Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Ben Stull
|
213f6862d5 |
-184
@@ -23,190 +23,6 @@ skip versions are the composition of each intervening adjacent
|
||||
release's steps in order — no A-to-B path is pre-computed beyond
|
||||
that.
|
||||
|
||||
## 0.18.0 — 2026-05-28
|
||||
|
||||
**Minor — schema migration required; one env var now mandatory; no
|
||||
new secrets.** This release lands the framework-side half of OHM
|
||||
roadmap items #18 (Secure the SMTP relay + Gitea webhook) and #20
|
||||
(Email deliverability). It is the framework counterpart to the
|
||||
operator-side SMTP / DNS hardening covered in the
|
||||
`EMAIL-AND-WEBHOOK-HARDENING-RUNBOOK.md` companion doc.
|
||||
|
||||
The release is shipped in five atomic slices per the v0.18.0 proposal:
|
||||
|
||||
1. **`build_envelope` shared helper.** A single place where every
|
||||
outbound `EmailMessage` is constructed. Lands the
|
||||
deliverability-critical headers (`Date`, `Message-ID`,
|
||||
`Auto-Submitted`) uniformly across OTC, invite, watcher
|
||||
notification, bundle, and digest paths; exposes per-kind
|
||||
unsubscribe semantics (none for OTC, mailto: for invites, full
|
||||
one-click for bulk-adjacent paths) as explicit kwargs.
|
||||
2. **Migrated send paths.** `email_otc.py`, `email_invite.py`,
|
||||
`email._deliver`, `email._send_bundle`, and `digest.py` now
|
||||
build their envelopes through the helper. Per-RFC invite
|
||||
(v0.16.0) rides through `email_invite.py`'s helper and picks
|
||||
up the change for free. The shared `_SENT` test buffer also
|
||||
carries the constructed `EmailMessage` under `envelope["message"]`
|
||||
so tests can assert on the header surface directly.
|
||||
3. **Webhook handler tightening.** `GITEA_WEBHOOK_SECRET` is now
|
||||
mandatory at startup — the framework refuses to load_config()
|
||||
if it's empty, unless the operator explicitly opts into the
|
||||
dev-bypass with `RFC_APP_INSECURE_WEBHOOKS=1`. The
|
||||
`/api/webhooks/gitea` receiver carries defense-in-depth checks
|
||||
that surface the misconfiguration loudly at the request layer
|
||||
too. Mis-targeted webhooks (a hook on a fork or a stale Gitea
|
||||
binding) now log at INFO instead of silently 200-OK'ing.
|
||||
4. **`outbound_emails` audit table + admin endpoint.** Every send
|
||||
helper writes one row to `outbound_emails` before returning,
|
||||
capturing the send attempt regardless of outcome
|
||||
(status='sent' / 'failed' / 'deferred'). The new admin endpoint
|
||||
`GET /api/admin/outbound-emails` (filterable by kind / status /
|
||||
to_address) lets the operator answer "did this person ever get
|
||||
their invite?" without grepping VM logs. No admin UI ships
|
||||
with v0.18.0; operator queries via curl + jq for now.
|
||||
5. **Bounce correlation.** The `POST /api/webhooks/email-bounce`
|
||||
body accepts a new optional `message_id` field; when supplied,
|
||||
the handler stamps status='bounced' on the matching
|
||||
`outbound_emails` row and returns the row id as
|
||||
`correlated_id`. The pre-existing hard-bounce ->
|
||||
`email_opt_out_all = 1` flow still fires.
|
||||
|
||||
The `POST /api/email/unsubscribe` endpoint also lands in Slice 2
|
||||
as the matching receiver for the new `List-Unsubscribe-Post:
|
||||
List-Unsubscribe=One-Click` header (Gmail and Yahoo POST that
|
||||
payload on the user's one-click action per RFC 8058 — the GET
|
||||
endpoint alone is no longer sufficient for senders at OHM's tier).
|
||||
|
||||
### Added
|
||||
|
||||
- **`backend/app/email_envelope.py`** — the `build_envelope` helper.
|
||||
Single source of truth for every outbound `EmailMessage`'s
|
||||
headers + body shape. Standalone module so tests can exercise it
|
||||
without booting the FastAPI app.
|
||||
- **`backend/migrations/020_outbound_emails.sql`** — the audit
|
||||
table. Single new table with three indexes (to_address, sent_at,
|
||||
message_id); no changes to existing tables.
|
||||
- **`email.record_outbound(...)`** — best-effort write helper every
|
||||
send path calls. Catches `RuntimeError` (so pure-helper unit
|
||||
tests where `db.init()` was never called don't break) and any
|
||||
other exception (so the audit write never breaks a send).
|
||||
- **`GET /api/admin/outbound-emails`** in `api_admin.py` —
|
||||
admin-only listing of `outbound_emails`. Newest-first, filterable
|
||||
by kind, status, and to_address (case-insensitive). Returns
|
||||
`{items: [...], has_more}` per the rest of the admin endpoints'
|
||||
shape.
|
||||
- **`POST /api/email/unsubscribe`** in `api_notifications.py` —
|
||||
RFC 8058 one-click receiver. Accepts the same `?t=` token as the
|
||||
GET handler; idempotent; returns 200 + `{ok, category}` on
|
||||
success.
|
||||
- **`all` synthetic category** for unsubscribe URLs. Used by the
|
||||
bundle + digest paths (which can't honor per-category opt-outs
|
||||
because they span multiple categories); flips
|
||||
`email_opt_out_all = 1` rather than a per-category column.
|
||||
|
||||
### Changed
|
||||
|
||||
- **`backend/app/email.py`** — `EmailConfig` gains
|
||||
`unsubscribe_mailto` (env: `EMAIL_UNSUBSCRIBE_MAILTO`, default
|
||||
falls back to `EMAIL_FROM`). `_deliver` and `_send_bundle`
|
||||
build envelopes through `build_envelope` and thread `kind` +
|
||||
`notification_id` into the audit write.
|
||||
- **`backend/app/email_otc.py`** + **`email_invite.py`** — both
|
||||
call `build_envelope` and `record_outbound`. OTC carries no
|
||||
`List-Unsubscribe` (recipient explicitly requested the code);
|
||||
invite carries `List-Unsubscribe: <mailto:…>` only (no signed
|
||||
URL — the invitee isn't a user yet, no per-user opt-out row
|
||||
exists).
|
||||
- **`backend/app/digest.py`** — calls `_deliver` with `kind='digest'`
|
||||
+ the new `all`-category one-click unsubscribe.
|
||||
- **`backend/app/webhooks.py`** — refuses 500 at request time if
|
||||
the secret is empty + bypass isn't set; logs a loud warning
|
||||
per-request when running under the bypass; logs INFO when a
|
||||
hook targets a repo not in `cached_rfcs`.
|
||||
- **`backend/app/config.py`** — `load_config()` raises
|
||||
RuntimeError if `GITEA_WEBHOOK_SECRET` is empty unless
|
||||
`RFC_APP_INSECURE_WEBHOOKS=1`.
|
||||
- **`backend/app/api_notifications.py`** — GET unsubscribe handler
|
||||
accepts the `all` category (sets `email_opt_out_all = 1`).
|
||||
Bounce webhook body adds optional `message_id` field; response
|
||||
shape adds `correlated_id` field. (Tests that read the exact
|
||||
response shape — currently just
|
||||
`test_bounce_webhook_refuses_unsigned_when_secret_configured`
|
||||
in test_e2e_smoke.py — updated to assert on the new shape.)
|
||||
- **`backend/tests/test_propose_vertical.py`** — the shared
|
||||
`tmp_env` fixture binds a fake `GITEA_WEBHOOK_SECRET` so all
|
||||
252 pre-v0.18.0 tests boot cleanly under the new mandatory
|
||||
secret. Tests that want to exercise the dev-bypass path
|
||||
monkeypatch `RFC_APP_INSECURE_WEBHOOKS=1` explicitly.
|
||||
|
||||
### Tests
|
||||
|
||||
- 15 new unit tests in `test_email_envelope.py` for the helper.
|
||||
- 10 new integration tests across `test_otc_vertical`,
|
||||
`test_admin_create_user_invite_vertical`, and
|
||||
`test_notifications_vertical` covering: OTC has no
|
||||
List-Unsubscribe; invite has mailto: only; notification has
|
||||
full one-click; POST one-click flips per-category; `all` flips
|
||||
global; respects `EMAIL_UNSUBSCRIBE_MAILTO` override.
|
||||
- 7 new integration tests in `test_webhooks_vertical.py` covering
|
||||
the startup-time mandatory-secret check, the dev-bypass, and
|
||||
the request-time signature verification including the unknown-
|
||||
repo log line.
|
||||
- 11 new integration tests in `test_outbound_emails_vertical.py`
|
||||
covering the audit table write path (OTC / invite / notification),
|
||||
the admin endpoint (list, filter by kind, filter by to_address,
|
||||
non-admin refusal), and the bounce correlation (matched
|
||||
message_id stamps status='bounced'; unknown message_id is
|
||||
logged; absent message_id falls back to legacy behavior; bounced
|
||||
rows surface in admin endpoint with `?status=bounced`).
|
||||
- One test updated for intentional response-shape change:
|
||||
`test_e2e_smoke.test_bounce_webhook_refuses_unsigned_when_secret_configured`.
|
||||
|
||||
Full suite: 295 passed (was 252 pre-v0.18.0).
|
||||
|
||||
### Migration
|
||||
|
||||
- **`backend/migrations/020_outbound_emails.sql`** — auto-applied
|
||||
on next backend start. Single new table with three indexes; no
|
||||
changes to existing tables.
|
||||
|
||||
### Upgrade steps (from 0.17.0)
|
||||
|
||||
- Operators **MUST** ensure `GITEA_WEBHOOK_SECRET` is set in the
|
||||
deployment's env. The framework now refuses to start if it's
|
||||
empty. (For OHM-flotilla deployments,
|
||||
`flotilla secret list <deployment>` confirms the binding; OHM
|
||||
has carried this binding since v0.14.0, so the upgrade is
|
||||
gesture-free for OHM specifically.)
|
||||
- Operators **MAY** set `RFC_APP_INSECURE_WEBHOOKS=1` to bypass
|
||||
the requirement in local-dev environments. Production
|
||||
deployments **MUST NOT** set this; if they do, every webhook
|
||||
POST logs a loud warning line per request.
|
||||
- Operators **MAY** set `EMAIL_UNSUBSCRIBE_MAILTO` to route
|
||||
`List-Unsubscribe: <mailto:…>` opt-out courtesy mail to a
|
||||
humans-monitored mailbox distinct from the no-reply
|
||||
`EMAIL_FROM` sender. Default falls back to `EMAIL_FROM`.
|
||||
- You **MUST** apply schema migration
|
||||
`020_outbound_emails.sql`. The migration creates a single new
|
||||
table with three indexes; the framework runs migrations
|
||||
automatically at process start, so no manual step is required
|
||||
beyond restarting the backend so the migration runner picks
|
||||
the file up. Existing deployments pick it up on first start
|
||||
after upgrade with no operator action required.
|
||||
- You **MUST** rebuild the frontend and restart the backend
|
||||
after upgrading. `frontend/package.json#version` and `VERSION`
|
||||
both move to `0.18.0`. No new secrets (the `outbound_emails`
|
||||
table writes synchronously to the same SQLite file as every
|
||||
other write).
|
||||
- Operators **SHOULD** run a `mail-tester.com` probe against the
|
||||
upgraded deployment to confirm the new envelope headers
|
||||
(`Date`, `Message-ID`, `Auto-Submitted`, `List-Unsubscribe`,
|
||||
`List-Unsubscribe-Post`) land cleanly with the upstream SMTP
|
||||
relay's DKIM signing. The expected delta from pre-v0.18.0 is
|
||||
+2-3 points on the spam-score axis (typical 5-6/10 baseline
|
||||
→ 9+/10 post-upgrade).
|
||||
|
||||
|
||||
## 0.17.0 — 2026-05-28
|
||||
|
||||
**Minor — schema migration required; no new env vars; no new secrets.**
|
||||
|
||||
+407
@@ -0,0 +1,407 @@
|
||||
# Contributing to rfc-app
|
||||
|
||||
`rfc-app` is the framework that hosts RFC-shaped collections of
|
||||
documents — one repo per RFC, a meta repo per collection, a web app
|
||||
that turns the Git substrate into a writeable surface. The Open
|
||||
Human Model (OHM) deployment at `ohm.wiggleverse.org` is one
|
||||
instance. The framework is intended to host more.
|
||||
|
||||
This document explains how to propose a change to the framework
|
||||
itself — a new endpoint, a schema migration, a UI affordance, a
|
||||
spec clarification. For changes to *content* hosted by a specific
|
||||
deployment (the OHM RFCs, the OHM roadmap), see that deployment's
|
||||
own contribution guide (e.g. [`ohm-rfc/CONTRIBUTING.md`](https://git.wiggleverse.org/wiggleverse/ohm-rfc/src/branch/main/CONTRIBUTING.md)).
|
||||
|
||||
---
|
||||
|
||||
## How the project actually evolves
|
||||
|
||||
rfc-app is built in the open in the literal sense: **every build
|
||||
session produces a full transcript** at
|
||||
[`wiggleverse/ohm-session-history`](https://git.wiggleverse.org/wiggleverse/ohm-session-history)
|
||||
on `git.wiggleverse.org`. The transcripts are the authoritative
|
||||
record of how the framework got from one release to the next — the
|
||||
decisions, the friction, the dead ends, the reasoning. They are not
|
||||
curated retrospectives; wrong turns stay in.
|
||||
|
||||
If you are proposing a change to rfc-app, **read at least the most
|
||||
recent session transcript before opening a PR.** The transcripts
|
||||
show what shape a feature lands in, where the spec gets touched,
|
||||
what the operator pushes back on, and how the release rides into
|
||||
deployment. A PR that matches that texture is much more likely to
|
||||
land cleanly than one shaped by the README alone.
|
||||
|
||||
Worked examples to start with:
|
||||
|
||||
- **Session E** ([transcript](https://git.wiggleverse.org/wiggleverse/ohm-session-history)) —
|
||||
a clean small release. Read this for the simplest possible release
|
||||
shape: one feature, one version bump, one upgrade-steps block, no
|
||||
surprises.
|
||||
- **Session I** — recovery from a deploy fault. Read this for how
|
||||
the project handles things going wrong mid-deploy, and for the
|
||||
honest no-curation discipline.
|
||||
- **Session K** — a multi-feature wave with one item paused on an
|
||||
operator-provided secret. Read this for the subagent dispatch
|
||||
pattern (the model the project uses to ship multiple features in
|
||||
parallel), and for the binding rule that the assistant **never**
|
||||
asks the operator to paste secret bytes into the conversation.
|
||||
- **Session L** — squash-merge integration across three parallel
|
||||
features (v0.15.0 / v0.16.0 / v0.17.0), with `#21 Part C`
|
||||
identity-lifecycle Amplitude wiring folded inline across all
|
||||
three releases. Read this for how cross-cutting concerns (analytics,
|
||||
observability) get layered into already-in-flight features
|
||||
without scope-creeping any single release.
|
||||
|
||||
The repository where transcripts live —
|
||||
[`wiggleverse/ohm-session-history`](https://git.wiggleverse.org/wiggleverse/ohm-session-history) —
|
||||
is the canonical history. The `git log` of `rfc-app` is the artifact;
|
||||
the transcripts are the story behind it.
|
||||
|
||||
---
|
||||
|
||||
## How a contribution flows
|
||||
|
||||
The framework runs on a **subagents push feature branches; operator
|
||||
tags and deploys** model. Contributors — whether human or AI agents
|
||||
running in a Claude Code subsession — open feature branches and
|
||||
submit PRs. The operator (the person running the deployment) is the
|
||||
one who merges, tags, bumps `VERSION`, runs `flotilla deploy` (or
|
||||
the equivalent for non-OHM deployments), and moves the deployment's
|
||||
`.rfc-app-version` pin. The driver session transcripts inherit
|
||||
this shape; contributors inherit it from them.
|
||||
|
||||
Concretely:
|
||||
|
||||
1. Read the most recent session transcript. Understand what just
|
||||
shipped and what is in flight.
|
||||
2. Open an Issue first if your change is exploratory, structural,
|
||||
or might overlap with in-flight work. The operator will name
|
||||
any collision.
|
||||
3. Branch from `main`. Name the branch
|
||||
`feature/<short-description>` for additive work, `fix/<short-
|
||||
description>` for bug fixes, `docs/<short-description>` for
|
||||
documentation-only work. The driver sessions use
|
||||
`feature/v<target-version>-<slug>` (e.g.
|
||||
`feature/v0.16.0-owner-invite`) — that shape is welcome but not
|
||||
required for outside contributors, since contributors do not
|
||||
pick the target version.
|
||||
4. **Do not bump `VERSION` or `frontend/package.json#version` in
|
||||
your PR.** The operator picks the target version at integration
|
||||
time; bumping ahead causes cherry-pick conflicts. The same
|
||||
applies to the `CHANGELOG.md` entry header — see below.
|
||||
5. **Do not tag releases, do not run any deploy gesture, do not
|
||||
touch any deployment's `.rfc-app-version` pin.** The operator
|
||||
alone owns those gestures. (For OHM specifically: "I'm the only
|
||||
one that gets to yolo." See the boundary section in
|
||||
`ohm-rfc/CONTRIBUTING.md`.)
|
||||
6. Push your branch and open a PR. Describe what you're proposing
|
||||
and why, in language the operator can paste into the eventual
|
||||
release commit. If the change touches `SPEC.md`, name which
|
||||
section(s) and the contract change.
|
||||
|
||||
---
|
||||
|
||||
## CHANGELOG convention: strict descending
|
||||
|
||||
`CHANGELOG.md` is ordered **newest-on-top**. The header line for
|
||||
the in-progress version goes at the top of the file; older
|
||||
releases descend below it. This is the binding convention; the
|
||||
operator hand-resolves the conflict when two parallel feature
|
||||
branches both insert at the top of the file (the squash-merge
|
||||
integration that ships parallel-feature waves keeps the strict-
|
||||
descending shape — see Session K for the cherry-pick mechanics and
|
||||
Session L for the hand-resolved-with-a-small-script variant).
|
||||
|
||||
A new entry has this shape (read the existing 0.15.0 / 0.16.0 /
|
||||
0.17.0 entries for worked examples):
|
||||
|
||||
```markdown
|
||||
## 0.X.Y — YYYY-MM-DD
|
||||
|
||||
**Minor — schema migration auto-applied; no operator action.** This
|
||||
release ships <one or two sentences naming the feature and why>.
|
||||
|
||||
### Added
|
||||
- **<New module/endpoint/component>** — what it does, where it lives,
|
||||
why it exists. Include file paths inline so a reader can click through.
|
||||
### Changed
|
||||
- **<Existing surface>** — what changed and how a deployment notices.
|
||||
### Migration
|
||||
- **`<NNN_name>.sql`** — auto-applied by `db.run_migrations()` on
|
||||
backend start. <Describe the schema delta in one sentence.>
|
||||
### Upgrade steps (from 0.(X-1).Y)
|
||||
- You **MUST** … (per RFC 2119; see SPEC.md §20.4).
|
||||
- You **MUST NOT** …
|
||||
- You **SHOULD** …
|
||||
- You **MAY** …
|
||||
```
|
||||
|
||||
The header version number is filled in by the operator at merge
|
||||
time. Your PR's CHANGELOG diff can leave the version as
|
||||
`0.X.Y — YYYY-MM-DD` (literal placeholder), or use a guessed value
|
||||
the operator overwrites; either is fine.
|
||||
|
||||
---
|
||||
|
||||
## `Upgrade steps:` blocks use RFC 2119 keywords
|
||||
|
||||
If your change requires deployments to do anything when they
|
||||
upgrade — set an env var, apply a migration, restart a process,
|
||||
flip an overlay value, accept a behavioral change — your CHANGELOG
|
||||
entry **must** include an `### Upgrade steps` block, and that
|
||||
block **must** use the [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119)
|
||||
/ [RFC 8174](https://www.rfc-editor.org/rfc/rfc8174) keywords as
|
||||
defined in `SPEC.md` §20.4:
|
||||
|
||||
- **MUST** / **SHALL** / **REQUIRED** — without this step the
|
||||
deployment will not function correctly. Skipping is a regression
|
||||
the framework does not handle.
|
||||
- **MUST NOT** / **SHALL NOT** — previously valid, now no longer
|
||||
supported.
|
||||
- **SHOULD** / **RECOMMENDED** — the framework's tested path. A
|
||||
deployment may deviate when it has a reason.
|
||||
- **SHOULD NOT** / **NOT RECOMMENDED** — discouraged without being
|
||||
forbidden.
|
||||
- **MAY** / **OPTIONAL** — an affordance you can take or skip.
|
||||
|
||||
Cross-version upgrades (jumping more than one minor) are computed by
|
||||
the operator composing each intervening release's steps in order.
|
||||
Each adjacent step must therefore be locally unambiguous — this is
|
||||
the whole reason the keyword discipline is binding. Avoid words
|
||||
like "should probably" or "might want to" inside an upgrade step;
|
||||
either the framework needs the action or it doesn't.
|
||||
|
||||
If your change touches the env contract, **also update**
|
||||
`backend/.env.example` and/or `frontend/.env.example` in the same
|
||||
PR so the contract and the documentation land together (§20.4).
|
||||
|
||||
---
|
||||
|
||||
## SPEC.md and §19.2 candidates
|
||||
|
||||
`SPEC.md` is the framework's binding spec. It is honest about open
|
||||
questions — large sections of it carry "§19.2 candidates," which
|
||||
are decisions the project has deliberately deferred rather than
|
||||
guessed at.
|
||||
|
||||
The discipline: **architectural or process deferrals get noted as
|
||||
§19.2 candidates rather than scope-creeping a release.** When you
|
||||
notice that your change opens a question larger than the change
|
||||
itself (a different DB shape, a new auth contract, a cross-cutting
|
||||
UX rethink), the right move is usually to land the narrow change
|
||||
and add a §19.2 candidate naming the larger question. The candidate
|
||||
documents what was set aside and why, so a future session can pick
|
||||
it up with context.
|
||||
|
||||
Worked examples from recent sessions:
|
||||
|
||||
- v0.11.0 (Session K) shipped device trust and surfaced three new
|
||||
§19.2 candidates: cross-device session revocation, password-
|
||||
equivalent change invalidating trust, device-trust window
|
||||
tunables via env. None of those were in the v0.11.0 scope; they
|
||||
were noted in SPEC.md §19.2 so a future session can address them
|
||||
on their own terms.
|
||||
- v0.15.0 (Session L) shipped the Amplitude wrapper and added
|
||||
candidates around session-replay-specific consent category +
|
||||
bundle-size measurement, both deferred to the future Part-A audit.
|
||||
|
||||
When you spot a deferred decision in your PR's territory, name it
|
||||
in your PR description and add it to `SPEC.md` §19.2 in the same
|
||||
diff. Do not silently expand scope to settle it.
|
||||
|
||||
---
|
||||
|
||||
## Test-coverage expectations
|
||||
|
||||
The backend has the load-bearing test suite at
|
||||
`backend/tests/`. Tests are organized as `*_vertical.py` files,
|
||||
each covering one feature end-to-end through the FastAPI app
|
||||
(provisioning fixtures, hitting the HTTP surface, asserting on the
|
||||
database state). At time of writing, the suite is ~250 tests across
|
||||
~25 files. Examples:
|
||||
|
||||
- `test_admin_create_user_invite_vertical.py` — v0.17.0's
|
||||
admin-create user + invite + claim flow, 15 tests covering happy
|
||||
path + every refusal shape + the audit-trail row.
|
||||
- `test_rfc_invitations_vertical.py` — v0.16.0's per-RFC invite +
|
||||
accept flow, 18 tests.
|
||||
- `test_device_trust_vertical.py` — v0.11.0's 30-day device trust,
|
||||
14 tests including cookie shape, hash-vs-raw-token discipline,
|
||||
expired / revoked / forged / cross-user invariants.
|
||||
|
||||
Expected coverage for a new feature:
|
||||
|
||||
- **Backend feature** — one new `test_<feature>_vertical.py` file
|
||||
that covers the happy path, every documented refusal/error code,
|
||||
and any cross-surface effect (rows the feature writes to existing
|
||||
tables, fields it adds to existing endpoints). Reuse fixtures
|
||||
from neighboring test files (e.g. `test_propose_vertical.py`'s
|
||||
`FakeGitea` is widely reused).
|
||||
- **Migration** — verify migrations are reachable from `backend/.venv`
|
||||
before pushing: `cd backend && PYTHONPATH=. .venv/bin/pytest -q`
|
||||
exercises `db.run_migrations()` through the fixture setup.
|
||||
- **Frontend feature** — there is currently no frontend test
|
||||
runner. The discipline is: keep the change ships-clean
|
||||
(`cd frontend && npm run build` succeeds), and the backend
|
||||
vertical test exercises the HTTP contract the frontend
|
||||
consumes, which is the meaningful behavioral guarantee.
|
||||
Frontend changes that ride along with a backend feature land
|
||||
with the backend test as the regression boundary.
|
||||
- **Bug fix** — add a regression test in the same vertical file
|
||||
that proves the original failure mode and verifies the fix.
|
||||
|
||||
Run the backend suite before pushing. From `backend/`:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=. .venv/bin/pytest -q
|
||||
```
|
||||
|
||||
(The `PYTHONPATH=.` is a known ergonomic gap — see SPEC.md §19.2
|
||||
candidate; the suite does not pick up `app/` without it.)
|
||||
|
||||
If your PR doesn't include tests, the operator will ask for them
|
||||
before merge unless the change is genuinely test-irrelevant
|
||||
(documentation, comments, dev-only tooling).
|
||||
|
||||
---
|
||||
|
||||
## Analytics instrumentation checklist
|
||||
|
||||
> *(This section codifies `ohm-rfc/ROADMAP.md` #21 Part B's
|
||||
> CONTRIBUTING checklist. It is discipline, not a gate — but the
|
||||
> operator will push back on PRs that skip it.)*
|
||||
|
||||
If your PR adds or changes a user-facing feature, walk this
|
||||
checklist before opening the PR. The instrumentation conventions
|
||||
themselves are specified in `SPEC.md` §21 (Analytics instrumentation
|
||||
and identity); this section is the procedural reminder.
|
||||
|
||||
1. **What named event(s) does this feature need?**
|
||||
Open `frontend/src/lib/analytics.js` and look at the `EVENTS`
|
||||
constant. Does an existing event cover your feature? If not, is
|
||||
the new event in the spec's "Subject Verb" Title Case form
|
||||
(`Comment Posted`, `Invitation Sent`)? Are the prop families
|
||||
consistent with SPEC.md §21's required-prop catalog (opaque
|
||||
ids only, no PII, enums lowercased like `'otc'` not `'OTC'`)?
|
||||
|
||||
2. **Do interactive elements have stable text / ARIA labels /
|
||||
`data-amp-track-*` so autocapture is meaningful?**
|
||||
The frontend ships `autocapture: true`, which instruments
|
||||
every click and form interaction. The *value* of those events
|
||||
depends on the DOM the SDK sees: a `<button>` with stable
|
||||
visible text or an `aria-label` shows up as a meaningful
|
||||
dashboard row; an icon-only `<button>` with no label shows up
|
||||
as garbage. New components that introduce interactive elements
|
||||
should either carry meaningful labels (visible text or ARIA) or
|
||||
carry a `data-amp-track-name="<Stable Name>"` attribute. For
|
||||
repeated rows (per-RFC lists, comment lists), use a stable
|
||||
`data-amp-track-*` identifier so per-row click counts aggregate
|
||||
to the row's identity rather than to a generic label.
|
||||
|
||||
3. **Does any new form field need replay masking?**
|
||||
Session replay records at `sampleRate: 1` (100% of consented
|
||||
sessions). New form inputs that capture passwords, OTC codes,
|
||||
tokens, magic-link URLs, or other secret/credential-equivalent
|
||||
material **MUST** be masked with Amplitude's masking conventions
|
||||
(the `.amp-mask` class or the `data-amp-mask` attribute,
|
||||
whichever the wrapper integration expects in this version).
|
||||
New inputs that capture arguably-PII (email, real name, free-
|
||||
text drafts) **SHOULD** also be masked; if a deliberate
|
||||
un-masking decision is taken, document it in the PR description
|
||||
and in `SPEC.md` §21.
|
||||
|
||||
4. **Does the PR description name the instrumentation decisions?**
|
||||
A one-sentence summary in the PR description — "fires
|
||||
`Comment Posted` with `{rfc_slug, comment_id}`; no new form
|
||||
fields, no new replay-masking concerns" — is enough. If the
|
||||
decision is "we chose not to instrument this," say that too;
|
||||
the absence of an event is itself a decision the operator
|
||||
wants visible. The relevant SPEC chapter (§21) is the binding
|
||||
reference for what shapes are correct.
|
||||
|
||||
If your feature touches an identity-meaningful surface (sign-in,
|
||||
sign-out, invite-claim, role change, account state change), also
|
||||
walk the **identity lifecycle** contract in SPEC.md §21.6: every
|
||||
new claim/sign-in path **MUST** call `identify({ user_id, properties })`
|
||||
BEFORE the first `track()` event on that surface, so the Amplitude
|
||||
user record is created with the OHM user_id from the very first
|
||||
event rather than as an anonymous device that retroactively links.
|
||||
v0.16.0's `AcceptInvitation.jsx` and v0.17.0's `InviteClaim.jsx`
|
||||
are the worked examples; mirror their shape.
|
||||
|
||||
---
|
||||
|
||||
## The operator-only gestures
|
||||
|
||||
Some gestures are operator-only. Contributors do not perform them;
|
||||
PRs that perform them get rejected on principle, not on merit:
|
||||
|
||||
- **Tagging a release** (`git tag v0.X.Y` + `git push --tags`).
|
||||
- **Pushing to `main`** after merge (the operator merges; the
|
||||
framework's `main` branch tracks releases the operator has
|
||||
shipped).
|
||||
- **Bumping `VERSION` and `frontend/package.json#version` to the
|
||||
shipped value.** The operator does this at integration time so
|
||||
the version line is consistent across the release commit.
|
||||
- **Running `flotilla deploy` or any equivalent deployment gesture**
|
||||
in any deployment of rfc-app. Contributors do not deploy.
|
||||
- **Moving a deployment's `.rfc-app-version` pin.** That pin lives
|
||||
in the deployment's content repo (e.g. `ohm-rfc/.rfc-app-version`)
|
||||
and is moved by the deployment's operator. Contributors to that
|
||||
deployment do not move it; contributors to the framework
|
||||
certainly do not.
|
||||
- **Setting secrets** (anywhere — Secret Manager, env files,
|
||||
`flotilla secret set`, vendor dashboards, anything). The
|
||||
binding rule baked in mid-Session-K is: **the assistant never
|
||||
asks the operator to paste secret bytes into a conversation,
|
||||
even as one offered option**. The corollary for contributors:
|
||||
do not include secret values in PR descriptions, commit
|
||||
messages, or issue comments. Reference secrets by their binding
|
||||
name (`SMTP_PASSWORD`, `AMPLITUDE_API_KEY`) and let the
|
||||
operator handle the bytes.
|
||||
|
||||
If your change requires a new secret or env var, document the
|
||||
requirement in the CHANGELOG `### Upgrade steps` block in the
|
||||
RFC 2119 form ("operators **MUST** set `<NEW_VAR>` ...") and
|
||||
update the `*.env.example` file. The operator will run the
|
||||
secret/overlay-set gesture themselves at deploy time.
|
||||
|
||||
---
|
||||
|
||||
## When in doubt
|
||||
|
||||
- **Open an Issue first.** Especially for any change that touches
|
||||
SPEC.md, the auth/permissions model (§6), the storage shape (§4
|
||||
/ §5), or the deploy contract (§20). The operator (or a future
|
||||
driver session) will name what they want before you write code.
|
||||
- **Read the most recent session transcript.** It will tell you
|
||||
what shipped last and what's in flight.
|
||||
- **Cite SPEC.md sections in your PR description.** "Touches §15.4
|
||||
(per-category email toggles) and adds §19.2 candidate around
|
||||
per-channel mute granularity" gives the operator a map of where
|
||||
to read.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
The framework is released under the MIT License (see
|
||||
[`LICENSE`](./LICENSE)). By contributing, you agree your work
|
||||
ships under those terms.
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
- [`SPEC.md`](./SPEC.md) — the framework's binding spec. §19.2
|
||||
is the deferred-decisions queue; §20 is the versioning + deploy
|
||||
contract; §21 is the analytics instrumentation contract.
|
||||
- [`CHANGELOG.md`](./CHANGELOG.md) — release history in strict
|
||||
descending order. Read recent entries for the shape your PR's
|
||||
release-commit will take.
|
||||
- [`PHILOSOPHY.md`](./PHILOSOPHY.md) — what the framework is for.
|
||||
PRs whose shape conflicts with the philosophy get a longer
|
||||
conversation than PRs that fit.
|
||||
- [`wiggleverse/ohm-session-history`](https://git.wiggleverse.org/wiggleverse/ohm-session-history)
|
||||
— the authoritative record of how the project has actually
|
||||
evolved, session by session.
|
||||
@@ -2284,6 +2284,13 @@ a given signal, the **storage shape** that makes triage tractable, and
|
||||
the **out-of-session channels** (email, digest) that let asynchrony
|
||||
actually work.
|
||||
|
||||
(The framework's separate **analytics + session-replay** surface —
|
||||
Amplitude wiring, event taxonomy, identity lifecycle, consent
|
||||
contract — is a peer cross-cutting concern specified in §21.
|
||||
Notifications cover in-product signal-of-others-acting-on-your-work;
|
||||
analytics covers observability of how the product is used. The two
|
||||
surfaces do not overlap.)
|
||||
|
||||
### 15.1 The signal-surface stack
|
||||
|
||||
Five surfaces, each with one narrow job:
|
||||
@@ -4300,3 +4307,465 @@ Downstream deployments, in exchange for the contract above, commit to:
|
||||
order;
|
||||
- supply every required env var the framework documents at the
|
||||
version they are running.
|
||||
|
||||
---
|
||||
|
||||
## 21. Analytics instrumentation and identity
|
||||
|
||||
The framework ships an Amplitude Analytics + Session Replay wrapper
|
||||
in v0.15.0 (`frontend/src/lib/analytics.js`), gated by the v0.13.0
|
||||
cookie/privacy consent surface (`frontend/src/lib/consent.js`,
|
||||
§14.5). This section codifies the conventions that keep the
|
||||
instrumentation **quality** healthy as features land — taxonomy
|
||||
shape, autocapture hygiene, replay masking, the consent contract,
|
||||
and the identity lifecycle. The conventions are framework-neutral:
|
||||
every deployment of rfc-app that turns on the wrapper inherits
|
||||
them.
|
||||
|
||||
This chapter is placed semantically after §15 (Notifications) and
|
||||
§16 (deliberately deferred) as a peer cross-cutting framework
|
||||
concern. It was added after §20 in the chapter sequence to avoid
|
||||
renumbering the deferred-decisions surface §19.2, which is a
|
||||
load-bearing project noun referenced across CLAUDE.md, transcripts,
|
||||
and prior commits.
|
||||
|
||||
### 21.1 Event-taxonomy conventions
|
||||
|
||||
Events live in the public `EVENTS` constant in
|
||||
`frontend/src/lib/analytics.js`. Callers **SHOULD** use one of the
|
||||
named constants rather than firing arbitrary event strings — that
|
||||
keeps the Amplitude dashboard coherent over time and makes the
|
||||
taxonomy reviewable as a single source of truth.
|
||||
|
||||
- **Name form: Title Case, "Subject Verb".** E.g.
|
||||
`Comment Posted`, `Invitation Sent`, `RFC Viewed`,
|
||||
`User Signed In`, `Admin Permission Decision`. Spaces between
|
||||
words, no punctuation, no leading verbs (use `RFC Proposed`,
|
||||
not `Propose RFC`). The strings match the Amplitude dashboard
|
||||
names exactly.
|
||||
- **Stability.** New events **SHOULD** land via a release, not
|
||||
ad-hoc — adding an entry to `EVENTS` is a CHANGELOG-worthy
|
||||
change because it widens the framework's observable surface
|
||||
(§20.3). Renaming an event after it has shipped breaks the
|
||||
dashboard's historical continuity; renames **SHOULD** be
|
||||
treated as a deprecation cycle (ship both, dashboard-migrate,
|
||||
drop the old one).
|
||||
- **Opaque ids only in prop values.** Properties **MUST NOT**
|
||||
carry PII — no email, no display name, no IP, no free-text
|
||||
field bodies (titles, comment text, RFC drafts). Properties
|
||||
**SHOULD** be limited to:
|
||||
- opaque ids: `rfc_slug`, `rfc_id`, `pr_number`,
|
||||
`target_user_id`, `invited_by_admin_id`, `thread_id`,
|
||||
`comment_id`;
|
||||
- enums (lowercased): `method: 'otc' | 'passcode' |
|
||||
'device-trust' | 'admin-invite' | 'rfc-invite'`;
|
||||
- booleans: `trust_device`, `needs_passcode`, `passcode_set`;
|
||||
- timestamps (ISO 8601);
|
||||
- small bounded integers: `custom_message_chars` (coarse-grained
|
||||
signal of admin effort, NOT the message text itself).
|
||||
- **Casing consistency.** Prop keys use `snake_case` (matches the
|
||||
backend's JSON shape). Enum values use lowercase with hyphens
|
||||
(`'rfc-invite'`, not `'rfcInvite'` or `'RFC_INVITE'`). Drift
|
||||
here ruins dashboard aggregation; the operator-side audit
|
||||
(§21.7 / `ohm-rfc/ROADMAP.md` #21 Part A) checks for it.
|
||||
|
||||
The starting taxonomy as of v0.17.0:
|
||||
|
||||
```
|
||||
PAGE_VIEWED: 'Page Viewed'
|
||||
RFC_VIEWED: 'RFC Viewed'
|
||||
USER_SIGNED_IN: 'User Signed In'
|
||||
USER_SIGNED_OUT: 'User Signed Out'
|
||||
RFC_PROPOSED: 'RFC Proposed'
|
||||
PR_OPENED: 'PR Opened'
|
||||
COMMENT_POSTED: 'Comment Posted'
|
||||
BETA_ACCESS_REQUESTED: 'Beta Access Requested'
|
||||
ADMIN_PERMISSION_DECISION: 'Admin Permission Decision'
|
||||
INVITATION_SENT: 'Invitation Sent' # v0.16.0 / #12
|
||||
INVITATION_ACCEPTED: 'Invitation Accepted' # v0.16.0 / #12
|
||||
USER_INVITED: 'User Invited' # v0.17.0 / #16
|
||||
INVITE_CLAIMED: 'Invite Claimed' # v0.17.0 / #16
|
||||
```
|
||||
|
||||
### 21.2 Required prop families per event kind
|
||||
|
||||
Each event family carries a small required prop set. These are
|
||||
load-bearing for the dashboard's cohort analysis; releases that
|
||||
add a new event in an existing family **SHOULD** carry the
|
||||
family's required props.
|
||||
|
||||
- **Navigation events** (`Page Viewed`, `RFC Viewed`): carry
|
||||
`path` (string, pathname only — never the query string if it
|
||||
could carry a token) for `Page Viewed`; carry `rfc_slug` for
|
||||
`RFC Viewed`. `rfc_id` **MAY** be added when the cached row is
|
||||
in hand.
|
||||
- **Auth-state events** (`User Signed In`, `User Signed Out`):
|
||||
`User Signed In` carries `method` (one of `'otc'`,
|
||||
`'passcode'`, `'device-trust'`, `'admin-invite'`). `User
|
||||
Signed Out` carries no props (the identity binding is cleared
|
||||
separately via `anonymize()`).
|
||||
- **Authored-action events** (`RFC Proposed`, `PR Opened`,
|
||||
`Comment Posted`): carry `rfc_slug`. PRs additionally carry
|
||||
`pr_number` once the row exists. Comments additionally carry
|
||||
`thread_id`. None carry the body text.
|
||||
- **Admin-action events** (`Beta Access Requested`,
|
||||
`Admin Permission Decision`): the latter carries `action`
|
||||
(lowercase: `'grant'` / `'revoke'`) and `target_user_id`.
|
||||
- **Invite-side events** (`Invitation Sent`, `User Invited`): fire
|
||||
from the inviter's signed-in session. `Invitation Sent` (per-RFC,
|
||||
#12) carries `rfc_slug` + `role_in_rfc`. `User Invited`
|
||||
(admin-create, #16) carries `target_user_id` (the OHM user_id of
|
||||
the just-provisioned user) + `initial_role` +
|
||||
`custom_message_chars` (a bounded integer signal of admin
|
||||
effort, never the message text). Per #21 Part C: when the
|
||||
invitee is not yet a user (#12 per-RFC invitations to an email
|
||||
address that has never signed in), the invite-side event **MAY**
|
||||
carry a hashed `target_email` fingerprint (SHA-256 of the
|
||||
normalized lower-cased email) so the invite + claim pair can be
|
||||
correlated later. Plain-text `target_email` **MUST NOT** be
|
||||
carried.
|
||||
- **Claim-side events** (`Invitation Accepted`, `Invite Claimed`):
|
||||
fire from the invitee's session, immediately after an
|
||||
`identify({ user_id, properties })` call binds the OHM user_id
|
||||
to the Amplitude record (see §21.6). The events carry the
|
||||
invite context (`rfc_slug` + `role_in_rfc` for the per-RFC
|
||||
shape; `invited_by_admin_id` + `initial_role` + `needs_passcode`
|
||||
+ `trust_device` for the admin-create shape).
|
||||
|
||||
When in doubt, the principle: a property is correctly shaped iff
|
||||
the operator could publish it in a session transcript without
|
||||
hesitation.
|
||||
|
||||
### 21.3 Autocapture-friendly DOM patterns
|
||||
|
||||
The wrapper initializes Amplitude with `analytics.autocapture: true`,
|
||||
which auto-instruments page views, clicks, and form interactions.
|
||||
The *value* of those auto-captured events depends entirely on the
|
||||
DOM the SDK observes. Releases that add interactive UI **SHOULD**
|
||||
follow these patterns so the dashboard rows are readable rather
|
||||
than rows like "Click on `<button>` at `:nth-child(7)`".
|
||||
|
||||
- **Stable visible text on interactive elements.** Buttons and
|
||||
links **SHOULD** have stable, human-readable text content (the
|
||||
same string Amplitude uses to label the row). Avoid generic
|
||||
labels like "Read more" / "Click here" that lose context.
|
||||
- **`aria-label` on icon-only buttons.** Icon-only buttons (the
|
||||
chevron expanders, kebab menus, close `X`s) **MUST** carry a
|
||||
meaningful `aria-label`. Default autocapture for an unlabeled
|
||||
icon button reads as garbage. The `aria-label` is also an
|
||||
accessibility requirement — the two goals align.
|
||||
- **`data-amp-track-*` for repeating-list per-row identifiers.**
|
||||
When a list renders many rows of the same shape (RFC rows,
|
||||
comment rows, PR rows in a listing), per-row interactive elements
|
||||
**SHOULD** carry a `data-amp-track-name` attribute that
|
||||
identifies the row's *kind* and a `data-amp-track-*` attribute
|
||||
carrying the row's stable id. The convention:
|
||||
|
||||
```html
|
||||
<button
|
||||
data-amp-track-name="RFC Row Expand"
|
||||
data-amp-track-rfc-slug={slug}
|
||||
>…</button>
|
||||
```
|
||||
|
||||
This makes per-RFC click counts aggregate to the RFC rather
|
||||
than to a generic label, and lets the dashboard answer "which
|
||||
RFCs got the most engagement" rather than "how many buttons
|
||||
were clicked."
|
||||
- **`data-amp-track-suppress` for noise surfaces.** Crowded surfaces
|
||||
(the admin user-listing post-v0.9.0, the RFC discussion panel
|
||||
during heavy review) **MAY** apply
|
||||
`data-amp-track-suppress` (or its current equivalent in the
|
||||
SDK version in use) to elements whose clicks would flood the
|
||||
dashboard without informing anything. Suppression is a
|
||||
deliberate decision; document it inline.
|
||||
|
||||
### 21.4 Session-replay masking conventions
|
||||
|
||||
The wrapper initializes Amplitude with `sessionReplay.sampleRate: 1`
|
||||
(100% of consented sessions are recorded for full-DOM playback —
|
||||
vendor-recommended default for new Amplitude deployments). Replay
|
||||
has a meaningfully larger privacy footprint than event counters,
|
||||
and the masking discipline is binding.
|
||||
|
||||
- **Credentials MUST be masked.** The OTC code input, passcode
|
||||
input, any password-type field, the Turnstile widget internals,
|
||||
the magic-link-claim token if it survives in the URL bar
|
||||
(browser history, screenshot windows) — these **MUST** be masked
|
||||
with Amplitude's masking convention (the `.amp-mask` class or the
|
||||
`data-amp-mask` attribute, whichever the wrapper's SDK version
|
||||
uses; the wrapper's bootstrap comment names the current
|
||||
convention). Confirm each masking attribute survives the
|
||||
wrapper init by inspecting a recorded session before each
|
||||
release that touches an auth input.
|
||||
- **PII SHOULD be masked or carefully un-masked.** Email-entry
|
||||
fields, real-name capture fields (the v0.8.0 first/last/why
|
||||
panel), free-text RFC body drafts, comment-compose text —
|
||||
each is arguably PII or near-PII. The per-field decision is
|
||||
the release's responsibility; document the choice in `SPEC.md`
|
||||
§21 (this section) and in the release CHANGELOG so future
|
||||
deployments inherit the call rather than re-deciding.
|
||||
- **Privacy-policy alignment.** The recorded data **MUST** match
|
||||
what the deployment's privacy / cookies policy claims. If
|
||||
reality is broader than the document promises, update the policy
|
||||
text in the same release.
|
||||
- **Selective redaction.** Amplitude supports field-level mask
|
||||
classes that hide value while preserving DOM shape (so the
|
||||
session is replayable but the value is not). Prefer this over
|
||||
whole-form masking when only a subset is sensitive.
|
||||
|
||||
### 21.5 Consent-gate contract
|
||||
|
||||
The wrapper is bound to the v0.13.0 cookie/privacy consent surface
|
||||
(`frontend/src/lib/consent.js`, §14.5). The binding is **load-
|
||||
bearing**: the SDK and the session-replay recorder **MUST NOT**
|
||||
load before consent is granted, and any consent revocation **MUST**
|
||||
take effect within one tick of the consent flip.
|
||||
|
||||
The wrapper's bootstrap implements this contract; releases that
|
||||
touch the analytics surface **MUST** preserve it.
|
||||
|
||||
- **Pre-consent: no init, no network, no recording.** If
|
||||
`consent.analytics === true` is not currently true (either
|
||||
because the user denied, or because the banner is up and no
|
||||
decision has been recorded), the wrapper **MUST NOT** import
|
||||
the Amplitude SDK, **MUST NOT** open any network request to
|
||||
Amplitude, and **MUST NOT** start any session-replay recording.
|
||||
The consent-gated lazy `import('@amplitude/unified')` is the
|
||||
binding implementation; preserve it.
|
||||
- **Denied → granted: init at the consent moment.** When consent
|
||||
flips from denied/undecided to granted, the wrapper **MUST**
|
||||
initialize the SDK at that moment (a new `initAll(KEY, …)` call
|
||||
through the lazy-import path). Track and identify calls made
|
||||
before init resolves **MUST** be queued and drained on init,
|
||||
so the first signed-in user's first event is not dropped on
|
||||
the cold-load race.
|
||||
- **Granted → denied: setOptOut(true) within one tick.** When
|
||||
consent flips from granted to denied mid-session, the wrapper
|
||||
**MUST** call `amplitude.setOptOut(true)` so subsequent events
|
||||
are dropped client-side and session replay stops recording.
|
||||
The wrapper cannot unload the script tag (the SDK is already
|
||||
in memory), but the SDK's contract for "drop subsequent events"
|
||||
is `setOptOut(true)`. This **MUST** happen within one tick of
|
||||
the consent flip (i.e. synchronously inside the
|
||||
`onConsentChange` handler).
|
||||
- **No silent re-grant.** A granted → denied → granted sequence
|
||||
**MUST** call `setOptOut(false)` (re-enabling the SDK that was
|
||||
paused) rather than firing a second `initAll` (which would
|
||||
double-init). The wrapper's bootstrap implements this; releases
|
||||
that touch the consent integration **MUST** preserve the
|
||||
distinction.
|
||||
- **Build-time vs. runtime.** The API key is read from
|
||||
`import.meta.env.VITE_AMPLITUDE_API_KEY` at build time. When
|
||||
the env var is unset, the wrapper **MUST** log one console
|
||||
warning and no-op (every public function becomes a deterministic
|
||||
no-op) so dev environments without an Amplitude account keep
|
||||
working. The deploy gesture binds the key via the deployment's
|
||||
overlay verb (for OHM-shape deployments, `flotilla overlay set`);
|
||||
see §21.8 for the secret-vs-public discussion.
|
||||
|
||||
### 21.6 Identity lifecycle (per #21 Part C)
|
||||
|
||||
Amplitude's identity model has a specific pattern that the
|
||||
framework follows verbatim. Every release that touches an
|
||||
identity-meaningful surface **MUST** observe this pattern. The
|
||||
pattern shipped inline across v0.15.0 / v0.16.0 / v0.17.0
|
||||
(Session L's wave); this section codifies the contract so future
|
||||
releases inherit it.
|
||||
|
||||
**On sign-in success** (`App.jsx`'s `me.user` resolution):
|
||||
|
||||
- The wrapper's `identify({ user_id, properties })` call **MUST**
|
||||
carry both the OHM user_id (`amplitude.setUserId(<id>)`
|
||||
internally) AND the user's durable property bag. `setUserId`
|
||||
alone is **NOT** sufficient — without properties, the Amplitude
|
||||
user record carries only the id, and cohort analysis loses the
|
||||
shape (role distribution, sign-in-method distribution, etc.)
|
||||
the dashboard depends on.
|
||||
- Properties **MUST** be a bag of opaque ids, enums, booleans,
|
||||
and timestamps — no PII (no email, no display name, no IP).
|
||||
- Properties **MUST** be classified `set` vs `setOnce` deliberately
|
||||
(see §21.6.1 below).
|
||||
- The same `identify` call **MUST** be re-issued on every sign-in
|
||||
(idempotent at Amplitude's side; cheap; corrects any drift in
|
||||
the mutable property half).
|
||||
|
||||
**On user-state change mid-session** (role grant/revoke, trust-
|
||||
device add, passcode set, beta-permission flip):
|
||||
|
||||
- The wrapper's `setUserProperties(properties)` call **MUST** fire
|
||||
so the Amplitude record stays current. Mid-session state changes
|
||||
**MUST NOT** wait for the next sign-in to surface — the dashboard
|
||||
cohort an admin uses to grant permission is the same dashboard
|
||||
that next sees the granted user's behavior; staleness here breaks
|
||||
the cohort feedback loop.
|
||||
|
||||
**On sign-out**:
|
||||
|
||||
- The wrapper's `anonymize()` call (internally `amplitude.reset()`)
|
||||
**MUST** fire. `reset` clears the device-id linking AND **MUST**
|
||||
also clear the property cache so the next anonymous session is a
|
||||
fresh slate (the wrapper's `anonymize()` does both — releases
|
||||
that touch the wrapper **MUST** preserve this).
|
||||
- The `User Signed Out` `track()` call **MUST** fire *before*
|
||||
`anonymize()`, so the sign-out event is correctly attributed to
|
||||
the signing-out user rather than to the post-reset anonymous
|
||||
device.
|
||||
|
||||
**On invite-claim** (the v0.16.0 per-RFC invite + v0.17.0 admin-
|
||||
create invite paths):
|
||||
|
||||
- The wrapper's `identify({ user_id, properties })` call **MUST**
|
||||
fire BEFORE the first `track()` event on the claim surface, so
|
||||
the Amplitude user record is created with the OHM user_id from
|
||||
the first event. **MUST NOT** fire `track()` first and `identify`
|
||||
later — that creates an anonymous device record that
|
||||
retroactively links, and the cohort attribution for
|
||||
invite-driven onboarding loses precision.
|
||||
- The invite-context properties (`claim_method`,
|
||||
`invited_by_admin_id`, `invited_at`, `initial_role`) are
|
||||
`setOnce` (immutable user-history markers) — see §21.6.1.
|
||||
|
||||
**Inviter-side identification on invite-send events**:
|
||||
|
||||
- The inviter's `track('Invitation Sent', …)` and
|
||||
`track('User Invited', …)` events fire from the inviter's
|
||||
signed-in session, so the `user_id` attribution is already
|
||||
correct (it's the inviter's id). The event body carries
|
||||
`target_user_id` (#16 — admin-create, where the future user is
|
||||
provisioned at create-time) or a hashed `target_email`
|
||||
fingerprint (#12 — per-RFC invite, where the invitee is not
|
||||
yet a user) so the invite + claim pair can be correlated later
|
||||
in the dashboard.
|
||||
|
||||
#### 21.6.1 `set` vs `setOnce` taxonomy
|
||||
|
||||
Amplitude distinguishes two property-write semantics:
|
||||
|
||||
- **`set(k, v)`** — overwrites the property on every call. The
|
||||
user record reflects the most recent value.
|
||||
- **`setOnce(k, v)`** — writes only if the property is not
|
||||
already present. Subsequent calls are no-ops. The user record
|
||||
reflects the first value ever written.
|
||||
|
||||
The wrapper's `applyProperties` function accepts both: a bare
|
||||
value uses `.set()`; a sentinel-wrapped value
|
||||
`['__setOnce__', value]` uses `.setOnce()`. Releases that add new
|
||||
user properties **MUST** classify each one explicitly, by the
|
||||
following rule:
|
||||
|
||||
- A property whose value is **expected to change over the user's
|
||||
lifetime** is `set`. Examples: `role` (can flip from
|
||||
`contributor` to `owner`), `permission_state` (pending → granted),
|
||||
`passcode_set` (false → true), `device_trusted` (changes per
|
||||
active device). On each sign-in, the latest value is written;
|
||||
the dashboard always sees current state.
|
||||
- A property that is an **immutable historical marker** is
|
||||
`setOnce`. Examples: `first_sign_in_at` (the timestamp of the
|
||||
user's first observed sign-in — never re-write), `account_
|
||||
created_at` (the timestamp of provisioning),
|
||||
`invited_by_admin_id` (the admin who provisioned this user via
|
||||
the v0.17.0 path — preserved even if the user is later
|
||||
re-invited or has their role changed), `invited_at` (the
|
||||
timestamp at which the invite was sent — distinct from
|
||||
`claim_method` which is also setOnce because once claim_method
|
||||
is `'admin-invite'`, that's the path this user took).
|
||||
|
||||
The classification is part of the release's contract — flipping a
|
||||
property from `set` to `setOnce` (or vice versa) mid-life corrupts
|
||||
the user record and **MUST** be avoided. If a property's
|
||||
semantics genuinely change, retire the old key and introduce a new
|
||||
one (same deprecation discipline as event renames in §21.1).
|
||||
|
||||
### 21.7 Cohort-shape implications (informative)
|
||||
|
||||
The conventions above are designed so that the Amplitude dashboard
|
||||
can answer cohort questions the operator actually asks:
|
||||
|
||||
- *"How many users signed in via the admin-invite path in week N
|
||||
vs. organic OTC?"* — uses `claim_method` (setOnce) on the user
|
||||
record + `User Signed In` events with `method`.
|
||||
- *"Of admin-invited users, what fraction set a passcode within
|
||||
their first session?"* — uses `claim_method` + `passcode_set`
|
||||
on the user record + `Page Viewed` events to define "session."
|
||||
- *"Which RFCs have the most owner-invited contributors?"* — uses
|
||||
per-RFC `Invitation Sent` / `Invitation Accepted` correlated
|
||||
via `rfc_slug` + `role_in_rfc`.
|
||||
- *"What's the gap between invite-send and invite-claim, broken
|
||||
out by inviter?"* — uses `Invitation Sent` (inviter's session,
|
||||
inviter `user_id`) + `Invitation Accepted` (invitee's session,
|
||||
invitee `user_id` after the BEFORE-track identify) joined on
|
||||
`rfc_slug` + inviter (the inviter's id is the same id on both
|
||||
events because both invite and claim sides observe it).
|
||||
|
||||
The Part-A audit (per `ohm-rfc/ROADMAP.md` #21) confirms these
|
||||
shapes against real data once a week of beta traffic is in. The
|
||||
audit is a point-in-time pass; this chapter is the standing
|
||||
discipline that keeps future work in shape.
|
||||
|
||||
### 21.8 Secret vs. public — overlay binding
|
||||
|
||||
The Amplitude browser API key (`VITE_AMPLITUDE_API_KEY`) is
|
||||
**bundle-embedded by design**: it appears as a literal string in
|
||||
the deployment's JavaScript bundle, visible to anyone with browser
|
||||
dev tools. The vendor's installation prompt embeds it inline. This
|
||||
puts it in the same category as Cloudflare Turnstile's site key
|
||||
(`VITE_TURNSTILE_SITE_KEY`, v0.12.0) — public, not secret.
|
||||
|
||||
Deployments **MUST** bind such keys via their overlay verb (for
|
||||
OHM-shape deployments, `flotilla overlay set`), not via the secret
|
||||
binding. The matching secret half (the Cloudflare Turnstile
|
||||
**secret** key, `CLOUDFLARE_TURNSTILE_SECRET`, used server-side
|
||||
for siteverify) is a true secret bound via `flotilla secret set`.
|
||||
This per-key distinction is the deployment's responsibility; the
|
||||
framework's `*.env.example` files name the binding for each.
|
||||
|
||||
The binding rule baked in mid-Session-K is: **the operator's
|
||||
secret bytes never enter the conversation with an assistant**,
|
||||
even as one offered option. The conversation-layer corollary of
|
||||
the build-pipeline §3-invariant-1 rule from
|
||||
`ohm-rfc-app-flotilla/SPEC.md` is that sessions publish in full,
|
||||
and a secret in a transcript is a leaked secret. The canonical
|
||||
secret-set gesture for OHM is `pbpaste | flotilla secret set
|
||||
<deployment> <SECRET_NAME>` (the value goes clipboard → stdin →
|
||||
Secret Manager without ever appearing in shell history or the
|
||||
model context). Non-OHM deployments inherit the same discipline
|
||||
through their own deploy tooling.
|
||||
|
||||
### 21.9 §19.2 candidates surfaced by this chapter
|
||||
|
||||
- **Session-replay-specific consent category.** v0.13.0's cookie
|
||||
banner has a single `analytics` toggle that gates both event
|
||||
counters and full-DOM session replay. Recording has a larger
|
||||
privacy footprint than counters; a separate consent category
|
||||
for session replay is the cleaner shape. Captured here and in
|
||||
`ohm-rfc/ROADMAP.md` #21 Part A.
|
||||
- **Bundle-size budget for the analytics wrapper.** The
|
||||
`@amplitude/unified` package adds ~150 KB gzipped (the session-
|
||||
replay recorder is the bulk). The consent-gated lazy import
|
||||
keeps the cost off the initial bundle for users who haven't
|
||||
opted in; the post-consent init path has not been measured
|
||||
for jank. Captured in #21 Part A.
|
||||
- **Property-shape lint.** The conventions in §21.1 / §21.2 are
|
||||
enforced today by review discipline. A small lint (CI grep
|
||||
against `track(` / `identify(` callsites with a property-key
|
||||
allowlist + a PII-name denylist) is a future affordance that
|
||||
catches drift mechanically.
|
||||
- **Hashed `target_email` derivation.** §21.2 names SHA-256 of
|
||||
the normalized lower-cased email as the hashing function. The
|
||||
framework does not currently expose a helper for this — a
|
||||
small `frontend/src/lib/hash.js` or `backend/app/hash.py` that
|
||||
centralizes the normalization + hash would make the contract
|
||||
enforceable. Captured here.
|
||||
|
||||
### 21.10 Open question
|
||||
|
||||
The wrapper currently uses the Amplitude SDK's autocapture +
|
||||
session-replay defaults. The v0.13.0 consent surface has a single
|
||||
toggle for "analytics." Splitting the consent into "analytics"
|
||||
vs "session replay" is a §19.2 candidate (above) but settling it
|
||||
also requires a privacy-policy update and a re-prompt of
|
||||
existing consenters. The cleanest moment to do this is the next
|
||||
material privacy-policy revision; the conventions in §21.4 hold
|
||||
in the interim.
|
||||
|
||||
|
||||
@@ -672,73 +672,6 @@ def make_router(config: Config) -> APIRouter:
|
||||
"has_more": len(rows) == limit,
|
||||
}
|
||||
|
||||
@router.get("/api/admin/outbound-emails")
|
||||
async def list_outbound_emails(
|
||||
request: Request,
|
||||
kind: str | None = None,
|
||||
status: str | None = None,
|
||||
to_address: str | None = None,
|
||||
limit: int = Query(default=100, ge=1, le=500),
|
||||
before_id: int | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""v0.18.0 Slice 4: read-only inspection of the
|
||||
`outbound_emails` audit table.
|
||||
|
||||
Answers questions like "did this person ever get their
|
||||
invite?" without grepping VM logs. Filterable by kind
|
||||
('otc' | 'invite' | 'notification' | 'bundle' | 'digest'),
|
||||
status ('sent' | 'failed' | 'deferred' | 'bounced'), and
|
||||
to_address; the latter is exact-match because the audit
|
||||
question is usually "the specific person who said they
|
||||
didn't receive it." Per the proposal, no admin UI ships
|
||||
with v0.18.0 — operator queries via curl + jq for now.
|
||||
"""
|
||||
auth.require_admin(request)
|
||||
clauses: list[str] = []
|
||||
args: list[Any] = []
|
||||
if kind:
|
||||
clauses.append("kind = ?")
|
||||
args.append(kind)
|
||||
if status:
|
||||
clauses.append("status = ?")
|
||||
args.append(status)
|
||||
if to_address:
|
||||
clauses.append("LOWER(to_address) = LOWER(?)")
|
||||
args.append(to_address)
|
||||
if before_id is not None:
|
||||
clauses.append("id < ?")
|
||||
args.append(before_id)
|
||||
where = ("WHERE " + " AND ".join(clauses)) if clauses else ""
|
||||
rows = db.conn().execute(
|
||||
f"""
|
||||
SELECT id, to_address, from_address, subject, kind, sent_at,
|
||||
status, error, notification_id, message_id
|
||||
FROM outbound_emails
|
||||
{where}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
""",
|
||||
(*args, limit),
|
||||
).fetchall()
|
||||
return {
|
||||
"items": [
|
||||
{
|
||||
"id": r["id"],
|
||||
"to_address": r["to_address"],
|
||||
"from_address": r["from_address"],
|
||||
"subject": r["subject"],
|
||||
"kind": r["kind"],
|
||||
"sent_at": r["sent_at"],
|
||||
"status": r["status"],
|
||||
"error": r["error"],
|
||||
"notification_id": r["notification_id"],
|
||||
"message_id": r["message_id"],
|
||||
}
|
||||
for r in rows
|
||||
],
|
||||
"has_more": len(rows) == limit,
|
||||
}
|
||||
|
||||
@router.get("/api/admin/permission-events")
|
||||
async def list_permission_events(
|
||||
request: Request,
|
||||
|
||||
@@ -73,13 +73,6 @@ class MarkReadBody(BaseModel):
|
||||
class BounceBody(BaseModel):
|
||||
email: str = Field(min_length=3, max_length=320)
|
||||
kind: str = Field(default="hard") # 'hard' or 'complaint'
|
||||
# v0.18.0 Slice 5: when the bounce provider includes the
|
||||
# original Message-ID, the framework correlates it back to
|
||||
# the matching `outbound_emails` row and stamps
|
||||
# `status='bounced'`. Optional — providers that don't surface
|
||||
# the Message-ID still flip the global opt-out via the email
|
||||
# match, but lose the per-message attribution.
|
||||
message_id: str | None = Field(default=None, max_length=1000)
|
||||
|
||||
|
||||
class CookieConsentBody(BaseModel):
|
||||
@@ -450,40 +443,6 @@ def make_router(config: Config) -> APIRouter:
|
||||
|
||||
# ----- Email: one-click unsubscribe + bounce webhook -----
|
||||
|
||||
# v0.18.0: the category → column map. The `all` synthetic
|
||||
# category lands the bundle's one-click on the global opt-out
|
||||
# flag (per `email._send_bundle` in v0.18.0 Slice 2 — a bundle
|
||||
# spans multiple categories, so a per-category flip wouldn't
|
||||
# honor the user's intent).
|
||||
_CATEGORY_COLUMN: dict[str, str] = {
|
||||
"personal-direct": "email_personal_direct",
|
||||
"structural": "email_watched_structural",
|
||||
"admin-actionable": "email_admin_actionable",
|
||||
"all": "email_opt_out_all",
|
||||
}
|
||||
|
||||
def _apply_unsubscribe(user_id: int, category: str) -> bool:
|
||||
"""Flip the matching column. Returns True on success, False
|
||||
if the category is unknown. Idempotent — running twice on
|
||||
the same (user, category) is harmless (it sets the column
|
||||
to its current value)."""
|
||||
column = _CATEGORY_COLUMN.get(category)
|
||||
if column is None:
|
||||
return False
|
||||
# `all` sets the flag to 1 (opt out); per-category sets to 0
|
||||
# (turn that category off). The column semantic is "1 means
|
||||
# don't send"; the per-category booleans are "1 means do
|
||||
# send". Different polarities, hence the case split.
|
||||
if category == "all":
|
||||
db.conn().execute(
|
||||
f"UPDATE users SET {column} = 1 WHERE id = ?", (user_id,)
|
||||
)
|
||||
else:
|
||||
db.conn().execute(
|
||||
f"UPDATE users SET {column} = 0 WHERE id = ?", (user_id,)
|
||||
)
|
||||
return True
|
||||
|
||||
@router.get("/api/email/unsubscribe")
|
||||
async def email_unsubscribe(t: str = Query(..., description="Signed token from the email footer")) -> HTMLResponse:
|
||||
try:
|
||||
@@ -494,51 +453,20 @@ def make_router(config: Config) -> APIRouter:
|
||||
"<p>Open the app to manage your notification preferences directly.</p>",
|
||||
status_code=400,
|
||||
)
|
||||
if not _apply_unsubscribe(user_id, category):
|
||||
column = {
|
||||
"personal-direct": "email_personal_direct",
|
||||
"structural": "email_watched_structural",
|
||||
"admin-actionable": "email_admin_actionable",
|
||||
}.get(category)
|
||||
if column is None:
|
||||
return HTMLResponse(
|
||||
f"<h1>Unknown category</h1><p>{category}</p>", status_code=400
|
||||
)
|
||||
if category == "all":
|
||||
body = (
|
||||
"<h1>Unsubscribed</h1><p>You will no longer receive any email "
|
||||
"from this app. You can re-enable individual categories from "
|
||||
"your notification preferences after signing in.</p>"
|
||||
)
|
||||
else:
|
||||
body = (
|
||||
f"<h1>Unsubscribed</h1><p>You will no longer receive {category} emails. "
|
||||
f"You can re-enable them in your notification preferences.</p>"
|
||||
)
|
||||
return HTMLResponse(body)
|
||||
|
||||
@router.post("/api/email/unsubscribe")
|
||||
async def email_unsubscribe_post(
|
||||
request: Request,
|
||||
t: str = Query(..., description="Signed token from the List-Unsubscribe header"),
|
||||
) -> dict[str, Any]:
|
||||
"""v0.18.0: RFC 8058 one-click endpoint.
|
||||
|
||||
Gmail and Yahoo POST `List-Unsubscribe=One-Click` (as a
|
||||
form-encoded body) to the URL in the `List-Unsubscribe`
|
||||
header when the user clicks their MUA's "Unsubscribe"
|
||||
button. The endpoint MUST accept POST (per the
|
||||
`List-Unsubscribe-Post` header we advertise) and MUST be
|
||||
idempotent.
|
||||
|
||||
The body content is checked loosely — RFC 8058 says it
|
||||
SHOULD be exactly `List-Unsubscribe=One-Click`, but some
|
||||
intermediaries strip / re-encode the body, so the
|
||||
framework accepts any POST to the URL once the token
|
||||
verifies. The bar is that the token signature carries the
|
||||
authority; the body is hint-only.
|
||||
"""
|
||||
try:
|
||||
user_id, category = email_mod.verify_unsubscribe_token(t)
|
||||
except BadSignature:
|
||||
raise HTTPException(400, "Invalid or expired token")
|
||||
if not _apply_unsubscribe(user_id, category):
|
||||
raise HTTPException(400, f"Unknown category: {category}")
|
||||
return {"ok": True, "category": category}
|
||||
db.conn().execute(f"UPDATE users SET {column} = 0 WHERE id = ?", (user_id,))
|
||||
return HTMLResponse(
|
||||
f"<h1>Unsubscribed</h1><p>You will no longer receive {category} emails. "
|
||||
f"You can re-enable them in your notification preferences.</p>"
|
||||
)
|
||||
|
||||
@router.post("/api/webhooks/email-bounce")
|
||||
async def email_bounce(body: BounceBody, request: Request) -> dict[str, Any]:
|
||||
@@ -562,46 +490,16 @@ def make_router(config: Config) -> APIRouter:
|
||||
import hmac as _hmac
|
||||
if not received or not _hmac.compare_digest(expected, received):
|
||||
raise HTTPException(401, "Invalid webhook signature")
|
||||
# v0.18.0 Slice 5: correlate the bounce back to the
|
||||
# matching outbound_emails row if the provider supplied
|
||||
# the Message-ID. The hard-bounce -> global-opt-out
|
||||
# logic below still fires regardless; this is an
|
||||
# additional audit signal.
|
||||
correlated_row_id: int | None = None
|
||||
if body.message_id:
|
||||
correlated = db.conn().execute(
|
||||
"SELECT id FROM outbound_emails WHERE message_id = ?",
|
||||
(body.message_id,),
|
||||
).fetchone()
|
||||
if correlated is not None:
|
||||
correlated_row_id = correlated["id"]
|
||||
db.conn().execute(
|
||||
"UPDATE outbound_emails SET status = 'bounced', "
|
||||
"error = COALESCE(error, '') || ? WHERE id = ?",
|
||||
(f"bounce ({body.kind})", correlated_row_id),
|
||||
)
|
||||
log.info(
|
||||
"email-bounce: correlated message_id=%s -> outbound_emails.id=%s",
|
||||
body.message_id, correlated_row_id,
|
||||
)
|
||||
else:
|
||||
log.info(
|
||||
"email-bounce: message_id=%s did not match any "
|
||||
"outbound_emails row (provider may be replaying an old bounce, "
|
||||
"or the row was pruned)",
|
||||
body.message_id,
|
||||
)
|
||||
|
||||
row = db.conn().execute(
|
||||
"SELECT id FROM users WHERE LOWER(email) = LOWER(?)", (body.email,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return {"ok": True, "matched": False, "correlated_id": correlated_row_id}
|
||||
return {"ok": True, "matched": False}
|
||||
db.conn().execute(
|
||||
"UPDATE users SET email_opt_out_all = 1 WHERE id = ?", (row["id"],),
|
||||
)
|
||||
log.info("email-bounce: opted out user %s (%s)", row["id"], body.kind)
|
||||
return {"ok": True, "matched": True, "correlated_id": correlated_row_id}
|
||||
return {"ok": True, "matched": True}
|
||||
|
||||
return router
|
||||
|
||||
|
||||
+1
-15
@@ -60,20 +60,6 @@ def load_config() -> Config:
|
||||
|
||||
enabled = [m.strip() for m in _optional("ENABLED_MODELS", "claude").split(",") if m.strip()]
|
||||
|
||||
# v0.18.0: `GITEA_WEBHOOK_SECRET` is now mandatory (per the
|
||||
# email + webhook hygiene proposal). An empty value used to
|
||||
# silently accept unsigned webhook POSTs — that was the
|
||||
# invisible-failure shape the proposal targets. Now the
|
||||
# framework refuses to start when the secret is empty unless
|
||||
# the operator opts into the dev-bypass with
|
||||
# `RFC_APP_INSECURE_WEBHOOKS=1`. Local-dev deployments without
|
||||
# a wired Gitea hook set the bypass; production MUST NOT.
|
||||
insecure_webhooks = os.environ.get("RFC_APP_INSECURE_WEBHOOKS", "").strip() == "1"
|
||||
if insecure_webhooks:
|
||||
webhook_secret = _optional("GITEA_WEBHOOK_SECRET")
|
||||
else:
|
||||
webhook_secret = _required("GITEA_WEBHOOK_SECRET")
|
||||
|
||||
return Config(
|
||||
gitea_url=_required("GITEA_URL").rstrip("/"),
|
||||
gitea_bot_user=_required("GITEA_BOT_USER"),
|
||||
@@ -86,7 +72,7 @@ def load_config() -> Config:
|
||||
secret_key=_required("SECRET_KEY"),
|
||||
database_path=database_path,
|
||||
owner_gitea_login=_optional("OWNER_GITEA_LOGIN"),
|
||||
webhook_secret=webhook_secret,
|
||||
webhook_secret=_optional("GITEA_WEBHOOK_SECRET"),
|
||||
enabled_models=enabled,
|
||||
anthropic_api_key=_optional("ANTHROPIC_API_KEY"),
|
||||
google_api_key=_optional("GOOGLE_API_KEY"),
|
||||
|
||||
+1
-13
@@ -180,19 +180,7 @@ def assemble_for_user(
|
||||
|
||||
subject = _subject(eligible, cadence)
|
||||
body = _body(eligible, cadence, cfg)
|
||||
# v0.18.0: the digest is the bulk-adjacent surface par excellence
|
||||
# (it can carry weeks of accumulated activity), so it gets the
|
||||
# full one-click unsubscribe to the global opt-out. Per-category
|
||||
# opt-outs are managed from the preferences page; this footer is
|
||||
# the "stop sending me anything" escape hatch Gmail and Yahoo
|
||||
# expect for senders at this tier.
|
||||
unsubscribe_url = email_mod.make_unsubscribe_url(user_id, "all")
|
||||
sent = email_mod._deliver(
|
||||
cfg, email, subject, body,
|
||||
unsubscribe_mailto=cfg.unsubscribe_mailto,
|
||||
unsubscribe_url=unsubscribe_url,
|
||||
kind="digest",
|
||||
)
|
||||
sent = email_mod._deliver(cfg, email, subject, body)
|
||||
if not sent:
|
||||
return False
|
||||
ids = [r["id"] for r, _ in eligible]
|
||||
|
||||
+11
-166
@@ -24,6 +24,7 @@ import os
|
||||
import smtplib
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, time, timezone
|
||||
from email.message import EmailMessage
|
||||
from email.utils import formataddr
|
||||
from itertools import groupby
|
||||
from typing import Any
|
||||
@@ -32,7 +33,6 @@ from urllib.parse import urlencode
|
||||
from itsdangerous import BadSignature, URLSafeSerializer
|
||||
|
||||
from . import db
|
||||
from .email_envelope import build_envelope
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -69,7 +69,6 @@ class EmailConfig:
|
||||
app_url: str
|
||||
bundle_threshold: int
|
||||
enabled: bool
|
||||
unsubscribe_mailto: str
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> "EmailConfig":
|
||||
@@ -85,16 +84,6 @@ class EmailConfig:
|
||||
app_url=os.environ.get("APP_URL", "http://localhost:8000").rstrip("/"),
|
||||
bundle_threshold=int(os.environ.get("EMAIL_BUNDLE_THRESHOLD", "5")),
|
||||
enabled=os.environ.get("EMAIL_ENABLED", "1") not in ("0", "false", "False"),
|
||||
# v0.18.0: the `List-Unsubscribe: <mailto:…>` target on
|
||||
# invite + notification mail. Defaults to the From
|
||||
# address when unset; a deployment can route opt-out
|
||||
# mail to a separate mailbox (e.g., a humans-monitored
|
||||
# account distinct from the no-reply notifications
|
||||
# sender) by setting this explicitly.
|
||||
unsubscribe_mailto=os.environ.get(
|
||||
"EMAIL_UNSUBSCRIBE_MAILTO",
|
||||
os.environ.get("EMAIL_FROM", "notifications@wiggleverse.local"),
|
||||
).strip(),
|
||||
)
|
||||
|
||||
|
||||
@@ -109,14 +98,6 @@ def _signer() -> URLSafeSerializer:
|
||||
|
||||
|
||||
def make_unsubscribe_url(user_id: int, category: str) -> str:
|
||||
"""Build the §15.4 per-category one-click URL.
|
||||
|
||||
`category` is one of `personal-direct`, `structural`,
|
||||
`admin-actionable` (the three per-category flags) or `all`
|
||||
(v0.18.0: the bundle path, which sets `email_opt_out_all = 1`
|
||||
because a bundle covers multiple categories and a per-category
|
||||
opt-out wouldn't honor the user's intent).
|
||||
"""
|
||||
cfg = EmailConfig.from_env()
|
||||
token = _signer().dumps({"u": user_id, "c": category})
|
||||
qs = urlencode({"t": token})
|
||||
@@ -269,21 +250,7 @@ def _send_one(user: Any, notif_id: int, payload: dict, category: str) -> None:
|
||||
return
|
||||
subject = _subject(payload)
|
||||
body = _body(payload, user["id"], category, cfg)
|
||||
# v0.18.0: notification mail is bulk-adjacent (a watcher can
|
||||
# accumulate dozens of structural events on a busy RFC), so it
|
||||
# carries the full one-click unsubscribe — Gmail and Yahoo
|
||||
# require this for senders at OHM's volume tier per RFC 8058.
|
||||
unsubscribe_url = make_unsubscribe_url(user["id"], category)
|
||||
sent = _deliver(
|
||||
cfg,
|
||||
user["email"],
|
||||
subject,
|
||||
body,
|
||||
unsubscribe_mailto=cfg.unsubscribe_mailto,
|
||||
unsubscribe_url=unsubscribe_url,
|
||||
kind="notification",
|
||||
notification_id=notif_id,
|
||||
)
|
||||
sent = _deliver(cfg, user["email"], subject, body)
|
||||
if not sent:
|
||||
return
|
||||
db.conn().execute(
|
||||
@@ -338,65 +305,23 @@ def _deep_link(payload: dict, cfg: EmailConfig) -> str:
|
||||
return cfg.app_url
|
||||
|
||||
|
||||
def _deliver(
|
||||
cfg: EmailConfig,
|
||||
to_address: str,
|
||||
subject: str,
|
||||
body: str,
|
||||
*,
|
||||
unsubscribe_mailto: str | None = None,
|
||||
unsubscribe_url: str | None = None,
|
||||
kind: str = "notification",
|
||||
notification_id: int | None = None,
|
||||
) -> bool:
|
||||
"""Build the envelope via the shared `build_envelope` helper and
|
||||
hand it to SMTP.
|
||||
|
||||
The `_SENT` buffer carries the helper's `EmailMessage` under
|
||||
`message` plus the legacy `to`/`from`/`subject`/`body` keys for
|
||||
backward-compatibility with tests that read those directly.
|
||||
Newer tests can assert on the header surface by inspecting
|
||||
`envelope["message"]`.
|
||||
|
||||
v0.18.0 Slice 4: also writes one row to `outbound_emails`
|
||||
capturing the attempt. status='sent' on success, 'failed' on
|
||||
SMTP exception, 'deferred' on the dev-fallback path (no
|
||||
SMTP_HOST configured — the send didn't happen, but the row
|
||||
records the attempt so the admin endpoint can answer "did the
|
||||
framework try?").
|
||||
"""
|
||||
msg = build_envelope(
|
||||
to_address=to_address,
|
||||
from_address=cfg.from_address,
|
||||
from_name=cfg.from_name,
|
||||
subject=subject,
|
||||
body_plain=body,
|
||||
unsubscribe_mailto=unsubscribe_mailto,
|
||||
unsubscribe_url=unsubscribe_url,
|
||||
)
|
||||
def _deliver(cfg: EmailConfig, to_address: str, subject: str, body: str) -> bool:
|
||||
envelope = {
|
||||
"to": to_address,
|
||||
"from": formataddr((cfg.from_name, cfg.from_address)),
|
||||
"subject": subject,
|
||||
"body": body,
|
||||
"message": msg,
|
||||
"kind": kind,
|
||||
}
|
||||
_SENT.append(envelope)
|
||||
message_id = msg["Message-ID"]
|
||||
if not cfg.smtp_host:
|
||||
log.info("email (stdout fallback): to=%s subject=%s", to_address, subject)
|
||||
record_outbound(
|
||||
to_address=to_address,
|
||||
from_address=cfg.from_address,
|
||||
subject=subject,
|
||||
kind=kind,
|
||||
status="deferred",
|
||||
message_id=message_id,
|
||||
notification_id=notification_id,
|
||||
)
|
||||
return True
|
||||
try:
|
||||
msg = EmailMessage()
|
||||
msg["From"] = envelope["from"]
|
||||
msg["To"] = to_address
|
||||
msg["Subject"] = subject
|
||||
msg.set_content(body)
|
||||
smtp = smtplib.SMTP(cfg.smtp_host, cfg.smtp_port, timeout=30)
|
||||
try:
|
||||
if cfg.smtp_starttls:
|
||||
@@ -406,76 +331,10 @@ def _deliver(
|
||||
smtp.send_message(msg)
|
||||
finally:
|
||||
smtp.quit()
|
||||
record_outbound(
|
||||
to_address=to_address,
|
||||
from_address=cfg.from_address,
|
||||
subject=subject,
|
||||
kind=kind,
|
||||
status="sent",
|
||||
message_id=message_id,
|
||||
notification_id=notification_id,
|
||||
)
|
||||
return True
|
||||
except Exception as exc:
|
||||
log.exception("email send failed: to=%s subject=%s", to_address, subject)
|
||||
record_outbound(
|
||||
to_address=to_address,
|
||||
from_address=cfg.from_address,
|
||||
subject=subject,
|
||||
kind=kind,
|
||||
status="failed",
|
||||
error=f"{type(exc).__name__}: {exc}",
|
||||
message_id=message_id,
|
||||
notification_id=notification_id,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def record_outbound(
|
||||
*,
|
||||
to_address: str,
|
||||
from_address: str,
|
||||
subject: str,
|
||||
kind: str,
|
||||
status: str,
|
||||
error: str | None = None,
|
||||
notification_id: int | None = None,
|
||||
message_id: str | None = None,
|
||||
) -> int | None:
|
||||
"""v0.18.0 Slice 4: write one row to `outbound_emails`.
|
||||
|
||||
Returns the inserted row's id, or `None` if the DB connection
|
||||
isn't initialized (which happens in unit tests that don't boot
|
||||
the full app — the write is best-effort and never raises).
|
||||
"""
|
||||
try:
|
||||
cur = db.conn().execute(
|
||||
"""
|
||||
INSERT INTO outbound_emails
|
||||
(to_address, from_address, subject, kind, sent_at, status,
|
||||
error, notification_id, message_id)
|
||||
VALUES (?, ?, ?, ?, datetime('now'), ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
to_address,
|
||||
from_address,
|
||||
subject,
|
||||
kind,
|
||||
status,
|
||||
error,
|
||||
notification_id,
|
||||
message_id,
|
||||
),
|
||||
)
|
||||
return cur.lastrowid
|
||||
except RuntimeError:
|
||||
# db.conn() raises RuntimeError if init() hasn't been called.
|
||||
# Pure-helper unit tests for build_envelope hit this path; the
|
||||
# audit row is best-effort and not part of the contract.
|
||||
return None
|
||||
except Exception:
|
||||
log.exception("outbound_emails write failed: to=%s subject=%s", to_address, subject)
|
||||
return None
|
||||
log.exception("email send failed: to=%s subject=%s", to_address, subject)
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -581,27 +440,13 @@ def _send_bundle(cfg: EmailConfig, user: Any, emailable: list) -> int:
|
||||
for r, _cat, extras in group_rows:
|
||||
summary = _summary_for(r["event_kind"], r["actor_display"], r["rfc_title"], extras)
|
||||
sections.append(f" · {summary}")
|
||||
# v0.18.0: the bundle covers multiple categories, so a
|
||||
# per-category opt-out can't honor the user's intent. The
|
||||
# `all` category lands at the §15.4 endpoint and sets
|
||||
# `email_opt_out_all = 1`.
|
||||
unsubscribe_url = make_unsubscribe_url(user["id"], "all")
|
||||
body = (
|
||||
"Activity on RFCs you watch, accumulated during your quiet hours:\n"
|
||||
+ "\n".join(sections)
|
||||
+ f"\n\nOpen your inbox: {cfg.app_url}/inbox\n"
|
||||
+ f"Manage all preferences: {cfg.app_url}/settings/notifications\n"
|
||||
+ f"Unsubscribe from all email: {unsubscribe_url}\n"
|
||||
)
|
||||
sent = _deliver(
|
||||
cfg,
|
||||
user["email"],
|
||||
subject,
|
||||
body,
|
||||
unsubscribe_mailto=cfg.unsubscribe_mailto,
|
||||
unsubscribe_url=unsubscribe_url,
|
||||
kind="bundle",
|
||||
)
|
||||
sent = _deliver(cfg, user["email"], subject, body)
|
||||
if not sent:
|
||||
return 0
|
||||
ids = [r["id"] for r, _, _ in emailable]
|
||||
|
||||
@@ -1,143 +0,0 @@
|
||||
"""v0.18.0 / roadmap items #18 + #20: a shared envelope builder.
|
||||
|
||||
Every outbound mail in rfc-app today (OTC, admin-invite, watcher
|
||||
notification, "while you were away" bundle, per-RFC invite) constructs
|
||||
its own `email.message.EmailMessage` ad-hoc. The four sites diverged
|
||||
just enough to be a deliverability hazard: missing `Date`, missing
|
||||
`Message-ID`, no `Auto-Submitted`, no `List-Unsubscribe` on the
|
||||
bulk-adjacent paths, no `multipart/alternative` body.
|
||||
|
||||
This module is the one place an `EmailMessage` is constructed. Every
|
||||
send path imports `build_envelope` and calls it; the headers that
|
||||
matter for inbox placement (Date, Message-ID, Auto-Submitted) land
|
||||
uniformly, and the per-kind variations (unsubscribe semantics,
|
||||
HTML alternative) are explicit arguments rather than buried in
|
||||
each call site.
|
||||
|
||||
Per the v0.18.0 proposal at `~/git/ohm-infra/RFC-APP-EMAIL-HYGIENE-PROPOSAL.md`,
|
||||
the per-kind unsubscribe matrix is:
|
||||
|
||||
* OTC: no `List-Unsubscribe` (the recipient explicitly requested
|
||||
the code; advertising an unsubscribe header would imply OHM has
|
||||
them on a list, which it doesn't).
|
||||
* Admin invite / per-RFC invite: `mailto:` form only (the
|
||||
recipient isn't a user yet, so there's no per-user opt-out row
|
||||
to flip; the operator handles ad-hoc opt-outs manually).
|
||||
* Watcher notification / bundle: full `mailto:` + signed-URL
|
||||
`List-Unsubscribe` plus `List-Unsubscribe-Post:
|
||||
List-Unsubscribe=One-Click` per RFC 8058 (Gmail and Yahoo
|
||||
enforce this for bulk-adjacent senders).
|
||||
|
||||
The `is_transactional` flag governs `Auto-Submitted: auto-generated`,
|
||||
which prevents auto-responder loops on every kind of mail we send.
|
||||
All five mail kinds today are transactional in the SMTP sense (no
|
||||
human is at the From mailbox watching for replies), so the default
|
||||
is True; the argument is exposed for future symmetry.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from email.message import EmailMessage
|
||||
from email.utils import formataddr, formatdate, make_msgid
|
||||
|
||||
|
||||
def build_envelope(
|
||||
*,
|
||||
to_address: str,
|
||||
from_address: str,
|
||||
from_name: str,
|
||||
subject: str,
|
||||
body_plain: str,
|
||||
body_html: str | None = None,
|
||||
reply_to: str | None = None,
|
||||
unsubscribe_mailto: str | None = None,
|
||||
unsubscribe_url: str | None = None,
|
||||
is_transactional: bool = True,
|
||||
msgid_domain: str | None = None,
|
||||
) -> EmailMessage:
|
||||
"""Compose an `EmailMessage` with hardened headers.
|
||||
|
||||
`to_address` / `from_address` are bare RFC 5322 addresses;
|
||||
`from_name` is the display label that goes through `formataddr`
|
||||
so spaces / commas in the display string are encoded correctly.
|
||||
|
||||
`body_plain` is mandatory. `body_html`, if supplied, lands as the
|
||||
second part of a `multipart/alternative` body — mail clients
|
||||
that prefer HTML render it; clients that don't fall back to the
|
||||
plain part. The text/plain part comes first per RFC 2046, so a
|
||||
plain-text client that picks the first body gets the readable
|
||||
text.
|
||||
|
||||
`reply_to`, when set, lets a send path point replies at a
|
||||
different mailbox than the From line (e.g., a watcher
|
||||
notification with From=notifications@... but Reply-To=
|
||||
ohm@... so a confused recipient who hits Reply lands at a
|
||||
monitored mailbox).
|
||||
|
||||
`unsubscribe_mailto` / `unsubscribe_url` populate
|
||||
`List-Unsubscribe`. If `unsubscribe_url` is set, the helper also
|
||||
emits `List-Unsubscribe-Post: List-Unsubscribe=One-Click` per
|
||||
RFC 8058 — Gmail and Yahoo POST that payload on the user's
|
||||
one-click action. (Send paths that wire `unsubscribe_url`
|
||||
therefore MUST also expose a matching POST endpoint that accepts
|
||||
the same token; see `api_notifications.py:email_unsubscribe`.)
|
||||
|
||||
`msgid_domain` defaults to the @-domain of `from_address` so
|
||||
Message-IDs are aligned with the sending domain by default. A
|
||||
deployment that wants the Message-ID domain to track a different
|
||||
surface (e.g., a tracking-domain that's separate from the From
|
||||
domain) can override.
|
||||
|
||||
`Date` is RFC 5322 formatted via `email.utils.formatdate`; the
|
||||
`localtime=True` setting picks the running process's local
|
||||
timezone, which is what every popular MUA does too. (A
|
||||
deployment running in UTC stamps UTC; that's correct, not a
|
||||
bug.)
|
||||
"""
|
||||
msg = EmailMessage()
|
||||
msg["From"] = formataddr((from_name, from_address))
|
||||
msg["To"] = to_address
|
||||
msg["Subject"] = subject
|
||||
msg["Date"] = formatdate(localtime=True)
|
||||
# If the caller didn't pin a Message-ID domain, derive it from the
|
||||
# From address. `make_msgid` accepts None and falls back to the
|
||||
# local hostname, which is the wrong shape for a deliverable
|
||||
# message (the hostname might be `gke-pool-xxx`); a deployment
|
||||
# without a configured From would surface that as a build-time
|
||||
# config error elsewhere, so the fallback here is just defensive.
|
||||
if msgid_domain is None:
|
||||
if "@" in from_address:
|
||||
msgid_domain = from_address.split("@", 1)[1]
|
||||
else:
|
||||
msgid_domain = "localhost"
|
||||
msg["Message-ID"] = make_msgid(domain=msgid_domain)
|
||||
if reply_to:
|
||||
msg["Reply-To"] = reply_to
|
||||
if is_transactional:
|
||||
# RFC 3834: prevents auto-responders (vacation replies, etc.)
|
||||
# from triggering on this message. Every kind of mail rfc-app
|
||||
# sends today is transactional in this sense.
|
||||
msg["Auto-Submitted"] = "auto-generated"
|
||||
if unsubscribe_mailto or unsubscribe_url:
|
||||
parts: list[str] = []
|
||||
if unsubscribe_mailto:
|
||||
parts.append(f"<mailto:{unsubscribe_mailto}>")
|
||||
if unsubscribe_url:
|
||||
parts.append(f"<{unsubscribe_url}>")
|
||||
msg["List-Unsubscribe"] = ", ".join(parts)
|
||||
if unsubscribe_url:
|
||||
# RFC 8058 one-click. Gmail and Yahoo POST the payload
|
||||
# `List-Unsubscribe=One-Click` to the URL on the user's
|
||||
# one-click action; the matching POST endpoint must be
|
||||
# idempotent and not require auth. See
|
||||
# `api_notifications.py` for the receiver.
|
||||
msg["List-Unsubscribe-Post"] = "List-Unsubscribe=One-Click"
|
||||
if body_html:
|
||||
# multipart/alternative: text/plain first, text/html second.
|
||||
# `set_content` sets the first part (and the message's main
|
||||
# body); `add_alternative` adds the second part and
|
||||
# restructures the message as multipart/alternative.
|
||||
msg.set_content(body_plain)
|
||||
msg.add_alternative(body_html, subtype="html")
|
||||
else:
|
||||
msg.set_content(body_plain)
|
||||
return msg
|
||||
@@ -31,10 +31,10 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import smtplib
|
||||
from email.message import EmailMessage
|
||||
from email.utils import formataddr
|
||||
|
||||
from .email import EmailConfig, _SENT, record_outbound
|
||||
from .email_envelope import build_envelope
|
||||
from .email import EmailConfig, _SENT
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -59,52 +59,31 @@ def send_invite_email(
|
||||
cfg = EmailConfig.from_env()
|
||||
subject = _subject(inviter_display, cfg)
|
||||
body = _body(claim_url, inviter_display, inviter_email, custom_message, cfg)
|
||||
# v0.18.0: invite mail carries a `List-Unsubscribe: <mailto:…>`
|
||||
# only (no signed URL) — the invitee isn't a user yet, so there
|
||||
# is no per-user opt-out row to flip. The operator handles
|
||||
# ad-hoc opt-outs from the mailto: target. Per the proposal's
|
||||
# "Tradeoff discussion": the invite was unsolicited from the
|
||||
# recipient's perspective, so the courtesy header is right;
|
||||
# but it can't be a one-click URL because the row doesn't
|
||||
# exist yet.
|
||||
msg = build_envelope(
|
||||
to_address=to_address,
|
||||
from_address=cfg.from_address,
|
||||
from_name=cfg.from_name,
|
||||
subject=subject,
|
||||
body_plain=body,
|
||||
unsubscribe_mailto=cfg.unsubscribe_mailto,
|
||||
)
|
||||
envelope = {
|
||||
"to": to_address,
|
||||
"from": formataddr((cfg.from_name, cfg.from_address)),
|
||||
"subject": subject,
|
||||
"body": body,
|
||||
"kind": "invite",
|
||||
"message": msg,
|
||||
}
|
||||
_SENT.append(envelope)
|
||||
|
||||
message_id = msg["Message-ID"]
|
||||
if not cfg.enabled:
|
||||
log.info("invite email disabled (EMAIL_ENABLED=0): to=%s", to_address)
|
||||
record_outbound(
|
||||
to_address=to_address, from_address=cfg.from_address,
|
||||
subject=subject, kind="invite", status="deferred", message_id=message_id,
|
||||
)
|
||||
return True
|
||||
if not cfg.smtp_host:
|
||||
# Dev fallback: surface the claim URL at INFO so the operator can
|
||||
# complete a claim flow without an SMTP relay. In production
|
||||
# SMTP_HOST is always set per OHM's overlay.
|
||||
log.info("invite email (stdout fallback): to=%s claim_url=%s", to_address, claim_url)
|
||||
record_outbound(
|
||||
to_address=to_address, from_address=cfg.from_address,
|
||||
subject=subject, kind="invite", status="deferred", message_id=message_id,
|
||||
)
|
||||
return True
|
||||
|
||||
try:
|
||||
msg = EmailMessage()
|
||||
msg["From"] = envelope["from"]
|
||||
msg["To"] = to_address
|
||||
msg["Subject"] = subject
|
||||
msg.set_content(body)
|
||||
smtp = smtplib.SMTP(cfg.smtp_host, cfg.smtp_port, timeout=30)
|
||||
try:
|
||||
if cfg.smtp_starttls:
|
||||
@@ -114,18 +93,9 @@ def send_invite_email(
|
||||
smtp.send_message(msg)
|
||||
finally:
|
||||
smtp.quit()
|
||||
record_outbound(
|
||||
to_address=to_address, from_address=cfg.from_address,
|
||||
subject=subject, kind="invite", status="sent", message_id=message_id,
|
||||
)
|
||||
return True
|
||||
except Exception as exc:
|
||||
except Exception:
|
||||
log.exception("invite email send failed: to=%s", to_address)
|
||||
record_outbound(
|
||||
to_address=to_address, from_address=cfg.from_address,
|
||||
subject=subject, kind="invite", status="failed",
|
||||
error=f"{type(exc).__name__}: {exc}", message_id=message_id,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
|
||||
@@ -23,10 +23,10 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import smtplib
|
||||
from email.message import EmailMessage
|
||||
from email.utils import formataddr
|
||||
|
||||
from .email import EmailConfig, _SENT, record_outbound
|
||||
from .email_envelope import build_envelope
|
||||
from .email import EmailConfig, _SENT
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -44,48 +44,31 @@ def send_otc_email(to_address: str, code: str) -> bool:
|
||||
cfg = EmailConfig.from_env()
|
||||
subject = f"Your sign-in code for {cfg.from_name}"
|
||||
body = _body(code, cfg)
|
||||
# v0.18.0: OTC mail carries NO List-Unsubscribe — the recipient
|
||||
# explicitly requested the code; advertising an unsubscribe
|
||||
# header would imply OHM has them on a list, which it doesn't.
|
||||
# See the proposal's "Tradeoff discussion" for the binding
|
||||
# rationale.
|
||||
msg = build_envelope(
|
||||
to_address=to_address,
|
||||
from_address=cfg.from_address,
|
||||
from_name=cfg.from_name,
|
||||
subject=subject,
|
||||
body_plain=body,
|
||||
)
|
||||
envelope = {
|
||||
"to": to_address,
|
||||
"from": formataddr((cfg.from_name, cfg.from_address)),
|
||||
"subject": subject,
|
||||
"body": body,
|
||||
"kind": "otc",
|
||||
"message": msg,
|
||||
}
|
||||
_SENT.append(envelope)
|
||||
|
||||
message_id = msg["Message-ID"]
|
||||
if not cfg.enabled:
|
||||
log.info("otc email disabled (EMAIL_ENABLED=0): to=%s", to_address)
|
||||
record_outbound(
|
||||
to_address=to_address, from_address=cfg.from_address,
|
||||
subject=subject, kind="otc", status="deferred", message_id=message_id,
|
||||
)
|
||||
return True
|
||||
if not cfg.smtp_host:
|
||||
# Dev fallback: surface the code at INFO so the operator can
|
||||
# complete a sign-in flow without an SMTP relay. In production
|
||||
# SMTP_HOST is always set per OHM's overlay.
|
||||
log.info("otc email (stdout fallback): to=%s code=%s", to_address, code)
|
||||
record_outbound(
|
||||
to_address=to_address, from_address=cfg.from_address,
|
||||
subject=subject, kind="otc", status="deferred", message_id=message_id,
|
||||
)
|
||||
return True
|
||||
|
||||
try:
|
||||
msg = EmailMessage()
|
||||
msg["From"] = envelope["from"]
|
||||
msg["To"] = to_address
|
||||
msg["Subject"] = subject
|
||||
msg.set_content(body)
|
||||
smtp = smtplib.SMTP(cfg.smtp_host, cfg.smtp_port, timeout=30)
|
||||
try:
|
||||
if cfg.smtp_starttls:
|
||||
@@ -95,18 +78,9 @@ def send_otc_email(to_address: str, code: str) -> bool:
|
||||
smtp.send_message(msg)
|
||||
finally:
|
||||
smtp.quit()
|
||||
record_outbound(
|
||||
to_address=to_address, from_address=cfg.from_address,
|
||||
subject=subject, kind="otc", status="sent", message_id=message_id,
|
||||
)
|
||||
return True
|
||||
except Exception as exc:
|
||||
except Exception:
|
||||
log.exception("otc email send failed: to=%s", to_address)
|
||||
record_outbound(
|
||||
to_address=to_address, from_address=cfg.from_address,
|
||||
subject=subject, kind="otc", status="failed",
|
||||
error=f"{type(exc).__name__}: {exc}", message_id=message_id,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
|
||||
+1
-33
@@ -12,7 +12,6 @@ import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
|
||||
from fastapi import APIRouter, Header, HTTPException, Request
|
||||
|
||||
@@ -41,27 +40,7 @@ def make_router(config: Config, gitea: Gitea) -> APIRouter:
|
||||
x_gitea_signature: str = Header(default=""),
|
||||
):
|
||||
body = await request.body()
|
||||
# v0.18.0: defense in depth. config.py refuses to start
|
||||
# when the secret is empty unless `RFC_APP_INSECURE_WEBHOOKS=1`
|
||||
# is set; this branch catches the dev-bypass case (the only
|
||||
# path where `config.webhook_secret` can be empty) and surfaces
|
||||
# it loudly to the client. A POST that lands here with an
|
||||
# empty secret on a production deployment indicates a
|
||||
# mis-configuration (somebody flipped the bypass in prod),
|
||||
# and the loud 500 is the proposal's whole point.
|
||||
insecure = os.environ.get("RFC_APP_INSECURE_WEBHOOKS", "").strip() == "1"
|
||||
if not config.webhook_secret:
|
||||
if not insecure:
|
||||
log.error(
|
||||
"webhook receiver misconfigured: GITEA_WEBHOOK_SECRET is empty "
|
||||
"and RFC_APP_INSECURE_WEBHOOKS=1 is not set"
|
||||
)
|
||||
raise HTTPException(status_code=500, detail="Webhook receiver misconfigured")
|
||||
log.warning(
|
||||
"webhook receiver running with RFC_APP_INSECURE_WEBHOOKS=1 — "
|
||||
"signature verification is DISABLED. Production deployments MUST NOT set this."
|
||||
)
|
||||
else:
|
||||
if config.webhook_secret:
|
||||
if not _verify_signature(body, x_gitea_signature, config.webhook_secret):
|
||||
raise HTTPException(status_code=401, detail="Invalid signature")
|
||||
|
||||
@@ -89,17 +68,6 @@ def make_router(config: Config, gitea: Gitea) -> APIRouter:
|
||||
slug = _slug_for_repo(repo_full)
|
||||
if slug:
|
||||
await cache.refresh_rfc_repo(config, gitea, slug)
|
||||
else:
|
||||
# v0.18.0: the proposal's "unknown-repo logging"
|
||||
# gesture — a hook on a fork or a stale repo binding
|
||||
# used to silently 200-OK here, hiding the
|
||||
# misconfiguration. Now the operator sees it in
|
||||
# the log.
|
||||
log.info(
|
||||
"webhook received for unknown repo: repo_full=%s event=%s "
|
||||
"(no cached_rfcs row matched; hook may be on a fork or stale)",
|
||||
repo_full, event,
|
||||
)
|
||||
except Exception:
|
||||
log.exception("webhook refresh failed")
|
||||
raise HTTPException(status_code=500, detail="Refresh failed")
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
-- v0.18.0 Slice 4: outbound_emails audit table.
|
||||
--
|
||||
-- Per the v0.18.0 email + webhook hygiene proposal §3, every send
|
||||
-- helper writes a row to this table before returning, regardless
|
||||
-- of outcome. status='sent' on success, 'failed' on exception,
|
||||
-- 'deferred' on the dev-fallback path (no SMTP_HOST configured).
|
||||
--
|
||||
-- The table is queried by `GET /api/admin/outbound-emails` to
|
||||
-- answer "did this person ever get their invite?" without having
|
||||
-- to grep VM logs, and by the v0.18.0 Slice 5 bounce-correlation
|
||||
-- hook (which looks up message_id when a POST lands at
|
||||
-- /api/webhooks/email-bounce and marks the matching row
|
||||
-- status='bounced').
|
||||
|
||||
CREATE TABLE IF NOT EXISTS outbound_emails (
|
||||
id INTEGER PRIMARY KEY,
|
||||
to_address TEXT NOT NULL,
|
||||
from_address TEXT NOT NULL,
|
||||
subject TEXT NOT NULL,
|
||||
kind TEXT NOT NULL, -- 'otc' | 'invite' | 'notification' | 'bundle' | 'digest' | 'rfc-invite'
|
||||
sent_at TEXT NOT NULL, -- ISO 8601, time the send was attempted
|
||||
status TEXT NOT NULL, -- 'sent' | 'failed' | 'deferred' | 'bounced'
|
||||
error TEXT, -- exception class + message if status='failed'
|
||||
notification_id INTEGER, -- nullable FK to notifications.id for the watcher path
|
||||
message_id TEXT -- the Message-ID header value, for bounce correlation
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_outbound_emails_to ON outbound_emails(to_address);
|
||||
CREATE INDEX IF NOT EXISTS idx_outbound_emails_sent_at ON outbound_emails(sent_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_outbound_emails_message ON outbound_emails(message_id);
|
||||
@@ -647,82 +647,3 @@ def test_pending_invites_listing_admin_only(app_with_fake_gitea):
|
||||
)
|
||||
r = client.get("/api/admin/users/invites")
|
||||
assert r.status_code == 403
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# v0.18.0: invite-envelope header shape — Slice 2
|
||||
#
|
||||
# Invite mail goes through `build_envelope` and MUST land Date,
|
||||
# Message-ID, Auto-Submitted, AND a `List-Unsubscribe: <mailto:…>`
|
||||
# (no URL — the invitee isn't a user yet, so no per-user opt-out
|
||||
# row exists). The mailto: target is the operator's `EMAIL_FROM`
|
||||
# by default; the operator can override via `EMAIL_UNSUBSCRIBE_MAILTO`.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _provision_admin_and_send_invite(client, app_with_fake_gitea_fixture, *, to: str = "headers@ex.co"):
|
||||
provision_user_row(user_id=400, login="adminH", role="admin")
|
||||
sign_in_as(
|
||||
client, user_id=400, gitea_login="adminH",
|
||||
display_name="Admin H", role="admin",
|
||||
email="adminh@test",
|
||||
)
|
||||
_reset_outbound()
|
||||
r = client.post(
|
||||
"/api/admin/users",
|
||||
json={
|
||||
"email": to,
|
||||
"first_name": "Header",
|
||||
"last_name": "Test",
|
||||
"role": "contributor",
|
||||
"custom_message": "",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
return _outbound_invite_envelopes(to)[-1]
|
||||
|
||||
|
||||
def test_invite_envelope_sets_date_messageid_autosubmitted(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
from email.utils import parsedate_to_datetime
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
env = _provision_admin_and_send_invite(client, (app, _fake))
|
||||
msg = env["message"]
|
||||
assert parsedate_to_datetime(msg["Date"]) is not None
|
||||
assert msg["Message-ID"].startswith("<") and msg["Message-ID"].endswith(">")
|
||||
assert msg["Auto-Submitted"] == "auto-generated"
|
||||
|
||||
|
||||
def test_invite_envelope_has_mailto_list_unsubscribe_only(app_with_fake_gitea):
|
||||
"""The invitee isn't a user yet — no per-user opt-out URL is
|
||||
available. The `List-Unsubscribe` MUST be a mailto: form, and
|
||||
the `List-Unsubscribe-Post` header MUST be absent (the
|
||||
one-click semantic requires a URL the MUA can POST to)."""
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
env = _provision_admin_and_send_invite(client, (app, _fake))
|
||||
msg = env["message"]
|
||||
lu = msg["List-Unsubscribe"]
|
||||
assert lu is not None and lu.startswith("<mailto:")
|
||||
# No URL part — invite is mailto-only.
|
||||
assert "https://" not in lu and "http://" not in lu
|
||||
assert msg["List-Unsubscribe-Post"] is None
|
||||
|
||||
|
||||
def test_invite_envelope_respects_email_unsubscribe_mailto_override(app_with_fake_gitea, monkeypatch):
|
||||
"""When `EMAIL_UNSUBSCRIBE_MAILTO` is set, the mailto: target on
|
||||
`List-Unsubscribe` honors it (lets a deployment route opt-outs
|
||||
to a humans-monitored mailbox distinct from the no-reply
|
||||
sender)."""
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
monkeypatch.setenv("EMAIL_UNSUBSCRIBE_MAILTO", "ohm@wiggleverse.org?subject=remove")
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
env = _provision_admin_and_send_invite(client, (app, _fake))
|
||||
msg = env["message"]
|
||||
assert "ohm@wiggleverse.org?subject=remove" in msg["List-Unsubscribe"]
|
||||
|
||||
@@ -231,17 +231,13 @@ def test_bounce_webhook_refuses_unsigned_when_secret_configured(app_with_fake_gi
|
||||
|
||||
# With the right header, the call passes the guard. (No matching
|
||||
# user exists, so we get {matched: False} — that's the v1 contract.)
|
||||
# v0.18.0 Slice 5: the response now includes `correlated_id`
|
||||
# (the outbound_emails row id that matched the bounce's
|
||||
# `message_id`, if one was supplied). The body didn't pass a
|
||||
# message_id, so correlated_id is None.
|
||||
r = client.post(
|
||||
"/api/webhooks/email-bounce",
|
||||
json={"email": "stranger@example.com", "kind": "hard"},
|
||||
headers={"X-Webhook-Secret": "shhh"},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json() == {"ok": True, "matched": False, "correlated_id": None}
|
||||
assert r.json() == {"ok": True, "matched": False}
|
||||
|
||||
|
||||
def test_bounce_webhook_open_when_secret_unset(app_with_fake_gitea):
|
||||
|
||||
@@ -1,173 +0,0 @@
|
||||
"""Unit tests for `app.email_envelope.build_envelope` (v0.18.0 Slice 1).
|
||||
|
||||
These tests don't spin up the FastAPI app or touch the DB — they
|
||||
exercise the helper directly. The integration tests in
|
||||
test_otc_vertical / test_admin_create_user_invite_vertical /
|
||||
test_notifications_vertical exercise the helper's *use* via the
|
||||
shared `_SENT` buffer (the send path appends the envelope dict
|
||||
before invoking the helper).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from email.utils import parsedate_to_datetime
|
||||
|
||||
from app.email_envelope import build_envelope
|
||||
|
||||
|
||||
def _base_kwargs(**overrides):
|
||||
base = dict(
|
||||
to_address="recipient@example.com",
|
||||
from_address="notifications@ohm.wiggleverse.org",
|
||||
from_name="OHM",
|
||||
subject="A test subject",
|
||||
body_plain="Hello, world.\n",
|
||||
)
|
||||
base.update(overrides)
|
||||
return base
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Always-present headers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_envelope_sets_from_to_subject():
|
||||
msg = build_envelope(**_base_kwargs())
|
||||
assert msg["To"] == "recipient@example.com"
|
||||
assert msg["Subject"] == "A test subject"
|
||||
# `From` is the display-form: "OHM <notifications@ohm.wiggleverse.org>".
|
||||
assert "OHM" in msg["From"]
|
||||
assert "<notifications@ohm.wiggleverse.org>" in msg["From"]
|
||||
|
||||
|
||||
def test_envelope_sets_date_header_parseable():
|
||||
msg = build_envelope(**_base_kwargs())
|
||||
raw = msg["Date"]
|
||||
assert raw, "Date header must be set"
|
||||
# parsedate_to_datetime raises ValueError on malformed input.
|
||||
dt = parsedate_to_datetime(raw)
|
||||
assert dt is not None
|
||||
|
||||
|
||||
def test_envelope_sets_message_id_with_from_domain_by_default():
|
||||
msg = build_envelope(**_base_kwargs())
|
||||
mid = msg["Message-ID"]
|
||||
assert mid, "Message-ID must be set"
|
||||
# Shape per RFC 5322 / make_msgid: <random@domain>
|
||||
assert mid.startswith("<") and mid.endswith(">")
|
||||
assert "@ohm.wiggleverse.org>" in mid
|
||||
|
||||
|
||||
def test_envelope_message_id_domain_override():
|
||||
msg = build_envelope(**_base_kwargs(msgid_domain="example.test"))
|
||||
assert "@example.test>" in msg["Message-ID"]
|
||||
|
||||
|
||||
def test_envelope_message_id_falls_back_to_localhost_if_from_has_no_at():
|
||||
# Defensive: a malformed from_address shouldn't crash the helper.
|
||||
msg = build_envelope(**_base_kwargs(from_address="bare-no-at-sign"))
|
||||
assert "@localhost>" in msg["Message-ID"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auto-Submitted (RFC 3834)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_envelope_sets_auto_submitted_for_transactional_default():
|
||||
msg = build_envelope(**_base_kwargs())
|
||||
assert msg["Auto-Submitted"] == "auto-generated"
|
||||
|
||||
|
||||
def test_envelope_omits_auto_submitted_when_transactional_is_false():
|
||||
msg = build_envelope(**_base_kwargs(is_transactional=False))
|
||||
assert msg["Auto-Submitted"] is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Reply-To
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_envelope_sets_reply_to_when_provided():
|
||||
msg = build_envelope(**_base_kwargs(reply_to="ohm@wiggleverse.org"))
|
||||
assert msg["Reply-To"] == "ohm@wiggleverse.org"
|
||||
|
||||
|
||||
def test_envelope_omits_reply_to_when_absent():
|
||||
msg = build_envelope(**_base_kwargs())
|
||||
assert msg["Reply-To"] is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# List-Unsubscribe (the headers RFC 8058 / Gmail-Yahoo care about)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_envelope_no_list_unsubscribe_when_neither_given():
|
||||
"""OTC mail: the recipient explicitly requested the code; no
|
||||
unsubscribe semantics. The header MUST be absent (presence would
|
||||
imply OHM has the recipient on a list, which it doesn't)."""
|
||||
msg = build_envelope(**_base_kwargs())
|
||||
assert msg["List-Unsubscribe"] is None
|
||||
assert msg["List-Unsubscribe-Post"] is None
|
||||
|
||||
|
||||
def test_envelope_mailto_only_list_unsubscribe():
|
||||
"""Admin invite / per-RFC invite: `mailto:` form only, no URL.
|
||||
The recipient isn't a user yet, so there's no per-user opt-out
|
||||
URL to flip; the operator handles ad-hoc opt-outs manually."""
|
||||
msg = build_envelope(**_base_kwargs(
|
||||
unsubscribe_mailto="ohm@wiggleverse.org?subject=remove",
|
||||
))
|
||||
assert msg["List-Unsubscribe"] == "<mailto:ohm@wiggleverse.org?subject=remove>"
|
||||
# NO List-Unsubscribe-Post when only a mailto is present — the
|
||||
# one-click semantic requires a URL the MUA can POST to.
|
||||
assert msg["List-Unsubscribe-Post"] is None
|
||||
|
||||
|
||||
def test_envelope_full_one_click_list_unsubscribe():
|
||||
"""Watcher notification / bundle: `mailto:` + signed-URL +
|
||||
`List-Unsubscribe-Post: List-Unsubscribe=One-Click`. Gmail and
|
||||
Yahoo enforce this for bulk-adjacent mail per RFC 8058."""
|
||||
msg = build_envelope(**_base_kwargs(
|
||||
unsubscribe_mailto="ohm@wiggleverse.org?subject=remove",
|
||||
unsubscribe_url="https://ohm.wiggleverse.org/api/email/unsubscribe?t=abc",
|
||||
))
|
||||
lu = msg["List-Unsubscribe"]
|
||||
assert "<mailto:ohm@wiggleverse.org?subject=remove>" in lu
|
||||
assert "<https://ohm.wiggleverse.org/api/email/unsubscribe?t=abc>" in lu
|
||||
assert msg["List-Unsubscribe-Post"] == "List-Unsubscribe=One-Click"
|
||||
|
||||
|
||||
def test_envelope_url_only_list_unsubscribe_still_sets_post():
|
||||
msg = build_envelope(**_base_kwargs(
|
||||
unsubscribe_url="https://ohm.wiggleverse.org/api/email/unsubscribe?t=abc",
|
||||
))
|
||||
assert msg["List-Unsubscribe"] == "<https://ohm.wiggleverse.org/api/email/unsubscribe?t=abc>"
|
||||
assert msg["List-Unsubscribe-Post"] == "List-Unsubscribe=One-Click"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Body shape — plain-only vs multipart/alternative
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_envelope_plain_only_body_is_text_plain():
|
||||
msg = build_envelope(**_base_kwargs())
|
||||
# No HTML alternative -> single-part text/plain.
|
||||
assert msg.get_content_type() == "text/plain"
|
||||
assert msg.get_content().strip() == "Hello, world."
|
||||
|
||||
|
||||
def test_envelope_with_html_is_multipart_alternative():
|
||||
msg = build_envelope(**_base_kwargs(body_html="<p>Hello, <b>world</b>.</p>"))
|
||||
assert msg.get_content_type() == "multipart/alternative"
|
||||
# Two parts: text/plain first (so plain-text clients picking the
|
||||
# first part get the readable text), text/html second.
|
||||
parts = list(msg.iter_parts())
|
||||
assert len(parts) == 2
|
||||
assert parts[0].get_content_type() == "text/plain"
|
||||
assert parts[1].get_content_type() == "text/html"
|
||||
assert "Hello, world." in parts[0].get_content()
|
||||
assert "<b>world</b>" in parts[1].get_content()
|
||||
@@ -612,127 +612,3 @@ def test_explicit_watch_set_overrides_auto(app_with_fake_gitea):
|
||||
# the user put them.
|
||||
assert row["set_by"] == "explicit"
|
||||
assert row["state"] == "following"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# v0.18.0 — envelope headers + RFC 8058 one-click POST endpoint
|
||||
#
|
||||
# Watcher notifications are bulk-adjacent (a busy RFC can produce
|
||||
# dozens of structural events); per the proposal, they MUST carry
|
||||
# `Date`, `Message-ID`, `Auto-Submitted`, full `List-Unsubscribe`
|
||||
# (mailto + signed URL), AND `List-Unsubscribe-Post:
|
||||
# List-Unsubscribe=One-Click` per RFC 8058. Gmail and Yahoo
|
||||
# enforce this for senders at OHM's volume tier.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_notification_envelope_carries_full_one_click_headers(app_with_fake_gitea):
|
||||
"""A `proposal_merged` event lands a watcher notification email
|
||||
with the full one-click unsubscribe shape."""
|
||||
from fastapi.testclient import TestClient
|
||||
from email.utils import parsedate_to_datetime
|
||||
from app import db, email as email_mod
|
||||
|
||||
app, fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=2, login="alice", role="contributor")
|
||||
provision_user_row(user_id=1, login="ben", role="owner")
|
||||
|
||||
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test")
|
||||
r = client.post("/api/rfcs/propose", json={"title": "OHM", "slug": "ohm", "pitch": PITCH, "tags": []})
|
||||
assert r.status_code == 200
|
||||
email_mod.reset_sent_envelopes()
|
||||
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner", email="ben@test")
|
||||
merge_r = client.post(f"/api/proposals/{r.json()['pr_number']}/merge")
|
||||
assert merge_r.status_code == 200, merge_r.text
|
||||
|
||||
envelopes = [e for e in email_mod.sent_envelopes() if e["to"] == "alice@test"]
|
||||
assert envelopes, "watcher notification did not fire"
|
||||
msg = envelopes[-1]["message"]
|
||||
# Always-present headers from the helper.
|
||||
assert parsedate_to_datetime(msg["Date"]) is not None
|
||||
assert msg["Message-ID"].startswith("<") and msg["Message-ID"].endswith(">")
|
||||
assert msg["Auto-Submitted"] == "auto-generated"
|
||||
# Full one-click unsubscribe.
|
||||
lu = msg["List-Unsubscribe"]
|
||||
assert lu is not None
|
||||
assert "<mailto:" in lu
|
||||
# URL part carries the signed token per make_unsubscribe_url.
|
||||
assert "/api/email/unsubscribe?t=" in lu
|
||||
assert msg["List-Unsubscribe-Post"] == "List-Unsubscribe=One-Click"
|
||||
|
||||
|
||||
def test_email_unsubscribe_post_one_click_flips_category_off(app_with_fake_gitea):
|
||||
"""RFC 8058: Gmail/Yahoo POST `List-Unsubscribe=One-Click` to the
|
||||
URL in the List-Unsubscribe header. The endpoint MUST accept POST
|
||||
+ the same token shape as the GET handler + return 200 + flip the
|
||||
flag."""
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db, email as email_mod
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=2, login="alice", role="contributor")
|
||||
token = email_mod.make_unsubscribe_url(2, "personal-direct").split("t=", 1)[1]
|
||||
|
||||
r = client.post(
|
||||
f"/api/email/unsubscribe?t={token}",
|
||||
data={"List-Unsubscribe": "One-Click"},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
assert body["ok"] is True
|
||||
assert body["category"] == "personal-direct"
|
||||
|
||||
row = db.conn().execute(
|
||||
"SELECT email_personal_direct FROM users WHERE id = 2"
|
||||
).fetchone()
|
||||
assert row["email_personal_direct"] == 0
|
||||
|
||||
|
||||
def test_email_unsubscribe_post_all_sets_global_opt_out(app_with_fake_gitea):
|
||||
"""The v0.18.0 `all` synthetic category (used by the bundle +
|
||||
digest paths) MUST set `email_opt_out_all = 1`."""
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db, email as email_mod
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=2, login="alice", role="contributor")
|
||||
token = email_mod.make_unsubscribe_url(2, "all").split("t=", 1)[1]
|
||||
r = client.post(f"/api/email/unsubscribe?t={token}")
|
||||
assert r.status_code == 200
|
||||
assert r.json() == {"ok": True, "category": "all"}
|
||||
row = db.conn().execute(
|
||||
"SELECT email_opt_out_all FROM users WHERE id = 2"
|
||||
).fetchone()
|
||||
assert row["email_opt_out_all"] == 1
|
||||
|
||||
|
||||
def test_email_unsubscribe_get_all_sets_global_opt_out(app_with_fake_gitea):
|
||||
"""GET handler also accepts the `all` category and lands the
|
||||
global opt-out (so an MUA that doesn't honor RFC 8058 POST and
|
||||
just opens the URL in a browser still works)."""
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db, email as email_mod
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=2, login="alice", role="contributor")
|
||||
token = email_mod.make_unsubscribe_url(2, "all").split("t=", 1)[1]
|
||||
r = client.get(f"/api/email/unsubscribe?t={token}")
|
||||
assert r.status_code == 200
|
||||
assert "Unsubscribed" in r.text
|
||||
row = db.conn().execute(
|
||||
"SELECT email_opt_out_all FROM users WHERE id = 2"
|
||||
).fetchone()
|
||||
assert row["email_opt_out_all"] == 1
|
||||
|
||||
|
||||
def test_email_unsubscribe_post_refuses_invalid_token(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
r = client.post("/api/email/unsubscribe?t=not-a-valid-token")
|
||||
assert r.status_code == 400
|
||||
|
||||
@@ -347,53 +347,3 @@ def test_otc_re_request_invalidates_prior_unused_code(app_with_fake_gitea, monke
|
||||
# The new code still works.
|
||||
r = client.post("/auth/otc/verify", json={"email": "alice@example.com", "code": second})
|
||||
assert r.status_code == 200
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# v0.18.0: envelope headers — Slice 2
|
||||
#
|
||||
# OTC mail goes through `build_envelope` and MUST land Date,
|
||||
# Message-ID, and Auto-Submitted but MUST NOT carry a
|
||||
# List-Unsubscribe header (the recipient explicitly requested the
|
||||
# code; advertising a list semantic would be wrong).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _last_otc_envelope():
|
||||
from app import email as email_mod
|
||||
otc = [e for e in email_mod.sent_envelopes() if e.get("kind") == "otc"]
|
||||
assert otc, "no OTC envelope in the buffer"
|
||||
return otc[-1]
|
||||
|
||||
|
||||
def test_otc_envelope_sets_date_messageid_autosubmitted(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
from email.utils import parsedate_to_datetime
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
_reset_outbound()
|
||||
client.post("/auth/otc/request", json={"email": "headers@example.com"})
|
||||
msg = _last_otc_envelope()["message"]
|
||||
# Date is RFC 5322 parseable.
|
||||
assert parsedate_to_datetime(msg["Date"]) is not None
|
||||
# Message-ID is bracketed and carries the From-address @-domain.
|
||||
mid = msg["Message-ID"]
|
||||
assert mid.startswith("<") and mid.endswith(">")
|
||||
# Auto-Submitted prevents auto-responder loops.
|
||||
assert msg["Auto-Submitted"] == "auto-generated"
|
||||
|
||||
|
||||
def test_otc_envelope_has_no_list_unsubscribe(app_with_fake_gitea):
|
||||
"""The recipient explicitly typed their email and asked for a
|
||||
code; the framework MUST NOT advertise a list semantic on this
|
||||
mail. Per the v0.18.0 proposal's tradeoff discussion."""
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
_reset_outbound()
|
||||
client.post("/auth/otc/request", json={"email": "headers@example.com"})
|
||||
msg = _last_otc_envelope()["message"]
|
||||
assert msg["List-Unsubscribe"] is None
|
||||
assert msg["List-Unsubscribe-Post"] is None
|
||||
|
||||
@@ -1,368 +0,0 @@
|
||||
"""End-to-end integration tests for the v0.18.0 Slice 4
|
||||
outbound_emails audit table + admin endpoint.
|
||||
|
||||
The release adds:
|
||||
* `backend/migrations/020_outbound_emails.sql` — the audit table.
|
||||
* `record_outbound()` in `email.py` — the write helper every send
|
||||
path calls before returning, capturing status='sent' / 'failed'
|
||||
/ 'deferred' (the dev-fallback path when SMTP_HOST is unset).
|
||||
* `GET /api/admin/outbound-emails` — admin-only listing, filterable
|
||||
by kind / status / to_address.
|
||||
|
||||
These tests prove:
|
||||
* Sending OTC / invite / notification mail writes one row per send
|
||||
(status='deferred' under tests since SMTP_HOST is unset).
|
||||
* The Message-ID on the row matches the envelope's Message-ID
|
||||
header (the seam Slice 5 uses for bounce correlation).
|
||||
* `kind` is populated per send path.
|
||||
* `GET /api/admin/outbound-emails` lists rows newest-first,
|
||||
accepts filters, refuses non-admins.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from test_propose_vertical import ( # noqa: F401
|
||||
FakeGitea,
|
||||
app_with_fake_gitea,
|
||||
provision_user_row,
|
||||
sign_in_as,
|
||||
tmp_env,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Write-on-send wiring
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_otc_send_writes_outbound_row_with_message_id(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db, email as email_mod
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
email_mod.reset_sent_envelopes()
|
||||
r = client.post("/auth/otc/request", json={"email": "newcomer@ex.co"})
|
||||
assert r.status_code == 200
|
||||
|
||||
# Audit row landed.
|
||||
rows = db.conn().execute(
|
||||
"SELECT id, to_address, kind, status, message_id, error "
|
||||
"FROM outbound_emails WHERE to_address = 'newcomer@ex.co'"
|
||||
).fetchall()
|
||||
assert len(rows) == 1
|
||||
row = rows[0]
|
||||
assert row["kind"] == "otc"
|
||||
# No SMTP_HOST in tests -> 'deferred', not 'sent'.
|
||||
assert row["status"] == "deferred"
|
||||
assert row["error"] is None
|
||||
# Message-ID matches the envelope's header (the seam Slice 5 uses).
|
||||
envelopes = [e for e in email_mod.sent_envelopes() if e.get("kind") == "otc"]
|
||||
assert envelopes
|
||||
envelope_mid = envelopes[-1]["message"]["Message-ID"]
|
||||
assert row["message_id"] == envelope_mid
|
||||
|
||||
|
||||
def test_invite_send_writes_outbound_row(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=500, login="adminQ", role="admin")
|
||||
sign_in_as(
|
||||
client, user_id=500, gitea_login="adminQ",
|
||||
display_name="Admin Q", role="admin",
|
||||
email="adminq@test",
|
||||
)
|
||||
r = client.post(
|
||||
"/api/admin/users",
|
||||
json={
|
||||
"email": "invitee@ex.co",
|
||||
"first_name": "Inv", "last_name": "Itee",
|
||||
"role": "contributor", "custom_message": "",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
rows = db.conn().execute(
|
||||
"SELECT kind, status, message_id FROM outbound_emails "
|
||||
"WHERE to_address = 'invitee@ex.co'"
|
||||
).fetchall()
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["kind"] == "invite"
|
||||
assert rows[0]["status"] == "deferred"
|
||||
assert rows[0]["message_id"] is not None
|
||||
|
||||
|
||||
def test_notification_send_writes_outbound_row_with_notification_id(app_with_fake_gitea):
|
||||
"""Watcher notifications carry a `notification_id` FK so the
|
||||
admin can join through to the notifications table to see what
|
||||
triggered the send."""
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db, email as email_mod
|
||||
from test_notifications_vertical import PITCH
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=2, login="alice", role="contributor")
|
||||
provision_user_row(user_id=1, login="ben", role="owner")
|
||||
sign_in_as(
|
||||
client, user_id=2, gitea_login="alice",
|
||||
display_name="Alice", role="contributor", email="alice@test",
|
||||
)
|
||||
r = client.post("/api/rfcs/propose", json={
|
||||
"title": "OHM", "slug": "ohm", "pitch": PITCH, "tags": [],
|
||||
})
|
||||
email_mod.reset_sent_envelopes()
|
||||
# Wipe pre-merge audit rows so the assertion below is unambiguous.
|
||||
db.conn().execute("DELETE FROM outbound_emails")
|
||||
sign_in_as(
|
||||
client, user_id=1, gitea_login="ben",
|
||||
display_name="Ben", role="owner", email="ben@test",
|
||||
)
|
||||
merge_r = client.post(f"/api/proposals/{r.json()['pr_number']}/merge")
|
||||
assert merge_r.status_code == 200, merge_r.text
|
||||
|
||||
rows = db.conn().execute(
|
||||
"SELECT kind, status, notification_id, message_id "
|
||||
"FROM outbound_emails WHERE to_address = 'alice@test'"
|
||||
).fetchall()
|
||||
assert rows, "no outbound_emails row for alice@test"
|
||||
# At least one notification kind, with a populated FK.
|
||||
notif_rows = [r for r in rows if r["kind"] == "notification"]
|
||||
assert notif_rows
|
||||
for nr in notif_rows:
|
||||
assert nr["status"] == "deferred"
|
||||
assert nr["notification_id"] is not None
|
||||
assert nr["message_id"] is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Admin endpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_admin_outbound_emails_lists_rows(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
# Generate a few rows.
|
||||
client.post("/auth/otc/request", json={"email": "one@ex.co"})
|
||||
|
||||
provision_user_row(user_id=600, login="adminR", role="admin")
|
||||
sign_in_as(
|
||||
client, user_id=600, gitea_login="adminR",
|
||||
display_name="Admin R", role="admin", email="adminr@test",
|
||||
)
|
||||
client.post("/api/admin/users", json={
|
||||
"email": "two@ex.co", "first_name": "T", "last_name": "Wo",
|
||||
"role": "contributor", "custom_message": "",
|
||||
})
|
||||
|
||||
r = client.get("/api/admin/outbound-emails")
|
||||
assert r.status_code == 200, r.text
|
||||
items = r.json()["items"]
|
||||
kinds = {it["kind"] for it in items}
|
||||
assert "otc" in kinds
|
||||
assert "invite" in kinds
|
||||
# Newest-first.
|
||||
ids = [it["id"] for it in items]
|
||||
assert ids == sorted(ids, reverse=True)
|
||||
|
||||
|
||||
def test_admin_outbound_emails_filters_by_kind(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
client.post("/auth/otc/request", json={"email": "filter1@ex.co"})
|
||||
|
||||
provision_user_row(user_id=601, login="adminS", role="admin")
|
||||
sign_in_as(
|
||||
client, user_id=601, gitea_login="adminS",
|
||||
display_name="Admin S", role="admin", email="admins@test",
|
||||
)
|
||||
client.post("/api/admin/users", json={
|
||||
"email": "filter2@ex.co", "first_name": "F", "last_name": "Two",
|
||||
"role": "contributor", "custom_message": "",
|
||||
})
|
||||
|
||||
r = client.get("/api/admin/outbound-emails?kind=otc")
|
||||
assert r.status_code == 200
|
||||
items = r.json()["items"]
|
||||
assert items
|
||||
assert all(it["kind"] == "otc" for it in items)
|
||||
|
||||
|
||||
def test_admin_outbound_emails_filters_by_to_address(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
client.post("/auth/otc/request", json={"email": "TARGET@ex.co"})
|
||||
client.post("/auth/otc/request", json={"email": "other@ex.co"})
|
||||
|
||||
provision_user_row(user_id=602, login="adminT", role="admin")
|
||||
sign_in_as(
|
||||
client, user_id=602, gitea_login="adminT",
|
||||
display_name="Admin T", role="admin", email="admint@test",
|
||||
)
|
||||
|
||||
# to_address filter is case-insensitive.
|
||||
r = client.get("/api/admin/outbound-emails?to_address=target@ex.co")
|
||||
assert r.status_code == 200
|
||||
items = r.json()["items"]
|
||||
assert items
|
||||
assert all(it["to_address"].lower() == "target@ex.co" for it in items)
|
||||
|
||||
|
||||
def test_admin_outbound_emails_refuses_non_admin(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=700, login="contribU", role="contributor")
|
||||
sign_in_as(
|
||||
client, user_id=700, gitea_login="contribU",
|
||||
display_name="Contrib U", role="contributor",
|
||||
)
|
||||
r = client.get("/api/admin/outbound-emails")
|
||||
assert r.status_code == 403
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# v0.18.0 Slice 5: bounce correlation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_bounce_with_message_id_marks_outbound_row_bounced(app_with_fake_gitea):
|
||||
"""When the bounce body includes the original `message_id`, the
|
||||
framework looks it up in outbound_emails and stamps
|
||||
status='bounced' on the matching row. The hard-bounce ->
|
||||
global-opt-out logic still fires."""
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db, email as email_mod
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=800, login="bouncey", role="contributor")
|
||||
db.conn().execute("UPDATE users SET email = 'bouncey@ex.co' WHERE id = 800")
|
||||
|
||||
# Send something to bouncey to land an outbound_emails row.
|
||||
email_mod.reset_sent_envelopes()
|
||||
client.post("/auth/otc/request", json={"email": "bouncey@ex.co"})
|
||||
row = db.conn().execute(
|
||||
"SELECT id, message_id, status FROM outbound_emails "
|
||||
"WHERE to_address = 'bouncey@ex.co'"
|
||||
).fetchone()
|
||||
assert row is not None
|
||||
original_id = row["id"]
|
||||
message_id = row["message_id"]
|
||||
assert row["status"] == "deferred" # pre-bounce baseline
|
||||
|
||||
# Bounce comes in carrying that message_id.
|
||||
r = client.post(
|
||||
"/api/webhooks/email-bounce",
|
||||
json={
|
||||
"email": "bouncey@ex.co",
|
||||
"kind": "hard",
|
||||
"message_id": message_id,
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["matched"] is True
|
||||
assert body["correlated_id"] == original_id
|
||||
|
||||
# Audit row stamped.
|
||||
post = db.conn().execute(
|
||||
"SELECT status, error FROM outbound_emails WHERE id = ?",
|
||||
(original_id,),
|
||||
).fetchone()
|
||||
assert post["status"] == "bounced"
|
||||
assert "bounce (hard)" in (post["error"] or "")
|
||||
|
||||
# Hard-bounce global opt-out still fires.
|
||||
urow = db.conn().execute(
|
||||
"SELECT email_opt_out_all FROM users WHERE id = 800"
|
||||
).fetchone()
|
||||
assert urow["email_opt_out_all"] == 1
|
||||
|
||||
|
||||
def test_bounce_with_unknown_message_id_does_not_crash(app_with_fake_gitea):
|
||||
"""A message_id the framework doesn't recognize logs but does
|
||||
NOT 5xx — bounce providers replay old bounces, and the
|
||||
framework can't refuse just because the row was pruned."""
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
r = client.post(
|
||||
"/api/webhooks/email-bounce",
|
||||
json={
|
||||
"email": "nobody@ex.co",
|
||||
"kind": "hard",
|
||||
"message_id": "<not-in-our-db@ex.co>",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
assert r.json()["correlated_id"] is None
|
||||
|
||||
|
||||
def test_bounce_without_message_id_still_flips_opt_out(app_with_fake_gitea):
|
||||
"""Backward compat: providers that don't surface Message-ID
|
||||
still get the legacy v1 behavior — match by email + flip the
|
||||
global opt-out."""
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=801, login="legacybounce", role="contributor")
|
||||
db.conn().execute("UPDATE users SET email = 'legacy@ex.co' WHERE id = 801")
|
||||
|
||||
r = client.post(
|
||||
"/api/webhooks/email-bounce",
|
||||
json={"email": "legacy@ex.co", "kind": "hard"},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["matched"] is True
|
||||
assert body["correlated_id"] is None
|
||||
|
||||
urow = db.conn().execute(
|
||||
"SELECT email_opt_out_all FROM users WHERE id = 801"
|
||||
).fetchone()
|
||||
assert urow["email_opt_out_all"] == 1
|
||||
|
||||
|
||||
def test_bounced_rows_show_in_admin_endpoint(app_with_fake_gitea):
|
||||
"""The admin endpoint surfaces bounced rows alongside the rest;
|
||||
filtering by `status=bounced` isolates them."""
|
||||
from fastapi.testclient import TestClient
|
||||
from app import db, email as email_mod
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
provision_user_row(user_id=802, login="adminB", role="admin")
|
||||
sign_in_as(
|
||||
client, user_id=802, gitea_login="adminB",
|
||||
display_name="Admin B", role="admin", email="adminb@test",
|
||||
)
|
||||
email_mod.reset_sent_envelopes()
|
||||
client.post("/auth/otc/request", json={"email": "willbounce@ex.co"})
|
||||
row = db.conn().execute(
|
||||
"SELECT message_id FROM outbound_emails WHERE to_address = 'willbounce@ex.co'"
|
||||
).fetchone()
|
||||
client.post(
|
||||
"/api/webhooks/email-bounce",
|
||||
json={"email": "willbounce@ex.co", "kind": "hard", "message_id": row["message_id"]},
|
||||
)
|
||||
|
||||
r = client.get("/api/admin/outbound-emails?status=bounced")
|
||||
assert r.status_code == 200
|
||||
items = r.json()["items"]
|
||||
assert items
|
||||
assert all(it["status"] == "bounced" for it in items)
|
||||
assert any(it["to_address"] == "willbounce@ex.co" for it in items)
|
||||
@@ -438,11 +438,7 @@ def tmp_env(monkeypatch):
|
||||
"SECRET_KEY": "test-secret-key-for-cookies",
|
||||
"DATABASE_PATH": str(db_path),
|
||||
"OWNER_GITEA_LOGIN": "ben",
|
||||
# v0.18.0: `GITEA_WEBHOOK_SECRET` is now mandatory at startup
|
||||
# per the email + webhook hygiene proposal. Tests bind a fake
|
||||
# value so the framework boots; tests that want to exercise
|
||||
# the dev-bypass path monkeypatch `RFC_APP_INSECURE_WEBHOOKS=1`.
|
||||
"GITEA_WEBHOOK_SECRET": "test-webhook-secret-for-signature-verification",
|
||||
"GITEA_WEBHOOK_SECRET": "",
|
||||
"ENABLED_MODELS": "claude",
|
||||
}
|
||||
for k, v in env.items():
|
||||
|
||||
@@ -1,205 +0,0 @@
|
||||
"""End-to-end integration tests for the Gitea webhook receiver
|
||||
(v0.18.0 Slice 3 — webhook tightening per the email + webhook
|
||||
hygiene proposal).
|
||||
|
||||
The release changes the receiver from "verifies the signature only
|
||||
when a secret is configured; silently accepts unsigned POSTs
|
||||
otherwise" to "requires the secret unless `RFC_APP_INSECURE_WEBHOOKS=1`
|
||||
is set as an explicit dev-bypass." The startup-time check lives in
|
||||
`config.load_config()`; the request-time check lives in
|
||||
`webhooks.receive`.
|
||||
|
||||
These tests prove:
|
||||
|
||||
* The framework refuses to start when `GITEA_WEBHOOK_SECRET` is
|
||||
empty and the dev-bypass is not set.
|
||||
* The dev-bypass (`RFC_APP_INSECURE_WEBHOOKS=1`) lets the
|
||||
framework boot with an empty secret AND lets webhook POSTs
|
||||
land without signature verification (a loud-warning log line
|
||||
surfaces, but the request is accepted).
|
||||
* Default path (secret bound): a POST with a valid signature
|
||||
lands; a POST with an invalid signature gets 401; a POST with
|
||||
no signature gets 401.
|
||||
* Unknown-repo POSTs surface in the log (the "stale Gitea hook"
|
||||
case the proposal targets).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
|
||||
import pytest
|
||||
|
||||
from test_propose_vertical import ( # noqa: F401
|
||||
FakeGitea,
|
||||
app_with_fake_gitea,
|
||||
tmp_env,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Startup-time secret check (config.load_config)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_config_refuses_to_load_with_empty_secret_and_no_bypass(monkeypatch, tmp_path):
|
||||
"""The framework MUST refuse to start when `GITEA_WEBHOOK_SECRET`
|
||||
is empty unless `RFC_APP_INSECURE_WEBHOOKS=1` is set. This is
|
||||
the v0.18.0 startup-loud-failure shape — silent acceptance was
|
||||
the bug."""
|
||||
monkeypatch.setenv("GITEA_URL", "http://gitea.test")
|
||||
monkeypatch.setenv("GITEA_BOT_USER", "rfc-bot")
|
||||
monkeypatch.setenv("GITEA_BOT_TOKEN", "bot-token")
|
||||
monkeypatch.setenv("GITEA_ORG", "wiggleverse")
|
||||
monkeypatch.setenv("OAUTH_CLIENT_ID", "cid")
|
||||
monkeypatch.setenv("OAUTH_CLIENT_SECRET", "csec")
|
||||
monkeypatch.setenv("SECRET_KEY", "test-secret-key")
|
||||
monkeypatch.setenv("DATABASE_PATH", str(tmp_path / "test.db"))
|
||||
monkeypatch.setenv("GITEA_WEBHOOK_SECRET", "")
|
||||
monkeypatch.delenv("RFC_APP_INSECURE_WEBHOOKS", raising=False)
|
||||
|
||||
from app.config import load_config
|
||||
with pytest.raises(RuntimeError, match="GITEA_WEBHOOK_SECRET"):
|
||||
load_config()
|
||||
|
||||
|
||||
def test_config_loads_with_empty_secret_when_bypass_is_set(monkeypatch, tmp_path):
|
||||
"""The explicit `RFC_APP_INSECURE_WEBHOOKS=1` opt-in lets the
|
||||
framework boot with an empty webhook secret. This is the
|
||||
local-dev escape hatch."""
|
||||
monkeypatch.setenv("GITEA_URL", "http://gitea.test")
|
||||
monkeypatch.setenv("GITEA_BOT_USER", "rfc-bot")
|
||||
monkeypatch.setenv("GITEA_BOT_TOKEN", "bot-token")
|
||||
monkeypatch.setenv("GITEA_ORG", "wiggleverse")
|
||||
monkeypatch.setenv("OAUTH_CLIENT_ID", "cid")
|
||||
monkeypatch.setenv("OAUTH_CLIENT_SECRET", "csec")
|
||||
monkeypatch.setenv("SECRET_KEY", "test-secret-key")
|
||||
monkeypatch.setenv("DATABASE_PATH", str(tmp_path / "test.db"))
|
||||
monkeypatch.setenv("GITEA_WEBHOOK_SECRET", "")
|
||||
monkeypatch.setenv("RFC_APP_INSECURE_WEBHOOKS", "1")
|
||||
|
||||
from app.config import load_config
|
||||
cfg = load_config() # MUST NOT raise
|
||||
assert cfg.webhook_secret == ""
|
||||
|
||||
|
||||
def test_config_loads_with_secret_set(monkeypatch, tmp_path):
|
||||
"""Sanity: the happy path (secret bound, bypass not set) loads
|
||||
cleanly."""
|
||||
monkeypatch.setenv("GITEA_URL", "http://gitea.test")
|
||||
monkeypatch.setenv("GITEA_BOT_USER", "rfc-bot")
|
||||
monkeypatch.setenv("GITEA_BOT_TOKEN", "bot-token")
|
||||
monkeypatch.setenv("GITEA_ORG", "wiggleverse")
|
||||
monkeypatch.setenv("OAUTH_CLIENT_ID", "cid")
|
||||
monkeypatch.setenv("OAUTH_CLIENT_SECRET", "csec")
|
||||
monkeypatch.setenv("SECRET_KEY", "test-secret-key")
|
||||
monkeypatch.setenv("DATABASE_PATH", str(tmp_path / "test.db"))
|
||||
monkeypatch.setenv("GITEA_WEBHOOK_SECRET", "my-real-secret")
|
||||
monkeypatch.delenv("RFC_APP_INSECURE_WEBHOOKS", raising=False)
|
||||
|
||||
from app.config import load_config
|
||||
cfg = load_config()
|
||||
assert cfg.webhook_secret == "my-real-secret"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Request-time signature verification (webhooks.receive)
|
||||
#
|
||||
# The default `app_with_fake_gitea` fixture binds
|
||||
# `GITEA_WEBHOOK_SECRET=test-webhook-secret-for-signature-verification`,
|
||||
# so these tests exercise the production path.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
_SECRET = "test-webhook-secret-for-signature-verification"
|
||||
|
||||
|
||||
def _sign(body: bytes) -> str:
|
||||
return hmac.new(_SECRET.encode("utf-8"), body, hashlib.sha256).hexdigest()
|
||||
|
||||
|
||||
def test_webhook_post_with_valid_signature_accepted(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
body = json.dumps({"repository": {"full_name": "wiggleverse/meta"}}).encode()
|
||||
sig = _sign(body)
|
||||
r = client.post(
|
||||
"/api/webhooks/gitea",
|
||||
content=body,
|
||||
headers={
|
||||
"X-Gitea-Event": "push",
|
||||
"X-Gitea-Signature": sig,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
|
||||
def test_webhook_post_with_invalid_signature_refused_401(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
body = json.dumps({"repository": {"full_name": "wiggleverse/meta"}}).encode()
|
||||
r = client.post(
|
||||
"/api/webhooks/gitea",
|
||||
content=body,
|
||||
headers={
|
||||
"X-Gitea-Event": "push",
|
||||
"X-Gitea-Signature": "0" * 64, # wrong signature
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 401
|
||||
|
||||
|
||||
def test_webhook_post_with_missing_signature_refused_401(app_with_fake_gitea):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
body = json.dumps({"repository": {"full_name": "wiggleverse/meta"}}).encode()
|
||||
r = client.post(
|
||||
"/api/webhooks/gitea",
|
||||
content=body,
|
||||
headers={
|
||||
"X-Gitea-Event": "push",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 401
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Unknown-repo logging (the "stale hook on a fork" surface)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_webhook_unknown_repo_logs_at_info(app_with_fake_gitea, caplog):
|
||||
"""Per the proposal: a hook on a fork or a stale Gitea binding
|
||||
used to silently 200-OK. v0.18.0 surfaces it as an INFO log."""
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
app, _fake = app_with_fake_gitea
|
||||
with TestClient(app) as client:
|
||||
body = json.dumps({"repository": {"full_name": "someone-else/unrelated"}}).encode()
|
||||
sig = _sign(body)
|
||||
with caplog.at_level(logging.INFO, logger="app.webhooks"):
|
||||
r = client.post(
|
||||
"/api/webhooks/gitea",
|
||||
content=body,
|
||||
headers={
|
||||
"X-Gitea-Event": "push",
|
||||
"X-Gitea-Signature": sig,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200 # the handler still 200s; surface is the log line
|
||||
assert any(
|
||||
"unknown repo" in rec.message and "someone-else/unrelated" in rec.message
|
||||
for rec in caplog.records
|
||||
), f"expected unknown-repo log line; got: {[r.message for r in caplog.records]}"
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "rfc-app-frontend",
|
||||
"private": true,
|
||||
"version": "0.18.0",
|
||||
"version": "0.17.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
Reference in New Issue
Block a user