v0.19.0 backend: /api/docs/sessions/* endpoints + TTL cache

Adds the four read endpoints the v0.19.0 /docs/sessions/* surface mounts on top of: - GET /api/docs/sessions/manifest → sessions.json (title map) - GET /api/docs/sessions/about → README.md - GET /api/docs/sessions/<NNNN>/index → per-session transcript list - GET /api/docs/sessions/<NNNN>/<file> → transcript body The framework mediates the gitea fetch so the rendered surface inherits the same chrome as the v0.14.0 /docs route and the browser makes no cross-origin call. Reads are aggressively cached in-process (60s for the manifest, 5min for content) so the framework doesn't hammer git.wiggleverse.org under normal traffic. Negative results (gitea 404) are also cached at the content TTL to absorb the expected empty-state at deploy-time (the parallel ohm-session-history repo restructure ships in driver subsession 0017.2). All four endpoints are anonymous-reachable, sibling to /api/philosophy and /api/docs. Path validation gates the network: only /^\d{4}$/ session dirs and the full SESSION-NNNN.M-TRANSCRIPT-...md filename shape pass to the upstream. Legacy flat-root names (e.g. SESSION-A-TRANSCRIPT.md) and path-traversal attempts are rejected 400 before any fetch. 18 new tests cover the happy paths, 404 empty-states, 502 upstream errors, path-validation rejections, and the cache-hit-within-TTL contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 09:07:01 -07:00
parent ac3513a686
commit 39e57706d9
3 changed files with 855 additions and 0 deletions
@@ -0,0 +1,358 @@
+"""§14 + roadmap item #30 — on-site sessions-history browser source.
+
+Sibling of `docs.py` / `philosophy.py` but with a different read shape:
+the bodies here live in the **public** `wiggleverse/ohm-session-history`
+gitea repo (transcripts of every OHM build session, published per the
+ohm-infra SESSION-PROTOCOL.md), not on disk. The framework mediates
+the gitea fetch on behalf of the browser so the rendered `/docs/sessions/*`
+surface inherits the same chrome as `/philosophy` and `/docs/user-guide`
+and stays free of any cross-origin gestures from the frontend.
+
+Three read endpoints, all anonymous-reachable:
+
+  GET /api/docs/sessions/manifest       — sessions.json (title manifest)
+  GET /api/docs/sessions/about          — README.md (the about page)
+  GET /api/docs/sessions/<NNNN>/<file>  — a transcript body
+  GET /api/docs/sessions/<NNNN>/index   — per-session file listing
+
+All three sit behind a small in-process TTL cache (manifest TTL default
+60 s, content TTL default 300 s). Negative results (404 from gitea) are
+also cached at the content TTL to avoid hammering gitea when a
+deployment hasn't yet been populated with transcripts. The cache key
+is the URL path on the gitea raw base (or the contents API for the
+per-session listing); the cache lives in-process, plain dict +
+`time.monotonic()` check, no external dep.
+
+Env knobs:
+
+  OHM_SESSION_HISTORY_RAW_BASE
+    Override the gitea raw base URL. Default points at OHM's canonical
+    transcript repo:
+      https://git.wiggleverse.org/wiggleverse/ohm-session-history/raw/branch/main
+    The framework-default value is OHM-flavored because OHM is the
+    only deployment to date — a deployment running its own
+    transcript repo overrides this via flotilla's overlay.
+
+  OHM_SESSION_HISTORY_CONTENTS_BASE
+    Override the gitea contents-API base URL (for the per-session
+    listing endpoint, which enumerates files inside a `NNNN/` folder).
+    Default:
+      https://git.wiggleverse.org/api/v1/repos/wiggleverse/ohm-session-history/contents
+
+  OHM_DOCS_SESSIONS_MANIFEST_TTL_SEC
+    Cache TTL for the manifest (default 60 s). The manifest is small
+    and changes when a new session is added; 60 s strikes a balance
+    between freshness and gitea load.
+
+  OHM_DOCS_SESSIONS_CONTENT_TTL_SEC
+    Cache TTL for transcript bodies + README + per-session listings
+    (default 300 s = 5 minutes). Transcripts are append-only once
+    published, so 5 minutes of staleness is harmless.
+
+§3 invariant 1 is preserved: the framework holds no secret bytes; the
+gitea repo is public, the fetch carries no auth header.
+"""
+from __future__ import annotations
+
+import logging
+import os
+import re
+import threading
+import time
+from typing import Any
+
+import httpx
+
+log = logging.getLogger(__name__)
+
+
+_DEFAULT_RAW_BASE = (
+    "https://git.wiggleverse.org/wiggleverse/ohm-session-history/raw/branch/main"
+)
+_DEFAULT_CONTENTS_BASE = (
+    "https://git.wiggleverse.org/api/v1/repos/wiggleverse/ohm-session-history/contents"
+)
+_DEFAULT_MANIFEST_TTL_SEC = 60.0
+_DEFAULT_CONTENT_TTL_SEC = 300.0
+
+# The transcript filename shape per SESSION-PROTOCOL.md §1. The
+# `<start>--<end>` suffix is optional so legacy renamed-letter
+# transcripts (e.g. `SESSION-0009.0-TRANSCRIPT.md` without timestamps)
+# remain reachable. The `\.\d+(\.\d+)*` after the session number
+# accommodates `0017.0`, `0017.1`, `0017.1.1`, etc.
+_TRANSCRIPT_FILENAME_RE = re.compile(
+    r"^SESSION-\d{4}\.\d+(\.\d+)*-TRANSCRIPT"
+    r"(-\d{4}-\d{2}-\d{2}T\d{2}-\d{2}--\d{4}-\d{2}-\d{2}T\d{2}-\d{2})?"
+    r"\.md$"
+)
+_SESSION_DIR_RE = re.compile(r"^\d{4}$")
+
+_HTTP_TIMEOUT_SEC = 5.0
+
+
+def _env_float(name: str, default: float) -> float:
+    raw = os.environ.get(name, "").strip()
+    if not raw:
+        return default
+    try:
+        return float(raw)
+    except ValueError:
+        log.warning("invalid %s=%r — falling back to %s", name, raw, default)
+        return default
+
+
+def _raw_base() -> str:
+    return os.environ.get("OHM_SESSION_HISTORY_RAW_BASE", "").strip() or _DEFAULT_RAW_BASE
+
+
+def _contents_base() -> str:
+    return (
+        os.environ.get("OHM_SESSION_HISTORY_CONTENTS_BASE", "").strip()
+        or _DEFAULT_CONTENTS_BASE
+    )
+
+
+def _manifest_ttl() -> float:
+    return _env_float("OHM_DOCS_SESSIONS_MANIFEST_TTL_SEC", _DEFAULT_MANIFEST_TTL_SEC)
+
+
+def _content_ttl() -> float:
+    return _env_float("OHM_DOCS_SESSIONS_CONTENT_TTL_SEC", _DEFAULT_CONTENT_TTL_SEC)
+
+
+# ---------------------------------------------------------------------------
+# In-process TTL cache
+# ---------------------------------------------------------------------------
+#
+# Plain dict + `time.monotonic()` check, no external dep. The cache
+# value is a `(stored_at, payload)` tuple; `payload` may carry an
+# error-shape sentinel for negative caching (404s). Lock guards
+# read-modify-write across worker tasks; entries are immutable once
+# stored so reads under the lock are fast.
+
+_lock = threading.Lock()
+_cache: dict[str, tuple[float, dict[str, Any]]] = {}
+
+
+def _cache_get(key: str, ttl_sec: float) -> dict[str, Any] | None:
+    with _lock:
+        entry = _cache.get(key)
+        if entry is None:
+            return None
+        stored_at, payload = entry
+        if time.monotonic() - stored_at > ttl_sec:
+            # Don't evict here; let _cache_put overwrite on next fetch.
+            # The stale entry is gated by the TTL check, so it stays
+            # invisible to readers regardless.
+            return None
+        return payload
+
+
+def _cache_put(key: str, payload: dict[str, Any]) -> None:
+    with _lock:
+        _cache[key] = (time.monotonic(), payload)
+
+
+def reset_cache() -> None:
+    """Drop every cached entry. Test seam — not called in production."""
+    with _lock:
+        _cache.clear()
+
+
+# ---------------------------------------------------------------------------
+# Public fetch surface
+# ---------------------------------------------------------------------------
+#
+# Each fetcher returns a `{status, ...}` dict. `status` is one of:
+#   "ok"     — payload field carries the body
+#   "404"    — gitea returned 404 (or content was missing)
+#   "error"  — gitea returned 5xx, timed out, or returned malformed data
+#
+# The route layer maps these onto HTTP responses; keeping the mapping
+# out of this module makes the cache transparent to the test harness.
+
+
+async def _http_get(url: str) -> tuple[int, str]:
+    """Perform a single GET against `url`; return (status_code, body).
+
+    On timeout or network error, returns (599, error_message). The 599
+    pseudo-status maps to a 502 at the route layer the same way an
+    upstream 5xx does — the caller doesn't care which leg of the
+    network broke.
+    """
+    try:
+        async with httpx.AsyncClient(timeout=_HTTP_TIMEOUT_SEC) as client:
+            r = await client.get(url)
+            return r.status_code, r.text
+    except httpx.HTTPError as e:
+        log.warning("gitea fetch failed for %s: %s", url, e)
+        return 599, f"fetch error: {e}"
+
+
+def _is_valid_session_dir(nnnn: str) -> bool:
+    return bool(_SESSION_DIR_RE.match(nnnn))
+
+
+def _is_valid_transcript_filename(filename: str) -> bool:
+    return bool(_TRANSCRIPT_FILENAME_RE.match(filename))
+
+
+async def fetch_manifest() -> dict[str, Any]:
+    """Fetch and parse `sessions.json` from the public repo.
+
+    Returns one of:
+      {"status": "ok", "manifest": {...}}        — successful parse
+      {"status": "404"}                          — gitea 404 (empty state)
+      {"status": "error", "detail": "..."}       — 5xx / timeout / bad JSON
+    """
+    cache_key = "manifest"
+    cached = _cache_get(cache_key, _manifest_ttl())
+    if cached is not None:
+        return cached
+
+    url = f"{_raw_base()}/sessions.json"
+    status, body = await _http_get(url)
+
+    if status == 200:
+        try:
+            import json
+
+            data = json.loads(body)
+        except (json.JSONDecodeError, ValueError) as e:
+            payload: dict[str, Any] = {
+                "status": "error",
+                "detail": f"sessions.json malformed: {e}",
+            }
+            # Don't cache parse errors — give the upstream a chance to
+            # fix the file without waiting for TTL expiry.
+            return payload
+        if not isinstance(data, dict):
+            return {
+                "status": "error",
+                "detail": "sessions.json is not a JSON object",
+            }
+        payload = {"status": "ok", "manifest": data}
+        _cache_put(cache_key, payload)
+        return payload
+
+    if status == 404:
+        payload = {"status": "404"}
+        _cache_put(cache_key, payload)
+        return payload
+
+    return {"status": "error", "detail": f"upstream returned {status}"}
+
+
+async def fetch_about() -> dict[str, Any]:
+    """Fetch the repo's README.md (rendered as the /docs/sessions/about page).
+
+    Returns one of:
+      {"status": "ok", "body": "..."}
+      {"status": "404"}
+      {"status": "error", "detail": "..."}
+    """
+    cache_key = "about:README.md"
+    cached = _cache_get(cache_key, _content_ttl())
+    if cached is not None:
+        return cached
+
+    url = f"{_raw_base()}/README.md"
+    status, body = await _http_get(url)
+
+    if status == 200:
+        payload: dict[str, Any] = {"status": "ok", "body": body}
+        _cache_put(cache_key, payload)
+        return payload
+    if status == 404:
+        payload = {"status": "404"}
+        _cache_put(cache_key, payload)
+        return payload
+    return {"status": "error", "detail": f"upstream returned {status}"}
+
+
+async def fetch_transcript(nnnn: str, filename: str) -> dict[str, Any]:
+    """Fetch a single transcript body from `{nnnn}/{filename}` in the repo.
+
+    The caller is expected to have validated `nnnn` and `filename`
+    against `_is_valid_session_dir` / `_is_valid_transcript_filename`
+    before calling this — invalid paths shouldn't reach the network.
+    """
+    cache_key = f"transcript:{nnnn}/{filename}"
+    cached = _cache_get(cache_key, _content_ttl())
+    if cached is not None:
+        return cached
+
+    url = f"{_raw_base()}/{nnnn}/{filename}"
+    status, body = await _http_get(url)
+
+    if status == 200:
+        payload: dict[str, Any] = {"status": "ok", "body": body}
+        _cache_put(cache_key, payload)
+        return payload
+    if status == 404:
+        payload = {"status": "404"}
+        _cache_put(cache_key, payload)
+        return payload
+    return {"status": "error", "detail": f"upstream returned {status}"}
+
+
+async def fetch_session_index(nnnn: str) -> dict[str, Any]:
+    """List the transcript filenames inside the `{nnnn}/` folder.
+
+    Uses gitea's contents API (one HTTP per session-index page-view per
+    cache-TTL) rather than the raw URL — there's no flat way to list a
+    folder via the raw mount.
+
+    Returns one of:
+      {"status": "ok", "files": ["SESSION-...md", ...]}
+      {"status": "404"}
+      {"status": "error", "detail": "..."}
+
+    Only filenames that match `_is_valid_transcript_filename` are
+    surfaced — sibling files (e.g. an attached `notes.md`) are ignored
+    so the /docs/sessions/<NNNN> page never lists a non-transcript
+    masquerading as one.
+    """
+    cache_key = f"index:{nnnn}"
+    cached = _cache_get(cache_key, _content_ttl())
+    if cached is not None:
+        return cached
+
+    url = f"{_contents_base()}/{nnnn}"
+    status, body = await _http_get(url)
+
+    if status == 200:
+        try:
+            import json
+
+            data = json.loads(body)
+        except (json.JSONDecodeError, ValueError) as e:
+            return {
+                "status": "error",
+                "detail": f"contents API response malformed: {e}",
+            }
+        if not isinstance(data, list):
+            return {
+                "status": "error",
+                "detail": "contents API returned non-list",
+            }
+        files: list[str] = []
+        for entry in data:
+            if not isinstance(entry, dict):
+                continue
+            if entry.get("type") != "file":
+                continue
+            name = entry.get("name")
+            if not isinstance(name, str):
+                continue
+            if _is_valid_transcript_filename(name):
+                files.append(name)
+        files.sort()
+        payload: dict[str, Any] = {"status": "ok", "files": files}
+        _cache_put(cache_key, payload)
+        return payload
+    if status == 404:
+        payload = {"status": "404"}
+        _cache_put(cache_key, payload)
+        return payload
+    return {"status": "error", "detail": f"upstream returned {status}"}