From e5ca07e2a475c12a60e0367d88629ab1e4796596 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Sun, 7 Jun 2026 22:52:01 -0700 Subject: [PATCH] docs(plan): implementation plan for reconciled simulator-alteration slice Co-Authored-By: Claude Opus 4.8 (1M context) --- ...7-reconciled-simulator-alteration-slice.md | 1208 +++++++++++++++++ 1 file changed, 1208 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md diff --git a/docs/superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md b/docs/superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md new file mode 100644 index 0000000..4166be1 --- /dev/null +++ b/docs/superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md @@ -0,0 +1,1208 @@ +# Reconciled Simulator-First Alteration Slice — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire the alteration engine into the simulator — live Dark/Light grade, a live Left analytical overlay (authored annotation track + runtime-shaped text), and discrete pre-baked flow-stabilized Right variants over ONE neutral clip — so the look can be tuned by eye before any hardware. + +**Architecture:** Python stays the single source of truth for the alteration math (`player/alteration.py`); the browser only renders. The engine model changes are surgical: a continuous `Restyle.blend` becomes a discrete `Restyle.variant` (selecting a pre-baked clip), `AnalyticalOverlay` gains a Left `level`, and a new frozen `Calibration` parameterizes the knob→strength curves (default behavior-preserving). The simulator retires its selection-era surface (`/api/select`, X-ray, `fixtures.py`) for an alteration surface (`/api/alteration`, `/api/clips`, `clips.py`) reading a base-clip + variant + annotation manifest. + +**Tech Stack:** Python 3.13, FastAPI + pydantic, pytest, frozen dataclasses; vanilla JS + CSS filters / SVG for the browser; ffmpeg (via `imageio-ffmpeg` or system `ffmpeg`) for placeholder media. + +**Spec:** `docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md` + +--- + +## File Structure + +**Engine (modify):** +- `player/alteration.py` — add `Calibration` + `DEFAULT_CALIBRATION`; `Restyle.blend: float` → `Restyle.variant: int`; `AnalyticalOverlay` gains `level: int`; `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`; a `render_plan_to_dict` serializer for the API. +- `player/state.py` — no logic change (compares whole `Restyle` objects); docstring touch only. + +**Simulator (create/rewrite):** +- `simulator/clips.py` (create) — load a base-clip + variant + annotation manifest; replaces `simulator/fixtures.py` (delete). +- `simulator/app.py` (rewrite) — `POST /api/alteration`, `GET /api/clips`; remove `/api/select`, `/api/catalog/meta`. +- `simulator/static/{index.html,app.js,style.css}` (rewrite) — alteration preview. +- `simulator/sample_media/manifest.json` (create, committed) + `simulator/sample_media/README.md`; media files themselves gitignored. +- `simulator/setup_sample_media.py` (create) — copy POC clips + generate placeholder Right strengths. + +**Tests (create/rewrite):** +- `tests/test_player_alteration.py` (rewrite for new model). +- `tests/test_player_state.py` (update `.blend`→`.variant` references). +- `tests/test_clips.py` (create); `tests/test_fixtures.py` (delete). +- `tests/test_simulator_api.py` (rewrite for new endpoints). + +**Docs:** +- `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` — §4.3/§10 pointer to the reconciliation. +- `docs/ROADMAP.md` — §3 slice status. +- `docs/USER_GUIDE.md` — "Playing with the simulator" rewrite. + +--- + +## Task 1: Engine — `Calibration`, discrete `Restyle.variant`, overlay `level` + +**Files:** +- Modify: `player/alteration.py` +- Test: `tests/test_player_alteration.py` (rewrite) + +- [ ] **Step 1: Rewrite the failing tests for the new model** + +Replace the entire contents of `tests/test_player_alteration.py`: + +```python +import pytest + +from hef.selection import Coordinate +from player.alteration import ( + DEFAULT_CALIBRATION, + AnalyticalOverlay, + Calibration, + ColorGrade, + RenderPlan, + Restyle, + plan_alteration, + render_plan_to_dict, +) + + +def _coord(left=0, right=0, dark=0, light=0): + return Coordinate(left=left, right=right, dark=dark, light=light) + + +def test_all_zero_knobs_is_the_unaltered_base(): + plan = plan_alteration(_coord()) + assert plan.is_identity + assert plan.overlay.level == 0 + assert plan.overlay.intensity == 0.0 + assert plan.restyle.variant == 0 + assert plan.grade.tone == 0.0 + assert plan.grade.is_identity + + +def test_left_drives_the_analytical_overlay_only(): + plan = plan_alteration(_coord(left=4)) + assert plan.overlay.level == 4 + assert plan.overlay.intensity == 1.0 + assert plan.restyle.variant == 0 # Left does not touch the substrate + assert plan.grade.tone == 0.0 + + +def test_right_selects_a_discrete_restyle_variant_only(): + plan = plan_alteration(_coord(right=2)) + assert plan.restyle.variant == 2 + assert plan.overlay.level == 0 # Right does not add overlay + + +def test_left_and_right_stack_not_cancel(): + # design §4.2: whole-brain corner = dreamlike substrate WITH labels on top + plan = plan_alteration(_coord(left=4, right=4)) + assert plan.overlay.level == 4 + assert plan.restyle.variant == 4 + + +def test_light_pole_grades_warm_positive_tone(): + plan = plan_alteration(_coord(light=4)) + assert plan.grade.tone == 1.0 + assert not plan.grade.is_identity + + +def test_dark_pole_grades_cool_negative_tone(): + assert plan_alteration(_coord(dark=4)).grade.tone == -1.0 + + +def test_equal_dark_and_light_is_identity_grade(): + # design §5: the mood center is the raw, ungraded footage + assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity + assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity + + +def test_dark_minus_light_sets_intermediate_tone(): + assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5) + assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5) + + +def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay(): + plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0)) + assert plan.overlay.level == 4 + assert plan.restyle.variant == 2 + assert plan.grade.tone == -1.0 + assert not plan.is_identity + + +def test_default_calibration_is_behavior_preserving(): + # DEFAULT_CALIBRATION must reproduce the original three helpers exactly. + for left in range(5): + assert plan_alteration(_coord(left=left)).overlay.intensity == pytest.approx(left / 4) + for right in range(5): + assert plan_alteration(_coord(right=right)).restyle.variant == right + for dark in range(5): + for light in range(5): + expected = (light - dark) / 4 + assert plan_alteration(_coord(dark=dark, light=light)).grade.tone == pytest.approx(expected) + + +def test_custom_calibration_scales_mood_and_overlay(): + cal = Calibration(mood_gain=0.5, overlay_gain=0.5, right_variant_map=(0, 0, 1, 1, 2)) + assert plan_alteration(_coord(light=4), cal).grade.tone == pytest.approx(0.5) + assert plan_alteration(_coord(left=4), cal).overlay.intensity == pytest.approx(0.5) + assert plan_alteration(_coord(right=3), cal).restyle.variant == 1 + + +def test_calibration_gain_is_clamped_to_unit_range(): + cal = Calibration(mood_gain=10.0, overlay_gain=10.0) + assert plan_alteration(_coord(light=4), cal).grade.tone == 1.0 # clamped, not 10 + assert plan_alteration(_coord(left=4), cal).overlay.intensity == 1.0 + + +def test_render_plan_to_dict_round_trips_the_numbers(): + d = render_plan_to_dict(plan_alteration(_coord(left=4, right=2, dark=4, light=0))) + assert d == { + "grade": {"tone": -1.0}, + "overlay": {"level": 4, "intensity": 1.0}, + "restyle": {"variant": 2}, + "is_identity": False, + } + + +def test_render_plan_is_frozen(): + plan = plan_alteration(_coord()) + with pytest.raises(Exception): + plan.grade.tone = 0.5 # type: ignore[misc] +``` + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: `python -m pytest tests/test_player_alteration.py -q` +Expected: FAIL — `ImportError` for `Calibration` / `DEFAULT_CALIBRATION` / `render_plan_to_dict` (and `Restyle.variant` / `overlay.level` attribute errors). + +- [ ] **Step 3: Rewrite `player/alteration.py` for the new model** + +Replace the entire contents of `player/alteration.py`: + +```python +"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5). + +Reconciled slice (2026-06-07): the Right axis is a DISCRETE selection of a +pre-baked, flow-stabilized restyle variant (not a continuous blend), the Left +axis carries its knob LEVEL so a runtime annotation track can pick which labels +show, and a frozen `Calibration` parameterizes the knob->strength curves so they +can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION. + +Layers compose per §4.2: + - Substrate: ColorGrade (Dark/Light mood, center = identity §5) + a pre-baked + Right restyle variant. + - Overlay: AnalyticalOverlay (Left), composited on top at runtime. +Left and Right stack (different layers); Dark/Light are the two poles of one +mood grade. See docs/superpowers/specs/2026-06-07-reconciled-simulator- +alteration-slice-design.md. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field + +from hef.selection import Coordinate + +KNOB_MAX = 4 # knob full-scale (0..4) + + +def _clamp(x: float, lo: float, hi: float) -> float: + return max(lo, min(hi, x)) + + +@dataclass(frozen=True) +class Calibration: + """Tunable knob->strength curves (settled by eye in the sim, then baked). + + - mood_gain: scales the signed Dark/Light tone (result clamped to [-1, 1]). + - overlay_gain: scales the Left overlay intensity (clamped to [0, 1]). + - right_variant_map: knob value (0..4) -> pre-baked Right variant index. + """ + + mood_gain: float = 1.0 + overlay_gain: float = 1.0 + right_variant_map: tuple = (0, 1, 2, 3, 4) + + +DEFAULT_CALIBRATION = Calibration() + + +@dataclass(frozen=True) +class ColorGrade: + """Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light), + <0 cool blue->black (dark), 0 = identity (raw, ungraded).""" + + tone: float + + @property + def is_identity(self) -> bool: + return self.tone == 0.0 + + +@dataclass(frozen=True) +class AnalyticalOverlay: + """Left axis (§4.1): analytical HUD/annotation, composited on top at runtime. + `level` is the Left knob (0..4); a runtime annotation track uses it to pick + which labels appear. `intensity` 0..1 is the overlay opacity/strength.""" + + level: int + intensity: float + + +@dataclass(frozen=True) +class Restyle: + """Right axis (§4.1): selects a pre-baked, flow-stabilized restyle variant. + `variant` is a discrete index (0 = raw base, no restyle).""" + + variant: int + + +@dataclass(frozen=True) +class RenderPlan: + """The full layered alteration for one knob vector (§4.2).""" + + grade: ColorGrade + overlay: AnalyticalOverlay + restyle: Restyle + + @property + def is_identity(self) -> bool: + """True when the plan leaves the neutral base un-altered.""" + return ( + self.grade.is_identity + and self.overlay.level == 0 + and self.restyle.variant == 0 + ) + + +def _overlay_intensity(left: int, cal: Calibration) -> float: + return _clamp(cal.overlay_gain * left / KNOB_MAX, 0.0, 1.0) + + +def _right_variant(right: int, cal: Calibration) -> int: + return cal.right_variant_map[right] + + +def _mood_tone(dark: int, light: int, cal: Calibration) -> float: + return _clamp(cal.mood_gain * (light - dark) / KNOB_MAX, -1.0, 1.0) + + +def plan_alteration( + coord: Coordinate, calibration: Calibration = DEFAULT_CALIBRATION +) -> RenderPlan: + """Map a knob vector to its layered RenderPlan (design §4).""" + return RenderPlan( + grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light, calibration)), + overlay=AnalyticalOverlay( + level=coord.left, + intensity=_overlay_intensity(coord.left, calibration), + ), + restyle=Restyle(variant=_right_variant(coord.right, calibration)), + ) + + +def render_plan_to_dict(plan: RenderPlan) -> dict: + """JSON-serializable form for the simulator API.""" + return { + "grade": {"tone": plan.grade.tone}, + "overlay": {"level": plan.overlay.level, "intensity": plan.overlay.intensity}, + "restyle": {"variant": plan.restyle.variant}, + "is_identity": plan.is_identity, + } +``` + +Note: `field` import is unused — drop it; only `dataclass` is needed. (Final import line: `from dataclasses import dataclass`.) + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: `python -m pytest tests/test_player_alteration.py -q` +Expected: PASS (all). + +- [ ] **Step 5: Commit** + +```bash +git add player/alteration.py tests/test_player_alteration.py +git commit -m "feat(player): discrete Right variant + Left level + Calibration (slice design §2)" +``` + +--- + +## Task 2: `state.py` reconcile (field rename in tests; no logic change) + +**Files:** +- Modify: `player/state.py` (docstring only) +- Test: `tests/test_player_state.py` + +- [ ] **Step 1: Update the failing test references** + +In `tests/test_player_state.py`, change the two `.blend` assertions to `.variant` and the overlay assertion to `.level`: + +Replace `test_overlay_change_is_a_live_update` body's last assertion line: +```python + assert t.playback.plan.overlay.level == 4 +``` +Replace `test_restyle_change_crossfades_the_substrate` body's last assertion line: +```python + assert t.playback.plan.restyle.variant == 4 +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `python -m pytest tests/test_player_state.py -q` +Expected: FAIL — `AttributeError: 'Restyle' object has no attribute 'blend'` is already gone after Task 1, so the failure is now the *test's own* stale references resolved by Step 1; re-running confirms. If Step 1 is applied, this should already pass — run to confirm the suite is green. + +- [ ] **Step 3: Touch the `state.py` docstring** + +In `player/state.py`, update the module docstring sentence mentioning the Right variant to read: +``` +swapping the clip or the pre-baked Right restyle variant (a discrete index) +needs a CROSSFADE +``` +(No code change — `_classify` already compares whole `Restyle` objects via `nxt.plan.restyle != prev.plan.restyle`.) + +- [ ] **Step 4: Run to verify pass** + +Run: `python -m pytest tests/test_player_state.py -q` +Expected: PASS (all). + +- [ ] **Step 5: Commit** + +```bash +git add player/state.py tests/test_player_state.py +git commit -m "refactor(player): state.py tests track discrete Restyle.variant" +``` + +--- + +## Task 3: `simulator/clips.py` — the base-clip + variant + annotation manifest + +**Files:** +- Create: `simulator/clips.py` +- Create: `tests/test_clips.py` +- Delete: `simulator/fixtures.py`, `tests/test_fixtures.py` + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_clips.py`: + +```python +import json + +import pytest + +from simulator.clips import Clip, load_manifest + + +def _manifest_dict(): + return { + "clips": [ + { + "id": "forest", + "title": "Yosemite Falls (neutral)", + "base_file": "forest/base.mp4", + "license": "poc-sample", + "source": "hef-poc", + "right_variants": { + "4": {"file": "forest/right4.mp4", "model": "sd-turbo+flow"}, + "1": {"file": "forest/right1.mp4"}, + }, + "annotations": [ + {"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1}, + {"key": "detected.conifer", "box": [0.6, 0.1, 0.2, 0.2], "min_level": 3}, + ], + "strings": {"en": {"detected.water": "flowing water", "detected.conifer": "conifer"}}, + } + ] + } + + +def test_load_manifest_parses_clips(tmp_path): + p = tmp_path / "manifest.json" + p.write_text(json.dumps(_manifest_dict())) + clips = load_manifest(p) + assert len(clips) == 1 + c = clips[0] + assert isinstance(c, Clip) + assert c.id == "forest" + assert c.base_file == "forest/base.mp4" + + +def test_clip_lists_variant_files_by_strength(tmp_path): + p = tmp_path / "manifest.json" + p.write_text(json.dumps(_manifest_dict())) + c = load_manifest(p)[0] + # variant 0 is always the raw base; authored strengths come from the manifest + assert c.variant_file(0) == "forest/base.mp4" + assert c.variant_file(4) == "forest/right4.mp4" + assert c.variant_file(1) == "forest/right1.mp4" + # an unauthored strength falls back to the raw base + assert c.variant_file(2) == "forest/base.mp4" + + +def test_clip_serializes_to_dict_for_the_api(tmp_path): + p = tmp_path / "manifest.json" + p.write_text(json.dumps(_manifest_dict())) + d = load_manifest(p)[0].to_dict() + assert d["id"] == "forest" + assert d["base_file"] == "forest/base.mp4" + assert d["annotations"][0]["key"] == "detected.water" + assert d["strings"]["en"]["detected.water"] == "flowing water" + # variant map is exposed keyed by strength string, including 0 -> base + assert d["right_variants"]["0"]["file"] == "forest/base.mp4" + assert d["right_variants"]["4"]["file"] == "forest/right4.mp4" + + +def test_missing_manifest_raises(tmp_path): + with pytest.raises(FileNotFoundError): + load_manifest(tmp_path / "nope.json") +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `python -m pytest tests/test_clips.py -q` +Expected: FAIL — `ModuleNotFoundError: No module named 'simulator.clips'`. + +- [ ] **Step 3: Create `simulator/clips.py`** + +```python +"""The base-clip + variant + annotation manifest the simulator renders. + +Replaces simulator/fixtures.py (the selection-era synthetic catalog). Each base +clip carries: the raw base file, a map of pre-baked Right-strength variant files +(strength 0 is always the raw base), an authored Left annotation track (box + +label key + the minimum Left level at which it appears), and per-language string +tables. See the reconciled-simulator-alteration-slice design §3.2. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from pathlib import Path +from typing import Any + + +@dataclass(frozen=True) +class Clip: + id: str + title: str + base_file: str + license: str + source: str + right_variants: dict # {"1": {"file": ...}, "4": {...}} (no "0") + annotations: list # [{"key", "box":[x,y,w,h], "min_level"}, ...] + strings: dict # {"en": {key: text}} + + def variant_file(self, strength: int) -> str: + """The video file for a Right strength; 0 and any unauthored strength + fall back to the raw base file.""" + entry = self.right_variants.get(str(strength)) + return entry["file"] if entry else self.base_file + + def to_dict(self) -> dict: + variants = {"0": {"file": self.base_file, "raw": True}} + for k, v in self.right_variants.items(): + variants[k] = v + return { + "id": self.id, + "title": self.title, + "base_file": self.base_file, + "license": self.license, + "source": self.source, + "right_variants": variants, + "annotations": self.annotations, + "strings": self.strings, + } + + +def _clip_from_dict(d: dict[str, Any]) -> Clip: + return Clip( + id=d["id"], + title=d["title"], + base_file=d["base_file"], + license=d.get("license", ""), + source=d.get("source", ""), + right_variants=d.get("right_variants", {}), + annotations=d.get("annotations", []), + strings=d.get("strings", {}), + ) + + +def load_manifest(path: str | Path) -> list[Clip]: + """Load the base-clip manifest. Raises FileNotFoundError if missing.""" + path = Path(path) + data = json.loads(path.read_text()) + return [_clip_from_dict(c) for c in data["clips"]] +``` + +- [ ] **Step 4: Run to verify it passes** + +Run: `python -m pytest tests/test_clips.py -q` +Expected: PASS (all). + +- [ ] **Step 5: Delete the retired fixtures** + +```bash +git rm simulator/fixtures.py tests/test_fixtures.py +``` + +- [ ] **Step 6: Commit** + +```bash +git add simulator/clips.py tests/test_clips.py +git commit -m "feat(simulator): clips.py manifest model; retire selection fixtures" +``` + +--- + +## Task 4: `simulator/app.py` — `/api/alteration` + `/api/clips` + +**Files:** +- Modify: `simulator/app.py` (rewrite) +- Test: `tests/test_simulator_api.py` (rewrite) + +- [ ] **Step 1: Rewrite the failing tests** + +Replace the entire contents of `tests/test_simulator_api.py`: + +```python +import json + +import pytest +from fastapi.testclient import TestClient + +from simulator.app import create_app + + +@pytest.fixture +def manifest_path(tmp_path): + p = tmp_path / "manifest.json" + p.write_text(json.dumps({ + "clips": [{ + "id": "forest", + "title": "neutral forest", + "base_file": "forest/base.mp4", + "license": "poc", "source": "hef-poc", + "right_variants": {"4": {"file": "forest/right4.mp4"}}, + "annotations": [{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1}], + "strings": {"en": {"detected.water": "flowing water"}}, + }] + })) + return p + + +@pytest.fixture +def client(manifest_path): + return TestClient(create_app(manifest_path=manifest_path)) + + +def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2): + return dict(content=content, left=left, right=right, dark=dark, + light=light, volume=volume, brightness=brightness) + + +def test_alteration_returns_the_engine_plan(client): + resp = client.post("/api/alteration", json={"controls": _controls(left=4, right=2, dark=4)}) + assert resp.status_code == 200 + data = resp.json() + assert data["plan"]["overlay"]["level"] == 4 + assert data["plan"]["restyle"]["variant"] == 2 + assert data["plan"]["grade"]["tone"] == -1.0 + assert data["content"]["video"] is True + + +def test_alteration_honors_off_as_black(client): + resp = client.post("/api/alteration", json={"controls": _controls(content="off")}) + data = resp.json() + assert data["content"]["video"] is False + + +def test_alteration_accepts_calibration(client): + body = {"controls": _controls(light=4), + "calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "right_variant_map": [0, 1, 2, 3, 4]}} + resp = client.post("/api/alteration", json=body) + assert resp.json()["plan"]["grade"]["tone"] == 0.5 + + +def test_alteration_rejects_out_of_range_knob(client): + resp = client.post("/api/alteration", json={"controls": _controls(left=7)}) + assert resp.status_code == 422 + + +def test_alteration_rejects_bad_content(client): + resp = client.post("/api/alteration", json={"controls": _controls(content="banana")}) + assert resp.status_code == 422 + + +def test_clips_returns_the_manifest(client): + resp = client.get("/api/clips") + assert resp.status_code == 200 + data = resp.json() + assert data["clips"][0]["id"] == "forest" + assert data["clips"][0]["right_variants"]["0"]["file"] == "forest/base.mp4" + assert data["clips"][0]["annotations"][0]["key"] == "detected.water" + + +def test_retired_selection_endpoints_are_gone(client): + assert client.post("/api/select", json={}).status_code == 404 + assert client.get("/api/catalog/meta").status_code == 404 + + +def test_index_is_served(): + client = TestClient(create_app()) + resp = client.get("/") + assert resp.status_code == 200 + assert "text/html" in resp.headers["content-type"] + assert "Alteration" in resp.text +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `python -m pytest tests/test_simulator_api.py -q` +Expected: FAIL — `create_app` has no `manifest_path` kwarg; `/api/alteration` 404; old selection imports break. + +- [ ] **Step 3: Rewrite `simulator/app.py`** + +```python +"""FastAPI service: controls -> the real alteration engine -> a RenderPlan. + +The simulator's alteration surface (reconciled slice). It calls the canonical +player.alteration.plan_alteration; the browser only renders. The selection-era +endpoints (/api/select, /api/catalog/meta) and the X-ray are retired. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Optional + +from fastapi import FastAPI +from fastapi.staticfiles import StaticFiles +from pydantic import BaseModel, Field + +from hef.selection import Coordinate +from player.alteration import ( + DEFAULT_CALIBRATION, + Calibration, + plan_alteration, + render_plan_to_dict, +) +from player.content import resolve_content +from player.controls import CONTENT_POSITIONS +from simulator.clips import load_manifest + +STATIC_DIR = Path(__file__).parent / "static" +DEFAULT_MANIFEST = Path(__file__).parent / "sample_media" / "manifest.json" +MEDIA_DIR = Path(__file__).parent / "sample_media" + + +class ControlsModel(BaseModel): + content: str + left: int = Field(ge=0, le=4) + right: int = Field(ge=0, le=4) + dark: int = Field(ge=0, le=4) + light: int = Field(ge=0, le=4) + volume: int = Field(ge=0, le=4) + brightness: int = Field(ge=0, le=4) + + +class CalibrationModel(BaseModel): + mood_gain: float = 1.0 + overlay_gain: float = 1.0 + right_variant_map: list[int] = [0, 1, 2, 3, 4] + + +class AlterationRequest(BaseModel): + controls: ControlsModel + calibration: Optional[CalibrationModel] = None + + +def _load_clips(manifest_path: Optional[Path]): + path = Path(manifest_path) if manifest_path else DEFAULT_MANIFEST + if path.exists(): + return load_manifest(path) + return [] + + +def create_app(manifest_path: Optional[Path] = None) -> FastAPI: + app = FastAPI(title="HEF Alteration Simulator") + app.state.clips = _load_clips(manifest_path) + + @app.post("/api/alteration") + def api_alteration(req: AlterationRequest): + c = req.controls + if c.content not in CONTENT_POSITIONS: + # mirror Controls validation as a 422 + from fastapi import HTTPException + raise HTTPException(status_code=422, detail=f"invalid content {c.content!r}") + coord = Coordinate(c.left, c.right, c.dark, c.light) + cal = ( + Calibration( + mood_gain=req.calibration.mood_gain, + overlay_gain=req.calibration.overlay_gain, + right_variant_map=tuple(req.calibration.right_variant_map), + ) + if req.calibration + else DEFAULT_CALIBRATION + ) + plan = plan_alteration(coord, cal) + content = resolve_content(c.content) + return { + "plan": render_plan_to_dict(plan), + "content": {"audio_source": content.audio_source, "video": content.video}, + } + + @app.get("/api/clips") + def api_clips(): + return {"clips": [c.to_dict() for c in app.state.clips]} + + if MEDIA_DIR.exists(): + app.mount("/media", StaticFiles(directory=MEDIA_DIR), name="media") + if STATIC_DIR.exists(): + app.mount("/", StaticFiles(directory=STATIC_DIR, html=True), name="static") + + return app + + +app = create_app() +``` + +Note: confirm `player/content.py`'s `ContentResolution` exposes `audio_source` and `video` (it does — `state.py` reads both). If the attribute is named differently, match it. + +- [ ] **Step 4: Run to verify it passes** + +Run: `python -m pytest tests/test_simulator_api.py -q` +Expected: PASS. The `test_index_is_served` test needs the rewritten `index.html` (Task 6) containing "Alteration" — if Task 6 is not yet done, that one test fails; run the rest with `-k "not index"` until Task 6. + +- [ ] **Step 5: Commit** + +```bash +git add simulator/app.py tests/test_simulator_api.py +git commit -m "feat(simulator): /api/alteration + /api/clips; retire selection endpoints" +``` + +--- + +## Task 5: Sample media + placeholder Right strengths + +**Files:** +- Create: `simulator/sample_media/manifest.json` (committed) +- Create: `simulator/sample_media/README.md` (committed) +- Create: `simulator/setup_sample_media.py` (committed) +- Modify: `.gitignore` (ignore the media binaries) + +- [ ] **Step 1: Author the manifest** + +Create `simulator/sample_media/manifest.json`: + +```json +{ + "clips": [ + { + "id": "forest", + "title": "Yosemite Falls (neutral base, POC sample)", + "base_file": "forest/base.mp4", + "license": "poc-sample (look-tuning only; not shipped content)", + "source": "hef-poc/out/neutral.mp4", + "right_variants": { + "1": {"file": "forest/right1.mp4", "model": "placeholder"}, + "2": {"file": "forest/right2.mp4", "model": "placeholder"}, + "3": {"file": "forest/right3.mp4", "model": "placeholder"}, + "4": {"file": "forest/right4.mp4", "model": "sd-turbo+farneback-flow"} + }, + "annotations": [ + {"key": "detected.water", "box": [0.30, 0.10, 0.18, 0.70], "min_level": 1}, + {"key": "detected.rock_face", "box": [0.05, 0.30, 0.20, 0.55], "min_level": 2}, + {"key": "detected.conifer", "box": [0.70, 0.20, 0.22, 0.45], "min_level": 3}, + {"key": "measure.flow_rate", "box": [0.34, 0.55, 0.14, 0.08], "min_level": 4} + ], + "strings": { + "en": { + "detected.water": "flowing water", + "detected.rock_face": "granite face", + "detected.conifer": "conifer stand", + "measure.flow_rate": "~2.1 m³/s" + } + } + } + ] +} +``` + +- [ ] **Step 2: Write the setup script** + +Create `simulator/setup_sample_media.py`: + +```python +"""Populate simulator/sample_media/forest/ from the session-0008 POC artifacts. + +Copies the real neutral base + the real flow-stabilized Right restyle out of +~/hef-poc/out/, and generates placeholder intermediate Right strengths (1..3) by +blending the base toward the real restyle with ffmpeg. The media binaries are +gitignored; only the manifest is committed. Sample footage is for look-tuning +only, not shipped content. + +Usage: python simulator/setup_sample_media.py +Requires: ffmpeg on PATH (or `pip install imageio-ffmpeg`), and ~/hef-poc/out/. +""" + +from __future__ import annotations + +import shutil +import subprocess +from pathlib import Path + +POC = Path.home() / "hef-poc" / "out" +DEST = Path(__file__).parent / "sample_media" / "forest" + + +def _ffmpeg() -> str: + if shutil.which("ffmpeg"): + return "ffmpeg" + import imageio_ffmpeg + return imageio_ffmpeg.get_ffmpeg_exe() + + +def main() -> None: + DEST.mkdir(parents=True, exist_ok=True) + base = DEST / "base.mp4" + right4 = DEST / "right4.mp4" + shutil.copyfile(POC / "neutral.mp4", base) + shutil.copyfile(POC / "right_flow.mp4", right4) + ff = _ffmpeg() + # Placeholder strengths 1..3: opacity-blend base toward the real restyle. + for strength, alpha in ((1, 0.25), (2, 0.5), (3, 0.75)): + out = DEST / f"right{strength}.mp4" + subprocess.run( + [ff, "-y", "-i", str(base), "-i", str(right4), + "-filter_complex", + f"[1:v]format=yuva444p,colorchannelmixer=aa={alpha}[top];" + f"[0:v][top]overlay=shortest=1[v]", + "-map", "[v]", "-an", str(out)], + check=True, + ) + print(f"generated {out.name} (alpha {alpha})") + print(f"sample media ready in {DEST}") + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 3: Gitignore the media binaries, document them** + +Append to `.gitignore`: +``` +# Simulator sample media (look-tuning only; populate via setup_sample_media.py) +simulator/sample_media/forest/*.mp4 +``` + +Create `simulator/sample_media/README.md`: +```markdown +# Simulator sample media + +`manifest.json` is committed; the `.mp4` binaries are **not** (gitignored). They +are look-tuning samples, not shipped installation content. + +Populate them from the session-0008 POC artifacts: + + python simulator/setup_sample_media.py + +This copies `~/hef-poc/out/neutral.mp4` → `forest/base.mp4` and +`~/hef-poc/out/right_flow.mp4` → `forest/right4.mp4` (the real flow-stabilized +restyle), and generates placeholder strengths `forest/right1..3.mp4`. +``` + +- [ ] **Step 4: Run the setup script and verify media exists** + +Run: `python simulator/setup_sample_media.py && ls -la simulator/sample_media/forest/` +Expected: `base.mp4`, `right1.mp4`, `right2.mp4`, `right3.mp4`, `right4.mp4` present. + +- [ ] **Step 5: Commit (manifest, README, script, gitignore — NOT the mp4s)** + +```bash +git add simulator/sample_media/manifest.json simulator/sample_media/README.md simulator/setup_sample_media.py .gitignore +git commit -m "feat(simulator): sample-media manifest + POC setup/placeholder generator" +``` + +--- + +## Task 6: Browser preview (`simulator/static/`) + +**Files:** +- Modify: `simulator/static/index.html` (rewrite) +- Modify: `simulator/static/app.js` (rewrite) +- Modify: `simulator/static/style.css` (rewrite) + +- [ ] **Step 1: Rewrite `index.html`** + +```html + + + + + + HEF — Alteration Simulator + + + +

Human Experience Filter — Alteration Preview

+
+
+
+ + + +
+
+ +
+
+ Content dial + +
+ +
+ Experience knobs (0–4) + + + + +
+ +
+ Calibration + + +
+ +
+ RenderPlan readout + 
—
+
+
+
+ + + +``` + +- [ ] **Step 2: Rewrite `app.js`** + +```javascript +// Thin renderer: post controls+calibration -> RenderPlan; render grade, Right +// variant crossfade, and the live Left overlay. All math stays in Python. +const $ = (id) => document.getElementById(id); +const vid = $("vid"), overlay = $("overlay"), black = $("black"), readout = $("readout"); + +let clip = null; // active clip manifest entry +let currentVariant = -1; // last loaded Right strength + +async function loadClips() { + const data = await (await fetch("/api/clips")).json(); + clip = data.clips[0] || null; +} + +function mediaUrl(file) { return "/media/" + file; } + +function variantFile(strength) { + const v = clip.right_variants[String(strength)]; + return v ? v.file : clip.base_file; +} + +function applyGrade(tone) { + // Light: warm + brighten; Dark: cool + darken; 0: raw. + const warm = tone > 0 ? tone : 0, cool = tone < 0 ? -tone : 0; + const bright = 1 + 0.25 * tone, sat = 1 + 0.15 * Math.abs(tone); + vid.style.filter = + `brightness(${bright}) saturate(${sat}) ` + + `sepia(${(warm * 0.5).toFixed(3)}) hue-rotate(${(-cool * 200).toFixed(0)}deg)`; +} + +function loadVariant(strength) { + if (strength === currentVariant) return; + currentVariant = strength; + vid.style.opacity = "0"; + setTimeout(() => { + vid.src = mediaUrl(variantFile(strength)); + vid.play().catch(() => {}); + vid.style.opacity = "1"; + }, 150); +} + +function renderOverlay(level, intensity) { + overlay.innerHTML = ""; + if (!clip || level <= 0) { overlay.style.opacity = "0"; return; } + overlay.style.opacity = String(intensity); + const strings = (clip.strings && clip.strings.en) || {}; + for (const a of clip.annotations) { + if (a.min_level > level) continue; + const [x, y, w, h] = a.box.map((n) => n * 100); + const rect = document.createElementNS("http://www.w3.org/2000/svg", "rect"); + rect.setAttribute("x", x); rect.setAttribute("y", y); + rect.setAttribute("width", w); rect.setAttribute("height", h); + rect.setAttribute("class", "anno-box"); + overlay.appendChild(rect); + const text = document.createElementNS("http://www.w3.org/2000/svg", "text"); + text.setAttribute("x", x + 0.5); text.setAttribute("y", Math.max(y - 0.5, 2)); + text.setAttribute("class", "anno-label"); + text.textContent = strings[a.key] || a.key; + overlay.appendChild(text); + } +} + +function controls() { + return { + content: $("content").value, + left: +$("left").value, right: +$("right").value, + dark: +$("dark").value, light: +$("light").value, + volume: 2, brightness: 2, + }; +} + +function calibration() { + return { mood_gain: +$("mood_gain").value, overlay_gain: +$("overlay_gain").value, + right_variant_map: [0, 1, 2, 3, 4] }; +} + +let timer = null; +async function update() { + const resp = await fetch("/api/alteration", { + method: "POST", headers: { "content-type": "application/json" }, + body: JSON.stringify({ controls: controls(), calibration: calibration() }), + }); + if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; } + const data = await resp.json(); + readout.textContent = JSON.stringify(data, null, 2); + if (!data.content.video) { black.classList.remove("hidden"); return; } + black.classList.add("hidden"); + applyGrade(data.plan.grade.tone); + loadVariant(data.plan.restyle.variant); + renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity); +} + +function debounced() { clearTimeout(timer); timer = setTimeout(update, 80); } + +async function main() { + await loadClips(); + for (const id of ["content", "left", "right", "dark", "light", "mood_gain", "overlay_gain"]) { + $(id).addEventListener("input", debounced); + } + update(); +} +main(); +``` + +- [ ] **Step 3: Rewrite `style.css`** + +```css +* { box-sizing: border-box; } +body { margin: 0; font: 14px/1.4 system-ui, sans-serif; background: #111; color: #eee; } +header { padding: 0.6rem 1rem; background: #000; } +h1 { font-size: 1rem; margin: 0; font-weight: 600; } +main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; } +.stage { flex: 1 1 640px; } +.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000; + border-radius: 6px; overflow: hidden; } +#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; } +#overlay { position: absolute; inset: 0; width: 100%; height: 100%; + pointer-events: none; transition: opacity 0.2s ease; } +.anno-box { fill: none; stroke: #6cf; stroke-width: 0.4; vector-effect: non-scaling-stroke; } +.anno-label { fill: #6cf; font-size: 3px; font-family: monospace; } +.black { position: absolute; inset: 0; background: #000; } +.hidden { display: none; } +.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem; } +fieldset { border: 1px solid #333; border-radius: 6px; } +legend { color: #9af; padding: 0 0.4rem; } +label { display: block; margin: 0.4rem 0; } +input[type=range], select { width: 100%; } +#readout { background: #000; padding: 0.5rem; border-radius: 4px; font-size: 12px; + white-space: pre-wrap; max-height: 240px; overflow: auto; } +``` + +- [ ] **Step 4: Run the full simulator test suite (index test now passes)** + +Run: `python -m pytest tests/test_simulator_api.py -q` +Expected: PASS (all, including `test_index_is_served`). + +- [ ] **Step 5: Manual smoke (optional, needs media from Task 5)** + +Run: `python -m uvicorn simulator.app:app --port 8000` then open `http://localhost:8000`. +Expected: the base clip loops; Right slider crossfades variants; Dark/Light grade the footage live; Left slider reveals boxes + labels; "off" → black; the readout shows the engine numbers. + +- [ ] **Step 6: Commit** + +```bash +git add simulator/static/ +git commit -m "feat(simulator): alteration preview UI (grade + variant crossfade + Left overlay)" +``` + +--- + +## Task 7: Docs — update parent design, roadmap, user guide + +**Files:** +- Modify: `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` +- Modify: `docs/ROADMAP.md` +- Modify: `docs/USER_GUIDE.md` + +- [ ] **Step 1: Point the parent design at the reconciliation** + +In `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md`, near the §4.3 and §10 headings, add a note (one line each): +``` +> **Reconciled (2026-06-07):** Left HUD is a runtime overlay driven by an authored +> annotation track + per-language string tables; Right is a discrete pre-baked +> variant. See 2026-06-07-reconciled-simulator-alteration-slice-design.md §1. +``` + +- [ ] **Step 2: Update the roadmap slice status** + +In `docs/ROADMAP.md` §3 (sub-project 3 slices), mark the simulator-alteration slice as built: add a bullet: +``` +- [x] Simulator-first alteration preview — live Dark/Light grade, runtime Left + overlay (authored annotation track + string tables), discrete pre-baked Right + variants over one neutral clip, parameterized Calibration. (session 0009) +``` +Keep the deferred slices (Pi renderer, serial, endless-encoder + AI transitions, +real multi-strength re-bake) listed as not-started. + +- [ ] **Step 3: Rewrite the user-guide simulator section** + +In `docs/USER_GUIDE.md`, replace the "Playing with the simulator" section to describe the alteration preview: run `python simulator/setup_sample_media.py` once, then `uvicorn simulator.app:app`; turn the four knobs + content dial; tune the calibration sliders; read the RenderPlan readout. Remove references to the retired X-ray / selection view. + +- [ ] **Step 4: Verify the whole suite is green** + +Run: `python -m pytest -q` +Expected: PASS (no failures; previously-passing unrelated suites unaffected). + +- [ ] **Step 5: Commit** + +```bash +git add docs/ +git commit -m "docs: point parent design at reconciliation; roadmap + user guide for sim alteration" +``` + +--- + +## Self-Review + +**Spec coverage:** +- §1 decision (Left runtime overlay, Right discrete pre-baked, Dark/Light live) → Tasks 1 (engine), 6 (renderer), 5 (annotation track). ✓ +- §2 engine reconciliation (`Calibration`, `Restyle.variant`, `AnalyticalOverlay.level`, `state.py`) → Tasks 1, 2. ✓ +- §3 simulator (retire selection; `clips.py`; `/api/alteration` + `/api/clips`; browser preview) → Tasks 3, 4, 6. ✓ +- §4 sample clip + real Right variant + placeholder generator + Left annotation track → Task 5 (media/script), Task 5 manifest (annotations). ✓ +- §5 testing (player unit, sim API, removed endpoints, fixtures retired) → Tasks 1–4. ✓ +- §6 ships (incl. doc updates) → Task 7. ✓ + +**Placeholder scan:** No TBD/TODO; every code step shows complete code. ✓ + +**Type consistency:** `Restyle.variant` (int), `AnalyticalOverlay.level`/`.intensity`, `Calibration(mood_gain, overlay_gain, right_variant_map)`, `render_plan_to_dict` keys (`grade.tone`, `overlay.level/intensity`, `restyle.variant`, `is_identity`), `Clip.variant_file`/`to_dict`, `load_manifest`, `create_app(manifest_path=)` — used identically across Tasks 1, 3, 4, 6. ✓ + +**Note for the executor:** verify `player/content.py`'s `ContentResolution` field names (`audio_source`, `video`) before Task 4 Step 3; match them if different. `CONTENT_POSITIONS` is importable from `player/controls.py`.