diff --git a/.gitignore b/.gitignore index 2c62fc9..cdd216c 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,5 @@ __pycache__/ media/ .superpowers/ *.egg-info/ +# Simulator sample media (look-tuning only; populate via setup_sample_media.py) +simulator/sample_media/forest/*.mp4 diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 50efbc7..5667f11 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -123,6 +123,22 @@ longer *select* a pre-tagged clip; they drive an **alteration engine** (design Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-player-alteration-core.md). +**Slice 2 — simulator-first alteration preview ✅ Done.** Merged to `main` +(session 0009). Wires the alteration engine into the web simulator so the look is +tunable by eye before any hardware, and **reconciles** the unmerged session-0007 +design with the merged session-0008 design (the Left-HUD conflict): Left is a +**runtime overlay** (authored annotation track + per-language string tables), +Right is a **discrete pre-baked** flow-stabilized variant, Dark/Light a **live** +grade — superseding 0007's baked-HUD 5×5 grid. Engine: a parameterized +`Calibration` (tuned by eye, baked into `DEFAULT_CALIBRATION`), `Restyle.variant` +(discrete) replacing the continuous blend, and `AnalyticalOverlay.level`. Sim: +`/api/alteration` + `/api/clips` over `simulator/clips.py`, an alteration-preview +UI, and one neutral clip with a real flow-stabilized Right variant from the POC. +Design: +[`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md); +plan: +[`2026-06-07-reconciled-simulator-alteration-slice.md`](./superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md). + **Remaining slices (not started):** - **Runtime renderer** — drive the single panoramic projector via mpv/ffmpeg; @@ -134,7 +150,12 @@ Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-pl `Controls` stream). - **White-noise generation** — runtime white/pink noise for that content position. - **Offline v2v variant pipeline** — author the pre-baked Right restyle variants - (the only paid AI step, design §9) + the multilingual label/string tables + TTS. + via the local flow-stabilized SD pipeline (now ~free, not a paid API — see the + scales-library/right-axis design §1/§4); a real multi-strength flow-stabilized + re-bake per base clip + the multilingual label/string tables + TTS. +- **Scale-ring navigation** — the endless rotary encoder + short pre-baked AI + zoom/warp transitions between neutral "scales of nature" clips on a closed ring + (scales-library design §3); a new control + offline pipeline element. - **Catalog model changes** — audio *source* + "neutral base" vs "altered variant" flag (sub-project 2 territory, design §13). diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index 232e515..ad95237 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -287,11 +287,23 @@ Records point at files via `file_path`. Those files live on the player's drive, `file_path` values consistent with wherever you mount that drive on the machine that will eventually run the room. -## Playing with the simulator (curator's X-ray) +## Playing with the simulator (alteration preview) The simulator is a web stand-in for the installation's control panel. It runs the -real `hef.selection` code against a synthetic catalog so you can feel whether the -dials surface fitting pieces before any hardware exists. +real `player.alteration` engine and **alters** a neutral base clip toward the knob +state in the browser, so you can tune the *look* of the filter before any hardware +exists. (The earlier selection-era "curator's X-ray" view was retired when the +piece moved from *selecting* clips to *altering* them.) + +**One-time setup — populate the sample footage** (look-tuning only; not shipped +content): + + python simulator/setup_sample_media.py + +This copies the session-0008 POC artifacts (`~/hef-poc/out/`) into +`simulator/sample_media/forest/` — the neutral base clip and the real +flow-stabilized Right restyle — and generates placeholder intermediate Right +strengths. The `.mp4` binaries are gitignored. **Run it (Docker):** @@ -304,11 +316,22 @@ then open http://localhost:8000. pip install -e ".[sim]" make sim-local -**What you see:** the five real dials (mode + Left/Right/Dark/Light), the model -knobs (brain/mood weights, pool size, approved-only), and the X-ray — the picked -piece, the ranked candidate pool with distances, and brain/mood coordinate maps -showing where your knob point and the candidates sit. +**What you see and can do:** -By default it loads a generated fixture catalog. To point it at a real catalog, -set `HEF_SIM_CATALOG=catalog/library.jsonl` (used automatically when that file is -non-empty). +- **Content dial** — picks audio/video channel; "off" and audio-only positions go + to black walls. +- **Four experience knobs (0–4):** + - **Dark / Light** — a live runtime color grade (cool/dark ↔ warm/bright; equal + or zero = the raw footage). + - **Right (dreamlike)** — selects a discrete pre-baked, flow-stabilized restyle + variant and crossfades to it (strength 0 = raw base). + - **Left (analytical)** — a live overlay: labelled boxes from the clip's authored + annotation track, with more annotations appearing at higher levels. Text is + shaped live (the simulator analogue of the Pi's Pango/HarfBuzz path). +- **Calibration sliders** — adjust the grade/overlay gain curves live; once a look + is liked, bake the values into `DEFAULT_CALIBRATION` in `player/alteration.py`. +- **RenderPlan readout** — always shows the exact numbers the engine produced (the + project's honesty "X-ray," now over the alteration model). + +The base clips, Right variants, Left annotation track, and string tables come from +`simulator/sample_media/manifest.json`. diff --git a/docs/superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md b/docs/superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md new file mode 100644 index 0000000..4166be1 --- /dev/null +++ b/docs/superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md @@ -0,0 +1,1208 @@ +# Reconciled Simulator-First Alteration Slice — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire the alteration engine into the simulator — live Dark/Light grade, a live Left analytical overlay (authored annotation track + runtime-shaped text), and discrete pre-baked flow-stabilized Right variants over ONE neutral clip — so the look can be tuned by eye before any hardware. + +**Architecture:** Python stays the single source of truth for the alteration math (`player/alteration.py`); the browser only renders. The engine model changes are surgical: a continuous `Restyle.blend` becomes a discrete `Restyle.variant` (selecting a pre-baked clip), `AnalyticalOverlay` gains a Left `level`, and a new frozen `Calibration` parameterizes the knob→strength curves (default behavior-preserving). The simulator retires its selection-era surface (`/api/select`, X-ray, `fixtures.py`) for an alteration surface (`/api/alteration`, `/api/clips`, `clips.py`) reading a base-clip + variant + annotation manifest. + +**Tech Stack:** Python 3.13, FastAPI + pydantic, pytest, frozen dataclasses; vanilla JS + CSS filters / SVG for the browser; ffmpeg (via `imageio-ffmpeg` or system `ffmpeg`) for placeholder media. + +**Spec:** `docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md` + +--- + +## File Structure + +**Engine (modify):** +- `player/alteration.py` — add `Calibration` + `DEFAULT_CALIBRATION`; `Restyle.blend: float` → `Restyle.variant: int`; `AnalyticalOverlay` gains `level: int`; `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`; a `render_plan_to_dict` serializer for the API. +- `player/state.py` — no logic change (compares whole `Restyle` objects); docstring touch only. + +**Simulator (create/rewrite):** +- `simulator/clips.py` (create) — load a base-clip + variant + annotation manifest; replaces `simulator/fixtures.py` (delete). +- `simulator/app.py` (rewrite) — `POST /api/alteration`, `GET /api/clips`; remove `/api/select`, `/api/catalog/meta`. +- `simulator/static/{index.html,app.js,style.css}` (rewrite) — alteration preview. +- `simulator/sample_media/manifest.json` (create, committed) + `simulator/sample_media/README.md`; media files themselves gitignored. +- `simulator/setup_sample_media.py` (create) — copy POC clips + generate placeholder Right strengths. + +**Tests (create/rewrite):** +- `tests/test_player_alteration.py` (rewrite for new model). +- `tests/test_player_state.py` (update `.blend`→`.variant` references). +- `tests/test_clips.py` (create); `tests/test_fixtures.py` (delete). +- `tests/test_simulator_api.py` (rewrite for new endpoints). + +**Docs:** +- `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` — §4.3/§10 pointer to the reconciliation. +- `docs/ROADMAP.md` — §3 slice status. +- `docs/USER_GUIDE.md` — "Playing with the simulator" rewrite. + +--- + +## Task 1: Engine — `Calibration`, discrete `Restyle.variant`, overlay `level` + +**Files:** +- Modify: `player/alteration.py` +- Test: `tests/test_player_alteration.py` (rewrite) + +- [ ] **Step 1: Rewrite the failing tests for the new model** + +Replace the entire contents of `tests/test_player_alteration.py`: + +```python +import pytest + +from hef.selection import Coordinate +from player.alteration import ( + DEFAULT_CALIBRATION, + AnalyticalOverlay, + Calibration, + ColorGrade, + RenderPlan, + Restyle, + plan_alteration, + render_plan_to_dict, +) + + +def _coord(left=0, right=0, dark=0, light=0): + return Coordinate(left=left, right=right, dark=dark, light=light) + + +def test_all_zero_knobs_is_the_unaltered_base(): + plan = plan_alteration(_coord()) + assert plan.is_identity + assert plan.overlay.level == 0 + assert plan.overlay.intensity == 0.0 + assert plan.restyle.variant == 0 + assert plan.grade.tone == 0.0 + assert plan.grade.is_identity + + +def test_left_drives_the_analytical_overlay_only(): + plan = plan_alteration(_coord(left=4)) + assert plan.overlay.level == 4 + assert plan.overlay.intensity == 1.0 + assert plan.restyle.variant == 0 # Left does not touch the substrate + assert plan.grade.tone == 0.0 + + +def test_right_selects_a_discrete_restyle_variant_only(): + plan = plan_alteration(_coord(right=2)) + assert plan.restyle.variant == 2 + assert plan.overlay.level == 0 # Right does not add overlay + + +def test_left_and_right_stack_not_cancel(): + # design §4.2: whole-brain corner = dreamlike substrate WITH labels on top + plan = plan_alteration(_coord(left=4, right=4)) + assert plan.overlay.level == 4 + assert plan.restyle.variant == 4 + + +def test_light_pole_grades_warm_positive_tone(): + plan = plan_alteration(_coord(light=4)) + assert plan.grade.tone == 1.0 + assert not plan.grade.is_identity + + +def test_dark_pole_grades_cool_negative_tone(): + assert plan_alteration(_coord(dark=4)).grade.tone == -1.0 + + +def test_equal_dark_and_light_is_identity_grade(): + # design §5: the mood center is the raw, ungraded footage + assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity + assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity + + +def test_dark_minus_light_sets_intermediate_tone(): + assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5) + assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5) + + +def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay(): + plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0)) + assert plan.overlay.level == 4 + assert plan.restyle.variant == 2 + assert plan.grade.tone == -1.0 + assert not plan.is_identity + + +def test_default_calibration_is_behavior_preserving(): + # DEFAULT_CALIBRATION must reproduce the original three helpers exactly. + for left in range(5): + assert plan_alteration(_coord(left=left)).overlay.intensity == pytest.approx(left / 4) + for right in range(5): + assert plan_alteration(_coord(right=right)).restyle.variant == right + for dark in range(5): + for light in range(5): + expected = (light - dark) / 4 + assert plan_alteration(_coord(dark=dark, light=light)).grade.tone == pytest.approx(expected) + + +def test_custom_calibration_scales_mood_and_overlay(): + cal = Calibration(mood_gain=0.5, overlay_gain=0.5, right_variant_map=(0, 0, 1, 1, 2)) + assert plan_alteration(_coord(light=4), cal).grade.tone == pytest.approx(0.5) + assert plan_alteration(_coord(left=4), cal).overlay.intensity == pytest.approx(0.5) + assert plan_alteration(_coord(right=3), cal).restyle.variant == 1 + + +def test_calibration_gain_is_clamped_to_unit_range(): + cal = Calibration(mood_gain=10.0, overlay_gain=10.0) + assert plan_alteration(_coord(light=4), cal).grade.tone == 1.0 # clamped, not 10 + assert plan_alteration(_coord(left=4), cal).overlay.intensity == 1.0 + + +def test_render_plan_to_dict_round_trips_the_numbers(): + d = render_plan_to_dict(plan_alteration(_coord(left=4, right=2, dark=4, light=0))) + assert d == { + "grade": {"tone": -1.0}, + "overlay": {"level": 4, "intensity": 1.0}, + "restyle": {"variant": 2}, + "is_identity": False, + } + + +def test_render_plan_is_frozen(): + plan = plan_alteration(_coord()) + with pytest.raises(Exception): + plan.grade.tone = 0.5 # type: ignore[misc] +``` + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: `python -m pytest tests/test_player_alteration.py -q` +Expected: FAIL — `ImportError` for `Calibration` / `DEFAULT_CALIBRATION` / `render_plan_to_dict` (and `Restyle.variant` / `overlay.level` attribute errors). + +- [ ] **Step 3: Rewrite `player/alteration.py` for the new model** + +Replace the entire contents of `player/alteration.py`: + +```python +"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5). + +Reconciled slice (2026-06-07): the Right axis is a DISCRETE selection of a +pre-baked, flow-stabilized restyle variant (not a continuous blend), the Left +axis carries its knob LEVEL so a runtime annotation track can pick which labels +show, and a frozen `Calibration` parameterizes the knob->strength curves so they +can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION. + +Layers compose per §4.2: + - Substrate: ColorGrade (Dark/Light mood, center = identity §5) + a pre-baked + Right restyle variant. + - Overlay: AnalyticalOverlay (Left), composited on top at runtime. +Left and Right stack (different layers); Dark/Light are the two poles of one +mood grade. See docs/superpowers/specs/2026-06-07-reconciled-simulator- +alteration-slice-design.md. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field + +from hef.selection import Coordinate + +KNOB_MAX = 4 # knob full-scale (0..4) + + +def _clamp(x: float, lo: float, hi: float) -> float: + return max(lo, min(hi, x)) + + +@dataclass(frozen=True) +class Calibration: + """Tunable knob->strength curves (settled by eye in the sim, then baked). + + - mood_gain: scales the signed Dark/Light tone (result clamped to [-1, 1]). + - overlay_gain: scales the Left overlay intensity (clamped to [0, 1]). + - right_variant_map: knob value (0..4) -> pre-baked Right variant index. + """ + + mood_gain: float = 1.0 + overlay_gain: float = 1.0 + right_variant_map: tuple = (0, 1, 2, 3, 4) + + +DEFAULT_CALIBRATION = Calibration() + + +@dataclass(frozen=True) +class ColorGrade: + """Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light), + <0 cool blue->black (dark), 0 = identity (raw, ungraded).""" + + tone: float + + @property + def is_identity(self) -> bool: + return self.tone == 0.0 + + +@dataclass(frozen=True) +class AnalyticalOverlay: + """Left axis (§4.1): analytical HUD/annotation, composited on top at runtime. + `level` is the Left knob (0..4); a runtime annotation track uses it to pick + which labels appear. `intensity` 0..1 is the overlay opacity/strength.""" + + level: int + intensity: float + + +@dataclass(frozen=True) +class Restyle: + """Right axis (§4.1): selects a pre-baked, flow-stabilized restyle variant. + `variant` is a discrete index (0 = raw base, no restyle).""" + + variant: int + + +@dataclass(frozen=True) +class RenderPlan: + """The full layered alteration for one knob vector (§4.2).""" + + grade: ColorGrade + overlay: AnalyticalOverlay + restyle: Restyle + + @property + def is_identity(self) -> bool: + """True when the plan leaves the neutral base un-altered.""" + return ( + self.grade.is_identity + and self.overlay.level == 0 + and self.restyle.variant == 0 + ) + + +def _overlay_intensity(left: int, cal: Calibration) -> float: + return _clamp(cal.overlay_gain * left / KNOB_MAX, 0.0, 1.0) + + +def _right_variant(right: int, cal: Calibration) -> int: + return cal.right_variant_map[right] + + +def _mood_tone(dark: int, light: int, cal: Calibration) -> float: + return _clamp(cal.mood_gain * (light - dark) / KNOB_MAX, -1.0, 1.0) + + +def plan_alteration( + coord: Coordinate, calibration: Calibration = DEFAULT_CALIBRATION +) -> RenderPlan: + """Map a knob vector to its layered RenderPlan (design §4).""" + return RenderPlan( + grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light, calibration)), + overlay=AnalyticalOverlay( + level=coord.left, + intensity=_overlay_intensity(coord.left, calibration), + ), + restyle=Restyle(variant=_right_variant(coord.right, calibration)), + ) + + +def render_plan_to_dict(plan: RenderPlan) -> dict: + """JSON-serializable form for the simulator API.""" + return { + "grade": {"tone": plan.grade.tone}, + "overlay": {"level": plan.overlay.level, "intensity": plan.overlay.intensity}, + "restyle": {"variant": plan.restyle.variant}, + "is_identity": plan.is_identity, + } +``` + +Note: `field` import is unused — drop it; only `dataclass` is needed. (Final import line: `from dataclasses import dataclass`.) + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: `python -m pytest tests/test_player_alteration.py -q` +Expected: PASS (all). + +- [ ] **Step 5: Commit** + +```bash +git add player/alteration.py tests/test_player_alteration.py +git commit -m "feat(player): discrete Right variant + Left level + Calibration (slice design §2)" +``` + +--- + +## Task 2: `state.py` reconcile (field rename in tests; no logic change) + +**Files:** +- Modify: `player/state.py` (docstring only) +- Test: `tests/test_player_state.py` + +- [ ] **Step 1: Update the failing test references** + +In `tests/test_player_state.py`, change the two `.blend` assertions to `.variant` and the overlay assertion to `.level`: + +Replace `test_overlay_change_is_a_live_update` body's last assertion line: +```python + assert t.playback.plan.overlay.level == 4 +``` +Replace `test_restyle_change_crossfades_the_substrate` body's last assertion line: +```python + assert t.playback.plan.restyle.variant == 4 +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `python -m pytest tests/test_player_state.py -q` +Expected: FAIL — `AttributeError: 'Restyle' object has no attribute 'blend'` is already gone after Task 1, so the failure is now the *test's own* stale references resolved by Step 1; re-running confirms. If Step 1 is applied, this should already pass — run to confirm the suite is green. + +- [ ] **Step 3: Touch the `state.py` docstring** + +In `player/state.py`, update the module docstring sentence mentioning the Right variant to read: +``` +swapping the clip or the pre-baked Right restyle variant (a discrete index) +needs a CROSSFADE +``` +(No code change — `_classify` already compares whole `Restyle` objects via `nxt.plan.restyle != prev.plan.restyle`.) + +- [ ] **Step 4: Run to verify pass** + +Run: `python -m pytest tests/test_player_state.py -q` +Expected: PASS (all). + +- [ ] **Step 5: Commit** + +```bash +git add player/state.py tests/test_player_state.py +git commit -m "refactor(player): state.py tests track discrete Restyle.variant" +``` + +--- + +## Task 3: `simulator/clips.py` — the base-clip + variant + annotation manifest + +**Files:** +- Create: `simulator/clips.py` +- Create: `tests/test_clips.py` +- Delete: `simulator/fixtures.py`, `tests/test_fixtures.py` + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_clips.py`: + +```python +import json + +import pytest + +from simulator.clips import Clip, load_manifest + + +def _manifest_dict(): + return { + "clips": [ + { + "id": "forest", + "title": "Yosemite Falls (neutral)", + "base_file": "forest/base.mp4", + "license": "poc-sample", + "source": "hef-poc", + "right_variants": { + "4": {"file": "forest/right4.mp4", "model": "sd-turbo+flow"}, + "1": {"file": "forest/right1.mp4"}, + }, + "annotations": [ + {"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1}, + {"key": "detected.conifer", "box": [0.6, 0.1, 0.2, 0.2], "min_level": 3}, + ], + "strings": {"en": {"detected.water": "flowing water", "detected.conifer": "conifer"}}, + } + ] + } + + +def test_load_manifest_parses_clips(tmp_path): + p = tmp_path / "manifest.json" + p.write_text(json.dumps(_manifest_dict())) + clips = load_manifest(p) + assert len(clips) == 1 + c = clips[0] + assert isinstance(c, Clip) + assert c.id == "forest" + assert c.base_file == "forest/base.mp4" + + +def test_clip_lists_variant_files_by_strength(tmp_path): + p = tmp_path / "manifest.json" + p.write_text(json.dumps(_manifest_dict())) + c = load_manifest(p)[0] + # variant 0 is always the raw base; authored strengths come from the manifest + assert c.variant_file(0) == "forest/base.mp4" + assert c.variant_file(4) == "forest/right4.mp4" + assert c.variant_file(1) == "forest/right1.mp4" + # an unauthored strength falls back to the raw base + assert c.variant_file(2) == "forest/base.mp4" + + +def test_clip_serializes_to_dict_for_the_api(tmp_path): + p = tmp_path / "manifest.json" + p.write_text(json.dumps(_manifest_dict())) + d = load_manifest(p)[0].to_dict() + assert d["id"] == "forest" + assert d["base_file"] == "forest/base.mp4" + assert d["annotations"][0]["key"] == "detected.water" + assert d["strings"]["en"]["detected.water"] == "flowing water" + # variant map is exposed keyed by strength string, including 0 -> base + assert d["right_variants"]["0"]["file"] == "forest/base.mp4" + assert d["right_variants"]["4"]["file"] == "forest/right4.mp4" + + +def test_missing_manifest_raises(tmp_path): + with pytest.raises(FileNotFoundError): + load_manifest(tmp_path / "nope.json") +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `python -m pytest tests/test_clips.py -q` +Expected: FAIL — `ModuleNotFoundError: No module named 'simulator.clips'`. + +- [ ] **Step 3: Create `simulator/clips.py`** + +```python +"""The base-clip + variant + annotation manifest the simulator renders. + +Replaces simulator/fixtures.py (the selection-era synthetic catalog). Each base +clip carries: the raw base file, a map of pre-baked Right-strength variant files +(strength 0 is always the raw base), an authored Left annotation track (box + +label key + the minimum Left level at which it appears), and per-language string +tables. See the reconciled-simulator-alteration-slice design §3.2. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from pathlib import Path +from typing import Any + + +@dataclass(frozen=True) +class Clip: + id: str + title: str + base_file: str + license: str + source: str + right_variants: dict # {"1": {"file": ...}, "4": {...}} (no "0") + annotations: list # [{"key", "box":[x,y,w,h], "min_level"}, ...] + strings: dict # {"en": {key: text}} + + def variant_file(self, strength: int) -> str: + """The video file for a Right strength; 0 and any unauthored strength + fall back to the raw base file.""" + entry = self.right_variants.get(str(strength)) + return entry["file"] if entry else self.base_file + + def to_dict(self) -> dict: + variants = {"0": {"file": self.base_file, "raw": True}} + for k, v in self.right_variants.items(): + variants[k] = v + return { + "id": self.id, + "title": self.title, + "base_file": self.base_file, + "license": self.license, + "source": self.source, + "right_variants": variants, + "annotations": self.annotations, + "strings": self.strings, + } + + +def _clip_from_dict(d: dict[str, Any]) -> Clip: + return Clip( + id=d["id"], + title=d["title"], + base_file=d["base_file"], + license=d.get("license", ""), + source=d.get("source", ""), + right_variants=d.get("right_variants", {}), + annotations=d.get("annotations", []), + strings=d.get("strings", {}), + ) + + +def load_manifest(path: str | Path) -> list[Clip]: + """Load the base-clip manifest. Raises FileNotFoundError if missing.""" + path = Path(path) + data = json.loads(path.read_text()) + return [_clip_from_dict(c) for c in data["clips"]] +``` + +- [ ] **Step 4: Run to verify it passes** + +Run: `python -m pytest tests/test_clips.py -q` +Expected: PASS (all). + +- [ ] **Step 5: Delete the retired fixtures** + +```bash +git rm simulator/fixtures.py tests/test_fixtures.py +``` + +- [ ] **Step 6: Commit** + +```bash +git add simulator/clips.py tests/test_clips.py +git commit -m "feat(simulator): clips.py manifest model; retire selection fixtures" +``` + +--- + +## Task 4: `simulator/app.py` — `/api/alteration` + `/api/clips` + +**Files:** +- Modify: `simulator/app.py` (rewrite) +- Test: `tests/test_simulator_api.py` (rewrite) + +- [ ] **Step 1: Rewrite the failing tests** + +Replace the entire contents of `tests/test_simulator_api.py`: + +```python +import json + +import pytest +from fastapi.testclient import TestClient + +from simulator.app import create_app + + +@pytest.fixture +def manifest_path(tmp_path): + p = tmp_path / "manifest.json" + p.write_text(json.dumps({ + "clips": [{ + "id": "forest", + "title": "neutral forest", + "base_file": "forest/base.mp4", + "license": "poc", "source": "hef-poc", + "right_variants": {"4": {"file": "forest/right4.mp4"}}, + "annotations": [{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1}], + "strings": {"en": {"detected.water": "flowing water"}}, + }] + })) + return p + + +@pytest.fixture +def client(manifest_path): + return TestClient(create_app(manifest_path=manifest_path)) + + +def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2): + return dict(content=content, left=left, right=right, dark=dark, + light=light, volume=volume, brightness=brightness) + + +def test_alteration_returns_the_engine_plan(client): + resp = client.post("/api/alteration", json={"controls": _controls(left=4, right=2, dark=4)}) + assert resp.status_code == 200 + data = resp.json() + assert data["plan"]["overlay"]["level"] == 4 + assert data["plan"]["restyle"]["variant"] == 2 + assert data["plan"]["grade"]["tone"] == -1.0 + assert data["content"]["video"] is True + + +def test_alteration_honors_off_as_black(client): + resp = client.post("/api/alteration", json={"controls": _controls(content="off")}) + data = resp.json() + assert data["content"]["video"] is False + + +def test_alteration_accepts_calibration(client): + body = {"controls": _controls(light=4), + "calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "right_variant_map": [0, 1, 2, 3, 4]}} + resp = client.post("/api/alteration", json=body) + assert resp.json()["plan"]["grade"]["tone"] == 0.5 + + +def test_alteration_rejects_out_of_range_knob(client): + resp = client.post("/api/alteration", json={"controls": _controls(left=7)}) + assert resp.status_code == 422 + + +def test_alteration_rejects_bad_content(client): + resp = client.post("/api/alteration", json={"controls": _controls(content="banana")}) + assert resp.status_code == 422 + + +def test_clips_returns_the_manifest(client): + resp = client.get("/api/clips") + assert resp.status_code == 200 + data = resp.json() + assert data["clips"][0]["id"] == "forest" + assert data["clips"][0]["right_variants"]["0"]["file"] == "forest/base.mp4" + assert data["clips"][0]["annotations"][0]["key"] == "detected.water" + + +def test_retired_selection_endpoints_are_gone(client): + assert client.post("/api/select", json={}).status_code == 404 + assert client.get("/api/catalog/meta").status_code == 404 + + +def test_index_is_served(): + client = TestClient(create_app()) + resp = client.get("/") + assert resp.status_code == 200 + assert "text/html" in resp.headers["content-type"] + assert "Alteration" in resp.text +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `python -m pytest tests/test_simulator_api.py -q` +Expected: FAIL — `create_app` has no `manifest_path` kwarg; `/api/alteration` 404; old selection imports break. + +- [ ] **Step 3: Rewrite `simulator/app.py`** + +```python +"""FastAPI service: controls -> the real alteration engine -> a RenderPlan. + +The simulator's alteration surface (reconciled slice). It calls the canonical +player.alteration.plan_alteration; the browser only renders. The selection-era +endpoints (/api/select, /api/catalog/meta) and the X-ray are retired. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Optional + +from fastapi import FastAPI +from fastapi.staticfiles import StaticFiles +from pydantic import BaseModel, Field + +from hef.selection import Coordinate +from player.alteration import ( + DEFAULT_CALIBRATION, + Calibration, + plan_alteration, + render_plan_to_dict, +) +from player.content import resolve_content +from player.controls import CONTENT_POSITIONS +from simulator.clips import load_manifest + +STATIC_DIR = Path(__file__).parent / "static" +DEFAULT_MANIFEST = Path(__file__).parent / "sample_media" / "manifest.json" +MEDIA_DIR = Path(__file__).parent / "sample_media" + + +class ControlsModel(BaseModel): + content: str + left: int = Field(ge=0, le=4) + right: int = Field(ge=0, le=4) + dark: int = Field(ge=0, le=4) + light: int = Field(ge=0, le=4) + volume: int = Field(ge=0, le=4) + brightness: int = Field(ge=0, le=4) + + +class CalibrationModel(BaseModel): + mood_gain: float = 1.0 + overlay_gain: float = 1.0 + right_variant_map: list[int] = [0, 1, 2, 3, 4] + + +class AlterationRequest(BaseModel): + controls: ControlsModel + calibration: Optional[CalibrationModel] = None + + +def _load_clips(manifest_path: Optional[Path]): + path = Path(manifest_path) if manifest_path else DEFAULT_MANIFEST + if path.exists(): + return load_manifest(path) + return [] + + +def create_app(manifest_path: Optional[Path] = None) -> FastAPI: + app = FastAPI(title="HEF Alteration Simulator") + app.state.clips = _load_clips(manifest_path) + + @app.post("/api/alteration") + def api_alteration(req: AlterationRequest): + c = req.controls + if c.content not in CONTENT_POSITIONS: + # mirror Controls validation as a 422 + from fastapi import HTTPException + raise HTTPException(status_code=422, detail=f"invalid content {c.content!r}") + coord = Coordinate(c.left, c.right, c.dark, c.light) + cal = ( + Calibration( + mood_gain=req.calibration.mood_gain, + overlay_gain=req.calibration.overlay_gain, + right_variant_map=tuple(req.calibration.right_variant_map), + ) + if req.calibration + else DEFAULT_CALIBRATION + ) + plan = plan_alteration(coord, cal) + content = resolve_content(c.content) + return { + "plan": render_plan_to_dict(plan), + "content": {"audio_source": content.audio_source, "video": content.video}, + } + + @app.get("/api/clips") + def api_clips(): + return {"clips": [c.to_dict() for c in app.state.clips]} + + if MEDIA_DIR.exists(): + app.mount("/media", StaticFiles(directory=MEDIA_DIR), name="media") + if STATIC_DIR.exists(): + app.mount("/", StaticFiles(directory=STATIC_DIR, html=True), name="static") + + return app + + +app = create_app() +``` + +Note: confirm `player/content.py`'s `ContentResolution` exposes `audio_source` and `video` (it does — `state.py` reads both). If the attribute is named differently, match it. + +- [ ] **Step 4: Run to verify it passes** + +Run: `python -m pytest tests/test_simulator_api.py -q` +Expected: PASS. The `test_index_is_served` test needs the rewritten `index.html` (Task 6) containing "Alteration" — if Task 6 is not yet done, that one test fails; run the rest with `-k "not index"` until Task 6. + +- [ ] **Step 5: Commit** + +```bash +git add simulator/app.py tests/test_simulator_api.py +git commit -m "feat(simulator): /api/alteration + /api/clips; retire selection endpoints" +``` + +--- + +## Task 5: Sample media + placeholder Right strengths + +**Files:** +- Create: `simulator/sample_media/manifest.json` (committed) +- Create: `simulator/sample_media/README.md` (committed) +- Create: `simulator/setup_sample_media.py` (committed) +- Modify: `.gitignore` (ignore the media binaries) + +- [ ] **Step 1: Author the manifest** + +Create `simulator/sample_media/manifest.json`: + +```json +{ + "clips": [ + { + "id": "forest", + "title": "Yosemite Falls (neutral base, POC sample)", + "base_file": "forest/base.mp4", + "license": "poc-sample (look-tuning only; not shipped content)", + "source": "hef-poc/out/neutral.mp4", + "right_variants": { + "1": {"file": "forest/right1.mp4", "model": "placeholder"}, + "2": {"file": "forest/right2.mp4", "model": "placeholder"}, + "3": {"file": "forest/right3.mp4", "model": "placeholder"}, + "4": {"file": "forest/right4.mp4", "model": "sd-turbo+farneback-flow"} + }, + "annotations": [ + {"key": "detected.water", "box": [0.30, 0.10, 0.18, 0.70], "min_level": 1}, + {"key": "detected.rock_face", "box": [0.05, 0.30, 0.20, 0.55], "min_level": 2}, + {"key": "detected.conifer", "box": [0.70, 0.20, 0.22, 0.45], "min_level": 3}, + {"key": "measure.flow_rate", "box": [0.34, 0.55, 0.14, 0.08], "min_level": 4} + ], + "strings": { + "en": { + "detected.water": "flowing water", + "detected.rock_face": "granite face", + "detected.conifer": "conifer stand", + "measure.flow_rate": "~2.1 m³/s" + } + } + } + ] +} +``` + +- [ ] **Step 2: Write the setup script** + +Create `simulator/setup_sample_media.py`: + +```python +"""Populate simulator/sample_media/forest/ from the session-0008 POC artifacts. + +Copies the real neutral base + the real flow-stabilized Right restyle out of +~/hef-poc/out/, and generates placeholder intermediate Right strengths (1..3) by +blending the base toward the real restyle with ffmpeg. The media binaries are +gitignored; only the manifest is committed. Sample footage is for look-tuning +only, not shipped content. + +Usage: python simulator/setup_sample_media.py +Requires: ffmpeg on PATH (or `pip install imageio-ffmpeg`), and ~/hef-poc/out/. +""" + +from __future__ import annotations + +import shutil +import subprocess +from pathlib import Path + +POC = Path.home() / "hef-poc" / "out" +DEST = Path(__file__).parent / "sample_media" / "forest" + + +def _ffmpeg() -> str: + if shutil.which("ffmpeg"): + return "ffmpeg" + import imageio_ffmpeg + return imageio_ffmpeg.get_ffmpeg_exe() + + +def main() -> None: + DEST.mkdir(parents=True, exist_ok=True) + base = DEST / "base.mp4" + right4 = DEST / "right4.mp4" + shutil.copyfile(POC / "neutral.mp4", base) + shutil.copyfile(POC / "right_flow.mp4", right4) + ff = _ffmpeg() + # Placeholder strengths 1..3: opacity-blend base toward the real restyle. + for strength, alpha in ((1, 0.25), (2, 0.5), (3, 0.75)): + out = DEST / f"right{strength}.mp4" + subprocess.run( + [ff, "-y", "-i", str(base), "-i", str(right4), + "-filter_complex", + f"[1:v]format=yuva444p,colorchannelmixer=aa={alpha}[top];" + f"[0:v][top]overlay=shortest=1[v]", + "-map", "[v]", "-an", str(out)], + check=True, + ) + print(f"generated {out.name} (alpha {alpha})") + print(f"sample media ready in {DEST}") + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 3: Gitignore the media binaries, document them** + +Append to `.gitignore`: +``` +# Simulator sample media (look-tuning only; populate via setup_sample_media.py) +simulator/sample_media/forest/*.mp4 +``` + +Create `simulator/sample_media/README.md`: +```markdown +# Simulator sample media + +`manifest.json` is committed; the `.mp4` binaries are **not** (gitignored). They +are look-tuning samples, not shipped installation content. + +Populate them from the session-0008 POC artifacts: + + python simulator/setup_sample_media.py + +This copies `~/hef-poc/out/neutral.mp4` → `forest/base.mp4` and +`~/hef-poc/out/right_flow.mp4` → `forest/right4.mp4` (the real flow-stabilized +restyle), and generates placeholder strengths `forest/right1..3.mp4`. +``` + +- [ ] **Step 4: Run the setup script and verify media exists** + +Run: `python simulator/setup_sample_media.py && ls -la simulator/sample_media/forest/` +Expected: `base.mp4`, `right1.mp4`, `right2.mp4`, `right3.mp4`, `right4.mp4` present. + +- [ ] **Step 5: Commit (manifest, README, script, gitignore — NOT the mp4s)** + +```bash +git add simulator/sample_media/manifest.json simulator/sample_media/README.md simulator/setup_sample_media.py .gitignore +git commit -m "feat(simulator): sample-media manifest + POC setup/placeholder generator" +``` + +--- + +## Task 6: Browser preview (`simulator/static/`) + +**Files:** +- Modify: `simulator/static/index.html` (rewrite) +- Modify: `simulator/static/app.js` (rewrite) +- Modify: `simulator/static/style.css` (rewrite) + +- [ ] **Step 1: Rewrite `index.html`** + +```html + + + + + + HEF — Alteration Simulator + + + +

Human Experience Filter — Alteration Preview

+
+
+
+ + + +
+
+ +
+
+ Content dial + +
+ +
+ Experience knobs (0–4) + + + + +
+ +
+ Calibration + + +
+ +
+ RenderPlan readout + 
—
+
+
+
+ + + +``` + +- [ ] **Step 2: Rewrite `app.js`** + +```javascript +// Thin renderer: post controls+calibration -> RenderPlan; render grade, Right +// variant crossfade, and the live Left overlay. All math stays in Python. +const $ = (id) => document.getElementById(id); +const vid = $("vid"), overlay = $("overlay"), black = $("black"), readout = $("readout"); + +let clip = null; // active clip manifest entry +let currentVariant = -1; // last loaded Right strength + +async function loadClips() { + const data = await (await fetch("/api/clips")).json(); + clip = data.clips[0] || null; +} + +function mediaUrl(file) { return "/media/" + file; } + +function variantFile(strength) { + const v = clip.right_variants[String(strength)]; + return v ? v.file : clip.base_file; +} + +function applyGrade(tone) { + // Light: warm + brighten; Dark: cool + darken; 0: raw. + const warm = tone > 0 ? tone : 0, cool = tone < 0 ? -tone : 0; + const bright = 1 + 0.25 * tone, sat = 1 + 0.15 * Math.abs(tone); + vid.style.filter = + `brightness(${bright}) saturate(${sat}) ` + + `sepia(${(warm * 0.5).toFixed(3)}) hue-rotate(${(-cool * 200).toFixed(0)}deg)`; +} + +function loadVariant(strength) { + if (strength === currentVariant) return; + currentVariant = strength; + vid.style.opacity = "0"; + setTimeout(() => { + vid.src = mediaUrl(variantFile(strength)); + vid.play().catch(() => {}); + vid.style.opacity = "1"; + }, 150); +} + +function renderOverlay(level, intensity) { + overlay.innerHTML = ""; + if (!clip || level <= 0) { overlay.style.opacity = "0"; return; } + overlay.style.opacity = String(intensity); + const strings = (clip.strings && clip.strings.en) || {}; + for (const a of clip.annotations) { + if (a.min_level > level) continue; + const [x, y, w, h] = a.box.map((n) => n * 100); + const rect = document.createElementNS("http://www.w3.org/2000/svg", "rect"); + rect.setAttribute("x", x); rect.setAttribute("y", y); + rect.setAttribute("width", w); rect.setAttribute("height", h); + rect.setAttribute("class", "anno-box"); + overlay.appendChild(rect); + const text = document.createElementNS("http://www.w3.org/2000/svg", "text"); + text.setAttribute("x", x + 0.5); text.setAttribute("y", Math.max(y - 0.5, 2)); + text.setAttribute("class", "anno-label"); + text.textContent = strings[a.key] || a.key; + overlay.appendChild(text); + } +} + +function controls() { + return { + content: $("content").value, + left: +$("left").value, right: +$("right").value, + dark: +$("dark").value, light: +$("light").value, + volume: 2, brightness: 2, + }; +} + +function calibration() { + return { mood_gain: +$("mood_gain").value, overlay_gain: +$("overlay_gain").value, + right_variant_map: [0, 1, 2, 3, 4] }; +} + +let timer = null; +async function update() { + const resp = await fetch("/api/alteration", { + method: "POST", headers: { "content-type": "application/json" }, + body: JSON.stringify({ controls: controls(), calibration: calibration() }), + }); + if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; } + const data = await resp.json(); + readout.textContent = JSON.stringify(data, null, 2); + if (!data.content.video) { black.classList.remove("hidden"); return; } + black.classList.add("hidden"); + applyGrade(data.plan.grade.tone); + loadVariant(data.plan.restyle.variant); + renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity); +} + +function debounced() { clearTimeout(timer); timer = setTimeout(update, 80); } + +async function main() { + await loadClips(); + for (const id of ["content", "left", "right", "dark", "light", "mood_gain", "overlay_gain"]) { + $(id).addEventListener("input", debounced); + } + update(); +} +main(); +``` + +- [ ] **Step 3: Rewrite `style.css`** + +```css +* { box-sizing: border-box; } +body { margin: 0; font: 14px/1.4 system-ui, sans-serif; background: #111; color: #eee; } +header { padding: 0.6rem 1rem; background: #000; } +h1 { font-size: 1rem; margin: 0; font-weight: 600; } +main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; } +.stage { flex: 1 1 640px; } +.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000; + border-radius: 6px; overflow: hidden; } +#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; } +#overlay { position: absolute; inset: 0; width: 100%; height: 100%; + pointer-events: none; transition: opacity 0.2s ease; } +.anno-box { fill: none; stroke: #6cf; stroke-width: 0.4; vector-effect: non-scaling-stroke; } +.anno-label { fill: #6cf; font-size: 3px; font-family: monospace; } +.black { position: absolute; inset: 0; background: #000; } +.hidden { display: none; } +.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem; } +fieldset { border: 1px solid #333; border-radius: 6px; } +legend { color: #9af; padding: 0 0.4rem; } +label { display: block; margin: 0.4rem 0; } +input[type=range], select { width: 100%; } +#readout { background: #000; padding: 0.5rem; border-radius: 4px; font-size: 12px; + white-space: pre-wrap; max-height: 240px; overflow: auto; } +``` + +- [ ] **Step 4: Run the full simulator test suite (index test now passes)** + +Run: `python -m pytest tests/test_simulator_api.py -q` +Expected: PASS (all, including `test_index_is_served`). + +- [ ] **Step 5: Manual smoke (optional, needs media from Task 5)** + +Run: `python -m uvicorn simulator.app:app --port 8000` then open `http://localhost:8000`. +Expected: the base clip loops; Right slider crossfades variants; Dark/Light grade the footage live; Left slider reveals boxes + labels; "off" → black; the readout shows the engine numbers. + +- [ ] **Step 6: Commit** + +```bash +git add simulator/static/ +git commit -m "feat(simulator): alteration preview UI (grade + variant crossfade + Left overlay)" +``` + +--- + +## Task 7: Docs — update parent design, roadmap, user guide + +**Files:** +- Modify: `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` +- Modify: `docs/ROADMAP.md` +- Modify: `docs/USER_GUIDE.md` + +- [ ] **Step 1: Point the parent design at the reconciliation** + +In `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md`, near the §4.3 and §10 headings, add a note (one line each): +``` +> **Reconciled (2026-06-07):** Left HUD is a runtime overlay driven by an authored +> annotation track + per-language string tables; Right is a discrete pre-baked +> variant. See 2026-06-07-reconciled-simulator-alteration-slice-design.md §1. +``` + +- [ ] **Step 2: Update the roadmap slice status** + +In `docs/ROADMAP.md` §3 (sub-project 3 slices), mark the simulator-alteration slice as built: add a bullet: +``` +- [x] Simulator-first alteration preview — live Dark/Light grade, runtime Left + overlay (authored annotation track + string tables), discrete pre-baked Right + variants over one neutral clip, parameterized Calibration. (session 0009) +``` +Keep the deferred slices (Pi renderer, serial, endless-encoder + AI transitions, +real multi-strength re-bake) listed as not-started. + +- [ ] **Step 3: Rewrite the user-guide simulator section** + +In `docs/USER_GUIDE.md`, replace the "Playing with the simulator" section to describe the alteration preview: run `python simulator/setup_sample_media.py` once, then `uvicorn simulator.app:app`; turn the four knobs + content dial; tune the calibration sliders; read the RenderPlan readout. Remove references to the retired X-ray / selection view. + +- [ ] **Step 4: Verify the whole suite is green** + +Run: `python -m pytest -q` +Expected: PASS (no failures; previously-passing unrelated suites unaffected). + +- [ ] **Step 5: Commit** + +```bash +git add docs/ +git commit -m "docs: point parent design at reconciliation; roadmap + user guide for sim alteration" +``` + +--- + +## Self-Review + +**Spec coverage:** +- §1 decision (Left runtime overlay, Right discrete pre-baked, Dark/Light live) → Tasks 1 (engine), 6 (renderer), 5 (annotation track). ✓ +- §2 engine reconciliation (`Calibration`, `Restyle.variant`, `AnalyticalOverlay.level`, `state.py`) → Tasks 1, 2. ✓ +- §3 simulator (retire selection; `clips.py`; `/api/alteration` + `/api/clips`; browser preview) → Tasks 3, 4, 6. ✓ +- §4 sample clip + real Right variant + placeholder generator + Left annotation track → Task 5 (media/script), Task 5 manifest (annotations). ✓ +- §5 testing (player unit, sim API, removed endpoints, fixtures retired) → Tasks 1–4. ✓ +- §6 ships (incl. doc updates) → Task 7. ✓ + +**Placeholder scan:** No TBD/TODO; every code step shows complete code. ✓ + +**Type consistency:** `Restyle.variant` (int), `AnalyticalOverlay.level`/`.intensity`, `Calibration(mood_gain, overlay_gain, right_variant_map)`, `render_plan_to_dict` keys (`grade.tone`, `overlay.level/intensity`, `restyle.variant`, `is_identity`), `Clip.variant_file`/`to_dict`, `load_manifest`, `create_app(manifest_path=)` — used identically across Tasks 1, 3, 4, 6. ✓ + +**Note for the executor:** verify `player/content.py`'s `ContentResolution` field names (`audio_source`, `video`) before Task 4 Step 3; match them if different. `CONTENT_POSITIONS` is importable from `player/controls.py`. diff --git a/docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md b/docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md index c855cdc..7298a4b 100644 --- a/docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md +++ b/docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md @@ -121,6 +121,13 @@ layers. This single rule covers the whole `5×5×5×5` space: ### 4.3 Where each transform runs +> **Reconciled (2026-06-07, session 0009):** the Left HUD is a **runtime overlay** +> driven by an authored annotation track + per-language string tables (text shaped +> live); the Right axis selects a **discrete pre-baked** flow-stabilized variant +> (not a continuous blend). See +> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md) +> §1, which supersedes the session-0007 baked-HUD / 5×5-grid proposal. + - **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade and the Left analytical overlay. These are cheap (LUT/curves + luma key, and text/graphics compositing) and can move continuously with the knob. @@ -235,6 +242,13 @@ one-time build. ## 10. Accessibility, i18n, and the translation-cost finding +> **Reconciled (2026-06-07, session 0009):** the near-free-i18n path is kept — the +> Left HUD is a runtime overlay (authored annotation track + per-language string +> tables, shaped live via Pango/HarfBuzz on the Pi). The session-0007 baked-HUD +> reversal is **not** adopted. See +> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md) +> §1. + The piece is operable **blind, in the dark, in your language**, via four redundant channels on the control panel: **touch** (engraved symbol shape), **low-light color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local* diff --git a/docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md b/docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md new file mode 100644 index 0000000..94d8194 --- /dev/null +++ b/docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md @@ -0,0 +1,242 @@ +# Human Experience Filter — Simulator Alteration Preview (Design) + +**Date:** 2026-06-06 +**Status:** Approved design (pre-implementation) +**Repo:** `human-experience-filter-art` +**Builds on:** the alteration engine of +[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md) +(the "design" below) and the simulator scaffold of +[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md). +**Revises the design:** §4.3 and §10 — the Left analytical HUD is **no longer a +runtime overlay**; it is **baked into authored variant videos** (see §8 below). +This is a deliberate trade of near-free multilingual support for authorial +precision over the HUD. +**Retires:** the simulator's selection-era "curator's X-ray" view and its +`/api/select` + `/api/catalog/meta` endpoints (the selection model they +visualize was superseded by the alteration model). + +> **Why this exists.** Operator directive (session 0007): *build and design only +> things that run in the simulator, and get the whole experience working the way +> we like in the simulator before moving to hardware.* The slice-1 alteration +> engine (PR #5) is pure logic with **no simulator surface** — you cannot yet turn +> the experience knobs and *see* the result. This design brings the alteration +> into the simulator so the look can be tuned and liked before any Pi/serial work. + +--- + +## 1. Scope + +A browser **alteration preview**: turn the four experience knobs and the content +dial, and see the neutral base footage altered toward the knob state, in real +time, on a looping clip. The purpose is to **tune the look** of the filter and to +**settle the knob→strength calibration by eye** (the open §3-vs-§4.2/§5 question +from session 0006). + +**In scope** +- Live preview of the four experience knobs (Left/Right/Dark/Light) on a looping + base clip. +- **Dark/Light** rendered as a live, deterministic runtime color grade. +- **Left/Right** rendered by selecting a **pre-baked authored variant clip** from + a 5×5 grid and crossfading on change. +- A **calibration panel** that adjusts the grade curve live; the chosen values are + baked back into `player/alteration.py` defaults. +- A **RenderPlan readout** (the project's "X-ray" honesty): always show the exact + numbers the engine produced. +- The content dial's **video on/off** behavior (so "Off" goes to black), driven by + the existing `resolve_content`. +- Placeholder variant generation so the mechanism is testable before the operator + authors real videos. + +**Out of scope (this slice)** — real generative v2v; serial input / the 3⇄4 +framing contract; audio playback (music / white-noise / audio-track); the Pi/mpv +runtime renderer; the full transition/crossfade timing engine; multilingual label +tables; catalog-model changes. These remain later roadmap slices. + +--- + +## 2. Architecture — Python-canonical, thin browser renderer + +`player/alteration.py` stays the **single source of truth** for the alteration +math (handbook §4.2 — deterministic core, thin I/O). The browser owns only +*rendering*. Data flow: + +```mermaid +flowchart LR + subgraph Browser + K[4 experience knobs +
content dial +
calibration sliders] + V["<video> variant + canvas grade"] + RO[RenderPlan readout] + end + subgraph FastAPI [simulator/app.py] + EP["POST /api/alteration"] + CL["GET /api/clips"] + end + ENG["player.alteration.plan_alteration(coord, calibration)"] + K -- "debounced POST {controls, calibration}" --> EP + EP --> ENG --> EP -- "RenderPlan {variant, grade}" --> V + EP --> RO + CL -- "base clip + variant manifest" --> V +``` + +- The browser sends control/calibration changes (debounced) and receives a + `RenderPlan`. Video filtering itself runs continuously in the browser; only a + *plan recompute* makes a round-trip. On localhost these JSON round-trips are + imperceptible. +- "Bake the calibration winner in" = change the `Calibration` defaults in Python. + Nothing in the browser is canonical. + +--- + +## 3. The four axes — how each is rendered + +| Axis | Engine output | Browser rendering | +|---|---|---| +| **Dark / Light** | `ColorGrade.tone` ∈ [−1, 1], center = identity | live color grade on the `