# Human Experience Filter — Simulator Alteration Preview (Design)

**Date:** 2026-06-06
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
**Builds on:** the alteration engine of
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
(the "design" below) and the simulator scaffold of
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md).
**Revises the design:** §4.3 and §10 — the Left analytical HUD is **no longer a
runtime overlay**; it is **baked into authored variant videos** (see §8 below).
This is a deliberate trade of near-free multilingual support for authorial
precision over the HUD.
**Retires:** the simulator's selection-era "curator's X-ray" view and its
`/api/select` + `/api/catalog/meta` endpoints (the selection model they
visualize was superseded by the alteration model).

> **Why this exists.** Operator directive (session 0007): *build and design only
> things that run in the simulator, and get the whole experience working the way
> we like in the simulator before moving to hardware.* The slice-1 alteration
> engine (PR #5) is pure logic with **no simulator surface** — you cannot yet turn
> the experience knobs and *see* the result. This design brings the alteration
> into the simulator so the look can be tuned and liked before any Pi/serial work.

---

## 1. Scope

A browser **alteration preview**: turn the four experience knobs and the content
dial, and see the neutral base footage altered toward the knob state, in real
time, on a looping clip. The purpose is to **tune the look** of the filter and to
**settle the knob→strength calibration by eye** (the open §3-vs-§4.2/§5 question
from session 0006).

**In scope**
- Live preview of the four experience knobs (Left/Right/Dark/Light) on a looping
  base clip.
- **Dark/Light** rendered as a live, deterministic runtime color grade.
- **Left/Right** rendered by selecting a **pre-baked authored variant clip** from
  a 5×5 grid and crossfading on change.
- A **calibration panel** that adjusts the grade curve live; the chosen values are
  baked back into `player/alteration.py` defaults.
- A **RenderPlan readout** (the project's "X-ray" honesty): always show the exact
  numbers the engine produced.
- The content dial's **video on/off** behavior (so "Off" goes to black), driven by
  the existing `resolve_content`.
- Placeholder variant generation so the mechanism is testable before the operator
  authors real videos.

**Out of scope (this slice)** — real generative v2v; serial input / the 3⇄4
framing contract; audio playback (music / white-noise / audio-track); the Pi/mpv
runtime renderer; the full transition/crossfade timing engine; multilingual label
tables; catalog-model changes. These remain later roadmap slices.

---

## 2. Architecture — Python-canonical, thin browser renderer

`player/alteration.py` stays the **single source of truth** for the alteration
math (handbook §4.2 — deterministic core, thin I/O). The browser owns only
*rendering*. Data flow:

```mermaid
flowchart LR
  subgraph Browser
    K[4 experience knobs +<br/>content dial +<br/>calibration sliders]
    V["&lt;video&gt; variant + canvas grade"]
    RO[RenderPlan readout]
  end
  subgraph FastAPI [simulator/app.py]
    EP["POST /api/alteration"]
    CL["GET /api/clips"]
  end
  ENG["player.alteration.plan_alteration(coord, calibration)"]
  K -- "debounced POST {controls, calibration}" --> EP
  EP --> ENG --> EP -- "RenderPlan {variant, grade}" --> V
  EP --> RO
  CL -- "base clip + variant manifest" --> V
```

- The browser sends control/calibration changes (debounced) and receives a
  `RenderPlan`. Video filtering itself runs continuously in the browser; only a
  *plan recompute* makes a round-trip. On localhost these JSON round-trips are
  imperceptible.
- "Bake the calibration winner in" = change the `Calibration` defaults in Python.
  Nothing in the browser is canonical.

---

## 3. The four axes — how each is rendered

| Axis | Engine output | Browser rendering |
|---|---|---|
| **Dark / Light** | `ColorGrade.tone` ∈ [−1, 1], center = identity | live color grade on the `<video>`: Light → warm/yellow + negative space toward white; Dark → cool/blue + negative space toward black; 0 = raw. (Confirmed direction in brainstorm.) |
| **Left / Right** | `VariantRef(left, right)` | select the authored variant clip for `(left, right)`; **crossfade** when it changes; `(0,0)` → raw base. The analytical HUD (Left) and dreamlike restyle (Right) are **baked into the clip** (§8). |

Dark/Light is the only continuously-tunable axis in this slice; Left/Right is a
discrete selection by coordinate.

---

## 4. The variant grid (per base clip)

Each base clip carries a 5×5 grid keyed by the two baked axes:

- Rows = **Left** (analytical / HUD), 0–4. Columns = **Right** (artistic /
  dreamlike restyle), 0–4.
- `(0,0)` = the raw base clip — no authored file needed.
- The other **24 cells are authored videos** (4 analytical-edge + 4 dreamlike-edge
  + 16 combined core). This is **per base clip**; authoring load multiplies by the
  number of base clips (and by language, since the HUD is baked — §8).

The grid is **data**, not code: a manifest the simulator reads (see §6).

---

## 5. Engine reconciliation (`player/alteration.py`)

The slice-1 engine modeled Left/Right as continuous runtime layers
(`AnalyticalOverlay.intensity`, `Restyle.blend`). Baked variants make that
obsolete. Changes:

- **`RenderPlan`** becomes `{ grade: ColorGrade, variant: VariantRef }`.
  - `ColorGrade` — unchanged.
  - `VariantRef(left, right)` — new; identity selection by coordinate.
- **Remove** `AnalyticalOverlay` and `Restyle` (their content is now baked into the
  variant clip) rather than leaving dead fields.
- **`Calibration`** — a new frozen dataclass parameterizing the grade curve (e.g.
  `mood_center`, per-axis curve), defaulting to **today's exact behavior**
  (behavior-preserving). `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`.
- **`player/state.py`** — the `CROSSFADE` trigger switches from "restyle changed"
  to "variant changed"; `LIVE_UPDATE` still covers a grade-only change. Video
  on/off fades unchanged.

These are framework-code changes that keep the engine pure and unit-tested; they
do **not** bake any deployment-shape decision into the engine.

---

## 6. Data & endpoints

- **`simulator/clips.py`** (replaces `simulator/fixtures.py`): reads a base-clip +
  variant manifest. Each base clip: `{ id, title, base_file, license, source,
  variants: { "L,R": { file, model?, hud_lang? } } }`. Missing cells fall back to
  the raw base (and are flagged in the readout as "raw / unauthored").
- **`POST /api/alteration`**: body `{ controls, calibration }` → `RenderPlan`
  (plus the `ContentResolution` from `resolve_content`, so the dial's video-on/off
  is honored). The endpoint calls the real `plan_alteration`.
- **`GET /api/clips`**: the base-clip list + variant manifest for the active base.
- Video files served as static assets from a sample-media directory.
- **Removed**: `POST /api/select`, `GET /api/catalog/meta`.
- `hef.selection` (the library, incl. `Coordinate`) is **untouched**; only the
  simulator's selection view/endpoints retire. (`ranked_candidates`, added for the
  old X-ray, may become unused by the simulator — noted, not removed here.)

---

## 7. Calibration tuning

The calibration panel exposes the grade-curve parameters (the two conventions in
tension: `(2,2,2,2)`-centered vs the `value/4` reading currently implemented, plus
curve shape). Changing them re-requests the plan and the footage responds live.
When the operator is happy, the chosen values are written into
`DEFAULT_CALIBRATION` in `player/alteration.py` and locked in by a unit test.

---

## 8. Baked HUD — the §4.3/§10 revision (accepted trade)

The design's §4.3/§10 kept the Left HUD a **runtime overlay** specifically so the
v2v substrate is language-agnostic and multilingual support is near-free. This
design **reverses that for the Left/Right plane**: the operator authors the HUD
**into** the variant videos to get pixel-precise control over the HUD and the
analytical↔feeling balance.

**Consequences (accepted):**
- Multilingual support is **no longer near-free**: baked HUD text means a new
  language re-renders every label-bearing variant (the §10 "budget landmine").
  The piece is English-first; broad i18n is deferred/expensive.
- The live Dark/Light grade is applied **on top** of the baked cell, so it **tints
  the baked HUD** too (a "Dark" mood cools/darkens the HUD). With baked overlays
  the grade cannot skip the HUD. Escape hatch: author HUD colors that survive
  grading, or (future) reinstate a runtime HUD layer.

The parent design's §4.3/§10 must be updated to point at this revision (a task for
the implementation plan).

---

## 9. Bootstrapping before authored videos exist

The operator will author the 24 variants per base clip; none exist yet. So the
mechanism is testable immediately, the build includes a **placeholder generator**:
for at least one base clip, produce the 24 cells by burning the cell's `L,R` (and
a stub HUD caption) into the base loop via ffmpeg. Real authored clips drop into
the manifest with **no code change**.

---

## 10. Testing

- **`player/` unit tests:** `DEFAULT_CALIBRATION` is behavior-preserving vs the
  current helpers; the two calibration conventions produce the expected plans;
  `VariantRef` selection (incl. `(0,0)` → raw) and the `state.py` crossfade
  trigger on variant change.
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
  `POST /api/alteration` returns the engine's plan for given controls/calibration;
  `GET /api/clips` returns the manifest; the removed endpoints are gone.
- **No browser/E2E automation** this slice (manual visual tuning is the point);
  the rendering JS is kept thin and the engine logic stays in tested Python.

---

## 11. What ships

- `player/alteration.py` (parameterized `Calibration`, `VariantRef`, slimmed
  `RenderPlan`) + `player/state.py` crossfade-trigger update.
- `simulator/clips.py` (variant manifest) replacing `fixtures.py`.
- `simulator/app.py` new endpoints; selection endpoints removed.
- `simulator/static/` rewritten as the Player preview (variant `<video>` +
  crossfade, live grade, calibration panel, RenderPlan readout, content dial).
- Placeholder-variant generator + a sample base clip.
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten;
  parent design §4.3/§10 pointer updated; `docs/ROADMAP.md` §3 updated.

---

## 12. Open questions (for the plan, not blockers)

- **Grade vs baked HUD interaction** — accepted that the grade tints the HUD;
  revisit only if it reads badly once real authored clips exist.
- **Sample base clip** — pick one CC0 nature loop for the placeholder grid.
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for
  tuning; the real timing engine is a later slice.

---

## 13. Out of scope (YAGNI)

Real generative v2v; serial / firmware; audio playback; the Pi renderer; the full
transition engine; multilingual tables; catalog-model changes; retiring
`hef.selection.ranked_candidates`. All remain later roadmap work.
