Compare commits

..

3 Commits

4 changed files with 127 additions and 242 deletions
@@ -1,242 +0,0 @@
# Human Experience Filter — Simulator Alteration Preview (Design)
**Date:** 2026-06-06
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
**Builds on:** the alteration engine of
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
(the "design" below) and the simulator scaffold of
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md).
**Revises the design:** §4.3 and §10 — the Left analytical HUD is **no longer a
runtime overlay**; it is **baked into authored variant videos** (see §8 below).
This is a deliberate trade of near-free multilingual support for authorial
precision over the HUD.
**Retires:** the simulator's selection-era "curator's X-ray" view and its
`/api/select` + `/api/catalog/meta` endpoints (the selection model they
visualize was superseded by the alteration model).
> **Why this exists.** Operator directive (session 0007): *build and design only
> things that run in the simulator, and get the whole experience working the way
> we like in the simulator before moving to hardware.* The slice-1 alteration
> engine (PR #5) is pure logic with **no simulator surface** — you cannot yet turn
> the experience knobs and *see* the result. This design brings the alteration
> into the simulator so the look can be tuned and liked before any Pi/serial work.
---
## 1. Scope
A browser **alteration preview**: turn the four experience knobs and the content
dial, and see the neutral base footage altered toward the knob state, in real
time, on a looping clip. The purpose is to **tune the look** of the filter and to
**settle the knob→strength calibration by eye** (the open §3-vs-§4.2/§5 question
from session 0006).
**In scope**
- Live preview of the four experience knobs (Left/Right/Dark/Light) on a looping
base clip.
- **Dark/Light** rendered as a live, deterministic runtime color grade.
- **Left/Right** rendered by selecting a **pre-baked authored variant clip** from
a 5×5 grid and crossfading on change.
- A **calibration panel** that adjusts the grade curve live; the chosen values are
baked back into `player/alteration.py` defaults.
- A **RenderPlan readout** (the project's "X-ray" honesty): always show the exact
numbers the engine produced.
- The content dial's **video on/off** behavior (so "Off" goes to black), driven by
the existing `resolve_content`.
- Placeholder variant generation so the mechanism is testable before the operator
authors real videos.
**Out of scope (this slice)** — real generative v2v; serial input / the 3⇄4
framing contract; audio playback (music / white-noise / audio-track); the Pi/mpv
runtime renderer; the full transition/crossfade timing engine; multilingual label
tables; catalog-model changes. These remain later roadmap slices.
---
## 2. Architecture — Python-canonical, thin browser renderer
`player/alteration.py` stays the **single source of truth** for the alteration
math (handbook §4.2 — deterministic core, thin I/O). The browser owns only
*rendering*. Data flow:
```mermaid
flowchart LR
subgraph Browser
K[4 experience knobs +<br/>content dial +<br/>calibration sliders]
V["&lt;video&gt; variant + canvas grade"]
RO[RenderPlan readout]
end
subgraph FastAPI [simulator/app.py]
EP["POST /api/alteration"]
CL["GET /api/clips"]
end
ENG["player.alteration.plan_alteration(coord, calibration)"]
K -- "debounced POST {controls, calibration}" --> EP
EP --> ENG --> EP -- "RenderPlan {variant, grade}" --> V
EP --> RO
CL -- "base clip + variant manifest" --> V
```
- The browser sends control/calibration changes (debounced) and receives a
`RenderPlan`. Video filtering itself runs continuously in the browser; only a
*plan recompute* makes a round-trip. On localhost these JSON round-trips are
imperceptible.
- "Bake the calibration winner in" = change the `Calibration` defaults in Python.
Nothing in the browser is canonical.
---
## 3. The four axes — how each is rendered
| Axis | Engine output | Browser rendering |
|---|---|---|
| **Dark / Light** | `ColorGrade.tone` ∈ [1, 1], center = identity | live color grade on the `<video>`: Light → warm/yellow + negative space toward white; Dark → cool/blue + negative space toward black; 0 = raw. (Confirmed direction in brainstorm.) |
| **Left / Right** | `VariantRef(left, right)` | select the authored variant clip for `(left, right)`; **crossfade** when it changes; `(0,0)` → raw base. The analytical HUD (Left) and dreamlike restyle (Right) are **baked into the clip** (§8). |
Dark/Light is the only continuously-tunable axis in this slice; Left/Right is a
discrete selection by coordinate.
---
## 4. The variant grid (per base clip)
Each base clip carries a 5×5 grid keyed by the two baked axes:
- Rows = **Left** (analytical / HUD), 04. Columns = **Right** (artistic /
dreamlike restyle), 04.
- `(0,0)` = the raw base clip — no authored file needed.
- The other **24 cells are authored videos** (4 analytical-edge + 4 dreamlike-edge
+ 16 combined core). This is **per base clip**; authoring load multiplies by the
number of base clips (and by language, since the HUD is baked — §8).
The grid is **data**, not code: a manifest the simulator reads (see §6).
---
## 5. Engine reconciliation (`player/alteration.py`)
The slice-1 engine modeled Left/Right as continuous runtime layers
(`AnalyticalOverlay.intensity`, `Restyle.blend`). Baked variants make that
obsolete. Changes:
- **`RenderPlan`** becomes `{ grade: ColorGrade, variant: VariantRef }`.
- `ColorGrade` — unchanged.
- `VariantRef(left, right)` — new; identity selection by coordinate.
- **Remove** `AnalyticalOverlay` and `Restyle` (their content is now baked into the
variant clip) rather than leaving dead fields.
- **`Calibration`** — a new frozen dataclass parameterizing the grade curve (e.g.
`mood_center`, per-axis curve), defaulting to **today's exact behavior**
(behavior-preserving). `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`.
- **`player/state.py`** — the `CROSSFADE` trigger switches from "restyle changed"
to "variant changed"; `LIVE_UPDATE` still covers a grade-only change. Video
on/off fades unchanged.
These are framework-code changes that keep the engine pure and unit-tested; they
do **not** bake any deployment-shape decision into the engine.
---
## 6. Data & endpoints
- **`simulator/clips.py`** (replaces `simulator/fixtures.py`): reads a base-clip +
variant manifest. Each base clip: `{ id, title, base_file, license, source,
variants: { "L,R": { file, model?, hud_lang? } } }`. Missing cells fall back to
the raw base (and are flagged in the readout as "raw / unauthored").
- **`POST /api/alteration`**: body `{ controls, calibration }``RenderPlan`
(plus the `ContentResolution` from `resolve_content`, so the dial's video-on/off
is honored). The endpoint calls the real `plan_alteration`.
- **`GET /api/clips`**: the base-clip list + variant manifest for the active base.
- Video files served as static assets from a sample-media directory.
- **Removed**: `POST /api/select`, `GET /api/catalog/meta`.
- `hef.selection` (the library, incl. `Coordinate`) is **untouched**; only the
simulator's selection view/endpoints retire. (`ranked_candidates`, added for the
old X-ray, may become unused by the simulator — noted, not removed here.)
---
## 7. Calibration tuning
The calibration panel exposes the grade-curve parameters (the two conventions in
tension: `(2,2,2,2)`-centered vs the `value/4` reading currently implemented, plus
curve shape). Changing them re-requests the plan and the footage responds live.
When the operator is happy, the chosen values are written into
`DEFAULT_CALIBRATION` in `player/alteration.py` and locked in by a unit test.
---
## 8. Baked HUD — the §4.3/§10 revision (accepted trade)
The design's §4.3/§10 kept the Left HUD a **runtime overlay** specifically so the
v2v substrate is language-agnostic and multilingual support is near-free. This
design **reverses that for the Left/Right plane**: the operator authors the HUD
**into** the variant videos to get pixel-precise control over the HUD and the
analytical↔feeling balance.
**Consequences (accepted):**
- Multilingual support is **no longer near-free**: baked HUD text means a new
language re-renders every label-bearing variant (the §10 "budget landmine").
The piece is English-first; broad i18n is deferred/expensive.
- The live Dark/Light grade is applied **on top** of the baked cell, so it **tints
the baked HUD** too (a "Dark" mood cools/darkens the HUD). With baked overlays
the grade cannot skip the HUD. Escape hatch: author HUD colors that survive
grading, or (future) reinstate a runtime HUD layer.
The parent design's §4.3/§10 must be updated to point at this revision (a task for
the implementation plan).
---
## 9. Bootstrapping before authored videos exist
The operator will author the 24 variants per base clip; none exist yet. So the
mechanism is testable immediately, the build includes a **placeholder generator**:
for at least one base clip, produce the 24 cells by burning the cell's `L,R` (and
a stub HUD caption) into the base loop via ffmpeg. Real authored clips drop into
the manifest with **no code change**.
---
## 10. Testing
- **`player/` unit tests:** `DEFAULT_CALIBRATION` is behavior-preserving vs the
current helpers; the two calibration conventions produce the expected plans;
`VariantRef` selection (incl. `(0,0)` → raw) and the `state.py` crossfade
trigger on variant change.
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
`POST /api/alteration` returns the engine's plan for given controls/calibration;
`GET /api/clips` returns the manifest; the removed endpoints are gone.
- **No browser/E2E automation** this slice (manual visual tuning is the point);
the rendering JS is kept thin and the engine logic stays in tested Python.
---
## 11. What ships
- `player/alteration.py` (parameterized `Calibration`, `VariantRef`, slimmed
`RenderPlan`) + `player/state.py` crossfade-trigger update.
- `simulator/clips.py` (variant manifest) replacing `fixtures.py`.
- `simulator/app.py` new endpoints; selection endpoints removed.
- `simulator/static/` rewritten as the Player preview (variant `<video>` +
crossfade, live grade, calibration panel, RenderPlan readout, content dial).
- Placeholder-variant generator + a sample base clip.
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten;
parent design §4.3/§10 pointer updated; `docs/ROADMAP.md` §3 updated.
---
## 12. Open questions (for the plan, not blockers)
- **Grade vs baked HUD interaction** — accepted that the grade tints the HUD;
revisit only if it reads badly once real authored clips exist.
- **Sample base clip** — pick one CC0 nature loop for the placeholder grid.
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for
tuning; the real timing engine is a later slice.
---
## 13. Out of scope (YAGNI)
Real generative v2v; serial / firmware; audio playback; the Pi renderer; the full
transition engine; multilingual tables; catalog-model changes; retiring
`hef.selection.ranked_candidates`. All remain later roadmap work.
@@ -0,0 +1,101 @@
# Session 0007.0 — Transcript
> App: human-experience-filter-art
> Date: 2026-06-06 (PST) · 14:12 → 17:57
> Type: coding
> Goal: continue the roadmap; reframed by the operator to **simulator-first** —
> bring the slice-1 alteration engine into the simulator so the experience can be
> tuned before any hardware.
> Outcome: **design spec written, committed, and pushed** on branch
> `feature/simulator-alteration-preview`. No code changes. Implementation deferred
> to the next session (design + implementation to ship as one PR).
## Plan
Bring the alteration engine into the simulator (scope: tune the look), per
`docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md`:
- Python-canonical engine, thin browser renderer.
- Dark/Light = live runtime color grade; Left/Right = 5×5 grid of pre-baked
authored variant clips selected by coordinate (24 videos/base clip).
- Revises design §4.3/§10: HUD baked into variant videos (authorial precision)
rather than runtime overlay — trades away near-free i18n.
- Engine reconciliation: parameterize `Calibration`, add `VariantRef`, slim
`RenderPlan` (drop runtime `AnalyticalOverlay`/`Restyle`); `state.py` crossfade
on variant change.
- Simulator: retire selection X-ray + `/api/select`; add `/api/alteration` +
`/api/clips` + variant manifest; placeholder-variant generator.
## Pre-session state
- `main` at `5290785`, clean and pushed. Sub-projects 1 & 2 done; sub-project 3
(Player Runtime) in progress — slice 1 (pure-logic `player/` core) merged via
PR #5 (session 0006).
- A **stale nested worktree** (`.worktrees/feature-BuildSimulator11`, branch
`feature/player-alteration-core`, already merged) was present and breaking the
session resolver ("ambiguous app.json"), exactly as the session-0006 memory
gotcha predicted.
## Turn-by-turn arc
1. **Session gate.** Opened with "What's next?" → classified as a coding session;
ran `wgl-session-coding-init`.
2. **Baseline cleanup.** Claim dry-run failed on the resolver ambiguity. Verified
the stale worktree's branch was fully merged + clean, removed the worktree and
branch, and deleted the empty `.worktrees/` dir. Tree back to clean `main`.
3. **Claimed session 0007** (no other sessions in flight).
4. **Operator steer:** "only design things that will run in the simulator; get
everything working the way we like in the simulator before hardware." Saved as
a `feedback` memory ([[simulator-first-before-hardware]]).
5. **Discovered** the simulator (`simulator/`, built session 0004) is fully
decoupled from `player/` — it exercises the OLD *selection* model and has no
alteration surface. That gap became the work.
6. **Brainstorming** (with the visual companion). Decisions, one at a time:
scope = tune the alteration look; medium = short looping video; Right restyle =
labeled stand-in shader (swappable for real variants); calibration =
live-adjustable curves; architecture = **A, Python-canonical**; integration =
new view, **retire the selection X-ray**.
7. **Interactive look mockup** (browser): a real nature image with the 4 knobs as
live sliders. Operator confirmed Dark/Light read well, but said for Left/Right
they'll **author videos with the overlay baked in** for precise HUD control.
8. **Surfaced the tradeoff:** baking the HUD reverses design §4.3/§10's near-free
i18n. Operator chose **precision wins**. Confirmed the variant grid as **full
5×5 = 24 authored clips per base** (shown as a grid mockup).
9. **Revised model** presented and approved: base clip + variant manifest;
`RenderPlan = {grade, variant}`; runtime overlay/restyle layers retired;
placeholder-variant bootstrapping via ffmpeg.
10. **Wrote the spec**, self-reviewed, committed on a feature branch.
11. Operator: "do [implementation] next session. finalize this one." →
`wgl-session-finalize`.
## Cut state (end of session)
| Repo | Branch | Commit | State |
|---|---|---|---|
| human-experience-filter-art | `feature/simulator-alteration-preview` | `3ef21fb` | pushed to origin, **not merged** |
- `docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md`
new, committed.
- `main` unchanged at `5290785`.
- Working tree clean. Visual-companion server stopped; `.superpowers/` gitignored.
- No tests run (no code changed).
## What lands on the operator's plate
- **Author the 24 variant videos** per base clip (the Left×Right grid), with HUD
baked in. The simulator will consume them via the manifest; placeholder variants
cover the gap until then.
- **Deferred decisions** (also surfaced in chat):
- *knob→strength calibration* — unresolved since session 0006; will be settled
by eye in the simulator's calibration panel next session, then baked into
`DEFAULT_CALIBRATION`.
- *grade tints baked HUD* — accepted under "precision wins"; revisit only if it
reads badly once real authored clips exist.
## Prompt the operator can paste into the next session
```
/goal Implement the simulator alteration preview on branch feature/simulator-alteration-preview, per docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md — begin with the writing-plans skill, then build: parameterize Calibration + add VariantRef and slim RenderPlan in player/, add /api/alteration + /api/clips + simulator/clips.py variant manifest, rewrite the simulator UI as the player preview, retire the selection X-ray + /api/select, add the placeholder-variant generator, and update tests + USER_GUIDE + ROADMAP §3 + the parent design §4.3/§10 pointer.
```
(Or resume with `/goal next` — the `Next /goal:` field is stored in memory
`sub-project-3-player-progress.md`.)
@@ -0,0 +1,20 @@
# Session 0008.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-07T19-42 (PST)
> Type: spec
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
>
> This file reserves session ID 0008 for human-experience-filter-art. The driver replaces this
> body with the full transcript and renames the file to its final
> SESSION-0008.0-TRANSCRIPT-2026-06-07T19-42--<end>.md form at session end.
## Launch prompt
_(launch prompt not captured at claim time)_
## Deferred decisions
_Autonomous-mode low-confidence calls the driver made and would have
liked operator input on. Appended as the session runs; surfaced at
finalize. Empty if none._
+6
View File
@@ -16,5 +16,11 @@
},
"0006": {
"title": ""
},
"0007": {
"title": ""
},
"0008": {
"title": ""
}
}