feat(sim): Right axis = deterministic real-time painterly dream

Replace the pre-baked SD restyle variants (too fluid — the painting re-rolled
every keyframe and the flow-warp melted frames) with a deterministic, real-time
dream that holds STILL across the loop; only the footage's own motion moves.
Design: docs/superpowers/specs/2026-06-22-right-axis-deterministic-dream-design.md

Engine (player/alteration.py): Restyle(variant) -> Dream(strength, intensity);
Calibration.right_variant_map -> dream_gain. player/state.py: a Right change is a
LIVE_UPDATE now, not a crossfade (only a clip swap crossfades). Plan dict
restyle -> dream. Tests migrated; 233 green. (No new Python behavior for the
client-render swap below.)

Renderer (simulator front-end): the Right dream is a WebGL2 Kuwahara shader on a
<canvas> over the live video — edge-preserving, so motion/structure stay crisp;
the dream is the same footage, just stylized (no blur). Ramped by the knob; max
goes "trippy" (vivid saturation + ~45deg hue drift + posterize, concentrated near
max via t=intensity^2). Phase-1 luminous-haze (CSS+bloom) was built then retired
by eye (blur read as out-of-focus video). Mood grade rides as a CSS filter on the
canvas; bloom layer removed.

Dev ergonomics that fell out of the iteration:
- /dev/version + a 1s client poll = live-reload (open tab never runs stale
  renderer code; reloads on my edits AND server restarts).
- no-cache on the dev server; an on-screen error banner + crash-proof render so a
  silent throw can't masquerade as "nothing is changing".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ben Stull
2026-06-23 09:55:35 -07:00
parent 6adb93407b
commit cbb7c9f53d
10 changed files with 404 additions and 92 deletions
@@ -0,0 +1,109 @@
# Right axis as a real-time deterministic dream
**Status:** built (session 0013, 2026-06-22) — Phase 1 superseded by Phase 2 in the
same session after by-eye review (the haze "looked like blurred video, not stylized").
The shipped Right dream is the **Phase 2 painterly** path.
**Surface:** simulator preview (`simulator/`, `player/alteration.py`)
**Supersedes:** the "discrete pre-baked, flow-stabilized restyle variant" definition
of the Right axis in the machine-altered-perception design SPEC §4.1 — a deliberate
change to a core decision (see Why).
## Why
The pre-baked SD img2img variants (`bake_right_variants.py`) are **too fluid**: the
painting re-rolls every keyframe (~2 s) and the optical-flow tween warps frames into
each other, so across the loop the dream churns and melts. The right brain should be
a **still altered state** — a calm, holistic counterpart to the busy, accreting
analytical Left — not a restless morph.
## Concept
The Right knob no longer selects a pre-baked variant. It drives a **deterministic,
real-time filter** applied live to the base footage, scaled 0→4. The same filter runs
every frame, so the look is **stable by construction** — locked to scene content,
moving only where the footage already moves. Bonus: no baked variant videos to ship,
which simplifies the eventual hardware (Pi) story.
Staged delivery (both built this session; Phase 2 is what shipped):
- **Phase 1: luminous haze** — soft-focus, blooming highlights, pastel desaturation.
CSS + a bloom layer. Built, then **retired** by eye: blur reads as *out-of-focus
video*, not stylized, and softens the base's crisp motion.
- **Phase 2: painterly flatten (shipped)** — a WebGL2 **Kuwahara** shader on a
`<canvas>` restyling the live video frames. Edge-preserving, so motion/structure
stay as crisp as the source — the dream is the same footage, just stylized. No blur.
## Phase 2 — the shipped painterly dream
- **Renderer:** `<canvas id="paint">` over `<video>`; a `requestAnimationFrame` loop
uploads each video frame to a texture and runs the Kuwahara fragment shader (radius
`R=6`). `intensity = 0` ⇒ passthrough (raw base); ring transitions render passthrough
(`busy`). WebGL-less fallback: canvas hidden, mood grade applied to `#vid`.
- **Painterly:** `mix(base, kuwahara, intensity)` — flatten into painted regions, edges
preserved.
- **"Trippy" at max** (operator-tuned by eye): the psychedelic terms ramp with
`t = intensity²` so low/mid Right stays gently dreamy and only **max** goes
psychedelic — vivid saturation (`×(1 + 1.1·t)`), a surreal **hue drift** (~45° at max),
and **posterize** banding (`mix(255, 6, t)` levels), plus a slight luminous lift.
- **Mood grade** (Dark/Light) rides as a CSS filter on the canvas; `#tint` unchanged.
- The Python plan is unchanged — Phase 2 is a client-render swap; `dream.intensity`
drives the shader.
## Phase 1 — components
All scale with the Right strength `s = intensity` (0→1):
- **soft-focus** — gentle blur, ~`1.5·s` px.
- **pastel** — `saturate(1 0.35·s)`, `contrast(1 0.15·s)` (lift shadows / flatten).
- **luminous** — `brightness(1 + 0.08·s)`.
- **bloom** — a second, synced copy of the video, blurred + brightened and
`screen`-blended over the main, opacity `~0.6·s` — highlights glow and bleed.
Starting coefficients are by-eye seeds; the look is tuned live in the sim.
## Engine — `player/alteration.py`
- Replace `Restyle(variant)` with `Dream(strength, intensity)`:
`strength = right` (04, discrete), `intensity = clamp(dream_gain · right / KNOB_MAX,
0, 1)`.
- `Calibration` gains `dream_gain: float = 1.0`; `DEFAULT_CALIBRATION` sets it 1.0.
- `render_plan_to_dict`: `"restyle": {"variant"}``"dream": {"strength", "intensity"}`.
- `is_identity`: Right contributes when `strength > 0` (replaces the
`restyle.variant == 0` clause).
- Pure + unit-tested. The client maps `dream.intensity` → CSS params, mirroring how
`grade.tone` → video filter is mapped client-side today.
## Rendering — simulator
- **Remove** the variant-crossfade machinery (`loadVariant`, `variantFile`, the
`right_variants` / `currentVariant` handling). The main video always plays
`base_file`.
- **Compose one video filter** from *both* the mood grade (`grade.tone`) and the dream
(`dream.intensity`) — today `applyGrade` owns `vid.style.filter`; it becomes a single
`applyVideoLook(tone, dreamIntensity)` so the two don't clobber each other.
- **Add a bloom layer**: a second `<video id="bloom">` (same `base_file`, muted,
looped, started in sync), `screen`-blended, blurred/brightened, opacity from
`dream.intensity`. Both videos' `src` are set together on scale settle / ring move.
- `#tint` (the Dark cool wash) is unchanged.
## Migration
- The baked SD variants (`*/right*.mp4`) and `bake_right_variants.py` become **legacy:
parked in-repo, not deleted, unused at runtime**. `right_variants` stays in the
manifest/`clips.py` (ignored by the client) to avoid churn; it can be removed later.
- The scale ring and its transitions are untouched.
- The affect channel is unaffected — Right is still a 04 knob, so `min(left, right)`
emotions keep working.
## Testing
- Python unit tests: `Dream.strength = right`, `intensity` scaling + `dream_gain`,
clamping, `0` at `right=0`, presence/shape in the serialized plan, `is_identity`.
- By-eye in the live sim (forest, Right up, then Left+Right for the whole picture) —
the established judge-by-eye loop. Success = the dream holds still across the loop;
only real motion (the waterfall) moves.
## Out of scope
- Removing the legacy baker / baked media (left parked, unused at runtime).
- Any change to the scale ring, transitions, Left HUD, or affect channel.
- A Pi-GPU performance pass on the Kuwahara shader (desktop sim is the current target).