feat(sim): Right axis = deterministic real-time painterly dream
Replace the pre-baked SD restyle variants (too fluid — the painting re-rolled every keyframe and the flow-warp melted frames) with a deterministic, real-time dream that holds STILL across the loop; only the footage's own motion moves. Design: docs/superpowers/specs/2026-06-22-right-axis-deterministic-dream-design.md Engine (player/alteration.py): Restyle(variant) -> Dream(strength, intensity); Calibration.right_variant_map -> dream_gain. player/state.py: a Right change is a LIVE_UPDATE now, not a crossfade (only a clip swap crossfades). Plan dict restyle -> dream. Tests migrated; 233 green. (No new Python behavior for the client-render swap below.) Renderer (simulator front-end): the Right dream is a WebGL2 Kuwahara shader on a <canvas> over the live video — edge-preserving, so motion/structure stay crisp; the dream is the same footage, just stylized (no blur). Ramped by the knob; max goes "trippy" (vivid saturation + ~45deg hue drift + posterize, concentrated near max via t=intensity^2). Phase-1 luminous-haze (CSS+bloom) was built then retired by eye (blur read as out-of-focus video). Mood grade rides as a CSS filter on the canvas; bloom layer removed. Dev ergonomics that fell out of the iteration: - /dev/version + a 1s client poll = live-reload (open tab never runs stale renderer code; reloads on my edits AND server restarts). - no-cache on the dev server; an on-screen error banner + crash-proof render so a silent throw can't masquerade as "nothing is changing". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,109 @@
|
||||
# Right axis as a real-time deterministic dream
|
||||
|
||||
**Status:** built (session 0013, 2026-06-22) — Phase 1 superseded by Phase 2 in the
|
||||
same session after by-eye review (the haze "looked like blurred video, not stylized").
|
||||
The shipped Right dream is the **Phase 2 painterly** path.
|
||||
**Surface:** simulator preview (`simulator/`, `player/alteration.py`)
|
||||
**Supersedes:** the "discrete pre-baked, flow-stabilized restyle variant" definition
|
||||
of the Right axis in the machine-altered-perception design SPEC §4.1 — a deliberate
|
||||
change to a core decision (see Why).
|
||||
|
||||
## Why
|
||||
|
||||
The pre-baked SD img2img variants (`bake_right_variants.py`) are **too fluid**: the
|
||||
painting re-rolls every keyframe (~2 s) and the optical-flow tween warps frames into
|
||||
each other, so across the loop the dream churns and melts. The right brain should be
|
||||
a **still altered state** — a calm, holistic counterpart to the busy, accreting
|
||||
analytical Left — not a restless morph.
|
||||
|
||||
## Concept
|
||||
|
||||
The Right knob no longer selects a pre-baked variant. It drives a **deterministic,
|
||||
real-time filter** applied live to the base footage, scaled 0→4. The same filter runs
|
||||
every frame, so the look is **stable by construction** — locked to scene content,
|
||||
moving only where the footage already moves. Bonus: no baked variant videos to ship,
|
||||
which simplifies the eventual hardware (Pi) story.
|
||||
|
||||
Staged delivery (both built this session; Phase 2 is what shipped):
|
||||
- **Phase 1: luminous haze** — soft-focus, blooming highlights, pastel desaturation.
|
||||
CSS + a bloom layer. Built, then **retired** by eye: blur reads as *out-of-focus
|
||||
video*, not stylized, and softens the base's crisp motion.
|
||||
- **Phase 2: painterly flatten (shipped)** — a WebGL2 **Kuwahara** shader on a
|
||||
`<canvas>` restyling the live video frames. Edge-preserving, so motion/structure
|
||||
stay as crisp as the source — the dream is the same footage, just stylized. No blur.
|
||||
|
||||
## Phase 2 — the shipped painterly dream
|
||||
|
||||
- **Renderer:** `<canvas id="paint">` over `<video>`; a `requestAnimationFrame` loop
|
||||
uploads each video frame to a texture and runs the Kuwahara fragment shader (radius
|
||||
`R=6`). `intensity = 0` ⇒ passthrough (raw base); ring transitions render passthrough
|
||||
(`busy`). WebGL-less fallback: canvas hidden, mood grade applied to `#vid`.
|
||||
- **Painterly:** `mix(base, kuwahara, intensity)` — flatten into painted regions, edges
|
||||
preserved.
|
||||
- **"Trippy" at max** (operator-tuned by eye): the psychedelic terms ramp with
|
||||
`t = intensity²` so low/mid Right stays gently dreamy and only **max** goes
|
||||
psychedelic — vivid saturation (`×(1 + 1.1·t)`), a surreal **hue drift** (~45° at max),
|
||||
and **posterize** banding (`mix(255, 6, t)` levels), plus a slight luminous lift.
|
||||
- **Mood grade** (Dark/Light) rides as a CSS filter on the canvas; `#tint` unchanged.
|
||||
- The Python plan is unchanged — Phase 2 is a client-render swap; `dream.intensity`
|
||||
drives the shader.
|
||||
|
||||
## Phase 1 — components
|
||||
|
||||
All scale with the Right strength `s = intensity` (0→1):
|
||||
|
||||
- **soft-focus** — gentle blur, ~`1.5·s` px.
|
||||
- **pastel** — `saturate(1 − 0.35·s)`, `contrast(1 − 0.15·s)` (lift shadows / flatten).
|
||||
- **luminous** — `brightness(1 + 0.08·s)`.
|
||||
- **bloom** — a second, synced copy of the video, blurred + brightened and
|
||||
`screen`-blended over the main, opacity `~0.6·s` — highlights glow and bleed.
|
||||
|
||||
Starting coefficients are by-eye seeds; the look is tuned live in the sim.
|
||||
|
||||
## Engine — `player/alteration.py`
|
||||
|
||||
- Replace `Restyle(variant)` with `Dream(strength, intensity)`:
|
||||
`strength = right` (0–4, discrete), `intensity = clamp(dream_gain · right / KNOB_MAX,
|
||||
0, 1)`.
|
||||
- `Calibration` gains `dream_gain: float = 1.0`; `DEFAULT_CALIBRATION` sets it 1.0.
|
||||
- `render_plan_to_dict`: `"restyle": {"variant"}` → `"dream": {"strength", "intensity"}`.
|
||||
- `is_identity`: Right contributes when `strength > 0` (replaces the
|
||||
`restyle.variant == 0` clause).
|
||||
- Pure + unit-tested. The client maps `dream.intensity` → CSS params, mirroring how
|
||||
`grade.tone` → video filter is mapped client-side today.
|
||||
|
||||
## Rendering — simulator
|
||||
|
||||
- **Remove** the variant-crossfade machinery (`loadVariant`, `variantFile`, the
|
||||
`right_variants` / `currentVariant` handling). The main video always plays
|
||||
`base_file`.
|
||||
- **Compose one video filter** from *both* the mood grade (`grade.tone`) and the dream
|
||||
(`dream.intensity`) — today `applyGrade` owns `vid.style.filter`; it becomes a single
|
||||
`applyVideoLook(tone, dreamIntensity)` so the two don't clobber each other.
|
||||
- **Add a bloom layer**: a second `<video id="bloom">` (same `base_file`, muted,
|
||||
looped, started in sync), `screen`-blended, blurred/brightened, opacity from
|
||||
`dream.intensity`. Both videos' `src` are set together on scale settle / ring move.
|
||||
- `#tint` (the Dark cool wash) is unchanged.
|
||||
|
||||
## Migration
|
||||
|
||||
- The baked SD variants (`*/right*.mp4`) and `bake_right_variants.py` become **legacy:
|
||||
parked in-repo, not deleted, unused at runtime**. `right_variants` stays in the
|
||||
manifest/`clips.py` (ignored by the client) to avoid churn; it can be removed later.
|
||||
- The scale ring and its transitions are untouched.
|
||||
- The affect channel is unaffected — Right is still a 0–4 knob, so `min(left, right)`
|
||||
emotions keep working.
|
||||
|
||||
## Testing
|
||||
|
||||
- Python unit tests: `Dream.strength = right`, `intensity` scaling + `dream_gain`,
|
||||
clamping, `0` at `right=0`, presence/shape in the serialized plan, `is_identity`.
|
||||
- By-eye in the live sim (forest, Right up, then Left+Right for the whole picture) —
|
||||
the established judge-by-eye loop. Success = the dream holds still across the loop;
|
||||
only real motion (the waterfall) moves.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Removing the legacy baker / baked media (left parked, unused at runtime).
|
||||
- Any change to the scale ring, transitions, Left HUD, or affect channel.
|
||||
- A Pi-GPU performance pass on the Kuwahara shader (desktop sim is the current target).
|
||||
Reference in New Issue
Block a user