feat(sim): Right axis = deterministic real-time painterly dream

Replace the pre-baked SD restyle variants (too fluid — the painting re-rolled
every keyframe and the flow-warp melted frames) with a deterministic, real-time
dream that holds STILL across the loop; only the footage's own motion moves.
Design: docs/superpowers/specs/2026-06-22-right-axis-deterministic-dream-design.md

Engine (player/alteration.py): Restyle(variant) -> Dream(strength, intensity);
Calibration.right_variant_map -> dream_gain. player/state.py: a Right change is a
LIVE_UPDATE now, not a crossfade (only a clip swap crossfades). Plan dict
restyle -> dream. Tests migrated; 233 green. (No new Python behavior for the
client-render swap below.)

Renderer (simulator front-end): the Right dream is a WebGL2 Kuwahara shader on a
<canvas> over the live video — edge-preserving, so motion/structure stay crisp;
the dream is the same footage, just stylized (no blur). Ramped by the knob; max
goes "trippy" (vivid saturation + ~45deg hue drift + posterize, concentrated near
max via t=intensity^2). Phase-1 luminous-haze (CSS+bloom) was built then retired
by eye (blur read as out-of-focus video). Mood grade rides as a CSS filter on the
canvas; bloom layer removed.

Dev ergonomics that fell out of the iteration:
- /dev/version + a 1s client poll = live-reload (open tab never runs stale
  renderer code; reloads on my edits AND server restarts).
- no-cache on the dev server; an on-screen error banner + crash-proof render so a
  silent throw can't masquerade as "nothing is changing".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ben Stull
2026-06-23 09:55:35 -07:00
parent 6adb93407b
commit cbb7c9f53d
10 changed files with 404 additions and 92 deletions
@@ -0,0 +1,109 @@
# Right axis as a real-time deterministic dream
**Status:** built (session 0013, 2026-06-22) — Phase 1 superseded by Phase 2 in the
same session after by-eye review (the haze "looked like blurred video, not stylized").
The shipped Right dream is the **Phase 2 painterly** path.
**Surface:** simulator preview (`simulator/`, `player/alteration.py`)
**Supersedes:** the "discrete pre-baked, flow-stabilized restyle variant" definition
of the Right axis in the machine-altered-perception design SPEC §4.1 — a deliberate
change to a core decision (see Why).
## Why
The pre-baked SD img2img variants (`bake_right_variants.py`) are **too fluid**: the
painting re-rolls every keyframe (~2 s) and the optical-flow tween warps frames into
each other, so across the loop the dream churns and melts. The right brain should be
a **still altered state** — a calm, holistic counterpart to the busy, accreting
analytical Left — not a restless morph.
## Concept
The Right knob no longer selects a pre-baked variant. It drives a **deterministic,
real-time filter** applied live to the base footage, scaled 0→4. The same filter runs
every frame, so the look is **stable by construction** — locked to scene content,
moving only where the footage already moves. Bonus: no baked variant videos to ship,
which simplifies the eventual hardware (Pi) story.
Staged delivery (both built this session; Phase 2 is what shipped):
- **Phase 1: luminous haze** — soft-focus, blooming highlights, pastel desaturation.
CSS + a bloom layer. Built, then **retired** by eye: blur reads as *out-of-focus
video*, not stylized, and softens the base's crisp motion.
- **Phase 2: painterly flatten (shipped)** — a WebGL2 **Kuwahara** shader on a
`<canvas>` restyling the live video frames. Edge-preserving, so motion/structure
stay as crisp as the source — the dream is the same footage, just stylized. No blur.
## Phase 2 — the shipped painterly dream
- **Renderer:** `<canvas id="paint">` over `<video>`; a `requestAnimationFrame` loop
uploads each video frame to a texture and runs the Kuwahara fragment shader (radius
`R=6`). `intensity = 0` ⇒ passthrough (raw base); ring transitions render passthrough
(`busy`). WebGL-less fallback: canvas hidden, mood grade applied to `#vid`.
- **Painterly:** `mix(base, kuwahara, intensity)` — flatten into painted regions, edges
preserved.
- **"Trippy" at max** (operator-tuned by eye): the psychedelic terms ramp with
`t = intensity²` so low/mid Right stays gently dreamy and only **max** goes
psychedelic — vivid saturation (`×(1 + 1.1·t)`), a surreal **hue drift** (~45° at max),
and **posterize** banding (`mix(255, 6, t)` levels), plus a slight luminous lift.
- **Mood grade** (Dark/Light) rides as a CSS filter on the canvas; `#tint` unchanged.
- The Python plan is unchanged — Phase 2 is a client-render swap; `dream.intensity`
drives the shader.
## Phase 1 — components
All scale with the Right strength `s = intensity` (0→1):
- **soft-focus** — gentle blur, ~`1.5·s` px.
- **pastel** — `saturate(1 0.35·s)`, `contrast(1 0.15·s)` (lift shadows / flatten).
- **luminous** — `brightness(1 + 0.08·s)`.
- **bloom** — a second, synced copy of the video, blurred + brightened and
`screen`-blended over the main, opacity `~0.6·s` — highlights glow and bleed.
Starting coefficients are by-eye seeds; the look is tuned live in the sim.
## Engine — `player/alteration.py`
- Replace `Restyle(variant)` with `Dream(strength, intensity)`:
`strength = right` (04, discrete), `intensity = clamp(dream_gain · right / KNOB_MAX,
0, 1)`.
- `Calibration` gains `dream_gain: float = 1.0`; `DEFAULT_CALIBRATION` sets it 1.0.
- `render_plan_to_dict`: `"restyle": {"variant"}``"dream": {"strength", "intensity"}`.
- `is_identity`: Right contributes when `strength > 0` (replaces the
`restyle.variant == 0` clause).
- Pure + unit-tested. The client maps `dream.intensity` → CSS params, mirroring how
`grade.tone` → video filter is mapped client-side today.
## Rendering — simulator
- **Remove** the variant-crossfade machinery (`loadVariant`, `variantFile`, the
`right_variants` / `currentVariant` handling). The main video always plays
`base_file`.
- **Compose one video filter** from *both* the mood grade (`grade.tone`) and the dream
(`dream.intensity`) — today `applyGrade` owns `vid.style.filter`; it becomes a single
`applyVideoLook(tone, dreamIntensity)` so the two don't clobber each other.
- **Add a bloom layer**: a second `<video id="bloom">` (same `base_file`, muted,
looped, started in sync), `screen`-blended, blurred/brightened, opacity from
`dream.intensity`. Both videos' `src` are set together on scale settle / ring move.
- `#tint` (the Dark cool wash) is unchanged.
## Migration
- The baked SD variants (`*/right*.mp4`) and `bake_right_variants.py` become **legacy:
parked in-repo, not deleted, unused at runtime**. `right_variants` stays in the
manifest/`clips.py` (ignored by the client) to avoid churn; it can be removed later.
- The scale ring and its transitions are untouched.
- The affect channel is unaffected — Right is still a 04 knob, so `min(left, right)`
emotions keep working.
## Testing
- Python unit tests: `Dream.strength = right`, `intensity` scaling + `dream_gain`,
clamping, `0` at `right=0`, presence/shape in the serialized plan, `is_identity`.
- By-eye in the live sim (forest, Right up, then Left+Right for the whole picture) —
the established judge-by-eye loop. Success = the dream holds still across the loop;
only real motion (the waterfall) moves.
## Out of scope
- Removing the legacy baker / baked media (left parked, unused at runtime).
- Any change to the scale ring, transitions, Left HUD, or affect channel.
- A Pi-GPU performance pass on the Kuwahara shader (desktop sim is the current target).
+37 -29
View File
@@ -1,18 +1,19 @@
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
Reconciled slice (2026-06-07): the Right axis is a DISCRETE selection of a
pre-baked, flow-stabilized restyle variant (not a continuous blend), the Left
axis carries its knob LEVEL so a runtime annotation track can pick which labels
show, and a frozen `Calibration` parameterizes the knob->strength curves so they
can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION.
Right axis (reframed session 0013): a DETERMINISTIC, real-time dream applied live
to the base footage — a STILL altered state, not the old pre-baked SD variant that
churned across the loop (Right-axis dream design,
docs/superpowers/specs/2026-06-22-right-axis-deterministic-dream-design.md). The
Left axis carries its knob LEVEL so a runtime annotation track can pick which
labels show, and a frozen `Calibration` parameterizes the knob->strength gains so
they can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION.
Layers compose per §4.2:
- Substrate: ColorGrade (Dark/Light mood, center = identity §5) + a pre-baked
Right restyle variant.
- Overlay: AnalyticalOverlay (Left), composited on top at runtime.
- Substrate: ColorGrade (Dark/Light mood, center = identity §5) + the Right Dream
(deterministic luminous haze, applied client-side from `dream.intensity`).
- Overlay: AnalyticalOverlay (Left) + the affect channel, composited on top.
Left and Right stack (different layers); Dark/Light are the two poles of one
mood grade. See docs/superpowers/specs/2026-06-07-reconciled-simulator-
alteration-slice-design.md.
mood grade.
"""
from __future__ import annotations
@@ -33,13 +34,13 @@ class Calibration:
"""Tunable knob->strength curves (settled by eye in the sim, then baked).
- mood_gain: scales the signed Dark/Light tone (result clamped to [-1, 1]).
- overlay_gain: scales the Left overlay intensity (clamped to [0, 1]).
- right_variant_map: knob value (0..4) -> pre-baked Right variant index.
- overlay_gain: scales the Left overlay (and affect) intensity (clamped [0, 1]).
- dream_gain: scales the Right deterministic-dream intensity (clamped [0, 1]).
"""
mood_gain: float = 1.0
overlay_gain: float = 1.0
right_variant_map: tuple = (0, 1, 2, 3, 4)
dream_gain: float = 1.0
# The LOCKED calibration (session 0010, 2026-06-07) — settled by eye in the
@@ -51,15 +52,15 @@ class Calibration:
# full tilt is peaceful on every axis, so no softening is warranted.
# - overlay_gain = 1.0: full Left = opacity 1.0; the HUD is legible, not
# overwhelming, sitting above the grade.
# - right_variant_map linear: the 5 knob notches map 1:1 onto the 5 discrete
# pre-baked Right strengths (0 = raw base).
# These are unity/linear by deliberate choice, not as placeholders. The curve
# stays parameterized so a future re-bake or a different feel is one edit away;
# test_default_calibration_is_locked guards the values from drifting silently.
# - dream_gain = 1.0: full Right = dream intensity 1.0; the deterministic
# luminous haze (Right-axis dream design, session 0013) reaches full strength.
# These are unity by deliberate choice, not as placeholders. The gains stay
# parameterized so a different feel is one edit away; test_default_calibration_is_locked
# guards the values from drifting silently.
DEFAULT_CALIBRATION = Calibration(
mood_gain=1.0,
overlay_gain=1.0,
right_variant_map=(0, 1, 2, 3, 4),
dream_gain=1.0,
)
@@ -98,11 +99,15 @@ class AffectOverlay:
@dataclass(frozen=True)
class Restyle:
"""Right axis (§4.1): selects a pre-baked, flow-stabilized restyle variant.
`variant` is a discrete index (0 = raw base, no restyle)."""
class Dream:
"""Right axis: a deterministic, real-time dream applied live to the base
footage — a STILL altered state (Right-axis dream design, session 0013),
NOT the old pre-baked SD variant that churned across the loop. `strength` is
the Right knob (0..4); `intensity` 0..1 is the dream's strength (the client
maps it to the luminous-haze look, the way it maps `grade.tone` to a filter)."""
variant: int
strength: int
intensity: float
@dataclass(frozen=True)
@@ -112,7 +117,7 @@ class RenderPlan:
grade: ColorGrade
overlay: AnalyticalOverlay
affect: AffectOverlay
restyle: Restyle
dream: Dream
@property
def is_identity(self) -> bool:
@@ -122,7 +127,7 @@ class RenderPlan:
return (
self.grade.is_identity
and self.overlay.level == 0
and self.restyle.variant == 0
and self.dream.strength == 0
)
@@ -139,8 +144,8 @@ def _affect_intensity(left: int, right: int, cal: Calibration) -> float:
return _clamp(cal.overlay_gain * _affect_strength(left, right) / KNOB_MAX, 0.0, 1.0)
def _right_variant(right: int, cal: Calibration) -> int:
return cal.right_variant_map[right]
def _dream_intensity(right: int, cal: Calibration) -> float:
return _clamp(cal.dream_gain * right / KNOB_MAX, 0.0, 1.0)
def _mood_tone(dark: int, light: int, cal: Calibration) -> float:
@@ -161,7 +166,10 @@ def plan_alteration(
strength=_affect_strength(coord.left, coord.right),
intensity=_affect_intensity(coord.left, coord.right, calibration),
),
restyle=Restyle(variant=_right_variant(coord.right, calibration)),
dream=Dream(
strength=coord.right,
intensity=_dream_intensity(coord.right, calibration),
),
)
@@ -171,6 +179,6 @@ def render_plan_to_dict(plan: RenderPlan) -> dict:
"grade": {"tone": plan.grade.tone},
"overlay": {"level": plan.overlay.level, "intensity": plan.overlay.intensity},
"affect": {"strength": plan.affect.strength, "intensity": plan.affect.intensity},
"restyle": {"variant": plan.restyle.variant},
"dream": {"strength": plan.dream.strength, "intensity": plan.dream.intensity},
"is_identity": plan.is_identity,
}
+7 -5
View File
@@ -3,10 +3,10 @@
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
desired Playback (which neutral base clip, how it is altered, what audio plays,
at what levels) and returns the Transition from the previous Playback. The
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
pre-baked Right restyle variant (a discrete index) needs a CROSSFADE, and
toggling video on/off fades to/from black.
transition KIND encodes design §4.3: the Dark/Light grade, the Left overlay AND
the Right dream (a deterministic, live luminous haze since session 0013) are all
continuous runtime ops (LIVE_UPDATE); only swapping the clip needs a CROSSFADE,
and toggling video on/off fades to/from black.
Knobs no longer *select* a clip (the base library is neutral by construction);
they *transform* it. Which neutral base to show is an injected policy
@@ -107,7 +107,9 @@ class Player:
if next_video and not prev_video:
return TransitionKind.FADE_FROM_BLACK
if next_video and prev_video:
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
# The Right dream is a live filter now (not a substrate swap), so it
# no longer forces a crossfade — only a clip (scale) change does.
if nxt.clip_id != prev.clip_id:
return TransitionKind.CROSSFADE
return TransitionKind.LIVE_UPDATE
# both black: only audio/levels could have changed
+29 -2
View File
@@ -7,6 +7,9 @@ endpoints (/api/select, /api/catalog/meta) and the X-ray are retired.
from __future__ import annotations
import hashlib
import os
import time
from pathlib import Path
from typing import Optional
@@ -30,6 +33,24 @@ STATIC_DIR = Path(__file__).parent / "static"
MEDIA_DIR = Path(__file__).parent / "sample_media"
DEFAULT_MANIFEST = MEDIA_DIR / "manifest.json"
# Per-process boot token: changes on every server restart so the dev live-reload
# (below) also fires when Python code changes (which needs a restart), not just
# when a static asset's mtime changes.
_BOOT_ID = f"{os.getpid()}-{int(time.time())}"
def _asset_version() -> str:
"""A short digest of the renderer assets' mtimes + this process's boot token.
Changes whenever app.js/style.css/index.html change on disk OR the server is
restarted — the signal the browser polls to know it's running stale code."""
parts = [_BOOT_ID]
for name in ("app.js", "style.css", "index.html"):
try:
parts.append(f"{name}:{(STATIC_DIR / name).stat().st_mtime_ns}")
except OSError:
parts.append(f"{name}:?")
return hashlib.sha1("|".join(parts).encode()).hexdigest()[:16]
class ControlsModel(BaseModel):
content: str
@@ -44,7 +65,7 @@ class ControlsModel(BaseModel):
class CalibrationModel(BaseModel):
mood_gain: float = 1.0
overlay_gain: float = 1.0
right_variant_map: list[int] = [0, 1, 2, 3, 4]
dream_gain: float = 1.0
class AlterationRequest(BaseModel):
@@ -86,7 +107,7 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
Calibration(
mood_gain=req.calibration.mood_gain,
overlay_gain=req.calibration.overlay_gain,
right_variant_map=tuple(req.calibration.right_variant_map),
dream_gain=req.calibration.dream_gain,
)
if req.calibration
else DEFAULT_CALIBRATION
@@ -98,6 +119,12 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
"content": {"audio_source": content.audio_source, "video": content.video},
}
@app.get("/dev/version")
def dev_version():
# Dev live-reload signal: the browser polls this and reloads when it
# changes, so it never silently runs a stale renderer during iteration.
return {"version": _asset_version()}
@app.get("/api/clips")
def api_clips():
return {"clips": [c.to_dict() for c in app.state.clips]}
+181 -34
View File
@@ -1,16 +1,39 @@
// Thin renderer: post controls+calibration -> RenderPlan; render grade, Right
// variant crossfade, and the live Left overlay. All alteration math stays in
// Python. The scale RING (endless encoder) is navigated via /api/ring/advance —
// Python owns the step/wrap/transition-selection math; the browser only plays
// the returned transition clip(s) then settles on the target scale's clip.
// Thin renderer: post controls+calibration -> RenderPlan; render the mood grade,
// the live Right DREAM (a deterministic luminous haze, NOT a baked variant — it
// holds still across the loop), and the live Left overlay + affect. All alteration
// math stays in Python. The scale RING (endless encoder) is navigated via
// /api/ring/advance — Python owns the step/wrap/transition-selection math; the
// browser only plays the returned transition clip(s) then settles on the target.
const $ = (id) => document.getElementById(id);
const vid = $("vid"), tint = $("tint"), overlay = $("overlay"), black = $("black"), readout = $("readout");
const vid = $("vid"), paint = $("paint"), tint = $("tint"), overlay = $("overlay"),
black = $("black"), readout = $("readout");
const affectLayer = $("affect");
// Right-dream state, read by the painterly render loop each frame. update() (debounced
// on knob moves) writes these; the WebGL loop renders the live video continuously.
let dreamIntensity = 0; // 0..1 — painterly strength (and dream pastel/luminous)
let gradeFilter = "none"; // CSS filter string for the Dark/Light mood grade
// Surface any uncaught JS error on-screen (a silent throw in update() otherwise
// looks like "nothing is changing" — the render dies but the page sits there).
function showError(msg) {
let b = document.getElementById("err-banner");
if (!b) {
b = document.createElement("div");
b.id = "err-banner";
b.style.cssText = "position:fixed;top:0;left:0;right:0;z-index:9999;background:#a00;" +
"color:#fff;font:12px/1.4 monospace;padding:6px 10px;white-space:pre-wrap;";
document.body.appendChild(b);
}
b.textContent = "⚠ " + msg;
}
window.addEventListener("error", (e) =>
showError(`${e.message} @ ${(e.filename || "").split("/").pop()}:${e.lineno}`));
let clipsById = {}; // id -> clip manifest entry
let ring = null; // {scales:[{id,clip_id,title}], transitions:[...]} or null
let ringIndex = 0; // current scale position on the ring
let currentVariant = -1; // last loaded Right strength (reset on clip change)
let currentClipId = null; // base media currently loaded (reset to force a reload)
let busy = false; // true while a ring transition is playing
async function loadData() {
@@ -32,36 +55,134 @@ function activeClip() {
function mediaUrl(file) { return "/media/" + file; }
function variantFile(strength) {
// Load the active scale's BASE footage into the video (the painterly canvas reads
// its frames live). The Right knob no longer swaps the source — it only drives the
// deterministic dream restyle — so media reloads only when the SCALE changes.
function ensureClipMedia() {
const clip = activeClip();
if (!clip) return "";
const v = clip.right_variants[String(strength)];
return v ? v.file : clip.base_file;
if (!clip || clip.id === currentClipId) return;
currentClipId = clip.id;
vid.src = mediaUrl(clip.base_file);
vid.loop = true;
vid.muted = true;
vid.play().catch(() => {});
vid.style.opacity = "1";
}
function applyGrade(tone) {
// Light: warm + brighten (sepia). Dark: cool + darken via a multiply-blended
// blue wash (#tint) that lifts shadows toward blue while keeping natural
// greens — the peaceful POC dark look, NOT a full-frame hue spin.
// Compose the video look. The Right DREAM is now a painterly WebGL restyle of the
// live frames (see the render loop below) — so here we only stash its intensity for
// that loop, and build the Dark/Light mood grade as a CSS filter. The grade rides
// on the painterly canvas (or #vid in the WebGL-less fallback). No blur: the dream
// must keep the base's crisp motion, just stylized.
function applyVideoLook(tone, dream) {
const warm = tone > 0 ? tone : 0, cool = tone < 0 ? -tone : 0;
const bright = 1 + 0.25 * warm - 0.35 * cool;
const sat = 1 + 0.15 * warm - 0.30 * cool;
vid.style.filter =
`brightness(${bright.toFixed(3)}) saturate(${sat.toFixed(3)}) ` +
`sepia(${(warm * 0.5).toFixed(3)})`;
const sepia = warm * 0.5;
gradeFilter = `brightness(${bright.toFixed(3)}) saturate(${sat.toFixed(3)}) sepia(${sepia.toFixed(3)})`;
dreamIntensity = dream;
tint.style.opacity = (cool * 0.6).toFixed(3);
// WebGL-less fallback: the canvas is hidden, so grade the visible #vid directly.
if (!paintOK) vid.style.filter = gradeFilter;
}
function loadVariant(strength) {
if (strength === currentVariant) return;
currentVariant = strength;
vid.style.opacity = "0";
setTimeout(() => {
vid.src = mediaUrl(variantFile(strength));
vid.loop = true;
vid.play().catch(() => {});
vid.style.opacity = "1";
}, 150);
// --- Right dream: real-time painterly restyle (WebGL2 Kuwahara) ---
// An edge-preserving "oil painting" filter on the LIVE video frames: each pixel
// becomes the mean of whichever surrounding quadrant is most uniform, flattening
// the image into painted regions WHILE KEEPING EDGES — so motion stays as crisp as
// the source. Strength + a gentle pastel/luminous dream-grade scale with intensity.
// Deterministic (a pure function of each frame) => the dream holds still across the
// loop; only the footage's own motion moves.
let paintOK = false;
const PAINT_VS = `#version 300 es
in vec2 p; out vec2 v_uv;
void main(){
// Fullscreen triangle (verts to +3); map the visible [-1,1] to uv [0,1] with Y
// flipped for the video's top-left origin. (Visible region stays in range; the
// off-screen excess is clipped.)
v_uv = vec2(p.x * 0.5 + 0.5, 0.5 - p.y * 0.5);
gl_Position = vec4(p, 0.0, 1.0);
}`;
const PAINT_FS = `#version 300 es
precision highp float;
in vec2 v_uv; out vec4 frag;
uniform sampler2D u_tex; uniform vec2 u_texel; uniform float u_amt;
#define R 6
#define QUAD(x0,x1,y0,y1,MO,VO) { vec3 m=vec3(0.0),s=vec3(0.0); \
for(int j=y0;j<=y1;j++){ for(int i=x0;i<=x1;i++){ \
vec3 c=texture(u_tex, v_uv+vec2(float(i),float(j))*u_texel).rgb; m+=c; s+=c*c; } } \
float n=float((x1-x0+1)*(y1-y0+1)); m/=n; vec3 vv=s/n-m*m; MO=m; VO=vv.r+vv.g+vv.b; }
vec3 hueRotate(vec3 c, float a){
float co=cos(a), si=sin(a);
return vec3(
c.r*(0.299+0.701*co+0.168*si) + c.g*(0.587-0.587*co+0.330*si) + c.b*(0.114-0.114*co-0.497*si),
c.r*(0.299-0.299*co-0.328*si) + c.g*(0.587+0.413*co+0.035*si) + c.b*(0.114-0.114*co+0.292*si),
c.r*(0.299-0.300*co+1.250*si) + c.g*(0.587-0.588*co-1.050*si) + c.b*(0.114+0.886*co-0.203*si));
}
void main(){
vec3 base = texture(u_tex, v_uv).rgb;
if (u_amt <= 0.001) { frag = vec4(base, 1.0); return; }
vec3 m0,m1,m2,m3; float v0,v1,v2,v3;
QUAD(-R,0,-R,0,m0,v0) QUAD(0,R,-R,0,m1,v1) QUAD(-R,0,0,R,m2,v2) QUAD(0,R,0,R,m3,v3)
vec3 painted=m0; float mv=v0;
if(v1<mv){mv=v1;painted=m1;} if(v2<mv){mv=v2;painted=m2;} if(v3<mv){mv=v3;painted=m3;}
vec3 styl = mix(base, painted, u_amt); // painterly restyle, linear in the knob
// The "trippy" terms ramp with t = amt^2, so low Right stays gently dreamy and
// only MAX goes psychedelic: vivid saturation, a surreal hue drift, poster banding.
float t = u_amt * u_amt;
float luma = dot(styl, vec3(0.299, 0.587, 0.114));
styl = mix(vec3(luma), styl, 1.0 + 1.1 * t); // saturation: vivid toward max
styl = hueRotate(styl, 0.8 * t); // surreal hue drift (~45deg at max)
float levels = mix(255.0, 6.0, t); // posterize: color banding toward max
styl = floor(styl * levels + 0.5) / levels;
styl *= 1.0 + 0.06 * u_amt; // luminous lift
frag = vec4(clamp(styl, 0.0, 1.0), 1.0);
}`;
let gl = null, uAmt = null, uTexel = null;
function _shader(kind, src) {
const s = gl.createShader(kind);
gl.shaderSource(s, src); gl.compileShader(s);
if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) throw new Error(gl.getShaderInfoLog(s));
return s;
}
function initPaint() {
gl = paint.getContext("webgl2");
if (!gl) { paint.style.display = "none"; return false; } // CSS fallback grades #vid
const prog = gl.createProgram();
gl.attachShader(prog, _shader(gl.VERTEX_SHADER, PAINT_VS));
gl.attachShader(prog, _shader(gl.FRAGMENT_SHADER, PAINT_FS));
gl.bindAttribLocation(prog, 0, "p"); gl.linkProgram(prog);
if (!gl.getProgramParameter(prog, gl.LINK_STATUS)) throw new Error(gl.getProgramInfoLog(prog));
gl.useProgram(prog);
const buf = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buf);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1, -1, 3, -1, -1, 3]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(0); gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);
const t = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, t);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
uAmt = gl.getUniformLocation(prog, "u_amt");
uTexel = gl.getUniformLocation(prog, "u_texel");
paintOK = true;
requestAnimationFrame(paintLoop);
return true;
}
function paintLoop() {
if (paintOK && vid.readyState >= 2 && vid.videoWidth) {
const w = vid.videoWidth, h = vid.videoHeight;
if (paint.width !== w || paint.height !== h) { paint.width = w; paint.height = h; }
gl.viewport(0, 0, w, h);
try { gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, vid); } catch (_) {}
// No painterly restyle during a ring transition (busy) — show it raw.
gl.uniform1f(uAmt, busy ? 0 : dreamIntensity);
gl.uniform2f(uTexel, 1 / w, 1 / h);
gl.drawArrays(gl.TRIANGLES, 0, 3);
paint.style.filter = busy ? "none" : gradeFilter;
}
requestAnimationFrame(paintLoop);
}
const SVGNS = "http://www.w3.org/2000/svg";
@@ -153,6 +274,7 @@ function renderOverlay(level, intensity) {
// softer than the clinical reticles — the dream leaking into the machine's read.
function renderAffect(strength, intensity) {
const clip = activeClip();
if (!affectLayer) return; // tolerate a stale page missing the affect layer
affectLayer.innerHTML = "";
if (!clip || !clip.affect || strength <= 0) { affectLayer.style.opacity = "0"; return; }
affectLayer.style.opacity = String(intensity);
@@ -182,7 +304,7 @@ function controls() {
function calibration() {
return { mood_gain: +$("mood_gain").value, overlay_gain: +$("overlay_gain").value,
right_variant_map: [0, 1, 2, 3, 4] };
dream_gain: 1.0 };
}
let timer = null;
@@ -197,10 +319,17 @@ async function update() {
readout.textContent = JSON.stringify(data, null, 2);
if (!data.content.video) { black.classList.remove("hidden"); return; }
black.classList.add("hidden");
applyGrade(data.plan.grade.tone);
loadVariant(data.plan.restyle.variant);
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
renderAffect(data.plan.affect.strength, data.plan.affect.intensity);
try {
ensureClipMedia();
applyVideoLook(data.plan.grade.tone, data.plan.dream.intensity);
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
renderAffect(data.plan.affect.strength, data.plan.affect.intensity);
const b = document.getElementById("err-banner");
if (b) b.remove(); // render succeeded — clear any prior error
} catch (err) {
showError("render failed: " + (err && err.message ? err.message : err));
throw err;
}
}
function debounced() { clearTimeout(timer); timer = setTimeout(update, 80); }
@@ -216,6 +345,7 @@ function playTransition(file, blended) {
// Resolves on 'ended' (or a safety timeout that scales with playback rate).
return new Promise((resolve) => {
overlay.style.opacity = "0";
affectLayer.style.opacity = "0";
tint.style.opacity = "0";
vid.style.filter = "none";
vid.loop = false;
@@ -252,7 +382,7 @@ async function advance(delta) {
await playTransition(step.file, step.blended);
}
ringIndex = move.to_index;
currentVariant = -1; // force the target clip's variant to (re)load
currentClipId = null; // force the target scale's base media to (re)load
renderScaleReadout();
} finally {
busy = false;
@@ -272,7 +402,24 @@ function onWheel(e) {
}, 90);
}
// Dev live-reload: poll the asset version and reload when it changes, so an open
// tab never keeps running a stale renderer while we iterate (the readout updates
// from the live API and masks it otherwise). Reloads on my edits AND on a server
// restart. Dev-only convenience; harmless if the endpoint is absent.
function devLiveReload() {
let seen = null;
setInterval(async () => {
try {
const v = (await (await fetch("/dev/version", { cache: "no-store" })).json()).version;
if (seen === null) seen = v;
else if (v !== seen) location.reload();
} catch (_) { /* server restarting / endpoint absent — ignore */ }
}, 1000);
}
async function main() {
devLiveReload();
try { initPaint(); } catch (e) { paintOK = false; paint.style.display = "none"; showError("WebGL init: " + e.message); }
await loadData();
renderScaleReadout();
for (const id of ["content", "left", "right", "dark", "light", "mood_gain", "overlay_gain"]) {
+1
View File
@@ -12,6 +12,7 @@
<section class="stage" id="stage">
<div class="screen">
<video id="vid" loop muted playsinline></video>
<canvas id="paint"></canvas>
<div id="tint"></div>
<svg id="overlay" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
<svg id="affect" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
+6
View File
@@ -7,6 +7,12 @@ main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; }
.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000;
border-radius: 6px; overflow: hidden; }
#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; }
/* Right-dream painterly canvas: a WebGL Kuwahara restyle of the LIVE video frames,
drawn over the base. Edge-preserving, so motion stays as crisp as the source
the dream is the same footage, just stylized. The mood grade rides as a CSS
filter here. Hidden when WebGL is unavailable (CSS fallback shows #vid). */
#paint { position: absolute; inset: 0; width: 100%; height: 100%; object-fit: cover;
pointer-events: none; transition: filter 0.2s ease; }
#tint { position: absolute; inset: 0; pointer-events: none; opacity: 0;
background: #28425f; mix-blend-mode: multiply;
transition: opacity 0.2s ease; }
+25 -16
View File
@@ -7,8 +7,8 @@ from player.alteration import (
AnalyticalOverlay,
Calibration,
ColorGrade,
Dream,
RenderPlan,
Restyle,
plan_alteration,
render_plan_to_dict,
)
@@ -23,7 +23,8 @@ def test_all_zero_knobs_is_the_unaltered_base():
assert plan.is_identity
assert plan.overlay.level == 0
assert plan.overlay.intensity == 0.0
assert plan.restyle.variant == 0
assert plan.dream.strength == 0
assert plan.dream.intensity == 0.0
assert plan.grade.tone == 0.0
assert plan.grade.is_identity
@@ -32,21 +33,26 @@ def test_left_drives_the_analytical_overlay_only():
plan = plan_alteration(_coord(left=4))
assert plan.overlay.level == 4
assert plan.overlay.intensity == 1.0
assert plan.restyle.variant == 0 # Left does not touch the substrate
assert plan.dream.strength == 0 # Left does not touch the substrate
assert plan.dream.intensity == 0.0
assert plan.grade.tone == 0.0
def test_right_selects_a_discrete_restyle_variant_only():
def test_right_drives_the_deterministic_dream_only():
# Right-axis dream reframe (session 0013): Right is a live deterministic
# haze, carried as strength (0..4) + intensity (0..1), not a baked variant.
plan = plan_alteration(_coord(right=2))
assert plan.restyle.variant == 2
assert plan.dream.strength == 2
assert plan.dream.intensity == pytest.approx(0.5)
assert plan.overlay.level == 0 # Right does not add overlay
def test_left_and_right_stack_not_cancel():
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
# design §4.2: whole-brain corner = dream substrate WITH labels on top
plan = plan_alteration(_coord(left=4, right=4))
assert plan.overlay.level == 4
assert plan.restyle.variant == 4
assert plan.dream.strength == 4
assert plan.dream.intensity == 1.0
def test_affect_needs_both_knobs_up():
@@ -112,7 +118,8 @@ def test_dark_minus_light_sets_intermediate_tone():
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
assert plan.overlay.level == 4
assert plan.restyle.variant == 2
assert plan.dream.strength == 2
assert plan.dream.intensity == pytest.approx(0.5)
assert plan.grade.tone == -1.0
assert not plan.is_identity
@@ -124,32 +131,34 @@ def test_default_calibration_is_locked():
# locked feel is a deliberate edit here + in alteration.py.
assert DEFAULT_CALIBRATION.mood_gain == 1.0
assert DEFAULT_CALIBRATION.overlay_gain == 1.0
assert DEFAULT_CALIBRATION.right_variant_map == (0, 1, 2, 3, 4)
assert DEFAULT_CALIBRATION.dream_gain == 1.0
def test_default_calibration_is_behavior_preserving():
# DEFAULT_CALIBRATION must reproduce the original three helpers exactly.
# DEFAULT_CALIBRATION reproduces the per-axis helpers exactly.
for left in range(5):
assert plan_alteration(_coord(left=left)).overlay.intensity == pytest.approx(left / 4)
for right in range(5):
assert plan_alteration(_coord(right=right)).restyle.variant == right
assert plan_alteration(_coord(right=right)).dream.strength == right
assert plan_alteration(_coord(right=right)).dream.intensity == pytest.approx(right / 4)
for dark in range(5):
for light in range(5):
expected = (light - dark) / 4
assert plan_alteration(_coord(dark=dark, light=light)).grade.tone == pytest.approx(expected)
def test_custom_calibration_scales_mood_and_overlay():
cal = Calibration(mood_gain=0.5, overlay_gain=0.5, right_variant_map=(0, 0, 1, 1, 2))
def test_custom_calibration_scales_mood_overlay_and_dream():
cal = Calibration(mood_gain=0.5, overlay_gain=0.5, dream_gain=0.5)
assert plan_alteration(_coord(light=4), cal).grade.tone == pytest.approx(0.5)
assert plan_alteration(_coord(left=4), cal).overlay.intensity == pytest.approx(0.5)
assert plan_alteration(_coord(right=3), cal).restyle.variant == 1
assert plan_alteration(_coord(right=4), cal).dream.intensity == pytest.approx(0.5)
def test_calibration_gain_is_clamped_to_unit_range():
cal = Calibration(mood_gain=10.0, overlay_gain=10.0)
cal = Calibration(mood_gain=10.0, overlay_gain=10.0, dream_gain=10.0)
assert plan_alteration(_coord(light=4), cal).grade.tone == 1.0 # clamped, not 10
assert plan_alteration(_coord(left=4), cal).overlay.intensity == 1.0
assert plan_alteration(_coord(right=4), cal).dream.intensity == 1.0
def test_render_plan_to_dict_round_trips_the_numbers():
@@ -158,7 +167,7 @@ def test_render_plan_to_dict_round_trips_the_numbers():
"grade": {"tone": -1.0},
"overlay": {"level": 4, "intensity": 1.0},
"affect": {"strength": 2, "intensity": 0.5},
"restyle": {"variant": 2},
"dream": {"strength": 2, "intensity": 0.5},
"is_identity": False,
}
+6 -4
View File
@@ -59,13 +59,15 @@ def test_overlay_change_is_a_live_update():
assert t.playback.plan.overlay.level == 4
def test_restyle_change_crossfades_the_substrate():
# design §4.3: the Right v2v substrate is a pre-baked variant swap
def test_dream_change_is_a_live_update_not_a_crossfade():
# Right-axis dream reframe (session 0013): the Right dream is a deterministic
# LIVE filter now, so changing it no longer crossfades — only a clip swap does.
p = Player(LIB)
p.update(_controls(content="video", right=0))
t = p.update(_controls(content="video", right=4))
assert t.kind == TransitionKind.CROSSFADE
assert t.playback.plan.restyle.variant == 4
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.dream.strength == 4
assert t.playback.plan.dream.intensity == 1.0
def test_volume_only_change_is_a_live_update():
+3 -2
View File
@@ -38,7 +38,8 @@ def test_alteration_returns_the_engine_plan(client):
assert resp.status_code == 200
data = resp.json()
assert data["plan"]["overlay"]["level"] == 4
assert data["plan"]["restyle"]["variant"] == 2
assert data["plan"]["dream"]["strength"] == 2
assert data["plan"]["dream"]["intensity"] == 0.5
assert data["plan"]["grade"]["tone"] == -1.0
assert data["content"]["video"] is True
@@ -51,7 +52,7 @@ def test_alteration_honors_off_as_black(client):
def test_alteration_accepts_calibration(client):
body = {"controls": _controls(light=4),
"calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "right_variant_map": [0, 1, 2, 3, 4]}}
"calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "dream_gain": 1.0}}
resp = client.post("/api/alteration", json=body)
assert resp.json()["plan"]["grade"]["tone"] == 0.5