Merge pull request 'feat(sim): Left HUD redesign · emotion channel · deterministic painterly Right dream · 3-knob control' (#12) from feature/left-hud-look-and-feel into main
This commit was merged in pull request #12.
This commit is contained in:
@@ -0,0 +1,85 @@
|
||||
# Affect channel — emotions in the Left-brain HUD
|
||||
|
||||
**Status:** approved (session 0013, 2026-06-22)
|
||||
**Surface:** simulator preview (`simulator/`, `player/alteration.py`)
|
||||
**Builds on:** the machine-altered-perception design SPEC; the Left HUD look-and-feel
|
||||
pass (same session).
|
||||
|
||||
## Idea
|
||||
|
||||
A third HUD channel beside the cyan *detections* and amber *measurements*: soft,
|
||||
glowing **emotion-words** that surface the feelings the experience may invoke.
|
||||
They appear only when *both* the analytical (Left) and dreamlike (Right) knobs are
|
||||
up — the dream leaking into the machine's reading. This is the uncanny edge of the
|
||||
thesis: the machine trying to quantify not just what is there, but how you are
|
||||
meant to feel.
|
||||
|
||||
## Behaviour
|
||||
|
||||
- **Trigger / intensity.** Affect strength = `min(left, right)` (0–4). Either knob
|
||||
at 0 → no emotions at all. Opacity scales with that strength × `overlay_gain`.
|
||||
- **Escalation.** A **stable emotional palette per scene** — higher combined
|
||||
strength reveals *more* words at higher opacity. Each word carries a `min_level`
|
||||
(the same mechanic the detection channel uses), but compared against the
|
||||
*combined* `min(left, right)` strength rather than Left alone.
|
||||
- **Rendering.** Floating words at authored scene positions — **no boxes or
|
||||
reticles**, lowercase, semi-transparent with a soft glow, in a distinct **violet**
|
||||
register so they read as affective and clearly *softer* than the hard clinical
|
||||
reticles.
|
||||
|
||||
## Where the math lives
|
||||
|
||||
Consistent with the existing split (alteration math in Python, label-selection in
|
||||
the client):
|
||||
|
||||
- `player/alteration.py` gains an `AffectOverlay(strength, intensity)` on the
|
||||
`RenderPlan`. `strength = min(left, right)`; `intensity = clamp(overlay_gain *
|
||||
min(left,right) / KNOB_MAX, 0, 1)`. Returned in `render_plan_to_dict` under
|
||||
`"affect"`. Pure, unit-tested.
|
||||
- The simulator client picks *which* words to show by comparing each affect
|
||||
entry's `min_level` to `plan.affect.strength`, and renders them at
|
||||
`plan.affect.intensity` opacity.
|
||||
|
||||
`is_identity` is unaffected: `min(left,right) > 0` implies `left > 0`, which already
|
||||
makes `overlay.level > 0`.
|
||||
|
||||
## Data
|
||||
|
||||
New optional per-clip `affect` list in the manifest, parallel to `annotations`:
|
||||
|
||||
```json
|
||||
"affect": [
|
||||
{"key": "feel.awe", "at": [0.5, 0.4], "min_level": 1},
|
||||
{"key": "feel.serenity", "at": [0.2, 0.62], "min_level": 2}
|
||||
],
|
||||
"strings": {"en": {"feel.awe": "awe", "feel.serenity": "serenity"}}
|
||||
```
|
||||
|
||||
`at` is a normalized `[x, y]` anchor point (not a box — feelings are scene-level).
|
||||
The words live in the existing `strings.en` map. `Clip` gains an `affect` field
|
||||
(defaulting to `[]`) threaded through `_clip_from_dict` and `to_dict`.
|
||||
|
||||
Starter palettes (words are art-direction, easily tuned):
|
||||
|
||||
| scale | palette (min_level 1 → 4) |
|
||||
|--------|-----------------------------------------------|
|
||||
| forest | awe · serenity · reverence · stillness |
|
||||
| cosmos | wonder · vastness · insignificance · longing |
|
||||
| abyss | unease · fascination · isolation · dread |
|
||||
|
||||
All three scales carry the data; forest is fully realized (it has the real Right
|
||||
variants).
|
||||
|
||||
## Testing
|
||||
|
||||
- Python unit tests: affect `strength = min(left, right)` across knob combos;
|
||||
intensity scaling and `overlay_gain`; `0` when either knob is 0; presence in the
|
||||
serialized plan.
|
||||
- By-eye in the live simulator (forest, Left & Right both up) — the established
|
||||
judge-by-eye loop for this surface.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- No drift-toward-unsettling arc (rejected in favour of a stable palette).
|
||||
- No per-object emotion tags; no animated drift (words are statically placed for
|
||||
v1; gentle motion can come later if wanted).
|
||||
@@ -0,0 +1,109 @@
|
||||
# Right axis as a real-time deterministic dream
|
||||
|
||||
**Status:** built (session 0013, 2026-06-22) — Phase 1 superseded by Phase 2 in the
|
||||
same session after by-eye review (the haze "looked like blurred video, not stylized").
|
||||
The shipped Right dream is the **Phase 2 painterly** path.
|
||||
**Surface:** simulator preview (`simulator/`, `player/alteration.py`)
|
||||
**Supersedes:** the "discrete pre-baked, flow-stabilized restyle variant" definition
|
||||
of the Right axis in the machine-altered-perception design SPEC §4.1 — a deliberate
|
||||
change to a core decision (see Why).
|
||||
|
||||
## Why
|
||||
|
||||
The pre-baked SD img2img variants (`bake_right_variants.py`) are **too fluid**: the
|
||||
painting re-rolls every keyframe (~2 s) and the optical-flow tween warps frames into
|
||||
each other, so across the loop the dream churns and melts. The right brain should be
|
||||
a **still altered state** — a calm, holistic counterpart to the busy, accreting
|
||||
analytical Left — not a restless morph.
|
||||
|
||||
## Concept
|
||||
|
||||
The Right knob no longer selects a pre-baked variant. It drives a **deterministic,
|
||||
real-time filter** applied live to the base footage, scaled 0→4. The same filter runs
|
||||
every frame, so the look is **stable by construction** — locked to scene content,
|
||||
moving only where the footage already moves. Bonus: no baked variant videos to ship,
|
||||
which simplifies the eventual hardware (Pi) story.
|
||||
|
||||
Staged delivery (both built this session; Phase 2 is what shipped):
|
||||
- **Phase 1: luminous haze** — soft-focus, blooming highlights, pastel desaturation.
|
||||
CSS + a bloom layer. Built, then **retired** by eye: blur reads as *out-of-focus
|
||||
video*, not stylized, and softens the base's crisp motion.
|
||||
- **Phase 2: painterly flatten (shipped)** — a WebGL2 **Kuwahara** shader on a
|
||||
`<canvas>` restyling the live video frames. Edge-preserving, so motion/structure
|
||||
stay as crisp as the source — the dream is the same footage, just stylized. No blur.
|
||||
|
||||
## Phase 2 — the shipped painterly dream
|
||||
|
||||
- **Renderer:** `<canvas id="paint">` over `<video>`; a `requestAnimationFrame` loop
|
||||
uploads each video frame to a texture and runs the Kuwahara fragment shader (radius
|
||||
`R=6`). `intensity = 0` ⇒ passthrough (raw base); ring transitions render passthrough
|
||||
(`busy`). WebGL-less fallback: canvas hidden, mood grade applied to `#vid`.
|
||||
- **Painterly:** `mix(base, kuwahara, intensity)` — flatten into painted regions, edges
|
||||
preserved.
|
||||
- **"Trippy" at max** (operator-tuned by eye): the psychedelic terms ramp with
|
||||
`t = intensity²` so low/mid Right stays gently dreamy and only **max** goes
|
||||
psychedelic — vivid saturation (`×(1 + 1.1·t)`), a surreal **hue drift** (~45° at max),
|
||||
and **posterize** banding (`mix(255, 6, t)` levels), plus a slight luminous lift.
|
||||
- **Mood grade** (Dark/Light) rides as a CSS filter on the canvas; `#tint` unchanged.
|
||||
- The Python plan is unchanged — Phase 2 is a client-render swap; `dream.intensity`
|
||||
drives the shader.
|
||||
|
||||
## Phase 1 — components
|
||||
|
||||
All scale with the Right strength `s = intensity` (0→1):
|
||||
|
||||
- **soft-focus** — gentle blur, ~`1.5·s` px.
|
||||
- **pastel** — `saturate(1 − 0.35·s)`, `contrast(1 − 0.15·s)` (lift shadows / flatten).
|
||||
- **luminous** — `brightness(1 + 0.08·s)`.
|
||||
- **bloom** — a second, synced copy of the video, blurred + brightened and
|
||||
`screen`-blended over the main, opacity `~0.6·s` — highlights glow and bleed.
|
||||
|
||||
Starting coefficients are by-eye seeds; the look is tuned live in the sim.
|
||||
|
||||
## Engine — `player/alteration.py`
|
||||
|
||||
- Replace `Restyle(variant)` with `Dream(strength, intensity)`:
|
||||
`strength = right` (0–4, discrete), `intensity = clamp(dream_gain · right / KNOB_MAX,
|
||||
0, 1)`.
|
||||
- `Calibration` gains `dream_gain: float = 1.0`; `DEFAULT_CALIBRATION` sets it 1.0.
|
||||
- `render_plan_to_dict`: `"restyle": {"variant"}` → `"dream": {"strength", "intensity"}`.
|
||||
- `is_identity`: Right contributes when `strength > 0` (replaces the
|
||||
`restyle.variant == 0` clause).
|
||||
- Pure + unit-tested. The client maps `dream.intensity` → CSS params, mirroring how
|
||||
`grade.tone` → video filter is mapped client-side today.
|
||||
|
||||
## Rendering — simulator
|
||||
|
||||
- **Remove** the variant-crossfade machinery (`loadVariant`, `variantFile`, the
|
||||
`right_variants` / `currentVariant` handling). The main video always plays
|
||||
`base_file`.
|
||||
- **Compose one video filter** from *both* the mood grade (`grade.tone`) and the dream
|
||||
(`dream.intensity`) — today `applyGrade` owns `vid.style.filter`; it becomes a single
|
||||
`applyVideoLook(tone, dreamIntensity)` so the two don't clobber each other.
|
||||
- **Add a bloom layer**: a second `<video id="bloom">` (same `base_file`, muted,
|
||||
looped, started in sync), `screen`-blended, blurred/brightened, opacity from
|
||||
`dream.intensity`. Both videos' `src` are set together on scale settle / ring move.
|
||||
- `#tint` (the Dark cool wash) is unchanged.
|
||||
|
||||
## Migration
|
||||
|
||||
- The baked SD variants (`*/right*.mp4`) and `bake_right_variants.py` become **legacy:
|
||||
parked in-repo, not deleted, unused at runtime**. `right_variants` stays in the
|
||||
manifest/`clips.py` (ignored by the client) to avoid churn; it can be removed later.
|
||||
- The scale ring and its transitions are untouched.
|
||||
- The affect channel is unaffected — Right is still a 0–4 knob, so `min(left, right)`
|
||||
emotions keep working.
|
||||
|
||||
## Testing
|
||||
|
||||
- Python unit tests: `Dream.strength = right`, `intensity` scaling + `dream_gain`,
|
||||
clamping, `0` at `right=0`, presence/shape in the serialized plan, `is_identity`.
|
||||
- By-eye in the live sim (forest, Right up, then Left+Right for the whole picture) —
|
||||
the established judge-by-eye loop. Success = the dream holds still across the loop;
|
||||
only real motion (the waterfall) moves.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Removing the legacy baker / baked media (left parked, unused at runtime).
|
||||
- Any change to the scale ring, transitions, Left HUD, or affect channel.
|
||||
- A Pi-GPU performance pass on the Kuwahara shader (desktop sim is the current target).
|
||||
+67
-30
@@ -1,18 +1,19 @@
|
||||
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
|
||||
|
||||
Reconciled slice (2026-06-07): the Right axis is a DISCRETE selection of a
|
||||
pre-baked, flow-stabilized restyle variant (not a continuous blend), the Left
|
||||
axis carries its knob LEVEL so a runtime annotation track can pick which labels
|
||||
show, and a frozen `Calibration` parameterizes the knob->strength curves so they
|
||||
can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION.
|
||||
Right axis (reframed session 0013): a DETERMINISTIC, real-time dream applied live
|
||||
to the base footage — a STILL altered state, not the old pre-baked SD variant that
|
||||
churned across the loop (Right-axis dream design,
|
||||
docs/superpowers/specs/2026-06-22-right-axis-deterministic-dream-design.md). The
|
||||
Left axis carries its knob LEVEL so a runtime annotation track can pick which
|
||||
labels show, and a frozen `Calibration` parameterizes the knob->strength gains so
|
||||
they can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION.
|
||||
|
||||
Layers compose per §4.2:
|
||||
- Substrate: ColorGrade (Dark/Light mood, center = identity §5) + a pre-baked
|
||||
Right restyle variant.
|
||||
- Overlay: AnalyticalOverlay (Left), composited on top at runtime.
|
||||
- Substrate: ColorGrade (Dark/Light mood, center = identity §5) + the Right Dream
|
||||
(deterministic luminous haze, applied client-side from `dream.intensity`).
|
||||
- Overlay: AnalyticalOverlay (Left) + the affect channel, composited on top.
|
||||
Left and Right stack (different layers); Dark/Light are the two poles of one
|
||||
mood grade. See docs/superpowers/specs/2026-06-07-reconciled-simulator-
|
||||
alteration-slice-design.md.
|
||||
mood grade.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -33,13 +34,13 @@ class Calibration:
|
||||
"""Tunable knob->strength curves (settled by eye in the sim, then baked).
|
||||
|
||||
- mood_gain: scales the signed Dark/Light tone (result clamped to [-1, 1]).
|
||||
- overlay_gain: scales the Left overlay intensity (clamped to [0, 1]).
|
||||
- right_variant_map: knob value (0..4) -> pre-baked Right variant index.
|
||||
- overlay_gain: scales the Left overlay (and affect) intensity (clamped [0, 1]).
|
||||
- dream_gain: scales the Right deterministic-dream intensity (clamped [0, 1]).
|
||||
"""
|
||||
|
||||
mood_gain: float = 1.0
|
||||
overlay_gain: float = 1.0
|
||||
right_variant_map: tuple = (0, 1, 2, 3, 4)
|
||||
dream_gain: float = 1.0
|
||||
|
||||
|
||||
# The LOCKED calibration (session 0010, 2026-06-07) — settled by eye in the
|
||||
@@ -51,15 +52,15 @@ class Calibration:
|
||||
# full tilt is peaceful on every axis, so no softening is warranted.
|
||||
# - overlay_gain = 1.0: full Left = opacity 1.0; the HUD is legible, not
|
||||
# overwhelming, sitting above the grade.
|
||||
# - right_variant_map linear: the 5 knob notches map 1:1 onto the 5 discrete
|
||||
# pre-baked Right strengths (0 = raw base).
|
||||
# These are unity/linear by deliberate choice, not as placeholders. The curve
|
||||
# stays parameterized so a future re-bake or a different feel is one edit away;
|
||||
# test_default_calibration_is_locked guards the values from drifting silently.
|
||||
# - dream_gain = 1.0: full Right = dream intensity 1.0; the deterministic
|
||||
# luminous haze (Right-axis dream design, session 0013) reaches full strength.
|
||||
# These are unity by deliberate choice, not as placeholders. The gains stay
|
||||
# parameterized so a different feel is one edit away; test_default_calibration_is_locked
|
||||
# guards the values from drifting silently.
|
||||
DEFAULT_CALIBRATION = Calibration(
|
||||
mood_gain=1.0,
|
||||
overlay_gain=1.0,
|
||||
right_variant_map=(0, 1, 2, 3, 4),
|
||||
dream_gain=1.0,
|
||||
)
|
||||
|
||||
|
||||
@@ -86,11 +87,27 @@ class AnalyticalOverlay:
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Restyle:
|
||||
"""Right axis (§4.1): selects a pre-baked, flow-stabilized restyle variant.
|
||||
`variant` is a discrete index (0 = raw base, no restyle)."""
|
||||
class AffectOverlay:
|
||||
"""Affect channel (Left x Right): emotion-words surfaced only when BOTH the
|
||||
analytical (Left) and dreamlike (Right) knobs are up. `strength` is
|
||||
`min(left, right)` (0..4) — a runtime affect track picks which feeling words
|
||||
appear by it; `intensity` 0..1 is their opacity. The dream leaking into the
|
||||
machine's reading (affect-channel design, session 0013)."""
|
||||
|
||||
variant: int
|
||||
strength: int
|
||||
intensity: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Dream:
|
||||
"""Right axis: a deterministic, real-time dream applied live to the base
|
||||
footage — a STILL altered state (Right-axis dream design, session 0013),
|
||||
NOT the old pre-baked SD variant that churned across the loop. `strength` is
|
||||
the Right knob (0..4); `intensity` 0..1 is the dream's strength (the client
|
||||
maps it to the luminous-haze look, the way it maps `grade.tone` to a filter)."""
|
||||
|
||||
strength: int
|
||||
intensity: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -99,15 +116,18 @@ class RenderPlan:
|
||||
|
||||
grade: ColorGrade
|
||||
overlay: AnalyticalOverlay
|
||||
restyle: Restyle
|
||||
affect: AffectOverlay
|
||||
dream: Dream
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
"""True when the plan leaves the neutral base un-altered."""
|
||||
"""True when the plan leaves the neutral base un-altered. Affect needs no
|
||||
clause: `min(left,right) > 0` implies `left > 0`, so `overlay.level > 0`
|
||||
already makes a plan with active affect non-identity."""
|
||||
return (
|
||||
self.grade.is_identity
|
||||
and self.overlay.level == 0
|
||||
and self.restyle.variant == 0
|
||||
and self.dream.strength == 0
|
||||
)
|
||||
|
||||
|
||||
@@ -115,8 +135,17 @@ def _overlay_intensity(left: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.overlay_gain * left / KNOB_MAX, 0.0, 1.0)
|
||||
|
||||
|
||||
def _right_variant(right: int, cal: Calibration) -> int:
|
||||
return cal.right_variant_map[right]
|
||||
def _affect_strength(left: int, right: int) -> int:
|
||||
"""Affect surfaces only when BOTH knobs are up — gated by the smaller one."""
|
||||
return min(left, right)
|
||||
|
||||
|
||||
def _affect_intensity(left: int, right: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.overlay_gain * _affect_strength(left, right) / KNOB_MAX, 0.0, 1.0)
|
||||
|
||||
|
||||
def _dream_intensity(right: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.dream_gain * right / KNOB_MAX, 0.0, 1.0)
|
||||
|
||||
|
||||
def _mood_tone(dark: int, light: int, cal: Calibration) -> float:
|
||||
@@ -133,7 +162,14 @@ def plan_alteration(
|
||||
level=coord.left,
|
||||
intensity=_overlay_intensity(coord.left, calibration),
|
||||
),
|
||||
restyle=Restyle(variant=_right_variant(coord.right, calibration)),
|
||||
affect=AffectOverlay(
|
||||
strength=_affect_strength(coord.left, coord.right),
|
||||
intensity=_affect_intensity(coord.left, coord.right, calibration),
|
||||
),
|
||||
dream=Dream(
|
||||
strength=coord.right,
|
||||
intensity=_dream_intensity(coord.right, calibration),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -142,6 +178,7 @@ def render_plan_to_dict(plan: RenderPlan) -> dict:
|
||||
return {
|
||||
"grade": {"tone": plan.grade.tone},
|
||||
"overlay": {"level": plan.overlay.level, "intensity": plan.overlay.intensity},
|
||||
"restyle": {"variant": plan.restyle.variant},
|
||||
"affect": {"strength": plan.affect.strength, "intensity": plan.affect.intensity},
|
||||
"dream": {"strength": plan.dream.strength, "intensity": plan.dream.intensity},
|
||||
"is_identity": plan.is_identity,
|
||||
}
|
||||
|
||||
+7
-5
@@ -3,10 +3,10 @@
|
||||
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
|
||||
desired Playback (which neutral base clip, how it is altered, what audio plays,
|
||||
at what levels) and returns the Transition from the previous Playback. The
|
||||
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
|
||||
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
|
||||
pre-baked Right restyle variant (a discrete index) needs a CROSSFADE, and
|
||||
toggling video on/off fades to/from black.
|
||||
transition KIND encodes design §4.3: the Dark/Light grade, the Left overlay AND
|
||||
the Right dream (a deterministic, live luminous haze since session 0013) are all
|
||||
continuous runtime ops (LIVE_UPDATE); only swapping the clip needs a CROSSFADE,
|
||||
and toggling video on/off fades to/from black.
|
||||
|
||||
Knobs no longer *select* a clip (the base library is neutral by construction);
|
||||
they *transform* it. Which neutral base to show is an injected policy
|
||||
@@ -107,7 +107,9 @@ class Player:
|
||||
if next_video and not prev_video:
|
||||
return TransitionKind.FADE_FROM_BLACK
|
||||
if next_video and prev_video:
|
||||
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
|
||||
# The Right dream is a live filter now (not a substrate swap), so it
|
||||
# no longer forces a crossfade — only a clip (scale) change does.
|
||||
if nxt.clip_id != prev.clip_id:
|
||||
return TransitionKind.CROSSFADE
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
# both black: only audio/levels could have changed
|
||||
|
||||
+38
-2
@@ -7,6 +7,9 @@ endpoints (/api/select, /api/catalog/meta) and the X-ray are retired.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
@@ -30,6 +33,24 @@ STATIC_DIR = Path(__file__).parent / "static"
|
||||
MEDIA_DIR = Path(__file__).parent / "sample_media"
|
||||
DEFAULT_MANIFEST = MEDIA_DIR / "manifest.json"
|
||||
|
||||
# Per-process boot token: changes on every server restart so the dev live-reload
|
||||
# (below) also fires when Python code changes (which needs a restart), not just
|
||||
# when a static asset's mtime changes.
|
||||
_BOOT_ID = f"{os.getpid()}-{int(time.time())}"
|
||||
|
||||
|
||||
def _asset_version() -> str:
|
||||
"""A short digest of the renderer assets' mtimes + this process's boot token.
|
||||
Changes whenever app.js/style.css/index.html change on disk OR the server is
|
||||
restarted — the signal the browser polls to know it's running stale code."""
|
||||
parts = [_BOOT_ID]
|
||||
for name in ("app.js", "style.css", "index.html"):
|
||||
try:
|
||||
parts.append(f"{name}:{(STATIC_DIR / name).stat().st_mtime_ns}")
|
||||
except OSError:
|
||||
parts.append(f"{name}:?")
|
||||
return hashlib.sha1("|".join(parts).encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
class ControlsModel(BaseModel):
|
||||
content: str
|
||||
@@ -44,7 +65,7 @@ class ControlsModel(BaseModel):
|
||||
class CalibrationModel(BaseModel):
|
||||
mood_gain: float = 1.0
|
||||
overlay_gain: float = 1.0
|
||||
right_variant_map: list[int] = [0, 1, 2, 3, 4]
|
||||
dream_gain: float = 1.0
|
||||
|
||||
|
||||
class AlterationRequest(BaseModel):
|
||||
@@ -86,7 +107,7 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
|
||||
Calibration(
|
||||
mood_gain=req.calibration.mood_gain,
|
||||
overlay_gain=req.calibration.overlay_gain,
|
||||
right_variant_map=tuple(req.calibration.right_variant_map),
|
||||
dream_gain=req.calibration.dream_gain,
|
||||
)
|
||||
if req.calibration
|
||||
else DEFAULT_CALIBRATION
|
||||
@@ -98,6 +119,12 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
|
||||
"content": {"audio_source": content.audio_source, "video": content.video},
|
||||
}
|
||||
|
||||
@app.get("/dev/version")
|
||||
def dev_version():
|
||||
# Dev live-reload signal: the browser polls this and reloads when it
|
||||
# changes, so it never silently runs a stale renderer during iteration.
|
||||
return {"version": _asset_version()}
|
||||
|
||||
@app.get("/api/clips")
|
||||
def api_clips():
|
||||
return {"clips": [c.to_dict() for c in app.state.clips]}
|
||||
@@ -120,6 +147,15 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
|
||||
)
|
||||
return ring_move_to_dict(move, app.state.ring)
|
||||
|
||||
@app.middleware("http")
|
||||
async def _no_cache(request, call_next):
|
||||
# Dev preview server: never let a browser serve a stale app.js/style.css.
|
||||
# The whole point is by-eye iteration, so a plain refresh must always get
|
||||
# the latest assets (the heuristic cache otherwise hides JS/CSS edits).
|
||||
response = await call_next(request)
|
||||
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
|
||||
return response
|
||||
|
||||
if MEDIA_DIR.exists():
|
||||
app.mount("/media", StaticFiles(directory=MEDIA_DIR), name="media")
|
||||
if STATIC_DIR.exists():
|
||||
|
||||
@@ -26,6 +26,7 @@ class Clip:
|
||||
source: str
|
||||
right_variants: dict # {"1": {"file": ...}, "4": {...}} (no "0")
|
||||
annotations: list # [{"key", "box":[x,y,w,h], "min_level"}, ...]
|
||||
affect: list # [{"key", "at":[x,y], "min_level"}, ...] (Left x Right)
|
||||
strings: dict # {"en": {key: text}}
|
||||
|
||||
def variant_file(self, strength: int) -> str:
|
||||
@@ -46,6 +47,7 @@ class Clip:
|
||||
"source": self.source,
|
||||
"right_variants": variants,
|
||||
"annotations": self.annotations,
|
||||
"affect": self.affect,
|
||||
"strings": self.strings,
|
||||
}
|
||||
|
||||
@@ -59,6 +61,7 @@ def _clip_from_dict(d: dict[str, Any]) -> Clip:
|
||||
source=d.get("source", ""),
|
||||
right_variants=d.get("right_variants", {}),
|
||||
annotations=d.get("annotations", []),
|
||||
affect=d.get("affect", []),
|
||||
strings=d.get("strings", {}),
|
||||
)
|
||||
|
||||
|
||||
@@ -11,10 +11,20 @@
|
||||
{"key": "detected.galaxy", "box": [0.34, 0.30, 0.30, 0.34], "min_level": 1},
|
||||
{"key": "measure.redshift", "box": [0.36, 0.66, 0.18, 0.08], "min_level": 4}
|
||||
],
|
||||
"affect": [
|
||||
{"key": "feel.wonder", "at": [0.50, 0.46], "min_level": 1},
|
||||
{"key": "feel.vastness", "at": [0.20, 0.70], "min_level": 2},
|
||||
{"key": "feel.insignificance", "at": [0.64, 0.60], "min_level": 3},
|
||||
{"key": "feel.longing", "at": [0.42, 0.84], "min_level": 4}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.galaxy": "spiral galaxy",
|
||||
"measure.redshift": "z ≈ 0.04"
|
||||
"measure.redshift": "z ≈ 0.04",
|
||||
"feel.wonder": "wonder",
|
||||
"feel.vastness": "vastness",
|
||||
"feel.insignificance": "insignificance",
|
||||
"feel.longing": "longing"
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -36,12 +46,22 @@
|
||||
{"key": "detected.conifer", "box": [0.70, 0.20, 0.22, 0.45], "min_level": 3},
|
||||
{"key": "measure.flow_rate", "box": [0.34, 0.55, 0.14, 0.08], "min_level": 4}
|
||||
],
|
||||
"affect": [
|
||||
{"key": "feel.awe", "at": [0.48, 0.44], "min_level": 1},
|
||||
{"key": "feel.serenity", "at": [0.18, 0.66], "min_level": 2},
|
||||
{"key": "feel.reverence", "at": [0.66, 0.56], "min_level": 3},
|
||||
{"key": "feel.stillness", "at": [0.42, 0.82], "min_level": 4}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.water": "flowing water",
|
||||
"detected.rock_face": "granite face",
|
||||
"detected.conifer": "conifer stand",
|
||||
"measure.flow_rate": "~2.1 m³/s"
|
||||
"measure.flow_rate": "~2.1 m³/s",
|
||||
"feel.awe": "awe",
|
||||
"feel.serenity": "serenity",
|
||||
"feel.reverence": "reverence",
|
||||
"feel.stillness": "stillness"
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -56,10 +76,20 @@
|
||||
{"key": "detected.organism", "box": [0.40, 0.38, 0.22, 0.26], "min_level": 1},
|
||||
{"key": "measure.depth", "box": [0.06, 0.06, 0.16, 0.08], "min_level": 2}
|
||||
],
|
||||
"affect": [
|
||||
{"key": "feel.unease", "at": [0.50, 0.46], "min_level": 1},
|
||||
{"key": "feel.fascination", "at": [0.22, 0.68], "min_level": 2},
|
||||
{"key": "feel.isolation", "at": [0.66, 0.58], "min_level": 3},
|
||||
{"key": "feel.dread", "at": [0.42, 0.82], "min_level": 4}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.organism": "siphonophore",
|
||||
"measure.depth": "−2,140 m"
|
||||
"measure.depth": "−2,140 m",
|
||||
"feel.unease": "unease",
|
||||
"feel.fascination": "fascination",
|
||||
"feel.isolation": "isolation",
|
||||
"feel.dread": "dread"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+277
-50
@@ -1,15 +1,39 @@
|
||||
// Thin renderer: post controls+calibration -> RenderPlan; render grade, Right
|
||||
// variant crossfade, and the live Left overlay. All alteration math stays in
|
||||
// Python. The scale RING (endless encoder) is navigated via /api/ring/advance —
|
||||
// Python owns the step/wrap/transition-selection math; the browser only plays
|
||||
// the returned transition clip(s) then settles on the target scale's clip.
|
||||
// Thin renderer: post controls+calibration -> RenderPlan; render the mood grade,
|
||||
// the live Right DREAM (a deterministic luminous haze, NOT a baked variant — it
|
||||
// holds still across the loop), and the live Left overlay + affect. All alteration
|
||||
// math stays in Python. The scale RING (endless encoder) is navigated via
|
||||
// /api/ring/advance — Python owns the step/wrap/transition-selection math; the
|
||||
// browser only plays the returned transition clip(s) then settles on the target.
|
||||
const $ = (id) => document.getElementById(id);
|
||||
const vid = $("vid"), tint = $("tint"), overlay = $("overlay"), black = $("black"), readout = $("readout");
|
||||
const vid = $("vid"), paint = $("paint"), tint = $("tint"), overlay = $("overlay"),
|
||||
black = $("black"), readout = $("readout");
|
||||
const affectLayer = $("affect");
|
||||
|
||||
// Right-dream state, read by the painterly render loop each frame. update() (debounced
|
||||
// on knob moves) writes these; the WebGL loop renders the live video continuously.
|
||||
let dreamIntensity = 0; // 0..1 — painterly strength (and dream pastel/luminous)
|
||||
let gradeFilter = "none"; // CSS filter string for the Dark/Light mood grade
|
||||
|
||||
// Surface any uncaught JS error on-screen (a silent throw in update() otherwise
|
||||
// looks like "nothing is changing" — the render dies but the page sits there).
|
||||
function showError(msg) {
|
||||
let b = document.getElementById("err-banner");
|
||||
if (!b) {
|
||||
b = document.createElement("div");
|
||||
b.id = "err-banner";
|
||||
b.style.cssText = "position:fixed;top:0;left:0;right:0;z-index:9999;background:#a00;" +
|
||||
"color:#fff;font:12px/1.4 monospace;padding:6px 10px;white-space:pre-wrap;";
|
||||
document.body.appendChild(b);
|
||||
}
|
||||
b.textContent = "⚠ " + msg;
|
||||
}
|
||||
window.addEventListener("error", (e) =>
|
||||
showError(`${e.message} @ ${(e.filename || "").split("/").pop()}:${e.lineno}`));
|
||||
|
||||
let clipsById = {}; // id -> clip manifest entry
|
||||
let ring = null; // {scales:[{id,clip_id,title}], transitions:[...]} or null
|
||||
let ringIndex = 0; // current scale position on the ring
|
||||
let currentVariant = -1; // last loaded Right strength (reset on clip change)
|
||||
let currentClipId = null; // base media currently loaded (reset to force a reload)
|
||||
let busy = false; // true while a ring transition is playing
|
||||
|
||||
async function loadData() {
|
||||
@@ -31,36 +55,181 @@ function activeClip() {
|
||||
|
||||
function mediaUrl(file) { return "/media/" + file; }
|
||||
|
||||
function variantFile(strength) {
|
||||
// Load the active scale's BASE footage into the video (the painterly canvas reads
|
||||
// its frames live). The Right knob no longer swaps the source — it only drives the
|
||||
// deterministic dream restyle — so media reloads only when the SCALE changes.
|
||||
function ensureClipMedia() {
|
||||
const clip = activeClip();
|
||||
if (!clip) return "";
|
||||
const v = clip.right_variants[String(strength)];
|
||||
return v ? v.file : clip.base_file;
|
||||
if (!clip || clip.id === currentClipId) return;
|
||||
currentClipId = clip.id;
|
||||
vid.src = mediaUrl(clip.base_file);
|
||||
vid.loop = true;
|
||||
vid.muted = true;
|
||||
vid.play().catch(() => {});
|
||||
vid.style.opacity = "1";
|
||||
}
|
||||
|
||||
function applyGrade(tone) {
|
||||
// Light: warm + brighten (sepia). Dark: cool + darken via a multiply-blended
|
||||
// blue wash (#tint) that lifts shadows toward blue while keeping natural
|
||||
// greens — the peaceful POC dark look, NOT a full-frame hue spin.
|
||||
// Compose the video look. The Right DREAM is now a painterly WebGL restyle of the
|
||||
// live frames (see the render loop below) — so here we only stash its intensity for
|
||||
// that loop, and build the Dark/Light mood grade as a CSS filter. The grade rides
|
||||
// on the painterly canvas (or #vid in the WebGL-less fallback). No blur: the dream
|
||||
// must keep the base's crisp motion, just stylized.
|
||||
function applyVideoLook(tone, dream) {
|
||||
const warm = tone > 0 ? tone : 0, cool = tone < 0 ? -tone : 0;
|
||||
const bright = 1 + 0.25 * warm - 0.35 * cool;
|
||||
const sat = 1 + 0.15 * warm - 0.30 * cool;
|
||||
vid.style.filter =
|
||||
`brightness(${bright.toFixed(3)}) saturate(${sat.toFixed(3)}) ` +
|
||||
`sepia(${(warm * 0.5).toFixed(3)})`;
|
||||
const sepia = warm * 0.5;
|
||||
gradeFilter = `brightness(${bright.toFixed(3)}) saturate(${sat.toFixed(3)}) sepia(${sepia.toFixed(3)})`;
|
||||
dreamIntensity = dream;
|
||||
tint.style.opacity = (cool * 0.6).toFixed(3);
|
||||
// WebGL-less fallback: the canvas is hidden, so grade the visible #vid directly.
|
||||
if (!paintOK) vid.style.filter = gradeFilter;
|
||||
}
|
||||
|
||||
function loadVariant(strength) {
|
||||
if (strength === currentVariant) return;
|
||||
currentVariant = strength;
|
||||
vid.style.opacity = "0";
|
||||
setTimeout(() => {
|
||||
vid.src = mediaUrl(variantFile(strength));
|
||||
vid.loop = true;
|
||||
vid.play().catch(() => {});
|
||||
vid.style.opacity = "1";
|
||||
}, 150);
|
||||
// --- Right dream: real-time painterly restyle (WebGL2 Kuwahara) ---
|
||||
// An edge-preserving "oil painting" filter on the LIVE video frames: each pixel
|
||||
// becomes the mean of whichever surrounding quadrant is most uniform, flattening
|
||||
// the image into painted regions WHILE KEEPING EDGES — so motion stays as crisp as
|
||||
// the source. Strength + a gentle pastel/luminous dream-grade scale with intensity.
|
||||
// Deterministic (a pure function of each frame) => the dream holds still across the
|
||||
// loop; only the footage's own motion moves.
|
||||
let paintOK = false;
|
||||
const PAINT_VS = `#version 300 es
|
||||
in vec2 p; out vec2 v_uv;
|
||||
void main(){
|
||||
// Fullscreen triangle (verts to +3); map the visible [-1,1] to uv [0,1] with Y
|
||||
// flipped for the video's top-left origin. (Visible region stays in range; the
|
||||
// off-screen excess is clipped.)
|
||||
v_uv = vec2(p.x * 0.5 + 0.5, 0.5 - p.y * 0.5);
|
||||
gl_Position = vec4(p, 0.0, 1.0);
|
||||
}`;
|
||||
const PAINT_FS = `#version 300 es
|
||||
precision highp float;
|
||||
in vec2 v_uv; out vec4 frag;
|
||||
uniform sampler2D u_tex; uniform vec2 u_texel; uniform float u_amt;
|
||||
#define R 6
|
||||
#define QUAD(x0,x1,y0,y1,MO,VO) { vec3 m=vec3(0.0),s=vec3(0.0); \
|
||||
for(int j=y0;j<=y1;j++){ for(int i=x0;i<=x1;i++){ \
|
||||
vec3 c=texture(u_tex, v_uv+vec2(float(i),float(j))*u_texel).rgb; m+=c; s+=c*c; } } \
|
||||
float n=float((x1-x0+1)*(y1-y0+1)); m/=n; vec3 vv=s/n-m*m; MO=m; VO=vv.r+vv.g+vv.b; }
|
||||
vec3 hueRotate(vec3 c, float a){
|
||||
float co=cos(a), si=sin(a);
|
||||
return vec3(
|
||||
c.r*(0.299+0.701*co+0.168*si) + c.g*(0.587-0.587*co+0.330*si) + c.b*(0.114-0.114*co-0.497*si),
|
||||
c.r*(0.299-0.299*co-0.328*si) + c.g*(0.587+0.413*co+0.035*si) + c.b*(0.114-0.114*co+0.292*si),
|
||||
c.r*(0.299-0.300*co+1.250*si) + c.g*(0.587-0.588*co-1.050*si) + c.b*(0.114+0.886*co-0.203*si));
|
||||
}
|
||||
void main(){
|
||||
vec3 base = texture(u_tex, v_uv).rgb;
|
||||
if (u_amt <= 0.001) { frag = vec4(base, 1.0); return; }
|
||||
vec3 m0,m1,m2,m3; float v0,v1,v2,v3;
|
||||
QUAD(-R,0,-R,0,m0,v0) QUAD(0,R,-R,0,m1,v1) QUAD(-R,0,0,R,m2,v2) QUAD(0,R,0,R,m3,v3)
|
||||
vec3 painted=m0; float mv=v0;
|
||||
if(v1<mv){mv=v1;painted=m1;} if(v2<mv){mv=v2;painted=m2;} if(v3<mv){mv=v3;painted=m3;}
|
||||
vec3 styl = mix(base, painted, u_amt); // painterly restyle, linear in the knob
|
||||
// The "trippy" terms ramp with t = amt^2, so low Right stays gently dreamy and
|
||||
// only MAX goes psychedelic: vivid saturation, a surreal hue drift, poster banding.
|
||||
float t = u_amt * u_amt;
|
||||
float luma = dot(styl, vec3(0.299, 0.587, 0.114));
|
||||
styl = mix(vec3(luma), styl, 1.0 + 1.1 * t); // saturation: vivid toward max
|
||||
styl = hueRotate(styl, 0.8 * t); // surreal hue drift (~45deg at max)
|
||||
float levels = mix(255.0, 6.0, t); // posterize: color banding toward max
|
||||
styl = floor(styl * levels + 0.5) / levels;
|
||||
styl *= 1.0 + 0.06 * u_amt; // luminous lift
|
||||
frag = vec4(clamp(styl, 0.0, 1.0), 1.0);
|
||||
}`;
|
||||
let gl = null, uAmt = null, uTexel = null;
|
||||
function _shader(kind, src) {
|
||||
const s = gl.createShader(kind);
|
||||
gl.shaderSource(s, src); gl.compileShader(s);
|
||||
if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) throw new Error(gl.getShaderInfoLog(s));
|
||||
return s;
|
||||
}
|
||||
function initPaint() {
|
||||
gl = paint.getContext("webgl2");
|
||||
if (!gl) { paint.style.display = "none"; return false; } // CSS fallback grades #vid
|
||||
const prog = gl.createProgram();
|
||||
gl.attachShader(prog, _shader(gl.VERTEX_SHADER, PAINT_VS));
|
||||
gl.attachShader(prog, _shader(gl.FRAGMENT_SHADER, PAINT_FS));
|
||||
gl.bindAttribLocation(prog, 0, "p"); gl.linkProgram(prog);
|
||||
if (!gl.getProgramParameter(prog, gl.LINK_STATUS)) throw new Error(gl.getProgramInfoLog(prog));
|
||||
gl.useProgram(prog);
|
||||
const buf = gl.createBuffer();
|
||||
gl.bindBuffer(gl.ARRAY_BUFFER, buf);
|
||||
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1, -1, 3, -1, -1, 3]), gl.STATIC_DRAW);
|
||||
gl.enableVertexAttribArray(0); gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);
|
||||
const t = gl.createTexture();
|
||||
gl.bindTexture(gl.TEXTURE_2D, t);
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
|
||||
uAmt = gl.getUniformLocation(prog, "u_amt");
|
||||
uTexel = gl.getUniformLocation(prog, "u_texel");
|
||||
paintOK = true;
|
||||
requestAnimationFrame(paintLoop);
|
||||
return true;
|
||||
}
|
||||
function paintLoop() {
|
||||
if (paintOK && vid.readyState >= 2 && vid.videoWidth) {
|
||||
const w = vid.videoWidth, h = vid.videoHeight;
|
||||
if (paint.width !== w || paint.height !== h) { paint.width = w; paint.height = h; }
|
||||
gl.viewport(0, 0, w, h);
|
||||
try { gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, vid); } catch (_) {}
|
||||
// No painterly restyle during a ring transition (busy) — show it raw.
|
||||
gl.uniform1f(uAmt, busy ? 0 : dreamIntensity);
|
||||
gl.uniform2f(uTexel, 1 / w, 1 / h);
|
||||
gl.drawArrays(gl.TRIANGLES, 0, 3);
|
||||
paint.style.filter = busy ? "none" : gradeFilter;
|
||||
}
|
||||
requestAnimationFrame(paintLoop);
|
||||
}
|
||||
|
||||
const SVGNS = "http://www.w3.org/2000/svg";
|
||||
function svg(tag, attrs, parent) {
|
||||
const el = document.createElementNS(SVGNS, tag);
|
||||
for (const k in attrs) el.setAttribute(k, attrs[k]);
|
||||
if (parent) parent.appendChild(el);
|
||||
return el;
|
||||
}
|
||||
|
||||
// Deterministic per-key "confidence" in [0.78, 0.99] — stable across renders so
|
||||
// a detection always reports the same score (looks like a real model, not noise).
|
||||
function confidence(key) {
|
||||
let h = 0;
|
||||
for (let i = 0; i < key.length; i++) h = (h * 31 + key.charCodeAt(i)) >>> 0;
|
||||
return (0.78 + (h % 2200) / 10000).toFixed(2);
|
||||
}
|
||||
|
||||
// Four L-shaped corner brackets framing [x,y,w,h] — a targeting reticle rather
|
||||
// than a plain rectangle. Arm length scales to the box but is clamped legible.
|
||||
function reticle(x, y, w, h, cls, parent) {
|
||||
const L = Math.min(Math.max(Math.min(w, h) * 0.28, 1.4), 4);
|
||||
const corners = [
|
||||
[[x + L, y], [x, y], [x, y + L]],
|
||||
[[x + w - L, y], [x + w, y], [x + w, y + L]],
|
||||
[[x + L, y + h], [x, y + h], [x, y + h - L]],
|
||||
[[x + w - L, y + h], [x + w, y + h], [x + w, y + h - L]],
|
||||
];
|
||||
for (const pts of corners) {
|
||||
svg("polyline", { points: pts.map((p) => p.join(",")).join(" "), class: cls }, parent);
|
||||
}
|
||||
}
|
||||
|
||||
// A label chip: translucent plate + monospace text, anchored top-left of the box.
|
||||
// `conf` (detections only) is appended as a dimmer score tspan.
|
||||
function chip(x, y, label, conf, kind, parent) {
|
||||
const fs = 2.4, pad = 0.6;
|
||||
const chars = label.length + (conf ? conf.length + 1 : 0);
|
||||
const w = chars * fs * 0.6 + pad * 2, h = fs + pad * 1.4;
|
||||
const cy = Math.max(y - h - 0.6, 0.4);
|
||||
svg("rect", { x, y: cy, width: w, height: h, rx: 0.4, class: "hud-chip-bg " + kind }, parent);
|
||||
const t = svg("text", { x: x + pad, y: cy + h - pad * 1.1, class: "hud-chip-text " + kind }, parent);
|
||||
t.appendChild(document.createTextNode(label));
|
||||
if (conf) {
|
||||
const sp = svg("tspan", { class: "hud-conf" }, t);
|
||||
sp.textContent = " " + conf;
|
||||
}
|
||||
}
|
||||
|
||||
function renderOverlay(level, intensity) {
|
||||
@@ -69,19 +238,52 @@ function renderOverlay(level, intensity) {
|
||||
if (!clip || level <= 0) { overlay.style.opacity = "0"; return; }
|
||||
overlay.style.opacity = String(intensity);
|
||||
const strings = (clip.strings && clip.strings.en) || {};
|
||||
let shown = 0;
|
||||
for (const a of clip.annotations) {
|
||||
if (a.min_level > level) continue;
|
||||
shown++;
|
||||
const [x, y, w, h] = a.box.map((n) => n * 100);
|
||||
const rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
|
||||
rect.setAttribute("x", x); rect.setAttribute("y", y);
|
||||
rect.setAttribute("width", w); rect.setAttribute("height", h);
|
||||
rect.setAttribute("class", "anno-box");
|
||||
overlay.appendChild(rect);
|
||||
const text = document.createElementNS("http://www.w3.org/2000/svg", "text");
|
||||
text.setAttribute("x", x + 0.5); text.setAttribute("y", Math.max(y - 0.5, 2));
|
||||
text.setAttribute("class", "anno-label");
|
||||
text.textContent = strings[a.key] || a.key;
|
||||
overlay.appendChild(text);
|
||||
const measure = a.key.startsWith("measure.");
|
||||
const kind = measure ? "measure" : "detect";
|
||||
reticle(x, y, w, h, "hud-reticle " + kind, overlay);
|
||||
if (measure) {
|
||||
// Point measurement: a center crosshair tick to read as "sampled here".
|
||||
const cx = x + w / 2, cyc = y + h / 2;
|
||||
svg("line", { x1: cx - 1, y1: cyc, x2: cx + 1, y2: cyc, class: "hud-reticle measure" }, overlay);
|
||||
svg("line", { x1: cx, y1: cyc - 1, x2: cx, y2: cyc + 1, class: "hud-reticle measure" }, overlay);
|
||||
}
|
||||
chip(x, y, strings[a.key] || a.key, measure ? null : confidence(a.key), kind, overlay);
|
||||
}
|
||||
// Global status tag — the machine announcing it is analyzing, escalating with Left.
|
||||
// Right-anchored at the frame edge; the plate is sized from the text's real bbox
|
||||
// (the ◉/· glyphs are wider than monospace, so an estimate clips).
|
||||
if (shown) {
|
||||
const pad = 0.8;
|
||||
const st = svg("text", { x: 98, y: 5, "text-anchor": "end", class: "hud-status" }, overlay);
|
||||
st.textContent = "◉ ANALYSIS · L" + level + " · " + shown + " OBJ";
|
||||
const b = st.getBBox();
|
||||
const bg = svg("rect", { x: b.x - pad, y: b.y - pad * 0.6, width: b.width + pad * 2,
|
||||
height: b.height + pad * 1.2, rx: 0.4, class: "hud-chip-bg detect" });
|
||||
overlay.insertBefore(bg, st);
|
||||
}
|
||||
}
|
||||
|
||||
// Affect channel: soft, glowing emotion-words surfaced only when BOTH knobs are
|
||||
// up. `strength` = min(left, right); `intensity` is the layer opacity. Words are
|
||||
// placed at authored scene points (no boxes — feelings are scene-level) and read
|
||||
// softer than the clinical reticles — the dream leaking into the machine's read.
|
||||
function renderAffect(strength, intensity) {
|
||||
const clip = activeClip();
|
||||
if (!affectLayer) return; // tolerate a stale page missing the affect layer
|
||||
affectLayer.innerHTML = "";
|
||||
if (!clip || !clip.affect || strength <= 0) { affectLayer.style.opacity = "0"; return; }
|
||||
affectLayer.style.opacity = String(intensity);
|
||||
const strings = (clip.strings && clip.strings.en) || {};
|
||||
for (const f of clip.affect) {
|
||||
if (f.min_level > strength) continue;
|
||||
const [x, y] = f.at.map((n) => n * 100);
|
||||
const t = svg("text", { x, y, "text-anchor": "middle", class: "hud-affect" }, affectLayer);
|
||||
t.textContent = strings[f.key] || f.key;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -92,34 +294,41 @@ function renderScaleReadout() {
|
||||
}
|
||||
|
||||
function controls() {
|
||||
// One bipolar Mood dial (-4 dark .. 0 neutral .. +4 light) maps onto the engine's
|
||||
// two poles. The calibration gains are pinned to their locked 1.0 (full dial = full
|
||||
// effect), so the simulator sends no calibration — the server uses DEFAULT_CALIBRATION.
|
||||
const mood = +$("mood").value;
|
||||
return {
|
||||
content: $("content").value,
|
||||
left: +$("left").value, right: +$("right").value,
|
||||
dark: +$("dark").value, light: +$("light").value,
|
||||
dark: mood < 0 ? -mood : 0, light: mood > 0 ? mood : 0,
|
||||
volume: 2, brightness: 2,
|
||||
};
|
||||
}
|
||||
|
||||
function calibration() {
|
||||
return { mood_gain: +$("mood_gain").value, overlay_gain: +$("overlay_gain").value,
|
||||
right_variant_map: [0, 1, 2, 3, 4] };
|
||||
}
|
||||
|
||||
let timer = null;
|
||||
async function update() {
|
||||
if (busy) return;
|
||||
const resp = await fetch("/api/alteration", {
|
||||
method: "POST", headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ controls: controls(), calibration: calibration() }),
|
||||
body: JSON.stringify({ controls: controls() }),
|
||||
});
|
||||
if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; }
|
||||
const data = await resp.json();
|
||||
readout.textContent = JSON.stringify(data, null, 2);
|
||||
if (!data.content.video) { black.classList.remove("hidden"); return; }
|
||||
black.classList.add("hidden");
|
||||
applyGrade(data.plan.grade.tone);
|
||||
loadVariant(data.plan.restyle.variant);
|
||||
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
|
||||
try {
|
||||
ensureClipMedia();
|
||||
applyVideoLook(data.plan.grade.tone, data.plan.dream.intensity);
|
||||
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
|
||||
renderAffect(data.plan.affect.strength, data.plan.affect.intensity);
|
||||
const b = document.getElementById("err-banner");
|
||||
if (b) b.remove(); // render succeeded — clear any prior error
|
||||
} catch (err) {
|
||||
showError("render failed: " + (err && err.message ? err.message : err));
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
function debounced() { clearTimeout(timer); timer = setTimeout(update, 80); }
|
||||
@@ -135,6 +344,7 @@ function playTransition(file, blended) {
|
||||
// Resolves on 'ended' (or a safety timeout that scales with playback rate).
|
||||
return new Promise((resolve) => {
|
||||
overlay.style.opacity = "0";
|
||||
affectLayer.style.opacity = "0";
|
||||
tint.style.opacity = "0";
|
||||
vid.style.filter = "none";
|
||||
vid.loop = false;
|
||||
@@ -171,7 +381,7 @@ async function advance(delta) {
|
||||
await playTransition(step.file, step.blended);
|
||||
}
|
||||
ringIndex = move.to_index;
|
||||
currentVariant = -1; // force the target clip's variant to (re)load
|
||||
currentClipId = null; // force the target scale's base media to (re)load
|
||||
renderScaleReadout();
|
||||
} finally {
|
||||
busy = false;
|
||||
@@ -191,10 +401,27 @@ function onWheel(e) {
|
||||
}, 90);
|
||||
}
|
||||
|
||||
// Dev live-reload: poll the asset version and reload when it changes, so an open
|
||||
// tab never keeps running a stale renderer while we iterate (the readout updates
|
||||
// from the live API and masks it otherwise). Reloads on my edits AND on a server
|
||||
// restart. Dev-only convenience; harmless if the endpoint is absent.
|
||||
function devLiveReload() {
|
||||
let seen = null;
|
||||
setInterval(async () => {
|
||||
try {
|
||||
const v = (await (await fetch("/dev/version", { cache: "no-store" })).json()).version;
|
||||
if (seen === null) seen = v;
|
||||
else if (v !== seen) location.reload();
|
||||
} catch (_) { /* server restarting / endpoint absent — ignore */ }
|
||||
}, 1000);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
devLiveReload();
|
||||
try { initPaint(); } catch (e) { paintOK = false; paint.style.display = "none"; showError("WebGL init: " + e.message); }
|
||||
await loadData();
|
||||
renderScaleReadout();
|
||||
for (const id of ["content", "left", "right", "dark", "light", "mood_gain", "overlay_gain"]) {
|
||||
for (const id of ["content", "left", "right", "mood"]) {
|
||||
$(id).addEventListener("input", debounced);
|
||||
}
|
||||
$("zoom-in").addEventListener("click", () => advance(1));
|
||||
|
||||
@@ -12,8 +12,10 @@
|
||||
<section class="stage" id="stage">
|
||||
<div class="screen">
|
||||
<video id="vid" loop muted playsinline></video>
|
||||
<canvas id="paint"></canvas>
|
||||
<div id="tint"></div>
|
||||
<svg id="overlay" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
|
||||
<svg id="affect" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
|
||||
<div id="black" class="black hidden"></div>
|
||||
</div>
|
||||
</section>
|
||||
@@ -46,15 +48,9 @@
|
||||
<legend>Experience knobs (0–4)</legend>
|
||||
<label>Left (analytical) <input type="range" id="left" min="0" max="4" value="0" /></label>
|
||||
<label>Right (dreamlike) <input type="range" id="right" min="0" max="4" value="0" /></label>
|
||||
<label>Dark <input type="range" id="dark" min="0" max="4" value="0" /></label>
|
||||
<label>Light <input type="range" id="light" min="0" max="4" value="0" /></label>
|
||||
<label>Mood — dark ◀ 0 ▶ light <input type="range" id="mood" min="-4" max="4" value="0" /></label>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>Calibration</legend>
|
||||
<label>mood gain <input type="range" id="mood_gain" min="0" max="2" step="0.05" value="1" /></label>
|
||||
<label>overlay gain <input type="range" id="overlay_gain" min="0" max="2" step="0.05" value="1" /></label>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>RenderPlan readout</legend>
|
||||
|
||||
@@ -7,13 +7,38 @@ main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; }
|
||||
.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000;
|
||||
border-radius: 6px; overflow: hidden; }
|
||||
#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; }
|
||||
/* Right-dream painterly canvas: a WebGL Kuwahara restyle of the LIVE video frames,
|
||||
drawn over the base. Edge-preserving, so motion stays as crisp as the source —
|
||||
the dream is the same footage, just stylized. The mood grade rides as a CSS
|
||||
filter here. Hidden when WebGL is unavailable (CSS fallback shows #vid). */
|
||||
#paint { position: absolute; inset: 0; width: 100%; height: 100%; object-fit: cover;
|
||||
pointer-events: none; transition: filter 0.2s ease; }
|
||||
#tint { position: absolute; inset: 0; pointer-events: none; opacity: 0;
|
||||
background: #28425f; mix-blend-mode: multiply;
|
||||
transition: opacity 0.2s ease; }
|
||||
#overlay { position: absolute; inset: 0; width: 100%; height: 100%;
|
||||
pointer-events: none; transition: opacity 0.2s ease; }
|
||||
.anno-box { fill: none; stroke: #6cf; stroke-width: 0.4; vector-effect: non-scaling-stroke; }
|
||||
.anno-label { fill: #6cf; font-size: 3px; font-family: monospace; }
|
||||
/* Affect channel — its own layer so its opacity tracks min(left,right), not Left.
|
||||
Soft violet glow, distinct from the clinical reticles (the dream leaking in). */
|
||||
#affect { position: absolute; inset: 0; width: 100%; height: 100%;
|
||||
pointer-events: none; transition: opacity 0.35s ease;
|
||||
filter: drop-shadow(0 0 0.9px rgba(176, 120, 255, 0.95)); }
|
||||
.hud-affect { fill: #e3cfff; font: italic 600 3.6px Georgia, "Times New Roman", serif;
|
||||
letter-spacing: 0.06px; }
|
||||
/* Left-brain analytical HUD. Two sensor channels: detection (cyan) + measurement
|
||||
(amber). Corner-bracket reticles, translucent label chips, a status tag. */
|
||||
.hud-reticle { fill: none; stroke-width: 0.6; vector-effect: non-scaling-stroke;
|
||||
stroke-linecap: square; }
|
||||
.hud-reticle.detect { stroke: #6cf; }
|
||||
.hud-reticle.measure { stroke: #ffc46b; }
|
||||
.hud-chip-bg { fill-opacity: 0.72; }
|
||||
.hud-chip-bg.detect { fill: #001722; }
|
||||
.hud-chip-bg.measure { fill: #1c1200; }
|
||||
.hud-chip-text { font: 2.4px monospace; letter-spacing: 0.04px; }
|
||||
.hud-chip-text.detect { fill: #aee6ff; }
|
||||
.hud-chip-text.measure { fill: #ffd79a; }
|
||||
.hud-conf { fill: #6cf; opacity: 0.8; }
|
||||
.hud-status { fill: #8fdcff; font: 2.4px monospace; opacity: 0.9; letter-spacing: 0.15px; }
|
||||
.black { position: absolute; inset: 0; background: #000; }
|
||||
.hidden { display: none; }
|
||||
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem; }
|
||||
|
||||
@@ -3,11 +3,12 @@ import pytest
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import (
|
||||
DEFAULT_CALIBRATION,
|
||||
AffectOverlay,
|
||||
AnalyticalOverlay,
|
||||
Calibration,
|
||||
ColorGrade,
|
||||
Dream,
|
||||
RenderPlan,
|
||||
Restyle,
|
||||
plan_alteration,
|
||||
render_plan_to_dict,
|
||||
)
|
||||
@@ -22,7 +23,8 @@ def test_all_zero_knobs_is_the_unaltered_base():
|
||||
assert plan.is_identity
|
||||
assert plan.overlay.level == 0
|
||||
assert plan.overlay.intensity == 0.0
|
||||
assert plan.restyle.variant == 0
|
||||
assert plan.dream.strength == 0
|
||||
assert plan.dream.intensity == 0.0
|
||||
assert plan.grade.tone == 0.0
|
||||
assert plan.grade.is_identity
|
||||
|
||||
@@ -31,21 +33,65 @@ def test_left_drives_the_analytical_overlay_only():
|
||||
plan = plan_alteration(_coord(left=4))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.variant == 0 # Left does not touch the substrate
|
||||
assert plan.dream.strength == 0 # Left does not touch the substrate
|
||||
assert plan.dream.intensity == 0.0
|
||||
assert plan.grade.tone == 0.0
|
||||
|
||||
|
||||
def test_right_selects_a_discrete_restyle_variant_only():
|
||||
def test_right_drives_the_deterministic_dream_only():
|
||||
# Right-axis dream reframe (session 0013): Right is a live deterministic
|
||||
# haze, carried as strength (0..4) + intensity (0..1), not a baked variant.
|
||||
plan = plan_alteration(_coord(right=2))
|
||||
assert plan.restyle.variant == 2
|
||||
assert plan.dream.strength == 2
|
||||
assert plan.dream.intensity == pytest.approx(0.5)
|
||||
assert plan.overlay.level == 0 # Right does not add overlay
|
||||
|
||||
|
||||
def test_left_and_right_stack_not_cancel():
|
||||
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
|
||||
# design §4.2: whole-brain corner = dream substrate WITH labels on top
|
||||
plan = plan_alteration(_coord(left=4, right=4))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.restyle.variant == 4
|
||||
assert plan.dream.strength == 4
|
||||
assert plan.dream.intensity == 1.0
|
||||
|
||||
|
||||
def test_affect_needs_both_knobs_up():
|
||||
# affect-channel design (session 0013): emotion-words surface only when BOTH
|
||||
# the Left and Right knobs are up — either at 0 means no affect at all.
|
||||
assert plan_alteration(_coord(left=4, right=0)).affect.strength == 0
|
||||
assert plan_alteration(_coord(left=0, right=4)).affect.strength == 0
|
||||
assert plan_alteration(_coord(left=4, right=0)).affect.intensity == 0.0
|
||||
assert plan_alteration(_coord(left=0, right=4)).affect.intensity == 0.0
|
||||
|
||||
|
||||
def test_affect_strength_is_the_smaller_knob():
|
||||
# strength = min(left, right) — the smaller knob gates the channel.
|
||||
assert plan_alteration(_coord(left=2, right=4)).affect.strength == 2
|
||||
assert plan_alteration(_coord(left=4, right=2)).affect.strength == 2
|
||||
assert plan_alteration(_coord(left=4, right=4)).affect.strength == 4
|
||||
assert plan_alteration(_coord(left=3, right=1)).affect.strength == 1
|
||||
|
||||
|
||||
def test_affect_intensity_scales_with_strength_over_full_scale():
|
||||
for lo in range(5):
|
||||
plan = plan_alteration(_coord(left=lo, right=4))
|
||||
assert plan.affect.intensity == pytest.approx(lo / 4)
|
||||
assert plan_alteration(_coord(left=4, right=4)).affect.intensity == 1.0
|
||||
|
||||
|
||||
def test_affect_intensity_honors_overlay_gain():
|
||||
# affect rides the same overlay_gain as the analytical HUD it belongs to.
|
||||
cal = Calibration(overlay_gain=0.5)
|
||||
assert plan_alteration(_coord(left=4, right=4), cal).affect.intensity == pytest.approx(0.5)
|
||||
clamp = Calibration(overlay_gain=10.0)
|
||||
assert plan_alteration(_coord(left=4, right=4), clamp).affect.intensity == 1.0
|
||||
|
||||
|
||||
def test_affect_active_implies_non_identity():
|
||||
# min(left,right) > 0 implies left > 0, so an active affect plan is never identity.
|
||||
plan = plan_alteration(_coord(left=2, right=3))
|
||||
assert plan.affect.strength == 2
|
||||
assert not plan.is_identity
|
||||
|
||||
|
||||
def test_light_pole_grades_warm_positive_tone():
|
||||
@@ -72,7 +118,8 @@ def test_dark_minus_light_sets_intermediate_tone():
|
||||
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
|
||||
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.restyle.variant == 2
|
||||
assert plan.dream.strength == 2
|
||||
assert plan.dream.intensity == pytest.approx(0.5)
|
||||
assert plan.grade.tone == -1.0
|
||||
assert not plan.is_identity
|
||||
|
||||
@@ -84,32 +131,34 @@ def test_default_calibration_is_locked():
|
||||
# locked feel is a deliberate edit here + in alteration.py.
|
||||
assert DEFAULT_CALIBRATION.mood_gain == 1.0
|
||||
assert DEFAULT_CALIBRATION.overlay_gain == 1.0
|
||||
assert DEFAULT_CALIBRATION.right_variant_map == (0, 1, 2, 3, 4)
|
||||
assert DEFAULT_CALIBRATION.dream_gain == 1.0
|
||||
|
||||
|
||||
def test_default_calibration_is_behavior_preserving():
|
||||
# DEFAULT_CALIBRATION must reproduce the original three helpers exactly.
|
||||
# DEFAULT_CALIBRATION reproduces the per-axis helpers exactly.
|
||||
for left in range(5):
|
||||
assert plan_alteration(_coord(left=left)).overlay.intensity == pytest.approx(left / 4)
|
||||
for right in range(5):
|
||||
assert plan_alteration(_coord(right=right)).restyle.variant == right
|
||||
assert plan_alteration(_coord(right=right)).dream.strength == right
|
||||
assert plan_alteration(_coord(right=right)).dream.intensity == pytest.approx(right / 4)
|
||||
for dark in range(5):
|
||||
for light in range(5):
|
||||
expected = (light - dark) / 4
|
||||
assert plan_alteration(_coord(dark=dark, light=light)).grade.tone == pytest.approx(expected)
|
||||
|
||||
|
||||
def test_custom_calibration_scales_mood_and_overlay():
|
||||
cal = Calibration(mood_gain=0.5, overlay_gain=0.5, right_variant_map=(0, 0, 1, 1, 2))
|
||||
def test_custom_calibration_scales_mood_overlay_and_dream():
|
||||
cal = Calibration(mood_gain=0.5, overlay_gain=0.5, dream_gain=0.5)
|
||||
assert plan_alteration(_coord(light=4), cal).grade.tone == pytest.approx(0.5)
|
||||
assert plan_alteration(_coord(left=4), cal).overlay.intensity == pytest.approx(0.5)
|
||||
assert plan_alteration(_coord(right=3), cal).restyle.variant == 1
|
||||
assert plan_alteration(_coord(right=4), cal).dream.intensity == pytest.approx(0.5)
|
||||
|
||||
|
||||
def test_calibration_gain_is_clamped_to_unit_range():
|
||||
cal = Calibration(mood_gain=10.0, overlay_gain=10.0)
|
||||
cal = Calibration(mood_gain=10.0, overlay_gain=10.0, dream_gain=10.0)
|
||||
assert plan_alteration(_coord(light=4), cal).grade.tone == 1.0 # clamped, not 10
|
||||
assert plan_alteration(_coord(left=4), cal).overlay.intensity == 1.0
|
||||
assert plan_alteration(_coord(right=4), cal).dream.intensity == 1.0
|
||||
|
||||
|
||||
def test_render_plan_to_dict_round_trips_the_numbers():
|
||||
@@ -117,7 +166,8 @@ def test_render_plan_to_dict_round_trips_the_numbers():
|
||||
assert d == {
|
||||
"grade": {"tone": -1.0},
|
||||
"overlay": {"level": 4, "intensity": 1.0},
|
||||
"restyle": {"variant": 2},
|
||||
"affect": {"strength": 2, "intensity": 0.5},
|
||||
"dream": {"strength": 2, "intensity": 0.5},
|
||||
"is_identity": False,
|
||||
}
|
||||
|
||||
|
||||
@@ -59,13 +59,15 @@ def test_overlay_change_is_a_live_update():
|
||||
assert t.playback.plan.overlay.level == 4
|
||||
|
||||
|
||||
def test_restyle_change_crossfades_the_substrate():
|
||||
# design §4.3: the Right v2v substrate is a pre-baked variant swap
|
||||
def test_dream_change_is_a_live_update_not_a_crossfade():
|
||||
# Right-axis dream reframe (session 0013): the Right dream is a deterministic
|
||||
# LIVE filter now, so changing it no longer crossfades — only a clip swap does.
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", right=0))
|
||||
t = p.update(_controls(content="video", right=4))
|
||||
assert t.kind == TransitionKind.CROSSFADE
|
||||
assert t.playback.plan.restyle.variant == 4
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.dream.strength == 4
|
||||
assert t.playback.plan.dream.intensity == 1.0
|
||||
|
||||
|
||||
def test_volume_only_change_is_a_live_update():
|
||||
|
||||
@@ -38,7 +38,8 @@ def test_alteration_returns_the_engine_plan(client):
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["plan"]["overlay"]["level"] == 4
|
||||
assert data["plan"]["restyle"]["variant"] == 2
|
||||
assert data["plan"]["dream"]["strength"] == 2
|
||||
assert data["plan"]["dream"]["intensity"] == 0.5
|
||||
assert data["plan"]["grade"]["tone"] == -1.0
|
||||
assert data["content"]["video"] is True
|
||||
|
||||
@@ -51,7 +52,7 @@ def test_alteration_honors_off_as_black(client):
|
||||
|
||||
def test_alteration_accepts_calibration(client):
|
||||
body = {"controls": _controls(light=4),
|
||||
"calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "right_variant_map": [0, 1, 2, 3, 4]}}
|
||||
"calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "dream_gain": 1.0}}
|
||||
resp = client.post("/api/alteration", json=body)
|
||||
assert resp.json()["plan"]["grade"]["tone"] == 0.5
|
||||
|
||||
|
||||
Reference in New Issue
Block a user