spec(design): scrub-driven altitude transitions (knob position drives morph + audio)

Approved design for the next increment: continuous knob position scrubs the morph
video + crossfades audio (hold-partway, reversible, re-roll-on-fresh-approach,
wheel auto-scrubs one altitude). Includes morph all-intra re-bake for smooth
seeking. To be planned + built in a separate session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
BenStullsBets
2026-06-27 15:19:32 -07:00
parent 9b4884b466
commit 972ad8832b
@@ -0,0 +1,105 @@
# Scrub-driven Altitude Transitions — Design
**Status:** Approved (brainstormed 2026-06-27, session 0024). Ready for writing-plans.
## Goal
Make the altitude transition **driven by the knob position** rather than auto-played.
Turning the physical knob (or, in the simulator, dragging the Altitude dial) at any
speed scrubs the morph video and crossfades the audio in lockstep with the knob. The
knob position *is* the scene: turn slow → the morph eases slowly; turn fast → it
races; stop halfway → it holds in a mid-morph blend; turn back → it reverses exactly.
This supersedes the timed dial-needle sweep (the dial-vs-video sync problem dissolves
because the knob and the transition become the same thing).
## Core model
A continuous **altitude position** `pos` (float): integers are altitudes/detents,
fractions are mid-morph blend. `pos = 2.4` ⇒ 40 % from coast (index 2) toward reef
(index 3). The knob/needle maps directly to `pos`; the morph video's `currentTime`
and the audio crossfade are scrubbed by `frac(pos)`.
**Decisions (locked):**
- **In-between state:** hold the blend wherever the knob stops — no auto-complete
(continuous encoder model). *(Q1 → a)*
- **Turn-back:** scrubs the same morph in reverse; lands on exactly the previous
altitude's clip you started from (fully reversible).
- **Destination randomness:** the next altitude's clip is a random pick from its pool,
**fixed during a single continuous gesture**, **re-rolled on each fresh approach**.
*(Q → a)*
- **Scroll wheel:** auto-scrub one altitude over ~0.6 s, then lock. *(Q2 → a)*
- **Lock-per-altitude** still holds: a resting altitude's clip stays locked until you
commit into a different one.
## Components
### 1. Scrub controller (frontend — `simulator/static/app.js`)
Replaces the discrete `advance(delta)` "commit then play" for drag/knob input.
- Drives `pos` from the dial pointer angle: `pos = restPos + accum / dialStep`. The
needle follows continuously (the needle *is* the knob — live follow is correct here).
- For the active segment `[floor, floor+1]` in the travel direction, sets
`vid.currentTime = frac(pos) × vid.duration`, **throttled to one seek per
`requestAnimationFrame`** (avoid seek thrash).
- Crossing an integer **commits**: the crossed altitude's chosen clip becomes the
locked `activeClipId`, `ringIndex` updates, and the next segment begins (re-roll).
- Reversible: `pos` decreasing scrubs the morph backward.
- Keep the pure, testable bits separable (position→segment, `frac`→currentTime,
integer-crossing commit, re-roll-on-fresh-approach) from the DOM/event glue.
### 2. Clip selection / lock
Entering a segment toward neighbor `k±1`, pick a random clip from that scale's pool
(client-side) and resolve the directed morph via the existing
`morphByPair["<fromClip>→<toClip>"]` lookup (built from `ring.transitions`). Fixed for
the gesture; re-rolled on a fresh approach. *(This moves the random pick client-side
for responsiveness vs the current server pick in `/api/ring/advance`; acceptable for
the simulator — the Pi player can mirror the same client-side pick. `pick_clip_id` in
`player/ring.py` stays the canonical pure helper.)*
### 3. Audio crossfade (frontend)
During a segment, crossfade the two adjacent scale soundtracks by `frac(pos)` (gains
`1-frac` / `frac`); at rest only the current scale plays. Hook into the existing audio
layer (`applyAudio` / the per-scale `audio` field in the ring).
### 4. Morph media re-bake (pipeline — `simulator/build_pool_manifest.py`)
Smooth scrubbing = seeking to arbitrary frames, which needs **dense keyframes**. The
current morphs use a sparse GOP, so they scrub "steppy." Re-bake the 154 morphs
**all-intra** (e.g. x264 `-g 1` / `keyint=1`, or `-intra`) so every frame is seekable.
Files grow (~176 MB → ~0.50.9 GB) but each clip stays small (front-end upload limit is
a non-issue; that was the 68 MB *base*, unrelated). Media is committed via **git-LFS**
(see `.gitattributes`); re-push after re-baking.
**Phasing:** build the interaction first against the *current* morphs to validate the
feel (will scrub somewhat steppy), then re-bake all-intra for smoothness. Two plan
phases.
### 5. Wheel + tap
- **Wheel:** auto-scrub `pos` by ±1 over ~0.6 s (drives the same scrub path), then lock.
- **Tap a dial label:** auto-scrub to that altitude the shortest way around.
## Testing
- **Unit (pytest or JS):** the pure controller logic — position→segment mapping,
`frac``currentTime`, commit+lock on integer crossing, re-roll on fresh approach,
reverse direction.
- **E2E (Playwright, `simulator/e2e`):** drag the dial partway → assert the morph
`currentTime` and the two audio gains track the angle; drag back → values reverse;
a full turn → commits and locks the destination clip; the wheel → auto-scrubs one
altitude and locks. (Extends the existing `altitude-lock.spec.ts` tier.)
## Out of scope
- Physical hardware / encoder firmware (the simulator drag is the stand-in; the model
is designed to map 1:1 to an encoder position later — see the networked control-surface
spec).
- Changing the clip-pair morph *set* or naming (`<src>__<dst>.mp4` + `.rev`) — reused as-is.
## Key existing context (for the implementer)
- Dial + transition logic: `simulator/static/app.js` (`advance`, `playTransition`,
`onDialDown/Move/Up`, `renderDial/setNeedle`, `morphByPair`, the preload helpers).
- Ring/morph contract: `player/ring.py` (`Scale.pool`, `pick_clip_id`, `morph_for`,
`resolve_move`), `simulator/clips.py` (`ring_to_dict`, `resolved_move_to_dict`),
`/api/ring/advance` in `simulator/app.py`.
- Morph manifest + baking: `simulator/build_pool_manifest.py` (`_make_transition`,
`_make_reverse`, `generate_media`, `build_manifest`); morphs at
`simulator/sample_media/transitions/<src>__<dst>.mp4` (+ `.rev`).
- The clip-pair morph + lock feature this builds on shipped in session 0024 (plan:
`docs/superpowers/plans/2026-06-27-altitude-clip-pair-morph-lock.md`).