Compare commits

..

31 Commits

Author SHA1 Message Date
benstull 108e6207ba Merge pull request 'feat(sim): scrub-driven altitude transitions' (#28) from session-0026 into main 2026-06-28 01:46:54 +00:00
BenStullsBets b710b259f2 update sessions/0026/SESSION-0026.0-TRANSCRIPT-2026-06-27T15-26--INPROGRESS.md 2026-06-27 17:39:07 -07:00
BenStullsBets 0e401c9a42 feat(sim): auto-scrub at constant per-altitude speed, 2x slower
Operator: slow click/auto transitions 2x and make them the same speed between any two
altitudes (2 away should take twice as long as 1). autoScrub duration is now
PROPORTIONAL to distance (PER_ALTITUDE_MS=1200, ~2x the old fixed 600ms total): a
label-click or wheel jump of N altitudes takes N*1200ms, so per-step speed is constant.
Manual drag is unaffected (operator drives its speed). E2E asserts a 2-step jump is
~2x a 1-step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 17:36:45 -07:00
BenStullsBets 7f99c3cdc4 update sessions/0026/SESSION-0026.0-TRANSCRIPT-2026-06-27T15-26--INPROGRESS.md 2026-06-27 17:08:39 -07:00
BenStullsBets 4b1d3afe7b fix(sim): cancel in-flight volume fades so audio isn't silenced after landing
Operator: audio turned off sometimes after landing on a new altitude. Root cause:
fadeVolume started a setInterval ramp that was never cancelled. When restAudio faded
the idle element to 0 and — within FADE_MS — that same element became the newly-landed
ACTIVE scale (assignElements reuses an element per soundtrack url), the stale fade-to-0
kept running and dragged the now-active element back to silence. Fix: cancelFade()
supersedes any in-flight fade when a new fade starts (fadeVolume) or a direct volume
is set (ensurePlaying). E2E proves a new fade supersedes a stale one (fails without
the fix).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 17:07:52 -07:00
BenStullsBets 153681a402 update sessions/0026/SESSION-0026.0-TRANSCRIPT-2026-06-27T15-26--INPROGRESS.md 2026-06-27 17:03:10 -07:00
BenStullsBets e862fb498b feat(sim): audio on/off toggle -> 0-10 level dial; first video lifts it to 3/10
Operator request. Replaces the Audio checkbox with a 0-10 range slider that sets the
master soundtrack gain (audioVol = level/10), scaling both the per-scale rest volume
and the two-track scrub crossfade. Level 0 = silent. Turning Video on for the first
time now raises audio to 3/10 (was: full on) so the experience opens gently; raising
the slider from 0 unlocks playback in-gesture (Safari). E2E: dial is range 0-10
default 0, and first video-on lifts it to 3/10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 17:02:19 -07:00
BenStullsBets ea530b3df6 fix(sim): double-buffer video so morph->loop handoff doesn't stall
Operator eyeball: a jolt/pause when the morph ends and the next altitude's loop
loads. Root cause: swapping vid.src to the base + seeking to the tail reloaded the
ON-SCREEN element (decode + seek stall). Now a dedicated #vid-loop element carries
the steady-state base loop; during a scrub it is PRELOADED to the clip we're heading
toward, seeked to the tail and paused on that exact frame. The paint shader reads the
active source (busy ? morph : loop), so landing is an instant source swap — no reload,
no seek. E2E asserts the loop element is armed + playing from ~3s after landing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 16:58:08 -07:00
BenStullsBets 80952d7b8e update sessions/0026/SESSION-0026.0-TRANSCRIPT-2026-06-27T15-26--INPROGRESS.md 2026-06-27 16:33:42 -07:00
BenStullsBets 0752746f7b fix(content): strip NOAA title card from reef_snapper base + re-bake its morphs
Operator eyeball feedback: the red snapper clip opened with a black 'Northern Red
Snapper / fishwatch.gov' title card. Re-trimmed reef_snapper/base.mp4 to drop the
first 3.0s (the card; footage is clean from 3.0s), 23s->20s, and re-baked the 7
reef_snapper morph pairs (+ reverses) all-intra from the clean base. Verified the
snapper-dominant morph frames are text-free.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 16:32:14 -07:00
BenStullsBets 614411b5e4 fix(sim): loop base from morph-tail offset so landed morphs don't jitter
Operator eyeball feedback: landing on an altitude jump-cut the loop back to frame 0.
Each morph reproduces the destination clip's first 3s (pipeline trim=0:3) and ends on
dst@~3s, so the steady-state loop now starts at LOOP_TAIL_S=3.0 and loops within
[3.0, duration] — the morph's last frame hands straight off to the loop, and the
morph-covered intro never re-shows at rest (removes the overlapping 'altitude video
that is part of the morph'). E2E asserts a landed clip loops from ~3s, not 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 16:29:22 -07:00
BenStullsBets a5e44ea9c7 update sessions/0026/SESSION-0026.0-TRANSCRIPT-2026-06-27T15-26--INPROGRESS.md 2026-06-27 16:18:33 -07:00
BenStullsBets 70fc9a46ab feat(pipeline): re-bake 154 morphs all-intra for smooth scrub seeking (LFS)
Per plan Task 7. Every frame now a keyframe (verified 93/93 I-frames on a sample),
so the scrub interaction seeks to arbitrary frames smoothly. Total transitions
173M -> 504M; largest clip ~5.8M (well under the 25MB LFS/proxy ceiling).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:51:53 -07:00
BenStullsBets f2f242a411 feat(pipeline): all-intra ffmpeg builders for seekable morph re-bake
Per plan Task 7. Extracts transition_cmd/reverse_cmd argv builders and adds the
ALL_INTRA flag set (-g 1 -keyint_min 1 -sc_threshold 0) so every baked frame is a
keyframe — smooth arbitrary-frame seeking for the scrub interaction. Unit-tested.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:50:31 -07:00
BenStullsBets ad0a59ac04 update sessions/0026/SESSION-0026.0-TRANSCRIPT-2026-06-27T15-26--INPROGRESS.md 2026-06-27 15:49:02 -07:00
BenStullsBets 31a3fd733f test(e2e): scrub tier — currentTime + gains track dial, reverse, commit-lock, wheel
Per plan Task 5. Replaces the .rev-asserting zoom-out test with a turn-back test
(same forward morph seeked backward, audio recedes, re-locks the start clip); adds
a detent-crossing commit+lock test via a new window.__hefState diagnostic seam.
Full tier (6 tests) green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:47:25 -07:00
BenStullsBets 8458ab59eb feat(sim): wheel + label-tap auto-scrub then lock; retire discrete advance()
Per plan Task 4. onWheel + jumpToScale now animate pos via autoScrub() (drives the
same setPos scrub path, ~600ms, lands exactly on the integer + locks). Removes the
now-dead advance()/playTransition()/FAST_BLEND_RATE (the server /api/ring/advance
round-trip is no longer used by the client). Post-landing morph warm-up folded into
setPos's settle branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:45:37 -07:00
BenStullsBets 20f49e936a feat(sim): scrub engine — drag drives pos, seeks morph, commits+locks+re-rolls
Per plan Task 3. A continuous float pos is the knob: setPos() seeks the segment's
morph currentTime by frac (throttled one seek/rAF), crossfades audio, sweeps the
needle live, and commits+locks+re-rolls on each integer crossing. Drag live-scrubs
(no more commit-on-release); release holds the blend (no auto-complete). Client-side
pool pick replaces the server round-trip for drag.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:43:48 -07:00
BenStullsBets 0548e9a82b feat(sim): two-element audio crossfade keyed by scrub fraction
Per plan Task 2. Adds <audio id=aud-b>; blendAudio(lo,frac) mixes the two
adjacent scale soundtracks by knob fraction (element reuse across crossings so
only the genuinely-new scale reloads); restAudio settles a single scale at rest.
Diagnostic readout now reports whichever element is audible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:40:04 -07:00
BenStullsBets 3dc3e3491f feat(sim): pure altitude-scrub math module + node unit suite
Per plan Task 1. position->segment, frac->currentTime, audio gains, and
integer-crossing detection; 9 node --test cases green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:37:27 -07:00
BenStullsBets 937461e3ba docs(plan): scrub-driven altitude transitions implementation plan
Plan for the approved design at
docs/superpowers/specs/2026-06-27-scrub-driven-altitude-transitions-design.md.
Two phases: (1) scrub interaction against current morphs, (2) all-intra re-bake.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:35:23 -07:00
BenStullsBets cf74d2c70a claim human-experience-filter-art session 0026 (placeholder) + sessions.json entry 2026-06-27 15:28:02 -07:00
BenStullsBets 6781e40c94 add sessions/0024/SESSION-0024.0-TRANSCRIPT-2026-06-26T21-12--2026-06-27T15-23.md + replace placeholder/variant SESSION-0024.0-TRANSCRIPT-2026-06-26T21-12--INPROGRESS.md 2026-06-27 15:25:28 -07:00
BenStullsBets 66c65e78b8 update sessions/0024/SESSION-0024.0-TRANSCRIPT-2026-06-26T21-12--INPROGRESS.md 2026-06-27 15:24:43 -07:00
BenStullsBets 528fb5c438 Merge origin/main (session-history commits 0023-0025) into 0024 feature work 2026-06-27 15:22:13 -07:00
BenStullsBets c2cc503a9c Merge session-0024: altitude clip-pair morph + lock; scrub-driven design spec
Ships the altitude clip-pair morph transition system (154 directional morphs,
pick-then-morph, chained, lock-per-altitude), graduated experience media into
git-LFS, the Playwright E2E tier, and the dial-needle sweep. Also lands the
approved design spec for the next increment (scrub-driven transitions).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 15:21:31 -07:00
BenStullsBets e2b54dd0cc add sessions/0025/SESSION-0025.0-TRANSCRIPT-2026-06-27T06-05--2026-06-27T07-05.md + replace placeholder/variant SESSION-0025.0-TRANSCRIPT-2026-06-27T06-05--INPROGRESS.md 2026-06-27 07:05:51 -07:00
BenStullsBets a581fa41ed claim human-experience-filter-art session 0025 (placeholder) + sessions.json entry 2026-06-27 06:05:49 -07:00
BenStullsBets afec816bb0 claim human-experience-filter-art session 0024 (placeholder) + sessions.json entry 2026-06-26 21:13:13 -07:00
BenStullsBets df869d6978 add sessions/0023/SESSION-0023.0-TRANSCRIPT-2026-06-26T09-05--2026-06-26T09-20.md + replace placeholder/variant SESSION-0023.0-TRANSCRIPT-2026-06-26T09-05--INPROGRESS.md 2026-06-26 18:59:46 -07:00
BenStullsBets 8aa5ee4019 claim human-experience-filter-art session 0023 (placeholder) + sessions.json entry 2026-06-26 18:59:05 -07:00
170 changed files with 2192 additions and 513 deletions
@@ -0,0 +1,803 @@
# Scrub-driven Altitude Transitions Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make the Altitude knob position *continuously* drive the scale transition — dragging the dial scrubs the morph video's `currentTime` and crossfades the two adjacent scale soundtracks by knob angle, holding wherever the knob stops, fully reversible.
**Architecture:** Introduce a continuous float **position** `pos` (integers = altitudes/detents, fractions = mid-morph blend). A small pure module (`scrub.js`) owns the math (position→segment, frac→currentTime, audio gains, integer-crossing detection); `app.js` becomes the DOM/event glue that drives `pos` from the dial, seeks the morph `<video>`, crossfades two `<audio>` elements, and commits/locks/re-rolls on integer crossings. Phase 1 builds the interaction against the *current* (sparse-GOP) morphs to validate feel; Phase 2 re-bakes all 154 morphs all-intra for smooth seeking.
**Tech Stack:** Vanilla JS (no framework, plain `<script>`), Node built-in `node --test` for pure-logic unit tests, Playwright for E2E, Python + ffmpeg/x264 for the morph re-bake (`simulator/build_pool_manifest.py`), git-LFS for media.
## Global Constraints
- **Spec contract:** `docs/superpowers/specs/2026-06-27-scrub-driven-altitude-transitions-design.md` — implement its locked decisions verbatim.
- **In-between state:** hold the blend wherever the knob stops — **no auto-complete** (continuous-encoder model).
- **Turn-back:** scrubs the **same** segment morph in reverse; landing back on the start integer re-locks **exactly** the clip you left (fully reversible).
- **Destination randomness:** the destination altitude's clip is a random pick from its pool, **fixed for a single continuous gesture**, **re-rolled on each fresh approach** (each fresh entry into a segment from an integer).
- **Scroll wheel:** auto-scrub one altitude over ~0.6 s, then lock. **Tap a dial label:** auto-scrub to that altitude the shortest way around.
- **Lock-per-altitude** still holds: a resting altitude's clip stays locked until you commit into a different one.
- **Canonical segment file (locked interpretation):** for the segment between altitude indices `lo` and `lo+1`, always use the **descend/forward** morph `morphByPair["<clip@lo>→<clip@lo+1>"]`, with `currentTime = frac × duration`. Reverse travel seeks the **same** file backward. `.rev` files are not used by the scrub interaction (they remain baked for back-compat).
- **Git transport:** SSH only (`ssh://git@git.benstull.org/...`). No inline trailing comments on shell/CLI command lines.
- **Pipeline:** ship via §9 — localhost + E2E green → PPE + E2E green → prod (prod human-gated). Every UI change carries its E2E browser tests as first-class tasks.
---
## File Structure
- **Create** `simulator/static/scrub.js` — pure scrub math, UMD (browser global `HEFScrub` + CommonJS `module.exports`). No DOM.
- **Create** `simulator/unit/scrub.test.js``node --test` unit suite for `scrub.js`.
- **Create** `simulator/unit/README.md` — one-liner on running the unit suite.
- **Modify** `simulator/static/index.html` — load `scrub.js` before `app.js`; add a second `<audio id="aud-b" loop preload="auto">`.
- **Modify** `simulator/static/app.js` — replace the discrete drag/wheel/tap `advance()` flow with the scrub engine (`setPos`, `rebuildSegment`, `commitCrossings`, auto-scrub animator); two-element audio crossfade; client-side clip pick from `scale.pool`.
- **Modify** `simulator/e2e/tests/altitude-lock.spec.ts` (or add `scrub.spec.ts`) — assert `currentTime` + the two audio gains track the dial angle and reverse on turn-back; full turn commits+locks; wheel auto-scrubs one altitude and locks.
- **Modify** `simulator/build_pool_manifest.py``_make_transition` / `_make_reverse` emit **all-intra** H.264 (`-g 1 -keyint_min 1 -sc_threshold 0`) for seekable frames.
- **Modify** `tests/test_build_pool_manifest.py` — assert the ffmpeg arg list carries the all-intra flags.
---
## Phase 1 — Scrub interaction against current morphs
### Task 1: Pure scrub math module (`scrub.js`) + Node unit tests
**Files:**
- Create: `simulator/static/scrub.js`
- Create: `simulator/unit/scrub.test.js`
- Create: `simulator/unit/README.md`
**Interfaces:**
- Produces (all pure, no DOM), exported on `HEFScrub` and `module.exports`:
- `clamp01(x: number): number`
- `wrapIndex(i: number, n: number): number``((i % n) + n) % n`
- `segmentOf(pos: number): { lo: number, frac: number }``lo = floor(pos)`, `frac = pos - lo`
- `accumToPos(restPos: number, accumDeg: number, stepDeg: number): number``restPos + accumDeg/stepDeg`
- `fracToTime(frac: number, duration: number): number``clamp01(frac) * (duration || 0)`
- `crossfadeGains(frac: number): { from: number, to: number }``{ from: 1-clamp01(frac), to: clamp01(frac) }`
- `integerCrossings(prevPos: number, pos: number): Array<{ index: number, dir: number }>` — integers strictly crossed going `prevPos → pos`, in travel order, `dir` = `+1` descending (pos increasing) / `-1` ascending. Landing exactly on an integer counts as crossing it.
- [ ] **Step 1: Write the failing unit suite**
Create `simulator/unit/scrub.test.js`:
```js
"use strict";
const test = require("node:test");
const assert = require("node:assert/strict");
const S = require("../static/scrub.js");
test("clamp01 bounds to [0,1]", () => {
assert.equal(S.clamp01(-0.2), 0);
assert.equal(S.clamp01(0.4), 0.4);
assert.equal(S.clamp01(1.5), 1);
});
test("wrapIndex wraps both directions", () => {
assert.equal(S.wrapIndex(0, 5), 0);
assert.equal(S.wrapIndex(5, 5), 0);
assert.equal(S.wrapIndex(-1, 5), 4);
assert.equal(S.wrapIndex(7, 5), 2);
});
test("segmentOf splits floor and fraction", () => {
assert.deepEqual(S.segmentOf(2), { lo: 2, frac: 0 });
const s = S.segmentOf(2.4);
assert.equal(s.lo, 2);
assert.ok(Math.abs(s.frac - 0.4) < 1e-9);
});
test("accumToPos maps knob degrees to position", () => {
assert.equal(S.accumToPos(2, 0, 72), 2);
assert.equal(S.accumToPos(2, 72, 72), 3);
assert.equal(S.accumToPos(2, -36, 72), 1.5);
});
test("fracToTime clamps and scales by duration", () => {
assert.equal(S.fracToTime(0, 1.5), 0);
assert.equal(S.fracToTime(0.5, 1.5), 0.75);
assert.equal(S.fracToTime(1.2, 1.5), 1.5);
assert.equal(S.fracToTime(0.5, 0), 0);
});
test("crossfadeGains splits energy by fraction", () => {
assert.deepEqual(S.crossfadeGains(0), { from: 1, to: 0 });
assert.deepEqual(S.crossfadeGains(1), { from: 0, to: 1 });
assert.deepEqual(S.crossfadeGains(0.25), { from: 0.75, to: 0.25 });
});
test("integerCrossings: none while inside a segment", () => {
assert.deepEqual(S.integerCrossings(2.1, 2.9), []);
assert.deepEqual(S.integerCrossings(2.0, 2.5), []);
assert.deepEqual(S.integerCrossings(3.0, 2.5), []);
});
test("integerCrossings: ascending and descending single crossings", () => {
assert.deepEqual(S.integerCrossings(2.4, 3.0), [{ index: 3, dir: 1 }]);
assert.deepEqual(S.integerCrossings(2.0, 3.0), [{ index: 3, dir: 1 }]);
assert.deepEqual(S.integerCrossings(3.4, 3.0), [{ index: 3, dir: -1 }]);
});
test("integerCrossings: multiple crossings in travel order", () => {
assert.deepEqual(S.integerCrossings(2.4, 4.1), [{ index: 3, dir: 1 }, { index: 4, dir: 1 }]);
assert.deepEqual(S.integerCrossings(3.2, 1.8), [{ index: 3, dir: -1 }, { index: 2, dir: -1 }]);
});
```
Create `simulator/unit/README.md`:
```markdown
# Simulator unit tests
Pure-logic JS unit tests (no browser), run with Node's built-in test runner:
node --test simulator/unit/
`scrub.test.js` covers `simulator/static/scrub.js` — the pure altitude-scrub math
(position→segment, frac→currentTime, audio gains, integer-crossing detection).
```
- [ ] **Step 2: Run the suite to verify it fails**
Run: `node --test simulator/unit/scrub.test.js`
Expected: FAIL — `Cannot find module '../static/scrub.js'`.
- [ ] **Step 3: Write `scrub.js` to pass**
Create `simulator/static/scrub.js`:
```js
// Pure altitude-scrub math — no DOM. A continuous `pos` (float) is the knob:
// integers are altitudes/detents, fractions are mid-morph blend. UMD so the
// browser gets a `HEFScrub` global and `node --test` can require() it.
(function (root, factory) {
const api = factory();
if (typeof module !== "undefined" && module.exports) module.exports = api;
else root.HEFScrub = api;
})(typeof self !== "undefined" ? self : this, function () {
"use strict";
const clamp01 = (x) => Math.max(0, Math.min(1, x));
const wrapIndex = (i, n) => ((i % n) + n) % n;
function segmentOf(pos) {
const lo = Math.floor(pos);
return { lo, frac: pos - lo };
}
function accumToPos(restPos, accumDeg, stepDeg) {
return restPos + accumDeg / stepDeg;
}
function fracToTime(frac, duration) {
return clamp01(frac) * (duration || 0);
}
function crossfadeGains(frac) {
const f = clamp01(frac);
return { from: 1 - f, to: f };
}
// Integers strictly crossed moving prevPos -> pos, in travel order. Landing
// exactly on an integer counts as crossing it (it commits that altitude).
function integerCrossings(prevPos, pos) {
const out = [];
if (pos > prevPos) {
for (let k = Math.floor(prevPos) + 1; k <= pos + 1e-9; k++) out.push({ index: k, dir: 1 });
} else if (pos < prevPos) {
for (let k = Math.ceil(prevPos) - 1; k >= pos - 1e-9; k--) out.push({ index: k, dir: -1 });
}
return out;
}
return { clamp01, wrapIndex, segmentOf, accumToPos, fracToTime, crossfadeGains, integerCrossings };
});
```
- [ ] **Step 4: Run the suite to verify it passes**
Run: `node --test simulator/unit/scrub.test.js`
Expected: PASS — all tests, 0 failures.
- [ ] **Step 5: Commit**
```bash
git add simulator/static/scrub.js simulator/unit/scrub.test.js simulator/unit/README.md
git commit -m "feat(sim): pure altitude-scrub math module + node unit suite"
```
---
### Task 2: Two-element audio crossfade
**Files:**
- Modify: `simulator/static/index.html` (add `<audio id="aud-b">`; load `scrub.js`)
- Modify: `simulator/static/app.js` (audio layer ~lines 9501045)
- Test: `simulator/unit/scrub.test.js` (gains already covered in Task 1 — no new pure logic)
**Interfaces:**
- Consumes: `HEFScrub.crossfadeGains`, `soundtrackUrl()`-style per-scale URL resolution.
- Produces:
- `scaleAudioUrl(index: number): string|null` — the soundtrack URL for ring scale `index` (wrapped), using the ring `audio` field then `SCALE_AUDIO_FALLBACK`.
- `blendAudio(loIndex: number, frac: number): void` — element A plays scale `loIndex`, element B plays scale `loIndex+1` (wrapped), gains `crossfadeGains(frac)`; loads each element's src on first use per gesture. No-op when Audio toggle is off.
- `restAudio(index: number): void` — settle to a single scale at rest (full gain on the element holding `index`, fade the other to 0). Replaces the tail of `applyAudio()` for the at-rest case.
- [ ] **Step 1: Add the second audio element + scrub.js script tag**
In `simulator/static/index.html`, beside the existing `<audio id="aud" loop preload="auto"></audio>` (line ~21), add:
```html
<audio id="aud" loop preload="auto"></audio>
<audio id="aud-b" loop preload="auto"></audio>
```
And load the pure module before `app.js` (line ~112):
```html
<script src="/scrub.js"></script>
<script src="/app.js"></script>
```
- [ ] **Step 2: Write the failing E2E expectation (gains exist on two elements)**
Add to `simulator/e2e/tests/altitude-lock.spec.ts` a focused check (full assertions land in Task 5; this verifies the element + hook exist):
```ts
test("two audio elements exist for crossfade", async ({ page }) => {
await boot(page);
const n = await page.evaluate(() => document.querySelectorAll("audio#aud, audio#aud-b").length);
expect(n).toBe(2);
});
```
Run: `cd simulator/e2e && npx playwright test -g "two audio elements"`
Expected: FAIL until the markup change is loaded (and PASS once the running server serves the new `index.html`).
- [ ] **Step 3: Implement the crossfade audio layer in `app.js`**
Replace the single-element logic around `playUrl`/`applyAudio` (app.js ~10191045). Keep `aud` as element A and add `audB` as element B; generalize URL resolution by index:
```js
const aud = $("aud");
const audB = $("aud-b");
let audLastErr = "";
// Soundtrack URL for ring scale `index` (wrapped), ring `audio` field then fallback.
function scaleAudioUrl(index) {
if (!ring || !ring.scales.length) return null;
const s = ring.scales[HEFScrub.wrapIndex(index, ring.scales.length)];
const a = s.audio || SCALE_AUDIO_FALLBACK[s.id];
return a ? "/media/audio/" + a : null;
}
// Start (once) and set the volume of one element to a given url. Safari needs the
// first play() inside a user gesture; later programmatic plays reuse the element.
function ensurePlaying(el, url, vol) {
if (!url) { el.volume = 0; if (!el.paused) el.pause(); return; }
if (el.dataset.url !== url) {
el.dataset.url = url;
el.src = mediaUrl(url);
}
if (el.paused) {
const pr = el.play();
if (pr) pr.then(() => { audLastErr = ""; updateAudioStatus(); })
.catch((e) => { audLastErr = "play BLOCKED: " + ((e && e.name) || e); updateAudioStatus(); });
}
el.volume = HEFScrub.clamp01(vol);
}
// Mid-segment: A = scale `loIndex`, B = scale loIndex+1, gains by frac.
function blendAudio(loIndex, frac) {
if (!$("audio").checked) return;
const g = HEFScrub.crossfadeGains(frac);
ensurePlaying(aud, scaleAudioUrl(loIndex), g.from);
ensurePlaying(audB, scaleAudioUrl(loIndex + 1), g.to);
updateAudioStatus();
}
// At rest on a single altitude: the element already holding it stays at full gain,
// the other fades to 0. Picks whichever element currently carries `index`'s url.
function restAudio(index) {
if (!$("audio").checked) { fadeVolume(aud, 0, FADE_MS, () => aud.pause()); fadeVolume(audB, 0, FADE_MS, () => audB.pause()); return; }
const url = scaleAudioUrl(index);
const onB = audB.dataset.url === url && url;
const active = onB ? audB : aud;
const idle = onB ? aud : audB;
ensurePlaying(active, url, 1);
fadeVolume(idle, 0, FADE_MS);
}
```
Then make the Audio toggle and altitude-rest paths call `restAudio(ringIndex)` instead of the old `applyAudio()` swap. Keep `applyAudio()` as a thin shim that calls `restAudio(ringIndex)` so existing call sites (e.g. the toggle handler at ~1073, `update()`) keep working:
```js
function applyAudio() { restAudio(ringIndex); }
```
- [ ] **Step 4: Run unit suite + the focused E2E**
Run: `node --test simulator/unit/scrub.test.js`
Expected: PASS.
Run: `cd simulator/e2e && npx playwright test -g "two audio elements"` (with the dev server running)
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add simulator/static/index.html simulator/static/app.js simulator/e2e/tests/altitude-lock.spec.ts
git commit -m "feat(sim): two-element audio crossfade keyed by scrub fraction"
```
---
### Task 3: Scrub engine — drag drives `pos`, seeks the morph, commits/locks/re-rolls
**Files:**
- Modify: `simulator/static/app.js` (`onDialDown/Move/Up` ~766795; `advance` ~640679; `playTransition` ~595636; needle/render ~739751)
- Test: `simulator/unit/scrub.test.js` (pure parts covered; engine glue is DOM, covered by E2E in Task 5)
**Interfaces:**
- Consumes: `HEFScrub.*`, `morphByPair`, `scale.pool` (client ring), `mediaUrl`, `setNeedle`, `dialStep`, `ensureClipMedia`/`update` (base-clip loop), `blendAudio`/`restAudio`.
- Produces:
- `pos` (module-level float; rest value = `ringIndex`).
- `pickPoolClip(index: number): string` — random `clip_id` from ring scale `index`'s pool (wrapped).
- `rebuildSegment(lo: number, enteredFrom: number): void` — sets `activeSeg = { lo, clipLo, clipHi, file }`; the side equal to the just-rested altitude takes `activeClipId`, the other is a fresh `pickPoolClip` (re-roll); `file = morphByPair["<clipLo>→<clipHi>"]` (descend canonical).
- `setPos(next: number, opts?: { commit?: boolean }): void` — clamps thrash via one seek per rAF; seeks `vid.currentTime = fracToTime(frac, dur)`; `setNeedle(next * dialStep())`; `blendAudio(lo, frac)`; processes `integerCrossings(pos, next)` to commit/lock/re-roll; updates `pos`.
- `commitTo(index: number): void``ringIndex = wrapIndex(index,n)`, `activeClipId = clip at that index for the active segment`, settle base loop + `restAudio` when frac becomes 0.
- [ ] **Step 1: Write the failing E2E (drag partway tracks currentTime + gains)**
Add to the E2E spec (full version in Task 5; this drives the engine):
```ts
test("dragging the dial scrubs morph currentTime and audio gains", async ({ page }) => {
await boot(page);
// Drag the dial ~40% of one detent (clockwise = descend).
const box = await page.locator("#dial").boundingBox();
const cx = box!.x + box!.width / 2, cy = box!.y + box!.height / 2;
await page.mouse.move(cx, cy - 30);
await page.mouse.down();
await page.mouse.move(cx + 18, cy - 24, { steps: 8 }); // partial turn
const state = await page.evaluate(() => ({
t: (document.querySelector("video") as HTMLVideoElement).currentTime,
ga: (document.querySelector("#aud") as HTMLAudioElement).volume,
gb: (document.querySelector("#aud-b") as HTMLAudioElement).volume,
}));
expect(state.t).toBeGreaterThan(0); // morph scrubbed off frame 0
expect(state.gb).toBeGreaterThan(0); // next scale audio fading in
expect(state.ga).toBeLessThan(1); // current scale fading out
await page.mouse.up();
});
```
Run: `cd simulator/e2e && npx playwright test -g "scrubs morph currentTime"`
Expected: FAIL (engine not built yet).
- [ ] **Step 2: Implement the scrub engine in `app.js`**
Add the position state + segment + setPos near the dial logic, and rewrite `onDialMove`/`onDialUp` to drive it live:
```js
let pos = 0; // continuous knob position; rest value == ringIndex
let activeSeg = null; // { lo, clipLo, clipHi, file }
let seekPending = false; // throttle: one currentTime seek per rAF
function pickPoolClip(index) {
const n = ring.scales.length;
const s = ring.scales[HEFScrub.wrapIndex(index, n)];
const pool = (s.pool && s.pool.length) ? s.pool : [{ clip_id: s.clip_id }];
return pool[Math.floor(Math.random() * pool.length)].clip_id;
}
// Build (or re-roll) the segment [lo, lo+1]. The side equal to the rested altitude
// keeps the locked clip; the other end is a fresh random pick (re-roll).
function rebuildSegment(lo, enteredFrom) {
const n = ring.scales.length;
const loId = (enteredFrom === HEFScrub.wrapIndex(lo, n)) ? activeClipId : pickPoolClip(lo);
const hiId = (enteredFrom === HEFScrub.wrapIndex(lo + 1, n)) ? activeClipId : pickPoolClip(lo + 1);
const file = morphByPair[`${loId}${hiId}`] || null;
activeSeg = { lo, clipLo: loId, clipHi: hiId, file };
if (file) {
overlay.style.opacity = "0"; affectLayer.style.opacity = "0"; tint.style.opacity = "0";
vid.style.filter = "none"; vid.loop = false; vid.style.opacity = "1";
if (vid.dataset.morph !== file) { vid.dataset.morph = file; vid.src = mediaUrl(file); vid.pause(); }
(window.__hefMorphs || (window.__hefMorphs = [])).push(file);
}
}
function setPos(next) {
if (!ring || ring.scales.length < 2) return;
const n = ring.scales.length;
// Commit every integer crossed between the old and new position.
for (const c of HEFScrub.integerCrossings(pos, next)) {
ringIndex = HEFScrub.wrapIndex(c.index, n);
// The clip at the crossed integer is whichever segment end matches it.
if (activeSeg) activeClipId = (HEFScrub.wrapIndex(activeSeg.lo, n) === ringIndex) ? activeSeg.clipLo : activeSeg.clipHi;
activeSeg = null; // force a fresh-approach re-roll for the next segment
}
pos = next;
const { lo, frac } = HEFScrub.segmentOf(pos);
if (frac === 0) { // settled on an altitude
currentClipId = null; // force ensureClipMedia to reload + loop the locked clip
update();
restAudio(ringIndex);
setNeedle(ringIndex * dialStep());
return;
}
if (!activeSeg || activeSeg.lo !== lo) rebuildSegment(lo, ringIndex);
setNeedle(pos * dialStep());
blendAudio(lo, frac);
if (activeSeg.file && !seekPending) { // throttle seeks to one per frame
seekPending = true;
requestAnimationFrame(() => {
seekPending = false;
if (vid.dataset.morph === activeSeg.file) vid.currentTime = HEFScrub.fracToTime(frac, vid.duration);
});
}
}
```
Rewrite the drag handlers to drive `pos` live (replacing the no-live-follow `onDialMove` and the commit-on-release `onDialUp`):
```js
function onDialDown(e) {
if (!ring || ring.scales.length < 2) return;
e.preventDefault();
dialDrag = { lastAng: dialAngle(e), accum: 0, moved: 0, target: e.target, restPos: ringIndex };
}
function onDialMove(e) {
if (!dialDrag) return;
const a = dialAngle(e);
dialDrag.accum += angDelta(a, dialDrag.lastAng);
dialDrag.lastAng = a;
dialDrag.moved += Math.abs(angDelta(a, dialDrag.lastAng));
setPos(HEFScrub.accumToPos(dialDrag.restPos, dialDrag.accum, dialStep())); // LIVE scrub
}
function onDialUp(e) {
if (!dialDrag) return;
const { moved, target } = dialDrag;
dialDrag = null;
if (moved < 6) { // a tap, not a turn
if (target && target.classList && target.classList.contains("dial-label")) jumpToScale(+target.getAttribute("data-index"));
return;
}
// No auto-complete: hold wherever the knob stopped (continuous-encoder model).
}
```
Keep `renderDial()` reading `pos` for the needle when mid-segment, else `ringIndex`. Retire `advance()`'s body (or leave it unused) — wheel/tap use the auto-scrub in Task 4.
- [ ] **Step 3: Run the focused E2E**
Run: `cd simulator/e2e && npx playwright test -g "scrubs morph currentTime"` (dev server running)
Expected: PASS.
- [ ] **Step 4: Run the unit suite (no regressions)**
Run: `node --test simulator/unit/scrub.test.js`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add simulator/static/app.js simulator/e2e/tests/altitude-lock.spec.ts
git commit -m "feat(sim): scrub engine — drag drives pos, seeks morph, commits+locks+re-rolls"
```
---
### Task 4: Wheel + tap become auto-scrubs, then lock
**Files:**
- Modify: `simulator/static/app.js` (`onWheel` ~681691; `jumpToScale` ~798805)
**Interfaces:**
- Consumes: `setPos`, `ringIndex`, `pos`, `HEFScrub.wrapIndex`.
- Produces:
- `autoScrub(targetPos: number, ms = 600): void` — rAF-animates `pos` from its current value to `targetPos` (calling `setPos` each frame), landing exactly on the integer target, then locks (frac 0 settles in `setPos`).
- [ ] **Step 1: Write the failing E2E (wheel auto-scrubs one altitude then locks)**
Add to the E2E spec:
```ts
test("wheel auto-scrubs one altitude and locks", async ({ page }) => {
await boot(page);
const start = (await page.locator("#scale-name").textContent())!;
await page.locator("#stage").dispatchEvent("wheel", { deltaY: 60 });
await page.waitForFunction((s) => document.querySelector("#scale-name")?.textContent !== s, start, { timeout: 20000 });
const landed = (await page.locator("#scale-name").textContent())!;
expect(landed).toContain("orbit");
// Mid-flight currentTime moved; after landing the morph is settled (frac 0).
await page.waitForTimeout(1500);
expect(await page.locator("#scale-name").textContent()).toBe(landed); // locked
});
```
Run: `cd simulator/e2e && npx playwright test -g "wheel auto-scrubs"`
Expected: FAIL.
- [ ] **Step 2: Implement `autoScrub` and rewire wheel + tap**
```js
let autoRaf = 0;
function autoScrub(targetPos, ms = 600) {
if (autoRaf) cancelAnimationFrame(autoRaf);
const fromPos = pos, dist = targetPos - fromPos;
if (!dist) { setPos(targetPos); return; }
let startTs = null;
const tick = (ts) => {
if (startTs === null) startTs = ts;
const k = Math.min(1, (ts - startTs) / ms);
setPos(fromPos + dist * k);
if (k < 1) autoRaf = requestAnimationFrame(tick);
else { autoRaf = 0; setPos(targetPos); } // land exactly + lock (frac 0)
};
autoRaf = requestAnimationFrame(tick);
}
let wheelAccum = 0, wheelTimer = null;
function onWheel(e) {
e.preventDefault();
wheelAccum += e.deltaY;
clearTimeout(wheelTimer);
wheelTimer = setTimeout(() => {
const detents = Math.trunc(wheelAccum / 60) || (wheelAccum > 0 ? 1 : -1);
wheelAccum = 0;
if (detents) autoScrub(ringIndex + detents);
}, 90);
}
function jumpToScale(idx) {
if (!ring) return;
const n = ring.scales.length;
let d = (idx - ringIndex) % n;
if (d > n / 2) d -= n;
if (d < -n / 2) d += n;
if (d) autoScrub(ringIndex + d);
}
```
- [ ] **Step 3: Run the focused E2E**
Run: `cd simulator/e2e && npx playwright test -g "wheel auto-scrubs"` (dev server running)
Expected: PASS.
- [ ] **Step 4: Commit**
```bash
git add simulator/static/app.js
git commit -m "feat(sim): wheel + label-tap auto-scrub one/path of altitudes, then lock"
```
---
### Task 5: Full E2E tier for scrub-driven transitions
**Files:**
- Modify: `simulator/e2e/tests/altitude-lock.spec.ts`
**Interfaces:**
- Consumes: the running simulator, `window.__hefMorphs`, `#scale-name`, `#dial`, `video`, `#aud`/`#aud-b`.
- [ ] **Step 1: Replace the `.rev`-asserting reverse test with a scrub-reverse test**
The old "zoom back out plays a reverse morph" test asserted a `.rev.mp4` file. Under the scrub model, turn-back seeks the **same** segment morph backward. Replace it:
```ts
test("turn-back scrubs the same morph in reverse and re-locks the start clip", async ({ page }) => {
await boot(page);
const box = await page.locator("#dial").boundingBox();
const cx = box!.x + box!.width / 2, cy = box!.y + box!.height / 2;
const read = () => page.evaluate(() => ({
t: (document.querySelector("video") as HTMLVideoElement).currentTime,
ga: (document.querySelector("#aud") as HTMLAudioElement).volume,
gb: (document.querySelector("#aud-b") as HTMLAudioElement).volume,
name: document.querySelector("#scale-name")?.textContent,
}));
await page.mouse.move(cx, cy - 30);
await page.mouse.down();
await page.mouse.move(cx + 22, cy - 20, { steps: 10 }); // descend partway
const fwd = await read();
await page.mouse.move(cx + 4, cy - 30, { steps: 10 }); // turn back toward start
const back = await read();
await page.mouse.up();
expect(back.t).toBeLessThan(fwd.t); // morph seeking backward
expect(back.gb).toBeLessThan(fwd.gb); // next-scale audio receding
expect(back.name).toContain("cosmos"); // re-locked the altitude we left
});
```
- [ ] **Step 2: Add the full-turn commit+lock test**
```ts
test("a full turn commits and locks the destination clip", async ({ page }) => {
await boot(page);
const start = (await page.locator("#scale-name").textContent())!;
const box = await page.locator("#dial").boundingBox();
const cx = box!.x + box!.width / 2, cy = box!.y + box!.height / 2;
await page.mouse.move(cx, cy - 30);
await page.mouse.down();
await page.mouse.move(cx + 30, cy, { steps: 6 });
await page.mouse.move(cx, cy + 30, { steps: 6 }); // ~full detent clockwise
await page.mouse.up();
await page.waitForFunction((s) => document.querySelector("#scale-name")?.textContent !== s, start, { timeout: 20000 });
const landed = (await page.locator("#scale-name").textContent())!;
expect(landed).toContain("orbit");
const played = await page.evaluate(() => (window as any).__hefMorphs || []);
expect(played[0]).toMatch(/^transitions\/cosmos.*\.mp4$/);
await page.waitForTimeout(2000);
expect(await page.locator("#scale-name").textContent()).toBe(landed); // locked
});
```
- [ ] **Step 3: Keep the wheel + drag tests from Tasks 24; run the whole tier**
Run: `cd simulator/e2e && npx playwright test`
Expected: PASS — all scrub tests green (drag-scrub, turn-back, full-turn, wheel, two-audio-elements).
- [ ] **Step 4: Commit**
```bash
git add simulator/e2e/tests/altitude-lock.spec.ts
git commit -m "test(e2e): scrub tier asserts currentTime + audio gains track the dial, reverse, commit-lock, wheel"
```
---
### Task 6: Phase-1 localhost verification
**Files:** none (verification task).
- [ ] **Step 1: Run the unit suite**
Run: `node --test simulator/unit/`
Expected: PASS.
- [ ] **Step 2: Run the full Python test suite (no regressions)**
Run: `python -m pytest -q`
Expected: PASS (existing tally; no new failures).
- [ ] **Step 3: Boot the simulator and run E2E against it**
Run (server): start the simulator dev server per `simulator/` README on its configured port with the venv python.
Run (tests): `cd simulator/e2e && npx playwright test`
Expected: all PASS.
- [ ] **Step 4: Eyeball the feel**
Confirm by hand: slow drag eases the morph slowly; fast drag races it; stopping mid-drag holds a blended frame with mixed audio; turning back reverses; the wheel auto-scrubs one altitude and locks; a label tap travels the shortest way. Note any steppiness (expected on sparse-GOP morphs — Phase 2 fixes it).
- [ ] **Step 5: Checkpoint the transcript**
Re-publish the in-progress transcript (no commit needed beyond what tasks already pushed).
---
## Phase 2 — All-intra morph re-bake for smooth seeking
### Task 7: Re-bake the 154 morphs all-intra (dense keyframes)
**Files:**
- Modify: `simulator/build_pool_manifest.py` (`_make_transition` ~402417, `_make_reverse` ~420429)
- Test: `tests/test_build_pool_manifest.py`
**Interfaces:**
- Produces: every baked morph encoded all-intra (each frame a keyframe) so arbitrary-frame seeking is smooth.
- [ ] **Step 1: Write the failing arg-builder test**
`_make_transition`/`_make_reverse` currently call `subprocess.run` directly, so factor the ffmpeg command into a pure builder to test it. Add to `simulator/build_pool_manifest.py`:
```python
ALL_INTRA = ["-c:v", "libx264", "-g", "1", "-keyint_min", "1", "-sc_threshold", "0", "-pix_fmt", "yuv420p"]
```
Add a pure helper and a test. In `tests/test_build_pool_manifest.py`:
```python
from simulator import build_pool_manifest as B
def test_transition_cmd_is_all_intra():
cmd = B.transition_cmd("ffmpeg", "/a/base.mp4", "/b/base.mp4", "/out/x__y.mp4")
assert "-g" in cmd and cmd[cmd.index("-g") + 1] == "1"
assert "-keyint_min" in cmd and cmd[cmd.index("-keyint_min") + 1] == "1"
assert "-sc_threshold" in cmd and cmd[cmd.index("-sc_threshold") + 1] == "0"
def test_reverse_cmd_is_all_intra():
cmd = B.reverse_cmd("ffmpeg", "/out/x__y.mp4", "/out/x__y.rev.mp4")
assert "-g" in cmd and cmd[cmd.index("-g") + 1] == "1"
assert "reverse" in " ".join(cmd)
```
Run: `python -m pytest tests/test_build_pool_manifest.py -q`
Expected: FAIL — `transition_cmd`/`reverse_cmd` not defined.
- [ ] **Step 2: Extract `transition_cmd`/`reverse_cmd` and apply all-intra flags**
```python
def transition_cmd(ff, a, b, out):
norm = "trim=0:3,setpts=PTS-STARTPTS,scale=1280:720,fps=25,setsar=1,format=yuv420p"
return [
ff, "-y", "-i", str(a), "-i", str(b), "-filter_complex",
f"[0:v]{norm}[a];[1:v]{norm}[b];"
"[a][b]xfade=transition=zoomin:duration=1.5:offset=0.75,format=yuv420p[v]",
"-map", "[v]", "-an", *ALL_INTRA, str(out),
]
def reverse_cmd(ff, forward, out):
return [ff, "-y", "-i", str(forward), "-vf", "reverse", "-an", *ALL_INTRA, str(out)]
```
Then make `_make_transition`/`_make_reverse` call these builders inside `subprocess.run(..., check=True, capture_output=True)`.
- [ ] **Step 3: Run the arg-builder test**
Run: `python -m pytest tests/test_build_pool_manifest.py -q`
Expected: PASS.
- [ ] **Step 4: Regenerate the morph media**
Run: `python simulator/build_pool_manifest.py` (or its documented `generate_media` entrypoint) to re-bake all 154 morphs all-intra. Confirm files regrew (sparse → dense keyframes; expect ~0.50.9 GB total, each clip well under the 25 MB LFS/proxy ceiling noted in memory).
- [ ] **Step 5: Commit the re-baked media via git-LFS**
```bash
git add simulator/sample_media/transitions
git commit -m "feat(pipeline): re-bake 154 morphs all-intra for smooth scrub seeking (LFS)"
```
Confirm they are LFS pointers: `git lfs ls-files | head`.
---
### Task 8: Phase-2 verification — smooth scrub on dense keyframes
**Files:** none (verification task).
- [ ] **Step 1: Re-run the unit + E2E suites against re-baked media**
Run: `node --test simulator/unit/` → PASS.
Run (server up): `cd simulator/e2e && npx playwright test` → PASS.
- [ ] **Step 2: Eyeball smoothness**
Confirm the scrub now seeks smoothly frame-to-frame (no stepping) on a slow drag across a full segment.
- [ ] **Step 3: Push the branch**
```bash
git push -u origin session-0026
```
---
## Ship via §9 (execution, not a code task)
After Phase 1 + Phase 2 are green on localhost (unit + E2E), ship through the mandatory pipeline:
1. **localhost + E2E green** (Tasks 6 + 8).
2. **PPE deploy + E2E green** (flotilla; provision PPE if it doesn't exist yet). Stamp the `release/<YYYY-MM-DDTHH-MM>` tag.
3. **prod is human-gated** — the operator promotes the validated tag; the session stops at PPE-green.
---
## Self-Review
**Spec coverage:**
- Core model (continuous `pos`, frac drives currentTime + audio) → Tasks 1, 3, 2.
- In-between hold / no auto-complete → Task 3 (`onDialUp` holds).
- Turn-back reversible / re-lock start clip → Tasks 3, 5.
- Destination randomness fixed-per-gesture, re-roll on fresh approach → Task 3 (`rebuildSegment`, re-roll on `activeSeg` reset at crossing).
- Scroll wheel auto-scrub then lock; tap shortest way → Task 4.
- Lock-per-altitude preserved → Task 3 (`commit` sets `activeClipId`; frac 0 settles base loop).
- Audio crossfade two adjacent soundtracks by frac → Task 2.
- All-intra re-bake (154 morphs, LFS) → Task 7.
- Unit tests (position→segment, frac→time, commit on crossing, re-roll, reverse) → Task 1 (pure) + E2E for glue.
- Playwright E2E (currentTime + gains track angle, reverse, full-turn commit-lock, wheel) → Task 5.
- Phasing (interaction first, re-bake second) → Phase 1 then Phase 2.
**Placeholder scan:** none — every code step carries real code and exact commands.
**Type consistency:** `setPos`, `rebuildSegment`, `activeSeg {lo,clipLo,clipHi,file}`, `pickPoolClip`, `blendAudio(loIndex,frac)`, `restAudio(index)`, `scaleAudioUrl(index)`, `autoScrub(targetPos,ms)`, `HEFScrub.*` names are used consistently across Tasks 15. `transition_cmd`/`reverse_cmd`/`ALL_INTRA` consistent in Tasks 7.
**Open interpretation (logged as deferred decision):** the canonical-segment-file choice (always the descend/forward morph, scrubbed bidirectionally; `.rev` unused by scrub) reinterprets the spec's literal `morphByPair["<from>→<to>"]` directed lookup in favor of its "scrub the same morph in reverse" + full-reversibility requirements. Surfaced for operator awareness at finalize.
@@ -0,0 +1,83 @@
# Session 0023.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-26T09-05 (PST)
> End: 2026-06-26T09-20 (PST)
> Type: executing-plans (operator-driven debugging — "still can't hear" / "video didn't come on")
> Posture: yolo
> Claude-Session: e0cc053a-41fa-416d-a4f5-493f14984e75
> Status: FINALIZED
## Launch prompt (the thread)
> "I still can't hear" → (after diagnostics) operator screenshot: `audio: on, PAUSED
> (readyState 0)` and "the audio check works" with `ring=server · url=NONE` → "the
> video didn't come on when I turned it on" → after the fix: **"yes, both"** (video
> and audio confirmed working).
## Pre-state
- Sessions 00200022 had shipped the audio/video feature + three "Safari autoplay"
fixes that were verified only in headless browsers and **did not work** for the
operator. Audio was silent; this session set out to find the real cause.
## Session arc — the debugging
1. **Stop guessing; instrument.** Three blind fixes had failed because headless
Playwright (Chromium *and* WebKit) relaxes autoplay, so the bug was never
reproducible. Added a **live `#audio-status` readout** (surfacing the previously
*swallowed* `play()` errors + `aud.error`) and a **native `<audio controls>` test
player** that plays the file with zero app code involved. Confirmed the server
already honors HTTP Range (`206`) + `audio/mpeg` — not a serving problem.
2. **The decisive readout.** Operator reported: the **native player works** (file +
codec + browser audio all fine), but the toggle showed
**`ring=server · url=NONE`** with `readyState 0` and no media error — meaning my
code never set a source because `soundtrackUrl()` returned null.
3. **Root cause = a STALE uvicorn server.** The operator's Python process predated the
0020 audio/video work, so it (a) returns `/api/ring` without the per-scale `audio`
field, and (b) requires the old 7-way `content` — it **422s** the new
`{visual,audio}` payload and returns `{content:{video}}` instead of
`{render:{video:{shown}}}`. So audio had no url AND the Video toggle's POST failed
(→ "video didn't come on"). Reloading never fixes it (only refreshes static assets,
not the Python process).
4. **Fix = make the client resilient to a stale server** (so no restart is needed):
- `soundtrackUrl()` falls back to a scale-id→file map (`SCALE_AUDIO_FALLBACK`,
mirrors `build_pool_manifest.SCALE_AUDIO`).
- `controls()` also sends a back-compat `content` field (current server ignores it;
an old one ignores visual/audio).
- `update()` reads "video shown" from **either** response shape.
Verified end-to-end against a **simulated fully-stale server** (routed old
`/api/alteration` shape + stripped ring audio) in Chromium **and** WebKit: Video on
→ video shows + soundtrack plays, zero errors. **Operator confirmed "yes, both".**
5. **Cleanup.** Tucked the audio diagnostics (status readout + native player) under the
**Dev Mode** panel — kept for future debugging, out of the installation UI.
## Deferred decisions
_No low-confidence calls. One judgment worth noting: I added permanent client
back-compat for a pre-release internal API to unblock the operator without a server
restart. The clean long-term fix is restarting the dev server; the shim is harmless and
removes the dependency. Flagging in case a future cleanup wants to drop it once servers
are reliably current._
## Cut state
- `main` @ `6427ab4`, clean, pushed. **292 tests + 7 Playwright E2E pass** (2 skipped).
- Audio + Video experience **works and is operator-confirmed** on the real device.
- New E2E regressions: stale-ring soundtrack fallback, audio-then-video ordering.
- The operator's actual server is still stale — recommended (not required) they restart
it so the API is current; the client no longer depends on it.
## Operator plate / lesson
- **The big lesson (recorded in memory):** headless Playwright relaxes BOTH autoplay and
GPU compositing, so it cannot reproduce real-browser/real-server issues. When a fix
"passes headless" but the operator still sees the bug, **instrument with on-screen
readouts + a native control** to get their real data — don't ship another blind guess.
Three rounds were lost to this before instrumenting.
## Next /goal
```
/goal source/compose the deferred per-altitude music layer (the reserved Audio dial position) per docs/audio-candidate-pool.md — the Audio+Video experience is working & operator-confirmed; OR re-enable white-noise as a 3rd Audio position if wanted
```
@@ -0,0 +1,94 @@
# Session 0024.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-26T21-12 (PST)
> Type: planning-and-executing
> Posture: yolo
> Claude-Session: e0aea412-c853-4236-9536-cfd85cbc4975
> End: 2026-06-27T15-23 (PST)
> Status: **FINALIZED.**
>
> Started as writing-plans; transitioned to planning-and-executing (yolo) when the
> operator approved the plan and asked to execute in-session.
## Next /goal
Build **scrub-driven altitude transitions** per the approved design at
`docs/superpowers/specs/2026-06-27-scrub-driven-altitude-transitions-design.md`:
knob position continuously scrubs the morph `currentTime` + crossfades adjacent
scale audio (hold-partway, reversible, re-roll on fresh approach, wheel auto-scrubs
one altitude). Phase 1: interaction on current morphs. Phase 2: re-bake all 154
morphs all-intra for smooth seeking + re-push LFS. Writing-plans → executing-plans
in a fresh session. (Operator was handed a ready-to-paste prompt.)
## Launch prompt
> Fix altitude/video transition: video shouldn't switch when changing altitudes;
> good zoom transition for every video combination; video must not change after
> zooming to a new altitude.
Operator refinements during orientation:
- **Lock per altitude until the altitude is changed** — on arrival, pick the
clip once and keep it; no secondary swap, no re-roll while parked.
- **Choose the destination clip BEFORE transitioning, then play the appropriate
morph footage.** We need morph footage for ALL transitions between ALL videos
in adjacent altitudes (both directions).
## Plan
Anchor: leaf change extending player `design.md §3` (scale navigation & zoom
transitions) + content-pipeline. **Scope reality:** clip-pair-keyed morph system
**154 morph clips** (77 adjacent video pairs × 2 directions) up from 5 per-edge
morphs; touches the bake pipeline (`build_pool_manifest.py`), the ring contract
(`player/ring.py`), manifest/transition schema (`simulator/clips.py`), the
`/api/ring/advance` endpoint (`simulator/app.py`), the frontend `advance()`/lock
(`simulator/static/app.js`), and a new Playwright E2E tier. Writing-plans session:
author ONE plan, stop at the review gate.
## Deferred decisions
- The dial-needle **sweep** (synced to the morph) is verified working in Chromium +
WebKit both directions, but the operator's real Safari still showed it "not
working" — root cause likely stale cached `app.js` (Safari refresh confusion),
never fully confirmed. Moot: the **scrub-driven redesign supersedes** the dial
interaction entirely. The sweep commit merged to main as an interim improvement.
- **git-LFS front-proxy upload limit** (HTTP 413 on the 68 MB base): resolved by
transcoding rather than raising the server limit — the operator couldn't locate
the front-proxy VM (`35.238.203.16`, nginx, not in any gcloud project I could
reach) this session. Raising that limit is a deferred infra task.
## Arc
**Pre-state.** Altitude dial (PR #18, s0016) navigated a 5-scale ring with **5
per-edge** morphs (scale-primary→primary). Landing picked a random pool member, but
`advance()` played the edge morph (ending on the primary) then did a jarring
fade-to-black **secondary swap** to the random member. Operator: "video switches when
changing altitudes; must not change after zooming."
**What we built (writing-plans → executing, planning-and-executing).**
1. Orientation + clarification: lock-per-altitude; **pick destination first then play
the matching morph**; morphs for **all adjacent video pairs both directions** (154);
multi-detent **chains** through each altitude.
2. Wrote + reviewed the plan (`docs/superpowers/plans/2026-06-27-altitude-clip-pair-morph-lock.md`);
operator approved + asked to execute in-session → switched to yolo.
3. Executed TDD: per-clip-pair manifest + member-pair baking; `Transition`/`morph_for`/
`resolve_move` in `player/ring.py`; `resolved_move_to_dict`; `/api/ring/advance`
`from_clip_id`; frontend pick→morph→lock (no secondary swap); regenerated 154
morphs. pytest 302 passed; new **Playwright E2E tier** 2 passed.
4. **Media graduation:** operator flagged that "sample" media is actually the
experience — committed bases + audio + 154 morphs (~526 MB) via **git-LFS → Gitea
LFS store**. Hit HTTP 413 on the 68 MB `coast_birdrock` base at the front proxy;
**transcoded it 68→22 MB** (re-baked its 16 morphs) to ship. Pushed; merged to main.
5. Polish loop on the **dial/transition timing** (sweep → removed live drag pre-move),
then operator **reframed**: knob position should *drive* the transition (scrub),
not the dial follow the video. Brainstormed + wrote the **scrub-driven design spec**;
handed off a prompt to build it in a fresh session.
**Recurring lesson re-confirmed:** the operator's mid-session "no transitions" was a
**stale uvicorn** (served new `app.js` from disk but old in-memory API returning the
deleted old-named morph files → 404). Fix = restart uvicorn with the venv python.
**Cut state.** `main` (528fb5c) has the shipped feature, LFS media, E2E tier, both the
plan and the scrub design spec. No content repo → plan archived in-repo under `docs/`
(gap flagged). Simulator is localhost-only (no PPE/prod) → no §9 deploy stage; verified
green locally. Next increment (scrub) deferred to a fresh session.
@@ -0,0 +1,96 @@
# Session 0025.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-27T06-05 (PST)
> End: 2026-06-27T07-05 (PST)
> Type: capture
> Posture: careful
> Claude-Session: d6ed1203-92b6-471f-80a4-4c773b645ca1
> Status: **FINALIZED**
## Launch prompt
```
capture this as an issue. We'll implement the network feature later but want it as an Epic, then the remote simulator (e.g. ipad) as a feature with this design
```
## Pre-state
- Immediately followed session 0019 (brainstorming), which merged the
**networked control surface** design SPEC to `main`
(`docs/superpowers/specs/2026-06-26-networked-control-surface-design.md`).
- Operator wants the work captured as backlog (deferred — "implement later"): an
**Epic** for the whole networked feature, and a **Feature** for the iPad remote
twin anchored to that spec.
- Single-repo app: tracker = `benstull/human-experience-filter-art` on
git.benstull.org; no separate content repo (`CONTENT_REMOTE` empty).
- Checkout shared with concurrent sessions (0024 live on its own branch).
## Arc
1. **Routing.** Classified the request as **capture**; invoked `wgl-capture`,
claimed tracked-lite session **0025** (capture / careful).
2. **Target resolve.** Single-repo app → drafted into this repo's working-tree
`issues/` (ephemeral, INV-8); tracker is this repo on git.benstull.org.
3. **Two asks identified** (per handbook §4.3 sizing): an **Epic** (multi-Feature
umbrella — the networked control surface) and a **Feature** (single increment —
the iPad web remote twin, the spec's v1 scope). Both type-fit cleanly.
4. **Drafted both** in the content repo's `issues/` at business altitude (Epic +
Feature business sections free of implementation specifics; the design spec
referenced as orientation). Presented for review; operator approved filing both
at **P2** via `AskUserQuestion`.
5. **Filing.** Label-ensure first hit a **401** — the default issue-token service
resolved to the wrong host. Fixed by pointing `WGL_CAPTURE_TOKEN_SERVICE` at the
host-specific Keychain entry `wgl-gitea-issues-readwrite-token-git.benstull.org`.
Then ensured capture labels (9 created) and filed:
- **Epic #26** — "Networked control surface — drive the experience from a
separate control device" (`type/epic`, P2).
- **Feature #27** — "Remote controller simulator — drive the experience from an
iPad web twin over wifi" (`type/feature`, P2; parent #26).
(Typecheck advisory labels 401'd — fail-open, skipped; no effect on the issues.)
6. **Cleanup.** Deleted both filed drafts (Gitea = system of record, INV-8);
removed the now-empty `issues/` dir; working tree carries no drafts.
## Cut state
- Filed: **Epic #26**, **Feature #27** (parent #26), both `priority/P2`, deferred.
- No repo commits this session (capture is tracked-lite; drafts ephemeral). The
dirty tree on branch `session-0024` is the **concurrent session's** work
(`build_pool_manifest.py` + a test) — left untouched.
- Memory updated: `networked-control-surface-spec.md` records the filed issues +
a deferred "when resumed" pointer; `MEMORY.md` index refreshed.
## Deferred decisions
_None — no autonomous low-confidence calls. Type assignment and P2 priority were
operator-approved in-session._
## Type-fit tally
`type-fit: judged 2, mismatch 0, overridden 0` (judged against §4.3 informally;
the typecheck-label markers could not be stamped — advisory token 401, fail-open).
## Operator plate
- The networked-control work is now backlog: **#26** (Epic) / **#27** (Feature),
deferred per "implement later".
- Token note for future capture on this host: use
`WGL_CAPTURE_TOKEN_SERVICE=wgl-gitea-issues-readwrite-token-git.benstull.org`
(the unscoped default resolves to the wrong host and 401s).
## Next /goal
No new immediate next from this capture — the work is parked on the tracker
(#26/#27, deferred). When the operator resumes hardware:
```
/goal Write the implementation plan for the remote controller simulator, per docs/superpowers/specs/2026-06-26-networked-control-surface-design.md (Feature #27)
```
The active project frontier remains the per-altitude music layer (see the
`audio-soundtrack-sourcing` memory), not this deferred thread.
@@ -0,0 +1,130 @@
# Session 0026.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-27T15-26 (PST)
> Type: planning-and-executing
> Posture: yolo
> Claude-Session: c6b9bebd-f94a-4fc0-982d-0a9dc00b3148
> Checkout: /Users/benstull/git/benstull.org/benstull/human-experience-filter-art
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
>
> This file reserves session ID 0026 for human-experience-filter-art. The driver replaces this
> body with the full transcript and renames the file to its final
> SESSION-0026.0-TRANSCRIPT-2026-06-27T15-26--<end>.md form at session end.
## Launch prompt
Build scrub-driven altitude transitions in the simulator, per the approved design at
`docs/superpowers/specs/2026-06-27-scrub-driven-altitude-transitions-design.md`. Write the
implementation plan from that spec, review it, then execute it. Make the Altitude knob
position continuously drive the transition: dragging the dial scrubs the morph video's
currentTime and crossfades the two adjacent scale soundtracks by knob angle; hold-partway
(no auto-complete), fully reversible, re-roll the destination clip on each fresh approach,
scroll-wheel auto-scrubs one altitude then locks, lock-per-altitude preserved. Phase 1:
build against current morphs. Phase 2: re-bake all 154 morphs all-intra for smooth seeking,
re-push via git-LFS. Unit tests for pure scrub logic + extend Playwright E2E.
## Plan
> Anchor: design docs/superpowers/specs/2026-06-27-scrub-driven-altitude-transitions-design.md (R2a — ELIGIBLE)
> Session type: planning-and-executing (fused) — write the plan, then execute it, ship via §9.
Scrub-driven altitude transitions. Continuous `pos` (float) drives morph `currentTime` +
two-soundtrack audio crossfade by `frac(pos)`; integer crossing commits + locks + re-rolls.
Phase 1 = interaction against current morphs; Phase 2 = all-intra re-bake (154 morphs, LFS).
Pure scrub logic kept separable + unit-tested; Playwright E2E extended to assert currentTime
+ audio gains track the dial and reverse on turn-back.
### Progress (checkpoint @ Phase-1 done)
- Plan: `docs/superpowers/plans/2026-06-27-scrub-driven-altitude-transitions.md` (approved "execute as-is").
- **Phase 1 DONE** (branch `session-0026`, 5 commits):
- T1 `simulator/static/scrub.js` (pure UMD math) + `simulator/unit/scrub.test.js` (9 `node --test`).
- T2 two-element audio crossfade (`<audio id=aud-b>`, `blendAudio`/`restAudio`/`ensurePlaying`/`assignElements`).
- T3 scrub engine: drag live-drives `pos`; `setPos` seeks morph currentTime (1 seek/rAF), crossfades audio,
commits+locks+re-rolls on integer crossings; release holds (no auto-complete); client-side `pickPoolClip`.
- T4 wheel + label-tap → `autoScrub` (animate `pos`, land exact, lock); removed dead `advance()`/`playTransition()`.
- T5 E2E scrub tier (6 tests green): drag tracks currentTime+gains, turn-back reverses same file + re-locks start,
detent-crossing commits+locks (via `window.__hefState` seam), wheel auto-scrubs+locks.
- **Verification:** `node --test` 9 green · `pytest` 302 passed / 2 skipped · Playwright 6 green.
- **Phase 2 DONE** (T7T8): `transition_cmd`/`reverse_cmd` builders + `ALL_INTRA` (`-g 1 -keyint_min 1
-sc_threshold 0`), unit-tested; re-baked all 154 morphs all-intra (verified 93/93 I-frames on a sample),
173M→504M, largest clip ~5.8M; committed via git-LFS and pushed (154 LFS objects, 453MB).
- **All 8 plan tasks complete. Branch `session-0026` pushed.** Final verification: `node --test` 9 green ·
Playwright 6 green · `pytest` 302 passed/2 skipped.
- **Remaining:** merge to main + §9. Operator by-eye "feel/smoothness liked?" review is the one acceptance
criterion automated tests can't cover (simulator-first directive) — flagged for operator before/at merge.
- **DECISION (operator): HOLD for eyeball first.** Branch `session-0026` stays UNMERGED; session NOT finalized.
Operator pulls + runs the sim locally to judge the scrub feel/smoothness, then says merge (or requests tweaks).
### Eyeball round 1 — operator found 2 issues, both FIXED (2 more commits)
- **Fix 1 — landing jitter (frontend).** Landing jump-cut the loop back to frame 0. Each morph reproduces the
dst clip's first 3s (`trim=0:3`), ending dst@~3s, so steady-state loop now starts at `LOOP_TAIL_S=3.0` and
loops `[3.0, duration]` (`ensureClipMedia` custom loop; disarmed during morph scrub). Morph's last frame hands
straight off to the loop. E2E asserts a landed clip loops from ~3s, not 0. No re-bake. (8th E2E test added.)
- **Fix 2 — red-snapper text card (content).** `reef_snapper` opened with a black NOAA title card (~0.92.9s).
Re-trimmed `reef_snapper/base.mp4` to drop the first 3.0s (23s→20s, footage clean from 3.0s) and re-baked the
7 reef_snapper morph pairs (+reverses, all-intra) from the clean base; verified snapper morph frames text-free.
- **Verification:** Playwright 7 green · all-intra re-confirmed (93/93 I-frames). Pushed to `session-0026`.
- **D2 (deferred) — reef_snapper tracked test-label.** The trim shifts the footage, so reef_snapper's one
loop-normalized tracked label (`appear 0.0`/`disappear 0.7` + 3-keyframe track) is now slightly misaligned to
the subject. It's demo/test material gated behind both knobs up; left as-is — re-author via `/author.html` if
the operator wants it pixel-accurate.
### Eyeball round 2 — 2 more items, both FIXED (2 commits)
- **Fix 3 — morph→loop load jolt (double-buffer).** A pause/jolt when the morph ended and the next altitude's
loop loaded: swapping `vid.src` to the base + seeking the tail reloaded the ON-SCREEN element (decode+seek
stall). Added a dedicated `#vid-loop` element for the steady-state base loop; during a scrub it is PRELOADED to
the clip we're heading toward (`loadLoop`, paused at the tail frame). The paint shader reads the active source
(`displayVid() = busy ? vid : loopVid`), so landing is an instant source swap — no reload, no seek. E2E asserts
the loop element is armed + playing from ~3s after landing.
- **Fix 4 — audio on/off → 010 level dial.** Replaced the Audio checkbox with a 010 range slider = master gain
(`audioVol = level/10`), scaling rest volume + the two-track scrub crossfade. First Video-on lifts audio to
**3/10** (was full). E2E: range 010 default 0; first video-on → 3.
- **Verification:** Playwright **9 green** · node unit 9 · screenshot-confirmed the dial layout.
- **D3 residual still applies:** double-buffer kills the LOAD pause both directions; the reverse/turn-back FRAME
asymmetry (morph src-start@0 vs loop tail@3s) and the ~loop-wrap seek remain (per-clip loop files would fully
resolve if wanted).
### Eyeball round 3 — audio-off bug (systematic-debugging), FIXED (1 commit)
- **Bug:** audio turned off *sometimes* after landing on a new altitude. **Root cause (code-traced, not guessed):**
`fadeVolume` started a `setInterval` ramp that was **never cancelled**. `restAudio` fades the idle element to 0;
if within FADE_MS (500ms) that same element becomes the newly-landed ACTIVE scale (`assignElements` reuses one
element per soundtrack url), the stale fade-to-0 kept running and dragged the now-active element to silence —
hence "only sometimes." **Fix:** `cancelFade(el)` supersedes any in-flight fade when a new fade starts
(`fadeVolume`) or a direct volume is set (`ensurePlaying`). Added `window.__hefFade` seam + a deterministic E2E
that PROVES it (fails with the fix disabled, passes with it). Playwright **10 green**.
### Eyeball round 4 — transition speed (1 commit)
- **Request:** slow click/auto transitions ~2x and make them the SAME speed between any two altitudes (2 away =
2× as long as 1). `autoScrub` was a fixed 600ms total regardless of distance. Now duration is PROPORTIONAL:
`PER_ALTITUDE_MS=1200` (≈2× the old 600) × |distance|, so a label-click/wheel jump of N altitudes takes
N×1200ms — constant per-step speed. Manual drag unaffected. E2E asserts a 2-step jump ≈ 2× a 1-step. 11 green.
- **Resume:** merge + `wgl-session-finalize` on operator go, OR next eyeball round of tweaks.
## Deferred decisions
_Autonomous-mode low-confidence calls the driver made and would have
liked operator input on. Appended as the session runs; surfaced at
finalize. Empty if none._
- **D1 (plan) — canonical segment morph file.** The spec's component 2 says resolve the
directed morph via `morphByPair["<fromClip>→<toClip>"]`, but its turn-back decision says
"scrubs the SAME morph in reverse" with full reversibility. These are only mutually
consistent if a segment uses ONE file scrubbed bidirectionally. I locked the plan to:
per segment `[lo, lo+1]`, always use the **descend/forward** morph
`morphByPair["<clip@lo>→<clip@lo+1>"]` and drive `currentTime = frac × duration` in both
travel directions; the `.rev` files become unused by the scrub interaction (kept baked
for back-compat). This means the old E2E "zoom-out plays a `.rev`" assertion is replaced
by a "turn-back seeks the same file backward" assertion. Low-confidence on whether the
operator specifically wanted `.rev` retained in the drag path.
- **D2 — reef_snapper tracked test-label misalignment after the text-card trim.** Trimming the snapper base by
3.0s (to remove the title card) shifts the footage, so reef_snapper's one loop-normalized tracked label
(`appear 0.0`/`disappear 0.7` + a 3-keyframe track) no longer follows the same subject. Left as-is: it's
demo/test authoring gated behind both knobs up; re-author via `/author.html` if pixel-accuracy is wanted.
- **D3 — landing-jitter fix approach.** Chose a frontend loop-from-tail (`LOOP_TAIL_S=3.0`, custom
`[3.0, duration]` loop) over re-baking morphs or baking dedicated loop clips — cheap, no media churn,
eye-tweakable. Residual: a one-seek hitch at each loop wrap (~every 1720s) and scrub-START (vs landing) is
not made seamless (a looping clip is at arbitrary phase when a drag begins). If the wrap hitch bothers the
operator, bake per-clip loop files (`base[3:end]`) for native seamless looping.
+12
View File
@@ -64,5 +64,17 @@
},
"0022": {
"title": ""
},
"0023": {
"title": ""
},
"0024": {
"title": ""
},
"0025": {
"title": ""
},
"0026": {
"title": ""
}
}
+24 -10
View File
@@ -399,6 +399,28 @@ def _adjacent_edges() -> list[tuple[str, str]]:
return [(RING_ORDER[i], RING_ORDER[(i + 1) % n]) for i in range(n)]
# All-intra H.264: every frame a keyframe, so the scrub interaction can seek to an
# arbitrary frame smoothly (a sparse GOP scrubs "steppy"). Files grow, but each clip
# stays well under the LFS/proxy ceiling. (Scrub-driven-transitions design §4.)
ALL_INTRA = ["-c:v", "libx264", "-g", "1", "-keyint_min", "1", "-sc_threshold", "0", "-pix_fmt", "yuv420p"]
def transition_cmd(ff: str, a: str, b: str, out: str) -> list[str]:
"""The ffmpeg argv for a forward (zoom-in) member-pair morph, baked all-intra."""
norm = "trim=0:3,setpts=PTS-STARTPTS,scale=1280:720,fps=25,setsar=1,format=yuv420p"
return [
ff, "-y", "-i", str(a), "-i", str(b), "-filter_complex",
f"[0:v]{norm}[a];[1:v]{norm}[b];"
"[a][b]xfade=transition=zoomin:duration=1.5:offset=0.75,format=yuv420p[v]",
"-map", "[v]", "-an", *ALL_INTRA, str(out),
]
def reverse_cmd(ff: str, forward: str, out: str) -> list[str]:
"""The ffmpeg argv for the zoom-OUT companion (forward played backward), all-intra."""
return [ff, "-y", "-i", str(forward), "-vf", "reverse", "-an", *ALL_INTRA, str(out)]
def _make_transition(ff: str, src_clip: str, dst_clip: str) -> Path:
"""A zoom/warp morph between two CLIP MEMBERS' base footage (the well-liked
orbit-coast recipe), keyed by clip id so every adjacent member pair gets its
@@ -407,13 +429,7 @@ def _make_transition(ff: str, src_clip: str, dst_clip: str) -> Path:
b = MEDIA / dst_clip / "base.mp4"
out = MEDIA / "transitions" / f"{src_clip}__{dst_clip}.mp4"
out.parent.mkdir(parents=True, exist_ok=True)
norm = "trim=0:3,setpts=PTS-STARTPTS,scale=1280:720,fps=25,setsar=1,format=yuv420p"
subprocess.run([
ff, "-y", "-i", str(a), "-i", str(b), "-filter_complex",
f"[0:v]{norm}[a];[1:v]{norm}[b];"
"[a][b]xfade=transition=zoomin:duration=1.5:offset=0.75,format=yuv420p[v]",
"-map", "[v]", "-an", str(out),
], check=True, capture_output=True)
subprocess.run(transition_cmd(ff, str(a), str(b), str(out)), check=True, capture_output=True)
return out
@@ -423,9 +439,7 @@ def _make_reverse(ff: str, forward: Path) -> Path:
crosses its edge `reversed`; the renderer then plays this file, so zooming out
recedes from the current scale back to the higher one instead of zooming in."""
out = forward.with_suffix(".rev.mp4")
subprocess.run([
ff, "-y", "-i", str(forward), "-vf", "reverse", "-an", str(out),
], check=True, capture_output=True)
subprocess.run(reverse_cmd(ff, str(forward), str(out)), check=True, capture_output=True)
return out
+183 -27
View File
@@ -29,6 +29,140 @@ async function wheelOnStage(page: Page, deltaY: number) {
await page.locator("#stage").dispatchEvent("wheel", { deltaY });
}
// The Audio control is a 010 level slider; set it to full + fire `input` directly
// (headless Chromium relaxes the autoplay gesture requirement).
async function enableAudio(page: Page) {
await page.evaluate(() => {
const c = document.querySelector("#audio") as HTMLInputElement;
c.value = "10";
c.dispatchEvent(new Event("input", { bubbles: true }));
});
}
// The Video toggle gates the base-clip loop (update() only loads media when video is
// shown). Same hidden-checkbox pattern.
async function enableVideo(page: Page) {
await page.evaluate(() => {
const c = document.querySelector("#visual") as HTMLInputElement;
c.checked = true;
c.dispatchEvent(new Event("change", { bubbles: true }));
});
}
test("two audio elements exist for crossfade", async ({ page }) => {
await boot(page);
const n = await page.evaluate(() => document.querySelectorAll("audio#aud, audio#aud-b").length);
expect(n).toBe(2);
});
test("a new volume fade supersedes a stale one (audio not silenced after landing)", async ({ page }) => {
await boot(page);
// Reproduce the race: a fade-to-0 in flight, then the same element is brought back
// up (as happens when a faded idle element becomes the newly-landed active scale).
// The stale fade must NOT drag it back to 0.
const vol = await page.evaluate(async () => {
const a = document.querySelector("#aud") as HTMLAudioElement;
a.volume = 1;
(window as any).__hefFade(a, 0, 600); // start a slow fade to 0
await new Promise((r) => setTimeout(r, 120)); // let it ramp partway down
(window as any).__hefFade(a, 0.8, 90); // new fade up — must cancel the old one
await new Promise((r) => setTimeout(r, 350)); // well past both fades
return a.volume;
});
expect(vol).toBeGreaterThan(0.7); // ended near 0.8, not dragged to 0
});
test("audio is a 0-10 level dial, defaulting to 0", async ({ page }) => {
await boot(page);
const ctl = await page.evaluate(() => {
const c = document.querySelector("#audio") as HTMLInputElement;
return { type: c.type, min: c.min, max: c.max, value: c.value };
});
expect(ctl.type).toBe("range");
expect(ctl.min).toBe("0");
expect(ctl.max).toBe("10");
expect(ctl.value).toBe("0"); // silent until raised
});
test("turning video on for the first time lifts audio to 3/10", async ({ page }) => {
await boot(page);
expect(await page.evaluate(() => (document.querySelector("#audio") as HTMLInputElement).value)).toBe("0");
await enableVideo(page); // first video-on couples audio at a gentle level
await expect.poll(() => page.evaluate(() => (document.querySelector("#audio") as HTMLInputElement).value)).toBe("3");
// The level label reflects it.
expect(await page.evaluate(() => document.querySelector("#audio-level-val")?.textContent)).toBe("3");
});
test("dragging the dial scrubs morph currentTime and audio gains", async ({ page }) => {
await boot(page);
await enableAudio(page); // hidden custom toggle → set + dispatch change
const box = (await page.locator("#dial").boundingBox())!;
const cx = box.x + box.width / 2, cy = box.y + box.height / 2;
await page.mouse.move(cx, cy - 30);
await page.mouse.down();
await page.mouse.move(cx + 18, cy - 24, { steps: 8 }); // partial clockwise turn (~0.5 detent)
// The currentTime seek is throttled to one rAF; wait for it to land.
await page.waitForFunction(() => (document.querySelector("video") as HTMLVideoElement).currentTime > 0, null, { timeout: 5000 });
const state = await page.evaluate(() => ({
t: (document.querySelector("video") as HTMLVideoElement).currentTime,
ga: (document.querySelector("#aud") as HTMLAudioElement).volume,
gb: (document.querySelector("#aud-b") as HTMLAudioElement).volume,
}));
expect(state.t).toBeGreaterThan(0); // morph scrubbed off frame 0
expect(state.gb).toBeGreaterThan(0); // next scale fading in
expect(state.ga).toBeLessThan(1); // current scale fading out
await page.mouse.up();
});
test("wheel auto-scrubs one altitude and locks", async ({ page }) => {
await boot(page);
const start = (await page.locator("#scale-name").textContent())!;
await wheelOnStage(page, 60); // wheel down = descend one altitude
await page.waitForFunction((s) => document.querySelector("#scale-name")?.textContent !== s, start, { timeout: 20000 });
const landed = (await page.locator("#scale-name").textContent())!;
expect(landed).toContain("orbit");
await page.waitForTimeout(1500);
expect(await page.locator("#scale-name").textContent()).toBe(landed); // locked
});
test("auto-scrub speed is constant per altitude (a 2-step jump takes ~2x a 1-step)", async ({ page }) => {
await boot(page);
async function timeJump(deltaY: number, target: string): Promise<number> {
const t0 = await page.evaluate(() => performance.now());
await page.locator("#stage").dispatchEvent("wheel", { deltaY });
await page.waitForFunction((tgt) => !!document.querySelector("#scale-name")?.textContent?.includes(tgt), target, { timeout: 20000 });
return (await page.evaluate(() => performance.now())) - t0;
}
const t1 = await timeJump(60, "orbit"); // cosmos -> orbit (1 altitude)
const t2 = await timeJump(120, "reef"); // orbit -> reef (2 altitudes)
const ratio = t2 / t1;
expect(ratio).toBeGreaterThan(1.5); // proportional to distance, not a fixed total
expect(ratio).toBeLessThan(2.7); // ~2x, not more
});
test("a landed clip loops from the morph-tail offset (no jump-back to frame 0)", async ({ page }) => {
await boot(page);
await enableVideo(page); // base loop only loads when video is shown
const start = (await page.locator("#scale-name").textContent())!;
await wheelOnStage(page, 60);
await page.waitForFunction((s) => document.querySelector("#scale-name")?.textContent !== s, start, { timeout: 20000 });
// After landing, the dedicated loop element (#vid-loop) is armed (loopTail="1") and
// playing from the tail (~3s) — preloaded during the scrub, so the landing was a
// source swap, not a reload that restarts at 0.
await page.waitForFunction(
() => { const v = document.querySelector("#vid-loop") as HTMLVideoElement; return v.dataset.loopTail === "1" && v.currentTime >= 2.5; },
null,
{ timeout: 10000 },
);
const v = await page.evaluate(() => {
const el = document.querySelector("#vid-loop") as HTMLVideoElement;
return { t: el.currentTime, loopTail: el.dataset.loopTail, paused: el.paused };
});
expect(v.loopTail).toBe("1"); // tail-loop armed
expect(v.paused).toBe(false); // the loop is actually playing
expect(v.t).toBeGreaterThanOrEqual(2.5); // looping from ~3s, not 0
});
test("zoom in plays the matching morph, lands locked, and stays put while parked", async ({ page }) => {
await boot(page);
const start = (await page.locator("#scale-name").textContent())!;
@@ -57,35 +191,57 @@ test("zoom in plays the matching morph, lands locked, and stays put while parked
expect((await morphs(page)).length).toBe(countAfterLanding); // no extra swap
});
test("zoom back out plays a reverse morph and lands locked", async ({ page }) => {
test("turn-back scrubs the same morph in reverse and re-locks the start clip", async ({ page }) => {
await boot(page);
const start = (await page.locator("#scale-name").textContent())!;
await enableAudio(page);
const box = (await page.locator("#dial").boundingBox())!;
const cx = box.x + box.width / 2, cy = box.y + box.height / 2;
const read = () => page.evaluate(() => ({
t: (document.querySelector("video") as HTMLVideoElement).currentTime,
ga: (document.querySelector("#aud") as HTMLAudioElement).volume,
gb: (document.querySelector("#aud-b") as HTMLAudioElement).volume,
name: document.querySelector("#scale-name")?.textContent,
morphs: ((window as any).__hefMorphs || []).slice(),
}));
// Zoom in one detent to get off cosmos (-> orbit)...
await wheelOnStage(page, 60);
await page.waitForFunction(
(s) => document.querySelector("#scale-name")?.textContent !== s,
start,
{ timeout: 20_000 },
);
const atOrbit = (await page.locator("#scale-name").textContent())!;
expect(atOrbit).toContain("orbit");
await page.mouse.move(cx, cy - 30);
await page.mouse.down();
await page.mouse.move(cx + 22, cy - 20, { steps: 10 }); // descend partway into the segment
await page.waitForFunction(() => (document.querySelector("video") as HTMLVideoElement).currentTime > 0, null, { timeout: 5000 });
const fwd = await read();
// The single segment morph is the descend/forward file (cosmos__orbit), NOT a `.rev`.
expect(fwd.morphs[fwd.morphs.length - 1]).toMatch(/^transitions\/cosmos.*\.mp4$/);
expect(fwd.morphs[fwd.morphs.length - 1]).not.toMatch(/\.rev\.mp4$/);
// ...then zoom back OUT to cosmos.
await wheelOnStage(page, -60);
await page.waitForFunction(
(s) => document.querySelector("#scale-name")?.textContent !== s,
atOrbit,
{ timeout: 20_000 },
);
await page.mouse.move(cx + 4, cy - 30, { steps: 10 }); // turn back toward the start
await page.waitForTimeout(120); // let the throttled rAF seek land
const back = await read();
await page.mouse.up();
const played = await morphs(page);
// The most recent morph is the zoom-out: a `.rev` companion. Its file is named
// for the descending pair (cosmos<...>__orbit<...>), so it starts "cosmos".
expect(played[played.length - 1]).toMatch(/^transitions\/cosmos.*\.rev\.mp4$/);
const landed = (await page.locator("#scale-name").textContent())!;
expect(landed).toContain("cosmos");
await page.waitForTimeout(2500);
expect(await page.locator("#scale-name").textContent()).toBe(landed); // locked
expect(back.t).toBeLessThan(fwd.t); // SAME morph file, seeking backward
expect(back.gb).toBeLessThan(fwd.gb); // next-scale audio receding
expect(back.name).toContain("cosmos"); // re-locked the altitude we left
});
test("crossing a detent commits and locks the destination clip", async ({ page }) => {
await boot(page);
const box = (await page.locator("#dial").boundingBox())!;
const cx = box.x + box.width / 2, cy = box.y + box.height / 2;
// Drag clockwise across one full detent (top -> right ~= 90deg = 1.25 detents),
// crossing integer 1 (orbit) so it commits, then hold past it (no auto-complete).
await page.mouse.move(cx, cy - 30);
await page.mouse.down();
await page.mouse.move(cx + 30, cy + 6, { steps: 14 });
await page.waitForFunction(() => (window as any).__hefState().ringIndex === 1, null, { timeout: 5000 });
const st = await page.evaluate(() => (window as any).__hefState());
expect(st.ringIndex).toBe(1); // committed to orbit
expect(String(st.activeClipId)).toMatch(/^orbit/); // locked the destination pool pick
const played = await page.evaluate(() => (window as any).__hefMorphs || []);
expect(played[0]).toMatch(/^transitions\/cosmos.*\.mp4$/);
expect(played[0]).not.toMatch(/\.rev\.mp4$/);
// Hold: the committed clip does not change while parked mid-gesture.
await page.waitForTimeout(800);
const st2 = await page.evaluate(() => (window as any).__hefState());
expect(st2.activeClipId).toBe(st.activeClipId);
await page.mouse.up();
});
Binary file not shown.
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More