Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 625d32c0f3 | |||
| 4ac39c9034 | |||
| 30ec0c9c26 | |||
| 3d46b96ecb | |||
| 3b19d3eb92 | |||
| fddb6d65d4 | |||
| d446238b79 | |||
| c51fdc422a |
@@ -13,6 +13,16 @@
|
||||
> hide), and the soundtrack now plays on Safari/iOS (audio elements unlocked
|
||||
> synchronously inside the start gesture). Sections below describe the fuller
|
||||
> off/soundtrack/white-noise design; v1 live = off/soundtrack.
|
||||
>
|
||||
> **Startup revision (session 0022):** the tap-to-start wall is **removed** — the app
|
||||
> opens behind a **"Loading Universe…"** splash (shown until all media is preloaded
|
||||
> and the experience can run smoothly), then both toggles start **off** (black +
|
||||
> silent). Autoplay is satisfied without a wall: a **single** `<audio>` element is
|
||||
> played **synchronously inside the toggle's own click** (the reliable Safari unlock),
|
||||
> replacing the A/B-crossfade-after-gesture approach that failed on real Safari. The
|
||||
> first time **Video** is turned on, **Audio** turns on with it (one flip = the full
|
||||
> experience); turning Audio on first stays audio-only. Altitude changes swap the
|
||||
> single element's source with a brief fade.
|
||||
**Extends / reframes:** [`2026-06-26-networked-control-surface-design.md`](./2026-06-26-networked-control-surface-design.md)
|
||||
§3 (control inventory) and §5 (server contract) — its bundled **Content** selector
|
||||
is split into two orthogonal controls. Everything else in that spec (server-of-truth
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
# Session 0020.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-26T06-47 (PST)
|
||||
> End: 2026-06-26T07-58 (PST)
|
||||
> Type: writing-plans (extended into execution — operator approved "then build it" at the plan gate)
|
||||
> Posture: careful (execution proceeded under explicit operator approval)
|
||||
> Claude-Session: e0cc053a-41fa-416d-a4f5-493f14984e75
|
||||
> Anchor: design docs/superpowers/specs/2026-06-26-audio-video-separated-controls-design.md (R2a)
|
||||
> Status: FINALIZED
|
||||
|
||||
## Launch prompt
|
||||
|
||||
> Write the implementation plan for the "Audio + Video — separated Visual & Audio
|
||||
> controls" feature, then build it. (Spec:
|
||||
> docs/superpowers/specs/2026-06-26-audio-video-separated-controls-design.md.)
|
||||
> Split the bundled 7-way Content selector into orthogonal Visual (on/off) ×
|
||||
> Audio (off / soundtrack / white noise). Audio follows the single Altitude dial;
|
||||
> white noise synthesized (ffmpeg anoisesrc). Reframe the (unbuilt) networked-
|
||||
> control-surface spec §3/§5 as a clean pre-implementation edit. Honor the testing
|
||||
> tiers; resolve the three §12 flagged decisions.
|
||||
|
||||
## Pre-state
|
||||
|
||||
- `main` clean at 99a751c. Two stale `--INPROGRESS` placeholders (0015, 0017) from
|
||||
2026-06-24/25 — prior dead sessions, not live; left untouched.
|
||||
- The networked-control-surface architecture (SSE / `/remote` / renderer split) was
|
||||
**unbuilt** — the sim is a single fused page (`simulator/static/index.html`) talking
|
||||
to FastAPI via `POST /api/alteration` (knobs) + `POST /api/ring/advance` (altitude).
|
||||
- Audio assets (5 sourced ambiences) sat gitignored under `simulator/sample_media/audio/<scale>/`,
|
||||
unreferenced by the manifest, unserved; no `<audio>` element anywhere; video muted.
|
||||
|
||||
## Session arc (turn by turn)
|
||||
|
||||
1. **Session gate → writing-plans.** Classified the two-part "write the plan, then
|
||||
build it" prompt as the build track's entry; claimed session **0020** (`--type
|
||||
writing-plans`, careful posture). Baseline clean; created branch `session-0020`.
|
||||
2. **Scoping.** Read both specs + the audio candidate pool; dispatched an Explore
|
||||
agent to map the current simulator. Key finding: the control-surface architecture
|
||||
is unbuilt, so the audio spec's §6 server contract had no host. **Resolved by the
|
||||
standing simulator-first directive:** build the audio feature into the current
|
||||
single-page sim (ships sound now); reframe the control-surface spec on paper only;
|
||||
do NOT build the SSE/`/remote`/renderer split (it's "v1 hardware," deferred). This
|
||||
build is the "Content step" the audio spec §11 anticipates.
|
||||
3. **Plan.** Wrote `docs/superpowers/plans/2026-06-26-audio-video-separated-controls.md`
|
||||
(10 tasks, all testing tiers). Committed + pushed; presented at the careful-posture
|
||||
plan gate. Operator answered **"Approve — build it."** §12 decisions resolved:
|
||||
audio=live (~0.6 s crossfade), noise=pink, autoplay=tap-to-start overlay; asset
|
||||
params −18 LUFS / mp3 / loop mirrors the proven video crossfade recipe.
|
||||
4. **Build (TDD, commit per task):**
|
||||
- T1–2: pure ffmpeg audio builders (`tools/pipeline/audio_ops.py`) + runner
|
||||
(`audio_run.py`) + `simulator/build_audio_media.py`; produced the 5 normalized
|
||||
loops + pink-noise bed (gitignored).
|
||||
- T3: `Scale.audio` manifest field + read/emit/serialize + `SCALE_AUDIO`; manifest
|
||||
regenerated (clean diff — only 5 audio lines).
|
||||
- T4–6: pure `player/audio.py` resolver; retired `player/content.py` (+ test);
|
||||
reframed `player/controls.py` + `player/state.py`; `/api/alteration` validates
|
||||
visual+audio and returns `render.video` + server-resolved `render.audio.url` from
|
||||
`altitude_index`. Full Python suite green.
|
||||
- T7–8: client — Content `<select>` → Visual toggle + Audio selector; A/B `<audio>`
|
||||
gain-crossfade layer; tap-to-start gesture. Smoke-tested page + media (200s).
|
||||
- T9: reframed the networked-control-surface spec §3/§4/§5/§6; recorded §12
|
||||
resolutions; ROADMAP note.
|
||||
- T10: Playwright E2E (`[e2e]` extra). Installed Chromium and ran it for real —
|
||||
surfaced + fixed three client issues (see Cut state). **3 E2E pass**, full suite
|
||||
**288 passed, 2 skipped**.
|
||||
5. **Verify + ship.** Screenshotted the running app (cosmos plays, soundtrack audibly
|
||||
playing `cosmos/pillars.loop.mp3`, split controls visible, zero JS errors). Merged
|
||||
`session-0020` → `main` (`--no-ff`) and pushed. (`gh` PR create denied — Gitea host;
|
||||
merged locally with branch→merge discipline.)
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_No low-confidence autonomous calls. The scope decision (current-sim vs full control
|
||||
surface) was approved at the plan gate; the three §12 resolutions were operator-
|
||||
delegated by the launch prompt ("resolve the three flagged decisions"); asset params
|
||||
(−18 LUFS, mp3, pink) follow the spec's stated direction._
|
||||
|
||||
## Cut state
|
||||
|
||||
- `main` @ `aa76cd1` (merge of session-0020), clean, synced with origin.
|
||||
- **288 tests + 3 Playwright E2E pass; 2 skipped** (opt-in media integration).
|
||||
- Three client robustness fixes the E2E drove out (worth remembering):
|
||||
- `fadeVolume` uses `setInterval`, not `requestAnimationFrame` (rAF stalls when the
|
||||
tab is backgrounded → the gain ramp hung in headless).
|
||||
- The start gesture drops per-element "priming": a single page click grants Chrome
|
||||
autoplay document-wide; priming srcless `<audio>` only hung the handler / polluted
|
||||
the paused-state.
|
||||
- `applyAudio` does not `await` `play()` (awaiting a srcless/buffering play can hang).
|
||||
- Plan archival: this app has **no content repo** (`CONTENT_REMOTE` empty), so the plan
|
||||
was not archived to a `plans/` collection — it lives in-repo at
|
||||
`docs/superpowers/plans/` and is already on `main`.
|
||||
- §9 pipeline: localhost + E2E stage is **green**; there is no PPE/prod infra for this
|
||||
simulator-first art app (deferred with the Pi renderer), so no further ship stage ran.
|
||||
|
||||
## Operator plate (what's true now / next)
|
||||
|
||||
- The simulator has **sound**: per-altitude soundtracks that follow the Altitude dial +
|
||||
a pink white-noise bed, behind orthogonal **Visual** × **Audio** controls. The unheard
|
||||
audio wants a by-ear review/tune.
|
||||
- Deferred (already roadmap-tracked, not re-parked in memory): the **music** layer
|
||||
(reserved Audio position, no assets), the networked-control-surface SSE/`/remote`
|
||||
build, i2v ring transitions, the Pi renderer.
|
||||
|
||||
## Next /goal
|
||||
|
||||
```
|
||||
/goal review/tune the new audio by ear in the sim (per-altitude soundtracks + pink white-noise + the ~0.6s A/B crossfade; tap-to-start), then source/compose the deferred per-altitude music layer (the reserved Audio position), per docs/audio-candidate-pool.md
|
||||
```
|
||||
@@ -1,22 +0,0 @@
|
||||
# Session 0020.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-26T06-47 (PST)
|
||||
> Type: writing-plans
|
||||
> Posture: careful
|
||||
> Claude-Session: e0cc053a-41fa-416d-a4f5-493f14984e75
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0020 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0020.0-TRANSCRIPT-2026-06-26T06-47--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
_(launch prompt not captured at claim time)_
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
@@ -0,0 +1,88 @@
|
||||
# Session 0021.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-26T08-12 (PST)
|
||||
> End: 2026-06-26T08-20 (PST)
|
||||
> Type: executing-plans (operator-driven hotfix/revision of session 0020's audio feature)
|
||||
> Posture: yolo
|
||||
> Claude-Session: e0cc053a-41fa-416d-a4f5-493f14984e75
|
||||
> Status: FINALIZED
|
||||
|
||||
## Launch prompt
|
||||
|
||||
> Show video unchecked doesn't turn off the video. Make "show video" just "Video"
|
||||
> and have a toggle like "Dev Mode" has. I can't hear any audio when "soundtrack
|
||||
> (follows altitude)" is selected. Let's just have Audio as off/on like Video for
|
||||
> now and get rid of white noise.
|
||||
|
||||
## Pre-state
|
||||
|
||||
- Session 0020 had just merged the audio feature to `main` (Visual×Audio split,
|
||||
per-altitude soundtracks, pink white-noise, A/B crossfade). The operator tested it
|
||||
live and reported three issues.
|
||||
|
||||
## Session arc
|
||||
|
||||
1. **Diagnosis (the key work).** Both reported bugs were reproduced *only* in the
|
||||
operator's real browser — neither appeared in headless Chromium **or** WebKit
|
||||
(installed `playwright install webkit` to test Safari's engine). Root causes:
|
||||
- **Video-off doesn't blank:** `#vid` (`<video>`) and `#paint` (WebGL `<canvas>`)
|
||||
are GPU-composited layers; the `#black` cover had *auto* z-index, so on a real
|
||||
GPU (Safari/Chrome) the video layer can composite *above* it despite DOM order.
|
||||
Headless software-raster hides this.
|
||||
- **No soundtrack audio:** real Safari/iOS unlocks an `<audio>` element only when
|
||||
`play()` is called *synchronously inside* the user gesture; the 0020 gesture
|
||||
played in an async continuation (after `await fetch`), which Chrome allows but
|
||||
Safari blocks. Headless WebKit relaxes autoplay, so it passed.
|
||||
This is the session's main lesson: **Playwright headless relaxes both GPU
|
||||
compositing and autoplay**, so the 0020 E2E couldn't have caught either — real
|
||||
device / by-eye review is required.
|
||||
2. **Fixes (branch `session-0021`):**
|
||||
- **Video** + **Audio** are now Dev-Mode-style on/off toggles (`.dev-switch`),
|
||||
wired on `change`. Audio on → soundtrack; **white-noise deferred** (removed from
|
||||
the live control; `AUDIO_SOURCES={off,soundtrack}`; the pure pink-noise
|
||||
machinery + tests stay for later).
|
||||
- Video-off: `.black` gets `z-index:50` + `translateZ(0)` (own layer) **and** the
|
||||
video layers are set `opacity:0` on video-off (belt-and-suspenders blanking).
|
||||
- Safari audio: the tap-to-start gesture now primes both A/B `<audio>` elements
|
||||
**synchronously in-gesture** (set src, vol 0, `play().then(pause)`), unlocking
|
||||
the later async crossfades.
|
||||
- Server: `player/audio.py` resolver + `controls.py`/`state.py` + `/api/alteration`
|
||||
drop white-noise; `build_audio_media.py` stops emitting the noise bed.
|
||||
3. **Verify + ship.** Updated unit/contract/E2E tests; rewrote the 3 E2E for the new
|
||||
toggles (click the visible `.dev-switch-track`; wait for the opacity transition).
|
||||
**288 passed, 2 skipped.** Re-verified in **both Chromium and WebKit**: Audio toggle
|
||||
plays `cosmos/pillars.loop.mp3`, video-off sets vid+paint opacity 0 with `#black`
|
||||
shown, zero JS errors; screenshotted the new toggle UI. Merged `session-0021` →
|
||||
`main` and pushed. (Also untracked a stray `docs/.threads/*.json` editor artifact
|
||||
swept in by `git add -A`, and gitignored `.threads/`.)
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_No low-confidence calls. All three changes were explicit operator requests; the
|
||||
root-cause fixes (GPU z-index/layer-hide; Safari in-gesture unlock) are the standard
|
||||
remedies for these well-known browser behaviors._
|
||||
|
||||
## Cut state
|
||||
|
||||
- `main` @ `fddb6d6` (merge of session-0021), clean, pushed.
|
||||
- **288 tests + 3 Playwright E2E pass** (2 skipped: opt-in media integration).
|
||||
- White-noise is deferred, not deleted: `tools/pipeline/audio_ops.white_noise_args` +
|
||||
`audio_run.generate_white_noise` + their tests remain; only the live control + the
|
||||
build-script emission were removed.
|
||||
- **Unvalidated by this session:** real-device audio (Safari autoplay) and real-GPU
|
||||
video-off — headless can't exercise either; needs the operator's by-ear/by-eye pass.
|
||||
|
||||
## Operator plate
|
||||
|
||||
- Video + Audio are simple on/off toggles (Dev-Mode style). Audio on = the current
|
||||
altitude's soundtrack. The two real-browser bugs should now be fixed — but please
|
||||
confirm on a real device (especially iPad/Safari).
|
||||
- Deferred: white-noise (easy to re-add as a 3rd Audio position), the per-altitude
|
||||
music layer, the networked-control-surface build, Pi renderer.
|
||||
|
||||
## Next /goal
|
||||
|
||||
```
|
||||
/goal by-ear/by-eye review the audio + Video/Audio toggles on a REAL device (esp. iPad/Safari — headless can't validate Safari autoplay or GPU compositing), then source/compose the deferred per-altitude music layer per docs/audio-candidate-pool.md (or re-enable white-noise as a 3rd Audio position if wanted)
|
||||
```
|
||||
@@ -58,5 +58,8 @@
|
||||
},
|
||||
"0020": {
|
||||
"title": ""
|
||||
},
|
||||
"0021": {
|
||||
"title": ""
|
||||
}
|
||||
}
|
||||
|
||||
+100
-75
@@ -141,7 +141,10 @@ async function preloadAllMedia(concurrency = 4) {
|
||||
let done = 0;
|
||||
const total = files.length;
|
||||
const chip = preloadChip();
|
||||
const tick = () => { if (chip) chip.textContent = `⬇ caching clips ${done}/${total}`; };
|
||||
const tick = () => {
|
||||
if (chip) chip.textContent = `⬇ caching clips ${done}/${total}`;
|
||||
setLoadingProgress(done, total); // drive the "Loading Universe…" bar
|
||||
};
|
||||
tick();
|
||||
let i = 0;
|
||||
async function worker() {
|
||||
@@ -519,7 +522,6 @@ async function update() {
|
||||
if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; }
|
||||
const data = await resp.json();
|
||||
readout.textContent = JSON.stringify(data, null, 2);
|
||||
applyAudio(data.render.audio); // reconcile the audio layer
|
||||
if (!data.render.video.shown) {
|
||||
// Blank the screen. Cover with #black AND hide the video layers themselves —
|
||||
// the <video>/<canvas> are GPU-composited and can otherwise show through the
|
||||
@@ -637,6 +639,7 @@ async function advance(delta) {
|
||||
} finally {
|
||||
busy = false;
|
||||
update();
|
||||
applyAudio(); // soundtrack follows the Altitude dial (crossfade to the new scale)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -909,107 +912,129 @@ function devLiveReload() {
|
||||
}, 1000);
|
||||
}
|
||||
|
||||
// --- Audio layer: two <audio> elements, gain-crossfaded (audio spec §7/§8) ---
|
||||
const audA = $("audA"), audB = $("audB");
|
||||
let audActive = audA, audIdle = audB; // which element is currently audible
|
||||
let audUrl = null; // the url currently playing (null = silence)
|
||||
let audioReady = false; // unlocked after the first user gesture (autoplay policy)
|
||||
const XFADE_MS = 600; // ~0.6s gain crossfade — "live", no busy ack (§7)
|
||||
// --- Audio layer: ONE <audio> element, started in-gesture (Safari-safe) ---
|
||||
// Real Safari/iOS only "unlocks" an <audio> element when play() is called
|
||||
// SYNCHRONOUSLY inside a user gesture (Chrome is lenient; Safari is not). So we use
|
||||
// a single element and start it directly from the Video/Audio toggle's click — not
|
||||
// via the async server roundtrip; once unlocked it can be re-played programmatically
|
||||
// (e.g. on an altitude change). audio on = the current scale's soundtrack.
|
||||
const aud = $("aud");
|
||||
let audUrl = null; // soundtrack url currently loaded (null = silent)
|
||||
let audLastErr = ""; // last play() rejection / element error (for the readout)
|
||||
const FADE_MS = 500;
|
||||
|
||||
function fadeVolume(el, to, ms) {
|
||||
// --- Live audio status readout (diagnostic) — so a "no sound" report is legible ---
|
||||
const AUD_ERR = ["", "aborted", "network", "decode", "format-not-supported"];
|
||||
function audioStatusText() {
|
||||
if (!$("audio").checked) return "audio: off";
|
||||
if (aud.error) return `audio: MEDIA ERROR ${aud.error.code} (${AUD_ERR[aud.error.code] || "?"})`;
|
||||
if (audLastErr) return "audio: " + audLastErr;
|
||||
if (aud.paused) return `audio: on, PAUSED (readyState ${aud.readyState})`;
|
||||
return `audio: ▶ playing ${aud.currentTime.toFixed(1)}s · vol ${Math.round(aud.volume * 100)}% · rs ${aud.readyState}`;
|
||||
}
|
||||
function updateAudioStatus() {
|
||||
const el = $("audio-status");
|
||||
if (!el) return;
|
||||
el.textContent = audioStatusText();
|
||||
const playing = $("audio").checked && !aud.paused && !aud.error && !audLastErr;
|
||||
el.classList.toggle("playing", playing);
|
||||
el.classList.toggle("blocked", !!aud.error || !!audLastErr);
|
||||
}
|
||||
aud.addEventListener("error", () => { audLastErr = ""; updateAudioStatus(); });
|
||||
setInterval(updateAudioStatus, 400);
|
||||
|
||||
function fadeVolume(el, to, ms, done) {
|
||||
// setInterval (not requestAnimationFrame): rAF throttles/pauses when the tab is
|
||||
// backgrounded, which would stall the gain ramp; a timer fires regardless.
|
||||
const from = el.volume, steps = Math.max(1, Math.round(ms / 30));
|
||||
let i = 0;
|
||||
return new Promise((resolve) => {
|
||||
const iv = setInterval(() => {
|
||||
i += 1;
|
||||
const k = Math.min(1, i / steps);
|
||||
el.volume = Math.min(1, Math.max(0, from + (to - from) * k));
|
||||
if (k >= 1) { clearInterval(iv); resolve(); }
|
||||
}, 30);
|
||||
});
|
||||
const iv = setInterval(() => {
|
||||
i += 1;
|
||||
const k = Math.min(1, i / steps);
|
||||
el.volume = Math.min(1, Math.max(0, from + (to - from) * k));
|
||||
if (k >= 1) { clearInterval(iv); if (done) done(); }
|
||||
}, 30);
|
||||
}
|
||||
|
||||
// Reconcile the audio layer to render.audio: crossfade the active element to the
|
||||
// new url (or fade to silence for off). White-noise and soundtrack are the same
|
||||
// mechanism — only the url differs; an unchanged url is a no-op so altitude
|
||||
// re-rolls on the same scale don't restart the bed.
|
||||
async function applyAudio(audio) {
|
||||
const url = audio ? audio.url : null;
|
||||
if (url === audUrl) return;
|
||||
audUrl = url;
|
||||
if (!audioReady) return; // deferred until the start gesture (§8)
|
||||
if (!url) { // fade current out to silence
|
||||
await fadeVolume(audActive, 0, XFADE_MS);
|
||||
audActive.pause();
|
||||
return;
|
||||
}
|
||||
audIdle.src = mediaUrl(url); // absolute /media/... passes through mediaUrl
|
||||
audIdle.volume = 0;
|
||||
audIdle.play().catch(() => {}); // start (fire-and-forget — awaiting can hang)
|
||||
await Promise.all([fadeVolume(audIdle, 1, XFADE_MS), fadeVolume(audActive, 0, XFADE_MS)]);
|
||||
audActive.pause();
|
||||
[audActive, audIdle] = [audIdle, audActive]; // swap roles
|
||||
}
|
||||
|
||||
// The current altitude's soundtrack url (the ring carries each scale's `audio`),
|
||||
// resolved client-side so the start gesture can unlock the elements synchronously.
|
||||
// The current altitude's soundtrack url (the ring carries each scale's `audio`).
|
||||
function soundtrackUrl() {
|
||||
const s = ring && ring.scales[ringIndex];
|
||||
return s && s.audio ? "/media/audio/" + s.audio : null;
|
||||
}
|
||||
|
||||
// Browsers block autoplay until a user gesture. The renderer is launched
|
||||
// full-screen by the operator; a one-time tap unlocks audio, then it follows
|
||||
// state (audio spec §8 — the documented launch step).
|
||||
function showStartGesture() {
|
||||
if (audioReady || document.getElementById("start-gesture")) return;
|
||||
const ov = document.createElement("div");
|
||||
ov.id = "start-gesture";
|
||||
ov.textContent = "▶ tap to start sound";
|
||||
ov.addEventListener("click", () => {
|
||||
audioReady = true;
|
||||
ov.remove();
|
||||
// Safari unlocks an <audio> element ONLY when play() is called inside the
|
||||
// user gesture itself — not in a later async continuation (Chrome is lenient;
|
||||
// Safari/iOS is not). Prime BOTH A/B elements now, synchronously, at volume 0,
|
||||
// so the async applyAudio() crossfades that follow are allowed to play.
|
||||
const url = soundtrackUrl();
|
||||
if (url) for (const el of [audA, audB]) {
|
||||
try {
|
||||
el.src = mediaUrl(url);
|
||||
el.volume = 0;
|
||||
const pr = el.play();
|
||||
if (pr) pr.then(() => el.pause()).catch(() => {});
|
||||
} catch (_) { /* priming is best-effort */ }
|
||||
}
|
||||
audUrl = null; // force the next applyAudio to (re)load
|
||||
update(); // re-apply current state, now with audio
|
||||
}, { once: true });
|
||||
document.body.appendChild(ov);
|
||||
// Load `url` into the single element and fade it in. Call this SYNCHRONOUSLY from
|
||||
// the toggle's click the first time so play() is allowed; later calls reuse the
|
||||
// now-unlocked element.
|
||||
function playUrl(url) {
|
||||
aud.src = mediaUrl(url);
|
||||
aud.volume = 0;
|
||||
audLastErr = "";
|
||||
const pr = aud.play();
|
||||
if (pr) {
|
||||
pr.then(() => { audLastErr = ""; updateAudioStatus(); })
|
||||
.catch((e) => { audLastErr = "play BLOCKED: " + ((e && e.name) || e); updateAudioStatus(); });
|
||||
}
|
||||
fadeVolume(aud, 1, FADE_MS);
|
||||
}
|
||||
|
||||
// Reconcile audio to the Audio toggle + current altitude: on → the scale's
|
||||
// soundtrack, off → silence. Idempotent on an unchanged url (so a same-scale pool
|
||||
// re-roll doesn't restart it). MUST be called synchronously from the toggle gesture
|
||||
// the first time it plays.
|
||||
function applyAudio() {
|
||||
const url = $("audio").checked ? soundtrackUrl() : null;
|
||||
if (url === audUrl) return;
|
||||
audUrl = url;
|
||||
if (!url) { fadeVolume(aud, 0, FADE_MS, () => aud.pause()); return; }
|
||||
if (aud.paused || !aud.src) playUrl(url); // start fresh
|
||||
else fadeVolume(aud, 0, 250, () => playUrl(url)); // dip out, then swap + fade in
|
||||
}
|
||||
|
||||
// "Loading Universe…" splash — hidden once media is preloaded; reflects progress.
|
||||
function setLoadingProgress(done, total) {
|
||||
const fill = document.getElementById("loading-fill");
|
||||
if (fill && total) fill.style.width = Math.round((done / total) * 100) + "%";
|
||||
}
|
||||
function hideLoading() {
|
||||
const el = document.getElementById("loading");
|
||||
if (!el) return;
|
||||
el.classList.add("done");
|
||||
setTimeout(() => el.remove(), 700);
|
||||
}
|
||||
|
||||
let videoEverOn = false; // has Video ever been switched on this session?
|
||||
|
||||
async function main() {
|
||||
devLiveReload();
|
||||
try { initPaint(); } catch (e) { paintOK = false; paint.style.display = "none"; showError("WebGL init: " + e.message); }
|
||||
await loadData();
|
||||
await landScale(); // pick the initial scale's pool member before first render
|
||||
preloadAllMedia(); // background: cache every clip in memory for instant altitude swaps
|
||||
buildDial(); // draw the altitude knob from the ring's scales
|
||||
initDev(); // wire the Dev Mode toggle + pool picker (reads persisted state)
|
||||
renderScaleReadout();
|
||||
// Sliders stream on "input"; the Video/Audio toggles fire on "change" (matches
|
||||
// the Dev Mode switch and is the reliable checkbox event across browsers).
|
||||
// Sliders stream on "input"; the toggles fire on "change".
|
||||
for (const id of ["left", "right", "mood"]) $(id).addEventListener("input", debounced);
|
||||
for (const id of ["visual", "audio"]) $(id).addEventListener("change", debounced);
|
||||
showStartGesture(); // one-time tap unlocks audio (browser autoplay policy, §8)
|
||||
// Audio toggle: start/stop the soundtrack SYNCHRONOUSLY in this gesture so it
|
||||
// unlocks + plays on Safari (which blocks play() outside a user gesture).
|
||||
$("audio").addEventListener("change", () => { applyAudio(); debounced(); });
|
||||
// Video toggle: the FIRST time video turns on, bring audio with it (one flip = the
|
||||
// full experience). Audio toggled on its own first stays audio-only. The audio is
|
||||
// set + played in THIS gesture so it unlocks on Safari.
|
||||
$("visual").addEventListener("change", () => {
|
||||
if ($("visual").checked && !videoEverOn) {
|
||||
videoEverOn = true;
|
||||
if (!$("audio").checked) { $("audio").checked = true; applyAudio(); }
|
||||
}
|
||||
debounced();
|
||||
});
|
||||
// Altitude knob: drag to turn (commit detents on release), scroll to step, tap a label to jump.
|
||||
dial.addEventListener("pointerdown", onDialDown);
|
||||
window.addEventListener("pointermove", onDialMove);
|
||||
window.addEventListener("pointerup", onDialUp);
|
||||
dial.addEventListener("wheel", onWheel, { passive: false });
|
||||
$("stage").addEventListener("wheel", onWheel, { passive: false });
|
||||
update();
|
||||
update(); // render the initial state (both toggles off → black, silent)
|
||||
await preloadAllMedia(); // download all media, updating the loading bar
|
||||
hideLoading(); // experience is ready to run smoothly
|
||||
}
|
||||
main();
|
||||
|
||||
@@ -7,13 +7,18 @@
|
||||
<link rel="stylesheet" href="/style.css" />
|
||||
</head>
|
||||
<body>
|
||||
<div id="loading">
|
||||
<div class="loading-inner">
|
||||
<div class="loading-title">Loading Universe<span class="loading-dots"></span></div>
|
||||
<div class="loading-bar"><div id="loading-fill"></div></div>
|
||||
</div>
|
||||
</div>
|
||||
<header><h1>Human Experience Filter — Alteration Preview</h1></header>
|
||||
<main>
|
||||
<section class="stage" id="stage">
|
||||
<div class="screen">
|
||||
<video id="vid" loop muted playsinline></video>
|
||||
<audio id="audA" loop preload="auto"></audio>
|
||||
<audio id="audB" loop preload="auto"></audio>
|
||||
<audio id="aud" loop preload="auto"></audio>
|
||||
<canvas id="paint"></canvas>
|
||||
<div id="tint"></div>
|
||||
<svg id="overlay" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
|
||||
@@ -26,7 +31,7 @@
|
||||
<fieldset>
|
||||
<legend>Output</legend>
|
||||
<label class="dev-switch" for="visual">
|
||||
<input type="checkbox" id="visual" checked />
|
||||
<input type="checkbox" id="visual" />
|
||||
<span class="dev-switch-track"><span class="dev-switch-thumb"></span></span>
|
||||
<span class="dev-switch-label">Video</span>
|
||||
</label>
|
||||
@@ -35,6 +40,15 @@
|
||||
<span class="dev-switch-track"><span class="dev-switch-thumb"></span></span>
|
||||
<span class="dev-switch-label">Audio</span>
|
||||
</label>
|
||||
<div id="audio-status" class="audio-status">audio: off</div>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>Audio check (diagnostic)</legend>
|
||||
<p class="hint">Press play below to test the soundtrack file directly in the
|
||||
browser's own player — this bypasses all app code. If you hear it here but
|
||||
not from the Audio toggle, the file is fine and it's a code/autoplay issue.</p>
|
||||
<audio id="aud-test" controls preload="auto" src="/media/audio/cosmos/pillars.loop.mp3" style="width:100%"></audio>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
|
||||
@@ -49,6 +49,10 @@ main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; align-items: fl
|
||||
.hidden { display: none; }
|
||||
/* Stack the two Output toggles (Video / Audio) with a little breathing room. */
|
||||
.dev-switch + .dev-switch { margin-top: 0.45rem; }
|
||||
/* Live audio status readout (diagnostic) — turns green when actually playing. */
|
||||
.audio-status { margin-top: 0.5rem; font: 11px/1.4 monospace; color: #9ab; }
|
||||
.audio-status.playing { color: #4e9; }
|
||||
.audio-status.blocked { color: #f97; }
|
||||
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem;
|
||||
max-height: calc(100vh - 2rem); overflow-y: auto;
|
||||
position: sticky; top: 1rem; }
|
||||
@@ -115,12 +119,28 @@ input[type=range], select { width: 100%; }
|
||||
.dev-anno .anno-meta { color: #678; }
|
||||
.dev-anno .anno-empty { color: #567; font-style: italic; }
|
||||
|
||||
/* One-time tap-to-start overlay — unlocks audio under the browser autoplay
|
||||
policy (audio spec §8). Removed after the first gesture. */
|
||||
#start-gesture {
|
||||
/* "Loading Universe…" splash — shown until all media is preloaded and the
|
||||
experience is ready to run smoothly, then faded out. */
|
||||
#loading {
|
||||
position: fixed; inset: 0; z-index: 10000;
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
background: rgba(0, 0, 0, 0.82); color: #fff;
|
||||
font: 600 28px/1.2 system-ui, sans-serif; letter-spacing: 0.02em;
|
||||
cursor: pointer; user-select: none;
|
||||
background: radial-gradient(ellipse at center, #0a1230 0%, #02030a 70%);
|
||||
color: #cfe3ff; user-select: none;
|
||||
transition: opacity 0.6s ease;
|
||||
}
|
||||
#loading.done { opacity: 0; pointer-events: none; }
|
||||
.loading-inner { display: flex; flex-direction: column; align-items: center; gap: 1.1rem; }
|
||||
.loading-title { font: 600 30px/1.2 system-ui, sans-serif; letter-spacing: 0.04em; }
|
||||
.loading-dots::after {
|
||||
content: ""; animation: loading-dots 1.4s steps(4, end) infinite;
|
||||
}
|
||||
@keyframes loading-dots { 0% { content: ""; } 25% { content: "."; } 50% { content: ".."; } 75% { content: "..."; } 100% { content: ""; } }
|
||||
.loading-bar {
|
||||
width: 260px; height: 4px; border-radius: 2px;
|
||||
background: rgba(255, 255, 255, 0.12); overflow: hidden;
|
||||
}
|
||||
#loading-fill {
|
||||
height: 100%; width: 0%; border-radius: 2px;
|
||||
background: linear-gradient(90deg, #4e9cff, #9af);
|
||||
transition: width 0.25s ease;
|
||||
}
|
||||
|
||||
+48
-35
@@ -1,7 +1,9 @@
|
||||
"""Playwright E2E for the audio layer (audio spec §9). Skips cleanly when
|
||||
Playwright or its browser binary is absent (the headless dev box) — the unit +
|
||||
contract tiers cover the logic; this asserts the rendered <audio> element src and
|
||||
playback survives a Visual fade. Runs at the PPE stage where a browser exists.
|
||||
"""Playwright E2E for the audio + Video/Audio toggles (audio spec §9). Skips
|
||||
cleanly when Playwright or its browser binary is absent.
|
||||
|
||||
NOTE: headless engines relax BOTH autoplay and GPU compositing, so these assert
|
||||
the WIRING (toggle→play, the first-video-on→audio coupling, video-off blanking) —
|
||||
real Safari/iOS autoplay and real-GPU compositing still need a device by-ear/eye check.
|
||||
|
||||
Install: pip install -e '.[e2e]' && python -m playwright install chromium
|
||||
"""
|
||||
@@ -12,7 +14,6 @@ from contextlib import closing
|
||||
|
||||
import pytest
|
||||
|
||||
# Skip the whole module unless Playwright is importable.
|
||||
pytest.importorskip("playwright.sync_api")
|
||||
|
||||
|
||||
@@ -50,50 +51,62 @@ def page(app_url):
|
||||
|
||||
with sync_playwright() as p:
|
||||
try:
|
||||
# Headless Chromium blocks media playback without a real audio device;
|
||||
# this flag lets play() actually start so `paused` is meaningful (the
|
||||
# spec §9 asserts playback via element src/paused, not a waveform).
|
||||
# headless blocks media playback without a device; this flag lets play()
|
||||
# actually start so `paused` is meaningful (spec §9 asserts src/paused)
|
||||
browser = p.chromium.launch(args=["--autoplay-policy=no-user-gesture-required"])
|
||||
except Exception as exc: # no browser binary installed
|
||||
pytest.skip(f"chromium not available: {exc}")
|
||||
pg = browser.new_page()
|
||||
pg.goto(app_url)
|
||||
pg.click("#start-gesture") # unlock autoplay
|
||||
# wait out the "Loading Universe…" splash (it overlays + blocks clicks)
|
||||
pg.wait_for_selector("#loading", state="detached", timeout=60000)
|
||||
yield pg
|
||||
browser.close()
|
||||
|
||||
|
||||
def _toggle(page, which):
|
||||
"""Click the visible toggle track for #visual / #audio (the input is hidden)."""
|
||||
page.click(f"label[for='{which}'] .dev-switch-track")
|
||||
|
||||
|
||||
def test_app_starts_blanked_and_silent(page):
|
||||
# both toggles default off → black screen, no audio (no tap-to-start wall)
|
||||
assert page.is_checked("#visual") is False
|
||||
assert page.is_checked("#audio") is False
|
||||
page.wait_for_selector("#black:not(.hidden)")
|
||||
assert page.evaluate("document.getElementById('aud').paused") is True
|
||||
|
||||
|
||||
def test_audio_toggle_plays_the_current_scale_soundtrack(page):
|
||||
# Audio on (toggle) → the current altitude's soundtrack (cosmos at index 0) plays
|
||||
page.click("label[for='audio'] .dev-switch-track")
|
||||
_toggle(page, "audio")
|
||||
page.wait_for_function(
|
||||
"['audA','audB'].some(id => {"
|
||||
" const a = document.getElementById(id);"
|
||||
" return /\\/media\\/audio\\/.+\\.mp3/.test(a.src) && !a.paused;"
|
||||
"})"
|
||||
"(() => { const a = document.getElementById('aud');"
|
||||
" return /\\/media\\/audio\\/.+\\.mp3/.test(a.src) && !a.paused; })()"
|
||||
)
|
||||
|
||||
|
||||
def test_video_toggle_off_blanks_the_screen_and_hides_layers(page):
|
||||
page.click("label[for='visual'] .dev-switch-track")
|
||||
page.wait_for_selector("#black:not(.hidden)") # black cover shown
|
||||
# the video layers themselves are hidden too (GPU-composite-proof blanking);
|
||||
# wait for the opacity transition to settle to 0
|
||||
def test_first_video_on_also_enables_audio(page):
|
||||
_toggle(page, "visual") # first video-on couples audio
|
||||
page.wait_for_function("document.getElementById('audio').checked === true")
|
||||
page.wait_for_function(
|
||||
"(() => { const a = document.getElementById('aud'); return !!a.src && !a.paused; })()"
|
||||
)
|
||||
page.wait_for_selector("#black.hidden", state="attached") # video showing (black hidden=display:none)
|
||||
|
||||
|
||||
def test_audio_before_video_stays_audio_only(page):
|
||||
_toggle(page, "audio") # audio on first
|
||||
page.wait_for_function("!document.getElementById('aud').paused")
|
||||
assert page.is_checked("#visual") is False # video untouched
|
||||
page.wait_for_selector("#black:not(.hidden)") # still blanked
|
||||
|
||||
|
||||
def test_video_off_blanks_after_being_on(page):
|
||||
_toggle(page, "visual") # on
|
||||
page.wait_for_selector("#black.hidden", state="attached")
|
||||
_toggle(page, "visual") # off
|
||||
page.wait_for_selector("#black:not(.hidden)")
|
||||
# the GPU-composited video layers are hidden too (settle the opacity transition)
|
||||
page.wait_for_function(
|
||||
"['vid','paint'].every(id => getComputedStyle(document.getElementById(id)).opacity === '0')"
|
||||
)
|
||||
|
||||
|
||||
def test_audio_survives_video_off(page):
|
||||
page.click("label[for='audio'] .dev-switch-track")
|
||||
page.wait_for_function(
|
||||
"['audA','audB'].some(id => !document.getElementById(id).paused "
|
||||
"&& document.getElementById(id).src)"
|
||||
)
|
||||
page.click("label[for='visual'] .dev-switch-track")
|
||||
page.wait_for_selector("#black:not(.hidden)")
|
||||
playing = page.evaluate(
|
||||
"['audA','audB'].some(id => "
|
||||
"{const a = document.getElementById(id); return !a.paused && !!a.src;})"
|
||||
)
|
||||
assert playing is True # audio keeps playing with video off
|
||||
|
||||
Reference in New Issue
Block a user