Compare commits

..

2 Commits

5 changed files with 83 additions and 39 deletions
@@ -0,0 +1,79 @@
# Session 0022.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-26T08-40 (PST)
> End: 2026-06-26T08-50 (PST)
> Type: executing-plans (operator-driven follow-up to the audio feature)
> Posture: yolo
> Claude-Session: e0cc053a-41fa-416d-a4f5-493f14984e75
> Status: FINALIZED
## Launch prompt
> (1) "It shouldn't say 'Tap to Start Sound'… it can start with both sound and video
> off but immediately go to the app. We should have a 'Loading Universe…' loading
> screen until all media has been downloaded and the experience is ready to run."
> (2) "I still can't hear the sound, even in private mode."
> (3) "The first time video is turned on, also turn on audio. If audio is turned on
> before video after the app loads, just audio should turn on."
## Pre-state
- Sessions 0020/0021 had shipped the audio feature + a 0021 "Safari fix" (tap-to-start
overlay that primed A/B `<audio>` elements). The operator reported audio STILL
silent on their real browser (even in private mode), plus the UX requests above.
## Session arc
1. **Why the 0021 fix failed.** It primed *srcless* A/B elements in the tap gesture
and played the real audio in an *async continuation* (after `await fetch`). Safari
unlocks an element only when `play()` runs **synchronously inside** the gesture and
on an element **with a src** — so neither condition was met. Headless WebKit relaxes
autoplay, which is why automated tests never caught it.
2. **Redesign (client-only, branch `session-0022`):**
- **Single `<audio id="aud">`** element (dropped A/B). It's played **synchronously
inside the toggle's own `change` handler** — the canonical Safari unlock. Altitude
changes reuse the now-unlocked element with a brief fade-swap.
- **"Loading Universe…" splash** (`#loading`, in the HTML so it shows pre-JS): `main()`
now `await`s `preloadAllMedia()` (driving a progress bar), then `hideLoading()`.
Replaces the tap-to-start wall.
- **Both toggles default OFF** (black + silent at start) — no autoplay needed until a
toggle is flipped.
- **First Video-on couples Audio:** the Video `change` handler, on the first on,
sets `#audio` checked and plays — in the same gesture (so it unlocks on Safari).
Audio toggled on first stays audio-only.
- Server contract unchanged (`{off,soundtrack}`); the client now drives playback
directly from the ring's `scale.audio`, ignoring `render.audio`.
3. **Verify + ship.** Rewrote the 5 E2E for the new flow (single `#aud`, no gesture,
loading wait, the coupling, video-off blanking). Caught a test bug — `#black.hidden`
is `display:none`, so `wait_for_selector` default (visible) timed out; switched to
`state="attached"`. **290 passed, 2 skipped; 5 E2E pass.** Verified in headless
**Chromium AND WebKit**: first-Video-on couples audio, soundtrack plays, video shows,
zero JS errors; screenshotted the splash + running app. Merged to `main`.
## Deferred decisions
_No low-confidence calls — all three were explicit operator requests. The
single-element synchronous-in-gesture play is the standard Safari remedy._
## Cut state
- `main` @ `4ac39c9`, clean, pushed. 290 tests + 5 Playwright E2E pass.
- **Honest caveat — still unverifiable headlessly:** every engine Playwright ships
(Chromium, WebKit) relaxes autoplay, so I cannot *prove* real-Safari/iOS now plays.
The fix is the canonical pattern (sync `play()` in the gesture, real src) and is a
genuine change from 0021's broken approach — but it needs the operator's real-device ear.
## Operator plate
- No tap wall: "Loading Universe…" → app with both toggles off (black/silent). Flip
**Video** → video + audio together; flip **Audio** alone → audio only.
- If sound is STILL silent on the real device, that points to something beyond autoplay
(codec/MIME, file, or element error) — next session should capture `aud.error` and the
exact browser.
## Next /goal
```
/goal confirm on a REAL device (esp. iPad/Safari) that audio is now audible when Video/Audio is toggled on and the "Loading Universe…" → app flow feels right; if still silent, capture aud.error + the exact browser. Then the deferred per-altitude music layer per docs/audio-candidate-pool.md
```
+3
View File
@@ -61,5 +61,8 @@
},
"0021": {
"title": ""
},
"0022": {
"title": ""
}
}
+1 -26
View File
@@ -920,29 +920,8 @@ function devLiveReload() {
// (e.g. on an altitude change). audio on = the current scale's soundtrack.
const aud = $("aud");
let audUrl = null; // soundtrack url currently loaded (null = silent)
let audLastErr = ""; // last play() rejection / element error (for the readout)
const FADE_MS = 500;
// --- Live audio status readout (diagnostic) — so a "no sound" report is legible ---
const AUD_ERR = ["", "aborted", "network", "decode", "format-not-supported"];
function audioStatusText() {
if (!$("audio").checked) return "audio: off";
if (aud.error) return `audio: MEDIA ERROR ${aud.error.code} (${AUD_ERR[aud.error.code] || "?"})`;
if (audLastErr) return "audio: " + audLastErr;
if (aud.paused) return `audio: on, PAUSED (readyState ${aud.readyState})`;
return `audio: ▶ playing ${aud.currentTime.toFixed(1)}s · vol ${Math.round(aud.volume * 100)}% · rs ${aud.readyState}`;
}
function updateAudioStatus() {
const el = $("audio-status");
if (!el) return;
el.textContent = audioStatusText();
const playing = $("audio").checked && !aud.paused && !aud.error && !audLastErr;
el.classList.toggle("playing", playing);
el.classList.toggle("blocked", !!aud.error || !!audLastErr);
}
aud.addEventListener("error", () => { audLastErr = ""; updateAudioStatus(); });
setInterval(updateAudioStatus, 400);
function fadeVolume(el, to, ms, done) {
// setInterval (not requestAnimationFrame): rAF throttles/pauses when the tab is
// backgrounded, which would stall the gain ramp; a timer fires regardless.
@@ -968,12 +947,8 @@ function soundtrackUrl() {
function playUrl(url) {
aud.src = mediaUrl(url);
aud.volume = 0;
audLastErr = "";
const pr = aud.play();
if (pr) {
pr.then(() => { audLastErr = ""; updateAudioStatus(); })
.catch((e) => { audLastErr = "play BLOCKED: " + ((e && e.name) || e); updateAudioStatus(); });
}
if (pr) pr.catch(() => {});
fadeVolume(aud, 1, FADE_MS);
}
-9
View File
@@ -40,15 +40,6 @@
<span class="dev-switch-track"><span class="dev-switch-thumb"></span></span>
<span class="dev-switch-label">Audio</span>
</label>
<div id="audio-status" class="audio-status">audio: off</div>
</fieldset>
<fieldset>
<legend>Audio check (diagnostic)</legend>
<p class="hint">Press play below to test the soundtrack file directly in the
browser's own player — this bypasses all app code. If you hear it here but
not from the Audio toggle, the file is fine and it's a code/autoplay issue.</p>
<audio id="aud-test" controls preload="auto" src="/media/audio/cosmos/pillars.loop.mp3" style="width:100%"></audio>
</fieldset>
<fieldset>
-4
View File
@@ -49,10 +49,6 @@ main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; align-items: fl
.hidden { display: none; }
/* Stack the two Output toggles (Video / Audio) with a little breathing room. */
.dev-switch + .dev-switch { margin-top: 0.45rem; }
/* Live audio status readout (diagnostic) — turns green when actually playing. */
.audio-status { margin-top: 0.5rem; font: 11px/1.4 monospace; color: #9ab; }
.audio-status.playing { color: #4e9; }
.audio-status.blocked { color: #f97; }
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem;
max-height: calc(100vh - 2rem); overflow-y: auto;
position: sticky; top: 1rem; }