Compare commits

..

8 Commits

Author SHA1 Message Date
BenStullsBets 3d46b96ecb add sessions/0021/SESSION-0021.0-TRANSCRIPT-2026-06-26T08-12--2026-06-26T08-20.md + replace placeholder/variant SESSION-0021.0-TRANSCRIPT-2026-06-26T08-12--INPROGRESS.md 2026-06-26 12:57:38 -07:00
BenStullsBets 3b19d3eb92 claim human-experience-filter-art session 0021 (placeholder) + sessions.json entry 2026-06-26 12:56:57 -07:00
BenStullsBets fddb6d65d4 Merge session-0021: off/on Audio+Video toggles; Safari autoplay + GPU video-off fixes
Operator-driven revision of the audio feature: Dev-Mode-style Video/Audio toggles,
white-noise deferred (Audio = off/soundtrack), and fixes for two real-browser bugs
(video-off not blanking GPU-composited layers; no soundtrack on Safari/iOS).
Verified in headless Chromium + WebKit. 288 tests + 3 Playwright E2E pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 12:56:20 -07:00
BenStullsBets 2722949805 fix(audio): off/on Audio + Video toggles; fix Safari autoplay + GPU video-off
Operator-driven live-UI revision:
- Video + Audio are now Dev-Mode-style on/off toggles (Audio on = soundtrack);
  white-noise deferred (removed from the live control; pure machinery kept).
- FIX video-off not blanking: the <video>/<canvas> are GPU-composited and could
  paint above the auto-z #black on a real GPU — give #black z-index + its own
  layer AND hide the video layers (opacity 0) on video-off.
- FIX no soundtrack on Safari/iOS: Safari unlocks <audio> only when play() runs
  INSIDE the user gesture; the start gesture now primes both A/B elements
  synchronously, so the async crossfades are allowed to play.
- Toggles wired on 'change' (matches the Dev Mode switch).
- player/audio.py AUDIO_SOURCES -> {off, soundtrack}; tests + E2E updated.
- Verified in headless Chromium AND WebKit: audio plays, video-off blanks both
  layers, zero JS errors. 288 tests + 3 Playwright E2E pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 12:56:07 -07:00
BenStullsBets d446238b79 add sessions/0020/SESSION-0020.0-TRANSCRIPT-2026-06-26T06-47--2026-06-26T07-58.md + replace placeholder/variant SESSION-0020.0-TRANSCRIPT-2026-06-26T06-47--INPROGRESS.md 2026-06-26 08:06:49 -07:00
BenStullsBets c51fdc422a update sessions/0020/SESSION-0020.0-TRANSCRIPT-2026-06-26T06-47--INPROGRESS.md 2026-06-26 08:06:25 -07:00
BenStullsBets aa76cd1c9f Merge session-0020: Audio + separated Visual/Audio controls
Splits bundled Content into orthogonal Visual×Audio; per-altitude soundtracks +
synthesized pink white-noise; server-resolved render.audio; client A/B crossfade
layer; control-surface spec reframed. 288 tests + 3 Playwright E2E pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:02:45 -07:00
BenStullsBets adc39fe648 claim human-experience-filter-art session 0020 (placeholder) + sessions.json entry 2026-06-26 06:48:09 -07:00
19 changed files with 349 additions and 106 deletions
+3
View File
@@ -17,3 +17,6 @@ simulator/sample_media/**/*.m4a
# Local-only candidate review galleries (re-buildable; not shipped content).
simulator/static/review.html
simulator/static/review_audio.html
# Editor annotation/comment-thread metadata (not project content)
.threads/
@@ -1,8 +1,18 @@
# Audio + Video — separated Visual & Audio controls — design
**Status:** proposed → in implementation (plan `docs/superpowers/plans/2026-06-26-audio-video-separated-controls.md`, session 0020)
**Status:** proposed → built (sessions 0020 build + 0021 revision)
**Date:** 2026-06-26
**Session:** 0020 (audio)
**Session:** 0020 (audio) · 0021 (live-UI revision)
> **v1 build revision (session 0021, operator-driven):** the live **Audio** control
> ships as a simple **off / soundtrack** toggle (styled like the Dev Mode switch),
> and **Video** is the matching on/off toggle. **White-noise is deferred** with music
> (the orthogonal Visual × Audio model and the synthesized-pink-noise machinery
> remain in the codebase for when it's wanted). Two real-browser bugs were fixed in
> this revision: video-off now blanks GPU-composited layers (z-index + explicit layer
> hide), and the soundtrack now plays on Safari/iOS (audio elements unlocked
> synchronously inside the start gesture). Sections below describe the fuller
> off/soundtrack/white-noise design; v1 live = off/soundtrack.
**Extends / reframes:** [`2026-06-26-networked-control-surface-design.md`](./2026-06-26-networked-control-surface-design.md)
§3 (control inventory) and §5 (server contract) — its bundled **Content** selector
is split into two orthogonal controls. Everything else in that spec (server-of-truth
@@ -75,7 +75,7 @@ inventory is exactly what the sim exposes today:
| Control | Sim today | Physical form (later) |
|---|---|---|
| **Visual** | toggle (show video on/off) | 2-position switch |
| **Audio** | 3-way `<select>` (off / soundtrack / white noise; ·music deferred) | rotary selector switch (3 pos) |
| **Audio** | on/off toggle (on = per-altitude soundtrack; ·white-noise & music deferred) | 2-position switch (rotary selector when more sources land) |
| **Altitude** | endless circular dial — walks the 5 scales (cosmos→abyss), wraps | rotary encoder (endless, detented) |
| **Left** (analytical) | slider 04 | knob/pot, 5 detents |
| **Right** (dreamlike) | slider 04 | knob/pot, 5 detents |
+11 -12
View File
@@ -2,17 +2,18 @@
§4/§6). The single source of truth for: whether the projector shows video, and
which audio url plays for a given (audio source, current altitude).
Replaces player/content.py's 7-way bundled table. White-noise is a global,
altitude-independent bed; soundtrack follows the single Altitude dial; off is
silence. `music` is a reserved, deferred source — not in v1."""
Replaces player/content.py's 7-way bundled table. Soundtrack follows the single
Altitude dial; off is silence. `white_noise` and `music` are deferred sources —
not in v1 (the live control is a simple Audio on/off toggle where on=soundtrack)."""
from __future__ import annotations
from dataclasses import dataclass
VISUAL_POSITIONS = frozenset({"on", "off"})
# v1 audio sources. "music" is reserved/deferred (no assets) and intentionally absent.
AUDIO_SOURCES = frozenset({"off", "soundtrack", "white_noise"})
# v1 audio sources. The live UI is an Audio on/off toggle (on=soundtrack).
# "white_noise" and "music" are deferred (no live control) and intentionally absent.
AUDIO_SOURCES = frozenset({"off", "soundtrack"})
@dataclass(frozen=True)
@@ -31,24 +32,22 @@ def resolve_visual(position: str) -> bool:
return position == "on"
def resolve_audio_url(source: str, *, scale_audio: str, noise_url: str) -> str | None:
def resolve_audio_url(source: str, *, scale_audio: str) -> str | None:
"""The audio url for (source, current altitude): soundtrack → the current
scale's asset (or None if none authored); white_noise → the global bed; off →
None. `scale_audio` is the ring scale's `audio` path (relative to /media/audio/)."""
scale's asset (or None if none authored); off → None. `scale_audio` is the ring
scale's `audio` path (relative to /media/audio/)."""
if source not in AUDIO_SOURCES:
raise ValueError(
f"unknown audio source {source!r}; expected one of {sorted(AUDIO_SOURCES)}"
)
if source == "off":
return None
if source == "white_noise":
return noise_url
# soundtrack — couple to the current altitude's asset
return f"/media/audio/{scale_audio}" if scale_audio else None
def resolve_audio(source: str, *, scale_audio: str, noise_url: str) -> AudioResolution:
def resolve_audio(source: str, *, scale_audio: str) -> AudioResolution:
"""The full render.audio view: source, resolved url, and whether it follows the
Altitude dial (true only for soundtrack)."""
url = resolve_audio_url(source, scale_audio=scale_audio, noise_url=noise_url)
url = resolve_audio_url(source, scale_audio=scale_audio)
return AudioResolution(source=source, url=url, altitude_coupled=(source == "soundtrack"))
+1 -1
View File
@@ -2,7 +2,7 @@
This is the serial-contract payload shared with sub-project 4 (firmware). It
models the full panel from the audio spec §3: a Visual toggle (on/off) + an
Audio source selector (off/soundtrack/white_noise), the four experience knobs
Audio source selector (off/soundtrack; white-noise/music deferred), the four experience knobs
(0..4), and the two intensity levels (volume, brightness). The wire framing
itself is the separate 3<->4 serial contract; this module is the *decoded* form.
"""
+2 -2
View File
@@ -36,12 +36,12 @@ class TransitionKind:
class Playback:
"""What should currently be playing. Visual and audio are orthogonal (audio
spec §2): `video` is whether the projector shows footage, `audio_source` is
the independent audio dial (off / soundtrack / white_noise)."""
the independent audio dial (off / soundtrack)."""
clip_id: Optional[str] # None = black walls
plan: Optional[RenderPlan] # None when black
video: bool # whether footage is shown (Visual on/off)
audio_source: str # the Audio dial source (off/soundtrack/white_noise)
audio_source: str # the Audio dial source (off/soundtrack)
volume: int
brightness: int
@@ -0,0 +1,109 @@
# Session 0020.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-26T06-47 (PST)
> End: 2026-06-26T07-58 (PST)
> Type: writing-plans (extended into execution — operator approved "then build it" at the plan gate)
> Posture: careful (execution proceeded under explicit operator approval)
> Claude-Session: e0cc053a-41fa-416d-a4f5-493f14984e75
> Anchor: design docs/superpowers/specs/2026-06-26-audio-video-separated-controls-design.md (R2a)
> Status: FINALIZED
## Launch prompt
> Write the implementation plan for the "Audio + Video — separated Visual & Audio
> controls" feature, then build it. (Spec:
> docs/superpowers/specs/2026-06-26-audio-video-separated-controls-design.md.)
> Split the bundled 7-way Content selector into orthogonal Visual (on/off) ×
> Audio (off / soundtrack / white noise). Audio follows the single Altitude dial;
> white noise synthesized (ffmpeg anoisesrc). Reframe the (unbuilt) networked-
> control-surface spec §3/§5 as a clean pre-implementation edit. Honor the testing
> tiers; resolve the three §12 flagged decisions.
## Pre-state
- `main` clean at 99a751c. Two stale `--INPROGRESS` placeholders (0015, 0017) from
2026-06-24/25 — prior dead sessions, not live; left untouched.
- The networked-control-surface architecture (SSE / `/remote` / renderer split) was
**unbuilt** — the sim is a single fused page (`simulator/static/index.html`) talking
to FastAPI via `POST /api/alteration` (knobs) + `POST /api/ring/advance` (altitude).
- Audio assets (5 sourced ambiences) sat gitignored under `simulator/sample_media/audio/<scale>/`,
unreferenced by the manifest, unserved; no `<audio>` element anywhere; video muted.
## Session arc (turn by turn)
1. **Session gate → writing-plans.** Classified the two-part "write the plan, then
build it" prompt as the build track's entry; claimed session **0020** (`--type
writing-plans`, careful posture). Baseline clean; created branch `session-0020`.
2. **Scoping.** Read both specs + the audio candidate pool; dispatched an Explore
agent to map the current simulator. Key finding: the control-surface architecture
is unbuilt, so the audio spec's §6 server contract had no host. **Resolved by the
standing simulator-first directive:** build the audio feature into the current
single-page sim (ships sound now); reframe the control-surface spec on paper only;
do NOT build the SSE/`/remote`/renderer split (it's "v1 hardware," deferred). This
build is the "Content step" the audio spec §11 anticipates.
3. **Plan.** Wrote `docs/superpowers/plans/2026-06-26-audio-video-separated-controls.md`
(10 tasks, all testing tiers). Committed + pushed; presented at the careful-posture
plan gate. Operator answered **"Approve — build it."** §12 decisions resolved:
audio=live (~0.6 s crossfade), noise=pink, autoplay=tap-to-start overlay; asset
params 18 LUFS / mp3 / loop mirrors the proven video crossfade recipe.
4. **Build (TDD, commit per task):**
- T12: pure ffmpeg audio builders (`tools/pipeline/audio_ops.py`) + runner
(`audio_run.py`) + `simulator/build_audio_media.py`; produced the 5 normalized
loops + pink-noise bed (gitignored).
- T3: `Scale.audio` manifest field + read/emit/serialize + `SCALE_AUDIO`; manifest
regenerated (clean diff — only 5 audio lines).
- T46: pure `player/audio.py` resolver; retired `player/content.py` (+ test);
reframed `player/controls.py` + `player/state.py`; `/api/alteration` validates
visual+audio and returns `render.video` + server-resolved `render.audio.url` from
`altitude_index`. Full Python suite green.
- T78: client — Content `<select>` → Visual toggle + Audio selector; A/B `<audio>`
gain-crossfade layer; tap-to-start gesture. Smoke-tested page + media (200s).
- T9: reframed the networked-control-surface spec §3/§4/§5/§6; recorded §12
resolutions; ROADMAP note.
- T10: Playwright E2E (`[e2e]` extra). Installed Chromium and ran it for real —
surfaced + fixed three client issues (see Cut state). **3 E2E pass**, full suite
**288 passed, 2 skipped**.
5. **Verify + ship.** Screenshotted the running app (cosmos plays, soundtrack audibly
playing `cosmos/pillars.loop.mp3`, split controls visible, zero JS errors). Merged
`session-0020``main` (`--no-ff`) and pushed. (`gh` PR create denied — Gitea host;
merged locally with branch→merge discipline.)
## Deferred decisions
_No low-confidence autonomous calls. The scope decision (current-sim vs full control
surface) was approved at the plan gate; the three §12 resolutions were operator-
delegated by the launch prompt ("resolve the three flagged decisions"); asset params
(18 LUFS, mp3, pink) follow the spec's stated direction._
## Cut state
- `main` @ `aa76cd1` (merge of session-0020), clean, synced with origin.
- **288 tests + 3 Playwright E2E pass; 2 skipped** (opt-in media integration).
- Three client robustness fixes the E2E drove out (worth remembering):
- `fadeVolume` uses `setInterval`, not `requestAnimationFrame` (rAF stalls when the
tab is backgrounded → the gain ramp hung in headless).
- The start gesture drops per-element "priming": a single page click grants Chrome
autoplay document-wide; priming srcless `<audio>` only hung the handler / polluted
the paused-state.
- `applyAudio` does not `await` `play()` (awaiting a srcless/buffering play can hang).
- Plan archival: this app has **no content repo** (`CONTENT_REMOTE` empty), so the plan
was not archived to a `plans/` collection — it lives in-repo at
`docs/superpowers/plans/` and is already on `main`.
- §9 pipeline: localhost + E2E stage is **green**; there is no PPE/prod infra for this
simulator-first art app (deferred with the Pi renderer), so no further ship stage ran.
## Operator plate (what's true now / next)
- The simulator has **sound**: per-altitude soundtracks that follow the Altitude dial +
a pink white-noise bed, behind orthogonal **Visual** × **Audio** controls. The unheard
audio wants a by-ear review/tune.
- Deferred (already roadmap-tracked, not re-parked in memory): the **music** layer
(reserved Audio position, no assets), the networked-control-surface SSE/`/remote`
build, i2v ring transitions, the Pi renderer.
## Next /goal
```
/goal review/tune the new audio by ear in the sim (per-altitude soundtracks + pink white-noise + the ~0.6s A/B crossfade; tap-to-start), then source/compose the deferred per-altitude music layer (the reserved Audio position), per docs/audio-candidate-pool.md
```
@@ -0,0 +1,88 @@
# Session 0021.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-26T08-12 (PST)
> End: 2026-06-26T08-20 (PST)
> Type: executing-plans (operator-driven hotfix/revision of session 0020's audio feature)
> Posture: yolo
> Claude-Session: e0cc053a-41fa-416d-a4f5-493f14984e75
> Status: FINALIZED
## Launch prompt
> Show video unchecked doesn't turn off the video. Make "show video" just "Video"
> and have a toggle like "Dev Mode" has. I can't hear any audio when "soundtrack
> (follows altitude)" is selected. Let's just have Audio as off/on like Video for
> now and get rid of white noise.
## Pre-state
- Session 0020 had just merged the audio feature to `main` (Visual×Audio split,
per-altitude soundtracks, pink white-noise, A/B crossfade). The operator tested it
live and reported three issues.
## Session arc
1. **Diagnosis (the key work).** Both reported bugs were reproduced *only* in the
operator's real browser — neither appeared in headless Chromium **or** WebKit
(installed `playwright install webkit` to test Safari's engine). Root causes:
- **Video-off doesn't blank:** `#vid` (`<video>`) and `#paint` (WebGL `<canvas>`)
are GPU-composited layers; the `#black` cover had *auto* z-index, so on a real
GPU (Safari/Chrome) the video layer can composite *above* it despite DOM order.
Headless software-raster hides this.
- **No soundtrack audio:** real Safari/iOS unlocks an `<audio>` element only when
`play()` is called *synchronously inside* the user gesture; the 0020 gesture
played in an async continuation (after `await fetch`), which Chrome allows but
Safari blocks. Headless WebKit relaxes autoplay, so it passed.
This is the session's main lesson: **Playwright headless relaxes both GPU
compositing and autoplay**, so the 0020 E2E couldn't have caught either — real
device / by-eye review is required.
2. **Fixes (branch `session-0021`):**
- **Video** + **Audio** are now Dev-Mode-style on/off toggles (`.dev-switch`),
wired on `change`. Audio on → soundtrack; **white-noise deferred** (removed from
the live control; `AUDIO_SOURCES={off,soundtrack}`; the pure pink-noise
machinery + tests stay for later).
- Video-off: `.black` gets `z-index:50` + `translateZ(0)` (own layer) **and** the
video layers are set `opacity:0` on video-off (belt-and-suspenders blanking).
- Safari audio: the tap-to-start gesture now primes both A/B `<audio>` elements
**synchronously in-gesture** (set src, vol 0, `play().then(pause)`), unlocking
the later async crossfades.
- Server: `player/audio.py` resolver + `controls.py`/`state.py` + `/api/alteration`
drop white-noise; `build_audio_media.py` stops emitting the noise bed.
3. **Verify + ship.** Updated unit/contract/E2E tests; rewrote the 3 E2E for the new
toggles (click the visible `.dev-switch-track`; wait for the opacity transition).
**288 passed, 2 skipped.** Re-verified in **both Chromium and WebKit**: Audio toggle
plays `cosmos/pillars.loop.mp3`, video-off sets vid+paint opacity 0 with `#black`
shown, zero JS errors; screenshotted the new toggle UI. Merged `session-0021`
`main` and pushed. (Also untracked a stray `docs/.threads/*.json` editor artifact
swept in by `git add -A`, and gitignored `.threads/`.)
## Deferred decisions
_No low-confidence calls. All three changes were explicit operator requests; the
root-cause fixes (GPU z-index/layer-hide; Safari in-gesture unlock) are the standard
remedies for these well-known browser behaviors._
## Cut state
- `main` @ `fddb6d6` (merge of session-0021), clean, pushed.
- **288 tests + 3 Playwright E2E pass** (2 skipped: opt-in media integration).
- White-noise is deferred, not deleted: `tools/pipeline/audio_ops.white_noise_args` +
`audio_run.generate_white_noise` + their tests remain; only the live control + the
build-script emission were removed.
- **Unvalidated by this session:** real-device audio (Safari autoplay) and real-GPU
video-off — headless can't exercise either; needs the operator's by-ear/by-eye pass.
## Operator plate
- Video + Audio are simple on/off toggles (Dev-Mode style). Audio on = the current
altitude's soundtrack. The two real-browser bugs should now be fixed — but please
confirm on a real device (especially iPad/Safari).
- Deferred: white-noise (easy to re-add as a 3rd Audio position), the per-altitude
music layer, the networked-control-surface build, Pi renderer.
## Next /goal
```
/goal by-ear/by-eye review the audio + Video/Audio toggles on a REAL device (esp. iPad/Safari — headless can't validate Safari autoplay or GPU compositing), then source/compose the deferred per-altitude music layer per docs/audio-candidate-pool.md (or re-enable white-noise as a 3rd Audio position if wanted)
```
+6
View File
@@ -55,5 +55,11 @@
},
"0019": {
"title": ""
},
"0020": {
"title": ""
},
"0021": {
"title": ""
}
}
+1 -4
View File
@@ -39,9 +39,6 @@ STATIC_DIR = Path(__file__).parent / "static"
MEDIA_DIR = Path(__file__).parent / "sample_media"
DEFAULT_MANIFEST = MEDIA_DIR / "manifest.json"
# The global white-noise bed (audio spec §5.2): synthesized, altitude-independent.
NOISE_URL = "/media/audio/noise/pink.mp3"
# Per-process boot token: changes on every server restart so the dev live-reload
# (below) also fires when Python code changes (which needs a restart), not just
# when a static asset's mtime changes.
@@ -190,7 +187,7 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
scale_audio = ""
if app.state.ring is not None:
scale_audio = scale_at(app.state.ring, req.altitude_index).audio
audio = resolve_audio(c.audio, scale_audio=scale_audio, noise_url=NOISE_URL)
audio = resolve_audio(c.audio, scale_audio=scale_audio)
return {
"plan": render_plan_to_dict(plan),
"render": {
+11 -9
View File
@@ -1,18 +1,22 @@
"""Production pass for the 5 per-altitude soundtracks + the white-noise bed
(audio spec §5.3). The audio analogue of build_pool_manifest.py --media: read the
sourced ambience clips (re-downloadable per docs/audio-candidate-pool.md), make
each a seamless loudness-normalized loop, and synthesize the pink-noise bed.
"""Production pass for the 5 per-altitude soundtracks (audio spec §5.3). The audio
analogue of build_pool_manifest.py --media: read the sourced ambience clips
(re-downloadable per docs/audio-candidate-pool.md), and make each a seamless
loudness-normalized loop.
Media is gitignored; this script is the reproducible record. Outputs land under
simulator/sample_media/audio/<scale>/<name>.loop.mp3 and audio/noise/pink.mp3,
served at /media/audio/... Run: python simulator/build_audio_media.py
simulator/sample_media/audio/<scale>/<name>.loop.mp3, served at /media/audio/...
Run: python simulator/build_audio_media.py
(White-noise is deferred — the live Audio control is a simple on/off soundtrack
toggle. `tools.pipeline.audio_run.generate_white_noise` remains for when it's
wanted, but no noise bed is built here.)
"""
from __future__ import annotations
from pathlib import Path
from tools.pipeline.audio_run import generate_white_noise, process_soundtrack
from tools.pipeline.audio_run import process_soundtrack
AUDIO = Path(__file__).parent / "sample_media" / "audio"
@@ -35,8 +39,6 @@ def main() -> None:
continue
dst = process_soundtrack(src, AUDIO / scale / out)
print(f" produced {dst.relative_to(AUDIO.parent)}")
noise = generate_white_noise(AUDIO / "noise" / "pink.mp3")
print(f" produced {noise.relative_to(AUDIO.parent)}")
if __name__ == "__main__":
+39 -12
View File
@@ -495,14 +495,14 @@ function renderScaleReadout() {
}
function controls() {
// Visual (on/off) and Audio (off/soundtrack/white_noise) are orthogonal dials
// (audio spec §2). One bipolar Mood dial (-4 dark .. +4 light) maps onto the
// engine's two poles. Calibration gains are pinned to 1.0, so the simulator
// sends no calibration — the server uses DEFAULT_CALIBRATION.
// Video (on/off) and Audio (on/off) are orthogonal toggles (audio spec §2).
// Audio on = the per-altitude soundtrack (white-noise is deferred). One bipolar
// Mood dial (-4 dark .. +4 light) maps onto the engine's two poles. Calibration
// gains are pinned to 1.0, so the simulator sends no calibration.
const mood = +$("mood").value;
return {
visual: $("visual").checked ? "on" : "off",
audio: $("audio").value,
audio: $("audio").checked ? "soundtrack" : "off",
left: +$("left").value, right: +$("right").value,
dark: mood < 0 ? -mood : 0, light: mood > 0 ? mood : 0,
volume: 2, brightness: 2,
@@ -520,8 +520,17 @@ async function update() {
const data = await resp.json();
readout.textContent = JSON.stringify(data, null, 2);
applyAudio(data.render.audio); // reconcile the audio layer
if (!data.render.video.shown) { black.style.opacity = "1"; black.classList.remove("hidden"); return; }
if (!data.render.video.shown) {
// Blank the screen. Cover with #black AND hide the video layers themselves —
// the <video>/<canvas> are GPU-composited and can otherwise show through the
// overlay on a real GPU (Safari/Chrome).
black.style.opacity = "1"; black.classList.remove("hidden");
vid.style.opacity = "0"; paint.style.opacity = "0";
return;
}
black.classList.add("hidden");
vid.style.opacity = ""; paint.style.opacity = ""; // restore the video layers
try {
ensureClipMedia();
applyVideoLook(data.plan.grade.tone, data.plan.dream.intensity);
@@ -944,6 +953,13 @@ async function applyAudio(audio) {
[audActive, audIdle] = [audIdle, audActive]; // swap roles
}
// The current altitude's soundtrack url (the ring carries each scale's `audio`),
// resolved client-side so the start gesture can unlock the elements synchronously.
function soundtrackUrl() {
const s = ring && ring.scales[ringIndex];
return s && s.audio ? "/media/audio/" + s.audio : null;
}
// Browsers block autoplay until a user gesture. The renderer is launched
// full-screen by the operator; a one-time tap unlocks audio, then it follows
// state (audio spec §8 — the documented launch step).
@@ -953,11 +969,21 @@ function showStartGesture() {
ov.id = "start-gesture";
ov.textContent = "▶ tap to start sound";
ov.addEventListener("click", () => {
// A click anywhere on the page grants Chrome's autoplay permission
// document-wide for the session, so later programmatic play() is allowed —
// no per-element priming needed (priming srcless elements only pollutes state).
audioReady = true;
ov.remove();
// Safari unlocks an <audio> element ONLY when play() is called inside the
// user gesture itself — not in a later async continuation (Chrome is lenient;
// Safari/iOS is not). Prime BOTH A/B elements now, synchronously, at volume 0,
// so the async applyAudio() crossfades that follow are allowed to play.
const url = soundtrackUrl();
if (url) for (const el of [audA, audB]) {
try {
el.src = mediaUrl(url);
el.volume = 0;
const pr = el.play();
if (pr) pr.then(() => el.pause()).catch(() => {});
} catch (_) { /* priming is best-effort */ }
}
audUrl = null; // force the next applyAudio to (re)load
update(); // re-apply current state, now with audio
}, { once: true });
@@ -973,9 +999,10 @@ async function main() {
buildDial(); // draw the altitude knob from the ring's scales
initDev(); // wire the Dev Mode toggle + pool picker (reads persisted state)
renderScaleReadout();
for (const id of ["visual", "audio", "left", "right", "mood"]) {
$(id).addEventListener("input", debounced);
}
// Sliders stream on "input"; the Video/Audio toggles fire on "change" (matches
// the Dev Mode switch and is the reliable checkbox event across browsers).
for (const id of ["left", "right", "mood"]) $(id).addEventListener("input", debounced);
for (const id of ["visual", "audio"]) $(id).addEventListener("change", debounced);
showStartGesture(); // one-time tap unlocks audio (browser autoplay policy, §8)
// Altitude knob: drag to turn (commit detents on release), scroll to step, tap a label to jump.
dial.addEventListener("pointerdown", onDialDown);
+11 -11
View File
@@ -24,17 +24,17 @@
<section class="panel">
<fieldset>
<legend>Visual</legend>
<label class="toggle"><input type="checkbox" id="visual" checked /> show video</label>
</fieldset>
<fieldset>
<legend>Audio</legend>
<select id="audio">
<option value="off">off (silence)</option>
<option value="soundtrack">soundtrack (follows altitude)</option>
<option value="white_noise">white noise</option>
</select>
<legend>Output</legend>
<label class="dev-switch" for="visual">
<input type="checkbox" id="visual" checked />
<span class="dev-switch-track"><span class="dev-switch-thumb"></span></span>
<span class="dev-switch-label">Video</span>
</label>
<label class="dev-switch" for="audio">
<input type="checkbox" id="audio" />
<span class="dev-switch-track"><span class="dev-switch-thumb"></span></span>
<span class="dev-switch-label">Audio</span>
</label>
</fieldset>
<fieldset>
+8 -1
View File
@@ -40,8 +40,15 @@ main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; align-items: fl
.hud-chip-text.measure { fill: #ffd79a; }
.hud-conf { fill: #6cf; opacity: 0.8; }
.hud-status { fill: #8fdcff; font: 2.4px monospace; opacity: 0.9; letter-spacing: 0.15px; }
.black { position: absolute; inset: 0; background: #000; opacity: 1; transition: opacity 200ms ease; }
/* z-index + own compositing layer: the <video> and WebGL <canvas> are GPU-
composited layers that can paint ABOVE a plain auto-z sibling (Safari/Chrome
with a real GPU) despite DOM order — so the black cover needs an explicit
stacking order above them to actually blank the screen. */
.black { position: absolute; inset: 0; background: #000; opacity: 1;
z-index: 50; transform: translateZ(0); transition: opacity 200ms ease; }
.hidden { display: none; }
/* Stack the two Output toggles (Video / Audio) with a little breathing room. */
.dev-switch + .dev-switch { margin-top: 0.45rem; }
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem;
max-height: calc(100vh - 2rem); overflow-y: auto;
position: sticky; top: 1rem; }
+22 -21
View File
@@ -63,36 +63,37 @@ def page(app_url):
browser.close()
def test_white_noise_loads_the_noise_asset(page):
page.select_option("#audio", "white_noise")
page.wait_for_function(
"['audA','audB'].some(id => "
"document.getElementById(id).src.includes('/media/audio/noise/pink.mp3'))"
)
def test_soundtrack_loads_the_current_scale_asset(page):
# initial altitude is cosmos (index 0) → soundtrack resolves its scale asset,
# not the noise bed (the altitude-coupling itself is covered at the API tier)
page.select_option("#audio", "soundtrack")
def test_audio_toggle_plays_the_current_scale_soundtrack(page):
# Audio on (toggle) → the current altitude's soundtrack (cosmos at index 0) plays
page.click("label[for='audio'] .dev-switch-track")
page.wait_for_function(
"['audA','audB'].some(id => {"
" const s = document.getElementById(id).src;"
" return /\\/media\\/audio\\/.+\\.mp3/.test(s) && !s.includes('noise');"
" const a = document.getElementById(id);"
" return /\\/media\\/audio\\/.+\\.mp3/.test(a.src) && !a.paused;"
"})"
)
def test_visual_off_keeps_audio_and_fades_video(page):
page.select_option("#audio", "white_noise")
def test_video_toggle_off_blanks_the_screen_and_hides_layers(page):
page.click("label[for='visual'] .dev-switch-track")
page.wait_for_selector("#black:not(.hidden)") # black cover shown
# the video layers themselves are hidden too (GPU-composite-proof blanking);
# wait for the opacity transition to settle to 0
page.wait_for_function(
"['audA','audB'].some(id => "
"document.getElementById(id).src.includes('/media/audio/noise/pink.mp3'))"
"['vid','paint'].every(id => getComputedStyle(document.getElementById(id)).opacity === '0')"
)
page.uncheck("#visual")
page.wait_for_selector("#black:not(.hidden)") # video faded to black
def test_audio_survives_video_off(page):
page.click("label[for='audio'] .dev-switch-track")
page.wait_for_function(
"['audA','audB'].some(id => !document.getElementById(id).paused "
"&& document.getElementById(id).src)"
)
page.click("label[for='visual'] .dev-switch-track")
page.wait_for_selector("#black:not(.hidden)")
playing = page.evaluate(
"['audA','audB'].some(id => "
"{const a = document.getElementById(id); return !a.paused && !!a.src;})"
)
assert playing is True # audio survives the video fade
assert playing is True # audio keeps playing with video off
+12 -16
View File
@@ -10,9 +10,6 @@ from player.audio import (
resolve_visual,
)
NOISE = "/media/audio/noise/pink.mp3"
def test_visual_on_off():
assert resolve_visual("on") is True
assert resolve_visual("off") is False
@@ -27,37 +24,36 @@ def test_visual_positions_are_on_off():
assert VISUAL_POSITIONS == frozenset({"on", "off"})
def test_audio_sources_are_off_and_soundtrack():
assert AUDIO_SOURCES == frozenset({"off", "soundtrack"})
def test_soundtrack_resolves_to_the_current_scales_asset():
url = resolve_audio_url("soundtrack", scale_audio="coast/waves.loop.mp3", noise_url=NOISE)
url = resolve_audio_url("soundtrack", scale_audio="coast/waves.loop.mp3")
assert url == "/media/audio/coast/waves.loop.mp3"
def test_white_noise_resolves_to_the_global_bed_independent_of_altitude():
url = resolve_audio_url("white_noise", scale_audio="coast/waves.loop.mp3", noise_url=NOISE)
assert url == NOISE
def test_off_resolves_to_silence():
assert resolve_audio_url("off", scale_audio="coast/waves.loop.mp3", noise_url=NOISE) is None
assert resolve_audio_url("off", scale_audio="coast/waves.loop.mp3") is None
def test_soundtrack_with_no_scale_asset_is_silent():
assert resolve_audio_url("soundtrack", scale_audio="", noise_url=NOISE) is None
assert resolve_audio_url("soundtrack", scale_audio="") is None
def test_resolve_audio_marks_only_soundtrack_altitude_coupled():
r = resolve_audio("soundtrack", scale_audio="reef/soundscape.loop.mp3", noise_url=NOISE)
r = resolve_audio("soundtrack", scale_audio="reef/soundscape.loop.mp3")
assert isinstance(r, AudioResolution)
assert r.source == "soundtrack" and r.altitude_coupled is True
assert r.url == "/media/audio/reef/soundscape.loop.mp3"
assert resolve_audio("white_noise", scale_audio="x/y.mp3", noise_url=NOISE).altitude_coupled is False
assert resolve_audio("off", scale_audio="x/y.mp3", noise_url=NOISE).altitude_coupled is False
assert resolve_audio("off", scale_audio="x/y.mp3").altitude_coupled is False
def test_resolve_audio_url_rejects_unknown_source():
with pytest.raises(ValueError):
resolve_audio_url("podcast", scale_audio="", noise_url=NOISE)
resolve_audio_url("white_noise", scale_audio="") # deferred, not a v1 source
def test_music_is_not_a_v1_source():
def test_music_and_white_noise_are_not_v1_sources():
assert "music" not in AUDIO_SOURCES
assert "white_noise" not in AUDIO_SOURCES
+3 -3
View File
@@ -11,7 +11,7 @@ from player.controls import (
def test_visual_and_audio_enums_are_the_v1_positions():
assert VISUAL_POSITIONS == frozenset({"on", "off"})
assert AUDIO_SOURCES == frozenset({"off", "soundtrack", "white_noise"})
assert AUDIO_SOURCES == frozenset({"off", "soundtrack"})
def test_valid_controls_pass_validation():
@@ -47,10 +47,10 @@ def test_out_of_range_or_non_int_knob_rejected(field, bad):
def test_parse_controls_from_mapping():
c = parse_controls(
{"visual": "on", "audio": "white_noise", "left": 1, "right": 2, "dark": 3,
{"visual": "on", "audio": "soundtrack", "left": 1, "right": 2, "dark": 3,
"light": 0, "volume": 2, "brightness": 1}
)
assert c == Controls("on", "white_noise", 1, 2, 3, 0, 2, 1)
assert c == Controls("on", "soundtrack", 1, 2, 3, 0, 2, 1)
def test_parse_controls_rejects_unknown_keys():
+4 -4
View File
@@ -38,11 +38,11 @@ def test_visual_off_from_video_fades_to_black():
def test_audio_survives_visual_off():
# white noise + black is now reachable (the gap the bundled dial omitted)
# soundtrack + black is reachable (a gap the bundled dial omitted)
p = Player(LIB)
t = p.update(_controls(visual="off", audio="white_noise"))
t = p.update(_controls(visual="off", audio="soundtrack"))
assert t.playback.video is False
assert t.playback.audio_source == "white_noise"
assert t.playback.audio_source == "soundtrack"
def test_no_change_yields_none_transition():
@@ -90,7 +90,7 @@ def test_volume_only_change_is_a_live_update():
def test_audio_source_change_while_black_is_a_live_update():
p = Player(LIB)
p.update(_controls(visual="off", audio="white_noise"))
p.update(_controls(visual="off", audio="off"))
t = p.update(_controls(visual="off", audio="soundtrack"))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.audio_source == "soundtrack"
+5 -7
View File
@@ -56,13 +56,11 @@ def test_alteration_visual_off_hides_video(client):
assert body["render"]["video"]["shown"] is False
def test_alteration_white_noise_is_altitude_independent(ring_client):
for idx in (0, 1, 2):
body = ring_client.post("/api/alteration", json={
"controls": _controls(audio="white_noise"), "altitude_index": idx,
}).json()
assert body["render"]["audio"]["url"] == "/media/audio/noise/pink.mp3"
assert body["render"]["audio"]["altitude_coupled"] is False
def test_alteration_off_is_silent(ring_client):
body = ring_client.post("/api/alteration", json={
"controls": _controls(audio="off"), "altitude_index": 0,
}).json()
assert body["render"]["audio"] == {"source": "off", "url": None, "altitude_coupled": False}
def test_alteration_soundtrack_couples_to_the_given_altitude(ring_client):