Compare commits
22 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 BenStullsBets
|
2722949805 | ||
|
BenStullsBets
|
aa76cd1c9f | ||
|
BenStullsBets
|
c07067492b | ||
|
BenStullsBets
|
fd0aad6610 | ||
|
BenStullsBets
|
68f9d7342a | ||
|
BenStullsBets
|
29c6d822bd | ||
|
BenStullsBets
|
1f9b2a0bec | ||
|
BenStullsBets
|
39fb9108db | ||
|
BenStullsBets
|
062a283de4 | ||
|
BenStullsBets
|
adc39fe648 | ||
|
BenStullsBets
|
99a751cb46 | ||
|
BenStullsBets
|
6f2d1c4f7b | ||
|
BenStullsBets
|
a3ec09af7e | ||
|
BenStullsBets
|
e0b29c032c | ||
|
BenStullsBets
|
1b96e7065b | ||
|
BenStullsBets
|
f28a06cc7f | ||
 benstull
|
229af60f1f | ||
|
BenStullsBets
|
c389e2a834 | ||
|
BenStullsBets
|
32e74ca446 | ||
|
BenStullsBets
|
e3a9b7cff6 | ||
|
BenStullsBets
|
7a2f37af7e | ||
|
benstull
|
d0ad9a9ac9 |
+11
-1
@@ -1,5 +1,6 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.DS_Store
|
||||
.pytest_cache/
|
||||
.venv/
|
||||
media/
|
||||
@@ -8,5 +9,14 @@ media/
|
||||
# Simulator sample media (look-tuning only; populate via setup_sample_media.py
|
||||
# / build_pool_manifest.py). All scale + transition binaries are gitignored.
|
||||
simulator/sample_media/**/*.mp4
|
||||
# Local-only candidate review gallery (re-buildable; not shipped content).
|
||||
# Audio candidate clips (re-sourceable via docs/audio-candidate-pool.md).
|
||||
simulator/sample_media/**/*.wav
|
||||
simulator/sample_media/**/*.mp3
|
||||
simulator/sample_media/**/*.ogg
|
||||
simulator/sample_media/**/*.m4a
|
||||
# Local-only candidate review galleries (re-buildable; not shipped content).
|
||||
simulator/static/review.html
|
||||
simulator/static/review_audio.html
|
||||
|
||||
# Editor annotation/comment-thread metadata (not project content)
|
||||
.threads/
|
||||
|
||||
@@ -1,7 +1,11 @@
|
||||
.PHONY: sim sim-local
|
||||
|
||||
# Prefer the project venv's interpreter (this box has no bare `python` on PATH),
|
||||
# fall back to python3, then python.
|
||||
PYTHON ?= $(firstword $(wildcard .venv/bin/python) python3 python)
|
||||
|
||||
sim:
|
||||
docker compose -f simulator/docker-compose.yml up --build
|
||||
|
||||
sim-local:
|
||||
python -m uvicorn simulator.app:app --reload --port 8000
|
||||
$(PYTHON) -m uvicorn simulator.app:app --reload --port 8000
|
||||
|
||||
+16
-1
@@ -196,8 +196,23 @@ every transition, so fast navigation stays responsive on placeholder media.
|
||||
browser. Real strict-PD bases for all five scales sourced per
|
||||
[`docs/content-sourcing.md`](./content-sourcing.md).
|
||||
|
||||
- **Audio + separated Visual/Audio controls (session 0020) ✅ Done.** The
|
||||
simulator now has **sound**: per-altitude soundtracks that follow the Altitude
|
||||
dial + a synthesized pink **white-noise** bed, behind two orthogonal controls —
|
||||
**Visual** (on/off) × **Audio** (off / soundtrack / white_noise) — replacing
|
||||
the bundled 7-way Content dial. `tools/pipeline/audio_ops` + `audio_run` +
|
||||
`simulator/build_audio_media.py` (ffmpeg loop/loudnorm/noise, gitignored
|
||||
assets), `Scale.audio` in the manifest, a pure `player/audio.py` resolver
|
||||
(retiring `player/content.py`), server-resolved `render.audio.url`, and a client
|
||||
A/B `<audio>` crossfade layer. This is the "Content step" of the future control
|
||||
surface (it reframes the proposed networked-control-surface spec §3/§5). Spec:
|
||||
[`2026-06-26-audio-video-separated-controls-design.md`](./superpowers/specs/2026-06-26-audio-video-separated-controls-design.md);
|
||||
plan:
|
||||
[`2026-06-26-audio-video-separated-controls.md`](./superpowers/plans/2026-06-26-audio-video-separated-controls.md).
|
||||
|
||||
- **Still deferred:** real **i2v ring transitions** (need a generative-video
|
||||
model + adjacent real bases); Pi renderer + serial/firmware.
|
||||
model + adjacent real bases); Pi renderer + serial/firmware; the **music**
|
||||
audio layer (reserved dial position, no assets yet).
|
||||
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
|
||||
variant" flag (sub-project 2 territory, design §13).
|
||||
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
# Audio candidate pool — rotating pools per altitude (2026-06-26)
|
||||
|
||||
Candidate soundtrack/ambience for the **"audio + video" setting**. Following operator
|
||||
feedback, each altitude now has a **rotating pool** (mirrors the video `Scale.pool`
|
||||
model — random pick per landing, or pairable per-clip).
|
||||
|
||||
## Two audio layers per altitude (operator, 2026-06-26)
|
||||
|
||||
The "audio + video" setting layers **two** kinds of audio per altitude:
|
||||
1. **Ambience / audio track** — the soundscape (waves, reef crackle, chorus, etc.).
|
||||
**This is what we're sourcing now.**
|
||||
2. **Music track** — a separate composed musical layer. **Deferred** until the
|
||||
ambience tracks are settled across all altitudes.
|
||||
|
||||
Pools below are the **ambience** layer unless marked `music`. **Distinction (operator):**
|
||||
the ambience/soundtrack layer is an *environmental, real-world soundscape* (waves, reef,
|
||||
whale, room-tone hum) — **not composed music**. A musical pad/track, however nice, belongs
|
||||
in the music layer, even for "atmospheric" altitudes like cosmos/orbit.
|
||||
|
||||
**Media is gitignored** (`simulator/sample_media/**/*.{wav,mp3,…}`); this file is the
|
||||
re-sourceable record. Files: `simulator/sample_media/audio/<altitude>/<name>.<ext>`,
|
||||
served at `/media/audio/<altitude>/<name>.<ext>`. Review gallery (gitignored):
|
||||
`simulator/static/review_audio.html` → `/review_audio.html` with the sim running.
|
||||
|
||||
## Round-2 feedback applied (operator, session 2026-06-26)
|
||||
|
||||
- **abyss** ✓ kept (blue-whale moan) · **coast** ✓ kept (waves + terns)
|
||||
- **cosmos** ✗ plasma static "hurt my ears" → replaced with 3 NASA **sonifications**
|
||||
- **orbit** ✗ whistlers "sound like lasers" → replaced with gentle Earth **chorus** (+ wind)
|
||||
- **reef** ✗ "shouldn't have whales (too deep for a reef)" → whale removed; now crackle + diver
|
||||
- Operator: *"try all three… rotation of audio like the video, or per-video audio track."*
|
||||
|
||||
## Design arc
|
||||
|
||||
Descending the dial is one journey: **cosmos & abyss** = vast/dark/sparse siblings;
|
||||
**coast** = warm human middle; **orbit & reef** = transitional membranes. The science
|
||||
recordings (NASA/NOAA) keep the "machines reshaping perception" thesis literal.
|
||||
|
||||
## Pools
|
||||
|
||||
| altitude | id | clip | source | dur | license |
|
||||
|---|---|---|---|---|---|
|
||||
| 🌌 cosmos | `cosmos/pillars` | **✓ KEEP** — Pillars of Creation (M16), twinkly sweep | NASA/CXC/SAO — https://chandra.si.edu/sound/m16.html (`sounds/m16_all.mp4`) | 30s | PD (NASA, credit) |
|
||||
| 🌌 cosmos | `cosmos/music` | 🎵 **music track — TBD** (deferred). Black-hole drone DROPPED (too creepy, not awe-inspiring); Galactic Center dropped earlier | — | — | — |
|
||||
| 🛰 orbit | `orbit/spaceamb` | **✓ KEEP** — station room-tone hum ("inside the orbiting craft") | Sonicfreak — https://freesound.org/s/174450/ (`cdn.freesound.org/previews/174/174450_746632-hq.mp3`) | 2:31 | CC0 |
|
||||
| 🛰 (music) | _deferred_ | 🎵 "Frozen Star" warm pad — moved to MUSIC layer (it's music, not ambience) | Kevin MacLeod / incompetech — https://incompetech.com/music/royalty-free/mp3-royaltyfree/Frozen%20Star.mp3 | 3:41 | CC-BY |
|
||||
| 🐠 reef | `reef/soundscape` | **✓** richer reef (round 4) — 6-layer bake: submerged underwater bed (bubbles/sloosh) + shrimp crackle + fish chorus + toadfish + grouper + damselfish | NOAA SanctSound (FK+GR `*_snappingshrimp_*`, `FK02_01_fishchorus`, `GR01_01_toadfish`, `FK03_01_redgrouper`, `PM05_01_damselfish`) + DCSFX "Underwater [Loop] AMB" https://freesound.org/s/366159/ (`cdn.freesound.org/previews/366/366159_6725579-hq.mp3`); loop-to-60 + amix + dynaudnorm | 60s | PD (NOAA) + CC0 (underwater) |
|
||||
| 🏖 coast | `coast/waves` | Sea waves + tern calls | BigSoundBank #0267 — https://bigsoundbank.com/sea-waves-and-seagulls-s0267.html (`/UPLOAD/mp3/0267.mp3`) | 57s | CC0 |
|
||||
| 🕳 abyss | `abyss/whale` | NE Pacific blue-whale "AB call" | NOAA PMEL — https://www.pmel.noaa.gov/acoustics/multimedia/NETS_bluWhale.wav | 13s | PD (NOAA) |
|
||||
|
||||
## Sourcing rule (learned 2026-06-26)
|
||||
|
||||
**Source only from a browsable page the operator can open and whose license is stated.**
|
||||
The Mixkit clips were pulled by raw CDN asset id (`assets.mixkit.co/.../<id>-preview.mp3`)
|
||||
with no findable catalog page — operator couldn't see them, so they were removed. Good
|
||||
hubs: NASA SVS / Chandra, NOAA PMEL/SanctSound (PD), BigSoundBank (CC0), Pixabay
|
||||
(visible license). Avoid asset-id grabs and ND/NC licenses.
|
||||
|
||||
## Open items / next pass
|
||||
|
||||
- **orbit** — ✅ SETTLED: `orbit/spaceamb` (Sonicfreak station room-tone hum, CC0). Rejected
|
||||
en route: laser whistlers, Earth chorus, plain wind, "Frozen Star" pad (music). A real NASA
|
||||
ISS recording (Hadfield "Space Station Noise", SoundCloud) remains the authentic upgrade if
|
||||
ever wanted — needs a harder fetch (no yt-dlp; brew install denied).
|
||||
- **ambience layer COMPLETE** — all 5 picked: cosmos/pillars · orbit/spaceamb · coast/waves ·
|
||||
reef/soundscape · abyss/whale. Next is the production pass + wiring (below) and the music layer.
|
||||
- **reef** — ✅ round-4 6-layer bake (`reef/soundscape`): NOAA PD biological layers + a CC0
|
||||
submerged underwater "you're under" bed (DCSFX freesound #366159, hq preview). License clean.
|
||||
SanctSound clips: `files/SanctSound_*.mp4` at `https://sanctsound.ioos.us/files/` (extract audio).
|
||||
- **music layer** — separate composed music track per altitude, deferred (cosmos slot reserved).
|
||||
- **Production (after picks):** loop seams, length normalization, cross-altitude crossfade
|
||||
(mirror video); cosmos/abyss clips are short → loop.
|
||||
|
||||
## Re-download / re-bake
|
||||
|
||||
```
|
||||
A=simulator/sample_media/audio; FF=$(python -c "import imageio_ffmpeg;print(imageio_ffmpeg.get_ffmpeg_exe())")
|
||||
mkdir -p $A/cosmos $A/orbit $A/reef $A/coast $A/abyss
|
||||
# NASA sonifications (mp4 -> mp3)
|
||||
for u in perseus_sonification:blackhole m16_all:pillars galactic_all:galaxy; do
|
||||
f=${u%%:*}; o=${u##*:}; curl -fsSL "https://chandra.si.edu/sound/sounds/$f.mp4" -o /tmp/$o.mp4
|
||||
"$FF" -y -i /tmp/$o.mp4 -vn -q:a 4 $A/cosmos/$o.mp3; done
|
||||
curl -fsSL "https://svs.gsfc.nasa.gov/vis/a010000/a011000/a011073/Earthsong-540-MASTER_high.mp4" -o /tmp/es.mp4
|
||||
"$FF" -y -i /tmp/es.mp4 -vn -q:a 4 $A/orbit/earthsong.mp3
|
||||
curl -fsSL "https://assets.mixkit.co/active_storage/sfx/1162/1162-preview.mp3" -o $A/orbit/wind.mp3
|
||||
curl -fsSL "https://www.nhm.ac.uk/content/dam/nhm-www/discover/audio/sound-of-coral-reef.mp3" -o $A/reef/crackle.mp3
|
||||
curl -fsSL "https://assets.mixkit.co/active_storage/sfx/1242/1242-preview.mp3" -o $A/reef/scuba.mp3
|
||||
curl -fsSL "https://bigsoundbank.com/UPLOAD/mp3/0267.mp3" -o $A/coast/waves.mp3
|
||||
curl -fsSL "https://www.pmel.noaa.gov/acoustics/multimedia/NETS_bluWhale.wav" -o $A/abyss/whale.wav
|
||||
# layered reef
|
||||
"$FF" -y -i $A/reef/crackle.mp3 -i $A/reef/scuba.mp3 \
|
||||
-filter_complex "[1:a]volume=0.55[s];[0:a][s]amix=inputs=2:duration=first,dynaudnorm" -q:a 4 $A/reef/layered.mp3
|
||||
```
|
||||
@@ -23,11 +23,12 @@ a credit line** (our overlays + dream are derivatives, which these allow);
|
||||
**CC‑BY‑ND is NOT usable** (no derivatives); **CC‑BY‑NC only if the installation
|
||||
stays non‑commercial**.
|
||||
|
||||
## 🌌 cosmos (pool of 4)
|
||||
## 🌌 cosmos (pool of 5)
|
||||
|
||||
| id | clip | source | window (s) | license / credit |
|
||||
|----|------|--------|-----------|------------------|
|
||||
| `cosmos` | Cosmic Cliffs — Carina Nebula flythrough | NASA SVS 31348 — https://svs.gsfc.nasa.gov/31348/ (`Clifs-3d-STScI.mp4`, *Exploring the Cosmic Cliffs in 3D*, Webb NIRCam, **4K**) | 10–34 | CC‑BY‑class — credit **NASA, ESA, CSA, STScI** |
|
||||
| `cosmos_miri` | Cosmic Cliffs — Carina Nebula, **mid-infrared** (same towers, muted gray/teal/gold dust palette) | ESA/Webb weic2205c — https://esawebb.org/videos/weic2205c/ (*Pan of the Carina Nebula, combined NIRCam + MIRI*, **4K** `weic2205c.mp4`) | 4–26 | CC‑BY‑class — credit **NASA, ESA, CSA, STScI** |
|
||||
|
||||
> ℹ️ **Cosmos primary swapped to the Webb "Cosmic Cliffs" flythrough (2026-06-25, session 0018).**
|
||||
> History: the original primary was NASA/JPL's *"Orion: Dust and Death"* (captions
|
||||
@@ -43,6 +44,17 @@ stays non‑commercial**.
|
||||
| `cosmos_hudf` | Hubble Ultra Deep Field zoom | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030600/a030687/hudf-b-1920x1080p30.mov | 20–44 | CC‑BY‑class — credit **NASA, ESA, STScI** |
|
||||
| `cosmos_xdf` | eXtreme Deep Field flythrough | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030600/a030681/hxdf_fly-b-1920x1080p30.mov | 2–26 | CC‑BY‑class — credit **NASA, ESA, STScI** |
|
||||
|
||||
> ➕ **`cosmos_miri` added (2026-06-26).** Operator wanted more *Cosmic Cliffs*
|
||||
> footage and a different-color rendering. The keep-the-flythrough-primary +
|
||||
> add-the-mid-infrared-pan pair: ESA/Webb's **weic2205c** is the *same* Carina
|
||||
> towers shot with NIRCam **+ MIRI**, so it renders in a muted gray/teal/gold dust
|
||||
> palette instead of the iconic orange/blue — a distinct look from the same scene.
|
||||
> Sourced text-free/human-free from the **4K** master, trim 4–26s (clears the
|
||||
> opening fade-from-black, ends before any tail fade) → 1080p base + 4K master,
|
||||
> 1 s crossfade-loop (~21 s). ESA/Webb's narration-free *"Diving Into the Cosmic
|
||||
> Cliffs"* pan (`carina_revisit_pan`, NIRCam palette) was the runner-up — left
|
||||
> out for now since it repeats the existing palette; easy to add if wanted.
|
||||
|
||||
> ⚠️ **DISCREPANCY to fix next session.** Operator selected cosmos **#1, #3
|
||||
> (Orion), #4, #5**. During processing, **#6 Galaxy Traverse was run by mistake
|
||||
> instead of #3 Orion**. The cosmos primary now lives at
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,264 @@
|
||||
# Audio + Video — separated Visual & Audio controls — design
|
||||
|
||||
**Status:** proposed → built (sessions 0020 build + 0021 revision)
|
||||
**Date:** 2026-06-26
|
||||
**Session:** 0020 (audio) · 0021 (live-UI revision)
|
||||
|
||||
> **v1 build revision (session 0021, operator-driven):** the live **Audio** control
|
||||
> ships as a simple **off / soundtrack** toggle (styled like the Dev Mode switch),
|
||||
> and **Video** is the matching on/off toggle. **White-noise is deferred** with music
|
||||
> (the orthogonal Visual × Audio model and the synthesized-pink-noise machinery
|
||||
> remain in the codebase for when it's wanted). Two real-browser bugs were fixed in
|
||||
> this revision: video-off now blanks GPU-composited layers (z-index + explicit layer
|
||||
> hide), and the soundtrack now plays on Safari/iOS (audio elements unlocked
|
||||
> synchronously inside the start gesture). Sections below describe the fuller
|
||||
> off/soundtrack/white-noise design; v1 live = off/soundtrack.
|
||||
**Extends / reframes:** [`2026-06-26-networked-control-surface-design.md`](./2026-06-26-networked-control-surface-design.md)
|
||||
§3 (control inventory) and §5 (server contract) — its bundled **Content** selector
|
||||
is split into two orthogonal controls. Everything else in that spec (server-of-truth
|
||||
+ SSE, the two pages, the Arduino seam, transition lifecycle) stands unchanged.
|
||||
**Asset record:** [`docs/audio-candidate-pool.md`](../../audio-candidate-pool.md).
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Give the experience **sound**, and make audio and video **independently
|
||||
controllable**. Today the control surface bundles audio+video into a single 7-way
|
||||
**Content** selector (*video / audio+video / music+video / off / white noise /
|
||||
music / audio track*). That bundle is both awkward to operate and **incomplete** —
|
||||
it has no "white noise **with** video," for instance.
|
||||
|
||||
Replace it with **two orthogonal dials** — **Visual** and **Audio** — so every
|
||||
combination is reachable, including the ones the bundle omitted. Operator framing
|
||||
(2026-06-26): *"separate these dials… it's ok to let people have e.g. white noise
|
||||
with video."*
|
||||
|
||||
## 2. Core idea: orthogonalize Content into Visual × Audio
|
||||
|
||||
```
|
||||
BEFORE (bundled) AFTER (orthogonal)
|
||||
Content: 7-way selector Visual: on / off
|
||||
video Audio: off / soundtrack / white noise
|
||||
audio+video (· music — deferred)
|
||||
music+video ──▶ ───────────────────────────────────────
|
||||
off Any Visual × Any Audio is now reachable:
|
||||
white noise video + soundtrack (was "audio+video")
|
||||
music video + white noise (NEW — was missing)
|
||||
audio track black + white noise (was "white noise")
|
||||
black + soundtrack (was "audio track")
|
||||
```
|
||||
|
||||
The 7 bundled modes were really points in a **2 × N grid** (visual on/off ×
|
||||
audio source). Making the grid explicit removes the awkwardness and fills the gap.
|
||||
The **Altitude** dial is unchanged and remains the single scale selector.
|
||||
|
||||
## 3. The controls (reframes control-surface §3)
|
||||
|
||||
| Control | Positions (v1) | Drives | Class (§4 of control-surface spec) |
|
||||
|---|---|---|---|
|
||||
| **Altitude** | endless dial, cosmos→abyss (wraps) | the visual **scale** **and**, when Audio=soundtrack, the matching ambience | transitional |
|
||||
| **Visual** | on / off | video shown vs. black | transitional (video fade) |
|
||||
| **Audio** | off / soundtrack / white noise (· *music* deferred) | the audio **source** | live + short crossfade (§7) |
|
||||
| **Left** / **Right** / **Mood** | unchanged | the alteration engine | live |
|
||||
|
||||
So the panel grows from 5 controls to **6** (Content → Visual + Audio). The
|
||||
physical form (later): Visual = a toggle/2-pos switch; Audio = a small rotary
|
||||
selector (3-pos in v1, 4 with music).
|
||||
|
||||
## 4. Audio model (the coupling decision)
|
||||
|
||||
**Decision (operator):** the audio dial is a **source** selector; the per-altitude
|
||||
soundtrack **follows the single Altitude dial**. There is **no** separate audio
|
||||
altitude.
|
||||
|
||||
- **`soundtrack`** → plays the ambience of the **current Altitude scale**. Turning
|
||||
Altitude to coast shows coast video *and* plays coast waves; the soundtrack
|
||||
**crossfades** as the altitude transition settles (mirrors the video ring
|
||||
crossfade). One dial, two coupled outputs.
|
||||
- **`white noise`** → a single global bed, **altitude-independent**. Changing
|
||||
Altitude does not change it. This is what makes "white noise + any video" work.
|
||||
- **`off`** → silence. Video (if Visual=on) plays mute.
|
||||
- **`music`** *(deferred)* → the composed per-altitude music layer. Reserved as a
|
||||
future Audio position; **not in v1** (no assets yet — see
|
||||
[`docs/audio-candidate-pool.md`](../../audio-candidate-pool.md) "two audio layers").
|
||||
|
||||
Independence summary: **Visual ⟂ Audio** (any pairing allowed); **Audio source ⟂
|
||||
Altitude** for white noise/music, **coupled to Altitude** for soundtrack.
|
||||
|
||||
## 5. Assets & manifest
|
||||
|
||||
### 5.1 Per-scale soundtrack
|
||||
|
||||
The five ambiences are sourced and reviewed (PD/CC0; record in
|
||||
[`docs/audio-candidate-pool.md`](../../audio-candidate-pool.md), local-only review
|
||||
gallery `simulator/static/review_audio.html`):
|
||||
|
||||
| scale | soundtrack | license |
|
||||
|---|---|---|
|
||||
| cosmos | Pillars of Creation sonification | PD (NASA) |
|
||||
| orbit | station room-tone hum | CC0 |
|
||||
| coast | sea waves + terns | CC0 |
|
||||
| reef | 6-layer reef soundscape | PD (NOAA) + CC0 |
|
||||
| abyss | blue-whale moan | PD (NOAA) |
|
||||
|
||||
The manifest's `Scale` gains an **`audio`** field (path to the scale's soundtrack),
|
||||
parallel to how clips are referenced; media stays gitignored and is served at
|
||||
`/media/audio/<scale>/<file>`. Soundtracks are a **per-scale single** (not a
|
||||
rotating pool) in v1 — one bed per altitude. (A rotating audio pool, like the clip
|
||||
pool, is a possible later extension; out of v1.)
|
||||
|
||||
### 5.2 White noise
|
||||
|
||||
Altitude-independent global asset. **Synthesized, not sourced** — a calm
|
||||
**pink/brown** noise (gentler than pure white) generated deterministically with
|
||||
`ffmpeg -f lavfi -i anoisesrc=color=pink` → a clean ~60 s loop. Zero licensing
|
||||
concerns. (Colloquially "white noise"; spec a calm colored noise.)
|
||||
|
||||
### 5.3 Production pass (assets)
|
||||
|
||||
Before ship the source clips get a light, deterministic pass (the existing
|
||||
`tools/pipeline` ffmpeg approach): **seamless loop** (crossfade the loop point),
|
||||
**normalize loudness** across the five so altitudes match, and a consistent format
|
||||
(mp3/ogg). Several are short (abyss 13 s, cosmos 30 s) and must loop cleanly. The
|
||||
6-layer reef bake is already loop-length (60 s).
|
||||
|
||||
## 6. Server contract (reframes control-surface §5)
|
||||
|
||||
The control-surface spec's authoritative `SessionState` + four endpoints are
|
||||
unchanged in shape; only the **controls** payload changes: `content` is replaced
|
||||
by `visual` + `audio`, and the derived `render.content` is split.
|
||||
|
||||
### 6.1 `GET /api/state` — updated `controls` + `render`
|
||||
|
||||
```json
|
||||
{
|
||||
"seq": 42,
|
||||
"controls": { "visual": "on", "audio": "soundtrack", "left": 3, "right": 1, "mood": -2 },
|
||||
"altitude": { "index": 2, "scale": "coast", "clip_id": "coast_07" },
|
||||
"transitions": { "altitude": "idle", "visual": "idle" },
|
||||
"render": {
|
||||
"plan": { },
|
||||
"video": { "shown": true },
|
||||
"audio": { "source": "soundtrack", "url": "/media/audio/coast/waves.mp3", "altitude_coupled": true }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `audio.url` is resolved **server-side** from `(audio source, altitude)`:
|
||||
`soundtrack` → the current scale's `audio`; `white_noise` → the global noise
|
||||
asset; `off` → `null`. The renderer just plays `audio.url` (or nothing).
|
||||
- `transitions` tracks `visual` instead of `content` (Audio is not transitional —
|
||||
§7).
|
||||
|
||||
### 6.2 `POST /api/control`
|
||||
|
||||
```json
|
||||
{ "set": { "audio": "white_noise" } } // audio source change
|
||||
{ "set": { "visual": "off" } } // visual fade to black
|
||||
{ "set": {}, "altitude_delta": -1 } // altitude tick (also re-resolves audio.url when source=soundtrack)
|
||||
```
|
||||
|
||||
Server applies, bumps `seq`, recomputes `render.audio.url`, and broadcasts.
|
||||
Absolute `set` values keep retries idempotent (control-surface §5.6).
|
||||
|
||||
### 6.3 SSE events
|
||||
|
||||
- `state` — carries the new controls/render; emitted on Audio change and on any
|
||||
live change (Audio source change rides the `state` event — §7).
|
||||
- `ring` — altitude move; now **also** carries the new `audio.url` so the renderer
|
||||
can crossfade the soundtrack as it plays the visual transition (when
|
||||
source=soundtrack).
|
||||
- `mode` — repurposed to the **Visual** fade (on/off). (Same renderer fade machinery
|
||||
the bundled Content used.)
|
||||
|
||||
## 7. Live vs. transitional classification
|
||||
|
||||
- **Visual on/off → transitional.** It's a visible video fade (to/from black); it
|
||||
uses the renderer's existing fade + `settled` ack, exactly as the bundled
|
||||
Content's fade did. `transitions.visual` flips `transitioning`→`idle`.
|
||||
- **Audio source → live, with a short renderer-side crossfade.** Swapping
|
||||
soundtrack/white-noise/off is a ~0.5–1 s **gain crossfade** between two `<audio>`
|
||||
elements — perceptible but **non-blocking** and not a renderer animation with
|
||||
unknown timing. So it's classified **live** (no busy spinner, no `settled` ack),
|
||||
like Left/Right/Mood. *Flagged decision:* if the operator wants the Audio knob to
|
||||
feel "busy" during its fade, it can be promoted to transitional later; v1 keeps it
|
||||
live for simplicity.
|
||||
- **Altitude → transitional** (unchanged). When source=soundtrack, the renderer
|
||||
crossfades the audio bed in step with the ring transition; the audio crossfade is
|
||||
slaved to the visual transition and needs no separate ack.
|
||||
|
||||
## 8. Renderer behavior (extends control-surface §6.1)
|
||||
|
||||
The display page adds an **audio layer**:
|
||||
|
||||
- Two `<audio>` elements (A/B) for gapless gain crossfades; a tiny crossfade helper.
|
||||
- On `state`: reconcile `audio.url` — if it changed, crossfade A↔B to the new url
|
||||
(or fade to silence for `off`). White-noise and soundtrack are the same mechanism;
|
||||
only the url differs.
|
||||
- On `ring` (altitude move) with source=soundtrack: start the soundtrack crossfade
|
||||
to the event's new `audio.url` so it lands with the visual scale.
|
||||
- On `mode` (Visual): fade the **video** to/from black; audio is untouched (you can
|
||||
hear soundtrack/white-noise over black).
|
||||
- Each soundtrack loops (the production-pass clean loop); white-noise loops.
|
||||
- Autoplay: the renderer is launched full-screen by the operator; a one-time user
|
||||
gesture (or `--autoplay`/muted-then-unmute) satisfies browser autoplay policy.
|
||||
*Flagged:* document the launch gesture so the projection starts audio reliably.
|
||||
|
||||
## 9. Testing (mandatory pipeline tiers)
|
||||
|
||||
- **Pure unit** — `render.audio.url` resolution from `(source, altitude)`:
|
||||
soundtrack→scale asset, white_noise→global, off→null; altitude tick re-resolves
|
||||
only when source=soundtrack; `seq` bump. No I/O.
|
||||
- **API/contract** — `POST /api/control {visual}` flips `transitions.visual`;
|
||||
`{audio}` rides a `state` event with the new url and **no** transition flag; bad
|
||||
enum → 422; `GET /api/state` shape.
|
||||
- **E2E (Playwright)** — renderer + `/remote`: set Audio=white_noise → assert the
|
||||
renderer's audio element src is the noise asset; tick Altitude with
|
||||
Audio=soundtrack → assert the soundtrack url crossfades to the new scale; set
|
||||
Visual=off → assert the video fades and audio keeps playing. (Audio *playback*
|
||||
asserted via element `src`/`paused`, not waveform.)
|
||||
|
||||
## 10. Scope
|
||||
|
||||
**In v1:**
|
||||
|
||||
- Split **Content** → **Visual** (on/off) + **Audio** (off / soundtrack / white
|
||||
noise) across `SessionState`, `/api/state`, `/api/control`, SSE, and both pages.
|
||||
- Per-scale `audio` in the manifest + the 5 sourced soundtracks (production-passed
|
||||
loops); synthesized white-noise asset.
|
||||
- Server-side `audio.url` resolution; renderer audio layer with crossfades;
|
||||
altitude-coupled soundtrack crossfade.
|
||||
|
||||
**Explicitly NOT in v1 (deferred):**
|
||||
|
||||
- **Music layer** (`audio=music`) — reserved dial position; no assets yet.
|
||||
- **Separate audio altitude** (hear reef while seeing coast) — rejected coupling
|
||||
option; soundtrack follows the one Altitude dial.
|
||||
- **Rotating audio pools** per scale (one bed per altitude in v1).
|
||||
- Per-altitude white-noise variants (one global noise).
|
||||
- Physical Visual/Audio switches (Arduino) — contract only, per control-surface §7.
|
||||
|
||||
## 11. Spec & roadmap impact
|
||||
|
||||
- **Reframes** control-surface §3/§5: `content` → `visual` + `audio`;
|
||||
`transitions.content` → `transitions.visual`; `render.content` → `render.video`
|
||||
+ `render.audio`. That spec is still *proposed* (unbuilt), so this is a clean
|
||||
pre-implementation edit, not a migration. Both can be built together, or this
|
||||
one folds into the control-surface build as its Content step.
|
||||
- The **music layer** becomes a tracked follow-on (compose + source per-altitude
|
||||
music; flip on the reserved Audio position).
|
||||
- `ROADMAP.md` to note the audio capability at implementation time.
|
||||
|
||||
## 12. Open questions / flagged — RESOLVED (session 0020 build)
|
||||
|
||||
- **Audio knob feel** (§7) — **RESOLVED: live**, with a ~0.6 s renderer-side gain
|
||||
crossfade (`XFADE_MS=600`), no `settled` ack. Promote to transitional later only
|
||||
if the iPad makes it feel like it needs a "busy" state.
|
||||
- **White-noise color** — **RESOLVED: pink** (`anoisesrc=color=pink`), a balanced
|
||||
calm bed. Brown is a one-parameter alternative (`color=brown`) if a deeper rumble
|
||||
is wanted; the build is parameterized for it.
|
||||
- **Autoplay gesture** (§8) — **RESOLVED: a one-time tap-to-start overlay**
|
||||
("▶ tap to start sound", `#start-gesture`). The first click unlocks both `<audio>`
|
||||
elements within the user gesture; thereafter audio follows state. This is the
|
||||
documented launch step for the full-screen projection.
|
||||
@@ -0,0 +1,331 @@
|
||||
# Networked Control Surface — design
|
||||
|
||||
**Status:** proposed
|
||||
**Date:** 2026-06-26
|
||||
**Session:** 0019
|
||||
**Supersedes/reframes:** ROADMAP sub-project 4 (Arduino Firmware / control panel) and
|
||||
the "serial input" slice of sub-project 3 (Player Runtime).
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Split the existing simulator into a **renderer** (the laptop full-screen
|
||||
projection — "the experience") and a separate **controller** (a touch remote),
|
||||
talking over the local network, so the experience can be driven from a handheld
|
||||
device the way the real installation's panel drives its screen.
|
||||
|
||||
This is **v1 hardware**, deliberately simulator-friendly:
|
||||
|
||||
- The **laptop/computer renders** (no Raspberry Pi yet).
|
||||
- The controller is a **separate device** over **wifi** (LAN), not a USB-serial
|
||||
link to a Pi.
|
||||
- The controller is **prototyped as a web page** (an iPad/tablet/phone twin),
|
||||
and a **physical Arduino remote** drops in later as just another control source
|
||||
speaking the same contract.
|
||||
|
||||
Two goals, equally weighted (operator, session 0019): **decouple control from
|
||||
display cleanly**, *and* get the **handheld "stand back and drive it" feel**.
|
||||
|
||||
## 2. Core idea: one server of truth, interchangeable control sources
|
||||
|
||||
```
|
||||
┌──────────────────┐ POST /api/control ┌─────────────────────┐
|
||||
│ CONTROL SOURCE │ ───── (knob change) ────▶ │ FastAPI server │
|
||||
│ │ │ (laptop) │
|
||||
│ • iPad web remote│ ◀──── GET /api/state ──── │ ┌───────────────┐ │
|
||||
│ (v1) │ (sync on load) │ │ SessionState │ │
|
||||
│ • Arduino (later)│ │ │ controls + │ │
|
||||
└──────────────────┘ │ │ ring index + │ │
|
||||
│ │ transitions + │ │
|
||||
┌──────────────────┐ GET /api/events (SSE) │ │ seq (version) │ │
|
||||
│ RENDERER │ ◀═══ state/ring/mode ════ │ └───────┬───────┘ │
|
||||
│ laptop full- │ (live push) └──────────┼──────────┘
|
||||
│ screen │ │
|
||||
│ (the experience) │ ──── POST /api/render/event ────────▶│ (settled ack)
|
||||
└──────────────────┘ recompute RenderPlan + ring move
|
||||
```
|
||||
|
||||
- **Server** owns the single authoritative `SessionState` (current controls,
|
||||
altitude/ring index, per-field transition state, a monotonic `seq`). Every
|
||||
change recomputes the derived view and broadcasts it.
|
||||
- **Control source** = anything that `POST`s a control change. v1 has exactly
|
||||
one: the iPad web remote. A future Arduino is *another instance of the same
|
||||
role* — no new server concept.
|
||||
- **Renderer** = the laptop projection. It holds **no** input UI; it subscribes
|
||||
to the SSE stream and renders whatever state arrives. It is today's
|
||||
`simulator/static/index.html`, stripped of its sliders, given a `state`
|
||||
listener plus one small upstream ack channel.
|
||||
|
||||
The property that matters: controller and renderer are now **separate web pages
|
||||
off the same server**, decoupled exactly as a real installation's panel and
|
||||
screen are. (Chosen architecture: server-authoritative session + **Server-Sent
|
||||
Events** push. SSE fits because the directions are asymmetric — the renderer only
|
||||
*receives*, the control source only *sends* — so no bidirectional WebSocket is
|
||||
needed. Alternatives considered and rejected: renderer polling — laggy/wasteful;
|
||||
peer-to-peer WebRTC — most plumbing and it breaks the clean single-contract seam
|
||||
that makes the Arduino a drop-in.)
|
||||
|
||||
## 3. Control surface: the six controls (the faithful twin)
|
||||
|
||||
The iPad remote is a **faithful on-screen twin** of the intended physical panel —
|
||||
what you like on the iPad is the spec for the Arduino panel. The control
|
||||
inventory is exactly what the sim exposes today:
|
||||
|
||||
| Control | Sim today | Physical form (later) |
|
||||
|---|---|---|
|
||||
| **Visual** | toggle (show video on/off) | 2-position switch |
|
||||
| **Audio** | on/off toggle (on = per-altitude soundtrack; ·white-noise & music deferred) | 2-position switch (rotary selector when more sources land) |
|
||||
| **Altitude** | endless circular dial — walks the 5 scales (cosmos→abyss), wraps | rotary encoder (endless, detented) |
|
||||
| **Left** (analytical) | slider 0–4 | knob/pot, 5 detents |
|
||||
| **Right** (dreamlike) | slider 0–4 | knob/pot, 5 detents |
|
||||
| **Mood** | slider −4…+4 (dark↔light) | center-detented knob |
|
||||
|
||||
(This is the original sub-project-4 inventory, *updated*: Dark+Light collapsed
|
||||
into one **Mood** knob in session 0013, the **Altitude** encoder added in session
|
||||
0018, and the bundled 7-way **Content** dial split into orthogonal **Visual** ×
|
||||
**Audio** controls in session 0020 — see
|
||||
[`2026-06-26-audio-video-separated-controls-design.md`](./2026-06-26-audio-video-separated-controls-design.md).)
|
||||
|
||||
## 4. Live vs. transitional controls (per-knob transition state)
|
||||
|
||||
The five controls split into two classes by how a **human** perceives the change:
|
||||
|
||||
- **Live controls — Left, Right, Mood, Audio.** The alteration engine treats
|
||||
Left/Right/Mood as `LIVE_UPDATE` (the WebGL Kuwahara dream + HUD overlay + color
|
||||
grade re-tune within a frame). **Audio** source changes are also live: a short
|
||||
(~0.6 s) renderer-side gain crossfade between two `<audio>` elements —
|
||||
perceptible but non-blocking, with no `settled` ack (audio spec §7). These are
|
||||
*always idle*; the remote just shows the value.
|
||||
- **Transitional controls — Altitude, Visual.** These trigger a renderer
|
||||
animation that takes real, perceptible time: Altitude plays ring-transition
|
||||
clips (zoom/warp between scales); **Visual** on/off does fade-to-black /
|
||||
fade-from-black. During that window the control is **busy**, and the control
|
||||
source must model that so its feedback is honest.
|
||||
|
||||
Per-transitional-control lifecycle:
|
||||
|
||||
```
|
||||
idle ──(source sends)──▶ pending ──(renderer starts)──▶ transitioning ──(renderer settles)──▶ idle
|
||||
```
|
||||
|
||||
- **pending** — set locally by the remote the instant the knob is turned
|
||||
(immediate tactile feedback before the server replies).
|
||||
- **transitioning** — the server flags the field once the renderer actually
|
||||
begins the animation; broadcast to *all* control sources.
|
||||
- **idle** — cleared when the **renderer acks completion** (the only party that
|
||||
knows real playback timing), with a server-side max-duration timeout as a
|
||||
safety net so a dropped renderer can't wedge a knob.
|
||||
|
||||
Hardware payoff: a physical Altitude encoder can light an LED while
|
||||
`transitioning`; the iPad twin shows the matching spinner/disabled-detent.
|
||||
|
||||
`SessionState` therefore carries
|
||||
`transitions: { altitude: idle|transitioning, visual: idle|transitioning }`
|
||||
alongside the controls and `seq`. (Live controls — Left/Right/Mood/Audio — need
|
||||
no entry.)
|
||||
|
||||
## 5. Server contract
|
||||
|
||||
Four endpoints carry the whole thing. The existing `/api/alteration`,
|
||||
`/api/ring`, `/api/ring/advance` remain as **internal helpers the server calls**;
|
||||
control sources never hit them directly.
|
||||
|
||||
### 5.1 `GET /api/state` — snapshot (sync-on-load / polling fallback)
|
||||
|
||||
```json
|
||||
{
|
||||
"seq": 42,
|
||||
"controls": { "visual": "on", "audio": "soundtrack", "left": 3, "right": 1, "mood": -2 },
|
||||
"altitude": { "index": 2, "scale": "coast", "clip_id": "coast_07" },
|
||||
"transitions": { "altitude": "idle", "visual": "idle" },
|
||||
"render": {
|
||||
"plan": { },
|
||||
"video": { "shown": true },
|
||||
"audio": { "source": "soundtrack", "url": "/media/audio/coast/waves.loop.mp3", "altitude_coupled": true }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(The `render.video` + `render.audio` split and server-side `audio.url` resolution
|
||||
are specified in the audio spec §6.1.)
|
||||
|
||||
A control source GETs this on load to draw its knobs in the right positions; the
|
||||
renderer can use it to cold-start or resync after a reconnect.
|
||||
|
||||
### 5.2 `POST /api/control` — the one control-source contract
|
||||
|
||||
A partial patch — only the fields that changed:
|
||||
|
||||
```json
|
||||
{ "set": { "left": 4 }, "altitude_delta": 0 } // live knob
|
||||
{ "set": { "audio": "white_noise" }, "altitude_delta": 0 } // live (audio source)
|
||||
{ "set": { "visual": "off" }, "altitude_delta": 0 } // transitional (video fade)
|
||||
{ "set": {}, "altitude_delta": -1 } // one encoder tick down
|
||||
```
|
||||
|
||||
Server applies it, bumps `seq`, recomputes, and:
|
||||
|
||||
- **live field** (Left/Right/Mood, **or an `audio` source change**) → broadcasts
|
||||
a `state` event immediately (the renderer crossfades audio on its own, §7);
|
||||
- **transitional field** (a `visual` change, or a non-zero `altitude_delta`) →
|
||||
computes the ring move / video fade, marks that field `transitioning`,
|
||||
broadcasts a `ring` (or `mode`) event for the renderer to animate.
|
||||
|
||||
Returns the new snapshot (an immediate authoritative echo for the poster).
|
||||
Absolute `set` values make retries idempotent; `altitude_delta` is relative
|
||||
(encoder ticks) and the server coalesces rapid ticks via the existing
|
||||
`advance_ring` fast-spin logic.
|
||||
|
||||
### 5.3 `GET /api/events` — SSE stream (renderer subscribes)
|
||||
|
||||
Event types:
|
||||
|
||||
- `state` — full snapshot; sent on connect and on every live change (including an
|
||||
**audio** source change, which the renderer crossfades — §7).
|
||||
- `ring` — an altitude move: the transition clip(s) to play + the landed
|
||||
scale/clip, **plus the new `audio.url`** so the renderer can crossfade the
|
||||
soundtrack in step (when `audio=soundtrack`). Renderer plays them, then settles.
|
||||
- `mode` — a **Visual** on/off change: the video fade the renderer should run.
|
||||
- `ping` — keepalive.
|
||||
|
||||
The SSE event payloads reuse the existing serializers
|
||||
(`ring_move_to_dict`, `render_plan_to_dict`, etc.).
|
||||
|
||||
### 5.4 `POST /api/render/event` — the renderer's one upstream ack
|
||||
|
||||
```json
|
||||
{ "type": "settled", "seq": 42, "field": "altitude" }
|
||||
```
|
||||
|
||||
Sent when the renderer finishes a transition. Server flips that field `idle` and
|
||||
broadcasts a `state` so every remote clears its spinner. A server-side timeout
|
||||
(max transition duration + margin) clears it anyway if the renderer never acks.
|
||||
|
||||
### 5.5 End-to-end flow for one altitude tick
|
||||
|
||||
```
|
||||
iPad: turn dial → field=pending locally → POST /api/control {altitude_delta:-1}
|
||||
server: apply, seq++, transitions.altitude="transitioning"
|
||||
→ SSE "ring" to renderer → SSE "state" to all remotes (altitude busy)
|
||||
renderer: play transition clip(s), settle on forest_07
|
||||
→ POST /api/render/event {type:"settled", field:"altitude"}
|
||||
server: transitions.altitude="idle" → SSE "state" to all → iPad clears spinner
|
||||
```
|
||||
|
||||
### 5.6 Decisions flagged (not assumed silently)
|
||||
|
||||
- **Concurrency = last-write-wins.** Multiple control sources are allowed;
|
||||
per-field last write wins, `seq` orders everything. No locking in v1.
|
||||
- **No auth.** v1 trusts the LAN — any device that can reach the laptop can drive
|
||||
it. A room-token is deferred until wanted.
|
||||
|
||||
## 6. The two pages
|
||||
|
||||
### 6.1 Renderer — `/` (display), refactored from today's `index.html`
|
||||
|
||||
- **Removes** the input controls (the Visual toggle, the Audio `<select>`, the
|
||||
Altitude SVG, the three sliders) and their handlers.
|
||||
- **Keeps** everything that draws the experience: the video element, the **A/B
|
||||
`<audio>` crossfade layer**, the WebGL Kuwahara dream shader, the
|
||||
HUD/annotation + affect overlay, the ring-transition playback, the media
|
||||
preload pool, the crossfade-loop logic.
|
||||
- **Adds** a thin `state` client: open `EventSource('/api/events')`; on `state`
|
||||
apply controls→render live (incl. the audio crossfade); on `ring` play the
|
||||
transition clip(s) — crossfading the soundtrack to the event's `audio.url` —
|
||||
then settle and `POST /api/render/event {settled}`; on `mode` run the video
|
||||
fade. On (re)connect,
|
||||
pull `GET /api/state` to resync. SSE auto-reconnects, so a flaky link
|
||||
self-heals.
|
||||
- Runs full-screen on the laptop driving the projector; no keyboard/mouse needed
|
||||
once launched.
|
||||
|
||||
### 6.2 Controller — `/remote`, a new touch-first page (the faithful twin)
|
||||
|
||||
- The six controls laid out like the intended physical panel: **Visual** toggle +
|
||||
**Audio** selector, **Altitude** encoder dial (reuse the existing SVG dial
|
||||
component), **Left** / **Right** / **Mood** knobs.
|
||||
- On any input → `POST /api/control` with just the changed field. Live knobs post
|
||||
on drag; Altitude posts encoder ticks; Visual/Audio post on change.
|
||||
- Per-knob transition state (§4): a transitional control goes **pending** on
|
||||
touch, reflects **transitioning** from the SSE feed, clears on **settled**.
|
||||
Altitude shows a settling indicator and its detents soft-lock (further ticks
|
||||
queue as the next target rather than stacking); **Visual** shows the toggle as
|
||||
pending until the video fade completes; **Audio** is live (no pending state).
|
||||
- It **also** subscribes to `/api/events` so it stays in sync with other
|
||||
sources (turn the future Arduino's knob and the iPad reflects it, and
|
||||
vice-versa). On load it GETs `/api/state` to draw current positions.
|
||||
- Touch-sized hit targets, no hover dependence, works in mobile Safari with no
|
||||
install.
|
||||
|
||||
## 7. The future-Arduino seam (contract only in v1)
|
||||
|
||||
No firmware in this feature, but the design **proves the seam** so sub-project 4
|
||||
is a true drop-in:
|
||||
|
||||
- **ESP32 (wifi Arduino)** → posts straight to `POST /api/control` over the LAN.
|
||||
Zero server change. Recommended hardware target when the time comes.
|
||||
- **Classic USB Arduino** → emits serial frames to a tiny `bridge.py` on the
|
||||
laptop that translates each frame into a `POST /api/control`. The old roadmap
|
||||
"3⇄4 serial framing contract" shrinks to *just* the Arduino↔bridge link; the
|
||||
renderer/server never know the difference.
|
||||
- Either way the Arduino sends **absolute pot values** (`set`) + **encoder
|
||||
ticks** (`altitude_delta`) — exactly the iPad's message shape.
|
||||
|
||||
This is what makes "faithful twin" real: the iPad and the Arduino are the same
|
||||
control source speaking the same contract.
|
||||
|
||||
## 8. Error handling / resilience
|
||||
|
||||
- **Renderer drops:** SSE auto-reconnects; on reconnect it GETs `/api/state` and
|
||||
resyncs. The per-field transition timeout clears any knob left `transitioning`
|
||||
by a renderer that vanished mid-animation.
|
||||
- **Controller drops / flaky wifi:** POSTs are fire-and-retry; absolute `set`
|
||||
values are idempotent, so a retried knob value is harmless. `altitude_delta` is
|
||||
the one relative input — on a failed POST the remote keeps the tick in its
|
||||
local pending target and resends rather than double-counting.
|
||||
- **No renderer connected:** the server still accepts control changes and holds
|
||||
state; a renderer that connects later catches up from the snapshot.
|
||||
- **Stale control source:** every event carries `seq`; a source ignores events
|
||||
older than what it has applied.
|
||||
|
||||
## 9. Testing (satisfies the mandatory pipeline tiers)
|
||||
|
||||
- **Pure unit** — a `SessionState` store in Python: apply patch → live vs.
|
||||
transitional classification; `altitude_delta` → ring move via the existing
|
||||
`advance_ring`; `seq` bump; transition flag set/clear; timeout. No I/O, tested
|
||||
like the rest of `player/`.
|
||||
- **API/contract** — `GET /api/state`; `POST /api/control` (live + transitional +
|
||||
bad input → 422); `POST /api/render/event` clears the flag; the SSE
|
||||
event-builder tested as a pure function.
|
||||
- **E2E (Playwright, required for the UI surface)** — open the renderer and
|
||||
`/remote` as two pages: move a live knob → assert the renderer's render
|
||||
updates; tick Altitude → assert a transition plays, the remote shows
|
||||
`transitioning`, then clears on settle.
|
||||
|
||||
## 10. Scope
|
||||
|
||||
**In v1:**
|
||||
|
||||
- Server `SessionState` + the four endpoints (`/api/state`, `/api/control`,
|
||||
`/api/events`, `/api/render/event`).
|
||||
- Renderer refactor to display-only + SSE client + settled ack.
|
||||
- The `/remote` iPad twin: the five controls + per-knob transition state.
|
||||
- Wifi/LAN only; laptop renders.
|
||||
|
||||
**Explicitly NOT in v1 (deferred):**
|
||||
|
||||
- Any Arduino/firmware (contract defined here; hardware later).
|
||||
- Bluetooth / USB transport.
|
||||
- Auth / room-token.
|
||||
- The Pi renderer.
|
||||
- Multi-room / multi-renderer fan-out (the SSE design allows it; not a v1 goal).
|
||||
|
||||
## 11. Roadmap impact
|
||||
|
||||
- **Sub-project 4** is reframed: the panel is now "any control source over the
|
||||
`/api/control` contract," not USB-serial-to-Pi first. The iPad twin is the v1
|
||||
deliverable; the Arduino is a later drop-in.
|
||||
- **Sub-project 3's "serial input" slice** is absorbed into a transport detail
|
||||
(the Arduino↔bridge link), no longer a Pi-coupled concern.
|
||||
- `ROADMAP.md` to be updated at implementation time.
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Resolve the orthogonal Visual × Audio controls into render outputs (audio spec
|
||||
§4/§6). The single source of truth for: whether the projector shows video, and
|
||||
which audio url plays for a given (audio source, current altitude).
|
||||
|
||||
Replaces player/content.py's 7-way bundled table. Soundtrack follows the single
|
||||
Altitude dial; off is silence. `white_noise` and `music` are deferred sources —
|
||||
not in v1 (the live control is a simple Audio on/off toggle where on=soundtrack)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
VISUAL_POSITIONS = frozenset({"on", "off"})
|
||||
# v1 audio sources. The live UI is an Audio on/off toggle (on=soundtrack).
|
||||
# "white_noise" and "music" are deferred (no live control) and intentionally absent.
|
||||
AUDIO_SOURCES = frozenset({"off", "soundtrack"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AudioResolution:
|
||||
source: str
|
||||
url: str | None
|
||||
altitude_coupled: bool
|
||||
|
||||
|
||||
def resolve_visual(position: str) -> bool:
|
||||
"""Visual on/off → whether the renderer shows video (vs fade-to-black)."""
|
||||
if position not in VISUAL_POSITIONS:
|
||||
raise ValueError(
|
||||
f"unknown visual position {position!r}; expected one of {sorted(VISUAL_POSITIONS)}"
|
||||
)
|
||||
return position == "on"
|
||||
|
||||
|
||||
def resolve_audio_url(source: str, *, scale_audio: str) -> str | None:
|
||||
"""The audio url for (source, current altitude): soundtrack → the current
|
||||
scale's asset (or None if none authored); off → None. `scale_audio` is the ring
|
||||
scale's `audio` path (relative to /media/audio/)."""
|
||||
if source not in AUDIO_SOURCES:
|
||||
raise ValueError(
|
||||
f"unknown audio source {source!r}; expected one of {sorted(AUDIO_SOURCES)}"
|
||||
)
|
||||
if source == "off":
|
||||
return None
|
||||
# soundtrack — couple to the current altitude's asset
|
||||
return f"/media/audio/{scale_audio}" if scale_audio else None
|
||||
|
||||
|
||||
def resolve_audio(source: str, *, scale_audio: str) -> AudioResolution:
|
||||
"""The full render.audio view: source, resolved url, and whether it follows the
|
||||
Altitude dial (true only for soundtrack)."""
|
||||
url = resolve_audio_url(source, scale_audio=scale_audio)
|
||||
return AudioResolution(source=source, url=url, altitude_coupled=(source == "soundtrack"))
|
||||
@@ -1,42 +0,0 @@
|
||||
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
|
||||
|
||||
The single source of truth for design §6's table: which audio source plays and
|
||||
whether the projector shows video, for each dial position.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
# The four distinct audio sources. "white_noise" is generated at runtime;
|
||||
# "music" is the public-domain classical pool; "audio_track" is the clip's own
|
||||
# field audio; "none" is silence.
|
||||
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContentResolution:
|
||||
audio_source: str
|
||||
video: bool
|
||||
|
||||
|
||||
# Design §6, one row per dial position.
|
||||
_TABLE = {
|
||||
"off": ContentResolution("none", False),
|
||||
"white_noise": ContentResolution("white_noise", False),
|
||||
"music": ContentResolution("music", False),
|
||||
"audio_track": ContentResolution("audio_track", False),
|
||||
"video": ContentResolution("none", True),
|
||||
"music_video": ContentResolution("music", True),
|
||||
"audio_video": ContentResolution("audio_track", True),
|
||||
}
|
||||
|
||||
|
||||
def resolve_content(position: str) -> ContentResolution:
|
||||
"""Map a content-dial position to its audio source and video flag (§6)."""
|
||||
try:
|
||||
return _TABLE[position]
|
||||
except KeyError:
|
||||
raise ValueError(
|
||||
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
|
||||
) from None
|
||||
+15
-12
@@ -1,20 +1,17 @@
|
||||
"""Control-panel state: the data shape read from the Arduino over serial.
|
||||
|
||||
This is the serial-contract payload shared with sub-project 4 (firmware). It
|
||||
models the full panel from design §6/§7: the 7-way content dial, the four
|
||||
experience knobs (0..4), and the two intensity levels (volume, brightness).
|
||||
The wire framing itself is the separate 3<->4 serial contract; this module is
|
||||
the *decoded* form.
|
||||
models the full panel from the audio spec §3: a Visual toggle (on/off) + an
|
||||
Audio source selector (off/soundtrack; white-noise/music deferred), the four experience knobs
|
||||
(0..4), and the two intensity levels (volume, brightness). The wire framing
|
||||
itself is the separate 3<->4 serial contract; this module is the *decoded* form.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, fields
|
||||
|
||||
# The seven positions of the content dial (design §6).
|
||||
CONTENT_POSITIONS = frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
from player.audio import AUDIO_SOURCES, VISUAL_POSITIONS
|
||||
|
||||
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
|
||||
KNOB_MIN = 0
|
||||
@@ -27,7 +24,8 @@ class ControlsError(ValueError):
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Controls:
|
||||
content: str
|
||||
visual: str
|
||||
audio: str
|
||||
left: int
|
||||
right: int
|
||||
dark: int
|
||||
@@ -38,10 +36,15 @@ class Controls:
|
||||
|
||||
def validate_controls(c: Controls) -> None:
|
||||
"""Raise ControlsError if the payload is structurally invalid."""
|
||||
if c.content not in CONTENT_POSITIONS:
|
||||
if c.visual not in VISUAL_POSITIONS:
|
||||
raise ControlsError(
|
||||
f"invalid content position {c.content!r}; "
|
||||
f"expected one of {sorted(CONTENT_POSITIONS)}"
|
||||
f"invalid visual position {c.visual!r}; "
|
||||
f"expected one of {sorted(VISUAL_POSITIONS)}"
|
||||
)
|
||||
if c.audio not in AUDIO_SOURCES:
|
||||
raise ControlsError(
|
||||
f"invalid audio source {c.audio!r}; "
|
||||
f"expected one of {sorted(AUDIO_SOURCES)}"
|
||||
)
|
||||
for name in KNOB_FIELDS:
|
||||
value = getattr(c, name)
|
||||
|
||||
@@ -63,6 +63,7 @@ class Scale:
|
||||
id: str
|
||||
clip_id: str
|
||||
pool: tuple[str, ...] = ()
|
||||
audio: str = "" # per-scale soundtrack, relative to /media/audio/ (audio spec §5.1)
|
||||
|
||||
@property
|
||||
def members(self) -> tuple[str, ...]:
|
||||
|
||||
+14
-8
@@ -20,7 +20,7 @@ from typing import Callable, Optional
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import RenderPlan, plan_alteration
|
||||
from player.content import ContentResolution, resolve_content
|
||||
from player.audio import resolve_visual
|
||||
from player.controls import Controls
|
||||
|
||||
|
||||
@@ -34,11 +34,14 @@ class TransitionKind:
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Playback:
|
||||
"""What should currently be playing."""
|
||||
"""What should currently be playing. Visual and audio are orthogonal (audio
|
||||
spec §2): `video` is whether the projector shows footage, `audio_source` is
|
||||
the independent audio dial (off / soundtrack)."""
|
||||
|
||||
clip_id: Optional[str] # None = black walls
|
||||
plan: Optional[RenderPlan] # None when black
|
||||
content: ContentResolution
|
||||
video: bool # whether footage is shown (Visual on/off)
|
||||
audio_source: str # the Audio dial source (off/soundtrack)
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
@@ -58,7 +61,8 @@ def _first(library):
|
||||
_BLACK = Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=ContentResolution("none", False),
|
||||
video=False,
|
||||
audio_source="off",
|
||||
volume=0,
|
||||
brightness=0,
|
||||
)
|
||||
@@ -78,12 +82,13 @@ class Player:
|
||||
self._current = _BLACK
|
||||
|
||||
def _resolve(self, controls: Controls) -> Playback:
|
||||
content = resolve_content(controls.content)
|
||||
if not content.video:
|
||||
video = resolve_visual(controls.visual)
|
||||
if not video:
|
||||
return Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=content,
|
||||
video=False,
|
||||
audio_source=controls.audio,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
@@ -92,7 +97,8 @@ class Player:
|
||||
return Playback(
|
||||
clip_id=clip.id,
|
||||
plan=plan_alteration(coord),
|
||||
content=content,
|
||||
video=True,
|
||||
audio_source=controls.audio,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
|
||||
@@ -24,3 +24,9 @@ sim = [
|
||||
"uvicorn[standard]>=0.29",
|
||||
"httpx>=0.27",
|
||||
]
|
||||
# Playwright E2E browser tests (audio spec §9). Install with the browser binary:
|
||||
# pip install -e '.[e2e]' && python -m playwright install chromium
|
||||
# The E2E suite skips cleanly when Playwright/Chromium is absent (headless box).
|
||||
e2e = [
|
||||
"pytest-playwright>=0.4",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
# Session 0018.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-25T09-42 (PST)
|
||||
> End: 2026-06-26T05-56 (PST)
|
||||
> Type: planning-and-executing
|
||||
> Posture: yolo
|
||||
> Claude-Session: b96041e5-c682-49f5-88ee-380ea7abb6af
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
Find a better public-domain nebula flythrough video for the cosmos scale (the current text-free SVS Orion clip lost quality when cropped to remove its text overlay), or upscale the existing one if no better source exists.
|
||||
```
|
||||
|
||||
A content-and-polish session on the simulator's cosmos scale that grew, by
|
||||
operator follow-ups, into four merged PRs: the nebula swap, two cache fixes, a
|
||||
ring-landing artifact fix, and a Right-brain affect rework.
|
||||
|
||||
## Pre-state
|
||||
|
||||
- Branch `feature/ring-transitions-real` (later merged): HEAD `129bb23` had just
|
||||
swapped the cosmos primary off the captioned/cropped JPL "Orion: Dust and Death"
|
||||
clip onto the text-free SVS 30957 Orion flythrough.
|
||||
- Sim served by a manually-launched `uvicorn ... --port 8000` (no `--reload`),
|
||||
booted before the session's media changes.
|
||||
- Affect channel gated on `min(left, right)` (session 0013 rule).
|
||||
- Concurrent unfinalized sessions present (0015, 0017).
|
||||
|
||||
## Arc (turn by turn)
|
||||
|
||||
1. **Anchor + research.** Classified as planning-and-executing, claimed session
|
||||
0018. Found the current cosmos clip was already the uncropped text-free SVS
|
||||
30957 Orion (the crop damage the prompt described was already fixed at HEAD).
|
||||
Researched NASA SVS: (a) the same Orion clip exists in 4K; (b) the Webb
|
||||
**"Cosmic Cliffs"** Carina flythrough (SVS 31348) is a 4K PD alternative.
|
||||
Offered three options; **operator chose #2 — switch to Cosmic Cliffs.**
|
||||
|
||||
2. **Cosmos swap (PR #21).** Downloaded the Carina 4K master
|
||||
(`Clifs-3d-STScI.mp4`, 3840×2160, 60 fps, 103 s), sampled frames to find a
|
||||
clean window, trimmed **10–34 s** (clears the title card ~8 s and the
|
||||
end-credit card ~102 s), ran `tools.pipeline.run.process_clip` → 4K master +
|
||||
supersampled 1080p base (5.5 Mbps, crisper than the old 2.9 Mbps Orion encode).
|
||||
Updated `build_pool_manifest.py` META (new `CCBY_WEBB` credit, Carina labels:
|
||||
emission nebula / protostar, redshift→distance ≈7,500 ly), regenerated
|
||||
`manifest.json` (also corrected stale JPL-Orion provenance) + all ring-edge
|
||||
transitions, and `docs/content-candidate-pool.md`. Operator authorized the
|
||||
merge → opened + merged **PR #21** (the whole `feature/ring-transitions-real`
|
||||
line), synced main, deleted the local branch. Remote branch delete returned
|
||||
HTTP 405 (known Gitea perm limit).
|
||||
|
||||
3. **"Still seeing Orion" — caching (PRs #22, #23).** Diagnosed: the sim server
|
||||
(PID 33672) had been running since before the media swap and was launched
|
||||
**without `--reload`**, holding the pre-swap manifest in memory; the browser
|
||||
also held the old clip via the in-memory blob preload and a possible pre-
|
||||
`no-cache` immutable HTTP entry. Restarted the server with `make sim-local`
|
||||
(`--reload`). Shipped **PR #22** (`cache: "reload"` on the preload fetch), then
|
||||
— at operator request for a permanent fix — **PR #23**: a content-hash
|
||||
`GET /api/media-versions` endpoint (sha1[:12] per served file, cached by
|
||||
mtime/size) and client `?v=<hash>` on every /media URL, so a re-baked clip's
|
||||
URL changes with its bytes.
|
||||
|
||||
4. **Coast landing artifact (in PR #23).** Operator noticed zooming into coast
|
||||
showed birdrock then switched to another clip. Root cause: the baked ring
|
||||
transition lands on the scale **primary** (birdrock), but the rotating pool
|
||||
picks a **random member** on landing — a non-primary pick hard-cut from primary
|
||||
to chosen. Fix: `advance()` masks the swap behind a short fade through the
|
||||
existing `#black` overlay when `target_clip_id !== ring.scales[to_index].clip_id`;
|
||||
general across all pooled scales.
|
||||
|
||||
5. **Right-brain affect (PR #24).** Operator: emotions should be controlled by the
|
||||
right brain, not the left. Reversed `affect.strength = min(left, right)` →
|
||||
`strength = right`; intensity keyed off Right; Left no longer gates emotions.
|
||||
Updated engine + client comments + the affect design spec (revision note) +
|
||||
tests. Live-verified: left=0/right=4 → strength 4; left=4/right=0 → 0.
|
||||
|
||||
## Cut state
|
||||
|
||||
- **`main` @ (session-0018 merges all landed):** PRs **#21** (`7da7446`), **#22**
|
||||
(`aa56dfe`), **#23** (`5bb762a`), **#24** (`d0ad9a9`) — all confirmed in main's
|
||||
ancestry. Session 0019 has since advanced main further (cosmos_miri pool member,
|
||||
networked-control-surface spec, sim-local venv fix).
|
||||
- **Tests:** 271 passed, 2 skipped (+ media-version tests + rewritten affect
|
||||
tests).
|
||||
- **Sim:** running on :8000 as `uvicorn --reload` (PID 50199, restarted this
|
||||
session). The cosmos Cosmic Cliffs clip, content-hash cache-bust, coast fade,
|
||||
and Right-only affect are all live and API/logic-verified.
|
||||
- **No plan artifact** archived — leaf content/polish task executed directly
|
||||
(no `superpowers:writing-plans` doc).
|
||||
|
||||
## Operator plate
|
||||
|
||||
- **Cosmic Cliffs looks "incredible"** (operator) — the cosmos primary is now the
|
||||
Webb Carina flythrough.
|
||||
- **Unverified by eye** (no Chrome on the box): the coast landing fade timing, and
|
||||
the Right-only affect look (Left=0 + Right up should show emotion words over an
|
||||
un-annotated scene). Worth a by-eye pass in `make sim-local`.
|
||||
- **Pipeline §9:** changes merged to `main` (= done); no PPE/prod run — the E2E +
|
||||
PPE machinery is still being built (§10.6), and the simulator is a local dev
|
||||
surface. Not shipped to prod.
|
||||
- **Left untouched (not this session's work):** session 0019's in-flight audio
|
||||
work is dirty in the tree (`docs/audio-candidate-pool.md`, `.gitignore` audio
|
||||
entries) — left for 0019 to finalize. Pre-existing unfinalized sessions 0015,
|
||||
0017, 0019 remain `--INPROGRESS`.
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
None logged — every call this session was operator-directed (the option-2 clip
|
||||
choice, the permanent cache-bust request, the Right-brain affect direction).
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
```
|
||||
/goal By-eye review of session 0018's sim changes in `make sim-local` (:8000): the
|
||||
Webb Cosmic Cliffs cosmos clip, the coast/pool landing fade (no hard cut to the
|
||||
random member), and Right-only emotions (Left=0 + Right up shows feeling words over
|
||||
an un-annotated scene). Tune the ~200ms fade timing if it feels off. Then continue
|
||||
the experience/content line — deferred i2v ring transitions and/or the Pi renderer
|
||||
(per [[simulator-first-before-hardware]]). Note: session 0019 separately owns the
|
||||
audio pool + networked-control-surface work.
|
||||
```
|
||||
@@ -1,25 +0,0 @@
|
||||
# Session 0018.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-25T09-42 (PST)
|
||||
> Type: planning-and-executing
|
||||
> Posture: yolo
|
||||
> Claude-Session: b96041e5-c682-49f5-88ee-380ea7abb6af
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0018 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0018.0-TRANSCRIPT-2026-06-25T09-42--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
Find a better public-domain nebula flythrough video for the cosmos scale (the current text-free SVS Orion clip lost quality when cropped to remove its text overlay), or upscale the existing one if no better source exists.
|
||||
|
||||
```
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
@@ -0,0 +1,95 @@
|
||||
# Session 0019.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-26T04-47 (PST)
|
||||
> End: 2026-06-26T05-47 (PST)
|
||||
> Type: brainstorming
|
||||
> Posture: careful
|
||||
> Claude-Session: d6ed1203-92b6-471f-80a4-4c773b645ca1
|
||||
> Anchor: ROADMAP sub-project 4 (Arduino Firmware / control panel) — reframed
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
Let's start talking about hardware. Let's assume for the first version I'm ok having my computer or laptop do the rendering. We'd still want a controller but it could be connected to the computer via wifi, USB, or bluetooth and would send commands to the software we've already written that would adjust left brain, right brain, etc. We'd prototype this by having a controller simulator run on another device, such as an ipad (just riffing here) but eventually move to a physical remote running on an Arduino or similar
|
||||
```
|
||||
|
||||
## Pre-state
|
||||
|
||||
- Simulator is a **FastAPI** app, **stateless**: the single browser holds knob
|
||||
state in JS and POSTs to `/api/alteration` for a `RenderPlan`. Renderer and
|
||||
controller are the same browser tab.
|
||||
- Operator-facing control surface (sim today): Content (7-way `<select>`),
|
||||
Altitude (endless circular dial across 5 scales), Left/Right (sliders 0–4),
|
||||
Mood (slider −4…+4). (Dark+Light → Mood in s0013; Altitude dial in s0018.)
|
||||
- ROADMAP sub-project 4 = "Arduino Firmware (control panel)" assuming a **Pi**
|
||||
runs the room and the panel talks **USB serial** to it.
|
||||
- Standing directive: simulator-first before hardware.
|
||||
- Checkout shared with a concurrent audio-sourcing session (0017/0018), with
|
||||
uncommitted `.gitignore` + untracked files on `main`.
|
||||
|
||||
## Arc
|
||||
|
||||
1. **Session gate / routing.** Classified the opening ("let's talk about
|
||||
hardware… just riffing") as **brainstorming**; invoked `wgl-brainstorming`,
|
||||
claimed session **0019** (`--type brainstorming`, careful posture). Three stale
|
||||
`--INPROGRESS` sessions (0015/0017/0018) noted as leftovers, not live overlaps;
|
||||
no worktree (the audio session's edits are unrelated/uncommitted).
|
||||
|
||||
2. **Orientation.** Read `docs/ROADMAP.md` + the simulator (`app.py`, `index.html`)
|
||||
to ground the discussion; confirmed the real control surface and the stateless
|
||||
architecture. Recognized the request as a **reframe** of sub-project 4.
|
||||
|
||||
3. **Brainstorming dialogue** (`superpowers:brainstorming`), decisions reached:
|
||||
- **Intent:** both — decouple control/display cleanly AND want the handheld feel.
|
||||
- **Transport (v1):** wifi/web (iPad opens a controller page off the FastAPI
|
||||
server; least friction; Arduino transport left open).
|
||||
- **Remote fidelity:** faithful twin — decide the physical panel inventory now,
|
||||
iPad mirrors it.
|
||||
- Operator refinement: **per-knob transition state** — live controls
|
||||
(Left/Right/Mood) are instant; transitional ones (Altitude, Content) carry a
|
||||
lifecycle so the remote's feedback is honest.
|
||||
- **Architecture:** approach A — server-authoritative `SessionState` + **SSE**
|
||||
push (chosen over polling / WebRTC).
|
||||
|
||||
4. **Design presented section by section** (roles → transition model → server
|
||||
contract → two pages + Arduino seam → error/test/scope), each approved.
|
||||
|
||||
5. **Spec written** to
|
||||
`docs/superpowers/specs/2026-06-26-networked-control-surface-design.md`,
|
||||
committed on `feat/networked-control-surface`, pushed; operator approved going
|
||||
straight to finalize.
|
||||
|
||||
6. **Finalize.** Single-repo app (CONTENT_REMOTE empty) → merging the branch IS
|
||||
the submission. Discovered the shared checkout had switched to `main` (the
|
||||
audio session) and `main` had advanced to PR #25; landed the spec via a safe
|
||||
local `--no-ff` merge that added only the new spec file (their uncommitted work
|
||||
untouched), pushed `main` (`229af60..f28a06c`), deleted the feature branch.
|
||||
|
||||
## Cut state
|
||||
|
||||
- `main` @ `f28a06c` — spec merged. Feature branch deleted.
|
||||
- Spec `status: proposed` (careful posture); operator reviewed it live, approving
|
||||
every section before write.
|
||||
- No code written (brainstorming session). 0 new tests (none applicable).
|
||||
- Concurrent audio session's uncommitted `.gitignore`/untracked files left intact.
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_None — no autonomous low-confidence calls. The two flagged design choices
|
||||
(last-write-wins concurrency; no auth in v1) were presented to the operator and
|
||||
approved in-session, not deferred._
|
||||
|
||||
## Operator plate
|
||||
|
||||
- Review the merged spec if desired:
|
||||
`docs/superpowers/specs/2026-06-26-networked-control-surface-design.md` on `main`.
|
||||
- Next: open a **writing-plans** session to turn the spec into an implementation
|
||||
plan (the `/goal` below).
|
||||
|
||||
## Next /goal
|
||||
|
||||
```
|
||||
/goal Write the implementation plan for the networked control surface, per docs/superpowers/specs/2026-06-26-networked-control-surface-design.md
|
||||
```
|
||||
@@ -0,0 +1,22 @@
|
||||
# Session 0020.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-26T06-47 (PST)
|
||||
> Type: writing-plans
|
||||
> Posture: careful
|
||||
> Claude-Session: e0cc053a-41fa-416d-a4f5-493f14984e75
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0020 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0020.0-TRANSCRIPT-2026-06-26T06-47--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
_(launch prompt not captured at claim time)_
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
@@ -52,5 +52,11 @@
|
||||
},
|
||||
"0018": {
|
||||
"title": ""
|
||||
},
|
||||
"0019": {
|
||||
"title": ""
|
||||
},
|
||||
"0020": {
|
||||
"title": ""
|
||||
}
|
||||
}
|
||||
|
||||
+22
-7
@@ -26,8 +26,7 @@ from player.alteration import (
|
||||
plan_alteration,
|
||||
render_plan_to_dict,
|
||||
)
|
||||
from player.content import resolve_content
|
||||
from player.controls import CONTENT_POSITIONS
|
||||
from player.audio import AUDIO_SOURCES, VISUAL_POSITIONS, resolve_audio, resolve_visual
|
||||
from player.ring import (
|
||||
DEFAULT_FAST_SPIN_THRESHOLD,
|
||||
advance_ring,
|
||||
@@ -95,7 +94,8 @@ def _rev_file(file: str) -> str:
|
||||
|
||||
|
||||
class ControlsModel(BaseModel):
|
||||
content: str
|
||||
visual: str
|
||||
audio: str
|
||||
left: int = Field(ge=0, le=4)
|
||||
right: int = Field(ge=0, le=4)
|
||||
dark: int = Field(ge=0, le=4)
|
||||
@@ -112,6 +112,7 @@ class CalibrationModel(BaseModel):
|
||||
|
||||
class AlterationRequest(BaseModel):
|
||||
controls: ControlsModel
|
||||
altitude_index: int = 0
|
||||
calibration: Optional[CalibrationModel] = None
|
||||
|
||||
|
||||
@@ -166,8 +167,10 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
|
||||
@app.post("/api/alteration")
|
||||
def api_alteration(req: AlterationRequest):
|
||||
c = req.controls
|
||||
if c.content not in CONTENT_POSITIONS:
|
||||
raise HTTPException(status_code=422, detail=f"invalid content {c.content!r}")
|
||||
if c.visual not in VISUAL_POSITIONS:
|
||||
raise HTTPException(status_code=422, detail=f"invalid visual {c.visual!r}")
|
||||
if c.audio not in AUDIO_SOURCES:
|
||||
raise HTTPException(status_code=422, detail=f"invalid audio {c.audio!r}")
|
||||
coord = Coordinate(c.left, c.right, c.dark, c.light)
|
||||
cal = (
|
||||
Calibration(
|
||||
@@ -179,10 +182,22 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
|
||||
else DEFAULT_CALIBRATION
|
||||
)
|
||||
plan = plan_alteration(coord, cal)
|
||||
content = resolve_content(c.content)
|
||||
# Resolve the soundtrack url against the CURRENT altitude (server-side, audio
|
||||
# spec §6.1). White-noise/off ignore the scale; soundtrack follows it.
|
||||
scale_audio = ""
|
||||
if app.state.ring is not None:
|
||||
scale_audio = scale_at(app.state.ring, req.altitude_index).audio
|
||||
audio = resolve_audio(c.audio, scale_audio=scale_audio)
|
||||
return {
|
||||
"plan": render_plan_to_dict(plan),
|
||||
"content": {"audio_source": content.audio_source, "video": content.video},
|
||||
"render": {
|
||||
"video": {"shown": resolve_visual(c.visual)},
|
||||
"audio": {
|
||||
"source": audio.source,
|
||||
"url": audio.url,
|
||||
"altitude_coupled": audio.altitude_coupled,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
@app.get("/dev/version")
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
"""Production pass for the 5 per-altitude soundtracks (audio spec §5.3). The audio
|
||||
analogue of build_pool_manifest.py --media: read the sourced ambience clips
|
||||
(re-downloadable per docs/audio-candidate-pool.md), and make each a seamless
|
||||
loudness-normalized loop.
|
||||
|
||||
Media is gitignored; this script is the reproducible record. Outputs land under
|
||||
simulator/sample_media/audio/<scale>/<name>.loop.mp3, served at /media/audio/...
|
||||
Run: python simulator/build_audio_media.py
|
||||
|
||||
(White-noise is deferred — the live Audio control is a simple on/off soundtrack
|
||||
toggle. `tools.pipeline.audio_run.generate_white_noise` remains for when it's
|
||||
wanted, but no noise bed is built here.)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from tools.pipeline.audio_run import process_soundtrack
|
||||
|
||||
AUDIO = Path(__file__).parent / "sample_media" / "audio"
|
||||
|
||||
# scale -> (sourced raw file, production output file), both under audio/<scale>/.
|
||||
# Sources per docs/audio-candidate-pool.md (the "✓ KEEP" picks).
|
||||
SOURCES: dict[str, tuple[str, str]] = {
|
||||
"cosmos": ("pillars.mp3", "pillars.loop.mp3"),
|
||||
"orbit": ("spaceamb.mp3", "spaceamb.loop.mp3"),
|
||||
"coast": ("waves.mp3", "waves.loop.mp3"),
|
||||
"reef": ("soundscape.mp3", "soundscape.loop.mp3"),
|
||||
"abyss": ("whale.wav", "whale.loop.mp3"),
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
for scale, (raw, out) in SOURCES.items():
|
||||
src = AUDIO / scale / raw
|
||||
if not src.exists():
|
||||
print(f" SKIP {scale}: source missing ({src}) — re-source per audio-candidate-pool.md")
|
||||
continue
|
||||
dst = process_soundtrack(src, AUDIO / scale / out)
|
||||
print(f" produced {dst.relative_to(AUDIO.parent)}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -38,7 +38,7 @@ MANIFEST = MEDIA / "manifest.json"
|
||||
|
||||
# --- Pools: scale id -> ordered clip ids (primary first) (content-candidate-pool.md) ---
|
||||
POOLS: dict[str, list[str]] = {
|
||||
"cosmos": ["cosmos", "cosmos_galaxies", "cosmos_hudf", "cosmos_xdf"],
|
||||
"cosmos": ["cosmos", "cosmos_miri", "cosmos_galaxies", "cosmos_hudf", "cosmos_xdf"],
|
||||
"orbit": ["orbit_planetearth", "orbit_crewobs", "orbit_bluemarble"],
|
||||
"coast": ["coast_birdrock", "coast_surfgrass", "coast_elkbeach", "coast_drakesbeach"],
|
||||
"reef": ["reef_lionfish", "reef_spawning", "reef_hawkfish", "reef_snapper", "reef_coralspacific"],
|
||||
@@ -48,6 +48,16 @@ POOLS: dict[str, list[str]] = {
|
||||
# Ring order (large -> small, wraps): cosmos -> orbit -> coast -> reef -> abyss -> cosmos.
|
||||
RING_ORDER = ["cosmos", "orbit", "coast", "reef", "abyss"]
|
||||
|
||||
# Per-scale soundtrack (audio spec §5.1) — the production-pass output of
|
||||
# simulator/build_audio_media.py, served at /media/audio/<path>.
|
||||
SCALE_AUDIO: dict[str, str] = {
|
||||
"cosmos": "cosmos/pillars.loop.mp3",
|
||||
"orbit": "orbit/spaceamb.loop.mp3",
|
||||
"coast": "coast/waves.loop.mp3",
|
||||
"reef": "reef/soundscape.loop.mp3",
|
||||
"abyss": "abyss/whale.loop.mp3",
|
||||
}
|
||||
|
||||
PD = "public-domain (US Gov, 17 U.S.C. §105)"
|
||||
PD_NPS = "public-domain (NPS, no © — 17 U.S.C. §105)"
|
||||
CCBY_STSCI = "CC-BY-class — credit NASA, ESA, STScI"
|
||||
@@ -59,6 +69,10 @@ META: dict[str, tuple[str, str, str]] = {
|
||||
"cosmos": ("Cosmic Cliffs — Carina Nebula flythrough (NASA/ESA/CSA/STScI)", CCBY_WEBB,
|
||||
"NASA SVS 31348 Clifs-3d-STScI (Exploring the Cosmic Cliffs in 3D, Webb NIRCam); "
|
||||
"4K source, trim 10–34s, crossfade-loop. Text-free — replaces the Orion SVS 30957 flythrough."),
|
||||
"cosmos_miri": ("Cosmic Cliffs — Carina Nebula, mid-infrared (NASA/ESA/CSA/STScI)", CCBY_WEBB,
|
||||
"ESA/Webb weic2205c (Pan of the Carina Nebula, combined NIRCam + MIRI); 4K source, "
|
||||
"trim 4–26s, crossfade-loop. The MIRI mid-infrared composite — a muted gray/teal/gold "
|
||||
"dust palette distinct from the NIRCam flythrough, same Cosmic Cliffs towers. Text-free."),
|
||||
"cosmos_galaxies": ("Flying Through Galaxies (NASA SVS)", PD,
|
||||
"NASA SVS a014950 14950_Galaxies_FlyThrough_4k; trim 8–32s, crossfade-loop"),
|
||||
"cosmos_hudf": ("Hubble Ultra Deep Field zoom (NASA/ESA/STScI)", CCBY_STSCI,
|
||||
@@ -167,6 +181,11 @@ LABELS: dict[str, list[dict]] = {
|
||||
static_label("detected.star", 2, ["star", "young star", "protostar", "protostar · a newborn star, <1 Myr old"], [0.54, 0.12, 0.12, 0.14]),
|
||||
measure("measure.distance", "≈7,500 ly", [0.06, 0.06, 0.2, 0.08], 3),
|
||||
],
|
||||
"cosmos_miri": [
|
||||
static_label("detected.nebula", 4, ["dust", "nebula", "dust ridge", "dust ridge · mid-infrared glow of the Carina cliffs"], [0.10, 0.45, 0.7, 0.45]),
|
||||
static_label("detected.star", 2, ["star", "young star", "protostar", "protostar · its dust laid bare in mid-infrared"], [0.48, 0.18, 0.14, 0.16]),
|
||||
measure("measure.distance", "≈7,500 ly", [0.06, 0.06, 0.2, 0.08], 3),
|
||||
],
|
||||
"cosmos_galaxies": [
|
||||
static_label("detected.galaxy", 4, ["galaxy", "spiral galaxy", "barred spiral", "barred spiral · ~10¹¹ stars"], [0.34, 0.30, 0.30, 0.34]),
|
||||
measure("measure.distance", "~Mly", [0.06, 0.06, 0.18, 0.08], 3),
|
||||
@@ -348,7 +367,10 @@ def build_manifest() -> dict:
|
||||
for scale in RING_ORDER:
|
||||
for clip_id in POOLS[scale]:
|
||||
clips.append(_clip_entry(scale, clip_id))
|
||||
scales = [{"id": s, "clip_id": POOLS[s][0], "pool": POOLS[s]} for s in RING_ORDER]
|
||||
scales = [
|
||||
{"id": s, "clip_id": POOLS[s][0], "pool": POOLS[s], "audio": SCALE_AUDIO[s]}
|
||||
for s in RING_ORDER
|
||||
]
|
||||
transitions = []
|
||||
n = len(RING_ORDER)
|
||||
for i in range(n):
|
||||
|
||||
+2
-1
@@ -82,7 +82,7 @@ def _scale_from_dict(s: dict[str, Any]) -> Scale:
|
||||
"""
|
||||
pool = tuple(s.get("pool", []))
|
||||
clip_id = s.get("clip_id") or (pool[0] if pool else "")
|
||||
return Scale(id=s["id"], clip_id=clip_id, pool=pool)
|
||||
return Scale(id=s["id"], clip_id=clip_id, pool=pool, audio=s.get("audio", ""))
|
||||
|
||||
|
||||
def load_ring(path: str | Path) -> ScaleRing | None:
|
||||
@@ -119,6 +119,7 @@ def ring_to_dict(ring: ScaleRing, clips: list[Clip]) -> dict:
|
||||
"id": s.id,
|
||||
"clip_id": s.clip_id,
|
||||
"title": titles.get(s.clip_id, s.id),
|
||||
"audio": s.audio,
|
||||
"pool": [
|
||||
{"clip_id": cid, "title": titles.get(cid, cid)}
|
||||
for cid in s.members
|
||||
|
||||
@@ -115,6 +115,121 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cosmos_miri",
|
||||
"title": "Cosmic Cliffs — Carina Nebula, mid-infrared (NASA/ESA/CSA/STScI)",
|
||||
"base_file": "cosmos_miri/base.mp4",
|
||||
"license": "CC-BY-class — credit NASA, ESA, CSA, STScI",
|
||||
"source": "ESA/Webb weic2205c (Pan of the Carina Nebula, combined NIRCam + MIRI); 4K source, trim 4–26s, crossfade-loop. The MIRI mid-infrared composite — a muted gray/teal/gold dust palette distinct from the NIRCam flythrough, same Cosmic Cliffs towers. Text-free.",
|
||||
"right_variants": {},
|
||||
"annotations": [
|
||||
{
|
||||
"key": "detected.nebula",
|
||||
"salience": 4,
|
||||
"box": [
|
||||
0.1,
|
||||
0.45,
|
||||
0.7,
|
||||
0.45
|
||||
]
|
||||
},
|
||||
{
|
||||
"key": "detected.star",
|
||||
"salience": 2,
|
||||
"box": [
|
||||
0.48,
|
||||
0.18,
|
||||
0.14,
|
||||
0.16
|
||||
]
|
||||
},
|
||||
{
|
||||
"key": "measure.distance",
|
||||
"box": [
|
||||
0.06,
|
||||
0.06,
|
||||
0.2,
|
||||
0.08
|
||||
],
|
||||
"min_level": 3
|
||||
}
|
||||
],
|
||||
"affect": [
|
||||
{
|
||||
"key": "feel.wonder",
|
||||
"at": [
|
||||
0.5,
|
||||
0.44
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "feel.vastness",
|
||||
"at": [
|
||||
0.2,
|
||||
0.7
|
||||
],
|
||||
"min_level": 2
|
||||
},
|
||||
{
|
||||
"key": "feel.insignificance",
|
||||
"at": [
|
||||
0.66,
|
||||
0.6
|
||||
],
|
||||
"min_level": 3
|
||||
},
|
||||
{
|
||||
"key": "feel.longing",
|
||||
"at": [
|
||||
0.42,
|
||||
0.84
|
||||
],
|
||||
"min_level": 4
|
||||
}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.nebula": [
|
||||
"dust",
|
||||
"nebula",
|
||||
"dust ridge",
|
||||
"dust ridge · mid-infrared glow of the Carina cliffs"
|
||||
],
|
||||
"detected.star": [
|
||||
"star",
|
||||
"young star",
|
||||
"protostar",
|
||||
"protostar · its dust laid bare in mid-infrared"
|
||||
],
|
||||
"measure.distance": "≈7,500 ly",
|
||||
"feel.wonder": [
|
||||
"wow",
|
||||
"wonder",
|
||||
"awe",
|
||||
"transcendent awe"
|
||||
],
|
||||
"feel.vastness": [
|
||||
"big",
|
||||
"vastness",
|
||||
"immensity",
|
||||
"a dizzying immensity"
|
||||
],
|
||||
"feel.insignificance": [
|
||||
"small",
|
||||
"smallness",
|
||||
"insignificance",
|
||||
"a humbling insignificance"
|
||||
],
|
||||
"feel.longing": [
|
||||
"want",
|
||||
"longing",
|
||||
"yearning",
|
||||
"an aching yearning"
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cosmos_galaxies",
|
||||
"title": "Flying Through Galaxies (NASA SVS)",
|
||||
@@ -2252,10 +2367,12 @@
|
||||
"clip_id": "cosmos",
|
||||
"pool": [
|
||||
"cosmos",
|
||||
"cosmos_miri",
|
||||
"cosmos_galaxies",
|
||||
"cosmos_hudf",
|
||||
"cosmos_xdf"
|
||||
]
|
||||
],
|
||||
"audio": "cosmos/pillars.loop.mp3"
|
||||
},
|
||||
{
|
||||
"id": "orbit",
|
||||
@@ -2264,7 +2381,8 @@
|
||||
"orbit_planetearth",
|
||||
"orbit_crewobs",
|
||||
"orbit_bluemarble"
|
||||
]
|
||||
],
|
||||
"audio": "orbit/spaceamb.loop.mp3"
|
||||
},
|
||||
{
|
||||
"id": "coast",
|
||||
@@ -2274,7 +2392,8 @@
|
||||
"coast_surfgrass",
|
||||
"coast_elkbeach",
|
||||
"coast_drakesbeach"
|
||||
]
|
||||
],
|
||||
"audio": "coast/waves.loop.mp3"
|
||||
},
|
||||
{
|
||||
"id": "reef",
|
||||
@@ -2285,7 +2404,8 @@
|
||||
"reef_hawkfish",
|
||||
"reef_snapper",
|
||||
"reef_coralspacific"
|
||||
]
|
||||
],
|
||||
"audio": "reef/soundscape.loop.mp3"
|
||||
},
|
||||
{
|
||||
"id": "abyss",
|
||||
@@ -2294,7 +2414,8 @@
|
||||
"abyss_wow",
|
||||
"abyss_midwaterexp",
|
||||
"abyss_hiding"
|
||||
]
|
||||
],
|
||||
"audio": "abyss/whale.loop.mp3"
|
||||
}
|
||||
],
|
||||
"transitions": [
|
||||
|
||||
+108
-10
@@ -96,7 +96,10 @@ let mediaVersions = {}; // file -> content-hash token (from /api/media-version
|
||||
// changes with its bytes (permanent cache-bust). The blob cache, when present,
|
||||
// wins — those bytes are already the current ones for this session.
|
||||
function mediaNetUrl(file) { const v = mediaVersions[file]; return "/media/" + file + (v ? "?v=" + v : ""); }
|
||||
function mediaUrl(file) { return mediaBlobs[file] || mediaNetUrl(file); }
|
||||
function mediaUrl(file) {
|
||||
if (file.startsWith("/media/")) return file; // already a resolved absolute url (audio layer)
|
||||
return mediaBlobs[file] || mediaNetUrl(file);
|
||||
}
|
||||
|
||||
// Every media file the ring can show: each scale-pool clip's base footage plus the
|
||||
// per-edge transition clips. This is the full preload set (~two dozen files).
|
||||
@@ -492,12 +495,14 @@ function renderScaleReadout() {
|
||||
}
|
||||
|
||||
function controls() {
|
||||
// One bipolar Mood dial (-4 dark .. 0 neutral .. +4 light) maps onto the engine's
|
||||
// two poles. The calibration gains are pinned to their locked 1.0 (full dial = full
|
||||
// effect), so the simulator sends no calibration — the server uses DEFAULT_CALIBRATION.
|
||||
// Video (on/off) and Audio (on/off) are orthogonal toggles (audio spec §2).
|
||||
// Audio on = the per-altitude soundtrack (white-noise is deferred). One bipolar
|
||||
// Mood dial (-4 dark .. +4 light) maps onto the engine's two poles. Calibration
|
||||
// gains are pinned to 1.0, so the simulator sends no calibration.
|
||||
const mood = +$("mood").value;
|
||||
return {
|
||||
content: $("content").value,
|
||||
visual: $("visual").checked ? "on" : "off",
|
||||
audio: $("audio").checked ? "soundtrack" : "off",
|
||||
left: +$("left").value, right: +$("right").value,
|
||||
dark: mood < 0 ? -mood : 0, light: mood > 0 ? mood : 0,
|
||||
volume: 2, brightness: 2,
|
||||
@@ -509,13 +514,23 @@ async function update() {
|
||||
if (busy) return;
|
||||
const resp = await fetch("/api/alteration", {
|
||||
method: "POST", headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ controls: controls() }),
|
||||
body: JSON.stringify({ controls: controls(), altitude_index: ringIndex }),
|
||||
});
|
||||
if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; }
|
||||
const data = await resp.json();
|
||||
readout.textContent = JSON.stringify(data, null, 2);
|
||||
if (!data.content.video) { black.style.opacity = "1"; black.classList.remove("hidden"); return; }
|
||||
applyAudio(data.render.audio); // reconcile the audio layer
|
||||
if (!data.render.video.shown) {
|
||||
// Blank the screen. Cover with #black AND hide the video layers themselves —
|
||||
// the <video>/<canvas> are GPU-composited and can otherwise show through the
|
||||
// overlay on a real GPU (Safari/Chrome).
|
||||
black.style.opacity = "1"; black.classList.remove("hidden");
|
||||
vid.style.opacity = "0"; paint.style.opacity = "0";
|
||||
return;
|
||||
}
|
||||
black.classList.add("hidden");
|
||||
vid.style.opacity = ""; paint.style.opacity = ""; // restore the video layers
|
||||
|
||||
try {
|
||||
ensureClipMedia();
|
||||
applyVideoLook(data.plan.grade.tone, data.plan.dream.intensity);
|
||||
@@ -894,6 +909,87 @@ function devLiveReload() {
|
||||
}, 1000);
|
||||
}
|
||||
|
||||
// --- Audio layer: two <audio> elements, gain-crossfaded (audio spec §7/§8) ---
|
||||
const audA = $("audA"), audB = $("audB");
|
||||
let audActive = audA, audIdle = audB; // which element is currently audible
|
||||
let audUrl = null; // the url currently playing (null = silence)
|
||||
let audioReady = false; // unlocked after the first user gesture (autoplay policy)
|
||||
const XFADE_MS = 600; // ~0.6s gain crossfade — "live", no busy ack (§7)
|
||||
|
||||
function fadeVolume(el, to, ms) {
|
||||
// setInterval (not requestAnimationFrame): rAF throttles/pauses when the tab is
|
||||
// backgrounded, which would stall the gain ramp; a timer fires regardless.
|
||||
const from = el.volume, steps = Math.max(1, Math.round(ms / 30));
|
||||
let i = 0;
|
||||
return new Promise((resolve) => {
|
||||
const iv = setInterval(() => {
|
||||
i += 1;
|
||||
const k = Math.min(1, i / steps);
|
||||
el.volume = Math.min(1, Math.max(0, from + (to - from) * k));
|
||||
if (k >= 1) { clearInterval(iv); resolve(); }
|
||||
}, 30);
|
||||
});
|
||||
}
|
||||
|
||||
// Reconcile the audio layer to render.audio: crossfade the active element to the
|
||||
// new url (or fade to silence for off). White-noise and soundtrack are the same
|
||||
// mechanism — only the url differs; an unchanged url is a no-op so altitude
|
||||
// re-rolls on the same scale don't restart the bed.
|
||||
async function applyAudio(audio) {
|
||||
const url = audio ? audio.url : null;
|
||||
if (url === audUrl) return;
|
||||
audUrl = url;
|
||||
if (!audioReady) return; // deferred until the start gesture (§8)
|
||||
if (!url) { // fade current out to silence
|
||||
await fadeVolume(audActive, 0, XFADE_MS);
|
||||
audActive.pause();
|
||||
return;
|
||||
}
|
||||
audIdle.src = mediaUrl(url); // absolute /media/... passes through mediaUrl
|
||||
audIdle.volume = 0;
|
||||
audIdle.play().catch(() => {}); // start (fire-and-forget — awaiting can hang)
|
||||
await Promise.all([fadeVolume(audIdle, 1, XFADE_MS), fadeVolume(audActive, 0, XFADE_MS)]);
|
||||
audActive.pause();
|
||||
[audActive, audIdle] = [audIdle, audActive]; // swap roles
|
||||
}
|
||||
|
||||
// The current altitude's soundtrack url (the ring carries each scale's `audio`),
|
||||
// resolved client-side so the start gesture can unlock the elements synchronously.
|
||||
function soundtrackUrl() {
|
||||
const s = ring && ring.scales[ringIndex];
|
||||
return s && s.audio ? "/media/audio/" + s.audio : null;
|
||||
}
|
||||
|
||||
// Browsers block autoplay until a user gesture. The renderer is launched
|
||||
// full-screen by the operator; a one-time tap unlocks audio, then it follows
|
||||
// state (audio spec §8 — the documented launch step).
|
||||
function showStartGesture() {
|
||||
if (audioReady || document.getElementById("start-gesture")) return;
|
||||
const ov = document.createElement("div");
|
||||
ov.id = "start-gesture";
|
||||
ov.textContent = "▶ tap to start sound";
|
||||
ov.addEventListener("click", () => {
|
||||
audioReady = true;
|
||||
ov.remove();
|
||||
// Safari unlocks an <audio> element ONLY when play() is called inside the
|
||||
// user gesture itself — not in a later async continuation (Chrome is lenient;
|
||||
// Safari/iOS is not). Prime BOTH A/B elements now, synchronously, at volume 0,
|
||||
// so the async applyAudio() crossfades that follow are allowed to play.
|
||||
const url = soundtrackUrl();
|
||||
if (url) for (const el of [audA, audB]) {
|
||||
try {
|
||||
el.src = mediaUrl(url);
|
||||
el.volume = 0;
|
||||
const pr = el.play();
|
||||
if (pr) pr.then(() => el.pause()).catch(() => {});
|
||||
} catch (_) { /* priming is best-effort */ }
|
||||
}
|
||||
audUrl = null; // force the next applyAudio to (re)load
|
||||
update(); // re-apply current state, now with audio
|
||||
}, { once: true });
|
||||
document.body.appendChild(ov);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
devLiveReload();
|
||||
try { initPaint(); } catch (e) { paintOK = false; paint.style.display = "none"; showError("WebGL init: " + e.message); }
|
||||
@@ -903,9 +999,11 @@ async function main() {
|
||||
buildDial(); // draw the altitude knob from the ring's scales
|
||||
initDev(); // wire the Dev Mode toggle + pool picker (reads persisted state)
|
||||
renderScaleReadout();
|
||||
for (const id of ["content", "left", "right", "mood"]) {
|
||||
$(id).addEventListener("input", debounced);
|
||||
}
|
||||
// Sliders stream on "input"; the Video/Audio toggles fire on "change" (matches
|
||||
// the Dev Mode switch and is the reliable checkbox event across browsers).
|
||||
for (const id of ["left", "right", "mood"]) $(id).addEventListener("input", debounced);
|
||||
for (const id of ["visual", "audio"]) $(id).addEventListener("change", debounced);
|
||||
showStartGesture(); // one-time tap unlocks audio (browser autoplay policy, §8)
|
||||
// Altitude knob: drag to turn (commit detents on release), scroll to step, tap a label to jump.
|
||||
dial.addEventListener("pointerdown", onDialDown);
|
||||
window.addEventListener("pointermove", onDialMove);
|
||||
|
||||
+19
-10
@@ -12,6 +12,8 @@
|
||||
<section class="stage" id="stage">
|
||||
<div class="screen">
|
||||
<video id="vid" loop muted playsinline></video>
|
||||
<audio id="audA" loop preload="auto"></audio>
|
||||
<audio id="audB" loop preload="auto"></audio>
|
||||
<canvas id="paint"></canvas>
|
||||
<div id="tint"></div>
|
||||
<svg id="overlay" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
|
||||
@@ -22,16 +24,17 @@
|
||||
|
||||
<section class="panel">
|
||||
<fieldset>
|
||||
<legend>Content dial</legend>
|
||||
<select id="content">
|
||||
<option value="video">video</option>
|
||||
<option value="audio_video">audio + video</option>
|
||||
<option value="music_video">music + video</option>
|
||||
<option value="off">off (black)</option>
|
||||
<option value="white_noise">white noise (no video)</option>
|
||||
<option value="music">music (no video)</option>
|
||||
<option value="audio_track">audio track (no video)</option>
|
||||
</select>
|
||||
<legend>Output</legend>
|
||||
<label class="dev-switch" for="visual">
|
||||
<input type="checkbox" id="visual" checked />
|
||||
<span class="dev-switch-track"><span class="dev-switch-thumb"></span></span>
|
||||
<span class="dev-switch-label">Video</span>
|
||||
</label>
|
||||
<label class="dev-switch" for="audio">
|
||||
<input type="checkbox" id="audio" />
|
||||
<span class="dev-switch-track"><span class="dev-switch-thumb"></span></span>
|
||||
<span class="dev-switch-label">Audio</span>
|
||||
</label>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
@@ -83,6 +86,12 @@
|
||||
<legend>Cache & playback</legend>
|
||||
<dl id="dev-stats" class="dev-dl"></dl>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>Review galleries</legend>
|
||||
<a class="dev-link" href="/review.html" target="_blank" rel="noopener">🎞️ Clip pool review</a>
|
||||
<a class="dev-link" href="/review_audio.html" target="_blank" rel="noopener">🔊 Audio pool review</a>
|
||||
</fieldset>
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
body { margin: 0; font: 14px/1.4 system-ui, sans-serif; background: #111; color: #eee; }
|
||||
header { padding: 0.6rem 1rem; background: #000; }
|
||||
h1 { font-size: 1rem; margin: 0; font-weight: 600; }
|
||||
main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; }
|
||||
.stage { flex: 1 1 640px; }
|
||||
main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; align-items: flex-start; }
|
||||
/* Stage stays pinned in view while the right pane scrolls on its own. */
|
||||
.stage { flex: 1 1 640px; position: sticky; top: 1rem; }
|
||||
.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000;
|
||||
border-radius: 6px; overflow: hidden; }
|
||||
#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; }
|
||||
@@ -39,9 +40,18 @@ main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; }
|
||||
.hud-chip-text.measure { fill: #ffd79a; }
|
||||
.hud-conf { fill: #6cf; opacity: 0.8; }
|
||||
.hud-status { fill: #8fdcff; font: 2.4px monospace; opacity: 0.9; letter-spacing: 0.15px; }
|
||||
.black { position: absolute; inset: 0; background: #000; opacity: 1; transition: opacity 200ms ease; }
|
||||
/* z-index + own compositing layer: the <video> and WebGL <canvas> are GPU-
|
||||
composited layers that can paint ABOVE a plain auto-z sibling (Safari/Chrome
|
||||
with a real GPU) despite DOM order — so the black cover needs an explicit
|
||||
stacking order above them to actually blank the screen. */
|
||||
.black { position: absolute; inset: 0; background: #000; opacity: 1;
|
||||
z-index: 50; transform: translateZ(0); transition: opacity 200ms ease; }
|
||||
.hidden { display: none; }
|
||||
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem; }
|
||||
/* Stack the two Output toggles (Video / Audio) with a little breathing room. */
|
||||
.dev-switch + .dev-switch { margin-top: 0.45rem; }
|
||||
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem;
|
||||
max-height: calc(100vh - 2rem); overflow-y: auto;
|
||||
position: sticky; top: 1rem; }
|
||||
fieldset { border: 1px solid #333; border-radius: 6px; }
|
||||
legend { color: #9af; padding: 0 0.4rem; }
|
||||
label { display: block; margin: 0.4rem 0; }
|
||||
@@ -87,6 +97,10 @@ input[type=range], select { width: 100%; }
|
||||
.dev-btn { margin-top: 0.5rem; width: 100%; padding: 0.4rem; cursor: pointer;
|
||||
background: #182028; color: #cde; border: 1px solid #2c3c5c; border-radius: 4px; }
|
||||
.dev-btn:hover { background: #20303f; }
|
||||
.dev-link { display: block; margin-top: 0.4rem; padding: 0.4rem; text-align: center;
|
||||
background: #182028; color: #cde; border: 1px solid #2c3c5c; border-radius: 4px;
|
||||
text-decoration: none; }
|
||||
.dev-link:hover { background: #20303f; }
|
||||
.dev-dl { margin: 0; display: grid; grid-template-columns: auto 1fr; gap: 0.15rem 0.6rem; font-size: 12px; }
|
||||
.dev-dl dt { color: #789; }
|
||||
.dev-dl dd { margin: 0; color: #dde; word-break: break-word; }
|
||||
@@ -100,3 +114,13 @@ input[type=range], select { width: 100%; }
|
||||
.dev-anno .anno-key.measure { color: #ffd79a; }
|
||||
.dev-anno .anno-meta { color: #678; }
|
||||
.dev-anno .anno-empty { color: #567; font-style: italic; }
|
||||
|
||||
/* One-time tap-to-start overlay — unlocks audio under the browser autoplay
|
||||
policy (audio spec §8). Removed after the first gesture. */
|
||||
#start-gesture {
|
||||
position: fixed; inset: 0; z-index: 10000;
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
background: rgba(0, 0, 0, 0.82); color: #fff;
|
||||
font: 600 28px/1.2 system-ui, sans-serif; letter-spacing: 0.02em;
|
||||
cursor: pointer; user-select: none;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
"""Playwright E2E for the audio layer (audio spec §9). Skips cleanly when
|
||||
Playwright or its browser binary is absent (the headless dev box) — the unit +
|
||||
contract tiers cover the logic; this asserts the rendered <audio> element src and
|
||||
playback survives a Visual fade. Runs at the PPE stage where a browser exists.
|
||||
|
||||
Install: pip install -e '.[e2e]' && python -m playwright install chromium
|
||||
"""
|
||||
import socket
|
||||
import threading
|
||||
import time
|
||||
from contextlib import closing
|
||||
|
||||
import pytest
|
||||
|
||||
# Skip the whole module unless Playwright is importable.
|
||||
pytest.importorskip("playwright.sync_api")
|
||||
|
||||
|
||||
def _free_port() -> int:
|
||||
with closing(socket.socket()) as s:
|
||||
s.bind(("127.0.0.1", 0))
|
||||
return s.getsockname()[1]
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_url():
|
||||
import uvicorn
|
||||
|
||||
from simulator.app import app
|
||||
|
||||
port = _free_port()
|
||||
config = uvicorn.Config(app, host="127.0.0.1", port=port, log_level="error")
|
||||
server = uvicorn.Server(config)
|
||||
thread = threading.Thread(target=server.run, daemon=True)
|
||||
thread.start()
|
||||
for _ in range(50):
|
||||
if server.started:
|
||||
break
|
||||
time.sleep(0.1)
|
||||
if not server.started:
|
||||
pytest.skip("uvicorn did not start")
|
||||
yield f"http://127.0.0.1:{port}"
|
||||
server.should_exit = True
|
||||
thread.join(timeout=5)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def page(app_url):
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
with sync_playwright() as p:
|
||||
try:
|
||||
# Headless Chromium blocks media playback without a real audio device;
|
||||
# this flag lets play() actually start so `paused` is meaningful (the
|
||||
# spec §9 asserts playback via element src/paused, not a waveform).
|
||||
browser = p.chromium.launch(args=["--autoplay-policy=no-user-gesture-required"])
|
||||
except Exception as exc: # no browser binary installed
|
||||
pytest.skip(f"chromium not available: {exc}")
|
||||
pg = browser.new_page()
|
||||
pg.goto(app_url)
|
||||
pg.click("#start-gesture") # unlock autoplay
|
||||
yield pg
|
||||
browser.close()
|
||||
|
||||
|
||||
def test_audio_toggle_plays_the_current_scale_soundtrack(page):
|
||||
# Audio on (toggle) → the current altitude's soundtrack (cosmos at index 0) plays
|
||||
page.click("label[for='audio'] .dev-switch-track")
|
||||
page.wait_for_function(
|
||||
"['audA','audB'].some(id => {"
|
||||
" const a = document.getElementById(id);"
|
||||
" return /\\/media\\/audio\\/.+\\.mp3/.test(a.src) && !a.paused;"
|
||||
"})"
|
||||
)
|
||||
|
||||
|
||||
def test_video_toggle_off_blanks_the_screen_and_hides_layers(page):
|
||||
page.click("label[for='visual'] .dev-switch-track")
|
||||
page.wait_for_selector("#black:not(.hidden)") # black cover shown
|
||||
# the video layers themselves are hidden too (GPU-composite-proof blanking);
|
||||
# wait for the opacity transition to settle to 0
|
||||
page.wait_for_function(
|
||||
"['vid','paint'].every(id => getComputedStyle(document.getElementById(id)).opacity === '0')"
|
||||
)
|
||||
|
||||
|
||||
def test_audio_survives_video_off(page):
|
||||
page.click("label[for='audio'] .dev-switch-track")
|
||||
page.wait_for_function(
|
||||
"['audA','audB'].some(id => !document.getElementById(id).paused "
|
||||
"&& document.getElementById(id).src)"
|
||||
)
|
||||
page.click("label[for='visual'] .dev-switch-track")
|
||||
page.wait_for_selector("#black:not(.hidden)")
|
||||
playing = page.evaluate(
|
||||
"['audA','audB'].some(id => "
|
||||
"{const a = document.getElementById(id); return !a.paused && !!a.src;})"
|
||||
)
|
||||
assert playing is True # audio keeps playing with video off
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Unit tests for the pure audio ffmpeg arg builders (no ffmpeg run), plus an
|
||||
opt-in integration test that actually synthesizes a noise bed when ffmpeg is
|
||||
present."""
|
||||
|
||||
import shutil
|
||||
|
||||
import pytest
|
||||
|
||||
from tools.pipeline.audio_ops import audio_loop_args, loudnorm_args, white_noise_args
|
||||
|
||||
|
||||
def test_loop_args_mirror_the_video_crossfade_recipe():
|
||||
args = audio_loop_args("in.mp3", "out.mp3", duration=30.0, overlap=2.0, ff="FF")
|
||||
assert args[0] == "FF"
|
||||
joined = " ".join(args)
|
||||
# tail crossfades over head, then concats the middle — the audio analogue of
|
||||
# crossfade_loop_args. Output length = duration - overlap.
|
||||
assert "acrossfade=d=2.0" in joined
|
||||
assert "atrim=0:2.0" in joined # head
|
||||
assert "atrim=28.0:30.0" in joined # tail (d-overlap : d)
|
||||
assert "concat=n=2:v=0:a=1" in joined
|
||||
assert "-map" in args and "[out]" in args
|
||||
assert "-q:a" in args and "4" in args
|
||||
|
||||
|
||||
def test_loop_args_reject_overlap_past_half():
|
||||
with pytest.raises(ValueError):
|
||||
audio_loop_args("in.mp3", "out.mp3", duration=10.0, overlap=5.0)
|
||||
|
||||
|
||||
def test_loudnorm_args_carry_the_locked_targets():
|
||||
args = loudnorm_args("in.mp3", "out.mp3", ff="FF")
|
||||
joined = " ".join(args)
|
||||
assert "loudnorm=I=-18.0:TP=-1.5:LRA=11.0" in joined
|
||||
assert args[0] == "FF" and args[-1] == "out.mp3"
|
||||
|
||||
|
||||
def test_white_noise_args_are_pink_and_clean():
|
||||
args = white_noise_args("noise.mp3", duration=60.0, color="pink", ff="FF")
|
||||
joined = " ".join(args)
|
||||
assert "anoisesrc=color=pink" in joined
|
||||
assert "-t" in args and "60.0" in args
|
||||
assert args[-1] == "noise.mp3"
|
||||
|
||||
|
||||
def test_white_noise_rejects_unknown_color():
|
||||
with pytest.raises(ValueError):
|
||||
white_noise_args("noise.mp3", color="ultraviolet")
|
||||
|
||||
|
||||
def _have_ffmpeg() -> bool:
|
||||
if shutil.which("ffmpeg"):
|
||||
return True
|
||||
try:
|
||||
import imageio_ffmpeg # noqa: F401
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
@pytest.mark.skipif(not _have_ffmpeg(), reason="no ffmpeg available")
|
||||
def test_generate_white_noise_writes_a_playable_loop(tmp_path):
|
||||
from tools.pipeline.audio_run import generate_white_noise
|
||||
out = generate_white_noise(tmp_path / "pink.mp3", duration=2.0)
|
||||
assert out.exists() and out.stat().st_size > 0
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Unit tests for the pure Visual×Audio resolver (audio spec §4/§6)."""
|
||||
import pytest
|
||||
|
||||
from player.audio import (
|
||||
AUDIO_SOURCES,
|
||||
VISUAL_POSITIONS,
|
||||
AudioResolution,
|
||||
resolve_audio,
|
||||
resolve_audio_url,
|
||||
resolve_visual,
|
||||
)
|
||||
|
||||
def test_visual_on_off():
|
||||
assert resolve_visual("on") is True
|
||||
assert resolve_visual("off") is False
|
||||
|
||||
|
||||
def test_visual_rejects_unknown():
|
||||
with pytest.raises(ValueError):
|
||||
resolve_visual("flicker")
|
||||
|
||||
|
||||
def test_visual_positions_are_on_off():
|
||||
assert VISUAL_POSITIONS == frozenset({"on", "off"})
|
||||
|
||||
|
||||
def test_audio_sources_are_off_and_soundtrack():
|
||||
assert AUDIO_SOURCES == frozenset({"off", "soundtrack"})
|
||||
|
||||
|
||||
def test_soundtrack_resolves_to_the_current_scales_asset():
|
||||
url = resolve_audio_url("soundtrack", scale_audio="coast/waves.loop.mp3")
|
||||
assert url == "/media/audio/coast/waves.loop.mp3"
|
||||
|
||||
|
||||
def test_off_resolves_to_silence():
|
||||
assert resolve_audio_url("off", scale_audio="coast/waves.loop.mp3") is None
|
||||
|
||||
|
||||
def test_soundtrack_with_no_scale_asset_is_silent():
|
||||
assert resolve_audio_url("soundtrack", scale_audio="") is None
|
||||
|
||||
|
||||
def test_resolve_audio_marks_only_soundtrack_altitude_coupled():
|
||||
r = resolve_audio("soundtrack", scale_audio="reef/soundscape.loop.mp3")
|
||||
assert isinstance(r, AudioResolution)
|
||||
assert r.source == "soundtrack" and r.altitude_coupled is True
|
||||
assert r.url == "/media/audio/reef/soundscape.loop.mp3"
|
||||
assert resolve_audio("off", scale_audio="x/y.mp3").altitude_coupled is False
|
||||
|
||||
|
||||
def test_resolve_audio_url_rejects_unknown_source():
|
||||
with pytest.raises(ValueError):
|
||||
resolve_audio_url("white_noise", scale_audio="") # deferred, not a v1 source
|
||||
|
||||
|
||||
def test_music_and_white_noise_are_not_v1_sources():
|
||||
assert "music" not in AUDIO_SOURCES
|
||||
assert "white_noise" not in AUDIO_SOURCES
|
||||
@@ -1,34 +0,0 @@
|
||||
import pytest
|
||||
|
||||
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
|
||||
|
||||
|
||||
def test_audio_sources_are_the_four_distinct_sources():
|
||||
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
# The §6 table, row by row: position -> (audio_source, video).
|
||||
@pytest.mark.parametrize(
|
||||
"position,audio_source,video",
|
||||
[
|
||||
("off", "none", False),
|
||||
("white_noise", "white_noise", False),
|
||||
("music", "music", False),
|
||||
("audio_track", "audio_track", False),
|
||||
("video", "none", True),
|
||||
("music_video", "music", True),
|
||||
("audio_video", "audio_track", True),
|
||||
],
|
||||
)
|
||||
def test_resolve_content_matches_spec_table(position, audio_source, video):
|
||||
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
|
||||
|
||||
|
||||
def test_off_is_void_state_black_and_silent():
|
||||
r = resolve_content("off")
|
||||
assert r.video is False and r.audio_source == "none"
|
||||
|
||||
|
||||
def test_resolve_content_rejects_unknown_position():
|
||||
with pytest.raises(ValueError):
|
||||
resolve_content("bogus")
|
||||
@@ -1,27 +1,36 @@
|
||||
import pytest
|
||||
|
||||
from player.audio import AUDIO_SOURCES, VISUAL_POSITIONS
|
||||
from player.controls import (
|
||||
Controls,
|
||||
CONTENT_POSITIONS,
|
||||
ControlsError,
|
||||
validate_controls,
|
||||
parse_controls,
|
||||
)
|
||||
|
||||
|
||||
def test_content_positions_are_the_seven_from_the_spec():
|
||||
assert CONTENT_POSITIONS == frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
def test_visual_and_audio_enums_are_the_v1_positions():
|
||||
assert VISUAL_POSITIONS == frozenset({"on", "off"})
|
||||
assert AUDIO_SOURCES == frozenset({"off", "soundtrack"})
|
||||
|
||||
|
||||
def test_valid_controls_pass_validation():
|
||||
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
|
||||
c = Controls(visual="on", audio="soundtrack", left=0, right=4, dark=2, light=2,
|
||||
volume=3, brightness=4)
|
||||
validate_controls(c) # does not raise
|
||||
|
||||
|
||||
def test_invalid_content_position_rejected():
|
||||
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
def test_invalid_visual_position_rejected():
|
||||
c = Controls(visual="flicker", audio="off", left=0, right=0, dark=0, light=0,
|
||||
volume=0, brightness=0)
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(c)
|
||||
|
||||
|
||||
def test_invalid_audio_source_rejected():
|
||||
# "music" is a reserved/deferred dial position, not a v1 source
|
||||
c = Controls(visual="on", audio="music", left=0, right=0, dark=0, light=0,
|
||||
volume=0, brightness=0)
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(c)
|
||||
|
||||
@@ -29,7 +38,8 @@ def test_invalid_content_position_rejected():
|
||||
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
|
||||
@pytest.mark.parametrize("bad", [-1, 5, True])
|
||||
def test_out_of_range_or_non_int_knob_rejected(field, bad):
|
||||
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
kwargs = dict(visual="off", audio="off", left=0, right=0, dark=0, light=0,
|
||||
volume=0, brightness=0)
|
||||
kwargs[field] = bad
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(Controls(**kwargs))
|
||||
@@ -37,17 +47,18 @@ def test_out_of_range_or_non_int_knob_rejected(field, bad):
|
||||
|
||||
def test_parse_controls_from_mapping():
|
||||
c = parse_controls(
|
||||
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
|
||||
{"visual": "on", "audio": "soundtrack", "left": 1, "right": 2, "dark": 3,
|
||||
"light": 0, "volume": 2, "brightness": 1}
|
||||
)
|
||||
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
|
||||
assert c == Controls("on", "soundtrack", 1, 2, 3, 0, 2, 1)
|
||||
|
||||
|
||||
def test_parse_controls_rejects_unknown_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
|
||||
parse_controls({"visual": "off", "audio": "off", "left": 0, "right": 0, "dark": 0,
|
||||
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
|
||||
|
||||
|
||||
def test_parse_controls_rejects_missing_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0})
|
||||
parse_controls({"visual": "off", "left": 0})
|
||||
|
||||
@@ -240,3 +240,10 @@ def test_pick_clip_id_clamps_out_of_range_r():
|
||||
assert pick_clip_id(s, -5.0) == "a" # below 0 clamps to first
|
||||
assert pick_clip_id(s, 1.0) == "c" # 1.0 must not index past the end
|
||||
assert pick_clip_id(s, 999.0) == "c"
|
||||
|
||||
|
||||
def test_scale_carries_an_optional_audio_path():
|
||||
s = Scale(id="coast", clip_id="coast_birdrock", audio="coast/waves.loop.mp3")
|
||||
assert s.audio == "coast/waves.loop.mp3"
|
||||
# back-compat: audio defaults to empty
|
||||
assert Scale(id="x", clip_id="x").audio == ""
|
||||
|
||||
+35
-25
@@ -14,47 +14,57 @@ class FakeClip:
|
||||
LIB = [FakeClip("base-a"), FakeClip("base-b")]
|
||||
|
||||
|
||||
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
|
||||
return Controls(content, left, right, dark, light, volume, brightness)
|
||||
def _controls(visual="on", audio="off", left=0, right=0, dark=0, light=0,
|
||||
volume=2, brightness=2):
|
||||
return Controls(visual, audio, left, right, dark, light, volume, brightness)
|
||||
|
||||
|
||||
def test_first_update_to_video_fades_in_from_black():
|
||||
p = Player(LIB)
|
||||
t = p.update(_controls(content="video"))
|
||||
t = p.update(_controls(visual="on"))
|
||||
assert t.kind == TransitionKind.FADE_FROM_BLACK
|
||||
assert t.playback.clip_id == "base-a"
|
||||
assert t.playback.content.video is True
|
||||
assert t.playback.video is True
|
||||
|
||||
|
||||
def test_off_from_video_fades_to_black_and_silences():
|
||||
def test_visual_off_from_video_fades_to_black():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video"))
|
||||
t = p.update(_controls(content="off"))
|
||||
p.update(_controls(visual="on"))
|
||||
t = p.update(_controls(visual="off"))
|
||||
assert t.kind == TransitionKind.FADE_TO_BLACK
|
||||
assert t.playback.clip_id is None
|
||||
assert t.playback.content.audio_source == "none"
|
||||
assert t.playback.video is False
|
||||
assert t.playback.audio_source == "off"
|
||||
|
||||
|
||||
def test_audio_survives_visual_off():
|
||||
# soundtrack + black is reachable (a gap the bundled dial omitted)
|
||||
p = Player(LIB)
|
||||
t = p.update(_controls(visual="off", audio="soundtrack"))
|
||||
assert t.playback.video is False
|
||||
assert t.playback.audio_source == "soundtrack"
|
||||
|
||||
|
||||
def test_no_change_yields_none_transition():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=1))
|
||||
t = p.update(_controls(content="video", left=1))
|
||||
p.update(_controls(visual="on", left=1))
|
||||
t = p.update(_controls(visual="on", left=1))
|
||||
assert t.kind == TransitionKind.NONE
|
||||
|
||||
|
||||
def test_grade_change_is_a_live_update_not_a_crossfade():
|
||||
# design §4.3: the Dark/Light grade is a continuous runtime op
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", dark=0, light=0))
|
||||
t = p.update(_controls(content="video", dark=4, light=0))
|
||||
p.update(_controls(visual="on", dark=0, light=0))
|
||||
t = p.update(_controls(visual="on", dark=4, light=0))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_overlay_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=0))
|
||||
t = p.update(_controls(content="video", left=4))
|
||||
p.update(_controls(visual="on", left=0))
|
||||
t = p.update(_controls(visual="on", left=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.overlay.level == 4
|
||||
|
||||
@@ -63,8 +73,8 @@ def test_dream_change_is_a_live_update_not_a_crossfade():
|
||||
# Right-axis dream reframe (session 0013): the Right dream is a deterministic
|
||||
# LIVE filter now, so changing it no longer crossfades — only a clip swap does.
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", right=0))
|
||||
t = p.update(_controls(content="video", right=4))
|
||||
p.update(_controls(visual="on", right=0))
|
||||
t = p.update(_controls(visual="on", right=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.dream.strength == 4
|
||||
assert t.playback.plan.dream.intensity == 1.0
|
||||
@@ -72,43 +82,43 @@ def test_dream_change_is_a_live_update_not_a_crossfade():
|
||||
|
||||
def test_volume_only_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", volume=1))
|
||||
t = p.update(_controls(content="video", volume=4))
|
||||
p.update(_controls(visual="on", volume=1))
|
||||
t = p.update(_controls(visual="on", volume=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.volume == 4
|
||||
|
||||
|
||||
def test_audio_source_change_while_black_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="white_noise"))
|
||||
t = p.update(_controls(content="music"))
|
||||
p.update(_controls(visual="off", audio="off"))
|
||||
t = p.update(_controls(visual="off", audio="soundtrack"))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.content.audio_source == "music"
|
||||
assert t.playback.audio_source == "soundtrack"
|
||||
|
||||
|
||||
def test_injected_base_chooser_is_used():
|
||||
p = Player(LIB, choose_base=lambda lib: lib[1])
|
||||
t = p.update(_controls(content="video"))
|
||||
t = p.update(_controls(visual="on"))
|
||||
assert t.playback.clip_id == "base-b"
|
||||
|
||||
|
||||
def test_empty_library_with_video_raises():
|
||||
p = Player([])
|
||||
with pytest.raises(ValueError):
|
||||
p.update(_controls(content="video"))
|
||||
p.update(_controls(visual="on"))
|
||||
|
||||
|
||||
def test_off_with_empty_library_is_fine():
|
||||
p = Player([])
|
||||
# levels at 0 match the initial black state, so this is a no-op transition
|
||||
t = p.update(_controls(content="off", volume=0, brightness=0))
|
||||
t = p.update(_controls(visual="off", volume=0, brightness=0))
|
||||
assert t.kind == TransitionKind.NONE # already black at init
|
||||
assert t.playback.clip_id is None
|
||||
|
||||
|
||||
def test_off_with_levels_from_black_is_a_live_update():
|
||||
p = Player([])
|
||||
t = p.update(_controls(content="off", volume=3, brightness=2))
|
||||
t = p.update(_controls(visual="off", volume=3, brightness=2))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE # black->black, levels set
|
||||
assert t.playback.clip_id is None
|
||||
assert t.playback.volume == 3
|
||||
|
||||
+54
-12
@@ -28,8 +28,9 @@ def client(manifest_path):
|
||||
return TestClient(create_app(manifest_path=manifest_path))
|
||||
|
||||
|
||||
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
|
||||
return dict(content=content, left=left, right=right, dark=dark,
|
||||
def _controls(visual="on", audio="off", left=0, right=0, dark=0, light=0,
|
||||
volume=2, brightness=2):
|
||||
return dict(visual=visual, audio=audio, left=left, right=right, dark=dark,
|
||||
light=light, volume=volume, brightness=brightness)
|
||||
|
||||
|
||||
@@ -41,13 +42,40 @@ def test_alteration_returns_the_engine_plan(client):
|
||||
assert data["plan"]["dream"]["strength"] == 2
|
||||
assert data["plan"]["dream"]["intensity"] == 0.5
|
||||
assert data["plan"]["grade"]["tone"] == -1.0
|
||||
assert data["content"]["video"] is True
|
||||
assert data["render"]["video"]["shown"] is True
|
||||
|
||||
|
||||
def test_alteration_honors_off_as_black(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(content="off")})
|
||||
data = resp.json()
|
||||
assert data["content"]["video"] is False
|
||||
def test_alteration_returns_render_video_and_audio(client):
|
||||
body = client.post("/api/alteration", json={"controls": _controls()}).json()
|
||||
assert body["render"]["video"]["shown"] is True
|
||||
assert body["render"]["audio"] == {"source": "off", "url": None, "altitude_coupled": False}
|
||||
|
||||
|
||||
def test_alteration_visual_off_hides_video(client):
|
||||
body = client.post("/api/alteration", json={"controls": _controls(visual="off")}).json()
|
||||
assert body["render"]["video"]["shown"] is False
|
||||
|
||||
|
||||
def test_alteration_off_is_silent(ring_client):
|
||||
body = ring_client.post("/api/alteration", json={
|
||||
"controls": _controls(audio="off"), "altitude_index": 0,
|
||||
}).json()
|
||||
assert body["render"]["audio"] == {"source": "off", "url": None, "altitude_coupled": False}
|
||||
|
||||
|
||||
def test_alteration_soundtrack_couples_to_the_given_altitude(ring_client):
|
||||
# ring fixture order: cosmos(0), forest(1), abyss(2) — each carries its audio
|
||||
body = ring_client.post("/api/alteration", json={
|
||||
"controls": _controls(audio="soundtrack"), "altitude_index": 0,
|
||||
}).json()
|
||||
a = body["render"]["audio"]
|
||||
assert a["source"] == "soundtrack" and a["altitude_coupled"] is True
|
||||
assert a["url"] == "/media/audio/cosmos/pillars.loop.mp3"
|
||||
# a different altitude resolves a different soundtrack url
|
||||
body2 = ring_client.post("/api/alteration", json={
|
||||
"controls": _controls(audio="soundtrack"), "altitude_index": 2,
|
||||
}).json()
|
||||
assert body2["render"]["audio"]["url"] == "/media/audio/abyss/whale.loop.mp3"
|
||||
|
||||
|
||||
def test_alteration_accepts_calibration(client):
|
||||
@@ -62,8 +90,13 @@ def test_alteration_rejects_out_of_range_knob(client):
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
def test_alteration_rejects_bad_content(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(content="banana")})
|
||||
def test_alteration_rejects_bad_audio(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(audio="music")})
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
def test_alteration_rejects_bad_visual(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(visual="dim")})
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
@@ -162,9 +195,9 @@ def ring_manifest_path(tmp_path):
|
||||
],
|
||||
"ring": {
|
||||
"scales": [
|
||||
{"id": "cosmos", "clip_id": "cosmos"},
|
||||
{"id": "forest", "clip_id": "forest"},
|
||||
{"id": "abyss", "clip_id": "abyss"},
|
||||
{"id": "cosmos", "clip_id": "cosmos", "audio": "cosmos/pillars.loop.mp3"},
|
||||
{"id": "forest", "clip_id": "forest", "audio": "forest/amb.loop.mp3"},
|
||||
{"id": "abyss", "clip_id": "abyss", "audio": "abyss/whale.loop.mp3"},
|
||||
],
|
||||
"transitions": [
|
||||
{"file": "transitions/cosmos-forest.mp4", "model": "placeholder"},
|
||||
@@ -190,6 +223,15 @@ def test_ring_returns_scales_and_transitions(ring_client):
|
||||
assert len(data["transitions"]) == 3
|
||||
|
||||
|
||||
def test_ring_exposes_each_scales_audio(ring_client):
|
||||
body = ring_client.get("/api/ring").json()
|
||||
by_id = {s["id"]: s for s in body["scales"]}
|
||||
# every scale advertises its soundtrack path (audio spec §5.1)
|
||||
assert by_id["cosmos"]["audio"] == "cosmos/pillars.loop.mp3"
|
||||
assert by_id["abyss"]["audio"].endswith(".mp3")
|
||||
assert "audio" in by_id["forest"]
|
||||
|
||||
|
||||
def test_ring_advance_inward_one_step(ring_client):
|
||||
resp = ring_client.post("/api/ring/advance", json={"from_index": 0, "delta": 1})
|
||||
assert resp.status_code == 200
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
"""Pure ffmpeg argument builders (no I/O) for the audio production pass.
|
||||
|
||||
The audio analogue of tools/pipeline/ffmpeg_ops.py: each function returns a
|
||||
list[str] ready for ffmpeg, so command construction is unit-testable without
|
||||
running ffmpeg. tools/pipeline/audio_run.py executes them. The ffmpeg binary
|
||||
name is injected (default "ffmpeg")."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
_NOISE_COLORS = frozenset({"white", "pink", "brown", "blue", "violet"})
|
||||
|
||||
|
||||
def audio_loop_args(src, dst, *, duration: float, overlap: float,
|
||||
ff: str = "ffmpeg") -> list[str]:
|
||||
"""Make `src` loop seamlessly by crossfading its tail over its head — the
|
||||
audio mirror of ffmpeg_ops.crossfade_loop_args. Output length = duration -
|
||||
overlap. Requires overlap < duration/2 so a non-empty middle remains."""
|
||||
if overlap <= 0 or overlap >= duration / 2:
|
||||
raise ValueError(f"overlap {overlap} must be in (0, duration/2={duration / 2})")
|
||||
d, o = duration, overlap
|
||||
fc = (
|
||||
f"[0:a]atrim=0:{o},asetpts=N/SR/TB[head];"
|
||||
f"[0:a]atrim={o}:{d - o},asetpts=N/SR/TB[mid];"
|
||||
f"[0:a]atrim={d - o}:{d},asetpts=N/SR/TB[tail];"
|
||||
f"[tail][head]acrossfade=d={o}:c1=tri:c2=tri[xf];"
|
||||
f"[xf][mid]concat=n=2:v=0:a=1[out]"
|
||||
)
|
||||
return [
|
||||
ff, "-y", "-i", str(src), "-filter_complex", fc,
|
||||
"-map", "[out]", "-c:a", "libmp3lame", "-q:a", "4", str(dst),
|
||||
]
|
||||
|
||||
|
||||
def loudnorm_args(src, dst, *, i: float = -18.0, tp: float = -1.5,
|
||||
lra: float = 11.0, ff: str = "ffmpeg") -> list[str]:
|
||||
"""EBU R128 loudness normalize to the locked installation target so all five
|
||||
soundtracks (and the noise bed) sit at one comfortable level."""
|
||||
return [
|
||||
ff, "-y", "-i", str(src),
|
||||
"-af", f"loudnorm=I={i}:TP={tp}:LRA={lra}",
|
||||
"-c:a", "libmp3lame", "-q:a", "4", str(dst),
|
||||
]
|
||||
|
||||
|
||||
def white_noise_args(dst, *, duration: float = 60.0, color: str = "pink",
|
||||
ff: str = "ffmpeg") -> list[str]:
|
||||
"""Synthesize a calm colored-noise bed (default pink) of `duration` seconds —
|
||||
deterministic, zero licensing. Loops cleanly (steady-state noise has no seam)."""
|
||||
if color not in _NOISE_COLORS:
|
||||
raise ValueError(
|
||||
f"unknown noise color {color!r}; expected one of {sorted(_NOISE_COLORS)}"
|
||||
)
|
||||
return [
|
||||
ff, "-y", "-f", "lavfi",
|
||||
"-i", f"anoisesrc=color={color}:amplitude=0.5:duration={duration}",
|
||||
"-t", str(duration), "-c:a", "libmp3lame", "-q:a", "4", str(dst),
|
||||
]
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Thin runner: probe duration, then run the audio production pass (seamless
|
||||
loop -> loudness normalize) and synthesize the white-noise bed. The pure arg
|
||||
builders (tools.pipeline.audio_ops) are unit-tested; this glue is covered by the
|
||||
opt-in integration test."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from .audio_ops import audio_loop_args, loudnorm_args, white_noise_args
|
||||
from .run import resolve_ffmpeg
|
||||
|
||||
|
||||
def _probe_audio_duration(src, ff: str) -> float:
|
||||
"""Audio length in seconds via ffmpeg (no system ffprobe): decode to null and
|
||||
read the last reported time off stderr."""
|
||||
proc = subprocess.run(
|
||||
[ff, "-i", str(src), "-f", "null", "-"],
|
||||
capture_output=True, text=True,
|
||||
)
|
||||
times = re.findall(r"time=(\d+):(\d+):(\d+\.\d+)", proc.stderr)
|
||||
if not times:
|
||||
raise ValueError(f"could not probe audio duration of {src}")
|
||||
h, m, s = times[-1]
|
||||
return int(h) * 3600 + int(m) * 60 + float(s)
|
||||
|
||||
|
||||
def _run(args: list[str]) -> None:
|
||||
subprocess.run(args, check=True, capture_output=True)
|
||||
|
||||
|
||||
def process_soundtrack(src, dst, *, overlap: float = 2.0, ff: str | None = None) -> Path:
|
||||
"""Produce a seamless, loudness-normalized loop from a sourced ambience clip.
|
||||
Clips too short to crossfade-loop (<= 2*overlap) are normalized only."""
|
||||
ff = ff or resolve_ffmpeg()
|
||||
src, dst = Path(src), Path(dst)
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
duration = _probe_audio_duration(src, ff)
|
||||
tmp = dst.with_suffix(".loop.tmp.mp3")
|
||||
if duration > 2 * overlap + 0.5:
|
||||
_run(audio_loop_args(src, tmp, duration=duration, overlap=overlap, ff=ff))
|
||||
_run(loudnorm_args(tmp, dst, ff=ff))
|
||||
tmp.unlink(missing_ok=True)
|
||||
else:
|
||||
_run(loudnorm_args(src, dst, ff=ff))
|
||||
return dst
|
||||
|
||||
|
||||
def generate_white_noise(dst, *, duration: float = 60.0, color: str = "pink",
|
||||
ff: str | None = None) -> Path:
|
||||
"""Synthesize + normalize the global white-noise bed."""
|
||||
ff = ff or resolve_ffmpeg()
|
||||
dst = Path(dst)
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = dst.with_suffix(".raw.tmp.mp3")
|
||||
_run(white_noise_args(tmp, duration=duration, color=color, ff=ff))
|
||||
_run(loudnorm_args(tmp, dst, ff=ff))
|
||||
tmp.unlink(missing_ok=True)
|
||||
return dst
|
||||
Reference in New Issue
Block a user