Compare commits

...

25 Commits

Author SHA1 Message Date
BenStullsBets edca8a9615 content(sim): rotating clip pools + progressive tracked labels & emotions
Content-pipeline Increment 2, part 1 (the experience). Implements design §11
(docs/superpowers/specs/2026-06-24-content-pipeline-design.md):

- Rotating pool (§11.1): each ring scale references a POOL of vetted clips
  (docs/content-candidate-pool.md); the player picks a random member on landing.
  player.ring.Scale gains `pool` + pure `pick_clip_id(scale, r)` (injected
  randomness); the pick happens at the API boundary (random.random()), a delta=0
  advance is the initial/re-roll pick. clips.py parses/exposes the pool; the
  client lands on move.target_clip_id. Back-compat: a scale with only clip_id is
  a pool of one.
- Rename ring scale forest -> coast (new land<->sea scale, orbit..reef); ring
  order cosmos -> orbit -> coast -> reef -> abyss -> wrap. Cleanup: dropped the
  leftover local media (cosmos_traverse, orbit_westcoast, old forest/reef, the
  Increment-1 orbit/abyss single bases); new coast-edge transition placeholders.
- Time-windowed tracked labels (§11.2): annotations gain appear/disappear
  (loop-normalized, wraps across the seam); a label shows only while on screen
  and its box tracks the subject. abyss + reef pools authored as test material.
- Progressive LEFT detail tiers (§11.3): per-object `salience` (first shows at
  Left 5-salience) + tiered strings (general -> specific -> scientific -> +fact)
  escalating with the Left knob; replaces flat min_level gating. Strings may be a
  tier list or a static string (back-compat). No engine change (client reads
  overlay.level).
- Progressive RIGHT emotion tiers (§11.4): affect vocabulary escalates basic ->
  compound with the Right knob (dream.strength); appearance still gated by
  min(left,right). No engine change.

New simulator/build_pool_manifest.py holds the hand-authored content and emits
the 19-clip pool manifest (the reproducible baseline). setup_scales_media.py
marked superseded. 251 passed / 4 skipped (+ ring pool + API pool/window tests).
By-eye visual review deferred to the operator (no Chrome on this box).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:05:13 -07:00
BenStullsBets 4270c265e3 claim human-experience-filter-art session 0016 (placeholder) + sessions.json entry 2026-06-24 17:49:11 -07:00
benstull b28f930e11 Merge pull request 'docs: operator-selected content candidate pool' (#15) from content-candidate-pool into main 2026-06-25 00:39:59 +00:00
BenStullsBets 7be4f18e77 docs: operator-selected content candidate pool (rotating-pool model)
Records the 5-scale rotating clip pools the operator selected + vetted from the
candidate gallery (19 clips: cosmos 4, orbit 3, coast 4, reef 5, abyss 3), each
strict-PD/CC-BY-class, word-free, human-free, with source URL + trim window so
they are re-sourceable (media stays gitignored). Captures: the rotating-pool model
(random clip per scale on ring landing), the forest→coast rename, the cosmos
#3-Orion vs #6-Traverse processing mix-up to fix, and the rejected/leftover clips
to clean up. Feeds the next session (Increment 2 expanded: pool wiring + tracked
progressive labels + progressive emotions).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 17:39:54 -07:00
BenStullsBets 6256bc10e4 update sessions/0015/SESSION-0015.0-TRANSCRIPT-2026-06-24T13-16--INPROGRESS.md 2026-06-24 13:29:09 -07:00
BenStullsBets 317a6be6b0 claim human-experience-filter-art session 0015 (placeholder) + sessions.json entry 2026-06-24 13:16:43 -07:00
benstull bd4b65684c Merge pull request 'content(sim): real strict-PD bases for all 5 scales' (#14) from content-real-pd-bases into main 2026-06-24 16:16:37 +00:00
BenStullsBets 6369d0c5ab content(sim): real strict-PD bases for all 5 scales (provenance + license)
Sourced, license-verified, and ran through tools/pipeline/run.py process_clip
(trim → crossfade-loop → 1080p proxy) the five real public-domain neutral bases,
replacing the synthetic placeholders. Media stays gitignored; this records the
real provenance/license + trim window per clip so it is re-sourceable.

- cosmos — NASA/JPL-Caltech, Orion Nebula flythrough (17 U.S.C. §105)
- orbit  — NASA/JSC, Earth from ISS (Expedition 65) (§105)
- forest — NPS Yosemite stock b-roll (PD, no ©)
- reef   — NOAA Fisheries coral b-roll (§105)
- abyss  — NOAA Ocean Exploration comb jelly / ctenophore (§105)

All transcoded in ~2.6 min total (well under the 1-hour budget). Labels/tracks
remain the Increment-1 placeholders (real per-clip authoring is Increment 2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 09:16:21 -07:00
BenStullsBets 4b8ebf11ea add sessions/0014/SESSION-0014.0-TRANSCRIPT-2026-06-24T06-58--2026-06-24T08-29.md + replace placeholder/variant SESSION-0014.0-TRANSCRIPT-2026-06-24T06-58--INPROGRESS.md 2026-06-24 08:30:02 -07:00
BenStullsBets 73e5c4a3c7 docs(spec): stamp content-pipeline design graduated + Increment 1 shipped (session 0014)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 08:28:02 -07:00
benstull bb5ecf09b4 Merge pull request 'feat(content): content pipeline Increment 1 — tooling + annotation tracks + 5-scale ring' (#13) from feature/content-pipeline into main 2026-06-24 15:26:17 +00:00
BenStullsBets e812f41ef1 docs: content-sourcing procedure + roadmap update + setup docstring (content pipeline Increment 1)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 08:19:41 -07:00
BenStullsBets 8789d6a6d5 feat(sim): extend scale ring to 5 (orbit + reef) (content pipeline §5)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 08:15:41 -07:00
BenStullsBets 2933498c51 feat(sim): client annotation-track interpolation + hand-authored forest track (content pipeline §4) 2026-06-24 08:11:40 -07:00
BenStullsBets 02c762fd45 fix(pipeline): probe duration via cv2 (no ffprobe dep); wire MASTER_MAX_W; fps-sync note
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 08:09:01 -07:00
BenStullsBets f1c83c1a9a feat(pipeline): resolve bundled ffmpeg fallback; integration test runs on real forest base
- run.py: add resolve_ffmpeg() mirroring setup_scales_media.py; change
  process_clip ff param to None and resolve at call time
- test_pipeline_integration.py: gate on resolve_ffmpeg() not PATH ffmpeg;
  verify proxy dims with cv2 instead of ffprobe
- ffmpeg_ops.py: add fps= to trim segments in crossfade_loop_args so xfade
  receives CFR input (xfade rejects 1/0 rate from raw trim output)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 08:05:11 -07:00
BenStullsBets 38a1046433 feat(pipeline): runner + opt-in integration test on real forest base (content pipeline §3)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 08:01:25 -07:00
BenStullsBets 60d27aa662 test(pipeline): cover ring-scale append fallback + N=1 self-wrap edge 2026-06-24 07:59:34 -07:00
BenStullsBets dc122fa0bf feat(pipeline): manifest assembly — upsert clip + ring scale/edges (content pipeline §3.6) 2026-06-24 07:57:38 -07:00
BenStullsBets 0fd8b1203d feat(pipeline): strict-PD provenance record (content pipeline §3.1) 2026-06-24 07:57:07 -07:00
BenStullsBets d3b6cd7bbd test(pipeline): cover non-positive overlap guard in crossfade_loop_args 2026-06-24 07:56:03 -07:00
BenStullsBets 4b5fd20374 feat(pipeline): master/proxy transcode arg builder (content pipeline §3.4/§6) 2026-06-24 07:54:36 -07:00
BenStullsBets 7217daeb44 feat(pipeline): crossfade seamless-loop arg builder (content pipeline §3.3) 2026-06-24 07:54:01 -07:00
BenStullsBets 642451925f feat(pipeline): frame stage ffmpeg arg builder (content pipeline §3.2) 2026-06-24 07:53:28 -07:00
Ben Stull daa6bddca6 claim human-experience-filter-art session 0014 (placeholder) + sessions.json entry 2026-06-24 06:59:52 -07:00
27 changed files with 3921 additions and 126 deletions
+3 -1
View File
@@ -6,5 +6,7 @@ media/
.superpowers/
*.egg-info/
# Simulator sample media (look-tuning only; populate via setup_sample_media.py
# / setup_scales_media.py). All scale + transition binaries are gitignored.
# / build_pool_manifest.py). All scale + transition binaries are gitignored.
simulator/sample_media/**/*.mp4
# Local-only candidate review gallery (re-buildable; not shipped content).
simulator/static/review.html
+28 -15
View File
@@ -170,21 +170,34 @@ every transition, so fast navigation stays responsive on placeholder media.
contract** with the firmware (the keyboard/serial stand-in already works via a
`Controls` stream).
- **White-noise generation** — runtime white/pink noise for that content position.
- **Offline v2v variant pipeline** — author the pre-baked Right restyle variants
via the local flow-stabilized SD pipeline (now ~free, not a paid API — see the
scales-library/right-axis design §1/§4); a real multi-strength flow-stabilized
re-bake per base clip + the multilingual label/string tables + TTS. The
**scale-ring transitions** (forward/reverse per-edge morphs) are part of this
offline pipeline too.
- **Vertical slice done (session 0012):** the Right-variant half is
productionized and run for the **forest** scale — `simulator/bake_right_variants.py`
(the POC flow-restyle algorithm, repo-resident) bakes real strengths 14 on a
by-eye keyframe-strength ramp; forest now carries real Right variants in the
sim, so the dreamlike-axis look is judgeable on real footage. **Still
deferred:** real Right variants for cosmos/abyss (need their real strict-PD
bases sourced first) and the **real i2v ring transitions** (need a 2nd real
base + a generative-video model not in the POC). Plan:
[`2026-06-08-v2v-vertical-slice-forest-right-variants.md`](./superpowers/plans/2026-06-08-v2v-vertical-slice-forest-right-variants.md).
- **Content pipeline + label track** — source strict-PD neutral bases, run them
through the offline pipeline, and author the annotation tracks that drive the
Left-HUD overlay. The Right dream is now **real-time** (Kuwahara WebGL shader,
session 0013 — no per-clip ML bake), so the pipeline is
**source → crossfade-loop → master+proxy transcode → hand-authored /
motion-tracked labels → manifest**, not ML baking. Historical note: the
flow-stabilized SD-turbo v2v bake (session 0012, `simulator/bake_right_variants.py`)
was superseded when the operator found the baked-SD Right "too fluid" and the
design pivoted to the deterministic real-time dream; `bake_right_variants.py`
is now parked. Spec:
[`2026-06-24-content-pipeline-design.md`](./superpowers/specs/2026-06-24-content-pipeline-design.md).
- **Increment 1 — deterministic tooling + 5-scale ring (session 0014) ✅ Done.**
`tools/pipeline/` (frame/loop/transcode/manifest, pure + unit-tested + an
opt-in integration test on real footage); the backward-compatible keyframed
annotation-track schema with client interpolation; and the **full 5-scale ring**
(cosmos → orbit → forest → reef → abyss) with orbit + reef as placeholder
scales alongside the existing cosmos/abyss/forest. Plan:
[`2026-06-24-content-pipeline-increment-1.md`](./superpowers/plans/2026-06-24-content-pipeline-increment-1.md).
- **Increment 2 — hybrid motion-track pass + author mode (pending).** The
`tools/pipeline/track.py` classical OpenCV tracker + optional ML detect+track,
plus a simulator **author mode** for reviewing/editing annotation tracks in the
browser. Real strict-PD bases for all five scales sourced per
[`docs/content-sourcing.md`](./content-sourcing.md).
- **Still deferred:** real **i2v ring transitions** (need a generative-video
model + adjacent real bases); Pi renderer + serial/firmware.
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
variant" flag (sub-project 2 territory, design §13).
+112
View File
@@ -0,0 +1,112 @@
# Content candidate pool — operator-selected, vetted (2026-06-24)
> **WIRED (session 0016).** The rotating pool below is now the live manifest:
> `simulator/build_pool_manifest.py` regenerates `simulator/sample_media/manifest.json`
> from this pool; the ring scale `forest`→`coast` rename + the cleanup list (bottom)
> are done; the player picks a random pool member on landing
> (`player.ring.pick_clip_id`). See content-pipeline design §11.
The 5 scales each get a **rotating pool** of clips. When the viewer lands on a
scale by zooming the ring in/out, the player **randomly picks one clip from that
scale's pool**. (This supersedes the one-clip-per-scale model — see
[`docs/superpowers/specs/2026-06-24-content-pipeline-design.md`](./superpowers/specs/2026-06-24-content-pipeline-design.md).)
Every clip below was operator-selected from the candidate gallery, is **strict
public-domain or CCBYclass with attribution**, and was vetted **wordfree /
humanfree** (windows trimmed past title cards, captions, telemetry burnin,
divers, construction, logos). All processed via `tools/pipeline/run.py
process_clip` (trim → crossfadeloop → 1080p proxy). **Media is gitignored**;
this file is the resourceable record. Sim ids are the `simulator/sample_media/<id>/` dirs.
License rule confirmed by operator: **CC0 / CCBY / CCBYSA are acceptable with
a credit line** (our overlays + dream are derivatives, which these allow);
**CCBYND is NOT usable** (no derivatives); **CCBYNC only if the installation
stays noncommercial**.
## 🌌 cosmos (pool of 4)
| id | clip | source | window (s) | license / credit |
|----|------|--------|-----------|------------------|
| `cosmos` | Orion Nebula flythrough | NASA/JPLCaltech — https://images.nasa.gov/details/JPL-20221122-SOLSYSf-0001-Orion%20Dust%20and%20Death | ~3054 | PD (17 U.S.C. §105) |
| `cosmos_galaxies` | Flying Through Galaxies | NASA SVS — https://svs.gsfc.nasa.gov/vis/a010000/a014900/a014950/14950_Galaxies_FlyThrough_4k.mp4 | 832 | PD |
| `cosmos_hudf` | Hubble Ultra Deep Field zoom | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030600/a030687/hudf-b-1920x1080p30.mov | 2044 | CCBYclass — credit **NASA, ESA, STScI** |
| `cosmos_xdf` | eXtreme Deep Field flythrough | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030600/a030681/hxdf_fly-b-1920x1080p30.mov | 226 | CCBYclass — credit **NASA, ESA, STScI** |
> ⚠️ **DISCREPANCY to fix next session.** Operator selected cosmos **#1, #3
> (Orion), #4, #5**. During processing, **#6 Galaxy Traverse was run by mistake
> instead of #3 Orion**. The Orion clip (#3) is textfree and already lives at
> `simulator/sample_media/cosmos/base.mp4` (the prior real base), so the pool above
> is correct. **`cosmos_traverse` is an UNSELECTED extra on disk — drop it** (or
> confirm with operator). The current ring scale id is `cosmos`; when the pool is
> wired, `cosmos` becomes one pool member.
## 🛰️ orbit (pool of 3)
| id | clip | source | window (s) | license |
|----|------|--------|-----------|---------|
| `orbit_planetearth` | ISS — View of Planet Earth | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030700/a030771/ISS_View_of_Planet_Earth_2160p.mp4 | 1034 | PD |
| `orbit_crewobs` | ISS — Crew Earth Observations | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030700/a030771/ISS_Crew_Earth_Observations_2160p.mp4 | 933 | PD |
| `orbit_bluemarble` | NPP "Blue Marble" rotating globe (CG) | NASA SVS — https://svs.gsfc.nasa.gov/vis/a000000/a004500/a004550/BlueMarble_starfield_4k_2160p30_v2.mp4 | 1034 | PD |
> Operator also selected aurora→Perth (#4) and westcoast lights (#5); both
> **rejected as UNUSABLE** — #4 has a date/time burned into every frame, #5 has the
> ISS module + solar panel in frame throughout. No clean window exists in either.
## 🏖️ coast (pool of 4) — NEW scale, replaces the granite/waterfall `forest`
| id | clip | source | window (s) | license |
|----|------|--------|-----------|---------|
| `coast_birdrock` | Bird rock + ocean (aerial) | NPS Channel Islands — https://www.nps.gov/nps-audiovideo/audiovideo/663a2f13-6183-4b3e-a4a9-6698a84619d01080p.mp4 | 327 | PD |
| `coast_surfgrass` | Surfgrass / coralline tidepool | NPS Cabrillo — https://www.nps.gov/nps-audiovideo/audiovideo/de7d1cf2-e422-45b5-aa66-4c69e29a4d811080p.mp4 | 226 | PD |
| `coast_elkbeach` | Elk resting in coastal grass/fog | NPS Redwood — https://www.nps.gov/nps-audiovideo/legacy/redw/4634070E-F68D-4F74-E43174448168AC8A/redw-elkbeach_1280x720.mp4 | 2038 | PD (720p source) |
| `coast_drakesbeach` | Elephant seals on dark sand | NPS Point Reyes — https://www.nps.gov/nps-audiovideo/audiovideo/1cfd8165-1061-4d75-af53-b46c2e3ed2701080p.mp4 | 2035 | PD |
> Operator also selected the Lifeboat Station "elephant seal timelapse" (moreoptions
> #3) — **FAILED**: the NPS page's video resolved to the wrong footage (an aerial of
> the bay with road/parking/boats, no seals). An alternate source UUID
> (`50a7a20a-9e52-4643-b186-c61ce1cef0f7`) appeared in the page HTML and could be
> retried next session if the seal timelapse is still wanted.
>
> **Scale rename:** the live ring scale is still `forest` (Yosemite). Next session
> renames it `coast` and points it at this pool. The old real Yosemite base
> (`simulator/sample_media/forest/`) becomes unused.
## 🐠 reef (pool of 5)
All NOAA Fisheries broll (PD, credit "NOAA Fisheries"); Brightcove download
endpoint `https://download.gallery.brightcove.com/api/gallery/account/659677166001/site/site-600408/video/<VIDEO_ID>/download`.
| id | clip | VIDEO_ID | window (s) | notes |
|----|------|----------|-----------|-------|
| `reef_spawning` | Snapper school + yellow fish over sunlit reef | 1553921798001 | 5078 | clean window from a 12min reel |
| `reef_hawkfish` | Humphead parrotfish over reef (Hawaiian atoll) | 6039896460001 | 250274 | |
| `reef_lionfish` | Lionfish hovering + blue snapper | 4088881464001 | 1236 | underwater clean; humans on land after ~190s |
| `reef_snapper` | Red snapper schooling (Gulf, greenish water) | 5305427942001 | 731 | |
| `reef_coralspacific` | Pacific coral macro, spotted fish | 5231464663001 | 3054 | diverfree window (divers at 625s) |
## 🦑 abyss (pool of 3)
All NOAA Ocean Exploration (PD, 17 U.S.C. §105). These have creatures drifting
**into and out of frame** — the prime material for timewindowed tracked labels.
| id | clip | source | window (s) | creatures |
|----|------|--------|-----------|-----------|
| `abyss_wow` | "World of Water" | https://oceanexplorer.noaa.gov/wp-content/uploads/2018/05/wow-1280x720-1.mp4 | 1034 | jelly + ctenophore + siphonophore enter/exit |
| `abyss_midwaterexp` | "Midwater Exploration" | https://oceanexplorer.noaa.gov/wp-content/uploads/2019/09/midwater-exploration-1920x1080-1.mp4 | 1640 | white medusa + red worm enter/exit |
| `abyss_hiding` | "Hiding in the Dark" | https://oceanexplorer.noaa.gov/wp-content/uploads/2018/05/dark-1280x720-1.mp4 | 2044 | crimson jelly + ctenophore enter/exit |
> Operator also selected Crossota (#4) and comb jelly (#5); both **rejected** —
> Crossota has only ~17s clean (too short for a 24s window), comb jelly has a
> species caption covering the whole clip.
## On disk but NOT in any pool — ✅ CLEANED UP (session 0016)
Removed from local media (all were gitignored, so repo-invisible):
- `cosmos_traverse` — unselected (the #6/#3 mixup above). **dropped.**
- `orbit_westcoast` — rejected (ISS hardware in frame). **dropped.**
- `forest` — the old Yosemite base, superseded by the `coast` pool. **dropped.**
- `reef` — the original placeholder/early reef base, superseded by `reef_*`. **dropped.**
- `orbit` / `abyss` — the Increment1 single real bases (NASA ISS Exp65 / ctenophore),
not in the new pools. **dropped** (the cosmos Orion base stays as the cosmos pool primary).
- `review.html` gallery — now `.gitignore`d (localonly, rebuildable).
+29
View File
@@ -0,0 +1,29 @@
# Content sourcing — strict-PD neutral bases
For each scale, acquire a calm, neutral source clip from a **strict public-domain**
US-government source, verify the license, then run it through the pipeline.
| Scale | Source | Where |
|--------|--------------------------------|------------------------------------|
| cosmos | NASA / Hubble / JWST | https://images.nasa.gov/ |
| orbit | NASA / ISS | https://images.nasa.gov/ |
| forest | NPS / USGS | https://www.nps.gov/ , https://www.usgs.gov/ |
| reef | NOAA Ocean Exploration | https://oceanexplorer.noaa.gov/ |
| abyss | NOAA Ocean Exploration | https://oceanexplorer.noaa.gov/ |
**License check (mandatory):** confirm the asset is US-gov PD (17 U.S.C. §105) or
explicitly CC0. "Free stock" (Pexels/Pixabay/etc.) is royalty-free but NOT public
domain — do not use. Record a `Provenance` (tools/pipeline/provenance.py) per clip.
**Run the pipeline** (replaces the placeholder bytes for a scale):
```bash
python -c "from tools.pipeline.run import process_clip; \
process_clip('downloads/<scale>.mp4', 'simulator/sample_media/<scale>', \
start=<in_s>, duration=<len_s>, overlap=<0.5..1.0>)"
```
This writes `simulator/sample_media/<scale>/base.mp4` (proxy) + `master.mp4`. Then
author the labels (Increment 2: the hybrid track pass + simulator author mode) or
hand-edit the annotation track for now (spec §4). The ring + manifest entry already
exist; `process_clip` only replaces the media bytes.
@@ -1,7 +1,7 @@
# HEF — Content Pipeline (source → process → label → manifest)
**Date:** 2026-06-24
**Status:** Proposed design (pre-implementation) — session 0014
**Status:** Graduated — operator-approved (session 0014); **Increment 1 built & merged** (PR #13); real PD sourcing + rotating pool curated (PR #14/#15). **Increment 2 schema pinned + building** (session 0016 — see §11).
**Repo:** `human-experience-filter-art`
**Anchor:** `docs/ROADMAP.md` sub-project 3 (Player Runtime) content needs; the
spiritual successor to sub-project 2 (Ingest & Tagging tools), retargeted from the
@@ -239,7 +239,114 @@ master encode (waits on a measured pano spec).
download + provenance record for v1.
- **Crossfade overlap length** per scale (slow cosmos vs busier reef) — by eye.
## 10. Out of scope (YAGNI)
## 11. Increment 2 — schema decisions (session 0016)
Increment 2 is the second build order from §8. It is mostly **schema work**, so
this section pins the concrete data shapes before the build (the operator asked for
a short design pass first). Each subsection is a new, **backward-compatible**
extension — Increment 1 manifests keep rendering unchanged.
### 11.1 Rotating clip pool (supersedes one-clip-per-scale)
Each ring scale references a **pool** of vetted clips
(`docs/content-candidate-pool.md`); the player picks **one member at random when
the viewer lands on that scale** (zooming the ring in/out). This supersedes
decision §5's one-base-per-scale.
- **Manifest:** `ring.scales[i]` gains `"pool": ["clip_id", …]` (≥1). The legacy
`"clip_id"` stays as the **primary** member for back-compat (= `pool[0]` when a
pool is written); a scale with only `clip_id` and no `pool` behaves exactly as
Increment 1 (pool of one).
- **Canonical logic (`player/ring.py`):** `Scale` gains `pool: tuple[str, …]`;
`members` = `pool or (clip_id,)`. A **pure** `pick_clip_id(scale, r)` takes an
injected `r ∈ [0,1)` and returns `members[int(r·len)]` — randomness stays
**injected** (DI), so the function is pure/testable and the Pi reuses it.
- **Where the pick happens:** at the **API boundary** (`simulator/app.py`), which
may be impure: the advance endpoint calls `pick_clip_id(landed_scale,
random.random())` and returns it as `target_clip_id`. A `delta=0` advance is the
**initial / re-roll** pick (no-op move, fresh random member). The client loads
`move.target_clip_id` directly instead of deriving the clip from the index. One
selection implementation, Python-canonical (browser still "only renders").
- **Rename + cleanup:** ring scale `forest`**`coast`** (new land↔sea scale
between `orbit` and `reef`). Drop the leftovers `cosmos_traverse`,
`orbit_westcoast`, old `forest`, old `reef` (per the pool doc's cleanup list);
the cosmos pool's primary stays `cosmos` (Orion). New ring order:
`cosmos → orbit → coast → reef → abyss → (wrap) cosmos`; edges regenerated
(`orbit-coast`, `coast-reef` placeholders via `setup_scales_media.py`).
### 11.2 Time-windowed tracked labels
Objects drift **into and out of frame**, so a label appears only **while its
object is on screen** and its box **tracks** the object (the §4 keyframed `track`).
- **Manifest:** an annotation gains optional `"appear"` and `"disappear"`
(loop-normalized `t ∈ [0,1]`, same clock as `track`). The label renders only
while `appear ≤ loopT ≤ disappear`. To let an object straddle the loop seam, a
**`disappear < appear` window wraps** (visible when `loopT ≥ appear` **or**
`loopT ≤ disappear`). Absent both fields → always-present (Increment-1 behavior).
- **Runtime:** client-side in `renderOverlay` — a window test gates the annotation
before the existing `boxAt(loopT)` placement. Engine unchanged (labels are not in
`RenderPlan`). The **abyss + reef** pool clips (creatures entering/leaving) are
the authored test material.
### 11.3 Progressive LEFT-brain detail tiers (replaces flat `min_level` gating)
The Left knob now drives **how many** labels show **and how analytical** each is —
no fading. Low Left = **fewer, more general** labels; high Left = **more** labels +
**more analytical** detail.
- **Per-object `salience` (14):** how prominent the object is. It sets the Left
level at which the label first appears: `first_level = 5 salience` (salience 4 →
appears at Left 1; salience 1 → only at Left 4). So raising Left brings in
lower-salience objects.
- **Detail tiers (escalate with Left):** the displayed string escalates
`tier 1 general ("eel") → 2 specific ("moray eel") → 3 scientific
(*Gymnothorax*) → 4 scientific + a fact (lifespan)`. The tier shown =
`clamp(left first_level + 1, 1, n_tiers)` — an object shows tier 1 the moment
it appears and climbs one tier per further Left notch until its tiers run out.
- **Strings carry the tiers:** `strings.en[key]` may be a **list** (indexed by
`tier1`) instead of a string; a plain string stays static (back-compat, tier
ignored). Keeps one key per object and the i18n table as the single source.
- **Back-compat:** an annotation with `min_level` and no `salience` keeps flat
gating (`first_level = min_level`, single-tier string). `measure.*` readouts stay
single-tier, gated by their level (a measurement is not a tiered object).
- **No engine change:** tier selection is client-side against the existing
`overlay.level` (the Left knob) already in the `RenderPlan`.
### 11.4 Progressive RIGHT-brain emotion tiers
The affect **vocabulary** gets more sophisticated as Right rises — basic words at
Right 1, nuanced/compound feelings at high Right — mirroring §11.3 for affect.
- **Appearance rule unchanged:** affect surfaces only when **both** knobs are up;
an anchor's `min_level` is tested against `strength = min(left, right)` as today.
- **Emotion tiers:** `strings.en[feel.key]` may be a **list**; the word shown =
`clamp(right, 1, n_tiers)` (the Right knob, read from `dream.strength` already in
the `RenderPlan`). A plain string stays static (back-compat).
- **No engine change:** `AffectOverlay` already carries `strength`; the Right level
comes from `dream.strength`. Tier selection is client-side.
### 11.5 `tools/pipeline/track.py` (hybrid) + simulator author mode
The stage-5 tooling from §3.5 / §7, delivered as a **second PR** after the
experience PR (§8's split note).
- **`track.py`:** a **classical** path (OpenCV CSRT / optical-flow) seeded by a
hand-drawn box on a keyframe, propagated and **sampled to a sparse keyframed
`track`** (normalized `{t, box}` over the loop, §4); plus an **optional,
lazy-imported ML** detect+track path (YOLO+ByteTrack-class) that proposes tracks
the author maps to keys. Both emit the same `track` artifact + an `appear`/
`disappear` window. Pure helpers (normalization, keyframe sampling,
track→manifest) are unit-tested; an **opt-in** integration test runs the real
tracker on a pool clip (mirrors the Increment-1 real-footage test).
- **Author mode (simulator):** reuse the preview UI as the editor — scrub a pool
clip, draw/seed/correct boxes, run the tracker, assign `key` / `salience` /
tiered strings / `appear`-`disappear` / affect anchors, and **write the manifest
entry** (an endpoint upserts via `tools/pipeline/manifest.upsert_clip` and
persists). Semantics (keys, strings, scientific names, facts) are **always
hand-authored** — the tracker only propagates **box geometry** (§3.5).
## 12. Out of scope (YAGNI)
- Generative i2v **ring transitions** for the new edges (deferred; placeholders hold).
- **Audio** sourcing/alteration (separate content-dial channel; already deferred).
+30 -1
View File
@@ -50,10 +50,25 @@ class RingError(ValueError):
@dataclass(frozen=True)
class Scale:
"""A node on the ring: a scale of nature and the base clip it plays."""
"""A node on the ring: a scale of nature and the clip(s) it can play.
A scale carries a rotating POOL of vetted clips (content-pipeline design
§11.1): when the viewer lands on the scale, the player picks one member at
random (`pick_clip_id`). `clip_id` is the PRIMARY member (the deterministic
default / pool[0]); `pool` is the full set. A scale written with only
`clip_id` and no `pool` is a pool of one — exactly the Increment-1 behavior,
so old manifests are unchanged.
"""
id: str
clip_id: str
pool: tuple[str, ...] = ()
@property
def members(self) -> tuple[str, ...]:
"""The clip ids this scale can play — the pool, or just `clip_id` when
no pool was authored (a pool of one)."""
return self.pool if self.pool else (self.clip_id,)
@dataclass(frozen=True)
@@ -134,6 +149,20 @@ def scale_at(ring: ScaleRing, index: int) -> Scale:
return ring.scales[index % len(ring)]
def pick_clip_id(scale: Scale, r: float) -> str:
"""Pick one clip id from a scale's rotating pool, given an injected uniform
`r` in [0, 1) (content-pipeline design §11.1).
Randomness is INJECTED so this stays a pure function — testable and shared
with the Pi player; the impure draw (`random.random()`) happens once at the
API/runtime boundary. `r` is clamped into [0, 1) so an off-by-epsilon caller
never indexes out of range; a pool of one always returns its single member.
"""
members = scale.members
r = min(max(r, 0.0), 0.999999)
return members[int(r * len(members))]
def advance_ring(
ring: ScaleRing,
from_index: int,
@@ -0,0 +1,129 @@
# Session 0014.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-24T06-58 (PST)
> End: 2026-06-24T08-29 (PST)
> Type: brainstorming
> Posture: careful
> Claude-Session: 50301aba-c534-4a7c-9f33-8b0881920ab2
> Anchor: docs/ROADMAP.md sub-project 3 (Player Runtime) — content pipeline
> Status: **FINALIZED**
## Launch prompt
```
Launch: /goal next → resolved from memory to the recorded Next /goal:
START A NEW SESSION for the content pipeline — sourcing full neutral video clips + processing + labeling. Brainstorm/scope it into a content-pipeline spec FIRST, deciding: (1) sourcing (strict-PD clips, which scales); (2) clip length + seamless-loop strategy; (3) labeling approach (static vs motion-tracked; hand-authored vs detection-assisted; same for affect); (4) format target (res/fps/codec for sim now + Pi later). Anchor fact: Right dream is real-time, no per-clip bake → pipeline is transcode + authoring, NOT ML baking.
```
## Plan
> Anchor: docs/ROADMAP.md sub-project 3 content pipeline. Roadmap-anchored repo
> (no Gitea issue tracker used 00010013); content pipeline is one cohesive
> Feature → design-eligible. See Deferred decisions for the R1 deviation.
**Goal:** Explore the content pipeline (source full neutral PD clips → process →
label/author) and write its Solution-Design spec. Anchor fact: the Right dream is
real-time (session 0013), so the pipeline is **transcode + human authoring**, not
ML baking.
## What happened (arc)
This opened as `/goal next`; the session gate resolved the stored `Next /goal:`
from memory (content pipeline) and routed to **brainstorming** (the goal said
"brainstorm/scope it into a spec first"). Init claimed session **0014** (careful
posture, the brainstorming default), no concurrent sessions, clean `main`.
**Orientation.** Read the roadmap (sub-project 3), the scales-library/right-axis
design (§1/§3/§4), the two session-0013 specs (real-time dream supersedes the SD
bake; affect channel), and the current content shape (`simulator/clips.py`,
`manifest.json`, `setup_scales_media.py`). Confirmed the key simplifier: the Right
dream is real-time, so the pipeline is transcode + authoring, **not** ML baking.
**Brainstorming (careful, agree-up-front → draft-whole → review-whole).** Walked
the four launch-prompt decisions as `AskUserQuestion` checkpoints:
1. Scale set → **lean 5** (cosmos · orbit · forest/land · reef · abyss; microscopic
dropped — thinnest PD-video pool).
2. Loop → **crossfade loop** (tail→head overlap).
3. Labels → **motion-tracked, HYBRID** (ML proposes where it can; hand-seed +
classical tracker elsewhere). Surfaced that label *semantics* must always be
hand-authored (a generic detector can't produce "siphonophore"/"z ≈ 0.04");
tracking only propagates box geometry.
4. Format → **master + 1080p H.264 proxy**.
Drafted the full spec `docs/superpowers/specs/2026-06-24-content-pipeline-design.md`,
pushed to `feature/content-pipeline`, handed over the rendered Gitea URL for
whole-document review. Operator chose **"Approve + plan now"**.
**Planning.** Wrote `docs/superpowers/plans/2026-06-24-content-pipeline-increment-1.md`
(Increment 1 = real footage, no ML): 9 bite-sized TDD tasks. Self-reviewed against
the spec (coverage, types, one dead-variable fix). Operator chose **"Execute now
(subagent-driven)"**.
**Execution (subagent-driven, 2-stage review per group).** Implementer + reviewer
subagents (sonnet), grouped by file coupling:
- Tasks 13 (`tools/pipeline/ffmpeg_ops.py` pure builders) — DONE; review added an
`overlap<=0` test.
- Tasks 45 (`provenance.py`, `manifest.py`) — DONE; review added append-fallback +
N=1 self-wrap tests.
- Task 6 (`run.py` runner + integration test) — the test gated on `shutil.which`
and SKIPPED (ffmpeg not on PATH). Increment 1's goal is to *prove the pipeline on
real footage*, so I hardened the runner to resolve the **bundled imageio-ffmpeg**
(matching `setup_scales_media.py`) and switched duration probing to **cv2** (no
ffprobe dep — Pi-friendly). Re-run **actually exercised real ffmpeg on the real
forest base and surfaced a genuine bug**: `trim` yields `1/0` framerate which
`xfade` rejects (needs CFR) — fixed with `fps=30` after each trim segment. Test
now PASSES (proxy 1920×1080 verified via cv2). Closed an ffprobe-gap minor too.
- Task 7 (client annotation-track interpolation, `app.js`) — DONE; `node --check`
clean, API serves the track. Visual drift deferred to operator by-eye.
- Task 8 (ring 3→5: orbit + reef placeholders) — DONE; `/api/ring` shows 5 scales
+ 5 edges. Ring spin deferred to operator by-eye.
- Task 9 (sourcing doc + ROADMAP + stale-docstring fix) — DONE.
Final whole-branch review: **READY TO MERGE** (no critical/blocking). 246 passed /
2 skipped. Operator chose **"Merge to main now"** → pushed, created **PR #13** via
the Gitea API, merged (delete branch), synced main, deleted local branch. Stamped
the spec **graduated** on main.
## Outcome
- Content-pipeline **spec** (graduated) + **Increment-1 plan** (archived in-repo)
+ **Increment 1 implementation** all merged to `main` (PR #13, merge `bb5ecf0`;
spec stamp `73e5c4a`). Suite **246 passed / 2 skipped**.
- New `tools/pipeline/` package; keyframed annotation-`track` schema
(client-interpolated, backward compatible); scale ring extended 3→5.
- `docs/content-sourcing.md` (strict-PD sourcing procedure); ROADMAP updated.
## Deferred decisions
- **R1 anchor (no Gitea issue):** the brainstorming gate wants a non-`epic`
`type/*` tracker issue as the design anchor. This repo has never used a Gitea
issue tracker (roadmap-driven, 00010013), so the spec is anchored to
`docs/ROADMAP.md` sub-project 3 — matching every prior brainstorming session
here (0005/0007/0008/0013). Flagged rather than force-filing an issue that would
break established repo practice; operator can request issue-tracking if wanted.
- **Session-type blur (operator-directed):** a brainstorming session normally ends
at the spec; the operator explicitly extended it through plan → execute → merge.
Each transition was an explicit operator choice via `AskUserQuestion`, so the
careful-posture gates were honored, not bypassed.
- **Crossfade duration source (minor, non-blocking):** `process_clip` probes the
duration from the *source*, not the post-frame mezzanine; GOP alignment can make
the tail segment ±~33 ms off. ffmpeg handles EOF gracefully; imperceptible on
calm footage. Noted for a future fast-motion clip.
## Pending for next session
- **Operator by-eye review of Increment 1** in the sim: the track drift
(water/fish boxes follow playback + loop seamlessly) and spinning the 5-scale
ring cosmos→orbit→forest→reef→abyss→wrap. A dev sim (session-0013 live-reload)
was running on :8000 during the session.
## Next /goal
```
/goal content-pipeline Increment 2 — source real strict-PD bases per docs/content-sourcing.md (run each through tools/pipeline/run.py process_clip) + build the hybrid motion-track pass tools/pipeline/track.py + a simulator author mode; FIRST do the pending by-eye review of Increment 1. Per docs/superpowers/specs/2026-06-24-content-pipeline-design.md
```
> Still-deferred (tracked in `docs/ROADMAP.md`, this repo's queue): real i2v ring
> transitions (need a generative-video model + adjacent real bases); Pi renderer +
> serial/firmware (simulator-first).
@@ -0,0 +1,85 @@
# Session 0015.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-24T13-16 (PST)
> Type: planning-and-executing
> Posture: yolo
> Claude-Session: 50301aba-c534-4a7c-9f33-8b0881920ab2
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
>
> This file reserves session ID 0015 for human-experience-filter-art. The driver replaces this
> body with the full transcript and renames the file to its final
> SESSION-0015.0-TRANSCRIPT-2026-06-24T13-16--<end>.md form at session end.
## Launch prompt
```
Process 5 candidate ORBIT (Earth-from-space) clips for the human-experience-filter art installation: resolve URLs, download, vet for text + humans, trim to a word-free calm window, transcode. Clips: orbit_planetearth, orbit_crewobs, orbit_aurora_perth (SVS 31281), orbit_westcoast (SVS 30180), orbit_bluemarble (SVS 4550).
```
## Plan
Process 5 candidate ORBIT (Earth-from-space) clips: resolve URLs, download, vet for text/telemetry/humans via contact sheets, trim to word-free 24s window, transcode via process_clip.
Tasks:
1. Resolve direct mp4 URLs for SVS pages (aurora_perth/31281, westcoast/30180, bluemarble/4550)
2. Download all 5 clips
3. Build contact sheets (overview + first-8s + last-8s) for each clip
4. Vet each clip for text overlays, telemetry, ISS hardware, humans
5. Determine clean window or mark UNUSABLE
6. process_clip → simulator/sample_media/<id>/ for usable clips
7. Generate preview frames
## Clip results
### orbit_planetearth
- URL: https://svs.gsfc.nasa.gov/vis/a030000/a030700/a030771/ISS_View_of_Planet_Earth_2160p.mp4
- Duration: 97.2s
- Text/telemetry: "INTERNATIONAL SPACE STATION" title card t=08s; ISS logo end card t≈8097s
- Humans: none
- Window: 1034s (24s, clean daytime Earth clouds)
- Status: USABLE
- Output: simulator/sample_media/orbit_planetearth/
- Preview: /tmp/hef-cand/orbit_planetearth/preview.png
### orbit_crewobs
- URL: https://svs.gsfc.nasa.gov/vis/a030000/a030700/a030771/ISS_Crew_Earth_Observations_2160p.mp4
- Duration: 66.4s
- Text/telemetry: "INTERNATIONAL SPACE STATION" + date/location title card t=08s; ISS logo end card t≈5566s
- Humans: none
- Window: 933s (24s, clean desert/cloud Earth)
- Status: USABLE
- Output: simulator/sample_media/orbit_crewobs/
- Preview: /tmp/hef-cand/orbit_crewobs/preview.png
### orbit_aurora_perth
- URL: https://svs.gsfc.nasa.gov/vis/a030000/a031200/a031281/ISS067_20220817_aurora_1080p25.mp4
- Duration: 49.6s
- Text/telemetry: Date+timestamp burned in top-left throughout (e.g. "2022-08-17 19:15:25", updating every second)
- Humans: none
- Window: none (telemetry throughout)
- Status: UNUSABLE
### orbit_westcoast
- URL: https://svs.gsfc.nasa.gov/vis/a030000/a030100/a030180/iss028_nighttime_20110819_720p.mp4
- Duration: 54.7s
- Text/telemetry: none
- Humans: none
- ISS hardware: cylindrical ISS module + solar panel grid visible in top-right corner throughout entire clip (confirmed by exhaustive per-second crop inspection t=1444s)
- Window: none (ISS hardware throughout)
- Status: UNUSABLE — orbit_westcoast output dir deleted
### orbit_bluemarble
- URL: https://svs.gsfc.nasa.gov/vis/a000000/a004500/a004550/BlueMarble_starfield_4k_2160p30_v2.mp4
- Duration: 121.0s
- Text/telemetry: none (CG visualization, no overlays anywhere)
- Humans: none
- Window: 1034s (24s, rotating Earth globe on starfield)
- Status: USABLE
- Output: simulator/sample_media/orbit_bluemarble/
- Preview: /tmp/hef-cand/orbit_bluemarble/preview.png
## Deferred decisions
- orbit_westcoast output directory (simulator/sample_media/orbit_westcoast/) was written before the ISS hardware was confirmed throughout. It needs to be deleted. The rm -rf was attempted but permission was denied in the sandbox — operator should delete it manually: `rm -rf simulator/sample_media/orbit_westcoast`
@@ -0,0 +1,22 @@
# Session 0016.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-24T17-48 (PST)
> Type: planning-and-executing
> Posture: yolo
> Claude-Session: 0fde1d36-7c64-414f-8f49-8669b5dd2263
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
>
> This file reserves session ID 0016 for human-experience-filter-art. The driver replaces this
> body with the full transcript and renames the file to its final
> SESSION-0016.0-TRANSCRIPT-2026-06-24T17-48--<end>.md form at session end.
## Launch prompt
_(launch prompt not captured at claim time)_
## Deferred decisions
_Autonomous-mode low-confidence calls the driver made and would have
liked operator input on. Appended as the session runs; surfaced at
finalize. Empty if none._
+9
View File
@@ -37,5 +37,14 @@
},
"0013": {
"title": ""
},
"0014": {
"title": ""
},
"0015": {
"title": ""
},
"0016": {
"title": ""
}
}
+13 -2
View File
@@ -9,6 +9,7 @@ from __future__ import annotations
import hashlib
import os
import random
import time
from pathlib import Path
from typing import Optional
@@ -26,7 +27,12 @@ from player.alteration import (
)
from player.content import resolve_content
from player.controls import CONTENT_POSITIONS
from player.ring import DEFAULT_FAST_SPIN_THRESHOLD, advance_ring
from player.ring import (
DEFAULT_FAST_SPIN_THRESHOLD,
advance_ring,
pick_clip_id,
scale_at,
)
from simulator.clips import load_manifest, load_ring, ring_move_to_dict, ring_to_dict
STATIC_DIR = Path(__file__).parent / "static"
@@ -145,7 +151,12 @@ def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
req.delta,
fast_spin_threshold=DEFAULT_FAST_SPIN_THRESHOLD,
)
return ring_move_to_dict(move, app.state.ring)
# Rotating pool: pick a random member of the LANDED scale (content-pipeline
# §11.1). A delta=0 advance is the initial / re-roll pick (a no-op move that
# still yields a fresh random clip). The pure pick takes injected randomness.
landed = scale_at(app.state.ring, move.to_index)
chosen = pick_clip_id(landed, random.random())
return ring_move_to_dict(move, app.state.ring, chosen)
@app.middleware("http")
async def _no_cache(request, call_next):
+403
View File
@@ -0,0 +1,403 @@
"""Build the rotating-pool manifest for the 5 ring scales (content-pipeline §11).
This is the reproducible, hand-authored BASELINE for Increment 2's manifest:
each ring scale carries a POOL of vetted clips (docs/content-candidate-pool.md);
the player picks one member at random on landing (player.ring.pick_clip_id). The
authored label/affect content lives here in readable Python and is emitted to
`simulator/sample_media/manifest.json` — far less error-prone than hand-editing 19
clips of JSON, and a clean place for the simulator author mode to regenerate from.
Authoring model (design §11.3 / §11.4):
- LEFT detail tiers: each label carries `salience` (1-4; first shows at Left
level `5 - salience`) and a tiered string list (general -> specific ->
scientific -> +fact) that escalates with the Left knob.
- RIGHT emotion tiers: affect words are a tiered list per feeling (basic ->
compound) escalating with the Right knob; still gated on min(Left, Right).
- TIME-WINDOWED + TRACKED labels: a label may carry `appear`/`disappear`
(loop-normalized) and a keyframed `track`; it shows only while on screen and
its box follows the subject. The abyss + reef pools are the test material.
Strings: a value may be a LIST (indexed by tier-1) or a plain string (static,
back-compat). Affect appears only when both knobs are up.
Usage:
python simulator/build_pool_manifest.py # write manifest.json
python simulator/build_pool_manifest.py --media # + generate new transition placeholders
"""
from __future__ import annotations
import json
import shutil
import subprocess
import sys
from pathlib import Path
MEDIA = Path(__file__).parent / "sample_media"
MANIFEST = MEDIA / "manifest.json"
# --- Pools: scale id -> ordered clip ids (primary first) (content-candidate-pool.md) ---
POOLS: dict[str, list[str]] = {
"cosmos": ["cosmos", "cosmos_galaxies", "cosmos_hudf", "cosmos_xdf"],
"orbit": ["orbit_planetearth", "orbit_crewobs", "orbit_bluemarble"],
"coast": ["coast_birdrock", "coast_surfgrass", "coast_elkbeach", "coast_drakesbeach"],
"reef": ["reef_lionfish", "reef_spawning", "reef_hawkfish", "reef_snapper", "reef_coralspacific"],
"abyss": ["abyss_wow", "abyss_midwaterexp", "abyss_hiding"],
}
# Ring order (large -> small, wraps): cosmos -> orbit -> coast -> reef -> abyss -> cosmos.
RING_ORDER = ["cosmos", "orbit", "coast", "reef", "abyss"]
PD = "public-domain (US Gov, 17 U.S.C. §105)"
PD_NPS = "public-domain (NPS, no © — 17 U.S.C. §105)"
CCBY_STSCI = "CC-BY-class — credit NASA, ESA, STScI"
# --- Per-clip provenance: id -> (title, license, source) ---
META: dict[str, tuple[str, str, str]] = {
# cosmos
"cosmos": ("Orion Nebula flythrough (NASA/JPL-Caltech)", PD,
"NASA/JPL-Caltech — images.nasa.gov JPL-20221122-SOLSYSf-0001 (Orion); trim ~3054s, crossfade-loop"),
"cosmos_galaxies": ("Flying Through Galaxies (NASA SVS)", PD,
"NASA SVS a014950 14950_Galaxies_FlyThrough_4k; trim 832s, crossfade-loop"),
"cosmos_hudf": ("Hubble Ultra Deep Field zoom (NASA/ESA/STScI)", CCBY_STSCI,
"NASA SVS a030687 hudf-b-1920x1080p30; trim 2044s, crossfade-loop"),
"cosmos_xdf": ("eXtreme Deep Field flythrough (NASA/ESA/STScI)", CCBY_STSCI,
"NASA SVS a030681 hxdf_fly-b-1920x1080p30; trim 226s, crossfade-loop"),
# orbit
"orbit_planetearth": ("ISS — View of Planet Earth (NASA SVS)", PD,
"NASA SVS a030771 ISS_View_of_Planet_Earth_2160p; trim 1034s, crossfade-loop"),
"orbit_crewobs": ("ISS — Crew Earth Observations (NASA SVS)", PD,
"NASA SVS a030771 ISS_Crew_Earth_Observations_2160p; trim 933s, crossfade-loop"),
"orbit_bluemarble": ("NPP “Blue Marble” rotating globe (NASA SVS)", PD,
"NASA SVS a004550 BlueMarble_starfield_4k_2160p30_v2; trim 1034s, crossfade-loop"),
# coast
"coast_birdrock": ("Bird rock + ocean, aerial (NPS Channel Islands)", PD_NPS,
"NPS Channel Islands AV 663a2f13; trim 327s, crossfade-loop"),
"coast_surfgrass": ("Surfgrass / coralline tidepool (NPS Cabrillo)", PD_NPS,
"NPS Cabrillo AV de7d1cf2; trim 226s, crossfade-loop"),
"coast_elkbeach": ("Elk resting in coastal fog (NPS Redwood)", PD_NPS,
"NPS Redwood redw-elkbeach_1280x720; trim 2038s, crossfade-loop (720p source)"),
"coast_drakesbeach": ("Elephant seals on dark sand (NPS Point Reyes)", PD_NPS,
"NPS Point Reyes AV 1cfd8165; trim 2035s, crossfade-loop"),
# reef
"reef_lionfish": ("Lionfish hovering over reef (NOAA Fisheries)", PD,
"NOAA Fisheries b-roll VIDEO_ID 4088881464001; trim 1236s, crossfade-loop"),
"reef_spawning": ("Snapper school over sunlit reef (NOAA Fisheries)", PD,
"NOAA Fisheries b-roll VIDEO_ID 1553921798001; trim 5078s, crossfade-loop"),
"reef_hawkfish": ("Humphead parrotfish over reef (NOAA Fisheries)", PD,
"NOAA Fisheries b-roll VIDEO_ID 6039896460001; trim 250274s, crossfade-loop"),
"reef_snapper": ("Red snapper schooling (NOAA Fisheries)", PD,
"NOAA Fisheries b-roll VIDEO_ID 5305427942001; trim 731s, crossfade-loop"),
"reef_coralspacific": ("Pacific coral macro (NOAA Fisheries)", PD,
"NOAA Fisheries b-roll VIDEO_ID 5231464663001; trim 3054s, crossfade-loop"),
# abyss
"abyss_wow": ("“World of Water” (NOAA Ocean Exploration)", PD,
"NOAA Ocean Exploration wow-1280x720-1; trim 1034s, crossfade-loop"),
"abyss_midwaterexp": ("“Midwater Exploration” (NOAA Ocean Exploration)", PD,
"NOAA Ocean Exploration midwater-exploration-1920x1080-1; trim 1640s, crossfade-loop"),
"abyss_hiding": ("“Hiding in the Dark” (NOAA Ocean Exploration)", PD,
"NOAA Ocean Exploration dark-1280x720-1; trim 2044s, crossfade-loop"),
}
# --- Affect vocabulary per SCALE (shared by all the scale's pool members) -------
# Each: (key, [x, y], min_level, [tier1 basic .. tier4 compound]). The word shown
# escalates with the RIGHT knob; appears only when both knobs are up (§11.4).
AFFECT: dict[str, list[tuple]] = {
"cosmos": [
("feel.wonder", [0.50, 0.44], 1, ["wow", "wonder", "awe", "transcendent awe"]),
("feel.vastness", [0.20, 0.70], 2, ["big", "vastness", "immensity", "a dizzying immensity"]),
("feel.insignificance", [0.66, 0.60], 3, ["small", "smallness", "insignificance", "a humbling insignificance"]),
("feel.longing", [0.42, 0.84], 4, ["want", "longing", "yearning", "an aching yearning"]),
],
"orbit": [
("feel.serenity", [0.50, 0.44], 1, ["calm", "serenity", "peace", "a quiet, weightless peace"]),
("feel.fragility", [0.22, 0.68], 2, ["thin", "fragility", "vulnerability", "a tender fragility"]),
("feel.unity", [0.66, 0.58], 3, ["one", "unity", "belonging", "a borderless belonging"]),
("feel.distance", [0.42, 0.82], 4, ["far", "distance", "remoteness", "an exquisite remoteness"]),
],
"coast": [
("feel.ease", [0.50, 0.44], 1, ["nice", "ease", "calm", "an unhurried calm"]),
("feel.nostalgia", [0.22, 0.68], 2, ["miss", "nostalgia", "wistfulness", "a salt-air wistfulness"]),
("feel.belonging", [0.66, 0.58], 3, ["home", "belonging", "rootedness", "a tidal rootedness"]),
("feel.melancholy", [0.42, 0.82], 4, ["sad", "melancholy", "longing", "a soft seaward longing"]),
],
"reef": [
("feel.delight", [0.50, 0.44], 1, ["fun", "delight", "joy", "a darting, bright joy"]),
("feel.abundance", [0.22, 0.68], 2, ["full", "abundance", "richness", "a teeming richness"]),
("feel.curiosity", [0.66, 0.58], 3, ["look", "curiosity", "fascination", "an absorbed fascination"]),
("feel.immersion", [0.42, 0.82], 4, ["in", "immersion", "absorption", "a held, breathless absorption"]),
],
"abyss": [
("feel.unease", [0.50, 0.44], 1, ["uh", "unease", "disquiet", "a creeping disquiet"]),
("feel.fascination", [0.22, 0.68], 2, ["ooh", "fascination", "wonder", "a forbidden wonder"]),
("feel.isolation", [0.66, 0.58], 3, ["alone", "isolation", "solitude", "a crushing solitude"]),
("feel.dread", [0.42, 0.82], 4, ["fear", "dread", "foreboding", "a slow, cold foreboding"]),
],
}
def static_label(key, salience, tiers, box):
"""A fixed-box tiered label (always on-screen, salience-gated by Left)."""
return {"key": key, "salience": salience, "box": box, "_tiers": tiers}
def tracked_label(key, salience, tiers, appear, disappear, track):
"""A time-windowed tracked label: shows only in [appear, disappear] and its
box follows the subject (§11.2). `track` = [(t, [x,y,w,h]), ...]."""
return {
"key": key, "salience": salience, "appear": appear, "disappear": disappear,
"track": [{"t": t, "box": b} for t, b in track], "_tiers": tiers,
}
def measure(key, value, box, min_level):
"""A single-tier measurement readout (not a tiered object), gated by Left."""
return {"key": key, "box": box, "min_level": min_level, "_tiers": value}
# --- Labels per clip. `_tiers` is lifted into the strings table by build(). ------
# Showcase tracked/windowed labels live on the abyss + reef pools (the creatures
# drift through frame). Other clips get a few salience-gated static tiered labels.
LABELS: dict[str, list[dict]] = {
# ---------- cosmos ----------
"cosmos": [
static_label("detected.nebula", 4, ["cloud", "nebula", "emission nebula", "emission nebula · ionized H II region"], [0.32, 0.28, 0.34, 0.36]),
static_label("detected.star", 2, ["star", "young star", "protostar", "protostar · <1 Myr old"], [0.60, 0.30, 0.10, 0.12]),
measure("measure.redshift", "z ≈ 0.0", [0.36, 0.70, 0.18, 0.08], 4),
],
"cosmos_galaxies": [
static_label("detected.galaxy", 4, ["galaxy", "spiral galaxy", "barred spiral", "barred spiral · ~10¹¹ stars"], [0.34, 0.30, 0.30, 0.34]),
measure("measure.distance", "~Mly", [0.06, 0.06, 0.18, 0.08], 3),
],
"cosmos_hudf": [
static_label("detected.galaxy", 4, ["smudge", "galaxy", "early galaxy", "early galaxy · z ≈ 16, light Gyr-old"], [0.40, 0.36, 0.20, 0.22]),
measure("measure.field", "deep field", [0.06, 0.06, 0.2, 0.08], 3),
],
"cosmos_xdf": [
static_label("detected.galaxy", 4, ["dot", "galaxy", "faint galaxy", "faint galaxy · among the earliest seen"], [0.42, 0.38, 0.18, 0.2]),
],
# ---------- orbit ----------
"orbit_planetearth": [
static_label("detected.cloud_band", 4, ["clouds", "cloud band", "cumulus field", "cumulus field · tropical convection"], [0.30, 0.30, 0.30, 0.20]),
static_label("detected.limb", 2, ["edge", "Earths limb", "atmospheric limb", "atmospheric limb · ~100 km of air"], [0.05, 0.70, 0.9, 0.12]),
measure("measure.altitude", "~408 km", [0.06, 0.06, 0.18, 0.08], 3),
],
"orbit_crewobs": [
static_label("detected.coastline", 4, ["land", "coastline", "continental margin", "continental margin · land meets sea"], [0.30, 0.34, 0.34, 0.26]),
measure("measure.altitude", "~408 km", [0.06, 0.06, 0.18, 0.08], 3),
],
"orbit_bluemarble": [
static_label("detected.globe", 4, ["Earth", "the globe", "terrestrial planet", "terrestrial planet · 12,742 km across"], [0.28, 0.18, 0.44, 0.6]),
],
# ---------- coast ----------
"coast_birdrock": [
static_label("detected.surf", 4, ["waves", "surf", "breaking swell", "breaking swell · wind-driven, ~10 s period"], [0.20, 0.55, 0.6, 0.3]),
static_label("detected.searock", 2, ["rock", "sea stack", "coastal sea stack", "sea stack · wave-cut residual rock"], [0.30, 0.25, 0.22, 0.3]),
],
"coast_surfgrass": [
static_label("detected.surfgrass", 4, ["grass", "surfgrass", "Phyllospadix", "Phyllospadix · a marine seagrass, not algae"], [0.18, 0.40, 0.5, 0.4]),
static_label("detected.coralline", 2, ["pink", "coralline algae", "crustose coralline", "crustose coralline · calcified red algae"], [0.6, 0.55, 0.2, 0.2]),
],
"coast_elkbeach": [
static_label("detected.elk", 4, ["elk", "Roosevelt elk", "Cervus canadensis roosevelti", "C. c. roosevelti · largest elk subspecies"], [0.34, 0.42, 0.3, 0.3]),
static_label("detected.fog", 2, ["fog", "coastal fog", "advection fog", "advection fog · warm air over cold upwelling"], [0.05, 0.05, 0.9, 0.25]),
],
"coast_drakesbeach": [
static_label("detected.seal", 4, ["seals", "elephant seals", "Mirounga angustirostris", "M. angustirostris · males to 2,000 kg"], [0.25, 0.55, 0.5, 0.3]),
static_label("detected.sand", 1, ["sand", "dark sand", "mineral-dark beach", "dark beach · eroded coastal sediment"], [0.05, 0.8, 0.9, 0.15]),
],
# ---------- reef (showcase: tracked + windowed) ----------
"reef_lionfish": [
tracked_label(
"detected.lionfish", 4,
["fish", "lionfish", "Pterois volitans", "Pterois volitans · venomous spines, invasive in the Atlantic"],
0.05, 0.95,
[(0.05, [0.15, 0.30, 0.16, 0.18]), (0.5, [0.50, 0.38, 0.16, 0.18]), (0.95, [0.72, 0.34, 0.16, 0.18])],
),
tracked_label(
"detected.snapper", 2,
["fish", "blue snapper", "Lutjanus", "Lutjanus · schools over reef structure"],
0.0, 0.6,
[(0.0, [0.05, 0.55, 0.12, 0.1]), (0.3, [0.22, 0.5, 0.12, 0.1]), (0.6, [0.40, 0.56, 0.12, 0.1])],
),
measure("measure.depth", "18 m", [0.06, 0.06, 0.16, 0.08], 3),
],
"reef_spawning": [
tracked_label(
"detected.school", 4,
["fish", "fish school", "snapper aggregation", "snapper aggregation · synchronized spawning run"],
0.0, 1.0,
[(0.0, [0.20, 0.30, 0.2, 0.18]), (0.5, [0.55, 0.34, 0.2, 0.18]), (1.0, [0.20, 0.30, 0.2, 0.18])],
),
static_label("detected.coral", 2, ["coral", "reef coral", "scleractinian", "scleractinian · reef-building stony coral"], [0.15, 0.6, 0.35, 0.3]),
measure("measure.depth", "22 m", [0.06, 0.06, 0.16, 0.08], 3),
],
"reef_hawkfish": [
tracked_label(
"detected.parrotfish", 4,
["fish", "parrotfish", "Bolbometopon muricatum", "B. muricatum · humphead, grazes reef into sand"],
0.1, 0.9,
[(0.1, [0.55, 0.25, 0.2, 0.22]), (0.5, [0.35, 0.34, 0.2, 0.22]), (0.9, [0.12, 0.40, 0.2, 0.22])],
),
measure("measure.depth", "12 m", [0.06, 0.06, 0.16, 0.08], 3),
],
"reef_snapper": [
tracked_label(
"detected.snapper", 4,
["fish", "red snapper", "Lutjanus campechanus", "L. campechanus · long-lived, can reach 50+ yr"],
0.0, 0.7,
[(0.0, [0.10, 0.3, 0.16, 0.14]), (0.35, [0.40, 0.36, 0.16, 0.14]), (0.7, [0.68, 0.30, 0.16, 0.14])],
),
],
"reef_coralspacific": [
static_label("detected.coral", 4, ["coral", "coral colony", "Pacific scleractinian", "Pacific scleractinian · a symbiosis with algae"], [0.2, 0.4, 0.4, 0.4]),
tracked_label(
"detected.spotfish", 2,
["fish", "spotted fish", "reef damselfish", "reef damselfish · territorial over its coral"],
0.2, 0.8,
[(0.2, [0.6, 0.25, 0.1, 0.09]), (0.5, [0.45, 0.3, 0.1, 0.09]), (0.8, [0.3, 0.26, 0.1, 0.09])],
),
],
# ---------- abyss (showcase: tracked + windowed; creatures drift through) ----------
"abyss_wow": [
tracked_label(
"detected.jelly", 4,
["jelly", "comb jelly", "Ctenophora", "Ctenophora · swims by beating rows of cilia"],
0.0, 0.55,
[(0.0, [0.05, 0.25, 0.14, 0.18]), (0.3, [0.30, 0.32, 0.14, 0.18]), (0.55, [0.52, 0.28, 0.14, 0.18])],
),
tracked_label(
"detected.siphonophore", 3,
["chain", "siphonophore", "Siphonophorae", "Siphonophorae · a colony of clones, can exceed 40 m"],
0.45, 1.0,
[(0.45, [0.80, 0.6, 0.1, 0.3]), (0.7, [0.6, 0.45, 0.1, 0.34]), (1.0, [0.45, 0.3, 0.1, 0.38])],
),
measure("measure.depth", "1,200 m", [0.06, 0.06, 0.16, 0.08], 2),
],
"abyss_midwaterexp": [
tracked_label(
"detected.medusa", 4,
["jelly", "white medusa", "Scyphozoa", "Scyphozoa · the swimming medusa stage of a jellyfish"],
0.0, 0.6,
[(0.0, [0.55, 0.20, 0.16, 0.2]), (0.3, [0.40, 0.34, 0.16, 0.2]), (0.6, [0.22, 0.30, 0.16, 0.2])],
),
tracked_label(
"detected.worm", 2,
["worm", "red worm", "polychaete", "polychaete · a free-swimming bristle worm"],
0.5, 1.0,
[(0.5, [0.10, 0.7, 0.12, 0.1]), (0.75, [0.3, 0.6, 0.12, 0.1]), (1.0, [0.5, 0.66, 0.12, 0.1])],
),
measure("measure.depth", "1,600 m", [0.06, 0.06, 0.16, 0.08], 2),
],
"abyss_hiding": [
tracked_label(
"detected.jelly", 4,
["jelly", "crimson jelly", "Scyphozoa", "Scyphozoa · red is invisible in the lightless deep"],
0.0, 0.7,
[(0.0, [0.15, 0.28, 0.16, 0.2]), (0.35, [0.42, 0.36, 0.16, 0.2]), (0.7, [0.66, 0.30, 0.16, 0.2])],
),
measure("measure.depth", "2,140 m", [0.06, 0.06, 0.16, 0.08], 2),
],
}
# Human-readable strings for affect/measurement keys are produced from the tiered
# data above; this maps measurement keys to nothing extra (their value IS the string).
def _affect_for_clip(scale: str) -> tuple[list, dict]:
"""Build the affect list + its strings for a scale's pool member."""
entries, strings = [], {}
for key, at, min_level, tiers in AFFECT[scale]:
entries.append({"key": key, "at": at, "min_level": min_level})
strings[key] = tiers
return entries, strings
def _labels_for_clip(clip_id: str) -> tuple[list, dict]:
"""Build the annotation list + its strings, lifting `_tiers` into the table."""
anns, strings = [], {}
for raw in LABELS.get(clip_id, []):
a = {k: v for k, v in raw.items() if k != "_tiers"}
anns.append(a)
strings[a["key"]] = raw["_tiers"]
return anns, strings
def _clip_entry(scale: str, clip_id: str) -> dict:
title, license_, source = META[clip_id]
anns, lab_strings = _labels_for_clip(clip_id)
affect, aff_strings = _affect_for_clip(scale)
return {
"id": clip_id,
"title": title,
"base_file": f"{clip_id}/base.mp4",
"license": license_,
"source": source,
"right_variants": {},
"annotations": anns,
"affect": affect,
"strings": {"en": {**lab_strings, **aff_strings}},
}
def build_manifest() -> dict:
"""Assemble the full rotating-pool manifest dict."""
clips = []
for scale in RING_ORDER:
for clip_id in POOLS[scale]:
clips.append(_clip_entry(scale, clip_id))
scales = [{"id": s, "clip_id": POOLS[s][0], "pool": POOLS[s]} for s in RING_ORDER]
transitions = []
n = len(RING_ORDER)
for i in range(n):
a, b = RING_ORDER[i], RING_ORDER[(i + 1) % n]
transitions.append({"file": f"transitions/{a}-{b}.mp4", "model": "placeholder-zoom"})
return {"clips": clips, "ring": {"scales": scales, "transitions": transitions}}
# --- Optional: generate the NEW transition placeholders (orbit-coast, coast-reef) ---
def _ffmpeg() -> str:
if shutil.which("ffmpeg"):
return "ffmpeg"
import imageio_ffmpeg
return imageio_ffmpeg.get_ffmpeg_exe()
def _make_transition(ff: str, scale_a: str, scale_b: str) -> Path:
"""A placeholder zoom/warp morph between two scales, from their PRIMARY pool
members' bases (mirrors setup_scales_media.py but keyed by scale->primary)."""
a = MEDIA / POOLS[scale_a][0] / "base.mp4"
b = MEDIA / POOLS[scale_b][0] / "base.mp4"
out = MEDIA / "transitions" / f"{scale_a}-{scale_b}.mp4"
out.parent.mkdir(parents=True, exist_ok=True)
norm = "trim=0:3,setpts=PTS-STARTPTS,scale=1280:720,fps=25,setsar=1,format=yuv420p"
subprocess.run([
ff, "-y", "-i", str(a), "-i", str(b), "-filter_complex",
f"[0:v]{norm}[a];[1:v]{norm}[b];"
"[a][b]xfade=transition=zoomin:duration=1.5:offset=0.75,format=yuv420p[v]",
"-map", "[v]", "-an", str(out),
], check=True, capture_output=True)
return out
def generate_media() -> None:
"""Generate the new ring edges introduced by the coast scale; the cosmos-orbit,
reef-abyss and abyss-cosmos edges already exist from the prior 5-scale ring."""
ff = _ffmpeg()
for a, b in [("orbit", "coast"), ("coast", "reef")]:
_make_transition(ff, a, b)
print(f"generated transitions/{a}-{b}.mp4 (placeholder zoom)")
def main(argv: list[str]) -> None:
manifest = build_manifest()
MANIFEST.write_text(json.dumps(manifest, indent=2, ensure_ascii=False) + "\n")
print(f"wrote {MANIFEST}{len(manifest['clips'])} clips, "
f"{len(manifest['ring']['scales'])} ring scales")
if "--media" in argv:
generate_media()
if __name__ == "__main__":
main(sys.argv[1:])
+37 -8
View File
@@ -73,6 +73,18 @@ def load_manifest(path: str | Path) -> list[Clip]:
return [_clip_from_dict(c) for c in data["clips"]]
def _scale_from_dict(s: dict[str, Any]) -> Scale:
"""A ring scale node, with its rotating clip pool (content-pipeline §11.1).
Accepts a `pool` list (the rotating model) and/or a legacy single `clip_id`.
The primary `clip_id` is the explicit one if given, else the first pool
member; `pool` defaults to `(clip_id,)` for a pre-pool manifest.
"""
pool = tuple(s.get("pool", []))
clip_id = s.get("clip_id") or (pool[0] if pool else "")
return Scale(id=s["id"], clip_id=clip_id, pool=pool)
def load_ring(path: str | Path) -> ScaleRing | None:
"""Load the optional `ring` section as a `ScaleRing`, or None if absent.
@@ -85,9 +97,7 @@ def load_ring(path: str | Path) -> ScaleRing | None:
ring = data.get("ring")
if not ring:
return None
scales = tuple(
Scale(id=s["id"], clip_id=s["clip_id"]) for s in ring.get("scales", [])
)
scales = tuple(_scale_from_dict(s) for s in ring.get("scales", []))
transitions = tuple(
Transition(file=t["file"], model=t.get("model", ""))
for t in ring.get("transitions", [])
@@ -97,27 +107,46 @@ def load_ring(path: str | Path) -> ScaleRing | None:
def ring_to_dict(ring: ScaleRing, clips: list[Clip]) -> dict:
"""JSON form of the ring for the API: ordered scales (with their clip title)
and the per-edge transitions."""
and the per-edge transitions.
Each scale carries its full rotating `pool` (clip id + title per member) plus
the primary `clip_id`/`title` for back-compat. The client lands on a member
chosen at the API boundary (`/api/ring/advance` → `target_clip_id`)."""
titles = {c.id: c.title for c in clips}
return {
"scales": [
{"id": s.id, "clip_id": s.clip_id, "title": titles.get(s.clip_id, s.id)}
{
"id": s.id,
"clip_id": s.clip_id,
"title": titles.get(s.clip_id, s.id),
"pool": [
{"clip_id": cid, "title": titles.get(cid, cid)}
for cid in s.members
],
}
for s in ring.scales
],
"transitions": [{"file": t.file, "model": t.model} for t in ring.transitions],
}
def ring_move_to_dict(move: RingMove, ring: ScaleRing) -> dict:
def ring_move_to_dict(
move: RingMove, ring: ScaleRing, chosen_clip_id: str | None = None
) -> dict:
"""JSON form of an encoder move: the landing scale's clip and the ordered
transition clips to play (with direction). `fast` flags a collapsed fast-spin
pass; the single step then carries `blended` so the renderer plays it quick."""
pass; the single step then carries `blended` so the renderer plays it quick.
`target_clip_id` is the pool member the player should load on landing. The
caller passes the random pick (`pick_clip_id(landed_scale, random.random())`,
content-pipeline §11.1); when omitted it falls back to the scale's primary
`clip_id` (deterministic — keeps pre-pool callers/tests working)."""
return {
"from_index": move.from_index,
"to_index": move.to_index,
"wrapped": move.wrapped,
"fast": move.fast,
"target_clip_id": scale_at(ring, move.to_index).clip_id,
"target_clip_id": chosen_clip_id or scale_at(ring, move.to_index).clip_id,
"steps": [
{
"edge": st.edge,
File diff suppressed because it is too large Load Diff
+25 -12
View File
@@ -1,16 +1,26 @@
"""Generate cheap placeholder media for the scale RING (scales design §3).
"""Generate cheap placeholder media for the full 5-scale RING (scales design §3).
The ring needs >=2 neutral scales to be demonstrable. This script produces
cheap, offline, synthetic placeholders for the two true-PD scales the ring adds —
**cosmos** (NASA/Hubble) and **abyss** (NOAA Ocean Exploration) — plus the short
zoom/warp **transition** clips between adjacent scales. Forest reuses the real
POC base from `setup_sample_media.py` (or a synthetic placeholder if absent).
SUPERSEDED (session 0016): the ring now uses real strict-PD footage in a ROTATING
POOL per scale, regenerated by `simulator/build_pool_manifest.py` (which also makes
the new coast-edge transition placeholders). This generator built the older
one-base-per-scale forest ring (cosmos/orbit/forest/reef/abyss) and is kept only
for reference / a clean-room placeholder rebuild; it does NOT match the current
pool manifest or ring order (cosmos → orbit → coast → reef → abyss).
Produces offline synthetic placeholders for all five ring scales — **cosmos**
(NASA/Hubble), **orbit** (NASA/ISS), **forest** (NPS/USGS), **reef** (NOAA Ocean
Exploration), and **abyss** (NOAA Ocean Exploration) — plus the short zoom/warp
**transition** clips between adjacent scales. Forest reuses the real POC base from
`setup_sample_media.py` (or a synthetic placeholder if absent); all others are
generated as labelled solid-tint clips.
This is deliberately the CHEAP path: the manifest records the real true-PD
provenance (NASA 17 U.S.C. §105 / NOAA PD), but the bytes here are labelled
synthetic placeholders for look-tuning the ring navigation. The expensive real
multi-strength flow-stabilized Right re-bake is DEFERRED until the ring is liked,
so the new scales carry a raw base only (no Right variants).
provenance (NASA / NPS / USGS / NOAA, 17 U.S.C. §105), but the bytes here are
labelled synthetic placeholders for look-tuning the ring navigation. The Right dream
is real-time (Kuwahara WebGL shader), so no per-clip ML bake is needed; each scale
carries a raw base only. Real strict-PD bases replace these placeholders via the
content pipeline (see `docs/content-sourcing.md`).
Usage: python simulator/setup_scales_media.py
Requires: ffmpeg on PATH (or `pip install imageio-ffmpeg`).
@@ -31,12 +41,15 @@ FPS = 25
# Each scale: (dir, base color, on-screen placeholder label).
SCALES = {
"cosmos": ("0x05060f", "COSMOS — NASA/Hubble (PD placeholder)"),
"forest": ("0x12301a", "FOREST — Yosemite (POC/placeholder)"),
"orbit": ("0x081427", "ORBIT — NASA/ISS (PD placeholder)"),
"forest": ("0x12301a", "FOREST — NPS/USGS (POC/placeholder)"),
"reef": ("0x06303a", "REEF — NOAA Ocean Exploration (PD placeholder)"),
"abyss": ("0x021016", "ABYSS — NOAA Ocean Exploration (PD placeholder)"),
}
# Ring edges (adjacent pairs + the abyss->cosmos wrap), matching the manifest.
EDGES = [("cosmos", "forest"), ("forest", "abyss"), ("abyss", "cosmos")]
EDGES = [("cosmos", "orbit"), ("orbit", "forest"), ("forest", "reef"),
("reef", "abyss"), ("abyss", "cosmos")]
def _ffmpeg() -> str:
+113 -13
View File
@@ -31,26 +31,48 @@ window.addEventListener("error", (e) =>
showError(`${e.message} @ ${(e.filename || "").split("/").pop()}:${e.lineno}`));
let clipsById = {}; // id -> clip manifest entry
let ring = null; // {scales:[{id,clip_id,title}], transitions:[...]} or null
let ring = null; // {scales:[{id,clip_id,title,pool:[...]}], transitions:[...]} or null
let serverRing = false; // true when /api/ring served a real ring (vs the synthesized fallback)
let ringIndex = 0; // current scale position on the ring
let activeClipId = null; // the pool member chosen for the current scale landing
let currentClipId = null; // base media currently loaded (reset to force a reload)
let busy = false; // true while a ring transition is playing
let lastOverlay = null; // {level, intensity} from the most-recent renderOverlay call; drives per-frame track animation
async function loadData() {
const clips = (await (await fetch("/api/clips")).json()).clips || [];
clipsById = Object.fromEntries(clips.map((c) => [c.id, c]));
const r = await fetch("/api/ring");
serverRing = r.ok;
ring = r.ok ? await r.json() : null;
if (!ring && clips.length) {
// No ring: single-clip mode — synthesize a 1-scale ring over clip 0.
ring = { scales: [{ id: clips[0].id, clip_id: clips[0].id, title: clips[0].title }], transitions: [] };
// No ring: single-clip mode — synthesize a 1-scale ring (pool of one) over clip 0.
ring = { scales: [{ id: clips[0].id, clip_id: clips[0].id, title: clips[0].title, pool: [{ clip_id: clips[0].id, title: clips[0].title }] }], transitions: [] };
}
ringIndex = 0;
}
// Land on the current scale: pick a random pool member (content-pipeline §11.1).
// A real server ring picks via a delta=0 advance (Python-canonical pick); the
// synthesized fallback ring has a pool of one, so it just takes that member.
async function landScale() {
const scale = ring && ring.scales[ringIndex];
if (!scale) { activeClipId = null; return; }
if (serverRing) {
try {
const resp = await fetch("/api/ring/advance", {
method: "POST", headers: { "content-type": "application/json" },
body: JSON.stringify({ from_index: ringIndex, delta: 0 }),
});
if (resp.ok) { activeClipId = (await resp.json()).target_clip_id; currentClipId = null; return; }
} catch (_) { /* fall through to the primary member */ }
}
activeClipId = scale.clip_id;
currentClipId = null;
}
function activeClip() {
if (!ring) return null;
return clipsById[ring.scales[ringIndex].clip_id] || null;
return clipsById[activeClipId] || null;
}
function mediaUrl(file) { return "/media/" + file; }
@@ -182,6 +204,14 @@ function paintLoop() {
gl.drawArrays(gl.TRIANGLES, 0, 3);
paint.style.filter = busy ? "none" : gradeFilter;
}
// Animate per-frame: re-place keyframed tracks AND re-evaluate time-windowed
// labels (a label enters/leaves frame as the loop plays) against playback time.
const c = activeClip();
if (lastOverlay && lastOverlay.level > 0 && c &&
c.annotations.some((a) => (a.track && a.track.length) ||
typeof a.appear === "number" || typeof a.disappear === "number")) {
renderOverlay(lastOverlay.level, lastOverlay.intensity);
}
requestAnimationFrame(paintLoop);
}
@@ -232,17 +262,72 @@ function chip(x, y, label, conf, kind, parent) {
}
}
// Interpolate an annotation's box at loop-normalized time t (0..1). Static labels
// (no track) return their fixed box. Linear between sorted keyframes; clamps to ends.
function boxAt(a, t) {
if (!a.track || !a.track.length) return a.box;
const ks = a.track;
if (t <= ks[0].t) return ks[0].box;
if (t >= ks[ks.length - 1].t) return ks[ks.length - 1].box;
for (let i = 1; i < ks.length; i++) {
if (t <= ks[i].t) {
const a0 = ks[i - 1], a1 = ks[i];
const f = (t - a0.t) / (a1.t - a0.t);
return a0.box.map((v, j) => v + (a1.box[j] - v) * f);
}
}
return ks[ks.length - 1].box;
}
// Loop-normalized playback time of the base video (0..1), for track interpolation.
function loopT() {
return vid && vid.duration ? (vid.currentTime % vid.duration) / vid.duration : 0;
}
// --- Progressive labels & emotions (content-pipeline §11.3 / §11.4) ---
const clamp = (v, lo, hi) => Math.max(lo, Math.min(hi, v));
// The Left level at which a label first appears. From `salience` (1..4): high
// salience appears early (first_level = 5 - salience), so raising Left brings in
// lower-salience objects. Falls back to a legacy flat `min_level`, else 1.
function firstLevel(a) {
if (typeof a.salience === "number") return clamp(5 - a.salience, 1, 4);
if (typeof a.min_level === "number") return a.min_level;
return 1;
}
// A tiered string escalates with a knob: a LIST is indexed by tier-1 (clamped to
// the available tiers); a plain string is static (tier ignored — back-compat).
function pickTier(value, tier) {
if (Array.isArray(value)) return value[clamp(tier - 1, 0, value.length - 1)];
return value;
}
function tierCount(value) { return Array.isArray(value) ? value.length : 1; }
// Time-windowed presence (§11.2): a label with `appear`/`disappear` (loop-norm t)
// shows only while on screen. A `disappear < appear` window wraps across the loop
// seam (visible when t >= appear OR t <= disappear). No window => always present.
function inWindow(a, t) {
const hasA = typeof a.appear === "number", hasD = typeof a.disappear === "number";
if (!hasA && !hasD) return true;
const ap = hasA ? a.appear : 0, dis = hasD ? a.disappear : 1;
return ap <= dis ? (t >= ap && t <= dis) : (t >= ap || t <= dis);
}
function renderOverlay(level, intensity) {
const clip = activeClip();
overlay.innerHTML = "";
if (!clip || level <= 0) { overlay.style.opacity = "0"; return; }
overlay.style.opacity = String(intensity);
const strings = (clip.strings && clip.strings.en) || {};
const t = loopT();
let shown = 0;
for (const a of clip.annotations) {
if (a.min_level > level) continue;
const first = firstLevel(a);
if (level < first) continue; // salience gating: low Left = fewer/general labels
if (!inWindow(a, t)) continue; // time-windowed: only while the object is on screen
shown++;
const [x, y, w, h] = a.box.map((n) => n * 100);
const [x, y, w, h] = boxAt(a, t).map((n) => n * 100);
const measure = a.key.startsWith("measure.");
const kind = measure ? "measure" : "detect";
reticle(x, y, w, h, "hud-reticle " + kind, overlay);
@@ -252,7 +337,12 @@ function renderOverlay(level, intensity) {
svg("line", { x1: cx - 1, y1: cyc, x2: cx + 1, y2: cyc, class: "hud-reticle measure" }, overlay);
svg("line", { x1: cx, y1: cyc - 1, x2: cx, y2: cyc + 1, class: "hud-reticle measure" }, overlay);
}
chip(x, y, strings[a.key] || a.key, measure ? null : confidence(a.key), kind, overlay);
// LEFT detail tier: a detection escalates general -> scientific+fact as Left
// rises above where it first appeared; measurements stay single-tier.
const raw = strings[a.key];
const tier = measure ? 1 : clamp(level - first + 1, 1, tierCount(raw));
const label = pickTier(raw !== undefined ? raw : a.key, tier);
chip(x, y, label, measure ? null : confidence(a.key), kind, overlay);
}
// Global status tag — the machine announcing it is analyzing, escalating with Left.
// Right-anchored at the frame edge; the plate is sized from the text's real bbox
@@ -266,13 +356,14 @@ function renderOverlay(level, intensity) {
height: b.height + pad * 1.2, rx: 0.4, class: "hud-chip-bg detect" });
overlay.insertBefore(bg, st);
}
lastOverlay = { level, intensity };
}
// Affect channel: soft, glowing emotion-words surfaced only when BOTH knobs are
// up. `strength` = min(left, right); `intensity` is the layer opacity. Words are
// placed at authored scene points (no boxes — feelings are scene-level) and read
// softer than the clinical reticles — the dream leaking into the machine's read.
function renderAffect(strength, intensity) {
function renderAffect(strength, intensity, right) {
const clip = activeClip();
if (!affectLayer) return; // tolerate a stale page missing the affect layer
affectLayer.innerHTML = "";
@@ -283,14 +374,21 @@ function renderAffect(strength, intensity) {
if (f.min_level > strength) continue;
const [x, y] = f.at.map((n) => n * 100);
const t = svg("text", { x, y, "text-anchor": "middle", class: "hud-affect" }, affectLayer);
t.textContent = strings[f.key] || f.key;
// RIGHT emotion tier: the vocabulary escalates basic -> compound as the Right
// knob rises (appearance still gated by min(left,right) above).
const raw = strings[f.key];
t.textContent = pickTier(raw !== undefined ? raw : f.key, clamp(right || 0, 1, tierCount(raw)));
}
}
function renderScaleReadout() {
if (!ring) return;
const s = ring.scales[ringIndex];
$("scale-name").textContent = `${s.title} (${ringIndex + 1}/${ring.scales.length})`;
const poolN = (s.pool && s.pool.length) || 1;
const member = (activeClip() && activeClip().title) || s.title;
// scale id · the chosen pool member · position on the ring (pool size if >1)
const poolTag = poolN > 1 ? ` · pool ${poolN}` : "";
$("scale-name").textContent = `${s.id} · ${member} (${ringIndex + 1}/${ring.scales.length}${poolTag})`;
}
function controls() {
@@ -322,7 +420,7 @@ async function update() {
ensureClipMedia();
applyVideoLook(data.plan.grade.tone, data.plan.dream.intensity);
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
renderAffect(data.plan.affect.strength, data.plan.affect.intensity);
renderAffect(data.plan.affect.strength, data.plan.affect.intensity, data.plan.dream.strength);
const b = document.getElementById("err-banner");
if (b) b.remove(); // render succeeded — clear any prior error
} catch (err) {
@@ -381,7 +479,8 @@ async function advance(delta) {
await playTransition(step.file, step.blended);
}
ringIndex = move.to_index;
currentClipId = null; // force the target scale's base media to (re)load
activeClipId = move.target_clip_id; // the randomly-picked pool member to load
currentClipId = null; // force the target scale's base media to (re)load
renderScaleReadout();
} finally {
busy = false;
@@ -420,6 +519,7 @@ async function main() {
devLiveReload();
try { initPaint(); } catch (e) { paintOK = false; paint.style.display = "none"; showError("WebGL init: " + e.message); }
await loadData();
await landScale(); // pick the initial scale's pool member before first render
renderScaleReadout();
for (const id of ["content", "left", "right", "mood"]) {
$(id).addEventListener("input", debounced);
+54
View File
@@ -0,0 +1,54 @@
from tools.pipeline.ffmpeg_ops import frame_args, crossfade_loop_args, transcode_args
def test_frame_args_trims_scales_pads_and_strips_audio():
args = frame_args(
"src.mp4", "out.mp4",
start=2.0, duration=8.0, width=1920, height=1080, ff="ffmpeg",
)
assert args[0] == "ffmpeg"
assert "-ss" in args and args[args.index("-ss") + 1] == "2.0"
assert "-t" in args and args[args.index("-t") + 1] == "8.0"
assert "-an" in args # audio stripped (bases are video-only)
vf = args[args.index("-vf") + 1]
assert "scale=1920:1080:force_original_aspect_ratio=decrease" in vf
assert "pad=1920:1080" in vf
assert "format=yuv420p" in vf
assert args[-1] == "out.mp4"
def test_crossfade_loop_builds_head_mid_tail_xfade_concat():
args = crossfade_loop_args("in.mp4", "loop.mp4", duration=8.0, overlap=1.0)
fc = args[args.index("-filter_complex") + 1]
assert "trim=0:1.0" in fc # head = first O secs
assert "trim=1.0:7.0" in fc # mid = [O, D-O]
assert "trim=7.0:8.0" in fc # tail = last O secs
assert "xfade=transition=fade:duration=1.0:offset=0" in fc
assert "concat=n=2:v=1" in fc
assert "-an" in args
assert args[-1] == "loop.mp4"
def test_crossfade_loop_rejects_overlap_too_large():
import pytest
with pytest.raises(ValueError):
crossfade_loop_args("in.mp4", "loop.mp4", duration=4.0, overlap=2.0)
def test_crossfade_loop_rejects_nonpositive_overlap():
import pytest
with pytest.raises(ValueError):
crossfade_loop_args("in.mp4", "loop.mp4", duration=8.0, overlap=0.0)
def test_transcode_args_high_profile_even_dims_faststart():
args = transcode_args("loop.mp4", "proxy.mp4", width=1920, height=1080,
crf=20, fps=30)
assert "-profile:v" in args and args[args.index("-profile:v") + 1] == "high"
assert "-crf" in args and args[args.index("-crf") + 1] == "20"
vf = args[args.index("-vf") + 1]
assert "scale=1920:1080:force_original_aspect_ratio=decrease" in vf
assert "fps=30" in vf
assert "+faststart" in args
assert "-an" in args
assert args[-1] == "proxy.mp4"
+42
View File
@@ -0,0 +1,42 @@
from pathlib import Path
import pytest
from tools.pipeline.run import probe_duration, process_clip, resolve_ffmpeg
_FOREST = Path("simulator/sample_media/forest/base.mp4")
def _ffmpeg_available() -> bool:
try:
resolve_ffmpeg()
return True
except Exception:
return False
_HAS_FF = _ffmpeg_available()
@pytest.mark.skipif(not _FOREST.exists(),
reason="forest POC base absent (run simulator/setup_sample_media.py)")
def test_probe_duration_reads_forest_base():
if not _FOREST.exists():
pytest.skip("forest POC base absent")
assert probe_duration(_FOREST) > 0
@pytest.mark.skipif(not _HAS_FF,
reason="neither system ffmpeg nor imageio-ffmpeg available")
@pytest.mark.skipif(not _FOREST.exists(),
reason="forest POC base absent (run simulator/setup_sample_media.py)")
def test_process_clip_emits_proxy_and_master(tmp_path):
out = process_clip(_FOREST, tmp_path, overlap=0.5, start=0.0, duration=4.0)
assert out["proxy"].exists() and out["master"].exists()
# Proxy is exactly 1920×1080 — verified with cv2, no ffprobe dependency.
import cv2
cap = cv2.VideoCapture(str(out["proxy"]))
w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
cap.release()
assert (w, h) == (1920, 1080)
+47
View File
@@ -0,0 +1,47 @@
from tools.pipeline.manifest import upsert_clip, add_ring_scale, rebuild_ring_edges
def _base():
return {"clips": [{"id": "cosmos", "title": "Cosmos"}],
"ring": {"scales": [{"id": "cosmos", "clip_id": "cosmos"}],
"transitions": []}}
def test_upsert_clip_replaces_by_id_else_appends():
m = _base()
m = upsert_clip(m, {"id": "reef", "title": "Reef"})
assert [c["id"] for c in m["clips"]] == ["cosmos", "reef"]
m = upsert_clip(m, {"id": "reef", "title": "Coral Reef"}) # replace, not dup
assert [c["id"] for c in m["clips"]] == ["cosmos", "reef"]
assert m["clips"][1]["title"] == "Coral Reef"
def test_add_ring_scale_inserts_after_named_scale():
m = _base()
m = add_ring_scale(m, "orbit", "orbit", after="cosmos")
assert [s["id"] for s in m["ring"]["scales"]] == ["cosmos", "orbit"]
def test_rebuild_ring_edges_one_transition_per_edge_with_wrap():
m = _base()
m = add_ring_scale(m, "orbit", "orbit", after="cosmos")
m = add_ring_scale(m, "abyss", "abyss", after="orbit")
m = rebuild_ring_edges(m)
files = [t["file"] for t in m["ring"]["transitions"]]
assert files == [
"transitions/cosmos-orbit.mp4",
"transitions/orbit-abyss.mp4",
"transitions/abyss-cosmos.mp4", # wrap edge
]
def test_add_ring_scale_appends_when_after_missing():
m = _base()
m = add_ring_scale(m, "abyss", "abyss", after="nope")
assert [s["id"] for s in m["ring"]["scales"]] == ["cosmos", "abyss"]
def test_rebuild_ring_edges_single_scale_self_wrap():
m = _base() # one scale: cosmos
m = rebuild_ring_edges(m)
assert [t["file"] for t in m["ring"]["transitions"]] == ["transitions/cosmos-cosmos.mp4"]
+20
View File
@@ -0,0 +1,20 @@
from tools.pipeline.provenance import Provenance
def test_provenance_to_dict_roundtrip():
p = Provenance(
scale="abyss",
license="public-domain (US Gov, 17 U.S.C. §105)",
source="NOAA Ocean Exploration",
url="https://oceanexplorer.noaa.gov/",
fetched_at="2026-06-24",
)
d = p.to_dict()
assert d == {
"scale": "abyss",
"license": "public-domain (US Gov, 17 U.S.C. §105)",
"source": "NOAA Ocean Exploration",
"url": "https://oceanexplorer.noaa.gov/",
"fetched_at": "2026-06-24",
}
assert Provenance.from_dict(d) == p
+34
View File
@@ -8,6 +8,7 @@ from player.ring import (
Transition,
TransitionStep,
advance_ring,
pick_clip_id,
scale_at,
)
@@ -206,3 +207,36 @@ def test_fast_spin_noop_on_degenerate_ring():
move = advance_ring(r, from_index=0, delta=9, fast_spin_threshold=3)
assert move.fast is False
assert move.steps == ()
# --- rotating clip pool (content-pipeline design §11.1) ---
def test_scale_without_pool_is_a_pool_of_one():
s = Scale(id="cosmos", clip_id="cosmos")
assert s.members == ("cosmos",)
# any r maps to the single member
assert pick_clip_id(s, 0.0) == "cosmos"
assert pick_clip_id(s, 0.99) == "cosmos"
def test_scale_with_pool_exposes_all_members():
s = Scale(id="coast", clip_id="coast_a", pool=("coast_a", "coast_b", "coast_c"))
assert s.members == ("coast_a", "coast_b", "coast_c")
def test_pick_clip_id_selects_member_by_injected_r():
s = Scale(id="coast", clip_id="a", pool=("a", "b", "c", "d"))
# r in [0,1) partitions the pool into len equal buckets
assert pick_clip_id(s, 0.0) == "a"
assert pick_clip_id(s, 0.24) == "a"
assert pick_clip_id(s, 0.25) == "b"
assert pick_clip_id(s, 0.5) == "c"
assert pick_clip_id(s, 0.75) == "d"
def test_pick_clip_id_clamps_out_of_range_r():
s = Scale(id="coast", clip_id="a", pool=("a", "b", "c"))
assert pick_clip_id(s, -5.0) == "a" # below 0 clamps to first
assert pick_clip_id(s, 1.0) == "c" # 1.0 must not index past the end
assert pick_clip_id(s, 999.0) == "c"
+61
View File
@@ -189,3 +189,64 @@ def test_ring_404_when_no_ring_in_manifest(client):
def test_ring_advance_rejects_bad_delta(ring_client):
resp = ring_client.post("/api/ring/advance", json={"from_index": 0, "delta": "x"})
assert resp.status_code == 422
# --- rotating clip pool (content-pipeline §11.1) ---
@pytest.fixture
def pool_client(tmp_path):
p = tmp_path / "manifest.json"
members = ["coast_a", "coast_b", "coast_c", "coast_d"]
clips = [{"id": "cosmos", "title": "Cosmos", "base_file": "cosmos/base.mp4",
"license": "PD", "source": "NASA", "right_variants": {},
"annotations": [], "affect": [], "strings": {}}]
clips += [{"id": m, "title": m, "base_file": f"{m}/base.mp4", "license": "PD",
"source": "NPS", "right_variants": {}, "annotations": [], "affect": [],
"strings": {}} for m in members]
p.write_text(json.dumps({
"clips": clips,
"ring": {
"scales": [
{"id": "cosmos", "clip_id": "cosmos"},
{"id": "coast", "clip_id": members[0], "pool": members},
],
"transitions": [
{"file": "transitions/cosmos-coast.mp4"},
{"file": "transitions/coast-cosmos.mp4"},
],
},
}))
return TestClient(create_app(manifest_path=p)), members
def test_ring_exposes_each_scales_pool(pool_client):
client, members = pool_client
data = client.get("/api/ring").json()
coast = next(s for s in data["scales"] if s["id"] == "coast")
assert [m["clip_id"] for m in coast["pool"]] == members
# a single-clip scale still reports a pool of one
cosmos = next(s for s in data["scales"] if s["id"] == "cosmos")
assert [m["clip_id"] for m in cosmos["pool"]] == ["cosmos"]
def test_advance_into_a_pool_lands_on_a_member(pool_client):
client, members = pool_client
# landing on the coast scale (index 1) must pick one of its pool members
seen = set()
for _ in range(40):
data = client.post("/api/ring/advance", json={"from_index": 0, "delta": 1}).json()
assert data["to_index"] == 1
assert data["target_clip_id"] in members
seen.add(data["target_clip_id"])
# over many landings the random pick should cover more than one member
assert len(seen) > 1
def test_delta_zero_is_an_initial_pool_pick(pool_client):
client, members = pool_client
# a no-op (delta 0) advance on the pool scale is the initial / re-roll pick
data = client.post("/api/ring/advance", json={"from_index": 1, "delta": 0}).json()
assert data["to_index"] == 1
assert data["steps"] == []
assert data["target_clip_id"] in members
+5
View File
@@ -0,0 +1,5 @@
"""Content pipeline: source -> frame -> loop -> transcode -> manifest.
Pure ffmpeg-argument builders (ffmpeg_ops), a thin runner (run), provenance
records, and manifest assembly. See docs/superpowers/specs/2026-06-24-content-pipeline-design.md.
"""
+74
View File
@@ -0,0 +1,74 @@
"""Pure ffmpeg argument builders (no I/O) for the content pipeline.
Each function returns a list[str] ready for ffmpeg, so command construction is
unit-testable without running ffmpeg. tools/pipeline/run.py executes them. The
ffmpeg binary name is injected (default "ffmpeg").
"""
from __future__ import annotations
def _fit(width: int, height: int) -> str:
"""A scale+pad filter chain fitting any input into width×height (letterbox),
fixing SAR and pixel format so downstream concat/xfade get identical geometry."""
return (
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
"setsar=1,format=yuv420p"
)
def frame_args(src, dst, *, start: float, duration: float,
width: int, height: int, ff: str = "ffmpeg") -> list[str]:
"""Stage 2 — trim [start, start+duration], fit to width×height, strip audio.
Produces a high-quality mezzanine for the loop stage."""
return [
ff, "-y", "-ss", str(start), "-t", str(duration),
"-i", str(src), "-vf", _fit(width, height), "-an",
"-c:v", "libx264", "-preset", "medium", "-crf", "16",
"-pix_fmt", "yuv420p", "-movflags", "+faststart", str(dst),
]
def crossfade_loop_args(src, dst, *, duration: float, overlap: float,
fps: int = 30, ff: str = "ffmpeg") -> list[str]:
"""Stage 3 — make `src` loop seamlessly by crossfading its tail over its head.
Output length = duration overlap. Requires overlap < duration/2 so a non-empty
middle remains.
fps is applied after each trim so xfade receives a constant-frame-rate stream
(xfade requires CFR; a raw trim output reports rate 1/0 which xfade rejects)."""
if overlap <= 0 or overlap >= duration / 2:
raise ValueError(f"overlap {overlap} must be in (0, duration/2={duration/2})")
d, o = duration, overlap
fc = (
f"[0:v]trim=0:{o},setpts=PTS-STARTPTS,fps={fps}[head];"
f"[0:v]trim={o}:{d - o},setpts=PTS-STARTPTS,fps={fps}[mid];"
f"[0:v]trim={d - o}:{d},setpts=PTS-STARTPTS,fps={fps}[tail];"
f"[tail][head]xfade=transition=fade:duration={o}:offset=0[xf];"
f"[xf][mid]concat=n=2:v=1,format=yuv420p[out]"
)
return [
ff, "-y", "-i", str(src), "-filter_complex", fc,
"-map", "[out]", "-an",
"-c:v", "libx264", "-preset", "medium", "-crf", "16",
"-pix_fmt", "yuv420p", "-movflags", "+faststart", str(dst),
]
def transcode_args(src, dst, *, width: int, height: int, crf: int,
fps: int = 30, ff: str = "ffmpeg") -> list[str]:
"""Stage 4 — encode a final deliverable (master or proxy). Master: source res
capped ~3840 wide, crf 18. Proxy: 1920×1080, crf 20. Both H.264 High, yuv420p,
even dims, +faststart, video-only (spec §6)."""
vf = (
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
f"setsar=1,fps={fps},format=yuv420p"
)
return [
ff, "-y", "-i", str(src), "-vf", vf, "-an",
"-c:v", "libx264", "-profile:v", "high", "-preset", "slow",
"-crf", str(crf), "-pix_fmt", "yuv420p",
"-movflags", "+faststart", str(dst),
]
+55
View File
@@ -0,0 +1,55 @@
"""Stage 6 — assemble manifest entries (spec §3.6). Pure dict transforms over a
loaded manifest dict; the caller does the json read/write. Idempotent by id so
re-running earlier stages never clobbers authored labels."""
from __future__ import annotations
import copy
def upsert_clip(manifest: dict, clip: dict) -> dict:
"""Replace the clip with the same id, or append it. Returns a new manifest."""
m = copy.deepcopy(manifest)
clips = m.setdefault("clips", [])
for i, c in enumerate(clips):
if c["id"] == clip["id"]:
clips[i] = clip
return m
clips.append(clip)
return m
def add_ring_scale(manifest: dict, scale_id: str, clip_id: str,
after: str | None = None) -> dict:
"""Insert a scale into ring.scales after the named scale (or append if `after`
is None / not found). No-op if scale_id already present. Returns a new manifest."""
m = copy.deepcopy(manifest)
ring = m.setdefault("ring", {"scales": [], "transitions": []})
scales = ring.setdefault("scales", [])
if any(s["id"] == scale_id for s in scales):
return m
entry = {"id": scale_id, "clip_id": clip_id}
idx = next((i for i, s in enumerate(scales) if s["id"] == after), None)
if idx is None:
scales.append(entry)
else:
scales.insert(idx + 1, entry)
return m
def rebuild_ring_edges(manifest: dict) -> dict:
"""Rebuild ring.transitions to one placeholder entry per ring edge (N scales ->
N edges incl. the wrap), preserving any existing model string for an edge.
Returns a new manifest."""
m = copy.deepcopy(manifest)
ring = m.setdefault("ring", {"scales": [], "transitions": []})
scales = ring.get("scales", [])
existing = {t["file"]: t.get("model", "") for t in ring.get("transitions", [])}
edges = []
n = len(scales)
for i in range(n):
a, b = scales[i]["id"], scales[(i + 1) % n]["id"]
file = f"transitions/{a}-{b}.mp4"
edges.append({"file": file, "model": existing.get(file, "placeholder-zoom")})
ring["transitions"] = edges
return m
+29
View File
@@ -0,0 +1,29 @@
"""Stage 1 — the per-clip strict-PD provenance record (spec §3.1).
Captured at source time so license + source + URL travel with the media. The
'no explicit license -> assume PD, verify' trap (scales design §2.1) applies:
"free stock" (Pexels/Pixabay) is NOT public domain.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
@dataclass(frozen=True)
class Provenance:
scale: str
license: str
source: str
url: str
fetched_at: str # ISO date; passed in (no clock here -> deterministic)
def to_dict(self) -> dict:
return asdict(self)
@classmethod
def from_dict(cls, d: dict) -> "Provenance":
return cls(
scale=d["scale"], license=d["license"], source=d["source"],
url=d["url"], fetched_at=d["fetched_at"],
)
+67
View File
@@ -0,0 +1,67 @@
"""Thin runner: probe the source duration, then execute the frame -> loop ->
transcode stages with tools.pipeline.ffmpeg_ops. The pure arg builders are
unit-tested; this glue is covered by the opt-in integration test."""
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
from .ffmpeg_ops import crossfade_loop_args, frame_args, transcode_args
MASTER_MAX_W = 3840
def resolve_ffmpeg() -> str:
"""ffmpeg from PATH, else the bundled imageio-ffmpeg binary (matches
simulator/setup_scales_media.py). The Pi/dev box may lack a system ffmpeg."""
if shutil.which("ffmpeg"):
return "ffmpeg"
import imageio_ffmpeg
return imageio_ffmpeg.get_ffmpeg_exe()
def probe_duration(src) -> float:
"""Clip duration in seconds via cv2 (frame_count / fps) — avoids a system
ffprobe dependency (imageio-ffmpeg ships ffmpeg only; the Pi has no ffprobe)."""
import cv2
cap = cv2.VideoCapture(str(src))
fps = cap.get(cv2.CAP_PROP_FPS) or 0.0
frames = cap.get(cv2.CAP_PROP_FRAME_COUNT) or 0.0
cap.release()
if fps <= 0:
raise ValueError(f"could not read fps from {src}")
return frames / fps
def _run(args: list[str]) -> None:
subprocess.run(args, check=True, capture_output=True)
def process_clip(src, out_dir, *, start: float = 0.0, duration: float | None = None,
overlap: float = 1.0, master_w: int = MASTER_MAX_W, master_h: int = 2160,
ff: str | None = None) -> dict:
"""Run stages 24 for one clip; return {'master','proxy','loop','mezzanine'} paths.
`duration` defaults to the full source length (minus the trim start).
Note: `crossfade_loop_args` uses a fixed fps only to satisfy xfade's
constant-frame-rate requirement; the final output fps is set by `transcode_args`
(both default 30 — keep in sync if overridden)."""
ff = ff or resolve_ffmpeg()
src = Path(src)
out_dir = Path(out_dir)
out_dir.mkdir(parents=True, exist_ok=True)
if duration is None:
duration = probe_duration(src) - start
mezz = out_dir / "mezzanine.mp4"
loop = out_dir / "loop.mp4"
master = out_dir / "master.mp4"
proxy = out_dir / "base.mp4" # the proxy is what the manifest's base_file points at
_run(frame_args(src, mezz, start=start, duration=duration,
width=master_w, height=master_h, ff=ff))
_run(crossfade_loop_args(mezz, loop, duration=duration, overlap=overlap, ff=ff))
_run(transcode_args(loop, master, width=master_w, height=master_h, crf=18, ff=ff))
_run(transcode_args(loop, proxy, width=1920, height=1080, crf=20, ff=ff))
return {"mezzanine": mezz, "loop": loop, "master": master, "proxy": proxy}