Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Ben Stull
|
3ef21fb7f7 |
@@ -5,6 +5,3 @@ __pycache__/
|
||||
media/
|
||||
.superpowers/
|
||||
*.egg-info/
|
||||
# Simulator sample media (look-tuning only; populate via setup_sample_media.py
|
||||
# / setup_scales_media.py). All scale + transition binaries are gitignored.
|
||||
simulator/sample_media/**/*.mp4
|
||||
|
||||
+2
-65
@@ -123,43 +123,6 @@ longer *select* a pre-tagged clip; they drive an **alteration engine** (design
|
||||
|
||||
Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-player-alteration-core.md).
|
||||
|
||||
**Slice 2 — simulator-first alteration preview ✅ Done.** Merged to `main`
|
||||
(session 0009). Wires the alteration engine into the web simulator so the look is
|
||||
tunable by eye before any hardware, and **reconciles** the unmerged session-0007
|
||||
design with the merged session-0008 design (the Left-HUD conflict): Left is a
|
||||
**runtime overlay** (authored annotation track + per-language string tables),
|
||||
Right is a **discrete pre-baked** flow-stabilized variant, Dark/Light a **live**
|
||||
grade — superseding 0007's baked-HUD 5×5 grid. Engine: a parameterized
|
||||
`Calibration` (tuned by eye, baked into `DEFAULT_CALIBRATION`), `Restyle.variant`
|
||||
(discrete) replacing the continuous blend, and `AnalyticalOverlay.level`. Sim:
|
||||
`/api/alteration` + `/api/clips` over `simulator/clips.py`, an alteration-preview
|
||||
UI, and one neutral clip with a real flow-stabilized Right variant from the POC.
|
||||
Design:
|
||||
[`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md);
|
||||
plan:
|
||||
[`2026-06-07-reconciled-simulator-alteration-slice.md`](./superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md).
|
||||
|
||||
**Slice 3 — scale-ring navigation (simulator) ✅ Done.** Merged to `main`
|
||||
(session 0011). Adds the **endless rotary-encoder** control — *relative* (vs the
|
||||
absolute 0–4 experience knobs) — that walks a **closed ring** of neutral
|
||||
"scales of nature" clips, with placeholder AI zoom/warp **transitions** between
|
||||
adjacent scales and a small→large **wrap** (the infinite-zoom payoff, scales
|
||||
design §3). The navigation math is canonical Python (`player/ring.py`:
|
||||
`ScaleRing` + `advance_ring`, one transition clip per edge, played forward
|
||||
inward / reversed outward); the sim exposes `GET /api/ring` + `POST
|
||||
/api/ring/advance` and the browser only plays the returned transition(s) then
|
||||
settles on the target scale, keeping the live knob alteration on top. Two cheap
|
||||
**true-PD** scales were added so the ring is demonstrable — `cosmos` (NASA/Hubble)
|
||||
and `abyss` (NOAA Ocean Exploration), provenance recorded in the manifest, bytes
|
||||
generated as labelled placeholders by `simulator/setup_scales_media.py`. The
|
||||
expensive real multi-strength flow-stabilized Right re-bake is **deferred** (new
|
||||
scales carry a raw base only) until the ring is liked. Plan:
|
||||
[`2026-06-07-scale-ring-navigation.md`](./superpowers/plans/2026-06-07-scale-ring-navigation.md).
|
||||
Session 0012 added the design §3 **fast-spin** behavior on top: past an opt-in
|
||||
speed threshold (`DEFAULT_FAST_SPIN_THRESHOLD = 3`, `abs(delta)` as the spin-speed
|
||||
proxy) a quick spin collapses to a single **blended pass** instead of chaining
|
||||
every transition, so fast navigation stays responsive on placeholder media.
|
||||
|
||||
**Remaining slices (not started):**
|
||||
|
||||
- **Runtime renderer** — drive the single panoramic projector via mpv/ffmpeg;
|
||||
@@ -170,34 +133,8 @@ every transition, so fast navigation stays responsive on placeholder media.
|
||||
contract** with the firmware (the keyboard/serial stand-in already works via a
|
||||
`Controls` stream).
|
||||
- **White-noise generation** — runtime white/pink noise for that content position.
|
||||
- **Content pipeline + label track** — source strict-PD neutral bases, run them
|
||||
through the offline pipeline, and author the annotation tracks that drive the
|
||||
Left-HUD overlay. The Right dream is now **real-time** (Kuwahara WebGL shader,
|
||||
session 0013 — no per-clip ML bake), so the pipeline is
|
||||
**source → crossfade-loop → master+proxy transcode → hand-authored /
|
||||
motion-tracked labels → manifest**, not ML baking. Historical note: the
|
||||
flow-stabilized SD-turbo v2v bake (session 0012, `simulator/bake_right_variants.py`)
|
||||
was superseded when the operator found the baked-SD Right "too fluid" and the
|
||||
design pivoted to the deterministic real-time dream; `bake_right_variants.py`
|
||||
is now parked. Spec:
|
||||
[`2026-06-24-content-pipeline-design.md`](./superpowers/specs/2026-06-24-content-pipeline-design.md).
|
||||
|
||||
- **Increment 1 — deterministic tooling + 5-scale ring (session 0014) ✅ Done.**
|
||||
`tools/pipeline/` (frame/loop/transcode/manifest, pure + unit-tested + an
|
||||
opt-in integration test on real footage); the backward-compatible keyframed
|
||||
annotation-track schema with client interpolation; and the **full 5-scale ring**
|
||||
(cosmos → orbit → forest → reef → abyss) with orbit + reef as placeholder
|
||||
scales alongside the existing cosmos/abyss/forest. Plan:
|
||||
[`2026-06-24-content-pipeline-increment-1.md`](./superpowers/plans/2026-06-24-content-pipeline-increment-1.md).
|
||||
|
||||
- **Increment 2 — hybrid motion-track pass + author mode (pending).** The
|
||||
`tools/pipeline/track.py` classical OpenCV tracker + optional ML detect+track,
|
||||
plus a simulator **author mode** for reviewing/editing annotation tracks in the
|
||||
browser. Real strict-PD bases for all five scales sourced per
|
||||
[`docs/content-sourcing.md`](./content-sourcing.md).
|
||||
|
||||
- **Still deferred:** real **i2v ring transitions** (need a generative-video
|
||||
model + adjacent real bases); Pi renderer + serial/firmware.
|
||||
- **Offline v2v variant pipeline** — author the pre-baked Right restyle variants
|
||||
(the only paid AI step, design §9) + the multilingual label/string tables + TTS.
|
||||
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
|
||||
variant" flag (sub-project 2 territory, design §13).
|
||||
|
||||
|
||||
+10
-53
@@ -287,33 +287,11 @@ Records point at files via `file_path`. Those files live on the player's drive,
|
||||
`file_path` values consistent with wherever you mount that drive on the machine
|
||||
that will eventually run the room.
|
||||
|
||||
## Playing with the simulator (alteration preview)
|
||||
## Playing with the simulator (curator's X-ray)
|
||||
|
||||
The simulator is a web stand-in for the installation's control panel. It runs the
|
||||
real `player.alteration` engine and **alters** a neutral base clip toward the knob
|
||||
state in the browser, so you can tune the *look* of the filter before any hardware
|
||||
exists. (The earlier selection-era "curator's X-ray" view was retired when the
|
||||
piece moved from *selecting* clips to *altering* them.)
|
||||
|
||||
**One-time setup — populate the sample footage** (look-tuning only; not shipped
|
||||
content):
|
||||
|
||||
python simulator/setup_sample_media.py # stage the forest neutral base
|
||||
python -m simulator.bake_right_variants --scale forest # real Right variants (SD on MPS, ~11 min)
|
||||
python simulator/setup_scales_media.py # the cosmos + abyss scales + ring transitions
|
||||
|
||||
`setup_sample_media.py` stages the forest neutral base from the session-0008 POC
|
||||
artifacts (`~/hef-poc/out/neutral.mp4`). `bake_right_variants.py` then produces
|
||||
the **real** flow-stabilized painterly Right variants (strengths 1–4) for the
|
||||
forest scale — SD img2img keyframes + Farneback optical-flow tweens on Apple MPS,
|
||||
the temporally-calm POC algorithm, productionized (scales design §1/§4). The
|
||||
keyframe-strength ramp per Right level is by-eye tunable in
|
||||
`simulator/bake_right_variants.py`. `setup_scales_media.py` makes the scale
|
||||
**ring** demonstrable: cheap synthetic placeholder bases for the two true-PD
|
||||
scales (`cosmos` = NASA/Hubble, `abyss` = NOAA Ocean Exploration) plus the
|
||||
per-edge zoom/warp transition clips — these scales carry a raw base only (no real
|
||||
Right variants yet; the baker can extend to them once their real footage is
|
||||
sourced). The `.mp4` binaries are gitignored.
|
||||
real `hef.selection` code against a synthetic catalog so you can feel whether the
|
||||
dials surface fitting pieces before any hardware exists.
|
||||
|
||||
**Run it (Docker):**
|
||||
|
||||
@@ -326,32 +304,11 @@ then open http://localhost:8000.
|
||||
pip install -e ".[sim]"
|
||||
make sim-local
|
||||
|
||||
**What you see and can do:**
|
||||
**What you see:** the five real dials (mode + Left/Right/Dark/Light), the model
|
||||
knobs (brain/mood weights, pool size, approved-only), and the X-ray — the picked
|
||||
piece, the ranked candidate pool with distances, and brain/mood coordinate maps
|
||||
showing where your knob point and the candidates sit.
|
||||
|
||||
- **Content dial** — picks audio/video channel; "off" and audio-only positions go
|
||||
to black walls.
|
||||
- **Scale ring (endless encoder)** — `⊖ out` / `in ⊕` (or scroll the stage) walk a
|
||||
*closed ring* of neutral "scales of nature" clips — cosmos → forest → abyss and
|
||||
back around (diving past the smallest **wraps** to the largest). It is *relative*
|
||||
(an endless encoder), distinct from the absolute 0–4 knobs: each step plays a
|
||||
short placeholder zoom/warp **transition**, then settles on the next scale, with
|
||||
the current knob alteration still applied. The current scale is named beside the
|
||||
buttons (`name (i/N)`). A **fast spin** (scroll several detents at once, ≥3)
|
||||
collapses to a single quick **blended pass** straight to the destination scale
|
||||
instead of grinding through every transition (scales design §3); slow single
|
||||
steps still chain one full transition per scale crossed.
|
||||
- **Four experience knobs (0–4):**
|
||||
- **Dark / Light** — a live runtime color grade (cool/dark ↔ warm/bright; equal
|
||||
or zero = the raw footage).
|
||||
- **Right (dreamlike)** — selects a discrete pre-baked, flow-stabilized restyle
|
||||
variant and crossfades to it (strength 0 = raw base).
|
||||
- **Left (analytical)** — a live overlay: labelled boxes from the clip's authored
|
||||
annotation track, with more annotations appearing at higher levels. Text is
|
||||
shaped live (the simulator analogue of the Pi's Pango/HarfBuzz path).
|
||||
- **Calibration sliders** — adjust the grade/overlay gain curves live; once a look
|
||||
is liked, bake the values into `DEFAULT_CALIBRATION` in `player/alteration.py`.
|
||||
- **RenderPlan readout** — always shows the exact numbers the engine produced (the
|
||||
project's honesty "X-ray," now over the alteration model).
|
||||
|
||||
The base clips, Right variants, Left annotation track, and string tables come from
|
||||
`simulator/sample_media/manifest.json`.
|
||||
By default it loads a generated fixture catalog. To point it at a real catalog,
|
||||
set `HEF_SIM_CATALOG=catalog/library.jsonl` (used automatically when that file is
|
||||
non-empty).
|
||||
|
||||
@@ -1,103 +0,0 @@
|
||||
# Content candidate pool — operator-selected, vetted (2026-06-24)
|
||||
|
||||
The 5 scales each get a **rotating pool** of clips. When the viewer lands on a
|
||||
scale by zooming the ring in/out, the player **randomly picks one clip from that
|
||||
scale's pool**. (This supersedes the one-clip-per-scale model — see
|
||||
[`docs/superpowers/specs/2026-06-24-content-pipeline-design.md`](./superpowers/specs/2026-06-24-content-pipeline-design.md).)
|
||||
|
||||
Every clip below was operator-selected from the candidate gallery, is **strict
|
||||
public-domain or CC‑BY‑class with attribution**, and was vetted **word‑free /
|
||||
human‑free** (windows trimmed past title cards, captions, telemetry burn‑in,
|
||||
divers, construction, logos). All processed via `tools/pipeline/run.py
|
||||
process_clip` (trim → crossfade‑loop → 1080p proxy). **Media is gitignored**;
|
||||
this file is the re‑sourceable record. Sim ids are the `simulator/sample_media/<id>/` dirs.
|
||||
|
||||
License rule confirmed by operator: **CC0 / CC‑BY / CC‑BY‑SA are acceptable with
|
||||
a credit line** (our overlays + dream are derivatives, which these allow);
|
||||
**CC‑BY‑ND is NOT usable** (no derivatives); **CC‑BY‑NC only if the installation
|
||||
stays non‑commercial**.
|
||||
|
||||
## 🌌 cosmos (pool of 4)
|
||||
|
||||
| id | clip | source | window (s) | license / credit |
|
||||
|----|------|--------|-----------|------------------|
|
||||
| `cosmos` | Orion Nebula flythrough | NASA/JPL‑Caltech — https://images.nasa.gov/details/JPL-20221122-SOLSYSf-0001-Orion%20Dust%20and%20Death | ~30–54 | PD (17 U.S.C. §105) |
|
||||
| `cosmos_galaxies` | Flying Through Galaxies | NASA SVS — https://svs.gsfc.nasa.gov/vis/a010000/a014900/a014950/14950_Galaxies_FlyThrough_4k.mp4 | 8–32 | PD |
|
||||
| `cosmos_hudf` | Hubble Ultra Deep Field zoom | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030600/a030687/hudf-b-1920x1080p30.mov | 20–44 | CC‑BY‑class — credit **NASA, ESA, STScI** |
|
||||
| `cosmos_xdf` | eXtreme Deep Field flythrough | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030600/a030681/hxdf_fly-b-1920x1080p30.mov | 2–26 | CC‑BY‑class — credit **NASA, ESA, STScI** |
|
||||
|
||||
> ⚠️ **DISCREPANCY to fix next session.** Operator selected cosmos **#1, #3
|
||||
> (Orion), #4, #5**. During processing, **#6 Galaxy Traverse was run by mistake
|
||||
> instead of #3 Orion**. The Orion clip (#3) is text‑free and already lives at
|
||||
> `simulator/sample_media/cosmos/base.mp4` (the prior real base), so the pool above
|
||||
> is correct. **`cosmos_traverse` is an UNSELECTED extra on disk — drop it** (or
|
||||
> confirm with operator). The current ring scale id is `cosmos`; when the pool is
|
||||
> wired, `cosmos` becomes one pool member.
|
||||
|
||||
## 🛰️ orbit (pool of 3)
|
||||
|
||||
| id | clip | source | window (s) | license |
|
||||
|----|------|--------|-----------|---------|
|
||||
| `orbit_planetearth` | ISS — View of Planet Earth | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030700/a030771/ISS_View_of_Planet_Earth_2160p.mp4 | 10–34 | PD |
|
||||
| `orbit_crewobs` | ISS — Crew Earth Observations | NASA SVS — https://svs.gsfc.nasa.gov/vis/a030000/a030700/a030771/ISS_Crew_Earth_Observations_2160p.mp4 | 9–33 | PD |
|
||||
| `orbit_bluemarble` | NPP "Blue Marble" rotating globe (CG) | NASA SVS — https://svs.gsfc.nasa.gov/vis/a000000/a004500/a004550/BlueMarble_starfield_4k_2160p30_v2.mp4 | 10–34 | PD |
|
||||
|
||||
> Operator also selected aurora→Perth (#4) and west‑coast lights (#5); both
|
||||
> **rejected as UNUSABLE** — #4 has a date/time burned into every frame, #5 has the
|
||||
> ISS module + solar panel in frame throughout. No clean window exists in either.
|
||||
|
||||
## 🏖️ coast (pool of 4) — NEW scale, replaces the granite/waterfall `forest`
|
||||
|
||||
| id | clip | source | window (s) | license |
|
||||
|----|------|--------|-----------|---------|
|
||||
| `coast_birdrock` | Bird rock + ocean (aerial) | NPS Channel Islands — https://www.nps.gov/nps-audiovideo/audiovideo/663a2f13-6183-4b3e-a4a9-6698a84619d01080p.mp4 | 3–27 | PD |
|
||||
| `coast_surfgrass` | Surfgrass / coralline tidepool | NPS Cabrillo — https://www.nps.gov/nps-audiovideo/audiovideo/de7d1cf2-e422-45b5-aa66-4c69e29a4d811080p.mp4 | 2–26 | PD |
|
||||
| `coast_elkbeach` | Elk resting in coastal grass/fog | NPS Redwood — https://www.nps.gov/nps-audiovideo/legacy/redw/4634070E-F68D-4F74-E43174448168AC8A/redw-elkbeach_1280x720.mp4 | 20–38 | PD (720p source) |
|
||||
| `coast_drakesbeach` | Elephant seals on dark sand | NPS Point Reyes — https://www.nps.gov/nps-audiovideo/audiovideo/1cfd8165-1061-4d75-af53-b46c2e3ed2701080p.mp4 | 20–35 | PD |
|
||||
|
||||
> Operator also selected the Lifeboat Station "elephant seal timelapse" (more‑options
|
||||
> #3) — **FAILED**: the NPS page's video resolved to the wrong footage (an aerial of
|
||||
> the bay with road/parking/boats, no seals). An alternate source UUID
|
||||
> (`50a7a20a-9e52-4643-b186-c61ce1cef0f7`) appeared in the page HTML and could be
|
||||
> retried next session if the seal timelapse is still wanted.
|
||||
>
|
||||
> **Scale rename:** the live ring scale is still `forest` (Yosemite). Next session
|
||||
> renames it `coast` and points it at this pool. The old real Yosemite base
|
||||
> (`simulator/sample_media/forest/`) becomes unused.
|
||||
|
||||
## 🐠 reef (pool of 5)
|
||||
|
||||
All NOAA Fisheries b‑roll (PD, credit "NOAA Fisheries"); Brightcove download
|
||||
endpoint `https://download.gallery.brightcove.com/api/gallery/account/659677166001/site/site-600408/video/<VIDEO_ID>/download`.
|
||||
|
||||
| id | clip | VIDEO_ID | window (s) | notes |
|
||||
|----|------|----------|-----------|-------|
|
||||
| `reef_spawning` | Snapper school + yellow fish over sunlit reef | 1553921798001 | 50–78 | clean window from a 12‑min reel |
|
||||
| `reef_hawkfish` | Humphead parrotfish over reef (Hawaiian atoll) | 6039896460001 | 250–274 | |
|
||||
| `reef_lionfish` | Lionfish hovering + blue snapper | 4088881464001 | 12–36 | underwater clean; humans on land after ~190s |
|
||||
| `reef_snapper` | Red snapper schooling (Gulf, greenish water) | 5305427942001 | 7–31 | |
|
||||
| `reef_coralspacific` | Pacific coral macro, spotted fish | 5231464663001 | 30–54 | diver‑free window (divers at 6–25s) |
|
||||
|
||||
## 🦑 abyss (pool of 3)
|
||||
|
||||
All NOAA Ocean Exploration (PD, 17 U.S.C. §105). These have creatures drifting
|
||||
**into and out of frame** — the prime material for time‑windowed tracked labels.
|
||||
|
||||
| id | clip | source | window (s) | creatures |
|
||||
|----|------|--------|-----------|-----------|
|
||||
| `abyss_wow` | "World of Water" | https://oceanexplorer.noaa.gov/wp-content/uploads/2018/05/wow-1280x720-1.mp4 | 10–34 | jelly + ctenophore + siphonophore enter/exit |
|
||||
| `abyss_midwaterexp` | "Midwater Exploration" | https://oceanexplorer.noaa.gov/wp-content/uploads/2019/09/midwater-exploration-1920x1080-1.mp4 | 16–40 | white medusa + red worm enter/exit |
|
||||
| `abyss_hiding` | "Hiding in the Dark" | https://oceanexplorer.noaa.gov/wp-content/uploads/2018/05/dark-1280x720-1.mp4 | 20–44 | crimson jelly + ctenophore enter/exit |
|
||||
|
||||
> Operator also selected Crossota (#4) and comb jelly (#5); both **rejected** —
|
||||
> Crossota has only ~17s clean (too short for a 24s window), comb jelly has a
|
||||
> species caption covering the whole clip.
|
||||
|
||||
## On disk but NOT in any pool (clean up next session)
|
||||
|
||||
- `cosmos_traverse` — unselected (the #6/#3 mix‑up above).
|
||||
- `orbit_westcoast` — rejected (ISS hardware in frame); leftover output dir.
|
||||
- `forest` — the old Yosemite base, superseded by the `coast` pool.
|
||||
- `reef` — the original placeholder/early reef base, superseded by the `reef_*` pool.
|
||||
- The `review.html` gallery + all `cosmos/orbit/coast/reef/abyss` candidate dirs are
|
||||
local‑only (gitignored).
|
||||
@@ -1,29 +0,0 @@
|
||||
# Content sourcing — strict-PD neutral bases
|
||||
|
||||
For each scale, acquire a calm, neutral source clip from a **strict public-domain**
|
||||
US-government source, verify the license, then run it through the pipeline.
|
||||
|
||||
| Scale | Source | Where |
|
||||
|--------|--------------------------------|------------------------------------|
|
||||
| cosmos | NASA / Hubble / JWST | https://images.nasa.gov/ |
|
||||
| orbit | NASA / ISS | https://images.nasa.gov/ |
|
||||
| forest | NPS / USGS | https://www.nps.gov/ , https://www.usgs.gov/ |
|
||||
| reef | NOAA Ocean Exploration | https://oceanexplorer.noaa.gov/ |
|
||||
| abyss | NOAA Ocean Exploration | https://oceanexplorer.noaa.gov/ |
|
||||
|
||||
**License check (mandatory):** confirm the asset is US-gov PD (17 U.S.C. §105) or
|
||||
explicitly CC0. "Free stock" (Pexels/Pixabay/etc.) is royalty-free but NOT public
|
||||
domain — do not use. Record a `Provenance` (tools/pipeline/provenance.py) per clip.
|
||||
|
||||
**Run the pipeline** (replaces the placeholder bytes for a scale):
|
||||
|
||||
```bash
|
||||
python -c "from tools.pipeline.run import process_clip; \
|
||||
process_clip('downloads/<scale>.mp4', 'simulator/sample_media/<scale>', \
|
||||
start=<in_s>, duration=<len_s>, overlap=<0.5..1.0>)"
|
||||
```
|
||||
|
||||
This writes `simulator/sample_media/<scale>/base.mp4` (proxy) + `master.mp4`. Then
|
||||
author the labels (Increment 2: the hybrid track pass + simulator author mode) or
|
||||
hand-edit the annotation track for now (spec §4). The ring + manifest entry already
|
||||
exist; `process_clip` only replaces the media bytes.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,97 +0,0 @@
|
||||
# Scale-Ring Navigation (Simulator) — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans or
|
||||
> superpowers:subagent-driven-development to implement task-by-task. Steps use
|
||||
> checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add scale-ring navigation to the simulator — an endless rotary-encoder
|
||||
control (relative, vs the absolute 0–4 experience knobs) that walks a *closed
|
||||
ring* of neutral "scales of nature" clips, with placeholder AI zoom/warp
|
||||
transitions between adjacent scales. Add 1–2 cheap, true-PD neutral base clips
|
||||
(NASA/Hubble cosmos + NOAA Ocean Exploration deep sea) so the ring is
|
||||
demonstrable. **Defer** the expensive real multi-strength flow-stabilized Right
|
||||
re-bake until the ring is liked; keep deferring the Pi renderer + serial/firmware.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-design.md` §3 (scale navigation & zoom transitions), §2 (scales library), §2.1 (strict-PD sourcing map).
|
||||
|
||||
**Architecture:** Python stays the single source of truth. The ring *navigation
|
||||
math* — step, wrap, which transition clip to play and in which direction — lives
|
||||
in a new canonical pure module `player/ring.py` (mirrors `player/content.py` /
|
||||
`player/alteration.py`: frozen dataclasses, pure functions). The simulator's data
|
||||
layer (`simulator/clips.py`) parses a `ring` section of the manifest into the
|
||||
ring types; the API exposes the ring and a stateless `advance` move; the browser
|
||||
holds the current ring index and only *renders* (plays the returned transition
|
||||
file, then the target scale's clip, with the existing per-knob alteration on top).
|
||||
|
||||
**Ordering convention (canonical):** ring index 0 = the *largest* scale (cosmos);
|
||||
increasing index zooms **inward** toward the smallest. `+1` detent = zoom in,
|
||||
`-1` = zoom out — both wrap (past the smallest wraps to the largest, the
|
||||
infinite-zoom payoff, §3). Edge `i` connects `scales[i] → scales[(i+1) % N]`; the
|
||||
last edge is the small→large wrap seam. One transition clip per edge plays
|
||||
**forward** when zooming inward and **reversed** when zooming outward, so N scales
|
||||
need only N transition clips (§3: "one clip per ring edge").
|
||||
|
||||
**Deferral, made explicit:** the new cosmos/abyss scale clips carry a raw base
|
||||
only (no pre-baked Right variants); the `Clip` model already falls back any
|
||||
unauthored strength to the raw base, so Right on a new scale is a no-op until the
|
||||
multi-strength flow-stabilized re-bake happens (deliberately deferred). Forest
|
||||
keeps its existing POC variants.
|
||||
|
||||
**Tech Stack:** Python 3.13, FastAPI + pydantic, pytest, frozen dataclasses;
|
||||
vanilla JS + CSS/SVG for the browser; ffmpeg (system or `imageio-ffmpeg`) for
|
||||
placeholder media.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
**Engine (create):**
|
||||
- `player/ring.py` — `Scale`, `Transition`, `ScaleRing`, `RingMove`,
|
||||
`TransitionStep`, `scale_at`, `advance_ring`. Pure logic, no I/O/JSON.
|
||||
|
||||
**Simulator (modify):**
|
||||
- `simulator/clips.py` — add `load_ring(manifest)` building a `ScaleRing` from the
|
||||
manifest `ring` section + `ring_to_dict` / `ring_move_to_dict` serializers.
|
||||
- `simulator/app.py` — `GET /api/ring`, `POST /api/ring/advance`.
|
||||
- `simulator/static/{index.html,app.js,style.css}` — endless-encoder control
|
||||
(zoom-in/zoom-out buttons + mouse-wheel on the stage), current-scale readout,
|
||||
multi-clip active-clip selection, transition playback.
|
||||
- `simulator/sample_media/manifest.json` (committed) — add cosmos + abyss clips
|
||||
and the `ring` section; media binaries stay gitignored.
|
||||
- `simulator/setup_sample_media.py` (or sibling) — generate cheap cosmos/abyss
|
||||
base placeholders + per-edge transition placeholders (ffmpeg); record true-PD
|
||||
provenance in the manifest. Real PD fetch is a best-effort `--fetch` bonus.
|
||||
|
||||
**Tests (create/extend):**
|
||||
- `tests/test_player_ring.py` (create) — ring navigation logic (step, chain, wrap,
|
||||
direction/reverse, degenerate N≤1).
|
||||
- `tests/test_clips.py` (extend) — `load_ring`, serializers.
|
||||
- `tests/test_simulator_api.py` (extend) — `/api/ring`, `/api/ring/advance`.
|
||||
|
||||
**Docs:**
|
||||
- `docs/ROADMAP.md` — scale-ring status.
|
||||
- `docs/USER_GUIDE.md` — simulator "scale ring" gesture (if present).
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Engine — `player/ring.py` (pure ring navigation)
|
||||
- [ ] TDD `Scale`/`Transition`/`ScaleRing` construction + validation (N edges for N scales; ≥1 scale; degenerate N≤1 ring).
|
||||
- [ ] TDD `scale_at` (mod indexing) and `advance_ring(ring, from_index, delta)` → `RingMove` (to_index by mod; per-unit `TransitionStep` list; `reversed` on outward; `wrapped` flag on crossing the seam; chained steps for |delta|>1).
|
||||
|
||||
## Task 2: Simulator data layer — `load_ring` + serializers
|
||||
- [ ] TDD `load_ring(manifest)` parsing the `ring` section into a `ScaleRing`.
|
||||
- [ ] TDD `ring_to_dict` / `ring_move_to_dict`.
|
||||
|
||||
## Task 3: API — `/api/ring`, `/api/ring/advance`
|
||||
- [ ] TDD `GET /api/ring` (scales + transitions, with each scale's title) and `POST /api/ring/advance` (validates from_index/delta; returns the move + target clip_id).
|
||||
|
||||
## Task 4: Media — cosmos + abyss bases + transitions; manifest
|
||||
- [ ] Add cosmos + abyss clips and the `ring` section to the committed manifest with true-PD provenance.
|
||||
- [ ] Extend the media-setup script to generate cheap base + transition placeholders (and optional real PD fetch).
|
||||
|
||||
## Task 5: Frontend — endless encoder + multi-clip + transition playback
|
||||
- [ ] Active-clip selection by ring index; zoom-in/out buttons + wheel; current-scale readout; play returned transition file(s), then settle on target clip; keep live alteration on top.
|
||||
|
||||
## Task 6: Verify + docs
|
||||
- [ ] Full `pytest -q` green; boot the sim and exercise the ring by eye/screenshot.
|
||||
- [ ] Update ROADMAP / USER_GUIDE.
|
||||
@@ -1,239 +0,0 @@
|
||||
# v2v Vertical Slice — Real Forest Right Variants Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Produce the first *real* (not placeholder) multi-strength Right-axis
|
||||
restyle media for the simulator — flow-stabilized painterly variants of the
|
||||
forest scale at Right strengths 1–4 — so the dreamlike-axis *look* can finally be
|
||||
judged by eye on real footage, breaking the "can't judge the ring liked without
|
||||
real content" chicken-and-egg (operator chose a vertical slice, session 0012).
|
||||
|
||||
**Architecture:** Productionize the proven session-0008 POC restyle algorithm
|
||||
(`~/hef-poc/flow_restyle.py`) into a repo-resident offline baker,
|
||||
`simulator/bake_right_variants.py`, a sibling of the existing
|
||||
`setup_sample_media.py` / `setup_scales_media.py` media-generation scripts. The
|
||||
algorithm is unchanged (extract frames → SD img2img on keyframes → Farneback
|
||||
optical-flow warp + light refine on tweens → reassemble), which sessions 0008
|
||||
proved is temporally calm (no boiling). The one new idea is a **strength curve**:
|
||||
Right level 1–4 maps to an increasing keyframe img2img `strength`, so the four
|
||||
discrete pre-baked variants form a subtle→strong dreamlike ramp. Only the curve
|
||||
helper is unit-tested; the rendered media is verified by eye (screenshots), as
|
||||
the other media-setup scripts are.
|
||||
|
||||
**Tech Stack:** Python, diffusers `AutoPipelineForImage2Image` + `stabilityai/sd-turbo`
|
||||
(cached, 4.8 G) on Apple MPS (torch 2.12, available in the project `.venv`),
|
||||
OpenCV Farneback optical flow, imageio-ffmpeg. ~2.7 min/clip × 4 ≈ ~11 min local
|
||||
compute.
|
||||
|
||||
**Scope note — what this slice is NOT:** the *real i2v ring transition* half of
|
||||
the v2v build is deferred: it needs a second real scale base (cosmos/abyss are
|
||||
still synthetic placeholders) plus a generative-video model not in the POC, and
|
||||
real strict-PD footage sourcing (NASA/NOAA, sub-project 2). This slice does the
|
||||
Right-variant half, which is fully achievable now (forest has a real base).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: The Right strength curve (pure, testable)
|
||||
|
||||
**Files:**
|
||||
- Create: `simulator/bake_right_variants.py`
|
||||
- Test: `tests/test_bake_right_variants.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# tests/test_bake_right_variants.py
|
||||
import pytest
|
||||
|
||||
from simulator.bake_right_variants import right_strength_params, RIGHT_LEVELS
|
||||
|
||||
|
||||
def test_levels_are_one_through_four():
|
||||
assert RIGHT_LEVELS == (1, 2, 3, 4)
|
||||
|
||||
|
||||
def test_strength_increases_with_level():
|
||||
keys = [right_strength_params(l)[0] for l in RIGHT_LEVELS]
|
||||
assert keys == sorted(keys) # monotonic ramp
|
||||
assert keys[0] < keys[-1] # subtle -> strong
|
||||
|
||||
|
||||
def test_tween_strength_is_a_fraction_of_key_strength():
|
||||
for l in RIGHT_LEVELS:
|
||||
key, tween = right_strength_params(l)
|
||||
assert 0.0 < tween < key # tween is a light refine only
|
||||
|
||||
|
||||
def test_rejects_out_of_range_level():
|
||||
with pytest.raises(ValueError):
|
||||
right_strength_params(0)
|
||||
with pytest.raises(ValueError):
|
||||
right_strength_params(5)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/test_bake_right_variants.py -q`
|
||||
Expected: FAIL (ModuleNotFoundError / ImportError — module not created yet)
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation (curve only, no torch import at module load)**
|
||||
|
||||
```python
|
||||
# simulator/bake_right_variants.py (top of file)
|
||||
"""Offline baker: real flow-stabilized painterly Right-axis variants (scales
|
||||
design §1/§4). Productionizes the session-0008 POC (~/hef-poc/flow_restyle.py):
|
||||
SD img2img keyframes + Farneback optical-flow tweens = temporally calm restyle.
|
||||
|
||||
Right level -> keyframe img2img strength ramp (subtle -> strong); tween strength
|
||||
is a light refine fraction of the keyframe strength. Curve is by-eye tunable.
|
||||
|
||||
Usage: .venv/bin/python -m simulator.bake_right_variants [--scale forest]
|
||||
Requires: torch+MPS, diffusers, sd-turbo (cached), opencv, imageio-ffmpeg.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
RIGHT_LEVELS = (1, 2, 3, 4)
|
||||
|
||||
# Keyframe img2img strength per Right level (by-eye ramp). Level 4 ~ the POC's
|
||||
# proven 0.5-0.58 painterly look; level 1 is a barely-there dream haze.
|
||||
_KEY_STRENGTH = {1: 0.28, 2: 0.38, 3: 0.48, 4: 0.58}
|
||||
_TWEEN_RATIO = 0.36 # POC ratio (0.18 / 0.5): tweens are a light refine only
|
||||
|
||||
|
||||
def right_strength_params(level: int) -> tuple[float, float]:
|
||||
"""(keyframe_strength, tween_strength) for a Right level in RIGHT_LEVELS."""
|
||||
if level not in _KEY_STRENGTH:
|
||||
raise ValueError(f"Right level must be one of {RIGHT_LEVELS}, got {level}")
|
||||
key = _KEY_STRENGTH[level]
|
||||
return key, round(key * _TWEEN_RATIO, 4)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/test_bake_right_variants.py -q`
|
||||
Expected: PASS (4 passed)
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add simulator/bake_right_variants.py tests/test_bake_right_variants.py
|
||||
git commit -m "feat(pipeline): Right strength curve for the v2v variant baker"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: The baker (flow restyle, productionized)
|
||||
|
||||
**Files:**
|
||||
- Modify: `simulator/bake_right_variants.py`
|
||||
|
||||
No unit test for the render itself (non-trivial ML output; verified by eye in
|
||||
Task 4). The pure curve is already covered by Task 1.
|
||||
|
||||
- [ ] **Step 1: Add the restyle engine + per-level bake, porting the POC algorithm verbatim**
|
||||
|
||||
Port `extract`, `warp`, and the keyframe/tween loop from
|
||||
`~/hef-poc/flow_restyle.py` unchanged (it is the approved calm algorithm). Wrap
|
||||
it as `restyle_clip(src, out, key_strength, tween_strength, *, fps=12.0, keyint=24)`
|
||||
and add a `bake_scale(scale="forest")` that loops `RIGHT_LEVELS`, calling
|
||||
`restyle_clip` with `right_strength_params(level)` and writing
|
||||
`simulator/sample_media/<scale>/right<level>.mp4`. Load the SD pipeline ONCE and
|
||||
reuse it across levels. `PROMPT`/`NEG`/Farneback params/seed = the POC's.
|
||||
|
||||
Source base = `simulator/sample_media/<scale>/base.mp4` (forest = the real
|
||||
Yosemite POC clip). `main()` parses `--scale` (default `forest`), `--fps`,
|
||||
`--keyint` and calls `bake_scale`.
|
||||
|
||||
- [ ] **Step 2: Smoke-check the module imports and the curve still passes**
|
||||
|
||||
Run: `.venv/bin/python -m pytest tests/test_bake_right_variants.py -q`
|
||||
Expected: PASS (torch imported lazily inside the bake funcs, not at module load,
|
||||
so the test stays fast and import-safe)
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add simulator/bake_right_variants.py
|
||||
git commit -m "feat(pipeline): forest Right-variant baker (POC flow restyle, productionized)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Run the bake (real media, ~11 min)
|
||||
|
||||
**Files:**
|
||||
- Writes (gitignored): `simulator/sample_media/forest/right{1,2,3,4}.mp4`
|
||||
|
||||
- [ ] **Step 1: Run the baker**
|
||||
|
||||
Run: `.venv/bin/python -m simulator.bake_right_variants --scale forest`
|
||||
Expected: 4 clips written; per-level timing printed; ~11 min total. Replaces the
|
||||
3 placeholder strengths (1–3) and the single real strength (4) with a consistent
|
||||
real ramp.
|
||||
|
||||
- [ ] **Step 2: Confirm the outputs exist and are real video**
|
||||
|
||||
Run: `ls -la simulator/sample_media/forest/right*.mp4`
|
||||
Expected: right1–4.mp4 present, each a multi-MB H.264 clip.
|
||||
|
||||
(No commit — media is gitignored; it is reproduced by the baker.)
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Wire the manifest + verify by eye
|
||||
|
||||
**Files:**
|
||||
- Modify: `simulator/sample_media/manifest.json` (forest `right_variants` models)
|
||||
- Modify: `docs/USER_GUIDE.md` (note real forest Right variants + the baker)
|
||||
- Modify: `docs/ROADMAP.md` (v2v slice progress)
|
||||
|
||||
- [ ] **Step 1: Update the forest variant models in the manifest**
|
||||
|
||||
Set every forest `right_variants` entry (1–4) `model` to
|
||||
`"sd-turbo+farneback-flow"` (drop the `"placeholder"` labels) — they are all real
|
||||
bakes now.
|
||||
|
||||
- [ ] **Step 2: Boot the sim and drive the Right axis across 0→4 on forest**
|
||||
|
||||
Run: `.venv/bin/python -m uvicorn simulator.app:app --port 8013` (background),
|
||||
navigate to the forest scale, and capture headless screenshots at Right 0/1/2/3/4.
|
||||
Expected: a coherent subtle→strong painterly ramp on the real Yosemite footage,
|
||||
temporally calm (no boiling), grade/overlay still compose on top.
|
||||
|
||||
- [ ] **Step 3: Update docs**
|
||||
|
||||
USER_GUIDE: note forest now carries real flow-stabilized Right variants generated
|
||||
by `simulator/bake_right_variants.py`. ROADMAP: mark the v2v Right-variant vertical
|
||||
slice done; note the i2v ring-transition half still deferred (needs a 2nd real
|
||||
base + video model).
|
||||
|
||||
- [ ] **Step 4: Run the full suite**
|
||||
|
||||
Run: `.venv/bin/python -m pytest -q`
|
||||
Expected: PASS (existing + Task-1 curve tests).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add simulator/sample_media/manifest.json docs/USER_GUIDE.md docs/ROADMAP.md
|
||||
git commit -m "feat(sim): forest carries real multi-strength Right variants; docs"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:** scales design §1 (local restyle pipeline) + §4 (economics:
|
||||
local bake) → Tasks 2–3 productionize + run it. The operator's vertical-slice
|
||||
choice (real Right variants for one scale to judge the look) → Tasks 3–4. The
|
||||
deferred i2v transition half is explicitly scoped out (noted in header + Task 4).
|
||||
The strength curve (the one new design element) → Task 1.
|
||||
|
||||
**Placeholder scan:** none — the baker algorithm is a verbatim port of the
|
||||
existing `~/hef-poc/flow_restyle.py` (cited), the curve is concrete constants,
|
||||
commands are exact.
|
||||
|
||||
**Type consistency:** `right_strength_params(level) -> (key, tween)` defined in
|
||||
Task 1, consumed by `bake_scale` in Task 2. `RIGHT_LEVELS` shared. Output paths
|
||||
`simulator/sample_media/forest/right<level>.mp4` consistent with the manifest's
|
||||
existing forest `right_variants` map (Task 4).
|
||||
@@ -1,929 +0,0 @@
|
||||
# Content Pipeline — Increment 1 (real footage, no ML) Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build the deterministic content-pipeline tooling (frame → crossfade-loop → master+proxy transcode → manifest assembly), add the backward-compatible keyframed annotation-track schema with client interpolation, and extend the scale ring to 5 scales — proving the pipeline end-to-end on real footage without any ML.
|
||||
|
||||
**Architecture:** A new pure-Python `tools/pipeline/` package whose stage functions return ffmpeg argument lists (unit-tested without running ffmpeg); a thin `run.py` executes them and is covered by an opt-in integration test gated on `ffmpeg` + the real forest POC base. Manifest assembly is pure dict transforms. The annotation `track` is optional data that rides through the existing pass-through `annotations` list (no Python model change); interpolation is client-side JS, consistent with the existing "alteration math in Python, label placement in the client" split. This implements §3 (stages 1–4, 6), §4 (track schema), §6 (format), §7 (tooling), and Increment 1 of §8 in [`docs/superpowers/specs/2026-06-24-content-pipeline-design.md`](../specs/2026-06-24-content-pipeline-design.md).
|
||||
|
||||
**Tech Stack:** Python 3.13, pytest, ffmpeg (libx264), vanilla JS (SVG overlay). Tests run with `pytest -q` from the repo root in the project `.venv`.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Scaffold `tools/pipeline/` + the frame stage
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/pipeline/__init__.py`
|
||||
- Create: `tools/pipeline/ffmpeg_ops.py`
|
||||
- Test: `tests/test_pipeline_ffmpeg_ops.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# tests/test_pipeline_ffmpeg_ops.py
|
||||
from tools.pipeline.ffmpeg_ops import frame_args
|
||||
|
||||
|
||||
def test_frame_args_trims_scales_pads_and_strips_audio():
|
||||
args = frame_args(
|
||||
"src.mp4", "out.mp4",
|
||||
start=2.0, duration=8.0, width=1920, height=1080, ff="ffmpeg",
|
||||
)
|
||||
assert args[0] == "ffmpeg"
|
||||
assert "-ss" in args and args[args.index("-ss") + 1] == "2.0"
|
||||
assert "-t" in args and args[args.index("-t") + 1] == "8.0"
|
||||
assert "-an" in args # audio stripped (bases are video-only)
|
||||
vf = args[args.index("-vf") + 1]
|
||||
assert "scale=1920:1080:force_original_aspect_ratio=decrease" in vf
|
||||
assert "pad=1920:1080" in vf
|
||||
assert "format=yuv420p" in vf
|
||||
assert args[-1] == "out.mp4"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest tests/test_pipeline_ffmpeg_ops.py -q`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'tools.pipeline'`
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# tools/pipeline/__init__.py
|
||||
"""Content pipeline: source -> frame -> loop -> transcode -> manifest.
|
||||
|
||||
Pure ffmpeg-argument builders (ffmpeg_ops), a thin runner (run), provenance
|
||||
records, and manifest assembly. See docs/superpowers/specs/2026-06-24-content-pipeline-design.md.
|
||||
"""
|
||||
```
|
||||
|
||||
```python
|
||||
# tools/pipeline/ffmpeg_ops.py
|
||||
"""Pure ffmpeg argument builders (no I/O) for the content pipeline.
|
||||
|
||||
Each function returns a list[str] ready for ffmpeg, so command construction is
|
||||
unit-testable without running ffmpeg. tools/pipeline/run.py executes them. The
|
||||
ffmpeg binary name is injected (default "ffmpeg").
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
def _fit(width: int, height: int) -> str:
|
||||
"""A scale+pad filter chain fitting any input into width×height (letterbox),
|
||||
fixing SAR and pixel format so downstream concat/xfade get identical geometry."""
|
||||
return (
|
||||
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
|
||||
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
|
||||
"setsar=1,format=yuv420p"
|
||||
)
|
||||
|
||||
|
||||
def frame_args(src, dst, *, start: float, duration: float,
|
||||
width: int, height: int, ff: str = "ffmpeg") -> list[str]:
|
||||
"""Stage 2 — trim [start, start+duration], fit to width×height, strip audio.
|
||||
Produces a high-quality mezzanine for the loop stage."""
|
||||
return [
|
||||
ff, "-y", "-ss", str(start), "-t", str(duration),
|
||||
"-i", str(src), "-vf", _fit(width, height), "-an",
|
||||
"-c:v", "libx264", "-preset", "medium", "-crf", "16",
|
||||
"-pix_fmt", "yuv420p", "-movflags", "+faststart", str(dst),
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `pytest tests/test_pipeline_ffmpeg_ops.py -q`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/pipeline/__init__.py tools/pipeline/ffmpeg_ops.py tests/test_pipeline_ffmpeg_ops.py
|
||||
git commit -m "feat(pipeline): frame stage ffmpeg arg builder (content pipeline §3.2)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Crossfade seamless-loop stage
|
||||
|
||||
**Files:**
|
||||
- Modify: `tools/pipeline/ffmpeg_ops.py`
|
||||
- Test: `tests/test_pipeline_ffmpeg_ops.py`
|
||||
|
||||
The seamless loop (spec §3.3) crossfades the clip's tail over its head. For a clip
|
||||
of duration `D` with overlap `O`, the output (length `D − O`) is
|
||||
`concat( xfade(tail, head, O), mid )` where `head`=`[0,O]`, `mid`=`[O, D−O]`,
|
||||
`tail`=`[D−O, D]`. Both internal joins and the loop seam are then continuous, and
|
||||
the tail→head jump is blended away over `O`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# add to tests/test_pipeline_ffmpeg_ops.py
|
||||
from tools.pipeline.ffmpeg_ops import crossfade_loop_args
|
||||
|
||||
|
||||
def test_crossfade_loop_builds_head_mid_tail_xfade_concat():
|
||||
args = crossfade_loop_args("in.mp4", "loop.mp4", duration=8.0, overlap=1.0)
|
||||
fc = args[args.index("-filter_complex") + 1]
|
||||
assert "trim=0:1.0" in fc # head = first O secs
|
||||
assert "trim=1.0:7.0" in fc # mid = [O, D-O]
|
||||
assert "trim=7.0:8.0" in fc # tail = last O secs
|
||||
assert "xfade=transition=fade:duration=1.0:offset=0" in fc
|
||||
assert "concat=n=2:v=1" in fc
|
||||
assert "-an" in args
|
||||
assert args[-1] == "loop.mp4"
|
||||
|
||||
|
||||
def test_crossfade_loop_rejects_overlap_too_large():
|
||||
import pytest
|
||||
with pytest.raises(ValueError):
|
||||
crossfade_loop_args("in.mp4", "loop.mp4", duration=4.0, overlap=2.0)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest tests/test_pipeline_ffmpeg_ops.py -q`
|
||||
Expected: FAIL with `ImportError: cannot import name 'crossfade_loop_args'`
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# add to tools/pipeline/ffmpeg_ops.py
|
||||
def crossfade_loop_args(src, dst, *, duration: float, overlap: float,
|
||||
ff: str = "ffmpeg") -> list[str]:
|
||||
"""Stage 3 — make `src` loop seamlessly by crossfading its tail over its head.
|
||||
Output length = duration − overlap. Requires overlap < duration/2 so a non-empty
|
||||
middle remains."""
|
||||
if overlap <= 0 or overlap >= duration / 2:
|
||||
raise ValueError(f"overlap {overlap} must be in (0, duration/2={duration/2})")
|
||||
d, o = duration, overlap
|
||||
fc = (
|
||||
f"[0:v]trim=0:{o},setpts=PTS-STARTPTS[head];"
|
||||
f"[0:v]trim={o}:{d - o},setpts=PTS-STARTPTS[mid];"
|
||||
f"[0:v]trim={d - o}:{d},setpts=PTS-STARTPTS[tail];"
|
||||
f"[tail][head]xfade=transition=fade:duration={o}:offset=0[xf];"
|
||||
f"[xf][mid]concat=n=2:v=1,format=yuv420p[out]"
|
||||
)
|
||||
return [
|
||||
ff, "-y", "-i", str(src), "-filter_complex", fc,
|
||||
"-map", "[out]", "-an",
|
||||
"-c:v", "libx264", "-preset", "medium", "-crf", "16",
|
||||
"-pix_fmt", "yuv420p", "-movflags", "+faststart", str(dst),
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `pytest tests/test_pipeline_ffmpeg_ops.py -q`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/pipeline/ffmpeg_ops.py tests/test_pipeline_ffmpeg_ops.py
|
||||
git commit -m "feat(pipeline): crossfade seamless-loop arg builder (content pipeline §3.3)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Master + proxy transcode stage
|
||||
|
||||
**Files:**
|
||||
- Modify: `tools/pipeline/ffmpeg_ops.py`
|
||||
- Test: `tests/test_pipeline_ffmpeg_ops.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# add to tests/test_pipeline_ffmpeg_ops.py
|
||||
from tools.pipeline.ffmpeg_ops import transcode_args
|
||||
|
||||
|
||||
def test_transcode_args_high_profile_even_dims_faststart():
|
||||
args = transcode_args("loop.mp4", "proxy.mp4", width=1920, height=1080,
|
||||
crf=20, fps=30)
|
||||
assert "-profile:v" in args and args[args.index("-profile:v") + 1] == "high"
|
||||
assert "-crf" in args and args[args.index("-crf") + 1] == "20"
|
||||
vf = args[args.index("-vf") + 1]
|
||||
assert "scale=1920:1080:force_original_aspect_ratio=decrease" in vf
|
||||
assert "fps=30" in vf
|
||||
assert "+faststart" in args
|
||||
assert "-an" in args
|
||||
assert args[-1] == "proxy.mp4"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest tests/test_pipeline_ffmpeg_ops.py -q`
|
||||
Expected: FAIL with `ImportError: cannot import name 'transcode_args'`
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# add to tools/pipeline/ffmpeg_ops.py
|
||||
def transcode_args(src, dst, *, width: int, height: int, crf: int,
|
||||
fps: int = 30, ff: str = "ffmpeg") -> list[str]:
|
||||
"""Stage 4 — encode a final deliverable (master or proxy). Master: source res
|
||||
capped ~3840 wide, crf 18. Proxy: 1920×1080, crf 20. Both H.264 High, yuv420p,
|
||||
even dims, +faststart, video-only (spec §6)."""
|
||||
vf = (
|
||||
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
|
||||
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
|
||||
f"setsar=1,fps={fps},format=yuv420p"
|
||||
)
|
||||
return [
|
||||
ff, "-y", "-i", str(src), "-vf", vf, "-an",
|
||||
"-c:v", "libx264", "-profile:v", "high", "-preset", "slow",
|
||||
"-crf", str(crf), "-pix_fmt", "yuv420p",
|
||||
"-movflags", "+faststart", str(dst),
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `pytest tests/test_pipeline_ffmpeg_ops.py -q`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/pipeline/ffmpeg_ops.py tests/test_pipeline_ffmpeg_ops.py
|
||||
git commit -m "feat(pipeline): master/proxy transcode arg builder (content pipeline §3.4/§6)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Provenance record (stage 1 output)
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/pipeline/provenance.py`
|
||||
- Test: `tests/test_pipeline_provenance.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# tests/test_pipeline_provenance.py
|
||||
from tools.pipeline.provenance import Provenance
|
||||
|
||||
|
||||
def test_provenance_to_dict_roundtrip():
|
||||
p = Provenance(
|
||||
scale="abyss",
|
||||
license="public-domain (US Gov, 17 U.S.C. §105)",
|
||||
source="NOAA Ocean Exploration",
|
||||
url="https://oceanexplorer.noaa.gov/",
|
||||
fetched_at="2026-06-24",
|
||||
)
|
||||
d = p.to_dict()
|
||||
assert d == {
|
||||
"scale": "abyss",
|
||||
"license": "public-domain (US Gov, 17 U.S.C. §105)",
|
||||
"source": "NOAA Ocean Exploration",
|
||||
"url": "https://oceanexplorer.noaa.gov/",
|
||||
"fetched_at": "2026-06-24",
|
||||
}
|
||||
assert Provenance.from_dict(d) == p
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest tests/test_pipeline_provenance.py -q`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'tools.pipeline.provenance'`
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# tools/pipeline/provenance.py
|
||||
"""Stage 1 — the per-clip strict-PD provenance record (spec §3.1).
|
||||
|
||||
Captured at source time so license + source + URL travel with the media. The
|
||||
'no explicit license -> assume PD, verify' trap (scales design §2.1) applies:
|
||||
"free stock" (Pexels/Pixabay) is NOT public domain.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict, dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Provenance:
|
||||
scale: str
|
||||
license: str
|
||||
source: str
|
||||
url: str
|
||||
fetched_at: str # ISO date; passed in (no clock here -> deterministic)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, d: dict) -> "Provenance":
|
||||
return cls(
|
||||
scale=d["scale"], license=d["license"], source=d["source"],
|
||||
url=d["url"], fetched_at=d["fetched_at"],
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `pytest tests/test_pipeline_provenance.py -q`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/pipeline/provenance.py tests/test_pipeline_provenance.py
|
||||
git commit -m "feat(pipeline): strict-PD provenance record (content pipeline §3.1)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Manifest assembly (stage 6)
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/pipeline/manifest.py`
|
||||
- Test: `tests/test_pipeline_manifest.py`
|
||||
|
||||
Pure dict transforms over a loaded manifest: upsert a clip entry, place a scale in
|
||||
the ring order, and rebuild the per-edge transition list so it always has exactly
|
||||
N entries for N scales (one transition per ring edge, including the wrap).
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# tests/test_pipeline_manifest.py
|
||||
from tools.pipeline.manifest import upsert_clip, add_ring_scale, rebuild_ring_edges
|
||||
|
||||
|
||||
def _base():
|
||||
return {"clips": [{"id": "cosmos", "title": "Cosmos"}],
|
||||
"ring": {"scales": [{"id": "cosmos", "clip_id": "cosmos"}],
|
||||
"transitions": []}}
|
||||
|
||||
|
||||
def test_upsert_clip_replaces_by_id_else_appends():
|
||||
m = _base()
|
||||
m = upsert_clip(m, {"id": "reef", "title": "Reef"})
|
||||
assert [c["id"] for c in m["clips"]] == ["cosmos", "reef"]
|
||||
m = upsert_clip(m, {"id": "reef", "title": "Coral Reef"}) # replace, not dup
|
||||
assert [c["id"] for c in m["clips"]] == ["cosmos", "reef"]
|
||||
assert m["clips"][1]["title"] == "Coral Reef"
|
||||
|
||||
|
||||
def test_add_ring_scale_inserts_after_named_scale():
|
||||
m = _base()
|
||||
m = add_ring_scale(m, "orbit", "orbit", after="cosmos")
|
||||
assert [s["id"] for s in m["ring"]["scales"]] == ["cosmos", "orbit"]
|
||||
|
||||
|
||||
def test_rebuild_ring_edges_one_transition_per_edge_with_wrap():
|
||||
m = _base()
|
||||
m = add_ring_scale(m, "orbit", "orbit", after="cosmos")
|
||||
m = add_ring_scale(m, "abyss", "abyss", after="orbit")
|
||||
m = rebuild_ring_edges(m)
|
||||
files = [t["file"] for t in m["ring"]["transitions"]]
|
||||
assert files == [
|
||||
"transitions/cosmos-orbit.mp4",
|
||||
"transitions/orbit-abyss.mp4",
|
||||
"transitions/abyss-cosmos.mp4", # wrap edge
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest tests/test_pipeline_manifest.py -q`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'tools.pipeline.manifest'`
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# tools/pipeline/manifest.py
|
||||
"""Stage 6 — assemble manifest entries (spec §3.6). Pure dict transforms over a
|
||||
loaded manifest dict; the caller does the json read/write. Idempotent by id so
|
||||
re-running earlier stages never clobbers authored labels."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
|
||||
|
||||
def upsert_clip(manifest: dict, clip: dict) -> dict:
|
||||
"""Replace the clip with the same id, or append it. Returns a new manifest."""
|
||||
m = copy.deepcopy(manifest)
|
||||
clips = m.setdefault("clips", [])
|
||||
for i, c in enumerate(clips):
|
||||
if c["id"] == clip["id"]:
|
||||
clips[i] = clip
|
||||
return m
|
||||
clips.append(clip)
|
||||
return m
|
||||
|
||||
|
||||
def add_ring_scale(manifest: dict, scale_id: str, clip_id: str,
|
||||
after: str | None = None) -> dict:
|
||||
"""Insert a scale into ring.scales after the named scale (or append if `after`
|
||||
is None / not found). No-op if scale_id already present. Returns a new manifest."""
|
||||
m = copy.deepcopy(manifest)
|
||||
ring = m.setdefault("ring", {"scales": [], "transitions": []})
|
||||
scales = ring.setdefault("scales", [])
|
||||
if any(s["id"] == scale_id for s in scales):
|
||||
return m
|
||||
entry = {"id": scale_id, "clip_id": clip_id}
|
||||
idx = next((i for i, s in enumerate(scales) if s["id"] == after), None)
|
||||
if idx is None:
|
||||
scales.append(entry)
|
||||
else:
|
||||
scales.insert(idx + 1, entry)
|
||||
return m
|
||||
|
||||
|
||||
def rebuild_ring_edges(manifest: dict) -> dict:
|
||||
"""Rebuild ring.transitions to one placeholder entry per ring edge (N scales ->
|
||||
N edges incl. the wrap), preserving any existing model string for an edge.
|
||||
Returns a new manifest."""
|
||||
m = copy.deepcopy(manifest)
|
||||
ring = m.setdefault("ring", {"scales": [], "transitions": []})
|
||||
scales = ring.get("scales", [])
|
||||
existing = {t["file"]: t.get("model", "") for t in ring.get("transitions", [])}
|
||||
edges = []
|
||||
n = len(scales)
|
||||
for i in range(n):
|
||||
a, b = scales[i]["id"], scales[(i + 1) % n]["id"]
|
||||
file = f"transitions/{a}-{b}.mp4"
|
||||
edges.append({"file": file, "model": existing.get(file, "placeholder-zoom")})
|
||||
ring["transitions"] = edges
|
||||
return m
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `pytest tests/test_pipeline_manifest.py -q`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/pipeline/manifest.py tests/test_pipeline_manifest.py
|
||||
git commit -m "feat(pipeline): manifest assembly — upsert clip + ring scale/edges (content pipeline §3.6)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Runner + opt-in integration test on the real forest POC base
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/pipeline/run.py`
|
||||
- Test: `tests/test_pipeline_integration.py`
|
||||
|
||||
The runner shells out ffmpeg with the builders from Tasks 1–3, ffprobing the source
|
||||
duration so the loop stage gets real `D`. The integration test runs the **real**
|
||||
forest POC base (`simulator/sample_media/forest/base.mp4`, populated by
|
||||
`simulator/setup_sample_media.py`) through frame → loop → transcode and ffprobes the
|
||||
proxy — gated on `ffmpeg`/`ffprobe` and on the base existing (it's gitignored media),
|
||||
matching `tests/test_tools_integration.py`'s gating.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# tests/test_pipeline_integration.py
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from tools.pipeline.run import process_clip
|
||||
|
||||
_HAS_FF = shutil.which("ffmpeg") is not None and shutil.which("ffprobe") is not None
|
||||
_FOREST = Path("simulator/sample_media/forest/base.mp4")
|
||||
|
||||
|
||||
@pytest.mark.skipif(not _HAS_FF, reason="ffmpeg/ffprobe not installed")
|
||||
@pytest.mark.skipif(not _FOREST.exists(),
|
||||
reason="forest POC base absent (run simulator/setup_sample_media.py)")
|
||||
def test_process_clip_emits_proxy_and_master(tmp_path):
|
||||
out = process_clip(_FOREST, tmp_path, overlap=0.5, start=0.0, duration=4.0)
|
||||
assert out["proxy"].exists() and out["master"].exists()
|
||||
# Proxy is exactly 1920x1080.
|
||||
probe = subprocess.run(
|
||||
["ffprobe", "-v", "quiet", "-select_streams", "v:0",
|
||||
"-show_entries", "stream=width,height", "-of", "csv=p=0", str(out["proxy"])],
|
||||
capture_output=True, text=True, check=True,
|
||||
).stdout.strip()
|
||||
assert probe == "1920,1080"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest tests/test_pipeline_integration.py -q`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'tools.pipeline.run'`
|
||||
(or SKIP if ffmpeg/the base are absent — then verify by importing in `python -c`.)
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# tools/pipeline/run.py
|
||||
"""Thin runner: ffprobe the source, then execute the frame -> loop -> transcode
|
||||
stages with tools.pipeline.ffmpeg_ops. The pure arg builders are unit-tested;
|
||||
this glue is covered by the opt-in integration test."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from .ffmpeg_ops import crossfade_loop_args, frame_args, transcode_args
|
||||
|
||||
MASTER_MAX_W = 3840
|
||||
|
||||
|
||||
def probe_duration(src, *, ff: str = "ffprobe") -> float:
|
||||
out = subprocess.run(
|
||||
[ff, "-v", "quiet", "-show_entries", "format=duration",
|
||||
"-of", "csv=p=0", str(src)],
|
||||
capture_output=True, text=True, check=True,
|
||||
).stdout.strip()
|
||||
return float(out)
|
||||
|
||||
|
||||
def _run(args: list[str]) -> None:
|
||||
subprocess.run(args, check=True, capture_output=True)
|
||||
|
||||
|
||||
def process_clip(src, out_dir, *, start: float = 0.0, duration: float | None = None,
|
||||
overlap: float = 1.0, master_w: int = 3840, master_h: int = 2160,
|
||||
ff: str = "ffmpeg") -> dict:
|
||||
"""Run stages 2–4 for one clip; return {'master','proxy','loop','mezzanine'} paths.
|
||||
`duration` defaults to the full source length (minus the trim start)."""
|
||||
src = Path(src)
|
||||
out_dir = Path(out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
if duration is None:
|
||||
duration = probe_duration(src) - start
|
||||
|
||||
mezz = out_dir / "mezzanine.mp4"
|
||||
loop = out_dir / "loop.mp4"
|
||||
master = out_dir / "master.mp4"
|
||||
proxy = out_dir / "base.mp4" # the proxy is what the manifest's base_file points at
|
||||
|
||||
_run(frame_args(src, mezz, start=start, duration=duration,
|
||||
width=master_w, height=master_h, ff=ff))
|
||||
_run(crossfade_loop_args(mezz, loop, duration=duration, overlap=overlap, ff=ff))
|
||||
_run(transcode_args(loop, master, width=master_w, height=master_h, crf=18, ff=ff))
|
||||
_run(transcode_args(loop, proxy, width=1920, height=1080, crf=20, ff=ff))
|
||||
return {"mezzanine": mezz, "loop": loop, "master": master, "proxy": proxy}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes (or skips cleanly)**
|
||||
|
||||
Run: `pytest tests/test_pipeline_integration.py -q`
|
||||
Expected: PASS if ffmpeg + the forest base are present; otherwise SKIP with the gate
|
||||
reason. If skipped, also run `python -c "from tools.pipeline.run import process_clip"`
|
||||
and expect no error.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/pipeline/run.py tests/test_pipeline_integration.py
|
||||
git commit -m "feat(pipeline): runner + opt-in integration test on real forest base (content pipeline §3)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Client annotation-track interpolation + a hand-authored track
|
||||
|
||||
**Files:**
|
||||
- Modify: `simulator/static/app.js` (`renderOverlay` at lines ~235–269; `paintLoop` at ~173–186)
|
||||
- Modify: `simulator/sample_media/manifest.json` (add a `track` to the forest `detected.water` annotation)
|
||||
|
||||
The track is *data* on an existing annotation; the Python `Clip` already passes
|
||||
`annotations` through opaquely (`simulator/clips.py` `_clip_from_dict` does
|
||||
`d.get("annotations", [])`), so **no Python change is needed**. Interpolation is
|
||||
client-side (spec §4). Because the overlay currently re-renders only on knob change,
|
||||
a tracked box must update as the video plays — drive a re-render from the existing
|
||||
`paintLoop` rAF when the active clip has any track and the overlay is visible.
|
||||
|
||||
- [ ] **Step 1: Add the data — a hand-authored track on forest's water detection**
|
||||
|
||||
In `simulator/sample_media/manifest.json`, replace the forest `detected.water`
|
||||
annotation (currently `{"key": "detected.water", "box": [0.30, 0.10, 0.18, 0.70], "min_level": 1}`)
|
||||
with a keyframed track (a small horizontal drift, looping at t=0 and t=1 so it is
|
||||
seamless):
|
||||
|
||||
```json
|
||||
{
|
||||
"key": "detected.water",
|
||||
"min_level": 1,
|
||||
"track": [
|
||||
{"t": 0.0, "box": [0.30, 0.10, 0.18, 0.70]},
|
||||
{"t": 0.5, "box": [0.33, 0.10, 0.18, 0.70]},
|
||||
{"t": 1.0, "box": [0.30, 0.10, 0.18, 0.70]}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add the interpolation helper + use it in `renderOverlay`**
|
||||
|
||||
In `simulator/static/app.js`, add a helper above `renderOverlay` and use it instead
|
||||
of `a.box`:
|
||||
|
||||
```javascript
|
||||
// Interpolate an annotation's box at loop-normalized time t (0..1). Static labels
|
||||
// (no track) return their fixed box. Linear between sorted keyframes; clamps to ends.
|
||||
function boxAt(a, t) {
|
||||
if (!a.track || !a.track.length) return a.box;
|
||||
const ks = a.track;
|
||||
if (t <= ks[0].t) return ks[0].box;
|
||||
if (t >= ks[ks.length - 1].t) return ks[ks.length - 1].box;
|
||||
for (let i = 1; i < ks.length; i++) {
|
||||
if (t <= ks[i].t) {
|
||||
const a0 = ks[i - 1], a1 = ks[i];
|
||||
const f = (t - a0.t) / (a1.t - a0.t);
|
||||
return a0.box.map((v, j) => v + (a1.box[j] - v) * f);
|
||||
}
|
||||
}
|
||||
return ks[ks.length - 1].box;
|
||||
}
|
||||
|
||||
// Loop-normalized playback time of the base video (0..1), for track interpolation.
|
||||
function loopT() {
|
||||
return vid && vid.duration ? (vid.currentTime % vid.duration) / vid.duration : 0;
|
||||
}
|
||||
```
|
||||
|
||||
Then in `renderOverlay`, change the box line:
|
||||
|
||||
```javascript
|
||||
const [x, y, w, h] = boxAt(a, loopT()).map((n) => n * 100);
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Re-render tracked overlays each frame from `paintLoop`**
|
||||
|
||||
Store the last overlay args and, in `paintLoop`, re-run `renderOverlay` when the
|
||||
active clip has any tracked annotation and the overlay is visible. Add near the top
|
||||
of `app.js` (module scope): `let lastOverlay = null;`. At the end of `renderOverlay`,
|
||||
record the call: set `lastOverlay = { level, intensity };` (add this line just before
|
||||
the closing brace). In `paintLoop` (after the existing shader work, before
|
||||
`requestAnimationFrame(paintLoop)`), add:
|
||||
|
||||
```javascript
|
||||
// Animate keyframed annotation tracks: re-place boxes against playback time.
|
||||
const c = activeClip();
|
||||
if (lastOverlay && lastOverlay.level > 0 && c &&
|
||||
c.annotations.some((a) => a.track && a.track.length)) {
|
||||
renderOverlay(lastOverlay.level, lastOverlay.intensity);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify by eye (no JS unit harness in this repo)**
|
||||
|
||||
Run the sim and watch the forest water reticle drift with playback, looping
|
||||
seamlessly; static labels (rock_face, conifer, the measurement) stay put.
|
||||
|
||||
```bash
|
||||
python -m simulator.app # then open the printed localhost URL
|
||||
```
|
||||
|
||||
Expected: with Left up on forest, the `flowing water` box drifts right then back
|
||||
over the loop with no jump at the loop seam; affect/measure labels unchanged.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add simulator/static/app.js simulator/sample_media/manifest.json
|
||||
git commit -m "feat(sim): client annotation-track interpolation + hand-authored forest track (content pipeline §4)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Extend the ring to 5 scales (orbit + reef placeholders)
|
||||
|
||||
**Files:**
|
||||
- Modify: `simulator/setup_scales_media.py` (`SCALES` lines ~32–36; `EDGES` line ~39)
|
||||
- Modify: `simulator/sample_media/manifest.json` (add `orbit` + `reef` clips; reorder `ring.scales`; rebuild `ring.transitions`)
|
||||
|
||||
Real strict-PD bases are sourced separately (Task 9); to keep the 5-scale ring
|
||||
demonstrable now, generate labelled placeholders for the two new scales with the
|
||||
existing generator, exactly as cosmos/abyss are today.
|
||||
|
||||
- [ ] **Step 1: Add orbit + reef to the placeholder generator**
|
||||
|
||||
In `simulator/setup_scales_media.py`, extend `SCALES` and `EDGES` to the 5-scale
|
||||
ring (order: cosmos → orbit → forest → reef → abyss → wrap):
|
||||
|
||||
```python
|
||||
SCALES = {
|
||||
"cosmos": ("0x05060f", "COSMOS — NASA/Hubble (PD placeholder)"),
|
||||
"orbit": ("0x081427", "ORBIT — NASA/ISS (PD placeholder)"),
|
||||
"forest": ("0x12301a", "FOREST — NPS/USGS (POC/placeholder)"),
|
||||
"reef": ("0x06303a", "REEF — NOAA Ocean Exploration (PD placeholder)"),
|
||||
"abyss": ("0x021016", "ABYSS — NOAA Ocean Exploration (PD placeholder)"),
|
||||
}
|
||||
|
||||
EDGES = [("cosmos", "orbit"), ("orbit", "forest"), ("forest", "reef"),
|
||||
("reef", "abyss"), ("abyss", "cosmos")]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add the orbit + reef clip entries to the manifest**
|
||||
|
||||
In `simulator/sample_media/manifest.json`, add two clip objects (after `cosmos` and
|
||||
after `forest` respectively) mirroring the existing placeholder clips' shape, e.g.
|
||||
orbit:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "orbit",
|
||||
"title": "Earth from orbit (NASA/ISS, neutral base)",
|
||||
"base_file": "orbit/base.mp4",
|
||||
"license": "public-domain (US Gov, 17 U.S.C. §105)",
|
||||
"source": "NASA/ISS — https://images.nasa.gov/ (true PD; placeholder bytes until ingest)",
|
||||
"right_variants": {},
|
||||
"annotations": [
|
||||
{"key": "detected.cloud_band", "box": [0.30, 0.30, 0.30, 0.20], "min_level": 1},
|
||||
{"key": "measure.altitude", "box": [0.06, 0.06, 0.18, 0.08], "min_level": 2}
|
||||
],
|
||||
"affect": [
|
||||
{"key": "feel.serenity", "at": [0.50, 0.46], "min_level": 1},
|
||||
{"key": "feel.fragility", "at": [0.22, 0.68], "min_level": 2},
|
||||
{"key": "feel.unity", "at": [0.66, 0.58], "min_level": 3},
|
||||
{"key": "feel.distance", "at": [0.42, 0.82], "min_level": 4}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.cloud_band": "cloud band",
|
||||
"measure.altitude": "~408 km",
|
||||
"feel.serenity": "serenity", "feel.fragility": "fragility",
|
||||
"feel.unity": "unity", "feel.distance": "distance"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
And reef:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "reef",
|
||||
"title": "Coral reef (NOAA Ocean Exploration, neutral base)",
|
||||
"base_file": "reef/base.mp4",
|
||||
"license": "public-domain (US Gov, 17 U.S.C. §105)",
|
||||
"source": "NOAA Ocean Exploration — https://oceanexplorer.noaa.gov/ (true PD, worldwide; placeholder bytes until ingest)",
|
||||
"right_variants": {},
|
||||
"annotations": [
|
||||
{"key": "detected.coral", "box": [0.20, 0.45, 0.30, 0.30], "min_level": 1},
|
||||
{"key": "detected.fish", "min_level": 2,
|
||||
"track": [
|
||||
{"t": 0.0, "box": [0.20, 0.30, 0.10, 0.08]},
|
||||
{"t": 0.5, "box": [0.62, 0.34, 0.10, 0.08]},
|
||||
{"t": 1.0, "box": [0.20, 0.30, 0.10, 0.08]}
|
||||
]},
|
||||
{"key": "measure.depth", "box": [0.06, 0.06, 0.16, 0.08], "min_level": 3}
|
||||
],
|
||||
"affect": [
|
||||
{"key": "feel.delight", "at": [0.50, 0.46], "min_level": 1},
|
||||
{"key": "feel.abundance", "at": [0.22, 0.68], "min_level": 2},
|
||||
{"key": "feel.curiosity", "at": [0.66, 0.58], "min_level": 3},
|
||||
{"key": "feel.immersion", "at": [0.42, 0.82], "min_level": 4}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.coral": "coral colony", "detected.fish": "reef fish",
|
||||
"measure.depth": "−18 m",
|
||||
"feel.delight": "delight", "feel.abundance": "abundance",
|
||||
"feel.curiosity": "curiosity", "feel.immersion": "immersion"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Reorder the ring + rebuild edges in the manifest**
|
||||
|
||||
Set `ring.scales` to the 5-scale order and `ring.transitions` to the 5 edges:
|
||||
|
||||
```json
|
||||
"ring": {
|
||||
"scales": [
|
||||
{"id": "cosmos", "clip_id": "cosmos"},
|
||||
{"id": "orbit", "clip_id": "orbit"},
|
||||
{"id": "forest", "clip_id": "forest"},
|
||||
{"id": "reef", "clip_id": "reef"},
|
||||
{"id": "abyss", "clip_id": "abyss"}
|
||||
],
|
||||
"transitions": [
|
||||
{"file": "transitions/cosmos-orbit.mp4", "model": "placeholder-zoom"},
|
||||
{"file": "transitions/orbit-forest.mp4", "model": "placeholder-zoom"},
|
||||
{"file": "transitions/forest-reef.mp4", "model": "placeholder-zoom"},
|
||||
{"file": "transitions/reef-abyss.mp4", "model": "placeholder-zoom"},
|
||||
{"file": "transitions/abyss-cosmos.mp4", "model": "placeholder-zoom"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Regenerate placeholder media + verify the 5-scale ring**
|
||||
|
||||
```bash
|
||||
python simulator/setup_scales_media.py
|
||||
python -m simulator.app # open the URL; spin the encoder full circle
|
||||
```
|
||||
|
||||
Expected: the encoder walks cosmos → orbit → forest → reef → abyss → (wrap) cosmos
|
||||
with a transition on every edge; `GET /api/ring` lists 5 scales + 5 transitions; the
|
||||
reef `reef fish` box drifts with playback (Task 7 interpolation) when Left ≥ 2.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add simulator/setup_scales_media.py simulator/sample_media/manifest.json
|
||||
git commit -m "feat(sim): extend scale ring to 5 (orbit + reef) (content pipeline §5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Sourcing procedure doc + ROADMAP update
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/content-sourcing.md`
|
||||
- Modify: `docs/ROADMAP.md` (sub-project 3 "Offline v2v variant pipeline" bullet, lines ~174–187)
|
||||
|
||||
Sourcing the real strict-PD bytes is a curatorial task, not a code step. Document the
|
||||
procedure so anyone can run the pipeline per clip, and record the new content-pipeline
|
||||
work on the roadmap.
|
||||
|
||||
- [ ] **Step 1: Write the sourcing procedure**
|
||||
|
||||
```markdown
|
||||
# Content sourcing — strict-PD neutral bases
|
||||
|
||||
For each scale, acquire a calm, neutral source clip from a **strict public-domain**
|
||||
US-government source, verify the license, then run it through the pipeline.
|
||||
|
||||
| Scale | Source | Where |
|
||||
|--------|--------------------------------|------------------------------------|
|
||||
| cosmos | NASA / Hubble / JWST | https://images.nasa.gov/ |
|
||||
| orbit | NASA / ISS | https://images.nasa.gov/ |
|
||||
| forest | NPS / USGS | https://www.nps.gov/ , https://www.usgs.gov/ |
|
||||
| reef | NOAA Ocean Exploration | https://oceanexplorer.noaa.gov/ |
|
||||
| abyss | NOAA Ocean Exploration | https://oceanexplorer.noaa.gov/ |
|
||||
|
||||
**License check (mandatory):** confirm the asset is US-gov PD (17 U.S.C. §105) or
|
||||
explicitly CC0. "Free stock" (Pexels/Pixabay/etc.) is royalty-free but NOT public
|
||||
domain — do not use. Record a `Provenance` (tools/pipeline/provenance.py) per clip.
|
||||
|
||||
**Run the pipeline** (replaces the placeholder bytes for a scale):
|
||||
|
||||
```bash
|
||||
python -c "from tools.pipeline.run import process_clip; \
|
||||
process_clip('downloads/<scale>.mp4', 'simulator/sample_media/<scale>', \
|
||||
start=<in_s>, duration=<len_s>, overlap=<0.5..1.0>)"
|
||||
```
|
||||
|
||||
This writes `simulator/sample_media/<scale>/base.mp4` (proxy) + `master.mp4`. Then
|
||||
author the labels (Increment 2: the hybrid track pass + simulator author mode) or
|
||||
hand-edit the annotation track for now (spec §4). The ring + manifest entry already
|
||||
exist (Task 8); `process_clip` only replaces the media bytes.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update the roadmap**
|
||||
|
||||
In `docs/ROADMAP.md`, under sub-project 3, replace the "Offline v2v variant pipeline"
|
||||
framing with the content-pipeline reality: the Right dream is real-time (no ML bake),
|
||||
so the content pipeline is **source → crossfade-loop → master+proxy transcode →
|
||||
hand-authored/motion-tracked labels → manifest**. Note Increment 1 (this plan: tooling
|
||||
+ track schema + 5-scale ring, no ML) done on merge; Increment 2 (hybrid track pass +
|
||||
simulator author mode) and real i2v ring transitions still pending. Cite the spec
|
||||
`docs/superpowers/specs/2026-06-24-content-pipeline-design.md`.
|
||||
|
||||
- [ ] **Step 3: Run the full suite**
|
||||
|
||||
Run: `pytest -q`
|
||||
Expected: all prior tests still pass + the new pipeline tests (ffmpeg_ops,
|
||||
provenance, manifest) pass; the integration test passes or skips cleanly.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/content-sourcing.md docs/ROADMAP.md
|
||||
git commit -m "docs: content-sourcing procedure + roadmap update (content pipeline Increment 1)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes for the executor
|
||||
|
||||
- **Run from the main clone, not a nested worktree** — `resolve-app.py` errors
|
||||
"ambiguous" when a `.worktrees/` checkout shadows the main clone (session-0013
|
||||
gotcha). If you must use a worktree, expect the `WGL_APP_SCAN_DEPTH` / explicit
|
||||
`publish-transcript.sh` flag workarounds.
|
||||
- **PRs go through the Gitea API** (`wgl-gitea-admin` `gitea-api.sh`, host
|
||||
`git.benstull.org`) — no `tea`/`gh` here: create via `POST /repos/.../pulls`,
|
||||
merge via `.../merge`.
|
||||
- The project `.venv` has the deps (pytest, cv2/torch for later ML work). Activate it
|
||||
before running tests.
|
||||
- **Increment 2 (separate plan):** `tools/pipeline/track.py` (classical OpenCV
|
||||
tracker + optional lazy-imported ML detect+track) and the simulator **author mode**
|
||||
(draw/seed/correct boxes, assign keys/min_level/strings, place affect, write the
|
||||
manifest). Plus real i2v ring transitions once a generative-video model + adjacent
|
||||
real bases exist.
|
||||
```
|
||||
@@ -121,13 +121,6 @@ layers. This single rule covers the whole `5×5×5×5` space:
|
||||
|
||||
### 4.3 Where each transform runs
|
||||
|
||||
> **Reconciled (2026-06-07, session 0009):** the Left HUD is a **runtime overlay**
|
||||
> driven by an authored annotation track + per-language string tables (text shaped
|
||||
> live); the Right axis selects a **discrete pre-baked** flow-stabilized variant
|
||||
> (not a continuous blend). See
|
||||
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
|
||||
> §1, which supersedes the session-0007 baked-HUD / 5×5-grid proposal.
|
||||
|
||||
- **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade
|
||||
and the Left analytical overlay. These are cheap (LUT/curves + luma key, and
|
||||
text/graphics compositing) and can move continuously with the knob.
|
||||
@@ -242,13 +235,6 @@ one-time build.
|
||||
|
||||
## 10. Accessibility, i18n, and the translation-cost finding
|
||||
|
||||
> **Reconciled (2026-06-07, session 0009):** the near-free-i18n path is kept — the
|
||||
> Left HUD is a runtime overlay (authored annotation track + per-language string
|
||||
> tables, shaped live via Pango/HarfBuzz on the Pi). The session-0007 baked-HUD
|
||||
> reversal is **not** adopted. See
|
||||
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
|
||||
> §1.
|
||||
|
||||
The piece is operable **blind, in the dark, in your language**, via four redundant
|
||||
channels on the control panel: **touch** (engraved symbol shape), **low-light
|
||||
color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local*
|
||||
|
||||
@@ -1,299 +0,0 @@
|
||||
# HEF — Reconciled Simulator-First Alteration Slice (Design)
|
||||
|
||||
**Date:** 2026-06-07
|
||||
**Status:** Approved design (pre-implementation) — reconciliation approved this session (0009)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Reconciles:**
|
||||
[`2026-06-06-simulator-alteration-preview-design.md`](./2026-06-06-simulator-alteration-preview-design.md)
|
||||
(session 0007, the *unmerged* `feature/simulator-alteration-preview` branch) **with**
|
||||
[`2026-06-07-scales-library-and-right-axis-pipeline-design.md`](./2026-06-07-scales-library-and-right-axis-pipeline-design.md)
|
||||
(session 0008, merged to `main`).
|
||||
**Parents:**
|
||||
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
|
||||
(the alteration engine) and
|
||||
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md)
|
||||
(the simulator scaffold).
|
||||
**Supersedes:** the **baked-HUD / 5×5-grid** position of the 0007 design (§4, §5, §8 there).
|
||||
|
||||
> **Why this exists.** Sessions 0007 and 0008 left two unmerged design threads that
|
||||
> disagree on one load-bearing point — how the **Left** analytical HUD is rendered —
|
||||
> and, downstream of that, on the **shape of the pre-baked variant set**. 0007 baked
|
||||
> the Left HUD into a 5×5 grid of authored Left×Right variant clips (pixel-precise,
|
||||
> but i18n becomes expensive). 0008 (the later, merged design) kept the Left HUD a
|
||||
> **runtime overlay** driven by an authored annotation track + per-language string
|
||||
> tables (near-free i18n). This document picks the runtime-overlay position, follows
|
||||
> its consequences through the engine and the simulator, and scopes the first
|
||||
> **simulator-runnable** slice that realizes it. It then hands off to an
|
||||
> implementation plan.
|
||||
|
||||
---
|
||||
|
||||
## 1. The decision (operator-approved this session)
|
||||
|
||||
**Left is a runtime overlay, not baked pixels.** Concretely:
|
||||
|
||||
- **Left axis** → a **runtime `AnalyticalOverlay`** driven by an **offline-authored
|
||||
annotation track** (box positions, anchor points, and which label *keys* appear at
|
||||
each Left level 0–4) plus a per-language **string table** (key → translated text).
|
||||
Text is **shaped live** — natively by the browser in the simulator; by
|
||||
**Pango + HarfBuzz + Noto** on the Pi at runtime (the 0008 §1.2 requirement). This
|
||||
is the "authored box positions + runtime-shaped text" hybrid: authorial control
|
||||
over *layout*, near-free *i18n*.
|
||||
- **Right axis** → the **only pre-baked axis**: a small set of **flow-stabilized
|
||||
restyle-strength variants** (0008 §1), selected **discretely** by the Right knob
|
||||
(0–4), with a **crossfade** on change.
|
||||
- **Dark/Light** → a **live runtime `ColorGrade`** (mood center = identity, §5 of the
|
||||
parent).
|
||||
|
||||
**What this supersedes.** The 0007 design's 5×5 Left×Right **grid of 24 authored
|
||||
clips** and its **removal of `AnalyticalOverlay`/`Restyle` from `RenderPlan`** are
|
||||
dropped. The Left HUD is *not* baked; the variant set is **1-D over Right strength**,
|
||||
not a 2-D grid. (Accepted loss vs. fully-baked HUD: no pixel-painted HUD *artwork* —
|
||||
the HUD is shaped text + drawn boxes. Accepted gain: near-free i18n + far less
|
||||
authoring — N Right variants per clip, not 24.)
|
||||
|
||||
**What it preserves from 0007.** The valuable, non-conflicting parts: Python-canonical
|
||||
engine with a thin browser renderer; a parameterized **`Calibration`** tuned by eye in
|
||||
the sim; retiring the simulator's selection-era surface; placeholder-variant generation
|
||||
so the mechanism is testable before real authored media exists.
|
||||
|
||||
### 1.1 The merged engine is already most of the way there
|
||||
|
||||
The slice-1 engine on `main` (`player/alteration.py`) **already** models
|
||||
`RenderPlan = { grade, overlay, restyle }` with a runtime `AnalyticalOverlay` — only
|
||||
0007's *unmerged doc* proposed removing it. So this reconciliation is **surgical**, not
|
||||
a rewrite: keep `grade` and `overlay`; change only how the **Right** axis and the
|
||||
**calibration** are modeled.
|
||||
|
||||
---
|
||||
|
||||
## 2. Engine reconciliation (`player/alteration.py`, `player/state.py`)
|
||||
|
||||
### 2.1 `RenderPlan` layers
|
||||
|
||||
| Layer | Axis | Type | Change from `main` |
|
||||
|---|---|---|---|
|
||||
| `grade: ColorGrade` | Dark/Light | `tone ∈ [−1,1]`, 0 = identity | unchanged shape; curve now from `Calibration` |
|
||||
| `overlay: AnalyticalOverlay` | Left | gains discrete `level: int` (0–4) + keeps `intensity` | `level` added so the renderer selects which annotations show |
|
||||
| `restyle: Restyle` | Right | **discrete** `variant: int` (0–4); 0 = raw | replaces continuous `blend: float` |
|
||||
|
||||
- **`Restyle.variant`** is a discrete index selecting a **pre-baked** Right-strength
|
||||
clip. `variant == 0` means the raw base (no restyle). This matches "select a
|
||||
pre-baked variant and crossfade," and replaces the continuous `blend` that no longer
|
||||
has a runtime meaning (restyle is pre-baked, not blended live).
|
||||
- **`AnalyticalOverlay.level`** (0–4) is the Left knob value; the renderer uses it to
|
||||
choose which annotations from the authored track are active. `intensity` (0..1)
|
||||
stays for overlay opacity/strength and is derived from `level` via `Calibration`.
|
||||
- **`RenderPlan.is_identity`** holds when `grade.is_identity and overlay.level == 0
|
||||
and restyle.variant == 0`.
|
||||
|
||||
### 2.2 `Calibration` (new frozen dataclass)
|
||||
|
||||
A frozen `Calibration` parameterizes the knob→strength maps so they can be tuned **by
|
||||
eye in the sim** and then baked into a `DEFAULT_CALIBRATION` constant:
|
||||
|
||||
- `mood_center` and a per-axis curve for `_mood_tone` (Dark/Light).
|
||||
- the Left `level → intensity` curve.
|
||||
- the Right `knob → variant` map (which knob positions select which pre-baked
|
||||
strength; identity-preserving so knob 0 → variant 0).
|
||||
|
||||
`plan_alteration(coord, calibration: Calibration = DEFAULT_CALIBRATION) -> RenderPlan`.
|
||||
**`DEFAULT_CALIBRATION` reproduces today's exact behavior** (the three current helpers:
|
||||
`value/4` for Left intensity, `(light−dark)/4` for mood, `right` → `variant=right`), so
|
||||
the change is behavior-preserving until the operator tunes it. The session-0006
|
||||
knob→strength open decision is then settled **by eye** in the sim's calibration panel
|
||||
and locked into `DEFAULT_CALIBRATION` by a unit test.
|
||||
|
||||
### 2.3 `player/state.py`
|
||||
|
||||
`state.py` already classifies a change to `plan.restyle` as a `CROSSFADE` and a grade-/
|
||||
overlay-only change as `LIVE_UPDATE`. With `Restyle.variant` discrete, the same
|
||||
classifier works unchanged: a new Right variant → `CROSSFADE`; a grade or Left-overlay
|
||||
change → `LIVE_UPDATE`; clip swap or video on/off → crossfade / fade-to-black as today.
|
||||
The only edit is to the `_classify` comparison if the field name changes
|
||||
(`restyle.blend` → `restyle.variant`).
|
||||
|
||||
These are pure framework-code changes; no deployment-shape decision enters the engine.
|
||||
|
||||
---
|
||||
|
||||
## 3. Simulator (`simulator/`)
|
||||
|
||||
### 3.1 Retire the selection era
|
||||
|
||||
The simulator currently visualizes the **old selection model** (the "curator's X-ray").
|
||||
Remove:
|
||||
|
||||
- `POST /api/select`, `GET /api/catalog/meta`;
|
||||
- the X-ray static UI (`simulator/static/*` rewritten, see §3.4);
|
||||
- `simulator/fixtures.py` (the 625-record synthetic *catalog*) — replaced by
|
||||
`simulator/clips.py`.
|
||||
|
||||
`hef.selection` (the library, incl. `Coordinate`, `ranked_candidates`) is **untouched**;
|
||||
only the simulator's selection *surface* retires. `Coordinate` is still used by the
|
||||
alteration engine.
|
||||
|
||||
### 3.2 `simulator/clips.py` (replaces `fixtures.py`)
|
||||
|
||||
Reads a base-clip + variant manifest. Per base clip:
|
||||
|
||||
```
|
||||
{
|
||||
"id": "forest",
|
||||
"title": "...",
|
||||
"base_file": "forest/base.mp4",
|
||||
"license": "...", "source": "...",
|
||||
"right_variants": { "1": {"file": "forest/right1.mp4", "model": "..."},
|
||||
"...": {...}, "4": {"file": "forest/right4.mp4"} },
|
||||
"annotations": [ {"key": "detected.conifer", "box": [x,y,w,h], "min_level": 1}, ... ],
|
||||
"strings": { "en": { "detected.conifer": "conifer", ... } }
|
||||
}
|
||||
```
|
||||
|
||||
- `right_variants` is keyed by Right strength `1..4` (strength `0` = raw `base_file`).
|
||||
Missing strengths fall back to the raw base and are flagged "raw / unauthored" in the
|
||||
readout.
|
||||
- `annotations` is the **authored annotation track** (box + label key + the minimum
|
||||
Left level at which it appears). `strings` is the per-language table (English only
|
||||
this slice).
|
||||
|
||||
### 3.3 Endpoints
|
||||
|
||||
- **`POST /api/alteration`** — body `{ controls, calibration? }` → `RenderPlan`
|
||||
(serialized) **plus** the `ContentResolution` from `resolve_content` (so the content
|
||||
dial's video-on/off is honored — "Off" → black). Calls the real
|
||||
`plan_alteration(coord, calibration)`.
|
||||
- **`GET /api/clips`** — the base-clip list + the active clip's manifest (variants +
|
||||
annotation track + string table).
|
||||
- Video served as static assets from a sample-media directory.
|
||||
- **Removed:** `POST /api/select`, `GET /api/catalog/meta`.
|
||||
|
||||
### 3.4 Browser preview (`simulator/static/`)
|
||||
|
||||
Thin renderer; all alteration math stays in Python.
|
||||
|
||||
- **Right** — `<video>` showing the selected Right-variant file; **opacity crossfade**
|
||||
to the new file when `restyle.variant` changes; variant 0 → the raw base.
|
||||
- **Dark/Light** — **live color grade** over the video via CSS/canvas filters: Light →
|
||||
warm + lifted toward white; Dark → cool + crushed toward black; tone 0 → raw.
|
||||
- **Left** — overlay drawn **live** from the annotation track: for each annotation with
|
||||
`min_level ≤ overlay.level`, draw its box and the **shaped** string (browser-native
|
||||
shaping) for the active language; opacity from `overlay.intensity`. No baked HUD.
|
||||
- **Content dial** — drives `<video>` visibility; "Off"/audio-only → black walls.
|
||||
- **Calibration panel** — sliders for the `Calibration` params; changing them
|
||||
re-requests the plan and the footage responds live.
|
||||
- **RenderPlan readout** — always shows the exact engine numbers (grade tone, overlay
|
||||
level/intensity, restyle variant) — the project's honesty "X-ray," now over the
|
||||
alteration model.
|
||||
|
||||
The Left overlay being browser-drawn is the **simulator analogue** of the Pi's
|
||||
Pango/HarfBuzz path: both take the *same* annotation track + string table; the browser
|
||||
shapes natively, the Pi shapes with HarfBuzz. The manifest is the shared contract.
|
||||
|
||||
---
|
||||
|
||||
## 4. The sample clip + a real Right variant (this slice)
|
||||
|
||||
To make the look **evaluable now**, wire one real clip end-to-end using the session-0008
|
||||
POC artifacts (`~/hef-poc/out/`, outside the repo):
|
||||
|
||||
- **Base clip** = the POC's `neutral.mp4` (an 8 s nature loop) → copied to the
|
||||
sample-media dir as the one base clip.
|
||||
- **Right variant (top strength)** = the POC's **`right_flow.mp4`** — the real,
|
||||
operator-approved **flow-stabilized** restyle — wired as Right strength 4.
|
||||
- **Intermediate Right strengths (1–3)** = generated by a small **ffmpeg placeholder
|
||||
generator** (e.g. graded/blended stand-ins) so the crossfade mechanism is exercised
|
||||
across the full knob range; the real high-end look is present for tuning.
|
||||
- **Left annotation track** = a minimal authored track (a few boxes + English label
|
||||
keys) for that clip, so Left renders as real shaped text over drawn boxes.
|
||||
|
||||
> **Licensing note.** The simulator sample footage exists **only to tune the look**; it
|
||||
> is not shipped installation content. Strict-PD scale-library sourcing (NASA/NOAA/NPS
|
||||
> per 0008 §2.1) remains a later slice and is unaffected by this choice.
|
||||
|
||||
A real multi-strength SD re-bake (4 genuine flow-stabilized strengths) is **out of scope
|
||||
this slice** — one real strength + placeholders is enough to settle the mechanism and the
|
||||
calibration. The re-bake is a later offline-pipeline task.
|
||||
|
||||
---
|
||||
|
||||
## 5. Testing
|
||||
|
||||
- **`player/` unit tests** (`tests/test_player_alteration.py`, `test_player_state.py`):
|
||||
- `DEFAULT_CALIBRATION` reproduces the current helpers exactly (behavior-preserving).
|
||||
- `Restyle.variant` is discrete; knob 0 → variant 0 (identity); a non-default
|
||||
`Calibration` changes the plan as specified.
|
||||
- `AnalyticalOverlay.level` maps from the Left knob; `intensity` derives from it.
|
||||
- `state.py`: a Right-variant change → `CROSSFADE`; a grade-/overlay-only change →
|
||||
`LIVE_UPDATE`; video on/off unchanged.
|
||||
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
|
||||
- `POST /api/alteration` returns the engine's plan (+ `ContentResolution`) for given
|
||||
controls/calibration.
|
||||
- `GET /api/clips` returns the manifest.
|
||||
- the removed endpoints (`/api/select`, `/api/catalog/meta`) are gone (404).
|
||||
- `test_fixtures.py` retired/rewritten for `clips.py`.
|
||||
- **No browser/E2E automation** this slice — manual visual tuning is the point; the JS
|
||||
stays thin and the tested logic stays in Python.
|
||||
|
||||
---
|
||||
|
||||
## 6. What ships
|
||||
|
||||
- `player/alteration.py` — `Calibration` + `DEFAULT_CALIBRATION`, discrete
|
||||
`Restyle.variant`, `AnalyticalOverlay.level`; `plan_alteration(coord, calibration)`.
|
||||
- `player/state.py` — crossfade-trigger field rename only.
|
||||
- `simulator/clips.py` (variant + annotation manifest) replacing `fixtures.py`.
|
||||
- `simulator/app.py` — `/api/alteration` + `/api/clips`; selection endpoints removed.
|
||||
- `simulator/static/` — rewritten as the alteration preview (variant `<video>` +
|
||||
crossfade, live grade, live Left overlay, content dial, calibration panel,
|
||||
RenderPlan readout).
|
||||
- Sample base clip + one real Right variant + placeholder generator + minimal Left
|
||||
annotation track / English strings.
|
||||
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten; the parent
|
||||
design §4.3/§10 pointer updated to cite this reconciliation; `docs/ROADMAP.md` §3
|
||||
updated.
|
||||
|
||||
---
|
||||
|
||||
## 7. Out of scope (YAGNI) — later slices
|
||||
|
||||
- Serial input / the 3⇄4 framing contract; the Pi/mpv/GPU runtime renderer (deferred by
|
||||
`simulator-first-before-hardware`).
|
||||
- Audio playback (music / white-noise / audio-track).
|
||||
- The **endless rotary encoder** + **AI zoom/warp transitions** between scales
|
||||
(0008 §3) — a separate control + offline pipeline element.
|
||||
- A real multi-strength SD flow-stabilized re-bake; strict-PD scale-library sourcing
|
||||
(0008 §2.1).
|
||||
- Catalog-model changes (audio source / neutral-vs-variant flag); retiring
|
||||
`hef.selection.ranked_candidates`.
|
||||
- Broad multilingual string tables (English-first; the runtime path keeps i18n cheap,
|
||||
but authoring other languages is later).
|
||||
|
||||
---
|
||||
|
||||
## 8. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Calibration curve shape** — **RESOLVED (session 0010, by eye).**
|
||||
`DEFAULT_CALIBRATION` is **locked** to unity gains + a linear variant map
|
||||
(`mood_gain=1.0`, `overlay_gain=1.0`, `right_variant_map=(0,1,2,3,4)`), as a
|
||||
deliberate choice: with the dark-grade fix below, full knob is peaceful on
|
||||
every axis (POC + sim), so full tilt = full look and the 5 notches map 1:1 to
|
||||
the 5 discrete Right bakes. This also closes the **session-0006** convention
|
||||
question — knobs run 0=off..4=max, equal Dark/Light = identity; no
|
||||
"centered at 2 = no push." Guarded by `test_default_calibration_is_locked`.
|
||||
- **Grade vs. Left overlay interaction** — **RESOLVED: overlay above the grade.**
|
||||
The simulator composites the Left HUD (SVG) above the mood grade and the cool
|
||||
tint, so the HUD stays legible regardless of mood. The Pi renderer should do
|
||||
the same.
|
||||
- **Dark-pole grade look** — **FIXED (session 0010).** The first by-eye pass found
|
||||
the sim's dark grade used a full-frame `hue-rotate(-200deg)`, which turned the
|
||||
rock orange and trees purple — the disorienting look rejected in 0008, not the
|
||||
peaceful POC `dark_frame`. Replaced with darken + slight desaturate on the video
|
||||
filter plus a `multiply`-blended deep-blue wash (`#tint`) that lifts shadows
|
||||
toward blue while preserving natural greens. The Pi renderer (later slice) will
|
||||
do proper grading; this matches the approved POC dark look closely enough to tune
|
||||
by eye in the sim.
|
||||
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for tuning;
|
||||
the real timing engine is a later slice.
|
||||
- **Placeholder fidelity** — how close the strength-1–3 placeholders should look to real
|
||||
restyle; cheap stand-ins are fine for mechanism + calibration.
|
||||
@@ -1,247 +0,0 @@
|
||||
# HEF — Scales-of-Nature Library + Stabilized Right-Axis Pipeline (Design Revision)
|
||||
|
||||
**Date:** 2026-06-07
|
||||
**Status:** Approved design (pre-implementation) — operator-approved this session (0008)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Refines:** [`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
|
||||
— specifically its Right-axis pipeline (§4.1/§4.3), content sourcing/model (§6/§8),
|
||||
and economics (§9), and it **adds a scale-navigation control + zoom transitions**
|
||||
to the §2 selector / §11 control panel. The thesis (§1), coordinate model (§3),
|
||||
Dark/Light/Left treatment, and accessibility (§10) are **preserved**.
|
||||
**Grounded in:** a local proof-of-concept run this session on the operator's Mac
|
||||
mini (M4 Pro, 64 GB, MPS) — all numbers below are measured, not estimated.
|
||||
|
||||
> **Why this revision exists.** The 2026-06-05 design specified the Right axis as
|
||||
> "generative video-to-video restyle, pre-baked offline" and assumed that meant a
|
||||
> **paid cloud API** (§9 priced Kling/Sora/Veo/Runway at $0.05–0.10/s). A POC this
|
||||
> session established two things that change the design: (1) the Right restyle runs
|
||||
> **entirely locally and offline** on the operator's existing hardware, for the
|
||||
> cost of electricity; and (2) naïve per-frame restyle **boils/flickers** in a way
|
||||
> the operator found disorienting — disqualifying for a piece meant to be peaceful
|
||||
> — and the fix is **optical-flow keyframe propagation**. Separately, the operator
|
||||
> chose how the "scales of nature" idea enters the piece: as the curatorial theme
|
||||
> of a **small neutral base library**, not a single fixed journey.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Right axis is a local, flow-stabilized restyle (refines §4.1, §4.3)
|
||||
|
||||
The §4.1 mapping is unchanged in spirit — **Right = dissolve realism toward
|
||||
painterly/dreamlike via generative video-to-video** — but the *implementation* is
|
||||
now pinned:
|
||||
|
||||
- **Engine:** Stable Diffusion **img2img** (POC used `stabilityai/sd-turbo`) run on
|
||||
**Apple MPS**, locally, offline, at authoring time. No cloud API.
|
||||
- **Temporal coherence is a hard requirement, not a nicety.** Per-frame img2img
|
||||
independently re-imagines each frame, producing a shimmering "boil" that reads as
|
||||
disorienting — the **opposite** of the piece's peaceful intent. This was caught
|
||||
in the POC and is now a named design constraint: *the Right substrate must be
|
||||
temporally coherent.*
|
||||
- **Stabilization: optical-flow keyframe propagation.** Fully stylize **keyframes**
|
||||
at a fixed interval; for in-between frames, **warp the previous stylized frame
|
||||
forward by optical flow** (so motion is continuous) and apply only a *light*
|
||||
diffusion refine. The flow warp removes the boil; periodic keyframes bound drift.
|
||||
(This is the EbSynth principle. Genuine `ebsynth`/`ezsynth` are NVIDIA/Windows-
|
||||
leaning and don't install cleanly on Apple Silicon, so the POC implemented the
|
||||
same idea directly with OpenCV Farneback flow + the existing diffusers pipeline.)
|
||||
|
||||
This stays consistent with §4.3's crucial invariant: **the Left analytical labels
|
||||
remain a runtime overlay, never baked into the restyled pixels** — the POC's Left
|
||||
HUD is composited deterministically on top, preserving the near-free i18n of §10.
|
||||
|
||||
### 1.1 Where each transform runs (updated §4.3 table)
|
||||
|
||||
| Pole | Operation | Where it runs | Measured cost (8 s, 1080p clip) |
|
||||
|---|---|---|---|
|
||||
| **Dark** | color grade | runtime, live on the Pi | ~2.4 s offline; live at runtime |
|
||||
| **Light** | color grade | runtime, live on the Pi | ~2.5 s offline; live at runtime |
|
||||
| **Left** | analytical overlay (HUD) | runtime, live on the Pi | ~2.2 s offline; live at runtime |
|
||||
| **Right** | local generative restyle + flow propagation | **pre-baked offline, locally** | ~2.7 min/restyle-strength |
|
||||
|
||||
The three deterministic axes are confirmed cheap enough to run **live**; only the
|
||||
Right restyle is pre-baked. A peaceful **deterministic** alternative for Right (soft
|
||||
edge-preserving smoothing + bloom, zero flicker by construction, ~5 s/clip) was
|
||||
prototyped and set aside — the operator preferred the true generative repaint once
|
||||
the flow stabilization made it calm. It remains a documented fallback.
|
||||
|
||||
### 1.2 Runtime label rendering & i18n (sharpens §10)
|
||||
|
||||
The Left analytical labels are drawn **live by the Pi as a 2D graphics overlay** —
|
||||
**not** baked into video, and specifically **not** a pre-rendered overlay *video*. A
|
||||
baked overlay would have to exist per language × per Left level × per scale, which
|
||||
re-introduces the "× number of languages" cost §10 exists to avoid. So the runtime
|
||||
path is:
|
||||
|
||||
- **Architecture.** Per base clip, an offline-authored **annotation track** (box
|
||||
positions, anchor points, which annotations appear at each Left level) referencing
|
||||
language-agnostic label **keys** (e.g. `detected.conifer`). Per language, a cheap
|
||||
**string table** (key → translated text) + font + TTS voice. At runtime the Pi
|
||||
reads the Left knob, selects the active annotations, **shapes** the current
|
||||
language's strings, and composites over the altered video — updating only on change
|
||||
(knob move, language switch, timeline cue), not every frame.
|
||||
- **Correctness needs a real shaping stack.** Rendering *every* language correctly
|
||||
(Arabic joining, Indic conjuncts, CJK, RTL) requires **Pango + HarfBuzz + Noto
|
||||
fonts**, not naïve text drawing. (The POC's HUD used ffmpeg `drawtext`/Menlo —
|
||||
Latin-only; it would mis-render complex scripts and is **not** the runtime path.)
|
||||
This shaping stack is the load-bearing requirement behind §10's "label things
|
||||
correctly."
|
||||
- **Feasibility.** This is OSD/subtitle-class compositing; a Pi 5 (VideoCore VII,
|
||||
GLES/Vulkan, hardware decode) handles it. Headroom at the *panoramic* resolution is
|
||||
the one unmeasured variable — see §6.
|
||||
|
||||
---
|
||||
|
||||
## 2. Content = a small NEUTRAL "scales of nature" library (refines §6, §8)
|
||||
|
||||
The operator's "cosmic zoom" concept (space → continents → birds → ocean → abyss →
|
||||
microscopic → galaxy) enters the piece **as a curatorial theme, not a fixed film.**
|
||||
|
||||
- **Structure:** a **small library (~4–6 to start) of calm, neutral base clips**,
|
||||
each drawn from a *different scale of nature* — e.g. an orbital Earth, a forest, a
|
||||
coral reef, the deep-sea abyss, the microscopic, the cosmos. The machine alters
|
||||
whichever clip is playing, exactly as for any neutral base.
|
||||
- **Why this and not a single stitched journey.** A literal galaxy→cell "how small
|
||||
we are" journey carries its **own** emotion (awe, cosmic insignificance) *before
|
||||
the machine acts*, which contradicts the §1 neutral-base thesis, and as a fixed
|
||||
film it becomes "a journey you watch" rather than "a reality you bend." Keeping the
|
||||
scales as the *theme of a neutral library* preserves the awe-of-scale richness and
|
||||
the piece's coherence **while keeping the base neutral and the experience
|
||||
interactive.** (Rejected alternatives: single stitched base; cosmic-zoom as
|
||||
intro/reset; rethinking the thesis.)
|
||||
- **Cost is not the constraint.** Per §4 below, ~5 base clips is ~1 hour of overnight
|
||||
local pre-bake — so this is an *artistic* choice, made on artistic grounds.
|
||||
- **Mechanism unchanged:** this slots into the existing §6 content model and the
|
||||
sub-project-2 ingest/tagging/review tooling; "scales of nature" is simply the
|
||||
selection principle for which neutral clips to source.
|
||||
|
||||
### 2.1 Strict-PD sourcing map (refines §8)
|
||||
|
||||
License stance is unchanged (prefer Public Domain / CC0; record license + source per
|
||||
clip). The "scales" theme maps onto genuinely public-domain pools cleanly at the
|
||||
*ends* and is softer in the terrestrial *middle*:
|
||||
|
||||
| Scale | Best strict-PD source | Status |
|
||||
|---|---|---|
|
||||
| Cosmos / galaxy / deep space | NASA, Hubble, JWST | 🟢 abundant, true PD |
|
||||
| Earth from orbit / continents | NASA / ISS | 🟢 true PD |
|
||||
| Ocean & **deep sea / abyss** (global) | **NOAA Ocean Exploration** | 🟢 true PD, worldwide |
|
||||
| Microscopic / single-celled | NIH / NSF | 🟢 thinner but PD |
|
||||
| US land / wildlife | NPS, USGS, USFWS | 🟢 true PD (US locations only) |
|
||||
| **Non-US terrestrial, high-flying birds** | — | 🟡 mostly CC-BY; the PD soft spot |
|
||||
|
||||
US-government works are public domain by statute (17 U.S.C. §105); the installation
|
||||
is US-based, so this is the cleanest possible legal footing. **Caveat the design
|
||||
must respect:** "free stock" sites (Pexels, Pixabay, Mitch Martinez's free 4K, etc.)
|
||||
are *royalty-free but NOT public domain* — they restrict redistribution and retain
|
||||
copyright. The ingest tool's "no explicit license → assume PD, verify before use"
|
||||
flag exists precisely for this trap and must not be trusted blindly.
|
||||
|
||||
---
|
||||
|
||||
## 3. Scale navigation & zoom transitions (new element; refines §2 selector, §11)
|
||||
|
||||
The scales-of-nature library is navigated as a **closed loop (a ring), not a line.**
|
||||
A dedicated **scale ("zoom") control** lets the viewer journey through scales;
|
||||
advancing it triggers a short **AI zoom/warp transition** to the next scale,
|
||||
pre-baked offline. Diving past the smallest (single-celled) **wraps around** to the
|
||||
largest (cosmos) — the infinite-zoom payoff that unifies micro and macro and makes
|
||||
the ring continuous.
|
||||
|
||||
- **The control is an *endless* rotary encoder — infinitely turnable, no end stops.**
|
||||
The form embodies the concept: a ring of scales has no beginning or end, so neither
|
||||
does the knob. Keep turning one way and you zoom inward forever (…reef →
|
||||
microscopic → **cosmos** → continents → …); turn back to zoom out. This sets it
|
||||
apart from the four **experience knobs**, which are *absolute* 0–4 pots: the zoom
|
||||
control reports **relative** rotation (encoder detents), and the player/firmware
|
||||
advances or retreats one ring-step per increment. It is distinct too from the §6
|
||||
content dial (audio/video channel) — it chooses *where in the ring* you are, while
|
||||
the knobs still bend whichever scale is present.
|
||||
- **Transitions:** between each adjacent pair of scale clips, a short (~few-second)
|
||||
generative morph — **first-last-frame-conditioned image-to-video** (Wan/LTX-class)
|
||||
or SD "infinite-zoom" outpainting for the literal zoom-through. Pre-baked offline,
|
||||
local; one clip per ring edge (N scales → N transitions, including the micro→cosmos
|
||||
closer). A fast spin may cross several scales — transitions chain, or past a speed
|
||||
threshold a faster blended pass is used.
|
||||
- **Implemented (session 0012):** `player.ring.advance_ring` takes an opt-in
|
||||
`fast_spin_threshold` (canonical default `DEFAULT_FAST_SPIN_THRESHOLD = 3`).
|
||||
A spin batches its detents into one `advance` call, so `abs(delta)` is the
|
||||
input layer's proxy for spin speed; at/above the threshold the move collapses
|
||||
to a **single blended pass** — one `TransitionStep` (the arrival edge, marked
|
||||
`blended`, landing straight on the destination, `RingMove.fast=True`) instead
|
||||
of chaining every transition. The policy is opt-in because the pure function
|
||||
cannot observe wall-clock speed — the simulator/firmware enables it. The
|
||||
simulator plays the blended step at 2.5× (`FAST_BLEND_RATE`); when real
|
||||
transition clips exist a dedicated fast-blend clip can replace the
|
||||
accelerated arrival-edge placeholder. Threshold + rate are tunable by eye.
|
||||
- **Thesis-safe:** dwells on a scale are the neutral, knob-altered interactive cores;
|
||||
transitions are fixed connective moments (un-altered, or at most carrying the
|
||||
current mood grade). The awe lives in the *movement between* scales, not the base.
|
||||
- **Heavier than the restyle, still bounded:** generative video synthesis costs more
|
||||
per second than img2img, but transitions are few and short — a handful of ~3–5 s
|
||||
morphs is an overnight local batch.
|
||||
|
||||
---
|
||||
|
||||
## 4. Economics update — local authoring ≈ free (refines §9)
|
||||
|
||||
§9 priced the Right axis at **~$300–3k** of cloud generative-v2v API. The POC
|
||||
collapses that: the restyle runs on **hardware the operator already owns**, offline,
|
||||
so the marginal cost is electricity.
|
||||
|
||||
- **Per base clip:** ~4 Right restyle strengths × ~2.7 min ≈ **~11 min of local AI
|
||||
pre-bake**, plus seconds for the deterministic Dark/Light/Left grid.
|
||||
- **Whole small library (~5 clips):** **~1 hour** of overnight batch rendering.
|
||||
- **Scale transitions:** ~N short generative-video morphs (one per ring edge), a few
|
||||
seconds each — a separate, heavier offline batch (video synthesis > img2img), but
|
||||
still overnight-local.
|
||||
- **Cloud API: no longer required** for the build. (It remains an option if a
|
||||
higher-quality video model than a local one is wanted for a final pass.)
|
||||
|
||||
This also tightens the piece's "all-local" ethos: not just *runtime* is local
|
||||
(§1 of the prior design) — now *authoring* is too.
|
||||
|
||||
---
|
||||
|
||||
## 5. POC evidence (this session)
|
||||
|
||||
A throwaway spike (outside the repo, `~/hef-poc/`) validated the full engine on one
|
||||
real nature clip (a 4K Yosemite waterfall, trimmed to 8 s @ 1080p):
|
||||
|
||||
- **All four axes rendered** and read as distinct: Dark (cold/somber), Light
|
||||
(warm/serene), Left (analytical HUD overlay), Right (painterly).
|
||||
- **Right per-frame:** ~3.4 min/8 s clip, **flickers badly** (disqualifying).
|
||||
- **Right flow-propagated:** ~2.7 min/8 s clip, **calm** (operator-approved).
|
||||
- **Deterministic axes:** ~2.4 s each, ~3× faster than real-time → confirmed
|
||||
runtime-capable.
|
||||
- **Stack:** `imageio-ffmpeg`, `diffusers` + `sd-turbo` on MPS, OpenCV Farneback
|
||||
flow; Python 3.13; 64 GB unified memory comfortably ran models that OOM consumer
|
||||
GPUs.
|
||||
|
||||
---
|
||||
|
||||
## 6. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Flow quality at scale:** the OpenCV flow propagation was validated on one short
|
||||
clip; longer clips / faster motion may need shorter keyframe intervals, bidirectional
|
||||
blending, or a stronger flow model (RAFT).
|
||||
- **Painterly strength:** the POC kept the restyle gentle; the dreamlike *range* and
|
||||
the per-axis restyle-strength count (the §4.3 "small set of variants") are
|
||||
unfixed.
|
||||
- **Base-clip sourcing:** select and ingest the actual ~4–6 strictly-PD neutral
|
||||
scale clips (NASA/NOAA/NPS) via sub-project 2.
|
||||
- **Model choice:** `sd-turbo` was the POC's speed pick; a higher-quality local
|
||||
model (or a final cloud pass) may be worth a comparison for the shipped variants.
|
||||
- **Scale transitions:** generation method (first-last-frame i2v vs. infinite-zoom
|
||||
outpainting) and local model; per-transition length; whether transitions carry the
|
||||
current mood grade; the ring ordering of the scales; and behavior on fast or
|
||||
continuous spins of the endless encoder (chain transitions vs. blended skip).
|
||||
- **Pi compositing headroom:** confirm the Pi 5 can decode the altered video **and**
|
||||
render the live Pango/HarfBuzz label overlay at the *actual* panoramic projector
|
||||
resolution (ultra-wide / high-res) — low risk but unmeasured (see §1.2).
|
||||
|
||||
## 7. Out of scope (YAGNI)
|
||||
|
||||
- A single continuous **one-take** zoom through *all* scales (we use discrete neutral
|
||||
clips joined by short AI transitions on a navigable ring — not one unbroken shot).
|
||||
- Audio-axis alteration (already deferred by the prior §6).
|
||||
- Cloud rendering pipeline (local supersedes it for the base build).
|
||||
@@ -1,85 +0,0 @@
|
||||
# Affect channel — emotions in the Left-brain HUD
|
||||
|
||||
**Status:** approved (session 0013, 2026-06-22)
|
||||
**Surface:** simulator preview (`simulator/`, `player/alteration.py`)
|
||||
**Builds on:** the machine-altered-perception design SPEC; the Left HUD look-and-feel
|
||||
pass (same session).
|
||||
|
||||
## Idea
|
||||
|
||||
A third HUD channel beside the cyan *detections* and amber *measurements*: soft,
|
||||
glowing **emotion-words** that surface the feelings the experience may invoke.
|
||||
They appear only when *both* the analytical (Left) and dreamlike (Right) knobs are
|
||||
up — the dream leaking into the machine's reading. This is the uncanny edge of the
|
||||
thesis: the machine trying to quantify not just what is there, but how you are
|
||||
meant to feel.
|
||||
|
||||
## Behaviour
|
||||
|
||||
- **Trigger / intensity.** Affect strength = `min(left, right)` (0–4). Either knob
|
||||
at 0 → no emotions at all. Opacity scales with that strength × `overlay_gain`.
|
||||
- **Escalation.** A **stable emotional palette per scene** — higher combined
|
||||
strength reveals *more* words at higher opacity. Each word carries a `min_level`
|
||||
(the same mechanic the detection channel uses), but compared against the
|
||||
*combined* `min(left, right)` strength rather than Left alone.
|
||||
- **Rendering.** Floating words at authored scene positions — **no boxes or
|
||||
reticles**, lowercase, semi-transparent with a soft glow, in a distinct **violet**
|
||||
register so they read as affective and clearly *softer* than the hard clinical
|
||||
reticles.
|
||||
|
||||
## Where the math lives
|
||||
|
||||
Consistent with the existing split (alteration math in Python, label-selection in
|
||||
the client):
|
||||
|
||||
- `player/alteration.py` gains an `AffectOverlay(strength, intensity)` on the
|
||||
`RenderPlan`. `strength = min(left, right)`; `intensity = clamp(overlay_gain *
|
||||
min(left,right) / KNOB_MAX, 0, 1)`. Returned in `render_plan_to_dict` under
|
||||
`"affect"`. Pure, unit-tested.
|
||||
- The simulator client picks *which* words to show by comparing each affect
|
||||
entry's `min_level` to `plan.affect.strength`, and renders them at
|
||||
`plan.affect.intensity` opacity.
|
||||
|
||||
`is_identity` is unaffected: `min(left,right) > 0` implies `left > 0`, which already
|
||||
makes `overlay.level > 0`.
|
||||
|
||||
## Data
|
||||
|
||||
New optional per-clip `affect` list in the manifest, parallel to `annotations`:
|
||||
|
||||
```json
|
||||
"affect": [
|
||||
{"key": "feel.awe", "at": [0.5, 0.4], "min_level": 1},
|
||||
{"key": "feel.serenity", "at": [0.2, 0.62], "min_level": 2}
|
||||
],
|
||||
"strings": {"en": {"feel.awe": "awe", "feel.serenity": "serenity"}}
|
||||
```
|
||||
|
||||
`at` is a normalized `[x, y]` anchor point (not a box — feelings are scene-level).
|
||||
The words live in the existing `strings.en` map. `Clip` gains an `affect` field
|
||||
(defaulting to `[]`) threaded through `_clip_from_dict` and `to_dict`.
|
||||
|
||||
Starter palettes (words are art-direction, easily tuned):
|
||||
|
||||
| scale | palette (min_level 1 → 4) |
|
||||
|--------|-----------------------------------------------|
|
||||
| forest | awe · serenity · reverence · stillness |
|
||||
| cosmos | wonder · vastness · insignificance · longing |
|
||||
| abyss | unease · fascination · isolation · dread |
|
||||
|
||||
All three scales carry the data; forest is fully realized (it has the real Right
|
||||
variants).
|
||||
|
||||
## Testing
|
||||
|
||||
- Python unit tests: affect `strength = min(left, right)` across knob combos;
|
||||
intensity scaling and `overlay_gain`; `0` when either knob is 0; presence in the
|
||||
serialized plan.
|
||||
- By-eye in the live simulator (forest, Left & Right both up) — the established
|
||||
judge-by-eye loop for this surface.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- No drift-toward-unsettling arc (rejected in favour of a stable palette).
|
||||
- No per-object emotion tags; no animated drift (words are statically placed for
|
||||
v1; gentle motion can come later if wanted).
|
||||
@@ -1,109 +0,0 @@
|
||||
# Right axis as a real-time deterministic dream
|
||||
|
||||
**Status:** built (session 0013, 2026-06-22) — Phase 1 superseded by Phase 2 in the
|
||||
same session after by-eye review (the haze "looked like blurred video, not stylized").
|
||||
The shipped Right dream is the **Phase 2 painterly** path.
|
||||
**Surface:** simulator preview (`simulator/`, `player/alteration.py`)
|
||||
**Supersedes:** the "discrete pre-baked, flow-stabilized restyle variant" definition
|
||||
of the Right axis in the machine-altered-perception design SPEC §4.1 — a deliberate
|
||||
change to a core decision (see Why).
|
||||
|
||||
## Why
|
||||
|
||||
The pre-baked SD img2img variants (`bake_right_variants.py`) are **too fluid**: the
|
||||
painting re-rolls every keyframe (~2 s) and the optical-flow tween warps frames into
|
||||
each other, so across the loop the dream churns and melts. The right brain should be
|
||||
a **still altered state** — a calm, holistic counterpart to the busy, accreting
|
||||
analytical Left — not a restless morph.
|
||||
|
||||
## Concept
|
||||
|
||||
The Right knob no longer selects a pre-baked variant. It drives a **deterministic,
|
||||
real-time filter** applied live to the base footage, scaled 0→4. The same filter runs
|
||||
every frame, so the look is **stable by construction** — locked to scene content,
|
||||
moving only where the footage already moves. Bonus: no baked variant videos to ship,
|
||||
which simplifies the eventual hardware (Pi) story.
|
||||
|
||||
Staged delivery (both built this session; Phase 2 is what shipped):
|
||||
- **Phase 1: luminous haze** — soft-focus, blooming highlights, pastel desaturation.
|
||||
CSS + a bloom layer. Built, then **retired** by eye: blur reads as *out-of-focus
|
||||
video*, not stylized, and softens the base's crisp motion.
|
||||
- **Phase 2: painterly flatten (shipped)** — a WebGL2 **Kuwahara** shader on a
|
||||
`<canvas>` restyling the live video frames. Edge-preserving, so motion/structure
|
||||
stay as crisp as the source — the dream is the same footage, just stylized. No blur.
|
||||
|
||||
## Phase 2 — the shipped painterly dream
|
||||
|
||||
- **Renderer:** `<canvas id="paint">` over `<video>`; a `requestAnimationFrame` loop
|
||||
uploads each video frame to a texture and runs the Kuwahara fragment shader (radius
|
||||
`R=6`). `intensity = 0` ⇒ passthrough (raw base); ring transitions render passthrough
|
||||
(`busy`). WebGL-less fallback: canvas hidden, mood grade applied to `#vid`.
|
||||
- **Painterly:** `mix(base, kuwahara, intensity)` — flatten into painted regions, edges
|
||||
preserved.
|
||||
- **"Trippy" at max** (operator-tuned by eye): the psychedelic terms ramp with
|
||||
`t = intensity²` so low/mid Right stays gently dreamy and only **max** goes
|
||||
psychedelic — vivid saturation (`×(1 + 1.1·t)`), a surreal **hue drift** (~45° at max),
|
||||
and **posterize** banding (`mix(255, 6, t)` levels), plus a slight luminous lift.
|
||||
- **Mood grade** (Dark/Light) rides as a CSS filter on the canvas; `#tint` unchanged.
|
||||
- The Python plan is unchanged — Phase 2 is a client-render swap; `dream.intensity`
|
||||
drives the shader.
|
||||
|
||||
## Phase 1 — components
|
||||
|
||||
All scale with the Right strength `s = intensity` (0→1):
|
||||
|
||||
- **soft-focus** — gentle blur, ~`1.5·s` px.
|
||||
- **pastel** — `saturate(1 − 0.35·s)`, `contrast(1 − 0.15·s)` (lift shadows / flatten).
|
||||
- **luminous** — `brightness(1 + 0.08·s)`.
|
||||
- **bloom** — a second, synced copy of the video, blurred + brightened and
|
||||
`screen`-blended over the main, opacity `~0.6·s` — highlights glow and bleed.
|
||||
|
||||
Starting coefficients are by-eye seeds; the look is tuned live in the sim.
|
||||
|
||||
## Engine — `player/alteration.py`
|
||||
|
||||
- Replace `Restyle(variant)` with `Dream(strength, intensity)`:
|
||||
`strength = right` (0–4, discrete), `intensity = clamp(dream_gain · right / KNOB_MAX,
|
||||
0, 1)`.
|
||||
- `Calibration` gains `dream_gain: float = 1.0`; `DEFAULT_CALIBRATION` sets it 1.0.
|
||||
- `render_plan_to_dict`: `"restyle": {"variant"}` → `"dream": {"strength", "intensity"}`.
|
||||
- `is_identity`: Right contributes when `strength > 0` (replaces the
|
||||
`restyle.variant == 0` clause).
|
||||
- Pure + unit-tested. The client maps `dream.intensity` → CSS params, mirroring how
|
||||
`grade.tone` → video filter is mapped client-side today.
|
||||
|
||||
## Rendering — simulator
|
||||
|
||||
- **Remove** the variant-crossfade machinery (`loadVariant`, `variantFile`, the
|
||||
`right_variants` / `currentVariant` handling). The main video always plays
|
||||
`base_file`.
|
||||
- **Compose one video filter** from *both* the mood grade (`grade.tone`) and the dream
|
||||
(`dream.intensity`) — today `applyGrade` owns `vid.style.filter`; it becomes a single
|
||||
`applyVideoLook(tone, dreamIntensity)` so the two don't clobber each other.
|
||||
- **Add a bloom layer**: a second `<video id="bloom">` (same `base_file`, muted,
|
||||
looped, started in sync), `screen`-blended, blurred/brightened, opacity from
|
||||
`dream.intensity`. Both videos' `src` are set together on scale settle / ring move.
|
||||
- `#tint` (the Dark cool wash) is unchanged.
|
||||
|
||||
## Migration
|
||||
|
||||
- The baked SD variants (`*/right*.mp4`) and `bake_right_variants.py` become **legacy:
|
||||
parked in-repo, not deleted, unused at runtime**. `right_variants` stays in the
|
||||
manifest/`clips.py` (ignored by the client) to avoid churn; it can be removed later.
|
||||
- The scale ring and its transitions are untouched.
|
||||
- The affect channel is unaffected — Right is still a 0–4 knob, so `min(left, right)`
|
||||
emotions keep working.
|
||||
|
||||
## Testing
|
||||
|
||||
- Python unit tests: `Dream.strength = right`, `intensity` scaling + `dream_gain`,
|
||||
clamping, `0` at `right=0`, presence/shape in the serialized plan, `is_identity`.
|
||||
- By-eye in the live sim (forest, Right up, then Left+Right for the whole picture) —
|
||||
the established judge-by-eye loop. Success = the dream holds still across the loop;
|
||||
only real motion (the waterfall) moves.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Removing the legacy baker / baked media (left parked, unused at runtime).
|
||||
- Any change to the scale ring, transitions, Left HUD, or affect channel.
|
||||
- A Pi-GPU performance pass on the Kuwahara shader (desktop sim is the current target).
|
||||
@@ -1,248 +0,0 @@
|
||||
# HEF — Content Pipeline (source → process → label → manifest)
|
||||
|
||||
**Date:** 2026-06-24
|
||||
**Status:** Graduated — operator-approved (session 0014); **Increment 1 built & merged** (PR #13). Increment 2 (hybrid track pass + author mode) + real PD sourcing pending.
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Anchor:** `docs/ROADMAP.md` sub-project 3 (Player Runtime) content needs; the
|
||||
spiritual successor to sub-project 2 (Ingest & Tagging tools), retargeted from the
|
||||
retired coordinate-catalog model to the manifest content model.
|
||||
**Refines:** [`2026-06-07-scales-library-and-right-axis-pipeline-design.md`](./2026-06-07-scales-library-and-right-axis-pipeline-design.md)
|
||||
§2 (scales library), §2.1 (strict-PD sourcing), §6 (open sourcing questions).
|
||||
**Builds on:** [`2026-06-22-right-axis-deterministic-dream-design.md`](./2026-06-22-right-axis-deterministic-dream-design.md)
|
||||
(the real-time dream — **no per-clip ML bake**) and
|
||||
[`2026-06-22-affect-channel-hud-design.md`](./2026-06-22-affect-channel-hud-design.md)
|
||||
(scene-anchored affect words).
|
||||
|
||||
---
|
||||
|
||||
## 1. Why this exists
|
||||
|
||||
The installation has an alteration engine, a scale ring, a Left analytical HUD, an
|
||||
affect channel, and a real-time Right "dream" — but it has **no real content**. The
|
||||
three ring scales today are: `cosmos` and `abyss` carrying *placeholder bytes* (PD
|
||||
provenance recorded, but the media is synthetic), and `forest` carrying a *POC
|
||||
sample* (a trimmed Yosemite clip licensed "look-tuning only; not shipped content").
|
||||
There is no repeatable way to turn a public-domain source clip into a shippable,
|
||||
seamless-looping, labelled neutral base.
|
||||
|
||||
This design defines that pipeline: **source → frame → seamless-loop → transcode →
|
||||
label/author → manifest entry.** One big anchor fact reshapes it versus the old
|
||||
plan — the Right dream is now a **real-time deterministic shader** (session 0013),
|
||||
so there is **no per-clip ML restyle bake** anymore. The pipeline's offline cost is
|
||||
therefore **transcode + human authoring**, not generative-video baking. The one
|
||||
deliberate exception is the motion-tracking label pass chosen this session (§3),
|
||||
which re-introduces a bounded offline compute step — by choice, for realism.
|
||||
|
||||
## 2. Decisions settled this session
|
||||
|
||||
| # | Decision | Choice | Consequence |
|
||||
|---|----------|--------|-------------|
|
||||
| 1 | Scale set | **Lean 5** — cosmos · orbit · forest/land · reef · abyss (microscopic dropped) | Ring wraps **abyss → cosmos**; replaces the POC forest + both placeholder bases with real strict-PD footage |
|
||||
| 2 | Seamless loop | **Crossfade loop** (tail→head overlap) | Any source clip becomes an indefinite seamless dwell loop; a transcode-time step |
|
||||
| 3 | Label placement | **Motion-tracked, hybrid** | ML proposes tracks where it can; hand-seed + classical tracker elsewhere. Re-introduces an offline pass + an authoring step. Semantics always hand-authored |
|
||||
| 4 | Format | **Master + 1080p proxy** | High-quality master retained per clip; a 1080p H.264 proxy drives the sim; Pi chooses later |
|
||||
|
||||
Microscopic was dropped because PD microscopy *video* (vs stills) is the thinnest
|
||||
pool and hardest to source well; it can be added later by re-running the pipeline.
|
||||
|
||||
## 3. The pipeline — six stages
|
||||
|
||||
Each stage is a deterministic unit with one job (handbook §4.2: deterministic tools,
|
||||
not prose an agent re-derives). Stages 1–4 and 6 are mechanical; stage 5 is the one
|
||||
human-in-the-loop step.
|
||||
|
||||
```
|
||||
(1) source ──▶ (2) frame ──▶ (3) loop ──▶ (4) transcode ──▶ (6) manifest
|
||||
provenance trim/crop crossfade master + proxy entry
|
||||
& scale seam │ ▲
|
||||
└──▶ (5) label/author (hybrid track)
|
||||
```
|
||||
|
||||
### 3.1 Stage 1 — source & provenance
|
||||
Acquire the strict-PD source clip and record license + source URL + capture notes.
|
||||
Reuse sub-project-2 fetchers where they map (a NASA fetcher already exists); add
|
||||
NOAA Ocean Exploration / NPS / USGS fetchers (or accept a manual download + a
|
||||
provenance record). The "no explicit license → assume PD, verify" trap from the
|
||||
scales design §2.1 applies: **"free stock" (Pexels/Pixabay) is NOT public domain.**
|
||||
Output: the raw source file + a provenance record (license, source, url, fetched_at).
|
||||
|
||||
### 3.2 Stage 2 — frame (trim / crop / scale)
|
||||
Pick the in/out points; crop to the target aspect; scale toward the format target
|
||||
(§6). The panoramic projector aspect is still unmeasured (open question), so v1
|
||||
frames to a standard wide 16:9 for the sim; the master retains maximum available
|
||||
width so a later pano crop/extension has pixels to work with. Output: a framed,
|
||||
silent (audio stripped — bases are video-only; audio is the separate content-dial
|
||||
channel) intermediate.
|
||||
|
||||
### 3.3 Stage 3 — seamless crossfade loop
|
||||
Make the clip loop indefinitely with no visible seam by overlapping the tail into
|
||||
the head with a short crossfade (~0.5–1.0 s). The final looped clip is shorter than
|
||||
the source by the overlap; during the overlap region motion is a blend (acceptable
|
||||
for calm ambient nature footage). This is the chosen loop strategy (clean-cut and
|
||||
palindrome were rejected — seam risk and reversed motion respectively). Output: a
|
||||
loop-clean clip whose first and last frames join seamlessly.
|
||||
|
||||
### 3.4 Stage 4 — transcode (master + proxy)
|
||||
Emit two encodes per clip (§6 for exact params): a **master** (high quality, up to
|
||||
4K, retained as the archival/Pi-candidate source) and a **1080p H.264 proxy** (what
|
||||
the sim plays today, matching current `base.mp4` behaviour). Both video-only,
|
||||
`yuv420p`, even dimensions, `+faststart`.
|
||||
|
||||
### 3.5 Stage 5 — label & author (the hybrid motion-track pass)
|
||||
The only human-in-the-loop stage, and the only one that adds offline ML compute.
|
||||
|
||||
- **Semantics are always hand-authored.** A generic detector cannot produce the
|
||||
installation's semantic keys — "spiral galaxy", "siphonophore", `z ≈ 0.04`,
|
||||
`~2.1 m³/s` are out-of-distribution. The author owns the label **keys**,
|
||||
**strings**, and `min_level`; tracking only propagates **box geometry**.
|
||||
- **Hybrid geometry (decision 3).** Where a model can lock onto a subject (a fish
|
||||
on the reef, an animal in the forest), an ML detect+track pass (YOLO+ByteTrack /
|
||||
SAM2-class, lazy-imported like the parked baker imported torch) proposes a track
|
||||
the author maps to a key. Where it can't (cosmos, abyss, microscopic-type
|
||||
content), the author **hand-seeds** a box on a keyframe and a **classical tracker**
|
||||
(OpenCV CSRT / optical-flow) propagates it; the author fixes drift. Both paths
|
||||
emit the same artifact.
|
||||
- **Channel rules.** `detected.*` object labels **track**; `measure.*` readouts and
|
||||
`feel.*` affect words stay **static / scene-anchored** (a depth readout or "awe"
|
||||
is not pinned to a moving pixel). This matches the affect-channel design (affect
|
||||
is scene-level) and keeps measurement chips legible.
|
||||
- Output: an **annotation track** per object label (§4) plus the static measurement
|
||||
and affect placements, all keyed and stringed.
|
||||
|
||||
### 3.6 Stage 6 — manifest assembly
|
||||
Write the clip entry (base/proxy/master paths, license, source, annotations with
|
||||
tracks, affect, strings) into `simulator/sample_media/manifest.json`, and register
|
||||
the scale in the `ring` section with its transition edges (§5). Idempotent: stage 6
|
||||
owns the manifest entry so re-running earlier stages never clobbers authored labels
|
||||
(mirrors how `setup_sample_media.py` was made to never clobber a real bake).
|
||||
|
||||
## 4. Data model — annotation tracks
|
||||
|
||||
Today an annotation is a single static box: `{"key", "box":[x,y,w,h], "min_level"}`.
|
||||
Add an **optional keyframed track**; static labels are unchanged (backward
|
||||
compatible):
|
||||
|
||||
```json
|
||||
{
|
||||
"key": "detected.fish",
|
||||
"min_level": 1,
|
||||
"track": [
|
||||
{"t": 0.0, "box": [0.30, 0.40, 0.10, 0.08]},
|
||||
{"t": 0.5, "box": [0.52, 0.38, 0.10, 0.08]},
|
||||
{"t": 1.0, "box": [0.30, 0.40, 0.10, 0.08]}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `t` is **normalized 0→1 over the looped clip** — resolution- and fps-independent,
|
||||
so the same track is correct for both master and proxy and survives a re-transcode.
|
||||
(A track that loops cleanly has matching boxes at `t=0` and `t=1`.)
|
||||
- **Runtime:** if `track` is present, the client interpolates the box at the current
|
||||
loop-normalized playback time (linear between keyframes); otherwise it uses the
|
||||
static `box`. This lives **client-side**, consistent with the existing split
|
||||
(alteration math in Python; label selection/placement in the client). `Clip` gains
|
||||
the track only as data inside `annotations`; `player/alteration.py` is **unchanged**
|
||||
(labels are not part of `RenderPlan`).
|
||||
- `measure.*` keep a single static `box`; `feel.*` keep a single static `at:[x,y]`.
|
||||
|
||||
## 5. The 5 scales, sources, and ring
|
||||
|
||||
| Scale | Strict-PD source | Notes |
|
||||
|-------|------------------|-------|
|
||||
| `cosmos` | NASA / Hubble / JWST | 🟢 abundant true PD (17 U.S.C. §105); replaces placeholder |
|
||||
| `orbit` | NASA / ISS (Earth from orbit) | 🟢 true PD; **new** scale |
|
||||
| `forest` | NPS / USGS (US land/wildlife) | 🟢 true PD (US locations); replaces the non-shippable POC sample |
|
||||
| `reef` | NOAA Ocean Exploration (shallow) | 🟢 true PD worldwide; **new** scale |
|
||||
| `abyss` | NOAA Ocean Exploration (deep sea) | 🟢 true PD worldwide; replaces placeholder |
|
||||
|
||||
**Ring order (large → small, zooms inward; wraps small → large):**
|
||||
`cosmos → orbit → forest → reef → abyss → (wrap) cosmos`. Five scales → five ring
|
||||
edges. The wrap seam is now **abyss → cosmos** (deep sea back out to the cosmos) —
|
||||
the infinite-zoom payoff, minus the microscopic step.
|
||||
|
||||
**Transitions** between the new adjacent edges remain the **deferred i2v generative
|
||||
work** (scales design §3 / §6) — they need a generative-video model and two real
|
||||
adjacent bases, heavier than anything in this pipeline. Until then new edges carry
|
||||
**placeholder transitions** (the `setup_scales_media.py` ffmpeg `xfade=zoomin`
|
||||
generator already produces these), exactly as cosmos/forest/abyss do today. This
|
||||
pipeline's job is the **bases + labels**; transitions are tracked but out of scope
|
||||
here (§8).
|
||||
|
||||
## 6. Format target
|
||||
|
||||
| | Master | Proxy (sim) |
|
||||
|---|--------|-------------|
|
||||
| Resolution | source, capped ~3840 wide | 1920×1080 (fit wide) |
|
||||
| Codec | H.264 High | H.264 High |
|
||||
| Quality | CRF ~18 | CRF ~20 |
|
||||
| FPS | preserve source (cap 30) | 30 |
|
||||
| Pixel fmt | `yuv420p`, even dims | `yuv420p`, even dims |
|
||||
| Audio | none (stripped) | none (stripped) |
|
||||
| Flags | `+faststart` | `+faststart` |
|
||||
|
||||
The sim plays the **proxy** as today's `base_file`. The **master** is retained per
|
||||
clip for the eventual Pi/pano encode once the panoramic resolution is measured.
|
||||
Browser HEVC support is too spotty to make HEVC the sim path, which is why the proxy
|
||||
stays H.264 (the master can be re-encoded to HEVC for the Pi later without
|
||||
re-sourcing).
|
||||
|
||||
## 7. Tooling & where it lives
|
||||
|
||||
- **New `tools/pipeline/` package** — deterministic ffmpeg-backed verbs for stages
|
||||
1–4 and 6 (`fetch`/provenance, `frame`, `loop`, `transcode`, `manifest`), reusing
|
||||
sub-project-2 fetchers where valid. This **supersedes** the coordinate-era
|
||||
ingest *drafting* (the `left/right/dark/light` Record model retired with the
|
||||
machine-altered-perception pivot); the pipeline targets the **manifest**, not
|
||||
catalog `Record`s.
|
||||
- **Stage 5 tracking** — `tools/pipeline/track.py`: classical OpenCV tracker
|
||||
(always available) + an optional lazy-imported ML detect+track path. Emits
|
||||
candidate keyframed tracks for the author to accept/correct.
|
||||
- **Authoring surface** — extend the **simulator** with an **author mode** (it
|
||||
already renders the exact HUD overlay and is the by-eye judge): scrub the clip,
|
||||
draw/seed/adjust boxes, run the tracker, assign keys/`min_level`/strings, place
|
||||
affect anchors, write the manifest entry. Reusing the preview as the editor keeps
|
||||
"what you author is what plays." **This is the largest new piece** — see the
|
||||
delivery order (§8), which lets v1 prove the schema before the full editor exists.
|
||||
|
||||
## 8. Delivery / rollout order
|
||||
|
||||
Build order within this Feature (Solution-Design rollout strategy — not a task
|
||||
list):
|
||||
|
||||
1. **Increment 1 — real footage, no ML.** Stages 1–4 + 6 + the §4 track schema +
|
||||
client interpolation. Source the 5 strict-PD bases, crossfade-loop them, emit
|
||||
master+proxy, wire them into the ring. Author a *couple* of tracks **by hand as
|
||||
JSON** to prove the track schema and the runtime interpolation. Outcome: the
|
||||
whole ring is **real, seamless-looping footage** judgeable by eye — the
|
||||
highest-value, lowest-risk step, and it stands alone.
|
||||
2. **Increment 2 — the hybrid track pass + author mode.** `track.py` (classical +
|
||||
optional ML) and the simulator author mode; label all five scales properly.
|
||||
|
||||
**Deferred (tracked, not built here):** the real **i2v ring transitions** for the
|
||||
new edges (need a generative-video model + adjacent real bases); the Pi renderer +
|
||||
serial/firmware (per the simulator-first directive); the panoramic-resolution
|
||||
master encode (waits on a measured pano spec).
|
||||
|
||||
> Scope note: if Increment 2's author mode grows large enough to be its own
|
||||
> increment in practice, split it at execution time. The design is one coherent
|
||||
> pipeline; the increments above are its sensible build order, not separate
|
||||
> Features.
|
||||
|
||||
## 9. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Panoramic resolution/aspect** — unmeasured (scales design §6). Drives the master
|
||||
crop/extension strategy; v1 frames 16:9 for the sim and retains max width.
|
||||
- **ML model choice** for the hybrid track path (YOLO+ByteTrack vs SAM2-class) and
|
||||
whether it earns its place given how few scales it can usefully fire on.
|
||||
- **Classical tracker drift** on long/fast clips — keyframe interval, re-seed cadence.
|
||||
- **Source fetchers** — which of NOAA / NPS / USGS get real fetchers vs manual
|
||||
download + provenance record for v1.
|
||||
- **Crossfade overlap length** per scale (slow cosmos vs busier reef) — by eye.
|
||||
|
||||
## 10. Out of scope (YAGNI)
|
||||
|
||||
- Generative i2v **ring transitions** for the new edges (deferred; placeholders hold).
|
||||
- **Audio** sourcing/alteration (separate content-dial channel; already deferred).
|
||||
- The **Pi/pano** encode + renderer and serial/firmware (simulator-first).
|
||||
- Reviving the retired coordinate-catalog `Record` drafting.
|
||||
- Per-frame **dense** tracks (sparse keyframed tracks interpolated at runtime instead).
|
||||
+39
-125
@@ -1,19 +1,22 @@
|
||||
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
|
||||
|
||||
Right axis (reframed session 0013): a DETERMINISTIC, real-time dream applied live
|
||||
to the base footage — a STILL altered state, not the old pre-baked SD variant that
|
||||
churned across the loop (Right-axis dream design,
|
||||
docs/superpowers/specs/2026-06-22-right-axis-deterministic-dream-design.md). The
|
||||
Left axis carries its knob LEVEL so a runtime annotation track can pick which
|
||||
labels show, and a frozen `Calibration` parameterizes the knob->strength gains so
|
||||
they can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION.
|
||||
Replaces the 2026-06-04 nearest-match *selection* with a *transformation* of a
|
||||
neutral base clip. Given the four experience knobs, it produces three layers
|
||||
that compose per §4.2:
|
||||
|
||||
Layers compose per §4.2:
|
||||
- Substrate: ColorGrade (Dark/Light mood, center = identity §5) + the Right Dream
|
||||
(deterministic luminous haze, applied client-side from `dream.intensity`).
|
||||
- Overlay: AnalyticalOverlay (Left) + the affect channel, composited on top.
|
||||
Left and Right stack (different layers); Dark/Light are the two poles of one
|
||||
mood grade.
|
||||
- Substrate transforms (alter pixels, blend with each other):
|
||||
* Restyle -- the Right axis: a pre-baked generative v2v dreamlike restyle.
|
||||
* ColorGrade -- the mood axis (Dark/Light): a deterministic color grade.
|
||||
- Overlay (composited on top of the substrate):
|
||||
* AnalyticalOverlay -- the Left axis: HUD/labels/measurement.
|
||||
|
||||
Left and Right are NOT opposites; they live on different layers and stack
|
||||
(§4.2). Dark and Light are the two poles of one mood grade whose center is the
|
||||
identity (§5).
|
||||
|
||||
Calibration note: the knob->strength curves below are the single source of
|
||||
truth for how a 0..4 position maps to a transform strength. See the session
|
||||
0006 transcript Deferred decisions for the §3-vs-§4.2/§5 reconciliation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -25,45 +28,6 @@ from hef.selection import Coordinate
|
||||
KNOB_MAX = 4 # knob full-scale (0..4)
|
||||
|
||||
|
||||
def _clamp(x: float, lo: float, hi: float) -> float:
|
||||
return max(lo, min(hi, x))
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Calibration:
|
||||
"""Tunable knob->strength curves (settled by eye in the sim, then baked).
|
||||
|
||||
- mood_gain: scales the signed Dark/Light tone (result clamped to [-1, 1]).
|
||||
- overlay_gain: scales the Left overlay (and affect) intensity (clamped [0, 1]).
|
||||
- dream_gain: scales the Right deterministic-dream intensity (clamped [0, 1]).
|
||||
"""
|
||||
|
||||
mood_gain: float = 1.0
|
||||
overlay_gain: float = 1.0
|
||||
dream_gain: float = 1.0
|
||||
|
||||
|
||||
# The LOCKED calibration (session 0010, 2026-06-07) — settled by eye in the
|
||||
# simulator, closing the open session-0006 knob->strength decision. Convention:
|
||||
# every experience knob runs 0 = off .. 4 = max, and equal Dark/Light = identity
|
||||
# (raw footage); there is no "centered at 2 = no push" coordinate.
|
||||
# - mood_gain = 1.0: full Dark/Light knob reaches the full mood grade. By-eye
|
||||
# evidence (POC renders + sim, after the dark-grade fix this session) shows
|
||||
# full tilt is peaceful on every axis, so no softening is warranted.
|
||||
# - overlay_gain = 1.0: full Left = opacity 1.0; the HUD is legible, not
|
||||
# overwhelming, sitting above the grade.
|
||||
# - dream_gain = 1.0: full Right = dream intensity 1.0; the deterministic
|
||||
# luminous haze (Right-axis dream design, session 0013) reaches full strength.
|
||||
# These are unity by deliberate choice, not as placeholders. The gains stay
|
||||
# parameterized so a different feel is one edit away; test_default_calibration_is_locked
|
||||
# guards the values from drifting silently.
|
||||
DEFAULT_CALIBRATION = Calibration(
|
||||
mood_gain=1.0,
|
||||
overlay_gain=1.0,
|
||||
dream_gain=1.0,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ColorGrade:
|
||||
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
|
||||
@@ -78,36 +42,18 @@ class ColorGrade:
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AnalyticalOverlay:
|
||||
"""Left axis (§4.1): analytical HUD/annotation, composited on top at runtime.
|
||||
`level` is the Left knob (0..4); a runtime annotation track uses it to pick
|
||||
which labels appear. `intensity` 0..1 is the overlay opacity/strength."""
|
||||
"""Left axis (§4.1): analytical HUD/annotation/labels, composited on top.
|
||||
`intensity` 0..1 (0 = no overlay)."""
|
||||
|
||||
level: int
|
||||
intensity: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AffectOverlay:
|
||||
"""Affect channel (Left x Right): emotion-words surfaced only when BOTH the
|
||||
analytical (Left) and dreamlike (Right) knobs are up. `strength` is
|
||||
`min(left, right)` (0..4) — a runtime affect track picks which feeling words
|
||||
appear by it; `intensity` 0..1 is their opacity. The dream leaking into the
|
||||
machine's reading (affect-channel design, session 0013)."""
|
||||
class Restyle:
|
||||
"""Right axis (§4.1): pre-baked generative v2v dreamlike substrate.
|
||||
`blend` 0..1 (0 = raw substrate, no restyle)."""
|
||||
|
||||
strength: int
|
||||
intensity: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Dream:
|
||||
"""Right axis: a deterministic, real-time dream applied live to the base
|
||||
footage — a STILL altered state (Right-axis dream design, session 0013),
|
||||
NOT the old pre-baked SD variant that churned across the loop. `strength` is
|
||||
the Right knob (0..4); `intensity` 0..1 is the dream's strength (the client
|
||||
maps it to the luminous-haze look, the way it maps `grade.tone` to a filter)."""
|
||||
|
||||
strength: int
|
||||
intensity: float
|
||||
blend: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -116,69 +62,37 @@ class RenderPlan:
|
||||
|
||||
grade: ColorGrade
|
||||
overlay: AnalyticalOverlay
|
||||
affect: AffectOverlay
|
||||
dream: Dream
|
||||
restyle: Restyle
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
"""True when the plan leaves the neutral base un-altered. Affect needs no
|
||||
clause: `min(left,right) > 0` implies `left > 0`, so `overlay.level > 0`
|
||||
already makes a plan with active affect non-identity."""
|
||||
"""True when the plan leaves the neutral base un-altered."""
|
||||
return (
|
||||
self.grade.is_identity
|
||||
and self.overlay.level == 0
|
||||
and self.dream.strength == 0
|
||||
and self.overlay.intensity == 0.0
|
||||
and self.restyle.blend == 0.0
|
||||
)
|
||||
|
||||
|
||||
def _overlay_intensity(left: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.overlay_gain * left / KNOB_MAX, 0.0, 1.0)
|
||||
def _overlay_intensity(left: int) -> float:
|
||||
"""Left knob -> analytical-overlay intensity (0..1)."""
|
||||
return left / KNOB_MAX
|
||||
|
||||
|
||||
def _affect_strength(left: int, right: int) -> int:
|
||||
"""Affect surfaces only when BOTH knobs are up — gated by the smaller one."""
|
||||
return min(left, right)
|
||||
def _restyle_blend(right: int) -> float:
|
||||
"""Right knob -> v2v restyle blend (0..1)."""
|
||||
return right / KNOB_MAX
|
||||
|
||||
|
||||
def _affect_intensity(left: int, right: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.overlay_gain * _affect_strength(left, right) / KNOB_MAX, 0.0, 1.0)
|
||||
def _mood_tone(dark: int, light: int) -> float:
|
||||
"""(dark, light) -> signed mood grade in [-1, 1]; equal -> 0 identity (§5)."""
|
||||
return (light - dark) / KNOB_MAX
|
||||
|
||||
|
||||
def _dream_intensity(right: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.dream_gain * right / KNOB_MAX, 0.0, 1.0)
|
||||
|
||||
|
||||
def _mood_tone(dark: int, light: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.mood_gain * (light - dark) / KNOB_MAX, -1.0, 1.0)
|
||||
|
||||
|
||||
def plan_alteration(
|
||||
coord: Coordinate, calibration: Calibration = DEFAULT_CALIBRATION
|
||||
) -> RenderPlan:
|
||||
def plan_alteration(coord: Coordinate) -> RenderPlan:
|
||||
"""Map a knob vector to its layered RenderPlan (design §4)."""
|
||||
return RenderPlan(
|
||||
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light, calibration)),
|
||||
overlay=AnalyticalOverlay(
|
||||
level=coord.left,
|
||||
intensity=_overlay_intensity(coord.left, calibration),
|
||||
),
|
||||
affect=AffectOverlay(
|
||||
strength=_affect_strength(coord.left, coord.right),
|
||||
intensity=_affect_intensity(coord.left, coord.right, calibration),
|
||||
),
|
||||
dream=Dream(
|
||||
strength=coord.right,
|
||||
intensity=_dream_intensity(coord.right, calibration),
|
||||
),
|
||||
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light)),
|
||||
overlay=AnalyticalOverlay(intensity=_overlay_intensity(coord.left)),
|
||||
restyle=Restyle(blend=_restyle_blend(coord.right)),
|
||||
)
|
||||
|
||||
|
||||
def render_plan_to_dict(plan: RenderPlan) -> dict:
|
||||
"""JSON-serializable form for the simulator API."""
|
||||
return {
|
||||
"grade": {"tone": plan.grade.tone},
|
||||
"overlay": {"level": plan.overlay.level, "intensity": plan.overlay.intensity},
|
||||
"affect": {"strength": plan.affect.strength, "intensity": plan.affect.intensity},
|
||||
"dream": {"strength": plan.dream.strength, "intensity": plan.dream.intensity},
|
||||
"is_identity": plan.is_identity,
|
||||
}
|
||||
|
||||
-211
@@ -1,211 +0,0 @@
|
||||
"""Scale-ring navigation: the endless-encoder closed loop of nature scales (§3).
|
||||
|
||||
Design: docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-
|
||||
design.md §3 (scale navigation & zoom transitions).
|
||||
|
||||
The scales-of-nature library is navigated as a CLOSED RING, not a line. A
|
||||
dedicated endless rotary encoder — relative, vs the absolute 0..4 experience
|
||||
knobs — walks the ring one step per detent; diving past the smallest scale wraps
|
||||
around to the largest (the infinite-zoom payoff). Between each adjacent pair of
|
||||
scales a short pre-baked AI zoom/warp transition plays.
|
||||
|
||||
This is the canonical navigation engine (Python-canonical, shared with the Pi
|
||||
player/firmware); the simulator browser only renders what `advance_ring` returns.
|
||||
|
||||
Ordering convention:
|
||||
- index 0 = the LARGEST scale (cosmos); increasing index zooms INWARD toward
|
||||
the smallest.
|
||||
- +1 detent zooms in, -1 zooms out; BOTH wrap (past the smallest wraps to the
|
||||
largest, and vice versa).
|
||||
- edge i connects scales[i] -> scales[(i+1) % N]; the last edge is the
|
||||
small->large wrap seam. One transition clip per edge plays FORWARD when
|
||||
zooming inward and REVERSED when zooming outward, so N scales need only N
|
||||
transition clips.
|
||||
|
||||
Fast spin (§3, "transitions chain, or past a speed threshold a faster blended
|
||||
pass is used"): a quick encoder spin batches many detents into one advance, so
|
||||
`abs(delta)` is the input adapter's proxy for spin speed. Past an opt-in
|
||||
`fast_spin_threshold` the move collapses to a SINGLE blended pass (the
|
||||
arrival-edge clip, played fast) rather than chaining N full transitions, so a
|
||||
fast spin stays responsive instead of grinding through every morph. The policy
|
||||
is opt-in (default off) because this pure function cannot observe wall-clock
|
||||
spin speed — the input layer (simulator/firmware), which can, enables it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
# Default spin-speed cutoff for the input layer (simulator/firmware) to enable
|
||||
# the fast-spin blended pass: 3+ detents batched into one advance() is a fast
|
||||
# spin crossing several scales; 1-2 are deliberate single steps that chain. Tune
|
||||
# by eye — it is a UX policy, not canonical ring structure.
|
||||
DEFAULT_FAST_SPIN_THRESHOLD = 3
|
||||
|
||||
|
||||
class RingError(ValueError):
|
||||
"""Raised when a ScaleRing is structurally invalid."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Scale:
|
||||
"""A node on the ring: a scale of nature and the base clip it plays."""
|
||||
|
||||
id: str
|
||||
clip_id: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Transition:
|
||||
"""A pre-baked zoom/warp morph along ONE ring edge.
|
||||
|
||||
`file` plays FORWARD when zooming inward (scales[i] -> scales[i+1] in ring
|
||||
order); the renderer reverses it for the outward direction, so one clip
|
||||
covers both directions of an edge.
|
||||
"""
|
||||
|
||||
file: str
|
||||
model: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ScaleRing:
|
||||
"""A closed ring of scales joined by per-edge transitions.
|
||||
|
||||
Invariant: a ring of N>=2 scales has exactly N transitions (one per edge,
|
||||
including the small->large wrap seam). A degenerate single-scale ring has no
|
||||
edges. An empty ring is rejected.
|
||||
"""
|
||||
|
||||
scales: tuple[Scale, ...]
|
||||
transitions: tuple[Transition, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
n = len(self.scales)
|
||||
if n == 0:
|
||||
raise RingError("a ScaleRing needs at least one scale")
|
||||
expected = 0 if n == 1 else n
|
||||
if len(self.transitions) != expected:
|
||||
raise RingError(
|
||||
f"a {n}-scale ring needs {expected} transitions, "
|
||||
f"got {len(self.transitions)}"
|
||||
)
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self.scales)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TransitionStep:
|
||||
"""One transition to play during a move: which edge clip, in which direction,
|
||||
and the scale index it lands on.
|
||||
|
||||
`blended` marks the single collapsed step of a fast-spin pass (see
|
||||
`advance_ring`'s `fast_spin_threshold`): the renderer plays it as one quick
|
||||
accelerated morph straight to the destination instead of a full transition.
|
||||
"""
|
||||
|
||||
edge: int
|
||||
reversed: bool
|
||||
file: str
|
||||
to_index: int
|
||||
blended: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RingMove:
|
||||
"""The result of advancing the encoder: where you end up and the ordered
|
||||
transitions to play getting there (chained for a multi-detent spin).
|
||||
|
||||
`fast` is True when the move was collapsed to a single blended pass because
|
||||
the spin crossed the speed threshold (see `advance_ring`).
|
||||
"""
|
||||
|
||||
from_index: int
|
||||
to_index: int
|
||||
steps: tuple[TransitionStep, ...]
|
||||
wrapped: bool
|
||||
fast: bool = False
|
||||
|
||||
|
||||
def scale_at(ring: ScaleRing, index: int) -> Scale:
|
||||
"""The scale at `index`, taken modulo the ring length (wraps both ways)."""
|
||||
return ring.scales[index % len(ring)]
|
||||
|
||||
|
||||
def advance_ring(
|
||||
ring: ScaleRing,
|
||||
from_index: int,
|
||||
delta: int,
|
||||
*,
|
||||
fast_spin_threshold: int = 0,
|
||||
) -> RingMove:
|
||||
"""Walk the endless encoder `delta` detents from `from_index` (signed).
|
||||
|
||||
Returns a `RingMove` with the landing index and the ordered `TransitionStep`s
|
||||
to play. Inward steps (+) play edge i forward; outward steps (-) play the
|
||||
crossed edge reversed. `wrapped` is True if any step crossed the small<->large
|
||||
seam. A degenerate single-scale ring (or delta 0) is a no-op.
|
||||
|
||||
`fast_spin_threshold` (opt-in, default off): when >= 2 and `abs(delta)` meets
|
||||
it, the spin is treated as fast and the move collapses to a single blended
|
||||
pass — one `TransitionStep` (the arrival edge, marked `blended`) landing
|
||||
straight on the destination, with `RingMove.fast=True` — instead of chaining
|
||||
every transition. See the module docstring (§3).
|
||||
"""
|
||||
n = len(ring)
|
||||
start = from_index % n
|
||||
if n == 1 or delta == 0:
|
||||
return RingMove(from_index=start, to_index=start, steps=(), wrapped=False)
|
||||
|
||||
direction = 1 if delta > 0 else -1
|
||||
steps: list[TransitionStep] = []
|
||||
wrapped = False
|
||||
index = start
|
||||
for _ in range(abs(delta)):
|
||||
if direction > 0:
|
||||
# zoom inward: edge `index` forward, land at index+1 (mod n)
|
||||
edge = index
|
||||
nxt = (index + 1) % n
|
||||
reversed_ = False
|
||||
else:
|
||||
# zoom outward: cross edge `index-1` reversed, land at index-1 (mod n)
|
||||
edge = (index - 1) % n
|
||||
nxt = (index - 1) % n
|
||||
reversed_ = True
|
||||
if edge == n - 1:
|
||||
wrapped = True
|
||||
steps.append(
|
||||
TransitionStep(
|
||||
edge=edge,
|
||||
reversed=reversed_,
|
||||
file=ring.transitions[edge].file,
|
||||
to_index=nxt,
|
||||
)
|
||||
)
|
||||
index = nxt
|
||||
|
||||
# Fast spin: collapse the whole chain to a single blended arrival pass. The
|
||||
# landing index, seam-crossing, and arrival edge/direction are exactly those
|
||||
# of the full chain — only the in-between transitions are dropped.
|
||||
if fast_spin_threshold >= 2 and abs(delta) >= fast_spin_threshold:
|
||||
arrival = steps[-1]
|
||||
blended = TransitionStep(
|
||||
edge=arrival.edge,
|
||||
reversed=arrival.reversed,
|
||||
file=arrival.file,
|
||||
to_index=arrival.to_index,
|
||||
blended=True,
|
||||
)
|
||||
return RingMove(
|
||||
from_index=start,
|
||||
to_index=index,
|
||||
steps=(blended,),
|
||||
wrapped=wrapped,
|
||||
fast=True,
|
||||
)
|
||||
|
||||
return RingMove(
|
||||
from_index=start, to_index=index, steps=tuple(steps), wrapped=wrapped
|
||||
)
|
||||
+5
-7
@@ -3,10 +3,10 @@
|
||||
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
|
||||
desired Playback (which neutral base clip, how it is altered, what audio plays,
|
||||
at what levels) and returns the Transition from the previous Playback. The
|
||||
transition KIND encodes design §4.3: the Dark/Light grade, the Left overlay AND
|
||||
the Right dream (a deterministic, live luminous haze since session 0013) are all
|
||||
continuous runtime ops (LIVE_UPDATE); only swapping the clip needs a CROSSFADE,
|
||||
and toggling video on/off fades to/from black.
|
||||
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
|
||||
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
|
||||
pre-baked Right v2v variant needs a CROSSFADE, and toggling video on/off
|
||||
fades to/from black.
|
||||
|
||||
Knobs no longer *select* a clip (the base library is neutral by construction);
|
||||
they *transform* it. Which neutral base to show is an injected policy
|
||||
@@ -107,9 +107,7 @@ class Player:
|
||||
if next_video and not prev_video:
|
||||
return TransitionKind.FADE_FROM_BLACK
|
||||
if next_video and prev_video:
|
||||
# The Right dream is a live filter now (not a substrate swap), so it
|
||||
# no longer forces a crossfade — only a clip (scale) change does.
|
||||
if nxt.clip_id != prev.clip_id:
|
||||
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
|
||||
return TransitionKind.CROSSFADE
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
# both black: only audio/levels could have changed
|
||||
|
||||
@@ -1,101 +0,0 @@
|
||||
# Session 0007.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Date: 2026-06-06 (PST) · 14:12 → 17:57
|
||||
> Type: coding
|
||||
> Goal: continue the roadmap; reframed by the operator to **simulator-first** —
|
||||
> bring the slice-1 alteration engine into the simulator so the experience can be
|
||||
> tuned before any hardware.
|
||||
> Outcome: **design spec written, committed, and pushed** on branch
|
||||
> `feature/simulator-alteration-preview`. No code changes. Implementation deferred
|
||||
> to the next session (design + implementation to ship as one PR).
|
||||
|
||||
## Plan
|
||||
|
||||
Bring the alteration engine into the simulator (scope: tune the look), per
|
||||
`docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md`:
|
||||
- Python-canonical engine, thin browser renderer.
|
||||
- Dark/Light = live runtime color grade; Left/Right = 5×5 grid of pre-baked
|
||||
authored variant clips selected by coordinate (24 videos/base clip).
|
||||
- Revises design §4.3/§10: HUD baked into variant videos (authorial precision)
|
||||
rather than runtime overlay — trades away near-free i18n.
|
||||
- Engine reconciliation: parameterize `Calibration`, add `VariantRef`, slim
|
||||
`RenderPlan` (drop runtime `AnalyticalOverlay`/`Restyle`); `state.py` crossfade
|
||||
on variant change.
|
||||
- Simulator: retire selection X-ray + `/api/select`; add `/api/alteration` +
|
||||
`/api/clips` + variant manifest; placeholder-variant generator.
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` at `5290785`, clean and pushed. Sub-projects 1 & 2 done; sub-project 3
|
||||
(Player Runtime) in progress — slice 1 (pure-logic `player/` core) merged via
|
||||
PR #5 (session 0006).
|
||||
- A **stale nested worktree** (`.worktrees/feature-BuildSimulator11`, branch
|
||||
`feature/player-alteration-core`, already merged) was present and breaking the
|
||||
session resolver ("ambiguous app.json"), exactly as the session-0006 memory
|
||||
gotcha predicted.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Session gate.** Opened with "What's next?" → classified as a coding session;
|
||||
ran `wgl-session-coding-init`.
|
||||
2. **Baseline cleanup.** Claim dry-run failed on the resolver ambiguity. Verified
|
||||
the stale worktree's branch was fully merged + clean, removed the worktree and
|
||||
branch, and deleted the empty `.worktrees/` dir. Tree back to clean `main`.
|
||||
3. **Claimed session 0007** (no other sessions in flight).
|
||||
4. **Operator steer:** "only design things that will run in the simulator; get
|
||||
everything working the way we like in the simulator before hardware." Saved as
|
||||
a `feedback` memory ([[simulator-first-before-hardware]]).
|
||||
5. **Discovered** the simulator (`simulator/`, built session 0004) is fully
|
||||
decoupled from `player/` — it exercises the OLD *selection* model and has no
|
||||
alteration surface. That gap became the work.
|
||||
6. **Brainstorming** (with the visual companion). Decisions, one at a time:
|
||||
scope = tune the alteration look; medium = short looping video; Right restyle =
|
||||
labeled stand-in shader (swappable for real variants); calibration =
|
||||
live-adjustable curves; architecture = **A, Python-canonical**; integration =
|
||||
new view, **retire the selection X-ray**.
|
||||
7. **Interactive look mockup** (browser): a real nature image with the 4 knobs as
|
||||
live sliders. Operator confirmed Dark/Light read well, but said for Left/Right
|
||||
they'll **author videos with the overlay baked in** for precise HUD control.
|
||||
8. **Surfaced the tradeoff:** baking the HUD reverses design §4.3/§10's near-free
|
||||
i18n. Operator chose **precision wins**. Confirmed the variant grid as **full
|
||||
5×5 = 24 authored clips per base** (shown as a grid mockup).
|
||||
9. **Revised model** presented and approved: base clip + variant manifest;
|
||||
`RenderPlan = {grade, variant}`; runtime overlay/restyle layers retired;
|
||||
placeholder-variant bootstrapping via ffmpeg.
|
||||
10. **Wrote the spec**, self-reviewed, committed on a feature branch.
|
||||
11. Operator: "do [implementation] next session. finalize this one." →
|
||||
`wgl-session-finalize`.
|
||||
|
||||
## Cut state (end of session)
|
||||
|
||||
| Repo | Branch | Commit | State |
|
||||
|---|---|---|---|
|
||||
| human-experience-filter-art | `feature/simulator-alteration-preview` | `3ef21fb` | pushed to origin, **not merged** |
|
||||
|
||||
- `docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md` —
|
||||
new, committed.
|
||||
- `main` unchanged at `5290785`.
|
||||
- Working tree clean. Visual-companion server stopped; `.superpowers/` gitignored.
|
||||
- No tests run (no code changed).
|
||||
|
||||
## What lands on the operator's plate
|
||||
|
||||
- **Author the 24 variant videos** per base clip (the Left×Right grid), with HUD
|
||||
baked in. The simulator will consume them via the manifest; placeholder variants
|
||||
cover the gap until then.
|
||||
- **Deferred decisions** (also surfaced in chat):
|
||||
- *knob→strength calibration* — unresolved since session 0006; will be settled
|
||||
by eye in the simulator's calibration panel next session, then baked into
|
||||
`DEFAULT_CALIBRATION`.
|
||||
- *grade tints baked HUD* — accepted under "precision wins"; revisit only if it
|
||||
reads badly once real authored clips exist.
|
||||
|
||||
## Prompt the operator can paste into the next session
|
||||
|
||||
```
|
||||
/goal Implement the simulator alteration preview on branch feature/simulator-alteration-preview, per docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md — begin with the writing-plans skill, then build: parameterize Calibration + add VariantRef and slim RenderPlan in player/, add /api/alteration + /api/clips + simulator/clips.py variant manifest, rewrite the simulator UI as the player preview, retire the selection X-ray + /api/select, add the placeholder-variant generator, and update tests + USER_GUIDE + ROADMAP §3 + the parent design §4.3/§10 pointer.
|
||||
```
|
||||
|
||||
(Or resume with `/goal next` — the `Next /goal:` field is stored in memory
|
||||
`sub-project-3-player-progress.md`.)
|
||||
@@ -1,102 +0,0 @@
|
||||
# Session 0008.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-07T19-42 (PST)
|
||||
> End: 2026-06-07T22-37 (PST)
|
||||
> Type: spec
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
Session opened as `wgl-session-none` (research errand: "Let's find a great public
|
||||
domain nature video for this project"), then upgraded mid-session to a **spec**
|
||||
session once the work became tracked. Operator chose "Spec session, comprehensive":
|
||||
fold the session's POC findings into the design AND resolve the open cosmic-zoom
|
||||
structure fork.
|
||||
|
||||
## Plan
|
||||
|
||||
Develop a design revision to the machine-altered-perception design, grounded in a
|
||||
local POC run this session: (1) Right-axis alteration pipeline; (2) content
|
||||
structure (cosmic-zoom fork); (3) strictly-PD sourcing.
|
||||
|
||||
## Pre-state
|
||||
|
||||
- On `feature/simulator-alteration-preview` (session 0007's unmerged design), clean
|
||||
tree; `main` at `5290785`.
|
||||
- Catalog empty; `tools/ingest/internet_archive.py` is the only wired fetcher.
|
||||
- No ML stack, no ffmpeg in the venv (pure-stdlib project).
|
||||
|
||||
## Session arc (uncurated)
|
||||
|
||||
1. **PD nature-video research.** Surveyed pools and verified licenses against the
|
||||
actual metadata, not marketing. Key finding: **Pexels / Pixabay / Mitch Martinez
|
||||
"free 4K" are royalty-free but NOT public domain** (restrict redistribution,
|
||||
retain copyright) — exactly the trap the ingest tool's "no explicit license →
|
||||
assume PD, verify" flag exists for. Genuinely-PD: NASA/Hubble (cosmos), **NOAA
|
||||
Ocean Exploration** (deep sea, global), **NPS/USGS** (US land). Wikimedia nature
|
||||
4K timelapses are mostly CC-BY, not PD.
|
||||
|
||||
2. **Operator pivot → "cosmic zoom".** Operator proposed a Powers-of-Ten journey
|
||||
(space → continents → birds → ocean → abyss → microscopic → cosmos). Brainstormed
|
||||
the thesis tension (a scale-journey carries built-in awe vs. the "neutral base"
|
||||
thesis) and the cost (per-base pre-bake × altitudes). Surfaced that the *ends* of
|
||||
the zoom are strict-PD-rich; the terrestrial *middle* (birds, non-US land) is the
|
||||
CC-BY soft spot.
|
||||
|
||||
3. **"Can this run locally?" → POC.** Operator's machine: **Mac mini M4 Pro, 64 GB,
|
||||
16-core GPU.** Built a throwaway POC in `~/hef-poc/` (outside the repo).
|
||||
- *Wrong turns:* `brew install ffmpeg` and `pip install` were **denied by the
|
||||
harness** (it blocks package installs) — switched to venv-local `imageio-ffmpeg`
|
||||
run by the operator via `!`. zsh **doesn't word-split** unquoted `$EARGS` →
|
||||
mangled encoder flags; inlined them. sd-turbo img2img **crashed** ("reshape
|
||||
tensor of 0 elements") because `int(steps×strength)=int(2×0.45)=0` denoise steps
|
||||
→ guarded to ≥1.
|
||||
- *Results on an 8s/1080p clip:* deterministic **Dark/Light/Left** grades/overlay
|
||||
~2.4s each (~3× faster than realtime, runtime-capable). **Right** painterly
|
||||
restyle (SD img2img on MPS) ~3.4 min/clip and **flickers badly**.
|
||||
|
||||
4. **Flicker is disqualifying.** Operator: "crazy and disorienting … this project is
|
||||
meant to be peaceful." Prototyped two fixes: (a) a **deterministic** soft-dreamy
|
||||
filter (smartblur + RGB bloom — first attempt had a magenta cast from blending on
|
||||
YUV chroma planes; fixed by blending in RGB), ~5s, zero flicker by construction;
|
||||
(b) **optical-flow keyframe propagation** (EbSynth principle; genuine ebsynth is
|
||||
NVIDIA/Windows so implemented with OpenCV Farneback + the diffusers pipeline),
|
||||
~2.7 min/clip, **calm**. Operator chose **AI + flow**.
|
||||
|
||||
5. **Spec session (comprehensive).** Upgraded the session via
|
||||
`wgl-session-spec-init` → claimed 0008. Resolved the cosmic-zoom fork: operator
|
||||
chose a **small NEUTRAL "scales of nature" library** (not a single stitched film).
|
||||
Then operator added the **infinite-zoom ring** (AI zoom transitions between
|
||||
scales, micro→cosmos wraparound) and the **infinitely-turnable endless encoder**.
|
||||
Probed i18n feasibility → **§1.2**: Left HUD is a Pi-rendered **runtime overlay**
|
||||
(Pango/HarfBuzz + Noto, annotation-track + per-language string tables), NOT a baked
|
||||
per-language video.
|
||||
- Wrote `docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-design.md`,
|
||||
committed on `feature/scales-library-right-axis`.
|
||||
|
||||
## Cut state (what landed)
|
||||
|
||||
- **PR #6 merged to `main`** (merge `42a72fe`): the design revision doc.
|
||||
- No spec-RFC submission: this app keeps specs in-repo (`app.json` `contains:["specs"]`),
|
||||
so `submit-spec.sh` is N/A — surfaced, not dropped.
|
||||
- POC artifacts left in `~/hef-poc/` (throwaway, outside repo): `restyle.py`,
|
||||
`flow_restyle.py`, and the comparison clips/contact sheets.
|
||||
- Memory updated: `sub-project-3-player-progress.md` (+ `MEMORY.md` index).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **Left-HUD treatment conflict (low confidence — flag).** 0008 §1.2 specifies the
|
||||
Left HUD as a runtime Pango/HarfBuzz overlay (to keep i18n near-free, per parent
|
||||
§10). This **reverses session 0007's decision** to bake the HUD into the 5×5
|
||||
variant grid for authorial precision. I recommended the runtime-overlay path
|
||||
following the operator's i18n question without flagging the 0007 reversal at the
|
||||
time. `main` now carries the runtime-overlay position; the unmerged
|
||||
`feature/simulator-alteration-preview` carries baked-HUD. **Reconciliation deferred
|
||||
to next session** (pick one, or hybrid: authored positions + runtime-shaped text).
|
||||
|
||||
## Next /goal
|
||||
|
||||
```
|
||||
/goal Reconcile the 0007 simulator-alteration-preview design (feature/simulator-alteration-preview, UNMERGED) with the merged 0008 scales-library + flow-stabilized Right-axis design — resolve the Left-HUD conflict (baked-into-5×5-grid vs runtime Pango/HarfBuzz overlay) FIRST — then writing-plans → build the simulator-first slice: deterministic Dark/Light/Left + a flow-stabilized Right restyle over ONE neutral scale clip wired into the simulator. Defer Pi renderer, endless-encoder/serial, and AI zoom-transitions per simulator-first.
|
||||
```
|
||||
@@ -1,130 +0,0 @@
|
||||
# Session 0009.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-07T22-41 (PST)
|
||||
> End: 2026-06-07T23-03 (PST)
|
||||
> Type: coding
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
Opened with `/goal next` — resume the goal recorded at the 0008 finalize. The
|
||||
session gate resolved the stored `Next /goal:` from memory:
|
||||
|
||||
> Reconcile the 0007 simulator-alteration-preview design (UNMERGED) with the
|
||||
> merged 0008 scales-library + flow-stabilized Right-axis design — resolve the
|
||||
> Left-HUD conflict (baked-into-5×5-grid vs runtime Pango/HarfBuzz overlay) FIRST
|
||||
> — then writing-plans → build the simulator-first slice: deterministic
|
||||
> Dark/Light/Left + a flow-stabilized Right restyle over ONE neutral scale clip
|
||||
> wired into the simulator. Defer Pi renderer, endless-encoder/serial, AI
|
||||
> zoom-transitions.
|
||||
|
||||
## Plan
|
||||
|
||||
Brainstorm the reconciliation (resolve Left-HUD first) → write a unified design
|
||||
doc → writing-plans → build & merge the simulator-first alteration slice.
|
||||
|
||||
## Pre-state
|
||||
|
||||
- On `main` at `c99a669`, clean tree; ran from the **main clone** (not a nested
|
||||
worktree — avoiding the session-resolver ambiguity gotcha flagged in 0008).
|
||||
- Two unmerged design threads: session-0007's
|
||||
`feature/simulator-alteration-preview` (baked-HUD 5×5 grid, design doc only, no
|
||||
code) vs. session-0008's merged scales-library/right-axis design (runtime-overlay
|
||||
HUD). The Left-HUD conflict between them was the thing to resolve.
|
||||
- Slice-1 engine (`player/`) on `main`: `RenderPlan {grade, overlay, restyle}` with
|
||||
a continuous `Restyle.blend` and a runtime `AnalyticalOverlay`. Simulator still on
|
||||
the old *selection* surface (`/api/select`, X-ray, `fixtures.py`).
|
||||
- 0008 POC artifacts present in `~/hef-poc/out/` (outside repo): real `neutral.mp4`
|
||||
base + `right_flow.mp4` flow-stabilized restyle.
|
||||
|
||||
## Session arc (uncurated)
|
||||
|
||||
1. **Claimed 0009** (coding) from the main clone; resolver worked cleanly. Read
|
||||
both specs + memory in full.
|
||||
|
||||
2. **Surfaced the conflict precisely — it was 2-D, not 1-D.** Beyond the Left-HUD
|
||||
rendering question, the two designs disagreed on the *shape of the pre-baked set*:
|
||||
0007 made BOTH Left and Right a 5×5 grid of 24 authored clips; 0008 made only
|
||||
Right pre-baked and Left a runtime overlay. Key realization: **0008 §1.2 already
|
||||
IS the hybrid the operator floated** ("authored box positions + runtime-shaped
|
||||
text") — authored annotation track for layout, runtime shaping for cheap i18n.
|
||||
|
||||
3. **Brainstorming → one operator question.** Asked the single load-bearing fork
|
||||
(Left HUD: runtime-overlay+authored-track vs fully-baked vs defer). Operator chose
|
||||
**runtime overlay + authored track** (the 0008 / hybrid position). Everything else
|
||||
followed from approved material, so I proceeded autonomously.
|
||||
|
||||
4. **Found the engine was already most of the way there.** The merged slice-1 engine
|
||||
already keeps a runtime `AnalyticalOverlay` — only 0007's *unmerged doc* proposed
|
||||
removing it. So the reconciliation was surgical: continuous `Restyle.blend` →
|
||||
discrete `Restyle.variant`; add `AnalyticalOverlay.level`; add a frozen
|
||||
`Calibration`. Wrote the reconciled design doc (brought the 0007 doc onto the
|
||||
branch so its superseded position is preserved + links resolve), committed.
|
||||
|
||||
5. **writing-plans → a full TDD plan** (`docs/superpowers/plans/...`), then executed
|
||||
it task-by-task with executing-plans:
|
||||
- **Task 1–2 (engine):** `Calibration`+`DEFAULT_CALIBRATION` (behavior-preserving),
|
||||
`Restyle.variant`, `AnalyticalOverlay.level`, `render_plan_to_dict`. `state.py`
|
||||
needed no logic change (compares whole `Restyle`). Tests rewritten/green.
|
||||
- **Task 3 (clips):** `simulator/clips.py` manifest model; retired `fixtures.py` +
|
||||
`test_fixtures.py`.
|
||||
- **Task 4 (API):** `/api/alteration` + `/api/clips`; removed `/api/select` +
|
||||
`/api/catalog/meta`. *Wrong turn:* asserted retired POST returns 404, but the
|
||||
static catch-all yields **405** for an unrouted POST — relaxed the assertion to
|
||||
"404 or 405 = gone."
|
||||
- **Task 5 (media):** `sample_media/manifest.json` + `setup_sample_media.py` that
|
||||
copies the real POC `neutral.mp4`→base and `right_flow.mp4`→Right-strength-4 and
|
||||
ffmpeg-blends placeholder strengths 1–3. mp4s gitignored. Ran it — 5 files
|
||||
produced.
|
||||
- **Task 6 (UI):** rewrote `simulator/static/` as the alteration preview (live
|
||||
grade via CSS filters, Right-variant `<video>` crossfade, live SVG Left overlay
|
||||
from the annotation track + string table, calibration panel, RenderPlan readout).
|
||||
- **Task 7 (docs):** parent design §4.3/§10 pointers; ROADMAP slice 2 done +
|
||||
deferred slices; USER_GUIDE simulator section rewritten.
|
||||
- *Environment:* `python` not on PATH; used `.venv/bin/python` throughout.
|
||||
|
||||
6. **Verified end-to-end.** `pytest -q` → **192 passed, 2 skipped**. Booted the sim
|
||||
on a scratch port: `/api/clips` returns the manifest (variant 0→base), the engine
|
||||
plan is correct by eye (left=3→level 3/intensity 0.75, right=4→variant 4,
|
||||
dark=4→tone −1.0), and the **real** `right4.mp4` + base served as `video/mp4`.
|
||||
|
||||
7. **Shipped.** Pushed the branch, created **Gitea PR #7** via the keychain-token API
|
||||
helper, **merged** it (autonomous posture), synced `main`, re-verified tests green
|
||||
on the merged result, deleted my merged feature branch (local + remote).
|
||||
|
||||
## Cut state (what landed)
|
||||
|
||||
- **PR #7 merged to `main`** (merge `554eb50`): the reconciliation design + the built
|
||||
simulator-first alteration slice (engine + simulator + media tooling + docs). 9
|
||||
commits.
|
||||
- `main` green: 192 passed / 2 skipped. Sim verified booting + serving real media.
|
||||
- Memory updated: `sub-project-3-player-progress.md` (+ `MEMORY.md` index) — conflict
|
||||
resolved, slice 2 shipped, new Next /goal.
|
||||
- POC artifacts untouched in `~/hef-poc/` (still the source for `setup_sample_media.py`).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **Brought the 0007 design doc into `main` + intended to delete the superseded
|
||||
branch.** Decided autonomously to carry `2026-06-06-simulator-alteration-preview-design.md`
|
||||
forward (so the reconciliation's links resolve and the superseded position is
|
||||
preserved) and to delete the now-stale `feature/simulator-alteration-preview`.
|
||||
**The branch deletion (`git branch -D` / `push --delete`) was permission-denied by
|
||||
the harness** — left in place. Safe to delete later (its content is in `main`); or
|
||||
the operator may want to keep it. Flagging rather than forcing.
|
||||
- **Merged under autonomous posture without a separate `/code-review`.** TDD + the
|
||||
written plan were the quality gates; no independent review pass was run before
|
||||
merge. Low risk for a slice this size, but noting it.
|
||||
- **Used the POC's Yosemite clip as the sim's sample base.** It's a "forest"-scale
|
||||
neutral clip, good enough to tune the look; framed in the manifest/USER_GUIDE as
|
||||
**look-tuning only, not shipped content**. Strict-PD scale-library sourcing stays a
|
||||
later slice — unaffected.
|
||||
- **`Calibration` defaults are still behavior-preserving, not operator-tuned.** The
|
||||
knob→strength calibration (open since session 0006) is now tunable by eye in the
|
||||
sim but **not yet locked** — that's the next goal.
|
||||
|
||||
## Next /goal
|
||||
|
||||
```
|
||||
/goal Tune the alteration look by eye in the simulator (python simulator/setup_sample_media.py then make sim-local) and LOCK the knob→strength calibration into DEFAULT_CALIBRATION in player/alteration.py (+ a unit test) — settling the open session-0006 calibration decision. While there, judge whether more neutral "scales of nature" base clips + a real multi-strength flow-stabilized Right re-bake are worth doing next vs. moving to scale-ring navigation (endless encoder + AI zoom transitions). Keep deferring Pi renderer + serial/firmware. Read sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md (§8) first.
|
||||
```
|
||||
@@ -1,129 +0,0 @@
|
||||
# Session 0010.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-07T23-09 (PST)
|
||||
> End: 2026-06-07T23-22 (PST)
|
||||
> Type: coding
|
||||
> Status: **FINALIZED**
|
||||
> Posture: autonomous (yolo)
|
||||
> Landed: PR #8 (merge `638cf58`) — calibration lock + dark-grade look fix
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
Tune the alteration look by eye in the simulator (python simulator/setup_sample_media.py then make sim-local) and LOCK the knob→strength calibration into DEFAULT_CALIBRATION in player/alteration.py (+ a unit test) — settling the open session-0006 calibration decision. While there, judge whether more neutral "scales of nature" base clips + a real multi-strength flow-stabilized Right re-bake are worth doing next vs. moving to scale-ring navigation (endless encoder + AI zoom transitions). Keep deferring Pi renderer + serial/firmware. Read sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md (§8) first.
|
||||
```
|
||||
|
||||
## Pre-state
|
||||
|
||||
- `main` clean at `e753a68` (then ff'd to the 0010 claim commit `7d2a306`).
|
||||
- Sub-project 3 slice 2 (sim alteration) merged in session 0009 (PR #7). Engine
|
||||
in `player/alteration.py` carried a `Calibration` + `DEFAULT_CALIBRATION` with
|
||||
**behavior-preserving placeholder** values; the knob→strength calibration had
|
||||
been the open decision since session 0006.
|
||||
- Sample media already populated under `simulator/sample_media/forest/` (base +
|
||||
Right strengths 1–4; only strength 4 is the real flow-stabilized restyle, 1–3
|
||||
are ffmpeg blend placeholders). POC artifacts present in `~/hef-poc/out/`.
|
||||
|
||||
## The arc
|
||||
|
||||
### 1. Session open + grounding
|
||||
- Classified the launch prompt as a **coding** session; ran
|
||||
`wgl-session-coding-init`, claimed session **0010** (no other sessions in
|
||||
flight). Verified clean pushed `main` baseline + CLAUDE.md stub.
|
||||
- Read the resume context: `sub-project-3-player-progress` memory + the reconciled
|
||||
slice design §8 (open questions: calibration curve shape, grade-vs-overlay
|
||||
ordering, crossfade timing, placeholder fidelity).
|
||||
- Inspected the engine (`player/alteration.py`), its test, the sim frontend
|
||||
(`simulator/static/app.js`), and the content/clip plumbing. Established that the
|
||||
calibration's three params (`mood_gain`, `overlay_gain`, `right_variant_map`)
|
||||
are the lock target, and that the *grade look* itself lives in `app.js` CSS
|
||||
filters (frontend), not in `DEFAULT_CALIBRATION`.
|
||||
|
||||
### 2. Tuning the look by eye (the key finding)
|
||||
- `python`/`ffmpeg` weren't on the non-interactive shell PATH; used the project
|
||||
`.venv`. Baseline suite: **192 passed / 2 skipped**.
|
||||
- Viewed the operator's own POC renders in `~/hef-poc/out/` (`all_axes.png`,
|
||||
`right_three.png`, `dark_frame.png`, `light_frame.png`, `lefthud_frame.png`) —
|
||||
the by-eye-approved (session 0008) looks: tasteful cool-blue dark, warm golden
|
||||
light, soft dreamy flow restyle, legible analytical HUD.
|
||||
- Booted the sim (`uvicorn simulator.app:app`) and drove it with **headless
|
||||
Chrome (puppeteer-core)** to screenshot the *actual* rendered look across all
|
||||
four axes at full knob, plus the whole-brain-dark corner.
|
||||
- **Finding:** the sim's **dark** mood pole was rendering a full-frame
|
||||
`hue-rotate(-cool*200deg)` → rock turned orange, trees purple — a **psychedelic**
|
||||
look, exactly the "disorienting, not peaceful" effect the operator rejected in
|
||||
0008, and nothing like the POC `dark_frame`. Light / Left / Right all looked
|
||||
good and peaceful. So the dark grade was the one thing blocking a likable look.
|
||||
|
||||
### 3. Dark-grade fix
|
||||
- Replaced the hue-rotate in `applyGrade` (`simulator/static/app.js`) with:
|
||||
darken + slight desaturate on the video filter for the cool pole, plus a
|
||||
`multiply`-blended deep-blue wash (`#tint`, new element) that lifts shadows
|
||||
toward blue while preserving natural greens. Placed `#tint` **below** the SVG
|
||||
overlay so the Left HUD stays legible regardless of mood (resolves the §8
|
||||
grade-vs-overlay ordering question → overlay above).
|
||||
- Re-screenshotted: full dark now reads cool/somber with natural greens (matches
|
||||
POC `dark_frame`); whole-brain-dark corner peaceful + HUD legible; light/neutral
|
||||
unchanged (the warm path is algebraically identical for tone ≥ 0).
|
||||
|
||||
### 4. Locking the calibration
|
||||
- With full tilt now tasteful on **every** axis (POC + sim evidence), the data
|
||||
says full knob = full look, 5 notches map 1:1 to the 5 discrete Right bakes,
|
||||
equal Dark/Light = identity → **unity gains + linear variant map**.
|
||||
- Promoted `DEFAULT_CALIBRATION` from placeholder to a **deliberately locked**
|
||||
`Calibration(mood_gain=1.0, overlay_gain=1.0, right_variant_map=(0,1,2,3,4))`
|
||||
with a provenance comment; this also confirms the session-0006 convention
|
||||
(0=off..4=max, equal Dark/Light = identity — no "centered at 2 = no push").
|
||||
- Added `test_default_calibration_is_locked` pinning the literal constants.
|
||||
Suite: **193 passed / 2 skipped**.
|
||||
- Updated design §8 (calibration-curve + grade-vs-overlay resolved; dark-grade
|
||||
fix recorded).
|
||||
|
||||
### 5. Land + build-vs-next judgment
|
||||
- Committed on `feature/lock-alteration-calibration`, pushed, opened **PR #8**
|
||||
(Gitea API via `wgl-gitea-admin` helper), merged to `main` (`638cf58`), deleted
|
||||
the branch.
|
||||
- Presented the build-vs-next judgment the prompt asked for. Recommendation
|
||||
(operator-confirmed via AskUserQuestion): **move to scale-ring navigation next**
|
||||
— pull in 1–2 *cheap, true-PD* neutral base clips (NASA cosmos, NOAA deep-sea)
|
||||
so the ring is demonstrable, build the endless-encoder control + placeholder
|
||||
zoom transitions in the sim, and **defer** the expensive real multi-strength
|
||||
flow-stabilized Right re-bake until the ring experience is liked (re-baking
|
||||
before the clip set is final risks wasted renders). Keep deferring Pi renderer +
|
||||
serial/firmware.
|
||||
|
||||
## Cut state
|
||||
|
||||
- `main` @ `638cf58`, clean, pushed. Suite **193 passed / 2 skipped**.
|
||||
- `DEFAULT_CALIBRATION` locked; dark-grade look fixed; design §8 updated; memory
|
||||
updated with the session outcome + the new `Next /goal`.
|
||||
- Deferred (unchanged): Pi renderer, serial/firmware; the expensive Right re-bake.
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **Locked `DEFAULT_CALIBRATION` to unity/linear autonomously.** The operator
|
||||
asked to "tune by eye" and lock; I made the by-eye judgment (via POC renders +
|
||||
headless-Chrome sim screenshots) that full tilt is peaceful on every axis after
|
||||
the dark fix, so no softening of gains was warranted, and locked the unity/linear
|
||||
values. *Alternative:* soften `mood_gain`/`overlay_gain` (<1.0) so full knob is
|
||||
gentler — set aside because the evidence showed full tilt is already calm. The
|
||||
calibration stays fully parameterized, so flipping the feel is a one-line edit +
|
||||
the lock test if the operator disagrees on review.
|
||||
- **Dark-grade fix landed in the sim frontend (not the final renderer).** The fix
|
||||
makes the *simulator* look right for tuning; the Pi/mpv renderer (later slice)
|
||||
will do proper grading. Recorded in §8.
|
||||
- The build-vs-next direction was **not** a deferred call — it was put to the
|
||||
operator directly and confirmed.
|
||||
|
||||
## Operator plate
|
||||
|
||||
- Calibration is settled and guarded; the long-open session-0006 decision is
|
||||
closed. The sim look is now likable on the one forest clip across all axes.
|
||||
- Next session's first move is the `/goal` below (scale-ring navigation).
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
```
|
||||
/goal Build scale-ring navigation in the simulator — add an endless rotary-encoder control (relative, vs the absolute 0-4 experience knobs) that walks a closed ring of neutral "scales of nature" clips with placeholder AI zoom/warp transitions between scales. Add 1–2 cheap, true-PD neutral base clips so the ring is demonstrable (NASA/Hubble cosmos + NOAA Ocean Exploration deep sea). Defer the expensive real multi-strength flow-stabilized Right re-bake until the ring is liked; keep deferring the Pi renderer + serial/firmware. Read the sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-design.md first.
|
||||
```
|
||||
@@ -1,167 +0,0 @@
|
||||
# Session 0011.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: coding
|
||||
> Start: 2026-06-07T23-26 (PST) · End: 2026-06-07T23-42 (PST)
|
||||
> Goal: `/goal` — Build **scale-ring navigation** in the simulator: an endless
|
||||
> rotary-encoder control (relative, vs the absolute 0–4 experience knobs)
|
||||
> walking a closed ring of neutral "scales of nature" clips with placeholder
|
||||
> AI zoom/warp transitions; add 1–2 cheap true-PD base clips (NASA cosmos +
|
||||
> NOAA deep sea) so the ring is demonstrable; defer the expensive
|
||||
> multi-strength flow-stabilized Right re-bake + the Pi renderer + firmware.
|
||||
> Outcome: **Sub-project 3 slice 3 — scale-ring navigation — built and MERGED to
|
||||
> `main` via PR #9** (merge commit `435f201`). New canonical
|
||||
> `player/ring.py`; `/api/ring` + `/api/ring/advance`; endless-encoder UI;
|
||||
> cosmos (NASA/Hubble) + abyss (NOAA Ocean Exploration) true-PD scales with
|
||||
> placeholder media. 215 passed / 2 skipped (+22 new). Deferral honored.
|
||||
|
||||
## Plan
|
||||
|
||||
Implement the scales-library design §3 (scale navigation & zoom transitions) in
|
||||
the simulator. The "what" was already approved in session 0008's design, so no
|
||||
fresh brainstorm: keep the navigation math **Python-canonical** in a new pure
|
||||
`player/ring.py` (mirrors `player/content.py`/`alteration.py`), expose it via the
|
||||
simulator API, and have the browser only *render* (play returned transition(s),
|
||||
settle on the target scale, keep the live knob alteration on top). Add two cheap
|
||||
**true-PD** scales (cosmos, abyss) so the ring has ≥2 demonstrable scales;
|
||||
**defer** the expensive multi-strength flow-stabilized Right re-bake (new scales
|
||||
carry a raw base only) and keep deferring the Pi renderer + serial/firmware per
|
||||
[[simulator-first-before-hardware]]. Process: short plan doc → TDD pure logic →
|
||||
API → media → UI → verify (boot + headless drive) → branch → PR → merge.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
/goal Build scale-ring navigation in the simulator — add an endless rotary-encoder
|
||||
control (relative, vs the absolute 0-4 experience knobs) that walks a closed ring
|
||||
of neutral "scales of nature" clips with placeholder AI zoom/warp transitions
|
||||
between scales. Add 1–2 cheap, true-PD neutral base clips so the ring is
|
||||
demonstrable (NASA/Hubble cosmos + NOAA Ocean Exploration deep sea). Defer the
|
||||
expensive real multi-strength flow-stabilized Right re-bake until the ring is
|
||||
liked; keep deferring the Pi renderer + serial/firmware.
|
||||
```
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` @ `7cf550a` locally, clean; behind `origin/main` by 1 after the claim
|
||||
(sessions live in this same repo under `sessions/`); fast-forwarded during init.
|
||||
- Slice 1 (player core, PR #5) + slice 2 (sim alteration, PR #7) + calibration
|
||||
lock & dark-grade fix (PR #8) all merged. `DEFAULT_CALIBRATION` locked unity/linear.
|
||||
- Simulator had ONE neutral clip (forest) wired with the real flow-stabilized
|
||||
Right variant; no ring, no second scale.
|
||||
- Ran from the **main clone** (not a nested worktree) — the 0010 resolver gotcha
|
||||
did **not** recur.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Gate / classify + claim.** `/goal …` → unambiguously a coding session. Ran
|
||||
`wgl-session-coding-init`; dry-run peek showed no in-flight sessions; claimed
|
||||
session **0011** (type coding) at `6cf1ae0`. Verified clean pushed `main`
|
||||
baseline + the `@~/.claude/wiggleverse.md` stub.
|
||||
2. **Read context.** `[[sub-project-3-player-progress]]` memory + the scales
|
||||
design (`2026-06-07-scales-library-and-right-axis-pipeline-design.md` §3 ring /
|
||||
§2 scales / §2.1 strict-PD map). Surveyed `simulator/` (`clips.py`, `app.py`,
|
||||
`setup_sample_media.py`, static), `player/` (content/controls/alteration/state),
|
||||
and the `test_clips.py` / `test_simulator_api.py` contracts.
|
||||
3. **Plan.** Wrote `docs/superpowers/plans/2026-06-07-scale-ring-navigation.md`
|
||||
(6 tasks). Key design choices: ordering index 0 = largest (cosmos), +1 zooms
|
||||
inward, both wrap; **one transition clip per edge**, forward inward / reversed
|
||||
outward (N transitions for N scales); ring math is canonical Python, the browser
|
||||
only renders; new scales carry a raw base only (Right re-bake deferred).
|
||||
4. **TDD Task 1 — `player/ring.py`.** Red `tests/test_player_ring.py` (12 cases:
|
||||
construction/validation, `scale_at` mod, single-step in/out, wrap both ways,
|
||||
multi-detent chaining, full-loop, no-op, degenerate N≤1) → implemented
|
||||
`Scale`/`Transition`/`ScaleRing`/`TransitionStep`/`RingMove`/`scale_at`/
|
||||
`advance_ring` → green.
|
||||
5. **TDD Task 2 — data layer.** Extended `tests/test_clips.py` → added
|
||||
`load_ring` + `ring_to_dict` (scales carry their clip title) +
|
||||
`ring_move_to_dict` (`target_clip_id`) to `simulator/clips.py` → green.
|
||||
6. **TDD Task 3 — API.** Extended `tests/test_simulator_api.py` → added
|
||||
`GET /api/ring` (404 if no ring) + `POST /api/ring/advance` to `simulator/app.py`
|
||||
(loads the ring into `app.state.ring`) → green.
|
||||
7. **Task 4 — media + manifest.** Rewrote the committed
|
||||
`simulator/sample_media/manifest.json`: added `cosmos` (NASA/Hubble) + `abyss`
|
||||
(NOAA Ocean Exploration) clips with true-PD provenance (17 U.S.C. §105 / NOAA),
|
||||
empty `right_variants` (deferral), illustrative annotations, and the `ring`
|
||||
section. Broadened `.gitignore` to `simulator/sample_media/**/*.mp4`. Wrote
|
||||
`simulator/setup_scales_media.py` — synthetic labelled placeholder bases +
|
||||
per-edge `xfade=zoomin` transitions. First run failed (xfade `-22`: forest base
|
||||
is 1920×1080@29.97 vs synthetic 1280×720@25) → fixed by normalizing both inputs
|
||||
(scale/fps/sar/format/trim) before xfade. Re-ran clean: 3 bases + 3 transitions.
|
||||
8. **Task 5 — frontend.** Rewrote `app.js` for multi-clip active-scale selection
|
||||
by ring index, an endless-encoder control (⊖ out / in ⊕ buttons + stage
|
||||
mouse-wheel), current-scale readout, and sequential transition playback that
|
||||
restores the live alteration on settle; added the control to `index.html` + CSS.
|
||||
9. **Verify.** Full suite **215 passed / 2 skipped** (+22 new). Booted the sim on
|
||||
:8011; `/api/ring`, `/api/ring/advance` (incl. the abyss→cosmos wrap), and
|
||||
`/media` all correct; headless-Chrome initial screenshot showed the cosmos
|
||||
placeholder + the ring control. Then a **CDP drive** (websockets) called
|
||||
`advance(1)` ×3, walking cosmos → forest → abyss → (wrap) → cosmos with the
|
||||
active video reloading `cosmos/base.mp4`; post-nav screenshot confirmed.
|
||||
10. **Docs.** ROADMAP slice-3 = done (+ folded the ring-transitions into the
|
||||
offline-pipeline bullet); USER_GUIDE gained the two-step media setup + the
|
||||
scale-ring gesture.
|
||||
11. **Ship.** Committed; pushed `feature/scale-ring-navigation`; created **PR #9**
|
||||
and merged it via the Gitea API helper (`gitea-api.sh git.benstull.org`, merge
|
||||
commit `435f201`, branch auto-deleted); synced `main`; deleted the local
|
||||
branch. Post-merge suite green. Updated memory + finalized.
|
||||
|
||||
## Cut state
|
||||
|
||||
- `main` @ `435f201` (PR #9 merged). New `player/ring.py` + `tests/test_player_ring.py`;
|
||||
`simulator/clips.py`/`app.py`/`static/*` extended; `simulator/setup_scales_media.py`
|
||||
+ manifest cosmos/abyss/ring; ROADMAP + USER_GUIDE updated. Full suite 215 passed /
|
||||
2 skipped. Working tree clean; local + remote feature branch deleted.
|
||||
- Sub-project 3 = **⏳ in progress**. Slices 1–3 done; remaining slices (runtime
|
||||
renderer, serial framing, white-noise, **offline v2v pipeline now incl. real ring
|
||||
transitions**, catalog model) in ROADMAP §3 + [[sub-project-3-player-progress]].
|
||||
- Scale-ring media is placeholder (gitignored); repopulate with
|
||||
`python simulator/setup_sample_media.py && python simulator/setup_scales_media.py`.
|
||||
|
||||
## Deferred decisions (operator plate)
|
||||
|
||||
Low-confidence / judgment calls made autonomously this session:
|
||||
|
||||
1. **Ring shape = cosmos → forest → abyss (3 scales).** The goal asked for 1–2 new
|
||||
true-PD clips; added two (cosmos, abyss) to the existing forest for a 3-scale
|
||||
ring — the minimum that exercises both an interior edge and the wrap seam.
|
||||
Forest sits between them as a stand-in for the terrestrial/mid scales (the
|
||||
design's PD soft spot, §2.1); real ordering (orbit/forest/reef/abyss/micro/cosmos)
|
||||
is an artistic call for when real footage is ingested.
|
||||
2. **One transition clip per edge, reversed for the outward direction.** Halves the
|
||||
bake count (N not 2N) and matches the design's "one clip per ring edge." The
|
||||
browser plays the placeholder forward in both directions for now (true reverse
|
||||
playback is a renderer concern); the `reversed` flag is carried through the API
|
||||
so the real pipeline / Pi can honor it.
|
||||
3. **"True-PD" = provenance recorded, bytes are labelled placeholders.** Media is
|
||||
gitignored, look-tuning-only, and real ingest is sub-project-2 territory (an
|
||||
open question in the design). The manifest records the real NASA/NOAA PD
|
||||
sources + license; `setup_scales_media.py` generates cheap synthetic stand-ins
|
||||
so the ring is always demonstrable offline. A `--fetch` real-download path was
|
||||
noted as a future bonus but not built (network-fragile; deterministic offline
|
||||
verification preferred).
|
||||
4. **Right axis on the new scales is a no-op (deferral).** New scales have empty
|
||||
`right_variants`; the `Clip` model already falls back any unauthored strength to
|
||||
the raw base, so this is a clean structural expression of "defer the expensive
|
||||
re-bake until the ring is liked."
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
Tune the ring by eye and decide whether the experience is "liked"; once it is, the
|
||||
big next build is the offline v2v pipeline (real Right re-bake **and** real ring
|
||||
transitions) + strict-PD ingest. Read [[sub-project-3-player-progress]] and the
|
||||
scales design §3/§4 first.
|
||||
|
||||
```
|
||||
/goal Tune the scale-ring experience in the simulator by eye — transition feel/length,
|
||||
scale ordering, the wrap moment, fast-spin behavior — and decide whether it's "liked."
|
||||
If liked, begin the offline v2v variant pipeline: the real multi-strength
|
||||
flow-stabilized Right re-bake per base clip PLUS real first-last-frame-i2v /
|
||||
infinite-zoom ring transitions to replace the placeholders, and source/ingest the
|
||||
actual strict-PD scale clips (NASA/NOAA/NPS) via sub-project 2. Keep deferring the
|
||||
Pi renderer + serial/firmware. Per docs/ROADMAP.md §3 and the scales design §3/§4.
|
||||
```
|
||||
|
||||
Note: PRs on this repo go through the **Gitea API** (`wgl-gitea-admin`
|
||||
`gitea-api.sh`, host `git.benstull.org`) — no `tea`/`gh` installed. Running from
|
||||
the main clone (not a nested worktree) avoided the 0010 resolver-ambiguity gotcha.
|
||||
@@ -1,113 +0,0 @@
|
||||
# Session 0012.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-08T00-05 (PST)
|
||||
> End: 2026-06-08T07-00 (PST)
|
||||
> Type: coding
|
||||
> Status: **FINALIZED.**
|
||||
> Outcome: two PRs merged to `main` — **#10 fast-spin blended pass** + **#11 v2v
|
||||
> vertical slice (real forest Right variants)**. Suite 228 passed / 2 skipped.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
`/goal next` → resumed the stored Next /goal from memory: **tune the ring
|
||||
experience by eye (transition feel/length, scale ordering, wrap moment,
|
||||
fast-spin behavior) and decide whether it's "liked"; once liked, the next big
|
||||
build is the offline v2v variant pipeline.**
|
||||
|
||||
## Plan
|
||||
|
||||
Goal frontier = tune/judge the scale ring (sub-project 3, scales design §3).
|
||||
|
||||
**Findings (early in session, after driving the sim + ring API):**
|
||||
- **Ring mechanics are sound & correct** — single step, wrap (correct reversed
|
||||
seam edge), multi-detent chaining, full-loop all verified via the canonical
|
||||
`/api/ring` + `/api/ring/advance`. UI renders the endless-encoder control +
|
||||
live RenderPlan readout.
|
||||
- **Aesthetic "feel/liked" is gated on real footage.** The current scale bases
|
||||
are near-black solid-color placeholders with text labels; transitions are
|
||||
ffmpeg `zoomin`-xfades. Transition feel / scale ordering / wrap-moment cannot
|
||||
be meaningfully judged by eye on these — that is exactly what the deferred v2v
|
||||
build (real strict-PD footage + real first-last-frame i2v transitions)
|
||||
produces. Placeholder by-eye tuning has a hard ceiling here.
|
||||
- **Fast-spin is the one genuine, footage-independent gap.** A multi-detent spin
|
||||
returns N steps and the browser plays ALL N full transitions sequentially
|
||||
(+5 ≈ 5 transitions ≈ 12–15 s of placeholder morphs). Design §3 explicitly
|
||||
anticipates this: "transitions chain, **or past a speed threshold a faster
|
||||
blended pass is used**." Not implemented.
|
||||
|
||||
**This session's work:**
|
||||
1. Implement the design §3 fast-spin policy (Python-canonical step-reduction +
|
||||
thin frontend), TDD. The one tuning dimension that is real, footage-
|
||||
independent, and named in the goal.
|
||||
2. Verify (suite + drive the ring + screenshot), update USER_GUIDE + design
|
||||
note + memory.
|
||||
3. Surface the genuine **"liked?" → greenlight v2v build** decision to the
|
||||
operator with this honest assessment + recommendation (the aesthetic sign-off
|
||||
is the artist's call; v2v also needs real footage sourcing, so it is the
|
||||
natural next session).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
|
||||
- **Fast-spin policy shape (PR #10).** Made fast-spin opt-in (`fast_spin_threshold`,
|
||||
default off) rather than default-on, so the pure `advance_ring` keeps its
|
||||
complete-chain contract + all existing tests, and the input layer (sim/firmware)
|
||||
— which actually knows wall-clock spin speed — enables it. Used `abs(delta)` as
|
||||
the spin-speed proxy (a spin batches detents into one advance call). Threshold
|
||||
default = 3 and the sim's blended playback rate = 2.5× are by-eye guesses, easily
|
||||
tuned. Confidence: high on the architecture, medium on the exact threshold/rate.
|
||||
- **Right strength curve (v2v slice).** Picked the keyframe-strength ramp per Right
|
||||
level by eye against the first real bake. First attempt was a linear 0.28–0.58 —
|
||||
but sd-turbo's painterly effect is sharply non-linear (barely registers <0.5,
|
||||
ramps hard through ~0.5–0.65), so levels 1–3 looked ~identical to raw. Re-baked
|
||||
with a clustered curve (0.46/0.52/0.58/0.64) for real perceptual progression.
|
||||
This is fundamentally an aesthetic + ML-param tuning loop (prompt/steps/strength)
|
||||
that wants the operator's eye; the curve is one-line tunable in
|
||||
`simulator/bake_right_variants.py` + re-bake. Confidence: medium — a sound,
|
||||
usable first real ramp, but the operator will likely want to fine-tune it (and
|
||||
possibly the prompt) before the look is locked.
|
||||
|
||||
## Session arc
|
||||
|
||||
1. **Opened** via `/goal next` → routed to a coding session, claimed ID 0012,
|
||||
synced the clean `main` baseline (transcripts live in this same repo under
|
||||
`sessions/`).
|
||||
2. **Oriented** on `player/ring.py`, the simulator, the manifest, and scales
|
||||
design §3/§4. Booted the sim and drove the ring via the canonical API +
|
||||
a headless screenshot.
|
||||
3. **Diagnosed** the goal honestly: ring *mechanics* are sound; the *aesthetic*
|
||||
judgment the goal asks for is gated on real footage (placeholders can't show
|
||||
it). The one footage-independent gap was **fast-spin**.
|
||||
4. **Built fast-spin** (TDD) — `advance_ring` opt-in `fast_spin_threshold`,
|
||||
serializer + API + thin frontend. Verified live (slow chains, fast collapses,
|
||||
correct wrap/arrival). PR #10, merged.
|
||||
5. **Surfaced the "liked?" gate** to the operator with the honest assessment
|
||||
(`AskUserQuestion`). Operator chose **vertical-slice the v2v build** — get
|
||||
real content for one scale to break the chicken-and-egg.
|
||||
6. **Built the v2v vertical slice** — wrote a plan, productionized the POC
|
||||
flow-restyle into `simulator/bake_right_variants.py` (curve TDD'd), baked real
|
||||
forest Right variants (~11 min MPS). First ramp was back-loaded (linear
|
||||
0.28–0.58 → levels 1–3 looked raw); re-baked with a clustered 0.46–0.64 curve;
|
||||
verified the perceptual ramp frame-by-frame. PR #11, merged.
|
||||
7. **Finalized** — memory updated, deferred decisions recorded, transcript
|
||||
published.
|
||||
|
||||
## Outcome & next session
|
||||
|
||||
The goal's "**decide if liked**" is now the operator's to make *with real footage
|
||||
in hand*: `make sim` → forest scale → sweep Right 0→4 to judge the dreamlike axis
|
||||
on real Yosemite footage. Mechanics (incl. fast-spin) are complete.
|
||||
|
||||
**Deferred (next):** fine-tune the forest curve/prompt if the look isn't liked;
|
||||
source real strict-PD bases for cosmos/abyss (sub-project 2 ingest) so their Right
|
||||
variants can bake; build the real **i2v ring transitions** (needs a generative
|
||||
video model + two real adjacent bases). Pi renderer + serial/firmware stay
|
||||
deferred per the simulator-first directive.
|
||||
|
||||
```
|
||||
/goal Judge the now-real forest Right (dreamlike) axis by eye (make sim → forest → sweep Right 0→4) and decide if the look + strength ramp is liked; fine-tune the curve/prompt in simulator/bake_right_variants.py + re-bake if not. Then continue the v2v build: source real strict-PD bases for cosmos/abyss (sub-project 2) and build the real i2v ring transitions. Per docs/ROADMAP.md + scales design §1/§3/§4.
|
||||
```
|
||||
@@ -1,115 +0,0 @@
|
||||
# Session 0013.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-22T08-34 (PST)
|
||||
> End: 2026-06-23T11-29 (PST)
|
||||
> Type: planning-and-executing
|
||||
> Posture: yolo (autonomous)
|
||||
> Claude-Session: f0a3044a-e9cb-467e-991f-f1ce618fc418
|
||||
> Status: **FINALIZED**
|
||||
> Lands: PR #12 → `main` (merge `23306fa`). Suite 233 passed / 2 skipped.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
"Let's keep working on the simulation. What's next? Figuring out the labeling for left
|
||||
brain HUD?" — opened against the stored 0012 next-goal (judge the real forest Right look,
|
||||
tune the v2v curve). Reconciled live: operator said "I liked the forest Right," which
|
||||
closed the judgment half and mooted the curve tuning, freeing the session for HUD + a
|
||||
Right-axis rethink.
|
||||
|
||||
## Pre-state
|
||||
|
||||
Through session 0012: engine (`player/`) + simulator built; scale ring (cosmos/forest/abyss);
|
||||
forest carried real flow-stabilized **pre-baked SD Right variants** (1–4); Left HUD was
|
||||
functional but never art-directed (plain cyan boxes); Dark/Light were two separate knobs with
|
||||
`mood_gain`/`overlay_gain` calibration sliders. 228 tests.
|
||||
|
||||
## Arc (turn by turn, wrong turns included)
|
||||
|
||||
1. **Session gate + claim.** Classified planning-and-executing; claimed ID 0013 from the main
|
||||
clone (no nested-worktree resolver issue this time). Clean `main` baseline.
|
||||
|
||||
2. **Left HUD art-direction pass.** Operator chose "look & feel first". Redesigned the analytical
|
||||
overlay: corner-bracket targeting reticles (capped arm length), translucent legible label
|
||||
chips, two sensor channels — cyan `detected.*` with deterministic per-key confidence scores
|
||||
vs amber `measure.*` with a center crosshair — and a bbox-sized `◉ ANALYSIS · L{n} · {k} OBJ`
|
||||
status tag that escalates with the Left knob. Judged by eye via headless-Chrome screenshots.
|
||||
Fixed two self-spotted flaws (status-tag clipping → measure from real `getBBox`; over-long
|
||||
reticle arms → cap).
|
||||
|
||||
3. **Affect / emotion channel (brainstormed → built).** Operator wanted emotions surfaced when
|
||||
Left AND Right are both up. Brainstormed: trigger = `min(left,right)`; soft ambient violet
|
||||
words (not boxed); stable per-scene palette, more words as combined strength rises. Built:
|
||||
`AffectOverlay(strength,intensity)` in the engine (math in Python), own SVG layer in the
|
||||
client, palettes in the manifest. 5 new unit tests. Design doc committed.
|
||||
|
||||
4. **mood_gain "does nothing" (not a bug).** Operator reported it. Traced: it's a gain on the
|
||||
Dark/Light grade — invisible with both at 0. Added a clarifying hint, then (later) removed the
|
||||
whole calibration UI anyway.
|
||||
|
||||
5. **Stale-cache saga (several rounds — the time sink).** "I don't see the emotions" / "Nothing
|
||||
is changing" — repeatedly diagnosed as the open tab running stale `app.js` (the live RenderPlan
|
||||
readout masked it). Added `no-cache` to the dev server; then a real `/dev/version` + 1 s client
|
||||
poll **live-reload**; then an on-screen error banner + crash-proof `update()`. Confirmed by the
|
||||
operator: "Works in incognito mode" — code was always correct; environment state was stale.
|
||||
|
||||
6. **Right axis reframe — the centerpiece.** Operator: the dreamlike knob is "too fluid... changes
|
||||
across the loop more than it should." Brainstormed the concept: Right should be a STILL altered
|
||||
state. Chose a **deterministic real-time filter** over the baked SD churn. Phase 1 (CSS luminous
|
||||
haze) built — then the operator clarified "same movement as the main video, just stylized": the
|
||||
haze read as blurred video, not stylized. Pivoted to **Phase 2: a WebGL2 Kuwahara painterly
|
||||
shader** on the live frames (edge-preserving → crisp motion, holds still across the loop). Caught
|
||||
and fixed a UV-mapping bug (lower-half streaking) mid-build. Operator: "That looks great. Let's
|
||||
get even a little more stylized. Max should be pretty trippy" → added saturation + ~45° hue-drift
|
||||
+ posterize, ramped by `intensity²` so only max goes psychedelic. "Perfect."
|
||||
|
||||
7. **Control surface 4→3 knobs.** Operator: make Dark/Light one dial; fold in mood_gain; drop
|
||||
overlay_gain. Collapsed Dark+Light into one bipolar Mood dial; removed the Calibration section
|
||||
(gains pinned to the locked `1.0` constants). Sim panel is now Left · Right · Mood.
|
||||
|
||||
8. **Processing-time question.** Operator asked compute/min for the coming content pipeline.
|
||||
Measured bundled ffmpeg at ~8× realtime (~10 s/min). Headline: the reframe killed the dominant
|
||||
per-clip cost (the SD bake) — per-minute compute is now trivial; the real cost is human authoring.
|
||||
|
||||
9. **Wrap.** Committed in three feature commits; PR #12 → `main`, merged; branch deleted; memory
|
||||
updated; transcript published.
|
||||
|
||||
## Cut state
|
||||
|
||||
`main` @ `23306fa`. 233 passed / 2 skipped. Sim runs locally (uvicorn :8000); Right dream is
|
||||
real-time WebGL (no bake); 3-knob control. Legacy `bake_right_variants.py` + baked `right*.mp4`
|
||||
parked, unused. Three design docs in `docs/superpowers/specs/` (affect channel; right-axis
|
||||
deterministic dream; + the Phase-1/2 staging recorded there).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **No formal implementation-plan artifact** — built by-eye in the fast iteration loop; the two new
|
||||
design docs are the record. No content repo configured (`CONTENT_REMOTE` empty), so nothing to
|
||||
`submit-plan.sh`; docs live in-repo and merged.
|
||||
- **Pipeline (§9):** no PPE/prod stage exists for the simulator (simulator-first phase) — merge to
|
||||
`main` is the completion; no deploy ran. Not a skipped gate; there is no stage yet.
|
||||
- **4→3 knob change** is a real experience-design change not yet reflected in the canonical design
|
||||
SPEC (still describes 4 knobs). Engine keeps dark/light internally. To record in the SPEC later.
|
||||
- **Legacy SD baker parked, not deleted** — kept for reference; can be removed once the real-time
|
||||
dream is fully settled.
|
||||
|
||||
## Operator plate (what the operator did / decided)
|
||||
|
||||
Drove every aesthetic call by eye: liked forest Right; chose HUD look-and-feel; specified emotions
|
||||
on Left+Right; rejected the "too fluid" SD dream and the blurred haze; approved the painterly
|
||||
direction and asked for a trippy max; asked to merge Dark/Light into one Mood dial and remove the
|
||||
calibration knobs; asked the processing-time question; directed commit + push + finalize and to
|
||||
start a fresh session for the content pipeline.
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
`/goal` Start the **content pipeline** for the installation: source full neutral video clips +
|
||||
processing + labeling. **Brainstorm/scope into a content-pipeline spec first**, deciding:
|
||||
(1) sourcing — which strict-PD clips, which scales (beyond cosmos/forest/abyss); reuse the 0008
|
||||
strict-PD map (NASA/Hubble, NOAA Ocean Exploration, NPS/USGS = true PD; "free stock" is NOT PD);
|
||||
(2) clip length + seamless-loop strategy; (3) labeling approach — STATIC boxes (current, 0 compute)
|
||||
vs MOTION-TRACKED (a detection+tracking pass), hand-authored vs detection-assisted, same for affect
|
||||
placement; (4) format target (res/fps/codec for the sim now + the Pi later). Anchor fact: the Right
|
||||
dream is **real-time, no per-clip bake**, so the pipeline is transcode + authoring, not ML baking.
|
||||
Read `memory/sub-project-3-player-progress.md` + the scales design
|
||||
(`docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-design.md` §1/§3/§4) first.
|
||||
@@ -1,129 +0,0 @@
|
||||
# Session 0014.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-24T06-58 (PST)
|
||||
> End: 2026-06-24T08-29 (PST)
|
||||
> Type: brainstorming
|
||||
> Posture: careful
|
||||
> Claude-Session: 50301aba-c534-4a7c-9f33-8b0881920ab2
|
||||
> Anchor: docs/ROADMAP.md sub-project 3 (Player Runtime) — content pipeline
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
Launch: /goal next → resolved from memory to the recorded Next /goal:
|
||||
START A NEW SESSION for the content pipeline — sourcing full neutral video clips + processing + labeling. Brainstorm/scope it into a content-pipeline spec FIRST, deciding: (1) sourcing (strict-PD clips, which scales); (2) clip length + seamless-loop strategy; (3) labeling approach (static vs motion-tracked; hand-authored vs detection-assisted; same for affect); (4) format target (res/fps/codec for sim now + Pi later). Anchor fact: Right dream is real-time, no per-clip bake → pipeline is transcode + authoring, NOT ML baking.
|
||||
```
|
||||
|
||||
## Plan
|
||||
|
||||
> Anchor: docs/ROADMAP.md sub-project 3 content pipeline. Roadmap-anchored repo
|
||||
> (no Gitea issue tracker used 0001–0013); content pipeline is one cohesive
|
||||
> Feature → design-eligible. See Deferred decisions for the R1 deviation.
|
||||
|
||||
**Goal:** Explore the content pipeline (source full neutral PD clips → process →
|
||||
label/author) and write its Solution-Design spec. Anchor fact: the Right dream is
|
||||
real-time (session 0013), so the pipeline is **transcode + human authoring**, not
|
||||
ML baking.
|
||||
|
||||
## What happened (arc)
|
||||
|
||||
This opened as `/goal next`; the session gate resolved the stored `Next /goal:`
|
||||
from memory (content pipeline) and routed to **brainstorming** (the goal said
|
||||
"brainstorm/scope it into a spec first"). Init claimed session **0014** (careful
|
||||
posture, the brainstorming default), no concurrent sessions, clean `main`.
|
||||
|
||||
**Orientation.** Read the roadmap (sub-project 3), the scales-library/right-axis
|
||||
design (§1/§3/§4), the two session-0013 specs (real-time dream supersedes the SD
|
||||
bake; affect channel), and the current content shape (`simulator/clips.py`,
|
||||
`manifest.json`, `setup_scales_media.py`). Confirmed the key simplifier: the Right
|
||||
dream is real-time, so the pipeline is transcode + authoring, **not** ML baking.
|
||||
|
||||
**Brainstorming (careful, agree-up-front → draft-whole → review-whole).** Walked
|
||||
the four launch-prompt decisions as `AskUserQuestion` checkpoints:
|
||||
1. Scale set → **lean 5** (cosmos · orbit · forest/land · reef · abyss; microscopic
|
||||
dropped — thinnest PD-video pool).
|
||||
2. Loop → **crossfade loop** (tail→head overlap).
|
||||
3. Labels → **motion-tracked, HYBRID** (ML proposes where it can; hand-seed +
|
||||
classical tracker elsewhere). Surfaced that label *semantics* must always be
|
||||
hand-authored (a generic detector can't produce "siphonophore"/"z ≈ 0.04");
|
||||
tracking only propagates box geometry.
|
||||
4. Format → **master + 1080p H.264 proxy**.
|
||||
|
||||
Drafted the full spec `docs/superpowers/specs/2026-06-24-content-pipeline-design.md`,
|
||||
pushed to `feature/content-pipeline`, handed over the rendered Gitea URL for
|
||||
whole-document review. Operator chose **"Approve + plan now"**.
|
||||
|
||||
**Planning.** Wrote `docs/superpowers/plans/2026-06-24-content-pipeline-increment-1.md`
|
||||
(Increment 1 = real footage, no ML): 9 bite-sized TDD tasks. Self-reviewed against
|
||||
the spec (coverage, types, one dead-variable fix). Operator chose **"Execute now
|
||||
(subagent-driven)"**.
|
||||
|
||||
**Execution (subagent-driven, 2-stage review per group).** Implementer + reviewer
|
||||
subagents (sonnet), grouped by file coupling:
|
||||
- Tasks 1–3 (`tools/pipeline/ffmpeg_ops.py` pure builders) — DONE; review added an
|
||||
`overlap<=0` test.
|
||||
- Tasks 4–5 (`provenance.py`, `manifest.py`) — DONE; review added append-fallback +
|
||||
N=1 self-wrap tests.
|
||||
- Task 6 (`run.py` runner + integration test) — the test gated on `shutil.which`
|
||||
and SKIPPED (ffmpeg not on PATH). Increment 1's goal is to *prove the pipeline on
|
||||
real footage*, so I hardened the runner to resolve the **bundled imageio-ffmpeg**
|
||||
(matching `setup_scales_media.py`) and switched duration probing to **cv2** (no
|
||||
ffprobe dep — Pi-friendly). Re-run **actually exercised real ffmpeg on the real
|
||||
forest base and surfaced a genuine bug**: `trim` yields `1/0` framerate which
|
||||
`xfade` rejects (needs CFR) — fixed with `fps=30` after each trim segment. Test
|
||||
now PASSES (proxy 1920×1080 verified via cv2). Closed an ffprobe-gap minor too.
|
||||
- Task 7 (client annotation-track interpolation, `app.js`) — DONE; `node --check`
|
||||
clean, API serves the track. Visual drift deferred to operator by-eye.
|
||||
- Task 8 (ring 3→5: orbit + reef placeholders) — DONE; `/api/ring` shows 5 scales
|
||||
+ 5 edges. Ring spin deferred to operator by-eye.
|
||||
- Task 9 (sourcing doc + ROADMAP + stale-docstring fix) — DONE.
|
||||
|
||||
Final whole-branch review: **READY TO MERGE** (no critical/blocking). 246 passed /
|
||||
2 skipped. Operator chose **"Merge to main now"** → pushed, created **PR #13** via
|
||||
the Gitea API, merged (delete branch), synced main, deleted local branch. Stamped
|
||||
the spec **graduated** on main.
|
||||
|
||||
## Outcome
|
||||
|
||||
- Content-pipeline **spec** (graduated) + **Increment-1 plan** (archived in-repo)
|
||||
+ **Increment 1 implementation** all merged to `main` (PR #13, merge `bb5ecf0`;
|
||||
spec stamp `73e5c4a`). Suite **246 passed / 2 skipped**.
|
||||
- New `tools/pipeline/` package; keyframed annotation-`track` schema
|
||||
(client-interpolated, backward compatible); scale ring extended 3→5.
|
||||
- `docs/content-sourcing.md` (strict-PD sourcing procedure); ROADMAP updated.
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **R1 anchor (no Gitea issue):** the brainstorming gate wants a non-`epic`
|
||||
`type/*` tracker issue as the design anchor. This repo has never used a Gitea
|
||||
issue tracker (roadmap-driven, 0001–0013), so the spec is anchored to
|
||||
`docs/ROADMAP.md` sub-project 3 — matching every prior brainstorming session
|
||||
here (0005/0007/0008/0013). Flagged rather than force-filing an issue that would
|
||||
break established repo practice; operator can request issue-tracking if wanted.
|
||||
- **Session-type blur (operator-directed):** a brainstorming session normally ends
|
||||
at the spec; the operator explicitly extended it through plan → execute → merge.
|
||||
Each transition was an explicit operator choice via `AskUserQuestion`, so the
|
||||
careful-posture gates were honored, not bypassed.
|
||||
- **Crossfade duration source (minor, non-blocking):** `process_clip` probes the
|
||||
duration from the *source*, not the post-frame mezzanine; GOP alignment can make
|
||||
the tail segment ±~33 ms off. ffmpeg handles EOF gracefully; imperceptible on
|
||||
calm footage. Noted for a future fast-motion clip.
|
||||
|
||||
## Pending for next session
|
||||
|
||||
- **Operator by-eye review of Increment 1** in the sim: the track drift
|
||||
(water/fish boxes follow playback + loop seamlessly) and spinning the 5-scale
|
||||
ring cosmos→orbit→forest→reef→abyss→wrap. A dev sim (session-0013 live-reload)
|
||||
was running on :8000 during the session.
|
||||
|
||||
## Next /goal
|
||||
|
||||
```
|
||||
/goal content-pipeline Increment 2 — source real strict-PD bases per docs/content-sourcing.md (run each through tools/pipeline/run.py process_clip) + build the hybrid motion-track pass tools/pipeline/track.py + a simulator author mode; FIRST do the pending by-eye review of Increment 1. Per docs/superpowers/specs/2026-06-24-content-pipeline-design.md
|
||||
```
|
||||
|
||||
> Still-deferred (tracked in `docs/ROADMAP.md`, this repo's queue): real i2v ring
|
||||
> transitions (need a generative-video model + adjacent real bases); Pi renderer +
|
||||
> serial/firmware (simulator-first).
|
||||
@@ -1,85 +0,0 @@
|
||||
# Session 0015.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-24T13-16 (PST)
|
||||
> Type: planning-and-executing
|
||||
> Posture: yolo
|
||||
> Claude-Session: 50301aba-c534-4a7c-9f33-8b0881920ab2
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0015 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0015.0-TRANSCRIPT-2026-06-24T13-16--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
Process 5 candidate ORBIT (Earth-from-space) clips for the human-experience-filter art installation: resolve URLs, download, vet for text + humans, trim to a word-free calm window, transcode. Clips: orbit_planetearth, orbit_crewobs, orbit_aurora_perth (SVS 31281), orbit_westcoast (SVS 30180), orbit_bluemarble (SVS 4550).
|
||||
|
||||
```
|
||||
|
||||
## Plan
|
||||
|
||||
Process 5 candidate ORBIT (Earth-from-space) clips: resolve URLs, download, vet for text/telemetry/humans via contact sheets, trim to word-free 24s window, transcode via process_clip.
|
||||
|
||||
Tasks:
|
||||
1. Resolve direct mp4 URLs for SVS pages (aurora_perth/31281, westcoast/30180, bluemarble/4550)
|
||||
2. Download all 5 clips
|
||||
3. Build contact sheets (overview + first-8s + last-8s) for each clip
|
||||
4. Vet each clip for text overlays, telemetry, ISS hardware, humans
|
||||
5. Determine clean window or mark UNUSABLE
|
||||
6. process_clip → simulator/sample_media/<id>/ for usable clips
|
||||
7. Generate preview frames
|
||||
|
||||
## Clip results
|
||||
|
||||
### orbit_planetearth
|
||||
- URL: https://svs.gsfc.nasa.gov/vis/a030000/a030700/a030771/ISS_View_of_Planet_Earth_2160p.mp4
|
||||
- Duration: 97.2s
|
||||
- Text/telemetry: "INTERNATIONAL SPACE STATION" title card t=0–8s; ISS logo end card t≈80–97s
|
||||
- Humans: none
|
||||
- Window: 10–34s (24s, clean daytime Earth clouds)
|
||||
- Status: USABLE
|
||||
- Output: simulator/sample_media/orbit_planetearth/
|
||||
- Preview: /tmp/hef-cand/orbit_planetearth/preview.png
|
||||
|
||||
### orbit_crewobs
|
||||
- URL: https://svs.gsfc.nasa.gov/vis/a030000/a030700/a030771/ISS_Crew_Earth_Observations_2160p.mp4
|
||||
- Duration: 66.4s
|
||||
- Text/telemetry: "INTERNATIONAL SPACE STATION" + date/location title card t=0–8s; ISS logo end card t≈55–66s
|
||||
- Humans: none
|
||||
- Window: 9–33s (24s, clean desert/cloud Earth)
|
||||
- Status: USABLE
|
||||
- Output: simulator/sample_media/orbit_crewobs/
|
||||
- Preview: /tmp/hef-cand/orbit_crewobs/preview.png
|
||||
|
||||
### orbit_aurora_perth
|
||||
- URL: https://svs.gsfc.nasa.gov/vis/a030000/a031200/a031281/ISS067_20220817_aurora_1080p25.mp4
|
||||
- Duration: 49.6s
|
||||
- Text/telemetry: Date+timestamp burned in top-left throughout (e.g. "2022-08-17 19:15:25", updating every second)
|
||||
- Humans: none
|
||||
- Window: none (telemetry throughout)
|
||||
- Status: UNUSABLE
|
||||
|
||||
### orbit_westcoast
|
||||
- URL: https://svs.gsfc.nasa.gov/vis/a030000/a030100/a030180/iss028_nighttime_20110819_720p.mp4
|
||||
- Duration: 54.7s
|
||||
- Text/telemetry: none
|
||||
- Humans: none
|
||||
- ISS hardware: cylindrical ISS module + solar panel grid visible in top-right corner throughout entire clip (confirmed by exhaustive per-second crop inspection t=14–44s)
|
||||
- Window: none (ISS hardware throughout)
|
||||
- Status: UNUSABLE — orbit_westcoast output dir deleted
|
||||
|
||||
### orbit_bluemarble
|
||||
- URL: https://svs.gsfc.nasa.gov/vis/a000000/a004500/a004550/BlueMarble_starfield_4k_2160p30_v2.mp4
|
||||
- Duration: 121.0s
|
||||
- Text/telemetry: none (CG visualization, no overlays anywhere)
|
||||
- Humans: none
|
||||
- Window: 10–34s (24s, rotating Earth globe on starfield)
|
||||
- Status: USABLE
|
||||
- Output: simulator/sample_media/orbit_bluemarble/
|
||||
- Preview: /tmp/hef-cand/orbit_bluemarble/preview.png
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- orbit_westcoast output directory (simulator/sample_media/orbit_westcoast/) was written before the ISS hardware was confirmed throughout. It needs to be deleted. The rm -rf was attempted but permission was denied in the sandbox — operator should delete it manually: `rm -rf simulator/sample_media/orbit_westcoast`
|
||||
@@ -16,32 +16,5 @@
|
||||
},
|
||||
"0006": {
|
||||
"title": ""
|
||||
},
|
||||
"0007": {
|
||||
"title": ""
|
||||
},
|
||||
"0008": {
|
||||
"title": ""
|
||||
},
|
||||
"0009": {
|
||||
"title": ""
|
||||
},
|
||||
"0010": {
|
||||
"title": ""
|
||||
},
|
||||
"0011": {
|
||||
"title": ""
|
||||
},
|
||||
"0012": {
|
||||
"title": ""
|
||||
},
|
||||
"0013": {
|
||||
"title": ""
|
||||
},
|
||||
"0014": {
|
||||
"title": ""
|
||||
},
|
||||
"0015": {
|
||||
"title": ""
|
||||
}
|
||||
}
|
||||
|
||||
+61
-132
@@ -1,163 +1,92 @@
|
||||
"""FastAPI service: controls -> the real alteration engine -> a RenderPlan.
|
||||
|
||||
The simulator's alteration surface (reconciled slice). It calls the canonical
|
||||
player.alteration.plan_alteration; the browser only renders. The selection-era
|
||||
endpoints (/api/select, /api/catalog/meta) and the X-ray are retired.
|
||||
"""
|
||||
"""FastAPI service: dials -> real hef.selection -> X-ray (pick + ranked pool)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import time
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from typing import Literal, Optional
|
||||
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from fastapi import FastAPI
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import (
|
||||
DEFAULT_CALIBRATION,
|
||||
Calibration,
|
||||
plan_alteration,
|
||||
render_plan_to_dict,
|
||||
from hef.catalog import load_catalog, record_to_dict
|
||||
from hef.selection import (
|
||||
Coordinate,
|
||||
Weights,
|
||||
candidates_for_mode,
|
||||
ranked_candidates,
|
||||
select,
|
||||
)
|
||||
from player.content import resolve_content
|
||||
from player.controls import CONTENT_POSITIONS
|
||||
from player.ring import DEFAULT_FAST_SPIN_THRESHOLD, advance_ring
|
||||
from simulator.clips import load_manifest, load_ring, ring_move_to_dict, ring_to_dict
|
||||
from simulator.fixtures import generate_fixture_catalog
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "static"
|
||||
MEDIA_DIR = Path(__file__).parent / "sample_media"
|
||||
DEFAULT_MANIFEST = MEDIA_DIR / "manifest.json"
|
||||
|
||||
# Per-process boot token: changes on every server restart so the dev live-reload
|
||||
# (below) also fires when Python code changes (which needs a restart), not just
|
||||
# when a static asset's mtime changes.
|
||||
_BOOT_ID = f"{os.getpid()}-{int(time.time())}"
|
||||
|
||||
|
||||
def _asset_version() -> str:
|
||||
"""A short digest of the renderer assets' mtimes + this process's boot token.
|
||||
Changes whenever app.js/style.css/index.html change on disk OR the server is
|
||||
restarted — the signal the browser polls to know it's running stale code."""
|
||||
parts = [_BOOT_ID]
|
||||
for name in ("app.js", "style.css", "index.html"):
|
||||
try:
|
||||
parts.append(f"{name}:{(STATIC_DIR / name).stat().st_mtime_ns}")
|
||||
except OSError:
|
||||
parts.append(f"{name}:?")
|
||||
return hashlib.sha1("|".join(parts).encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
class ControlsModel(BaseModel):
|
||||
content: str
|
||||
class SelectRequest(BaseModel):
|
||||
left: int = Field(ge=0, le=4)
|
||||
right: int = Field(ge=0, le=4)
|
||||
dark: int = Field(ge=0, le=4)
|
||||
light: int = Field(ge=0, le=4)
|
||||
volume: int = Field(ge=0, le=4)
|
||||
brightness: int = Field(ge=0, le=4)
|
||||
mode: Literal["none", "audio", "video", "av"]
|
||||
pool_size: int = Field(default=4, ge=1, le=25)
|
||||
brain_weight: float = Field(default=1.0, ge=0.0)
|
||||
mood_weight: float = Field(default=1.0, ge=0.0)
|
||||
approved_only: bool = False
|
||||
|
||||
|
||||
class CalibrationModel(BaseModel):
|
||||
mood_gain: float = 1.0
|
||||
overlay_gain: float = 1.0
|
||||
dream_gain: float = 1.0
|
||||
def load_catalog_or_fixtures() -> list:
|
||||
"""Use the real catalog if a non-empty one is configured/exists, else fixtures."""
|
||||
configured = os.environ.get("HEF_SIM_CATALOG")
|
||||
path = Path(configured) if configured else Path("catalog/library.jsonl")
|
||||
if path.exists() and path.stat().st_size > 0:
|
||||
return load_catalog(path)
|
||||
return generate_fixture_catalog()
|
||||
|
||||
|
||||
class AlterationRequest(BaseModel):
|
||||
controls: ControlsModel
|
||||
calibration: Optional[CalibrationModel] = None
|
||||
def create_app(records: Optional[list] = None) -> FastAPI:
|
||||
app = FastAPI(title="HEF Experience Simulator")
|
||||
app.state.catalog = records if records is not None else load_catalog_or_fixtures()
|
||||
|
||||
|
||||
class RingAdvanceRequest(BaseModel):
|
||||
from_index: int = 0
|
||||
delta: int
|
||||
|
||||
|
||||
def _load_clips(manifest_path: Optional[Path]):
|
||||
path = Path(manifest_path) if manifest_path else DEFAULT_MANIFEST
|
||||
if path.exists():
|
||||
return load_manifest(path)
|
||||
return []
|
||||
|
||||
|
||||
def _load_ring(manifest_path: Optional[Path]):
|
||||
path = Path(manifest_path) if manifest_path else DEFAULT_MANIFEST
|
||||
if path.exists():
|
||||
return load_ring(path)
|
||||
return None
|
||||
|
||||
|
||||
def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
|
||||
app = FastAPI(title="HEF Alteration Simulator")
|
||||
app.state.clips = _load_clips(manifest_path)
|
||||
app.state.ring = _load_ring(manifest_path)
|
||||
|
||||
@app.post("/api/alteration")
|
||||
def api_alteration(req: AlterationRequest):
|
||||
c = req.controls
|
||||
if c.content not in CONTENT_POSITIONS:
|
||||
raise HTTPException(status_code=422, detail=f"invalid content {c.content!r}")
|
||||
coord = Coordinate(c.left, c.right, c.dark, c.light)
|
||||
cal = (
|
||||
Calibration(
|
||||
mood_gain=req.calibration.mood_gain,
|
||||
overlay_gain=req.calibration.overlay_gain,
|
||||
dream_gain=req.calibration.dream_gain,
|
||||
)
|
||||
if req.calibration
|
||||
else DEFAULT_CALIBRATION
|
||||
@app.post("/api/select")
|
||||
def api_select(req: SelectRequest):
|
||||
catalog = app.state.catalog
|
||||
coord = Coordinate(req.left, req.right, req.dark, req.light)
|
||||
weights = Weights(brain=req.brain_weight, mood=req.mood_weight)
|
||||
if req.mode == "none":
|
||||
return {"pick": None, "pool": [], "coverage": {"candidates_in_mode": 0}}
|
||||
pool = catalog
|
||||
if req.approved_only:
|
||||
pool = [r for r in pool if r.review_status == "approved"]
|
||||
eligible = candidates_for_mode(pool, req.mode, req.pool_size)
|
||||
ranked = ranked_candidates(
|
||||
catalog, coord, req.mode,
|
||||
pool_size=req.pool_size, weights=weights, approved_only=req.approved_only,
|
||||
)
|
||||
pick = select(
|
||||
catalog, coord, req.mode,
|
||||
pool_size=req.pool_size, weights=weights, approved_only=req.approved_only,
|
||||
rng=None,
|
||||
)
|
||||
plan = plan_alteration(coord, cal)
|
||||
content = resolve_content(c.content)
|
||||
return {
|
||||
"plan": render_plan_to_dict(plan),
|
||||
"content": {"audio_source": content.audio_source, "video": content.video},
|
||||
"pick": record_to_dict(pick) if pick else None,
|
||||
"pool": [
|
||||
{"record": record_to_dict(r), "distance": d, "rank": i + 1}
|
||||
for i, (r, d) in enumerate(ranked)
|
||||
],
|
||||
"coverage": {"candidates_in_mode": len(eligible)},
|
||||
}
|
||||
|
||||
@app.get("/dev/version")
|
||||
def dev_version():
|
||||
# Dev live-reload signal: the browser polls this and reloads when it
|
||||
# changes, so it never silently runs a stale renderer during iteration.
|
||||
return {"version": _asset_version()}
|
||||
@app.get("/api/catalog/meta")
|
||||
def api_meta():
|
||||
catalog = app.state.catalog
|
||||
return {
|
||||
"total": len(catalog),
|
||||
"by_mode": dict(Counter(r.mode for r in catalog)),
|
||||
"by_status": dict(Counter(r.review_status for r in catalog)),
|
||||
}
|
||||
|
||||
@app.get("/api/clips")
|
||||
def api_clips():
|
||||
return {"clips": [c.to_dict() for c in app.state.clips]}
|
||||
|
||||
@app.get("/api/ring")
|
||||
def api_ring():
|
||||
if app.state.ring is None:
|
||||
raise HTTPException(status_code=404, detail="no scale ring in manifest")
|
||||
return ring_to_dict(app.state.ring, app.state.clips)
|
||||
|
||||
@app.post("/api/ring/advance")
|
||||
def api_ring_advance(req: RingAdvanceRequest):
|
||||
if app.state.ring is None:
|
||||
raise HTTPException(status_code=404, detail="no scale ring in manifest")
|
||||
move = advance_ring(
|
||||
app.state.ring,
|
||||
req.from_index,
|
||||
req.delta,
|
||||
fast_spin_threshold=DEFAULT_FAST_SPIN_THRESHOLD,
|
||||
)
|
||||
return ring_move_to_dict(move, app.state.ring)
|
||||
|
||||
@app.middleware("http")
|
||||
async def _no_cache(request, call_next):
|
||||
# Dev preview server: never let a browser serve a stale app.js/style.css.
|
||||
# The whole point is by-eye iteration, so a plain refresh must always get
|
||||
# the latest assets (the heuristic cache otherwise hides JS/CSS edits).
|
||||
response = await call_next(request)
|
||||
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
|
||||
return response
|
||||
|
||||
if MEDIA_DIR.exists():
|
||||
app.mount("/media", StaticFiles(directory=MEDIA_DIR), name="media")
|
||||
if STATIC_DIR.exists():
|
||||
app.mount("/", StaticFiles(directory=STATIC_DIR, html=True), name="static")
|
||||
|
||||
|
||||
@@ -1,193 +0,0 @@
|
||||
"""Offline baker: real flow-stabilized painterly Right-axis variants (scales
|
||||
design §1/§4).
|
||||
|
||||
Productionizes the session-0008 POC (``~/hef-poc/flow_restyle.py``): SD img2img
|
||||
on KEYFRAMES + Farneback optical-flow warp on in-between frames = a temporally
|
||||
calm painterly restyle (no per-frame "boiling" — the look the operator approved
|
||||
in 0008). The one new idea on top of the POC is a **strength curve**: each Right
|
||||
level 1-4 maps to an increasing keyframe img2img ``strength`` so the four
|
||||
discrete pre-baked variants form a subtle -> strong dreamlike ramp. Level 0 is
|
||||
always the raw base (handled by the runtime, not baked here).
|
||||
|
||||
The curve (``right_strength_params``) is pure + unit-tested; the rendered media
|
||||
is verified by eye (like the other ``simulator/setup_*_media.py`` scripts). torch
|
||||
/ diffusers are imported lazily inside the bake functions so importing this
|
||||
module (e.g. for the curve test) is fast and dependency-light.
|
||||
|
||||
Usage:
|
||||
.venv/bin/python -m simulator.bake_right_variants [--scale forest]
|
||||
[--fps 12] [--keyint 24]
|
||||
|
||||
Requires: torch + Apple MPS, diffusers, ``stabilityai/sd-turbo`` (cached),
|
||||
opencv-python, imageio-ffmpeg. ~2.7 min/clip x 4 levels ~= ~11 min local compute.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import glob
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
MEDIA = Path(__file__).parent / "sample_media"
|
||||
|
||||
RIGHT_LEVELS = (1, 2, 3, 4)
|
||||
|
||||
# Keyframe img2img strength per Right level (by-eye ramp). sd-turbo's painterly
|
||||
# transformation is sharply NON-LINEAR in strength — it barely registers below
|
||||
# ~0.5 and ramps hard through ~0.5-0.65 (verified by eye, session 0012: a linear
|
||||
# 0.28-0.58 ramp left levels 1-3 ~indistinguishable from raw). So the curve is
|
||||
# clustered in that active band for real perceptual progression: a gentle dream
|
||||
# haze (L1) -> clearly painterly (L3, ~the POC's proven look) -> strongest (L4).
|
||||
# Still by-eye tunable — re-run the baker after editing.
|
||||
_KEY_STRENGTH = {1: 0.46, 2: 0.52, 3: 0.58, 4: 0.64}
|
||||
_TWEEN_RATIO = 0.36 # POC ratio (0.18 / 0.5): tweens are a light refine only.
|
||||
|
||||
# The restyle target (matches the POC spike that the operator approved).
|
||||
PROMPT = (
|
||||
"impressionist oil painting, dreamlike, soft painterly brushstrokes, "
|
||||
"ethereal, luminous, fine art"
|
||||
)
|
||||
NEG = "text, watermark, frame, border, ugly, deformed, lowres"
|
||||
SEED = 42
|
||||
|
||||
|
||||
def right_strength_params(level: int) -> tuple[float, float]:
|
||||
"""(keyframe_strength, tween_strength) for a Right level in RIGHT_LEVELS."""
|
||||
if level not in _KEY_STRENGTH:
|
||||
raise ValueError(f"Right level must be one of {RIGHT_LEVELS}, got {level}")
|
||||
key = _KEY_STRENGTH[level]
|
||||
return key, round(key * _TWEEN_RATIO, 4)
|
||||
|
||||
|
||||
# --- the restyle engine (verbatim port of ~/hef-poc/flow_restyle.py) ---
|
||||
|
||||
|
||||
def _ffmpeg() -> str:
|
||||
import imageio_ffmpeg
|
||||
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
|
||||
|
||||
def _run(cmd: list[str]) -> None:
|
||||
subprocess.run(cmd, check=True, capture_output=True)
|
||||
|
||||
|
||||
def _extract(ff: str, src: str, d: str, fps: float) -> list[str]:
|
||||
_run([ff, "-hide_banner", "-v", "error", "-i", src, "-vf", f"fps={fps}",
|
||||
os.path.join(d, "f_%05d.png"), "-y"])
|
||||
return sorted(glob.glob(os.path.join(d, "f_*.png")))
|
||||
|
||||
|
||||
def _warp(prev_bgr, src_prev_gray, src_cur_gray):
|
||||
"""Warp the previous stylized frame to align with the current source frame
|
||||
via Farneback optical flow — this is what kills the per-frame flicker."""
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
flow = cv2.calcOpticalFlowFarneback(
|
||||
src_prev_gray, src_cur_gray, None,
|
||||
pyr_scale=0.5, levels=3, winsize=21, iterations=3,
|
||||
poly_n=7, poly_sigma=1.5, flags=0)
|
||||
h, w = src_cur_gray.shape
|
||||
gx, gy = np.meshgrid(np.arange(w), np.arange(h))
|
||||
mx = (gx + flow[..., 0]).astype(np.float32)
|
||||
my = (gy + flow[..., 1]).astype(np.float32)
|
||||
return cv2.remap(prev_bgr, mx, my, cv2.INTER_LINEAR, borderMode=cv2.BORDER_REFLECT)
|
||||
|
||||
|
||||
def _load_pipe(model: str):
|
||||
"""Load the SD img2img pipeline once (reused across all levels)."""
|
||||
import torch
|
||||
from diffusers import AutoPipelineForImage2Image
|
||||
|
||||
dev = "mps" if torch.backends.mps.is_available() else "cpu"
|
||||
pipe = AutoPipelineForImage2Image.from_pretrained(
|
||||
model, torch_dtype=torch.float16 if dev == "mps" else torch.float32,
|
||||
safety_checker=None).to(dev)
|
||||
return pipe, dev
|
||||
|
||||
|
||||
def restyle_clip(pipe, dev, src: str, out: str, key_strength: float,
|
||||
tween_strength: float, *, fps: float = 12.0, keyint: int = 24) -> None:
|
||||
"""Restyle one clip: SD img2img keyframes + flow-warped light-refine tweens."""
|
||||
import cv2
|
||||
import numpy as np
|
||||
import torch
|
||||
from PIL import Image
|
||||
|
||||
ff = _ffmpeg()
|
||||
gen = torch.Generator(device=dev).manual_seed(SEED)
|
||||
|
||||
def img2img(pil, strength):
|
||||
steps = max(2, int(np.ceil(1.0 / max(strength, 0.05))) + 1)
|
||||
return pipe(prompt=PROMPT, negative_prompt=NEG, image=pil,
|
||||
strength=strength, guidance_scale=0.0,
|
||||
num_inference_steps=steps, generator=gen).images[0]
|
||||
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
frames = _extract(ff, src, d, fps)
|
||||
print(f" [frames] {len(frames)} key_s={key_strength} tween_s={tween_strength}")
|
||||
prev_styled = None # BGR uint8
|
||||
prev_src_gray = None
|
||||
t0 = time.time()
|
||||
for i, fp in enumerate(frames):
|
||||
src_bgr = cv2.imread(fp)
|
||||
h0, w0 = src_bgr.shape[:2]
|
||||
scale = 768 / max(h0, w0)
|
||||
wh = (int(w0 * scale) // 8 * 8, int(h0 * scale) // 8 * 8)
|
||||
src_small = cv2.resize(src_bgr, wh)
|
||||
src_gray = cv2.cvtColor(src_small, cv2.COLOR_BGR2GRAY)
|
||||
|
||||
if i % keyint == 0 or prev_styled is None:
|
||||
pil = Image.fromarray(cv2.cvtColor(src_small, cv2.COLOR_BGR2RGB))
|
||||
out_img = img2img(pil, key_strength)
|
||||
else:
|
||||
warped = _warp(prev_styled, prev_src_gray, src_gray)
|
||||
pil = Image.fromarray(cv2.cvtColor(warped, cv2.COLOR_BGR2RGB))
|
||||
out_img = img2img(pil, tween_strength) # light refine only
|
||||
styled = cv2.cvtColor(np.array(out_img), cv2.COLOR_RGB2BGR)
|
||||
prev_styled, prev_src_gray = styled, src_gray
|
||||
cv2.imwrite(os.path.join(d, f"s_{i+1:05d}.png"),
|
||||
cv2.resize(styled, (w0, h0)))
|
||||
if i % 12 == 0:
|
||||
print(f" frame {i+1}/{len(frames)} {time.time()-t0:.1f}s")
|
||||
|
||||
_run([ff, "-hide_banner", "-v", "error", "-framerate", str(fps),
|
||||
"-i", os.path.join(d, "s_%05d.png"), "-c:v", "libx264", "-crf", "20",
|
||||
"-preset", "medium", "-pix_fmt", "yuv420p", out, "-y"])
|
||||
print(f" [done] {time.time()-t0:.1f}s -> {out}")
|
||||
|
||||
|
||||
def bake_scale(scale: str = "forest", *, fps: float = 12.0, keyint: int = 24,
|
||||
model: str = "stabilityai/sd-turbo") -> None:
|
||||
"""Bake all RIGHT_LEVELS variants for one scale's base clip."""
|
||||
base = MEDIA / scale / "base.mp4"
|
||||
if not base.exists():
|
||||
raise FileNotFoundError(
|
||||
f"no base clip at {base} — run the media-setup scripts first")
|
||||
print(f"[bake] scale={scale} base={base}")
|
||||
pipe, dev = _load_pipe(model)
|
||||
print(f"[device] {dev}")
|
||||
for level in RIGHT_LEVELS:
|
||||
key_s, tween_s = right_strength_params(level)
|
||||
out = MEDIA / scale / f"right{level}.mp4"
|
||||
print(f" [right {level}] -> {out}")
|
||||
restyle_clip(pipe, dev, str(base), str(out), key_s, tween_s,
|
||||
fps=fps, keyint=keyint)
|
||||
print(f"[bake] {scale} done — {len(RIGHT_LEVELS)} real Right variants")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("--scale", default="forest")
|
||||
ap.add_argument("--fps", type=float, default=12.0)
|
||||
ap.add_argument("--keyint", type=int, default=24)
|
||||
a = ap.parse_args()
|
||||
bake_scale(a.scale, fps=a.fps, keyint=a.keyint)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,131 +0,0 @@
|
||||
"""The base-clip + variant + annotation manifest the simulator renders.
|
||||
|
||||
Replaces simulator/fixtures.py (the selection-era synthetic catalog). Each base
|
||||
clip carries: the raw base file, a map of pre-baked Right-strength variant files
|
||||
(strength 0 is always the raw base), an authored Left annotation track (box +
|
||||
label key + the minimum Left level at which it appears), and per-language string
|
||||
tables. See the reconciled-simulator-alteration-slice design §3.2.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from player.ring import RingMove, Scale, ScaleRing, Transition, scale_at
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Clip:
|
||||
id: str
|
||||
title: str
|
||||
base_file: str
|
||||
license: str
|
||||
source: str
|
||||
right_variants: dict # {"1": {"file": ...}, "4": {...}} (no "0")
|
||||
annotations: list # [{"key", "box":[x,y,w,h], "min_level"}, ...]
|
||||
affect: list # [{"key", "at":[x,y], "min_level"}, ...] (Left x Right)
|
||||
strings: dict # {"en": {key: text}}
|
||||
|
||||
def variant_file(self, strength: int) -> str:
|
||||
"""The video file for a Right strength; 0 and any unauthored strength
|
||||
fall back to the raw base file."""
|
||||
entry = self.right_variants.get(str(strength))
|
||||
return entry["file"] if entry else self.base_file
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
variants = {"0": {"file": self.base_file, "raw": True}}
|
||||
for k, v in self.right_variants.items():
|
||||
variants[k] = v
|
||||
return {
|
||||
"id": self.id,
|
||||
"title": self.title,
|
||||
"base_file": self.base_file,
|
||||
"license": self.license,
|
||||
"source": self.source,
|
||||
"right_variants": variants,
|
||||
"annotations": self.annotations,
|
||||
"affect": self.affect,
|
||||
"strings": self.strings,
|
||||
}
|
||||
|
||||
|
||||
def _clip_from_dict(d: dict[str, Any]) -> Clip:
|
||||
return Clip(
|
||||
id=d["id"],
|
||||
title=d["title"],
|
||||
base_file=d["base_file"],
|
||||
license=d.get("license", ""),
|
||||
source=d.get("source", ""),
|
||||
right_variants=d.get("right_variants", {}),
|
||||
annotations=d.get("annotations", []),
|
||||
affect=d.get("affect", []),
|
||||
strings=d.get("strings", {}),
|
||||
)
|
||||
|
||||
|
||||
def load_manifest(path: str | Path) -> list[Clip]:
|
||||
"""Load the base-clip manifest. Raises FileNotFoundError if missing."""
|
||||
path = Path(path)
|
||||
data = json.loads(path.read_text())
|
||||
return [_clip_from_dict(c) for c in data["clips"]]
|
||||
|
||||
|
||||
def load_ring(path: str | Path) -> ScaleRing | None:
|
||||
"""Load the optional `ring` section as a `ScaleRing`, or None if absent.
|
||||
|
||||
The ring closes the scales-of-nature library into the navigable loop the
|
||||
endless encoder walks (scales design §3); the navigation math itself lives
|
||||
in player.ring.
|
||||
"""
|
||||
path = Path(path)
|
||||
data = json.loads(path.read_text())
|
||||
ring = data.get("ring")
|
||||
if not ring:
|
||||
return None
|
||||
scales = tuple(
|
||||
Scale(id=s["id"], clip_id=s["clip_id"]) for s in ring.get("scales", [])
|
||||
)
|
||||
transitions = tuple(
|
||||
Transition(file=t["file"], model=t.get("model", ""))
|
||||
for t in ring.get("transitions", [])
|
||||
)
|
||||
return ScaleRing(scales=scales, transitions=transitions)
|
||||
|
||||
|
||||
def ring_to_dict(ring: ScaleRing, clips: list[Clip]) -> dict:
|
||||
"""JSON form of the ring for the API: ordered scales (with their clip title)
|
||||
and the per-edge transitions."""
|
||||
titles = {c.id: c.title for c in clips}
|
||||
return {
|
||||
"scales": [
|
||||
{"id": s.id, "clip_id": s.clip_id, "title": titles.get(s.clip_id, s.id)}
|
||||
for s in ring.scales
|
||||
],
|
||||
"transitions": [{"file": t.file, "model": t.model} for t in ring.transitions],
|
||||
}
|
||||
|
||||
|
||||
def ring_move_to_dict(move: RingMove, ring: ScaleRing) -> dict:
|
||||
"""JSON form of an encoder move: the landing scale's clip and the ordered
|
||||
transition clips to play (with direction). `fast` flags a collapsed fast-spin
|
||||
pass; the single step then carries `blended` so the renderer plays it quick."""
|
||||
return {
|
||||
"from_index": move.from_index,
|
||||
"to_index": move.to_index,
|
||||
"wrapped": move.wrapped,
|
||||
"fast": move.fast,
|
||||
"target_clip_id": scale_at(ring, move.to_index).clip_id,
|
||||
"steps": [
|
||||
{
|
||||
"edge": st.edge,
|
||||
"reversed": st.reversed,
|
||||
"file": st.file,
|
||||
"to_index": st.to_index,
|
||||
"blended": st.blended,
|
||||
}
|
||||
for st in move.steps
|
||||
],
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Deterministic synthetic catalog so the selection model can be felt everywhere.
|
||||
|
||||
The real catalog (catalog/library.jsonl) is empty; this generates one record per
|
||||
cell of the 5x5 brain x 5x5 mood coordinate space (625 records), with a seeded
|
||||
mix of content modes and review statuses and no real media attached.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
|
||||
from hef.catalog import Record
|
||||
|
||||
MODES = ("audio", "video", "av")
|
||||
ARCHIVES = ("internet_archive", "musopen", "librivox", "nasa", "freesound")
|
||||
|
||||
_LEFT_WORDS = ("Treatise", "Lecture", "Field Notes", "Reading", "Documentary")
|
||||
_RIGHT_WORDS = ("Reverie", "Nocturne", "Bloom", "Drift", "Aurora")
|
||||
|
||||
|
||||
def _title(left: int, right: int, dark: int, light: int, mode: str) -> str:
|
||||
a = _LEFT_WORDS[left] if left >= right else _RIGHT_WORDS[right]
|
||||
return f"{a} ({mode}) L{left}R{right}D{dark}Li{light}"
|
||||
|
||||
|
||||
def generate_fixture_catalog(seed: int = 1729) -> list[Record]:
|
||||
"""One valid Record per coordinate cell (625 total), deterministic for a seed."""
|
||||
rng = random.Random(seed)
|
||||
records: list[Record] = []
|
||||
n = 0
|
||||
for left in range(5):
|
||||
for right in range(5):
|
||||
for dark in range(5):
|
||||
for light in range(5):
|
||||
mode = rng.choice(MODES)
|
||||
status = rng.choice(("proposed", "approved"))
|
||||
is_video = mode in ("video", "av")
|
||||
records.append(
|
||||
Record(
|
||||
id=f"fx-{n:04d}",
|
||||
title=_title(left, right, dark, light, mode),
|
||||
source_url=f"https://example.test/fx/{n:04d}",
|
||||
source_archive=rng.choice(ARCHIVES),
|
||||
license="public_domain",
|
||||
mode=mode,
|
||||
left=left,
|
||||
right=right,
|
||||
dark=dark,
|
||||
light=light,
|
||||
duration_s=rng.choice((300, 480, 600, 720, 900)),
|
||||
file_path="",
|
||||
review_status=status,
|
||||
resolution="1920x1080" if is_video else "",
|
||||
rationale=f"fixture at ({left},{right},{dark},{light})",
|
||||
reviewed_at="2026-06-04T00:00:00Z" if status == "approved" else None,
|
||||
)
|
||||
)
|
||||
n += 1
|
||||
return records
|
||||
@@ -1,21 +0,0 @@
|
||||
# Simulator sample media
|
||||
|
||||
`manifest.json` is committed; the `.mp4` binaries are **not** (gitignored). They
|
||||
are look-tuning samples, not shipped installation content.
|
||||
|
||||
Populate them in two steps:
|
||||
|
||||
python simulator/setup_sample_media.py # forest (the real POC base + Right variants)
|
||||
python simulator/setup_scales_media.py # cosmos + abyss scales + ring transitions
|
||||
|
||||
`setup_sample_media.py` copies `~/hef-poc/out/neutral.mp4` → `forest/base.mp4`
|
||||
and `~/hef-poc/out/right_flow.mp4` → `forest/right4.mp4` (the real flow-stabilized
|
||||
restyle), and generates placeholder strengths `forest/right1..3.mp4`.
|
||||
|
||||
`setup_scales_media.py` makes the scale **ring** demonstrable: cheap synthetic
|
||||
placeholder bases for the two true-PD scales — `cosmos/base.mp4` (NASA/Hubble)
|
||||
and `abyss/base.mp4` (NOAA Ocean Exploration) — plus the per-edge zoom/warp
|
||||
**transition** clips under `transitions/`. The manifest records the real true-PD
|
||||
provenance; these bytes are labelled placeholders for tuning ring navigation.
|
||||
The new scales carry a raw base only — the expensive multi-strength
|
||||
flow-stabilized Right re-bake is **deferred** until the ring is liked.
|
||||
@@ -1,516 +0,0 @@
|
||||
{
|
||||
"clips": [
|
||||
{
|
||||
"id": "cosmos",
|
||||
"title": "Orion Nebula flythrough (NASA/JPL-Caltech, neutral base)",
|
||||
"base_file": "cosmos/base.mp4",
|
||||
"license": "public-domain (US Gov, 17 U.S.C. §105)",
|
||||
"source": "NASA/JPL-Caltech — https://images.nasa.gov/details/JPL-20221122-SOLSYSf-0001-Orion Dust and Death (true PD; trimmed ~30–54s, crossfade-loop)",
|
||||
"right_variants": {},
|
||||
"annotations": [
|
||||
{
|
||||
"key": "detected.galaxy",
|
||||
"box": [
|
||||
0.34,
|
||||
0.3,
|
||||
0.3,
|
||||
0.34
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "measure.redshift",
|
||||
"box": [
|
||||
0.36,
|
||||
0.66,
|
||||
0.18,
|
||||
0.08
|
||||
],
|
||||
"min_level": 4
|
||||
}
|
||||
],
|
||||
"affect": [
|
||||
{
|
||||
"key": "feel.wonder",
|
||||
"at": [
|
||||
0.5,
|
||||
0.46
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "feel.vastness",
|
||||
"at": [
|
||||
0.2,
|
||||
0.7
|
||||
],
|
||||
"min_level": 2
|
||||
},
|
||||
{
|
||||
"key": "feel.insignificance",
|
||||
"at": [
|
||||
0.64,
|
||||
0.6
|
||||
],
|
||||
"min_level": 3
|
||||
},
|
||||
{
|
||||
"key": "feel.longing",
|
||||
"at": [
|
||||
0.42,
|
||||
0.84
|
||||
],
|
||||
"min_level": 4
|
||||
}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.galaxy": "spiral galaxy",
|
||||
"measure.redshift": "z ≈ 0.04",
|
||||
"feel.wonder": "wonder",
|
||||
"feel.vastness": "vastness",
|
||||
"feel.insignificance": "insignificance",
|
||||
"feel.longing": "longing"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "orbit",
|
||||
"title": "Earth from ISS — Expedition 65 (NASA/JSC, neutral base)",
|
||||
"base_file": "orbit/base.mp4",
|
||||
"license": "public-domain (US Gov, 17 U.S.C. §105)",
|
||||
"source": "NASA/JSC — https://images.nasa.gov/details/jsc2022m000172_Earth_in_4K_Expedition_65_Edition (true PD; trimmed ~10:00, crossfade-loop)",
|
||||
"right_variants": {},
|
||||
"annotations": [
|
||||
{
|
||||
"key": "detected.cloud_band",
|
||||
"box": [
|
||||
0.3,
|
||||
0.3,
|
||||
0.3,
|
||||
0.2
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "measure.altitude",
|
||||
"box": [
|
||||
0.06,
|
||||
0.06,
|
||||
0.18,
|
||||
0.08
|
||||
],
|
||||
"min_level": 2
|
||||
}
|
||||
],
|
||||
"affect": [
|
||||
{
|
||||
"key": "feel.serenity",
|
||||
"at": [
|
||||
0.5,
|
||||
0.46
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "feel.fragility",
|
||||
"at": [
|
||||
0.22,
|
||||
0.68
|
||||
],
|
||||
"min_level": 2
|
||||
},
|
||||
{
|
||||
"key": "feel.unity",
|
||||
"at": [
|
||||
0.66,
|
||||
0.58
|
||||
],
|
||||
"min_level": 3
|
||||
},
|
||||
{
|
||||
"key": "feel.distance",
|
||||
"at": [
|
||||
0.42,
|
||||
0.82
|
||||
],
|
||||
"min_level": 4
|
||||
}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.cloud_band": "cloud band",
|
||||
"measure.altitude": "~408 km",
|
||||
"feel.serenity": "serenity",
|
||||
"feel.fragility": "fragility",
|
||||
"feel.unity": "unity",
|
||||
"feel.distance": "distance"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "forest",
|
||||
"title": "Yosemite (NPS stock b-roll, neutral base)",
|
||||
"base_file": "forest/base.mp4",
|
||||
"license": "public-domain (NPS, no © — 17 U.S.C. §105)",
|
||||
"source": "NPS Yosemite stock footage — https://www.nps.gov/yose/learn/photosmultimedia/b-roll.htm (true PD; trimmed ~75–99s, crossfade-loop)",
|
||||
"right_variants": {
|
||||
"1": {
|
||||
"file": "forest/right1.mp4",
|
||||
"model": "sd-turbo+farneback-flow"
|
||||
},
|
||||
"2": {
|
||||
"file": "forest/right2.mp4",
|
||||
"model": "sd-turbo+farneback-flow"
|
||||
},
|
||||
"3": {
|
||||
"file": "forest/right3.mp4",
|
||||
"model": "sd-turbo+farneback-flow"
|
||||
},
|
||||
"4": {
|
||||
"file": "forest/right4.mp4",
|
||||
"model": "sd-turbo+farneback-flow"
|
||||
}
|
||||
},
|
||||
"annotations": [
|
||||
{
|
||||
"key": "detected.water",
|
||||
"min_level": 1,
|
||||
"track": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"box": [
|
||||
0.3,
|
||||
0.1,
|
||||
0.18,
|
||||
0.7
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 0.5,
|
||||
"box": [
|
||||
0.33,
|
||||
0.1,
|
||||
0.18,
|
||||
0.7
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 1.0,
|
||||
"box": [
|
||||
0.3,
|
||||
0.1,
|
||||
0.18,
|
||||
0.7
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"key": "detected.rock_face",
|
||||
"box": [
|
||||
0.05,
|
||||
0.3,
|
||||
0.2,
|
||||
0.55
|
||||
],
|
||||
"min_level": 2
|
||||
},
|
||||
{
|
||||
"key": "detected.conifer",
|
||||
"box": [
|
||||
0.7,
|
||||
0.2,
|
||||
0.22,
|
||||
0.45
|
||||
],
|
||||
"min_level": 3
|
||||
},
|
||||
{
|
||||
"key": "measure.flow_rate",
|
||||
"box": [
|
||||
0.34,
|
||||
0.55,
|
||||
0.14,
|
||||
0.08
|
||||
],
|
||||
"min_level": 4
|
||||
}
|
||||
],
|
||||
"affect": [
|
||||
{
|
||||
"key": "feel.awe",
|
||||
"at": [
|
||||
0.48,
|
||||
0.44
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "feel.serenity",
|
||||
"at": [
|
||||
0.18,
|
||||
0.66
|
||||
],
|
||||
"min_level": 2
|
||||
},
|
||||
{
|
||||
"key": "feel.reverence",
|
||||
"at": [
|
||||
0.66,
|
||||
0.56
|
||||
],
|
||||
"min_level": 3
|
||||
},
|
||||
{
|
||||
"key": "feel.stillness",
|
||||
"at": [
|
||||
0.42,
|
||||
0.82
|
||||
],
|
||||
"min_level": 4
|
||||
}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.water": "flowing water",
|
||||
"detected.rock_face": "granite face",
|
||||
"detected.conifer": "conifer stand",
|
||||
"measure.flow_rate": "~2.1 m³/s",
|
||||
"feel.awe": "awe",
|
||||
"feel.serenity": "serenity",
|
||||
"feel.reverence": "reverence",
|
||||
"feel.stillness": "stillness"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "reef",
|
||||
"title": "Coral reef b-roll (NOAA Fisheries, neutral base)",
|
||||
"base_file": "reef/base.mp4",
|
||||
"license": "public-domain (US Gov, 17 U.S.C. §105)",
|
||||
"source": "NOAA Fisheries — B-Roll: Corals of the Atlantic/Gulf/Keys/Caribbean — https://videos.fisheries.noaa.gov/detail/videos/b-roll:-corals/video/5231418011001 (true PD; trimmed 127–155s, crossfade-loop)",
|
||||
"right_variants": {},
|
||||
"annotations": [
|
||||
{
|
||||
"key": "detected.coral",
|
||||
"box": [
|
||||
0.2,
|
||||
0.45,
|
||||
0.3,
|
||||
0.3
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "detected.fish",
|
||||
"min_level": 2,
|
||||
"track": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"box": [
|
||||
0.2,
|
||||
0.3,
|
||||
0.1,
|
||||
0.08
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 0.5,
|
||||
"box": [
|
||||
0.62,
|
||||
0.34,
|
||||
0.1,
|
||||
0.08
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 1.0,
|
||||
"box": [
|
||||
0.2,
|
||||
0.3,
|
||||
0.1,
|
||||
0.08
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"key": "measure.depth",
|
||||
"box": [
|
||||
0.06,
|
||||
0.06,
|
||||
0.16,
|
||||
0.08
|
||||
],
|
||||
"min_level": 3
|
||||
}
|
||||
],
|
||||
"affect": [
|
||||
{
|
||||
"key": "feel.delight",
|
||||
"at": [
|
||||
0.5,
|
||||
0.46
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "feel.abundance",
|
||||
"at": [
|
||||
0.22,
|
||||
0.68
|
||||
],
|
||||
"min_level": 2
|
||||
},
|
||||
{
|
||||
"key": "feel.curiosity",
|
||||
"at": [
|
||||
0.66,
|
||||
0.58
|
||||
],
|
||||
"min_level": 3
|
||||
},
|
||||
{
|
||||
"key": "feel.immersion",
|
||||
"at": [
|
||||
0.42,
|
||||
0.82
|
||||
],
|
||||
"min_level": 4
|
||||
}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.coral": "coral colony",
|
||||
"detected.fish": "reef fish",
|
||||
"measure.depth": "−18 m",
|
||||
"feel.delight": "delight",
|
||||
"feel.abundance": "abundance",
|
||||
"feel.curiosity": "curiosity",
|
||||
"feel.immersion": "immersion"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "abyss",
|
||||
"title": "Comb jelly / ctenophore (NOAA Ocean Exploration, neutral base)",
|
||||
"base_file": "abyss/base.mp4",
|
||||
"license": "public-domain (US Gov, 17 U.S.C. §105)",
|
||||
"source": "NOAA Ocean Exploration — https://oceanexplorer.noaa.gov/multimedia/video-playlist-ex2103-ctenophore/ (true PD; trimmed 3–27s, crossfade-loop)",
|
||||
"right_variants": {},
|
||||
"annotations": [
|
||||
{
|
||||
"key": "detected.organism",
|
||||
"box": [
|
||||
0.4,
|
||||
0.38,
|
||||
0.22,
|
||||
0.26
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "measure.depth",
|
||||
"box": [
|
||||
0.06,
|
||||
0.06,
|
||||
0.16,
|
||||
0.08
|
||||
],
|
||||
"min_level": 2
|
||||
}
|
||||
],
|
||||
"affect": [
|
||||
{
|
||||
"key": "feel.unease",
|
||||
"at": [
|
||||
0.5,
|
||||
0.46
|
||||
],
|
||||
"min_level": 1
|
||||
},
|
||||
{
|
||||
"key": "feel.fascination",
|
||||
"at": [
|
||||
0.22,
|
||||
0.68
|
||||
],
|
||||
"min_level": 2
|
||||
},
|
||||
{
|
||||
"key": "feel.isolation",
|
||||
"at": [
|
||||
0.66,
|
||||
0.58
|
||||
],
|
||||
"min_level": 3
|
||||
},
|
||||
{
|
||||
"key": "feel.dread",
|
||||
"at": [
|
||||
0.42,
|
||||
0.82
|
||||
],
|
||||
"min_level": 4
|
||||
}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.organism": "siphonophore",
|
||||
"measure.depth": "−2,140 m",
|
||||
"feel.unease": "unease",
|
||||
"feel.fascination": "fascination",
|
||||
"feel.isolation": "isolation",
|
||||
"feel.dread": "dread"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"ring": {
|
||||
"scales": [
|
||||
{
|
||||
"id": "cosmos",
|
||||
"clip_id": "cosmos"
|
||||
},
|
||||
{
|
||||
"id": "orbit",
|
||||
"clip_id": "orbit"
|
||||
},
|
||||
{
|
||||
"id": "forest",
|
||||
"clip_id": "forest"
|
||||
},
|
||||
{
|
||||
"id": "reef",
|
||||
"clip_id": "reef"
|
||||
},
|
||||
{
|
||||
"id": "abyss",
|
||||
"clip_id": "abyss"
|
||||
}
|
||||
],
|
||||
"transitions": [
|
||||
{
|
||||
"file": "transitions/cosmos-orbit.mp4",
|
||||
"model": "placeholder-zoom"
|
||||
},
|
||||
{
|
||||
"file": "transitions/orbit-forest.mp4",
|
||||
"model": "placeholder-zoom"
|
||||
},
|
||||
{
|
||||
"file": "transitions/forest-reef.mp4",
|
||||
"model": "placeholder-zoom"
|
||||
},
|
||||
{
|
||||
"file": "transitions/reef-abyss.mp4",
|
||||
"model": "placeholder-zoom"
|
||||
},
|
||||
{
|
||||
"file": "transitions/abyss-cosmos.mp4",
|
||||
"model": "placeholder-zoom"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
"""Stage the forest scale's neutral BASE clip from the session-0008 POC artifacts.
|
||||
|
||||
Copies the real neutral Yosemite base out of ~/hef-poc/out/ into
|
||||
simulator/sample_media/forest/. The real multi-strength Right variants are then
|
||||
produced by the offline baker (`simulator/bake_right_variants.py`) — this script
|
||||
no longer generates placeholder Right strengths, so re-running it never clobbers
|
||||
a real bake. The media binaries are gitignored; only the manifest is committed.
|
||||
Sample footage is for look-tuning only, not shipped content.
|
||||
|
||||
Usage:
|
||||
python simulator/setup_sample_media.py # stage the base
|
||||
python -m simulator.bake_right_variants --scale forest # then bake variants
|
||||
Requires: ~/hef-poc/out/neutral.mp4 (the POC neutral base).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
POC = Path.home() / "hef-poc" / "out"
|
||||
DEST = Path(__file__).parent / "sample_media" / "forest"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
DEST.mkdir(parents=True, exist_ok=True)
|
||||
base = DEST / "base.mp4"
|
||||
shutil.copyfile(POC / "neutral.mp4", base)
|
||||
print(f"staged {base} (real POC neutral base)")
|
||||
print("next: python -m simulator.bake_right_variants --scale forest")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,113 +0,0 @@
|
||||
"""Generate cheap placeholder media for the full 5-scale RING (scales design §3).
|
||||
|
||||
Produces offline synthetic placeholders for all five ring scales — **cosmos**
|
||||
(NASA/Hubble), **orbit** (NASA/ISS), **forest** (NPS/USGS), **reef** (NOAA Ocean
|
||||
Exploration), and **abyss** (NOAA Ocean Exploration) — plus the short zoom/warp
|
||||
**transition** clips between adjacent scales. Forest reuses the real POC base from
|
||||
`setup_sample_media.py` (or a synthetic placeholder if absent); all others are
|
||||
generated as labelled solid-tint clips.
|
||||
|
||||
This is deliberately the CHEAP path: the manifest records the real true-PD
|
||||
provenance (NASA / NPS / USGS / NOAA, 17 U.S.C. §105), but the bytes here are
|
||||
labelled synthetic placeholders for look-tuning the ring navigation. The Right dream
|
||||
is real-time (Kuwahara WebGL shader), so no per-clip ML bake is needed; each scale
|
||||
carries a raw base only. Real strict-PD bases replace these placeholders via the
|
||||
content pipeline (see `docs/content-sourcing.md`).
|
||||
|
||||
Usage: python simulator/setup_scales_media.py
|
||||
Requires: ffmpeg on PATH (or `pip install imageio-ffmpeg`).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
MEDIA = Path(__file__).parent / "sample_media"
|
||||
DUR = 6 # base clip seconds
|
||||
XDUR = 1.5 # transition seconds
|
||||
SIZE = "1280x720"
|
||||
FPS = 25
|
||||
|
||||
# Each scale: (dir, base color, on-screen placeholder label).
|
||||
SCALES = {
|
||||
"cosmos": ("0x05060f", "COSMOS — NASA/Hubble (PD placeholder)"),
|
||||
"orbit": ("0x081427", "ORBIT — NASA/ISS (PD placeholder)"),
|
||||
"forest": ("0x12301a", "FOREST — NPS/USGS (POC/placeholder)"),
|
||||
"reef": ("0x06303a", "REEF — NOAA Ocean Exploration (PD placeholder)"),
|
||||
"abyss": ("0x021016", "ABYSS — NOAA Ocean Exploration (PD placeholder)"),
|
||||
}
|
||||
|
||||
# Ring edges (adjacent pairs + the abyss->cosmos wrap), matching the manifest.
|
||||
EDGES = [("cosmos", "orbit"), ("orbit", "forest"), ("forest", "reef"),
|
||||
("reef", "abyss"), ("abyss", "cosmos")]
|
||||
|
||||
|
||||
def _ffmpeg() -> str:
|
||||
if shutil.which("ffmpeg"):
|
||||
return "ffmpeg"
|
||||
import imageio_ffmpeg
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
|
||||
|
||||
def _run(args: list[str]) -> None:
|
||||
subprocess.run(args, check=True, capture_output=True)
|
||||
|
||||
|
||||
def _make_base(ff: str, name: str, color: str, label: str) -> Path:
|
||||
out = MEDIA / name / "base.mp4"
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
# A calm solid tint + faint noise grain (reads as stars/particles), with a
|
||||
# placeholder label so it is never mistaken for shipped footage.
|
||||
vf = (
|
||||
"noise=alls=8:allf=t,"
|
||||
"drawtext=text='" + label + "':fontcolor=white@0.55:fontsize=24:"
|
||||
"x=(w-text_w)/2:y=h-48"
|
||||
)
|
||||
_run([
|
||||
ff, "-y", "-f", "lavfi", "-i",
|
||||
f"color=c={color}:s={SIZE}:d={DUR}:r={FPS}",
|
||||
"-vf", vf, "-pix_fmt", "yuv420p", "-an", str(out),
|
||||
])
|
||||
return out
|
||||
|
||||
|
||||
def _make_transition(ff: str, src: str, dst: str) -> Path:
|
||||
a = MEDIA / src / "base.mp4"
|
||||
b = MEDIA / dst / "base.mp4"
|
||||
out = MEDIA / "transitions" / f"{src}-{dst}.mp4"
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
w, h = SIZE.split("x")
|
||||
# A placeholder zoom/warp morph: zoom-in crossfade from src into dst. Both
|
||||
# bases are normalized first (size/fps/sar/format) because the real forest
|
||||
# POC base differs in resolution + fps from the synthetic scale placeholders,
|
||||
# and xfade requires identical frame geometry on both inputs.
|
||||
norm = f"trim=0:3,setpts=PTS-STARTPTS,scale={w}:{h},fps={FPS},setsar=1,format=yuv420p"
|
||||
_run([
|
||||
ff, "-y", "-i", str(a), "-i", str(b), "-filter_complex",
|
||||
f"[0:v]{norm}[a];[1:v]{norm}[b];"
|
||||
f"[a][b]xfade=transition=zoomin:duration={XDUR}:offset=0.75,"
|
||||
"format=yuv420p[v]",
|
||||
"-map", "[v]", "-an", str(out),
|
||||
])
|
||||
return out
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ff = _ffmpeg()
|
||||
for name, (color, label) in SCALES.items():
|
||||
base = MEDIA / name / "base.mp4"
|
||||
if name == "forest" and base.exists():
|
||||
print(f"forest/base.mp4 present (real POC base) — keeping")
|
||||
continue
|
||||
_make_base(ff, name, color, label)
|
||||
print(f"generated {name}/base.mp4 (synthetic placeholder)")
|
||||
for src, dst in EDGES:
|
||||
_make_transition(ff, src, dst)
|
||||
print(f"generated transitions/{src}-{dst}.mp4 (placeholder zoom)")
|
||||
print(f"scale-ring media ready in {MEDIA}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+91
-442
@@ -1,462 +1,111 @@
|
||||
// Thin renderer: post controls+calibration -> RenderPlan; render the mood grade,
|
||||
// the live Right DREAM (a deterministic luminous haze, NOT a baked variant — it
|
||||
// holds still across the loop), and the live Left overlay + affect. All alteration
|
||||
// math stays in Python. The scale RING (endless encoder) is navigated via
|
||||
// /api/ring/advance — Python owns the step/wrap/transition-selection math; the
|
||||
// browser only plays the returned transition clip(s) then settles on the target.
|
||||
const $ = (id) => document.getElementById(id);
|
||||
const vid = $("vid"), paint = $("paint"), tint = $("tint"), overlay = $("overlay"),
|
||||
black = $("black"), readout = $("readout");
|
||||
const affectLayer = $("affect");
|
||||
const DIALS = ["left", "right", "dark", "light"];
|
||||
const MODEL = ["brain_weight", "mood_weight", "pool_size"];
|
||||
|
||||
// Right-dream state, read by the painterly render loop each frame. update() (debounced
|
||||
// on knob moves) writes these; the WebGL loop renders the live video continuously.
|
||||
let dreamIntensity = 0; // 0..1 — painterly strength (and dream pastel/luminous)
|
||||
let gradeFilter = "none"; // CSS filter string for the Dark/Light mood grade
|
||||
|
||||
// Surface any uncaught JS error on-screen (a silent throw in update() otherwise
|
||||
// looks like "nothing is changing" — the render dies but the page sits there).
|
||||
function showError(msg) {
|
||||
let b = document.getElementById("err-banner");
|
||||
if (!b) {
|
||||
b = document.createElement("div");
|
||||
b.id = "err-banner";
|
||||
b.style.cssText = "position:fixed;top:0;left:0;right:0;z-index:9999;background:#a00;" +
|
||||
"color:#fff;font:12px/1.4 monospace;padding:6px 10px;white-space:pre-wrap;";
|
||||
document.body.appendChild(b);
|
||||
function buildGrid(el) {
|
||||
el.innerHTML = "";
|
||||
// rows = first axis 0..4 top->bottom, cols = second axis 0..4 left->right
|
||||
for (let a = 0; a < 5; a++) {
|
||||
for (let b = 0; b < 5; b++) {
|
||||
const cell = document.createElement("div");
|
||||
cell.className = "cell";
|
||||
cell.dataset.a = a;
|
||||
cell.dataset.b = b;
|
||||
el.appendChild(cell);
|
||||
}
|
||||
}
|
||||
b.textContent = "⚠ " + msg;
|
||||
}
|
||||
window.addEventListener("error", (e) =>
|
||||
showError(`${e.message} @ ${(e.filename || "").split("/").pop()}:${e.lineno}`));
|
||||
|
||||
let clipsById = {}; // id -> clip manifest entry
|
||||
let ring = null; // {scales:[{id,clip_id,title}], transitions:[...]} or null
|
||||
let ringIndex = 0; // current scale position on the ring
|
||||
let currentClipId = null; // base media currently loaded (reset to force a reload)
|
||||
let busy = false; // true while a ring transition is playing
|
||||
let lastOverlay = null; // {level, intensity} from the most-recent renderOverlay call; drives per-frame track animation
|
||||
|
||||
async function loadData() {
|
||||
const clips = (await (await fetch("/api/clips")).json()).clips || [];
|
||||
clipsById = Object.fromEntries(clips.map((c) => [c.id, c]));
|
||||
const r = await fetch("/api/ring");
|
||||
ring = r.ok ? await r.json() : null;
|
||||
if (!ring && clips.length) {
|
||||
// No ring: single-clip mode — synthesize a 1-scale ring over clip 0.
|
||||
ring = { scales: [{ id: clips[0].id, clip_id: clips[0].id, title: clips[0].title }], transitions: [] };
|
||||
}
|
||||
ringIndex = 0;
|
||||
}
|
||||
|
||||
function activeClip() {
|
||||
if (!ring) return null;
|
||||
return clipsById[ring.scales[ringIndex].clip_id] || null;
|
||||
function paintGrid(el, axisA, axisB, point, pool) {
|
||||
// clear
|
||||
el.querySelectorAll(".cell").forEach((c) => {
|
||||
c.className = "cell";
|
||||
c.innerHTML = "";
|
||||
});
|
||||
const counts = {};
|
||||
pool.forEach((c) => {
|
||||
const r = c.record;
|
||||
const key = `${r[axisA]},${r[axisB]}`;
|
||||
counts[key] = (counts[key] || 0) + 1;
|
||||
});
|
||||
el.querySelectorAll(".cell").forEach((c) => {
|
||||
const a = +c.dataset.a, b = +c.dataset.b;
|
||||
const key = `${a},${b}`;
|
||||
if (counts[key]) {
|
||||
c.classList.add("cand");
|
||||
const n = document.createElement("span");
|
||||
n.className = "n";
|
||||
n.textContent = counts[key];
|
||||
c.appendChild(n);
|
||||
}
|
||||
if (a === point[axisA] && b === point[axisB]) c.classList.add("point");
|
||||
});
|
||||
}
|
||||
|
||||
function mediaUrl(file) { return "/media/" + file; }
|
||||
|
||||
// Load the active scale's BASE footage into the video (the painterly canvas reads
|
||||
// its frames live). The Right knob no longer swaps the source — it only drives the
|
||||
// deterministic dream restyle — so media reloads only when the SCALE changes.
|
||||
function ensureClipMedia() {
|
||||
const clip = activeClip();
|
||||
if (!clip || clip.id === currentClipId) return;
|
||||
currentClipId = clip.id;
|
||||
vid.src = mediaUrl(clip.base_file);
|
||||
vid.loop = true;
|
||||
vid.muted = true;
|
||||
vid.play().catch(() => {});
|
||||
vid.style.opacity = "1";
|
||||
}
|
||||
|
||||
// Compose the video look. The Right DREAM is now a painterly WebGL restyle of the
|
||||
// live frames (see the render loop below) — so here we only stash its intensity for
|
||||
// that loop, and build the Dark/Light mood grade as a CSS filter. The grade rides
|
||||
// on the painterly canvas (or #vid in the WebGL-less fallback). No blur: the dream
|
||||
// must keep the base's crisp motion, just stylized.
|
||||
function applyVideoLook(tone, dream) {
|
||||
const warm = tone > 0 ? tone : 0, cool = tone < 0 ? -tone : 0;
|
||||
const bright = 1 + 0.25 * warm - 0.35 * cool;
|
||||
const sat = 1 + 0.15 * warm - 0.30 * cool;
|
||||
const sepia = warm * 0.5;
|
||||
gradeFilter = `brightness(${bright.toFixed(3)}) saturate(${sat.toFixed(3)}) sepia(${sepia.toFixed(3)})`;
|
||||
dreamIntensity = dream;
|
||||
tint.style.opacity = (cool * 0.6).toFixed(3);
|
||||
// WebGL-less fallback: the canvas is hidden, so grade the visible #vid directly.
|
||||
if (!paintOK) vid.style.filter = gradeFilter;
|
||||
}
|
||||
|
||||
// --- Right dream: real-time painterly restyle (WebGL2 Kuwahara) ---
|
||||
// An edge-preserving "oil painting" filter on the LIVE video frames: each pixel
|
||||
// becomes the mean of whichever surrounding quadrant is most uniform, flattening
|
||||
// the image into painted regions WHILE KEEPING EDGES — so motion stays as crisp as
|
||||
// the source. Strength + a gentle pastel/luminous dream-grade scale with intensity.
|
||||
// Deterministic (a pure function of each frame) => the dream holds still across the
|
||||
// loop; only the footage's own motion moves.
|
||||
let paintOK = false;
|
||||
const PAINT_VS = `#version 300 es
|
||||
in vec2 p; out vec2 v_uv;
|
||||
void main(){
|
||||
// Fullscreen triangle (verts to +3); map the visible [-1,1] to uv [0,1] with Y
|
||||
// flipped for the video's top-left origin. (Visible region stays in range; the
|
||||
// off-screen excess is clipped.)
|
||||
v_uv = vec2(p.x * 0.5 + 0.5, 0.5 - p.y * 0.5);
|
||||
gl_Position = vec4(p, 0.0, 1.0);
|
||||
}`;
|
||||
const PAINT_FS = `#version 300 es
|
||||
precision highp float;
|
||||
in vec2 v_uv; out vec4 frag;
|
||||
uniform sampler2D u_tex; uniform vec2 u_texel; uniform float u_amt;
|
||||
#define R 6
|
||||
#define QUAD(x0,x1,y0,y1,MO,VO) { vec3 m=vec3(0.0),s=vec3(0.0); \
|
||||
for(int j=y0;j<=y1;j++){ for(int i=x0;i<=x1;i++){ \
|
||||
vec3 c=texture(u_tex, v_uv+vec2(float(i),float(j))*u_texel).rgb; m+=c; s+=c*c; } } \
|
||||
float n=float((x1-x0+1)*(y1-y0+1)); m/=n; vec3 vv=s/n-m*m; MO=m; VO=vv.r+vv.g+vv.b; }
|
||||
vec3 hueRotate(vec3 c, float a){
|
||||
float co=cos(a), si=sin(a);
|
||||
return vec3(
|
||||
c.r*(0.299+0.701*co+0.168*si) + c.g*(0.587-0.587*co+0.330*si) + c.b*(0.114-0.114*co-0.497*si),
|
||||
c.r*(0.299-0.299*co-0.328*si) + c.g*(0.587+0.413*co+0.035*si) + c.b*(0.114-0.114*co+0.292*si),
|
||||
c.r*(0.299-0.300*co+1.250*si) + c.g*(0.587-0.588*co-1.050*si) + c.b*(0.114+0.886*co-0.203*si));
|
||||
}
|
||||
void main(){
|
||||
vec3 base = texture(u_tex, v_uv).rgb;
|
||||
if (u_amt <= 0.001) { frag = vec4(base, 1.0); return; }
|
||||
vec3 m0,m1,m2,m3; float v0,v1,v2,v3;
|
||||
QUAD(-R,0,-R,0,m0,v0) QUAD(0,R,-R,0,m1,v1) QUAD(-R,0,0,R,m2,v2) QUAD(0,R,0,R,m3,v3)
|
||||
vec3 painted=m0; float mv=v0;
|
||||
if(v1<mv){mv=v1;painted=m1;} if(v2<mv){mv=v2;painted=m2;} if(v3<mv){mv=v3;painted=m3;}
|
||||
vec3 styl = mix(base, painted, u_amt); // painterly restyle, linear in the knob
|
||||
// The "trippy" terms ramp with t = amt^2, so low Right stays gently dreamy and
|
||||
// only MAX goes psychedelic: vivid saturation, a surreal hue drift, poster banding.
|
||||
float t = u_amt * u_amt;
|
||||
float luma = dot(styl, vec3(0.299, 0.587, 0.114));
|
||||
styl = mix(vec3(luma), styl, 1.0 + 1.1 * t); // saturation: vivid toward max
|
||||
styl = hueRotate(styl, 0.8 * t); // surreal hue drift (~45deg at max)
|
||||
float levels = mix(255.0, 6.0, t); // posterize: color banding toward max
|
||||
styl = floor(styl * levels + 0.5) / levels;
|
||||
styl *= 1.0 + 0.06 * u_amt; // luminous lift
|
||||
frag = vec4(clamp(styl, 0.0, 1.0), 1.0);
|
||||
}`;
|
||||
let gl = null, uAmt = null, uTexel = null;
|
||||
function _shader(kind, src) {
|
||||
const s = gl.createShader(kind);
|
||||
gl.shaderSource(s, src); gl.compileShader(s);
|
||||
if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) throw new Error(gl.getShaderInfoLog(s));
|
||||
function readState() {
|
||||
const s = { mode: document.getElementById("mode").value, approved_only: document.getElementById("approved_only").checked };
|
||||
DIALS.forEach((d) => (s[d] = +document.getElementById(d).value));
|
||||
s.brain_weight = +document.getElementById("brain_weight").value;
|
||||
s.mood_weight = +document.getElementById("mood_weight").value;
|
||||
s.pool_size = +document.getElementById("pool_size").value;
|
||||
return s;
|
||||
}
|
||||
function initPaint() {
|
||||
gl = paint.getContext("webgl2");
|
||||
if (!gl) { paint.style.display = "none"; return false; } // CSS fallback grades #vid
|
||||
const prog = gl.createProgram();
|
||||
gl.attachShader(prog, _shader(gl.VERTEX_SHADER, PAINT_VS));
|
||||
gl.attachShader(prog, _shader(gl.FRAGMENT_SHADER, PAINT_FS));
|
||||
gl.bindAttribLocation(prog, 0, "p"); gl.linkProgram(prog);
|
||||
if (!gl.getProgramParameter(prog, gl.LINK_STATUS)) throw new Error(gl.getProgramInfoLog(prog));
|
||||
gl.useProgram(prog);
|
||||
const buf = gl.createBuffer();
|
||||
gl.bindBuffer(gl.ARRAY_BUFFER, buf);
|
||||
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1, -1, 3, -1, -1, 3]), gl.STATIC_DRAW);
|
||||
gl.enableVertexAttribArray(0); gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);
|
||||
const t = gl.createTexture();
|
||||
gl.bindTexture(gl.TEXTURE_2D, t);
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
|
||||
uAmt = gl.getUniformLocation(prog, "u_amt");
|
||||
uTexel = gl.getUniformLocation(prog, "u_texel");
|
||||
paintOK = true;
|
||||
requestAnimationFrame(paintLoop);
|
||||
return true;
|
||||
}
|
||||
function paintLoop() {
|
||||
if (paintOK && vid.readyState >= 2 && vid.videoWidth) {
|
||||
const w = vid.videoWidth, h = vid.videoHeight;
|
||||
if (paint.width !== w || paint.height !== h) { paint.width = w; paint.height = h; }
|
||||
gl.viewport(0, 0, w, h);
|
||||
try { gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, vid); } catch (_) {}
|
||||
// No painterly restyle during a ring transition (busy) — show it raw.
|
||||
gl.uniform1f(uAmt, busy ? 0 : dreamIntensity);
|
||||
gl.uniform2f(uTexel, 1 / w, 1 / h);
|
||||
gl.drawArrays(gl.TRIANGLES, 0, 3);
|
||||
paint.style.filter = busy ? "none" : gradeFilter;
|
||||
}
|
||||
// Animate keyframed annotation tracks: re-place boxes against playback time.
|
||||
const c = activeClip();
|
||||
if (lastOverlay && lastOverlay.level > 0 && c &&
|
||||
c.annotations.some((a) => a.track && a.track.length)) {
|
||||
renderOverlay(lastOverlay.level, lastOverlay.intensity);
|
||||
}
|
||||
requestAnimationFrame(paintLoop);
|
||||
}
|
||||
|
||||
const SVGNS = "http://www.w3.org/2000/svg";
|
||||
function svg(tag, attrs, parent) {
|
||||
const el = document.createElementNS(SVGNS, tag);
|
||||
for (const k in attrs) el.setAttribute(k, attrs[k]);
|
||||
if (parent) parent.appendChild(el);
|
||||
return el;
|
||||
}
|
||||
|
||||
// Deterministic per-key "confidence" in [0.78, 0.99] — stable across renders so
|
||||
// a detection always reports the same score (looks like a real model, not noise).
|
||||
function confidence(key) {
|
||||
let h = 0;
|
||||
for (let i = 0; i < key.length; i++) h = (h * 31 + key.charCodeAt(i)) >>> 0;
|
||||
return (0.78 + (h % 2200) / 10000).toFixed(2);
|
||||
}
|
||||
|
||||
// Four L-shaped corner brackets framing [x,y,w,h] — a targeting reticle rather
|
||||
// than a plain rectangle. Arm length scales to the box but is clamped legible.
|
||||
function reticle(x, y, w, h, cls, parent) {
|
||||
const L = Math.min(Math.max(Math.min(w, h) * 0.28, 1.4), 4);
|
||||
const corners = [
|
||||
[[x + L, y], [x, y], [x, y + L]],
|
||||
[[x + w - L, y], [x + w, y], [x + w, y + L]],
|
||||
[[x + L, y + h], [x, y + h], [x, y + h - L]],
|
||||
[[x + w - L, y + h], [x + w, y + h], [x + w, y + h - L]],
|
||||
];
|
||||
for (const pts of corners) {
|
||||
svg("polyline", { points: pts.map((p) => p.join(",")).join(" "), class: cls }, parent);
|
||||
}
|
||||
}
|
||||
|
||||
// A label chip: translucent plate + monospace text, anchored top-left of the box.
|
||||
// `conf` (detections only) is appended as a dimmer score tspan.
|
||||
function chip(x, y, label, conf, kind, parent) {
|
||||
const fs = 2.4, pad = 0.6;
|
||||
const chars = label.length + (conf ? conf.length + 1 : 0);
|
||||
const w = chars * fs * 0.6 + pad * 2, h = fs + pad * 1.4;
|
||||
const cy = Math.max(y - h - 0.6, 0.4);
|
||||
svg("rect", { x, y: cy, width: w, height: h, rx: 0.4, class: "hud-chip-bg " + kind }, parent);
|
||||
const t = svg("text", { x: x + pad, y: cy + h - pad * 1.1, class: "hud-chip-text " + kind }, parent);
|
||||
t.appendChild(document.createTextNode(label));
|
||||
if (conf) {
|
||||
const sp = svg("tspan", { class: "hud-conf" }, t);
|
||||
sp.textContent = " " + conf;
|
||||
}
|
||||
}
|
||||
|
||||
// Interpolate an annotation's box at loop-normalized time t (0..1). Static labels
|
||||
// (no track) return their fixed box. Linear between sorted keyframes; clamps to ends.
|
||||
function boxAt(a, t) {
|
||||
if (!a.track || !a.track.length) return a.box;
|
||||
const ks = a.track;
|
||||
if (t <= ks[0].t) return ks[0].box;
|
||||
if (t >= ks[ks.length - 1].t) return ks[ks.length - 1].box;
|
||||
for (let i = 1; i < ks.length; i++) {
|
||||
if (t <= ks[i].t) {
|
||||
const a0 = ks[i - 1], a1 = ks[i];
|
||||
const f = (t - a0.t) / (a1.t - a0.t);
|
||||
return a0.box.map((v, j) => v + (a1.box[j] - v) * f);
|
||||
}
|
||||
}
|
||||
return ks[ks.length - 1].box;
|
||||
}
|
||||
|
||||
// Loop-normalized playback time of the base video (0..1), for track interpolation.
|
||||
function loopT() {
|
||||
return vid && vid.duration ? (vid.currentTime % vid.duration) / vid.duration : 0;
|
||||
}
|
||||
|
||||
function renderOverlay(level, intensity) {
|
||||
const clip = activeClip();
|
||||
overlay.innerHTML = "";
|
||||
if (!clip || level <= 0) { overlay.style.opacity = "0"; return; }
|
||||
overlay.style.opacity = String(intensity);
|
||||
const strings = (clip.strings && clip.strings.en) || {};
|
||||
let shown = 0;
|
||||
for (const a of clip.annotations) {
|
||||
if (a.min_level > level) continue;
|
||||
shown++;
|
||||
const [x, y, w, h] = boxAt(a, loopT()).map((n) => n * 100);
|
||||
const measure = a.key.startsWith("measure.");
|
||||
const kind = measure ? "measure" : "detect";
|
||||
reticle(x, y, w, h, "hud-reticle " + kind, overlay);
|
||||
if (measure) {
|
||||
// Point measurement: a center crosshair tick to read as "sampled here".
|
||||
const cx = x + w / 2, cyc = y + h / 2;
|
||||
svg("line", { x1: cx - 1, y1: cyc, x2: cx + 1, y2: cyc, class: "hud-reticle measure" }, overlay);
|
||||
svg("line", { x1: cx, y1: cyc - 1, x2: cx, y2: cyc + 1, class: "hud-reticle measure" }, overlay);
|
||||
}
|
||||
chip(x, y, strings[a.key] || a.key, measure ? null : confidence(a.key), kind, overlay);
|
||||
}
|
||||
// Global status tag — the machine announcing it is analyzing, escalating with Left.
|
||||
// Right-anchored at the frame edge; the plate is sized from the text's real bbox
|
||||
// (the ◉/· glyphs are wider than monospace, so an estimate clips).
|
||||
if (shown) {
|
||||
const pad = 0.8;
|
||||
const st = svg("text", { x: 98, y: 5, "text-anchor": "end", class: "hud-status" }, overlay);
|
||||
st.textContent = "◉ ANALYSIS · L" + level + " · " + shown + " OBJ";
|
||||
const b = st.getBBox();
|
||||
const bg = svg("rect", { x: b.x - pad, y: b.y - pad * 0.6, width: b.width + pad * 2,
|
||||
height: b.height + pad * 1.2, rx: 0.4, class: "hud-chip-bg detect" });
|
||||
overlay.insertBefore(bg, st);
|
||||
}
|
||||
lastOverlay = { level, intensity };
|
||||
}
|
||||
|
||||
// Affect channel: soft, glowing emotion-words surfaced only when BOTH knobs are
|
||||
// up. `strength` = min(left, right); `intensity` is the layer opacity. Words are
|
||||
// placed at authored scene points (no boxes — feelings are scene-level) and read
|
||||
// softer than the clinical reticles — the dream leaking into the machine's read.
|
||||
function renderAffect(strength, intensity) {
|
||||
const clip = activeClip();
|
||||
if (!affectLayer) return; // tolerate a stale page missing the affect layer
|
||||
affectLayer.innerHTML = "";
|
||||
if (!clip || !clip.affect || strength <= 0) { affectLayer.style.opacity = "0"; return; }
|
||||
affectLayer.style.opacity = String(intensity);
|
||||
const strings = (clip.strings && clip.strings.en) || {};
|
||||
for (const f of clip.affect) {
|
||||
if (f.min_level > strength) continue;
|
||||
const [x, y] = f.at.map((n) => n * 100);
|
||||
const t = svg("text", { x, y, "text-anchor": "middle", class: "hud-affect" }, affectLayer);
|
||||
t.textContent = strings[f.key] || f.key;
|
||||
}
|
||||
}
|
||||
|
||||
function renderScaleReadout() {
|
||||
if (!ring) return;
|
||||
const s = ring.scales[ringIndex];
|
||||
$("scale-name").textContent = `${s.title} (${ringIndex + 1}/${ring.scales.length})`;
|
||||
}
|
||||
|
||||
function controls() {
|
||||
// One bipolar Mood dial (-4 dark .. 0 neutral .. +4 light) maps onto the engine's
|
||||
// two poles. The calibration gains are pinned to their locked 1.0 (full dial = full
|
||||
// effect), so the simulator sends no calibration — the server uses DEFAULT_CALIBRATION.
|
||||
const mood = +$("mood").value;
|
||||
return {
|
||||
content: $("content").value,
|
||||
left: +$("left").value, right: +$("right").value,
|
||||
dark: mood < 0 ? -mood : 0, light: mood > 0 ? mood : 0,
|
||||
volume: 2, brightness: 2,
|
||||
};
|
||||
}
|
||||
|
||||
let timer = null;
|
||||
async function update() {
|
||||
if (busy) return;
|
||||
const resp = await fetch("/api/alteration", {
|
||||
method: "POST", headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ controls: controls() }),
|
||||
function syncOutputs() {
|
||||
[...DIALS, ...MODEL].forEach((id) => {
|
||||
const out = document.getElementById(`${id}-out`);
|
||||
if (out) out.textContent = document.getElementById(id).value;
|
||||
});
|
||||
}
|
||||
|
||||
async function refresh() {
|
||||
syncOutputs();
|
||||
const state = readState();
|
||||
const resp = await fetch("/api/select", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(state),
|
||||
});
|
||||
if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; }
|
||||
const data = await resp.json();
|
||||
readout.textContent = JSON.stringify(data, null, 2);
|
||||
if (!data.content.video) { black.classList.remove("hidden"); return; }
|
||||
black.classList.add("hidden");
|
||||
try {
|
||||
ensureClipMedia();
|
||||
applyVideoLook(data.plan.grade.tone, data.plan.dream.intensity);
|
||||
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
|
||||
renderAffect(data.plan.affect.strength, data.plan.affect.intensity);
|
||||
const b = document.getElementById("err-banner");
|
||||
if (b) b.remove(); // render succeeded — clear any prior error
|
||||
} catch (err) {
|
||||
showError("render failed: " + (err && err.message ? err.message : err));
|
||||
throw err;
|
||||
|
||||
const pickEl = document.getElementById("pick");
|
||||
if (!data.pick) {
|
||||
pickEl.innerHTML = '<div class="void">∅ Void / rest — walls dark, audio silent.</div>';
|
||||
} else {
|
||||
const p = data.pick;
|
||||
pickEl.innerHTML =
|
||||
`<div class="title">${p.title}</div>` +
|
||||
`<div>mode ${p.mode} · coord (${p.left},${p.right},${p.dark},${p.light})</div>` +
|
||||
`<div>${p.rationale || ""}</div>`;
|
||||
}
|
||||
}
|
||||
|
||||
function debounced() { clearTimeout(timer); timer = setTimeout(update, 80); }
|
||||
|
||||
// --- Scale ring (endless encoder) ---
|
||||
|
||||
const FAST_BLEND_RATE = 2.5; // playback speed for a collapsed fast-spin pass
|
||||
|
||||
function playTransition(file, blended) {
|
||||
// Play one zoom/warp transition clip start-to-finish, with the alteration
|
||||
// layers hidden. A `blended` (fast-spin) pass runs at FAST_BLEND_RATE so a
|
||||
// quick encoder spin lands fast instead of grinding through every morph.
|
||||
// Resolves on 'ended' (or a safety timeout that scales with playback rate).
|
||||
return new Promise((resolve) => {
|
||||
overlay.style.opacity = "0";
|
||||
affectLayer.style.opacity = "0";
|
||||
tint.style.opacity = "0";
|
||||
vid.style.filter = "none";
|
||||
vid.loop = false;
|
||||
vid.style.opacity = "1";
|
||||
vid.src = mediaUrl(file);
|
||||
vid.playbackRate = blended ? FAST_BLEND_RATE : 1;
|
||||
let done = false;
|
||||
const finish = () => {
|
||||
if (done) return;
|
||||
done = true;
|
||||
vid.removeEventListener("ended", finish);
|
||||
vid.playbackRate = 1;
|
||||
resolve();
|
||||
};
|
||||
vid.addEventListener("ended", finish);
|
||||
vid.play().catch(finish);
|
||||
setTimeout(finish, blended ? 3000 : 6000);
|
||||
const poolEl = document.getElementById("pool");
|
||||
poolEl.innerHTML = "";
|
||||
data.pool.forEach((c, i) => {
|
||||
const li = document.createElement("li");
|
||||
if (i === 0) li.className = "winner";
|
||||
li.innerHTML = `${c.record.title} <span class="dist">d=${c.distance.toFixed(2)}</span>`;
|
||||
poolEl.appendChild(li);
|
||||
});
|
||||
|
||||
const point = { left: state.left, right: state.right, dark: state.dark, light: state.light };
|
||||
paintGrid(document.getElementById("brain-grid"), "left", "right", point, data.pool);
|
||||
paintGrid(document.getElementById("mood-grid"), "dark", "light", point, data.pool);
|
||||
}
|
||||
|
||||
async function advance(delta) {
|
||||
if (busy || !ring || ring.scales.length < 2) return;
|
||||
busy = true;
|
||||
try {
|
||||
const resp = await fetch("/api/ring/advance", {
|
||||
method: "POST", headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ from_index: ringIndex, delta }),
|
||||
});
|
||||
if (!resp.ok) return;
|
||||
const move = await resp.json();
|
||||
// A fast spin comes back collapsed to a single blended step (move.fast);
|
||||
// a slow spin chains one full transition per scale crossed.
|
||||
for (const step of move.steps) {
|
||||
await playTransition(step.file, step.blended);
|
||||
}
|
||||
ringIndex = move.to_index;
|
||||
currentClipId = null; // force the target scale's base media to (re)load
|
||||
renderScaleReadout();
|
||||
} finally {
|
||||
busy = false;
|
||||
update();
|
||||
}
|
||||
async function loadMeta() {
|
||||
const data = await (await fetch("/api/catalog/meta")).json();
|
||||
const byMode = Object.entries(data.by_mode).map(([k, v]) => `${k}:${v}`).join(" ");
|
||||
document.getElementById("meta").textContent = `${data.total} records · ${byMode}`;
|
||||
}
|
||||
|
||||
let wheelAccum = 0, wheelTimer = null;
|
||||
function onWheel(e) {
|
||||
e.preventDefault();
|
||||
wheelAccum += e.deltaY;
|
||||
clearTimeout(wheelTimer);
|
||||
wheelTimer = setTimeout(() => {
|
||||
const detents = Math.trunc(wheelAccum / 60) || (wheelAccum > 0 ? 1 : -1);
|
||||
wheelAccum = 0;
|
||||
if (detents) advance(detents); // wheel down = zoom inward (+)
|
||||
}, 90);
|
||||
function init() {
|
||||
buildGrid(document.getElementById("brain-grid"));
|
||||
buildGrid(document.getElementById("mood-grid"));
|
||||
document.querySelectorAll("input, select").forEach((el) =>
|
||||
el.addEventListener("input", refresh)
|
||||
);
|
||||
loadMeta();
|
||||
refresh();
|
||||
}
|
||||
|
||||
// Dev live-reload: poll the asset version and reload when it changes, so an open
|
||||
// tab never keeps running a stale renderer while we iterate (the readout updates
|
||||
// from the live API and masks it otherwise). Reloads on my edits AND on a server
|
||||
// restart. Dev-only convenience; harmless if the endpoint is absent.
|
||||
function devLiveReload() {
|
||||
let seen = null;
|
||||
setInterval(async () => {
|
||||
try {
|
||||
const v = (await (await fetch("/dev/version", { cache: "no-store" })).json()).version;
|
||||
if (seen === null) seen = v;
|
||||
else if (v !== seen) location.reload();
|
||||
} catch (_) { /* server restarting / endpoint absent — ignore */ }
|
||||
}, 1000);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
devLiveReload();
|
||||
try { initPaint(); } catch (e) { paintOK = false; paint.style.display = "none"; showError("WebGL init: " + e.message); }
|
||||
await loadData();
|
||||
renderScaleReadout();
|
||||
for (const id of ["content", "left", "right", "mood"]) {
|
||||
$(id).addEventListener("input", debounced);
|
||||
}
|
||||
$("zoom-in").addEventListener("click", () => advance(1));
|
||||
$("zoom-out").addEventListener("click", () => advance(-1));
|
||||
$("stage").addEventListener("wheel", onWheel, { passive: false });
|
||||
update();
|
||||
}
|
||||
main();
|
||||
document.addEventListener("DOMContentLoaded", init);
|
||||
|
||||
+53
-52
@@ -1,63 +1,64 @@
|
||||
<!doctype html>
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>HEF — Alteration Simulator</title>
|
||||
<link rel="stylesheet" href="/style.css" />
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>HEF — Curator's X-ray</title>
|
||||
<link rel="stylesheet" href="/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header><h1>Human Experience Filter — Alteration Preview</h1></header>
|
||||
<header>
|
||||
<h1>Experience Filter — Curator's X-ray</h1>
|
||||
<div id="meta" class="meta"></div>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<section class="stage" id="stage">
|
||||
<div class="screen">
|
||||
<video id="vid" loop muted playsinline></video>
|
||||
<canvas id="paint"></canvas>
|
||||
<div id="tint"></div>
|
||||
<svg id="overlay" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
|
||||
<svg id="affect" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
|
||||
<div id="black" class="black hidden"></div>
|
||||
<section class="controls">
|
||||
<h2>Dials</h2>
|
||||
<label>Mode
|
||||
<select id="mode">
|
||||
<option value="none">None</option>
|
||||
<option value="audio">Audio</option>
|
||||
<option value="video">Video</option>
|
||||
<option value="av" selected>A+V</option>
|
||||
</select>
|
||||
</label>
|
||||
<label>Left (analytical) <output id="left-out">0</output>
|
||||
<input type="range" id="left" min="0" max="4" step="1" value="0"></label>
|
||||
<label>Right (artistic) <output id="right-out">0</output>
|
||||
<input type="range" id="right" min="0" max="4" step="1" value="0"></label>
|
||||
<label>Dark (somber) <output id="dark-out">0</output>
|
||||
<input type="range" id="dark" min="0" max="4" step="1" value="0"></label>
|
||||
<label>Light (uplifting) <output id="light-out">0</output>
|
||||
<input type="range" id="light" min="0" max="4" step="1" value="0"></label>
|
||||
|
||||
<h2>Model knobs</h2>
|
||||
<label>Brain weight <output id="brain_weight-out">1</output>
|
||||
<input type="range" id="brain_weight" min="0" max="4" step="0.5" value="1"></label>
|
||||
<label>Mood weight <output id="mood_weight-out">1</output>
|
||||
<input type="range" id="mood_weight" min="0" max="4" step="0.5" value="1"></label>
|
||||
<label>Pool size <output id="pool_size-out">4</output>
|
||||
<input type="range" id="pool_size" min="1" max="10" step="1" value="4"></label>
|
||||
<label class="check"><input type="checkbox" id="approved_only"> Approved only</label>
|
||||
</section>
|
||||
|
||||
<section class="xray">
|
||||
<div class="pick">
|
||||
<h2>Picked</h2>
|
||||
<div id="pick"></div>
|
||||
</div>
|
||||
<div class="pool">
|
||||
<h2>Pool (nearest first)</h2>
|
||||
<ol id="pool"></ol>
|
||||
</div>
|
||||
<div class="maps">
|
||||
<h2>Coordinate maps</h2>
|
||||
<div class="map"><div class="label">Brain — Left × Right</div><div id="brain-grid" class="grid5"></div></div>
|
||||
<div class="map"><div class="label">Mood — Dark × Light</div><div id="mood-grid" class="grid5"></div></div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<fieldset>
|
||||
<legend>Content dial</legend>
|
||||
<select id="content">
|
||||
<option value="video">video</option>
|
||||
<option value="audio_video">audio + video</option>
|
||||
<option value="music_video">music + video</option>
|
||||
<option value="off">off (black)</option>
|
||||
<option value="white_noise">white noise (no video)</option>
|
||||
<option value="music">music (no video)</option>
|
||||
<option value="audio_track">audio track (no video)</option>
|
||||
</select>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>Scale ring (endless encoder)</legend>
|
||||
<div class="ring-control">
|
||||
<button type="button" id="zoom-out" title="zoom out (toward cosmos)">⊖ out</button>
|
||||
<span id="scale-name" class="scale-name">—</span>
|
||||
<button type="button" id="zoom-in" title="zoom in (toward microscopic)">in ⊕</button>
|
||||
</div>
|
||||
<p class="hint">Relative, endless — wraps past the smallest back to the largest. Scroll the stage too.</p>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>Experience knobs (0–4)</legend>
|
||||
<label>Left (analytical) <input type="range" id="left" min="0" max="4" value="0" /></label>
|
||||
<label>Right (dreamlike) <input type="range" id="right" min="0" max="4" value="0" /></label>
|
||||
<label>Mood — dark ◀ 0 ▶ light <input type="range" id="mood" min="-4" max="4" value="0" /></label>
|
||||
</fieldset>
|
||||
|
||||
|
||||
<fieldset>
|
||||
<legend>RenderPlan readout</legend>
|
||||
<pre id="readout">—</pre>
|
||||
</fieldset>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<script src="/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
+25
-56
@@ -1,57 +1,26 @@
|
||||
:root { color-scheme: dark; }
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; font: 14px/1.4 system-ui, sans-serif; background: #111; color: #eee; }
|
||||
header { padding: 0.6rem 1rem; background: #000; }
|
||||
h1 { font-size: 1rem; margin: 0; font-weight: 600; }
|
||||
main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; }
|
||||
.stage { flex: 1 1 640px; }
|
||||
.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000;
|
||||
border-radius: 6px; overflow: hidden; }
|
||||
#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; }
|
||||
/* Right-dream painterly canvas: a WebGL Kuwahara restyle of the LIVE video frames,
|
||||
drawn over the base. Edge-preserving, so motion stays as crisp as the source —
|
||||
the dream is the same footage, just stylized. The mood grade rides as a CSS
|
||||
filter here. Hidden when WebGL is unavailable (CSS fallback shows #vid). */
|
||||
#paint { position: absolute; inset: 0; width: 100%; height: 100%; object-fit: cover;
|
||||
pointer-events: none; transition: filter 0.2s ease; }
|
||||
#tint { position: absolute; inset: 0; pointer-events: none; opacity: 0;
|
||||
background: #28425f; mix-blend-mode: multiply;
|
||||
transition: opacity 0.2s ease; }
|
||||
#overlay { position: absolute; inset: 0; width: 100%; height: 100%;
|
||||
pointer-events: none; transition: opacity 0.2s ease; }
|
||||
/* Affect channel — its own layer so its opacity tracks min(left,right), not Left.
|
||||
Soft violet glow, distinct from the clinical reticles (the dream leaking in). */
|
||||
#affect { position: absolute; inset: 0; width: 100%; height: 100%;
|
||||
pointer-events: none; transition: opacity 0.35s ease;
|
||||
filter: drop-shadow(0 0 0.9px rgba(176, 120, 255, 0.95)); }
|
||||
.hud-affect { fill: #e3cfff; font: italic 600 3.6px Georgia, "Times New Roman", serif;
|
||||
letter-spacing: 0.06px; }
|
||||
/* Left-brain analytical HUD. Two sensor channels: detection (cyan) + measurement
|
||||
(amber). Corner-bracket reticles, translucent label chips, a status tag. */
|
||||
.hud-reticle { fill: none; stroke-width: 0.6; vector-effect: non-scaling-stroke;
|
||||
stroke-linecap: square; }
|
||||
.hud-reticle.detect { stroke: #6cf; }
|
||||
.hud-reticle.measure { stroke: #ffc46b; }
|
||||
.hud-chip-bg { fill-opacity: 0.72; }
|
||||
.hud-chip-bg.detect { fill: #001722; }
|
||||
.hud-chip-bg.measure { fill: #1c1200; }
|
||||
.hud-chip-text { font: 2.4px monospace; letter-spacing: 0.04px; }
|
||||
.hud-chip-text.detect { fill: #aee6ff; }
|
||||
.hud-chip-text.measure { fill: #ffd79a; }
|
||||
.hud-conf { fill: #6cf; opacity: 0.8; }
|
||||
.hud-status { fill: #8fdcff; font: 2.4px monospace; opacity: 0.9; letter-spacing: 0.15px; }
|
||||
.black { position: absolute; inset: 0; background: #000; }
|
||||
.hidden { display: none; }
|
||||
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem; }
|
||||
fieldset { border: 1px solid #333; border-radius: 6px; }
|
||||
legend { color: #9af; padding: 0 0.4rem; }
|
||||
label { display: block; margin: 0.4rem 0; }
|
||||
input[type=range], select { width: 100%; }
|
||||
#readout { background: #000; padding: 0.5rem; border-radius: 4px; font-size: 12px;
|
||||
white-space: pre-wrap; max-height: 240px; overflow: auto; }
|
||||
.ring-control { display: flex; align-items: center; gap: 0.4rem; }
|
||||
.ring-control button { flex: 0 0 auto; background: #1a2436; color: #9af;
|
||||
border: 1px solid #345; border-radius: 4px; padding: 0.3rem 0.5rem;
|
||||
cursor: pointer; font-size: 13px; }
|
||||
.ring-control button:hover { background: #243352; }
|
||||
.scale-name { flex: 1 1 auto; text-align: center; font-size: 12px; color: #cde; }
|
||||
.hint { margin: 0.3rem 0 0; font-size: 11px; color: #789; }
|
||||
body { margin: 0; font-family: -apple-system, system-ui, sans-serif; background: #0e0e16; color: #e6e6ee; }
|
||||
header { padding: 12px 20px; border-bottom: 1px solid #2a2a3a; display: flex; justify-content: space-between; align-items: baseline; }
|
||||
header h1 { font-size: 18px; margin: 0; }
|
||||
.meta { font-size: 12px; color: #9a9ab0; }
|
||||
main { display: grid; grid-template-columns: 280px 1fr; gap: 20px; padding: 20px; }
|
||||
.controls label { display: block; margin: 8px 0; font-size: 13px; }
|
||||
.controls input[type=range] { width: 100%; }
|
||||
.controls .check { display: flex; gap: 6px; align-items: center; }
|
||||
h2 { font-size: 13px; text-transform: uppercase; letter-spacing: .5px; color: #9a9ab0; }
|
||||
.xray { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; align-items: start; }
|
||||
.pick #pick { background: #1a1a2e; border: 1px solid #33334a; border-radius: 8px; padding: 14px; min-height: 80px; }
|
||||
.pick .title { font-size: 16px; font-weight: 600; }
|
||||
.pick .void { color: #7777aa; font-style: italic; }
|
||||
.pool ol { margin: 0; padding-left: 18px; font-size: 13px; }
|
||||
.pool li { margin: 4px 0; }
|
||||
.pool li.winner { color: #7fffd4; font-weight: 600; }
|
||||
.pool .dist { color: #9a9ab0; }
|
||||
.maps { grid-column: 1 / -1; display: flex; gap: 40px; }
|
||||
.grid5 { display: grid; grid-template-columns: repeat(5, 28px); grid-template-rows: repeat(5, 28px); gap: 3px; }
|
||||
.grid5 .cell { background: #1c1c2c; border: 1px solid #2a2a3a; border-radius: 3px; position: relative; }
|
||||
.grid5 .cell.cand { background: #3a3a66; }
|
||||
.grid5 .cell.point { outline: 2px solid #7fffd4; }
|
||||
.grid5 .cell .n { position: absolute; right: 2px; bottom: 1px; font-size: 9px; color: #aab; }
|
||||
.label { font-size: 11px; color: #9a9ab0; margin-bottom: 4px; }
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
import pytest
|
||||
|
||||
from simulator.bake_right_variants import RIGHT_LEVELS, right_strength_params
|
||||
|
||||
|
||||
def test_levels_are_one_through_four():
|
||||
assert RIGHT_LEVELS == (1, 2, 3, 4)
|
||||
|
||||
|
||||
def test_strength_increases_with_level():
|
||||
keys = [right_strength_params(l)[0] for l in RIGHT_LEVELS]
|
||||
assert keys == sorted(keys) # monotonic ramp
|
||||
assert keys[0] < keys[-1] # subtle -> strong
|
||||
|
||||
|
||||
def test_tween_strength_is_a_fraction_of_key_strength():
|
||||
for l in RIGHT_LEVELS:
|
||||
key, tween = right_strength_params(l)
|
||||
assert 0.0 < tween < key # tween is a light refine only
|
||||
|
||||
|
||||
def test_rejects_out_of_range_level():
|
||||
with pytest.raises(ValueError):
|
||||
right_strength_params(0)
|
||||
with pytest.raises(ValueError):
|
||||
right_strength_params(5)
|
||||
@@ -1,158 +0,0 @@
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from player.ring import RingMove, ScaleRing, TransitionStep, advance_ring
|
||||
from simulator.clips import (
|
||||
Clip,
|
||||
load_manifest,
|
||||
load_ring,
|
||||
ring_move_to_dict,
|
||||
ring_to_dict,
|
||||
)
|
||||
|
||||
|
||||
def _manifest_dict():
|
||||
return {
|
||||
"clips": [
|
||||
{
|
||||
"id": "forest",
|
||||
"title": "Yosemite Falls (neutral)",
|
||||
"base_file": "forest/base.mp4",
|
||||
"license": "poc-sample",
|
||||
"source": "hef-poc",
|
||||
"right_variants": {
|
||||
"4": {"file": "forest/right4.mp4", "model": "sd-turbo+flow"},
|
||||
"1": {"file": "forest/right1.mp4"},
|
||||
},
|
||||
"annotations": [
|
||||
{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1},
|
||||
{"key": "detected.conifer", "box": [0.6, 0.1, 0.2, 0.2], "min_level": 3},
|
||||
],
|
||||
"strings": {"en": {"detected.water": "flowing water", "detected.conifer": "conifer"}},
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
def test_load_manifest_parses_clips(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_manifest_dict()))
|
||||
clips = load_manifest(p)
|
||||
assert len(clips) == 1
|
||||
c = clips[0]
|
||||
assert isinstance(c, Clip)
|
||||
assert c.id == "forest"
|
||||
assert c.base_file == "forest/base.mp4"
|
||||
|
||||
|
||||
def test_clip_lists_variant_files_by_strength(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_manifest_dict()))
|
||||
c = load_manifest(p)[0]
|
||||
# variant 0 is always the raw base; authored strengths come from the manifest
|
||||
assert c.variant_file(0) == "forest/base.mp4"
|
||||
assert c.variant_file(4) == "forest/right4.mp4"
|
||||
assert c.variant_file(1) == "forest/right1.mp4"
|
||||
# an unauthored strength falls back to the raw base
|
||||
assert c.variant_file(2) == "forest/base.mp4"
|
||||
|
||||
|
||||
def test_clip_serializes_to_dict_for_the_api(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_manifest_dict()))
|
||||
d = load_manifest(p)[0].to_dict()
|
||||
assert d["id"] == "forest"
|
||||
assert d["base_file"] == "forest/base.mp4"
|
||||
assert d["annotations"][0]["key"] == "detected.water"
|
||||
assert d["strings"]["en"]["detected.water"] == "flowing water"
|
||||
# variant map is exposed keyed by strength string, including 0 -> base
|
||||
assert d["right_variants"]["0"]["file"] == "forest/base.mp4"
|
||||
assert d["right_variants"]["4"]["file"] == "forest/right4.mp4"
|
||||
|
||||
|
||||
def test_missing_manifest_raises(tmp_path):
|
||||
with pytest.raises(FileNotFoundError):
|
||||
load_manifest(tmp_path / "nope.json")
|
||||
|
||||
|
||||
def _ring_manifest_dict():
|
||||
return {
|
||||
"clips": [],
|
||||
"ring": {
|
||||
"scales": [
|
||||
{"id": "cosmos", "clip_id": "cosmos"},
|
||||
{"id": "forest", "clip_id": "forest"},
|
||||
{"id": "abyss", "clip_id": "abyss"},
|
||||
],
|
||||
"transitions": [
|
||||
{"file": "transitions/cosmos-forest.mp4", "model": "placeholder"},
|
||||
{"file": "transitions/forest-abyss.mp4", "model": "placeholder"},
|
||||
{"file": "transitions/abyss-cosmos.mp4", "model": "placeholder"},
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_load_ring_builds_a_scale_ring(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_ring_manifest_dict()))
|
||||
ring = load_ring(p)
|
||||
assert isinstance(ring, ScaleRing)
|
||||
assert len(ring) == 3
|
||||
assert ring.scales[0].id == "cosmos"
|
||||
assert ring.scales[0].clip_id == "cosmos"
|
||||
assert ring.transitions[2].file == "transitions/abyss-cosmos.mp4"
|
||||
assert ring.transitions[0].model == "placeholder"
|
||||
|
||||
|
||||
def test_load_ring_is_none_when_absent(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_manifest_dict())) # no "ring" key
|
||||
assert load_ring(p) is None
|
||||
|
||||
|
||||
def test_ring_to_dict_carries_scale_titles_from_clips(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
md = _ring_manifest_dict()
|
||||
md["clips"] = [
|
||||
{"id": "cosmos", "title": "NASA cosmos", "base_file": "cosmos/base.mp4"},
|
||||
{"id": "forest", "title": "Yosemite", "base_file": "forest/base.mp4"},
|
||||
{"id": "abyss", "title": "NOAA abyss", "base_file": "abyss/base.mp4"},
|
||||
]
|
||||
p.write_text(json.dumps(md))
|
||||
ring = load_ring(p)
|
||||
clips = load_manifest(p)
|
||||
d = ring_to_dict(ring, clips)
|
||||
assert [s["id"] for s in d["scales"]] == ["cosmos", "forest", "abyss"]
|
||||
assert d["scales"][0]["title"] == "NASA cosmos"
|
||||
assert d["scales"][0]["clip_id"] == "cosmos"
|
||||
assert d["transitions"][2]["file"] == "transitions/abyss-cosmos.mp4"
|
||||
|
||||
|
||||
def test_ring_move_to_dict_serializes_steps(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_ring_manifest_dict()))
|
||||
ring = load_ring(p)
|
||||
move = advance_ring(ring, from_index=2, delta=1) # abyss -> wrap -> cosmos
|
||||
d = ring_move_to_dict(move, ring)
|
||||
assert d["from_index"] == 2
|
||||
assert d["to_index"] == 0
|
||||
assert d["wrapped"] is True
|
||||
assert d["target_clip_id"] == "cosmos"
|
||||
assert d["steps"][0]["file"] == "transitions/abyss-cosmos.mp4"
|
||||
assert d["steps"][0]["reversed"] is False
|
||||
assert d["steps"][0]["to_index"] == 0
|
||||
assert d["fast"] is False
|
||||
assert d["steps"][0]["blended"] is False
|
||||
|
||||
|
||||
def test_ring_move_to_dict_marks_fast_blended_pass(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_ring_manifest_dict()))
|
||||
ring = load_ring(p)
|
||||
move = advance_ring(ring, from_index=0, delta=4, fast_spin_threshold=3)
|
||||
d = ring_move_to_dict(move, ring)
|
||||
assert d["fast"] is True
|
||||
assert len(d["steps"]) == 1
|
||||
assert d["steps"][0]["blended"] is True
|
||||
@@ -0,0 +1,40 @@
|
||||
from hef.catalog import validate_catalog
|
||||
from hef.selection import CONTENT_MODES
|
||||
from simulator.fixtures import generate_fixture_catalog
|
||||
|
||||
|
||||
def test_fixture_catalog_is_valid():
|
||||
records = generate_fixture_catalog()
|
||||
validate_catalog(records) # raises on any invalid record or duplicate id
|
||||
|
||||
|
||||
def test_fixture_catalog_spans_the_coordinate_space():
|
||||
records = generate_fixture_catalog()
|
||||
coords = {(r.left, r.right, r.dark, r.light) for r in records}
|
||||
# all 625 cells of the 5x5 brain x 5x5 mood space are present
|
||||
assert len(coords) == 625
|
||||
|
||||
|
||||
def test_fixture_catalog_has_every_content_mode():
|
||||
records = generate_fixture_catalog()
|
||||
present = {r.mode for r in records}
|
||||
assert CONTENT_MODES <= present
|
||||
|
||||
|
||||
def test_fixture_catalog_mixes_review_statuses():
|
||||
records = generate_fixture_catalog()
|
||||
statuses = {r.review_status for r in records}
|
||||
assert statuses == {"proposed", "approved"}
|
||||
|
||||
|
||||
def test_fixture_catalog_references_no_real_media():
|
||||
records = generate_fixture_catalog()
|
||||
assert all(r.file_path == "" for r in records)
|
||||
|
||||
|
||||
def test_fixture_catalog_is_deterministic():
|
||||
a = generate_fixture_catalog(seed=42)
|
||||
b = generate_fixture_catalog(seed=42)
|
||||
assert [r.id for r in a] == [r.id for r in b]
|
||||
assert [r.mode for r in a] == [r.mode for r in b]
|
||||
assert [r.review_status for r in a] == [r.review_status for r in b]
|
||||
@@ -1,54 +0,0 @@
|
||||
from tools.pipeline.ffmpeg_ops import frame_args, crossfade_loop_args, transcode_args
|
||||
|
||||
|
||||
def test_frame_args_trims_scales_pads_and_strips_audio():
|
||||
args = frame_args(
|
||||
"src.mp4", "out.mp4",
|
||||
start=2.0, duration=8.0, width=1920, height=1080, ff="ffmpeg",
|
||||
)
|
||||
assert args[0] == "ffmpeg"
|
||||
assert "-ss" in args and args[args.index("-ss") + 1] == "2.0"
|
||||
assert "-t" in args and args[args.index("-t") + 1] == "8.0"
|
||||
assert "-an" in args # audio stripped (bases are video-only)
|
||||
vf = args[args.index("-vf") + 1]
|
||||
assert "scale=1920:1080:force_original_aspect_ratio=decrease" in vf
|
||||
assert "pad=1920:1080" in vf
|
||||
assert "format=yuv420p" in vf
|
||||
assert args[-1] == "out.mp4"
|
||||
|
||||
|
||||
def test_crossfade_loop_builds_head_mid_tail_xfade_concat():
|
||||
args = crossfade_loop_args("in.mp4", "loop.mp4", duration=8.0, overlap=1.0)
|
||||
fc = args[args.index("-filter_complex") + 1]
|
||||
assert "trim=0:1.0" in fc # head = first O secs
|
||||
assert "trim=1.0:7.0" in fc # mid = [O, D-O]
|
||||
assert "trim=7.0:8.0" in fc # tail = last O secs
|
||||
assert "xfade=transition=fade:duration=1.0:offset=0" in fc
|
||||
assert "concat=n=2:v=1" in fc
|
||||
assert "-an" in args
|
||||
assert args[-1] == "loop.mp4"
|
||||
|
||||
|
||||
def test_crossfade_loop_rejects_overlap_too_large():
|
||||
import pytest
|
||||
with pytest.raises(ValueError):
|
||||
crossfade_loop_args("in.mp4", "loop.mp4", duration=4.0, overlap=2.0)
|
||||
|
||||
|
||||
def test_crossfade_loop_rejects_nonpositive_overlap():
|
||||
import pytest
|
||||
with pytest.raises(ValueError):
|
||||
crossfade_loop_args("in.mp4", "loop.mp4", duration=8.0, overlap=0.0)
|
||||
|
||||
|
||||
def test_transcode_args_high_profile_even_dims_faststart():
|
||||
args = transcode_args("loop.mp4", "proxy.mp4", width=1920, height=1080,
|
||||
crf=20, fps=30)
|
||||
assert "-profile:v" in args and args[args.index("-profile:v") + 1] == "high"
|
||||
assert "-crf" in args and args[args.index("-crf") + 1] == "20"
|
||||
vf = args[args.index("-vf") + 1]
|
||||
assert "scale=1920:1080:force_original_aspect_ratio=decrease" in vf
|
||||
assert "fps=30" in vf
|
||||
assert "+faststart" in args
|
||||
assert "-an" in args
|
||||
assert args[-1] == "proxy.mp4"
|
||||
@@ -1,42 +0,0 @@
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from tools.pipeline.run import probe_duration, process_clip, resolve_ffmpeg
|
||||
|
||||
_FOREST = Path("simulator/sample_media/forest/base.mp4")
|
||||
|
||||
|
||||
def _ffmpeg_available() -> bool:
|
||||
try:
|
||||
resolve_ffmpeg()
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
_HAS_FF = _ffmpeg_available()
|
||||
|
||||
|
||||
@pytest.mark.skipif(not _FOREST.exists(),
|
||||
reason="forest POC base absent (run simulator/setup_sample_media.py)")
|
||||
def test_probe_duration_reads_forest_base():
|
||||
if not _FOREST.exists():
|
||||
pytest.skip("forest POC base absent")
|
||||
assert probe_duration(_FOREST) > 0
|
||||
|
||||
|
||||
@pytest.mark.skipif(not _HAS_FF,
|
||||
reason="neither system ffmpeg nor imageio-ffmpeg available")
|
||||
@pytest.mark.skipif(not _FOREST.exists(),
|
||||
reason="forest POC base absent (run simulator/setup_sample_media.py)")
|
||||
def test_process_clip_emits_proxy_and_master(tmp_path):
|
||||
out = process_clip(_FOREST, tmp_path, overlap=0.5, start=0.0, duration=4.0)
|
||||
assert out["proxy"].exists() and out["master"].exists()
|
||||
# Proxy is exactly 1920×1080 — verified with cv2, no ffprobe dependency.
|
||||
import cv2
|
||||
cap = cv2.VideoCapture(str(out["proxy"]))
|
||||
w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
|
||||
h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
|
||||
cap.release()
|
||||
assert (w, h) == (1920, 1080)
|
||||
@@ -1,47 +0,0 @@
|
||||
from tools.pipeline.manifest import upsert_clip, add_ring_scale, rebuild_ring_edges
|
||||
|
||||
|
||||
def _base():
|
||||
return {"clips": [{"id": "cosmos", "title": "Cosmos"}],
|
||||
"ring": {"scales": [{"id": "cosmos", "clip_id": "cosmos"}],
|
||||
"transitions": []}}
|
||||
|
||||
|
||||
def test_upsert_clip_replaces_by_id_else_appends():
|
||||
m = _base()
|
||||
m = upsert_clip(m, {"id": "reef", "title": "Reef"})
|
||||
assert [c["id"] for c in m["clips"]] == ["cosmos", "reef"]
|
||||
m = upsert_clip(m, {"id": "reef", "title": "Coral Reef"}) # replace, not dup
|
||||
assert [c["id"] for c in m["clips"]] == ["cosmos", "reef"]
|
||||
assert m["clips"][1]["title"] == "Coral Reef"
|
||||
|
||||
|
||||
def test_add_ring_scale_inserts_after_named_scale():
|
||||
m = _base()
|
||||
m = add_ring_scale(m, "orbit", "orbit", after="cosmos")
|
||||
assert [s["id"] for s in m["ring"]["scales"]] == ["cosmos", "orbit"]
|
||||
|
||||
|
||||
def test_rebuild_ring_edges_one_transition_per_edge_with_wrap():
|
||||
m = _base()
|
||||
m = add_ring_scale(m, "orbit", "orbit", after="cosmos")
|
||||
m = add_ring_scale(m, "abyss", "abyss", after="orbit")
|
||||
m = rebuild_ring_edges(m)
|
||||
files = [t["file"] for t in m["ring"]["transitions"]]
|
||||
assert files == [
|
||||
"transitions/cosmos-orbit.mp4",
|
||||
"transitions/orbit-abyss.mp4",
|
||||
"transitions/abyss-cosmos.mp4", # wrap edge
|
||||
]
|
||||
|
||||
|
||||
def test_add_ring_scale_appends_when_after_missing():
|
||||
m = _base()
|
||||
m = add_ring_scale(m, "abyss", "abyss", after="nope")
|
||||
assert [s["id"] for s in m["ring"]["scales"]] == ["cosmos", "abyss"]
|
||||
|
||||
|
||||
def test_rebuild_ring_edges_single_scale_self_wrap():
|
||||
m = _base() # one scale: cosmos
|
||||
m = rebuild_ring_edges(m)
|
||||
assert [t["file"] for t in m["ring"]["transitions"]] == ["transitions/cosmos-cosmos.mp4"]
|
||||
@@ -1,20 +0,0 @@
|
||||
from tools.pipeline.provenance import Provenance
|
||||
|
||||
|
||||
def test_provenance_to_dict_roundtrip():
|
||||
p = Provenance(
|
||||
scale="abyss",
|
||||
license="public-domain (US Gov, 17 U.S.C. §105)",
|
||||
source="NOAA Ocean Exploration",
|
||||
url="https://oceanexplorer.noaa.gov/",
|
||||
fetched_at="2026-06-24",
|
||||
)
|
||||
d = p.to_dict()
|
||||
assert d == {
|
||||
"scale": "abyss",
|
||||
"license": "public-domain (US Gov, 17 U.S.C. §105)",
|
||||
"source": "NOAA Ocean Exploration",
|
||||
"url": "https://oceanexplorer.noaa.gov/",
|
||||
"fetched_at": "2026-06-24",
|
||||
}
|
||||
assert Provenance.from_dict(d) == p
|
||||
+14
-112
@@ -2,15 +2,11 @@ import pytest
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import (
|
||||
DEFAULT_CALIBRATION,
|
||||
AffectOverlay,
|
||||
AnalyticalOverlay,
|
||||
Calibration,
|
||||
ColorGrade,
|
||||
Dream,
|
||||
RenderPlan,
|
||||
Restyle,
|
||||
plan_alteration,
|
||||
render_plan_to_dict,
|
||||
)
|
||||
|
||||
|
||||
@@ -21,77 +17,30 @@ def _coord(left=0, right=0, dark=0, light=0):
|
||||
def test_all_zero_knobs_is_the_unaltered_base():
|
||||
plan = plan_alteration(_coord())
|
||||
assert plan.is_identity
|
||||
assert plan.overlay.level == 0
|
||||
assert plan.overlay.intensity == 0.0
|
||||
assert plan.dream.strength == 0
|
||||
assert plan.dream.intensity == 0.0
|
||||
assert plan.restyle.blend == 0.0
|
||||
assert plan.grade.tone == 0.0
|
||||
assert plan.grade.is_identity
|
||||
|
||||
|
||||
def test_left_drives_the_analytical_overlay_only():
|
||||
plan = plan_alteration(_coord(left=4))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.dream.strength == 0 # Left does not touch the substrate
|
||||
assert plan.dream.intensity == 0.0
|
||||
assert plan.restyle.blend == 0.0 # Left does not touch the substrate
|
||||
assert plan.grade.tone == 0.0
|
||||
|
||||
|
||||
def test_right_drives_the_deterministic_dream_only():
|
||||
# Right-axis dream reframe (session 0013): Right is a live deterministic
|
||||
# haze, carried as strength (0..4) + intensity (0..1), not a baked variant.
|
||||
def test_right_drives_the_restyle_substrate_only():
|
||||
plan = plan_alteration(_coord(right=2))
|
||||
assert plan.dream.strength == 2
|
||||
assert plan.dream.intensity == pytest.approx(0.5)
|
||||
assert plan.overlay.level == 0 # Right does not add overlay
|
||||
assert plan.restyle.blend == 0.5
|
||||
assert plan.overlay.intensity == 0.0 # Right does not add overlay
|
||||
|
||||
|
||||
def test_left_and_right_stack_not_cancel():
|
||||
# design §4.2: whole-brain corner = dream substrate WITH labels on top
|
||||
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
|
||||
plan = plan_alteration(_coord(left=4, right=4))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.dream.strength == 4
|
||||
assert plan.dream.intensity == 1.0
|
||||
|
||||
|
||||
def test_affect_needs_both_knobs_up():
|
||||
# affect-channel design (session 0013): emotion-words surface only when BOTH
|
||||
# the Left and Right knobs are up — either at 0 means no affect at all.
|
||||
assert plan_alteration(_coord(left=4, right=0)).affect.strength == 0
|
||||
assert plan_alteration(_coord(left=0, right=4)).affect.strength == 0
|
||||
assert plan_alteration(_coord(left=4, right=0)).affect.intensity == 0.0
|
||||
assert plan_alteration(_coord(left=0, right=4)).affect.intensity == 0.0
|
||||
|
||||
|
||||
def test_affect_strength_is_the_smaller_knob():
|
||||
# strength = min(left, right) — the smaller knob gates the channel.
|
||||
assert plan_alteration(_coord(left=2, right=4)).affect.strength == 2
|
||||
assert plan_alteration(_coord(left=4, right=2)).affect.strength == 2
|
||||
assert plan_alteration(_coord(left=4, right=4)).affect.strength == 4
|
||||
assert plan_alteration(_coord(left=3, right=1)).affect.strength == 1
|
||||
|
||||
|
||||
def test_affect_intensity_scales_with_strength_over_full_scale():
|
||||
for lo in range(5):
|
||||
plan = plan_alteration(_coord(left=lo, right=4))
|
||||
assert plan.affect.intensity == pytest.approx(lo / 4)
|
||||
assert plan_alteration(_coord(left=4, right=4)).affect.intensity == 1.0
|
||||
|
||||
|
||||
def test_affect_intensity_honors_overlay_gain():
|
||||
# affect rides the same overlay_gain as the analytical HUD it belongs to.
|
||||
cal = Calibration(overlay_gain=0.5)
|
||||
assert plan_alteration(_coord(left=4, right=4), cal).affect.intensity == pytest.approx(0.5)
|
||||
clamp = Calibration(overlay_gain=10.0)
|
||||
assert plan_alteration(_coord(left=4, right=4), clamp).affect.intensity == 1.0
|
||||
|
||||
|
||||
def test_affect_active_implies_non_identity():
|
||||
# min(left,right) > 0 implies left > 0, so an active affect plan is never identity.
|
||||
plan = plan_alteration(_coord(left=2, right=3))
|
||||
assert plan.affect.strength == 2
|
||||
assert not plan.is_identity
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 1.0
|
||||
|
||||
|
||||
def test_light_pole_grades_warm_positive_tone():
|
||||
@@ -101,7 +50,8 @@ def test_light_pole_grades_warm_positive_tone():
|
||||
|
||||
|
||||
def test_dark_pole_grades_cool_negative_tone():
|
||||
assert plan_alteration(_coord(dark=4)).grade.tone == -1.0
|
||||
plan = plan_alteration(_coord(dark=4))
|
||||
assert plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_equal_dark_and_light_is_identity_grade():
|
||||
@@ -116,62 +66,14 @@ def test_dark_minus_light_sets_intermediate_tone():
|
||||
|
||||
|
||||
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
|
||||
# design §4.2 "Dark + analytical": cold measurement over a melancholy scene
|
||||
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.dream.strength == 2
|
||||
assert plan.dream.intensity == pytest.approx(0.5)
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 0.5
|
||||
assert plan.grade.tone == -1.0
|
||||
assert not plan.is_identity
|
||||
|
||||
|
||||
def test_default_calibration_is_locked():
|
||||
# Session 0010: the knob->strength calibration is LOCKED to these values,
|
||||
# settled by eye in the simulator (closes the open session-0006 decision).
|
||||
# This pins the literal constants so they can't drift silently; changing the
|
||||
# locked feel is a deliberate edit here + in alteration.py.
|
||||
assert DEFAULT_CALIBRATION.mood_gain == 1.0
|
||||
assert DEFAULT_CALIBRATION.overlay_gain == 1.0
|
||||
assert DEFAULT_CALIBRATION.dream_gain == 1.0
|
||||
|
||||
|
||||
def test_default_calibration_is_behavior_preserving():
|
||||
# DEFAULT_CALIBRATION reproduces the per-axis helpers exactly.
|
||||
for left in range(5):
|
||||
assert plan_alteration(_coord(left=left)).overlay.intensity == pytest.approx(left / 4)
|
||||
for right in range(5):
|
||||
assert plan_alteration(_coord(right=right)).dream.strength == right
|
||||
assert plan_alteration(_coord(right=right)).dream.intensity == pytest.approx(right / 4)
|
||||
for dark in range(5):
|
||||
for light in range(5):
|
||||
expected = (light - dark) / 4
|
||||
assert plan_alteration(_coord(dark=dark, light=light)).grade.tone == pytest.approx(expected)
|
||||
|
||||
|
||||
def test_custom_calibration_scales_mood_overlay_and_dream():
|
||||
cal = Calibration(mood_gain=0.5, overlay_gain=0.5, dream_gain=0.5)
|
||||
assert plan_alteration(_coord(light=4), cal).grade.tone == pytest.approx(0.5)
|
||||
assert plan_alteration(_coord(left=4), cal).overlay.intensity == pytest.approx(0.5)
|
||||
assert plan_alteration(_coord(right=4), cal).dream.intensity == pytest.approx(0.5)
|
||||
|
||||
|
||||
def test_calibration_gain_is_clamped_to_unit_range():
|
||||
cal = Calibration(mood_gain=10.0, overlay_gain=10.0, dream_gain=10.0)
|
||||
assert plan_alteration(_coord(light=4), cal).grade.tone == 1.0 # clamped, not 10
|
||||
assert plan_alteration(_coord(left=4), cal).overlay.intensity == 1.0
|
||||
assert plan_alteration(_coord(right=4), cal).dream.intensity == 1.0
|
||||
|
||||
|
||||
def test_render_plan_to_dict_round_trips_the_numbers():
|
||||
d = render_plan_to_dict(plan_alteration(_coord(left=4, right=2, dark=4, light=0)))
|
||||
assert d == {
|
||||
"grade": {"tone": -1.0},
|
||||
"overlay": {"level": 4, "intensity": 1.0},
|
||||
"affect": {"strength": 2, "intensity": 0.5},
|
||||
"dream": {"strength": 2, "intensity": 0.5},
|
||||
"is_identity": False,
|
||||
}
|
||||
|
||||
|
||||
def test_render_plan_is_frozen():
|
||||
plan = plan_alteration(_coord())
|
||||
with pytest.raises(Exception):
|
||||
|
||||
@@ -1,208 +0,0 @@
|
||||
import pytest
|
||||
|
||||
from player.ring import (
|
||||
RingError,
|
||||
RingMove,
|
||||
Scale,
|
||||
ScaleRing,
|
||||
Transition,
|
||||
TransitionStep,
|
||||
advance_ring,
|
||||
scale_at,
|
||||
)
|
||||
|
||||
|
||||
def _ring():
|
||||
# cosmos (largest, idx 0) -> forest -> abyss (smallest) -> wraps to cosmos
|
||||
return ScaleRing(
|
||||
scales=(
|
||||
Scale(id="cosmos", clip_id="cosmos"),
|
||||
Scale(id="forest", clip_id="forest"),
|
||||
Scale(id="abyss", clip_id="abyss"),
|
||||
),
|
||||
transitions=(
|
||||
Transition(file="t/cosmos-forest.mp4"), # edge 0: cosmos->forest
|
||||
Transition(file="t/forest-abyss.mp4"), # edge 1: forest->abyss
|
||||
Transition(file="t/abyss-cosmos.mp4"), # edge 2: abyss->cosmos (wrap)
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def test_ring_needs_one_transition_per_scale():
|
||||
with pytest.raises(RingError):
|
||||
ScaleRing(
|
||||
scales=(Scale("a", "a"), Scale("b", "b")),
|
||||
transitions=(Transition("t/ab.mp4"),), # 1 edge for 2 scales
|
||||
)
|
||||
|
||||
|
||||
def test_ring_rejects_empty():
|
||||
with pytest.raises(RingError):
|
||||
ScaleRing(scales=(), transitions=())
|
||||
|
||||
|
||||
def test_scale_at_wraps_modulo():
|
||||
r = _ring()
|
||||
assert scale_at(r, 0).id == "cosmos"
|
||||
assert scale_at(r, 3).id == "cosmos" # wraps
|
||||
assert scale_at(r, -1).id == "abyss" # negative wraps
|
||||
assert scale_at(r, 4).id == "forest"
|
||||
|
||||
|
||||
def test_advance_one_step_inward_plays_edge_forward():
|
||||
r = _ring()
|
||||
move = advance_ring(r, from_index=0, delta=1)
|
||||
assert isinstance(move, RingMove)
|
||||
assert move.from_index == 0
|
||||
assert move.to_index == 1
|
||||
assert move.wrapped is False
|
||||
assert move.steps == (
|
||||
TransitionStep(edge=0, reversed=False, file="t/cosmos-forest.mp4", to_index=1),
|
||||
)
|
||||
|
||||
|
||||
def test_advance_one_step_outward_plays_edge_reversed():
|
||||
r = _ring()
|
||||
move = advance_ring(r, from_index=1, delta=-1)
|
||||
assert move.to_index == 0
|
||||
assert move.wrapped is False
|
||||
# going forest(1) -> cosmos(0) is edge 0 played in reverse
|
||||
assert move.steps == (
|
||||
TransitionStep(edge=0, reversed=True, file="t/cosmos-forest.mp4", to_index=0),
|
||||
)
|
||||
|
||||
|
||||
def test_advance_inward_past_smallest_wraps_to_largest():
|
||||
r = _ring()
|
||||
# at abyss (smallest, idx 2), zoom in -> wraps to cosmos (idx 0) via edge 2
|
||||
move = advance_ring(r, from_index=2, delta=1)
|
||||
assert move.to_index == 0
|
||||
assert move.wrapped is True
|
||||
assert move.steps == (
|
||||
TransitionStep(edge=2, reversed=False, file="t/abyss-cosmos.mp4", to_index=0),
|
||||
)
|
||||
|
||||
|
||||
def test_advance_outward_past_largest_wraps_to_smallest():
|
||||
r = _ring()
|
||||
# at cosmos (idx 0), zoom out -> wraps to abyss (idx 2) via edge 2 reversed
|
||||
move = advance_ring(r, from_index=0, delta=-1)
|
||||
assert move.to_index == 2
|
||||
assert move.wrapped is True
|
||||
assert move.steps == (
|
||||
TransitionStep(edge=2, reversed=True, file="t/abyss-cosmos.mp4", to_index=2),
|
||||
)
|
||||
|
||||
|
||||
def test_advance_multi_detent_chains_transitions():
|
||||
r = _ring()
|
||||
move = advance_ring(r, from_index=0, delta=2)
|
||||
assert move.to_index == 2
|
||||
assert move.wrapped is False
|
||||
assert move.steps == (
|
||||
TransitionStep(edge=0, reversed=False, file="t/cosmos-forest.mp4", to_index=1),
|
||||
TransitionStep(edge=1, reversed=False, file="t/forest-abyss.mp4", to_index=2),
|
||||
)
|
||||
|
||||
|
||||
def test_advance_full_loop_returns_to_start_and_marks_wrap():
|
||||
r = _ring()
|
||||
move = advance_ring(r, from_index=0, delta=3)
|
||||
assert move.to_index == 0
|
||||
assert move.wrapped is True
|
||||
assert len(move.steps) == 3
|
||||
|
||||
|
||||
def test_advance_zero_is_a_noop():
|
||||
r = _ring()
|
||||
move = advance_ring(r, from_index=1, delta=0)
|
||||
assert move.from_index == 1
|
||||
assert move.to_index == 1
|
||||
assert move.steps == ()
|
||||
assert move.wrapped is False
|
||||
|
||||
|
||||
def test_advance_normalizes_from_index():
|
||||
r = _ring()
|
||||
# from_index out of range is taken modulo N first
|
||||
move = advance_ring(r, from_index=3, delta=1)
|
||||
assert move.from_index == 0
|
||||
assert move.to_index == 1
|
||||
|
||||
|
||||
def test_degenerate_single_scale_ring_is_a_noop():
|
||||
r = ScaleRing(scales=(Scale("solo", "solo"),), transitions=())
|
||||
move = advance_ring(r, from_index=0, delta=5)
|
||||
assert move.to_index == 0
|
||||
assert move.steps == ()
|
||||
assert move.wrapped is False
|
||||
|
||||
|
||||
# --- fast-spin: a faster blended pass past a speed threshold (scales design §3) ---
|
||||
#
|
||||
# A fast encoder spin batches many detents into one advance() call, so |delta| is
|
||||
# the input adapter's proxy for spin speed. Past a configured threshold the move
|
||||
# collapses to a SINGLE blended pass (the arrival-edge clip, played fast) instead
|
||||
# of chaining N full transitions. The policy is opt-in (default off) because the
|
||||
# pure function cannot know wall-clock spin speed — the input layer enables it.
|
||||
|
||||
|
||||
def test_fast_spin_disabled_by_default_chains_every_detent():
|
||||
r = _ring()
|
||||
move = advance_ring(r, from_index=0, delta=5)
|
||||
assert move.fast is False
|
||||
assert len(move.steps) == 5
|
||||
assert all(not s.blended for s in move.steps)
|
||||
|
||||
|
||||
def test_fast_spin_below_threshold_still_chains():
|
||||
r = _ring()
|
||||
move = advance_ring(r, from_index=0, delta=2, fast_spin_threshold=3)
|
||||
assert move.fast is False
|
||||
assert len(move.steps) == 2
|
||||
assert move.steps[0].edge == 0 and move.steps[1].edge == 1
|
||||
|
||||
|
||||
def test_fast_spin_at_or_above_threshold_collapses_to_one_blended_step():
|
||||
r = _ring()
|
||||
# 4 inward detents from cosmos(0): 0->1->2->0->1, lands on forest(1),
|
||||
# crosses the abyss->cosmos seam, last edge crossed is edge 0 (forward).
|
||||
move = advance_ring(r, from_index=0, delta=4, fast_spin_threshold=3)
|
||||
assert move.fast is True
|
||||
assert move.to_index == 1
|
||||
assert move.wrapped is True
|
||||
assert move.steps == (
|
||||
TransitionStep(
|
||||
edge=0, reversed=False, file="t/cosmos-forest.mp4", to_index=1, blended=True
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def test_fast_spin_outward_blended_step_is_reversed():
|
||||
r = _ring()
|
||||
# 4 outward detents from cosmos(0): 0->2->1->0->2, lands on abyss(2);
|
||||
# outward arrival crosses edge 2 (abyss->cosmos) reversed.
|
||||
move = advance_ring(r, from_index=0, delta=-4, fast_spin_threshold=3)
|
||||
assert move.fast is True
|
||||
assert move.to_index == 2
|
||||
assert move.wrapped is True
|
||||
assert move.steps == (
|
||||
TransitionStep(
|
||||
edge=2, reversed=True, file="t/abyss-cosmos.mp4", to_index=2, blended=True
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def test_fast_spin_threshold_below_two_is_ignored():
|
||||
# A threshold of 0 or 1 would blend even single deliberate steps — treat as off.
|
||||
r = _ring()
|
||||
move = advance_ring(r, from_index=0, delta=3, fast_spin_threshold=1)
|
||||
assert move.fast is False
|
||||
assert len(move.steps) == 3
|
||||
|
||||
|
||||
def test_fast_spin_noop_on_degenerate_ring():
|
||||
r = ScaleRing(scales=(Scale("solo", "solo"),), transitions=())
|
||||
move = advance_ring(r, from_index=0, delta=9, fast_spin_threshold=3)
|
||||
assert move.fast is False
|
||||
assert move.steps == ()
|
||||
@@ -56,18 +56,16 @@ def test_overlay_change_is_a_live_update():
|
||||
p.update(_controls(content="video", left=0))
|
||||
t = p.update(_controls(content="video", left=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.overlay.level == 4
|
||||
assert t.playback.plan.overlay.intensity == 1.0
|
||||
|
||||
|
||||
def test_dream_change_is_a_live_update_not_a_crossfade():
|
||||
# Right-axis dream reframe (session 0013): the Right dream is a deterministic
|
||||
# LIVE filter now, so changing it no longer crossfades — only a clip swap does.
|
||||
def test_restyle_change_crossfades_the_substrate():
|
||||
# design §4.3: the Right v2v substrate is a pre-baked variant swap
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", right=0))
|
||||
t = p.update(_controls(content="video", right=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.dream.strength == 4
|
||||
assert t.playback.plan.dream.intensity == 1.0
|
||||
assert t.kind == TransitionKind.CROSSFADE
|
||||
assert t.playback.plan.restyle.blend == 1.0
|
||||
|
||||
|
||||
def test_volume_only_change_is_a_live_update():
|
||||
|
||||
+66
-155
@@ -1,191 +1,102 @@
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from hef.catalog import Record
|
||||
from simulator.app import create_app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def manifest_path(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps({
|
||||
"clips": [{
|
||||
"id": "forest",
|
||||
"title": "neutral forest",
|
||||
"base_file": "forest/base.mp4",
|
||||
"license": "poc", "source": "hef-poc",
|
||||
"right_variants": {"4": {"file": "forest/right4.mp4"}},
|
||||
"annotations": [{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1}],
|
||||
"strings": {"en": {"detected.water": "flowing water"}},
|
||||
}]
|
||||
}))
|
||||
return p
|
||||
def make_record(**overrides):
|
||||
base = dict(
|
||||
id="r",
|
||||
title="t",
|
||||
source_url="u",
|
||||
source_archive="internet_archive",
|
||||
license="public_domain",
|
||||
mode="video",
|
||||
left=0,
|
||||
right=0,
|
||||
dark=0,
|
||||
light=0,
|
||||
duration_s=600,
|
||||
file_path="",
|
||||
)
|
||||
base.update(overrides)
|
||||
return Record(**base)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(manifest_path):
|
||||
return TestClient(create_app(manifest_path=manifest_path))
|
||||
def client():
|
||||
records = [
|
||||
make_record(id="v-near", mode="video", left=0, right=0, dark=0, light=0),
|
||||
make_record(id="v-far", mode="video", left=4, right=4, dark=4, light=4),
|
||||
make_record(id="a-one", mode="audio", left=1, right=1, dark=1, light=1),
|
||||
make_record(id="prop", mode="video", left=0, right=0, dark=0, light=1,
|
||||
review_status="proposed"),
|
||||
make_record(id="appr", mode="video", left=0, right=0, dark=0, light=1,
|
||||
review_status="approved"),
|
||||
]
|
||||
return TestClient(create_app(records=records))
|
||||
|
||||
|
||||
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
|
||||
return dict(content=content, left=left, right=right, dark=dark,
|
||||
light=light, volume=volume, brightness=brightness)
|
||||
def _body(**overrides):
|
||||
base = dict(left=0, right=0, dark=0, light=0, mode="video")
|
||||
base.update(overrides)
|
||||
return base
|
||||
|
||||
|
||||
def test_alteration_returns_the_engine_plan(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(left=4, right=2, dark=4)})
|
||||
def test_select_returns_pick_and_ranked_pool(client):
|
||||
resp = client.post("/api/select", json=_body(mode="video", pool_size=4))
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["plan"]["overlay"]["level"] == 4
|
||||
assert data["plan"]["dream"]["strength"] == 2
|
||||
assert data["plan"]["dream"]["intensity"] == 0.5
|
||||
assert data["plan"]["grade"]["tone"] == -1.0
|
||||
assert data["content"]["video"] is True
|
||||
assert data["pick"]["id"] == "v-near"
|
||||
ids = [c["record"]["id"] for c in data["pool"]]
|
||||
assert ids[0] == "v-near"
|
||||
assert all("distance" in c and "rank" in c for c in data["pool"])
|
||||
assert [c["rank"] for c in data["pool"]] == list(range(1, len(data["pool"]) + 1))
|
||||
|
||||
|
||||
def test_alteration_honors_off_as_black(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(content="off")})
|
||||
def test_none_mode_is_the_void(client):
|
||||
resp = client.post("/api/select", json=_body(mode="none"))
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["content"]["video"] is False
|
||||
assert data["pick"] is None
|
||||
assert data["pool"] == []
|
||||
|
||||
|
||||
def test_alteration_accepts_calibration(client):
|
||||
body = {"controls": _controls(light=4),
|
||||
"calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "dream_gain": 1.0}}
|
||||
resp = client.post("/api/alteration", json=body)
|
||||
assert resp.json()["plan"]["grade"]["tone"] == 0.5
|
||||
|
||||
|
||||
def test_alteration_rejects_out_of_range_knob(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(left=7)})
|
||||
def test_dial_out_of_range_is_rejected(client):
|
||||
resp = client.post("/api/select", json=_body(left=7))
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
def test_alteration_rejects_bad_content(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(content="banana")})
|
||||
def test_bad_mode_is_rejected(client):
|
||||
resp = client.post("/api/select", json=_body(mode="banana"))
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
def test_clips_returns_the_manifest(client):
|
||||
resp = client.get("/api/clips")
|
||||
def test_approved_only_narrows_pool(client):
|
||||
resp = client.post("/api/select", json=_body(left=0, right=0, dark=0, light=1,
|
||||
mode="video", approved_only=True))
|
||||
data = resp.json()
|
||||
assert all(c["record"]["review_status"] == "approved" for c in data["pool"])
|
||||
|
||||
|
||||
def test_catalog_meta_reports_counts(client):
|
||||
resp = client.get("/api/catalog/meta")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["clips"][0]["id"] == "forest"
|
||||
assert data["clips"][0]["right_variants"]["0"]["file"] == "forest/base.mp4"
|
||||
assert data["clips"][0]["annotations"][0]["key"] == "detected.water"
|
||||
assert data["total"] == 5
|
||||
assert data["by_mode"]["video"] == 4
|
||||
assert data["by_mode"]["audio"] == 1
|
||||
assert set(data["by_status"]) == {"proposed", "approved"}
|
||||
|
||||
|
||||
def test_retired_selection_endpoints_are_gone(client):
|
||||
# The route no longer exists; the static catch-all yields 404 on GET and
|
||||
# 405 on the (now-unrouted) POST. Either proves the endpoint is gone.
|
||||
assert client.post("/api/select", json={}).status_code in (404, 405)
|
||||
assert client.get("/api/catalog/meta").status_code == 404
|
||||
from simulator.app import create_app as _create_app_for_static
|
||||
|
||||
|
||||
def test_index_is_served():
|
||||
client = TestClient(create_app())
|
||||
# The default app mounts the real static dir.
|
||||
client = TestClient(_create_app_for_static())
|
||||
resp = client.get("/")
|
||||
assert resp.status_code == 200
|
||||
assert "text/html" in resp.headers["content-type"]
|
||||
assert "Alteration" in resp.text
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ring_manifest_path(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps({
|
||||
"clips": [
|
||||
{"id": "cosmos", "title": "NASA cosmos", "base_file": "cosmos/base.mp4",
|
||||
"license": "PD", "source": "NASA", "right_variants": {},
|
||||
"annotations": [], "strings": {}},
|
||||
{"id": "forest", "title": "Yosemite", "base_file": "forest/base.mp4",
|
||||
"license": "poc", "source": "hef-poc",
|
||||
"right_variants": {"4": {"file": "forest/right4.mp4"}},
|
||||
"annotations": [], "strings": {}},
|
||||
{"id": "abyss", "title": "NOAA abyss", "base_file": "abyss/base.mp4",
|
||||
"license": "PD", "source": "NOAA", "right_variants": {},
|
||||
"annotations": [], "strings": {}},
|
||||
],
|
||||
"ring": {
|
||||
"scales": [
|
||||
{"id": "cosmos", "clip_id": "cosmos"},
|
||||
{"id": "forest", "clip_id": "forest"},
|
||||
{"id": "abyss", "clip_id": "abyss"},
|
||||
],
|
||||
"transitions": [
|
||||
{"file": "transitions/cosmos-forest.mp4", "model": "placeholder"},
|
||||
{"file": "transitions/forest-abyss.mp4", "model": "placeholder"},
|
||||
{"file": "transitions/abyss-cosmos.mp4", "model": "placeholder"},
|
||||
],
|
||||
},
|
||||
}))
|
||||
return p
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ring_client(ring_manifest_path):
|
||||
return TestClient(create_app(manifest_path=ring_manifest_path))
|
||||
|
||||
|
||||
def test_ring_returns_scales_and_transitions(ring_client):
|
||||
resp = ring_client.get("/api/ring")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert [s["id"] for s in data["scales"]] == ["cosmos", "forest", "abyss"]
|
||||
assert data["scales"][0]["title"] == "NASA cosmos"
|
||||
assert len(data["transitions"]) == 3
|
||||
|
||||
|
||||
def test_ring_advance_inward_one_step(ring_client):
|
||||
resp = ring_client.post("/api/ring/advance", json={"from_index": 0, "delta": 1})
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["to_index"] == 1
|
||||
assert data["target_clip_id"] == "forest"
|
||||
assert data["wrapped"] is False
|
||||
assert data["steps"][0]["file"] == "transitions/cosmos-forest.mp4"
|
||||
assert data["steps"][0]["reversed"] is False
|
||||
|
||||
|
||||
def test_ring_advance_wraps_smallest_to_largest(ring_client):
|
||||
resp = ring_client.post("/api/ring/advance", json={"from_index": 2, "delta": 1})
|
||||
data = resp.json()
|
||||
assert data["to_index"] == 0
|
||||
assert data["target_clip_id"] == "cosmos"
|
||||
assert data["wrapped"] is True
|
||||
|
||||
|
||||
def test_ring_advance_outward_reverses_edge(ring_client):
|
||||
resp = ring_client.post("/api/ring/advance", json={"from_index": 1, "delta": -1})
|
||||
data = resp.json()
|
||||
assert data["to_index"] == 0
|
||||
assert data["steps"][0]["reversed"] is True
|
||||
|
||||
|
||||
def test_ring_advance_small_delta_chains_not_fast(ring_client):
|
||||
# 2 detents is below the simulator's fast-spin threshold (3) -> full chain.
|
||||
resp = ring_client.post("/api/ring/advance", json={"from_index": 0, "delta": 2})
|
||||
data = resp.json()
|
||||
assert data["fast"] is False
|
||||
assert len(data["steps"]) == 2
|
||||
|
||||
|
||||
def test_ring_advance_fast_spin_collapses_to_one_blended_pass(ring_client):
|
||||
# A fast spin (4 detents batched) crosses the threshold -> one blended pass.
|
||||
resp = ring_client.post("/api/ring/advance", json={"from_index": 0, "delta": 4})
|
||||
data = resp.json()
|
||||
assert data["fast"] is True
|
||||
assert len(data["steps"]) == 1
|
||||
assert data["steps"][0]["blended"] is True
|
||||
assert data["to_index"] == 1
|
||||
|
||||
|
||||
def test_ring_404_when_no_ring_in_manifest(client):
|
||||
assert client.get("/api/ring").status_code == 404
|
||||
assert client.post("/api/ring/advance", json={"from_index": 0, "delta": 1}).status_code == 404
|
||||
|
||||
|
||||
def test_ring_advance_rejects_bad_delta(ring_client):
|
||||
resp = ring_client.post("/api/ring/advance", json={"from_index": 0, "delta": "x"})
|
||||
assert resp.status_code == 422
|
||||
assert "X-ray" in resp.text
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
"""Content pipeline: source -> frame -> loop -> transcode -> manifest.
|
||||
|
||||
Pure ffmpeg-argument builders (ffmpeg_ops), a thin runner (run), provenance
|
||||
records, and manifest assembly. See docs/superpowers/specs/2026-06-24-content-pipeline-design.md.
|
||||
"""
|
||||
@@ -1,74 +0,0 @@
|
||||
"""Pure ffmpeg argument builders (no I/O) for the content pipeline.
|
||||
|
||||
Each function returns a list[str] ready for ffmpeg, so command construction is
|
||||
unit-testable without running ffmpeg. tools/pipeline/run.py executes them. The
|
||||
ffmpeg binary name is injected (default "ffmpeg").
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
def _fit(width: int, height: int) -> str:
|
||||
"""A scale+pad filter chain fitting any input into width×height (letterbox),
|
||||
fixing SAR and pixel format so downstream concat/xfade get identical geometry."""
|
||||
return (
|
||||
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
|
||||
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
|
||||
"setsar=1,format=yuv420p"
|
||||
)
|
||||
|
||||
|
||||
def frame_args(src, dst, *, start: float, duration: float,
|
||||
width: int, height: int, ff: str = "ffmpeg") -> list[str]:
|
||||
"""Stage 2 — trim [start, start+duration], fit to width×height, strip audio.
|
||||
Produces a high-quality mezzanine for the loop stage."""
|
||||
return [
|
||||
ff, "-y", "-ss", str(start), "-t", str(duration),
|
||||
"-i", str(src), "-vf", _fit(width, height), "-an",
|
||||
"-c:v", "libx264", "-preset", "medium", "-crf", "16",
|
||||
"-pix_fmt", "yuv420p", "-movflags", "+faststart", str(dst),
|
||||
]
|
||||
|
||||
|
||||
def crossfade_loop_args(src, dst, *, duration: float, overlap: float,
|
||||
fps: int = 30, ff: str = "ffmpeg") -> list[str]:
|
||||
"""Stage 3 — make `src` loop seamlessly by crossfading its tail over its head.
|
||||
Output length = duration − overlap. Requires overlap < duration/2 so a non-empty
|
||||
middle remains.
|
||||
|
||||
fps is applied after each trim so xfade receives a constant-frame-rate stream
|
||||
(xfade requires CFR; a raw trim output reports rate 1/0 which xfade rejects)."""
|
||||
if overlap <= 0 or overlap >= duration / 2:
|
||||
raise ValueError(f"overlap {overlap} must be in (0, duration/2={duration/2})")
|
||||
d, o = duration, overlap
|
||||
fc = (
|
||||
f"[0:v]trim=0:{o},setpts=PTS-STARTPTS,fps={fps}[head];"
|
||||
f"[0:v]trim={o}:{d - o},setpts=PTS-STARTPTS,fps={fps}[mid];"
|
||||
f"[0:v]trim={d - o}:{d},setpts=PTS-STARTPTS,fps={fps}[tail];"
|
||||
f"[tail][head]xfade=transition=fade:duration={o}:offset=0[xf];"
|
||||
f"[xf][mid]concat=n=2:v=1,format=yuv420p[out]"
|
||||
)
|
||||
return [
|
||||
ff, "-y", "-i", str(src), "-filter_complex", fc,
|
||||
"-map", "[out]", "-an",
|
||||
"-c:v", "libx264", "-preset", "medium", "-crf", "16",
|
||||
"-pix_fmt", "yuv420p", "-movflags", "+faststart", str(dst),
|
||||
]
|
||||
|
||||
|
||||
def transcode_args(src, dst, *, width: int, height: int, crf: int,
|
||||
fps: int = 30, ff: str = "ffmpeg") -> list[str]:
|
||||
"""Stage 4 — encode a final deliverable (master or proxy). Master: source res
|
||||
capped ~3840 wide, crf 18. Proxy: 1920×1080, crf 20. Both H.264 High, yuv420p,
|
||||
even dims, +faststart, video-only (spec §6)."""
|
||||
vf = (
|
||||
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
|
||||
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
|
||||
f"setsar=1,fps={fps},format=yuv420p"
|
||||
)
|
||||
return [
|
||||
ff, "-y", "-i", str(src), "-vf", vf, "-an",
|
||||
"-c:v", "libx264", "-profile:v", "high", "-preset", "slow",
|
||||
"-crf", str(crf), "-pix_fmt", "yuv420p",
|
||||
"-movflags", "+faststart", str(dst),
|
||||
]
|
||||
@@ -1,55 +0,0 @@
|
||||
"""Stage 6 — assemble manifest entries (spec §3.6). Pure dict transforms over a
|
||||
loaded manifest dict; the caller does the json read/write. Idempotent by id so
|
||||
re-running earlier stages never clobbers authored labels."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
|
||||
|
||||
def upsert_clip(manifest: dict, clip: dict) -> dict:
|
||||
"""Replace the clip with the same id, or append it. Returns a new manifest."""
|
||||
m = copy.deepcopy(manifest)
|
||||
clips = m.setdefault("clips", [])
|
||||
for i, c in enumerate(clips):
|
||||
if c["id"] == clip["id"]:
|
||||
clips[i] = clip
|
||||
return m
|
||||
clips.append(clip)
|
||||
return m
|
||||
|
||||
|
||||
def add_ring_scale(manifest: dict, scale_id: str, clip_id: str,
|
||||
after: str | None = None) -> dict:
|
||||
"""Insert a scale into ring.scales after the named scale (or append if `after`
|
||||
is None / not found). No-op if scale_id already present. Returns a new manifest."""
|
||||
m = copy.deepcopy(manifest)
|
||||
ring = m.setdefault("ring", {"scales": [], "transitions": []})
|
||||
scales = ring.setdefault("scales", [])
|
||||
if any(s["id"] == scale_id for s in scales):
|
||||
return m
|
||||
entry = {"id": scale_id, "clip_id": clip_id}
|
||||
idx = next((i for i, s in enumerate(scales) if s["id"] == after), None)
|
||||
if idx is None:
|
||||
scales.append(entry)
|
||||
else:
|
||||
scales.insert(idx + 1, entry)
|
||||
return m
|
||||
|
||||
|
||||
def rebuild_ring_edges(manifest: dict) -> dict:
|
||||
"""Rebuild ring.transitions to one placeholder entry per ring edge (N scales ->
|
||||
N edges incl. the wrap), preserving any existing model string for an edge.
|
||||
Returns a new manifest."""
|
||||
m = copy.deepcopy(manifest)
|
||||
ring = m.setdefault("ring", {"scales": [], "transitions": []})
|
||||
scales = ring.get("scales", [])
|
||||
existing = {t["file"]: t.get("model", "") for t in ring.get("transitions", [])}
|
||||
edges = []
|
||||
n = len(scales)
|
||||
for i in range(n):
|
||||
a, b = scales[i]["id"], scales[(i + 1) % n]["id"]
|
||||
file = f"transitions/{a}-{b}.mp4"
|
||||
edges.append({"file": file, "model": existing.get(file, "placeholder-zoom")})
|
||||
ring["transitions"] = edges
|
||||
return m
|
||||
@@ -1,29 +0,0 @@
|
||||
"""Stage 1 — the per-clip strict-PD provenance record (spec §3.1).
|
||||
|
||||
Captured at source time so license + source + URL travel with the media. The
|
||||
'no explicit license -> assume PD, verify' trap (scales design §2.1) applies:
|
||||
"free stock" (Pexels/Pixabay) is NOT public domain.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict, dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Provenance:
|
||||
scale: str
|
||||
license: str
|
||||
source: str
|
||||
url: str
|
||||
fetched_at: str # ISO date; passed in (no clock here -> deterministic)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, d: dict) -> "Provenance":
|
||||
return cls(
|
||||
scale=d["scale"], license=d["license"], source=d["source"],
|
||||
url=d["url"], fetched_at=d["fetched_at"],
|
||||
)
|
||||
@@ -1,67 +0,0 @@
|
||||
"""Thin runner: probe the source duration, then execute the frame -> loop ->
|
||||
transcode stages with tools.pipeline.ffmpeg_ops. The pure arg builders are
|
||||
unit-tested; this glue is covered by the opt-in integration test."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from .ffmpeg_ops import crossfade_loop_args, frame_args, transcode_args
|
||||
|
||||
MASTER_MAX_W = 3840
|
||||
|
||||
|
||||
def resolve_ffmpeg() -> str:
|
||||
"""ffmpeg from PATH, else the bundled imageio-ffmpeg binary (matches
|
||||
simulator/setup_scales_media.py). The Pi/dev box may lack a system ffmpeg."""
|
||||
if shutil.which("ffmpeg"):
|
||||
return "ffmpeg"
|
||||
import imageio_ffmpeg
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
|
||||
|
||||
def probe_duration(src) -> float:
|
||||
"""Clip duration in seconds via cv2 (frame_count / fps) — avoids a system
|
||||
ffprobe dependency (imageio-ffmpeg ships ffmpeg only; the Pi has no ffprobe)."""
|
||||
import cv2
|
||||
cap = cv2.VideoCapture(str(src))
|
||||
fps = cap.get(cv2.CAP_PROP_FPS) or 0.0
|
||||
frames = cap.get(cv2.CAP_PROP_FRAME_COUNT) or 0.0
|
||||
cap.release()
|
||||
if fps <= 0:
|
||||
raise ValueError(f"could not read fps from {src}")
|
||||
return frames / fps
|
||||
|
||||
|
||||
def _run(args: list[str]) -> None:
|
||||
subprocess.run(args, check=True, capture_output=True)
|
||||
|
||||
|
||||
def process_clip(src, out_dir, *, start: float = 0.0, duration: float | None = None,
|
||||
overlap: float = 1.0, master_w: int = MASTER_MAX_W, master_h: int = 2160,
|
||||
ff: str | None = None) -> dict:
|
||||
"""Run stages 2–4 for one clip; return {'master','proxy','loop','mezzanine'} paths.
|
||||
`duration` defaults to the full source length (minus the trim start).
|
||||
Note: `crossfade_loop_args` uses a fixed fps only to satisfy xfade's
|
||||
constant-frame-rate requirement; the final output fps is set by `transcode_args`
|
||||
(both default 30 — keep in sync if overridden)."""
|
||||
ff = ff or resolve_ffmpeg()
|
||||
src = Path(src)
|
||||
out_dir = Path(out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
if duration is None:
|
||||
duration = probe_duration(src) - start
|
||||
|
||||
mezz = out_dir / "mezzanine.mp4"
|
||||
loop = out_dir / "loop.mp4"
|
||||
master = out_dir / "master.mp4"
|
||||
proxy = out_dir / "base.mp4" # the proxy is what the manifest's base_file points at
|
||||
|
||||
_run(frame_args(src, mezz, start=start, duration=duration,
|
||||
width=master_w, height=master_h, ff=ff))
|
||||
_run(crossfade_loop_args(mezz, loop, duration=duration, overlap=overlap, ff=ff))
|
||||
_run(transcode_args(loop, master, width=master_w, height=master_h, crf=18, ff=ff))
|
||||
_run(transcode_args(loop, proxy, width=1920, height=1080, crf=20, ff=ff))
|
||||
return {"mezzanine": mezz, "loop": loop, "master": master, "proxy": proxy}
|
||||
Reference in New Issue
Block a user