Compare commits
78 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 9c347d580f | |||
| 7d2a3064ad | |||
| e753a68147 | |||
| 554eb5076f | |||
| b8543906be | |||
| d2d63c0184 | |||
| c3f9262a73 | |||
| eb3aa0949d | |||
| c316309fc6 | |||
| 825d68c653 | |||
| 12a8177793 | |||
| e5ca07e2a4 | |||
| eda259e5b8 | |||
| 0a450bfd91 | |||
| c99a669194 | |||
| 42a72fe516 | |||
| 0238260908 | |||
| c653189397 | |||
| 92ed046602 | |||
| d58a23e5a6 | |||
| 5290785e2a | |||
| aeeed66f49 | |||
| 5403021e4a | |||
| 86b52ab8af | |||
| 9a13b01a41 | |||
| 79bb86c6a2 | |||
| 00969e2e44 | |||
| 57597be8af | |||
| 70834ae0ad | |||
| e69448922c | |||
| d8bb9661d4 | |||
| 77746e43c6 | |||
| c5952c6cfd | |||
| 850f67abea | |||
| 3510236a2a | |||
| 05c1f524da | |||
| 40dfbfd3db | |||
| 618b69182c | |||
| 859f868d3b | |||
| 80cde37b98 | |||
| c1745f0021 | |||
| cdf36c9b57 | |||
| aadfccf32d | |||
| a260b68d2b | |||
| 98fb86840c | |||
| 7f2f585600 | |||
| ddec45d39c | |||
| 9cb4fb62d8 | |||
| 0e5c3a1a0f | |||
| ee58c4cd78 | |||
| b7f19e01de | |||
| 593150b466 | |||
| 50ce6c9dfa | |||
| d7a2cbaa40 | |||
| 1e2a4dd966 | |||
| deb26b2575 | |||
| 376edd7819 | |||
| 3e07e86187 | |||
| 1d7f821ab8 | |||
| cecc5a0f61 | |||
| 2db2b7ac16 | |||
| 3d5c34491c | |||
| 222421e773 | |||
| d2a97823f5 | |||
| 542b5a5641 | |||
| 19104f94cd | |||
| b6fb635ef4 | |||
| db4f036729 | |||
| 6dd6afde19 | |||
| 1b8faf9142 | |||
| dd00d8bff5 | |||
| 6d6cbb5cad | |||
| dd02c2c36a | |||
| b8c406d517 | |||
| b5f5ee6663 | |||
| 3544092e5e | |||
| 4c1ec6e02e | |||
| f80ec65553 |
@@ -2,3 +2,8 @@ __pycache__/
|
||||
*.pyc
|
||||
.pytest_cache/
|
||||
.venv/
|
||||
media/
|
||||
.superpowers/
|
||||
*.egg-info/
|
||||
# Simulator sample media (look-tuning only; populate via setup_sample_media.py)
|
||||
simulator/sample_media/forest/*.mp4
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
.PHONY: sim sim-local
|
||||
|
||||
sim:
|
||||
docker compose -f simulator/docker-compose.yml up --build
|
||||
|
||||
sim-local:
|
||||
python -m uvicorn simulator.app:app --reload --port 8000
|
||||
+118
-57
@@ -14,13 +14,15 @@ Design reference: [`specs/2026-06-04-human-experience-filter-design.md`](./super
|
||||
| # | Sub-project | Status | Unblocks |
|
||||
|---|---------------------------------|---------------|----------|
|
||||
| 1 | Catalog & Selection Core | ✅ Done | everything |
|
||||
| 2 | Ingest & Tagging / Review tools | ⏳ Next | a real library |
|
||||
| 3 | Player Runtime (Pi) | ◻ Not started | the room runs |
|
||||
| 2 | Ingest & Tagging / Review tools | ✅ Done | a real library |
|
||||
| 3 | Player Runtime (Pi) | ⏳ In progress | the room runs |
|
||||
| 4 | Arduino Firmware (control panel)| ◻ Not started | real knobs |
|
||||
| 5 | Procedural Side Walls | ◻ Not started | full immersion |
|
||||
| 5 | ~~Procedural Side Walls~~ | ❌ Dropped | — (superseded) |
|
||||
|
||||
**Dependency order:** 1 → 2, and 1 → 3. Sub-projects 3 and 4 share a serial
|
||||
protocol contract and can then proceed in parallel. 5 depends on 3.
|
||||
protocol contract and can then proceed in parallel. Sub-project 5 was **dropped**
|
||||
(2026-06-04): the move to a single panoramic projector showing real video across
|
||||
the three walls removes the procedural side walls — see the design spec §6/§7.
|
||||
|
||||
```
|
||||
┌─────────────────────────┐
|
||||
@@ -29,15 +31,17 @@ protocol contract and can then proceed in parallel. 5 depends on 3.
|
||||
│
|
||||
┌───────────┴───────────┐
|
||||
▼ ▼
|
||||
┌───────────────┐ ┌─────────────────┐
|
||||
│ 2. Ingest / │ │ 3. Player │
|
||||
│ Review │─────▶│ Runtime (Pi) │
|
||||
└───────────────┘ feeds└───┬─────────┬────┘
|
||||
(fills catalog) │ │
|
||||
serial ▼ ▼ mood+color
|
||||
┌────────────────┐ ┌──────────────────┐
|
||||
│ 4. Firmware │ │ 5. Side Walls │
|
||||
└────────────────┘ └──────────────────┘
|
||||
┌───────────────┐ ┌─────────────────────┐
|
||||
│ 2. Ingest / │ │ 3. Player Runtime │
|
||||
│ Review │─────▶│ (Pi) → 1 pano │
|
||||
└───────────────┘ feeds│ projector │
|
||||
(fills catalog) └──────────┬──────────┘
|
||||
serial ▼
|
||||
┌────────────────┐
|
||||
│ 4. Firmware │
|
||||
└────────────────┘
|
||||
|
||||
(5. Procedural Side Walls — dropped 2026-06-04, single pano projector)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -55,11 +59,12 @@ Plan: [`2026-06-04-catalog-and-selection-core.md`](./superpowers/plans/2026-06-0
|
||||
|
||||
---
|
||||
|
||||
## 2. Ingest & Tagging / Review tools ⏳ (next)
|
||||
## 2. Ingest & Tagging / Review tools ✅
|
||||
|
||||
**Goal:** turn the manual catalog into the assisted *draft-then-review* flow from
|
||||
the spec, and actually populate the library from public-domain sources. This is
|
||||
where the content hunt lives.
|
||||
**Done — merged to `main` via PR #1 (session 0002, 2026-06-04); 111 tests pass
|
||||
(2 opt-in real-ffprobe tests skip when the binaries are absent).** The `tools/`
|
||||
package turns the manual catalog into the assisted *draft-then-review* flow from
|
||||
the spec, and can populate the library from public-domain sources.
|
||||
|
||||
**Delivers (`tools/`):**
|
||||
|
||||
@@ -75,40 +80,96 @@ where the content hunt lives.
|
||||
coordinates + rationale + a preview frame), accept or correct, flip to
|
||||
`approved` and stamp `reviewed_at`.
|
||||
|
||||
Spec: [`2026-06-04-ingest-tagging-review-tools.md`](./superpowers/specs/2026-06-04-ingest-tagging-review-tools.md).
|
||||
|
||||
**Depends on:** sub-project 1 (`hef.catalog`).
|
||||
**Done when:** you can run ingest against a source, get `proposed` records, review
|
||||
them to `approved`, and `load_catalog` validates the result — covered by tests on
|
||||
the mechanical-tagging and review state transitions.
|
||||
**External deps:** `ffmpeg`/`ffprobe`, an image lib for dominant color, per-archive
|
||||
download access.
|
||||
**Open decisions:** which archives ship first; download/caching layout for media
|
||||
files on the drive.
|
||||
**Done when:** ✅ you can run ingest against a source, get `proposed` records, review
|
||||
them to `approved`, and `load_catalog`/`validate_catalog` validate the result —
|
||||
covered by tests on the mechanical-tagging and review state transitions.
|
||||
**External deps:** `ffmpeg`/`ffprobe` only (no image lib — `dominant_color`, when
|
||||
requested, is ffmpeg-only and opt-in), per-archive download access.
|
||||
**Decisions settled:** first-ship archives = **LibriVox, NASA, Internet Archive**
|
||||
(Musopen/FMA/Freesound deferred stubs); media under a `--media-root` with
|
||||
`file_path` stored relative to it (spec §6.3); only additive `hef.catalog` change
|
||||
was `validate_catalog` + `index_by_id`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Player Runtime (Pi) ◻
|
||||
## 3. Player Runtime (Pi) ⏳ (in progress)
|
||||
|
||||
**Goal:** the thing that makes the room run — read the controls, pick media, play
|
||||
it on the primary wall.
|
||||
**Goal:** the thing that makes the room run — read the controls, **alter** the
|
||||
neutral base footage toward the knob state, and play it across the single
|
||||
panoramic projector. Per the machine-altered-perception revision the knobs no
|
||||
longer *select* a pre-tagged clip; they drive an **alteration engine** (design
|
||||
§4/§5) over a small neutral base library.
|
||||
|
||||
**Delivers (`player/`):**
|
||||
**Slice 1 — alteration engine + player core (pure logic) ✅ Done.** Merged to
|
||||
`main` (session 0006); 56 tests. The new `player/` package, with all I/O
|
||||
(video/audio/serial) behind injected interfaces:
|
||||
|
||||
- Read the five control values from the Arduino over USB serial.
|
||||
- Call `hef.selection.select()` on each change; loop the chosen 5–15 min segment
|
||||
until the knobs move; **crossfade** on change.
|
||||
- `None` mode → fade to black + silence.
|
||||
- Drive the **primary** projector; emit current `dominant_color` + dark/light to
|
||||
the side-wall renderer (sub-project 5).
|
||||
- `player/controls.py` — `Controls`, the full panel state (7-way content dial +
|
||||
4 experience knobs + volume + brightness); the serial-contract data shape
|
||||
shared with sub-project 4.
|
||||
- `player/content.py` — `resolve_content`, the §6 7-way dial → (audio source,
|
||||
video on/off) table.
|
||||
- `player/alteration.py` — `plan_alteration`, knob vector → `RenderPlan`
|
||||
(mood `ColorGrade` with center=identity §5; Left `AnalyticalOverlay`; Right
|
||||
`Restyle`), encoding the §4.2 layering rule (substrate vs overlay; Left & Right
|
||||
stack).
|
||||
- `player/state.py` — `Player`, the state machine that diffs successive controls
|
||||
into transitions (`FADE_TO_BLACK`/`FADE_FROM_BLACK` on video on/off,
|
||||
`CROSSFADE` on clip/restyle-variant swap, `LIVE_UPDATE` for the continuous
|
||||
grade/overlay/level ops per §4.3, `NONE` when unchanged).
|
||||
|
||||
**Depends on:** sub-project 1 (`select`); a populated catalog from sub-project 2
|
||||
to be meaningful, but can be developed against a hand-authored catalog and a
|
||||
**keyboard/serial stand-in** before firmware exists.
|
||||
**Done when:** given a catalog and a stream of control values, it plays the
|
||||
correct segment, loops, crossfades, and goes dark on `None` — testable with the
|
||||
serial input mocked.
|
||||
**Hardware:** Raspberry Pi 5 (holds the drive + catalog).
|
||||
Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-player-alteration-core.md).
|
||||
|
||||
**Slice 2 — simulator-first alteration preview ✅ Done.** Merged to `main`
|
||||
(session 0009). Wires the alteration engine into the web simulator so the look is
|
||||
tunable by eye before any hardware, and **reconciles** the unmerged session-0007
|
||||
design with the merged session-0008 design (the Left-HUD conflict): Left is a
|
||||
**runtime overlay** (authored annotation track + per-language string tables),
|
||||
Right is a **discrete pre-baked** flow-stabilized variant, Dark/Light a **live**
|
||||
grade — superseding 0007's baked-HUD 5×5 grid. Engine: a parameterized
|
||||
`Calibration` (tuned by eye, baked into `DEFAULT_CALIBRATION`), `Restyle.variant`
|
||||
(discrete) replacing the continuous blend, and `AnalyticalOverlay.level`. Sim:
|
||||
`/api/alteration` + `/api/clips` over `simulator/clips.py`, an alteration-preview
|
||||
UI, and one neutral clip with a real flow-stabilized Right variant from the POC.
|
||||
Design:
|
||||
[`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md);
|
||||
plan:
|
||||
[`2026-06-07-reconciled-simulator-alteration-slice.md`](./superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md).
|
||||
|
||||
**Remaining slices (not started):**
|
||||
|
||||
- **Runtime renderer** — drive the single panoramic projector via mpv/ffmpeg;
|
||||
GPU shaders for the luma-keyed mood grade + analytical-overlay compositing;
|
||||
realize crossfade/fade timing and the 5–15 min loop. (Open decision: player
|
||||
stack — mpv via IPC vs. ffmpeg vs. custom.)
|
||||
- **Serial input** — real USB-serial reader on the Pi + the **3⇄4 framing
|
||||
contract** with the firmware (the keyboard/serial stand-in already works via a
|
||||
`Controls` stream).
|
||||
- **White-noise generation** — runtime white/pink noise for that content position.
|
||||
- **Offline v2v variant pipeline** — author the pre-baked Right restyle variants
|
||||
via the local flow-stabilized SD pipeline (now ~free, not a paid API — see the
|
||||
scales-library/right-axis design §1/§4); a real multi-strength flow-stabilized
|
||||
re-bake per base clip + the multilingual label/string tables + TTS.
|
||||
- **Scale-ring navigation** — the endless rotary encoder + short pre-baked AI
|
||||
zoom/warp transitions between neutral "scales of nature" clips on a closed ring
|
||||
(scales-library design §3); a new control + offline pipeline element.
|
||||
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
|
||||
variant" flag (sub-project 2 territory, design §13).
|
||||
|
||||
**Depends on:** sub-project 1 (`Coordinate`); a neutral base library from
|
||||
sub-project 2 to be meaningful, but developable against a hand-authored library
|
||||
and a **keyboard/serial stand-in** before firmware exists.
|
||||
**Done when:** given a base library and a stream of control values, it plays the
|
||||
correct altered segment, loops, crossfades, and goes dark on `Off` — the
|
||||
decision logic is covered (slice 1, serial mocked); the renderer realizes it.
|
||||
**Hardware:** Raspberry Pi 5 (holds the drive + base library + variants).
|
||||
**Open decisions:** player stack (mpv via IPC vs. ffmpeg vs. custom); whether the
|
||||
player hard-restricts to `approved` records.
|
||||
player hard-restricts to `approved` records (the `approved_only` flag is plumbed
|
||||
but inert until backed by a real catalog); knob→strength calibration (the §3 vs
|
||||
§4.2/§5 reconciliation — see the slice-1 plan / session 0006 transcript).
|
||||
|
||||
---
|
||||
|
||||
@@ -132,20 +193,17 @@ a serial monitor) and the Pi player reacts.
|
||||
|
||||
---
|
||||
|
||||
## 5. Procedural Side Walls ◻
|
||||
## 5. ~~Procedural Side Walls~~ ❌ Dropped (2026-06-04)
|
||||
|
||||
**Goal:** make the mood axis physically felt in peripheral vision.
|
||||
**Superseded.** The move to a single panoramic projector showing the real selected
|
||||
video across the three walls (design spec §6/§7) removes the procedural ambient
|
||||
renderer entirely — the mood axis is felt through the chosen content itself. With
|
||||
it goes the only consumer of `dominant_color`, which becomes optional/opt-in in
|
||||
the ingest tooling (sub-project 2 spec §5.2).
|
||||
|
||||
**Delivers (`sidewalls/`):**
|
||||
|
||||
- Render a slow color wash on the three non-primary walls: **hue** from the
|
||||
playing piece's `dominant_color`, **brightness** from the dark/light knobs.
|
||||
- Fade to black in `None` mode.
|
||||
|
||||
**Depends on:** sub-project 3 emitting current `dominant_color` + mood state.
|
||||
**Done when:** given a color + mood input stream, the wash renders and tracks
|
||||
changes — testable headless by snapshotting rendered frames.
|
||||
**Open decisions:** runs on the primary Pi's extra outputs vs. a second Pi.
|
||||
> *Original goal (for the record): render a slow color wash on the three
|
||||
> non-primary walls — hue from the playing piece's `dominant_color`, brightness
|
||||
> from the dark/light knobs — to make the mood axis felt in peripheral vision.*
|
||||
|
||||
---
|
||||
|
||||
@@ -154,7 +212,10 @@ changes — testable headless by snapshotting rendered frames.
|
||||
Carried from the design spec's open-questions list — none block sub-project 2:
|
||||
|
||||
- **Serial protocol framing** between Arduino and Pi (needed before 3 ⇄ 4).
|
||||
- **Player stack** on the Pi (mpv / ffmpeg / custom).
|
||||
- **Side walls** on the primary Pi vs. a second Pi.
|
||||
- **Player stack** on the Pi (mpv / ffmpeg / custom) and how the single panoramic
|
||||
output is driven.
|
||||
- **`approved`-only enforcement** in the player.
|
||||
- **Media storage layout** on the drive (how `file_path` values are rooted).
|
||||
- ~~**Side walls** on the primary Pi vs. a second Pi~~ — resolved: side walls dropped.
|
||||
- ~~**Media storage layout** on the drive~~ — settled by the sub-project-2 spec
|
||||
§6.3: media under a `media-root`, `file_path` stored relative to it; the player
|
||||
joins it with its drive mount.
|
||||
|
||||
+144
-6
@@ -1,10 +1,11 @@
|
||||
# Human Experience Filter — User Guide
|
||||
|
||||
> **Scope (current build state).** Only the catalog + selection core is built so
|
||||
> far. This guide covers the one thing you can do today: **configure media by
|
||||
> hand-authoring and validating catalog records.** The automated ingest tool,
|
||||
> the tagging/review CLI, and the room player are separate, not-yet-built
|
||||
> sub-projects and are not covered here.
|
||||
> **Scope (current build state).** Two pieces are built: the catalog + selection
|
||||
> core, and the **`tools/` ingest & review pipeline** (sub-project 2). You can
|
||||
> populate the catalog two ways — **hand-author records** (below) or **assisted
|
||||
> ingest** that fetches from public-domain archives, mechanically tags, drafts
|
||||
> coordinates, and lets you review them to `approved` (see *Ingesting & reviewing
|
||||
> media*). The room **player** is a separate, not-yet-built sub-project.
|
||||
|
||||
---
|
||||
|
||||
@@ -66,7 +67,7 @@ defaults and may be omitted.
|
||||
| `review_status` | `"proposed"` | `proposed` or `approved` — whether a human has blessed it. |
|
||||
| `attribution` | `""` | **Required text when `license` is `cc_by` or `cc_by_nc`.** |
|
||||
| `resolution` | `""` | e.g. `1920x1080` (video). |
|
||||
| `dominant_color` | `""` | Hex color, used by the side walls later. |
|
||||
| `dominant_color` | `""` | Hex color. **Optional / opt-in** — computed only with `--dominant-color`. |
|
||||
| `rationale` | `""` | One-line note on why you chose the coordinate. |
|
||||
| `reviewed_at` | `null` | Timestamp when approved. |
|
||||
| `notes` | `""` | Free text. |
|
||||
@@ -146,6 +147,94 @@ print("added", r.id)
|
||||
`append_record` validates the record and raises `CatalogError` (printing what is
|
||||
wrong) before it writes, so a bad record never lands in the file.
|
||||
|
||||
### Option C: assisted ingest (fetch + tag + draft, then review)
|
||||
|
||||
See the next section.
|
||||
|
||||
---
|
||||
|
||||
## Ingesting & reviewing media
|
||||
|
||||
Instead of hand-authoring every record, the `tools/` pipeline can fetch a piece
|
||||
from a public-domain archive, fill in the mechanical fields for you, **draft** a
|
||||
coordinate, and write the record as `review_status: proposed`. You then walk the
|
||||
proposed records and bless each one to `approved`. The four coordinates are still
|
||||
a human act — the tool only *drafts* a starting point; nothing is selectable until
|
||||
you approve it.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **Python 3.11+**, run from the repo root (as everywhere in this guide).
|
||||
- **`ffmpeg` and `ffprobe`** on your `PATH` — used to read media properties
|
||||
(`mode`/`duration_s`/`resolution`), render review previews, and (opt-in)
|
||||
compute `dominant_color`. Install via your package manager (e.g.
|
||||
`brew install ffmpeg`). The unit tests don't need them; the live ingest/review
|
||||
do.
|
||||
|
||||
### Where downloaded media goes (`--media-root`)
|
||||
|
||||
Ingest downloads each file to `<media-root>/<archive>/<id>.<ext>` and records a
|
||||
`file_path` **relative to the media root** (e.g. `nasa/nasa-earthrise.mp4`).
|
||||
Relative paths are portable: the tagging workstation and the Pi's drive store the
|
||||
same tree under different mounts, and the player joins `file_path` with its own
|
||||
mount. The media root is `--media-root DIR` (or `HEF_MEDIA_ROOT`), default
|
||||
`./media/`, which is gitignored — **media never enters the repo**, only metadata.
|
||||
|
||||
> **Note on `file_path` style.** Hand-authored records (Option A/B above) often use
|
||||
> absolute paths like `/media/earthrise.mp4`; ingested records use archive-relative
|
||||
> paths. Both load and validate fine (`validate()` does not constrain the format).
|
||||
> Keep a catalog internally consistent where you can; the player spec will define
|
||||
> how mounts are resolved.
|
||||
|
||||
### First-ship archives
|
||||
|
||||
Three keyless, clean-license archives are wired today:
|
||||
|
||||
| `archive` arg | Pool | License |
|
||||
|--------------------|---------------------------------------|-------------------|
|
||||
| `librivox` | Public-domain audiobook recordings | `public_domain` |
|
||||
| `nasa` | NASA imagery / video | `public_domain` |
|
||||
| `internet_archive` | Internet Archive / Prelinger | per item (PD / CC)|
|
||||
|
||||
`musopen`, `fma`, and `freesound` are defined but **deferred** (auth or
|
||||
API-stability tax) — invoking them exits with a "deferred" message. **Freesound**
|
||||
will need an API token supplied via the `FREESOUND_API_TOKEN` environment variable
|
||||
(a secret — never put it on the command line or into a record).
|
||||
|
||||
### Running ingest
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m tools.ingest_cli nasa --query "earthrise" --limit 5
|
||||
.venv/bin/python -m tools.ingest_cli internet_archive --resolve prelinger_blast
|
||||
.venv/bin/python -m tools.ingest_cli librivox --query "meditations" --media-root ./media
|
||||
```
|
||||
|
||||
Flags: `--query` (search) or `--resolve <identifier>` (one item); `--limit N`;
|
||||
`--catalog` (default `catalog/library.jsonl`); `--media-root` (default `./media`,
|
||||
or `HEF_MEDIA_ROOT`); `--dominant-color` to compute `dominant_color` for
|
||||
video/`av` records (off by default — its only consumer, the procedural side walls,
|
||||
was dropped in the single-panoramic-projector design change). Re-running is
|
||||
idempotent: a candidate whose id already exists is skipped.
|
||||
|
||||
### Reviewing proposed records
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m tools.review_cli --catalog catalog/library.jsonl --media-root ./media
|
||||
```
|
||||
|
||||
For each `proposed` record it prints the id/title/source/license, the mechanical
|
||||
fields, the **drafted coordinates + rationale**, and opens a preview frame
|
||||
(video/`av`) or waveform image (audio). Then it prompts:
|
||||
|
||||
- **`a`** — accept the drafted coordinates and mark `approved`.
|
||||
- **`e`** — edit `left/right/dark/light` (enter blank to keep a value), then approve.
|
||||
- **`s`** — skip; leave it `proposed`.
|
||||
- **`q`** — save and quit.
|
||||
|
||||
Each approval stamps `reviewed_at` and is written immediately (full rewrite), so
|
||||
an interrupted session keeps its progress. **How far from done** is just the count
|
||||
of records still `proposed`. Add `--no-preview` to skip frame rendering.
|
||||
|
||||
---
|
||||
|
||||
## Validating the whole catalog
|
||||
@@ -197,3 +286,52 @@ Records point at files via `file_path`. Those files live on the player's drive,
|
||||
**not** in this git repo (the repo holds only the catalog metadata). Keep your
|
||||
`file_path` values consistent with wherever you mount that drive on the machine
|
||||
that will eventually run the room.
|
||||
|
||||
## Playing with the simulator (alteration preview)
|
||||
|
||||
The simulator is a web stand-in for the installation's control panel. It runs the
|
||||
real `player.alteration` engine and **alters** a neutral base clip toward the knob
|
||||
state in the browser, so you can tune the *look* of the filter before any hardware
|
||||
exists. (The earlier selection-era "curator's X-ray" view was retired when the
|
||||
piece moved from *selecting* clips to *altering* them.)
|
||||
|
||||
**One-time setup — populate the sample footage** (look-tuning only; not shipped
|
||||
content):
|
||||
|
||||
python simulator/setup_sample_media.py
|
||||
|
||||
This copies the session-0008 POC artifacts (`~/hef-poc/out/`) into
|
||||
`simulator/sample_media/forest/` — the neutral base clip and the real
|
||||
flow-stabilized Right restyle — and generates placeholder intermediate Right
|
||||
strengths. The `.mp4` binaries are gitignored.
|
||||
|
||||
**Run it (Docker):**
|
||||
|
||||
make sim
|
||||
|
||||
then open http://localhost:8000.
|
||||
|
||||
**Run it (no Docker):**
|
||||
|
||||
pip install -e ".[sim]"
|
||||
make sim-local
|
||||
|
||||
**What you see and can do:**
|
||||
|
||||
- **Content dial** — picks audio/video channel; "off" and audio-only positions go
|
||||
to black walls.
|
||||
- **Four experience knobs (0–4):**
|
||||
- **Dark / Light** — a live runtime color grade (cool/dark ↔ warm/bright; equal
|
||||
or zero = the raw footage).
|
||||
- **Right (dreamlike)** — selects a discrete pre-baked, flow-stabilized restyle
|
||||
variant and crossfades to it (strength 0 = raw base).
|
||||
- **Left (analytical)** — a live overlay: labelled boxes from the clip's authored
|
||||
annotation track, with more annotations appearing at higher levels. Text is
|
||||
shaped live (the simulator analogue of the Pi's Pango/HarfBuzz path).
|
||||
- **Calibration sliders** — adjust the grade/overlay gain curves live; once a look
|
||||
is liked, bake the values into `DEFAULT_CALIBRATION` in `player/alteration.py`.
|
||||
- **RenderPlan readout** — always shows the exact numbers the engine produced (the
|
||||
project's honesty "X-ray," now over the alteration model).
|
||||
|
||||
The base clips, Right variants, Left annotation track, and string tables come from
|
||||
`simulator/sample_media/manifest.json`.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,500 @@
|
||||
# Ingest & Tagging / Review Tools — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build the `tools/` package that turns the hand-authored catalog into the assisted *draft-then-review* pipeline: per-archive ingest fetchers, mechanical tagging (ffprobe/ffmpeg + origin license), heuristic coordinate drafting (`review_status=proposed` + one-line rationale), and an interactive review CLI that flips records to `approved` with a `reviewed_at` stamp.
|
||||
|
||||
**Spec:** [`2026-06-04-ingest-tagging-review-tools.md`](../specs/2026-06-04-ingest-tagging-review-tools.md). Read it first — this plan implements it task-by-task and inherits its decisions (esp. §3 the single required `hef.catalog` addition, §5.2 `dominant_color` is optional/opt-in, §6.4 first-ship = LibriVox + NASA + Internet Archive).
|
||||
|
||||
**Architecture:** A `tools/` package alongside the existing `hef/` (which it imports, never forks). The two external boundaries — `ffmpeg`/`ffprobe` (subprocess) and archive HTTP — are isolated behind injectable seams (a `Prober`, a `download` callable, an HTTP client) so the unit suite stays **hermetic** (no network, no binaries). Integration tests that actually invoke `ffprobe`/`ffmpeg` are opt-in and skipped when the binaries are absent.
|
||||
|
||||
**Tech Stack:** Python 3.11+, stdlib only in the testable core (`dataclasses`, `json`, `subprocess`, `urllib`, `pathlib`, `datetime`); `ffmpeg`/`ffprobe` system binaries; pytest. No `requests`, no image library (dominant color, when computed, is ffmpeg-only).
|
||||
|
||||
**Where this sits:** sub-project **2 of 5**. Depends on sub-project 1 (`hef.catalog`, `hef.selection`, built). Feeds sub-project 3 (the Pi player). Sub-project 5 (side walls) was dropped (design §7 revision), so `dominant_color` is optional here.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
pyproject.toml + add `tools` to packages; optional [project.scripts]
|
||||
.gitignore + media/
|
||||
hef/catalog.py + validate_catalog(), index_by_id() (ONLY hef change)
|
||||
tools/__init__.py package marker
|
||||
tools/http.py stdlib-urllib client (timeout/retry/user-agent), injectable
|
||||
tools/probe.py Prober: ffprobe wrapper -> parsed Probe result
|
||||
tools/tagging.py mode/duration_s/resolution from a Probe (+ cover-art guard)
|
||||
tools/mediatools.py ffmpeg: representative frame; optional dominant_color
|
||||
tools/licensing.py per-archive license/attribution -> hef LICENSES vocab
|
||||
tools/drafting.py Signals, Draft, Proposer, HeuristicProposer
|
||||
tools/ingest/__init__.py
|
||||
tools/ingest/base.py Candidate, Fetcher, ingest_candidate() pipeline
|
||||
tools/ingest/librivox.py first-ship fetcher
|
||||
tools/ingest/nasa.py first-ship fetcher
|
||||
tools/ingest/internet_archive.py first-ship fetcher
|
||||
tools/ingest/musopen.py deferred: docstring stub raising NotImplementedError
|
||||
tools/ingest/fma.py deferred: docstring stub
|
||||
tools/ingest/freesound.py deferred: docstring stub (needs FREESOUND_API_TOKEN)
|
||||
tools/ingest_cli.py `python -m tools.ingest_cli`
|
||||
tools/review.py proposed_records(), approve() (pure transition core)
|
||||
tools/review_cli.py `python -m tools.review_cli` (interactive walk)
|
||||
tests/test_catalog_integrity.py validate_catalog / index_by_id
|
||||
tests/test_probe.py ffprobe JSON parsing (canned)
|
||||
tests/test_tagging.py mode/duration/resolution + cover-art guard
|
||||
tests/test_mediatools.py dominant_color from canned rgb bytes; opt-in default off
|
||||
tests/test_licensing.py per-archive license normalization
|
||||
tests/test_drafting.py HeuristicProposer
|
||||
tests/test_fetchers.py librivox/nasa/internet_archive via fake HTTP client
|
||||
tests/test_ingest_pipeline.py ingest_candidate end-to-end with fakes
|
||||
tests/test_review.py proposed_records / approve transition
|
||||
tests/test_tools_integration.py end-to-end mocked; opt-in real-ffprobe (skipped if absent)
|
||||
```
|
||||
|
||||
`tools/` holds everything new; `hef/` gains only the two additive symbols of spec §3.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Scaffolding — `tools/` package, deps, http client, smoke test
|
||||
|
||||
**Files:** `pyproject.toml`, `.gitignore`, `tools/__init__.py`, `tools/http.py`, `tests/test_tools_smoke.py`
|
||||
|
||||
- [ ] **Step 1: Add `tools` to packaging.** In `pyproject.toml`, change `packages = ["hef"]` → `packages = ["hef", "tools"]`. (Optional: add `[project.scripts]` `hef-ingest = "tools.ingest_cli:main"` and `hef-review = "tools.review_cli:main"` for installed checkouts; `python -m` is the documented path either way.)
|
||||
|
||||
- [ ] **Step 2: gitignore media.** Append `media/` to `.gitignore`.
|
||||
|
||||
- [ ] **Step 3: Write the failing smoke test** `tests/test_tools_smoke.py`:
|
||||
```python
|
||||
def test_tools_package_imports():
|
||||
import tools
|
||||
import tools.http
|
||||
assert tools is not None
|
||||
```
|
||||
Run `python -m pytest tests/test_tools_smoke.py -v` → FAIL (`ModuleNotFoundError: tools`).
|
||||
|
||||
- [ ] **Step 4: Create `tools/__init__.py`** (empty) and `tools/http.py` — a tiny stdlib client:
|
||||
```python
|
||||
"""Minimal HTTP client over urllib: timeout, small retry, user-agent. Injectable."""
|
||||
from __future__ import annotations
|
||||
import json as _json
|
||||
import time
|
||||
import urllib.request
|
||||
|
||||
USER_AGENT = "human-experience-filter-ingest/0.1 (+local art installation)"
|
||||
|
||||
class HttpClient:
|
||||
def __init__(self, *, timeout=30.0, retries=2, opener=None):
|
||||
self.timeout, self.retries = timeout, retries
|
||||
self._opener = opener or urllib.request.urlopen
|
||||
|
||||
def get_bytes(self, url, *, headers=None):
|
||||
last = None
|
||||
for attempt in range(self.retries + 1):
|
||||
try:
|
||||
req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT, **(headers or {})})
|
||||
with self._opener(req, timeout=self.timeout) as resp:
|
||||
return resp.read()
|
||||
except Exception as exc: # noqa: BLE001 - retried below
|
||||
last = exc
|
||||
if attempt < self.retries:
|
||||
time.sleep(0.2 * (attempt + 1))
|
||||
raise last
|
||||
|
||||
def get_json(self, url, *, headers=None):
|
||||
return _json.loads(self.get_bytes(url, headers=headers).decode("utf-8"))
|
||||
```
|
||||
The `opener` injection point lets tests pass a fake (no network).
|
||||
|
||||
- [ ] **Step 5:** Run smoke test → PASS. Commit:
|
||||
```
|
||||
git add pyproject.toml .gitignore tools/__init__.py tools/http.py tests/test_tools_smoke.py
|
||||
git commit -m "chore: scaffold tools/ package + stdlib http client"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: `hef.catalog` additions — `validate_catalog` + `index_by_id`
|
||||
|
||||
The ONLY change to the shared spine (spec §3). Purely additive.
|
||||
|
||||
**Files:** `hef/catalog.py`, `tests/test_catalog_integrity.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** `tests/test_catalog_integrity.py` (reuse a local `make_record` like `tests/test_catalog.py`):
|
||||
```python
|
||||
import pytest
|
||||
from hef.catalog import Record, validate_catalog, index_by_id, CatalogError
|
||||
|
||||
def make_record(**o):
|
||||
base = dict(id="a", title="t", source_url="u", source_archive="nasa",
|
||||
license="public_domain", mode="video", left=0, right=0, dark=0,
|
||||
light=0, duration_s=1, file_path="p")
|
||||
base.update(o); return Record(**base)
|
||||
|
||||
def test_validate_catalog_accepts_unique_ids():
|
||||
validate_catalog([make_record(id="a"), make_record(id="b")]) # no raise
|
||||
|
||||
def test_validate_catalog_rejects_duplicate_id():
|
||||
with pytest.raises(CatalogError) as e:
|
||||
validate_catalog([make_record(id="dup"), make_record(id="dup")])
|
||||
assert "dup" in str(e.value)
|
||||
|
||||
def test_validate_catalog_validates_each_record():
|
||||
with pytest.raises(CatalogError):
|
||||
validate_catalog([make_record(left=9)])
|
||||
|
||||
def test_index_by_id_round_trips():
|
||||
recs = [make_record(id="a"), make_record(id="b")]
|
||||
idx = index_by_id(recs)
|
||||
assert idx["a"].id == "a" and idx["b"].id == "b"
|
||||
|
||||
def test_index_by_id_rejects_duplicates():
|
||||
with pytest.raises(CatalogError):
|
||||
index_by_id([make_record(id="x"), make_record(id="x")])
|
||||
```
|
||||
Run → FAIL (ImportError).
|
||||
|
||||
- [ ] **Step 2: Implement** — append to `hef/catalog.py` (do NOT touch existing functions):
|
||||
```python
|
||||
def index_by_id(records) -> dict:
|
||||
"""Map id -> Record, raising CatalogError on a duplicate id."""
|
||||
idx: dict = {}
|
||||
for r in records:
|
||||
if r.id in idx:
|
||||
raise CatalogError(f"duplicate record id: {r.id!r}")
|
||||
idx[r.id] = r
|
||||
return idx
|
||||
|
||||
|
||||
def validate_catalog(records) -> None:
|
||||
"""Validate every record AND cross-record invariants (currently: unique ids)."""
|
||||
for r in records:
|
||||
validate(r)
|
||||
index_by_id(records) # raises on duplicate id
|
||||
```
|
||||
|
||||
- [ ] **Step 3:** Run `python -m pytest tests/test_catalog_integrity.py -v` → PASS. Run full suite `python -m pytest -q` → still green (37 prior + new). Commit:
|
||||
```
|
||||
git add hef/catalog.py tests/test_catalog_integrity.py
|
||||
git commit -m "feat: catalog-level validate_catalog + index_by_id (unique ids)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: `tools/probe.py` — ffprobe wrapper
|
||||
|
||||
**Files:** `tools/probe.py`, `tests/test_probe.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** — feed canned `ffprobe` JSON via an injected runner; assert a parsed `Probe` (list of streams + format dict). Cover: a video+audio file, an audio-only file, and an audio file with an `attached_pic` cover stream. Example:
|
||||
```python
|
||||
import json
|
||||
from tools.probe import Probe, probe_file
|
||||
|
||||
def fake_runner(args): # mimics subprocess.run(...).stdout
|
||||
return json.dumps({
|
||||
"streams": [
|
||||
{"codec_type": "video", "width": 1920, "height": 1080, "disposition": {"attached_pic": 0}},
|
||||
{"codec_type": "audio"},
|
||||
],
|
||||
"format": {"duration": "12.5"},
|
||||
})
|
||||
|
||||
def test_probe_parses_streams_and_format():
|
||||
p = probe_file("x.mp4", runner=fake_runner)
|
||||
assert any(s["codec_type"] == "video" for s in p.streams)
|
||||
assert p.format["duration"] == "12.5"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement** `tools/probe.py`:
|
||||
```python
|
||||
"""ffprobe wrapper -> parsed streams/format. Subprocess runner is injectable."""
|
||||
from __future__ import annotations
|
||||
import json, subprocess
|
||||
from dataclasses import dataclass
|
||||
|
||||
@dataclass
|
||||
class Probe:
|
||||
streams: list
|
||||
format: dict
|
||||
|
||||
def _default_runner(args) -> str:
|
||||
return subprocess.run(args, capture_output=True, text=True, check=True).stdout
|
||||
|
||||
def probe_file(path, *, runner=_default_runner) -> Probe:
|
||||
out = runner(["ffprobe", "-v", "quiet", "-print_format", "json",
|
||||
"-show_format", "-show_streams", str(path)])
|
||||
data = json.loads(out)
|
||||
return Probe(streams=data.get("streams", []), format=data.get("format", {}))
|
||||
```
|
||||
|
||||
- [ ] **Step 3:** Run → PASS. Commit `feat: ffprobe wrapper (tools.probe)`.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: `tools/tagging.py` — mode / duration_s / resolution (with cover-art guard)
|
||||
|
||||
**Files:** `tools/tagging.py`, `tests/test_tagging.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** covering the spec §5.1 cases:
|
||||
- video+audio → `mode="av"`, `resolution="1920x1080"`, `duration_s=13` (round 12.5).
|
||||
- audio only → `mode="audio"`, `resolution=""`.
|
||||
- **audio + `attached_pic` cover image → `mode="audio"`** (the guard), `resolution=""`.
|
||||
- video only → `mode="video"`.
|
||||
- duration fallback to longest stream when `format.duration` absent.
|
||||
```python
|
||||
from tools.probe import Probe
|
||||
from tools.tagging import derive_tags
|
||||
|
||||
def test_cover_art_stays_audio():
|
||||
p = Probe(streams=[
|
||||
{"codec_type": "audio", "duration": "30.0"},
|
||||
{"codec_type": "video", "width": 600, "height": 600, "disposition": {"attached_pic": 1}},
|
||||
], format={"duration": "30.0"})
|
||||
t = derive_tags(p)
|
||||
assert t.mode == "audio" and t.resolution == "" and t.duration_s == 30
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement** — a `Tags(mode, duration_s, resolution)` dataclass and `derive_tags(probe)`:
|
||||
- real video streams = `codec_type == "video"` AND `disposition.attached_pic != 1`.
|
||||
- mode: both→`av`, video-only→`video`, else→`audio`.
|
||||
- resolution from the first real video stream (`f"{w}x{h}"`), else `""`.
|
||||
- duration: `round(float(format["duration"]))` else max stream duration else 0; never negative.
|
||||
|
||||
- [ ] **Step 3:** Run → PASS. Commit `feat: mechanical mode/duration/resolution tagging`.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: `tools/mediatools.py` — representative frame + optional dominant_color
|
||||
|
||||
**Files:** `tools/mediatools.py`, `tests/test_mediatools.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** (hermetic — inject the ffmpeg runner returning canned bytes):
|
||||
- `dominant_color_from_rgb(b"\xff\x00\x00") == "#ff0000"`.
|
||||
- `compute_dominant_color(path, runner=fake)` returns the hex when enabled.
|
||||
- The default ingest path does NOT call this (asserted in Task 8, not here).
|
||||
|
||||
- [ ] **Step 2: Implement**:
|
||||
```python
|
||||
"""ffmpeg helpers: representative frame; OPTIONAL dominant color. Runner injectable."""
|
||||
from __future__ import annotations
|
||||
import subprocess
|
||||
|
||||
def _run_bytes(args) -> bytes:
|
||||
return subprocess.run(args, capture_output=True, check=True).stdout
|
||||
|
||||
def dominant_color_from_rgb(rgb: bytes) -> str:
|
||||
r, g, b = rgb[0], rgb[1], rgb[2]
|
||||
return f"#{r:02x}{g:02x}{b:02x}"
|
||||
|
||||
def compute_dominant_color(path, *, midpoint_s=0.0, runner=_run_bytes) -> str:
|
||||
"""ffmpeg-only single-color palette of a mid-segment frame -> #rrggbb."""
|
||||
args = ["ffmpeg", "-v", "quiet", "-ss", str(midpoint_s), "-i", str(path),
|
||||
"-vf", "thumbnail,palettegen=max_colors=1", "-frames:v", "1",
|
||||
"-f", "rawvideo", "-pix_fmt", "rgb24", "-"]
|
||||
return dominant_color_from_rgb(runner(args))
|
||||
|
||||
def extract_frame(path, dest, *, midpoint_s=0.0, runner=None):
|
||||
"""Write one representative frame to dest (PNG) for the review preview."""
|
||||
runner = runner or (lambda a: subprocess.run(a, check=True))
|
||||
runner(["ffmpeg", "-v", "quiet", "-y", "-ss", str(midpoint_s), "-i", str(path),
|
||||
"-frames:v", "1", str(dest)])
|
||||
return dest
|
||||
```
|
||||
|
||||
- [ ] **Step 3:** Run → PASS. Commit `feat: ffmpeg frame extraction + optional dominant_color`.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: `tools/licensing.py` — origin → license/attribution
|
||||
|
||||
**Files:** `tools/licensing.py`, `tests/test_licensing.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** mapping sample origin metadata per archive to the `hef.catalog.LICENSES` vocab:
|
||||
- CC-BY url/identifier → `("cc_by", "<attribution string>")` (non-empty attribution).
|
||||
- CC0 / public-domain markers → `("cc0"|"public_domain", "")`.
|
||||
- LibriVox → always `("public_domain", "")`.
|
||||
- unmappable → raises a clear error (rejected at ingest).
|
||||
```python
|
||||
from tools.licensing import normalize_license
|
||||
|
||||
def test_cc_by_requires_attribution():
|
||||
lic, attr = normalize_license("https://creativecommons.org/licenses/by/4.0/",
|
||||
creator="Jane Doe")
|
||||
assert lic == "cc_by" and "Jane Doe" in attr
|
||||
|
||||
def test_unmappable_rejected():
|
||||
import pytest
|
||||
with pytest.raises(ValueError):
|
||||
normalize_license("All Rights Reserved")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement** `normalize_license(raw, *, creator="", license_name="")`:
|
||||
- regex/string-match CC URLs and identifiers → `cc0`/`cc_by`/`cc_by_nc`.
|
||||
- public-domain / "no known copyright" / "publicdomain" → `public_domain`.
|
||||
- for `ATTRIBUTION_LICENSES` build `attribution` from `creator` + license name/URL (must be non-empty; `validate()` enforces it downstream).
|
||||
- anything else → `raise ValueError(f"unmappable license: {raw!r}")`.
|
||||
- Per-archive helpers may wrap it (e.g. `librivox_license()` returns `("public_domain","")`).
|
||||
|
||||
- [ ] **Step 3:** Run → PASS. Commit `feat: per-archive license normalization`.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: `tools/drafting.py` — heuristic coordinate proposer
|
||||
|
||||
**Files:** `tools/drafting.py`, `tests/test_drafting.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** for `HeuristicProposer` (spec §7.1):
|
||||
- `librivox` archive → high `left`; `nasa`/`musopen`/`fma` → high `right`.
|
||||
- title/description containing a dark keyword ("storm") raises `dark`; a light keyword ("sunrise") raises `light`.
|
||||
- all coords clamped to 0..4.
|
||||
- `rationale` is a non-empty single line citing a signal.
|
||||
```python
|
||||
from tools.drafting import Signals, HeuristicProposer
|
||||
|
||||
def test_librivox_seeds_left():
|
||||
d = HeuristicProposer().propose(Signals(title="Meditations", description="",
|
||||
source_archive="librivox", mode="audio", duration_s=600))
|
||||
assert d.coordinate.left >= 3 and d.rationale
|
||||
|
||||
def test_storm_seeds_dark():
|
||||
d = HeuristicProposer().propose(Signals(title="Thunderstorm at Night",
|
||||
description="", source_archive="nasa", mode="video", duration_s=600))
|
||||
assert d.coordinate.dark >= 2
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement** `Signals`, `Draft` (wrapping `hef.selection.Coordinate`), `Proposer` protocol, and `HeuristicProposer` with: archive priors (brain plane), keyword sets for dark/light (mood plane), `max(0, min(4, v))` clamping, one-line rationale naming the dominant signal. Deterministic — no I/O.
|
||||
|
||||
- [ ] **Step 3:** Run → PASS. Commit `feat: heuristic coordinate proposer (drafting)`.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: `tools/ingest/base.py` — Candidate, Fetcher, pipeline
|
||||
|
||||
**Files:** `tools/ingest/__init__.py`, `tools/ingest/base.py`, `tests/test_ingest_pipeline.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** for `ingest_candidate` with ALL boundaries faked (no network, no ffprobe, no disk writes beyond tmp_path):
|
||||
- given a `Candidate` + fake `prober` (returns a Probe) + fake `downloader` (writes a tmp file) + `HeuristicProposer`, it appends ONE `proposed` record whose mechanical fields match the probe, `review_status=="proposed"`, `reviewed_at is None`, `rationale` non-empty, `file_path` relative to media-root.
|
||||
- re-running with the same `suggested_id` is idempotent (no duplicate; skipped).
|
||||
- `dominant_color` stays `""` by default; computed only when `compute_color=True`.
|
||||
- a candidate whose license is unmappable raises before writing (nothing appended).
|
||||
|
||||
- [ ] **Step 2: Implement** `Candidate` (spec §6.1), the `Fetcher` Protocol (`archive`, `search`, `resolve`), and:
|
||||
```python
|
||||
def ingest_candidate(candidate, *, catalog_path, media_root, proposer,
|
||||
prober=probe_file, downloader=..., compute_color=False):
|
||||
# 1. load_catalog + validate_catalog; skip if candidate.suggested_id exists
|
||||
# 2. download media_url -> <media_root>/<archive>/<id>.<ext> (skip if present)
|
||||
# 3. tags = derive_tags(prober(file)); color = compute_dominant_color(...) if compute_color and video
|
||||
# 4. draft = proposer.propose(Signals(...))
|
||||
# 5. build hef.catalog.Record(...) (proposed shape via defaults)
|
||||
# 6. validate(record); re-check uniqueness; append_record(record, catalog_path)
|
||||
```
|
||||
Plus `ingest_search(fetcher, query, *, limit, **kw)` looping the above over `fetcher.search`.
|
||||
|
||||
- [ ] **Step 3:** Run → PASS. Commit `feat: ingest pipeline (Candidate/Fetcher/ingest_candidate)`.
|
||||
|
||||
---
|
||||
|
||||
### Task 9: First-ship fetchers — LibriVox, NASA, Internet Archive
|
||||
|
||||
**Files:** `tools/ingest/librivox.py`, `tools/ingest/nasa.py`, `tools/ingest/internet_archive.py`, `tools/ingest/{musopen,fma,freesound}.py` (deferred stubs), `tests/test_fetchers.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** — one per first-ship fetcher, feeding canned API JSON through `HttpClient(opener=fake)`; assert `Candidate` fields + a stable `suggested_id` derivation and the normalized license:
|
||||
- **LibriVox** (`librivox.org/api/feed/audiobooks?...&format=json`): `license="public_domain"`, `mode` hint audio, id like `librivox-<slug>`.
|
||||
- **NASA** (`images-api.nasa.gov/search?q=...`): `license="public_domain"`, id like `nasa-<nasa_id>`, media_url from the asset collection.
|
||||
- **Internet Archive** (`archive.org/metadata/<id>`): license from `licenseurl`/`rights` via `tools.licensing`; ambiguous "no known copyright" → `public_domain` with a `notes` flag; id like `ia-<identifier>`.
|
||||
|
||||
- [ ] **Step 2: Implement** the three fetchers against the documented JSON APIs, each taking an injected `HttpClient`, each exposing `archive`, `search(query, limit)`, `resolve(identifier)`. Use `tools.licensing` for license/attribution.
|
||||
|
||||
- [ ] **Step 3: Deferred stubs.** `musopen.py`, `fma.py`, `freesound.py` define the class with the `Fetcher` shape but `raise NotImplementedError("deferred — see spec §6.4")`; `freesound.py` documents the `FREESOUND_API_TOKEN` env requirement (secret; never logged). A test asserts they raise `NotImplementedError` (so the seam is wired and the deferral is explicit, not forgotten).
|
||||
|
||||
- [ ] **Step 4:** Run → PASS. Commit `feat: LibriVox/NASA/Internet Archive fetchers (+ deferred stubs)`.
|
||||
|
||||
---
|
||||
|
||||
### Task 10: `tools/ingest_cli.py`
|
||||
|
||||
**Files:** `tools/ingest_cli.py` (light/manual test)
|
||||
|
||||
- [ ] **Step 1:** `argparse` entry: `python -m tools.ingest_cli <archive> --query "..." [--limit N] [--catalog catalog/library.jsonl] [--media-root ./media] [--dominant-color] [--resolve <identifier>]`. Wires the named fetcher + `HeuristicProposer` into `ingest_search`/`ingest_candidate`. `main()` returns an exit code; secrets read from env only.
|
||||
|
||||
- [ ] **Step 2:** A small test that `--help` parses and that an unknown archive errors cleanly (no network). Real fetches are manual/integration.
|
||||
|
||||
- [ ] **Step 3:** Commit `feat: ingest CLI entry point`.
|
||||
|
||||
---
|
||||
|
||||
### Task 11: `tools/review.py` — the transition core
|
||||
|
||||
**Files:** `tools/review.py`, `tests/test_review.py`
|
||||
|
||||
- [ ] **Step 1: Failing tests** (spec §8.1):
|
||||
```python
|
||||
from dataclasses import replace
|
||||
from tools.review import proposed_records, approve
|
||||
from hef.selection import Coordinate
|
||||
# make_record helper as elsewhere
|
||||
|
||||
def test_proposed_records_filters():
|
||||
a = make_record(id="a", review_status="proposed")
|
||||
b = make_record(id="b", review_status="approved")
|
||||
assert [r.id for r in proposed_records([a, b])] == ["a"]
|
||||
|
||||
def test_approve_sets_status_and_timestamp():
|
||||
r = make_record(review_status="proposed")
|
||||
out = approve(r, reviewed_at="2026-06-04T13:00:00+00:00")
|
||||
assert out.review_status == "approved" and out.reviewed_at == "2026-06-04T13:00:00+00:00"
|
||||
|
||||
def test_approve_can_override_coordinates():
|
||||
r = make_record(left=0, right=0, dark=0, light=0, review_status="proposed")
|
||||
out = approve(r, reviewed_at="t", coordinate=Coordinate(4, 1, 2, 3))
|
||||
assert (out.left, out.right, out.dark, out.light) == (4, 1, 2, 3)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement** `proposed_records(records)` and `approve(record, *, reviewed_at, coordinate=None)` using `dataclasses.replace` to return an approved copy (no in-place surprise); caller re-validates. Pure, no I/O, no clock (timestamp injected).
|
||||
|
||||
- [ ] **Step 3:** Run → PASS. Commit `feat: review transition core (proposed -> approved)`.
|
||||
|
||||
---
|
||||
|
||||
### Task 12: `tools/review_cli.py` — interactive walk
|
||||
|
||||
**Files:** `tools/review_cli.py` (manual; preview reuses `tools.mediatools`)
|
||||
|
||||
- [ ] **Step 1:** Implement the walk (spec §8.2): `load_catalog` → `validate_catalog`; for each `proposed` record print id/title/source/license/mode/duration/resolution + proposed coords + rationale; render a preview (video/av → `extract_frame` then `open`/`xdg-open`; audio → `showwavespic` thumbnail and/or optional `ffplay`); prompt `[a]ccept/[e]dit/[s]kip/[q]uit`; on accept/edit call `approve(...)` with `datetime.now(timezone.utc).isoformat()` and persist via `save_catalog` (rewrite) after each approval. `--catalog`, `--media-root` args.
|
||||
|
||||
- [ ] **Step 2:** Keep the shell thin — all decision logic already tested in Task 11. A tiny test that `--help` parses; the interactive loop is manual.
|
||||
|
||||
- [ ] **Step 3:** Commit `feat: interactive review CLI`.
|
||||
|
||||
---
|
||||
|
||||
### Task 13: End-to-end mocked integration + opt-in real-ffprobe
|
||||
|
||||
**Files:** `tests/test_tools_integration.py`
|
||||
|
||||
- [ ] **Step 1: Hermetic end-to-end** (spec §10 test 8): fake fetcher → `Candidate`; fake downloader + fake prober + `HeuristicProposer` → `ingest_candidate` appends a `proposed` record; `proposed_records` + `approve` flips it; `load_catalog` + `validate_catalog` pass; `select(loaded, coord, mode, approved_only=True)` returns it.
|
||||
|
||||
- [ ] **Step 2: Opt-in real-ffprobe test** — guarded by `shutil.which("ffprobe")` (`pytest.mark.skipif` when absent): generate a 1-second test clip with `ffmpeg lavfi` into `tmp_path`, probe it, assert `mode`/`duration_s`/`resolution`. Same pattern for a real `compute_dominant_color`.
|
||||
|
||||
- [ ] **Step 3:** Run the FULL suite `python -m pytest -q` → green. Commit `test: end-to-end ingest+review integration (+ opt-in ffprobe)`.
|
||||
|
||||
---
|
||||
|
||||
### Task 14: Docs — extend the User Guide for the tools
|
||||
|
||||
**Files:** `docs/USER_GUIDE.md`
|
||||
|
||||
- [ ] **Step 1:** Add an "Ingesting & reviewing media" section: prerequisites (`ffmpeg`/`ffprobe`), `--media-root`, the first-ship archives, `python -m tools.ingest_cli ...` examples, the review walk, the Freesound token note (env only), and that `dominant_color` is optional/opt-in. Update the scope banner (no longer "only catalog core is built"). Reconcile the absolute-vs-relative `file_path` note (spec §6.3).
|
||||
|
||||
- [ ] **Step 2:** Commit `docs: user guide for ingest + review tools`.
|
||||
|
||||
---
|
||||
|
||||
## Done criteria
|
||||
|
||||
- `python -m pytest -q` is green from the repo root inside `.venv` (prior 37 + all new unit tests; opt-in ffprobe tests skip cleanly when binaries are absent).
|
||||
- `hef.catalog` gains exactly `validate_catalog` and `index_by_id`; every pre-existing symbol/behavior is unchanged.
|
||||
- `python -m tools.ingest_cli <archive> --query ...` against LibriVox / NASA / Internet Archive produces `proposed` records with mechanical fields filled and a one-line rationale.
|
||||
- `python -m tools.review_cli` walks the `proposed` records (coords + rationale + preview frame) and flips accepted ones to `approved` with a `reviewed_at` stamp; `load_catalog` + `validate_catalog` validate the result.
|
||||
- `dominant_color` is computed only with `--dominant-color`; Freesound/Musopen/FMA fetchers are explicit deferred stubs (raise `NotImplementedError`).
|
||||
|
||||
This populated, human-reviewed catalog is what sub-project 3 (the Pi player) drives to the single panoramic projector.
|
||||
@@ -0,0 +1,867 @@
|
||||
# Player Alteration Core Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build the pure-logic core of sub-project 3 (the Player Runtime + alteration engine) — a new `player/` package that turns control-panel state into a render plan and playback transitions, fully unit-tested with all I/O (video/audio/serial) behind injected interfaces.
|
||||
|
||||
**Architecture:** Four small, pure modules mirroring `hef/` conventions (frozen dataclasses, `from __future__ import annotations`, string-constant frozensets, thorough unit tests, no heavy deps). `controls` models the panel state (the serial-contract data shape shared with sub-project 4); `content` resolves the 7-way content dial; `alteration` is the heart — knob vector → `RenderPlan` per design §4/§5; `state` is the player state machine that diffs successive controls into playback transitions (crossfade / fade-to-black / live-update). No mpv, no GPU, no real serial, no white-noise DSP, no v2v pipeline — those are later integration slices.
|
||||
|
||||
**Tech Stack:** Python 3.11+, stdlib only (`dataclasses`, `math`), pytest. Reuses `hef.selection.Coordinate` for the knob vector.
|
||||
|
||||
**Design reference:** `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` (§4 alteration engine, §5 mood grade, §6 7-way content dial, §7 intensity) and `docs/ROADMAP.md` §3 (Player Runtime "done when").
|
||||
|
||||
**Calibration decision (flagged for operator):** brain knobs use `strength = value/4` (0=off, 4=max); mood uses `tone = (light−dark)/4` (equal dark/light = identity per §5). This reads §3's "(2,2,2,2) neutral" as a vestige of the old coordinate-grid center. The calibration lives in three one-line helpers so the convention can be flipped trivially. See the session 0006 transcript's Deferred decisions.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- `player/__init__.py` — package marker (empty).
|
||||
- `player/controls.py` — `Controls` frozen dataclass (content position + 4 experience knobs + volume + brightness), `CONTENT_POSITIONS`, `validate_controls`, `parse_controls` (from a decoded mapping; wire framing is the separate 3⇄4 serial contract).
|
||||
- `player/content.py` — `ContentResolution` (audio source + video flag), `AUDIO_SOURCES`, `resolve_content` (the §6 7-row table, single source of truth).
|
||||
- `player/alteration.py` — `ColorGrade`, `AnalyticalOverlay`, `Restyle`, `RenderPlan`, `plan_alteration(coord)`, and the calibration helpers `_overlay_intensity`, `_restyle_blend`, `_mood_tone`.
|
||||
- `player/state.py` — `Playback`, `TransitionKind`, `Transition`, `Player` (consumes a stream of `Controls`, emits a `Transition` per update; output sink is injected/duck-typed; base-clip choice is an injected callable).
|
||||
- `pyproject.toml` — add `"player"` to `[tool.setuptools] packages`.
|
||||
|
||||
Tests (one per module): `tests/test_player_controls.py`, `tests/test_player_content.py`, `tests/test_player_alteration.py`, `tests/test_player_state.py`.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Controls model (the serial-contract data shape)
|
||||
|
||||
**Files:**
|
||||
- Create: `player/__init__.py`
|
||||
- Create: `player/controls.py`
|
||||
- Test: `tests/test_player_controls.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/test_player_controls.py
|
||||
import pytest
|
||||
|
||||
from player.controls import (
|
||||
Controls,
|
||||
CONTENT_POSITIONS,
|
||||
ControlsError,
|
||||
validate_controls,
|
||||
parse_controls,
|
||||
)
|
||||
|
||||
|
||||
def test_content_positions_are_the_seven_from_the_spec():
|
||||
assert CONTENT_POSITIONS == frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
|
||||
|
||||
def test_valid_controls_pass_validation():
|
||||
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
|
||||
validate_controls(c) # does not raise
|
||||
|
||||
|
||||
def test_invalid_content_position_rejected():
|
||||
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(c)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
|
||||
@pytest.mark.parametrize("bad", [-1, 5, True])
|
||||
def test_out_of_range_or_non_int_knob_rejected(field, bad):
|
||||
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
kwargs[field] = bad
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(Controls(**kwargs))
|
||||
|
||||
|
||||
def test_parse_controls_from_mapping():
|
||||
c = parse_controls(
|
||||
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
|
||||
)
|
||||
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
|
||||
|
||||
|
||||
def test_parse_controls_rejects_unknown_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
|
||||
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
|
||||
|
||||
|
||||
def test_parse_controls_rejects_missing_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0})
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pytest tests/test_player_controls.py -v`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'player'`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# player/__init__.py
|
||||
```
|
||||
(empty file)
|
||||
|
||||
```python
|
||||
# player/controls.py
|
||||
"""Control-panel state: the data shape read from the Arduino over serial.
|
||||
|
||||
This is the serial-contract payload shared with sub-project 4 (firmware). It
|
||||
models the full panel from design §6/§7: the 7-way content dial, the four
|
||||
experience knobs (0..4), and the two intensity levels (volume, brightness).
|
||||
The wire framing itself is the separate 3<->4 serial contract; this module is
|
||||
the *decoded* form.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, fields
|
||||
|
||||
# The seven positions of the content dial (design §6).
|
||||
CONTENT_POSITIONS = frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
|
||||
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
|
||||
KNOB_MIN = 0
|
||||
KNOB_MAX = 4
|
||||
|
||||
|
||||
class ControlsError(ValueError):
|
||||
"""Raised when a Controls payload is structurally invalid."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Controls:
|
||||
content: str
|
||||
left: int
|
||||
right: int
|
||||
dark: int
|
||||
light: int
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
|
||||
def validate_controls(c: Controls) -> None:
|
||||
"""Raise ControlsError if the payload is structurally invalid."""
|
||||
if c.content not in CONTENT_POSITIONS:
|
||||
raise ControlsError(
|
||||
f"invalid content position {c.content!r}; "
|
||||
f"expected one of {sorted(CONTENT_POSITIONS)}"
|
||||
)
|
||||
for name in KNOB_FIELDS:
|
||||
value = getattr(c, name)
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise ControlsError(f"knob {name} must be an int, got {value!r}")
|
||||
if not (KNOB_MIN <= value <= KNOB_MAX):
|
||||
raise ControlsError(
|
||||
f"knob {name}={value} out of range {KNOB_MIN}..{KNOB_MAX}"
|
||||
)
|
||||
|
||||
|
||||
def parse_controls(data: dict) -> Controls:
|
||||
"""Build a validated Controls from a decoded mapping, rejecting unknown or
|
||||
missing keys."""
|
||||
known = {f.name for f in fields(Controls)}
|
||||
unknown = set(data) - known
|
||||
if unknown:
|
||||
raise ControlsError(f"unknown keys: {sorted(unknown)}")
|
||||
missing = known - set(data)
|
||||
if missing:
|
||||
raise ControlsError(f"missing keys: {sorted(missing)}")
|
||||
c = Controls(**data)
|
||||
validate_controls(c)
|
||||
return c
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pytest tests/test_player_controls.py -v`
|
||||
Expected: PASS (all cases).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add player/__init__.py player/controls.py tests/test_player_controls.py
|
||||
git commit -m "feat(player): Controls panel model (serial-contract data shape)
|
||||
|
||||
Sub-project 3 slice 1, per ROADMAP.md §3 and design §6/§7."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Content-dial resolution (§6)
|
||||
|
||||
**Files:**
|
||||
- Create: `player/content.py`
|
||||
- Test: `tests/test_player_content.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/test_player_content.py
|
||||
import pytest
|
||||
|
||||
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
|
||||
|
||||
|
||||
def test_audio_sources_are_the_four_distinct_sources():
|
||||
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
# The §6 table, row by row: position -> (audio_source, video).
|
||||
@pytest.mark.parametrize(
|
||||
"position,audio_source,video",
|
||||
[
|
||||
("off", "none", False),
|
||||
("white_noise", "white_noise", False),
|
||||
("music", "music", False),
|
||||
("audio_track", "audio_track", False),
|
||||
("video", "none", True),
|
||||
("music_video", "music", True),
|
||||
("audio_video", "audio_track", True),
|
||||
],
|
||||
)
|
||||
def test_resolve_content_matches_spec_table(position, audio_source, video):
|
||||
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
|
||||
|
||||
|
||||
def test_off_is_void_state_black_and_silent():
|
||||
r = resolve_content("off")
|
||||
assert r.video is False and r.audio_source == "none"
|
||||
|
||||
|
||||
def test_resolve_content_rejects_unknown_position():
|
||||
with pytest.raises(ValueError):
|
||||
resolve_content("bogus")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pytest tests/test_player_content.py -v`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'player.content'`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# player/content.py
|
||||
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
|
||||
|
||||
The single source of truth for design §6's table: which audio source plays and
|
||||
whether the projector shows video, for each dial position.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
# The four distinct audio sources. "white_noise" is generated at runtime;
|
||||
# "music" is the public-domain classical pool; "audio_track" is the clip's own
|
||||
# field audio; "none" is silence.
|
||||
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContentResolution:
|
||||
audio_source: str
|
||||
video: bool
|
||||
|
||||
|
||||
# Design §6, one row per dial position.
|
||||
_TABLE = {
|
||||
"off": ContentResolution("none", False),
|
||||
"white_noise": ContentResolution("white_noise", False),
|
||||
"music": ContentResolution("music", False),
|
||||
"audio_track": ContentResolution("audio_track", False),
|
||||
"video": ContentResolution("none", True),
|
||||
"music_video": ContentResolution("music", True),
|
||||
"audio_video": ContentResolution("audio_track", True),
|
||||
}
|
||||
|
||||
|
||||
def resolve_content(position: str) -> ContentResolution:
|
||||
"""Map a content-dial position to its audio source and video flag (§6)."""
|
||||
try:
|
||||
return _TABLE[position]
|
||||
except KeyError:
|
||||
raise ValueError(
|
||||
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
|
||||
) from None
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pytest tests/test_player_content.py -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add player/content.py tests/test_player_content.py
|
||||
git commit -m "feat(player): 7-way content-dial resolution (design §6)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Alteration engine — knob vector to RenderPlan (§4, §5)
|
||||
|
||||
**Files:**
|
||||
- Create: `player/alteration.py`
|
||||
- Test: `tests/test_player_alteration.py`
|
||||
|
||||
The engine maps a knob vector `(left, right, dark, light)` to a layered `RenderPlan` per design §4.2:
|
||||
- **Substrate** transforms (alter pixels): `Restyle` (Right, v2v) blended with `ColorGrade` (mood, Dark/Light).
|
||||
- **Overlay** on top: `AnalyticalOverlay` (Left, HUD/labels).
|
||||
|
||||
`ColorGrade.tone` is signed: `>0` warm yellow→white (light pole), `<0` cool blue→black (dark pole), `0` identity/raw (§5). Calibration (flagged decision): `overlay = left/4`, `restyle = right/4`, `tone = (light − dark)/4`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/test_player_alteration.py
|
||||
import pytest
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import (
|
||||
AnalyticalOverlay,
|
||||
ColorGrade,
|
||||
RenderPlan,
|
||||
Restyle,
|
||||
plan_alteration,
|
||||
)
|
||||
|
||||
|
||||
def _coord(left=0, right=0, dark=0, light=0):
|
||||
return Coordinate(left=left, right=right, dark=dark, light=light)
|
||||
|
||||
|
||||
def test_all_zero_knobs_is_the_unaltered_base():
|
||||
plan = plan_alteration(_coord())
|
||||
assert plan.is_identity
|
||||
assert plan.overlay.intensity == 0.0
|
||||
assert plan.restyle.blend == 0.0
|
||||
assert plan.grade.tone == 0.0
|
||||
assert plan.grade.is_identity
|
||||
|
||||
|
||||
def test_left_drives_the_analytical_overlay_only():
|
||||
plan = plan_alteration(_coord(left=4))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 0.0 # Left does not touch the substrate
|
||||
assert plan.grade.tone == 0.0
|
||||
|
||||
|
||||
def test_right_drives_the_restyle_substrate_only():
|
||||
plan = plan_alteration(_coord(right=2))
|
||||
assert plan.restyle.blend == 0.5
|
||||
assert plan.overlay.intensity == 0.0 # Right does not add overlay
|
||||
|
||||
|
||||
def test_left_and_right_stack_not_cancel():
|
||||
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
|
||||
plan = plan_alteration(_coord(left=4, right=4))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 1.0
|
||||
|
||||
|
||||
def test_light_pole_grades_warm_positive_tone():
|
||||
plan = plan_alteration(_coord(light=4))
|
||||
assert plan.grade.tone == 1.0
|
||||
assert not plan.grade.is_identity
|
||||
|
||||
|
||||
def test_dark_pole_grades_cool_negative_tone():
|
||||
plan = plan_alteration(_coord(dark=4))
|
||||
assert plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_equal_dark_and_light_is_identity_grade():
|
||||
# design §5: the mood center is the raw, ungraded footage
|
||||
assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity
|
||||
assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity
|
||||
|
||||
|
||||
def test_dark_minus_light_sets_intermediate_tone():
|
||||
assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5)
|
||||
assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5)
|
||||
|
||||
|
||||
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
|
||||
# design §4.2 "Dark + analytical": cold measurement over a melancholy scene
|
||||
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 0.5
|
||||
assert plan.grade.tone == -1.0
|
||||
assert not plan.is_identity
|
||||
|
||||
|
||||
def test_render_plan_is_frozen():
|
||||
plan = plan_alteration(_coord())
|
||||
with pytest.raises(Exception):
|
||||
plan.grade.tone = 0.5 # type: ignore[misc]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pytest tests/test_player_alteration.py -v`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'player.alteration'`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# player/alteration.py
|
||||
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
|
||||
|
||||
Replaces the 2026-06-04 nearest-match *selection* with a *transformation* of a
|
||||
neutral base clip. Given the four experience knobs, it produces three layers
|
||||
that compose per §4.2:
|
||||
|
||||
- Substrate transforms (alter pixels, blend with each other):
|
||||
* Restyle -- the Right axis: a pre-baked generative v2v dreamlike restyle.
|
||||
* ColorGrade -- the mood axis (Dark/Light): a deterministic color grade.
|
||||
- Overlay (composited on top of the substrate):
|
||||
* AnalyticalOverlay -- the Left axis: HUD/labels/measurement.
|
||||
|
||||
Left and Right are NOT opposites; they live on different layers and stack
|
||||
(§4.2). Dark and Light are the two poles of one mood grade whose center is the
|
||||
identity (§5).
|
||||
|
||||
Calibration note: the knob->strength curves below are the single source of
|
||||
truth for how a 0..4 position maps to a transform strength. See the session
|
||||
0006 transcript Deferred decisions for the §3-vs-§4.2/§5 reconciliation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from hef.selection import Coordinate
|
||||
|
||||
KNOB_MAX = 4 # knob full-scale (0..4)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ColorGrade:
|
||||
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
|
||||
<0 cool blue->black (dark), 0 = identity (raw, ungraded)."""
|
||||
|
||||
tone: float
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
return self.tone == 0.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AnalyticalOverlay:
|
||||
"""Left axis (§4.1): analytical HUD/annotation/labels, composited on top.
|
||||
`intensity` 0..1 (0 = no overlay)."""
|
||||
|
||||
intensity: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Restyle:
|
||||
"""Right axis (§4.1): pre-baked generative v2v dreamlike substrate.
|
||||
`blend` 0..1 (0 = raw substrate, no restyle)."""
|
||||
|
||||
blend: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RenderPlan:
|
||||
"""The full layered alteration for one knob vector (§4.2)."""
|
||||
|
||||
grade: ColorGrade
|
||||
overlay: AnalyticalOverlay
|
||||
restyle: Restyle
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
"""True when the plan leaves the neutral base un-altered."""
|
||||
return (
|
||||
self.grade.is_identity
|
||||
and self.overlay.intensity == 0.0
|
||||
and self.restyle.blend == 0.0
|
||||
)
|
||||
|
||||
|
||||
def _overlay_intensity(left: int) -> float:
|
||||
"""Left knob -> analytical-overlay intensity (0..1)."""
|
||||
return left / KNOB_MAX
|
||||
|
||||
|
||||
def _restyle_blend(right: int) -> float:
|
||||
"""Right knob -> v2v restyle blend (0..1)."""
|
||||
return right / KNOB_MAX
|
||||
|
||||
|
||||
def _mood_tone(dark: int, light: int) -> float:
|
||||
"""(dark, light) -> signed mood grade in [-1, 1]; equal -> 0 identity (§5)."""
|
||||
return (light - dark) / KNOB_MAX
|
||||
|
||||
|
||||
def plan_alteration(coord: Coordinate) -> RenderPlan:
|
||||
"""Map a knob vector to its layered RenderPlan (design §4)."""
|
||||
return RenderPlan(
|
||||
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light)),
|
||||
overlay=AnalyticalOverlay(intensity=_overlay_intensity(coord.left)),
|
||||
restyle=Restyle(blend=_restyle_blend(coord.right)),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pytest tests/test_player_alteration.py -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add player/alteration.py tests/test_player_alteration.py
|
||||
git commit -m "feat(player): alteration engine — knob vector to RenderPlan (design §4/§5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Player state machine — controls stream to transitions
|
||||
|
||||
**Files:**
|
||||
- Create: `player/state.py`
|
||||
- Test: `tests/test_player_state.py`
|
||||
|
||||
The `Player` holds a base-clip library and the current `Playback`. Each `update(controls)` resolves the desired `Playback` (chosen base clip + RenderPlan + content resolution + volume/brightness) and returns the `Transition` from the previous playback:
|
||||
|
||||
- target video off, was on → `FADE_TO_BLACK`
|
||||
- target video on, was off → `FADE_FROM_BLACK`
|
||||
- both video on, clip or restyle variant changed → `CROSSFADE`
|
||||
- both video on, only grade/overlay/level changed → `LIVE_UPDATE` (§4.3: grade + overlay are continuous runtime ops)
|
||||
- both video off, audio/level changed → `LIVE_UPDATE`
|
||||
- nothing changed → `NONE`
|
||||
|
||||
Base-clip choice is an injected callable (default: first clip); knobs no longer *select* (the base is neutral) — they *transform*. The library is duck-typed objects with an `.id`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/test_player_state.py
|
||||
from dataclasses import dataclass
|
||||
|
||||
import pytest
|
||||
|
||||
from player.controls import Controls
|
||||
from player.state import Player, Playback, Transition, TransitionKind
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FakeClip:
|
||||
id: str
|
||||
|
||||
|
||||
LIB = [FakeClip("base-a"), FakeClip("base-b")]
|
||||
|
||||
|
||||
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
|
||||
return Controls(content, left, right, dark, light, volume, brightness)
|
||||
|
||||
|
||||
def test_first_update_to_video_fades_in_from_black():
|
||||
p = Player(LIB)
|
||||
t = p.update(_controls(content="video"))
|
||||
assert t.kind == TransitionKind.FADE_FROM_BLACK
|
||||
assert t.playback.clip_id == "base-a"
|
||||
assert t.playback.content.video is True
|
||||
|
||||
|
||||
def test_off_from_video_fades_to_black_and_silences():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video"))
|
||||
t = p.update(_controls(content="off"))
|
||||
assert t.kind == TransitionKind.FADE_TO_BLACK
|
||||
assert t.playback.clip_id is None
|
||||
assert t.playback.content.audio_source == "none"
|
||||
|
||||
|
||||
def test_no_change_yields_none_transition():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=1))
|
||||
t = p.update(_controls(content="video", left=1))
|
||||
assert t.kind == TransitionKind.NONE
|
||||
|
||||
|
||||
def test_grade_change_is_a_live_update_not_a_crossfade():
|
||||
# design §4.3: the Dark/Light grade is a continuous runtime op
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", dark=0, light=0))
|
||||
t = p.update(_controls(content="video", dark=4, light=0))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_overlay_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=0))
|
||||
t = p.update(_controls(content="video", left=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.overlay.intensity == 1.0
|
||||
|
||||
|
||||
def test_restyle_change_crossfades_the_substrate():
|
||||
# design §4.3: the Right v2v substrate is a pre-baked variant swap
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", right=0))
|
||||
t = p.update(_controls(content="video", right=4))
|
||||
assert t.kind == TransitionKind.CROSSFADE
|
||||
assert t.playback.plan.restyle.blend == 1.0
|
||||
|
||||
|
||||
def test_volume_only_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", volume=1))
|
||||
t = p.update(_controls(content="video", volume=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.volume == 4
|
||||
|
||||
|
||||
def test_audio_source_change_while_black_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="white_noise"))
|
||||
t = p.update(_controls(content="music"))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.content.audio_source == "music"
|
||||
|
||||
|
||||
def test_injected_base_chooser_is_used():
|
||||
p = Player(LIB, choose_base=lambda lib: lib[1])
|
||||
t = p.update(_controls(content="video"))
|
||||
assert t.playback.clip_id == "base-b"
|
||||
|
||||
|
||||
def test_empty_library_with_video_raises():
|
||||
p = Player([])
|
||||
with pytest.raises(ValueError):
|
||||
p.update(_controls(content="video"))
|
||||
|
||||
|
||||
def test_off_with_empty_library_is_fine():
|
||||
p = Player([])
|
||||
t = p.update(_controls(content="off"))
|
||||
assert t.kind == TransitionKind.NONE # already black at init
|
||||
assert t.playback.clip_id is None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pytest tests/test_player_state.py -v`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'player.state'`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# player/state.py
|
||||
"""The player state machine: a stream of Controls -> playback transitions.
|
||||
|
||||
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
|
||||
desired Playback (which neutral base clip, how it is altered, what audio plays,
|
||||
at what levels) and returns the Transition from the previous Playback. The
|
||||
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
|
||||
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
|
||||
pre-baked Right v2v variant needs a CROSSFADE, and toggling video on/off
|
||||
fades to/from black.
|
||||
|
||||
Knobs no longer *select* a clip (the base library is neutral by construction);
|
||||
they *transform* it. Which neutral base to show is an injected policy
|
||||
(`choose_base`), defaulting to the first clip; richer rotation is a later slice.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable, Optional
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import RenderPlan, plan_alteration
|
||||
from player.content import ContentResolution, resolve_content
|
||||
from player.controls import Controls
|
||||
|
||||
|
||||
class TransitionKind:
|
||||
NONE = "none"
|
||||
LIVE_UPDATE = "live_update"
|
||||
CROSSFADE = "crossfade"
|
||||
FADE_TO_BLACK = "fade_to_black"
|
||||
FADE_FROM_BLACK = "fade_from_black"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Playback:
|
||||
"""What should currently be playing."""
|
||||
|
||||
clip_id: Optional[str] # None = black walls
|
||||
plan: Optional[RenderPlan] # None when black
|
||||
content: ContentResolution
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Transition:
|
||||
kind: str
|
||||
playback: Playback
|
||||
|
||||
|
||||
def _first(library):
|
||||
if not library:
|
||||
raise ValueError("no base clips available to play video")
|
||||
return library[0]
|
||||
|
||||
|
||||
_BLACK = Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=ContentResolution("none", False),
|
||||
volume=0,
|
||||
brightness=0,
|
||||
)
|
||||
|
||||
|
||||
class Player:
|
||||
def __init__(
|
||||
self,
|
||||
base_library,
|
||||
*,
|
||||
choose_base: Callable = _first,
|
||||
approved_only: bool = False,
|
||||
):
|
||||
self._library = list(base_library)
|
||||
self._choose_base = choose_base
|
||||
self._approved_only = approved_only # reserved for catalog-backed libs
|
||||
self._current = _BLACK
|
||||
|
||||
def _resolve(self, controls: Controls) -> Playback:
|
||||
content = resolve_content(controls.content)
|
||||
if not content.video:
|
||||
return Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=content,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
clip = self._choose_base(self._library)
|
||||
coord = Coordinate(controls.left, controls.right, controls.dark, controls.light)
|
||||
return Playback(
|
||||
clip_id=clip.id,
|
||||
plan=plan_alteration(coord),
|
||||
content=content,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
|
||||
def _classify(self, prev: Playback, nxt: Playback) -> str:
|
||||
prev_video = prev.clip_id is not None
|
||||
next_video = nxt.clip_id is not None
|
||||
if prev == nxt:
|
||||
return TransitionKind.NONE
|
||||
if prev_video and not next_video:
|
||||
return TransitionKind.FADE_TO_BLACK
|
||||
if next_video and not prev_video:
|
||||
return TransitionKind.FADE_FROM_BLACK
|
||||
if next_video and prev_video:
|
||||
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
|
||||
return TransitionKind.CROSSFADE
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
# both black: only audio/levels could have changed
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
|
||||
def update(self, controls: Controls) -> Transition:
|
||||
nxt = self._resolve(controls)
|
||||
kind = self._classify(self._current, nxt)
|
||||
self._current = nxt
|
||||
return Transition(kind=kind, playback=nxt)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pytest tests/test_player_state.py -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add player/state.py tests/test_player_state.py
|
||||
git commit -m "feat(player): player state machine — controls stream to transitions"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Register the package and run the full suite
|
||||
|
||||
**Files:**
|
||||
- Modify: `pyproject.toml` (`[tool.setuptools] packages`)
|
||||
|
||||
- [ ] **Step 1: Add `"player"` to the packages list**
|
||||
|
||||
In `pyproject.toml`, change:
|
||||
|
||||
```toml
|
||||
[tool.setuptools]
|
||||
packages = ["hef", "tools", "simulator"]
|
||||
```
|
||||
to:
|
||||
```toml
|
||||
[tool.setuptools]
|
||||
packages = ["hef", "tools", "simulator", "player"]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the entire test suite**
|
||||
|
||||
Run: `pytest -q`
|
||||
Expected: all prior tests still pass plus the new `test_player_*` files; no failures.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add pyproject.toml
|
||||
git commit -m "build(player): register the player package"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Update the roadmap to reflect slice 1 shipped
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/ROADMAP.md` (§3 Player Runtime)
|
||||
|
||||
- [ ] **Step 1: Update §3 status and deliverables**
|
||||
|
||||
Under "## 3. Player Runtime (Pi)", note that the **alteration-engine + player-core slice** (pure logic) is done and tested, link this plan, and list the remaining slices (mpv/ffmpeg runtime integration + GPU grade/overlay shaders; real USB-serial reader + the 3⇄4 framing contract; white-noise DSP; offline v2v variant build pipeline; catalog model changes for audio-source + neutral-base flag). Keep the table's status marker for sub-project 3 as in-progress (⏳), not done.
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/ROADMAP.md
|
||||
git commit -m "docs(roadmap): sub-project 3 alteration-engine + player-core slice shipped"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:**
|
||||
- §4.1 per-axis alteration → Task 3 (overlay=Left, restyle=Right, grade=mood). ✓
|
||||
- §4.2 composition rule (substrate vs overlay; Left+Right stack) → Task 3 tests `test_left_and_right_stack_not_cancel`, `test_whole_brain_dark_corner_*`. ✓
|
||||
- §4.3 runtime vs pre-baked (grade/overlay continuous; restyle variant swap) → Task 4 transition kinds (`LIVE_UPDATE` vs `CROSSFADE`). ✓
|
||||
- §5 mood grade, center = identity → Task 3 `test_equal_dark_and_light_is_identity_grade`, signed tone. ✓
|
||||
- §6 7-way content dial → Task 2 full table. ✓
|
||||
- §7 volume + brightness intensity → Task 1 (modeled) + Task 4 (`test_volume_only_change_is_a_live_update`). ✓
|
||||
- Roadmap §3 "done when: plays correct segment, loops, crossfades, goes dark on None, testable with serial mocked" → Task 4 (transitions incl. fade-to-black; serial mocked as a `Controls` stream). Looping is implicit (a clip plays until the desired playback changes); explicit loop timing belongs to the mpv integration slice (deferred, noted in Task 6). ✓ (with the runtime-loop portion deferred and documented).
|
||||
|
||||
**Deferred (documented, not gaps):** mpv/ffmpeg + GPU; real serial + framing; white-noise DSP; v2v offline pipeline; catalog audio-source/neutral-base fields; base-clip rotation policy; `approved_only` enforcement against a real catalog (the flag is plumbed but inert here).
|
||||
|
||||
**Placeholder scan:** none — every step has concrete code/commands.
|
||||
|
||||
**Type consistency:** `Controls(content,left,right,dark,light,volume,brightness)` used identically in Tasks 1 & 4; `RenderPlan.{grade,overlay,restyle}` and `.is_identity` consistent across Tasks 3 & 4; `ContentResolution(audio_source,video)` consistent across Tasks 2 & 4; `TransitionKind` constants consistent. `Coordinate(left,right,dark,light)` matches `hef/selection.py`.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,193 @@
|
||||
# Experience Simulator (Curator's X-ray) — Design Spec
|
||||
|
||||
**Date:** 2026-06-04
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Session:** 0003 (discovery)
|
||||
|
||||
> A web-based stand-in for the installation's control panel, run on localhost in a
|
||||
> Docker container, built so a curator can **feel the selection model** — turn the
|
||||
> dials and see, in full, which catalog pieces the nearest-match algorithm
|
||||
> surfaces and why. It validates the *principles* (coordinate tagging + selection)
|
||||
> and the *interface* before the physical Pi-player / Arduino stack is built.
|
||||
|
||||
---
|
||||
|
||||
## 1. What this is
|
||||
|
||||
A single-page web tool that reproduces the installation's five controls and wires
|
||||
them to the **real** `hef.selection` code, then shows a transparent "X-ray" of the
|
||||
result: the picked piece, the full ranked candidate pool with distances, and a
|
||||
live map of where the knob point and candidates sit in the coordinate space.
|
||||
|
||||
It is a **curator's instrument**, not the installation itself. Its job is to let a
|
||||
human judge whether the tagging rubric (design spec §2) and the nearest-match
|
||||
selection (design spec §3) actually surface fitting pieces at every knob position —
|
||||
so a "wrong" result can be diagnosed as bad tagging vs. a sparse neighborhood.
|
||||
|
||||
Primary purpose (settled this session): **feel the selection model.** Faithful
|
||||
media playback is explicitly *not* a goal.
|
||||
|
||||
### Design constraints
|
||||
- **Real algorithm, single source of truth.** The simulator runs the shipped
|
||||
`hef.selection` code. No reimplementation of selection in JavaScript — a copy
|
||||
could drift from what the Pi will run, which would defeat the purpose.
|
||||
- **Runs locally in Docker.** One container, one command, `localhost`. No
|
||||
networking beyond the local browser, no auth, no multi-user.
|
||||
- **Reads the model; never curates into it.** The simulator does not write catalog
|
||||
records back. Editing/approval stays with the sub-project-2 review tools.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture
|
||||
|
||||
One Python service in one container. **FastAPI** serves both a JSON selection API
|
||||
and a dependency-free static frontend. The browser holds the dial state; each
|
||||
change POSTs to the API and re-renders the X-ray.
|
||||
|
||||
```
|
||||
browser (dials + X-ray) ──POST /api/select──▶ FastAPI ──▶ hef.selection (REAL)
|
||||
▲ │
|
||||
└────────── pool + distances + pick ◀─────┘
|
||||
```
|
||||
|
||||
- **Backend:** `simulator/app.py` (FastAPI), `simulator/fixtures.py` (synthetic
|
||||
catalog generator). Imports `hef.catalog` and `hef.selection` directly.
|
||||
- **Frontend:** `simulator/static/` — `index.html`, `app.js`, `style.css`. Vanilla
|
||||
JS, no build step.
|
||||
- **Packaging:** `simulator/Dockerfile` + a one-command run (`docker compose up`
|
||||
or a `make sim` target), serving on `localhost`.
|
||||
|
||||
This adds a `simulator/` top-level package, consistent with the repo layout in the
|
||||
main design spec §10 (`hef/`, `player/`, `firmware/`, `tools/`).
|
||||
|
||||
---
|
||||
|
||||
## 3. One additive change to `hef.selection`
|
||||
|
||||
The X-ray needs the ranked pool with distances — which `select()` already computes
|
||||
internally (`ranked`, `nearest`) but does not expose. To avoid duplicating ranking
|
||||
logic in the simulator (the very drift risk §1 forbids), extract it into a public
|
||||
function and refactor `select()` to call it:
|
||||
|
||||
```python
|
||||
def ranked_candidates(
|
||||
records, coord: Coordinate, mode: str, *,
|
||||
pool_size: int = 4, weights: Weights = Weights(), approved_only: bool = False,
|
||||
) -> list[tuple[Record, float]]:
|
||||
"""Mode-eligible records sorted nearest-first, each paired with its distance.
|
||||
|
||||
Honors the same 'av' fallback and approved_only filtering as select().
|
||||
"""
|
||||
```
|
||||
|
||||
`select()` becomes a thin wrapper over `ranked_candidates()` (take the pool, then
|
||||
return `nearest[0]` or `rng.choice(nearest)`). This is the **only** change to
|
||||
shipped code: additive, behavior-preserving for `select()`, and independently
|
||||
tested. The X-ray then displays exactly the pool the picker chooses from.
|
||||
|
||||
---
|
||||
|
||||
## 4. Synthetic fixture catalog (`simulator/fixtures.py`)
|
||||
|
||||
The real `catalog/library.jsonl` is empty. A deterministic (seeded) generator
|
||||
produces a dense, representative spread of `Record`s so nearest-match behavior can
|
||||
be felt everywhere immediately:
|
||||
|
||||
- Coverage across the `(left, right, dark, light)` space (the 5×5 brain plane ×
|
||||
5×5 mood plane = 625 configs) and across all content modes (`audio`, `video`,
|
||||
`av`).
|
||||
- Plausible titles and `rationale`s; `source_archive`/`license` drawn from the
|
||||
real pools so records validate.
|
||||
- A deliberate mix of `review_status: proposed` and `approved`, so the
|
||||
`approved_only` toggle is exercisable.
|
||||
- No real media: `file_path` empty, `dominant_color` empty (optional field).
|
||||
- Output validates against `hef.catalog.validate_catalog`.
|
||||
|
||||
Loaded at startup. A flag (`--catalog` / env var) lets the simulator point at
|
||||
`catalog/library.jsonl` instead when it is non-empty — **same loader**, so swapping
|
||||
in the real catalog later is free.
|
||||
|
||||
---
|
||||
|
||||
## 5. The API
|
||||
|
||||
### `POST /api/select`
|
||||
Request:
|
||||
```json
|
||||
{ "left":0-4, "right":0-4, "dark":0-4, "light":0-4,
|
||||
"mode":"none|audio|video|av",
|
||||
"pool_size":int, "weights":{"brain":float,"mood":float},
|
||||
"approved_only":bool }
|
||||
```
|
||||
Response:
|
||||
```json
|
||||
{ "pick": <record|null>,
|
||||
"pool": [ { "record": <record>, "distance": float, "rank": int }, ... ],
|
||||
"coverage": { "candidates_in_mode": int } }
|
||||
```
|
||||
- `none` selector mode → `pick:null`, `pool:[]` (the void/rest state).
|
||||
- No candidates → `pick:null`, `pool:[]`.
|
||||
- **Deterministic pick.** The displayed `pick` is the rank-1 (nearest) candidate —
|
||||
`select()` called with `rng=None` — so the X-ray is legible and stable for a
|
||||
given knob position. The room *shuffles* within the nearest pool at runtime;
|
||||
the returned `pool` IS that shuffle-set, so the curator sees every piece the
|
||||
real installation could surface there.
|
||||
- Knob values are constrained 0–4 and `mode` to the four values by Pydantic; bad
|
||||
input is a clean `422`.
|
||||
- Backed by `hef.selection.ranked_candidates()` (pool) + `select()` (the pick).
|
||||
|
||||
### `GET /api/catalog/meta`
|
||||
Counts, per-mode breakdown, `proposed`/`approved` split, and coverage stats for
|
||||
the header.
|
||||
|
||||
---
|
||||
|
||||
## 6. The X-ray UI
|
||||
|
||||
**Controls**
|
||||
- The five real dials: a 4-way mode selector (None/Audio/Video/A+V) and four 0–4
|
||||
knobs (Left, Right, Dark, Light).
|
||||
- The model knobs (so the algorithm's own parameters can be felt): brain/mood
|
||||
**weight sliders**, **pool-size** control, and an **approved-only** toggle.
|
||||
|
||||
**Readout**
|
||||
- **Picked piece:** title, mode, coordinate, distance, rationale.
|
||||
- **Ranked pool:** the N nearest with distances, winner highlighted.
|
||||
- **Two 5×5 grid maps** (brain L×R, mood D×L): the current knob point plus where
|
||||
candidates sit, so the neighborhood that produced the pick is visible at a
|
||||
glance.
|
||||
|
||||
---
|
||||
|
||||
## 7. Testing
|
||||
|
||||
- **Unit (`hef`):** `ranked_candidates()` ordering, the `av` pool fallback,
|
||||
`approved_only` filtering, weight effects; `select()` still behaves as before.
|
||||
- **Fixtures:** coverage test (every mode populated; space spanned; output passes
|
||||
`validate_catalog`).
|
||||
- **API:** FastAPI `TestClient` contract tests — valid select returns ranked pool;
|
||||
`none` returns the void; bad input → 422; `approved_only` narrows the pool.
|
||||
- **Behavior:** the felt behaviors are captured as gherkin scenarios under
|
||||
`features/` (this session's BDD deliverable).
|
||||
|
||||
---
|
||||
|
||||
## 8. Explicitly out of scope (YAGNI)
|
||||
|
||||
- Real media playback (audio/video) — the tool reads the model, not the content.
|
||||
- Writing/editing catalog records back — curation stays in the sub-project-2 tools.
|
||||
- Networking, auth, multi-user, session recording.
|
||||
- Any reimplementation of selection logic outside `hef`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Relationship to the roadmap
|
||||
|
||||
This is an **upstream discovery vehicle** for the physical stack (sub-projects 3
|
||||
and 4). It exercises sub-project 1's `hef.selection` against a stand-in catalog and
|
||||
is the richer, experienceable form of the "keyboard/serial stand-in" the roadmap
|
||||
anticipates for sub-project 3. It does not replace the Pi player or the firmware;
|
||||
it de-risks their *principles and interface* first. The synthetic catalog is a
|
||||
stand-in for the sub-project-2 output and is swapped for the real catalog through
|
||||
the same loader once ingest has populated it.
|
||||
@@ -4,15 +4,26 @@
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
|
||||
> **Revision — 2026-06-04 (display architecture):** the display moved from *four
|
||||
> projectors (one per wall) + a procedural side-wall renderer* to a **single
|
||||
> panoramic projector spanning the three walls the viewer faces, showing the real
|
||||
> selected video** (launch content focus: **nature video**). This supersedes the
|
||||
> original §6 (four projectors) and **removes §7 (procedural side walls)**, and
|
||||
> reshapes the roadmap's sub-project 3 (player drives one pano output) and
|
||||
> sub-project 5 (procedural side walls — **dropped**). Affected sections below are
|
||||
> updated in place and marked. The coordinate model (§2), selection (§3), and
|
||||
> catalog (§4) are unchanged.
|
||||
|
||||
---
|
||||
|
||||
## 1. What this is
|
||||
|
||||
A single-viewer immersive art installation. One person sits in a chair at the
|
||||
center of a small room with four walls, each fronted by a projector. A DJ-style
|
||||
control panel lets them "tune" their experience along a small set of felt axes;
|
||||
the system finds the public-domain media nearest that tuning and plays it. The
|
||||
piece is an *experience filter*: the viewer dials in how they want to feel and
|
||||
A single-viewer immersive art installation. One person sits in a chair in a small
|
||||
room; a **single panoramic projector wraps the real selected video across the
|
||||
three walls they face**. A DJ-style control panel lets them "tune" their
|
||||
experience along a small set of felt axes; the system finds the public-domain
|
||||
media nearest that tuning and plays it (launch content focus: **nature video**).
|
||||
The piece is an *experience filter*: the viewer dials in how they want to feel and
|
||||
think, and the room answers with found human artifacts that match.
|
||||
|
||||
Design constraints that shaped everything below:
|
||||
@@ -160,24 +171,37 @@ cannot decode HD video or drive projectors.
|
||||
- **Raspberry Pi 5 (or small mini-PC) — brain + player.** Holds the catalog,
|
||||
reads the Arduino's serial stream, runs the selection algorithm, and plays
|
||||
media. **The hard drive plugs into the Pi**, not the Arduino.
|
||||
- **Projectors.** The Pi drives the **primary** wall (the wall the viewer faces)
|
||||
with real content. The three **side walls** show procedural ambient (§7),
|
||||
driven from the same Pi's additional outputs or a second cheap Pi.
|
||||
- **Projector (single, panoramic).** *Revised 2026-06-04.* The Pi drives **one
|
||||
panoramic projector that spans the three walls the viewer faces**, showing the
|
||||
real selected video across the whole field. There is no longer a primary/side
|
||||
split and no procedural side-wall renderer (the former §7). This both simplifies
|
||||
the hardware (one output, one projector) and removes the need to *compute*
|
||||
`dominant_color` for ambient walls.
|
||||
|
||||
Parts delta vs. the original sketch: add a ~$80 Pi; everything else
|
||||
(Arduino, knobs, hard drive, projectors) stays.
|
||||
Parts delta vs. the original sketch: add a ~$80 Pi and use a single panoramic
|
||||
(ultra-wide / short-throw) projector instead of four; everything else (Arduino,
|
||||
knobs, hard drive) stays.
|
||||
|
||||
---
|
||||
|
||||
## 7. Procedural side walls
|
||||
## 7. ~~Procedural side walls~~ — REMOVED (2026-06-04)
|
||||
|
||||
The three non-primary walls render a slow gradient / color wash:
|
||||
**Superseded by the §6 single-panoramic-projector revision.** The three walls the
|
||||
viewer faces now show the **real** selected video (the pano projector spans them),
|
||||
so there is no separate procedural ambient renderer. The mood axis is felt through
|
||||
the chosen content itself rather than a synthetic side-wall wash.
|
||||
|
||||
- **Hue** comes from the playing piece's `dominant_color`.
|
||||
- **Brightness** tracks the Dark / Light knobs.
|
||||
Consequences:
|
||||
|
||||
This costs near-zero storage and makes the mood axis physically felt in
|
||||
peripheral vision. In `None` mode the side walls also fade to black.
|
||||
- **Sub-project 5 (procedural side walls) is dropped** from the roadmap.
|
||||
- `dominant_color` loses its only consumer; computing it becomes **optional /
|
||||
opt-in** in the ingest tooling (sub-project 2 spec §5.2) rather than a required
|
||||
mechanical tag. The field stays in the catalog schema (unused-but-harmless,
|
||||
default `""`) for a possible future ambient/lighting use.
|
||||
|
||||
> *Original intent (for the record): the three non-primary walls rendered a slow
|
||||
> color wash — hue from the playing piece's `dominant_color`, brightness from the
|
||||
> Dark/Light knobs — to make the mood axis felt in peripheral vision.*
|
||||
|
||||
---
|
||||
|
||||
@@ -218,14 +242,18 @@ defensible.
|
||||
## 10. Repo layout
|
||||
|
||||
```
|
||||
hef/ shared library: catalog model + selection (sub-project 1, built)
|
||||
catalog/ the tagged content catalog (JSONL)
|
||||
player/ selection algorithm + media player (runs on the Pi)
|
||||
player/ media player driving the single panoramic projector (runs on the Pi)
|
||||
firmware/ Arduino sketch (reads knobs/selector → USB serial)
|
||||
sidewalls/ procedural ambient renderer
|
||||
tools/ sourcing/ingest, dominant-color, license checker, review tool
|
||||
tools/ sourcing/ingest, mechanical tagging, review tool (sub-project 2)
|
||||
docs/ this spec + operator/build guide
|
||||
```
|
||||
|
||||
*(Revised 2026-06-04: `sidewalls/` removed with §7; the selection algorithm
|
||||
shipped in the shared `hef/` package, not under `player/`, since the tools import
|
||||
it too — see the sub-project-1 plan.)*
|
||||
|
||||
---
|
||||
|
||||
## 11. Explicitly out of scope (YAGNI)
|
||||
@@ -240,9 +268,13 @@ docs/ this spec + operator/build guide
|
||||
|
||||
## 12. Open implementation questions (for the plan, not blockers)
|
||||
|
||||
- Player stack on the Pi (e.g. mpv/ffmpeg-based vs. a custom renderer) and how
|
||||
crossfades are handled.
|
||||
- Player stack on the Pi (e.g. mpv/ffmpeg-based vs. a custom renderer), how
|
||||
crossfades are handled, and how a single wide/panoramic output is driven (one
|
||||
ultra-wide surface vs. spanned displays).
|
||||
- Exact serial protocol/framing between Arduino and Pi.
|
||||
- Whether side walls run on the primary Pi's extra outputs or a second Pi.
|
||||
- Ingest tooling language and where the per-archive downloaders live.
|
||||
- Whether the player hard-restricts to `approved` records.
|
||||
- ~~Whether side walls run on the primary Pi's extra outputs or a second Pi~~ —
|
||||
resolved: no procedural side walls (§7 removed).
|
||||
- ~~Ingest tooling language and where the per-archive downloaders live~~ —
|
||||
resolved by the sub-project-2 spec: Python under `tools/`, per-archive fetchers
|
||||
in `tools/ingest/`.
|
||||
|
||||
@@ -0,0 +1,560 @@
|
||||
# Sub-project 2 — Ingest & Tagging / Review Tools — Spec
|
||||
|
||||
**Date:** 2026-06-04
|
||||
**Status:** Proposed (sub-project 2 of 5) — pending operator approval
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Design reference:** [`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md) (esp. §4 catalog, §5 division-of-labor, §8 sourcing)
|
||||
**Roadmap item:** [`docs/ROADMAP.md`](../../ROADMAP.md) §2
|
||||
**Builds on:** sub-project 1 — `hef.catalog` + `hef.selection` (done, merged, 37 tests)
|
||||
|
||||
---
|
||||
|
||||
## 1. What this is
|
||||
|
||||
Sub-project 1 gave us the **spine**: a validated JSONL catalog (`hef.catalog`) and
|
||||
a nearest-match selector (`hef.selection`). Today the only way to fill that
|
||||
catalog is hand-authoring records (per [`docs/USER_GUIDE.md`](../../USER_GUIDE.md)).
|
||||
|
||||
Sub-project 2 builds the **assisted *draft-then-review* pipeline** the design spec
|
||||
calls for (§5), under `tools/`:
|
||||
|
||||
1. **Ingest** — per-archive fetchers pull a candidate piece from a public-domain
|
||||
pool and write a catalog record.
|
||||
2. **Mechanical tagging** — tooling auto-fills the non-curatorial fields: `mode`
|
||||
(via `ffprobe`), `license`/`attribution`/`source_*` (from origin),
|
||||
`duration_s`, `resolution`, and — optionally, opt-in only — `dominant_color`
|
||||
(computed from the video; see §5.2 design-change note).
|
||||
3. **Coordinate drafting** — a proposer suggests `left/right/dark/light` with a
|
||||
one-line `rationale`, written as `review_status: proposed`.
|
||||
4. **Review CLI** — walks `proposed` records one at a time (proposed coords +
|
||||
rationale + preview frame), accepts or corrects, flips to `approved`, and
|
||||
stamps `reviewed_at`.
|
||||
|
||||
The output is a populated catalog that `load_catalog` validates and the player
|
||||
(sub-project 3) can `select()` over. "How far from done" stays exactly what §4
|
||||
defines: the count of records still in `review_status: proposed`.
|
||||
|
||||
### Design invariants this honors
|
||||
|
||||
- **Curation is the artwork** (design §1). The four coordinates are a human
|
||||
curatorial act (design §11: *no automatic ML coordinate tagging*). The pipeline
|
||||
**drafts** coordinates as a starting point, but every coordinate is human-blessed
|
||||
before it can be selected. Drafting is deterministic and rule-based — there is no
|
||||
ML model in the tool.
|
||||
- **The catalog is the spine.** Tools only **read** and **append/rewrite** through
|
||||
`hef.catalog`; they never define a parallel record shape.
|
||||
- **Local, dependency-light, hermetically testable.** Like sub-project 1, the
|
||||
testable core is pure-stdlib Python. The two external boundaries — `ffmpeg`/`ffprobe`
|
||||
(subprocess) and archive HTTP — are isolated behind injectable seams so the unit
|
||||
suite never shells out or hits the network.
|
||||
|
||||
---
|
||||
|
||||
## 2. What `hef.catalog` already gives us (and why the schema needs no change)
|
||||
|
||||
The headline of the "build on without breaking" constraint: **the `Record` schema
|
||||
is already complete for this flow.** Sub-project 1 shipped, with defaults chosen
|
||||
for exactly the draft-then-review pipeline:
|
||||
|
||||
| Field group | Fields | Who sets it here |
|
||||
|---|---|---|
|
||||
| Identity / origin | `id`, `title`, `source_url`, `source_archive` | ingest (from the fetcher) |
|
||||
| License | `license`, `attribution` | mechanical tagging (from origin) |
|
||||
| Mechanical media | `mode`, `duration_s`, `resolution`, `dominant_color` | mechanical tagging (ffprobe/ffmpeg) |
|
||||
| Curatorial | `left`, `right`, `dark`, `light` | drafting (proposed) → review (approved) |
|
||||
| Review state | `review_status` (default `"proposed"`), `rationale`, `reviewed_at` (default `null`) | drafting sets `rationale`; review sets `reviewed_at` |
|
||||
| Misc | `file_path`, `notes` | ingest (`file_path`); either (`notes`) |
|
||||
|
||||
`Record`'s defaults — `review_status="proposed"`, `reviewed_at=None`,
|
||||
`rationale=""`, plus `attribution`/`resolution`/`dominant_color` defaulting to
|
||||
`""` — are precisely an ingested-but-unreviewed record. **Ingest constructs a
|
||||
`Record` and the proposed-state shape falls out of the defaults; nothing in the
|
||||
schema is added or changed.**
|
||||
|
||||
The tools consume these existing symbols unchanged:
|
||||
|
||||
- `hef.catalog`: `Record`, `validate`, `record_to_dict`, `record_from_dict`,
|
||||
`load_catalog`, `save_catalog`, `append_record`, `CatalogError`, and the
|
||||
vocab constants `MODES`, `LICENSES`, `REVIEW_STATUSES`, `ATTRIBUTION_LICENSES`,
|
||||
`COORD_FIELDS`, `COORD_MIN`, `COORD_MAX`.
|
||||
- `hef.selection`: `Coordinate`, `select` (used by the end-to-end test and by
|
||||
the player later).
|
||||
|
||||
---
|
||||
|
||||
## 3. The one required addition to `hef.catalog`
|
||||
|
||||
Per the constraint, every addition the tools need is called out explicitly. There
|
||||
is exactly **one** required change to the shared library, and it is purely
|
||||
additive (no existing symbol changes signature or behavior):
|
||||
|
||||
### 3.1 `validate_catalog(records)` — catalog-level integrity (unique ids)
|
||||
|
||||
`validate()` is per-record; it cannot see the whole file, so it cannot catch a
|
||||
**duplicate `id`**. The ingest pipeline appends repeatedly and the review CLI
|
||||
looks records up *by id* — both are unsafe if ids collide (ingest silently
|
||||
double-adds a re-fetched piece; review can't tell two records apart). So add a
|
||||
catalog-level check to the shared spine, where the player also benefits:
|
||||
|
||||
```python
|
||||
def validate_catalog(records) -> None:
|
||||
"""Validate every record AND the cross-record invariants (currently: unique ids).
|
||||
Raises CatalogError naming the first duplicate id."""
|
||||
```
|
||||
|
||||
- It calls `validate()` on each record, then asserts `id` uniqueness.
|
||||
- **Non-breaking:** new symbol; existing `validate`/`load_catalog`/`save_catalog`/
|
||||
`append_record` are untouched. Any catalog valid today (unique ids) stays valid.
|
||||
- **Used by tools** at two points: before `append_record` during ingest (reject a
|
||||
candidate whose derived `id` already exists), and after `load_catalog` in the
|
||||
review CLI (fail loudly on a corrupted/duplicated catalog before editing it).
|
||||
|
||||
> A companion convenience, `index_by_id(records) -> dict[str, Record]`, may be
|
||||
> added alongside it (the review CLI and dedupe both want id-keyed lookup). It is
|
||||
> a trivial helper; if we prefer to keep the spine minimal it can live in the
|
||||
> tools layer instead. **Recommendation:** add `index_by_id` to `hef.catalog`
|
||||
> too, since by-id lookup is a catalog concern the player will also want.
|
||||
|
||||
**Everything else this sub-project needs lives in `tools/` and only *consumes*
|
||||
`hef.catalog`.** No other change to `hef/` is required. (Considered and
|
||||
deliberately *not* added now — see §11: a `source_id` archive-identifier field,
|
||||
a `media_sha256`, an `ingested_at` timestamp. They are future schema extensions,
|
||||
not needed for the done-criteria.)
|
||||
|
||||
---
|
||||
|
||||
## 4. Package layout
|
||||
|
||||
Extends the existing repo (design §10; the sub-project-1 plan put shared code in
|
||||
`hef/` and data in `catalog/`):
|
||||
|
||||
```
|
||||
tools/
|
||||
__init__.py
|
||||
http.py # thin stdlib-urllib client: timeout, retry, user-agent (injectable)
|
||||
probe.py # Prober: ffprobe wrapper -> parsed streams (injectable)
|
||||
mediatools.py # ffmpeg helpers: extract frame, dominant color, waveform thumb
|
||||
tagging.py # mechanical tagging: assemble mode/duration/resolution/dominant_color
|
||||
licensing.py # per-archive license/attribution normalization -> LICENSES vocab
|
||||
drafting.py # Draft, Proposer protocol, HeuristicProposer
|
||||
ingest/
|
||||
__init__.py
|
||||
base.py # Candidate dataclass + Fetcher protocol + the ingest pipeline
|
||||
internet_archive.py
|
||||
musopen.py
|
||||
librivox.py
|
||||
nasa.py
|
||||
fma.py # Free Music Archive
|
||||
freesound.py
|
||||
ingest_cli.py # `python -m tools.ingest_cli` — run a fetcher, write proposed records
|
||||
review.py # review state-transition core (pure, tested)
|
||||
review_cli.py # `python -m tools.review_cli` — interactive walk of proposed records
|
||||
```
|
||||
|
||||
- `tools/` is added to `[tool.setuptools] packages` in `pyproject.toml`.
|
||||
- **Invocation matches the repo convention** (USER_GUIDE: run from repo root, no
|
||||
install, `import hef` resolves via the root `conftest.py`): the CLIs run as
|
||||
`.venv/bin/python -m tools.ingest_cli …` and `… -m tools.review_cli …`.
|
||||
`[project.scripts]` console-script aliases (`hef-ingest`, `hef-review`) may be
|
||||
declared too for an installed checkout, but `python -m` is the documented path.
|
||||
- `media/` (download target, §6.3) is added to `.gitignore`. Media never enters
|
||||
git — the repo holds metadata + pointers only (design §4).
|
||||
|
||||
---
|
||||
|
||||
## 5. Mechanical tagging (`tools/probe.py`, `tools/mediatools.py`, `tools/tagging.py`, `tools/licensing.py`)
|
||||
|
||||
All of §5's "mechanical / automatic" fields, set from the downloaded file and the
|
||||
origin metadata.
|
||||
|
||||
### 5.1 `mode`, `duration_s`, `resolution` — from `ffprobe`
|
||||
|
||||
`tools/probe.py` wraps:
|
||||
|
||||
```
|
||||
ffprobe -v quiet -print_format json -show_format -show_streams <file>
|
||||
```
|
||||
|
||||
and parses the JSON into a small `Probe` result (streams + format). `tools/tagging.py`
|
||||
derives:
|
||||
|
||||
- **`mode`** — from which *meaningful* streams exist:
|
||||
- audio stream present, no real video → `"audio"`
|
||||
- real video stream present, no audio → `"video"`
|
||||
- both → `"av"`
|
||||
- **Cover-art guard:** an embedded cover image in an audio file shows up as a
|
||||
video stream (`mjpeg`/`png`, `disposition.attached_pic == 1`, often 1 frame).
|
||||
These are **excluded** when deciding `mode`, so an MP3 with album art is
|
||||
correctly `"audio"`, not `"av"`. This guard is a tested case.
|
||||
- **`duration_s`** — `round(float(format.duration))`, falling back to the longest
|
||||
stream `duration` if `format.duration` is absent. Integer (schema requires int ≥ 0).
|
||||
- **`resolution`** — `"{width}x{height}"` from the chosen video stream; `""` for
|
||||
audio-only (schema default).
|
||||
|
||||
`tools/probe.py` takes the subprocess runner as an injectable dependency
|
||||
(default: `subprocess.run`); tests feed canned `ffprobe` JSON and never shell out.
|
||||
|
||||
### 5.2 `dominant_color` — optional, deferred (computed from the video only on demand)
|
||||
|
||||
> **Design-change note (2026-06-04):** `dominant_color`'s only consumer was the
|
||||
> **procedural side walls** (design §7): hue for the three non-primary walls. The
|
||||
> installation is moving to a **single panoramic projector that spans all three
|
||||
> walls showing the real (nature-video) content**, which removes the procedural
|
||||
> side-wall renderer — and with it the need to *compute* a dominant color. So this
|
||||
> spec **demotes `dominant_color` from a core mechanical-tagging requirement to an
|
||||
> optional, opt-in computation.** This is a knock-on of a larger design shift that
|
||||
> primarily affects sub-projects 3 and 5 and the design spec (see §11 follow-up).
|
||||
|
||||
Concretely:
|
||||
|
||||
- The `dominant_color` **field stays** in the schema (it is already there; default
|
||||
`""`) — no schema change either way.
|
||||
- Ingest does **not** compute it by default. An opt-in flag
|
||||
(`--dominant-color`) enables computation for `video`/`av` records, for the case
|
||||
the operator still wants it (e.g. a later ambient/lighting use).
|
||||
- **When enabled, it is `ffmpeg`-only — never an image library.** Extract a
|
||||
representative mid-segment frame and reduce it to one hex color:
|
||||
```
|
||||
ffmpeg -v quiet -ss <mid> -i <file> -vf "thumbnail,palettegen=max_colors=1" -frames:v 1 -f rawvideo -pix_fmt rgb24 -
|
||||
```
|
||||
The 3 output bytes are the dominant palette color → `"#rrggbb"` (a `scale=1:1`
|
||||
average-color path is the fallback if `palettegen` is unavailable). This honors
|
||||
the operator's "no video manipulation needed" steer: no Pillow, no new dep, and
|
||||
the work isn't done at all unless explicitly requested.
|
||||
|
||||
Audio-only records always keep `dominant_color=""`.
|
||||
|
||||
### 5.3 `license` / `attribution` / `source_*` — from origin (`tools/licensing.py`)
|
||||
|
||||
Each fetcher resolves origin metadata; `tools/licensing.py` normalizes it to the
|
||||
`hef.catalog` vocab (`LICENSES = {public_domain, cc0, cc_by, cc_by_nc}`):
|
||||
|
||||
- Map CC license URLs/identifiers → `cc0` | `cc_by` | `cc_by_nc`; explicit
|
||||
public-domain / "no known copyright" → `public_domain`.
|
||||
- For `cc_by`/`cc_by_nc` (the `ATTRIBUTION_LICENSES`), build the required
|
||||
`attribution` string (creator + license name/URL) — `validate()` already rejects
|
||||
these licenses with empty `attribution`, so the normalizer **must** produce it.
|
||||
- Anything that doesn't map to an allowed license is **rejected at ingest** with a
|
||||
clear error (the piece is not defensible per design §8's license stance) — it is
|
||||
never written as an invalid record.
|
||||
|
||||
This is the per-archive "license/attribution/source from origin" tagging; it is
|
||||
unit-tested with sample metadata payloads per archive.
|
||||
|
||||
---
|
||||
|
||||
## 6. Ingest (`tools/ingest/`)
|
||||
|
||||
### 6.1 The `Candidate` + `Fetcher` seam
|
||||
|
||||
Every archive differs in API but funnels into one shape. `tools/ingest/base.py`:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class Candidate:
|
||||
source_archive: str # origin label, e.g. internet_archive | librivox | nasa (§6.4)
|
||||
source_url: str # human/landing URL recorded in the record
|
||||
media_url: str # direct download URL of the chosen file
|
||||
title: str
|
||||
license: str # normalized (tools.licensing) -> hef LICENSES vocab
|
||||
attribution: str # "" unless the license requires it
|
||||
suggested_id: str # stable id derived from archive + archive-identifier
|
||||
media_ext: str # file extension for the download target
|
||||
description: str = "" # free text used by drafting signals; lands in notes
|
||||
```
|
||||
|
||||
```python
|
||||
class Fetcher(Protocol):
|
||||
archive: str # the source_archive label
|
||||
def search(self, query: str, *, limit: int) -> list[Candidate]: ...
|
||||
def resolve(self, identifier: str) -> Candidate: ... # one item by id/URL
|
||||
```
|
||||
|
||||
Fetchers receive an HTTP client (`tools/http.py`) by injection so tests feed canned
|
||||
API responses. Fetchers do **not** download or probe — they only resolve metadata.
|
||||
|
||||
### 6.2 The ingest pipeline
|
||||
|
||||
`tools/ingest/base.py` provides `ingest_candidate(candidate, *, catalog_path, media_root, proposer, prober, downloader)`:
|
||||
|
||||
1. **Dedupe.** `load_catalog` + `validate_catalog`; if `candidate.suggested_id`
|
||||
already exists, skip (log "already in catalog") — idempotent re-runs.
|
||||
2. **Download.** Fetch `candidate.media_url` → `<media_root>/<archive>/<id>.<ext>`
|
||||
(see §6.3). Skip the download if the file already exists.
|
||||
3. **Mechanical tag.** `prober` → `mode`, `duration_s`, `resolution`; `mediatools`
|
||||
→ `dominant_color` (video/av only). `licensing` already normalized
|
||||
`license`/`attribution` on the `Candidate`.
|
||||
4. **Draft coordinates.** `proposer.propose(signals)` → `Draft(coordinate, rationale)` (§7).
|
||||
5. **Build the record.** Construct a `hef.catalog.Record` from the above; the
|
||||
`proposed`/`reviewed_at=None` shape comes from the defaults (§2). `notes` gets
|
||||
`candidate.description`.
|
||||
6. **Validate + append.** `validate(record)`, then re-check uniqueness, then
|
||||
`append_record(record, catalog_path)`.
|
||||
|
||||
Batch helper `ingest_search(fetcher, query, *, limit, …)` runs steps over each
|
||||
search hit. Everything network/subprocess is injected, so the pipeline is tested
|
||||
end-to-end with a fake fetcher + fake prober + fake downloader (no I/O).
|
||||
|
||||
### 6.3 Download / caching layout (settles a roadmap open decision)
|
||||
|
||||
- **Media root:** `--media-root DIR` / env `HEF_MEDIA_ROOT`, default `./media/`
|
||||
(gitignored).
|
||||
- **On-disk path:** `<media_root>/<source_archive>/<suggested_id>.<media_ext>`.
|
||||
- **`file_path` recorded in the record:** the path **relative to the media root**
|
||||
(e.g. `nasa/nasa-as08-14-2383.mp4`). Relative-to-root is portable: the ingest
|
||||
workstation and the Pi's mounted drive store the same tree under different
|
||||
mounts, and the **player (sub-project 3) joins `file_path` with its own drive
|
||||
mount**. `validate()` does not constrain `file_path` format, so this is
|
||||
compatible with the hand-authoring guide's absolute-path examples.
|
||||
|
||||
> **Open decision (logged):** relative-to-media-root vs. absolute `file_path`. This
|
||||
> spec recommends **relative-to-root** for portability between the tagging
|
||||
> workstation and the Pi. The USER_GUIDE currently shows absolute paths for
|
||||
> hand-authoring; both load fine. A doc note will reconcile them; the player spec
|
||||
> will define mount resolution. Reversible.
|
||||
|
||||
### 6.4 The six archives (all specified; phased delivery)
|
||||
|
||||
License normalization and access per pool (design §8 maps these to the axes):
|
||||
|
||||
| Archive (`source_archive`) | Access | Typical license | Auth | Notes / risk |
|
||||
|---|---|---|---|---|
|
||||
| `internet_archive` (incl. Prelinger) | Metadata API `archive.org/metadata/<id>`; direct file URLs | `public_domain` / CC (per item `licenseurl`/`rights`) | none | Ambiguous "no known copyright" → `public_domain`, flagged in `notes` for review |
|
||||
| `librivox` | JSON API `librivox.org/api/feed/audiobooks` | `public_domain` (charter) | none | Reader credited in `notes`; attribution not required |
|
||||
| `nasa` | Images/video API `images-api.nasa.gov` | `public_domain` | none | NASA media guideline caveat: some items embed third-party content — flag in `notes` |
|
||||
| `musopen` | Musopen API / catalog | `public_domain` / CC | possible key | Access historically gated — may need an API key (secret, §9) |
|
||||
| `fma` (Free Music Archive) | Track/page resolution | CC (`cc_by`/`cc_by_nc`/`cc0`) | varies | **Highest risk:** the public FMA API has been deprecated/changed; fetcher resolves from a track URL + page metadata |
|
||||
| `freesound` | API v2 `freesound.org/apiv2` | `cc0` / `cc_by` / `cc_by_nc` | **token required** | API token is a secret (§9); uploader → `attribution` when license requires |
|
||||
|
||||
> **Recommended first-ship set:** the keyless, clean-license, stable-API pools —
|
||||
> **LibriVox, NASA, Internet Archive/Prelinger.** Defer the ones with an auth or
|
||||
> API-stability tax — **Freesound** (token), **Musopen** (possible gate),
|
||||
> **Free Music Archive** (deprecated API) — to a second pass. The `Fetcher` seam
|
||||
> means deferring them costs nothing structurally; the spec still defines all six.
|
||||
> *(Logged as an open decision for operator confirmation — see §11.)*
|
||||
|
||||
---
|
||||
|
||||
## 7. Coordinate drafting (`tools/drafting.py`)
|
||||
|
||||
The curatorial coordinates are a human act (design §11). The tool's job is to seed
|
||||
a **draft** to be reviewed, never to auto-tag authoritatively.
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class Draft:
|
||||
coordinate: Coordinate # hef.selection.Coordinate(left, right, dark, light)
|
||||
rationale: str # one line, cites the signal that drove the guess
|
||||
|
||||
@dataclass
|
||||
class Signals:
|
||||
title: str
|
||||
description: str
|
||||
source_archive: str
|
||||
mode: str
|
||||
duration_s: int
|
||||
|
||||
class Proposer(Protocol):
|
||||
def propose(self, signals: Signals) -> Draft: ...
|
||||
```
|
||||
|
||||
### 7.1 `HeuristicProposer` (the deterministic baseline)
|
||||
|
||||
A rule-based seed grounded in the §8 sourcing→axis map and the §2 rubric — **not
|
||||
an ML model** (honors design §11):
|
||||
|
||||
- **Archive priors** seed the brain plane: `librivox` → high `left`
|
||||
(spoken/verbal); `internet_archive`/Prelinger → high `left` (educational/
|
||||
industrial) by default; `musopen`/`fma` → high `right` (music); `nasa` → high
|
||||
`right` (wordless awe); `freesound` → high `right` (abstract field recordings).
|
||||
- **Keyword nudges** on title+description seed the mood plane: dark words
|
||||
(`storm`, `night`, `noir`, `requiem`, `minor`, `war`, `funeral`, `decay`, …)
|
||||
raise `dark`; light words (`sunrise`, `dawn`, `garden`, `spring`, `joy`, `hope`,
|
||||
`major`, `bright`, …) raise `light`.
|
||||
- Values **clamped to 0..4**.
|
||||
- `rationale` is one line naming the dominant signal, e.g.
|
||||
`"librivox spoken reading → strong left; 'storm' in title → dark"`.
|
||||
|
||||
Deterministic given `Signals` → fully unit-testable.
|
||||
|
||||
### 7.2 The assistant seam
|
||||
|
||||
`Proposer` is an interface. The `HeuristicProposer` is always-available and
|
||||
dependency-free. When a **session assistant** (e.g. Claude, or the operator) does
|
||||
the ingest, it can supply a considered coordinate + rationale through the *same*
|
||||
seam (a manual proposer that reads a per-candidate draft, or an LLM-backed
|
||||
proposer the operator wires in). Either way the record is written
|
||||
`review_status="proposed"` and is **still reviewed by a human**. The tool ships
|
||||
**no** ML dependency; the "assistant" is an optional, out-of-core seam.
|
||||
|
||||
---
|
||||
|
||||
## 8. Review CLI (`tools/review.py` + `tools/review_cli.py`)
|
||||
|
||||
### 8.1 The transition core (pure, tested) — `tools/review.py`
|
||||
|
||||
```python
|
||||
def proposed_records(records) -> list[Record]:
|
||||
"""Records still awaiting review."""
|
||||
|
||||
def approve(record, *, reviewed_at, coordinate=None) -> Record:
|
||||
"""Return an approved copy: review_status='approved', reviewed_at set,
|
||||
optionally overriding the four coordinates with a human correction.
|
||||
The returned record is re-validated by the caller before save."""
|
||||
```
|
||||
|
||||
- `reviewed_at` is an ISO-8601 UTC timestamp string
|
||||
(`datetime.now(timezone.utc).isoformat()`), injected by the CLI so the core stays
|
||||
pure/testable. `Record.reviewed_at` is `Optional[str]` — no schema change.
|
||||
- `approve` does not mutate in place gratuitously; it produces the approved record
|
||||
and the CLI persists via load → replace-by-id → `save_catalog` (rewrite). At
|
||||
120–800 records a full rewrite is trivial and crash-resilient when done after
|
||||
each approval.
|
||||
|
||||
### 8.2 The interactive walk — `tools/review_cli.py`
|
||||
|
||||
`python -m tools.review_cli --catalog catalog/library.jsonl [--media-root ./media]`
|
||||
|
||||
1. `load_catalog` → `validate_catalog` (fail loudly on a corrupt catalog before editing).
|
||||
2. For each `proposed` record, show:
|
||||
- `id`, `title`, `source_archive`, `source_url`, `license`/`attribution`
|
||||
- `mode`, `duration_s`, `resolution`, `dominant_color`
|
||||
- the **proposed coordinates** `left/right/dark/light` and the one-line `rationale`
|
||||
- a **preview frame**: for `video`/`av`, extract a representative frame (reusing
|
||||
the §5.2 mid-segment frame) to a temp PNG and open it with the OS viewer
|
||||
(`open` on macOS / `xdg-open` on Linux); for `audio`, render a waveform
|
||||
thumbnail (`ffmpeg … showwavespic`) and/or offer optional 10-second playback
|
||||
via `ffplay`. The preview resolves `file_path` against `--media-root`.
|
||||
3. Prompt: **[a]ccept · [e]dit coords · [s]kip · [q]uit**.
|
||||
- **accept** → `approve(record, reviewed_at=now)` (keep coords).
|
||||
- **edit** → prompt for new `left/right/dark/light` (and optionally a new
|
||||
`rationale`), then `approve(..., coordinate=corrected)`.
|
||||
- **skip** → leave `proposed`, advance.
|
||||
- **quit** → save and exit.
|
||||
4. Persist after each approval (rewrite via `save_catalog`), so an interrupted
|
||||
session keeps its progress.
|
||||
|
||||
The preview/prompt shell is thin; the **tested** logic is the §8.1 transition core
|
||||
(deterministic, no terminal, no subprocess). Preview rendering reuses
|
||||
`tools/mediatools.py` and is exercised only by opt-in integration tests.
|
||||
|
||||
---
|
||||
|
||||
## 9. Dependencies, configuration, secrets
|
||||
|
||||
- **System binaries:** `ffmpeg` + `ffprobe` (subprocess). Required for mechanical
|
||||
tagging, dominant color, and previews. The dominant-color recommendation (§5.2)
|
||||
keeps the dep surface to *just* these two — no Python image library in the core.
|
||||
Pillow is an optional documented fallback only.
|
||||
- **HTTP:** stdlib `urllib.request` via `tools/http.py` (timeout, small retry,
|
||||
user-agent) — no `requests` dependency, preserving the sub-project-1 ethos.
|
||||
`internetarchive`/other archive SDKs are explicitly avoided in favor of the
|
||||
documented JSON APIs.
|
||||
- **Config:** `--catalog` (default `catalog/library.jsonl`), `--media-root` /
|
||||
`HEF_MEDIA_ROOT` (default `./media/`, gitignored).
|
||||
- **Secrets (wgl hard rule — never in catalog, code, or transcript):** Freesound
|
||||
API token via `FREESOUND_API_TOKEN` (env or macOS Keychain); any Musopen/FMA key
|
||||
likewise. Referenced by name only; never written into a record or logged.
|
||||
- **Testing:** `pytest` (already a dep). The unit suite stays **hermetic** —
|
||||
`ffprobe`/`ffmpeg`/HTTP are injected and faked; no network, no binaries needed to
|
||||
run the core tests. Integration tests that actually invoke `ffprobe`/`ffmpeg` are
|
||||
opt-in and **skipped** when the binaries are absent.
|
||||
- **pyproject:** add `tools` to `[tool.setuptools] packages`; optionally add
|
||||
`[project.scripts]` (`hef-ingest`, `hef-review`) and a `[project.optional-dependencies]`
|
||||
`tools` extra (empty unless Pillow is later adopted).
|
||||
|
||||
---
|
||||
|
||||
## 10. Testing strategy & done-criteria
|
||||
|
||||
Mirrors sub-project 1's TDD discipline (test-first, stdlib, hermetic).
|
||||
|
||||
**Unit tests (hermetic):**
|
||||
|
||||
1. `hef.catalog.validate_catalog` — accepts unique ids; raises naming a duplicate;
|
||||
(`index_by_id` round-trip if added).
|
||||
2. **Mechanical tagging** — canned `ffprobe` JSON → expected `mode`/`duration_s`/
|
||||
`resolution`; the **cover-art `attached_pic` guard** keeps an art-bearing MP3 at
|
||||
`mode="audio"`.
|
||||
3. **Dominant color (opt-in path)** — with the flag on, canned ffmpeg `rgb24`
|
||||
bytes → expected `"#rrggbb"`; with the flag off (default), `dominant_color`
|
||||
stays `""`; audio-only → `""`.
|
||||
4. **License normalization** — sample per-archive metadata → correct `license`;
|
||||
`cc_by`/`cc_by_nc` produce non-empty `attribution`; unmappable license is
|
||||
rejected.
|
||||
5. **Fetchers** — canned API JSON via a fake HTTP client → `Candidate` fields and
|
||||
stable `suggested_id` derivation (one test per archive that ships).
|
||||
6. **Drafting** — `HeuristicProposer`: archive priors, keyword nudges, 0..4
|
||||
clamping, one-line rationale content.
|
||||
7. **Review transition** — `proposed → approved` sets `reviewed_at` + status; edit
|
||||
overrides coordinates; skip leaves `proposed`; result re-validates and
|
||||
round-trips through `save_catalog`/`load_catalog`.
|
||||
8. **End-to-end (mocked)** — fake fetcher + fake prober + fake downloader +
|
||||
`HeuristicProposer` → a `proposed` record appended; the review core approves it;
|
||||
`load_catalog` + `validate_catalog` pass; `select(..., approved_only=True)` can
|
||||
pick it.
|
||||
|
||||
**Opt-in integration tests** (skipped without binaries): real `ffprobe` on a tiny
|
||||
fixture clip → `mode`/`duration`/`resolution`; real dominant-color extraction.
|
||||
|
||||
**Done when (from ROADMAP §2):**
|
||||
|
||||
- You can run ingest against a source and get `proposed` records.
|
||||
- You can review them to `approved`.
|
||||
- `load_catalog` (and `validate_catalog`) validate the result.
|
||||
- Unit tests on mechanical tagging and the review state transitions pass, and the
|
||||
full suite is green from the repo root inside `.venv`.
|
||||
|
||||
---
|
||||
|
||||
## 11. Open decisions (recommended defaults; logged for operator)
|
||||
|
||||
None block writing the implementation plan; each has a recommended default the
|
||||
plan will assume unless the operator overrides.
|
||||
|
||||
1. **Dominant-color dependency** — *Decided (operator, 2026-06-04):* `dominant_color`
|
||||
is **optional / opt-in** and, when computed, **ffmpeg-only — no image lib**,
|
||||
because the move to a single panoramic projector showing real nature video
|
||||
removes the procedural side walls that were its only consumer (§5.2).
|
||||
2. **`file_path` style** — *Recommend:* relative-to-`media-root` for portability;
|
||||
reconcile the USER_GUIDE's absolute examples with a doc note; player spec
|
||||
defines mount resolution.
|
||||
3. **Which archives ship first** — *Decided (operator, 2026-06-04):* **LibriVox,
|
||||
NASA, Internet Archive/Prelinger** first; defer Freesound, Musopen, FMA (auth /
|
||||
API-stability tax). (Nature-video content focus weights NASA/IA; the audio
|
||||
pools still serve the catalog's audio modes.)
|
||||
4. **CLI surface** — *Recommend:* two `python -m` entry points (`tools.ingest_cli`,
|
||||
`tools.review_cli`), matching the no-install repo convention; console-script
|
||||
aliases optional.
|
||||
5. **`index_by_id` placement** — *Recommend:* add it to `hef.catalog` alongside
|
||||
`validate_catalog` (the player wants by-id lookup too).
|
||||
|
||||
**Deliberately deferred (YAGNI, no schema change now):** a `source_id`
|
||||
archive-identifier field, `media_sha256` integrity, an `ingested_at` timestamp, and
|
||||
any LLM-backed proposer in the core. All are future, additive extensions; none are
|
||||
needed for the done-criteria.
|
||||
|
||||
**Design-spec follow-up (beyond this sub-project):** the operator's 2026-06-04
|
||||
decision to use a **single panoramic projector spanning all three walls, showing
|
||||
real nature video**, supersedes the design spec's §6 (four projectors) / §7
|
||||
(procedural side-wall renderer) and reshapes sub-projects **3** (player drives one
|
||||
pano output, not primary + side feed) and **5** (procedural side walls
|
||||
**dropped**). The design spec
|
||||
(`2026-06-04-human-experience-filter-design.md` §6/§7) and ROADMAP were updated
|
||||
this session to reflect this. The only element affecting the sub-project-2 spec
|
||||
itself is the now-optional `dominant_color` (§5.2).
|
||||
|
||||
---
|
||||
|
||||
## 12. Relationship to the other sub-projects
|
||||
|
||||
- **Consumes** sub-project 1 (`hef.catalog`, `hef.selection`) — the only required
|
||||
change is the additive `validate_catalog` (+ optional `index_by_id`), §3.
|
||||
- **Feeds** sub-project 3 (the Pi player): a populated, `approved` catalog the
|
||||
player drives to the (now single, panoramic) projector. The player resolving
|
||||
`file_path` against its drive mount (§6.3) is a player-spec concern.
|
||||
- **Side walls (sub-project 5)** are likely **dropped** under the pano-projector
|
||||
design change (§11 follow-up); `dominant_color`, their only input, is
|
||||
correspondingly optional here (§5.2).
|
||||
- **Settles** two roadmap open decisions (download/caching layout §6.3; ingest
|
||||
language = Python under `tools/`, §4).
|
||||
@@ -0,0 +1,374 @@
|
||||
# Human Experience Filter — Machine-Altered Perception (Design Revision)
|
||||
|
||||
**Date:** 2026-06-05
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Supersedes:** the thesis (§1), selection model (§3), tagging division (§5), and
|
||||
sourcing strategy (§8) of
|
||||
[`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md),
|
||||
and substantially expands its control panel (§2 selector) and hardware (§6). The
|
||||
coordinate model (§2 axes) and the single-panoramic-projector display are
|
||||
**preserved**.
|
||||
|
||||
> **Why this revision exists.** The 2026-06-04 design framed the piece as an
|
||||
> *experience filter over found human artifacts* ("curation is the artwork; not
|
||||
> algorithmically generated filler"). This revision **changes the artistic
|
||||
> thesis** to one about **how humans interact with machines and how that interaction
|
||||
> reshapes their nervous systems** — and in doing so makes machine alteration of
|
||||
> the imagery the *subject* of the piece rather than a betrayal of it.
|
||||
|
||||
---
|
||||
|
||||
## 1. What this is (the new thesis)
|
||||
|
||||
A single-viewer immersive installation about **a machine reshaping your
|
||||
perception — for better and for worse.**
|
||||
|
||||
One person sits in a chair in a small room. A single panoramic projector wraps the
|
||||
walls they face. The source is **neutral**: real, calm, public-domain nature
|
||||
footage that sits at the *center* of the experience-space — neither analytical nor
|
||||
emotional, neither bright nor dark. The viewer turns knobs on a wooden control
|
||||
panel, and **the machine alters that neutral reality toward whatever they dial
|
||||
in.** The piece is the felt, bodily experience of a machine bending your inner
|
||||
state — and crucially it bends **both ways**: it can soothe you or disturb you,
|
||||
clarify the world or manipulate it, name your feelings or colonize them.
|
||||
|
||||
You don't read that thesis on a wall label. You feel it in your nervous system as
|
||||
your own hand moves the knobs.
|
||||
|
||||
Design constraints (carried from the original, one relaxed):
|
||||
|
||||
- **One viewer, one room, all-local at runtime.** No networking, no streaming, no
|
||||
multi-user, no session recording. (All AI alteration happens **offline at
|
||||
authoring time**; the room plays pre-built media + cheap runtime grading.)
|
||||
- **Real footage as the substrate.** The base library is real public-domain
|
||||
nature footage. The machine *alters* it — it does not invent from nothing. This
|
||||
preserves real-world motion/composition (and some lineage to the original
|
||||
found-media thesis) while making the alteration the point. *(This relaxes the
|
||||
original "no generated content" rule: generation/alteration is now the
|
||||
subject.)*
|
||||
- **Built to grow.** A small, deliberate base library; the data model and tooling
|
||||
designed to expand.
|
||||
|
||||
---
|
||||
|
||||
## 2. What changes vs. 2026-06-04
|
||||
|
||||
| Area | 2026-06-04 | This revision |
|
||||
|---|---|---|
|
||||
| Thesis (§1) | filter over found human artifacts | **machine reshapes perception, good & bad** |
|
||||
| Content | large found library pre-matched to coordinates | **small NEUTRAL base library, machine-altered to the knob state** |
|
||||
| Engine (§3) | nearest-match **lookup** in a tagged catalog | **transform from neutral** (knobs = a transformation, not a lookup) |
|
||||
| Sourcing (§8) | find media that already matches each cell | source **neutral** clips; the machine produces the variants |
|
||||
| Control panel (§2) | 4-way mode + 4 knobs | **7-way content dial + 4 experience knobs + volume + brightness**, on a tactile wooden panel |
|
||||
| Display | single pano projector | **unchanged** |
|
||||
| Coordinate axes (§2) | Left/Right/Dark/Light, two planes | **unchanged** (reinterpreted as transform targets) |
|
||||
|
||||
---
|
||||
|
||||
## 3. Coordinate model (preserved, reinterpreted)
|
||||
|
||||
The four experience knobs and two planes from the 2026-06-04 §2 are unchanged:
|
||||
|
||||
- **Brain plane** = Left × Right. Left = analytical / verbal / structured. Right =
|
||||
artistic / emotional / abstract. Both high → "whole brain."
|
||||
- **Mood plane** = Dark × Light. Dark = somber / heavy. Light = uplifting / serene.
|
||||
|
||||
Each knob is `0–4`. What changes is the *meaning of a knob position*: it is no
|
||||
longer "find the catalog piece nearest this coordinate" but **"how hard the machine
|
||||
pushes the neutral base toward this pole."** `(2,2,2,2)` is the neutral origin —
|
||||
the un-altered base clip.
|
||||
|
||||
---
|
||||
|
||||
## 4. The alteration engine (replaces the §3 selection algorithm)
|
||||
|
||||
The engine's job changes from **choosing** a clip to **altering** one. Given a
|
||||
neutral base clip and a knob vector `(left, right, dark, light)`, it applies a
|
||||
**transformation toward each active pole** and composites the result.
|
||||
|
||||
### 4.1 Per-axis alteration
|
||||
|
||||
| Pole | What the machine does | How |
|
||||
|---|---|---|
|
||||
| **Left** (analytical/verbal) | **imposes analytical structure** on the scene — annotation, measurement, labels, grids, tracking boxes, data HUD; the machine *dissecting* the world into data | **overlay** layer composited on top |
|
||||
| **Right** (artistic/emotional) | **dissolves realism** toward painterly / dreamlike / abstract | **generative video-to-video (v2v)** restyle of the pixels |
|
||||
| **Dark** (somber/heavy) | drains toward the cool/melancholy/night grade (see §5) | **color grade** (deterministic) |
|
||||
| **Light** (uplifting/serene) | warms toward the bright/serene grade (see §5) | **color grade** (deterministic) |
|
||||
|
||||
### 4.2 The composition rule
|
||||
|
||||
The four axes are **two kinds of operation** that **layer** rather than blend-and-
|
||||
cancel:
|
||||
|
||||
- **Substrate transforms** — Right (v2v restyle) + Dark + Light (color grades).
|
||||
They alter the **pixels** and blend with each other.
|
||||
- **Overlay** — Left (annotation / labels / HUD). It composites **on top** of
|
||||
whatever the substrate is.
|
||||
|
||||
So Left and Right are **not** mutually-exclusive opposites; they stack on different
|
||||
layers. This single rule covers the whole `5×5×5×5` space:
|
||||
|
||||
- **Max Left + Max Right** = maximally dissolved dreamlike substrate **with the
|
||||
machine's labels on the feelings themselves** — *"awe," "grief," "detected:
|
||||
longing 0.82,"* tracking boxes around emotional content. This is the "whole
|
||||
brain" corner (§3): the machine that both **induces** emotion and
|
||||
**names/quantifies** it — affective computing made visceral. (Good/bad duality:
|
||||
is being told your own feeling clarifying, or invasive?)
|
||||
- **Dark + analytical** = a drained, melancholy scene with cold measurement
|
||||
overlaid.
|
||||
- **Light + emotional** = a warm, dreamlike wash, unlabeled.
|
||||
|
||||
### 4.3 Where each transform runs
|
||||
|
||||
> **Reconciled (2026-06-07, session 0009):** the Left HUD is a **runtime overlay**
|
||||
> driven by an authored annotation track + per-language string tables (text shaped
|
||||
> live); the Right axis selects a **discrete pre-baked** flow-stabilized variant
|
||||
> (not a continuous blend). See
|
||||
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
|
||||
> §1, which supersedes the session-0007 baked-HUD / 5×5-grid proposal.
|
||||
|
||||
- **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade
|
||||
and the Left analytical overlay. These are cheap (LUT/curves + luma key, and
|
||||
text/graphics compositing) and can move continuously with the knob.
|
||||
- **Pre-baked offline (the only AI cost):** the Right generative v2v restyle.
|
||||
Generative v2v cannot run real-time on a Pi, so a small set of restyled variants
|
||||
is rendered offline and selected/blended at runtime.
|
||||
|
||||
> **Crucial consequence — the Left labels are a RUNTIME OVERLAY, never baked into
|
||||
> the v2v pixels.** This is what makes multilingual support nearly free (§10).
|
||||
|
||||
---
|
||||
|
||||
## 5. Mood-axis color grade
|
||||
|
||||
The Dark↔Light knob is a three-stop color ramp whose **center is the identity** —
|
||||
the raw, ungraded footage, no color transform applied. Turning toward a pole grades
|
||||
*away* from the raw clip. Hue carries emotional valence; the **negative-space
|
||||
value** (a luma-keyed grade — the brightest/darkest regions pushed toward
|
||||
white/black) carries the literal light/dark reading, so the image *is* lighter or
|
||||
darker even before the hue registers.
|
||||
|
||||
| Mood knob | Hue | Negative space | Feeling |
|
||||
|---|---|---|---|
|
||||
| **Light** | warm **yellow** | → **white** | uplifting, serene, airy |
|
||||
| **Center** | **none — raw, ungraded video** | unchanged | the un-altered base |
|
||||
| **Dark** | cool **blue** | → **black** | melancholy, somber, night, cold |
|
||||
|
||||
*(Red/alarm was considered for Dark and set aside: red reads as threat/alarm, not
|
||||
the intended melancholy. Dark is the cool, mournful blue. The center applies no
|
||||
grade at all — it is simply the raw clip.)*
|
||||
|
||||
---
|
||||
|
||||
## 6. Content model — the 7-way content dial
|
||||
|
||||
The 2026-06-04 4-way selector (None/Audio/Video/A+V) expands to **seven positions**,
|
||||
letting the viewer choose channel **and audio source**:
|
||||
|
||||
| Position | Audio source | Video |
|
||||
|---|---|---|
|
||||
| **Off** | — | black |
|
||||
| **White Noise** | generated white/pink noise (not from catalog) | black |
|
||||
| **Music** | public-domain classical (Musopen pool) | black |
|
||||
| **Audio Track** | the nature clip's **own** audio | black |
|
||||
| **Video** | — | the (altered) nature video |
|
||||
| **Music + Video** | public-domain classical (Musopen pool) | the (altered) nature video |
|
||||
| **Audio Track + Video** | the nature clip's own audio | the (altered) nature video |
|
||||
|
||||
Notes:
|
||||
|
||||
- "Music" (classical) and "Audio Track" (the clip's own field audio) are now
|
||||
**distinct sources**; "White Noise" is **generated** at runtime, not a catalog
|
||||
record. This touches the catalog/`mode` model (the 2026-06-04 §4 schema), which
|
||||
must represent audio *source* in addition to channel.
|
||||
- **Off** is the void/rest state: black walls, silence.
|
||||
- Audio alteration (steering the *sound* by knob, parallel to the video) is **out
|
||||
of scope for this revision** — the experience knobs alter video; audio is chosen
|
||||
by source only. (Future work.)
|
||||
|
||||
---
|
||||
|
||||
## 7. Intensity controls
|
||||
|
||||
Two knobs, orthogonal to the four experience axes, exist so a viewer can take the
|
||||
experience at **whatever intensity they want**:
|
||||
|
||||
- **Volume** — audio output level.
|
||||
- **Brightness** — projector output luminance.
|
||||
|
||||
These are pure output levels, distinct from the Dark/Light *mood grade* (which
|
||||
changes hue and tone, not raw output). Brightness is "how much light hits the
|
||||
wall"; Light is "how the world is colored."
|
||||
|
||||
---
|
||||
|
||||
## 8. Content sourcing — neutral base library
|
||||
|
||||
The library is now **small and neutral**. Instead of hunting for footage that
|
||||
matches 625 coordinate cells, source a deliberate set of **neutral, calm,
|
||||
real public-domain nature clips** (centered on all four axes) and let the machine
|
||||
produce every variant.
|
||||
|
||||
- **Reuse sub-project 2** (the built ingest/tagging/review tooling) to pull
|
||||
neutral public-domain nature footage and populate mechanical fields.
|
||||
- **License stance** unchanged: prefer Public Domain / CC0; record license +
|
||||
source per clip. Altered/generated variants record the model and that they are
|
||||
machine-derived.
|
||||
- **Size:** dozens of neutral base clips (not hundreds), each yielding many
|
||||
altered variants. Short, seamless loops (≈20–60 s) suit a looping installation
|
||||
and are the v2v models' native sweet spot.
|
||||
|
||||
---
|
||||
|
||||
## 9. Economics (why this is feasible)
|
||||
|
||||
At the original scale (120–800 pieces × ~10-min segments) AI involvement was
|
||||
~$15k–45k+ and not viable. Under this design it drops to **a few hundred to a few
|
||||
thousand dollars**, because:
|
||||
|
||||
- The library is small (generate-to-order, not find-to-cover).
|
||||
- Pieces are short seamless loops (cheaper **and** higher-quality for the models).
|
||||
- Only the **Right** transform uses paid generative v2v; Dark/Light/Left are free
|
||||
deterministic runtime operations.
|
||||
|
||||
Indicative offline v2v build (2026 rates — Kling 3.0 / Sora 2 Standard ≈ $0.10/s,
|
||||
Veo 3.1 Lite / Runway Turbo ≈ $0.05/s; ×~4 retry yield): **~$300–3k** for a small
|
||||
base library × a handful of restyle variants. A flat authoring subscription
|
||||
(Runway $28–76/mo, Kling from ~$6/mo) may be cheaper than per-second API for a
|
||||
one-time build.
|
||||
|
||||
---
|
||||
|
||||
## 10. Accessibility, i18n, and the translation-cost finding
|
||||
|
||||
> **Reconciled (2026-06-07, session 0009):** the near-free-i18n path is kept — the
|
||||
> Left HUD is a runtime overlay (authored annotation track + per-language string
|
||||
> tables, shaped live via Pango/HarfBuzz on the Pi). The session-0007 baked-HUD
|
||||
> reversal is **not** adopted. See
|
||||
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
|
||||
> §1.
|
||||
|
||||
The piece is operable **blind, in the dark, in your language**, via four redundant
|
||||
channels on the control panel: **touch** (engraved symbol shape), **low-light
|
||||
color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local*
|
||||
speaker, separate from the main audio system).
|
||||
|
||||
Because "the human experience from a communication perspective is not universal" is
|
||||
itself on-thesis, the piece supports **many languages**. The cost of that hinges on
|
||||
one architectural decision:
|
||||
|
||||
- **❌ Labels baked into the v2v render** → re-render every label-bearing variant
|
||||
per language → video cost **× number of languages** (≈ $9k raw / $27–45k with
|
||||
retries for ~50 languages). A budget landmine.
|
||||
- **✅ Labels as a runtime overlay** (which §4.2 already requires) → the v2v
|
||||
substrate is rendered **once**, language-agnostic; a new language is a translated
|
||||
string table + font + TTS voice. **Marginal video cost per language ≈ $0.**
|
||||
|
||||
So multilingual support costs only cheap text work: machine-translating a small
|
||||
label/script vocabulary into ~100 languages is a few dollars; optional human review
|
||||
of a curated tier is ~$0.10/word (~$50–200/language); TTS for the read-aloud button
|
||||
is pennies per language. **Net: supporting *every* language is essentially free on
|
||||
the expensive part, provided labels stay a runtime overlay.**
|
||||
|
||||
---
|
||||
|
||||
## 11. The physical control panel
|
||||
|
||||
A **wood** panel. Each control has a **tactile engraved symbol** below it —
|
||||
distinguishable by fingertip in the dark, and shaped to evoke the control's
|
||||
meaning — with **braille** beneath, a **dim color-coded LED** backlighting the
|
||||
engraving, and a **read-aloud button** that speaks the control's function on a
|
||||
small local speaker.
|
||||
|
||||
Layout principle: the **four experience knobs are two opposing pairs**, with the
|
||||
opposition encoded *in the touch* (angular vs flowing, heavy-down vs radiant-up).
|
||||
The **utility controls** use a more conventional AV shape-language so they are
|
||||
never confused with the experience four.
|
||||
|
||||
| Control | Engraved symbol | Touch | LED |
|
||||
|---|---|---|---|
|
||||
| **Left** (analytical) | parallel straight ridges / grid `≡` `▦` | flat, hard, orthogonal edges | — |
|
||||
| **Right** (emotional) | spiral / flowing wave `∿` | one continuous curve, no corners | — |
|
||||
| **Dark** (somber) | crescent / downward wedge `☾` `▼` | smooth solid mass, sinking | **blue** |
|
||||
| **Light** (serene) | radial sunburst `☀` | center with radial spikes | **yellow / white** |
|
||||
| **Content dial** (7) | per-position micro-icons: Off `○` · White-Noise `∴` · Music `♪` · Audio-Track `)))` · Video `▷` · Music+Video `♪▷` · A+V `▷)))` | detented clicks | neutral |
|
||||
| **Volume** | growing wedge `◁` | a slope that thickens | neutral |
|
||||
| **Brightness** | half-disc contrast `◐` | a circle, half raised | neutral |
|
||||
|
||||
Design notes:
|
||||
|
||||
- **Light vs Brightness** is the one collision trap (both are "bright"). Light =
|
||||
radial sunburst (a *mood*, glows the hue it imparts); Brightness = half-disc
|
||||
contrast (an *output level*) — distinct to finger and eye.
|
||||
- **LEDs equal the hue the knob imparts:** Dark glows blue, Light glows
|
||||
yellow/white, the mood center is neutral — tying the panel to the projection.
|
||||
- **Negative-space symbols:** symbols differ by gross **edge profile** (curve vs
|
||||
corner vs radial vs slope), not fine detail, so they read under a fingertip.
|
||||
|
||||
This substantially fleshes out **sub-project 4 (Arduino firmware / control panel)**,
|
||||
previously only sketched: 7 controls + a read-aloud button + LEDs + braille, with
|
||||
the Arduino reading positions and the read-aloud/LED behavior coordinated with the
|
||||
Pi.
|
||||
|
||||
---
|
||||
|
||||
## 12. Hardware (revises §6)
|
||||
|
||||
Unchanged from the 2026-06-04 single-pano revision except for the richer panel:
|
||||
|
||||
- **Arduino** — reads the 7-way content dial, 4 experience knobs, volume,
|
||||
brightness, and the read-aloud button; drives the panel LEDs; sends values to the
|
||||
Pi over USB serial. (Bigger than the original 5-value panel.)
|
||||
- **Raspberry Pi 5 (or mini-PC)** — holds the base library + altered variants,
|
||||
runs the alteration engine (runtime grading + overlay compositing + variant
|
||||
selection), drives the single panoramic projector, plays audio, and serves the
|
||||
read-aloud clips to the local panel speaker.
|
||||
- **Single panoramic projector** spanning the three walls — **unchanged**.
|
||||
- **Small local speaker** on/near the panel for the read-aloud button (separate
|
||||
from the main audio).
|
||||
|
||||
---
|
||||
|
||||
## 13. Roadmap impact
|
||||
|
||||
- **Sub-project 2 (ingest/tagging)** — reused to source the **neutral** base
|
||||
clips; coordinate *drafting* matters less (the base is neutral by selection), but
|
||||
mechanical tagging + license capture still apply; add a flag for "neutral base"
|
||||
vs "altered variant," and model audio *source*.
|
||||
- **Sub-project 3 (player)** — grows an **alteration engine**: runtime color grade
|
||||
(luma-keyed), runtime analytical-overlay compositor (multilingual, from string
|
||||
tables), generated white noise, and selection/blending of pre-baked v2v variants.
|
||||
- **Sub-project 4 (firmware/panel)** — grows from a 5-value panel to the full
|
||||
tactile/accessible panel in §11.
|
||||
- **New offline pipeline** — author neutral→restyled v2v variants (the only paid
|
||||
AI step) and the label/string tables + TTS per language.
|
||||
|
||||
---
|
||||
|
||||
## 14. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Pano resolution strategy** — v2v models output ~16:9 720p–1080p; the three-wall
|
||||
pano wants ultra-wide/high-res. Generate-at-max + upscale, native-ultrawide
|
||||
tooling, or accept the look?
|
||||
- **Negative-space luma key** — exact curve/mask for "brights→white, darks→black"
|
||||
on arbitrary nature footage.
|
||||
- **Transform composition order** — order of grade vs v2v substrate when several
|
||||
poles are active; how variant blending interpolates between pre-baked restyle
|
||||
levels.
|
||||
- **Neutral base count** — how many base clips for a satisfying launch.
|
||||
- **Language tier** — which languages get human-reviewed translation vs
|
||||
machine-only; default UI language and switching.
|
||||
- **Runtime grading performance** — confirm the Pi can do luma-keyed grade +
|
||||
overlay compositing at projector resolution/framerate (GPU shaders via mpv/ffmpeg).
|
||||
- **`approved`-only enforcement** in the player (carried from the original).
|
||||
|
||||
---
|
||||
|
||||
## 15. Out of scope (YAGNI)
|
||||
|
||||
- Networking / streaming at runtime — all alteration is offline; the room is local.
|
||||
- Runtime/on-demand generation (breaks all-local; slow; unbounded cost).
|
||||
- Audio alteration by knob (video-only this revision).
|
||||
- Multi-user / multi-viewer; session recording or analytics.
|
||||
- Automatic ML coordinate tagging — base clips are chosen neutral by hand.
|
||||
@@ -0,0 +1,242 @@
|
||||
# Human Experience Filter — Simulator Alteration Preview (Design)
|
||||
|
||||
**Date:** 2026-06-06
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Builds on:** the alteration engine of
|
||||
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
|
||||
(the "design" below) and the simulator scaffold of
|
||||
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md).
|
||||
**Revises the design:** §4.3 and §10 — the Left analytical HUD is **no longer a
|
||||
runtime overlay**; it is **baked into authored variant videos** (see §8 below).
|
||||
This is a deliberate trade of near-free multilingual support for authorial
|
||||
precision over the HUD.
|
||||
**Retires:** the simulator's selection-era "curator's X-ray" view and its
|
||||
`/api/select` + `/api/catalog/meta` endpoints (the selection model they
|
||||
visualize was superseded by the alteration model).
|
||||
|
||||
> **Why this exists.** Operator directive (session 0007): *build and design only
|
||||
> things that run in the simulator, and get the whole experience working the way
|
||||
> we like in the simulator before moving to hardware.* The slice-1 alteration
|
||||
> engine (PR #5) is pure logic with **no simulator surface** — you cannot yet turn
|
||||
> the experience knobs and *see* the result. This design brings the alteration
|
||||
> into the simulator so the look can be tuned and liked before any Pi/serial work.
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope
|
||||
|
||||
A browser **alteration preview**: turn the four experience knobs and the content
|
||||
dial, and see the neutral base footage altered toward the knob state, in real
|
||||
time, on a looping clip. The purpose is to **tune the look** of the filter and to
|
||||
**settle the knob→strength calibration by eye** (the open §3-vs-§4.2/§5 question
|
||||
from session 0006).
|
||||
|
||||
**In scope**
|
||||
- Live preview of the four experience knobs (Left/Right/Dark/Light) on a looping
|
||||
base clip.
|
||||
- **Dark/Light** rendered as a live, deterministic runtime color grade.
|
||||
- **Left/Right** rendered by selecting a **pre-baked authored variant clip** from
|
||||
a 5×5 grid and crossfading on change.
|
||||
- A **calibration panel** that adjusts the grade curve live; the chosen values are
|
||||
baked back into `player/alteration.py` defaults.
|
||||
- A **RenderPlan readout** (the project's "X-ray" honesty): always show the exact
|
||||
numbers the engine produced.
|
||||
- The content dial's **video on/off** behavior (so "Off" goes to black), driven by
|
||||
the existing `resolve_content`.
|
||||
- Placeholder variant generation so the mechanism is testable before the operator
|
||||
authors real videos.
|
||||
|
||||
**Out of scope (this slice)** — real generative v2v; serial input / the 3⇄4
|
||||
framing contract; audio playback (music / white-noise / audio-track); the Pi/mpv
|
||||
runtime renderer; the full transition/crossfade timing engine; multilingual label
|
||||
tables; catalog-model changes. These remain later roadmap slices.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture — Python-canonical, thin browser renderer
|
||||
|
||||
`player/alteration.py` stays the **single source of truth** for the alteration
|
||||
math (handbook §4.2 — deterministic core, thin I/O). The browser owns only
|
||||
*rendering*. Data flow:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Browser
|
||||
K[4 experience knobs +<br/>content dial +<br/>calibration sliders]
|
||||
V["<video> variant + canvas grade"]
|
||||
RO[RenderPlan readout]
|
||||
end
|
||||
subgraph FastAPI [simulator/app.py]
|
||||
EP["POST /api/alteration"]
|
||||
CL["GET /api/clips"]
|
||||
end
|
||||
ENG["player.alteration.plan_alteration(coord, calibration)"]
|
||||
K -- "debounced POST {controls, calibration}" --> EP
|
||||
EP --> ENG --> EP -- "RenderPlan {variant, grade}" --> V
|
||||
EP --> RO
|
||||
CL -- "base clip + variant manifest" --> V
|
||||
```
|
||||
|
||||
- The browser sends control/calibration changes (debounced) and receives a
|
||||
`RenderPlan`. Video filtering itself runs continuously in the browser; only a
|
||||
*plan recompute* makes a round-trip. On localhost these JSON round-trips are
|
||||
imperceptible.
|
||||
- "Bake the calibration winner in" = change the `Calibration` defaults in Python.
|
||||
Nothing in the browser is canonical.
|
||||
|
||||
---
|
||||
|
||||
## 3. The four axes — how each is rendered
|
||||
|
||||
| Axis | Engine output | Browser rendering |
|
||||
|---|---|---|
|
||||
| **Dark / Light** | `ColorGrade.tone` ∈ [−1, 1], center = identity | live color grade on the `<video>`: Light → warm/yellow + negative space toward white; Dark → cool/blue + negative space toward black; 0 = raw. (Confirmed direction in brainstorm.) |
|
||||
| **Left / Right** | `VariantRef(left, right)` | select the authored variant clip for `(left, right)`; **crossfade** when it changes; `(0,0)` → raw base. The analytical HUD (Left) and dreamlike restyle (Right) are **baked into the clip** (§8). |
|
||||
|
||||
Dark/Light is the only continuously-tunable axis in this slice; Left/Right is a
|
||||
discrete selection by coordinate.
|
||||
|
||||
---
|
||||
|
||||
## 4. The variant grid (per base clip)
|
||||
|
||||
Each base clip carries a 5×5 grid keyed by the two baked axes:
|
||||
|
||||
- Rows = **Left** (analytical / HUD), 0–4. Columns = **Right** (artistic /
|
||||
dreamlike restyle), 0–4.
|
||||
- `(0,0)` = the raw base clip — no authored file needed.
|
||||
- The other **24 cells are authored videos** (4 analytical-edge + 4 dreamlike-edge
|
||||
+ 16 combined core). This is **per base clip**; authoring load multiplies by the
|
||||
number of base clips (and by language, since the HUD is baked — §8).
|
||||
|
||||
The grid is **data**, not code: a manifest the simulator reads (see §6).
|
||||
|
||||
---
|
||||
|
||||
## 5. Engine reconciliation (`player/alteration.py`)
|
||||
|
||||
The slice-1 engine modeled Left/Right as continuous runtime layers
|
||||
(`AnalyticalOverlay.intensity`, `Restyle.blend`). Baked variants make that
|
||||
obsolete. Changes:
|
||||
|
||||
- **`RenderPlan`** becomes `{ grade: ColorGrade, variant: VariantRef }`.
|
||||
- `ColorGrade` — unchanged.
|
||||
- `VariantRef(left, right)` — new; identity selection by coordinate.
|
||||
- **Remove** `AnalyticalOverlay` and `Restyle` (their content is now baked into the
|
||||
variant clip) rather than leaving dead fields.
|
||||
- **`Calibration`** — a new frozen dataclass parameterizing the grade curve (e.g.
|
||||
`mood_center`, per-axis curve), defaulting to **today's exact behavior**
|
||||
(behavior-preserving). `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`.
|
||||
- **`player/state.py`** — the `CROSSFADE` trigger switches from "restyle changed"
|
||||
to "variant changed"; `LIVE_UPDATE` still covers a grade-only change. Video
|
||||
on/off fades unchanged.
|
||||
|
||||
These are framework-code changes that keep the engine pure and unit-tested; they
|
||||
do **not** bake any deployment-shape decision into the engine.
|
||||
|
||||
---
|
||||
|
||||
## 6. Data & endpoints
|
||||
|
||||
- **`simulator/clips.py`** (replaces `simulator/fixtures.py`): reads a base-clip +
|
||||
variant manifest. Each base clip: `{ id, title, base_file, license, source,
|
||||
variants: { "L,R": { file, model?, hud_lang? } } }`. Missing cells fall back to
|
||||
the raw base (and are flagged in the readout as "raw / unauthored").
|
||||
- **`POST /api/alteration`**: body `{ controls, calibration }` → `RenderPlan`
|
||||
(plus the `ContentResolution` from `resolve_content`, so the dial's video-on/off
|
||||
is honored). The endpoint calls the real `plan_alteration`.
|
||||
- **`GET /api/clips`**: the base-clip list + variant manifest for the active base.
|
||||
- Video files served as static assets from a sample-media directory.
|
||||
- **Removed**: `POST /api/select`, `GET /api/catalog/meta`.
|
||||
- `hef.selection` (the library, incl. `Coordinate`) is **untouched**; only the
|
||||
simulator's selection view/endpoints retire. (`ranked_candidates`, added for the
|
||||
old X-ray, may become unused by the simulator — noted, not removed here.)
|
||||
|
||||
---
|
||||
|
||||
## 7. Calibration tuning
|
||||
|
||||
The calibration panel exposes the grade-curve parameters (the two conventions in
|
||||
tension: `(2,2,2,2)`-centered vs the `value/4` reading currently implemented, plus
|
||||
curve shape). Changing them re-requests the plan and the footage responds live.
|
||||
When the operator is happy, the chosen values are written into
|
||||
`DEFAULT_CALIBRATION` in `player/alteration.py` and locked in by a unit test.
|
||||
|
||||
---
|
||||
|
||||
## 8. Baked HUD — the §4.3/§10 revision (accepted trade)
|
||||
|
||||
The design's §4.3/§10 kept the Left HUD a **runtime overlay** specifically so the
|
||||
v2v substrate is language-agnostic and multilingual support is near-free. This
|
||||
design **reverses that for the Left/Right plane**: the operator authors the HUD
|
||||
**into** the variant videos to get pixel-precise control over the HUD and the
|
||||
analytical↔feeling balance.
|
||||
|
||||
**Consequences (accepted):**
|
||||
- Multilingual support is **no longer near-free**: baked HUD text means a new
|
||||
language re-renders every label-bearing variant (the §10 "budget landmine").
|
||||
The piece is English-first; broad i18n is deferred/expensive.
|
||||
- The live Dark/Light grade is applied **on top** of the baked cell, so it **tints
|
||||
the baked HUD** too (a "Dark" mood cools/darkens the HUD). With baked overlays
|
||||
the grade cannot skip the HUD. Escape hatch: author HUD colors that survive
|
||||
grading, or (future) reinstate a runtime HUD layer.
|
||||
|
||||
The parent design's §4.3/§10 must be updated to point at this revision (a task for
|
||||
the implementation plan).
|
||||
|
||||
---
|
||||
|
||||
## 9. Bootstrapping before authored videos exist
|
||||
|
||||
The operator will author the 24 variants per base clip; none exist yet. So the
|
||||
mechanism is testable immediately, the build includes a **placeholder generator**:
|
||||
for at least one base clip, produce the 24 cells by burning the cell's `L,R` (and
|
||||
a stub HUD caption) into the base loop via ffmpeg. Real authored clips drop into
|
||||
the manifest with **no code change**.
|
||||
|
||||
---
|
||||
|
||||
## 10. Testing
|
||||
|
||||
- **`player/` unit tests:** `DEFAULT_CALIBRATION` is behavior-preserving vs the
|
||||
current helpers; the two calibration conventions produce the expected plans;
|
||||
`VariantRef` selection (incl. `(0,0)` → raw) and the `state.py` crossfade
|
||||
trigger on variant change.
|
||||
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
|
||||
`POST /api/alteration` returns the engine's plan for given controls/calibration;
|
||||
`GET /api/clips` returns the manifest; the removed endpoints are gone.
|
||||
- **No browser/E2E automation** this slice (manual visual tuning is the point);
|
||||
the rendering JS is kept thin and the engine logic stays in tested Python.
|
||||
|
||||
---
|
||||
|
||||
## 11. What ships
|
||||
|
||||
- `player/alteration.py` (parameterized `Calibration`, `VariantRef`, slimmed
|
||||
`RenderPlan`) + `player/state.py` crossfade-trigger update.
|
||||
- `simulator/clips.py` (variant manifest) replacing `fixtures.py`.
|
||||
- `simulator/app.py` new endpoints; selection endpoints removed.
|
||||
- `simulator/static/` rewritten as the Player preview (variant `<video>` +
|
||||
crossfade, live grade, calibration panel, RenderPlan readout, content dial).
|
||||
- Placeholder-variant generator + a sample base clip.
|
||||
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten;
|
||||
parent design §4.3/§10 pointer updated; `docs/ROADMAP.md` §3 updated.
|
||||
|
||||
---
|
||||
|
||||
## 12. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Grade vs baked HUD interaction** — accepted that the grade tints the HUD;
|
||||
revisit only if it reads badly once real authored clips exist.
|
||||
- **Sample base clip** — pick one CC0 nature loop for the placeholder grid.
|
||||
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for
|
||||
tuning; the real timing engine is a later slice.
|
||||
|
||||
---
|
||||
|
||||
## 13. Out of scope (YAGNI)
|
||||
|
||||
Real generative v2v; serial / firmware; audio playback; the Pi renderer; the full
|
||||
transition engine; multilingual tables; catalog-model changes; retiring
|
||||
`hef.selection.ranked_candidates`. All remain later roadmap work.
|
||||
@@ -0,0 +1,299 @@
|
||||
# HEF — Reconciled Simulator-First Alteration Slice (Design)
|
||||
|
||||
**Date:** 2026-06-07
|
||||
**Status:** Approved design (pre-implementation) — reconciliation approved this session (0009)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Reconciles:**
|
||||
[`2026-06-06-simulator-alteration-preview-design.md`](./2026-06-06-simulator-alteration-preview-design.md)
|
||||
(session 0007, the *unmerged* `feature/simulator-alteration-preview` branch) **with**
|
||||
[`2026-06-07-scales-library-and-right-axis-pipeline-design.md`](./2026-06-07-scales-library-and-right-axis-pipeline-design.md)
|
||||
(session 0008, merged to `main`).
|
||||
**Parents:**
|
||||
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
|
||||
(the alteration engine) and
|
||||
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md)
|
||||
(the simulator scaffold).
|
||||
**Supersedes:** the **baked-HUD / 5×5-grid** position of the 0007 design (§4, §5, §8 there).
|
||||
|
||||
> **Why this exists.** Sessions 0007 and 0008 left two unmerged design threads that
|
||||
> disagree on one load-bearing point — how the **Left** analytical HUD is rendered —
|
||||
> and, downstream of that, on the **shape of the pre-baked variant set**. 0007 baked
|
||||
> the Left HUD into a 5×5 grid of authored Left×Right variant clips (pixel-precise,
|
||||
> but i18n becomes expensive). 0008 (the later, merged design) kept the Left HUD a
|
||||
> **runtime overlay** driven by an authored annotation track + per-language string
|
||||
> tables (near-free i18n). This document picks the runtime-overlay position, follows
|
||||
> its consequences through the engine and the simulator, and scopes the first
|
||||
> **simulator-runnable** slice that realizes it. It then hands off to an
|
||||
> implementation plan.
|
||||
|
||||
---
|
||||
|
||||
## 1. The decision (operator-approved this session)
|
||||
|
||||
**Left is a runtime overlay, not baked pixels.** Concretely:
|
||||
|
||||
- **Left axis** → a **runtime `AnalyticalOverlay`** driven by an **offline-authored
|
||||
annotation track** (box positions, anchor points, and which label *keys* appear at
|
||||
each Left level 0–4) plus a per-language **string table** (key → translated text).
|
||||
Text is **shaped live** — natively by the browser in the simulator; by
|
||||
**Pango + HarfBuzz + Noto** on the Pi at runtime (the 0008 §1.2 requirement). This
|
||||
is the "authored box positions + runtime-shaped text" hybrid: authorial control
|
||||
over *layout*, near-free *i18n*.
|
||||
- **Right axis** → the **only pre-baked axis**: a small set of **flow-stabilized
|
||||
restyle-strength variants** (0008 §1), selected **discretely** by the Right knob
|
||||
(0–4), with a **crossfade** on change.
|
||||
- **Dark/Light** → a **live runtime `ColorGrade`** (mood center = identity, §5 of the
|
||||
parent).
|
||||
|
||||
**What this supersedes.** The 0007 design's 5×5 Left×Right **grid of 24 authored
|
||||
clips** and its **removal of `AnalyticalOverlay`/`Restyle` from `RenderPlan`** are
|
||||
dropped. The Left HUD is *not* baked; the variant set is **1-D over Right strength**,
|
||||
not a 2-D grid. (Accepted loss vs. fully-baked HUD: no pixel-painted HUD *artwork* —
|
||||
the HUD is shaped text + drawn boxes. Accepted gain: near-free i18n + far less
|
||||
authoring — N Right variants per clip, not 24.)
|
||||
|
||||
**What it preserves from 0007.** The valuable, non-conflicting parts: Python-canonical
|
||||
engine with a thin browser renderer; a parameterized **`Calibration`** tuned by eye in
|
||||
the sim; retiring the simulator's selection-era surface; placeholder-variant generation
|
||||
so the mechanism is testable before real authored media exists.
|
||||
|
||||
### 1.1 The merged engine is already most of the way there
|
||||
|
||||
The slice-1 engine on `main` (`player/alteration.py`) **already** models
|
||||
`RenderPlan = { grade, overlay, restyle }` with a runtime `AnalyticalOverlay` — only
|
||||
0007's *unmerged doc* proposed removing it. So this reconciliation is **surgical**, not
|
||||
a rewrite: keep `grade` and `overlay`; change only how the **Right** axis and the
|
||||
**calibration** are modeled.
|
||||
|
||||
---
|
||||
|
||||
## 2. Engine reconciliation (`player/alteration.py`, `player/state.py`)
|
||||
|
||||
### 2.1 `RenderPlan` layers
|
||||
|
||||
| Layer | Axis | Type | Change from `main` |
|
||||
|---|---|---|---|
|
||||
| `grade: ColorGrade` | Dark/Light | `tone ∈ [−1,1]`, 0 = identity | unchanged shape; curve now from `Calibration` |
|
||||
| `overlay: AnalyticalOverlay` | Left | gains discrete `level: int` (0–4) + keeps `intensity` | `level` added so the renderer selects which annotations show |
|
||||
| `restyle: Restyle` | Right | **discrete** `variant: int` (0–4); 0 = raw | replaces continuous `blend: float` |
|
||||
|
||||
- **`Restyle.variant`** is a discrete index selecting a **pre-baked** Right-strength
|
||||
clip. `variant == 0` means the raw base (no restyle). This matches "select a
|
||||
pre-baked variant and crossfade," and replaces the continuous `blend` that no longer
|
||||
has a runtime meaning (restyle is pre-baked, not blended live).
|
||||
- **`AnalyticalOverlay.level`** (0–4) is the Left knob value; the renderer uses it to
|
||||
choose which annotations from the authored track are active. `intensity` (0..1)
|
||||
stays for overlay opacity/strength and is derived from `level` via `Calibration`.
|
||||
- **`RenderPlan.is_identity`** holds when `grade.is_identity and overlay.level == 0
|
||||
and restyle.variant == 0`.
|
||||
|
||||
### 2.2 `Calibration` (new frozen dataclass)
|
||||
|
||||
A frozen `Calibration` parameterizes the knob→strength maps so they can be tuned **by
|
||||
eye in the sim** and then baked into a `DEFAULT_CALIBRATION` constant:
|
||||
|
||||
- `mood_center` and a per-axis curve for `_mood_tone` (Dark/Light).
|
||||
- the Left `level → intensity` curve.
|
||||
- the Right `knob → variant` map (which knob positions select which pre-baked
|
||||
strength; identity-preserving so knob 0 → variant 0).
|
||||
|
||||
`plan_alteration(coord, calibration: Calibration = DEFAULT_CALIBRATION) -> RenderPlan`.
|
||||
**`DEFAULT_CALIBRATION` reproduces today's exact behavior** (the three current helpers:
|
||||
`value/4` for Left intensity, `(light−dark)/4` for mood, `right` → `variant=right`), so
|
||||
the change is behavior-preserving until the operator tunes it. The session-0006
|
||||
knob→strength open decision is then settled **by eye** in the sim's calibration panel
|
||||
and locked into `DEFAULT_CALIBRATION` by a unit test.
|
||||
|
||||
### 2.3 `player/state.py`
|
||||
|
||||
`state.py` already classifies a change to `plan.restyle` as a `CROSSFADE` and a grade-/
|
||||
overlay-only change as `LIVE_UPDATE`. With `Restyle.variant` discrete, the same
|
||||
classifier works unchanged: a new Right variant → `CROSSFADE`; a grade or Left-overlay
|
||||
change → `LIVE_UPDATE`; clip swap or video on/off → crossfade / fade-to-black as today.
|
||||
The only edit is to the `_classify` comparison if the field name changes
|
||||
(`restyle.blend` → `restyle.variant`).
|
||||
|
||||
These are pure framework-code changes; no deployment-shape decision enters the engine.
|
||||
|
||||
---
|
||||
|
||||
## 3. Simulator (`simulator/`)
|
||||
|
||||
### 3.1 Retire the selection era
|
||||
|
||||
The simulator currently visualizes the **old selection model** (the "curator's X-ray").
|
||||
Remove:
|
||||
|
||||
- `POST /api/select`, `GET /api/catalog/meta`;
|
||||
- the X-ray static UI (`simulator/static/*` rewritten, see §3.4);
|
||||
- `simulator/fixtures.py` (the 625-record synthetic *catalog*) — replaced by
|
||||
`simulator/clips.py`.
|
||||
|
||||
`hef.selection` (the library, incl. `Coordinate`, `ranked_candidates`) is **untouched**;
|
||||
only the simulator's selection *surface* retires. `Coordinate` is still used by the
|
||||
alteration engine.
|
||||
|
||||
### 3.2 `simulator/clips.py` (replaces `fixtures.py`)
|
||||
|
||||
Reads a base-clip + variant manifest. Per base clip:
|
||||
|
||||
```
|
||||
{
|
||||
"id": "forest",
|
||||
"title": "...",
|
||||
"base_file": "forest/base.mp4",
|
||||
"license": "...", "source": "...",
|
||||
"right_variants": { "1": {"file": "forest/right1.mp4", "model": "..."},
|
||||
"...": {...}, "4": {"file": "forest/right4.mp4"} },
|
||||
"annotations": [ {"key": "detected.conifer", "box": [x,y,w,h], "min_level": 1}, ... ],
|
||||
"strings": { "en": { "detected.conifer": "conifer", ... } }
|
||||
}
|
||||
```
|
||||
|
||||
- `right_variants` is keyed by Right strength `1..4` (strength `0` = raw `base_file`).
|
||||
Missing strengths fall back to the raw base and are flagged "raw / unauthored" in the
|
||||
readout.
|
||||
- `annotations` is the **authored annotation track** (box + label key + the minimum
|
||||
Left level at which it appears). `strings` is the per-language table (English only
|
||||
this slice).
|
||||
|
||||
### 3.3 Endpoints
|
||||
|
||||
- **`POST /api/alteration`** — body `{ controls, calibration? }` → `RenderPlan`
|
||||
(serialized) **plus** the `ContentResolution` from `resolve_content` (so the content
|
||||
dial's video-on/off is honored — "Off" → black). Calls the real
|
||||
`plan_alteration(coord, calibration)`.
|
||||
- **`GET /api/clips`** — the base-clip list + the active clip's manifest (variants +
|
||||
annotation track + string table).
|
||||
- Video served as static assets from a sample-media directory.
|
||||
- **Removed:** `POST /api/select`, `GET /api/catalog/meta`.
|
||||
|
||||
### 3.4 Browser preview (`simulator/static/`)
|
||||
|
||||
Thin renderer; all alteration math stays in Python.
|
||||
|
||||
- **Right** — `<video>` showing the selected Right-variant file; **opacity crossfade**
|
||||
to the new file when `restyle.variant` changes; variant 0 → the raw base.
|
||||
- **Dark/Light** — **live color grade** over the video via CSS/canvas filters: Light →
|
||||
warm + lifted toward white; Dark → cool + crushed toward black; tone 0 → raw.
|
||||
- **Left** — overlay drawn **live** from the annotation track: for each annotation with
|
||||
`min_level ≤ overlay.level`, draw its box and the **shaped** string (browser-native
|
||||
shaping) for the active language; opacity from `overlay.intensity`. No baked HUD.
|
||||
- **Content dial** — drives `<video>` visibility; "Off"/audio-only → black walls.
|
||||
- **Calibration panel** — sliders for the `Calibration` params; changing them
|
||||
re-requests the plan and the footage responds live.
|
||||
- **RenderPlan readout** — always shows the exact engine numbers (grade tone, overlay
|
||||
level/intensity, restyle variant) — the project's honesty "X-ray," now over the
|
||||
alteration model.
|
||||
|
||||
The Left overlay being browser-drawn is the **simulator analogue** of the Pi's
|
||||
Pango/HarfBuzz path: both take the *same* annotation track + string table; the browser
|
||||
shapes natively, the Pi shapes with HarfBuzz. The manifest is the shared contract.
|
||||
|
||||
---
|
||||
|
||||
## 4. The sample clip + a real Right variant (this slice)
|
||||
|
||||
To make the look **evaluable now**, wire one real clip end-to-end using the session-0008
|
||||
POC artifacts (`~/hef-poc/out/`, outside the repo):
|
||||
|
||||
- **Base clip** = the POC's `neutral.mp4` (an 8 s nature loop) → copied to the
|
||||
sample-media dir as the one base clip.
|
||||
- **Right variant (top strength)** = the POC's **`right_flow.mp4`** — the real,
|
||||
operator-approved **flow-stabilized** restyle — wired as Right strength 4.
|
||||
- **Intermediate Right strengths (1–3)** = generated by a small **ffmpeg placeholder
|
||||
generator** (e.g. graded/blended stand-ins) so the crossfade mechanism is exercised
|
||||
across the full knob range; the real high-end look is present for tuning.
|
||||
- **Left annotation track** = a minimal authored track (a few boxes + English label
|
||||
keys) for that clip, so Left renders as real shaped text over drawn boxes.
|
||||
|
||||
> **Licensing note.** The simulator sample footage exists **only to tune the look**; it
|
||||
> is not shipped installation content. Strict-PD scale-library sourcing (NASA/NOAA/NPS
|
||||
> per 0008 §2.1) remains a later slice and is unaffected by this choice.
|
||||
|
||||
A real multi-strength SD re-bake (4 genuine flow-stabilized strengths) is **out of scope
|
||||
this slice** — one real strength + placeholders is enough to settle the mechanism and the
|
||||
calibration. The re-bake is a later offline-pipeline task.
|
||||
|
||||
---
|
||||
|
||||
## 5. Testing
|
||||
|
||||
- **`player/` unit tests** (`tests/test_player_alteration.py`, `test_player_state.py`):
|
||||
- `DEFAULT_CALIBRATION` reproduces the current helpers exactly (behavior-preserving).
|
||||
- `Restyle.variant` is discrete; knob 0 → variant 0 (identity); a non-default
|
||||
`Calibration` changes the plan as specified.
|
||||
- `AnalyticalOverlay.level` maps from the Left knob; `intensity` derives from it.
|
||||
- `state.py`: a Right-variant change → `CROSSFADE`; a grade-/overlay-only change →
|
||||
`LIVE_UPDATE`; video on/off unchanged.
|
||||
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
|
||||
- `POST /api/alteration` returns the engine's plan (+ `ContentResolution`) for given
|
||||
controls/calibration.
|
||||
- `GET /api/clips` returns the manifest.
|
||||
- the removed endpoints (`/api/select`, `/api/catalog/meta`) are gone (404).
|
||||
- `test_fixtures.py` retired/rewritten for `clips.py`.
|
||||
- **No browser/E2E automation** this slice — manual visual tuning is the point; the JS
|
||||
stays thin and the tested logic stays in Python.
|
||||
|
||||
---
|
||||
|
||||
## 6. What ships
|
||||
|
||||
- `player/alteration.py` — `Calibration` + `DEFAULT_CALIBRATION`, discrete
|
||||
`Restyle.variant`, `AnalyticalOverlay.level`; `plan_alteration(coord, calibration)`.
|
||||
- `player/state.py` — crossfade-trigger field rename only.
|
||||
- `simulator/clips.py` (variant + annotation manifest) replacing `fixtures.py`.
|
||||
- `simulator/app.py` — `/api/alteration` + `/api/clips`; selection endpoints removed.
|
||||
- `simulator/static/` — rewritten as the alteration preview (variant `<video>` +
|
||||
crossfade, live grade, live Left overlay, content dial, calibration panel,
|
||||
RenderPlan readout).
|
||||
- Sample base clip + one real Right variant + placeholder generator + minimal Left
|
||||
annotation track / English strings.
|
||||
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten; the parent
|
||||
design §4.3/§10 pointer updated to cite this reconciliation; `docs/ROADMAP.md` §3
|
||||
updated.
|
||||
|
||||
---
|
||||
|
||||
## 7. Out of scope (YAGNI) — later slices
|
||||
|
||||
- Serial input / the 3⇄4 framing contract; the Pi/mpv/GPU runtime renderer (deferred by
|
||||
`simulator-first-before-hardware`).
|
||||
- Audio playback (music / white-noise / audio-track).
|
||||
- The **endless rotary encoder** + **AI zoom/warp transitions** between scales
|
||||
(0008 §3) — a separate control + offline pipeline element.
|
||||
- A real multi-strength SD flow-stabilized re-bake; strict-PD scale-library sourcing
|
||||
(0008 §2.1).
|
||||
- Catalog-model changes (audio source / neutral-vs-variant flag); retiring
|
||||
`hef.selection.ranked_candidates`.
|
||||
- Broad multilingual string tables (English-first; the runtime path keeps i18n cheap,
|
||||
but authoring other languages is later).
|
||||
|
||||
---
|
||||
|
||||
## 8. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Calibration curve shape** — **RESOLVED (session 0010, by eye).**
|
||||
`DEFAULT_CALIBRATION` is **locked** to unity gains + a linear variant map
|
||||
(`mood_gain=1.0`, `overlay_gain=1.0`, `right_variant_map=(0,1,2,3,4)`), as a
|
||||
deliberate choice: with the dark-grade fix below, full knob is peaceful on
|
||||
every axis (POC + sim), so full tilt = full look and the 5 notches map 1:1 to
|
||||
the 5 discrete Right bakes. This also closes the **session-0006** convention
|
||||
question — knobs run 0=off..4=max, equal Dark/Light = identity; no
|
||||
"centered at 2 = no push." Guarded by `test_default_calibration_is_locked`.
|
||||
- **Grade vs. Left overlay interaction** — **RESOLVED: overlay above the grade.**
|
||||
The simulator composites the Left HUD (SVG) above the mood grade and the cool
|
||||
tint, so the HUD stays legible regardless of mood. The Pi renderer should do
|
||||
the same.
|
||||
- **Dark-pole grade look** — **FIXED (session 0010).** The first by-eye pass found
|
||||
the sim's dark grade used a full-frame `hue-rotate(-200deg)`, which turned the
|
||||
rock orange and trees purple — the disorienting look rejected in 0008, not the
|
||||
peaceful POC `dark_frame`. Replaced with darken + slight desaturate on the video
|
||||
filter plus a `multiply`-blended deep-blue wash (`#tint`) that lifts shadows
|
||||
toward blue while preserving natural greens. The Pi renderer (later slice) will
|
||||
do proper grading; this matches the approved POC dark look closely enough to tune
|
||||
by eye in the sim.
|
||||
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for tuning;
|
||||
the real timing engine is a later slice.
|
||||
- **Placeholder fidelity** — how close the strength-1–3 placeholders should look to real
|
||||
restyle; cheap stand-ins are fine for mechanism + calibration.
|
||||
@@ -0,0 +1,236 @@
|
||||
# HEF — Scales-of-Nature Library + Stabilized Right-Axis Pipeline (Design Revision)
|
||||
|
||||
**Date:** 2026-06-07
|
||||
**Status:** Approved design (pre-implementation) — operator-approved this session (0008)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Refines:** [`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
|
||||
— specifically its Right-axis pipeline (§4.1/§4.3), content sourcing/model (§6/§8),
|
||||
and economics (§9), and it **adds a scale-navigation control + zoom transitions**
|
||||
to the §2 selector / §11 control panel. The thesis (§1), coordinate model (§3),
|
||||
Dark/Light/Left treatment, and accessibility (§10) are **preserved**.
|
||||
**Grounded in:** a local proof-of-concept run this session on the operator's Mac
|
||||
mini (M4 Pro, 64 GB, MPS) — all numbers below are measured, not estimated.
|
||||
|
||||
> **Why this revision exists.** The 2026-06-05 design specified the Right axis as
|
||||
> "generative video-to-video restyle, pre-baked offline" and assumed that meant a
|
||||
> **paid cloud API** (§9 priced Kling/Sora/Veo/Runway at $0.05–0.10/s). A POC this
|
||||
> session established two things that change the design: (1) the Right restyle runs
|
||||
> **entirely locally and offline** on the operator's existing hardware, for the
|
||||
> cost of electricity; and (2) naïve per-frame restyle **boils/flickers** in a way
|
||||
> the operator found disorienting — disqualifying for a piece meant to be peaceful
|
||||
> — and the fix is **optical-flow keyframe propagation**. Separately, the operator
|
||||
> chose how the "scales of nature" idea enters the piece: as the curatorial theme
|
||||
> of a **small neutral base library**, not a single fixed journey.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Right axis is a local, flow-stabilized restyle (refines §4.1, §4.3)
|
||||
|
||||
The §4.1 mapping is unchanged in spirit — **Right = dissolve realism toward
|
||||
painterly/dreamlike via generative video-to-video** — but the *implementation* is
|
||||
now pinned:
|
||||
|
||||
- **Engine:** Stable Diffusion **img2img** (POC used `stabilityai/sd-turbo`) run on
|
||||
**Apple MPS**, locally, offline, at authoring time. No cloud API.
|
||||
- **Temporal coherence is a hard requirement, not a nicety.** Per-frame img2img
|
||||
independently re-imagines each frame, producing a shimmering "boil" that reads as
|
||||
disorienting — the **opposite** of the piece's peaceful intent. This was caught
|
||||
in the POC and is now a named design constraint: *the Right substrate must be
|
||||
temporally coherent.*
|
||||
- **Stabilization: optical-flow keyframe propagation.** Fully stylize **keyframes**
|
||||
at a fixed interval; for in-between frames, **warp the previous stylized frame
|
||||
forward by optical flow** (so motion is continuous) and apply only a *light*
|
||||
diffusion refine. The flow warp removes the boil; periodic keyframes bound drift.
|
||||
(This is the EbSynth principle. Genuine `ebsynth`/`ezsynth` are NVIDIA/Windows-
|
||||
leaning and don't install cleanly on Apple Silicon, so the POC implemented the
|
||||
same idea directly with OpenCV Farneback flow + the existing diffusers pipeline.)
|
||||
|
||||
This stays consistent with §4.3's crucial invariant: **the Left analytical labels
|
||||
remain a runtime overlay, never baked into the restyled pixels** — the POC's Left
|
||||
HUD is composited deterministically on top, preserving the near-free i18n of §10.
|
||||
|
||||
### 1.1 Where each transform runs (updated §4.3 table)
|
||||
|
||||
| Pole | Operation | Where it runs | Measured cost (8 s, 1080p clip) |
|
||||
|---|---|---|---|
|
||||
| **Dark** | color grade | runtime, live on the Pi | ~2.4 s offline; live at runtime |
|
||||
| **Light** | color grade | runtime, live on the Pi | ~2.5 s offline; live at runtime |
|
||||
| **Left** | analytical overlay (HUD) | runtime, live on the Pi | ~2.2 s offline; live at runtime |
|
||||
| **Right** | local generative restyle + flow propagation | **pre-baked offline, locally** | ~2.7 min/restyle-strength |
|
||||
|
||||
The three deterministic axes are confirmed cheap enough to run **live**; only the
|
||||
Right restyle is pre-baked. A peaceful **deterministic** alternative for Right (soft
|
||||
edge-preserving smoothing + bloom, zero flicker by construction, ~5 s/clip) was
|
||||
prototyped and set aside — the operator preferred the true generative repaint once
|
||||
the flow stabilization made it calm. It remains a documented fallback.
|
||||
|
||||
### 1.2 Runtime label rendering & i18n (sharpens §10)
|
||||
|
||||
The Left analytical labels are drawn **live by the Pi as a 2D graphics overlay** —
|
||||
**not** baked into video, and specifically **not** a pre-rendered overlay *video*. A
|
||||
baked overlay would have to exist per language × per Left level × per scale, which
|
||||
re-introduces the "× number of languages" cost §10 exists to avoid. So the runtime
|
||||
path is:
|
||||
|
||||
- **Architecture.** Per base clip, an offline-authored **annotation track** (box
|
||||
positions, anchor points, which annotations appear at each Left level) referencing
|
||||
language-agnostic label **keys** (e.g. `detected.conifer`). Per language, a cheap
|
||||
**string table** (key → translated text) + font + TTS voice. At runtime the Pi
|
||||
reads the Left knob, selects the active annotations, **shapes** the current
|
||||
language's strings, and composites over the altered video — updating only on change
|
||||
(knob move, language switch, timeline cue), not every frame.
|
||||
- **Correctness needs a real shaping stack.** Rendering *every* language correctly
|
||||
(Arabic joining, Indic conjuncts, CJK, RTL) requires **Pango + HarfBuzz + Noto
|
||||
fonts**, not naïve text drawing. (The POC's HUD used ffmpeg `drawtext`/Menlo —
|
||||
Latin-only; it would mis-render complex scripts and is **not** the runtime path.)
|
||||
This shaping stack is the load-bearing requirement behind §10's "label things
|
||||
correctly."
|
||||
- **Feasibility.** This is OSD/subtitle-class compositing; a Pi 5 (VideoCore VII,
|
||||
GLES/Vulkan, hardware decode) handles it. Headroom at the *panoramic* resolution is
|
||||
the one unmeasured variable — see §6.
|
||||
|
||||
---
|
||||
|
||||
## 2. Content = a small NEUTRAL "scales of nature" library (refines §6, §8)
|
||||
|
||||
The operator's "cosmic zoom" concept (space → continents → birds → ocean → abyss →
|
||||
microscopic → galaxy) enters the piece **as a curatorial theme, not a fixed film.**
|
||||
|
||||
- **Structure:** a **small library (~4–6 to start) of calm, neutral base clips**,
|
||||
each drawn from a *different scale of nature* — e.g. an orbital Earth, a forest, a
|
||||
coral reef, the deep-sea abyss, the microscopic, the cosmos. The machine alters
|
||||
whichever clip is playing, exactly as for any neutral base.
|
||||
- **Why this and not a single stitched journey.** A literal galaxy→cell "how small
|
||||
we are" journey carries its **own** emotion (awe, cosmic insignificance) *before
|
||||
the machine acts*, which contradicts the §1 neutral-base thesis, and as a fixed
|
||||
film it becomes "a journey you watch" rather than "a reality you bend." Keeping the
|
||||
scales as the *theme of a neutral library* preserves the awe-of-scale richness and
|
||||
the piece's coherence **while keeping the base neutral and the experience
|
||||
interactive.** (Rejected alternatives: single stitched base; cosmic-zoom as
|
||||
intro/reset; rethinking the thesis.)
|
||||
- **Cost is not the constraint.** Per §4 below, ~5 base clips is ~1 hour of overnight
|
||||
local pre-bake — so this is an *artistic* choice, made on artistic grounds.
|
||||
- **Mechanism unchanged:** this slots into the existing §6 content model and the
|
||||
sub-project-2 ingest/tagging/review tooling; "scales of nature" is simply the
|
||||
selection principle for which neutral clips to source.
|
||||
|
||||
### 2.1 Strict-PD sourcing map (refines §8)
|
||||
|
||||
License stance is unchanged (prefer Public Domain / CC0; record license + source per
|
||||
clip). The "scales" theme maps onto genuinely public-domain pools cleanly at the
|
||||
*ends* and is softer in the terrestrial *middle*:
|
||||
|
||||
| Scale | Best strict-PD source | Status |
|
||||
|---|---|---|
|
||||
| Cosmos / galaxy / deep space | NASA, Hubble, JWST | 🟢 abundant, true PD |
|
||||
| Earth from orbit / continents | NASA / ISS | 🟢 true PD |
|
||||
| Ocean & **deep sea / abyss** (global) | **NOAA Ocean Exploration** | 🟢 true PD, worldwide |
|
||||
| Microscopic / single-celled | NIH / NSF | 🟢 thinner but PD |
|
||||
| US land / wildlife | NPS, USGS, USFWS | 🟢 true PD (US locations only) |
|
||||
| **Non-US terrestrial, high-flying birds** | — | 🟡 mostly CC-BY; the PD soft spot |
|
||||
|
||||
US-government works are public domain by statute (17 U.S.C. §105); the installation
|
||||
is US-based, so this is the cleanest possible legal footing. **Caveat the design
|
||||
must respect:** "free stock" sites (Pexels, Pixabay, Mitch Martinez's free 4K, etc.)
|
||||
are *royalty-free but NOT public domain* — they restrict redistribution and retain
|
||||
copyright. The ingest tool's "no explicit license → assume PD, verify before use"
|
||||
flag exists precisely for this trap and must not be trusted blindly.
|
||||
|
||||
---
|
||||
|
||||
## 3. Scale navigation & zoom transitions (new element; refines §2 selector, §11)
|
||||
|
||||
The scales-of-nature library is navigated as a **closed loop (a ring), not a line.**
|
||||
A dedicated **scale ("zoom") control** lets the viewer journey through scales;
|
||||
advancing it triggers a short **AI zoom/warp transition** to the next scale,
|
||||
pre-baked offline. Diving past the smallest (single-celled) **wraps around** to the
|
||||
largest (cosmos) — the infinite-zoom payoff that unifies micro and macro and makes
|
||||
the ring continuous.
|
||||
|
||||
- **The control is an *endless* rotary encoder — infinitely turnable, no end stops.**
|
||||
The form embodies the concept: a ring of scales has no beginning or end, so neither
|
||||
does the knob. Keep turning one way and you zoom inward forever (…reef →
|
||||
microscopic → **cosmos** → continents → …); turn back to zoom out. This sets it
|
||||
apart from the four **experience knobs**, which are *absolute* 0–4 pots: the zoom
|
||||
control reports **relative** rotation (encoder detents), and the player/firmware
|
||||
advances or retreats one ring-step per increment. It is distinct too from the §6
|
||||
content dial (audio/video channel) — it chooses *where in the ring* you are, while
|
||||
the knobs still bend whichever scale is present.
|
||||
- **Transitions:** between each adjacent pair of scale clips, a short (~few-second)
|
||||
generative morph — **first-last-frame-conditioned image-to-video** (Wan/LTX-class)
|
||||
or SD "infinite-zoom" outpainting for the literal zoom-through. Pre-baked offline,
|
||||
local; one clip per ring edge (N scales → N transitions, including the micro→cosmos
|
||||
closer). A fast spin may cross several scales — transitions chain, or past a speed
|
||||
threshold a faster blended pass is used.
|
||||
- **Thesis-safe:** dwells on a scale are the neutral, knob-altered interactive cores;
|
||||
transitions are fixed connective moments (un-altered, or at most carrying the
|
||||
current mood grade). The awe lives in the *movement between* scales, not the base.
|
||||
- **Heavier than the restyle, still bounded:** generative video synthesis costs more
|
||||
per second than img2img, but transitions are few and short — a handful of ~3–5 s
|
||||
morphs is an overnight local batch.
|
||||
|
||||
---
|
||||
|
||||
## 4. Economics update — local authoring ≈ free (refines §9)
|
||||
|
||||
§9 priced the Right axis at **~$300–3k** of cloud generative-v2v API. The POC
|
||||
collapses that: the restyle runs on **hardware the operator already owns**, offline,
|
||||
so the marginal cost is electricity.
|
||||
|
||||
- **Per base clip:** ~4 Right restyle strengths × ~2.7 min ≈ **~11 min of local AI
|
||||
pre-bake**, plus seconds for the deterministic Dark/Light/Left grid.
|
||||
- **Whole small library (~5 clips):** **~1 hour** of overnight batch rendering.
|
||||
- **Scale transitions:** ~N short generative-video morphs (one per ring edge), a few
|
||||
seconds each — a separate, heavier offline batch (video synthesis > img2img), but
|
||||
still overnight-local.
|
||||
- **Cloud API: no longer required** for the build. (It remains an option if a
|
||||
higher-quality video model than a local one is wanted for a final pass.)
|
||||
|
||||
This also tightens the piece's "all-local" ethos: not just *runtime* is local
|
||||
(§1 of the prior design) — now *authoring* is too.
|
||||
|
||||
---
|
||||
|
||||
## 5. POC evidence (this session)
|
||||
|
||||
A throwaway spike (outside the repo, `~/hef-poc/`) validated the full engine on one
|
||||
real nature clip (a 4K Yosemite waterfall, trimmed to 8 s @ 1080p):
|
||||
|
||||
- **All four axes rendered** and read as distinct: Dark (cold/somber), Light
|
||||
(warm/serene), Left (analytical HUD overlay), Right (painterly).
|
||||
- **Right per-frame:** ~3.4 min/8 s clip, **flickers badly** (disqualifying).
|
||||
- **Right flow-propagated:** ~2.7 min/8 s clip, **calm** (operator-approved).
|
||||
- **Deterministic axes:** ~2.4 s each, ~3× faster than real-time → confirmed
|
||||
runtime-capable.
|
||||
- **Stack:** `imageio-ffmpeg`, `diffusers` + `sd-turbo` on MPS, OpenCV Farneback
|
||||
flow; Python 3.13; 64 GB unified memory comfortably ran models that OOM consumer
|
||||
GPUs.
|
||||
|
||||
---
|
||||
|
||||
## 6. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Flow quality at scale:** the OpenCV flow propagation was validated on one short
|
||||
clip; longer clips / faster motion may need shorter keyframe intervals, bidirectional
|
||||
blending, or a stronger flow model (RAFT).
|
||||
- **Painterly strength:** the POC kept the restyle gentle; the dreamlike *range* and
|
||||
the per-axis restyle-strength count (the §4.3 "small set of variants") are
|
||||
unfixed.
|
||||
- **Base-clip sourcing:** select and ingest the actual ~4–6 strictly-PD neutral
|
||||
scale clips (NASA/NOAA/NPS) via sub-project 2.
|
||||
- **Model choice:** `sd-turbo` was the POC's speed pick; a higher-quality local
|
||||
model (or a final cloud pass) may be worth a comparison for the shipped variants.
|
||||
- **Scale transitions:** generation method (first-last-frame i2v vs. infinite-zoom
|
||||
outpainting) and local model; per-transition length; whether transitions carry the
|
||||
current mood grade; the ring ordering of the scales; and behavior on fast or
|
||||
continuous spins of the endless encoder (chain transitions vs. blended skip).
|
||||
- **Pi compositing headroom:** confirm the Pi 5 can decode the altered video **and**
|
||||
render the live Pango/HarfBuzz label overlay at the *actual* panoramic projector
|
||||
resolution (ultra-wide / high-res) — low risk but unmeasured (see §1.2).
|
||||
|
||||
## 7. Out of scope (YAGNI)
|
||||
|
||||
- A single continuous **one-take** zoom through *all* scales (we use discrete neutral
|
||||
clips joined by short AI transitions on a navigable ring — not one unbroken shot).
|
||||
- Audio-axis alteration (already deferred by the prior §6).
|
||||
- Cloud rendering pipeline (local supersedes it for the base build).
|
||||
@@ -0,0 +1,25 @@
|
||||
Feature: Synthetic fixture catalog
|
||||
As a curator with an empty real catalog
|
||||
I want a dense, representative stand-in catalog generated on demand
|
||||
So that I can feel nearest-match behavior everywhere before any media is ingested
|
||||
|
||||
Scenario: The fixture catalog spans the coordinate space and all content modes
|
||||
When the fixture catalog is generated
|
||||
Then it contains records covering the brain plane and the mood plane
|
||||
And it contains records in each of the "audio", "video", and "av" modes
|
||||
And it contains a mix of "proposed" and "approved" records
|
||||
|
||||
Scenario: The fixture catalog is valid
|
||||
When the fixture catalog is generated
|
||||
Then every record passes hef.catalog validation
|
||||
And no record references real media
|
||||
|
||||
Scenario: Generation is deterministic
|
||||
Given a fixed seed
|
||||
When the fixture catalog is generated twice
|
||||
Then both runs produce identical records
|
||||
|
||||
Scenario: The real catalog is used when it is populated
|
||||
Given catalog/library.jsonl is non-empty
|
||||
When the simulator starts pointed at the real catalog
|
||||
Then it loads the real records through the same loader instead of the fixtures
|
||||
@@ -0,0 +1,32 @@
|
||||
Feature: Feeling the model's own parameters
|
||||
As a curator tuning the algorithm, not just the curation
|
||||
I want to adjust the selection model's parameters live
|
||||
So that I can feel how weighting, pool size, and the approved gate reshape picks
|
||||
|
||||
Background:
|
||||
Given the simulator is loaded with the synthetic fixture catalog
|
||||
And the dials are set to a fixed knob position with mode "av"
|
||||
|
||||
Scenario: Reweighting the brain plane can change the winner
|
||||
Given the brain and mood weights are equal
|
||||
And the current pick is recorded
|
||||
When I increase the brain weight relative to the mood weight
|
||||
Then the ranked pool may reorder so that brain-plane distance dominates
|
||||
And the picked piece can differ from the recorded pick
|
||||
|
||||
Scenario: Pool size controls how many candidates are shown and shuffled
|
||||
When I set the pool size to 1
|
||||
Then the pool shows exactly the single nearest candidate
|
||||
When I set the pool size to 5
|
||||
Then the pool shows up to the five nearest candidates
|
||||
|
||||
Scenario: Approved-only narrows the pool to human-blessed records
|
||||
Given the fixture catalog contains both "proposed" and "approved" records
|
||||
When I turn the approved-only toggle on
|
||||
Then every candidate in the pool has review_status "approved"
|
||||
When I turn the approved-only toggle off
|
||||
Then "proposed" records may appear in the pool again
|
||||
|
||||
Scenario: Default model parameters reproduce the shipped selection behavior
|
||||
Given the weights are equal, the pool size is the default, and approved-only is off
|
||||
Then the picked piece matches what hef.selection.select would return for the same dials
|
||||
@@ -0,0 +1,48 @@
|
||||
Feature: Curator's X-ray of the selection model
|
||||
As a curator
|
||||
I want to turn the installation's dials against a representative catalog and see
|
||||
the full ranked pool the nearest-match algorithm picks from
|
||||
So that I can judge whether the tagging rubric and selection surface fitting pieces
|
||||
|
||||
Background:
|
||||
Given the simulator is loaded with the synthetic fixture catalog
|
||||
And the fixture catalog spans the brain plane and the mood plane across all content modes
|
||||
|
||||
Scenario: A pick is always accompanied by its candidate pool
|
||||
When I set the mode to "av" and the dials to left=1 right=3 dark=4 light=1
|
||||
Then a piece is picked
|
||||
And the ranked pool is shown nearest-first with each candidate's distance
|
||||
And the picked piece is the rank-1 candidate in the pool
|
||||
|
||||
Scenario: Turning Dark up while Light stays low pulls the pick toward somber pieces
|
||||
Given the mode is "av" and the dials are left=2 right=2 dark=0 light=0
|
||||
When I raise the Dark dial from 0 to 4
|
||||
Then the picked piece's dark coordinate is higher than before
|
||||
And the pool's nearest candidates cluster toward the high-dark / low-light region
|
||||
|
||||
Scenario: Turning a dial moves the knob point on the coordinate map
|
||||
Given the mode is "av"
|
||||
When I change any of the Left, Right, Dark, or Light dials
|
||||
Then the knob point on the brain or mood grid map moves to the new coordinate
|
||||
And the candidate dots reflect the new neighborhood
|
||||
|
||||
Scenario: None mode is the void / rest state
|
||||
When I set the mode to "none"
|
||||
Then no piece is picked
|
||||
And the pool is empty
|
||||
And the screen shows the void / rest state
|
||||
|
||||
Scenario: A+V mode falls back to audio and video when av records are thin
|
||||
Given the fixture catalog has fewer than the pool size of native "av" records near the knob point
|
||||
When I set the mode to "av"
|
||||
Then the pool includes "audio" and "video" records so it is not starved
|
||||
|
||||
Scenario Outline: Each content mode restricts the pool to eligible records
|
||||
When I set the mode to "<mode>"
|
||||
Then every candidate in the pool is eligible for "<mode>"
|
||||
|
||||
Examples:
|
||||
| mode |
|
||||
| audio |
|
||||
| video |
|
||||
| av |
|
||||
@@ -0,0 +1,27 @@
|
||||
Feature: Simulator service and API
|
||||
As an operator
|
||||
I want to run the simulator locally in one container and drive it over a small API
|
||||
So that the browser X-ray and its behavior are testable without the physical stack
|
||||
|
||||
Scenario: The service runs locally in a container
|
||||
When I start the simulator container
|
||||
Then the X-ray is reachable in a browser on localhost
|
||||
And no external network access is required
|
||||
|
||||
Scenario: Selecting returns the pick and the ranked pool
|
||||
When I POST a valid selection request with dials and mode "av"
|
||||
Then the response contains a picked record
|
||||
And the response contains the ranked pool with a distance and rank per candidate
|
||||
|
||||
Scenario: None mode returns the void over the API
|
||||
When I POST a selection request with mode "none"
|
||||
Then the response pick is null
|
||||
And the response pool is empty
|
||||
|
||||
Scenario: Invalid dial values are rejected
|
||||
When I POST a selection request with a dial value outside 0 to 4
|
||||
Then the API responds with a validation error
|
||||
|
||||
Scenario: Catalog metadata is available for the header
|
||||
When I GET the catalog metadata
|
||||
Then the response reports record counts, a per-mode breakdown, and the proposed/approved split
|
||||
@@ -126,3 +126,20 @@ def append_record(record, path) -> None:
|
||||
path = Path(path)
|
||||
with path.open("a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(record_to_dict(record), ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
def index_by_id(records) -> dict:
|
||||
"""Map id -> Record, raising CatalogError on a duplicate id."""
|
||||
idx: dict = {}
|
||||
for r in records:
|
||||
if r.id in idx:
|
||||
raise CatalogError(f"duplicate record id: {r.id!r}")
|
||||
idx[r.id] = r
|
||||
return idx
|
||||
|
||||
|
||||
def validate_catalog(records) -> None:
|
||||
"""Validate every record AND cross-record invariants (currently: unique ids)."""
|
||||
for r in records:
|
||||
validate(r)
|
||||
index_by_id(records) # raises on duplicate id
|
||||
|
||||
+42
-10
@@ -58,6 +58,38 @@ def candidates_for_mode(records, mode: str, pool_size: int) -> list[Record]:
|
||||
from typing import Optional
|
||||
|
||||
|
||||
def ranked_candidates(
|
||||
records,
|
||||
coord: Coordinate,
|
||||
mode: str,
|
||||
*,
|
||||
pool_size: int = 4,
|
||||
weights: Weights = Weights(),
|
||||
approved_only: bool = False,
|
||||
) -> list[tuple[Record, float]]:
|
||||
"""Mode-eligible records sorted nearest-first, each paired with its distance.
|
||||
|
||||
Applies the same approved_only filter and 'av' pool fallback as select(),
|
||||
then returns up to pool_size nearest (record, distance) pairs. This is the
|
||||
single source of truth for "the pool"; select() picks from it.
|
||||
"""
|
||||
if mode not in CONTENT_MODES:
|
||||
raise ValueError(
|
||||
f"ranked_candidates expects a content mode {sorted(CONTENT_MODES)}, "
|
||||
f"got {mode!r}"
|
||||
)
|
||||
pool = records
|
||||
if approved_only:
|
||||
pool = [r for r in pool if r.review_status == "approved"]
|
||||
candidates = candidates_for_mode(pool, mode, pool_size)
|
||||
ranked = sorted(
|
||||
candidates,
|
||||
key=lambda r: (distance(coord, record_coordinate(r), weights), r.id),
|
||||
)
|
||||
nearest = ranked[:pool_size]
|
||||
return [(r, distance(coord, record_coordinate(r), weights)) for r in nearest]
|
||||
|
||||
|
||||
def select(
|
||||
records,
|
||||
coord: Coordinate,
|
||||
@@ -82,17 +114,17 @@ def select(
|
||||
)
|
||||
if mode == "none":
|
||||
return None
|
||||
pool = records
|
||||
if approved_only:
|
||||
pool = [r for r in pool if r.review_status == "approved"]
|
||||
candidates = candidates_for_mode(pool, mode, pool_size)
|
||||
if not candidates:
|
||||
return None
|
||||
ranked = sorted(
|
||||
candidates,
|
||||
key=lambda r: (distance(coord, record_coordinate(r), weights), r.id),
|
||||
ranked = ranked_candidates(
|
||||
records,
|
||||
coord,
|
||||
mode,
|
||||
pool_size=pool_size,
|
||||
weights=weights,
|
||||
approved_only=approved_only,
|
||||
)
|
||||
nearest = ranked[:pool_size]
|
||||
if not ranked:
|
||||
return None
|
||||
nearest = [r for r, _ in ranked]
|
||||
if rng is None:
|
||||
return nearest[0]
|
||||
return rng.choice(nearest)
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
|
||||
|
||||
Reconciled slice (2026-06-07): the Right axis is a DISCRETE selection of a
|
||||
pre-baked, flow-stabilized restyle variant (not a continuous blend), the Left
|
||||
axis carries its knob LEVEL so a runtime annotation track can pick which labels
|
||||
show, and a frozen `Calibration` parameterizes the knob->strength curves so they
|
||||
can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION.
|
||||
|
||||
Layers compose per §4.2:
|
||||
- Substrate: ColorGrade (Dark/Light mood, center = identity §5) + a pre-baked
|
||||
Right restyle variant.
|
||||
- Overlay: AnalyticalOverlay (Left), composited on top at runtime.
|
||||
Left and Right stack (different layers); Dark/Light are the two poles of one
|
||||
mood grade. See docs/superpowers/specs/2026-06-07-reconciled-simulator-
|
||||
alteration-slice-design.md.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from hef.selection import Coordinate
|
||||
|
||||
KNOB_MAX = 4 # knob full-scale (0..4)
|
||||
|
||||
|
||||
def _clamp(x: float, lo: float, hi: float) -> float:
|
||||
return max(lo, min(hi, x))
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Calibration:
|
||||
"""Tunable knob->strength curves (settled by eye in the sim, then baked).
|
||||
|
||||
- mood_gain: scales the signed Dark/Light tone (result clamped to [-1, 1]).
|
||||
- overlay_gain: scales the Left overlay intensity (clamped to [0, 1]).
|
||||
- right_variant_map: knob value (0..4) -> pre-baked Right variant index.
|
||||
"""
|
||||
|
||||
mood_gain: float = 1.0
|
||||
overlay_gain: float = 1.0
|
||||
right_variant_map: tuple = (0, 1, 2, 3, 4)
|
||||
|
||||
|
||||
# The LOCKED calibration (session 0010, 2026-06-07) — settled by eye in the
|
||||
# simulator, closing the open session-0006 knob->strength decision. Convention:
|
||||
# every experience knob runs 0 = off .. 4 = max, and equal Dark/Light = identity
|
||||
# (raw footage); there is no "centered at 2 = no push" coordinate.
|
||||
# - mood_gain = 1.0: full Dark/Light knob reaches the full mood grade. By-eye
|
||||
# evidence (POC renders + sim, after the dark-grade fix this session) shows
|
||||
# full tilt is peaceful on every axis, so no softening is warranted.
|
||||
# - overlay_gain = 1.0: full Left = opacity 1.0; the HUD is legible, not
|
||||
# overwhelming, sitting above the grade.
|
||||
# - right_variant_map linear: the 5 knob notches map 1:1 onto the 5 discrete
|
||||
# pre-baked Right strengths (0 = raw base).
|
||||
# These are unity/linear by deliberate choice, not as placeholders. The curve
|
||||
# stays parameterized so a future re-bake or a different feel is one edit away;
|
||||
# test_default_calibration_is_locked guards the values from drifting silently.
|
||||
DEFAULT_CALIBRATION = Calibration(
|
||||
mood_gain=1.0,
|
||||
overlay_gain=1.0,
|
||||
right_variant_map=(0, 1, 2, 3, 4),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ColorGrade:
|
||||
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
|
||||
<0 cool blue->black (dark), 0 = identity (raw, ungraded)."""
|
||||
|
||||
tone: float
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
return self.tone == 0.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AnalyticalOverlay:
|
||||
"""Left axis (§4.1): analytical HUD/annotation, composited on top at runtime.
|
||||
`level` is the Left knob (0..4); a runtime annotation track uses it to pick
|
||||
which labels appear. `intensity` 0..1 is the overlay opacity/strength."""
|
||||
|
||||
level: int
|
||||
intensity: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Restyle:
|
||||
"""Right axis (§4.1): selects a pre-baked, flow-stabilized restyle variant.
|
||||
`variant` is a discrete index (0 = raw base, no restyle)."""
|
||||
|
||||
variant: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RenderPlan:
|
||||
"""The full layered alteration for one knob vector (§4.2)."""
|
||||
|
||||
grade: ColorGrade
|
||||
overlay: AnalyticalOverlay
|
||||
restyle: Restyle
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
"""True when the plan leaves the neutral base un-altered."""
|
||||
return (
|
||||
self.grade.is_identity
|
||||
and self.overlay.level == 0
|
||||
and self.restyle.variant == 0
|
||||
)
|
||||
|
||||
|
||||
def _overlay_intensity(left: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.overlay_gain * left / KNOB_MAX, 0.0, 1.0)
|
||||
|
||||
|
||||
def _right_variant(right: int, cal: Calibration) -> int:
|
||||
return cal.right_variant_map[right]
|
||||
|
||||
|
||||
def _mood_tone(dark: int, light: int, cal: Calibration) -> float:
|
||||
return _clamp(cal.mood_gain * (light - dark) / KNOB_MAX, -1.0, 1.0)
|
||||
|
||||
|
||||
def plan_alteration(
|
||||
coord: Coordinate, calibration: Calibration = DEFAULT_CALIBRATION
|
||||
) -> RenderPlan:
|
||||
"""Map a knob vector to its layered RenderPlan (design §4)."""
|
||||
return RenderPlan(
|
||||
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light, calibration)),
|
||||
overlay=AnalyticalOverlay(
|
||||
level=coord.left,
|
||||
intensity=_overlay_intensity(coord.left, calibration),
|
||||
),
|
||||
restyle=Restyle(variant=_right_variant(coord.right, calibration)),
|
||||
)
|
||||
|
||||
|
||||
def render_plan_to_dict(plan: RenderPlan) -> dict:
|
||||
"""JSON-serializable form for the simulator API."""
|
||||
return {
|
||||
"grade": {"tone": plan.grade.tone},
|
||||
"overlay": {"level": plan.overlay.level, "intensity": plan.overlay.intensity},
|
||||
"restyle": {"variant": plan.restyle.variant},
|
||||
"is_identity": plan.is_identity,
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
|
||||
|
||||
The single source of truth for design §6's table: which audio source plays and
|
||||
whether the projector shows video, for each dial position.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
# The four distinct audio sources. "white_noise" is generated at runtime;
|
||||
# "music" is the public-domain classical pool; "audio_track" is the clip's own
|
||||
# field audio; "none" is silence.
|
||||
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContentResolution:
|
||||
audio_source: str
|
||||
video: bool
|
||||
|
||||
|
||||
# Design §6, one row per dial position.
|
||||
_TABLE = {
|
||||
"off": ContentResolution("none", False),
|
||||
"white_noise": ContentResolution("white_noise", False),
|
||||
"music": ContentResolution("music", False),
|
||||
"audio_track": ContentResolution("audio_track", False),
|
||||
"video": ContentResolution("none", True),
|
||||
"music_video": ContentResolution("music", True),
|
||||
"audio_video": ContentResolution("audio_track", True),
|
||||
}
|
||||
|
||||
|
||||
def resolve_content(position: str) -> ContentResolution:
|
||||
"""Map a content-dial position to its audio source and video flag (§6)."""
|
||||
try:
|
||||
return _TABLE[position]
|
||||
except KeyError:
|
||||
raise ValueError(
|
||||
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
|
||||
) from None
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Control-panel state: the data shape read from the Arduino over serial.
|
||||
|
||||
This is the serial-contract payload shared with sub-project 4 (firmware). It
|
||||
models the full panel from design §6/§7: the 7-way content dial, the four
|
||||
experience knobs (0..4), and the two intensity levels (volume, brightness).
|
||||
The wire framing itself is the separate 3<->4 serial contract; this module is
|
||||
the *decoded* form.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, fields
|
||||
|
||||
# The seven positions of the content dial (design §6).
|
||||
CONTENT_POSITIONS = frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
|
||||
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
|
||||
KNOB_MIN = 0
|
||||
KNOB_MAX = 4
|
||||
|
||||
|
||||
class ControlsError(ValueError):
|
||||
"""Raised when a Controls payload is structurally invalid."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Controls:
|
||||
content: str
|
||||
left: int
|
||||
right: int
|
||||
dark: int
|
||||
light: int
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
|
||||
def validate_controls(c: Controls) -> None:
|
||||
"""Raise ControlsError if the payload is structurally invalid."""
|
||||
if c.content not in CONTENT_POSITIONS:
|
||||
raise ControlsError(
|
||||
f"invalid content position {c.content!r}; "
|
||||
f"expected one of {sorted(CONTENT_POSITIONS)}"
|
||||
)
|
||||
for name in KNOB_FIELDS:
|
||||
value = getattr(c, name)
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise ControlsError(f"knob {name} must be an int, got {value!r}")
|
||||
if not (KNOB_MIN <= value <= KNOB_MAX):
|
||||
raise ControlsError(
|
||||
f"knob {name}={value} out of range {KNOB_MIN}..{KNOB_MAX}"
|
||||
)
|
||||
|
||||
|
||||
def parse_controls(data: dict) -> Controls:
|
||||
"""Build a validated Controls from a decoded mapping, rejecting unknown or
|
||||
missing keys."""
|
||||
known = {f.name for f in fields(Controls)}
|
||||
unknown = set(data) - known
|
||||
if unknown:
|
||||
raise ControlsError(f"unknown keys: {sorted(unknown)}")
|
||||
missing = known - set(data)
|
||||
if missing:
|
||||
raise ControlsError(f"missing keys: {sorted(missing)}")
|
||||
c = Controls(**data)
|
||||
validate_controls(c)
|
||||
return c
|
||||
+120
@@ -0,0 +1,120 @@
|
||||
"""The player state machine: a stream of Controls -> playback transitions.
|
||||
|
||||
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
|
||||
desired Playback (which neutral base clip, how it is altered, what audio plays,
|
||||
at what levels) and returns the Transition from the previous Playback. The
|
||||
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
|
||||
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
|
||||
pre-baked Right restyle variant (a discrete index) needs a CROSSFADE, and
|
||||
toggling video on/off fades to/from black.
|
||||
|
||||
Knobs no longer *select* a clip (the base library is neutral by construction);
|
||||
they *transform* it. Which neutral base to show is an injected policy
|
||||
(`choose_base`), defaulting to the first clip; richer rotation is a later slice.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable, Optional
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import RenderPlan, plan_alteration
|
||||
from player.content import ContentResolution, resolve_content
|
||||
from player.controls import Controls
|
||||
|
||||
|
||||
class TransitionKind:
|
||||
NONE = "none"
|
||||
LIVE_UPDATE = "live_update"
|
||||
CROSSFADE = "crossfade"
|
||||
FADE_TO_BLACK = "fade_to_black"
|
||||
FADE_FROM_BLACK = "fade_from_black"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Playback:
|
||||
"""What should currently be playing."""
|
||||
|
||||
clip_id: Optional[str] # None = black walls
|
||||
plan: Optional[RenderPlan] # None when black
|
||||
content: ContentResolution
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Transition:
|
||||
kind: str
|
||||
playback: Playback
|
||||
|
||||
|
||||
def _first(library):
|
||||
if not library:
|
||||
raise ValueError("no base clips available to play video")
|
||||
return library[0]
|
||||
|
||||
|
||||
_BLACK = Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=ContentResolution("none", False),
|
||||
volume=0,
|
||||
brightness=0,
|
||||
)
|
||||
|
||||
|
||||
class Player:
|
||||
def __init__(
|
||||
self,
|
||||
base_library,
|
||||
*,
|
||||
choose_base: Callable = _first,
|
||||
approved_only: bool = False,
|
||||
):
|
||||
self._library = list(base_library)
|
||||
self._choose_base = choose_base
|
||||
self._approved_only = approved_only # reserved for catalog-backed libs
|
||||
self._current = _BLACK
|
||||
|
||||
def _resolve(self, controls: Controls) -> Playback:
|
||||
content = resolve_content(controls.content)
|
||||
if not content.video:
|
||||
return Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=content,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
clip = self._choose_base(self._library)
|
||||
coord = Coordinate(controls.left, controls.right, controls.dark, controls.light)
|
||||
return Playback(
|
||||
clip_id=clip.id,
|
||||
plan=plan_alteration(coord),
|
||||
content=content,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
|
||||
def _classify(self, prev: Playback, nxt: Playback) -> str:
|
||||
prev_video = prev.clip_id is not None
|
||||
next_video = nxt.clip_id is not None
|
||||
if prev == nxt:
|
||||
return TransitionKind.NONE
|
||||
if prev_video and not next_video:
|
||||
return TransitionKind.FADE_TO_BLACK
|
||||
if next_video and not prev_video:
|
||||
return TransitionKind.FADE_FROM_BLACK
|
||||
if next_video and prev_video:
|
||||
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
|
||||
return TransitionKind.CROSSFADE
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
# both black: only audio/levels could have changed
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
|
||||
def update(self, controls: Controls) -> Transition:
|
||||
nxt = self._resolve(controls)
|
||||
kind = self._classify(self._current, nxt)
|
||||
self._current = nxt
|
||||
return Transition(kind=kind, playback=nxt)
|
||||
+12
-1
@@ -12,4 +12,15 @@ requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[tool.setuptools]
|
||||
packages = ["hef"]
|
||||
packages = ["hef", "tools", "simulator", "player"]
|
||||
|
||||
[project.scripts]
|
||||
hef-ingest = "tools.ingest_cli:main"
|
||||
hef-review = "tools.review_cli:main"
|
||||
|
||||
[project.optional-dependencies]
|
||||
sim = [
|
||||
"fastapi>=0.110",
|
||||
"uvicorn[standard]>=0.29",
|
||||
"httpx>=0.27",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
# Session 0001.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-04T05-36 (PST) · End: 2026-06-04T06-15 (PST)
|
||||
> Type: spec
|
||||
> Goal: Develop the spec for sub-project 2 — "Ingest & Tagging / Review tools".
|
||||
> Outcome: Spec written, self-reviewed, and operator-approved; implementation
|
||||
> plan written; design spec + ROADMAP revised for a mid-session display
|
||||
> architecture change. All merged to `main` and pushed; tree clean.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
/goal Develop the spec for sub-project 2 — "Ingest & Tagging / Review tools" — of
|
||||
the Human Experience Filter, the next item in docs/ROADMAP.md. Ground it in
|
||||
docs/superpowers/specs/2026-06-04-human-experience-filter-design.md (especially §5
|
||||
division-of-labor and §8 sourcing) and the roadmap's sub-project-2 section.
|
||||
|
||||
The spec must cover: per-archive ingest fetchers (Internet Archive/Prelinger,
|
||||
Musopen, LibriVox, NASA, Free Music Archive, Freesound); automatic mechanical
|
||||
tagging (mode via ffprobe, license/attribution/source from origin, duration_s,
|
||||
resolution, dominant_color computed from video); assistant coordinate-drafting
|
||||
that writes a one-line rationale and sets review_status=proposed; and the review
|
||||
CLI that walks proposed records (proposed coords + rationale + preview frame) and
|
||||
flips them to approved with reviewed_at. It must build on the existing
|
||||
hef.catalog API without breaking it — call out explicitly any additions the
|
||||
tools need.
|
||||
```
|
||||
|
||||
## Plan
|
||||
|
||||
Spec session — develop the sub-project-2 spec (Ingest & Tagging / Review tools),
|
||||
grounded in the approved design doc (§5 division-of-labor, §8 sourcing) and
|
||||
ROADMAP §2. Cover: per-archive fetchers, mechanical tagging, coordinate drafting,
|
||||
review CLI. Hard constraint: build on `hef.catalog` without breaking it; call out
|
||||
additions explicitly. Done when spec written + self-reviewed + approved; any plan
|
||||
saved under `plans/`; tree clean, merged to `main`, branches deleted, pushed.
|
||||
|
||||
Mid-session the operator added two follow-on asks: write the implementation plan,
|
||||
and update the design spec for a new display decision.
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- Sub-project 1 (Catalog & Selection Core) done and merged: `hef/catalog.py`
|
||||
(`Record`, `validate`, JSONL IO) + `hef/selection.py` (distance, mode filter,
|
||||
`select`), 37 passing tests.
|
||||
- Repo was **not yet registered** for the session protocol — no `app.json`, no
|
||||
`CLAUDE.md`. Clean `main`, pushed.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Session init (spec).** Got start time; surveyed git (clean `main`); read
|
||||
ROADMAP, memory (none), `app.json`/`CLAUDE.md` (absent). Resolver exited 3
|
||||
(repo unregistered). Authored `app.json` (One Name = `human-experience-filter-art`;
|
||||
sessions self-contained in this repo under `sessions/`, matching the benstull
|
||||
`docs` convention) and committed it to `main` (`4ddc113`). Created `CLAUDE.md`
|
||||
importing `@~/.claude/wiggleverse.md` (`5e4b794`, after a rebase — the claim
|
||||
push had advanced `main`). Claimed session **0001** with `--type spec`; no
|
||||
concurrent sessions. Seeded the transcript `## Plan`.
|
||||
2. **Orientation.** No BDD corpus (`BDD_REMOTE` empty), so grounding = the design
|
||||
spec, per the launch prompt. Read the design spec, `hef/catalog.py`,
|
||||
`hef/selection.py`, the sub-project-1 plan, and `USER_GUIDE.md`.
|
||||
3. **Key realization.** The `Record` schema already ships every field the
|
||||
draft-then-review flow writes, with defaults (`review_status="proposed"`,
|
||||
`reviewed_at=None`) that ARE the proposed shape — so the schema needs no change.
|
||||
4. **Wrote the spec** (`docs/superpowers/specs/2026-06-04-ingest-tagging-review-tools.md`,
|
||||
`f80ec65`) on branch `spec/sub-project-2-ingest-review`: 12 sections covering
|
||||
the `Candidate`/`Fetcher` seam + all six archives, ffprobe/ffmpeg mechanical
|
||||
tagging (with cover-art guard), per-archive license normalization, the
|
||||
deterministic `HeuristicProposer` (no ML, honoring design §11), the review
|
||||
transition core + interactive CLI. Single required `hef.catalog` addition:
|
||||
`validate_catalog` (unique-id integrity), optional `index_by_id`.
|
||||
5. **Self-review.** Verified against the live code: no name collision for the new
|
||||
symbols, all 37 existing tests still green, every launch-prompt requirement
|
||||
covered.
|
||||
6. **Approval (AskUserQuestion).** Operator **approved as written**, confirmed
|
||||
first-ship archives (LibriVox + NASA + IA), and — on the color question —
|
||||
dropped a design change: *single panoramic projector across all three walls,
|
||||
nature-video content, "may not need any video manipulation at all."*
|
||||
7. **Folded the change into the spec** (`4c1ec6e`): demoted `dominant_color` to
|
||||
optional/opt-in (its only consumer was the side walls), ffmpeg-only when
|
||||
computed. Merged the spec to `main` (`3544092`), deleted the branch, pushed.
|
||||
Linked the spec from ROADMAP §2 (`b5f5ee6`).
|
||||
8. **Operator chose all three follow-ups:** plan + design-spec update + finalize.
|
||||
On branch `docs/pano-update-and-sp2-plan`: revised the design spec (banner;
|
||||
§1, §6 single pano projector, §7 procedural side walls REMOVED, §10, §12) and
|
||||
ROADMAP (sub-project 5 Dropped; diagram; §3 player → pano; cross-cutting
|
||||
decisions) (`dd02c2c`); wrote the task-by-task implementation plan
|
||||
(`docs/superpowers/plans/2026-06-04-ingest-tagging-review-tools.md`, `6d6cbb5`).
|
||||
Merged to `main` (`dd00d8b`), deleted the branch, pushed.
|
||||
|
||||
## Cut state (end of session)
|
||||
|
||||
Branch `main`, clean, `0/0` vs `origin/main`. No feature branches. Key commits
|
||||
(all on `git.benstull.org:benstull/human-experience-filter-art`):
|
||||
|
||||
| SHA | What |
|
||||
|-----|------|
|
||||
| `4ddc113` | register `app.json` |
|
||||
| `5e4b794` | `CLAUDE.md` org import |
|
||||
| `f80ec65` | sub-project-2 spec (initial) |
|
||||
| `4c1ec6e` | spec: fold in approval + pano change |
|
||||
| `3544092` | merge spec → main |
|
||||
| `b5f5ee6` | ROADMAP links the spec |
|
||||
| `dd02c2c` | design + ROADMAP pano revision |
|
||||
| `6d6cbb5` | sub-project-2 implementation plan |
|
||||
| `dd00d8b` | merge pano revision + plan → main |
|
||||
|
||||
(Plus the session-protocol commits `18f0360`/`54fde0b`/`b8c406d` for the
|
||||
transcript itself.)
|
||||
|
||||
Tests: untouched this session — still 37 green (no code changed; docs only).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **Major design change surfaced mid-approval (operator, 2026-06-04):** single
|
||||
panoramic projector spanning all three walls, content = nature videos, "we may
|
||||
not need any video manipulation at all." Handled in the sub-project-2 spec by
|
||||
demoting `dominant_color` to optional/opt-in; then (operator-requested) the
|
||||
design spec §6/§7 and ROADMAP were revised and sub-project 5 dropped. Confirmed
|
||||
by the operator during the session.
|
||||
- Operator approved the spec "as written"; remaining §11 defaults taken as
|
||||
accepted: `file_path` relative-to-media-root, two `python -m` CLIs, `index_by_id`
|
||||
in `hef.catalog`. Revisit at implementation if any feels wrong.
|
||||
- **spec-RFC submission gap:** `app.json` has no `contains:["spec-rfc"]` repo, so
|
||||
the spec wasn't submitted to a spec-RFC renderer (it lives in-repo and is
|
||||
pushed). Add a spec-rfc target to `app.json` if rendered specs are wanted later.
|
||||
|
||||
## What lands on the operator's plate
|
||||
|
||||
- Nothing blocking. The spec + plan are approved and merged.
|
||||
- Optional: add a `contains:["spec-rfc"]` repo to `app.json` if you want specs
|
||||
rendered/tracked outside this repo.
|
||||
- The design spec + ROADMAP now describe the single-pano-projector / nature-video
|
||||
direction; confirm that matches your intent when you next look.
|
||||
|
||||
## Prompt the operator can paste into the next session
|
||||
|
||||
```
|
||||
/executing-plans docs/superpowers/plans/2026-06-04-ingest-tagging-review-tools.md
|
||||
```
|
||||
|
||||
This implements sub-project 2 (Ingest & Tagging / Review tools) task-by-task.
|
||||
First read the memory files `sub-project-2-ready-to-build.md` and
|
||||
`pano-projector-nature-video.md`. Task 1 scaffolds `tools/`; Task 2 is the only
|
||||
`hef.catalog` change (`validate_catalog` + `index_by_id`). Needs `ffmpeg`/`ffprobe`
|
||||
installed for the opt-in integration tests; the unit suite is hermetic.
|
||||
@@ -1,66 +0,0 @@
|
||||
# Session 0001.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-04T05-36 (PST)
|
||||
> Type: spec
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0001 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0001.0-TRANSCRIPT-2026-06-04T05-36--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
/goal Develop the spec for sub-project 2 — "Ingest & Tagging / Review tools" — of
|
||||
the Human Experience Filter, the next item in docs/ROADMAP.md. Ground it in
|
||||
docs/superpowers/specs/2026-06-04-human-experience-filter-design.md (especially §5
|
||||
division-of-labor and §8 sourcing) and the roadmap's sub-project-2 section.
|
||||
|
||||
The spec must cover: per-archive ingest fetchers (Internet Archive/Prelinger,
|
||||
Musopen, LibriVox, NASA, Free Music Archive, Freesound); automatic mechanical
|
||||
tagging (mode via ffprobe, license/attribution/source from origin, duration_s,
|
||||
resolution, dominant_color computed from video); assistant coordinate-drafting
|
||||
that writes a one-line rationale and sets review_status=proposed; and the review
|
||||
CLI that walks proposed records (proposed coords + rationale + preview frame) and
|
||||
flips them to approved with reviewed_at. It must build on the existing
|
||||
hef.catalog API without breaking it — call out explicitly any additions the
|
||||
tools need.
|
||||
|
||||
```
|
||||
|
||||
## Plan
|
||||
|
||||
**Spec session — develop the sub-project 2 spec.** Sub-project 1 (Catalog &
|
||||
Selection Core) is done and merged. This session writes the spec for
|
||||
sub-project 2 (**Ingest & Tagging / Review tools**) to
|
||||
`docs/superpowers/specs/`, grounded in the design doc
|
||||
(`2026-06-04-human-experience-filter-design.md`, esp. §5 division-of-labor and
|
||||
§8 sourcing) and the ROADMAP §2 section.
|
||||
|
||||
Scope the spec must cover:
|
||||
1. Per-archive ingest fetchers — Internet Archive/Prelinger, Musopen, LibriVox,
|
||||
NASA, Free Music Archive, Freesound.
|
||||
2. Mechanical tagging — `mode` (ffprobe streams), `license`/`attribution`/
|
||||
`source_*` (from origin), `duration_s`, `resolution`, `dominant_color`
|
||||
(computed from video).
|
||||
3. Assistant coordinate-drafting — propose `left/right/dark/light` + one-line
|
||||
`rationale`, write `review_status: proposed`.
|
||||
4. Review CLI — walk proposed records (coords + rationale + preview frame),
|
||||
accept/correct, flip to `approved`, stamp `reviewed_at`.
|
||||
|
||||
Hard constraint: build on existing `hef.catalog` API without breaking it; call
|
||||
out explicitly any additions the tools need.
|
||||
|
||||
Done when: spec written + self-reviewed + operator-approved; any plan saved
|
||||
under `docs/superpowers/plans/`; working tree clean, merged to `main`, branches
|
||||
deleted, `main` pushed to origin.
|
||||
|
||||
Note: app has no BDD corpus (`BDD_REMOTE` empty) — grounding input is the design
|
||||
spec, not a gherkin corpus.
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
@@ -0,0 +1,103 @@
|
||||
# Session 0002.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: coding
|
||||
> Start: 2026-06-04T06-19 (PST) · End: 2026-06-04T07-34 (PST)
|
||||
> Goal: Execute the sub-project-2 implementation plan (ingest & tagging / review tools).
|
||||
> Outcome: **Shipped.** All 14 plan tasks built TDD-first and merged to `main`
|
||||
> via PR #1; roadmap status updated via PR #2. Suite: 111 passed, 2 skipped.
|
||||
|
||||
## Plan
|
||||
|
||||
Execute `docs/superpowers/plans/2026-06-04-ingest-tagging-review-tools.md` via
|
||||
`superpowers:executing-plans`: build the `tools/` package — per-archive ingest
|
||||
fetchers, mechanical tagging (ffprobe/ffmpeg + license), heuristic coordinate
|
||||
drafting (`review_status=proposed`), and an interactive review CLI that flips
|
||||
records to `approved`. The only `hef/` change is additive: `validate_catalog` +
|
||||
`index_by_id` (spec §3). 14 tasks, TDD throughout; unit suite stays hermetic
|
||||
(network + binaries injected/faked); opt-in real-ffprobe tests skip when absent.
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- Clean `main`, synced with `origin/main`; 37 tests green (sub-project 1:
|
||||
`hef.catalog` + `hef.selection`).
|
||||
- Memory pointer said: spec + plan for sub-project 2 done & merged; next step is
|
||||
`/executing-plans` on the sub-project-2 plan.
|
||||
- ffmpeg/ffprobe **absent** on the build machine (so opt-in integration tests
|
||||
would skip — by design).
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Session gate → coding init.** Classified the `/executing-plans` launch as a
|
||||
coding session; ran `wgl-coding-session-init`. Claimed session **0002** (dry-run
|
||||
peek showed nothing in flight), verified clean `main` baseline, confirmed the
|
||||
`CLAUDE.md` context import.
|
||||
2. **Plan review.** Read the spec + plan + existing `hef/catalog.py`,
|
||||
`hef/selection.py`, `pyproject.toml`, `conftest.py`. Critical review: no
|
||||
blockers — additions are purely additive, all referenced symbols exist.
|
||||
3. **Workspace.** `using-git-worktrees` → chose a **feature branch**
|
||||
(`sub-project-2-ingest-tagging-review`) over a worktree: no parallel session,
|
||||
and `EnterWorktree` is gated to explicit requests. Empty `conftest.py` puts the
|
||||
repo root on `sys.path`; `hef` not installed editable, so the branch exercises
|
||||
its own code with the shared `.venv` (pytest-only dep).
|
||||
4. **Tasks 1–14, TDD (red → green → commit each):**
|
||||
- T1 scaffold `tools/` + `http.py`; T2 `hef.catalog.validate_catalog` +
|
||||
`index_by_id`; T3 `probe.py`; T4 `tagging.py` (cover-art guard +
|
||||
round-half-up duration — see deferred decision); T5 `mediatools.py`
|
||||
(frame + opt-in ffmpeg-only dominant_color); T6 `licensing.py`; T7
|
||||
`drafting.py` `HeuristicProposer`; T8 `ingest/base.py` pipeline; T9
|
||||
LibriVox/NASA/Internet Archive fetchers + Musopen/FMA/Freesound deferred
|
||||
stubs; T10 `ingest_cli.py`; T11 `review.py` transition core; T12
|
||||
`review_cli.py` (I/O seams injectable → scripted accept/edit/skip/quit tests);
|
||||
T13 hermetic e2e + opt-in real-ffprobe (skips cleanly); T14 USER_GUIDE.
|
||||
- One mid-flight catch: a `| tail` masked a pytest failure on T4 so a broken
|
||||
commit landed; fixed the round-half-up logic and `--amend`ed (branch unpushed).
|
||||
5. **Finish branch.** `finishing-a-development-branch` → operator chose **Push PR +
|
||||
auto-merge**. Gitea host (`git.benstull.org`), so no `gh`. No API token existed;
|
||||
searched both keychains (service names only, no secret bytes) — confirmed none
|
||||
for benstull.org. Operator minted + stored a `write:repository` PAT via
|
||||
`pbpaste` into `wgl-gitea-token-git.benstull.org`. Created **PR #1** and merged
|
||||
it (merge commit `d7a2cba`), deleted the branch, FF'd local `main`.
|
||||
6. **Finalize.** Survey clean. Updated memory. Marked the roadmap (§2 ✅ done,
|
||||
§3 ⏳ next) via **PR #2** (`593150b`). Published this transcript.
|
||||
|
||||
## Cut state (end of session)
|
||||
|
||||
- Branch: `main` @ `593150b`, clean, synced with `origin/main`.
|
||||
- **PR #1** (`d7a2cba`) — sub-project 2 `tools/` package + additive `hef.catalog`
|
||||
symbols + USER_GUIDE. 14 commits.
|
||||
- **PR #2** (`593150b`) — roadmap status update.
|
||||
- Tests: `python -m pytest -q` → **111 passed, 2 skipped** (opt-in real-ffprobe;
|
||||
ffmpeg absent here). Existing catalog still validates under `validate_catalog`.
|
||||
- New surface: `tools/{http,probe,tagging,mediatools,licensing,drafting,review,
|
||||
review_cli,ingest_cli}.py`, `tools/ingest/{base,librivox,nasa,internet_archive,
|
||||
musopen,fma,freesound}.py`; `hef.catalog` gained exactly `validate_catalog` +
|
||||
`index_by_id`.
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **Task 4 duration rounding (low risk):** spec/plan say `round(float(duration))`
|
||||
and the plan's test expects `12.5 → 13`. Python's `round()` uses banker's
|
||||
rounding (`round(12.5) == 12`), so implemented round-half-up
|
||||
(`math.floor(x + 0.5)`) to match the plan's stated behavior. Chose the
|
||||
documented expectation over the literal function name. Confirm or redirect if
|
||||
truncation/banker's was actually intended.
|
||||
|
||||
## What lands on the operator's plate
|
||||
|
||||
- A `write:repository` Gitea PAT for `git.benstull.org` now lives in the Keychain
|
||||
(`wgl-gitea-token-git.benstull.org`) — reused for future PR automation; revoke
|
||||
in the Gitea UI if/when you want it gone.
|
||||
- Live ingest/review need `ffmpeg`/`ffprobe` installed (absent on this machine);
|
||||
the unit suite does not. Freesound (when un-deferred) needs
|
||||
`FREESOUND_API_TOKEN` (env-only secret).
|
||||
|
||||
## Prompt the operator can paste into the next session
|
||||
|
||||
Sub-project 3 (Pi player) has no plan yet and §3 carries open decisions (player
|
||||
stack, approved-only enforcement, serial protocol contract) — so it likely opens
|
||||
as a spec/discovery session before coding.
|
||||
|
||||
```
|
||||
/goal Develop sub-project 3 — the Pi Player Runtime that drives the single panoramic projector from the approved catalog — starting from its open decisions (player stack: mpv-IPC vs ffmpeg vs custom; approved-only enforcement; serial-protocol contract with sub-project 4; file_path-vs-media-mount resolution), per docs/ROADMAP.md §3. Read memory sub-project-2-ready-to-build.md and pano-projector-nature-video.md first.
|
||||
```
|
||||
@@ -0,0 +1,99 @@
|
||||
# Session 0003.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-04T07-57 (PST)
|
||||
> End: 2026-06-05T03-36 (PST)
|
||||
> Type: discovery
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
> We should create a simulator, just something that is web-based, potentially that
|
||||
> runs on localhost, running in a Docker container. It allows us to play virtually
|
||||
> with the dials and see the content that has been curated, and then we can get to
|
||||
> the actual physical stack. I think we want to make sure that we have the
|
||||
> principles well done and the interface figured out, and then the middle will come
|
||||
> in to play.
|
||||
|
||||
## Plan
|
||||
|
||||
Discovery session: explore the proposed **web experience simulator** and produce
|
||||
its first artifacts — BDD scenarios and/or a spec — plus (chosen mid-session) an
|
||||
implementation plan. Orientation:
|
||||
|
||||
```
|
||||
/goal Discover the web-based experience simulator — produce BDD scenarios and/or
|
||||
a spec, per human-experience-filter-art/docs/ROADMAP.md
|
||||
```
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` clean and pushed at `b7f19e0`. Sub-projects 1 (catalog/selection) and 2
|
||||
(ingest/review) shipped; physical stack (3 Pi player, 4 firmware) ahead.
|
||||
- `catalog/library.jsonl` empty (0 bytes) — tooling exists, no media ingested.
|
||||
- No prior simulator artifacts. Memory pointed at sub-project 3 as next.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Session gate / routing.** Opening prompt straddled coding vs. discovery;
|
||||
asked the operator → **discovery**. Ran `wgl-discovery-session-init`: claimed
|
||||
ID **0003** (`--type discovery`), verified clean pushed `main`, read memory,
|
||||
oriented from `docs/ROADMAP.md`.
|
||||
2. **Brainstorming.** Accepted the visual companion (server on :61057). Settled
|
||||
scope via one-at-a-time questions:
|
||||
- Primary purpose → **feel the selection model** (curator's instrument).
|
||||
- Catalog source → **synthetic fixture catalog** (real catalog is empty).
|
||||
- Screen layout (browser mockups A/B/C) → **B · Curator's X-ray** (full
|
||||
transparency: pick + ranked pool + distances + brain/mood grid maps).
|
||||
- Approved the **additive `hef.selection.ranked_candidates()`** change.
|
||||
- **Include** the model knobs (weights, pool size, approved-only).
|
||||
3. **Architecture.** Chose **Approach 1** — FastAPI + vanilla-JS, real
|
||||
`hef.selection` as single source of truth; Docker on localhost.
|
||||
4. **Artifacts written + committed** (`76a9526`): design spec
|
||||
`docs/superpowers/specs/2026-06-04-experience-simulator-design.md` + 4 BDD
|
||||
feature files under `features/`. Self-reviewed; fixed the pick-determinism
|
||||
ambiguity (deterministic `rng=None` pick; pool = the room's shuffle-set).
|
||||
Operator approved the spec.
|
||||
5. **Implementation plan** (chose "draft the plan now"): wrote
|
||||
`docs/superpowers/plans/2026-06-04-experience-simulator.md` — 7 bite-sized TDD
|
||||
tasks; self-reviewed for spec coverage / placeholders / type consistency.
|
||||
Committed (`e7ab227`).
|
||||
6. **Memory + handoff.** Saved resume pointer
|
||||
`experience-simulator-ready-to-build.md` (+ MEMORY.md index) so the next
|
||||
coding session auto-surfaces the `/executing-plans` line.
|
||||
7. **Finalize.** Push raced twice against concurrent session-claim commits (0003
|
||||
then 0004); rebased onto each (disjoint paths, non-destructive) and pushed.
|
||||
|
||||
## Cut state (end of session)
|
||||
|
||||
- `main` synced with origin. Discovery artifacts landed:
|
||||
- spec `40d7434`→`9cb4fb6` (rebased): `docs/superpowers/specs/2026-06-04-experience-simulator-design.md` + `features/*.feature`
|
||||
- plan `19a2dbc`→`ddec45d` (rebased): `docs/superpowers/plans/2026-06-04-experience-simulator.md`
|
||||
- Final `main` tip: **`ddec45d`**.
|
||||
- No code written this session (discovery only). Test suite untouched (baseline
|
||||
111 passed / 2 skipped).
|
||||
- Visual-companion files persist under `.superpowers/brainstorm/` (gitignored).
|
||||
|
||||
## What lands on the operator's plate
|
||||
|
||||
- **Build is queued, not started.** The simulator exists only as spec + BDDs +
|
||||
plan. Next session executes the plan.
|
||||
- **Structural gap (flagged, not blocking):** this app's `app.json` has no
|
||||
`contains:["bdd"]` or `contains:["spec-rfc"]` repo, so there is no external
|
||||
corpus/spec-RFC target — the discovery artifacts live in-repo (committed) by
|
||||
design. If a dedicated BDD/spec-RFC repo is ever wanted, register it in
|
||||
`app.json`.
|
||||
- **Concurrent session 0004** was claimed during this finalize and may still be
|
||||
in flight.
|
||||
- No deferred (low-confidence autonomous) decisions this session.
|
||||
|
||||
## Prompt the operator can paste into the next session
|
||||
|
||||
```
|
||||
/goal Build the experience simulator — execute the plan task-by-task:
|
||||
/executing-plans docs/superpowers/plans/2026-06-04-experience-simulator.md
|
||||
```
|
||||
|
||||
Read `memory/experience-simulator-ready-to-build.md` first. The plan recommends
|
||||
subagent-driven execution with review between tasks; the only shipped-code change
|
||||
is the additive `hef.selection.ranked_candidates()` (Task 1).
|
||||
@@ -0,0 +1,125 @@
|
||||
# Session 0004.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: coding
|
||||
> Start: 2026-06-05T03-32 (PST) · End: 2026-06-05T03-56 (PST)
|
||||
> Goal: Execute the experience-simulator (curator's X-ray) implementation plan.
|
||||
> Outcome: **Shipped.** All 7 plan tasks built TDD-first and merged to `main`
|
||||
> via PR #3. Suite: 133 passed, 2 skipped. Docker image builds + serves.
|
||||
|
||||
## Plan
|
||||
|
||||
Execute `docs/superpowers/plans/2026-06-04-experience-simulator.md` via
|
||||
`superpowers:executing-plans`: build the `simulator/` package — a FastAPI service
|
||||
wiring the five dials to the **real** `hef.selection` code and serving a
|
||||
vanilla-JS "X-ray" (picked piece + ranked candidate pool with distances +
|
||||
brain/mood 5×5 maps), fed by a deterministic synthetic fixture catalog since
|
||||
`catalog/library.jsonl` is empty. The only `hef/` change is additive:
|
||||
`hef.selection.ranked_candidates()` with `select()` refactored onto it. 7 tasks,
|
||||
TDD throughout; Docker + compose + `make` targets package it for localhost.
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- Clean `main`, synced with `origin/main` at `ddec45d`; 111 tests green, 2 skipped.
|
||||
- Memory pointer: simulator spec + BDDs + plan produced in discovery 0003
|
||||
(2026-06-04); next coding session runs `/executing-plans` on the simulator plan.
|
||||
- Discovery session **0003** still `--INPROGRESS` (committed its artifacts to main
|
||||
but never finalized) — dormant, not actively editing the tree.
|
||||
- `fastapi` not yet installed; `python` not on PATH (use `.venv/bin/python`).
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Session gate → coding init.** Classified the `/executing-plans` launch as a
|
||||
coding session; ran `wgl-session-coding-init`. Dry-run peek surfaced the
|
||||
in-flight 0003 placeholder; autonomous default → noted it and proceeded.
|
||||
Claimed session **0004**. Reconciled an initially-stale "ahead 2" read:
|
||||
local `main` == `origin/main` == `ddec45d` (the 0003 spec/BDD/plan commits are
|
||||
on origin). Verified clean baseline: **111 passed, 2 skipped**.
|
||||
2. **Plan review.** Read the plan + `hef/selection.py`, `hef/catalog.py`,
|
||||
`pyproject.toml`. Critical review: all assumed symbols exist
|
||||
(`Coordinate`/`Weights`/`distance`/`candidates_for_mode`/`select`,
|
||||
`CONTENT_MODES`/`SELECTOR_MODES`; `Record` carries every fixture field). No
|
||||
blockers. Confirmed Docker present, `docs/USER_GUIDE.md` + `features/` exist.
|
||||
3. **Workspace.** `using-git-worktrees` → native `EnterWorktree`
|
||||
(`worktree-experience-simulator`, branched from `origin/main`). Fresh `.venv`
|
||||
in the worktree; base package + pytest installed; baseline re-confirmed
|
||||
**111 passed, 2 skipped**.
|
||||
4. **Tasks 1–7, TDD (red → green → commit each):**
|
||||
- **T1** `hef.selection.ranked_candidates()` + `select()` refactored onto it
|
||||
(single source of truth; behavior-preserving) — 24 passed (new + existing
|
||||
selection suite unchanged).
|
||||
- **T2** declare `simulator` package + `[project.optional-dependencies] sim`
|
||||
(fastapi/uvicorn/httpx); installed `-e ".[sim]"`; imports resolve.
|
||||
- **T3** `simulator/fixtures.py` — deterministic 625-cell synthetic catalog
|
||||
(all modes, mixed review status, no media) — 6 passed.
|
||||
- **T4** `simulator/app.py` — FastAPI `create_app`, `POST /api/select`
|
||||
(pick + ranked pool with distance/rank + coverage), `GET /api/catalog/meta`,
|
||||
real-catalog-or-fixtures loader, guarded static mount — 6 passed.
|
||||
- **T5** `simulator/static/` X-ray UI (dials, model knobs, ranked pool,
|
||||
brain/mood maps) + static smoke test — 7 passed. Manual browser eyeball
|
||||
substituted with live `curl` verification (no interactive browser): `/` 200,
|
||||
`/api/catalog/meta` 625 records all-modes/mixed-status, `/api/select` ranked
|
||||
pool ascending distances (1.0 → 1.41…), `none` → void, static assets 200.
|
||||
- **T6** Dockerfile + compose + `Makefile` (literal-tab recipes). First build
|
||||
**failed**: `package directory 'tools' does not exist` — `tools` is a
|
||||
declared setuptools package but wasn't copied into the image. Root-caused
|
||||
from the build log; fixed by adding `COPY tools ./tools`. Rebuilt → container
|
||||
serves (`/` 200, 625-record fixture catalog). Torn down.
|
||||
- **T7** USER_GUIDE "Playing with the simulator" section; **full suite: 133
|
||||
passed, 2 skipped** (baseline 111 + 22 new).
|
||||
5. **Finish branch.** `finishing-a-development-branch` (autonomous): pushed branch
|
||||
as `experience-simulator`; created **PR #3** via the Gitea API
|
||||
(`wgl-gitea-admin` token mechanism, `git.benstull.org`); merged it (merge
|
||||
commit `618b691`); deleted the merged remote branch; FF'd local `main`;
|
||||
re-verified the merged result **133 passed, 2 skipped**; removed the worktree
|
||||
(all 7 commits confirmed ancestors of `main` first).
|
||||
6. **Finalize.** Added `*.egg-info/` to `.gitignore` (editable-install artifact)
|
||||
and pushed to `main` (`40dfbfd`); updated memory; published this transcript.
|
||||
|
||||
## Cut state (final)
|
||||
|
||||
- `main` @ `40dfbfd`, clean, synced with `origin/main`. No open PRs, no dangling
|
||||
branches, worktree removed.
|
||||
- **133 passed, 2 skipped.** Docker `make sim` builds and serves; `make sim-local`
|
||||
runs uvicorn. Simulator is operator-runnable on localhost.
|
||||
- The simulator is a **curator's discovery tool**, not a roadmap sub-project — the
|
||||
roadmap frontier is unchanged.
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
Autonomous-mode calls made without operator input:
|
||||
|
||||
1. **Dockerfile `COPY tools ./tools` (plan deviation).** The plan's Dockerfile
|
||||
copied only `hef`/`simulator`/`catalog`, but `pyproject.toml` declares
|
||||
`packages = ["hef", "tools", "simulator"]`, so the in-image editable install
|
||||
failed. Added the missing COPY. *Alternative:* trim `tools` from the package
|
||||
list — rejected (it's a real, used package). High confidence; the image now
|
||||
matches the declared packages.
|
||||
2. **Manual browser eyeball → programmatic curl (T5 Step 7).** No interactive
|
||||
browser available, so verified the running server with `curl` (HTML, meta,
|
||||
`/api/select`, void, static assets) instead of visually. Behavior fully
|
||||
exercised, but no human visual confirmation of the rendered UI.
|
||||
3. **Autonomous PR-merge instead of the interactive finish menu.** Per §6.5
|
||||
autonomous default + §5.4 (branch→PR→merge), pushed + opened + merged PR #3
|
||||
without surfacing the 4-option finish prompt. The preferred path was
|
||||
unambiguous and the merge is revertible.
|
||||
4. **`*.egg-info/` gitignore committed straight to `main`.** Trivial one-line
|
||||
hygiene; landed directly rather than via PR.
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
The simulator is shipped. The roadmap's next milestone is **sub-project 3
|
||||
(Player Runtime / Pi)** — no spec/plan yet, so it starts with discovery.
|
||||
|
||||
```
|
||||
/goal Develop sub-project 3 (Player Runtime / Pi): read the controls, call hef.selection.select(), play the chosen segment across the single panoramic projector, crossfade on change, dark on None — start with a discovery session to produce BDDs + spec, per docs/ROADMAP.md §3
|
||||
```
|
||||
|
||||
Read first: memory `experience-simulator-ready-to-build.md` (what shipped + the
|
||||
Dockerfile-copies-every-package gotcha), `pano-projector-nature-video.md` (single
|
||||
pano projector, sub-project 5 dropped), and `docs/ROADMAP.md §3`. Open decisions
|
||||
to settle: player stack (mpv/ffmpeg/custom), `approved`-only enforcement, and the
|
||||
serial protocol framing shared with sub-project 4.
|
||||
|
||||
> Housekeeping: discovery session **0003** is still `--INPROGRESS` in
|
||||
> `sessions/0003/` (artifacts long since merged) — finalize or close it out.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Session 0005.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: discovery
|
||||
> Start: 2026-06-05T08-30 (PST) · End: 2026-06-05T17-48 (PST)
|
||||
> Goal: Explore whether AI-generated/altered video is a viable content path for the
|
||||
> pano projector — economics + fit with the design — and capture the outcome.
|
||||
> Outcome: **Artistic thesis pivoted** and a full design-spec revision written,
|
||||
> reviewed, and **merged to `main` via PR #4** (merge commit `77746e4`).
|
||||
> Implementation deferred to a sub-project-3 coding session.
|
||||
|
||||
## Plan
|
||||
|
||||
Discovery session opened from the operator's question: "can we write/generate
|
||||
scene descriptions and generate the video with AI — would the economics work?"
|
||||
The session ran as a brainstorming exploration that converged on a new thesis and
|
||||
produced a design spec as its artifact.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
I'm not sure if hte economics would work out (can you do a best effort calculation?) but maybe we could write up/generate scene descriptions and generate the video with AI. What do you think?
|
||||
```
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` @ `3510236`. Sub-projects 1 & 2 done/merged; roadmap frontier = sub-project
|
||||
3 (Pi player). A stale, already-merged worktree (`feature/BuildSimulator11`,
|
||||
same commit as main) was the session's launch cwd.
|
||||
- Design of record: `2026-06-04-human-experience-filter-design.md` — single pano
|
||||
projector + **found** nature video; §1 constraint "real found media, **not
|
||||
algorithmically generated filler. Curation is the artwork.**"
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Gate / classify.** Operator's prompt was ambiguous between discovery and a
|
||||
quick chat; asked → operator chose **Discovery (tracked)**. Ran
|
||||
`wgl-session-discovery-init`.
|
||||
2. **Claim snag (kept in the record).** `resolve-app.py` reported **ambiguous** —
|
||||
the nested worktree's `app.json` (`.worktrees/feature-BuildSimulator11/app.json`)
|
||||
was discovered alongside the canonical main-repo `app.json`, both listing the
|
||||
repo. Worked around with `WGL_APP_SCAN_DEPTH=3` (excludes the depth-5 worktree
|
||||
copy). Claimed session **0005**, type discovery, at `850f67a`.
|
||||
3. **Economics, v1 (the original question).** Researched 2026 AI-video API rates
|
||||
(Sora 2 ~$0.10/s, Kling 3.0 ~$0.10/s, Veo 3.1 Lite/Runway Turbo ~$0.05/s, Veo 3
|
||||
+audio ~$0.75/s). At the spec's scale (120–800 pieces × ~10-min segments) →
|
||||
**~$15k–45k+** with retries; **not viable**, and it doesn't reduce the real
|
||||
constraint (curation labor). Also flagged the §1 thesis conflict ("not generated
|
||||
filler").
|
||||
4. **Thesis pivot.** Operator: willing to change the thesis — the piece is "about
|
||||
**how humans interact with machines and how it impacts their nervous systems**."
|
||||
Under that thesis AI alteration *is* the subject. Invoked `brainstorming`.
|
||||
Scope = **re-skin the meaning** (keep apparatus); content = **AI-altered
|
||||
primary**.
|
||||
5. **Economics, v2 (the flip).** New thesis unlocks two cost levers: small
|
||||
**generate-to-order** library + short **seamless loops** → **~$300–3k**. Feasible.
|
||||
6. **The strong concept crystallizes.** Operator: start from **neutral** real
|
||||
nature footage and let the machine **alter it toward the knob state** — "a great
|
||||
expression of how machines alter our experience, good and bad." Established the
|
||||
per-axis mapping (Left = analytical overlay; Right = generative v2v dissolve;
|
||||
Dark/Light = color grade), the **substrate-vs-overlay composition rule**, and the
|
||||
"whole-brain" corner (feeling, *labeled*).
|
||||
7. **Control panel + i18n.** Operator added a 7-way content dial (Off/White
|
||||
Noise/Music/Audio Track/Video/Music+Video/Audio Track+Video), volume + brightness
|
||||
knobs, and a tactile **wooden** panel (engraved symbols, braille, color LEDs,
|
||||
read-aloud button). Designed touch-distinct/engravable symbols. Key finding on
|
||||
the **translation cost**: because the Left **labels are a runtime overlay**, the
|
||||
v2v renders once (language-agnostic) and supporting *every* language is ~free on
|
||||
the video — only cheap text translation + TTS. (Baking labels in would multiply
|
||||
video cost × #languages — the avoided landmine.)
|
||||
8. **Mood color grade.** Operator: Light = yellow / white negative space; center =
|
||||
raw ungraded video; Dark = blue / black negative space. (First draft mis-set
|
||||
Dark=red and center=grey; operator corrected → Dark = melancholy **blue** not
|
||||
alarm red; center = **raw video**, no grade.)
|
||||
9. **Wrote the spec.** Branched `feature/machine-altered-perception-spec` off main,
|
||||
wrote `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md`
|
||||
(15 sections), committed, self-reviewed; operator reviewed and gave 4
|
||||
corrections (center=raw, add Music+Video → 7-way, audio-alteration deferred,
|
||||
red gone) → applied, commit amended to `c5952c6`.
|
||||
10. **Wrap.** Operator chose to wrap the discovery session here (spec is the
|
||||
artifact; build later). Pushed branch; first PR-creation attempt was **blocked
|
||||
by the auto-mode classifier** (treated as routing around PR authorization) —
|
||||
surfaced it rather than working around. Operator then explicitly authorized
|
||||
opening + merging the PR → **PR #4 opened and merged to `main`** (merge commit
|
||||
`77746e4`, branch deleted), local `main` fast-forwarded.
|
||||
|
||||
## Cut state (end of session)
|
||||
|
||||
| Item | State |
|
||||
|---|---|
|
||||
| Design spec | **Merged to `main`** — `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` |
|
||||
| Branch commit | `c5952c6` (spec, amended with review fixes) |
|
||||
| Merge commit | `77746e4` (PR #4), feature branch deleted |
|
||||
| `main` | fast-forwarded to `77746e4` (local + origin) |
|
||||
| Roadmap | unchanged on disk; spec §13 describes how sub-projects 2/3/4 + a new offline v2v pipeline are reshaped (a roadmap-edit pass is future work) |
|
||||
| Memory | `ai-generated-content-thesis-pivot.md` (full design rationale + status) + `MEMORY.md` index updated |
|
||||
|
||||
## What lands on the operator's plate
|
||||
|
||||
- Nothing blocking. The spec is canonical and merged.
|
||||
- Optional future passes the spec itself lists (§14): pano resolution strategy,
|
||||
negative-space luma key, transform composition order, neutral-base count,
|
||||
language tier, runtime-grading perf on the Pi.
|
||||
- A roadmap-edit pass to reflect the reshaped sub-projects 2/3/4 (the design spec
|
||||
carries it for now).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
No substantive low-confidence design calls — every design decision (thesis,
|
||||
per-axis mapping, composition rule, color grade, the 7-way dial, the panel) was
|
||||
operator-confirmed in-session. Two mechanical judgment calls worth flagging:
|
||||
|
||||
- **`WGL_APP_SCAN_DEPTH=3` workaround** for the nested-worktree `app.json`
|
||||
ambiguity in `resolve-app.py`. This is a genuine plugin friction (a worktree
|
||||
nested under the main repo makes the resolver ambiguous) — a candidate for
|
||||
`wgl-dev-plugin-feedback` (resolver should skip `.worktrees/`).
|
||||
- **Repurposed the stale `feature/BuildSimulator11` worktree** to a fresh
|
||||
`feature/machine-altered-perception-spec` branch off its HEAD (= main) rather
|
||||
than removing it (harness had cwd anchored there; removal risked breaking the
|
||||
shell).
|
||||
|
||||
## Prompt the operator can paste into the next session
|
||||
|
||||
```
|
||||
/goal Implement sub-project 3 (the player / alteration engine) per docs/ROADMAP.md and the new docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md. Read memory ai-generated-content-thesis-pivot.md first. Build the alteration engine: runtime luma-keyed mood grade (Light=yellow/white → raw → Dark=blue/black), runtime analytical-overlay compositor (multilingual, from string tables), generated white noise, and selection/blending of pre-baked v2v variants over a small neutral base library; resolve the spec §14 open questions (pano resolution, composition order) as you go.
|
||||
```
|
||||
@@ -0,0 +1,134 @@
|
||||
# Session 0006.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: coding
|
||||
> Start: 2026-06-05T17-52 (PST) · End: 2026-06-05T18-15 (PST)
|
||||
> Goal: `/goal next` — drive the roadmap to its next frontier item (sub-project 3,
|
||||
> the player / machine-alteration engine), and build it.
|
||||
> Outcome: **Sub-project 3 slice 1 — the pure-logic alteration-engine + player
|
||||
> core — built and MERGED to `main` via PR #5** (merge commit `aeeed66`).
|
||||
> New `player/` package; 56 new tests (189 passed / 2 skipped total).
|
||||
> Sub-project 3 advanced from "next" to "in progress."
|
||||
|
||||
## Plan
|
||||
|
||||
Begin sub-project 3 (Player Runtime + alteration engine) per `docs/ROADMAP.md` §3
|
||||
and the approved machine-altered-perception design (§4 alteration engine, §5 mood
|
||||
grade, §6 7-way content dial, §7 intensity). Sub-project 3 is large (runtime
|
||||
grading + overlay compositing + v2v variant selection + serial input + mpv/GPU +
|
||||
white-noise DSP); this session ships **slice 1 — the pure-logic core** as a new
|
||||
`player/` package, fully unit-tested with all I/O behind injected interfaces,
|
||||
mirroring how sub-project 1 was the pure-logic dependency root. Process:
|
||||
writing-plans → TDD → branch → PR → merge.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
/goal next — drive the roadmap to its next frontier item.
|
||||
|
||||
Per project memory (session 0005, machine-altered-perception design SPEC merged
|
||||
to main via PR #4), the roadmap frontier is sub-project 3: the player /
|
||||
machine-alteration engine. This coding session builds that.
|
||||
```
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` @ `77746e4` locally (behind `origin/main` by 2 — session 0005's finalized
|
||||
transcript + this session's claim placeholder); fast-forwarded to `origin/main`
|
||||
during init.
|
||||
- Sub-projects 1 & 2 done/merged; design of record = the machine-altered-perception
|
||||
revision (PR #4). No `player/` package and no sub-project-3 plan existed yet.
|
||||
- Launch cwd was the stale, already-merged worktree
|
||||
`.worktrees/feature-BuildSimulator11/` (branch `feature/machine-altered-perception-spec`,
|
||||
remote gone) — same situation as session 0005.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Gate / classify.** `/goal next` → unambiguously a coding session (roadmap
|
||||
frontier = sub-project 3 build). Ran `wgl-session-coding-init`.
|
||||
2. **Claim snag (same as 0005, kept in the record).** `resolve-app.py` reported
|
||||
**ambiguous** — the nested worktree's `app.json` discovered alongside the main
|
||||
clone's. Read the resolver to confirm root cause (`find_app_jsons` prunes
|
||||
`.git/node_modules/…` but not `.worktrees/`). Worked around with
|
||||
`WGL_APP_SCAN_DEPTH=4` (prunes the depth-5 worktree copy, keeps the depth-3 main
|
||||
one). Claimed session **0006**, type coding, at `e694489`. Filed a plugin-feedback
|
||||
note for the recurring gap (`feedback/2026-06-05T18-13-resolver-nested-worktree-ambiguous.md`).
|
||||
3. **Baseline + branch.** Fast-forwarded the main clone; confirmed PR #4 merged;
|
||||
created `feature/player-alteration-core` off `origin/main` in the worktree.
|
||||
4. **Read the design.** Read ROADMAP §3 + the machine-altered-perception spec
|
||||
(§4 alteration engine, §5 mood grade, §6 7-way dial, §7 intensity), and the
|
||||
existing `hef/` conventions (frozen dataclasses, pure modules, heavy unit tests).
|
||||
5. **Scoped the session → slice 1.** Sub-project 3 is too large for one session;
|
||||
chose the pure-logic core (controls model, content-dial resolver, alteration
|
||||
render-plan, player state machine) with all I/O behind injected interfaces, and
|
||||
deferred the renderer/serial/white-noise/v2v/catalog slices (named in the plan).
|
||||
6. **Surfaced a spec ambiguity (Deferred decision).** §3 "(2,2,2,2) neutral" vs
|
||||
§4.2 (Left/Right stack as independent layers) vs §5 (mood center = identity)
|
||||
cannot all be literally true for four independent knobs. Resolved with a
|
||||
swappable calibration: brain `value/4`, mood `(light−dark)/4`; flagged for
|
||||
operator confirm.
|
||||
7. **writing-plans.** Wrote `docs/superpowers/plans/2026-06-05-player-alteration-core.md`
|
||||
(6 TDD tasks, full code + tests, self-review against the spec).
|
||||
8. **TDD implementation.** Red→green→commit per module: `player/controls.py`
|
||||
(Controls, validation, parse), `player/content.py` (§6 7-way table),
|
||||
`player/alteration.py` (RenderPlan: ColorGrade/AnalyticalOverlay/Restyle +
|
||||
calibration helpers), `player/state.py` (Player state machine + Transition
|
||||
kinds). Registered the package in `pyproject.toml`; updated ROADMAP §3.
|
||||
9. **Verify.** Full suite green: **189 passed, 2 skipped** (the opt-in ffprobe
|
||||
tests), 56 of them new player tests. No regressions.
|
||||
10. **Ship.** Pushed the branch; created **PR #5** and merged it via the Gitea API
|
||||
helper (merge commit `aeeed66`); synced the main clone; deleted the remote
|
||||
feature branch.
|
||||
11. **Wrap.** Filed plugin feedback, updated memory, finalized.
|
||||
|
||||
## Cut state
|
||||
|
||||
- `main` @ `aeeed66` (PR #5 merged). `player/` package present: `controls.py`,
|
||||
`content.py`, `alteration.py`, `state.py` + 4 test files. Full suite 189 passed /
|
||||
2 skipped.
|
||||
- Sub-project 3 = **⏳ in progress**. Slice 1 done; remaining slices listed in
|
||||
ROADMAP §3 and `[[sub-project-3-player-progress]]`.
|
||||
- Working tree clean. Remote feature branch deleted; the local worktree remains on
|
||||
the now-merged `feature/player-alteration-core` branch (harmless).
|
||||
|
||||
## Deferred decisions (operator plate)
|
||||
|
||||
Two low-confidence calls made autonomously this session — both flagged for operator
|
||||
confirmation:
|
||||
|
||||
1. **Knob→strength calibration (§3 vs §4.2/§5 conflict).** §3 says "(2,2,2,2) is
|
||||
neutral," but §4.2 (Left/Right stack as independent layers: overlay vs substrate)
|
||||
and §5 (mood center = identity) conflict with that for four independent physical
|
||||
knobs. Implemented: brain knobs `strength = value/4` (0 = off, 4 = max — full
|
||||
range, honors §4.2 stacking), mood `tone = (light − dark)/4` (equal dark/light →
|
||||
identity, honors §5). Under this the un-altered base is `(0,0,·,·)` with
|
||||
`dark==light`, **not** `(2,2,2,2)` — read as a vestige of the old coordinate-grid
|
||||
*center*. The calibration is three one-line helpers in `player/alteration.py`;
|
||||
flipping to a "centered at 2 = no push" convention is a localized change that
|
||||
leaves the RenderPlan shape and all downstream logic untouched. **Recommend
|
||||
operator confirm the intended panel UX.**
|
||||
2. **Volume/Brightness granularity.** Modeled as `0..4` ints for uniformity with the
|
||||
experience knobs and the Arduino's `0..4` quantization (roadmap §4). The spec
|
||||
calls them "levels" without fixing granularity; revisit if the panel warrants
|
||||
finer steps.
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
The roadmap frontier is now the remaining sub-project-3 slices. Read
|
||||
`[[sub-project-3-player-progress]]` and the slice-1 plan first. Two strong
|
||||
candidates for the next slice — the **runtime renderer** (makes the core visible)
|
||||
or the **serial framing contract + input adapter** (unblocks sub-project 4 in
|
||||
parallel):
|
||||
|
||||
```
|
||||
/goal Sub-project 3 slice 2 — settle the 3⇄4 serial framing contract and build the
|
||||
USB-serial / keyboard input adapter that feeds the player a `Controls` stream
|
||||
(unblocks sub-project 4), per docs/ROADMAP.md §3 and docs/superpowers/plans/2026-06-05-player-alteration-core.md.
|
||||
Alternatively start the runtime renderer (mpv/ffmpeg + grade/overlay). First
|
||||
confirm the open knob→strength calibration decision (§3 vs §4.2/§5).
|
||||
```
|
||||
|
||||
Gotcha to carry forward: running from the nested `.worktrees/…` checkout breaks
|
||||
`resolve-app.py` (ambiguous app.json) — run from the main clone, or use
|
||||
`WGL_APP_SCAN_DEPTH=4` on claim and explicit `--sessions-*` flags on publish, until
|
||||
the plugin fix lands.
|
||||
@@ -0,0 +1,101 @@
|
||||
# Session 0007.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Date: 2026-06-06 (PST) · 14:12 → 17:57
|
||||
> Type: coding
|
||||
> Goal: continue the roadmap; reframed by the operator to **simulator-first** —
|
||||
> bring the slice-1 alteration engine into the simulator so the experience can be
|
||||
> tuned before any hardware.
|
||||
> Outcome: **design spec written, committed, and pushed** on branch
|
||||
> `feature/simulator-alteration-preview`. No code changes. Implementation deferred
|
||||
> to the next session (design + implementation to ship as one PR).
|
||||
|
||||
## Plan
|
||||
|
||||
Bring the alteration engine into the simulator (scope: tune the look), per
|
||||
`docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md`:
|
||||
- Python-canonical engine, thin browser renderer.
|
||||
- Dark/Light = live runtime color grade; Left/Right = 5×5 grid of pre-baked
|
||||
authored variant clips selected by coordinate (24 videos/base clip).
|
||||
- Revises design §4.3/§10: HUD baked into variant videos (authorial precision)
|
||||
rather than runtime overlay — trades away near-free i18n.
|
||||
- Engine reconciliation: parameterize `Calibration`, add `VariantRef`, slim
|
||||
`RenderPlan` (drop runtime `AnalyticalOverlay`/`Restyle`); `state.py` crossfade
|
||||
on variant change.
|
||||
- Simulator: retire selection X-ray + `/api/select`; add `/api/alteration` +
|
||||
`/api/clips` + variant manifest; placeholder-variant generator.
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` at `5290785`, clean and pushed. Sub-projects 1 & 2 done; sub-project 3
|
||||
(Player Runtime) in progress — slice 1 (pure-logic `player/` core) merged via
|
||||
PR #5 (session 0006).
|
||||
- A **stale nested worktree** (`.worktrees/feature-BuildSimulator11`, branch
|
||||
`feature/player-alteration-core`, already merged) was present and breaking the
|
||||
session resolver ("ambiguous app.json"), exactly as the session-0006 memory
|
||||
gotcha predicted.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Session gate.** Opened with "What's next?" → classified as a coding session;
|
||||
ran `wgl-session-coding-init`.
|
||||
2. **Baseline cleanup.** Claim dry-run failed on the resolver ambiguity. Verified
|
||||
the stale worktree's branch was fully merged + clean, removed the worktree and
|
||||
branch, and deleted the empty `.worktrees/` dir. Tree back to clean `main`.
|
||||
3. **Claimed session 0007** (no other sessions in flight).
|
||||
4. **Operator steer:** "only design things that will run in the simulator; get
|
||||
everything working the way we like in the simulator before hardware." Saved as
|
||||
a `feedback` memory ([[simulator-first-before-hardware]]).
|
||||
5. **Discovered** the simulator (`simulator/`, built session 0004) is fully
|
||||
decoupled from `player/` — it exercises the OLD *selection* model and has no
|
||||
alteration surface. That gap became the work.
|
||||
6. **Brainstorming** (with the visual companion). Decisions, one at a time:
|
||||
scope = tune the alteration look; medium = short looping video; Right restyle =
|
||||
labeled stand-in shader (swappable for real variants); calibration =
|
||||
live-adjustable curves; architecture = **A, Python-canonical**; integration =
|
||||
new view, **retire the selection X-ray**.
|
||||
7. **Interactive look mockup** (browser): a real nature image with the 4 knobs as
|
||||
live sliders. Operator confirmed Dark/Light read well, but said for Left/Right
|
||||
they'll **author videos with the overlay baked in** for precise HUD control.
|
||||
8. **Surfaced the tradeoff:** baking the HUD reverses design §4.3/§10's near-free
|
||||
i18n. Operator chose **precision wins**. Confirmed the variant grid as **full
|
||||
5×5 = 24 authored clips per base** (shown as a grid mockup).
|
||||
9. **Revised model** presented and approved: base clip + variant manifest;
|
||||
`RenderPlan = {grade, variant}`; runtime overlay/restyle layers retired;
|
||||
placeholder-variant bootstrapping via ffmpeg.
|
||||
10. **Wrote the spec**, self-reviewed, committed on a feature branch.
|
||||
11. Operator: "do [implementation] next session. finalize this one." →
|
||||
`wgl-session-finalize`.
|
||||
|
||||
## Cut state (end of session)
|
||||
|
||||
| Repo | Branch | Commit | State |
|
||||
|---|---|---|---|
|
||||
| human-experience-filter-art | `feature/simulator-alteration-preview` | `3ef21fb` | pushed to origin, **not merged** |
|
||||
|
||||
- `docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md` —
|
||||
new, committed.
|
||||
- `main` unchanged at `5290785`.
|
||||
- Working tree clean. Visual-companion server stopped; `.superpowers/` gitignored.
|
||||
- No tests run (no code changed).
|
||||
|
||||
## What lands on the operator's plate
|
||||
|
||||
- **Author the 24 variant videos** per base clip (the Left×Right grid), with HUD
|
||||
baked in. The simulator will consume them via the manifest; placeholder variants
|
||||
cover the gap until then.
|
||||
- **Deferred decisions** (also surfaced in chat):
|
||||
- *knob→strength calibration* — unresolved since session 0006; will be settled
|
||||
by eye in the simulator's calibration panel next session, then baked into
|
||||
`DEFAULT_CALIBRATION`.
|
||||
- *grade tints baked HUD* — accepted under "precision wins"; revisit only if it
|
||||
reads badly once real authored clips exist.
|
||||
|
||||
## Prompt the operator can paste into the next session
|
||||
|
||||
```
|
||||
/goal Implement the simulator alteration preview on branch feature/simulator-alteration-preview, per docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md — begin with the writing-plans skill, then build: parameterize Calibration + add VariantRef and slim RenderPlan in player/, add /api/alteration + /api/clips + simulator/clips.py variant manifest, rewrite the simulator UI as the player preview, retire the selection X-ray + /api/select, add the placeholder-variant generator, and update tests + USER_GUIDE + ROADMAP §3 + the parent design §4.3/§10 pointer.
|
||||
```
|
||||
|
||||
(Or resume with `/goal next` — the `Next /goal:` field is stored in memory
|
||||
`sub-project-3-player-progress.md`.)
|
||||
@@ -0,0 +1,102 @@
|
||||
# Session 0008.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-07T19-42 (PST)
|
||||
> End: 2026-06-07T22-37 (PST)
|
||||
> Type: spec
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
Session opened as `wgl-session-none` (research errand: "Let's find a great public
|
||||
domain nature video for this project"), then upgraded mid-session to a **spec**
|
||||
session once the work became tracked. Operator chose "Spec session, comprehensive":
|
||||
fold the session's POC findings into the design AND resolve the open cosmic-zoom
|
||||
structure fork.
|
||||
|
||||
## Plan
|
||||
|
||||
Develop a design revision to the machine-altered-perception design, grounded in a
|
||||
local POC run this session: (1) Right-axis alteration pipeline; (2) content
|
||||
structure (cosmic-zoom fork); (3) strictly-PD sourcing.
|
||||
|
||||
## Pre-state
|
||||
|
||||
- On `feature/simulator-alteration-preview` (session 0007's unmerged design), clean
|
||||
tree; `main` at `5290785`.
|
||||
- Catalog empty; `tools/ingest/internet_archive.py` is the only wired fetcher.
|
||||
- No ML stack, no ffmpeg in the venv (pure-stdlib project).
|
||||
|
||||
## Session arc (uncurated)
|
||||
|
||||
1. **PD nature-video research.** Surveyed pools and verified licenses against the
|
||||
actual metadata, not marketing. Key finding: **Pexels / Pixabay / Mitch Martinez
|
||||
"free 4K" are royalty-free but NOT public domain** (restrict redistribution,
|
||||
retain copyright) — exactly the trap the ingest tool's "no explicit license →
|
||||
assume PD, verify" flag exists for. Genuinely-PD: NASA/Hubble (cosmos), **NOAA
|
||||
Ocean Exploration** (deep sea, global), **NPS/USGS** (US land). Wikimedia nature
|
||||
4K timelapses are mostly CC-BY, not PD.
|
||||
|
||||
2. **Operator pivot → "cosmic zoom".** Operator proposed a Powers-of-Ten journey
|
||||
(space → continents → birds → ocean → abyss → microscopic → cosmos). Brainstormed
|
||||
the thesis tension (a scale-journey carries built-in awe vs. the "neutral base"
|
||||
thesis) and the cost (per-base pre-bake × altitudes). Surfaced that the *ends* of
|
||||
the zoom are strict-PD-rich; the terrestrial *middle* (birds, non-US land) is the
|
||||
CC-BY soft spot.
|
||||
|
||||
3. **"Can this run locally?" → POC.** Operator's machine: **Mac mini M4 Pro, 64 GB,
|
||||
16-core GPU.** Built a throwaway POC in `~/hef-poc/` (outside the repo).
|
||||
- *Wrong turns:* `brew install ffmpeg` and `pip install` were **denied by the
|
||||
harness** (it blocks package installs) — switched to venv-local `imageio-ffmpeg`
|
||||
run by the operator via `!`. zsh **doesn't word-split** unquoted `$EARGS` →
|
||||
mangled encoder flags; inlined them. sd-turbo img2img **crashed** ("reshape
|
||||
tensor of 0 elements") because `int(steps×strength)=int(2×0.45)=0` denoise steps
|
||||
→ guarded to ≥1.
|
||||
- *Results on an 8s/1080p clip:* deterministic **Dark/Light/Left** grades/overlay
|
||||
~2.4s each (~3× faster than realtime, runtime-capable). **Right** painterly
|
||||
restyle (SD img2img on MPS) ~3.4 min/clip and **flickers badly**.
|
||||
|
||||
4. **Flicker is disqualifying.** Operator: "crazy and disorienting … this project is
|
||||
meant to be peaceful." Prototyped two fixes: (a) a **deterministic** soft-dreamy
|
||||
filter (smartblur + RGB bloom — first attempt had a magenta cast from blending on
|
||||
YUV chroma planes; fixed by blending in RGB), ~5s, zero flicker by construction;
|
||||
(b) **optical-flow keyframe propagation** (EbSynth principle; genuine ebsynth is
|
||||
NVIDIA/Windows so implemented with OpenCV Farneback + the diffusers pipeline),
|
||||
~2.7 min/clip, **calm**. Operator chose **AI + flow**.
|
||||
|
||||
5. **Spec session (comprehensive).** Upgraded the session via
|
||||
`wgl-session-spec-init` → claimed 0008. Resolved the cosmic-zoom fork: operator
|
||||
chose a **small NEUTRAL "scales of nature" library** (not a single stitched film).
|
||||
Then operator added the **infinite-zoom ring** (AI zoom transitions between
|
||||
scales, micro→cosmos wraparound) and the **infinitely-turnable endless encoder**.
|
||||
Probed i18n feasibility → **§1.2**: Left HUD is a Pi-rendered **runtime overlay**
|
||||
(Pango/HarfBuzz + Noto, annotation-track + per-language string tables), NOT a baked
|
||||
per-language video.
|
||||
- Wrote `docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-design.md`,
|
||||
committed on `feature/scales-library-right-axis`.
|
||||
|
||||
## Cut state (what landed)
|
||||
|
||||
- **PR #6 merged to `main`** (merge `42a72fe`): the design revision doc.
|
||||
- No spec-RFC submission: this app keeps specs in-repo (`app.json` `contains:["specs"]`),
|
||||
so `submit-spec.sh` is N/A — surfaced, not dropped.
|
||||
- POC artifacts left in `~/hef-poc/` (throwaway, outside repo): `restyle.py`,
|
||||
`flow_restyle.py`, and the comparison clips/contact sheets.
|
||||
- Memory updated: `sub-project-3-player-progress.md` (+ `MEMORY.md` index).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **Left-HUD treatment conflict (low confidence — flag).** 0008 §1.2 specifies the
|
||||
Left HUD as a runtime Pango/HarfBuzz overlay (to keep i18n near-free, per parent
|
||||
§10). This **reverses session 0007's decision** to bake the HUD into the 5×5
|
||||
variant grid for authorial precision. I recommended the runtime-overlay path
|
||||
following the operator's i18n question without flagging the 0007 reversal at the
|
||||
time. `main` now carries the runtime-overlay position; the unmerged
|
||||
`feature/simulator-alteration-preview` carries baked-HUD. **Reconciliation deferred
|
||||
to next session** (pick one, or hybrid: authored positions + runtime-shaped text).
|
||||
|
||||
## Next /goal
|
||||
|
||||
```
|
||||
/goal Reconcile the 0007 simulator-alteration-preview design (feature/simulator-alteration-preview, UNMERGED) with the merged 0008 scales-library + flow-stabilized Right-axis design — resolve the Left-HUD conflict (baked-into-5×5-grid vs runtime Pango/HarfBuzz overlay) FIRST — then writing-plans → build the simulator-first slice: deterministic Dark/Light/Left + a flow-stabilized Right restyle over ONE neutral scale clip wired into the simulator. Defer Pi renderer, endless-encoder/serial, and AI zoom-transitions per simulator-first.
|
||||
```
|
||||
@@ -0,0 +1,130 @@
|
||||
# Session 0009.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-07T22-41 (PST)
|
||||
> End: 2026-06-07T23-03 (PST)
|
||||
> Type: coding
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
Opened with `/goal next` — resume the goal recorded at the 0008 finalize. The
|
||||
session gate resolved the stored `Next /goal:` from memory:
|
||||
|
||||
> Reconcile the 0007 simulator-alteration-preview design (UNMERGED) with the
|
||||
> merged 0008 scales-library + flow-stabilized Right-axis design — resolve the
|
||||
> Left-HUD conflict (baked-into-5×5-grid vs runtime Pango/HarfBuzz overlay) FIRST
|
||||
> — then writing-plans → build the simulator-first slice: deterministic
|
||||
> Dark/Light/Left + a flow-stabilized Right restyle over ONE neutral scale clip
|
||||
> wired into the simulator. Defer Pi renderer, endless-encoder/serial, AI
|
||||
> zoom-transitions.
|
||||
|
||||
## Plan
|
||||
|
||||
Brainstorm the reconciliation (resolve Left-HUD first) → write a unified design
|
||||
doc → writing-plans → build & merge the simulator-first alteration slice.
|
||||
|
||||
## Pre-state
|
||||
|
||||
- On `main` at `c99a669`, clean tree; ran from the **main clone** (not a nested
|
||||
worktree — avoiding the session-resolver ambiguity gotcha flagged in 0008).
|
||||
- Two unmerged design threads: session-0007's
|
||||
`feature/simulator-alteration-preview` (baked-HUD 5×5 grid, design doc only, no
|
||||
code) vs. session-0008's merged scales-library/right-axis design (runtime-overlay
|
||||
HUD). The Left-HUD conflict between them was the thing to resolve.
|
||||
- Slice-1 engine (`player/`) on `main`: `RenderPlan {grade, overlay, restyle}` with
|
||||
a continuous `Restyle.blend` and a runtime `AnalyticalOverlay`. Simulator still on
|
||||
the old *selection* surface (`/api/select`, X-ray, `fixtures.py`).
|
||||
- 0008 POC artifacts present in `~/hef-poc/out/` (outside repo): real `neutral.mp4`
|
||||
base + `right_flow.mp4` flow-stabilized restyle.
|
||||
|
||||
## Session arc (uncurated)
|
||||
|
||||
1. **Claimed 0009** (coding) from the main clone; resolver worked cleanly. Read
|
||||
both specs + memory in full.
|
||||
|
||||
2. **Surfaced the conflict precisely — it was 2-D, not 1-D.** Beyond the Left-HUD
|
||||
rendering question, the two designs disagreed on the *shape of the pre-baked set*:
|
||||
0007 made BOTH Left and Right a 5×5 grid of 24 authored clips; 0008 made only
|
||||
Right pre-baked and Left a runtime overlay. Key realization: **0008 §1.2 already
|
||||
IS the hybrid the operator floated** ("authored box positions + runtime-shaped
|
||||
text") — authored annotation track for layout, runtime shaping for cheap i18n.
|
||||
|
||||
3. **Brainstorming → one operator question.** Asked the single load-bearing fork
|
||||
(Left HUD: runtime-overlay+authored-track vs fully-baked vs defer). Operator chose
|
||||
**runtime overlay + authored track** (the 0008 / hybrid position). Everything else
|
||||
followed from approved material, so I proceeded autonomously.
|
||||
|
||||
4. **Found the engine was already most of the way there.** The merged slice-1 engine
|
||||
already keeps a runtime `AnalyticalOverlay` — only 0007's *unmerged doc* proposed
|
||||
removing it. So the reconciliation was surgical: continuous `Restyle.blend` →
|
||||
discrete `Restyle.variant`; add `AnalyticalOverlay.level`; add a frozen
|
||||
`Calibration`. Wrote the reconciled design doc (brought the 0007 doc onto the
|
||||
branch so its superseded position is preserved + links resolve), committed.
|
||||
|
||||
5. **writing-plans → a full TDD plan** (`docs/superpowers/plans/...`), then executed
|
||||
it task-by-task with executing-plans:
|
||||
- **Task 1–2 (engine):** `Calibration`+`DEFAULT_CALIBRATION` (behavior-preserving),
|
||||
`Restyle.variant`, `AnalyticalOverlay.level`, `render_plan_to_dict`. `state.py`
|
||||
needed no logic change (compares whole `Restyle`). Tests rewritten/green.
|
||||
- **Task 3 (clips):** `simulator/clips.py` manifest model; retired `fixtures.py` +
|
||||
`test_fixtures.py`.
|
||||
- **Task 4 (API):** `/api/alteration` + `/api/clips`; removed `/api/select` +
|
||||
`/api/catalog/meta`. *Wrong turn:* asserted retired POST returns 404, but the
|
||||
static catch-all yields **405** for an unrouted POST — relaxed the assertion to
|
||||
"404 or 405 = gone."
|
||||
- **Task 5 (media):** `sample_media/manifest.json` + `setup_sample_media.py` that
|
||||
copies the real POC `neutral.mp4`→base and `right_flow.mp4`→Right-strength-4 and
|
||||
ffmpeg-blends placeholder strengths 1–3. mp4s gitignored. Ran it — 5 files
|
||||
produced.
|
||||
- **Task 6 (UI):** rewrote `simulator/static/` as the alteration preview (live
|
||||
grade via CSS filters, Right-variant `<video>` crossfade, live SVG Left overlay
|
||||
from the annotation track + string table, calibration panel, RenderPlan readout).
|
||||
- **Task 7 (docs):** parent design §4.3/§10 pointers; ROADMAP slice 2 done +
|
||||
deferred slices; USER_GUIDE simulator section rewritten.
|
||||
- *Environment:* `python` not on PATH; used `.venv/bin/python` throughout.
|
||||
|
||||
6. **Verified end-to-end.** `pytest -q` → **192 passed, 2 skipped**. Booted the sim
|
||||
on a scratch port: `/api/clips` returns the manifest (variant 0→base), the engine
|
||||
plan is correct by eye (left=3→level 3/intensity 0.75, right=4→variant 4,
|
||||
dark=4→tone −1.0), and the **real** `right4.mp4` + base served as `video/mp4`.
|
||||
|
||||
7. **Shipped.** Pushed the branch, created **Gitea PR #7** via the keychain-token API
|
||||
helper, **merged** it (autonomous posture), synced `main`, re-verified tests green
|
||||
on the merged result, deleted my merged feature branch (local + remote).
|
||||
|
||||
## Cut state (what landed)
|
||||
|
||||
- **PR #7 merged to `main`** (merge `554eb50`): the reconciliation design + the built
|
||||
simulator-first alteration slice (engine + simulator + media tooling + docs). 9
|
||||
commits.
|
||||
- `main` green: 192 passed / 2 skipped. Sim verified booting + serving real media.
|
||||
- Memory updated: `sub-project-3-player-progress.md` (+ `MEMORY.md` index) — conflict
|
||||
resolved, slice 2 shipped, new Next /goal.
|
||||
- POC artifacts untouched in `~/hef-poc/` (still the source for `setup_sample_media.py`).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
- **Brought the 0007 design doc into `main` + intended to delete the superseded
|
||||
branch.** Decided autonomously to carry `2026-06-06-simulator-alteration-preview-design.md`
|
||||
forward (so the reconciliation's links resolve and the superseded position is
|
||||
preserved) and to delete the now-stale `feature/simulator-alteration-preview`.
|
||||
**The branch deletion (`git branch -D` / `push --delete`) was permission-denied by
|
||||
the harness** — left in place. Safe to delete later (its content is in `main`); or
|
||||
the operator may want to keep it. Flagging rather than forcing.
|
||||
- **Merged under autonomous posture without a separate `/code-review`.** TDD + the
|
||||
written plan were the quality gates; no independent review pass was run before
|
||||
merge. Low risk for a slice this size, but noting it.
|
||||
- **Used the POC's Yosemite clip as the sim's sample base.** It's a "forest"-scale
|
||||
neutral clip, good enough to tune the look; framed in the manifest/USER_GUIDE as
|
||||
**look-tuning only, not shipped content**. Strict-PD scale-library sourcing stays a
|
||||
later slice — unaffected.
|
||||
- **`Calibration` defaults are still behavior-preserving, not operator-tuned.** The
|
||||
knob→strength calibration (open since session 0006) is now tunable by eye in the
|
||||
sim but **not yet locked** — that's the next goal.
|
||||
|
||||
## Next /goal
|
||||
|
||||
```
|
||||
/goal Tune the alteration look by eye in the simulator (python simulator/setup_sample_media.py then make sim-local) and LOCK the knob→strength calibration into DEFAULT_CALIBRATION in player/alteration.py (+ a unit test) — settling the open session-0006 calibration decision. While there, judge whether more neutral "scales of nature" base clips + a real multi-strength flow-stabilized Right re-bake are worth doing next vs. moving to scale-ring navigation (endless encoder + AI zoom transitions). Keep deferring Pi renderer + serial/firmware. Read sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md (§8) first.
|
||||
```
|
||||
@@ -0,0 +1,23 @@
|
||||
# Session 0010.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-07T23-09 (PST)
|
||||
> Type: coding
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0010 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0010.0-TRANSCRIPT-2026-06-07T23-09--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
Tune the alteration look by eye in the simulator (python simulator/setup_sample_media.py then make sim-local) and LOCK the knob→strength calibration into DEFAULT_CALIBRATION in player/alteration.py (+ a unit test) — settling the open session-0006 calibration decision. While there, judge whether more neutral "scales of nature" base clips + a real multi-strength flow-stabilized Right re-bake are worth doing next vs. moving to scale-ring navigation (endless encoder + AI zoom transitions). Keep deferring Pi renderer + serial/firmware. Read sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md (§8) first.
|
||||
|
||||
```
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
@@ -1,5 +1,32 @@
|
||||
{
|
||||
"0001": {
|
||||
"title": ""
|
||||
},
|
||||
"0002": {
|
||||
"title": ""
|
||||
},
|
||||
"0003": {
|
||||
"title": ""
|
||||
},
|
||||
"0004": {
|
||||
"title": ""
|
||||
},
|
||||
"0005": {
|
||||
"title": ""
|
||||
},
|
||||
"0006": {
|
||||
"title": ""
|
||||
},
|
||||
"0007": {
|
||||
"title": ""
|
||||
},
|
||||
"0008": {
|
||||
"title": ""
|
||||
},
|
||||
"0009": {
|
||||
"title": ""
|
||||
},
|
||||
"0010": {
|
||||
"title": ""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
FROM python:3.11-slim
|
||||
WORKDIR /app
|
||||
COPY pyproject.toml ./
|
||||
COPY hef ./hef
|
||||
COPY tools ./tools
|
||||
COPY simulator ./simulator
|
||||
COPY catalog ./catalog
|
||||
RUN pip install --no-cache-dir -e ".[sim]"
|
||||
EXPOSE 8000
|
||||
CMD ["uvicorn", "simulator.app:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
@@ -0,0 +1 @@
|
||||
"""Web-based curator's X-ray simulator for the experience filter."""
|
||||
@@ -0,0 +1,99 @@
|
||||
"""FastAPI service: controls -> the real alteration engine -> a RenderPlan.
|
||||
|
||||
The simulator's alteration surface (reconciled slice). It calls the canonical
|
||||
player.alteration.plan_alteration; the browser only renders. The selection-era
|
||||
endpoints (/api/select, /api/catalog/meta) and the X-ray are retired.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import (
|
||||
DEFAULT_CALIBRATION,
|
||||
Calibration,
|
||||
plan_alteration,
|
||||
render_plan_to_dict,
|
||||
)
|
||||
from player.content import resolve_content
|
||||
from player.controls import CONTENT_POSITIONS
|
||||
from simulator.clips import load_manifest
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "static"
|
||||
MEDIA_DIR = Path(__file__).parent / "sample_media"
|
||||
DEFAULT_MANIFEST = MEDIA_DIR / "manifest.json"
|
||||
|
||||
|
||||
class ControlsModel(BaseModel):
|
||||
content: str
|
||||
left: int = Field(ge=0, le=4)
|
||||
right: int = Field(ge=0, le=4)
|
||||
dark: int = Field(ge=0, le=4)
|
||||
light: int = Field(ge=0, le=4)
|
||||
volume: int = Field(ge=0, le=4)
|
||||
brightness: int = Field(ge=0, le=4)
|
||||
|
||||
|
||||
class CalibrationModel(BaseModel):
|
||||
mood_gain: float = 1.0
|
||||
overlay_gain: float = 1.0
|
||||
right_variant_map: list[int] = [0, 1, 2, 3, 4]
|
||||
|
||||
|
||||
class AlterationRequest(BaseModel):
|
||||
controls: ControlsModel
|
||||
calibration: Optional[CalibrationModel] = None
|
||||
|
||||
|
||||
def _load_clips(manifest_path: Optional[Path]):
|
||||
path = Path(manifest_path) if manifest_path else DEFAULT_MANIFEST
|
||||
if path.exists():
|
||||
return load_manifest(path)
|
||||
return []
|
||||
|
||||
|
||||
def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
|
||||
app = FastAPI(title="HEF Alteration Simulator")
|
||||
app.state.clips = _load_clips(manifest_path)
|
||||
|
||||
@app.post("/api/alteration")
|
||||
def api_alteration(req: AlterationRequest):
|
||||
c = req.controls
|
||||
if c.content not in CONTENT_POSITIONS:
|
||||
raise HTTPException(status_code=422, detail=f"invalid content {c.content!r}")
|
||||
coord = Coordinate(c.left, c.right, c.dark, c.light)
|
||||
cal = (
|
||||
Calibration(
|
||||
mood_gain=req.calibration.mood_gain,
|
||||
overlay_gain=req.calibration.overlay_gain,
|
||||
right_variant_map=tuple(req.calibration.right_variant_map),
|
||||
)
|
||||
if req.calibration
|
||||
else DEFAULT_CALIBRATION
|
||||
)
|
||||
plan = plan_alteration(coord, cal)
|
||||
content = resolve_content(c.content)
|
||||
return {
|
||||
"plan": render_plan_to_dict(plan),
|
||||
"content": {"audio_source": content.audio_source, "video": content.video},
|
||||
}
|
||||
|
||||
@app.get("/api/clips")
|
||||
def api_clips():
|
||||
return {"clips": [c.to_dict() for c in app.state.clips]}
|
||||
|
||||
if MEDIA_DIR.exists():
|
||||
app.mount("/media", StaticFiles(directory=MEDIA_DIR), name="media")
|
||||
if STATIC_DIR.exists():
|
||||
app.mount("/", StaticFiles(directory=STATIC_DIR, html=True), name="static")
|
||||
|
||||
return app
|
||||
|
||||
|
||||
app = create_app()
|
||||
@@ -0,0 +1,68 @@
|
||||
"""The base-clip + variant + annotation manifest the simulator renders.
|
||||
|
||||
Replaces simulator/fixtures.py (the selection-era synthetic catalog). Each base
|
||||
clip carries: the raw base file, a map of pre-baked Right-strength variant files
|
||||
(strength 0 is always the raw base), an authored Left annotation track (box +
|
||||
label key + the minimum Left level at which it appears), and per-language string
|
||||
tables. See the reconciled-simulator-alteration-slice design §3.2.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Clip:
|
||||
id: str
|
||||
title: str
|
||||
base_file: str
|
||||
license: str
|
||||
source: str
|
||||
right_variants: dict # {"1": {"file": ...}, "4": {...}} (no "0")
|
||||
annotations: list # [{"key", "box":[x,y,w,h], "min_level"}, ...]
|
||||
strings: dict # {"en": {key: text}}
|
||||
|
||||
def variant_file(self, strength: int) -> str:
|
||||
"""The video file for a Right strength; 0 and any unauthored strength
|
||||
fall back to the raw base file."""
|
||||
entry = self.right_variants.get(str(strength))
|
||||
return entry["file"] if entry else self.base_file
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
variants = {"0": {"file": self.base_file, "raw": True}}
|
||||
for k, v in self.right_variants.items():
|
||||
variants[k] = v
|
||||
return {
|
||||
"id": self.id,
|
||||
"title": self.title,
|
||||
"base_file": self.base_file,
|
||||
"license": self.license,
|
||||
"source": self.source,
|
||||
"right_variants": variants,
|
||||
"annotations": self.annotations,
|
||||
"strings": self.strings,
|
||||
}
|
||||
|
||||
|
||||
def _clip_from_dict(d: dict[str, Any]) -> Clip:
|
||||
return Clip(
|
||||
id=d["id"],
|
||||
title=d["title"],
|
||||
base_file=d["base_file"],
|
||||
license=d.get("license", ""),
|
||||
source=d.get("source", ""),
|
||||
right_variants=d.get("right_variants", {}),
|
||||
annotations=d.get("annotations", []),
|
||||
strings=d.get("strings", {}),
|
||||
)
|
||||
|
||||
|
||||
def load_manifest(path: str | Path) -> list[Clip]:
|
||||
"""Load the base-clip manifest. Raises FileNotFoundError if missing."""
|
||||
path = Path(path)
|
||||
data = json.loads(path.read_text())
|
||||
return [_clip_from_dict(c) for c in data["clips"]]
|
||||
@@ -0,0 +1,7 @@
|
||||
services:
|
||||
simulator:
|
||||
build:
|
||||
context: ..
|
||||
dockerfile: simulator/Dockerfile
|
||||
ports:
|
||||
- "8000:8000"
|
||||
@@ -0,0 +1,12 @@
|
||||
# Simulator sample media
|
||||
|
||||
`manifest.json` is committed; the `.mp4` binaries are **not** (gitignored). They
|
||||
are look-tuning samples, not shipped installation content.
|
||||
|
||||
Populate them from the session-0008 POC artifacts:
|
||||
|
||||
python simulator/setup_sample_media.py
|
||||
|
||||
This copies `~/hef-poc/out/neutral.mp4` → `forest/base.mp4` and
|
||||
`~/hef-poc/out/right_flow.mp4` → `forest/right4.mp4` (the real flow-stabilized
|
||||
restyle), and generates placeholder strengths `forest/right1..3.mp4`.
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"clips": [
|
||||
{
|
||||
"id": "forest",
|
||||
"title": "Yosemite Falls (neutral base, POC sample)",
|
||||
"base_file": "forest/base.mp4",
|
||||
"license": "poc-sample (look-tuning only; not shipped content)",
|
||||
"source": "hef-poc/out/neutral.mp4",
|
||||
"right_variants": {
|
||||
"1": {"file": "forest/right1.mp4", "model": "placeholder"},
|
||||
"2": {"file": "forest/right2.mp4", "model": "placeholder"},
|
||||
"3": {"file": "forest/right3.mp4", "model": "placeholder"},
|
||||
"4": {"file": "forest/right4.mp4", "model": "sd-turbo+farneback-flow"}
|
||||
},
|
||||
"annotations": [
|
||||
{"key": "detected.water", "box": [0.30, 0.10, 0.18, 0.70], "min_level": 1},
|
||||
{"key": "detected.rock_face", "box": [0.05, 0.30, 0.20, 0.55], "min_level": 2},
|
||||
{"key": "detected.conifer", "box": [0.70, 0.20, 0.22, 0.45], "min_level": 3},
|
||||
{"key": "measure.flow_rate", "box": [0.34, 0.55, 0.14, 0.08], "min_level": 4}
|
||||
],
|
||||
"strings": {
|
||||
"en": {
|
||||
"detected.water": "flowing water",
|
||||
"detected.rock_face": "granite face",
|
||||
"detected.conifer": "conifer stand",
|
||||
"measure.flow_rate": "~2.1 m³/s"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Populate simulator/sample_media/forest/ from the session-0008 POC artifacts.
|
||||
|
||||
Copies the real neutral base + the real flow-stabilized Right restyle out of
|
||||
~/hef-poc/out/, and generates placeholder intermediate Right strengths (1..3) by
|
||||
blending the base toward the real restyle with ffmpeg. The media binaries are
|
||||
gitignored; only the manifest is committed. Sample footage is for look-tuning
|
||||
only, not shipped content.
|
||||
|
||||
Usage: python simulator/setup_sample_media.py
|
||||
Requires: ffmpeg on PATH (or `pip install imageio-ffmpeg`), and ~/hef-poc/out/.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
POC = Path.home() / "hef-poc" / "out"
|
||||
DEST = Path(__file__).parent / "sample_media" / "forest"
|
||||
|
||||
|
||||
def _ffmpeg() -> str:
|
||||
if shutil.which("ffmpeg"):
|
||||
return "ffmpeg"
|
||||
import imageio_ffmpeg
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
DEST.mkdir(parents=True, exist_ok=True)
|
||||
base = DEST / "base.mp4"
|
||||
right4 = DEST / "right4.mp4"
|
||||
shutil.copyfile(POC / "neutral.mp4", base)
|
||||
shutil.copyfile(POC / "right_flow.mp4", right4)
|
||||
ff = _ffmpeg()
|
||||
# Placeholder strengths 1..3: opacity-blend base toward the real restyle.
|
||||
for strength, alpha in ((1, 0.25), (2, 0.5), (3, 0.75)):
|
||||
out = DEST / f"right{strength}.mp4"
|
||||
subprocess.run(
|
||||
[ff, "-y", "-i", str(base), "-i", str(right4),
|
||||
"-filter_complex",
|
||||
f"[1:v]format=yuva444p,colorchannelmixer=aa={alpha}[top];"
|
||||
f"[0:v][top]overlay=shortest=1[v]",
|
||||
"-map", "[v]", "-an", str(out)],
|
||||
check=True,
|
||||
)
|
||||
print(f"generated {out.name} (alpha {alpha})")
|
||||
print(f"sample media ready in {DEST}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,105 @@
|
||||
// Thin renderer: post controls+calibration -> RenderPlan; render grade, Right
|
||||
// variant crossfade, and the live Left overlay. All math stays in Python.
|
||||
const $ = (id) => document.getElementById(id);
|
||||
const vid = $("vid"), tint = $("tint"), overlay = $("overlay"), black = $("black"), readout = $("readout");
|
||||
|
||||
let clip = null; // active clip manifest entry
|
||||
let currentVariant = -1; // last loaded Right strength
|
||||
|
||||
async function loadClips() {
|
||||
const data = await (await fetch("/api/clips")).json();
|
||||
clip = data.clips[0] || null;
|
||||
}
|
||||
|
||||
function mediaUrl(file) { return "/media/" + file; }
|
||||
|
||||
function variantFile(strength) {
|
||||
const v = clip.right_variants[String(strength)];
|
||||
return v ? v.file : clip.base_file;
|
||||
}
|
||||
|
||||
function applyGrade(tone) {
|
||||
// Light: warm + brighten (sepia). Dark: cool + darken via a multiply-blended
|
||||
// blue wash (#tint) that lifts shadows toward blue while keeping natural
|
||||
// greens — the peaceful POC dark look, NOT a full-frame hue spin.
|
||||
const warm = tone > 0 ? tone : 0, cool = tone < 0 ? -tone : 0;
|
||||
const bright = 1 + 0.25 * warm - 0.35 * cool;
|
||||
const sat = 1 + 0.15 * warm - 0.30 * cool;
|
||||
vid.style.filter =
|
||||
`brightness(${bright.toFixed(3)}) saturate(${sat.toFixed(3)}) ` +
|
||||
`sepia(${(warm * 0.5).toFixed(3)})`;
|
||||
tint.style.opacity = (cool * 0.6).toFixed(3);
|
||||
}
|
||||
|
||||
function loadVariant(strength) {
|
||||
if (strength === currentVariant) return;
|
||||
currentVariant = strength;
|
||||
vid.style.opacity = "0";
|
||||
setTimeout(() => {
|
||||
vid.src = mediaUrl(variantFile(strength));
|
||||
vid.play().catch(() => {});
|
||||
vid.style.opacity = "1";
|
||||
}, 150);
|
||||
}
|
||||
|
||||
function renderOverlay(level, intensity) {
|
||||
overlay.innerHTML = "";
|
||||
if (!clip || level <= 0) { overlay.style.opacity = "0"; return; }
|
||||
overlay.style.opacity = String(intensity);
|
||||
const strings = (clip.strings && clip.strings.en) || {};
|
||||
for (const a of clip.annotations) {
|
||||
if (a.min_level > level) continue;
|
||||
const [x, y, w, h] = a.box.map((n) => n * 100);
|
||||
const rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
|
||||
rect.setAttribute("x", x); rect.setAttribute("y", y);
|
||||
rect.setAttribute("width", w); rect.setAttribute("height", h);
|
||||
rect.setAttribute("class", "anno-box");
|
||||
overlay.appendChild(rect);
|
||||
const text = document.createElementNS("http://www.w3.org/2000/svg", "text");
|
||||
text.setAttribute("x", x + 0.5); text.setAttribute("y", Math.max(y - 0.5, 2));
|
||||
text.setAttribute("class", "anno-label");
|
||||
text.textContent = strings[a.key] || a.key;
|
||||
overlay.appendChild(text);
|
||||
}
|
||||
}
|
||||
|
||||
function controls() {
|
||||
return {
|
||||
content: $("content").value,
|
||||
left: +$("left").value, right: +$("right").value,
|
||||
dark: +$("dark").value, light: +$("light").value,
|
||||
volume: 2, brightness: 2,
|
||||
};
|
||||
}
|
||||
|
||||
function calibration() {
|
||||
return { mood_gain: +$("mood_gain").value, overlay_gain: +$("overlay_gain").value,
|
||||
right_variant_map: [0, 1, 2, 3, 4] };
|
||||
}
|
||||
|
||||
let timer = null;
|
||||
async function update() {
|
||||
const resp = await fetch("/api/alteration", {
|
||||
method: "POST", headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ controls: controls(), calibration: calibration() }),
|
||||
});
|
||||
if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; }
|
||||
const data = await resp.json();
|
||||
readout.textContent = JSON.stringify(data, null, 2);
|
||||
if (!data.content.video) { black.classList.remove("hidden"); return; }
|
||||
black.classList.add("hidden");
|
||||
applyGrade(data.plan.grade.tone);
|
||||
loadVariant(data.plan.restyle.variant);
|
||||
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
|
||||
}
|
||||
|
||||
function debounced() { clearTimeout(timer); timer = setTimeout(update, 80); }
|
||||
|
||||
async function main() {
|
||||
await loadClips();
|
||||
for (const id of ["content", "left", "right", "dark", "light", "mood_gain", "overlay_gain"]) {
|
||||
$(id).addEventListener("input", debounced);
|
||||
}
|
||||
update();
|
||||
}
|
||||
main();
|
||||
@@ -0,0 +1,57 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>HEF — Alteration Simulator</title>
|
||||
<link rel="stylesheet" href="/style.css" />
|
||||
</head>
|
||||
<body>
|
||||
<header><h1>Human Experience Filter — Alteration Preview</h1></header>
|
||||
<main>
|
||||
<section class="stage">
|
||||
<div class="screen">
|
||||
<video id="vid" loop muted playsinline></video>
|
||||
<div id="tint"></div>
|
||||
<svg id="overlay" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
|
||||
<div id="black" class="black hidden"></div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<fieldset>
|
||||
<legend>Content dial</legend>
|
||||
<select id="content">
|
||||
<option value="video">video</option>
|
||||
<option value="audio_video">audio + video</option>
|
||||
<option value="music_video">music + video</option>
|
||||
<option value="off">off (black)</option>
|
||||
<option value="white_noise">white noise (no video)</option>
|
||||
<option value="music">music (no video)</option>
|
||||
<option value="audio_track">audio track (no video)</option>
|
||||
</select>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>Experience knobs (0–4)</legend>
|
||||
<label>Left (analytical) <input type="range" id="left" min="0" max="4" value="0" /></label>
|
||||
<label>Right (dreamlike) <input type="range" id="right" min="0" max="4" value="0" /></label>
|
||||
<label>Dark <input type="range" id="dark" min="0" max="4" value="0" /></label>
|
||||
<label>Light <input type="range" id="light" min="0" max="4" value="0" /></label>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>Calibration</legend>
|
||||
<label>mood gain <input type="range" id="mood_gain" min="0" max="2" step="0.05" value="1" /></label>
|
||||
<label>overlay gain <input type="range" id="overlay_gain" min="0" max="2" step="0.05" value="1" /></label>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>RenderPlan readout</legend>
|
||||
<pre id="readout">—</pre>
|
||||
</fieldset>
|
||||
</section>
|
||||
</main>
|
||||
<script src="/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,25 @@
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; font: 14px/1.4 system-ui, sans-serif; background: #111; color: #eee; }
|
||||
header { padding: 0.6rem 1rem; background: #000; }
|
||||
h1 { font-size: 1rem; margin: 0; font-weight: 600; }
|
||||
main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; }
|
||||
.stage { flex: 1 1 640px; }
|
||||
.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000;
|
||||
border-radius: 6px; overflow: hidden; }
|
||||
#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; }
|
||||
#tint { position: absolute; inset: 0; pointer-events: none; opacity: 0;
|
||||
background: #28425f; mix-blend-mode: multiply;
|
||||
transition: opacity 0.2s ease; }
|
||||
#overlay { position: absolute; inset: 0; width: 100%; height: 100%;
|
||||
pointer-events: none; transition: opacity 0.2s ease; }
|
||||
.anno-box { fill: none; stroke: #6cf; stroke-width: 0.4; vector-effect: non-scaling-stroke; }
|
||||
.anno-label { fill: #6cf; font-size: 3px; font-family: monospace; }
|
||||
.black { position: absolute; inset: 0; background: #000; }
|
||||
.hidden { display: none; }
|
||||
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem; }
|
||||
fieldset { border: 1px solid #333; border-radius: 6px; }
|
||||
legend { color: #9af; padding: 0 0.4rem; }
|
||||
label { display: block; margin: 0.4rem 0; }
|
||||
input[type=range], select { width: 100%; }
|
||||
#readout { background: #000; padding: 0.5rem; border-radius: 4px; font-size: 12px;
|
||||
white-space: pre-wrap; max-height: 240px; overflow: auto; }
|
||||
@@ -0,0 +1,48 @@
|
||||
import pytest
|
||||
|
||||
from hef.catalog import Record, validate_catalog, index_by_id, CatalogError
|
||||
|
||||
|
||||
def make_record(**o):
|
||||
base = dict(
|
||||
id="a",
|
||||
title="t",
|
||||
source_url="u",
|
||||
source_archive="nasa",
|
||||
license="public_domain",
|
||||
mode="video",
|
||||
left=0,
|
||||
right=0,
|
||||
dark=0,
|
||||
light=0,
|
||||
duration_s=1,
|
||||
file_path="p",
|
||||
)
|
||||
base.update(o)
|
||||
return Record(**base)
|
||||
|
||||
|
||||
def test_validate_catalog_accepts_unique_ids():
|
||||
validate_catalog([make_record(id="a"), make_record(id="b")]) # no raise
|
||||
|
||||
|
||||
def test_validate_catalog_rejects_duplicate_id():
|
||||
with pytest.raises(CatalogError) as e:
|
||||
validate_catalog([make_record(id="dup"), make_record(id="dup")])
|
||||
assert "dup" in str(e.value)
|
||||
|
||||
|
||||
def test_validate_catalog_validates_each_record():
|
||||
with pytest.raises(CatalogError):
|
||||
validate_catalog([make_record(left=9)])
|
||||
|
||||
|
||||
def test_index_by_id_round_trips():
|
||||
recs = [make_record(id="a"), make_record(id="b")]
|
||||
idx = index_by_id(recs)
|
||||
assert idx["a"].id == "a" and idx["b"].id == "b"
|
||||
|
||||
|
||||
def test_index_by_id_rejects_duplicates():
|
||||
with pytest.raises(CatalogError):
|
||||
index_by_id([make_record(id="x"), make_record(id="x")])
|
||||
@@ -0,0 +1,69 @@
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from simulator.clips import Clip, load_manifest
|
||||
|
||||
|
||||
def _manifest_dict():
|
||||
return {
|
||||
"clips": [
|
||||
{
|
||||
"id": "forest",
|
||||
"title": "Yosemite Falls (neutral)",
|
||||
"base_file": "forest/base.mp4",
|
||||
"license": "poc-sample",
|
||||
"source": "hef-poc",
|
||||
"right_variants": {
|
||||
"4": {"file": "forest/right4.mp4", "model": "sd-turbo+flow"},
|
||||
"1": {"file": "forest/right1.mp4"},
|
||||
},
|
||||
"annotations": [
|
||||
{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1},
|
||||
{"key": "detected.conifer", "box": [0.6, 0.1, 0.2, 0.2], "min_level": 3},
|
||||
],
|
||||
"strings": {"en": {"detected.water": "flowing water", "detected.conifer": "conifer"}},
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
def test_load_manifest_parses_clips(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_manifest_dict()))
|
||||
clips = load_manifest(p)
|
||||
assert len(clips) == 1
|
||||
c = clips[0]
|
||||
assert isinstance(c, Clip)
|
||||
assert c.id == "forest"
|
||||
assert c.base_file == "forest/base.mp4"
|
||||
|
||||
|
||||
def test_clip_lists_variant_files_by_strength(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_manifest_dict()))
|
||||
c = load_manifest(p)[0]
|
||||
# variant 0 is always the raw base; authored strengths come from the manifest
|
||||
assert c.variant_file(0) == "forest/base.mp4"
|
||||
assert c.variant_file(4) == "forest/right4.mp4"
|
||||
assert c.variant_file(1) == "forest/right1.mp4"
|
||||
# an unauthored strength falls back to the raw base
|
||||
assert c.variant_file(2) == "forest/base.mp4"
|
||||
|
||||
|
||||
def test_clip_serializes_to_dict_for_the_api(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps(_manifest_dict()))
|
||||
d = load_manifest(p)[0].to_dict()
|
||||
assert d["id"] == "forest"
|
||||
assert d["base_file"] == "forest/base.mp4"
|
||||
assert d["annotations"][0]["key"] == "detected.water"
|
||||
assert d["strings"]["en"]["detected.water"] == "flowing water"
|
||||
# variant map is exposed keyed by strength string, including 0 -> base
|
||||
assert d["right_variants"]["0"]["file"] == "forest/base.mp4"
|
||||
assert d["right_variants"]["4"]["file"] == "forest/right4.mp4"
|
||||
|
||||
|
||||
def test_missing_manifest_raises(tmp_path):
|
||||
with pytest.raises(FileNotFoundError):
|
||||
load_manifest(tmp_path / "nope.json")
|
||||
@@ -0,0 +1,79 @@
|
||||
from hef.selection import Coordinate
|
||||
from tools.drafting import Draft, HeuristicProposer, Signals
|
||||
|
||||
|
||||
def _sig(**o):
|
||||
base = dict(
|
||||
title="",
|
||||
description="",
|
||||
source_archive="nasa",
|
||||
mode="video",
|
||||
duration_s=600,
|
||||
)
|
||||
base.update(o)
|
||||
return Signals(**base)
|
||||
|
||||
|
||||
def test_returns_draft_with_coordinate_and_rationale():
|
||||
d = HeuristicProposer().propose(_sig())
|
||||
assert isinstance(d, Draft)
|
||||
assert isinstance(d.coordinate, Coordinate)
|
||||
assert d.rationale and "\n" not in d.rationale
|
||||
|
||||
|
||||
def test_librivox_seeds_left():
|
||||
d = HeuristicProposer().propose(
|
||||
_sig(title="Meditations", source_archive="librivox", mode="audio")
|
||||
)
|
||||
assert d.coordinate.left >= 3 and d.rationale
|
||||
|
||||
|
||||
def test_music_archives_seed_right():
|
||||
for arch in ("musopen", "fma", "nasa", "freesound"):
|
||||
d = HeuristicProposer().propose(_sig(source_archive=arch, mode="audio"))
|
||||
assert d.coordinate.right >= 3, arch
|
||||
|
||||
|
||||
def test_internet_archive_seeds_left():
|
||||
d = HeuristicProposer().propose(_sig(source_archive="internet_archive"))
|
||||
assert d.coordinate.left >= 3
|
||||
|
||||
|
||||
def test_storm_seeds_dark():
|
||||
d = HeuristicProposer().propose(
|
||||
_sig(title="Thunderstorm at Night", source_archive="nasa")
|
||||
)
|
||||
assert d.coordinate.dark >= 2
|
||||
|
||||
|
||||
def test_sunrise_seeds_light():
|
||||
d = HeuristicProposer().propose(
|
||||
_sig(title="Sunrise over the Garden", source_archive="nasa")
|
||||
)
|
||||
assert d.coordinate.light >= 2
|
||||
|
||||
|
||||
def test_coordinates_clamped_to_0_4():
|
||||
d = HeuristicProposer().propose(
|
||||
_sig(
|
||||
title="storm night war funeral decay requiem minor",
|
||||
description="noir death grief",
|
||||
source_archive="librivox",
|
||||
)
|
||||
)
|
||||
for v in (d.coordinate.left, d.coordinate.right, d.coordinate.dark, d.coordinate.light):
|
||||
assert 0 <= v <= 4
|
||||
|
||||
|
||||
def test_rationale_cites_a_signal():
|
||||
d = HeuristicProposer().propose(
|
||||
_sig(title="Storm", source_archive="librivox", mode="audio")
|
||||
)
|
||||
assert "librivox" in d.rationale.lower()
|
||||
|
||||
|
||||
def test_deterministic():
|
||||
s = _sig(title="Storm at Dawn", source_archive="nasa")
|
||||
a = HeuristicProposer().propose(s)
|
||||
b = HeuristicProposer().propose(s)
|
||||
assert a == b
|
||||
@@ -0,0 +1,181 @@
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from tools.http import HttpClient
|
||||
from tools.ingest.internet_archive import InternetArchiveFetcher
|
||||
from tools.ingest.librivox import LibriVoxFetcher
|
||||
from tools.ingest.nasa import NasaFetcher
|
||||
|
||||
|
||||
class _Resp:
|
||||
def __init__(self, data: bytes):
|
||||
self._data = data
|
||||
|
||||
def read(self):
|
||||
return self._data
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *a):
|
||||
return False
|
||||
|
||||
|
||||
def fake_opener(mapping):
|
||||
"""Dispatch a urllib Request to a canned payload by URL substring."""
|
||||
|
||||
def opener(req, timeout=None):
|
||||
url = req.full_url
|
||||
for key, payload in mapping.items():
|
||||
if key in url:
|
||||
if isinstance(payload, (bytes, bytearray)):
|
||||
return _Resp(bytes(payload))
|
||||
return _Resp(json.dumps(payload).encode("utf-8"))
|
||||
raise AssertionError(f"unexpected url: {url}")
|
||||
|
||||
return opener
|
||||
|
||||
|
||||
def test_librivox_fetcher():
|
||||
payload = {
|
||||
"books": [
|
||||
{
|
||||
"id": "123",
|
||||
"title": "Meditations",
|
||||
"url_librivox": "https://librivox.org/meditations/",
|
||||
"url_zip_file": "https://archive.org/download/meditations/meditations_mp3.zip",
|
||||
"authors": [{"first_name": "Marcus", "last_name": "Aurelius"}],
|
||||
"description": "Stoic philosophy.",
|
||||
}
|
||||
]
|
||||
}
|
||||
client = HttpClient(opener=fake_opener({"librivox.org/api": payload}))
|
||||
cands = LibriVoxFetcher(client).search("medit", limit=5)
|
||||
c = cands[0]
|
||||
assert c.source_archive == "librivox"
|
||||
assert c.license == "public_domain" and c.attribution == ""
|
||||
assert c.suggested_id == "librivox-meditations"
|
||||
assert c.media_url.endswith(".zip")
|
||||
assert c.media_ext == "zip"
|
||||
assert "Aurelius" in c.description
|
||||
|
||||
|
||||
def test_nasa_fetcher():
|
||||
search_payload = {
|
||||
"collection": {
|
||||
"items": [
|
||||
{
|
||||
"data": [
|
||||
{
|
||||
"nasa_id": "as08-14-2383",
|
||||
"title": "Earthrise",
|
||||
"description": "View of Earth from the Moon",
|
||||
"media_type": "video",
|
||||
}
|
||||
],
|
||||
"href": "https://images-assets.nasa.gov/video/as08-14-2383/collection.json",
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
asset_payload = [
|
||||
"https://images-assets.nasa.gov/video/as08-14-2383/as08-14-2383~orig.mp4",
|
||||
"https://images-assets.nasa.gov/video/as08-14-2383/as08-14-2383~thumb.jpg",
|
||||
]
|
||||
client = HttpClient(
|
||||
opener=fake_opener(
|
||||
{
|
||||
"images-api.nasa.gov/search": search_payload,
|
||||
"collection.json": asset_payload,
|
||||
}
|
||||
)
|
||||
)
|
||||
c = NasaFetcher(client).search("earth", limit=3)[0]
|
||||
assert c.suggested_id == "nasa-as08-14-2383"
|
||||
assert c.license == "public_domain" and c.attribution == ""
|
||||
assert c.media_url.endswith("orig.mp4")
|
||||
assert c.media_ext == "mp4"
|
||||
assert "Earth" in c.description
|
||||
|
||||
|
||||
def test_internet_archive_cc_by():
|
||||
meta_payload = {
|
||||
"metadata": {
|
||||
"identifier": "earthrise",
|
||||
"title": "Earthrise",
|
||||
"licenseurl": "http://creativecommons.org/licenses/by/4.0/",
|
||||
"creator": "NASA",
|
||||
"description": "Apollo 8 footage.",
|
||||
},
|
||||
"files": [
|
||||
{"name": "earthrise.mp4", "format": "h.264", "source": "original"},
|
||||
{"name": "earthrise.png", "format": "PNG", "source": "derivative"},
|
||||
],
|
||||
"server": "ia800100.us.archive.org",
|
||||
"dir": "/12/items/earthrise",
|
||||
}
|
||||
client = HttpClient(opener=fake_opener({"archive.org/metadata/earthrise": meta_payload}))
|
||||
c = InternetArchiveFetcher(client).resolve("earthrise")
|
||||
assert c.suggested_id == "ia-earthrise"
|
||||
assert c.license == "cc_by" and "NASA" in c.attribution
|
||||
assert c.media_url == "https://ia800100.us.archive.org/12/items/earthrise/earthrise.mp4"
|
||||
assert c.media_ext == "mp4"
|
||||
assert c.source_url == "https://archive.org/details/earthrise"
|
||||
|
||||
|
||||
def test_internet_archive_no_license_assumed_public_domain_and_flagged():
|
||||
meta_payload = {
|
||||
"metadata": {"identifier": "oldfilm", "title": "Old Film"},
|
||||
"files": [{"name": "oldfilm.mp4", "source": "original"}],
|
||||
"server": "ia.example.org",
|
||||
"dir": "/x/items/oldfilm",
|
||||
}
|
||||
client = HttpClient(opener=fake_opener({"archive.org/metadata/oldfilm": meta_payload}))
|
||||
c = InternetArchiveFetcher(client).resolve("oldfilm")
|
||||
assert c.license == "public_domain"
|
||||
assert "verify" in c.description.lower()
|
||||
|
||||
|
||||
def test_internet_archive_search_resolves_each_hit():
|
||||
search_payload = {"response": {"docs": [{"identifier": "earthrise"}]}}
|
||||
meta_payload = {
|
||||
"metadata": {
|
||||
"identifier": "earthrise",
|
||||
"title": "Earthrise",
|
||||
"licenseurl": "https://creativecommons.org/publicdomain/mark/1.0/",
|
||||
},
|
||||
"files": [{"name": "earthrise.mp4", "source": "original"}],
|
||||
"server": "ia.example.org",
|
||||
"dir": "/x",
|
||||
}
|
||||
client = HttpClient(
|
||||
opener=fake_opener(
|
||||
{
|
||||
"advancedsearch.php": search_payload,
|
||||
"archive.org/metadata/earthrise": meta_payload,
|
||||
}
|
||||
)
|
||||
)
|
||||
cands = InternetArchiveFetcher(client).search("earthrise", limit=5)
|
||||
assert [c.suggested_id for c in cands] == ["ia-earthrise"]
|
||||
assert cands[0].license == "public_domain"
|
||||
|
||||
|
||||
def test_deferred_stubs_raise_not_implemented():
|
||||
from tools.ingest.fma import FmaFetcher
|
||||
from tools.ingest.freesound import FreesoundFetcher
|
||||
from tools.ingest.musopen import MusopenFetcher
|
||||
|
||||
with pytest.raises(NotImplementedError):
|
||||
MusopenFetcher(None).search("x", limit=1)
|
||||
with pytest.raises(NotImplementedError):
|
||||
FmaFetcher(None).search("x", limit=1)
|
||||
with pytest.raises(NotImplementedError):
|
||||
FreesoundFetcher(None).search("x", limit=1)
|
||||
|
||||
|
||||
def test_fetchers_expose_archive_label():
|
||||
assert LibriVoxFetcher(None).archive == "librivox"
|
||||
assert NasaFetcher(None).archive == "nasa"
|
||||
assert InternetArchiveFetcher(None).archive == "internet_archive"
|
||||
@@ -0,0 +1,25 @@
|
||||
import pytest
|
||||
|
||||
from tools.ingest_cli import main
|
||||
|
||||
|
||||
def test_help_parses():
|
||||
with pytest.raises(SystemExit) as e:
|
||||
main(["--help"])
|
||||
assert e.value.code == 0
|
||||
|
||||
|
||||
def test_unknown_archive_errors_cleanly(capsys):
|
||||
rc = main(["bogus", "--query", "x"])
|
||||
assert rc == 2
|
||||
assert "unknown archive" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_missing_query_and_resolve_errors():
|
||||
assert main(["nasa"]) == 2
|
||||
|
||||
|
||||
def test_deferred_archive_reports_not_implemented(capsys):
|
||||
rc = main(["musopen", "--query", "bach"])
|
||||
assert rc == 3
|
||||
assert "deferred" in capsys.readouterr().err
|
||||
@@ -0,0 +1,193 @@
|
||||
import pytest
|
||||
|
||||
from hef.catalog import CatalogError, load_catalog, validate_catalog
|
||||
from tools.drafting import HeuristicProposer
|
||||
from tools.ingest.base import Candidate, ingest_candidate, ingest_search
|
||||
from tools.probe import Probe
|
||||
|
||||
|
||||
def make_candidate(**o):
|
||||
base = dict(
|
||||
source_archive="nasa",
|
||||
source_url="https://example.org/landing/x",
|
||||
media_url="https://example.org/media/x.mp4",
|
||||
title="Earthrise",
|
||||
license="public_domain",
|
||||
attribution="",
|
||||
suggested_id="nasa-earthrise",
|
||||
media_ext="mp4",
|
||||
description="a view of earth",
|
||||
)
|
||||
base.update(o)
|
||||
return Candidate(**base)
|
||||
|
||||
|
||||
def video_probe():
|
||||
return Probe(
|
||||
streams=[
|
||||
{
|
||||
"codec_type": "video",
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"disposition": {"attached_pic": 0},
|
||||
},
|
||||
{"codec_type": "audio"},
|
||||
],
|
||||
format={"duration": "10.0"},
|
||||
)
|
||||
|
||||
|
||||
def make_prober(probe):
|
||||
def prober(path):
|
||||
return probe
|
||||
|
||||
return prober
|
||||
|
||||
|
||||
def make_downloader(content=b"FAKEMEDIA"):
|
||||
def downloader(url, dest):
|
||||
dest.write_bytes(content)
|
||||
|
||||
return downloader
|
||||
|
||||
|
||||
def test_ingest_appends_one_proposed_record(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
media_root = tmp_path / "media"
|
||||
rec = ingest_candidate(
|
||||
make_candidate(),
|
||||
catalog_path=catalog,
|
||||
media_root=media_root,
|
||||
proposer=HeuristicProposer(),
|
||||
prober=make_prober(video_probe()),
|
||||
downloader=make_downloader(),
|
||||
)
|
||||
assert rec is not None
|
||||
records = load_catalog(catalog)
|
||||
assert len(records) == 1
|
||||
r = records[0]
|
||||
assert r.id == "nasa-earthrise"
|
||||
assert r.mode == "av"
|
||||
assert r.resolution == "1920x1080"
|
||||
assert r.duration_s == 10
|
||||
assert r.review_status == "proposed"
|
||||
assert r.reviewed_at is None
|
||||
assert r.rationale != ""
|
||||
assert r.file_path == "nasa/nasa-earthrise.mp4"
|
||||
assert r.notes == "a view of earth"
|
||||
# media written under media_root/<archive>/<id>.<ext>
|
||||
assert (media_root / "nasa" / "nasa-earthrise.mp4").exists()
|
||||
|
||||
|
||||
def test_ingest_is_idempotent(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
media_root = tmp_path / "media"
|
||||
kw = dict(
|
||||
catalog_path=catalog,
|
||||
media_root=media_root,
|
||||
proposer=HeuristicProposer(),
|
||||
prober=make_prober(video_probe()),
|
||||
downloader=make_downloader(),
|
||||
)
|
||||
first = ingest_candidate(make_candidate(), **kw)
|
||||
second = ingest_candidate(make_candidate(), **kw)
|
||||
assert first is not None
|
||||
assert second is None # skipped as duplicate
|
||||
assert len(load_catalog(catalog)) == 1
|
||||
|
||||
|
||||
def test_dominant_color_off_by_default(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
rec = ingest_candidate(
|
||||
make_candidate(),
|
||||
catalog_path=catalog,
|
||||
media_root=tmp_path / "media",
|
||||
proposer=HeuristicProposer(),
|
||||
prober=make_prober(video_probe()),
|
||||
downloader=make_downloader(),
|
||||
)
|
||||
assert rec.dominant_color == ""
|
||||
|
||||
|
||||
def test_dominant_color_computed_when_enabled(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
color_calls = []
|
||||
|
||||
def fake_color_fn(path, *, midpoint_s=0.0):
|
||||
color_calls.append(midpoint_s)
|
||||
return "#abcdef"
|
||||
|
||||
rec = ingest_candidate(
|
||||
make_candidate(),
|
||||
catalog_path=catalog,
|
||||
media_root=tmp_path / "media",
|
||||
proposer=HeuristicProposer(),
|
||||
prober=make_prober(video_probe()),
|
||||
downloader=make_downloader(),
|
||||
compute_color=True,
|
||||
color_fn=fake_color_fn,
|
||||
)
|
||||
assert rec.dominant_color == "#abcdef"
|
||||
assert color_calls # was invoked
|
||||
|
||||
|
||||
def test_audio_keeps_empty_color_even_when_enabled(tmp_path):
|
||||
audio = Probe(streams=[{"codec_type": "audio"}], format={"duration": "30.0"})
|
||||
|
||||
def boom(path, *, midpoint_s=0.0):
|
||||
raise AssertionError("must not compute color for audio")
|
||||
|
||||
rec = ingest_candidate(
|
||||
make_candidate(media_ext="mp3", source_archive="librivox", license="public_domain"),
|
||||
catalog_path=tmp_path / "library.jsonl",
|
||||
media_root=tmp_path / "media",
|
||||
proposer=HeuristicProposer(),
|
||||
prober=make_prober(audio),
|
||||
downloader=make_downloader(),
|
||||
compute_color=True,
|
||||
color_fn=boom,
|
||||
)
|
||||
assert rec.mode == "audio" and rec.dominant_color == ""
|
||||
|
||||
|
||||
def test_unmappable_license_raises_and_writes_nothing(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
with pytest.raises(CatalogError):
|
||||
ingest_candidate(
|
||||
make_candidate(license="all_rights_reserved"),
|
||||
catalog_path=catalog,
|
||||
media_root=tmp_path / "media",
|
||||
proposer=HeuristicProposer(),
|
||||
prober=make_prober(video_probe()),
|
||||
downloader=make_downloader(),
|
||||
)
|
||||
assert not catalog.exists() or load_catalog(catalog) == []
|
||||
|
||||
|
||||
def test_ingest_search_runs_over_hits(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
|
||||
class FakeFetcher:
|
||||
archive = "nasa"
|
||||
|
||||
def search(self, query, *, limit):
|
||||
return [
|
||||
make_candidate(suggested_id="nasa-a", media_url="u/a.mp4"),
|
||||
make_candidate(suggested_id="nasa-b", media_url="u/b.mp4"),
|
||||
][:limit]
|
||||
|
||||
def resolve(self, identifier): # pragma: no cover - not used here
|
||||
raise NotImplementedError
|
||||
|
||||
recs = ingest_search(
|
||||
FakeFetcher(),
|
||||
"earth",
|
||||
limit=2,
|
||||
catalog_path=catalog,
|
||||
media_root=tmp_path / "media",
|
||||
proposer=HeuristicProposer(),
|
||||
prober=make_prober(video_probe()),
|
||||
downloader=make_downloader(),
|
||||
)
|
||||
assert [r.id for r in recs] == ["nasa-a", "nasa-b"]
|
||||
validate_catalog(load_catalog(catalog))
|
||||
@@ -0,0 +1,84 @@
|
||||
import pytest
|
||||
|
||||
from hef.catalog import LICENSES
|
||||
from tools.licensing import librivox_license, normalize_license
|
||||
|
||||
|
||||
def test_cc_by_url_maps_and_requires_attribution():
|
||||
lic, attr = normalize_license(
|
||||
"https://creativecommons.org/licenses/by/4.0/", creator="Jane Doe"
|
||||
)
|
||||
assert lic == "cc_by" and "Jane Doe" in attr
|
||||
|
||||
|
||||
def test_cc_by_identifier_maps():
|
||||
lic, attr = normalize_license("CC BY 4.0", creator="Sam")
|
||||
assert lic == "cc_by" and attr
|
||||
|
||||
|
||||
def test_cc_by_nc_maps():
|
||||
lic, attr = normalize_license(
|
||||
"https://creativecommons.org/licenses/by-nc/4.0/", creator="Artist"
|
||||
)
|
||||
assert lic == "cc_by_nc" and "Artist" in attr
|
||||
|
||||
|
||||
def test_cc_by_without_creator_still_has_nonempty_attribution():
|
||||
# validate() rejects an attribution license with empty attribution, so the
|
||||
# normalizer must always produce a non-empty string for cc_by/cc_by_nc.
|
||||
lic, attr = normalize_license("https://creativecommons.org/licenses/by/4.0/")
|
||||
assert lic == "cc_by" and attr != ""
|
||||
|
||||
|
||||
def test_cc0_maps_with_empty_attribution():
|
||||
lic, attr = normalize_license("https://creativecommons.org/publicdomain/zero/1.0/")
|
||||
assert lic == "cc0" and attr == ""
|
||||
|
||||
|
||||
def test_cc0_identifier():
|
||||
lic, attr = normalize_license("CC0")
|
||||
assert lic == "cc0" and attr == ""
|
||||
|
||||
|
||||
def test_public_domain_mark():
|
||||
lic, attr = normalize_license(
|
||||
"https://creativecommons.org/publicdomain/mark/1.0/"
|
||||
)
|
||||
assert lic == "public_domain" and attr == ""
|
||||
|
||||
|
||||
def test_no_known_copyright_is_public_domain():
|
||||
lic, attr = normalize_license("No known copyright")
|
||||
assert lic == "public_domain" and attr == ""
|
||||
|
||||
|
||||
def test_public_domain_literal():
|
||||
lic, attr = normalize_license("public_domain")
|
||||
assert lic == "public_domain" and attr == ""
|
||||
|
||||
|
||||
def test_unmappable_rejected():
|
||||
with pytest.raises(ValueError):
|
||||
normalize_license("All Rights Reserved")
|
||||
|
||||
|
||||
def test_empty_rejected():
|
||||
with pytest.raises(ValueError):
|
||||
normalize_license("")
|
||||
|
||||
|
||||
def test_librivox_license_helper():
|
||||
lic, attr = librivox_license()
|
||||
assert lic == "public_domain" and attr == ""
|
||||
|
||||
|
||||
def test_all_outputs_in_vocab():
|
||||
samples = [
|
||||
"https://creativecommons.org/licenses/by/4.0/",
|
||||
"https://creativecommons.org/licenses/by-nc/4.0/",
|
||||
"CC0",
|
||||
"public domain",
|
||||
]
|
||||
for raw in samples:
|
||||
lic, _ = normalize_license(raw, creator="x")
|
||||
assert lic in LICENSES
|
||||
@@ -0,0 +1,38 @@
|
||||
from tools.mediatools import (
|
||||
compute_dominant_color,
|
||||
dominant_color_from_rgb,
|
||||
extract_frame,
|
||||
)
|
||||
|
||||
|
||||
def test_dominant_color_from_rgb_pure_red():
|
||||
assert dominant_color_from_rgb(b"\xff\x00\x00") == "#ff0000"
|
||||
|
||||
|
||||
def test_dominant_color_from_rgb_arbitrary():
|
||||
assert dominant_color_from_rgb(b"\x12\xab\x0f") == "#12ab0f"
|
||||
|
||||
|
||||
def test_compute_dominant_color_uses_runner():
|
||||
captured = {}
|
||||
|
||||
def fake_runner(args):
|
||||
captured["args"] = args
|
||||
return b"\x00\x80\xff"
|
||||
|
||||
color = compute_dominant_color("clip.mp4", midpoint_s=5.0, runner=fake_runner)
|
||||
assert color == "#0080ff"
|
||||
assert "ffmpeg" in captured["args"]
|
||||
assert "5.0" in captured["args"]
|
||||
|
||||
|
||||
def test_extract_frame_invokes_runner_and_returns_dest(tmp_path):
|
||||
dest = tmp_path / "frame.png"
|
||||
calls = []
|
||||
|
||||
def fake_runner(args):
|
||||
calls.append(args)
|
||||
|
||||
out = extract_frame("clip.mp4", dest, midpoint_s=2.0, runner=fake_runner)
|
||||
assert out == dest
|
||||
assert calls and "ffmpeg" in calls[0] and str(dest) in calls[0]
|
||||
@@ -0,0 +1,128 @@
|
||||
import pytest
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import (
|
||||
DEFAULT_CALIBRATION,
|
||||
AnalyticalOverlay,
|
||||
Calibration,
|
||||
ColorGrade,
|
||||
RenderPlan,
|
||||
Restyle,
|
||||
plan_alteration,
|
||||
render_plan_to_dict,
|
||||
)
|
||||
|
||||
|
||||
def _coord(left=0, right=0, dark=0, light=0):
|
||||
return Coordinate(left=left, right=right, dark=dark, light=light)
|
||||
|
||||
|
||||
def test_all_zero_knobs_is_the_unaltered_base():
|
||||
plan = plan_alteration(_coord())
|
||||
assert plan.is_identity
|
||||
assert plan.overlay.level == 0
|
||||
assert plan.overlay.intensity == 0.0
|
||||
assert plan.restyle.variant == 0
|
||||
assert plan.grade.tone == 0.0
|
||||
assert plan.grade.is_identity
|
||||
|
||||
|
||||
def test_left_drives_the_analytical_overlay_only():
|
||||
plan = plan_alteration(_coord(left=4))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.variant == 0 # Left does not touch the substrate
|
||||
assert plan.grade.tone == 0.0
|
||||
|
||||
|
||||
def test_right_selects_a_discrete_restyle_variant_only():
|
||||
plan = plan_alteration(_coord(right=2))
|
||||
assert plan.restyle.variant == 2
|
||||
assert plan.overlay.level == 0 # Right does not add overlay
|
||||
|
||||
|
||||
def test_left_and_right_stack_not_cancel():
|
||||
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
|
||||
plan = plan_alteration(_coord(left=4, right=4))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.restyle.variant == 4
|
||||
|
||||
|
||||
def test_light_pole_grades_warm_positive_tone():
|
||||
plan = plan_alteration(_coord(light=4))
|
||||
assert plan.grade.tone == 1.0
|
||||
assert not plan.grade.is_identity
|
||||
|
||||
|
||||
def test_dark_pole_grades_cool_negative_tone():
|
||||
assert plan_alteration(_coord(dark=4)).grade.tone == -1.0
|
||||
|
||||
|
||||
def test_equal_dark_and_light_is_identity_grade():
|
||||
# design §5: the mood center is the raw, ungraded footage
|
||||
assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity
|
||||
assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity
|
||||
|
||||
|
||||
def test_dark_minus_light_sets_intermediate_tone():
|
||||
assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5)
|
||||
assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5)
|
||||
|
||||
|
||||
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
|
||||
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
|
||||
assert plan.overlay.level == 4
|
||||
assert plan.restyle.variant == 2
|
||||
assert plan.grade.tone == -1.0
|
||||
assert not plan.is_identity
|
||||
|
||||
|
||||
def test_default_calibration_is_locked():
|
||||
# Session 0010: the knob->strength calibration is LOCKED to these values,
|
||||
# settled by eye in the simulator (closes the open session-0006 decision).
|
||||
# This pins the literal constants so they can't drift silently; changing the
|
||||
# locked feel is a deliberate edit here + in alteration.py.
|
||||
assert DEFAULT_CALIBRATION.mood_gain == 1.0
|
||||
assert DEFAULT_CALIBRATION.overlay_gain == 1.0
|
||||
assert DEFAULT_CALIBRATION.right_variant_map == (0, 1, 2, 3, 4)
|
||||
|
||||
|
||||
def test_default_calibration_is_behavior_preserving():
|
||||
# DEFAULT_CALIBRATION must reproduce the original three helpers exactly.
|
||||
for left in range(5):
|
||||
assert plan_alteration(_coord(left=left)).overlay.intensity == pytest.approx(left / 4)
|
||||
for right in range(5):
|
||||
assert plan_alteration(_coord(right=right)).restyle.variant == right
|
||||
for dark in range(5):
|
||||
for light in range(5):
|
||||
expected = (light - dark) / 4
|
||||
assert plan_alteration(_coord(dark=dark, light=light)).grade.tone == pytest.approx(expected)
|
||||
|
||||
|
||||
def test_custom_calibration_scales_mood_and_overlay():
|
||||
cal = Calibration(mood_gain=0.5, overlay_gain=0.5, right_variant_map=(0, 0, 1, 1, 2))
|
||||
assert plan_alteration(_coord(light=4), cal).grade.tone == pytest.approx(0.5)
|
||||
assert plan_alteration(_coord(left=4), cal).overlay.intensity == pytest.approx(0.5)
|
||||
assert plan_alteration(_coord(right=3), cal).restyle.variant == 1
|
||||
|
||||
|
||||
def test_calibration_gain_is_clamped_to_unit_range():
|
||||
cal = Calibration(mood_gain=10.0, overlay_gain=10.0)
|
||||
assert plan_alteration(_coord(light=4), cal).grade.tone == 1.0 # clamped, not 10
|
||||
assert plan_alteration(_coord(left=4), cal).overlay.intensity == 1.0
|
||||
|
||||
|
||||
def test_render_plan_to_dict_round_trips_the_numbers():
|
||||
d = render_plan_to_dict(plan_alteration(_coord(left=4, right=2, dark=4, light=0)))
|
||||
assert d == {
|
||||
"grade": {"tone": -1.0},
|
||||
"overlay": {"level": 4, "intensity": 1.0},
|
||||
"restyle": {"variant": 2},
|
||||
"is_identity": False,
|
||||
}
|
||||
|
||||
|
||||
def test_render_plan_is_frozen():
|
||||
plan = plan_alteration(_coord())
|
||||
with pytest.raises(Exception):
|
||||
plan.grade.tone = 0.5 # type: ignore[misc]
|
||||
@@ -0,0 +1,34 @@
|
||||
import pytest
|
||||
|
||||
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
|
||||
|
||||
|
||||
def test_audio_sources_are_the_four_distinct_sources():
|
||||
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
# The §6 table, row by row: position -> (audio_source, video).
|
||||
@pytest.mark.parametrize(
|
||||
"position,audio_source,video",
|
||||
[
|
||||
("off", "none", False),
|
||||
("white_noise", "white_noise", False),
|
||||
("music", "music", False),
|
||||
("audio_track", "audio_track", False),
|
||||
("video", "none", True),
|
||||
("music_video", "music", True),
|
||||
("audio_video", "audio_track", True),
|
||||
],
|
||||
)
|
||||
def test_resolve_content_matches_spec_table(position, audio_source, video):
|
||||
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
|
||||
|
||||
|
||||
def test_off_is_void_state_black_and_silent():
|
||||
r = resolve_content("off")
|
||||
assert r.video is False and r.audio_source == "none"
|
||||
|
||||
|
||||
def test_resolve_content_rejects_unknown_position():
|
||||
with pytest.raises(ValueError):
|
||||
resolve_content("bogus")
|
||||
@@ -0,0 +1,53 @@
|
||||
import pytest
|
||||
|
||||
from player.controls import (
|
||||
Controls,
|
||||
CONTENT_POSITIONS,
|
||||
ControlsError,
|
||||
validate_controls,
|
||||
parse_controls,
|
||||
)
|
||||
|
||||
|
||||
def test_content_positions_are_the_seven_from_the_spec():
|
||||
assert CONTENT_POSITIONS == frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
|
||||
|
||||
def test_valid_controls_pass_validation():
|
||||
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
|
||||
validate_controls(c) # does not raise
|
||||
|
||||
|
||||
def test_invalid_content_position_rejected():
|
||||
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(c)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
|
||||
@pytest.mark.parametrize("bad", [-1, 5, True])
|
||||
def test_out_of_range_or_non_int_knob_rejected(field, bad):
|
||||
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
kwargs[field] = bad
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(Controls(**kwargs))
|
||||
|
||||
|
||||
def test_parse_controls_from_mapping():
|
||||
c = parse_controls(
|
||||
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
|
||||
)
|
||||
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
|
||||
|
||||
|
||||
def test_parse_controls_rejects_unknown_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
|
||||
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
|
||||
|
||||
|
||||
def test_parse_controls_rejects_missing_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0})
|
||||
@@ -0,0 +1,112 @@
|
||||
from dataclasses import dataclass
|
||||
|
||||
import pytest
|
||||
|
||||
from player.controls import Controls
|
||||
from player.state import Player, Playback, Transition, TransitionKind
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FakeClip:
|
||||
id: str
|
||||
|
||||
|
||||
LIB = [FakeClip("base-a"), FakeClip("base-b")]
|
||||
|
||||
|
||||
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
|
||||
return Controls(content, left, right, dark, light, volume, brightness)
|
||||
|
||||
|
||||
def test_first_update_to_video_fades_in_from_black():
|
||||
p = Player(LIB)
|
||||
t = p.update(_controls(content="video"))
|
||||
assert t.kind == TransitionKind.FADE_FROM_BLACK
|
||||
assert t.playback.clip_id == "base-a"
|
||||
assert t.playback.content.video is True
|
||||
|
||||
|
||||
def test_off_from_video_fades_to_black_and_silences():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video"))
|
||||
t = p.update(_controls(content="off"))
|
||||
assert t.kind == TransitionKind.FADE_TO_BLACK
|
||||
assert t.playback.clip_id is None
|
||||
assert t.playback.content.audio_source == "none"
|
||||
|
||||
|
||||
def test_no_change_yields_none_transition():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=1))
|
||||
t = p.update(_controls(content="video", left=1))
|
||||
assert t.kind == TransitionKind.NONE
|
||||
|
||||
|
||||
def test_grade_change_is_a_live_update_not_a_crossfade():
|
||||
# design §4.3: the Dark/Light grade is a continuous runtime op
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", dark=0, light=0))
|
||||
t = p.update(_controls(content="video", dark=4, light=0))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_overlay_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=0))
|
||||
t = p.update(_controls(content="video", left=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.overlay.level == 4
|
||||
|
||||
|
||||
def test_restyle_change_crossfades_the_substrate():
|
||||
# design §4.3: the Right v2v substrate is a pre-baked variant swap
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", right=0))
|
||||
t = p.update(_controls(content="video", right=4))
|
||||
assert t.kind == TransitionKind.CROSSFADE
|
||||
assert t.playback.plan.restyle.variant == 4
|
||||
|
||||
|
||||
def test_volume_only_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", volume=1))
|
||||
t = p.update(_controls(content="video", volume=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.volume == 4
|
||||
|
||||
|
||||
def test_audio_source_change_while_black_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="white_noise"))
|
||||
t = p.update(_controls(content="music"))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.content.audio_source == "music"
|
||||
|
||||
|
||||
def test_injected_base_chooser_is_used():
|
||||
p = Player(LIB, choose_base=lambda lib: lib[1])
|
||||
t = p.update(_controls(content="video"))
|
||||
assert t.playback.clip_id == "base-b"
|
||||
|
||||
|
||||
def test_empty_library_with_video_raises():
|
||||
p = Player([])
|
||||
with pytest.raises(ValueError):
|
||||
p.update(_controls(content="video"))
|
||||
|
||||
|
||||
def test_off_with_empty_library_is_fine():
|
||||
p = Player([])
|
||||
# levels at 0 match the initial black state, so this is a no-op transition
|
||||
t = p.update(_controls(content="off", volume=0, brightness=0))
|
||||
assert t.kind == TransitionKind.NONE # already black at init
|
||||
assert t.playback.clip_id is None
|
||||
|
||||
|
||||
def test_off_with_levels_from_black_is_a_live_update():
|
||||
p = Player([])
|
||||
t = p.update(_controls(content="off", volume=3, brightness=2))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE # black->black, levels set
|
||||
assert t.playback.clip_id is None
|
||||
assert t.playback.volume == 3
|
||||
@@ -0,0 +1,65 @@
|
||||
import json
|
||||
|
||||
from tools.probe import Probe, probe_file
|
||||
|
||||
|
||||
def _runner_for(payload):
|
||||
def fake_runner(args): # mimics subprocess.run(...).stdout
|
||||
return json.dumps(payload)
|
||||
|
||||
return fake_runner
|
||||
|
||||
|
||||
def test_probe_parses_streams_and_format():
|
||||
runner = _runner_for(
|
||||
{
|
||||
"streams": [
|
||||
{
|
||||
"codec_type": "video",
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"disposition": {"attached_pic": 0},
|
||||
},
|
||||
{"codec_type": "audio"},
|
||||
],
|
||||
"format": {"duration": "12.5"},
|
||||
}
|
||||
)
|
||||
p = probe_file("x.mp4", runner=runner)
|
||||
assert isinstance(p, Probe)
|
||||
assert any(s["codec_type"] == "video" for s in p.streams)
|
||||
assert p.format["duration"] == "12.5"
|
||||
|
||||
|
||||
def test_probe_audio_only():
|
||||
runner = _runner_for(
|
||||
{"streams": [{"codec_type": "audio"}], "format": {"duration": "30.0"}}
|
||||
)
|
||||
p = probe_file("x.mp3", runner=runner)
|
||||
assert [s["codec_type"] for s in p.streams] == ["audio"]
|
||||
|
||||
|
||||
def test_probe_audio_with_cover_art():
|
||||
runner = _runner_for(
|
||||
{
|
||||
"streams": [
|
||||
{"codec_type": "audio"},
|
||||
{
|
||||
"codec_type": "video",
|
||||
"width": 600,
|
||||
"height": 600,
|
||||
"disposition": {"attached_pic": 1},
|
||||
},
|
||||
],
|
||||
"format": {"duration": "200.0"},
|
||||
}
|
||||
)
|
||||
p = probe_file("x.mp3", runner=runner)
|
||||
cover = [s for s in p.streams if s.get("disposition", {}).get("attached_pic")]
|
||||
assert len(cover) == 1
|
||||
|
||||
|
||||
def test_probe_handles_missing_keys():
|
||||
runner = _runner_for({})
|
||||
p = probe_file("x.mp4", runner=runner)
|
||||
assert p.streams == [] and p.format == {}
|
||||
@@ -0,0 +1,59 @@
|
||||
from hef.catalog import Record, validate
|
||||
from hef.selection import Coordinate
|
||||
from tools.review import approve, proposed_records
|
||||
|
||||
|
||||
def make_record(**o):
|
||||
base = dict(
|
||||
id="a",
|
||||
title="t",
|
||||
source_url="u",
|
||||
source_archive="nasa",
|
||||
license="public_domain",
|
||||
mode="video",
|
||||
left=0,
|
||||
right=0,
|
||||
dark=0,
|
||||
light=0,
|
||||
duration_s=1,
|
||||
file_path="p",
|
||||
)
|
||||
base.update(o)
|
||||
return Record(**base)
|
||||
|
||||
|
||||
def test_proposed_records_filters():
|
||||
a = make_record(id="a", review_status="proposed")
|
||||
b = make_record(id="b", review_status="approved")
|
||||
assert [r.id for r in proposed_records([a, b])] == ["a"]
|
||||
|
||||
|
||||
def test_approve_sets_status_and_timestamp():
|
||||
r = make_record(review_status="proposed")
|
||||
out = approve(r, reviewed_at="2026-06-04T13:00:00+00:00")
|
||||
assert out.review_status == "approved"
|
||||
assert out.reviewed_at == "2026-06-04T13:00:00+00:00"
|
||||
|
||||
|
||||
def test_approve_can_override_coordinates():
|
||||
r = make_record(left=0, right=0, dark=0, light=0, review_status="proposed")
|
||||
out = approve(r, reviewed_at="t", coordinate=Coordinate(4, 1, 2, 3))
|
||||
assert (out.left, out.right, out.dark, out.light) == (4, 1, 2, 3)
|
||||
|
||||
|
||||
def test_approve_does_not_mutate_input():
|
||||
r = make_record(review_status="proposed")
|
||||
approve(r, reviewed_at="t")
|
||||
assert r.review_status == "proposed" and r.reviewed_at is None
|
||||
|
||||
|
||||
def test_approve_can_set_rationale():
|
||||
r = make_record(review_status="proposed", rationale="auto")
|
||||
out = approve(r, reviewed_at="t", rationale="human note")
|
||||
assert out.rationale == "human note"
|
||||
|
||||
|
||||
def test_approved_record_revalidates():
|
||||
r = make_record(review_status="proposed")
|
||||
out = approve(r, reviewed_at="t", coordinate=Coordinate(2, 2, 1, 1))
|
||||
validate(out) # must not raise
|
||||
@@ -0,0 +1,103 @@
|
||||
import io
|
||||
|
||||
import pytest
|
||||
|
||||
from hef.catalog import Record, load_catalog, save_catalog
|
||||
from tools.review_cli import main
|
||||
|
||||
|
||||
def make_record(**o):
|
||||
base = dict(
|
||||
id="a",
|
||||
title="t",
|
||||
source_url="u",
|
||||
source_archive="nasa",
|
||||
license="public_domain",
|
||||
mode="video",
|
||||
left=0,
|
||||
right=0,
|
||||
dark=0,
|
||||
light=0,
|
||||
duration_s=1,
|
||||
file_path="nasa/a.mp4",
|
||||
)
|
||||
base.update(o)
|
||||
return Record(**base)
|
||||
|
||||
|
||||
def test_help_parses():
|
||||
with pytest.raises(SystemExit) as e:
|
||||
main(["--help"])
|
||||
assert e.value.code == 0
|
||||
|
||||
|
||||
def test_no_proposed_records(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
save_catalog([make_record(id="x", review_status="approved", reviewed_at="t")], catalog)
|
||||
out = io.StringIO()
|
||||
rc = main(["--catalog", str(catalog), "--no-preview"], out=out)
|
||||
assert rc == 0
|
||||
assert "no proposed records" in out.getvalue()
|
||||
|
||||
|
||||
def test_accept_flips_to_approved(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
save_catalog([make_record(id="a", review_status="proposed")], catalog)
|
||||
out = io.StringIO()
|
||||
rc = main(
|
||||
["--catalog", str(catalog), "--no-preview"],
|
||||
input_fn=lambda prompt: "a",
|
||||
now_fn=lambda: "2026-06-04T13:00:00+00:00",
|
||||
out=out,
|
||||
)
|
||||
assert rc == 0
|
||||
r = load_catalog(catalog)[0]
|
||||
assert r.review_status == "approved"
|
||||
assert r.reviewed_at == "2026-06-04T13:00:00+00:00"
|
||||
|
||||
|
||||
def test_skip_leaves_proposed(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
save_catalog([make_record(id="a", review_status="proposed")], catalog)
|
||||
rc = main(
|
||||
["--catalog", str(catalog), "--no-preview"],
|
||||
input_fn=lambda prompt: "s",
|
||||
out=io.StringIO(),
|
||||
)
|
||||
assert rc == 0
|
||||
assert load_catalog(catalog)[0].review_status == "proposed"
|
||||
|
||||
|
||||
def test_edit_overrides_coordinates(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
save_catalog([make_record(id="a", review_status="proposed")], catalog)
|
||||
answers = iter(["e", "4", "1", "2", "3"])
|
||||
rc = main(
|
||||
["--catalog", str(catalog), "--no-preview"],
|
||||
input_fn=lambda prompt: next(answers),
|
||||
now_fn=lambda: "t",
|
||||
out=io.StringIO(),
|
||||
)
|
||||
assert rc == 0
|
||||
r = load_catalog(catalog)[0]
|
||||
assert (r.left, r.right, r.dark, r.light) == (4, 1, 2, 3)
|
||||
assert r.review_status == "approved"
|
||||
|
||||
|
||||
def test_quit_stops_walk(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
save_catalog(
|
||||
[
|
||||
make_record(id="a", review_status="proposed"),
|
||||
make_record(id="b", review_status="proposed"),
|
||||
],
|
||||
catalog,
|
||||
)
|
||||
rc = main(
|
||||
["--catalog", str(catalog), "--no-preview"],
|
||||
input_fn=lambda prompt: "q",
|
||||
out=io.StringIO(),
|
||||
)
|
||||
assert rc == 0
|
||||
statuses = {r.id: r.review_status for r in load_catalog(catalog)}
|
||||
assert statuses == {"a": "proposed", "b": "proposed"}
|
||||
@@ -0,0 +1,87 @@
|
||||
import random
|
||||
|
||||
import pytest
|
||||
|
||||
from hef.catalog import Record
|
||||
from hef.selection import Coordinate, Weights, ranked_candidates, select
|
||||
|
||||
|
||||
def make_record(**overrides):
|
||||
base = dict(
|
||||
id="r",
|
||||
title="t",
|
||||
source_url="u",
|
||||
source_archive="internet_archive",
|
||||
license="public_domain",
|
||||
mode="video",
|
||||
left=0,
|
||||
right=0,
|
||||
dark=0,
|
||||
light=0,
|
||||
duration_s=600,
|
||||
file_path="",
|
||||
)
|
||||
base.update(overrides)
|
||||
return Record(**base)
|
||||
|
||||
|
||||
def test_ranked_candidates_sorts_nearest_first_with_distances():
|
||||
near = make_record(id="near", mode="video", left=1, right=1, dark=0, light=0)
|
||||
far = make_record(id="far", mode="video", left=4, right=4, dark=4, light=4)
|
||||
ranked = ranked_candidates([far, near], Coordinate(0, 0, 0, 0), "video")
|
||||
assert [r.id for r, _ in ranked] == ["near", "far"]
|
||||
assert ranked[0][1] < ranked[1][1]
|
||||
|
||||
|
||||
def test_ranked_candidates_caps_at_pool_size():
|
||||
recs = [make_record(id=f"r{i}", mode="video", left=i % 5) for i in range(10)]
|
||||
ranked = ranked_candidates(recs, Coordinate(0, 0, 0, 0), "video", pool_size=3)
|
||||
assert len(ranked) == 3
|
||||
|
||||
|
||||
def test_ranked_candidates_filters_by_mode():
|
||||
a = make_record(id="aud", mode="audio")
|
||||
v = make_record(id="vid", mode="video")
|
||||
ranked = ranked_candidates([a, v], Coordinate(0, 0, 0, 0), "audio")
|
||||
assert [r.id for r, _ in ranked] == ["aud"]
|
||||
|
||||
|
||||
def test_ranked_candidates_av_falls_back_to_audio_and_video():
|
||||
a = make_record(id="aud", mode="audio")
|
||||
v = make_record(id="vid", mode="video")
|
||||
ranked = ranked_candidates([a, v], Coordinate(0, 0, 0, 0), "av", pool_size=4)
|
||||
assert {r.id for r, _ in ranked} == {"aud", "vid"}
|
||||
|
||||
|
||||
def test_ranked_candidates_approved_only():
|
||||
p = make_record(id="prop", mode="video", review_status="proposed")
|
||||
ok = make_record(id="appr", mode="video", review_status="approved")
|
||||
ranked = ranked_candidates([p, ok], Coordinate(0, 0, 0, 0), "video", approved_only=True)
|
||||
assert [r.id for r, _ in ranked] == ["appr"]
|
||||
|
||||
|
||||
def test_ranked_candidates_rejects_none_mode():
|
||||
with pytest.raises(ValueError):
|
||||
ranked_candidates([], Coordinate(0, 0, 0, 0), "none")
|
||||
|
||||
|
||||
def test_select_returns_rank_one_of_ranked_candidates():
|
||||
recs = [
|
||||
make_record(id="near", mode="video", left=1),
|
||||
make_record(id="far", mode="video", left=4),
|
||||
]
|
||||
coord = Coordinate(0, 0, 0, 0)
|
||||
ranked = ranked_candidates(recs, coord, "video")
|
||||
assert select(recs, coord, "video", rng=None).id == ranked[0][0].id
|
||||
|
||||
|
||||
def test_select_none_mode_still_returns_none():
|
||||
recs = [make_record(id="v", mode="video")]
|
||||
assert select(recs, Coordinate(0, 0, 0, 0), "none") is None
|
||||
|
||||
|
||||
def test_select_shuffles_within_pool_when_rng_given():
|
||||
recs = [make_record(id=f"r{i}", mode="video", left=i) for i in range(5)]
|
||||
coord = Coordinate(0, 0, 0, 0)
|
||||
picks = {select(recs, coord, "video", pool_size=5, rng=random.Random(s)).id for s in range(20)}
|
||||
assert len(picks) > 1
|
||||
@@ -0,0 +1,90 @@
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from simulator.app import create_app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def manifest_path(tmp_path):
|
||||
p = tmp_path / "manifest.json"
|
||||
p.write_text(json.dumps({
|
||||
"clips": [{
|
||||
"id": "forest",
|
||||
"title": "neutral forest",
|
||||
"base_file": "forest/base.mp4",
|
||||
"license": "poc", "source": "hef-poc",
|
||||
"right_variants": {"4": {"file": "forest/right4.mp4"}},
|
||||
"annotations": [{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1}],
|
||||
"strings": {"en": {"detected.water": "flowing water"}},
|
||||
}]
|
||||
}))
|
||||
return p
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(manifest_path):
|
||||
return TestClient(create_app(manifest_path=manifest_path))
|
||||
|
||||
|
||||
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
|
||||
return dict(content=content, left=left, right=right, dark=dark,
|
||||
light=light, volume=volume, brightness=brightness)
|
||||
|
||||
|
||||
def test_alteration_returns_the_engine_plan(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(left=4, right=2, dark=4)})
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["plan"]["overlay"]["level"] == 4
|
||||
assert data["plan"]["restyle"]["variant"] == 2
|
||||
assert data["plan"]["grade"]["tone"] == -1.0
|
||||
assert data["content"]["video"] is True
|
||||
|
||||
|
||||
def test_alteration_honors_off_as_black(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(content="off")})
|
||||
data = resp.json()
|
||||
assert data["content"]["video"] is False
|
||||
|
||||
|
||||
def test_alteration_accepts_calibration(client):
|
||||
body = {"controls": _controls(light=4),
|
||||
"calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "right_variant_map": [0, 1, 2, 3, 4]}}
|
||||
resp = client.post("/api/alteration", json=body)
|
||||
assert resp.json()["plan"]["grade"]["tone"] == 0.5
|
||||
|
||||
|
||||
def test_alteration_rejects_out_of_range_knob(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(left=7)})
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
def test_alteration_rejects_bad_content(client):
|
||||
resp = client.post("/api/alteration", json={"controls": _controls(content="banana")})
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
def test_clips_returns_the_manifest(client):
|
||||
resp = client.get("/api/clips")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["clips"][0]["id"] == "forest"
|
||||
assert data["clips"][0]["right_variants"]["0"]["file"] == "forest/base.mp4"
|
||||
assert data["clips"][0]["annotations"][0]["key"] == "detected.water"
|
||||
|
||||
|
||||
def test_retired_selection_endpoints_are_gone(client):
|
||||
# The route no longer exists; the static catch-all yields 404 on GET and
|
||||
# 405 on the (now-unrouted) POST. Either proves the endpoint is gone.
|
||||
assert client.post("/api/select", json={}).status_code in (404, 405)
|
||||
assert client.get("/api/catalog/meta").status_code == 404
|
||||
|
||||
|
||||
def test_index_is_served():
|
||||
client = TestClient(create_app())
|
||||
resp = client.get("/")
|
||||
assert resp.status_code == 200
|
||||
assert "text/html" in resp.headers["content-type"]
|
||||
assert "Alteration" in resp.text
|
||||
@@ -0,0 +1,94 @@
|
||||
from tools.probe import Probe
|
||||
from tools.tagging import Tags, derive_tags
|
||||
|
||||
|
||||
def test_video_and_audio_is_av():
|
||||
p = Probe(
|
||||
streams=[
|
||||
{
|
||||
"codec_type": "video",
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"disposition": {"attached_pic": 0},
|
||||
},
|
||||
{"codec_type": "audio"},
|
||||
],
|
||||
format={"duration": "12.5"},
|
||||
)
|
||||
t = derive_tags(p)
|
||||
assert isinstance(t, Tags)
|
||||
assert t.mode == "av"
|
||||
assert t.resolution == "1920x1080"
|
||||
assert t.duration_s == 13 # round(12.5)
|
||||
|
||||
|
||||
def test_audio_only():
|
||||
p = Probe(streams=[{"codec_type": "audio"}], format={"duration": "30.0"})
|
||||
t = derive_tags(p)
|
||||
assert t.mode == "audio" and t.resolution == "" and t.duration_s == 30
|
||||
|
||||
|
||||
def test_cover_art_stays_audio():
|
||||
p = Probe(
|
||||
streams=[
|
||||
{"codec_type": "audio", "duration": "30.0"},
|
||||
{
|
||||
"codec_type": "video",
|
||||
"width": 600,
|
||||
"height": 600,
|
||||
"disposition": {"attached_pic": 1},
|
||||
},
|
||||
],
|
||||
format={"duration": "30.0"},
|
||||
)
|
||||
t = derive_tags(p)
|
||||
assert t.mode == "audio" and t.resolution == "" and t.duration_s == 30
|
||||
|
||||
|
||||
def test_video_only():
|
||||
p = Probe(
|
||||
streams=[
|
||||
{
|
||||
"codec_type": "video",
|
||||
"width": 1280,
|
||||
"height": 720,
|
||||
"disposition": {"attached_pic": 0},
|
||||
}
|
||||
],
|
||||
format={"duration": "5.0"},
|
||||
)
|
||||
t = derive_tags(p)
|
||||
assert t.mode == "video" and t.resolution == "1280x720"
|
||||
|
||||
|
||||
def test_duration_falls_back_to_longest_stream():
|
||||
p = Probe(
|
||||
streams=[
|
||||
{"codec_type": "audio", "duration": "10.0"},
|
||||
{
|
||||
"codec_type": "video",
|
||||
"width": 640,
|
||||
"height": 480,
|
||||
"duration": "42.4",
|
||||
"disposition": {"attached_pic": 0},
|
||||
},
|
||||
],
|
||||
format={},
|
||||
)
|
||||
t = derive_tags(p)
|
||||
assert t.duration_s == 42 # round(42.4), longest stream
|
||||
|
||||
|
||||
def test_duration_never_negative_and_zero_default():
|
||||
p = Probe(streams=[{"codec_type": "audio"}], format={})
|
||||
t = derive_tags(p)
|
||||
assert t.duration_s == 0
|
||||
|
||||
|
||||
def test_video_without_dimensions_yields_empty_resolution():
|
||||
p = Probe(
|
||||
streams=[{"codec_type": "video", "disposition": {"attached_pic": 0}}],
|
||||
format={"duration": "3.0"},
|
||||
)
|
||||
t = derive_tags(p)
|
||||
assert t.mode == "video" and t.resolution == ""
|
||||
@@ -0,0 +1,134 @@
|
||||
"""End-to-end: ingest (mocked boundaries) -> review -> select.
|
||||
|
||||
Plus opt-in tests that invoke real ffprobe/ffmpeg, skipped when absent.
|
||||
"""
|
||||
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from hef.catalog import load_catalog, validate_catalog
|
||||
from hef.selection import Coordinate, select
|
||||
from tools.drafting import HeuristicProposer
|
||||
from tools.ingest.base import Candidate, ingest_candidate
|
||||
from tools.mediatools import compute_dominant_color
|
||||
from tools.probe import probe_file
|
||||
from tools.review import approve, proposed_records
|
||||
from tools.tagging import derive_tags
|
||||
|
||||
|
||||
class _FakeFetcher:
|
||||
archive = "nasa"
|
||||
|
||||
def search(self, query, *, limit):
|
||||
return [
|
||||
Candidate(
|
||||
source_archive="nasa",
|
||||
source_url="https://example.org/landing",
|
||||
media_url="https://example.org/x.mp4",
|
||||
title="Earthrise",
|
||||
license="public_domain",
|
||||
attribution="",
|
||||
suggested_id="nasa-earthrise",
|
||||
media_ext="mp4",
|
||||
description="earth from the moon",
|
||||
)
|
||||
][:limit]
|
||||
|
||||
def resolve(self, identifier):
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
def _av_prober(path):
|
||||
from tools.probe import Probe
|
||||
|
||||
return Probe(
|
||||
streams=[
|
||||
{"codec_type": "video", "width": 1920, "height": 1080, "disposition": {"attached_pic": 0}},
|
||||
{"codec_type": "audio"},
|
||||
],
|
||||
format={"duration": "10.0"},
|
||||
)
|
||||
|
||||
|
||||
def _fake_downloader(url, dest):
|
||||
dest.write_bytes(b"FAKE")
|
||||
|
||||
|
||||
def test_end_to_end_ingest_review_select(tmp_path):
|
||||
catalog = tmp_path / "library.jsonl"
|
||||
media_root = tmp_path / "media"
|
||||
|
||||
# Ingest one candidate (all boundaries faked).
|
||||
candidate = _FakeFetcher().search("earth", limit=1)[0]
|
||||
rec = ingest_candidate(
|
||||
candidate,
|
||||
catalog_path=catalog,
|
||||
media_root=media_root,
|
||||
proposer=HeuristicProposer(),
|
||||
prober=_av_prober,
|
||||
downloader=_fake_downloader,
|
||||
)
|
||||
assert rec is not None and rec.review_status == "proposed"
|
||||
|
||||
# The proposed record can't be selected with approved_only yet.
|
||||
loaded = load_catalog(catalog)
|
||||
validate_catalog(loaded)
|
||||
coord = Coordinate(loaded[0].left, loaded[0].right, loaded[0].dark, loaded[0].light)
|
||||
assert select(loaded, coord, "av", approved_only=True) is None
|
||||
|
||||
# Review: approve it, persist.
|
||||
pending = proposed_records(loaded)
|
||||
assert len(pending) == 1
|
||||
approved = approve(pending[0], reviewed_at="2026-06-04T13:00:00+00:00")
|
||||
from hef.catalog import save_catalog
|
||||
|
||||
idx = {r.id: i for i, r in enumerate(loaded)}
|
||||
loaded[idx[approved.id]] = approved
|
||||
save_catalog(loaded, catalog)
|
||||
|
||||
# Reload, validate, and now select() finds it.
|
||||
final = load_catalog(catalog)
|
||||
validate_catalog(final)
|
||||
picked = select(final, coord, "av", approved_only=True)
|
||||
assert picked is not None and picked.id == "nasa-earthrise"
|
||||
assert picked.review_status == "approved"
|
||||
|
||||
|
||||
_HAS_FFMPEG = shutil.which("ffmpeg") is not None
|
||||
_HAS_FFPROBE = shutil.which("ffprobe") is not None
|
||||
|
||||
|
||||
@pytest.mark.skipif(not (_HAS_FFMPEG and _HAS_FFPROBE), reason="ffmpeg/ffprobe not installed")
|
||||
def test_real_ffprobe_on_generated_clip(tmp_path):
|
||||
clip = tmp_path / "clip.mp4"
|
||||
subprocess.run(
|
||||
[
|
||||
"ffmpeg", "-v", "quiet", "-y",
|
||||
"-f", "lavfi", "-i", "testsrc=duration=1:size=320x240:rate=30",
|
||||
"-f", "lavfi", "-i", "sine=frequency=440:duration=1",
|
||||
"-shortest", str(clip),
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
tags = derive_tags(probe_file(clip))
|
||||
assert tags.mode == "av"
|
||||
assert tags.resolution == "320x240"
|
||||
assert tags.duration_s == 1
|
||||
|
||||
|
||||
@pytest.mark.skipif(not _HAS_FFMPEG, reason="ffmpeg not installed")
|
||||
def test_real_dominant_color_on_generated_clip(tmp_path):
|
||||
clip = tmp_path / "clip.mp4"
|
||||
subprocess.run(
|
||||
[
|
||||
"ffmpeg", "-v", "quiet", "-y",
|
||||
"-f", "lavfi", "-i", "color=c=red:duration=1:size=320x240:rate=30",
|
||||
str(clip),
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
color = compute_dominant_color(clip, midpoint_s=0.5)
|
||||
assert re.fullmatch(r"#[0-9a-f]{6}", color)
|
||||
@@ -0,0 +1,5 @@
|
||||
def test_tools_package_imports():
|
||||
import tools
|
||||
import tools.http
|
||||
|
||||
assert tools is not None
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Heuristic coordinate proposer — a deterministic, rule-based DRAFT seed.
|
||||
|
||||
The four curatorial coordinates are a human act (design §11: no automatic ML
|
||||
coordinate tagging). This proposer only suggests a starting point written as
|
||||
review_status='proposed'; a human blesses every coordinate before selection.
|
||||
There is no ML model here — just archive priors (brain plane) and keyword
|
||||
nudges (mood plane), grounded in the §8 sourcing→axis map.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Protocol
|
||||
|
||||
from hef.selection import Coordinate
|
||||
|
||||
COORD_MIN, COORD_MAX = 0, 4
|
||||
|
||||
# Brain plane (left = verbal/spoken/educational, right = music/wordless/abstract).
|
||||
_ARCHIVE_PRIORS = {
|
||||
"librivox": (4, 0), # spoken readings
|
||||
"internet_archive": (3, 1), # educational / industrial / Prelinger
|
||||
"musopen": (0, 4), # classical music
|
||||
"fma": (0, 4), # music
|
||||
"nasa": (0, 4), # wordless awe
|
||||
"freesound": (0, 4), # abstract field recordings
|
||||
}
|
||||
_ARCHIVE_DEFAULT = (2, 2)
|
||||
|
||||
# Mood plane keywords (substring match so "thunderstorm" trips "storm").
|
||||
_DARK_WORDS = (
|
||||
"storm", "night", "noir", "requiem", "minor", "war", "funeral",
|
||||
"decay", "death", "grief", "dusk", "winter", "mourning", "shadow",
|
||||
)
|
||||
_LIGHT_WORDS = (
|
||||
"sunrise", "dawn", "garden", "spring", "joy", "hope", "major",
|
||||
"bright", "bloom", "summer", "smile", "light", "morning",
|
||||
)
|
||||
|
||||
|
||||
def _clamp(v: int) -> int:
|
||||
return max(COORD_MIN, min(COORD_MAX, v))
|
||||
|
||||
|
||||
@dataclass
|
||||
class Signals:
|
||||
title: str
|
||||
description: str
|
||||
source_archive: str
|
||||
mode: str
|
||||
duration_s: int
|
||||
|
||||
|
||||
@dataclass
|
||||
class Draft:
|
||||
coordinate: Coordinate
|
||||
rationale: str
|
||||
|
||||
|
||||
class Proposer(Protocol):
|
||||
def propose(self, signals: Signals) -> Draft: ...
|
||||
|
||||
|
||||
class HeuristicProposer:
|
||||
def propose(self, signals: Signals) -> Draft:
|
||||
left, right = _ARCHIVE_PRIORS.get(
|
||||
signals.source_archive, _ARCHIVE_DEFAULT
|
||||
)
|
||||
|
||||
text = f"{signals.title} {signals.description}".lower()
|
||||
dark_hits = [w for w in _DARK_WORDS if w in text]
|
||||
light_hits = [w for w in _LIGHT_WORDS if w in text]
|
||||
dark = _clamp(len(dark_hits))
|
||||
light = _clamp(len(light_hits))
|
||||
|
||||
coordinate = Coordinate(_clamp(left), _clamp(right), dark, light)
|
||||
|
||||
# Rationale: name the dominant brain prior + any mood keywords.
|
||||
brain = (
|
||||
f"{signals.source_archive} → "
|
||||
+ ("strong left" if left >= right else "strong right")
|
||||
)
|
||||
mood_bits = []
|
||||
if dark_hits:
|
||||
mood_bits.append(f"{dark_hits[0]!r} → dark")
|
||||
if light_hits:
|
||||
mood_bits.append(f"{light_hits[0]!r} → light")
|
||||
rationale = "; ".join([brain, *mood_bits])
|
||||
|
||||
return Draft(coordinate=coordinate, rationale=rationale)
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Minimal HTTP client over urllib: timeout, small retry, user-agent. Injectable."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json as _json
|
||||
import time
|
||||
import urllib.request
|
||||
|
||||
USER_AGENT = "human-experience-filter-ingest/0.1 (+local art installation)"
|
||||
|
||||
|
||||
class HttpClient:
|
||||
def __init__(self, *, timeout=30.0, retries=2, opener=None):
|
||||
self.timeout, self.retries = timeout, retries
|
||||
self._opener = opener or urllib.request.urlopen
|
||||
|
||||
def get_bytes(self, url, *, headers=None):
|
||||
last = None
|
||||
for attempt in range(self.retries + 1):
|
||||
try:
|
||||
req = urllib.request.Request(
|
||||
url, headers={"User-Agent": USER_AGENT, **(headers or {})}
|
||||
)
|
||||
with self._opener(req, timeout=self.timeout) as resp:
|
||||
return resp.read()
|
||||
except Exception as exc: # noqa: BLE001 - retried below
|
||||
last = exc
|
||||
if attempt < self.retries:
|
||||
time.sleep(0.2 * (attempt + 1))
|
||||
raise last
|
||||
|
||||
def get_json(self, url, *, headers=None):
|
||||
return _json.loads(self.get_bytes(url, headers=headers).decode("utf-8"))
|
||||
@@ -0,0 +1,148 @@
|
||||
"""Ingest seam: a Candidate shape, the Fetcher protocol, and the pipeline.
|
||||
|
||||
Every external boundary — archive HTTP (download), ffprobe (prober), and the
|
||||
optional ffmpeg dominant-color (color_fn) — is injected, so the whole pipeline
|
||||
is exercised hermetically (no network, no binaries, no disk beyond tmp).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Protocol
|
||||
|
||||
from hef.catalog import (
|
||||
Record,
|
||||
append_record,
|
||||
load_catalog,
|
||||
validate,
|
||||
validate_catalog,
|
||||
)
|
||||
from tools.drafting import Signals
|
||||
from tools.http import HttpClient
|
||||
from tools.mediatools import compute_dominant_color
|
||||
from tools.probe import probe_file
|
||||
from tools.tagging import derive_tags
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
_VIDEO_MODES = {"video", "av"}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Candidate:
|
||||
source_archive: str # origin label, e.g. internet_archive | librivox | nasa
|
||||
source_url: str # human/landing URL recorded in the record
|
||||
media_url: str # direct download URL of the chosen file
|
||||
title: str
|
||||
license: str # normalized (tools.licensing) -> hef LICENSES vocab
|
||||
attribution: str # "" unless the license requires it
|
||||
suggested_id: str # stable id derived from archive + archive-identifier
|
||||
media_ext: str # file extension for the download target
|
||||
description: str = "" # free text used by drafting signals; lands in notes
|
||||
|
||||
|
||||
class Fetcher(Protocol):
|
||||
archive: str
|
||||
|
||||
def search(self, query: str, *, limit: int) -> list[Candidate]: ...
|
||||
|
||||
def resolve(self, identifier: str) -> Candidate: ...
|
||||
|
||||
|
||||
def _default_downloader(url, dest) -> None:
|
||||
client = HttpClient()
|
||||
dest.write_bytes(client.get_bytes(url))
|
||||
|
||||
|
||||
def ingest_candidate(
|
||||
candidate: Candidate,
|
||||
*,
|
||||
catalog_path,
|
||||
media_root,
|
||||
proposer,
|
||||
prober=probe_file,
|
||||
downloader=_default_downloader,
|
||||
compute_color: bool = False,
|
||||
color_fn=compute_dominant_color,
|
||||
):
|
||||
"""Ingest one candidate into the catalog as a `proposed` record.
|
||||
|
||||
Returns the appended Record, or None if it was skipped as a duplicate.
|
||||
Raises CatalogError if the resulting record is invalid (e.g. an unmappable
|
||||
license) — nothing is appended in that case.
|
||||
"""
|
||||
catalog_path = Path(catalog_path)
|
||||
media_root = Path(media_root)
|
||||
|
||||
# 1. Dedupe.
|
||||
existing = load_catalog(catalog_path) if catalog_path.exists() else []
|
||||
validate_catalog(existing)
|
||||
if any(r.id == candidate.suggested_id for r in existing):
|
||||
log.info("already in catalog, skipping: %s", candidate.suggested_id)
|
||||
return None
|
||||
|
||||
# 2. Download (skip if already present).
|
||||
rel_path = f"{candidate.source_archive}/{candidate.suggested_id}.{candidate.media_ext}"
|
||||
dest = media_root / rel_path
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not dest.exists():
|
||||
downloader(candidate.media_url, dest)
|
||||
|
||||
# 3. Mechanical tagging.
|
||||
tags = derive_tags(prober(dest))
|
||||
dominant_color = ""
|
||||
if compute_color and tags.mode in _VIDEO_MODES:
|
||||
dominant_color = color_fn(dest, midpoint_s=tags.duration_s / 2)
|
||||
|
||||
# 4. Draft coordinates.
|
||||
draft = proposer.propose(
|
||||
Signals(
|
||||
title=candidate.title,
|
||||
description=candidate.description,
|
||||
source_archive=candidate.source_archive,
|
||||
mode=tags.mode,
|
||||
duration_s=tags.duration_s,
|
||||
)
|
||||
)
|
||||
coord = draft.coordinate
|
||||
|
||||
# 5. Build the record — proposed/reviewed_at=None falls out of the defaults.
|
||||
record = Record(
|
||||
id=candidate.suggested_id,
|
||||
title=candidate.title,
|
||||
source_url=candidate.source_url,
|
||||
source_archive=candidate.source_archive,
|
||||
license=candidate.license,
|
||||
mode=tags.mode,
|
||||
left=coord.left,
|
||||
right=coord.right,
|
||||
dark=coord.dark,
|
||||
light=coord.light,
|
||||
duration_s=tags.duration_s,
|
||||
file_path=rel_path,
|
||||
attribution=candidate.attribution,
|
||||
resolution=tags.resolution,
|
||||
dominant_color=dominant_color,
|
||||
rationale=draft.rationale,
|
||||
notes=candidate.description,
|
||||
)
|
||||
|
||||
# 6. Validate + re-check uniqueness + append.
|
||||
validate(record)
|
||||
if any(r.id == record.id for r in existing): # racey re-check
|
||||
log.info("already in catalog (race), skipping: %s", record.id)
|
||||
return None
|
||||
append_record(record, catalog_path)
|
||||
return record
|
||||
|
||||
|
||||
def ingest_search(fetcher: Fetcher, query: str, *, limit: int, **kw):
|
||||
"""Run ingest_candidate over each search hit. Returns the appended records."""
|
||||
appended = []
|
||||
for candidate in fetcher.search(query, limit=limit):
|
||||
rec = ingest_candidate(candidate, **kw)
|
||||
if rec is not None:
|
||||
appended.append(rec)
|
||||
return appended
|
||||
@@ -0,0 +1,26 @@
|
||||
"""Free Music Archive fetcher — DEFERRED (see spec §6.4).
|
||||
|
||||
CC-licensed music (cc_by / cc_by_nc / cc0). Highest API-stability risk: the
|
||||
public FMA API has been deprecated/changed, so a real implementation resolves
|
||||
from a track URL + page metadata. The Fetcher seam is wired; it raises until
|
||||
implemented.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from tools.ingest.base import Candidate
|
||||
|
||||
_DEFERRED = "deferred — see spec §6.4 (FMA public API deprecated/changed)"
|
||||
|
||||
|
||||
class FmaFetcher:
|
||||
archive = "fma"
|
||||
|
||||
def __init__(self, client):
|
||||
self.client = client
|
||||
|
||||
def search(self, query: str, *, limit: int) -> list[Candidate]:
|
||||
raise NotImplementedError(_DEFERRED)
|
||||
|
||||
def resolve(self, identifier: str) -> Candidate:
|
||||
raise NotImplementedError(_DEFERRED)
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Freesound fetcher — DEFERRED (see spec §6.4).
|
||||
|
||||
CC-licensed sound effects / field recordings (cc0 / cc_by / cc_by_nc) via API
|
||||
v2 (https://freesound.org/apiv2). Requires an API token supplied through the
|
||||
FREESOUND_API_TOKEN environment variable (or the macOS Keychain). That token is
|
||||
a SECRET: it is read from the environment only and must NEVER be written into a
|
||||
record, a log line, or a transcript (wgl hard secrets rule, spec §9).
|
||||
|
||||
The Fetcher seam is wired; it raises until implemented.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from tools.ingest.base import Candidate
|
||||
|
||||
_DEFERRED = "deferred — see spec §6.4 (Freesound requires FREESOUND_API_TOKEN)"
|
||||
TOKEN_ENV = "FREESOUND_API_TOKEN"
|
||||
|
||||
|
||||
class FreesoundFetcher:
|
||||
archive = "freesound"
|
||||
|
||||
def __init__(self, client, token: str | None = None):
|
||||
self.client = client
|
||||
# Read the secret from the environment only; never log or store it.
|
||||
self._token = token or os.environ.get(TOKEN_ENV)
|
||||
|
||||
def search(self, query: str, *, limit: int) -> list[Candidate]:
|
||||
raise NotImplementedError(_DEFERRED)
|
||||
|
||||
def resolve(self, identifier: str) -> Candidate:
|
||||
raise NotImplementedError(_DEFERRED)
|
||||
@@ -0,0 +1,91 @@
|
||||
"""Internet Archive fetcher (incl. Prelinger).
|
||||
|
||||
Metadata API: https://archive.org/metadata/<id> (keyless); search via
|
||||
advancedsearch.php. License is read from `licenseurl`/`rights`/`license` and
|
||||
normalized through tools.licensing. When no explicit license is present the
|
||||
item is assumed public_domain and flagged in notes for review (§6.4); an
|
||||
explicitly non-free license (e.g. All Rights Reserved) raises and is not
|
||||
ingested.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from tools.ingest.base import Candidate
|
||||
from tools.licensing import normalize_license
|
||||
|
||||
META = "https://archive.org/metadata/"
|
||||
SEARCH = "https://archive.org/advancedsearch.php"
|
||||
_NO_LICENSE_CAVEAT = "[IA: no explicit license — assumed public_domain, verify before use]"
|
||||
_MEDIA_EXTS = (
|
||||
".mp4", ".mov", ".m4v", ".ogv", ".webm",
|
||||
".mp3", ".m4a", ".wav", ".flac", ".ogg",
|
||||
)
|
||||
|
||||
|
||||
def _ext_from_name(name: str) -> str:
|
||||
return name.rsplit(".", 1)[-1].lower() if "." in name else ""
|
||||
|
||||
|
||||
def _pick_file(files):
|
||||
originals = [f for f in files if f.get("source") == "original"]
|
||||
for pool in (originals, files):
|
||||
for f in pool:
|
||||
if str(f.get("name", "")).lower().endswith(_MEDIA_EXTS):
|
||||
return f
|
||||
return None
|
||||
|
||||
|
||||
class InternetArchiveFetcher:
|
||||
archive = "internet_archive"
|
||||
|
||||
def __init__(self, client):
|
||||
self.client = client
|
||||
|
||||
def resolve(self, identifier: str) -> Candidate:
|
||||
data = self.client.get_json(f"{META}{identifier}")
|
||||
meta = data.get("metadata", {})
|
||||
files = data.get("files", []) or []
|
||||
server = data.get("server", "")
|
||||
directory = data.get("dir", "")
|
||||
|
||||
chosen = _pick_file(files)
|
||||
media_url = ""
|
||||
media_ext = ""
|
||||
if chosen and server:
|
||||
name = chosen["name"]
|
||||
media_url = f"https://{server}{directory}/{name}"
|
||||
media_ext = _ext_from_name(name)
|
||||
|
||||
raw_license = (
|
||||
meta.get("licenseurl") or meta.get("rights") or meta.get("license") or ""
|
||||
)
|
||||
creator = meta.get("creator", "") or ""
|
||||
if isinstance(creator, list):
|
||||
creator = ", ".join(str(c) for c in creator)
|
||||
description = (meta.get("description", "") or "").strip()
|
||||
|
||||
ambiguous = (not raw_license) or ("no known copyright" in raw_license.lower())
|
||||
if raw_license:
|
||||
lic, attr = normalize_license(raw_license, creator=creator)
|
||||
else:
|
||||
lic, attr = "public_domain", ""
|
||||
if ambiguous:
|
||||
description = f"{description} {_NO_LICENSE_CAVEAT}".strip()
|
||||
|
||||
return Candidate(
|
||||
source_archive=self.archive,
|
||||
source_url=f"https://archive.org/details/{identifier}",
|
||||
media_url=media_url,
|
||||
title=meta.get("title", identifier),
|
||||
license=lic,
|
||||
attribution=attr,
|
||||
suggested_id=f"ia-{identifier}",
|
||||
media_ext=media_ext,
|
||||
description=description,
|
||||
)
|
||||
|
||||
def search(self, query: str, *, limit: int) -> list[Candidate]:
|
||||
url = f"{SEARCH}?q={query}&fl[]=identifier&rows={limit}&output=json"
|
||||
data = self.client.get_json(url)
|
||||
docs = (data.get("response", {}).get("docs") or [])[:limit]
|
||||
return [self.resolve(doc["identifier"]) for doc in docs]
|
||||
@@ -0,0 +1,71 @@
|
||||
"""LibriVox fetcher — public-domain audiobook recordings.
|
||||
|
||||
JSON API: https://librivox.org/api/feed/audiobooks (keyless). LibriVox is
|
||||
public domain by charter; the reader/author is credited in notes but
|
||||
attribution is not required. Section-level (per-track) file resolution is a
|
||||
future refinement; this resolves the audiobook's zip as the media target.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from tools.ingest.base import Candidate
|
||||
from tools.licensing import librivox_license
|
||||
|
||||
BASE = "https://librivox.org/api/feed/audiobooks"
|
||||
|
||||
|
||||
def _slug(text: str) -> str:
|
||||
s = re.sub(r"[^a-z0-9]+", "-", (text or "").lower()).strip("-")
|
||||
return s or "untitled"
|
||||
|
||||
|
||||
def _ext_from_url(url: str, default: str) -> str:
|
||||
tail = url.rsplit("/", 1)[-1]
|
||||
return tail.rsplit(".", 1)[-1].lower() if "." in tail else default
|
||||
|
||||
|
||||
class LibriVoxFetcher:
|
||||
archive = "librivox"
|
||||
|
||||
def __init__(self, client):
|
||||
self.client = client
|
||||
|
||||
def _candidate(self, book) -> Candidate:
|
||||
title = book.get("title", "")
|
||||
authors = book.get("authors") or []
|
||||
credit = ", ".join(
|
||||
f"{a.get('first_name', '')} {a.get('last_name', '')}".strip()
|
||||
for a in authors
|
||||
).strip(", ")
|
||||
media_url = book.get("url_zip_file", "")
|
||||
description = (book.get("description", "") or "").strip()
|
||||
if credit:
|
||||
description = (description + f" (author: {credit})").strip()
|
||||
lic, attr = librivox_license()
|
||||
return Candidate(
|
||||
source_archive=self.archive,
|
||||
source_url=book.get("url_librivox", ""),
|
||||
media_url=media_url,
|
||||
title=title,
|
||||
license=lic,
|
||||
attribution=attr,
|
||||
suggested_id=f"librivox-{_slug(title)}",
|
||||
media_ext=_ext_from_url(media_url, "zip"),
|
||||
description=description,
|
||||
)
|
||||
|
||||
def search(self, query: str, *, limit: int) -> list[Candidate]:
|
||||
url = f"{BASE}/title/^{query}?format=json&limit={limit}"
|
||||
data = self.client.get_json(url)
|
||||
books = (data.get("books") or [])[:limit]
|
||||
return [self._candidate(b) for b in books]
|
||||
|
||||
def resolve(self, identifier: str) -> Candidate:
|
||||
url = f"{BASE}/id/{identifier}?format=json"
|
||||
data = self.client.get_json(url)
|
||||
books = data.get("books") or []
|
||||
if not books:
|
||||
raise ValueError(f"librivox: no audiobook for {identifier!r}")
|
||||
return self._candidate(books[0])
|
||||
@@ -0,0 +1,25 @@
|
||||
"""Musopen fetcher — DEFERRED (see spec §6.4).
|
||||
|
||||
Public-domain / CC classical music. Access has historically been gated and may
|
||||
require an API key (a secret — never logged, see spec §9). The Fetcher seam is
|
||||
wired so enabling it later is purely additive; it raises until implemented.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from tools.ingest.base import Candidate
|
||||
|
||||
_DEFERRED = "deferred — see spec §6.4 (Musopen access may require an API key)"
|
||||
|
||||
|
||||
class MusopenFetcher:
|
||||
archive = "musopen"
|
||||
|
||||
def __init__(self, client):
|
||||
self.client = client
|
||||
|
||||
def search(self, query: str, *, limit: int) -> list[Candidate]:
|
||||
raise NotImplementedError(_DEFERRED)
|
||||
|
||||
def resolve(self, identifier: str) -> Candidate:
|
||||
raise NotImplementedError(_DEFERRED)
|
||||
@@ -0,0 +1,79 @@
|
||||
"""NASA fetcher — public-domain imagery/video.
|
||||
|
||||
JSON API: https://images-api.nasa.gov/search (keyless). Each search item links
|
||||
an asset-collection JSON (the item's `href`) listing the concrete file URLs;
|
||||
the chosen media file is resolved from there. NASA media is public domain, but
|
||||
some items embed third-party content — flagged in notes for review (§6.4).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from tools.ingest.base import Candidate
|
||||
|
||||
SEARCH = "https://images-api.nasa.gov/search"
|
||||
_THIRD_PARTY_CAVEAT = "[NASA: may embed third-party content — verify before use]"
|
||||
_VIDEO_EXTS = (".mp4", ".mov", ".m4v", ".webm")
|
||||
_AUDIO_EXTS = (".mp3", ".m4a", ".wav", ".flac", ".ogg")
|
||||
|
||||
|
||||
def _ext_from_url(url: str) -> str:
|
||||
tail = url.rsplit("/", 1)[-1].split("?", 1)[0]
|
||||
return tail.rsplit(".", 1)[-1].lower() if "." in tail else ""
|
||||
|
||||
|
||||
def _pick_asset(assets, media_type: str) -> str:
|
||||
if not isinstance(assets, list):
|
||||
return ""
|
||||
prefs = _AUDIO_EXTS if media_type == "audio" else _VIDEO_EXTS
|
||||
for ext in prefs:
|
||||
for url in assets:
|
||||
if isinstance(url, str) and url.lower().endswith(ext):
|
||||
return url
|
||||
for url in assets:
|
||||
if isinstance(url, str) and not url.lower().endswith((".jpg", ".png", ".json")):
|
||||
return url
|
||||
return ""
|
||||
|
||||
|
||||
class NasaFetcher:
|
||||
archive = "nasa"
|
||||
|
||||
def __init__(self, client):
|
||||
self.client = client
|
||||
|
||||
def _candidate(self, item) -> Candidate:
|
||||
data0 = (item.get("data") or [{}])[0]
|
||||
nasa_id = data0.get("nasa_id", "")
|
||||
title = data0.get("title", "")
|
||||
media_type = data0.get("media_type", "")
|
||||
description = (data0.get("description", "") or "").strip()
|
||||
description = f"{description} {_THIRD_PARTY_CAVEAT}".strip()
|
||||
|
||||
href = item.get("href", "")
|
||||
assets = self.client.get_json(href) if href else []
|
||||
media_url = _pick_asset(assets, media_type)
|
||||
return Candidate(
|
||||
source_archive=self.archive,
|
||||
source_url=f"https://images.nasa.gov/details-{nasa_id}",
|
||||
media_url=media_url,
|
||||
title=title,
|
||||
license="public_domain",
|
||||
attribution="",
|
||||
suggested_id=f"nasa-{nasa_id}",
|
||||
media_ext=_ext_from_url(media_url),
|
||||
description=description,
|
||||
)
|
||||
|
||||
def search(self, query: str, *, limit: int) -> list[Candidate]:
|
||||
url = f"{SEARCH}?q={query}&media_type=video"
|
||||
data = self.client.get_json(url)
|
||||
items = (data.get("collection", {}).get("items") or [])[:limit]
|
||||
return [self._candidate(i) for i in items]
|
||||
|
||||
def resolve(self, identifier: str) -> Candidate:
|
||||
url = f"{SEARCH}?nasa_id={identifier}"
|
||||
data = self.client.get_json(url)
|
||||
items = data.get("collection", {}).get("items") or []
|
||||
if not items:
|
||||
raise ValueError(f"nasa: no item for {identifier!r}")
|
||||
return self._candidate(items[0])
|
||||
@@ -0,0 +1,99 @@
|
||||
"""CLI entry point for ingest: `python -m tools.ingest_cli <archive> ...`.
|
||||
|
||||
Wires a named fetcher + the HeuristicProposer into the ingest pipeline. Secrets
|
||||
(e.g. FREESOUND_API_TOKEN) are read from the environment only, never as flags.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
from tools.drafting import HeuristicProposer
|
||||
from tools.http import HttpClient
|
||||
from tools.ingest.base import ingest_candidate, ingest_search
|
||||
from tools.ingest.fma import FmaFetcher
|
||||
from tools.ingest.freesound import FreesoundFetcher
|
||||
from tools.ingest.internet_archive import InternetArchiveFetcher
|
||||
from tools.ingest.librivox import LibriVoxFetcher
|
||||
from tools.ingest.musopen import MusopenFetcher
|
||||
from tools.ingest.nasa import NasaFetcher
|
||||
|
||||
FETCHERS = {
|
||||
"librivox": LibriVoxFetcher,
|
||||
"nasa": NasaFetcher,
|
||||
"internet_archive": InternetArchiveFetcher,
|
||||
"musopen": MusopenFetcher,
|
||||
"fma": FmaFetcher,
|
||||
"freesound": FreesoundFetcher,
|
||||
}
|
||||
|
||||
DEFAULT_CATALOG = "catalog/library.jsonl"
|
||||
DEFAULT_MEDIA_ROOT = "./media"
|
||||
|
||||
|
||||
def _build_parser() -> argparse.ArgumentParser:
|
||||
p = argparse.ArgumentParser(
|
||||
prog="tools.ingest_cli",
|
||||
description="Ingest candidates from an archive into the catalog as proposed records.",
|
||||
)
|
||||
p.add_argument("archive", help=f"source archive ({', '.join(sorted(FETCHERS))})")
|
||||
p.add_argument("--query", help="search query")
|
||||
p.add_argument("--resolve", metavar="IDENTIFIER", help="ingest a single item by id/URL")
|
||||
p.add_argument("--limit", type=int, default=5, help="max search hits (default 5)")
|
||||
p.add_argument("--catalog", default=DEFAULT_CATALOG, help=f"catalog JSONL (default {DEFAULT_CATALOG})")
|
||||
p.add_argument(
|
||||
"--media-root",
|
||||
default=os.environ.get("HEF_MEDIA_ROOT", DEFAULT_MEDIA_ROOT),
|
||||
help="download root (env HEF_MEDIA_ROOT; default ./media)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--dominant-color",
|
||||
action="store_true",
|
||||
help="compute dominant_color for video/av (opt-in, ffmpeg-only)",
|
||||
)
|
||||
return p
|
||||
|
||||
|
||||
def main(argv=None) -> int:
|
||||
args = _build_parser().parse_args(argv)
|
||||
|
||||
cls = FETCHERS.get(args.archive)
|
||||
if cls is None:
|
||||
print(
|
||||
f"error: unknown archive {args.archive!r}; "
|
||||
f"choose one of {', '.join(sorted(FETCHERS))}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
if not args.query and not args.resolve:
|
||||
print("error: provide --query or --resolve", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
fetcher = cls(HttpClient())
|
||||
kw = dict(
|
||||
catalog_path=args.catalog,
|
||||
media_root=args.media_root,
|
||||
proposer=HeuristicProposer(),
|
||||
compute_color=args.dominant_color,
|
||||
)
|
||||
|
||||
try:
|
||||
if args.resolve:
|
||||
rec = ingest_candidate(fetcher.resolve(args.resolve), **kw)
|
||||
appended = [rec] if rec is not None else []
|
||||
else:
|
||||
appended = ingest_search(fetcher, args.query, limit=args.limit, **kw)
|
||||
except NotImplementedError as exc:
|
||||
print(f"error: {exc}", file=sys.stderr)
|
||||
return 3
|
||||
|
||||
print(f"ingested {len(appended)} proposed record(s) into {args.catalog}")
|
||||
for rec in appended:
|
||||
print(f" {rec.id} [{rec.mode}] {rec.title}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Normalize per-archive origin license metadata to the hef.catalog LICENSES vocab.
|
||||
|
||||
Maps Creative Commons URLs / identifiers and public-domain markers to
|
||||
{public_domain, cc0, cc_by, cc_by_nc}. For the attribution licenses (cc_by,
|
||||
cc_by_nc) it builds a non-empty attribution string — hef.catalog.validate()
|
||||
rejects those licenses with an empty attribution. Anything unmappable is
|
||||
rejected (a piece whose license can't be established is not ingested).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
|
||||
def _attribution(creator: str, license_name: str, raw: str, default_label: str) -> str:
|
||||
label = license_name or default_label or raw
|
||||
parts = []
|
||||
if creator:
|
||||
parts.append(creator)
|
||||
parts.append(label)
|
||||
return " — ".join(parts)
|
||||
|
||||
|
||||
def normalize_license(raw, *, creator="", license_name=""):
|
||||
"""Return (license, attribution) for an origin license string.
|
||||
|
||||
Raises ValueError when `raw` does not map to an allowed license.
|
||||
"""
|
||||
text = (raw or "").strip().lower()
|
||||
if not text:
|
||||
raise ValueError(f"unmappable license: {raw!r}")
|
||||
|
||||
# CC0 / public-domain dedication (check before the generic publicdomain
|
||||
# markers, since the CC0 URL also contains "publicdomain").
|
||||
if "publicdomain/zero" in text or re.search(r"\bcc[ _-]?0\b", text):
|
||||
return ("cc0", "")
|
||||
|
||||
# Public Domain Mark / explicit public domain / no-known-copyright.
|
||||
pd_markers = (
|
||||
"publicdomain/mark",
|
||||
"public_domain",
|
||||
"public domain",
|
||||
"publicdomain",
|
||||
"no known copyright",
|
||||
"pdm",
|
||||
)
|
||||
if any(m in text for m in pd_markers):
|
||||
return ("public_domain", "")
|
||||
|
||||
# CC BY-NC must be checked before the bare CC BY.
|
||||
if re.search(r"by[ _-]?nc", text):
|
||||
return ("cc_by_nc", _attribution(creator, license_name, raw, "CC BY-NC"))
|
||||
|
||||
# CC BY.
|
||||
if re.search(r"\bcc[ _-]?by\b", text) or "licenses/by" in text:
|
||||
return ("cc_by", _attribution(creator, license_name, raw, "CC BY"))
|
||||
|
||||
raise ValueError(f"unmappable license: {raw!r}")
|
||||
|
||||
|
||||
def librivox_license():
|
||||
"""LibriVox recordings are public domain by charter (attribution not required)."""
|
||||
return ("public_domain", "")
|
||||
@@ -0,0 +1,58 @@
|
||||
"""ffmpeg helpers: representative frame; OPTIONAL dominant color. Runner injectable."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
|
||||
def _run_bytes(args) -> bytes:
|
||||
return subprocess.run(args, capture_output=True, check=True).stdout
|
||||
|
||||
|
||||
def dominant_color_from_rgb(rgb: bytes) -> str:
|
||||
r, g, b = rgb[0], rgb[1], rgb[2]
|
||||
return f"#{r:02x}{g:02x}{b:02x}"
|
||||
|
||||
|
||||
def compute_dominant_color(path, *, midpoint_s=0.0, runner=_run_bytes) -> str:
|
||||
"""ffmpeg-only single-color palette of a mid-segment frame -> #rrggbb."""
|
||||
args = [
|
||||
"ffmpeg",
|
||||
"-v",
|
||||
"quiet",
|
||||
"-ss",
|
||||
str(midpoint_s),
|
||||
"-i",
|
||||
str(path),
|
||||
"-vf",
|
||||
"thumbnail,palettegen=max_colors=1",
|
||||
"-frames:v",
|
||||
"1",
|
||||
"-f",
|
||||
"rawvideo",
|
||||
"-pix_fmt",
|
||||
"rgb24",
|
||||
"-",
|
||||
]
|
||||
return dominant_color_from_rgb(runner(args))
|
||||
|
||||
|
||||
def extract_frame(path, dest, *, midpoint_s=0.0, runner=None):
|
||||
"""Write one representative frame to dest (PNG) for the review preview."""
|
||||
runner = runner or (lambda a: subprocess.run(a, check=True))
|
||||
runner(
|
||||
[
|
||||
"ffmpeg",
|
||||
"-v",
|
||||
"quiet",
|
||||
"-y",
|
||||
"-ss",
|
||||
str(midpoint_s),
|
||||
"-i",
|
||||
str(path),
|
||||
"-frames:v",
|
||||
"1",
|
||||
str(dest),
|
||||
]
|
||||
)
|
||||
return dest
|
||||
@@ -0,0 +1,34 @@
|
||||
"""ffprobe wrapper -> parsed streams/format. Subprocess runner is injectable."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class Probe:
|
||||
streams: list
|
||||
format: dict
|
||||
|
||||
|
||||
def _default_runner(args) -> str:
|
||||
return subprocess.run(args, capture_output=True, text=True, check=True).stdout
|
||||
|
||||
|
||||
def probe_file(path, *, runner=_default_runner) -> Probe:
|
||||
out = runner(
|
||||
[
|
||||
"ffprobe",
|
||||
"-v",
|
||||
"quiet",
|
||||
"-print_format",
|
||||
"json",
|
||||
"-show_format",
|
||||
"-show_streams",
|
||||
str(path),
|
||||
]
|
||||
)
|
||||
data = json.loads(out)
|
||||
return Probe(streams=data.get("streams", []), format=data.get("format", {}))
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Review state-transition core (pure, tested): proposed -> approved.
|
||||
|
||||
No I/O, no clock — the timestamp is injected so the logic stays deterministic.
|
||||
The CLI persists the result via hef.catalog.save_catalog after re-validating.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
|
||||
|
||||
def proposed_records(records):
|
||||
"""Records still awaiting review."""
|
||||
return [r for r in records if r.review_status == "proposed"]
|
||||
|
||||
|
||||
def approve(record, *, reviewed_at, coordinate=None, rationale=None):
|
||||
"""Return an approved copy of `record` (the input is not mutated).
|
||||
|
||||
review_status -> 'approved' and reviewed_at is stamped. Optionally override
|
||||
the four coordinates with a human correction and/or replace the rationale.
|
||||
The caller re-validates before saving.
|
||||
"""
|
||||
changes = {"review_status": "approved", "reviewed_at": reviewed_at}
|
||||
if coordinate is not None:
|
||||
changes.update(
|
||||
left=coordinate.left,
|
||||
right=coordinate.right,
|
||||
dark=coordinate.dark,
|
||||
light=coordinate.light,
|
||||
)
|
||||
if rationale is not None:
|
||||
changes["rationale"] = rationale
|
||||
return replace(record, **changes)
|
||||
@@ -0,0 +1,154 @@
|
||||
"""Interactive review walk: `python -m tools.review_cli`.
|
||||
|
||||
Thin shell over the tested review core (tools.review). Walks each proposed
|
||||
record showing its mechanical fields, the proposed coordinates + rationale, and
|
||||
a best-effort preview frame, then prompts accept/edit/skip/quit. Approvals are
|
||||
persisted (full rewrite) after each one, so an interrupted session keeps its
|
||||
progress.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from hef.catalog import load_catalog, save_catalog, validate_catalog
|
||||
from tools.mediatools import extract_frame
|
||||
from tools.review import approve, proposed_records
|
||||
|
||||
DEFAULT_CATALOG = "catalog/library.jsonl"
|
||||
DEFAULT_MEDIA_ROOT = "./media"
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def _build_parser() -> argparse.ArgumentParser:
|
||||
p = argparse.ArgumentParser(
|
||||
prog="tools.review_cli",
|
||||
description="Walk proposed records and approve/correct their coordinates.",
|
||||
)
|
||||
p.add_argument("--catalog", default=DEFAULT_CATALOG)
|
||||
p.add_argument(
|
||||
"--media-root",
|
||||
default=os.environ.get("HEF_MEDIA_ROOT", DEFAULT_MEDIA_ROOT),
|
||||
)
|
||||
p.add_argument("--no-preview", action="store_true", help="skip preview rendering")
|
||||
return p
|
||||
|
||||
|
||||
def _show(rec, out) -> None:
|
||||
print(f"\n{'=' * 60}", file=out)
|
||||
print(f" id {rec.id}", file=out)
|
||||
print(f" title {rec.title}", file=out)
|
||||
print(f" source {rec.source_archive} {rec.source_url}", file=out)
|
||||
attr = f" ({rec.attribution})" if rec.attribution else ""
|
||||
print(f" license {rec.license}{attr}", file=out)
|
||||
print(f" media mode={rec.mode} {rec.duration_s}s {rec.resolution}", file=out)
|
||||
if rec.dominant_color:
|
||||
print(f" color {rec.dominant_color}", file=out)
|
||||
print(
|
||||
f" proposed left={rec.left} right={rec.right} dark={rec.dark} light={rec.light}",
|
||||
file=out,
|
||||
)
|
||||
print(f" rationale {rec.rationale}", file=out)
|
||||
if rec.notes:
|
||||
print(f" notes {rec.notes}", file=out)
|
||||
|
||||
|
||||
def _open_cmd():
|
||||
return "open" if platform.system() == "Darwin" else "xdg-open"
|
||||
|
||||
|
||||
def _preview(rec, media_root, out) -> None:
|
||||
"""Best-effort preview; never fatal. Requires ffmpeg for video frames."""
|
||||
if not shutil.which("ffmpeg"):
|
||||
return
|
||||
src = Path(media_root) / rec.file_path
|
||||
if not src.exists():
|
||||
print(f" (preview: media not found at {src})", file=out)
|
||||
return
|
||||
try:
|
||||
midpoint = rec.duration_s / 2 if rec.duration_s else 0.0
|
||||
if rec.mode in ("video", "av"):
|
||||
dest = Path(tempfile.gettempdir()) / f"hef-preview-{rec.id}.png"
|
||||
extract_frame(src, dest, midpoint_s=midpoint)
|
||||
else: # audio -> waveform thumbnail
|
||||
dest = Path(tempfile.gettempdir()) / f"hef-preview-{rec.id}.png"
|
||||
subprocess.run(
|
||||
[
|
||||
"ffmpeg", "-v", "quiet", "-y", "-i", str(src),
|
||||
"-filter_complex", "showwavespic=s=640x240", "-frames:v", "1",
|
||||
str(dest),
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
opener = shutil.which(_open_cmd())
|
||||
if opener:
|
||||
subprocess.run([opener, str(dest)], check=False)
|
||||
except Exception as exc: # noqa: BLE001 - preview is best-effort
|
||||
print(f" (preview failed: {exc})", file=out)
|
||||
|
||||
|
||||
def _prompt_coords(rec, input_fn, out):
|
||||
from hef.selection import Coordinate
|
||||
|
||||
def _ask(axis, current):
|
||||
raw = input_fn(f" {axis} [{current}]: ").strip()
|
||||
if not raw:
|
||||
return current
|
||||
try:
|
||||
return max(0, min(4, int(raw)))
|
||||
except ValueError:
|
||||
print(f" (not an int, keeping {current})", file=out)
|
||||
return current
|
||||
|
||||
return Coordinate(
|
||||
_ask("left", rec.left),
|
||||
_ask("right", rec.right),
|
||||
_ask("dark", rec.dark),
|
||||
_ask("light", rec.light),
|
||||
)
|
||||
|
||||
|
||||
def main(argv=None, *, input_fn=input, now_fn=_now, out=sys.stdout) -> int:
|
||||
args = _build_parser().parse_args(argv)
|
||||
|
||||
records = load_catalog(args.catalog)
|
||||
validate_catalog(records)
|
||||
pending = proposed_records(records)
|
||||
if not pending:
|
||||
print("no proposed records to review", file=out)
|
||||
return 0
|
||||
|
||||
pos = {r.id: i for i, r in enumerate(records)}
|
||||
print(f"{len(pending)} proposed record(s) to review", file=out)
|
||||
|
||||
for rec in pending:
|
||||
_show(rec, out)
|
||||
if not args.no_preview:
|
||||
_preview(rec, args.media_root, out)
|
||||
choice = input_fn("[a]ccept / [e]dit / [s]kip / [q]uit > ").strip().lower()
|
||||
if choice == "q":
|
||||
break
|
||||
if choice == "s" or choice not in ("a", "e"):
|
||||
continue
|
||||
coordinate = _prompt_coords(rec, input_fn, out) if choice == "e" else None
|
||||
approved = approve(records[pos[rec.id]], reviewed_at=now_fn(), coordinate=coordinate)
|
||||
records[pos[rec.id]] = approved
|
||||
save_catalog(records, args.catalog)
|
||||
print(f" approved {rec.id}", file=out)
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Mechanical tagging: derive mode/duration_s/resolution from a Probe.
|
||||
|
||||
Honors the cover-art guard (spec §5.1): an embedded album-art image in an audio
|
||||
file shows up as a video stream with disposition.attached_pic == 1; it must NOT
|
||||
make the file 'av'.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
|
||||
from tools.probe import Probe
|
||||
|
||||
|
||||
@dataclass
|
||||
class Tags:
|
||||
mode: str
|
||||
duration_s: int
|
||||
resolution: str
|
||||
|
||||
|
||||
def _is_real_video(stream) -> bool:
|
||||
if stream.get("codec_type") != "video":
|
||||
return False
|
||||
return stream.get("disposition", {}).get("attached_pic", 0) != 1
|
||||
|
||||
|
||||
def _parse_duration(value) -> float:
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return 0.0
|
||||
|
||||
|
||||
def derive_tags(probe: Probe) -> Tags:
|
||||
real_video = [s for s in probe.streams if _is_real_video(s)]
|
||||
has_audio = any(s.get("codec_type") == "audio" for s in probe.streams)
|
||||
|
||||
if real_video and has_audio:
|
||||
mode = "av"
|
||||
elif real_video:
|
||||
mode = "video"
|
||||
else:
|
||||
mode = "audio"
|
||||
|
||||
resolution = ""
|
||||
if real_video:
|
||||
v = real_video[0]
|
||||
w, h = v.get("width"), v.get("height")
|
||||
if w and h:
|
||||
resolution = f"{w}x{h}"
|
||||
|
||||
if "duration" in probe.format:
|
||||
seconds = _parse_duration(probe.format.get("duration"))
|
||||
else:
|
||||
seconds = max(
|
||||
(_parse_duration(s.get("duration")) for s in probe.streams),
|
||||
default=0.0,
|
||||
)
|
||||
|
||||
# Round half up (math.floor(x + 0.5)) rather than Python's banker's
|
||||
# rounding, so a 12.5s clip becomes 13s as the spec intends.
|
||||
duration_s = max(0, math.floor(seconds + 0.5))
|
||||
return Tags(mode=mode, duration_s=duration_s, resolution=resolution)
|
||||
Reference in New Issue
Block a user