Compare commits

..

93 Commits

Author SHA1 Message Date
Ben Stull 9c347d580f feat(sim+engine): lock alteration calibration + fix psychedelic dark grade
Tuning the look by eye in the simulator (design §8) surfaced that the dark
mood pole rendered a full-frame hue-rotate(-200deg) — rock orange, trees
purple: the disorienting look rejected in session 0008, not the peaceful POC
dark_frame. Replace it with darken + slight desaturate on the video filter
plus a multiply-blended deep-blue wash (#tint, below the SVG HUD so the
overlay stays legible). Dark now reads cool/somber with natural greens, like
the approved POC look; light/left/right unchanged and confirmed peaceful.

With full tilt now tasteful on every axis, LOCK DEFAULT_CALIBRATION to unity
gains + linear variant map as a deliberate by-eye choice (not placeholders),
closing the open session-0006 knob->strength decision: knobs run 0=off..4=max,
equal Dark/Light = identity, the 5 notches map 1:1 to the 5 discrete Right
bakes. Guard the locked constants with test_default_calibration_is_locked.

Resolves design §8 open questions (calibration curve shape; grade-vs-overlay
ordering) and records the dark-grade fix. Full suite 193 passed / 2 skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 23:19:53 -07:00
Ben Stull 7d2a3064ad claim human-experience-filter-art session 0010 (placeholder) + sessions.json entry 2026-06-07 23:09:52 -07:00
Ben Stull e753a68147 add sessions/0009/SESSION-0009.0-TRANSCRIPT-2026-06-07T22-41--2026-06-07T23-03.md + replace placeholder/variant SESSION-0009.0-TRANSCRIPT-2026-06-07T22-41--INPROGRESS.md 2026-06-07 23:04:40 -07:00
benstull 554eb5076f Merge pull request 'feat: reconciled simulator-first alteration slice (sessions 0007+0008)' (#7) from feature/reconciled-simulator-alteration-slice into main 2026-06-08 06:00:31 +00:00
Ben Stull b8543906be docs: point parent design at reconciliation; roadmap + user guide for sim alteration
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:58:46 -07:00
Ben Stull d2d63c0184 feat(simulator): alteration preview UI (grade + variant crossfade + Left overlay)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:57:13 -07:00
Ben Stull c3f9262a73 feat(simulator): sample-media manifest + POC setup/placeholder generator
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:56:15 -07:00
Ben Stull eb3aa0949d feat(simulator): /api/alteration + /api/clips; retire selection endpoints
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:55:10 -07:00
Ben Stull c316309fc6 feat(simulator): clips.py manifest model; retire selection fixtures
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:54:09 -07:00
Ben Stull 825d68c653 refactor(player): state.py tests track discrete Restyle.variant
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:53:40 -07:00
Ben Stull 12a8177793 feat(player): discrete Right variant + Left level + Calibration (slice design §2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:53:19 -07:00
Ben Stull e5ca07e2a4 docs(plan): implementation plan for reconciled simulator-alteration slice
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:52:01 -07:00
Ben Stull eda259e5b8 docs(spec): reconcile 0007/0008 sim-alteration designs (session 0009)
Bring the unmerged session-0007 simulator-alteration-preview design onto a
single thread and add the reconciliation design that resolves the load-bearing
Left-HUD conflict between it and the merged session-0008 scales-library /
right-axis design.

Decision (operator-approved): Left is a RUNTIME overlay (authored annotation
track + per-language string tables, shaped live — browser in the sim, Pango/
HarfBuzz on the Pi), not baked pixels. Consequently the pre-baked variant set is
1-D over Right strength, not 0007's 2-D 5x5 Left x Right grid; 0007's baked-HUD
position (its sec 4/5/8) is superseded. Engine change is surgical — the merged
slice-1 engine already keeps a runtime AnalyticalOverlay.

Scopes the first simulator-runnable slice: deterministic Dark/Light/Left +
discrete pre-baked flow-stabilized Right variants over ONE neutral clip, wired
into the simulator, with a parameterized Calibration tuned by eye.

Spec: docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:47:56 -07:00
Ben Stull 0a450bfd91 claim human-experience-filter-art session 0009 (placeholder) + sessions.json entry 2026-06-07 22:41:44 -07:00
Ben Stull c99a669194 add sessions/0008/SESSION-0008.0-TRANSCRIPT-2026-06-07T19-42--2026-06-07T22-37.md + replace placeholder/variant SESSION-0008.0-TRANSCRIPT-2026-06-07T19-42--INPROGRESS.md 2026-06-07 22:37:56 -07:00
benstull 42a72fe516 Merge pull request 'docs(spec): scales-of-nature library + stabilized right-axis pipeline (session 0008)' (#6) from feature/scales-library-right-axis into main 2026-06-08 05:34:48 +00:00
Ben Stull 0238260908 docs(spec): scales-of-nature library + stabilized right-axis pipeline (session 0008)
Design revision refining the machine-altered-perception design (2026-06-05):

- Right axis is a LOCAL offline restyle (SD img2img on MPS), not a cloud API
  (refines §4.1/§4.3, §9). Per-frame restyle boils/flickers — disqualifying for
  a peaceful piece — so temporal coherence is now a hard constraint, met by
  optical-flow keyframe propagation (the EbSynth principle, OpenCV impl).
- Content = a small NEUTRAL "scales of nature" library (~4-6 clips, one per
  scale), not a single stitched cosmic-zoom film; keeps the neutral-base thesis
  and interactivity while retaining the awe-of-scale richness (refines §6/§8).
- NEW: the scales form a navigable closed RING joined by short AI zoom/warp
  transitions, driven by an infinitely-turnable endless rotary encoder; diving
  past the microscopic wraps around to the cosmos (refines §2 selector / §11).
- i18n sharpened (§1.2, sharpens §10): Left labels are a Pi-rendered runtime
  graphics overlay (annotation-track + per-language string tables, shaped via
  Pango/HarfBuzz + Noto) — NOT a baked per-language overlay video, which would
  break i18n economics. Added Pi-resolution headroom to open questions.
- Strict-PD sourcing map (NASA/NOAA/NPS/USGS); non-US terrestrial is the CC-BY
  soft spot. Economics: local authoring is ~free, not $300-3k of cloud API.

Grounded in a local POC this session (M4 Pro, sd-turbo, OpenCV flow): all four
axes validated on real footage; deterministic axes ~2.4s, Right flow-propagated
~2.7min/8s clip and calm.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:31:12 -07:00
Ben Stull c653189397 claim human-experience-filter-art session 0008 (placeholder) + sessions.json entry 2026-06-07 19:43:22 -07:00
Ben Stull 92ed046602 add sessions/0007/SESSION-0007.0-TRANSCRIPT-2026-06-06T14-12--2026-06-06T17-57.md + replace placeholder/variant SESSION-0007.0-TRANSCRIPT-2026-06-06T14-12--INPROGRESS.md 2026-06-06 17:58:05 -07:00
Ben Stull d58a23e5a6 claim human-experience-filter-art session 0007 (placeholder) + sessions.json entry 2026-06-06 14:13:36 -07:00
Ben Stull 5290785e2a add sessions/0006/SESSION-0006.0-TRANSCRIPT-2026-06-05T17-52--2026-06-05T18-15.md + replace placeholder/variant SESSION-0006.0-TRANSCRIPT-2026-06-05T17-52--INPROGRESS.md 2026-06-05 18:16:33 -07:00
benstull aeeed66f49 Merge pull request 'feat(player): sub-project 3 slice 1 — alteration engine + player core' (#5) from feature/player-alteration-core into main 2026-06-06 01:11:21 +00:00
Ben Stull 5403021e4a docs(roadmap): sub-project 3 alteration-engine + player-core slice shipped 2026-06-05 18:09:54 -07:00
Ben Stull 86b52ab8af build(player): register the player package 2026-06-05 18:08:55 -07:00
Ben Stull 9a13b01a41 feat(player): player state machine — controls stream to transitions 2026-06-05 18:08:34 -07:00
Ben Stull 79bb86c6a2 feat(player): alteration engine — knob vector to RenderPlan (design §4/§5) 2026-06-05 18:07:29 -07:00
Ben Stull 00969e2e44 feat(player): 7-way content-dial resolution (design §6) 2026-06-05 18:06:49 -07:00
Ben Stull 57597be8af feat(player): Controls panel model (serial-contract data shape)
Sub-project 3 slice 1, per ROADMAP.md §3 and design §6/§7.
2026-06-05 18:06:27 -07:00
Ben Stull 70834ae0ad docs(plan): sub-project 3 player alteration core (slice 1)
Pure-logic core per design §4/§5/§6/§7 and ROADMAP.md §3.
2026-06-05 18:05:34 -07:00
Ben Stull e69448922c claim human-experience-filter-art session 0006 (placeholder) + sessions.json entry 2026-06-05 17:55:09 -07:00
Ben Stull d8bb9661d4 add sessions/0005/SESSION-0005.0-TRANSCRIPT-2026-06-05T08-30--2026-06-05T17-48.md + replace placeholder/variant SESSION-0005.0-TRANSCRIPT-2026-06-05T08-30--INPROGRESS.md 2026-06-05 17:51:46 -07:00
benstull 77746e43c6 Merge pull request 'docs(design): machine-altered-perception thesis revision (session 0005)' (#4) from feature/machine-altered-perception-spec into main 2026-06-06 00:49:42 +00:00
Ben Stull c5952c6cfd docs(design): machine-altered-perception thesis revision
Revise the Human Experience Filter design: the piece becomes about how
humans interact with machines and how that reshapes the nervous system.
Neutral public-domain nature footage in, machine-altered perception out —
soothing or disturbing, good and bad.

Supersedes the 2026-06-04 design's thesis (§1), selection model (§3 — now
"transform from neutral", not catalog lookup), tagging division (§5), and
sourcing (§8); expands the control panel and hardware (§2/§6). Coordinate
axes and the single-pano projector are preserved.

Captures: the per-axis alteration mapping (Left=analytical overlay,
Right=generative v2v dissolve, Dark/Light=color grade), the substrate-vs-
overlay composition rule, the Light=yellow/white → neutral grey →
Dark=blue/black mood ramp, the 6-way content dial + volume/brightness, the
tactile/braille/LED/read-aloud wooden panel, and the runtime-overlay
architecture that makes multilingual labels nearly free.

Per docs/ROADMAP.md (reshapes sub-projects 2/3/4 + adds an offline v2v
authoring pipeline). Discovery session 0005.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:30:27 -07:00
Ben Stull 850f67abea claim human-experience-filter-art session 0005 (placeholder) + sessions.json entry 2026-06-05 08:32:48 -07:00
Ben Stull 3510236a2a add sessions/0004/SESSION-0004.0-TRANSCRIPT-2026-06-05T03-32--2026-06-05T03-56.md + replace placeholder/variant SESSION-0004.0-TRANSCRIPT-2026-06-05T03-32--INPROGRESS.md 2026-06-05 03:58:06 -07:00
Ben Stull 05c1f524da update sessions/0004/SESSION-0004.0-TRANSCRIPT-2026-06-05T03-32--INPROGRESS.md 2026-06-05 03:57:32 -07:00
Ben Stull 40dfbfd3db chore: gitignore *.egg-info/ build artifact
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:55:36 -07:00
benstull 618b69182c Merge pull request 'feat(simulator): curators X-ray experience simulator (sub-project 3)' (#3) from experience-simulator into main 2026-06-05 10:52:58 +00:00
Ben Stull 859f868d3b docs(simulator): user-guide section for the curator's X-ray
Per docs/superpowers/plans/2026-06-04-experience-simulator.md Task 7.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:51:06 -07:00
Ben Stull 80cde37b98 build(simulator): Dockerfile, compose, and make targets
Per docs/superpowers/plans/2026-06-04-experience-simulator.md Task 6.
Dockerfile also copies tools/ (a declared setuptools package) so the
editable install resolves inside the image; the plan's Dockerfile omitted it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:50:34 -07:00
Ben Stull c1745f0021 feat(simulator): X-ray frontend (dials, pool, brain/mood maps)
Per docs/superpowers/plans/2026-06-04-experience-simulator.md Task 5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:43:34 -07:00
Ben Stull cdf36c9b57 feat(simulator): FastAPI app with /api/select and /api/catalog/meta
Per docs/superpowers/plans/2026-06-04-experience-simulator.md Task 4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:42:13 -07:00
Ben Stull aadfccf32d feat(simulator): deterministic synthetic fixture catalog
Per docs/superpowers/plans/2026-06-04-experience-simulator.md Task 3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:41:35 -07:00
Ben Stull a260b68d2b build(simulator): add simulator package and sim optional-deps
Per docs/superpowers/plans/2026-06-04-experience-simulator.md Task 2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:41:05 -07:00
Ben Stull 98fb86840c feat(selection): expose ranked_candidates(); refactor select() onto it
Per docs/superpowers/plans/2026-06-04-experience-simulator.md Task 1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:40:38 -07:00
Ben Stull 7f2f585600 add sessions/0003/SESSION-0003.0-TRANSCRIPT-2026-06-04T07-57--2026-06-05T03-36.md + replace placeholder/variant SESSION-0003.0-TRANSCRIPT-2026-06-04T07-57--INPROGRESS.md 2026-06-05 03:37:25 -07:00
Ben Stull ddec45d39c docs(simulator): implementation plan for curator's X-ray simulator
Bite-sized TDD plan, 7 tasks: hef.selection.ranked_candidates() + select()
refactor; simulator package + deps; synthetic fixture catalog; FastAPI app
(/api/select, /api/catalog/meta); X-ray frontend; Docker/compose/make; user
guide + full-suite verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:36:25 -07:00
Ben Stull 9cb4fb62d8 docs(simulator): design spec + BDDs for curator's X-ray experience simulator
Discovery session 0003. Web-based, Docker-on-localhost simulator to feel the
selection model: real hef.selection wired to a synthetic fixture catalog, with
a full-transparency X-ray (ranked pool + distances + brain/mood grid maps) and
live model knobs (weights, pool size, approved-only).

- docs/superpowers/specs/2026-06-04-experience-simulator-design.md
- features/{selection_xray,model_knobs,fixture_catalog,simulator_service}.feature
- one additive hef.selection.ranked_candidates() change planned (single source of truth)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:36:25 -07:00
Ben Stull 0e5c3a1a0f claim human-experience-filter-art session 0004 (placeholder) + sessions.json entry 2026-06-05 03:36:01 -07:00
Ben Stull ee58c4cd78 claim human-experience-filter-art session 0003 (placeholder) + sessions.json entry 2026-06-04 07:57:27 -07:00
Ben Stull b7f19e01de add sessions/0002/SESSION-0002.0-TRANSCRIPT-2026-06-04T06-19--2026-06-04T07-34.md + replace placeholder/variant SESSION-0002.0-TRANSCRIPT-2026-06-04T06-19--INPROGRESS.md 2026-06-04 07:35:13 -07:00
benstull 593150b466 Merge pull request 'docs(roadmap): mark sub-project 2 done, sub-project 3 next' (#2) from roadmap-mark-subproject-2-done into main 2026-06-04 14:33:45 +00:00
Ben Stull 50ce6c9dfa docs(roadmap): mark sub-project 2 done, sub-project 3 next
Sub-project 2 (ingest & tagging / review tools) shipped via PR #1 this session.
Updates the status table and §2 detail, reconciles resolved decisions (first-ship
archives, media-root layout, ffmpeg-only opt-in dominant_color), and marks §3
(Pi player) as next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 07:33:32 -07:00
benstull d7a2cbaa40 Merge pull request 'Sub-project 2: ingest & tagging / review tools' (#1) from sub-project-2-ingest-tagging-review into main 2026-06-04 14:16:28 +00:00
Ben Stull 1e2a4dd966 docs: user guide for ingest + review tools
Per sub-project-2 plan Task 14. Updates the scope banner (catalog core + tools
now built), adds an 'Ingesting & reviewing media' section (ffmpeg prereqs,
--media-root, first-ship archives, ingest_cli/review_cli usage, Freesound token
note, opt-in dominant_color), and reconciles the absolute-vs-relative file_path
note (spec §6.3).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:36:15 -07:00
Ben Stull deb26b2575 test: end-to-end ingest+review integration (+ opt-in ffprobe)
Per sub-project-2 plan Task 13 / spec §10 test 8. Hermetic e2e: fake fetcher ->
ingest_candidate -> proposed record; approve flips it; reload + validate_catalog;
select(approved_only=True) finds it. Opt-in real-ffprobe/ffmpeg tests generate a
lavfi clip and assert mode/duration/resolution + dominant color, skipped when the
binaries are absent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:35:17 -07:00
Ben Stull 376edd7819 feat: interactive review CLI
Per sub-project-2 plan Task 12 / spec §8.2. Walks proposed records (fields +
coords + rationale + best-effort ffmpeg preview), prompts accept/edit/skip/quit,
and persists each approval via save_catalog rewrite. I/O seams (input_fn/now_fn/
out) and --no-preview make the walk testable hermetically; decision logic lives
in the tested review core.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:34:34 -07:00
Ben Stull 3e07e86187 feat: review transition core (proposed -> approved)
Per sub-project-2 plan Task 11 / spec §8.1. proposed_records filter and approve()
return an approved copy via dataclasses.replace (no in-place mutation), with
optional coordinate/rationale override and an injected reviewed_at timestamp.
Pure, no I/O, no clock.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:33:27 -07:00
Ben Stull 1d7f821ab8 feat: ingest CLI entry point
Per sub-project-2 plan Task 10. argparse entry wiring named fetcher +
HeuristicProposer into ingest_search/ingest_candidate; --query/--resolve,
--limit, --catalog, --media-root (env HEF_MEDIA_ROOT), --dominant-color.
Unknown archive -> exit 2, deferred archive -> exit 3. Secrets env-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:33:01 -07:00
Ben Stull cecc5a0f61 feat: LibriVox/NASA/Internet Archive fetchers (+ deferred stubs)
Per sub-project-2 plan Task 9 / spec §6.4. Three keyless first-ship fetchers
parse documented JSON APIs via an injected HttpClient; license/attribution go
through tools.licensing; NASA third-party + IA no-license cases are flagged in
notes for review. musopen/fma/freesound are explicit deferred stubs raising
NotImplementedError; freesound documents FREESOUND_API_TOKEN (secret, env-only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:32:18 -07:00
Ben Stull 2db2b7ac16 feat: ingest pipeline (Candidate/Fetcher/ingest_candidate)
Per sub-project-2 plan Task 8 / spec §6. ingest_candidate dedupes, downloads,
mechanically tags, drafts coordinates, builds a proposed Record and appends it;
ingest_search loops over fetcher hits. All boundaries (download/prober/color_fn)
injectable, so the pipeline is fully hermetic.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:29:09 -07:00
Ben Stull 3d5c34491c feat: heuristic coordinate proposer (drafting)
Per sub-project-2 plan Task 7 / spec §7. HeuristicProposer seeds the brain plane
from archive priors and the mood plane from title/description keyword nudges,
clamps to 0..4, and emits a one-line rationale. Deterministic, no I/O, no ML
(honors design §11) — only a DRAFT a human reviews.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:27:37 -07:00
Ben Stull 222421e773 feat: per-archive license normalization
Per sub-project-2 plan Task 6 / spec §5.3. normalize_license maps CC URLs /
identifiers and public-domain markers to the LICENSES vocab, builds non-empty
attribution for cc_by/cc_by_nc, and rejects unmappable licenses at ingest.
librivox_license() helper returns public_domain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:26:21 -07:00
Ben Stull d2a97823f5 feat: ffmpeg frame extraction + optional dominant_color
Per sub-project-2 plan Task 5 / spec §5.2. ffmpeg-only dominant color (no image
library), opt-in by design; extract_frame for review previews. Runners injectable
so the unit suite never shells out.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:25:11 -07:00
Ben Stull 542b5a5641 feat: mechanical mode/duration/resolution tagging
Per sub-project-2 plan Task 4 / spec §5.1. derive_tags(Probe) -> Tags with the
attached_pic cover-art guard so an art-bearing audio file stays mode=audio.
Duration falls back to the longest stream when format.duration is absent, and
rounds half-up so 12.5s -> 13s (Python's round() would give 12 via banker's
rounding) to match the spec's stated behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:24:33 -07:00
Ben Stull 19104f94cd feat: ffprobe wrapper (tools.probe)
Per sub-project-2 plan Task 3 / spec §5.1. Parses ffprobe JSON into a Probe
(streams + format); subprocess runner injectable so tests never shell out.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:23:24 -07:00
Ben Stull b6fb635ef4 feat: catalog-level validate_catalog + index_by_id (unique ids)
Per sub-project-2 plan Task 2 / spec §3. Purely additive to hef.catalog;
no existing symbol changes. validate_catalog runs per-record validate() then
asserts id uniqueness; index_by_id gives by-id lookup the player also wants.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:23:01 -07:00
Ben Stull db4f036729 chore: scaffold tools/ package + stdlib http client
Per sub-project-2 plan Task 1. Adds tools/ to setuptools packages,
gitignores media/, and a minimal injectable urllib HTTP client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:22:30 -07:00
Ben Stull 6dd6afde19 claim human-experience-filter-art session 0002 (placeholder) + sessions.json entry 2026-06-04 06:19:23 -07:00
Ben Stull 1b8faf9142 add sessions/0001/SESSION-0001.0-TRANSCRIPT-2026-06-04T05-36--2026-06-04T06-15.md + replace placeholder/variant SESSION-0001.0-TRANSCRIPT-2026-06-04T05-36--INPROGRESS.md 2026-06-04 06:16:43 -07:00
Ben Stull dd00d8bff5 Merge: pano design revision + sub-project 2 implementation plan
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:14:29 -07:00
Ben Stull 6d6cbb5cad plan: sub-project 2 — ingest & tagging / review tools
Task-by-task TDD plan implementing the approved sub-project-2 spec: tools/
scaffolding, the additive hef.catalog change (validate_catalog/index_by_id),
ffprobe/ffmpeg mechanical tagging with cover-art guard, per-archive license
normalization, heuristic coordinate drafting, the ingest pipeline + first-ship
fetchers (LibriVox/NASA/Internet Archive, others deferred stubs), the review
transition core + interactive CLI, and hermetic + opt-in integration tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:14:28 -07:00
Ben Stull dd02c2c36a docs: revise design + roadmap for single pano projector
Operator decision (2026-06-04): single panoramic projector spanning the
three walls showing the real selected video (nature-video focus), replacing
four projectors + a procedural side-wall renderer.

- Design spec: revision banner; updated §1, §6 (one pano projector), §7
  (procedural side walls REMOVED), §10 layout, §12 open questions.
- ROADMAP: sub-project 5 marked Dropped; diagram + dependency order updated;
  §3 player drives one pano output; cross-cutting decisions reconciled.
- Sub-project-2 spec: follow-up note updated to past tense (design spec done).

dominant_color's only consumer was the side walls, so it stays optional/opt-in
in the sub-project-2 tooling (already specced).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:12:09 -07:00
Ben Stull b8c406d517 update sessions/0001/SESSION-0001.0-TRANSCRIPT-2026-06-04T05-36--INPROGRESS.md 2026-06-04 06:07:44 -07:00
Ben Stull b5f5ee6663 docs: link the sub-project 2 spec from ROADMAP
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:07:32 -07:00
Ben Stull 3544092e5e Merge spec: sub-project 2 — ingest & tagging / review tools
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:06:50 -07:00
Ben Stull 4c1ec6e02e spec: fold in operator approval + pano-projector design change
Operator approved the spec and confirmed first-ship archives
(LibriVox + NASA + IA). Folds in the 2026-06-04 design change (single
panoramic projector across all three walls, nature-video content):
demotes dominant_color to optional/opt-in (its only consumer was the
procedural side walls), ffmpeg-only when computed, never an image lib.
Flags the design-spec/ROADMAP follow-up as out of scope here.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 06:06:45 -07:00
Ben Stull f80ec65553 spec: sub-project 2 — ingest & tagging / review tools
Develops the draft-then-review pipeline spec (design §5/§8, ROADMAP §2):
per-archive fetchers, mechanical tagging (ffprobe mode/duration/resolution,
ffmpeg dominant_color, origin license/attribution), heuristic coordinate
drafting (review_status=proposed + one-line rationale), and the review CLI
(walk proposed -> approve + reviewed_at).

Builds on hef.catalog additively: schema unchanged (Record defaults already
match the proposed shape); the only required core addition is a catalog-level
validate_catalog (unique-id integrity), with optional index_by_id.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 05:45:37 -07:00
Ben Stull 5e4b79486a chore: import wiggleverse org context in CLAUDE.md
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 05:38:44 -07:00
Ben Stull 54fde0b7a4 update sessions/0001/SESSION-0001.0-TRANSCRIPT-2026-06-04T05-36--INPROGRESS.md 2026-06-04 05:38:36 -07:00
Ben Stull 18f0360476 claim human-experience-filter-art session 0001 (placeholder) + sessions.json entry 2026-06-04 05:37:59 -07:00
Ben Stull 4ddc113375 chore: register app.json for session protocol
Registers the repo as a Wiggleverse app so its build sessions get tracked
transcripts. One Name = main repo name (human-experience-filter-art);
sessions live self-contained in this repo under sessions/, matching the
benstull-host convention (benstull/docs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 05:37:41 -07:00
Ben Stull 7f0e92d130 docs: roadmap for the five sub-projects (catalog core done, tools next)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 01:51:35 -07:00
Ben Stull 8c8291a94a docs: user guide for configuring media (hand-authoring + validating the catalog)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 01:48:14 -07:00
Ben Stull 062e13d42e test: end-to-end catalog+selection integration 2026-06-04 01:36:23 -07:00
Ben Stull 904c4bff52 feat: nearest-match select function 2026-06-04 01:36:07 -07:00
Ben Stull d28fb0a0a7 feat: selection mode filtering with av fallback 2026-06-04 01:35:38 -07:00
Ben Stull d1c5190bbb feat: selection Coordinate and weighted distance 2026-06-04 01:35:15 -07:00
Ben Stull 85119ed385 feat: catalog JSONL load/save/append 2026-06-04 01:34:51 -07:00
Ben Stull c36ef73342 feat: catalog Record dict conversion 2026-06-04 01:34:20 -07:00
Ben Stull 8b3d17a6aa feat: catalog Record model and validation 2026-06-04 01:33:50 -07:00
Ben Stull 1379751dda chore: scaffold catalog+selection core package 2026-06-04 01:33:23 -07:00
Ben Stull f15fe739ee docs: implementation plan for catalog+selection core (sub-project 1/5)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 01:31:10 -07:00
97 changed files with 12680 additions and 23 deletions
+9
View File
@@ -0,0 +1,9 @@
__pycache__/
*.pyc
.pytest_cache/
.venv/
media/
.superpowers/
*.egg-info/
# Simulator sample media (look-tuning only; populate via setup_sample_media.py)
simulator/sample_media/forest/*.mp4
+3
View File
@@ -0,0 +1,3 @@
# Human Experience Filter — art installation
@~/.claude/wiggleverse.md
+7
View File
@@ -0,0 +1,7 @@
.PHONY: sim sim-local
sim:
docker compose -f simulator/docker-compose.yml up --build
sim-local:
python -m uvicorn simulator.app:app --reload --port 8000
+20
View File
@@ -0,0 +1,20 @@
{
"schemaVersion": "1.0",
"name": "human-experience-filter-art",
"title": "Human Experience Filter — art installation",
"giteaHost": "ssh://git@git.benstull.org",
"repos": [
{
"namespace": "benstull",
"name": "human-experience-filter-art",
"contains": ["code", "specs", "documentation", "roadmap", "sessions"],
"sessions": {
"subdir": "sessions",
"layout": "folder-per-session",
"naming": "numeric",
"manifest": true,
"visibility": "private"
}
}
]
}
View File
View File
+221
View File
@@ -0,0 +1,221 @@
# Human Experience Filter — Roadmap
The installation is built as **five independent sub-projects**, each producing
working, testable software on its own. Each gets its own spec → plan → implement
cycle. This file is the index and dependency map; it is the source of truth for
"what's left."
Design reference: [`specs/2026-06-04-human-experience-filter-design.md`](./superpowers/specs/2026-06-04-human-experience-filter-design.md).
---
## Status at a glance
| # | Sub-project | Status | Unblocks |
|---|---------------------------------|---------------|----------|
| 1 | Catalog & Selection Core | ✅ Done | everything |
| 2 | Ingest & Tagging / Review tools | ✅ Done | a real library |
| 3 | Player Runtime (Pi) | ⏳ In progress | the room runs |
| 4 | Arduino Firmware (control panel)| ◻ Not started | real knobs |
| 5 | ~~Procedural Side Walls~~ | ❌ Dropped | — (superseded) |
**Dependency order:** 1 → 2, and 1 → 3. Sub-projects 3 and 4 share a serial
protocol contract and can then proceed in parallel. Sub-project 5 was **dropped**
(2026-06-04): the move to a single panoramic projector showing real video across
the three walls removes the procedural side walls — see the design spec §6/§7.
```
┌─────────────────────────┐
│ 1. Catalog & Selection │ ✅
└───────────┬─────────────┘
┌───────────┴───────────┐
▼ ▼
┌───────────────┐ ┌─────────────────────┐
│ 2. Ingest / │ │ 3. Player Runtime │
│ Review │─────▶│ (Pi) → 1 pano │
└───────────────┘ feeds│ projector │
(fills catalog) └──────────┬──────────┘
serial ▼
┌────────────────┐
│ 4. Firmware │
└────────────────┘
(5. Procedural Side Walls — dropped 2026-06-04, single pano projector)
```
---
## 1. Catalog & Selection Core ✅
**Done — merged to `main`, 37 tests.** The dependency root every other
sub-project imports.
- `hef/catalog.py``Record` model, validation, JSONL load/save/append.
- `hef/selection.py` — weighted-Euclidean distance, mode filtering with A+V
fallback, nearest-match `select()`.
Plan: [`2026-06-04-catalog-and-selection-core.md`](./superpowers/plans/2026-06-04-catalog-and-selection-core.md).
---
## 2. Ingest & Tagging / Review tools ✅
**Done — merged to `main` via PR #1 (session 0002, 2026-06-04); 111 tests pass
(2 opt-in real-ffprobe tests skip when the binaries are absent).** The `tools/`
package turns the manual catalog into the assisted *draft-then-review* flow from
the spec, and can populate the library from public-domain sources.
**Delivers (`tools/`):**
- **Ingest** — per-archive fetchers (Internet Archive / Prelinger, Musopen,
LibriVox, NASA, Free Music Archive, Freesound) that download a candidate and
write a catalog record.
- **Mechanical tagging** — auto-fill `mode` (probe streams with `ffprobe`),
`license` / `attribution` / `source_*` (from origin), `duration_s`,
`resolution`, and `dominant_color` (computed from the video).
- **Coordinate drafting** — propose `left/right/dark/light` with a one-line
`rationale`, written as `review_status: proposed`.
- **Review CLI** — walk unreviewed records one at a time (show proposed
coordinates + rationale + a preview frame), accept or correct, flip to
`approved` and stamp `reviewed_at`.
Spec: [`2026-06-04-ingest-tagging-review-tools.md`](./superpowers/specs/2026-06-04-ingest-tagging-review-tools.md).
**Depends on:** sub-project 1 (`hef.catalog`).
**Done when:** ✅ you can run ingest against a source, get `proposed` records, review
them to `approved`, and `load_catalog`/`validate_catalog` validate the result —
covered by tests on the mechanical-tagging and review state transitions.
**External deps:** `ffmpeg`/`ffprobe` only (no image lib — `dominant_color`, when
requested, is ffmpeg-only and opt-in), per-archive download access.
**Decisions settled:** first-ship archives = **LibriVox, NASA, Internet Archive**
(Musopen/FMA/Freesound deferred stubs); media under a `--media-root` with
`file_path` stored relative to it (spec §6.3); only additive `hef.catalog` change
was `validate_catalog` + `index_by_id`.
---
## 3. Player Runtime (Pi) ⏳ (in progress)
**Goal:** the thing that makes the room run — read the controls, **alter** the
neutral base footage toward the knob state, and play it across the single
panoramic projector. Per the machine-altered-perception revision the knobs no
longer *select* a pre-tagged clip; they drive an **alteration engine** (design
§4/§5) over a small neutral base library.
**Slice 1 — alteration engine + player core (pure logic) ✅ Done.** Merged to
`main` (session 0006); 56 tests. The new `player/` package, with all I/O
(video/audio/serial) behind injected interfaces:
- `player/controls.py``Controls`, the full panel state (7-way content dial +
4 experience knobs + volume + brightness); the serial-contract data shape
shared with sub-project 4.
- `player/content.py``resolve_content`, the §6 7-way dial → (audio source,
video on/off) table.
- `player/alteration.py``plan_alteration`, knob vector → `RenderPlan`
(mood `ColorGrade` with center=identity §5; Left `AnalyticalOverlay`; Right
`Restyle`), encoding the §4.2 layering rule (substrate vs overlay; Left & Right
stack).
- `player/state.py``Player`, the state machine that diffs successive controls
into transitions (`FADE_TO_BLACK`/`FADE_FROM_BLACK` on video on/off,
`CROSSFADE` on clip/restyle-variant swap, `LIVE_UPDATE` for the continuous
grade/overlay/level ops per §4.3, `NONE` when unchanged).
Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-player-alteration-core.md).
**Slice 2 — simulator-first alteration preview ✅ Done.** Merged to `main`
(session 0009). Wires the alteration engine into the web simulator so the look is
tunable by eye before any hardware, and **reconciles** the unmerged session-0007
design with the merged session-0008 design (the Left-HUD conflict): Left is a
**runtime overlay** (authored annotation track + per-language string tables),
Right is a **discrete pre-baked** flow-stabilized variant, Dark/Light a **live**
grade — superseding 0007's baked-HUD 5×5 grid. Engine: a parameterized
`Calibration` (tuned by eye, baked into `DEFAULT_CALIBRATION`), `Restyle.variant`
(discrete) replacing the continuous blend, and `AnalyticalOverlay.level`. Sim:
`/api/alteration` + `/api/clips` over `simulator/clips.py`, an alteration-preview
UI, and one neutral clip with a real flow-stabilized Right variant from the POC.
Design:
[`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md);
plan:
[`2026-06-07-reconciled-simulator-alteration-slice.md`](./superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md).
**Remaining slices (not started):**
- **Runtime renderer** — drive the single panoramic projector via mpv/ffmpeg;
GPU shaders for the luma-keyed mood grade + analytical-overlay compositing;
realize crossfade/fade timing and the 515 min loop. (Open decision: player
stack — mpv via IPC vs. ffmpeg vs. custom.)
- **Serial input** — real USB-serial reader on the Pi + the **3⇄4 framing
contract** with the firmware (the keyboard/serial stand-in already works via a
`Controls` stream).
- **White-noise generation** — runtime white/pink noise for that content position.
- **Offline v2v variant pipeline** — author the pre-baked Right restyle variants
via the local flow-stabilized SD pipeline (now ~free, not a paid API — see the
scales-library/right-axis design §1/§4); a real multi-strength flow-stabilized
re-bake per base clip + the multilingual label/string tables + TTS.
- **Scale-ring navigation** — the endless rotary encoder + short pre-baked AI
zoom/warp transitions between neutral "scales of nature" clips on a closed ring
(scales-library design §3); a new control + offline pipeline element.
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
variant" flag (sub-project 2 territory, design §13).
**Depends on:** sub-project 1 (`Coordinate`); a neutral base library from
sub-project 2 to be meaningful, but developable against a hand-authored library
and a **keyboard/serial stand-in** before firmware exists.
**Done when:** given a base library and a stream of control values, it plays the
correct altered segment, loops, crossfades, and goes dark on `Off` — the
decision logic is covered (slice 1, serial mocked); the renderer realizes it.
**Hardware:** Raspberry Pi 5 (holds the drive + base library + variants).
**Open decisions:** player stack (mpv via IPC vs. ffmpeg vs. custom); whether the
player hard-restricts to `approved` records (the `approved_only` flag is plumbed
but inert until backed by a real catalog); knob→strength calibration (the §3 vs
§4.2/§5 reconciliation — see the slice-1 plan / session 0006 transcript).
---
## 4. Arduino Firmware (control panel) ◻
**Goal:** the physical panel — turn knob/selector positions into serial messages.
**Delivers (`firmware/`):**
- Read four knobs (`left`, `right`, `dark`, `light`) and the 4-way content-mode
selector.
- Quantize/debounce analog inputs to the `0..4` integer scale.
- Send the five values to the Pi over USB serial in a defined framing.
**Depends on:** only the **serial protocol contract** shared with sub-project 3 —
agree that first, then 3 and 4 proceed in parallel.
**Done when:** turning a knob produces the expected serial message (verifiable on
a serial monitor) and the Pi player reacts.
**Hardware:** Arduino + pots/encoders + a 4-position selector.
**Open decisions:** pots vs. detented encoders; exact serial framing.
---
## 5. ~~Procedural Side Walls~~ ❌ Dropped (2026-06-04)
**Superseded.** The move to a single panoramic projector showing the real selected
video across the three walls (design spec §6/§7) removes the procedural ambient
renderer entirely — the mood axis is felt through the chosen content itself. With
it goes the only consumer of `dominant_color`, which becomes optional/opt-in in
the ingest tooling (sub-project 2 spec §5.2).
> *Original goal (for the record): render a slow color wash on the three
> non-primary walls — hue from the playing piece's `dominant_color`, brightness
> from the dark/light knobs — to make the mood axis felt in peripheral vision.*
---
## Cross-cutting decisions to settle as we go
Carried from the design spec's open-questions list — none block sub-project 2:
- **Serial protocol framing** between Arduino and Pi (needed before 3 ⇄ 4).
- **Player stack** on the Pi (mpv / ffmpeg / custom) and how the single panoramic
output is driven.
- **`approved`-only enforcement** in the player.
- ~~**Side walls** on the primary Pi vs. a second Pi~~ — resolved: side walls dropped.
- ~~**Media storage layout** on the drive~~ — settled by the sub-project-2 spec
§6.3: media under a `media-root`, `file_path` stored relative to it; the player
joins it with its drive mount.
+337
View File
@@ -0,0 +1,337 @@
# Human Experience Filter — User Guide
> **Scope (current build state).** Two pieces are built: the catalog + selection
> core, and the **`tools/` ingest & review pipeline** (sub-project 2). You can
> populate the catalog two ways — **hand-author records** (below) or **assisted
> ingest** that fetches from public-domain archives, mechanically tags, drafts
> coordinates, and lets you review them to `approved` (see *Ingesting & reviewing
> media*). The room **player** is a separate, not-yet-built sub-project.
---
## What "configuring media" means
The installation plays media chosen by where a viewer sets the control knobs.
Every piece of media is one **record** in a catalog file. A record says *what the
file is*, *where it lives on the drive*, *its license*, and — the important part —
*its coordinate* in the experience-space. At run time the player will pick the
record nearest the knob position; for now, you build and validate that catalog.
The catalog is a single file: **`catalog/library.jsonl`**. It is
[JSON Lines](https://jsonlines.org/) — **one JSON object per line**, one line per
piece of media. Blank lines are ignored.
---
## Prerequisites
The commands in this guide need **Python 3.11+** and must be run **from the repo
root**, where `import hef` resolves with no install step. Examples use
`.venv/bin/python`; if you made the project virtualenv they work as-is, otherwise
substitute your own `python3`.
Creating the virtualenv is optional but recommended:
```bash
python3 -m venv .venv
```
---
## The record schema
Each line is a JSON object. **Twelve fields are required**; the rest have
defaults and may be omitted.
### Required fields
| Field | Type | Meaning |
|------------------|--------|----------------------------------------------------------------|
| `id` | string | Stable unique id (non-empty). Your choice; keep it unique. |
| `title` | string | Human title. |
| `source_url` | string | Where the piece came from. |
| `source_archive` | string | Origin label, e.g. `internet_archive`, `musopen`, `librivox`. |
| `license` | string | One of: `public_domain`, `cc0`, `cc_by`, `cc_by_nc`. |
| `mode` | string | One of: `audio`, `video`, `av`. |
| `left` | int | Analytical / verbal axis, **04** (see rubric). |
| `right` | int | Artistic / abstract axis, **04**. |
| `dark` | int | Somber / heavy axis, **04**. |
| `light` | int | Uplifting / serene axis, **04**. |
| `duration_s` | int | Segment length in seconds (≥ 0). |
| `file_path` | string | Path to the media file on the player's drive. |
### Optional fields (defaults shown)
| Field | Default | Meaning |
|------------------|--------------|------------------------------------------------------------|
| `review_status` | `"proposed"` | `proposed` or `approved` — whether a human has blessed it. |
| `attribution` | `""` | **Required text when `license` is `cc_by` or `cc_by_nc`.** |
| `resolution` | `""` | e.g. `1920x1080` (video). |
| `dominant_color` | `""` | Hex color. **Optional / opt-in** — computed only with `--dominant-color`. |
| `rationale` | `""` | One-line note on why you chose the coordinate. |
| `reviewed_at` | `null` | Timestamp when approved. |
| `notes` | `""` | Free text. |
### Validation rules (enforced when you load/save)
- `id` must be non-empty.
- `mode`, `license`, `review_status` must be one of the allowed values above.
- `left`, `right`, `dark`, `light` must be **integers** in `0..4` (booleans are
rejected).
- `duration_s` must be `≥ 0`.
- `cc_by` and `cc_by_nc` require a non-empty `attribution`.
- Unknown fields and missing required fields are rejected.
> The file referenced by `file_path` is **not** checked for existence yet — only
> the record's structure is validated.
---
## The coordinate rubric
The four coordinates are the curatorial heart of the work. Each is a felt
judgment on a **04** scale. The two pairs are **independent knobs**, not opposite
ends of one slider — a piece can be high on both `left` and `right` ("whole
brain"), or high on both `dark` and `light` ("bittersweet").
| Axis | 0 means… | 4 means… |
|---------|---------------------|-------------------------------------------------------------|
| `left` | not analytical | strongly analytical / verbal / structured — narration, language, logic, sequence, instruction, documentary, spoken word |
| `right` | not artistic | strongly artistic / emotional / abstract — music, abstract visuals, dance, nature, awe, non-verbal, dreamlike |
| `dark` | not somber | strongly somber / heavy — ominous, melancholic, night, decay, minor-key, storms, noir |
| `light` | not light | strongly uplifting / serene — joyful, hopeful, bright, sunrise, gardens, major-key, calm |
`mode` describes the media's channels: `audio` (sound only), `video` (image
only), `av` (both). There is no `none` record — "None" is a control state the
player handles by going dark, not a kind of media.
**License stance:** prefer `public_domain` / `cc0`; `cc_by` is fine if you record
`attribution`; `cc_by_nc` only while the installation stays non-commercial.
---
## Adding media — two ways
### Option A: edit the JSONL file by hand
Append one line per piece to `catalog/library.jsonl`. A complete example record
(formatted across lines here for readability — **it must be a single line in the
file**):
```json
{"id": "nasa-earthrise", "title": "Earthrise", "source_url": "https://archive.org/details/earthrise", "source_archive": "internet_archive", "license": "public_domain", "mode": "video", "left": 0, "right": 4, "dark": 1, "light": 3, "duration_s": 600, "file_path": "/media/nasa-earthrise.mp4", "resolution": "1920x1080", "rationale": "wordless awe, bright Earth on black — strongly right-brain, gently light"}
```
### Option B: use the Python API (validates before writing)
```bash
.venv/bin/python -c '
from hef.catalog import Record, append_record
r = Record(
id="librivox-meditations",
title="Meditations, Book II (excerpt)",
source_url="https://librivox.org/meditations/",
source_archive="librivox",
license="public_domain",
mode="audio",
left=4, right=1, dark=2, light=2,
duration_s=720,
file_path="/media/librivox-meditations-ii.mp3",
rationale="spoken philosophy, verbal and reflective — strongly left-brain",
)
append_record(r, "catalog/library.jsonl")
print("added", r.id)
'
```
`append_record` validates the record and raises `CatalogError` (printing what is
wrong) before it writes, so a bad record never lands in the file.
### Option C: assisted ingest (fetch + tag + draft, then review)
See the next section.
---
## Ingesting & reviewing media
Instead of hand-authoring every record, the `tools/` pipeline can fetch a piece
from a public-domain archive, fill in the mechanical fields for you, **draft** a
coordinate, and write the record as `review_status: proposed`. You then walk the
proposed records and bless each one to `approved`. The four coordinates are still
a human act — the tool only *drafts* a starting point; nothing is selectable until
you approve it.
### Prerequisites
- **Python 3.11+**, run from the repo root (as everywhere in this guide).
- **`ffmpeg` and `ffprobe`** on your `PATH` — used to read media properties
(`mode`/`duration_s`/`resolution`), render review previews, and (opt-in)
compute `dominant_color`. Install via your package manager (e.g.
`brew install ffmpeg`). The unit tests don't need them; the live ingest/review
do.
### Where downloaded media goes (`--media-root`)
Ingest downloads each file to `<media-root>/<archive>/<id>.<ext>` and records a
`file_path` **relative to the media root** (e.g. `nasa/nasa-earthrise.mp4`).
Relative paths are portable: the tagging workstation and the Pi's drive store the
same tree under different mounts, and the player joins `file_path` with its own
mount. The media root is `--media-root DIR` (or `HEF_MEDIA_ROOT`), default
`./media/`, which is gitignored — **media never enters the repo**, only metadata.
> **Note on `file_path` style.** Hand-authored records (Option A/B above) often use
> absolute paths like `/media/earthrise.mp4`; ingested records use archive-relative
> paths. Both load and validate fine (`validate()` does not constrain the format).
> Keep a catalog internally consistent where you can; the player spec will define
> how mounts are resolved.
### First-ship archives
Three keyless, clean-license archives are wired today:
| `archive` arg | Pool | License |
|--------------------|---------------------------------------|-------------------|
| `librivox` | Public-domain audiobook recordings | `public_domain` |
| `nasa` | NASA imagery / video | `public_domain` |
| `internet_archive` | Internet Archive / Prelinger | per item (PD / CC)|
`musopen`, `fma`, and `freesound` are defined but **deferred** (auth or
API-stability tax) — invoking them exits with a "deferred" message. **Freesound**
will need an API token supplied via the `FREESOUND_API_TOKEN` environment variable
(a secret — never put it on the command line or into a record).
### Running ingest
```bash
.venv/bin/python -m tools.ingest_cli nasa --query "earthrise" --limit 5
.venv/bin/python -m tools.ingest_cli internet_archive --resolve prelinger_blast
.venv/bin/python -m tools.ingest_cli librivox --query "meditations" --media-root ./media
```
Flags: `--query` (search) or `--resolve <identifier>` (one item); `--limit N`;
`--catalog` (default `catalog/library.jsonl`); `--media-root` (default `./media`,
or `HEF_MEDIA_ROOT`); `--dominant-color` to compute `dominant_color` for
video/`av` records (off by default — its only consumer, the procedural side walls,
was dropped in the single-panoramic-projector design change). Re-running is
idempotent: a candidate whose id already exists is skipped.
### Reviewing proposed records
```bash
.venv/bin/python -m tools.review_cli --catalog catalog/library.jsonl --media-root ./media
```
For each `proposed` record it prints the id/title/source/license, the mechanical
fields, the **drafted coordinates + rationale**, and opens a preview frame
(video/`av`) or waveform image (audio). Then it prompts:
- **`a`** — accept the drafted coordinates and mark `approved`.
- **`e`** — edit `left/right/dark/light` (enter blank to keep a value), then approve.
- **`s`** — skip; leave it `proposed`.
- **`q`** — save and quit.
Each approval stamps `reviewed_at` and is written immediately (full rewrite), so
an interrupted session keeps its progress. **How far from done** is just the count
of records still `proposed`. Add `--no-preview` to skip frame rendering.
---
## Validating the whole catalog
Loading the catalog validates **every** record. Run this after editing by hand:
```bash
.venv/bin/python -c "from hef.catalog import load_catalog; print(len(load_catalog('catalog/library.jsonl')), 'records OK')"
```
- Prints e.g. `12 records OK` if everything is valid.
- Raises `CatalogError` describing the problem if not. A validation error names
the field and reason, e.g. `coordinate left=9 out of range 0..4`; malformed
JSON additionally names the line, e.g. `line 4: invalid JSON: ...`.
---
## Sanity-checking selection (optional)
To feel how tagging drives playback, you can ask the selection core which record
a given knob position would pick. Coordinates are `Coordinate(left, right, dark,
light)`; the second argument is the content-mode selector
(`audio` / `video` / `av` / `none`).
```bash
.venv/bin/python -c "
from hef.catalog import load_catalog
from hef.selection import Coordinate, select
lib = load_catalog('catalog/library.jsonl')
pick = select(lib, Coordinate(left=0, right=4, dark=1, light=3), 'video')
print(pick.id if pick else 'nothing')
"
```
Notes that shape how you tag:
- Selection is **nearest-match** — every knob position resolves to the closest
record, so a sparse catalog still works; you do not need to fill every
coordinate.
- `select(..., 'none')` always returns nothing (the void state).
- Pass `approved_only=True` to restrict to records whose `review_status` is
`approved`.
---
## Where the media files go
Records point at files via `file_path`. Those files live on the player's drive,
**not** in this git repo (the repo holds only the catalog metadata). Keep your
`file_path` values consistent with wherever you mount that drive on the machine
that will eventually run the room.
## Playing with the simulator (alteration preview)
The simulator is a web stand-in for the installation's control panel. It runs the
real `player.alteration` engine and **alters** a neutral base clip toward the knob
state in the browser, so you can tune the *look* of the filter before any hardware
exists. (The earlier selection-era "curator's X-ray" view was retired when the
piece moved from *selecting* clips to *altering* them.)
**One-time setup — populate the sample footage** (look-tuning only; not shipped
content):
python simulator/setup_sample_media.py
This copies the session-0008 POC artifacts (`~/hef-poc/out/`) into
`simulator/sample_media/forest/` — the neutral base clip and the real
flow-stabilized Right restyle — and generates placeholder intermediate Right
strengths. The `.mp4` binaries are gitignored.
**Run it (Docker):**
make sim
then open http://localhost:8000.
**Run it (no Docker):**
pip install -e ".[sim]"
make sim-local
**What you see and can do:**
- **Content dial** — picks audio/video channel; "off" and audio-only positions go
to black walls.
- **Four experience knobs (04):**
- **Dark / Light** — a live runtime color grade (cool/dark ↔ warm/bright; equal
or zero = the raw footage).
- **Right (dreamlike)** — selects a discrete pre-baked, flow-stabilized restyle
variant and crossfades to it (strength 0 = raw base).
- **Left (analytical)** — a live overlay: labelled boxes from the clip's authored
annotation track, with more annotations appearing at higher levels. Text is
shaped live (the simulator analogue of the Pi's Pango/HarfBuzz path).
- **Calibration sliders** — adjust the grade/overlay gain curves live; once a look
is liked, bake the values into `DEFAULT_CALIBRATION` in `player/alteration.py`.
- **RenderPlan readout** — always shows the exact numbers the engine produced (the
project's honesty "X-ray," now over the alteration model).
The base clips, Right variants, Left annotation track, and string tables come from
`simulator/sample_media/manifest.json`.
@@ -0,0 +1,965 @@
# Catalog & Selection Core Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the tested Python library that defines the content catalog (load/save/validate JSONL records) and the nearest-match selection algorithm that maps a knob coordinate to a media record.
**Architecture:** A small dependency-free package `hef/` with two modules — `catalog.py` (the `Record` data model + JSONL IO + validation) and `selection.py` (coordinate distance + mode filtering + nearest-match `select`). Everything is plain stdlib Python, fully unit-tested with pytest. The actual media files and hardware are out of scope here; this is the spine that the ingest tools, review tool, and Pi player (later sub-projects) all import.
**Tech Stack:** Python 3.11+, stdlib only (`dataclasses`, `json`, `math`, `pathlib`), pytest for tests.
**Where this sits:** This is sub-project **1 of 5** from the design spec (`docs/superpowers/specs/2026-06-04-human-experience-filter-design.md`). The others get their own plans later: (2) ingest & tagging tools + review CLI, (3) Pi player runtime, (4) Arduino firmware, (5) procedural side walls. This plan must be completed first because every other plan depends on `hef.catalog` and `hef.selection`.
---
## File Structure
```
pyproject.toml project + pytest config
conftest.py empty; puts repo root on sys.path so `import hef` works
.gitignore python artifacts + .venv
hef/__init__.py empty package marker
hef/catalog.py Record model, validation, JSONL load/save/append
hef/selection.py Coordinate, Weights, distance, mode filter, select()
catalog/library.jsonl the real catalog data file (starts empty)
tests/test_smoke.py package imports
tests/test_catalog.py Record validation + dict round-trip + JSONL IO
tests/test_selection.py distance + mode filtering + select()
```
`hef/` holds the shared library (imported by the future player and tools). `catalog/` holds data only. This is a small, intentional refinement of the spec's §10 layout: the spec put "selection algorithm" under `player/`, but selection is also needed by the tagging tools, so it lives in a shared `hef/` package rather than inside the Pi-only `player/`. `player/`, `tools/`, `firmware/`, and `sidewalls/` arrive in later sub-projects.
---
### Task 1: Project scaffolding + smoke test
**Files:**
- Create: `pyproject.toml`
- Create: `conftest.py`
- Create: `.gitignore`
- Create: `hef/__init__.py`
- Create: `catalog/library.jsonl`
- Test: `tests/test_smoke.py`
- [ ] **Step 1: Create the Python environment and install pytest**
Run:
```bash
cd /Users/benstull/git/benstull.org/benstull/human-experience-filter-art
python3 -m venv .venv
source .venv/bin/activate
pip install pytest
```
Expected: pytest installs without error. Use this `.venv` (activated) for every later `pytest` command.
- [ ] **Step 2: Create `pyproject.toml`**
```toml
[project]
name = "human-experience-filter"
version = "0.1.0"
description = "Coordinate-tuned public-domain media installation"
requires-python = ">=3.11"
[tool.pytest.ini_options]
testpaths = ["tests"]
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
packages = ["hef"]
```
- [ ] **Step 3: Create `conftest.py` (empty)**
An empty `conftest.py` at the repo root makes pytest insert the repo root into `sys.path`, so `import hef` resolves without `pip install -e .`.
```python
```
(The file is intentionally empty.)
- [ ] **Step 4: Create `.gitignore`**
```gitignore
__pycache__/
*.pyc
.pytest_cache/
.venv/
```
- [ ] **Step 5: Create `hef/__init__.py` (empty) and the empty catalog file**
`hef/__init__.py`:
```python
```
(Empty package marker.)
Create the empty data file:
```bash
mkdir -p hef catalog
touch hef/__init__.py catalog/library.jsonl
```
- [ ] **Step 6: Write the smoke test**
`tests/test_smoke.py`:
```python
def test_package_imports():
import hef
import hef.catalog
import hef.selection
assert hef is not None
```
- [ ] **Step 7: Run the smoke test (expect it to fail — modules don't exist yet)**
Run: `python -m pytest tests/test_smoke.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'hef.catalog'`.
- [ ] **Step 8: Create empty module stubs to make the smoke test pass**
`hef/catalog.py`:
```python
"""Catalog data model, validation, and JSONL IO."""
```
`hef/selection.py`:
```python
"""Nearest-match selection of a catalog record for a knob coordinate."""
```
- [ ] **Step 9: Run the smoke test (expect pass)**
Run: `python -m pytest tests/test_smoke.py -v`
Expected: PASS (1 passed).
- [ ] **Step 10: Commit**
```bash
git add pyproject.toml conftest.py .gitignore hef/ catalog/library.jsonl tests/test_smoke.py
git commit -m "chore: scaffold catalog+selection core package"
```
---
### Task 2: Record model + validation
**Files:**
- Modify: `hef/catalog.py`
- Test: `tests/test_catalog.py`
- [ ] **Step 1: Write the failing tests for `Record` + `validate`**
`tests/test_catalog.py`:
```python
import pytest
from hef.catalog import Record, validate, CatalogError
def make_record(**overrides):
base = dict(
id="nasa-apollo-earthrise",
title="Earthrise",
source_url="https://archive.org/details/earthrise",
source_archive="internet_archive",
license="public_domain",
mode="video",
left=1,
right=4,
dark=1,
light=3,
duration_s=600,
file_path="/media/earthrise.mp4",
)
base.update(overrides)
return Record(**base)
def test_valid_record_passes():
validate(make_record()) # must not raise
def test_defaults_are_set():
record = make_record()
assert record.review_status == "proposed"
assert record.attribution == ""
assert record.reviewed_at is None
def test_invalid_mode_raises():
with pytest.raises(CatalogError):
validate(make_record(mode="hologram"))
def test_invalid_license_raises():
with pytest.raises(CatalogError):
validate(make_record(license="all_rights_reserved"))
def test_coordinate_out_of_range_raises():
with pytest.raises(CatalogError):
validate(make_record(left=5))
with pytest.raises(CatalogError):
validate(make_record(dark=-1))
def test_boolean_coordinate_rejected():
with pytest.raises(CatalogError):
validate(make_record(right=True))
def test_cc_by_requires_attribution():
with pytest.raises(CatalogError):
validate(make_record(license="cc_by", attribution=""))
validate(make_record(license="cc_by", attribution="Jane Doe, CC BY 4.0"))
def test_empty_id_raises():
with pytest.raises(CatalogError):
validate(make_record(id=""))
def test_negative_duration_raises():
with pytest.raises(CatalogError):
validate(make_record(duration_s=-1))
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `python -m pytest tests/test_catalog.py -v`
Expected: FAIL with `ImportError: cannot import name 'Record' from 'hef.catalog'`.
- [ ] **Step 3: Implement `Record`, constants, and `validate`**
Replace the contents of `hef/catalog.py` with:
```python
"""Catalog data model, validation, and JSONL IO."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Optional
MODES = frozenset({"audio", "video", "av"})
LICENSES = frozenset({"public_domain", "cc0", "cc_by", "cc_by_nc"})
REVIEW_STATUSES = frozenset({"proposed", "approved"})
COORD_FIELDS = ("left", "right", "dark", "light")
COORD_MIN = 0
COORD_MAX = 4
ATTRIBUTION_LICENSES = frozenset({"cc_by", "cc_by_nc"})
class CatalogError(ValueError):
"""Raised when a catalog record is structurally invalid."""
@dataclass
class Record:
id: str
title: str
source_url: str
source_archive: str
license: str
mode: str
left: int
right: int
dark: int
light: int
duration_s: int
file_path: str
review_status: str = "proposed"
attribution: str = ""
resolution: str = ""
dominant_color: str = ""
rationale: str = ""
reviewed_at: Optional[str] = None
notes: str = ""
def validate(record: Record) -> None:
"""Raise CatalogError if the record is structurally invalid."""
if not record.id:
raise CatalogError("record id must be non-empty")
if record.mode not in MODES:
raise CatalogError(
f"invalid mode {record.mode!r}; expected one of {sorted(MODES)}"
)
if record.license not in LICENSES:
raise CatalogError(
f"invalid license {record.license!r}; expected one of {sorted(LICENSES)}"
)
if record.review_status not in REVIEW_STATUSES:
raise CatalogError(
f"invalid review_status {record.review_status!r}; "
f"expected one of {sorted(REVIEW_STATUSES)}"
)
for axis in COORD_FIELDS:
value = getattr(record, axis)
if isinstance(value, bool) or not isinstance(value, int):
raise CatalogError(f"coordinate {axis} must be an int, got {value!r}")
if not (COORD_MIN <= value <= COORD_MAX):
raise CatalogError(
f"coordinate {axis}={value} out of range {COORD_MIN}..{COORD_MAX}"
)
if record.duration_s < 0:
raise CatalogError("duration_s must be non-negative")
if record.license in ATTRIBUTION_LICENSES and not record.attribution:
raise CatalogError(f"license {record.license} requires attribution")
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `python -m pytest tests/test_catalog.py -v`
Expected: PASS (all tests in `test_catalog.py` pass).
- [ ] **Step 5: Commit**
```bash
git add hef/catalog.py tests/test_catalog.py
git commit -m "feat: catalog Record model and validation"
```
---
### Task 3: Record <-> dict conversion
**Files:**
- Modify: `hef/catalog.py`
- Test: `tests/test_catalog.py`
- [ ] **Step 1: Add failing tests for dict conversion**
Append to `tests/test_catalog.py`:
```python
from hef.catalog import record_to_dict, record_from_dict
def test_dict_round_trip():
record = make_record()
restored = record_from_dict(record_to_dict(record))
assert restored == record
def test_record_from_dict_rejects_unknown_field():
data = record_to_dict(make_record())
data["bogus"] = 1
with pytest.raises(CatalogError):
record_from_dict(data)
def test_record_from_dict_rejects_missing_required_field():
data = record_to_dict(make_record())
del data["title"]
with pytest.raises(CatalogError):
record_from_dict(data)
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `python -m pytest tests/test_catalog.py -k "dict" -v`
Expected: FAIL with `ImportError: cannot import name 'record_to_dict'`.
- [ ] **Step 3: Implement the conversion functions**
Add to the imports at the top of `hef/catalog.py`:
```python
from dataclasses import dataclass, asdict, fields
```
(Replace the existing `from dataclasses import dataclass` line.)
Append to `hef/catalog.py`:
```python
def record_to_dict(record: Record) -> dict:
"""Convert a Record to a plain dict suitable for JSON serialization."""
return asdict(record)
def record_from_dict(data: dict) -> Record:
"""Build a Record from a dict, rejecting unknown or missing fields."""
known = {f.name for f in fields(Record)}
unknown = set(data) - known
if unknown:
raise CatalogError(f"unknown fields: {sorted(unknown)}")
try:
return Record(**data)
except TypeError as exc:
raise CatalogError(str(exc)) from exc
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `python -m pytest tests/test_catalog.py -v`
Expected: PASS (all `test_catalog.py` tests, including the new dict tests).
- [ ] **Step 5: Commit**
```bash
git add hef/catalog.py tests/test_catalog.py
git commit -m "feat: catalog Record dict conversion"
```
---
### Task 4: JSONL load / save / append
**Files:**
- Modify: `hef/catalog.py`
- Test: `tests/test_catalog.py`
- [ ] **Step 1: Add failing tests for JSONL IO**
Append to `tests/test_catalog.py`:
```python
from hef.catalog import load_catalog, save_catalog, append_record
def test_save_then_load_round_trip(tmp_path):
path = tmp_path / "library.jsonl"
records = [make_record(id="a"), make_record(id="b", mode="audio")]
save_catalog(records, path)
loaded = load_catalog(path)
assert loaded == records
def test_load_skips_blank_lines(tmp_path):
path = tmp_path / "library.jsonl"
save_catalog([make_record(id="a")], path)
with path.open("a", encoding="utf-8") as fh:
fh.write("\n \n")
loaded = load_catalog(path)
assert len(loaded) == 1
def test_load_empty_file_returns_empty_list(tmp_path):
path = tmp_path / "library.jsonl"
path.write_text("", encoding="utf-8")
assert load_catalog(path) == []
def test_load_invalid_json_raises(tmp_path):
path = tmp_path / "library.jsonl"
path.write_text("{not json}\n", encoding="utf-8")
with pytest.raises(CatalogError):
load_catalog(path)
def test_load_validates_records(tmp_path):
path = tmp_path / "library.jsonl"
save_catalog([make_record(id="a")], path)
import json
bad = record_to_dict(make_record(id="bad"))
bad["left"] = 9
with path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(bad) + "\n")
with pytest.raises(CatalogError):
load_catalog(path)
def test_append_record(tmp_path):
path = tmp_path / "library.jsonl"
save_catalog([make_record(id="a")], path)
append_record(make_record(id="b"), path)
loaded = load_catalog(path)
assert [r.id for r in loaded] == ["a", "b"]
def test_save_rejects_invalid_record(tmp_path):
path = tmp_path / "library.jsonl"
with pytest.raises(CatalogError):
save_catalog([make_record(mode="hologram")], path)
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `python -m pytest tests/test_catalog.py -k "load or save or append" -v`
Expected: FAIL with `ImportError: cannot import name 'load_catalog'`.
- [ ] **Step 3: Implement the JSONL IO functions**
Add these two imports to `hef/catalog.py` **below** the existing
`from dataclasses import ...` / `from typing import Optional` lines (they must come
after `from __future__ import annotations`, which has to stay the first statement):
```python
import json
from pathlib import Path
```
Append to `hef/catalog.py`:
```python
def load_catalog(path) -> list[Record]:
"""Load and validate every record from a JSONL file. Blank lines skipped."""
path = Path(path)
records: list[Record] = []
with path.open("r", encoding="utf-8") as fh:
for lineno, raw in enumerate(fh, start=1):
line = raw.strip()
if not line:
continue
try:
data = json.loads(line)
except json.JSONDecodeError as exc:
raise CatalogError(f"line {lineno}: invalid JSON: {exc}") from exc
record = record_from_dict(data)
validate(record)
records.append(record)
return records
def save_catalog(records, path) -> None:
"""Validate and write all records to a JSONL file (overwrites)."""
path = Path(path)
with path.open("w", encoding="utf-8") as fh:
for record in records:
validate(record)
fh.write(json.dumps(record_to_dict(record), ensure_ascii=False) + "\n")
def append_record(record, path) -> None:
"""Validate and append a single record to a JSONL file."""
validate(record)
path = Path(path)
with path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(record_to_dict(record), ensure_ascii=False) + "\n")
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `python -m pytest tests/test_catalog.py -v`
Expected: PASS (all `test_catalog.py` tests).
- [ ] **Step 5: Commit**
```bash
git add hef/catalog.py tests/test_catalog.py
git commit -m "feat: catalog JSONL load/save/append"
```
---
### Task 5: Coordinate + distance
**Files:**
- Modify: `hef/selection.py`
- Test: `tests/test_selection.py`
- [ ] **Step 1: Write failing tests for distance**
`tests/test_selection.py`:
```python
import math
import pytest
from hef.catalog import Record
from hef.selection import Coordinate, Weights, distance, record_coordinate
def make_record(**overrides):
base = dict(
id="r",
title="t",
source_url="u",
source_archive="internet_archive",
license="public_domain",
mode="video",
left=0,
right=0,
dark=0,
light=0,
duration_s=600,
file_path="/media/r.mp4",
)
base.update(overrides)
return Record(**base)
def test_distance_zero_for_same_point():
c = Coordinate(2, 2, 2, 2)
assert distance(c, c) == 0.0
def test_distance_is_euclidean_by_default():
a = Coordinate(0, 0, 0, 0)
b = Coordinate(1, 1, 1, 1)
assert distance(a, b) == pytest.approx(2.0) # sqrt(1+1+1+1)
def test_weights_scale_planes_independently():
a = Coordinate(0, 0, 0, 0)
brain_only = Coordinate(1, 0, 0, 0)
mood_only = Coordinate(0, 0, 1, 0)
w = Weights(brain=4.0, mood=1.0)
assert distance(a, brain_only, w) == pytest.approx(2.0) # sqrt(4*1)
assert distance(a, mood_only, w) == pytest.approx(1.0) # sqrt(1*1)
def test_record_coordinate_extracts_axes():
record = make_record(left=1, right=2, dark=3, light=4)
assert record_coordinate(record) == Coordinate(1, 2, 3, 4)
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `python -m pytest tests/test_selection.py -v`
Expected: FAIL with `ImportError: cannot import name 'Coordinate' from 'hef.selection'`.
- [ ] **Step 3: Implement Coordinate, Weights, distance, record_coordinate**
Replace the contents of `hef/selection.py` with:
```python
"""Nearest-match selection of a catalog record for a knob coordinate."""
from __future__ import annotations
import math
from dataclasses import dataclass
from hef.catalog import Record
SELECTOR_MODES = frozenset({"none", "audio", "video", "av"})
CONTENT_MODES = frozenset({"audio", "video", "av"})
@dataclass(frozen=True)
class Coordinate:
left: int
right: int
dark: int
light: int
@dataclass(frozen=True)
class Weights:
brain: float = 1.0
mood: float = 1.0
def distance(a: Coordinate, b: Coordinate, weights: Weights = Weights()) -> float:
"""Weighted Euclidean distance; brain plane and mood plane weighted separately."""
brain_sq = (a.left - b.left) ** 2 + (a.right - b.right) ** 2
mood_sq = (a.dark - b.dark) ** 2 + (a.light - b.light) ** 2
return math.sqrt(weights.brain * brain_sq + weights.mood * mood_sq)
def record_coordinate(record: Record) -> Coordinate:
"""The (left, right, dark, light) coordinate of a catalog record."""
return Coordinate(record.left, record.right, record.dark, record.light)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `python -m pytest tests/test_selection.py -v`
Expected: PASS (the distance tests pass).
- [ ] **Step 5: Commit**
```bash
git add hef/selection.py tests/test_selection.py
git commit -m "feat: selection Coordinate and weighted distance"
```
---
### Task 6: Mode filtering with A+V fallback
**Files:**
- Modify: `hef/selection.py`
- Test: `tests/test_selection.py`
- [ ] **Step 1: Add failing tests for `candidates_for_mode`**
Append to `tests/test_selection.py`:
```python
from hef.selection import candidates_for_mode
def test_candidates_filter_to_exact_mode():
records = [make_record(id="v", mode="video"), make_record(id="a", mode="audio")]
result = candidates_for_mode(records, "audio", pool_size=4)
assert [r.id for r in result] == ["a"]
def test_av_does_not_fall_back_when_enough_av():
records = [make_record(id=f"av{i}", mode="av") for i in range(4)]
records.append(make_record(id="a", mode="audio"))
result = candidates_for_mode(records, "av", pool_size=4)
assert {r.id for r in result} == {"av0", "av1", "av2", "av3"}
def test_av_falls_back_to_audio_and_video_when_thin():
records = [
make_record(id="av0", mode="av"),
make_record(id="a", mode="audio"),
make_record(id="v", mode="video"),
]
result = candidates_for_mode(records, "av", pool_size=4)
assert {r.id for r in result} == {"av0", "a", "v"}
def test_candidates_rejects_none_mode():
with pytest.raises(ValueError):
candidates_for_mode([], "none", pool_size=4)
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `python -m pytest tests/test_selection.py -k "candidates or av" -v`
Expected: FAIL with `ImportError: cannot import name 'candidates_for_mode'`.
- [ ] **Step 3: Implement `candidates_for_mode`**
Append to `hef/selection.py`:
```python
def candidates_for_mode(records, mode: str, pool_size: int) -> list[Record]:
"""Records eligible for a content mode.
For 'av', if fewer than pool_size native 'av' records exist, fall back to
including 'audio' and 'video' records so the pool is never starved.
"""
if mode not in CONTENT_MODES:
raise ValueError(
f"candidates_for_mode expects a content mode {sorted(CONTENT_MODES)}, "
f"got {mode!r}"
)
primary = [r for r in records if r.mode == mode]
if mode == "av" and len(primary) < pool_size:
extra = [r for r in records if r.mode in {"audio", "video"}]
return primary + extra
return primary
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `python -m pytest tests/test_selection.py -v`
Expected: PASS (mode-filtering tests pass).
- [ ] **Step 5: Commit**
```bash
git add hef/selection.py tests/test_selection.py
git commit -m "feat: selection mode filtering with av fallback"
```
---
### Task 7: The `select` function
**Files:**
- Modify: `hef/selection.py`
- Test: `tests/test_selection.py`
- [ ] **Step 1: Add failing tests for `select`**
Append to `tests/test_selection.py`:
```python
import random
from hef.selection import select
def test_none_mode_returns_none():
records = [make_record(id="v", mode="video")]
assert select(records, Coordinate(0, 0, 0, 0), "none") is None
def test_empty_pool_returns_none():
assert select([], Coordinate(0, 0, 0, 0), "video") is None
def test_select_returns_nearest_by_default():
near = make_record(id="near", mode="video", left=2, right=2, dark=2, light=2)
far = make_record(id="far", mode="video", left=0, right=0, dark=0, light=0)
result = select([far, near], Coordinate(2, 2, 2, 2), "video")
assert result.id == "near"
def test_select_is_deterministic_without_rng():
records = [
make_record(id="b", mode="video", left=1),
make_record(id="a", mode="video", left=1),
]
# Equal distance -> tie broken by id, so "a" wins deterministically.
result = select(records, Coordinate(1, 0, 0, 0), "video")
assert result.id == "a"
def test_select_with_rng_picks_within_nearest_pool():
records = [make_record(id=f"r{i}", mode="video", left=i % 5) for i in range(10)]
coord = Coordinate(2, 0, 0, 0)
rng = random.Random(0)
chosen = {select(records, coord, "video", pool_size=3, rng=rng).id for _ in range(50)}
# Only records inside the 3-nearest pool may ever be chosen.
ranked = sorted(records, key=lambda r: abs(r.left - 2))
allowed = {r.id for r in ranked[:3]}
assert chosen <= allowed
def test_approved_only_excludes_proposed():
proposed = make_record(id="p", mode="video", review_status="proposed")
approved = make_record(id="ok", mode="video", review_status="approved")
result = select([proposed, approved], Coordinate(0, 0, 0, 0), "video",
approved_only=True)
assert result.id == "ok"
def test_invalid_selector_mode_raises():
with pytest.raises(ValueError):
select([], Coordinate(0, 0, 0, 0), "telepathy")
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `python -m pytest tests/test_selection.py -k "select or none or approved or rng" -v`
Expected: FAIL with `ImportError: cannot import name 'select'`.
- [ ] **Step 3: Implement `select`**
Append to `hef/selection.py`:
```python
from typing import Optional
def select(
records,
coord: Coordinate,
mode: str,
*,
pool_size: int = 4,
weights: Weights = Weights(),
approved_only: bool = False,
rng=None,
) -> Optional[Record]:
"""Pick the record nearest `coord` for the given selector `mode`.
- 'none' selector mode returns None (the void/rest state).
- With rng=None, returns the single nearest record (ties broken by id) for
deterministic behavior. Pass a random.Random to shuffle within the
pool_size nearest records.
- approved_only restricts to records the human has blessed.
"""
if mode not in SELECTOR_MODES:
raise ValueError(
f"invalid selector mode {mode!r}; expected one of {sorted(SELECTOR_MODES)}"
)
if mode == "none":
return None
pool = records
if approved_only:
pool = [r for r in pool if r.review_status == "approved"]
candidates = candidates_for_mode(pool, mode, pool_size)
if not candidates:
return None
ranked = sorted(
candidates,
key=lambda r: (distance(coord, record_coordinate(r), weights), r.id),
)
nearest = ranked[:pool_size]
if rng is None:
return nearest[0]
return rng.choice(nearest)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `python -m pytest tests/test_selection.py -v`
Expected: PASS (all selection tests).
- [ ] **Step 5: Commit**
```bash
git add hef/selection.py tests/test_selection.py
git commit -m "feat: nearest-match select function"
```
---
### Task 8: End-to-end integration test + empty catalog check
**Files:**
- Test: `tests/test_integration.py`
- [ ] **Step 1: Write the integration test**
This proves the full spine: build records, write them to JSONL, load them back, and select against a knob coordinate. It also asserts the committed empty `catalog/library.jsonl` loads cleanly to an empty list.
`tests/test_integration.py`:
```python
from pathlib import Path
from hef.catalog import Record, save_catalog, load_catalog
from hef.selection import Coordinate, select
REPO_ROOT = Path(__file__).resolve().parent.parent
def _rec(id, mode, left, right, dark, light):
return Record(
id=id,
title=id,
source_url=f"https://example.org/{id}",
source_archive="internet_archive",
license="public_domain",
mode=mode,
left=left,
right=right,
dark=dark,
light=light,
duration_s=600,
file_path=f"/media/{id}.mp4",
)
def test_catalog_round_trip_then_select(tmp_path):
library = [
_rec("lecture", "av", left=4, right=0, dark=1, light=2),
_rec("nebula", "video", left=0, right=4, dark=2, light=3),
_rec("storm", "av", left=0, right=3, dark=4, light=0),
_rec("sunrise", "av", left=1, right=3, dark=0, light=4),
]
path = tmp_path / "library.jsonl"
save_catalog(library, path)
loaded = load_catalog(path)
# A right-brain, very-light tuning should land on "sunrise".
chosen = select(loaded, Coordinate(left=1, right=3, dark=0, light=4), "av")
assert chosen.id == "sunrise"
# The void state returns nothing regardless of library.
assert select(loaded, Coordinate(0, 0, 0, 0), "none") is None
def test_committed_empty_catalog_loads():
path = REPO_ROOT / "catalog" / "library.jsonl"
assert load_catalog(path) == []
```
- [ ] **Step 2: Run the integration test**
Run: `python -m pytest tests/test_integration.py -v`
Expected: PASS (2 passed).
- [ ] **Step 3: Run the full suite**
Run: `python -m pytest -v`
Expected: PASS (every test across smoke, catalog, selection, integration).
- [ ] **Step 4: Commit**
```bash
git add tests/test_integration.py
git commit -m "test: end-to-end catalog+selection integration"
```
---
## Done criteria
- `python -m pytest -v` passes from the repo root inside the `.venv`.
- `hef.catalog` exposes `Record`, `validate`, `record_to_dict`, `record_from_dict`, `load_catalog`, `save_catalog`, `append_record`, `CatalogError`.
- `hef.selection` exposes `Coordinate`, `Weights`, `distance`, `record_coordinate`, `candidates_for_mode`, `select`, `SELECTOR_MODES`.
- `catalog/library.jsonl` exists, is empty, and loads to `[]`.
This spine is what sub-project 2 (ingest & review tools) will write records into, and what sub-project 3 (the Pi player) will call `select()` on each time a knob moves.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,500 @@
# Ingest & Tagging / Review Tools — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the `tools/` package that turns the hand-authored catalog into the assisted *draft-then-review* pipeline: per-archive ingest fetchers, mechanical tagging (ffprobe/ffmpeg + origin license), heuristic coordinate drafting (`review_status=proposed` + one-line rationale), and an interactive review CLI that flips records to `approved` with a `reviewed_at` stamp.
**Spec:** [`2026-06-04-ingest-tagging-review-tools.md`](../specs/2026-06-04-ingest-tagging-review-tools.md). Read it first — this plan implements it task-by-task and inherits its decisions (esp. §3 the single required `hef.catalog` addition, §5.2 `dominant_color` is optional/opt-in, §6.4 first-ship = LibriVox + NASA + Internet Archive).
**Architecture:** A `tools/` package alongside the existing `hef/` (which it imports, never forks). The two external boundaries — `ffmpeg`/`ffprobe` (subprocess) and archive HTTP — are isolated behind injectable seams (a `Prober`, a `download` callable, an HTTP client) so the unit suite stays **hermetic** (no network, no binaries). Integration tests that actually invoke `ffprobe`/`ffmpeg` are opt-in and skipped when the binaries are absent.
**Tech Stack:** Python 3.11+, stdlib only in the testable core (`dataclasses`, `json`, `subprocess`, `urllib`, `pathlib`, `datetime`); `ffmpeg`/`ffprobe` system binaries; pytest. No `requests`, no image library (dominant color, when computed, is ffmpeg-only).
**Where this sits:** sub-project **2 of 5**. Depends on sub-project 1 (`hef.catalog`, `hef.selection`, built). Feeds sub-project 3 (the Pi player). Sub-project 5 (side walls) was dropped (design §7 revision), so `dominant_color` is optional here.
---
## File Structure
```
pyproject.toml + add `tools` to packages; optional [project.scripts]
.gitignore + media/
hef/catalog.py + validate_catalog(), index_by_id() (ONLY hef change)
tools/__init__.py package marker
tools/http.py stdlib-urllib client (timeout/retry/user-agent), injectable
tools/probe.py Prober: ffprobe wrapper -> parsed Probe result
tools/tagging.py mode/duration_s/resolution from a Probe (+ cover-art guard)
tools/mediatools.py ffmpeg: representative frame; optional dominant_color
tools/licensing.py per-archive license/attribution -> hef LICENSES vocab
tools/drafting.py Signals, Draft, Proposer, HeuristicProposer
tools/ingest/__init__.py
tools/ingest/base.py Candidate, Fetcher, ingest_candidate() pipeline
tools/ingest/librivox.py first-ship fetcher
tools/ingest/nasa.py first-ship fetcher
tools/ingest/internet_archive.py first-ship fetcher
tools/ingest/musopen.py deferred: docstring stub raising NotImplementedError
tools/ingest/fma.py deferred: docstring stub
tools/ingest/freesound.py deferred: docstring stub (needs FREESOUND_API_TOKEN)
tools/ingest_cli.py `python -m tools.ingest_cli`
tools/review.py proposed_records(), approve() (pure transition core)
tools/review_cli.py `python -m tools.review_cli` (interactive walk)
tests/test_catalog_integrity.py validate_catalog / index_by_id
tests/test_probe.py ffprobe JSON parsing (canned)
tests/test_tagging.py mode/duration/resolution + cover-art guard
tests/test_mediatools.py dominant_color from canned rgb bytes; opt-in default off
tests/test_licensing.py per-archive license normalization
tests/test_drafting.py HeuristicProposer
tests/test_fetchers.py librivox/nasa/internet_archive via fake HTTP client
tests/test_ingest_pipeline.py ingest_candidate end-to-end with fakes
tests/test_review.py proposed_records / approve transition
tests/test_tools_integration.py end-to-end mocked; opt-in real-ffprobe (skipped if absent)
```
`tools/` holds everything new; `hef/` gains only the two additive symbols of spec §3.
---
### Task 1: Scaffolding — `tools/` package, deps, http client, smoke test
**Files:** `pyproject.toml`, `.gitignore`, `tools/__init__.py`, `tools/http.py`, `tests/test_tools_smoke.py`
- [ ] **Step 1: Add `tools` to packaging.** In `pyproject.toml`, change `packages = ["hef"]``packages = ["hef", "tools"]`. (Optional: add `[project.scripts]` `hef-ingest = "tools.ingest_cli:main"` and `hef-review = "tools.review_cli:main"` for installed checkouts; `python -m` is the documented path either way.)
- [ ] **Step 2: gitignore media.** Append `media/` to `.gitignore`.
- [ ] **Step 3: Write the failing smoke test** `tests/test_tools_smoke.py`:
```python
def test_tools_package_imports():
import tools
import tools.http
assert tools is not None
```
Run `python -m pytest tests/test_tools_smoke.py -v` → FAIL (`ModuleNotFoundError: tools`).
- [ ] **Step 4: Create `tools/__init__.py`** (empty) and `tools/http.py` — a tiny stdlib client:
```python
"""Minimal HTTP client over urllib: timeout, small retry, user-agent. Injectable."""
from __future__ import annotations
import json as _json
import time
import urllib.request
USER_AGENT = "human-experience-filter-ingest/0.1 (+local art installation)"
class HttpClient:
def __init__(self, *, timeout=30.0, retries=2, opener=None):
self.timeout, self.retries = timeout, retries
self._opener = opener or urllib.request.urlopen
def get_bytes(self, url, *, headers=None):
last = None
for attempt in range(self.retries + 1):
try:
req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT, **(headers or {})})
with self._opener(req, timeout=self.timeout) as resp:
return resp.read()
except Exception as exc: # noqa: BLE001 - retried below
last = exc
if attempt < self.retries:
time.sleep(0.2 * (attempt + 1))
raise last
def get_json(self, url, *, headers=None):
return _json.loads(self.get_bytes(url, headers=headers).decode("utf-8"))
```
The `opener` injection point lets tests pass a fake (no network).
- [ ] **Step 5:** Run smoke test → PASS. Commit:
```
git add pyproject.toml .gitignore tools/__init__.py tools/http.py tests/test_tools_smoke.py
git commit -m "chore: scaffold tools/ package + stdlib http client"
```
---
### Task 2: `hef.catalog` additions — `validate_catalog` + `index_by_id`
The ONLY change to the shared spine (spec §3). Purely additive.
**Files:** `hef/catalog.py`, `tests/test_catalog_integrity.py`
- [ ] **Step 1: Failing tests** `tests/test_catalog_integrity.py` (reuse a local `make_record` like `tests/test_catalog.py`):
```python
import pytest
from hef.catalog import Record, validate_catalog, index_by_id, CatalogError
def make_record(**o):
base = dict(id="a", title="t", source_url="u", source_archive="nasa",
license="public_domain", mode="video", left=0, right=0, dark=0,
light=0, duration_s=1, file_path="p")
base.update(o); return Record(**base)
def test_validate_catalog_accepts_unique_ids():
validate_catalog([make_record(id="a"), make_record(id="b")]) # no raise
def test_validate_catalog_rejects_duplicate_id():
with pytest.raises(CatalogError) as e:
validate_catalog([make_record(id="dup"), make_record(id="dup")])
assert "dup" in str(e.value)
def test_validate_catalog_validates_each_record():
with pytest.raises(CatalogError):
validate_catalog([make_record(left=9)])
def test_index_by_id_round_trips():
recs = [make_record(id="a"), make_record(id="b")]
idx = index_by_id(recs)
assert idx["a"].id == "a" and idx["b"].id == "b"
def test_index_by_id_rejects_duplicates():
with pytest.raises(CatalogError):
index_by_id([make_record(id="x"), make_record(id="x")])
```
Run → FAIL (ImportError).
- [ ] **Step 2: Implement** — append to `hef/catalog.py` (do NOT touch existing functions):
```python
def index_by_id(records) -> dict:
"""Map id -> Record, raising CatalogError on a duplicate id."""
idx: dict = {}
for r in records:
if r.id in idx:
raise CatalogError(f"duplicate record id: {r.id!r}")
idx[r.id] = r
return idx
def validate_catalog(records) -> None:
"""Validate every record AND cross-record invariants (currently: unique ids)."""
for r in records:
validate(r)
index_by_id(records) # raises on duplicate id
```
- [ ] **Step 3:** Run `python -m pytest tests/test_catalog_integrity.py -v` → PASS. Run full suite `python -m pytest -q` → still green (37 prior + new). Commit:
```
git add hef/catalog.py tests/test_catalog_integrity.py
git commit -m "feat: catalog-level validate_catalog + index_by_id (unique ids)"
```
---
### Task 3: `tools/probe.py` — ffprobe wrapper
**Files:** `tools/probe.py`, `tests/test_probe.py`
- [ ] **Step 1: Failing tests** — feed canned `ffprobe` JSON via an injected runner; assert a parsed `Probe` (list of streams + format dict). Cover: a video+audio file, an audio-only file, and an audio file with an `attached_pic` cover stream. Example:
```python
import json
from tools.probe import Probe, probe_file
def fake_runner(args): # mimics subprocess.run(...).stdout
return json.dumps({
"streams": [
{"codec_type": "video", "width": 1920, "height": 1080, "disposition": {"attached_pic": 0}},
{"codec_type": "audio"},
],
"format": {"duration": "12.5"},
})
def test_probe_parses_streams_and_format():
p = probe_file("x.mp4", runner=fake_runner)
assert any(s["codec_type"] == "video" for s in p.streams)
assert p.format["duration"] == "12.5"
```
- [ ] **Step 2: Implement** `tools/probe.py`:
```python
"""ffprobe wrapper -> parsed streams/format. Subprocess runner is injectable."""
from __future__ import annotations
import json, subprocess
from dataclasses import dataclass
@dataclass
class Probe:
streams: list
format: dict
def _default_runner(args) -> str:
return subprocess.run(args, capture_output=True, text=True, check=True).stdout
def probe_file(path, *, runner=_default_runner) -> Probe:
out = runner(["ffprobe", "-v", "quiet", "-print_format", "json",
"-show_format", "-show_streams", str(path)])
data = json.loads(out)
return Probe(streams=data.get("streams", []), format=data.get("format", {}))
```
- [ ] **Step 3:** Run → PASS. Commit `feat: ffprobe wrapper (tools.probe)`.
---
### Task 4: `tools/tagging.py` — mode / duration_s / resolution (with cover-art guard)
**Files:** `tools/tagging.py`, `tests/test_tagging.py`
- [ ] **Step 1: Failing tests** covering the spec §5.1 cases:
- video+audio → `mode="av"`, `resolution="1920x1080"`, `duration_s=13` (round 12.5).
- audio only → `mode="audio"`, `resolution=""`.
- **audio + `attached_pic` cover image → `mode="audio"`** (the guard), `resolution=""`.
- video only → `mode="video"`.
- duration fallback to longest stream when `format.duration` absent.
```python
from tools.probe import Probe
from tools.tagging import derive_tags
def test_cover_art_stays_audio():
p = Probe(streams=[
{"codec_type": "audio", "duration": "30.0"},
{"codec_type": "video", "width": 600, "height": 600, "disposition": {"attached_pic": 1}},
], format={"duration": "30.0"})
t = derive_tags(p)
assert t.mode == "audio" and t.resolution == "" and t.duration_s == 30
```
- [ ] **Step 2: Implement** — a `Tags(mode, duration_s, resolution)` dataclass and `derive_tags(probe)`:
- real video streams = `codec_type == "video"` AND `disposition.attached_pic != 1`.
- mode: both→`av`, video-only→`video`, else→`audio`.
- resolution from the first real video stream (`f"{w}x{h}"`), else `""`.
- duration: `round(float(format["duration"]))` else max stream duration else 0; never negative.
- [ ] **Step 3:** Run → PASS. Commit `feat: mechanical mode/duration/resolution tagging`.
---
### Task 5: `tools/mediatools.py` — representative frame + optional dominant_color
**Files:** `tools/mediatools.py`, `tests/test_mediatools.py`
- [ ] **Step 1: Failing tests** (hermetic — inject the ffmpeg runner returning canned bytes):
- `dominant_color_from_rgb(b"\xff\x00\x00") == "#ff0000"`.
- `compute_dominant_color(path, runner=fake)` returns the hex when enabled.
- The default ingest path does NOT call this (asserted in Task 8, not here).
- [ ] **Step 2: Implement**:
```python
"""ffmpeg helpers: representative frame; OPTIONAL dominant color. Runner injectable."""
from __future__ import annotations
import subprocess
def _run_bytes(args) -> bytes:
return subprocess.run(args, capture_output=True, check=True).stdout
def dominant_color_from_rgb(rgb: bytes) -> str:
r, g, b = rgb[0], rgb[1], rgb[2]
return f"#{r:02x}{g:02x}{b:02x}"
def compute_dominant_color(path, *, midpoint_s=0.0, runner=_run_bytes) -> str:
"""ffmpeg-only single-color palette of a mid-segment frame -> #rrggbb."""
args = ["ffmpeg", "-v", "quiet", "-ss", str(midpoint_s), "-i", str(path),
"-vf", "thumbnail,palettegen=max_colors=1", "-frames:v", "1",
"-f", "rawvideo", "-pix_fmt", "rgb24", "-"]
return dominant_color_from_rgb(runner(args))
def extract_frame(path, dest, *, midpoint_s=0.0, runner=None):
"""Write one representative frame to dest (PNG) for the review preview."""
runner = runner or (lambda a: subprocess.run(a, check=True))
runner(["ffmpeg", "-v", "quiet", "-y", "-ss", str(midpoint_s), "-i", str(path),
"-frames:v", "1", str(dest)])
return dest
```
- [ ] **Step 3:** Run → PASS. Commit `feat: ffmpeg frame extraction + optional dominant_color`.
---
### Task 6: `tools/licensing.py` — origin → license/attribution
**Files:** `tools/licensing.py`, `tests/test_licensing.py`
- [ ] **Step 1: Failing tests** mapping sample origin metadata per archive to the `hef.catalog.LICENSES` vocab:
- CC-BY url/identifier → `("cc_by", "<attribution string>")` (non-empty attribution).
- CC0 / public-domain markers → `("cc0"|"public_domain", "")`.
- LibriVox → always `("public_domain", "")`.
- unmappable → raises a clear error (rejected at ingest).
```python
from tools.licensing import normalize_license
def test_cc_by_requires_attribution():
lic, attr = normalize_license("https://creativecommons.org/licenses/by/4.0/",
creator="Jane Doe")
assert lic == "cc_by" and "Jane Doe" in attr
def test_unmappable_rejected():
import pytest
with pytest.raises(ValueError):
normalize_license("All Rights Reserved")
```
- [ ] **Step 2: Implement** `normalize_license(raw, *, creator="", license_name="")`:
- regex/string-match CC URLs and identifiers → `cc0`/`cc_by`/`cc_by_nc`.
- public-domain / "no known copyright" / "publicdomain" → `public_domain`.
- for `ATTRIBUTION_LICENSES` build `attribution` from `creator` + license name/URL (must be non-empty; `validate()` enforces it downstream).
- anything else → `raise ValueError(f"unmappable license: {raw!r}")`.
- Per-archive helpers may wrap it (e.g. `librivox_license()` returns `("public_domain","")`).
- [ ] **Step 3:** Run → PASS. Commit `feat: per-archive license normalization`.
---
### Task 7: `tools/drafting.py` — heuristic coordinate proposer
**Files:** `tools/drafting.py`, `tests/test_drafting.py`
- [ ] **Step 1: Failing tests** for `HeuristicProposer` (spec §7.1):
- `librivox` archive → high `left`; `nasa`/`musopen`/`fma` → high `right`.
- title/description containing a dark keyword ("storm") raises `dark`; a light keyword ("sunrise") raises `light`.
- all coords clamped to 0..4.
- `rationale` is a non-empty single line citing a signal.
```python
from tools.drafting import Signals, HeuristicProposer
def test_librivox_seeds_left():
d = HeuristicProposer().propose(Signals(title="Meditations", description="",
source_archive="librivox", mode="audio", duration_s=600))
assert d.coordinate.left >= 3 and d.rationale
def test_storm_seeds_dark():
d = HeuristicProposer().propose(Signals(title="Thunderstorm at Night",
description="", source_archive="nasa", mode="video", duration_s=600))
assert d.coordinate.dark >= 2
```
- [ ] **Step 2: Implement** `Signals`, `Draft` (wrapping `hef.selection.Coordinate`), `Proposer` protocol, and `HeuristicProposer` with: archive priors (brain plane), keyword sets for dark/light (mood plane), `max(0, min(4, v))` clamping, one-line rationale naming the dominant signal. Deterministic — no I/O.
- [ ] **Step 3:** Run → PASS. Commit `feat: heuristic coordinate proposer (drafting)`.
---
### Task 8: `tools/ingest/base.py` — Candidate, Fetcher, pipeline
**Files:** `tools/ingest/__init__.py`, `tools/ingest/base.py`, `tests/test_ingest_pipeline.py`
- [ ] **Step 1: Failing tests** for `ingest_candidate` with ALL boundaries faked (no network, no ffprobe, no disk writes beyond tmp_path):
- given a `Candidate` + fake `prober` (returns a Probe) + fake `downloader` (writes a tmp file) + `HeuristicProposer`, it appends ONE `proposed` record whose mechanical fields match the probe, `review_status=="proposed"`, `reviewed_at is None`, `rationale` non-empty, `file_path` relative to media-root.
- re-running with the same `suggested_id` is idempotent (no duplicate; skipped).
- `dominant_color` stays `""` by default; computed only when `compute_color=True`.
- a candidate whose license is unmappable raises before writing (nothing appended).
- [ ] **Step 2: Implement** `Candidate` (spec §6.1), the `Fetcher` Protocol (`archive`, `search`, `resolve`), and:
```python
def ingest_candidate(candidate, *, catalog_path, media_root, proposer,
prober=probe_file, downloader=..., compute_color=False):
# 1. load_catalog + validate_catalog; skip if candidate.suggested_id exists
# 2. download media_url -> <media_root>/<archive>/<id>.<ext> (skip if present)
# 3. tags = derive_tags(prober(file)); color = compute_dominant_color(...) if compute_color and video
# 4. draft = proposer.propose(Signals(...))
# 5. build hef.catalog.Record(...) (proposed shape via defaults)
# 6. validate(record); re-check uniqueness; append_record(record, catalog_path)
```
Plus `ingest_search(fetcher, query, *, limit, **kw)` looping the above over `fetcher.search`.
- [ ] **Step 3:** Run → PASS. Commit `feat: ingest pipeline (Candidate/Fetcher/ingest_candidate)`.
---
### Task 9: First-ship fetchers — LibriVox, NASA, Internet Archive
**Files:** `tools/ingest/librivox.py`, `tools/ingest/nasa.py`, `tools/ingest/internet_archive.py`, `tools/ingest/{musopen,fma,freesound}.py` (deferred stubs), `tests/test_fetchers.py`
- [ ] **Step 1: Failing tests** — one per first-ship fetcher, feeding canned API JSON through `HttpClient(opener=fake)`; assert `Candidate` fields + a stable `suggested_id` derivation and the normalized license:
- **LibriVox** (`librivox.org/api/feed/audiobooks?...&format=json`): `license="public_domain"`, `mode` hint audio, id like `librivox-<slug>`.
- **NASA** (`images-api.nasa.gov/search?q=...`): `license="public_domain"`, id like `nasa-<nasa_id>`, media_url from the asset collection.
- **Internet Archive** (`archive.org/metadata/<id>`): license from `licenseurl`/`rights` via `tools.licensing`; ambiguous "no known copyright" → `public_domain` with a `notes` flag; id like `ia-<identifier>`.
- [ ] **Step 2: Implement** the three fetchers against the documented JSON APIs, each taking an injected `HttpClient`, each exposing `archive`, `search(query, limit)`, `resolve(identifier)`. Use `tools.licensing` for license/attribution.
- [ ] **Step 3: Deferred stubs.** `musopen.py`, `fma.py`, `freesound.py` define the class with the `Fetcher` shape but `raise NotImplementedError("deferred — see spec §6.4")`; `freesound.py` documents the `FREESOUND_API_TOKEN` env requirement (secret; never logged). A test asserts they raise `NotImplementedError` (so the seam is wired and the deferral is explicit, not forgotten).
- [ ] **Step 4:** Run → PASS. Commit `feat: LibriVox/NASA/Internet Archive fetchers (+ deferred stubs)`.
---
### Task 10: `tools/ingest_cli.py`
**Files:** `tools/ingest_cli.py` (light/manual test)
- [ ] **Step 1:** `argparse` entry: `python -m tools.ingest_cli <archive> --query "..." [--limit N] [--catalog catalog/library.jsonl] [--media-root ./media] [--dominant-color] [--resolve <identifier>]`. Wires the named fetcher + `HeuristicProposer` into `ingest_search`/`ingest_candidate`. `main()` returns an exit code; secrets read from env only.
- [ ] **Step 2:** A small test that `--help` parses and that an unknown archive errors cleanly (no network). Real fetches are manual/integration.
- [ ] **Step 3:** Commit `feat: ingest CLI entry point`.
---
### Task 11: `tools/review.py` — the transition core
**Files:** `tools/review.py`, `tests/test_review.py`
- [ ] **Step 1: Failing tests** (spec §8.1):
```python
from dataclasses import replace
from tools.review import proposed_records, approve
from hef.selection import Coordinate
# make_record helper as elsewhere
def test_proposed_records_filters():
a = make_record(id="a", review_status="proposed")
b = make_record(id="b", review_status="approved")
assert [r.id for r in proposed_records([a, b])] == ["a"]
def test_approve_sets_status_and_timestamp():
r = make_record(review_status="proposed")
out = approve(r, reviewed_at="2026-06-04T13:00:00+00:00")
assert out.review_status == "approved" and out.reviewed_at == "2026-06-04T13:00:00+00:00"
def test_approve_can_override_coordinates():
r = make_record(left=0, right=0, dark=0, light=0, review_status="proposed")
out = approve(r, reviewed_at="t", coordinate=Coordinate(4, 1, 2, 3))
assert (out.left, out.right, out.dark, out.light) == (4, 1, 2, 3)
```
- [ ] **Step 2: Implement** `proposed_records(records)` and `approve(record, *, reviewed_at, coordinate=None)` using `dataclasses.replace` to return an approved copy (no in-place surprise); caller re-validates. Pure, no I/O, no clock (timestamp injected).
- [ ] **Step 3:** Run → PASS. Commit `feat: review transition core (proposed -> approved)`.
---
### Task 12: `tools/review_cli.py` — interactive walk
**Files:** `tools/review_cli.py` (manual; preview reuses `tools.mediatools`)
- [ ] **Step 1:** Implement the walk (spec §8.2): `load_catalog``validate_catalog`; for each `proposed` record print id/title/source/license/mode/duration/resolution + proposed coords + rationale; render a preview (video/av → `extract_frame` then `open`/`xdg-open`; audio → `showwavespic` thumbnail and/or optional `ffplay`); prompt `[a]ccept/[e]dit/[s]kip/[q]uit`; on accept/edit call `approve(...)` with `datetime.now(timezone.utc).isoformat()` and persist via `save_catalog` (rewrite) after each approval. `--catalog`, `--media-root` args.
- [ ] **Step 2:** Keep the shell thin — all decision logic already tested in Task 11. A tiny test that `--help` parses; the interactive loop is manual.
- [ ] **Step 3:** Commit `feat: interactive review CLI`.
---
### Task 13: End-to-end mocked integration + opt-in real-ffprobe
**Files:** `tests/test_tools_integration.py`
- [ ] **Step 1: Hermetic end-to-end** (spec §10 test 8): fake fetcher → `Candidate`; fake downloader + fake prober + `HeuristicProposer``ingest_candidate` appends a `proposed` record; `proposed_records` + `approve` flips it; `load_catalog` + `validate_catalog` pass; `select(loaded, coord, mode, approved_only=True)` returns it.
- [ ] **Step 2: Opt-in real-ffprobe test** — guarded by `shutil.which("ffprobe")` (`pytest.mark.skipif` when absent): generate a 1-second test clip with `ffmpeg lavfi` into `tmp_path`, probe it, assert `mode`/`duration_s`/`resolution`. Same pattern for a real `compute_dominant_color`.
- [ ] **Step 3:** Run the FULL suite `python -m pytest -q` → green. Commit `test: end-to-end ingest+review integration (+ opt-in ffprobe)`.
---
### Task 14: Docs — extend the User Guide for the tools
**Files:** `docs/USER_GUIDE.md`
- [ ] **Step 1:** Add an "Ingesting & reviewing media" section: prerequisites (`ffmpeg`/`ffprobe`), `--media-root`, the first-ship archives, `python -m tools.ingest_cli ...` examples, the review walk, the Freesound token note (env only), and that `dominant_color` is optional/opt-in. Update the scope banner (no longer "only catalog core is built"). Reconcile the absolute-vs-relative `file_path` note (spec §6.3).
- [ ] **Step 2:** Commit `docs: user guide for ingest + review tools`.
---
## Done criteria
- `python -m pytest -q` is green from the repo root inside `.venv` (prior 37 + all new unit tests; opt-in ffprobe tests skip cleanly when binaries are absent).
- `hef.catalog` gains exactly `validate_catalog` and `index_by_id`; every pre-existing symbol/behavior is unchanged.
- `python -m tools.ingest_cli <archive> --query ...` against LibriVox / NASA / Internet Archive produces `proposed` records with mechanical fields filled and a one-line rationale.
- `python -m tools.review_cli` walks the `proposed` records (coords + rationale + preview frame) and flips accepted ones to `approved` with a `reviewed_at` stamp; `load_catalog` + `validate_catalog` validate the result.
- `dominant_color` is computed only with `--dominant-color`; Freesound/Musopen/FMA fetchers are explicit deferred stubs (raise `NotImplementedError`).
This populated, human-reviewed catalog is what sub-project 3 (the Pi player) drives to the single panoramic projector.
@@ -0,0 +1,867 @@
# Player Alteration Core Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the pure-logic core of sub-project 3 (the Player Runtime + alteration engine) — a new `player/` package that turns control-panel state into a render plan and playback transitions, fully unit-tested with all I/O (video/audio/serial) behind injected interfaces.
**Architecture:** Four small, pure modules mirroring `hef/` conventions (frozen dataclasses, `from __future__ import annotations`, string-constant frozensets, thorough unit tests, no heavy deps). `controls` models the panel state (the serial-contract data shape shared with sub-project 4); `content` resolves the 7-way content dial; `alteration` is the heart — knob vector → `RenderPlan` per design §4/§5; `state` is the player state machine that diffs successive controls into playback transitions (crossfade / fade-to-black / live-update). No mpv, no GPU, no real serial, no white-noise DSP, no v2v pipeline — those are later integration slices.
**Tech Stack:** Python 3.11+, stdlib only (`dataclasses`, `math`), pytest. Reuses `hef.selection.Coordinate` for the knob vector.
**Design reference:** `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` (§4 alteration engine, §5 mood grade, §6 7-way content dial, §7 intensity) and `docs/ROADMAP.md` §3 (Player Runtime "done when").
**Calibration decision (flagged for operator):** brain knobs use `strength = value/4` (0=off, 4=max); mood uses `tone = (lightdark)/4` (equal dark/light = identity per §5). This reads §3's "(2,2,2,2) neutral" as a vestige of the old coordinate-grid center. The calibration lives in three one-line helpers so the convention can be flipped trivially. See the session 0006 transcript's Deferred decisions.
---
## File Structure
- `player/__init__.py` — package marker (empty).
- `player/controls.py``Controls` frozen dataclass (content position + 4 experience knobs + volume + brightness), `CONTENT_POSITIONS`, `validate_controls`, `parse_controls` (from a decoded mapping; wire framing is the separate 3⇄4 serial contract).
- `player/content.py``ContentResolution` (audio source + video flag), `AUDIO_SOURCES`, `resolve_content` (the §6 7-row table, single source of truth).
- `player/alteration.py``ColorGrade`, `AnalyticalOverlay`, `Restyle`, `RenderPlan`, `plan_alteration(coord)`, and the calibration helpers `_overlay_intensity`, `_restyle_blend`, `_mood_tone`.
- `player/state.py``Playback`, `TransitionKind`, `Transition`, `Player` (consumes a stream of `Controls`, emits a `Transition` per update; output sink is injected/duck-typed; base-clip choice is an injected callable).
- `pyproject.toml` — add `"player"` to `[tool.setuptools] packages`.
Tests (one per module): `tests/test_player_controls.py`, `tests/test_player_content.py`, `tests/test_player_alteration.py`, `tests/test_player_state.py`.
---
## Task 1: Controls model (the serial-contract data shape)
**Files:**
- Create: `player/__init__.py`
- Create: `player/controls.py`
- Test: `tests/test_player_controls.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/test_player_controls.py
import pytest
from player.controls import (
Controls,
CONTENT_POSITIONS,
ControlsError,
validate_controls,
parse_controls,
)
def test_content_positions_are_the_seven_from_the_spec():
assert CONTENT_POSITIONS == frozenset(
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
)
def test_valid_controls_pass_validation():
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
validate_controls(c) # does not raise
def test_invalid_content_position_rejected():
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
with pytest.raises(ControlsError):
validate_controls(c)
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
@pytest.mark.parametrize("bad", [-1, 5, True])
def test_out_of_range_or_non_int_knob_rejected(field, bad):
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
kwargs[field] = bad
with pytest.raises(ControlsError):
validate_controls(Controls(**kwargs))
def test_parse_controls_from_mapping():
c = parse_controls(
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
)
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
def test_parse_controls_rejects_unknown_keys():
with pytest.raises(ControlsError):
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
def test_parse_controls_rejects_missing_keys():
with pytest.raises(ControlsError):
parse_controls({"content": "off", "left": 0})
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `pytest tests/test_player_controls.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'player'`.
- [ ] **Step 3: Write minimal implementation**
```python
# player/__init__.py
```
(empty file)
```python
# player/controls.py
"""Control-panel state: the data shape read from the Arduino over serial.
This is the serial-contract payload shared with sub-project 4 (firmware). It
models the full panel from design §6/§7: the 7-way content dial, the four
experience knobs (0..4), and the two intensity levels (volume, brightness).
The wire framing itself is the separate 3<->4 serial contract; this module is
the *decoded* form.
"""
from __future__ import annotations
from dataclasses import dataclass, fields
# The seven positions of the content dial (design §6).
CONTENT_POSITIONS = frozenset(
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
)
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
KNOB_MIN = 0
KNOB_MAX = 4
class ControlsError(ValueError):
"""Raised when a Controls payload is structurally invalid."""
@dataclass(frozen=True)
class Controls:
content: str
left: int
right: int
dark: int
light: int
volume: int
brightness: int
def validate_controls(c: Controls) -> None:
"""Raise ControlsError if the payload is structurally invalid."""
if c.content not in CONTENT_POSITIONS:
raise ControlsError(
f"invalid content position {c.content!r}; "
f"expected one of {sorted(CONTENT_POSITIONS)}"
)
for name in KNOB_FIELDS:
value = getattr(c, name)
if isinstance(value, bool) or not isinstance(value, int):
raise ControlsError(f"knob {name} must be an int, got {value!r}")
if not (KNOB_MIN <= value <= KNOB_MAX):
raise ControlsError(
f"knob {name}={value} out of range {KNOB_MIN}..{KNOB_MAX}"
)
def parse_controls(data: dict) -> Controls:
"""Build a validated Controls from a decoded mapping, rejecting unknown or
missing keys."""
known = {f.name for f in fields(Controls)}
unknown = set(data) - known
if unknown:
raise ControlsError(f"unknown keys: {sorted(unknown)}")
missing = known - set(data)
if missing:
raise ControlsError(f"missing keys: {sorted(missing)}")
c = Controls(**data)
validate_controls(c)
return c
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `pytest tests/test_player_controls.py -v`
Expected: PASS (all cases).
- [ ] **Step 5: Commit**
```bash
git add player/__init__.py player/controls.py tests/test_player_controls.py
git commit -m "feat(player): Controls panel model (serial-contract data shape)
Sub-project 3 slice 1, per ROADMAP.md §3 and design §6/§7."
```
---
## Task 2: Content-dial resolution (§6)
**Files:**
- Create: `player/content.py`
- Test: `tests/test_player_content.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/test_player_content.py
import pytest
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
def test_audio_sources_are_the_four_distinct_sources():
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
# The §6 table, row by row: position -> (audio_source, video).
@pytest.mark.parametrize(
"position,audio_source,video",
[
("off", "none", False),
("white_noise", "white_noise", False),
("music", "music", False),
("audio_track", "audio_track", False),
("video", "none", True),
("music_video", "music", True),
("audio_video", "audio_track", True),
],
)
def test_resolve_content_matches_spec_table(position, audio_source, video):
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
def test_off_is_void_state_black_and_silent():
r = resolve_content("off")
assert r.video is False and r.audio_source == "none"
def test_resolve_content_rejects_unknown_position():
with pytest.raises(ValueError):
resolve_content("bogus")
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `pytest tests/test_player_content.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'player.content'`.
- [ ] **Step 3: Write minimal implementation**
```python
# player/content.py
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
The single source of truth for design §6's table: which audio source plays and
whether the projector shows video, for each dial position.
"""
from __future__ import annotations
from dataclasses import dataclass
# The four distinct audio sources. "white_noise" is generated at runtime;
# "music" is the public-domain classical pool; "audio_track" is the clip's own
# field audio; "none" is silence.
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
@dataclass(frozen=True)
class ContentResolution:
audio_source: str
video: bool
# Design §6, one row per dial position.
_TABLE = {
"off": ContentResolution("none", False),
"white_noise": ContentResolution("white_noise", False),
"music": ContentResolution("music", False),
"audio_track": ContentResolution("audio_track", False),
"video": ContentResolution("none", True),
"music_video": ContentResolution("music", True),
"audio_video": ContentResolution("audio_track", True),
}
def resolve_content(position: str) -> ContentResolution:
"""Map a content-dial position to its audio source and video flag (§6)."""
try:
return _TABLE[position]
except KeyError:
raise ValueError(
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
) from None
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `pytest tests/test_player_content.py -v`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add player/content.py tests/test_player_content.py
git commit -m "feat(player): 7-way content-dial resolution (design §6)"
```
---
## Task 3: Alteration engine — knob vector to RenderPlan (§4, §5)
**Files:**
- Create: `player/alteration.py`
- Test: `tests/test_player_alteration.py`
The engine maps a knob vector `(left, right, dark, light)` to a layered `RenderPlan` per design §4.2:
- **Substrate** transforms (alter pixels): `Restyle` (Right, v2v) blended with `ColorGrade` (mood, Dark/Light).
- **Overlay** on top: `AnalyticalOverlay` (Left, HUD/labels).
`ColorGrade.tone` is signed: `>0` warm yellow→white (light pole), `<0` cool blue→black (dark pole), `0` identity/raw (§5). Calibration (flagged decision): `overlay = left/4`, `restyle = right/4`, `tone = (light dark)/4`.
- [ ] **Step 1: Write the failing tests**
```python
# tests/test_player_alteration.py
import pytest
from hef.selection import Coordinate
from player.alteration import (
AnalyticalOverlay,
ColorGrade,
RenderPlan,
Restyle,
plan_alteration,
)
def _coord(left=0, right=0, dark=0, light=0):
return Coordinate(left=left, right=right, dark=dark, light=light)
def test_all_zero_knobs_is_the_unaltered_base():
plan = plan_alteration(_coord())
assert plan.is_identity
assert plan.overlay.intensity == 0.0
assert plan.restyle.blend == 0.0
assert plan.grade.tone == 0.0
assert plan.grade.is_identity
def test_left_drives_the_analytical_overlay_only():
plan = plan_alteration(_coord(left=4))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 0.0 # Left does not touch the substrate
assert plan.grade.tone == 0.0
def test_right_drives_the_restyle_substrate_only():
plan = plan_alteration(_coord(right=2))
assert plan.restyle.blend == 0.5
assert plan.overlay.intensity == 0.0 # Right does not add overlay
def test_left_and_right_stack_not_cancel():
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
plan = plan_alteration(_coord(left=4, right=4))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 1.0
def test_light_pole_grades_warm_positive_tone():
plan = plan_alteration(_coord(light=4))
assert plan.grade.tone == 1.0
assert not plan.grade.is_identity
def test_dark_pole_grades_cool_negative_tone():
plan = plan_alteration(_coord(dark=4))
assert plan.grade.tone == -1.0
def test_equal_dark_and_light_is_identity_grade():
# design §5: the mood center is the raw, ungraded footage
assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity
assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity
def test_dark_minus_light_sets_intermediate_tone():
assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5)
assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5)
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
# design §4.2 "Dark + analytical": cold measurement over a melancholy scene
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 0.5
assert plan.grade.tone == -1.0
assert not plan.is_identity
def test_render_plan_is_frozen():
plan = plan_alteration(_coord())
with pytest.raises(Exception):
plan.grade.tone = 0.5 # type: ignore[misc]
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `pytest tests/test_player_alteration.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'player.alteration'`.
- [ ] **Step 3: Write minimal implementation**
```python
# player/alteration.py
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
Replaces the 2026-06-04 nearest-match *selection* with a *transformation* of a
neutral base clip. Given the four experience knobs, it produces three layers
that compose per §4.2:
- Substrate transforms (alter pixels, blend with each other):
* Restyle -- the Right axis: a pre-baked generative v2v dreamlike restyle.
* ColorGrade -- the mood axis (Dark/Light): a deterministic color grade.
- Overlay (composited on top of the substrate):
* AnalyticalOverlay -- the Left axis: HUD/labels/measurement.
Left and Right are NOT opposites; they live on different layers and stack
(§4.2). Dark and Light are the two poles of one mood grade whose center is the
identity (§5).
Calibration note: the knob->strength curves below are the single source of
truth for how a 0..4 position maps to a transform strength. See the session
0006 transcript Deferred decisions for the §3-vs-§4.2/§5 reconciliation.
"""
from __future__ import annotations
from dataclasses import dataclass
from hef.selection import Coordinate
KNOB_MAX = 4 # knob full-scale (0..4)
@dataclass(frozen=True)
class ColorGrade:
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
<0 cool blue->black (dark), 0 = identity (raw, ungraded)."""
tone: float
@property
def is_identity(self) -> bool:
return self.tone == 0.0
@dataclass(frozen=True)
class AnalyticalOverlay:
"""Left axis (§4.1): analytical HUD/annotation/labels, composited on top.
`intensity` 0..1 (0 = no overlay)."""
intensity: float
@dataclass(frozen=True)
class Restyle:
"""Right axis (§4.1): pre-baked generative v2v dreamlike substrate.
`blend` 0..1 (0 = raw substrate, no restyle)."""
blend: float
@dataclass(frozen=True)
class RenderPlan:
"""The full layered alteration for one knob vector (§4.2)."""
grade: ColorGrade
overlay: AnalyticalOverlay
restyle: Restyle
@property
def is_identity(self) -> bool:
"""True when the plan leaves the neutral base un-altered."""
return (
self.grade.is_identity
and self.overlay.intensity == 0.0
and self.restyle.blend == 0.0
)
def _overlay_intensity(left: int) -> float:
"""Left knob -> analytical-overlay intensity (0..1)."""
return left / KNOB_MAX
def _restyle_blend(right: int) -> float:
"""Right knob -> v2v restyle blend (0..1)."""
return right / KNOB_MAX
def _mood_tone(dark: int, light: int) -> float:
"""(dark, light) -> signed mood grade in [-1, 1]; equal -> 0 identity (§5)."""
return (light - dark) / KNOB_MAX
def plan_alteration(coord: Coordinate) -> RenderPlan:
"""Map a knob vector to its layered RenderPlan (design §4)."""
return RenderPlan(
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light)),
overlay=AnalyticalOverlay(intensity=_overlay_intensity(coord.left)),
restyle=Restyle(blend=_restyle_blend(coord.right)),
)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `pytest tests/test_player_alteration.py -v`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add player/alteration.py tests/test_player_alteration.py
git commit -m "feat(player): alteration engine — knob vector to RenderPlan (design §4/§5)"
```
---
## Task 4: Player state machine — controls stream to transitions
**Files:**
- Create: `player/state.py`
- Test: `tests/test_player_state.py`
The `Player` holds a base-clip library and the current `Playback`. Each `update(controls)` resolves the desired `Playback` (chosen base clip + RenderPlan + content resolution + volume/brightness) and returns the `Transition` from the previous playback:
- target video off, was on → `FADE_TO_BLACK`
- target video on, was off → `FADE_FROM_BLACK`
- both video on, clip or restyle variant changed → `CROSSFADE`
- both video on, only grade/overlay/level changed → `LIVE_UPDATE` (§4.3: grade + overlay are continuous runtime ops)
- both video off, audio/level changed → `LIVE_UPDATE`
- nothing changed → `NONE`
Base-clip choice is an injected callable (default: first clip); knobs no longer *select* (the base is neutral) — they *transform*. The library is duck-typed objects with an `.id`.
- [ ] **Step 1: Write the failing tests**
```python
# tests/test_player_state.py
from dataclasses import dataclass
import pytest
from player.controls import Controls
from player.state import Player, Playback, Transition, TransitionKind
@dataclass(frozen=True)
class FakeClip:
id: str
LIB = [FakeClip("base-a"), FakeClip("base-b")]
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
return Controls(content, left, right, dark, light, volume, brightness)
def test_first_update_to_video_fades_in_from_black():
p = Player(LIB)
t = p.update(_controls(content="video"))
assert t.kind == TransitionKind.FADE_FROM_BLACK
assert t.playback.clip_id == "base-a"
assert t.playback.content.video is True
def test_off_from_video_fades_to_black_and_silences():
p = Player(LIB)
p.update(_controls(content="video"))
t = p.update(_controls(content="off"))
assert t.kind == TransitionKind.FADE_TO_BLACK
assert t.playback.clip_id is None
assert t.playback.content.audio_source == "none"
def test_no_change_yields_none_transition():
p = Player(LIB)
p.update(_controls(content="video", left=1))
t = p.update(_controls(content="video", left=1))
assert t.kind == TransitionKind.NONE
def test_grade_change_is_a_live_update_not_a_crossfade():
# design §4.3: the Dark/Light grade is a continuous runtime op
p = Player(LIB)
p.update(_controls(content="video", dark=0, light=0))
t = p.update(_controls(content="video", dark=4, light=0))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.grade.tone == -1.0
def test_overlay_change_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="video", left=0))
t = p.update(_controls(content="video", left=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.overlay.intensity == 1.0
def test_restyle_change_crossfades_the_substrate():
# design §4.3: the Right v2v substrate is a pre-baked variant swap
p = Player(LIB)
p.update(_controls(content="video", right=0))
t = p.update(_controls(content="video", right=4))
assert t.kind == TransitionKind.CROSSFADE
assert t.playback.plan.restyle.blend == 1.0
def test_volume_only_change_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="video", volume=1))
t = p.update(_controls(content="video", volume=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.volume == 4
def test_audio_source_change_while_black_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="white_noise"))
t = p.update(_controls(content="music"))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.content.audio_source == "music"
def test_injected_base_chooser_is_used():
p = Player(LIB, choose_base=lambda lib: lib[1])
t = p.update(_controls(content="video"))
assert t.playback.clip_id == "base-b"
def test_empty_library_with_video_raises():
p = Player([])
with pytest.raises(ValueError):
p.update(_controls(content="video"))
def test_off_with_empty_library_is_fine():
p = Player([])
t = p.update(_controls(content="off"))
assert t.kind == TransitionKind.NONE # already black at init
assert t.playback.clip_id is None
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `pytest tests/test_player_state.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'player.state'`.
- [ ] **Step 3: Write minimal implementation**
```python
# player/state.py
"""The player state machine: a stream of Controls -> playback transitions.
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
desired Playback (which neutral base clip, how it is altered, what audio plays,
at what levels) and returns the Transition from the previous Playback. The
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
pre-baked Right v2v variant needs a CROSSFADE, and toggling video on/off
fades to/from black.
Knobs no longer *select* a clip (the base library is neutral by construction);
they *transform* it. Which neutral base to show is an injected policy
(`choose_base`), defaulting to the first clip; richer rotation is a later slice.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Optional
from hef.selection import Coordinate
from player.alteration import RenderPlan, plan_alteration
from player.content import ContentResolution, resolve_content
from player.controls import Controls
class TransitionKind:
NONE = "none"
LIVE_UPDATE = "live_update"
CROSSFADE = "crossfade"
FADE_TO_BLACK = "fade_to_black"
FADE_FROM_BLACK = "fade_from_black"
@dataclass(frozen=True)
class Playback:
"""What should currently be playing."""
clip_id: Optional[str] # None = black walls
plan: Optional[RenderPlan] # None when black
content: ContentResolution
volume: int
brightness: int
@dataclass(frozen=True)
class Transition:
kind: str
playback: Playback
def _first(library):
if not library:
raise ValueError("no base clips available to play video")
return library[0]
_BLACK = Playback(
clip_id=None,
plan=None,
content=ContentResolution("none", False),
volume=0,
brightness=0,
)
class Player:
def __init__(
self,
base_library,
*,
choose_base: Callable = _first,
approved_only: bool = False,
):
self._library = list(base_library)
self._choose_base = choose_base
self._approved_only = approved_only # reserved for catalog-backed libs
self._current = _BLACK
def _resolve(self, controls: Controls) -> Playback:
content = resolve_content(controls.content)
if not content.video:
return Playback(
clip_id=None,
plan=None,
content=content,
volume=controls.volume,
brightness=controls.brightness,
)
clip = self._choose_base(self._library)
coord = Coordinate(controls.left, controls.right, controls.dark, controls.light)
return Playback(
clip_id=clip.id,
plan=plan_alteration(coord),
content=content,
volume=controls.volume,
brightness=controls.brightness,
)
def _classify(self, prev: Playback, nxt: Playback) -> str:
prev_video = prev.clip_id is not None
next_video = nxt.clip_id is not None
if prev == nxt:
return TransitionKind.NONE
if prev_video and not next_video:
return TransitionKind.FADE_TO_BLACK
if next_video and not prev_video:
return TransitionKind.FADE_FROM_BLACK
if next_video and prev_video:
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
return TransitionKind.CROSSFADE
return TransitionKind.LIVE_UPDATE
# both black: only audio/levels could have changed
return TransitionKind.LIVE_UPDATE
def update(self, controls: Controls) -> Transition:
nxt = self._resolve(controls)
kind = self._classify(self._current, nxt)
self._current = nxt
return Transition(kind=kind, playback=nxt)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `pytest tests/test_player_state.py -v`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add player/state.py tests/test_player_state.py
git commit -m "feat(player): player state machine — controls stream to transitions"
```
---
## Task 5: Register the package and run the full suite
**Files:**
- Modify: `pyproject.toml` (`[tool.setuptools] packages`)
- [ ] **Step 1: Add `"player"` to the packages list**
In `pyproject.toml`, change:
```toml
[tool.setuptools]
packages = ["hef", "tools", "simulator"]
```
to:
```toml
[tool.setuptools]
packages = ["hef", "tools", "simulator", "player"]
```
- [ ] **Step 2: Run the entire test suite**
Run: `pytest -q`
Expected: all prior tests still pass plus the new `test_player_*` files; no failures.
- [ ] **Step 3: Commit**
```bash
git add pyproject.toml
git commit -m "build(player): register the player package"
```
---
## Task 6: Update the roadmap to reflect slice 1 shipped
**Files:**
- Modify: `docs/ROADMAP.md` (§3 Player Runtime)
- [ ] **Step 1: Update §3 status and deliverables**
Under "## 3. Player Runtime (Pi)", note that the **alteration-engine + player-core slice** (pure logic) is done and tested, link this plan, and list the remaining slices (mpv/ffmpeg runtime integration + GPU grade/overlay shaders; real USB-serial reader + the 3⇄4 framing contract; white-noise DSP; offline v2v variant build pipeline; catalog model changes for audio-source + neutral-base flag). Keep the table's status marker for sub-project 3 as in-progress (⏳), not done.
- [ ] **Step 2: Commit**
```bash
git add docs/ROADMAP.md
git commit -m "docs(roadmap): sub-project 3 alteration-engine + player-core slice shipped"
```
---
## Self-Review
**Spec coverage:**
- §4.1 per-axis alteration → Task 3 (overlay=Left, restyle=Right, grade=mood). ✓
- §4.2 composition rule (substrate vs overlay; Left+Right stack) → Task 3 tests `test_left_and_right_stack_not_cancel`, `test_whole_brain_dark_corner_*`. ✓
- §4.3 runtime vs pre-baked (grade/overlay continuous; restyle variant swap) → Task 4 transition kinds (`LIVE_UPDATE` vs `CROSSFADE`). ✓
- §5 mood grade, center = identity → Task 3 `test_equal_dark_and_light_is_identity_grade`, signed tone. ✓
- §6 7-way content dial → Task 2 full table. ✓
- §7 volume + brightness intensity → Task 1 (modeled) + Task 4 (`test_volume_only_change_is_a_live_update`). ✓
- Roadmap §3 "done when: plays correct segment, loops, crossfades, goes dark on None, testable with serial mocked" → Task 4 (transitions incl. fade-to-black; serial mocked as a `Controls` stream). Looping is implicit (a clip plays until the desired playback changes); explicit loop timing belongs to the mpv integration slice (deferred, noted in Task 6). ✓ (with the runtime-loop portion deferred and documented).
**Deferred (documented, not gaps):** mpv/ffmpeg + GPU; real serial + framing; white-noise DSP; v2v offline pipeline; catalog audio-source/neutral-base fields; base-clip rotation policy; `approved_only` enforcement against a real catalog (the flag is plumbed but inert here).
**Placeholder scan:** none — every step has concrete code/commands.
**Type consistency:** `Controls(content,left,right,dark,light,volume,brightness)` used identically in Tasks 1 & 4; `RenderPlan.{grade,overlay,restyle}` and `.is_identity` consistent across Tasks 3 & 4; `ContentResolution(audio_source,video)` consistent across Tasks 2 & 4; `TransitionKind` constants consistent. `Coordinate(left,right,dark,light)` matches `hef/selection.py`.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,193 @@
# Experience Simulator (Curator's X-ray) — Design Spec
**Date:** 2026-06-04
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
**Session:** 0003 (discovery)
> A web-based stand-in for the installation's control panel, run on localhost in a
> Docker container, built so a curator can **feel the selection model** — turn the
> dials and see, in full, which catalog pieces the nearest-match algorithm
> surfaces and why. It validates the *principles* (coordinate tagging + selection)
> and the *interface* before the physical Pi-player / Arduino stack is built.
---
## 1. What this is
A single-page web tool that reproduces the installation's five controls and wires
them to the **real** `hef.selection` code, then shows a transparent "X-ray" of the
result: the picked piece, the full ranked candidate pool with distances, and a
live map of where the knob point and candidates sit in the coordinate space.
It is a **curator's instrument**, not the installation itself. Its job is to let a
human judge whether the tagging rubric (design spec §2) and the nearest-match
selection (design spec §3) actually surface fitting pieces at every knob position —
so a "wrong" result can be diagnosed as bad tagging vs. a sparse neighborhood.
Primary purpose (settled this session): **feel the selection model.** Faithful
media playback is explicitly *not* a goal.
### Design constraints
- **Real algorithm, single source of truth.** The simulator runs the shipped
`hef.selection` code. No reimplementation of selection in JavaScript — a copy
could drift from what the Pi will run, which would defeat the purpose.
- **Runs locally in Docker.** One container, one command, `localhost`. No
networking beyond the local browser, no auth, no multi-user.
- **Reads the model; never curates into it.** The simulator does not write catalog
records back. Editing/approval stays with the sub-project-2 review tools.
---
## 2. Architecture
One Python service in one container. **FastAPI** serves both a JSON selection API
and a dependency-free static frontend. The browser holds the dial state; each
change POSTs to the API and re-renders the X-ray.
```
browser (dials + X-ray) ──POST /api/select──▶ FastAPI ──▶ hef.selection (REAL)
▲ │
└────────── pool + distances + pick ◀─────┘
```
- **Backend:** `simulator/app.py` (FastAPI), `simulator/fixtures.py` (synthetic
catalog generator). Imports `hef.catalog` and `hef.selection` directly.
- **Frontend:** `simulator/static/``index.html`, `app.js`, `style.css`. Vanilla
JS, no build step.
- **Packaging:** `simulator/Dockerfile` + a one-command run (`docker compose up`
or a `make sim` target), serving on `localhost`.
This adds a `simulator/` top-level package, consistent with the repo layout in the
main design spec §10 (`hef/`, `player/`, `firmware/`, `tools/`).
---
## 3. One additive change to `hef.selection`
The X-ray needs the ranked pool with distances — which `select()` already computes
internally (`ranked`, `nearest`) but does not expose. To avoid duplicating ranking
logic in the simulator (the very drift risk §1 forbids), extract it into a public
function and refactor `select()` to call it:
```python
def ranked_candidates(
records, coord: Coordinate, mode: str, *,
pool_size: int = 4, weights: Weights = Weights(), approved_only: bool = False,
) -> list[tuple[Record, float]]:
"""Mode-eligible records sorted nearest-first, each paired with its distance.
Honors the same 'av' fallback and approved_only filtering as select().
"""
```
`select()` becomes a thin wrapper over `ranked_candidates()` (take the pool, then
return `nearest[0]` or `rng.choice(nearest)`). This is the **only** change to
shipped code: additive, behavior-preserving for `select()`, and independently
tested. The X-ray then displays exactly the pool the picker chooses from.
---
## 4. Synthetic fixture catalog (`simulator/fixtures.py`)
The real `catalog/library.jsonl` is empty. A deterministic (seeded) generator
produces a dense, representative spread of `Record`s so nearest-match behavior can
be felt everywhere immediately:
- Coverage across the `(left, right, dark, light)` space (the 5×5 brain plane ×
5×5 mood plane = 625 configs) and across all content modes (`audio`, `video`,
`av`).
- Plausible titles and `rationale`s; `source_archive`/`license` drawn from the
real pools so records validate.
- A deliberate mix of `review_status: proposed` and `approved`, so the
`approved_only` toggle is exercisable.
- No real media: `file_path` empty, `dominant_color` empty (optional field).
- Output validates against `hef.catalog.validate_catalog`.
Loaded at startup. A flag (`--catalog` / env var) lets the simulator point at
`catalog/library.jsonl` instead when it is non-empty — **same loader**, so swapping
in the real catalog later is free.
---
## 5. The API
### `POST /api/select`
Request:
```json
{ "left":0-4, "right":0-4, "dark":0-4, "light":0-4,
"mode":"none|audio|video|av",
"pool_size":int, "weights":{"brain":float,"mood":float},
"approved_only":bool }
```
Response:
```json
{ "pick": <record|null>,
"pool": [ { "record": <record>, "distance": float, "rank": int }, ... ],
"coverage": { "candidates_in_mode": int } }
```
- `none` selector mode → `pick:null`, `pool:[]` (the void/rest state).
- No candidates → `pick:null`, `pool:[]`.
- **Deterministic pick.** The displayed `pick` is the rank-1 (nearest) candidate —
`select()` called with `rng=None` — so the X-ray is legible and stable for a
given knob position. The room *shuffles* within the nearest pool at runtime;
the returned `pool` IS that shuffle-set, so the curator sees every piece the
real installation could surface there.
- Knob values are constrained 04 and `mode` to the four values by Pydantic; bad
input is a clean `422`.
- Backed by `hef.selection.ranked_candidates()` (pool) + `select()` (the pick).
### `GET /api/catalog/meta`
Counts, per-mode breakdown, `proposed`/`approved` split, and coverage stats for
the header.
---
## 6. The X-ray UI
**Controls**
- The five real dials: a 4-way mode selector (None/Audio/Video/A+V) and four 04
knobs (Left, Right, Dark, Light).
- The model knobs (so the algorithm's own parameters can be felt): brain/mood
**weight sliders**, **pool-size** control, and an **approved-only** toggle.
**Readout**
- **Picked piece:** title, mode, coordinate, distance, rationale.
- **Ranked pool:** the N nearest with distances, winner highlighted.
- **Two 5×5 grid maps** (brain L×R, mood D×L): the current knob point plus where
candidates sit, so the neighborhood that produced the pick is visible at a
glance.
---
## 7. Testing
- **Unit (`hef`):** `ranked_candidates()` ordering, the `av` pool fallback,
`approved_only` filtering, weight effects; `select()` still behaves as before.
- **Fixtures:** coverage test (every mode populated; space spanned; output passes
`validate_catalog`).
- **API:** FastAPI `TestClient` contract tests — valid select returns ranked pool;
`none` returns the void; bad input → 422; `approved_only` narrows the pool.
- **Behavior:** the felt behaviors are captured as gherkin scenarios under
`features/` (this session's BDD deliverable).
---
## 8. Explicitly out of scope (YAGNI)
- Real media playback (audio/video) — the tool reads the model, not the content.
- Writing/editing catalog records back — curation stays in the sub-project-2 tools.
- Networking, auth, multi-user, session recording.
- Any reimplementation of selection logic outside `hef`.
---
## 9. Relationship to the roadmap
This is an **upstream discovery vehicle** for the physical stack (sub-projects 3
and 4). It exercises sub-project 1's `hef.selection` against a stand-in catalog and
is the richer, experienceable form of the "keyboard/serial stand-in" the roadmap
anticipates for sub-project 3. It does not replace the Pi player or the firmware;
it de-risks their *principles and interface* first. The synthetic catalog is a
stand-in for the sub-project-2 output and is swapped for the real catalog through
the same loader once ingest has populated it.
@@ -4,15 +4,26 @@
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
> **Revision — 2026-06-04 (display architecture):** the display moved from *four
> projectors (one per wall) + a procedural side-wall renderer* to a **single
> panoramic projector spanning the three walls the viewer faces, showing the real
> selected video** (launch content focus: **nature video**). This supersedes the
> original §6 (four projectors) and **removes §7 (procedural side walls)**, and
> reshapes the roadmap's sub-project 3 (player drives one pano output) and
> sub-project 5 (procedural side walls — **dropped**). Affected sections below are
> updated in place and marked. The coordinate model (§2), selection (§3), and
> catalog (§4) are unchanged.
---
## 1. What this is
A single-viewer immersive art installation. One person sits in a chair at the
center of a small room with four walls, each fronted by a projector. A DJ-style
control panel lets them "tune" their experience along a small set of felt axes;
the system finds the public-domain media nearest that tuning and plays it. The
piece is an *experience filter*: the viewer dials in how they want to feel and
A single-viewer immersive art installation. One person sits in a chair in a small
room; a **single panoramic projector wraps the real selected video across the
three walls they face**. A DJ-style control panel lets them "tune" their
experience along a small set of felt axes; the system finds the public-domain
media nearest that tuning and plays it (launch content focus: **nature video**).
The piece is an *experience filter*: the viewer dials in how they want to feel and
think, and the room answers with found human artifacts that match.
Design constraints that shaped everything below:
@@ -160,24 +171,37 @@ cannot decode HD video or drive projectors.
- **Raspberry Pi 5 (or small mini-PC) — brain + player.** Holds the catalog,
reads the Arduino's serial stream, runs the selection algorithm, and plays
media. **The hard drive plugs into the Pi**, not the Arduino.
- **Projectors.** The Pi drives the **primary** wall (the wall the viewer faces)
with real content. The three **side walls** show procedural ambient (§7),
driven from the same Pi's additional outputs or a second cheap Pi.
- **Projector (single, panoramic).** *Revised 2026-06-04.* The Pi drives **one
panoramic projector that spans the three walls the viewer faces**, showing the
real selected video across the whole field. There is no longer a primary/side
split and no procedural side-wall renderer (the former §7). This both simplifies
the hardware (one output, one projector) and removes the need to *compute*
`dominant_color` for ambient walls.
Parts delta vs. the original sketch: add a ~$80 Pi; everything else
(Arduino, knobs, hard drive, projectors) stays.
Parts delta vs. the original sketch: add a ~$80 Pi and use a single panoramic
(ultra-wide / short-throw) projector instead of four; everything else (Arduino,
knobs, hard drive) stays.
---
## 7. Procedural side walls
## 7. ~~Procedural side walls~~ — REMOVED (2026-06-04)
The three non-primary walls render a slow gradient / color wash:
**Superseded by the §6 single-panoramic-projector revision.** The three walls the
viewer faces now show the **real** selected video (the pano projector spans them),
so there is no separate procedural ambient renderer. The mood axis is felt through
the chosen content itself rather than a synthetic side-wall wash.
- **Hue** comes from the playing piece's `dominant_color`.
- **Brightness** tracks the Dark / Light knobs.
Consequences:
This costs near-zero storage and makes the mood axis physically felt in
peripheral vision. In `None` mode the side walls also fade to black.
- **Sub-project 5 (procedural side walls) is dropped** from the roadmap.
- `dominant_color` loses its only consumer; computing it becomes **optional /
opt-in** in the ingest tooling (sub-project 2 spec §5.2) rather than a required
mechanical tag. The field stays in the catalog schema (unused-but-harmless,
default `""`) for a possible future ambient/lighting use.
> *Original intent (for the record): the three non-primary walls rendered a slow
> color wash — hue from the playing piece's `dominant_color`, brightness from the
> Dark/Light knobs — to make the mood axis felt in peripheral vision.*
---
@@ -218,14 +242,18 @@ defensible.
## 10. Repo layout
```
hef/ shared library: catalog model + selection (sub-project 1, built)
catalog/ the tagged content catalog (JSONL)
player/ selection algorithm + media player (runs on the Pi)
player/ media player driving the single panoramic projector (runs on the Pi)
firmware/ Arduino sketch (reads knobs/selector → USB serial)
sidewalls/ procedural ambient renderer
tools/ sourcing/ingest, dominant-color, license checker, review tool
tools/ sourcing/ingest, mechanical tagging, review tool (sub-project 2)
docs/ this spec + operator/build guide
```
*(Revised 2026-06-04: `sidewalls/` removed with §7; the selection algorithm
shipped in the shared `hef/` package, not under `player/`, since the tools import
it too — see the sub-project-1 plan.)*
---
## 11. Explicitly out of scope (YAGNI)
@@ -240,9 +268,13 @@ docs/ this spec + operator/build guide
## 12. Open implementation questions (for the plan, not blockers)
- Player stack on the Pi (e.g. mpv/ffmpeg-based vs. a custom renderer) and how
crossfades are handled.
- Player stack on the Pi (e.g. mpv/ffmpeg-based vs. a custom renderer), how
crossfades are handled, and how a single wide/panoramic output is driven (one
ultra-wide surface vs. spanned displays).
- Exact serial protocol/framing between Arduino and Pi.
- Whether side walls run on the primary Pi's extra outputs or a second Pi.
- Ingest tooling language and where the per-archive downloaders live.
- Whether the player hard-restricts to `approved` records.
- ~~Whether side walls run on the primary Pi's extra outputs or a second Pi~~ —
resolved: no procedural side walls (§7 removed).
- ~~Ingest tooling language and where the per-archive downloaders live~~ —
resolved by the sub-project-2 spec: Python under `tools/`, per-archive fetchers
in `tools/ingest/`.
@@ -0,0 +1,560 @@
# Sub-project 2 — Ingest & Tagging / Review Tools — Spec
**Date:** 2026-06-04
**Status:** Proposed (sub-project 2 of 5) — pending operator approval
**Repo:** `human-experience-filter-art`
**Design reference:** [`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md) (esp. §4 catalog, §5 division-of-labor, §8 sourcing)
**Roadmap item:** [`docs/ROADMAP.md`](../../ROADMAP.md) §2
**Builds on:** sub-project 1 — `hef.catalog` + `hef.selection` (done, merged, 37 tests)
---
## 1. What this is
Sub-project 1 gave us the **spine**: a validated JSONL catalog (`hef.catalog`) and
a nearest-match selector (`hef.selection`). Today the only way to fill that
catalog is hand-authoring records (per [`docs/USER_GUIDE.md`](../../USER_GUIDE.md)).
Sub-project 2 builds the **assisted *draft-then-review* pipeline** the design spec
calls for (§5), under `tools/`:
1. **Ingest** — per-archive fetchers pull a candidate piece from a public-domain
pool and write a catalog record.
2. **Mechanical tagging** — tooling auto-fills the non-curatorial fields: `mode`
(via `ffprobe`), `license`/`attribution`/`source_*` (from origin),
`duration_s`, `resolution`, and — optionally, opt-in only — `dominant_color`
(computed from the video; see §5.2 design-change note).
3. **Coordinate drafting** — a proposer suggests `left/right/dark/light` with a
one-line `rationale`, written as `review_status: proposed`.
4. **Review CLI** — walks `proposed` records one at a time (proposed coords +
rationale + preview frame), accepts or corrects, flips to `approved`, and
stamps `reviewed_at`.
The output is a populated catalog that `load_catalog` validates and the player
(sub-project 3) can `select()` over. "How far from done" stays exactly what §4
defines: the count of records still in `review_status: proposed`.
### Design invariants this honors
- **Curation is the artwork** (design §1). The four coordinates are a human
curatorial act (design §11: *no automatic ML coordinate tagging*). The pipeline
**drafts** coordinates as a starting point, but every coordinate is human-blessed
before it can be selected. Drafting is deterministic and rule-based — there is no
ML model in the tool.
- **The catalog is the spine.** Tools only **read** and **append/rewrite** through
`hef.catalog`; they never define a parallel record shape.
- **Local, dependency-light, hermetically testable.** Like sub-project 1, the
testable core is pure-stdlib Python. The two external boundaries — `ffmpeg`/`ffprobe`
(subprocess) and archive HTTP — are isolated behind injectable seams so the unit
suite never shells out or hits the network.
---
## 2. What `hef.catalog` already gives us (and why the schema needs no change)
The headline of the "build on without breaking" constraint: **the `Record` schema
is already complete for this flow.** Sub-project 1 shipped, with defaults chosen
for exactly the draft-then-review pipeline:
| Field group | Fields | Who sets it here |
|---|---|---|
| Identity / origin | `id`, `title`, `source_url`, `source_archive` | ingest (from the fetcher) |
| License | `license`, `attribution` | mechanical tagging (from origin) |
| Mechanical media | `mode`, `duration_s`, `resolution`, `dominant_color` | mechanical tagging (ffprobe/ffmpeg) |
| Curatorial | `left`, `right`, `dark`, `light` | drafting (proposed) → review (approved) |
| Review state | `review_status` (default `"proposed"`), `rationale`, `reviewed_at` (default `null`) | drafting sets `rationale`; review sets `reviewed_at` |
| Misc | `file_path`, `notes` | ingest (`file_path`); either (`notes`) |
`Record`'s defaults — `review_status="proposed"`, `reviewed_at=None`,
`rationale=""`, plus `attribution`/`resolution`/`dominant_color` defaulting to
`""` — are precisely an ingested-but-unreviewed record. **Ingest constructs a
`Record` and the proposed-state shape falls out of the defaults; nothing in the
schema is added or changed.**
The tools consume these existing symbols unchanged:
- `hef.catalog`: `Record`, `validate`, `record_to_dict`, `record_from_dict`,
`load_catalog`, `save_catalog`, `append_record`, `CatalogError`, and the
vocab constants `MODES`, `LICENSES`, `REVIEW_STATUSES`, `ATTRIBUTION_LICENSES`,
`COORD_FIELDS`, `COORD_MIN`, `COORD_MAX`.
- `hef.selection`: `Coordinate`, `select` (used by the end-to-end test and by
the player later).
---
## 3. The one required addition to `hef.catalog`
Per the constraint, every addition the tools need is called out explicitly. There
is exactly **one** required change to the shared library, and it is purely
additive (no existing symbol changes signature or behavior):
### 3.1 `validate_catalog(records)` — catalog-level integrity (unique ids)
`validate()` is per-record; it cannot see the whole file, so it cannot catch a
**duplicate `id`**. The ingest pipeline appends repeatedly and the review CLI
looks records up *by id* — both are unsafe if ids collide (ingest silently
double-adds a re-fetched piece; review can't tell two records apart). So add a
catalog-level check to the shared spine, where the player also benefits:
```python
def validate_catalog(records) -> None:
"""Validate every record AND the cross-record invariants (currently: unique ids).
Raises CatalogError naming the first duplicate id."""
```
- It calls `validate()` on each record, then asserts `id` uniqueness.
- **Non-breaking:** new symbol; existing `validate`/`load_catalog`/`save_catalog`/
`append_record` are untouched. Any catalog valid today (unique ids) stays valid.
- **Used by tools** at two points: before `append_record` during ingest (reject a
candidate whose derived `id` already exists), and after `load_catalog` in the
review CLI (fail loudly on a corrupted/duplicated catalog before editing it).
> A companion convenience, `index_by_id(records) -> dict[str, Record]`, may be
> added alongside it (the review CLI and dedupe both want id-keyed lookup). It is
> a trivial helper; if we prefer to keep the spine minimal it can live in the
> tools layer instead. **Recommendation:** add `index_by_id` to `hef.catalog`
> too, since by-id lookup is a catalog concern the player will also want.
**Everything else this sub-project needs lives in `tools/` and only *consumes*
`hef.catalog`.** No other change to `hef/` is required. (Considered and
deliberately *not* added now — see §11: a `source_id` archive-identifier field,
a `media_sha256`, an `ingested_at` timestamp. They are future schema extensions,
not needed for the done-criteria.)
---
## 4. Package layout
Extends the existing repo (design §10; the sub-project-1 plan put shared code in
`hef/` and data in `catalog/`):
```
tools/
__init__.py
http.py # thin stdlib-urllib client: timeout, retry, user-agent (injectable)
probe.py # Prober: ffprobe wrapper -> parsed streams (injectable)
mediatools.py # ffmpeg helpers: extract frame, dominant color, waveform thumb
tagging.py # mechanical tagging: assemble mode/duration/resolution/dominant_color
licensing.py # per-archive license/attribution normalization -> LICENSES vocab
drafting.py # Draft, Proposer protocol, HeuristicProposer
ingest/
__init__.py
base.py # Candidate dataclass + Fetcher protocol + the ingest pipeline
internet_archive.py
musopen.py
librivox.py
nasa.py
fma.py # Free Music Archive
freesound.py
ingest_cli.py # `python -m tools.ingest_cli` — run a fetcher, write proposed records
review.py # review state-transition core (pure, tested)
review_cli.py # `python -m tools.review_cli` — interactive walk of proposed records
```
- `tools/` is added to `[tool.setuptools] packages` in `pyproject.toml`.
- **Invocation matches the repo convention** (USER_GUIDE: run from repo root, no
install, `import hef` resolves via the root `conftest.py`): the CLIs run as
`.venv/bin/python -m tools.ingest_cli …` and `… -m tools.review_cli …`.
`[project.scripts]` console-script aliases (`hef-ingest`, `hef-review`) may be
declared too for an installed checkout, but `python -m` is the documented path.
- `media/` (download target, §6.3) is added to `.gitignore`. Media never enters
git — the repo holds metadata + pointers only (design §4).
---
## 5. Mechanical tagging (`tools/probe.py`, `tools/mediatools.py`, `tools/tagging.py`, `tools/licensing.py`)
All of §5's "mechanical / automatic" fields, set from the downloaded file and the
origin metadata.
### 5.1 `mode`, `duration_s`, `resolution` — from `ffprobe`
`tools/probe.py` wraps:
```
ffprobe -v quiet -print_format json -show_format -show_streams <file>
```
and parses the JSON into a small `Probe` result (streams + format). `tools/tagging.py`
derives:
- **`mode`** — from which *meaningful* streams exist:
- audio stream present, no real video → `"audio"`
- real video stream present, no audio → `"video"`
- both → `"av"`
- **Cover-art guard:** an embedded cover image in an audio file shows up as a
video stream (`mjpeg`/`png`, `disposition.attached_pic == 1`, often 1 frame).
These are **excluded** when deciding `mode`, so an MP3 with album art is
correctly `"audio"`, not `"av"`. This guard is a tested case.
- **`duration_s`** — `round(float(format.duration))`, falling back to the longest
stream `duration` if `format.duration` is absent. Integer (schema requires int ≥ 0).
- **`resolution`** — `"{width}x{height}"` from the chosen video stream; `""` for
audio-only (schema default).
`tools/probe.py` takes the subprocess runner as an injectable dependency
(default: `subprocess.run`); tests feed canned `ffprobe` JSON and never shell out.
### 5.2 `dominant_color` — optional, deferred (computed from the video only on demand)
> **Design-change note (2026-06-04):** `dominant_color`'s only consumer was the
> **procedural side walls** (design §7): hue for the three non-primary walls. The
> installation is moving to a **single panoramic projector that spans all three
> walls showing the real (nature-video) content**, which removes the procedural
> side-wall renderer — and with it the need to *compute* a dominant color. So this
> spec **demotes `dominant_color` from a core mechanical-tagging requirement to an
> optional, opt-in computation.** This is a knock-on of a larger design shift that
> primarily affects sub-projects 3 and 5 and the design spec (see §11 follow-up).
Concretely:
- The `dominant_color` **field stays** in the schema (it is already there; default
`""`) — no schema change either way.
- Ingest does **not** compute it by default. An opt-in flag
(`--dominant-color`) enables computation for `video`/`av` records, for the case
the operator still wants it (e.g. a later ambient/lighting use).
- **When enabled, it is `ffmpeg`-only — never an image library.** Extract a
representative mid-segment frame and reduce it to one hex color:
```
ffmpeg -v quiet -ss <mid> -i <file> -vf "thumbnail,palettegen=max_colors=1" -frames:v 1 -f rawvideo -pix_fmt rgb24 -
```
The 3 output bytes are the dominant palette color → `"#rrggbb"` (a `scale=1:1`
average-color path is the fallback if `palettegen` is unavailable). This honors
the operator's "no video manipulation needed" steer: no Pillow, no new dep, and
the work isn't done at all unless explicitly requested.
Audio-only records always keep `dominant_color=""`.
### 5.3 `license` / `attribution` / `source_*` — from origin (`tools/licensing.py`)
Each fetcher resolves origin metadata; `tools/licensing.py` normalizes it to the
`hef.catalog` vocab (`LICENSES = {public_domain, cc0, cc_by, cc_by_nc}`):
- Map CC license URLs/identifiers → `cc0` | `cc_by` | `cc_by_nc`; explicit
public-domain / "no known copyright" → `public_domain`.
- For `cc_by`/`cc_by_nc` (the `ATTRIBUTION_LICENSES`), build the required
`attribution` string (creator + license name/URL) — `validate()` already rejects
these licenses with empty `attribution`, so the normalizer **must** produce it.
- Anything that doesn't map to an allowed license is **rejected at ingest** with a
clear error (the piece is not defensible per design §8's license stance) — it is
never written as an invalid record.
This is the per-archive "license/attribution/source from origin" tagging; it is
unit-tested with sample metadata payloads per archive.
---
## 6. Ingest (`tools/ingest/`)
### 6.1 The `Candidate` + `Fetcher` seam
Every archive differs in API but funnels into one shape. `tools/ingest/base.py`:
```python
@dataclass
class Candidate:
source_archive: str # origin label, e.g. internet_archive | librivox | nasa (§6.4)
source_url: str # human/landing URL recorded in the record
media_url: str # direct download URL of the chosen file
title: str
license: str # normalized (tools.licensing) -> hef LICENSES vocab
attribution: str # "" unless the license requires it
suggested_id: str # stable id derived from archive + archive-identifier
media_ext: str # file extension for the download target
description: str = "" # free text used by drafting signals; lands in notes
```
```python
class Fetcher(Protocol):
archive: str # the source_archive label
def search(self, query: str, *, limit: int) -> list[Candidate]: ...
def resolve(self, identifier: str) -> Candidate: ... # one item by id/URL
```
Fetchers receive an HTTP client (`tools/http.py`) by injection so tests feed canned
API responses. Fetchers do **not** download or probe — they only resolve metadata.
### 6.2 The ingest pipeline
`tools/ingest/base.py` provides `ingest_candidate(candidate, *, catalog_path, media_root, proposer, prober, downloader)`:
1. **Dedupe.** `load_catalog` + `validate_catalog`; if `candidate.suggested_id`
already exists, skip (log "already in catalog") — idempotent re-runs.
2. **Download.** Fetch `candidate.media_url` → `<media_root>/<archive>/<id>.<ext>`
(see §6.3). Skip the download if the file already exists.
3. **Mechanical tag.** `prober` → `mode`, `duration_s`, `resolution`; `mediatools`
→ `dominant_color` (video/av only). `licensing` already normalized
`license`/`attribution` on the `Candidate`.
4. **Draft coordinates.** `proposer.propose(signals)` → `Draft(coordinate, rationale)` (§7).
5. **Build the record.** Construct a `hef.catalog.Record` from the above; the
`proposed`/`reviewed_at=None` shape comes from the defaults (§2). `notes` gets
`candidate.description`.
6. **Validate + append.** `validate(record)`, then re-check uniqueness, then
`append_record(record, catalog_path)`.
Batch helper `ingest_search(fetcher, query, *, limit, …)` runs steps over each
search hit. Everything network/subprocess is injected, so the pipeline is tested
end-to-end with a fake fetcher + fake prober + fake downloader (no I/O).
### 6.3 Download / caching layout (settles a roadmap open decision)
- **Media root:** `--media-root DIR` / env `HEF_MEDIA_ROOT`, default `./media/`
(gitignored).
- **On-disk path:** `<media_root>/<source_archive>/<suggested_id>.<media_ext>`.
- **`file_path` recorded in the record:** the path **relative to the media root**
(e.g. `nasa/nasa-as08-14-2383.mp4`). Relative-to-root is portable: the ingest
workstation and the Pi's mounted drive store the same tree under different
mounts, and the **player (sub-project 3) joins `file_path` with its own drive
mount**. `validate()` does not constrain `file_path` format, so this is
compatible with the hand-authoring guide's absolute-path examples.
> **Open decision (logged):** relative-to-media-root vs. absolute `file_path`. This
> spec recommends **relative-to-root** for portability between the tagging
> workstation and the Pi. The USER_GUIDE currently shows absolute paths for
> hand-authoring; both load fine. A doc note will reconcile them; the player spec
> will define mount resolution. Reversible.
### 6.4 The six archives (all specified; phased delivery)
License normalization and access per pool (design §8 maps these to the axes):
| Archive (`source_archive`) | Access | Typical license | Auth | Notes / risk |
|---|---|---|---|---|
| `internet_archive` (incl. Prelinger) | Metadata API `archive.org/metadata/<id>`; direct file URLs | `public_domain` / CC (per item `licenseurl`/`rights`) | none | Ambiguous "no known copyright" → `public_domain`, flagged in `notes` for review |
| `librivox` | JSON API `librivox.org/api/feed/audiobooks` | `public_domain` (charter) | none | Reader credited in `notes`; attribution not required |
| `nasa` | Images/video API `images-api.nasa.gov` | `public_domain` | none | NASA media guideline caveat: some items embed third-party content — flag in `notes` |
| `musopen` | Musopen API / catalog | `public_domain` / CC | possible key | Access historically gated — may need an API key (secret, §9) |
| `fma` (Free Music Archive) | Track/page resolution | CC (`cc_by`/`cc_by_nc`/`cc0`) | varies | **Highest risk:** the public FMA API has been deprecated/changed; fetcher resolves from a track URL + page metadata |
| `freesound` | API v2 `freesound.org/apiv2` | `cc0` / `cc_by` / `cc_by_nc` | **token required** | API token is a secret (§9); uploader → `attribution` when license requires |
> **Recommended first-ship set:** the keyless, clean-license, stable-API pools —
> **LibriVox, NASA, Internet Archive/Prelinger.** Defer the ones with an auth or
> API-stability tax — **Freesound** (token), **Musopen** (possible gate),
> **Free Music Archive** (deprecated API) — to a second pass. The `Fetcher` seam
> means deferring them costs nothing structurally; the spec still defines all six.
> *(Logged as an open decision for operator confirmation — see §11.)*
---
## 7. Coordinate drafting (`tools/drafting.py`)
The curatorial coordinates are a human act (design §11). The tool's job is to seed
a **draft** to be reviewed, never to auto-tag authoritatively.
```python
@dataclass
class Draft:
coordinate: Coordinate # hef.selection.Coordinate(left, right, dark, light)
rationale: str # one line, cites the signal that drove the guess
@dataclass
class Signals:
title: str
description: str
source_archive: str
mode: str
duration_s: int
class Proposer(Protocol):
def propose(self, signals: Signals) -> Draft: ...
```
### 7.1 `HeuristicProposer` (the deterministic baseline)
A rule-based seed grounded in the §8 sourcing→axis map and the §2 rubric — **not
an ML model** (honors design §11):
- **Archive priors** seed the brain plane: `librivox` → high `left`
(spoken/verbal); `internet_archive`/Prelinger → high `left` (educational/
industrial) by default; `musopen`/`fma` → high `right` (music); `nasa` → high
`right` (wordless awe); `freesound` → high `right` (abstract field recordings).
- **Keyword nudges** on title+description seed the mood plane: dark words
(`storm`, `night`, `noir`, `requiem`, `minor`, `war`, `funeral`, `decay`, …)
raise `dark`; light words (`sunrise`, `dawn`, `garden`, `spring`, `joy`, `hope`,
`major`, `bright`, …) raise `light`.
- Values **clamped to 0..4**.
- `rationale` is one line naming the dominant signal, e.g.
`"librivox spoken reading → strong left; 'storm' in title → dark"`.
Deterministic given `Signals` → fully unit-testable.
### 7.2 The assistant seam
`Proposer` is an interface. The `HeuristicProposer` is always-available and
dependency-free. When a **session assistant** (e.g. Claude, or the operator) does
the ingest, it can supply a considered coordinate + rationale through the *same*
seam (a manual proposer that reads a per-candidate draft, or an LLM-backed
proposer the operator wires in). Either way the record is written
`review_status="proposed"` and is **still reviewed by a human**. The tool ships
**no** ML dependency; the "assistant" is an optional, out-of-core seam.
---
## 8. Review CLI (`tools/review.py` + `tools/review_cli.py`)
### 8.1 The transition core (pure, tested) — `tools/review.py`
```python
def proposed_records(records) -> list[Record]:
"""Records still awaiting review."""
def approve(record, *, reviewed_at, coordinate=None) -> Record:
"""Return an approved copy: review_status='approved', reviewed_at set,
optionally overriding the four coordinates with a human correction.
The returned record is re-validated by the caller before save."""
```
- `reviewed_at` is an ISO-8601 UTC timestamp string
(`datetime.now(timezone.utc).isoformat()`), injected by the CLI so the core stays
pure/testable. `Record.reviewed_at` is `Optional[str]` — no schema change.
- `approve` does not mutate in place gratuitously; it produces the approved record
and the CLI persists via load → replace-by-id → `save_catalog` (rewrite). At
120800 records a full rewrite is trivial and crash-resilient when done after
each approval.
### 8.2 The interactive walk — `tools/review_cli.py`
`python -m tools.review_cli --catalog catalog/library.jsonl [--media-root ./media]`
1. `load_catalog` → `validate_catalog` (fail loudly on a corrupt catalog before editing).
2. For each `proposed` record, show:
- `id`, `title`, `source_archive`, `source_url`, `license`/`attribution`
- `mode`, `duration_s`, `resolution`, `dominant_color`
- the **proposed coordinates** `left/right/dark/light` and the one-line `rationale`
- a **preview frame**: for `video`/`av`, extract a representative frame (reusing
the §5.2 mid-segment frame) to a temp PNG and open it with the OS viewer
(`open` on macOS / `xdg-open` on Linux); for `audio`, render a waveform
thumbnail (`ffmpeg … showwavespic`) and/or offer optional 10-second playback
via `ffplay`. The preview resolves `file_path` against `--media-root`.
3. Prompt: **[a]ccept · [e]dit coords · [s]kip · [q]uit**.
- **accept** → `approve(record, reviewed_at=now)` (keep coords).
- **edit** → prompt for new `left/right/dark/light` (and optionally a new
`rationale`), then `approve(..., coordinate=corrected)`.
- **skip** → leave `proposed`, advance.
- **quit** → save and exit.
4. Persist after each approval (rewrite via `save_catalog`), so an interrupted
session keeps its progress.
The preview/prompt shell is thin; the **tested** logic is the §8.1 transition core
(deterministic, no terminal, no subprocess). Preview rendering reuses
`tools/mediatools.py` and is exercised only by opt-in integration tests.
---
## 9. Dependencies, configuration, secrets
- **System binaries:** `ffmpeg` + `ffprobe` (subprocess). Required for mechanical
tagging, dominant color, and previews. The dominant-color recommendation (§5.2)
keeps the dep surface to *just* these two — no Python image library in the core.
Pillow is an optional documented fallback only.
- **HTTP:** stdlib `urllib.request` via `tools/http.py` (timeout, small retry,
user-agent) — no `requests` dependency, preserving the sub-project-1 ethos.
`internetarchive`/other archive SDKs are explicitly avoided in favor of the
documented JSON APIs.
- **Config:** `--catalog` (default `catalog/library.jsonl`), `--media-root` /
`HEF_MEDIA_ROOT` (default `./media/`, gitignored).
- **Secrets (wgl hard rule — never in catalog, code, or transcript):** Freesound
API token via `FREESOUND_API_TOKEN` (env or macOS Keychain); any Musopen/FMA key
likewise. Referenced by name only; never written into a record or logged.
- **Testing:** `pytest` (already a dep). The unit suite stays **hermetic** —
`ffprobe`/`ffmpeg`/HTTP are injected and faked; no network, no binaries needed to
run the core tests. Integration tests that actually invoke `ffprobe`/`ffmpeg` are
opt-in and **skipped** when the binaries are absent.
- **pyproject:** add `tools` to `[tool.setuptools] packages`; optionally add
`[project.scripts]` (`hef-ingest`, `hef-review`) and a `[project.optional-dependencies]`
`tools` extra (empty unless Pillow is later adopted).
---
## 10. Testing strategy & done-criteria
Mirrors sub-project 1's TDD discipline (test-first, stdlib, hermetic).
**Unit tests (hermetic):**
1. `hef.catalog.validate_catalog` — accepts unique ids; raises naming a duplicate;
(`index_by_id` round-trip if added).
2. **Mechanical tagging** — canned `ffprobe` JSON → expected `mode`/`duration_s`/
`resolution`; the **cover-art `attached_pic` guard** keeps an art-bearing MP3 at
`mode="audio"`.
3. **Dominant color (opt-in path)** — with the flag on, canned ffmpeg `rgb24`
bytes → expected `"#rrggbb"`; with the flag off (default), `dominant_color`
stays `""`; audio-only → `""`.
4. **License normalization** — sample per-archive metadata → correct `license`;
`cc_by`/`cc_by_nc` produce non-empty `attribution`; unmappable license is
rejected.
5. **Fetchers** — canned API JSON via a fake HTTP client → `Candidate` fields and
stable `suggested_id` derivation (one test per archive that ships).
6. **Drafting** — `HeuristicProposer`: archive priors, keyword nudges, 0..4
clamping, one-line rationale content.
7. **Review transition** — `proposed → approved` sets `reviewed_at` + status; edit
overrides coordinates; skip leaves `proposed`; result re-validates and
round-trips through `save_catalog`/`load_catalog`.
8. **End-to-end (mocked)** — fake fetcher + fake prober + fake downloader +
`HeuristicProposer` → a `proposed` record appended; the review core approves it;
`load_catalog` + `validate_catalog` pass; `select(..., approved_only=True)` can
pick it.
**Opt-in integration tests** (skipped without binaries): real `ffprobe` on a tiny
fixture clip → `mode`/`duration`/`resolution`; real dominant-color extraction.
**Done when (from ROADMAP §2):**
- You can run ingest against a source and get `proposed` records.
- You can review them to `approved`.
- `load_catalog` (and `validate_catalog`) validate the result.
- Unit tests on mechanical tagging and the review state transitions pass, and the
full suite is green from the repo root inside `.venv`.
---
## 11. Open decisions (recommended defaults; logged for operator)
None block writing the implementation plan; each has a recommended default the
plan will assume unless the operator overrides.
1. **Dominant-color dependency** — *Decided (operator, 2026-06-04):* `dominant_color`
is **optional / opt-in** and, when computed, **ffmpeg-only — no image lib**,
because the move to a single panoramic projector showing real nature video
removes the procedural side walls that were its only consumer (§5.2).
2. **`file_path` style** — *Recommend:* relative-to-`media-root` for portability;
reconcile the USER_GUIDE's absolute examples with a doc note; player spec
defines mount resolution.
3. **Which archives ship first** — *Decided (operator, 2026-06-04):* **LibriVox,
NASA, Internet Archive/Prelinger** first; defer Freesound, Musopen, FMA (auth /
API-stability tax). (Nature-video content focus weights NASA/IA; the audio
pools still serve the catalog's audio modes.)
4. **CLI surface** — *Recommend:* two `python -m` entry points (`tools.ingest_cli`,
`tools.review_cli`), matching the no-install repo convention; console-script
aliases optional.
5. **`index_by_id` placement** — *Recommend:* add it to `hef.catalog` alongside
`validate_catalog` (the player wants by-id lookup too).
**Deliberately deferred (YAGNI, no schema change now):** a `source_id`
archive-identifier field, `media_sha256` integrity, an `ingested_at` timestamp, and
any LLM-backed proposer in the core. All are future, additive extensions; none are
needed for the done-criteria.
**Design-spec follow-up (beyond this sub-project):** the operator's 2026-06-04
decision to use a **single panoramic projector spanning all three walls, showing
real nature video**, supersedes the design spec's §6 (four projectors) / §7
(procedural side-wall renderer) and reshapes sub-projects **3** (player drives one
pano output, not primary + side feed) and **5** (procedural side walls
**dropped**). The design spec
(`2026-06-04-human-experience-filter-design.md` §6/§7) and ROADMAP were updated
this session to reflect this. The only element affecting the sub-project-2 spec
itself is the now-optional `dominant_color` (§5.2).
---
## 12. Relationship to the other sub-projects
- **Consumes** sub-project 1 (`hef.catalog`, `hef.selection`) — the only required
change is the additive `validate_catalog` (+ optional `index_by_id`), §3.
- **Feeds** sub-project 3 (the Pi player): a populated, `approved` catalog the
player drives to the (now single, panoramic) projector. The player resolving
`file_path` against its drive mount (§6.3) is a player-spec concern.
- **Side walls (sub-project 5)** are likely **dropped** under the pano-projector
design change (§11 follow-up); `dominant_color`, their only input, is
correspondingly optional here (§5.2).
- **Settles** two roadmap open decisions (download/caching layout §6.3; ingest
language = Python under `tools/`, §4).
@@ -0,0 +1,374 @@
# Human Experience Filter — Machine-Altered Perception (Design Revision)
**Date:** 2026-06-05
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
**Supersedes:** the thesis (§1), selection model (§3), tagging division (§5), and
sourcing strategy (§8) of
[`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md),
and substantially expands its control panel (§2 selector) and hardware (§6). The
coordinate model (§2 axes) and the single-panoramic-projector display are
**preserved**.
> **Why this revision exists.** The 2026-06-04 design framed the piece as an
> *experience filter over found human artifacts* ("curation is the artwork; not
> algorithmically generated filler"). This revision **changes the artistic
> thesis** to one about **how humans interact with machines and how that interaction
> reshapes their nervous systems** — and in doing so makes machine alteration of
> the imagery the *subject* of the piece rather than a betrayal of it.
---
## 1. What this is (the new thesis)
A single-viewer immersive installation about **a machine reshaping your
perception — for better and for worse.**
One person sits in a chair in a small room. A single panoramic projector wraps the
walls they face. The source is **neutral**: real, calm, public-domain nature
footage that sits at the *center* of the experience-space — neither analytical nor
emotional, neither bright nor dark. The viewer turns knobs on a wooden control
panel, and **the machine alters that neutral reality toward whatever they dial
in.** The piece is the felt, bodily experience of a machine bending your inner
state — and crucially it bends **both ways**: it can soothe you or disturb you,
clarify the world or manipulate it, name your feelings or colonize them.
You don't read that thesis on a wall label. You feel it in your nervous system as
your own hand moves the knobs.
Design constraints (carried from the original, one relaxed):
- **One viewer, one room, all-local at runtime.** No networking, no streaming, no
multi-user, no session recording. (All AI alteration happens **offline at
authoring time**; the room plays pre-built media + cheap runtime grading.)
- **Real footage as the substrate.** The base library is real public-domain
nature footage. The machine *alters* it — it does not invent from nothing. This
preserves real-world motion/composition (and some lineage to the original
found-media thesis) while making the alteration the point. *(This relaxes the
original "no generated content" rule: generation/alteration is now the
subject.)*
- **Built to grow.** A small, deliberate base library; the data model and tooling
designed to expand.
---
## 2. What changes vs. 2026-06-04
| Area | 2026-06-04 | This revision |
|---|---|---|
| Thesis (§1) | filter over found human artifacts | **machine reshapes perception, good & bad** |
| Content | large found library pre-matched to coordinates | **small NEUTRAL base library, machine-altered to the knob state** |
| Engine (§3) | nearest-match **lookup** in a tagged catalog | **transform from neutral** (knobs = a transformation, not a lookup) |
| Sourcing (§8) | find media that already matches each cell | source **neutral** clips; the machine produces the variants |
| Control panel (§2) | 4-way mode + 4 knobs | **7-way content dial + 4 experience knobs + volume + brightness**, on a tactile wooden panel |
| Display | single pano projector | **unchanged** |
| Coordinate axes (§2) | Left/Right/Dark/Light, two planes | **unchanged** (reinterpreted as transform targets) |
---
## 3. Coordinate model (preserved, reinterpreted)
The four experience knobs and two planes from the 2026-06-04 §2 are unchanged:
- **Brain plane** = Left × Right. Left = analytical / verbal / structured. Right =
artistic / emotional / abstract. Both high → "whole brain."
- **Mood plane** = Dark × Light. Dark = somber / heavy. Light = uplifting / serene.
Each knob is `04`. What changes is the *meaning of a knob position*: it is no
longer "find the catalog piece nearest this coordinate" but **"how hard the machine
pushes the neutral base toward this pole."** `(2,2,2,2)` is the neutral origin —
the un-altered base clip.
---
## 4. The alteration engine (replaces the §3 selection algorithm)
The engine's job changes from **choosing** a clip to **altering** one. Given a
neutral base clip and a knob vector `(left, right, dark, light)`, it applies a
**transformation toward each active pole** and composites the result.
### 4.1 Per-axis alteration
| Pole | What the machine does | How |
|---|---|---|
| **Left** (analytical/verbal) | **imposes analytical structure** on the scene — annotation, measurement, labels, grids, tracking boxes, data HUD; the machine *dissecting* the world into data | **overlay** layer composited on top |
| **Right** (artistic/emotional) | **dissolves realism** toward painterly / dreamlike / abstract | **generative video-to-video (v2v)** restyle of the pixels |
| **Dark** (somber/heavy) | drains toward the cool/melancholy/night grade (see §5) | **color grade** (deterministic) |
| **Light** (uplifting/serene) | warms toward the bright/serene grade (see §5) | **color grade** (deterministic) |
### 4.2 The composition rule
The four axes are **two kinds of operation** that **layer** rather than blend-and-
cancel:
- **Substrate transforms** — Right (v2v restyle) + Dark + Light (color grades).
They alter the **pixels** and blend with each other.
- **Overlay** — Left (annotation / labels / HUD). It composites **on top** of
whatever the substrate is.
So Left and Right are **not** mutually-exclusive opposites; they stack on different
layers. This single rule covers the whole `5×5×5×5` space:
- **Max Left + Max Right** = maximally dissolved dreamlike substrate **with the
machine's labels on the feelings themselves** — *"awe," "grief," "detected:
longing 0.82,"* tracking boxes around emotional content. This is the "whole
brain" corner (§3): the machine that both **induces** emotion and
**names/quantifies** it — affective computing made visceral. (Good/bad duality:
is being told your own feeling clarifying, or invasive?)
- **Dark + analytical** = a drained, melancholy scene with cold measurement
overlaid.
- **Light + emotional** = a warm, dreamlike wash, unlabeled.
### 4.3 Where each transform runs
> **Reconciled (2026-06-07, session 0009):** the Left HUD is a **runtime overlay**
> driven by an authored annotation track + per-language string tables (text shaped
> live); the Right axis selects a **discrete pre-baked** flow-stabilized variant
> (not a continuous blend). See
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
> §1, which supersedes the session-0007 baked-HUD / 5×5-grid proposal.
- **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade
and the Left analytical overlay. These are cheap (LUT/curves + luma key, and
text/graphics compositing) and can move continuously with the knob.
- **Pre-baked offline (the only AI cost):** the Right generative v2v restyle.
Generative v2v cannot run real-time on a Pi, so a small set of restyled variants
is rendered offline and selected/blended at runtime.
> **Crucial consequence — the Left labels are a RUNTIME OVERLAY, never baked into
> the v2v pixels.** This is what makes multilingual support nearly free (§10).
---
## 5. Mood-axis color grade
The Dark↔Light knob is a three-stop color ramp whose **center is the identity**
the raw, ungraded footage, no color transform applied. Turning toward a pole grades
*away* from the raw clip. Hue carries emotional valence; the **negative-space
value** (a luma-keyed grade — the brightest/darkest regions pushed toward
white/black) carries the literal light/dark reading, so the image *is* lighter or
darker even before the hue registers.
| Mood knob | Hue | Negative space | Feeling |
|---|---|---|---|
| **Light** | warm **yellow** | → **white** | uplifting, serene, airy |
| **Center** | **none — raw, ungraded video** | unchanged | the un-altered base |
| **Dark** | cool **blue** | → **black** | melancholy, somber, night, cold |
*(Red/alarm was considered for Dark and set aside: red reads as threat/alarm, not
the intended melancholy. Dark is the cool, mournful blue. The center applies no
grade at all — it is simply the raw clip.)*
---
## 6. Content model — the 7-way content dial
The 2026-06-04 4-way selector (None/Audio/Video/A+V) expands to **seven positions**,
letting the viewer choose channel **and audio source**:
| Position | Audio source | Video |
|---|---|---|
| **Off** | — | black |
| **White Noise** | generated white/pink noise (not from catalog) | black |
| **Music** | public-domain classical (Musopen pool) | black |
| **Audio Track** | the nature clip's **own** audio | black |
| **Video** | — | the (altered) nature video |
| **Music + Video** | public-domain classical (Musopen pool) | the (altered) nature video |
| **Audio Track + Video** | the nature clip's own audio | the (altered) nature video |
Notes:
- "Music" (classical) and "Audio Track" (the clip's own field audio) are now
**distinct sources**; "White Noise" is **generated** at runtime, not a catalog
record. This touches the catalog/`mode` model (the 2026-06-04 §4 schema), which
must represent audio *source* in addition to channel.
- **Off** is the void/rest state: black walls, silence.
- Audio alteration (steering the *sound* by knob, parallel to the video) is **out
of scope for this revision** — the experience knobs alter video; audio is chosen
by source only. (Future work.)
---
## 7. Intensity controls
Two knobs, orthogonal to the four experience axes, exist so a viewer can take the
experience at **whatever intensity they want**:
- **Volume** — audio output level.
- **Brightness** — projector output luminance.
These are pure output levels, distinct from the Dark/Light *mood grade* (which
changes hue and tone, not raw output). Brightness is "how much light hits the
wall"; Light is "how the world is colored."
---
## 8. Content sourcing — neutral base library
The library is now **small and neutral**. Instead of hunting for footage that
matches 625 coordinate cells, source a deliberate set of **neutral, calm,
real public-domain nature clips** (centered on all four axes) and let the machine
produce every variant.
- **Reuse sub-project 2** (the built ingest/tagging/review tooling) to pull
neutral public-domain nature footage and populate mechanical fields.
- **License stance** unchanged: prefer Public Domain / CC0; record license +
source per clip. Altered/generated variants record the model and that they are
machine-derived.
- **Size:** dozens of neutral base clips (not hundreds), each yielding many
altered variants. Short, seamless loops (≈2060 s) suit a looping installation
and are the v2v models' native sweet spot.
---
## 9. Economics (why this is feasible)
At the original scale (120800 pieces × ~10-min segments) AI involvement was
~$15k45k+ and not viable. Under this design it drops to **a few hundred to a few
thousand dollars**, because:
- The library is small (generate-to-order, not find-to-cover).
- Pieces are short seamless loops (cheaper **and** higher-quality for the models).
- Only the **Right** transform uses paid generative v2v; Dark/Light/Left are free
deterministic runtime operations.
Indicative offline v2v build (2026 rates — Kling 3.0 / Sora 2 Standard ≈ $0.10/s,
Veo 3.1 Lite / Runway Turbo ≈ $0.05/s; ×~4 retry yield): **~$3003k** for a small
base library × a handful of restyle variants. A flat authoring subscription
(Runway $2876/mo, Kling from ~$6/mo) may be cheaper than per-second API for a
one-time build.
---
## 10. Accessibility, i18n, and the translation-cost finding
> **Reconciled (2026-06-07, session 0009):** the near-free-i18n path is kept — the
> Left HUD is a runtime overlay (authored annotation track + per-language string
> tables, shaped live via Pango/HarfBuzz on the Pi). The session-0007 baked-HUD
> reversal is **not** adopted. See
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
> §1.
The piece is operable **blind, in the dark, in your language**, via four redundant
channels on the control panel: **touch** (engraved symbol shape), **low-light
color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local*
speaker, separate from the main audio system).
Because "the human experience from a communication perspective is not universal" is
itself on-thesis, the piece supports **many languages**. The cost of that hinges on
one architectural decision:
- **❌ Labels baked into the v2v render** → re-render every label-bearing variant
per language → video cost **× number of languages** (≈ $9k raw / $2745k with
retries for ~50 languages). A budget landmine.
- **✅ Labels as a runtime overlay** (which §4.2 already requires) → the v2v
substrate is rendered **once**, language-agnostic; a new language is a translated
string table + font + TTS voice. **Marginal video cost per language ≈ $0.**
So multilingual support costs only cheap text work: machine-translating a small
label/script vocabulary into ~100 languages is a few dollars; optional human review
of a curated tier is ~$0.10/word (~$50200/language); TTS for the read-aloud button
is pennies per language. **Net: supporting *every* language is essentially free on
the expensive part, provided labels stay a runtime overlay.**
---
## 11. The physical control panel
A **wood** panel. Each control has a **tactile engraved symbol** below it —
distinguishable by fingertip in the dark, and shaped to evoke the control's
meaning — with **braille** beneath, a **dim color-coded LED** backlighting the
engraving, and a **read-aloud button** that speaks the control's function on a
small local speaker.
Layout principle: the **four experience knobs are two opposing pairs**, with the
opposition encoded *in the touch* (angular vs flowing, heavy-down vs radiant-up).
The **utility controls** use a more conventional AV shape-language so they are
never confused with the experience four.
| Control | Engraved symbol | Touch | LED |
|---|---|---|---|
| **Left** (analytical) | parallel straight ridges / grid `≡` `▦` | flat, hard, orthogonal edges | — |
| **Right** (emotional) | spiral / flowing wave `∿` | one continuous curve, no corners | — |
| **Dark** (somber) | crescent / downward wedge `☾` `▼` | smooth solid mass, sinking | **blue** |
| **Light** (serene) | radial sunburst `☀` | center with radial spikes | **yellow / white** |
| **Content dial** (7) | per-position micro-icons: Off `○` · White-Noise `∴` · Music `♪` · Audio-Track `)))` · Video `▷` · Music+Video `♪▷` · A+V `▷)))` | detented clicks | neutral |
| **Volume** | growing wedge `◁` | a slope that thickens | neutral |
| **Brightness** | half-disc contrast `◐` | a circle, half raised | neutral |
Design notes:
- **Light vs Brightness** is the one collision trap (both are "bright"). Light =
radial sunburst (a *mood*, glows the hue it imparts); Brightness = half-disc
contrast (an *output level*) — distinct to finger and eye.
- **LEDs equal the hue the knob imparts:** Dark glows blue, Light glows
yellow/white, the mood center is neutral — tying the panel to the projection.
- **Negative-space symbols:** symbols differ by gross **edge profile** (curve vs
corner vs radial vs slope), not fine detail, so they read under a fingertip.
This substantially fleshes out **sub-project 4 (Arduino firmware / control panel)**,
previously only sketched: 7 controls + a read-aloud button + LEDs + braille, with
the Arduino reading positions and the read-aloud/LED behavior coordinated with the
Pi.
---
## 12. Hardware (revises §6)
Unchanged from the 2026-06-04 single-pano revision except for the richer panel:
- **Arduino** — reads the 7-way content dial, 4 experience knobs, volume,
brightness, and the read-aloud button; drives the panel LEDs; sends values to the
Pi over USB serial. (Bigger than the original 5-value panel.)
- **Raspberry Pi 5 (or mini-PC)** — holds the base library + altered variants,
runs the alteration engine (runtime grading + overlay compositing + variant
selection), drives the single panoramic projector, plays audio, and serves the
read-aloud clips to the local panel speaker.
- **Single panoramic projector** spanning the three walls — **unchanged**.
- **Small local speaker** on/near the panel for the read-aloud button (separate
from the main audio).
---
## 13. Roadmap impact
- **Sub-project 2 (ingest/tagging)** — reused to source the **neutral** base
clips; coordinate *drafting* matters less (the base is neutral by selection), but
mechanical tagging + license capture still apply; add a flag for "neutral base"
vs "altered variant," and model audio *source*.
- **Sub-project 3 (player)** — grows an **alteration engine**: runtime color grade
(luma-keyed), runtime analytical-overlay compositor (multilingual, from string
tables), generated white noise, and selection/blending of pre-baked v2v variants.
- **Sub-project 4 (firmware/panel)** — grows from a 5-value panel to the full
tactile/accessible panel in §11.
- **New offline pipeline** — author neutral→restyled v2v variants (the only paid
AI step) and the label/string tables + TTS per language.
---
## 14. Open questions (for the plan, not blockers)
- **Pano resolution strategy** — v2v models output ~16:9 720p1080p; the three-wall
pano wants ultra-wide/high-res. Generate-at-max + upscale, native-ultrawide
tooling, or accept the look?
- **Negative-space luma key** — exact curve/mask for "brights→white, darks→black"
on arbitrary nature footage.
- **Transform composition order** — order of grade vs v2v substrate when several
poles are active; how variant blending interpolates between pre-baked restyle
levels.
- **Neutral base count** — how many base clips for a satisfying launch.
- **Language tier** — which languages get human-reviewed translation vs
machine-only; default UI language and switching.
- **Runtime grading performance** — confirm the Pi can do luma-keyed grade +
overlay compositing at projector resolution/framerate (GPU shaders via mpv/ffmpeg).
- **`approved`-only enforcement** in the player (carried from the original).
---
## 15. Out of scope (YAGNI)
- Networking / streaming at runtime — all alteration is offline; the room is local.
- Runtime/on-demand generation (breaks all-local; slow; unbounded cost).
- Audio alteration by knob (video-only this revision).
- Multi-user / multi-viewer; session recording or analytics.
- Automatic ML coordinate tagging — base clips are chosen neutral by hand.
@@ -0,0 +1,242 @@
# Human Experience Filter — Simulator Alteration Preview (Design)
**Date:** 2026-06-06
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
**Builds on:** the alteration engine of
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
(the "design" below) and the simulator scaffold of
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md).
**Revises the design:** §4.3 and §10 — the Left analytical HUD is **no longer a
runtime overlay**; it is **baked into authored variant videos** (see §8 below).
This is a deliberate trade of near-free multilingual support for authorial
precision over the HUD.
**Retires:** the simulator's selection-era "curator's X-ray" view and its
`/api/select` + `/api/catalog/meta` endpoints (the selection model they
visualize was superseded by the alteration model).
> **Why this exists.** Operator directive (session 0007): *build and design only
> things that run in the simulator, and get the whole experience working the way
> we like in the simulator before moving to hardware.* The slice-1 alteration
> engine (PR #5) is pure logic with **no simulator surface** — you cannot yet turn
> the experience knobs and *see* the result. This design brings the alteration
> into the simulator so the look can be tuned and liked before any Pi/serial work.
---
## 1. Scope
A browser **alteration preview**: turn the four experience knobs and the content
dial, and see the neutral base footage altered toward the knob state, in real
time, on a looping clip. The purpose is to **tune the look** of the filter and to
**settle the knob→strength calibration by eye** (the open §3-vs-§4.2/§5 question
from session 0006).
**In scope**
- Live preview of the four experience knobs (Left/Right/Dark/Light) on a looping
base clip.
- **Dark/Light** rendered as a live, deterministic runtime color grade.
- **Left/Right** rendered by selecting a **pre-baked authored variant clip** from
a 5×5 grid and crossfading on change.
- A **calibration panel** that adjusts the grade curve live; the chosen values are
baked back into `player/alteration.py` defaults.
- A **RenderPlan readout** (the project's "X-ray" honesty): always show the exact
numbers the engine produced.
- The content dial's **video on/off** behavior (so "Off" goes to black), driven by
the existing `resolve_content`.
- Placeholder variant generation so the mechanism is testable before the operator
authors real videos.
**Out of scope (this slice)** — real generative v2v; serial input / the 3⇄4
framing contract; audio playback (music / white-noise / audio-track); the Pi/mpv
runtime renderer; the full transition/crossfade timing engine; multilingual label
tables; catalog-model changes. These remain later roadmap slices.
---
## 2. Architecture — Python-canonical, thin browser renderer
`player/alteration.py` stays the **single source of truth** for the alteration
math (handbook §4.2 — deterministic core, thin I/O). The browser owns only
*rendering*. Data flow:
```mermaid
flowchart LR
subgraph Browser
K[4 experience knobs +<br/>content dial +<br/>calibration sliders]
V["&lt;video&gt; variant + canvas grade"]
RO[RenderPlan readout]
end
subgraph FastAPI [simulator/app.py]
EP["POST /api/alteration"]
CL["GET /api/clips"]
end
ENG["player.alteration.plan_alteration(coord, calibration)"]
K -- "debounced POST {controls, calibration}" --> EP
EP --> ENG --> EP -- "RenderPlan {variant, grade}" --> V
EP --> RO
CL -- "base clip + variant manifest" --> V
```
- The browser sends control/calibration changes (debounced) and receives a
`RenderPlan`. Video filtering itself runs continuously in the browser; only a
*plan recompute* makes a round-trip. On localhost these JSON round-trips are
imperceptible.
- "Bake the calibration winner in" = change the `Calibration` defaults in Python.
Nothing in the browser is canonical.
---
## 3. The four axes — how each is rendered
| Axis | Engine output | Browser rendering |
|---|---|---|
| **Dark / Light** | `ColorGrade.tone` ∈ [1, 1], center = identity | live color grade on the `<video>`: Light → warm/yellow + negative space toward white; Dark → cool/blue + negative space toward black; 0 = raw. (Confirmed direction in brainstorm.) |
| **Left / Right** | `VariantRef(left, right)` | select the authored variant clip for `(left, right)`; **crossfade** when it changes; `(0,0)` → raw base. The analytical HUD (Left) and dreamlike restyle (Right) are **baked into the clip** (§8). |
Dark/Light is the only continuously-tunable axis in this slice; Left/Right is a
discrete selection by coordinate.
---
## 4. The variant grid (per base clip)
Each base clip carries a 5×5 grid keyed by the two baked axes:
- Rows = **Left** (analytical / HUD), 04. Columns = **Right** (artistic /
dreamlike restyle), 04.
- `(0,0)` = the raw base clip — no authored file needed.
- The other **24 cells are authored videos** (4 analytical-edge + 4 dreamlike-edge
+ 16 combined core). This is **per base clip**; authoring load multiplies by the
number of base clips (and by language, since the HUD is baked — §8).
The grid is **data**, not code: a manifest the simulator reads (see §6).
---
## 5. Engine reconciliation (`player/alteration.py`)
The slice-1 engine modeled Left/Right as continuous runtime layers
(`AnalyticalOverlay.intensity`, `Restyle.blend`). Baked variants make that
obsolete. Changes:
- **`RenderPlan`** becomes `{ grade: ColorGrade, variant: VariantRef }`.
- `ColorGrade` — unchanged.
- `VariantRef(left, right)` — new; identity selection by coordinate.
- **Remove** `AnalyticalOverlay` and `Restyle` (their content is now baked into the
variant clip) rather than leaving dead fields.
- **`Calibration`** — a new frozen dataclass parameterizing the grade curve (e.g.
`mood_center`, per-axis curve), defaulting to **today's exact behavior**
(behavior-preserving). `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`.
- **`player/state.py`** — the `CROSSFADE` trigger switches from "restyle changed"
to "variant changed"; `LIVE_UPDATE` still covers a grade-only change. Video
on/off fades unchanged.
These are framework-code changes that keep the engine pure and unit-tested; they
do **not** bake any deployment-shape decision into the engine.
---
## 6. Data & endpoints
- **`simulator/clips.py`** (replaces `simulator/fixtures.py`): reads a base-clip +
variant manifest. Each base clip: `{ id, title, base_file, license, source,
variants: { "L,R": { file, model?, hud_lang? } } }`. Missing cells fall back to
the raw base (and are flagged in the readout as "raw / unauthored").
- **`POST /api/alteration`**: body `{ controls, calibration }``RenderPlan`
(plus the `ContentResolution` from `resolve_content`, so the dial's video-on/off
is honored). The endpoint calls the real `plan_alteration`.
- **`GET /api/clips`**: the base-clip list + variant manifest for the active base.
- Video files served as static assets from a sample-media directory.
- **Removed**: `POST /api/select`, `GET /api/catalog/meta`.
- `hef.selection` (the library, incl. `Coordinate`) is **untouched**; only the
simulator's selection view/endpoints retire. (`ranked_candidates`, added for the
old X-ray, may become unused by the simulator — noted, not removed here.)
---
## 7. Calibration tuning
The calibration panel exposes the grade-curve parameters (the two conventions in
tension: `(2,2,2,2)`-centered vs the `value/4` reading currently implemented, plus
curve shape). Changing them re-requests the plan and the footage responds live.
When the operator is happy, the chosen values are written into
`DEFAULT_CALIBRATION` in `player/alteration.py` and locked in by a unit test.
---
## 8. Baked HUD — the §4.3/§10 revision (accepted trade)
The design's §4.3/§10 kept the Left HUD a **runtime overlay** specifically so the
v2v substrate is language-agnostic and multilingual support is near-free. This
design **reverses that for the Left/Right plane**: the operator authors the HUD
**into** the variant videos to get pixel-precise control over the HUD and the
analytical↔feeling balance.
**Consequences (accepted):**
- Multilingual support is **no longer near-free**: baked HUD text means a new
language re-renders every label-bearing variant (the §10 "budget landmine").
The piece is English-first; broad i18n is deferred/expensive.
- The live Dark/Light grade is applied **on top** of the baked cell, so it **tints
the baked HUD** too (a "Dark" mood cools/darkens the HUD). With baked overlays
the grade cannot skip the HUD. Escape hatch: author HUD colors that survive
grading, or (future) reinstate a runtime HUD layer.
The parent design's §4.3/§10 must be updated to point at this revision (a task for
the implementation plan).
---
## 9. Bootstrapping before authored videos exist
The operator will author the 24 variants per base clip; none exist yet. So the
mechanism is testable immediately, the build includes a **placeholder generator**:
for at least one base clip, produce the 24 cells by burning the cell's `L,R` (and
a stub HUD caption) into the base loop via ffmpeg. Real authored clips drop into
the manifest with **no code change**.
---
## 10. Testing
- **`player/` unit tests:** `DEFAULT_CALIBRATION` is behavior-preserving vs the
current helpers; the two calibration conventions produce the expected plans;
`VariantRef` selection (incl. `(0,0)` → raw) and the `state.py` crossfade
trigger on variant change.
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
`POST /api/alteration` returns the engine's plan for given controls/calibration;
`GET /api/clips` returns the manifest; the removed endpoints are gone.
- **No browser/E2E automation** this slice (manual visual tuning is the point);
the rendering JS is kept thin and the engine logic stays in tested Python.
---
## 11. What ships
- `player/alteration.py` (parameterized `Calibration`, `VariantRef`, slimmed
`RenderPlan`) + `player/state.py` crossfade-trigger update.
- `simulator/clips.py` (variant manifest) replacing `fixtures.py`.
- `simulator/app.py` new endpoints; selection endpoints removed.
- `simulator/static/` rewritten as the Player preview (variant `<video>` +
crossfade, live grade, calibration panel, RenderPlan readout, content dial).
- Placeholder-variant generator + a sample base clip.
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten;
parent design §4.3/§10 pointer updated; `docs/ROADMAP.md` §3 updated.
---
## 12. Open questions (for the plan, not blockers)
- **Grade vs baked HUD interaction** — accepted that the grade tints the HUD;
revisit only if it reads badly once real authored clips exist.
- **Sample base clip** — pick one CC0 nature loop for the placeholder grid.
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for
tuning; the real timing engine is a later slice.
---
## 13. Out of scope (YAGNI)
Real generative v2v; serial / firmware; audio playback; the Pi renderer; the full
transition engine; multilingual tables; catalog-model changes; retiring
`hef.selection.ranked_candidates`. All remain later roadmap work.
@@ -0,0 +1,299 @@
# HEF — Reconciled Simulator-First Alteration Slice (Design)
**Date:** 2026-06-07
**Status:** Approved design (pre-implementation) — reconciliation approved this session (0009)
**Repo:** `human-experience-filter-art`
**Reconciles:**
[`2026-06-06-simulator-alteration-preview-design.md`](./2026-06-06-simulator-alteration-preview-design.md)
(session 0007, the *unmerged* `feature/simulator-alteration-preview` branch) **with**
[`2026-06-07-scales-library-and-right-axis-pipeline-design.md`](./2026-06-07-scales-library-and-right-axis-pipeline-design.md)
(session 0008, merged to `main`).
**Parents:**
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
(the alteration engine) and
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md)
(the simulator scaffold).
**Supersedes:** the **baked-HUD / 5×5-grid** position of the 0007 design (§4, §5, §8 there).
> **Why this exists.** Sessions 0007 and 0008 left two unmerged design threads that
> disagree on one load-bearing point — how the **Left** analytical HUD is rendered —
> and, downstream of that, on the **shape of the pre-baked variant set**. 0007 baked
> the Left HUD into a 5×5 grid of authored Left×Right variant clips (pixel-precise,
> but i18n becomes expensive). 0008 (the later, merged design) kept the Left HUD a
> **runtime overlay** driven by an authored annotation track + per-language string
> tables (near-free i18n). This document picks the runtime-overlay position, follows
> its consequences through the engine and the simulator, and scopes the first
> **simulator-runnable** slice that realizes it. It then hands off to an
> implementation plan.
---
## 1. The decision (operator-approved this session)
**Left is a runtime overlay, not baked pixels.** Concretely:
- **Left axis** → a **runtime `AnalyticalOverlay`** driven by an **offline-authored
annotation track** (box positions, anchor points, and which label *keys* appear at
each Left level 04) plus a per-language **string table** (key → translated text).
Text is **shaped live** — natively by the browser in the simulator; by
**Pango + HarfBuzz + Noto** on the Pi at runtime (the 0008 §1.2 requirement). This
is the "authored box positions + runtime-shaped text" hybrid: authorial control
over *layout*, near-free *i18n*.
- **Right axis** → the **only pre-baked axis**: a small set of **flow-stabilized
restyle-strength variants** (0008 §1), selected **discretely** by the Right knob
(04), with a **crossfade** on change.
- **Dark/Light** → a **live runtime `ColorGrade`** (mood center = identity, §5 of the
parent).
**What this supersedes.** The 0007 design's 5×5 Left×Right **grid of 24 authored
clips** and its **removal of `AnalyticalOverlay`/`Restyle` from `RenderPlan`** are
dropped. The Left HUD is *not* baked; the variant set is **1-D over Right strength**,
not a 2-D grid. (Accepted loss vs. fully-baked HUD: no pixel-painted HUD *artwork*
the HUD is shaped text + drawn boxes. Accepted gain: near-free i18n + far less
authoring — N Right variants per clip, not 24.)
**What it preserves from 0007.** The valuable, non-conflicting parts: Python-canonical
engine with a thin browser renderer; a parameterized **`Calibration`** tuned by eye in
the sim; retiring the simulator's selection-era surface; placeholder-variant generation
so the mechanism is testable before real authored media exists.
### 1.1 The merged engine is already most of the way there
The slice-1 engine on `main` (`player/alteration.py`) **already** models
`RenderPlan = { grade, overlay, restyle }` with a runtime `AnalyticalOverlay` — only
0007's *unmerged doc* proposed removing it. So this reconciliation is **surgical**, not
a rewrite: keep `grade` and `overlay`; change only how the **Right** axis and the
**calibration** are modeled.
---
## 2. Engine reconciliation (`player/alteration.py`, `player/state.py`)
### 2.1 `RenderPlan` layers
| Layer | Axis | Type | Change from `main` |
|---|---|---|---|
| `grade: ColorGrade` | Dark/Light | `tone ∈ [1,1]`, 0 = identity | unchanged shape; curve now from `Calibration` |
| `overlay: AnalyticalOverlay` | Left | gains discrete `level: int` (04) + keeps `intensity` | `level` added so the renderer selects which annotations show |
| `restyle: Restyle` | Right | **discrete** `variant: int` (04); 0 = raw | replaces continuous `blend: float` |
- **`Restyle.variant`** is a discrete index selecting a **pre-baked** Right-strength
clip. `variant == 0` means the raw base (no restyle). This matches "select a
pre-baked variant and crossfade," and replaces the continuous `blend` that no longer
has a runtime meaning (restyle is pre-baked, not blended live).
- **`AnalyticalOverlay.level`** (04) is the Left knob value; the renderer uses it to
choose which annotations from the authored track are active. `intensity` (0..1)
stays for overlay opacity/strength and is derived from `level` via `Calibration`.
- **`RenderPlan.is_identity`** holds when `grade.is_identity and overlay.level == 0
and restyle.variant == 0`.
### 2.2 `Calibration` (new frozen dataclass)
A frozen `Calibration` parameterizes the knob→strength maps so they can be tuned **by
eye in the sim** and then baked into a `DEFAULT_CALIBRATION` constant:
- `mood_center` and a per-axis curve for `_mood_tone` (Dark/Light).
- the Left `level → intensity` curve.
- the Right `knob → variant` map (which knob positions select which pre-baked
strength; identity-preserving so knob 0 → variant 0).
`plan_alteration(coord, calibration: Calibration = DEFAULT_CALIBRATION) -> RenderPlan`.
**`DEFAULT_CALIBRATION` reproduces today's exact behavior** (the three current helpers:
`value/4` for Left intensity, `(lightdark)/4` for mood, `right` → `variant=right`), so
the change is behavior-preserving until the operator tunes it. The session-0006
knob→strength open decision is then settled **by eye** in the sim's calibration panel
and locked into `DEFAULT_CALIBRATION` by a unit test.
### 2.3 `player/state.py`
`state.py` already classifies a change to `plan.restyle` as a `CROSSFADE` and a grade-/
overlay-only change as `LIVE_UPDATE`. With `Restyle.variant` discrete, the same
classifier works unchanged: a new Right variant → `CROSSFADE`; a grade or Left-overlay
change → `LIVE_UPDATE`; clip swap or video on/off → crossfade / fade-to-black as today.
The only edit is to the `_classify` comparison if the field name changes
(`restyle.blend` → `restyle.variant`).
These are pure framework-code changes; no deployment-shape decision enters the engine.
---
## 3. Simulator (`simulator/`)
### 3.1 Retire the selection era
The simulator currently visualizes the **old selection model** (the "curator's X-ray").
Remove:
- `POST /api/select`, `GET /api/catalog/meta`;
- the X-ray static UI (`simulator/static/*` rewritten, see §3.4);
- `simulator/fixtures.py` (the 625-record synthetic *catalog*) — replaced by
`simulator/clips.py`.
`hef.selection` (the library, incl. `Coordinate`, `ranked_candidates`) is **untouched**;
only the simulator's selection *surface* retires. `Coordinate` is still used by the
alteration engine.
### 3.2 `simulator/clips.py` (replaces `fixtures.py`)
Reads a base-clip + variant manifest. Per base clip:
```
{
"id": "forest",
"title": "...",
"base_file": "forest/base.mp4",
"license": "...", "source": "...",
"right_variants": { "1": {"file": "forest/right1.mp4", "model": "..."},
"...": {...}, "4": {"file": "forest/right4.mp4"} },
"annotations": [ {"key": "detected.conifer", "box": [x,y,w,h], "min_level": 1}, ... ],
"strings": { "en": { "detected.conifer": "conifer", ... } }
}
```
- `right_variants` is keyed by Right strength `1..4` (strength `0` = raw `base_file`).
Missing strengths fall back to the raw base and are flagged "raw / unauthored" in the
readout.
- `annotations` is the **authored annotation track** (box + label key + the minimum
Left level at which it appears). `strings` is the per-language table (English only
this slice).
### 3.3 Endpoints
- **`POST /api/alteration`** — body `{ controls, calibration? }` → `RenderPlan`
(serialized) **plus** the `ContentResolution` from `resolve_content` (so the content
dial's video-on/off is honored — "Off" → black). Calls the real
`plan_alteration(coord, calibration)`.
- **`GET /api/clips`** — the base-clip list + the active clip's manifest (variants +
annotation track + string table).
- Video served as static assets from a sample-media directory.
- **Removed:** `POST /api/select`, `GET /api/catalog/meta`.
### 3.4 Browser preview (`simulator/static/`)
Thin renderer; all alteration math stays in Python.
- **Right** — `<video>` showing the selected Right-variant file; **opacity crossfade**
to the new file when `restyle.variant` changes; variant 0 → the raw base.
- **Dark/Light** — **live color grade** over the video via CSS/canvas filters: Light →
warm + lifted toward white; Dark → cool + crushed toward black; tone 0 → raw.
- **Left** — overlay drawn **live** from the annotation track: for each annotation with
`min_level ≤ overlay.level`, draw its box and the **shaped** string (browser-native
shaping) for the active language; opacity from `overlay.intensity`. No baked HUD.
- **Content dial** — drives `<video>` visibility; "Off"/audio-only → black walls.
- **Calibration panel** — sliders for the `Calibration` params; changing them
re-requests the plan and the footage responds live.
- **RenderPlan readout** — always shows the exact engine numbers (grade tone, overlay
level/intensity, restyle variant) — the project's honesty "X-ray," now over the
alteration model.
The Left overlay being browser-drawn is the **simulator analogue** of the Pi's
Pango/HarfBuzz path: both take the *same* annotation track + string table; the browser
shapes natively, the Pi shapes with HarfBuzz. The manifest is the shared contract.
---
## 4. The sample clip + a real Right variant (this slice)
To make the look **evaluable now**, wire one real clip end-to-end using the session-0008
POC artifacts (`~/hef-poc/out/`, outside the repo):
- **Base clip** = the POC's `neutral.mp4` (an 8 s nature loop) → copied to the
sample-media dir as the one base clip.
- **Right variant (top strength)** = the POC's **`right_flow.mp4`** — the real,
operator-approved **flow-stabilized** restyle — wired as Right strength 4.
- **Intermediate Right strengths (13)** = generated by a small **ffmpeg placeholder
generator** (e.g. graded/blended stand-ins) so the crossfade mechanism is exercised
across the full knob range; the real high-end look is present for tuning.
- **Left annotation track** = a minimal authored track (a few boxes + English label
keys) for that clip, so Left renders as real shaped text over drawn boxes.
> **Licensing note.** The simulator sample footage exists **only to tune the look**; it
> is not shipped installation content. Strict-PD scale-library sourcing (NASA/NOAA/NPS
> per 0008 §2.1) remains a later slice and is unaffected by this choice.
A real multi-strength SD re-bake (4 genuine flow-stabilized strengths) is **out of scope
this slice** — one real strength + placeholders is enough to settle the mechanism and the
calibration. The re-bake is a later offline-pipeline task.
---
## 5. Testing
- **`player/` unit tests** (`tests/test_player_alteration.py`, `test_player_state.py`):
- `DEFAULT_CALIBRATION` reproduces the current helpers exactly (behavior-preserving).
- `Restyle.variant` is discrete; knob 0 → variant 0 (identity); a non-default
`Calibration` changes the plan as specified.
- `AnalyticalOverlay.level` maps from the Left knob; `intensity` derives from it.
- `state.py`: a Right-variant change → `CROSSFADE`; a grade-/overlay-only change →
`LIVE_UPDATE`; video on/off unchanged.
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
- `POST /api/alteration` returns the engine's plan (+ `ContentResolution`) for given
controls/calibration.
- `GET /api/clips` returns the manifest.
- the removed endpoints (`/api/select`, `/api/catalog/meta`) are gone (404).
- `test_fixtures.py` retired/rewritten for `clips.py`.
- **No browser/E2E automation** this slice — manual visual tuning is the point; the JS
stays thin and the tested logic stays in Python.
---
## 6. What ships
- `player/alteration.py` — `Calibration` + `DEFAULT_CALIBRATION`, discrete
`Restyle.variant`, `AnalyticalOverlay.level`; `plan_alteration(coord, calibration)`.
- `player/state.py` — crossfade-trigger field rename only.
- `simulator/clips.py` (variant + annotation manifest) replacing `fixtures.py`.
- `simulator/app.py` — `/api/alteration` + `/api/clips`; selection endpoints removed.
- `simulator/static/` — rewritten as the alteration preview (variant `<video>` +
crossfade, live grade, live Left overlay, content dial, calibration panel,
RenderPlan readout).
- Sample base clip + one real Right variant + placeholder generator + minimal Left
annotation track / English strings.
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten; the parent
design §4.3/§10 pointer updated to cite this reconciliation; `docs/ROADMAP.md` §3
updated.
---
## 7. Out of scope (YAGNI) — later slices
- Serial input / the 3⇄4 framing contract; the Pi/mpv/GPU runtime renderer (deferred by
`simulator-first-before-hardware`).
- Audio playback (music / white-noise / audio-track).
- The **endless rotary encoder** + **AI zoom/warp transitions** between scales
(0008 §3) — a separate control + offline pipeline element.
- A real multi-strength SD flow-stabilized re-bake; strict-PD scale-library sourcing
(0008 §2.1).
- Catalog-model changes (audio source / neutral-vs-variant flag); retiring
`hef.selection.ranked_candidates`.
- Broad multilingual string tables (English-first; the runtime path keeps i18n cheap,
but authoring other languages is later).
---
## 8. Open questions (for the plan, not blockers)
- **Calibration curve shape** — **RESOLVED (session 0010, by eye).**
`DEFAULT_CALIBRATION` is **locked** to unity gains + a linear variant map
(`mood_gain=1.0`, `overlay_gain=1.0`, `right_variant_map=(0,1,2,3,4)`), as a
deliberate choice: with the dark-grade fix below, full knob is peaceful on
every axis (POC + sim), so full tilt = full look and the 5 notches map 1:1 to
the 5 discrete Right bakes. This also closes the **session-0006** convention
question — knobs run 0=off..4=max, equal Dark/Light = identity; no
"centered at 2 = no push." Guarded by `test_default_calibration_is_locked`.
- **Grade vs. Left overlay interaction** — **RESOLVED: overlay above the grade.**
The simulator composites the Left HUD (SVG) above the mood grade and the cool
tint, so the HUD stays legible regardless of mood. The Pi renderer should do
the same.
- **Dark-pole grade look** — **FIXED (session 0010).** The first by-eye pass found
the sim's dark grade used a full-frame `hue-rotate(-200deg)`, which turned the
rock orange and trees purple — the disorienting look rejected in 0008, not the
peaceful POC `dark_frame`. Replaced with darken + slight desaturate on the video
filter plus a `multiply`-blended deep-blue wash (`#tint`) that lifts shadows
toward blue while preserving natural greens. The Pi renderer (later slice) will
do proper grading; this matches the approved POC dark look closely enough to tune
by eye in the sim.
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for tuning;
the real timing engine is a later slice.
- **Placeholder fidelity** — how close the strength-13 placeholders should look to real
restyle; cheap stand-ins are fine for mechanism + calibration.
@@ -0,0 +1,236 @@
# HEF — Scales-of-Nature Library + Stabilized Right-Axis Pipeline (Design Revision)
**Date:** 2026-06-07
**Status:** Approved design (pre-implementation) — operator-approved this session (0008)
**Repo:** `human-experience-filter-art`
**Refines:** [`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
— specifically its Right-axis pipeline (§4.1/§4.3), content sourcing/model (§6/§8),
and economics (§9), and it **adds a scale-navigation control + zoom transitions**
to the §2 selector / §11 control panel. The thesis (§1), coordinate model (§3),
Dark/Light/Left treatment, and accessibility (§10) are **preserved**.
**Grounded in:** a local proof-of-concept run this session on the operator's Mac
mini (M4 Pro, 64 GB, MPS) — all numbers below are measured, not estimated.
> **Why this revision exists.** The 2026-06-05 design specified the Right axis as
> "generative video-to-video restyle, pre-baked offline" and assumed that meant a
> **paid cloud API** (§9 priced Kling/Sora/Veo/Runway at $0.050.10/s). A POC this
> session established two things that change the design: (1) the Right restyle runs
> **entirely locally and offline** on the operator's existing hardware, for the
> cost of electricity; and (2) naïve per-frame restyle **boils/flickers** in a way
> the operator found disorienting — disqualifying for a piece meant to be peaceful
> — and the fix is **optical-flow keyframe propagation**. Separately, the operator
> chose how the "scales of nature" idea enters the piece: as the curatorial theme
> of a **small neutral base library**, not a single fixed journey.
---
## 1. The Right axis is a local, flow-stabilized restyle (refines §4.1, §4.3)
The §4.1 mapping is unchanged in spirit — **Right = dissolve realism toward
painterly/dreamlike via generative video-to-video** — but the *implementation* is
now pinned:
- **Engine:** Stable Diffusion **img2img** (POC used `stabilityai/sd-turbo`) run on
**Apple MPS**, locally, offline, at authoring time. No cloud API.
- **Temporal coherence is a hard requirement, not a nicety.** Per-frame img2img
independently re-imagines each frame, producing a shimmering "boil" that reads as
disorienting — the **opposite** of the piece's peaceful intent. This was caught
in the POC and is now a named design constraint: *the Right substrate must be
temporally coherent.*
- **Stabilization: optical-flow keyframe propagation.** Fully stylize **keyframes**
at a fixed interval; for in-between frames, **warp the previous stylized frame
forward by optical flow** (so motion is continuous) and apply only a *light*
diffusion refine. The flow warp removes the boil; periodic keyframes bound drift.
(This is the EbSynth principle. Genuine `ebsynth`/`ezsynth` are NVIDIA/Windows-
leaning and don't install cleanly on Apple Silicon, so the POC implemented the
same idea directly with OpenCV Farneback flow + the existing diffusers pipeline.)
This stays consistent with §4.3's crucial invariant: **the Left analytical labels
remain a runtime overlay, never baked into the restyled pixels** — the POC's Left
HUD is composited deterministically on top, preserving the near-free i18n of §10.
### 1.1 Where each transform runs (updated §4.3 table)
| Pole | Operation | Where it runs | Measured cost (8 s, 1080p clip) |
|---|---|---|---|
| **Dark** | color grade | runtime, live on the Pi | ~2.4 s offline; live at runtime |
| **Light** | color grade | runtime, live on the Pi | ~2.5 s offline; live at runtime |
| **Left** | analytical overlay (HUD) | runtime, live on the Pi | ~2.2 s offline; live at runtime |
| **Right** | local generative restyle + flow propagation | **pre-baked offline, locally** | ~2.7 min/restyle-strength |
The three deterministic axes are confirmed cheap enough to run **live**; only the
Right restyle is pre-baked. A peaceful **deterministic** alternative for Right (soft
edge-preserving smoothing + bloom, zero flicker by construction, ~5 s/clip) was
prototyped and set aside — the operator preferred the true generative repaint once
the flow stabilization made it calm. It remains a documented fallback.
### 1.2 Runtime label rendering & i18n (sharpens §10)
The Left analytical labels are drawn **live by the Pi as a 2D graphics overlay**
**not** baked into video, and specifically **not** a pre-rendered overlay *video*. A
baked overlay would have to exist per language × per Left level × per scale, which
re-introduces the "× number of languages" cost §10 exists to avoid. So the runtime
path is:
- **Architecture.** Per base clip, an offline-authored **annotation track** (box
positions, anchor points, which annotations appear at each Left level) referencing
language-agnostic label **keys** (e.g. `detected.conifer`). Per language, a cheap
**string table** (key → translated text) + font + TTS voice. At runtime the Pi
reads the Left knob, selects the active annotations, **shapes** the current
language's strings, and composites over the altered video — updating only on change
(knob move, language switch, timeline cue), not every frame.
- **Correctness needs a real shaping stack.** Rendering *every* language correctly
(Arabic joining, Indic conjuncts, CJK, RTL) requires **Pango + HarfBuzz + Noto
fonts**, not naïve text drawing. (The POC's HUD used ffmpeg `drawtext`/Menlo —
Latin-only; it would mis-render complex scripts and is **not** the runtime path.)
This shaping stack is the load-bearing requirement behind §10's "label things
correctly."
- **Feasibility.** This is OSD/subtitle-class compositing; a Pi 5 (VideoCore VII,
GLES/Vulkan, hardware decode) handles it. Headroom at the *panoramic* resolution is
the one unmeasured variable — see §6.
---
## 2. Content = a small NEUTRAL "scales of nature" library (refines §6, §8)
The operator's "cosmic zoom" concept (space → continents → birds → ocean → abyss →
microscopic → galaxy) enters the piece **as a curatorial theme, not a fixed film.**
- **Structure:** a **small library (~46 to start) of calm, neutral base clips**,
each drawn from a *different scale of nature* — e.g. an orbital Earth, a forest, a
coral reef, the deep-sea abyss, the microscopic, the cosmos. The machine alters
whichever clip is playing, exactly as for any neutral base.
- **Why this and not a single stitched journey.** A literal galaxy→cell "how small
we are" journey carries its **own** emotion (awe, cosmic insignificance) *before
the machine acts*, which contradicts the §1 neutral-base thesis, and as a fixed
film it becomes "a journey you watch" rather than "a reality you bend." Keeping the
scales as the *theme of a neutral library* preserves the awe-of-scale richness and
the piece's coherence **while keeping the base neutral and the experience
interactive.** (Rejected alternatives: single stitched base; cosmic-zoom as
intro/reset; rethinking the thesis.)
- **Cost is not the constraint.** Per §4 below, ~5 base clips is ~1 hour of overnight
local pre-bake — so this is an *artistic* choice, made on artistic grounds.
- **Mechanism unchanged:** this slots into the existing §6 content model and the
sub-project-2 ingest/tagging/review tooling; "scales of nature" is simply the
selection principle for which neutral clips to source.
### 2.1 Strict-PD sourcing map (refines §8)
License stance is unchanged (prefer Public Domain / CC0; record license + source per
clip). The "scales" theme maps onto genuinely public-domain pools cleanly at the
*ends* and is softer in the terrestrial *middle*:
| Scale | Best strict-PD source | Status |
|---|---|---|
| Cosmos / galaxy / deep space | NASA, Hubble, JWST | 🟢 abundant, true PD |
| Earth from orbit / continents | NASA / ISS | 🟢 true PD |
| Ocean & **deep sea / abyss** (global) | **NOAA Ocean Exploration** | 🟢 true PD, worldwide |
| Microscopic / single-celled | NIH / NSF | 🟢 thinner but PD |
| US land / wildlife | NPS, USGS, USFWS | 🟢 true PD (US locations only) |
| **Non-US terrestrial, high-flying birds** | — | 🟡 mostly CC-BY; the PD soft spot |
US-government works are public domain by statute (17 U.S.C. §105); the installation
is US-based, so this is the cleanest possible legal footing. **Caveat the design
must respect:** "free stock" sites (Pexels, Pixabay, Mitch Martinez's free 4K, etc.)
are *royalty-free but NOT public domain* — they restrict redistribution and retain
copyright. The ingest tool's "no explicit license → assume PD, verify before use"
flag exists precisely for this trap and must not be trusted blindly.
---
## 3. Scale navigation & zoom transitions (new element; refines §2 selector, §11)
The scales-of-nature library is navigated as a **closed loop (a ring), not a line.**
A dedicated **scale ("zoom") control** lets the viewer journey through scales;
advancing it triggers a short **AI zoom/warp transition** to the next scale,
pre-baked offline. Diving past the smallest (single-celled) **wraps around** to the
largest (cosmos) — the infinite-zoom payoff that unifies micro and macro and makes
the ring continuous.
- **The control is an *endless* rotary encoder — infinitely turnable, no end stops.**
The form embodies the concept: a ring of scales has no beginning or end, so neither
does the knob. Keep turning one way and you zoom inward forever (…reef →
microscopic → **cosmos** → continents → …); turn back to zoom out. This sets it
apart from the four **experience knobs**, which are *absolute* 04 pots: the zoom
control reports **relative** rotation (encoder detents), and the player/firmware
advances or retreats one ring-step per increment. It is distinct too from the §6
content dial (audio/video channel) — it chooses *where in the ring* you are, while
the knobs still bend whichever scale is present.
- **Transitions:** between each adjacent pair of scale clips, a short (~few-second)
generative morph — **first-last-frame-conditioned image-to-video** (Wan/LTX-class)
or SD "infinite-zoom" outpainting for the literal zoom-through. Pre-baked offline,
local; one clip per ring edge (N scales → N transitions, including the micro→cosmos
closer). A fast spin may cross several scales — transitions chain, or past a speed
threshold a faster blended pass is used.
- **Thesis-safe:** dwells on a scale are the neutral, knob-altered interactive cores;
transitions are fixed connective moments (un-altered, or at most carrying the
current mood grade). The awe lives in the *movement between* scales, not the base.
- **Heavier than the restyle, still bounded:** generative video synthesis costs more
per second than img2img, but transitions are few and short — a handful of ~35 s
morphs is an overnight local batch.
---
## 4. Economics update — local authoring ≈ free (refines §9)
§9 priced the Right axis at **~$3003k** of cloud generative-v2v API. The POC
collapses that: the restyle runs on **hardware the operator already owns**, offline,
so the marginal cost is electricity.
- **Per base clip:** ~4 Right restyle strengths × ~2.7 min ≈ **~11 min of local AI
pre-bake**, plus seconds for the deterministic Dark/Light/Left grid.
- **Whole small library (~5 clips):** **~1 hour** of overnight batch rendering.
- **Scale transitions:** ~N short generative-video morphs (one per ring edge), a few
seconds each — a separate, heavier offline batch (video synthesis > img2img), but
still overnight-local.
- **Cloud API: no longer required** for the build. (It remains an option if a
higher-quality video model than a local one is wanted for a final pass.)
This also tightens the piece's "all-local" ethos: not just *runtime* is local
(§1 of the prior design) — now *authoring* is too.
---
## 5. POC evidence (this session)
A throwaway spike (outside the repo, `~/hef-poc/`) validated the full engine on one
real nature clip (a 4K Yosemite waterfall, trimmed to 8 s @ 1080p):
- **All four axes rendered** and read as distinct: Dark (cold/somber), Light
(warm/serene), Left (analytical HUD overlay), Right (painterly).
- **Right per-frame:** ~3.4 min/8 s clip, **flickers badly** (disqualifying).
- **Right flow-propagated:** ~2.7 min/8 s clip, **calm** (operator-approved).
- **Deterministic axes:** ~2.4 s each, ~3× faster than real-time → confirmed
runtime-capable.
- **Stack:** `imageio-ffmpeg`, `diffusers` + `sd-turbo` on MPS, OpenCV Farneback
flow; Python 3.13; 64 GB unified memory comfortably ran models that OOM consumer
GPUs.
---
## 6. Open questions (for the plan, not blockers)
- **Flow quality at scale:** the OpenCV flow propagation was validated on one short
clip; longer clips / faster motion may need shorter keyframe intervals, bidirectional
blending, or a stronger flow model (RAFT).
- **Painterly strength:** the POC kept the restyle gentle; the dreamlike *range* and
the per-axis restyle-strength count (the §4.3 "small set of variants") are
unfixed.
- **Base-clip sourcing:** select and ingest the actual ~46 strictly-PD neutral
scale clips (NASA/NOAA/NPS) via sub-project 2.
- **Model choice:** `sd-turbo` was the POC's speed pick; a higher-quality local
model (or a final cloud pass) may be worth a comparison for the shipped variants.
- **Scale transitions:** generation method (first-last-frame i2v vs. infinite-zoom
outpainting) and local model; per-transition length; whether transitions carry the
current mood grade; the ring ordering of the scales; and behavior on fast or
continuous spins of the endless encoder (chain transitions vs. blended skip).
- **Pi compositing headroom:** confirm the Pi 5 can decode the altered video **and**
render the live Pango/HarfBuzz label overlay at the *actual* panoramic projector
resolution (ultra-wide / high-res) — low risk but unmeasured (see §1.2).
## 7. Out of scope (YAGNI)
- A single continuous **one-take** zoom through *all* scales (we use discrete neutral
clips joined by short AI transitions on a navigable ring — not one unbroken shot).
- Audio-axis alteration (already deferred by the prior §6).
- Cloud rendering pipeline (local supersedes it for the base build).
+25
View File
@@ -0,0 +1,25 @@
Feature: Synthetic fixture catalog
As a curator with an empty real catalog
I want a dense, representative stand-in catalog generated on demand
So that I can feel nearest-match behavior everywhere before any media is ingested
Scenario: The fixture catalog spans the coordinate space and all content modes
When the fixture catalog is generated
Then it contains records covering the brain plane and the mood plane
And it contains records in each of the "audio", "video", and "av" modes
And it contains a mix of "proposed" and "approved" records
Scenario: The fixture catalog is valid
When the fixture catalog is generated
Then every record passes hef.catalog validation
And no record references real media
Scenario: Generation is deterministic
Given a fixed seed
When the fixture catalog is generated twice
Then both runs produce identical records
Scenario: The real catalog is used when it is populated
Given catalog/library.jsonl is non-empty
When the simulator starts pointed at the real catalog
Then it loads the real records through the same loader instead of the fixtures
+32
View File
@@ -0,0 +1,32 @@
Feature: Feeling the model's own parameters
As a curator tuning the algorithm, not just the curation
I want to adjust the selection model's parameters live
So that I can feel how weighting, pool size, and the approved gate reshape picks
Background:
Given the simulator is loaded with the synthetic fixture catalog
And the dials are set to a fixed knob position with mode "av"
Scenario: Reweighting the brain plane can change the winner
Given the brain and mood weights are equal
And the current pick is recorded
When I increase the brain weight relative to the mood weight
Then the ranked pool may reorder so that brain-plane distance dominates
And the picked piece can differ from the recorded pick
Scenario: Pool size controls how many candidates are shown and shuffled
When I set the pool size to 1
Then the pool shows exactly the single nearest candidate
When I set the pool size to 5
Then the pool shows up to the five nearest candidates
Scenario: Approved-only narrows the pool to human-blessed records
Given the fixture catalog contains both "proposed" and "approved" records
When I turn the approved-only toggle on
Then every candidate in the pool has review_status "approved"
When I turn the approved-only toggle off
Then "proposed" records may appear in the pool again
Scenario: Default model parameters reproduce the shipped selection behavior
Given the weights are equal, the pool size is the default, and approved-only is off
Then the picked piece matches what hef.selection.select would return for the same dials
+48
View File
@@ -0,0 +1,48 @@
Feature: Curator's X-ray of the selection model
As a curator
I want to turn the installation's dials against a representative catalog and see
the full ranked pool the nearest-match algorithm picks from
So that I can judge whether the tagging rubric and selection surface fitting pieces
Background:
Given the simulator is loaded with the synthetic fixture catalog
And the fixture catalog spans the brain plane and the mood plane across all content modes
Scenario: A pick is always accompanied by its candidate pool
When I set the mode to "av" and the dials to left=1 right=3 dark=4 light=1
Then a piece is picked
And the ranked pool is shown nearest-first with each candidate's distance
And the picked piece is the rank-1 candidate in the pool
Scenario: Turning Dark up while Light stays low pulls the pick toward somber pieces
Given the mode is "av" and the dials are left=2 right=2 dark=0 light=0
When I raise the Dark dial from 0 to 4
Then the picked piece's dark coordinate is higher than before
And the pool's nearest candidates cluster toward the high-dark / low-light region
Scenario: Turning a dial moves the knob point on the coordinate map
Given the mode is "av"
When I change any of the Left, Right, Dark, or Light dials
Then the knob point on the brain or mood grid map moves to the new coordinate
And the candidate dots reflect the new neighborhood
Scenario: None mode is the void / rest state
When I set the mode to "none"
Then no piece is picked
And the pool is empty
And the screen shows the void / rest state
Scenario: A+V mode falls back to audio and video when av records are thin
Given the fixture catalog has fewer than the pool size of native "av" records near the knob point
When I set the mode to "av"
Then the pool includes "audio" and "video" records so it is not starved
Scenario Outline: Each content mode restricts the pool to eligible records
When I set the mode to "<mode>"
Then every candidate in the pool is eligible for "<mode>"
Examples:
| mode |
| audio |
| video |
| av |
+27
View File
@@ -0,0 +1,27 @@
Feature: Simulator service and API
As an operator
I want to run the simulator locally in one container and drive it over a small API
So that the browser X-ray and its behavior are testable without the physical stack
Scenario: The service runs locally in a container
When I start the simulator container
Then the X-ray is reachable in a browser on localhost
And no external network access is required
Scenario: Selecting returns the pick and the ranked pool
When I POST a valid selection request with dials and mode "av"
Then the response contains a picked record
And the response contains the ranked pool with a distance and rank per candidate
Scenario: None mode returns the void over the API
When I POST a selection request with mode "none"
Then the response pick is null
And the response pool is empty
Scenario: Invalid dial values are rejected
When I POST a selection request with a dial value outside 0 to 4
Then the API responds with a validation error
Scenario: Catalog metadata is available for the header
When I GET the catalog metadata
Then the response reports record counts, a per-mode breakdown, and the proposed/approved split
View File
+145
View File
@@ -0,0 +1,145 @@
"""Catalog data model, validation, and JSONL IO."""
from __future__ import annotations
from dataclasses import dataclass, asdict, fields
from typing import Optional
import json
from pathlib import Path
MODES = frozenset({"audio", "video", "av"})
LICENSES = frozenset({"public_domain", "cc0", "cc_by", "cc_by_nc"})
REVIEW_STATUSES = frozenset({"proposed", "approved"})
COORD_FIELDS = ("left", "right", "dark", "light")
COORD_MIN = 0
COORD_MAX = 4
ATTRIBUTION_LICENSES = frozenset({"cc_by", "cc_by_nc"})
class CatalogError(ValueError):
"""Raised when a catalog record is structurally invalid."""
@dataclass
class Record:
id: str
title: str
source_url: str
source_archive: str
license: str
mode: str
left: int
right: int
dark: int
light: int
duration_s: int
file_path: str
review_status: str = "proposed"
attribution: str = ""
resolution: str = ""
dominant_color: str = ""
rationale: str = ""
reviewed_at: Optional[str] = None
notes: str = ""
def validate(record: Record) -> None:
"""Raise CatalogError if the record is structurally invalid."""
if not record.id:
raise CatalogError("record id must be non-empty")
if record.mode not in MODES:
raise CatalogError(
f"invalid mode {record.mode!r}; expected one of {sorted(MODES)}"
)
if record.license not in LICENSES:
raise CatalogError(
f"invalid license {record.license!r}; expected one of {sorted(LICENSES)}"
)
if record.review_status not in REVIEW_STATUSES:
raise CatalogError(
f"invalid review_status {record.review_status!r}; "
f"expected one of {sorted(REVIEW_STATUSES)}"
)
for axis in COORD_FIELDS:
value = getattr(record, axis)
if isinstance(value, bool) or not isinstance(value, int):
raise CatalogError(f"coordinate {axis} must be an int, got {value!r}")
if not (COORD_MIN <= value <= COORD_MAX):
raise CatalogError(
f"coordinate {axis}={value} out of range {COORD_MIN}..{COORD_MAX}"
)
if record.duration_s < 0:
raise CatalogError("duration_s must be non-negative")
if record.license in ATTRIBUTION_LICENSES and not record.attribution:
raise CatalogError(f"license {record.license} requires attribution")
def record_to_dict(record: Record) -> dict:
"""Convert a Record to a plain dict suitable for JSON serialization."""
return asdict(record)
def record_from_dict(data: dict) -> Record:
"""Build a Record from a dict, rejecting unknown or missing fields."""
known = {f.name for f in fields(Record)}
unknown = set(data) - known
if unknown:
raise CatalogError(f"unknown fields: {sorted(unknown)}")
try:
return Record(**data)
except TypeError as exc:
raise CatalogError(str(exc)) from exc
def load_catalog(path) -> list[Record]:
"""Load and validate every record from a JSONL file. Blank lines skipped."""
path = Path(path)
records: list[Record] = []
with path.open("r", encoding="utf-8") as fh:
for lineno, raw in enumerate(fh, start=1):
line = raw.strip()
if not line:
continue
try:
data = json.loads(line)
except json.JSONDecodeError as exc:
raise CatalogError(f"line {lineno}: invalid JSON: {exc}") from exc
record = record_from_dict(data)
validate(record)
records.append(record)
return records
def save_catalog(records, path) -> None:
"""Validate and write all records to a JSONL file (overwrites)."""
path = Path(path)
with path.open("w", encoding="utf-8") as fh:
for record in records:
validate(record)
fh.write(json.dumps(record_to_dict(record), ensure_ascii=False) + "\n")
def append_record(record, path) -> None:
"""Validate and append a single record to a JSONL file."""
validate(record)
path = Path(path)
with path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(record_to_dict(record), ensure_ascii=False) + "\n")
def index_by_id(records) -> dict:
"""Map id -> Record, raising CatalogError on a duplicate id."""
idx: dict = {}
for r in records:
if r.id in idx:
raise CatalogError(f"duplicate record id: {r.id!r}")
idx[r.id] = r
return idx
def validate_catalog(records) -> None:
"""Validate every record AND cross-record invariants (currently: unique ids)."""
for r in records:
validate(r)
index_by_id(records) # raises on duplicate id
+130
View File
@@ -0,0 +1,130 @@
"""Nearest-match selection of a catalog record for a knob coordinate."""
from __future__ import annotations
import math
from dataclasses import dataclass
from hef.catalog import Record
SELECTOR_MODES = frozenset({"none", "audio", "video", "av"})
CONTENT_MODES = frozenset({"audio", "video", "av"})
@dataclass(frozen=True)
class Coordinate:
left: int
right: int
dark: int
light: int
@dataclass(frozen=True)
class Weights:
brain: float = 1.0
mood: float = 1.0
def distance(a: Coordinate, b: Coordinate, weights: Weights = Weights()) -> float:
"""Weighted Euclidean distance; brain plane and mood plane weighted separately."""
brain_sq = (a.left - b.left) ** 2 + (a.right - b.right) ** 2
mood_sq = (a.dark - b.dark) ** 2 + (a.light - b.light) ** 2
return math.sqrt(weights.brain * brain_sq + weights.mood * mood_sq)
def record_coordinate(record: Record) -> Coordinate:
"""The (left, right, dark, light) coordinate of a catalog record."""
return Coordinate(record.left, record.right, record.dark, record.light)
def candidates_for_mode(records, mode: str, pool_size: int) -> list[Record]:
"""Records eligible for a content mode.
For 'av', if fewer than pool_size native 'av' records exist, fall back to
including 'audio' and 'video' records so the pool is never starved.
"""
if mode not in CONTENT_MODES:
raise ValueError(
f"candidates_for_mode expects a content mode {sorted(CONTENT_MODES)}, "
f"got {mode!r}"
)
primary = [r for r in records if r.mode == mode]
if mode == "av" and len(primary) < pool_size:
extra = [r for r in records if r.mode in {"audio", "video"}]
return primary + extra
return primary
from typing import Optional
def ranked_candidates(
records,
coord: Coordinate,
mode: str,
*,
pool_size: int = 4,
weights: Weights = Weights(),
approved_only: bool = False,
) -> list[tuple[Record, float]]:
"""Mode-eligible records sorted nearest-first, each paired with its distance.
Applies the same approved_only filter and 'av' pool fallback as select(),
then returns up to pool_size nearest (record, distance) pairs. This is the
single source of truth for "the pool"; select() picks from it.
"""
if mode not in CONTENT_MODES:
raise ValueError(
f"ranked_candidates expects a content mode {sorted(CONTENT_MODES)}, "
f"got {mode!r}"
)
pool = records
if approved_only:
pool = [r for r in pool if r.review_status == "approved"]
candidates = candidates_for_mode(pool, mode, pool_size)
ranked = sorted(
candidates,
key=lambda r: (distance(coord, record_coordinate(r), weights), r.id),
)
nearest = ranked[:pool_size]
return [(r, distance(coord, record_coordinate(r), weights)) for r in nearest]
def select(
records,
coord: Coordinate,
mode: str,
*,
pool_size: int = 4,
weights: Weights = Weights(),
approved_only: bool = False,
rng=None,
) -> Optional[Record]:
"""Pick the record nearest `coord` for the given selector `mode`.
- 'none' selector mode returns None (the void/rest state).
- With rng=None, returns the single nearest record (ties broken by id) for
deterministic behavior. Pass a random.Random to shuffle within the
pool_size nearest records.
- approved_only restricts to records the human has blessed.
"""
if mode not in SELECTOR_MODES:
raise ValueError(
f"invalid selector mode {mode!r}; expected one of {sorted(SELECTOR_MODES)}"
)
if mode == "none":
return None
ranked = ranked_candidates(
records,
coord,
mode,
pool_size=pool_size,
weights=weights,
approved_only=approved_only,
)
if not ranked:
return None
nearest = [r for r, _ in ranked]
if rng is None:
return nearest[0]
return rng.choice(nearest)
View File
+147
View File
@@ -0,0 +1,147 @@
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
Reconciled slice (2026-06-07): the Right axis is a DISCRETE selection of a
pre-baked, flow-stabilized restyle variant (not a continuous blend), the Left
axis carries its knob LEVEL so a runtime annotation track can pick which labels
show, and a frozen `Calibration` parameterizes the knob->strength curves so they
can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION.
Layers compose per §4.2:
- Substrate: ColorGrade (Dark/Light mood, center = identity §5) + a pre-baked
Right restyle variant.
- Overlay: AnalyticalOverlay (Left), composited on top at runtime.
Left and Right stack (different layers); Dark/Light are the two poles of one
mood grade. See docs/superpowers/specs/2026-06-07-reconciled-simulator-
alteration-slice-design.md.
"""
from __future__ import annotations
from dataclasses import dataclass
from hef.selection import Coordinate
KNOB_MAX = 4 # knob full-scale (0..4)
def _clamp(x: float, lo: float, hi: float) -> float:
return max(lo, min(hi, x))
@dataclass(frozen=True)
class Calibration:
"""Tunable knob->strength curves (settled by eye in the sim, then baked).
- mood_gain: scales the signed Dark/Light tone (result clamped to [-1, 1]).
- overlay_gain: scales the Left overlay intensity (clamped to [0, 1]).
- right_variant_map: knob value (0..4) -> pre-baked Right variant index.
"""
mood_gain: float = 1.0
overlay_gain: float = 1.0
right_variant_map: tuple = (0, 1, 2, 3, 4)
# The LOCKED calibration (session 0010, 2026-06-07) — settled by eye in the
# simulator, closing the open session-0006 knob->strength decision. Convention:
# every experience knob runs 0 = off .. 4 = max, and equal Dark/Light = identity
# (raw footage); there is no "centered at 2 = no push" coordinate.
# - mood_gain = 1.0: full Dark/Light knob reaches the full mood grade. By-eye
# evidence (POC renders + sim, after the dark-grade fix this session) shows
# full tilt is peaceful on every axis, so no softening is warranted.
# - overlay_gain = 1.0: full Left = opacity 1.0; the HUD is legible, not
# overwhelming, sitting above the grade.
# - right_variant_map linear: the 5 knob notches map 1:1 onto the 5 discrete
# pre-baked Right strengths (0 = raw base).
# These are unity/linear by deliberate choice, not as placeholders. The curve
# stays parameterized so a future re-bake or a different feel is one edit away;
# test_default_calibration_is_locked guards the values from drifting silently.
DEFAULT_CALIBRATION = Calibration(
mood_gain=1.0,
overlay_gain=1.0,
right_variant_map=(0, 1, 2, 3, 4),
)
@dataclass(frozen=True)
class ColorGrade:
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
<0 cool blue->black (dark), 0 = identity (raw, ungraded)."""
tone: float
@property
def is_identity(self) -> bool:
return self.tone == 0.0
@dataclass(frozen=True)
class AnalyticalOverlay:
"""Left axis (§4.1): analytical HUD/annotation, composited on top at runtime.
`level` is the Left knob (0..4); a runtime annotation track uses it to pick
which labels appear. `intensity` 0..1 is the overlay opacity/strength."""
level: int
intensity: float
@dataclass(frozen=True)
class Restyle:
"""Right axis (§4.1): selects a pre-baked, flow-stabilized restyle variant.
`variant` is a discrete index (0 = raw base, no restyle)."""
variant: int
@dataclass(frozen=True)
class RenderPlan:
"""The full layered alteration for one knob vector (§4.2)."""
grade: ColorGrade
overlay: AnalyticalOverlay
restyle: Restyle
@property
def is_identity(self) -> bool:
"""True when the plan leaves the neutral base un-altered."""
return (
self.grade.is_identity
and self.overlay.level == 0
and self.restyle.variant == 0
)
def _overlay_intensity(left: int, cal: Calibration) -> float:
return _clamp(cal.overlay_gain * left / KNOB_MAX, 0.0, 1.0)
def _right_variant(right: int, cal: Calibration) -> int:
return cal.right_variant_map[right]
def _mood_tone(dark: int, light: int, cal: Calibration) -> float:
return _clamp(cal.mood_gain * (light - dark) / KNOB_MAX, -1.0, 1.0)
def plan_alteration(
coord: Coordinate, calibration: Calibration = DEFAULT_CALIBRATION
) -> RenderPlan:
"""Map a knob vector to its layered RenderPlan (design §4)."""
return RenderPlan(
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light, calibration)),
overlay=AnalyticalOverlay(
level=coord.left,
intensity=_overlay_intensity(coord.left, calibration),
),
restyle=Restyle(variant=_right_variant(coord.right, calibration)),
)
def render_plan_to_dict(plan: RenderPlan) -> dict:
"""JSON-serializable form for the simulator API."""
return {
"grade": {"tone": plan.grade.tone},
"overlay": {"level": plan.overlay.level, "intensity": plan.overlay.intensity},
"restyle": {"variant": plan.restyle.variant},
"is_identity": plan.is_identity,
}
+42
View File
@@ -0,0 +1,42 @@
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
The single source of truth for design §6's table: which audio source plays and
whether the projector shows video, for each dial position.
"""
from __future__ import annotations
from dataclasses import dataclass
# The four distinct audio sources. "white_noise" is generated at runtime;
# "music" is the public-domain classical pool; "audio_track" is the clip's own
# field audio; "none" is silence.
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
@dataclass(frozen=True)
class ContentResolution:
audio_source: str
video: bool
# Design §6, one row per dial position.
_TABLE = {
"off": ContentResolution("none", False),
"white_noise": ContentResolution("white_noise", False),
"music": ContentResolution("music", False),
"audio_track": ContentResolution("audio_track", False),
"video": ContentResolution("none", True),
"music_video": ContentResolution("music", True),
"audio_video": ContentResolution("audio_track", True),
}
def resolve_content(position: str) -> ContentResolution:
"""Map a content-dial position to its audio source and video flag (§6)."""
try:
return _TABLE[position]
except KeyError:
raise ValueError(
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
) from None
+68
View File
@@ -0,0 +1,68 @@
"""Control-panel state: the data shape read from the Arduino over serial.
This is the serial-contract payload shared with sub-project 4 (firmware). It
models the full panel from design §6/§7: the 7-way content dial, the four
experience knobs (0..4), and the two intensity levels (volume, brightness).
The wire framing itself is the separate 3<->4 serial contract; this module is
the *decoded* form.
"""
from __future__ import annotations
from dataclasses import dataclass, fields
# The seven positions of the content dial (design §6).
CONTENT_POSITIONS = frozenset(
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
)
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
KNOB_MIN = 0
KNOB_MAX = 4
class ControlsError(ValueError):
"""Raised when a Controls payload is structurally invalid."""
@dataclass(frozen=True)
class Controls:
content: str
left: int
right: int
dark: int
light: int
volume: int
brightness: int
def validate_controls(c: Controls) -> None:
"""Raise ControlsError if the payload is structurally invalid."""
if c.content not in CONTENT_POSITIONS:
raise ControlsError(
f"invalid content position {c.content!r}; "
f"expected one of {sorted(CONTENT_POSITIONS)}"
)
for name in KNOB_FIELDS:
value = getattr(c, name)
if isinstance(value, bool) or not isinstance(value, int):
raise ControlsError(f"knob {name} must be an int, got {value!r}")
if not (KNOB_MIN <= value <= KNOB_MAX):
raise ControlsError(
f"knob {name}={value} out of range {KNOB_MIN}..{KNOB_MAX}"
)
def parse_controls(data: dict) -> Controls:
"""Build a validated Controls from a decoded mapping, rejecting unknown or
missing keys."""
known = {f.name for f in fields(Controls)}
unknown = set(data) - known
if unknown:
raise ControlsError(f"unknown keys: {sorted(unknown)}")
missing = known - set(data)
if missing:
raise ControlsError(f"missing keys: {sorted(missing)}")
c = Controls(**data)
validate_controls(c)
return c
+120
View File
@@ -0,0 +1,120 @@
"""The player state machine: a stream of Controls -> playback transitions.
Pure decision logic no mpv, no audio, no serial. Each update() resolves the
desired Playback (which neutral base clip, how it is altered, what audio plays,
at what levels) and returns the Transition from the previous Playback. The
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
pre-baked Right restyle variant (a discrete index) needs a CROSSFADE, and
toggling video on/off fades to/from black.
Knobs no longer *select* a clip (the base library is neutral by construction);
they *transform* it. Which neutral base to show is an injected policy
(`choose_base`), defaulting to the first clip; richer rotation is a later slice.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Optional
from hef.selection import Coordinate
from player.alteration import RenderPlan, plan_alteration
from player.content import ContentResolution, resolve_content
from player.controls import Controls
class TransitionKind:
NONE = "none"
LIVE_UPDATE = "live_update"
CROSSFADE = "crossfade"
FADE_TO_BLACK = "fade_to_black"
FADE_FROM_BLACK = "fade_from_black"
@dataclass(frozen=True)
class Playback:
"""What should currently be playing."""
clip_id: Optional[str] # None = black walls
plan: Optional[RenderPlan] # None when black
content: ContentResolution
volume: int
brightness: int
@dataclass(frozen=True)
class Transition:
kind: str
playback: Playback
def _first(library):
if not library:
raise ValueError("no base clips available to play video")
return library[0]
_BLACK = Playback(
clip_id=None,
plan=None,
content=ContentResolution("none", False),
volume=0,
brightness=0,
)
class Player:
def __init__(
self,
base_library,
*,
choose_base: Callable = _first,
approved_only: bool = False,
):
self._library = list(base_library)
self._choose_base = choose_base
self._approved_only = approved_only # reserved for catalog-backed libs
self._current = _BLACK
def _resolve(self, controls: Controls) -> Playback:
content = resolve_content(controls.content)
if not content.video:
return Playback(
clip_id=None,
plan=None,
content=content,
volume=controls.volume,
brightness=controls.brightness,
)
clip = self._choose_base(self._library)
coord = Coordinate(controls.left, controls.right, controls.dark, controls.light)
return Playback(
clip_id=clip.id,
plan=plan_alteration(coord),
content=content,
volume=controls.volume,
brightness=controls.brightness,
)
def _classify(self, prev: Playback, nxt: Playback) -> str:
prev_video = prev.clip_id is not None
next_video = nxt.clip_id is not None
if prev == nxt:
return TransitionKind.NONE
if prev_video and not next_video:
return TransitionKind.FADE_TO_BLACK
if next_video and not prev_video:
return TransitionKind.FADE_FROM_BLACK
if next_video and prev_video:
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
return TransitionKind.CROSSFADE
return TransitionKind.LIVE_UPDATE
# both black: only audio/levels could have changed
return TransitionKind.LIVE_UPDATE
def update(self, controls: Controls) -> Transition:
nxt = self._resolve(controls)
kind = self._classify(self._current, nxt)
self._current = nxt
return Transition(kind=kind, playback=nxt)
+26
View File
@@ -0,0 +1,26 @@
[project]
name = "human-experience-filter"
version = "0.1.0"
description = "Coordinate-tuned public-domain media installation"
requires-python = ">=3.11"
[tool.pytest.ini_options]
testpaths = ["tests"]
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
packages = ["hef", "tools", "simulator", "player"]
[project.scripts]
hef-ingest = "tools.ingest_cli:main"
hef-review = "tools.review_cli:main"
[project.optional-dependencies]
sim = [
"fastapi>=0.110",
"uvicorn[standard]>=0.29",
"httpx>=0.27",
]
@@ -0,0 +1,147 @@
# Session 0001.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-04T05-36 (PST) · End: 2026-06-04T06-15 (PST)
> Type: spec
> Goal: Develop the spec for sub-project 2 — "Ingest & Tagging / Review tools".
> Outcome: Spec written, self-reviewed, and operator-approved; implementation
> plan written; design spec + ROADMAP revised for a mid-session display
> architecture change. All merged to `main` and pushed; tree clean.
## Launch prompt
```
/goal Develop the spec for sub-project 2 — "Ingest & Tagging / Review tools" — of
the Human Experience Filter, the next item in docs/ROADMAP.md. Ground it in
docs/superpowers/specs/2026-06-04-human-experience-filter-design.md (especially §5
division-of-labor and §8 sourcing) and the roadmap's sub-project-2 section.
The spec must cover: per-archive ingest fetchers (Internet Archive/Prelinger,
Musopen, LibriVox, NASA, Free Music Archive, Freesound); automatic mechanical
tagging (mode via ffprobe, license/attribution/source from origin, duration_s,
resolution, dominant_color computed from video); assistant coordinate-drafting
that writes a one-line rationale and sets review_status=proposed; and the review
CLI that walks proposed records (proposed coords + rationale + preview frame) and
flips them to approved with reviewed_at. It must build on the existing
hef.catalog API without breaking it — call out explicitly any additions the
tools need.
```
## Plan
Spec session — develop the sub-project-2 spec (Ingest & Tagging / Review tools),
grounded in the approved design doc (§5 division-of-labor, §8 sourcing) and
ROADMAP §2. Cover: per-archive fetchers, mechanical tagging, coordinate drafting,
review CLI. Hard constraint: build on `hef.catalog` without breaking it; call out
additions explicitly. Done when spec written + self-reviewed + approved; any plan
saved under `plans/`; tree clean, merged to `main`, branches deleted, pushed.
Mid-session the operator added two follow-on asks: write the implementation plan,
and update the design spec for a new display decision.
## Pre-session state
- Sub-project 1 (Catalog & Selection Core) done and merged: `hef/catalog.py`
(`Record`, `validate`, JSONL IO) + `hef/selection.py` (distance, mode filter,
`select`), 37 passing tests.
- Repo was **not yet registered** for the session protocol — no `app.json`, no
`CLAUDE.md`. Clean `main`, pushed.
## Turn-by-turn arc
1. **Session init (spec).** Got start time; surveyed git (clean `main`); read
ROADMAP, memory (none), `app.json`/`CLAUDE.md` (absent). Resolver exited 3
(repo unregistered). Authored `app.json` (One Name = `human-experience-filter-art`;
sessions self-contained in this repo under `sessions/`, matching the benstull
`docs` convention) and committed it to `main` (`4ddc113`). Created `CLAUDE.md`
importing `@~/.claude/wiggleverse.md` (`5e4b794`, after a rebase — the claim
push had advanced `main`). Claimed session **0001** with `--type spec`; no
concurrent sessions. Seeded the transcript `## Plan`.
2. **Orientation.** No BDD corpus (`BDD_REMOTE` empty), so grounding = the design
spec, per the launch prompt. Read the design spec, `hef/catalog.py`,
`hef/selection.py`, the sub-project-1 plan, and `USER_GUIDE.md`.
3. **Key realization.** The `Record` schema already ships every field the
draft-then-review flow writes, with defaults (`review_status="proposed"`,
`reviewed_at=None`) that ARE the proposed shape — so the schema needs no change.
4. **Wrote the spec** (`docs/superpowers/specs/2026-06-04-ingest-tagging-review-tools.md`,
`f80ec65`) on branch `spec/sub-project-2-ingest-review`: 12 sections covering
the `Candidate`/`Fetcher` seam + all six archives, ffprobe/ffmpeg mechanical
tagging (with cover-art guard), per-archive license normalization, the
deterministic `HeuristicProposer` (no ML, honoring design §11), the review
transition core + interactive CLI. Single required `hef.catalog` addition:
`validate_catalog` (unique-id integrity), optional `index_by_id`.
5. **Self-review.** Verified against the live code: no name collision for the new
symbols, all 37 existing tests still green, every launch-prompt requirement
covered.
6. **Approval (AskUserQuestion).** Operator **approved as written**, confirmed
first-ship archives (LibriVox + NASA + IA), and — on the color question —
dropped a design change: *single panoramic projector across all three walls,
nature-video content, "may not need any video manipulation at all."*
7. **Folded the change into the spec** (`4c1ec6e`): demoted `dominant_color` to
optional/opt-in (its only consumer was the side walls), ffmpeg-only when
computed. Merged the spec to `main` (`3544092`), deleted the branch, pushed.
Linked the spec from ROADMAP §2 (`b5f5ee6`).
8. **Operator chose all three follow-ups:** plan + design-spec update + finalize.
On branch `docs/pano-update-and-sp2-plan`: revised the design spec (banner;
§1, §6 single pano projector, §7 procedural side walls REMOVED, §10, §12) and
ROADMAP (sub-project 5 Dropped; diagram; §3 player → pano; cross-cutting
decisions) (`dd02c2c`); wrote the task-by-task implementation plan
(`docs/superpowers/plans/2026-06-04-ingest-tagging-review-tools.md`, `6d6cbb5`).
Merged to `main` (`dd00d8b`), deleted the branch, pushed.
## Cut state (end of session)
Branch `main`, clean, `0/0` vs `origin/main`. No feature branches. Key commits
(all on `git.benstull.org:benstull/human-experience-filter-art`):
| SHA | What |
|-----|------|
| `4ddc113` | register `app.json` |
| `5e4b794` | `CLAUDE.md` org import |
| `f80ec65` | sub-project-2 spec (initial) |
| `4c1ec6e` | spec: fold in approval + pano change |
| `3544092` | merge spec → main |
| `b5f5ee6` | ROADMAP links the spec |
| `dd02c2c` | design + ROADMAP pano revision |
| `6d6cbb5` | sub-project-2 implementation plan |
| `dd00d8b` | merge pano revision + plan → main |
(Plus the session-protocol commits `18f0360`/`54fde0b`/`b8c406d` for the
transcript itself.)
Tests: untouched this session — still 37 green (no code changed; docs only).
## Deferred decisions
- **Major design change surfaced mid-approval (operator, 2026-06-04):** single
panoramic projector spanning all three walls, content = nature videos, "we may
not need any video manipulation at all." Handled in the sub-project-2 spec by
demoting `dominant_color` to optional/opt-in; then (operator-requested) the
design spec §6/§7 and ROADMAP were revised and sub-project 5 dropped. Confirmed
by the operator during the session.
- Operator approved the spec "as written"; remaining §11 defaults taken as
accepted: `file_path` relative-to-media-root, two `python -m` CLIs, `index_by_id`
in `hef.catalog`. Revisit at implementation if any feels wrong.
- **spec-RFC submission gap:** `app.json` has no `contains:["spec-rfc"]` repo, so
the spec wasn't submitted to a spec-RFC renderer (it lives in-repo and is
pushed). Add a spec-rfc target to `app.json` if rendered specs are wanted later.
## What lands on the operator's plate
- Nothing blocking. The spec + plan are approved and merged.
- Optional: add a `contains:["spec-rfc"]` repo to `app.json` if you want specs
rendered/tracked outside this repo.
- The design spec + ROADMAP now describe the single-pano-projector / nature-video
direction; confirm that matches your intent when you next look.
## Prompt the operator can paste into the next session
```
/executing-plans docs/superpowers/plans/2026-06-04-ingest-tagging-review-tools.md
```
This implements sub-project 2 (Ingest & Tagging / Review tools) task-by-task.
First read the memory files `sub-project-2-ready-to-build.md` and
`pano-projector-nature-video.md`. Task 1 scaffolds `tools/`; Task 2 is the only
`hef.catalog` change (`validate_catalog` + `index_by_id`). Needs `ffmpeg`/`ffprobe`
installed for the opt-in integration tests; the unit suite is hermetic.
@@ -0,0 +1,103 @@
# Session 0002.0 — Transcript
> App: human-experience-filter-art
> Type: coding
> Start: 2026-06-04T06-19 (PST) · End: 2026-06-04T07-34 (PST)
> Goal: Execute the sub-project-2 implementation plan (ingest & tagging / review tools).
> Outcome: **Shipped.** All 14 plan tasks built TDD-first and merged to `main`
> via PR #1; roadmap status updated via PR #2. Suite: 111 passed, 2 skipped.
## Plan
Execute `docs/superpowers/plans/2026-06-04-ingest-tagging-review-tools.md` via
`superpowers:executing-plans`: build the `tools/` package — per-archive ingest
fetchers, mechanical tagging (ffprobe/ffmpeg + license), heuristic coordinate
drafting (`review_status=proposed`), and an interactive review CLI that flips
records to `approved`. The only `hef/` change is additive: `validate_catalog` +
`index_by_id` (spec §3). 14 tasks, TDD throughout; unit suite stays hermetic
(network + binaries injected/faked); opt-in real-ffprobe tests skip when absent.
## Pre-session state
- Clean `main`, synced with `origin/main`; 37 tests green (sub-project 1:
`hef.catalog` + `hef.selection`).
- Memory pointer said: spec + plan for sub-project 2 done & merged; next step is
`/executing-plans` on the sub-project-2 plan.
- ffmpeg/ffprobe **absent** on the build machine (so opt-in integration tests
would skip — by design).
## Turn-by-turn arc
1. **Session gate → coding init.** Classified the `/executing-plans` launch as a
coding session; ran `wgl-coding-session-init`. Claimed session **0002** (dry-run
peek showed nothing in flight), verified clean `main` baseline, confirmed the
`CLAUDE.md` context import.
2. **Plan review.** Read the spec + plan + existing `hef/catalog.py`,
`hef/selection.py`, `pyproject.toml`, `conftest.py`. Critical review: no
blockers — additions are purely additive, all referenced symbols exist.
3. **Workspace.** `using-git-worktrees` → chose a **feature branch**
(`sub-project-2-ingest-tagging-review`) over a worktree: no parallel session,
and `EnterWorktree` is gated to explicit requests. Empty `conftest.py` puts the
repo root on `sys.path`; `hef` not installed editable, so the branch exercises
its own code with the shared `.venv` (pytest-only dep).
4. **Tasks 114, TDD (red → green → commit each):**
- T1 scaffold `tools/` + `http.py`; T2 `hef.catalog.validate_catalog` +
`index_by_id`; T3 `probe.py`; T4 `tagging.py` (cover-art guard +
round-half-up duration — see deferred decision); T5 `mediatools.py`
(frame + opt-in ffmpeg-only dominant_color); T6 `licensing.py`; T7
`drafting.py` `HeuristicProposer`; T8 `ingest/base.py` pipeline; T9
LibriVox/NASA/Internet Archive fetchers + Musopen/FMA/Freesound deferred
stubs; T10 `ingest_cli.py`; T11 `review.py` transition core; T12
`review_cli.py` (I/O seams injectable → scripted accept/edit/skip/quit tests);
T13 hermetic e2e + opt-in real-ffprobe (skips cleanly); T14 USER_GUIDE.
- One mid-flight catch: a `| tail` masked a pytest failure on T4 so a broken
commit landed; fixed the round-half-up logic and `--amend`ed (branch unpushed).
5. **Finish branch.** `finishing-a-development-branch` → operator chose **Push PR +
auto-merge**. Gitea host (`git.benstull.org`), so no `gh`. No API token existed;
searched both keychains (service names only, no secret bytes) — confirmed none
for benstull.org. Operator minted + stored a `write:repository` PAT via
`pbpaste` into `wgl-gitea-token-git.benstull.org`. Created **PR #1** and merged
it (merge commit `d7a2cba`), deleted the branch, FF'd local `main`.
6. **Finalize.** Survey clean. Updated memory. Marked the roadmap (§2 ✅ done,
§3 ⏳ next) via **PR #2** (`593150b`). Published this transcript.
## Cut state (end of session)
- Branch: `main` @ `593150b`, clean, synced with `origin/main`.
- **PR #1** (`d7a2cba`) — sub-project 2 `tools/` package + additive `hef.catalog`
symbols + USER_GUIDE. 14 commits.
- **PR #2** (`593150b`) — roadmap status update.
- Tests: `python -m pytest -q`**111 passed, 2 skipped** (opt-in real-ffprobe;
ffmpeg absent here). Existing catalog still validates under `validate_catalog`.
- New surface: `tools/{http,probe,tagging,mediatools,licensing,drafting,review,
review_cli,ingest_cli}.py`, `tools/ingest/{base,librivox,nasa,internet_archive,
musopen,fma,freesound}.py`; `hef.catalog` gained exactly `validate_catalog` +
`index_by_id`.
## Deferred decisions
- **Task 4 duration rounding (low risk):** spec/plan say `round(float(duration))`
and the plan's test expects `12.5 → 13`. Python's `round()` uses banker's
rounding (`round(12.5) == 12`), so implemented round-half-up
(`math.floor(x + 0.5)`) to match the plan's stated behavior. Chose the
documented expectation over the literal function name. Confirm or redirect if
truncation/banker's was actually intended.
## What lands on the operator's plate
- A `write:repository` Gitea PAT for `git.benstull.org` now lives in the Keychain
(`wgl-gitea-token-git.benstull.org`) — reused for future PR automation; revoke
in the Gitea UI if/when you want it gone.
- Live ingest/review need `ffmpeg`/`ffprobe` installed (absent on this machine);
the unit suite does not. Freesound (when un-deferred) needs
`FREESOUND_API_TOKEN` (env-only secret).
## Prompt the operator can paste into the next session
Sub-project 3 (Pi player) has no plan yet and §3 carries open decisions (player
stack, approved-only enforcement, serial protocol contract) — so it likely opens
as a spec/discovery session before coding.
```
/goal Develop sub-project 3 — the Pi Player Runtime that drives the single panoramic projector from the approved catalog — starting from its open decisions (player stack: mpv-IPC vs ffmpeg vs custom; approved-only enforcement; serial-protocol contract with sub-project 4; file_path-vs-media-mount resolution), per docs/ROADMAP.md §3. Read memory sub-project-2-ready-to-build.md and pano-projector-nature-video.md first.
```
@@ -0,0 +1,99 @@
# Session 0003.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-04T07-57 (PST)
> End: 2026-06-05T03-36 (PST)
> Type: discovery
> Status: **FINALIZED**
## Launch prompt
> We should create a simulator, just something that is web-based, potentially that
> runs on localhost, running in a Docker container. It allows us to play virtually
> with the dials and see the content that has been curated, and then we can get to
> the actual physical stack. I think we want to make sure that we have the
> principles well done and the interface figured out, and then the middle will come
> in to play.
## Plan
Discovery session: explore the proposed **web experience simulator** and produce
its first artifacts — BDD scenarios and/or a spec — plus (chosen mid-session) an
implementation plan. Orientation:
```
/goal Discover the web-based experience simulator — produce BDD scenarios and/or
a spec, per human-experience-filter-art/docs/ROADMAP.md
```
## Pre-session state
- `main` clean and pushed at `b7f19e0`. Sub-projects 1 (catalog/selection) and 2
(ingest/review) shipped; physical stack (3 Pi player, 4 firmware) ahead.
- `catalog/library.jsonl` empty (0 bytes) — tooling exists, no media ingested.
- No prior simulator artifacts. Memory pointed at sub-project 3 as next.
## Turn-by-turn arc
1. **Session gate / routing.** Opening prompt straddled coding vs. discovery;
asked the operator → **discovery**. Ran `wgl-discovery-session-init`: claimed
ID **0003** (`--type discovery`), verified clean pushed `main`, read memory,
oriented from `docs/ROADMAP.md`.
2. **Brainstorming.** Accepted the visual companion (server on :61057). Settled
scope via one-at-a-time questions:
- Primary purpose → **feel the selection model** (curator's instrument).
- Catalog source → **synthetic fixture catalog** (real catalog is empty).
- Screen layout (browser mockups A/B/C) → **B · Curator's X-ray** (full
transparency: pick + ranked pool + distances + brain/mood grid maps).
- Approved the **additive `hef.selection.ranked_candidates()`** change.
- **Include** the model knobs (weights, pool size, approved-only).
3. **Architecture.** Chose **Approach 1** — FastAPI + vanilla-JS, real
`hef.selection` as single source of truth; Docker on localhost.
4. **Artifacts written + committed** (`76a9526`): design spec
`docs/superpowers/specs/2026-06-04-experience-simulator-design.md` + 4 BDD
feature files under `features/`. Self-reviewed; fixed the pick-determinism
ambiguity (deterministic `rng=None` pick; pool = the room's shuffle-set).
Operator approved the spec.
5. **Implementation plan** (chose "draft the plan now"): wrote
`docs/superpowers/plans/2026-06-04-experience-simulator.md` — 7 bite-sized TDD
tasks; self-reviewed for spec coverage / placeholders / type consistency.
Committed (`e7ab227`).
6. **Memory + handoff.** Saved resume pointer
`experience-simulator-ready-to-build.md` (+ MEMORY.md index) so the next
coding session auto-surfaces the `/executing-plans` line.
7. **Finalize.** Push raced twice against concurrent session-claim commits (0003
then 0004); rebased onto each (disjoint paths, non-destructive) and pushed.
## Cut state (end of session)
- `main` synced with origin. Discovery artifacts landed:
- spec `40d7434``9cb4fb6` (rebased): `docs/superpowers/specs/2026-06-04-experience-simulator-design.md` + `features/*.feature`
- plan `19a2dbc``ddec45d` (rebased): `docs/superpowers/plans/2026-06-04-experience-simulator.md`
- Final `main` tip: **`ddec45d`**.
- No code written this session (discovery only). Test suite untouched (baseline
111 passed / 2 skipped).
- Visual-companion files persist under `.superpowers/brainstorm/` (gitignored).
## What lands on the operator's plate
- **Build is queued, not started.** The simulator exists only as spec + BDDs +
plan. Next session executes the plan.
- **Structural gap (flagged, not blocking):** this app's `app.json` has no
`contains:["bdd"]` or `contains:["spec-rfc"]` repo, so there is no external
corpus/spec-RFC target — the discovery artifacts live in-repo (committed) by
design. If a dedicated BDD/spec-RFC repo is ever wanted, register it in
`app.json`.
- **Concurrent session 0004** was claimed during this finalize and may still be
in flight.
- No deferred (low-confidence autonomous) decisions this session.
## Prompt the operator can paste into the next session
```
/goal Build the experience simulator — execute the plan task-by-task:
/executing-plans docs/superpowers/plans/2026-06-04-experience-simulator.md
```
Read `memory/experience-simulator-ready-to-build.md` first. The plan recommends
subagent-driven execution with review between tasks; the only shipped-code change
is the additive `hef.selection.ranked_candidates()` (Task 1).
@@ -0,0 +1,125 @@
# Session 0004.0 — Transcript
> App: human-experience-filter-art
> Type: coding
> Start: 2026-06-05T03-32 (PST) · End: 2026-06-05T03-56 (PST)
> Goal: Execute the experience-simulator (curator's X-ray) implementation plan.
> Outcome: **Shipped.** All 7 plan tasks built TDD-first and merged to `main`
> via PR #3. Suite: 133 passed, 2 skipped. Docker image builds + serves.
## Plan
Execute `docs/superpowers/plans/2026-06-04-experience-simulator.md` via
`superpowers:executing-plans`: build the `simulator/` package — a FastAPI service
wiring the five dials to the **real** `hef.selection` code and serving a
vanilla-JS "X-ray" (picked piece + ranked candidate pool with distances +
brain/mood 5×5 maps), fed by a deterministic synthetic fixture catalog since
`catalog/library.jsonl` is empty. The only `hef/` change is additive:
`hef.selection.ranked_candidates()` with `select()` refactored onto it. 7 tasks,
TDD throughout; Docker + compose + `make` targets package it for localhost.
## Pre-session state
- Clean `main`, synced with `origin/main` at `ddec45d`; 111 tests green, 2 skipped.
- Memory pointer: simulator spec + BDDs + plan produced in discovery 0003
(2026-06-04); next coding session runs `/executing-plans` on the simulator plan.
- Discovery session **0003** still `--INPROGRESS` (committed its artifacts to main
but never finalized) — dormant, not actively editing the tree.
- `fastapi` not yet installed; `python` not on PATH (use `.venv/bin/python`).
## Turn-by-turn arc
1. **Session gate → coding init.** Classified the `/executing-plans` launch as a
coding session; ran `wgl-session-coding-init`. Dry-run peek surfaced the
in-flight 0003 placeholder; autonomous default → noted it and proceeded.
Claimed session **0004**. Reconciled an initially-stale "ahead 2" read:
local `main` == `origin/main` == `ddec45d` (the 0003 spec/BDD/plan commits are
on origin). Verified clean baseline: **111 passed, 2 skipped**.
2. **Plan review.** Read the plan + `hef/selection.py`, `hef/catalog.py`,
`pyproject.toml`. Critical review: all assumed symbols exist
(`Coordinate`/`Weights`/`distance`/`candidates_for_mode`/`select`,
`CONTENT_MODES`/`SELECTOR_MODES`; `Record` carries every fixture field). No
blockers. Confirmed Docker present, `docs/USER_GUIDE.md` + `features/` exist.
3. **Workspace.** `using-git-worktrees` → native `EnterWorktree`
(`worktree-experience-simulator`, branched from `origin/main`). Fresh `.venv`
in the worktree; base package + pytest installed; baseline re-confirmed
**111 passed, 2 skipped**.
4. **Tasks 17, TDD (red → green → commit each):**
- **T1** `hef.selection.ranked_candidates()` + `select()` refactored onto it
(single source of truth; behavior-preserving) — 24 passed (new + existing
selection suite unchanged).
- **T2** declare `simulator` package + `[project.optional-dependencies] sim`
(fastapi/uvicorn/httpx); installed `-e ".[sim]"`; imports resolve.
- **T3** `simulator/fixtures.py` — deterministic 625-cell synthetic catalog
(all modes, mixed review status, no media) — 6 passed.
- **T4** `simulator/app.py` — FastAPI `create_app`, `POST /api/select`
(pick + ranked pool with distance/rank + coverage), `GET /api/catalog/meta`,
real-catalog-or-fixtures loader, guarded static mount — 6 passed.
- **T5** `simulator/static/` X-ray UI (dials, model knobs, ranked pool,
brain/mood maps) + static smoke test — 7 passed. Manual browser eyeball
substituted with live `curl` verification (no interactive browser): `/` 200,
`/api/catalog/meta` 625 records all-modes/mixed-status, `/api/select` ranked
pool ascending distances (1.0 → 1.41…), `none` → void, static assets 200.
- **T6** Dockerfile + compose + `Makefile` (literal-tab recipes). First build
**failed**: `package directory 'tools' does not exist``tools` is a
declared setuptools package but wasn't copied into the image. Root-caused
from the build log; fixed by adding `COPY tools ./tools`. Rebuilt → container
serves (`/` 200, 625-record fixture catalog). Torn down.
- **T7** USER_GUIDE "Playing with the simulator" section; **full suite: 133
passed, 2 skipped** (baseline 111 + 22 new).
5. **Finish branch.** `finishing-a-development-branch` (autonomous): pushed branch
as `experience-simulator`; created **PR #3** via the Gitea API
(`wgl-gitea-admin` token mechanism, `git.benstull.org`); merged it (merge
commit `618b691`); deleted the merged remote branch; FF'd local `main`;
re-verified the merged result **133 passed, 2 skipped**; removed the worktree
(all 7 commits confirmed ancestors of `main` first).
6. **Finalize.** Added `*.egg-info/` to `.gitignore` (editable-install artifact)
and pushed to `main` (`40dfbfd`); updated memory; published this transcript.
## Cut state (final)
- `main` @ `40dfbfd`, clean, synced with `origin/main`. No open PRs, no dangling
branches, worktree removed.
- **133 passed, 2 skipped.** Docker `make sim` builds and serves; `make sim-local`
runs uvicorn. Simulator is operator-runnable on localhost.
- The simulator is a **curator's discovery tool**, not a roadmap sub-project — the
roadmap frontier is unchanged.
## Deferred decisions
Autonomous-mode calls made without operator input:
1. **Dockerfile `COPY tools ./tools` (plan deviation).** The plan's Dockerfile
copied only `hef`/`simulator`/`catalog`, but `pyproject.toml` declares
`packages = ["hef", "tools", "simulator"]`, so the in-image editable install
failed. Added the missing COPY. *Alternative:* trim `tools` from the package
list — rejected (it's a real, used package). High confidence; the image now
matches the declared packages.
2. **Manual browser eyeball → programmatic curl (T5 Step 7).** No interactive
browser available, so verified the running server with `curl` (HTML, meta,
`/api/select`, void, static assets) instead of visually. Behavior fully
exercised, but no human visual confirmation of the rendered UI.
3. **Autonomous PR-merge instead of the interactive finish menu.** Per §6.5
autonomous default + §5.4 (branch→PR→merge), pushed + opened + merged PR #3
without surfacing the 4-option finish prompt. The preferred path was
unambiguous and the merge is revertible.
4. **`*.egg-info/` gitignore committed straight to `main`.** Trivial one-line
hygiene; landed directly rather than via PR.
## Next-session prompt
The simulator is shipped. The roadmap's next milestone is **sub-project 3
(Player Runtime / Pi)** — no spec/plan yet, so it starts with discovery.
```
/goal Develop sub-project 3 (Player Runtime / Pi): read the controls, call hef.selection.select(), play the chosen segment across the single panoramic projector, crossfade on change, dark on None — start with a discovery session to produce BDDs + spec, per docs/ROADMAP.md §3
```
Read first: memory `experience-simulator-ready-to-build.md` (what shipped + the
Dockerfile-copies-every-package gotcha), `pano-projector-nature-video.md` (single
pano projector, sub-project 5 dropped), and `docs/ROADMAP.md §3`. Open decisions
to settle: player stack (mpv/ffmpeg/custom), `approved`-only enforcement, and the
serial protocol framing shared with sub-project 4.
> Housekeeping: discovery session **0003** is still `--INPROGRESS` in
> `sessions/0003/` (artifacts long since merged) — finalize or close it out.
@@ -0,0 +1,126 @@
# Session 0005.0 — Transcript
> App: human-experience-filter-art
> Type: discovery
> Start: 2026-06-05T08-30 (PST) · End: 2026-06-05T17-48 (PST)
> Goal: Explore whether AI-generated/altered video is a viable content path for the
> pano projector — economics + fit with the design — and capture the outcome.
> Outcome: **Artistic thesis pivoted** and a full design-spec revision written,
> reviewed, and **merged to `main` via PR #4** (merge commit `77746e4`).
> Implementation deferred to a sub-project-3 coding session.
## Plan
Discovery session opened from the operator's question: "can we write/generate
scene descriptions and generate the video with AI — would the economics work?"
The session ran as a brainstorming exploration that converged on a new thesis and
produced a design spec as its artifact.
## Launch prompt
```
I'm not sure if hte economics would work out (can you do a best effort calculation?) but maybe we could write up/generate scene descriptions and generate the video with AI. What do you think?
```
## Pre-session state
- `main` @ `3510236`. Sub-projects 1 & 2 done/merged; roadmap frontier = sub-project
3 (Pi player). A stale, already-merged worktree (`feature/BuildSimulator11`,
same commit as main) was the session's launch cwd.
- Design of record: `2026-06-04-human-experience-filter-design.md` — single pano
projector + **found** nature video; §1 constraint "real found media, **not
algorithmically generated filler. Curation is the artwork.**"
## Turn-by-turn arc
1. **Gate / classify.** Operator's prompt was ambiguous between discovery and a
quick chat; asked → operator chose **Discovery (tracked)**. Ran
`wgl-session-discovery-init`.
2. **Claim snag (kept in the record).** `resolve-app.py` reported **ambiguous**
the nested worktree's `app.json` (`.worktrees/feature-BuildSimulator11/app.json`)
was discovered alongside the canonical main-repo `app.json`, both listing the
repo. Worked around with `WGL_APP_SCAN_DEPTH=3` (excludes the depth-5 worktree
copy). Claimed session **0005**, type discovery, at `850f67a`.
3. **Economics, v1 (the original question).** Researched 2026 AI-video API rates
(Sora 2 ~$0.10/s, Kling 3.0 ~$0.10/s, Veo 3.1 Lite/Runway Turbo ~$0.05/s, Veo 3
+audio ~$0.75/s). At the spec's scale (120800 pieces × ~10-min segments) →
**~$15k45k+** with retries; **not viable**, and it doesn't reduce the real
constraint (curation labor). Also flagged the §1 thesis conflict ("not generated
filler").
4. **Thesis pivot.** Operator: willing to change the thesis — the piece is "about
**how humans interact with machines and how it impacts their nervous systems**."
Under that thesis AI alteration *is* the subject. Invoked `brainstorming`.
Scope = **re-skin the meaning** (keep apparatus); content = **AI-altered
primary**.
5. **Economics, v2 (the flip).** New thesis unlocks two cost levers: small
**generate-to-order** library + short **seamless loops****~$3003k**. Feasible.
6. **The strong concept crystallizes.** Operator: start from **neutral** real
nature footage and let the machine **alter it toward the knob state** — "a great
expression of how machines alter our experience, good and bad." Established the
per-axis mapping (Left = analytical overlay; Right = generative v2v dissolve;
Dark/Light = color grade), the **substrate-vs-overlay composition rule**, and the
"whole-brain" corner (feeling, *labeled*).
7. **Control panel + i18n.** Operator added a 7-way content dial (Off/White
Noise/Music/Audio Track/Video/Music+Video/Audio Track+Video), volume + brightness
knobs, and a tactile **wooden** panel (engraved symbols, braille, color LEDs,
read-aloud button). Designed touch-distinct/engravable symbols. Key finding on
the **translation cost**: because the Left **labels are a runtime overlay**, the
v2v renders once (language-agnostic) and supporting *every* language is ~free on
the video — only cheap text translation + TTS. (Baking labels in would multiply
video cost × #languages — the avoided landmine.)
8. **Mood color grade.** Operator: Light = yellow / white negative space; center =
raw ungraded video; Dark = blue / black negative space. (First draft mis-set
Dark=red and center=grey; operator corrected → Dark = melancholy **blue** not
alarm red; center = **raw video**, no grade.)
9. **Wrote the spec.** Branched `feature/machine-altered-perception-spec` off main,
wrote `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md`
(15 sections), committed, self-reviewed; operator reviewed and gave 4
corrections (center=raw, add Music+Video → 7-way, audio-alteration deferred,
red gone) → applied, commit amended to `c5952c6`.
10. **Wrap.** Operator chose to wrap the discovery session here (spec is the
artifact; build later). Pushed branch; first PR-creation attempt was **blocked
by the auto-mode classifier** (treated as routing around PR authorization) —
surfaced it rather than working around. Operator then explicitly authorized
opening + merging the PR → **PR #4 opened and merged to `main`** (merge commit
`77746e4`, branch deleted), local `main` fast-forwarded.
## Cut state (end of session)
| Item | State |
|---|---|
| Design spec | **Merged to `main`**`docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` |
| Branch commit | `c5952c6` (spec, amended with review fixes) |
| Merge commit | `77746e4` (PR #4), feature branch deleted |
| `main` | fast-forwarded to `77746e4` (local + origin) |
| Roadmap | unchanged on disk; spec §13 describes how sub-projects 2/3/4 + a new offline v2v pipeline are reshaped (a roadmap-edit pass is future work) |
| Memory | `ai-generated-content-thesis-pivot.md` (full design rationale + status) + `MEMORY.md` index updated |
## What lands on the operator's plate
- Nothing blocking. The spec is canonical and merged.
- Optional future passes the spec itself lists (§14): pano resolution strategy,
negative-space luma key, transform composition order, neutral-base count,
language tier, runtime-grading perf on the Pi.
- A roadmap-edit pass to reflect the reshaped sub-projects 2/3/4 (the design spec
carries it for now).
## Deferred decisions
No substantive low-confidence design calls — every design decision (thesis,
per-axis mapping, composition rule, color grade, the 7-way dial, the panel) was
operator-confirmed in-session. Two mechanical judgment calls worth flagging:
- **`WGL_APP_SCAN_DEPTH=3` workaround** for the nested-worktree `app.json`
ambiguity in `resolve-app.py`. This is a genuine plugin friction (a worktree
nested under the main repo makes the resolver ambiguous) — a candidate for
`wgl-dev-plugin-feedback` (resolver should skip `.worktrees/`).
- **Repurposed the stale `feature/BuildSimulator11` worktree** to a fresh
`feature/machine-altered-perception-spec` branch off its HEAD (= main) rather
than removing it (harness had cwd anchored there; removal risked breaking the
shell).
## Prompt the operator can paste into the next session
```
/goal Implement sub-project 3 (the player / alteration engine) per docs/ROADMAP.md and the new docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md. Read memory ai-generated-content-thesis-pivot.md first. Build the alteration engine: runtime luma-keyed mood grade (Light=yellow/white → raw → Dark=blue/black), runtime analytical-overlay compositor (multilingual, from string tables), generated white noise, and selection/blending of pre-baked v2v variants over a small neutral base library; resolve the spec §14 open questions (pano resolution, composition order) as you go.
```
@@ -0,0 +1,134 @@
# Session 0006.0 — Transcript
> App: human-experience-filter-art
> Type: coding
> Start: 2026-06-05T17-52 (PST) · End: 2026-06-05T18-15 (PST)
> Goal: `/goal next` — drive the roadmap to its next frontier item (sub-project 3,
> the player / machine-alteration engine), and build it.
> Outcome: **Sub-project 3 slice 1 — the pure-logic alteration-engine + player
> core — built and MERGED to `main` via PR #5** (merge commit `aeeed66`).
> New `player/` package; 56 new tests (189 passed / 2 skipped total).
> Sub-project 3 advanced from "next" to "in progress."
## Plan
Begin sub-project 3 (Player Runtime + alteration engine) per `docs/ROADMAP.md` §3
and the approved machine-altered-perception design (§4 alteration engine, §5 mood
grade, §6 7-way content dial, §7 intensity). Sub-project 3 is large (runtime
grading + overlay compositing + v2v variant selection + serial input + mpv/GPU +
white-noise DSP); this session ships **slice 1 — the pure-logic core** as a new
`player/` package, fully unit-tested with all I/O behind injected interfaces,
mirroring how sub-project 1 was the pure-logic dependency root. Process:
writing-plans → TDD → branch → PR → merge.
## Launch prompt
```
/goal next — drive the roadmap to its next frontier item.
Per project memory (session 0005, machine-altered-perception design SPEC merged
to main via PR #4), the roadmap frontier is sub-project 3: the player /
machine-alteration engine. This coding session builds that.
```
## Pre-session state
- `main` @ `77746e4` locally (behind `origin/main` by 2 — session 0005's finalized
transcript + this session's claim placeholder); fast-forwarded to `origin/main`
during init.
- Sub-projects 1 & 2 done/merged; design of record = the machine-altered-perception
revision (PR #4). No `player/` package and no sub-project-3 plan existed yet.
- Launch cwd was the stale, already-merged worktree
`.worktrees/feature-BuildSimulator11/` (branch `feature/machine-altered-perception-spec`,
remote gone) — same situation as session 0005.
## Turn-by-turn arc
1. **Gate / classify.** `/goal next` → unambiguously a coding session (roadmap
frontier = sub-project 3 build). Ran `wgl-session-coding-init`.
2. **Claim snag (same as 0005, kept in the record).** `resolve-app.py` reported
**ambiguous** — the nested worktree's `app.json` discovered alongside the main
clone's. Read the resolver to confirm root cause (`find_app_jsons` prunes
`.git/node_modules/…` but not `.worktrees/`). Worked around with
`WGL_APP_SCAN_DEPTH=4` (prunes the depth-5 worktree copy, keeps the depth-3 main
one). Claimed session **0006**, type coding, at `e694489`. Filed a plugin-feedback
note for the recurring gap (`feedback/2026-06-05T18-13-resolver-nested-worktree-ambiguous.md`).
3. **Baseline + branch.** Fast-forwarded the main clone; confirmed PR #4 merged;
created `feature/player-alteration-core` off `origin/main` in the worktree.
4. **Read the design.** Read ROADMAP §3 + the machine-altered-perception spec
(§4 alteration engine, §5 mood grade, §6 7-way dial, §7 intensity), and the
existing `hef/` conventions (frozen dataclasses, pure modules, heavy unit tests).
5. **Scoped the session → slice 1.** Sub-project 3 is too large for one session;
chose the pure-logic core (controls model, content-dial resolver, alteration
render-plan, player state machine) with all I/O behind injected interfaces, and
deferred the renderer/serial/white-noise/v2v/catalog slices (named in the plan).
6. **Surfaced a spec ambiguity (Deferred decision).** §3 "(2,2,2,2) neutral" vs
§4.2 (Left/Right stack as independent layers) vs §5 (mood center = identity)
cannot all be literally true for four independent knobs. Resolved with a
swappable calibration: brain `value/4`, mood `(lightdark)/4`; flagged for
operator confirm.
7. **writing-plans.** Wrote `docs/superpowers/plans/2026-06-05-player-alteration-core.md`
(6 TDD tasks, full code + tests, self-review against the spec).
8. **TDD implementation.** Red→green→commit per module: `player/controls.py`
(Controls, validation, parse), `player/content.py` (§6 7-way table),
`player/alteration.py` (RenderPlan: ColorGrade/AnalyticalOverlay/Restyle +
calibration helpers), `player/state.py` (Player state machine + Transition
kinds). Registered the package in `pyproject.toml`; updated ROADMAP §3.
9. **Verify.** Full suite green: **189 passed, 2 skipped** (the opt-in ffprobe
tests), 56 of them new player tests. No regressions.
10. **Ship.** Pushed the branch; created **PR #5** and merged it via the Gitea API
helper (merge commit `aeeed66`); synced the main clone; deleted the remote
feature branch.
11. **Wrap.** Filed plugin feedback, updated memory, finalized.
## Cut state
- `main` @ `aeeed66` (PR #5 merged). `player/` package present: `controls.py`,
`content.py`, `alteration.py`, `state.py` + 4 test files. Full suite 189 passed /
2 skipped.
- Sub-project 3 = **⏳ in progress**. Slice 1 done; remaining slices listed in
ROADMAP §3 and `[[sub-project-3-player-progress]]`.
- Working tree clean. Remote feature branch deleted; the local worktree remains on
the now-merged `feature/player-alteration-core` branch (harmless).
## Deferred decisions (operator plate)
Two low-confidence calls made autonomously this session — both flagged for operator
confirmation:
1. **Knob→strength calibration (§3 vs §4.2/§5 conflict).** §3 says "(2,2,2,2) is
neutral," but §4.2 (Left/Right stack as independent layers: overlay vs substrate)
and §5 (mood center = identity) conflict with that for four independent physical
knobs. Implemented: brain knobs `strength = value/4` (0 = off, 4 = max — full
range, honors §4.2 stacking), mood `tone = (light dark)/4` (equal dark/light →
identity, honors §5). Under this the un-altered base is `(0,0,·,·)` with
`dark==light`, **not** `(2,2,2,2)` — read as a vestige of the old coordinate-grid
*center*. The calibration is three one-line helpers in `player/alteration.py`;
flipping to a "centered at 2 = no push" convention is a localized change that
leaves the RenderPlan shape and all downstream logic untouched. **Recommend
operator confirm the intended panel UX.**
2. **Volume/Brightness granularity.** Modeled as `0..4` ints for uniformity with the
experience knobs and the Arduino's `0..4` quantization (roadmap §4). The spec
calls them "levels" without fixing granularity; revisit if the panel warrants
finer steps.
## Next-session prompt
The roadmap frontier is now the remaining sub-project-3 slices. Read
`[[sub-project-3-player-progress]]` and the slice-1 plan first. Two strong
candidates for the next slice — the **runtime renderer** (makes the core visible)
or the **serial framing contract + input adapter** (unblocks sub-project 4 in
parallel):
```
/goal Sub-project 3 slice 2 — settle the 3⇄4 serial framing contract and build the
USB-serial / keyboard input adapter that feeds the player a `Controls` stream
(unblocks sub-project 4), per docs/ROADMAP.md §3 and docs/superpowers/plans/2026-06-05-player-alteration-core.md.
Alternatively start the runtime renderer (mpv/ffmpeg + grade/overlay). First
confirm the open knob→strength calibration decision (§3 vs §4.2/§5).
```
Gotcha to carry forward: running from the nested `.worktrees/…` checkout breaks
`resolve-app.py` (ambiguous app.json) — run from the main clone, or use
`WGL_APP_SCAN_DEPTH=4` on claim and explicit `--sessions-*` flags on publish, until
the plugin fix lands.
@@ -0,0 +1,101 @@
# Session 0007.0 — Transcript
> App: human-experience-filter-art
> Date: 2026-06-06 (PST) · 14:12 → 17:57
> Type: coding
> Goal: continue the roadmap; reframed by the operator to **simulator-first**
> bring the slice-1 alteration engine into the simulator so the experience can be
> tuned before any hardware.
> Outcome: **design spec written, committed, and pushed** on branch
> `feature/simulator-alteration-preview`. No code changes. Implementation deferred
> to the next session (design + implementation to ship as one PR).
## Plan
Bring the alteration engine into the simulator (scope: tune the look), per
`docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md`:
- Python-canonical engine, thin browser renderer.
- Dark/Light = live runtime color grade; Left/Right = 5×5 grid of pre-baked
authored variant clips selected by coordinate (24 videos/base clip).
- Revises design §4.3/§10: HUD baked into variant videos (authorial precision)
rather than runtime overlay — trades away near-free i18n.
- Engine reconciliation: parameterize `Calibration`, add `VariantRef`, slim
`RenderPlan` (drop runtime `AnalyticalOverlay`/`Restyle`); `state.py` crossfade
on variant change.
- Simulator: retire selection X-ray + `/api/select`; add `/api/alteration` +
`/api/clips` + variant manifest; placeholder-variant generator.
## Pre-session state
- `main` at `5290785`, clean and pushed. Sub-projects 1 & 2 done; sub-project 3
(Player Runtime) in progress — slice 1 (pure-logic `player/` core) merged via
PR #5 (session 0006).
- A **stale nested worktree** (`.worktrees/feature-BuildSimulator11`, branch
`feature/player-alteration-core`, already merged) was present and breaking the
session resolver ("ambiguous app.json"), exactly as the session-0006 memory
gotcha predicted.
## Turn-by-turn arc
1. **Session gate.** Opened with "What's next?" → classified as a coding session;
ran `wgl-session-coding-init`.
2. **Baseline cleanup.** Claim dry-run failed on the resolver ambiguity. Verified
the stale worktree's branch was fully merged + clean, removed the worktree and
branch, and deleted the empty `.worktrees/` dir. Tree back to clean `main`.
3. **Claimed session 0007** (no other sessions in flight).
4. **Operator steer:** "only design things that will run in the simulator; get
everything working the way we like in the simulator before hardware." Saved as
a `feedback` memory ([[simulator-first-before-hardware]]).
5. **Discovered** the simulator (`simulator/`, built session 0004) is fully
decoupled from `player/` — it exercises the OLD *selection* model and has no
alteration surface. That gap became the work.
6. **Brainstorming** (with the visual companion). Decisions, one at a time:
scope = tune the alteration look; medium = short looping video; Right restyle =
labeled stand-in shader (swappable for real variants); calibration =
live-adjustable curves; architecture = **A, Python-canonical**; integration =
new view, **retire the selection X-ray**.
7. **Interactive look mockup** (browser): a real nature image with the 4 knobs as
live sliders. Operator confirmed Dark/Light read well, but said for Left/Right
they'll **author videos with the overlay baked in** for precise HUD control.
8. **Surfaced the tradeoff:** baking the HUD reverses design §4.3/§10's near-free
i18n. Operator chose **precision wins**. Confirmed the variant grid as **full
5×5 = 24 authored clips per base** (shown as a grid mockup).
9. **Revised model** presented and approved: base clip + variant manifest;
`RenderPlan = {grade, variant}`; runtime overlay/restyle layers retired;
placeholder-variant bootstrapping via ffmpeg.
10. **Wrote the spec**, self-reviewed, committed on a feature branch.
11. Operator: "do [implementation] next session. finalize this one." →
`wgl-session-finalize`.
## Cut state (end of session)
| Repo | Branch | Commit | State |
|---|---|---|---|
| human-experience-filter-art | `feature/simulator-alteration-preview` | `3ef21fb` | pushed to origin, **not merged** |
- `docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md`
new, committed.
- `main` unchanged at `5290785`.
- Working tree clean. Visual-companion server stopped; `.superpowers/` gitignored.
- No tests run (no code changed).
## What lands on the operator's plate
- **Author the 24 variant videos** per base clip (the Left×Right grid), with HUD
baked in. The simulator will consume them via the manifest; placeholder variants
cover the gap until then.
- **Deferred decisions** (also surfaced in chat):
- *knob→strength calibration* — unresolved since session 0006; will be settled
by eye in the simulator's calibration panel next session, then baked into
`DEFAULT_CALIBRATION`.
- *grade tints baked HUD* — accepted under "precision wins"; revisit only if it
reads badly once real authored clips exist.
## Prompt the operator can paste into the next session
```
/goal Implement the simulator alteration preview on branch feature/simulator-alteration-preview, per docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md — begin with the writing-plans skill, then build: parameterize Calibration + add VariantRef and slim RenderPlan in player/, add /api/alteration + /api/clips + simulator/clips.py variant manifest, rewrite the simulator UI as the player preview, retire the selection X-ray + /api/select, add the placeholder-variant generator, and update tests + USER_GUIDE + ROADMAP §3 + the parent design §4.3/§10 pointer.
```
(Or resume with `/goal next` — the `Next /goal:` field is stored in memory
`sub-project-3-player-progress.md`.)
@@ -0,0 +1,102 @@
# Session 0008.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-07T19-42 (PST)
> End: 2026-06-07T22-37 (PST)
> Type: spec
> Status: **FINALIZED**
## Launch prompt
Session opened as `wgl-session-none` (research errand: "Let's find a great public
domain nature video for this project"), then upgraded mid-session to a **spec**
session once the work became tracked. Operator chose "Spec session, comprehensive":
fold the session's POC findings into the design AND resolve the open cosmic-zoom
structure fork.
## Plan
Develop a design revision to the machine-altered-perception design, grounded in a
local POC run this session: (1) Right-axis alteration pipeline; (2) content
structure (cosmic-zoom fork); (3) strictly-PD sourcing.
## Pre-state
- On `feature/simulator-alteration-preview` (session 0007's unmerged design), clean
tree; `main` at `5290785`.
- Catalog empty; `tools/ingest/internet_archive.py` is the only wired fetcher.
- No ML stack, no ffmpeg in the venv (pure-stdlib project).
## Session arc (uncurated)
1. **PD nature-video research.** Surveyed pools and verified licenses against the
actual metadata, not marketing. Key finding: **Pexels / Pixabay / Mitch Martinez
"free 4K" are royalty-free but NOT public domain** (restrict redistribution,
retain copyright) — exactly the trap the ingest tool's "no explicit license →
assume PD, verify" flag exists for. Genuinely-PD: NASA/Hubble (cosmos), **NOAA
Ocean Exploration** (deep sea, global), **NPS/USGS** (US land). Wikimedia nature
4K timelapses are mostly CC-BY, not PD.
2. **Operator pivot → "cosmic zoom".** Operator proposed a Powers-of-Ten journey
(space → continents → birds → ocean → abyss → microscopic → cosmos). Brainstormed
the thesis tension (a scale-journey carries built-in awe vs. the "neutral base"
thesis) and the cost (per-base pre-bake × altitudes). Surfaced that the *ends* of
the zoom are strict-PD-rich; the terrestrial *middle* (birds, non-US land) is the
CC-BY soft spot.
3. **"Can this run locally?" → POC.** Operator's machine: **Mac mini M4 Pro, 64 GB,
16-core GPU.** Built a throwaway POC in `~/hef-poc/` (outside the repo).
- *Wrong turns:* `brew install ffmpeg` and `pip install` were **denied by the
harness** (it blocks package installs) — switched to venv-local `imageio-ffmpeg`
run by the operator via `!`. zsh **doesn't word-split** unquoted `$EARGS`
mangled encoder flags; inlined them. sd-turbo img2img **crashed** ("reshape
tensor of 0 elements") because `int(steps×strength)=int(2×0.45)=0` denoise steps
→ guarded to ≥1.
- *Results on an 8s/1080p clip:* deterministic **Dark/Light/Left** grades/overlay
~2.4s each (~3× faster than realtime, runtime-capable). **Right** painterly
restyle (SD img2img on MPS) ~3.4 min/clip and **flickers badly**.
4. **Flicker is disqualifying.** Operator: "crazy and disorienting … this project is
meant to be peaceful." Prototyped two fixes: (a) a **deterministic** soft-dreamy
filter (smartblur + RGB bloom — first attempt had a magenta cast from blending on
YUV chroma planes; fixed by blending in RGB), ~5s, zero flicker by construction;
(b) **optical-flow keyframe propagation** (EbSynth principle; genuine ebsynth is
NVIDIA/Windows so implemented with OpenCV Farneback + the diffusers pipeline),
~2.7 min/clip, **calm**. Operator chose **AI + flow**.
5. **Spec session (comprehensive).** Upgraded the session via
`wgl-session-spec-init` → claimed 0008. Resolved the cosmic-zoom fork: operator
chose a **small NEUTRAL "scales of nature" library** (not a single stitched film).
Then operator added the **infinite-zoom ring** (AI zoom transitions between
scales, micro→cosmos wraparound) and the **infinitely-turnable endless encoder**.
Probed i18n feasibility → **§1.2**: Left HUD is a Pi-rendered **runtime overlay**
(Pango/HarfBuzz + Noto, annotation-track + per-language string tables), NOT a baked
per-language video.
- Wrote `docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-design.md`,
committed on `feature/scales-library-right-axis`.
## Cut state (what landed)
- **PR #6 merged to `main`** (merge `42a72fe`): the design revision doc.
- No spec-RFC submission: this app keeps specs in-repo (`app.json` `contains:["specs"]`),
so `submit-spec.sh` is N/A — surfaced, not dropped.
- POC artifacts left in `~/hef-poc/` (throwaway, outside repo): `restyle.py`,
`flow_restyle.py`, and the comparison clips/contact sheets.
- Memory updated: `sub-project-3-player-progress.md` (+ `MEMORY.md` index).
## Deferred decisions
- **Left-HUD treatment conflict (low confidence — flag).** 0008 §1.2 specifies the
Left HUD as a runtime Pango/HarfBuzz overlay (to keep i18n near-free, per parent
§10). This **reverses session 0007's decision** to bake the HUD into the 5×5
variant grid for authorial precision. I recommended the runtime-overlay path
following the operator's i18n question without flagging the 0007 reversal at the
time. `main` now carries the runtime-overlay position; the unmerged
`feature/simulator-alteration-preview` carries baked-HUD. **Reconciliation deferred
to next session** (pick one, or hybrid: authored positions + runtime-shaped text).
## Next /goal
```
/goal Reconcile the 0007 simulator-alteration-preview design (feature/simulator-alteration-preview, UNMERGED) with the merged 0008 scales-library + flow-stabilized Right-axis design — resolve the Left-HUD conflict (baked-into-5×5-grid vs runtime Pango/HarfBuzz overlay) FIRST — then writing-plans → build the simulator-first slice: deterministic Dark/Light/Left + a flow-stabilized Right restyle over ONE neutral scale clip wired into the simulator. Defer Pi renderer, endless-encoder/serial, and AI zoom-transitions per simulator-first.
```
@@ -0,0 +1,130 @@
# Session 0009.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-07T22-41 (PST)
> End: 2026-06-07T23-03 (PST)
> Type: coding
> Status: **FINALIZED**
## Launch prompt
Opened with `/goal next` — resume the goal recorded at the 0008 finalize. The
session gate resolved the stored `Next /goal:` from memory:
> Reconcile the 0007 simulator-alteration-preview design (UNMERGED) with the
> merged 0008 scales-library + flow-stabilized Right-axis design — resolve the
> Left-HUD conflict (baked-into-5×5-grid vs runtime Pango/HarfBuzz overlay) FIRST
> — then writing-plans → build the simulator-first slice: deterministic
> Dark/Light/Left + a flow-stabilized Right restyle over ONE neutral scale clip
> wired into the simulator. Defer Pi renderer, endless-encoder/serial, AI
> zoom-transitions.
## Plan
Brainstorm the reconciliation (resolve Left-HUD first) → write a unified design
doc → writing-plans → build & merge the simulator-first alteration slice.
## Pre-state
- On `main` at `c99a669`, clean tree; ran from the **main clone** (not a nested
worktree — avoiding the session-resolver ambiguity gotcha flagged in 0008).
- Two unmerged design threads: session-0007's
`feature/simulator-alteration-preview` (baked-HUD 5×5 grid, design doc only, no
code) vs. session-0008's merged scales-library/right-axis design (runtime-overlay
HUD). The Left-HUD conflict between them was the thing to resolve.
- Slice-1 engine (`player/`) on `main`: `RenderPlan {grade, overlay, restyle}` with
a continuous `Restyle.blend` and a runtime `AnalyticalOverlay`. Simulator still on
the old *selection* surface (`/api/select`, X-ray, `fixtures.py`).
- 0008 POC artifacts present in `~/hef-poc/out/` (outside repo): real `neutral.mp4`
base + `right_flow.mp4` flow-stabilized restyle.
## Session arc (uncurated)
1. **Claimed 0009** (coding) from the main clone; resolver worked cleanly. Read
both specs + memory in full.
2. **Surfaced the conflict precisely — it was 2-D, not 1-D.** Beyond the Left-HUD
rendering question, the two designs disagreed on the *shape of the pre-baked set*:
0007 made BOTH Left and Right a 5×5 grid of 24 authored clips; 0008 made only
Right pre-baked and Left a runtime overlay. Key realization: **0008 §1.2 already
IS the hybrid the operator floated** ("authored box positions + runtime-shaped
text") — authored annotation track for layout, runtime shaping for cheap i18n.
3. **Brainstorming → one operator question.** Asked the single load-bearing fork
(Left HUD: runtime-overlay+authored-track vs fully-baked vs defer). Operator chose
**runtime overlay + authored track** (the 0008 / hybrid position). Everything else
followed from approved material, so I proceeded autonomously.
4. **Found the engine was already most of the way there.** The merged slice-1 engine
already keeps a runtime `AnalyticalOverlay` — only 0007's *unmerged doc* proposed
removing it. So the reconciliation was surgical: continuous `Restyle.blend`
discrete `Restyle.variant`; add `AnalyticalOverlay.level`; add a frozen
`Calibration`. Wrote the reconciled design doc (brought the 0007 doc onto the
branch so its superseded position is preserved + links resolve), committed.
5. **writing-plans → a full TDD plan** (`docs/superpowers/plans/...`), then executed
it task-by-task with executing-plans:
- **Task 12 (engine):** `Calibration`+`DEFAULT_CALIBRATION` (behavior-preserving),
`Restyle.variant`, `AnalyticalOverlay.level`, `render_plan_to_dict`. `state.py`
needed no logic change (compares whole `Restyle`). Tests rewritten/green.
- **Task 3 (clips):** `simulator/clips.py` manifest model; retired `fixtures.py` +
`test_fixtures.py`.
- **Task 4 (API):** `/api/alteration` + `/api/clips`; removed `/api/select` +
`/api/catalog/meta`. *Wrong turn:* asserted retired POST returns 404, but the
static catch-all yields **405** for an unrouted POST — relaxed the assertion to
"404 or 405 = gone."
- **Task 5 (media):** `sample_media/manifest.json` + `setup_sample_media.py` that
copies the real POC `neutral.mp4`→base and `right_flow.mp4`→Right-strength-4 and
ffmpeg-blends placeholder strengths 13. mp4s gitignored. Ran it — 5 files
produced.
- **Task 6 (UI):** rewrote `simulator/static/` as the alteration preview (live
grade via CSS filters, Right-variant `<video>` crossfade, live SVG Left overlay
from the annotation track + string table, calibration panel, RenderPlan readout).
- **Task 7 (docs):** parent design §4.3/§10 pointers; ROADMAP slice 2 done +
deferred slices; USER_GUIDE simulator section rewritten.
- *Environment:* `python` not on PATH; used `.venv/bin/python` throughout.
6. **Verified end-to-end.** `pytest -q`**192 passed, 2 skipped**. Booted the sim
on a scratch port: `/api/clips` returns the manifest (variant 0→base), the engine
plan is correct by eye (left=3→level 3/intensity 0.75, right=4→variant 4,
dark=4→tone 1.0), and the **real** `right4.mp4` + base served as `video/mp4`.
7. **Shipped.** Pushed the branch, created **Gitea PR #7** via the keychain-token API
helper, **merged** it (autonomous posture), synced `main`, re-verified tests green
on the merged result, deleted my merged feature branch (local + remote).
## Cut state (what landed)
- **PR #7 merged to `main`** (merge `554eb50`): the reconciliation design + the built
simulator-first alteration slice (engine + simulator + media tooling + docs). 9
commits.
- `main` green: 192 passed / 2 skipped. Sim verified booting + serving real media.
- Memory updated: `sub-project-3-player-progress.md` (+ `MEMORY.md` index) — conflict
resolved, slice 2 shipped, new Next /goal.
- POC artifacts untouched in `~/hef-poc/` (still the source for `setup_sample_media.py`).
## Deferred decisions
- **Brought the 0007 design doc into `main` + intended to delete the superseded
branch.** Decided autonomously to carry `2026-06-06-simulator-alteration-preview-design.md`
forward (so the reconciliation's links resolve and the superseded position is
preserved) and to delete the now-stale `feature/simulator-alteration-preview`.
**The branch deletion (`git branch -D` / `push --delete`) was permission-denied by
the harness** — left in place. Safe to delete later (its content is in `main`); or
the operator may want to keep it. Flagging rather than forcing.
- **Merged under autonomous posture without a separate `/code-review`.** TDD + the
written plan were the quality gates; no independent review pass was run before
merge. Low risk for a slice this size, but noting it.
- **Used the POC's Yosemite clip as the sim's sample base.** It's a "forest"-scale
neutral clip, good enough to tune the look; framed in the manifest/USER_GUIDE as
**look-tuning only, not shipped content**. Strict-PD scale-library sourcing stays a
later slice — unaffected.
- **`Calibration` defaults are still behavior-preserving, not operator-tuned.** The
knob→strength calibration (open since session 0006) is now tunable by eye in the
sim but **not yet locked** — that's the next goal.
## Next /goal
```
/goal Tune the alteration look by eye in the simulator (python simulator/setup_sample_media.py then make sim-local) and LOCK the knob→strength calibration into DEFAULT_CALIBRATION in player/alteration.py (+ a unit test) — settling the open session-0006 calibration decision. While there, judge whether more neutral "scales of nature" base clips + a real multi-strength flow-stabilized Right re-bake are worth doing next vs. moving to scale-ring navigation (endless encoder + AI zoom transitions). Keep deferring Pi renderer + serial/firmware. Read sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md (§8) first.
```
@@ -0,0 +1,23 @@
# Session 0010.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-07T23-09 (PST)
> Type: coding
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
>
> This file reserves session ID 0010 for human-experience-filter-art. The driver replaces this
> body with the full transcript and renames the file to its final
> SESSION-0010.0-TRANSCRIPT-2026-06-07T23-09--<end>.md form at session end.
## Launch prompt
```
Tune the alteration look by eye in the simulator (python simulator/setup_sample_media.py then make sim-local) and LOCK the knob→strength calibration into DEFAULT_CALIBRATION in player/alteration.py (+ a unit test) — settling the open session-0006 calibration decision. While there, judge whether more neutral "scales of nature" base clips + a real multi-strength flow-stabilized Right re-bake are worth doing next vs. moving to scale-ring navigation (endless encoder + AI zoom transitions). Keep deferring Pi renderer + serial/firmware. Read sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md (§8) first.
```
## Deferred decisions
_Autonomous-mode low-confidence calls the driver made and would have
liked operator input on. Appended as the session runs; surfaced at
finalize. Empty if none._
+32
View File
@@ -0,0 +1,32 @@
{
"0001": {
"title": ""
},
"0002": {
"title": ""
},
"0003": {
"title": ""
},
"0004": {
"title": ""
},
"0005": {
"title": ""
},
"0006": {
"title": ""
},
"0007": {
"title": ""
},
"0008": {
"title": ""
},
"0009": {
"title": ""
},
"0010": {
"title": ""
}
}
+10
View File
@@ -0,0 +1,10 @@
FROM python:3.11-slim
WORKDIR /app
COPY pyproject.toml ./
COPY hef ./hef
COPY tools ./tools
COPY simulator ./simulator
COPY catalog ./catalog
RUN pip install --no-cache-dir -e ".[sim]"
EXPOSE 8000
CMD ["uvicorn", "simulator.app:app", "--host", "0.0.0.0", "--port", "8000"]
+1
View File
@@ -0,0 +1 @@
"""Web-based curator's X-ray simulator for the experience filter."""
+99
View File
@@ -0,0 +1,99 @@
"""FastAPI service: controls -> the real alteration engine -> a RenderPlan.
The simulator's alteration surface (reconciled slice). It calls the canonical
player.alteration.plan_alteration; the browser only renders. The selection-era
endpoints (/api/select, /api/catalog/meta) and the X-ray are retired.
"""
from __future__ import annotations
from pathlib import Path
from typing import Optional
from fastapi import FastAPI, HTTPException
from fastapi.staticfiles import StaticFiles
from pydantic import BaseModel, Field
from hef.selection import Coordinate
from player.alteration import (
DEFAULT_CALIBRATION,
Calibration,
plan_alteration,
render_plan_to_dict,
)
from player.content import resolve_content
from player.controls import CONTENT_POSITIONS
from simulator.clips import load_manifest
STATIC_DIR = Path(__file__).parent / "static"
MEDIA_DIR = Path(__file__).parent / "sample_media"
DEFAULT_MANIFEST = MEDIA_DIR / "manifest.json"
class ControlsModel(BaseModel):
content: str
left: int = Field(ge=0, le=4)
right: int = Field(ge=0, le=4)
dark: int = Field(ge=0, le=4)
light: int = Field(ge=0, le=4)
volume: int = Field(ge=0, le=4)
brightness: int = Field(ge=0, le=4)
class CalibrationModel(BaseModel):
mood_gain: float = 1.0
overlay_gain: float = 1.0
right_variant_map: list[int] = [0, 1, 2, 3, 4]
class AlterationRequest(BaseModel):
controls: ControlsModel
calibration: Optional[CalibrationModel] = None
def _load_clips(manifest_path: Optional[Path]):
path = Path(manifest_path) if manifest_path else DEFAULT_MANIFEST
if path.exists():
return load_manifest(path)
return []
def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
app = FastAPI(title="HEF Alteration Simulator")
app.state.clips = _load_clips(manifest_path)
@app.post("/api/alteration")
def api_alteration(req: AlterationRequest):
c = req.controls
if c.content not in CONTENT_POSITIONS:
raise HTTPException(status_code=422, detail=f"invalid content {c.content!r}")
coord = Coordinate(c.left, c.right, c.dark, c.light)
cal = (
Calibration(
mood_gain=req.calibration.mood_gain,
overlay_gain=req.calibration.overlay_gain,
right_variant_map=tuple(req.calibration.right_variant_map),
)
if req.calibration
else DEFAULT_CALIBRATION
)
plan = plan_alteration(coord, cal)
content = resolve_content(c.content)
return {
"plan": render_plan_to_dict(plan),
"content": {"audio_source": content.audio_source, "video": content.video},
}
@app.get("/api/clips")
def api_clips():
return {"clips": [c.to_dict() for c in app.state.clips]}
if MEDIA_DIR.exists():
app.mount("/media", StaticFiles(directory=MEDIA_DIR), name="media")
if STATIC_DIR.exists():
app.mount("/", StaticFiles(directory=STATIC_DIR, html=True), name="static")
return app
app = create_app()
+68
View File
@@ -0,0 +1,68 @@
"""The base-clip + variant + annotation manifest the simulator renders.
Replaces simulator/fixtures.py (the selection-era synthetic catalog). Each base
clip carries: the raw base file, a map of pre-baked Right-strength variant files
(strength 0 is always the raw base), an authored Left annotation track (box +
label key + the minimum Left level at which it appears), and per-language string
tables. See the reconciled-simulator-alteration-slice design §3.2.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Any
@dataclass(frozen=True)
class Clip:
id: str
title: str
base_file: str
license: str
source: str
right_variants: dict # {"1": {"file": ...}, "4": {...}} (no "0")
annotations: list # [{"key", "box":[x,y,w,h], "min_level"}, ...]
strings: dict # {"en": {key: text}}
def variant_file(self, strength: int) -> str:
"""The video file for a Right strength; 0 and any unauthored strength
fall back to the raw base file."""
entry = self.right_variants.get(str(strength))
return entry["file"] if entry else self.base_file
def to_dict(self) -> dict:
variants = {"0": {"file": self.base_file, "raw": True}}
for k, v in self.right_variants.items():
variants[k] = v
return {
"id": self.id,
"title": self.title,
"base_file": self.base_file,
"license": self.license,
"source": self.source,
"right_variants": variants,
"annotations": self.annotations,
"strings": self.strings,
}
def _clip_from_dict(d: dict[str, Any]) -> Clip:
return Clip(
id=d["id"],
title=d["title"],
base_file=d["base_file"],
license=d.get("license", ""),
source=d.get("source", ""),
right_variants=d.get("right_variants", {}),
annotations=d.get("annotations", []),
strings=d.get("strings", {}),
)
def load_manifest(path: str | Path) -> list[Clip]:
"""Load the base-clip manifest. Raises FileNotFoundError if missing."""
path = Path(path)
data = json.loads(path.read_text())
return [_clip_from_dict(c) for c in data["clips"]]
+7
View File
@@ -0,0 +1,7 @@
services:
simulator:
build:
context: ..
dockerfile: simulator/Dockerfile
ports:
- "8000:8000"
+12
View File
@@ -0,0 +1,12 @@
# Simulator sample media
`manifest.json` is committed; the `.mp4` binaries are **not** (gitignored). They
are look-tuning samples, not shipped installation content.
Populate them from the session-0008 POC artifacts:
python simulator/setup_sample_media.py
This copies `~/hef-poc/out/neutral.mp4``forest/base.mp4` and
`~/hef-poc/out/right_flow.mp4``forest/right4.mp4` (the real flow-stabilized
restyle), and generates placeholder strengths `forest/right1..3.mp4`.
+31
View File
@@ -0,0 +1,31 @@
{
"clips": [
{
"id": "forest",
"title": "Yosemite Falls (neutral base, POC sample)",
"base_file": "forest/base.mp4",
"license": "poc-sample (look-tuning only; not shipped content)",
"source": "hef-poc/out/neutral.mp4",
"right_variants": {
"1": {"file": "forest/right1.mp4", "model": "placeholder"},
"2": {"file": "forest/right2.mp4", "model": "placeholder"},
"3": {"file": "forest/right3.mp4", "model": "placeholder"},
"4": {"file": "forest/right4.mp4", "model": "sd-turbo+farneback-flow"}
},
"annotations": [
{"key": "detected.water", "box": [0.30, 0.10, 0.18, 0.70], "min_level": 1},
{"key": "detected.rock_face", "box": [0.05, 0.30, 0.20, 0.55], "min_level": 2},
{"key": "detected.conifer", "box": [0.70, 0.20, 0.22, 0.45], "min_level": 3},
{"key": "measure.flow_rate", "box": [0.34, 0.55, 0.14, 0.08], "min_level": 4}
],
"strings": {
"en": {
"detected.water": "flowing water",
"detected.rock_face": "granite face",
"detected.conifer": "conifer stand",
"measure.flow_rate": "~2.1 m³/s"
}
}
}
]
}
+53
View File
@@ -0,0 +1,53 @@
"""Populate simulator/sample_media/forest/ from the session-0008 POC artifacts.
Copies the real neutral base + the real flow-stabilized Right restyle out of
~/hef-poc/out/, and generates placeholder intermediate Right strengths (1..3) by
blending the base toward the real restyle with ffmpeg. The media binaries are
gitignored; only the manifest is committed. Sample footage is for look-tuning
only, not shipped content.
Usage: python simulator/setup_sample_media.py
Requires: ffmpeg on PATH (or `pip install imageio-ffmpeg`), and ~/hef-poc/out/.
"""
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
POC = Path.home() / "hef-poc" / "out"
DEST = Path(__file__).parent / "sample_media" / "forest"
def _ffmpeg() -> str:
if shutil.which("ffmpeg"):
return "ffmpeg"
import imageio_ffmpeg
return imageio_ffmpeg.get_ffmpeg_exe()
def main() -> None:
DEST.mkdir(parents=True, exist_ok=True)
base = DEST / "base.mp4"
right4 = DEST / "right4.mp4"
shutil.copyfile(POC / "neutral.mp4", base)
shutil.copyfile(POC / "right_flow.mp4", right4)
ff = _ffmpeg()
# Placeholder strengths 1..3: opacity-blend base toward the real restyle.
for strength, alpha in ((1, 0.25), (2, 0.5), (3, 0.75)):
out = DEST / f"right{strength}.mp4"
subprocess.run(
[ff, "-y", "-i", str(base), "-i", str(right4),
"-filter_complex",
f"[1:v]format=yuva444p,colorchannelmixer=aa={alpha}[top];"
f"[0:v][top]overlay=shortest=1[v]",
"-map", "[v]", "-an", str(out)],
check=True,
)
print(f"generated {out.name} (alpha {alpha})")
print(f"sample media ready in {DEST}")
if __name__ == "__main__":
main()
+105
View File
@@ -0,0 +1,105 @@
// Thin renderer: post controls+calibration -> RenderPlan; render grade, Right
// variant crossfade, and the live Left overlay. All math stays in Python.
const $ = (id) => document.getElementById(id);
const vid = $("vid"), tint = $("tint"), overlay = $("overlay"), black = $("black"), readout = $("readout");
let clip = null; // active clip manifest entry
let currentVariant = -1; // last loaded Right strength
async function loadClips() {
const data = await (await fetch("/api/clips")).json();
clip = data.clips[0] || null;
}
function mediaUrl(file) { return "/media/" + file; }
function variantFile(strength) {
const v = clip.right_variants[String(strength)];
return v ? v.file : clip.base_file;
}
function applyGrade(tone) {
// Light: warm + brighten (sepia). Dark: cool + darken via a multiply-blended
// blue wash (#tint) that lifts shadows toward blue while keeping natural
// greens — the peaceful POC dark look, NOT a full-frame hue spin.
const warm = tone > 0 ? tone : 0, cool = tone < 0 ? -tone : 0;
const bright = 1 + 0.25 * warm - 0.35 * cool;
const sat = 1 + 0.15 * warm - 0.30 * cool;
vid.style.filter =
`brightness(${bright.toFixed(3)}) saturate(${sat.toFixed(3)}) ` +
`sepia(${(warm * 0.5).toFixed(3)})`;
tint.style.opacity = (cool * 0.6).toFixed(3);
}
function loadVariant(strength) {
if (strength === currentVariant) return;
currentVariant = strength;
vid.style.opacity = "0";
setTimeout(() => {
vid.src = mediaUrl(variantFile(strength));
vid.play().catch(() => {});
vid.style.opacity = "1";
}, 150);
}
function renderOverlay(level, intensity) {
overlay.innerHTML = "";
if (!clip || level <= 0) { overlay.style.opacity = "0"; return; }
overlay.style.opacity = String(intensity);
const strings = (clip.strings && clip.strings.en) || {};
for (const a of clip.annotations) {
if (a.min_level > level) continue;
const [x, y, w, h] = a.box.map((n) => n * 100);
const rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
rect.setAttribute("x", x); rect.setAttribute("y", y);
rect.setAttribute("width", w); rect.setAttribute("height", h);
rect.setAttribute("class", "anno-box");
overlay.appendChild(rect);
const text = document.createElementNS("http://www.w3.org/2000/svg", "text");
text.setAttribute("x", x + 0.5); text.setAttribute("y", Math.max(y - 0.5, 2));
text.setAttribute("class", "anno-label");
text.textContent = strings[a.key] || a.key;
overlay.appendChild(text);
}
}
function controls() {
return {
content: $("content").value,
left: +$("left").value, right: +$("right").value,
dark: +$("dark").value, light: +$("light").value,
volume: 2, brightness: 2,
};
}
function calibration() {
return { mood_gain: +$("mood_gain").value, overlay_gain: +$("overlay_gain").value,
right_variant_map: [0, 1, 2, 3, 4] };
}
let timer = null;
async function update() {
const resp = await fetch("/api/alteration", {
method: "POST", headers: { "content-type": "application/json" },
body: JSON.stringify({ controls: controls(), calibration: calibration() }),
});
if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; }
const data = await resp.json();
readout.textContent = JSON.stringify(data, null, 2);
if (!data.content.video) { black.classList.remove("hidden"); return; }
black.classList.add("hidden");
applyGrade(data.plan.grade.tone);
loadVariant(data.plan.restyle.variant);
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
}
function debounced() { clearTimeout(timer); timer = setTimeout(update, 80); }
async function main() {
await loadClips();
for (const id of ["content", "left", "right", "dark", "light", "mood_gain", "overlay_gain"]) {
$(id).addEventListener("input", debounced);
}
update();
}
main();
+57
View File
@@ -0,0 +1,57 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>HEF — Alteration Simulator</title>
<link rel="stylesheet" href="/style.css" />
</head>
<body>
<header><h1>Human Experience Filter — Alteration Preview</h1></header>
<main>
<section class="stage">
<div class="screen">
<video id="vid" loop muted playsinline></video>
<div id="tint"></div>
<svg id="overlay" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
<div id="black" class="black hidden"></div>
</div>
</section>
<section class="panel">
<fieldset>
<legend>Content dial</legend>
<select id="content">
<option value="video">video</option>
<option value="audio_video">audio + video</option>
<option value="music_video">music + video</option>
<option value="off">off (black)</option>
<option value="white_noise">white noise (no video)</option>
<option value="music">music (no video)</option>
<option value="audio_track">audio track (no video)</option>
</select>
</fieldset>
<fieldset>
<legend>Experience knobs (04)</legend>
<label>Left (analytical) <input type="range" id="left" min="0" max="4" value="0" /></label>
<label>Right (dreamlike) <input type="range" id="right" min="0" max="4" value="0" /></label>
<label>Dark <input type="range" id="dark" min="0" max="4" value="0" /></label>
<label>Light <input type="range" id="light" min="0" max="4" value="0" /></label>
</fieldset>
<fieldset>
<legend>Calibration</legend>
<label>mood gain <input type="range" id="mood_gain" min="0" max="2" step="0.05" value="1" /></label>
<label>overlay gain <input type="range" id="overlay_gain" min="0" max="2" step="0.05" value="1" /></label>
</fieldset>
<fieldset>
<legend>RenderPlan readout</legend>
<pre id="readout"></pre>
</fieldset>
</section>
</main>
<script src="/app.js"></script>
</body>
</html>
+25
View File
@@ -0,0 +1,25 @@
* { box-sizing: border-box; }
body { margin: 0; font: 14px/1.4 system-ui, sans-serif; background: #111; color: #eee; }
header { padding: 0.6rem 1rem; background: #000; }
h1 { font-size: 1rem; margin: 0; font-weight: 600; }
main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; }
.stage { flex: 1 1 640px; }
.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000;
border-radius: 6px; overflow: hidden; }
#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; }
#tint { position: absolute; inset: 0; pointer-events: none; opacity: 0;
background: #28425f; mix-blend-mode: multiply;
transition: opacity 0.2s ease; }
#overlay { position: absolute; inset: 0; width: 100%; height: 100%;
pointer-events: none; transition: opacity 0.2s ease; }
.anno-box { fill: none; stroke: #6cf; stroke-width: 0.4; vector-effect: non-scaling-stroke; }
.anno-label { fill: #6cf; font-size: 3px; font-family: monospace; }
.black { position: absolute; inset: 0; background: #000; }
.hidden { display: none; }
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem; }
fieldset { border: 1px solid #333; border-radius: 6px; }
legend { color: #9af; padding: 0 0.4rem; }
label { display: block; margin: 0.4rem 0; }
input[type=range], select { width: 100%; }
#readout { background: #000; padding: 0.5rem; border-radius: 4px; font-size: 12px;
white-space: pre-wrap; max-height: 240px; overflow: auto; }
+153
View File
@@ -0,0 +1,153 @@
import pytest
from hef.catalog import Record, validate, CatalogError
def make_record(**overrides):
base = dict(
id="nasa-apollo-earthrise",
title="Earthrise",
source_url="https://archive.org/details/earthrise",
source_archive="internet_archive",
license="public_domain",
mode="video",
left=1,
right=4,
dark=1,
light=3,
duration_s=600,
file_path="/media/earthrise.mp4",
)
base.update(overrides)
return Record(**base)
def test_valid_record_passes():
validate(make_record()) # must not raise
def test_defaults_are_set():
record = make_record()
assert record.review_status == "proposed"
assert record.attribution == ""
assert record.reviewed_at is None
def test_invalid_mode_raises():
with pytest.raises(CatalogError):
validate(make_record(mode="hologram"))
def test_invalid_license_raises():
with pytest.raises(CatalogError):
validate(make_record(license="all_rights_reserved"))
def test_coordinate_out_of_range_raises():
with pytest.raises(CatalogError):
validate(make_record(left=5))
with pytest.raises(CatalogError):
validate(make_record(dark=-1))
def test_boolean_coordinate_rejected():
with pytest.raises(CatalogError):
validate(make_record(right=True))
def test_cc_by_requires_attribution():
with pytest.raises(CatalogError):
validate(make_record(license="cc_by", attribution=""))
validate(make_record(license="cc_by", attribution="Jane Doe, CC BY 4.0"))
def test_empty_id_raises():
with pytest.raises(CatalogError):
validate(make_record(id=""))
def test_negative_duration_raises():
with pytest.raises(CatalogError):
validate(make_record(duration_s=-1))
from hef.catalog import record_to_dict, record_from_dict
def test_dict_round_trip():
record = make_record()
restored = record_from_dict(record_to_dict(record))
assert restored == record
def test_record_from_dict_rejects_unknown_field():
data = record_to_dict(make_record())
data["bogus"] = 1
with pytest.raises(CatalogError):
record_from_dict(data)
def test_record_from_dict_rejects_missing_required_field():
data = record_to_dict(make_record())
del data["title"]
with pytest.raises(CatalogError):
record_from_dict(data)
from hef.catalog import load_catalog, save_catalog, append_record
def test_save_then_load_round_trip(tmp_path):
path = tmp_path / "library.jsonl"
records = [make_record(id="a"), make_record(id="b", mode="audio")]
save_catalog(records, path)
loaded = load_catalog(path)
assert loaded == records
def test_load_skips_blank_lines(tmp_path):
path = tmp_path / "library.jsonl"
save_catalog([make_record(id="a")], path)
with path.open("a", encoding="utf-8") as fh:
fh.write("\n \n")
loaded = load_catalog(path)
assert len(loaded) == 1
def test_load_empty_file_returns_empty_list(tmp_path):
path = tmp_path / "library.jsonl"
path.write_text("", encoding="utf-8")
assert load_catalog(path) == []
def test_load_invalid_json_raises(tmp_path):
path = tmp_path / "library.jsonl"
path.write_text("{not json}\n", encoding="utf-8")
with pytest.raises(CatalogError):
load_catalog(path)
def test_load_validates_records(tmp_path):
path = tmp_path / "library.jsonl"
save_catalog([make_record(id="a")], path)
import json
bad = record_to_dict(make_record(id="bad"))
bad["left"] = 9
with path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(bad) + "\n")
with pytest.raises(CatalogError):
load_catalog(path)
def test_append_record(tmp_path):
path = tmp_path / "library.jsonl"
save_catalog([make_record(id="a")], path)
append_record(make_record(id="b"), path)
loaded = load_catalog(path)
assert [r.id for r in loaded] == ["a", "b"]
def test_save_rejects_invalid_record(tmp_path):
path = tmp_path / "library.jsonl"
with pytest.raises(CatalogError):
save_catalog([make_record(mode="hologram")], path)
+48
View File
@@ -0,0 +1,48 @@
import pytest
from hef.catalog import Record, validate_catalog, index_by_id, CatalogError
def make_record(**o):
base = dict(
id="a",
title="t",
source_url="u",
source_archive="nasa",
license="public_domain",
mode="video",
left=0,
right=0,
dark=0,
light=0,
duration_s=1,
file_path="p",
)
base.update(o)
return Record(**base)
def test_validate_catalog_accepts_unique_ids():
validate_catalog([make_record(id="a"), make_record(id="b")]) # no raise
def test_validate_catalog_rejects_duplicate_id():
with pytest.raises(CatalogError) as e:
validate_catalog([make_record(id="dup"), make_record(id="dup")])
assert "dup" in str(e.value)
def test_validate_catalog_validates_each_record():
with pytest.raises(CatalogError):
validate_catalog([make_record(left=9)])
def test_index_by_id_round_trips():
recs = [make_record(id="a"), make_record(id="b")]
idx = index_by_id(recs)
assert idx["a"].id == "a" and idx["b"].id == "b"
def test_index_by_id_rejects_duplicates():
with pytest.raises(CatalogError):
index_by_id([make_record(id="x"), make_record(id="x")])
+69
View File
@@ -0,0 +1,69 @@
import json
import pytest
from simulator.clips import Clip, load_manifest
def _manifest_dict():
return {
"clips": [
{
"id": "forest",
"title": "Yosemite Falls (neutral)",
"base_file": "forest/base.mp4",
"license": "poc-sample",
"source": "hef-poc",
"right_variants": {
"4": {"file": "forest/right4.mp4", "model": "sd-turbo+flow"},
"1": {"file": "forest/right1.mp4"},
},
"annotations": [
{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1},
{"key": "detected.conifer", "box": [0.6, 0.1, 0.2, 0.2], "min_level": 3},
],
"strings": {"en": {"detected.water": "flowing water", "detected.conifer": "conifer"}},
}
]
}
def test_load_manifest_parses_clips(tmp_path):
p = tmp_path / "manifest.json"
p.write_text(json.dumps(_manifest_dict()))
clips = load_manifest(p)
assert len(clips) == 1
c = clips[0]
assert isinstance(c, Clip)
assert c.id == "forest"
assert c.base_file == "forest/base.mp4"
def test_clip_lists_variant_files_by_strength(tmp_path):
p = tmp_path / "manifest.json"
p.write_text(json.dumps(_manifest_dict()))
c = load_manifest(p)[0]
# variant 0 is always the raw base; authored strengths come from the manifest
assert c.variant_file(0) == "forest/base.mp4"
assert c.variant_file(4) == "forest/right4.mp4"
assert c.variant_file(1) == "forest/right1.mp4"
# an unauthored strength falls back to the raw base
assert c.variant_file(2) == "forest/base.mp4"
def test_clip_serializes_to_dict_for_the_api(tmp_path):
p = tmp_path / "manifest.json"
p.write_text(json.dumps(_manifest_dict()))
d = load_manifest(p)[0].to_dict()
assert d["id"] == "forest"
assert d["base_file"] == "forest/base.mp4"
assert d["annotations"][0]["key"] == "detected.water"
assert d["strings"]["en"]["detected.water"] == "flowing water"
# variant map is exposed keyed by strength string, including 0 -> base
assert d["right_variants"]["0"]["file"] == "forest/base.mp4"
assert d["right_variants"]["4"]["file"] == "forest/right4.mp4"
def test_missing_manifest_raises(tmp_path):
with pytest.raises(FileNotFoundError):
load_manifest(tmp_path / "nope.json")
+79
View File
@@ -0,0 +1,79 @@
from hef.selection import Coordinate
from tools.drafting import Draft, HeuristicProposer, Signals
def _sig(**o):
base = dict(
title="",
description="",
source_archive="nasa",
mode="video",
duration_s=600,
)
base.update(o)
return Signals(**base)
def test_returns_draft_with_coordinate_and_rationale():
d = HeuristicProposer().propose(_sig())
assert isinstance(d, Draft)
assert isinstance(d.coordinate, Coordinate)
assert d.rationale and "\n" not in d.rationale
def test_librivox_seeds_left():
d = HeuristicProposer().propose(
_sig(title="Meditations", source_archive="librivox", mode="audio")
)
assert d.coordinate.left >= 3 and d.rationale
def test_music_archives_seed_right():
for arch in ("musopen", "fma", "nasa", "freesound"):
d = HeuristicProposer().propose(_sig(source_archive=arch, mode="audio"))
assert d.coordinate.right >= 3, arch
def test_internet_archive_seeds_left():
d = HeuristicProposer().propose(_sig(source_archive="internet_archive"))
assert d.coordinate.left >= 3
def test_storm_seeds_dark():
d = HeuristicProposer().propose(
_sig(title="Thunderstorm at Night", source_archive="nasa")
)
assert d.coordinate.dark >= 2
def test_sunrise_seeds_light():
d = HeuristicProposer().propose(
_sig(title="Sunrise over the Garden", source_archive="nasa")
)
assert d.coordinate.light >= 2
def test_coordinates_clamped_to_0_4():
d = HeuristicProposer().propose(
_sig(
title="storm night war funeral decay requiem minor",
description="noir death grief",
source_archive="librivox",
)
)
for v in (d.coordinate.left, d.coordinate.right, d.coordinate.dark, d.coordinate.light):
assert 0 <= v <= 4
def test_rationale_cites_a_signal():
d = HeuristicProposer().propose(
_sig(title="Storm", source_archive="librivox", mode="audio")
)
assert "librivox" in d.rationale.lower()
def test_deterministic():
s = _sig(title="Storm at Dawn", source_archive="nasa")
a = HeuristicProposer().propose(s)
b = HeuristicProposer().propose(s)
assert a == b
+181
View File
@@ -0,0 +1,181 @@
import json
import pytest
from tools.http import HttpClient
from tools.ingest.internet_archive import InternetArchiveFetcher
from tools.ingest.librivox import LibriVoxFetcher
from tools.ingest.nasa import NasaFetcher
class _Resp:
def __init__(self, data: bytes):
self._data = data
def read(self):
return self._data
def __enter__(self):
return self
def __exit__(self, *a):
return False
def fake_opener(mapping):
"""Dispatch a urllib Request to a canned payload by URL substring."""
def opener(req, timeout=None):
url = req.full_url
for key, payload in mapping.items():
if key in url:
if isinstance(payload, (bytes, bytearray)):
return _Resp(bytes(payload))
return _Resp(json.dumps(payload).encode("utf-8"))
raise AssertionError(f"unexpected url: {url}")
return opener
def test_librivox_fetcher():
payload = {
"books": [
{
"id": "123",
"title": "Meditations",
"url_librivox": "https://librivox.org/meditations/",
"url_zip_file": "https://archive.org/download/meditations/meditations_mp3.zip",
"authors": [{"first_name": "Marcus", "last_name": "Aurelius"}],
"description": "Stoic philosophy.",
}
]
}
client = HttpClient(opener=fake_opener({"librivox.org/api": payload}))
cands = LibriVoxFetcher(client).search("medit", limit=5)
c = cands[0]
assert c.source_archive == "librivox"
assert c.license == "public_domain" and c.attribution == ""
assert c.suggested_id == "librivox-meditations"
assert c.media_url.endswith(".zip")
assert c.media_ext == "zip"
assert "Aurelius" in c.description
def test_nasa_fetcher():
search_payload = {
"collection": {
"items": [
{
"data": [
{
"nasa_id": "as08-14-2383",
"title": "Earthrise",
"description": "View of Earth from the Moon",
"media_type": "video",
}
],
"href": "https://images-assets.nasa.gov/video/as08-14-2383/collection.json",
}
]
}
}
asset_payload = [
"https://images-assets.nasa.gov/video/as08-14-2383/as08-14-2383~orig.mp4",
"https://images-assets.nasa.gov/video/as08-14-2383/as08-14-2383~thumb.jpg",
]
client = HttpClient(
opener=fake_opener(
{
"images-api.nasa.gov/search": search_payload,
"collection.json": asset_payload,
}
)
)
c = NasaFetcher(client).search("earth", limit=3)[0]
assert c.suggested_id == "nasa-as08-14-2383"
assert c.license == "public_domain" and c.attribution == ""
assert c.media_url.endswith("orig.mp4")
assert c.media_ext == "mp4"
assert "Earth" in c.description
def test_internet_archive_cc_by():
meta_payload = {
"metadata": {
"identifier": "earthrise",
"title": "Earthrise",
"licenseurl": "http://creativecommons.org/licenses/by/4.0/",
"creator": "NASA",
"description": "Apollo 8 footage.",
},
"files": [
{"name": "earthrise.mp4", "format": "h.264", "source": "original"},
{"name": "earthrise.png", "format": "PNG", "source": "derivative"},
],
"server": "ia800100.us.archive.org",
"dir": "/12/items/earthrise",
}
client = HttpClient(opener=fake_opener({"archive.org/metadata/earthrise": meta_payload}))
c = InternetArchiveFetcher(client).resolve("earthrise")
assert c.suggested_id == "ia-earthrise"
assert c.license == "cc_by" and "NASA" in c.attribution
assert c.media_url == "https://ia800100.us.archive.org/12/items/earthrise/earthrise.mp4"
assert c.media_ext == "mp4"
assert c.source_url == "https://archive.org/details/earthrise"
def test_internet_archive_no_license_assumed_public_domain_and_flagged():
meta_payload = {
"metadata": {"identifier": "oldfilm", "title": "Old Film"},
"files": [{"name": "oldfilm.mp4", "source": "original"}],
"server": "ia.example.org",
"dir": "/x/items/oldfilm",
}
client = HttpClient(opener=fake_opener({"archive.org/metadata/oldfilm": meta_payload}))
c = InternetArchiveFetcher(client).resolve("oldfilm")
assert c.license == "public_domain"
assert "verify" in c.description.lower()
def test_internet_archive_search_resolves_each_hit():
search_payload = {"response": {"docs": [{"identifier": "earthrise"}]}}
meta_payload = {
"metadata": {
"identifier": "earthrise",
"title": "Earthrise",
"licenseurl": "https://creativecommons.org/publicdomain/mark/1.0/",
},
"files": [{"name": "earthrise.mp4", "source": "original"}],
"server": "ia.example.org",
"dir": "/x",
}
client = HttpClient(
opener=fake_opener(
{
"advancedsearch.php": search_payload,
"archive.org/metadata/earthrise": meta_payload,
}
)
)
cands = InternetArchiveFetcher(client).search("earthrise", limit=5)
assert [c.suggested_id for c in cands] == ["ia-earthrise"]
assert cands[0].license == "public_domain"
def test_deferred_stubs_raise_not_implemented():
from tools.ingest.fma import FmaFetcher
from tools.ingest.freesound import FreesoundFetcher
from tools.ingest.musopen import MusopenFetcher
with pytest.raises(NotImplementedError):
MusopenFetcher(None).search("x", limit=1)
with pytest.raises(NotImplementedError):
FmaFetcher(None).search("x", limit=1)
with pytest.raises(NotImplementedError):
FreesoundFetcher(None).search("x", limit=1)
def test_fetchers_expose_archive_label():
assert LibriVoxFetcher(None).archive == "librivox"
assert NasaFetcher(None).archive == "nasa"
assert InternetArchiveFetcher(None).archive == "internet_archive"
+25
View File
@@ -0,0 +1,25 @@
import pytest
from tools.ingest_cli import main
def test_help_parses():
with pytest.raises(SystemExit) as e:
main(["--help"])
assert e.value.code == 0
def test_unknown_archive_errors_cleanly(capsys):
rc = main(["bogus", "--query", "x"])
assert rc == 2
assert "unknown archive" in capsys.readouterr().err
def test_missing_query_and_resolve_errors():
assert main(["nasa"]) == 2
def test_deferred_archive_reports_not_implemented(capsys):
rc = main(["musopen", "--query", "bach"])
assert rc == 3
assert "deferred" in capsys.readouterr().err
+193
View File
@@ -0,0 +1,193 @@
import pytest
from hef.catalog import CatalogError, load_catalog, validate_catalog
from tools.drafting import HeuristicProposer
from tools.ingest.base import Candidate, ingest_candidate, ingest_search
from tools.probe import Probe
def make_candidate(**o):
base = dict(
source_archive="nasa",
source_url="https://example.org/landing/x",
media_url="https://example.org/media/x.mp4",
title="Earthrise",
license="public_domain",
attribution="",
suggested_id="nasa-earthrise",
media_ext="mp4",
description="a view of earth",
)
base.update(o)
return Candidate(**base)
def video_probe():
return Probe(
streams=[
{
"codec_type": "video",
"width": 1920,
"height": 1080,
"disposition": {"attached_pic": 0},
},
{"codec_type": "audio"},
],
format={"duration": "10.0"},
)
def make_prober(probe):
def prober(path):
return probe
return prober
def make_downloader(content=b"FAKEMEDIA"):
def downloader(url, dest):
dest.write_bytes(content)
return downloader
def test_ingest_appends_one_proposed_record(tmp_path):
catalog = tmp_path / "library.jsonl"
media_root = tmp_path / "media"
rec = ingest_candidate(
make_candidate(),
catalog_path=catalog,
media_root=media_root,
proposer=HeuristicProposer(),
prober=make_prober(video_probe()),
downloader=make_downloader(),
)
assert rec is not None
records = load_catalog(catalog)
assert len(records) == 1
r = records[0]
assert r.id == "nasa-earthrise"
assert r.mode == "av"
assert r.resolution == "1920x1080"
assert r.duration_s == 10
assert r.review_status == "proposed"
assert r.reviewed_at is None
assert r.rationale != ""
assert r.file_path == "nasa/nasa-earthrise.mp4"
assert r.notes == "a view of earth"
# media written under media_root/<archive>/<id>.<ext>
assert (media_root / "nasa" / "nasa-earthrise.mp4").exists()
def test_ingest_is_idempotent(tmp_path):
catalog = tmp_path / "library.jsonl"
media_root = tmp_path / "media"
kw = dict(
catalog_path=catalog,
media_root=media_root,
proposer=HeuristicProposer(),
prober=make_prober(video_probe()),
downloader=make_downloader(),
)
first = ingest_candidate(make_candidate(), **kw)
second = ingest_candidate(make_candidate(), **kw)
assert first is not None
assert second is None # skipped as duplicate
assert len(load_catalog(catalog)) == 1
def test_dominant_color_off_by_default(tmp_path):
catalog = tmp_path / "library.jsonl"
rec = ingest_candidate(
make_candidate(),
catalog_path=catalog,
media_root=tmp_path / "media",
proposer=HeuristicProposer(),
prober=make_prober(video_probe()),
downloader=make_downloader(),
)
assert rec.dominant_color == ""
def test_dominant_color_computed_when_enabled(tmp_path):
catalog = tmp_path / "library.jsonl"
color_calls = []
def fake_color_fn(path, *, midpoint_s=0.0):
color_calls.append(midpoint_s)
return "#abcdef"
rec = ingest_candidate(
make_candidate(),
catalog_path=catalog,
media_root=tmp_path / "media",
proposer=HeuristicProposer(),
prober=make_prober(video_probe()),
downloader=make_downloader(),
compute_color=True,
color_fn=fake_color_fn,
)
assert rec.dominant_color == "#abcdef"
assert color_calls # was invoked
def test_audio_keeps_empty_color_even_when_enabled(tmp_path):
audio = Probe(streams=[{"codec_type": "audio"}], format={"duration": "30.0"})
def boom(path, *, midpoint_s=0.0):
raise AssertionError("must not compute color for audio")
rec = ingest_candidate(
make_candidate(media_ext="mp3", source_archive="librivox", license="public_domain"),
catalog_path=tmp_path / "library.jsonl",
media_root=tmp_path / "media",
proposer=HeuristicProposer(),
prober=make_prober(audio),
downloader=make_downloader(),
compute_color=True,
color_fn=boom,
)
assert rec.mode == "audio" and rec.dominant_color == ""
def test_unmappable_license_raises_and_writes_nothing(tmp_path):
catalog = tmp_path / "library.jsonl"
with pytest.raises(CatalogError):
ingest_candidate(
make_candidate(license="all_rights_reserved"),
catalog_path=catalog,
media_root=tmp_path / "media",
proposer=HeuristicProposer(),
prober=make_prober(video_probe()),
downloader=make_downloader(),
)
assert not catalog.exists() or load_catalog(catalog) == []
def test_ingest_search_runs_over_hits(tmp_path):
catalog = tmp_path / "library.jsonl"
class FakeFetcher:
archive = "nasa"
def search(self, query, *, limit):
return [
make_candidate(suggested_id="nasa-a", media_url="u/a.mp4"),
make_candidate(suggested_id="nasa-b", media_url="u/b.mp4"),
][:limit]
def resolve(self, identifier): # pragma: no cover - not used here
raise NotImplementedError
recs = ingest_search(
FakeFetcher(),
"earth",
limit=2,
catalog_path=catalog,
media_root=tmp_path / "media",
proposer=HeuristicProposer(),
prober=make_prober(video_probe()),
downloader=make_downloader(),
)
assert [r.id for r in recs] == ["nasa-a", "nasa-b"]
validate_catalog(load_catalog(catalog))
+47
View File
@@ -0,0 +1,47 @@
from pathlib import Path
from hef.catalog import Record, save_catalog, load_catalog
from hef.selection import Coordinate, select
REPO_ROOT = Path(__file__).resolve().parent.parent
def _rec(id, mode, left, right, dark, light):
return Record(
id=id,
title=id,
source_url=f"https://example.org/{id}",
source_archive="internet_archive",
license="public_domain",
mode=mode,
left=left,
right=right,
dark=dark,
light=light,
duration_s=600,
file_path=f"/media/{id}.mp4",
)
def test_catalog_round_trip_then_select(tmp_path):
library = [
_rec("lecture", "av", left=4, right=0, dark=1, light=2),
_rec("nebula", "video", left=0, right=4, dark=2, light=3),
_rec("storm", "av", left=0, right=3, dark=4, light=0),
_rec("sunrise", "av", left=1, right=3, dark=0, light=4),
]
path = tmp_path / "library.jsonl"
save_catalog(library, path)
loaded = load_catalog(path)
# A right-brain, very-light tuning should land on "sunrise".
chosen = select(loaded, Coordinate(left=1, right=3, dark=0, light=4), "av")
assert chosen.id == "sunrise"
# The void state returns nothing regardless of library.
assert select(loaded, Coordinate(0, 0, 0, 0), "none") is None
def test_committed_empty_catalog_loads():
path = REPO_ROOT / "catalog" / "library.jsonl"
assert load_catalog(path) == []
+84
View File
@@ -0,0 +1,84 @@
import pytest
from hef.catalog import LICENSES
from tools.licensing import librivox_license, normalize_license
def test_cc_by_url_maps_and_requires_attribution():
lic, attr = normalize_license(
"https://creativecommons.org/licenses/by/4.0/", creator="Jane Doe"
)
assert lic == "cc_by" and "Jane Doe" in attr
def test_cc_by_identifier_maps():
lic, attr = normalize_license("CC BY 4.0", creator="Sam")
assert lic == "cc_by" and attr
def test_cc_by_nc_maps():
lic, attr = normalize_license(
"https://creativecommons.org/licenses/by-nc/4.0/", creator="Artist"
)
assert lic == "cc_by_nc" and "Artist" in attr
def test_cc_by_without_creator_still_has_nonempty_attribution():
# validate() rejects an attribution license with empty attribution, so the
# normalizer must always produce a non-empty string for cc_by/cc_by_nc.
lic, attr = normalize_license("https://creativecommons.org/licenses/by/4.0/")
assert lic == "cc_by" and attr != ""
def test_cc0_maps_with_empty_attribution():
lic, attr = normalize_license("https://creativecommons.org/publicdomain/zero/1.0/")
assert lic == "cc0" and attr == ""
def test_cc0_identifier():
lic, attr = normalize_license("CC0")
assert lic == "cc0" and attr == ""
def test_public_domain_mark():
lic, attr = normalize_license(
"https://creativecommons.org/publicdomain/mark/1.0/"
)
assert lic == "public_domain" and attr == ""
def test_no_known_copyright_is_public_domain():
lic, attr = normalize_license("No known copyright")
assert lic == "public_domain" and attr == ""
def test_public_domain_literal():
lic, attr = normalize_license("public_domain")
assert lic == "public_domain" and attr == ""
def test_unmappable_rejected():
with pytest.raises(ValueError):
normalize_license("All Rights Reserved")
def test_empty_rejected():
with pytest.raises(ValueError):
normalize_license("")
def test_librivox_license_helper():
lic, attr = librivox_license()
assert lic == "public_domain" and attr == ""
def test_all_outputs_in_vocab():
samples = [
"https://creativecommons.org/licenses/by/4.0/",
"https://creativecommons.org/licenses/by-nc/4.0/",
"CC0",
"public domain",
]
for raw in samples:
lic, _ = normalize_license(raw, creator="x")
assert lic in LICENSES
+38
View File
@@ -0,0 +1,38 @@
from tools.mediatools import (
compute_dominant_color,
dominant_color_from_rgb,
extract_frame,
)
def test_dominant_color_from_rgb_pure_red():
assert dominant_color_from_rgb(b"\xff\x00\x00") == "#ff0000"
def test_dominant_color_from_rgb_arbitrary():
assert dominant_color_from_rgb(b"\x12\xab\x0f") == "#12ab0f"
def test_compute_dominant_color_uses_runner():
captured = {}
def fake_runner(args):
captured["args"] = args
return b"\x00\x80\xff"
color = compute_dominant_color("clip.mp4", midpoint_s=5.0, runner=fake_runner)
assert color == "#0080ff"
assert "ffmpeg" in captured["args"]
assert "5.0" in captured["args"]
def test_extract_frame_invokes_runner_and_returns_dest(tmp_path):
dest = tmp_path / "frame.png"
calls = []
def fake_runner(args):
calls.append(args)
out = extract_frame("clip.mp4", dest, midpoint_s=2.0, runner=fake_runner)
assert out == dest
assert calls and "ffmpeg" in calls[0] and str(dest) in calls[0]
+128
View File
@@ -0,0 +1,128 @@
import pytest
from hef.selection import Coordinate
from player.alteration import (
DEFAULT_CALIBRATION,
AnalyticalOverlay,
Calibration,
ColorGrade,
RenderPlan,
Restyle,
plan_alteration,
render_plan_to_dict,
)
def _coord(left=0, right=0, dark=0, light=0):
return Coordinate(left=left, right=right, dark=dark, light=light)
def test_all_zero_knobs_is_the_unaltered_base():
plan = plan_alteration(_coord())
assert plan.is_identity
assert plan.overlay.level == 0
assert plan.overlay.intensity == 0.0
assert plan.restyle.variant == 0
assert plan.grade.tone == 0.0
assert plan.grade.is_identity
def test_left_drives_the_analytical_overlay_only():
plan = plan_alteration(_coord(left=4))
assert plan.overlay.level == 4
assert plan.overlay.intensity == 1.0
assert plan.restyle.variant == 0 # Left does not touch the substrate
assert plan.grade.tone == 0.0
def test_right_selects_a_discrete_restyle_variant_only():
plan = plan_alteration(_coord(right=2))
assert plan.restyle.variant == 2
assert plan.overlay.level == 0 # Right does not add overlay
def test_left_and_right_stack_not_cancel():
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
plan = plan_alteration(_coord(left=4, right=4))
assert plan.overlay.level == 4
assert plan.restyle.variant == 4
def test_light_pole_grades_warm_positive_tone():
plan = plan_alteration(_coord(light=4))
assert plan.grade.tone == 1.0
assert not plan.grade.is_identity
def test_dark_pole_grades_cool_negative_tone():
assert plan_alteration(_coord(dark=4)).grade.tone == -1.0
def test_equal_dark_and_light_is_identity_grade():
# design §5: the mood center is the raw, ungraded footage
assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity
assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity
def test_dark_minus_light_sets_intermediate_tone():
assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5)
assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5)
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
assert plan.overlay.level == 4
assert plan.restyle.variant == 2
assert plan.grade.tone == -1.0
assert not plan.is_identity
def test_default_calibration_is_locked():
# Session 0010: the knob->strength calibration is LOCKED to these values,
# settled by eye in the simulator (closes the open session-0006 decision).
# This pins the literal constants so they can't drift silently; changing the
# locked feel is a deliberate edit here + in alteration.py.
assert DEFAULT_CALIBRATION.mood_gain == 1.0
assert DEFAULT_CALIBRATION.overlay_gain == 1.0
assert DEFAULT_CALIBRATION.right_variant_map == (0, 1, 2, 3, 4)
def test_default_calibration_is_behavior_preserving():
# DEFAULT_CALIBRATION must reproduce the original three helpers exactly.
for left in range(5):
assert plan_alteration(_coord(left=left)).overlay.intensity == pytest.approx(left / 4)
for right in range(5):
assert plan_alteration(_coord(right=right)).restyle.variant == right
for dark in range(5):
for light in range(5):
expected = (light - dark) / 4
assert plan_alteration(_coord(dark=dark, light=light)).grade.tone == pytest.approx(expected)
def test_custom_calibration_scales_mood_and_overlay():
cal = Calibration(mood_gain=0.5, overlay_gain=0.5, right_variant_map=(0, 0, 1, 1, 2))
assert plan_alteration(_coord(light=4), cal).grade.tone == pytest.approx(0.5)
assert plan_alteration(_coord(left=4), cal).overlay.intensity == pytest.approx(0.5)
assert plan_alteration(_coord(right=3), cal).restyle.variant == 1
def test_calibration_gain_is_clamped_to_unit_range():
cal = Calibration(mood_gain=10.0, overlay_gain=10.0)
assert plan_alteration(_coord(light=4), cal).grade.tone == 1.0 # clamped, not 10
assert plan_alteration(_coord(left=4), cal).overlay.intensity == 1.0
def test_render_plan_to_dict_round_trips_the_numbers():
d = render_plan_to_dict(plan_alteration(_coord(left=4, right=2, dark=4, light=0)))
assert d == {
"grade": {"tone": -1.0},
"overlay": {"level": 4, "intensity": 1.0},
"restyle": {"variant": 2},
"is_identity": False,
}
def test_render_plan_is_frozen():
plan = plan_alteration(_coord())
with pytest.raises(Exception):
plan.grade.tone = 0.5 # type: ignore[misc]
+34
View File
@@ -0,0 +1,34 @@
import pytest
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
def test_audio_sources_are_the_four_distinct_sources():
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
# The §6 table, row by row: position -> (audio_source, video).
@pytest.mark.parametrize(
"position,audio_source,video",
[
("off", "none", False),
("white_noise", "white_noise", False),
("music", "music", False),
("audio_track", "audio_track", False),
("video", "none", True),
("music_video", "music", True),
("audio_video", "audio_track", True),
],
)
def test_resolve_content_matches_spec_table(position, audio_source, video):
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
def test_off_is_void_state_black_and_silent():
r = resolve_content("off")
assert r.video is False and r.audio_source == "none"
def test_resolve_content_rejects_unknown_position():
with pytest.raises(ValueError):
resolve_content("bogus")
+53
View File
@@ -0,0 +1,53 @@
import pytest
from player.controls import (
Controls,
CONTENT_POSITIONS,
ControlsError,
validate_controls,
parse_controls,
)
def test_content_positions_are_the_seven_from_the_spec():
assert CONTENT_POSITIONS == frozenset(
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
)
def test_valid_controls_pass_validation():
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
validate_controls(c) # does not raise
def test_invalid_content_position_rejected():
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
with pytest.raises(ControlsError):
validate_controls(c)
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
@pytest.mark.parametrize("bad", [-1, 5, True])
def test_out_of_range_or_non_int_knob_rejected(field, bad):
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
kwargs[field] = bad
with pytest.raises(ControlsError):
validate_controls(Controls(**kwargs))
def test_parse_controls_from_mapping():
c = parse_controls(
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
)
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
def test_parse_controls_rejects_unknown_keys():
with pytest.raises(ControlsError):
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
def test_parse_controls_rejects_missing_keys():
with pytest.raises(ControlsError):
parse_controls({"content": "off", "left": 0})
+112
View File
@@ -0,0 +1,112 @@
from dataclasses import dataclass
import pytest
from player.controls import Controls
from player.state import Player, Playback, Transition, TransitionKind
@dataclass(frozen=True)
class FakeClip:
id: str
LIB = [FakeClip("base-a"), FakeClip("base-b")]
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
return Controls(content, left, right, dark, light, volume, brightness)
def test_first_update_to_video_fades_in_from_black():
p = Player(LIB)
t = p.update(_controls(content="video"))
assert t.kind == TransitionKind.FADE_FROM_BLACK
assert t.playback.clip_id == "base-a"
assert t.playback.content.video is True
def test_off_from_video_fades_to_black_and_silences():
p = Player(LIB)
p.update(_controls(content="video"))
t = p.update(_controls(content="off"))
assert t.kind == TransitionKind.FADE_TO_BLACK
assert t.playback.clip_id is None
assert t.playback.content.audio_source == "none"
def test_no_change_yields_none_transition():
p = Player(LIB)
p.update(_controls(content="video", left=1))
t = p.update(_controls(content="video", left=1))
assert t.kind == TransitionKind.NONE
def test_grade_change_is_a_live_update_not_a_crossfade():
# design §4.3: the Dark/Light grade is a continuous runtime op
p = Player(LIB)
p.update(_controls(content="video", dark=0, light=0))
t = p.update(_controls(content="video", dark=4, light=0))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.grade.tone == -1.0
def test_overlay_change_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="video", left=0))
t = p.update(_controls(content="video", left=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.overlay.level == 4
def test_restyle_change_crossfades_the_substrate():
# design §4.3: the Right v2v substrate is a pre-baked variant swap
p = Player(LIB)
p.update(_controls(content="video", right=0))
t = p.update(_controls(content="video", right=4))
assert t.kind == TransitionKind.CROSSFADE
assert t.playback.plan.restyle.variant == 4
def test_volume_only_change_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="video", volume=1))
t = p.update(_controls(content="video", volume=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.volume == 4
def test_audio_source_change_while_black_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="white_noise"))
t = p.update(_controls(content="music"))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.content.audio_source == "music"
def test_injected_base_chooser_is_used():
p = Player(LIB, choose_base=lambda lib: lib[1])
t = p.update(_controls(content="video"))
assert t.playback.clip_id == "base-b"
def test_empty_library_with_video_raises():
p = Player([])
with pytest.raises(ValueError):
p.update(_controls(content="video"))
def test_off_with_empty_library_is_fine():
p = Player([])
# levels at 0 match the initial black state, so this is a no-op transition
t = p.update(_controls(content="off", volume=0, brightness=0))
assert t.kind == TransitionKind.NONE # already black at init
assert t.playback.clip_id is None
def test_off_with_levels_from_black_is_a_live_update():
p = Player([])
t = p.update(_controls(content="off", volume=3, brightness=2))
assert t.kind == TransitionKind.LIVE_UPDATE # black->black, levels set
assert t.playback.clip_id is None
assert t.playback.volume == 3
+65
View File
@@ -0,0 +1,65 @@
import json
from tools.probe import Probe, probe_file
def _runner_for(payload):
def fake_runner(args): # mimics subprocess.run(...).stdout
return json.dumps(payload)
return fake_runner
def test_probe_parses_streams_and_format():
runner = _runner_for(
{
"streams": [
{
"codec_type": "video",
"width": 1920,
"height": 1080,
"disposition": {"attached_pic": 0},
},
{"codec_type": "audio"},
],
"format": {"duration": "12.5"},
}
)
p = probe_file("x.mp4", runner=runner)
assert isinstance(p, Probe)
assert any(s["codec_type"] == "video" for s in p.streams)
assert p.format["duration"] == "12.5"
def test_probe_audio_only():
runner = _runner_for(
{"streams": [{"codec_type": "audio"}], "format": {"duration": "30.0"}}
)
p = probe_file("x.mp3", runner=runner)
assert [s["codec_type"] for s in p.streams] == ["audio"]
def test_probe_audio_with_cover_art():
runner = _runner_for(
{
"streams": [
{"codec_type": "audio"},
{
"codec_type": "video",
"width": 600,
"height": 600,
"disposition": {"attached_pic": 1},
},
],
"format": {"duration": "200.0"},
}
)
p = probe_file("x.mp3", runner=runner)
cover = [s for s in p.streams if s.get("disposition", {}).get("attached_pic")]
assert len(cover) == 1
def test_probe_handles_missing_keys():
runner = _runner_for({})
p = probe_file("x.mp4", runner=runner)
assert p.streams == [] and p.format == {}
+59
View File
@@ -0,0 +1,59 @@
from hef.catalog import Record, validate
from hef.selection import Coordinate
from tools.review import approve, proposed_records
def make_record(**o):
base = dict(
id="a",
title="t",
source_url="u",
source_archive="nasa",
license="public_domain",
mode="video",
left=0,
right=0,
dark=0,
light=0,
duration_s=1,
file_path="p",
)
base.update(o)
return Record(**base)
def test_proposed_records_filters():
a = make_record(id="a", review_status="proposed")
b = make_record(id="b", review_status="approved")
assert [r.id for r in proposed_records([a, b])] == ["a"]
def test_approve_sets_status_and_timestamp():
r = make_record(review_status="proposed")
out = approve(r, reviewed_at="2026-06-04T13:00:00+00:00")
assert out.review_status == "approved"
assert out.reviewed_at == "2026-06-04T13:00:00+00:00"
def test_approve_can_override_coordinates():
r = make_record(left=0, right=0, dark=0, light=0, review_status="proposed")
out = approve(r, reviewed_at="t", coordinate=Coordinate(4, 1, 2, 3))
assert (out.left, out.right, out.dark, out.light) == (4, 1, 2, 3)
def test_approve_does_not_mutate_input():
r = make_record(review_status="proposed")
approve(r, reviewed_at="t")
assert r.review_status == "proposed" and r.reviewed_at is None
def test_approve_can_set_rationale():
r = make_record(review_status="proposed", rationale="auto")
out = approve(r, reviewed_at="t", rationale="human note")
assert out.rationale == "human note"
def test_approved_record_revalidates():
r = make_record(review_status="proposed")
out = approve(r, reviewed_at="t", coordinate=Coordinate(2, 2, 1, 1))
validate(out) # must not raise
+103
View File
@@ -0,0 +1,103 @@
import io
import pytest
from hef.catalog import Record, load_catalog, save_catalog
from tools.review_cli import main
def make_record(**o):
base = dict(
id="a",
title="t",
source_url="u",
source_archive="nasa",
license="public_domain",
mode="video",
left=0,
right=0,
dark=0,
light=0,
duration_s=1,
file_path="nasa/a.mp4",
)
base.update(o)
return Record(**base)
def test_help_parses():
with pytest.raises(SystemExit) as e:
main(["--help"])
assert e.value.code == 0
def test_no_proposed_records(tmp_path):
catalog = tmp_path / "library.jsonl"
save_catalog([make_record(id="x", review_status="approved", reviewed_at="t")], catalog)
out = io.StringIO()
rc = main(["--catalog", str(catalog), "--no-preview"], out=out)
assert rc == 0
assert "no proposed records" in out.getvalue()
def test_accept_flips_to_approved(tmp_path):
catalog = tmp_path / "library.jsonl"
save_catalog([make_record(id="a", review_status="proposed")], catalog)
out = io.StringIO()
rc = main(
["--catalog", str(catalog), "--no-preview"],
input_fn=lambda prompt: "a",
now_fn=lambda: "2026-06-04T13:00:00+00:00",
out=out,
)
assert rc == 0
r = load_catalog(catalog)[0]
assert r.review_status == "approved"
assert r.reviewed_at == "2026-06-04T13:00:00+00:00"
def test_skip_leaves_proposed(tmp_path):
catalog = tmp_path / "library.jsonl"
save_catalog([make_record(id="a", review_status="proposed")], catalog)
rc = main(
["--catalog", str(catalog), "--no-preview"],
input_fn=lambda prompt: "s",
out=io.StringIO(),
)
assert rc == 0
assert load_catalog(catalog)[0].review_status == "proposed"
def test_edit_overrides_coordinates(tmp_path):
catalog = tmp_path / "library.jsonl"
save_catalog([make_record(id="a", review_status="proposed")], catalog)
answers = iter(["e", "4", "1", "2", "3"])
rc = main(
["--catalog", str(catalog), "--no-preview"],
input_fn=lambda prompt: next(answers),
now_fn=lambda: "t",
out=io.StringIO(),
)
assert rc == 0
r = load_catalog(catalog)[0]
assert (r.left, r.right, r.dark, r.light) == (4, 1, 2, 3)
assert r.review_status == "approved"
def test_quit_stops_walk(tmp_path):
catalog = tmp_path / "library.jsonl"
save_catalog(
[
make_record(id="a", review_status="proposed"),
make_record(id="b", review_status="proposed"),
],
catalog,
)
rc = main(
["--catalog", str(catalog), "--no-preview"],
input_fn=lambda prompt: "q",
out=io.StringIO(),
)
assert rc == 0
statuses = {r.id: r.review_status for r in load_catalog(catalog)}
assert statuses == {"a": "proposed", "b": "proposed"}
+136
View File
@@ -0,0 +1,136 @@
import math
import pytest
from hef.catalog import Record
from hef.selection import Coordinate, Weights, distance, record_coordinate
def make_record(**overrides):
base = dict(
id="r",
title="t",
source_url="u",
source_archive="internet_archive",
license="public_domain",
mode="video",
left=0,
right=0,
dark=0,
light=0,
duration_s=600,
file_path="/media/r.mp4",
)
base.update(overrides)
return Record(**base)
def test_distance_zero_for_same_point():
c = Coordinate(2, 2, 2, 2)
assert distance(c, c) == 0.0
def test_distance_is_euclidean_by_default():
a = Coordinate(0, 0, 0, 0)
b = Coordinate(1, 1, 1, 1)
assert distance(a, b) == pytest.approx(2.0) # sqrt(1+1+1+1)
def test_weights_scale_planes_independently():
a = Coordinate(0, 0, 0, 0)
brain_only = Coordinate(1, 0, 0, 0)
mood_only = Coordinate(0, 0, 1, 0)
w = Weights(brain=4.0, mood=1.0)
assert distance(a, brain_only, w) == pytest.approx(2.0) # sqrt(4*1)
assert distance(a, mood_only, w) == pytest.approx(1.0) # sqrt(1*1)
def test_record_coordinate_extracts_axes():
record = make_record(left=1, right=2, dark=3, light=4)
assert record_coordinate(record) == Coordinate(1, 2, 3, 4)
from hef.selection import candidates_for_mode
def test_candidates_filter_to_exact_mode():
records = [make_record(id="v", mode="video"), make_record(id="a", mode="audio")]
result = candidates_for_mode(records, "audio", pool_size=4)
assert [r.id for r in result] == ["a"]
def test_av_does_not_fall_back_when_enough_av():
records = [make_record(id=f"av{i}", mode="av") for i in range(4)]
records.append(make_record(id="a", mode="audio"))
result = candidates_for_mode(records, "av", pool_size=4)
assert {r.id for r in result} == {"av0", "av1", "av2", "av3"}
def test_av_falls_back_to_audio_and_video_when_thin():
records = [
make_record(id="av0", mode="av"),
make_record(id="a", mode="audio"),
make_record(id="v", mode="video"),
]
result = candidates_for_mode(records, "av", pool_size=4)
assert {r.id for r in result} == {"av0", "a", "v"}
def test_candidates_rejects_none_mode():
with pytest.raises(ValueError):
candidates_for_mode([], "none", pool_size=4)
import random
from hef.selection import select
def test_none_mode_returns_none():
records = [make_record(id="v", mode="video")]
assert select(records, Coordinate(0, 0, 0, 0), "none") is None
def test_empty_pool_returns_none():
assert select([], Coordinate(0, 0, 0, 0), "video") is None
def test_select_returns_nearest_by_default():
near = make_record(id="near", mode="video", left=2, right=2, dark=2, light=2)
far = make_record(id="far", mode="video", left=0, right=0, dark=0, light=0)
result = select([far, near], Coordinate(2, 2, 2, 2), "video")
assert result.id == "near"
def test_select_is_deterministic_without_rng():
records = [
make_record(id="b", mode="video", left=1),
make_record(id="a", mode="video", left=1),
]
# Equal distance -> tie broken by id, so "a" wins deterministically.
result = select(records, Coordinate(1, 0, 0, 0), "video")
assert result.id == "a"
def test_select_with_rng_picks_within_nearest_pool():
records = [make_record(id=f"r{i}", mode="video", left=i % 5) for i in range(10)]
coord = Coordinate(2, 0, 0, 0)
rng = random.Random(0)
chosen = {select(records, coord, "video", pool_size=3, rng=rng).id for _ in range(50)}
# Only records inside the 3-nearest pool may ever be chosen.
ranked = sorted(records, key=lambda r: abs(r.left - 2))
allowed = {r.id for r in ranked[:3]}
assert chosen <= allowed
def test_approved_only_excludes_proposed():
proposed = make_record(id="p", mode="video", review_status="proposed")
approved = make_record(id="ok", mode="video", review_status="approved")
result = select([proposed, approved], Coordinate(0, 0, 0, 0), "video",
approved_only=True)
assert result.id == "ok"
def test_invalid_selector_mode_raises():
with pytest.raises(ValueError):
select([], Coordinate(0, 0, 0, 0), "telepathy")
+87
View File
@@ -0,0 +1,87 @@
import random
import pytest
from hef.catalog import Record
from hef.selection import Coordinate, Weights, ranked_candidates, select
def make_record(**overrides):
base = dict(
id="r",
title="t",
source_url="u",
source_archive="internet_archive",
license="public_domain",
mode="video",
left=0,
right=0,
dark=0,
light=0,
duration_s=600,
file_path="",
)
base.update(overrides)
return Record(**base)
def test_ranked_candidates_sorts_nearest_first_with_distances():
near = make_record(id="near", mode="video", left=1, right=1, dark=0, light=0)
far = make_record(id="far", mode="video", left=4, right=4, dark=4, light=4)
ranked = ranked_candidates([far, near], Coordinate(0, 0, 0, 0), "video")
assert [r.id for r, _ in ranked] == ["near", "far"]
assert ranked[0][1] < ranked[1][1]
def test_ranked_candidates_caps_at_pool_size():
recs = [make_record(id=f"r{i}", mode="video", left=i % 5) for i in range(10)]
ranked = ranked_candidates(recs, Coordinate(0, 0, 0, 0), "video", pool_size=3)
assert len(ranked) == 3
def test_ranked_candidates_filters_by_mode():
a = make_record(id="aud", mode="audio")
v = make_record(id="vid", mode="video")
ranked = ranked_candidates([a, v], Coordinate(0, 0, 0, 0), "audio")
assert [r.id for r, _ in ranked] == ["aud"]
def test_ranked_candidates_av_falls_back_to_audio_and_video():
a = make_record(id="aud", mode="audio")
v = make_record(id="vid", mode="video")
ranked = ranked_candidates([a, v], Coordinate(0, 0, 0, 0), "av", pool_size=4)
assert {r.id for r, _ in ranked} == {"aud", "vid"}
def test_ranked_candidates_approved_only():
p = make_record(id="prop", mode="video", review_status="proposed")
ok = make_record(id="appr", mode="video", review_status="approved")
ranked = ranked_candidates([p, ok], Coordinate(0, 0, 0, 0), "video", approved_only=True)
assert [r.id for r, _ in ranked] == ["appr"]
def test_ranked_candidates_rejects_none_mode():
with pytest.raises(ValueError):
ranked_candidates([], Coordinate(0, 0, 0, 0), "none")
def test_select_returns_rank_one_of_ranked_candidates():
recs = [
make_record(id="near", mode="video", left=1),
make_record(id="far", mode="video", left=4),
]
coord = Coordinate(0, 0, 0, 0)
ranked = ranked_candidates(recs, coord, "video")
assert select(recs, coord, "video", rng=None).id == ranked[0][0].id
def test_select_none_mode_still_returns_none():
recs = [make_record(id="v", mode="video")]
assert select(recs, Coordinate(0, 0, 0, 0), "none") is None
def test_select_shuffles_within_pool_when_rng_given():
recs = [make_record(id=f"r{i}", mode="video", left=i) for i in range(5)]
coord = Coordinate(0, 0, 0, 0)
picks = {select(recs, coord, "video", pool_size=5, rng=random.Random(s)).id for s in range(20)}
assert len(picks) > 1
+90
View File
@@ -0,0 +1,90 @@
import json
import pytest
from fastapi.testclient import TestClient
from simulator.app import create_app
@pytest.fixture
def manifest_path(tmp_path):
p = tmp_path / "manifest.json"
p.write_text(json.dumps({
"clips": [{
"id": "forest",
"title": "neutral forest",
"base_file": "forest/base.mp4",
"license": "poc", "source": "hef-poc",
"right_variants": {"4": {"file": "forest/right4.mp4"}},
"annotations": [{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1}],
"strings": {"en": {"detected.water": "flowing water"}},
}]
}))
return p
@pytest.fixture
def client(manifest_path):
return TestClient(create_app(manifest_path=manifest_path))
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
return dict(content=content, left=left, right=right, dark=dark,
light=light, volume=volume, brightness=brightness)
def test_alteration_returns_the_engine_plan(client):
resp = client.post("/api/alteration", json={"controls": _controls(left=4, right=2, dark=4)})
assert resp.status_code == 200
data = resp.json()
assert data["plan"]["overlay"]["level"] == 4
assert data["plan"]["restyle"]["variant"] == 2
assert data["plan"]["grade"]["tone"] == -1.0
assert data["content"]["video"] is True
def test_alteration_honors_off_as_black(client):
resp = client.post("/api/alteration", json={"controls": _controls(content="off")})
data = resp.json()
assert data["content"]["video"] is False
def test_alteration_accepts_calibration(client):
body = {"controls": _controls(light=4),
"calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "right_variant_map": [0, 1, 2, 3, 4]}}
resp = client.post("/api/alteration", json=body)
assert resp.json()["plan"]["grade"]["tone"] == 0.5
def test_alteration_rejects_out_of_range_knob(client):
resp = client.post("/api/alteration", json={"controls": _controls(left=7)})
assert resp.status_code == 422
def test_alteration_rejects_bad_content(client):
resp = client.post("/api/alteration", json={"controls": _controls(content="banana")})
assert resp.status_code == 422
def test_clips_returns_the_manifest(client):
resp = client.get("/api/clips")
assert resp.status_code == 200
data = resp.json()
assert data["clips"][0]["id"] == "forest"
assert data["clips"][0]["right_variants"]["0"]["file"] == "forest/base.mp4"
assert data["clips"][0]["annotations"][0]["key"] == "detected.water"
def test_retired_selection_endpoints_are_gone(client):
# The route no longer exists; the static catch-all yields 404 on GET and
# 405 on the (now-unrouted) POST. Either proves the endpoint is gone.
assert client.post("/api/select", json={}).status_code in (404, 405)
assert client.get("/api/catalog/meta").status_code == 404
def test_index_is_served():
client = TestClient(create_app())
resp = client.get("/")
assert resp.status_code == 200
assert "text/html" in resp.headers["content-type"]
assert "Alteration" in resp.text
+5
View File
@@ -0,0 +1,5 @@
def test_package_imports():
import hef
import hef.catalog
import hef.selection
assert hef is not None
+94
View File
@@ -0,0 +1,94 @@
from tools.probe import Probe
from tools.tagging import Tags, derive_tags
def test_video_and_audio_is_av():
p = Probe(
streams=[
{
"codec_type": "video",
"width": 1920,
"height": 1080,
"disposition": {"attached_pic": 0},
},
{"codec_type": "audio"},
],
format={"duration": "12.5"},
)
t = derive_tags(p)
assert isinstance(t, Tags)
assert t.mode == "av"
assert t.resolution == "1920x1080"
assert t.duration_s == 13 # round(12.5)
def test_audio_only():
p = Probe(streams=[{"codec_type": "audio"}], format={"duration": "30.0"})
t = derive_tags(p)
assert t.mode == "audio" and t.resolution == "" and t.duration_s == 30
def test_cover_art_stays_audio():
p = Probe(
streams=[
{"codec_type": "audio", "duration": "30.0"},
{
"codec_type": "video",
"width": 600,
"height": 600,
"disposition": {"attached_pic": 1},
},
],
format={"duration": "30.0"},
)
t = derive_tags(p)
assert t.mode == "audio" and t.resolution == "" and t.duration_s == 30
def test_video_only():
p = Probe(
streams=[
{
"codec_type": "video",
"width": 1280,
"height": 720,
"disposition": {"attached_pic": 0},
}
],
format={"duration": "5.0"},
)
t = derive_tags(p)
assert t.mode == "video" and t.resolution == "1280x720"
def test_duration_falls_back_to_longest_stream():
p = Probe(
streams=[
{"codec_type": "audio", "duration": "10.0"},
{
"codec_type": "video",
"width": 640,
"height": 480,
"duration": "42.4",
"disposition": {"attached_pic": 0},
},
],
format={},
)
t = derive_tags(p)
assert t.duration_s == 42 # round(42.4), longest stream
def test_duration_never_negative_and_zero_default():
p = Probe(streams=[{"codec_type": "audio"}], format={})
t = derive_tags(p)
assert t.duration_s == 0
def test_video_without_dimensions_yields_empty_resolution():
p = Probe(
streams=[{"codec_type": "video", "disposition": {"attached_pic": 0}}],
format={"duration": "3.0"},
)
t = derive_tags(p)
assert t.mode == "video" and t.resolution == ""
+134
View File
@@ -0,0 +1,134 @@
"""End-to-end: ingest (mocked boundaries) -> review -> select.
Plus opt-in tests that invoke real ffprobe/ffmpeg, skipped when absent.
"""
import re
import shutil
import subprocess
import pytest
from hef.catalog import load_catalog, validate_catalog
from hef.selection import Coordinate, select
from tools.drafting import HeuristicProposer
from tools.ingest.base import Candidate, ingest_candidate
from tools.mediatools import compute_dominant_color
from tools.probe import probe_file
from tools.review import approve, proposed_records
from tools.tagging import derive_tags
class _FakeFetcher:
archive = "nasa"
def search(self, query, *, limit):
return [
Candidate(
source_archive="nasa",
source_url="https://example.org/landing",
media_url="https://example.org/x.mp4",
title="Earthrise",
license="public_domain",
attribution="",
suggested_id="nasa-earthrise",
media_ext="mp4",
description="earth from the moon",
)
][:limit]
def resolve(self, identifier):
raise NotImplementedError
def _av_prober(path):
from tools.probe import Probe
return Probe(
streams=[
{"codec_type": "video", "width": 1920, "height": 1080, "disposition": {"attached_pic": 0}},
{"codec_type": "audio"},
],
format={"duration": "10.0"},
)
def _fake_downloader(url, dest):
dest.write_bytes(b"FAKE")
def test_end_to_end_ingest_review_select(tmp_path):
catalog = tmp_path / "library.jsonl"
media_root = tmp_path / "media"
# Ingest one candidate (all boundaries faked).
candidate = _FakeFetcher().search("earth", limit=1)[0]
rec = ingest_candidate(
candidate,
catalog_path=catalog,
media_root=media_root,
proposer=HeuristicProposer(),
prober=_av_prober,
downloader=_fake_downloader,
)
assert rec is not None and rec.review_status == "proposed"
# The proposed record can't be selected with approved_only yet.
loaded = load_catalog(catalog)
validate_catalog(loaded)
coord = Coordinate(loaded[0].left, loaded[0].right, loaded[0].dark, loaded[0].light)
assert select(loaded, coord, "av", approved_only=True) is None
# Review: approve it, persist.
pending = proposed_records(loaded)
assert len(pending) == 1
approved = approve(pending[0], reviewed_at="2026-06-04T13:00:00+00:00")
from hef.catalog import save_catalog
idx = {r.id: i for i, r in enumerate(loaded)}
loaded[idx[approved.id]] = approved
save_catalog(loaded, catalog)
# Reload, validate, and now select() finds it.
final = load_catalog(catalog)
validate_catalog(final)
picked = select(final, coord, "av", approved_only=True)
assert picked is not None and picked.id == "nasa-earthrise"
assert picked.review_status == "approved"
_HAS_FFMPEG = shutil.which("ffmpeg") is not None
_HAS_FFPROBE = shutil.which("ffprobe") is not None
@pytest.mark.skipif(not (_HAS_FFMPEG and _HAS_FFPROBE), reason="ffmpeg/ffprobe not installed")
def test_real_ffprobe_on_generated_clip(tmp_path):
clip = tmp_path / "clip.mp4"
subprocess.run(
[
"ffmpeg", "-v", "quiet", "-y",
"-f", "lavfi", "-i", "testsrc=duration=1:size=320x240:rate=30",
"-f", "lavfi", "-i", "sine=frequency=440:duration=1",
"-shortest", str(clip),
],
check=True,
)
tags = derive_tags(probe_file(clip))
assert tags.mode == "av"
assert tags.resolution == "320x240"
assert tags.duration_s == 1
@pytest.mark.skipif(not _HAS_FFMPEG, reason="ffmpeg not installed")
def test_real_dominant_color_on_generated_clip(tmp_path):
clip = tmp_path / "clip.mp4"
subprocess.run(
[
"ffmpeg", "-v", "quiet", "-y",
"-f", "lavfi", "-i", "color=c=red:duration=1:size=320x240:rate=30",
str(clip),
],
check=True,
)
color = compute_dominant_color(clip, midpoint_s=0.5)
assert re.fullmatch(r"#[0-9a-f]{6}", color)
+5
View File
@@ -0,0 +1,5 @@
def test_tools_package_imports():
import tools
import tools.http
assert tools is not None
View File
+90
View File
@@ -0,0 +1,90 @@
"""Heuristic coordinate proposer — a deterministic, rule-based DRAFT seed.
The four curatorial coordinates are a human act (design §11: no automatic ML
coordinate tagging). This proposer only suggests a starting point written as
review_status='proposed'; a human blesses every coordinate before selection.
There is no ML model here just archive priors (brain plane) and keyword
nudges (mood plane), grounded in the §8 sourcingaxis map.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol
from hef.selection import Coordinate
COORD_MIN, COORD_MAX = 0, 4
# Brain plane (left = verbal/spoken/educational, right = music/wordless/abstract).
_ARCHIVE_PRIORS = {
"librivox": (4, 0), # spoken readings
"internet_archive": (3, 1), # educational / industrial / Prelinger
"musopen": (0, 4), # classical music
"fma": (0, 4), # music
"nasa": (0, 4), # wordless awe
"freesound": (0, 4), # abstract field recordings
}
_ARCHIVE_DEFAULT = (2, 2)
# Mood plane keywords (substring match so "thunderstorm" trips "storm").
_DARK_WORDS = (
"storm", "night", "noir", "requiem", "minor", "war", "funeral",
"decay", "death", "grief", "dusk", "winter", "mourning", "shadow",
)
_LIGHT_WORDS = (
"sunrise", "dawn", "garden", "spring", "joy", "hope", "major",
"bright", "bloom", "summer", "smile", "light", "morning",
)
def _clamp(v: int) -> int:
return max(COORD_MIN, min(COORD_MAX, v))
@dataclass
class Signals:
title: str
description: str
source_archive: str
mode: str
duration_s: int
@dataclass
class Draft:
coordinate: Coordinate
rationale: str
class Proposer(Protocol):
def propose(self, signals: Signals) -> Draft: ...
class HeuristicProposer:
def propose(self, signals: Signals) -> Draft:
left, right = _ARCHIVE_PRIORS.get(
signals.source_archive, _ARCHIVE_DEFAULT
)
text = f"{signals.title} {signals.description}".lower()
dark_hits = [w for w in _DARK_WORDS if w in text]
light_hits = [w for w in _LIGHT_WORDS if w in text]
dark = _clamp(len(dark_hits))
light = _clamp(len(light_hits))
coordinate = Coordinate(_clamp(left), _clamp(right), dark, light)
# Rationale: name the dominant brain prior + any mood keywords.
brain = (
f"{signals.source_archive}"
+ ("strong left" if left >= right else "strong right")
)
mood_bits = []
if dark_hits:
mood_bits.append(f"{dark_hits[0]!r} → dark")
if light_hits:
mood_bits.append(f"{light_hits[0]!r} → light")
rationale = "; ".join([brain, *mood_bits])
return Draft(coordinate=coordinate, rationale=rationale)
+33
View File
@@ -0,0 +1,33 @@
"""Minimal HTTP client over urllib: timeout, small retry, user-agent. Injectable."""
from __future__ import annotations
import json as _json
import time
import urllib.request
USER_AGENT = "human-experience-filter-ingest/0.1 (+local art installation)"
class HttpClient:
def __init__(self, *, timeout=30.0, retries=2, opener=None):
self.timeout, self.retries = timeout, retries
self._opener = opener or urllib.request.urlopen
def get_bytes(self, url, *, headers=None):
last = None
for attempt in range(self.retries + 1):
try:
req = urllib.request.Request(
url, headers={"User-Agent": USER_AGENT, **(headers or {})}
)
with self._opener(req, timeout=self.timeout) as resp:
return resp.read()
except Exception as exc: # noqa: BLE001 - retried below
last = exc
if attempt < self.retries:
time.sleep(0.2 * (attempt + 1))
raise last
def get_json(self, url, *, headers=None):
return _json.loads(self.get_bytes(url, headers=headers).decode("utf-8"))
View File
+148
View File
@@ -0,0 +1,148 @@
"""Ingest seam: a Candidate shape, the Fetcher protocol, and the pipeline.
Every external boundary archive HTTP (download), ffprobe (prober), and the
optional ffmpeg dominant-color (color_fn) is injected, so the whole pipeline
is exercised hermetically (no network, no binaries, no disk beyond tmp).
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol
from hef.catalog import (
Record,
append_record,
load_catalog,
validate,
validate_catalog,
)
from tools.drafting import Signals
from tools.http import HttpClient
from tools.mediatools import compute_dominant_color
from tools.probe import probe_file
from tools.tagging import derive_tags
log = logging.getLogger(__name__)
_VIDEO_MODES = {"video", "av"}
@dataclass
class Candidate:
source_archive: str # origin label, e.g. internet_archive | librivox | nasa
source_url: str # human/landing URL recorded in the record
media_url: str # direct download URL of the chosen file
title: str
license: str # normalized (tools.licensing) -> hef LICENSES vocab
attribution: str # "" unless the license requires it
suggested_id: str # stable id derived from archive + archive-identifier
media_ext: str # file extension for the download target
description: str = "" # free text used by drafting signals; lands in notes
class Fetcher(Protocol):
archive: str
def search(self, query: str, *, limit: int) -> list[Candidate]: ...
def resolve(self, identifier: str) -> Candidate: ...
def _default_downloader(url, dest) -> None:
client = HttpClient()
dest.write_bytes(client.get_bytes(url))
def ingest_candidate(
candidate: Candidate,
*,
catalog_path,
media_root,
proposer,
prober=probe_file,
downloader=_default_downloader,
compute_color: bool = False,
color_fn=compute_dominant_color,
):
"""Ingest one candidate into the catalog as a `proposed` record.
Returns the appended Record, or None if it was skipped as a duplicate.
Raises CatalogError if the resulting record is invalid (e.g. an unmappable
license) nothing is appended in that case.
"""
catalog_path = Path(catalog_path)
media_root = Path(media_root)
# 1. Dedupe.
existing = load_catalog(catalog_path) if catalog_path.exists() else []
validate_catalog(existing)
if any(r.id == candidate.suggested_id for r in existing):
log.info("already in catalog, skipping: %s", candidate.suggested_id)
return None
# 2. Download (skip if already present).
rel_path = f"{candidate.source_archive}/{candidate.suggested_id}.{candidate.media_ext}"
dest = media_root / rel_path
dest.parent.mkdir(parents=True, exist_ok=True)
if not dest.exists():
downloader(candidate.media_url, dest)
# 3. Mechanical tagging.
tags = derive_tags(prober(dest))
dominant_color = ""
if compute_color and tags.mode in _VIDEO_MODES:
dominant_color = color_fn(dest, midpoint_s=tags.duration_s / 2)
# 4. Draft coordinates.
draft = proposer.propose(
Signals(
title=candidate.title,
description=candidate.description,
source_archive=candidate.source_archive,
mode=tags.mode,
duration_s=tags.duration_s,
)
)
coord = draft.coordinate
# 5. Build the record — proposed/reviewed_at=None falls out of the defaults.
record = Record(
id=candidate.suggested_id,
title=candidate.title,
source_url=candidate.source_url,
source_archive=candidate.source_archive,
license=candidate.license,
mode=tags.mode,
left=coord.left,
right=coord.right,
dark=coord.dark,
light=coord.light,
duration_s=tags.duration_s,
file_path=rel_path,
attribution=candidate.attribution,
resolution=tags.resolution,
dominant_color=dominant_color,
rationale=draft.rationale,
notes=candidate.description,
)
# 6. Validate + re-check uniqueness + append.
validate(record)
if any(r.id == record.id for r in existing): # racey re-check
log.info("already in catalog (race), skipping: %s", record.id)
return None
append_record(record, catalog_path)
return record
def ingest_search(fetcher: Fetcher, query: str, *, limit: int, **kw):
"""Run ingest_candidate over each search hit. Returns the appended records."""
appended = []
for candidate in fetcher.search(query, limit=limit):
rec = ingest_candidate(candidate, **kw)
if rec is not None:
appended.append(rec)
return appended
+26
View File
@@ -0,0 +1,26 @@
"""Free Music Archive fetcher — DEFERRED (see spec §6.4).
CC-licensed music (cc_by / cc_by_nc / cc0). Highest API-stability risk: the
public FMA API has been deprecated/changed, so a real implementation resolves
from a track URL + page metadata. The Fetcher seam is wired; it raises until
implemented.
"""
from __future__ import annotations
from tools.ingest.base import Candidate
_DEFERRED = "deferred — see spec §6.4 (FMA public API deprecated/changed)"
class FmaFetcher:
archive = "fma"
def __init__(self, client):
self.client = client
def search(self, query: str, *, limit: int) -> list[Candidate]:
raise NotImplementedError(_DEFERRED)
def resolve(self, identifier: str) -> Candidate:
raise NotImplementedError(_DEFERRED)
+34
View File
@@ -0,0 +1,34 @@
"""Freesound fetcher — DEFERRED (see spec §6.4).
CC-licensed sound effects / field recordings (cc0 / cc_by / cc_by_nc) via API
v2 (https://freesound.org/apiv2). Requires an API token supplied through the
FREESOUND_API_TOKEN environment variable (or the macOS Keychain). That token is
a SECRET: it is read from the environment only and must NEVER be written into a
record, a log line, or a transcript (wgl hard secrets rule, spec §9).
The Fetcher seam is wired; it raises until implemented.
"""
from __future__ import annotations
import os
from tools.ingest.base import Candidate
_DEFERRED = "deferred — see spec §6.4 (Freesound requires FREESOUND_API_TOKEN)"
TOKEN_ENV = "FREESOUND_API_TOKEN"
class FreesoundFetcher:
archive = "freesound"
def __init__(self, client, token: str | None = None):
self.client = client
# Read the secret from the environment only; never log or store it.
self._token = token or os.environ.get(TOKEN_ENV)
def search(self, query: str, *, limit: int) -> list[Candidate]:
raise NotImplementedError(_DEFERRED)
def resolve(self, identifier: str) -> Candidate:
raise NotImplementedError(_DEFERRED)
+91
View File
@@ -0,0 +1,91 @@
"""Internet Archive fetcher (incl. Prelinger).
Metadata API: https://archive.org/metadata/<id> (keyless); search via
advancedsearch.php. License is read from `licenseurl`/`rights`/`license` and
normalized through tools.licensing. When no explicit license is present the
item is assumed public_domain and flagged in notes for review (§6.4); an
explicitly non-free license (e.g. All Rights Reserved) raises and is not
ingested.
"""
from __future__ import annotations
from tools.ingest.base import Candidate
from tools.licensing import normalize_license
META = "https://archive.org/metadata/"
SEARCH = "https://archive.org/advancedsearch.php"
_NO_LICENSE_CAVEAT = "[IA: no explicit license — assumed public_domain, verify before use]"
_MEDIA_EXTS = (
".mp4", ".mov", ".m4v", ".ogv", ".webm",
".mp3", ".m4a", ".wav", ".flac", ".ogg",
)
def _ext_from_name(name: str) -> str:
return name.rsplit(".", 1)[-1].lower() if "." in name else ""
def _pick_file(files):
originals = [f for f in files if f.get("source") == "original"]
for pool in (originals, files):
for f in pool:
if str(f.get("name", "")).lower().endswith(_MEDIA_EXTS):
return f
return None
class InternetArchiveFetcher:
archive = "internet_archive"
def __init__(self, client):
self.client = client
def resolve(self, identifier: str) -> Candidate:
data = self.client.get_json(f"{META}{identifier}")
meta = data.get("metadata", {})
files = data.get("files", []) or []
server = data.get("server", "")
directory = data.get("dir", "")
chosen = _pick_file(files)
media_url = ""
media_ext = ""
if chosen and server:
name = chosen["name"]
media_url = f"https://{server}{directory}/{name}"
media_ext = _ext_from_name(name)
raw_license = (
meta.get("licenseurl") or meta.get("rights") or meta.get("license") or ""
)
creator = meta.get("creator", "") or ""
if isinstance(creator, list):
creator = ", ".join(str(c) for c in creator)
description = (meta.get("description", "") or "").strip()
ambiguous = (not raw_license) or ("no known copyright" in raw_license.lower())
if raw_license:
lic, attr = normalize_license(raw_license, creator=creator)
else:
lic, attr = "public_domain", ""
if ambiguous:
description = f"{description} {_NO_LICENSE_CAVEAT}".strip()
return Candidate(
source_archive=self.archive,
source_url=f"https://archive.org/details/{identifier}",
media_url=media_url,
title=meta.get("title", identifier),
license=lic,
attribution=attr,
suggested_id=f"ia-{identifier}",
media_ext=media_ext,
description=description,
)
def search(self, query: str, *, limit: int) -> list[Candidate]:
url = f"{SEARCH}?q={query}&fl[]=identifier&rows={limit}&output=json"
data = self.client.get_json(url)
docs = (data.get("response", {}).get("docs") or [])[:limit]
return [self.resolve(doc["identifier"]) for doc in docs]
+71
View File
@@ -0,0 +1,71 @@
"""LibriVox fetcher — public-domain audiobook recordings.
JSON API: https://librivox.org/api/feed/audiobooks (keyless). LibriVox is
public domain by charter; the reader/author is credited in notes but
attribution is not required. Section-level (per-track) file resolution is a
future refinement; this resolves the audiobook's zip as the media target.
"""
from __future__ import annotations
import re
from tools.ingest.base import Candidate
from tools.licensing import librivox_license
BASE = "https://librivox.org/api/feed/audiobooks"
def _slug(text: str) -> str:
s = re.sub(r"[^a-z0-9]+", "-", (text or "").lower()).strip("-")
return s or "untitled"
def _ext_from_url(url: str, default: str) -> str:
tail = url.rsplit("/", 1)[-1]
return tail.rsplit(".", 1)[-1].lower() if "." in tail else default
class LibriVoxFetcher:
archive = "librivox"
def __init__(self, client):
self.client = client
def _candidate(self, book) -> Candidate:
title = book.get("title", "")
authors = book.get("authors") or []
credit = ", ".join(
f"{a.get('first_name', '')} {a.get('last_name', '')}".strip()
for a in authors
).strip(", ")
media_url = book.get("url_zip_file", "")
description = (book.get("description", "") or "").strip()
if credit:
description = (description + f" (author: {credit})").strip()
lic, attr = librivox_license()
return Candidate(
source_archive=self.archive,
source_url=book.get("url_librivox", ""),
media_url=media_url,
title=title,
license=lic,
attribution=attr,
suggested_id=f"librivox-{_slug(title)}",
media_ext=_ext_from_url(media_url, "zip"),
description=description,
)
def search(self, query: str, *, limit: int) -> list[Candidate]:
url = f"{BASE}/title/^{query}?format=json&limit={limit}"
data = self.client.get_json(url)
books = (data.get("books") or [])[:limit]
return [self._candidate(b) for b in books]
def resolve(self, identifier: str) -> Candidate:
url = f"{BASE}/id/{identifier}?format=json"
data = self.client.get_json(url)
books = data.get("books") or []
if not books:
raise ValueError(f"librivox: no audiobook for {identifier!r}")
return self._candidate(books[0])
+25
View File
@@ -0,0 +1,25 @@
"""Musopen fetcher — DEFERRED (see spec §6.4).
Public-domain / CC classical music. Access has historically been gated and may
require an API key (a secret never logged, see spec §9). The Fetcher seam is
wired so enabling it later is purely additive; it raises until implemented.
"""
from __future__ import annotations
from tools.ingest.base import Candidate
_DEFERRED = "deferred — see spec §6.4 (Musopen access may require an API key)"
class MusopenFetcher:
archive = "musopen"
def __init__(self, client):
self.client = client
def search(self, query: str, *, limit: int) -> list[Candidate]:
raise NotImplementedError(_DEFERRED)
def resolve(self, identifier: str) -> Candidate:
raise NotImplementedError(_DEFERRED)
+79
View File
@@ -0,0 +1,79 @@
"""NASA fetcher — public-domain imagery/video.
JSON API: https://images-api.nasa.gov/search (keyless). Each search item links
an asset-collection JSON (the item's `href`) listing the concrete file URLs;
the chosen media file is resolved from there. NASA media is public domain, but
some items embed third-party content flagged in notes for review (§6.4).
"""
from __future__ import annotations
from tools.ingest.base import Candidate
SEARCH = "https://images-api.nasa.gov/search"
_THIRD_PARTY_CAVEAT = "[NASA: may embed third-party content — verify before use]"
_VIDEO_EXTS = (".mp4", ".mov", ".m4v", ".webm")
_AUDIO_EXTS = (".mp3", ".m4a", ".wav", ".flac", ".ogg")
def _ext_from_url(url: str) -> str:
tail = url.rsplit("/", 1)[-1].split("?", 1)[0]
return tail.rsplit(".", 1)[-1].lower() if "." in tail else ""
def _pick_asset(assets, media_type: str) -> str:
if not isinstance(assets, list):
return ""
prefs = _AUDIO_EXTS if media_type == "audio" else _VIDEO_EXTS
for ext in prefs:
for url in assets:
if isinstance(url, str) and url.lower().endswith(ext):
return url
for url in assets:
if isinstance(url, str) and not url.lower().endswith((".jpg", ".png", ".json")):
return url
return ""
class NasaFetcher:
archive = "nasa"
def __init__(self, client):
self.client = client
def _candidate(self, item) -> Candidate:
data0 = (item.get("data") or [{}])[0]
nasa_id = data0.get("nasa_id", "")
title = data0.get("title", "")
media_type = data0.get("media_type", "")
description = (data0.get("description", "") or "").strip()
description = f"{description} {_THIRD_PARTY_CAVEAT}".strip()
href = item.get("href", "")
assets = self.client.get_json(href) if href else []
media_url = _pick_asset(assets, media_type)
return Candidate(
source_archive=self.archive,
source_url=f"https://images.nasa.gov/details-{nasa_id}",
media_url=media_url,
title=title,
license="public_domain",
attribution="",
suggested_id=f"nasa-{nasa_id}",
media_ext=_ext_from_url(media_url),
description=description,
)
def search(self, query: str, *, limit: int) -> list[Candidate]:
url = f"{SEARCH}?q={query}&media_type=video"
data = self.client.get_json(url)
items = (data.get("collection", {}).get("items") or [])[:limit]
return [self._candidate(i) for i in items]
def resolve(self, identifier: str) -> Candidate:
url = f"{SEARCH}?nasa_id={identifier}"
data = self.client.get_json(url)
items = data.get("collection", {}).get("items") or []
if not items:
raise ValueError(f"nasa: no item for {identifier!r}")
return self._candidate(items[0])
+99
View File
@@ -0,0 +1,99 @@
"""CLI entry point for ingest: `python -m tools.ingest_cli <archive> ...`.
Wires a named fetcher + the HeuristicProposer into the ingest pipeline. Secrets
(e.g. FREESOUND_API_TOKEN) are read from the environment only, never as flags.
"""
from __future__ import annotations
import argparse
import os
import sys
from tools.drafting import HeuristicProposer
from tools.http import HttpClient
from tools.ingest.base import ingest_candidate, ingest_search
from tools.ingest.fma import FmaFetcher
from tools.ingest.freesound import FreesoundFetcher
from tools.ingest.internet_archive import InternetArchiveFetcher
from tools.ingest.librivox import LibriVoxFetcher
from tools.ingest.musopen import MusopenFetcher
from tools.ingest.nasa import NasaFetcher
FETCHERS = {
"librivox": LibriVoxFetcher,
"nasa": NasaFetcher,
"internet_archive": InternetArchiveFetcher,
"musopen": MusopenFetcher,
"fma": FmaFetcher,
"freesound": FreesoundFetcher,
}
DEFAULT_CATALOG = "catalog/library.jsonl"
DEFAULT_MEDIA_ROOT = "./media"
def _build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(
prog="tools.ingest_cli",
description="Ingest candidates from an archive into the catalog as proposed records.",
)
p.add_argument("archive", help=f"source archive ({', '.join(sorted(FETCHERS))})")
p.add_argument("--query", help="search query")
p.add_argument("--resolve", metavar="IDENTIFIER", help="ingest a single item by id/URL")
p.add_argument("--limit", type=int, default=5, help="max search hits (default 5)")
p.add_argument("--catalog", default=DEFAULT_CATALOG, help=f"catalog JSONL (default {DEFAULT_CATALOG})")
p.add_argument(
"--media-root",
default=os.environ.get("HEF_MEDIA_ROOT", DEFAULT_MEDIA_ROOT),
help="download root (env HEF_MEDIA_ROOT; default ./media)",
)
p.add_argument(
"--dominant-color",
action="store_true",
help="compute dominant_color for video/av (opt-in, ffmpeg-only)",
)
return p
def main(argv=None) -> int:
args = _build_parser().parse_args(argv)
cls = FETCHERS.get(args.archive)
if cls is None:
print(
f"error: unknown archive {args.archive!r}; "
f"choose one of {', '.join(sorted(FETCHERS))}",
file=sys.stderr,
)
return 2
if not args.query and not args.resolve:
print("error: provide --query or --resolve", file=sys.stderr)
return 2
fetcher = cls(HttpClient())
kw = dict(
catalog_path=args.catalog,
media_root=args.media_root,
proposer=HeuristicProposer(),
compute_color=args.dominant_color,
)
try:
if args.resolve:
rec = ingest_candidate(fetcher.resolve(args.resolve), **kw)
appended = [rec] if rec is not None else []
else:
appended = ingest_search(fetcher, args.query, limit=args.limit, **kw)
except NotImplementedError as exc:
print(f"error: {exc}", file=sys.stderr)
return 3
print(f"ingested {len(appended)} proposed record(s) into {args.catalog}")
for rec in appended:
print(f" {rec.id} [{rec.mode}] {rec.title}")
return 0
if __name__ == "__main__": # pragma: no cover
raise SystemExit(main())
+63
View File
@@ -0,0 +1,63 @@
"""Normalize per-archive origin license metadata to the hef.catalog LICENSES vocab.
Maps Creative Commons URLs / identifiers and public-domain markers to
{public_domain, cc0, cc_by, cc_by_nc}. For the attribution licenses (cc_by,
cc_by_nc) it builds a non-empty attribution string hef.catalog.validate()
rejects those licenses with an empty attribution. Anything unmappable is
rejected (a piece whose license can't be established is not ingested).
"""
from __future__ import annotations
import re
def _attribution(creator: str, license_name: str, raw: str, default_label: str) -> str:
label = license_name or default_label or raw
parts = []
if creator:
parts.append(creator)
parts.append(label)
return "".join(parts)
def normalize_license(raw, *, creator="", license_name=""):
"""Return (license, attribution) for an origin license string.
Raises ValueError when `raw` does not map to an allowed license.
"""
text = (raw or "").strip().lower()
if not text:
raise ValueError(f"unmappable license: {raw!r}")
# CC0 / public-domain dedication (check before the generic publicdomain
# markers, since the CC0 URL also contains "publicdomain").
if "publicdomain/zero" in text or re.search(r"\bcc[ _-]?0\b", text):
return ("cc0", "")
# Public Domain Mark / explicit public domain / no-known-copyright.
pd_markers = (
"publicdomain/mark",
"public_domain",
"public domain",
"publicdomain",
"no known copyright",
"pdm",
)
if any(m in text for m in pd_markers):
return ("public_domain", "")
# CC BY-NC must be checked before the bare CC BY.
if re.search(r"by[ _-]?nc", text):
return ("cc_by_nc", _attribution(creator, license_name, raw, "CC BY-NC"))
# CC BY.
if re.search(r"\bcc[ _-]?by\b", text) or "licenses/by" in text:
return ("cc_by", _attribution(creator, license_name, raw, "CC BY"))
raise ValueError(f"unmappable license: {raw!r}")
def librivox_license():
"""LibriVox recordings are public domain by charter (attribution not required)."""
return ("public_domain", "")
+58
View File
@@ -0,0 +1,58 @@
"""ffmpeg helpers: representative frame; OPTIONAL dominant color. Runner injectable."""
from __future__ import annotations
import subprocess
def _run_bytes(args) -> bytes:
return subprocess.run(args, capture_output=True, check=True).stdout
def dominant_color_from_rgb(rgb: bytes) -> str:
r, g, b = rgb[0], rgb[1], rgb[2]
return f"#{r:02x}{g:02x}{b:02x}"
def compute_dominant_color(path, *, midpoint_s=0.0, runner=_run_bytes) -> str:
"""ffmpeg-only single-color palette of a mid-segment frame -> #rrggbb."""
args = [
"ffmpeg",
"-v",
"quiet",
"-ss",
str(midpoint_s),
"-i",
str(path),
"-vf",
"thumbnail,palettegen=max_colors=1",
"-frames:v",
"1",
"-f",
"rawvideo",
"-pix_fmt",
"rgb24",
"-",
]
return dominant_color_from_rgb(runner(args))
def extract_frame(path, dest, *, midpoint_s=0.0, runner=None):
"""Write one representative frame to dest (PNG) for the review preview."""
runner = runner or (lambda a: subprocess.run(a, check=True))
runner(
[
"ffmpeg",
"-v",
"quiet",
"-y",
"-ss",
str(midpoint_s),
"-i",
str(path),
"-frames:v",
"1",
str(dest),
]
)
return dest
+34
View File
@@ -0,0 +1,34 @@
"""ffprobe wrapper -> parsed streams/format. Subprocess runner is injectable."""
from __future__ import annotations
import json
import subprocess
from dataclasses import dataclass
@dataclass
class Probe:
streams: list
format: dict
def _default_runner(args) -> str:
return subprocess.run(args, capture_output=True, text=True, check=True).stdout
def probe_file(path, *, runner=_default_runner) -> Probe:
out = runner(
[
"ffprobe",
"-v",
"quiet",
"-print_format",
"json",
"-show_format",
"-show_streams",
str(path),
]
)
data = json.loads(out)
return Probe(streams=data.get("streams", []), format=data.get("format", {}))
+34
View File
@@ -0,0 +1,34 @@
"""Review state-transition core (pure, tested): proposed -> approved.
No I/O, no clock the timestamp is injected so the logic stays deterministic.
The CLI persists the result via hef.catalog.save_catalog after re-validating.
"""
from __future__ import annotations
from dataclasses import replace
def proposed_records(records):
"""Records still awaiting review."""
return [r for r in records if r.review_status == "proposed"]
def approve(record, *, reviewed_at, coordinate=None, rationale=None):
"""Return an approved copy of `record` (the input is not mutated).
review_status -> 'approved' and reviewed_at is stamped. Optionally override
the four coordinates with a human correction and/or replace the rationale.
The caller re-validates before saving.
"""
changes = {"review_status": "approved", "reviewed_at": reviewed_at}
if coordinate is not None:
changes.update(
left=coordinate.left,
right=coordinate.right,
dark=coordinate.dark,
light=coordinate.light,
)
if rationale is not None:
changes["rationale"] = rationale
return replace(record, **changes)
+154
View File
@@ -0,0 +1,154 @@
"""Interactive review walk: `python -m tools.review_cli`.
Thin shell over the tested review core (tools.review). Walks each proposed
record showing its mechanical fields, the proposed coordinates + rationale, and
a best-effort preview frame, then prompts accept/edit/skip/quit. Approvals are
persisted (full rewrite) after each one, so an interrupted session keeps its
progress.
"""
from __future__ import annotations
import argparse
import os
import platform
import shutil
import subprocess
import sys
import tempfile
from datetime import datetime, timezone
from pathlib import Path
from hef.catalog import load_catalog, save_catalog, validate_catalog
from tools.mediatools import extract_frame
from tools.review import approve, proposed_records
DEFAULT_CATALOG = "catalog/library.jsonl"
DEFAULT_MEDIA_ROOT = "./media"
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(
prog="tools.review_cli",
description="Walk proposed records and approve/correct their coordinates.",
)
p.add_argument("--catalog", default=DEFAULT_CATALOG)
p.add_argument(
"--media-root",
default=os.environ.get("HEF_MEDIA_ROOT", DEFAULT_MEDIA_ROOT),
)
p.add_argument("--no-preview", action="store_true", help="skip preview rendering")
return p
def _show(rec, out) -> None:
print(f"\n{'=' * 60}", file=out)
print(f" id {rec.id}", file=out)
print(f" title {rec.title}", file=out)
print(f" source {rec.source_archive} {rec.source_url}", file=out)
attr = f" ({rec.attribution})" if rec.attribution else ""
print(f" license {rec.license}{attr}", file=out)
print(f" media mode={rec.mode} {rec.duration_s}s {rec.resolution}", file=out)
if rec.dominant_color:
print(f" color {rec.dominant_color}", file=out)
print(
f" proposed left={rec.left} right={rec.right} dark={rec.dark} light={rec.light}",
file=out,
)
print(f" rationale {rec.rationale}", file=out)
if rec.notes:
print(f" notes {rec.notes}", file=out)
def _open_cmd():
return "open" if platform.system() == "Darwin" else "xdg-open"
def _preview(rec, media_root, out) -> None:
"""Best-effort preview; never fatal. Requires ffmpeg for video frames."""
if not shutil.which("ffmpeg"):
return
src = Path(media_root) / rec.file_path
if not src.exists():
print(f" (preview: media not found at {src})", file=out)
return
try:
midpoint = rec.duration_s / 2 if rec.duration_s else 0.0
if rec.mode in ("video", "av"):
dest = Path(tempfile.gettempdir()) / f"hef-preview-{rec.id}.png"
extract_frame(src, dest, midpoint_s=midpoint)
else: # audio -> waveform thumbnail
dest = Path(tempfile.gettempdir()) / f"hef-preview-{rec.id}.png"
subprocess.run(
[
"ffmpeg", "-v", "quiet", "-y", "-i", str(src),
"-filter_complex", "showwavespic=s=640x240", "-frames:v", "1",
str(dest),
],
check=True,
)
opener = shutil.which(_open_cmd())
if opener:
subprocess.run([opener, str(dest)], check=False)
except Exception as exc: # noqa: BLE001 - preview is best-effort
print(f" (preview failed: {exc})", file=out)
def _prompt_coords(rec, input_fn, out):
from hef.selection import Coordinate
def _ask(axis, current):
raw = input_fn(f" {axis} [{current}]: ").strip()
if not raw:
return current
try:
return max(0, min(4, int(raw)))
except ValueError:
print(f" (not an int, keeping {current})", file=out)
return current
return Coordinate(
_ask("left", rec.left),
_ask("right", rec.right),
_ask("dark", rec.dark),
_ask("light", rec.light),
)
def main(argv=None, *, input_fn=input, now_fn=_now, out=sys.stdout) -> int:
args = _build_parser().parse_args(argv)
records = load_catalog(args.catalog)
validate_catalog(records)
pending = proposed_records(records)
if not pending:
print("no proposed records to review", file=out)
return 0
pos = {r.id: i for i, r in enumerate(records)}
print(f"{len(pending)} proposed record(s) to review", file=out)
for rec in pending:
_show(rec, out)
if not args.no_preview:
_preview(rec, args.media_root, out)
choice = input_fn("[a]ccept / [e]dit / [s]kip / [q]uit > ").strip().lower()
if choice == "q":
break
if choice == "s" or choice not in ("a", "e"):
continue
coordinate = _prompt_coords(rec, input_fn, out) if choice == "e" else None
approved = approve(records[pos[rec.id]], reviewed_at=now_fn(), coordinate=coordinate)
records[pos[rec.id]] = approved
save_catalog(records, args.catalog)
print(f" approved {rec.id}", file=out)
return 0
if __name__ == "__main__": # pragma: no cover
raise SystemExit(main())
+65
View File
@@ -0,0 +1,65 @@
"""Mechanical tagging: derive mode/duration_s/resolution from a Probe.
Honors the cover-art guard (spec §5.1): an embedded album-art image in an audio
file shows up as a video stream with disposition.attached_pic == 1; it must NOT
make the file 'av'.
"""
from __future__ import annotations
import math
from dataclasses import dataclass
from tools.probe import Probe
@dataclass
class Tags:
mode: str
duration_s: int
resolution: str
def _is_real_video(stream) -> bool:
if stream.get("codec_type") != "video":
return False
return stream.get("disposition", {}).get("attached_pic", 0) != 1
def _parse_duration(value) -> float:
try:
return float(value)
except (TypeError, ValueError):
return 0.0
def derive_tags(probe: Probe) -> Tags:
real_video = [s for s in probe.streams if _is_real_video(s)]
has_audio = any(s.get("codec_type") == "audio" for s in probe.streams)
if real_video and has_audio:
mode = "av"
elif real_video:
mode = "video"
else:
mode = "audio"
resolution = ""
if real_video:
v = real_video[0]
w, h = v.get("width"), v.get("height")
if w and h:
resolution = f"{w}x{h}"
if "duration" in probe.format:
seconds = _parse_duration(probe.format.get("duration"))
else:
seconds = max(
(_parse_duration(s.get("duration")) for s in probe.streams),
default=0.0,
)
# Round half up (math.floor(x + 0.5)) rather than Python's banker's
# rounding, so a 12.5s clip becomes 13s as the spec intends.
duration_s = max(0, math.floor(seconds + 0.5))
return Tags(mode=mode, duration_s=duration_s, resolution=resolution)