human-experience-filter-art/docs/ROADMAP.md

Human Experience Filter — Roadmap

The installation is built as five independent sub-projects, each producing working, testable software on its own. Each gets its own spec → plan → implement cycle. This file is the index and dependency map; it is the source of truth for "what's left."

Design reference: specs/2026-06-04-human-experience-filter-design.md.

---

Status at a glance

#	Sub-project	Status	Unblocks
1	Catalog & Selection Core	✅ Done	everything
2	Ingest & Tagging / Review tools	⏳ Next	a real library
3	Player Runtime (Pi)	◻ Not started	the room runs
4	Arduino Firmware (control panel)	◻ Not started	real knobs
5	Procedural Side Walls	◻ Not started	full immersion

Dependency order: 1 → 2, and 1 → 3. Sub-projects 3 and 4 share a serial protocol contract and can then proceed in parallel. 5 depends on 3.

        ┌─────────────────────────┐
        │ 1. Catalog & Selection  │ ✅
        └───────────┬─────────────┘
                    │
        ┌───────────┴───────────┐
        ▼                       ▼
┌───────────────┐      ┌─────────────────┐
│ 2. Ingest /   │      │ 3. Player        │
│    Review     │─────▶│    Runtime (Pi)  │
└───────────────┘ feeds└───┬─────────┬────┘
   (fills catalog)         │         │
                  serial   ▼         ▼ mood+color
              ┌────────────────┐ ┌──────────────────┐
              │ 4. Firmware    │ │ 5. Side Walls    │
              └────────────────┘ └──────────────────┘

---

1. Catalog & Selection Core ✅

Done — merged to main, 37 tests. The dependency root every other sub-project imports.

• hef/catalog.py — Record model, validation, JSONL load/save/append.
• hef/selection.py — weighted-Euclidean distance, mode filtering with A+V fallback, nearest-match select().

Plan: 2026-06-04-catalog-and-selection-core.md.

---

2. Ingest & Tagging / Review tools ⏳ (next)

Goal: turn the manual catalog into the assisted draft-then-review flow from the spec, and actually populate the library from public-domain sources. This is where the content hunt lives.

Delivers (tools/):

• Ingest — per-archive fetchers (Internet Archive / Prelinger, Musopen, LibriVox, NASA, Free Music Archive, Freesound) that download a candidate and write a catalog record.
• Mechanical tagging — auto-fill mode (probe streams with ffprobe), license / attribution / source_* (from origin), duration_s, resolution, and dominant_color (computed from the video).
• Coordinate drafting — propose left/right/dark/light with a one-line rationale, written as review_status: proposed.
• Review CLI — walk unreviewed records one at a time (show proposed coordinates + rationale + a preview frame), accept or correct, flip to approved and stamp reviewed_at.

Spec: 2026-06-04-ingest-tagging-review-tools.md.

Depends on: sub-project 1 (hef.catalog). Done when: you can run ingest against a source, get proposed records, review them to approved, and load_catalog validates the result — covered by tests on the mechanical-tagging and review state transitions. External deps: ffmpeg/ffprobe, an image lib for dominant color, per-archive download access. Open decisions: which archives ship first; download/caching layout for media files on the drive.

---

3. Player Runtime (Pi) ◻

Goal: the thing that makes the room run — read the controls, pick media, play it on the primary wall.

Delivers (player/):

• Read the five control values from the Arduino over USB serial.
• Call hef.selection.select() on each change; loop the chosen 5–15 min segment until the knobs move; crossfade on change.
• None mode → fade to black + silence.
• Drive the primary projector; emit current dominant_color + dark/light to the side-wall renderer (sub-project 5).

Depends on: sub-project 1 (select); a populated catalog from sub-project 2 to be meaningful, but can be developed against a hand-authored catalog and a keyboard/serial stand-in before firmware exists. Done when: given a catalog and a stream of control values, it plays the correct segment, loops, crossfades, and goes dark on None — testable with the serial input mocked. Hardware: Raspberry Pi 5 (holds the drive + catalog). Open decisions: player stack (mpv via IPC vs. ffmpeg vs. custom); whether the player hard-restricts to approved records.

---

4. Arduino Firmware (control panel) ◻

Goal: the physical panel — turn knob/selector positions into serial messages.

Delivers (firmware/):

• Read four knobs (left, right, dark, light) and the 4-way content-mode selector.
• Quantize/debounce analog inputs to the 0..4 integer scale.
• Send the five values to the Pi over USB serial in a defined framing.

Depends on: only the serial protocol contract shared with sub-project 3 — agree that first, then 3 and 4 proceed in parallel. Done when: turning a knob produces the expected serial message (verifiable on a serial monitor) and the Pi player reacts. Hardware: Arduino + pots/encoders + a 4-position selector. Open decisions: pots vs. detented encoders; exact serial framing.

---

5. Procedural Side Walls ◻

Goal: make the mood axis physically felt in peripheral vision.

Delivers (sidewalls/):

• Render a slow color wash on the three non-primary walls: hue from the playing piece's dominant_color, brightness from the dark/light knobs.
• Fade to black in None mode.

Depends on: sub-project 3 emitting current dominant_color + mood state. Done when: given a color + mood input stream, the wash renders and tracks changes — testable headless by snapshotting rendered frames. Open decisions: runs on the primary Pi's extra outputs vs. a second Pi.

---

Cross-cutting decisions to settle as we go

Carried from the design spec's open-questions list — none block sub-project 2:

• Serial protocol framing between Arduino and Pi (needed before 3 ⇄ 4).
• Player stack on the Pi (mpv / ffmpeg / custom).
• Side walls on the primary Pi vs. a second Pi.
• approved-only enforcement in the player.
• Media storage layout on the drive (how file_path values are rooted).