commit 46e5d8e4751d314f6ff81ddfbe35b1bf70c2e2d1 Author: Ben Stull Date: Thu Jun 4 01:22:01 2026 -0700 docs: human-experience-filter design spec Single-viewer four-wall projection installation. Coordinate model (content mode + brain plane + mood plane), nearest-match selection from a tagged JSONL catalog, Arduino-controller/Pi-player hardware split, procedural side walls, public-domain sourcing strategy, lean-to-dense sizing. Co-Authored-By: Claude Opus 4.8 diff --git a/docs/superpowers/specs/2026-06-04-human-experience-filter-design.md b/docs/superpowers/specs/2026-06-04-human-experience-filter-design.md new file mode 100644 index 0000000..7391026 --- /dev/null +++ b/docs/superpowers/specs/2026-06-04-human-experience-filter-design.md @@ -0,0 +1,248 @@ +# Human Experience Filter — Design Spec + +**Date:** 2026-06-04 +**Status:** Approved design (pre-implementation) +**Repo:** `human-experience-filter-art` + +--- + +## 1. What this is + +A single-viewer immersive art installation. One person sits in a chair at the +center of a small room with four walls, each fronted by a projector. A DJ-style +control panel lets them "tune" their experience along a small set of felt axes; +the system finds the public-domain media nearest that tuning and plays it. The +piece is an *experience filter*: the viewer dials in how they want to feel and +think, and the room answers with found human artifacts that match. + +Design constraints that shaped everything below: + +- **One viewer, one room, all local.** No networking, no streaming, no + multi-user, no session recording. +- **Made of real found media.** The library is curated public-domain / openly + licensed work, not algorithmically generated filler. Curation is the artwork. +- **Built to grow.** Launch lean; the data model and tooling are designed so the + library can expand over time without rework. + +--- + +## 2. Coordinate model + +The control panel defines a point in a 5-control experience-space. + +| Control | Type | Range | Meaning | +|---------------|----------------|---------------------------|-------------------------------------------| +| Content mode | 4-way selector | None / Audio / Video / A+V | Which media channels are active | +| Left | knob | 0–4 | How analytical / verbal / structured | +| Right | knob | 0–4 | How artistic / emotional / abstract | +| Dark | knob | 0–4 | How somber / heavy / ominous | +| Light | knob | 0–4 | How uplifting / serene / joyful | + +Two **independent planes**, each formed by two knobs that are *not* opposite ends +of one slider: + +- **Brain plane** = Left × Right. Both high → "whole brain." Both low → "quiet + mind." One-sided → pure analytical or pure artistic. +- **Mood plane** = Dark × Light. Both high → "bittersweet." Both low → "neutral." + One-sided → pure somber or pure uplifting. + +Each plane is a 5×5 grid (25 cells). Combined space is 25 × 25 = 625 knob +configurations per content mode. + +**Axis semantics (the tagging rubric):** + +- **Left (analytical/verbal/structured):** narration, language, logic, sequence, + instruction, explanation, documentary, spoken word. +- **Right (artistic/emotional/abstract):** music, abstract visuals, dance, + nature, awe, non-verbal, dreamlike, intuitive. +- **Dark (somber/heavy):** ominous, melancholic, night, decay, minor-key, + storms, noir. +- **Light (uplifting/serene):** joyful, hopeful, bright, sunrise, gardens, + major-key, calm. + +**None** is the void / rest state: walls fall to black, audio goes silent. + +--- + +## 3. Selection algorithm + +Every catalog record carries a coordinate `(left, right, dark, light)` and a +`mode`. Selection is **nearest-match**, not exact-cell lookup, so a sparse +library still resolves meaningfully at every knob position. + +On a knob change: + +1. **Filter by mode.** Restrict to records whose `mode` matches the selector. + In `A+V` mode the pool may also fall back to `audio` and `video` records when + `av` records near the point are thin. +2. **Distance.** Compute weighted Euclidean distance from the knob coordinate to + each candidate's coordinate. Weights let the brain plane and mood plane be + tuned independently later (default: equal weight). +3. **Pool + pick.** Take the N nearest (default N = 3–5) and choose one, + shuffling within the pool so the same corner doesn't always yield the + identical piece. +4. **Play.** Crossfade from the current piece. The chosen 5–15 min segment loops + until the knobs move again. + +`None` short-circuits the algorithm: fade to black + silence. + +--- + +## 4. Content catalog (the spine of the repo) + +A version-controlled catalog with **one record per piece**. The media files +themselves live on the player's drive (too large for git); the repo holds the +**metadata + pointers**. Format: **JSON Lines** (append-friendly, diff-friendly). + +Record schema: + +``` +id stable unique id +title human title +source_url where it came from +source_archive e.g. internet_archive | musopen | librivox | nasa | freesound +license public_domain | cc0 | cc_by | cc_by_nc +attribution required text when license demands it +mode audio | video | av +left 0–4 (curatorial) +right 0–4 (curatorial) +dark 0–4 (curatorial) +light 0–4 (curatorial) +duration_s segment length in seconds +file_path path on the player drive +resolution e.g. 1920x1080 (video only) +dominant_color hex, computed from the media (video only) +review_status proposed | approved +rationale one-line justification for the proposed coordinates +reviewed_at timestamp when the human approved (null while proposed) +notes free text +``` + +This catalog is what grows from lean → dense, and what *both* the player and the +sourcing/review tools read. "How far from done" is simply the count of records +still in `review_status: proposed`. + +--- + +## 5. Division of labor — tagging + +Tagging decomposes into mechanical fields and one curatorial field group. + +- **Mechanical / automatic** (tooling sets these): `mode` (from whether the file + has meaningful audio/video), `license` + `attribution` + `source_*` (from + origin), `dominant_color` (computed), `duration_s`, `resolution`. +- **Curatorial** (`left`, `right`, `dark`, `light`): the genuine artistic act. + +**Workflow — "I draft everything, you review every piece":** + +1. **Discover & ingest** (tooling + assistant): pull candidates from the + public-domain pools, populate all mechanical fields, write records as + `review_status: proposed` with **assistant-proposed coordinates** and a + one-line `rationale` each. +2. **Review** (human, every piece): a review tool walks unreviewed records one + at a time — showing proposed coordinates, the rationale, and a preview frame — + and the human accepts or types a correction. Acceptance flips the record to + `approved` and stamps `reviewed_at`. + +Every coordinate is human-blessed before it can be selected in the room +(optionally the player can be restricted to `approved` records only). + +--- + +## 6. Hardware + +The microcontroller and the media player are **separate roles** — an Arduino +cannot decode HD video or drive projectors. + +- **Arduino — control panel only.** Reads the four knobs + the 4-way selector and + sends the five values to the player over USB serial. This is what Arduinos are + good at. +- **Raspberry Pi 5 (or small mini-PC) — brain + player.** Holds the catalog, + reads the Arduino's serial stream, runs the selection algorithm, and plays + media. **The hard drive plugs into the Pi**, not the Arduino. +- **Projectors.** The Pi drives the **primary** wall (the wall the viewer faces) + with real content. The three **side walls** show procedural ambient (§7), + driven from the same Pi's additional outputs or a second cheap Pi. + +Parts delta vs. the original sketch: add a ~$80 Pi; everything else +(Arduino, knobs, hard drive, projectors) stays. + +--- + +## 7. Procedural side walls + +The three non-primary walls render a slow gradient / color wash: + +- **Hue** comes from the playing piece's `dominant_color`. +- **Brightness** tracks the Dark / Light knobs. + +This costs near-zero storage and makes the mood axis physically felt in +peripheral vision. In `None` mode the side walls also fade to black. + +--- + +## 8. Sourcing strategy (lean now, dense later) + +Pools, prioritized by license cleanliness, mapped to the axes: + +- **Left-brain (verbal/analytical):** LibriVox (readings), Prelinger / Internet + Archive educational & industrial films, public-domain lectures, instructional + animation. +- **Right-brain (abstract/artistic):** Musopen (public-domain classical), NASA + imagery & space footage (public domain), early/silent cinema dream sequences, + abstract animation, nature. +- **Dark mood:** storms, night, noir, minor-key works, decay, somber readings. +- **Light mood:** sunrise/gardens, major-key works, bright nature, uplifting + verse. +- **Audio pools:** Musopen, Free Music Archive (CC), Internet Archive audio, + Freesound (CC0 field recordings). + +**License stance:** prefer Public Domain / CC0; accept CC-BY (attribution +recorded in the catalog); CC-BY-NC acceptable only while the installation stays +non-commercial. Every record stores its license + source so each piece is +defensible. + +--- + +## 9. Sizing + +- **Pieces:** lean launch ~**120–200** tagged pieces; design supports growth to + ~**800**. +- **Per piece:** medium segment ≈ 10 min. Primary-wall video at 1080p ≈ + 0.6–0.8 GB (~3 GB at 4K). Audio ≈ 20–100 MB. +- **Drive:** **1 TB** SSD for a 1080p library, **2 TB** for 4K, both with + comfortable headroom. The drive is not the constraint; curation labor is. + +--- + +## 10. Repo layout + +``` +catalog/ the tagged content catalog (JSONL) +player/ selection algorithm + media player (runs on the Pi) +firmware/ Arduino sketch (reads knobs/selector → USB serial) +sidewalls/ procedural ambient renderer +tools/ sourcing/ingest, dominant-color, license checker, review tool +docs/ this spec + operator/build guide +``` + +--- + +## 11. Explicitly out of scope (YAGNI) + +- Networking / streaming — everything runs locally on the Pi. +- Multi-user / multi-viewer. +- Session recording or analytics. +- Automatic ML coordinate tagging — coordinates are a human curatorial act by + design. + +--- + +## 12. Open implementation questions (for the plan, not blockers) + +- Player stack on the Pi (e.g. mpv/ffmpeg-based vs. a custom renderer) and how + crossfades are handled. +- Exact serial protocol/framing between Arduino and Pi. +- Whether side walls run on the primary Pi's extra outputs or a second Pi. +- Ingest tooling language and where the per-archive downloaders live. +- Whether the player hard-restricts to `approved` records.