docs: human-experience-filter design spec
Single-viewer four-wall projection installation. Coordinate model (content mode + brain plane + mood plane), nearest-match selection from a tagged JSONL catalog, Arduino-controller/Pi-player hardware split, procedural side walls, public-domain sourcing strategy, lean-to-dense sizing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Ben Stull
@@ -0,0 +1,248 @@
|
||||
# Human Experience Filter — Design Spec
|
||||
|
||||
**Date:** 2026-06-04
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
|
||||
---
|
||||
|
||||
## 1. What this is
|
||||
|
||||
A single-viewer immersive art installation. One person sits in a chair at the
|
||||
center of a small room with four walls, each fronted by a projector. A DJ-style
|
||||
control panel lets them "tune" their experience along a small set of felt axes;
|
||||
the system finds the public-domain media nearest that tuning and plays it. The
|
||||
piece is an *experience filter*: the viewer dials in how they want to feel and
|
||||
think, and the room answers with found human artifacts that match.
|
||||
|
||||
Design constraints that shaped everything below:
|
||||
|
||||
- **One viewer, one room, all local.** No networking, no streaming, no
|
||||
multi-user, no session recording.
|
||||
- **Made of real found media.** The library is curated public-domain / openly
|
||||
licensed work, not algorithmically generated filler. Curation is the artwork.
|
||||
- **Built to grow.** Launch lean; the data model and tooling are designed so the
|
||||
library can expand over time without rework.
|
||||
|
||||
---
|
||||
|
||||
## 2. Coordinate model
|
||||
|
||||
The control panel defines a point in a 5-control experience-space.
|
||||
|
||||
| Control | Type | Range | Meaning |
|
||||
|---------------|----------------|---------------------------|-------------------------------------------|
|
||||
| Content mode | 4-way selector | None / Audio / Video / A+V | Which media channels are active |
|
||||
| Left | knob | 0–4 | How analytical / verbal / structured |
|
||||
| Right | knob | 0–4 | How artistic / emotional / abstract |
|
||||
| Dark | knob | 0–4 | How somber / heavy / ominous |
|
||||
| Light | knob | 0–4 | How uplifting / serene / joyful |
|
||||
|
||||
Two **independent planes**, each formed by two knobs that are *not* opposite ends
|
||||
of one slider:
|
||||
|
||||
- **Brain plane** = Left × Right. Both high → "whole brain." Both low → "quiet
|
||||
mind." One-sided → pure analytical or pure artistic.
|
||||
- **Mood plane** = Dark × Light. Both high → "bittersweet." Both low → "neutral."
|
||||
One-sided → pure somber or pure uplifting.
|
||||
|
||||
Each plane is a 5×5 grid (25 cells). Combined space is 25 × 25 = 625 knob
|
||||
configurations per content mode.
|
||||
|
||||
**Axis semantics (the tagging rubric):**
|
||||
|
||||
- **Left (analytical/verbal/structured):** narration, language, logic, sequence,
|
||||
instruction, explanation, documentary, spoken word.
|
||||
- **Right (artistic/emotional/abstract):** music, abstract visuals, dance,
|
||||
nature, awe, non-verbal, dreamlike, intuitive.
|
||||
- **Dark (somber/heavy):** ominous, melancholic, night, decay, minor-key,
|
||||
storms, noir.
|
||||
- **Light (uplifting/serene):** joyful, hopeful, bright, sunrise, gardens,
|
||||
major-key, calm.
|
||||
|
||||
**None** is the void / rest state: walls fall to black, audio goes silent.
|
||||
|
||||
---
|
||||
|
||||
## 3. Selection algorithm
|
||||
|
||||
Every catalog record carries a coordinate `(left, right, dark, light)` and a
|
||||
`mode`. Selection is **nearest-match**, not exact-cell lookup, so a sparse
|
||||
library still resolves meaningfully at every knob position.
|
||||
|
||||
On a knob change:
|
||||
|
||||
1. **Filter by mode.** Restrict to records whose `mode` matches the selector.
|
||||
In `A+V` mode the pool may also fall back to `audio` and `video` records when
|
||||
`av` records near the point are thin.
|
||||
2. **Distance.** Compute weighted Euclidean distance from the knob coordinate to
|
||||
each candidate's coordinate. Weights let the brain plane and mood plane be
|
||||
tuned independently later (default: equal weight).
|
||||
3. **Pool + pick.** Take the N nearest (default N = 3–5) and choose one,
|
||||
shuffling within the pool so the same corner doesn't always yield the
|
||||
identical piece.
|
||||
4. **Play.** Crossfade from the current piece. The chosen 5–15 min segment loops
|
||||
until the knobs move again.
|
||||
|
||||
`None` short-circuits the algorithm: fade to black + silence.
|
||||
|
||||
---
|
||||
|
||||
## 4. Content catalog (the spine of the repo)
|
||||
|
||||
A version-controlled catalog with **one record per piece**. The media files
|
||||
themselves live on the player's drive (too large for git); the repo holds the
|
||||
**metadata + pointers**. Format: **JSON Lines** (append-friendly, diff-friendly).
|
||||
|
||||
Record schema:
|
||||
|
||||
```
|
||||
id stable unique id
|
||||
title human title
|
||||
source_url where it came from
|
||||
source_archive e.g. internet_archive | musopen | librivox | nasa | freesound
|
||||
license public_domain | cc0 | cc_by | cc_by_nc
|
||||
attribution required text when license demands it
|
||||
mode audio | video | av
|
||||
left 0–4 (curatorial)
|
||||
right 0–4 (curatorial)
|
||||
dark 0–4 (curatorial)
|
||||
light 0–4 (curatorial)
|
||||
duration_s segment length in seconds
|
||||
file_path path on the player drive
|
||||
resolution e.g. 1920x1080 (video only)
|
||||
dominant_color hex, computed from the media (video only)
|
||||
review_status proposed | approved
|
||||
rationale one-line justification for the proposed coordinates
|
||||
reviewed_at timestamp when the human approved (null while proposed)
|
||||
notes free text
|
||||
```
|
||||
|
||||
This catalog is what grows from lean → dense, and what *both* the player and the
|
||||
sourcing/review tools read. "How far from done" is simply the count of records
|
||||
still in `review_status: proposed`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Division of labor — tagging
|
||||
|
||||
Tagging decomposes into mechanical fields and one curatorial field group.
|
||||
|
||||
- **Mechanical / automatic** (tooling sets these): `mode` (from whether the file
|
||||
has meaningful audio/video), `license` + `attribution` + `source_*` (from
|
||||
origin), `dominant_color` (computed), `duration_s`, `resolution`.
|
||||
- **Curatorial** (`left`, `right`, `dark`, `light`): the genuine artistic act.
|
||||
|
||||
**Workflow — "I draft everything, you review every piece":**
|
||||
|
||||
1. **Discover & ingest** (tooling + assistant): pull candidates from the
|
||||
public-domain pools, populate all mechanical fields, write records as
|
||||
`review_status: proposed` with **assistant-proposed coordinates** and a
|
||||
one-line `rationale` each.
|
||||
2. **Review** (human, every piece): a review tool walks unreviewed records one
|
||||
at a time — showing proposed coordinates, the rationale, and a preview frame —
|
||||
and the human accepts or types a correction. Acceptance flips the record to
|
||||
`approved` and stamps `reviewed_at`.
|
||||
|
||||
Every coordinate is human-blessed before it can be selected in the room
|
||||
(optionally the player can be restricted to `approved` records only).
|
||||
|
||||
---
|
||||
|
||||
## 6. Hardware
|
||||
|
||||
The microcontroller and the media player are **separate roles** — an Arduino
|
||||
cannot decode HD video or drive projectors.
|
||||
|
||||
- **Arduino — control panel only.** Reads the four knobs + the 4-way selector and
|
||||
sends the five values to the player over USB serial. This is what Arduinos are
|
||||
good at.
|
||||
- **Raspberry Pi 5 (or small mini-PC) — brain + player.** Holds the catalog,
|
||||
reads the Arduino's serial stream, runs the selection algorithm, and plays
|
||||
media. **The hard drive plugs into the Pi**, not the Arduino.
|
||||
- **Projectors.** The Pi drives the **primary** wall (the wall the viewer faces)
|
||||
with real content. The three **side walls** show procedural ambient (§7),
|
||||
driven from the same Pi's additional outputs or a second cheap Pi.
|
||||
|
||||
Parts delta vs. the original sketch: add a ~$80 Pi; everything else
|
||||
(Arduino, knobs, hard drive, projectors) stays.
|
||||
|
||||
---
|
||||
|
||||
## 7. Procedural side walls
|
||||
|
||||
The three non-primary walls render a slow gradient / color wash:
|
||||
|
||||
- **Hue** comes from the playing piece's `dominant_color`.
|
||||
- **Brightness** tracks the Dark / Light knobs.
|
||||
|
||||
This costs near-zero storage and makes the mood axis physically felt in
|
||||
peripheral vision. In `None` mode the side walls also fade to black.
|
||||
|
||||
---
|
||||
|
||||
## 8. Sourcing strategy (lean now, dense later)
|
||||
|
||||
Pools, prioritized by license cleanliness, mapped to the axes:
|
||||
|
||||
- **Left-brain (verbal/analytical):** LibriVox (readings), Prelinger / Internet
|
||||
Archive educational & industrial films, public-domain lectures, instructional
|
||||
animation.
|
||||
- **Right-brain (abstract/artistic):** Musopen (public-domain classical), NASA
|
||||
imagery & space footage (public domain), early/silent cinema dream sequences,
|
||||
abstract animation, nature.
|
||||
- **Dark mood:** storms, night, noir, minor-key works, decay, somber readings.
|
||||
- **Light mood:** sunrise/gardens, major-key works, bright nature, uplifting
|
||||
verse.
|
||||
- **Audio pools:** Musopen, Free Music Archive (CC), Internet Archive audio,
|
||||
Freesound (CC0 field recordings).
|
||||
|
||||
**License stance:** prefer Public Domain / CC0; accept CC-BY (attribution
|
||||
recorded in the catalog); CC-BY-NC acceptable only while the installation stays
|
||||
non-commercial. Every record stores its license + source so each piece is
|
||||
defensible.
|
||||
|
||||
---
|
||||
|
||||
## 9. Sizing
|
||||
|
||||
- **Pieces:** lean launch ~**120–200** tagged pieces; design supports growth to
|
||||
~**800**.
|
||||
- **Per piece:** medium segment ≈ 10 min. Primary-wall video at 1080p ≈
|
||||
0.6–0.8 GB (~3 GB at 4K). Audio ≈ 20–100 MB.
|
||||
- **Drive:** **1 TB** SSD for a 1080p library, **2 TB** for 4K, both with
|
||||
comfortable headroom. The drive is not the constraint; curation labor is.
|
||||
|
||||
---
|
||||
|
||||
## 10. Repo layout
|
||||
|
||||
```
|
||||
catalog/ the tagged content catalog (JSONL)
|
||||
player/ selection algorithm + media player (runs on the Pi)
|
||||
firmware/ Arduino sketch (reads knobs/selector → USB serial)
|
||||
sidewalls/ procedural ambient renderer
|
||||
tools/ sourcing/ingest, dominant-color, license checker, review tool
|
||||
docs/ this spec + operator/build guide
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Explicitly out of scope (YAGNI)
|
||||
|
||||
- Networking / streaming — everything runs locally on the Pi.
|
||||
- Multi-user / multi-viewer.
|
||||
- Session recording or analytics.
|
||||
- Automatic ML coordinate tagging — coordinates are a human curatorial act by
|
||||
design.
|
||||
|
||||
---
|
||||
|
||||
## 12. Open implementation questions (for the plan, not blockers)
|
||||
|
||||
- Player stack on the Pi (e.g. mpv/ffmpeg-based vs. a custom renderer) and how
|
||||
crossfades are handled.
|
||||
- Exact serial protocol/framing between Arduino and Pi.
|
||||
- Whether side walls run on the primary Pi's extra outputs or a second Pi.
|
||||
- Ingest tooling language and where the per-archive downloaders live.
|
||||
- Whether the player hard-restricts to `approved` records.
|
||||
Reference in New Issue
Block a user