# Human Experience Filter — Machine-Altered Perception (Design Revision)

**Date:** 2026-06-05
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
**Supersedes:** the thesis (§1), selection model (§3), tagging division (§5), and
sourcing strategy (§8) of
[`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md),
and substantially expands its control panel (§2 selector) and hardware (§6). The
coordinate model (§2 axes) and the single-panoramic-projector display are
**preserved**.

> **Why this revision exists.** The 2026-06-04 design framed the piece as an
> *experience filter over found human artifacts* ("curation is the artwork; not
> algorithmically generated filler"). This revision **changes the artistic
> thesis** to one about **how humans interact with machines and how that interaction
> reshapes their nervous systems** — and in doing so makes machine alteration of
> the imagery the *subject* of the piece rather than a betrayal of it.

---

## 1. What this is (the new thesis)

A single-viewer immersive installation about **a machine reshaping your
perception — for better and for worse.**

One person sits in a chair in a small room. A single panoramic projector wraps the
walls they face. The source is **neutral**: real, calm, public-domain nature
footage that sits at the *center* of the experience-space — neither analytical nor
emotional, neither bright nor dark. The viewer turns knobs on a wooden control
panel, and **the machine alters that neutral reality toward whatever they dial
in.** The piece is the felt, bodily experience of a machine bending your inner
state — and crucially it bends **both ways**: it can soothe you or disturb you,
clarify the world or manipulate it, name your feelings or colonize them.

You don't read that thesis on a wall label. You feel it in your nervous system as
your own hand moves the knobs.

Design constraints (carried from the original, one relaxed):

- **One viewer, one room, all-local at runtime.** No networking, no streaming, no
  multi-user, no session recording. (All AI alteration happens **offline at
  authoring time**; the room plays pre-built media + cheap runtime grading.)
- **Real footage as the substrate.** The base library is real public-domain
  nature footage. The machine *alters* it — it does not invent from nothing. This
  preserves real-world motion/composition (and some lineage to the original
  found-media thesis) while making the alteration the point. *(This relaxes the
  original "no generated content" rule: generation/alteration is now the
  subject.)*
- **Built to grow.** A small, deliberate base library; the data model and tooling
  designed to expand.

---

## 2. What changes vs. 2026-06-04

| Area | 2026-06-04 | This revision |
|---|---|---|
| Thesis (§1) | filter over found human artifacts | **machine reshapes perception, good & bad** |
| Content | large found library pre-matched to coordinates | **small NEUTRAL base library, machine-altered to the knob state** |
| Engine (§3) | nearest-match **lookup** in a tagged catalog | **transform from neutral** (knobs = a transformation, not a lookup) |
| Sourcing (§8) | find media that already matches each cell | source **neutral** clips; the machine produces the variants |
| Control panel (§2) | 4-way mode + 4 knobs | **7-way content dial + 4 experience knobs + volume + brightness**, on a tactile wooden panel |
| Display | single pano projector | **unchanged** |
| Coordinate axes (§2) | Left/Right/Dark/Light, two planes | **unchanged** (reinterpreted as transform targets) |

---

## 3. Coordinate model (preserved, reinterpreted)

The four experience knobs and two planes from the 2026-06-04 §2 are unchanged:

- **Brain plane** = Left × Right. Left = analytical / verbal / structured. Right =
  artistic / emotional / abstract. Both high → "whole brain."
- **Mood plane** = Dark × Light. Dark = somber / heavy. Light = uplifting / serene.

Each knob is `0–4`. What changes is the *meaning of a knob position*: it is no
longer "find the catalog piece nearest this coordinate" but **"how hard the machine
pushes the neutral base toward this pole."** `(2,2,2,2)` is the neutral origin —
the un-altered base clip.

---

## 4. The alteration engine (replaces the §3 selection algorithm)

The engine's job changes from **choosing** a clip to **altering** one. Given a
neutral base clip and a knob vector `(left, right, dark, light)`, it applies a
**transformation toward each active pole** and composites the result.

### 4.1 Per-axis alteration

| Pole | What the machine does | How |
|---|---|---|
| **Left** (analytical/verbal) | **imposes analytical structure** on the scene — annotation, measurement, labels, grids, tracking boxes, data HUD; the machine *dissecting* the world into data | **overlay** layer composited on top |
| **Right** (artistic/emotional) | **dissolves realism** toward painterly / dreamlike / abstract | **generative video-to-video (v2v)** restyle of the pixels |
| **Dark** (somber/heavy) | drains toward the cool/melancholy/night grade (see §5) | **color grade** (deterministic) |
| **Light** (uplifting/serene) | warms toward the bright/serene grade (see §5) | **color grade** (deterministic) |

### 4.2 The composition rule

The four axes are **two kinds of operation** that **layer** rather than blend-and-
cancel:

- **Substrate transforms** — Right (v2v restyle) + Dark + Light (color grades).
  They alter the **pixels** and blend with each other.
- **Overlay** — Left (annotation / labels / HUD). It composites **on top** of
  whatever the substrate is.

So Left and Right are **not** mutually-exclusive opposites; they stack on different
layers. This single rule covers the whole `5×5×5×5` space:

- **Max Left + Max Right** = maximally dissolved dreamlike substrate **with the
  machine's labels on the feelings themselves** — *"awe," "grief," "detected:
  longing 0.82,"* tracking boxes around emotional content. This is the "whole
  brain" corner (§3): the machine that both **induces** emotion and
  **names/quantifies** it — affective computing made visceral. (Good/bad duality:
  is being told your own feeling clarifying, or invasive?)
- **Dark + analytical** = a drained, melancholy scene with cold measurement
  overlaid.
- **Light + emotional** = a warm, dreamlike wash, unlabeled.

### 4.3 Where each transform runs

> **Reconciled (2026-06-07, session 0009):** the Left HUD is a **runtime overlay**
> driven by an authored annotation track + per-language string tables (text shaped
> live); the Right axis selects a **discrete pre-baked** flow-stabilized variant
> (not a continuous blend). See
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
> §1, which supersedes the session-0007 baked-HUD / 5×5-grid proposal.

- **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade
  and the Left analytical overlay. These are cheap (LUT/curves + luma key, and
  text/graphics compositing) and can move continuously with the knob.
- **Pre-baked offline (the only AI cost):** the Right generative v2v restyle.
  Generative v2v cannot run real-time on a Pi, so a small set of restyled variants
  is rendered offline and selected/blended at runtime.

> **Crucial consequence — the Left labels are a RUNTIME OVERLAY, never baked into
> the v2v pixels.** This is what makes multilingual support nearly free (§10).

---

## 5. Mood-axis color grade

The Dark↔Light knob is a three-stop color ramp whose **center is the identity** —
the raw, ungraded footage, no color transform applied. Turning toward a pole grades
*away* from the raw clip. Hue carries emotional valence; the **negative-space
value** (a luma-keyed grade — the brightest/darkest regions pushed toward
white/black) carries the literal light/dark reading, so the image *is* lighter or
darker even before the hue registers.

| Mood knob | Hue | Negative space | Feeling |
|---|---|---|---|
| **Light** | warm **yellow** | → **white** | uplifting, serene, airy |
| **Center** | **none — raw, ungraded video** | unchanged | the un-altered base |
| **Dark** | cool **blue** | → **black** | melancholy, somber, night, cold |

*(Red/alarm was considered for Dark and set aside: red reads as threat/alarm, not
the intended melancholy. Dark is the cool, mournful blue. The center applies no
grade at all — it is simply the raw clip.)*

---

## 6. Content model — the 7-way content dial

The 2026-06-04 4-way selector (None/Audio/Video/A+V) expands to **seven positions**,
letting the viewer choose channel **and audio source**:

| Position | Audio source | Video |
|---|---|---|
| **Off** | — | black |
| **White Noise** | generated white/pink noise (not from catalog) | black |
| **Music** | public-domain classical (Musopen pool) | black |
| **Audio Track** | the nature clip's **own** audio | black |
| **Video** | — | the (altered) nature video |
| **Music + Video** | public-domain classical (Musopen pool) | the (altered) nature video |
| **Audio Track + Video** | the nature clip's own audio | the (altered) nature video |

Notes:

- "Music" (classical) and "Audio Track" (the clip's own field audio) are now
  **distinct sources**; "White Noise" is **generated** at runtime, not a catalog
  record. This touches the catalog/`mode` model (the 2026-06-04 §4 schema), which
  must represent audio *source* in addition to channel.
- **Off** is the void/rest state: black walls, silence.
- Audio alteration (steering the *sound* by knob, parallel to the video) is **out
  of scope for this revision** — the experience knobs alter video; audio is chosen
  by source only. (Future work.)

---

## 7. Intensity controls

Two knobs, orthogonal to the four experience axes, exist so a viewer can take the
experience at **whatever intensity they want**:

- **Volume** — audio output level.
- **Brightness** — projector output luminance.

These are pure output levels, distinct from the Dark/Light *mood grade* (which
changes hue and tone, not raw output). Brightness is "how much light hits the
wall"; Light is "how the world is colored."

---

## 8. Content sourcing — neutral base library

The library is now **small and neutral**. Instead of hunting for footage that
matches 625 coordinate cells, source a deliberate set of **neutral, calm,
real public-domain nature clips** (centered on all four axes) and let the machine
produce every variant.

- **Reuse sub-project 2** (the built ingest/tagging/review tooling) to pull
  neutral public-domain nature footage and populate mechanical fields.
- **License stance** unchanged: prefer Public Domain / CC0; record license +
  source per clip. Altered/generated variants record the model and that they are
  machine-derived.
- **Size:** dozens of neutral base clips (not hundreds), each yielding many
  altered variants. Short, seamless loops (≈20–60 s) suit a looping installation
  and are the v2v models' native sweet spot.

---

## 9. Economics (why this is feasible)

At the original scale (120–800 pieces × ~10-min segments) AI involvement was
~$15k–45k+ and not viable. Under this design it drops to **a few hundred to a few
thousand dollars**, because:

- The library is small (generate-to-order, not find-to-cover).
- Pieces are short seamless loops (cheaper **and** higher-quality for the models).
- Only the **Right** transform uses paid generative v2v; Dark/Light/Left are free
  deterministic runtime operations.

Indicative offline v2v build (2026 rates — Kling 3.0 / Sora 2 Standard ≈ $0.10/s,
Veo 3.1 Lite / Runway Turbo ≈ $0.05/s; ×~4 retry yield): **~$300–3k** for a small
base library × a handful of restyle variants. A flat authoring subscription
(Runway $28–76/mo, Kling from ~$6/mo) may be cheaper than per-second API for a
one-time build.

---

## 10. Accessibility, i18n, and the translation-cost finding

> **Reconciled (2026-06-07, session 0009):** the near-free-i18n path is kept — the
> Left HUD is a runtime overlay (authored annotation track + per-language string
> tables, shaped live via Pango/HarfBuzz on the Pi). The session-0007 baked-HUD
> reversal is **not** adopted. See
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
> §1.

The piece is operable **blind, in the dark, in your language**, via four redundant
channels on the control panel: **touch** (engraved symbol shape), **low-light
color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local*
speaker, separate from the main audio system).

Because "the human experience from a communication perspective is not universal" is
itself on-thesis, the piece supports **many languages**. The cost of that hinges on
one architectural decision:

- **❌ Labels baked into the v2v render** → re-render every label-bearing variant
  per language → video cost **× number of languages** (≈ $9k raw / $27–45k with
  retries for ~50 languages). A budget landmine.
- **✅ Labels as a runtime overlay** (which §4.2 already requires) → the v2v
  substrate is rendered **once**, language-agnostic; a new language is a translated
  string table + font + TTS voice. **Marginal video cost per language ≈ $0.**

So multilingual support costs only cheap text work: machine-translating a small
label/script vocabulary into ~100 languages is a few dollars; optional human review
of a curated tier is ~$0.10/word (~$50–200/language); TTS for the read-aloud button
is pennies per language. **Net: supporting *every* language is essentially free on
the expensive part, provided labels stay a runtime overlay.**

---

## 11. The physical control panel

A **wood** panel. Each control has a **tactile engraved symbol** below it —
distinguishable by fingertip in the dark, and shaped to evoke the control's
meaning — with **braille** beneath, a **dim color-coded LED** backlighting the
engraving, and a **read-aloud button** that speaks the control's function on a
small local speaker.

Layout principle: the **four experience knobs are two opposing pairs**, with the
opposition encoded *in the touch* (angular vs flowing, heavy-down vs radiant-up).
The **utility controls** use a more conventional AV shape-language so they are
never confused with the experience four.

| Control | Engraved symbol | Touch | LED |
|---|---|---|---|
| **Left** (analytical) | parallel straight ridges / grid `≡` `▦` | flat, hard, orthogonal edges | — |
| **Right** (emotional) | spiral / flowing wave `∿` | one continuous curve, no corners | — |
| **Dark** (somber) | crescent / downward wedge `☾` `▼` | smooth solid mass, sinking | **blue** |
| **Light** (serene) | radial sunburst `☀` | center with radial spikes | **yellow / white** |
| **Content dial** (7) | per-position micro-icons: Off `○` · White-Noise `∴` · Music `♪` · Audio-Track `)))` · Video `▷` · Music+Video `♪▷` · A+V `▷)))` | detented clicks | neutral |
| **Volume** | growing wedge `◁` | a slope that thickens | neutral |
| **Brightness** | half-disc contrast `◐` | a circle, half raised | neutral |

Design notes:

- **Light vs Brightness** is the one collision trap (both are "bright"). Light =
  radial sunburst (a *mood*, glows the hue it imparts); Brightness = half-disc
  contrast (an *output level*) — distinct to finger and eye.
- **LEDs equal the hue the knob imparts:** Dark glows blue, Light glows
  yellow/white, the mood center is neutral — tying the panel to the projection.
- **Negative-space symbols:** symbols differ by gross **edge profile** (curve vs
  corner vs radial vs slope), not fine detail, so they read under a fingertip.

This substantially fleshes out **sub-project 4 (Arduino firmware / control panel)**,
previously only sketched: 7 controls + a read-aloud button + LEDs + braille, with
the Arduino reading positions and the read-aloud/LED behavior coordinated with the
Pi.

---

## 12. Hardware (revises §6)

Unchanged from the 2026-06-04 single-pano revision except for the richer panel:

- **Arduino** — reads the 7-way content dial, 4 experience knobs, volume,
  brightness, and the read-aloud button; drives the panel LEDs; sends values to the
  Pi over USB serial. (Bigger than the original 5-value panel.)
- **Raspberry Pi 5 (or mini-PC)** — holds the base library + altered variants,
  runs the alteration engine (runtime grading + overlay compositing + variant
  selection), drives the single panoramic projector, plays audio, and serves the
  read-aloud clips to the local panel speaker.
- **Single panoramic projector** spanning the three walls — **unchanged**.
- **Small local speaker** on/near the panel for the read-aloud button (separate
  from the main audio).

---

## 13. Roadmap impact

- **Sub-project 2 (ingest/tagging)** — reused to source the **neutral** base
  clips; coordinate *drafting* matters less (the base is neutral by selection), but
  mechanical tagging + license capture still apply; add a flag for "neutral base"
  vs "altered variant," and model audio *source*.
- **Sub-project 3 (player)** — grows an **alteration engine**: runtime color grade
  (luma-keyed), runtime analytical-overlay compositor (multilingual, from string
  tables), generated white noise, and selection/blending of pre-baked v2v variants.
- **Sub-project 4 (firmware/panel)** — grows from a 5-value panel to the full
  tactile/accessible panel in §11.
- **New offline pipeline** — author neutral→restyled v2v variants (the only paid
  AI step) and the label/string tables + TTS per language.

---

## 14. Open questions (for the plan, not blockers)

- **Pano resolution strategy** — v2v models output ~16:9 720p–1080p; the three-wall
  pano wants ultra-wide/high-res. Generate-at-max + upscale, native-ultrawide
  tooling, or accept the look?
- **Negative-space luma key** — exact curve/mask for "brights→white, darks→black"
  on arbitrary nature footage.
- **Transform composition order** — order of grade vs v2v substrate when several
  poles are active; how variant blending interpolates between pre-baked restyle
  levels.
- **Neutral base count** — how many base clips for a satisfying launch.
- **Language tier** — which languages get human-reviewed translation vs
  machine-only; default UI language and switching.
- **Runtime grading performance** — confirm the Pi can do luma-keyed grade +
  overlay compositing at projector resolution/framerate (GPU shaders via mpv/ffmpeg).
- **`approved`-only enforcement** in the player (carried from the original).

---

## 15. Out of scope (YAGNI)

- Networking / streaming at runtime — all alteration is offline; the room is local.
- Runtime/on-demand generation (breaks all-local; slow; unbounded cost).
- Audio alteration by knob (video-only this revision).
- Multi-user / multi-viewer; session recording or analytics.
- Automatic ML coordinate tagging — base clips are chosen neutral by hand.
