From c5952c6cfdaeb969cd9395c22c81c2095ac84a31 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 17:27:19 -0700 Subject: [PATCH] docs(design): machine-altered-perception thesis revision MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Revise the Human Experience Filter design: the piece becomes about how humans interact with machines and how that reshapes the nervous system. Neutral public-domain nature footage in, machine-altered perception out — soothing or disturbing, good and bad. Supersedes the 2026-06-04 design's thesis (§1), selection model (§3 — now "transform from neutral", not catalog lookup), tagging division (§5), and sourcing (§8); expands the control panel and hardware (§2/§6). Coordinate axes and the single-pano projector are preserved. Captures: the per-axis alteration mapping (Left=analytical overlay, Right=generative v2v dissolve, Dark/Light=color grade), the substrate-vs- overlay composition rule, the Light=yellow/white → neutral grey → Dark=blue/black mood ramp, the 6-way content dial + volume/brightness, the tactile/braille/LED/read-aloud wooden panel, and the runtime-overlay architecture that makes multilingual labels nearly free. Per docs/ROADMAP.md (reshapes sub-projects 2/3/4 + adds an offline v2v authoring pipeline). Discovery session 0005. Co-Authored-By: Claude Opus 4.8 --- ...06-05-machine-altered-perception-design.md | 360 ++++++++++++++++++ 1 file changed, 360 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md diff --git a/docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md b/docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md new file mode 100644 index 0000000..c855cdc --- /dev/null +++ b/docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md @@ -0,0 +1,360 @@ +# Human Experience Filter — Machine-Altered Perception (Design Revision) + +**Date:** 2026-06-05 +**Status:** Approved design (pre-implementation) +**Repo:** `human-experience-filter-art` +**Supersedes:** the thesis (§1), selection model (§3), tagging division (§5), and +sourcing strategy (§8) of +[`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md), +and substantially expands its control panel (§2 selector) and hardware (§6). The +coordinate model (§2 axes) and the single-panoramic-projector display are +**preserved**. + +> **Why this revision exists.** The 2026-06-04 design framed the piece as an +> *experience filter over found human artifacts* ("curation is the artwork; not +> algorithmically generated filler"). This revision **changes the artistic +> thesis** to one about **how humans interact with machines and how that interaction +> reshapes their nervous systems** — and in doing so makes machine alteration of +> the imagery the *subject* of the piece rather than a betrayal of it. + +--- + +## 1. What this is (the new thesis) + +A single-viewer immersive installation about **a machine reshaping your +perception — for better and for worse.** + +One person sits in a chair in a small room. A single panoramic projector wraps the +walls they face. The source is **neutral**: real, calm, public-domain nature +footage that sits at the *center* of the experience-space — neither analytical nor +emotional, neither bright nor dark. The viewer turns knobs on a wooden control +panel, and **the machine alters that neutral reality toward whatever they dial +in.** The piece is the felt, bodily experience of a machine bending your inner +state — and crucially it bends **both ways**: it can soothe you or disturb you, +clarify the world or manipulate it, name your feelings or colonize them. + +You don't read that thesis on a wall label. You feel it in your nervous system as +your own hand moves the knobs. + +Design constraints (carried from the original, one relaxed): + +- **One viewer, one room, all-local at runtime.** No networking, no streaming, no + multi-user, no session recording. (All AI alteration happens **offline at + authoring time**; the room plays pre-built media + cheap runtime grading.) +- **Real footage as the substrate.** The base library is real public-domain + nature footage. The machine *alters* it — it does not invent from nothing. This + preserves real-world motion/composition (and some lineage to the original + found-media thesis) while making the alteration the point. *(This relaxes the + original "no generated content" rule: generation/alteration is now the + subject.)* +- **Built to grow.** A small, deliberate base library; the data model and tooling + designed to expand. + +--- + +## 2. What changes vs. 2026-06-04 + +| Area | 2026-06-04 | This revision | +|---|---|---| +| Thesis (§1) | filter over found human artifacts | **machine reshapes perception, good & bad** | +| Content | large found library pre-matched to coordinates | **small NEUTRAL base library, machine-altered to the knob state** | +| Engine (§3) | nearest-match **lookup** in a tagged catalog | **transform from neutral** (knobs = a transformation, not a lookup) | +| Sourcing (§8) | find media that already matches each cell | source **neutral** clips; the machine produces the variants | +| Control panel (§2) | 4-way mode + 4 knobs | **7-way content dial + 4 experience knobs + volume + brightness**, on a tactile wooden panel | +| Display | single pano projector | **unchanged** | +| Coordinate axes (§2) | Left/Right/Dark/Light, two planes | **unchanged** (reinterpreted as transform targets) | + +--- + +## 3. Coordinate model (preserved, reinterpreted) + +The four experience knobs and two planes from the 2026-06-04 §2 are unchanged: + +- **Brain plane** = Left × Right. Left = analytical / verbal / structured. Right = + artistic / emotional / abstract. Both high → "whole brain." +- **Mood plane** = Dark × Light. Dark = somber / heavy. Light = uplifting / serene. + +Each knob is `0–4`. What changes is the *meaning of a knob position*: it is no +longer "find the catalog piece nearest this coordinate" but **"how hard the machine +pushes the neutral base toward this pole."** `(2,2,2,2)` is the neutral origin — +the un-altered base clip. + +--- + +## 4. The alteration engine (replaces the §3 selection algorithm) + +The engine's job changes from **choosing** a clip to **altering** one. Given a +neutral base clip and a knob vector `(left, right, dark, light)`, it applies a +**transformation toward each active pole** and composites the result. + +### 4.1 Per-axis alteration + +| Pole | What the machine does | How | +|---|---|---| +| **Left** (analytical/verbal) | **imposes analytical structure** on the scene — annotation, measurement, labels, grids, tracking boxes, data HUD; the machine *dissecting* the world into data | **overlay** layer composited on top | +| **Right** (artistic/emotional) | **dissolves realism** toward painterly / dreamlike / abstract | **generative video-to-video (v2v)** restyle of the pixels | +| **Dark** (somber/heavy) | drains toward the cool/melancholy/night grade (see §5) | **color grade** (deterministic) | +| **Light** (uplifting/serene) | warms toward the bright/serene grade (see §5) | **color grade** (deterministic) | + +### 4.2 The composition rule + +The four axes are **two kinds of operation** that **layer** rather than blend-and- +cancel: + +- **Substrate transforms** — Right (v2v restyle) + Dark + Light (color grades). + They alter the **pixels** and blend with each other. +- **Overlay** — Left (annotation / labels / HUD). It composites **on top** of + whatever the substrate is. + +So Left and Right are **not** mutually-exclusive opposites; they stack on different +layers. This single rule covers the whole `5×5×5×5` space: + +- **Max Left + Max Right** = maximally dissolved dreamlike substrate **with the + machine's labels on the feelings themselves** — *"awe," "grief," "detected: + longing 0.82,"* tracking boxes around emotional content. This is the "whole + brain" corner (§3): the machine that both **induces** emotion and + **names/quantifies** it — affective computing made visceral. (Good/bad duality: + is being told your own feeling clarifying, or invasive?) +- **Dark + analytical** = a drained, melancholy scene with cold measurement + overlaid. +- **Light + emotional** = a warm, dreamlike wash, unlabeled. + +### 4.3 Where each transform runs + +- **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade + and the Left analytical overlay. These are cheap (LUT/curves + luma key, and + text/graphics compositing) and can move continuously with the knob. +- **Pre-baked offline (the only AI cost):** the Right generative v2v restyle. + Generative v2v cannot run real-time on a Pi, so a small set of restyled variants + is rendered offline and selected/blended at runtime. + +> **Crucial consequence — the Left labels are a RUNTIME OVERLAY, never baked into +> the v2v pixels.** This is what makes multilingual support nearly free (§10). + +--- + +## 5. Mood-axis color grade + +The Dark↔Light knob is a three-stop color ramp whose **center is the identity** — +the raw, ungraded footage, no color transform applied. Turning toward a pole grades +*away* from the raw clip. Hue carries emotional valence; the **negative-space +value** (a luma-keyed grade — the brightest/darkest regions pushed toward +white/black) carries the literal light/dark reading, so the image *is* lighter or +darker even before the hue registers. + +| Mood knob | Hue | Negative space | Feeling | +|---|---|---|---| +| **Light** | warm **yellow** | → **white** | uplifting, serene, airy | +| **Center** | **none — raw, ungraded video** | unchanged | the un-altered base | +| **Dark** | cool **blue** | → **black** | melancholy, somber, night, cold | + +*(Red/alarm was considered for Dark and set aside: red reads as threat/alarm, not +the intended melancholy. Dark is the cool, mournful blue. The center applies no +grade at all — it is simply the raw clip.)* + +--- + +## 6. Content model — the 7-way content dial + +The 2026-06-04 4-way selector (None/Audio/Video/A+V) expands to **seven positions**, +letting the viewer choose channel **and audio source**: + +| Position | Audio source | Video | +|---|---|---| +| **Off** | — | black | +| **White Noise** | generated white/pink noise (not from catalog) | black | +| **Music** | public-domain classical (Musopen pool) | black | +| **Audio Track** | the nature clip's **own** audio | black | +| **Video** | — | the (altered) nature video | +| **Music + Video** | public-domain classical (Musopen pool) | the (altered) nature video | +| **Audio Track + Video** | the nature clip's own audio | the (altered) nature video | + +Notes: + +- "Music" (classical) and "Audio Track" (the clip's own field audio) are now + **distinct sources**; "White Noise" is **generated** at runtime, not a catalog + record. This touches the catalog/`mode` model (the 2026-06-04 §4 schema), which + must represent audio *source* in addition to channel. +- **Off** is the void/rest state: black walls, silence. +- Audio alteration (steering the *sound* by knob, parallel to the video) is **out + of scope for this revision** — the experience knobs alter video; audio is chosen + by source only. (Future work.) + +--- + +## 7. Intensity controls + +Two knobs, orthogonal to the four experience axes, exist so a viewer can take the +experience at **whatever intensity they want**: + +- **Volume** — audio output level. +- **Brightness** — projector output luminance. + +These are pure output levels, distinct from the Dark/Light *mood grade* (which +changes hue and tone, not raw output). Brightness is "how much light hits the +wall"; Light is "how the world is colored." + +--- + +## 8. Content sourcing — neutral base library + +The library is now **small and neutral**. Instead of hunting for footage that +matches 625 coordinate cells, source a deliberate set of **neutral, calm, +real public-domain nature clips** (centered on all four axes) and let the machine +produce every variant. + +- **Reuse sub-project 2** (the built ingest/tagging/review tooling) to pull + neutral public-domain nature footage and populate mechanical fields. +- **License stance** unchanged: prefer Public Domain / CC0; record license + + source per clip. Altered/generated variants record the model and that they are + machine-derived. +- **Size:** dozens of neutral base clips (not hundreds), each yielding many + altered variants. Short, seamless loops (≈20–60 s) suit a looping installation + and are the v2v models' native sweet spot. + +--- + +## 9. Economics (why this is feasible) + +At the original scale (120–800 pieces × ~10-min segments) AI involvement was +~$15k–45k+ and not viable. Under this design it drops to **a few hundred to a few +thousand dollars**, because: + +- The library is small (generate-to-order, not find-to-cover). +- Pieces are short seamless loops (cheaper **and** higher-quality for the models). +- Only the **Right** transform uses paid generative v2v; Dark/Light/Left are free + deterministic runtime operations. + +Indicative offline v2v build (2026 rates — Kling 3.0 / Sora 2 Standard ≈ $0.10/s, +Veo 3.1 Lite / Runway Turbo ≈ $0.05/s; ×~4 retry yield): **~$300–3k** for a small +base library × a handful of restyle variants. A flat authoring subscription +(Runway $28–76/mo, Kling from ~$6/mo) may be cheaper than per-second API for a +one-time build. + +--- + +## 10. Accessibility, i18n, and the translation-cost finding + +The piece is operable **blind, in the dark, in your language**, via four redundant +channels on the control panel: **touch** (engraved symbol shape), **low-light +color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local* +speaker, separate from the main audio system). + +Because "the human experience from a communication perspective is not universal" is +itself on-thesis, the piece supports **many languages**. The cost of that hinges on +one architectural decision: + +- **❌ Labels baked into the v2v render** → re-render every label-bearing variant + per language → video cost **× number of languages** (≈ $9k raw / $27–45k with + retries for ~50 languages). A budget landmine. +- **✅ Labels as a runtime overlay** (which §4.2 already requires) → the v2v + substrate is rendered **once**, language-agnostic; a new language is a translated + string table + font + TTS voice. **Marginal video cost per language ≈ $0.** + +So multilingual support costs only cheap text work: machine-translating a small +label/script vocabulary into ~100 languages is a few dollars; optional human review +of a curated tier is ~$0.10/word (~$50–200/language); TTS for the read-aloud button +is pennies per language. **Net: supporting *every* language is essentially free on +the expensive part, provided labels stay a runtime overlay.** + +--- + +## 11. The physical control panel + +A **wood** panel. Each control has a **tactile engraved symbol** below it — +distinguishable by fingertip in the dark, and shaped to evoke the control's +meaning — with **braille** beneath, a **dim color-coded LED** backlighting the +engraving, and a **read-aloud button** that speaks the control's function on a +small local speaker. + +Layout principle: the **four experience knobs are two opposing pairs**, with the +opposition encoded *in the touch* (angular vs flowing, heavy-down vs radiant-up). +The **utility controls** use a more conventional AV shape-language so they are +never confused with the experience four. + +| Control | Engraved symbol | Touch | LED | +|---|---|---|---| +| **Left** (analytical) | parallel straight ridges / grid `≡` `▦` | flat, hard, orthogonal edges | — | +| **Right** (emotional) | spiral / flowing wave `∿` | one continuous curve, no corners | — | +| **Dark** (somber) | crescent / downward wedge `☾` `▼` | smooth solid mass, sinking | **blue** | +| **Light** (serene) | radial sunburst `☀` | center with radial spikes | **yellow / white** | +| **Content dial** (7) | per-position micro-icons: Off `○` · White-Noise `∴` · Music `♪` · Audio-Track `)))` · Video `▷` · Music+Video `♪▷` · A+V `▷)))` | detented clicks | neutral | +| **Volume** | growing wedge `◁` | a slope that thickens | neutral | +| **Brightness** | half-disc contrast `◐` | a circle, half raised | neutral | + +Design notes: + +- **Light vs Brightness** is the one collision trap (both are "bright"). Light = + radial sunburst (a *mood*, glows the hue it imparts); Brightness = half-disc + contrast (an *output level*) — distinct to finger and eye. +- **LEDs equal the hue the knob imparts:** Dark glows blue, Light glows + yellow/white, the mood center is neutral — tying the panel to the projection. +- **Negative-space symbols:** symbols differ by gross **edge profile** (curve vs + corner vs radial vs slope), not fine detail, so they read under a fingertip. + +This substantially fleshes out **sub-project 4 (Arduino firmware / control panel)**, +previously only sketched: 7 controls + a read-aloud button + LEDs + braille, with +the Arduino reading positions and the read-aloud/LED behavior coordinated with the +Pi. + +--- + +## 12. Hardware (revises §6) + +Unchanged from the 2026-06-04 single-pano revision except for the richer panel: + +- **Arduino** — reads the 7-way content dial, 4 experience knobs, volume, + brightness, and the read-aloud button; drives the panel LEDs; sends values to the + Pi over USB serial. (Bigger than the original 5-value panel.) +- **Raspberry Pi 5 (or mini-PC)** — holds the base library + altered variants, + runs the alteration engine (runtime grading + overlay compositing + variant + selection), drives the single panoramic projector, plays audio, and serves the + read-aloud clips to the local panel speaker. +- **Single panoramic projector** spanning the three walls — **unchanged**. +- **Small local speaker** on/near the panel for the read-aloud button (separate + from the main audio). + +--- + +## 13. Roadmap impact + +- **Sub-project 2 (ingest/tagging)** — reused to source the **neutral** base + clips; coordinate *drafting* matters less (the base is neutral by selection), but + mechanical tagging + license capture still apply; add a flag for "neutral base" + vs "altered variant," and model audio *source*. +- **Sub-project 3 (player)** — grows an **alteration engine**: runtime color grade + (luma-keyed), runtime analytical-overlay compositor (multilingual, from string + tables), generated white noise, and selection/blending of pre-baked v2v variants. +- **Sub-project 4 (firmware/panel)** — grows from a 5-value panel to the full + tactile/accessible panel in §11. +- **New offline pipeline** — author neutral→restyled v2v variants (the only paid + AI step) and the label/string tables + TTS per language. + +--- + +## 14. Open questions (for the plan, not blockers) + +- **Pano resolution strategy** — v2v models output ~16:9 720p–1080p; the three-wall + pano wants ultra-wide/high-res. Generate-at-max + upscale, native-ultrawide + tooling, or accept the look? +- **Negative-space luma key** — exact curve/mask for "brights→white, darks→black" + on arbitrary nature footage. +- **Transform composition order** — order of grade vs v2v substrate when several + poles are active; how variant blending interpolates between pre-baked restyle + levels. +- **Neutral base count** — how many base clips for a satisfying launch. +- **Language tier** — which languages get human-reviewed translation vs + machine-only; default UI language and switching. +- **Runtime grading performance** — confirm the Pi can do luma-keyed grade + + overlay compositing at projector resolution/framerate (GPU shaders via mpv/ffmpeg). +- **`approved`-only enforcement** in the player (carried from the original). + +--- + +## 15. Out of scope (YAGNI) + +- Networking / streaming at runtime — all alteration is offline; the room is local. +- Runtime/on-demand generation (breaks all-local; slow; unbounded cost). +- Audio alteration by knob (video-only this revision). +- Multi-user / multi-viewer; session recording or analytics. +- Automatic ML coordinate tagging — base clips are chosen neutral by hand. -- 2.39.5