Compare commits

...

18 Commits

Author SHA1 Message Date
Ben Stull 3ef21fb7f7 docs(spec): simulator alteration preview design (session 0007)
Bring the slice-1 alteration engine into the simulator so the experience
look can be tuned before hardware (operator simulator-first directive).
Python-canonical engine, thin browser renderer; Dark/Light as a live
runtime grade, Left/Right as a 5x5 grid of pre-baked authored variant
clips selected by coordinate.

Revises the machine-altered-perception design SS4.3/SS10: the Left HUD is
now baked into authored variant videos (authorial precision over the HUD)
rather than a runtime overlay, trading away near-free multilingual support.
Retires the simulator's selection-era X-ray view + endpoints.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:18:26 -07:00
Ben Stull 5290785e2a add sessions/0006/SESSION-0006.0-TRANSCRIPT-2026-06-05T17-52--2026-06-05T18-15.md + replace placeholder/variant SESSION-0006.0-TRANSCRIPT-2026-06-05T17-52--INPROGRESS.md 2026-06-05 18:16:33 -07:00
benstull aeeed66f49 Merge pull request 'feat(player): sub-project 3 slice 1 — alteration engine + player core' (#5) from feature/player-alteration-core into main 2026-06-06 01:11:21 +00:00
Ben Stull 5403021e4a docs(roadmap): sub-project 3 alteration-engine + player-core slice shipped 2026-06-05 18:09:54 -07:00
Ben Stull 86b52ab8af build(player): register the player package 2026-06-05 18:08:55 -07:00
Ben Stull 9a13b01a41 feat(player): player state machine — controls stream to transitions 2026-06-05 18:08:34 -07:00
Ben Stull 79bb86c6a2 feat(player): alteration engine — knob vector to RenderPlan (design §4/§5) 2026-06-05 18:07:29 -07:00
Ben Stull 00969e2e44 feat(player): 7-way content-dial resolution (design §6) 2026-06-05 18:06:49 -07:00
Ben Stull 57597be8af feat(player): Controls panel model (serial-contract data shape)
Sub-project 3 slice 1, per ROADMAP.md §3 and design §6/§7.
2026-06-05 18:06:27 -07:00
Ben Stull 70834ae0ad docs(plan): sub-project 3 player alteration core (slice 1)
Pure-logic core per design §4/§5/§6/§7 and ROADMAP.md §3.
2026-06-05 18:05:34 -07:00
Ben Stull e69448922c claim human-experience-filter-art session 0006 (placeholder) + sessions.json entry 2026-06-05 17:55:09 -07:00
Ben Stull d8bb9661d4 add sessions/0005/SESSION-0005.0-TRANSCRIPT-2026-06-05T08-30--2026-06-05T17-48.md + replace placeholder/variant SESSION-0005.0-TRANSCRIPT-2026-06-05T08-30--INPROGRESS.md 2026-06-05 17:51:46 -07:00
benstull 77746e43c6 Merge pull request 'docs(design): machine-altered-perception thesis revision (session 0005)' (#4) from feature/machine-altered-perception-spec into main 2026-06-06 00:49:42 +00:00
Ben Stull c5952c6cfd docs(design): machine-altered-perception thesis revision
Revise the Human Experience Filter design: the piece becomes about how
humans interact with machines and how that reshapes the nervous system.
Neutral public-domain nature footage in, machine-altered perception out —
soothing or disturbing, good and bad.

Supersedes the 2026-06-04 design's thesis (§1), selection model (§3 — now
"transform from neutral", not catalog lookup), tagging division (§5), and
sourcing (§8); expands the control panel and hardware (§2/§6). Coordinate
axes and the single-pano projector are preserved.

Captures: the per-axis alteration mapping (Left=analytical overlay,
Right=generative v2v dissolve, Dark/Light=color grade), the substrate-vs-
overlay composition rule, the Light=yellow/white → neutral grey →
Dark=blue/black mood ramp, the 6-way content dial + volume/brightness, the
tactile/braille/LED/read-aloud wooden panel, and the runtime-overlay
architecture that makes multilingual labels nearly free.

Per docs/ROADMAP.md (reshapes sub-projects 2/3/4 + adds an offline v2v
authoring pipeline). Discovery session 0005.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:30:27 -07:00
Ben Stull 850f67abea claim human-experience-filter-art session 0005 (placeholder) + sessions.json entry 2026-06-05 08:32:48 -07:00
Ben Stull 3510236a2a add sessions/0004/SESSION-0004.0-TRANSCRIPT-2026-06-05T03-32--2026-06-05T03-56.md + replace placeholder/variant SESSION-0004.0-TRANSCRIPT-2026-06-05T03-32--INPROGRESS.md 2026-06-05 03:58:06 -07:00
Ben Stull 05c1f524da update sessions/0004/SESSION-0004.0-TRANSCRIPT-2026-06-05T03-32--INPROGRESS.md 2026-06-05 03:57:32 -07:00
Ben Stull 40dfbfd3db chore: gitignore *.egg-info/ build artifact
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 03:55:36 -07:00
20 changed files with 2519 additions and 40 deletions
+1
View File
@@ -4,3 +4,4 @@ __pycache__/
.venv/
media/
.superpowers/
*.egg-info/
+50 -19
View File
@@ -15,7 +15,7 @@ Design reference: [`specs/2026-06-04-human-experience-filter-design.md`](./super
|---|---------------------------------|---------------|----------|
| 1 | Catalog & Selection Core | ✅ Done | everything |
| 2 | Ingest & Tagging / Review tools | ✅ Done | a real library |
| 3 | Player Runtime (Pi) | ⏳ Next | the room runs |
| 3 | Player Runtime (Pi) | ⏳ In progress | the room runs |
| 4 | Arduino Firmware (control panel)| ◻ Not started | real knobs |
| 5 | ~~Procedural Side Walls~~ | ❌ Dropped | — (superseded) |
@@ -95,29 +95,60 @@ was `validate_catalog` + `index_by_id`.
---
## 3. Player Runtime (Pi) ⏳ (next)
## 3. Player Runtime (Pi) ⏳ (in progress)
**Goal:** the thing that makes the room run — read the controls, pick media, play
it across the single panoramic projector.
**Goal:** the thing that makes the room run — read the controls, **alter** the
neutral base footage toward the knob state, and play it across the single
panoramic projector. Per the machine-altered-perception revision the knobs no
longer *select* a pre-tagged clip; they drive an **alteration engine** (design
§4/§5) over a small neutral base library.
**Delivers (`player/`):**
**Slice 1 — alteration engine + player core (pure logic) ✅ Done.** Merged to
`main` (session 0006); 56 tests. The new `player/` package, with all I/O
(video/audio/serial) behind injected interfaces:
- Read the five control values from the Arduino over USB serial.
- Call `hef.selection.select()` on each change; loop the chosen 515 min segment
until the knobs move; **crossfade** on change.
- `None` mode → fade to black + silence.
- Drive the **single panoramic projector** spanning the three walls with the real
selected video (no primary/side split; no side-wall feed — sub-project 5 dropped).
- `player/controls.py``Controls`, the full panel state (7-way content dial +
4 experience knobs + volume + brightness); the serial-contract data shape
shared with sub-project 4.
- `player/content.py``resolve_content`, the §6 7-way dial → (audio source,
video on/off) table.
- `player/alteration.py``plan_alteration`, knob vector → `RenderPlan`
(mood `ColorGrade` with center=identity §5; Left `AnalyticalOverlay`; Right
`Restyle`), encoding the §4.2 layering rule (substrate vs overlay; Left & Right
stack).
- `player/state.py``Player`, the state machine that diffs successive controls
into transitions (`FADE_TO_BLACK`/`FADE_FROM_BLACK` on video on/off,
`CROSSFADE` on clip/restyle-variant swap, `LIVE_UPDATE` for the continuous
grade/overlay/level ops per §4.3, `NONE` when unchanged).
**Depends on:** sub-project 1 (`select`); a populated catalog from sub-project 2
to be meaningful, but can be developed against a hand-authored catalog and a
**keyboard/serial stand-in** before firmware exists.
**Done when:** given a catalog and a stream of control values, it plays the
correct segment, loops, crossfades, and goes dark on `None` — testable with the
serial input mocked.
**Hardware:** Raspberry Pi 5 (holds the drive + catalog).
Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-player-alteration-core.md).
**Remaining slices (not started):**
- **Runtime renderer** — drive the single panoramic projector via mpv/ffmpeg;
GPU shaders for the luma-keyed mood grade + analytical-overlay compositing;
realize crossfade/fade timing and the 515 min loop. (Open decision: player
stack — mpv via IPC vs. ffmpeg vs. custom.)
- **Serial input** — real USB-serial reader on the Pi + the **3⇄4 framing
contract** with the firmware (the keyboard/serial stand-in already works via a
`Controls` stream).
- **White-noise generation** — runtime white/pink noise for that content position.
- **Offline v2v variant pipeline** — author the pre-baked Right restyle variants
(the only paid AI step, design §9) + the multilingual label/string tables + TTS.
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
variant" flag (sub-project 2 territory, design §13).
**Depends on:** sub-project 1 (`Coordinate`); a neutral base library from
sub-project 2 to be meaningful, but developable against a hand-authored library
and a **keyboard/serial stand-in** before firmware exists.
**Done when:** given a base library and a stream of control values, it plays the
correct altered segment, loops, crossfades, and goes dark on `Off` — the
decision logic is covered (slice 1, serial mocked); the renderer realizes it.
**Hardware:** Raspberry Pi 5 (holds the drive + base library + variants).
**Open decisions:** player stack (mpv via IPC vs. ffmpeg vs. custom); whether the
player hard-restricts to `approved` records.
player hard-restricts to `approved` records (the `approved_only` flag is plumbed
but inert until backed by a real catalog); knob→strength calibration (the §3 vs
§4.2/§5 reconciliation — see the slice-1 plan / session 0006 transcript).
---
@@ -0,0 +1,867 @@
# Player Alteration Core Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the pure-logic core of sub-project 3 (the Player Runtime + alteration engine) — a new `player/` package that turns control-panel state into a render plan and playback transitions, fully unit-tested with all I/O (video/audio/serial) behind injected interfaces.
**Architecture:** Four small, pure modules mirroring `hef/` conventions (frozen dataclasses, `from __future__ import annotations`, string-constant frozensets, thorough unit tests, no heavy deps). `controls` models the panel state (the serial-contract data shape shared with sub-project 4); `content` resolves the 7-way content dial; `alteration` is the heart — knob vector → `RenderPlan` per design §4/§5; `state` is the player state machine that diffs successive controls into playback transitions (crossfade / fade-to-black / live-update). No mpv, no GPU, no real serial, no white-noise DSP, no v2v pipeline — those are later integration slices.
**Tech Stack:** Python 3.11+, stdlib only (`dataclasses`, `math`), pytest. Reuses `hef.selection.Coordinate` for the knob vector.
**Design reference:** `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` (§4 alteration engine, §5 mood grade, §6 7-way content dial, §7 intensity) and `docs/ROADMAP.md` §3 (Player Runtime "done when").
**Calibration decision (flagged for operator):** brain knobs use `strength = value/4` (0=off, 4=max); mood uses `tone = (lightdark)/4` (equal dark/light = identity per §5). This reads §3's "(2,2,2,2) neutral" as a vestige of the old coordinate-grid center. The calibration lives in three one-line helpers so the convention can be flipped trivially. See the session 0006 transcript's Deferred decisions.
---
## File Structure
- `player/__init__.py` — package marker (empty).
- `player/controls.py``Controls` frozen dataclass (content position + 4 experience knobs + volume + brightness), `CONTENT_POSITIONS`, `validate_controls`, `parse_controls` (from a decoded mapping; wire framing is the separate 3⇄4 serial contract).
- `player/content.py``ContentResolution` (audio source + video flag), `AUDIO_SOURCES`, `resolve_content` (the §6 7-row table, single source of truth).
- `player/alteration.py``ColorGrade`, `AnalyticalOverlay`, `Restyle`, `RenderPlan`, `plan_alteration(coord)`, and the calibration helpers `_overlay_intensity`, `_restyle_blend`, `_mood_tone`.
- `player/state.py``Playback`, `TransitionKind`, `Transition`, `Player` (consumes a stream of `Controls`, emits a `Transition` per update; output sink is injected/duck-typed; base-clip choice is an injected callable).
- `pyproject.toml` — add `"player"` to `[tool.setuptools] packages`.
Tests (one per module): `tests/test_player_controls.py`, `tests/test_player_content.py`, `tests/test_player_alteration.py`, `tests/test_player_state.py`.
---
## Task 1: Controls model (the serial-contract data shape)
**Files:**
- Create: `player/__init__.py`
- Create: `player/controls.py`
- Test: `tests/test_player_controls.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/test_player_controls.py
import pytest
from player.controls import (
Controls,
CONTENT_POSITIONS,
ControlsError,
validate_controls,
parse_controls,
)
def test_content_positions_are_the_seven_from_the_spec():
assert CONTENT_POSITIONS == frozenset(
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
)
def test_valid_controls_pass_validation():
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
validate_controls(c) # does not raise
def test_invalid_content_position_rejected():
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
with pytest.raises(ControlsError):
validate_controls(c)
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
@pytest.mark.parametrize("bad", [-1, 5, True])
def test_out_of_range_or_non_int_knob_rejected(field, bad):
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
kwargs[field] = bad
with pytest.raises(ControlsError):
validate_controls(Controls(**kwargs))
def test_parse_controls_from_mapping():
c = parse_controls(
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
)
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
def test_parse_controls_rejects_unknown_keys():
with pytest.raises(ControlsError):
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
def test_parse_controls_rejects_missing_keys():
with pytest.raises(ControlsError):
parse_controls({"content": "off", "left": 0})
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `pytest tests/test_player_controls.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'player'`.
- [ ] **Step 3: Write minimal implementation**
```python
# player/__init__.py
```
(empty file)
```python
# player/controls.py
"""Control-panel state: the data shape read from the Arduino over serial.
This is the serial-contract payload shared with sub-project 4 (firmware). It
models the full panel from design §6/§7: the 7-way content dial, the four
experience knobs (0..4), and the two intensity levels (volume, brightness).
The wire framing itself is the separate 3<->4 serial contract; this module is
the *decoded* form.
"""
from __future__ import annotations
from dataclasses import dataclass, fields
# The seven positions of the content dial (design §6).
CONTENT_POSITIONS = frozenset(
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
)
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
KNOB_MIN = 0
KNOB_MAX = 4
class ControlsError(ValueError):
"""Raised when a Controls payload is structurally invalid."""
@dataclass(frozen=True)
class Controls:
content: str
left: int
right: int
dark: int
light: int
volume: int
brightness: int
def validate_controls(c: Controls) -> None:
"""Raise ControlsError if the payload is structurally invalid."""
if c.content not in CONTENT_POSITIONS:
raise ControlsError(
f"invalid content position {c.content!r}; "
f"expected one of {sorted(CONTENT_POSITIONS)}"
)
for name in KNOB_FIELDS:
value = getattr(c, name)
if isinstance(value, bool) or not isinstance(value, int):
raise ControlsError(f"knob {name} must be an int, got {value!r}")
if not (KNOB_MIN <= value <= KNOB_MAX):
raise ControlsError(
f"knob {name}={value} out of range {KNOB_MIN}..{KNOB_MAX}"
)
def parse_controls(data: dict) -> Controls:
"""Build a validated Controls from a decoded mapping, rejecting unknown or
missing keys."""
known = {f.name for f in fields(Controls)}
unknown = set(data) - known
if unknown:
raise ControlsError(f"unknown keys: {sorted(unknown)}")
missing = known - set(data)
if missing:
raise ControlsError(f"missing keys: {sorted(missing)}")
c = Controls(**data)
validate_controls(c)
return c
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `pytest tests/test_player_controls.py -v`
Expected: PASS (all cases).
- [ ] **Step 5: Commit**
```bash
git add player/__init__.py player/controls.py tests/test_player_controls.py
git commit -m "feat(player): Controls panel model (serial-contract data shape)
Sub-project 3 slice 1, per ROADMAP.md §3 and design §6/§7."
```
---
## Task 2: Content-dial resolution (§6)
**Files:**
- Create: `player/content.py`
- Test: `tests/test_player_content.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/test_player_content.py
import pytest
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
def test_audio_sources_are_the_four_distinct_sources():
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
# The §6 table, row by row: position -> (audio_source, video).
@pytest.mark.parametrize(
"position,audio_source,video",
[
("off", "none", False),
("white_noise", "white_noise", False),
("music", "music", False),
("audio_track", "audio_track", False),
("video", "none", True),
("music_video", "music", True),
("audio_video", "audio_track", True),
],
)
def test_resolve_content_matches_spec_table(position, audio_source, video):
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
def test_off_is_void_state_black_and_silent():
r = resolve_content("off")
assert r.video is False and r.audio_source == "none"
def test_resolve_content_rejects_unknown_position():
with pytest.raises(ValueError):
resolve_content("bogus")
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `pytest tests/test_player_content.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'player.content'`.
- [ ] **Step 3: Write minimal implementation**
```python
# player/content.py
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
The single source of truth for design §6's table: which audio source plays and
whether the projector shows video, for each dial position.
"""
from __future__ import annotations
from dataclasses import dataclass
# The four distinct audio sources. "white_noise" is generated at runtime;
# "music" is the public-domain classical pool; "audio_track" is the clip's own
# field audio; "none" is silence.
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
@dataclass(frozen=True)
class ContentResolution:
audio_source: str
video: bool
# Design §6, one row per dial position.
_TABLE = {
"off": ContentResolution("none", False),
"white_noise": ContentResolution("white_noise", False),
"music": ContentResolution("music", False),
"audio_track": ContentResolution("audio_track", False),
"video": ContentResolution("none", True),
"music_video": ContentResolution("music", True),
"audio_video": ContentResolution("audio_track", True),
}
def resolve_content(position: str) -> ContentResolution:
"""Map a content-dial position to its audio source and video flag (§6)."""
try:
return _TABLE[position]
except KeyError:
raise ValueError(
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
) from None
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `pytest tests/test_player_content.py -v`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add player/content.py tests/test_player_content.py
git commit -m "feat(player): 7-way content-dial resolution (design §6)"
```
---
## Task 3: Alteration engine — knob vector to RenderPlan (§4, §5)
**Files:**
- Create: `player/alteration.py`
- Test: `tests/test_player_alteration.py`
The engine maps a knob vector `(left, right, dark, light)` to a layered `RenderPlan` per design §4.2:
- **Substrate** transforms (alter pixels): `Restyle` (Right, v2v) blended with `ColorGrade` (mood, Dark/Light).
- **Overlay** on top: `AnalyticalOverlay` (Left, HUD/labels).
`ColorGrade.tone` is signed: `>0` warm yellow→white (light pole), `<0` cool blue→black (dark pole), `0` identity/raw (§5). Calibration (flagged decision): `overlay = left/4`, `restyle = right/4`, `tone = (light dark)/4`.
- [ ] **Step 1: Write the failing tests**
```python
# tests/test_player_alteration.py
import pytest
from hef.selection import Coordinate
from player.alteration import (
AnalyticalOverlay,
ColorGrade,
RenderPlan,
Restyle,
plan_alteration,
)
def _coord(left=0, right=0, dark=0, light=0):
return Coordinate(left=left, right=right, dark=dark, light=light)
def test_all_zero_knobs_is_the_unaltered_base():
plan = plan_alteration(_coord())
assert plan.is_identity
assert plan.overlay.intensity == 0.0
assert plan.restyle.blend == 0.0
assert plan.grade.tone == 0.0
assert plan.grade.is_identity
def test_left_drives_the_analytical_overlay_only():
plan = plan_alteration(_coord(left=4))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 0.0 # Left does not touch the substrate
assert plan.grade.tone == 0.0
def test_right_drives_the_restyle_substrate_only():
plan = plan_alteration(_coord(right=2))
assert plan.restyle.blend == 0.5
assert plan.overlay.intensity == 0.0 # Right does not add overlay
def test_left_and_right_stack_not_cancel():
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
plan = plan_alteration(_coord(left=4, right=4))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 1.0
def test_light_pole_grades_warm_positive_tone():
plan = plan_alteration(_coord(light=4))
assert plan.grade.tone == 1.0
assert not plan.grade.is_identity
def test_dark_pole_grades_cool_negative_tone():
plan = plan_alteration(_coord(dark=4))
assert plan.grade.tone == -1.0
def test_equal_dark_and_light_is_identity_grade():
# design §5: the mood center is the raw, ungraded footage
assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity
assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity
def test_dark_minus_light_sets_intermediate_tone():
assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5)
assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5)
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
# design §4.2 "Dark + analytical": cold measurement over a melancholy scene
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 0.5
assert plan.grade.tone == -1.0
assert not plan.is_identity
def test_render_plan_is_frozen():
plan = plan_alteration(_coord())
with pytest.raises(Exception):
plan.grade.tone = 0.5 # type: ignore[misc]
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `pytest tests/test_player_alteration.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'player.alteration'`.
- [ ] **Step 3: Write minimal implementation**
```python
# player/alteration.py
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
Replaces the 2026-06-04 nearest-match *selection* with a *transformation* of a
neutral base clip. Given the four experience knobs, it produces three layers
that compose per §4.2:
- Substrate transforms (alter pixels, blend with each other):
* Restyle -- the Right axis: a pre-baked generative v2v dreamlike restyle.
* ColorGrade -- the mood axis (Dark/Light): a deterministic color grade.
- Overlay (composited on top of the substrate):
* AnalyticalOverlay -- the Left axis: HUD/labels/measurement.
Left and Right are NOT opposites; they live on different layers and stack
(§4.2). Dark and Light are the two poles of one mood grade whose center is the
identity (§5).
Calibration note: the knob->strength curves below are the single source of
truth for how a 0..4 position maps to a transform strength. See the session
0006 transcript Deferred decisions for the §3-vs-§4.2/§5 reconciliation.
"""
from __future__ import annotations
from dataclasses import dataclass
from hef.selection import Coordinate
KNOB_MAX = 4 # knob full-scale (0..4)
@dataclass(frozen=True)
class ColorGrade:
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
<0 cool blue->black (dark), 0 = identity (raw, ungraded)."""
tone: float
@property
def is_identity(self) -> bool:
return self.tone == 0.0
@dataclass(frozen=True)
class AnalyticalOverlay:
"""Left axis (§4.1): analytical HUD/annotation/labels, composited on top.
`intensity` 0..1 (0 = no overlay)."""
intensity: float
@dataclass(frozen=True)
class Restyle:
"""Right axis (§4.1): pre-baked generative v2v dreamlike substrate.
`blend` 0..1 (0 = raw substrate, no restyle)."""
blend: float
@dataclass(frozen=True)
class RenderPlan:
"""The full layered alteration for one knob vector (§4.2)."""
grade: ColorGrade
overlay: AnalyticalOverlay
restyle: Restyle
@property
def is_identity(self) -> bool:
"""True when the plan leaves the neutral base un-altered."""
return (
self.grade.is_identity
and self.overlay.intensity == 0.0
and self.restyle.blend == 0.0
)
def _overlay_intensity(left: int) -> float:
"""Left knob -> analytical-overlay intensity (0..1)."""
return left / KNOB_MAX
def _restyle_blend(right: int) -> float:
"""Right knob -> v2v restyle blend (0..1)."""
return right / KNOB_MAX
def _mood_tone(dark: int, light: int) -> float:
"""(dark, light) -> signed mood grade in [-1, 1]; equal -> 0 identity (§5)."""
return (light - dark) / KNOB_MAX
def plan_alteration(coord: Coordinate) -> RenderPlan:
"""Map a knob vector to its layered RenderPlan (design §4)."""
return RenderPlan(
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light)),
overlay=AnalyticalOverlay(intensity=_overlay_intensity(coord.left)),
restyle=Restyle(blend=_restyle_blend(coord.right)),
)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `pytest tests/test_player_alteration.py -v`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add player/alteration.py tests/test_player_alteration.py
git commit -m "feat(player): alteration engine — knob vector to RenderPlan (design §4/§5)"
```
---
## Task 4: Player state machine — controls stream to transitions
**Files:**
- Create: `player/state.py`
- Test: `tests/test_player_state.py`
The `Player` holds a base-clip library and the current `Playback`. Each `update(controls)` resolves the desired `Playback` (chosen base clip + RenderPlan + content resolution + volume/brightness) and returns the `Transition` from the previous playback:
- target video off, was on → `FADE_TO_BLACK`
- target video on, was off → `FADE_FROM_BLACK`
- both video on, clip or restyle variant changed → `CROSSFADE`
- both video on, only grade/overlay/level changed → `LIVE_UPDATE` (§4.3: grade + overlay are continuous runtime ops)
- both video off, audio/level changed → `LIVE_UPDATE`
- nothing changed → `NONE`
Base-clip choice is an injected callable (default: first clip); knobs no longer *select* (the base is neutral) — they *transform*. The library is duck-typed objects with an `.id`.
- [ ] **Step 1: Write the failing tests**
```python
# tests/test_player_state.py
from dataclasses import dataclass
import pytest
from player.controls import Controls
from player.state import Player, Playback, Transition, TransitionKind
@dataclass(frozen=True)
class FakeClip:
id: str
LIB = [FakeClip("base-a"), FakeClip("base-b")]
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
return Controls(content, left, right, dark, light, volume, brightness)
def test_first_update_to_video_fades_in_from_black():
p = Player(LIB)
t = p.update(_controls(content="video"))
assert t.kind == TransitionKind.FADE_FROM_BLACK
assert t.playback.clip_id == "base-a"
assert t.playback.content.video is True
def test_off_from_video_fades_to_black_and_silences():
p = Player(LIB)
p.update(_controls(content="video"))
t = p.update(_controls(content="off"))
assert t.kind == TransitionKind.FADE_TO_BLACK
assert t.playback.clip_id is None
assert t.playback.content.audio_source == "none"
def test_no_change_yields_none_transition():
p = Player(LIB)
p.update(_controls(content="video", left=1))
t = p.update(_controls(content="video", left=1))
assert t.kind == TransitionKind.NONE
def test_grade_change_is_a_live_update_not_a_crossfade():
# design §4.3: the Dark/Light grade is a continuous runtime op
p = Player(LIB)
p.update(_controls(content="video", dark=0, light=0))
t = p.update(_controls(content="video", dark=4, light=0))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.grade.tone == -1.0
def test_overlay_change_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="video", left=0))
t = p.update(_controls(content="video", left=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.overlay.intensity == 1.0
def test_restyle_change_crossfades_the_substrate():
# design §4.3: the Right v2v substrate is a pre-baked variant swap
p = Player(LIB)
p.update(_controls(content="video", right=0))
t = p.update(_controls(content="video", right=4))
assert t.kind == TransitionKind.CROSSFADE
assert t.playback.plan.restyle.blend == 1.0
def test_volume_only_change_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="video", volume=1))
t = p.update(_controls(content="video", volume=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.volume == 4
def test_audio_source_change_while_black_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="white_noise"))
t = p.update(_controls(content="music"))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.content.audio_source == "music"
def test_injected_base_chooser_is_used():
p = Player(LIB, choose_base=lambda lib: lib[1])
t = p.update(_controls(content="video"))
assert t.playback.clip_id == "base-b"
def test_empty_library_with_video_raises():
p = Player([])
with pytest.raises(ValueError):
p.update(_controls(content="video"))
def test_off_with_empty_library_is_fine():
p = Player([])
t = p.update(_controls(content="off"))
assert t.kind == TransitionKind.NONE # already black at init
assert t.playback.clip_id is None
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `pytest tests/test_player_state.py -v`
Expected: FAIL with `ModuleNotFoundError: No module named 'player.state'`.
- [ ] **Step 3: Write minimal implementation**
```python
# player/state.py
"""The player state machine: a stream of Controls -> playback transitions.
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
desired Playback (which neutral base clip, how it is altered, what audio plays,
at what levels) and returns the Transition from the previous Playback. The
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
pre-baked Right v2v variant needs a CROSSFADE, and toggling video on/off
fades to/from black.
Knobs no longer *select* a clip (the base library is neutral by construction);
they *transform* it. Which neutral base to show is an injected policy
(`choose_base`), defaulting to the first clip; richer rotation is a later slice.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Optional
from hef.selection import Coordinate
from player.alteration import RenderPlan, plan_alteration
from player.content import ContentResolution, resolve_content
from player.controls import Controls
class TransitionKind:
NONE = "none"
LIVE_UPDATE = "live_update"
CROSSFADE = "crossfade"
FADE_TO_BLACK = "fade_to_black"
FADE_FROM_BLACK = "fade_from_black"
@dataclass(frozen=True)
class Playback:
"""What should currently be playing."""
clip_id: Optional[str] # None = black walls
plan: Optional[RenderPlan] # None when black
content: ContentResolution
volume: int
brightness: int
@dataclass(frozen=True)
class Transition:
kind: str
playback: Playback
def _first(library):
if not library:
raise ValueError("no base clips available to play video")
return library[0]
_BLACK = Playback(
clip_id=None,
plan=None,
content=ContentResolution("none", False),
volume=0,
brightness=0,
)
class Player:
def __init__(
self,
base_library,
*,
choose_base: Callable = _first,
approved_only: bool = False,
):
self._library = list(base_library)
self._choose_base = choose_base
self._approved_only = approved_only # reserved for catalog-backed libs
self._current = _BLACK
def _resolve(self, controls: Controls) -> Playback:
content = resolve_content(controls.content)
if not content.video:
return Playback(
clip_id=None,
plan=None,
content=content,
volume=controls.volume,
brightness=controls.brightness,
)
clip = self._choose_base(self._library)
coord = Coordinate(controls.left, controls.right, controls.dark, controls.light)
return Playback(
clip_id=clip.id,
plan=plan_alteration(coord),
content=content,
volume=controls.volume,
brightness=controls.brightness,
)
def _classify(self, prev: Playback, nxt: Playback) -> str:
prev_video = prev.clip_id is not None
next_video = nxt.clip_id is not None
if prev == nxt:
return TransitionKind.NONE
if prev_video and not next_video:
return TransitionKind.FADE_TO_BLACK
if next_video and not prev_video:
return TransitionKind.FADE_FROM_BLACK
if next_video and prev_video:
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
return TransitionKind.CROSSFADE
return TransitionKind.LIVE_UPDATE
# both black: only audio/levels could have changed
return TransitionKind.LIVE_UPDATE
def update(self, controls: Controls) -> Transition:
nxt = self._resolve(controls)
kind = self._classify(self._current, nxt)
self._current = nxt
return Transition(kind=kind, playback=nxt)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `pytest tests/test_player_state.py -v`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add player/state.py tests/test_player_state.py
git commit -m "feat(player): player state machine — controls stream to transitions"
```
---
## Task 5: Register the package and run the full suite
**Files:**
- Modify: `pyproject.toml` (`[tool.setuptools] packages`)
- [ ] **Step 1: Add `"player"` to the packages list**
In `pyproject.toml`, change:
```toml
[tool.setuptools]
packages = ["hef", "tools", "simulator"]
```
to:
```toml
[tool.setuptools]
packages = ["hef", "tools", "simulator", "player"]
```
- [ ] **Step 2: Run the entire test suite**
Run: `pytest -q`
Expected: all prior tests still pass plus the new `test_player_*` files; no failures.
- [ ] **Step 3: Commit**
```bash
git add pyproject.toml
git commit -m "build(player): register the player package"
```
---
## Task 6: Update the roadmap to reflect slice 1 shipped
**Files:**
- Modify: `docs/ROADMAP.md` (§3 Player Runtime)
- [ ] **Step 1: Update §3 status and deliverables**
Under "## 3. Player Runtime (Pi)", note that the **alteration-engine + player-core slice** (pure logic) is done and tested, link this plan, and list the remaining slices (mpv/ffmpeg runtime integration + GPU grade/overlay shaders; real USB-serial reader + the 3⇄4 framing contract; white-noise DSP; offline v2v variant build pipeline; catalog model changes for audio-source + neutral-base flag). Keep the table's status marker for sub-project 3 as in-progress (⏳), not done.
- [ ] **Step 2: Commit**
```bash
git add docs/ROADMAP.md
git commit -m "docs(roadmap): sub-project 3 alteration-engine + player-core slice shipped"
```
---
## Self-Review
**Spec coverage:**
- §4.1 per-axis alteration → Task 3 (overlay=Left, restyle=Right, grade=mood). ✓
- §4.2 composition rule (substrate vs overlay; Left+Right stack) → Task 3 tests `test_left_and_right_stack_not_cancel`, `test_whole_brain_dark_corner_*`. ✓
- §4.3 runtime vs pre-baked (grade/overlay continuous; restyle variant swap) → Task 4 transition kinds (`LIVE_UPDATE` vs `CROSSFADE`). ✓
- §5 mood grade, center = identity → Task 3 `test_equal_dark_and_light_is_identity_grade`, signed tone. ✓
- §6 7-way content dial → Task 2 full table. ✓
- §7 volume + brightness intensity → Task 1 (modeled) + Task 4 (`test_volume_only_change_is_a_live_update`). ✓
- Roadmap §3 "done when: plays correct segment, loops, crossfades, goes dark on None, testable with serial mocked" → Task 4 (transitions incl. fade-to-black; serial mocked as a `Controls` stream). Looping is implicit (a clip plays until the desired playback changes); explicit loop timing belongs to the mpv integration slice (deferred, noted in Task 6). ✓ (with the runtime-loop portion deferred and documented).
**Deferred (documented, not gaps):** mpv/ffmpeg + GPU; real serial + framing; white-noise DSP; v2v offline pipeline; catalog audio-source/neutral-base fields; base-clip rotation policy; `approved_only` enforcement against a real catalog (the flag is plumbed but inert here).
**Placeholder scan:** none — every step has concrete code/commands.
**Type consistency:** `Controls(content,left,right,dark,light,volume,brightness)` used identically in Tasks 1 & 4; `RenderPlan.{grade,overlay,restyle}` and `.is_identity` consistent across Tasks 3 & 4; `ContentResolution(audio_source,video)` consistent across Tasks 2 & 4; `TransitionKind` constants consistent. `Coordinate(left,right,dark,light)` matches `hef/selection.py`.
@@ -0,0 +1,360 @@
# Human Experience Filter — Machine-Altered Perception (Design Revision)
**Date:** 2026-06-05
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
**Supersedes:** the thesis (§1), selection model (§3), tagging division (§5), and
sourcing strategy (§8) of
[`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md),
and substantially expands its control panel (§2 selector) and hardware (§6). The
coordinate model (§2 axes) and the single-panoramic-projector display are
**preserved**.
> **Why this revision exists.** The 2026-06-04 design framed the piece as an
> *experience filter over found human artifacts* ("curation is the artwork; not
> algorithmically generated filler"). This revision **changes the artistic
> thesis** to one about **how humans interact with machines and how that interaction
> reshapes their nervous systems** — and in doing so makes machine alteration of
> the imagery the *subject* of the piece rather than a betrayal of it.
---
## 1. What this is (the new thesis)
A single-viewer immersive installation about **a machine reshaping your
perception — for better and for worse.**
One person sits in a chair in a small room. A single panoramic projector wraps the
walls they face. The source is **neutral**: real, calm, public-domain nature
footage that sits at the *center* of the experience-space — neither analytical nor
emotional, neither bright nor dark. The viewer turns knobs on a wooden control
panel, and **the machine alters that neutral reality toward whatever they dial
in.** The piece is the felt, bodily experience of a machine bending your inner
state — and crucially it bends **both ways**: it can soothe you or disturb you,
clarify the world or manipulate it, name your feelings or colonize them.
You don't read that thesis on a wall label. You feel it in your nervous system as
your own hand moves the knobs.
Design constraints (carried from the original, one relaxed):
- **One viewer, one room, all-local at runtime.** No networking, no streaming, no
multi-user, no session recording. (All AI alteration happens **offline at
authoring time**; the room plays pre-built media + cheap runtime grading.)
- **Real footage as the substrate.** The base library is real public-domain
nature footage. The machine *alters* it — it does not invent from nothing. This
preserves real-world motion/composition (and some lineage to the original
found-media thesis) while making the alteration the point. *(This relaxes the
original "no generated content" rule: generation/alteration is now the
subject.)*
- **Built to grow.** A small, deliberate base library; the data model and tooling
designed to expand.
---
## 2. What changes vs. 2026-06-04
| Area | 2026-06-04 | This revision |
|---|---|---|
| Thesis (§1) | filter over found human artifacts | **machine reshapes perception, good & bad** |
| Content | large found library pre-matched to coordinates | **small NEUTRAL base library, machine-altered to the knob state** |
| Engine (§3) | nearest-match **lookup** in a tagged catalog | **transform from neutral** (knobs = a transformation, not a lookup) |
| Sourcing (§8) | find media that already matches each cell | source **neutral** clips; the machine produces the variants |
| Control panel (§2) | 4-way mode + 4 knobs | **7-way content dial + 4 experience knobs + volume + brightness**, on a tactile wooden panel |
| Display | single pano projector | **unchanged** |
| Coordinate axes (§2) | Left/Right/Dark/Light, two planes | **unchanged** (reinterpreted as transform targets) |
---
## 3. Coordinate model (preserved, reinterpreted)
The four experience knobs and two planes from the 2026-06-04 §2 are unchanged:
- **Brain plane** = Left × Right. Left = analytical / verbal / structured. Right =
artistic / emotional / abstract. Both high → "whole brain."
- **Mood plane** = Dark × Light. Dark = somber / heavy. Light = uplifting / serene.
Each knob is `04`. What changes is the *meaning of a knob position*: it is no
longer "find the catalog piece nearest this coordinate" but **"how hard the machine
pushes the neutral base toward this pole."** `(2,2,2,2)` is the neutral origin —
the un-altered base clip.
---
## 4. The alteration engine (replaces the §3 selection algorithm)
The engine's job changes from **choosing** a clip to **altering** one. Given a
neutral base clip and a knob vector `(left, right, dark, light)`, it applies a
**transformation toward each active pole** and composites the result.
### 4.1 Per-axis alteration
| Pole | What the machine does | How |
|---|---|---|
| **Left** (analytical/verbal) | **imposes analytical structure** on the scene — annotation, measurement, labels, grids, tracking boxes, data HUD; the machine *dissecting* the world into data | **overlay** layer composited on top |
| **Right** (artistic/emotional) | **dissolves realism** toward painterly / dreamlike / abstract | **generative video-to-video (v2v)** restyle of the pixels |
| **Dark** (somber/heavy) | drains toward the cool/melancholy/night grade (see §5) | **color grade** (deterministic) |
| **Light** (uplifting/serene) | warms toward the bright/serene grade (see §5) | **color grade** (deterministic) |
### 4.2 The composition rule
The four axes are **two kinds of operation** that **layer** rather than blend-and-
cancel:
- **Substrate transforms** — Right (v2v restyle) + Dark + Light (color grades).
They alter the **pixels** and blend with each other.
- **Overlay** — Left (annotation / labels / HUD). It composites **on top** of
whatever the substrate is.
So Left and Right are **not** mutually-exclusive opposites; they stack on different
layers. This single rule covers the whole `5×5×5×5` space:
- **Max Left + Max Right** = maximally dissolved dreamlike substrate **with the
machine's labels on the feelings themselves** — *"awe," "grief," "detected:
longing 0.82,"* tracking boxes around emotional content. This is the "whole
brain" corner (§3): the machine that both **induces** emotion and
**names/quantifies** it — affective computing made visceral. (Good/bad duality:
is being told your own feeling clarifying, or invasive?)
- **Dark + analytical** = a drained, melancholy scene with cold measurement
overlaid.
- **Light + emotional** = a warm, dreamlike wash, unlabeled.
### 4.3 Where each transform runs
- **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade
and the Left analytical overlay. These are cheap (LUT/curves + luma key, and
text/graphics compositing) and can move continuously with the knob.
- **Pre-baked offline (the only AI cost):** the Right generative v2v restyle.
Generative v2v cannot run real-time on a Pi, so a small set of restyled variants
is rendered offline and selected/blended at runtime.
> **Crucial consequence — the Left labels are a RUNTIME OVERLAY, never baked into
> the v2v pixels.** This is what makes multilingual support nearly free (§10).
---
## 5. Mood-axis color grade
The Dark↔Light knob is a three-stop color ramp whose **center is the identity**
the raw, ungraded footage, no color transform applied. Turning toward a pole grades
*away* from the raw clip. Hue carries emotional valence; the **negative-space
value** (a luma-keyed grade — the brightest/darkest regions pushed toward
white/black) carries the literal light/dark reading, so the image *is* lighter or
darker even before the hue registers.
| Mood knob | Hue | Negative space | Feeling |
|---|---|---|---|
| **Light** | warm **yellow** | → **white** | uplifting, serene, airy |
| **Center** | **none — raw, ungraded video** | unchanged | the un-altered base |
| **Dark** | cool **blue** | → **black** | melancholy, somber, night, cold |
*(Red/alarm was considered for Dark and set aside: red reads as threat/alarm, not
the intended melancholy. Dark is the cool, mournful blue. The center applies no
grade at all — it is simply the raw clip.)*
---
## 6. Content model — the 7-way content dial
The 2026-06-04 4-way selector (None/Audio/Video/A+V) expands to **seven positions**,
letting the viewer choose channel **and audio source**:
| Position | Audio source | Video |
|---|---|---|
| **Off** | — | black |
| **White Noise** | generated white/pink noise (not from catalog) | black |
| **Music** | public-domain classical (Musopen pool) | black |
| **Audio Track** | the nature clip's **own** audio | black |
| **Video** | — | the (altered) nature video |
| **Music + Video** | public-domain classical (Musopen pool) | the (altered) nature video |
| **Audio Track + Video** | the nature clip's own audio | the (altered) nature video |
Notes:
- "Music" (classical) and "Audio Track" (the clip's own field audio) are now
**distinct sources**; "White Noise" is **generated** at runtime, not a catalog
record. This touches the catalog/`mode` model (the 2026-06-04 §4 schema), which
must represent audio *source* in addition to channel.
- **Off** is the void/rest state: black walls, silence.
- Audio alteration (steering the *sound* by knob, parallel to the video) is **out
of scope for this revision** — the experience knobs alter video; audio is chosen
by source only. (Future work.)
---
## 7. Intensity controls
Two knobs, orthogonal to the four experience axes, exist so a viewer can take the
experience at **whatever intensity they want**:
- **Volume** — audio output level.
- **Brightness** — projector output luminance.
These are pure output levels, distinct from the Dark/Light *mood grade* (which
changes hue and tone, not raw output). Brightness is "how much light hits the
wall"; Light is "how the world is colored."
---
## 8. Content sourcing — neutral base library
The library is now **small and neutral**. Instead of hunting for footage that
matches 625 coordinate cells, source a deliberate set of **neutral, calm,
real public-domain nature clips** (centered on all four axes) and let the machine
produce every variant.
- **Reuse sub-project 2** (the built ingest/tagging/review tooling) to pull
neutral public-domain nature footage and populate mechanical fields.
- **License stance** unchanged: prefer Public Domain / CC0; record license +
source per clip. Altered/generated variants record the model and that they are
machine-derived.
- **Size:** dozens of neutral base clips (not hundreds), each yielding many
altered variants. Short, seamless loops (≈2060 s) suit a looping installation
and are the v2v models' native sweet spot.
---
## 9. Economics (why this is feasible)
At the original scale (120800 pieces × ~10-min segments) AI involvement was
~$15k45k+ and not viable. Under this design it drops to **a few hundred to a few
thousand dollars**, because:
- The library is small (generate-to-order, not find-to-cover).
- Pieces are short seamless loops (cheaper **and** higher-quality for the models).
- Only the **Right** transform uses paid generative v2v; Dark/Light/Left are free
deterministic runtime operations.
Indicative offline v2v build (2026 rates — Kling 3.0 / Sora 2 Standard ≈ $0.10/s,
Veo 3.1 Lite / Runway Turbo ≈ $0.05/s; ×~4 retry yield): **~$3003k** for a small
base library × a handful of restyle variants. A flat authoring subscription
(Runway $2876/mo, Kling from ~$6/mo) may be cheaper than per-second API for a
one-time build.
---
## 10. Accessibility, i18n, and the translation-cost finding
The piece is operable **blind, in the dark, in your language**, via four redundant
channels on the control panel: **touch** (engraved symbol shape), **low-light
color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local*
speaker, separate from the main audio system).
Because "the human experience from a communication perspective is not universal" is
itself on-thesis, the piece supports **many languages**. The cost of that hinges on
one architectural decision:
- **❌ Labels baked into the v2v render** → re-render every label-bearing variant
per language → video cost **× number of languages** (≈ $9k raw / $2745k with
retries for ~50 languages). A budget landmine.
- **✅ Labels as a runtime overlay** (which §4.2 already requires) → the v2v
substrate is rendered **once**, language-agnostic; a new language is a translated
string table + font + TTS voice. **Marginal video cost per language ≈ $0.**
So multilingual support costs only cheap text work: machine-translating a small
label/script vocabulary into ~100 languages is a few dollars; optional human review
of a curated tier is ~$0.10/word (~$50200/language); TTS for the read-aloud button
is pennies per language. **Net: supporting *every* language is essentially free on
the expensive part, provided labels stay a runtime overlay.**
---
## 11. The physical control panel
A **wood** panel. Each control has a **tactile engraved symbol** below it —
distinguishable by fingertip in the dark, and shaped to evoke the control's
meaning — with **braille** beneath, a **dim color-coded LED** backlighting the
engraving, and a **read-aloud button** that speaks the control's function on a
small local speaker.
Layout principle: the **four experience knobs are two opposing pairs**, with the
opposition encoded *in the touch* (angular vs flowing, heavy-down vs radiant-up).
The **utility controls** use a more conventional AV shape-language so they are
never confused with the experience four.
| Control | Engraved symbol | Touch | LED |
|---|---|---|---|
| **Left** (analytical) | parallel straight ridges / grid `≡` `▦` | flat, hard, orthogonal edges | — |
| **Right** (emotional) | spiral / flowing wave `∿` | one continuous curve, no corners | — |
| **Dark** (somber) | crescent / downward wedge `☾` `▼` | smooth solid mass, sinking | **blue** |
| **Light** (serene) | radial sunburst `☀` | center with radial spikes | **yellow / white** |
| **Content dial** (7) | per-position micro-icons: Off `○` · White-Noise `∴` · Music `♪` · Audio-Track `)))` · Video `▷` · Music+Video `♪▷` · A+V `▷)))` | detented clicks | neutral |
| **Volume** | growing wedge `◁` | a slope that thickens | neutral |
| **Brightness** | half-disc contrast `◐` | a circle, half raised | neutral |
Design notes:
- **Light vs Brightness** is the one collision trap (both are "bright"). Light =
radial sunburst (a *mood*, glows the hue it imparts); Brightness = half-disc
contrast (an *output level*) — distinct to finger and eye.
- **LEDs equal the hue the knob imparts:** Dark glows blue, Light glows
yellow/white, the mood center is neutral — tying the panel to the projection.
- **Negative-space symbols:** symbols differ by gross **edge profile** (curve vs
corner vs radial vs slope), not fine detail, so they read under a fingertip.
This substantially fleshes out **sub-project 4 (Arduino firmware / control panel)**,
previously only sketched: 7 controls + a read-aloud button + LEDs + braille, with
the Arduino reading positions and the read-aloud/LED behavior coordinated with the
Pi.
---
## 12. Hardware (revises §6)
Unchanged from the 2026-06-04 single-pano revision except for the richer panel:
- **Arduino** — reads the 7-way content dial, 4 experience knobs, volume,
brightness, and the read-aloud button; drives the panel LEDs; sends values to the
Pi over USB serial. (Bigger than the original 5-value panel.)
- **Raspberry Pi 5 (or mini-PC)** — holds the base library + altered variants,
runs the alteration engine (runtime grading + overlay compositing + variant
selection), drives the single panoramic projector, plays audio, and serves the
read-aloud clips to the local panel speaker.
- **Single panoramic projector** spanning the three walls — **unchanged**.
- **Small local speaker** on/near the panel for the read-aloud button (separate
from the main audio).
---
## 13. Roadmap impact
- **Sub-project 2 (ingest/tagging)** — reused to source the **neutral** base
clips; coordinate *drafting* matters less (the base is neutral by selection), but
mechanical tagging + license capture still apply; add a flag for "neutral base"
vs "altered variant," and model audio *source*.
- **Sub-project 3 (player)** — grows an **alteration engine**: runtime color grade
(luma-keyed), runtime analytical-overlay compositor (multilingual, from string
tables), generated white noise, and selection/blending of pre-baked v2v variants.
- **Sub-project 4 (firmware/panel)** — grows from a 5-value panel to the full
tactile/accessible panel in §11.
- **New offline pipeline** — author neutral→restyled v2v variants (the only paid
AI step) and the label/string tables + TTS per language.
---
## 14. Open questions (for the plan, not blockers)
- **Pano resolution strategy** — v2v models output ~16:9 720p1080p; the three-wall
pano wants ultra-wide/high-res. Generate-at-max + upscale, native-ultrawide
tooling, or accept the look?
- **Negative-space luma key** — exact curve/mask for "brights→white, darks→black"
on arbitrary nature footage.
- **Transform composition order** — order of grade vs v2v substrate when several
poles are active; how variant blending interpolates between pre-baked restyle
levels.
- **Neutral base count** — how many base clips for a satisfying launch.
- **Language tier** — which languages get human-reviewed translation vs
machine-only; default UI language and switching.
- **Runtime grading performance** — confirm the Pi can do luma-keyed grade +
overlay compositing at projector resolution/framerate (GPU shaders via mpv/ffmpeg).
- **`approved`-only enforcement** in the player (carried from the original).
---
## 15. Out of scope (YAGNI)
- Networking / streaming at runtime — all alteration is offline; the room is local.
- Runtime/on-demand generation (breaks all-local; slow; unbounded cost).
- Audio alteration by knob (video-only this revision).
- Multi-user / multi-viewer; session recording or analytics.
- Automatic ML coordinate tagging — base clips are chosen neutral by hand.
@@ -0,0 +1,242 @@
# Human Experience Filter — Simulator Alteration Preview (Design)
**Date:** 2026-06-06
**Status:** Approved design (pre-implementation)
**Repo:** `human-experience-filter-art`
**Builds on:** the alteration engine of
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
(the "design" below) and the simulator scaffold of
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md).
**Revises the design:** §4.3 and §10 — the Left analytical HUD is **no longer a
runtime overlay**; it is **baked into authored variant videos** (see §8 below).
This is a deliberate trade of near-free multilingual support for authorial
precision over the HUD.
**Retires:** the simulator's selection-era "curator's X-ray" view and its
`/api/select` + `/api/catalog/meta` endpoints (the selection model they
visualize was superseded by the alteration model).
> **Why this exists.** Operator directive (session 0007): *build and design only
> things that run in the simulator, and get the whole experience working the way
> we like in the simulator before moving to hardware.* The slice-1 alteration
> engine (PR #5) is pure logic with **no simulator surface** — you cannot yet turn
> the experience knobs and *see* the result. This design brings the alteration
> into the simulator so the look can be tuned and liked before any Pi/serial work.
---
## 1. Scope
A browser **alteration preview**: turn the four experience knobs and the content
dial, and see the neutral base footage altered toward the knob state, in real
time, on a looping clip. The purpose is to **tune the look** of the filter and to
**settle the knob→strength calibration by eye** (the open §3-vs-§4.2/§5 question
from session 0006).
**In scope**
- Live preview of the four experience knobs (Left/Right/Dark/Light) on a looping
base clip.
- **Dark/Light** rendered as a live, deterministic runtime color grade.
- **Left/Right** rendered by selecting a **pre-baked authored variant clip** from
a 5×5 grid and crossfading on change.
- A **calibration panel** that adjusts the grade curve live; the chosen values are
baked back into `player/alteration.py` defaults.
- A **RenderPlan readout** (the project's "X-ray" honesty): always show the exact
numbers the engine produced.
- The content dial's **video on/off** behavior (so "Off" goes to black), driven by
the existing `resolve_content`.
- Placeholder variant generation so the mechanism is testable before the operator
authors real videos.
**Out of scope (this slice)** — real generative v2v; serial input / the 3⇄4
framing contract; audio playback (music / white-noise / audio-track); the Pi/mpv
runtime renderer; the full transition/crossfade timing engine; multilingual label
tables; catalog-model changes. These remain later roadmap slices.
---
## 2. Architecture — Python-canonical, thin browser renderer
`player/alteration.py` stays the **single source of truth** for the alteration
math (handbook §4.2 — deterministic core, thin I/O). The browser owns only
*rendering*. Data flow:
```mermaid
flowchart LR
subgraph Browser
K[4 experience knobs +<br/>content dial +<br/>calibration sliders]
V["&lt;video&gt; variant + canvas grade"]
RO[RenderPlan readout]
end
subgraph FastAPI [simulator/app.py]
EP["POST /api/alteration"]
CL["GET /api/clips"]
end
ENG["player.alteration.plan_alteration(coord, calibration)"]
K -- "debounced POST {controls, calibration}" --> EP
EP --> ENG --> EP -- "RenderPlan {variant, grade}" --> V
EP --> RO
CL -- "base clip + variant manifest" --> V
```
- The browser sends control/calibration changes (debounced) and receives a
`RenderPlan`. Video filtering itself runs continuously in the browser; only a
*plan recompute* makes a round-trip. On localhost these JSON round-trips are
imperceptible.
- "Bake the calibration winner in" = change the `Calibration` defaults in Python.
Nothing in the browser is canonical.
---
## 3. The four axes — how each is rendered
| Axis | Engine output | Browser rendering |
|---|---|---|
| **Dark / Light** | `ColorGrade.tone` ∈ [1, 1], center = identity | live color grade on the `<video>`: Light → warm/yellow + negative space toward white; Dark → cool/blue + negative space toward black; 0 = raw. (Confirmed direction in brainstorm.) |
| **Left / Right** | `VariantRef(left, right)` | select the authored variant clip for `(left, right)`; **crossfade** when it changes; `(0,0)` → raw base. The analytical HUD (Left) and dreamlike restyle (Right) are **baked into the clip** (§8). |
Dark/Light is the only continuously-tunable axis in this slice; Left/Right is a
discrete selection by coordinate.
---
## 4. The variant grid (per base clip)
Each base clip carries a 5×5 grid keyed by the two baked axes:
- Rows = **Left** (analytical / HUD), 04. Columns = **Right** (artistic /
dreamlike restyle), 04.
- `(0,0)` = the raw base clip — no authored file needed.
- The other **24 cells are authored videos** (4 analytical-edge + 4 dreamlike-edge
+ 16 combined core). This is **per base clip**; authoring load multiplies by the
number of base clips (and by language, since the HUD is baked — §8).
The grid is **data**, not code: a manifest the simulator reads (see §6).
---
## 5. Engine reconciliation (`player/alteration.py`)
The slice-1 engine modeled Left/Right as continuous runtime layers
(`AnalyticalOverlay.intensity`, `Restyle.blend`). Baked variants make that
obsolete. Changes:
- **`RenderPlan`** becomes `{ grade: ColorGrade, variant: VariantRef }`.
- `ColorGrade` — unchanged.
- `VariantRef(left, right)` — new; identity selection by coordinate.
- **Remove** `AnalyticalOverlay` and `Restyle` (their content is now baked into the
variant clip) rather than leaving dead fields.
- **`Calibration`** — a new frozen dataclass parameterizing the grade curve (e.g.
`mood_center`, per-axis curve), defaulting to **today's exact behavior**
(behavior-preserving). `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`.
- **`player/state.py`** — the `CROSSFADE` trigger switches from "restyle changed"
to "variant changed"; `LIVE_UPDATE` still covers a grade-only change. Video
on/off fades unchanged.
These are framework-code changes that keep the engine pure and unit-tested; they
do **not** bake any deployment-shape decision into the engine.
---
## 6. Data & endpoints
- **`simulator/clips.py`** (replaces `simulator/fixtures.py`): reads a base-clip +
variant manifest. Each base clip: `{ id, title, base_file, license, source,
variants: { "L,R": { file, model?, hud_lang? } } }`. Missing cells fall back to
the raw base (and are flagged in the readout as "raw / unauthored").
- **`POST /api/alteration`**: body `{ controls, calibration }``RenderPlan`
(plus the `ContentResolution` from `resolve_content`, so the dial's video-on/off
is honored). The endpoint calls the real `plan_alteration`.
- **`GET /api/clips`**: the base-clip list + variant manifest for the active base.
- Video files served as static assets from a sample-media directory.
- **Removed**: `POST /api/select`, `GET /api/catalog/meta`.
- `hef.selection` (the library, incl. `Coordinate`) is **untouched**; only the
simulator's selection view/endpoints retire. (`ranked_candidates`, added for the
old X-ray, may become unused by the simulator — noted, not removed here.)
---
## 7. Calibration tuning
The calibration panel exposes the grade-curve parameters (the two conventions in
tension: `(2,2,2,2)`-centered vs the `value/4` reading currently implemented, plus
curve shape). Changing them re-requests the plan and the footage responds live.
When the operator is happy, the chosen values are written into
`DEFAULT_CALIBRATION` in `player/alteration.py` and locked in by a unit test.
---
## 8. Baked HUD — the §4.3/§10 revision (accepted trade)
The design's §4.3/§10 kept the Left HUD a **runtime overlay** specifically so the
v2v substrate is language-agnostic and multilingual support is near-free. This
design **reverses that for the Left/Right plane**: the operator authors the HUD
**into** the variant videos to get pixel-precise control over the HUD and the
analytical↔feeling balance.
**Consequences (accepted):**
- Multilingual support is **no longer near-free**: baked HUD text means a new
language re-renders every label-bearing variant (the §10 "budget landmine").
The piece is English-first; broad i18n is deferred/expensive.
- The live Dark/Light grade is applied **on top** of the baked cell, so it **tints
the baked HUD** too (a "Dark" mood cools/darkens the HUD). With baked overlays
the grade cannot skip the HUD. Escape hatch: author HUD colors that survive
grading, or (future) reinstate a runtime HUD layer.
The parent design's §4.3/§10 must be updated to point at this revision (a task for
the implementation plan).
---
## 9. Bootstrapping before authored videos exist
The operator will author the 24 variants per base clip; none exist yet. So the
mechanism is testable immediately, the build includes a **placeholder generator**:
for at least one base clip, produce the 24 cells by burning the cell's `L,R` (and
a stub HUD caption) into the base loop via ffmpeg. Real authored clips drop into
the manifest with **no code change**.
---
## 10. Testing
- **`player/` unit tests:** `DEFAULT_CALIBRATION` is behavior-preserving vs the
current helpers; the two calibration conventions produce the expected plans;
`VariantRef` selection (incl. `(0,0)` → raw) and the `state.py` crossfade
trigger on variant change.
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
`POST /api/alteration` returns the engine's plan for given controls/calibration;
`GET /api/clips` returns the manifest; the removed endpoints are gone.
- **No browser/E2E automation** this slice (manual visual tuning is the point);
the rendering JS is kept thin and the engine logic stays in tested Python.
---
## 11. What ships
- `player/alteration.py` (parameterized `Calibration`, `VariantRef`, slimmed
`RenderPlan`) + `player/state.py` crossfade-trigger update.
- `simulator/clips.py` (variant manifest) replacing `fixtures.py`.
- `simulator/app.py` new endpoints; selection endpoints removed.
- `simulator/static/` rewritten as the Player preview (variant `<video>` +
crossfade, live grade, calibration panel, RenderPlan readout, content dial).
- Placeholder-variant generator + a sample base clip.
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten;
parent design §4.3/§10 pointer updated; `docs/ROADMAP.md` §3 updated.
---
## 12. Open questions (for the plan, not blockers)
- **Grade vs baked HUD interaction** — accepted that the grade tints the HUD;
revisit only if it reads badly once real authored clips exist.
- **Sample base clip** — pick one CC0 nature loop for the placeholder grid.
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for
tuning; the real timing engine is a later slice.
---
## 13. Out of scope (YAGNI)
Real generative v2v; serial / firmware; audio playback; the Pi renderer; the full
transition engine; multilingual tables; catalog-model changes; retiring
`hef.selection.ranked_candidates`. All remain later roadmap work.
View File
+98
View File
@@ -0,0 +1,98 @@
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
Replaces the 2026-06-04 nearest-match *selection* with a *transformation* of a
neutral base clip. Given the four experience knobs, it produces three layers
that compose per §4.2:
- Substrate transforms (alter pixels, blend with each other):
* Restyle -- the Right axis: a pre-baked generative v2v dreamlike restyle.
* ColorGrade -- the mood axis (Dark/Light): a deterministic color grade.
- Overlay (composited on top of the substrate):
* AnalyticalOverlay -- the Left axis: HUD/labels/measurement.
Left and Right are NOT opposites; they live on different layers and stack
(§4.2). Dark and Light are the two poles of one mood grade whose center is the
identity (§5).
Calibration note: the knob->strength curves below are the single source of
truth for how a 0..4 position maps to a transform strength. See the session
0006 transcript Deferred decisions for the §3-vs-§4.2/§5 reconciliation.
"""
from __future__ import annotations
from dataclasses import dataclass
from hef.selection import Coordinate
KNOB_MAX = 4 # knob full-scale (0..4)
@dataclass(frozen=True)
class ColorGrade:
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
<0 cool blue->black (dark), 0 = identity (raw, ungraded)."""
tone: float
@property
def is_identity(self) -> bool:
return self.tone == 0.0
@dataclass(frozen=True)
class AnalyticalOverlay:
"""Left axis (§4.1): analytical HUD/annotation/labels, composited on top.
`intensity` 0..1 (0 = no overlay)."""
intensity: float
@dataclass(frozen=True)
class Restyle:
"""Right axis (§4.1): pre-baked generative v2v dreamlike substrate.
`blend` 0..1 (0 = raw substrate, no restyle)."""
blend: float
@dataclass(frozen=True)
class RenderPlan:
"""The full layered alteration for one knob vector (§4.2)."""
grade: ColorGrade
overlay: AnalyticalOverlay
restyle: Restyle
@property
def is_identity(self) -> bool:
"""True when the plan leaves the neutral base un-altered."""
return (
self.grade.is_identity
and self.overlay.intensity == 0.0
and self.restyle.blend == 0.0
)
def _overlay_intensity(left: int) -> float:
"""Left knob -> analytical-overlay intensity (0..1)."""
return left / KNOB_MAX
def _restyle_blend(right: int) -> float:
"""Right knob -> v2v restyle blend (0..1)."""
return right / KNOB_MAX
def _mood_tone(dark: int, light: int) -> float:
"""(dark, light) -> signed mood grade in [-1, 1]; equal -> 0 identity (§5)."""
return (light - dark) / KNOB_MAX
def plan_alteration(coord: Coordinate) -> RenderPlan:
"""Map a knob vector to its layered RenderPlan (design §4)."""
return RenderPlan(
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light)),
overlay=AnalyticalOverlay(intensity=_overlay_intensity(coord.left)),
restyle=Restyle(blend=_restyle_blend(coord.right)),
)
+42
View File
@@ -0,0 +1,42 @@
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
The single source of truth for design §6's table: which audio source plays and
whether the projector shows video, for each dial position.
"""
from __future__ import annotations
from dataclasses import dataclass
# The four distinct audio sources. "white_noise" is generated at runtime;
# "music" is the public-domain classical pool; "audio_track" is the clip's own
# field audio; "none" is silence.
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
@dataclass(frozen=True)
class ContentResolution:
audio_source: str
video: bool
# Design §6, one row per dial position.
_TABLE = {
"off": ContentResolution("none", False),
"white_noise": ContentResolution("white_noise", False),
"music": ContentResolution("music", False),
"audio_track": ContentResolution("audio_track", False),
"video": ContentResolution("none", True),
"music_video": ContentResolution("music", True),
"audio_video": ContentResolution("audio_track", True),
}
def resolve_content(position: str) -> ContentResolution:
"""Map a content-dial position to its audio source and video flag (§6)."""
try:
return _TABLE[position]
except KeyError:
raise ValueError(
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
) from None
+68
View File
@@ -0,0 +1,68 @@
"""Control-panel state: the data shape read from the Arduino over serial.
This is the serial-contract payload shared with sub-project 4 (firmware). It
models the full panel from design §6/§7: the 7-way content dial, the four
experience knobs (0..4), and the two intensity levels (volume, brightness).
The wire framing itself is the separate 3<->4 serial contract; this module is
the *decoded* form.
"""
from __future__ import annotations
from dataclasses import dataclass, fields
# The seven positions of the content dial (design §6).
CONTENT_POSITIONS = frozenset(
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
)
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
KNOB_MIN = 0
KNOB_MAX = 4
class ControlsError(ValueError):
"""Raised when a Controls payload is structurally invalid."""
@dataclass(frozen=True)
class Controls:
content: str
left: int
right: int
dark: int
light: int
volume: int
brightness: int
def validate_controls(c: Controls) -> None:
"""Raise ControlsError if the payload is structurally invalid."""
if c.content not in CONTENT_POSITIONS:
raise ControlsError(
f"invalid content position {c.content!r}; "
f"expected one of {sorted(CONTENT_POSITIONS)}"
)
for name in KNOB_FIELDS:
value = getattr(c, name)
if isinstance(value, bool) or not isinstance(value, int):
raise ControlsError(f"knob {name} must be an int, got {value!r}")
if not (KNOB_MIN <= value <= KNOB_MAX):
raise ControlsError(
f"knob {name}={value} out of range {KNOB_MIN}..{KNOB_MAX}"
)
def parse_controls(data: dict) -> Controls:
"""Build a validated Controls from a decoded mapping, rejecting unknown or
missing keys."""
known = {f.name for f in fields(Controls)}
unknown = set(data) - known
if unknown:
raise ControlsError(f"unknown keys: {sorted(unknown)}")
missing = known - set(data)
if missing:
raise ControlsError(f"missing keys: {sorted(missing)}")
c = Controls(**data)
validate_controls(c)
return c
+120
View File
@@ -0,0 +1,120 @@
"""The player state machine: a stream of Controls -> playback transitions.
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
desired Playback (which neutral base clip, how it is altered, what audio plays,
at what levels) and returns the Transition from the previous Playback. The
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
pre-baked Right v2v variant needs a CROSSFADE, and toggling video on/off
fades to/from black.
Knobs no longer *select* a clip (the base library is neutral by construction);
they *transform* it. Which neutral base to show is an injected policy
(`choose_base`), defaulting to the first clip; richer rotation is a later slice.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Optional
from hef.selection import Coordinate
from player.alteration import RenderPlan, plan_alteration
from player.content import ContentResolution, resolve_content
from player.controls import Controls
class TransitionKind:
NONE = "none"
LIVE_UPDATE = "live_update"
CROSSFADE = "crossfade"
FADE_TO_BLACK = "fade_to_black"
FADE_FROM_BLACK = "fade_from_black"
@dataclass(frozen=True)
class Playback:
"""What should currently be playing."""
clip_id: Optional[str] # None = black walls
plan: Optional[RenderPlan] # None when black
content: ContentResolution
volume: int
brightness: int
@dataclass(frozen=True)
class Transition:
kind: str
playback: Playback
def _first(library):
if not library:
raise ValueError("no base clips available to play video")
return library[0]
_BLACK = Playback(
clip_id=None,
plan=None,
content=ContentResolution("none", False),
volume=0,
brightness=0,
)
class Player:
def __init__(
self,
base_library,
*,
choose_base: Callable = _first,
approved_only: bool = False,
):
self._library = list(base_library)
self._choose_base = choose_base
self._approved_only = approved_only # reserved for catalog-backed libs
self._current = _BLACK
def _resolve(self, controls: Controls) -> Playback:
content = resolve_content(controls.content)
if not content.video:
return Playback(
clip_id=None,
plan=None,
content=content,
volume=controls.volume,
brightness=controls.brightness,
)
clip = self._choose_base(self._library)
coord = Coordinate(controls.left, controls.right, controls.dark, controls.light)
return Playback(
clip_id=clip.id,
plan=plan_alteration(coord),
content=content,
volume=controls.volume,
brightness=controls.brightness,
)
def _classify(self, prev: Playback, nxt: Playback) -> str:
prev_video = prev.clip_id is not None
next_video = nxt.clip_id is not None
if prev == nxt:
return TransitionKind.NONE
if prev_video and not next_video:
return TransitionKind.FADE_TO_BLACK
if next_video and not prev_video:
return TransitionKind.FADE_FROM_BLACK
if next_video and prev_video:
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
return TransitionKind.CROSSFADE
return TransitionKind.LIVE_UPDATE
# both black: only audio/levels could have changed
return TransitionKind.LIVE_UPDATE
def update(self, controls: Controls) -> Transition:
nxt = self._resolve(controls)
kind = self._classify(self._current, nxt)
self._current = nxt
return Transition(kind=kind, playback=nxt)
+1 -1
View File
@@ -12,7 +12,7 @@ requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
packages = ["hef", "tools", "simulator"]
packages = ["hef", "tools", "simulator", "player"]
[project.scripts]
hef-ingest = "tools.ingest_cli:main"
@@ -0,0 +1,125 @@
# Session 0004.0 — Transcript
> App: human-experience-filter-art
> Type: coding
> Start: 2026-06-05T03-32 (PST) · End: 2026-06-05T03-56 (PST)
> Goal: Execute the experience-simulator (curator's X-ray) implementation plan.
> Outcome: **Shipped.** All 7 plan tasks built TDD-first and merged to `main`
> via PR #3. Suite: 133 passed, 2 skipped. Docker image builds + serves.
## Plan
Execute `docs/superpowers/plans/2026-06-04-experience-simulator.md` via
`superpowers:executing-plans`: build the `simulator/` package — a FastAPI service
wiring the five dials to the **real** `hef.selection` code and serving a
vanilla-JS "X-ray" (picked piece + ranked candidate pool with distances +
brain/mood 5×5 maps), fed by a deterministic synthetic fixture catalog since
`catalog/library.jsonl` is empty. The only `hef/` change is additive:
`hef.selection.ranked_candidates()` with `select()` refactored onto it. 7 tasks,
TDD throughout; Docker + compose + `make` targets package it for localhost.
## Pre-session state
- Clean `main`, synced with `origin/main` at `ddec45d`; 111 tests green, 2 skipped.
- Memory pointer: simulator spec + BDDs + plan produced in discovery 0003
(2026-06-04); next coding session runs `/executing-plans` on the simulator plan.
- Discovery session **0003** still `--INPROGRESS` (committed its artifacts to main
but never finalized) — dormant, not actively editing the tree.
- `fastapi` not yet installed; `python` not on PATH (use `.venv/bin/python`).
## Turn-by-turn arc
1. **Session gate → coding init.** Classified the `/executing-plans` launch as a
coding session; ran `wgl-session-coding-init`. Dry-run peek surfaced the
in-flight 0003 placeholder; autonomous default → noted it and proceeded.
Claimed session **0004**. Reconciled an initially-stale "ahead 2" read:
local `main` == `origin/main` == `ddec45d` (the 0003 spec/BDD/plan commits are
on origin). Verified clean baseline: **111 passed, 2 skipped**.
2. **Plan review.** Read the plan + `hef/selection.py`, `hef/catalog.py`,
`pyproject.toml`. Critical review: all assumed symbols exist
(`Coordinate`/`Weights`/`distance`/`candidates_for_mode`/`select`,
`CONTENT_MODES`/`SELECTOR_MODES`; `Record` carries every fixture field). No
blockers. Confirmed Docker present, `docs/USER_GUIDE.md` + `features/` exist.
3. **Workspace.** `using-git-worktrees` → native `EnterWorktree`
(`worktree-experience-simulator`, branched from `origin/main`). Fresh `.venv`
in the worktree; base package + pytest installed; baseline re-confirmed
**111 passed, 2 skipped**.
4. **Tasks 17, TDD (red → green → commit each):**
- **T1** `hef.selection.ranked_candidates()` + `select()` refactored onto it
(single source of truth; behavior-preserving) — 24 passed (new + existing
selection suite unchanged).
- **T2** declare `simulator` package + `[project.optional-dependencies] sim`
(fastapi/uvicorn/httpx); installed `-e ".[sim]"`; imports resolve.
- **T3** `simulator/fixtures.py` — deterministic 625-cell synthetic catalog
(all modes, mixed review status, no media) — 6 passed.
- **T4** `simulator/app.py` — FastAPI `create_app`, `POST /api/select`
(pick + ranked pool with distance/rank + coverage), `GET /api/catalog/meta`,
real-catalog-or-fixtures loader, guarded static mount — 6 passed.
- **T5** `simulator/static/` X-ray UI (dials, model knobs, ranked pool,
brain/mood maps) + static smoke test — 7 passed. Manual browser eyeball
substituted with live `curl` verification (no interactive browser): `/` 200,
`/api/catalog/meta` 625 records all-modes/mixed-status, `/api/select` ranked
pool ascending distances (1.0 → 1.41…), `none` → void, static assets 200.
- **T6** Dockerfile + compose + `Makefile` (literal-tab recipes). First build
**failed**: `package directory 'tools' does not exist``tools` is a
declared setuptools package but wasn't copied into the image. Root-caused
from the build log; fixed by adding `COPY tools ./tools`. Rebuilt → container
serves (`/` 200, 625-record fixture catalog). Torn down.
- **T7** USER_GUIDE "Playing with the simulator" section; **full suite: 133
passed, 2 skipped** (baseline 111 + 22 new).
5. **Finish branch.** `finishing-a-development-branch` (autonomous): pushed branch
as `experience-simulator`; created **PR #3** via the Gitea API
(`wgl-gitea-admin` token mechanism, `git.benstull.org`); merged it (merge
commit `618b691`); deleted the merged remote branch; FF'd local `main`;
re-verified the merged result **133 passed, 2 skipped**; removed the worktree
(all 7 commits confirmed ancestors of `main` first).
6. **Finalize.** Added `*.egg-info/` to `.gitignore` (editable-install artifact)
and pushed to `main` (`40dfbfd`); updated memory; published this transcript.
## Cut state (final)
- `main` @ `40dfbfd`, clean, synced with `origin/main`. No open PRs, no dangling
branches, worktree removed.
- **133 passed, 2 skipped.** Docker `make sim` builds and serves; `make sim-local`
runs uvicorn. Simulator is operator-runnable on localhost.
- The simulator is a **curator's discovery tool**, not a roadmap sub-project — the
roadmap frontier is unchanged.
## Deferred decisions
Autonomous-mode calls made without operator input:
1. **Dockerfile `COPY tools ./tools` (plan deviation).** The plan's Dockerfile
copied only `hef`/`simulator`/`catalog`, but `pyproject.toml` declares
`packages = ["hef", "tools", "simulator"]`, so the in-image editable install
failed. Added the missing COPY. *Alternative:* trim `tools` from the package
list — rejected (it's a real, used package). High confidence; the image now
matches the declared packages.
2. **Manual browser eyeball → programmatic curl (T5 Step 7).** No interactive
browser available, so verified the running server with `curl` (HTML, meta,
`/api/select`, void, static assets) instead of visually. Behavior fully
exercised, but no human visual confirmation of the rendered UI.
3. **Autonomous PR-merge instead of the interactive finish menu.** Per §6.5
autonomous default + §5.4 (branch→PR→merge), pushed + opened + merged PR #3
without surfacing the 4-option finish prompt. The preferred path was
unambiguous and the merge is revertible.
4. **`*.egg-info/` gitignore committed straight to `main`.** Trivial one-line
hygiene; landed directly rather than via PR.
## Next-session prompt
The simulator is shipped. The roadmap's next milestone is **sub-project 3
(Player Runtime / Pi)** — no spec/plan yet, so it starts with discovery.
```
/goal Develop sub-project 3 (Player Runtime / Pi): read the controls, call hef.selection.select(), play the chosen segment across the single panoramic projector, crossfade on change, dark on None — start with a discovery session to produce BDDs + spec, per docs/ROADMAP.md §3
```
Read first: memory `experience-simulator-ready-to-build.md` (what shipped + the
Dockerfile-copies-every-package gotcha), `pano-projector-nature-video.md` (single
pano projector, sub-project 5 dropped), and `docs/ROADMAP.md §3`. Open decisions
to settle: player stack (mpv/ffmpeg/custom), `approved`-only enforcement, and the
serial protocol framing shared with sub-project 4.
> Housekeeping: discovery session **0003** is still `--INPROGRESS` in
> `sessions/0003/` (artifacts long since merged) — finalize or close it out.
@@ -1,20 +0,0 @@
# Session 0004.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-05T03-32 (PST)
> Type: coding
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
>
> This file reserves session ID 0004 for human-experience-filter-art. The driver replaces this
> body with the full transcript and renames the file to its final
> SESSION-0004.0-TRANSCRIPT-2026-06-05T03-32--<end>.md form at session end.
## Launch prompt
_(launch prompt not captured at claim time)_
## Deferred decisions
_Autonomous-mode low-confidence calls the driver made and would have
liked operator input on. Appended as the session runs; surfaced at
finalize. Empty if none._
@@ -0,0 +1,126 @@
# Session 0005.0 — Transcript
> App: human-experience-filter-art
> Type: discovery
> Start: 2026-06-05T08-30 (PST) · End: 2026-06-05T17-48 (PST)
> Goal: Explore whether AI-generated/altered video is a viable content path for the
> pano projector — economics + fit with the design — and capture the outcome.
> Outcome: **Artistic thesis pivoted** and a full design-spec revision written,
> reviewed, and **merged to `main` via PR #4** (merge commit `77746e4`).
> Implementation deferred to a sub-project-3 coding session.
## Plan
Discovery session opened from the operator's question: "can we write/generate
scene descriptions and generate the video with AI — would the economics work?"
The session ran as a brainstorming exploration that converged on a new thesis and
produced a design spec as its artifact.
## Launch prompt
```
I'm not sure if hte economics would work out (can you do a best effort calculation?) but maybe we could write up/generate scene descriptions and generate the video with AI. What do you think?
```
## Pre-session state
- `main` @ `3510236`. Sub-projects 1 & 2 done/merged; roadmap frontier = sub-project
3 (Pi player). A stale, already-merged worktree (`feature/BuildSimulator11`,
same commit as main) was the session's launch cwd.
- Design of record: `2026-06-04-human-experience-filter-design.md` — single pano
projector + **found** nature video; §1 constraint "real found media, **not
algorithmically generated filler. Curation is the artwork.**"
## Turn-by-turn arc
1. **Gate / classify.** Operator's prompt was ambiguous between discovery and a
quick chat; asked → operator chose **Discovery (tracked)**. Ran
`wgl-session-discovery-init`.
2. **Claim snag (kept in the record).** `resolve-app.py` reported **ambiguous**
the nested worktree's `app.json` (`.worktrees/feature-BuildSimulator11/app.json`)
was discovered alongside the canonical main-repo `app.json`, both listing the
repo. Worked around with `WGL_APP_SCAN_DEPTH=3` (excludes the depth-5 worktree
copy). Claimed session **0005**, type discovery, at `850f67a`.
3. **Economics, v1 (the original question).** Researched 2026 AI-video API rates
(Sora 2 ~$0.10/s, Kling 3.0 ~$0.10/s, Veo 3.1 Lite/Runway Turbo ~$0.05/s, Veo 3
+audio ~$0.75/s). At the spec's scale (120800 pieces × ~10-min segments) →
**~$15k45k+** with retries; **not viable**, and it doesn't reduce the real
constraint (curation labor). Also flagged the §1 thesis conflict ("not generated
filler").
4. **Thesis pivot.** Operator: willing to change the thesis — the piece is "about
**how humans interact with machines and how it impacts their nervous systems**."
Under that thesis AI alteration *is* the subject. Invoked `brainstorming`.
Scope = **re-skin the meaning** (keep apparatus); content = **AI-altered
primary**.
5. **Economics, v2 (the flip).** New thesis unlocks two cost levers: small
**generate-to-order** library + short **seamless loops****~$3003k**. Feasible.
6. **The strong concept crystallizes.** Operator: start from **neutral** real
nature footage and let the machine **alter it toward the knob state** — "a great
expression of how machines alter our experience, good and bad." Established the
per-axis mapping (Left = analytical overlay; Right = generative v2v dissolve;
Dark/Light = color grade), the **substrate-vs-overlay composition rule**, and the
"whole-brain" corner (feeling, *labeled*).
7. **Control panel + i18n.** Operator added a 7-way content dial (Off/White
Noise/Music/Audio Track/Video/Music+Video/Audio Track+Video), volume + brightness
knobs, and a tactile **wooden** panel (engraved symbols, braille, color LEDs,
read-aloud button). Designed touch-distinct/engravable symbols. Key finding on
the **translation cost**: because the Left **labels are a runtime overlay**, the
v2v renders once (language-agnostic) and supporting *every* language is ~free on
the video — only cheap text translation + TTS. (Baking labels in would multiply
video cost × #languages — the avoided landmine.)
8. **Mood color grade.** Operator: Light = yellow / white negative space; center =
raw ungraded video; Dark = blue / black negative space. (First draft mis-set
Dark=red and center=grey; operator corrected → Dark = melancholy **blue** not
alarm red; center = **raw video**, no grade.)
9. **Wrote the spec.** Branched `feature/machine-altered-perception-spec` off main,
wrote `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md`
(15 sections), committed, self-reviewed; operator reviewed and gave 4
corrections (center=raw, add Music+Video → 7-way, audio-alteration deferred,
red gone) → applied, commit amended to `c5952c6`.
10. **Wrap.** Operator chose to wrap the discovery session here (spec is the
artifact; build later). Pushed branch; first PR-creation attempt was **blocked
by the auto-mode classifier** (treated as routing around PR authorization) —
surfaced it rather than working around. Operator then explicitly authorized
opening + merging the PR → **PR #4 opened and merged to `main`** (merge commit
`77746e4`, branch deleted), local `main` fast-forwarded.
## Cut state (end of session)
| Item | State |
|---|---|
| Design spec | **Merged to `main`**`docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` |
| Branch commit | `c5952c6` (spec, amended with review fixes) |
| Merge commit | `77746e4` (PR #4), feature branch deleted |
| `main` | fast-forwarded to `77746e4` (local + origin) |
| Roadmap | unchanged on disk; spec §13 describes how sub-projects 2/3/4 + a new offline v2v pipeline are reshaped (a roadmap-edit pass is future work) |
| Memory | `ai-generated-content-thesis-pivot.md` (full design rationale + status) + `MEMORY.md` index updated |
## What lands on the operator's plate
- Nothing blocking. The spec is canonical and merged.
- Optional future passes the spec itself lists (§14): pano resolution strategy,
negative-space luma key, transform composition order, neutral-base count,
language tier, runtime-grading perf on the Pi.
- A roadmap-edit pass to reflect the reshaped sub-projects 2/3/4 (the design spec
carries it for now).
## Deferred decisions
No substantive low-confidence design calls — every design decision (thesis,
per-axis mapping, composition rule, color grade, the 7-way dial, the panel) was
operator-confirmed in-session. Two mechanical judgment calls worth flagging:
- **`WGL_APP_SCAN_DEPTH=3` workaround** for the nested-worktree `app.json`
ambiguity in `resolve-app.py`. This is a genuine plugin friction (a worktree
nested under the main repo makes the resolver ambiguous) — a candidate for
`wgl-dev-plugin-feedback` (resolver should skip `.worktrees/`).
- **Repurposed the stale `feature/BuildSimulator11` worktree** to a fresh
`feature/machine-altered-perception-spec` branch off its HEAD (= main) rather
than removing it (harness had cwd anchored there; removal risked breaking the
shell).
## Prompt the operator can paste into the next session
```
/goal Implement sub-project 3 (the player / alteration engine) per docs/ROADMAP.md and the new docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md. Read memory ai-generated-content-thesis-pivot.md first. Build the alteration engine: runtime luma-keyed mood grade (Light=yellow/white → raw → Dark=blue/black), runtime analytical-overlay compositor (multilingual, from string tables), generated white noise, and selection/blending of pre-baked v2v variants over a small neutral base library; resolve the spec §14 open questions (pano resolution, composition order) as you go.
```
@@ -0,0 +1,134 @@
# Session 0006.0 — Transcript
> App: human-experience-filter-art
> Type: coding
> Start: 2026-06-05T17-52 (PST) · End: 2026-06-05T18-15 (PST)
> Goal: `/goal next` — drive the roadmap to its next frontier item (sub-project 3,
> the player / machine-alteration engine), and build it.
> Outcome: **Sub-project 3 slice 1 — the pure-logic alteration-engine + player
> core — built and MERGED to `main` via PR #5** (merge commit `aeeed66`).
> New `player/` package; 56 new tests (189 passed / 2 skipped total).
> Sub-project 3 advanced from "next" to "in progress."
## Plan
Begin sub-project 3 (Player Runtime + alteration engine) per `docs/ROADMAP.md` §3
and the approved machine-altered-perception design (§4 alteration engine, §5 mood
grade, §6 7-way content dial, §7 intensity). Sub-project 3 is large (runtime
grading + overlay compositing + v2v variant selection + serial input + mpv/GPU +
white-noise DSP); this session ships **slice 1 — the pure-logic core** as a new
`player/` package, fully unit-tested with all I/O behind injected interfaces,
mirroring how sub-project 1 was the pure-logic dependency root. Process:
writing-plans → TDD → branch → PR → merge.
## Launch prompt
```
/goal next — drive the roadmap to its next frontier item.
Per project memory (session 0005, machine-altered-perception design SPEC merged
to main via PR #4), the roadmap frontier is sub-project 3: the player /
machine-alteration engine. This coding session builds that.
```
## Pre-session state
- `main` @ `77746e4` locally (behind `origin/main` by 2 — session 0005's finalized
transcript + this session's claim placeholder); fast-forwarded to `origin/main`
during init.
- Sub-projects 1 & 2 done/merged; design of record = the machine-altered-perception
revision (PR #4). No `player/` package and no sub-project-3 plan existed yet.
- Launch cwd was the stale, already-merged worktree
`.worktrees/feature-BuildSimulator11/` (branch `feature/machine-altered-perception-spec`,
remote gone) — same situation as session 0005.
## Turn-by-turn arc
1. **Gate / classify.** `/goal next` → unambiguously a coding session (roadmap
frontier = sub-project 3 build). Ran `wgl-session-coding-init`.
2. **Claim snag (same as 0005, kept in the record).** `resolve-app.py` reported
**ambiguous** — the nested worktree's `app.json` discovered alongside the main
clone's. Read the resolver to confirm root cause (`find_app_jsons` prunes
`.git/node_modules/…` but not `.worktrees/`). Worked around with
`WGL_APP_SCAN_DEPTH=4` (prunes the depth-5 worktree copy, keeps the depth-3 main
one). Claimed session **0006**, type coding, at `e694489`. Filed a plugin-feedback
note for the recurring gap (`feedback/2026-06-05T18-13-resolver-nested-worktree-ambiguous.md`).
3. **Baseline + branch.** Fast-forwarded the main clone; confirmed PR #4 merged;
created `feature/player-alteration-core` off `origin/main` in the worktree.
4. **Read the design.** Read ROADMAP §3 + the machine-altered-perception spec
(§4 alteration engine, §5 mood grade, §6 7-way dial, §7 intensity), and the
existing `hef/` conventions (frozen dataclasses, pure modules, heavy unit tests).
5. **Scoped the session → slice 1.** Sub-project 3 is too large for one session;
chose the pure-logic core (controls model, content-dial resolver, alteration
render-plan, player state machine) with all I/O behind injected interfaces, and
deferred the renderer/serial/white-noise/v2v/catalog slices (named in the plan).
6. **Surfaced a spec ambiguity (Deferred decision).** §3 "(2,2,2,2) neutral" vs
§4.2 (Left/Right stack as independent layers) vs §5 (mood center = identity)
cannot all be literally true for four independent knobs. Resolved with a
swappable calibration: brain `value/4`, mood `(lightdark)/4`; flagged for
operator confirm.
7. **writing-plans.** Wrote `docs/superpowers/plans/2026-06-05-player-alteration-core.md`
(6 TDD tasks, full code + tests, self-review against the spec).
8. **TDD implementation.** Red→green→commit per module: `player/controls.py`
(Controls, validation, parse), `player/content.py` (§6 7-way table),
`player/alteration.py` (RenderPlan: ColorGrade/AnalyticalOverlay/Restyle +
calibration helpers), `player/state.py` (Player state machine + Transition
kinds). Registered the package in `pyproject.toml`; updated ROADMAP §3.
9. **Verify.** Full suite green: **189 passed, 2 skipped** (the opt-in ffprobe
tests), 56 of them new player tests. No regressions.
10. **Ship.** Pushed the branch; created **PR #5** and merged it via the Gitea API
helper (merge commit `aeeed66`); synced the main clone; deleted the remote
feature branch.
11. **Wrap.** Filed plugin feedback, updated memory, finalized.
## Cut state
- `main` @ `aeeed66` (PR #5 merged). `player/` package present: `controls.py`,
`content.py`, `alteration.py`, `state.py` + 4 test files. Full suite 189 passed /
2 skipped.
- Sub-project 3 = **⏳ in progress**. Slice 1 done; remaining slices listed in
ROADMAP §3 and `[[sub-project-3-player-progress]]`.
- Working tree clean. Remote feature branch deleted; the local worktree remains on
the now-merged `feature/player-alteration-core` branch (harmless).
## Deferred decisions (operator plate)
Two low-confidence calls made autonomously this session — both flagged for operator
confirmation:
1. **Knob→strength calibration (§3 vs §4.2/§5 conflict).** §3 says "(2,2,2,2) is
neutral," but §4.2 (Left/Right stack as independent layers: overlay vs substrate)
and §5 (mood center = identity) conflict with that for four independent physical
knobs. Implemented: brain knobs `strength = value/4` (0 = off, 4 = max — full
range, honors §4.2 stacking), mood `tone = (light dark)/4` (equal dark/light →
identity, honors §5). Under this the un-altered base is `(0,0,·,·)` with
`dark==light`, **not** `(2,2,2,2)` — read as a vestige of the old coordinate-grid
*center*. The calibration is three one-line helpers in `player/alteration.py`;
flipping to a "centered at 2 = no push" convention is a localized change that
leaves the RenderPlan shape and all downstream logic untouched. **Recommend
operator confirm the intended panel UX.**
2. **Volume/Brightness granularity.** Modeled as `0..4` ints for uniformity with the
experience knobs and the Arduino's `0..4` quantization (roadmap §4). The spec
calls them "levels" without fixing granularity; revisit if the panel warrants
finer steps.
## Next-session prompt
The roadmap frontier is now the remaining sub-project-3 slices. Read
`[[sub-project-3-player-progress]]` and the slice-1 plan first. Two strong
candidates for the next slice — the **runtime renderer** (makes the core visible)
or the **serial framing contract + input adapter** (unblocks sub-project 4 in
parallel):
```
/goal Sub-project 3 slice 2 — settle the 3⇄4 serial framing contract and build the
USB-serial / keyboard input adapter that feeds the player a `Controls` stream
(unblocks sub-project 4), per docs/ROADMAP.md §3 and docs/superpowers/plans/2026-06-05-player-alteration-core.md.
Alternatively start the runtime renderer (mpv/ffmpeg + grade/overlay). First
confirm the open knob→strength calibration decision (§3 vs §4.2/§5).
```
Gotcha to carry forward: running from the nested `.worktrees/…` checkout breaks
`resolve-app.py` (ambiguous app.json) — run from the main clone, or use
`WGL_APP_SCAN_DEPTH=4` on claim and explicit `--sessions-*` flags on publish, until
the plugin fix lands.
+6
View File
@@ -10,5 +10,11 @@
},
"0004": {
"title": ""
},
"0005": {
"title": ""
},
"0006": {
"title": ""
}
}
+80
View File
@@ -0,0 +1,80 @@
import pytest
from hef.selection import Coordinate
from player.alteration import (
AnalyticalOverlay,
ColorGrade,
RenderPlan,
Restyle,
plan_alteration,
)
def _coord(left=0, right=0, dark=0, light=0):
return Coordinate(left=left, right=right, dark=dark, light=light)
def test_all_zero_knobs_is_the_unaltered_base():
plan = plan_alteration(_coord())
assert plan.is_identity
assert plan.overlay.intensity == 0.0
assert plan.restyle.blend == 0.0
assert plan.grade.tone == 0.0
assert plan.grade.is_identity
def test_left_drives_the_analytical_overlay_only():
plan = plan_alteration(_coord(left=4))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 0.0 # Left does not touch the substrate
assert plan.grade.tone == 0.0
def test_right_drives_the_restyle_substrate_only():
plan = plan_alteration(_coord(right=2))
assert plan.restyle.blend == 0.5
assert plan.overlay.intensity == 0.0 # Right does not add overlay
def test_left_and_right_stack_not_cancel():
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
plan = plan_alteration(_coord(left=4, right=4))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 1.0
def test_light_pole_grades_warm_positive_tone():
plan = plan_alteration(_coord(light=4))
assert plan.grade.tone == 1.0
assert not plan.grade.is_identity
def test_dark_pole_grades_cool_negative_tone():
plan = plan_alteration(_coord(dark=4))
assert plan.grade.tone == -1.0
def test_equal_dark_and_light_is_identity_grade():
# design §5: the mood center is the raw, ungraded footage
assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity
assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity
def test_dark_minus_light_sets_intermediate_tone():
assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5)
assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5)
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
# design §4.2 "Dark + analytical": cold measurement over a melancholy scene
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 0.5
assert plan.grade.tone == -1.0
assert not plan.is_identity
def test_render_plan_is_frozen():
plan = plan_alteration(_coord())
with pytest.raises(Exception):
plan.grade.tone = 0.5 # type: ignore[misc]
+34
View File
@@ -0,0 +1,34 @@
import pytest
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
def test_audio_sources_are_the_four_distinct_sources():
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
# The §6 table, row by row: position -> (audio_source, video).
@pytest.mark.parametrize(
"position,audio_source,video",
[
("off", "none", False),
("white_noise", "white_noise", False),
("music", "music", False),
("audio_track", "audio_track", False),
("video", "none", True),
("music_video", "music", True),
("audio_video", "audio_track", True),
],
)
def test_resolve_content_matches_spec_table(position, audio_source, video):
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
def test_off_is_void_state_black_and_silent():
r = resolve_content("off")
assert r.video is False and r.audio_source == "none"
def test_resolve_content_rejects_unknown_position():
with pytest.raises(ValueError):
resolve_content("bogus")
+53
View File
@@ -0,0 +1,53 @@
import pytest
from player.controls import (
Controls,
CONTENT_POSITIONS,
ControlsError,
validate_controls,
parse_controls,
)
def test_content_positions_are_the_seven_from_the_spec():
assert CONTENT_POSITIONS == frozenset(
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
)
def test_valid_controls_pass_validation():
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
validate_controls(c) # does not raise
def test_invalid_content_position_rejected():
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
with pytest.raises(ControlsError):
validate_controls(c)
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
@pytest.mark.parametrize("bad", [-1, 5, True])
def test_out_of_range_or_non_int_knob_rejected(field, bad):
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
kwargs[field] = bad
with pytest.raises(ControlsError):
validate_controls(Controls(**kwargs))
def test_parse_controls_from_mapping():
c = parse_controls(
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
)
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
def test_parse_controls_rejects_unknown_keys():
with pytest.raises(ControlsError):
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
def test_parse_controls_rejects_missing_keys():
with pytest.raises(ControlsError):
parse_controls({"content": "off", "left": 0})
+112
View File
@@ -0,0 +1,112 @@
from dataclasses import dataclass
import pytest
from player.controls import Controls
from player.state import Player, Playback, Transition, TransitionKind
@dataclass(frozen=True)
class FakeClip:
id: str
LIB = [FakeClip("base-a"), FakeClip("base-b")]
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
return Controls(content, left, right, dark, light, volume, brightness)
def test_first_update_to_video_fades_in_from_black():
p = Player(LIB)
t = p.update(_controls(content="video"))
assert t.kind == TransitionKind.FADE_FROM_BLACK
assert t.playback.clip_id == "base-a"
assert t.playback.content.video is True
def test_off_from_video_fades_to_black_and_silences():
p = Player(LIB)
p.update(_controls(content="video"))
t = p.update(_controls(content="off"))
assert t.kind == TransitionKind.FADE_TO_BLACK
assert t.playback.clip_id is None
assert t.playback.content.audio_source == "none"
def test_no_change_yields_none_transition():
p = Player(LIB)
p.update(_controls(content="video", left=1))
t = p.update(_controls(content="video", left=1))
assert t.kind == TransitionKind.NONE
def test_grade_change_is_a_live_update_not_a_crossfade():
# design §4.3: the Dark/Light grade is a continuous runtime op
p = Player(LIB)
p.update(_controls(content="video", dark=0, light=0))
t = p.update(_controls(content="video", dark=4, light=0))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.grade.tone == -1.0
def test_overlay_change_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="video", left=0))
t = p.update(_controls(content="video", left=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.overlay.intensity == 1.0
def test_restyle_change_crossfades_the_substrate():
# design §4.3: the Right v2v substrate is a pre-baked variant swap
p = Player(LIB)
p.update(_controls(content="video", right=0))
t = p.update(_controls(content="video", right=4))
assert t.kind == TransitionKind.CROSSFADE
assert t.playback.plan.restyle.blend == 1.0
def test_volume_only_change_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="video", volume=1))
t = p.update(_controls(content="video", volume=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.volume == 4
def test_audio_source_change_while_black_is_a_live_update():
p = Player(LIB)
p.update(_controls(content="white_noise"))
t = p.update(_controls(content="music"))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.content.audio_source == "music"
def test_injected_base_chooser_is_used():
p = Player(LIB, choose_base=lambda lib: lib[1])
t = p.update(_controls(content="video"))
assert t.playback.clip_id == "base-b"
def test_empty_library_with_video_raises():
p = Player([])
with pytest.raises(ValueError):
p.update(_controls(content="video"))
def test_off_with_empty_library_is_fine():
p = Player([])
# levels at 0 match the initial black state, so this is a no-op transition
t = p.update(_controls(content="off", volume=0, brightness=0))
assert t.kind == TransitionKind.NONE # already black at init
assert t.playback.clip_id is None
def test_off_with_levels_from_black_is_a_live_update():
p = Player([])
t = p.update(_controls(content="off", volume=3, brightness=2))
assert t.kind == TransitionKind.LIVE_UPDATE # black->black, levels set
assert t.playback.clip_id is None
assert t.playback.volume == 3