Compare commits
29 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 3ef21fb7f7 | |||
| 5290785e2a | |||
| aeeed66f49 | |||
| 5403021e4a | |||
| 86b52ab8af | |||
| 9a13b01a41 | |||
| 79bb86c6a2 | |||
| 00969e2e44 | |||
| 57597be8af | |||
| 70834ae0ad | |||
| e69448922c | |||
| d8bb9661d4 | |||
| 77746e43c6 | |||
| c5952c6cfd | |||
| 850f67abea | |||
| 3510236a2a | |||
| 05c1f524da | |||
| 40dfbfd3db | |||
| 618b69182c | |||
| 859f868d3b | |||
| 80cde37b98 | |||
| c1745f0021 | |||
| cdf36c9b57 | |||
| aadfccf32d | |||
| a260b68d2b | |||
| 98fb86840c | |||
| 7f2f585600 | |||
| ddec45d39c | |||
| 9cb4fb62d8 |
@@ -3,3 +3,5 @@ __pycache__/
|
||||
.pytest_cache/
|
||||
.venv/
|
||||
media/
|
||||
.superpowers/
|
||||
*.egg-info/
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
.PHONY: sim sim-local
|
||||
|
||||
sim:
|
||||
docker compose -f simulator/docker-compose.yml up --build
|
||||
|
||||
sim-local:
|
||||
python -m uvicorn simulator.app:app --reload --port 8000
|
||||
+50
-19
@@ -15,7 +15,7 @@ Design reference: [`specs/2026-06-04-human-experience-filter-design.md`](./super
|
||||
|---|---------------------------------|---------------|----------|
|
||||
| 1 | Catalog & Selection Core | ✅ Done | everything |
|
||||
| 2 | Ingest & Tagging / Review tools | ✅ Done | a real library |
|
||||
| 3 | Player Runtime (Pi) | ⏳ Next | the room runs |
|
||||
| 3 | Player Runtime (Pi) | ⏳ In progress | the room runs |
|
||||
| 4 | Arduino Firmware (control panel)| ◻ Not started | real knobs |
|
||||
| 5 | ~~Procedural Side Walls~~ | ❌ Dropped | — (superseded) |
|
||||
|
||||
@@ -95,29 +95,60 @@ was `validate_catalog` + `index_by_id`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Player Runtime (Pi) ⏳ (next)
|
||||
## 3. Player Runtime (Pi) ⏳ (in progress)
|
||||
|
||||
**Goal:** the thing that makes the room run — read the controls, pick media, play
|
||||
it across the single panoramic projector.
|
||||
**Goal:** the thing that makes the room run — read the controls, **alter** the
|
||||
neutral base footage toward the knob state, and play it across the single
|
||||
panoramic projector. Per the machine-altered-perception revision the knobs no
|
||||
longer *select* a pre-tagged clip; they drive an **alteration engine** (design
|
||||
§4/§5) over a small neutral base library.
|
||||
|
||||
**Delivers (`player/`):**
|
||||
**Slice 1 — alteration engine + player core (pure logic) ✅ Done.** Merged to
|
||||
`main` (session 0006); 56 tests. The new `player/` package, with all I/O
|
||||
(video/audio/serial) behind injected interfaces:
|
||||
|
||||
- Read the five control values from the Arduino over USB serial.
|
||||
- Call `hef.selection.select()` on each change; loop the chosen 5–15 min segment
|
||||
until the knobs move; **crossfade** on change.
|
||||
- `None` mode → fade to black + silence.
|
||||
- Drive the **single panoramic projector** spanning the three walls with the real
|
||||
selected video (no primary/side split; no side-wall feed — sub-project 5 dropped).
|
||||
- `player/controls.py` — `Controls`, the full panel state (7-way content dial +
|
||||
4 experience knobs + volume + brightness); the serial-contract data shape
|
||||
shared with sub-project 4.
|
||||
- `player/content.py` — `resolve_content`, the §6 7-way dial → (audio source,
|
||||
video on/off) table.
|
||||
- `player/alteration.py` — `plan_alteration`, knob vector → `RenderPlan`
|
||||
(mood `ColorGrade` with center=identity §5; Left `AnalyticalOverlay`; Right
|
||||
`Restyle`), encoding the §4.2 layering rule (substrate vs overlay; Left & Right
|
||||
stack).
|
||||
- `player/state.py` — `Player`, the state machine that diffs successive controls
|
||||
into transitions (`FADE_TO_BLACK`/`FADE_FROM_BLACK` on video on/off,
|
||||
`CROSSFADE` on clip/restyle-variant swap, `LIVE_UPDATE` for the continuous
|
||||
grade/overlay/level ops per §4.3, `NONE` when unchanged).
|
||||
|
||||
**Depends on:** sub-project 1 (`select`); a populated catalog from sub-project 2
|
||||
to be meaningful, but can be developed against a hand-authored catalog and a
|
||||
**keyboard/serial stand-in** before firmware exists.
|
||||
**Done when:** given a catalog and a stream of control values, it plays the
|
||||
correct segment, loops, crossfades, and goes dark on `None` — testable with the
|
||||
serial input mocked.
|
||||
**Hardware:** Raspberry Pi 5 (holds the drive + catalog).
|
||||
Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-player-alteration-core.md).
|
||||
|
||||
**Remaining slices (not started):**
|
||||
|
||||
- **Runtime renderer** — drive the single panoramic projector via mpv/ffmpeg;
|
||||
GPU shaders for the luma-keyed mood grade + analytical-overlay compositing;
|
||||
realize crossfade/fade timing and the 5–15 min loop. (Open decision: player
|
||||
stack — mpv via IPC vs. ffmpeg vs. custom.)
|
||||
- **Serial input** — real USB-serial reader on the Pi + the **3⇄4 framing
|
||||
contract** with the firmware (the keyboard/serial stand-in already works via a
|
||||
`Controls` stream).
|
||||
- **White-noise generation** — runtime white/pink noise for that content position.
|
||||
- **Offline v2v variant pipeline** — author the pre-baked Right restyle variants
|
||||
(the only paid AI step, design §9) + the multilingual label/string tables + TTS.
|
||||
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
|
||||
variant" flag (sub-project 2 territory, design §13).
|
||||
|
||||
**Depends on:** sub-project 1 (`Coordinate`); a neutral base library from
|
||||
sub-project 2 to be meaningful, but developable against a hand-authored library
|
||||
and a **keyboard/serial stand-in** before firmware exists.
|
||||
**Done when:** given a base library and a stream of control values, it plays the
|
||||
correct altered segment, loops, crossfades, and goes dark on `Off` — the
|
||||
decision logic is covered (slice 1, serial mocked); the renderer realizes it.
|
||||
**Hardware:** Raspberry Pi 5 (holds the drive + base library + variants).
|
||||
**Open decisions:** player stack (mpv via IPC vs. ffmpeg vs. custom); whether the
|
||||
player hard-restricts to `approved` records.
|
||||
player hard-restricts to `approved` records (the `approved_only` flag is plumbed
|
||||
but inert until backed by a real catalog); knob→strength calibration (the §3 vs
|
||||
§4.2/§5 reconciliation — see the slice-1 plan / session 0006 transcript).
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -286,3 +286,29 @@ Records point at files via `file_path`. Those files live on the player's drive,
|
||||
**not** in this git repo (the repo holds only the catalog metadata). Keep your
|
||||
`file_path` values consistent with wherever you mount that drive on the machine
|
||||
that will eventually run the room.
|
||||
|
||||
## Playing with the simulator (curator's X-ray)
|
||||
|
||||
The simulator is a web stand-in for the installation's control panel. It runs the
|
||||
real `hef.selection` code against a synthetic catalog so you can feel whether the
|
||||
dials surface fitting pieces before any hardware exists.
|
||||
|
||||
**Run it (Docker):**
|
||||
|
||||
make sim
|
||||
|
||||
then open http://localhost:8000.
|
||||
|
||||
**Run it (no Docker):**
|
||||
|
||||
pip install -e ".[sim]"
|
||||
make sim-local
|
||||
|
||||
**What you see:** the five real dials (mode + Left/Right/Dark/Light), the model
|
||||
knobs (brain/mood weights, pool size, approved-only), and the X-ray — the picked
|
||||
piece, the ranked candidate pool with distances, and brain/mood coordinate maps
|
||||
showing where your knob point and the candidates sit.
|
||||
|
||||
By default it loads a generated fixture catalog. To point it at a real catalog,
|
||||
set `HEF_SIM_CATALOG=catalog/library.jsonl` (used automatically when that file is
|
||||
non-empty).
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,867 @@
|
||||
# Player Alteration Core Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build the pure-logic core of sub-project 3 (the Player Runtime + alteration engine) — a new `player/` package that turns control-panel state into a render plan and playback transitions, fully unit-tested with all I/O (video/audio/serial) behind injected interfaces.
|
||||
|
||||
**Architecture:** Four small, pure modules mirroring `hef/` conventions (frozen dataclasses, `from __future__ import annotations`, string-constant frozensets, thorough unit tests, no heavy deps). `controls` models the panel state (the serial-contract data shape shared with sub-project 4); `content` resolves the 7-way content dial; `alteration` is the heart — knob vector → `RenderPlan` per design §4/§5; `state` is the player state machine that diffs successive controls into playback transitions (crossfade / fade-to-black / live-update). No mpv, no GPU, no real serial, no white-noise DSP, no v2v pipeline — those are later integration slices.
|
||||
|
||||
**Tech Stack:** Python 3.11+, stdlib only (`dataclasses`, `math`), pytest. Reuses `hef.selection.Coordinate` for the knob vector.
|
||||
|
||||
**Design reference:** `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` (§4 alteration engine, §5 mood grade, §6 7-way content dial, §7 intensity) and `docs/ROADMAP.md` §3 (Player Runtime "done when").
|
||||
|
||||
**Calibration decision (flagged for operator):** brain knobs use `strength = value/4` (0=off, 4=max); mood uses `tone = (light−dark)/4` (equal dark/light = identity per §5). This reads §3's "(2,2,2,2) neutral" as a vestige of the old coordinate-grid center. The calibration lives in three one-line helpers so the convention can be flipped trivially. See the session 0006 transcript's Deferred decisions.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- `player/__init__.py` — package marker (empty).
|
||||
- `player/controls.py` — `Controls` frozen dataclass (content position + 4 experience knobs + volume + brightness), `CONTENT_POSITIONS`, `validate_controls`, `parse_controls` (from a decoded mapping; wire framing is the separate 3⇄4 serial contract).
|
||||
- `player/content.py` — `ContentResolution` (audio source + video flag), `AUDIO_SOURCES`, `resolve_content` (the §6 7-row table, single source of truth).
|
||||
- `player/alteration.py` — `ColorGrade`, `AnalyticalOverlay`, `Restyle`, `RenderPlan`, `plan_alteration(coord)`, and the calibration helpers `_overlay_intensity`, `_restyle_blend`, `_mood_tone`.
|
||||
- `player/state.py` — `Playback`, `TransitionKind`, `Transition`, `Player` (consumes a stream of `Controls`, emits a `Transition` per update; output sink is injected/duck-typed; base-clip choice is an injected callable).
|
||||
- `pyproject.toml` — add `"player"` to `[tool.setuptools] packages`.
|
||||
|
||||
Tests (one per module): `tests/test_player_controls.py`, `tests/test_player_content.py`, `tests/test_player_alteration.py`, `tests/test_player_state.py`.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Controls model (the serial-contract data shape)
|
||||
|
||||
**Files:**
|
||||
- Create: `player/__init__.py`
|
||||
- Create: `player/controls.py`
|
||||
- Test: `tests/test_player_controls.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/test_player_controls.py
|
||||
import pytest
|
||||
|
||||
from player.controls import (
|
||||
Controls,
|
||||
CONTENT_POSITIONS,
|
||||
ControlsError,
|
||||
validate_controls,
|
||||
parse_controls,
|
||||
)
|
||||
|
||||
|
||||
def test_content_positions_are_the_seven_from_the_spec():
|
||||
assert CONTENT_POSITIONS == frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
|
||||
|
||||
def test_valid_controls_pass_validation():
|
||||
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
|
||||
validate_controls(c) # does not raise
|
||||
|
||||
|
||||
def test_invalid_content_position_rejected():
|
||||
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(c)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
|
||||
@pytest.mark.parametrize("bad", [-1, 5, True])
|
||||
def test_out_of_range_or_non_int_knob_rejected(field, bad):
|
||||
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
kwargs[field] = bad
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(Controls(**kwargs))
|
||||
|
||||
|
||||
def test_parse_controls_from_mapping():
|
||||
c = parse_controls(
|
||||
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
|
||||
)
|
||||
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
|
||||
|
||||
|
||||
def test_parse_controls_rejects_unknown_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
|
||||
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
|
||||
|
||||
|
||||
def test_parse_controls_rejects_missing_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0})
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pytest tests/test_player_controls.py -v`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'player'`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# player/__init__.py
|
||||
```
|
||||
(empty file)
|
||||
|
||||
```python
|
||||
# player/controls.py
|
||||
"""Control-panel state: the data shape read from the Arduino over serial.
|
||||
|
||||
This is the serial-contract payload shared with sub-project 4 (firmware). It
|
||||
models the full panel from design §6/§7: the 7-way content dial, the four
|
||||
experience knobs (0..4), and the two intensity levels (volume, brightness).
|
||||
The wire framing itself is the separate 3<->4 serial contract; this module is
|
||||
the *decoded* form.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, fields
|
||||
|
||||
# The seven positions of the content dial (design §6).
|
||||
CONTENT_POSITIONS = frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
|
||||
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
|
||||
KNOB_MIN = 0
|
||||
KNOB_MAX = 4
|
||||
|
||||
|
||||
class ControlsError(ValueError):
|
||||
"""Raised when a Controls payload is structurally invalid."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Controls:
|
||||
content: str
|
||||
left: int
|
||||
right: int
|
||||
dark: int
|
||||
light: int
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
|
||||
def validate_controls(c: Controls) -> None:
|
||||
"""Raise ControlsError if the payload is structurally invalid."""
|
||||
if c.content not in CONTENT_POSITIONS:
|
||||
raise ControlsError(
|
||||
f"invalid content position {c.content!r}; "
|
||||
f"expected one of {sorted(CONTENT_POSITIONS)}"
|
||||
)
|
||||
for name in KNOB_FIELDS:
|
||||
value = getattr(c, name)
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise ControlsError(f"knob {name} must be an int, got {value!r}")
|
||||
if not (KNOB_MIN <= value <= KNOB_MAX):
|
||||
raise ControlsError(
|
||||
f"knob {name}={value} out of range {KNOB_MIN}..{KNOB_MAX}"
|
||||
)
|
||||
|
||||
|
||||
def parse_controls(data: dict) -> Controls:
|
||||
"""Build a validated Controls from a decoded mapping, rejecting unknown or
|
||||
missing keys."""
|
||||
known = {f.name for f in fields(Controls)}
|
||||
unknown = set(data) - known
|
||||
if unknown:
|
||||
raise ControlsError(f"unknown keys: {sorted(unknown)}")
|
||||
missing = known - set(data)
|
||||
if missing:
|
||||
raise ControlsError(f"missing keys: {sorted(missing)}")
|
||||
c = Controls(**data)
|
||||
validate_controls(c)
|
||||
return c
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pytest tests/test_player_controls.py -v`
|
||||
Expected: PASS (all cases).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add player/__init__.py player/controls.py tests/test_player_controls.py
|
||||
git commit -m "feat(player): Controls panel model (serial-contract data shape)
|
||||
|
||||
Sub-project 3 slice 1, per ROADMAP.md §3 and design §6/§7."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Content-dial resolution (§6)
|
||||
|
||||
**Files:**
|
||||
- Create: `player/content.py`
|
||||
- Test: `tests/test_player_content.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/test_player_content.py
|
||||
import pytest
|
||||
|
||||
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
|
||||
|
||||
|
||||
def test_audio_sources_are_the_four_distinct_sources():
|
||||
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
# The §6 table, row by row: position -> (audio_source, video).
|
||||
@pytest.mark.parametrize(
|
||||
"position,audio_source,video",
|
||||
[
|
||||
("off", "none", False),
|
||||
("white_noise", "white_noise", False),
|
||||
("music", "music", False),
|
||||
("audio_track", "audio_track", False),
|
||||
("video", "none", True),
|
||||
("music_video", "music", True),
|
||||
("audio_video", "audio_track", True),
|
||||
],
|
||||
)
|
||||
def test_resolve_content_matches_spec_table(position, audio_source, video):
|
||||
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
|
||||
|
||||
|
||||
def test_off_is_void_state_black_and_silent():
|
||||
r = resolve_content("off")
|
||||
assert r.video is False and r.audio_source == "none"
|
||||
|
||||
|
||||
def test_resolve_content_rejects_unknown_position():
|
||||
with pytest.raises(ValueError):
|
||||
resolve_content("bogus")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pytest tests/test_player_content.py -v`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'player.content'`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# player/content.py
|
||||
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
|
||||
|
||||
The single source of truth for design §6's table: which audio source plays and
|
||||
whether the projector shows video, for each dial position.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
# The four distinct audio sources. "white_noise" is generated at runtime;
|
||||
# "music" is the public-domain classical pool; "audio_track" is the clip's own
|
||||
# field audio; "none" is silence.
|
||||
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContentResolution:
|
||||
audio_source: str
|
||||
video: bool
|
||||
|
||||
|
||||
# Design §6, one row per dial position.
|
||||
_TABLE = {
|
||||
"off": ContentResolution("none", False),
|
||||
"white_noise": ContentResolution("white_noise", False),
|
||||
"music": ContentResolution("music", False),
|
||||
"audio_track": ContentResolution("audio_track", False),
|
||||
"video": ContentResolution("none", True),
|
||||
"music_video": ContentResolution("music", True),
|
||||
"audio_video": ContentResolution("audio_track", True),
|
||||
}
|
||||
|
||||
|
||||
def resolve_content(position: str) -> ContentResolution:
|
||||
"""Map a content-dial position to its audio source and video flag (§6)."""
|
||||
try:
|
||||
return _TABLE[position]
|
||||
except KeyError:
|
||||
raise ValueError(
|
||||
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
|
||||
) from None
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pytest tests/test_player_content.py -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add player/content.py tests/test_player_content.py
|
||||
git commit -m "feat(player): 7-way content-dial resolution (design §6)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Alteration engine — knob vector to RenderPlan (§4, §5)
|
||||
|
||||
**Files:**
|
||||
- Create: `player/alteration.py`
|
||||
- Test: `tests/test_player_alteration.py`
|
||||
|
||||
The engine maps a knob vector `(left, right, dark, light)` to a layered `RenderPlan` per design §4.2:
|
||||
- **Substrate** transforms (alter pixels): `Restyle` (Right, v2v) blended with `ColorGrade` (mood, Dark/Light).
|
||||
- **Overlay** on top: `AnalyticalOverlay` (Left, HUD/labels).
|
||||
|
||||
`ColorGrade.tone` is signed: `>0` warm yellow→white (light pole), `<0` cool blue→black (dark pole), `0` identity/raw (§5). Calibration (flagged decision): `overlay = left/4`, `restyle = right/4`, `tone = (light − dark)/4`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/test_player_alteration.py
|
||||
import pytest
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import (
|
||||
AnalyticalOverlay,
|
||||
ColorGrade,
|
||||
RenderPlan,
|
||||
Restyle,
|
||||
plan_alteration,
|
||||
)
|
||||
|
||||
|
||||
def _coord(left=0, right=0, dark=0, light=0):
|
||||
return Coordinate(left=left, right=right, dark=dark, light=light)
|
||||
|
||||
|
||||
def test_all_zero_knobs_is_the_unaltered_base():
|
||||
plan = plan_alteration(_coord())
|
||||
assert plan.is_identity
|
||||
assert plan.overlay.intensity == 0.0
|
||||
assert plan.restyle.blend == 0.0
|
||||
assert plan.grade.tone == 0.0
|
||||
assert plan.grade.is_identity
|
||||
|
||||
|
||||
def test_left_drives_the_analytical_overlay_only():
|
||||
plan = plan_alteration(_coord(left=4))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 0.0 # Left does not touch the substrate
|
||||
assert plan.grade.tone == 0.0
|
||||
|
||||
|
||||
def test_right_drives_the_restyle_substrate_only():
|
||||
plan = plan_alteration(_coord(right=2))
|
||||
assert plan.restyle.blend == 0.5
|
||||
assert plan.overlay.intensity == 0.0 # Right does not add overlay
|
||||
|
||||
|
||||
def test_left_and_right_stack_not_cancel():
|
||||
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
|
||||
plan = plan_alteration(_coord(left=4, right=4))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 1.0
|
||||
|
||||
|
||||
def test_light_pole_grades_warm_positive_tone():
|
||||
plan = plan_alteration(_coord(light=4))
|
||||
assert plan.grade.tone == 1.0
|
||||
assert not plan.grade.is_identity
|
||||
|
||||
|
||||
def test_dark_pole_grades_cool_negative_tone():
|
||||
plan = plan_alteration(_coord(dark=4))
|
||||
assert plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_equal_dark_and_light_is_identity_grade():
|
||||
# design §5: the mood center is the raw, ungraded footage
|
||||
assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity
|
||||
assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity
|
||||
|
||||
|
||||
def test_dark_minus_light_sets_intermediate_tone():
|
||||
assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5)
|
||||
assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5)
|
||||
|
||||
|
||||
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
|
||||
# design §4.2 "Dark + analytical": cold measurement over a melancholy scene
|
||||
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 0.5
|
||||
assert plan.grade.tone == -1.0
|
||||
assert not plan.is_identity
|
||||
|
||||
|
||||
def test_render_plan_is_frozen():
|
||||
plan = plan_alteration(_coord())
|
||||
with pytest.raises(Exception):
|
||||
plan.grade.tone = 0.5 # type: ignore[misc]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pytest tests/test_player_alteration.py -v`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'player.alteration'`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# player/alteration.py
|
||||
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
|
||||
|
||||
Replaces the 2026-06-04 nearest-match *selection* with a *transformation* of a
|
||||
neutral base clip. Given the four experience knobs, it produces three layers
|
||||
that compose per §4.2:
|
||||
|
||||
- Substrate transforms (alter pixels, blend with each other):
|
||||
* Restyle -- the Right axis: a pre-baked generative v2v dreamlike restyle.
|
||||
* ColorGrade -- the mood axis (Dark/Light): a deterministic color grade.
|
||||
- Overlay (composited on top of the substrate):
|
||||
* AnalyticalOverlay -- the Left axis: HUD/labels/measurement.
|
||||
|
||||
Left and Right are NOT opposites; they live on different layers and stack
|
||||
(§4.2). Dark and Light are the two poles of one mood grade whose center is the
|
||||
identity (§5).
|
||||
|
||||
Calibration note: the knob->strength curves below are the single source of
|
||||
truth for how a 0..4 position maps to a transform strength. See the session
|
||||
0006 transcript Deferred decisions for the §3-vs-§4.2/§5 reconciliation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from hef.selection import Coordinate
|
||||
|
||||
KNOB_MAX = 4 # knob full-scale (0..4)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ColorGrade:
|
||||
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
|
||||
<0 cool blue->black (dark), 0 = identity (raw, ungraded)."""
|
||||
|
||||
tone: float
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
return self.tone == 0.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AnalyticalOverlay:
|
||||
"""Left axis (§4.1): analytical HUD/annotation/labels, composited on top.
|
||||
`intensity` 0..1 (0 = no overlay)."""
|
||||
|
||||
intensity: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Restyle:
|
||||
"""Right axis (§4.1): pre-baked generative v2v dreamlike substrate.
|
||||
`blend` 0..1 (0 = raw substrate, no restyle)."""
|
||||
|
||||
blend: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RenderPlan:
|
||||
"""The full layered alteration for one knob vector (§4.2)."""
|
||||
|
||||
grade: ColorGrade
|
||||
overlay: AnalyticalOverlay
|
||||
restyle: Restyle
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
"""True when the plan leaves the neutral base un-altered."""
|
||||
return (
|
||||
self.grade.is_identity
|
||||
and self.overlay.intensity == 0.0
|
||||
and self.restyle.blend == 0.0
|
||||
)
|
||||
|
||||
|
||||
def _overlay_intensity(left: int) -> float:
|
||||
"""Left knob -> analytical-overlay intensity (0..1)."""
|
||||
return left / KNOB_MAX
|
||||
|
||||
|
||||
def _restyle_blend(right: int) -> float:
|
||||
"""Right knob -> v2v restyle blend (0..1)."""
|
||||
return right / KNOB_MAX
|
||||
|
||||
|
||||
def _mood_tone(dark: int, light: int) -> float:
|
||||
"""(dark, light) -> signed mood grade in [-1, 1]; equal -> 0 identity (§5)."""
|
||||
return (light - dark) / KNOB_MAX
|
||||
|
||||
|
||||
def plan_alteration(coord: Coordinate) -> RenderPlan:
|
||||
"""Map a knob vector to its layered RenderPlan (design §4)."""
|
||||
return RenderPlan(
|
||||
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light)),
|
||||
overlay=AnalyticalOverlay(intensity=_overlay_intensity(coord.left)),
|
||||
restyle=Restyle(blend=_restyle_blend(coord.right)),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pytest tests/test_player_alteration.py -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add player/alteration.py tests/test_player_alteration.py
|
||||
git commit -m "feat(player): alteration engine — knob vector to RenderPlan (design §4/§5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Player state machine — controls stream to transitions
|
||||
|
||||
**Files:**
|
||||
- Create: `player/state.py`
|
||||
- Test: `tests/test_player_state.py`
|
||||
|
||||
The `Player` holds a base-clip library and the current `Playback`. Each `update(controls)` resolves the desired `Playback` (chosen base clip + RenderPlan + content resolution + volume/brightness) and returns the `Transition` from the previous playback:
|
||||
|
||||
- target video off, was on → `FADE_TO_BLACK`
|
||||
- target video on, was off → `FADE_FROM_BLACK`
|
||||
- both video on, clip or restyle variant changed → `CROSSFADE`
|
||||
- both video on, only grade/overlay/level changed → `LIVE_UPDATE` (§4.3: grade + overlay are continuous runtime ops)
|
||||
- both video off, audio/level changed → `LIVE_UPDATE`
|
||||
- nothing changed → `NONE`
|
||||
|
||||
Base-clip choice is an injected callable (default: first clip); knobs no longer *select* (the base is neutral) — they *transform*. The library is duck-typed objects with an `.id`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/test_player_state.py
|
||||
from dataclasses import dataclass
|
||||
|
||||
import pytest
|
||||
|
||||
from player.controls import Controls
|
||||
from player.state import Player, Playback, Transition, TransitionKind
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FakeClip:
|
||||
id: str
|
||||
|
||||
|
||||
LIB = [FakeClip("base-a"), FakeClip("base-b")]
|
||||
|
||||
|
||||
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
|
||||
return Controls(content, left, right, dark, light, volume, brightness)
|
||||
|
||||
|
||||
def test_first_update_to_video_fades_in_from_black():
|
||||
p = Player(LIB)
|
||||
t = p.update(_controls(content="video"))
|
||||
assert t.kind == TransitionKind.FADE_FROM_BLACK
|
||||
assert t.playback.clip_id == "base-a"
|
||||
assert t.playback.content.video is True
|
||||
|
||||
|
||||
def test_off_from_video_fades_to_black_and_silences():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video"))
|
||||
t = p.update(_controls(content="off"))
|
||||
assert t.kind == TransitionKind.FADE_TO_BLACK
|
||||
assert t.playback.clip_id is None
|
||||
assert t.playback.content.audio_source == "none"
|
||||
|
||||
|
||||
def test_no_change_yields_none_transition():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=1))
|
||||
t = p.update(_controls(content="video", left=1))
|
||||
assert t.kind == TransitionKind.NONE
|
||||
|
||||
|
||||
def test_grade_change_is_a_live_update_not_a_crossfade():
|
||||
# design §4.3: the Dark/Light grade is a continuous runtime op
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", dark=0, light=0))
|
||||
t = p.update(_controls(content="video", dark=4, light=0))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_overlay_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=0))
|
||||
t = p.update(_controls(content="video", left=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.overlay.intensity == 1.0
|
||||
|
||||
|
||||
def test_restyle_change_crossfades_the_substrate():
|
||||
# design §4.3: the Right v2v substrate is a pre-baked variant swap
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", right=0))
|
||||
t = p.update(_controls(content="video", right=4))
|
||||
assert t.kind == TransitionKind.CROSSFADE
|
||||
assert t.playback.plan.restyle.blend == 1.0
|
||||
|
||||
|
||||
def test_volume_only_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", volume=1))
|
||||
t = p.update(_controls(content="video", volume=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.volume == 4
|
||||
|
||||
|
||||
def test_audio_source_change_while_black_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="white_noise"))
|
||||
t = p.update(_controls(content="music"))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.content.audio_source == "music"
|
||||
|
||||
|
||||
def test_injected_base_chooser_is_used():
|
||||
p = Player(LIB, choose_base=lambda lib: lib[1])
|
||||
t = p.update(_controls(content="video"))
|
||||
assert t.playback.clip_id == "base-b"
|
||||
|
||||
|
||||
def test_empty_library_with_video_raises():
|
||||
p = Player([])
|
||||
with pytest.raises(ValueError):
|
||||
p.update(_controls(content="video"))
|
||||
|
||||
|
||||
def test_off_with_empty_library_is_fine():
|
||||
p = Player([])
|
||||
t = p.update(_controls(content="off"))
|
||||
assert t.kind == TransitionKind.NONE # already black at init
|
||||
assert t.playback.clip_id is None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pytest tests/test_player_state.py -v`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'player.state'`.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# player/state.py
|
||||
"""The player state machine: a stream of Controls -> playback transitions.
|
||||
|
||||
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
|
||||
desired Playback (which neutral base clip, how it is altered, what audio plays,
|
||||
at what levels) and returns the Transition from the previous Playback. The
|
||||
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
|
||||
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
|
||||
pre-baked Right v2v variant needs a CROSSFADE, and toggling video on/off
|
||||
fades to/from black.
|
||||
|
||||
Knobs no longer *select* a clip (the base library is neutral by construction);
|
||||
they *transform* it. Which neutral base to show is an injected policy
|
||||
(`choose_base`), defaulting to the first clip; richer rotation is a later slice.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable, Optional
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import RenderPlan, plan_alteration
|
||||
from player.content import ContentResolution, resolve_content
|
||||
from player.controls import Controls
|
||||
|
||||
|
||||
class TransitionKind:
|
||||
NONE = "none"
|
||||
LIVE_UPDATE = "live_update"
|
||||
CROSSFADE = "crossfade"
|
||||
FADE_TO_BLACK = "fade_to_black"
|
||||
FADE_FROM_BLACK = "fade_from_black"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Playback:
|
||||
"""What should currently be playing."""
|
||||
|
||||
clip_id: Optional[str] # None = black walls
|
||||
plan: Optional[RenderPlan] # None when black
|
||||
content: ContentResolution
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Transition:
|
||||
kind: str
|
||||
playback: Playback
|
||||
|
||||
|
||||
def _first(library):
|
||||
if not library:
|
||||
raise ValueError("no base clips available to play video")
|
||||
return library[0]
|
||||
|
||||
|
||||
_BLACK = Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=ContentResolution("none", False),
|
||||
volume=0,
|
||||
brightness=0,
|
||||
)
|
||||
|
||||
|
||||
class Player:
|
||||
def __init__(
|
||||
self,
|
||||
base_library,
|
||||
*,
|
||||
choose_base: Callable = _first,
|
||||
approved_only: bool = False,
|
||||
):
|
||||
self._library = list(base_library)
|
||||
self._choose_base = choose_base
|
||||
self._approved_only = approved_only # reserved for catalog-backed libs
|
||||
self._current = _BLACK
|
||||
|
||||
def _resolve(self, controls: Controls) -> Playback:
|
||||
content = resolve_content(controls.content)
|
||||
if not content.video:
|
||||
return Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=content,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
clip = self._choose_base(self._library)
|
||||
coord = Coordinate(controls.left, controls.right, controls.dark, controls.light)
|
||||
return Playback(
|
||||
clip_id=clip.id,
|
||||
plan=plan_alteration(coord),
|
||||
content=content,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
|
||||
def _classify(self, prev: Playback, nxt: Playback) -> str:
|
||||
prev_video = prev.clip_id is not None
|
||||
next_video = nxt.clip_id is not None
|
||||
if prev == nxt:
|
||||
return TransitionKind.NONE
|
||||
if prev_video and not next_video:
|
||||
return TransitionKind.FADE_TO_BLACK
|
||||
if next_video and not prev_video:
|
||||
return TransitionKind.FADE_FROM_BLACK
|
||||
if next_video and prev_video:
|
||||
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
|
||||
return TransitionKind.CROSSFADE
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
# both black: only audio/levels could have changed
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
|
||||
def update(self, controls: Controls) -> Transition:
|
||||
nxt = self._resolve(controls)
|
||||
kind = self._classify(self._current, nxt)
|
||||
self._current = nxt
|
||||
return Transition(kind=kind, playback=nxt)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pytest tests/test_player_state.py -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add player/state.py tests/test_player_state.py
|
||||
git commit -m "feat(player): player state machine — controls stream to transitions"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Register the package and run the full suite
|
||||
|
||||
**Files:**
|
||||
- Modify: `pyproject.toml` (`[tool.setuptools] packages`)
|
||||
|
||||
- [ ] **Step 1: Add `"player"` to the packages list**
|
||||
|
||||
In `pyproject.toml`, change:
|
||||
|
||||
```toml
|
||||
[tool.setuptools]
|
||||
packages = ["hef", "tools", "simulator"]
|
||||
```
|
||||
to:
|
||||
```toml
|
||||
[tool.setuptools]
|
||||
packages = ["hef", "tools", "simulator", "player"]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the entire test suite**
|
||||
|
||||
Run: `pytest -q`
|
||||
Expected: all prior tests still pass plus the new `test_player_*` files; no failures.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add pyproject.toml
|
||||
git commit -m "build(player): register the player package"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Update the roadmap to reflect slice 1 shipped
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/ROADMAP.md` (§3 Player Runtime)
|
||||
|
||||
- [ ] **Step 1: Update §3 status and deliverables**
|
||||
|
||||
Under "## 3. Player Runtime (Pi)", note that the **alteration-engine + player-core slice** (pure logic) is done and tested, link this plan, and list the remaining slices (mpv/ffmpeg runtime integration + GPU grade/overlay shaders; real USB-serial reader + the 3⇄4 framing contract; white-noise DSP; offline v2v variant build pipeline; catalog model changes for audio-source + neutral-base flag). Keep the table's status marker for sub-project 3 as in-progress (⏳), not done.
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/ROADMAP.md
|
||||
git commit -m "docs(roadmap): sub-project 3 alteration-engine + player-core slice shipped"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:**
|
||||
- §4.1 per-axis alteration → Task 3 (overlay=Left, restyle=Right, grade=mood). ✓
|
||||
- §4.2 composition rule (substrate vs overlay; Left+Right stack) → Task 3 tests `test_left_and_right_stack_not_cancel`, `test_whole_brain_dark_corner_*`. ✓
|
||||
- §4.3 runtime vs pre-baked (grade/overlay continuous; restyle variant swap) → Task 4 transition kinds (`LIVE_UPDATE` vs `CROSSFADE`). ✓
|
||||
- §5 mood grade, center = identity → Task 3 `test_equal_dark_and_light_is_identity_grade`, signed tone. ✓
|
||||
- §6 7-way content dial → Task 2 full table. ✓
|
||||
- §7 volume + brightness intensity → Task 1 (modeled) + Task 4 (`test_volume_only_change_is_a_live_update`). ✓
|
||||
- Roadmap §3 "done when: plays correct segment, loops, crossfades, goes dark on None, testable with serial mocked" → Task 4 (transitions incl. fade-to-black; serial mocked as a `Controls` stream). Looping is implicit (a clip plays until the desired playback changes); explicit loop timing belongs to the mpv integration slice (deferred, noted in Task 6). ✓ (with the runtime-loop portion deferred and documented).
|
||||
|
||||
**Deferred (documented, not gaps):** mpv/ffmpeg + GPU; real serial + framing; white-noise DSP; v2v offline pipeline; catalog audio-source/neutral-base fields; base-clip rotation policy; `approved_only` enforcement against a real catalog (the flag is plumbed but inert here).
|
||||
|
||||
**Placeholder scan:** none — every step has concrete code/commands.
|
||||
|
||||
**Type consistency:** `Controls(content,left,right,dark,light,volume,brightness)` used identically in Tasks 1 & 4; `RenderPlan.{grade,overlay,restyle}` and `.is_identity` consistent across Tasks 3 & 4; `ContentResolution(audio_source,video)` consistent across Tasks 2 & 4; `TransitionKind` constants consistent. `Coordinate(left,right,dark,light)` matches `hef/selection.py`.
|
||||
@@ -0,0 +1,193 @@
|
||||
# Experience Simulator (Curator's X-ray) — Design Spec
|
||||
|
||||
**Date:** 2026-06-04
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Session:** 0003 (discovery)
|
||||
|
||||
> A web-based stand-in for the installation's control panel, run on localhost in a
|
||||
> Docker container, built so a curator can **feel the selection model** — turn the
|
||||
> dials and see, in full, which catalog pieces the nearest-match algorithm
|
||||
> surfaces and why. It validates the *principles* (coordinate tagging + selection)
|
||||
> and the *interface* before the physical Pi-player / Arduino stack is built.
|
||||
|
||||
---
|
||||
|
||||
## 1. What this is
|
||||
|
||||
A single-page web tool that reproduces the installation's five controls and wires
|
||||
them to the **real** `hef.selection` code, then shows a transparent "X-ray" of the
|
||||
result: the picked piece, the full ranked candidate pool with distances, and a
|
||||
live map of where the knob point and candidates sit in the coordinate space.
|
||||
|
||||
It is a **curator's instrument**, not the installation itself. Its job is to let a
|
||||
human judge whether the tagging rubric (design spec §2) and the nearest-match
|
||||
selection (design spec §3) actually surface fitting pieces at every knob position —
|
||||
so a "wrong" result can be diagnosed as bad tagging vs. a sparse neighborhood.
|
||||
|
||||
Primary purpose (settled this session): **feel the selection model.** Faithful
|
||||
media playback is explicitly *not* a goal.
|
||||
|
||||
### Design constraints
|
||||
- **Real algorithm, single source of truth.** The simulator runs the shipped
|
||||
`hef.selection` code. No reimplementation of selection in JavaScript — a copy
|
||||
could drift from what the Pi will run, which would defeat the purpose.
|
||||
- **Runs locally in Docker.** One container, one command, `localhost`. No
|
||||
networking beyond the local browser, no auth, no multi-user.
|
||||
- **Reads the model; never curates into it.** The simulator does not write catalog
|
||||
records back. Editing/approval stays with the sub-project-2 review tools.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture
|
||||
|
||||
One Python service in one container. **FastAPI** serves both a JSON selection API
|
||||
and a dependency-free static frontend. The browser holds the dial state; each
|
||||
change POSTs to the API and re-renders the X-ray.
|
||||
|
||||
```
|
||||
browser (dials + X-ray) ──POST /api/select──▶ FastAPI ──▶ hef.selection (REAL)
|
||||
▲ │
|
||||
└────────── pool + distances + pick ◀─────┘
|
||||
```
|
||||
|
||||
- **Backend:** `simulator/app.py` (FastAPI), `simulator/fixtures.py` (synthetic
|
||||
catalog generator). Imports `hef.catalog` and `hef.selection` directly.
|
||||
- **Frontend:** `simulator/static/` — `index.html`, `app.js`, `style.css`. Vanilla
|
||||
JS, no build step.
|
||||
- **Packaging:** `simulator/Dockerfile` + a one-command run (`docker compose up`
|
||||
or a `make sim` target), serving on `localhost`.
|
||||
|
||||
This adds a `simulator/` top-level package, consistent with the repo layout in the
|
||||
main design spec §10 (`hef/`, `player/`, `firmware/`, `tools/`).
|
||||
|
||||
---
|
||||
|
||||
## 3. One additive change to `hef.selection`
|
||||
|
||||
The X-ray needs the ranked pool with distances — which `select()` already computes
|
||||
internally (`ranked`, `nearest`) but does not expose. To avoid duplicating ranking
|
||||
logic in the simulator (the very drift risk §1 forbids), extract it into a public
|
||||
function and refactor `select()` to call it:
|
||||
|
||||
```python
|
||||
def ranked_candidates(
|
||||
records, coord: Coordinate, mode: str, *,
|
||||
pool_size: int = 4, weights: Weights = Weights(), approved_only: bool = False,
|
||||
) -> list[tuple[Record, float]]:
|
||||
"""Mode-eligible records sorted nearest-first, each paired with its distance.
|
||||
|
||||
Honors the same 'av' fallback and approved_only filtering as select().
|
||||
"""
|
||||
```
|
||||
|
||||
`select()` becomes a thin wrapper over `ranked_candidates()` (take the pool, then
|
||||
return `nearest[0]` or `rng.choice(nearest)`). This is the **only** change to
|
||||
shipped code: additive, behavior-preserving for `select()`, and independently
|
||||
tested. The X-ray then displays exactly the pool the picker chooses from.
|
||||
|
||||
---
|
||||
|
||||
## 4. Synthetic fixture catalog (`simulator/fixtures.py`)
|
||||
|
||||
The real `catalog/library.jsonl` is empty. A deterministic (seeded) generator
|
||||
produces a dense, representative spread of `Record`s so nearest-match behavior can
|
||||
be felt everywhere immediately:
|
||||
|
||||
- Coverage across the `(left, right, dark, light)` space (the 5×5 brain plane ×
|
||||
5×5 mood plane = 625 configs) and across all content modes (`audio`, `video`,
|
||||
`av`).
|
||||
- Plausible titles and `rationale`s; `source_archive`/`license` drawn from the
|
||||
real pools so records validate.
|
||||
- A deliberate mix of `review_status: proposed` and `approved`, so the
|
||||
`approved_only` toggle is exercisable.
|
||||
- No real media: `file_path` empty, `dominant_color` empty (optional field).
|
||||
- Output validates against `hef.catalog.validate_catalog`.
|
||||
|
||||
Loaded at startup. A flag (`--catalog` / env var) lets the simulator point at
|
||||
`catalog/library.jsonl` instead when it is non-empty — **same loader**, so swapping
|
||||
in the real catalog later is free.
|
||||
|
||||
---
|
||||
|
||||
## 5. The API
|
||||
|
||||
### `POST /api/select`
|
||||
Request:
|
||||
```json
|
||||
{ "left":0-4, "right":0-4, "dark":0-4, "light":0-4,
|
||||
"mode":"none|audio|video|av",
|
||||
"pool_size":int, "weights":{"brain":float,"mood":float},
|
||||
"approved_only":bool }
|
||||
```
|
||||
Response:
|
||||
```json
|
||||
{ "pick": <record|null>,
|
||||
"pool": [ { "record": <record>, "distance": float, "rank": int }, ... ],
|
||||
"coverage": { "candidates_in_mode": int } }
|
||||
```
|
||||
- `none` selector mode → `pick:null`, `pool:[]` (the void/rest state).
|
||||
- No candidates → `pick:null`, `pool:[]`.
|
||||
- **Deterministic pick.** The displayed `pick` is the rank-1 (nearest) candidate —
|
||||
`select()` called with `rng=None` — so the X-ray is legible and stable for a
|
||||
given knob position. The room *shuffles* within the nearest pool at runtime;
|
||||
the returned `pool` IS that shuffle-set, so the curator sees every piece the
|
||||
real installation could surface there.
|
||||
- Knob values are constrained 0–4 and `mode` to the four values by Pydantic; bad
|
||||
input is a clean `422`.
|
||||
- Backed by `hef.selection.ranked_candidates()` (pool) + `select()` (the pick).
|
||||
|
||||
### `GET /api/catalog/meta`
|
||||
Counts, per-mode breakdown, `proposed`/`approved` split, and coverage stats for
|
||||
the header.
|
||||
|
||||
---
|
||||
|
||||
## 6. The X-ray UI
|
||||
|
||||
**Controls**
|
||||
- The five real dials: a 4-way mode selector (None/Audio/Video/A+V) and four 0–4
|
||||
knobs (Left, Right, Dark, Light).
|
||||
- The model knobs (so the algorithm's own parameters can be felt): brain/mood
|
||||
**weight sliders**, **pool-size** control, and an **approved-only** toggle.
|
||||
|
||||
**Readout**
|
||||
- **Picked piece:** title, mode, coordinate, distance, rationale.
|
||||
- **Ranked pool:** the N nearest with distances, winner highlighted.
|
||||
- **Two 5×5 grid maps** (brain L×R, mood D×L): the current knob point plus where
|
||||
candidates sit, so the neighborhood that produced the pick is visible at a
|
||||
glance.
|
||||
|
||||
---
|
||||
|
||||
## 7. Testing
|
||||
|
||||
- **Unit (`hef`):** `ranked_candidates()` ordering, the `av` pool fallback,
|
||||
`approved_only` filtering, weight effects; `select()` still behaves as before.
|
||||
- **Fixtures:** coverage test (every mode populated; space spanned; output passes
|
||||
`validate_catalog`).
|
||||
- **API:** FastAPI `TestClient` contract tests — valid select returns ranked pool;
|
||||
`none` returns the void; bad input → 422; `approved_only` narrows the pool.
|
||||
- **Behavior:** the felt behaviors are captured as gherkin scenarios under
|
||||
`features/` (this session's BDD deliverable).
|
||||
|
||||
---
|
||||
|
||||
## 8. Explicitly out of scope (YAGNI)
|
||||
|
||||
- Real media playback (audio/video) — the tool reads the model, not the content.
|
||||
- Writing/editing catalog records back — curation stays in the sub-project-2 tools.
|
||||
- Networking, auth, multi-user, session recording.
|
||||
- Any reimplementation of selection logic outside `hef`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Relationship to the roadmap
|
||||
|
||||
This is an **upstream discovery vehicle** for the physical stack (sub-projects 3
|
||||
and 4). It exercises sub-project 1's `hef.selection` against a stand-in catalog and
|
||||
is the richer, experienceable form of the "keyboard/serial stand-in" the roadmap
|
||||
anticipates for sub-project 3. It does not replace the Pi player or the firmware;
|
||||
it de-risks their *principles and interface* first. The synthetic catalog is a
|
||||
stand-in for the sub-project-2 output and is swapped for the real catalog through
|
||||
the same loader once ingest has populated it.
|
||||
@@ -0,0 +1,360 @@
|
||||
# Human Experience Filter — Machine-Altered Perception (Design Revision)
|
||||
|
||||
**Date:** 2026-06-05
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Supersedes:** the thesis (§1), selection model (§3), tagging division (§5), and
|
||||
sourcing strategy (§8) of
|
||||
[`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md),
|
||||
and substantially expands its control panel (§2 selector) and hardware (§6). The
|
||||
coordinate model (§2 axes) and the single-panoramic-projector display are
|
||||
**preserved**.
|
||||
|
||||
> **Why this revision exists.** The 2026-06-04 design framed the piece as an
|
||||
> *experience filter over found human artifacts* ("curation is the artwork; not
|
||||
> algorithmically generated filler"). This revision **changes the artistic
|
||||
> thesis** to one about **how humans interact with machines and how that interaction
|
||||
> reshapes their nervous systems** — and in doing so makes machine alteration of
|
||||
> the imagery the *subject* of the piece rather than a betrayal of it.
|
||||
|
||||
---
|
||||
|
||||
## 1. What this is (the new thesis)
|
||||
|
||||
A single-viewer immersive installation about **a machine reshaping your
|
||||
perception — for better and for worse.**
|
||||
|
||||
One person sits in a chair in a small room. A single panoramic projector wraps the
|
||||
walls they face. The source is **neutral**: real, calm, public-domain nature
|
||||
footage that sits at the *center* of the experience-space — neither analytical nor
|
||||
emotional, neither bright nor dark. The viewer turns knobs on a wooden control
|
||||
panel, and **the machine alters that neutral reality toward whatever they dial
|
||||
in.** The piece is the felt, bodily experience of a machine bending your inner
|
||||
state — and crucially it bends **both ways**: it can soothe you or disturb you,
|
||||
clarify the world or manipulate it, name your feelings or colonize them.
|
||||
|
||||
You don't read that thesis on a wall label. You feel it in your nervous system as
|
||||
your own hand moves the knobs.
|
||||
|
||||
Design constraints (carried from the original, one relaxed):
|
||||
|
||||
- **One viewer, one room, all-local at runtime.** No networking, no streaming, no
|
||||
multi-user, no session recording. (All AI alteration happens **offline at
|
||||
authoring time**; the room plays pre-built media + cheap runtime grading.)
|
||||
- **Real footage as the substrate.** The base library is real public-domain
|
||||
nature footage. The machine *alters* it — it does not invent from nothing. This
|
||||
preserves real-world motion/composition (and some lineage to the original
|
||||
found-media thesis) while making the alteration the point. *(This relaxes the
|
||||
original "no generated content" rule: generation/alteration is now the
|
||||
subject.)*
|
||||
- **Built to grow.** A small, deliberate base library; the data model and tooling
|
||||
designed to expand.
|
||||
|
||||
---
|
||||
|
||||
## 2. What changes vs. 2026-06-04
|
||||
|
||||
| Area | 2026-06-04 | This revision |
|
||||
|---|---|---|
|
||||
| Thesis (§1) | filter over found human artifacts | **machine reshapes perception, good & bad** |
|
||||
| Content | large found library pre-matched to coordinates | **small NEUTRAL base library, machine-altered to the knob state** |
|
||||
| Engine (§3) | nearest-match **lookup** in a tagged catalog | **transform from neutral** (knobs = a transformation, not a lookup) |
|
||||
| Sourcing (§8) | find media that already matches each cell | source **neutral** clips; the machine produces the variants |
|
||||
| Control panel (§2) | 4-way mode + 4 knobs | **7-way content dial + 4 experience knobs + volume + brightness**, on a tactile wooden panel |
|
||||
| Display | single pano projector | **unchanged** |
|
||||
| Coordinate axes (§2) | Left/Right/Dark/Light, two planes | **unchanged** (reinterpreted as transform targets) |
|
||||
|
||||
---
|
||||
|
||||
## 3. Coordinate model (preserved, reinterpreted)
|
||||
|
||||
The four experience knobs and two planes from the 2026-06-04 §2 are unchanged:
|
||||
|
||||
- **Brain plane** = Left × Right. Left = analytical / verbal / structured. Right =
|
||||
artistic / emotional / abstract. Both high → "whole brain."
|
||||
- **Mood plane** = Dark × Light. Dark = somber / heavy. Light = uplifting / serene.
|
||||
|
||||
Each knob is `0–4`. What changes is the *meaning of a knob position*: it is no
|
||||
longer "find the catalog piece nearest this coordinate" but **"how hard the machine
|
||||
pushes the neutral base toward this pole."** `(2,2,2,2)` is the neutral origin —
|
||||
the un-altered base clip.
|
||||
|
||||
---
|
||||
|
||||
## 4. The alteration engine (replaces the §3 selection algorithm)
|
||||
|
||||
The engine's job changes from **choosing** a clip to **altering** one. Given a
|
||||
neutral base clip and a knob vector `(left, right, dark, light)`, it applies a
|
||||
**transformation toward each active pole** and composites the result.
|
||||
|
||||
### 4.1 Per-axis alteration
|
||||
|
||||
| Pole | What the machine does | How |
|
||||
|---|---|---|
|
||||
| **Left** (analytical/verbal) | **imposes analytical structure** on the scene — annotation, measurement, labels, grids, tracking boxes, data HUD; the machine *dissecting* the world into data | **overlay** layer composited on top |
|
||||
| **Right** (artistic/emotional) | **dissolves realism** toward painterly / dreamlike / abstract | **generative video-to-video (v2v)** restyle of the pixels |
|
||||
| **Dark** (somber/heavy) | drains toward the cool/melancholy/night grade (see §5) | **color grade** (deterministic) |
|
||||
| **Light** (uplifting/serene) | warms toward the bright/serene grade (see §5) | **color grade** (deterministic) |
|
||||
|
||||
### 4.2 The composition rule
|
||||
|
||||
The four axes are **two kinds of operation** that **layer** rather than blend-and-
|
||||
cancel:
|
||||
|
||||
- **Substrate transforms** — Right (v2v restyle) + Dark + Light (color grades).
|
||||
They alter the **pixels** and blend with each other.
|
||||
- **Overlay** — Left (annotation / labels / HUD). It composites **on top** of
|
||||
whatever the substrate is.
|
||||
|
||||
So Left and Right are **not** mutually-exclusive opposites; they stack on different
|
||||
layers. This single rule covers the whole `5×5×5×5` space:
|
||||
|
||||
- **Max Left + Max Right** = maximally dissolved dreamlike substrate **with the
|
||||
machine's labels on the feelings themselves** — *"awe," "grief," "detected:
|
||||
longing 0.82,"* tracking boxes around emotional content. This is the "whole
|
||||
brain" corner (§3): the machine that both **induces** emotion and
|
||||
**names/quantifies** it — affective computing made visceral. (Good/bad duality:
|
||||
is being told your own feeling clarifying, or invasive?)
|
||||
- **Dark + analytical** = a drained, melancholy scene with cold measurement
|
||||
overlaid.
|
||||
- **Light + emotional** = a warm, dreamlike wash, unlabeled.
|
||||
|
||||
### 4.3 Where each transform runs
|
||||
|
||||
- **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade
|
||||
and the Left analytical overlay. These are cheap (LUT/curves + luma key, and
|
||||
text/graphics compositing) and can move continuously with the knob.
|
||||
- **Pre-baked offline (the only AI cost):** the Right generative v2v restyle.
|
||||
Generative v2v cannot run real-time on a Pi, so a small set of restyled variants
|
||||
is rendered offline and selected/blended at runtime.
|
||||
|
||||
> **Crucial consequence — the Left labels are a RUNTIME OVERLAY, never baked into
|
||||
> the v2v pixels.** This is what makes multilingual support nearly free (§10).
|
||||
|
||||
---
|
||||
|
||||
## 5. Mood-axis color grade
|
||||
|
||||
The Dark↔Light knob is a three-stop color ramp whose **center is the identity** —
|
||||
the raw, ungraded footage, no color transform applied. Turning toward a pole grades
|
||||
*away* from the raw clip. Hue carries emotional valence; the **negative-space
|
||||
value** (a luma-keyed grade — the brightest/darkest regions pushed toward
|
||||
white/black) carries the literal light/dark reading, so the image *is* lighter or
|
||||
darker even before the hue registers.
|
||||
|
||||
| Mood knob | Hue | Negative space | Feeling |
|
||||
|---|---|---|---|
|
||||
| **Light** | warm **yellow** | → **white** | uplifting, serene, airy |
|
||||
| **Center** | **none — raw, ungraded video** | unchanged | the un-altered base |
|
||||
| **Dark** | cool **blue** | → **black** | melancholy, somber, night, cold |
|
||||
|
||||
*(Red/alarm was considered for Dark and set aside: red reads as threat/alarm, not
|
||||
the intended melancholy. Dark is the cool, mournful blue. The center applies no
|
||||
grade at all — it is simply the raw clip.)*
|
||||
|
||||
---
|
||||
|
||||
## 6. Content model — the 7-way content dial
|
||||
|
||||
The 2026-06-04 4-way selector (None/Audio/Video/A+V) expands to **seven positions**,
|
||||
letting the viewer choose channel **and audio source**:
|
||||
|
||||
| Position | Audio source | Video |
|
||||
|---|---|---|
|
||||
| **Off** | — | black |
|
||||
| **White Noise** | generated white/pink noise (not from catalog) | black |
|
||||
| **Music** | public-domain classical (Musopen pool) | black |
|
||||
| **Audio Track** | the nature clip's **own** audio | black |
|
||||
| **Video** | — | the (altered) nature video |
|
||||
| **Music + Video** | public-domain classical (Musopen pool) | the (altered) nature video |
|
||||
| **Audio Track + Video** | the nature clip's own audio | the (altered) nature video |
|
||||
|
||||
Notes:
|
||||
|
||||
- "Music" (classical) and "Audio Track" (the clip's own field audio) are now
|
||||
**distinct sources**; "White Noise" is **generated** at runtime, not a catalog
|
||||
record. This touches the catalog/`mode` model (the 2026-06-04 §4 schema), which
|
||||
must represent audio *source* in addition to channel.
|
||||
- **Off** is the void/rest state: black walls, silence.
|
||||
- Audio alteration (steering the *sound* by knob, parallel to the video) is **out
|
||||
of scope for this revision** — the experience knobs alter video; audio is chosen
|
||||
by source only. (Future work.)
|
||||
|
||||
---
|
||||
|
||||
## 7. Intensity controls
|
||||
|
||||
Two knobs, orthogonal to the four experience axes, exist so a viewer can take the
|
||||
experience at **whatever intensity they want**:
|
||||
|
||||
- **Volume** — audio output level.
|
||||
- **Brightness** — projector output luminance.
|
||||
|
||||
These are pure output levels, distinct from the Dark/Light *mood grade* (which
|
||||
changes hue and tone, not raw output). Brightness is "how much light hits the
|
||||
wall"; Light is "how the world is colored."
|
||||
|
||||
---
|
||||
|
||||
## 8. Content sourcing — neutral base library
|
||||
|
||||
The library is now **small and neutral**. Instead of hunting for footage that
|
||||
matches 625 coordinate cells, source a deliberate set of **neutral, calm,
|
||||
real public-domain nature clips** (centered on all four axes) and let the machine
|
||||
produce every variant.
|
||||
|
||||
- **Reuse sub-project 2** (the built ingest/tagging/review tooling) to pull
|
||||
neutral public-domain nature footage and populate mechanical fields.
|
||||
- **License stance** unchanged: prefer Public Domain / CC0; record license +
|
||||
source per clip. Altered/generated variants record the model and that they are
|
||||
machine-derived.
|
||||
- **Size:** dozens of neutral base clips (not hundreds), each yielding many
|
||||
altered variants. Short, seamless loops (≈20–60 s) suit a looping installation
|
||||
and are the v2v models' native sweet spot.
|
||||
|
||||
---
|
||||
|
||||
## 9. Economics (why this is feasible)
|
||||
|
||||
At the original scale (120–800 pieces × ~10-min segments) AI involvement was
|
||||
~$15k–45k+ and not viable. Under this design it drops to **a few hundred to a few
|
||||
thousand dollars**, because:
|
||||
|
||||
- The library is small (generate-to-order, not find-to-cover).
|
||||
- Pieces are short seamless loops (cheaper **and** higher-quality for the models).
|
||||
- Only the **Right** transform uses paid generative v2v; Dark/Light/Left are free
|
||||
deterministic runtime operations.
|
||||
|
||||
Indicative offline v2v build (2026 rates — Kling 3.0 / Sora 2 Standard ≈ $0.10/s,
|
||||
Veo 3.1 Lite / Runway Turbo ≈ $0.05/s; ×~4 retry yield): **~$300–3k** for a small
|
||||
base library × a handful of restyle variants. A flat authoring subscription
|
||||
(Runway $28–76/mo, Kling from ~$6/mo) may be cheaper than per-second API for a
|
||||
one-time build.
|
||||
|
||||
---
|
||||
|
||||
## 10. Accessibility, i18n, and the translation-cost finding
|
||||
|
||||
The piece is operable **blind, in the dark, in your language**, via four redundant
|
||||
channels on the control panel: **touch** (engraved symbol shape), **low-light
|
||||
color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local*
|
||||
speaker, separate from the main audio system).
|
||||
|
||||
Because "the human experience from a communication perspective is not universal" is
|
||||
itself on-thesis, the piece supports **many languages**. The cost of that hinges on
|
||||
one architectural decision:
|
||||
|
||||
- **❌ Labels baked into the v2v render** → re-render every label-bearing variant
|
||||
per language → video cost **× number of languages** (≈ $9k raw / $27–45k with
|
||||
retries for ~50 languages). A budget landmine.
|
||||
- **✅ Labels as a runtime overlay** (which §4.2 already requires) → the v2v
|
||||
substrate is rendered **once**, language-agnostic; a new language is a translated
|
||||
string table + font + TTS voice. **Marginal video cost per language ≈ $0.**
|
||||
|
||||
So multilingual support costs only cheap text work: machine-translating a small
|
||||
label/script vocabulary into ~100 languages is a few dollars; optional human review
|
||||
of a curated tier is ~$0.10/word (~$50–200/language); TTS for the read-aloud button
|
||||
is pennies per language. **Net: supporting *every* language is essentially free on
|
||||
the expensive part, provided labels stay a runtime overlay.**
|
||||
|
||||
---
|
||||
|
||||
## 11. The physical control panel
|
||||
|
||||
A **wood** panel. Each control has a **tactile engraved symbol** below it —
|
||||
distinguishable by fingertip in the dark, and shaped to evoke the control's
|
||||
meaning — with **braille** beneath, a **dim color-coded LED** backlighting the
|
||||
engraving, and a **read-aloud button** that speaks the control's function on a
|
||||
small local speaker.
|
||||
|
||||
Layout principle: the **four experience knobs are two opposing pairs**, with the
|
||||
opposition encoded *in the touch* (angular vs flowing, heavy-down vs radiant-up).
|
||||
The **utility controls** use a more conventional AV shape-language so they are
|
||||
never confused with the experience four.
|
||||
|
||||
| Control | Engraved symbol | Touch | LED |
|
||||
|---|---|---|---|
|
||||
| **Left** (analytical) | parallel straight ridges / grid `≡` `▦` | flat, hard, orthogonal edges | — |
|
||||
| **Right** (emotional) | spiral / flowing wave `∿` | one continuous curve, no corners | — |
|
||||
| **Dark** (somber) | crescent / downward wedge `☾` `▼` | smooth solid mass, sinking | **blue** |
|
||||
| **Light** (serene) | radial sunburst `☀` | center with radial spikes | **yellow / white** |
|
||||
| **Content dial** (7) | per-position micro-icons: Off `○` · White-Noise `∴` · Music `♪` · Audio-Track `)))` · Video `▷` · Music+Video `♪▷` · A+V `▷)))` | detented clicks | neutral |
|
||||
| **Volume** | growing wedge `◁` | a slope that thickens | neutral |
|
||||
| **Brightness** | half-disc contrast `◐` | a circle, half raised | neutral |
|
||||
|
||||
Design notes:
|
||||
|
||||
- **Light vs Brightness** is the one collision trap (both are "bright"). Light =
|
||||
radial sunburst (a *mood*, glows the hue it imparts); Brightness = half-disc
|
||||
contrast (an *output level*) — distinct to finger and eye.
|
||||
- **LEDs equal the hue the knob imparts:** Dark glows blue, Light glows
|
||||
yellow/white, the mood center is neutral — tying the panel to the projection.
|
||||
- **Negative-space symbols:** symbols differ by gross **edge profile** (curve vs
|
||||
corner vs radial vs slope), not fine detail, so they read under a fingertip.
|
||||
|
||||
This substantially fleshes out **sub-project 4 (Arduino firmware / control panel)**,
|
||||
previously only sketched: 7 controls + a read-aloud button + LEDs + braille, with
|
||||
the Arduino reading positions and the read-aloud/LED behavior coordinated with the
|
||||
Pi.
|
||||
|
||||
---
|
||||
|
||||
## 12. Hardware (revises §6)
|
||||
|
||||
Unchanged from the 2026-06-04 single-pano revision except for the richer panel:
|
||||
|
||||
- **Arduino** — reads the 7-way content dial, 4 experience knobs, volume,
|
||||
brightness, and the read-aloud button; drives the panel LEDs; sends values to the
|
||||
Pi over USB serial. (Bigger than the original 5-value panel.)
|
||||
- **Raspberry Pi 5 (or mini-PC)** — holds the base library + altered variants,
|
||||
runs the alteration engine (runtime grading + overlay compositing + variant
|
||||
selection), drives the single panoramic projector, plays audio, and serves the
|
||||
read-aloud clips to the local panel speaker.
|
||||
- **Single panoramic projector** spanning the three walls — **unchanged**.
|
||||
- **Small local speaker** on/near the panel for the read-aloud button (separate
|
||||
from the main audio).
|
||||
|
||||
---
|
||||
|
||||
## 13. Roadmap impact
|
||||
|
||||
- **Sub-project 2 (ingest/tagging)** — reused to source the **neutral** base
|
||||
clips; coordinate *drafting* matters less (the base is neutral by selection), but
|
||||
mechanical tagging + license capture still apply; add a flag for "neutral base"
|
||||
vs "altered variant," and model audio *source*.
|
||||
- **Sub-project 3 (player)** — grows an **alteration engine**: runtime color grade
|
||||
(luma-keyed), runtime analytical-overlay compositor (multilingual, from string
|
||||
tables), generated white noise, and selection/blending of pre-baked v2v variants.
|
||||
- **Sub-project 4 (firmware/panel)** — grows from a 5-value panel to the full
|
||||
tactile/accessible panel in §11.
|
||||
- **New offline pipeline** — author neutral→restyled v2v variants (the only paid
|
||||
AI step) and the label/string tables + TTS per language.
|
||||
|
||||
---
|
||||
|
||||
## 14. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Pano resolution strategy** — v2v models output ~16:9 720p–1080p; the three-wall
|
||||
pano wants ultra-wide/high-res. Generate-at-max + upscale, native-ultrawide
|
||||
tooling, or accept the look?
|
||||
- **Negative-space luma key** — exact curve/mask for "brights→white, darks→black"
|
||||
on arbitrary nature footage.
|
||||
- **Transform composition order** — order of grade vs v2v substrate when several
|
||||
poles are active; how variant blending interpolates between pre-baked restyle
|
||||
levels.
|
||||
- **Neutral base count** — how many base clips for a satisfying launch.
|
||||
- **Language tier** — which languages get human-reviewed translation vs
|
||||
machine-only; default UI language and switching.
|
||||
- **Runtime grading performance** — confirm the Pi can do luma-keyed grade +
|
||||
overlay compositing at projector resolution/framerate (GPU shaders via mpv/ffmpeg).
|
||||
- **`approved`-only enforcement** in the player (carried from the original).
|
||||
|
||||
---
|
||||
|
||||
## 15. Out of scope (YAGNI)
|
||||
|
||||
- Networking / streaming at runtime — all alteration is offline; the room is local.
|
||||
- Runtime/on-demand generation (breaks all-local; slow; unbounded cost).
|
||||
- Audio alteration by knob (video-only this revision).
|
||||
- Multi-user / multi-viewer; session recording or analytics.
|
||||
- Automatic ML coordinate tagging — base clips are chosen neutral by hand.
|
||||
@@ -0,0 +1,242 @@
|
||||
# Human Experience Filter — Simulator Alteration Preview (Design)
|
||||
|
||||
**Date:** 2026-06-06
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Builds on:** the alteration engine of
|
||||
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
|
||||
(the "design" below) and the simulator scaffold of
|
||||
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md).
|
||||
**Revises the design:** §4.3 and §10 — the Left analytical HUD is **no longer a
|
||||
runtime overlay**; it is **baked into authored variant videos** (see §8 below).
|
||||
This is a deliberate trade of near-free multilingual support for authorial
|
||||
precision over the HUD.
|
||||
**Retires:** the simulator's selection-era "curator's X-ray" view and its
|
||||
`/api/select` + `/api/catalog/meta` endpoints (the selection model they
|
||||
visualize was superseded by the alteration model).
|
||||
|
||||
> **Why this exists.** Operator directive (session 0007): *build and design only
|
||||
> things that run in the simulator, and get the whole experience working the way
|
||||
> we like in the simulator before moving to hardware.* The slice-1 alteration
|
||||
> engine (PR #5) is pure logic with **no simulator surface** — you cannot yet turn
|
||||
> the experience knobs and *see* the result. This design brings the alteration
|
||||
> into the simulator so the look can be tuned and liked before any Pi/serial work.
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope
|
||||
|
||||
A browser **alteration preview**: turn the four experience knobs and the content
|
||||
dial, and see the neutral base footage altered toward the knob state, in real
|
||||
time, on a looping clip. The purpose is to **tune the look** of the filter and to
|
||||
**settle the knob→strength calibration by eye** (the open §3-vs-§4.2/§5 question
|
||||
from session 0006).
|
||||
|
||||
**In scope**
|
||||
- Live preview of the four experience knobs (Left/Right/Dark/Light) on a looping
|
||||
base clip.
|
||||
- **Dark/Light** rendered as a live, deterministic runtime color grade.
|
||||
- **Left/Right** rendered by selecting a **pre-baked authored variant clip** from
|
||||
a 5×5 grid and crossfading on change.
|
||||
- A **calibration panel** that adjusts the grade curve live; the chosen values are
|
||||
baked back into `player/alteration.py` defaults.
|
||||
- A **RenderPlan readout** (the project's "X-ray" honesty): always show the exact
|
||||
numbers the engine produced.
|
||||
- The content dial's **video on/off** behavior (so "Off" goes to black), driven by
|
||||
the existing `resolve_content`.
|
||||
- Placeholder variant generation so the mechanism is testable before the operator
|
||||
authors real videos.
|
||||
|
||||
**Out of scope (this slice)** — real generative v2v; serial input / the 3⇄4
|
||||
framing contract; audio playback (music / white-noise / audio-track); the Pi/mpv
|
||||
runtime renderer; the full transition/crossfade timing engine; multilingual label
|
||||
tables; catalog-model changes. These remain later roadmap slices.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture — Python-canonical, thin browser renderer
|
||||
|
||||
`player/alteration.py` stays the **single source of truth** for the alteration
|
||||
math (handbook §4.2 — deterministic core, thin I/O). The browser owns only
|
||||
*rendering*. Data flow:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Browser
|
||||
K[4 experience knobs +<br/>content dial +<br/>calibration sliders]
|
||||
V["<video> variant + canvas grade"]
|
||||
RO[RenderPlan readout]
|
||||
end
|
||||
subgraph FastAPI [simulator/app.py]
|
||||
EP["POST /api/alteration"]
|
||||
CL["GET /api/clips"]
|
||||
end
|
||||
ENG["player.alteration.plan_alteration(coord, calibration)"]
|
||||
K -- "debounced POST {controls, calibration}" --> EP
|
||||
EP --> ENG --> EP -- "RenderPlan {variant, grade}" --> V
|
||||
EP --> RO
|
||||
CL -- "base clip + variant manifest" --> V
|
||||
```
|
||||
|
||||
- The browser sends control/calibration changes (debounced) and receives a
|
||||
`RenderPlan`. Video filtering itself runs continuously in the browser; only a
|
||||
*plan recompute* makes a round-trip. On localhost these JSON round-trips are
|
||||
imperceptible.
|
||||
- "Bake the calibration winner in" = change the `Calibration` defaults in Python.
|
||||
Nothing in the browser is canonical.
|
||||
|
||||
---
|
||||
|
||||
## 3. The four axes — how each is rendered
|
||||
|
||||
| Axis | Engine output | Browser rendering |
|
||||
|---|---|---|
|
||||
| **Dark / Light** | `ColorGrade.tone` ∈ [−1, 1], center = identity | live color grade on the `<video>`: Light → warm/yellow + negative space toward white; Dark → cool/blue + negative space toward black; 0 = raw. (Confirmed direction in brainstorm.) |
|
||||
| **Left / Right** | `VariantRef(left, right)` | select the authored variant clip for `(left, right)`; **crossfade** when it changes; `(0,0)` → raw base. The analytical HUD (Left) and dreamlike restyle (Right) are **baked into the clip** (§8). |
|
||||
|
||||
Dark/Light is the only continuously-tunable axis in this slice; Left/Right is a
|
||||
discrete selection by coordinate.
|
||||
|
||||
---
|
||||
|
||||
## 4. The variant grid (per base clip)
|
||||
|
||||
Each base clip carries a 5×5 grid keyed by the two baked axes:
|
||||
|
||||
- Rows = **Left** (analytical / HUD), 0–4. Columns = **Right** (artistic /
|
||||
dreamlike restyle), 0–4.
|
||||
- `(0,0)` = the raw base clip — no authored file needed.
|
||||
- The other **24 cells are authored videos** (4 analytical-edge + 4 dreamlike-edge
|
||||
+ 16 combined core). This is **per base clip**; authoring load multiplies by the
|
||||
number of base clips (and by language, since the HUD is baked — §8).
|
||||
|
||||
The grid is **data**, not code: a manifest the simulator reads (see §6).
|
||||
|
||||
---
|
||||
|
||||
## 5. Engine reconciliation (`player/alteration.py`)
|
||||
|
||||
The slice-1 engine modeled Left/Right as continuous runtime layers
|
||||
(`AnalyticalOverlay.intensity`, `Restyle.blend`). Baked variants make that
|
||||
obsolete. Changes:
|
||||
|
||||
- **`RenderPlan`** becomes `{ grade: ColorGrade, variant: VariantRef }`.
|
||||
- `ColorGrade` — unchanged.
|
||||
- `VariantRef(left, right)` — new; identity selection by coordinate.
|
||||
- **Remove** `AnalyticalOverlay` and `Restyle` (their content is now baked into the
|
||||
variant clip) rather than leaving dead fields.
|
||||
- **`Calibration`** — a new frozen dataclass parameterizing the grade curve (e.g.
|
||||
`mood_center`, per-axis curve), defaulting to **today's exact behavior**
|
||||
(behavior-preserving). `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`.
|
||||
- **`player/state.py`** — the `CROSSFADE` trigger switches from "restyle changed"
|
||||
to "variant changed"; `LIVE_UPDATE` still covers a grade-only change. Video
|
||||
on/off fades unchanged.
|
||||
|
||||
These are framework-code changes that keep the engine pure and unit-tested; they
|
||||
do **not** bake any deployment-shape decision into the engine.
|
||||
|
||||
---
|
||||
|
||||
## 6. Data & endpoints
|
||||
|
||||
- **`simulator/clips.py`** (replaces `simulator/fixtures.py`): reads a base-clip +
|
||||
variant manifest. Each base clip: `{ id, title, base_file, license, source,
|
||||
variants: { "L,R": { file, model?, hud_lang? } } }`. Missing cells fall back to
|
||||
the raw base (and are flagged in the readout as "raw / unauthored").
|
||||
- **`POST /api/alteration`**: body `{ controls, calibration }` → `RenderPlan`
|
||||
(plus the `ContentResolution` from `resolve_content`, so the dial's video-on/off
|
||||
is honored). The endpoint calls the real `plan_alteration`.
|
||||
- **`GET /api/clips`**: the base-clip list + variant manifest for the active base.
|
||||
- Video files served as static assets from a sample-media directory.
|
||||
- **Removed**: `POST /api/select`, `GET /api/catalog/meta`.
|
||||
- `hef.selection` (the library, incl. `Coordinate`) is **untouched**; only the
|
||||
simulator's selection view/endpoints retire. (`ranked_candidates`, added for the
|
||||
old X-ray, may become unused by the simulator — noted, not removed here.)
|
||||
|
||||
---
|
||||
|
||||
## 7. Calibration tuning
|
||||
|
||||
The calibration panel exposes the grade-curve parameters (the two conventions in
|
||||
tension: `(2,2,2,2)`-centered vs the `value/4` reading currently implemented, plus
|
||||
curve shape). Changing them re-requests the plan and the footage responds live.
|
||||
When the operator is happy, the chosen values are written into
|
||||
`DEFAULT_CALIBRATION` in `player/alteration.py` and locked in by a unit test.
|
||||
|
||||
---
|
||||
|
||||
## 8. Baked HUD — the §4.3/§10 revision (accepted trade)
|
||||
|
||||
The design's §4.3/§10 kept the Left HUD a **runtime overlay** specifically so the
|
||||
v2v substrate is language-agnostic and multilingual support is near-free. This
|
||||
design **reverses that for the Left/Right plane**: the operator authors the HUD
|
||||
**into** the variant videos to get pixel-precise control over the HUD and the
|
||||
analytical↔feeling balance.
|
||||
|
||||
**Consequences (accepted):**
|
||||
- Multilingual support is **no longer near-free**: baked HUD text means a new
|
||||
language re-renders every label-bearing variant (the §10 "budget landmine").
|
||||
The piece is English-first; broad i18n is deferred/expensive.
|
||||
- The live Dark/Light grade is applied **on top** of the baked cell, so it **tints
|
||||
the baked HUD** too (a "Dark" mood cools/darkens the HUD). With baked overlays
|
||||
the grade cannot skip the HUD. Escape hatch: author HUD colors that survive
|
||||
grading, or (future) reinstate a runtime HUD layer.
|
||||
|
||||
The parent design's §4.3/§10 must be updated to point at this revision (a task for
|
||||
the implementation plan).
|
||||
|
||||
---
|
||||
|
||||
## 9. Bootstrapping before authored videos exist
|
||||
|
||||
The operator will author the 24 variants per base clip; none exist yet. So the
|
||||
mechanism is testable immediately, the build includes a **placeholder generator**:
|
||||
for at least one base clip, produce the 24 cells by burning the cell's `L,R` (and
|
||||
a stub HUD caption) into the base loop via ffmpeg. Real authored clips drop into
|
||||
the manifest with **no code change**.
|
||||
|
||||
---
|
||||
|
||||
## 10. Testing
|
||||
|
||||
- **`player/` unit tests:** `DEFAULT_CALIBRATION` is behavior-preserving vs the
|
||||
current helpers; the two calibration conventions produce the expected plans;
|
||||
`VariantRef` selection (incl. `(0,0)` → raw) and the `state.py` crossfade
|
||||
trigger on variant change.
|
||||
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
|
||||
`POST /api/alteration` returns the engine's plan for given controls/calibration;
|
||||
`GET /api/clips` returns the manifest; the removed endpoints are gone.
|
||||
- **No browser/E2E automation** this slice (manual visual tuning is the point);
|
||||
the rendering JS is kept thin and the engine logic stays in tested Python.
|
||||
|
||||
---
|
||||
|
||||
## 11. What ships
|
||||
|
||||
- `player/alteration.py` (parameterized `Calibration`, `VariantRef`, slimmed
|
||||
`RenderPlan`) + `player/state.py` crossfade-trigger update.
|
||||
- `simulator/clips.py` (variant manifest) replacing `fixtures.py`.
|
||||
- `simulator/app.py` new endpoints; selection endpoints removed.
|
||||
- `simulator/static/` rewritten as the Player preview (variant `<video>` +
|
||||
crossfade, live grade, calibration panel, RenderPlan readout, content dial).
|
||||
- Placeholder-variant generator + a sample base clip.
|
||||
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten;
|
||||
parent design §4.3/§10 pointer updated; `docs/ROADMAP.md` §3 updated.
|
||||
|
||||
---
|
||||
|
||||
## 12. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Grade vs baked HUD interaction** — accepted that the grade tints the HUD;
|
||||
revisit only if it reads badly once real authored clips exist.
|
||||
- **Sample base clip** — pick one CC0 nature loop for the placeholder grid.
|
||||
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for
|
||||
tuning; the real timing engine is a later slice.
|
||||
|
||||
---
|
||||
|
||||
## 13. Out of scope (YAGNI)
|
||||
|
||||
Real generative v2v; serial / firmware; audio playback; the Pi renderer; the full
|
||||
transition engine; multilingual tables; catalog-model changes; retiring
|
||||
`hef.selection.ranked_candidates`. All remain later roadmap work.
|
||||
@@ -0,0 +1,25 @@
|
||||
Feature: Synthetic fixture catalog
|
||||
As a curator with an empty real catalog
|
||||
I want a dense, representative stand-in catalog generated on demand
|
||||
So that I can feel nearest-match behavior everywhere before any media is ingested
|
||||
|
||||
Scenario: The fixture catalog spans the coordinate space and all content modes
|
||||
When the fixture catalog is generated
|
||||
Then it contains records covering the brain plane and the mood plane
|
||||
And it contains records in each of the "audio", "video", and "av" modes
|
||||
And it contains a mix of "proposed" and "approved" records
|
||||
|
||||
Scenario: The fixture catalog is valid
|
||||
When the fixture catalog is generated
|
||||
Then every record passes hef.catalog validation
|
||||
And no record references real media
|
||||
|
||||
Scenario: Generation is deterministic
|
||||
Given a fixed seed
|
||||
When the fixture catalog is generated twice
|
||||
Then both runs produce identical records
|
||||
|
||||
Scenario: The real catalog is used when it is populated
|
||||
Given catalog/library.jsonl is non-empty
|
||||
When the simulator starts pointed at the real catalog
|
||||
Then it loads the real records through the same loader instead of the fixtures
|
||||
@@ -0,0 +1,32 @@
|
||||
Feature: Feeling the model's own parameters
|
||||
As a curator tuning the algorithm, not just the curation
|
||||
I want to adjust the selection model's parameters live
|
||||
So that I can feel how weighting, pool size, and the approved gate reshape picks
|
||||
|
||||
Background:
|
||||
Given the simulator is loaded with the synthetic fixture catalog
|
||||
And the dials are set to a fixed knob position with mode "av"
|
||||
|
||||
Scenario: Reweighting the brain plane can change the winner
|
||||
Given the brain and mood weights are equal
|
||||
And the current pick is recorded
|
||||
When I increase the brain weight relative to the mood weight
|
||||
Then the ranked pool may reorder so that brain-plane distance dominates
|
||||
And the picked piece can differ from the recorded pick
|
||||
|
||||
Scenario: Pool size controls how many candidates are shown and shuffled
|
||||
When I set the pool size to 1
|
||||
Then the pool shows exactly the single nearest candidate
|
||||
When I set the pool size to 5
|
||||
Then the pool shows up to the five nearest candidates
|
||||
|
||||
Scenario: Approved-only narrows the pool to human-blessed records
|
||||
Given the fixture catalog contains both "proposed" and "approved" records
|
||||
When I turn the approved-only toggle on
|
||||
Then every candidate in the pool has review_status "approved"
|
||||
When I turn the approved-only toggle off
|
||||
Then "proposed" records may appear in the pool again
|
||||
|
||||
Scenario: Default model parameters reproduce the shipped selection behavior
|
||||
Given the weights are equal, the pool size is the default, and approved-only is off
|
||||
Then the picked piece matches what hef.selection.select would return for the same dials
|
||||
@@ -0,0 +1,48 @@
|
||||
Feature: Curator's X-ray of the selection model
|
||||
As a curator
|
||||
I want to turn the installation's dials against a representative catalog and see
|
||||
the full ranked pool the nearest-match algorithm picks from
|
||||
So that I can judge whether the tagging rubric and selection surface fitting pieces
|
||||
|
||||
Background:
|
||||
Given the simulator is loaded with the synthetic fixture catalog
|
||||
And the fixture catalog spans the brain plane and the mood plane across all content modes
|
||||
|
||||
Scenario: A pick is always accompanied by its candidate pool
|
||||
When I set the mode to "av" and the dials to left=1 right=3 dark=4 light=1
|
||||
Then a piece is picked
|
||||
And the ranked pool is shown nearest-first with each candidate's distance
|
||||
And the picked piece is the rank-1 candidate in the pool
|
||||
|
||||
Scenario: Turning Dark up while Light stays low pulls the pick toward somber pieces
|
||||
Given the mode is "av" and the dials are left=2 right=2 dark=0 light=0
|
||||
When I raise the Dark dial from 0 to 4
|
||||
Then the picked piece's dark coordinate is higher than before
|
||||
And the pool's nearest candidates cluster toward the high-dark / low-light region
|
||||
|
||||
Scenario: Turning a dial moves the knob point on the coordinate map
|
||||
Given the mode is "av"
|
||||
When I change any of the Left, Right, Dark, or Light dials
|
||||
Then the knob point on the brain or mood grid map moves to the new coordinate
|
||||
And the candidate dots reflect the new neighborhood
|
||||
|
||||
Scenario: None mode is the void / rest state
|
||||
When I set the mode to "none"
|
||||
Then no piece is picked
|
||||
And the pool is empty
|
||||
And the screen shows the void / rest state
|
||||
|
||||
Scenario: A+V mode falls back to audio and video when av records are thin
|
||||
Given the fixture catalog has fewer than the pool size of native "av" records near the knob point
|
||||
When I set the mode to "av"
|
||||
Then the pool includes "audio" and "video" records so it is not starved
|
||||
|
||||
Scenario Outline: Each content mode restricts the pool to eligible records
|
||||
When I set the mode to "<mode>"
|
||||
Then every candidate in the pool is eligible for "<mode>"
|
||||
|
||||
Examples:
|
||||
| mode |
|
||||
| audio |
|
||||
| video |
|
||||
| av |
|
||||
@@ -0,0 +1,27 @@
|
||||
Feature: Simulator service and API
|
||||
As an operator
|
||||
I want to run the simulator locally in one container and drive it over a small API
|
||||
So that the browser X-ray and its behavior are testable without the physical stack
|
||||
|
||||
Scenario: The service runs locally in a container
|
||||
When I start the simulator container
|
||||
Then the X-ray is reachable in a browser on localhost
|
||||
And no external network access is required
|
||||
|
||||
Scenario: Selecting returns the pick and the ranked pool
|
||||
When I POST a valid selection request with dials and mode "av"
|
||||
Then the response contains a picked record
|
||||
And the response contains the ranked pool with a distance and rank per candidate
|
||||
|
||||
Scenario: None mode returns the void over the API
|
||||
When I POST a selection request with mode "none"
|
||||
Then the response pick is null
|
||||
And the response pool is empty
|
||||
|
||||
Scenario: Invalid dial values are rejected
|
||||
When I POST a selection request with a dial value outside 0 to 4
|
||||
Then the API responds with a validation error
|
||||
|
||||
Scenario: Catalog metadata is available for the header
|
||||
When I GET the catalog metadata
|
||||
Then the response reports record counts, a per-mode breakdown, and the proposed/approved split
|
||||
+42
-10
@@ -58,6 +58,38 @@ def candidates_for_mode(records, mode: str, pool_size: int) -> list[Record]:
|
||||
from typing import Optional
|
||||
|
||||
|
||||
def ranked_candidates(
|
||||
records,
|
||||
coord: Coordinate,
|
||||
mode: str,
|
||||
*,
|
||||
pool_size: int = 4,
|
||||
weights: Weights = Weights(),
|
||||
approved_only: bool = False,
|
||||
) -> list[tuple[Record, float]]:
|
||||
"""Mode-eligible records sorted nearest-first, each paired with its distance.
|
||||
|
||||
Applies the same approved_only filter and 'av' pool fallback as select(),
|
||||
then returns up to pool_size nearest (record, distance) pairs. This is the
|
||||
single source of truth for "the pool"; select() picks from it.
|
||||
"""
|
||||
if mode not in CONTENT_MODES:
|
||||
raise ValueError(
|
||||
f"ranked_candidates expects a content mode {sorted(CONTENT_MODES)}, "
|
||||
f"got {mode!r}"
|
||||
)
|
||||
pool = records
|
||||
if approved_only:
|
||||
pool = [r for r in pool if r.review_status == "approved"]
|
||||
candidates = candidates_for_mode(pool, mode, pool_size)
|
||||
ranked = sorted(
|
||||
candidates,
|
||||
key=lambda r: (distance(coord, record_coordinate(r), weights), r.id),
|
||||
)
|
||||
nearest = ranked[:pool_size]
|
||||
return [(r, distance(coord, record_coordinate(r), weights)) for r in nearest]
|
||||
|
||||
|
||||
def select(
|
||||
records,
|
||||
coord: Coordinate,
|
||||
@@ -82,17 +114,17 @@ def select(
|
||||
)
|
||||
if mode == "none":
|
||||
return None
|
||||
pool = records
|
||||
if approved_only:
|
||||
pool = [r for r in pool if r.review_status == "approved"]
|
||||
candidates = candidates_for_mode(pool, mode, pool_size)
|
||||
if not candidates:
|
||||
return None
|
||||
ranked = sorted(
|
||||
candidates,
|
||||
key=lambda r: (distance(coord, record_coordinate(r), weights), r.id),
|
||||
ranked = ranked_candidates(
|
||||
records,
|
||||
coord,
|
||||
mode,
|
||||
pool_size=pool_size,
|
||||
weights=weights,
|
||||
approved_only=approved_only,
|
||||
)
|
||||
nearest = ranked[:pool_size]
|
||||
if not ranked:
|
||||
return None
|
||||
nearest = [r for r, _ in ranked]
|
||||
if rng is None:
|
||||
return nearest[0]
|
||||
return rng.choice(nearest)
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
|
||||
|
||||
Replaces the 2026-06-04 nearest-match *selection* with a *transformation* of a
|
||||
neutral base clip. Given the four experience knobs, it produces three layers
|
||||
that compose per §4.2:
|
||||
|
||||
- Substrate transforms (alter pixels, blend with each other):
|
||||
* Restyle -- the Right axis: a pre-baked generative v2v dreamlike restyle.
|
||||
* ColorGrade -- the mood axis (Dark/Light): a deterministic color grade.
|
||||
- Overlay (composited on top of the substrate):
|
||||
* AnalyticalOverlay -- the Left axis: HUD/labels/measurement.
|
||||
|
||||
Left and Right are NOT opposites; they live on different layers and stack
|
||||
(§4.2). Dark and Light are the two poles of one mood grade whose center is the
|
||||
identity (§5).
|
||||
|
||||
Calibration note: the knob->strength curves below are the single source of
|
||||
truth for how a 0..4 position maps to a transform strength. See the session
|
||||
0006 transcript Deferred decisions for the §3-vs-§4.2/§5 reconciliation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from hef.selection import Coordinate
|
||||
|
||||
KNOB_MAX = 4 # knob full-scale (0..4)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ColorGrade:
|
||||
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
|
||||
<0 cool blue->black (dark), 0 = identity (raw, ungraded)."""
|
||||
|
||||
tone: float
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
return self.tone == 0.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AnalyticalOverlay:
|
||||
"""Left axis (§4.1): analytical HUD/annotation/labels, composited on top.
|
||||
`intensity` 0..1 (0 = no overlay)."""
|
||||
|
||||
intensity: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Restyle:
|
||||
"""Right axis (§4.1): pre-baked generative v2v dreamlike substrate.
|
||||
`blend` 0..1 (0 = raw substrate, no restyle)."""
|
||||
|
||||
blend: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RenderPlan:
|
||||
"""The full layered alteration for one knob vector (§4.2)."""
|
||||
|
||||
grade: ColorGrade
|
||||
overlay: AnalyticalOverlay
|
||||
restyle: Restyle
|
||||
|
||||
@property
|
||||
def is_identity(self) -> bool:
|
||||
"""True when the plan leaves the neutral base un-altered."""
|
||||
return (
|
||||
self.grade.is_identity
|
||||
and self.overlay.intensity == 0.0
|
||||
and self.restyle.blend == 0.0
|
||||
)
|
||||
|
||||
|
||||
def _overlay_intensity(left: int) -> float:
|
||||
"""Left knob -> analytical-overlay intensity (0..1)."""
|
||||
return left / KNOB_MAX
|
||||
|
||||
|
||||
def _restyle_blend(right: int) -> float:
|
||||
"""Right knob -> v2v restyle blend (0..1)."""
|
||||
return right / KNOB_MAX
|
||||
|
||||
|
||||
def _mood_tone(dark: int, light: int) -> float:
|
||||
"""(dark, light) -> signed mood grade in [-1, 1]; equal -> 0 identity (§5)."""
|
||||
return (light - dark) / KNOB_MAX
|
||||
|
||||
|
||||
def plan_alteration(coord: Coordinate) -> RenderPlan:
|
||||
"""Map a knob vector to its layered RenderPlan (design §4)."""
|
||||
return RenderPlan(
|
||||
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light)),
|
||||
overlay=AnalyticalOverlay(intensity=_overlay_intensity(coord.left)),
|
||||
restyle=Restyle(blend=_restyle_blend(coord.right)),
|
||||
)
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Resolve the 7-way content dial into an audio source + video on/off (§6).
|
||||
|
||||
The single source of truth for design §6's table: which audio source plays and
|
||||
whether the projector shows video, for each dial position.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
# The four distinct audio sources. "white_noise" is generated at runtime;
|
||||
# "music" is the public-domain classical pool; "audio_track" is the clip's own
|
||||
# field audio; "none" is silence.
|
||||
AUDIO_SOURCES = frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContentResolution:
|
||||
audio_source: str
|
||||
video: bool
|
||||
|
||||
|
||||
# Design §6, one row per dial position.
|
||||
_TABLE = {
|
||||
"off": ContentResolution("none", False),
|
||||
"white_noise": ContentResolution("white_noise", False),
|
||||
"music": ContentResolution("music", False),
|
||||
"audio_track": ContentResolution("audio_track", False),
|
||||
"video": ContentResolution("none", True),
|
||||
"music_video": ContentResolution("music", True),
|
||||
"audio_video": ContentResolution("audio_track", True),
|
||||
}
|
||||
|
||||
|
||||
def resolve_content(position: str) -> ContentResolution:
|
||||
"""Map a content-dial position to its audio source and video flag (§6)."""
|
||||
try:
|
||||
return _TABLE[position]
|
||||
except KeyError:
|
||||
raise ValueError(
|
||||
f"unknown content position {position!r}; expected one of {sorted(_TABLE)}"
|
||||
) from None
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Control-panel state: the data shape read from the Arduino over serial.
|
||||
|
||||
This is the serial-contract payload shared with sub-project 4 (firmware). It
|
||||
models the full panel from design §6/§7: the 7-way content dial, the four
|
||||
experience knobs (0..4), and the two intensity levels (volume, brightness).
|
||||
The wire framing itself is the separate 3<->4 serial contract; this module is
|
||||
the *decoded* form.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, fields
|
||||
|
||||
# The seven positions of the content dial (design §6).
|
||||
CONTENT_POSITIONS = frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
|
||||
KNOB_FIELDS = ("left", "right", "dark", "light", "volume", "brightness")
|
||||
KNOB_MIN = 0
|
||||
KNOB_MAX = 4
|
||||
|
||||
|
||||
class ControlsError(ValueError):
|
||||
"""Raised when a Controls payload is structurally invalid."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Controls:
|
||||
content: str
|
||||
left: int
|
||||
right: int
|
||||
dark: int
|
||||
light: int
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
|
||||
def validate_controls(c: Controls) -> None:
|
||||
"""Raise ControlsError if the payload is structurally invalid."""
|
||||
if c.content not in CONTENT_POSITIONS:
|
||||
raise ControlsError(
|
||||
f"invalid content position {c.content!r}; "
|
||||
f"expected one of {sorted(CONTENT_POSITIONS)}"
|
||||
)
|
||||
for name in KNOB_FIELDS:
|
||||
value = getattr(c, name)
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise ControlsError(f"knob {name} must be an int, got {value!r}")
|
||||
if not (KNOB_MIN <= value <= KNOB_MAX):
|
||||
raise ControlsError(
|
||||
f"knob {name}={value} out of range {KNOB_MIN}..{KNOB_MAX}"
|
||||
)
|
||||
|
||||
|
||||
def parse_controls(data: dict) -> Controls:
|
||||
"""Build a validated Controls from a decoded mapping, rejecting unknown or
|
||||
missing keys."""
|
||||
known = {f.name for f in fields(Controls)}
|
||||
unknown = set(data) - known
|
||||
if unknown:
|
||||
raise ControlsError(f"unknown keys: {sorted(unknown)}")
|
||||
missing = known - set(data)
|
||||
if missing:
|
||||
raise ControlsError(f"missing keys: {sorted(missing)}")
|
||||
c = Controls(**data)
|
||||
validate_controls(c)
|
||||
return c
|
||||
+120
@@ -0,0 +1,120 @@
|
||||
"""The player state machine: a stream of Controls -> playback transitions.
|
||||
|
||||
Pure decision logic — no mpv, no audio, no serial. Each update() resolves the
|
||||
desired Playback (which neutral base clip, how it is altered, what audio plays,
|
||||
at what levels) and returns the Transition from the previous Playback. The
|
||||
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
|
||||
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
|
||||
pre-baked Right v2v variant needs a CROSSFADE, and toggling video on/off
|
||||
fades to/from black.
|
||||
|
||||
Knobs no longer *select* a clip (the base library is neutral by construction);
|
||||
they *transform* it. Which neutral base to show is an injected policy
|
||||
(`choose_base`), defaulting to the first clip; richer rotation is a later slice.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable, Optional
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import RenderPlan, plan_alteration
|
||||
from player.content import ContentResolution, resolve_content
|
||||
from player.controls import Controls
|
||||
|
||||
|
||||
class TransitionKind:
|
||||
NONE = "none"
|
||||
LIVE_UPDATE = "live_update"
|
||||
CROSSFADE = "crossfade"
|
||||
FADE_TO_BLACK = "fade_to_black"
|
||||
FADE_FROM_BLACK = "fade_from_black"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Playback:
|
||||
"""What should currently be playing."""
|
||||
|
||||
clip_id: Optional[str] # None = black walls
|
||||
plan: Optional[RenderPlan] # None when black
|
||||
content: ContentResolution
|
||||
volume: int
|
||||
brightness: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Transition:
|
||||
kind: str
|
||||
playback: Playback
|
||||
|
||||
|
||||
def _first(library):
|
||||
if not library:
|
||||
raise ValueError("no base clips available to play video")
|
||||
return library[0]
|
||||
|
||||
|
||||
_BLACK = Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=ContentResolution("none", False),
|
||||
volume=0,
|
||||
brightness=0,
|
||||
)
|
||||
|
||||
|
||||
class Player:
|
||||
def __init__(
|
||||
self,
|
||||
base_library,
|
||||
*,
|
||||
choose_base: Callable = _first,
|
||||
approved_only: bool = False,
|
||||
):
|
||||
self._library = list(base_library)
|
||||
self._choose_base = choose_base
|
||||
self._approved_only = approved_only # reserved for catalog-backed libs
|
||||
self._current = _BLACK
|
||||
|
||||
def _resolve(self, controls: Controls) -> Playback:
|
||||
content = resolve_content(controls.content)
|
||||
if not content.video:
|
||||
return Playback(
|
||||
clip_id=None,
|
||||
plan=None,
|
||||
content=content,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
clip = self._choose_base(self._library)
|
||||
coord = Coordinate(controls.left, controls.right, controls.dark, controls.light)
|
||||
return Playback(
|
||||
clip_id=clip.id,
|
||||
plan=plan_alteration(coord),
|
||||
content=content,
|
||||
volume=controls.volume,
|
||||
brightness=controls.brightness,
|
||||
)
|
||||
|
||||
def _classify(self, prev: Playback, nxt: Playback) -> str:
|
||||
prev_video = prev.clip_id is not None
|
||||
next_video = nxt.clip_id is not None
|
||||
if prev == nxt:
|
||||
return TransitionKind.NONE
|
||||
if prev_video and not next_video:
|
||||
return TransitionKind.FADE_TO_BLACK
|
||||
if next_video and not prev_video:
|
||||
return TransitionKind.FADE_FROM_BLACK
|
||||
if next_video and prev_video:
|
||||
if nxt.clip_id != prev.clip_id or nxt.plan.restyle != prev.plan.restyle:
|
||||
return TransitionKind.CROSSFADE
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
# both black: only audio/levels could have changed
|
||||
return TransitionKind.LIVE_UPDATE
|
||||
|
||||
def update(self, controls: Controls) -> Transition:
|
||||
nxt = self._resolve(controls)
|
||||
kind = self._classify(self._current, nxt)
|
||||
self._current = nxt
|
||||
return Transition(kind=kind, playback=nxt)
|
||||
+8
-1
@@ -12,8 +12,15 @@ requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[tool.setuptools]
|
||||
packages = ["hef", "tools"]
|
||||
packages = ["hef", "tools", "simulator", "player"]
|
||||
|
||||
[project.scripts]
|
||||
hef-ingest = "tools.ingest_cli:main"
|
||||
hef-review = "tools.review_cli:main"
|
||||
|
||||
[project.optional-dependencies]
|
||||
sim = [
|
||||
"fastapi>=0.110",
|
||||
"uvicorn[standard]>=0.29",
|
||||
"httpx>=0.27",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# Session 0003.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-04T07-57 (PST)
|
||||
> End: 2026-06-05T03-36 (PST)
|
||||
> Type: discovery
|
||||
> Status: **FINALIZED**
|
||||
|
||||
## Launch prompt
|
||||
|
||||
> We should create a simulator, just something that is web-based, potentially that
|
||||
> runs on localhost, running in a Docker container. It allows us to play virtually
|
||||
> with the dials and see the content that has been curated, and then we can get to
|
||||
> the actual physical stack. I think we want to make sure that we have the
|
||||
> principles well done and the interface figured out, and then the middle will come
|
||||
> in to play.
|
||||
|
||||
## Plan
|
||||
|
||||
Discovery session: explore the proposed **web experience simulator** and produce
|
||||
its first artifacts — BDD scenarios and/or a spec — plus (chosen mid-session) an
|
||||
implementation plan. Orientation:
|
||||
|
||||
```
|
||||
/goal Discover the web-based experience simulator — produce BDD scenarios and/or
|
||||
a spec, per human-experience-filter-art/docs/ROADMAP.md
|
||||
```
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` clean and pushed at `b7f19e0`. Sub-projects 1 (catalog/selection) and 2
|
||||
(ingest/review) shipped; physical stack (3 Pi player, 4 firmware) ahead.
|
||||
- `catalog/library.jsonl` empty (0 bytes) — tooling exists, no media ingested.
|
||||
- No prior simulator artifacts. Memory pointed at sub-project 3 as next.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Session gate / routing.** Opening prompt straddled coding vs. discovery;
|
||||
asked the operator → **discovery**. Ran `wgl-discovery-session-init`: claimed
|
||||
ID **0003** (`--type discovery`), verified clean pushed `main`, read memory,
|
||||
oriented from `docs/ROADMAP.md`.
|
||||
2. **Brainstorming.** Accepted the visual companion (server on :61057). Settled
|
||||
scope via one-at-a-time questions:
|
||||
- Primary purpose → **feel the selection model** (curator's instrument).
|
||||
- Catalog source → **synthetic fixture catalog** (real catalog is empty).
|
||||
- Screen layout (browser mockups A/B/C) → **B · Curator's X-ray** (full
|
||||
transparency: pick + ranked pool + distances + brain/mood grid maps).
|
||||
- Approved the **additive `hef.selection.ranked_candidates()`** change.
|
||||
- **Include** the model knobs (weights, pool size, approved-only).
|
||||
3. **Architecture.** Chose **Approach 1** — FastAPI + vanilla-JS, real
|
||||
`hef.selection` as single source of truth; Docker on localhost.
|
||||
4. **Artifacts written + committed** (`76a9526`): design spec
|
||||
`docs/superpowers/specs/2026-06-04-experience-simulator-design.md` + 4 BDD
|
||||
feature files under `features/`. Self-reviewed; fixed the pick-determinism
|
||||
ambiguity (deterministic `rng=None` pick; pool = the room's shuffle-set).
|
||||
Operator approved the spec.
|
||||
5. **Implementation plan** (chose "draft the plan now"): wrote
|
||||
`docs/superpowers/plans/2026-06-04-experience-simulator.md` — 7 bite-sized TDD
|
||||
tasks; self-reviewed for spec coverage / placeholders / type consistency.
|
||||
Committed (`e7ab227`).
|
||||
6. **Memory + handoff.** Saved resume pointer
|
||||
`experience-simulator-ready-to-build.md` (+ MEMORY.md index) so the next
|
||||
coding session auto-surfaces the `/executing-plans` line.
|
||||
7. **Finalize.** Push raced twice against concurrent session-claim commits (0003
|
||||
then 0004); rebased onto each (disjoint paths, non-destructive) and pushed.
|
||||
|
||||
## Cut state (end of session)
|
||||
|
||||
- `main` synced with origin. Discovery artifacts landed:
|
||||
- spec `40d7434`→`9cb4fb6` (rebased): `docs/superpowers/specs/2026-06-04-experience-simulator-design.md` + `features/*.feature`
|
||||
- plan `19a2dbc`→`ddec45d` (rebased): `docs/superpowers/plans/2026-06-04-experience-simulator.md`
|
||||
- Final `main` tip: **`ddec45d`**.
|
||||
- No code written this session (discovery only). Test suite untouched (baseline
|
||||
111 passed / 2 skipped).
|
||||
- Visual-companion files persist under `.superpowers/brainstorm/` (gitignored).
|
||||
|
||||
## What lands on the operator's plate
|
||||
|
||||
- **Build is queued, not started.** The simulator exists only as spec + BDDs +
|
||||
plan. Next session executes the plan.
|
||||
- **Structural gap (flagged, not blocking):** this app's `app.json` has no
|
||||
`contains:["bdd"]` or `contains:["spec-rfc"]` repo, so there is no external
|
||||
corpus/spec-RFC target — the discovery artifacts live in-repo (committed) by
|
||||
design. If a dedicated BDD/spec-RFC repo is ever wanted, register it in
|
||||
`app.json`.
|
||||
- **Concurrent session 0004** was claimed during this finalize and may still be
|
||||
in flight.
|
||||
- No deferred (low-confidence autonomous) decisions this session.
|
||||
|
||||
## Prompt the operator can paste into the next session
|
||||
|
||||
```
|
||||
/goal Build the experience simulator — execute the plan task-by-task:
|
||||
/executing-plans docs/superpowers/plans/2026-06-04-experience-simulator.md
|
||||
```
|
||||
|
||||
Read `memory/experience-simulator-ready-to-build.md` first. The plan recommends
|
||||
subagent-driven execution with review between tasks; the only shipped-code change
|
||||
is the additive `hef.selection.ranked_candidates()` (Task 1).
|
||||
@@ -1,20 +0,0 @@
|
||||
# Session 0003.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-04T07-57 (PST)
|
||||
> Type: discovery
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0003 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0003.0-TRANSCRIPT-2026-06-04T07-57--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
_(launch prompt not captured at claim time)_
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
@@ -0,0 +1,125 @@
|
||||
# Session 0004.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: coding
|
||||
> Start: 2026-06-05T03-32 (PST) · End: 2026-06-05T03-56 (PST)
|
||||
> Goal: Execute the experience-simulator (curator's X-ray) implementation plan.
|
||||
> Outcome: **Shipped.** All 7 plan tasks built TDD-first and merged to `main`
|
||||
> via PR #3. Suite: 133 passed, 2 skipped. Docker image builds + serves.
|
||||
|
||||
## Plan
|
||||
|
||||
Execute `docs/superpowers/plans/2026-06-04-experience-simulator.md` via
|
||||
`superpowers:executing-plans`: build the `simulator/` package — a FastAPI service
|
||||
wiring the five dials to the **real** `hef.selection` code and serving a
|
||||
vanilla-JS "X-ray" (picked piece + ranked candidate pool with distances +
|
||||
brain/mood 5×5 maps), fed by a deterministic synthetic fixture catalog since
|
||||
`catalog/library.jsonl` is empty. The only `hef/` change is additive:
|
||||
`hef.selection.ranked_candidates()` with `select()` refactored onto it. 7 tasks,
|
||||
TDD throughout; Docker + compose + `make` targets package it for localhost.
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- Clean `main`, synced with `origin/main` at `ddec45d`; 111 tests green, 2 skipped.
|
||||
- Memory pointer: simulator spec + BDDs + plan produced in discovery 0003
|
||||
(2026-06-04); next coding session runs `/executing-plans` on the simulator plan.
|
||||
- Discovery session **0003** still `--INPROGRESS` (committed its artifacts to main
|
||||
but never finalized) — dormant, not actively editing the tree.
|
||||
- `fastapi` not yet installed; `python` not on PATH (use `.venv/bin/python`).
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Session gate → coding init.** Classified the `/executing-plans` launch as a
|
||||
coding session; ran `wgl-session-coding-init`. Dry-run peek surfaced the
|
||||
in-flight 0003 placeholder; autonomous default → noted it and proceeded.
|
||||
Claimed session **0004**. Reconciled an initially-stale "ahead 2" read:
|
||||
local `main` == `origin/main` == `ddec45d` (the 0003 spec/BDD/plan commits are
|
||||
on origin). Verified clean baseline: **111 passed, 2 skipped**.
|
||||
2. **Plan review.** Read the plan + `hef/selection.py`, `hef/catalog.py`,
|
||||
`pyproject.toml`. Critical review: all assumed symbols exist
|
||||
(`Coordinate`/`Weights`/`distance`/`candidates_for_mode`/`select`,
|
||||
`CONTENT_MODES`/`SELECTOR_MODES`; `Record` carries every fixture field). No
|
||||
blockers. Confirmed Docker present, `docs/USER_GUIDE.md` + `features/` exist.
|
||||
3. **Workspace.** `using-git-worktrees` → native `EnterWorktree`
|
||||
(`worktree-experience-simulator`, branched from `origin/main`). Fresh `.venv`
|
||||
in the worktree; base package + pytest installed; baseline re-confirmed
|
||||
**111 passed, 2 skipped**.
|
||||
4. **Tasks 1–7, TDD (red → green → commit each):**
|
||||
- **T1** `hef.selection.ranked_candidates()` + `select()` refactored onto it
|
||||
(single source of truth; behavior-preserving) — 24 passed (new + existing
|
||||
selection suite unchanged).
|
||||
- **T2** declare `simulator` package + `[project.optional-dependencies] sim`
|
||||
(fastapi/uvicorn/httpx); installed `-e ".[sim]"`; imports resolve.
|
||||
- **T3** `simulator/fixtures.py` — deterministic 625-cell synthetic catalog
|
||||
(all modes, mixed review status, no media) — 6 passed.
|
||||
- **T4** `simulator/app.py` — FastAPI `create_app`, `POST /api/select`
|
||||
(pick + ranked pool with distance/rank + coverage), `GET /api/catalog/meta`,
|
||||
real-catalog-or-fixtures loader, guarded static mount — 6 passed.
|
||||
- **T5** `simulator/static/` X-ray UI (dials, model knobs, ranked pool,
|
||||
brain/mood maps) + static smoke test — 7 passed. Manual browser eyeball
|
||||
substituted with live `curl` verification (no interactive browser): `/` 200,
|
||||
`/api/catalog/meta` 625 records all-modes/mixed-status, `/api/select` ranked
|
||||
pool ascending distances (1.0 → 1.41…), `none` → void, static assets 200.
|
||||
- **T6** Dockerfile + compose + `Makefile` (literal-tab recipes). First build
|
||||
**failed**: `package directory 'tools' does not exist` — `tools` is a
|
||||
declared setuptools package but wasn't copied into the image. Root-caused
|
||||
from the build log; fixed by adding `COPY tools ./tools`. Rebuilt → container
|
||||
serves (`/` 200, 625-record fixture catalog). Torn down.
|
||||
- **T7** USER_GUIDE "Playing with the simulator" section; **full suite: 133
|
||||
passed, 2 skipped** (baseline 111 + 22 new).
|
||||
5. **Finish branch.** `finishing-a-development-branch` (autonomous): pushed branch
|
||||
as `experience-simulator`; created **PR #3** via the Gitea API
|
||||
(`wgl-gitea-admin` token mechanism, `git.benstull.org`); merged it (merge
|
||||
commit `618b691`); deleted the merged remote branch; FF'd local `main`;
|
||||
re-verified the merged result **133 passed, 2 skipped**; removed the worktree
|
||||
(all 7 commits confirmed ancestors of `main` first).
|
||||
6. **Finalize.** Added `*.egg-info/` to `.gitignore` (editable-install artifact)
|
||||
and pushed to `main` (`40dfbfd`); updated memory; published this transcript.
|
||||
|
||||
## Cut state (final)
|
||||
|
||||
- `main` @ `40dfbfd`, clean, synced with `origin/main`. No open PRs, no dangling
|
||||
branches, worktree removed.
|
||||
- **133 passed, 2 skipped.** Docker `make sim` builds and serves; `make sim-local`
|
||||
runs uvicorn. Simulator is operator-runnable on localhost.
|
||||
- The simulator is a **curator's discovery tool**, not a roadmap sub-project — the
|
||||
roadmap frontier is unchanged.
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
Autonomous-mode calls made without operator input:
|
||||
|
||||
1. **Dockerfile `COPY tools ./tools` (plan deviation).** The plan's Dockerfile
|
||||
copied only `hef`/`simulator`/`catalog`, but `pyproject.toml` declares
|
||||
`packages = ["hef", "tools", "simulator"]`, so the in-image editable install
|
||||
failed. Added the missing COPY. *Alternative:* trim `tools` from the package
|
||||
list — rejected (it's a real, used package). High confidence; the image now
|
||||
matches the declared packages.
|
||||
2. **Manual browser eyeball → programmatic curl (T5 Step 7).** No interactive
|
||||
browser available, so verified the running server with `curl` (HTML, meta,
|
||||
`/api/select`, void, static assets) instead of visually. Behavior fully
|
||||
exercised, but no human visual confirmation of the rendered UI.
|
||||
3. **Autonomous PR-merge instead of the interactive finish menu.** Per §6.5
|
||||
autonomous default + §5.4 (branch→PR→merge), pushed + opened + merged PR #3
|
||||
without surfacing the 4-option finish prompt. The preferred path was
|
||||
unambiguous and the merge is revertible.
|
||||
4. **`*.egg-info/` gitignore committed straight to `main`.** Trivial one-line
|
||||
hygiene; landed directly rather than via PR.
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
The simulator is shipped. The roadmap's next milestone is **sub-project 3
|
||||
(Player Runtime / Pi)** — no spec/plan yet, so it starts with discovery.
|
||||
|
||||
```
|
||||
/goal Develop sub-project 3 (Player Runtime / Pi): read the controls, call hef.selection.select(), play the chosen segment across the single panoramic projector, crossfade on change, dark on None — start with a discovery session to produce BDDs + spec, per docs/ROADMAP.md §3
|
||||
```
|
||||
|
||||
Read first: memory `experience-simulator-ready-to-build.md` (what shipped + the
|
||||
Dockerfile-copies-every-package gotcha), `pano-projector-nature-video.md` (single
|
||||
pano projector, sub-project 5 dropped), and `docs/ROADMAP.md §3`. Open decisions
|
||||
to settle: player stack (mpv/ffmpeg/custom), `approved`-only enforcement, and the
|
||||
serial protocol framing shared with sub-project 4.
|
||||
|
||||
> Housekeeping: discovery session **0003** is still `--INPROGRESS` in
|
||||
> `sessions/0003/` (artifacts long since merged) — finalize or close it out.
|
||||
@@ -1,20 +0,0 @@
|
||||
# Session 0004.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-05T03-32 (PST)
|
||||
> Type: coding
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0004 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0004.0-TRANSCRIPT-2026-06-05T03-32--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
_(launch prompt not captured at claim time)_
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
@@ -0,0 +1,126 @@
|
||||
# Session 0005.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: discovery
|
||||
> Start: 2026-06-05T08-30 (PST) · End: 2026-06-05T17-48 (PST)
|
||||
> Goal: Explore whether AI-generated/altered video is a viable content path for the
|
||||
> pano projector — economics + fit with the design — and capture the outcome.
|
||||
> Outcome: **Artistic thesis pivoted** and a full design-spec revision written,
|
||||
> reviewed, and **merged to `main` via PR #4** (merge commit `77746e4`).
|
||||
> Implementation deferred to a sub-project-3 coding session.
|
||||
|
||||
## Plan
|
||||
|
||||
Discovery session opened from the operator's question: "can we write/generate
|
||||
scene descriptions and generate the video with AI — would the economics work?"
|
||||
The session ran as a brainstorming exploration that converged on a new thesis and
|
||||
produced a design spec as its artifact.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
I'm not sure if hte economics would work out (can you do a best effort calculation?) but maybe we could write up/generate scene descriptions and generate the video with AI. What do you think?
|
||||
```
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` @ `3510236`. Sub-projects 1 & 2 done/merged; roadmap frontier = sub-project
|
||||
3 (Pi player). A stale, already-merged worktree (`feature/BuildSimulator11`,
|
||||
same commit as main) was the session's launch cwd.
|
||||
- Design of record: `2026-06-04-human-experience-filter-design.md` — single pano
|
||||
projector + **found** nature video; §1 constraint "real found media, **not
|
||||
algorithmically generated filler. Curation is the artwork.**"
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Gate / classify.** Operator's prompt was ambiguous between discovery and a
|
||||
quick chat; asked → operator chose **Discovery (tracked)**. Ran
|
||||
`wgl-session-discovery-init`.
|
||||
2. **Claim snag (kept in the record).** `resolve-app.py` reported **ambiguous** —
|
||||
the nested worktree's `app.json` (`.worktrees/feature-BuildSimulator11/app.json`)
|
||||
was discovered alongside the canonical main-repo `app.json`, both listing the
|
||||
repo. Worked around with `WGL_APP_SCAN_DEPTH=3` (excludes the depth-5 worktree
|
||||
copy). Claimed session **0005**, type discovery, at `850f67a`.
|
||||
3. **Economics, v1 (the original question).** Researched 2026 AI-video API rates
|
||||
(Sora 2 ~$0.10/s, Kling 3.0 ~$0.10/s, Veo 3.1 Lite/Runway Turbo ~$0.05/s, Veo 3
|
||||
+audio ~$0.75/s). At the spec's scale (120–800 pieces × ~10-min segments) →
|
||||
**~$15k–45k+** with retries; **not viable**, and it doesn't reduce the real
|
||||
constraint (curation labor). Also flagged the §1 thesis conflict ("not generated
|
||||
filler").
|
||||
4. **Thesis pivot.** Operator: willing to change the thesis — the piece is "about
|
||||
**how humans interact with machines and how it impacts their nervous systems**."
|
||||
Under that thesis AI alteration *is* the subject. Invoked `brainstorming`.
|
||||
Scope = **re-skin the meaning** (keep apparatus); content = **AI-altered
|
||||
primary**.
|
||||
5. **Economics, v2 (the flip).** New thesis unlocks two cost levers: small
|
||||
**generate-to-order** library + short **seamless loops** → **~$300–3k**. Feasible.
|
||||
6. **The strong concept crystallizes.** Operator: start from **neutral** real
|
||||
nature footage and let the machine **alter it toward the knob state** — "a great
|
||||
expression of how machines alter our experience, good and bad." Established the
|
||||
per-axis mapping (Left = analytical overlay; Right = generative v2v dissolve;
|
||||
Dark/Light = color grade), the **substrate-vs-overlay composition rule**, and the
|
||||
"whole-brain" corner (feeling, *labeled*).
|
||||
7. **Control panel + i18n.** Operator added a 7-way content dial (Off/White
|
||||
Noise/Music/Audio Track/Video/Music+Video/Audio Track+Video), volume + brightness
|
||||
knobs, and a tactile **wooden** panel (engraved symbols, braille, color LEDs,
|
||||
read-aloud button). Designed touch-distinct/engravable symbols. Key finding on
|
||||
the **translation cost**: because the Left **labels are a runtime overlay**, the
|
||||
v2v renders once (language-agnostic) and supporting *every* language is ~free on
|
||||
the video — only cheap text translation + TTS. (Baking labels in would multiply
|
||||
video cost × #languages — the avoided landmine.)
|
||||
8. **Mood color grade.** Operator: Light = yellow / white negative space; center =
|
||||
raw ungraded video; Dark = blue / black negative space. (First draft mis-set
|
||||
Dark=red and center=grey; operator corrected → Dark = melancholy **blue** not
|
||||
alarm red; center = **raw video**, no grade.)
|
||||
9. **Wrote the spec.** Branched `feature/machine-altered-perception-spec` off main,
|
||||
wrote `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md`
|
||||
(15 sections), committed, self-reviewed; operator reviewed and gave 4
|
||||
corrections (center=raw, add Music+Video → 7-way, audio-alteration deferred,
|
||||
red gone) → applied, commit amended to `c5952c6`.
|
||||
10. **Wrap.** Operator chose to wrap the discovery session here (spec is the
|
||||
artifact; build later). Pushed branch; first PR-creation attempt was **blocked
|
||||
by the auto-mode classifier** (treated as routing around PR authorization) —
|
||||
surfaced it rather than working around. Operator then explicitly authorized
|
||||
opening + merging the PR → **PR #4 opened and merged to `main`** (merge commit
|
||||
`77746e4`, branch deleted), local `main` fast-forwarded.
|
||||
|
||||
## Cut state (end of session)
|
||||
|
||||
| Item | State |
|
||||
|---|---|
|
||||
| Design spec | **Merged to `main`** — `docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md` |
|
||||
| Branch commit | `c5952c6` (spec, amended with review fixes) |
|
||||
| Merge commit | `77746e4` (PR #4), feature branch deleted |
|
||||
| `main` | fast-forwarded to `77746e4` (local + origin) |
|
||||
| Roadmap | unchanged on disk; spec §13 describes how sub-projects 2/3/4 + a new offline v2v pipeline are reshaped (a roadmap-edit pass is future work) |
|
||||
| Memory | `ai-generated-content-thesis-pivot.md` (full design rationale + status) + `MEMORY.md` index updated |
|
||||
|
||||
## What lands on the operator's plate
|
||||
|
||||
- Nothing blocking. The spec is canonical and merged.
|
||||
- Optional future passes the spec itself lists (§14): pano resolution strategy,
|
||||
negative-space luma key, transform composition order, neutral-base count,
|
||||
language tier, runtime-grading perf on the Pi.
|
||||
- A roadmap-edit pass to reflect the reshaped sub-projects 2/3/4 (the design spec
|
||||
carries it for now).
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
No substantive low-confidence design calls — every design decision (thesis,
|
||||
per-axis mapping, composition rule, color grade, the 7-way dial, the panel) was
|
||||
operator-confirmed in-session. Two mechanical judgment calls worth flagging:
|
||||
|
||||
- **`WGL_APP_SCAN_DEPTH=3` workaround** for the nested-worktree `app.json`
|
||||
ambiguity in `resolve-app.py`. This is a genuine plugin friction (a worktree
|
||||
nested under the main repo makes the resolver ambiguous) — a candidate for
|
||||
`wgl-dev-plugin-feedback` (resolver should skip `.worktrees/`).
|
||||
- **Repurposed the stale `feature/BuildSimulator11` worktree** to a fresh
|
||||
`feature/machine-altered-perception-spec` branch off its HEAD (= main) rather
|
||||
than removing it (harness had cwd anchored there; removal risked breaking the
|
||||
shell).
|
||||
|
||||
## Prompt the operator can paste into the next session
|
||||
|
||||
```
|
||||
/goal Implement sub-project 3 (the player / alteration engine) per docs/ROADMAP.md and the new docs/superpowers/specs/2026-06-05-machine-altered-perception-design.md. Read memory ai-generated-content-thesis-pivot.md first. Build the alteration engine: runtime luma-keyed mood grade (Light=yellow/white → raw → Dark=blue/black), runtime analytical-overlay compositor (multilingual, from string tables), generated white noise, and selection/blending of pre-baked v2v variants over a small neutral base library; resolve the spec §14 open questions (pano resolution, composition order) as you go.
|
||||
```
|
||||
@@ -0,0 +1,134 @@
|
||||
# Session 0006.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: coding
|
||||
> Start: 2026-06-05T17-52 (PST) · End: 2026-06-05T18-15 (PST)
|
||||
> Goal: `/goal next` — drive the roadmap to its next frontier item (sub-project 3,
|
||||
> the player / machine-alteration engine), and build it.
|
||||
> Outcome: **Sub-project 3 slice 1 — the pure-logic alteration-engine + player
|
||||
> core — built and MERGED to `main` via PR #5** (merge commit `aeeed66`).
|
||||
> New `player/` package; 56 new tests (189 passed / 2 skipped total).
|
||||
> Sub-project 3 advanced from "next" to "in progress."
|
||||
|
||||
## Plan
|
||||
|
||||
Begin sub-project 3 (Player Runtime + alteration engine) per `docs/ROADMAP.md` §3
|
||||
and the approved machine-altered-perception design (§4 alteration engine, §5 mood
|
||||
grade, §6 7-way content dial, §7 intensity). Sub-project 3 is large (runtime
|
||||
grading + overlay compositing + v2v variant selection + serial input + mpv/GPU +
|
||||
white-noise DSP); this session ships **slice 1 — the pure-logic core** as a new
|
||||
`player/` package, fully unit-tested with all I/O behind injected interfaces,
|
||||
mirroring how sub-project 1 was the pure-logic dependency root. Process:
|
||||
writing-plans → TDD → branch → PR → merge.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
/goal next — drive the roadmap to its next frontier item.
|
||||
|
||||
Per project memory (session 0005, machine-altered-perception design SPEC merged
|
||||
to main via PR #4), the roadmap frontier is sub-project 3: the player /
|
||||
machine-alteration engine. This coding session builds that.
|
||||
```
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` @ `77746e4` locally (behind `origin/main` by 2 — session 0005's finalized
|
||||
transcript + this session's claim placeholder); fast-forwarded to `origin/main`
|
||||
during init.
|
||||
- Sub-projects 1 & 2 done/merged; design of record = the machine-altered-perception
|
||||
revision (PR #4). No `player/` package and no sub-project-3 plan existed yet.
|
||||
- Launch cwd was the stale, already-merged worktree
|
||||
`.worktrees/feature-BuildSimulator11/` (branch `feature/machine-altered-perception-spec`,
|
||||
remote gone) — same situation as session 0005.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Gate / classify.** `/goal next` → unambiguously a coding session (roadmap
|
||||
frontier = sub-project 3 build). Ran `wgl-session-coding-init`.
|
||||
2. **Claim snag (same as 0005, kept in the record).** `resolve-app.py` reported
|
||||
**ambiguous** — the nested worktree's `app.json` discovered alongside the main
|
||||
clone's. Read the resolver to confirm root cause (`find_app_jsons` prunes
|
||||
`.git/node_modules/…` but not `.worktrees/`). Worked around with
|
||||
`WGL_APP_SCAN_DEPTH=4` (prunes the depth-5 worktree copy, keeps the depth-3 main
|
||||
one). Claimed session **0006**, type coding, at `e694489`. Filed a plugin-feedback
|
||||
note for the recurring gap (`feedback/2026-06-05T18-13-resolver-nested-worktree-ambiguous.md`).
|
||||
3. **Baseline + branch.** Fast-forwarded the main clone; confirmed PR #4 merged;
|
||||
created `feature/player-alteration-core` off `origin/main` in the worktree.
|
||||
4. **Read the design.** Read ROADMAP §3 + the machine-altered-perception spec
|
||||
(§4 alteration engine, §5 mood grade, §6 7-way dial, §7 intensity), and the
|
||||
existing `hef/` conventions (frozen dataclasses, pure modules, heavy unit tests).
|
||||
5. **Scoped the session → slice 1.** Sub-project 3 is too large for one session;
|
||||
chose the pure-logic core (controls model, content-dial resolver, alteration
|
||||
render-plan, player state machine) with all I/O behind injected interfaces, and
|
||||
deferred the renderer/serial/white-noise/v2v/catalog slices (named in the plan).
|
||||
6. **Surfaced a spec ambiguity (Deferred decision).** §3 "(2,2,2,2) neutral" vs
|
||||
§4.2 (Left/Right stack as independent layers) vs §5 (mood center = identity)
|
||||
cannot all be literally true for four independent knobs. Resolved with a
|
||||
swappable calibration: brain `value/4`, mood `(light−dark)/4`; flagged for
|
||||
operator confirm.
|
||||
7. **writing-plans.** Wrote `docs/superpowers/plans/2026-06-05-player-alteration-core.md`
|
||||
(6 TDD tasks, full code + tests, self-review against the spec).
|
||||
8. **TDD implementation.** Red→green→commit per module: `player/controls.py`
|
||||
(Controls, validation, parse), `player/content.py` (§6 7-way table),
|
||||
`player/alteration.py` (RenderPlan: ColorGrade/AnalyticalOverlay/Restyle +
|
||||
calibration helpers), `player/state.py` (Player state machine + Transition
|
||||
kinds). Registered the package in `pyproject.toml`; updated ROADMAP §3.
|
||||
9. **Verify.** Full suite green: **189 passed, 2 skipped** (the opt-in ffprobe
|
||||
tests), 56 of them new player tests. No regressions.
|
||||
10. **Ship.** Pushed the branch; created **PR #5** and merged it via the Gitea API
|
||||
helper (merge commit `aeeed66`); synced the main clone; deleted the remote
|
||||
feature branch.
|
||||
11. **Wrap.** Filed plugin feedback, updated memory, finalized.
|
||||
|
||||
## Cut state
|
||||
|
||||
- `main` @ `aeeed66` (PR #5 merged). `player/` package present: `controls.py`,
|
||||
`content.py`, `alteration.py`, `state.py` + 4 test files. Full suite 189 passed /
|
||||
2 skipped.
|
||||
- Sub-project 3 = **⏳ in progress**. Slice 1 done; remaining slices listed in
|
||||
ROADMAP §3 and `[[sub-project-3-player-progress]]`.
|
||||
- Working tree clean. Remote feature branch deleted; the local worktree remains on
|
||||
the now-merged `feature/player-alteration-core` branch (harmless).
|
||||
|
||||
## Deferred decisions (operator plate)
|
||||
|
||||
Two low-confidence calls made autonomously this session — both flagged for operator
|
||||
confirmation:
|
||||
|
||||
1. **Knob→strength calibration (§3 vs §4.2/§5 conflict).** §3 says "(2,2,2,2) is
|
||||
neutral," but §4.2 (Left/Right stack as independent layers: overlay vs substrate)
|
||||
and §5 (mood center = identity) conflict with that for four independent physical
|
||||
knobs. Implemented: brain knobs `strength = value/4` (0 = off, 4 = max — full
|
||||
range, honors §4.2 stacking), mood `tone = (light − dark)/4` (equal dark/light →
|
||||
identity, honors §5). Under this the un-altered base is `(0,0,·,·)` with
|
||||
`dark==light`, **not** `(2,2,2,2)` — read as a vestige of the old coordinate-grid
|
||||
*center*. The calibration is three one-line helpers in `player/alteration.py`;
|
||||
flipping to a "centered at 2 = no push" convention is a localized change that
|
||||
leaves the RenderPlan shape and all downstream logic untouched. **Recommend
|
||||
operator confirm the intended panel UX.**
|
||||
2. **Volume/Brightness granularity.** Modeled as `0..4` ints for uniformity with the
|
||||
experience knobs and the Arduino's `0..4` quantization (roadmap §4). The spec
|
||||
calls them "levels" without fixing granularity; revisit if the panel warrants
|
||||
finer steps.
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
The roadmap frontier is now the remaining sub-project-3 slices. Read
|
||||
`[[sub-project-3-player-progress]]` and the slice-1 plan first. Two strong
|
||||
candidates for the next slice — the **runtime renderer** (makes the core visible)
|
||||
or the **serial framing contract + input adapter** (unblocks sub-project 4 in
|
||||
parallel):
|
||||
|
||||
```
|
||||
/goal Sub-project 3 slice 2 — settle the 3⇄4 serial framing contract and build the
|
||||
USB-serial / keyboard input adapter that feeds the player a `Controls` stream
|
||||
(unblocks sub-project 4), per docs/ROADMAP.md §3 and docs/superpowers/plans/2026-06-05-player-alteration-core.md.
|
||||
Alternatively start the runtime renderer (mpv/ffmpeg + grade/overlay). First
|
||||
confirm the open knob→strength calibration decision (§3 vs §4.2/§5).
|
||||
```
|
||||
|
||||
Gotcha to carry forward: running from the nested `.worktrees/…` checkout breaks
|
||||
`resolve-app.py` (ambiguous app.json) — run from the main clone, or use
|
||||
`WGL_APP_SCAN_DEPTH=4` on claim and explicit `--sessions-*` flags on publish, until
|
||||
the plugin fix lands.
|
||||
@@ -10,5 +10,11 @@
|
||||
},
|
||||
"0004": {
|
||||
"title": ""
|
||||
},
|
||||
"0005": {
|
||||
"title": ""
|
||||
},
|
||||
"0006": {
|
||||
"title": ""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
FROM python:3.11-slim
|
||||
WORKDIR /app
|
||||
COPY pyproject.toml ./
|
||||
COPY hef ./hef
|
||||
COPY tools ./tools
|
||||
COPY simulator ./simulator
|
||||
COPY catalog ./catalog
|
||||
RUN pip install --no-cache-dir -e ".[sim]"
|
||||
EXPOSE 8000
|
||||
CMD ["uvicorn", "simulator.app:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
@@ -0,0 +1 @@
|
||||
"""Web-based curator's X-ray simulator for the experience filter."""
|
||||
@@ -0,0 +1,96 @@
|
||||
"""FastAPI service: dials -> real hef.selection -> X-ray (pick + ranked pool)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
from typing import Literal, Optional
|
||||
|
||||
from fastapi import FastAPI
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from hef.catalog import load_catalog, record_to_dict
|
||||
from hef.selection import (
|
||||
Coordinate,
|
||||
Weights,
|
||||
candidates_for_mode,
|
||||
ranked_candidates,
|
||||
select,
|
||||
)
|
||||
from simulator.fixtures import generate_fixture_catalog
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "static"
|
||||
|
||||
|
||||
class SelectRequest(BaseModel):
|
||||
left: int = Field(ge=0, le=4)
|
||||
right: int = Field(ge=0, le=4)
|
||||
dark: int = Field(ge=0, le=4)
|
||||
light: int = Field(ge=0, le=4)
|
||||
mode: Literal["none", "audio", "video", "av"]
|
||||
pool_size: int = Field(default=4, ge=1, le=25)
|
||||
brain_weight: float = Field(default=1.0, ge=0.0)
|
||||
mood_weight: float = Field(default=1.0, ge=0.0)
|
||||
approved_only: bool = False
|
||||
|
||||
|
||||
def load_catalog_or_fixtures() -> list:
|
||||
"""Use the real catalog if a non-empty one is configured/exists, else fixtures."""
|
||||
configured = os.environ.get("HEF_SIM_CATALOG")
|
||||
path = Path(configured) if configured else Path("catalog/library.jsonl")
|
||||
if path.exists() and path.stat().st_size > 0:
|
||||
return load_catalog(path)
|
||||
return generate_fixture_catalog()
|
||||
|
||||
|
||||
def create_app(records: Optional[list] = None) -> FastAPI:
|
||||
app = FastAPI(title="HEF Experience Simulator")
|
||||
app.state.catalog = records if records is not None else load_catalog_or_fixtures()
|
||||
|
||||
@app.post("/api/select")
|
||||
def api_select(req: SelectRequest):
|
||||
catalog = app.state.catalog
|
||||
coord = Coordinate(req.left, req.right, req.dark, req.light)
|
||||
weights = Weights(brain=req.brain_weight, mood=req.mood_weight)
|
||||
if req.mode == "none":
|
||||
return {"pick": None, "pool": [], "coverage": {"candidates_in_mode": 0}}
|
||||
pool = catalog
|
||||
if req.approved_only:
|
||||
pool = [r for r in pool if r.review_status == "approved"]
|
||||
eligible = candidates_for_mode(pool, req.mode, req.pool_size)
|
||||
ranked = ranked_candidates(
|
||||
catalog, coord, req.mode,
|
||||
pool_size=req.pool_size, weights=weights, approved_only=req.approved_only,
|
||||
)
|
||||
pick = select(
|
||||
catalog, coord, req.mode,
|
||||
pool_size=req.pool_size, weights=weights, approved_only=req.approved_only,
|
||||
rng=None,
|
||||
)
|
||||
return {
|
||||
"pick": record_to_dict(pick) if pick else None,
|
||||
"pool": [
|
||||
{"record": record_to_dict(r), "distance": d, "rank": i + 1}
|
||||
for i, (r, d) in enumerate(ranked)
|
||||
],
|
||||
"coverage": {"candidates_in_mode": len(eligible)},
|
||||
}
|
||||
|
||||
@app.get("/api/catalog/meta")
|
||||
def api_meta():
|
||||
catalog = app.state.catalog
|
||||
return {
|
||||
"total": len(catalog),
|
||||
"by_mode": dict(Counter(r.mode for r in catalog)),
|
||||
"by_status": dict(Counter(r.review_status for r in catalog)),
|
||||
}
|
||||
|
||||
if STATIC_DIR.exists():
|
||||
app.mount("/", StaticFiles(directory=STATIC_DIR, html=True), name="static")
|
||||
|
||||
return app
|
||||
|
||||
|
||||
app = create_app()
|
||||
@@ -0,0 +1,7 @@
|
||||
services:
|
||||
simulator:
|
||||
build:
|
||||
context: ..
|
||||
dockerfile: simulator/Dockerfile
|
||||
ports:
|
||||
- "8000:8000"
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Deterministic synthetic catalog so the selection model can be felt everywhere.
|
||||
|
||||
The real catalog (catalog/library.jsonl) is empty; this generates one record per
|
||||
cell of the 5x5 brain x 5x5 mood coordinate space (625 records), with a seeded
|
||||
mix of content modes and review statuses and no real media attached.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
|
||||
from hef.catalog import Record
|
||||
|
||||
MODES = ("audio", "video", "av")
|
||||
ARCHIVES = ("internet_archive", "musopen", "librivox", "nasa", "freesound")
|
||||
|
||||
_LEFT_WORDS = ("Treatise", "Lecture", "Field Notes", "Reading", "Documentary")
|
||||
_RIGHT_WORDS = ("Reverie", "Nocturne", "Bloom", "Drift", "Aurora")
|
||||
|
||||
|
||||
def _title(left: int, right: int, dark: int, light: int, mode: str) -> str:
|
||||
a = _LEFT_WORDS[left] if left >= right else _RIGHT_WORDS[right]
|
||||
return f"{a} ({mode}) L{left}R{right}D{dark}Li{light}"
|
||||
|
||||
|
||||
def generate_fixture_catalog(seed: int = 1729) -> list[Record]:
|
||||
"""One valid Record per coordinate cell (625 total), deterministic for a seed."""
|
||||
rng = random.Random(seed)
|
||||
records: list[Record] = []
|
||||
n = 0
|
||||
for left in range(5):
|
||||
for right in range(5):
|
||||
for dark in range(5):
|
||||
for light in range(5):
|
||||
mode = rng.choice(MODES)
|
||||
status = rng.choice(("proposed", "approved"))
|
||||
is_video = mode in ("video", "av")
|
||||
records.append(
|
||||
Record(
|
||||
id=f"fx-{n:04d}",
|
||||
title=_title(left, right, dark, light, mode),
|
||||
source_url=f"https://example.test/fx/{n:04d}",
|
||||
source_archive=rng.choice(ARCHIVES),
|
||||
license="public_domain",
|
||||
mode=mode,
|
||||
left=left,
|
||||
right=right,
|
||||
dark=dark,
|
||||
light=light,
|
||||
duration_s=rng.choice((300, 480, 600, 720, 900)),
|
||||
file_path="",
|
||||
review_status=status,
|
||||
resolution="1920x1080" if is_video else "",
|
||||
rationale=f"fixture at ({left},{right},{dark},{light})",
|
||||
reviewed_at="2026-06-04T00:00:00Z" if status == "approved" else None,
|
||||
)
|
||||
)
|
||||
n += 1
|
||||
return records
|
||||
@@ -0,0 +1,111 @@
|
||||
const DIALS = ["left", "right", "dark", "light"];
|
||||
const MODEL = ["brain_weight", "mood_weight", "pool_size"];
|
||||
|
||||
function buildGrid(el) {
|
||||
el.innerHTML = "";
|
||||
// rows = first axis 0..4 top->bottom, cols = second axis 0..4 left->right
|
||||
for (let a = 0; a < 5; a++) {
|
||||
for (let b = 0; b < 5; b++) {
|
||||
const cell = document.createElement("div");
|
||||
cell.className = "cell";
|
||||
cell.dataset.a = a;
|
||||
cell.dataset.b = b;
|
||||
el.appendChild(cell);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function paintGrid(el, axisA, axisB, point, pool) {
|
||||
// clear
|
||||
el.querySelectorAll(".cell").forEach((c) => {
|
||||
c.className = "cell";
|
||||
c.innerHTML = "";
|
||||
});
|
||||
const counts = {};
|
||||
pool.forEach((c) => {
|
||||
const r = c.record;
|
||||
const key = `${r[axisA]},${r[axisB]}`;
|
||||
counts[key] = (counts[key] || 0) + 1;
|
||||
});
|
||||
el.querySelectorAll(".cell").forEach((c) => {
|
||||
const a = +c.dataset.a, b = +c.dataset.b;
|
||||
const key = `${a},${b}`;
|
||||
if (counts[key]) {
|
||||
c.classList.add("cand");
|
||||
const n = document.createElement("span");
|
||||
n.className = "n";
|
||||
n.textContent = counts[key];
|
||||
c.appendChild(n);
|
||||
}
|
||||
if (a === point[axisA] && b === point[axisB]) c.classList.add("point");
|
||||
});
|
||||
}
|
||||
|
||||
function readState() {
|
||||
const s = { mode: document.getElementById("mode").value, approved_only: document.getElementById("approved_only").checked };
|
||||
DIALS.forEach((d) => (s[d] = +document.getElementById(d).value));
|
||||
s.brain_weight = +document.getElementById("brain_weight").value;
|
||||
s.mood_weight = +document.getElementById("mood_weight").value;
|
||||
s.pool_size = +document.getElementById("pool_size").value;
|
||||
return s;
|
||||
}
|
||||
|
||||
function syncOutputs() {
|
||||
[...DIALS, ...MODEL].forEach((id) => {
|
||||
const out = document.getElementById(`${id}-out`);
|
||||
if (out) out.textContent = document.getElementById(id).value;
|
||||
});
|
||||
}
|
||||
|
||||
async function refresh() {
|
||||
syncOutputs();
|
||||
const state = readState();
|
||||
const resp = await fetch("/api/select", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(state),
|
||||
});
|
||||
const data = await resp.json();
|
||||
|
||||
const pickEl = document.getElementById("pick");
|
||||
if (!data.pick) {
|
||||
pickEl.innerHTML = '<div class="void">∅ Void / rest — walls dark, audio silent.</div>';
|
||||
} else {
|
||||
const p = data.pick;
|
||||
pickEl.innerHTML =
|
||||
`<div class="title">${p.title}</div>` +
|
||||
`<div>mode ${p.mode} · coord (${p.left},${p.right},${p.dark},${p.light})</div>` +
|
||||
`<div>${p.rationale || ""}</div>`;
|
||||
}
|
||||
|
||||
const poolEl = document.getElementById("pool");
|
||||
poolEl.innerHTML = "";
|
||||
data.pool.forEach((c, i) => {
|
||||
const li = document.createElement("li");
|
||||
if (i === 0) li.className = "winner";
|
||||
li.innerHTML = `${c.record.title} <span class="dist">d=${c.distance.toFixed(2)}</span>`;
|
||||
poolEl.appendChild(li);
|
||||
});
|
||||
|
||||
const point = { left: state.left, right: state.right, dark: state.dark, light: state.light };
|
||||
paintGrid(document.getElementById("brain-grid"), "left", "right", point, data.pool);
|
||||
paintGrid(document.getElementById("mood-grid"), "dark", "light", point, data.pool);
|
||||
}
|
||||
|
||||
async function loadMeta() {
|
||||
const data = await (await fetch("/api/catalog/meta")).json();
|
||||
const byMode = Object.entries(data.by_mode).map(([k, v]) => `${k}:${v}`).join(" ");
|
||||
document.getElementById("meta").textContent = `${data.total} records · ${byMode}`;
|
||||
}
|
||||
|
||||
function init() {
|
||||
buildGrid(document.getElementById("brain-grid"));
|
||||
buildGrid(document.getElementById("mood-grid"));
|
||||
document.querySelectorAll("input, select").forEach((el) =>
|
||||
el.addEventListener("input", refresh)
|
||||
);
|
||||
loadMeta();
|
||||
refresh();
|
||||
}
|
||||
|
||||
document.addEventListener("DOMContentLoaded", init);
|
||||
@@ -0,0 +1,64 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>HEF — Curator's X-ray</title>
|
||||
<link rel="stylesheet" href="/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<h1>Experience Filter — Curator's X-ray</h1>
|
||||
<div id="meta" class="meta"></div>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<section class="controls">
|
||||
<h2>Dials</h2>
|
||||
<label>Mode
|
||||
<select id="mode">
|
||||
<option value="none">None</option>
|
||||
<option value="audio">Audio</option>
|
||||
<option value="video">Video</option>
|
||||
<option value="av" selected>A+V</option>
|
||||
</select>
|
||||
</label>
|
||||
<label>Left (analytical) <output id="left-out">0</output>
|
||||
<input type="range" id="left" min="0" max="4" step="1" value="0"></label>
|
||||
<label>Right (artistic) <output id="right-out">0</output>
|
||||
<input type="range" id="right" min="0" max="4" step="1" value="0"></label>
|
||||
<label>Dark (somber) <output id="dark-out">0</output>
|
||||
<input type="range" id="dark" min="0" max="4" step="1" value="0"></label>
|
||||
<label>Light (uplifting) <output id="light-out">0</output>
|
||||
<input type="range" id="light" min="0" max="4" step="1" value="0"></label>
|
||||
|
||||
<h2>Model knobs</h2>
|
||||
<label>Brain weight <output id="brain_weight-out">1</output>
|
||||
<input type="range" id="brain_weight" min="0" max="4" step="0.5" value="1"></label>
|
||||
<label>Mood weight <output id="mood_weight-out">1</output>
|
||||
<input type="range" id="mood_weight" min="0" max="4" step="0.5" value="1"></label>
|
||||
<label>Pool size <output id="pool_size-out">4</output>
|
||||
<input type="range" id="pool_size" min="1" max="10" step="1" value="4"></label>
|
||||
<label class="check"><input type="checkbox" id="approved_only"> Approved only</label>
|
||||
</section>
|
||||
|
||||
<section class="xray">
|
||||
<div class="pick">
|
||||
<h2>Picked</h2>
|
||||
<div id="pick"></div>
|
||||
</div>
|
||||
<div class="pool">
|
||||
<h2>Pool (nearest first)</h2>
|
||||
<ol id="pool"></ol>
|
||||
</div>
|
||||
<div class="maps">
|
||||
<h2>Coordinate maps</h2>
|
||||
<div class="map"><div class="label">Brain — Left × Right</div><div id="brain-grid" class="grid5"></div></div>
|
||||
<div class="map"><div class="label">Mood — Dark × Light</div><div id="mood-grid" class="grid5"></div></div>
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<script src="/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,26 @@
|
||||
:root { color-scheme: dark; }
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; font-family: -apple-system, system-ui, sans-serif; background: #0e0e16; color: #e6e6ee; }
|
||||
header { padding: 12px 20px; border-bottom: 1px solid #2a2a3a; display: flex; justify-content: space-between; align-items: baseline; }
|
||||
header h1 { font-size: 18px; margin: 0; }
|
||||
.meta { font-size: 12px; color: #9a9ab0; }
|
||||
main { display: grid; grid-template-columns: 280px 1fr; gap: 20px; padding: 20px; }
|
||||
.controls label { display: block; margin: 8px 0; font-size: 13px; }
|
||||
.controls input[type=range] { width: 100%; }
|
||||
.controls .check { display: flex; gap: 6px; align-items: center; }
|
||||
h2 { font-size: 13px; text-transform: uppercase; letter-spacing: .5px; color: #9a9ab0; }
|
||||
.xray { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; align-items: start; }
|
||||
.pick #pick { background: #1a1a2e; border: 1px solid #33334a; border-radius: 8px; padding: 14px; min-height: 80px; }
|
||||
.pick .title { font-size: 16px; font-weight: 600; }
|
||||
.pick .void { color: #7777aa; font-style: italic; }
|
||||
.pool ol { margin: 0; padding-left: 18px; font-size: 13px; }
|
||||
.pool li { margin: 4px 0; }
|
||||
.pool li.winner { color: #7fffd4; font-weight: 600; }
|
||||
.pool .dist { color: #9a9ab0; }
|
||||
.maps { grid-column: 1 / -1; display: flex; gap: 40px; }
|
||||
.grid5 { display: grid; grid-template-columns: repeat(5, 28px); grid-template-rows: repeat(5, 28px); gap: 3px; }
|
||||
.grid5 .cell { background: #1c1c2c; border: 1px solid #2a2a3a; border-radius: 3px; position: relative; }
|
||||
.grid5 .cell.cand { background: #3a3a66; }
|
||||
.grid5 .cell.point { outline: 2px solid #7fffd4; }
|
||||
.grid5 .cell .n { position: absolute; right: 2px; bottom: 1px; font-size: 9px; color: #aab; }
|
||||
.label { font-size: 11px; color: #9a9ab0; margin-bottom: 4px; }
|
||||
@@ -0,0 +1,40 @@
|
||||
from hef.catalog import validate_catalog
|
||||
from hef.selection import CONTENT_MODES
|
||||
from simulator.fixtures import generate_fixture_catalog
|
||||
|
||||
|
||||
def test_fixture_catalog_is_valid():
|
||||
records = generate_fixture_catalog()
|
||||
validate_catalog(records) # raises on any invalid record or duplicate id
|
||||
|
||||
|
||||
def test_fixture_catalog_spans_the_coordinate_space():
|
||||
records = generate_fixture_catalog()
|
||||
coords = {(r.left, r.right, r.dark, r.light) for r in records}
|
||||
# all 625 cells of the 5x5 brain x 5x5 mood space are present
|
||||
assert len(coords) == 625
|
||||
|
||||
|
||||
def test_fixture_catalog_has_every_content_mode():
|
||||
records = generate_fixture_catalog()
|
||||
present = {r.mode for r in records}
|
||||
assert CONTENT_MODES <= present
|
||||
|
||||
|
||||
def test_fixture_catalog_mixes_review_statuses():
|
||||
records = generate_fixture_catalog()
|
||||
statuses = {r.review_status for r in records}
|
||||
assert statuses == {"proposed", "approved"}
|
||||
|
||||
|
||||
def test_fixture_catalog_references_no_real_media():
|
||||
records = generate_fixture_catalog()
|
||||
assert all(r.file_path == "" for r in records)
|
||||
|
||||
|
||||
def test_fixture_catalog_is_deterministic():
|
||||
a = generate_fixture_catalog(seed=42)
|
||||
b = generate_fixture_catalog(seed=42)
|
||||
assert [r.id for r in a] == [r.id for r in b]
|
||||
assert [r.mode for r in a] == [r.mode for r in b]
|
||||
assert [r.review_status for r in a] == [r.review_status for r in b]
|
||||
@@ -0,0 +1,80 @@
|
||||
import pytest
|
||||
|
||||
from hef.selection import Coordinate
|
||||
from player.alteration import (
|
||||
AnalyticalOverlay,
|
||||
ColorGrade,
|
||||
RenderPlan,
|
||||
Restyle,
|
||||
plan_alteration,
|
||||
)
|
||||
|
||||
|
||||
def _coord(left=0, right=0, dark=0, light=0):
|
||||
return Coordinate(left=left, right=right, dark=dark, light=light)
|
||||
|
||||
|
||||
def test_all_zero_knobs_is_the_unaltered_base():
|
||||
plan = plan_alteration(_coord())
|
||||
assert plan.is_identity
|
||||
assert plan.overlay.intensity == 0.0
|
||||
assert plan.restyle.blend == 0.0
|
||||
assert plan.grade.tone == 0.0
|
||||
assert plan.grade.is_identity
|
||||
|
||||
|
||||
def test_left_drives_the_analytical_overlay_only():
|
||||
plan = plan_alteration(_coord(left=4))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 0.0 # Left does not touch the substrate
|
||||
assert plan.grade.tone == 0.0
|
||||
|
||||
|
||||
def test_right_drives_the_restyle_substrate_only():
|
||||
plan = plan_alteration(_coord(right=2))
|
||||
assert plan.restyle.blend == 0.5
|
||||
assert plan.overlay.intensity == 0.0 # Right does not add overlay
|
||||
|
||||
|
||||
def test_left_and_right_stack_not_cancel():
|
||||
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
|
||||
plan = plan_alteration(_coord(left=4, right=4))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 1.0
|
||||
|
||||
|
||||
def test_light_pole_grades_warm_positive_tone():
|
||||
plan = plan_alteration(_coord(light=4))
|
||||
assert plan.grade.tone == 1.0
|
||||
assert not plan.grade.is_identity
|
||||
|
||||
|
||||
def test_dark_pole_grades_cool_negative_tone():
|
||||
plan = plan_alteration(_coord(dark=4))
|
||||
assert plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_equal_dark_and_light_is_identity_grade():
|
||||
# design §5: the mood center is the raw, ungraded footage
|
||||
assert plan_alteration(_coord(dark=3, light=3)).grade.is_identity
|
||||
assert plan_alteration(_coord(dark=2, light=2)).grade.is_identity
|
||||
|
||||
|
||||
def test_dark_minus_light_sets_intermediate_tone():
|
||||
assert plan_alteration(_coord(dark=4, light=2)).grade.tone == pytest.approx(-0.5)
|
||||
assert plan_alteration(_coord(dark=1, light=3)).grade.tone == pytest.approx(0.5)
|
||||
|
||||
|
||||
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
|
||||
# design §4.2 "Dark + analytical": cold measurement over a melancholy scene
|
||||
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
|
||||
assert plan.overlay.intensity == 1.0
|
||||
assert plan.restyle.blend == 0.5
|
||||
assert plan.grade.tone == -1.0
|
||||
assert not plan.is_identity
|
||||
|
||||
|
||||
def test_render_plan_is_frozen():
|
||||
plan = plan_alteration(_coord())
|
||||
with pytest.raises(Exception):
|
||||
plan.grade.tone = 0.5 # type: ignore[misc]
|
||||
@@ -0,0 +1,34 @@
|
||||
import pytest
|
||||
|
||||
from player.content import AUDIO_SOURCES, ContentResolution, resolve_content
|
||||
|
||||
|
||||
def test_audio_sources_are_the_four_distinct_sources():
|
||||
assert AUDIO_SOURCES == frozenset({"none", "white_noise", "music", "audio_track"})
|
||||
|
||||
|
||||
# The §6 table, row by row: position -> (audio_source, video).
|
||||
@pytest.mark.parametrize(
|
||||
"position,audio_source,video",
|
||||
[
|
||||
("off", "none", False),
|
||||
("white_noise", "white_noise", False),
|
||||
("music", "music", False),
|
||||
("audio_track", "audio_track", False),
|
||||
("video", "none", True),
|
||||
("music_video", "music", True),
|
||||
("audio_video", "audio_track", True),
|
||||
],
|
||||
)
|
||||
def test_resolve_content_matches_spec_table(position, audio_source, video):
|
||||
assert resolve_content(position) == ContentResolution(audio_source=audio_source, video=video)
|
||||
|
||||
|
||||
def test_off_is_void_state_black_and_silent():
|
||||
r = resolve_content("off")
|
||||
assert r.video is False and r.audio_source == "none"
|
||||
|
||||
|
||||
def test_resolve_content_rejects_unknown_position():
|
||||
with pytest.raises(ValueError):
|
||||
resolve_content("bogus")
|
||||
@@ -0,0 +1,53 @@
|
||||
import pytest
|
||||
|
||||
from player.controls import (
|
||||
Controls,
|
||||
CONTENT_POSITIONS,
|
||||
ControlsError,
|
||||
validate_controls,
|
||||
parse_controls,
|
||||
)
|
||||
|
||||
|
||||
def test_content_positions_are_the_seven_from_the_spec():
|
||||
assert CONTENT_POSITIONS == frozenset(
|
||||
{"off", "white_noise", "music", "audio_track", "video", "music_video", "audio_video"}
|
||||
)
|
||||
|
||||
|
||||
def test_valid_controls_pass_validation():
|
||||
c = Controls(content="video", left=0, right=4, dark=2, light=2, volume=3, brightness=4)
|
||||
validate_controls(c) # does not raise
|
||||
|
||||
|
||||
def test_invalid_content_position_rejected():
|
||||
c = Controls(content="bogus", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(c)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["left", "right", "dark", "light", "volume", "brightness"])
|
||||
@pytest.mark.parametrize("bad", [-1, 5, True])
|
||||
def test_out_of_range_or_non_int_knob_rejected(field, bad):
|
||||
kwargs = dict(content="off", left=0, right=0, dark=0, light=0, volume=0, brightness=0)
|
||||
kwargs[field] = bad
|
||||
with pytest.raises(ControlsError):
|
||||
validate_controls(Controls(**kwargs))
|
||||
|
||||
|
||||
def test_parse_controls_from_mapping():
|
||||
c = parse_controls(
|
||||
{"content": "music_video", "left": 1, "right": 2, "dark": 3, "light": 0, "volume": 2, "brightness": 1}
|
||||
)
|
||||
assert c == Controls("music_video", 1, 2, 3, 0, 2, 1)
|
||||
|
||||
|
||||
def test_parse_controls_rejects_unknown_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0, "right": 0, "dark": 0,
|
||||
"light": 0, "volume": 0, "brightness": 0, "bogus": 1})
|
||||
|
||||
|
||||
def test_parse_controls_rejects_missing_keys():
|
||||
with pytest.raises(ControlsError):
|
||||
parse_controls({"content": "off", "left": 0})
|
||||
@@ -0,0 +1,112 @@
|
||||
from dataclasses import dataclass
|
||||
|
||||
import pytest
|
||||
|
||||
from player.controls import Controls
|
||||
from player.state import Player, Playback, Transition, TransitionKind
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FakeClip:
|
||||
id: str
|
||||
|
||||
|
||||
LIB = [FakeClip("base-a"), FakeClip("base-b")]
|
||||
|
||||
|
||||
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
|
||||
return Controls(content, left, right, dark, light, volume, brightness)
|
||||
|
||||
|
||||
def test_first_update_to_video_fades_in_from_black():
|
||||
p = Player(LIB)
|
||||
t = p.update(_controls(content="video"))
|
||||
assert t.kind == TransitionKind.FADE_FROM_BLACK
|
||||
assert t.playback.clip_id == "base-a"
|
||||
assert t.playback.content.video is True
|
||||
|
||||
|
||||
def test_off_from_video_fades_to_black_and_silences():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video"))
|
||||
t = p.update(_controls(content="off"))
|
||||
assert t.kind == TransitionKind.FADE_TO_BLACK
|
||||
assert t.playback.clip_id is None
|
||||
assert t.playback.content.audio_source == "none"
|
||||
|
||||
|
||||
def test_no_change_yields_none_transition():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=1))
|
||||
t = p.update(_controls(content="video", left=1))
|
||||
assert t.kind == TransitionKind.NONE
|
||||
|
||||
|
||||
def test_grade_change_is_a_live_update_not_a_crossfade():
|
||||
# design §4.3: the Dark/Light grade is a continuous runtime op
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", dark=0, light=0))
|
||||
t = p.update(_controls(content="video", dark=4, light=0))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.grade.tone == -1.0
|
||||
|
||||
|
||||
def test_overlay_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", left=0))
|
||||
t = p.update(_controls(content="video", left=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.plan.overlay.intensity == 1.0
|
||||
|
||||
|
||||
def test_restyle_change_crossfades_the_substrate():
|
||||
# design §4.3: the Right v2v substrate is a pre-baked variant swap
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", right=0))
|
||||
t = p.update(_controls(content="video", right=4))
|
||||
assert t.kind == TransitionKind.CROSSFADE
|
||||
assert t.playback.plan.restyle.blend == 1.0
|
||||
|
||||
|
||||
def test_volume_only_change_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="video", volume=1))
|
||||
t = p.update(_controls(content="video", volume=4))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.volume == 4
|
||||
|
||||
|
||||
def test_audio_source_change_while_black_is_a_live_update():
|
||||
p = Player(LIB)
|
||||
p.update(_controls(content="white_noise"))
|
||||
t = p.update(_controls(content="music"))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE
|
||||
assert t.playback.content.audio_source == "music"
|
||||
|
||||
|
||||
def test_injected_base_chooser_is_used():
|
||||
p = Player(LIB, choose_base=lambda lib: lib[1])
|
||||
t = p.update(_controls(content="video"))
|
||||
assert t.playback.clip_id == "base-b"
|
||||
|
||||
|
||||
def test_empty_library_with_video_raises():
|
||||
p = Player([])
|
||||
with pytest.raises(ValueError):
|
||||
p.update(_controls(content="video"))
|
||||
|
||||
|
||||
def test_off_with_empty_library_is_fine():
|
||||
p = Player([])
|
||||
# levels at 0 match the initial black state, so this is a no-op transition
|
||||
t = p.update(_controls(content="off", volume=0, brightness=0))
|
||||
assert t.kind == TransitionKind.NONE # already black at init
|
||||
assert t.playback.clip_id is None
|
||||
|
||||
|
||||
def test_off_with_levels_from_black_is_a_live_update():
|
||||
p = Player([])
|
||||
t = p.update(_controls(content="off", volume=3, brightness=2))
|
||||
assert t.kind == TransitionKind.LIVE_UPDATE # black->black, levels set
|
||||
assert t.playback.clip_id is None
|
||||
assert t.playback.volume == 3
|
||||
@@ -0,0 +1,87 @@
|
||||
import random
|
||||
|
||||
import pytest
|
||||
|
||||
from hef.catalog import Record
|
||||
from hef.selection import Coordinate, Weights, ranked_candidates, select
|
||||
|
||||
|
||||
def make_record(**overrides):
|
||||
base = dict(
|
||||
id="r",
|
||||
title="t",
|
||||
source_url="u",
|
||||
source_archive="internet_archive",
|
||||
license="public_domain",
|
||||
mode="video",
|
||||
left=0,
|
||||
right=0,
|
||||
dark=0,
|
||||
light=0,
|
||||
duration_s=600,
|
||||
file_path="",
|
||||
)
|
||||
base.update(overrides)
|
||||
return Record(**base)
|
||||
|
||||
|
||||
def test_ranked_candidates_sorts_nearest_first_with_distances():
|
||||
near = make_record(id="near", mode="video", left=1, right=1, dark=0, light=0)
|
||||
far = make_record(id="far", mode="video", left=4, right=4, dark=4, light=4)
|
||||
ranked = ranked_candidates([far, near], Coordinate(0, 0, 0, 0), "video")
|
||||
assert [r.id for r, _ in ranked] == ["near", "far"]
|
||||
assert ranked[0][1] < ranked[1][1]
|
||||
|
||||
|
||||
def test_ranked_candidates_caps_at_pool_size():
|
||||
recs = [make_record(id=f"r{i}", mode="video", left=i % 5) for i in range(10)]
|
||||
ranked = ranked_candidates(recs, Coordinate(0, 0, 0, 0), "video", pool_size=3)
|
||||
assert len(ranked) == 3
|
||||
|
||||
|
||||
def test_ranked_candidates_filters_by_mode():
|
||||
a = make_record(id="aud", mode="audio")
|
||||
v = make_record(id="vid", mode="video")
|
||||
ranked = ranked_candidates([a, v], Coordinate(0, 0, 0, 0), "audio")
|
||||
assert [r.id for r, _ in ranked] == ["aud"]
|
||||
|
||||
|
||||
def test_ranked_candidates_av_falls_back_to_audio_and_video():
|
||||
a = make_record(id="aud", mode="audio")
|
||||
v = make_record(id="vid", mode="video")
|
||||
ranked = ranked_candidates([a, v], Coordinate(0, 0, 0, 0), "av", pool_size=4)
|
||||
assert {r.id for r, _ in ranked} == {"aud", "vid"}
|
||||
|
||||
|
||||
def test_ranked_candidates_approved_only():
|
||||
p = make_record(id="prop", mode="video", review_status="proposed")
|
||||
ok = make_record(id="appr", mode="video", review_status="approved")
|
||||
ranked = ranked_candidates([p, ok], Coordinate(0, 0, 0, 0), "video", approved_only=True)
|
||||
assert [r.id for r, _ in ranked] == ["appr"]
|
||||
|
||||
|
||||
def test_ranked_candidates_rejects_none_mode():
|
||||
with pytest.raises(ValueError):
|
||||
ranked_candidates([], Coordinate(0, 0, 0, 0), "none")
|
||||
|
||||
|
||||
def test_select_returns_rank_one_of_ranked_candidates():
|
||||
recs = [
|
||||
make_record(id="near", mode="video", left=1),
|
||||
make_record(id="far", mode="video", left=4),
|
||||
]
|
||||
coord = Coordinate(0, 0, 0, 0)
|
||||
ranked = ranked_candidates(recs, coord, "video")
|
||||
assert select(recs, coord, "video", rng=None).id == ranked[0][0].id
|
||||
|
||||
|
||||
def test_select_none_mode_still_returns_none():
|
||||
recs = [make_record(id="v", mode="video")]
|
||||
assert select(recs, Coordinate(0, 0, 0, 0), "none") is None
|
||||
|
||||
|
||||
def test_select_shuffles_within_pool_when_rng_given():
|
||||
recs = [make_record(id=f"r{i}", mode="video", left=i) for i in range(5)]
|
||||
coord = Coordinate(0, 0, 0, 0)
|
||||
picks = {select(recs, coord, "video", pool_size=5, rng=random.Random(s)).id for s in range(20)}
|
||||
assert len(picks) > 1
|
||||
@@ -0,0 +1,102 @@
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from hef.catalog import Record
|
||||
from simulator.app import create_app
|
||||
|
||||
|
||||
def make_record(**overrides):
|
||||
base = dict(
|
||||
id="r",
|
||||
title="t",
|
||||
source_url="u",
|
||||
source_archive="internet_archive",
|
||||
license="public_domain",
|
||||
mode="video",
|
||||
left=0,
|
||||
right=0,
|
||||
dark=0,
|
||||
light=0,
|
||||
duration_s=600,
|
||||
file_path="",
|
||||
)
|
||||
base.update(overrides)
|
||||
return Record(**base)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client():
|
||||
records = [
|
||||
make_record(id="v-near", mode="video", left=0, right=0, dark=0, light=0),
|
||||
make_record(id="v-far", mode="video", left=4, right=4, dark=4, light=4),
|
||||
make_record(id="a-one", mode="audio", left=1, right=1, dark=1, light=1),
|
||||
make_record(id="prop", mode="video", left=0, right=0, dark=0, light=1,
|
||||
review_status="proposed"),
|
||||
make_record(id="appr", mode="video", left=0, right=0, dark=0, light=1,
|
||||
review_status="approved"),
|
||||
]
|
||||
return TestClient(create_app(records=records))
|
||||
|
||||
|
||||
def _body(**overrides):
|
||||
base = dict(left=0, right=0, dark=0, light=0, mode="video")
|
||||
base.update(overrides)
|
||||
return base
|
||||
|
||||
|
||||
def test_select_returns_pick_and_ranked_pool(client):
|
||||
resp = client.post("/api/select", json=_body(mode="video", pool_size=4))
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["pick"]["id"] == "v-near"
|
||||
ids = [c["record"]["id"] for c in data["pool"]]
|
||||
assert ids[0] == "v-near"
|
||||
assert all("distance" in c and "rank" in c for c in data["pool"])
|
||||
assert [c["rank"] for c in data["pool"]] == list(range(1, len(data["pool"]) + 1))
|
||||
|
||||
|
||||
def test_none_mode_is_the_void(client):
|
||||
resp = client.post("/api/select", json=_body(mode="none"))
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["pick"] is None
|
||||
assert data["pool"] == []
|
||||
|
||||
|
||||
def test_dial_out_of_range_is_rejected(client):
|
||||
resp = client.post("/api/select", json=_body(left=7))
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
def test_bad_mode_is_rejected(client):
|
||||
resp = client.post("/api/select", json=_body(mode="banana"))
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
def test_approved_only_narrows_pool(client):
|
||||
resp = client.post("/api/select", json=_body(left=0, right=0, dark=0, light=1,
|
||||
mode="video", approved_only=True))
|
||||
data = resp.json()
|
||||
assert all(c["record"]["review_status"] == "approved" for c in data["pool"])
|
||||
|
||||
|
||||
def test_catalog_meta_reports_counts(client):
|
||||
resp = client.get("/api/catalog/meta")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["total"] == 5
|
||||
assert data["by_mode"]["video"] == 4
|
||||
assert data["by_mode"]["audio"] == 1
|
||||
assert set(data["by_status"]) == {"proposed", "approved"}
|
||||
|
||||
|
||||
from simulator.app import create_app as _create_app_for_static
|
||||
|
||||
|
||||
def test_index_is_served():
|
||||
# The default app mounts the real static dir.
|
||||
client = TestClient(_create_app_for_static())
|
||||
resp = client.get("/")
|
||||
assert resp.status_code == 200
|
||||
assert "text/html" in resp.headers["content-type"]
|
||||
assert "X-ray" in resp.text
|
||||
Reference in New Issue
Block a user