Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 3ef21fb7f7 | |||
| 5290785e2a |
@@ -0,0 +1,242 @@
|
||||
# Human Experience Filter — Simulator Alteration Preview (Design)
|
||||
|
||||
**Date:** 2026-06-06
|
||||
**Status:** Approved design (pre-implementation)
|
||||
**Repo:** `human-experience-filter-art`
|
||||
**Builds on:** the alteration engine of
|
||||
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
|
||||
(the "design" below) and the simulator scaffold of
|
||||
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md).
|
||||
**Revises the design:** §4.3 and §10 — the Left analytical HUD is **no longer a
|
||||
runtime overlay**; it is **baked into authored variant videos** (see §8 below).
|
||||
This is a deliberate trade of near-free multilingual support for authorial
|
||||
precision over the HUD.
|
||||
**Retires:** the simulator's selection-era "curator's X-ray" view and its
|
||||
`/api/select` + `/api/catalog/meta` endpoints (the selection model they
|
||||
visualize was superseded by the alteration model).
|
||||
|
||||
> **Why this exists.** Operator directive (session 0007): *build and design only
|
||||
> things that run in the simulator, and get the whole experience working the way
|
||||
> we like in the simulator before moving to hardware.* The slice-1 alteration
|
||||
> engine (PR #5) is pure logic with **no simulator surface** — you cannot yet turn
|
||||
> the experience knobs and *see* the result. This design brings the alteration
|
||||
> into the simulator so the look can be tuned and liked before any Pi/serial work.
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope
|
||||
|
||||
A browser **alteration preview**: turn the four experience knobs and the content
|
||||
dial, and see the neutral base footage altered toward the knob state, in real
|
||||
time, on a looping clip. The purpose is to **tune the look** of the filter and to
|
||||
**settle the knob→strength calibration by eye** (the open §3-vs-§4.2/§5 question
|
||||
from session 0006).
|
||||
|
||||
**In scope**
|
||||
- Live preview of the four experience knobs (Left/Right/Dark/Light) on a looping
|
||||
base clip.
|
||||
- **Dark/Light** rendered as a live, deterministic runtime color grade.
|
||||
- **Left/Right** rendered by selecting a **pre-baked authored variant clip** from
|
||||
a 5×5 grid and crossfading on change.
|
||||
- A **calibration panel** that adjusts the grade curve live; the chosen values are
|
||||
baked back into `player/alteration.py` defaults.
|
||||
- A **RenderPlan readout** (the project's "X-ray" honesty): always show the exact
|
||||
numbers the engine produced.
|
||||
- The content dial's **video on/off** behavior (so "Off" goes to black), driven by
|
||||
the existing `resolve_content`.
|
||||
- Placeholder variant generation so the mechanism is testable before the operator
|
||||
authors real videos.
|
||||
|
||||
**Out of scope (this slice)** — real generative v2v; serial input / the 3⇄4
|
||||
framing contract; audio playback (music / white-noise / audio-track); the Pi/mpv
|
||||
runtime renderer; the full transition/crossfade timing engine; multilingual label
|
||||
tables; catalog-model changes. These remain later roadmap slices.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture — Python-canonical, thin browser renderer
|
||||
|
||||
`player/alteration.py` stays the **single source of truth** for the alteration
|
||||
math (handbook §4.2 — deterministic core, thin I/O). The browser owns only
|
||||
*rendering*. Data flow:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Browser
|
||||
K[4 experience knobs +<br/>content dial +<br/>calibration sliders]
|
||||
V["<video> variant + canvas grade"]
|
||||
RO[RenderPlan readout]
|
||||
end
|
||||
subgraph FastAPI [simulator/app.py]
|
||||
EP["POST /api/alteration"]
|
||||
CL["GET /api/clips"]
|
||||
end
|
||||
ENG["player.alteration.plan_alteration(coord, calibration)"]
|
||||
K -- "debounced POST {controls, calibration}" --> EP
|
||||
EP --> ENG --> EP -- "RenderPlan {variant, grade}" --> V
|
||||
EP --> RO
|
||||
CL -- "base clip + variant manifest" --> V
|
||||
```
|
||||
|
||||
- The browser sends control/calibration changes (debounced) and receives a
|
||||
`RenderPlan`. Video filtering itself runs continuously in the browser; only a
|
||||
*plan recompute* makes a round-trip. On localhost these JSON round-trips are
|
||||
imperceptible.
|
||||
- "Bake the calibration winner in" = change the `Calibration` defaults in Python.
|
||||
Nothing in the browser is canonical.
|
||||
|
||||
---
|
||||
|
||||
## 3. The four axes — how each is rendered
|
||||
|
||||
| Axis | Engine output | Browser rendering |
|
||||
|---|---|---|
|
||||
| **Dark / Light** | `ColorGrade.tone` ∈ [−1, 1], center = identity | live color grade on the `<video>`: Light → warm/yellow + negative space toward white; Dark → cool/blue + negative space toward black; 0 = raw. (Confirmed direction in brainstorm.) |
|
||||
| **Left / Right** | `VariantRef(left, right)` | select the authored variant clip for `(left, right)`; **crossfade** when it changes; `(0,0)` → raw base. The analytical HUD (Left) and dreamlike restyle (Right) are **baked into the clip** (§8). |
|
||||
|
||||
Dark/Light is the only continuously-tunable axis in this slice; Left/Right is a
|
||||
discrete selection by coordinate.
|
||||
|
||||
---
|
||||
|
||||
## 4. The variant grid (per base clip)
|
||||
|
||||
Each base clip carries a 5×5 grid keyed by the two baked axes:
|
||||
|
||||
- Rows = **Left** (analytical / HUD), 0–4. Columns = **Right** (artistic /
|
||||
dreamlike restyle), 0–4.
|
||||
- `(0,0)` = the raw base clip — no authored file needed.
|
||||
- The other **24 cells are authored videos** (4 analytical-edge + 4 dreamlike-edge
|
||||
+ 16 combined core). This is **per base clip**; authoring load multiplies by the
|
||||
number of base clips (and by language, since the HUD is baked — §8).
|
||||
|
||||
The grid is **data**, not code: a manifest the simulator reads (see §6).
|
||||
|
||||
---
|
||||
|
||||
## 5. Engine reconciliation (`player/alteration.py`)
|
||||
|
||||
The slice-1 engine modeled Left/Right as continuous runtime layers
|
||||
(`AnalyticalOverlay.intensity`, `Restyle.blend`). Baked variants make that
|
||||
obsolete. Changes:
|
||||
|
||||
- **`RenderPlan`** becomes `{ grade: ColorGrade, variant: VariantRef }`.
|
||||
- `ColorGrade` — unchanged.
|
||||
- `VariantRef(left, right)` — new; identity selection by coordinate.
|
||||
- **Remove** `AnalyticalOverlay` and `Restyle` (their content is now baked into the
|
||||
variant clip) rather than leaving dead fields.
|
||||
- **`Calibration`** — a new frozen dataclass parameterizing the grade curve (e.g.
|
||||
`mood_center`, per-axis curve), defaulting to **today's exact behavior**
|
||||
(behavior-preserving). `plan_alteration(coord, calibration=DEFAULT_CALIBRATION)`.
|
||||
- **`player/state.py`** — the `CROSSFADE` trigger switches from "restyle changed"
|
||||
to "variant changed"; `LIVE_UPDATE` still covers a grade-only change. Video
|
||||
on/off fades unchanged.
|
||||
|
||||
These are framework-code changes that keep the engine pure and unit-tested; they
|
||||
do **not** bake any deployment-shape decision into the engine.
|
||||
|
||||
---
|
||||
|
||||
## 6. Data & endpoints
|
||||
|
||||
- **`simulator/clips.py`** (replaces `simulator/fixtures.py`): reads a base-clip +
|
||||
variant manifest. Each base clip: `{ id, title, base_file, license, source,
|
||||
variants: { "L,R": { file, model?, hud_lang? } } }`. Missing cells fall back to
|
||||
the raw base (and are flagged in the readout as "raw / unauthored").
|
||||
- **`POST /api/alteration`**: body `{ controls, calibration }` → `RenderPlan`
|
||||
(plus the `ContentResolution` from `resolve_content`, so the dial's video-on/off
|
||||
is honored). The endpoint calls the real `plan_alteration`.
|
||||
- **`GET /api/clips`**: the base-clip list + variant manifest for the active base.
|
||||
- Video files served as static assets from a sample-media directory.
|
||||
- **Removed**: `POST /api/select`, `GET /api/catalog/meta`.
|
||||
- `hef.selection` (the library, incl. `Coordinate`) is **untouched**; only the
|
||||
simulator's selection view/endpoints retire. (`ranked_candidates`, added for the
|
||||
old X-ray, may become unused by the simulator — noted, not removed here.)
|
||||
|
||||
---
|
||||
|
||||
## 7. Calibration tuning
|
||||
|
||||
The calibration panel exposes the grade-curve parameters (the two conventions in
|
||||
tension: `(2,2,2,2)`-centered vs the `value/4` reading currently implemented, plus
|
||||
curve shape). Changing them re-requests the plan and the footage responds live.
|
||||
When the operator is happy, the chosen values are written into
|
||||
`DEFAULT_CALIBRATION` in `player/alteration.py` and locked in by a unit test.
|
||||
|
||||
---
|
||||
|
||||
## 8. Baked HUD — the §4.3/§10 revision (accepted trade)
|
||||
|
||||
The design's §4.3/§10 kept the Left HUD a **runtime overlay** specifically so the
|
||||
v2v substrate is language-agnostic and multilingual support is near-free. This
|
||||
design **reverses that for the Left/Right plane**: the operator authors the HUD
|
||||
**into** the variant videos to get pixel-precise control over the HUD and the
|
||||
analytical↔feeling balance.
|
||||
|
||||
**Consequences (accepted):**
|
||||
- Multilingual support is **no longer near-free**: baked HUD text means a new
|
||||
language re-renders every label-bearing variant (the §10 "budget landmine").
|
||||
The piece is English-first; broad i18n is deferred/expensive.
|
||||
- The live Dark/Light grade is applied **on top** of the baked cell, so it **tints
|
||||
the baked HUD** too (a "Dark" mood cools/darkens the HUD). With baked overlays
|
||||
the grade cannot skip the HUD. Escape hatch: author HUD colors that survive
|
||||
grading, or (future) reinstate a runtime HUD layer.
|
||||
|
||||
The parent design's §4.3/§10 must be updated to point at this revision (a task for
|
||||
the implementation plan).
|
||||
|
||||
---
|
||||
|
||||
## 9. Bootstrapping before authored videos exist
|
||||
|
||||
The operator will author the 24 variants per base clip; none exist yet. So the
|
||||
mechanism is testable immediately, the build includes a **placeholder generator**:
|
||||
for at least one base clip, produce the 24 cells by burning the cell's `L,R` (and
|
||||
a stub HUD caption) into the base loop via ffmpeg. Real authored clips drop into
|
||||
the manifest with **no code change**.
|
||||
|
||||
---
|
||||
|
||||
## 10. Testing
|
||||
|
||||
- **`player/` unit tests:** `DEFAULT_CALIBRATION` is behavior-preserving vs the
|
||||
current helpers; the two calibration conventions produce the expected plans;
|
||||
`VariantRef` selection (incl. `(0,0)` → raw) and the `state.py` crossfade
|
||||
trigger on variant change.
|
||||
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
|
||||
`POST /api/alteration` returns the engine's plan for given controls/calibration;
|
||||
`GET /api/clips` returns the manifest; the removed endpoints are gone.
|
||||
- **No browser/E2E automation** this slice (manual visual tuning is the point);
|
||||
the rendering JS is kept thin and the engine logic stays in tested Python.
|
||||
|
||||
---
|
||||
|
||||
## 11. What ships
|
||||
|
||||
- `player/alteration.py` (parameterized `Calibration`, `VariantRef`, slimmed
|
||||
`RenderPlan`) + `player/state.py` crossfade-trigger update.
|
||||
- `simulator/clips.py` (variant manifest) replacing `fixtures.py`.
|
||||
- `simulator/app.py` new endpoints; selection endpoints removed.
|
||||
- `simulator/static/` rewritten as the Player preview (variant `<video>` +
|
||||
crossfade, live grade, calibration panel, RenderPlan readout, content dial).
|
||||
- Placeholder-variant generator + a sample base clip.
|
||||
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten;
|
||||
parent design §4.3/§10 pointer updated; `docs/ROADMAP.md` §3 updated.
|
||||
|
||||
---
|
||||
|
||||
## 12. Open questions (for the plan, not blockers)
|
||||
|
||||
- **Grade vs baked HUD interaction** — accepted that the grade tints the HUD;
|
||||
revisit only if it reads badly once real authored clips exist.
|
||||
- **Sample base clip** — pick one CC0 nature loop for the placeholder grid.
|
||||
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for
|
||||
tuning; the real timing engine is a later slice.
|
||||
|
||||
---
|
||||
|
||||
## 13. Out of scope (YAGNI)
|
||||
|
||||
Real generative v2v; serial / firmware; audio playback; the Pi renderer; the full
|
||||
transition engine; multilingual tables; catalog-model changes; retiring
|
||||
`hef.selection.ranked_candidates`. All remain later roadmap work.
|
||||
@@ -0,0 +1,134 @@
|
||||
# Session 0006.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Type: coding
|
||||
> Start: 2026-06-05T17-52 (PST) · End: 2026-06-05T18-15 (PST)
|
||||
> Goal: `/goal next` — drive the roadmap to its next frontier item (sub-project 3,
|
||||
> the player / machine-alteration engine), and build it.
|
||||
> Outcome: **Sub-project 3 slice 1 — the pure-logic alteration-engine + player
|
||||
> core — built and MERGED to `main` via PR #5** (merge commit `aeeed66`).
|
||||
> New `player/` package; 56 new tests (189 passed / 2 skipped total).
|
||||
> Sub-project 3 advanced from "next" to "in progress."
|
||||
|
||||
## Plan
|
||||
|
||||
Begin sub-project 3 (Player Runtime + alteration engine) per `docs/ROADMAP.md` §3
|
||||
and the approved machine-altered-perception design (§4 alteration engine, §5 mood
|
||||
grade, §6 7-way content dial, §7 intensity). Sub-project 3 is large (runtime
|
||||
grading + overlay compositing + v2v variant selection + serial input + mpv/GPU +
|
||||
white-noise DSP); this session ships **slice 1 — the pure-logic core** as a new
|
||||
`player/` package, fully unit-tested with all I/O behind injected interfaces,
|
||||
mirroring how sub-project 1 was the pure-logic dependency root. Process:
|
||||
writing-plans → TDD → branch → PR → merge.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
/goal next — drive the roadmap to its next frontier item.
|
||||
|
||||
Per project memory (session 0005, machine-altered-perception design SPEC merged
|
||||
to main via PR #4), the roadmap frontier is sub-project 3: the player /
|
||||
machine-alteration engine. This coding session builds that.
|
||||
```
|
||||
|
||||
## Pre-session state
|
||||
|
||||
- `main` @ `77746e4` locally (behind `origin/main` by 2 — session 0005's finalized
|
||||
transcript + this session's claim placeholder); fast-forwarded to `origin/main`
|
||||
during init.
|
||||
- Sub-projects 1 & 2 done/merged; design of record = the machine-altered-perception
|
||||
revision (PR #4). No `player/` package and no sub-project-3 plan existed yet.
|
||||
- Launch cwd was the stale, already-merged worktree
|
||||
`.worktrees/feature-BuildSimulator11/` (branch `feature/machine-altered-perception-spec`,
|
||||
remote gone) — same situation as session 0005.
|
||||
|
||||
## Turn-by-turn arc
|
||||
|
||||
1. **Gate / classify.** `/goal next` → unambiguously a coding session (roadmap
|
||||
frontier = sub-project 3 build). Ran `wgl-session-coding-init`.
|
||||
2. **Claim snag (same as 0005, kept in the record).** `resolve-app.py` reported
|
||||
**ambiguous** — the nested worktree's `app.json` discovered alongside the main
|
||||
clone's. Read the resolver to confirm root cause (`find_app_jsons` prunes
|
||||
`.git/node_modules/…` but not `.worktrees/`). Worked around with
|
||||
`WGL_APP_SCAN_DEPTH=4` (prunes the depth-5 worktree copy, keeps the depth-3 main
|
||||
one). Claimed session **0006**, type coding, at `e694489`. Filed a plugin-feedback
|
||||
note for the recurring gap (`feedback/2026-06-05T18-13-resolver-nested-worktree-ambiguous.md`).
|
||||
3. **Baseline + branch.** Fast-forwarded the main clone; confirmed PR #4 merged;
|
||||
created `feature/player-alteration-core` off `origin/main` in the worktree.
|
||||
4. **Read the design.** Read ROADMAP §3 + the machine-altered-perception spec
|
||||
(§4 alteration engine, §5 mood grade, §6 7-way dial, §7 intensity), and the
|
||||
existing `hef/` conventions (frozen dataclasses, pure modules, heavy unit tests).
|
||||
5. **Scoped the session → slice 1.** Sub-project 3 is too large for one session;
|
||||
chose the pure-logic core (controls model, content-dial resolver, alteration
|
||||
render-plan, player state machine) with all I/O behind injected interfaces, and
|
||||
deferred the renderer/serial/white-noise/v2v/catalog slices (named in the plan).
|
||||
6. **Surfaced a spec ambiguity (Deferred decision).** §3 "(2,2,2,2) neutral" vs
|
||||
§4.2 (Left/Right stack as independent layers) vs §5 (mood center = identity)
|
||||
cannot all be literally true for four independent knobs. Resolved with a
|
||||
swappable calibration: brain `value/4`, mood `(light−dark)/4`; flagged for
|
||||
operator confirm.
|
||||
7. **writing-plans.** Wrote `docs/superpowers/plans/2026-06-05-player-alteration-core.md`
|
||||
(6 TDD tasks, full code + tests, self-review against the spec).
|
||||
8. **TDD implementation.** Red→green→commit per module: `player/controls.py`
|
||||
(Controls, validation, parse), `player/content.py` (§6 7-way table),
|
||||
`player/alteration.py` (RenderPlan: ColorGrade/AnalyticalOverlay/Restyle +
|
||||
calibration helpers), `player/state.py` (Player state machine + Transition
|
||||
kinds). Registered the package in `pyproject.toml`; updated ROADMAP §3.
|
||||
9. **Verify.** Full suite green: **189 passed, 2 skipped** (the opt-in ffprobe
|
||||
tests), 56 of them new player tests. No regressions.
|
||||
10. **Ship.** Pushed the branch; created **PR #5** and merged it via the Gitea API
|
||||
helper (merge commit `aeeed66`); synced the main clone; deleted the remote
|
||||
feature branch.
|
||||
11. **Wrap.** Filed plugin feedback, updated memory, finalized.
|
||||
|
||||
## Cut state
|
||||
|
||||
- `main` @ `aeeed66` (PR #5 merged). `player/` package present: `controls.py`,
|
||||
`content.py`, `alteration.py`, `state.py` + 4 test files. Full suite 189 passed /
|
||||
2 skipped.
|
||||
- Sub-project 3 = **⏳ in progress**. Slice 1 done; remaining slices listed in
|
||||
ROADMAP §3 and `[[sub-project-3-player-progress]]`.
|
||||
- Working tree clean. Remote feature branch deleted; the local worktree remains on
|
||||
the now-merged `feature/player-alteration-core` branch (harmless).
|
||||
|
||||
## Deferred decisions (operator plate)
|
||||
|
||||
Two low-confidence calls made autonomously this session — both flagged for operator
|
||||
confirmation:
|
||||
|
||||
1. **Knob→strength calibration (§3 vs §4.2/§5 conflict).** §3 says "(2,2,2,2) is
|
||||
neutral," but §4.2 (Left/Right stack as independent layers: overlay vs substrate)
|
||||
and §5 (mood center = identity) conflict with that for four independent physical
|
||||
knobs. Implemented: brain knobs `strength = value/4` (0 = off, 4 = max — full
|
||||
range, honors §4.2 stacking), mood `tone = (light − dark)/4` (equal dark/light →
|
||||
identity, honors §5). Under this the un-altered base is `(0,0,·,·)` with
|
||||
`dark==light`, **not** `(2,2,2,2)` — read as a vestige of the old coordinate-grid
|
||||
*center*. The calibration is three one-line helpers in `player/alteration.py`;
|
||||
flipping to a "centered at 2 = no push" convention is a localized change that
|
||||
leaves the RenderPlan shape and all downstream logic untouched. **Recommend
|
||||
operator confirm the intended panel UX.**
|
||||
2. **Volume/Brightness granularity.** Modeled as `0..4` ints for uniformity with the
|
||||
experience knobs and the Arduino's `0..4` quantization (roadmap §4). The spec
|
||||
calls them "levels" without fixing granularity; revisit if the panel warrants
|
||||
finer steps.
|
||||
|
||||
## Next-session prompt
|
||||
|
||||
The roadmap frontier is now the remaining sub-project-3 slices. Read
|
||||
`[[sub-project-3-player-progress]]` and the slice-1 plan first. Two strong
|
||||
candidates for the next slice — the **runtime renderer** (makes the core visible)
|
||||
or the **serial framing contract + input adapter** (unblocks sub-project 4 in
|
||||
parallel):
|
||||
|
||||
```
|
||||
/goal Sub-project 3 slice 2 — settle the 3⇄4 serial framing contract and build the
|
||||
USB-serial / keyboard input adapter that feeds the player a `Controls` stream
|
||||
(unblocks sub-project 4), per docs/ROADMAP.md §3 and docs/superpowers/plans/2026-06-05-player-alteration-core.md.
|
||||
Alternatively start the runtime renderer (mpv/ffmpeg + grade/overlay). First
|
||||
confirm the open knob→strength calibration decision (§3 vs §4.2/§5).
|
||||
```
|
||||
|
||||
Gotcha to carry forward: running from the nested `.worktrees/…` checkout breaks
|
||||
`resolve-app.py` (ambiguous app.json) — run from the main clone, or use
|
||||
`WGL_APP_SCAN_DEPTH=4` on claim and explicit `--sessions-*` flags on publish, until
|
||||
the plugin fix lands.
|
||||
@@ -1,27 +0,0 @@
|
||||
# Session 0006.0 — Transcript
|
||||
|
||||
> App: human-experience-filter-art
|
||||
> Start: 2026-06-05T17-52 (PST)
|
||||
> Type: coding
|
||||
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
|
||||
>
|
||||
> This file reserves session ID 0006 for human-experience-filter-art. The driver replaces this
|
||||
> body with the full transcript and renames the file to its final
|
||||
> SESSION-0006.0-TRANSCRIPT-2026-06-05T17-52--<end>.md form at session end.
|
||||
|
||||
## Launch prompt
|
||||
|
||||
```
|
||||
/goal next — drive the roadmap to its next frontier item.
|
||||
|
||||
Per project memory (session 0005, machine-altered-perception design SPEC merged
|
||||
to main via PR #4), the roadmap frontier is sub-project 3: the player /
|
||||
machine-alteration engine. This coding session builds that.
|
||||
|
||||
```
|
||||
|
||||
## Deferred decisions
|
||||
|
||||
_Autonomous-mode low-confidence calls the driver made and would have
|
||||
liked operator input on. Appended as the session runs; surfaced at
|
||||
finalize. Empty if none._
|
||||
Reference in New Issue
Block a user