Compare commits

..

23 Commits

Author SHA1 Message Date
Ben Stull 6cf1ae08ff claim human-experience-filter-art session 0011 (placeholder) + sessions.json entry 2026-06-07 23:27:02 -07:00
Ben Stull 7cf550ac60 add sessions/0010/SESSION-0010.0-TRANSCRIPT-2026-06-07T23-09--2026-06-07T23-22.md + replace placeholder/variant SESSION-0010.0-TRANSCRIPT-2026-06-07T23-09--INPROGRESS.md 2026-06-07 23:23:47 -07:00
benstull 638cf5808f Merge pull request 'feat(sim+engine): lock alteration calibration + fix psychedelic dark grade' (#8) from feature/lock-alteration-calibration into main 2026-06-08 06:20:53 +00:00
Ben Stull 9c347d580f feat(sim+engine): lock alteration calibration + fix psychedelic dark grade
Tuning the look by eye in the simulator (design §8) surfaced that the dark
mood pole rendered a full-frame hue-rotate(-200deg) — rock orange, trees
purple: the disorienting look rejected in session 0008, not the peaceful POC
dark_frame. Replace it with darken + slight desaturate on the video filter
plus a multiply-blended deep-blue wash (#tint, below the SVG HUD so the
overlay stays legible). Dark now reads cool/somber with natural greens, like
the approved POC look; light/left/right unchanged and confirmed peaceful.

With full tilt now tasteful on every axis, LOCK DEFAULT_CALIBRATION to unity
gains + linear variant map as a deliberate by-eye choice (not placeholders),
closing the open session-0006 knob->strength decision: knobs run 0=off..4=max,
equal Dark/Light = identity, the 5 notches map 1:1 to the 5 discrete Right
bakes. Guard the locked constants with test_default_calibration_is_locked.

Resolves design §8 open questions (calibration curve shape; grade-vs-overlay
ordering) and records the dark-grade fix. Full suite 193 passed / 2 skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 23:19:53 -07:00
Ben Stull 7d2a3064ad claim human-experience-filter-art session 0010 (placeholder) + sessions.json entry 2026-06-07 23:09:52 -07:00
Ben Stull e753a68147 add sessions/0009/SESSION-0009.0-TRANSCRIPT-2026-06-07T22-41--2026-06-07T23-03.md + replace placeholder/variant SESSION-0009.0-TRANSCRIPT-2026-06-07T22-41--INPROGRESS.md 2026-06-07 23:04:40 -07:00
benstull 554eb5076f Merge pull request 'feat: reconciled simulator-first alteration slice (sessions 0007+0008)' (#7) from feature/reconciled-simulator-alteration-slice into main 2026-06-08 06:00:31 +00:00
Ben Stull b8543906be docs: point parent design at reconciliation; roadmap + user guide for sim alteration
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:58:46 -07:00
Ben Stull d2d63c0184 feat(simulator): alteration preview UI (grade + variant crossfade + Left overlay)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:57:13 -07:00
Ben Stull c3f9262a73 feat(simulator): sample-media manifest + POC setup/placeholder generator
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:56:15 -07:00
Ben Stull eb3aa0949d feat(simulator): /api/alteration + /api/clips; retire selection endpoints
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:55:10 -07:00
Ben Stull c316309fc6 feat(simulator): clips.py manifest model; retire selection fixtures
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:54:09 -07:00
Ben Stull 825d68c653 refactor(player): state.py tests track discrete Restyle.variant
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:53:40 -07:00
Ben Stull 12a8177793 feat(player): discrete Right variant + Left level + Calibration (slice design §2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:53:19 -07:00
Ben Stull e5ca07e2a4 docs(plan): implementation plan for reconciled simulator-alteration slice
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:52:01 -07:00
Ben Stull eda259e5b8 docs(spec): reconcile 0007/0008 sim-alteration designs (session 0009)
Bring the unmerged session-0007 simulator-alteration-preview design onto a
single thread and add the reconciliation design that resolves the load-bearing
Left-HUD conflict between it and the merged session-0008 scales-library /
right-axis design.

Decision (operator-approved): Left is a RUNTIME overlay (authored annotation
track + per-language string tables, shaped live — browser in the sim, Pango/
HarfBuzz on the Pi), not baked pixels. Consequently the pre-baked variant set is
1-D over Right strength, not 0007's 2-D 5x5 Left x Right grid; 0007's baked-HUD
position (its sec 4/5/8) is superseded. Engine change is surgical — the merged
slice-1 engine already keeps a runtime AnalyticalOverlay.

Scopes the first simulator-runnable slice: deterministic Dark/Light/Left +
discrete pre-baked flow-stabilized Right variants over ONE neutral clip, wired
into the simulator, with a parameterized Calibration tuned by eye.

Spec: docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:47:56 -07:00
Ben Stull 0a450bfd91 claim human-experience-filter-art session 0009 (placeholder) + sessions.json entry 2026-06-07 22:41:44 -07:00
Ben Stull c99a669194 add sessions/0008/SESSION-0008.0-TRANSCRIPT-2026-06-07T19-42--2026-06-07T22-37.md + replace placeholder/variant SESSION-0008.0-TRANSCRIPT-2026-06-07T19-42--INPROGRESS.md 2026-06-07 22:37:56 -07:00
benstull 42a72fe516 Merge pull request 'docs(spec): scales-of-nature library + stabilized right-axis pipeline (session 0008)' (#6) from feature/scales-library-right-axis into main 2026-06-08 05:34:48 +00:00
Ben Stull 0238260908 docs(spec): scales-of-nature library + stabilized right-axis pipeline (session 0008)
Design revision refining the machine-altered-perception design (2026-06-05):

- Right axis is a LOCAL offline restyle (SD img2img on MPS), not a cloud API
  (refines §4.1/§4.3, §9). Per-frame restyle boils/flickers — disqualifying for
  a peaceful piece — so temporal coherence is now a hard constraint, met by
  optical-flow keyframe propagation (the EbSynth principle, OpenCV impl).
- Content = a small NEUTRAL "scales of nature" library (~4-6 clips, one per
  scale), not a single stitched cosmic-zoom film; keeps the neutral-base thesis
  and interactivity while retaining the awe-of-scale richness (refines §6/§8).
- NEW: the scales form a navigable closed RING joined by short AI zoom/warp
  transitions, driven by an infinitely-turnable endless rotary encoder; diving
  past the microscopic wraps around to the cosmos (refines §2 selector / §11).
- i18n sharpened (§1.2, sharpens §10): Left labels are a Pi-rendered runtime
  graphics overlay (annotation-track + per-language string tables, shaped via
  Pango/HarfBuzz + Noto) — NOT a baked per-language overlay video, which would
  break i18n economics. Added Pi-resolution headroom to open questions.
- Strict-PD sourcing map (NASA/NOAA/NPS/USGS); non-US terrestrial is the CC-BY
  soft spot. Economics: local authoring is ~free, not $300-3k of cloud API.

Grounded in a local POC this session (M4 Pro, sd-turbo, OpenCV flow): all four
axes validated on real footage; deterministic axes ~2.4s, Right flow-propagated
~2.7min/8s clip and calm.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:31:12 -07:00
Ben Stull c653189397 claim human-experience-filter-art session 0008 (placeholder) + sessions.json entry 2026-06-07 19:43:22 -07:00
Ben Stull 92ed046602 add sessions/0007/SESSION-0007.0-TRANSCRIPT-2026-06-06T14-12--2026-06-06T17-57.md + replace placeholder/variant SESSION-0007.0-TRANSCRIPT-2026-06-06T14-12--INPROGRESS.md 2026-06-06 17:58:05 -07:00
Ben Stull d58a23e5a6 claim human-experience-filter-art session 0007 (placeholder) + sessions.json entry 2026-06-06 14:13:36 -07:00
29 changed files with 2975 additions and 465 deletions
+2
View File
@@ -5,3 +5,5 @@ __pycache__/
media/
.superpowers/
*.egg-info/
# Simulator sample media (look-tuning only; populate via setup_sample_media.py)
simulator/sample_media/forest/*.mp4
+22 -1
View File
@@ -123,6 +123,22 @@ longer *select* a pre-tagged clip; they drive an **alteration engine** (design
Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-player-alteration-core.md).
**Slice 2 — simulator-first alteration preview ✅ Done.** Merged to `main`
(session 0009). Wires the alteration engine into the web simulator so the look is
tunable by eye before any hardware, and **reconciles** the unmerged session-0007
design with the merged session-0008 design (the Left-HUD conflict): Left is a
**runtime overlay** (authored annotation track + per-language string tables),
Right is a **discrete pre-baked** flow-stabilized variant, Dark/Light a **live**
grade — superseding 0007's baked-HUD 5×5 grid. Engine: a parameterized
`Calibration` (tuned by eye, baked into `DEFAULT_CALIBRATION`), `Restyle.variant`
(discrete) replacing the continuous blend, and `AnalyticalOverlay.level`. Sim:
`/api/alteration` + `/api/clips` over `simulator/clips.py`, an alteration-preview
UI, and one neutral clip with a real flow-stabilized Right variant from the POC.
Design:
[`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md);
plan:
[`2026-06-07-reconciled-simulator-alteration-slice.md`](./superpowers/plans/2026-06-07-reconciled-simulator-alteration-slice.md).
**Remaining slices (not started):**
- **Runtime renderer** — drive the single panoramic projector via mpv/ffmpeg;
@@ -134,7 +150,12 @@ Plan: [`2026-06-05-player-alteration-core.md`](./superpowers/plans/2026-06-05-pl
`Controls` stream).
- **White-noise generation** — runtime white/pink noise for that content position.
- **Offline v2v variant pipeline** — author the pre-baked Right restyle variants
(the only paid AI step, design §9) + the multilingual label/string tables + TTS.
via the local flow-stabilized SD pipeline (now ~free, not a paid API — see the
scales-library/right-axis design §1/§4); a real multi-strength flow-stabilized
re-bake per base clip + the multilingual label/string tables + TTS.
- **Scale-ring navigation** — the endless rotary encoder + short pre-baked AI
zoom/warp transitions between neutral "scales of nature" clips on a closed ring
(scales-library design §3); a new control + offline pipeline element.
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
variant" flag (sub-project 2 territory, design §13).
+33 -10
View File
@@ -287,11 +287,23 @@ Records point at files via `file_path`. Those files live on the player's drive,
`file_path` values consistent with wherever you mount that drive on the machine
that will eventually run the room.
## Playing with the simulator (curator's X-ray)
## Playing with the simulator (alteration preview)
The simulator is a web stand-in for the installation's control panel. It runs the
real `hef.selection` code against a synthetic catalog so you can feel whether the
dials surface fitting pieces before any hardware exists.
real `player.alteration` engine and **alters** a neutral base clip toward the knob
state in the browser, so you can tune the *look* of the filter before any hardware
exists. (The earlier selection-era "curator's X-ray" view was retired when the
piece moved from *selecting* clips to *altering* them.)
**One-time setup — populate the sample footage** (look-tuning only; not shipped
content):
python simulator/setup_sample_media.py
This copies the session-0008 POC artifacts (`~/hef-poc/out/`) into
`simulator/sample_media/forest/` — the neutral base clip and the real
flow-stabilized Right restyle — and generates placeholder intermediate Right
strengths. The `.mp4` binaries are gitignored.
**Run it (Docker):**
@@ -304,11 +316,22 @@ then open http://localhost:8000.
pip install -e ".[sim]"
make sim-local
**What you see:** the five real dials (mode + Left/Right/Dark/Light), the model
knobs (brain/mood weights, pool size, approved-only), and the X-ray — the picked
piece, the ranked candidate pool with distances, and brain/mood coordinate maps
showing where your knob point and the candidates sit.
**What you see and can do:**
By default it loads a generated fixture catalog. To point it at a real catalog,
set `HEF_SIM_CATALOG=catalog/library.jsonl` (used automatically when that file is
non-empty).
- **Content dial** — picks audio/video channel; "off" and audio-only positions go
to black walls.
- **Four experience knobs (04):**
- **Dark / Light** — a live runtime color grade (cool/dark ↔ warm/bright; equal
or zero = the raw footage).
- **Right (dreamlike)** — selects a discrete pre-baked, flow-stabilized restyle
variant and crossfades to it (strength 0 = raw base).
- **Left (analytical)** — a live overlay: labelled boxes from the clip's authored
annotation track, with more annotations appearing at higher levels. Text is
shaped live (the simulator analogue of the Pi's Pango/HarfBuzz path).
- **Calibration sliders** — adjust the grade/overlay gain curves live; once a look
is liked, bake the values into `DEFAULT_CALIBRATION` in `player/alteration.py`.
- **RenderPlan readout** — always shows the exact numbers the engine produced (the
project's honesty "X-ray," now over the alteration model).
The base clips, Right variants, Left annotation track, and string tables come from
`simulator/sample_media/manifest.json`.
File diff suppressed because it is too large Load Diff
@@ -121,6 +121,13 @@ layers. This single rule covers the whole `5×5×5×5` space:
### 4.3 Where each transform runs
> **Reconciled (2026-06-07, session 0009):** the Left HUD is a **runtime overlay**
> driven by an authored annotation track + per-language string tables (text shaped
> live); the Right axis selects a **discrete pre-baked** flow-stabilized variant
> (not a continuous blend). See
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
> §1, which supersedes the session-0007 baked-HUD / 5×5-grid proposal.
- **Runtime, on the Pi (free, continuous, full-res):** the Dark/Light color grade
and the Left analytical overlay. These are cheap (LUT/curves + luma key, and
text/graphics compositing) and can move continuously with the knob.
@@ -235,6 +242,13 @@ one-time build.
## 10. Accessibility, i18n, and the translation-cost finding
> **Reconciled (2026-06-07, session 0009):** the near-free-i18n path is kept — the
> Left HUD is a runtime overlay (authored annotation track + per-language string
> tables, shaped live via Pango/HarfBuzz on the Pi). The session-0007 baked-HUD
> reversal is **not** adopted. See
> [`2026-06-07-reconciled-simulator-alteration-slice-design.md`](./2026-06-07-reconciled-simulator-alteration-slice-design.md)
> §1.
The piece is operable **blind, in the dark, in your language**, via four redundant
channels on the control panel: **touch** (engraved symbol shape), **low-light
color** (LEDs), **braille**, and **audio** (a read-aloud button on a small *local*
@@ -0,0 +1,299 @@
# HEF — Reconciled Simulator-First Alteration Slice (Design)
**Date:** 2026-06-07
**Status:** Approved design (pre-implementation) — reconciliation approved this session (0009)
**Repo:** `human-experience-filter-art`
**Reconciles:**
[`2026-06-06-simulator-alteration-preview-design.md`](./2026-06-06-simulator-alteration-preview-design.md)
(session 0007, the *unmerged* `feature/simulator-alteration-preview` branch) **with**
[`2026-06-07-scales-library-and-right-axis-pipeline-design.md`](./2026-06-07-scales-library-and-right-axis-pipeline-design.md)
(session 0008, merged to `main`).
**Parents:**
[`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
(the alteration engine) and
[`2026-06-04-experience-simulator-design.md`](./2026-06-04-experience-simulator-design.md)
(the simulator scaffold).
**Supersedes:** the **baked-HUD / 5×5-grid** position of the 0007 design (§4, §5, §8 there).
> **Why this exists.** Sessions 0007 and 0008 left two unmerged design threads that
> disagree on one load-bearing point — how the **Left** analytical HUD is rendered —
> and, downstream of that, on the **shape of the pre-baked variant set**. 0007 baked
> the Left HUD into a 5×5 grid of authored Left×Right variant clips (pixel-precise,
> but i18n becomes expensive). 0008 (the later, merged design) kept the Left HUD a
> **runtime overlay** driven by an authored annotation track + per-language string
> tables (near-free i18n). This document picks the runtime-overlay position, follows
> its consequences through the engine and the simulator, and scopes the first
> **simulator-runnable** slice that realizes it. It then hands off to an
> implementation plan.
---
## 1. The decision (operator-approved this session)
**Left is a runtime overlay, not baked pixels.** Concretely:
- **Left axis** → a **runtime `AnalyticalOverlay`** driven by an **offline-authored
annotation track** (box positions, anchor points, and which label *keys* appear at
each Left level 04) plus a per-language **string table** (key → translated text).
Text is **shaped live** — natively by the browser in the simulator; by
**Pango + HarfBuzz + Noto** on the Pi at runtime (the 0008 §1.2 requirement). This
is the "authored box positions + runtime-shaped text" hybrid: authorial control
over *layout*, near-free *i18n*.
- **Right axis** → the **only pre-baked axis**: a small set of **flow-stabilized
restyle-strength variants** (0008 §1), selected **discretely** by the Right knob
(04), with a **crossfade** on change.
- **Dark/Light** → a **live runtime `ColorGrade`** (mood center = identity, §5 of the
parent).
**What this supersedes.** The 0007 design's 5×5 Left×Right **grid of 24 authored
clips** and its **removal of `AnalyticalOverlay`/`Restyle` from `RenderPlan`** are
dropped. The Left HUD is *not* baked; the variant set is **1-D over Right strength**,
not a 2-D grid. (Accepted loss vs. fully-baked HUD: no pixel-painted HUD *artwork*
the HUD is shaped text + drawn boxes. Accepted gain: near-free i18n + far less
authoring — N Right variants per clip, not 24.)
**What it preserves from 0007.** The valuable, non-conflicting parts: Python-canonical
engine with a thin browser renderer; a parameterized **`Calibration`** tuned by eye in
the sim; retiring the simulator's selection-era surface; placeholder-variant generation
so the mechanism is testable before real authored media exists.
### 1.1 The merged engine is already most of the way there
The slice-1 engine on `main` (`player/alteration.py`) **already** models
`RenderPlan = { grade, overlay, restyle }` with a runtime `AnalyticalOverlay` — only
0007's *unmerged doc* proposed removing it. So this reconciliation is **surgical**, not
a rewrite: keep `grade` and `overlay`; change only how the **Right** axis and the
**calibration** are modeled.
---
## 2. Engine reconciliation (`player/alteration.py`, `player/state.py`)
### 2.1 `RenderPlan` layers
| Layer | Axis | Type | Change from `main` |
|---|---|---|---|
| `grade: ColorGrade` | Dark/Light | `tone ∈ [1,1]`, 0 = identity | unchanged shape; curve now from `Calibration` |
| `overlay: AnalyticalOverlay` | Left | gains discrete `level: int` (04) + keeps `intensity` | `level` added so the renderer selects which annotations show |
| `restyle: Restyle` | Right | **discrete** `variant: int` (04); 0 = raw | replaces continuous `blend: float` |
- **`Restyle.variant`** is a discrete index selecting a **pre-baked** Right-strength
clip. `variant == 0` means the raw base (no restyle). This matches "select a
pre-baked variant and crossfade," and replaces the continuous `blend` that no longer
has a runtime meaning (restyle is pre-baked, not blended live).
- **`AnalyticalOverlay.level`** (04) is the Left knob value; the renderer uses it to
choose which annotations from the authored track are active. `intensity` (0..1)
stays for overlay opacity/strength and is derived from `level` via `Calibration`.
- **`RenderPlan.is_identity`** holds when `grade.is_identity and overlay.level == 0
and restyle.variant == 0`.
### 2.2 `Calibration` (new frozen dataclass)
A frozen `Calibration` parameterizes the knob→strength maps so they can be tuned **by
eye in the sim** and then baked into a `DEFAULT_CALIBRATION` constant:
- `mood_center` and a per-axis curve for `_mood_tone` (Dark/Light).
- the Left `level → intensity` curve.
- the Right `knob → variant` map (which knob positions select which pre-baked
strength; identity-preserving so knob 0 → variant 0).
`plan_alteration(coord, calibration: Calibration = DEFAULT_CALIBRATION) -> RenderPlan`.
**`DEFAULT_CALIBRATION` reproduces today's exact behavior** (the three current helpers:
`value/4` for Left intensity, `(lightdark)/4` for mood, `right` → `variant=right`), so
the change is behavior-preserving until the operator tunes it. The session-0006
knob→strength open decision is then settled **by eye** in the sim's calibration panel
and locked into `DEFAULT_CALIBRATION` by a unit test.
### 2.3 `player/state.py`
`state.py` already classifies a change to `plan.restyle` as a `CROSSFADE` and a grade-/
overlay-only change as `LIVE_UPDATE`. With `Restyle.variant` discrete, the same
classifier works unchanged: a new Right variant → `CROSSFADE`; a grade or Left-overlay
change → `LIVE_UPDATE`; clip swap or video on/off → crossfade / fade-to-black as today.
The only edit is to the `_classify` comparison if the field name changes
(`restyle.blend` → `restyle.variant`).
These are pure framework-code changes; no deployment-shape decision enters the engine.
---
## 3. Simulator (`simulator/`)
### 3.1 Retire the selection era
The simulator currently visualizes the **old selection model** (the "curator's X-ray").
Remove:
- `POST /api/select`, `GET /api/catalog/meta`;
- the X-ray static UI (`simulator/static/*` rewritten, see §3.4);
- `simulator/fixtures.py` (the 625-record synthetic *catalog*) — replaced by
`simulator/clips.py`.
`hef.selection` (the library, incl. `Coordinate`, `ranked_candidates`) is **untouched**;
only the simulator's selection *surface* retires. `Coordinate` is still used by the
alteration engine.
### 3.2 `simulator/clips.py` (replaces `fixtures.py`)
Reads a base-clip + variant manifest. Per base clip:
```
{
"id": "forest",
"title": "...",
"base_file": "forest/base.mp4",
"license": "...", "source": "...",
"right_variants": { "1": {"file": "forest/right1.mp4", "model": "..."},
"...": {...}, "4": {"file": "forest/right4.mp4"} },
"annotations": [ {"key": "detected.conifer", "box": [x,y,w,h], "min_level": 1}, ... ],
"strings": { "en": { "detected.conifer": "conifer", ... } }
}
```
- `right_variants` is keyed by Right strength `1..4` (strength `0` = raw `base_file`).
Missing strengths fall back to the raw base and are flagged "raw / unauthored" in the
readout.
- `annotations` is the **authored annotation track** (box + label key + the minimum
Left level at which it appears). `strings` is the per-language table (English only
this slice).
### 3.3 Endpoints
- **`POST /api/alteration`** — body `{ controls, calibration? }` → `RenderPlan`
(serialized) **plus** the `ContentResolution` from `resolve_content` (so the content
dial's video-on/off is honored — "Off" → black). Calls the real
`plan_alteration(coord, calibration)`.
- **`GET /api/clips`** — the base-clip list + the active clip's manifest (variants +
annotation track + string table).
- Video served as static assets from a sample-media directory.
- **Removed:** `POST /api/select`, `GET /api/catalog/meta`.
### 3.4 Browser preview (`simulator/static/`)
Thin renderer; all alteration math stays in Python.
- **Right** — `<video>` showing the selected Right-variant file; **opacity crossfade**
to the new file when `restyle.variant` changes; variant 0 → the raw base.
- **Dark/Light** — **live color grade** over the video via CSS/canvas filters: Light →
warm + lifted toward white; Dark → cool + crushed toward black; tone 0 → raw.
- **Left** — overlay drawn **live** from the annotation track: for each annotation with
`min_level ≤ overlay.level`, draw its box and the **shaped** string (browser-native
shaping) for the active language; opacity from `overlay.intensity`. No baked HUD.
- **Content dial** — drives `<video>` visibility; "Off"/audio-only → black walls.
- **Calibration panel** — sliders for the `Calibration` params; changing them
re-requests the plan and the footage responds live.
- **RenderPlan readout** — always shows the exact engine numbers (grade tone, overlay
level/intensity, restyle variant) — the project's honesty "X-ray," now over the
alteration model.
The Left overlay being browser-drawn is the **simulator analogue** of the Pi's
Pango/HarfBuzz path: both take the *same* annotation track + string table; the browser
shapes natively, the Pi shapes with HarfBuzz. The manifest is the shared contract.
---
## 4. The sample clip + a real Right variant (this slice)
To make the look **evaluable now**, wire one real clip end-to-end using the session-0008
POC artifacts (`~/hef-poc/out/`, outside the repo):
- **Base clip** = the POC's `neutral.mp4` (an 8 s nature loop) → copied to the
sample-media dir as the one base clip.
- **Right variant (top strength)** = the POC's **`right_flow.mp4`** — the real,
operator-approved **flow-stabilized** restyle — wired as Right strength 4.
- **Intermediate Right strengths (13)** = generated by a small **ffmpeg placeholder
generator** (e.g. graded/blended stand-ins) so the crossfade mechanism is exercised
across the full knob range; the real high-end look is present for tuning.
- **Left annotation track** = a minimal authored track (a few boxes + English label
keys) for that clip, so Left renders as real shaped text over drawn boxes.
> **Licensing note.** The simulator sample footage exists **only to tune the look**; it
> is not shipped installation content. Strict-PD scale-library sourcing (NASA/NOAA/NPS
> per 0008 §2.1) remains a later slice and is unaffected by this choice.
A real multi-strength SD re-bake (4 genuine flow-stabilized strengths) is **out of scope
this slice** — one real strength + placeholders is enough to settle the mechanism and the
calibration. The re-bake is a later offline-pipeline task.
---
## 5. Testing
- **`player/` unit tests** (`tests/test_player_alteration.py`, `test_player_state.py`):
- `DEFAULT_CALIBRATION` reproduces the current helpers exactly (behavior-preserving).
- `Restyle.variant` is discrete; knob 0 → variant 0 (identity); a non-default
`Calibration` changes the plan as specified.
- `AnalyticalOverlay.level` maps from the Left knob; `intensity` derives from it.
- `state.py`: a Right-variant change → `CROSSFADE`; a grade-/overlay-only change →
`LIVE_UPDATE`; video on/off unchanged.
- **Simulator API tests** (rewrite `tests/test_simulator_api.py`):
- `POST /api/alteration` returns the engine's plan (+ `ContentResolution`) for given
controls/calibration.
- `GET /api/clips` returns the manifest.
- the removed endpoints (`/api/select`, `/api/catalog/meta`) are gone (404).
- `test_fixtures.py` retired/rewritten for `clips.py`.
- **No browser/E2E automation** this slice — manual visual tuning is the point; the JS
stays thin and the tested logic stays in Python.
---
## 6. What ships
- `player/alteration.py` — `Calibration` + `DEFAULT_CALIBRATION`, discrete
`Restyle.variant`, `AnalyticalOverlay.level`; `plan_alteration(coord, calibration)`.
- `player/state.py` — crossfade-trigger field rename only.
- `simulator/clips.py` (variant + annotation manifest) replacing `fixtures.py`.
- `simulator/app.py` — `/api/alteration` + `/api/clips`; selection endpoints removed.
- `simulator/static/` — rewritten as the alteration preview (variant `<video>` +
crossfade, live grade, live Left overlay, content dial, calibration panel,
RenderPlan readout).
- Sample base clip + one real Right variant + placeholder generator + minimal Left
annotation track / English strings.
- Tests above; `docs/USER_GUIDE.md` "Playing with the simulator" rewritten; the parent
design §4.3/§10 pointer updated to cite this reconciliation; `docs/ROADMAP.md` §3
updated.
---
## 7. Out of scope (YAGNI) — later slices
- Serial input / the 3⇄4 framing contract; the Pi/mpv/GPU runtime renderer (deferred by
`simulator-first-before-hardware`).
- Audio playback (music / white-noise / audio-track).
- The **endless rotary encoder** + **AI zoom/warp transitions** between scales
(0008 §3) — a separate control + offline pipeline element.
- A real multi-strength SD flow-stabilized re-bake; strict-PD scale-library sourcing
(0008 §2.1).
- Catalog-model changes (audio source / neutral-vs-variant flag); retiring
`hef.selection.ranked_candidates`.
- Broad multilingual string tables (English-first; the runtime path keeps i18n cheap,
but authoring other languages is later).
---
## 8. Open questions (for the plan, not blockers)
- **Calibration curve shape** — **RESOLVED (session 0010, by eye).**
`DEFAULT_CALIBRATION` is **locked** to unity gains + a linear variant map
(`mood_gain=1.0`, `overlay_gain=1.0`, `right_variant_map=(0,1,2,3,4)`), as a
deliberate choice: with the dark-grade fix below, full knob is peaceful on
every axis (POC + sim), so full tilt = full look and the 5 notches map 1:1 to
the 5 discrete Right bakes. This also closes the **session-0006** convention
question — knobs run 0=off..4=max, equal Dark/Light = identity; no
"centered at 2 = no push." Guarded by `test_default_calibration_is_locked`.
- **Grade vs. Left overlay interaction** — **RESOLVED: overlay above the grade.**
The simulator composites the Left HUD (SVG) above the mood grade and the cool
tint, so the HUD stays legible regardless of mood. The Pi renderer should do
the same.
- **Dark-pole grade look** — **FIXED (session 0010).** The first by-eye pass found
the sim's dark grade used a full-frame `hue-rotate(-200deg)`, which turned the
rock orange and trees purple — the disorienting look rejected in 0008, not the
peaceful POC `dark_frame`. Replaced with darken + slight desaturate on the video
filter plus a `multiply`-blended deep-blue wash (`#tint`) that lifts shadows
toward blue while preserving natural greens. The Pi renderer (later slice) will
do proper grading; this matches the approved POC dark look closely enough to tune
by eye in the sim.
- **Crossfade timing in the browser** — a simple opacity crossfade is enough for tuning;
the real timing engine is a later slice.
- **Placeholder fidelity** — how close the strength-13 placeholders should look to real
restyle; cheap stand-ins are fine for mechanism + calibration.
@@ -0,0 +1,236 @@
# HEF — Scales-of-Nature Library + Stabilized Right-Axis Pipeline (Design Revision)
**Date:** 2026-06-07
**Status:** Approved design (pre-implementation) — operator-approved this session (0008)
**Repo:** `human-experience-filter-art`
**Refines:** [`2026-06-05-machine-altered-perception-design.md`](./2026-06-05-machine-altered-perception-design.md)
— specifically its Right-axis pipeline (§4.1/§4.3), content sourcing/model (§6/§8),
and economics (§9), and it **adds a scale-navigation control + zoom transitions**
to the §2 selector / §11 control panel. The thesis (§1), coordinate model (§3),
Dark/Light/Left treatment, and accessibility (§10) are **preserved**.
**Grounded in:** a local proof-of-concept run this session on the operator's Mac
mini (M4 Pro, 64 GB, MPS) — all numbers below are measured, not estimated.
> **Why this revision exists.** The 2026-06-05 design specified the Right axis as
> "generative video-to-video restyle, pre-baked offline" and assumed that meant a
> **paid cloud API** (§9 priced Kling/Sora/Veo/Runway at $0.050.10/s). A POC this
> session established two things that change the design: (1) the Right restyle runs
> **entirely locally and offline** on the operator's existing hardware, for the
> cost of electricity; and (2) naïve per-frame restyle **boils/flickers** in a way
> the operator found disorienting — disqualifying for a piece meant to be peaceful
> — and the fix is **optical-flow keyframe propagation**. Separately, the operator
> chose how the "scales of nature" idea enters the piece: as the curatorial theme
> of a **small neutral base library**, not a single fixed journey.
---
## 1. The Right axis is a local, flow-stabilized restyle (refines §4.1, §4.3)
The §4.1 mapping is unchanged in spirit — **Right = dissolve realism toward
painterly/dreamlike via generative video-to-video** — but the *implementation* is
now pinned:
- **Engine:** Stable Diffusion **img2img** (POC used `stabilityai/sd-turbo`) run on
**Apple MPS**, locally, offline, at authoring time. No cloud API.
- **Temporal coherence is a hard requirement, not a nicety.** Per-frame img2img
independently re-imagines each frame, producing a shimmering "boil" that reads as
disorienting — the **opposite** of the piece's peaceful intent. This was caught
in the POC and is now a named design constraint: *the Right substrate must be
temporally coherent.*
- **Stabilization: optical-flow keyframe propagation.** Fully stylize **keyframes**
at a fixed interval; for in-between frames, **warp the previous stylized frame
forward by optical flow** (so motion is continuous) and apply only a *light*
diffusion refine. The flow warp removes the boil; periodic keyframes bound drift.
(This is the EbSynth principle. Genuine `ebsynth`/`ezsynth` are NVIDIA/Windows-
leaning and don't install cleanly on Apple Silicon, so the POC implemented the
same idea directly with OpenCV Farneback flow + the existing diffusers pipeline.)
This stays consistent with §4.3's crucial invariant: **the Left analytical labels
remain a runtime overlay, never baked into the restyled pixels** — the POC's Left
HUD is composited deterministically on top, preserving the near-free i18n of §10.
### 1.1 Where each transform runs (updated §4.3 table)
| Pole | Operation | Where it runs | Measured cost (8 s, 1080p clip) |
|---|---|---|---|
| **Dark** | color grade | runtime, live on the Pi | ~2.4 s offline; live at runtime |
| **Light** | color grade | runtime, live on the Pi | ~2.5 s offline; live at runtime |
| **Left** | analytical overlay (HUD) | runtime, live on the Pi | ~2.2 s offline; live at runtime |
| **Right** | local generative restyle + flow propagation | **pre-baked offline, locally** | ~2.7 min/restyle-strength |
The three deterministic axes are confirmed cheap enough to run **live**; only the
Right restyle is pre-baked. A peaceful **deterministic** alternative for Right (soft
edge-preserving smoothing + bloom, zero flicker by construction, ~5 s/clip) was
prototyped and set aside — the operator preferred the true generative repaint once
the flow stabilization made it calm. It remains a documented fallback.
### 1.2 Runtime label rendering & i18n (sharpens §10)
The Left analytical labels are drawn **live by the Pi as a 2D graphics overlay**
**not** baked into video, and specifically **not** a pre-rendered overlay *video*. A
baked overlay would have to exist per language × per Left level × per scale, which
re-introduces the "× number of languages" cost §10 exists to avoid. So the runtime
path is:
- **Architecture.** Per base clip, an offline-authored **annotation track** (box
positions, anchor points, which annotations appear at each Left level) referencing
language-agnostic label **keys** (e.g. `detected.conifer`). Per language, a cheap
**string table** (key → translated text) + font + TTS voice. At runtime the Pi
reads the Left knob, selects the active annotations, **shapes** the current
language's strings, and composites over the altered video — updating only on change
(knob move, language switch, timeline cue), not every frame.
- **Correctness needs a real shaping stack.** Rendering *every* language correctly
(Arabic joining, Indic conjuncts, CJK, RTL) requires **Pango + HarfBuzz + Noto
fonts**, not naïve text drawing. (The POC's HUD used ffmpeg `drawtext`/Menlo —
Latin-only; it would mis-render complex scripts and is **not** the runtime path.)
This shaping stack is the load-bearing requirement behind §10's "label things
correctly."
- **Feasibility.** This is OSD/subtitle-class compositing; a Pi 5 (VideoCore VII,
GLES/Vulkan, hardware decode) handles it. Headroom at the *panoramic* resolution is
the one unmeasured variable — see §6.
---
## 2. Content = a small NEUTRAL "scales of nature" library (refines §6, §8)
The operator's "cosmic zoom" concept (space → continents → birds → ocean → abyss →
microscopic → galaxy) enters the piece **as a curatorial theme, not a fixed film.**
- **Structure:** a **small library (~46 to start) of calm, neutral base clips**,
each drawn from a *different scale of nature* — e.g. an orbital Earth, a forest, a
coral reef, the deep-sea abyss, the microscopic, the cosmos. The machine alters
whichever clip is playing, exactly as for any neutral base.
- **Why this and not a single stitched journey.** A literal galaxy→cell "how small
we are" journey carries its **own** emotion (awe, cosmic insignificance) *before
the machine acts*, which contradicts the §1 neutral-base thesis, and as a fixed
film it becomes "a journey you watch" rather than "a reality you bend." Keeping the
scales as the *theme of a neutral library* preserves the awe-of-scale richness and
the piece's coherence **while keeping the base neutral and the experience
interactive.** (Rejected alternatives: single stitched base; cosmic-zoom as
intro/reset; rethinking the thesis.)
- **Cost is not the constraint.** Per §4 below, ~5 base clips is ~1 hour of overnight
local pre-bake — so this is an *artistic* choice, made on artistic grounds.
- **Mechanism unchanged:** this slots into the existing §6 content model and the
sub-project-2 ingest/tagging/review tooling; "scales of nature" is simply the
selection principle for which neutral clips to source.
### 2.1 Strict-PD sourcing map (refines §8)
License stance is unchanged (prefer Public Domain / CC0; record license + source per
clip). The "scales" theme maps onto genuinely public-domain pools cleanly at the
*ends* and is softer in the terrestrial *middle*:
| Scale | Best strict-PD source | Status |
|---|---|---|
| Cosmos / galaxy / deep space | NASA, Hubble, JWST | 🟢 abundant, true PD |
| Earth from orbit / continents | NASA / ISS | 🟢 true PD |
| Ocean & **deep sea / abyss** (global) | **NOAA Ocean Exploration** | 🟢 true PD, worldwide |
| Microscopic / single-celled | NIH / NSF | 🟢 thinner but PD |
| US land / wildlife | NPS, USGS, USFWS | 🟢 true PD (US locations only) |
| **Non-US terrestrial, high-flying birds** | — | 🟡 mostly CC-BY; the PD soft spot |
US-government works are public domain by statute (17 U.S.C. §105); the installation
is US-based, so this is the cleanest possible legal footing. **Caveat the design
must respect:** "free stock" sites (Pexels, Pixabay, Mitch Martinez's free 4K, etc.)
are *royalty-free but NOT public domain* — they restrict redistribution and retain
copyright. The ingest tool's "no explicit license → assume PD, verify before use"
flag exists precisely for this trap and must not be trusted blindly.
---
## 3. Scale navigation & zoom transitions (new element; refines §2 selector, §11)
The scales-of-nature library is navigated as a **closed loop (a ring), not a line.**
A dedicated **scale ("zoom") control** lets the viewer journey through scales;
advancing it triggers a short **AI zoom/warp transition** to the next scale,
pre-baked offline. Diving past the smallest (single-celled) **wraps around** to the
largest (cosmos) — the infinite-zoom payoff that unifies micro and macro and makes
the ring continuous.
- **The control is an *endless* rotary encoder — infinitely turnable, no end stops.**
The form embodies the concept: a ring of scales has no beginning or end, so neither
does the knob. Keep turning one way and you zoom inward forever (…reef →
microscopic → **cosmos** → continents → …); turn back to zoom out. This sets it
apart from the four **experience knobs**, which are *absolute* 04 pots: the zoom
control reports **relative** rotation (encoder detents), and the player/firmware
advances or retreats one ring-step per increment. It is distinct too from the §6
content dial (audio/video channel) — it chooses *where in the ring* you are, while
the knobs still bend whichever scale is present.
- **Transitions:** between each adjacent pair of scale clips, a short (~few-second)
generative morph — **first-last-frame-conditioned image-to-video** (Wan/LTX-class)
or SD "infinite-zoom" outpainting for the literal zoom-through. Pre-baked offline,
local; one clip per ring edge (N scales → N transitions, including the micro→cosmos
closer). A fast spin may cross several scales — transitions chain, or past a speed
threshold a faster blended pass is used.
- **Thesis-safe:** dwells on a scale are the neutral, knob-altered interactive cores;
transitions are fixed connective moments (un-altered, or at most carrying the
current mood grade). The awe lives in the *movement between* scales, not the base.
- **Heavier than the restyle, still bounded:** generative video synthesis costs more
per second than img2img, but transitions are few and short — a handful of ~35 s
morphs is an overnight local batch.
---
## 4. Economics update — local authoring ≈ free (refines §9)
§9 priced the Right axis at **~$3003k** of cloud generative-v2v API. The POC
collapses that: the restyle runs on **hardware the operator already owns**, offline,
so the marginal cost is electricity.
- **Per base clip:** ~4 Right restyle strengths × ~2.7 min ≈ **~11 min of local AI
pre-bake**, plus seconds for the deterministic Dark/Light/Left grid.
- **Whole small library (~5 clips):** **~1 hour** of overnight batch rendering.
- **Scale transitions:** ~N short generative-video morphs (one per ring edge), a few
seconds each — a separate, heavier offline batch (video synthesis > img2img), but
still overnight-local.
- **Cloud API: no longer required** for the build. (It remains an option if a
higher-quality video model than a local one is wanted for a final pass.)
This also tightens the piece's "all-local" ethos: not just *runtime* is local
(§1 of the prior design) — now *authoring* is too.
---
## 5. POC evidence (this session)
A throwaway spike (outside the repo, `~/hef-poc/`) validated the full engine on one
real nature clip (a 4K Yosemite waterfall, trimmed to 8 s @ 1080p):
- **All four axes rendered** and read as distinct: Dark (cold/somber), Light
(warm/serene), Left (analytical HUD overlay), Right (painterly).
- **Right per-frame:** ~3.4 min/8 s clip, **flickers badly** (disqualifying).
- **Right flow-propagated:** ~2.7 min/8 s clip, **calm** (operator-approved).
- **Deterministic axes:** ~2.4 s each, ~3× faster than real-time → confirmed
runtime-capable.
- **Stack:** `imageio-ffmpeg`, `diffusers` + `sd-turbo` on MPS, OpenCV Farneback
flow; Python 3.13; 64 GB unified memory comfortably ran models that OOM consumer
GPUs.
---
## 6. Open questions (for the plan, not blockers)
- **Flow quality at scale:** the OpenCV flow propagation was validated on one short
clip; longer clips / faster motion may need shorter keyframe intervals, bidirectional
blending, or a stronger flow model (RAFT).
- **Painterly strength:** the POC kept the restyle gentle; the dreamlike *range* and
the per-axis restyle-strength count (the §4.3 "small set of variants") are
unfixed.
- **Base-clip sourcing:** select and ingest the actual ~46 strictly-PD neutral
scale clips (NASA/NOAA/NPS) via sub-project 2.
- **Model choice:** `sd-turbo` was the POC's speed pick; a higher-quality local
model (or a final cloud pass) may be worth a comparison for the shipped variants.
- **Scale transitions:** generation method (first-last-frame i2v vs. infinite-zoom
outpainting) and local model; per-transition length; whether transitions carry the
current mood grade; the ring ordering of the scales; and behavior on fast or
continuous spins of the endless encoder (chain transitions vs. blended skip).
- **Pi compositing headroom:** confirm the Pi 5 can decode the altered video **and**
render the live Pango/HarfBuzz label overlay at the *actual* panoramic projector
resolution (ultra-wide / high-res) — low risk but unmeasured (see §1.2).
## 7. Out of scope (YAGNI)
- A single continuous **one-take** zoom through *all* scales (we use discrete neutral
clips joined by short AI transitions on a navigable ring — not one unbroken shot).
- Audio-axis alteration (already deferred by the prior §6).
- Cloud rendering pipeline (local supersedes it for the base build).
+85 -36
View File
@@ -1,22 +1,18 @@
"""The alteration engine: a knob vector -> a layered RenderPlan (design §4, §5).
Replaces the 2026-06-04 nearest-match *selection* with a *transformation* of a
neutral base clip. Given the four experience knobs, it produces three layers
that compose per §4.2:
Reconciled slice (2026-06-07): the Right axis is a DISCRETE selection of a
pre-baked, flow-stabilized restyle variant (not a continuous blend), the Left
axis carries its knob LEVEL so a runtime annotation track can pick which labels
show, and a frozen `Calibration` parameterizes the knob->strength curves so they
can be tuned by eye in the simulator and baked into DEFAULT_CALIBRATION.
- Substrate transforms (alter pixels, blend with each other):
* Restyle -- the Right axis: a pre-baked generative v2v dreamlike restyle.
* ColorGrade -- the mood axis (Dark/Light): a deterministic color grade.
- Overlay (composited on top of the substrate):
* AnalyticalOverlay -- the Left axis: HUD/labels/measurement.
Left and Right are NOT opposites; they live on different layers and stack
(§4.2). Dark and Light are the two poles of one mood grade whose center is the
identity (§5).
Calibration note: the knob->strength curves below are the single source of
truth for how a 0..4 position maps to a transform strength. See the session
0006 transcript Deferred decisions for the §3-vs-§4.2/§5 reconciliation.
Layers compose per §4.2:
- Substrate: ColorGrade (Dark/Light mood, center = identity §5) + a pre-baked
Right restyle variant.
- Overlay: AnalyticalOverlay (Left), composited on top at runtime.
Left and Right stack (different layers); Dark/Light are the two poles of one
mood grade. See docs/superpowers/specs/2026-06-07-reconciled-simulator-
alteration-slice-design.md.
"""
from __future__ import annotations
@@ -28,6 +24,45 @@ from hef.selection import Coordinate
KNOB_MAX = 4 # knob full-scale (0..4)
def _clamp(x: float, lo: float, hi: float) -> float:
return max(lo, min(hi, x))
@dataclass(frozen=True)
class Calibration:
"""Tunable knob->strength curves (settled by eye in the sim, then baked).
- mood_gain: scales the signed Dark/Light tone (result clamped to [-1, 1]).
- overlay_gain: scales the Left overlay intensity (clamped to [0, 1]).
- right_variant_map: knob value (0..4) -> pre-baked Right variant index.
"""
mood_gain: float = 1.0
overlay_gain: float = 1.0
right_variant_map: tuple = (0, 1, 2, 3, 4)
# The LOCKED calibration (session 0010, 2026-06-07) — settled by eye in the
# simulator, closing the open session-0006 knob->strength decision. Convention:
# every experience knob runs 0 = off .. 4 = max, and equal Dark/Light = identity
# (raw footage); there is no "centered at 2 = no push" coordinate.
# - mood_gain = 1.0: full Dark/Light knob reaches the full mood grade. By-eye
# evidence (POC renders + sim, after the dark-grade fix this session) shows
# full tilt is peaceful on every axis, so no softening is warranted.
# - overlay_gain = 1.0: full Left = opacity 1.0; the HUD is legible, not
# overwhelming, sitting above the grade.
# - right_variant_map linear: the 5 knob notches map 1:1 onto the 5 discrete
# pre-baked Right strengths (0 = raw base).
# These are unity/linear by deliberate choice, not as placeholders. The curve
# stays parameterized so a future re-bake or a different feel is one edit away;
# test_default_calibration_is_locked guards the values from drifting silently.
DEFAULT_CALIBRATION = Calibration(
mood_gain=1.0,
overlay_gain=1.0,
right_variant_map=(0, 1, 2, 3, 4),
)
@dataclass(frozen=True)
class ColorGrade:
"""Mood-axis grade (§5). `tone` is signed: >0 warm yellow->white (light),
@@ -42,18 +77,20 @@ class ColorGrade:
@dataclass(frozen=True)
class AnalyticalOverlay:
"""Left axis (§4.1): analytical HUD/annotation/labels, composited on top.
`intensity` 0..1 (0 = no overlay)."""
"""Left axis (§4.1): analytical HUD/annotation, composited on top at runtime.
`level` is the Left knob (0..4); a runtime annotation track uses it to pick
which labels appear. `intensity` 0..1 is the overlay opacity/strength."""
level: int
intensity: float
@dataclass(frozen=True)
class Restyle:
"""Right axis (§4.1): pre-baked generative v2v dreamlike substrate.
`blend` 0..1 (0 = raw substrate, no restyle)."""
"""Right axis (§4.1): selects a pre-baked, flow-stabilized restyle variant.
`variant` is a discrete index (0 = raw base, no restyle)."""
blend: float
variant: int
@dataclass(frozen=True)
@@ -69,30 +106,42 @@ class RenderPlan:
"""True when the plan leaves the neutral base un-altered."""
return (
self.grade.is_identity
and self.overlay.intensity == 0.0
and self.restyle.blend == 0.0
and self.overlay.level == 0
and self.restyle.variant == 0
)
def _overlay_intensity(left: int) -> float:
"""Left knob -> analytical-overlay intensity (0..1)."""
return left / KNOB_MAX
def _overlay_intensity(left: int, cal: Calibration) -> float:
return _clamp(cal.overlay_gain * left / KNOB_MAX, 0.0, 1.0)
def _restyle_blend(right: int) -> float:
"""Right knob -> v2v restyle blend (0..1)."""
return right / KNOB_MAX
def _right_variant(right: int, cal: Calibration) -> int:
return cal.right_variant_map[right]
def _mood_tone(dark: int, light: int) -> float:
"""(dark, light) -> signed mood grade in [-1, 1]; equal -> 0 identity (§5)."""
return (light - dark) / KNOB_MAX
def _mood_tone(dark: int, light: int, cal: Calibration) -> float:
return _clamp(cal.mood_gain * (light - dark) / KNOB_MAX, -1.0, 1.0)
def plan_alteration(coord: Coordinate) -> RenderPlan:
def plan_alteration(
coord: Coordinate, calibration: Calibration = DEFAULT_CALIBRATION
) -> RenderPlan:
"""Map a knob vector to its layered RenderPlan (design §4)."""
return RenderPlan(
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light)),
overlay=AnalyticalOverlay(intensity=_overlay_intensity(coord.left)),
restyle=Restyle(blend=_restyle_blend(coord.right)),
grade=ColorGrade(tone=_mood_tone(coord.dark, coord.light, calibration)),
overlay=AnalyticalOverlay(
level=coord.left,
intensity=_overlay_intensity(coord.left, calibration),
),
restyle=Restyle(variant=_right_variant(coord.right, calibration)),
)
def render_plan_to_dict(plan: RenderPlan) -> dict:
"""JSON-serializable form for the simulator API."""
return {
"grade": {"tone": plan.grade.tone},
"overlay": {"level": plan.overlay.level, "intensity": plan.overlay.intensity},
"restyle": {"variant": plan.restyle.variant},
"is_identity": plan.is_identity,
}
+2 -2
View File
@@ -5,8 +5,8 @@ desired Playback (which neutral base clip, how it is altered, what audio plays,
at what levels) and returns the Transition from the previous Playback. The
transition KIND encodes design §4.3: the Dark/Light grade and the Left overlay
are continuous runtime ops (LIVE_UPDATE), whereas swapping the clip or the
pre-baked Right v2v variant needs a CROSSFADE, and toggling video on/off
fades to/from black.
pre-baked Right restyle variant (a discrete index) needs a CROSSFADE, and
toggling video on/off fades to/from black.
Knobs no longer *select* a clip (the base library is neutral by construction);
they *transform* it. Which neutral base to show is an injected policy
@@ -0,0 +1,101 @@
# Session 0007.0 — Transcript
> App: human-experience-filter-art
> Date: 2026-06-06 (PST) · 14:12 → 17:57
> Type: coding
> Goal: continue the roadmap; reframed by the operator to **simulator-first** —
> bring the slice-1 alteration engine into the simulator so the experience can be
> tuned before any hardware.
> Outcome: **design spec written, committed, and pushed** on branch
> `feature/simulator-alteration-preview`. No code changes. Implementation deferred
> to the next session (design + implementation to ship as one PR).
## Plan
Bring the alteration engine into the simulator (scope: tune the look), per
`docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md`:
- Python-canonical engine, thin browser renderer.
- Dark/Light = live runtime color grade; Left/Right = 5×5 grid of pre-baked
authored variant clips selected by coordinate (24 videos/base clip).
- Revises design §4.3/§10: HUD baked into variant videos (authorial precision)
rather than runtime overlay — trades away near-free i18n.
- Engine reconciliation: parameterize `Calibration`, add `VariantRef`, slim
`RenderPlan` (drop runtime `AnalyticalOverlay`/`Restyle`); `state.py` crossfade
on variant change.
- Simulator: retire selection X-ray + `/api/select`; add `/api/alteration` +
`/api/clips` + variant manifest; placeholder-variant generator.
## Pre-session state
- `main` at `5290785`, clean and pushed. Sub-projects 1 & 2 done; sub-project 3
(Player Runtime) in progress — slice 1 (pure-logic `player/` core) merged via
PR #5 (session 0006).
- A **stale nested worktree** (`.worktrees/feature-BuildSimulator11`, branch
`feature/player-alteration-core`, already merged) was present and breaking the
session resolver ("ambiguous app.json"), exactly as the session-0006 memory
gotcha predicted.
## Turn-by-turn arc
1. **Session gate.** Opened with "What's next?" → classified as a coding session;
ran `wgl-session-coding-init`.
2. **Baseline cleanup.** Claim dry-run failed on the resolver ambiguity. Verified
the stale worktree's branch was fully merged + clean, removed the worktree and
branch, and deleted the empty `.worktrees/` dir. Tree back to clean `main`.
3. **Claimed session 0007** (no other sessions in flight).
4. **Operator steer:** "only design things that will run in the simulator; get
everything working the way we like in the simulator before hardware." Saved as
a `feedback` memory ([[simulator-first-before-hardware]]).
5. **Discovered** the simulator (`simulator/`, built session 0004) is fully
decoupled from `player/` — it exercises the OLD *selection* model and has no
alteration surface. That gap became the work.
6. **Brainstorming** (with the visual companion). Decisions, one at a time:
scope = tune the alteration look; medium = short looping video; Right restyle =
labeled stand-in shader (swappable for real variants); calibration =
live-adjustable curves; architecture = **A, Python-canonical**; integration =
new view, **retire the selection X-ray**.
7. **Interactive look mockup** (browser): a real nature image with the 4 knobs as
live sliders. Operator confirmed Dark/Light read well, but said for Left/Right
they'll **author videos with the overlay baked in** for precise HUD control.
8. **Surfaced the tradeoff:** baking the HUD reverses design §4.3/§10's near-free
i18n. Operator chose **precision wins**. Confirmed the variant grid as **full
5×5 = 24 authored clips per base** (shown as a grid mockup).
9. **Revised model** presented and approved: base clip + variant manifest;
`RenderPlan = {grade, variant}`; runtime overlay/restyle layers retired;
placeholder-variant bootstrapping via ffmpeg.
10. **Wrote the spec**, self-reviewed, committed on a feature branch.
11. Operator: "do [implementation] next session. finalize this one." →
`wgl-session-finalize`.
## Cut state (end of session)
| Repo | Branch | Commit | State |
|---|---|---|---|
| human-experience-filter-art | `feature/simulator-alteration-preview` | `3ef21fb` | pushed to origin, **not merged** |
- `docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md`
new, committed.
- `main` unchanged at `5290785`.
- Working tree clean. Visual-companion server stopped; `.superpowers/` gitignored.
- No tests run (no code changed).
## What lands on the operator's plate
- **Author the 24 variant videos** per base clip (the Left×Right grid), with HUD
baked in. The simulator will consume them via the manifest; placeholder variants
cover the gap until then.
- **Deferred decisions** (also surfaced in chat):
- *knob→strength calibration* — unresolved since session 0006; will be settled
by eye in the simulator's calibration panel next session, then baked into
`DEFAULT_CALIBRATION`.
- *grade tints baked HUD* — accepted under "precision wins"; revisit only if it
reads badly once real authored clips exist.
## Prompt the operator can paste into the next session
```
/goal Implement the simulator alteration preview on branch feature/simulator-alteration-preview, per docs/superpowers/specs/2026-06-06-simulator-alteration-preview-design.md — begin with the writing-plans skill, then build: parameterize Calibration + add VariantRef and slim RenderPlan in player/, add /api/alteration + /api/clips + simulator/clips.py variant manifest, rewrite the simulator UI as the player preview, retire the selection X-ray + /api/select, add the placeholder-variant generator, and update tests + USER_GUIDE + ROADMAP §3 + the parent design §4.3/§10 pointer.
```
(Or resume with `/goal next` — the `Next /goal:` field is stored in memory
`sub-project-3-player-progress.md`.)
@@ -0,0 +1,102 @@
# Session 0008.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-07T19-42 (PST)
> End: 2026-06-07T22-37 (PST)
> Type: spec
> Status: **FINALIZED**
## Launch prompt
Session opened as `wgl-session-none` (research errand: "Let's find a great public
domain nature video for this project"), then upgraded mid-session to a **spec**
session once the work became tracked. Operator chose "Spec session, comprehensive":
fold the session's POC findings into the design AND resolve the open cosmic-zoom
structure fork.
## Plan
Develop a design revision to the machine-altered-perception design, grounded in a
local POC run this session: (1) Right-axis alteration pipeline; (2) content
structure (cosmic-zoom fork); (3) strictly-PD sourcing.
## Pre-state
- On `feature/simulator-alteration-preview` (session 0007's unmerged design), clean
tree; `main` at `5290785`.
- Catalog empty; `tools/ingest/internet_archive.py` is the only wired fetcher.
- No ML stack, no ffmpeg in the venv (pure-stdlib project).
## Session arc (uncurated)
1. **PD nature-video research.** Surveyed pools and verified licenses against the
actual metadata, not marketing. Key finding: **Pexels / Pixabay / Mitch Martinez
"free 4K" are royalty-free but NOT public domain** (restrict redistribution,
retain copyright) — exactly the trap the ingest tool's "no explicit license →
assume PD, verify" flag exists for. Genuinely-PD: NASA/Hubble (cosmos), **NOAA
Ocean Exploration** (deep sea, global), **NPS/USGS** (US land). Wikimedia nature
4K timelapses are mostly CC-BY, not PD.
2. **Operator pivot → "cosmic zoom".** Operator proposed a Powers-of-Ten journey
(space → continents → birds → ocean → abyss → microscopic → cosmos). Brainstormed
the thesis tension (a scale-journey carries built-in awe vs. the "neutral base"
thesis) and the cost (per-base pre-bake × altitudes). Surfaced that the *ends* of
the zoom are strict-PD-rich; the terrestrial *middle* (birds, non-US land) is the
CC-BY soft spot.
3. **"Can this run locally?" → POC.** Operator's machine: **Mac mini M4 Pro, 64 GB,
16-core GPU.** Built a throwaway POC in `~/hef-poc/` (outside the repo).
- *Wrong turns:* `brew install ffmpeg` and `pip install` were **denied by the
harness** (it blocks package installs) — switched to venv-local `imageio-ffmpeg`
run by the operator via `!`. zsh **doesn't word-split** unquoted `$EARGS`
mangled encoder flags; inlined them. sd-turbo img2img **crashed** ("reshape
tensor of 0 elements") because `int(steps×strength)=int(2×0.45)=0` denoise steps
→ guarded to ≥1.
- *Results on an 8s/1080p clip:* deterministic **Dark/Light/Left** grades/overlay
~2.4s each (~3× faster than realtime, runtime-capable). **Right** painterly
restyle (SD img2img on MPS) ~3.4 min/clip and **flickers badly**.
4. **Flicker is disqualifying.** Operator: "crazy and disorienting … this project is
meant to be peaceful." Prototyped two fixes: (a) a **deterministic** soft-dreamy
filter (smartblur + RGB bloom — first attempt had a magenta cast from blending on
YUV chroma planes; fixed by blending in RGB), ~5s, zero flicker by construction;
(b) **optical-flow keyframe propagation** (EbSynth principle; genuine ebsynth is
NVIDIA/Windows so implemented with OpenCV Farneback + the diffusers pipeline),
~2.7 min/clip, **calm**. Operator chose **AI + flow**.
5. **Spec session (comprehensive).** Upgraded the session via
`wgl-session-spec-init` → claimed 0008. Resolved the cosmic-zoom fork: operator
chose a **small NEUTRAL "scales of nature" library** (not a single stitched film).
Then operator added the **infinite-zoom ring** (AI zoom transitions between
scales, micro→cosmos wraparound) and the **infinitely-turnable endless encoder**.
Probed i18n feasibility → **§1.2**: Left HUD is a Pi-rendered **runtime overlay**
(Pango/HarfBuzz + Noto, annotation-track + per-language string tables), NOT a baked
per-language video.
- Wrote `docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-design.md`,
committed on `feature/scales-library-right-axis`.
## Cut state (what landed)
- **PR #6 merged to `main`** (merge `42a72fe`): the design revision doc.
- No spec-RFC submission: this app keeps specs in-repo (`app.json` `contains:["specs"]`),
so `submit-spec.sh` is N/A — surfaced, not dropped.
- POC artifacts left in `~/hef-poc/` (throwaway, outside repo): `restyle.py`,
`flow_restyle.py`, and the comparison clips/contact sheets.
- Memory updated: `sub-project-3-player-progress.md` (+ `MEMORY.md` index).
## Deferred decisions
- **Left-HUD treatment conflict (low confidence — flag).** 0008 §1.2 specifies the
Left HUD as a runtime Pango/HarfBuzz overlay (to keep i18n near-free, per parent
§10). This **reverses session 0007's decision** to bake the HUD into the 5×5
variant grid for authorial precision. I recommended the runtime-overlay path
following the operator's i18n question without flagging the 0007 reversal at the
time. `main` now carries the runtime-overlay position; the unmerged
`feature/simulator-alteration-preview` carries baked-HUD. **Reconciliation deferred
to next session** (pick one, or hybrid: authored positions + runtime-shaped text).
## Next /goal
```
/goal Reconcile the 0007 simulator-alteration-preview design (feature/simulator-alteration-preview, UNMERGED) with the merged 0008 scales-library + flow-stabilized Right-axis design — resolve the Left-HUD conflict (baked-into-5×5-grid vs runtime Pango/HarfBuzz overlay) FIRST — then writing-plans → build the simulator-first slice: deterministic Dark/Light/Left + a flow-stabilized Right restyle over ONE neutral scale clip wired into the simulator. Defer Pi renderer, endless-encoder/serial, and AI zoom-transitions per simulator-first.
```
@@ -0,0 +1,130 @@
# Session 0009.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-07T22-41 (PST)
> End: 2026-06-07T23-03 (PST)
> Type: coding
> Status: **FINALIZED**
## Launch prompt
Opened with `/goal next` — resume the goal recorded at the 0008 finalize. The
session gate resolved the stored `Next /goal:` from memory:
> Reconcile the 0007 simulator-alteration-preview design (UNMERGED) with the
> merged 0008 scales-library + flow-stabilized Right-axis design — resolve the
> Left-HUD conflict (baked-into-5×5-grid vs runtime Pango/HarfBuzz overlay) FIRST
> — then writing-plans → build the simulator-first slice: deterministic
> Dark/Light/Left + a flow-stabilized Right restyle over ONE neutral scale clip
> wired into the simulator. Defer Pi renderer, endless-encoder/serial, AI
> zoom-transitions.
## Plan
Brainstorm the reconciliation (resolve Left-HUD first) → write a unified design
doc → writing-plans → build & merge the simulator-first alteration slice.
## Pre-state
- On `main` at `c99a669`, clean tree; ran from the **main clone** (not a nested
worktree — avoiding the session-resolver ambiguity gotcha flagged in 0008).
- Two unmerged design threads: session-0007's
`feature/simulator-alteration-preview` (baked-HUD 5×5 grid, design doc only, no
code) vs. session-0008's merged scales-library/right-axis design (runtime-overlay
HUD). The Left-HUD conflict between them was the thing to resolve.
- Slice-1 engine (`player/`) on `main`: `RenderPlan {grade, overlay, restyle}` with
a continuous `Restyle.blend` and a runtime `AnalyticalOverlay`. Simulator still on
the old *selection* surface (`/api/select`, X-ray, `fixtures.py`).
- 0008 POC artifacts present in `~/hef-poc/out/` (outside repo): real `neutral.mp4`
base + `right_flow.mp4` flow-stabilized restyle.
## Session arc (uncurated)
1. **Claimed 0009** (coding) from the main clone; resolver worked cleanly. Read
both specs + memory in full.
2. **Surfaced the conflict precisely — it was 2-D, not 1-D.** Beyond the Left-HUD
rendering question, the two designs disagreed on the *shape of the pre-baked set*:
0007 made BOTH Left and Right a 5×5 grid of 24 authored clips; 0008 made only
Right pre-baked and Left a runtime overlay. Key realization: **0008 §1.2 already
IS the hybrid the operator floated** ("authored box positions + runtime-shaped
text") — authored annotation track for layout, runtime shaping for cheap i18n.
3. **Brainstorming → one operator question.** Asked the single load-bearing fork
(Left HUD: runtime-overlay+authored-track vs fully-baked vs defer). Operator chose
**runtime overlay + authored track** (the 0008 / hybrid position). Everything else
followed from approved material, so I proceeded autonomously.
4. **Found the engine was already most of the way there.** The merged slice-1 engine
already keeps a runtime `AnalyticalOverlay` — only 0007's *unmerged doc* proposed
removing it. So the reconciliation was surgical: continuous `Restyle.blend`
discrete `Restyle.variant`; add `AnalyticalOverlay.level`; add a frozen
`Calibration`. Wrote the reconciled design doc (brought the 0007 doc onto the
branch so its superseded position is preserved + links resolve), committed.
5. **writing-plans → a full TDD plan** (`docs/superpowers/plans/...`), then executed
it task-by-task with executing-plans:
- **Task 12 (engine):** `Calibration`+`DEFAULT_CALIBRATION` (behavior-preserving),
`Restyle.variant`, `AnalyticalOverlay.level`, `render_plan_to_dict`. `state.py`
needed no logic change (compares whole `Restyle`). Tests rewritten/green.
- **Task 3 (clips):** `simulator/clips.py` manifest model; retired `fixtures.py` +
`test_fixtures.py`.
- **Task 4 (API):** `/api/alteration` + `/api/clips`; removed `/api/select` +
`/api/catalog/meta`. *Wrong turn:* asserted retired POST returns 404, but the
static catch-all yields **405** for an unrouted POST — relaxed the assertion to
"404 or 405 = gone."
- **Task 5 (media):** `sample_media/manifest.json` + `setup_sample_media.py` that
copies the real POC `neutral.mp4`→base and `right_flow.mp4`→Right-strength-4 and
ffmpeg-blends placeholder strengths 13. mp4s gitignored. Ran it — 5 files
produced.
- **Task 6 (UI):** rewrote `simulator/static/` as the alteration preview (live
grade via CSS filters, Right-variant `<video>` crossfade, live SVG Left overlay
from the annotation track + string table, calibration panel, RenderPlan readout).
- **Task 7 (docs):** parent design §4.3/§10 pointers; ROADMAP slice 2 done +
deferred slices; USER_GUIDE simulator section rewritten.
- *Environment:* `python` not on PATH; used `.venv/bin/python` throughout.
6. **Verified end-to-end.** `pytest -q`**192 passed, 2 skipped**. Booted the sim
on a scratch port: `/api/clips` returns the manifest (variant 0→base), the engine
plan is correct by eye (left=3→level 3/intensity 0.75, right=4→variant 4,
dark=4→tone 1.0), and the **real** `right4.mp4` + base served as `video/mp4`.
7. **Shipped.** Pushed the branch, created **Gitea PR #7** via the keychain-token API
helper, **merged** it (autonomous posture), synced `main`, re-verified tests green
on the merged result, deleted my merged feature branch (local + remote).
## Cut state (what landed)
- **PR #7 merged to `main`** (merge `554eb50`): the reconciliation design + the built
simulator-first alteration slice (engine + simulator + media tooling + docs). 9
commits.
- `main` green: 192 passed / 2 skipped. Sim verified booting + serving real media.
- Memory updated: `sub-project-3-player-progress.md` (+ `MEMORY.md` index) — conflict
resolved, slice 2 shipped, new Next /goal.
- POC artifacts untouched in `~/hef-poc/` (still the source for `setup_sample_media.py`).
## Deferred decisions
- **Brought the 0007 design doc into `main` + intended to delete the superseded
branch.** Decided autonomously to carry `2026-06-06-simulator-alteration-preview-design.md`
forward (so the reconciliation's links resolve and the superseded position is
preserved) and to delete the now-stale `feature/simulator-alteration-preview`.
**The branch deletion (`git branch -D` / `push --delete`) was permission-denied by
the harness** — left in place. Safe to delete later (its content is in `main`); or
the operator may want to keep it. Flagging rather than forcing.
- **Merged under autonomous posture without a separate `/code-review`.** TDD + the
written plan were the quality gates; no independent review pass was run before
merge. Low risk for a slice this size, but noting it.
- **Used the POC's Yosemite clip as the sim's sample base.** It's a "forest"-scale
neutral clip, good enough to tune the look; framed in the manifest/USER_GUIDE as
**look-tuning only, not shipped content**. Strict-PD scale-library sourcing stays a
later slice — unaffected.
- **`Calibration` defaults are still behavior-preserving, not operator-tuned.** The
knob→strength calibration (open since session 0006) is now tunable by eye in the
sim but **not yet locked** — that's the next goal.
## Next /goal
```
/goal Tune the alteration look by eye in the simulator (python simulator/setup_sample_media.py then make sim-local) and LOCK the knob→strength calibration into DEFAULT_CALIBRATION in player/alteration.py (+ a unit test) — settling the open session-0006 calibration decision. While there, judge whether more neutral "scales of nature" base clips + a real multi-strength flow-stabilized Right re-bake are worth doing next vs. moving to scale-ring navigation (endless encoder + AI zoom transitions). Keep deferring Pi renderer + serial/firmware. Read sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md (§8) first.
```
@@ -0,0 +1,129 @@
# Session 0010.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-07T23-09 (PST)
> End: 2026-06-07T23-22 (PST)
> Type: coding
> Status: **FINALIZED**
> Posture: autonomous (yolo)
> Landed: PR #8 (merge `638cf58`) — calibration lock + dark-grade look fix
## Launch prompt
```
Tune the alteration look by eye in the simulator (python simulator/setup_sample_media.py then make sim-local) and LOCK the knob→strength calibration into DEFAULT_CALIBRATION in player/alteration.py (+ a unit test) — settling the open session-0006 calibration decision. While there, judge whether more neutral "scales of nature" base clips + a real multi-strength flow-stabilized Right re-bake are worth doing next vs. moving to scale-ring navigation (endless encoder + AI zoom transitions). Keep deferring Pi renderer + serial/firmware. Read sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-reconciled-simulator-alteration-slice-design.md (§8) first.
```
## Pre-state
- `main` clean at `e753a68` (then ff'd to the 0010 claim commit `7d2a306`).
- Sub-project 3 slice 2 (sim alteration) merged in session 0009 (PR #7). Engine
in `player/alteration.py` carried a `Calibration` + `DEFAULT_CALIBRATION` with
**behavior-preserving placeholder** values; the knob→strength calibration had
been the open decision since session 0006.
- Sample media already populated under `simulator/sample_media/forest/` (base +
Right strengths 14; only strength 4 is the real flow-stabilized restyle, 13
are ffmpeg blend placeholders). POC artifacts present in `~/hef-poc/out/`.
## The arc
### 1. Session open + grounding
- Classified the launch prompt as a **coding** session; ran
`wgl-session-coding-init`, claimed session **0010** (no other sessions in
flight). Verified clean pushed `main` baseline + CLAUDE.md stub.
- Read the resume context: `sub-project-3-player-progress` memory + the reconciled
slice design §8 (open questions: calibration curve shape, grade-vs-overlay
ordering, crossfade timing, placeholder fidelity).
- Inspected the engine (`player/alteration.py`), its test, the sim frontend
(`simulator/static/app.js`), and the content/clip plumbing. Established that the
calibration's three params (`mood_gain`, `overlay_gain`, `right_variant_map`)
are the lock target, and that the *grade look* itself lives in `app.js` CSS
filters (frontend), not in `DEFAULT_CALIBRATION`.
### 2. Tuning the look by eye (the key finding)
- `python`/`ffmpeg` weren't on the non-interactive shell PATH; used the project
`.venv`. Baseline suite: **192 passed / 2 skipped**.
- Viewed the operator's own POC renders in `~/hef-poc/out/` (`all_axes.png`,
`right_three.png`, `dark_frame.png`, `light_frame.png`, `lefthud_frame.png`) —
the by-eye-approved (session 0008) looks: tasteful cool-blue dark, warm golden
light, soft dreamy flow restyle, legible analytical HUD.
- Booted the sim (`uvicorn simulator.app:app`) and drove it with **headless
Chrome (puppeteer-core)** to screenshot the *actual* rendered look across all
four axes at full knob, plus the whole-brain-dark corner.
- **Finding:** the sim's **dark** mood pole was rendering a full-frame
`hue-rotate(-cool*200deg)` → rock turned orange, trees purple — a **psychedelic**
look, exactly the "disorienting, not peaceful" effect the operator rejected in
0008, and nothing like the POC `dark_frame`. Light / Left / Right all looked
good and peaceful. So the dark grade was the one thing blocking a likable look.
### 3. Dark-grade fix
- Replaced the hue-rotate in `applyGrade` (`simulator/static/app.js`) with:
darken + slight desaturate on the video filter for the cool pole, plus a
`multiply`-blended deep-blue wash (`#tint`, new element) that lifts shadows
toward blue while preserving natural greens. Placed `#tint` **below** the SVG
overlay so the Left HUD stays legible regardless of mood (resolves the §8
grade-vs-overlay ordering question → overlay above).
- Re-screenshotted: full dark now reads cool/somber with natural greens (matches
POC `dark_frame`); whole-brain-dark corner peaceful + HUD legible; light/neutral
unchanged (the warm path is algebraically identical for tone ≥ 0).
### 4. Locking the calibration
- With full tilt now tasteful on **every** axis (POC + sim evidence), the data
says full knob = full look, 5 notches map 1:1 to the 5 discrete Right bakes,
equal Dark/Light = identity → **unity gains + linear variant map**.
- Promoted `DEFAULT_CALIBRATION` from placeholder to a **deliberately locked**
`Calibration(mood_gain=1.0, overlay_gain=1.0, right_variant_map=(0,1,2,3,4))`
with a provenance comment; this also confirms the session-0006 convention
(0=off..4=max, equal Dark/Light = identity — no "centered at 2 = no push").
- Added `test_default_calibration_is_locked` pinning the literal constants.
Suite: **193 passed / 2 skipped**.
- Updated design §8 (calibration-curve + grade-vs-overlay resolved; dark-grade
fix recorded).
### 5. Land + build-vs-next judgment
- Committed on `feature/lock-alteration-calibration`, pushed, opened **PR #8**
(Gitea API via `wgl-gitea-admin` helper), merged to `main` (`638cf58`), deleted
the branch.
- Presented the build-vs-next judgment the prompt asked for. Recommendation
(operator-confirmed via AskUserQuestion): **move to scale-ring navigation next**
— pull in 12 *cheap, true-PD* neutral base clips (NASA cosmos, NOAA deep-sea)
so the ring is demonstrable, build the endless-encoder control + placeholder
zoom transitions in the sim, and **defer** the expensive real multi-strength
flow-stabilized Right re-bake until the ring experience is liked (re-baking
before the clip set is final risks wasted renders). Keep deferring Pi renderer +
serial/firmware.
## Cut state
- `main` @ `638cf58`, clean, pushed. Suite **193 passed / 2 skipped**.
- `DEFAULT_CALIBRATION` locked; dark-grade look fixed; design §8 updated; memory
updated with the session outcome + the new `Next /goal`.
- Deferred (unchanged): Pi renderer, serial/firmware; the expensive Right re-bake.
## Deferred decisions
- **Locked `DEFAULT_CALIBRATION` to unity/linear autonomously.** The operator
asked to "tune by eye" and lock; I made the by-eye judgment (via POC renders +
headless-Chrome sim screenshots) that full tilt is peaceful on every axis after
the dark fix, so no softening of gains was warranted, and locked the unity/linear
values. *Alternative:* soften `mood_gain`/`overlay_gain` (<1.0) so full knob is
gentler — set aside because the evidence showed full tilt is already calm. The
calibration stays fully parameterized, so flipping the feel is a one-line edit +
the lock test if the operator disagrees on review.
- **Dark-grade fix landed in the sim frontend (not the final renderer).** The fix
makes the *simulator* look right for tuning; the Pi/mpv renderer (later slice)
will do proper grading. Recorded in §8.
- The build-vs-next direction was **not** a deferred call — it was put to the
operator directly and confirmed.
## Operator plate
- Calibration is settled and guarded; the long-open session-0006 decision is
closed. The sim look is now likable on the one forest clip across all axes.
- Next session's first move is the `/goal` below (scale-ring navigation).
## Next-session prompt
```
/goal Build scale-ring navigation in the simulator — add an endless rotary-encoder control (relative, vs the absolute 0-4 experience knobs) that walks a closed ring of neutral "scales of nature" clips with placeholder AI zoom/warp transitions between scales. Add 12 cheap, true-PD neutral base clips so the ring is demonstrable (NASA/Hubble cosmos + NOAA Ocean Exploration deep sea). Defer the expensive real multi-strength flow-stabilized Right re-bake until the ring is liked; keep deferring the Pi renderer + serial/firmware. Read the sub-project-3-player-progress memory + docs/superpowers/specs/2026-06-07-scales-library-and-right-axis-pipeline-design.md first.
```
@@ -0,0 +1,22 @@
# Session 0011.0 — Transcript
> App: human-experience-filter-art
> Start: 2026-06-07T23-26 (PST)
> Type: coding
> Status: **PLACEHOLDER — claimed at session start; finalized at session end.**
>
> This file reserves session ID 0011 for human-experience-filter-art. The driver replaces this
> body with the full transcript and renames the file to its final
> SESSION-0011.0-TRANSCRIPT-2026-06-07T23-26--<end>.md form at session end.
## Launch prompt
```
/goal Build scale-ring navigation in the simulator — add an endless rotary-encoder control (relative, vs the absolute 0-4 experience knobs) that walks a closed ring of neutral "scales of nature" clips with placeholder AI zoom/warp transitions between scales. Add 12 cheap, true-PD neutral base clips so the ring is demonstrable (NASA/Hubble cosmos + NOAA Ocean Exploration deep sea). Defer the expensive real multi-strength flow-stabilized Right re-bake until the ring is liked; keep deferring the Pi renderer + serial/firmware.
```
## Deferred decisions
_Autonomous-mode low-confidence calls the driver made and would have
liked operator input on. Appended as the session runs; surfaced at
finalize. Empty if none._
+15
View File
@@ -16,5 +16,20 @@
},
"0006": {
"title": ""
},
"0007": {
"title": ""
},
"0008": {
"title": ""
},
"0009": {
"title": ""
},
"0010": {
"title": ""
},
"0011": {
"title": ""
}
}
+65 -62
View File
@@ -1,92 +1,95 @@
"""FastAPI service: dials -> real hef.selection -> X-ray (pick + ranked pool)."""
"""FastAPI service: controls -> the real alteration engine -> a RenderPlan.
The simulator's alteration surface (reconciled slice). It calls the canonical
player.alteration.plan_alteration; the browser only renders. The selection-era
endpoints (/api/select, /api/catalog/meta) and the X-ray are retired.
"""
from __future__ import annotations
import os
from collections import Counter
from pathlib import Path
from typing import Literal, Optional
from typing import Optional
from fastapi import FastAPI
from fastapi import FastAPI, HTTPException
from fastapi.staticfiles import StaticFiles
from pydantic import BaseModel, Field
from hef.catalog import load_catalog, record_to_dict
from hef.selection import (
Coordinate,
Weights,
candidates_for_mode,
ranked_candidates,
select,
from hef.selection import Coordinate
from player.alteration import (
DEFAULT_CALIBRATION,
Calibration,
plan_alteration,
render_plan_to_dict,
)
from simulator.fixtures import generate_fixture_catalog
from player.content import resolve_content
from player.controls import CONTENT_POSITIONS
from simulator.clips import load_manifest
STATIC_DIR = Path(__file__).parent / "static"
MEDIA_DIR = Path(__file__).parent / "sample_media"
DEFAULT_MANIFEST = MEDIA_DIR / "manifest.json"
class SelectRequest(BaseModel):
class ControlsModel(BaseModel):
content: str
left: int = Field(ge=0, le=4)
right: int = Field(ge=0, le=4)
dark: int = Field(ge=0, le=4)
light: int = Field(ge=0, le=4)
mode: Literal["none", "audio", "video", "av"]
pool_size: int = Field(default=4, ge=1, le=25)
brain_weight: float = Field(default=1.0, ge=0.0)
mood_weight: float = Field(default=1.0, ge=0.0)
approved_only: bool = False
volume: int = Field(ge=0, le=4)
brightness: int = Field(ge=0, le=4)
def load_catalog_or_fixtures() -> list:
"""Use the real catalog if a non-empty one is configured/exists, else fixtures."""
configured = os.environ.get("HEF_SIM_CATALOG")
path = Path(configured) if configured else Path("catalog/library.jsonl")
if path.exists() and path.stat().st_size > 0:
return load_catalog(path)
return generate_fixture_catalog()
class CalibrationModel(BaseModel):
mood_gain: float = 1.0
overlay_gain: float = 1.0
right_variant_map: list[int] = [0, 1, 2, 3, 4]
def create_app(records: Optional[list] = None) -> FastAPI:
app = FastAPI(title="HEF Experience Simulator")
app.state.catalog = records if records is not None else load_catalog_or_fixtures()
class AlterationRequest(BaseModel):
controls: ControlsModel
calibration: Optional[CalibrationModel] = None
@app.post("/api/select")
def api_select(req: SelectRequest):
catalog = app.state.catalog
coord = Coordinate(req.left, req.right, req.dark, req.light)
weights = Weights(brain=req.brain_weight, mood=req.mood_weight)
if req.mode == "none":
return {"pick": None, "pool": [], "coverage": {"candidates_in_mode": 0}}
pool = catalog
if req.approved_only:
pool = [r for r in pool if r.review_status == "approved"]
eligible = candidates_for_mode(pool, req.mode, req.pool_size)
ranked = ranked_candidates(
catalog, coord, req.mode,
pool_size=req.pool_size, weights=weights, approved_only=req.approved_only,
)
pick = select(
catalog, coord, req.mode,
pool_size=req.pool_size, weights=weights, approved_only=req.approved_only,
rng=None,
def _load_clips(manifest_path: Optional[Path]):
path = Path(manifest_path) if manifest_path else DEFAULT_MANIFEST
if path.exists():
return load_manifest(path)
return []
def create_app(manifest_path: Optional[Path] = None) -> FastAPI:
app = FastAPI(title="HEF Alteration Simulator")
app.state.clips = _load_clips(manifest_path)
@app.post("/api/alteration")
def api_alteration(req: AlterationRequest):
c = req.controls
if c.content not in CONTENT_POSITIONS:
raise HTTPException(status_code=422, detail=f"invalid content {c.content!r}")
coord = Coordinate(c.left, c.right, c.dark, c.light)
cal = (
Calibration(
mood_gain=req.calibration.mood_gain,
overlay_gain=req.calibration.overlay_gain,
right_variant_map=tuple(req.calibration.right_variant_map),
)
if req.calibration
else DEFAULT_CALIBRATION
)
plan = plan_alteration(coord, cal)
content = resolve_content(c.content)
return {
"pick": record_to_dict(pick) if pick else None,
"pool": [
{"record": record_to_dict(r), "distance": d, "rank": i + 1}
for i, (r, d) in enumerate(ranked)
],
"coverage": {"candidates_in_mode": len(eligible)},
"plan": render_plan_to_dict(plan),
"content": {"audio_source": content.audio_source, "video": content.video},
}
@app.get("/api/catalog/meta")
def api_meta():
catalog = app.state.catalog
return {
"total": len(catalog),
"by_mode": dict(Counter(r.mode for r in catalog)),
"by_status": dict(Counter(r.review_status for r in catalog)),
}
@app.get("/api/clips")
def api_clips():
return {"clips": [c.to_dict() for c in app.state.clips]}
if MEDIA_DIR.exists():
app.mount("/media", StaticFiles(directory=MEDIA_DIR), name="media")
if STATIC_DIR.exists():
app.mount("/", StaticFiles(directory=STATIC_DIR, html=True), name="static")
+68
View File
@@ -0,0 +1,68 @@
"""The base-clip + variant + annotation manifest the simulator renders.
Replaces simulator/fixtures.py (the selection-era synthetic catalog). Each base
clip carries: the raw base file, a map of pre-baked Right-strength variant files
(strength 0 is always the raw base), an authored Left annotation track (box +
label key + the minimum Left level at which it appears), and per-language string
tables. See the reconciled-simulator-alteration-slice design §3.2.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Any
@dataclass(frozen=True)
class Clip:
id: str
title: str
base_file: str
license: str
source: str
right_variants: dict # {"1": {"file": ...}, "4": {...}} (no "0")
annotations: list # [{"key", "box":[x,y,w,h], "min_level"}, ...]
strings: dict # {"en": {key: text}}
def variant_file(self, strength: int) -> str:
"""The video file for a Right strength; 0 and any unauthored strength
fall back to the raw base file."""
entry = self.right_variants.get(str(strength))
return entry["file"] if entry else self.base_file
def to_dict(self) -> dict:
variants = {"0": {"file": self.base_file, "raw": True}}
for k, v in self.right_variants.items():
variants[k] = v
return {
"id": self.id,
"title": self.title,
"base_file": self.base_file,
"license": self.license,
"source": self.source,
"right_variants": variants,
"annotations": self.annotations,
"strings": self.strings,
}
def _clip_from_dict(d: dict[str, Any]) -> Clip:
return Clip(
id=d["id"],
title=d["title"],
base_file=d["base_file"],
license=d.get("license", ""),
source=d.get("source", ""),
right_variants=d.get("right_variants", {}),
annotations=d.get("annotations", []),
strings=d.get("strings", {}),
)
def load_manifest(path: str | Path) -> list[Clip]:
"""Load the base-clip manifest. Raises FileNotFoundError if missing."""
path = Path(path)
data = json.loads(path.read_text())
return [_clip_from_dict(c) for c in data["clips"]]
-59
View File
@@ -1,59 +0,0 @@
"""Deterministic synthetic catalog so the selection model can be felt everywhere.
The real catalog (catalog/library.jsonl) is empty; this generates one record per
cell of the 5x5 brain x 5x5 mood coordinate space (625 records), with a seeded
mix of content modes and review statuses and no real media attached.
"""
from __future__ import annotations
import random
from hef.catalog import Record
MODES = ("audio", "video", "av")
ARCHIVES = ("internet_archive", "musopen", "librivox", "nasa", "freesound")
_LEFT_WORDS = ("Treatise", "Lecture", "Field Notes", "Reading", "Documentary")
_RIGHT_WORDS = ("Reverie", "Nocturne", "Bloom", "Drift", "Aurora")
def _title(left: int, right: int, dark: int, light: int, mode: str) -> str:
a = _LEFT_WORDS[left] if left >= right else _RIGHT_WORDS[right]
return f"{a} ({mode}) L{left}R{right}D{dark}Li{light}"
def generate_fixture_catalog(seed: int = 1729) -> list[Record]:
"""One valid Record per coordinate cell (625 total), deterministic for a seed."""
rng = random.Random(seed)
records: list[Record] = []
n = 0
for left in range(5):
for right in range(5):
for dark in range(5):
for light in range(5):
mode = rng.choice(MODES)
status = rng.choice(("proposed", "approved"))
is_video = mode in ("video", "av")
records.append(
Record(
id=f"fx-{n:04d}",
title=_title(left, right, dark, light, mode),
source_url=f"https://example.test/fx/{n:04d}",
source_archive=rng.choice(ARCHIVES),
license="public_domain",
mode=mode,
left=left,
right=right,
dark=dark,
light=light,
duration_s=rng.choice((300, 480, 600, 720, 900)),
file_path="",
review_status=status,
resolution="1920x1080" if is_video else "",
rationale=f"fixture at ({left},{right},{dark},{light})",
reviewed_at="2026-06-04T00:00:00Z" if status == "approved" else None,
)
)
n += 1
return records
+12
View File
@@ -0,0 +1,12 @@
# Simulator sample media
`manifest.json` is committed; the `.mp4` binaries are **not** (gitignored). They
are look-tuning samples, not shipped installation content.
Populate them from the session-0008 POC artifacts:
python simulator/setup_sample_media.py
This copies `~/hef-poc/out/neutral.mp4``forest/base.mp4` and
`~/hef-poc/out/right_flow.mp4``forest/right4.mp4` (the real flow-stabilized
restyle), and generates placeholder strengths `forest/right1..3.mp4`.
+31
View File
@@ -0,0 +1,31 @@
{
"clips": [
{
"id": "forest",
"title": "Yosemite Falls (neutral base, POC sample)",
"base_file": "forest/base.mp4",
"license": "poc-sample (look-tuning only; not shipped content)",
"source": "hef-poc/out/neutral.mp4",
"right_variants": {
"1": {"file": "forest/right1.mp4", "model": "placeholder"},
"2": {"file": "forest/right2.mp4", "model": "placeholder"},
"3": {"file": "forest/right3.mp4", "model": "placeholder"},
"4": {"file": "forest/right4.mp4", "model": "sd-turbo+farneback-flow"}
},
"annotations": [
{"key": "detected.water", "box": [0.30, 0.10, 0.18, 0.70], "min_level": 1},
{"key": "detected.rock_face", "box": [0.05, 0.30, 0.20, 0.55], "min_level": 2},
{"key": "detected.conifer", "box": [0.70, 0.20, 0.22, 0.45], "min_level": 3},
{"key": "measure.flow_rate", "box": [0.34, 0.55, 0.14, 0.08], "min_level": 4}
],
"strings": {
"en": {
"detected.water": "flowing water",
"detected.rock_face": "granite face",
"detected.conifer": "conifer stand",
"measure.flow_rate": "~2.1 m³/s"
}
}
}
]
}
+53
View File
@@ -0,0 +1,53 @@
"""Populate simulator/sample_media/forest/ from the session-0008 POC artifacts.
Copies the real neutral base + the real flow-stabilized Right restyle out of
~/hef-poc/out/, and generates placeholder intermediate Right strengths (1..3) by
blending the base toward the real restyle with ffmpeg. The media binaries are
gitignored; only the manifest is committed. Sample footage is for look-tuning
only, not shipped content.
Usage: python simulator/setup_sample_media.py
Requires: ffmpeg on PATH (or `pip install imageio-ffmpeg`), and ~/hef-poc/out/.
"""
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
POC = Path.home() / "hef-poc" / "out"
DEST = Path(__file__).parent / "sample_media" / "forest"
def _ffmpeg() -> str:
if shutil.which("ffmpeg"):
return "ffmpeg"
import imageio_ffmpeg
return imageio_ffmpeg.get_ffmpeg_exe()
def main() -> None:
DEST.mkdir(parents=True, exist_ok=True)
base = DEST / "base.mp4"
right4 = DEST / "right4.mp4"
shutil.copyfile(POC / "neutral.mp4", base)
shutil.copyfile(POC / "right_flow.mp4", right4)
ff = _ffmpeg()
# Placeholder strengths 1..3: opacity-blend base toward the real restyle.
for strength, alpha in ((1, 0.25), (2, 0.5), (3, 0.75)):
out = DEST / f"right{strength}.mp4"
subprocess.run(
[ff, "-y", "-i", str(base), "-i", str(right4),
"-filter_complex",
f"[1:v]format=yuva444p,colorchannelmixer=aa={alpha}[top];"
f"[0:v][top]overlay=shortest=1[v]",
"-map", "[v]", "-an", str(out)],
check=True,
)
print(f"generated {out.name} (alpha {alpha})")
print(f"sample media ready in {DEST}")
if __name__ == "__main__":
main()
+92 -98
View File
@@ -1,111 +1,105 @@
const DIALS = ["left", "right", "dark", "light"];
const MODEL = ["brain_weight", "mood_weight", "pool_size"];
// Thin renderer: post controls+calibration -> RenderPlan; render grade, Right
// variant crossfade, and the live Left overlay. All math stays in Python.
const $ = (id) => document.getElementById(id);
const vid = $("vid"), tint = $("tint"), overlay = $("overlay"), black = $("black"), readout = $("readout");
function buildGrid(el) {
el.innerHTML = "";
// rows = first axis 0..4 top->bottom, cols = second axis 0..4 left->right
for (let a = 0; a < 5; a++) {
for (let b = 0; b < 5; b++) {
const cell = document.createElement("div");
cell.className = "cell";
cell.dataset.a = a;
cell.dataset.b = b;
el.appendChild(cell);
}
let clip = null; // active clip manifest entry
let currentVariant = -1; // last loaded Right strength
async function loadClips() {
const data = await (await fetch("/api/clips")).json();
clip = data.clips[0] || null;
}
function mediaUrl(file) { return "/media/" + file; }
function variantFile(strength) {
const v = clip.right_variants[String(strength)];
return v ? v.file : clip.base_file;
}
function applyGrade(tone) {
// Light: warm + brighten (sepia). Dark: cool + darken via a multiply-blended
// blue wash (#tint) that lifts shadows toward blue while keeping natural
// greens — the peaceful POC dark look, NOT a full-frame hue spin.
const warm = tone > 0 ? tone : 0, cool = tone < 0 ? -tone : 0;
const bright = 1 + 0.25 * warm - 0.35 * cool;
const sat = 1 + 0.15 * warm - 0.30 * cool;
vid.style.filter =
`brightness(${bright.toFixed(3)}) saturate(${sat.toFixed(3)}) ` +
`sepia(${(warm * 0.5).toFixed(3)})`;
tint.style.opacity = (cool * 0.6).toFixed(3);
}
function loadVariant(strength) {
if (strength === currentVariant) return;
currentVariant = strength;
vid.style.opacity = "0";
setTimeout(() => {
vid.src = mediaUrl(variantFile(strength));
vid.play().catch(() => {});
vid.style.opacity = "1";
}, 150);
}
function renderOverlay(level, intensity) {
overlay.innerHTML = "";
if (!clip || level <= 0) { overlay.style.opacity = "0"; return; }
overlay.style.opacity = String(intensity);
const strings = (clip.strings && clip.strings.en) || {};
for (const a of clip.annotations) {
if (a.min_level > level) continue;
const [x, y, w, h] = a.box.map((n) => n * 100);
const rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
rect.setAttribute("x", x); rect.setAttribute("y", y);
rect.setAttribute("width", w); rect.setAttribute("height", h);
rect.setAttribute("class", "anno-box");
overlay.appendChild(rect);
const text = document.createElementNS("http://www.w3.org/2000/svg", "text");
text.setAttribute("x", x + 0.5); text.setAttribute("y", Math.max(y - 0.5, 2));
text.setAttribute("class", "anno-label");
text.textContent = strings[a.key] || a.key;
overlay.appendChild(text);
}
}
function paintGrid(el, axisA, axisB, point, pool) {
// clear
el.querySelectorAll(".cell").forEach((c) => {
c.className = "cell";
c.innerHTML = "";
});
const counts = {};
pool.forEach((c) => {
const r = c.record;
const key = `${r[axisA]},${r[axisB]}`;
counts[key] = (counts[key] || 0) + 1;
});
el.querySelectorAll(".cell").forEach((c) => {
const a = +c.dataset.a, b = +c.dataset.b;
const key = `${a},${b}`;
if (counts[key]) {
c.classList.add("cand");
const n = document.createElement("span");
n.className = "n";
n.textContent = counts[key];
c.appendChild(n);
}
if (a === point[axisA] && b === point[axisB]) c.classList.add("point");
});
function controls() {
return {
content: $("content").value,
left: +$("left").value, right: +$("right").value,
dark: +$("dark").value, light: +$("light").value,
volume: 2, brightness: 2,
};
}
function readState() {
const s = { mode: document.getElementById("mode").value, approved_only: document.getElementById("approved_only").checked };
DIALS.forEach((d) => (s[d] = +document.getElementById(d).value));
s.brain_weight = +document.getElementById("brain_weight").value;
s.mood_weight = +document.getElementById("mood_weight").value;
s.pool_size = +document.getElementById("pool_size").value;
return s;
function calibration() {
return { mood_gain: +$("mood_gain").value, overlay_gain: +$("overlay_gain").value,
right_variant_map: [0, 1, 2, 3, 4] };
}
function syncOutputs() {
[...DIALS, ...MODEL].forEach((id) => {
const out = document.getElementById(`${id}-out`);
if (out) out.textContent = document.getElementById(id).value;
});
}
async function refresh() {
syncOutputs();
const state = readState();
const resp = await fetch("/api/select", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(state),
let timer = null;
async function update() {
const resp = await fetch("/api/alteration", {
method: "POST", headers: { "content-type": "application/json" },
body: JSON.stringify({ controls: controls(), calibration: calibration() }),
});
if (!resp.ok) { readout.textContent = "invalid: " + resp.status; return; }
const data = await resp.json();
readout.textContent = JSON.stringify(data, null, 2);
if (!data.content.video) { black.classList.remove("hidden"); return; }
black.classList.add("hidden");
applyGrade(data.plan.grade.tone);
loadVariant(data.plan.restyle.variant);
renderOverlay(data.plan.overlay.level, data.plan.overlay.intensity);
}
const pickEl = document.getElementById("pick");
if (!data.pick) {
pickEl.innerHTML = '<div class="void">∅ Void / rest — walls dark, audio silent.</div>';
} else {
const p = data.pick;
pickEl.innerHTML =
`<div class="title">${p.title}</div>` +
`<div>mode ${p.mode} · coord (${p.left},${p.right},${p.dark},${p.light})</div>` +
`<div>${p.rationale || ""}</div>`;
function debounced() { clearTimeout(timer); timer = setTimeout(update, 80); }
async function main() {
await loadClips();
for (const id of ["content", "left", "right", "dark", "light", "mood_gain", "overlay_gain"]) {
$(id).addEventListener("input", debounced);
}
const poolEl = document.getElementById("pool");
poolEl.innerHTML = "";
data.pool.forEach((c, i) => {
const li = document.createElement("li");
if (i === 0) li.className = "winner";
li.innerHTML = `${c.record.title} <span class="dist">d=${c.distance.toFixed(2)}</span>`;
poolEl.appendChild(li);
});
const point = { left: state.left, right: state.right, dark: state.dark, light: state.light };
paintGrid(document.getElementById("brain-grid"), "left", "right", point, data.pool);
paintGrid(document.getElementById("mood-grid"), "dark", "light", point, data.pool);
update();
}
async function loadMeta() {
const data = await (await fetch("/api/catalog/meta")).json();
const byMode = Object.entries(data.by_mode).map(([k, v]) => `${k}:${v}`).join(" ");
document.getElementById("meta").textContent = `${data.total} records · ${byMode}`;
}
function init() {
buildGrid(document.getElementById("brain-grid"));
buildGrid(document.getElementById("mood-grid"));
document.querySelectorAll("input, select").forEach((el) =>
el.addEventListener("input", refresh)
);
loadMeta();
refresh();
}
document.addEventListener("DOMContentLoaded", init);
main();
+45 -52
View File
@@ -1,64 +1,57 @@
<!DOCTYPE html>
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>HEF — Curator's X-ray</title>
<link rel="stylesheet" href="/style.css">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>HEF — Alteration Simulator</title>
<link rel="stylesheet" href="/style.css" />
</head>
<body>
<header>
<h1>Experience Filter — Curator's X-ray</h1>
<div id="meta" class="meta"></div>
</header>
<header><h1>Human Experience Filter — Alteration Preview</h1></header>
<main>
<section class="controls">
<h2>Dials</h2>
<label>Mode
<select id="mode">
<option value="none">None</option>
<option value="audio">Audio</option>
<option value="video">Video</option>
<option value="av" selected>A+V</option>
</select>
</label>
<label>Left (analytical) <output id="left-out">0</output>
<input type="range" id="left" min="0" max="4" step="1" value="0"></label>
<label>Right (artistic) <output id="right-out">0</output>
<input type="range" id="right" min="0" max="4" step="1" value="0"></label>
<label>Dark (somber) <output id="dark-out">0</output>
<input type="range" id="dark" min="0" max="4" step="1" value="0"></label>
<label>Light (uplifting) <output id="light-out">0</output>
<input type="range" id="light" min="0" max="4" step="1" value="0"></label>
<h2>Model knobs</h2>
<label>Brain weight <output id="brain_weight-out">1</output>
<input type="range" id="brain_weight" min="0" max="4" step="0.5" value="1"></label>
<label>Mood weight <output id="mood_weight-out">1</output>
<input type="range" id="mood_weight" min="0" max="4" step="0.5" value="1"></label>
<label>Pool size <output id="pool_size-out">4</output>
<input type="range" id="pool_size" min="1" max="10" step="1" value="4"></label>
<label class="check"><input type="checkbox" id="approved_only"> Approved only</label>
<section class="stage">
<div class="screen">
<video id="vid" loop muted playsinline></video>
<div id="tint"></div>
<svg id="overlay" viewBox="0 0 100 100" preserveAspectRatio="none"></svg>
<div id="black" class="black hidden"></div>
</div>
</section>
<section class="xray">
<div class="pick">
<h2>Picked</h2>
<div id="pick"></div>
</div>
<div class="pool">
<h2>Pool (nearest first)</h2>
<ol id="pool"></ol>
</div>
<div class="maps">
<h2>Coordinate maps</h2>
<div class="map"><div class="label">Brain — Left × Right</div><div id="brain-grid" class="grid5"></div></div>
<div class="map"><div class="label">Mood — Dark × Light</div><div id="mood-grid" class="grid5"></div></div>
</div>
<section class="panel">
<fieldset>
<legend>Content dial</legend>
<select id="content">
<option value="video">video</option>
<option value="audio_video">audio + video</option>
<option value="music_video">music + video</option>
<option value="off">off (black)</option>
<option value="white_noise">white noise (no video)</option>
<option value="music">music (no video)</option>
<option value="audio_track">audio track (no video)</option>
</select>
</fieldset>
<fieldset>
<legend>Experience knobs (04)</legend>
<label>Left (analytical) <input type="range" id="left" min="0" max="4" value="0" /></label>
<label>Right (dreamlike) <input type="range" id="right" min="0" max="4" value="0" /></label>
<label>Dark <input type="range" id="dark" min="0" max="4" value="0" /></label>
<label>Light <input type="range" id="light" min="0" max="4" value="0" /></label>
</fieldset>
<fieldset>
<legend>Calibration</legend>
<label>mood gain <input type="range" id="mood_gain" min="0" max="2" step="0.05" value="1" /></label>
<label>overlay gain <input type="range" id="overlay_gain" min="0" max="2" step="0.05" value="1" /></label>
</fieldset>
<fieldset>
<legend>RenderPlan readout</legend>
<pre id="readout"></pre>
</fieldset>
</section>
</main>
<script src="/app.js"></script>
</body>
</html>
+24 -25
View File
@@ -1,26 +1,25 @@
:root { color-scheme: dark; }
* { box-sizing: border-box; }
body { margin: 0; font-family: -apple-system, system-ui, sans-serif; background: #0e0e16; color: #e6e6ee; }
header { padding: 12px 20px; border-bottom: 1px solid #2a2a3a; display: flex; justify-content: space-between; align-items: baseline; }
header h1 { font-size: 18px; margin: 0; }
.meta { font-size: 12px; color: #9a9ab0; }
main { display: grid; grid-template-columns: 280px 1fr; gap: 20px; padding: 20px; }
.controls label { display: block; margin: 8px 0; font-size: 13px; }
.controls input[type=range] { width: 100%; }
.controls .check { display: flex; gap: 6px; align-items: center; }
h2 { font-size: 13px; text-transform: uppercase; letter-spacing: .5px; color: #9a9ab0; }
.xray { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; align-items: start; }
.pick #pick { background: #1a1a2e; border: 1px solid #33334a; border-radius: 8px; padding: 14px; min-height: 80px; }
.pick .title { font-size: 16px; font-weight: 600; }
.pick .void { color: #7777aa; font-style: italic; }
.pool ol { margin: 0; padding-left: 18px; font-size: 13px; }
.pool li { margin: 4px 0; }
.pool li.winner { color: #7fffd4; font-weight: 600; }
.pool .dist { color: #9a9ab0; }
.maps { grid-column: 1 / -1; display: flex; gap: 40px; }
.grid5 { display: grid; grid-template-columns: repeat(5, 28px); grid-template-rows: repeat(5, 28px); gap: 3px; }
.grid5 .cell { background: #1c1c2c; border: 1px solid #2a2a3a; border-radius: 3px; position: relative; }
.grid5 .cell.cand { background: #3a3a66; }
.grid5 .cell.point { outline: 2px solid #7fffd4; }
.grid5 .cell .n { position: absolute; right: 2px; bottom: 1px; font-size: 9px; color: #aab; }
.label { font-size: 11px; color: #9a9ab0; margin-bottom: 4px; }
body { margin: 0; font: 14px/1.4 system-ui, sans-serif; background: #111; color: #eee; }
header { padding: 0.6rem 1rem; background: #000; }
h1 { font-size: 1rem; margin: 0; font-weight: 600; }
main { display: flex; gap: 1rem; padding: 1rem; flex-wrap: wrap; }
.stage { flex: 1 1 640px; }
.screen { position: relative; width: 100%; aspect-ratio: 16 / 9; background: #000;
border-radius: 6px; overflow: hidden; }
#vid { width: 100%; height: 100%; object-fit: cover; transition: opacity 0.15s ease; }
#tint { position: absolute; inset: 0; pointer-events: none; opacity: 0;
background: #28425f; mix-blend-mode: multiply;
transition: opacity 0.2s ease; }
#overlay { position: absolute; inset: 0; width: 100%; height: 100%;
pointer-events: none; transition: opacity 0.2s ease; }
.anno-box { fill: none; stroke: #6cf; stroke-width: 0.4; vector-effect: non-scaling-stroke; }
.anno-label { fill: #6cf; font-size: 3px; font-family: monospace; }
.black { position: absolute; inset: 0; background: #000; }
.hidden { display: none; }
.panel { flex: 0 0 280px; display: flex; flex-direction: column; gap: 0.8rem; }
fieldset { border: 1px solid #333; border-radius: 6px; }
legend { color: #9af; padding: 0 0.4rem; }
label { display: block; margin: 0.4rem 0; }
input[type=range], select { width: 100%; }
#readout { background: #000; padding: 0.5rem; border-radius: 4px; font-size: 12px;
white-space: pre-wrap; max-height: 240px; overflow: auto; }
+69
View File
@@ -0,0 +1,69 @@
import json
import pytest
from simulator.clips import Clip, load_manifest
def _manifest_dict():
return {
"clips": [
{
"id": "forest",
"title": "Yosemite Falls (neutral)",
"base_file": "forest/base.mp4",
"license": "poc-sample",
"source": "hef-poc",
"right_variants": {
"4": {"file": "forest/right4.mp4", "model": "sd-turbo+flow"},
"1": {"file": "forest/right1.mp4"},
},
"annotations": [
{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1},
{"key": "detected.conifer", "box": [0.6, 0.1, 0.2, 0.2], "min_level": 3},
],
"strings": {"en": {"detected.water": "flowing water", "detected.conifer": "conifer"}},
}
]
}
def test_load_manifest_parses_clips(tmp_path):
p = tmp_path / "manifest.json"
p.write_text(json.dumps(_manifest_dict()))
clips = load_manifest(p)
assert len(clips) == 1
c = clips[0]
assert isinstance(c, Clip)
assert c.id == "forest"
assert c.base_file == "forest/base.mp4"
def test_clip_lists_variant_files_by_strength(tmp_path):
p = tmp_path / "manifest.json"
p.write_text(json.dumps(_manifest_dict()))
c = load_manifest(p)[0]
# variant 0 is always the raw base; authored strengths come from the manifest
assert c.variant_file(0) == "forest/base.mp4"
assert c.variant_file(4) == "forest/right4.mp4"
assert c.variant_file(1) == "forest/right1.mp4"
# an unauthored strength falls back to the raw base
assert c.variant_file(2) == "forest/base.mp4"
def test_clip_serializes_to_dict_for_the_api(tmp_path):
p = tmp_path / "manifest.json"
p.write_text(json.dumps(_manifest_dict()))
d = load_manifest(p)[0].to_dict()
assert d["id"] == "forest"
assert d["base_file"] == "forest/base.mp4"
assert d["annotations"][0]["key"] == "detected.water"
assert d["strings"]["en"]["detected.water"] == "flowing water"
# variant map is exposed keyed by strength string, including 0 -> base
assert d["right_variants"]["0"]["file"] == "forest/base.mp4"
assert d["right_variants"]["4"]["file"] == "forest/right4.mp4"
def test_missing_manifest_raises(tmp_path):
with pytest.raises(FileNotFoundError):
load_manifest(tmp_path / "nope.json")
-40
View File
@@ -1,40 +0,0 @@
from hef.catalog import validate_catalog
from hef.selection import CONTENT_MODES
from simulator.fixtures import generate_fixture_catalog
def test_fixture_catalog_is_valid():
records = generate_fixture_catalog()
validate_catalog(records) # raises on any invalid record or duplicate id
def test_fixture_catalog_spans_the_coordinate_space():
records = generate_fixture_catalog()
coords = {(r.left, r.right, r.dark, r.light) for r in records}
# all 625 cells of the 5x5 brain x 5x5 mood space are present
assert len(coords) == 625
def test_fixture_catalog_has_every_content_mode():
records = generate_fixture_catalog()
present = {r.mode for r in records}
assert CONTENT_MODES <= present
def test_fixture_catalog_mixes_review_statuses():
records = generate_fixture_catalog()
statuses = {r.review_status for r in records}
assert statuses == {"proposed", "approved"}
def test_fixture_catalog_references_no_real_media():
records = generate_fixture_catalog()
assert all(r.file_path == "" for r in records)
def test_fixture_catalog_is_deterministic():
a = generate_fixture_catalog(seed=42)
b = generate_fixture_catalog(seed=42)
assert [r.id for r in a] == [r.id for r in b]
assert [r.mode for r in a] == [r.mode for r in b]
assert [r.review_status for r in a] == [r.review_status for r in b]
+60 -12
View File
@@ -2,11 +2,14 @@ import pytest
from hef.selection import Coordinate
from player.alteration import (
DEFAULT_CALIBRATION,
AnalyticalOverlay,
Calibration,
ColorGrade,
RenderPlan,
Restyle,
plan_alteration,
render_plan_to_dict,
)
@@ -17,30 +20,32 @@ def _coord(left=0, right=0, dark=0, light=0):
def test_all_zero_knobs_is_the_unaltered_base():
plan = plan_alteration(_coord())
assert plan.is_identity
assert plan.overlay.level == 0
assert plan.overlay.intensity == 0.0
assert plan.restyle.blend == 0.0
assert plan.restyle.variant == 0
assert plan.grade.tone == 0.0
assert plan.grade.is_identity
def test_left_drives_the_analytical_overlay_only():
plan = plan_alteration(_coord(left=4))
assert plan.overlay.level == 4
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 0.0 # Left does not touch the substrate
assert plan.restyle.variant == 0 # Left does not touch the substrate
assert plan.grade.tone == 0.0
def test_right_drives_the_restyle_substrate_only():
def test_right_selects_a_discrete_restyle_variant_only():
plan = plan_alteration(_coord(right=2))
assert plan.restyle.blend == 0.5
assert plan.overlay.intensity == 0.0 # Right does not add overlay
assert plan.restyle.variant == 2
assert plan.overlay.level == 0 # Right does not add overlay
def test_left_and_right_stack_not_cancel():
# design §4.2: whole-brain corner = dreamlike substrate WITH labels on top
plan = plan_alteration(_coord(left=4, right=4))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 1.0
assert plan.overlay.level == 4
assert plan.restyle.variant == 4
def test_light_pole_grades_warm_positive_tone():
@@ -50,8 +55,7 @@ def test_light_pole_grades_warm_positive_tone():
def test_dark_pole_grades_cool_negative_tone():
plan = plan_alteration(_coord(dark=4))
assert plan.grade.tone == -1.0
assert plan_alteration(_coord(dark=4)).grade.tone == -1.0
def test_equal_dark_and_light_is_identity_grade():
@@ -66,14 +70,58 @@ def test_dark_minus_light_sets_intermediate_tone():
def test_whole_brain_dark_corner_stacks_grade_substrate_and_overlay():
# design §4.2 "Dark + analytical": cold measurement over a melancholy scene
plan = plan_alteration(_coord(left=4, right=2, dark=4, light=0))
assert plan.overlay.intensity == 1.0
assert plan.restyle.blend == 0.5
assert plan.overlay.level == 4
assert plan.restyle.variant == 2
assert plan.grade.tone == -1.0
assert not plan.is_identity
def test_default_calibration_is_locked():
# Session 0010: the knob->strength calibration is LOCKED to these values,
# settled by eye in the simulator (closes the open session-0006 decision).
# This pins the literal constants so they can't drift silently; changing the
# locked feel is a deliberate edit here + in alteration.py.
assert DEFAULT_CALIBRATION.mood_gain == 1.0
assert DEFAULT_CALIBRATION.overlay_gain == 1.0
assert DEFAULT_CALIBRATION.right_variant_map == (0, 1, 2, 3, 4)
def test_default_calibration_is_behavior_preserving():
# DEFAULT_CALIBRATION must reproduce the original three helpers exactly.
for left in range(5):
assert plan_alteration(_coord(left=left)).overlay.intensity == pytest.approx(left / 4)
for right in range(5):
assert plan_alteration(_coord(right=right)).restyle.variant == right
for dark in range(5):
for light in range(5):
expected = (light - dark) / 4
assert plan_alteration(_coord(dark=dark, light=light)).grade.tone == pytest.approx(expected)
def test_custom_calibration_scales_mood_and_overlay():
cal = Calibration(mood_gain=0.5, overlay_gain=0.5, right_variant_map=(0, 0, 1, 1, 2))
assert plan_alteration(_coord(light=4), cal).grade.tone == pytest.approx(0.5)
assert plan_alteration(_coord(left=4), cal).overlay.intensity == pytest.approx(0.5)
assert plan_alteration(_coord(right=3), cal).restyle.variant == 1
def test_calibration_gain_is_clamped_to_unit_range():
cal = Calibration(mood_gain=10.0, overlay_gain=10.0)
assert plan_alteration(_coord(light=4), cal).grade.tone == 1.0 # clamped, not 10
assert plan_alteration(_coord(left=4), cal).overlay.intensity == 1.0
def test_render_plan_to_dict_round_trips_the_numbers():
d = render_plan_to_dict(plan_alteration(_coord(left=4, right=2, dark=4, light=0)))
assert d == {
"grade": {"tone": -1.0},
"overlay": {"level": 4, "intensity": 1.0},
"restyle": {"variant": 2},
"is_identity": False,
}
def test_render_plan_is_frozen():
plan = plan_alteration(_coord())
with pytest.raises(Exception):
+2 -2
View File
@@ -56,7 +56,7 @@ def test_overlay_change_is_a_live_update():
p.update(_controls(content="video", left=0))
t = p.update(_controls(content="video", left=4))
assert t.kind == TransitionKind.LIVE_UPDATE
assert t.playback.plan.overlay.intensity == 1.0
assert t.playback.plan.overlay.level == 4
def test_restyle_change_crossfades_the_substrate():
@@ -65,7 +65,7 @@ def test_restyle_change_crossfades_the_substrate():
p.update(_controls(content="video", right=0))
t = p.update(_controls(content="video", right=4))
assert t.kind == TransitionKind.CROSSFADE
assert t.playback.plan.restyle.blend == 1.0
assert t.playback.plan.restyle.variant == 4
def test_volume_only_change_is_a_live_update():
+54 -66
View File
@@ -1,102 +1,90 @@
import json
import pytest
from fastapi.testclient import TestClient
from hef.catalog import Record
from simulator.app import create_app
def make_record(**overrides):
base = dict(
id="r",
title="t",
source_url="u",
source_archive="internet_archive",
license="public_domain",
mode="video",
left=0,
right=0,
dark=0,
light=0,
duration_s=600,
file_path="",
)
base.update(overrides)
return Record(**base)
@pytest.fixture
def manifest_path(tmp_path):
p = tmp_path / "manifest.json"
p.write_text(json.dumps({
"clips": [{
"id": "forest",
"title": "neutral forest",
"base_file": "forest/base.mp4",
"license": "poc", "source": "hef-poc",
"right_variants": {"4": {"file": "forest/right4.mp4"}},
"annotations": [{"key": "detected.water", "box": [0.1, 0.2, 0.3, 0.4], "min_level": 1}],
"strings": {"en": {"detected.water": "flowing water"}},
}]
}))
return p
@pytest.fixture
def client():
records = [
make_record(id="v-near", mode="video", left=0, right=0, dark=0, light=0),
make_record(id="v-far", mode="video", left=4, right=4, dark=4, light=4),
make_record(id="a-one", mode="audio", left=1, right=1, dark=1, light=1),
make_record(id="prop", mode="video", left=0, right=0, dark=0, light=1,
review_status="proposed"),
make_record(id="appr", mode="video", left=0, right=0, dark=0, light=1,
review_status="approved"),
]
return TestClient(create_app(records=records))
def client(manifest_path):
return TestClient(create_app(manifest_path=manifest_path))
def _body(**overrides):
base = dict(left=0, right=0, dark=0, light=0, mode="video")
base.update(overrides)
return base
def _controls(content="video", left=0, right=0, dark=0, light=0, volume=2, brightness=2):
return dict(content=content, left=left, right=right, dark=dark,
light=light, volume=volume, brightness=brightness)
def test_select_returns_pick_and_ranked_pool(client):
resp = client.post("/api/select", json=_body(mode="video", pool_size=4))
def test_alteration_returns_the_engine_plan(client):
resp = client.post("/api/alteration", json={"controls": _controls(left=4, right=2, dark=4)})
assert resp.status_code == 200
data = resp.json()
assert data["pick"]["id"] == "v-near"
ids = [c["record"]["id"] for c in data["pool"]]
assert ids[0] == "v-near"
assert all("distance" in c and "rank" in c for c in data["pool"])
assert [c["rank"] for c in data["pool"]] == list(range(1, len(data["pool"]) + 1))
assert data["plan"]["overlay"]["level"] == 4
assert data["plan"]["restyle"]["variant"] == 2
assert data["plan"]["grade"]["tone"] == -1.0
assert data["content"]["video"] is True
def test_none_mode_is_the_void(client):
resp = client.post("/api/select", json=_body(mode="none"))
assert resp.status_code == 200
def test_alteration_honors_off_as_black(client):
resp = client.post("/api/alteration", json={"controls": _controls(content="off")})
data = resp.json()
assert data["pick"] is None
assert data["pool"] == []
assert data["content"]["video"] is False
def test_dial_out_of_range_is_rejected(client):
resp = client.post("/api/select", json=_body(left=7))
def test_alteration_accepts_calibration(client):
body = {"controls": _controls(light=4),
"calibration": {"mood_gain": 0.5, "overlay_gain": 1.0, "right_variant_map": [0, 1, 2, 3, 4]}}
resp = client.post("/api/alteration", json=body)
assert resp.json()["plan"]["grade"]["tone"] == 0.5
def test_alteration_rejects_out_of_range_knob(client):
resp = client.post("/api/alteration", json={"controls": _controls(left=7)})
assert resp.status_code == 422
def test_bad_mode_is_rejected(client):
resp = client.post("/api/select", json=_body(mode="banana"))
def test_alteration_rejects_bad_content(client):
resp = client.post("/api/alteration", json={"controls": _controls(content="banana")})
assert resp.status_code == 422
def test_approved_only_narrows_pool(client):
resp = client.post("/api/select", json=_body(left=0, right=0, dark=0, light=1,
mode="video", approved_only=True))
data = resp.json()
assert all(c["record"]["review_status"] == "approved" for c in data["pool"])
def test_catalog_meta_reports_counts(client):
resp = client.get("/api/catalog/meta")
def test_clips_returns_the_manifest(client):
resp = client.get("/api/clips")
assert resp.status_code == 200
data = resp.json()
assert data["total"] == 5
assert data["by_mode"]["video"] == 4
assert data["by_mode"]["audio"] == 1
assert set(data["by_status"]) == {"proposed", "approved"}
assert data["clips"][0]["id"] == "forest"
assert data["clips"][0]["right_variants"]["0"]["file"] == "forest/base.mp4"
assert data["clips"][0]["annotations"][0]["key"] == "detected.water"
from simulator.app import create_app as _create_app_for_static
def test_retired_selection_endpoints_are_gone(client):
# The route no longer exists; the static catch-all yields 404 on GET and
# 405 on the (now-unrouted) POST. Either proves the endpoint is gone.
assert client.post("/api/select", json={}).status_code in (404, 405)
assert client.get("/api/catalog/meta").status_code == 404
def test_index_is_served():
# The default app mounts the real static dir.
client = TestClient(_create_app_for_static())
client = TestClient(create_app())
resp = client.get("/")
assert resp.status_code == 200
assert "text/html" in resp.headers["content-type"]
assert "X-ray" in resp.text
assert "Alteration" in resp.text