diff --git a/docs/superpowers/plans/2026-06-08-v2v-vertical-slice-forest-right-variants.md b/docs/superpowers/plans/2026-06-08-v2v-vertical-slice-forest-right-variants.md new file mode 100644 index 0000000..c982f70 --- /dev/null +++ b/docs/superpowers/plans/2026-06-08-v2v-vertical-slice-forest-right-variants.md @@ -0,0 +1,239 @@ +# v2v Vertical Slice — Real Forest Right Variants Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Produce the first *real* (not placeholder) multi-strength Right-axis +restyle media for the simulator — flow-stabilized painterly variants of the +forest scale at Right strengths 1–4 — so the dreamlike-axis *look* can finally be +judged by eye on real footage, breaking the "can't judge the ring liked without +real content" chicken-and-egg (operator chose a vertical slice, session 0012). + +**Architecture:** Productionize the proven session-0008 POC restyle algorithm +(`~/hef-poc/flow_restyle.py`) into a repo-resident offline baker, +`simulator/bake_right_variants.py`, a sibling of the existing +`setup_sample_media.py` / `setup_scales_media.py` media-generation scripts. The +algorithm is unchanged (extract frames → SD img2img on keyframes → Farneback +optical-flow warp + light refine on tweens → reassemble), which sessions 0008 +proved is temporally calm (no boiling). The one new idea is a **strength curve**: +Right level 1–4 maps to an increasing keyframe img2img `strength`, so the four +discrete pre-baked variants form a subtle→strong dreamlike ramp. Only the curve +helper is unit-tested; the rendered media is verified by eye (screenshots), as +the other media-setup scripts are. + +**Tech Stack:** Python, diffusers `AutoPipelineForImage2Image` + `stabilityai/sd-turbo` +(cached, 4.8 G) on Apple MPS (torch 2.12, available in the project `.venv`), +OpenCV Farneback optical flow, imageio-ffmpeg. ~2.7 min/clip × 4 ≈ ~11 min local +compute. + +**Scope note — what this slice is NOT:** the *real i2v ring transition* half of +the v2v build is deferred: it needs a second real scale base (cosmos/abyss are +still synthetic placeholders) plus a generative-video model not in the POC, and +real strict-PD footage sourcing (NASA/NOAA, sub-project 2). This slice does the +Right-variant half, which is fully achievable now (forest has a real base). + +--- + +### Task 1: The Right strength curve (pure, testable) + +**Files:** +- Create: `simulator/bake_right_variants.py` +- Test: `tests/test_bake_right_variants.py` + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_bake_right_variants.py +import pytest + +from simulator.bake_right_variants import right_strength_params, RIGHT_LEVELS + + +def test_levels_are_one_through_four(): + assert RIGHT_LEVELS == (1, 2, 3, 4) + + +def test_strength_increases_with_level(): + keys = [right_strength_params(l)[0] for l in RIGHT_LEVELS] + assert keys == sorted(keys) # monotonic ramp + assert keys[0] < keys[-1] # subtle -> strong + + +def test_tween_strength_is_a_fraction_of_key_strength(): + for l in RIGHT_LEVELS: + key, tween = right_strength_params(l) + assert 0.0 < tween < key # tween is a light refine only + + +def test_rejects_out_of_range_level(): + with pytest.raises(ValueError): + right_strength_params(0) + with pytest.raises(ValueError): + right_strength_params(5) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_bake_right_variants.py -q` +Expected: FAIL (ModuleNotFoundError / ImportError — module not created yet) + +- [ ] **Step 3: Write minimal implementation (curve only, no torch import at module load)** + +```python +# simulator/bake_right_variants.py (top of file) +"""Offline baker: real flow-stabilized painterly Right-axis variants (scales +design §1/§4). Productionizes the session-0008 POC (~/hef-poc/flow_restyle.py): +SD img2img keyframes + Farneback optical-flow tweens = temporally calm restyle. + +Right level -> keyframe img2img strength ramp (subtle -> strong); tween strength +is a light refine fraction of the keyframe strength. Curve is by-eye tunable. + +Usage: .venv/bin/python -m simulator.bake_right_variants [--scale forest] +Requires: torch+MPS, diffusers, sd-turbo (cached), opencv, imageio-ffmpeg. +""" +from __future__ import annotations + +RIGHT_LEVELS = (1, 2, 3, 4) + +# Keyframe img2img strength per Right level (by-eye ramp). Level 4 ~ the POC's +# proven 0.5-0.58 painterly look; level 1 is a barely-there dream haze. +_KEY_STRENGTH = {1: 0.28, 2: 0.38, 3: 0.48, 4: 0.58} +_TWEEN_RATIO = 0.36 # POC ratio (0.18 / 0.5): tweens are a light refine only + + +def right_strength_params(level: int) -> tuple[float, float]: + """(keyframe_strength, tween_strength) for a Right level in RIGHT_LEVELS.""" + if level not in _KEY_STRENGTH: + raise ValueError(f"Right level must be one of {RIGHT_LEVELS}, got {level}") + key = _KEY_STRENGTH[level] + return key, round(key * _TWEEN_RATIO, 4) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_bake_right_variants.py -q` +Expected: PASS (4 passed) + +- [ ] **Step 5: Commit** + +```bash +git add simulator/bake_right_variants.py tests/test_bake_right_variants.py +git commit -m "feat(pipeline): Right strength curve for the v2v variant baker" +``` + +--- + +### Task 2: The baker (flow restyle, productionized) + +**Files:** +- Modify: `simulator/bake_right_variants.py` + +No unit test for the render itself (non-trivial ML output; verified by eye in +Task 4). The pure curve is already covered by Task 1. + +- [ ] **Step 1: Add the restyle engine + per-level bake, porting the POC algorithm verbatim** + +Port `extract`, `warp`, and the keyframe/tween loop from +`~/hef-poc/flow_restyle.py` unchanged (it is the approved calm algorithm). Wrap +it as `restyle_clip(src, out, key_strength, tween_strength, *, fps=12.0, keyint=24)` +and add a `bake_scale(scale="forest")` that loops `RIGHT_LEVELS`, calling +`restyle_clip` with `right_strength_params(level)` and writing +`simulator/sample_media//right.mp4`. Load the SD pipeline ONCE and +reuse it across levels. `PROMPT`/`NEG`/Farneback params/seed = the POC's. + +Source base = `simulator/sample_media//base.mp4` (forest = the real +Yosemite POC clip). `main()` parses `--scale` (default `forest`), `--fps`, +`--keyint` and calls `bake_scale`. + +- [ ] **Step 2: Smoke-check the module imports and the curve still passes** + +Run: `.venv/bin/python -m pytest tests/test_bake_right_variants.py -q` +Expected: PASS (torch imported lazily inside the bake funcs, not at module load, +so the test stays fast and import-safe) + +- [ ] **Step 3: Commit** + +```bash +git add simulator/bake_right_variants.py +git commit -m "feat(pipeline): forest Right-variant baker (POC flow restyle, productionized)" +``` + +--- + +### Task 3: Run the bake (real media, ~11 min) + +**Files:** +- Writes (gitignored): `simulator/sample_media/forest/right{1,2,3,4}.mp4` + +- [ ] **Step 1: Run the baker** + +Run: `.venv/bin/python -m simulator.bake_right_variants --scale forest` +Expected: 4 clips written; per-level timing printed; ~11 min total. Replaces the +3 placeholder strengths (1–3) and the single real strength (4) with a consistent +real ramp. + +- [ ] **Step 2: Confirm the outputs exist and are real video** + +Run: `ls -la simulator/sample_media/forest/right*.mp4` +Expected: right1–4.mp4 present, each a multi-MB H.264 clip. + +(No commit — media is gitignored; it is reproduced by the baker.) + +--- + +### Task 4: Wire the manifest + verify by eye + +**Files:** +- Modify: `simulator/sample_media/manifest.json` (forest `right_variants` models) +- Modify: `docs/USER_GUIDE.md` (note real forest Right variants + the baker) +- Modify: `docs/ROADMAP.md` (v2v slice progress) + +- [ ] **Step 1: Update the forest variant models in the manifest** + +Set every forest `right_variants` entry (1–4) `model` to +`"sd-turbo+farneback-flow"` (drop the `"placeholder"` labels) — they are all real +bakes now. + +- [ ] **Step 2: Boot the sim and drive the Right axis across 0→4 on forest** + +Run: `.venv/bin/python -m uvicorn simulator.app:app --port 8013` (background), +navigate to the forest scale, and capture headless screenshots at Right 0/1/2/3/4. +Expected: a coherent subtle→strong painterly ramp on the real Yosemite footage, +temporally calm (no boiling), grade/overlay still compose on top. + +- [ ] **Step 3: Update docs** + +USER_GUIDE: note forest now carries real flow-stabilized Right variants generated +by `simulator/bake_right_variants.py`. ROADMAP: mark the v2v Right-variant vertical +slice done; note the i2v ring-transition half still deferred (needs a 2nd real +base + video model). + +- [ ] **Step 4: Run the full suite** + +Run: `.venv/bin/python -m pytest -q` +Expected: PASS (existing + Task-1 curve tests). + +- [ ] **Step 5: Commit** + +```bash +git add simulator/sample_media/manifest.json docs/USER_GUIDE.md docs/ROADMAP.md +git commit -m "feat(sim): forest carries real multi-strength Right variants; docs" +``` + +--- + +## Self-Review + +**Spec coverage:** scales design §1 (local restyle pipeline) + §4 (economics: +local bake) → Tasks 2–3 productionize + run it. The operator's vertical-slice +choice (real Right variants for one scale to judge the look) → Tasks 3–4. The +deferred i2v transition half is explicitly scoped out (noted in header + Task 4). +The strength curve (the one new design element) → Task 1. + +**Placeholder scan:** none — the baker algorithm is a verbatim port of the +existing `~/hef-poc/flow_restyle.py` (cited), the curve is concrete constants, +commands are exact. + +**Type consistency:** `right_strength_params(level) -> (key, tween)` defined in +Task 1, consumed by `bake_scale` in Task 2. `RIGHT_LEVELS` shared. Output paths +`simulator/sample_media/forest/right.mp4` consistent with the manifest's +existing forest `right_variants` map (Task 4).