feat(v2v): real multi-strength Right variants for the forest scale (vertical slice) #11
+10
-1
@@ -175,7 +175,16 @@ every transition, so fast navigation stays responsive on placeholder media.
|
||||
scales-library/right-axis design §1/§4); a real multi-strength flow-stabilized
|
||||
re-bake per base clip + the multilingual label/string tables + TTS. The
|
||||
**scale-ring transitions** (forward/reverse per-edge morphs) are part of this
|
||||
offline pipeline too — the simulator currently uses placeholders.
|
||||
offline pipeline too.
|
||||
- **Vertical slice done (session 0012):** the Right-variant half is
|
||||
productionized and run for the **forest** scale — `simulator/bake_right_variants.py`
|
||||
(the POC flow-restyle algorithm, repo-resident) bakes real strengths 1–4 on a
|
||||
by-eye keyframe-strength ramp; forest now carries real Right variants in the
|
||||
sim, so the dreamlike-axis look is judgeable on real footage. **Still
|
||||
deferred:** real Right variants for cosmos/abyss (need their real strict-PD
|
||||
bases sourced first) and the **real i2v ring transitions** (need a 2nd real
|
||||
base + a generative-video model not in the POC). Plan:
|
||||
[`2026-06-08-v2v-vertical-slice-forest-right-variants.md`](./superpowers/plans/2026-06-08-v2v-vertical-slice-forest-right-variants.md).
|
||||
- **Catalog model changes** — audio *source* + "neutral base" vs "altered
|
||||
variant" flag (sub-project 2 territory, design §13).
|
||||
|
||||
|
||||
+15
-9
@@ -298,16 +298,22 @@ piece moved from *selecting* clips to *altering* them.)
|
||||
**One-time setup — populate the sample footage** (look-tuning only; not shipped
|
||||
content):
|
||||
|
||||
python simulator/setup_sample_media.py # the forest scale (real POC base + Right variants)
|
||||
python simulator/setup_scales_media.py # the cosmos + abyss scales + ring transitions
|
||||
python simulator/setup_sample_media.py # stage the forest neutral base
|
||||
python -m simulator.bake_right_variants --scale forest # real Right variants (SD on MPS, ~11 min)
|
||||
python simulator/setup_scales_media.py # the cosmos + abyss scales + ring transitions
|
||||
|
||||
`setup_sample_media.py` copies the session-0008 POC artifacts (`~/hef-poc/out/`)
|
||||
into `simulator/sample_media/forest/` — the neutral base clip and the real
|
||||
flow-stabilized Right restyle — and generates placeholder intermediate Right
|
||||
strengths. `setup_scales_media.py` makes the scale **ring** demonstrable: cheap
|
||||
synthetic placeholder bases for the two true-PD scales (`cosmos` = NASA/Hubble,
|
||||
`abyss` = NOAA Ocean Exploration) plus the per-edge zoom/warp transition clips.
|
||||
The `.mp4` binaries are gitignored.
|
||||
`setup_sample_media.py` stages the forest neutral base from the session-0008 POC
|
||||
artifacts (`~/hef-poc/out/neutral.mp4`). `bake_right_variants.py` then produces
|
||||
the **real** flow-stabilized painterly Right variants (strengths 1–4) for the
|
||||
forest scale — SD img2img keyframes + Farneback optical-flow tweens on Apple MPS,
|
||||
the temporally-calm POC algorithm, productionized (scales design §1/§4). The
|
||||
keyframe-strength ramp per Right level is by-eye tunable in
|
||||
`simulator/bake_right_variants.py`. `setup_scales_media.py` makes the scale
|
||||
**ring** demonstrable: cheap synthetic placeholder bases for the two true-PD
|
||||
scales (`cosmos` = NASA/Hubble, `abyss` = NOAA Ocean Exploration) plus the
|
||||
per-edge zoom/warp transition clips — these scales carry a raw base only (no real
|
||||
Right variants yet; the baker can extend to them once their real footage is
|
||||
sourced). The `.mp4` binaries are gitignored.
|
||||
|
||||
**Run it (Docker):**
|
||||
|
||||
|
||||
@@ -35,10 +35,14 @@ MEDIA = Path(__file__).parent / "sample_media"
|
||||
|
||||
RIGHT_LEVELS = (1, 2, 3, 4)
|
||||
|
||||
# Keyframe img2img strength per Right level (by-eye ramp). Level 4 ~ the POC's
|
||||
# proven painterly look (key-strength 0.5-0.58); level 1 is a barely-there dream
|
||||
# haze. Tune by eye in the sim, then re-bake.
|
||||
_KEY_STRENGTH = {1: 0.28, 2: 0.38, 3: 0.48, 4: 0.58}
|
||||
# Keyframe img2img strength per Right level (by-eye ramp). sd-turbo's painterly
|
||||
# transformation is sharply NON-LINEAR in strength — it barely registers below
|
||||
# ~0.5 and ramps hard through ~0.5-0.65 (verified by eye, session 0012: a linear
|
||||
# 0.28-0.58 ramp left levels 1-3 ~indistinguishable from raw). So the curve is
|
||||
# clustered in that active band for real perceptual progression: a gentle dream
|
||||
# haze (L1) -> clearly painterly (L3, ~the POC's proven look) -> strongest (L4).
|
||||
# Still by-eye tunable — re-run the baker after editing.
|
||||
_KEY_STRENGTH = {1: 0.46, 2: 0.52, 3: 0.58, 4: 0.64}
|
||||
_TWEEN_RATIO = 0.36 # POC ratio (0.18 / 0.5): tweens are a light refine only.
|
||||
|
||||
# The restyle target (matches the POC spike that the operator approved).
|
||||
|
||||
@@ -25,9 +25,9 @@
|
||||
"license": "poc-sample (look-tuning only; not shipped content)",
|
||||
"source": "hef-poc/out/neutral.mp4",
|
||||
"right_variants": {
|
||||
"1": {"file": "forest/right1.mp4", "model": "placeholder"},
|
||||
"2": {"file": "forest/right2.mp4", "model": "placeholder"},
|
||||
"3": {"file": "forest/right3.mp4", "model": "placeholder"},
|
||||
"1": {"file": "forest/right1.mp4", "model": "sd-turbo+farneback-flow"},
|
||||
"2": {"file": "forest/right2.mp4", "model": "sd-turbo+farneback-flow"},
|
||||
"3": {"file": "forest/right3.mp4", "model": "sd-turbo+farneback-flow"},
|
||||
"4": {"file": "forest/right4.mp4", "model": "sd-turbo+farneback-flow"}
|
||||
},
|
||||
"annotations": [
|
||||
|
||||
@@ -1,52 +1,33 @@
|
||||
"""Populate simulator/sample_media/forest/ from the session-0008 POC artifacts.
|
||||
"""Stage the forest scale's neutral BASE clip from the session-0008 POC artifacts.
|
||||
|
||||
Copies the real neutral base + the real flow-stabilized Right restyle out of
|
||||
~/hef-poc/out/, and generates placeholder intermediate Right strengths (1..3) by
|
||||
blending the base toward the real restyle with ffmpeg. The media binaries are
|
||||
gitignored; only the manifest is committed. Sample footage is for look-tuning
|
||||
only, not shipped content.
|
||||
Copies the real neutral Yosemite base out of ~/hef-poc/out/ into
|
||||
simulator/sample_media/forest/. The real multi-strength Right variants are then
|
||||
produced by the offline baker (`simulator/bake_right_variants.py`) — this script
|
||||
no longer generates placeholder Right strengths, so re-running it never clobbers
|
||||
a real bake. The media binaries are gitignored; only the manifest is committed.
|
||||
Sample footage is for look-tuning only, not shipped content.
|
||||
|
||||
Usage: python simulator/setup_sample_media.py
|
||||
Requires: ffmpeg on PATH (or `pip install imageio-ffmpeg`), and ~/hef-poc/out/.
|
||||
Usage:
|
||||
python simulator/setup_sample_media.py # stage the base
|
||||
python -m simulator.bake_right_variants --scale forest # then bake variants
|
||||
Requires: ~/hef-poc/out/neutral.mp4 (the POC neutral base).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
POC = Path.home() / "hef-poc" / "out"
|
||||
DEST = Path(__file__).parent / "sample_media" / "forest"
|
||||
|
||||
|
||||
def _ffmpeg() -> str:
|
||||
if shutil.which("ffmpeg"):
|
||||
return "ffmpeg"
|
||||
import imageio_ffmpeg
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
DEST.mkdir(parents=True, exist_ok=True)
|
||||
base = DEST / "base.mp4"
|
||||
right4 = DEST / "right4.mp4"
|
||||
shutil.copyfile(POC / "neutral.mp4", base)
|
||||
shutil.copyfile(POC / "right_flow.mp4", right4)
|
||||
ff = _ffmpeg()
|
||||
# Placeholder strengths 1..3: opacity-blend base toward the real restyle.
|
||||
for strength, alpha in ((1, 0.25), (2, 0.5), (3, 0.75)):
|
||||
out = DEST / f"right{strength}.mp4"
|
||||
subprocess.run(
|
||||
[ff, "-y", "-i", str(base), "-i", str(right4),
|
||||
"-filter_complex",
|
||||
f"[1:v]format=yuva444p,colorchannelmixer=aa={alpha}[top];"
|
||||
f"[0:v][top]overlay=shortest=1[v]",
|
||||
"-map", "[v]", "-an", str(out)],
|
||||
check=True,
|
||||
)
|
||||
print(f"generated {out.name} (alpha {alpha})")
|
||||
print(f"sample media ready in {DEST}")
|
||||
print(f"staged {base} (real POC neutral base)")
|
||||
print("next: python -m simulator.bake_right_variants --scale forest")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user