# Human Experience Filter — User Guide > **Scope (current build state).** Two pieces are built: the catalog + selection > core, and the **`tools/` ingest & review pipeline** (sub-project 2). You can > populate the catalog two ways — **hand-author records** (below) or **assisted > ingest** that fetches from public-domain archives, mechanically tags, drafts > coordinates, and lets you review them to `approved` (see *Ingesting & reviewing > media*). The room **player** is a separate, not-yet-built sub-project. --- ## What "configuring media" means The installation plays media chosen by where a viewer sets the control knobs. Every piece of media is one **record** in a catalog file. A record says *what the file is*, *where it lives on the drive*, *its license*, and — the important part — *its coordinate* in the experience-space. At run time the player will pick the record nearest the knob position; for now, you build and validate that catalog. The catalog is a single file: **`catalog/library.jsonl`**. It is [JSON Lines](https://jsonlines.org/) — **one JSON object per line**, one line per piece of media. Blank lines are ignored. --- ## Prerequisites The commands in this guide need **Python 3.11+** and must be run **from the repo root**, where `import hef` resolves with no install step. Examples use `.venv/bin/python`; if you made the project virtualenv they work as-is, otherwise substitute your own `python3`. Creating the virtualenv is optional but recommended: ```bash python3 -m venv .venv ``` --- ## The record schema Each line is a JSON object. **Twelve fields are required**; the rest have defaults and may be omitted. ### Required fields | Field | Type | Meaning | |------------------|--------|----------------------------------------------------------------| | `id` | string | Stable unique id (non-empty). Your choice; keep it unique. | | `title` | string | Human title. | | `source_url` | string | Where the piece came from. | | `source_archive` | string | Origin label, e.g. `internet_archive`, `musopen`, `librivox`. | | `license` | string | One of: `public_domain`, `cc0`, `cc_by`, `cc_by_nc`. | | `mode` | string | One of: `audio`, `video`, `av`. | | `left` | int | Analytical / verbal axis, **0–4** (see rubric). | | `right` | int | Artistic / abstract axis, **0–4**. | | `dark` | int | Somber / heavy axis, **0–4**. | | `light` | int | Uplifting / serene axis, **0–4**. | | `duration_s` | int | Segment length in seconds (≥ 0). | | `file_path` | string | Path to the media file on the player's drive. | ### Optional fields (defaults shown) | Field | Default | Meaning | |------------------|--------------|------------------------------------------------------------| | `review_status` | `"proposed"` | `proposed` or `approved` — whether a human has blessed it. | | `attribution` | `""` | **Required text when `license` is `cc_by` or `cc_by_nc`.** | | `resolution` | `""` | e.g. `1920x1080` (video). | | `dominant_color` | `""` | Hex color. **Optional / opt-in** — computed only with `--dominant-color`. | | `rationale` | `""` | One-line note on why you chose the coordinate. | | `reviewed_at` | `null` | Timestamp when approved. | | `notes` | `""` | Free text. | ### Validation rules (enforced when you load/save) - `id` must be non-empty. - `mode`, `license`, `review_status` must be one of the allowed values above. - `left`, `right`, `dark`, `light` must be **integers** in `0..4` (booleans are rejected). - `duration_s` must be `≥ 0`. - `cc_by` and `cc_by_nc` require a non-empty `attribution`. - Unknown fields and missing required fields are rejected. > The file referenced by `file_path` is **not** checked for existence yet — only > the record's structure is validated. --- ## The coordinate rubric The four coordinates are the curatorial heart of the work. Each is a felt judgment on a **0–4** scale. The two pairs are **independent knobs**, not opposite ends of one slider — a piece can be high on both `left` and `right` ("whole brain"), or high on both `dark` and `light` ("bittersweet"). | Axis | 0 means… | 4 means… | |---------|---------------------|-------------------------------------------------------------| | `left` | not analytical | strongly analytical / verbal / structured — narration, language, logic, sequence, instruction, documentary, spoken word | | `right` | not artistic | strongly artistic / emotional / abstract — music, abstract visuals, dance, nature, awe, non-verbal, dreamlike | | `dark` | not somber | strongly somber / heavy — ominous, melancholic, night, decay, minor-key, storms, noir | | `light` | not light | strongly uplifting / serene — joyful, hopeful, bright, sunrise, gardens, major-key, calm | `mode` describes the media's channels: `audio` (sound only), `video` (image only), `av` (both). There is no `none` record — "None" is a control state the player handles by going dark, not a kind of media. **License stance:** prefer `public_domain` / `cc0`; `cc_by` is fine if you record `attribution`; `cc_by_nc` only while the installation stays non-commercial. --- ## Adding media — two ways ### Option A: edit the JSONL file by hand Append one line per piece to `catalog/library.jsonl`. A complete example record (formatted across lines here for readability — **it must be a single line in the file**): ```json {"id": "nasa-earthrise", "title": "Earthrise", "source_url": "https://archive.org/details/earthrise", "source_archive": "internet_archive", "license": "public_domain", "mode": "video", "left": 0, "right": 4, "dark": 1, "light": 3, "duration_s": 600, "file_path": "/media/nasa-earthrise.mp4", "resolution": "1920x1080", "rationale": "wordless awe, bright Earth on black — strongly right-brain, gently light"} ``` ### Option B: use the Python API (validates before writing) ```bash .venv/bin/python -c ' from hef.catalog import Record, append_record r = Record( id="librivox-meditations", title="Meditations, Book II (excerpt)", source_url="https://librivox.org/meditations/", source_archive="librivox", license="public_domain", mode="audio", left=4, right=1, dark=2, light=2, duration_s=720, file_path="/media/librivox-meditations-ii.mp3", rationale="spoken philosophy, verbal and reflective — strongly left-brain", ) append_record(r, "catalog/library.jsonl") print("added", r.id) ' ``` `append_record` validates the record and raises `CatalogError` (printing what is wrong) before it writes, so a bad record never lands in the file. ### Option C: assisted ingest (fetch + tag + draft, then review) See the next section. --- ## Ingesting & reviewing media Instead of hand-authoring every record, the `tools/` pipeline can fetch a piece from a public-domain archive, fill in the mechanical fields for you, **draft** a coordinate, and write the record as `review_status: proposed`. You then walk the proposed records and bless each one to `approved`. The four coordinates are still a human act — the tool only *drafts* a starting point; nothing is selectable until you approve it. ### Prerequisites - **Python 3.11+**, run from the repo root (as everywhere in this guide). - **`ffmpeg` and `ffprobe`** on your `PATH` — used to read media properties (`mode`/`duration_s`/`resolution`), render review previews, and (opt-in) compute `dominant_color`. Install via your package manager (e.g. `brew install ffmpeg`). The unit tests don't need them; the live ingest/review do. ### Where downloaded media goes (`--media-root`) Ingest downloads each file to `//.` and records a `file_path` **relative to the media root** (e.g. `nasa/nasa-earthrise.mp4`). Relative paths are portable: the tagging workstation and the Pi's drive store the same tree under different mounts, and the player joins `file_path` with its own mount. The media root is `--media-root DIR` (or `HEF_MEDIA_ROOT`), default `./media/`, which is gitignored — **media never enters the repo**, only metadata. > **Note on `file_path` style.** Hand-authored records (Option A/B above) often use > absolute paths like `/media/earthrise.mp4`; ingested records use archive-relative > paths. Both load and validate fine (`validate()` does not constrain the format). > Keep a catalog internally consistent where you can; the player spec will define > how mounts are resolved. ### First-ship archives Three keyless, clean-license archives are wired today: | `archive` arg | Pool | License | |--------------------|---------------------------------------|-------------------| | `librivox` | Public-domain audiobook recordings | `public_domain` | | `nasa` | NASA imagery / video | `public_domain` | | `internet_archive` | Internet Archive / Prelinger | per item (PD / CC)| `musopen`, `fma`, and `freesound` are defined but **deferred** (auth or API-stability tax) — invoking them exits with a "deferred" message. **Freesound** will need an API token supplied via the `FREESOUND_API_TOKEN` environment variable (a secret — never put it on the command line or into a record). ### Running ingest ```bash .venv/bin/python -m tools.ingest_cli nasa --query "earthrise" --limit 5 .venv/bin/python -m tools.ingest_cli internet_archive --resolve prelinger_blast .venv/bin/python -m tools.ingest_cli librivox --query "meditations" --media-root ./media ``` Flags: `--query` (search) or `--resolve ` (one item); `--limit N`; `--catalog` (default `catalog/library.jsonl`); `--media-root` (default `./media`, or `HEF_MEDIA_ROOT`); `--dominant-color` to compute `dominant_color` for video/`av` records (off by default — its only consumer, the procedural side walls, was dropped in the single-panoramic-projector design change). Re-running is idempotent: a candidate whose id already exists is skipped. ### Reviewing proposed records ```bash .venv/bin/python -m tools.review_cli --catalog catalog/library.jsonl --media-root ./media ``` For each `proposed` record it prints the id/title/source/license, the mechanical fields, the **drafted coordinates + rationale**, and opens a preview frame (video/`av`) or waveform image (audio). Then it prompts: - **`a`** — accept the drafted coordinates and mark `approved`. - **`e`** — edit `left/right/dark/light` (enter blank to keep a value), then approve. - **`s`** — skip; leave it `proposed`. - **`q`** — save and quit. Each approval stamps `reviewed_at` and is written immediately (full rewrite), so an interrupted session keeps its progress. **How far from done** is just the count of records still `proposed`. Add `--no-preview` to skip frame rendering. --- ## Validating the whole catalog Loading the catalog validates **every** record. Run this after editing by hand: ```bash .venv/bin/python -c "from hef.catalog import load_catalog; print(len(load_catalog('catalog/library.jsonl')), 'records OK')" ``` - Prints e.g. `12 records OK` if everything is valid. - Raises `CatalogError` describing the problem if not. A validation error names the field and reason, e.g. `coordinate left=9 out of range 0..4`; malformed JSON additionally names the line, e.g. `line 4: invalid JSON: ...`. --- ## Sanity-checking selection (optional) To feel how tagging drives playback, you can ask the selection core which record a given knob position would pick. Coordinates are `Coordinate(left, right, dark, light)`; the second argument is the content-mode selector (`audio` / `video` / `av` / `none`). ```bash .venv/bin/python -c " from hef.catalog import load_catalog from hef.selection import Coordinate, select lib = load_catalog('catalog/library.jsonl') pick = select(lib, Coordinate(left=0, right=4, dark=1, light=3), 'video') print(pick.id if pick else 'nothing') " ``` Notes that shape how you tag: - Selection is **nearest-match** — every knob position resolves to the closest record, so a sparse catalog still works; you do not need to fill every coordinate. - `select(..., 'none')` always returns nothing (the void state). - Pass `approved_only=True` to restrict to records whose `review_status` is `approved`. --- ## Where the media files go Records point at files via `file_path`. Those files live on the player's drive, **not** in this git repo (the repo holds only the catalog metadata). Keep your `file_path` values consistent with wherever you mount that drive on the machine that will eventually run the room. ## Playing with the simulator (alteration preview) The simulator is a web stand-in for the installation's control panel. It runs the real `player.alteration` engine and **alters** a neutral base clip toward the knob state in the browser, so you can tune the *look* of the filter before any hardware exists. (The earlier selection-era "curator's X-ray" view was retired when the piece moved from *selecting* clips to *altering* them.) **One-time setup — populate the sample footage** (look-tuning only; not shipped content): python simulator/setup_sample_media.py # stage the forest neutral base python -m simulator.bake_right_variants --scale forest # real Right variants (SD on MPS, ~11 min) python simulator/setup_scales_media.py # the cosmos + abyss scales + ring transitions `setup_sample_media.py` stages the forest neutral base from the session-0008 POC artifacts (`~/hef-poc/out/neutral.mp4`). `bake_right_variants.py` then produces the **real** flow-stabilized painterly Right variants (strengths 1–4) for the forest scale — SD img2img keyframes + Farneback optical-flow tweens on Apple MPS, the temporally-calm POC algorithm, productionized (scales design §1/§4). The keyframe-strength ramp per Right level is by-eye tunable in `simulator/bake_right_variants.py`. `setup_scales_media.py` makes the scale **ring** demonstrable: cheap synthetic placeholder bases for the two true-PD scales (`cosmos` = NASA/Hubble, `abyss` = NOAA Ocean Exploration) plus the per-edge zoom/warp transition clips — these scales carry a raw base only (no real Right variants yet; the baker can extend to them once their real footage is sourced). The `.mp4` binaries are gitignored. **Run it (Docker):** make sim then open http://localhost:8000. **Run it (no Docker):** pip install -e ".[sim]" make sim-local **What you see and can do:** - **Content dial** — picks audio/video channel; "off" and audio-only positions go to black walls. - **Scale ring (endless encoder)** — `⊖ out` / `in ⊕` (or scroll the stage) walk a *closed ring* of neutral "scales of nature" clips — cosmos → forest → abyss and back around (diving past the smallest **wraps** to the largest). It is *relative* (an endless encoder), distinct from the absolute 0–4 knobs: each step plays a short placeholder zoom/warp **transition**, then settles on the next scale, with the current knob alteration still applied. The current scale is named beside the buttons (`name (i/N)`). A **fast spin** (scroll several detents at once, ≥3) collapses to a single quick **blended pass** straight to the destination scale instead of grinding through every transition (scales design §3); slow single steps still chain one full transition per scale crossed. - **Four experience knobs (0–4):** - **Dark / Light** — a live runtime color grade (cool/dark ↔ warm/bright; equal or zero = the raw footage). - **Right (dreamlike)** — selects a discrete pre-baked, flow-stabilized restyle variant and crossfades to it (strength 0 = raw base). - **Left (analytical)** — a live overlay: labelled boxes from the clip's authored annotation track, with more annotations appearing at higher levels. Text is shaped live (the simulator analogue of the Pi's Pango/HarfBuzz path). - **Calibration sliders** — adjust the grade/overlay gain curves live; once a look is liked, bake the values into `DEFAULT_CALIBRATION` in `player/alteration.py`. - **RenderPlan readout** — always shows the exact numbers the engine produced (the project's honesty "X-ray," now over the alteration model). The base clips, Right variants, Left annotation track, and string tables come from `simulator/sample_media/manifest.json`.