# Human Experience Filter — User Guide

> **Scope (current build state).** Two pieces are built: the catalog + selection
> core, and the **`tools/` ingest & review pipeline** (sub-project 2). You can
> populate the catalog two ways — **hand-author records** (below) or **assisted
> ingest** that fetches from public-domain archives, mechanically tags, drafts
> coordinates, and lets you review them to `approved` (see *Ingesting & reviewing
> media*). The room **player** is a separate, not-yet-built sub-project.

---

## What "configuring media" means

The installation plays media chosen by where a viewer sets the control knobs.
Every piece of media is one **record** in a catalog file. A record says *what the
file is*, *where it lives on the drive*, *its license*, and — the important part —
*its coordinate* in the experience-space. At run time the player will pick the
record nearest the knob position; for now, you build and validate that catalog.

The catalog is a single file: **`catalog/library.jsonl`**. It is
[JSON Lines](https://jsonlines.org/) — **one JSON object per line**, one line per
piece of media. Blank lines are ignored.

---

## Prerequisites

The commands in this guide need **Python 3.11+** and must be run **from the repo
root**, where `import hef` resolves with no install step. Examples use
`.venv/bin/python`; if you made the project virtualenv they work as-is, otherwise
substitute your own `python3`.

Creating the virtualenv is optional but recommended:

```bash
python3 -m venv .venv
```

---

## The record schema

Each line is a JSON object. **Twelve fields are required**; the rest have
defaults and may be omitted.

### Required fields

| Field            | Type   | Meaning                                                        |
|------------------|--------|----------------------------------------------------------------|
| `id`             | string | Stable unique id (non-empty). Your choice; keep it unique.      |
| `title`          | string | Human title.                                                   |
| `source_url`     | string | Where the piece came from.                                     |
| `source_archive` | string | Origin label, e.g. `internet_archive`, `musopen`, `librivox`.  |
| `license`        | string | One of: `public_domain`, `cc0`, `cc_by`, `cc_by_nc`.           |
| `mode`           | string | One of: `audio`, `video`, `av`.                                |
| `left`           | int    | Analytical / verbal axis, **0–4** (see rubric).                |
| `right`          | int    | Artistic / abstract axis, **0–4**.                             |
| `dark`           | int    | Somber / heavy axis, **0–4**.                                  |
| `light`          | int    | Uplifting / serene axis, **0–4**.                              |
| `duration_s`     | int    | Segment length in seconds (≥ 0).                               |
| `file_path`      | string | Path to the media file on the player's drive.                  |

### Optional fields (defaults shown)

| Field            | Default      | Meaning                                                   |
|------------------|--------------|------------------------------------------------------------|
| `review_status`  | `"proposed"` | `proposed` or `approved` — whether a human has blessed it.  |
| `attribution`    | `""`         | **Required text when `license` is `cc_by` or `cc_by_nc`.** |
| `resolution`     | `""`         | e.g. `1920x1080` (video).                                  |
| `dominant_color` | `""`         | Hex color. **Optional / opt-in** — computed only with `--dominant-color`. |
| `rationale`      | `""`         | One-line note on why you chose the coordinate.            |
| `reviewed_at`    | `null`       | Timestamp when approved.                                  |
| `notes`          | `""`         | Free text.                                                |

### Validation rules (enforced when you load/save)

- `id` must be non-empty.
- `mode`, `license`, `review_status` must be one of the allowed values above.
- `left`, `right`, `dark`, `light` must be **integers** in `0..4` (booleans are
  rejected).
- `duration_s` must be `≥ 0`.
- `cc_by` and `cc_by_nc` require a non-empty `attribution`.
- Unknown fields and missing required fields are rejected.

> The file referenced by `file_path` is **not** checked for existence yet — only
> the record's structure is validated.

---

## The coordinate rubric

The four coordinates are the curatorial heart of the work. Each is a felt
judgment on a **0–4** scale. The two pairs are **independent knobs**, not opposite
ends of one slider — a piece can be high on both `left` and `right` ("whole
brain"), or high on both `dark` and `light` ("bittersweet").

| Axis    | 0 means…            | 4 means…                                                    |
|---------|---------------------|-------------------------------------------------------------|
| `left`  | not analytical      | strongly analytical / verbal / structured — narration, language, logic, sequence, instruction, documentary, spoken word |
| `right` | not artistic        | strongly artistic / emotional / abstract — music, abstract visuals, dance, nature, awe, non-verbal, dreamlike |
| `dark`  | not somber          | strongly somber / heavy — ominous, melancholic, night, decay, minor-key, storms, noir |
| `light` | not light           | strongly uplifting / serene — joyful, hopeful, bright, sunrise, gardens, major-key, calm |

`mode` describes the media's channels: `audio` (sound only), `video` (image
only), `av` (both). There is no `none` record — "None" is a control state the
player handles by going dark, not a kind of media.

**License stance:** prefer `public_domain` / `cc0`; `cc_by` is fine if you record
`attribution`; `cc_by_nc` only while the installation stays non-commercial.

---

## Adding media — two ways

### Option A: edit the JSONL file by hand

Append one line per piece to `catalog/library.jsonl`. A complete example record
(formatted across lines here for readability — **it must be a single line in the
file**):

```json
{"id": "nasa-earthrise", "title": "Earthrise", "source_url": "https://archive.org/details/earthrise", "source_archive": "internet_archive", "license": "public_domain", "mode": "video", "left": 0, "right": 4, "dark": 1, "light": 3, "duration_s": 600, "file_path": "/media/nasa-earthrise.mp4", "resolution": "1920x1080", "rationale": "wordless awe, bright Earth on black — strongly right-brain, gently light"}
```

### Option B: use the Python API (validates before writing)

```bash
.venv/bin/python -c '
from hef.catalog import Record, append_record
r = Record(
    id="librivox-meditations",
    title="Meditations, Book II (excerpt)",
    source_url="https://librivox.org/meditations/",
    source_archive="librivox",
    license="public_domain",
    mode="audio",
    left=4, right=1, dark=2, light=2,
    duration_s=720,
    file_path="/media/librivox-meditations-ii.mp3",
    rationale="spoken philosophy, verbal and reflective — strongly left-brain",
)
append_record(r, "catalog/library.jsonl")
print("added", r.id)
'
```

`append_record` validates the record and raises `CatalogError` (printing what is
wrong) before it writes, so a bad record never lands in the file.

### Option C: assisted ingest (fetch + tag + draft, then review)

See the next section.

---

## Ingesting & reviewing media

Instead of hand-authoring every record, the `tools/` pipeline can fetch a piece
from a public-domain archive, fill in the mechanical fields for you, **draft** a
coordinate, and write the record as `review_status: proposed`. You then walk the
proposed records and bless each one to `approved`. The four coordinates are still
a human act — the tool only *drafts* a starting point; nothing is selectable until
you approve it.

### Prerequisites

- **Python 3.11+**, run from the repo root (as everywhere in this guide).
- **`ffmpeg` and `ffprobe`** on your `PATH` — used to read media properties
  (`mode`/`duration_s`/`resolution`), render review previews, and (opt-in)
  compute `dominant_color`. Install via your package manager (e.g.
  `brew install ffmpeg`). The unit tests don't need them; the live ingest/review
  do.

### Where downloaded media goes (`--media-root`)

Ingest downloads each file to `<media-root>/<archive>/<id>.<ext>` and records a
`file_path` **relative to the media root** (e.g. `nasa/nasa-earthrise.mp4`).
Relative paths are portable: the tagging workstation and the Pi's drive store the
same tree under different mounts, and the player joins `file_path` with its own
mount. The media root is `--media-root DIR` (or `HEF_MEDIA_ROOT`), default
`./media/`, which is gitignored — **media never enters the repo**, only metadata.

> **Note on `file_path` style.** Hand-authored records (Option A/B above) often use
> absolute paths like `/media/earthrise.mp4`; ingested records use archive-relative
> paths. Both load and validate fine (`validate()` does not constrain the format).
> Keep a catalog internally consistent where you can; the player spec will define
> how mounts are resolved.

### First-ship archives

Three keyless, clean-license archives are wired today:

| `archive` arg      | Pool                                  | License           |
|--------------------|---------------------------------------|-------------------|
| `librivox`         | Public-domain audiobook recordings    | `public_domain`   |
| `nasa`             | NASA imagery / video                  | `public_domain`   |
| `internet_archive` | Internet Archive / Prelinger          | per item (PD / CC)|

`musopen`, `fma`, and `freesound` are defined but **deferred** (auth or
API-stability tax) — invoking them exits with a "deferred" message. **Freesound**
will need an API token supplied via the `FREESOUND_API_TOKEN` environment variable
(a secret — never put it on the command line or into a record).

### Running ingest

```bash
.venv/bin/python -m tools.ingest_cli nasa --query "earthrise" --limit 5
.venv/bin/python -m tools.ingest_cli internet_archive --resolve prelinger_blast
.venv/bin/python -m tools.ingest_cli librivox --query "meditations" --media-root ./media
```

Flags: `--query` (search) or `--resolve <identifier>` (one item); `--limit N`;
`--catalog` (default `catalog/library.jsonl`); `--media-root` (default `./media`,
or `HEF_MEDIA_ROOT`); `--dominant-color` to compute `dominant_color` for
video/`av` records (off by default — its only consumer, the procedural side walls,
was dropped in the single-panoramic-projector design change). Re-running is
idempotent: a candidate whose id already exists is skipped.

### Reviewing proposed records

```bash
.venv/bin/python -m tools.review_cli --catalog catalog/library.jsonl --media-root ./media
```

For each `proposed` record it prints the id/title/source/license, the mechanical
fields, the **drafted coordinates + rationale**, and opens a preview frame
(video/`av`) or waveform image (audio). Then it prompts:

- **`a`** — accept the drafted coordinates and mark `approved`.
- **`e`** — edit `left/right/dark/light` (enter blank to keep a value), then approve.
- **`s`** — skip; leave it `proposed`.
- **`q`** — save and quit.

Each approval stamps `reviewed_at` and is written immediately (full rewrite), so
an interrupted session keeps its progress. **How far from done** is just the count
of records still `proposed`. Add `--no-preview` to skip frame rendering.

---

## Validating the whole catalog

Loading the catalog validates **every** record. Run this after editing by hand:

```bash
.venv/bin/python -c "from hef.catalog import load_catalog; print(len(load_catalog('catalog/library.jsonl')), 'records OK')"
```

- Prints e.g. `12 records OK` if everything is valid.
- Raises `CatalogError` describing the problem if not. A validation error names
  the field and reason, e.g. `coordinate left=9 out of range 0..4`; malformed
  JSON additionally names the line, e.g. `line 4: invalid JSON: ...`.

---

## Sanity-checking selection (optional)

To feel how tagging drives playback, you can ask the selection core which record
a given knob position would pick. Coordinates are `Coordinate(left, right, dark,
light)`; the second argument is the content-mode selector
(`audio` / `video` / `av` / `none`).

```bash
.venv/bin/python -c "
from hef.catalog import load_catalog
from hef.selection import Coordinate, select
lib = load_catalog('catalog/library.jsonl')
pick = select(lib, Coordinate(left=0, right=4, dark=1, light=3), 'video')
print(pick.id if pick else 'nothing')
"
```

Notes that shape how you tag:

- Selection is **nearest-match** — every knob position resolves to the closest
  record, so a sparse catalog still works; you do not need to fill every
  coordinate.
- `select(..., 'none')` always returns nothing (the void state).
- Pass `approved_only=True` to restrict to records whose `review_status` is
  `approved`.

---

## Where the media files go

Records point at files via `file_path`. Those files live on the player's drive,
**not** in this git repo (the repo holds only the catalog metadata). Keep your
`file_path` values consistent with wherever you mount that drive on the machine
that will eventually run the room.

## Playing with the simulator (alteration preview)

The simulator is a web stand-in for the installation's control panel. It runs the
real `player.alteration` engine and **alters** a neutral base clip toward the knob
state in the browser, so you can tune the *look* of the filter before any hardware
exists. (The earlier selection-era "curator's X-ray" view was retired when the
piece moved from *selecting* clips to *altering* them.)

**One-time setup — populate the sample footage** (look-tuning only; not shipped
content). The ring uses real strict-PD footage in a **rotating pool** per scale
(`docs/content-candidate-pool.md`); regenerate the manifest + new transition
placeholders with:

    python simulator/build_pool_manifest.py --media     # write the pool manifest + coast-edge transitions

`build_pool_manifest.py` holds the hand-authored label/affect content and emits
`simulator/sample_media/manifest.json` (the 19-clip pool baseline). The pool clips
themselves are sourced via the content pipeline (`tools/pipeline/run.py
process_clip`); the `.mp4` binaries are gitignored. (`setup_scales_media.py` is the
older one-base-per-scale generator, now superseded.)

**Run it (Docker):**

    make sim

then open http://localhost:8000.

**Run it (no Docker):**

    pip install -e ".[sim]"
    make sim-local

**What you see and can do:**

- **Content dial** — picks audio/video channel; "off" and audio-only positions go
  to black walls.
- **Scale ring (endless encoder)** — `⊖ out` / `in ⊕` (or scroll the stage) walk a
  *closed ring* of neutral "scales of nature" — **cosmos → orbit → coast → reef →
  abyss** and back around (diving past the smallest **wraps** to the largest). Each
  scale is a **rotating pool** of vetted clips: landing on a scale plays a random
  pool member, so the ring feels fresh each pass. It is *relative* (an endless
  encoder), distinct from the absolute 0–4 knobs: each step plays a short
  placeholder zoom/warp **transition**, then settles, with the current alteration
  still applied. The current scale + chosen member is named beside the buttons
  (`scale · member (i/N · pool N)`). A **fast spin** (≥3 detents at once) collapses
  to one quick **blended pass** instead of grinding through every transition.
- **Three experience knobs:**
  - **Mood (−4 dark .. 0 .. +4 light)** — a live runtime color grade (cool/dark ↔
    warm/bright; 0 = the raw footage).
  - **Right (dreamlike, 0–4)** — a **deterministic real-time painterly dream** (a
    WebGL Kuwahara restyle of the live frames; holds still across the loop, goes
    trippy at max). It also drives **progressive emotion tiers**: when both knobs
    are up, the affect words escalate from basic to compound as Right rises.
  - **Left (analytical, 0–4)** — a live HUD of labelled boxes from the clip's
    authored annotation track. Labels use **progressive detail tiers**: low Left
    shows fewer, more general labels (high-salience objects only); raising Left
    brings in more objects *and* escalates each label general → specific →
    scientific → +fact. Time-windowed tracked labels appear only while their
    subject is on screen and follow it.
- **RenderPlan readout** — always shows the exact numbers the engine produced (the
  project's honesty "X-ray," over the alteration model).

The base clips, the Left annotation tracks (with tiers + appear/disappear
windows), affect anchors, and per-tier string tables all come from
`simulator/sample_media/manifest.json`.

### Authoring labels — the author mode

Open **http://localhost:8000/author.html** to author a clip's labels by eye
(content-pipeline §11.5). Pick a pool clip, **scrub** to where a subject is on
screen, **drag a box** over it, set the label key + salience + the four detail
tiers, then **Run tracker** to propagate the box into a keyframed track (classical
OpenCV optical flow) with an appear/disappear window — or **Add as static box**
for a fixed label. Shift-click the stage to place an **affect anchor** and type its
emotion tiers. **Save to manifest** writes the entry back via the pipeline's
idempotent upsert (keeping the clip's media + provenance, replacing only the
authored content). The tracker only propagates **box geometry** — every label
string, scientific name, and fact is **hand-authored**, because a generic detector
can't produce "*Gymnothorax*" or a lifespan.
