docs: user guide for ingest + review tools
Per sub-project-2 plan Task 14. Updates the scope banner (catalog core + tools now built), adds an 'Ingesting & reviewing media' section (ffmpeg prereqs, --media-root, first-ship archives, ingest_cli/review_cli usage, Freesound token note, opt-in dominant_color), and reconciles the absolute-vs-relative file_path note (spec §6.3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ben Stull
+95
-6
@@ -1,10 +1,11 @@
|
||||
# Human Experience Filter — User Guide
|
||||
|
||||
> **Scope (current build state).** Only the catalog + selection core is built so
|
||||
> far. This guide covers the one thing you can do today: **configure media by
|
||||
> hand-authoring and validating catalog records.** The automated ingest tool,
|
||||
> the tagging/review CLI, and the room player are separate, not-yet-built
|
||||
> sub-projects and are not covered here.
|
||||
> **Scope (current build state).** Two pieces are built: the catalog + selection
|
||||
> core, and the **`tools/` ingest & review pipeline** (sub-project 2). You can
|
||||
> populate the catalog two ways — **hand-author records** (below) or **assisted
|
||||
> ingest** that fetches from public-domain archives, mechanically tags, drafts
|
||||
> coordinates, and lets you review them to `approved` (see *Ingesting & reviewing
|
||||
> media*). The room **player** is a separate, not-yet-built sub-project.
|
||||
|
||||
---
|
||||
|
||||
@@ -66,7 +67,7 @@ defaults and may be omitted.
|
||||
| `review_status` | `"proposed"` | `proposed` or `approved` — whether a human has blessed it. |
|
||||
| `attribution` | `""` | **Required text when `license` is `cc_by` or `cc_by_nc`.** |
|
||||
| `resolution` | `""` | e.g. `1920x1080` (video). |
|
||||
| `dominant_color` | `""` | Hex color, used by the side walls later. |
|
||||
| `dominant_color` | `""` | Hex color. **Optional / opt-in** — computed only with `--dominant-color`. |
|
||||
| `rationale` | `""` | One-line note on why you chose the coordinate. |
|
||||
| `reviewed_at` | `null` | Timestamp when approved. |
|
||||
| `notes` | `""` | Free text. |
|
||||
@@ -146,6 +147,94 @@ print("added", r.id)
|
||||
`append_record` validates the record and raises `CatalogError` (printing what is
|
||||
wrong) before it writes, so a bad record never lands in the file.
|
||||
|
||||
### Option C: assisted ingest (fetch + tag + draft, then review)
|
||||
|
||||
See the next section.
|
||||
|
||||
---
|
||||
|
||||
## Ingesting & reviewing media
|
||||
|
||||
Instead of hand-authoring every record, the `tools/` pipeline can fetch a piece
|
||||
from a public-domain archive, fill in the mechanical fields for you, **draft** a
|
||||
coordinate, and write the record as `review_status: proposed`. You then walk the
|
||||
proposed records and bless each one to `approved`. The four coordinates are still
|
||||
a human act — the tool only *drafts* a starting point; nothing is selectable until
|
||||
you approve it.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **Python 3.11+**, run from the repo root (as everywhere in this guide).
|
||||
- **`ffmpeg` and `ffprobe`** on your `PATH` — used to read media properties
|
||||
(`mode`/`duration_s`/`resolution`), render review previews, and (opt-in)
|
||||
compute `dominant_color`. Install via your package manager (e.g.
|
||||
`brew install ffmpeg`). The unit tests don't need them; the live ingest/review
|
||||
do.
|
||||
|
||||
### Where downloaded media goes (`--media-root`)
|
||||
|
||||
Ingest downloads each file to `<media-root>/<archive>/<id>.<ext>` and records a
|
||||
`file_path` **relative to the media root** (e.g. `nasa/nasa-earthrise.mp4`).
|
||||
Relative paths are portable: the tagging workstation and the Pi's drive store the
|
||||
same tree under different mounts, and the player joins `file_path` with its own
|
||||
mount. The media root is `--media-root DIR` (or `HEF_MEDIA_ROOT`), default
|
||||
`./media/`, which is gitignored — **media never enters the repo**, only metadata.
|
||||
|
||||
> **Note on `file_path` style.** Hand-authored records (Option A/B above) often use
|
||||
> absolute paths like `/media/earthrise.mp4`; ingested records use archive-relative
|
||||
> paths. Both load and validate fine (`validate()` does not constrain the format).
|
||||
> Keep a catalog internally consistent where you can; the player spec will define
|
||||
> how mounts are resolved.
|
||||
|
||||
### First-ship archives
|
||||
|
||||
Three keyless, clean-license archives are wired today:
|
||||
|
||||
| `archive` arg | Pool | License |
|
||||
|--------------------|---------------------------------------|-------------------|
|
||||
| `librivox` | Public-domain audiobook recordings | `public_domain` |
|
||||
| `nasa` | NASA imagery / video | `public_domain` |
|
||||
| `internet_archive` | Internet Archive / Prelinger | per item (PD / CC)|
|
||||
|
||||
`musopen`, `fma`, and `freesound` are defined but **deferred** (auth or
|
||||
API-stability tax) — invoking them exits with a "deferred" message. **Freesound**
|
||||
will need an API token supplied via the `FREESOUND_API_TOKEN` environment variable
|
||||
(a secret — never put it on the command line or into a record).
|
||||
|
||||
### Running ingest
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m tools.ingest_cli nasa --query "earthrise" --limit 5
|
||||
.venv/bin/python -m tools.ingest_cli internet_archive --resolve prelinger_blast
|
||||
.venv/bin/python -m tools.ingest_cli librivox --query "meditations" --media-root ./media
|
||||
```
|
||||
|
||||
Flags: `--query` (search) or `--resolve <identifier>` (one item); `--limit N`;
|
||||
`--catalog` (default `catalog/library.jsonl`); `--media-root` (default `./media`,
|
||||
or `HEF_MEDIA_ROOT`); `--dominant-color` to compute `dominant_color` for
|
||||
video/`av` records (off by default — its only consumer, the procedural side walls,
|
||||
was dropped in the single-panoramic-projector design change). Re-running is
|
||||
idempotent: a candidate whose id already exists is skipped.
|
||||
|
||||
### Reviewing proposed records
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m tools.review_cli --catalog catalog/library.jsonl --media-root ./media
|
||||
```
|
||||
|
||||
For each `proposed` record it prints the id/title/source/license, the mechanical
|
||||
fields, the **drafted coordinates + rationale**, and opens a preview frame
|
||||
(video/`av`) or waveform image (audio). Then it prompts:
|
||||
|
||||
- **`a`** — accept the drafted coordinates and mark `approved`.
|
||||
- **`e`** — edit `left/right/dark/light` (enter blank to keep a value), then approve.
|
||||
- **`s`** — skip; leave it `proposed`.
|
||||
- **`q`** — save and quit.
|
||||
|
||||
Each approval stamps `reviewed_at` and is written immediately (full rewrite), so
|
||||
an interrupted session keeps its progress. **How far from done** is just the count
|
||||
of records still `proposed`. Add `--no-preview` to skip frame rendering.
|
||||
|
||||
---
|
||||
|
||||
## Validating the whole catalog
|
||||
|
||||
Reference in New Issue
Block a user