diff --git a/docs/superpowers/specs/2026-06-04-ingest-tagging-review-tools.md b/docs/superpowers/specs/2026-06-04-ingest-tagging-review-tools.md new file mode 100644 index 0000000..f940769 --- /dev/null +++ b/docs/superpowers/specs/2026-06-04-ingest-tagging-review-tools.md @@ -0,0 +1,535 @@ +# Sub-project 2 — Ingest & Tagging / Review Tools — Spec + +**Date:** 2026-06-04 +**Status:** Proposed (sub-project 2 of 5) — pending operator approval +**Repo:** `human-experience-filter-art` +**Design reference:** [`2026-06-04-human-experience-filter-design.md`](./2026-06-04-human-experience-filter-design.md) (esp. §4 catalog, §5 division-of-labor, §8 sourcing) +**Roadmap item:** [`docs/ROADMAP.md`](../../ROADMAP.md) §2 +**Builds on:** sub-project 1 — `hef.catalog` + `hef.selection` (done, merged, 37 tests) + +--- + +## 1. What this is + +Sub-project 1 gave us the **spine**: a validated JSONL catalog (`hef.catalog`) and +a nearest-match selector (`hef.selection`). Today the only way to fill that +catalog is hand-authoring records (per [`docs/USER_GUIDE.md`](../../USER_GUIDE.md)). + +Sub-project 2 builds the **assisted *draft-then-review* pipeline** the design spec +calls for (§5), under `tools/`: + +1. **Ingest** — per-archive fetchers pull a candidate piece from a public-domain + pool and write a catalog record. +2. **Mechanical tagging** — tooling auto-fills the non-curatorial fields: `mode` + (via `ffprobe`), `license`/`attribution`/`source_*` (from origin), + `duration_s`, `resolution`, and `dominant_color` (computed from the video). +3. **Coordinate drafting** — a proposer suggests `left/right/dark/light` with a + one-line `rationale`, written as `review_status: proposed`. +4. **Review CLI** — walks `proposed` records one at a time (proposed coords + + rationale + preview frame), accepts or corrects, flips to `approved`, and + stamps `reviewed_at`. + +The output is a populated catalog that `load_catalog` validates and the player +(sub-project 3) can `select()` over. "How far from done" stays exactly what §4 +defines: the count of records still in `review_status: proposed`. + +### Design invariants this honors + +- **Curation is the artwork** (design §1). The four coordinates are a human + curatorial act (design §11: *no automatic ML coordinate tagging*). The pipeline + **drafts** coordinates as a starting point, but every coordinate is human-blessed + before it can be selected. Drafting is deterministic and rule-based — there is no + ML model in the tool. +- **The catalog is the spine.** Tools only **read** and **append/rewrite** through + `hef.catalog`; they never define a parallel record shape. +- **Local, dependency-light, hermetically testable.** Like sub-project 1, the + testable core is pure-stdlib Python. The two external boundaries — `ffmpeg`/`ffprobe` + (subprocess) and archive HTTP — are isolated behind injectable seams so the unit + suite never shells out or hits the network. + +--- + +## 2. What `hef.catalog` already gives us (and why the schema needs no change) + +The headline of the "build on without breaking" constraint: **the `Record` schema +is already complete for this flow.** Sub-project 1 shipped, with defaults chosen +for exactly the draft-then-review pipeline: + +| Field group | Fields | Who sets it here | +|---|---|---| +| Identity / origin | `id`, `title`, `source_url`, `source_archive` | ingest (from the fetcher) | +| License | `license`, `attribution` | mechanical tagging (from origin) | +| Mechanical media | `mode`, `duration_s`, `resolution`, `dominant_color` | mechanical tagging (ffprobe/ffmpeg) | +| Curatorial | `left`, `right`, `dark`, `light` | drafting (proposed) → review (approved) | +| Review state | `review_status` (default `"proposed"`), `rationale`, `reviewed_at` (default `null`) | drafting sets `rationale`; review sets `reviewed_at` | +| Misc | `file_path`, `notes` | ingest (`file_path`); either (`notes`) | + +`Record`'s defaults — `review_status="proposed"`, `reviewed_at=None`, +`rationale=""`, plus `attribution`/`resolution`/`dominant_color` defaulting to +`""` — are precisely an ingested-but-unreviewed record. **Ingest constructs a +`Record` and the proposed-state shape falls out of the defaults; nothing in the +schema is added or changed.** + +The tools consume these existing symbols unchanged: + +- `hef.catalog`: `Record`, `validate`, `record_to_dict`, `record_from_dict`, + `load_catalog`, `save_catalog`, `append_record`, `CatalogError`, and the + vocab constants `MODES`, `LICENSES`, `REVIEW_STATUSES`, `ATTRIBUTION_LICENSES`, + `COORD_FIELDS`, `COORD_MIN`, `COORD_MAX`. +- `hef.selection`: `Coordinate`, `select` (used by the end-to-end test and by + the player later). + +--- + +## 3. The one required addition to `hef.catalog` + +Per the constraint, every addition the tools need is called out explicitly. There +is exactly **one** required change to the shared library, and it is purely +additive (no existing symbol changes signature or behavior): + +### 3.1 `validate_catalog(records)` — catalog-level integrity (unique ids) + +`validate()` is per-record; it cannot see the whole file, so it cannot catch a +**duplicate `id`**. The ingest pipeline appends repeatedly and the review CLI +looks records up *by id* — both are unsafe if ids collide (ingest silently +double-adds a re-fetched piece; review can't tell two records apart). So add a +catalog-level check to the shared spine, where the player also benefits: + +```python +def validate_catalog(records) -> None: + """Validate every record AND the cross-record invariants (currently: unique ids). + Raises CatalogError naming the first duplicate id.""" +``` + +- It calls `validate()` on each record, then asserts `id` uniqueness. +- **Non-breaking:** new symbol; existing `validate`/`load_catalog`/`save_catalog`/ + `append_record` are untouched. Any catalog valid today (unique ids) stays valid. +- **Used by tools** at two points: before `append_record` during ingest (reject a + candidate whose derived `id` already exists), and after `load_catalog` in the + review CLI (fail loudly on a corrupted/duplicated catalog before editing it). + +> A companion convenience, `index_by_id(records) -> dict[str, Record]`, may be +> added alongside it (the review CLI and dedupe both want id-keyed lookup). It is +> a trivial helper; if we prefer to keep the spine minimal it can live in the +> tools layer instead. **Recommendation:** add `index_by_id` to `hef.catalog` +> too, since by-id lookup is a catalog concern the player will also want. + +**Everything else this sub-project needs lives in `tools/` and only *consumes* +`hef.catalog`.** No other change to `hef/` is required. (Considered and +deliberately *not* added now — see §11: a `source_id` archive-identifier field, +a `media_sha256`, an `ingested_at` timestamp. They are future schema extensions, +not needed for the done-criteria.) + +--- + +## 4. Package layout + +Extends the existing repo (design §10; the sub-project-1 plan put shared code in +`hef/` and data in `catalog/`): + +``` +tools/ + __init__.py + http.py # thin stdlib-urllib client: timeout, retry, user-agent (injectable) + probe.py # Prober: ffprobe wrapper -> parsed streams (injectable) + mediatools.py # ffmpeg helpers: extract frame, dominant color, waveform thumb + tagging.py # mechanical tagging: assemble mode/duration/resolution/dominant_color + licensing.py # per-archive license/attribution normalization -> LICENSES vocab + drafting.py # Draft, Proposer protocol, HeuristicProposer + ingest/ + __init__.py + base.py # Candidate dataclass + Fetcher protocol + the ingest pipeline + internet_archive.py + musopen.py + librivox.py + nasa.py + fma.py # Free Music Archive + freesound.py + ingest_cli.py # `python -m tools.ingest_cli` — run a fetcher, write proposed records + review.py # review state-transition core (pure, tested) + review_cli.py # `python -m tools.review_cli` — interactive walk of proposed records +``` + +- `tools/` is added to `[tool.setuptools] packages` in `pyproject.toml`. +- **Invocation matches the repo convention** (USER_GUIDE: run from repo root, no + install, `import hef` resolves via the root `conftest.py`): the CLIs run as + `.venv/bin/python -m tools.ingest_cli …` and `… -m tools.review_cli …`. + `[project.scripts]` console-script aliases (`hef-ingest`, `hef-review`) may be + declared too for an installed checkout, but `python -m` is the documented path. +- `media/` (download target, §6.3) is added to `.gitignore`. Media never enters + git — the repo holds metadata + pointers only (design §4). + +--- + +## 5. Mechanical tagging (`tools/probe.py`, `tools/mediatools.py`, `tools/tagging.py`, `tools/licensing.py`) + +All of §5's "mechanical / automatic" fields, set from the downloaded file and the +origin metadata. + +### 5.1 `mode`, `duration_s`, `resolution` — from `ffprobe` + +`tools/probe.py` wraps: + +``` +ffprobe -v quiet -print_format json -show_format -show_streams +``` + +and parses the JSON into a small `Probe` result (streams + format). `tools/tagging.py` +derives: + +- **`mode`** — from which *meaningful* streams exist: + - audio stream present, no real video → `"audio"` + - real video stream present, no audio → `"video"` + - both → `"av"` + - **Cover-art guard:** an embedded cover image in an audio file shows up as a + video stream (`mjpeg`/`png`, `disposition.attached_pic == 1`, often 1 frame). + These are **excluded** when deciding `mode`, so an MP3 with album art is + correctly `"audio"`, not `"av"`. This guard is a tested case. +- **`duration_s`** — `round(float(format.duration))`, falling back to the longest + stream `duration` if `format.duration` is absent. Integer (schema requires int ≥ 0). +- **`resolution`** — `"{width}x{height}"` from the chosen video stream; `""` for + audio-only (schema default). + +`tools/probe.py` takes the subprocess runner as an injectable dependency +(default: `subprocess.run`); tests feed canned `ffprobe` JSON and never shell out. + +### 5.2 `dominant_color` — computed from the video (`tools/mediatools.py`) + +For `video`/`av` records, extract a representative frame mid-segment and reduce it +to one hex color. **Primary approach is `ffmpeg`-only** (keeps the dependency +surface to just `ffmpeg`/`ffprobe` — no image library): + +``` +# representative frame near the middle, reduced to a single-entry palette +ffmpeg -v quiet -ss -i -vf "thumbnail,palettegen=max_colors=1" -frames:v 1 -f rawvideo -pix_fmt rgb24 - +``` + +The 3 output bytes are the dominant palette color → `"#rrggbb"`. (A simpler +`scale=1:1` average-color path is the fallback if `palettegen` is unavailable.) +Audio-only records keep `dominant_color=""`. + +> **Open decision (logged):** the roadmap assumed "an image lib for dominant +> color." This spec instead recommends the **ffmpeg-only** path so the whole +> sub-project depends on nothing beyond `ffmpeg`/`ffprobe`. **Pillow** (k-means / +> `quantize` over a sampled frame) is the documented fallback if ffmpeg-derived +> color quality proves insufficient in review. Chosen for the lean dependency +> story; reversible. + +`dominant_color` feeds the side walls (sub-project 5, design §7). + +### 5.3 `license` / `attribution` / `source_*` — from origin (`tools/licensing.py`) + +Each fetcher resolves origin metadata; `tools/licensing.py` normalizes it to the +`hef.catalog` vocab (`LICENSES = {public_domain, cc0, cc_by, cc_by_nc}`): + +- Map CC license URLs/identifiers → `cc0` | `cc_by` | `cc_by_nc`; explicit + public-domain / "no known copyright" → `public_domain`. +- For `cc_by`/`cc_by_nc` (the `ATTRIBUTION_LICENSES`), build the required + `attribution` string (creator + license name/URL) — `validate()` already rejects + these licenses with empty `attribution`, so the normalizer **must** produce it. +- Anything that doesn't map to an allowed license is **rejected at ingest** with a + clear error (the piece is not defensible per design §8's license stance) — it is + never written as an invalid record. + +This is the per-archive "license/attribution/source from origin" tagging; it is +unit-tested with sample metadata payloads per archive. + +--- + +## 6. Ingest (`tools/ingest/`) + +### 6.1 The `Candidate` + `Fetcher` seam + +Every archive differs in API but funnels into one shape. `tools/ingest/base.py`: + +```python +@dataclass +class Candidate: + source_archive: str # origin label, e.g. internet_archive | librivox | nasa (§6.4) + source_url: str # human/landing URL recorded in the record + media_url: str # direct download URL of the chosen file + title: str + license: str # normalized (tools.licensing) -> hef LICENSES vocab + attribution: str # "" unless the license requires it + suggested_id: str # stable id derived from archive + archive-identifier + media_ext: str # file extension for the download target + description: str = "" # free text used by drafting signals; lands in notes +``` + +```python +class Fetcher(Protocol): + archive: str # the source_archive label + def search(self, query: str, *, limit: int) -> list[Candidate]: ... + def resolve(self, identifier: str) -> Candidate: ... # one item by id/URL +``` + +Fetchers receive an HTTP client (`tools/http.py`) by injection so tests feed canned +API responses. Fetchers do **not** download or probe — they only resolve metadata. + +### 6.2 The ingest pipeline + +`tools/ingest/base.py` provides `ingest_candidate(candidate, *, catalog_path, media_root, proposer, prober, downloader)`: + +1. **Dedupe.** `load_catalog` + `validate_catalog`; if `candidate.suggested_id` + already exists, skip (log "already in catalog") — idempotent re-runs. +2. **Download.** Fetch `candidate.media_url` → `//.` + (see §6.3). Skip the download if the file already exists. +3. **Mechanical tag.** `prober` → `mode`, `duration_s`, `resolution`; `mediatools` + → `dominant_color` (video/av only). `licensing` already normalized + `license`/`attribution` on the `Candidate`. +4. **Draft coordinates.** `proposer.propose(signals)` → `Draft(coordinate, rationale)` (§7). +5. **Build the record.** Construct a `hef.catalog.Record` from the above; the + `proposed`/`reviewed_at=None` shape comes from the defaults (§2). `notes` gets + `candidate.description`. +6. **Validate + append.** `validate(record)`, then re-check uniqueness, then + `append_record(record, catalog_path)`. + +Batch helper `ingest_search(fetcher, query, *, limit, …)` runs steps over each +search hit. Everything network/subprocess is injected, so the pipeline is tested +end-to-end with a fake fetcher + fake prober + fake downloader (no I/O). + +### 6.3 Download / caching layout (settles a roadmap open decision) + +- **Media root:** `--media-root DIR` / env `HEF_MEDIA_ROOT`, default `./media/` + (gitignored). +- **On-disk path:** `//.`. +- **`file_path` recorded in the record:** the path **relative to the media root** + (e.g. `nasa/nasa-as08-14-2383.mp4`). Relative-to-root is portable: the ingest + workstation and the Pi's mounted drive store the same tree under different + mounts, and the **player (sub-project 3) joins `file_path` with its own drive + mount**. `validate()` does not constrain `file_path` format, so this is + compatible with the hand-authoring guide's absolute-path examples. + +> **Open decision (logged):** relative-to-media-root vs. absolute `file_path`. This +> spec recommends **relative-to-root** for portability between the tagging +> workstation and the Pi. The USER_GUIDE currently shows absolute paths for +> hand-authoring; both load fine. A doc note will reconcile them; the player spec +> will define mount resolution. Reversible. + +### 6.4 The six archives (all specified; phased delivery) + +License normalization and access per pool (design §8 maps these to the axes): + +| Archive (`source_archive`) | Access | Typical license | Auth | Notes / risk | +|---|---|---|---|---| +| `internet_archive` (incl. Prelinger) | Metadata API `archive.org/metadata/`; direct file URLs | `public_domain` / CC (per item `licenseurl`/`rights`) | none | Ambiguous "no known copyright" → `public_domain`, flagged in `notes` for review | +| `librivox` | JSON API `librivox.org/api/feed/audiobooks` | `public_domain` (charter) | none | Reader credited in `notes`; attribution not required | +| `nasa` | Images/video API `images-api.nasa.gov` | `public_domain` | none | NASA media guideline caveat: some items embed third-party content — flag in `notes` | +| `musopen` | Musopen API / catalog | `public_domain` / CC | possible key | Access historically gated — may need an API key (secret, §9) | +| `fma` (Free Music Archive) | Track/page resolution | CC (`cc_by`/`cc_by_nc`/`cc0`) | varies | **Highest risk:** the public FMA API has been deprecated/changed; fetcher resolves from a track URL + page metadata | +| `freesound` | API v2 `freesound.org/apiv2` | `cc0` / `cc_by` / `cc_by_nc` | **token required** | API token is a secret (§9); uploader → `attribution` when license requires | + +> **Recommended first-ship set:** the keyless, clean-license, stable-API pools — +> **LibriVox, NASA, Internet Archive/Prelinger.** Defer the ones with an auth or +> API-stability tax — **Freesound** (token), **Musopen** (possible gate), +> **Free Music Archive** (deprecated API) — to a second pass. The `Fetcher` seam +> means deferring them costs nothing structurally; the spec still defines all six. +> *(Logged as an open decision for operator confirmation — see §11.)* + +--- + +## 7. Coordinate drafting (`tools/drafting.py`) + +The curatorial coordinates are a human act (design §11). The tool's job is to seed +a **draft** to be reviewed, never to auto-tag authoritatively. + +```python +@dataclass +class Draft: + coordinate: Coordinate # hef.selection.Coordinate(left, right, dark, light) + rationale: str # one line, cites the signal that drove the guess + +@dataclass +class Signals: + title: str + description: str + source_archive: str + mode: str + duration_s: int + +class Proposer(Protocol): + def propose(self, signals: Signals) -> Draft: ... +``` + +### 7.1 `HeuristicProposer` (the deterministic baseline) + +A rule-based seed grounded in the §8 sourcing→axis map and the §2 rubric — **not +an ML model** (honors design §11): + +- **Archive priors** seed the brain plane: `librivox` → high `left` + (spoken/verbal); `internet_archive`/Prelinger → high `left` (educational/ + industrial) by default; `musopen`/`fma` → high `right` (music); `nasa` → high + `right` (wordless awe); `freesound` → high `right` (abstract field recordings). +- **Keyword nudges** on title+description seed the mood plane: dark words + (`storm`, `night`, `noir`, `requiem`, `minor`, `war`, `funeral`, `decay`, …) + raise `dark`; light words (`sunrise`, `dawn`, `garden`, `spring`, `joy`, `hope`, + `major`, `bright`, …) raise `light`. +- Values **clamped to 0..4**. +- `rationale` is one line naming the dominant signal, e.g. + `"librivox spoken reading → strong left; 'storm' in title → dark"`. + +Deterministic given `Signals` → fully unit-testable. + +### 7.2 The assistant seam + +`Proposer` is an interface. The `HeuristicProposer` is always-available and +dependency-free. When a **session assistant** (e.g. Claude, or the operator) does +the ingest, it can supply a considered coordinate + rationale through the *same* +seam (a manual proposer that reads a per-candidate draft, or an LLM-backed +proposer the operator wires in). Either way the record is written +`review_status="proposed"` and is **still reviewed by a human**. The tool ships +**no** ML dependency; the "assistant" is an optional, out-of-core seam. + +--- + +## 8. Review CLI (`tools/review.py` + `tools/review_cli.py`) + +### 8.1 The transition core (pure, tested) — `tools/review.py` + +```python +def proposed_records(records) -> list[Record]: + """Records still awaiting review.""" + +def approve(record, *, reviewed_at, coordinate=None) -> Record: + """Return an approved copy: review_status='approved', reviewed_at set, + optionally overriding the four coordinates with a human correction. + The returned record is re-validated by the caller before save.""" +``` + +- `reviewed_at` is an ISO-8601 UTC timestamp string + (`datetime.now(timezone.utc).isoformat()`), injected by the CLI so the core stays + pure/testable. `Record.reviewed_at` is `Optional[str]` — no schema change. +- `approve` does not mutate in place gratuitously; it produces the approved record + and the CLI persists via load → replace-by-id → `save_catalog` (rewrite). At + 120–800 records a full rewrite is trivial and crash-resilient when done after + each approval. + +### 8.2 The interactive walk — `tools/review_cli.py` + +`python -m tools.review_cli --catalog catalog/library.jsonl [--media-root ./media]` + +1. `load_catalog` → `validate_catalog` (fail loudly on a corrupt catalog before editing). +2. For each `proposed` record, show: + - `id`, `title`, `source_archive`, `source_url`, `license`/`attribution` + - `mode`, `duration_s`, `resolution`, `dominant_color` + - the **proposed coordinates** `left/right/dark/light` and the one-line `rationale` + - a **preview frame**: for `video`/`av`, extract a representative frame (reusing + the §5.2 mid-segment frame) to a temp PNG and open it with the OS viewer + (`open` on macOS / `xdg-open` on Linux); for `audio`, render a waveform + thumbnail (`ffmpeg … showwavespic`) and/or offer optional 10-second playback + via `ffplay`. The preview resolves `file_path` against `--media-root`. +3. Prompt: **[a]ccept · [e]dit coords · [s]kip · [q]uit**. + - **accept** → `approve(record, reviewed_at=now)` (keep coords). + - **edit** → prompt for new `left/right/dark/light` (and optionally a new + `rationale`), then `approve(..., coordinate=corrected)`. + - **skip** → leave `proposed`, advance. + - **quit** → save and exit. +4. Persist after each approval (rewrite via `save_catalog`), so an interrupted + session keeps its progress. + +The preview/prompt shell is thin; the **tested** logic is the §8.1 transition core +(deterministic, no terminal, no subprocess). Preview rendering reuses +`tools/mediatools.py` and is exercised only by opt-in integration tests. + +--- + +## 9. Dependencies, configuration, secrets + +- **System binaries:** `ffmpeg` + `ffprobe` (subprocess). Required for mechanical + tagging, dominant color, and previews. The dominant-color recommendation (§5.2) + keeps the dep surface to *just* these two — no Python image library in the core. + Pillow is an optional documented fallback only. +- **HTTP:** stdlib `urllib.request` via `tools/http.py` (timeout, small retry, + user-agent) — no `requests` dependency, preserving the sub-project-1 ethos. + `internetarchive`/other archive SDKs are explicitly avoided in favor of the + documented JSON APIs. +- **Config:** `--catalog` (default `catalog/library.jsonl`), `--media-root` / + `HEF_MEDIA_ROOT` (default `./media/`, gitignored). +- **Secrets (wgl hard rule — never in catalog, code, or transcript):** Freesound + API token via `FREESOUND_API_TOKEN` (env or macOS Keychain); any Musopen/FMA key + likewise. Referenced by name only; never written into a record or logged. +- **Testing:** `pytest` (already a dep). The unit suite stays **hermetic** — + `ffprobe`/`ffmpeg`/HTTP are injected and faked; no network, no binaries needed to + run the core tests. Integration tests that actually invoke `ffprobe`/`ffmpeg` are + opt-in and **skipped** when the binaries are absent. +- **pyproject:** add `tools` to `[tool.setuptools] packages`; optionally add + `[project.scripts]` (`hef-ingest`, `hef-review`) and a `[project.optional-dependencies]` + `tools` extra (empty unless Pillow is later adopted). + +--- + +## 10. Testing strategy & done-criteria + +Mirrors sub-project 1's TDD discipline (test-first, stdlib, hermetic). + +**Unit tests (hermetic):** + +1. `hef.catalog.validate_catalog` — accepts unique ids; raises naming a duplicate; + (`index_by_id` round-trip if added). +2. **Mechanical tagging** — canned `ffprobe` JSON → expected `mode`/`duration_s`/ + `resolution`; the **cover-art `attached_pic` guard** keeps an art-bearing MP3 at + `mode="audio"`. +3. **Dominant color** — canned ffmpeg `rgb24` bytes → expected `"#rrggbb"`; + audio-only → `""`. +4. **License normalization** — sample per-archive metadata → correct `license`; + `cc_by`/`cc_by_nc` produce non-empty `attribution`; unmappable license is + rejected. +5. **Fetchers** — canned API JSON via a fake HTTP client → `Candidate` fields and + stable `suggested_id` derivation (one test per archive that ships). +6. **Drafting** — `HeuristicProposer`: archive priors, keyword nudges, 0..4 + clamping, one-line rationale content. +7. **Review transition** — `proposed → approved` sets `reviewed_at` + status; edit + overrides coordinates; skip leaves `proposed`; result re-validates and + round-trips through `save_catalog`/`load_catalog`. +8. **End-to-end (mocked)** — fake fetcher + fake prober + fake downloader + + `HeuristicProposer` → a `proposed` record appended; the review core approves it; + `load_catalog` + `validate_catalog` pass; `select(..., approved_only=True)` can + pick it. + +**Opt-in integration tests** (skipped without binaries): real `ffprobe` on a tiny +fixture clip → `mode`/`duration`/`resolution`; real dominant-color extraction. + +**Done when (from ROADMAP §2):** + +- You can run ingest against a source and get `proposed` records. +- You can review them to `approved`. +- `load_catalog` (and `validate_catalog`) validate the result. +- Unit tests on mechanical tagging and the review state transitions pass, and the + full suite is green from the repo root inside `.venv`. + +--- + +## 11. Open decisions (recommended defaults; logged for operator) + +None block writing the implementation plan; each has a recommended default the +plan will assume unless the operator overrides. + +1. **Dominant-color dependency** — *Recommend:* ffmpeg-only (`palettegen`), no + image lib; Pillow as fallback. (Roadmap had assumed an image lib.) +2. **`file_path` style** — *Recommend:* relative-to-`media-root` for portability; + reconcile the USER_GUIDE's absolute examples with a doc note; player spec + defines mount resolution. +3. **Which archives ship first** — *Recommend:* LibriVox, NASA, Internet + Archive/Prelinger first; defer Freesound, Musopen, FMA (auth / API-stability tax). +4. **CLI surface** — *Recommend:* two `python -m` entry points (`tools.ingest_cli`, + `tools.review_cli`), matching the no-install repo convention; console-script + aliases optional. +5. **`index_by_id` placement** — *Recommend:* add it to `hef.catalog` alongside + `validate_catalog` (the player wants by-id lookup too). + +**Deliberately deferred (YAGNI, no schema change now):** a `source_id` +archive-identifier field, `media_sha256` integrity, an `ingested_at` timestamp, and +any LLM-backed proposer in the core. All are future, additive extensions; none are +needed for the done-criteria. + +--- + +## 12. Relationship to the other sub-projects + +- **Consumes** sub-project 1 (`hef.catalog`, `hef.selection`) — the only required + change is the additive `validate_catalog` (+ optional `index_by_id`), §3. +- **Feeds** sub-project 3 (the Pi player): a populated, `approved` catalog with + `dominant_color` ready for sub-project 5's side walls. The player resolving + `file_path` against its drive mount (§6.3) is a player-spec concern. +- **Settles** two roadmap open decisions (download/caching layout §6.3; ingest + language = Python under `tools/`, §4).