Remote controller simulator — drive the experience from an iPad web twin over wifi #27

New Issue

2026-06-27T13:16:34Z

benstull commented

2026-06-27 13:16:34 +00:00

Remote controller simulator — drive the experience from an iPad web twin over wifi

Summary

As the operator, I can drive the live experience (altitude, left, right, mood,
content) from a separate touch device (e.g. an iPad) over wifi while the
laptop renders — so I can stand back and feel the installation, and prototype the
controller in software before any hardware exists.

Problem / pain

Today you must sit at the rendering machine and move on-screen sliders. You can't
experience the piece from a distance, you can't feel what a handheld panel will be
like, and you have no way to validate the control ergonomics before committing
to hardware — so any physical-panel decision is made blind. The result is wasted
hardware iteration and an experience that can only be operated from the one spot
nobody actually stands.

Who feels it

persona	role	what they need
Operator / artist	tunes and runs the piece	to drive the experience from a handheld device, away from the render machine
Participant (proxy)	future in-room participant	an early read on how a handheld controller feels driving the piece
Hardware builder	builds the physical panel next	a liked, faithful software twin that is the spec for the hardware

Desired outcome / value

A touch remote on a separate device drives the live experience over wifi. It
is a faithful twin of the planned physical panel, so what is liked on the iPad
becomes the specification for the hardware — no redesign later — and the control
feel is validated in software, honoring simulator-first.

Acceptance — "done right"

Open the remote page on a separate device on the same wifi; turning Altitude /
Left / Right / Mood / Content changes the projected experience live.
Transitional controls (Altitude, Content) show honest busy/settling
feedback while their change plays; live controls (Left/Right/Mood) respond
immediately.
The renderer holds no input UI — it only renders what the controller drives.
Works over wifi/LAN with no install (a web page on the device).
A second control source (now or a future panel) speaking the same contract drives
the same experience with no renderer change.

Scope / non-goals

In scope: the design spec's v1 — server-authoritative session state + the
networked control endpoints; the renderer refactored to display-only + live
updates; the touch remote page with the five controls and per-knob transition
state; wifi/LAN.

Non-goals: any Arduino/firmware; Bluetooth/USB transport; access control/auth;
moving rendering off the laptop. (All belong to the parent Epic's later Features.)

Assumptions · constraints · dependencies

Parent: the Networked control surface Epic (#26).
Design: docs/superpowers/specs/2026-06-26-networked-control-surface-design.md
is the binding engineering design (architecture, contract, the four endpoints,
the transition model, testing).
Reconcile the control inventory with the then-current simulator surface at
build time — the content/mode controls have evolved since the spec (session
0020's separate Video/Audio toggles); the implementation plan should fold that
in rather than treat the spec's content selector as literal.
Honors simulator-first; the laptop renders (no Pi).

Source / signal

Capture session human-experience-filter-art-0025; design spec
docs/superpowers/specs/2026-06-26-networked-control-surface-design.md (session
0019 brainstorming).

Priority (WSJF)

Value: high (it is the installation-feel payoff and the hardware spec) ·
Time-criticality: low (deferred with the Epic) · Opportunity: high (first child,
unblocks the panel) ÷ Size: medium-large → provisional priority/P2.

Decomposition

A single increment (one SPEC = one plan = one execution). A recommended
implementation Task breakdown is in Solution notes; the binding breakdown is the
implementation plan's Tasks.

Solution notes (optional — NOT a design, non-binding)

Recommended Task shape (per the design spec): (1) server SessionState store +
the control endpoints; (2) the live push channel to the renderer; (3) renderer
refactor to display-only + reconnect/resync; (4) the touch remote page (five
controls + per-knob transition state); (5) the renderer's settled-ack +
transition-timeout safety net; (6) end-to-end browser tests across the two pages.
The binding detail is in the design spec.

# Remote controller simulator — drive the experience from an iPad web twin over wifi ## Summary As the operator, I can drive the live experience (altitude, left, right, mood, content) from a **separate touch device** (e.g. an iPad) over wifi while the laptop renders — so I can stand back and feel the installation, and prototype the controller in software before any hardware exists. ## Problem / pain Today you must sit at the rendering machine and move on-screen sliders. You can't experience the piece from a distance, you can't feel what a handheld panel will be like, and you have **no way to validate the control ergonomics before committing to hardware** — so any physical-panel decision is made blind. The result is wasted hardware iteration and an experience that can only be operated from the one spot nobody actually stands. ## Who feels it | persona | role | what they need | | --- | --- | --- | | Operator / artist | tunes and runs the piece | to drive the experience from a handheld device, away from the render machine | | Participant (proxy) | future in-room participant | an early read on how a handheld controller feels driving the piece | | Hardware builder | builds the physical panel next | a liked, faithful software twin that *is* the spec for the hardware | ## Desired outcome / value A **touch remote on a separate device** drives the live experience over wifi. It is a **faithful twin of the planned physical panel**, so what is liked on the iPad becomes the specification for the hardware — no redesign later — and the control feel is validated in software, honoring simulator-first. ## Acceptance — "done right" - Open the remote page on a separate device on the same wifi; turning **Altitude / Left / Right / Mood / Content** changes the projected experience **live**. - **Transitional controls** (Altitude, Content) show honest **busy/settling feedback** while their change plays; **live controls** (Left/Right/Mood) respond immediately. - The **renderer holds no input UI** — it only renders what the controller drives. - Works over **wifi/LAN with no install** (a web page on the device). - A second control source (now or a future panel) speaking the same contract drives the same experience with no renderer change. ## Scope / non-goals **In scope:** the design spec's **v1** — server-authoritative session state + the networked control endpoints; the renderer refactored to display-only + live updates; the touch remote page with the five controls and per-knob transition state; wifi/LAN. **Non-goals:** any Arduino/firmware; Bluetooth/USB transport; access control/auth; moving rendering off the laptop. (All belong to the parent Epic's later Features.) ## Assumptions · constraints · dependencies - **Parent:** the Networked control surface Epic (#26). - **Design:** `docs/superpowers/specs/2026-06-26-networked-control-surface-design.md` is the binding engineering design (architecture, contract, the four endpoints, the transition model, testing). - **Reconcile the control inventory** with the then-current simulator surface at build time — the content/mode controls have evolved since the spec (session 0020's separate Video/Audio toggles); the implementation plan should fold that in rather than treat the spec's content selector as literal. - Honors **simulator-first**; the laptop renders (no Pi). ## Source / signal Capture session `human-experience-filter-art-0025`; design spec `docs/superpowers/specs/2026-06-26-networked-control-surface-design.md` (session 0019 brainstorming). ## Priority (WSJF) Value: high (it is the installation-feel payoff and the hardware spec) · Time-criticality: low (deferred with the Epic) · Opportunity: high (first child, unblocks the panel) ÷ Size: medium-large → **provisional priority/P2**. ## Decomposition A single increment (one SPEC = one plan = one execution). A recommended implementation Task breakdown is in Solution notes; the binding breakdown is the implementation plan's Tasks. ## Solution notes (optional — NOT a design, non-binding) Recommended Task shape (per the design spec): (1) server `SessionState` store + the control endpoints; (2) the live push channel to the renderer; (3) renderer refactor to display-only + reconnect/resync; (4) the touch remote page (five controls + per-knob transition state); (5) the renderer's settled-ack + transition-timeout safety net; (6) end-to-end browser tests across the two pages. The binding detail is in the design spec.

benstull added the type/feature priority/P2 labels 2026-06-27 13:16:34 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: benstull/human-experience-filter-art#27