Implementation¶

Overview¶

A Python control program running on a Raspberry Pi that serves as the brain for an art installation. The program listens to MIDI media controllers and drives three types of outputs: smart WiZ bulbs, a NeoPixel strip, and a video screen. The number of controllers and the NeoPixel strip length are defined in the configuration.

Hardware setup¶

• Controller: Raspberry Pi (acts as a Wi-Fi hotspot)
• Inputs: MIDI media controllers connected through a hub (count from config)
• Outputs:
• Smart WiZ bulbs (Wi-Fi, connected directly to the Pi's hotspot)
• NeoPixel strip (length from config, one pixel per controller)
• Screen (connected to the Pi, plays video)

Inputs — MIDI controllers¶

• MIDI controllers (one per config entry), each identified by USB port path (see MIDI device mapping).
• Controllers send notes continuously while active; they do not send discrete note-on/note-off pairs in the traditional sense.
• The program listens for note-off events to determine when a note has ended.
• After a note-off, a configurable debounce timeout (e.g. 2 seconds) must elapse before transitioning to idle. If a new note-on arrives during this timeout, the controller returns to the active state immediately.
• SYSEX messages can optionally be logged for debugging and do not affect the state machine.

State machine (per controller)¶

Each controller independently maintains three states:

Idle¶

• No note is currently playing.
• Reached after a note-off event and the debounce timeout elapses with no new note-on.

Debounce¶

• A note has just ended; the timeout is running.
• If a new note-on arrives, the controller transitions back to active.
• When the timeout expires with no note-on, the controller transitions to idle.

Active¶

• A note is currently playing.
• Entered on note-on.
• If a new note-on arrives during debounce, the controller goes active again (resets to full brightness).

Output 1 — Smart WiZ bulbs¶

Configuration¶

• A config file maps each MIDI device to the IP address of its corresponding bulb.
• Each bulb is also assigned a color in the config.
• Optionally, bulb commands can be sent in a fire-and-forget way so that slow or missing bulb responses do not block the animation loop; otherwise the program waits for each bulb response.

Behavior¶

Brightness and timing¶

• Bulb active: max 255, min configurable (e.g. 25).
• Fade is stepped over a configurable duration and step count (e.g. 0.25 s, 12 steps).
• Idle pulse uses its own min/max and period.

Output 2 — NeoPixel strip¶

• A strip of NeoPixels (length from config), one pixel per controller.
• Each pixel position is mapped to a controller in the configuration (led_index).
• When color is disabled in settings, the strip shows white at the current brightness instead of the controller's configured color.

Output 3 — Video screen¶

The screen operates on an aggregate state derived from all controllers combined, not individual controller states.

Aggregate state logic¶

Notes¶

• The idle video is short and loops continuously as a waiting state.
• The active video plays to completion before returning to idle; it is not interrupted mid-playback.
• If controllers go idle and then all become active again while the active video is still playing, the active video is not restarted. It only triggers on a fresh idle-to-active transition after the current playback cycle completes.
• Implementation uses a single VLC instance (python-vlc) with two media (idle and active) and one player; on end-of-media the player switches back to idle.

Architecture¶

Single-process async with crash-isolated video¶

• All controllers, the LED strip, and the supervision loop run as asyncio tasks in a single process alongside the FastAPI server.
• The video player runs in a separate child process for crash isolation. VLC (libvlc) is a native C library with documented segfault issues during media switching and MediaPlayerEndReached callbacks. A segfault is an OS signal (SIGSEGV) that kills the process immediately — no Python exception, no try/except, no cleanup. Running VLC in a child process ensures a crash only kills that child; the supervisor detects it and restarts automatically.
• Controller tasks update a multiprocessing.Value (an atomic shared integer, 1 = all controllers active, 0 = not) on every state transition. The video process reads this value — no multiprocessing.Manager or coordinator task needed.
• All other shared state (state_dict, led_dict) is held in plain Python dicts — no IPC overhead for controllers and LED.

MIDI input¶

• Each controller opens its MIDI port with a callback via mido.open_input(port_name, callback=...). The rtmidi backend invokes the callback from its own thread the instant a message arrives.
• Messages are bridged into asyncio via loop.call_soon_threadsafe() and an asyncio.Queue. The consumer task wakes on await queue.get() — no polling, sub-millisecond reaction time.
• Port disconnect is detected by a background task that checks mido.get_input_names() every 2 seconds.

Time-based animations¶

• Fade and pulse brightness are computed from time.monotonic(), not from a step counter with fixed sleeps. Each animation loop asks "what should brightness be right now?" based on elapsed wall-clock time.
• This absorbs variable bulb command latency: if a UDP round-trip is slow, the next brightness value jumps forward to match the clock rather than running late.
• When sync is enabled, all idle controllers share a single pulse_reference_time (set once at supervisor start), so their pulses are always in phase. When sync is disabled, each controller independently varies its pulse duration between idle_pulse_min_duration and idle_pulse_max_duration using the selected variation mode. Fades remain independent (triggered by individual MIDI events).

LED updates decoupled from bulbs¶

• set_lights() writes the LED dict before awaiting bulb commands. LED brightness updates at the rate of the animation loop, not gated by the slowest WiZ bulb response.
• The LED task reads the dict and pushes SPI writes via asyncio.to_thread().

Resilience¶

• The application is resilient to disconnections of any device (controller or bulb).
• If a device disconnects, its task handles the failure without affecting other tasks.
• Devices are hot-pluggable: unplugging and replugging causes the supervisor to detect the change and start or restart the corresponding task.
• If a bulb becomes unreachable, the controller task logs errors and continues; no errors propagate to other tasks.
• The supervisor periodically rebuilds the USB-path-to-MIDI-port map and starts one controller task per config entry whose usb_path is present. Missing or unplugged devices are skipped; reconnecting a device causes its task to start on the next poll.

Supervisor¶

The Supervisor is the component that owns all task lifecycle and shared state. It is a single class (Supervisor in server.py) used by the FastAPI application to start, stop, or restart the installation. External systems can control the installation via the HTTP API.

Role¶

• Single owner of task and process lifecycle: Only the Supervisor starts and stops controller tasks, the LED task, and the video child process. It loads config.json, creates shared state (plain dicts + a shared Value for IPC), and runs the supervision loop that keeps tasks in sync with connected devices.
• Clean start/stop/restart: Each start() loads config from disk and creates fresh shared state. Each stop() cancels all asyncio tasks, terminates the video process, and clears all internal references. There is no long-lived global state outside the Supervisor instance.
• Safe for concurrent API calls: An internal asyncio.Lock serializes start(), stop(), and restart() so that concurrent HTTP requests cannot leave the system in an inconsistent state.

State held by the Supervisor¶

Lifecycle semantics¶

• start(): If already running, no-op. Otherwise: load config.json, initialize state_dict, led_dict, all_active Value, and pulse_reference_time, set _running = True, and start the supervision loop as an async task. The loop will start controller tasks, the LED task, and the video process as devices are detected.
• stop(): If not running, no-op. Otherwise: set _running = False, await the supervision loop task until it exits, cancel all asyncio tasks (controllers, LED), await them, terminate the video process, and clear all internal references. After stop(), the Supervisor holds no tasks, processes, or shared state.
• restart(): Equivalent to stop() then start(). Used when config has changed: the new start() reloads config.json and creates fresh state, so no stale or dangling state from the previous run remains.

Supervision loop¶

The supervision loop runs inside the main process as an asyncio task. Each iteration:

1. Builds the USB-path-to-MIDI-port map (via asyncio.to_thread() so the blocking sysfs/aconnect work does not block the event loop).
2. Starts or restarts controller tasks for each config entry whose usb_path is present; cancels tasks for disconnected devices.
3. Starts or restarts the video process and the LED task when at least one controller is running. If the video process died (e.g. VLC segfault), it is automatically restarted.
4. Sleeps ~5 seconds (50 × 0.1 s) using asyncio.sleep(), so the FastAPI server remains responsive.

Controllers and LED share the same event loop. The video player runs in a separate process for crash isolation.

Integration with FastAPI¶

• The FastAPI app is created with a lifespan context manager. On startup, if the app was not started with --paused, it calls supervisor.start(). On shutdown (e.g. SIGTERM), it always calls supervisor.stop(), so all tasks and processes are cleaned up when the server exits.
• The CLI (uv run server.py) supports --paused: when set, the server starts without calling supervisor.start(). An external system can then call POST /start when the installation should run. This allows the API to be up and queryable (GET /status) while the installation tasks are stopped.
• All task control is done by calling Supervisor methods; the API endpoints (POST /start, POST /stop, POST /restart) delegate directly to them. Future APIs that modify config.json can apply changes and then call supervisor.restart() so the new config is loaded and any previous state is cleared.

MIDI device mapping¶

Identical USB MIDI devices (e.g. five Instruo Scions) share the same product name and often the same serial number, so they cannot be identified by name or serial alone. The program uses the USB port path instead (e.g. 1-1.4 on a hub, 3-1 on the host). As long as each controller stays in the same physical port, the mapping is stable across reboots.

The mapping from usb_path (config) to the mido port name (used to open the MIDI input) is built in three steps in utils/midi.py:

1. sysfs — build_usbpath_to_cardnum_map(): scan /sys/class/sound/cardN/device, resolve the symlink, walk up to the USB device directory that has a serial file, and take its basename (e.g. 1-1.4). This yields usb_path -> ALSA card number.
2. aconnect -l — build_cardnum_to_client_map(): run the ALSA utility and parse lines like client 24: 'Instruo Scion' [type=kernel,card=2] to get ALSA card number -> sequencer client number.
3. mido — Inside build_usbpath_to_port_map(): mido port names end with the sequencer client:port (e.g. Instruo Scion:Instruo Scion MIDI 1 24:0). Extract the client number with a regex and build client number -> port name.

Chaining these gives usb_path -> mido port name. Each config entry's usb_path is looked up in this map; if present, that controller task opens the corresponding MIDI port. Non-existent or unplugged USB paths are skipped. With all devices plugged in, the USB path for each port can be determined to fill in config.json.

Dependency: aconnect (from alsa-utils) must be installed so that card-to-client mapping can be obtained.

Configuration file¶

The configuration is a JSON object with three top-level keys: bulbs, controllers, and settings.

bulbs¶

An array of known WiZ bulbs. Each entry has:

• ip — IP address of the bulb on the network.
• name — Display label for the bulb (used in the control panel).

controllers¶

An array of MIDI controller entries. Each entry includes:

• name — Log and display label for this controller.
• usb_path — USB port path (e.g. 1-1.4, 3-1) identifying which physical MIDI device this entry is for. Hub ports look like 1-1.1, 1-1.2; a device on the host may be 3-1. Must match the port the device is plugged into.
• bulb_ips — List of WiZ bulb IPs to drive when this controller is active; all use the same color.
• color — Bulb color as RGB [R, G, B] (0–255).
• led_index — NeoPixel strip position (0–4).

settings¶

An object with global behaviour settings:

• led_count — NeoPixel strip length.
• idle_timeout — Seconds before a note-off is treated as idle.
• bulb_active_min_brightness / bulb_active_max_brightness — Bulb brightness range during active fade.
• led_active_min_brightness / led_active_max_brightness — LED brightness range during active fade.
• fade_duration / fade_steps — Active fade timing.
• bulb_color_enabled — Whether bulbs use per-controller color.
• bulb_idle_pulse_min_brightness / bulb_idle_pulse_max_brightness — Bulb brightness range during idle pulse.
• led_idle_pulse_min_brightness / led_idle_pulse_max_brightness — LED brightness range during idle pulse.
• idle_pulse_min_duration / idle_pulse_max_duration — Idle pulse duration range. In sync mode, max_duration is used as the fixed duration. In other modes, each controller varies its duration between min and max.
• idle_pulse_mode — Pulse timing algorithm: sync (all controllers in phase, fixed duration), harmonic (superimposed sine waves), golden (golden ratio sequence), meanrevert (Ornstein-Uhlenbeck process), breathing (asymmetric inhale/exhale with jitter), or levy (Levy flight with occasional dramatic shifts).
• idle_pulse_steps — Number of brightness steps per half-cycle.
• video_idle_filename / video_active_filename — Video files (under uploads/) for idle and active playback.
• midi_sysex_logging_enabled — When true, incoming SYSEX messages are logged to the MIDI log.
• led_color_enabled — When true, the NeoPixel strip uses per-controller color; when false, it shows white at the current brightness.
• fire_and_forget_enabled — When true, bulb commands are sent as fire-and-forget UDP (animation not blocked by slow responses); when false, the program awaits each bulb response.
• audio_enabled — When true, video playback uses sound; when false, video is silent.

Sample config.json¶

{
    "bulbs": [
        { "ip": "10.42.0.81", "name": "bulb-1" },
        { "ip": "10.42.0.192", "name": "bulb-2" }
    ],
    "controllers": [
        {
            "name": "Rock 1",
            "usb_path": "1-1.1",
            "bulb_ips": ["10.42.0.81"],
            "led_index": 0,
            "color": [255, 0, 0]
        },
        {
            "name": "Rock 2",
            "usb_path": "1-1.2",
            "bulb_ips": ["10.42.0.192"],
            "led_index": 1,
            "color": [0, 0, 255]
        }
    ],
    "settings": {
        "led_count": 5,
        "idle_timeout": 2.0,
        "bulb_active_min_brightness": 25,
        "bulb_active_max_brightness": 255,
        "led_active_min_brightness": 2,
        "led_active_max_brightness": 255,
        "fade_duration": 0.25,
        "fade_steps": 12,
        "bulb_color_enabled": true,
        "bulb_idle_pulse_min_brightness": 25,
        "bulb_idle_pulse_max_brightness": 50,
        "led_idle_pulse_min_brightness": 2,
        "led_idle_pulse_max_brightness": 25,
        "idle_pulse_min_duration": 2.0,
        "idle_pulse_max_duration": 4.0,
        "idle_pulse_mode": "sync",
        "idle_pulse_steps": 12,
        "video_idle_filename": "",
        "video_active_filename": ""
    }
}

Implementation summary¶

• server.py — Entry point: parses CLI (--paused, --host, --port), creates the Supervisor instance, and runs uvicorn. The Supervisor loads config and runs the supervision loop as an asyncio task. The loop builds the USB-path-to-MIDI-port map via utils.midi.build_usbpath_to_port_map(), starts one controller task per config entry whose usb_path is present, and runs the LED task and video process. Controller tasks use mido's callback API to listen for MIDI messages and run the per-controller state machine and bulb/LED logic. On each state transition, controller tasks update a multiprocessing.Value that the video process reads. Fade and pulse animations are time-based (driven by time.monotonic()). The LED task reads a plain dict and pushes SPI writes via asyncio.to_thread(). Video playback uses a single VLC instance with two media (idle and active) and one player in a crash-isolated child process.
• api.py — Defines the FastAPI application and all HTTP API endpoints (status, start/stop/restart, uploads, devices, controllers, bulbs, settings). Mounts the webapp/ directory as static files at /. See API for endpoint documentation.
• utils/midi.py — build_usbpath_to_cardnum_map(), build_cardnum_to_client_map(), and build_usbpath_to_port_map() implement the three-step mapping from USB path to mido port name.

Dependencies: vlc must be installed on the system (e.g. apt install vlc). The stack uses mido, pywizlight, python-vlc, pi5neo, FastAPI, and uvicorn.

Summary of key constraints¶

1. Single-process async concurrency for controllers and LED; crash-isolated child process for video (VLC/libvlc can segfault on media switching).
2. Event-driven MIDI input via callbacks — no polling, sub-millisecond reaction time.
3. Time-based animations — fade and pulse durations are accurate regardless of bulb command latency.
4. Synchronized or organic idle pulses across all controllers: sync mode uses a shared reference clock; desync mode applies per-controller duration variation via configurable algorithms (harmonic drift, golden ratio, mean-reverting walk, breathing rhythm, Levy flight).
5. Debounce timeout between note-off and idle transition (e.g. 2 seconds), implemented as a cancellable asyncio task.
6. Global video state requires all controllers active to trigger the active video.
7. Active video plays to completion before returning to idle loop.
8. Full resilience to device disconnections, reconnections, and video process crashes; supervisor restarts tasks and processes as needed.
9. Task and process lifecycle is owned by the Supervisor; start/stop/restart reload config and clear state so external APIs can drive the installation consistently.