camera-webui/README.md

# camera-webui

A camera service for single-board computers: a live video feed served through a small web UI, with
face recognition as a later stage.

**Target hardware is a Jetson Orin Nano.** A Raspberry Pi 5 is the development and test platform —
convenient, and available first — but it is not where this is meant to end up. That distinction is
load-bearing: the two boards do not share a camera stack, and only one of them can realistically run
face recognition on a live stream.

## Platforms

| | Jetson Orin Nano | Raspberry Pi 5 |
|---|---|---|
| Role | **target** | test / development |
| Arch | aarch64 | aarch64 |
| Stack | L4T / JetPack, CUDA, TensorRT | Raspberry Pi OS (Debian 12) |
| Camera path | GStreamer / Argus (`nvarguscamerasrc`) for CSI Bayer sensors, V4L2 for USB | libcamera / `rpicam` / `picamera2` |
| Inference | GPU + DLA | CPU only |

Recorded for `mikkeli-orin-nano-2` (`192.168.2.209`): L4T 36.4.4, CUDA 12.6, TensorRT 10.7. **Confirm
against whichever unit is actually used** — there is more than one Orin here, and the camera is going
to whichever one gets it wired first.

## ⚠ The camera stacks are not the same, and that is the main design constraint

This is the thing to get right early, because retrofitting it is expensive:

- **Pi 5** uses libcamera. `picamera2` is the idiomatic Python entry point.
- **Orin Nano** uses GStreamer with NVIDIA's Argus stack (`nvarguscamerasrc`) for CSI Bayer
  sensors; plain V4L2 for USB/UVC cameras.
- Code written directly against `picamera2` **will not run on the Orin at all.**

So: **put capture behind an interface** with one backend per platform, and let everything above it —
streaming, the web UI, recognition — depend only on "a source of frames". Pick the backend at
runtime from what the machine actually has, not from a build flag.

The same applies to inference. On the Orin, face recognition should go through TensorRT and can use
the GPU or DLA. On the Pi 5 it is CPU-only and will not keep up with a live stream at full
resolution. Treat the Pi as proof the *pipeline* works, never as evidence the *performance* works.

## Current state

**Stages 1-4 are complete.** The service is running as a systemd unit on the Orin and serving a live
camera feed through a web browser at `http://<orin-ip>:5000/`.

### Camera: working

An IMX219 on CAM0 is live. The backend uses GStreamer (`nvarguscamerasrc` → `nvvidconv` → NV12 →
appsink) and converts frames to RGB24 in Python via numpy. Resolution is 1920×1080 at 30fps.

### ⚠ The Orin needed a device-tree overlay — it does NOT auto-detect

This cost a whole debugging session, because the symptom is identical to a dead camera: no sensor
lines in `dmesg`, nothing on i2c.

**The Pi auto-detects cameras. The Jetson does not.** `camera_auto_detect=1` has no equivalent — the
sensor's overlay must be selected explicitly, and until it is, a perfectly good camera is invisible:

```bash
sudo /opt/nvidia/jetson-io/config-by-hardware.py -l                     # list modules per header
sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="Camera IMX219-A" # header 2 = 24-pin CSI
sudo reboot                                                            # required
```

⚠ **`-n` needs the header number** (`2=` for the CSI connector). Without it the tool defaults to the
40-pin header and fails with `No configuration found for Camera IMX219-A on Jetson 40pin Header!`,
which reads like the module is unsupported.

Naming: **`-A` is CAM0, `-C` is CAM1.** `extlinux.conf` is backed up before the change, and the
result is one `OVERLAYS` line — remove it and reboot to undo.

⚠ **`v4l2-ctl` is not installed by default** (`sudo apt install v4l-utils`), and its absence reports
as `command not found`, which is easy to misread as "no camera".

### The Pi 5 camera is still dead

Separate fault, genuinely hardware: not detected, traced to a bad cable with the module possibly
damaged too. **Do not generalise the Orin's fix to it** — the Pi's auto-detect had nothing to
configure, so an overlay is not the answer there.

Evidence, for contrast with the Orin's *configuration* fault above:

```console
$ rpicam-hello --list-cameras
No cameras available!
```

The imaging pipeline was up (`pisp_be` loaded, `/dev/media0-2` present), but
`/sys/bus/i2c/devices/` held only `i2c-13` and `i2c-14` — **no camera i2c bus and no CFE bound** —
and `dmesg` had no sensor probe lines. On the Pi, `camera_auto_detect=1` loads an overlay when it
finds something, so an absent bus means the firmware found nothing to probe. Unchanged across a
reboot, with the module's IR LEDs lit.

⚠ **Lit IR LEDs prove power, not a working sensor.** The illuminator is wired to the 3.3V rail
independently of the i2c and CSI lanes, so it lights whenever the ribbon is seated well enough to
carry power — while a creased or partly-seated cable can still have broken exactly the i2c traces
detection depends on. Which is what happened.

⚠ **Do not treat `/dev/video*` as evidence of a camera.** Those nodes exist on both boards with
nothing attached — on the Pi 5 they are the codec and ISP blocks, on the Orin
`tegra-camrtc-ca`/`/dev/media0` is the VI platform block.

### Checking a connection

```bash
# Pi 5
rpicam-hello --list-cameras
dmesg | grep -iE 'imx|ov5647|cfe'
ls /sys/bus/i2c/devices/          # a camera bus should appear

# Orin Nano
grep -i OVERLAYS /boot/extlinux/extlinux.conf   # ⚠ CHECK THIS FIRST — no line, no camera
v4l2-ctl --list-devices
dmesg | grep -iE 'imx|camera|argus|vi:'
```

On the Orin, check the overlay **before** suspecting hardware. An unconfigured Jetson and a dead
camera look exactly alike.

Cable notes worth keeping, since they cost a module here:

- The **Pi 5 uses the narrow 22-pin FPC**; Pi 4-era modules ship with a **15-pin** cable and need the
  adapter. Both Pi 5 connectors (`CAM/DISP 0` and `1`) are dual-purpose, so either accepts a camera.
- **Jetson carrier boards use their own pinout** — a cable that fits a Pi does not necessarily carry
  the same signals. Match the cable to the carrier, not to the sensor.
- Ribbon orientation differs at each end. Always power off first.

## Application structure

```
src/camera_webui/
├── app.py                  # Flask app, backend auto-detect, routes
├── stream.py               # MJPEG streaming (multipart/x-mixed-replace)
├── templates/index.html    # Web UI (dark, full-viewport video feed)
└── camera/
    ├── __init__.py
    ├── interface.py        # Abstract Camera base class + FrameMeta
    ├── gstreamer_backend.py # Jetson Orin — nvarguscamerasrc via GStreamer
    ├── v4l2_backend.py     # USB cameras / generic V4L2 devices
    ├── picamera2_backend.py # Raspberry Pi — picamera2 (for when camera is replaced)
    ├── null_backend.py     # Blank frames for testing without hardware
    └── utils.py            # JPEG encoding helpers (numpy + Pillow)
```

### Backend auto-detection

On startup the app tries backends in order:
1. **picamera2** (Pi 5) — fails silently if library absent
2. **GStreamer / Argus** (Orin) — fails silently if GStreamer unavailable
3. **V4L2** (USB / generic) — fails silently if no device supports it
4. **NullCamera** — produces blank frames as a safe fallback

Each failure is logged at INFO level, so you can see which path was taken.

### Systemd service

The service is installed at `/etc/systemd/system/camera-webui.service`:

```bash
sudo systemctl start  camera-webui   # Start now
sudo systemctl stop   camera-webui   # Stop
sudo systemctl restart camera-webui  # Restart (e.g. after code changes)
sudo systemctl status camera-webui   # Check status
sudo journalctl -u camera-webui -f   # Live logs
```

The service depends on `nvargus-daemon.service` and restarts automatically on failure.

## Planned stages

1. ~~**Capture** — get a sensor detected on the target, grab a still, establish resolution and~~
   format.~~ **Done on the Orin** (IMX219, GStreamer/Argus, 1920×1080@30fps).
2. ~~**Capture abstraction** — one interface, a backend per platform, chosen at runtime.~~ **Done**.
3. ~~**Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later**
   only if latency demands it.~~ **Done** (MJPEG at `/feed`).
4. ~~**Web UI** — one page: live feed and basic controls.~~ **Done**.
5. **Face recognition** — detection before recognition, on downscaled frames, off the capture thread.
   TensorRT on the Orin.

Each stage should be usable on its own before the next begins.

## Development

Codex runs on the boards themselves, so development happens on the target rather than cross-compiled
or deployed. The model endpoint and MCP gateway live on `halogen` and are reachable from both boards
by name. See [AGENTS.md](AGENTS.md).

## Dependencies

Installed via `uv sync` in the project virtual environment:

- **flask** — web server
- **numpy** — array manipulation (NV12→RGB conversion)
- **pillow** — JPEG encoding

GStreamer is provided by the system (Ubuntu 22.04 on L4T) — `libgstreamer1.0-0` and the NVIDIA
GStreamer plugins (`gstreamer1.0-plugins-bad`, `gstreamer1.0-nvvconv-plugin`, etc.). No pip
packages for GStreamer.