Mikkeli
d10574191c
An IMX219 is live on the Orin's CAM0 at /dev/video0, verified by capture rather than by a bound driver: a 16,163,840-byte frame, exactly 3280x2464x2 for 10-bit Bayer, 99.8% non-zero across 40 distinct values. Real sensor data, not an allocated buffer. ⚠ The fault was configuration, and it is indistinguishable from dead hardware. The Pi auto-detects cameras; the Jetson does not. Until the sensor's device-tree overlay is selected there is no /dev/video0, no sensor line in dmesg and nothing on i2c — the same evidence a broken cable produces, which is how it got blamed on a cable here. Fixed with config-by-hardware.py, and the README now says to check the OVERLAYS line BEFORE suspecting hardware. Two traps recorded with it: the -n flag needs the header number (2= for the CSI connector) or it defaults to the 40-pin header and reports the module as unsupported, and v4l-utils is not installed by default so v4l2-ctl reports "command not found", which reads as "no camera". The Pi 5 fault was genuinely a cable and is kept for contrast, along with why its lit IR LEDs proved nothing: the illuminator sits on the 3.3V rail independently of the i2c and CSI lanes, so it lights whenever the ribbon carries power, even when the traces detection needs are broken. AGENTS.md also gains the rules that came out of a 27-attempt loop in this repo: write files with apply_patch and never write_stdin, never invent or increment a session id, stop after two identical failures instead of retrying, and emit tool calls as JSON arguments rather than leaking <parameter=...> markup into strings.
154 lines
7.2 KiB
Markdown
154 lines
7.2 KiB
Markdown
# camera-webui
|
|
|
|
A camera service for single-board computers: a live video feed served through a small web UI, with
|
|
face recognition as a later stage.
|
|
|
|
**Target hardware is a Jetson Orin Nano.** A Raspberry Pi 5 is the development and test platform —
|
|
convenient, and available first — but it is not where this is meant to end up. That distinction is
|
|
load-bearing: the two boards do not share a camera stack, and only one of them can realistically run
|
|
face recognition on a live stream.
|
|
|
|
## Platforms
|
|
|
|
| | Jetson Orin Nano | Raspberry Pi 5 |
|
|
|---|---|---|
|
|
| Role | **target** | test / development |
|
|
| Arch | aarch64 | aarch64 |
|
|
| Stack | L4T / JetPack, CUDA, TensorRT | Raspberry Pi OS (Debian 12) |
|
|
| Camera path | V4L2 / GStreamer (Argus for Bayer CSI sensors) | libcamera / `rpicam` / `picamera2` |
|
|
| Inference | GPU + DLA | CPU only |
|
|
|
|
Recorded for `mikkeli-orin-nano-2` (`192.168.2.209`): L4T 36.4.4, CUDA 12.6, TensorRT 10.7. **Confirm
|
|
against whichever unit is actually used** — there is more than one Orin here, and the camera is going
|
|
to whichever one gets it wired first.
|
|
|
|
## ⚠ The camera stacks are not the same, and that is the main design constraint
|
|
|
|
This is the thing to get right early, because retrofitting it is expensive:
|
|
|
|
- **Pi 5** uses libcamera. `picamera2` is the idiomatic Python entry point.
|
|
- **Orin Nano** uses V4L2 and GStreamer. CSI Bayer sensors go through NVIDIA's Argus stack
|
|
(`nvarguscamerasrc`); USB/UVC cameras are plain V4L2.
|
|
- Code written directly against `picamera2` **will not run on the Orin at all.**
|
|
|
|
So: **put capture behind an interface** with one backend per platform, and let everything above it —
|
|
streaming, the web UI, recognition — depend only on "a source of frames". Pick the backend at
|
|
runtime from what the machine actually has, not from a build flag.
|
|
|
|
The same applies to inference. On the Orin, face recognition should go through TensorRT and can use
|
|
the GPU or DLA. On the Pi 5 it is CPU-only and will not keep up with a live stream at full
|
|
resolution. Treat the Pi as proof the *pipeline* works, never as evidence the *performance* works.
|
|
|
|
## Current state: working camera on the Orin
|
|
|
|
**An IMX219 is live on the Orin Nano's CAM0**, confirmed by capture, not just by a bound driver:
|
|
|
|
```console
|
|
$ v4l2-ctl --list-devices
|
|
vi-output, imx219 9-0010 (platform:tegra-capture-vi:1):
|
|
/dev/video0
|
|
|
|
$ v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=1 --stream-to=/tmp/frame.raw
|
|
$ ls -l /tmp/frame.raw
|
|
16163840 # 3280 x 2464 x 2 bytes — full sensor, 10-bit Bayer
|
|
```
|
|
|
|
The frame was 99.8% non-zero across 40 distinct values, so it is real sensor data rather than an
|
|
allocated buffer. Formats: `RG10` (10-bit Bayer) at 3280x2464/21fps, 1920x1080/30fps, 1640x1232/30fps.
|
|
|
|
### ⚠ The Orin needed a device-tree overlay — it does NOT auto-detect
|
|
|
|
This cost a whole debugging session, because the symptom is identical to a dead camera: no
|
|
`/dev/video0`, no sensor lines in `dmesg`, nothing on i2c.
|
|
|
|
**The Pi auto-detects cameras. The Jetson does not.** `camera_auto_detect=1` has no equivalent — the
|
|
sensor's overlay must be selected explicitly, and until it is, a perfectly good camera is invisible:
|
|
|
|
```bash
|
|
sudo /opt/nvidia/jetson-io/config-by-hardware.py -l # list modules per header
|
|
sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="Camera IMX219-A" # header 2 = 24-pin CSI
|
|
sudo reboot # required
|
|
```
|
|
|
|
⚠ **`-n` needs the header number** (`2=` for the CSI connector). Without it the tool defaults to the
|
|
40-pin header and fails with `No configuration found for Camera IMX219-A on Jetson 40pin Header!`,
|
|
which reads like the module is unsupported.
|
|
|
|
Naming: **`-A` is CAM0, `-C` is CAM1.** `extlinux.conf` is backed up before the change, and the
|
|
result is one `OVERLAYS` line — remove it and reboot to undo.
|
|
|
|
⚠ **`v4l2-ctl` is not installed by default** (`sudo apt install v4l-utils`), and its absence reports
|
|
as `command not found`, which is easy to misread as "no camera".
|
|
|
|
### The Pi 5 camera is still dead
|
|
|
|
Separate fault, genuinely hardware: not detected, traced to a bad cable with the module possibly
|
|
damaged too. **Do not generalise the Orin's fix to it** — the Pi's auto-detect had nothing to
|
|
configure, so an overlay is not the answer there.
|
|
|
|
Evidence, for contrast with the Orin's *configuration* fault above:
|
|
|
|
```console
|
|
$ rpicam-hello --list-cameras
|
|
No cameras available!
|
|
```
|
|
|
|
The imaging pipeline was up (`pisp_be` loaded, `/dev/media0-2` present), but
|
|
`/sys/bus/i2c/devices/` held only `i2c-13` and `i2c-14` — **no camera i2c bus and no CFE bound** —
|
|
and `dmesg` had no sensor probe lines. On the Pi, `camera_auto_detect=1` loads an overlay when it
|
|
finds something, so an absent bus means the firmware found nothing to probe. Unchanged across a
|
|
reboot, with the module's IR LEDs lit.
|
|
|
|
⚠ **Lit IR LEDs prove power, not a working sensor.** The illuminator is wired to the 3.3V rail
|
|
independently of the i2c and CSI lanes, so it lights whenever the ribbon is seated well enough to
|
|
carry power — while a creased or partly-seated cable can still have broken exactly the i2c traces
|
|
detection depends on. Which is what happened.
|
|
|
|
⚠ **Do not treat `/dev/video*` as evidence of a camera.** Those nodes exist on both boards with
|
|
nothing attached — on the Pi 5 they are the codec and ISP blocks, on the Orin
|
|
`tegra-camrtc-ca`/`/dev/media0` is the VI platform block.
|
|
|
|
### Checking a connection
|
|
|
|
```bash
|
|
# Pi 5
|
|
rpicam-hello --list-cameras
|
|
dmesg | grep -iE 'imx|ov5647|cfe'
|
|
ls /sys/bus/i2c/devices/ # a camera bus should appear
|
|
|
|
# Orin Nano
|
|
grep -i OVERLAYS /boot/extlinux/extlinux.conf # ⚠ CHECK THIS FIRST — no line, no camera
|
|
v4l2-ctl --list-devices
|
|
dmesg | grep -iE 'imx|camera|argus|vi:'
|
|
```
|
|
|
|
On the Orin, check the overlay **before** suspecting hardware. An unconfigured Jetson and a dead
|
|
camera look exactly alike.
|
|
|
|
Cable notes worth keeping, since they cost a module here:
|
|
|
|
- The **Pi 5 uses the narrow 22-pin FPC**; Pi 4-era modules ship with a **15-pin** cable and need the
|
|
adapter. Both Pi 5 connectors (`CAM/DISP 0` and `1`) are dual-purpose, so either accepts a camera.
|
|
- **Jetson carrier boards use their own pinout** — a cable that fits a Pi does not necessarily carry
|
|
the same signals. Match the cable to the carrier, not to the sensor.
|
|
- Ribbon orientation differs at each end. Always power off first.
|
|
|
|
## Planned stages
|
|
|
|
1. ~~**Capture** — get a sensor detected on the target, grab a still, establish resolution and
|
|
format.~~ **Done on the Orin** (IMX219, `/dev/video0`, `RG10`, full frame captured).
|
|
2. **Capture abstraction** — one interface, a backend per platform, chosen at runtime.
|
|
3. **Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later
|
|
only if latency demands it.
|
|
4. **Web UI** — one page: live feed and basic controls.
|
|
5. **Face recognition** — detection before recognition, on downscaled frames, off the capture thread.
|
|
TensorRT on the Orin.
|
|
|
|
Each stage should be usable on its own before the next begins.
|
|
|
|
## Development
|
|
|
|
Codex runs on the boards themselves, so development happens on the target rather than cross-compiled
|
|
or deployed. The model endpoint and MCP gateway live on `halogen` and are reachable from both boards
|
|
by name. See [AGENTS.md](AGENTS.md).
|