Mikkeli
55b7a6f6d9
Written as a Pi 5 project because that is where the first camera went. That was never the destination — the camera is meant for an Orin Nano, and the Pi was convenient and available first. ⚠ This is not a wording change. The two boards do not share a camera stack: libcamera/picamera2 on the Pi, V4L2/GStreamer with Argus for CSI Bayer sensors on the Jetson. Code written directly against picamera2 does not run on the Orin at all. So capture goes behind an interface with a backend per platform, chosen at runtime from what the hardware reports, and everything above it depends only on "a source of frames". Recorded as the main design constraint rather than left to be discovered when the code moves. The same split decides where face recognition can live: TensorRT on GPU/DLA on the Orin, CPU-only on the Pi. The Pi proves the pipeline, never the performance, and AGENTS.md now requires a measurement to say which board it came from. Also generalises the camera-not-detected section to cover both boards, and keeps the cable notes that cost a module: the Pi 5's 22-pin FPC versus 15-pin Pi 4-era cables, and that Jetson carriers use their own pinout, so a cable fitting a Pi does not necessarily carry the same signals. Status corrected to "no camera connected anywhere" — the first module's cable is confirmed faulty and the module may be damaged; a second is being tried on an Orin.
103 lines
4.6 KiB
Markdown
103 lines
4.6 KiB
Markdown
# camera-webui
|
|
|
|
A camera service for single-board computers: a live video feed served through a small web UI, with
|
|
face recognition as a later stage.
|
|
|
|
**Target hardware is a Jetson Orin Nano.** A Raspberry Pi 5 is the development and test platform —
|
|
convenient, and available first — but it is not where this is meant to end up. That distinction is
|
|
load-bearing: the two boards do not share a camera stack, and only one of them can realistically run
|
|
face recognition on a live stream.
|
|
|
|
## Platforms
|
|
|
|
| | Jetson Orin Nano | Raspberry Pi 5 |
|
|
|---|---|---|
|
|
| Role | **target** | test / development |
|
|
| Arch | aarch64 | aarch64 |
|
|
| Stack | L4T / JetPack, CUDA, TensorRT | Raspberry Pi OS (Debian 12) |
|
|
| Camera path | V4L2 / GStreamer (Argus for Bayer CSI sensors) | libcamera / `rpicam` / `picamera2` |
|
|
| Inference | GPU + DLA | CPU only |
|
|
|
|
Recorded for `mikkeli-orin-nano-2` (`192.168.2.209`): L4T 36.4.4, CUDA 12.6, TensorRT 10.7. **Confirm
|
|
against whichever unit is actually used** — there is more than one Orin here, and the camera is going
|
|
to whichever one gets it wired first.
|
|
|
|
## ⚠ The camera stacks are not the same, and that is the main design constraint
|
|
|
|
This is the thing to get right early, because retrofitting it is expensive:
|
|
|
|
- **Pi 5** uses libcamera. `picamera2` is the idiomatic Python entry point.
|
|
- **Orin Nano** uses V4L2 and GStreamer. CSI Bayer sensors go through NVIDIA's Argus stack
|
|
(`nvarguscamerasrc`); USB/UVC cameras are plain V4L2.
|
|
- Code written directly against `picamera2` **will not run on the Orin at all.**
|
|
|
|
So: **put capture behind an interface** with one backend per platform, and let everything above it —
|
|
streaming, the web UI, recognition — depend only on "a source of frames". Pick the backend at
|
|
runtime from what the machine actually has, not from a build flag.
|
|
|
|
The same applies to inference. On the Orin, face recognition should go through TensorRT and can use
|
|
the GPU or DLA. On the Pi 5 it is CPU-only and will not keep up with a live stream at full
|
|
resolution. Treat the Pi as proof the *pipeline* works, never as evidence the *performance* works.
|
|
|
|
## ⚠ Current state: no camera is connected anywhere
|
|
|
|
Nothing is built yet, and no working camera has been attached to either board.
|
|
|
|
The first module, on the Pi 5, was **not detected at all**:
|
|
|
|
```console
|
|
$ rpicam-hello --list-cameras
|
|
No cameras available!
|
|
```
|
|
|
|
Diagnosed to hardware, not software. The imaging pipeline was up (`pisp_be` loaded, `/dev/media0-2`
|
|
present), but `/sys/bus/i2c/devices/` held only `i2c-13` and `i2c-14` — **no camera i2c bus was
|
|
instantiated and no CFE bound**, and `dmesg` had no sensor probe lines. `camera_auto_detect=1` loads
|
|
a sensor overlay when it finds something, so an absent bus means the firmware found nothing to probe.
|
|
Unchanged across a reboot.
|
|
|
|
**Confirmed a cable fault; the module itself may also be damaged.** A second module is being tried on
|
|
an Orin Nano.
|
|
|
|
⚠ **Do not treat `/dev/video*` as evidence of a camera.** Those nodes exist on both boards with
|
|
nothing attached — on the Pi 5 they are the codec and ISP blocks.
|
|
|
|
### Checking a connection
|
|
|
|
```bash
|
|
# Pi 5
|
|
rpicam-hello --list-cameras
|
|
dmesg | grep -iE 'imx|ov5647|cfe'
|
|
ls /sys/bus/i2c/devices/ # a camera bus should appear
|
|
|
|
# Orin Nano
|
|
v4l2-ctl --list-devices
|
|
dmesg | grep -iE 'imx|camera|argus|vi:'
|
|
```
|
|
|
|
Cable notes worth keeping, since they cost a module here:
|
|
|
|
- The **Pi 5 uses the narrow 22-pin FPC**; Pi 4-era modules ship with a **15-pin** cable and need the
|
|
adapter. Both Pi 5 connectors (`CAM/DISP 0` and `1`) are dual-purpose, so either accepts a camera.
|
|
- **Jetson carrier boards use their own pinout** — a cable that fits a Pi does not necessarily carry
|
|
the same signals. Match the cable to the carrier, not to the sensor.
|
|
- Ribbon orientation differs at each end. Always power off first.
|
|
|
|
## Planned stages
|
|
|
|
1. **Capture** — get a sensor detected on the target, grab a still, establish resolution and format.
|
|
2. **Capture abstraction** — one interface, a backend per platform, chosen at runtime.
|
|
3. **Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later
|
|
only if latency demands it.
|
|
4. **Web UI** — one page: live feed and basic controls.
|
|
5. **Face recognition** — detection before recognition, on downscaled frames, off the capture thread.
|
|
TensorRT on the Orin.
|
|
|
|
Each stage should be usable on its own before the next begins.
|
|
|
|
## Development
|
|
|
|
Codex runs on the boards themselves, so development happens on the target rather than cross-compiled
|
|
or deployed. The model endpoint and MCP gateway live on `halogen` and are reachable from both boards
|
|
by name. See [AGENTS.md](AGENTS.md).
|