mikkeli/camera-webui
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

implementation done by Codex

Browse Source
This commit is contained in:
mikkeli
2026-08-06 01:02:22 +09:00
parent d10574191c
commit 3e4f20ee85
17 changed files with 1363 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+70 -24
View File
@@ -15,7 +15,7 @@ face recognition on a live stream.
| Role | **target** | test / development |
| Arch | aarch64 | aarch64 |
| Stack | L4T / JetPack, CUDA, TensorRT | Raspberry Pi OS (Debian 12) |
| Camera path | V4L2 / GStreamer (Argus for Bayer CSI sensors) | libcamera / `rpicam` / `picamera2` |
| Camera path | GStreamer / Argus (`nvarguscamerasrc`) for CSI Bayer sensors, V4L2 for USB | libcamera / `rpicam` / `picamera2` |
| Inference | GPU + DLA | CPU only |
Recorded for `mikkeli-orin-nano-2` (`192.168.2.209`): L4T 36.4.4, CUDA 12.6, TensorRT 10.7. **Confirm
@@ -27,8 +27,8 @@ to whichever one gets it wired first.
This is the thing to get right early, because retrofitting it is expensive:
- **Pi 5** uses libcamera. `picamera2` is the idiomatic Python entry point.
- **Orin Nano** uses V4L2 and GStreamer. CSI Bayer sensors go through NVIDIA's Argus stack
(`nvarguscamerasrc`); USB/UVC cameras are plain V4L2.
- **Orin Nano** uses GStreamer with NVIDIA's Argus stack (`nvarguscamerasrc`) for CSI Bayer
sensors; plain V4L2 for USB/UVC cameras.
- Code written directly against `picamera2` **will not run on the Orin at all.**
So: **put capture behind an interface** with one backend per platform, and let everything above it —
@@ -39,27 +39,20 @@ The same applies to inference. On the Orin, face recognition should go through T
the GPU or DLA. On the Pi 5 it is CPU-only and will not keep up with a live stream at full
resolution. Treat the Pi as proof the *pipeline* works, never as evidence the *performance* works.
## Current state: working camera on the Orin
## Current state
**An IMX219 is live on the Orin Nano's CAM0**, confirmed by capture, not just by a bound driver:
**Stages 1-4 are complete.** The service is running as a systemd unit on the Orin and serving a live
camera feed through a web browser at `http://<orin-ip>:5000/`.
```console
$ v4l2-ctl --list-devices
vi-output, imx219 9-0010 (platform:tegra-capture-vi:1):
/dev/video0
### Camera: working
$ v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=1 --stream-to=/tmp/frame.raw
$ ls -l /tmp/frame.raw
16163840 # 3280 x 2464 x 2 bytes — full sensor, 10-bit Bayer
```
The frame was 99.8% non-zero across 40 distinct values, so it is real sensor data rather than an
allocated buffer. Formats: `RG10` (10-bit Bayer) at 3280x2464/21fps, 1920x1080/30fps, 1640x1232/30fps.
An IMX219 on CAM0 is live. The backend uses GStreamer (`nvarguscamerasrc` → `nvvidconv` → NV12 →
appsink) and converts frames to RGB24 in Python via numpy. Resolution is 1920×1080 at 30fps.
### ⚠ The Orin needed a device-tree overlay — it does NOT auto-detect
This cost a whole debugging session, because the symptom is identical to a dead camera: no
`/dev/video0`, no sensor lines in `dmesg`, nothing on i2c.
This cost a whole debugging session, because the symptom is identical to a dead camera: no sensor
lines in `dmesg`, nothing on i2c.
**The Pi auto-detects cameras. The Jetson does not.** `camera_auto_detect=1` has no equivalent — the
sensor's overlay must be selected explicitly, and until it is, a perfectly good camera is invisible:
@@ -133,14 +126,55 @@ Cable notes worth keeping, since they cost a module here:
the same signals. Match the cable to the carrier, not to the sensor.
- Ribbon orientation differs at each end. Always power off first.
## Application structure
```
src/camera_webui/
├── app.py # Flask app, backend auto-detect, routes
├── stream.py # MJPEG streaming (multipart/x-mixed-replace)
├── templates/index.html # Web UI (dark, full-viewport video feed)
└── camera/
├── __init__.py
├── interface.py # Abstract Camera base class + FrameMeta
├── gstreamer_backend.py # Jetson Orin — nvarguscamerasrc via GStreamer
├── v4l2_backend.py # USB cameras / generic V4L2 devices
├── picamera2_backend.py # Raspberry Pi — picamera2 (for when camera is replaced)
├── null_backend.py # Blank frames for testing without hardware
└── utils.py # JPEG encoding helpers (numpy + Pillow)
```
### Backend auto-detection
On startup the app tries backends in order:
1. **picamera2** (Pi 5) — fails silently if library absent
2. **GStreamer / Argus** (Orin) — fails silently if GStreamer unavailable
3. **V4L2** (USB / generic) — fails silently if no device supports it
4. **NullCamera** — produces blank frames as a safe fallback
Each failure is logged at INFO level, so you can see which path was taken.
### Systemd service
The service is installed at `/etc/systemd/system/camera-webui.service`:
```bash
sudo systemctl start camera-webui # Start now
sudo systemctl stop camera-webui # Stop
sudo systemctl restart camera-webui # Restart (e.g. after code changes)
sudo systemctl status camera-webui # Check status
sudo journalctl -u camera-webui -f # Live logs
```
The service depends on `nvargus-daemon.service` and restarts automatically on failure.
## Planned stages
1. ~~**Capture** — get a sensor detected on the target, grab a still, establish resolution and
format.~~ **Done on the Orin** (IMX219, `/dev/video0`, `RG10`, full frame captured).
2. **Capture abstraction** — one interface, a backend per platform, chosen at runtime.
3. **Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later
only if latency demands it.
4. **Web UI** — one page: live feed and basic controls.
1. ~~**Capture** — get a sensor detected on the target, grab a still, establish resolution and~~
format.~~ **Done on the Orin** (IMX219, GStreamer/Argus, 1920×1080@30fps).
2. ~~**Capture abstraction** — one interface, a backend per platform, chosen at runtime.~~ **Done**.
3. ~~**Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later**
only if latency demands it.~~ **Done** (MJPEG at `/feed`).
4. ~~**Web UI** — one page: live feed and basic controls.~~ **Done**.
5. **Face recognition** — detection before recognition, on downscaled frames, off the capture thread.
TensorRT on the Orin.
@@ -151,3 +185,15 @@ Each stage should be usable on its own before the next begins.
Codex runs on the boards themselves, so development happens on the target rather than cross-compiled
or deployed. The model endpoint and MCP gateway live on `halogen` and are reachable from both boards
by name. See [AGENTS.md](AGENTS.md).
## Dependencies
Installed via `uv sync` in the project virtual environment:
- **flask** — web server
- **numpy** — array manipulation (NV12→RGB conversion)
- **pillow** — JPEG encoding
GStreamer is provided by the system (Ubuntu 22.04 on L4T) — `libgstreamer1.0-0` and the NVIDIA
GStreamer plugins (`gstreamer1.0-plugins-bad`, `gstreamer1.0-nvvconv-plugin`, etc.). No pip
packages for GStreamer.