From 55b7a6f6d9516ffb2e112c8d59668f2c819d3499 Mon Sep 17 00:00:00 2001 From: Mikkeli Date: Wed, 5 Aug 2026 23:20:55 +0900 Subject: [PATCH] Broaden scope: the Orin Nano is the target, the Pi 5 was the test rig MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Written as a Pi 5 project because that is where the first camera went. That was never the destination — the camera is meant for an Orin Nano, and the Pi was convenient and available first. ⚠ This is not a wording change. The two boards do not share a camera stack: libcamera/picamera2 on the Pi, V4L2/GStreamer with Argus for CSI Bayer sensors on the Jetson. Code written directly against picamera2 does not run on the Orin at all. So capture goes behind an interface with a backend per platform, chosen at runtime from what the hardware reports, and everything above it depends only on "a source of frames". Recorded as the main design constraint rather than left to be discovered when the code moves. The same split decides where face recognition can live: TensorRT on GPU/DLA on the Orin, CPU-only on the Pi. The Pi proves the pipeline, never the performance, and AGENTS.md now requires a measurement to say which board it came from. Also generalises the camera-not-detected section to cover both boards, and keeps the cable notes that cost a module: the Pi 5's 22-pin FPC versus 15-pin Pi 4-era cables, and that Jetson carriers use their own pinout, so a cable fitting a Pi does not necessarily carry the same signals. Status corrected to "no camera connected anywhere" — the first module's cable is confirmed faulty and the module may be damaged; a second is being tried on an Orin. --- AGENTS.md | 92 +++++++++++++++++++++---------------- README.md | 134 ++++++++++++++++++++++++++++++------------------------ 2 files changed, 128 insertions(+), 98 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 15bcb86..34ff3cf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,62 +1,78 @@ # Working in this repo -Project instructions for Codex on `pi5`. The client-wide prompt lives in `~/.codex/AGENTS.md`; this -file is the project-specific part. +Project instructions for Codex. The client-wide prompt lives in `~/.codex/AGENTS.md`; this file is +the project-specific part. -## Where you are +## Know which board you are on -**You are running on the target hardware.** `pi5` is the Raspberry Pi 5 this software is *for*, so -you can test against the real camera, the real CPU and the real network — there is no emulation step -and no deploy step. Use that: prefer running the thing over reasoning about whether it would run. +This project spans **two different single-board computers with incompatible camera stacks**, and +Codex is installed on both. Getting this wrong produces code that runs where you tested it and +nowhere else. -Call `platform_info` if you need to confirm which machine you are on. It is client-local and reports -this Pi, not `halogen`. +- **Jetson Orin Nano** — the *target*. L4T / JetPack, CUDA, TensorRT. V4L2 / GStreamer, with Argus + (`nvarguscamerasrc`) for CSI Bayer sensors. +- **Raspberry Pi 5** — the *test platform*. libcamera / `rpicam` / `picamera2`. CPU-only inference. -## The camera is not detected yet +**Call `platform_info` before writing anything platform-specific.** It is client-local and reports +the machine you are actually on, not `halogen`. Do not infer the board from the fact that both are +aarch64 — that is the one thing they have in common. -`rpicam-hello --list-cameras` reports **"No cameras available!"**, and `dmesg` has no sensor probe -lines. See README.md for the diagnosis. +⚠ **Code written directly against `picamera2` will not run on the Orin.** Put capture behind an +interface with a backend per platform, selected at runtime from what the hardware reports. Everything +above capture — streaming, UI, recognition — depends only on "a source of frames". -⚠ **This is a hardware/cabling problem, and you cannot fix it from software.** Do not add -`dtoverlay=` lines, edit `/boot/firmware/config.txt`, or install packages to try to make it appear — -`camera_auto_detect=1` is already correct, and a silent `dmesg` means the sensor is not being reached -electrically. Report the state and ask the operator to reseat the cable. +⚠ **The Pi proves the pipeline, never the performance.** Face recognition on the Orin goes through +TensorRT on GPU/DLA; on the Pi it is CPU-only and will not hold a live stream. Never present a Pi +timing as evidence the target is fast enough, and say which board a measurement came from. -⚠ **Do not take `/dev/video*` as proof of a camera.** Those nodes are the Pi's codec and ISP blocks -and exist with nothing attached. +## No camera is connected yet -Until a sensor appears, work that does not depend on live capture is still available: the web UI -shell, the streaming plumbing against a test pattern or a still image, project structure, tests. -**Say plainly when you are working against a placeholder** rather than a real frame. +No working camera has been attached to either board. The first module was not detected on the Pi 5 at +all — traced to a cable fault, with the module possibly damaged too. See README.md for the evidence +and the check commands. + +⚠ **You cannot fix camera detection from software.** Do not add `dtoverlay=` lines, edit +`/boot/firmware/config.txt`, or install packages to make a sensor appear. On the Pi, +`camera_auto_detect=1` is already correct; a silent `dmesg` and a missing i2c bus mean the sensor is +not being reached electrically. Report the state and stop. + +⚠ **`/dev/video*` is not evidence of a camera** — those nodes exist on both boards with nothing +attached. + +Work that does not need live capture is still available: the capture interface and a synthetic or +still-image backend, the streaming plumbing, the web UI, project structure, tests. **Say plainly when +you are working against a placeholder** rather than a real frame. ## Constraints -- **Python 3 with `picamera2`** is the expected capture path once a sensor exists. It is not - installed yet; install it via `apt` (`python3-picamera2`), not `pip` — it binds to system - libcamera, and the pip build will not match. -- **Do not commit captured images or video.** Frames of a real room are not test fixtures. If a - fixture is genuinely needed, generate a synthetic one. -- This is a **4-core Pi 5**. Face recognition must run on downscaled frames and off the capture - thread; a per-frame full-resolution model will not hold a live stream. -- Prefer the **stdlib and system packages** over adding dependencies. Every dependency here is one - more thing that has to build on aarch64. +- **Install capture libraries from system packages, not pip.** `python3-picamera2` on the Pi; the + Jetson camera stack ships with L4T. Both bind to system libraries and a pip build will not match. +- **Do not commit captured images or video.** Frames of a real room are not test fixtures. Generate a + synthetic fixture if one is genuinely needed. +- Keep dependencies few. Everything here has to build on aarch64, and on the Jetson it has to + coexist with a vendor-pinned CUDA and Python. +- Face recognition runs on **downscaled frames, off the capture thread**. ## Verifying your work -Claims about the camera or the stream must be backed by a command that ran: +Claims about hardware, the stream, or performance need a command that ran: ```bash -rpicam-hello --list-cameras # is a sensor present at all -dmesg | grep -iE 'imx|ov5647|cfe' # did it probe -curl -sI http://localhost:/ # is the server actually serving +# is a sensor present +rpicam-hello --list-cameras # Pi 5 +v4l2-ctl --list-devices # Orin + +# is the server actually serving frames +curl -sI http://localhost:/ ``` -⚠ **The absence of an error is not evidence something works.** A stream endpoint that returns 200 -with no frames, and a working one, look identical to `curl -o /dev/null`. Check what came back. +⚠ **The absence of an error is not evidence that something works.** A stream endpoint returning 200 +with no frames and a working one are indistinguishable to `curl -o /dev/null` — check what actually +came back. The same applies to a capture backend that constructs cleanly and yields nothing. ## Git -The remote is Gitea at `192.168.2.199:3005`, reachable from this Pi over SSH on port 2222. +The remote is Gitea at `192.168.2.199:3005`, reachable over SSH on port 2222. -⚠ **`tea` (the Gitea CLI) is denied by policy** and is blocked by a `PreToolUse` hook. Ordinary -`git` is fine. **Do not push** unless the operator asks — commit locally and say what is ready. +⚠ **`tea` (the Gitea CLI) is denied by policy** and blocked by a `PreToolUse` hook. Ordinary `git` is +fine. **Do not push** unless the operator asks — commit locally and say what is ready. diff --git a/README.md b/README.md index 6fc5fc1..b46b8a4 100644 --- a/README.md +++ b/README.md @@ -1,88 +1,102 @@ -# pi5-camera-webui +# camera-webui -Camera service for a Raspberry Pi 5: a live video feed served over a small web UI, with face -recognition planned as a later stage. +A camera service for single-board computers: a live video feed served through a small web UI, with +face recognition as a later stage. -Developed on the machine it runs on (`pi5`), with Codex as the coding client — see [AGENTS.md](AGENTS.md). +**Target hardware is a Jetson Orin Nano.** A Raspberry Pi 5 is the development and test platform — +convenient, and available first — but it is not where this is meant to end up. That distinction is +load-bearing: the two boards do not share a camera stack, and only one of them can realistically run +face recognition on a live stream. -## Hardware +## Platforms -| | | -|---|---| -| Board | Raspberry Pi 5 Model B Rev 1.1 | -| OS | Debian 12 (bookworm), aarch64 | -| Camera | Pi camera module, connected to a CAM port | -| Host | `pi5`, `192.168.2.154` | +| | Jetson Orin Nano | Raspberry Pi 5 | +|---|---|---| +| Role | **target** | test / development | +| Arch | aarch64 | aarch64 | +| Stack | L4T / JetPack, CUDA, TensorRT | Raspberry Pi OS (Debian 12) | +| Camera path | V4L2 / GStreamer (Argus for Bayer CSI sensors) | libcamera / `rpicam` / `picamera2` | +| Inference | GPU + DLA | CPU only | -## ⚠ Current state: the camera is NOT detected +Recorded for `mikkeli-orin-nano-2` (`192.168.2.209`): L4T 36.4.4, CUDA 12.6, TensorRT 10.7. **Confirm +against whichever unit is actually used** — there is more than one Orin here, and the camera is going +to whichever one gets it wired first. -Nothing is built yet, and **the camera is not visible to the system**, so start here: +## ⚠ The camera stacks are not the same, and that is the main design constraint + +This is the thing to get right early, because retrofitting it is expensive: + +- **Pi 5** uses libcamera. `picamera2` is the idiomatic Python entry point. +- **Orin Nano** uses V4L2 and GStreamer. CSI Bayer sensors go through NVIDIA's Argus stack + (`nvarguscamerasrc`); USB/UVC cameras are plain V4L2. +- Code written directly against `picamera2` **will not run on the Orin at all.** + +So: **put capture behind an interface** with one backend per platform, and let everything above it — +streaming, the web UI, recognition — depend only on "a source of frames". Pick the backend at +runtime from what the machine actually has, not from a build flag. + +The same applies to inference. On the Orin, face recognition should go through TensorRT and can use +the GPU or DLA. On the Pi 5 it is CPU-only and will not keep up with a live stream at full +resolution. Treat the Pi as proof the *pipeline* works, never as evidence the *performance* works. + +## ⚠ Current state: no camera is connected anywhere + +Nothing is built yet, and no working camera has been attached to either board. + +The first module, on the Pi 5, was **not detected at all**: ```console $ rpicam-hello --list-cameras No cameras available! ``` -This is a *detection* problem, not a software one — the userland is already in place: +Diagnosed to hardware, not software. The imaging pipeline was up (`pisp_be` loaded, `/dev/media0-2` +present), but `/sys/bus/i2c/devices/` held only `i2c-13` and `i2c-14` — **no camera i2c bus was +instantiated and no CFE bound**, and `dmesg` had no sensor probe lines. `camera_auto_detect=1` loads +a sensor overlay when it finds something, so an absent bus means the firmware found nothing to probe. +Unchanged across a reboot. -- `camera_auto_detect=1` is set in `/boot/firmware/config.txt` (the correct default; nothing to add) -- 12 `libcamera`/`rpicam` packages are installed -- **`dmesg` contains no camera probe lines at all** — no sensor (`imx*`, `ov5647`), no CFE +**Confirmed a cable fault; the module itself may also be damaged.** A second module is being tried on +an Orin Nano. -That last point is the diagnostic one. A camera that is seen but misconfigured still leaves probe -messages; **silence means the sensor is not being reached electrically.** +⚠ **Do not treat `/dev/video*` as evidence of a camera.** Those nodes exist on both boards with +nothing attached — on the Pi 5 they are the codec and ISP blocks. -The imaging pipeline itself is fine, which narrows it further: - -```console -$ lsmod | grep -iE 'pisp|cfe' -pisp_be 49152 0 # ISP backend loaded -$ ls /sys/bus/i2c/devices/ -i2c-13 i2c-14 # no camera i2c bus -``` - -`camera_auto_detect=1` probes the connectors at boot and loads a sensor overlay when it finds -something. **No sensor i2c bus was instantiated and no CFE bound**, so the firmware found nothing to -load an overlay for. Nothing in software will change that. - -So check the physical connection, power off first: - -1. The Pi 5 uses the **narrower 22-pin** FPC connector. Cameras from the Pi 4 era ship with a - **15-pin** cable, which needs the 22-pin-to-15-pin adapter cable — this is the most common cause - of a camera that is connected but invisible. -2. Ribbon **orientation**: contacts face the correct side at *each* end, and the two ends are not the - same way round. -3. The connector latch is fully seated at both ends. -4. Try the **other connector** — the Pi 5 has two, `CAM/DISP 0` and `CAM/DISP 1`, and **both are - dual-purpose**, so either accepts a camera. Swapping isolates a faulty port. - -Re-check with: +### Checking a connection ```bash -rpicam-hello --list-cameras # should list a sensor -dmesg | grep -iE 'imx|ov5647|cfe' # should show a probe -ls /sys/bus/i2c/devices/ # a new bus should appear +# Pi 5 +rpicam-hello --list-cameras +dmesg | grep -iE 'imx|ov5647|cfe' +ls /sys/bus/i2c/devices/ # a camera bus should appear + +# Orin Nano +v4l2-ctl --list-devices +dmesg | grep -iE 'imx|camera|argus|vi:' ``` -⚠ **Do not treat `/dev/video*` as evidence the camera works.** Those nodes exist on this Pi already -and belong to the video codec and ISP blocks — they are present with no camera attached at all. +Cable notes worth keeping, since they cost a module here: -`picamera2` is **not** installed yet (`import picamera2` fails); it is the expected capture library -once a sensor is detected. +- The **Pi 5 uses the narrow 22-pin FPC**; Pi 4-era modules ship with a **15-pin** cable and need the + adapter. Both Pi 5 connectors (`CAM/DISP 0` and `1`) are dual-purpose, so either accepts a camera. +- **Jetson carrier boards use their own pinout** — a cable that fits a Pi does not necessarily carry + the same signals. Match the cable to the carrier, not to the sensor. +- Ribbon orientation differs at each end. Always power off first. ## Planned stages -1. **Capture** — confirm a sensor, grab a still, establish resolution/format. -2. **Live feed** — MJPEG stream first (simplest thing that works in a browser); WebRTC later if - latency demands it. -3. **Web UI** — single page showing the live feed, plus basic controls. -4. **Face recognition** — detection first, recognition after; run it on a downscaled frame, not the - full stream. +1. **Capture** — get a sensor detected on the target, grab a still, establish resolution and format. +2. **Capture abstraction** — one interface, a backend per platform, chosen at runtime. +3. **Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later + only if latency demands it. +4. **Web UI** — one page: live feed and basic controls. +5. **Face recognition** — detection before recognition, on downscaled frames, off the capture thread. + TensorRT on the Orin. Each stage should be usable on its own before the next begins. ## Development -The model endpoint and MCP gateway live on `halogen` and are reachable from this Pi by name. -Codex is installed and configured here; see [AGENTS.md](AGENTS.md) for how it is expected to work in -this repo. +Codex runs on the boards themselves, so development happens on the target rather than cross-compiled +or deployed. The model endpoint and MCP gateway live on `halogen` and are reachable from both boards +by name. See [AGENTS.md](AGENTS.md).