diff --git a/AGENTS.md b/AGENTS.md index 15bcb86..34ff3cf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,62 +1,78 @@ # Working in this repo -Project instructions for Codex on `pi5`. The client-wide prompt lives in `~/.codex/AGENTS.md`; this -file is the project-specific part. +Project instructions for Codex. The client-wide prompt lives in `~/.codex/AGENTS.md`; this file is +the project-specific part. -## Where you are +## Know which board you are on -**You are running on the target hardware.** `pi5` is the Raspberry Pi 5 this software is *for*, so -you can test against the real camera, the real CPU and the real network — there is no emulation step -and no deploy step. Use that: prefer running the thing over reasoning about whether it would run. +This project spans **two different single-board computers with incompatible camera stacks**, and +Codex is installed on both. Getting this wrong produces code that runs where you tested it and +nowhere else. -Call `platform_info` if you need to confirm which machine you are on. It is client-local and reports -this Pi, not `halogen`. +- **Jetson Orin Nano** — the *target*. L4T / JetPack, CUDA, TensorRT. V4L2 / GStreamer, with Argus + (`nvarguscamerasrc`) for CSI Bayer sensors. +- **Raspberry Pi 5** — the *test platform*. libcamera / `rpicam` / `picamera2`. CPU-only inference. -## The camera is not detected yet +**Call `platform_info` before writing anything platform-specific.** It is client-local and reports +the machine you are actually on, not `halogen`. Do not infer the board from the fact that both are +aarch64 — that is the one thing they have in common. -`rpicam-hello --list-cameras` reports **"No cameras available!"**, and `dmesg` has no sensor probe -lines. See README.md for the diagnosis. +⚠ **Code written directly against `picamera2` will not run on the Orin.** Put capture behind an +interface with a backend per platform, selected at runtime from what the hardware reports. Everything +above capture — streaming, UI, recognition — depends only on "a source of frames". -⚠ **This is a hardware/cabling problem, and you cannot fix it from software.** Do not add -`dtoverlay=` lines, edit `/boot/firmware/config.txt`, or install packages to try to make it appear — -`camera_auto_detect=1` is already correct, and a silent `dmesg` means the sensor is not being reached -electrically. Report the state and ask the operator to reseat the cable. +⚠ **The Pi proves the pipeline, never the performance.** Face recognition on the Orin goes through +TensorRT on GPU/DLA; on the Pi it is CPU-only and will not hold a live stream. Never present a Pi +timing as evidence the target is fast enough, and say which board a measurement came from. -⚠ **Do not take `/dev/video*` as proof of a camera.** Those nodes are the Pi's codec and ISP blocks -and exist with nothing attached. +## No camera is connected yet -Until a sensor appears, work that does not depend on live capture is still available: the web UI -shell, the streaming plumbing against a test pattern or a still image, project structure, tests. -**Say plainly when you are working against a placeholder** rather than a real frame. +No working camera has been attached to either board. The first module was not detected on the Pi 5 at +all — traced to a cable fault, with the module possibly damaged too. See README.md for the evidence +and the check commands. + +⚠ **You cannot fix camera detection from software.** Do not add `dtoverlay=` lines, edit +`/boot/firmware/config.txt`, or install packages to make a sensor appear. On the Pi, +`camera_auto_detect=1` is already correct; a silent `dmesg` and a missing i2c bus mean the sensor is +not being reached electrically. Report the state and stop. + +⚠ **`/dev/video*` is not evidence of a camera** — those nodes exist on both boards with nothing +attached. + +Work that does not need live capture is still available: the capture interface and a synthetic or +still-image backend, the streaming plumbing, the web UI, project structure, tests. **Say plainly when +you are working against a placeholder** rather than a real frame. ## Constraints -- **Python 3 with `picamera2`** is the expected capture path once a sensor exists. It is not - installed yet; install it via `apt` (`python3-picamera2`), not `pip` — it binds to system - libcamera, and the pip build will not match. -- **Do not commit captured images or video.** Frames of a real room are not test fixtures. If a - fixture is genuinely needed, generate a synthetic one. -- This is a **4-core Pi 5**. Face recognition must run on downscaled frames and off the capture - thread; a per-frame full-resolution model will not hold a live stream. -- Prefer the **stdlib and system packages** over adding dependencies. Every dependency here is one - more thing that has to build on aarch64. +- **Install capture libraries from system packages, not pip.** `python3-picamera2` on the Pi; the + Jetson camera stack ships with L4T. Both bind to system libraries and a pip build will not match. +- **Do not commit captured images or video.** Frames of a real room are not test fixtures. Generate a + synthetic fixture if one is genuinely needed. +- Keep dependencies few. Everything here has to build on aarch64, and on the Jetson it has to + coexist with a vendor-pinned CUDA and Python. +- Face recognition runs on **downscaled frames, off the capture thread**. ## Verifying your work -Claims about the camera or the stream must be backed by a command that ran: +Claims about hardware, the stream, or performance need a command that ran: ```bash -rpicam-hello --list-cameras # is a sensor present at all -dmesg | grep -iE 'imx|ov5647|cfe' # did it probe -curl -sI http://localhost:/ # is the server actually serving +# is a sensor present +rpicam-hello --list-cameras # Pi 5 +v4l2-ctl --list-devices # Orin + +# is the server actually serving frames +curl -sI http://localhost:/ ``` -⚠ **The absence of an error is not evidence something works.** A stream endpoint that returns 200 -with no frames, and a working one, look identical to `curl -o /dev/null`. Check what came back. +⚠ **The absence of an error is not evidence that something works.** A stream endpoint returning 200 +with no frames and a working one are indistinguishable to `curl -o /dev/null` — check what actually +came back. The same applies to a capture backend that constructs cleanly and yields nothing. ## Git -The remote is Gitea at `192.168.2.199:3005`, reachable from this Pi over SSH on port 2222. +The remote is Gitea at `192.168.2.199:3005`, reachable over SSH on port 2222. -⚠ **`tea` (the Gitea CLI) is denied by policy** and is blocked by a `PreToolUse` hook. Ordinary -`git` is fine. **Do not push** unless the operator asks — commit locally and say what is ready. +⚠ **`tea` (the Gitea CLI) is denied by policy** and blocked by a `PreToolUse` hook. Ordinary `git` is +fine. **Do not push** unless the operator asks — commit locally and say what is ready. diff --git a/README.md b/README.md index 6fc5fc1..b46b8a4 100644 --- a/README.md +++ b/README.md @@ -1,88 +1,102 @@ -# pi5-camera-webui +# camera-webui -Camera service for a Raspberry Pi 5: a live video feed served over a small web UI, with face -recognition planned as a later stage. +A camera service for single-board computers: a live video feed served through a small web UI, with +face recognition as a later stage. -Developed on the machine it runs on (`pi5`), with Codex as the coding client — see [AGENTS.md](AGENTS.md). +**Target hardware is a Jetson Orin Nano.** A Raspberry Pi 5 is the development and test platform — +convenient, and available first — but it is not where this is meant to end up. That distinction is +load-bearing: the two boards do not share a camera stack, and only one of them can realistically run +face recognition on a live stream. -## Hardware +## Platforms -| | | -|---|---| -| Board | Raspberry Pi 5 Model B Rev 1.1 | -| OS | Debian 12 (bookworm), aarch64 | -| Camera | Pi camera module, connected to a CAM port | -| Host | `pi5`, `192.168.2.154` | +| | Jetson Orin Nano | Raspberry Pi 5 | +|---|---|---| +| Role | **target** | test / development | +| Arch | aarch64 | aarch64 | +| Stack | L4T / JetPack, CUDA, TensorRT | Raspberry Pi OS (Debian 12) | +| Camera path | V4L2 / GStreamer (Argus for Bayer CSI sensors) | libcamera / `rpicam` / `picamera2` | +| Inference | GPU + DLA | CPU only | -## ⚠ Current state: the camera is NOT detected +Recorded for `mikkeli-orin-nano-2` (`192.168.2.209`): L4T 36.4.4, CUDA 12.6, TensorRT 10.7. **Confirm +against whichever unit is actually used** — there is more than one Orin here, and the camera is going +to whichever one gets it wired first. -Nothing is built yet, and **the camera is not visible to the system**, so start here: +## ⚠ The camera stacks are not the same, and that is the main design constraint + +This is the thing to get right early, because retrofitting it is expensive: + +- **Pi 5** uses libcamera. `picamera2` is the idiomatic Python entry point. +- **Orin Nano** uses V4L2 and GStreamer. CSI Bayer sensors go through NVIDIA's Argus stack + (`nvarguscamerasrc`); USB/UVC cameras are plain V4L2. +- Code written directly against `picamera2` **will not run on the Orin at all.** + +So: **put capture behind an interface** with one backend per platform, and let everything above it — +streaming, the web UI, recognition — depend only on "a source of frames". Pick the backend at +runtime from what the machine actually has, not from a build flag. + +The same applies to inference. On the Orin, face recognition should go through TensorRT and can use +the GPU or DLA. On the Pi 5 it is CPU-only and will not keep up with a live stream at full +resolution. Treat the Pi as proof the *pipeline* works, never as evidence the *performance* works. + +## ⚠ Current state: no camera is connected anywhere + +Nothing is built yet, and no working camera has been attached to either board. + +The first module, on the Pi 5, was **not detected at all**: ```console $ rpicam-hello --list-cameras No cameras available! ``` -This is a *detection* problem, not a software one — the userland is already in place: +Diagnosed to hardware, not software. The imaging pipeline was up (`pisp_be` loaded, `/dev/media0-2` +present), but `/sys/bus/i2c/devices/` held only `i2c-13` and `i2c-14` — **no camera i2c bus was +instantiated and no CFE bound**, and `dmesg` had no sensor probe lines. `camera_auto_detect=1` loads +a sensor overlay when it finds something, so an absent bus means the firmware found nothing to probe. +Unchanged across a reboot. -- `camera_auto_detect=1` is set in `/boot/firmware/config.txt` (the correct default; nothing to add) -- 12 `libcamera`/`rpicam` packages are installed -- **`dmesg` contains no camera probe lines at all** — no sensor (`imx*`, `ov5647`), no CFE +**Confirmed a cable fault; the module itself may also be damaged.** A second module is being tried on +an Orin Nano. -That last point is the diagnostic one. A camera that is seen but misconfigured still leaves probe -messages; **silence means the sensor is not being reached electrically.** +⚠ **Do not treat `/dev/video*` as evidence of a camera.** Those nodes exist on both boards with +nothing attached — on the Pi 5 they are the codec and ISP blocks. -The imaging pipeline itself is fine, which narrows it further: - -```console -$ lsmod | grep -iE 'pisp|cfe' -pisp_be 49152 0 # ISP backend loaded -$ ls /sys/bus/i2c/devices/ -i2c-13 i2c-14 # no camera i2c bus -``` - -`camera_auto_detect=1` probes the connectors at boot and loads a sensor overlay when it finds -something. **No sensor i2c bus was instantiated and no CFE bound**, so the firmware found nothing to -load an overlay for. Nothing in software will change that. - -So check the physical connection, power off first: - -1. The Pi 5 uses the **narrower 22-pin** FPC connector. Cameras from the Pi 4 era ship with a - **15-pin** cable, which needs the 22-pin-to-15-pin adapter cable — this is the most common cause - of a camera that is connected but invisible. -2. Ribbon **orientation**: contacts face the correct side at *each* end, and the two ends are not the - same way round. -3. The connector latch is fully seated at both ends. -4. Try the **other connector** — the Pi 5 has two, `CAM/DISP 0` and `CAM/DISP 1`, and **both are - dual-purpose**, so either accepts a camera. Swapping isolates a faulty port. - -Re-check with: +### Checking a connection ```bash -rpicam-hello --list-cameras # should list a sensor -dmesg | grep -iE 'imx|ov5647|cfe' # should show a probe -ls /sys/bus/i2c/devices/ # a new bus should appear +# Pi 5 +rpicam-hello --list-cameras +dmesg | grep -iE 'imx|ov5647|cfe' +ls /sys/bus/i2c/devices/ # a camera bus should appear + +# Orin Nano +v4l2-ctl --list-devices +dmesg | grep -iE 'imx|camera|argus|vi:' ``` -⚠ **Do not treat `/dev/video*` as evidence the camera works.** Those nodes exist on this Pi already -and belong to the video codec and ISP blocks — they are present with no camera attached at all. +Cable notes worth keeping, since they cost a module here: -`picamera2` is **not** installed yet (`import picamera2` fails); it is the expected capture library -once a sensor is detected. +- The **Pi 5 uses the narrow 22-pin FPC**; Pi 4-era modules ship with a **15-pin** cable and need the + adapter. Both Pi 5 connectors (`CAM/DISP 0` and `1`) are dual-purpose, so either accepts a camera. +- **Jetson carrier boards use their own pinout** — a cable that fits a Pi does not necessarily carry + the same signals. Match the cable to the carrier, not to the sensor. +- Ribbon orientation differs at each end. Always power off first. ## Planned stages -1. **Capture** — confirm a sensor, grab a still, establish resolution/format. -2. **Live feed** — MJPEG stream first (simplest thing that works in a browser); WebRTC later if - latency demands it. -3. **Web UI** — single page showing the live feed, plus basic controls. -4. **Face recognition** — detection first, recognition after; run it on a downscaled frame, not the - full stream. +1. **Capture** — get a sensor detected on the target, grab a still, establish resolution and format. +2. **Capture abstraction** — one interface, a backend per platform, chosen at runtime. +3. **Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later + only if latency demands it. +4. **Web UI** — one page: live feed and basic controls. +5. **Face recognition** — detection before recognition, on downscaled frames, off the capture thread. + TensorRT on the Orin. Each stage should be usable on its own before the next begins. ## Development -The model endpoint and MCP gateway live on `halogen` and are reachable from this Pi by name. -Codex is installed and configured here; see [AGENTS.md](AGENTS.md) for how it is expected to work in -this repo. +Codex runs on the boards themselves, so development happens on the target rather than cross-compiled +or deployed. The model endpoint and MCP gateway live on `halogen` and are reachable from both boards +by name. See [AGENTS.md](AGENTS.md).