diff --git a/AGENTS.md b/AGENTS.md index 34ff3cf..0c9d611 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,23 +25,47 @@ above capture — streaming, UI, recognition — depends only on "a source of fr TensorRT on GPU/DLA; on the Pi it is CPU-only and will not hold a live stream. Never present a Pi timing as evidence the target is fast enough, and say which board a measurement came from. -## No camera is connected yet +## Camera state -No working camera has been attached to either board. The first module was not detected on the Pi 5 at -all — traced to a cable fault, with the module possibly damaged too. See README.md for the evidence -and the check commands. +**Orin Nano: working.** IMX219 on CAM0 at `/dev/video0`, format `RG10` (10-bit Bayer), full frame +captured and verified as real data. This is the board to develop capture against. -⚠ **You cannot fix camera detection from software.** Do not add `dtoverlay=` lines, edit -`/boot/firmware/config.txt`, or install packages to make a sensor appear. On the Pi, -`camera_auto_detect=1` is already correct; a silent `dmesg` and a missing i2c bus mean the sensor is -not being reached electrically. Report the state and stop. +**Pi 5: dead.** Not detected; a cable fault, with the module possibly damaged. Nothing to do there +until the hardware is replaced. -⚠ **`/dev/video*` is not evidence of a camera** — those nodes exist on both boards with nothing +⚠ **The two boards failed for completely different reasons, and the fixes do not transfer.** The +Orin was a missing device-tree overlay — a configuration fault that looks exactly like dead hardware. +The Pi was genuinely a broken cable. See README.md. + +⚠ **On the Pi, you cannot fix camera detection from software.** Do not add `dtoverlay=` lines or edit +`/boot/firmware/config.txt`; `camera_auto_detect=1` is already correct there. On the Orin the overlay +*is* the mechanism, but it is already configured — do not change it without being asked. + +⚠ **`/dev/video*` is not evidence of a camera.** Those nodes exist on both boards with nothing attached. -Work that does not need live capture is still available: the capture interface and a synthetic or -still-image backend, the streaming plumbing, the web UI, project structure, tests. **Say plainly when -you are working against a placeholder** rather than a real frame. +If you are ever on a board with no working camera, work that does not need live capture is still +available: the capture interface and a synthetic backend, streaming plumbing, the web UI, tests. +**Say plainly when you are working against a placeholder** rather than a real frame. + +## ⚠ Writing files, and not looping + +**Write files with `apply_patch`**, or a heredoc through the shell. **Never** use `write_stdin` to +create file contents — it writes to the stdin of an already-running process, which is a different +thing and will not create anything. `write_stdin` takes a session id that an actual `exec_command` +returned to you; never invent one and never increment one. + +**If the same tool call fails twice with the same error, STOP and report it.** Do not retry, and do +not vary an identifier hoping one works. A repeated identical failure means the assumption is wrong, +not that the call needs another attempt. + +This happened here, in this repo. A session got as far as creating `src/camera_webui/capture/`, then +tried to write `base.py` via `write_stdin`, was told `Unknown process id 45`, and answered by +incrementing the id — 46, 47, 48 — for **27 attempts over nine minutes** until a human killed it. + +⚠ **Emit tool calls as the API's JSON arguments only.** If you find yourself writing markup like +`` inside a string argument, you are mixing in a different tool-calling format and the +call will not do what you mean. That is what the loop above degenerated into. ## Constraints diff --git a/README.md b/README.md index b46b8a4..aafd99f 100644 --- a/README.md +++ b/README.md @@ -39,28 +39,74 @@ The same applies to inference. On the Orin, face recognition should go through T the GPU or DLA. On the Pi 5 it is CPU-only and will not keep up with a live stream at full resolution. Treat the Pi as proof the *pipeline* works, never as evidence the *performance* works. -## ⚠ Current state: no camera is connected anywhere +## Current state: working camera on the Orin -Nothing is built yet, and no working camera has been attached to either board. +**An IMX219 is live on the Orin Nano's CAM0**, confirmed by capture, not just by a bound driver: -The first module, on the Pi 5, was **not detected at all**: +```console +$ v4l2-ctl --list-devices +vi-output, imx219 9-0010 (platform:tegra-capture-vi:1): + /dev/video0 + +$ v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=1 --stream-to=/tmp/frame.raw +$ ls -l /tmp/frame.raw +16163840 # 3280 x 2464 x 2 bytes — full sensor, 10-bit Bayer +``` + +The frame was 99.8% non-zero across 40 distinct values, so it is real sensor data rather than an +allocated buffer. Formats: `RG10` (10-bit Bayer) at 3280x2464/21fps, 1920x1080/30fps, 1640x1232/30fps. + +### ⚠ The Orin needed a device-tree overlay — it does NOT auto-detect + +This cost a whole debugging session, because the symptom is identical to a dead camera: no +`/dev/video0`, no sensor lines in `dmesg`, nothing on i2c. + +**The Pi auto-detects cameras. The Jetson does not.** `camera_auto_detect=1` has no equivalent — the +sensor's overlay must be selected explicitly, and until it is, a perfectly good camera is invisible: + +```bash +sudo /opt/nvidia/jetson-io/config-by-hardware.py -l # list modules per header +sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="Camera IMX219-A" # header 2 = 24-pin CSI +sudo reboot # required +``` + +⚠ **`-n` needs the header number** (`2=` for the CSI connector). Without it the tool defaults to the +40-pin header and fails with `No configuration found for Camera IMX219-A on Jetson 40pin Header!`, +which reads like the module is unsupported. + +Naming: **`-A` is CAM0, `-C` is CAM1.** `extlinux.conf` is backed up before the change, and the +result is one `OVERLAYS` line — remove it and reboot to undo. + +⚠ **`v4l2-ctl` is not installed by default** (`sudo apt install v4l-utils`), and its absence reports +as `command not found`, which is easy to misread as "no camera". + +### The Pi 5 camera is still dead + +Separate fault, genuinely hardware: not detected, traced to a bad cable with the module possibly +damaged too. **Do not generalise the Orin's fix to it** — the Pi's auto-detect had nothing to +configure, so an overlay is not the answer there. + +Evidence, for contrast with the Orin's *configuration* fault above: ```console $ rpicam-hello --list-cameras No cameras available! ``` -Diagnosed to hardware, not software. The imaging pipeline was up (`pisp_be` loaded, `/dev/media0-2` -present), but `/sys/bus/i2c/devices/` held only `i2c-13` and `i2c-14` — **no camera i2c bus was -instantiated and no CFE bound**, and `dmesg` had no sensor probe lines. `camera_auto_detect=1` loads -a sensor overlay when it finds something, so an absent bus means the firmware found nothing to probe. -Unchanged across a reboot. +The imaging pipeline was up (`pisp_be` loaded, `/dev/media0-2` present), but +`/sys/bus/i2c/devices/` held only `i2c-13` and `i2c-14` — **no camera i2c bus and no CFE bound** — +and `dmesg` had no sensor probe lines. On the Pi, `camera_auto_detect=1` loads an overlay when it +finds something, so an absent bus means the firmware found nothing to probe. Unchanged across a +reboot, with the module's IR LEDs lit. -**Confirmed a cable fault; the module itself may also be damaged.** A second module is being tried on -an Orin Nano. +⚠ **Lit IR LEDs prove power, not a working sensor.** The illuminator is wired to the 3.3V rail +independently of the i2c and CSI lanes, so it lights whenever the ribbon is seated well enough to +carry power — while a creased or partly-seated cable can still have broken exactly the i2c traces +detection depends on. Which is what happened. ⚠ **Do not treat `/dev/video*` as evidence of a camera.** Those nodes exist on both boards with -nothing attached — on the Pi 5 they are the codec and ISP blocks. +nothing attached — on the Pi 5 they are the codec and ISP blocks, on the Orin +`tegra-camrtc-ca`/`/dev/media0` is the VI platform block. ### Checking a connection @@ -71,10 +117,14 @@ dmesg | grep -iE 'imx|ov5647|cfe' ls /sys/bus/i2c/devices/ # a camera bus should appear # Orin Nano +grep -i OVERLAYS /boot/extlinux/extlinux.conf # ⚠ CHECK THIS FIRST — no line, no camera v4l2-ctl --list-devices dmesg | grep -iE 'imx|camera|argus|vi:' ``` +On the Orin, check the overlay **before** suspecting hardware. An unconfigured Jetson and a dead +camera look exactly alike. + Cable notes worth keeping, since they cost a module here: - The **Pi 5 uses the narrow 22-pin FPC**; Pi 4-era modules ship with a **15-pin** cable and need the @@ -85,7 +135,8 @@ Cable notes worth keeping, since they cost a module here: ## Planned stages -1. **Capture** — get a sensor detected on the target, grab a still, establish resolution and format. +1. ~~**Capture** — get a sensor detected on the target, grab a still, establish resolution and + format.~~ **Done on the Orin** (IMX219, `/dev/video0`, `RG10`, full frame captured). 2. **Capture abstraction** — one interface, a backend per platform, chosen at runtime. 3. **Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later only if latency demands it.