Mikkeli
d10574191c
An IMX219 is live on the Orin's CAM0 at /dev/video0, verified by capture rather than by a bound driver: a 16,163,840-byte frame, exactly 3280x2464x2 for 10-bit Bayer, 99.8% non-zero across 40 distinct values. Real sensor data, not an allocated buffer. ⚠ The fault was configuration, and it is indistinguishable from dead hardware. The Pi auto-detects cameras; the Jetson does not. Until the sensor's device-tree overlay is selected there is no /dev/video0, no sensor line in dmesg and nothing on i2c — the same evidence a broken cable produces, which is how it got blamed on a cable here. Fixed with config-by-hardware.py, and the README now says to check the OVERLAYS line BEFORE suspecting hardware. Two traps recorded with it: the -n flag needs the header number (2= for the CSI connector) or it defaults to the 40-pin header and reports the module as unsupported, and v4l-utils is not installed by default so v4l2-ctl reports "command not found", which reads as "no camera". The Pi 5 fault was genuinely a cable and is kept for contrast, along with why its lit IR LEDs proved nothing: the illuminator sits on the 3.3V rail independently of the i2c and CSI lanes, so it lights whenever the ribbon carries power, even when the traces detection needs are broken. AGENTS.md also gains the rules that came out of a 27-attempt loop in this repo: write files with apply_patch and never write_stdin, never invent or increment a session id, stop after two identical failures instead of retrying, and emit tool calls as JSON arguments rather than leaking <parameter=...> markup into strings.
103 lines
5.3 KiB
Markdown
103 lines
5.3 KiB
Markdown
# Working in this repo
|
|
|
|
Project instructions for Codex. The client-wide prompt lives in `~/.codex/AGENTS.md`; this file is
|
|
the project-specific part.
|
|
|
|
## Know which board you are on
|
|
|
|
This project spans **two different single-board computers with incompatible camera stacks**, and
|
|
Codex is installed on both. Getting this wrong produces code that runs where you tested it and
|
|
nowhere else.
|
|
|
|
- **Jetson Orin Nano** — the *target*. L4T / JetPack, CUDA, TensorRT. V4L2 / GStreamer, with Argus
|
|
(`nvarguscamerasrc`) for CSI Bayer sensors.
|
|
- **Raspberry Pi 5** — the *test platform*. libcamera / `rpicam` / `picamera2`. CPU-only inference.
|
|
|
|
**Call `platform_info` before writing anything platform-specific.** It is client-local and reports
|
|
the machine you are actually on, not `halogen`. Do not infer the board from the fact that both are
|
|
aarch64 — that is the one thing they have in common.
|
|
|
|
⚠ **Code written directly against `picamera2` will not run on the Orin.** Put capture behind an
|
|
interface with a backend per platform, selected at runtime from what the hardware reports. Everything
|
|
above capture — streaming, UI, recognition — depends only on "a source of frames".
|
|
|
|
⚠ **The Pi proves the pipeline, never the performance.** Face recognition on the Orin goes through
|
|
TensorRT on GPU/DLA; on the Pi it is CPU-only and will not hold a live stream. Never present a Pi
|
|
timing as evidence the target is fast enough, and say which board a measurement came from.
|
|
|
|
## Camera state
|
|
|
|
**Orin Nano: working.** IMX219 on CAM0 at `/dev/video0`, format `RG10` (10-bit Bayer), full frame
|
|
captured and verified as real data. This is the board to develop capture against.
|
|
|
|
**Pi 5: dead.** Not detected; a cable fault, with the module possibly damaged. Nothing to do there
|
|
until the hardware is replaced.
|
|
|
|
⚠ **The two boards failed for completely different reasons, and the fixes do not transfer.** The
|
|
Orin was a missing device-tree overlay — a configuration fault that looks exactly like dead hardware.
|
|
The Pi was genuinely a broken cable. See README.md.
|
|
|
|
⚠ **On the Pi, you cannot fix camera detection from software.** Do not add `dtoverlay=` lines or edit
|
|
`/boot/firmware/config.txt`; `camera_auto_detect=1` is already correct there. On the Orin the overlay
|
|
*is* the mechanism, but it is already configured — do not change it without being asked.
|
|
|
|
⚠ **`/dev/video*` is not evidence of a camera.** Those nodes exist on both boards with nothing
|
|
attached.
|
|
|
|
If you are ever on a board with no working camera, work that does not need live capture is still
|
|
available: the capture interface and a synthetic backend, streaming plumbing, the web UI, tests.
|
|
**Say plainly when you are working against a placeholder** rather than a real frame.
|
|
|
|
## ⚠ Writing files, and not looping
|
|
|
|
**Write files with `apply_patch`**, or a heredoc through the shell. **Never** use `write_stdin` to
|
|
create file contents — it writes to the stdin of an already-running process, which is a different
|
|
thing and will not create anything. `write_stdin` takes a session id that an actual `exec_command`
|
|
returned to you; never invent one and never increment one.
|
|
|
|
**If the same tool call fails twice with the same error, STOP and report it.** Do not retry, and do
|
|
not vary an identifier hoping one works. A repeated identical failure means the assumption is wrong,
|
|
not that the call needs another attempt.
|
|
|
|
This happened here, in this repo. A session got as far as creating `src/camera_webui/capture/`, then
|
|
tried to write `base.py` via `write_stdin`, was told `Unknown process id 45`, and answered by
|
|
incrementing the id — 46, 47, 48 — for **27 attempts over nine minutes** until a human killed it.
|
|
|
|
⚠ **Emit tool calls as the API's JSON arguments only.** If you find yourself writing markup like
|
|
`<parameter=...>` inside a string argument, you are mixing in a different tool-calling format and the
|
|
call will not do what you mean. That is what the loop above degenerated into.
|
|
|
|
## Constraints
|
|
|
|
- **Install capture libraries from system packages, not pip.** `python3-picamera2` on the Pi; the
|
|
Jetson camera stack ships with L4T. Both bind to system libraries and a pip build will not match.
|
|
- **Do not commit captured images or video.** Frames of a real room are not test fixtures. Generate a
|
|
synthetic fixture if one is genuinely needed.
|
|
- Keep dependencies few. Everything here has to build on aarch64, and on the Jetson it has to
|
|
coexist with a vendor-pinned CUDA and Python.
|
|
- Face recognition runs on **downscaled frames, off the capture thread**.
|
|
|
|
## Verifying your work
|
|
|
|
Claims about hardware, the stream, or performance need a command that ran:
|
|
|
|
```bash
|
|
# is a sensor present
|
|
rpicam-hello --list-cameras # Pi 5
|
|
v4l2-ctl --list-devices # Orin
|
|
|
|
# is the server actually serving frames
|
|
curl -sI http://localhost:<port>/
|
|
```
|
|
|
|
⚠ **The absence of an error is not evidence that something works.** A stream endpoint returning 200
|
|
with no frames and a working one are indistinguishable to `curl -o /dev/null` — check what actually
|
|
came back. The same applies to a capture backend that constructs cleanly and yields nothing.
|
|
|
|
## Git
|
|
|
|
The remote is Gitea at `192.168.2.199:3005`, reachable over SSH on port 2222.
|
|
|
|
⚠ **`tea` (the Gitea CLI) is denied by policy** and blocked by a `PreToolUse` hook. Ordinary `git` is
|
|
fine. **Do not push** unless the operator asks — commit locally and say what is ready.
|