From 6aa0d940f6aad83f606fcf031188fb16b2f775db Mon Sep 17 00:00:00 2001 From: Mikkeli Date: Sun, 11 Oct 2026 17:21:10 +0900 Subject: [PATCH] Add the USB camera (IMX662, Sunplus UVC) to the BoM, with a first measurement report Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DDwfd6Ki8UfG4hK6W5YpJz --- docs/BOM.md | 19 ++++++ docs/impl/reports/USB-CAMERA.md | 114 ++++++++++++++++++++++++++++++++ 2 files changed, 133 insertions(+) create mode 100644 docs/impl/reports/USB-CAMERA.md diff --git a/docs/BOM.md b/docs/BOM.md index 724775a..5ee2dc8 100644 --- a/docs/BOM.md +++ b/docs/BOM.md @@ -77,6 +77,25 @@ type: reference The shunt value is unconfirmed; check the schematic before converting shunt mV to amps. - No GPIO or shutdown signal on the module; the BOOT button may be needed to wake a battery that stopped outputting (FAQ). +## USB camera (IMX662, Sunplus UVC bridge) + +| Field | Value | +|-------|-------| +| **Sensor** | Sony IMX662 per the seller (1/2.8", STARVIS 2, low-light); the USB bridge does not report it | +| **USB ID** | `1bcf:2d4f` (Sunplus Innovation Technology, "USB Camera 1080P"), USB 2.0 high-speed | +| **Purpose** | Camera for the Orin: recording, live view on the tablet, later inference (roles open; see project-yukart roadmap, Phase 5) | +| **Interface** | UVC (`uvcvideo`, `/dev/video0`) + UAC microphone (ALSA card `U1080P`, 48 kHz stereo). In-tree drivers, no setup | +| **Formats** | MJPEG up to 1920×1080 at 60 fps nominal; YUYV only at low rates (5 fps at 1080p). **No H.264** | +| **Documentation** | None beyond the seller's listing; facts from the device: [USB camera report](impl/reports/USB-CAMERA.md) | + +### Notes + +- Delivered frame rate follows the light: 17–20 fps indoors with auto exposure, full rate with a short fixed exposure. +- MJPEG is ~245 KB per 1080p frame, ~7.3 MB/s at 30 fps; too much for the tablet link (~5 MB/s) or for recording. The Orin Nano has + **no hardware video encoder**; software H.264 (`x264enc` ultrafast) costs ~1.3 cores at 19 fps, ~2 at 30. +- Shares the carrier board's internal USB 2.0 hub with the SIM7600; headroom is ample. +- UVC controls (exposure, white balance, power-line frequency) reset on re-enumeration; set them when opening the device. + ## RTC backup battery **Status: optional, low priority.** Battery choice settled (CR2032); footprint/R560 still to verify against NVIDIA's schematic. diff --git a/docs/impl/reports/USB-CAMERA.md b/docs/impl/reports/USB-CAMERA.md new file mode 100644 index 0000000..3aa9211 --- /dev/null +++ b/docs/impl/reports/USB-CAMERA.md @@ -0,0 +1,114 @@ +--- +type: report +date: 2026-10-11 +--- +# USB camera (IMX662, Sunplus UVC bridge) + +First look at the camera on the car Orin: what it presents over USB, what it +actually delivers, and what that means for recording and for the tablet link. +Indoor bench measurements, camera pointing at a lit ceiling. + +## Short version + +- A generic **UVC webcam**: `uvcvideo` and `snd-usb-audio` bind with no setup, + `/dev/video0` (capture) and `/dev/video1` (metadata), plus a USB microphone. +- **MJPEG only** at useful rates (up to 1920×1080 at 60 fps, nominally). Raw + YUYV is 5 fps at 1080p. **No H.264 from the camera, and the Orin Nano has no + hardware video encoder** (no NVENC; no `nvhost-*enc` device nodes). +- **The frame rate follows the light.** With auto exposure indoors it gives + 17–20 fps whatever was requested; with a fixed 10 ms exposure it gives 27.6 fps + at 1080p30 and 55.9 fps at 720p60 (but a near-black image indoors). +- **MJPEG is heavy:** about 245 KB per 1080p frame and 113 KB per 720p frame, + for this scene. At a full 30 fps that is ~7.3 MB/s and ~3.4 MB/s. A busier + road scene will likely be larger. +- **Software H.264 works:** 1080p MJPEG → `jpegdec` → `x264enc` ultrafast at + 6 Mbit/s took **1.3 cores at 19 fps** (≈ 2 cores at 30 fps, of 6); `veryfast` + took 2.4 cores. Output ~0.5 MB/s for this static scene. + +## Device + +| Field | Value | +|---|---| +| USB ID | `1bcf:2d4f`, Sunplus Innovation Technology, product "USB Camera 1080P" | +| Sensor | Sony IMX662 per the seller (1/2.8", STARVIS 2). **Not visible over USB**: the bridge identifies only itself | +| Speed | USB 2.0 high-speed (480 Mbit/s) | +| Topology | Bus 1, behind the carrier board's internal USB 2.0 hub, **shared with the SIM7600 modem** | +| Interfaces | UVC video (`/dev/video0` capture, `/dev/video1` metadata); UAC audio (ALSA card `U1080P`, capture only: S16_LE, 2 channels, 48 kHz) | +| Driver | In-tree `uvcvideo`, `snd-usb-audio`; nothing to build | +| Documentation | None beyond the seller's listing; everything here is from the device itself | + +### Formats + +| Format | Sizes | Frame rates | +|---|---|---| +| MJPEG | 1920×1080, 1280×960, 1280×720, 960×536, 800×600, 640×480, 320×240 | 60, 30, 25, 20, 15, 10, 5 at every size | +| YUYV 4:2:2 | 1920×1080, 1280×960 | 5, 2 | +| | 1280×720, 960×536 | 10, 5, 2 | +| | 800×600 | 20 and below | +| | 640×480 | 30 and below | +| | 320×240 | 60 and below | + +### Controls (`v4l2-ctl -d /dev/video0 --list-ctrls-menus`) + +Brightness, contrast, saturation, hue, gamma, gain, sharpness, backlight +compensation; white balance auto or 2800–6500 K; **power-line frequency** +(currently 60 Hz, default 50 Hz: right for western Japan, flicker otherwise); +auto exposure (*aperture priority*) or manual, exposure 1–10000 in 100 µs units; +pan, tilt and zoom (digital). + +UVC controls reset when the camera is replugged or re-enumerated: whatever +captures from it should set them on open, not rely on the last session. + +## Measurements + +### Frame rate and MJPEG size + +| Mode requested | Exposure | Delivered | MJPEG per frame | Rate | +|---|---|---|---|---| +| 1920×1080 @ 30 | auto | 17.5 fps | 243 KB | 4.35 MB/s | +| 1920×1080 @ 60 | auto | 19.6 fps | 252 KB | 5.05 MB/s | +| 1280×720 @ 30 | auto | 19.1 fps | 113 KB | 2.22 MB/s | +| 1280×720 @ 60 | auto | 19.6 fps | 113 KB | 2.28 MB/s | +| 1920×1080 @ 30 | fixed 10 ms | 27.6 fps | 61 KB (dark) | — | +| 1280×720 @ 60 | fixed 10 ms | 55.9 fps | 29 KB (dark) | — | + +Frame size depends on the scene and the exposure: a dark frame compresses to a +quarter of a lit one. Daylight in a car should allow full frame rate with auto +exposure; at night the IMX662's low-light sensitivity is the reason to have it, +and the frame rate will drop. + +### Software H.264 (1080p, 200 frames, camera-limited at ~19 fps) + +`v4l2src ! image/jpeg,1920x1080 ! jpegdec ! videoconvert ! x264enc tune=zerolatency bitrate=6000 key-int-max=60 ! h264parse ! matroskamux` + +| `speed-preset` | CPU (whole system, idle baseline 0.01) | Output | +|---|---|---| +| ultrafast | 1.29 cores at 18.8 fps | 5.2 MB per 10.6 s ≈ 0.49 MB/s | +| veryfast | 2.40 cores at 19.0 fps | 5.2 MB per 10.5 s ≈ 0.49 MB/s | + +The hardware JPEG decoder (`nvjpegdec`) rejected the camera's frames (*no valid +frames found*). UVC MJPEG commonly omits the Huffman tables, which `jpegdec` +fills in and NVIDIA's decoder does not; not investigated further. The CPU +figures include software JPEG decoding. + +## What it means + +| Use | Fits? | +|---|---| +| **Live view on the tablet** (link: ~5 MB/s Orin → tablet, paced; app-flutter `docs/impl/reports/LINK-TEST-2026-10-11.md`) | 720p MJPEG passthrough at 30 fps, ~3.4 MB/s: yes, no CPU. 1080p MJPEG at 30 fps, ~7.3 MB/s: no. H.264 at any size: yes, at 1.3–2 cores | +| **Recording** (recorder bound: 50 GB) | MJPEG at 1080p30 is ~26 GB per hour: no. H.264 at 6 Mbit/s is ~2.7 GB per hour: yes, ~18 hours in the bound | +| **Inference** | Frames are local; decoding is the cost. No constraint from the link | +| **ASR microphone** | Possible, as a USB capture device on the Orin; quality and placement untested | + +The USB 2.0 hub it shares with the modem has headroom: 7.3 MB/s of MJPEG is +~60 Mbit/s of 480, and LTE Cat 4 tops out at 150 Mbit/s down. + +## Open + +- Daylight and night frame rates and frame sizes in the car, on a real road + scene. +- Mounting and orientation (the image can be flipped in the pipeline; the + camera has no flip control). +- Whether the bridge's MJPEG can be made acceptable to `nvjpegdec` (inserting + standard Huffman tables), which would free about a core. +- The microphone: quality, and whether it is the ASR input (Whisper takes 16 kHz mono; resampling is trivial).