mikkeli
3827c5c381
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DDwfd6Ki8UfG4hK6W5YpJz
143 lines
7.0 KiB
Markdown
143 lines
7.0 KiB
Markdown
---
|
||
type: report
|
||
date: 2026-10-11
|
||
---
|
||
# USB camera (IMX662, Sunplus UVC bridge)
|
||
|
||
First look at the camera on the car Orin: what it presents over USB, what it
|
||
actually delivers, and what that means for recording and for the tablet link.
|
||
Indoor bench measurements, camera pointing at a lit ceiling.
|
||
|
||
## Short version
|
||
|
||
- A generic **UVC webcam**: `uvcvideo` and `snd-usb-audio` bind with no setup,
|
||
`/dev/video0` (capture) and `/dev/video1` (metadata), plus a USB microphone.
|
||
- **MJPEG only** at useful rates (up to 1920×1080 at 60 fps, nominally). Raw
|
||
YUYV is 5 fps at 1080p. **No H.264 from the camera, and the Orin Nano has no
|
||
hardware video encoder** (no NVENC; no `nvhost-*enc` device nodes).
|
||
- **The frame rate follows the light.** With auto exposure indoors it gives
|
||
17–20 fps whatever was requested; with a fixed 10 ms exposure it gives 27.6 fps
|
||
at 1080p30 and 55.9 fps at 720p60 (but a near-black image indoors).
|
||
- **MJPEG is heavy:** about 245 KB per 1080p frame and 113 KB per 720p frame,
|
||
for this scene. At a full 30 fps that is ~7.3 MB/s and ~3.4 MB/s. A busier
|
||
road scene will likely be larger.
|
||
- **Software H.264 works:** 1080p MJPEG → `jpegdec` → `x264enc` ultrafast at
|
||
6 Mbit/s took **1.3 cores at 19 fps** (≈ 2 cores at 30 fps, of 6); `veryfast`
|
||
took 2.4 cores. Output ~0.5 MB/s for this static scene.
|
||
|
||
## Device
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| USB ID | `1bcf:2d4f`, Sunplus Innovation Technology, product "USB Camera 1080P" |
|
||
| Sensor | Sony IMX662 per the seller (1/2.8", STARVIS 2). **Not visible over USB**: the bridge identifies only itself |
|
||
| Speed | USB 2.0 high-speed (480 Mbit/s) |
|
||
| Topology | Bus 1, behind the carrier board's internal USB 2.0 hub, **shared with the SIM7600 modem** |
|
||
| Interfaces | UVC video (`/dev/video0` capture, `/dev/video1` metadata); UAC audio (ALSA card `U1080P`, capture only: S16_LE, 2 channels, 48 kHz) |
|
||
| Driver | In-tree `uvcvideo`, `snd-usb-audio`; nothing to build |
|
||
| Documentation | None beyond the seller's listing; everything here is from the device itself |
|
||
|
||
### Formats
|
||
|
||
| Format | Sizes | Frame rates |
|
||
|---|---|---|
|
||
| MJPEG | 1920×1080, 1280×960, 1280×720, 960×536, 800×600, 640×480, 320×240 | 60, 30, 25, 20, 15, 10, 5 at every size |
|
||
| YUYV 4:2:2 | 1920×1080, 1280×960 | 5, 2 |
|
||
| | 1280×720, 960×536 | 10, 5, 2 |
|
||
| | 800×600 | 20 and below |
|
||
| | 640×480 | 30 and below |
|
||
| | 320×240 | 60 and below |
|
||
|
||
### Controls (`v4l2-ctl -d /dev/video0 --list-ctrls-menus`)
|
||
|
||
Brightness, contrast, saturation, hue, gamma, gain, sharpness, backlight
|
||
compensation; white balance auto or 2800–6500 K; **power-line frequency**
|
||
(currently 60 Hz, default 50 Hz: right for western Japan, flicker otherwise);
|
||
auto exposure (*aperture priority*) or manual, exposure 1–10000 in 100 µs units;
|
||
pan, tilt and zoom (digital).
|
||
|
||
UVC controls reset when the camera is replugged or re-enumerated: whatever
|
||
captures from it should set them on open, not rely on the last session.
|
||
|
||
## Measurements
|
||
|
||
### Frame rate and MJPEG size
|
||
|
||
| Mode requested | Exposure | Delivered | MJPEG per frame | Rate |
|
||
|---|---|---|---|---|
|
||
| 1920×1080 @ 30 | auto | 17.5 fps | 243 KB | 4.35 MB/s |
|
||
| 1920×1080 @ 60 | auto | 19.6 fps | 252 KB | 5.05 MB/s |
|
||
| 1280×720 @ 30 | auto | 19.1 fps | 113 KB | 2.22 MB/s |
|
||
| 1280×720 @ 60 | auto | 19.6 fps | 113 KB | 2.28 MB/s |
|
||
| 1920×1080 @ 30 | fixed 10 ms | 27.6 fps | 61 KB (dark) | — |
|
||
| 1280×720 @ 60 | fixed 10 ms | 55.9 fps | 29 KB (dark) | — |
|
||
|
||
Frame size depends on the scene and the exposure: a dark frame compresses to a
|
||
quarter of a lit one. Daylight in a car should allow full frame rate with auto
|
||
exposure; at night the IMX662's low-light sensitivity is the reason to have it,
|
||
and the frame rate will drop.
|
||
|
||
### Software H.264 (1080p, 200 frames, camera-limited at ~19 fps)
|
||
|
||
`v4l2src ! image/jpeg,1920x1080 ! jpegdec ! videoconvert ! x264enc tune=zerolatency bitrate=6000 key-int-max=60 ! h264parse ! matroskamux`
|
||
|
||
| `speed-preset` | CPU (whole system, idle baseline 0.01) | Output |
|
||
|---|---|---|
|
||
| ultrafast | 1.29 cores at 18.8 fps | 5.2 MB per 10.6 s ≈ 0.49 MB/s |
|
||
| veryfast | 2.40 cores at 19.0 fps | 5.2 MB per 10.5 s ≈ 0.49 MB/s |
|
||
|
||
The hardware JPEG decoder (`nvjpegdec`) rejected the camera's frames (*no valid
|
||
frames found*). UVC MJPEG commonly omits the Huffman tables, which `jpegdec`
|
||
fills in and NVIDIA's decoder does not; not investigated further. The CPU
|
||
figures include software JPEG decoding.
|
||
|
||
### Manual exposure and gain (1280×720, 60 fps requested, same scene)
|
||
|
||
Both controls are honored and read back as set. Brightness is the frame mean (0–255);
|
||
noise is the standard deviation over a flat patch of ceiling.
|
||
|
||
| Exposure | Gain | Delivered | Mean | Flat patch: mean / std |
|
||
|---|---|---|---|---|
|
||
| auto | auto | 18.7 fps | 137 | 160 / 3.9 |
|
||
| 0.1 ms | 0 | 50 fps | 3 | 2 / 0.1 |
|
||
| 0.1 ms | 240 | 50 fps | 135 | 151 / **31.3** |
|
||
| 1 ms | 240 | 50 fps | 219 | 250 (clipped) |
|
||
| 10 ms | 0 | 50 fps | 9 | 7 / 0.9 |
|
||
| 16.6 ms | 0 | 50 fps | 16 | 12 / 1.0 |
|
||
| 33.3 ms | 0 | **23 fps** | 31 | 26 / 1.9 |
|
||
| 100 ms requested | 0 | **16.5 fps** | 44 | 41 / 2.5 |
|
||
|
||
- **Frame rate holds to ~16.6 ms of exposure**, then the frame time follows the
|
||
exposure. Auto exposure takes that trade freely, and **nothing tells it not to**:
|
||
there is no `exposure_dynamic_framerate` control. A pipeline that needs a fixed
|
||
rate has to run exposure manually and do its own metering.
|
||
- **Exposure tops out near 50–60 ms**, whatever the control's 1 s range says:
|
||
asking for 100 ms gave 1.45× the brightness of 33 ms and a 60 ms frame time.
|
||
- **Gain is very large:** 0.1 ms at full gain matches auto exposure's brightness,
|
||
roughly 1000–2000× over gain 0, and is very noisy (std 31 against 4).
|
||
- **Sensor identity stays unproven.** The gain range is consistent with a
|
||
STARVIS-class sensor; a night test against a reference camera would settle
|
||
low-light performance, and the bridge cannot.
|
||
|
||
## What it means
|
||
|
||
| Use | Fits? |
|
||
|---|---|
|
||
| **Live view on the tablet** (link: ~5 MB/s Orin → tablet, paced; app-flutter `docs/impl/reports/LINK-TEST-2026-10-11.md`) | 720p MJPEG passthrough at 30 fps, ~3.4 MB/s: yes, no CPU. 1080p MJPEG at 30 fps, ~7.3 MB/s: no. H.264 at any size: yes, at 1.3–2 cores |
|
||
| **Recording** (recorder bound: 50 GB) | MJPEG at 1080p30 is ~26 GB per hour: no. H.264 at 6 Mbit/s is ~2.7 GB per hour: yes, ~18 hours in the bound |
|
||
| **Inference** | Frames are local; decoding is the cost. No constraint from the link |
|
||
| **ASR microphone** | Possible, as a USB capture device on the Orin; quality and placement untested |
|
||
|
||
The USB 2.0 hub it shares with the modem has headroom: 7.3 MB/s of MJPEG is
|
||
~60 Mbit/s of 480, and LTE Cat 4 tops out at 150 Mbit/s down.
|
||
|
||
## Open
|
||
|
||
- Daylight and night frame rates and frame sizes in the car, on a real road
|
||
scene; noise at the gain night driving needs.
|
||
- Mounting and orientation (the image can be flipped in the pipeline; the
|
||
camera has no flip control).
|
||
- Whether the bridge's MJPEG can be made acceptable to `nvjpegdec` (inserting
|
||
standard Huffman tables), which would free about a core.
|
||
- The microphone: quality, and whether it is the ASR input (Whisper takes 16 kHz mono; resampling is trivial).
|