Add recorder: raw CAN and GNSS sessions with failsafes and a storage bound

Reads SocketCAN and gpsd directly. Sessions start on the first frame or on
'ctl start', end after 15 s of bus idle or on 'ctl stop'; blips under 3 s are
discarded. candump -L chunks fdatasync'd every second and zstd'd once closed;
oldest sessions deleted to stay under 50 GB / above 20 GB free. Kernel and
queue drops, link changes and clock steps are recorded as events.

Bench-tested on vcan with the first car capture (byte-identical) and installed
on the car Orin as yukart-recorder.service.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDwfd6Ki8UfG4hK6W5YpJz
This commit is contained in:
2026-10-09 20:15:13 +09:00
commit 1f678fe87f
19 changed files with 1784 additions and 0 deletions
+108
View File
@@ -0,0 +1,108 @@
---
type: plan
status: active
updated: 2026-10-09
---
# Initial plan
## Order
1. **`recorder`** — raw CAN and GNSS to disk. Built first because every later
piece is developed against what it records.
2. `obd-poller` — standard OBD-II PIDs at a low rate, the cross-check for
reverse-engineered signals. The first transmitter: `can_pcan` leaves
listen-only for it, a deliberate step.
3. `can-bridge` and `gnss` — publish onto the broker once it exists (project-yukart
Phase 3; MQTT at ~890 frames/s is still unproven).
4. `decoder` — frames to named signals via the DBC in `contracts`.
## Recorder
Reads SocketCAN and gpsd directly. It is tier 1, and the broker is a single point
of failure that raw capture should not share. When `can-bridge` exists, the
recorder still owns raw capture; it subscribes to the bus only for decoded
signals.
### Session triggers
| Trigger | Starts on | Ends on |
|---|---|---|
| auto | the first CAN frame | `--idle` (15 s) with no frames |
| manual | `ctl start [label]` | `ctl stop`, or shutdown |
- **No traffic, no session.** The PEAK attached at a desk, attached to a parked
car with the bus asleep, or unplugged, produces nothing — the socket is retried
every second in silence.
- **Cranking stays inside one session.** A bus dropout or adapter re-enumeration
shorter than the idle timeout is recorded as `can_link` events, not as a split.
- **Blips are discarded.** An auto session with under `--min-secs` (3 s) of
traffic and no marks is deleted at close — door unlocks and brief plug-ins.
- `ctl start` during an auto session promotes it to manual. After `ctl stop`, auto
start waits for the bus to go idle, so a stop sticks.
- `ctl auto off` (or `--no-auto`) leaves only manual sessions.
The power service planned in orin-nano-env-setup (`docs/impl/plans/POWER.md`)
is the intended second client: `ctl mark crank` for a recovered input loss, and
`systemctl stop yukart-recorder` before poweroff.
### Session layout
```
/var/lib/yukart/recordings/
.last-seq high-water mark; numbers are never reused
000042-20261009T163736/ sequence number orders; the local time is for humans
session.json status open → closed | truncated; trigger, label, host,
boot_id, start/end realtime + monotonic + clock sync,
frame and drop counts, end reason
can-0001.log.zst candump -L, closed chunks (5 min or 64 MB), zstd
can-0002.log the open chunk, plain
gnss.ndjson {"rx": <local receive time>, "msg": <gpsd report>}
events.ndjson marks, can_link, gnss_link, clock_step, drops, storage_bound
```
`zstdcat can-*.log.zst can-*.log | canplayer` replays a session.
### Failsafes
| Failure | Behaviour |
|---|---|
| Power loss, crash, SIGKILL | open chunk is fdatasync'd every second; at the next start the session is marked `truncated` and its chunks compressed |
| Crash mid-compression | compression writes `.zst.tmp` then renames; the plain chunk is removed only after |
| Adapter unplugged, link down | reopen every second; recorded as `can_link` events |
| gpsd down | reconnect every 3 s; CAN capture unaffected |
| Writer slower than the bus | 100 000-frame queue (~110 s); overflow counted as `queue_drops` |
| Kernel socket overflow | `SO_RXQ_OVFL` counted as `kernel_drops` |
| Disk filling | oldest closed sessions deleted to stay under `--cap-gb` (50) and above `--min-free-gb` (20); the open session is never deleted |
| Control socket fails | logged; auto capture continues |
Measured on the first capture: ~33 KB/s of candump text, ~5.5 KB/s after zstd —
**about 20 MB per hour of driving**. The 50 GB default holds years of it; the cap
is there for the failure where something else fills the disk.
### Time
The car unit has no RTC battery yet and can start with a stale clock that chrony
steps later. CAN frames carry the kernel's receive timestamp (`SO_TIMESTAMP`,
CLOCK_REALTIME), and GNSS reports carry their local receive time — **the same
clock**, so CAN and GNSS line up within a session however wrong that clock is.
To get absolute time back afterwards:
- `session.json` records the kernel's sync state at start and end.
- `clock_step` events record every step over 50 ms with its size.
- Each TPV holds GNSS UTC beside its local receive time; their difference is the
clock error, plus the SIM7600's NMEA latency (no PPS: expect tens to hundreds
of milliseconds, constant enough to calibrate out).
At 1 Hz GNSS against ~890 frames/s, alignment to a few hundred milliseconds is
what this gives. It is enough to match CAN speed against GNSS speed; it is not
enough to time individual frames against position.
## Open
- **Session marking from the app.** `app-flutter`'s mark button reaches the
recorder through `svc-gw`, which will forward to the control socket.
- **Native builds.** `scripts/deploy-recorder.sh` cross-compiles from the
workstation while the development Orin is offline; `check-glibc.sh` gates it.
- **Copy-off.** Recordings stay on the car until the parked-at-home step in the
power plan, or a manual `rsync`.