mikkeli
1f678fe87f
Reads SocketCAN and gpsd directly. Sessions start on the first frame or on 'ctl start', end after 15 s of bus idle or on 'ctl stop'; blips under 3 s are discarded. candump -L chunks fdatasync'd every second and zstd'd once closed; oldest sessions deleted to stay under 50 GB / above 20 GB free. Kernel and queue drops, link changes and clock steps are recorded as events. Bench-tested on vcan with the first car capture (byte-identical) and installed on the car Orin as yukart-recorder.service. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DDwfd6Ki8UfG4hK6W5YpJz
109 lines
5.1 KiB
Markdown
109 lines
5.1 KiB
Markdown
---
|
|
type: plan
|
|
status: active
|
|
updated: 2026-10-09
|
|
---
|
|
# Initial plan
|
|
|
|
## Order
|
|
|
|
1. **`recorder`** — raw CAN and GNSS to disk. Built first because every later
|
|
piece is developed against what it records.
|
|
2. `obd-poller` — standard OBD-II PIDs at a low rate, the cross-check for
|
|
reverse-engineered signals. The first transmitter: `can_pcan` leaves
|
|
listen-only for it, a deliberate step.
|
|
3. `can-bridge` and `gnss` — publish onto the broker once it exists (project-yukart
|
|
Phase 3; MQTT at ~890 frames/s is still unproven).
|
|
4. `decoder` — frames to named signals via the DBC in `contracts`.
|
|
|
|
## Recorder
|
|
|
|
Reads SocketCAN and gpsd directly. It is tier 1, and the broker is a single point
|
|
of failure that raw capture should not share. When `can-bridge` exists, the
|
|
recorder still owns raw capture; it subscribes to the bus only for decoded
|
|
signals.
|
|
|
|
### Session triggers
|
|
|
|
| Trigger | Starts on | Ends on |
|
|
|---|---|---|
|
|
| auto | the first CAN frame | `--idle` (15 s) with no frames |
|
|
| manual | `ctl start [label]` | `ctl stop`, or shutdown |
|
|
|
|
- **No traffic, no session.** The PEAK attached at a desk, attached to a parked
|
|
car with the bus asleep, or unplugged, produces nothing — the socket is retried
|
|
every second in silence.
|
|
- **Cranking stays inside one session.** A bus dropout or adapter re-enumeration
|
|
shorter than the idle timeout is recorded as `can_link` events, not as a split.
|
|
- **Blips are discarded.** An auto session with under `--min-secs` (3 s) of
|
|
traffic and no marks is deleted at close — door unlocks and brief plug-ins.
|
|
- `ctl start` during an auto session promotes it to manual. After `ctl stop`, auto
|
|
start waits for the bus to go idle, so a stop sticks.
|
|
- `ctl auto off` (or `--no-auto`) leaves only manual sessions.
|
|
|
|
The power service planned in orin-nano-env-setup (`docs/impl/plans/POWER.md`)
|
|
is the intended second client: `ctl mark crank` for a recovered input loss, and
|
|
`systemctl stop yukart-recorder` before poweroff.
|
|
|
|
### Session layout
|
|
|
|
```
|
|
/var/lib/yukart/recordings/
|
|
.last-seq high-water mark; numbers are never reused
|
|
000042-20261009T163736/ sequence number orders; the local time is for humans
|
|
session.json status open → closed | truncated; trigger, label, host,
|
|
boot_id, start/end realtime + monotonic + clock sync,
|
|
frame and drop counts, end reason
|
|
can-0001.log.zst candump -L, closed chunks (5 min or 64 MB), zstd
|
|
can-0002.log the open chunk, plain
|
|
gnss.ndjson {"rx": <local receive time>, "msg": <gpsd report>}
|
|
events.ndjson marks, can_link, gnss_link, clock_step, drops, storage_bound
|
|
```
|
|
|
|
`zstdcat can-*.log.zst can-*.log | canplayer` replays a session.
|
|
|
|
### Failsafes
|
|
|
|
| Failure | Behaviour |
|
|
|---|---|
|
|
| Power loss, crash, SIGKILL | open chunk is fdatasync'd every second; at the next start the session is marked `truncated` and its chunks compressed |
|
|
| Crash mid-compression | compression writes `.zst.tmp` then renames; the plain chunk is removed only after |
|
|
| Adapter unplugged, link down | reopen every second; recorded as `can_link` events |
|
|
| gpsd down | reconnect every 3 s; CAN capture unaffected |
|
|
| Writer slower than the bus | 100 000-frame queue (~110 s); overflow counted as `queue_drops` |
|
|
| Kernel socket overflow | `SO_RXQ_OVFL` counted as `kernel_drops` |
|
|
| Disk filling | oldest closed sessions deleted to stay under `--cap-gb` (50) and above `--min-free-gb` (20); the open session is never deleted |
|
|
| Control socket fails | logged; auto capture continues |
|
|
|
|
Measured on the first capture: ~33 KB/s of candump text, ~5.5 KB/s after zstd —
|
|
**about 20 MB per hour of driving**. The 50 GB default holds years of it; the cap
|
|
is there for the failure where something else fills the disk.
|
|
|
|
### Time
|
|
|
|
The car unit has no RTC battery yet and can start with a stale clock that chrony
|
|
steps later. CAN frames carry the kernel's receive timestamp (`SO_TIMESTAMP`,
|
|
CLOCK_REALTIME), and GNSS reports carry their local receive time — **the same
|
|
clock**, so CAN and GNSS line up within a session however wrong that clock is.
|
|
|
|
To get absolute time back afterwards:
|
|
|
|
- `session.json` records the kernel's sync state at start and end.
|
|
- `clock_step` events record every step over 50 ms with its size.
|
|
- Each TPV holds GNSS UTC beside its local receive time; their difference is the
|
|
clock error, plus the SIM7600's NMEA latency (no PPS: expect tens to hundreds
|
|
of milliseconds, constant enough to calibrate out).
|
|
|
|
At 1 Hz GNSS against ~890 frames/s, alignment to a few hundred milliseconds is
|
|
what this gives. It is enough to match CAN speed against GNSS speed; it is not
|
|
enough to time individual frames against position.
|
|
|
|
## Open
|
|
|
|
- **Session marking from the app.** `app-flutter`'s mark button reaches the
|
|
recorder through `svc-gw`, which will forward to the control socket.
|
|
- **Native builds.** `scripts/deploy-recorder.sh` cross-compiles from the
|
|
workstation while the development Orin is offline; `check-glibc.sh` gates it.
|
|
- **Copy-off.** Recordings stay on the car until the parked-at-home step in the
|
|
power plan, or a manual `rsync`.
|