Persistent journal on the car unit, 1 GB cap, 30 s sync

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DDwfd6Ki8UfG4hK6W5YpJz
2026-10-13 22:44:10 +09:00
parent 654ec96e3b
commit 1551e5648a
1 changed files with 61 additions and 0 deletions
@@ -0,0 +1,61 @@
+---
+type: report
+date: 2026-10-13
+---
+# Persistent journal on the car unit
+
+## Why
+
+On 2026-10-13 the car Orin hung (unreachable over WiFi, LTE and the tablet's USB
+link) during repeated failed BLE connections to the ELM327, and needed a hard power
+cycle. Nothing could be said about the cause afterwards: the journal was volatile.
+JetPack ships `Storage=auto` with no `/var/log/journal`, so logs lived in `/run`
+and died with the boot.
+
+## What is set
+
+`/etc/systemd/journald.conf.d/50-yukart-persistent.conf`:
+
+```ini
+[Journal]
+Storage=persistent
+SystemMaxUse=1G
+SystemKeepFree=10G
+SystemMaxFileSize=64M
+SyncIntervalSec=30s
+```
+
+- **1 GB cap:** weeks of logs at the current volume (16 MB for the first boot's
+  half hour, mostly boot noise), next to a 467 GB NVMe. `SystemKeepFree` stops
+  the journal growing into a nearly full disk, which the recorder needs first.
+- **64 MB files:** rotation drops old logs in small steps.
+- **Sync every 30 s** (default 5 min): a hang or power loss costs at most the last
+  30 s of ordinary messages. `crit` and worse are synced at once regardless.
+
+Applied by hand (install time, sudo):
+
+```bash
+sudo mkdir -p /etc/systemd/journald.conf.d /var/log/journal
+# write the file above
+sudo systemd-tmpfiles --create --prefix /var/log/journal
+sudo systemctl restart systemd-journald
+sudo journalctl --flush
+```
+
+## Use
+
+```bash
+journalctl --list-boots              # one line per boot, kept across reboots
+journalctl -b -1 -e                  # the end of the previous boot: how it ended
+journalctl -b -1 -k                  # its kernel messages (driver crashes, USB resets)
+journalctl --disk-usage
+```
+
+`mikkeli` reads the journal through the `adm` group; no sudo needed.
+
+A boot that ended cleanly ends in the shutdown sequence. One that ends mid-stream
+was a hang or a power loss; the last kernel messages before the gap are the lead.
+Timestamps of an early boot can be off until NTP or GPS sets the clock (no RTC
+battery), so order by `-o short-monotonic` within a boot.
+
+Only the car unit (`mikkeli-orin-nano`) has this. The development Orin is unchanged.