project-yukart/docs/ORIN-RUNTIME.md

type

type
default

Orin runtime

ARCHITECTURE.md describes services and the bus without saying how they coexist on one 8 GB board. This document covers that: what runs, in what order, under what budget, and what happens when it goes wrong.

Cross-cutting by nature — it spans svc-core, svc-nav, svc-gw and the host repo, so it lives in the hub rather than in any one of them.

Layers

  clients        app-flutter, app-esp32          off the Orin
  ─────────────────────────────────────────────────────────
  serving        svc-gw, svc-nav
  processing     decoder, recorder
  acquisition    can-bridge, obd-poller, gnss
  bus            mosquitto
  ─────────────────────────────────────────────────────────
  host           kernel modules, gpsd, PipeWire, network     orin-nano-env-setup

A layer may depend on the one below it and on the bus. Nothing depends sideways: no service waits on another service's process, only on the broker.

The binding constraint: 8 GB, shared

The Orin Nano has 8 GB of LPDDR5 shared between CPU and GPU. There is no separate VRAM to fall back on — a model loaded onto the GPU takes memory away from everything else on the board. Roughly 6–7 GB is left once JetPack and the OS have taken their share.

Consumer	Resident	Note
OS + JetPack	~1–1.5 GB	before anything of ours starts
mosquitto, acquisition, svc-gw	tens of MB	not a factor
Valhalla server	small; tiles are mmap'd page cache	evictable — see below
Whisper	273 MB (tiny) → 3.9 GB (large)	small is 244 M params; large does not fit
GPT-SoVITS	upstream guidance assumes an 8 GB VRAM card	the largest unknown; measure before designing around it
vision (Phase 5)	—	competes directly with voice for the same pool

Four rules follow.

The cloud LLM is load-bearing. OVERVIEW.md puts reasoning in the cloud. That decision is what makes a local voice stack fit at all — moving the LLM on-device collapses this budget entirely. It is not a free choice to revisit.

Size Whisper by vocabulary, not by benchmark. In-car commands are a narrow, predictable set. large buys accuracy this application cannot spend, at a cost this board cannot pay.

Voice and vision cannot both be resident. Phase 5 has to plan for eviction, not addition. Worth knowing now, while vision is still deferred.

Valhalla's memory-mapped tiles are not a leak. They appear as page cache and are evictable. Under pressure the kernel drops tile pages and re-faults them on next use — that costs routing latency, not an OOM. This is the graceful failure in the system and should be left alone.

Startup tiers

Ordered by what is lost if a tier is late.

Tier	Units	Why here
0	kernel modules, gpsd, network	Host layer; nothing of ours can precede it
1	mosquitto, can-bridge, recorder	Data is lost if late. Must reach ready fastest
2	decoder, gnss, obd-poller	Derive from tier 1; a late start loses nothing
3	svc-gw, svc-nav	Serving; a display can wait
4	voice — ASR, TTS	Interactive; seconds late is harmless

Capture first is the whole principle. A CAN frame missed at tier 1 is gone forever; a map that loads five seconds into the drive costs nothing. Ordering the boot the intuitive way — get the screen up quickly — trades irreplaceable data for a cosmetic gain.

Failure isolation

• Restart=always on every unit. A service dying mid-drive is a normal event.
• Units order themselves After= the broker and nothing else. No service may wait on another service, or a restart cascades.
• The broker is the single point of failure, and that is accepted: retained topics make reconnection cheap, so a client that comes back has state on its first message rather than waiting for the next publish.
• No service holds another's state. Anything worth surviving a restart is either retained on the bus or on disk.

Ignition off

Shutdown is the reverse of startup, so tier 1 dies last.

The only thing that must complete is the recorder's flush. It therefore owns its own flush and shutdown handling rather than trusting the OS to get there in time — a truncated session is the one failure that cannot be recovered by restarting anything.

Storage

Self-built binaries live under /home/mikkeli/infra/ per the host repo's guidelines. Data does not.

Data	Size	Growth
Map artifacts	GB	Replaced wholesale, quarterly at most
Voice models	GB	Static
Recorded sessions	GB per drive	Unbounded

Recordings are the problem. Without a retention policy the disk fills mid-drive and capture stops — silently, because a full disk is not a crash. A retention policy is a tier-1 concern, not housekeeping.

Audio

The Orin owns the car's single aux input, via a USB DAC.

  GPT-SoVITS (TTS) ──┐
                     ├──▶ PipeWire mixer ──▶ USB DAC ──▶ aux in
  A2DP sink ─────────┘         │
  (media from the tablet)      └── ducking: media drops under TTS

  mic ──▶ Whisper (ASR)

Three reasons the aux cable belongs here rather than on the tablet:

• The aux cable should belong to the fixed unit. The display is meant to be removable; anything holding a permanent cable to the car is not removable.
• Ducking becomes a local policy on one box instead of coordination between two. Without it the assistant talks over the music.
• The direction works. Linux is a capable A2DP sink, so the tablet can send media here. The reverse — this box as an A2DP source into an Android sink — is not viable, since stock Android builds generally do not enable A2DP sink.

Unplug the tablet and the car keeps audio and voice. The Orin depends on the display unit for nothing.