--- type: default --- # Orin runtime [ARCHITECTURE.md](ARCHITECTURE.md) describes services and the bus without saying how they coexist on one 8 GB board. This document covers that: what runs, in what order, under what budget, and what happens when it goes wrong. Cross-cutting by nature — it spans `svc-core`, `svc-nav`, `svc-gw` and the host repo, so it lives in the hub rather than in any one of them. ## Layers ``` clients app-flutter, app-esp32 off the Orin ───────────────────────────────────────────────────────── serving svc-gw, svc-nav processing decoder, recorder acquisition can-bridge, obd-poller, gnss bus mosquitto ───────────────────────────────────────────────────────── host kernel modules, gpsd, PipeWire, network orin-nano-env-setup ``` A layer may depend on the one below it and on the bus. Nothing depends sideways: no service waits on another service's process, only on the broker. ## The binding constraint: 8 GB, shared The Orin Nano has 8 GB of LPDDR5 **shared between CPU and GPU**. There is no separate VRAM to fall back on — a model loaded onto the GPU takes memory away from everything else on the board. Roughly 6–7 GB is left once JetPack and the OS have taken their share. | Consumer | Resident | Note | |---|---|---| | OS + JetPack | ~1–1.5 GB | before anything of ours starts | | mosquitto, acquisition, `svc-gw` | tens of MB | not a factor | | Valhalla server | small; tiles are **mmap'd page cache** | evictable — see below | | Whisper | 273 MB (tiny) → 3.9 GB (large) | `small` is 244 M params; `large` does not fit | | GPT-SoVITS | upstream guidance assumes an 8 GB VRAM card | the largest unknown; **measure before designing around it** | | vision (Phase 5) | — | competes directly with voice for the same pool | Four rules follow. **The cloud LLM is load-bearing.** [OVERVIEW.md](../OVERVIEW.md) puts reasoning in the cloud. That decision is what makes a local voice stack fit at all — moving the LLM on-device collapses this budget entirely. It is not a free choice to revisit. **Size Whisper by vocabulary, not by benchmark.** In-car commands are a narrow, predictable set. `large` buys accuracy this application cannot spend, at a cost this board cannot pay. **Voice and vision cannot both be resident.** Phase 5 has to plan for eviction, not addition. Worth knowing now, while vision is still deferred. **Valhalla's memory-mapped tiles are not a leak.** They appear as page cache and are evictable. Under pressure the kernel drops tile pages and re-faults them on next use — that costs routing latency, not an OOM. This is the graceful failure in the system and should be left alone. ## Startup tiers Ordered by what is lost if a tier is late. | Tier | Units | Why here | |---|---|---| | 0 | kernel modules, gpsd, network | Host layer; nothing of ours can precede it | | 1 | `mosquitto`, `can-bridge`, `recorder` | **Data is lost if late.** Must reach ready fastest | | 2 | `decoder`, `gnss`, `obd-poller` | Derive from tier 1; a late start loses nothing | | 3 | `svc-gw`, `svc-nav` | Serving; a display can wait | | 4 | voice — ASR, TTS | Interactive; seconds late is harmless | **Capture first** is the whole principle. A CAN frame missed at tier 1 is gone forever; a map that loads five seconds into the drive costs nothing. Ordering the boot the intuitive way — get the screen up quickly — trades irreplaceable data for a cosmetic gain. ## Failure isolation - `Restart=always` on every unit. A service dying mid-drive is a normal event. - Units order themselves `After=` the broker and nothing else. No service may wait on another service, or a restart cascades. - The broker is the single point of failure, and that is accepted: retained topics make reconnection cheap, so a client that comes back has state on its first message rather than waiting for the next publish. - No service holds another's state. Anything worth surviving a restart is either retained on the bus or on disk. ## Ignition off Shutdown is the reverse of startup, so tier 1 dies last. The only thing that *must* complete is the recorder's flush. It therefore owns its own flush and shutdown handling rather than trusting the OS to get there in time — a truncated session is the one failure that cannot be recovered by restarting anything. ## Storage Self-built binaries live under `/home/mikkeli/infra/` per the host repo's guidelines. Data does not. | Data | Size | Growth | |---|---|---| | Map artifacts | GB | Replaced wholesale, quarterly at most | | Voice models | GB | Static | | Recorded sessions | GB per drive | **Unbounded** | Recordings are the problem. Without a bound the disk fills mid-drive and capture stops — silently, because a full disk is not a crash. **Recordings are bounded with rotation.** The recorder enforces its own ceiling and discards oldest-first; it does not rely on an external cleanup job that might not have run. Two properties matter more than the specific size: - The bound is enforced by the writer, so the failure mode is *losing the oldest session*, which is recoverable, rather than *silently stopping capture*, which is not. - A session in progress is never the one rotated away. ## Audio The **tablet** owns the car's single aux input. It is the music player, and that role is not moving to the Orin. ``` tablet: media (Spotify etc.) ──┐ ├──▶ Android mixer ──▶ aux in Orin: GPT-SoVITS (TTS) ──────┘ └── utterances sent over the reliable boundary channel Orin: mic ──▶ Whisper (ASR) stays local, never crosses ``` The tablet is the media source, so it holds the aux cable whatever is decided about speech. Giving the Orin its own aux line would not free the display unit — it would add a second permanent cable to the car and leave the first one in place. ### TTS crosses as utterances, not as a stream An utterance is short, finite, and fully synthesised before it is needed. So it ships as a **discrete object over the reliable channel**, not as a real-time audio stream: no jitter buffer, no underrun handling, no clock synchronisation, and a dropped packet is a retry rather than an audible gap. The cost is that speech begins only once synthesis *and* transfer complete. For a car assistant that is fine — nothing here is lip-synced. If streaming TTS is ever wanted, so that speech starts before synthesis finishes, this is the decision to revisit. ### Ducking is the tablet's job Android's `AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK` ducks other applications' audio — Spotify included — for the duration of a request, then restores it. This is the mechanism every navigation app already uses, and it is less work than the PipeWire policy it replaces. Cross-application ducking is a solved problem on the tablet and would have been a bespoke one on the Orin. ### The microphone stays on the Orin Whisper runs on the Orin, so its input does too. Audio *in* on one box and audio *out* on the other is asymmetric on purpose: each side does the simple thing, and a microphone has no reason to reach the head unit. ### Degrade when there is no sink With no tablet connected the Orin can still hear — the mic is local — but cannot speak. The voice stack expires or drops pending utterances rather than queueing them unboundedly or blocking on a sink that may never arrive. The consequence to accept: **unplug the tablet and the car has no voice.** That is tolerable because unplugging it also removes the music, so the tablet was already load-bearing for audio. Speech does not make it more so.