mikkeli
d234a2da37
Replaces `type: default` — one value across every document, read by nothing, so it discriminated nothing. Types are now reference, guidelines, plan and report, applied to every markdown file with no exceptions for the hub. Plans carry status and updated, reports carry date. REPOS.md already noted that plans go stale and reports do not, but nothing made that visible: a plan from months ago looked identical to one written today. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013kj58Jugkic783b8ZrtTVZ
194 lines
8.5 KiB
Markdown
194 lines
8.5 KiB
Markdown
---
|
||
type: reference
|
||
---
|
||
# Orin runtime
|
||
|
||
[ARCHITECTURE.md](ARCHITECTURE.md) describes services and the bus without saying
|
||
how they coexist on one 8 GB board. This document covers that: what runs, in what
|
||
order, under what budget, and what happens when it goes wrong.
|
||
|
||
Cross-cutting by nature — it spans `svc-core`, `svc-nav`, `svc-gw` and the host
|
||
repo, so it lives in the hub rather than in any one of them.
|
||
|
||
## Layers
|
||
|
||
```
|
||
clients app-flutter, app-esp32 off the Orin
|
||
─────────────────────────────────────────────────────────
|
||
serving svc-gw, svc-nav
|
||
processing decoder, recorder
|
||
acquisition can-bridge, obd-poller, gnss
|
||
bus mosquitto
|
||
─────────────────────────────────────────────────────────
|
||
host kernel modules, gpsd, PipeWire, network orin-nano-env-setup
|
||
```
|
||
|
||
A layer may depend on the one below it and on the bus. Nothing depends sideways:
|
||
no service waits on another service's process, only on the broker.
|
||
|
||
## The binding constraint: 8 GB, shared
|
||
|
||
The Orin Nano has 8 GB of LPDDR5 **shared between CPU and GPU**. There is no
|
||
separate VRAM to fall back on — a model loaded onto the GPU takes memory away
|
||
from everything else on the board. Roughly 6–7 GB is left once JetPack and the OS
|
||
have taken their share.
|
||
|
||
| Consumer | Resident | Note |
|
||
|---|---|---|
|
||
| OS + JetPack | ~1–1.5 GB | before anything of ours starts |
|
||
| mosquitto, acquisition, `svc-gw` | tens of MB | not a factor |
|
||
| Valhalla server | small; tiles are **mmap'd page cache** | evictable — see below |
|
||
| Whisper | 273 MB (tiny) → 3.9 GB (large) | `small` is 244 M params; `large` does not fit |
|
||
| GPT-SoVITS | upstream guidance assumes an 8 GB VRAM card | the largest unknown; **measure before designing around it** |
|
||
| vision (Phase 5) | — | competes directly with voice for the same pool |
|
||
|
||
Four rules follow.
|
||
|
||
**The cloud LLM is load-bearing.** [OVERVIEW.md](../OVERVIEW.md) puts reasoning in
|
||
the cloud. That decision is what makes a local voice stack fit at all — moving the
|
||
LLM on-device collapses this budget entirely. It is not a free choice to revisit.
|
||
|
||
**Size Whisper by vocabulary, not by benchmark.** In-car commands are a narrow,
|
||
predictable set. `large` buys accuracy this application cannot spend, at a cost
|
||
this board cannot pay.
|
||
|
||
**Voice and vision cannot both be resident.** Phase 5 has to plan for eviction,
|
||
not addition. Worth knowing now, while vision is still deferred.
|
||
|
||
**Valhalla's memory-mapped tiles are not a leak.** They appear as page cache and
|
||
are evictable. Under pressure the kernel drops tile pages and re-faults them on
|
||
next use — that costs routing latency, not an OOM. This is the graceful failure
|
||
in the system and should be left alone.
|
||
|
||
## Startup tiers
|
||
|
||
Ordered by what is lost if a tier is late.
|
||
|
||
| Tier | Units | Why here |
|
||
|---|---|---|
|
||
| 0 | kernel modules, gpsd, network | Host layer; nothing of ours can precede it |
|
||
| 1 | `mosquitto`, `can-bridge`, `recorder` | **Data is lost if late.** Must reach ready fastest |
|
||
| 2 | `decoder`, `gnss`, `obd-poller` | Derive from tier 1; a late start loses nothing |
|
||
| 3 | `svc-gw`, `svc-nav` | Serving; a display can wait |
|
||
| 4 | voice — ASR, TTS | Interactive; seconds late is harmless |
|
||
|
||
**Capture first** is the whole principle. A CAN frame missed at tier 1 is gone
|
||
forever; a map that loads five seconds into the drive costs nothing. Ordering the
|
||
boot the intuitive way — get the screen up quickly — trades irreplaceable data for
|
||
a cosmetic gain.
|
||
|
||
## Failure isolation
|
||
|
||
- `Restart=always` on every unit. A service dying mid-drive is a normal event.
|
||
- Units order themselves `After=` the broker and nothing else. No service may wait
|
||
on another service, or a restart cascades.
|
||
- The broker is the single point of failure, and that is accepted: retained topics
|
||
make reconnection cheap, so a client that comes back has state on its first
|
||
message rather than waiting for the next publish.
|
||
- No service holds another's state. Anything worth surviving a restart is either
|
||
retained on the bus or on disk.
|
||
|
||
## Ignition off
|
||
|
||
Shutdown is the reverse of startup, so tier 1 dies last.
|
||
|
||
The only thing that *must* complete is the recorder's flush. It therefore owns its
|
||
own flush and shutdown handling rather than trusting the OS to get there in time —
|
||
a truncated session is the one failure that cannot be recovered by restarting
|
||
anything.
|
||
|
||
## Storage
|
||
|
||
Self-built binaries live under `/home/mikkeli/infra/` per the host repo's
|
||
guidelines. Data does not.
|
||
|
||
| Data | Size | Growth |
|
||
|---|---|---|
|
||
| Map artifacts | GB | Replaced wholesale, quarterly at most |
|
||
| Voice models | GB | Static |
|
||
| Recorded sessions | GB per drive | **Unbounded** |
|
||
|
||
Recordings are the problem. Without a bound the disk fills mid-drive and capture
|
||
stops — silently, because a full disk is not a crash.
|
||
|
||
**Recordings are bounded with rotation.** The recorder enforces its own ceiling and
|
||
discards oldest-first; it does not rely on an external cleanup job that might not
|
||
have run. Two properties matter more than the specific size:
|
||
|
||
- The bound is enforced by the writer, so the failure mode is *losing the oldest
|
||
session*, which is recoverable, rather than *silently stopping capture*, which
|
||
is not.
|
||
- A session in progress is never the one rotated away.
|
||
|
||
## Audio
|
||
|
||
The **tablet** owns the car's single aux input. It is the music player, and that
|
||
role is not moving to the Orin.
|
||
|
||
```
|
||
tablet: media (Spotify etc.) ──┐
|
||
├──▶ Android mixer ──▶ aux in
|
||
Orin: GPT-SoVITS (TTS) ──────┘
|
||
└── utterances sent over the reliable boundary channel
|
||
|
||
Orin: mic ──▶ Whisper (ASR) stays local, never crosses
|
||
```
|
||
|
||
The tablet is the media source, so it holds the aux cable whatever is decided
|
||
about speech. Giving the Orin its own aux line would not free the display unit —
|
||
it would add a second permanent cable to the car and leave the first one in place.
|
||
|
||
### TTS crosses as chunked utterances
|
||
|
||
An utterance travels as a **sequence of chunks with an explicit terminator**, over
|
||
the reliable channel. Not as one object, and not as a clock-synchronised audio
|
||
stream.
|
||
|
||
This is deliberately the shape that keeps both modes open, because whole-utterance
|
||
is simply a stream of one chunk:
|
||
|
||
| TTS capability | What it sends |
|
||
|---|---|
|
||
| Synthesises fully, then returns | one chunk, then terminator |
|
||
| Streams as it synthesises | chunks as produced, then terminator |
|
||
|
||
Same protocol, same receiver. The receiver starts playback on the first chunk,
|
||
appends as more arrive, and finishes on the terminator — which is what Android's
|
||
`AudioTrack` streaming mode does natively.
|
||
|
||
**What must not be built:** an utterance as a single response body with a
|
||
"download the file, then play the file" player. That works today and forecloses
|
||
streaming permanently. Whether the TTS we land on streams is still open, so the
|
||
transport must not assume either way.
|
||
|
||
### The transport is testable with no real TTS
|
||
|
||
Audio is the lowest-priority part of the system, but its transport does not have
|
||
to wait for GPT-SoVITS. A canned WAV, or `espeak-ng` — instant, tiny, and awful —
|
||
exercises chunking, ordering, the terminator, ducking and the no-sink path
|
||
completely. Voice quality is irrelevant to every one of those.
|
||
|
||
### Ducking is the tablet's job
|
||
|
||
Android's `AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK` ducks other applications' audio —
|
||
Spotify included — for the duration of a request, then restores it. This is the
|
||
mechanism every navigation app already uses, and it is less work than the PipeWire
|
||
policy it replaces. Cross-application ducking is a solved problem on the tablet
|
||
and would have been a bespoke one on the Orin.
|
||
|
||
### The microphone stays on the Orin
|
||
|
||
Whisper runs on the Orin, so its input does too. Audio *in* on one box and audio
|
||
*out* on the other is asymmetric on purpose: each side does the simple thing, and
|
||
a microphone has no reason to reach the head unit.
|
||
|
||
### Degrade when there is no sink
|
||
|
||
With no tablet connected the Orin can still hear — the mic is local — but cannot
|
||
speak. The voice stack expires or drops pending utterances rather than queueing
|
||
them unboundedly or blocking on a sink that may never arrive.
|
||
|
||
The consequence to accept: **unplug the tablet and the car has no voice.** That is
|
||
tolerable because unplugging it also removes the music, so the tablet was already
|
||
load-bearing for audio. Speech does not make it more so.
|