From 2fe9665664cb0e696aadee7c462b60e1e5edb5d5 Mon Sep 17 00:00:00 2001 From: Mikkeli Date: Fri, 4 Sep 2026 14:02:41 +0900 Subject: [PATCH] Keep streaming TTS open; name the swappable-provider pattern MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Utterances cross as chunks with an explicit terminator, so whole-utterance is a stream of one chunk and a streaming TTS needs no protocol change. Records what must not be built — a single-body download-then-play player — since that forecloses streaming silently. Also notes the transport is fully exercisable with espeak-ng or a canned WAV, so audio transport need not wait on GPT-SoVITS. The provider-behind-an-interface shape has now been picked three times independently, so it is named as the project idiom rather than re-derived. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013kj58Jugkic783b8ZrtTVZ --- docs/ARCHITECTURE.md | 21 ++++++++++++++++++++- docs/ORIN-RUNTIME.md | 36 +++++++++++++++++++++++++++--------- 2 files changed, 47 insertions(+), 10 deletions(-) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 69f9085..10cc5f8 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -86,6 +86,25 @@ Topic names and payload schemas are a **contract shared across repos** — Andro and ESP32 clients depend on them — so they are versioned in their own repo rather than duplicated per client. +### Swappable providers + +The same shape has now been chosen independently three times, so it is the +project's idiom rather than three coincidences: + +| Component | Interface | Real | Trivial | +|---|---|---|---| +| `svc-nav` | routing provider | local Valhalla | cloud routing API | +| `svc-gw` | input source | the bus | synthetic state | +| voice | TTS provider | GPT-SoVITS | `espeak-ng`, or a canned WAV | + +The rule in each case: **one implementation of the contract, a swappable source +behind it.** That is what lets a service mock itself without a second codebase to +drift (decision 19), and what keeps a heavyweight dependency from blocking work on +everything downstream of it. + +A new component that has an expensive or not-yet-existing dependency should assume +this shape rather than rediscovering it. + ## Data flow and direction ### Inside the Orin, direction belongs to topics, not links @@ -279,7 +298,7 @@ rewriting the document. | 9 | Nav guidance published on the bus; route requests over HTTP via `telemetry-gw` | **Proposed** | 2026-09-03 | | 10 | Map artifacts delivered by `rsync` over LAN only; Valhalla tiles kept as a directory, not a tar | Accepted | 2026-09-03 | | 11 | Boot ordering is capture-first: acquisition reaches ready before serving or voice | Accepted | 2026-09-03 | -| 12 | The **tablet** owns the car's aux input; TTS crosses as discrete utterances, not a stream | Accepted | 2026-09-04 | +| 12 | The **tablet** owns the car's aux input; TTS crosses as chunked utterances, streaming-capable | Accepted | 2026-09-04 | | 13 | Reasoning stays in the cloud — a local LLM does not fit beside the voice stack | Accepted | 2026-09-03 | | 14 | Recordings are bounded with rotation | Accepted | 2026-09-04 | | 15 | Boundary is two simplex channels — lossy stream down, acknowledged request/response up | Accepted | 2026-09-03 | diff --git a/docs/ORIN-RUNTIME.md b/docs/ORIN-RUNTIME.md index 5f5cfcf..877b696 100644 --- a/docs/ORIN-RUNTIME.md +++ b/docs/ORIN-RUNTIME.md @@ -138,17 +138,35 @@ The tablet is the media source, so it holds the aux cable whatever is decided about speech. Giving the Orin its own aux line would not free the display unit — it would add a second permanent cable to the car and leave the first one in place. -### TTS crosses as utterances, not as a stream +### TTS crosses as chunked utterances -An utterance is short, finite, and fully synthesised before it is needed. So it -ships as a **discrete object over the reliable channel**, not as a real-time audio -stream: no jitter buffer, no underrun handling, no clock synchronisation, and a -dropped packet is a retry rather than an audible gap. +An utterance travels as a **sequence of chunks with an explicit terminator**, over +the reliable channel. Not as one object, and not as a clock-synchronised audio +stream. -The cost is that speech begins only once synthesis *and* transfer complete. For a -car assistant that is fine — nothing here is lip-synced. If streaming TTS is ever -wanted, so that speech starts before synthesis finishes, this is the decision to -revisit. +This is deliberately the shape that keeps both modes open, because whole-utterance +is simply a stream of one chunk: + +| TTS capability | What it sends | +|---|---| +| Synthesises fully, then returns | one chunk, then terminator | +| Streams as it synthesises | chunks as produced, then terminator | + +Same protocol, same receiver. The receiver starts playback on the first chunk, +appends as more arrive, and finishes on the terminator — which is what Android's +`AudioTrack` streaming mode does natively. + +**What must not be built:** an utterance as a single response body with a +"download the file, then play the file" player. That works today and forecloses +streaming permanently. Whether the TTS we land on streams is still open, so the +transport must not assume either way. + +### The transport is testable with no real TTS + +Audio is the lowest-priority part of the system, but its transport does not have +to wait for GPT-SoVITS. A canned WAV, or `espeak-ng` — instant, tiny, and awful — +exercises chunking, ordering, the terminator, ducking and the no-sink path +completely. Voice quality is irrelevant to every one of those. ### Ducking is the tablet's job