Files
app-flutter/docs/impl/plans/INITIAL.md
T
mikkeli 62c6a79f89 TTS: chunked utterances; buffer-then-play allowed as a first version
Brings the audio sink notes in line with project-yukart decision 12 and
ORIN-RUNTIME: the wire format is chunks plus terminator from the start,
and the first receiver may buffer to the terminator, so moving to
streaming playback is a receiver-only change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 16:10:47 +09:00

4.6 KiB

type, status, updated
type status updated
plan active 2026-10-09

Initial plan

Order

  1. MQTT client, subscribing to the retained vehicle topics. Text readout only — prove the contract before drawing anything.
  2. Gauges.
  3. Moving map: MapLibre against the PMTiles basemap served by the Orin, with the position from yukart/gnss/fix. No routing involved.
  4. Guidance from yukart/nav/guidance, once svc-nav publishes it.

A moving map with position, heading and trace is most of the value in a car and needs no routing engine at all. Routing is the expensive last fraction.

Open

  • Transport. Whether the app reaches the broker directly or goes through svc-gw is decided in that repo — see its initial plan. The physical link is settled (below).
  • CJK labels. MapLibre cannot ship Japanese glyph sets as SDF ranges. Local font rendering must be configured, or labels will be slow or absent.
  • Sunlight. A typical tablet is unreadable through a windscreen in daylight. This is a hardware problem, but the UI should not make it worse.

The boundary, from this side

Two channels with different guarantees, not one socket:

Down (state) Up (commands)
Transport push stream request/response
Loss tolerated; next sample supersedes not tolerated; retry until acknowledged
On reconnect expect current state immediately resend anything pending

This client always initiates and owns reconnection — the Orin never dials out. Assume the link drops constantly: cable knocked, screen off, power changes.

Tested with the actual tablet (Redmi Pad SE 8.7 4G, Android 16) against the car Orin. Details: USB tablet link report.

  • Physical: tablet USB-C → PD hub (charging from the car) → Orin's USB-C port. The tablet is the USB host; the Orin is a USB network gadget. The tablet picks NCM and gets 192.168.55.100; the Orin is 192.168.55.1.
  • Android won't route over it by default. The link has no internet, so it never becomes the default network while WiFi or cellular is up, and ordinary sockets go elsewhere. Verified: the Orin was unreachable until WiFi and cellular were off. The app must request the Ethernet network (ConnectivityManager.requestNetwork with TRANSPORT_ETHERNET) and bind its sockets to it.
  • About 3 s blank per power change. Cutting the hub's PD costs ~3.5 s (full USB detach); restoring it ~2.2 s. Ignition and cranking produce these.
  • An asleep tablet is a dead link. Off charger and idle, Android suspends about a minute after screen-off and answers nothing, while the Orin still sees the USB link up. The app keeps the device awake while it is the display (screen-on flag in the full UI; the same for any overlay widget).
  • Reconnect without user action, and expect the Orin to drop a silent client by heartbeat (project-yukart decision 24).

Session marking is worth building early

A button that stamps an event (indicator on, cold start, hard braking) into the recording, on the same clock as the CAN frames, turns the Phase 1 manual event log from a notebook-and-wristwatch exercise into structured data. It is what makes a raw capture decodable later, and it is a Phase 1 capability that happens to live in this repo.

Mock data must look like mock data

svc-gw can synthesise vehicle state without any real readings, which is how this app is developed with no Jetson present. The stream carries a flag when it does.

Rendering synthetic readings indistinguishable from real ones, on a screen in a moving car, is a safety problem. The indication belongs on the display surface itself, not in a settings page or a startup log.

This app is the audio sink

The tablet owns the car's aux input and is the music player, so the Orin's speech has to arrive here to be heard.

  • TTS arrives as an utterance: a sequence of chunks with a terminator over the reliable channel, never as one opaque file body. Streaming playback (start on the first chunk, append, finish on the terminator via AudioTrack streaming mode) is the target.
  • Acceptable first version: buffer the chunks and play on the terminator. With a non-streaming TTS that is a single chunk anyway. Swapping to streaming then changes only this receiver, not the protocol or the Orin.
  • Duck other audio with AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK for the duration of the utterance, then release. That is what makes Spotify drop under the voice without either side coordinating.
  • The microphone stays on the Orin. This app never captures audio; ASR input never crosses the boundary.