Link test: per-socket pacing (SO_MAX_PACING_RATE) at 5 MB/s removes the NCM holds

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDwfd6Ki8UfG4hK6W5YpJz

docs/impl/reports/LINK-TEST-2026-10-11.md

@@ -117,5 +117,28 @@ readable without root.
 - `adb push`/`install` is exactly an unpaced bulk transfer, which explains its
   0.14 MB/s.
 
-Open: whether RNDIS instead of NCM avoids it, and whether pacing all egress on
-the Orin (`tc` `fq` with `maxrate`) fixes unpaced senders too.
+## Pacing fixes it
+
+The Tegra kernel (5.15.148-tegra) ships without `fq`, `tbf`, `htb` or `fq_codel`,
+so egress cannot be shaped per interface with `tc`. TCP's own pacing needs no
+qdisc: the server capped each download socket with `SO_MAX_PACING_RATE` at
+5 MB/s (`LINKTEST_PACE=5000000`), just under the ~5.6 MB/s the link carries
+when it flows. Labels `pace5-bi`, `pace5-dl`:
+
+| | Unpaced | Paced to 5 MB/s |
+|---|---|---|
+| Bulk, 5 × 10 s | 2.3–5.4 MB/s, stalls up to 14 s | 4.96–4.99 MB/s every run, longest stall 94 ms |
+| Bulk, 250 ms windows | min 0, up to 56 empty | min 4.1 MB/s, none empty |
+| Burst 256 KB, p50 | 12–14 s | 58–78 ms |
+| Burst 1 MB / 4 MB, p50 | 14 s / 4.5 s | 225 ms / 863 ms |
+| Burst-connection retransmits | ~50 | 0–1 |
+| Idle gap before burst | erratic | no effect |
+| Paced 30 fps streams, ≤ 4 MB/s | clean | clean |
+
+**Every Orin → tablet sender should cap its socket at ~5 MB/s**
+(`SO_MAX_PACING_RATE`, one `setsockopt`). For svc-gw that is the WebSocket
+socket; for a camera relay, its stream socket. Senders that cannot be changed
+(sshd for the adb tunnel) stay affected.
+
+Open: whether RNDIS instead of NCM avoids the holding in the first place (not
+tested: changing the live gadget configuration needs a root action on the Orin).

tool/linktest_server.py

@@ -16,6 +16,9 @@ traffic goes over WiFi), so the app drives the test and this end only serves:
 
 Every TCP test also logs this end's TCP_INFO (retransmits, RTT), which the app
 cannot read. Run: python3 linktest_server.py [results.jsonl]
+
+LINKTEST_PACE=<bytes/s> caps every download socket with SO_MAX_PACING_RATE, TCP's
+own pacing (no qdisc needed; the Tegra kernel ships without fq/tbf/htb).
 """
 
 import json
@@ -26,6 +29,8 @@ import sys
 import threading
 import time
 
+PACE = int(os.environ.get("LINKTEST_PACE", "0"))
+SO_MAX_PACING_RATE = 47
 RESULTS = os.path.expanduser(sys.argv[1] if len(sys.argv) > 1 else "~/linktest/results.jsonl")
 CHUNK = b"\0" * 65536
 lock = threading.Lock()
@@ -58,6 +63,8 @@ def readline(conn):
 def down(conn, peer):
     test_id, *args = readline(conn).split()
     mode = args[0] if args[0] in ("burst", "paced") else "bulk"
+    if PACE:
+        conn.setsockopt(socket.SOL_SOCKET, SO_MAX_PACING_RATE, PACE)
     sent = 0
     try:
         if mode == "burst":
@@ -85,7 +92,7 @@ def down(conn, peer):
                 sent += conn.send(CHUNK)
     except OSError as e:
         log({"id": test_id, "kind": "tcp_down_orin", "mode": mode, "error": str(e)})
-    log({"id": test_id, "kind": "tcp_down_orin", "mode": mode, "args": args, "peer": peer, "bytes": sent,
+    log({"id": test_id, "kind": "tcp_down_orin", "mode": mode, "args": args, "pace": PACE, "peer": peer, "bytes": sent,
          **tcp_info(conn)})