Correct two disproved claims and record what is actually verified

The TLS-handshake explanation for the 403 was wrong. The split is by token
form, not by client: Python fetches the master (query token) fine and fails
only on the media playlist (path token). That is why record_live.py mints a
master and why --downloader ffmpeg is required.

"The stream never ends" was one video observation stated as a rule, and an
audio stream has since contradicted it by terminating cleanly well inside the
cap. Both are now recorded as single observations rather than conclusions.

Also records the state of play: unattended recording works, login has never
been tested, and the decision not to automate it -- the cookie outlives a
quarter, so automating the riskiest step to save four manual logins a year is
a bad trade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kyt7QHQvLJtQRuiY4wAqKB
This commit is contained in:
Mikkeli Matlock
2026-09-05 18:49:00 +09:00
parent f476e85209
commit 84c29cc0b7
+70 -11
View File
@@ -96,13 +96,25 @@ The master response rewrites each variant URL with a 24h `hdntl` token. So the
- Player is **Bitmovin**, embedded on the club site. `0_xm5vrrw6` is a Kaltura
entry id and identifies the match -- the catcher dedupes on it.
### Gotcha: use ffmpeg/yt-dlp, not a hand-rolled HTTP client
### Gotcha: which URL form a Python client can fetch
Python's `http.client` gets a **403 on every request**, with any header set,
including ffmpeg's exact headers, on an unexpired token from the same IP. The
difference is below HTTP (TLS handshake). ffmpeg fetches the same URL fine, so
just use yt-dlp. `probe.py` is diagnostic scaffolding and inherits this 403 --
do not use it to conclude a token is dead.
Earlier notes here blamed a TLS handshake difference. That was wrong. The 403
splits by **token form**, not by client:
| URL form | Python | ffmpeg |
|---|---|---|
| `index.m3u8?hdnts=...` (master, query token) | 200 | 200 |
| `hdntl=.../index_N.m3u8` (media, path token) | 403 | 200 |
Consequences:
- Mint a **master** URL and hand that to the downloader. `record_live.py` does
this: it calls the entry API for a fresh master immediately before recording.
- Force `--downloader ffmpeg`. yt-dlp's native HLS downloader fetches variant
playlists with Python and dies on the path-token URLs. This is required, not
a preference.
- `probe.py` is diagnostic scaffolding and inherits the path-token 403 -- do
not use it to conclude a token is dead.
### Firefox, not Chrome
@@ -118,9 +130,56 @@ is no `#EXT-X-ENDLIST`, so ffmpeg never finishes, and there is no stall to
detect either -- from the downloader's point of view it is a healthy live
stream forever.
**A hard duration cap is therefore the only stop condition.** `catcher.py`
passes `ffmpeg_o:-t $DURATION_MIN*60`. Do not remove it; without it an
unattended run fills the disk with a logo.
**Keep the duration cap.** `catcher.py` and `record_live.py` pass
`ffmpeg_o:-t $DURATION_MIN*60`. Without it, a feed that behaves like the video
one above fills the disk with a logo.
Observed: 116 min recorded for a 23:00 kickoff, still going at 01:04 and only
stopped manually. Budget ~3.5-4 GB per copy per match at 40 MB/min.
Note the sample size. This is **one video observation**, and it is contradicted
by **one audio observation**: on 2026-09-02 an audio stream ended cleanly by
itself (`rc=0` at 165 min, against a 180 min cap that never fired). So "the
stream never ends" is a property of that video feed, not an established rule.
Treat the cap as a safety net rather than as the only stop condition.
Observed: video, 116 min recorded for a 23:00 kickoff, still running at 01:04
and stopped manually. Audio, 165 min, self-terminated. Budget ~3.5-4 GB per copy per match at 40 MB/min.
---
## Status
Unattended recording works. A scheduled task started `record_live.py` at 03:15
for an 03:45 kickoff, found the stream in 16 seconds, and recorded 165 min of
audio to a playable MPEG-TS with nobody awake.
Sample sizes are one. One video capture, one audio capture. Where the two
disagree (stream termination, whether an entry id persists between matches),
neither is established -- do not build on either without more matches.
### Verified
- Token chain end to end in plain Python: session cookie -> JWT -> entry API
-> master playlist. No browser, no fingerprint forging.
- Recording via ffmpeg, video (1080p) and audio (~122 kbps AAC).
- Scheduled unattended run, including that Task Scheduler launches it correctly.
- Concurrency: several readers of one token, alongside a playing browser.
### Not verified
- **Login.** Everything above rides an `SSESS` cookie lifted from a desktop
Firefox profile. Obtaining a session *from credentials* has never been run.
`/user/login` is a standard Drupal form (`name`, `pass`, `form_build_id`,
`form_id`, `captcha_token`) with no reCAPTCHA script and no 2FA mention, but
`captcha_token` is an unknown and testing it risks a lockout that would take
out the working session too.
### Decision: do not automate login
The cookie lasts ~87 days and matches are weekly, so automating login saves
about four manual actions a year -- while being the riskiest step and the only
one needing a stored password. Refresh the cookie by logging in manually on the
desktop each quarter and provisioning it to the recording host instead. That
makes cookie *provisioning* the open problem, not the captcha.
Use Firefox as the session holder. Two independent reasons: its cookie DB stays
readable while running (Chrome/Edge lock theirs), and it holds the login
durably where Chrome has been observed dropping it within a day.