diff --git a/README.md b/README.md index a9c9b82..1be37ac 100644 --- a/README.md +++ b/README.md @@ -96,13 +96,25 @@ The master response rewrites each variant URL with a 24h `hdntl` token. So the - Player is **Bitmovin**, embedded on the club site. `0_xm5vrrw6` is a Kaltura entry id and identifies the match -- the catcher dedupes on it. -### Gotcha: use ffmpeg/yt-dlp, not a hand-rolled HTTP client +### Gotcha: which URL form a Python client can fetch -Python's `http.client` gets a **403 on every request**, with any header set, -including ffmpeg's exact headers, on an unexpired token from the same IP. The -difference is below HTTP (TLS handshake). ffmpeg fetches the same URL fine, so -just use yt-dlp. `probe.py` is diagnostic scaffolding and inherits this 403 -- -do not use it to conclude a token is dead. +Earlier notes here blamed a TLS handshake difference. That was wrong. The 403 +splits by **token form**, not by client: + +| URL form | Python | ffmpeg | +|---|---|---| +| `index.m3u8?hdnts=...` (master, query token) | 200 | 200 | +| `hdntl=.../index_N.m3u8` (media, path token) | 403 | 200 | + +Consequences: + +- Mint a **master** URL and hand that to the downloader. `record_live.py` does + this: it calls the entry API for a fresh master immediately before recording. +- Force `--downloader ffmpeg`. yt-dlp's native HLS downloader fetches variant + playlists with Python and dies on the path-token URLs. This is required, not + a preference. +- `probe.py` is diagnostic scaffolding and inherits the path-token 403 -- do + not use it to conclude a token is dead. ### Firefox, not Chrome @@ -118,9 +130,56 @@ is no `#EXT-X-ENDLIST`, so ffmpeg never finishes, and there is no stall to detect either -- from the downloader's point of view it is a healthy live stream forever. -**A hard duration cap is therefore the only stop condition.** `catcher.py` -passes `ffmpeg_o:-t $DURATION_MIN*60`. Do not remove it; without it an -unattended run fills the disk with a logo. +**Keep the duration cap.** `catcher.py` and `record_live.py` pass +`ffmpeg_o:-t $DURATION_MIN*60`. Without it, a feed that behaves like the video +one above fills the disk with a logo. -Observed: 116 min recorded for a 23:00 kickoff, still going at 01:04 and only -stopped manually. Budget ~3.5-4 GB per copy per match at 40 MB/min. +Note the sample size. This is **one video observation**, and it is contradicted +by **one audio observation**: on 2026-09-02 an audio stream ended cleanly by +itself (`rc=0` at 165 min, against a 180 min cap that never fired). So "the +stream never ends" is a property of that video feed, not an established rule. +Treat the cap as a safety net rather than as the only stop condition. + +Observed: video, 116 min recorded for a 23:00 kickoff, still running at 01:04 +and stopped manually. Audio, 165 min, self-terminated. Budget ~3.5-4 GB per copy per match at 40 MB/min. + +--- + +## Status + +Unattended recording works. A scheduled task started `record_live.py` at 03:15 +for an 03:45 kickoff, found the stream in 16 seconds, and recorded 165 min of +audio to a playable MPEG-TS with nobody awake. + +Sample sizes are one. One video capture, one audio capture. Where the two +disagree (stream termination, whether an entry id persists between matches), +neither is established -- do not build on either without more matches. + +### Verified + +- Token chain end to end in plain Python: session cookie -> JWT -> entry API + -> master playlist. No browser, no fingerprint forging. +- Recording via ffmpeg, video (1080p) and audio (~122 kbps AAC). +- Scheduled unattended run, including that Task Scheduler launches it correctly. +- Concurrency: several readers of one token, alongside a playing browser. + +### Not verified + +- **Login.** Everything above rides an `SSESS` cookie lifted from a desktop + Firefox profile. Obtaining a session *from credentials* has never been run. + `/user/login` is a standard Drupal form (`name`, `pass`, `form_build_id`, + `form_id`, `captcha_token`) with no reCAPTCHA script and no 2FA mention, but + `captcha_token` is an unknown and testing it risks a lockout that would take + out the working session too. + +### Decision: do not automate login + +The cookie lasts ~87 days and matches are weekly, so automating login saves +about four manual actions a year -- while being the riskiest step and the only +one needing a stored password. Refresh the cookie by logging in manually on the +desktop each quarter and provisioning it to the recording host instead. That +makes cookie *provisioning* the open problem, not the captcha. + +Use Firefox as the session holder. Two independent reasons: its cookie DB stays +readable while running (Chrome/Edge lock theirs), and it holds the login +durably where Chrome has been observed dropping it within a day.