Compare commits
8 Commits
79e82ef597
..
master
| Author | SHA1 | Date | |
|---|---|---|---|
| 9c22a02fab | |||
| 847fe1e13f | |||
| c39de9a017 | |||
| 14ee5ec26c | |||
| fcf509fda5 | |||
| 84c29cc0b7 | |||
| f476e85209 | |||
| ce92e55255 |
@@ -5,3 +5,4 @@ cookies.txt
|
|||||||
*.mkv
|
*.mkv
|
||||||
*.mp4
|
*.mp4
|
||||||
*.ts
|
*.ts
|
||||||
|
__pycache__/
|
||||||
|
|||||||
@@ -11,116 +11,185 @@ makes, using your own paid session.
|
|||||||
|
|
||||||
| File | Job |
|
| File | Job |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `tampermonkey/brfc-stream-sniffer.user.js` | Hooks fetch/XHR in the page, captures the manifest URL verbatim, POSTs it to the catcher |
|
| `record_live.py` | Unattended recorder: finds the live entry, mints a master, records |
|
||||||
| `catcher.py` | Listens on 127.0.0.1:8787, hands the URL to yt-dlp, records to `recordings/` |
|
| `preflight.py` | Pre-match check: is the session live, can this host record |
|
||||||
|
| `watchdog.py` | Backup starter: if nothing is *progressing*, start a recorder |
|
||||||
|
| `catcher.py` | Listens on 127.0.0.1:8787, hands a sniffed URL to yt-dlp |
|
||||||
|
| `tampermonkey/brfc-stream-sniffer.user.js` | Hooks fetch/XHR, POSTs the manifest URL to the catcher |
|
||||||
|
| `discover.py` | Drives the player with Playwright to capture token-minting traffic |
|
||||||
| `probe.py` | Byte-exact manifest fetch + concurrency test |
|
| `probe.py` | Byte-exact manifest fetch + concurrency test |
|
||||||
|
|
||||||
|
## Recording a match
|
||||||
|
|
||||||
|
python preflight.py # exit 0 only if the session is live
|
||||||
|
|
||||||
|
Mandatory, hours before kickoff. A dead session is invisible locally and the
|
||||||
|
only fix needs a human at a browser. Then schedule the recorder for ~30 min
|
||||||
|
before kickoff:
|
||||||
|
|
||||||
|
python record_live.py --max-wait 75 --duration 180 --feed video
|
||||||
|
|
||||||
|
**Pick the feed, not just the rendition.** The live page can expose two entries
|
||||||
|
for one match -- an audio feed and a video feed -- and which manifest loads
|
||||||
|
first is a race that has come out differently on the same night. `--feed`
|
||||||
|
chooses between entries and is the setting that matters:
|
||||||
|
|
||||||
|
| `--feed` | Behaviour |
|
||||||
|
|---|---|
|
||||||
|
| `video` (default) | record a video feed; fall back to audio if none is offered |
|
||||||
|
| `audio` | the reverse |
|
||||||
|
| `any` | whichever was discovered first -- the old behaviour |
|
||||||
|
| `both` | record every feed in parallel |
|
||||||
|
| `--entry <id>` | skip discovery entirely and pin one |
|
||||||
|
|
||||||
|
Feeds are classified by the codecs in their master, never by entry id. A
|
||||||
|
non-preferred feed stays on the list as a fallback, because a feed you hold no
|
||||||
|
pass for resolves a master fine and only fails later on the segments; the
|
||||||
|
recorder falls through to the next candidate when a download dies immediately.
|
||||||
|
|
||||||
|
`--prefer` is the *other* choice -- which rendition inside one master -- and
|
||||||
|
`best` is almost always right. A wrong `--feed` cannot be rescued by `--prefer`,
|
||||||
|
and it fails quietly: a complete, playable, healthy recording of the wrong
|
||||||
|
thing. `--feed both` costs ~210 MB extra and removes the question.
|
||||||
|
|
||||||
|
**Warm the scheduled task.** After registering it, launch it once by hand and
|
||||||
|
stop it again. A freshly registered task has been seen to hang at interpreter
|
||||||
|
startup -- alive, zero CPU, nothing logged, frozen before any of our code runs.
|
||||||
|
Relaunching fixed it; root cause unknown. Stopping the test run matters as much:
|
||||||
|
`MultipleInstances IgnoreNew` means a test left running swallows the real
|
||||||
|
trigger silently.
|
||||||
|
|
||||||
|
Optionally schedule `watchdog.py` ~20 min later as a backup. It checks whether
|
||||||
|
recording is *progressing* (a growing `.ts`, or a freshly written log), not
|
||||||
|
whether a process exists -- a hung recorder passes the process check.
|
||||||
|
|
||||||
|
### Manual alternative
|
||||||
|
|
||||||
|
python catcher.py # leave running
|
||||||
|
|
||||||
|
Install the userscript, open the match page, press play. The sniffer catches the
|
||||||
|
manifest, the catcher starts yt-dlp. Config via environment:
|
||||||
|
|
||||||
|
| Variable | Default | Effect |
|
||||||
|
|---|---|---|
|
||||||
|
| `BRFC_BROWSER` | `firefox` | which browser's cookie jar yt-dlp reads |
|
||||||
|
| `BRFC_DURATION_MIN` | `150` | catcher's recording cap, minutes |
|
||||||
|
|
||||||
|
Check concurrency against a live token with `probe.py --url "<manifest>"
|
||||||
|
--parallel 3`.
|
||||||
|
|
||||||
## The token
|
## The token
|
||||||
|
|
||||||
Manifest URLs carry an Akamai path token:
|
Manifest URLs carry an Akamai path token:
|
||||||
|
|
||||||
hdntl=exp=<unix>~acl=/*~id=<n>~data=hdntl~hmac=<sig>
|
hdntl=exp=<unix>~acl=/*~id=<n>~data=hdntl~hmac=<sig>
|
||||||
|
|
||||||
- `exp` -- ~24h validity
|
It is a **bearer credential** with `acl=/*` over the whole host, valid to `exp`.
|
||||||
- `acl=/*` -- wildcard over the whole host, not just your match
|
Anyone holding a live URL can pull the stream. Keep it out of pastebins and
|
||||||
- `hmac` -- signature over the literal path
|
issue trackers; `stream-url.txt` and `cookies.txt` are gitignored.
|
||||||
|
|
||||||
It is a bearer credential. Anyone holding a live URL can pull the stream until
|
Two tokens, not one:
|
||||||
`exp`. Keep it out of pastebins and issue trackers. `stream-url.txt` and
|
|
||||||
`cookies.txt` are gitignored.
|
|
||||||
|
|
||||||
**Do not re-encode the path.** curl normalises `%2f` to `%2F`, which
|
| Token | Where | TTL | Fetches |
|
||||||
invalidates the HMAC. `probe.py` sends the request target verbatim.
|
|
||||||
|
|
||||||
## Use Firefox for watching
|
|
||||||
|
|
||||||
yt-dlp lifts session cookies from the browser. Chrome 127+ on Windows uses
|
|
||||||
app-bound cookie encryption and extraction fails. Firefox works with no fuss.
|
|
||||||
|
|
||||||
set BRFC_BROWSER=firefox # default
|
|
||||||
|
|
||||||
## Workflow
|
|
||||||
|
|
||||||
pip install yt-dlp # done
|
|
||||||
python catcher.py # leave running
|
|
||||||
|
|
||||||
Install the userscript in Tampermonkey, open the match page, press play. The
|
|
||||||
sniffer catches the manifest, the catcher starts yt-dlp. Recording is MPEG-TS
|
|
||||||
(`--hls-use-mpegts`) so a killed process still leaves a playable file.
|
|
||||||
|
|
||||||
For a 3am kickoff: Task Scheduler opens the match page a few minutes before,
|
|
||||||
the browser autoplays, the sniffer fires, the catcher records. Nobody is awake.
|
|
||||||
|
|
||||||
## Concurrency check
|
|
||||||
|
|
||||||
An Akamai edge token is a stateless HMAC -- the edge validates signature and
|
|
||||||
expiry and serves; it holds no session table, so it cannot count viewers. Any
|
|
||||||
concurrent-stream limit lives in the entitlement API that *issues* tokens,
|
|
||||||
upstream of the CDN. Confirm rather than assume, while the browser is playing:
|
|
||||||
|
|
||||||
python probe.py --url "<manifest from the sniffer>" --parallel 3
|
|
||||||
|
|
||||||
3/3 alongside the playing browser means four concurrent readers of one token
|
|
||||||
and the edge does not care.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Findings (2026-08-29, live 2300 JST match)
|
|
||||||
|
|
||||||
Verified against a live stream. Recorded 1080p successfully end to end.
|
|
||||||
|
|
||||||
### Two token types, not one
|
|
||||||
|
|
||||||
The player uses a **two-stage token chain**:
|
|
||||||
|
|
||||||
| Token | Where | TTL | Purpose |
|
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| `hdnts=` | query string, `acl=/*` unencoded | **~25 seconds** | fetch the master playlist |
|
| `hdnts=` | query string, `acl=/*` unencoded | ~25 seconds | master playlist |
|
||||||
| `hdntl=` | URL path, `acl=%2f*` encoded | **~24 hours** | media playlists + segments |
|
| `hdntl=` | URL path, `acl=%2f*` encoded | ~24 hours | media playlists, segments |
|
||||||
|
|
||||||
The master response rewrites each variant URL with a 24h `hdntl` token. So the
|
The master response rewrites each variant URL with a 24h `hdntl`. Chasing a
|
||||||
`hdntl` token is the durable one worth handing to a downloader. Chasing a
|
`hdnts` URL by hand is pointless -- it expires before you can paste it.
|
||||||
`hdnts` URL by hand is pointless -- it dies before you can paste it.
|
|
||||||
|
|
||||||
### Confirmed by experiment
|
**Do not re-encode the path.** curl normalises `%2f` to `%2F`, invalidating the
|
||||||
|
HMAC. `probe.py` sends the request target verbatim.
|
||||||
|
|
||||||
- **Concurrency is not limited.** Two independent ffmpeg downloads ran at full
|
## Rules that are not preferences
|
||||||
bitrate *while Firefox played the same match*. Three concurrent readers of
|
|
||||||
one token, no complaint from the edge. Akamai token auth is a stateless HMAC
|
|
||||||
check; it has no viewer counter.
|
|
||||||
- **No CDN cookies exist.** All 13 relevant cookies are on `bristolrovers.co.uk`;
|
|
||||||
nothing is scoped to `akamaized.net`. The token in the URL is the whole auth.
|
|
||||||
- **Stream is 1920x1080 H.264 + AAC ~5.2 Mbit/s** -> roughly 4.7 GB per match.
|
|
||||||
- **Output is MPEG-TS** even though yt-dlp names it `.mp4`. That is what
|
|
||||||
`--hls-use-mpegts` buys: a killed process still leaves a playable file.
|
|
||||||
- Player is **Bitmovin**, embedded on the club site. `0_xm5vrrw6` is a Kaltura
|
|
||||||
entry id and identifies the match -- the catcher dedupes on it.
|
|
||||||
|
|
||||||
### Gotcha: use ffmpeg/yt-dlp, not a hand-rolled HTTP client
|
- **Hand the downloader a master URL, not a media one.** Python gets 403 on
|
||||||
|
path-token media playlists; ffmpeg gets 200. yt-dlp fetches playlists with
|
||||||
|
Python before ffmpeg runs, so it must be given a `hdnts` master.
|
||||||
|
`record_live.py` mints one immediately before recording.
|
||||||
|
- **Force `--downloader ffmpeg`.** yt-dlp's native HLS downloader dies on the
|
||||||
|
path-token URLs for the same reason.
|
||||||
|
- **Keep the duration cap.** A feed can keep serving after full time -- a static
|
||||||
|
slate on video, digital silence on audio -- at full bitrate, with no
|
||||||
|
`#EXT-X-ENDLIST` and no stall to detect. Of five observed endings three
|
||||||
|
self-terminated and two ran forever, on both feed types, with no pattern that
|
||||||
|
predicts which. Treat a stream that stops by itself as luck.
|
||||||
|
- **Use Firefox as the session holder.** Chrome/Edge lock their cookie DB while
|
||||||
|
running (yt-dlp issue 7271) and Chrome 127+ on Windows adds app-bound cookie
|
||||||
|
encryption; an unattended job cannot close a browser to read them.
|
||||||
|
- **Do not judge a token by `probe.py`.** It inherits the path-token 403, so a
|
||||||
|
live token looks dead through it.
|
||||||
|
|
||||||
Python's `http.client` gets a **403 on every request**, with any header set,
|
## Facts
|
||||||
including ffmpeg's exact headers, on an unexpired token from the same IP. The
|
|
||||||
difference is below HTTP (TLS handshake). ffmpeg fetches the same URL fine, so
|
|
||||||
just use yt-dlp. `probe.py` is diagnostic scaffolding and inherits this 403 --
|
|
||||||
do not use it to conclude a token is dead.
|
|
||||||
|
|
||||||
### Firefox, not Chrome
|
- 1920x1080 H.264 + AAC ~5.2 Mbit/s. Audio-only feeds are ~140 kbps AAC.
|
||||||
|
- Output is MPEG-TS even when named `.mp4` -- that is what `--hls-use-mpegts`
|
||||||
|
buys: a killed process still leaves a playable file.
|
||||||
|
- Budget **~36 MB/min video** (~6.5 GB for a 180 min cap), **~1.2 MB/min audio**
|
||||||
|
(~210 MB).
|
||||||
|
- **Concurrency is not limited.** An Akamai edge token is a stateless HMAC with
|
||||||
|
no viewer counter. Three readers of one token ran at full bitrate alongside a
|
||||||
|
playing browser.
|
||||||
|
- **No CDN cookies exist.** All relevant cookies are on `bristolrovers.co.uk`,
|
||||||
|
none on `akamaized.net`. The token in the URL is the whole auth.
|
||||||
|
- Player is Bitmovin. The Kaltura entry id names the **feed, not the match**:
|
||||||
|
`0_xm5vrrw6` (video) and `0_z236v0e7` (audio) each recurred weeks apart.
|
||||||
|
Nothing may use it to answer "have I recorded this game" -- use the date.
|
||||||
|
`catcher.py` is unaffected; it dedupes on liveness, not identity.
|
||||||
|
|
||||||
yt-dlp reads cookies from a running Firefox. Chrome/Edge lock their cookie DB
|
## The session
|
||||||
while running (yt-dlp issue 7271), and an unattended job cannot close the
|
|
||||||
browser to read them.
|
|
||||||
|
|
||||||
### The stream never ends
|
The `SSESS` cookie is the whole login. Everything downstream -- JWT, entry API,
|
||||||
|
master playlist -- rides it, in plain Python, with no browser or fingerprint
|
||||||
|
forging.
|
||||||
|
|
||||||
After full time the feed switches to a static league slate and keeps serving
|
**Cookie expiry is not session lifetime.** The server can drop a session while
|
||||||
indefinitely at full CBR bitrate (~4.7 Mbit/s to encode a still image). There
|
the browser holds a cookie that looks perfectly healthy:
|
||||||
is no `#EXT-X-ENDLIST`, so ffmpeg never finishes, and there is no stall to
|
|
||||||
detect either -- from the downloader's point of view it is a healthy live
|
|
||||||
stream forever.
|
|
||||||
|
|
||||||
**A hard duration cap is therefore the only stop condition.** `catcher.py`
|
SSESS... 76 days left on the clock
|
||||||
passes `ffmpeg_o:-t $DURATION_MIN*60`. Do not remove it; without it an
|
GET /oauth/browser-token -> 403 {"error":"Unauthenticated"}
|
||||||
unattended run fills the disk with a logo.
|
Set-Cookie: SSESS...=deleted; expires=Thu, 01 Jan 1970
|
||||||
|
|
||||||
Observed: 116 min recorded for a 23:00 kickoff, still going at 01:04 and only
|
One session died after six days; its replacement survived seven and counting.
|
||||||
stopped manually. Budget ~3.5-4 GB per copy per match at 40 MB/min.
|
The lifetime is unpredictable and only the server knows it, so a local cookie
|
||||||
|
check proves nothing and `preflight.py` is not a formality.
|
||||||
|
|
||||||
|
Other session behaviour:
|
||||||
|
|
||||||
|
- **Sessions are concurrent.** Logging in via a second browser does not
|
||||||
|
invalidate the first -- Drupal issues a session per browser. So a second
|
||||||
|
device can hold its own session without disturbing the recorder's. *One
|
||||||
|
observation, checked immediately after the second login.*
|
||||||
|
- `/oauth/browser-token` returns the session's **existing** JWT rather than
|
||||||
|
minting a fresh one per call, so the token's expiry is not a freshness
|
||||||
|
signal. A dead session returns 403, which is what the check relies on.
|
||||||
|
- The cookie *value* changed once between weeks with no re-login recorded.
|
||||||
|
Unexplained. Treat a copied cookie as a snapshot that can go stale on its own.
|
||||||
|
- The session cookie is **httpOnly**, so `document.cookie` cannot read it. Any
|
||||||
|
provisioning scheme must use a platform cookie store, or it will silently
|
||||||
|
collect every cookie except the one that matters.
|
||||||
|
|
||||||
|
### Decision: do not automate login
|
||||||
|
|
||||||
|
Automating it is the riskiest step in the chain and the only one needing a
|
||||||
|
stored password, against an unknown `captcha_token` and a lockout that would
|
||||||
|
take out the working session too. Instead assume the session can be dead before
|
||||||
|
any given match and make `preflight.py` routine.
|
||||||
|
|
||||||
|
Cookie *provisioning* to a second host remains the open problem.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Unattended recording works end to end, scheduled, with nobody awake. Five
|
||||||
|
matches recorded as of 2026-09-20, including one night capturing both feeds in
|
||||||
|
parallel.
|
||||||
|
|
||||||
|
**Verified:** the token chain in plain Python; recording both feed types via
|
||||||
|
ffmpeg; Task Scheduler launching the job; concurrent readers of one token; a
|
||||||
|
dead session being detectable before kickoff; the duration cap firing at `rc=0`
|
||||||
|
on a feed that would not stop.
|
||||||
|
|
||||||
|
**Not verified:** obtaining a session *from credentials*. `/user/login` is a
|
||||||
|
standard Drupal form (`name`, `pass`, `form_build_id`, `form_id`,
|
||||||
|
`captcha_token`) with no reCAPTCHA script and no 2FA mention, but
|
||||||
|
`captcha_token` is unknown and probing it risks a lockout.
|
||||||
|
|||||||
+6
-3
@@ -32,8 +32,10 @@ BROWSER = os.environ.get("BRFC_BROWSER", "firefox")
|
|||||||
# ends cleanly. Pad past 90 min for half-time, stoppage and post-match.
|
# ends cleanly. Pad past 90 min for half-time, stoppage and post-match.
|
||||||
DURATION_MIN = int(os.environ.get("BRFC_DURATION_MIN", "150"))
|
DURATION_MIN = int(os.environ.get("BRFC_DURATION_MIN", "150"))
|
||||||
|
|
||||||
# Kaltura entry id (e.g. 0_xm5vrrw6) identifies the match. Every manifest for
|
# Kaltura entry id (e.g. 0_xm5vrrw6). Every manifest for one game -- master and
|
||||||
# one game -- master and each variant, each with its own token -- shares it.
|
# each variant, each with its own token -- shares it. It names the *feed*, not
|
||||||
|
# the match: the same id came back three weeks apart (see README). That is fine
|
||||||
|
# here, because the dedupe below is on liveness, not on "seen this before".
|
||||||
ASSET_RE = re.compile(r"/(\d+_[a-z0-9]+)/", re.I)
|
ASSET_RE = re.compile(r"/(\d+_[a-z0-9]+)/", re.I)
|
||||||
|
|
||||||
_procs = {} # asset key -> Popen of the running downloader
|
_procs = {} # asset key -> Popen of the running downloader
|
||||||
@@ -68,8 +70,9 @@ def start_recording(key, url, user_agent, page):
|
|||||||
# yt-dlp's own retry flags do NOT reach the ffmpeg downloader, which is
|
# yt-dlp's own retry flags do NOT reach the ffmpeg downloader, which is
|
||||||
# what actually pulls live HLS. These are ffmpeg's own reconnect knobs.
|
# what actually pulls live HLS. These are ffmpeg's own reconnect knobs.
|
||||||
"--downloader-args",
|
"--downloader-args",
|
||||||
|
# ffmpeg 4.3.1 has no -reconnect_on_network_error.
|
||||||
"ffmpeg_i:-reconnect 1 -reconnect_streamed 1 "
|
"ffmpeg_i:-reconnect 1 -reconnect_streamed 1 "
|
||||||
"-reconnect_on_network_error 1 -reconnect_delay_max 30",
|
"-reconnect_delay_max 30",
|
||||||
"--downloader-args", f"ffmpeg_o:-t {DURATION_MIN * 60}",
|
"--downloader-args", f"ffmpeg_o:-t {DURATION_MIN * 60}",
|
||||||
"-o", out,
|
"-o", out,
|
||||||
url,
|
url,
|
||||||
|
|||||||
+203
@@ -0,0 +1,203 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Drive the player with Playwright and capture the traffic that mints a token.
|
||||||
|
|
||||||
|
Purpose: find which request produces a stream URL, so the mint can eventually
|
||||||
|
run without a GUI. Playwright sees request AND response bodies natively, so
|
||||||
|
this replaces the userscript's hooks with something far less fragile.
|
||||||
|
|
||||||
|
No password is needed. Your existing Firefox session cookies are injected into
|
||||||
|
a fresh browser context, so this rides the login you already have.
|
||||||
|
|
||||||
|
python discover.py --url "<match page url>"
|
||||||
|
python discover.py --url "..." --headless
|
||||||
|
|
||||||
|
Ports to Camoufox on halogen by swapping the launcher: Camoufox speaks the
|
||||||
|
Playwright protocol, so the flow below is unchanged.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
ROOT = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
LOGS = os.path.join(ROOT, "logs")
|
||||||
|
TRACE = os.path.join(LOGS, "pw-trace.jsonl")
|
||||||
|
|
||||||
|
# Only these domains' cookies are handed to the browser. Everything else in
|
||||||
|
# the Firefox jar is irrelevant here and should not be copied around.
|
||||||
|
COOKIE_DOMAINS = ("bristolrovers", "streamamg", "kaltura", "akamaized")
|
||||||
|
|
||||||
|
# Bodies we never want to read: media segments would be dumped into the log.
|
||||||
|
SKIP_BODY = (".ts", ".mp4", ".m4s", ".aac", ".jpg", ".jpeg", ".png", ".gif",
|
||||||
|
".webp", ".svg", ".woff", ".woff2", ".css", ".ico")
|
||||||
|
|
||||||
|
MAX_BODY = 20000
|
||||||
|
|
||||||
|
|
||||||
|
def firefox_cookies():
|
||||||
|
"""Lift the live session from Firefox. Requires no password."""
|
||||||
|
from yt_dlp.cookies import extract_cookies_from_browser
|
||||||
|
import io, contextlib
|
||||||
|
buf = io.StringIO()
|
||||||
|
with contextlib.redirect_stderr(buf):
|
||||||
|
jar = extract_cookies_from_browser("firefox")
|
||||||
|
|
||||||
|
out = []
|
||||||
|
for c in jar:
|
||||||
|
dom = (c.domain or "")
|
||||||
|
if not any(k in dom.lower() for k in COOKIE_DOMAINS):
|
||||||
|
continue
|
||||||
|
out.append({
|
||||||
|
"name": c.name,
|
||||||
|
"value": c.value or "",
|
||||||
|
"domain": dom,
|
||||||
|
"path": c.path or "/",
|
||||||
|
"expires": float(c.expires) if c.expires else -1,
|
||||||
|
"httpOnly": False,
|
||||||
|
"secure": bool(c.secure),
|
||||||
|
"sameSite": "Lax",
|
||||||
|
})
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--url", required=True, help="match page URL")
|
||||||
|
ap.add_argument("--headless", action="store_true",
|
||||||
|
help="run without a window (what halogen will do)")
|
||||||
|
ap.add_argument("--seconds", type=int, default=90,
|
||||||
|
help="how long to watch after load")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
try:
|
||||||
|
from playwright.sync_api import sync_playwright
|
||||||
|
except ImportError:
|
||||||
|
sys.exit("playwright not installed: pip install playwright "
|
||||||
|
"&& python -m playwright install firefox")
|
||||||
|
|
||||||
|
os.makedirs(LOGS, exist_ok=True)
|
||||||
|
cookies = firefox_cookies()
|
||||||
|
print(f"injecting {len(cookies)} session cookies")
|
||||||
|
|
||||||
|
manifests, mints, jwts, seen = [], [], [], set()
|
||||||
|
fh = open(TRACE, "a", encoding="utf-8")
|
||||||
|
|
||||||
|
def write(rec):
|
||||||
|
rec["ts"] = datetime.now().isoformat()
|
||||||
|
fh.write(json.dumps(rec, ensure_ascii=False) + chr(10))
|
||||||
|
fh.flush()
|
||||||
|
|
||||||
|
with sync_playwright() as p:
|
||||||
|
browser = p.firefox.launch(headless=args.headless)
|
||||||
|
ctx = browser.new_context(
|
||||||
|
user_agent=("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) "
|
||||||
|
"Gecko/20100101 Firefox/131.0"),
|
||||||
|
viewport={"width": 1440, "height": 900},
|
||||||
|
)
|
||||||
|
if cookies:
|
||||||
|
try:
|
||||||
|
ctx.add_cookies(cookies)
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"cookie injection failed: {exc}")
|
||||||
|
|
||||||
|
page = ctx.new_page()
|
||||||
|
|
||||||
|
def on_response(resp):
|
||||||
|
url = resp.url
|
||||||
|
if url in seen:
|
||||||
|
return
|
||||||
|
low = url.split("?")[0].lower()
|
||||||
|
if ".m3u8" in low:
|
||||||
|
seen.add(url)
|
||||||
|
manifests.append(url)
|
||||||
|
print(f" manifest: {url[:120]}")
|
||||||
|
write({"kind": "manifest", "url": url, "status": resp.status})
|
||||||
|
return
|
||||||
|
if any(low.endswith(e) for e in SKIP_BODY):
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
body = resp.text()
|
||||||
|
except Exception:
|
||||||
|
return
|
||||||
|
if not body:
|
||||||
|
return
|
||||||
|
# The JWT that authorises the mint is issued somewhere on the club
|
||||||
|
# site off the session cookie. Detect it the same way: by output.
|
||||||
|
# A JWT is three base64 segments starting with a b64 '{"alg"'.
|
||||||
|
if re.search(r"eyJ[A-Za-z0-9_-]{8,}\.eyJ[A-Za-z0-9_-]{8,}\.", body):
|
||||||
|
seen.add(url)
|
||||||
|
req = resp.request
|
||||||
|
rec = {
|
||||||
|
"kind": "JWT-ISSUER",
|
||||||
|
"method": req.method,
|
||||||
|
"url": url,
|
||||||
|
"status": resp.status,
|
||||||
|
"reqHeaders": dict(req.headers),
|
||||||
|
"reqBody": (req.post_data or None),
|
||||||
|
"body": body[:MAX_BODY],
|
||||||
|
}
|
||||||
|
jwts.append(rec)
|
||||||
|
write(rec)
|
||||||
|
print(f" *** JWT ISSUER *** {req.method} {url[:100]}")
|
||||||
|
return
|
||||||
|
|
||||||
|
# The mint call is whatever RETURNS a manifest URL, whatever it is
|
||||||
|
# named. Detect by output, not by guessing endpoint names.
|
||||||
|
if "m3u8" in body.lower():
|
||||||
|
seen.add(url)
|
||||||
|
req = resp.request
|
||||||
|
rec = {
|
||||||
|
"kind": "MINT",
|
||||||
|
"method": req.method,
|
||||||
|
"url": url,
|
||||||
|
"status": resp.status,
|
||||||
|
"reqHeaders": dict(req.headers),
|
||||||
|
"reqBody": (req.post_data or None),
|
||||||
|
"body": body[:MAX_BODY],
|
||||||
|
}
|
||||||
|
mints.append(rec)
|
||||||
|
write(rec)
|
||||||
|
print(f" *** MINT *** {req.method} {url[:110]}")
|
||||||
|
|
||||||
|
page.on("response", on_response)
|
||||||
|
|
||||||
|
print(f"loading {args.url}")
|
||||||
|
page.goto(args.url, wait_until="domcontentloaded", timeout=60000)
|
||||||
|
|
||||||
|
# The player needs a click. Muted playback is enough to trigger the
|
||||||
|
# manifest fetch, and muted autoplay is not blocked.
|
||||||
|
page.wait_for_timeout(5000)
|
||||||
|
for sel in (".bmpui-ui-hugeplaybacktogglebutton",
|
||||||
|
".bmpui-ui-playbacktogglebutton",
|
||||||
|
"button[aria-label*='play' i]"):
|
||||||
|
try:
|
||||||
|
el = page.query_selector(sel)
|
||||||
|
if el and el.is_visible():
|
||||||
|
el.click()
|
||||||
|
print(f"clicked {sel}")
|
||||||
|
break
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
page.evaluate("() => { const v = document.querySelector('video');"
|
||||||
|
" if (v) { v.muted = true; v.play(); } }")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
print(f"watching for {args.seconds}s ...")
|
||||||
|
page.wait_for_timeout(args.seconds * 1000)
|
||||||
|
browser.close()
|
||||||
|
|
||||||
|
fh.close()
|
||||||
|
print(f"\nmanifests seen : {len(manifests)}")
|
||||||
|
print(f"mint calls : {len(mints)}")
|
||||||
|
for m in mints:
|
||||||
|
print(f"\n {m['method']} {m['url']}")
|
||||||
|
print(f" status {m['status']} reqBody={str(m['reqBody'])[:200]}")
|
||||||
|
print(f"\nfull trace -> {TRACE}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
+231
@@ -0,0 +1,231 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Pre-match check: is the session still good, and can this host record?
|
||||||
|
|
||||||
|
Run it a few hours before kickoff, not five minutes before. The one thing it
|
||||||
|
cannot fix for you is a dead login: refreshing that means opening Firefox and
|
||||||
|
signing in by hand, which takes a human (see README, "Decision: do not
|
||||||
|
automate login").
|
||||||
|
|
||||||
|
python preflight.py # session + host checks
|
||||||
|
python preflight.py --entry 0_z236v0e7 # also exercise the entry API
|
||||||
|
|
||||||
|
Exit code is 0 only if the server returned a JWT for this session. Anything
|
||||||
|
else means tonight's scheduled run would fail, and it would fail at kickoff
|
||||||
|
with nobody watching.
|
||||||
|
|
||||||
|
Prints no secrets. Cookie values and tokens are reported by length only.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import base64
|
||||||
|
import contextlib
|
||||||
|
import http.cookiejar
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
ROOT = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
RECORDINGS = os.path.join(ROOT, "recordings")
|
||||||
|
UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) "
|
||||||
|
"Gecko/20100101 Firefox/131.0")
|
||||||
|
TOKEN_URL = ("https://www.bristolrovers.co.uk/oauth/browser-token"
|
||||||
|
"?client_id=streamamg&token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aid_token")
|
||||||
|
# Public client key, shipped to every browser -- not a secret.
|
||||||
|
API_KEY = "50ivbQ3IXpF2mwT4tPAJ2V6cVm4CGRT2YKmDK5v6"
|
||||||
|
|
||||||
|
OK, BAD, WARN = "ok ", "FAIL", "warn"
|
||||||
|
|
||||||
|
|
||||||
|
def say(status, msg):
|
||||||
|
print(f"[{status}] {msg}", flush=True)
|
||||||
|
|
||||||
|
|
||||||
|
def check_deps():
|
||||||
|
"""Everything record_live.py reaches for, checked before kickoff."""
|
||||||
|
good = True
|
||||||
|
for mod, hint in (("yt_dlp", "pip install yt-dlp"),
|
||||||
|
("playwright", "pip install playwright && "
|
||||||
|
"python -m playwright install firefox")):
|
||||||
|
try:
|
||||||
|
__import__(mod)
|
||||||
|
say(OK, f"{mod} importable")
|
||||||
|
except ImportError:
|
||||||
|
say(BAD, f"{mod} missing -- {hint}")
|
||||||
|
good = False
|
||||||
|
if shutil.which("ffmpeg"):
|
||||||
|
say(OK, "ffmpeg on PATH")
|
||||||
|
else:
|
||||||
|
say(BAD, "ffmpeg not on PATH -- the downloader is forced to ffmpeg")
|
||||||
|
good = False
|
||||||
|
return good
|
||||||
|
|
||||||
|
|
||||||
|
def check_disk(need_gb=8):
|
||||||
|
"""A video match is ~4.7 GB. Audio is ~200 MB. Budget for video."""
|
||||||
|
os.makedirs(RECORDINGS, exist_ok=True)
|
||||||
|
free_gb = shutil.disk_usage(RECORDINGS).free / 2**30
|
||||||
|
if free_gb >= need_gb:
|
||||||
|
say(OK, f"{free_gb:.0f} GB free on the recordings volume")
|
||||||
|
return True
|
||||||
|
say(BAD, f"only {free_gb:.1f} GB free, want >= {need_gb} GB for a video match")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def firefox_jar():
|
||||||
|
from yt_dlp.cookies import extract_cookies_from_browser
|
||||||
|
with contextlib.redirect_stderr(io.StringIO()):
|
||||||
|
return extract_cookies_from_browser("firefox")
|
||||||
|
|
||||||
|
|
||||||
|
def check_cookies(jar):
|
||||||
|
"""Cookie *present and unexpired* is necessary, not sufficient. The server
|
||||||
|
can invalidate a session while the browser still holds the cookie, which is
|
||||||
|
exactly what check_session() is for."""
|
||||||
|
brfc = [c for c in jar if "bristolrovers" in (c.domain or "").lower()]
|
||||||
|
say(OK if brfc else BAD, f"{len(brfc)} bristolrovers cookies in the Firefox jar")
|
||||||
|
sess = [c for c in brfc if c.name.startswith(("SSESS", "SESS"))]
|
||||||
|
if not sess:
|
||||||
|
say(BAD, "no SSESS/SESS cookie -- Firefox is not logged in")
|
||||||
|
return brfc, False
|
||||||
|
for c in sess:
|
||||||
|
if not c.expires:
|
||||||
|
say(WARN, f"{c.name[:16]}... is a session cookie (dies with Firefox)")
|
||||||
|
continue
|
||||||
|
days = (c.expires - time.time()) / 86400
|
||||||
|
say(OK if days > 1 else BAD,
|
||||||
|
f"{c.name[:16]}... len={len(c.value or '')} expires in {days:.0f}d")
|
||||||
|
return brfc, True
|
||||||
|
|
||||||
|
|
||||||
|
def check_session(brfc):
|
||||||
|
"""The real login test: will the server hand back a JWT for this cookie?
|
||||||
|
|
||||||
|
It returns the session's existing token rather than minting a fresh one per
|
||||||
|
call, so the expiry printed below is the token's remaining life and not a
|
||||||
|
freshness signal. That does not weaken the check: a dead session answers
|
||||||
|
403, which is the thing we are looking for.
|
||||||
|
"""
|
||||||
|
cj = http.cookiejar.CookieJar()
|
||||||
|
for c in brfc:
|
||||||
|
cj.set_cookie(c)
|
||||||
|
op = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
|
||||||
|
hdrs = {"User-Agent": UA, "Accept": "application/json, text/plain, */*"}
|
||||||
|
try:
|
||||||
|
resp = op.open(urllib.request.Request(TOKEN_URL, headers=hdrs), timeout=20)
|
||||||
|
body = resp.read().decode("utf-8", "replace")
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
detail = exc.read().decode("utf-8", "replace")[:200]
|
||||||
|
say(BAD, f"browser-token returned HTTP {exc.code}: {detail}")
|
||||||
|
if exc.code in (401, 403):
|
||||||
|
say(BAD, "session is dead server-side -- sign in again in Firefox")
|
||||||
|
return None
|
||||||
|
except Exception as exc:
|
||||||
|
say(BAD, f"browser-token unreachable: {type(exc).__name__}: {exc}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
m = re.search(r"(eyJ[\w-]+\.eyJ[\w-]+\.[\w-]+)", body)
|
||||||
|
if not m:
|
||||||
|
say(BAD, "no JWT in the browser-token response -- session is stale")
|
||||||
|
return None
|
||||||
|
jwt = m.group(1)
|
||||||
|
seg = jwt.split(".")[1]
|
||||||
|
try:
|
||||||
|
claims = json.loads(base64.urlsafe_b64decode(seg + "=" * (-len(seg) % 4)))
|
||||||
|
except Exception:
|
||||||
|
claims = {}
|
||||||
|
exp = claims.get("exp")
|
||||||
|
when = (f", expires {datetime.fromtimestamp(exp, timezone.utc):%H:%M:%S}Z "
|
||||||
|
f"({(exp - time.time()) / 60:+.0f} min)" if exp else "")
|
||||||
|
say(OK, f"session is live, server returned a JWT ({len(jwt)} chars{when})")
|
||||||
|
return jwt
|
||||||
|
|
||||||
|
|
||||||
|
def check_entry(jwt, entry_id):
|
||||||
|
"""Optional: exercise the rest of the chain against a known entry id.
|
||||||
|
|
||||||
|
A master playlist outlives its stream, so a 200 here does not prove the
|
||||||
|
match is live -- it proves the entitlement still resolves.
|
||||||
|
"""
|
||||||
|
req = urllib.request.Request(
|
||||||
|
f"https://api.playback.streamamg.com/v1/entry/{entry_id}",
|
||||||
|
headers={"User-Agent": UA, "Authorization": "Bearer " + jwt,
|
||||||
|
"x-api-key": API_KEY,
|
||||||
|
"Origin": "https://www.bristolrovers.co.uk",
|
||||||
|
"Referer": "https://www.bristolrovers.co.uk/"})
|
||||||
|
try:
|
||||||
|
data = json.loads(urllib.request.urlopen(req, timeout=20)
|
||||||
|
.read().decode("utf-8", "replace"))
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
say(WARN, f"entry {entry_id}: HTTP {exc.code} "
|
||||||
|
f"(stale entry id, or not entitled)")
|
||||||
|
return
|
||||||
|
except Exception as exc:
|
||||||
|
say(WARN, f"entry {entry_id}: {type(exc).__name__}: {exc}")
|
||||||
|
return
|
||||||
|
|
||||||
|
hls = (data.get("media") or {}).get("hls")
|
||||||
|
if not hls:
|
||||||
|
say(WARN, f"entry {entry_id} resolved but carries no HLS master")
|
||||||
|
return
|
||||||
|
say(OK, f"entry {entry_id} resolves to a master playlist")
|
||||||
|
try:
|
||||||
|
body = urllib.request.urlopen(urllib.request.Request(
|
||||||
|
hls, headers={"User-Agent": UA,
|
||||||
|
"Referer": "https://www.bristolrovers.co.uk/"}),
|
||||||
|
timeout=15).read().decode("utf-8", "replace")
|
||||||
|
except Exception as exc:
|
||||||
|
say(WARN, f"could not read the master: {type(exc).__name__}: {exc}")
|
||||||
|
return
|
||||||
|
PREFIX = "#EXT-X-STREAM-INF:"
|
||||||
|
streams = [ln[len(PREFIX):] for ln in body.splitlines()
|
||||||
|
if ln.startswith(PREFIX)]
|
||||||
|
say(OK, f"master offers {len(streams)} rendition(s):")
|
||||||
|
for st in streams:
|
||||||
|
bw = re.search(r"BANDWIDTH=(\d+)", st)
|
||||||
|
res = re.search(r"RESOLUTION=([\dx]+)", st)
|
||||||
|
cod = re.search(r'CODECS="([^"]+)"', st)
|
||||||
|
codecs = cod.group(1) if cod else "?"
|
||||||
|
kind = "audio-only" if ("avc" not in codecs and "hvc" not in codecs) else "video"
|
||||||
|
print(f" {kind:11s} {int(bw.group(1)) // 1000 if bw else '?':>6} kbps "
|
||||||
|
f"{res.group(1) if res else '-':>10} {codecs}")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||||
|
ap.add_argument("--entry", help="entry id to exercise the entry API against")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
print(f"preflight {datetime.now():%Y-%m-%d %H:%M:%S}\n")
|
||||||
|
host_ok = check_deps() & check_disk()
|
||||||
|
|
||||||
|
try:
|
||||||
|
jar = firefox_jar()
|
||||||
|
except Exception as exc:
|
||||||
|
say(BAD, f"cannot read Firefox cookies: {type(exc).__name__}: {exc}")
|
||||||
|
return 1
|
||||||
|
brfc, have_cookie = check_cookies(jar)
|
||||||
|
|
||||||
|
jwt = check_session(brfc) if have_cookie else None
|
||||||
|
if jwt and args.entry:
|
||||||
|
check_entry(jwt, args.entry)
|
||||||
|
|
||||||
|
print()
|
||||||
|
if not jwt:
|
||||||
|
say(BAD, "NOT READY -- sign in at https://www.bristolrovers.co.uk/user/login "
|
||||||
|
"in Firefox, then run this again")
|
||||||
|
return 1
|
||||||
|
if not host_ok:
|
||||||
|
say(BAD, "session is good but this host cannot record -- fix the above")
|
||||||
|
return 1
|
||||||
|
say(OK, "READY")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
+448
@@ -0,0 +1,448 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Unattended recorder: find the live stream, then record it.
|
||||||
|
|
||||||
|
Why this needs a headless browser at all: auth and the token mint are plain
|
||||||
|
HTTP (see discover.py), but the *entry id* for a given match only appears in
|
||||||
|
the page at runtime -- the static HTML carries no id when nothing is live. So
|
||||||
|
we render the page headlessly just long enough to learn the manifest URL, then
|
||||||
|
hand it to ffmpeg and close the browser.
|
||||||
|
|
||||||
|
Nothing here needs your password. It rides the Firefox session cookies, which
|
||||||
|
outlive kickoff by months.
|
||||||
|
|
||||||
|
# start now, keep looking for up to 45 min, then record 150 min
|
||||||
|
python record_live.py --max-wait 45 --duration 150
|
||||||
|
|
||||||
|
# take both feeds at once, or pin one when you know which you hold
|
||||||
|
python record_live.py --feed both
|
||||||
|
python record_live.py --entry 0_xm5vrrw6
|
||||||
|
|
||||||
|
Two different choices, often confused. **--feed** picks which *entry* to record
|
||||||
|
when the page exposes more than one for the same match, typically an audio feed
|
||||||
|
and a video feed. **--prefer** picks a rendition *inside* one entry's master.
|
||||||
|
Getting --feed wrong cannot be rescued by --prefer, and it fails quietly: you
|
||||||
|
get a complete, playable, perfectly healthy recording of the wrong thing.
|
||||||
|
|
||||||
|
# or wait until a wall-clock time first (03:15 before an 03:45 kickoff)
|
||||||
|
python record_live.py --at "2026-09-02 03:15" --max-wait 60
|
||||||
|
|
||||||
|
Recording stops on the duration cap. That is deliberate and load-bearing: the
|
||||||
|
feed does not emit #EXT-X-ENDLIST -- after full time it switches to a static
|
||||||
|
slate and streams forever.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import contextlib
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
ROOT = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
RECORDINGS = os.path.join(ROOT, "recordings")
|
||||||
|
LOGS = os.path.join(ROOT, "logs")
|
||||||
|
LIVE_URL = "https://www.bristolrovers.co.uk/video/live"
|
||||||
|
UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) "
|
||||||
|
"Gecko/20100101 Firefox/131.0")
|
||||||
|
COOKIE_DOMAINS = ("bristolrovers", "streamamg", "kaltura", "akamaized")
|
||||||
|
|
||||||
|
|
||||||
|
def log(msg):
|
||||||
|
line = f"[{datetime.now():%Y-%m-%d %H:%M:%S}] {msg}"
|
||||||
|
print(line, flush=True)
|
||||||
|
os.makedirs(LOGS, exist_ok=True)
|
||||||
|
with open(os.path.join(LOGS, "record_live.log"), "a", encoding="utf-8") as fh:
|
||||||
|
fh.write(line + chr(10))
|
||||||
|
|
||||||
|
|
||||||
|
def firefox_cookies():
|
||||||
|
from yt_dlp.cookies import extract_cookies_from_browser
|
||||||
|
with contextlib.redirect_stderr(io.StringIO()):
|
||||||
|
jar = extract_cookies_from_browser("firefox")
|
||||||
|
out = []
|
||||||
|
for c in jar:
|
||||||
|
dom = c.domain or ""
|
||||||
|
if not any(k in dom.lower() for k in COOKIE_DOMAINS):
|
||||||
|
continue
|
||||||
|
out.append({"name": c.name, "value": c.value or "", "domain": dom,
|
||||||
|
"path": c.path or "/",
|
||||||
|
"expires": float(c.expires) if c.expires else -1,
|
||||||
|
"httpOnly": False, "secure": bool(c.secure),
|
||||||
|
"sameSite": "Lax"})
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
TOKEN_URL = ("https://www.bristolrovers.co.uk/oauth/browser-token"
|
||||||
|
"?client_id=streamamg&token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aid_token")
|
||||||
|
# Public client key, shipped to every browser -- not a secret.
|
||||||
|
API_KEY = "50ivbQ3IXpF2mwT4tPAJ2V6cVm4CGRT2YKmDK5v6"
|
||||||
|
|
||||||
|
|
||||||
|
def mint_master(entry_id, quiet=False):
|
||||||
|
"""Mint a fresh master playlist URL in plain Python. No browser.
|
||||||
|
|
||||||
|
Returns (master_url, rendition_kinds). The master token lives ~25 seconds,
|
||||||
|
so mint immediately before you use the URL -- classify with one call, then
|
||||||
|
mint again for the recording.
|
||||||
|
|
||||||
|
Handing yt-dlp the *media* playlist (hdntl, path token) fails: Python
|
||||||
|
clients get 403 on that URL form, and yt-dlp fetches the playlist with
|
||||||
|
urllib before ffmpeg ever runs. The *master* (hdnts, query token) is fetched
|
||||||
|
fine by Python, so we mint one here and pass that instead. It lives only
|
||||||
|
~25 seconds, which is why it is minted immediately before recording.
|
||||||
|
"""
|
||||||
|
import http.cookiejar, json, re, urllib.request
|
||||||
|
from yt_dlp.cookies import extract_cookies_from_browser
|
||||||
|
|
||||||
|
with contextlib.redirect_stderr(io.StringIO()):
|
||||||
|
jar = extract_cookies_from_browser("firefox")
|
||||||
|
cj = http.cookiejar.CookieJar()
|
||||||
|
for c in jar:
|
||||||
|
if "bristolrovers" in (c.domain or "").lower():
|
||||||
|
cj.set_cookie(c)
|
||||||
|
op = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
|
||||||
|
hdrs = {"User-Agent": UA, "Accept": "application/json, text/plain, */*"}
|
||||||
|
|
||||||
|
body = op.open(urllib.request.Request(TOKEN_URL, headers=hdrs),
|
||||||
|
timeout=20).read().decode("utf-8", "replace")
|
||||||
|
m = re.search(r"(eyJ[\w-]+\.eyJ[\w-]+\.[\w-]+)", body)
|
||||||
|
if not m:
|
||||||
|
log(" mint: no JWT in browser-token response (session expired?)")
|
||||||
|
return None, []
|
||||||
|
jwt = m.group(1)
|
||||||
|
|
||||||
|
req = urllib.request.Request(
|
||||||
|
f"https://api.playback.streamamg.com/v1/entry/{entry_id}",
|
||||||
|
headers={**hdrs, "Authorization": "Bearer " + jwt, "x-api-key": API_KEY,
|
||||||
|
"Origin": "https://www.bristolrovers.co.uk",
|
||||||
|
"Referer": "https://www.bristolrovers.co.uk/"})
|
||||||
|
data = json.loads(urllib.request.urlopen(req, timeout=20)
|
||||||
|
.read().decode("utf-8", "replace"))
|
||||||
|
hls = (data.get("media") or {}).get("hls") or None
|
||||||
|
kinds = log_renditions(hls, quiet=quiet) if hls else []
|
||||||
|
return hls, kinds
|
||||||
|
|
||||||
|
|
||||||
|
def log_renditions(master_url, quiet=False):
|
||||||
|
"""Log what the master offers, and return the kind of each rendition.
|
||||||
|
|
||||||
|
The return value is what feed_kind() classifies an entry on. Codecs are the
|
||||||
|
only trustworthy signal: entry ids look alike, and the same id comes back
|
||||||
|
week after week for the same feed.
|
||||||
|
"""
|
||||||
|
import urllib.request
|
||||||
|
try:
|
||||||
|
req = urllib.request.Request(master_url, headers={"User-Agent": UA,
|
||||||
|
"Referer": "https://www.bristolrovers.co.uk/"})
|
||||||
|
body = urllib.request.urlopen(req, timeout=15).read().decode("utf-8", "replace")
|
||||||
|
except Exception as exc:
|
||||||
|
log(f" (could not read master for rendition list: {exc})")
|
||||||
|
return []
|
||||||
|
PREFIX = "#EXT-X-STREAM-INF:"
|
||||||
|
streams = [ln[len(PREFIX):] for ln in body.splitlines()
|
||||||
|
if ln.startswith(PREFIX)]
|
||||||
|
kinds = []
|
||||||
|
if not quiet:
|
||||||
|
log(f" master offers {len(streams)} rendition(s):")
|
||||||
|
for st in streams:
|
||||||
|
bw = re.search(r"BANDWIDTH=(\d+)", st)
|
||||||
|
res = re.search(r"RESOLUTION=([\dx]+)", st)
|
||||||
|
cod = re.search(r'CODECS="([^"]+)"', st)
|
||||||
|
codecs = cod.group(1) if cod else "?"
|
||||||
|
kind = "audio-only" if ("avc" not in codecs and "hvc" not in codecs) else "video"
|
||||||
|
kinds.append(kind)
|
||||||
|
if not quiet:
|
||||||
|
log(f" {kind:11s} {int(bw.group(1))//1000 if bw else '?':>6} kbps "
|
||||||
|
f"{res.group(1) if res else '-':>10} {codecs}")
|
||||||
|
return kinds
|
||||||
|
|
||||||
|
|
||||||
|
def feed_kind(kinds):
|
||||||
|
"""'video' if the master carries any video rendition, else 'audio'."""
|
||||||
|
if not kinds:
|
||||||
|
return "unknown"
|
||||||
|
return "video" if any(k == "video" for k in kinds) else "audio"
|
||||||
|
|
||||||
|
|
||||||
|
def find_manifest(timeout_s=90, settle_s=15):
|
||||||
|
"""Load the live page headlessly and return every entry id it exposes.
|
||||||
|
|
||||||
|
The browser is needed only for this: the entry id appears at runtime, and
|
||||||
|
the static HTML carries none when nothing is live. Everything after this
|
||||||
|
is plain HTTP.
|
||||||
|
|
||||||
|
Returns a list, not one id, because the page can expose more than one feed
|
||||||
|
for the same match -- an audio entry and a video entry. Which manifest
|
||||||
|
loads first is a race, and it has been observed to come out differently on
|
||||||
|
the same night. Returning the first one silently records the wrong feed.
|
||||||
|
So keep watching for `settle_s` after the first hit, and let the caller
|
||||||
|
choose.
|
||||||
|
"""
|
||||||
|
from playwright.sync_api import sync_playwright
|
||||||
|
|
||||||
|
entries = [] # discovery order, de-duplicated
|
||||||
|
first_seen = [None]
|
||||||
|
|
||||||
|
with sync_playwright() as p:
|
||||||
|
browser = p.firefox.launch(headless=True)
|
||||||
|
ctx = browser.new_context(user_agent=UA,
|
||||||
|
viewport={"width": 1440, "height": 900})
|
||||||
|
ctx.add_cookies(firefox_cookies())
|
||||||
|
page = ctx.new_page()
|
||||||
|
|
||||||
|
def on_response(resp):
|
||||||
|
u = resp.url
|
||||||
|
if ".m3u8" not in u.split("?")[0].lower():
|
||||||
|
return
|
||||||
|
if "hdntl=" not in u and "hdnts=" not in u:
|
||||||
|
return
|
||||||
|
m = re.search(r"/(\d+_[a-z0-9]+)/", u, re.I)
|
||||||
|
if not m or m.group(1) in entries:
|
||||||
|
return
|
||||||
|
entries.append(m.group(1))
|
||||||
|
if first_seen[0] is None:
|
||||||
|
first_seen[0] = time.time()
|
||||||
|
|
||||||
|
page.on("response", on_response)
|
||||||
|
try:
|
||||||
|
page.goto(LIVE_URL, wait_until="domcontentloaded", timeout=60000)
|
||||||
|
except Exception as exc:
|
||||||
|
log(f" page load failed: {exc}")
|
||||||
|
browser.close()
|
||||||
|
return None
|
||||||
|
|
||||||
|
# The manifest is fetched on load, before any click. We still click,
|
||||||
|
# because a stream that has only just gone live may need the nudge.
|
||||||
|
deadline = time.time() + timeout_s
|
||||||
|
clicked = False
|
||||||
|
while time.time() < deadline:
|
||||||
|
# Stop once the page has settled: a feed the player never requests
|
||||||
|
# will not appear no matter how long we wait.
|
||||||
|
if first_seen[0] and time.time() - first_seen[0] > settle_s:
|
||||||
|
break
|
||||||
|
if not clicked:
|
||||||
|
for sel in (".bmpui-ui-hugeplaybacktogglebutton",
|
||||||
|
".bmpui-ui-playbacktogglebutton",
|
||||||
|
"button[aria-label*='play' i]"):
|
||||||
|
try:
|
||||||
|
el = page.query_selector(sel)
|
||||||
|
if el and el.is_visible():
|
||||||
|
el.click()
|
||||||
|
clicked = True
|
||||||
|
break
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
with contextlib.suppress(Exception):
|
||||||
|
page.evaluate("() => { const v=document.querySelector('video');"
|
||||||
|
" if (v) { v.muted = true; v.play(); } }")
|
||||||
|
page.wait_for_timeout(2000)
|
||||||
|
|
||||||
|
browser.close()
|
||||||
|
|
||||||
|
return entries
|
||||||
|
|
||||||
|
|
||||||
|
def record(url, duration_min, prefer='best', tag=None):
|
||||||
|
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||||
|
name = f"brfc-{stamp}-{tag}.ts" if tag else f"brfc-{stamp}.ts"
|
||||||
|
out = os.path.join(RECORDINGS, name)
|
||||||
|
os.makedirs(RECORDINGS, exist_ok=True)
|
||||||
|
logfile = os.path.join(LOGS, f"yt-dlp-{stamp}.log")
|
||||||
|
cmd = [
|
||||||
|
sys.executable, "-m", "yt_dlp",
|
||||||
|
# Force ffmpeg. yt-dlp's native HLS downloader fetches variant
|
||||||
|
# playlists with Python and gets 403/404 on the path-token URLs;
|
||||||
|
# ffmpeg fetches them fine. This is not a preference, it is required.
|
||||||
|
"--downloader", "ffmpeg",
|
||||||
|
"--hls-use-mpegts", # a killed process still leaves a playable file
|
||||||
|
"--no-part",
|
||||||
|
"--cookies-from-browser", "firefox",
|
||||||
|
"--user-agent", UA,
|
||||||
|
"--referer", "https://www.bristolrovers.co.uk/",
|
||||||
|
"--downloader-args",
|
||||||
|
# ffmpeg 4.3.1 has no -reconnect_on_network_error.
|
||||||
|
"ffmpeg_i:-reconnect 1 -reconnect_streamed 1 "
|
||||||
|
"-reconnect_delay_max 30",
|
||||||
|
"--downloader-args", f"ffmpeg_o:-t {duration_min * 60}",
|
||||||
|
"-o", out,
|
||||||
|
]
|
||||||
|
if prefer == "audio":
|
||||||
|
# Pin audio when the entitlement is audio-only: if a video rendition
|
||||||
|
# ever appears in the master, taking it would 403 on the segments.
|
||||||
|
cmd += ["-f", "bestaudio/best"]
|
||||||
|
elif prefer == "video":
|
||||||
|
cmd += ["-f", "bestvideo*+bestaudio/best"]
|
||||||
|
cmd.append(url)
|
||||||
|
log(f"recording -> {os.path.basename(out)} (cap {duration_min} min)")
|
||||||
|
log(f" yt-dlp log -> {os.path.basename(logfile)}")
|
||||||
|
with open(logfile, "w", encoding="utf-8") as fh:
|
||||||
|
return subprocess.Popen(cmd, stdout=fh, stderr=subprocess.STDOUT, cwd=ROOT)
|
||||||
|
|
||||||
|
|
||||||
|
def classify_entries(entries):
|
||||||
|
"""Label each discovered entry video or audio, by minting its master once.
|
||||||
|
|
||||||
|
Returns [(entry_id, kind)] in discovery order, skipping anything that will
|
||||||
|
not mint. The master minted here is thrown away -- its token lives ~25
|
||||||
|
seconds, so the recording mints its own.
|
||||||
|
"""
|
||||||
|
out = []
|
||||||
|
for entry in entries:
|
||||||
|
try:
|
||||||
|
url, kinds = mint_master(entry)
|
||||||
|
except Exception as exc:
|
||||||
|
log(f" {entry}: mint failed: {type(exc).__name__}: {exc}")
|
||||||
|
continue
|
||||||
|
if not url:
|
||||||
|
continue
|
||||||
|
kind = feed_kind(kinds)
|
||||||
|
log(f" {entry} is a {kind} feed")
|
||||||
|
out.append((entry, kind))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def choose_feeds(chosen, want):
|
||||||
|
"""Order classified entries by preference. 'both' and 'any' take them all.
|
||||||
|
|
||||||
|
A preference is not a filter: the non-preferred feeds stay on the list as
|
||||||
|
fallbacks, because a feed you hold no pass for still resolves a master and
|
||||||
|
only fails later, on the segments.
|
||||||
|
"""
|
||||||
|
if want in ("both", "any"):
|
||||||
|
return chosen
|
||||||
|
first = [c for c in chosen if c[1] == want]
|
||||||
|
rest = [c for c in chosen if c[1] != want]
|
||||||
|
if not first and rest:
|
||||||
|
log(f" no {want} feed on offer -- falling back to "
|
||||||
|
f"{', '.join(k for _, k in rest)}")
|
||||||
|
return first + rest
|
||||||
|
|
||||||
|
|
||||||
|
def run_recorders(targets, args):
|
||||||
|
"""Record, and report whether a recording actually ran.
|
||||||
|
|
||||||
|
'both' starts every feed at once: concurrency is not limited and a second
|
||||||
|
copy is cheap insurance. Otherwise take the first choice and fall through
|
||||||
|
to the next only when the downloader dies immediately -- which is what an
|
||||||
|
unentitled feed looks like, the master resolving and the segments 403ing.
|
||||||
|
"""
|
||||||
|
def start(entry, kind):
|
||||||
|
try:
|
||||||
|
url, _ = mint_master(entry, quiet=True)
|
||||||
|
except Exception as exc:
|
||||||
|
log(f" {entry}: mint failed: {type(exc).__name__}: {exc}")
|
||||||
|
return None
|
||||||
|
if not url:
|
||||||
|
log(f" {entry}: could not re-mint a master; skipping")
|
||||||
|
return None
|
||||||
|
proc = record(url, args.duration, args.prefer, tag=kind)
|
||||||
|
log(f" {entry} ({kind}): downloader pid {proc.pid}")
|
||||||
|
return proc, time.time()
|
||||||
|
|
||||||
|
def finish(entry, kind, proc, t0):
|
||||||
|
proc.wait()
|
||||||
|
ran = time.time() - t0
|
||||||
|
log(f" {entry} ({kind}): rc={proc.returncode} after {ran:.0f}s")
|
||||||
|
# A master outlives its stream, so minting one proves nothing about
|
||||||
|
# whether the match is up. Only a download that actually ran does.
|
||||||
|
if proc.returncode == 0 or ran > 120:
|
||||||
|
return True
|
||||||
|
log(" died immediately -- not live yet, or no pass for this feed")
|
||||||
|
return False
|
||||||
|
|
||||||
|
if args.feed == "both":
|
||||||
|
started = []
|
||||||
|
for entry, kind in targets:
|
||||||
|
s = start(entry, kind)
|
||||||
|
if s is not None:
|
||||||
|
started.append((entry, kind) + s)
|
||||||
|
# Every one must be waited on, so collect results before reducing --
|
||||||
|
# any() would short-circuit and orphan a running downloader.
|
||||||
|
results = [finish(*row) for row in started]
|
||||||
|
return any(results)
|
||||||
|
|
||||||
|
# One feed at a time, falling through on immediate death so an unentitled
|
||||||
|
# first choice does not cost the match.
|
||||||
|
for entry, kind in targets:
|
||||||
|
s = start(entry, kind)
|
||||||
|
if s is None:
|
||||||
|
continue
|
||||||
|
if finish(entry, kind, *s):
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--at", help='wall-clock start, e.g. "2026-09-02 03:15"')
|
||||||
|
ap.add_argument("--max-wait", type=int, default=45,
|
||||||
|
help="minutes to keep looking for a live stream")
|
||||||
|
ap.add_argument("--duration", type=int, default=150,
|
||||||
|
help="recording cap in minutes")
|
||||||
|
ap.add_argument("--poll", type=int, default=60, help="seconds between attempts")
|
||||||
|
ap.add_argument("--prefer", choices=("best", "audio", "video"), default="best",
|
||||||
|
help="which rendition to take when the master offers both; "
|
||||||
|
"'best' is yt-dlp's default and prefers video")
|
||||||
|
ap.add_argument("--entry", help="skip discovery and record this entry id, "
|
||||||
|
"bypassing --feed entirely")
|
||||||
|
ap.add_argument("--feed", choices=("video", "audio", "any", "both"),
|
||||||
|
default="video",
|
||||||
|
help="which FEED to record when the page exposes more than "
|
||||||
|
"one entry. 'video' prefers a video feed and falls "
|
||||||
|
"back to audio; 'both' records every feed in parallel. "
|
||||||
|
"Distinct from --prefer, which picks a rendition "
|
||||||
|
"inside one feed and cannot rescue a wrong feed.")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
if args.at:
|
||||||
|
target = datetime.strptime(args.at, "%Y-%m-%d %H:%M")
|
||||||
|
wait = (target - datetime.now()).total_seconds()
|
||||||
|
if wait > 0:
|
||||||
|
log(f"sleeping {wait/3600:.2f}h until {target:%Y-%m-%d %H:%M}")
|
||||||
|
time.sleep(wait)
|
||||||
|
|
||||||
|
deadline = time.time() + args.max_wait * 60
|
||||||
|
attempt = 0
|
||||||
|
while time.time() < deadline:
|
||||||
|
attempt += 1
|
||||||
|
if args.entry:
|
||||||
|
entries = [args.entry]
|
||||||
|
log(f"attempt {attempt}: pinned entry {args.entry}")
|
||||||
|
else:
|
||||||
|
log(f"attempt {attempt}: looking for a live stream")
|
||||||
|
try:
|
||||||
|
entries = find_manifest()
|
||||||
|
except Exception as exc:
|
||||||
|
log(f" error: {type(exc).__name__}: {exc}")
|
||||||
|
entries = []
|
||||||
|
|
||||||
|
if not entries:
|
||||||
|
log(f" nothing live yet; retrying in {args.poll}s")
|
||||||
|
time.sleep(args.poll)
|
||||||
|
continue
|
||||||
|
|
||||||
|
log(f" live entries: {', '.join(entries)}")
|
||||||
|
chosen = classify_entries(entries)
|
||||||
|
if not chosen:
|
||||||
|
log(" could not mint a manifest for any entry; retrying")
|
||||||
|
time.sleep(args.poll)
|
||||||
|
continue
|
||||||
|
|
||||||
|
targets = choose_feeds(chosen, args.feed)
|
||||||
|
if not targets:
|
||||||
|
log(f" no {args.feed} feed among {[k for _, k in chosen]}; retrying")
|
||||||
|
time.sleep(args.poll)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if run_recorders(targets, args) or time.time() >= deadline:
|
||||||
|
return 0
|
||||||
|
time.sleep(args.poll)
|
||||||
|
|
||||||
|
log("gave up: no live stream appeared within the wait window")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -27,6 +27,13 @@
|
|||||||
// once the endpoint is known -- it writes a lot.
|
// once the endpoint is known -- it writes a lot.
|
||||||
const TRACE = true;
|
const TRACE = true;
|
||||||
|
|
||||||
|
// Start playback without a human. The player needs a click, but the browser
|
||||||
|
// only has to trigger the manifest fetch -- ffmpeg records the real audio
|
||||||
|
// independently, so playing MUTED costs nothing and muted autoplay is
|
||||||
|
// permitted by Firefox policy where audible autoplay is not.
|
||||||
|
const AUTOPLAY = true;
|
||||||
|
const AUTOPLAY_TIMEOUT_MS = 180000; // pre-match margin is long; keep trying
|
||||||
|
|
||||||
const IDLE_OPACITY = 0.063; // 16/255
|
const IDLE_OPACITY = 0.063; // 16/255
|
||||||
const MAX_BODY = 20000; // never log a whole media segment
|
const MAX_BODY = 20000; // never log a whole media segment
|
||||||
|
|
||||||
@@ -188,4 +195,67 @@
|
|||||||
panel.appendChild(row);
|
panel.appendChild(row);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- autoplay -------------------------------------------------------------
|
||||||
|
// Runs in whichever frame owns the <video>, which may be the player iframe.
|
||||||
|
function findVideo() {
|
||||||
|
const v = document.querySelector('video');
|
||||||
|
return (v && v.isConnected) ? v : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function clickPlayButton() {
|
||||||
|
const sels = [
|
||||||
|
'.bmpui-ui-hugeplaybacktogglebutton', // Bitmovin centre button
|
||||||
|
'.bmpui-ui-playbacktogglebutton', // Bitmovin control bar
|
||||||
|
'button[aria-label*="play" i]',
|
||||||
|
'[class*="play-button"]', '[class*="playButton"]'
|
||||||
|
];
|
||||||
|
for (const s of sels) {
|
||||||
|
const el = document.querySelector(s);
|
||||||
|
if (el && el.offsetParent !== null) { // must be visible to be real
|
||||||
|
el.click();
|
||||||
|
console.log('[brfc-sniffer] autoplay: clicked', s);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
function startAutoplay() {
|
||||||
|
if (!AUTOPLAY) return;
|
||||||
|
const deadline = Date.now() + AUTOPLAY_TIMEOUT_MS;
|
||||||
|
|
||||||
|
const timer = setInterval(() => {
|
||||||
|
const v = findVideo();
|
||||||
|
|
||||||
|
// Done once something is actually playing. Critically, we must stop
|
||||||
|
// here: the play control is a TOGGLE, so clicking it again would pause
|
||||||
|
// playback and silently end the recording.
|
||||||
|
if (v && !v.paused && !v.ended && v.readyState > 2) {
|
||||||
|
console.log('[brfc-sniffer] autoplay: playing, standing down');
|
||||||
|
clearInterval(timer);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (Date.now() > deadline) {
|
||||||
|
console.log('[brfc-sniffer] autoplay: gave up after',
|
||||||
|
AUTOPLAY_TIMEOUT_MS / 1000, 's');
|
||||||
|
clearInterval(timer);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (v) {
|
||||||
|
v.muted = true; // audible autoplay is blocked; muted is not
|
||||||
|
const p = v.play();
|
||||||
|
if (p && p.catch) p.catch(() => { clickPlayButton(); });
|
||||||
|
} else {
|
||||||
|
clickPlayButton(); // player chrome may exist before <video>
|
||||||
|
}
|
||||||
|
}, 2000);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (document.readyState === 'loading') {
|
||||||
|
document.addEventListener('DOMContentLoaded', startAutoplay);
|
||||||
|
} else {
|
||||||
|
startAutoplay();
|
||||||
|
}
|
||||||
})();
|
})();
|
||||||
|
|||||||
+128
@@ -0,0 +1,128 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Backup starter: confirm something is actually recording, else start it.
|
||||||
|
|
||||||
|
Schedule this a few minutes before kickoff, behind the primary recording task.
|
||||||
|
It exists because a scheduled launch of record_live.py has been seen to hang
|
||||||
|
dead at interpreter startup -- process alive, zero CPU, nothing logged, forever.
|
||||||
|
Unattended, that loses the match silently.
|
||||||
|
|
||||||
|
So "is a recorder process running" is the wrong question: the hung process
|
||||||
|
answers yes. This asks whether recording is *progressing*:
|
||||||
|
|
||||||
|
1. a .ts in recordings/ grew in the last --quiet-secs -> healthy, leave it
|
||||||
|
2. record_live.log was written in the last --quiet-secs -> mid search loop,
|
||||||
|
leave it
|
||||||
|
3. neither -> stale. Kill any
|
||||||
|
recorder process
|
||||||
|
and start a fresh
|
||||||
|
one.
|
||||||
|
|
||||||
|
Killing only ever targets a recorder that has gone quiet on both signals, so a
|
||||||
|
healthy capture is never interrupted.
|
||||||
|
|
||||||
|
python watchdog.py --duration 170 --max-wait 55
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import glob
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
ROOT = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
RECORDINGS = os.path.join(ROOT, "recordings")
|
||||||
|
LOGS = os.path.join(ROOT, "logs")
|
||||||
|
RECORDER = os.path.join(ROOT, "record_live.py")
|
||||||
|
|
||||||
|
|
||||||
|
def log(msg):
|
||||||
|
line = f"[{datetime.now():%Y-%m-%d %H:%M:%S}] {msg}"
|
||||||
|
print(line, flush=True)
|
||||||
|
os.makedirs(LOGS, exist_ok=True)
|
||||||
|
with open(os.path.join(LOGS, "watchdog.log"), "a", encoding="utf-8") as fh:
|
||||||
|
fh.write(line + chr(10))
|
||||||
|
|
||||||
|
|
||||||
|
def powershell(cmd):
|
||||||
|
try:
|
||||||
|
out = subprocess.run(["powershell", "-NoProfile", "-Command", cmd],
|
||||||
|
capture_output=True, text=True, timeout=60)
|
||||||
|
return out.stdout.strip()
|
||||||
|
except Exception as exc:
|
||||||
|
log(f" powershell failed: {type(exc).__name__}: {exc}")
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def recorder_pids():
|
||||||
|
"""PIDs of any running record_live.py, hung or healthy."""
|
||||||
|
out = powershell(
|
||||||
|
"Get-CimInstance Win32_Process -Filter \"Name='python.exe'\" | "
|
||||||
|
"Where-Object { $_.CommandLine -match 'record_live' } | "
|
||||||
|
"ForEach-Object { $_.ProcessId }")
|
||||||
|
return [int(x) for x in out.split() if x.strip().isdigit()]
|
||||||
|
|
||||||
|
|
||||||
|
def newest_mtime(pattern):
|
||||||
|
files = glob.glob(pattern)
|
||||||
|
return max((os.path.getmtime(f) for f in files), default=0)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||||
|
ap.add_argument("--duration", type=int, default=170,
|
||||||
|
help="recording cap in minutes for a run we start")
|
||||||
|
ap.add_argument("--max-wait", type=int, default=55,
|
||||||
|
help="minutes to keep looking for a live stream")
|
||||||
|
ap.add_argument("--prefer", choices=("best", "audio", "video"), default="best")
|
||||||
|
ap.add_argument("--quiet-secs", type=int, default=180,
|
||||||
|
help="how stale both signals must be before we intervene")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
log("watchdog: checking whether anything is recording")
|
||||||
|
now = time.time()
|
||||||
|
|
||||||
|
ts_age = now - newest_mtime(os.path.join(RECORDINGS, "*.ts"))
|
||||||
|
log_age = now - newest_mtime(os.path.join(LOGS, "record_live.log"))
|
||||||
|
pids = recorder_pids()
|
||||||
|
log(f" recorder pids {pids or 'none'}; newest .ts {ts_age:.0f}s old; "
|
||||||
|
f"record_live.log {log_age:.0f}s old")
|
||||||
|
|
||||||
|
if ts_age < args.quiet_secs:
|
||||||
|
log(" a recording is growing -- healthy, nothing to do")
|
||||||
|
return 0
|
||||||
|
if pids and log_age < args.quiet_secs:
|
||||||
|
log(" recorder is logging -- mid search loop, nothing to do")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if pids:
|
||||||
|
log(f" recorder {pids} is alive but both signals are stale -- killing it")
|
||||||
|
for pid in pids:
|
||||||
|
powershell(f"Stop-Process -Id {pid} -Force -ErrorAction SilentlyContinue")
|
||||||
|
time.sleep(3)
|
||||||
|
still = recorder_pids()
|
||||||
|
if still:
|
||||||
|
log(f" WARNING: {still} survived the kill; starting anyway")
|
||||||
|
else:
|
||||||
|
log(" nothing is recording")
|
||||||
|
|
||||||
|
cmd = [sys.executable, RECORDER,
|
||||||
|
"--max-wait", str(args.max_wait),
|
||||||
|
"--duration", str(args.duration),
|
||||||
|
"--prefer", args.prefer]
|
||||||
|
log(f" starting: {' '.join(cmd[1:])}")
|
||||||
|
# Detached: the watchdog task exits immediately, the recorder outlives it.
|
||||||
|
flags = 0
|
||||||
|
if os.name == "nt":
|
||||||
|
flags = subprocess.CREATE_NEW_PROCESS_GROUP | getattr(
|
||||||
|
subprocess, "DETACHED_PROCESS", 0)
|
||||||
|
proc = subprocess.Popen(cmd, cwd=ROOT, creationflags=flags,
|
||||||
|
stdin=subprocess.DEVNULL,
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
stderr=subprocess.DEVNULL)
|
||||||
|
log(f" started recorder pid {proc.pid}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Reference in New Issue
Block a user