Capture Bristol Rovers video pass streams for personal time-shifting
Verified end to end against a live match: 1080p H.264/AAC MPEG-TS at ~5.2 Mbit/s, recorded while the browser played the same stream. Components: - tampermonkey sniffer hooks fetch/XHR at document-start, captures the manifest URL verbatim and POSTs it to the local catcher. v0.4 also traces the API chain to find which call mints the token. - catcher receives the URL and launches yt-dlp. Dedupes on the Kaltura asset id and checks downloader liveness, so a mid-match page reload is both safe and the recovery path for a dead downloader. - creds stores secrets in the OS vault rather than a gitignored file, and refuses to run on a backend that does not encrypt. Protocol notes are in the README: the stream uses a two-stage token chain (a ~25s hdnts token for the master playlist, a ~24h hdntl token for media), concurrency is unrestricted, and the live window is ~30s with no DVR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kyt7QHQvLJtQRuiY4wAqKB
This commit is contained in:
Mikkeli Matlock
@@ -0,0 +1,7 @@
|
||||
recordings/
|
||||
logs/
|
||||
stream-url.txt
|
||||
cookies.txt
|
||||
*.mkv
|
||||
*.mp4
|
||||
*.ts
|
||||
@@ -0,0 +1,111 @@
|
||||
# brfc-stream-pirate
|
||||
|
||||
Personal time-shifting archive for Bristol Rovers video pass streams.
|
||||
Kickoffs land at unfriendly hours in JST; this records them unattended.
|
||||
|
||||
Stream is **plain HLS, no DRM** (StreamAMG behind Akamai). Nothing here breaks
|
||||
a protection measure -- it replays the same authenticated request the player
|
||||
makes, using your own paid session.
|
||||
|
||||
## Pieces
|
||||
|
||||
| File | Job |
|
||||
|---|---|
|
||||
| `tampermonkey/brfc-stream-sniffer.user.js` | Hooks fetch/XHR in the page, captures the manifest URL verbatim, POSTs it to the catcher |
|
||||
| `catcher.py` | Listens on 127.0.0.1:8787, hands the URL to yt-dlp, records to `recordings/` |
|
||||
| `probe.py` | Byte-exact manifest fetch + concurrency test |
|
||||
|
||||
## The token
|
||||
|
||||
Manifest URLs carry an Akamai path token:
|
||||
|
||||
hdntl=exp=<unix>~acl=/*~id=<n>~data=hdntl~hmac=<sig>
|
||||
|
||||
- `exp` -- ~24h validity
|
||||
- `acl=/*` -- wildcard over the whole host, not just your match
|
||||
- `hmac` -- signature over the literal path
|
||||
|
||||
It is a bearer credential. Anyone holding a live URL can pull the stream until
|
||||
`exp`. Keep it out of pastebins and issue trackers. `stream-url.txt` and
|
||||
`cookies.txt` are gitignored.
|
||||
|
||||
**Do not re-encode the path.** curl normalises `%2f` to `%2F`, which
|
||||
invalidates the HMAC. `probe.py` sends the request target verbatim.
|
||||
|
||||
## Use Firefox for watching
|
||||
|
||||
yt-dlp lifts session cookies from the browser. Chrome 127+ on Windows uses
|
||||
app-bound cookie encryption and extraction fails. Firefox works with no fuss.
|
||||
|
||||
set BRFC_BROWSER=firefox # default
|
||||
|
||||
## Workflow
|
||||
|
||||
pip install yt-dlp # done
|
||||
python catcher.py # leave running
|
||||
|
||||
Install the userscript in Tampermonkey, open the match page, press play. The
|
||||
sniffer catches the manifest, the catcher starts yt-dlp. Recording is MPEG-TS
|
||||
(`--hls-use-mpegts`) so a killed process still leaves a playable file.
|
||||
|
||||
For a 3am kickoff: Task Scheduler opens the match page a few minutes before,
|
||||
the browser autoplays, the sniffer fires, the catcher records. Nobody is awake.
|
||||
|
||||
## Concurrency check
|
||||
|
||||
An Akamai edge token is a stateless HMAC -- the edge validates signature and
|
||||
expiry and serves; it holds no session table, so it cannot count viewers. Any
|
||||
concurrent-stream limit lives in the entitlement API that *issues* tokens,
|
||||
upstream of the CDN. Confirm rather than assume, while the browser is playing:
|
||||
|
||||
python probe.py --url "<manifest from the sniffer>" --parallel 3
|
||||
|
||||
3/3 alongside the playing browser means four concurrent readers of one token
|
||||
and the edge does not care.
|
||||
|
||||
---
|
||||
|
||||
## Findings (2026-08-29, live 2300 JST match)
|
||||
|
||||
Verified against a live stream. Recorded 1080p successfully end to end.
|
||||
|
||||
### Two token types, not one
|
||||
|
||||
The player uses a **two-stage token chain**:
|
||||
|
||||
| Token | Where | TTL | Purpose |
|
||||
|---|---|---|---|
|
||||
| `hdnts=` | query string, `acl=/*` unencoded | **~25 seconds** | fetch the master playlist |
|
||||
| `hdntl=` | URL path, `acl=%2f*` encoded | **~24 hours** | media playlists + segments |
|
||||
|
||||
The master response rewrites each variant URL with a 24h `hdntl` token. So the
|
||||
`hdntl` token is the durable one worth handing to a downloader. Chasing a
|
||||
`hdnts` URL by hand is pointless -- it dies before you can paste it.
|
||||
|
||||
### Confirmed by experiment
|
||||
|
||||
- **Concurrency is not limited.** Two independent ffmpeg downloads ran at full
|
||||
bitrate *while Firefox played the same match*. Three concurrent readers of
|
||||
one token, no complaint from the edge. Akamai token auth is a stateless HMAC
|
||||
check; it has no viewer counter.
|
||||
- **No CDN cookies exist.** All 13 relevant cookies are on `bristolrovers.co.uk`;
|
||||
nothing is scoped to `akamaized.net`. The token in the URL is the whole auth.
|
||||
- **Stream is 1920x1080 H.264 + AAC ~5.2 Mbit/s** -> roughly 4.7 GB per match.
|
||||
- **Output is MPEG-TS** even though yt-dlp names it `.mp4`. That is what
|
||||
`--hls-use-mpegts` buys: a killed process still leaves a playable file.
|
||||
- Player is **Bitmovin**, embedded on the club site. `0_xm5vrrw6` is a Kaltura
|
||||
entry id and identifies the match -- the catcher dedupes on it.
|
||||
|
||||
### Gotcha: use ffmpeg/yt-dlp, not a hand-rolled HTTP client
|
||||
|
||||
Python's `http.client` gets a **403 on every request**, with any header set,
|
||||
including ffmpeg's exact headers, on an unexpired token from the same IP. The
|
||||
difference is below HTTP (TLS handshake). ffmpeg fetches the same URL fine, so
|
||||
just use yt-dlp. `probe.py` is diagnostic scaffolding and inherits this 403 --
|
||||
do not use it to conclude a token is dead.
|
||||
|
||||
### Firefox, not Chrome
|
||||
|
||||
yt-dlp reads cookies from a running Firefox. Chrome/Edge lock their cookie DB
|
||||
while running (yt-dlp issue 7271), and an unattended job cannot close the
|
||||
browser to read them.
|
||||
+151
@@ -0,0 +1,151 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Receive a manifest URL from the Tampermonkey sniffer and record the match.
|
||||
|
||||
Runs on localhost only. The userscript POSTs whatever manifest URL the player
|
||||
actually requested; we hand it to yt-dlp.
|
||||
|
||||
Reload resilience is the whole point of the liveness check below. A page
|
||||
reload mid-match is expected (network blips, tab crashes), and it must be
|
||||
*safe* -- it must not start a duplicate recording, and it must restart a
|
||||
recording that has died. So we dedupe on "is a downloader for this match
|
||||
still alive", not "have I seen this match before".
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from datetime import datetime
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
|
||||
HOST, PORT = "127.0.0.1", 8787
|
||||
ROOT = os.path.dirname(os.path.abspath(__file__))
|
||||
RECORDINGS = os.path.join(ROOT, "recordings")
|
||||
LOGS = os.path.join(ROOT, "logs")
|
||||
|
||||
# yt-dlp lifts session cookies from here. Firefox is required on Windows:
|
||||
# Chrome/Edge lock their cookie DB while running (yt-dlp issue 7271).
|
||||
BROWSER = os.environ.get("BRFC_BROWSER", "firefox")
|
||||
|
||||
# Hard stop, so an unattended run cannot record forever if the stream never
|
||||
# ends cleanly. Pad past 90 min for half-time, stoppage and post-match.
|
||||
DURATION_MIN = int(os.environ.get("BRFC_DURATION_MIN", "150"))
|
||||
|
||||
# Kaltura entry id (e.g. 0_xm5vrrw6) identifies the match. Every manifest for
|
||||
# one game -- master and each variant, each with its own token -- shares it.
|
||||
ASSET_RE = re.compile(r"/(\d+_[a-z0-9]+)/", re.I)
|
||||
|
||||
_procs = {} # asset key -> Popen of the running downloader
|
||||
_lock = threading.Lock()
|
||||
|
||||
|
||||
def log(msg):
|
||||
line = f"[{datetime.now():%Y-%m-%d %H:%M:%S}] {msg}"
|
||||
print(line, flush=True)
|
||||
os.makedirs(LOGS, exist_ok=True)
|
||||
with open(os.path.join(LOGS, "catcher.log"), "a", encoding="utf-8") as fh:
|
||||
fh.write(line + "\n")
|
||||
|
||||
|
||||
def asset_key(url):
|
||||
m = ASSET_RE.search(url)
|
||||
return m.group(1) if m else url
|
||||
|
||||
|
||||
def start_recording(key, url, user_agent, page):
|
||||
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||
out = os.path.join(RECORDINGS, f"brfc-{key}-{stamp}.ts")
|
||||
os.makedirs(RECORDINGS, exist_ok=True)
|
||||
|
||||
cmd = [
|
||||
sys.executable, "-m", "yt_dlp",
|
||||
"--hls-use-mpegts", # truncated file still plays
|
||||
"--no-part",
|
||||
"--cookies-from-browser", BROWSER,
|
||||
"--user-agent", user_agent or "Mozilla/5.0",
|
||||
"--referer", page or "https://www.bristolrovers.co.uk/",
|
||||
# yt-dlp's own retry flags do NOT reach the ffmpeg downloader, which is
|
||||
# what actually pulls live HLS. These are ffmpeg's own reconnect knobs.
|
||||
"--downloader-args",
|
||||
"ffmpeg_i:-reconnect 1 -reconnect_streamed 1 "
|
||||
"-reconnect_on_network_error 1 -reconnect_delay_max 30",
|
||||
"--downloader-args", f"ffmpeg_o:-t {DURATION_MIN * 60}",
|
||||
"-o", out,
|
||||
url,
|
||||
]
|
||||
logfile = os.path.join(LOGS, f"yt-dlp-{stamp}.log")
|
||||
fh = open(logfile, "w", encoding="utf-8")
|
||||
proc = subprocess.Popen(cmd, stdout=fh, stderr=subprocess.STDOUT, cwd=ROOT)
|
||||
log(f"START {key} -> {os.path.basename(out)} (pid {proc.pid}, cap {DURATION_MIN}min)")
|
||||
return proc
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def do_POST(self):
|
||||
if self.path not in ("/stream", "/trace"):
|
||||
self.send_error(404)
|
||||
return
|
||||
n = int(self.headers.get("Content-Length", 0))
|
||||
try:
|
||||
body = json.loads(self.rfile.read(n) or b"{}")
|
||||
except json.JSONDecodeError:
|
||||
self.send_error(400, "bad json")
|
||||
return
|
||||
|
||||
if self.path == "/trace":
|
||||
self.do_TRACE_post(body)
|
||||
self.send_response(200)
|
||||
self.send_header("Access-Control-Allow-Origin", "*")
|
||||
self.end_headers()
|
||||
self.wfile.write(b'{"ok":true}')
|
||||
return
|
||||
|
||||
url = body.get("url", "")
|
||||
if not url:
|
||||
self.send_error(400, "no url")
|
||||
return
|
||||
|
||||
key = asset_key(url)
|
||||
action = "ignored"
|
||||
with _lock:
|
||||
proc = _procs.get(key)
|
||||
if proc is not None and proc.poll() is None:
|
||||
action = "already recording" # healthy: reload is a no-op
|
||||
else:
|
||||
if proc is not None:
|
||||
log(f"RESTART {key}: downloader had exited (rc={proc.returncode})")
|
||||
_procs[key] = start_recording(key, url, body.get("userAgent"),
|
||||
body.get("page"))
|
||||
action = "started"
|
||||
|
||||
# Always 200 the browser; the player re-requests manifests constantly.
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Access-Control-Allow-Origin", "*")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps({"ok": True, "action": action}).encode())
|
||||
|
||||
def do_TRACE_post(self, body):
|
||||
"""Discovery mode: append the request/response the page made.
|
||||
|
||||
Used to find which API call mints the stream token, so the chain can
|
||||
run without a GUI browser. Written as JSONL for easy grepping.
|
||||
"""
|
||||
os.makedirs(LOGS, exist_ok=True)
|
||||
with open(os.path.join(LOGS, "trace.jsonl"), "a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(body, ensure_ascii=False) + chr(10))
|
||||
|
||||
def do_OPTIONS(self):
|
||||
self.send_response(204)
|
||||
self.send_header("Access-Control-Allow-Origin", "*")
|
||||
self.send_header("Access-Control-Allow-Headers", "Content-Type")
|
||||
self.end_headers()
|
||||
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
log(f"catcher listening on http://{HOST}:{PORT}/stream (cookies: {BROWSER})")
|
||||
HTTPServer((HOST, PORT), Handler).serve_forever()
|
||||
@@ -0,0 +1,156 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Credential storage for unattended recording.
|
||||
|
||||
Secrets live in the Windows Credential Manager (DPAPI, keyed to your Windows
|
||||
account), not in a file. Nothing here can be committed by accident because
|
||||
nothing here is written to the repo.
|
||||
|
||||
python creds.py --set-login store email + password (prompts)
|
||||
python creds.py --show show what is stored (never the password)
|
||||
python creds.py --clear remove everything
|
||||
|
||||
What this does and does not protect against:
|
||||
|
||||
DOES -- plaintext at rest, and the classic "gitignored file gets committed
|
||||
after a git add -A" accident.
|
||||
DOES -- readable by another Windows user on this machine.
|
||||
NOT -- a process already running as you. Credential Manager unlocks with
|
||||
your login session, so anything running as you can read it. That is
|
||||
inherent to unattended operation: a job that logs in without you
|
||||
present must be able to reach the secret without you present.
|
||||
|
||||
Caveat for Task Scheduler: "Run whether user is logged on or not" runs in a
|
||||
session where DPAPI may not be able to decrypt these entries. Prefer "Run only
|
||||
when user is logged on" for any task that needs credentials.
|
||||
|
||||
PLATFORM NOTE -- this module is only safe where a real vault backend exists:
|
||||
|
||||
Windows WinVaultKeyring (Credential Manager, DPAPI). Good.
|
||||
Linux+GUI SecretService (GNOME Keyring / KWallet). Good.
|
||||
Headless No D-Bus session, no unlocked keyring. `keyring` NOT SAFE.
|
||||
Linux falls back to a fail or plaintext backend.
|
||||
|
||||
On a headless Pi or server, do NOT use this module. Use systemd-creds, which
|
||||
encrypts to the host/TPM key and is decrypted only for the service:
|
||||
|
||||
systemd-creds encrypt --name=brfc-pass - /etc/credstore.encrypted/brfc-pass
|
||||
# then in the unit: LoadCredential=brfc-pass
|
||||
# and read it from: $CREDENTIALS_DIRECTORY/brfc-pass
|
||||
|
||||
Run `python creds.py --check` on each host before trusting it with a secret.
|
||||
"""
|
||||
import argparse
|
||||
import getpass
|
||||
import sys
|
||||
|
||||
try:
|
||||
import keyring
|
||||
except ImportError:
|
||||
sys.exit("keyring not installed. Run: pip install keyring")
|
||||
|
||||
SERVICE = "brfc-stream-pirate"
|
||||
|
||||
# Keys held in the vault.
|
||||
K_USER = "username"
|
||||
K_PASS = "password"
|
||||
K_COOKIE = "session-cookie" # cached login session, refreshed periodically
|
||||
|
||||
|
||||
def get(key):
|
||||
return keyring.get_password(SERVICE, key)
|
||||
|
||||
|
||||
def put(key, value):
|
||||
keyring.set_password(SERVICE, key, value)
|
||||
|
||||
|
||||
def drop(key):
|
||||
try:
|
||||
keyring.delete_password(SERVICE, key)
|
||||
except keyring.errors.PasswordDeleteError:
|
||||
pass # already absent
|
||||
|
||||
|
||||
def load_login():
|
||||
"""Return (username, password), or None if not configured yet."""
|
||||
user, pw = get(K_USER), get(K_PASS)
|
||||
return (user, pw) if user and pw else None
|
||||
|
||||
|
||||
def load_cookie():
|
||||
"""Cached session cookie, or None. Callers must handle it being stale."""
|
||||
return get(K_COOKIE)
|
||||
|
||||
|
||||
def save_cookie(value):
|
||||
put(K_COOKIE, value)
|
||||
|
||||
|
||||
# Backends that do not actually protect anything. `keyring` selects these
|
||||
# silently on a headless box, so a secret stored there is effectively plaintext.
|
||||
UNSAFE_BACKENDS = {"PlaintextKeyring", "Keyring", "fail.Keyring", "EnvironCredential"}
|
||||
|
||||
|
||||
def backend():
|
||||
return keyring.get_keyring().__class__.__name__
|
||||
|
||||
|
||||
def backend_is_safe():
|
||||
name = backend()
|
||||
return name not in UNSAFE_BACKENDS and "fail" not in name.lower()
|
||||
|
||||
|
||||
def _mask(s):
|
||||
if not s:
|
||||
return "(not set)"
|
||||
return s[:2] + "*" * max(0, len(s) - 4) + s[-2:] if len(s) > 4 else "*" * len(s)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
g = ap.add_mutually_exclusive_group(required=True)
|
||||
g.add_argument("--set-login", action="store_true", help="store email + password")
|
||||
g.add_argument("--show", action="store_true", help="show stored username")
|
||||
g.add_argument("--clear", action="store_true", help="remove all stored secrets")
|
||||
g.add_argument("--check", action="store_true",
|
||||
help="report whether this host has a real vault backend")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.check:
|
||||
name = backend()
|
||||
safe = backend_is_safe()
|
||||
print(f"backend : {name}")
|
||||
print(f"verdict : {'SAFE' if safe else 'NOT SAFE -- do not store secrets here'}")
|
||||
if not safe:
|
||||
print(" use systemd-creds on this host (see module docstring)")
|
||||
sys.exit(0 if safe else 1)
|
||||
|
||||
if not backend_is_safe():
|
||||
sys.exit(f"refusing to use backend {backend()!r}: it does not encrypt. "
|
||||
"Run 'python creds.py --check' for guidance.")
|
||||
|
||||
if args.set_login:
|
||||
user = input("email: ").strip()
|
||||
if not user:
|
||||
sys.exit("aborted: empty email")
|
||||
pw = getpass.getpass("password: ")
|
||||
if not pw:
|
||||
sys.exit("aborted: empty password")
|
||||
put(K_USER, user)
|
||||
put(K_PASS, pw)
|
||||
print(f"stored in Windows Credential Manager under '{SERVICE}'")
|
||||
|
||||
elif args.show:
|
||||
print(f"service : {SERVICE}")
|
||||
print(f"username: {get(K_USER) or '(not set)'}")
|
||||
print(f"password: {'(set)' if get(K_PASS) else '(not set)'}")
|
||||
print(f"cookie : {_mask(get(K_COOKIE))}")
|
||||
|
||||
elif args.clear:
|
||||
for k in (K_USER, K_PASS, K_COOKIE):
|
||||
drop(k)
|
||||
print("cleared")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Byte-exact manifest fetcher + concurrency test.
|
||||
|
||||
curl normalises percent-escapes (%2f -> %2F), which breaks an Akamai path
|
||||
token because the HMAC covers the literal path. This sends the request target
|
||||
verbatim via http.client, and can fire N of them at once to see whether the
|
||||
edge objects to concurrent readers of one token.
|
||||
|
||||
python probe.py --url "<manifest url>" # single fetch
|
||||
python probe.py --url "<...>" --parallel 3 # concurrency test
|
||||
python probe.py --url "<...>" --cookie "name=val" # with a CDN cookie
|
||||
"""
|
||||
import argparse
|
||||
import http.client
|
||||
import sys
|
||||
import threading
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
||||
"(KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36")
|
||||
|
||||
|
||||
def fetch(url, cookie=None, referer="https://www.bristolrovers.co.uk/", tag=""):
|
||||
parts = urlsplit(url)
|
||||
# Rebuild the target from the raw split so nothing is re-encoded.
|
||||
target = parts.path + (("?" + parts.query) if parts.query else "")
|
||||
conn = http.client.HTTPSConnection(parts.netloc, timeout=15)
|
||||
try:
|
||||
conn.putrequest("GET", target, skip_accept_encoding=True)
|
||||
conn.putheader("User-Agent", UA)
|
||||
conn.putheader("Accept", "*/*")
|
||||
conn.putheader("Referer", referer)
|
||||
conn.putheader("Origin", referer.rstrip("/"))
|
||||
if cookie:
|
||||
conn.putheader("Cookie", cookie)
|
||||
conn.endheaders()
|
||||
resp = conn.getresponse()
|
||||
body = resp.read()
|
||||
print(f"{tag}HTTP {resp.status} {resp.reason} ({len(body)} bytes)")
|
||||
if resp.status == 200:
|
||||
text = body.decode("utf-8", "replace")
|
||||
variants = [l for l in text.splitlines()
|
||||
if l.startswith("#EXT-X-STREAM-INF") or l.endswith(".m3u8")]
|
||||
segs = [l for l in text.splitlines() if l and not l.startswith("#")]
|
||||
if variants:
|
||||
print(f"{tag} master playlist, {len(variants)//2 or len(variants)} rendition(s):")
|
||||
for v in variants[:12]:
|
||||
print(f"{tag} {v}")
|
||||
else:
|
||||
print(f"{tag} media playlist, {len(segs)} segment(s)")
|
||||
for s in segs[:2]:
|
||||
print(f"{tag} {s[:110]}")
|
||||
else:
|
||||
print(f"{tag} {body.decode('utf-8', 'replace')[:200].strip()}")
|
||||
return resp.status
|
||||
except Exception as exc:
|
||||
print(f"{tag}ERROR {type(exc).__name__}: {exc}")
|
||||
return -1
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--url", required=True)
|
||||
ap.add_argument("--cookie", default=None)
|
||||
ap.add_argument("--parallel", type=int, default=1,
|
||||
help="fire N simultaneous fetches to test concurrency")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.parallel <= 1:
|
||||
sys.exit(0 if fetch(args.url, args.cookie) == 200 else 1)
|
||||
|
||||
print(f"firing {args.parallel} simultaneous requests "
|
||||
f"(keep the browser playing while this runs)\n")
|
||||
results = {}
|
||||
start = threading.Barrier(args.parallel)
|
||||
|
||||
def worker(i):
|
||||
start.wait() # release them together
|
||||
results[i] = fetch(args.url, args.cookie, tag=f"[{i}] ")
|
||||
|
||||
threads = [threading.Thread(target=worker, args=(i,)) for i in range(args.parallel)]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
ok = sum(1 for v in results.values() if v == 200)
|
||||
print(f"\n{ok}/{args.parallel} succeeded concurrently"
|
||||
f"{' -- plus the browser itself' if ok else ''}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,191 @@
|
||||
// ==UserScript==
|
||||
// @name BRFC stream sniffer
|
||||
// @namespace local.brfc.archive
|
||||
// @version 0.4
|
||||
// @description Capture the manifest URL the player requests, and trace the API call that mints its token.
|
||||
// @match *://*.bristolrovers.co.uk/*
|
||||
// @match *://*.streamamg.com/*
|
||||
// @match *://*.streamamg.co.uk/*
|
||||
// @match *://*.kaltura.com/*
|
||||
// @match *://*.kaltura.co.uk/*
|
||||
// @match *://*.akamaized.net/*
|
||||
// @match *://*.akamaihd.net/*
|
||||
// @noframes false
|
||||
// @run-at document-start
|
||||
// @grant GM_xmlhttpRequest
|
||||
// @connect 127.0.0.1
|
||||
// ==/UserScript==
|
||||
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
const CATCHER = 'http://127.0.0.1:8787/stream';
|
||||
const TRACER = 'http://127.0.0.1:8787/trace';
|
||||
|
||||
// Discovery mode: record the request/response chain that produces a stream
|
||||
// URL, so the token mint can be replayed without a GUI browser. Set false
|
||||
// once the endpoint is known -- it writes a lot.
|
||||
const TRACE = true;
|
||||
|
||||
const IDLE_OPACITY = 0.063; // 16/255
|
||||
const MAX_BODY = 20000; // never log a whole media segment
|
||||
|
||||
console.log('[brfc-sniffer] v0.4 active in frame:', location.href);
|
||||
|
||||
const seen = new Map();
|
||||
|
||||
const isManifest = u => typeof u === 'string' && /\.m3u8(\?|$)/i.test(u);
|
||||
// Media and assets are noise. Everything else is a mint-call candidate.
|
||||
const isNoise = u => /\.(ts|mp4|m4s|aac|jpe?g|png|gif|webp|svg|woff2?|css)(\?|$)/i.test(u);
|
||||
|
||||
function post(endpoint, payload) {
|
||||
try {
|
||||
GM_xmlhttpRequest({
|
||||
method: 'POST',
|
||||
url: endpoint,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
data: JSON.stringify(payload),
|
||||
onerror: () => {}
|
||||
});
|
||||
} catch (e) { /* catcher not running; never break the page */ }
|
||||
}
|
||||
|
||||
function record(url, via) {
|
||||
let abs = url;
|
||||
if (!/^https?:/i.test(url)) {
|
||||
try { abs = new URL(url, location.href).href; } catch (e) { /* keep raw */ }
|
||||
}
|
||||
if (seen.has(abs)) return;
|
||||
seen.set(abs, { url: abs, via });
|
||||
console.log('[brfc-sniffer]', via, abs);
|
||||
render();
|
||||
post(CATCHER, {
|
||||
url: abs, via, page: location.href,
|
||||
userAgent: navigator.userAgent, ts: new Date().toISOString()
|
||||
});
|
||||
}
|
||||
|
||||
// The mint call is identified by what it RETURNS, not by its name. Any
|
||||
// response mentioning m3u8 is a candidate, whatever the endpoint is called.
|
||||
function trace(entry, bodyText) {
|
||||
if (!TRACE) return;
|
||||
const mints = !!bodyText && /m3u8/i.test(bodyText);
|
||||
if (!mints && !/api|token|entitle|session|auth|manifest|playback|licen/i.test(entry.url)) return;
|
||||
entry.mintsManifest = mints;
|
||||
entry.body = bodyText ? bodyText.slice(0, MAX_BODY) : null;
|
||||
entry.page = location.href;
|
||||
entry.ts = new Date().toISOString();
|
||||
if (mints) console.log('[brfc-sniffer] *** MINT CALL ***', entry.method, entry.url);
|
||||
post(TRACER, entry);
|
||||
}
|
||||
|
||||
// --- fetch ----------------------------------------------------------------
|
||||
const _fetch = window.fetch;
|
||||
if (_fetch) {
|
||||
window.fetch = function (input, init) {
|
||||
let url = null, method = 'GET';
|
||||
try {
|
||||
url = (typeof input === 'string') ? input : (input && input.url) || null;
|
||||
method = (init && init.method) || (input && input.method) || 'GET';
|
||||
if (isManifest(url)) record(url, 'fetch');
|
||||
} catch (e) { /* ignore */ }
|
||||
|
||||
const p = _fetch.apply(this, arguments);
|
||||
if (TRACE && url && !isNoise(url)) {
|
||||
p.then(resp => {
|
||||
try {
|
||||
const reqBody = (init && typeof init.body === 'string')
|
||||
? init.body.slice(0, MAX_BODY) : null;
|
||||
resp.clone().text().then(
|
||||
t => trace({ via: 'fetch', method, url, status: resp.status, reqBody }, t),
|
||||
() => {}
|
||||
);
|
||||
} catch (e) { /* ignore */ }
|
||||
}, () => {});
|
||||
}
|
||||
return p;
|
||||
};
|
||||
}
|
||||
|
||||
// --- XHR ------------------------------------------------------------------
|
||||
const _open = XMLHttpRequest.prototype.open;
|
||||
const _send = XMLHttpRequest.prototype.send;
|
||||
|
||||
XMLHttpRequest.prototype.open = function (method, url) {
|
||||
try {
|
||||
this.__brfc = { method: method, url: url };
|
||||
if (isManifest(url)) record(url, 'xhr');
|
||||
} catch (e) { /* ignore */ }
|
||||
return _open.apply(this, arguments);
|
||||
};
|
||||
|
||||
XMLHttpRequest.prototype.send = function (body) {
|
||||
try {
|
||||
const info = this.__brfc;
|
||||
if (TRACE && info && info.url && !isNoise(info.url)) {
|
||||
const reqBody = (typeof body === 'string') ? body.slice(0, MAX_BODY) : null;
|
||||
this.addEventListener('load', () => {
|
||||
let text = null;
|
||||
try {
|
||||
if (this.responseType === '' || this.responseType === 'text') {
|
||||
text = this.responseText;
|
||||
}
|
||||
} catch (e) { /* binary response */ }
|
||||
trace({ via: 'xhr', method: info.method, url: info.url,
|
||||
status: this.status, reqBody: reqBody }, text);
|
||||
});
|
||||
}
|
||||
} catch (e) { /* ignore */ }
|
||||
return _send.apply(this, arguments);
|
||||
};
|
||||
|
||||
// --- <video src> ----------------------------------------------------------
|
||||
const _setAttr = Element.prototype.setAttribute;
|
||||
Element.prototype.setAttribute = function (name, value) {
|
||||
try {
|
||||
if (/^src$/i.test(name) && isManifest(value)) record(value, 'attr');
|
||||
} catch (e) { /* ignore */ }
|
||||
return _setAttr.apply(this, arguments);
|
||||
};
|
||||
|
||||
// --- panel ----------------------------------------------------------------
|
||||
let panel;
|
||||
function render() {
|
||||
if (window.top !== window.self) return;
|
||||
if (!panel) {
|
||||
panel = document.createElement('div');
|
||||
panel.style.cssText = [
|
||||
'position:fixed', 'right:12px', 'bottom:12px', 'z-index:2147483647',
|
||||
'max-width:min(560px,90vw)', 'max-height:40vh', 'overflow:auto',
|
||||
'background:#101418', 'color:#e6edf3',
|
||||
'font:12px/1.45 ui-monospace,Menlo,Consolas,monospace',
|
||||
'border:1px solid #2b333c', 'border-radius:8px', 'padding:10px 12px',
|
||||
'box-shadow:0 8px 24px rgba(0,0,0,.45)',
|
||||
'opacity:' + IDLE_OPACITY,
|
||||
'transition:opacity .18s ease'
|
||||
].join(';');
|
||||
panel.addEventListener('mouseenter', () => { panel.style.opacity = '1'; });
|
||||
panel.addEventListener('mouseleave', () => { panel.style.opacity = String(IDLE_OPACITY); });
|
||||
(document.body || document.documentElement).appendChild(panel);
|
||||
}
|
||||
panel.innerHTML = '<b style="color:#5ac8fa">BRFC sniffer v0.4</b> '
|
||||
+ seen.size + ' manifest(s)'
|
||||
+ (TRACE ? ' <span style="color:#e3b341">[tracing]</span>' : '') + '<br>';
|
||||
for (const rec of seen.values()) {
|
||||
const row = document.createElement('div');
|
||||
row.style.cssText = 'margin-top:8px;word-break:break-all;'
|
||||
+ 'border-top:1px solid #2b333c;padding-top:6px';
|
||||
const btn = document.createElement('button');
|
||||
btn.textContent = 'copy';
|
||||
btn.style.cssText = 'margin-right:8px;cursor:pointer;background:#1f6feb;'
|
||||
+ 'color:#fff;border:0;border-radius:4px;padding:2px 8px';
|
||||
btn.onclick = () => navigator.clipboard.writeText(rec.url).then(() => {
|
||||
btn.textContent = 'copied';
|
||||
setTimeout(() => { btn.textContent = 'copy'; }, 1200);
|
||||
});
|
||||
row.appendChild(btn);
|
||||
row.appendChild(document.createTextNode('[' + rec.via + '] ' + rec.url));
|
||||
panel.appendChild(row);
|
||||
}
|
||||
}
|
||||
})();
|
||||
Reference in New Issue
Block a user