From 6ae4d5e8e3f21af83dc32d52c05735e850cc96cc Mon Sep 17 00:00:00 2001 From: Mikkeli Matlock Date: Mon, 14 Sep 2026 10:16:52 +0900 Subject: [PATCH] Document the in-browser web UI and its parity tests Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_015UXA39Sh9coAx4FhRkd8R2 --- CLAUDE.md | 96 +++++++++++++++------------------ README.md | 159 ++++++++++++++++++++++++------------------------------ 2 files changed, 115 insertions(+), 140 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index c5dd9d3..bdfc9d5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,6 +11,8 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th - **Modular GUI Architecture**: Complete PyQt5 interface with drag-and-drop and file dialog support - **Font Management**: CJK-capable, fixed UI font (M PLUS 1 Code @ 10pt) with system fallback - **Threading & Logging**: Robust background processing with detailed logging system +- **Web UI** (`web/`): the same metrics ported to in-browser JavaScript — a static + site; audio never leaves the tab ### Technical stack - **Audio Processing**: librosa, numpy @@ -109,49 +111,41 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th gone with the matplotlib render path. If wanted back, it belongs in the renderer, applied uniformly to every metric — not per-metric -#### `web_session.py` -- Server-side equivalent of `AnalysisResultsManager`, without Qt: owns the - per-track metric cache and the overlay set, and mirrors the desktop's - compute-off-the-request-thread behaviour -- **No audio is stored.** Tracks are keyed by `file_id` = SHA-256 of the audio, - and the session keeps only analysis results and metadata. There is no upload - directory and no cache of bytes -- Because the key is a content hash, re-sending the same master for a second - metric reuses the first decode — the client only re-uploads what the server has - never seen (checked via `metrics_cached`) +#### `web/` (browser version — no backend) +- A static site: decoding and every metric run in the visitor's tab; nothing is + uploaded or stored. Served with `Content-Security-Policy: connect-src 'none'` + (`deploy/nginx.conf`), so the page and its worker cannot make network requests +- `decode.js` — `AudioFile.__init__` counterpart. WAV/AIFF PCM & float are parsed + directly with libsndfile scaling (bit-identical to `librosa.load`; Chromium's + own WAV decoder scales positive int16 by 1/32767). Other formats go through + `decodeAudioData` on an `OfflineAudioContext` at the header-sniffed native rate + (no resampling). ID3 artist/title for MP3. Downmix = float32 channel mean +- `dsp.js` — ports of every `Metric.compute`: pure functions over the mono + Float32Array (run in the worker and under Node). Same numerics as the Python: + sequential float64 prefix sums sampled only at window edges (no N-length + arrays), pyloudnorm K-weighting via scipy `lfilter`'s DF2T recurrence (streamed), + `resample_poly(y, 4, 1)`'s 81-tap Kaiser(5) filter + monotonic-deque window max, + `librosa.stft` centred framing + `amplitude_to_db(ref=max, top_db=80)`. + Loudness (short-term, integrated, LRA) is one memoised pass shared by LUFS and PSR +- `metrics.js` — registry + `build_spec` ports producing the plot payload + (`{axes, hlines, annotations, segments, series, heatmap}`), plus `applyRelativeX` +- `analysis-worker.js` — Web Worker: holds each track's mono signal, computes one + metric per request, transfers typed arrays back +- `app.js` — `MainWindow` + `AnalysisResultsManager` + renderer: track list, + overlay set, per-metric result cache, job queue (foreground jobs jump ahead of + background prefetch), uPlot for curves/bands + canvas spectrogram (log axis + drawn from linear rows), draggable reference lines per metric, metric hlines +- `vendor/` — uPlot 1.6.32, vendored (the CSP only allows same-origin scripts) +- Needs an HTTP server (module scripts and workers don't load from `file://`), + but no secure context — plain `http://` on a LAN works -#### `webapp.py` -- FastAPI routes + static mount + the `ujm-web` entry point (binds loopback by - default; `--host`/`--port`/`--open` to override) -- Analysis endpoints take the audio as a **raw request body**, not a multipart - upload: spool to a temp file, `SESSION.observe(...)`, unlink in `finally`. The - bytes exist on disk only for the duration of that one request -- `POST /api/analyse` (analyse + plot data), `POST /api/file` (register without - plotting), `DELETE /api/file`, `POST /api/overlay`, `GET /api/plot`, - `GET /api/metadata`, `GET /api/metrics`, `GET /api/session` - -#### `metrics_json.py` -- The renderer seam: `PlotSpec`/metric data → JSON, so a non-pyqtgraph renderer - consumes the identical spec (curves, bands, hlines, heatmap, axis specs) that - the desktop widget draws. `hlines` carry the integrated LUFS reference line — - it is not an annotation - -#### `static/` (web front end) -- `audio-store.js` — OPFS store. Copies each picked/dropped `File` into Origin - Private File System (browser-sandboxed, not the host's filesystem), hashes it - with WebCrypto in chunks while streaming (never loads a whole master into - memory), and re-sends bytes to the server per metric -- `app.js` — the web renderer, an ES module importing the store. Draws the same - `PlotSpec` the desktop does: uPlot for curves/bands, a plain canvas for the - spectrogram heatmap (a canvas can map rows through an arbitrary function, so it - draws native linear rows and maps them, unlike pyqtgraph's affine-only - `ImageItem`). Mirrors the desktop's per-metric colour cycle, lin/log toggles, - absolute/relative time, and draggable reference lines -- `vendor/` — uPlot 1.6.32 (IIFE + CSS), vendored so nothing is fetched from a - CDN at runtime -- Boot rehydrates: the server still has the last session's analysis, so it - re-offers audio only for metric/track combinations the server reports as - uncached, and reports unavailable combinations rather than faking them +#### `tests/parity/` +- `fixtures.py` synthesises edge-case audio (short / silent / below the relative + gate, 96 kHz, every PCM layout, lossy formats via ffmpeg); `reference.py` runs + `metrics.py` and saves the exact mono signal it analysed; `parity.mjs` runs + `web/dsp.js` on that signal and compares every field (float64 paths ~1e-12, + float32 paths ~1e-6) +- Any change to `metrics.py` numerics must be mirrored in `web/dsp.js` and pass this #### `master_core.py` - Defines the `AudioFile` class: librosa loading, rolling RMS power. BPM detection @@ -269,7 +263,8 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th - matplotlib: Colormaps only (consumed by pyqtgraph) + librosa dependency - mutagen: Audio metadata extraction - PyQt5: GUI framework -- fastapi / uvicorn: web UI (optional extra; the desktop app does not need them) +- Web UI: no Python dependencies; uPlot is vendored in `web/vendor/`. The parity + tests need Node 18+ ### Architecture considerations - Three-stage split: `metrics.compute` (heavy, worker thread, backend-neutral @@ -294,19 +289,16 @@ uv run ujm # launch the GUI ``` ### Running the web UI -Branch `feature/web-ui`. Same backend, browser front end. +Branch `feature/web-js`. Static files in `web/`; everything runs in the browser. ```bash -uv sync --extra web -uv run ujm-web # http://127.0.0.1:8000 -uv run ujm-web --host 0.0.0.0 --port 8080 # reachable from another machine +docker compose up -d --build # nginx + CSP, http://localhost:8080 +python3 -m http.server -d web 8080 # quick local try (no CSP header) ``` -Uploaded audio is held in the browser's Origin Private File System and streamed -to the server per request; the server spools each body to a temp file and unlinks -it before responding. Nothing is written to the host beyond that transient spool. -Verify with `python3 scripts/webapi_smoke.py` against a running server (needs -`ffmpeg` for the fixtures). +Audio is decoded and analysed in the tab and never leaves it (CSP +`connect-src 'none'`). Check the port against `metrics.py` with the parity tests +(`tests/parity`, see README). Optional flags (handled by `logger_setup.parse_log_args`): ```bash diff --git a/README.md b/README.md index 3f41d34..2f9827f 100644 --- a/README.md +++ b/README.md @@ -39,12 +39,11 @@ uv sync --extra desktop uv run ujm ``` -`uv run ujm` boots the desktop GUI (see *Web UI* for the browser front end). +`uv run ujm` boots the desktop GUI (see *Web UI* for the browser version). `uv sync` on its own installs only the analysis dependencies (librosa, numpy, scipy, mutagen, pyloudnorm). The GUI stack — PyQt5, pyqtgraph — is the -`desktop` extra, kept separate because nothing on the web path imports it. Add -`--all-extras` to get both front ends. +`desktop` extra. ### Logging flags ```bash @@ -59,102 +58,81 @@ See [CJK_FONTS.md](CJK_FONTS.md) for details. ## Web UI -The same analysis backend with a browser front end, on branch `feature/web-ui`. -No PyQt5, no matplotlib: `metrics.py` was already split into a backend-neutral -`compute` and a view-aware `build_spec`, so the web UI reuses the analysis code -unchanged and reimplements only the drawing. +A browser version that runs entirely inside the tab, in `web/` (branch +`feature/web-js`). There is no Python backend: decoding and all seven metrics are +JavaScript ports of `master_core.py` / `metrics.py`, computed in a Web Worker and +drawn from the same `PlotSpec`-shaped payloads the desktop renderer uses. -```bash -uv sync --extra web -uv run ujm-web # http://127.0.0.1:8000 -uv run ujm-web --host 0.0.0.0 --port 8080 --open -``` - -A server install needs no Qt at all, which is also what keeps the container -image small. +**Audio never leaves the browser.** A dropped file is read into memory, decoded, +reduced to the mono signal the metrics use and analysed — nothing is uploaded, +cached or written to storage, and closing the tab discards it. This is enforced, +not just promised: the page is served with `Content-Security-Policy: connect-src +'none'`, so neither the page nor its worker is permitted to make any network +request. Everything the desktop app does works the same way: all seven metrics, the -overlay, lin/log toggles, absolute/relative time, and draggable reference lines -(kept per metric, as on the desktop). +overlay, lin/log toggles, absolute/relative time and draggable reference lines +(kept per metric), plus the metrics' own reference lines (−14 LUFS, integrated +loudness, 0 dBTP…). Analysis runs at the file's native sample rate. -**Uploaded audio is never written to the host.** The browser copies each file -into Origin Private File System — its own sandboxed storage — and streams the -bytes to the server for one request at a time. The server spools each body to a -temp file, decodes, computes, and unlinks it in a `finally` before the response -is sent. There is no upload directory, no cache of audio, and no session state -keyed on a file path: `file_id` is the SHA-256 of the audio, so re-sending the -same master for a second metric reuses the first decode. +Serve `web/` with any static file server, over `http://` or `https://` — no +secure context is needed. (ES modules and workers don't load from `file://`, so +opening `index.html` directly won't work.) -Being precise about the guarantee: the bytes do have to reach the server, because -`librosa` decodes from a path and the analysis is CPU-bound numpy/scipy. -*Running this does not accumulate the user's masters on the host* — nothing -persists between requests. It is not a claim that audio is never on the machine -mid-request. +```bash +docker compose up -d --build # nginx + the CSP, http://localhost:8080 +python3 -m http.server -d web 8080 # quick local try (no CSP header) +``` -The front end is served entirely from `static/` with uPlot vendored locally, so -nothing is fetched from a CDN, and the server binds loopback by default. +### How the port matches the Python + +- **Decoding** (`web/decode.js`): WAV and AIFF (PCM 8/16/24/32-bit, float 32/64, + WAVE_FORMAT_EXTENSIBLE) are parsed directly with libsndfile's sample scaling, + so the signal is bit-identical to `librosa.load`. Other formats (FLAC, MP3, Ogg + Vorbis/Opus, AAC) use the browser's decoder through an `OfflineAudioContext` + running at the sample rate read from the file header, so nothing is resampled. + MP3 artist/title come from ID3 tags, as on the desktop. +- **Metrics** (`web/dsp.js`): the same algorithms and numerics — pyloudnorm's + K-weighting through scipy's `lfilter` recurrence, BS.1770 gating and EBU LRA, + `resample_poly`'s 81-tap Kaiser filter for true peak, `librosa.stft` framing and + `amplitude_to_db` for the spectrogram. Running sums and window maxima are + single streaming passes, so no signal-length intermediate arrays are allocated. + +### Parity tests + +`tests/parity` checks the port against the Python on synthetic fixtures that +cover what the metric code branches on — shorter than each window, silence, +passages below the relative gate, 96 kHz, every PCM layout, lossy formats: + +```bash +uv run --no-project --with numpy --with soundfile \ + python tests/parity/fixtures.py /tmp/ujm-parity +uv run --no-project --with librosa --with pyloudnorm --with scipy --with mutagen \ + python tests/parity/reference.py /tmp/ujm-ref /tmp/ujm-parity/* +node tests/parity/parity.mjs /tmp/ujm-ref +``` + +`reference.py` saves the exact mono signal `metrics.py` analysed, and +`parity.mjs` runs `web/dsp.js` on that same signal and compares every output +field. The float64 paths (loudness, crest factor, PSR) agree to ~1e-12, the +float32 paths (true peak, spectrogram dB, RMS) to ~1e-6. ### Deploying with Docker -A `docker compose` template is provided: - -```bash -docker compose up --build # build and start -open http://localhost:8000 -``` - -The image is `web` extra only — no Qt, no display server — and runs as an -unprivileged user (uid 10001). It is built multi-stage so the compiler toolchain -that scipy/numba need never reaches the runtime layer; the runtime image installs -the built wheel, so the same image works from a checkout or a release tarball. - -What the template assumes, and why: - -- **There is no audio volume.** Not an oversight: the app keeps no audio, so - there is nothing to persist. `read_only: true` with a `/tmp` tmpfs means - request spools are memory-backed and never touch a disk at all. -- **Loopback publishing.** The app binds `0.0.0.0` inside the container (the CLI - default of `127.0.0.1` would be unreachable from the host), but compose - publishes `127.0.0.1:8000:8000`. This tool has no authentication, so it should - not appear on the LAN by accident. Change to `"8000:8000"` or put it behind a - reverse proxy to expose it deliberately. -- **CPU limits, not just memory.** Analysis is CPU-bound numpy/scipy, so the BLAS - thread pools are pinned (`OMP_NUM_THREADS` etc.) to stop overlapping requests - from oversubscribing. Raise them if you raise `cpus`. -- `cap_drop: [ALL]` and `no-new-privileges` — nothing in the image needs - capabilities. - -For a read-only root filesystem with no writable layer at all, the defaults -already give you that. - -### Verifying it - -With a server running: - -```bash -python3 scripts/make_fixtures.py # synthesise test audio -python3 scripts/webapi_smoke.py # 61 assertions -python3 scripts/webapi_smoke.py http://host:port /path/to/audio -``` - -`make_fixtures.py` generates the audio the suite needs (`sine440.wav`, -`pinknoise.wav`) with numpy alone, so no network and no ffmpeg are required — a -tagged MP3 is added only if ffmpeg happens to be present, and the tag assertions -skip cleanly when it is not. The suite drives the real HTTP surface with real -audio and checks the contract the browser depends on — payload shapes per -metric, physics cross-checked against an independent `librosa` computation, view -toggles that must not alter data, error paths, and that no spool file survives a -request. +The image is nginx (unprivileged, read-only root filesystem) serving `web/` with +`deploy/nginx.conf`, which sets the CSP and turns access logging off. There is +nothing to persist, upload or log on the server side, so there are no volumes. ### Web architecture | Module | Responsibility | | --- | --- | -| `web_session.py` | Headless analysis session: content-addressed track cache, metric cache, overlay | -| `metrics_json.py` | `PlotSpec` → JSON (the renderer seam; the desktop renderer is the other consumer) | -| `webapp.py` | FastAPI routes, raw-body analysis endpoints, static mount, `ujm-web` entry point | -| `static/audio-store.js` | OPFS audio store, content hashing, streamed upload | -| `static/app.js` | Web renderer: same `PlotSpec` drawn with uPlot + canvas | +| `web/decode.js` | File → mono Float32Array at native rate (direct WAV/AIFF, browser decoder otherwise); ID3 tags | +| `web/dsp.js` | Ports of every `Metric.compute`; pure functions (also run under Node for the parity tests) | +| `web/metrics.js` | Metric registry + `build_spec` ports producing plot payloads; relative-time transform | +| `web/analysis-worker.js` | Web Worker holding each track's signal and computing one metric per request | +| `web/app.js` | Track list, overlay, result cache, foreground/prefetch job queue, uPlot + canvas renderer | +| `deploy/nginx.conf` | Static serving, CSP (`connect-src 'none'`), no access log | ## Dependencies Core (always installed): `librosa`, `numpy`, `matplotlib`, `mutagen`, @@ -162,7 +140,9 @@ Core (always installed): `librosa`, `numpy`, `matplotlib`, `mutagen`, Optional extras: - `desktop` — `PyQt5`, `pyqtgraph` (the GUI) -- `web` — `fastapi`, `uvicorn`, `python-multipart` (the browser front end) + +The web UI has no dependencies beyond uPlot, vendored in `web/vendor/`. The parity +tests need Node 18+ plus the Python analysis dependencies. ## Architecture @@ -186,5 +166,8 @@ Subclass `Metric` in `metrics.py`, implement `compute(audio_file) -> data` (the heavy part, runs on the worker thread) and `build_spec(data, view) -> PlotSpec` (cheap, view-aware, reruns on every scale toggle without recomputation). Register the instance in the `METRICS` dict at the bottom of the file — it shows up in the -desktop dropdown and in the web UI's metric list automatically, because both -renderers consume the same `PlotSpec`. +desktop dropdown automatically. + +For the web UI, port `compute` to `web/dsp.js` (add it to `COMPUTE`) and +`build_spec` to `web/metrics.js` (add it to `METRICS`), then add its fields to the +tolerance table in `tests/parity/parity.mjs` so the parity run checks it.