uj-mastering-master/CLAUDE.md

uj-mastering-master

A custom mastering toolkit that provides metrics to evaluate audio masterings through visual analysis.

Current implementation

Core features

• Audio Analysis: Uses librosa to analyze audio files (MP3/WAV/FLAC support) at native sample rate (no resampling)
• Pluggable Metrics: Switchable visualizations (RMS Power, Waveform, LUFS, Crest Factor, PSR, True Peak, Spectrogram; DR next) via a Metric ABC
• Metadata Extraction: Reads ID3 tags from MP3 files for better file identification
• Modular GUI Architecture: Complete PyQt5 interface with drag-and-drop and file dialog support
• Font Management: CJK-capable, fixed UI font (M PLUS 1 Code @ 10pt) with system fallback
• Threading & Logging: Robust background processing with detailed logging system

Technical stack

• Audio Processing: librosa, numpy
• Visualization: pyqtgraph — persistent, interactive (mouse zoom/pan, lin/log toggle, multi-dataset overlay). matplotlib remains only for its colormaps (consumed by pyqtgraph) and as a librosa dependency
• GUI Framework: PyQt5 with modular widget architecture
• Metadata: mutagen for audio tag reading
• Font Support: Custom font loading system with CJK fallback

Key components

main.py

• Complete GUI application with modular architecture
• Drag-and-drop and file dialog support for audio files
• Integrated font control system
• Real-time analysis display and file management

analysis_results_manager.py

• Background threading for audio analysis (AudioAnalysisWorker = load + first metric; MetricComputeWorker = one metric on an already-loaded file)
• Caches both the loaded AudioFile and per-metric compute() output, so metric/font switches re-render from cache without reloading librosa
• Prefetch (PrefetchWorker): after a file loads, the remaining metrics are computed in the background (one at a time, cooperatively cancellable) so the first switch to any metric is instant too. Superseded when a new file loads
• Timing: workers measure compute time; metricTiming + phase/duration progress messages drive the status slip ("X computed in Ys", "Loaded in Ns — computing…")
• shutdown() stops all threads on window close (MainWindow.closeEvent)

audio_visualization_widget.py

• Persistent pyqtgraph plot — the PlotItem is reused across renders, never torn down, so mouse zoom/pan and scale toggles survive every redraw
• show_specs([(label, PlotSpec), ...], view) draws one or more datasets onto the shared axes, assigning a distinct colour per dataset for overlay/compare
• Spectrogram log-frequency is realised by resampling STFT rows onto a log grid (ImageItem is affine-only and won't follow a log axis) — see _render_heatmap

plotspec.py

• Backend-agnostic drawing descriptors: Curve, Band, HLine, Heatmap, AxisSpec, PlotSpec, plus the ViewState (recompute-free lin/log options)
• The seam that decouples metrics from the plotting library: metrics emit intent, the renderer owns colour/layout/library specifics

font_manager.py

• Auto-detection of custom fonts from fonts/ directory; CJK fallbacks
• apply_fixed_font(family, size) locks the Qt app font (used at startup to pin the UI to M PLUS 1 Code @ 10pt, falling back to the system default if the family isn't found). There is no runtime font picker — the old font_control_widget.py was removed as wasted panel space
• pyqtgraph and the Qt widgets both read the app font, so this covers the plot too (M PLUS 1 Code has full Japanese coverage, so titles stay CJK-safe)

plot_control_widget.py

• Metric selector dropdown driven by the metrics.METRICS registry
• Log-frequency toggle and a time-axis mode selector — Absolute (seconds) vs Relative (% of each track's own length) — both view-state, recompute-free
• Refresh Plot button. Compare/overlay membership is the file-list checkboxes; reference lines have their own cluster

ref_line_widget.py

• RefLineControlWidget: side-panel list of custom reference lines with Add / Edit… / Remove / Clear; a pure view over the RefLineProps list the main window owns, emitting intents
• RefLineDialog: edits one line's value, colour, line style, and tag
• The plot draws each line with a triangle drag-handle; dragging writes the new value back into the shared RefLineProps and refreshes the list

metrics.py

• Pluggable Metric ABC: compute(audio_file) -> data (heavy, worker thread, backend-neutral numpy/scalars) and build_spec(data, view) -> PlotSpec (cheap, GUI thread, view-aware). Metrics no longer touch the plotting library
• Compute-time vs view-time split: scale (lin/log) is a ViewState argument to build_spec, so toggling it never recomputes
• Current registry:
  • RMSPowerMetric — 10 s rolling RMS with adaptive colour scale
  • WaveformMetric — min/max envelope, fixed ±1.1 y-range
  • LUFSMetric — true (ungated) EBU R128 short-term (3 s) via a single K-weighting pass (_kweight, cached) + a vectorised sliding mean-square. Integrated (_integrated_lufs) and LRA (_loudness_range) are reimplemented from the same cached K-weighted signal — validated bit-equal to pyloudnorm — so nothing re-filters the signal. ~3.8 s → ~0.6 s. pyloudnorm is now used only to source the BS.1770 filter coefficients
  • CrestFactorMetric — 20·log10(peak/RMS) per 1 s window; peaks via O(N) running max
  • PSRMetric — sample-peak minus short-term LUFS (3 s window); reuses LUFSMetric's short-term series (memoised on the AudioFile), so PSR is near-free once LUFS is computed
  • TruePeakMetric — 4× oversampled dBTP; the whole signal is oversampled once (scipy.signal.resample_poly) then an O(N) running max over windows
  • SpectrogramMetric — log-frequency STFT heatmap; adaptive hop caps time bins at ~4000, N_FFT=4096. Log/linear frequency is a view toggle
• Drop in new ones (DR, spectral balance) by appending an instance to METRICS; return a PlotSpec from build_spec (curves overlay automatically; heatmaps show one dataset at a time)
• Note: the old matplotlib _show_axis_extents exact-endpoint tick labelling is gone with the matplotlib render path. If wanted back, it belongs in the renderer, applied uniformly to every metric — not per-metric

web_session.py

• Server-side equivalent of AnalysisResultsManager, without Qt: owns the per-track metric cache and the overlay set, and mirrors the desktop's compute-off-the-request-thread behaviour
• No audio is stored. Tracks are keyed by file_id = SHA-256 of the audio, and the session keeps only analysis results and metadata. There is no upload directory and no cache of bytes
• Because the key is a content hash, re-sending the same master for a second metric reuses the first decode — the client only re-uploads what the server has never seen (checked via metrics_cached)

webapp.py

• FastAPI routes + static mount + the ujm-web entry point (binds loopback by default; --host/--port/--open to override)
• Analysis endpoints take the audio as a raw request body, not a multipart upload: spool to a temp file, SESSION.observe(...), unlink in finally. The bytes exist on disk only for the duration of that one request
• POST /api/analyse (analyse + plot data), POST /api/file (register without plotting), DELETE /api/file, POST /api/overlay, GET /api/plot, GET /api/metadata, GET /api/metrics, GET /api/session

metrics_json.py

• The renderer seam: PlotSpec/metric data → JSON, so a non-pyqtgraph renderer consumes the identical spec (curves, bands, hlines, heatmap, axis specs) that the desktop widget draws. hlines carry the integrated LUFS reference line — it is not an annotation

static/ (web front end)

• audio-store.js — OPFS store. Copies each picked/dropped File into Origin Private File System (browser-sandboxed, not the host's filesystem), hashes it with WebCrypto in chunks while streaming (never loads a whole master into memory), and re-sends bytes to the server per metric
• app.js — the web renderer, an ES module importing the store. Draws the same PlotSpec the desktop does: uPlot for curves/bands, a plain canvas for the spectrogram heatmap (a canvas can map rows through an arbitrary function, so it draws native linear rows and maps them, unlike pyqtgraph's affine-only ImageItem). Mirrors the desktop's per-metric colour cycle, lin/log toggles, absolute/relative time, and draggable reference lines
• vendor/ — uPlot 1.6.32 (IIFE + CSS), vendored so nothing is fetched from a CDN at runtime
• Boot rehydrates: the server still has the last session's analysis, so it re-offers audio only for metric/track combinations the server reports as uncached, and reports unavailable combinations rather than faking them

master_core.py

• Defines the AudioFile class: librosa loading, rolling RMS power. BPM detection was removed — librosa.beat.beat_track cost ~3.7 s on every load for a number no better than tapping by hand
• Loads at native sample rate (librosa.load(..., sr=None)) so the full band is preserved — analysis runs ~2× heavier on 44.1/48 kHz files than the old 22050 Hz default, by design
• No batch / CLI mode — all analysis is driven from main.py via AnalysisResultsManager

Current analysis features

• Native-rate loading: full-band analysis up to the file's own nyquist
• RMS power analysis: 10-second rolling window with 2-second hops
• Adaptive colour mapping: Automatically adjusts scale based on detected headroom
  • High dynamic range: 0-0.6 scale for loud masters
  • Conservative mastering: 0-0.3 scale for quiet masters
• Loudness metrics: LUFS (ungated short-term + gated integrated + LRA), PSR, Crest Factor
• Peak analysis: True Peak (4× oversampled dBTP)
• Spectral view: log-frequency spectrogram heatmap over time
• Metadata display: Artist and title from audio tags
• Real-time visualization: Embedded matplotlib plots with font-aware rendering

GUI features

• File management: Drag-and-drop and file dialog for audio selection
• Compare/overlay: each analysed file has a checkbox; the ticked set is overlaid on one graph for the current metric (curve metrics overlay; the spectrogram shows one track at a time). Highlighting a row drives the metadata panel, independent of the overlay set
• Interactive plot: mouse drag-zoom, scroll-wheel zoom, pan, right-click menu (pyqtgraph ViewBox); log/linear frequency toggle. Scroll zooms both axes; Ctrl+scroll zooms time only, Shift+scroll zooms the value axis only (_AxisZoomViewBox); scrolling over an axis also zooms just that axis
• Time-axis mode: a Relative-time toggle — off = seconds, on = 0-100% of each track's own length, so tracks of very different durations line up by position
• Custom reference lines: side-panel list (Add/Edit/Remove/Clear) of draggable horizontal markers with value/colour/style/tag; dragged via a triangle handle. Kept per metric (so switching metrics doesn't lose them) and expressed in the metric's own units — on the spectrogram they read and edit in Hz (the renderer converts Hz<->row index, since the heatmap y-axis is a row index)
• Plot control: Metric selector + log-frequency toggle + relative-time toggle
  • refresh-plot button
• Analysis display: Real-time visualization with metadata panels
• Modular architecture: Self-contained widgets for easy layout management

Future development plans

Short-term (urgent)

1. Plot control widget cluster (metric selector + Refresh Plot done; still TODO)
   • Plot style controller (colormap, line vs bar, etc.)
   • Foundation for mastering comparison features

Short-term (not urgent)

1. Enhanced metrics (plug new ones into metrics.METRICS)

   • Dynamic range measurement (DR meter)
   • Long-term average spectrum (LTAS) / tonal-balance curve
   • Stereo metrics (correlation, mid/side) — needs AudioFile to retain stereo

2. Interactive plot features (zoom/pan, axis-range select, lin/log done via pyqtgraph)

   • GUI-controllable plotting styles (colormap, visualization type)
   • Export analysis results to CSV/JSON

3. Advanced GUI controls

   • Plot style customization interface
   • Real-time axis range selection (zooming in/out)
   • Interactive plot manipulation tools

4. Better looking UI

   • Graphical loading bar
   • Graphical logging text box

Mid-to-long-term (very not urgent)

1. Audio comparison system (multi-file overlay done via file-list checkboxes; each song has a stable palette colour keyed to its list row)

   • Per-song colour picker: clickable swatch in the file list (overlay already accepts a caller-supplied colour per dataset via show_specs, so this is a UI + override-map addition, not a render change)
   • Reference vs. comparee designation (vs. flat overlay)
   • Side-by-side track comparison interface (incl. spectrogram, which can't overlay)
   • A/B testing for mastering versions

2. Distribution & deployment

   • Self-contained executable releases
   • Cross-platform packaging
   • Installer creation and distribution

Future vision

1. Advanced analysis tools

   • Spectral centroid and bandwidth analysis
   • Stereo width measurements
   • Transient detection and analysis
   • Harmonic distortion detection

2. Professional features

   • EBU R128 compliance checking
   • Custom target curves
   • Professional reporting formats
   • Multi-format export capabilities

3. VST plugin development

   • Real-time analysis during mixing/mastering
   • Integration with DAWs
   • Live feedback during production

Development notes

Dependencies

• librosa: Audio analysis and feature extraction
• numpy: Numerical computations
• scipy: Signal processing (true-peak polyphase oversampling, K-weighting filters, spectrogram log-frequency resample, O(N) running-max via ndimage)
• pyloudnorm: source of the BS.1770 K-weighting filter coefficients (the LUFS short-term / integrated / LRA math is now computed directly, validated against it)
• pyqtgraph: Interactive plotting (zoom/pan, overlay, lin/log)
• matplotlib: Colormaps only (consumed by pyqtgraph) + librosa dependency
• mutagen: Audio metadata extraction
• PyQt5: GUI framework
• fastapi / uvicorn: web UI (optional extra; the desktop app does not need them)

Architecture considerations

• Three-stage split: metrics.compute (heavy, worker thread, backend-neutral data) → metrics.build_spec (cheap, GUI thread, view-aware PlotSpec) → AudioVisualizationWidget.show_specs (pyqtgraph rendering, overlay, colours)
• File path handling needs improvement for cross-platform compatibility
• Error handling should be enhanced for production use
• Consider moving from PyQt5 to PyQt6 or PySide for better licensing

Testing requirements

• Unit tests for audio analysis functions
• GUI component testing
• File format compatibility testing
• Performance testing with large audio files

Usage

Running the app

uv sync          # one-time, after cloning
uv run ujm       # launch the GUI

Running the web UI

Branch feature/web-ui. Same backend, browser front end.

uv sync --extra web
uv run ujm-web                             # http://127.0.0.1:8000
uv run ujm-web --host 0.0.0.0 --port 8080  # reachable from another machine

Uploaded audio is held in the browser's Origin Private File System and streamed to the server per request; the server spools each body to a temp file and unlinks it before responding. Nothing is written to the host beyond that transient spool. Verify with python3 scripts/webapi_smoke.py against a running server (needs ffmpeg for the fixtures).

Optional flags (handled by logger_setup.parse_log_args):

uv run ujm --log-level DEBUG    # ERROR | WARN | INFO | DEBUG | TRACE
uv run ujm --log-file            # also write audio_analysis.log

The only entry point is ujm (defined in pyproject.toml as ujm = "main:main"). The previous files.txt batch mode and the python master_core.py workflow have been removed.

Planned usage enhancements

1. Interactive plot manipulation and style customization
2. Audio file comparison features (reference vs. comparee)
3. Self-contained executable releases