uj-mastering-master/CLAUDE.md

# uj-mastering-master

A custom mastering toolkit that provides metrics to evaluate audio masterings through visual analysis.

## Current implementation

### Core features
- **Audio Analysis**: Uses librosa to analyze audio files (MP3/WAV/FLAC support) at native sample rate (no resampling)
- **Pluggable Metrics**: Switchable visualizations (RMS Power, Waveform, LUFS, Crest Factor, PSR, True Peak, Spectrogram; DR next) via a `Metric` ABC
- **Metadata Extraction**: Reads ID3 tags from MP3 files for better file identification
- **Modular GUI Architecture**: Complete PyQt5 interface with drag-and-drop and file dialog support
- **Font Management**: CJK-capable, fixed UI font (M PLUS 1 Code @ 10pt) with system fallback
- **Threading & Logging**: Robust background processing with detailed logging system

### Technical stack
- **Audio Processing**: librosa, numpy
- **Visualization**: pyqtgraph — persistent, interactive (mouse zoom/pan, lin/log
  toggle, multi-dataset overlay). matplotlib remains only for its colormaps
  (consumed by pyqtgraph) and as a librosa dependency
- **GUI Framework**: PyQt5 with modular widget architecture
- **Metadata**: mutagen for audio tag reading
- **Font Support**: Custom font loading system with CJK fallback

### Key components

#### `main.py`
- Complete GUI application with modular architecture
- Drag-and-drop and file dialog support for audio files
- Integrated font control system
- Real-time analysis display and file management

#### `analysis_results_manager.py`
- Background threading for audio analysis (`AudioAnalysisWorker` = load + first
  metric; `MetricComputeWorker` = one metric on an already-loaded file)
- Caches both the loaded `AudioFile` and per-metric `compute()` output, so
  metric/font switches re-render from cache without reloading librosa
- **Prefetch** (`PrefetchWorker`): after a file loads, the remaining metrics are
  computed in the background (one at a time, cooperatively cancellable) so the
  first switch to any metric is instant too. Superseded when a new file loads
- Timing: workers measure compute time; `metricTiming` + phase/duration progress
  messages drive the status slip ("X computed in Ys", "Loaded in Ns — computing…")
- `shutdown()` stops all threads on window close (`MainWindow.closeEvent`)

#### `audio_visualization_widget.py`
- Persistent pyqtgraph plot — the PlotItem is reused across renders, never torn
  down, so mouse zoom/pan and scale toggles survive every redraw
- `show_specs([(label, PlotSpec), ...], view)` draws one or more datasets onto
  the shared axes, assigning a distinct colour per dataset for overlay/compare
- Spectrogram log-frequency is realised by resampling STFT rows onto a log grid
  (`ImageItem` is affine-only and won't follow a log axis) — see `_render_heatmap`

#### `plotspec.py`
- Backend-agnostic drawing descriptors: `Curve`, `Band`, `HLine`, `Heatmap`,
  `AxisSpec`, `PlotSpec`, plus the `ViewState` (recompute-free lin/log options)
- The seam that decouples metrics from the plotting library: metrics emit
  *intent*, the renderer owns colour/layout/library specifics

#### `font_manager.py`
- Auto-detection of custom fonts from `fonts/` directory; CJK fallbacks
- `apply_fixed_font(family, size)` locks the Qt app font (used at startup to pin
  the UI to **M PLUS 1 Code @ 10pt**, falling back to the system default if the
  family isn't found). There is no runtime font picker — the old
  `font_control_widget.py` was removed as wasted panel space
- pyqtgraph and the Qt widgets both read the app font, so this covers the plot
  too (M PLUS 1 Code has full Japanese coverage, so titles stay CJK-safe)

#### `plot_control_widget.py`
- Metric selector dropdown driven by the `metrics.METRICS` registry
- Log-frequency toggle and a time-axis mode selector — Absolute (seconds) vs
  Relative (% of each track's own length) — both view-state, recompute-free
- `Refresh Plot` button. Compare/overlay membership is the file-list checkboxes;
  reference lines have their own cluster

#### `ref_line_widget.py`
- `RefLineControlWidget`: side-panel list of custom reference lines with
  Add / Edit… / Remove / Clear; a pure view over the `RefLineProps` list the
  main window owns, emitting intents
- `RefLineDialog`: edits one line's value, colour, line style, and tag
- The plot draws each line with a triangle drag-handle; dragging writes the new
  value back into the shared `RefLineProps` and refreshes the list

#### `metrics.py`
- Pluggable `Metric` ABC: `compute(audio_file) -> data` (heavy, worker thread,
  backend-neutral numpy/scalars) and `build_spec(data, view) -> PlotSpec` (cheap,
  GUI thread, view-aware). Metrics no longer touch the plotting library
- Compute-time vs view-time split: scale (lin/log) is a `ViewState` argument to
  `build_spec`, so toggling it never recomputes
- Current registry:
  - `RMSPowerMetric` — 10 s rolling RMS with adaptive colour scale
  - `WaveformMetric` — min/max envelope, fixed ±1.1 y-range
  - `LUFSMetric` — true (ungated) EBU R128 short-term (3 s) via a single
    K-weighting pass (`_kweight`, cached) + a vectorised sliding mean-square.
    Integrated (`_integrated_lufs`) and LRA (`_loudness_range`) are reimplemented
    from the same cached K-weighted signal — validated **bit-equal** to
    pyloudnorm — so nothing re-filters the signal. ~3.8 s → ~0.6 s. pyloudnorm is
    now used only to source the BS.1770 filter coefficients
  - `CrestFactorMetric` — 20·log10(peak/RMS) per 1 s window; peaks via O(N) running max
  - `PSRMetric` — sample-peak minus short-term LUFS (3 s window); reuses
    `LUFSMetric`'s short-term series (memoised on the `AudioFile`), so PSR is
    near-free once LUFS is computed
  - `TruePeakMetric` — 4× oversampled dBTP; the whole signal is oversampled once
    (`scipy.signal.resample_poly`) then an O(N) running max over windows
  - `SpectrogramMetric` — log-frequency STFT heatmap; adaptive hop caps time
    bins at ~4000, `N_FFT=4096`. Log/linear frequency is a view toggle
- Drop in new ones (DR, spectral balance) by appending an instance to `METRICS`;
  return a `PlotSpec` from `build_spec` (curves overlay automatically; heatmaps
  show one dataset at a time)
- Note: the old matplotlib `_show_axis_extents` exact-endpoint tick labelling is
  gone with the matplotlib render path. If wanted back, it belongs in the
  renderer, applied uniformly to every metric — not per-metric

#### `web_session.py`
- Server-side equivalent of `AnalysisResultsManager`, without Qt: owns the
  per-track metric cache and the overlay set, and mirrors the desktop's
  compute-off-the-request-thread behaviour
- **No audio is stored.** Tracks are keyed by `file_id` = SHA-256 of the audio,
  and the session keeps only analysis results and metadata. There is no upload
  directory and no cache of bytes
- Because the key is a content hash, re-sending the same master for a second
  metric reuses the first decode — the client only re-uploads what the server has
  never seen (checked via `metrics_cached`)

#### `webapp.py`
- FastAPI routes + static mount + the `ujm-web` entry point (binds loopback by
  default; `--host`/`--port`/`--open` to override)
- Analysis endpoints take the audio as a **raw request body**, not a multipart
  upload: spool to a temp file, `SESSION.observe(...)`, unlink in `finally`. The
  bytes exist on disk only for the duration of that one request
- `POST /api/analyse` (analyse + plot data), `POST /api/file` (register without
  plotting), `DELETE /api/file`, `POST /api/overlay`, `GET /api/plot`,
  `GET /api/metadata`, `GET /api/metrics`, `GET /api/session`

#### `metrics_json.py`
- The renderer seam: `PlotSpec`/metric data → JSON, so a non-pyqtgraph renderer
  consumes the identical spec (curves, bands, hlines, heatmap, axis specs) that
  the desktop widget draws. `hlines` carry the integrated LUFS reference line —
  it is not an annotation

#### `static/` (web front end)
- `audio-store.js` — OPFS store. Copies each picked/dropped `File` into Origin
  Private File System (browser-sandboxed, not the host's filesystem), hashes it
  with WebCrypto in chunks while streaming (never loads a whole master into
  memory), and re-sends bytes to the server per metric
- `app.js` — the web renderer, an ES module importing the store. Draws the same
  `PlotSpec` the desktop does: uPlot for curves/bands, a plain canvas for the
  spectrogram heatmap (a canvas can map rows through an arbitrary function, so it
  draws native linear rows and maps them, unlike pyqtgraph's affine-only
  `ImageItem`). Mirrors the desktop's per-metric colour cycle, lin/log toggles,
  absolute/relative time, and draggable reference lines
- `vendor/` — uPlot 1.6.32 (IIFE + CSS), vendored so nothing is fetched from a
  CDN at runtime
- Boot rehydrates: the server still has the last session's analysis, so it
  re-offers audio only for metric/track combinations the server reports as
  uncached, and reports unavailable combinations rather than faking them

#### `master_core.py`
- Defines the `AudioFile` class: librosa loading, rolling RMS power. BPM detection
  was **removed** — `librosa.beat.beat_track` cost ~3.7 s on every load for a
  number no better than tapping by hand
- Loads at **native sample rate** (`librosa.load(..., sr=None)`) so the full
  band is preserved — analysis runs ~2× heavier on 44.1/48 kHz files than the
  old 22050 Hz default, by design
- No batch / CLI mode — all analysis is driven from `main.py` via `AnalysisResultsManager`

### Current analysis features
- **Native-rate loading**: full-band analysis up to the file's own nyquist
- **RMS power analysis**: 10-second rolling window with 2-second hops
- **Adaptive colour mapping**: Automatically adjusts scale based on detected headroom
  - High dynamic range: 0-0.6 scale for loud masters  
  - Conservative mastering: 0-0.3 scale for quiet masters
- **Loudness metrics**: LUFS (ungated short-term + gated integrated + LRA), PSR, Crest Factor
- **Peak analysis**: True Peak (4× oversampled dBTP)
- **Spectral view**: log-frequency spectrogram heatmap over time
- **Metadata display**: Artist and title from audio tags
- **Real-time visualization**: Embedded matplotlib plots with font-aware rendering

### GUI features
- **File management**: Drag-and-drop and file dialog for audio selection
- **Compare/overlay**: each analysed file has a checkbox; the ticked set is
  overlaid on one graph for the current metric (curve metrics overlay; the
  spectrogram shows one track at a time). Highlighting a row drives the metadata
  panel, independent of the overlay set
- **Interactive plot**: mouse drag-zoom, scroll-wheel zoom, pan, right-click menu
  (pyqtgraph ViewBox); log/linear frequency toggle. Scroll zooms both axes;
  **Ctrl+scroll** zooms time only, **Shift+scroll** zooms the value axis only
  (`_AxisZoomViewBox`); scrolling over an axis also zooms just that axis
- **Time-axis mode**: a Relative-time toggle — off = seconds, on = 0-100% of each
  track's own length, so tracks of very different durations line up by position
- **Custom reference lines**: side-panel list (Add/Edit/Remove/Clear) of draggable
  horizontal markers with value/colour/style/tag; dragged via a triangle handle.
  Kept **per metric** (so switching metrics doesn't lose them) and expressed in
  the metric's own units — on the spectrogram they read and edit in **Hz** (the
  renderer converts Hz<->row index, since the heatmap y-axis is a row index)
- **Plot control**: Metric selector + log-frequency toggle + relative-time toggle
  + refresh-plot button
- **Analysis display**: Real-time visualization with metadata panels
- **Modular architecture**: Self-contained widgets for easy layout management

## Future development plans

### Short-term (urgent)
1. **Plot control widget cluster** *(metric selector + Refresh Plot done; still TODO)*
   - Plot style controller (colormap, line vs bar, etc.)
   - Foundation for mastering comparison features

### Short-term (not urgent)
1. **Enhanced metrics** *(plug new ones into `metrics.METRICS`)*
   - Dynamic range measurement (DR meter)
   - Long-term average spectrum (LTAS) / tonal-balance curve
   - Stereo metrics (correlation, mid/side) — needs `AudioFile` to retain stereo

2. **Interactive plot features** *(zoom/pan, axis-range select, lin/log done via
   pyqtgraph)*
   - GUI-controllable plotting styles (colormap, visualization type)
   - Export analysis results to CSV/JSON

3. **Advanced GUI controls**
   - Plot style customization interface
   - Real-time axis range selection (zooming in/out)
   - Interactive plot manipulation tools

4. **Better looking UI**
   - Graphical loading bar
   - Graphical logging text box

### Mid-to-long-term (very not urgent)
1. **Audio comparison system** *(multi-file overlay done via file-list checkboxes;
   each song has a stable palette colour keyed to its list row)*
   - Per-song colour picker: clickable swatch in the file list (overlay already
     accepts a caller-supplied colour per dataset via `show_specs`, so this is a
     UI + override-map addition, not a render change)
   - Reference vs. comparee designation (vs. flat overlay)
   - Side-by-side track comparison interface (incl. spectrogram, which can't overlay)
   - A/B testing for mastering versions

2. **Distribution & deployment**
   - Self-contained executable releases
   - Cross-platform packaging
   - Installer creation and distribution

### Future vision
1. **Advanced analysis tools**
   - Spectral centroid and bandwidth analysis
   - Stereo width measurements
   - Transient detection and analysis
   - Harmonic distortion detection

2. **Professional features**
   - EBU R128 compliance checking
   - Custom target curves
   - Professional reporting formats
   - Multi-format export capabilities

3. **VST plugin development**
   - Real-time analysis during mixing/mastering
   - Integration with DAWs
   - Live feedback during production

## Development notes

### Dependencies
- librosa: Audio analysis and feature extraction
- numpy: Numerical computations
- scipy: Signal processing (true-peak polyphase oversampling, K-weighting
  filters, spectrogram log-frequency resample, O(N) running-max via ndimage)
- pyloudnorm: source of the BS.1770 K-weighting filter coefficients (the LUFS
  short-term / integrated / LRA math is now computed directly, validated against it)
- pyqtgraph: Interactive plotting (zoom/pan, overlay, lin/log)
- matplotlib: Colormaps only (consumed by pyqtgraph) + librosa dependency
- mutagen: Audio metadata extraction
- PyQt5: GUI framework
- fastapi / uvicorn: web UI (optional extra; the desktop app does not need them)

### Architecture considerations
- Three-stage split: `metrics.compute` (heavy, worker thread, backend-neutral
  data) → `metrics.build_spec` (cheap, GUI thread, view-aware `PlotSpec`) →
  `AudioVisualizationWidget.show_specs` (pyqtgraph rendering, overlay, colours)
- File path handling needs improvement for cross-platform compatibility
- Error handling should be enhanced for production use
- Consider moving from PyQt5 to PyQt6 or PySide for better licensing

### Testing requirements
- Unit tests for audio analysis functions
- GUI component testing
- File format compatibility testing
- Performance testing with large audio files

## Usage

### Running the app
```bash
uv sync          # one-time, after cloning
uv run ujm       # launch the GUI
```

### Running the web UI
Branch `feature/web-ui`. Same backend, browser front end.

```bash
uv sync --extra web
uv run ujm-web                             # http://127.0.0.1:8000
uv run ujm-web --host 0.0.0.0 --port 8080  # reachable from another machine
```

Uploaded audio is held in the browser's Origin Private File System and streamed
to the server per request; the server spools each body to a temp file and unlinks
it before responding. Nothing is written to the host beyond that transient spool.
Verify with `python3 scripts/webapi_smoke.py` against a running server (needs
`ffmpeg` for the fixtures).

Optional flags (handled by `logger_setup.parse_log_args`):
```bash
uv run ujm --log-level DEBUG    # ERROR | WARN | INFO | DEBUG | TRACE
uv run ujm --log-file            # also write audio_analysis.log
```

The only entry point is `ujm` (defined in `pyproject.toml` as
`ujm = "main:main"`). The previous `files.txt` batch mode and the
`python master_core.py` workflow have been removed.

### Planned usage enhancements
1. Interactive plot manipulation and style customization
2. Audio file comparison features (reference vs. comparee)
3. Self-contained executable releases