Files
ncmr-songs-videos/scripts
irene 2c6e227c82 feat(bom): add --hash, and fix the BOM marker breaking YAML
`feat/bom-generator` carried one commit that never landed on master: file
content hashes in the BOM ("v2 with hashes"). The Python rewrite dropped that
capability, so port it forward rather than losing it when the branch goes.

- `--hash` records a short BLAKE2b digest per file. Opt-in, because it reads
  every asset in full and that is wasted work for a plain inventory.
- Hashes are streamed in chunks with `hashlib.blake2b`, not `md5` via haveged
  imports, and not whole-file reads — these assets run to hundreds of megabytes.
- YAML gains `hash:` after `size:`; Markdown gains a hash column only when
  hashes were requested, so the common table stays readable.

Fix a pre-existing bug found while testing: the BOM marker was written as
`; generated by ...`, but `;` is not a YAML comment, so every generated BOM
failed to parse with "mapping values are not allowed here". Now `#`. Verified by
re-parsing output from both the PyYAML path and the built-in fallback writer.

Sizes alone cannot tell you whether an asset changed; a same-length re-render
keeps the byte count identical. That is the gap the hash closes.
2026-09-10 18:37:18 +09:00
..

NCMR songs-videos — tools

Helper scripts for building NCMR song videos: converting lyrics to subtitles, burning them in, muxing audio, and working out what this machine can encode.

Everything here is Python standard library only. No virtualenv, no install step, no dependencies to break in six months. Run them straight from the repo:

python3 scripts/tools.py --help

Quick start

cd ~/dev/mikkeli/ncmr-songs-videos

# 1. What can this machine actually encode?
python3 scripts/tools.py hwprobe

# 2. Lyrics -> subtitles
python3 scripts/tools.py lrc2srt song.lrc -o song.srt
python3 scripts/tools.py srt2ass song.srt -o song.ass

# 3. Audio + still image -> a video (the "PV" case)
python3 scripts/tools.py ffmpeg still cover.png song.flac -o pv.mkv

# 4. Burn the subtitles in, copy the audio through untouched
python3 scripts/tools.py ffmpeg bake pv.mkv song.ass -o final.mp4 --format flac

# 5. Record what untracked assets went in
python3 scripts/tools.py bom --dir .

Every subcommand takes --help. Every subcommand accepts --dry-run (-n) to print the work it would do without doing it, which is the fastest way to see what ffmpeg command you are about to run.


The tools

Tool Purpose
tools.py Dispatcher. python3 scripts/tools.py <tool> [args...].
hwprobe.py Report which encoders this machine can really use, and why not.
lrc2srt.py LRC lyrics → SRT subtitles.
srt2ass.py SRT → ASS, for burn-in via ffmpeg's ass filter.
ffmpeg_tools.py mux, bake, still, strip, probe — the ffmpeg wrappers.
bom_gen.py Write a BOM.yaml (or BOM.md) of a song's untracked assets.
_common.py Shared helpers. Not a command; imported by the rest.

Each script also runs standalone (python3 scripts/hwprobe.py) and can be imported as a module. The dispatcher is a convenience, not a requirement.


Hardware encoding

This machine is an AMD Strix Halo (Radeon 8060S). There is no NVIDIA card, so nvenc is unavailable here. What does work is VA-API via the Mesa radeonsi driver:

Codec Encoder Status
H.264 h264_vaapi works
HEVC hevc_vaapi works
AV1 av1_vaapi works
H.264/HEVC libx264/libx265 software fallback, always available

ffmpeg_tools.py picks a hardware encoder automatically when one is usable and falls back to libx264 when it is not, so the same command works on a machine without a usable GPU. Override it when you want to:

--video auto      # default: hardware if possible, else software
--video x264      # force software H.264
--video x265      # force software HEVC
--video copy      # no re-encode at all
--codec hevc      # which hardware codec family to try with --video auto
--device PATH     # VA-API render node (default /dev/dri/renderD128)

If hardware encoding stops working

The usual cause is permissions on the DRM render node. /dev/dri/renderD128 is owned by root:render, so the user needs to be in the render group:

getent group render                    # who is in it
python3 scripts/tools.py hwprobe       # says plainly whether the device opens

Group membership only applies to new logins. If id shows render but hwprobe still reports the device as unreadable, the running process is holding stale credentials — log out and back in, or start the process under the group explicitly:

sg render -c 'python3 scripts/hwprobe.py'

hwprobe never guesses. It opens the device, runs a two-frame test encode through each candidate encoder, and reports what actually worked.

python3 scripts/tools.py hwprobe --shell   # export-friendly KEY=value output

Subtitles

LRC is the timestamped-lyrics format lyric sites emit; SRT is what most players read; ASS is what you need to burn styled text into a video with ffmpeg.

python3 scripts/tools.py lrc2srt song.lrc -o song.srt --shift -0.5

lrc2srt handles the messy real-world cases: [mm:ss], [mm:ss.xx] and [mm:ss.xxx] timestamps, several timestamps on one line (repeated choruses — each becomes its own cue), [ar:]/[ti:] metadata tags, and blank lines inside a cue. The final cue is held for a few seconds rather than dropped; adjust with --tail. Use --shift to nudge everything earlier or later if the lyrics drift against the audio.

python3 scripts/tools.py srt2ass song.srt -o song.ass --play-res 1920x1080

srt2ass converts to ASS and gives ffmpeg something it can burn in. --play-res sets the script resolution used for font scaling; match it to your video so text sizes come out as designed. --dry-run prints the ASS to stdout for inspection.


ffmpeg wrappers

Long ffmpeg command lines are easy to get subtly wrong and impossible to remember. These wrap the five the project actually needs.

# Attach audio to a video-only file; video stream copied, never re-encoded.
python3 scripts/tools.py ffmpeg mux render.mp4 song.flac -o pv.mkv --format flac

# Burn subtitles into a video.
python3 scripts/tools.py ffmpeg bake pv.mkv song.ass -o final.mp4 --format flac

# One still image + audio -> a video.
python3 scripts/tools.py ffmpeg still cover.png song.flac -o pv.mkv --fps 60

# Pull the audio back out as MP3.
python3 scripts/tools.py ffmpeg strip final.mp4 -o final.mp3

# What is actually in this file?
python3 scripts/tools.py ffmpeg probe final.mp4

Notes on the choices baked in:

  • mux uses -c:v copy. The Remotion render is already encoded; re-encoding it to add audio costs quality and time for nothing. Containers are picked from the output extension.
  • --format presets are aac, mp3, flac, opus and copy. flac is the default because these are archival masters; use aac/opus for anything you are actually shipping. --bitrate applies to the lossy ones.
  • -shortest is set where a still image is looped, so the video ends with the audio instead of running on for ever.
  • -movflags +faststart is set for MP4 output, so the file starts playing before it has fully downloaded.

For anything these do not cover, use --dry-run to print the generated command, copy it, and edit it — the wrapper is a convenience, not a cage. hwprobe.py reports what this host can actually encode, so you can tell at a glance whether a recipe will need a hardware-specific encoder.


Bill of materials

Binary assets are git-ignored, so a clone does not tell you what went into a song. bom_gen.py walks the asset directories and records it:

python3 scripts/tools.py bom --dir . --format yaml

It writes BOM.yaml listing every untracked file under audio/, images/, fonts/ and videos/ with its size, plus whatever description you give it. Use --format md for Markdown, --format json for machine-readable output, and -y to skip the interactive description prompt. If PyYAML is available it is used for clean output; otherwise a small built-in writer produces the same schema. Neither is required.

Add --hash to record a short content hash per file:

python3 scripts/tools.py bom --dir . --hash

That is how you answer "has this asset changed since the BOM was written?" — sizes stay identical when you re-render a take at the same length, but the hash will not. Hashing reads every asset in full, so it is off by default; when you only want the inventory, leave it out.


Conventions

  • master is the shared template. It is intended to freeze once the workspace is finalised; until then, tooling work can land on it directly. Per-song work goes on <song>/master, and features on <song>/<feature>. See README.md.
  • Binary assets stay out of git; the BOM is how you know what was used.
  • Keep new tools in scripts/, stdlib only, with a --dry-run and --help.