feat(scripts): unified Python toolchain, stdlib only
Replaces the scattered TypeScript/shell helpers with one Python toolchain that runs as plain files (`python3 scripts/tools.py <tool> ...`) with no third-party dependencies. New tools - `_common.py` shared plumbing (Report, run, ffmpeg discovery, argparse ergonomics) - `hwprobe.py` what this machine can actually encode (software + hardware) - `srt2ass.py` SRT -> ASS, the missing step between lrc2srt and baking - `ffmpeg_tools.py` mux / bake / still / strip / probe, with hardware encode - `bom_gen.py` BOM generator, ported from scripts/bom-gen.ts - `tools.py` dispatcher so any tool is one command away - `README.md` usage for all of the above Rewritten - `lrc2srt.py` now handles multiple timestamps per line, timestamp-less cue lines and blank/comment lines, and survives malformed input. The legacy version emitted an empty cue plus a stray cue for every line. Hardware encoding This host is AMD (Strix Halo, VCN 4.0.5), so nvenc in the legacy notes is reference only. `hwprobe.py` probes and reports what genuinely works here: VA-API via /dev/dri/renderD128 gives h264_vaapi, hevc_vaapi and av1_vaapi. `ffmpeg_tools.py --video auto` picks the hardware encoder when the render node is usable and falls back to libx264/libx265 when it is not. Two bugs found and fixed while wiring this up: - the VA-API capability probe fed frames straight to the encoder without `format=nv12,hwupload`, so ffmpeg failed with "Function not implemented" and the toolchain silently fell back to software on a machine where hardware works. VA-API encoders only accept GPU-resident frames; the probe now uploads. - `bake --video copy` only failed when the copied stream happened to be VA-API. Burning subtitles always changes pixels, so it now refuses `copy` outright with a clear message. Also makes -q/-n usable on either side of the mode word, and ignores __pycache__ in .gitignore.
This commit is contained in:
@@ -0,0 +1,204 @@
|
||||
# NCMR songs-videos — tools
|
||||
|
||||
Helper scripts for building NCMR song videos: converting lyrics to subtitles,
|
||||
burning them in, muxing audio, and working out what this machine can encode.
|
||||
|
||||
Everything here is **Python standard library only**. No virtualenv, no install
|
||||
step, no dependencies to break in six months. Run them straight from the repo:
|
||||
|
||||
```
|
||||
python3 scripts/tools.py --help
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick start
|
||||
|
||||
```sh
|
||||
cd ~/dev/mikkeli/ncmr-songs-videos
|
||||
|
||||
# 1. What can this machine actually encode?
|
||||
python3 scripts/tools.py hwprobe
|
||||
|
||||
# 2. Lyrics -> subtitles
|
||||
python3 scripts/tools.py lrc2srt song.lrc -o song.srt
|
||||
python3 scripts/tools.py srt2ass song.srt -o song.ass
|
||||
|
||||
# 3. Audio + still image -> a video (the "PV" case)
|
||||
python3 scripts/tools.py ffmpeg still cover.png song.flac -o pv.mkv
|
||||
|
||||
# 4. Burn the subtitles in, copy the audio through untouched
|
||||
python3 scripts/tools.py ffmpeg bake pv.mkv song.ass -o final.mp4 --format flac
|
||||
|
||||
# 5. Record what untracked assets went in
|
||||
python3 scripts/tools.py bom --dir .
|
||||
```
|
||||
|
||||
Every subcommand takes `--help`. Every subcommand accepts `--dry-run` (`-n`) to
|
||||
print the work it would do without doing it, which is the fastest way to see
|
||||
what ffmpeg command you are about to run.
|
||||
|
||||
---
|
||||
|
||||
## The tools
|
||||
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `tools.py` | Dispatcher. `python3 scripts/tools.py <tool> [args...]`. |
|
||||
| `hwprobe.py` | Report which encoders this machine can really use, and why not. |
|
||||
| `lrc2srt.py` | LRC lyrics → SRT subtitles. |
|
||||
| `srt2ass.py` | SRT → ASS, for burn-in via ffmpeg's `ass` filter. |
|
||||
| `ffmpeg_tools.py` | `mux`, `bake`, `still`, `strip`, `probe` — the ffmpeg wrappers. |
|
||||
| `bom_gen.py` | Write a `BOM.yaml` (or `BOM.md`) of a song's untracked assets. |
|
||||
| `_common.py` | Shared helpers. Not a command; imported by the rest. |
|
||||
|
||||
Each script also runs standalone (`python3 scripts/hwprobe.py`) and can be
|
||||
imported as a module. The dispatcher is a convenience, not a requirement.
|
||||
|
||||
---
|
||||
|
||||
## Hardware encoding
|
||||
|
||||
This machine is an **AMD Strix Halo** (Radeon 8060S). There is no NVIDIA card,
|
||||
so the `nvenc` recipes in `ffmpegusage.txt` are reference only — they will not
|
||||
run here. What does work is **VA-API** via the Mesa `radeonsi` driver:
|
||||
|
||||
| Codec | Encoder | Status |
|
||||
| --- | --- | --- |
|
||||
| H.264 | `h264_vaapi` | works |
|
||||
| HEVC | `hevc_vaapi` | works |
|
||||
| AV1 | `av1_vaapi` | works |
|
||||
| H.264/HEVC | `libx264`/`libx265` | software fallback, always available |
|
||||
|
||||
`ffmpeg_tools.py` picks a hardware encoder automatically when one is usable and
|
||||
falls back to `libx264` when it is not, so the same command works on a machine
|
||||
without a usable GPU. Override it when you want to:
|
||||
|
||||
```sh
|
||||
--video auto # default: hardware if possible, else software
|
||||
--video x264 # force software H.264
|
||||
--video x265 # force software HEVC
|
||||
--video copy # no re-encode at all
|
||||
--codec hevc # which hardware codec family to try with --video auto
|
||||
--device PATH # VA-API render node (default /dev/dri/renderD128)
|
||||
```
|
||||
|
||||
### If hardware encoding stops working
|
||||
|
||||
The usual cause is permissions on the DRM render node. `/dev/dri/renderD128` is
|
||||
owned by `root:render`, so the user needs to be in the `render` group:
|
||||
|
||||
```sh
|
||||
getent group render # who is in it
|
||||
python3 scripts/tools.py hwprobe # says plainly whether the device opens
|
||||
```
|
||||
|
||||
Group membership only applies to *new* logins. If `id` shows `render` but
|
||||
`hwprobe` still reports the device as unreadable, the running process is holding
|
||||
stale credentials — log out and back in, or start the process under the group
|
||||
explicitly:
|
||||
|
||||
```sh
|
||||
sg render -c 'python3 scripts/hwprobe.py'
|
||||
```
|
||||
|
||||
`hwprobe` never guesses. It opens the device, runs a two-frame test encode
|
||||
through each candidate encoder, and reports what actually worked.
|
||||
|
||||
```sh
|
||||
python3 scripts/tools.py hwprobe --shell # export-friendly KEY=value output
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Subtitles
|
||||
|
||||
LRC is the timestamped-lyrics format lyric sites emit; SRT is what most players
|
||||
read; ASS is what you need to burn styled text into a video with ffmpeg.
|
||||
|
||||
```sh
|
||||
python3 scripts/tools.py lrc2srt song.lrc -o song.srt --shift -0.5
|
||||
```
|
||||
|
||||
`lrc2srt` handles the messy real-world cases: `[mm:ss]`, `[mm:ss.xx]` and
|
||||
`[mm:ss.xxx]` timestamps, several timestamps on one line (repeated choruses —
|
||||
each becomes its own cue), `[ar:]`/`[ti:]` metadata tags, and blank lines inside
|
||||
a cue. The final cue is held for a few seconds rather than dropped; adjust with
|
||||
`--tail`. Use `--shift` to nudge everything earlier or later if the lyrics drift
|
||||
against the audio.
|
||||
|
||||
```sh
|
||||
python3 scripts/tools.py srt2ass song.srt -o song.ass --play-res 1920x1080
|
||||
```
|
||||
|
||||
`srt2ass` converts to ASS and gives ffmpeg something it can burn in. `--play-res`
|
||||
sets the script resolution used for font scaling; match it to your video so text
|
||||
sizes come out as designed. `--dry-run` prints the ASS to stdout for inspection.
|
||||
|
||||
---
|
||||
|
||||
## ffmpeg wrappers
|
||||
|
||||
Long ffmpeg command lines are easy to get subtly wrong and impossible to
|
||||
remember. These wrap the five the project actually needs.
|
||||
|
||||
```sh
|
||||
# Attach audio to a video-only file; video stream copied, never re-encoded.
|
||||
python3 scripts/tools.py ffmpeg mux render.mp4 song.flac -o pv.mkv --format flac
|
||||
|
||||
# Burn subtitles into a video.
|
||||
python3 scripts/tools.py ffmpeg bake pv.mkv song.ass -o final.mp4 --format flac
|
||||
|
||||
# One still image + audio -> a video.
|
||||
python3 scripts/tools.py ffmpeg still cover.png song.flac -o pv.mkv --fps 60
|
||||
|
||||
# Pull the audio back out as MP3.
|
||||
python3 scripts/tools.py ffmpeg strip final.mp4 -o final.mp3
|
||||
|
||||
# What is actually in this file?
|
||||
python3 scripts/tools.py ffmpeg probe final.mp4
|
||||
```
|
||||
|
||||
Notes on the choices baked in:
|
||||
|
||||
- **`mux` uses `-c:v copy`.** The Remotion render is already encoded; re-encoding
|
||||
it to add audio costs quality and time for nothing. Containers are picked from
|
||||
the output extension.
|
||||
- **`--format` presets** are `aac`, `mp3`, `flac`, `opus` and `copy`. `flac` is
|
||||
the default because these are archival masters; use `aac`/`opus` for anything
|
||||
you are actually shipping. `--bitrate` applies to the lossy ones.
|
||||
- **`-shortest`** is set where a still image is looped, so the video ends with
|
||||
the audio instead of running on for ever.
|
||||
- **`-movflags +faststart`** is set for MP4 output, so the file starts playing
|
||||
before it has fully downloaded.
|
||||
|
||||
For anything these do not cover, use `--dry-run` to print the generated command,
|
||||
copy it, and edit it — the wrapper is a convenience, not a cage. The legacy
|
||||
recipes this grew out of are preserved in `ffmpegusage.txt`.
|
||||
|
||||
---
|
||||
|
||||
## Bill of materials
|
||||
|
||||
Binary assets are git-ignored, so a clone does not tell you what went into a
|
||||
song. `bom_gen.py` walks the asset directories and records it:
|
||||
|
||||
```sh
|
||||
python3 scripts/tools.py bom --dir . --format yaml
|
||||
```
|
||||
|
||||
It writes `BOM.yaml` listing every untracked file under `audio/`, `images/`,
|
||||
`fonts/` and `videos/` with its size, plus whatever description you give it. Use
|
||||
`--format md` for Markdown, `--format json` for machine-readable output, and
|
||||
`-y` to skip the interactive description prompt. If PyYAML is available it is
|
||||
used for clean output; otherwise a small built-in writer produces the same
|
||||
schema. Neither is required.
|
||||
|
||||
---
|
||||
|
||||
## Conventions
|
||||
|
||||
- `master` is the protected template. Do per-song work on `<song>/master`, and
|
||||
features on `<song>/<feature>`. See `README.md`.
|
||||
- Binary assets stay out of git; the BOM is how you know what was used.
|
||||
- Keep new tools in `scripts/`, stdlib only, with a `--dry-run` and `--help`.
|
||||
Reference in New Issue
Block a user