90 lines
4.3 KiB
Markdown
90 lines
4.3 KiB
Markdown
# pipelines/songs
|
|
|
|
Publishes lyric pages from `mikkeli/ncmr-songs` to `https://wiki.novoyuuparosk.org`.
|
|
|
|
- Source-file contract (frontmatter and body conventions): [SCHEMA.md](SCHEMA.md)
|
|
- Cross-cutting setup (wiki URL, bot identity, runner, Gitea secrets/variables): [repo root README](../../README.md)
|
|
|
|
## v1 scope
|
|
|
|
Implemented:
|
|
|
|
- YAML frontmatter parsing per [SCHEMA.md](SCHEMA.md)
|
|
- Pandoc-based markdown → wikitext body rendering
|
|
- First-line h1 stripping
|
|
- Auto-generated banner + album category injection
|
|
- MediaWiki bot API write with idempotency (no-op edits skipped)
|
|
|
|
Deferred (schema reserves the fields; renderer doesn't yet act on them):
|
|
|
|
- LRC parsing and embedding
|
|
- Inter-page sibling placeholder resolution
|
|
- Multi-language metadata (the body conveys what languages exist; renderer doesn't introspect)
|
|
- Auto-generated album index pages (intentionally NOT done — user writes album category-page descriptions freeform)
|
|
|
|
## Source repo and branch convention
|
|
|
|
Source repo: `mikkeli/ncmr-songs`. The pipeline triggers on push to `master`.
|
|
|
|
**Branch convention in `ncmr-songs`**: per-song short-lived branches named `autowiki/<song-slug>` (e.g., `autowiki/pulse`). Create when staging edits, merge to master when ready to publish, delete after one cycle.
|
|
|
|
**Excluded paths** (won't trigger the pipeline):
|
|
|
|
- `wip/**`
|
|
- `README.md`, `.gitignore`, `.claude/**`
|
|
|
|
## Invocation
|
|
|
|
The pipeline is invoked via a reusable Gitea Actions workflow (`.gitea/workflows/publish-songs.yml` in this repo). The `ncmr-songs` repo holds a thin stub workflow (`.gitea/workflows/publish.yml`) that calls into this one on push to `master`.
|
|
|
|
### Workflow inputs and secrets
|
|
|
|
| Name | Kind | Source | Purpose |
|
|
|---|---|---|---|
|
|
| `WIKI_API_URL` | variable | Gitea (cross-cutting) | MediaWiki action API endpoint |
|
|
| `WIKI_BASE_URL` | variable | Gitea (cross-cutting) | Wiki base URL (used in the banner link) |
|
|
| `WIKI_BOT_USER` | secret | Gitea (cross-cutting) | Bot login |
|
|
| `WIKI_BOT_PASSWORD` | secret | Gitea (cross-cutting) | BotPasswords value |
|
|
| `SOURCE_DIR` | workflow input | from the stub | Path to checked-out `ncmr-songs` working tree |
|
|
| `SOURCE_REF` | workflow input | from the stub | The pushed-to commit SHA |
|
|
| `SOURCE_BASE_REF` | workflow input | from the stub | The SHA prior to the push (for diff-based change detection) |
|
|
|
|
## Dependencies
|
|
|
|
- Python 3.12+
|
|
- Pandoc 2.x or newer (apt-installable in the runner container)
|
|
- `mwclient` or `requests` for the MediaWiki API
|
|
- `PyYAML` for frontmatter parsing
|
|
- `python-frontmatter` (convenience wrapper around PyYAML for markdown frontmatter)
|
|
|
|
`requirements.txt` will be added when the implementation lands.
|
|
|
|
## One-off wiki setup
|
|
|
|
Before the first run, two things must exist on the wiki:
|
|
|
|
1. **`Template:Auto-generated`** — the banner injected at the top of every auto-published page. Wikitext for this template is included in the implementation step (not yet written).
|
|
2. **The bot user has edit rights** for the namespace(s) the pipeline writes to. The default main namespace is fine; verify by attempting a manual edit via the bot account before relying on the pipeline.
|
|
|
|
## Modes
|
|
|
|
- **Incremental** (default, triggered by push to `master`): publishes only files changed between `SOURCE_BASE_REF` and `SOURCE_REF`.
|
|
- **Full** (`--all` flag, triggered by `workflow_dispatch`): re-publishes every publishable file in the source tree. Use after template or renderer changes.
|
|
|
|
## Decisions log (songs pipeline)
|
|
|
|
| Decision | Value | Date |
|
|
|---|---|---|
|
|
| Metadata source | YAML frontmatter inside each `.md` (see [SCHEMA.md](SCHEMA.md)) | 2026-06-09 |
|
|
| Body rendering | Pandoc-based, with a thin Python pre/post-processor | 2026-06-09 |
|
|
| Excluded paths | `wip/**`, plus repo-meta files | 2026-06-09 |
|
|
| Page template | Designed from scratch (no existing wiki pages to mirror) | 2026-06-09 |
|
|
| Album landing pages | Auto-injected `[[Category:<album>]]`; the wiki category page is user-written and not overwritten by the pipeline | 2026-06-09 |
|
|
| Source-repo branch convention | `autowiki/<song-slug>` short-lived branches in `ncmr-songs` | 2026-06-09 |
|
|
|
|
## Status
|
|
|
|
Schema and pipeline overview committed. No executable code yet.
|
|
|
|
**Next**: implement renderer (Python module), workflow YAMLs (this repo + `ncmr-songs` stub), `Template:Auto-generated` wikitext, and end-to-end dry-run on `ses/pulse.md`.
|