mikkeli
87874fa935
- `album` no longer required; category format changed to Category:Album:<name> - New optional `categories` list for additional arbitrary categories - `wiki.siblings` bill of materials: path+tag required for resolution, label optional - Post-Pandoc [[TAG]] substitution with four-tier fallback (link+label, link, plain label, plain tag) - Sibling map built from all published posts in the same run; unresolved siblings degrade gracefully - SCHEMA.md fully updated; README.md deferred items moved to implemented Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
96 lines
5.2 KiB
Markdown
96 lines
5.2 KiB
Markdown
# pipelines/songs
|
|
|
|
Publishes lyric pages from `mikkeli/ncmr-songs` to `https://wiki.novoyuuparosk.org`.
|
|
|
|
- Source-file contract (frontmatter and body conventions): [SCHEMA.md](SCHEMA.md)
|
|
- Cross-cutting setup (wiki URL, bot identity, runner, Gitea secrets/variables): [repo root README](../../README.md)
|
|
|
|
## v1 scope
|
|
|
|
Implemented:
|
|
|
|
- YAML frontmatter parsing per [SCHEMA.md](SCHEMA.md)
|
|
- Pandoc-based markdown → wikitext body rendering
|
|
- First-line h1 stripping
|
|
- Auto-generated banner (`{{Auto-generated|source=...|commit=<sha>}}`)
|
|
- Album category injection (`[[Category:Album:<name>]]`); additional categories via `categories` list
|
|
- LRC file upload (idempotent SHA1 check) + footer link
|
|
- Inter-page sibling placeholder resolution (`[[TAG]]` → `[[wiki page title|label]]`)
|
|
- MediaWiki bot API write with idempotency — no-op if wiki content matches generated output
|
|
- Files without `wiki.publish: true` silently skipped
|
|
- Pages with existing non-auto-generated content skipped (logged to stderr) — delete the page or add the banner to hand ownership to the bot
|
|
|
|
Not in scope (intentional):
|
|
|
|
- Multi-language metadata (the body conveys what languages exist; renderer doesn't introspect)
|
|
- Auto-generated album index pages — user writes album category-page descriptions freeform
|
|
|
|
## Source repo and branch convention
|
|
|
|
Source repo: `mikkeli/ncmr-songs`. The pipeline triggers on push to `master`.
|
|
|
|
**Branch convention in `ncmr-songs`**: per-song short-lived branches named `autowiki/<song-slug>` (e.g., `autowiki/pulse`). Create when staging edits, merge to master when ready to publish, delete after one cycle.
|
|
|
|
**Excluded paths** (won't trigger the pipeline):
|
|
|
|
- `wip/**`
|
|
- `README.md`, `.gitignore`, `.claude/**`
|
|
|
|
## Invocation
|
|
|
|
The pipeline runs as a self-contained Gitea Actions workflow in `mikkeli/ncmr-songs` (`.gitea/workflows/publish.yml`). It clones this repo at runtime to get the renderer.
|
|
|
|
Triggers:
|
|
- `push` to `master` (path-filtered as above)
|
|
- `workflow_dispatch` — manual trigger from the Gitea Actions UI; useful after renderer or template changes without needing a dummy commit
|
|
|
|
### Environment (secrets and variables)
|
|
|
|
| Name | Kind | Source | Purpose |
|
|
|---|---|---|---|
|
|
| `WIKI_API_URL` | variable | Gitea (cross-cutting) | MediaWiki action API endpoint |
|
|
| `WIKI_BASE_URL` | variable | Gitea (cross-cutting) | Wiki base URL |
|
|
| `WIKI_BOT_USER` | secret | Gitea (cross-cutting) | Bot login (`Dubrowski@giteaAutomaton`) |
|
|
| `WIKI_BOT_PASSWORD` | secret | Gitea (cross-cutting) | BotPasswords value |
|
|
| `URL_TO_GITEA` | variable | Gitea (cross-cutting) | Gitea instance base URL for cloning |
|
|
| `FAPAT` | secret | Gitea (cross-cutting) | Full-Access PAT for cloning private repos |
|
|
| `SOURCE_REF` | env (set in workflow) | `github.sha` | Pushed commit SHA, used in banner and edit summary |
|
|
| `GITEA_REPO_URL` | env (set in workflow) | composed from vars | Source-file URL base for the banner link |
|
|
|
|
## Dependencies
|
|
|
|
- Python 3.12+ (job container: `python:3.12-slim`)
|
|
- Pandoc (apt-installed in the job container)
|
|
- `python-frontmatter`, `mwclient`, `PyYAML` — see `requirements.txt`
|
|
|
|
## Modes
|
|
|
|
The pipeline always runs `--all`: every file with `wiki.publish: true` (not under `wip/`) is checked against the current wiki content. Writes only happen when content has changed (commit SHA in the banner drives idempotency). This is appropriate for the current repo size.
|
|
|
|
## One-off wiki setup
|
|
|
|
- [x] **`Template:Auto-generated`** — exists on the wiki. Wikitext source: `template_auto_generated.wikitext` in this directory.
|
|
- [x] **Bot user in `bot` group** — grants `skipcaptcha`; verified working.
|
|
|
|
## Decisions log (songs pipeline)
|
|
|
|
| Decision | Value | Date |
|
|
|---|---|---|
|
|
| Metadata source | YAML frontmatter inside each `.md` (see [SCHEMA.md](SCHEMA.md)) | 2026-06-09 |
|
|
| Body rendering | Pandoc-based, with a thin Python pre/post-processor | 2026-06-09 |
|
|
| Excluded paths | `wip/**`, plus repo-meta files | 2026-06-09 |
|
|
| Page template | Designed from scratch (no existing wiki pages to mirror) | 2026-06-09 |
|
|
| Album categories | Auto-injected `[[Category:Album:<name>]]`; category page is user-written and not overwritten | 2026-06-09 |
|
|
| Additional categories | Optional `categories` list in frontmatter; each entry → `[[Category:<value>]]` | 2026-06-10 |
|
|
| `album` optional | Companion/sibling pages don't need an album; omitting the field emits no album category | 2026-06-10 |
|
|
| Sibling resolution | Parent declares siblings as a bill of materials (`path` + `tag` + optional `label`); `[[TAG]]` substituted post-Pandoc | 2026-06-10 |
|
|
| Source-repo branch convention | `autowiki/<song-slug>` short-lived branches in `ncmr-songs` | 2026-06-09 |
|
|
| Idempotency | Commit SHA in banner (`commit=<sha>`), not timestamp — stable across identical re-runs | 2026-06-09 |
|
|
| Publish mode | Always `--all`; no diff detection — small repo, simpler than fragile git-diff gating | 2026-06-09 |
|
|
| Manual-page protection | Bot skips pages without the `{{Auto-generated` banner to avoid overwriting hand-written content | 2026-06-09 |
|
|
| `workflow_call` abandoned | Cross-repo `workflow_call` fails — run token scoped to triggering repo; cannot clone private callee | 2026-06-09 |
|
|
|
|
## Status
|
|
|
|
v1 live. Pipeline active and publishing to https://wiki.novoyuuparosk.org.
|