wan
e1810facd6
tests / pytest (push) Successful in 7s
Every auto-published page, in all three pipelines, carried a source link to
`localhost:3005`. The template was never at fault — it links `{{{source}}}`
verbatim; the bad URL was baked into each page's wikitext by the publisher.
Root cause was one variable serving two incompatible purposes. `URL_TO_GITEA`
has to be an address the runner container can clone from, and since the runner
uses host networking that is `http://localhost:3005`. All three workflows then
composed `GITEA_REPO_URL` — the base of the public banner link, written into
the wiki for anyone to read — out of that same loopback value.
Adds a second org variable, `PUBLIC_URL_TO_GITEA`, for the public base
(`https://git.novoyuuparosk.org`), and points the banner at it. Cloning keeps
using `URL_TO_GITEA`. The two are now documented as deliberately separate,
because collapsing them looks like tidying up.
`lib/wiki.public_source_base()` validates the banner base rather than trusting
it: unset, relative, loopback, mDNS (`*.local`) or private-range values raise
with the variable to set. A misconfiguration that would publish an unreachable
link now stops the run instead. Note the failure mode this creates on purpose:
until `PUBLIC_URL_TO_GITEA` exists, publishing aborts rather than proceeding
with a wrong URL.
Also builds the source path with `rel.as_posix()`, so the separator in a URL
does not depend on the OS the publisher runs on — it produced backslashes when
the renderer was exercised on Windows.
Prerequisites before this can publish:
1. Add the org variable `PUBLIC_URL_TO_GITEA` = https://git.novoyuuparosk.org
2. Flush the act_runner callee cache — `docker compose up -d --force-recreate
act_runner`. Per the cross-cutting decisions log, edits to `publish-*.yml`
keep executing the stale cached copy, and this change is in those files.
Pages already published still carry the old URL in their stored wikitext. They
are only corrected by re-publishing each pipeline in full.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
8.0 KiB
8.0 KiB
pipelines/songs
Publishes lyric pages from novoyuuparosk-wiki/ncmr-songs to https://wiki.novoyuuparosk.org.
- Source-file contract (frontmatter and body conventions): SCHEMA.md
- Cross-cutting setup (wiki URL, bot identity, runner, Gitea secrets/variables): repo root README
v1 scope
Implemented:
- YAML frontmatter parsing per SCHEMA.md
- Pandoc-based markdown → wikitext body rendering
- First-line h1 stripping
- Auto-generated banner (
{{Auto-generated|source=...|commit=<sha>}}) - Album category injection (
[[Category:Songs]]+[[Category:Album:<name>]]whenalbumis set); additional categories viacategorieslist - LRC file upload (idempotent SHA1 check) + footer link
- Inter-page link resolution: a relative Markdown link (
[label](notes.md)) becomes[[wiki page title|label]], resolved against a repo-wide index so it never depends on which files a push touched - Repo-wide page-title collision detection, and render-before-write, both ahead of any wiki contact
- MediaWiki bot API write with idempotency — no-op if wiki content matches generated output
- Files without
wiki.publish: truesilently skipped - Pages with existing non-auto-generated content skipped (logged to stderr) — delete the page or add the banner to hand ownership to the bot
Not in scope (intentional):
- Multi-language metadata (the body conveys what languages exist; renderer doesn't introspect)
- Auto-generated album index pages — user writes album category-page descriptions freeform
Source repo and branch convention
Source repo: novoyuuparosk-wiki/ncmr-songs. The pipeline triggers on push to master.
Branch convention in ncmr-songs: per-song short-lived branches named autowiki/<song-slug> (e.g., autowiki/pulse). Create when staging edits, merge to master when ready to publish, delete after one cycle.
Excluded paths (won't trigger the pipeline):
wip/**README.md,.gitignore,.claude/**
Invocation
The pipeline runs as a self-contained Gitea Actions workflow in novoyuuparosk-wiki/ncmr-songs (.gitea/workflows/publish.yml). It clones this repo at runtime to get the renderer.
Triggers:
pushtomaster(path-filtered as above)workflow_dispatch— manual trigger from the Gitea Actions UI; useful after renderer or template changes without needing a dummy commit
Environment (secrets and variables)
| Name | Kind | Source | Purpose |
|---|---|---|---|
WIKI_API_URL |
variable | Gitea (cross-cutting) | MediaWiki action API endpoint |
WIKI_BASE_URL |
variable | Gitea (cross-cutting) | Wiki base URL |
WIKI_BOT_USER |
secret | Gitea (cross-cutting) | Bot login (Mikkeli@giteaBot) |
WIKI_BOT_PASSWORD |
secret | Gitea (cross-cutting) | BotPasswords value |
URL_TO_GITEA |
variable | Gitea (cross-cutting) | Runner-internal Gitea base URL, for cloning only |
PUBLIC_URL_TO_GITEA |
variable | Gitea (cross-cutting) | Public Gitea base URL, for the banner source link only |
FAPAT |
secret | Gitea (cross-cutting) | Full-Access PAT for cloning private repos |
SOURCE_REF |
env (set in workflow) | github.sha |
Pushed commit SHA, used in banner and edit summary |
GITEA_REPO_URL |
env (set in workflow) | PUBLIC_URL_TO_GITEA + repo path |
Source-file URL base for the banner link. Validated by public_source_base(): a loopback, mDNS or private-range host aborts the run rather than publishing an unreachable link |
Dependencies
Job container: novoyuuparosk-wiki-runner:latest (pre-built, stored in the local Docker daemon on the runner host). Bakes in Python 3.12, Pandoc, and all pipeline Python packages — no install steps at job runtime.
See the Dockerfile at the repo root and requirements.txt in this directory. Rebuild the image after changes to either.
Modes
Reading is always repo-wide; writing is scoped to the push. Every run scans every file with wiki.publish: true (not under wip/), because link resolution and title-collision detection have to see the whole tree. What gets written is the push diff, plus one hop in each link direction: any page a changed page links to, and any page that links to a changed page.
--all— publish every publishable file. Used on the first push and onworkflow_dispatch.--files <paths…>— publish these files and their one-hop link neighbours. Used for an ordinary push.
Writes only happen when content has changed (commit SHA in the banner drives idempotency), so a page pulled in by the link closure is normally a no-op API read.
One-off wiki setup
Template:Auto-generated— exists on the wiki. Wikitext source:template_auto_generated.wikitextin this directory.- Bot user in
botgroup — grantsskipcaptcha; verified working.
Decisions log (songs pipeline)
| Decision | Value | Date |
|---|---|---|
| Metadata source | YAML frontmatter inside each .md (see SCHEMA.md) |
2026-06-09 |
| Body rendering | Pandoc-based, with a thin Python pre/post-processor | 2026-06-09 |
| Excluded paths | wip/**, plus repo-meta files |
2026-06-09 |
| Page template | Designed from scratch (no existing wiki pages to mirror) | 2026-06-09 |
| Album categories | Auto-injected [[Category:Songs]] + [[Category:Album:<name>]] when album is set; category pages are user-written and not overwritten |
2026-06-09 |
| Additional categories | Optional categories list in frontmatter; each entry → [[Category:<value>]] |
2026-06-10 |
album optional |
Companion pages don't need an album; omitting the field emits no album category | 2026-06-10 |
path + tag + optional label); [[TAG]] substituted post-Pandoc |
2026-06-10 | |
| Inter-page links | Plain relative Markdown link ([label](notes.md)); target's own title frontmatter is the page name. No frontmatter declaration, no macro namespace, and the link still works in Gitea's blob view. Replaces the wiki.siblings bill of materials, which substituted a bare tag by unanchored string replace — no word boundary, no collision check, and it degraded a link to plain text whenever the target fell outside the batch |
2026-09-09 |
| Unresolvable link | Hard error, never graceful degradation. A link to a missing or unpublished file fails the run; linking to a file does not publish it | 2026-09-09 |
| Render before write | All pages in a run are rendered before the first is written, so a broken page leaves the wiki untouched rather than half-updated | 2026-09-09 |
| Source-repo branch convention | autowiki/<song-slug> short-lived branches in ncmr-songs |
2026-06-09 |
| Idempotency | Commit SHA in banner (commit=<sha>), not timestamp — stable across identical re-runs |
2026-06-09 |
| Publish mode | Scan repo-wide, write the push diff plus one-hop link neighbours. Publishing is not instant, so the write set is bounded; resolution never is, which is what makes bounding the write set safe | 2026-09-09 |
| Title collisions | Detected across the whole repository, before any wiki contact — not only among the files one push happened to touch | 2026-09-09 |
| Tests | tests/ run by .gitea/workflows/test.yml in the publish runner image, so the pandoc-output contract the link resolver depends on is pinned by a test rather than by prose |
2026-09-09 |
| Manual-page protection | Bot skips pages without the {{Auto-generated banner to avoid overwriting hand-written content |
2026-06-09 |
workflow_call abandoned |
Cross-repo workflow_call fails — run token scoped to triggering repo; cannot clone private callee |
2026-06-09 |
| Side-by-side columns | Authored as a ```columns fenced block (separator ===); Pandoc emits <pre class="columns">, expanded post-Pandoc into a flex <div> of <poem> columns. No wiki template or extension — all Pi-side before the API write. See SCHEMA.md |
2026-06-14 |
Status
v1 live. Pipeline active and publishing to https://wiki.novoyuuparosk.org.