Every auto-published page, in all three pipelines, carried a source link to
`localhost:3005`. The template was never at fault — it links `{{{source}}}`
verbatim; the bad URL was baked into each page's wikitext by the publisher.
Root cause was one variable serving two incompatible purposes. `URL_TO_GITEA`
has to be an address the runner container can clone from, and since the runner
uses host networking that is `http://localhost:3005`. All three workflows then
composed `GITEA_REPO_URL` — the base of the public banner link, written into
the wiki for anyone to read — out of that same loopback value.
Adds a second org variable, `PUBLIC_URL_TO_GITEA`, for the public base
(`https://git.novoyuuparosk.org`), and points the banner at it. Cloning keeps
using `URL_TO_GITEA`. The two are now documented as deliberately separate,
because collapsing them looks like tidying up.
`lib/wiki.public_source_base()` validates the banner base rather than trusting
it: unset, relative, loopback, mDNS (`*.local`) or private-range values raise
with the variable to set. A misconfiguration that would publish an unreachable
link now stops the run instead. Note the failure mode this creates on purpose:
until `PUBLIC_URL_TO_GITEA` exists, publishing aborts rather than proceeding
with a wrong URL.
Also builds the source path with `rel.as_posix()`, so the separator in a URL
does not depend on the OS the publisher runs on — it produced backslashes when
the renderer was exercised on Windows.
Prerequisites before this can publish:
1. Add the org variable `PUBLIC_URL_TO_GITEA` = https://git.novoyuuparosk.org
2. Flush the act_runner callee cache — `docker compose up -d --force-recreate
act_runner`. Per the cross-cutting decisions log, edits to `publish-*.yml`
keep executing the stale cached copy, and this change is in those files.
Pages already published still carry the old URL in their stored wikitext. They
are only corrected by re-publishing each pipeline in full.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `wiki.siblings` bill of materials is gone. A related co-publication is
now nominated with an ordinary relative Markdown link — `[label](notes.md)` —
resolved at publish time against the target file's own `title` frontmatter.
No macro namespace, no per-file declaration, and the link still works in
Gitea's blob view and any editor preview.
The macro could not be made safe. Substitution was `str.replace(tag, …)` over
the whole rendered page: no word boundary, so a tag also occurring in lyrics or
a code block got rewritten; no collision check, so overlapping tags corrupted
each other in declaration order. SCHEMA.md documented the placeholder as
`[[TAG]]` while the code replaced the bare tag, so following the schema
produced `[[[[Title|label]]]]`.
Fixes a live corruption bug in the process. `sibling_map` was built only from
the files in the publish batch, and an unresolved sibling degraded to plain
text — so pushing `ses/pulse.md` alone republished it with its wikilink
silently stripped to the bare words "Dev gibberish". SCHEMA.md asserted the
workflow "always runs --all", which it never did on an ordinary push.
The repository is now always scanned in full, whatever a run publishes, so
link resolution and title-collision detection can never depend on which files
a push touched. Writes stay bounded to the push diff plus one hop in each link
direction: the page a changed page links to (so a link cannot point at an
unpublished page) and the pages linking to it (so an edited `title` does not
leave stale inbound links). Publishing is not instant; reading is cheap.
An unresolvable `.md` link is now a hard error, never graceful degradation, and
linking to a file still does not publish it — `wiki.publish: true` remains the
only gate. A leftover-link guard fails the run on any link shape the resolver
did not recognise, including a Markdown link written inside a verbatim
```columns fence, so a pandoc upgrade cannot silently publish literal text.
Also:
- render before write: every page in a run is rendered before the first is
published, so a broken page leaves the wiki untouched instead of
half-updated. `publish_one` is split into `render_page` (pure) and the write.
- pin pandoc's stdout to UTF-8. `text=True` decoded it with the ambient locale
encoding; on any non-UTF-8 locale the Japanese and Chinese lyrics died inside
subprocess's reader thread, `stdout` came back None, and the real error was
lost behind a TypeError. Latent on the Debian runner via PEP 538 only.
- title collisions are detected repo-wide, not just within a push, and before
any wiki contact.
- tests/, run by .gitea/workflows/test.yml in the publish runner image. The
pandoc output contract the resolver depends on is now pinned by a test
instead of by prose — which is how SCHEMA.md came to assert something false.
- drop an f-string with nothing to interpolate.
- quote the SOURCE_REF interpolation in the workflow instead of pasting an
input straight into shell.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This repo and the three source repos moved from the `mikkeli` account
into the `novoyuuparosk-wiki` org. Rewrites cover repo paths only —
the `mikkeli:` HTTP basic-auth usernames stay, since FAPAT/PKGRW_PAT
remain personally owned, as do the `3005/mikkeli/*` image paths
(Gitea cannot transfer packages).
Bot identity moves to `Mikkeli@giteaBot`; both accounts hold the bot
right, so edits stay flagged as bot edits.
Inside a columns fenced block (verbatim, so Pandoc doesn't process
Markdown), convert *italic* / **bold** / ***bold-italic*** to wikitext
emphasis. Asterisk style, single line; precedence bold-italic > bold >
italic. Raw inline HTML still works via the existing HTML-unescape path.
Docs + example updated to use ** ** rather than <b>.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Authors write a ```columns fenced block (columns separated by a line of
===); Pandoc passes the body through verbatim as <pre class="columns">,
and a new post-Pandoc transform in lib.wiki.expand_columns expands it
into a flex <div> of <poem> columns. Runs entirely Pi-side before the
MediaWiki API write — no wiki template or PHP extension required.
The transform lives in shared lib/wiki.py (called from markdown_to_wikitext),
so it is universal across all pipelines. Stdlib only; no new deps.
Docs: SCHEMA.md author contract + songs/root decision logs.
Also ignore __pycache__/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dateless posts silently lost their Blog:<year> category. build_wikitext
now falls back to datetime.now().year when no date is declared, so every
published post lands in a year category. Docs corrected accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Publishes mikkeli/tech-blogs to the wiki at bare titles (no prefix).
Every page gets Category:Tech blog; Category:Blog:<year> from the date
field; Category:<tag> per tag entry (no prefix).
Fields: title (required), wiki.publish (required), date (optional,
YYYY-MM-DD), tags (optional list).
Source repo tech-blogs not yet initialised — pipeline ships first.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Songs always carries a # Song title line so stripping makes sense there,
but SES body structure is freeform and may use h1 deliberately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New pipeline at pipelines/ses/ publishes mikkeli/ses-light-novel to
the SES: MediaWiki namespace. All pages get Category:SES and
Category:SES:<type> (type from frontmatter, fallback to parent dir name).
Shared functions (connect_wiki, markdown_to_wikitext, strip_first_h1,
AUTO_BANNER_PREFIX) extracted from songs/publish.py into lib/wiki.py;
songs refactored to import from there.
Also adds publish-ses.yml workflow stub and updates Dockerfile and
root README.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pages with a valid `album` field now get [[Category:Songs]] prepended
before [[Category:Album:<name>]]. Companion/sibling pages without an
album are unaffected. Updated SCHEMA.md and pipeline README accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Credentials moved to Gitea user-scope vault — removed from README.
Architecture diagram and songs pipeline deps section updated to reflect
prebuilt novoyuuparosk-wiki-runner:latest image (no runtime dep installs).
Noted workflow stubs in .gitea/workflows/ may be stale.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- `album` no longer required; category format changed to Category:Album:<name>
- New optional `categories` list for additional arbitrary categories
- `wiki.siblings` bill of materials: path+tag required for resolution, label optional
- Post-Pandoc [[TAG]] substitution with four-tier fallback (link+label, link, plain label, plain tag)
- Sibling map built from all published posts in the same run; unresolved siblings degrade gracefully
- SCHEMA.md fully updated; README.md deferred items moved to implemented
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For songs with an `lrc` field in frontmatter:
- Verifies the file exists and is valid UTF-8 (warns and skips otherwise)
- Uploads to wiki as `File:<title>.lrc` via MediaWiki file API
- Idempotent: skips upload if SHA1 matches existing wiki file
- Appends `[[Media:<title>.lrc|Synced lyrics (.lrc)]]` before the
category tag at the bottom of the wiki page
Songs without `lrc` in frontmatter are unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
render.py was a misleading name — the script validates, renders,
and writes to MediaWiki (and will soon upload LRC files too).
publish.py better describes the full scope of the entry point.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If a wiki page exists but was written manually (no Auto-generated
banner), the bot now skips it instead of attempting an overwrite
that triggers MediaWiki CAPTCHA. Clear stderr warning tells user
to delete the page or add the banner to hand ownership to the bot.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Files with no frontmatter or wiki.publish not set should be silently
ignored rather than failing the whole pipeline run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Timestamps made the idempotency check always false — every run would
rewrite every page. Commit SHA is stable: same source commit produces
identical wikitext, so unchanged pages are correctly skipped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
wan
mikkeli