songs: replace the wiki.siblings macro with plain Markdown links
The `wiki.siblings` bill of materials is gone. A related co-publication is now nominated with an ordinary relative Markdown link — `[label](notes.md)` — resolved at publish time against the target file's own `title` frontmatter. No macro namespace, no per-file declaration, and the link still works in Gitea's blob view and any editor preview. The macro could not be made safe. Substitution was `str.replace(tag, …)` over the whole rendered page: no word boundary, so a tag also occurring in lyrics or a code block got rewritten; no collision check, so overlapping tags corrupted each other in declaration order. SCHEMA.md documented the placeholder as `[[TAG]]` while the code replaced the bare tag, so following the schema produced `[[[[Title|label]]]]`. Fixes a live corruption bug in the process. `sibling_map` was built only from the files in the publish batch, and an unresolved sibling degraded to plain text — so pushing `ses/pulse.md` alone republished it with its wikilink silently stripped to the bare words "Dev gibberish". SCHEMA.md asserted the workflow "always runs --all", which it never did on an ordinary push. The repository is now always scanned in full, whatever a run publishes, so link resolution and title-collision detection can never depend on which files a push touched. Writes stay bounded to the push diff plus one hop in each link direction: the page a changed page links to (so a link cannot point at an unpublished page) and the pages linking to it (so an edited `title` does not leave stale inbound links). Publishing is not instant; reading is cheap. An unresolvable `.md` link is now a hard error, never graceful degradation, and linking to a file still does not publish it — `wiki.publish: true` remains the only gate. A leftover-link guard fails the run on any link shape the resolver did not recognise, including a Markdown link written inside a verbatim ```columns fence, so a pandoc upgrade cannot silently publish literal text. Also: - render before write: every page in a run is rendered before the first is published, so a broken page leaves the wiki untouched instead of half-updated. `publish_one` is split into `render_page` (pure) and the write. - pin pandoc's stdout to UTF-8. `text=True` decoded it with the ambient locale encoding; on any non-UTF-8 locale the Japanese and Chinese lyrics died inside subprocess's reader thread, `stdout` came back None, and the real error was lost behind a TypeError. Latent on the Debian runner via PEP 538 only. - title collisions are detected repo-wide, not just within a push, and before any wiki contact. - tests/, run by .gitea/workflows/test.yml in the publish runner image. The pandoc output contract the resolver depends on is now pinned by a test instead of by prose — which is how SCHEMA.md came to assert something false. - drop an f-string with nothing to interpolate. - quote the SOURCE_REF interpolation in the workflow instead of pasting an input straight into shell. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -6,14 +6,9 @@ on:
|
||||
required: true
|
||||
type: string
|
||||
SOURCE_BASE_REF:
|
||||
description: "Commit SHA before the push (all-zeros = first push)"
|
||||
description: "Commit SHA before the push (empty or all-zeros = first push)"
|
||||
required: true
|
||||
type: string
|
||||
FULL_PUBLISH:
|
||||
description: "Re-publish every file regardless of diff"
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
@@ -26,9 +21,10 @@ jobs:
|
||||
env:
|
||||
FAPAT: ${{ secrets.FAPAT }}
|
||||
URL_TO_GITEA: ${{ vars.URL_TO_GITEA }}
|
||||
SOURCE_REF: ${{ inputs.SOURCE_REF }}
|
||||
run: |
|
||||
git clone "http://mikkeli:${FAPAT}@${URL_TO_GITEA#http://}/novoyuuparosk-wiki/ncmr-songs" ncmr-songs
|
||||
git -C ncmr-songs checkout ${{ inputs.SOURCE_REF }}
|
||||
git -C ncmr-songs checkout "$SOURCE_REF"
|
||||
|
||||
- name: Checkout auto-wiki
|
||||
env:
|
||||
@@ -36,24 +32,28 @@ jobs:
|
||||
URL_TO_GITEA: ${{ vars.URL_TO_GITEA }}
|
||||
run: git clone "http://mikkeli:${FAPAT}@${URL_TO_GITEA#http://}/novoyuuparosk-wiki/novoyuuparosk-auto-wiki" auto-wiki
|
||||
|
||||
# The diff bounds what gets *written*, not what gets *read*. The publisher
|
||||
# always scans the whole repository to resolve links and page titles, then
|
||||
# writes only these files (plus, one hop out, any page they link to or that
|
||||
# links to them). Passing a subset here is safe for that reason and only
|
||||
# that reason — resolution no longer depends on the batch.
|
||||
- name: Detect changed files
|
||||
id: diff
|
||||
env:
|
||||
BASE: ${{ inputs.SOURCE_BASE_REF }}
|
||||
HEAD: ${{ inputs.SOURCE_REF }}
|
||||
run: |
|
||||
BASE="${{ inputs.SOURCE_BASE_REF }}"
|
||||
HEAD="${{ inputs.SOURCE_REF }}"
|
||||
# All-zeros base means first push — publish everything
|
||||
# Empty or all-zeros base means first push or manual dispatch — publish everything
|
||||
if [ -z "$BASE" ] || echo "$BASE" | grep -qE '^0+$'; then
|
||||
echo "First push or manual dispatch — publishing everything."
|
||||
echo "mode=all" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
git -C ncmr-songs diff --name-only "$BASE" "$HEAD" -- '*.md' \
|
||||
| grep -v '^wip/' \
|
||||
| grep -v '^README\.md' \
|
||||
> /tmp/changed.txt 2>/dev/null || true
|
||||
echo "Changed files:"
|
||||
cat /tmp/changed.txt
|
||||
echo "mode=incremental" >> "$GITHUB_OUTPUT"
|
||||
echo "files=$(tr '\n' ' ' < /tmp/changed.txt)" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
CHANGED=$(git -C ncmr-songs diff --name-only "$BASE" "$HEAD" -- '*.md' | grep -v '^wip/' || true)
|
||||
echo "Changed files:"
|
||||
echo "$CHANGED"
|
||||
echo "mode=incremental" >> "$GITHUB_OUTPUT"
|
||||
echo "files=$(echo "$CHANGED" | tr '\n' ' ')" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Publish
|
||||
env:
|
||||
@@ -63,20 +63,14 @@ jobs:
|
||||
WIKI_BOT_PASSWORD: ${{ secrets.WIKI_BOT_PASSWORD }}
|
||||
SOURCE_REF: ${{ inputs.SOURCE_REF }}
|
||||
GITEA_REPO_URL: ${{ vars.URL_TO_GITEA }}/novoyuuparosk-wiki/ncmr-songs
|
||||
MODE: ${{ steps.diff.outputs.mode }}
|
||||
FILES: ${{ steps.diff.outputs.files }}
|
||||
run: |
|
||||
MODE="${{ steps.diff.outputs.mode }}"
|
||||
if [ "$MODE" = "all" ] || [ "${{ inputs.FULL_PUBLISH }}" = "true" ]; then
|
||||
python auto-wiki/pipelines/songs/publish.py \
|
||||
--source-dir ncmr-songs \
|
||||
--all
|
||||
else
|
||||
FILES="${{ steps.diff.outputs.files }}"
|
||||
if [ -z "$FILES" ]; then
|
||||
echo "No changed .md files detected. Nothing to publish."
|
||||
exit 0
|
||||
fi
|
||||
if [ "$MODE" = "all" ]; then
|
||||
python auto-wiki/pipelines/songs/publish.py --source-dir ncmr-songs --all
|
||||
elif [ -n "$(echo "$FILES" | tr -d '[:space:]')" ]; then
|
||||
# shellcheck disable=SC2086
|
||||
python auto-wiki/pipelines/songs/publish.py \
|
||||
--source-dir ncmr-songs \
|
||||
--files $FILES
|
||||
python auto-wiki/pipelines/songs/publish.py --source-dir ncmr-songs --files $FILES
|
||||
else
|
||||
echo "No changed .md files outside wip/. Nothing to publish."
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user