novoyuuparosk-auto-wiki/pipelines/songs/SCHEMA.md

SCHEMA — song source files

Source-file contract for the songs pipeline. Files in mikkeli/ncmr-songs must follow this contract to be picked up by the auto-publisher.

The schema lives entirely in the YAML frontmatter block at the top of each .md file. The body below the frontmatter is plain markdown, rendered to MediaWiki wikitext by Pandoc — no special body conventions are enforced beyond the standard markdown grammar, except that [[TAG]] placeholders declared via wiki.siblings are substituted with resolved wikilinks after rendering.

File layout

---
title: pulse under latent semantic envelopes
album: Shirakaba Express Service
wiki:
  publish: true
  siblings:
    - path: pulse_dev.md
      tag: PULSE_DEV
      label: development notes
release_date: 2026-06-18
lrc: pulse.lrc
---

# pulse under latent semantic envelopes

... body markdown, PULSE_DEV used inline ...

The renderer parses the frontmatter, validates required fields, then hands the body (with the first # Heading line removed) to Pandoc, then applies sibling substitutions on the wikitext output.

Fields

Required

title (string)

The MediaWiki page name that the bot writes or updates. MediaWiki capitalises the first letter automatically; lowercase input is fine.

Example: title: pulse under latent semantic envelopes → wiki page Pulse under latent semantic envelopes at URL /wiki/Pulse_under_latent_semantic_envelopes.

The body should contain a matching # Title first-line heading for human readability when viewing the source file directly (in an editor, on Gitea's blob view, etc.). The renderer strips this first-line heading during conversion — the wiki page already supplies its own title.

wiki.publish (boolean)

Publishing gate. true means the pipeline writes this file to the wiki. false means the file is ignored.

The wiki key is a mapping rather than a flat field so future toggles (wiki.protected, wiki.summary_template, etc.) can be added without restructuring.

Files under wip/ are also skipped, regardless of wiki.publish.

Optional

album (string)

The full album name. Drives [[Category:Songs]] and [[Category:Album:<album name>]] injection at the bottom of the rendered wiki page.

Example: album: Shirakaba Express Service → category tags [[Category:Songs]] and [[Category:Album:Shirakaba Express Service]].

Pages without an album (companion documents, placeholders, songs without an assigned album) simply omit this field — neither category is injected unless categories provides one.

categories (list of strings)

Additional wiki categories to inject, beyond the album category. Each entry becomes [[Category:<value>]].

categories:
  - Song development notes
  - Work in progress

album and categories are additive — a page can have both.

release_date (ISO 8601 date)

The song's release date in YYYY-MM-DD format. Skipped if absent. Currently informational; may drive sort order on future auto-generated indexes.

lrc (string, relative path)

Pointer to a companion LRC file, relative to the .md file's location. The pipeline uploads the file to the wiki as <title>.lrc (if the content has changed) and appends a [[Media:<title>.lrc|Synced lyrics (.lrc)]] link to the page footer.

Example: in ses/pulse.md, lrc: pulse.lrc declares that ses/pulse.lrc is the companion synced-lyrics file.

If the LRC file is missing or not valid UTF-8, the upload is skipped with a warning — the page still publishes.

wiki.siblings (list of sibling declarations)

Bill of materials for inter-page wikilinks. Each entry declares a sibling .md file and a tag that can be used as a [[TAG]] placeholder anywhere in the body.

wiki:
  siblings:
    - path: pulse_dev.md   # relative to this file's directory
      tag: PULSE_DEV       # placeholder used in body as [[PULSE_DEV]]
      label: development notes  # optional display text

Fields per entry:

Field	Required for resolution	Purpose
path	yes	Relative path to the sibling .md file
tag	yes	Placeholder identifier; pipeline does a case-sensitive string replace of TAG in the rendered wikitext
label	no	Display text for the resolved link

An entry missing either path or tag is silently skipped — no substitution is attempted for that entry.

Resolution:

The pipeline resolves each declared sibling's path to its wiki page title by looking it up in the set of all publishable files processed in the same run. Substitution is a case-sensitive string replace of the bare tag. Substitution rules:

Condition	TAG becomes
path resolves to a published page, label present	[[wiki page title|label]]
path resolves to a published page, no label	[[wiki page title]]
path not in published set (file exists but not published), label present	label (plain text)
path not in published set, no label	TAG (plain text)

Declared siblings whose file doesn't exist on disk are a validation error. Siblings that exist but aren't published (no wiki.publish: true) degrade gracefully per the table above.

The siblings list is a bill of materials — declaring a sibling has no effect unless its [[TAG]] appears in the body. Unused entries are inert.

--files mode: sibling resolution is limited to the files in that batch. Siblings pointing to files outside the batch degrade to plain text. This is acceptable since the pipeline always runs --all in the workflow.

Body

Plain markdown. The renderer applies these transformations before and after Pandoc:

1. Pre-Pandoc: strip the first-line # Heading if present.
2. Post-Pandoc: expand columns fenced blocks into side-by-side wikitext (see below).
3. Post-Pandoc: substitute TAG placeholders declared in wiki.siblings.

Do not embed raw wikitext-specific syntax ({{Template}}, raw [[Wikilink]] not declared via siblings, etc.) in the body unless you intend the literal output. Use markdown idioms; the renderer adds the metadata-derived bits (banner, LRC link, categories, sibling resolution) around Pandoc's output.

Columns (side-by-side) shorthand

For parallel content — e.g. an original and its translation — write a fenced code block tagged columns and separate the columns with a line containing only ===:

```columns
<b>原題</b>

一行目
二行目
===
<b>Title</b>

first line
second line
```

The renderer turns this into a flexbox row of <poem> columns (one per ===-delimited section). Notes:

• Line breaks and blank lines are preserved verbatim — that's the point of using a fenced block; Pandoc passes the body through untouched, and each column is wrapped in <poem> so MediaWiki keeps the line breaks.
• N columns: use N−1 === separators. Two is the common case (original / translation).
• Inline markup is allowed: HTML such as <b>…</b> or <br> written inside the block is HTML-unescaped on the way out, so it renders rather than showing as literal text.
• This is a shared transform (lib/wiki.py), so it works for any pipeline, not just songs. Column width/gap styling currently lives in that transform.

Renderer behaviour

For each .md with wiki.publish: true and not under wip/:

1. Parse and validate frontmatter.
2. Strip the leading # Heading from the body if present.
3. Pipe the body through pandoc -f markdown -t mediawiki, then expand any columns fenced blocks into side-by-side <poem> columns.
4. Substitute TAG placeholders with resolved [[Page|Display]] wikilinks (or fallback text).
5. Prepend the auto-generated banner: {{Auto-generated|source=<source URL>|commit=<sha>}}.
6. If lrc is declared: upload the LRC file if its SHA1 has changed; append a [[Media:…]] link.
7. Append category tags: [[Category:Songs]] and [[Category:Album:<album>]] if album is set; [[Category:<cat>]] for each entry in categories.
8. Read the current wiki page content via the MediaWiki API; if identical to the generated output, skip the write (idempotency — keeps the wiki history clean).
9. If the page exists and does not start with {{Auto-generated, skip with a warning (manual page protection).
10. Otherwise, write the page with an edit summary referencing the source commit.

Implemented scope

• Frontmatter parsing and validation
• Pandoc-based markdown → wikitext body rendering
• First-line h1 stripping
• Banner template injection
• LRC file upload and footer link
• Sibling placeholder resolution ([[TAG]] → [[wiki page title|label]])
• Album category injection ([[Category:Songs]] + [[Category:Album:<name>]]) when album is set
• Additional categories via categories list
• Idempotent writes (no-op skip when content matches)
• Manual-page protection (skip pages without auto-gen banner)
• Files without wiki.publish: true silently skipped

Not in scope (intentional)

• Multi-page aggregation: no album index pages generated; each file → its own wiki page. Album landing pages are wiki category pages, written freeform by the user.
• Language-aware rendering: the body conveys what languages exist; the renderer doesn't introspect.

Examples

A released song

---
title: pulse under latent semantic envelopes
album: Shirakaba Express Service
wiki:
  publish: true
  siblings:
    - path: pulse_dev.md
      tag: PULSE_DEV
      label: development notes
release_date: 2026-06-18
lrc: pulse.lrc
---

A companion/sibling document (no album)

---
title: pulse under latent semantic envelopes - development notes
wiki:
  publish: true
categories:
  - Song development notes
---

A placeholder (not published)

---
title: (placeholder)
album: kairo
wiki:
  publish: false
---

A WIP file (in wip/, skipped regardless)

---
title: 海淀
wiki:
  publish: false
---

Validation errors the renderer must produce

Files without a wiki.publish: true frontmatter field are silently skipped — no error. The pipeline fails fast and loud only on files that are opted in (wiki.publish: true) with invalid data:

• Missing or empty title on a file with wiki.publish: true
• wiki.publish present but not a boolean
• release_date (if present) does not parse as YYYY-MM-DD
• categories (if present) is not a list
• wiki.siblings entry that has both path and tag declared, but the path file does not exist on disk
• wiki.siblings entry that is not a mapping
• Two publishable source files declare the same title (page-name collision)

Validation runs before any wiki API calls. A failed validation aborts the run with a non-zero exit and leaves the wiki untouched.