novoyuuparosk-auto-wiki/pipelines/songs/SCHEMA.md

# SCHEMA — song source files

Source-file contract for the songs pipeline. Files in `mikkeli/ncmr-songs` must follow this contract to be picked up by the auto-publisher.

The schema lives entirely in the YAML frontmatter block at the top of each `.md` file. The body below the frontmatter is plain markdown, rendered to MediaWiki wikitext by Pandoc — no special body conventions are enforced beyond the standard markdown grammar, except that `[[TAG]]` placeholders declared via `wiki.siblings` are substituted with resolved wikilinks after rendering.

## File layout

```markdown
---
title: pulse under latent semantic envelopes
album: Shirakaba Express Service
wiki:
  publish: true
  siblings:
    - path: pulse_dev.md
      tag: PULSE_DEV
      label: development notes
release_date: 2026-06-18
lrc: pulse.lrc
---

# pulse under latent semantic envelopes

... body markdown, PULSE_DEV used inline ...
```

The renderer parses the frontmatter, validates required fields, then hands the body (with the first `# Heading` line removed) to Pandoc, then applies sibling substitutions on the wikitext output.

## Fields

### Required

#### `title` (string)

The MediaWiki page name that the bot writes or updates. MediaWiki capitalises the first letter automatically; lowercase input is fine.

Example: `title: pulse under latent semantic envelopes` → wiki page `Pulse under latent semantic envelopes` at URL `/wiki/Pulse_under_latent_semantic_envelopes`.

The body should contain a matching `# Title` first-line heading for human readability when viewing the source file directly (in an editor, on Gitea's blob view, etc.). The renderer strips this first-line heading during conversion — the wiki page already supplies its own title.

#### `wiki.publish` (boolean)

Publishing gate. `true` means the pipeline writes this file to the wiki. `false` means the file is ignored.

The `wiki` key is a mapping rather than a flat field so future toggles (`wiki.protected`, `wiki.summary_template`, etc.) can be added without restructuring.

Files under `wip/` are also skipped, regardless of `wiki.publish`.

### Optional

#### `album` (string)

The full album name. Drives `[[Category:Songs]]` and `[[Category:Album:<album name>]]` injection at the bottom of the rendered wiki page.

Example: `album: Shirakaba Express Service` → category tags `[[Category:Songs]]` and `[[Category:Album:Shirakaba Express Service]]`.

Pages without an album (companion documents, placeholders, songs without an assigned album) simply omit this field — neither category is injected unless `categories` provides one.

#### `categories` (list of strings)

Additional wiki categories to inject, beyond the album category. Each entry becomes `[[Category:<value>]]`.

```yaml
categories:
  - Song development notes
  - Work in progress
```

`album` and `categories` are additive — a page can have both.

#### `release_date` (ISO 8601 date)

The song's release date in `YYYY-MM-DD` format. Skipped if absent. Currently informational; may drive sort order on future auto-generated indexes.

#### `lrc` (string, relative path)

Pointer to a companion LRC file, relative to the `.md` file's location. The pipeline uploads the file to the wiki as `<title>.lrc` (if the content has changed) and appends a `[[Media:<title>.lrc|Synced lyrics (.lrc)]]` link to the page footer.

Example: in `ses/pulse.md`, `lrc: pulse.lrc` declares that `ses/pulse.lrc` is the companion synced-lyrics file.

If the LRC file is missing or not valid UTF-8, the upload is skipped with a warning — the page still publishes.

#### `wiki.siblings` (list of sibling declarations)

Bill of materials for inter-page wikilinks. Each entry declares a sibling `.md` file and a tag that can be used as a `[[TAG]]` placeholder anywhere in the body.

```yaml
wiki:
  siblings:
    - path: pulse_dev.md   # relative to this file's directory
      tag: PULSE_DEV       # placeholder used in body as [[PULSE_DEV]]
      label: development notes  # optional display text
```

**Fields per entry:**

| Field | Required for resolution | Purpose |
|---|---|---|
| `path` | yes | Relative path to the sibling `.md` file |
| `tag` | yes | Placeholder identifier; pipeline does a case-sensitive string replace of `TAG` in the rendered wikitext |
| `label` | no | Display text for the resolved link |

An entry missing either `path` or `tag` is silently skipped — no substitution is attempted for that entry.

**Resolution:**

The pipeline resolves each declared sibling's `path` to its wiki page title by looking it up in the set of all publishable files processed in the same run. Substitution is a case-sensitive string replace of the bare tag. Substitution rules:

| Condition | `TAG` becomes |
|---|---|
| `path` resolves to a published page, `label` present | `[[wiki page title\|label]]` |
| `path` resolves to a published page, no `label` | `[[wiki page title]]` |
| `path` not in published set (file exists but not published), `label` present | `label` (plain text) |
| `path` not in published set, no `label` | `TAG` (plain text) |

Declared siblings whose file doesn't exist on disk are a **validation error**. Siblings that exist but aren't published (no `wiki.publish: true`) degrade gracefully per the table above.

The siblings list is a bill of materials — declaring a sibling has no effect unless its `[[TAG]]` appears in the body. Unused entries are inert.

**`--files` mode**: sibling resolution is limited to the files in that batch. Siblings pointing to files outside the batch degrade to plain text. This is acceptable since the pipeline always runs `--all` in the workflow.

## Body

Plain markdown. The renderer applies these transformations before and after Pandoc:

1. **Pre-Pandoc**: strip the first-line `# Heading` if present.
2. **Post-Pandoc**: expand `columns` fenced blocks into side-by-side wikitext (see below).
3. **Post-Pandoc**: substitute `TAG` placeholders declared in `wiki.siblings`.

Do not embed raw wikitext-specific syntax (`{{Template}}`, raw `[[Wikilink]]` not declared via `siblings`, etc.) in the body unless you intend the literal output. Use markdown idioms; the renderer adds the metadata-derived bits (banner, LRC link, categories, sibling resolution) around Pandoc's output.

### Columns (side-by-side) shorthand

For parallel content — e.g. an original and its translation — write a fenced code block tagged `columns` and separate the columns with a line containing only `===`:

````markdown
```columns
**原題**

一行目
二行目
===
**Title**

first line
second line
```
````

The renderer turns this into a flexbox row of `<poem>` columns (one per `===`-delimited section). Notes:

- **Line breaks and blank lines are preserved verbatim** — that's the point of using a fenced block; Pandoc passes the body through untouched, and each column is wrapped in `<poem>` so MediaWiki keeps the line breaks.
- **N columns**: use N−1 `===` separators. Two is the common case (original / translation).
- **Emphasis in Markdown**: write `*italic*`, `**bold**`, or `***bold-italic***` (asterisk style, single line) — these are converted to wikitext. Raw inline HTML such as `<b>…</b>` or `<br>` also still works (it's HTML-unescaped on the way out), but you shouldn't need it.
- This is a shared transform (`lib/wiki.py`), so it works for any pipeline, not just songs. Column width/gap styling currently lives in that transform.

## Renderer behaviour

For each `.md` with `wiki.publish: true` and not under `wip/`:

1. Parse and validate frontmatter.
2. Strip the leading `# Heading` from the body if present.
3. Pipe the body through `pandoc -f markdown -t mediawiki`, then expand any `columns` fenced blocks into side-by-side `<poem>` columns.
4. Substitute `TAG` placeholders with resolved `[[Page|Display]]` wikilinks (or fallback text).
5. Prepend the auto-generated banner: `{{Auto-generated|source=<source URL>|commit=<sha>}}`.
6. If `lrc` is declared: upload the LRC file if its SHA1 has changed; append a `[[Media:…]]` link.
7. Append category tags: `[[Category:Songs]]` and `[[Category:Album:<album>]]` if `album` is set; `[[Category:<cat>]]` for each entry in `categories`.
8. Read the current wiki page content via the MediaWiki API; if identical to the generated output, skip the write (idempotency — keeps the wiki history clean).
9. If the page exists and does not start with `{{Auto-generated`, skip with a warning (manual page protection).
10. Otherwise, write the page with an edit summary referencing the source commit.

## Implemented scope

- Frontmatter parsing and validation
- Pandoc-based markdown → wikitext body rendering
- First-line h1 stripping
- Banner template injection
- LRC file upload and footer link
- Sibling placeholder resolution (`[[TAG]]` → `[[wiki page title|label]]`)
- Album category injection (`[[Category:Songs]]` + `[[Category:Album:<name>]]`) when `album` is set
- Additional categories via `categories` list
- Idempotent writes (no-op skip when content matches)
- Manual-page protection (skip pages without auto-gen banner)
- Files without `wiki.publish: true` silently skipped

## Not in scope (intentional)

- Multi-page aggregation: no album index pages generated; each file → its own wiki page. Album landing pages are wiki category pages, written freeform by the user.
- Language-aware rendering: the body conveys what languages exist; the renderer doesn't introspect.

## Examples

### A released song

```yaml
---
title: pulse under latent semantic envelopes
album: Shirakaba Express Service
wiki:
  publish: true
  siblings:
    - path: pulse_dev.md
      tag: PULSE_DEV
      label: development notes
release_date: 2026-06-18
lrc: pulse.lrc
---
```

### A companion/sibling document (no album)

```yaml
---
title: pulse under latent semantic envelopes - development notes
wiki:
  publish: true
categories:
  - Song development notes
---
```

### A placeholder (not published)

```yaml
---
title: (placeholder)
album: kairo
wiki:
  publish: false
---
```

### A WIP file (in `wip/`, skipped regardless)

```yaml
---
title: 海淀
wiki:
  publish: false
---
```

## Validation errors the renderer must produce

Files without a `wiki.publish: true` frontmatter field are silently skipped — no error. The pipeline fails fast and loud only on files that are opted in (`wiki.publish: true`) with invalid data:

- Missing or empty `title` on a file with `wiki.publish: true`
- `wiki.publish` present but not a boolean
- `release_date` (if present) does not parse as YYYY-MM-DD
- `categories` (if present) is not a list
- `wiki.siblings` entry that has both `path` and `tag` declared, but the `path` file does not exist on disk
- `wiki.siblings` entry that is not a mapping
- Two publishable source files declare the same `title` (page-name collision)

Validation runs before any wiki API calls. A failed validation aborts the run with a non-zero exit and leaves the wiki untouched.