novoyuuparosk-wiki/novoyuuparosk-auto-wiki
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(songs): optional album, categories list, sibling placeholder resolution

Browse Source 
- `album` no longer required; category format changed to Category:Album:<name>
- New optional `categories` list for additional arbitrary categories
- `wiki.siblings` bill of materials: path+tag required for resolution, label optional
- Post-Pandoc [[TAG]] substitution with four-tier fallback (link+label, link, plain label, plain tag)
- Sibling map built from all published posts in the same run; unresolved siblings degrade gracefully
- SCHEMA.md fully updated; README.md deferred items moved to implemented

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Mikkeli Matlock
2026-06-10 00:45:42 +09:00
parent ef8ba3f01f
commit 87874fa935
3 changed files with 177 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pipelines/songs/SCHEMA.md
+109 -54
View File
@@ -2,7 +2,7 @@
Source-file contract for the songs pipeline. Files in `mikkeli/ncmr-songs` must follow this contract to be picked up by the auto-publisher.
The schema lives entirely in the YAML frontmatter block at the top of each `.md` file. The body below the frontmatter is plain markdown, rendered to MediaWiki wikitext by Pandoc — no special body conventions are enforced beyond the standard markdown grammar.
The schema lives entirely in the YAML frontmatter block at the top of each `.md` file. The body below the frontmatter is plain markdown, rendered to MediaWiki wikitext by Pandoc — no special body conventions are enforced beyond the standard markdown grammar, except that `[[TAG]]` placeholders declared via `wiki.siblings` are substituted with resolved wikilinks after rendering.
## File layout
@@ -12,15 +12,20 @@ title: pulse under latent semantic envelopes
album: Shirakaba Express Service
wiki:
publish: true
release_date: 2026-06-09
siblings:
- path: pulse_dev.md
tag: PULSE_DEV
label: development notes
release_date: 2026-06-18
lrc: pulse.lrc
---
# pulse under latent semantic envelopes
... body markdown ...
... body markdown, [[PULSE_DEV]] used inline ...
```
The renderer parses the frontmatter, validates required fields, then hands the body (with the first `# Heading` line removed) to Pandoc.
The renderer parses the frontmatter, validates required fields, then hands the body (with the first `# Heading` line removed) to Pandoc, then applies sibling substitutions on the wikitext output.
## Fields
@@ -34,56 +39,95 @@ Example: `title: pulse under latent semantic envelopes` → wiki page `Pulse und
The body should contain a matching `# Title` first-line heading for human readability when viewing the source file directly (in an editor, on Gitea's blob view, etc.). The renderer strips this first-line heading during conversion — the wiki page already supplies its own title.
#### `album` (string)
The full album name. Drives `[[Category:<album name>]]` injection at the bottom of the rendered wiki page. MediaWiki auto-creates the category page when at least one page references it.
The pipeline **does not** create or update a dedicated album info/index page. Album landing pages are the wiki's category pages, which you write freeform yourself.
Example: `album: Shirakaba Express Service` → category tag `[[Category:Shirakaba Express Service]]`.
For files where no album applies (placeholders, songs without an assigned album), use a sensible literal like `(unassigned)` and accept that they'll get a `[[Category:(unassigned)]]` tag. The field is required to keep the schema's contract simple.
#### `wiki.publish` (boolean)
Publishing gate. `true` means the pipeline writes this file to the wiki. `false` means the file is ignored.
The `wiki` key is a mapping rather than a flat field so future toggles (`wiki.protected`, `wiki.summary_template`, etc.) can be added without restructuring. None are defined yet.
The `wiki` key is a mapping rather than a flat field so future toggles (`wiki.protected`, `wiki.summary_template`, etc.) can be added without restructuring.
Files under `wip/` are also skipped, regardless of `wiki.publish`.
### Optional
#### `album` (string)
The full album name. Drives `[[Category:Album:<album name>]]` injection at the bottom of the rendered wiki page.
Example: `album: Shirakaba Express Service` → category tag `[[Category:Album:Shirakaba Express Service]]`.
Pages without an album (companion documents, placeholders, songs without an assigned album) simply omit this field — no category is injected unless `categories` provides one.
#### `categories` (list of strings)
Additional wiki categories to inject, beyond the album category. Each entry becomes `[[Category:<value>]]`.
```yaml
categories:
- Song development notes
- Work in progress
```
`album` and `categories` are additive — a page can have both.
#### `release_date` (ISO 8601 date)
The song's release date in `YYYY-MM-DD` format. Skipped if absent. Currently informational; may drive sort order on future auto-generated indexes.
#### `lrc` (string, relative path)
Pointer to a companion LRC file, relative to the `.md` file's location.
Pointer to a companion LRC file, relative to the `.md` file's location. The pipeline uploads the file to the wiki as `<title>.lrc` (if the content has changed) and appends a `[[Media:<title>.lrc|Synced lyrics (.lrc)]]` link to the page footer.
Example: in `ses/tunnels.md`, `lrc: tunnels.lrc` declares that `ses/tunnels.lrc` is the companion synced-lyrics file.
Example: in `ses/pulse.md`, `lrc: pulse.lrc` declares that `ses/pulse.lrc` is the companion synced-lyrics file.
**Deferred for v1**: the field is permitted by the schema but the renderer doesn't yet read or embed LRC contents.
If the LRC file is missing or not valid UTF-8, the upload is skipped with a warning — the page still publishes.
#### `siblings` (mapping of placeholder → relative path)
#### `wiki.siblings` (list of sibling declarations)
Placeholder-to-file mapping for inter-page wikilinks.
Bill of materials for inter-page wikilinks. Each entry declares a sibling `.md` file and a tag that can be used as a `[[TAG]]` placeholder anywhere in the body.
```yaml
siblings:
PULSE_DEV: pulse_dev.md
wiki:
siblings:
- path: pulse_dev.md # relative to this file's directory
tag: PULSE_DEV # placeholder used in body as [[PULSE_DEV]]
label: development notes # optional display text
```
In the body markdown, write `[[PULSE_DEV]]` for a wikilink that displays the target's title, or `[[PULSE_DEV|custom display]]` for an aliased display. Resolution uses the target file's `title:` frontmatter as the wiki page name.
**Fields per entry:**
**Deferred for v1**: the field is permitted by the schema but placeholders pass through untouched.
| Field | Required for resolution | Purpose |
|---|---|---|
| `path` | yes | Relative path to the sibling `.md` file |
| `tag` | yes | Placeholder identifier; pipeline hunts for `[[TAG]]` in the rendered wikitext |
| `label` | no | Display text for the resolved link |
An entry missing either `path` or `tag` is silently skipped — no substitution is attempted and any `[[TAG]]` in the body passes through as a literal wikilink to a page named `TAG`.
**Resolution:**
The pipeline resolves each declared sibling's `path` to its wiki page title by looking it up in the set of all publishable files processed in the same run. Substitution rules:
| Condition | `[[TAG]]` becomes |
|---|---|
| `path` resolves to a published page, `label` present | `[[wiki page title\|label]]` |
| `path` resolves to a published page, no `label` | `[[wiki page title]]` |
| `path` not in published set (file exists but not published), `label` present | `label` (plain text) |
| `path` not in published set, no `label` | `TAG` (plain text) |
Declared siblings whose file doesn't exist on disk are a **validation error**. Siblings that exist but aren't published (no `wiki.publish: true`) degrade gracefully per the table above.
The siblings list is a bill of materials — declaring a sibling has no effect unless its `[[TAG]]` appears in the body. Unused entries are inert.
**`--files` mode**: sibling resolution is limited to the files in that batch. Siblings pointing to files outside the batch degrade to plain text. This is acceptable since the pipeline always runs `--all` in the workflow.
## Body
Plain markdown. The renderer applies a single transformation before handing the body to Pandoc: the first-line `# Heading`, if present, is stripped.
Plain markdown. The renderer applies two transformations before and after Pandoc:
Do not embed wikitext-specific syntax (`{{Template}}`, raw `[[Wikilink]]` not declared via `siblings`, etc.) in the body. Use markdown idioms only; the renderer adds the metadata-derived bits (banner, categories, sibling resolution) around Pandoc's output.
1. **Pre-Pandoc**: strip the first-line `# Heading` if present.
2. **Post-Pandoc**: substitute `[[TAG]]` placeholders declared in `wiki.siblings`.
Do not embed raw wikitext-specific syntax (`{{Template}}`, raw `[[Wikilink]]` not declared via `siblings`, etc.) in the body unless you intend the literal output. Use markdown idioms; the renderer adds the metadata-derived bits (banner, LRC link, categories, sibling resolution) around Pandoc's output.
## Renderer behaviour
@@ -91,29 +135,33 @@ For each `.md` with `wiki.publish: true` and not under `wip/`:
1. Parse and validate frontmatter.
2. Strip the leading `# Heading` from the body if present.
3. Replace `[[PLACEHOLDER]]` markers with safe tokens. *(Deferred v1; no-op until siblings are implemented.)*
4. Pipe the body through `pandoc -f markdown -t mediawiki`.
5. Re-substitute sibling tokens with resolved `[[Page|Display]]` wikilinks. *(Deferred v1.)*
6. Prepend the auto-generated banner: `{{Auto-generated|source=<source URL>|commit=<sha>}}`.
7. Append the album category: `[[Category:<album>]]`.
3. Pipe the body through `pandoc -f markdown -t mediawiki`.
4. Substitute `[[TAG]]` placeholders with resolved `[[Page|Display]]` wikilinks (or fallback text).
5. Prepend the auto-generated banner: `{{Auto-generated|source=<source URL>|commit=<sha>}}`.
6. If `lrc` is declared: upload the LRC file if its SHA1 has changed; append a `[[Media:…]]` link.
7. Append category tags: `[[Category:Album:<album>]]` if `album` is set; `[[Category:<cat>]]` for each entry in `categories`.
8. Read the current wiki page content via the MediaWiki API; if identical to the generated output, skip the write (idempotency — keeps the wiki history clean).
9. Otherwise, write the page with an edit summary referencing the source commit.
9. If the page exists and does not start with `{{Auto-generated`, skip with a warning (manual page protection).
10. Otherwise, write the page with an edit summary referencing the source commit.
## v1 scope
## Implemented scope
**Implemented:**
- Frontmatter parsing and validation for `title`, `album`, `wiki.publish`, `release_date`
- Markdown body → wikitext via Pandoc
- Frontmatter parsing and validation
- Pandoc-based markdown → wikitext body rendering
- First-line h1 stripping
- Banner template injection (requires `Template:Auto-generated` to exist on the wiki — see the pipeline `README.md` for the one-off setup)
- Album category injection
- Idempotent writes (no-op skip)
- Banner template injection
- LRC file upload and footer link
- Sibling placeholder resolution (`[[TAG]]` → `[[wiki page title|label]]`)
- Album category injection (`[[Category:Album:<name>]]`)
- Additional categories via `categories` list
- Idempotent writes (no-op skip when content matches)
- Manual-page protection (skip pages without auto-gen banner)
- Files without `wiki.publish: true` silently skipped
**Deferred:**
- `lrc` field consumption
- `siblings` placeholder resolution
- Multi-page aggregation (no album index pages generated; each file → its own wiki page)
- Language-aware rendering (the body conveys what languages exist; the renderer doesn't introspect)
## Not in scope (intentional)
- Multi-page aggregation: no album index pages generated; each file → its own wiki page. Album landing pages are wiki category pages, written freeform by the user.
- Language-aware rendering: the body conveys what languages exist; the renderer doesn't introspect.
## Examples
@@ -125,22 +173,28 @@ title: pulse under latent semantic envelopes
album: Shirakaba Express Service
wiki:
publish: true
release_date: 2026-06-09
siblings:
- path: pulse_dev.md
tag: PULSE_DEV
label: development notes
release_date: 2026-06-18
lrc: pulse.lrc
---
```
### A dev sister file (not published)
### A companion/sibling document (no album)
```yaml
---
title: pulse under latent semantic envelopes — personal dev notes
album: Shirakaba Express Service
title: pulse under latent semantic envelopes - development notes
wiki:
publish: false
publish: true
categories:
- Song development notes
---
```
### A placeholder file (no real song yet)
### A placeholder (not published)
```yaml
---
@@ -151,12 +205,11 @@ wiki:
---
```
### A WIP file (in `wip/`, schema still applies but the file is skipped regardless)
### A WIP file (in `wip/`, skipped regardless)
```yaml
---
title: 海淀
album: (unassigned)
wiki:
publish: false
---
@@ -166,10 +219,12 @@ wiki:
Files without a `wiki.publish: true` frontmatter field are silently skipped — no error. The pipeline fails fast and loud only on files that are opted in (`wiki.publish: true`) with invalid data:
- Missing required field (`title` or `album`) on a file with `wiki.publish: true`
- Missing or empty `title` on a file with `wiki.publish: true`
- `wiki.publish` present but not a boolean
- `release_date` (if present) does not parse as YYYY-MM-DD
- `siblings` (if present) references a file that doesn't exist
- `categories` (if present) is not a list
- `wiki.siblings` entry that has both `path` and `tag` declared, but the `path` file does not exist on disk
- `wiki.siblings` entry that is not a mapping
- Two publishable source files declare the same `title` (page-name collision)
Validation runs before any wiki API calls. A failed validation aborts the run with a non-zero exit and leaves the wiki untouched.