feat(songs): optional album, categories list, sibling placeholder resolution
- `album` no longer required; category format changed to Category:Album:<name> - New optional `categories` list for additional arbitrary categories - `wiki.siblings` bill of materials: path+tag required for resolution, label optional - Post-Pandoc [[TAG]] substitution with four-tier fallback (link+label, link, plain label, plain tag) - Sibling map built from all published posts in the same run; unresolved siblings degrade gracefully - SCHEMA.md fully updated; README.md deferred items moved to implemented Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -12,17 +12,18 @@ Implemented:
|
|||||||
- YAML frontmatter parsing per [SCHEMA.md](SCHEMA.md)
|
- YAML frontmatter parsing per [SCHEMA.md](SCHEMA.md)
|
||||||
- Pandoc-based markdown → wikitext body rendering
|
- Pandoc-based markdown → wikitext body rendering
|
||||||
- First-line h1 stripping
|
- First-line h1 stripping
|
||||||
- Auto-generated banner (`{{Auto-generated|source=...|commit=<sha>}}`) + album category injection
|
- Auto-generated banner (`{{Auto-generated|source=...|commit=<sha>}}`)
|
||||||
|
- Album category injection (`[[Category:Album:<name>]]`); additional categories via `categories` list
|
||||||
|
- LRC file upload (idempotent SHA1 check) + footer link
|
||||||
|
- Inter-page sibling placeholder resolution (`[[TAG]]` → `[[wiki page title|label]]`)
|
||||||
- MediaWiki bot API write with idempotency — no-op if wiki content matches generated output
|
- MediaWiki bot API write with idempotency — no-op if wiki content matches generated output
|
||||||
- Files without `wiki.publish: true` silently skipped
|
- Files without `wiki.publish: true` silently skipped
|
||||||
- Pages with existing non-auto-generated content skipped (logged to stderr) — delete the page or add the banner to hand ownership to the bot
|
- Pages with existing non-auto-generated content skipped (logged to stderr) — delete the page or add the banner to hand ownership to the bot
|
||||||
|
|
||||||
Deferred (schema reserves the fields; renderer doesn't yet act on them):
|
Not in scope (intentional):
|
||||||
|
|
||||||
- LRC parsing and embedding
|
|
||||||
- Inter-page sibling placeholder resolution
|
|
||||||
- Multi-language metadata (the body conveys what languages exist; renderer doesn't introspect)
|
- Multi-language metadata (the body conveys what languages exist; renderer doesn't introspect)
|
||||||
- Auto-generated album index pages (intentionally NOT done — user writes album category-page descriptions freeform)
|
- Auto-generated album index pages — user writes album category-page descriptions freeform
|
||||||
|
|
||||||
## Source repo and branch convention
|
## Source repo and branch convention
|
||||||
|
|
||||||
@@ -79,7 +80,10 @@ The pipeline always runs `--all`: every file with `wiki.publish: true` (not unde
|
|||||||
| Body rendering | Pandoc-based, with a thin Python pre/post-processor | 2026-06-09 |
|
| Body rendering | Pandoc-based, with a thin Python pre/post-processor | 2026-06-09 |
|
||||||
| Excluded paths | `wip/**`, plus repo-meta files | 2026-06-09 |
|
| Excluded paths | `wip/**`, plus repo-meta files | 2026-06-09 |
|
||||||
| Page template | Designed from scratch (no existing wiki pages to mirror) | 2026-06-09 |
|
| Page template | Designed from scratch (no existing wiki pages to mirror) | 2026-06-09 |
|
||||||
| Album landing pages | Auto-injected `[[Category:<album>]]`; the wiki category page is user-written and not overwritten | 2026-06-09 |
|
| Album categories | Auto-injected `[[Category:Album:<name>]]`; category page is user-written and not overwritten | 2026-06-09 |
|
||||||
|
| Additional categories | Optional `categories` list in frontmatter; each entry → `[[Category:<value>]]` | 2026-06-10 |
|
||||||
|
| `album` optional | Companion/sibling pages don't need an album; omitting the field emits no album category | 2026-06-10 |
|
||||||
|
| Sibling resolution | Parent declares siblings as a bill of materials (`path` + `tag` + optional `label`); `[[TAG]]` substituted post-Pandoc | 2026-06-10 |
|
||||||
| Source-repo branch convention | `autowiki/<song-slug>` short-lived branches in `ncmr-songs` | 2026-06-09 |
|
| Source-repo branch convention | `autowiki/<song-slug>` short-lived branches in `ncmr-songs` | 2026-06-09 |
|
||||||
| Idempotency | Commit SHA in banner (`commit=<sha>`), not timestamp — stable across identical re-runs | 2026-06-09 |
|
| Idempotency | Commit SHA in banner (`commit=<sha>`), not timestamp — stable across identical re-runs | 2026-06-09 |
|
||||||
| Publish mode | Always `--all`; no diff detection — small repo, simpler than fragile git-diff gating | 2026-06-09 |
|
| Publish mode | Always `--all`; no diff detection — small repo, simpler than fragile git-diff gating | 2026-06-09 |
|
||||||
|
|||||||
+109
-54
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
Source-file contract for the songs pipeline. Files in `mikkeli/ncmr-songs` must follow this contract to be picked up by the auto-publisher.
|
Source-file contract for the songs pipeline. Files in `mikkeli/ncmr-songs` must follow this contract to be picked up by the auto-publisher.
|
||||||
|
|
||||||
The schema lives entirely in the YAML frontmatter block at the top of each `.md` file. The body below the frontmatter is plain markdown, rendered to MediaWiki wikitext by Pandoc — no special body conventions are enforced beyond the standard markdown grammar.
|
The schema lives entirely in the YAML frontmatter block at the top of each `.md` file. The body below the frontmatter is plain markdown, rendered to MediaWiki wikitext by Pandoc — no special body conventions are enforced beyond the standard markdown grammar, except that `[[TAG]]` placeholders declared via `wiki.siblings` are substituted with resolved wikilinks after rendering.
|
||||||
|
|
||||||
## File layout
|
## File layout
|
||||||
|
|
||||||
@@ -12,15 +12,20 @@ title: pulse under latent semantic envelopes
|
|||||||
album: Shirakaba Express Service
|
album: Shirakaba Express Service
|
||||||
wiki:
|
wiki:
|
||||||
publish: true
|
publish: true
|
||||||
release_date: 2026-06-09
|
siblings:
|
||||||
|
- path: pulse_dev.md
|
||||||
|
tag: PULSE_DEV
|
||||||
|
label: development notes
|
||||||
|
release_date: 2026-06-18
|
||||||
|
lrc: pulse.lrc
|
||||||
---
|
---
|
||||||
|
|
||||||
# pulse under latent semantic envelopes
|
# pulse under latent semantic envelopes
|
||||||
|
|
||||||
... body markdown ...
|
... body markdown, [[PULSE_DEV]] used inline ...
|
||||||
```
|
```
|
||||||
|
|
||||||
The renderer parses the frontmatter, validates required fields, then hands the body (with the first `# Heading` line removed) to Pandoc.
|
The renderer parses the frontmatter, validates required fields, then hands the body (with the first `# Heading` line removed) to Pandoc, then applies sibling substitutions on the wikitext output.
|
||||||
|
|
||||||
## Fields
|
## Fields
|
||||||
|
|
||||||
@@ -34,56 +39,95 @@ Example: `title: pulse under latent semantic envelopes` → wiki page `Pulse und
|
|||||||
|
|
||||||
The body should contain a matching `# Title` first-line heading for human readability when viewing the source file directly (in an editor, on Gitea's blob view, etc.). The renderer strips this first-line heading during conversion — the wiki page already supplies its own title.
|
The body should contain a matching `# Title` first-line heading for human readability when viewing the source file directly (in an editor, on Gitea's blob view, etc.). The renderer strips this first-line heading during conversion — the wiki page already supplies its own title.
|
||||||
|
|
||||||
#### `album` (string)
|
|
||||||
|
|
||||||
The full album name. Drives `[[Category:<album name>]]` injection at the bottom of the rendered wiki page. MediaWiki auto-creates the category page when at least one page references it.
|
|
||||||
|
|
||||||
The pipeline **does not** create or update a dedicated album info/index page. Album landing pages are the wiki's category pages, which you write freeform yourself.
|
|
||||||
|
|
||||||
Example: `album: Shirakaba Express Service` → category tag `[[Category:Shirakaba Express Service]]`.
|
|
||||||
|
|
||||||
For files where no album applies (placeholders, songs without an assigned album), use a sensible literal like `(unassigned)` and accept that they'll get a `[[Category:(unassigned)]]` tag. The field is required to keep the schema's contract simple.
|
|
||||||
|
|
||||||
#### `wiki.publish` (boolean)
|
#### `wiki.publish` (boolean)
|
||||||
|
|
||||||
Publishing gate. `true` means the pipeline writes this file to the wiki. `false` means the file is ignored.
|
Publishing gate. `true` means the pipeline writes this file to the wiki. `false` means the file is ignored.
|
||||||
|
|
||||||
The `wiki` key is a mapping rather than a flat field so future toggles (`wiki.protected`, `wiki.summary_template`, etc.) can be added without restructuring. None are defined yet.
|
The `wiki` key is a mapping rather than a flat field so future toggles (`wiki.protected`, `wiki.summary_template`, etc.) can be added without restructuring.
|
||||||
|
|
||||||
Files under `wip/` are also skipped, regardless of `wiki.publish`.
|
Files under `wip/` are also skipped, regardless of `wiki.publish`.
|
||||||
|
|
||||||
### Optional
|
### Optional
|
||||||
|
|
||||||
|
#### `album` (string)
|
||||||
|
|
||||||
|
The full album name. Drives `[[Category:Album:<album name>]]` injection at the bottom of the rendered wiki page.
|
||||||
|
|
||||||
|
Example: `album: Shirakaba Express Service` → category tag `[[Category:Album:Shirakaba Express Service]]`.
|
||||||
|
|
||||||
|
Pages without an album (companion documents, placeholders, songs without an assigned album) simply omit this field — no category is injected unless `categories` provides one.
|
||||||
|
|
||||||
|
#### `categories` (list of strings)
|
||||||
|
|
||||||
|
Additional wiki categories to inject, beyond the album category. Each entry becomes `[[Category:<value>]]`.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
categories:
|
||||||
|
- Song development notes
|
||||||
|
- Work in progress
|
||||||
|
```
|
||||||
|
|
||||||
|
`album` and `categories` are additive — a page can have both.
|
||||||
|
|
||||||
#### `release_date` (ISO 8601 date)
|
#### `release_date` (ISO 8601 date)
|
||||||
|
|
||||||
The song's release date in `YYYY-MM-DD` format. Skipped if absent. Currently informational; may drive sort order on future auto-generated indexes.
|
The song's release date in `YYYY-MM-DD` format. Skipped if absent. Currently informational; may drive sort order on future auto-generated indexes.
|
||||||
|
|
||||||
#### `lrc` (string, relative path)
|
#### `lrc` (string, relative path)
|
||||||
|
|
||||||
Pointer to a companion LRC file, relative to the `.md` file's location.
|
Pointer to a companion LRC file, relative to the `.md` file's location. The pipeline uploads the file to the wiki as `<title>.lrc` (if the content has changed) and appends a `[[Media:<title>.lrc|Synced lyrics (.lrc)]]` link to the page footer.
|
||||||
|
|
||||||
Example: in `ses/tunnels.md`, `lrc: tunnels.lrc` declares that `ses/tunnels.lrc` is the companion synced-lyrics file.
|
Example: in `ses/pulse.md`, `lrc: pulse.lrc` declares that `ses/pulse.lrc` is the companion synced-lyrics file.
|
||||||
|
|
||||||
**Deferred for v1**: the field is permitted by the schema but the renderer doesn't yet read or embed LRC contents.
|
If the LRC file is missing or not valid UTF-8, the upload is skipped with a warning — the page still publishes.
|
||||||
|
|
||||||
#### `siblings` (mapping of placeholder → relative path)
|
#### `wiki.siblings` (list of sibling declarations)
|
||||||
|
|
||||||
Placeholder-to-file mapping for inter-page wikilinks.
|
Bill of materials for inter-page wikilinks. Each entry declares a sibling `.md` file and a tag that can be used as a `[[TAG]]` placeholder anywhere in the body.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
siblings:
|
wiki:
|
||||||
PULSE_DEV: pulse_dev.md
|
siblings:
|
||||||
|
- path: pulse_dev.md # relative to this file's directory
|
||||||
|
tag: PULSE_DEV # placeholder used in body as [[PULSE_DEV]]
|
||||||
|
label: development notes # optional display text
|
||||||
```
|
```
|
||||||
|
|
||||||
In the body markdown, write `[[PULSE_DEV]]` for a wikilink that displays the target's title, or `[[PULSE_DEV|custom display]]` for an aliased display. Resolution uses the target file's `title:` frontmatter as the wiki page name.
|
**Fields per entry:**
|
||||||
|
|
||||||
**Deferred for v1**: the field is permitted by the schema but placeholders pass through untouched.
|
| Field | Required for resolution | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| `path` | yes | Relative path to the sibling `.md` file |
|
||||||
|
| `tag` | yes | Placeholder identifier; pipeline hunts for `[[TAG]]` in the rendered wikitext |
|
||||||
|
| `label` | no | Display text for the resolved link |
|
||||||
|
|
||||||
|
An entry missing either `path` or `tag` is silently skipped — no substitution is attempted and any `[[TAG]]` in the body passes through as a literal wikilink to a page named `TAG`.
|
||||||
|
|
||||||
|
**Resolution:**
|
||||||
|
|
||||||
|
The pipeline resolves each declared sibling's `path` to its wiki page title by looking it up in the set of all publishable files processed in the same run. Substitution rules:
|
||||||
|
|
||||||
|
| Condition | `[[TAG]]` becomes |
|
||||||
|
|---|---|
|
||||||
|
| `path` resolves to a published page, `label` present | `[[wiki page title\|label]]` |
|
||||||
|
| `path` resolves to a published page, no `label` | `[[wiki page title]]` |
|
||||||
|
| `path` not in published set (file exists but not published), `label` present | `label` (plain text) |
|
||||||
|
| `path` not in published set, no `label` | `TAG` (plain text) |
|
||||||
|
|
||||||
|
Declared siblings whose file doesn't exist on disk are a **validation error**. Siblings that exist but aren't published (no `wiki.publish: true`) degrade gracefully per the table above.
|
||||||
|
|
||||||
|
The siblings list is a bill of materials — declaring a sibling has no effect unless its `[[TAG]]` appears in the body. Unused entries are inert.
|
||||||
|
|
||||||
|
**`--files` mode**: sibling resolution is limited to the files in that batch. Siblings pointing to files outside the batch degrade to plain text. This is acceptable since the pipeline always runs `--all` in the workflow.
|
||||||
|
|
||||||
## Body
|
## Body
|
||||||
|
|
||||||
Plain markdown. The renderer applies a single transformation before handing the body to Pandoc: the first-line `# Heading`, if present, is stripped.
|
Plain markdown. The renderer applies two transformations before and after Pandoc:
|
||||||
|
|
||||||
Do not embed wikitext-specific syntax (`{{Template}}`, raw `[[Wikilink]]` not declared via `siblings`, etc.) in the body. Use markdown idioms only; the renderer adds the metadata-derived bits (banner, categories, sibling resolution) around Pandoc's output.
|
1. **Pre-Pandoc**: strip the first-line `# Heading` if present.
|
||||||
|
2. **Post-Pandoc**: substitute `[[TAG]]` placeholders declared in `wiki.siblings`.
|
||||||
|
|
||||||
|
Do not embed raw wikitext-specific syntax (`{{Template}}`, raw `[[Wikilink]]` not declared via `siblings`, etc.) in the body unless you intend the literal output. Use markdown idioms; the renderer adds the metadata-derived bits (banner, LRC link, categories, sibling resolution) around Pandoc's output.
|
||||||
|
|
||||||
## Renderer behaviour
|
## Renderer behaviour
|
||||||
|
|
||||||
@@ -91,29 +135,33 @@ For each `.md` with `wiki.publish: true` and not under `wip/`:
|
|||||||
|
|
||||||
1. Parse and validate frontmatter.
|
1. Parse and validate frontmatter.
|
||||||
2. Strip the leading `# Heading` from the body if present.
|
2. Strip the leading `# Heading` from the body if present.
|
||||||
3. Replace `[[PLACEHOLDER]]` markers with safe tokens. *(Deferred v1; no-op until siblings are implemented.)*
|
3. Pipe the body through `pandoc -f markdown -t mediawiki`.
|
||||||
4. Pipe the body through `pandoc -f markdown -t mediawiki`.
|
4. Substitute `[[TAG]]` placeholders with resolved `[[Page|Display]]` wikilinks (or fallback text).
|
||||||
5. Re-substitute sibling tokens with resolved `[[Page|Display]]` wikilinks. *(Deferred v1.)*
|
5. Prepend the auto-generated banner: `{{Auto-generated|source=<source URL>|commit=<sha>}}`.
|
||||||
6. Prepend the auto-generated banner: `{{Auto-generated|source=<source URL>|commit=<sha>}}`.
|
6. If `lrc` is declared: upload the LRC file if its SHA1 has changed; append a `[[Media:…]]` link.
|
||||||
7. Append the album category: `[[Category:<album>]]`.
|
7. Append category tags: `[[Category:Album:<album>]]` if `album` is set; `[[Category:<cat>]]` for each entry in `categories`.
|
||||||
8. Read the current wiki page content via the MediaWiki API; if identical to the generated output, skip the write (idempotency — keeps the wiki history clean).
|
8. Read the current wiki page content via the MediaWiki API; if identical to the generated output, skip the write (idempotency — keeps the wiki history clean).
|
||||||
9. Otherwise, write the page with an edit summary referencing the source commit.
|
9. If the page exists and does not start with `{{Auto-generated`, skip with a warning (manual page protection).
|
||||||
|
10. Otherwise, write the page with an edit summary referencing the source commit.
|
||||||
|
|
||||||
## v1 scope
|
## Implemented scope
|
||||||
|
|
||||||
**Implemented:**
|
- Frontmatter parsing and validation
|
||||||
- Frontmatter parsing and validation for `title`, `album`, `wiki.publish`, `release_date`
|
- Pandoc-based markdown → wikitext body rendering
|
||||||
- Markdown body → wikitext via Pandoc
|
|
||||||
- First-line h1 stripping
|
- First-line h1 stripping
|
||||||
- Banner template injection (requires `Template:Auto-generated` to exist on the wiki — see the pipeline `README.md` for the one-off setup)
|
- Banner template injection
|
||||||
- Album category injection
|
- LRC file upload and footer link
|
||||||
- Idempotent writes (no-op skip)
|
- Sibling placeholder resolution (`[[TAG]]` → `[[wiki page title|label]]`)
|
||||||
|
- Album category injection (`[[Category:Album:<name>]]`)
|
||||||
|
- Additional categories via `categories` list
|
||||||
|
- Idempotent writes (no-op skip when content matches)
|
||||||
|
- Manual-page protection (skip pages without auto-gen banner)
|
||||||
|
- Files without `wiki.publish: true` silently skipped
|
||||||
|
|
||||||
**Deferred:**
|
## Not in scope (intentional)
|
||||||
- `lrc` field consumption
|
|
||||||
- `siblings` placeholder resolution
|
- Multi-page aggregation: no album index pages generated; each file → its own wiki page. Album landing pages are wiki category pages, written freeform by the user.
|
||||||
- Multi-page aggregation (no album index pages generated; each file → its own wiki page)
|
- Language-aware rendering: the body conveys what languages exist; the renderer doesn't introspect.
|
||||||
- Language-aware rendering (the body conveys what languages exist; the renderer doesn't introspect)
|
|
||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
|
||||||
@@ -125,22 +173,28 @@ title: pulse under latent semantic envelopes
|
|||||||
album: Shirakaba Express Service
|
album: Shirakaba Express Service
|
||||||
wiki:
|
wiki:
|
||||||
publish: true
|
publish: true
|
||||||
release_date: 2026-06-09
|
siblings:
|
||||||
|
- path: pulse_dev.md
|
||||||
|
tag: PULSE_DEV
|
||||||
|
label: development notes
|
||||||
|
release_date: 2026-06-18
|
||||||
|
lrc: pulse.lrc
|
||||||
---
|
---
|
||||||
```
|
```
|
||||||
|
|
||||||
### A dev sister file (not published)
|
### A companion/sibling document (no album)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
---
|
---
|
||||||
title: pulse under latent semantic envelopes — personal dev notes
|
title: pulse under latent semantic envelopes - development notes
|
||||||
album: Shirakaba Express Service
|
|
||||||
wiki:
|
wiki:
|
||||||
publish: false
|
publish: true
|
||||||
|
categories:
|
||||||
|
- Song development notes
|
||||||
---
|
---
|
||||||
```
|
```
|
||||||
|
|
||||||
### A placeholder file (no real song yet)
|
### A placeholder (not published)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
---
|
---
|
||||||
@@ -151,12 +205,11 @@ wiki:
|
|||||||
---
|
---
|
||||||
```
|
```
|
||||||
|
|
||||||
### A WIP file (in `wip/`, schema still applies but the file is skipped regardless)
|
### A WIP file (in `wip/`, skipped regardless)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
---
|
---
|
||||||
title: 海淀
|
title: 海淀
|
||||||
album: (unassigned)
|
|
||||||
wiki:
|
wiki:
|
||||||
publish: false
|
publish: false
|
||||||
---
|
---
|
||||||
@@ -166,10 +219,12 @@ wiki:
|
|||||||
|
|
||||||
Files without a `wiki.publish: true` frontmatter field are silently skipped — no error. The pipeline fails fast and loud only on files that are opted in (`wiki.publish: true`) with invalid data:
|
Files without a `wiki.publish: true` frontmatter field are silently skipped — no error. The pipeline fails fast and loud only on files that are opted in (`wiki.publish: true`) with invalid data:
|
||||||
|
|
||||||
- Missing required field (`title` or `album`) on a file with `wiki.publish: true`
|
- Missing or empty `title` on a file with `wiki.publish: true`
|
||||||
- `wiki.publish` present but not a boolean
|
- `wiki.publish` present but not a boolean
|
||||||
- `release_date` (if present) does not parse as YYYY-MM-DD
|
- `release_date` (if present) does not parse as YYYY-MM-DD
|
||||||
- `siblings` (if present) references a file that doesn't exist
|
- `categories` (if present) is not a list
|
||||||
|
- `wiki.siblings` entry that has both `path` and `tag` declared, but the `path` file does not exist on disk
|
||||||
|
- `wiki.siblings` entry that is not a mapping
|
||||||
- Two publishable source files declare the same `title` (page-name collision)
|
- Two publishable source files declare the same `title` (page-name collision)
|
||||||
|
|
||||||
Validation runs before any wiki API calls. A failed validation aborts the run with a non-zero exit and leaves the wiki untouched.
|
Validation runs before any wiki API calls. A failed validation aborts the run with a non-zero exit and leaves the wiki untouched.
|
||||||
|
|||||||
@@ -17,10 +17,9 @@ import mwclient
|
|||||||
def validate(fm: dict, path: Path) -> None:
|
def validate(fm: dict, path: Path) -> None:
|
||||||
errors = []
|
errors = []
|
||||||
|
|
||||||
for field in ("title", "album"):
|
val = fm.get("title")
|
||||||
val = fm.get(field)
|
|
||||||
if not val or not str(val).strip():
|
if not val or not str(val).strip():
|
||||||
errors.append(f"missing or empty '{field}'")
|
errors.append("missing or empty 'title'")
|
||||||
|
|
||||||
if not isinstance(fm.get("wiki", {}).get("publish"), bool):
|
if not isinstance(fm.get("wiki", {}).get("publish"), bool):
|
||||||
errors.append("'wiki.publish' must be a boolean (true/false), not a string")
|
errors.append("'wiki.publish' must be a boolean (true/false), not a string")
|
||||||
@@ -31,6 +30,19 @@ def validate(fm: dict, path: Path) -> None:
|
|||||||
except ValueError:
|
except ValueError:
|
||||||
errors.append("'release_date' must be YYYY-MM-DD")
|
errors.append("'release_date' must be YYYY-MM-DD")
|
||||||
|
|
||||||
|
if "categories" in fm and not isinstance(fm["categories"], list):
|
||||||
|
errors.append("'categories' must be a list")
|
||||||
|
|
||||||
|
for i, sib in enumerate(fm.get("wiki", {}).get("siblings") or []):
|
||||||
|
if not isinstance(sib, dict):
|
||||||
|
errors.append(f"wiki.siblings[{i}] must be a mapping")
|
||||||
|
continue
|
||||||
|
sib_path = sib.get("path")
|
||||||
|
sib_tag = sib.get("tag")
|
||||||
|
if sib_path and sib_tag:
|
||||||
|
if not (path.parent / str(sib_path)).resolve().exists():
|
||||||
|
errors.append(f"wiki.siblings[{i}]: path '{sib_path}' not found")
|
||||||
|
|
||||||
if errors:
|
if errors:
|
||||||
raise ValueError(f"{path}: " + "; ".join(errors))
|
raise ValueError(f"{path}: " + "; ".join(errors))
|
||||||
|
|
||||||
@@ -59,11 +71,47 @@ def markdown_to_wikitext(body: str) -> str:
|
|||||||
return result.stdout
|
return result.stdout
|
||||||
|
|
||||||
|
|
||||||
|
def build_sibling_map(posts: dict) -> dict[str, str]:
|
||||||
|
"""Map abs_path_str → wiki_title for all published posts."""
|
||||||
|
return {str(path): str(post.metadata["title"]) for path, post in posts.items()}
|
||||||
|
|
||||||
|
|
||||||
|
def apply_sibling_substitutions(wikitext: str, this_path: Path, fm_wiki: dict, sibling_map: dict[str, str]) -> str:
|
||||||
|
"""Replace [[TAG]] placeholders using the declared sibling bill of materials."""
|
||||||
|
for sib in (fm_wiki.get("siblings") or []):
|
||||||
|
if not isinstance(sib, dict):
|
||||||
|
continue
|
||||||
|
tag = sib.get("tag")
|
||||||
|
path = sib.get("path")
|
||||||
|
if not tag or not path:
|
||||||
|
continue
|
||||||
|
label = (sib.get("label") or "").strip()
|
||||||
|
abs_sib = str((this_path.parent / str(path)).resolve())
|
||||||
|
title = sibling_map.get(abs_sib)
|
||||||
|
if title and label:
|
||||||
|
replacement = f"[[{title}|{label}]]"
|
||||||
|
elif title:
|
||||||
|
replacement = f"[[{title}]]"
|
||||||
|
elif label:
|
||||||
|
replacement = label
|
||||||
|
else:
|
||||||
|
replacement = tag
|
||||||
|
wikitext = wikitext.replace(f"[[{tag}]]", replacement)
|
||||||
|
return wikitext
|
||||||
|
|
||||||
|
|
||||||
def build_wikitext(fm: dict, body_wikitext: str, source_url: str, source_ref: str, lrc_filename: str | None = None) -> str:
|
def build_wikitext(fm: dict, body_wikitext: str, source_url: str, source_ref: str, lrc_filename: str | None = None) -> str:
|
||||||
banner = f"{{{{Auto-generated|source={source_url}|commit={source_ref}}}}}"
|
banner = f"{{{{Auto-generated|source={source_url}|commit={source_ref}}}}}"
|
||||||
category = f"[[Category:{fm['album']}]]"
|
|
||||||
lrc_line = f"[[Media:{lrc_filename}|Synced lyrics (.lrc)]]\n" if lrc_filename else ""
|
lrc_line = f"[[Media:{lrc_filename}|Synced lyrics (.lrc)]]\n" if lrc_filename else ""
|
||||||
return f"{banner}\n\n{body_wikitext}{lrc_line}{category}\n"
|
|
||||||
|
cat_parts = []
|
||||||
|
if fm.get("album"):
|
||||||
|
cat_parts.append(f"[[Category:Album:{fm['album']}]]")
|
||||||
|
for cat in (fm.get("categories") or []):
|
||||||
|
cat_parts.append(f"[[Category:{cat}]]")
|
||||||
|
categories = "\n".join(cat_parts) + "\n" if cat_parts else ""
|
||||||
|
|
||||||
|
return f"{banner}\n\n{body_wikitext}{lrc_line}{categories}"
|
||||||
|
|
||||||
|
|
||||||
def connect_wiki() -> mwclient.Site:
|
def connect_wiki() -> mwclient.Site:
|
||||||
@@ -110,13 +158,14 @@ def upload_lrc(lrc_path: Path, title: str, site: mwclient.Site) -> bool:
|
|||||||
AUTO_BANNER_PREFIX = "{{Auto-generated"
|
AUTO_BANNER_PREFIX = "{{Auto-generated"
|
||||||
|
|
||||||
|
|
||||||
def publish_one(abs_path: Path, source_dir: Path, post, site: mwclient.Site, source_ref: str, gitea_repo_url: str) -> str:
|
def publish_one(abs_path: Path, source_dir: Path, post, site: mwclient.Site, source_ref: str, gitea_repo_url: str, sibling_map: dict[str, str]) -> str:
|
||||||
"""Returns 'published', 'noop', or 'skipped-manual'."""
|
"""Returns 'published', 'noop', or 'skipped-manual'."""
|
||||||
rel = abs_path.relative_to(source_dir)
|
rel = abs_path.relative_to(source_dir)
|
||||||
fm = post.metadata
|
fm = post.metadata
|
||||||
title = str(fm["title"])
|
title = str(fm["title"])
|
||||||
|
|
||||||
body_wikitext = markdown_to_wikitext(strip_first_h1(post.content))
|
body_wikitext = markdown_to_wikitext(strip_first_h1(post.content))
|
||||||
|
body_wikitext = apply_sibling_substitutions(body_wikitext, abs_path, fm.get("wiki", {}), sibling_map)
|
||||||
source_url = f"{gitea_repo_url}/src/commit/{source_ref}/{rel}"
|
source_url = f"{gitea_repo_url}/src/commit/{source_ref}/{rel}"
|
||||||
|
|
||||||
lrc_filename = None
|
lrc_filename = None
|
||||||
@@ -204,6 +253,7 @@ def main() -> None:
|
|||||||
|
|
||||||
print(f"Validating {len(files)} file(s)...")
|
print(f"Validating {len(files)} file(s)...")
|
||||||
posts = load_and_validate(files, source_dir)
|
posts = load_and_validate(files, source_dir)
|
||||||
|
sibling_map = build_sibling_map(posts)
|
||||||
print("Validation passed.")
|
print("Validation passed.")
|
||||||
|
|
||||||
site = connect_wiki()
|
site = connect_wiki()
|
||||||
@@ -211,7 +261,7 @@ def main() -> None:
|
|||||||
counts = {"published": 0, "noop": 0, "skipped-manual": 0}
|
counts = {"published": 0, "noop": 0, "skipped-manual": 0}
|
||||||
for abs_path, post in posts.items():
|
for abs_path, post in posts.items():
|
||||||
rel = abs_path.relative_to(source_dir)
|
rel = abs_path.relative_to(source_dir)
|
||||||
outcome = publish_one(abs_path, source_dir, post, site, source_ref, gitea_repo_url)
|
outcome = publish_one(abs_path, source_dir, post, site, source_ref, gitea_repo_url, sibling_map)
|
||||||
counts[outcome] += 1
|
counts[outcome] += 1
|
||||||
if outcome == "skipped-manual":
|
if outcome == "skipped-manual":
|
||||||
print(f" [skipped-manual] {rel} ← page exists without auto-gen banner; delete or add banner to hand over to bot", file=sys.stderr)
|
print(f" [skipped-manual] {rel} ← page exists without auto-gen banner; delete or add banner to hand over to bot", file=sys.stderr)
|
||||||
|
|||||||
Reference in New Issue
Block a user