Merge branch 'automation/columns-shorthand'
Side-by-side columns shorthand: a 'columns' fenced block (columns split by a line of ===) is expanded post-Pandoc in shared lib/wiki.py into a flex div of poem columns. Universal across pipelines; no wiki template or PHP extension.
This commit is contained in:
+4
-1
@@ -1,2 +1,5 @@
|
|||||||
# claude local settings
|
# claude local settings
|
||||||
.claude/
|
.claude/
|
||||||
|
|
||||||
|
# python
|
||||||
|
__pycache__/
|
||||||
@@ -86,6 +86,7 @@ Secrets and variables are scoped to user `mikkeli` (no orgs on this instance), i
|
|||||||
| MediaWiki API path | `api.php` (classic action API) | 2026-06-09 |
|
| MediaWiki API path | `api.php` (classic action API) | 2026-06-09 |
|
||||||
| Branch naming (this repo) | `automation/<pipeline>` for pipeline-development branches | 2026-06-09 |
|
| Branch naming (this repo) | `automation/<pipeline>` for pipeline-development branches | 2026-06-09 |
|
||||||
| Variable naming | `URL_TO_GITEA` not `GITEA_URL` — Gitea blocks `GITEA_`/`GITHUB_` prefixes | 2026-06-09 |
|
| Variable naming | `URL_TO_GITEA` not `GITEA_URL` — Gitea blocks `GITEA_`/`GITHUB_` prefixes | 2026-06-09 |
|
||||||
|
| Columns shorthand | Side-by-side columns authored as a ` ```columns ` fenced block, expanded post-Pandoc in shared `lib/wiki.py` (universal across pipelines). No wiki template or PHP extension — runs Pi-side before the API call | 2026-06-14 |
|
||||||
|
|
||||||
Per-pipeline decisions live in each pipeline's README.
|
Per-pipeline decisions live in each pipeline's README.
|
||||||
|
|
||||||
|
|||||||
+29
-1
@@ -1,6 +1,8 @@
|
|||||||
"""Shared utilities for novoyuuparosk-auto-wiki pipelines."""
|
"""Shared utilities for novoyuuparosk-auto-wiki pipelines."""
|
||||||
|
|
||||||
|
import html
|
||||||
import os
|
import os
|
||||||
|
import re
|
||||||
import subprocess
|
import subprocess
|
||||||
from urllib.parse import urlparse
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
@@ -8,6 +10,11 @@ import mwclient
|
|||||||
|
|
||||||
AUTO_BANNER_PREFIX = "{{Auto-generated"
|
AUTO_BANNER_PREFIX = "{{Auto-generated"
|
||||||
|
|
||||||
|
# A ```columns fenced block is passed through Pandoc verbatim as
|
||||||
|
# <pre class="columns">…</pre>; columns within it are separated by a line of ===.
|
||||||
|
_COLUMNS_BLOCK_RE = re.compile(r'<pre class="columns">(.*?)</pre>', re.DOTALL)
|
||||||
|
_COLUMN_SEP_RE = re.compile(r"^\s*===\s*$", re.MULTILINE)
|
||||||
|
|
||||||
|
|
||||||
def strip_first_h1(text: str) -> str:
|
def strip_first_h1(text: str) -> str:
|
||||||
"""Remove the first '# Heading' line and any immediately following blank line."""
|
"""Remove the first '# Heading' line and any immediately following blank line."""
|
||||||
@@ -21,6 +28,27 @@ def strip_first_h1(text: str) -> str:
|
|||||||
return "\n".join(lines)
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def expand_columns(wikitext: str) -> str:
|
||||||
|
"""Expand ```columns fenced blocks into a flex row of <poem> columns.
|
||||||
|
|
||||||
|
Authors write a fenced code block tagged ``columns``; Pandoc passes its body
|
||||||
|
through verbatim as ``<pre class="columns">…</pre>`` (line breaks and blank
|
||||||
|
lines preserved, inline markup entity-escaped). Columns within the block are
|
||||||
|
separated by a line containing only ``===``. Each column is wrapped in
|
||||||
|
<poem> so its line breaks survive MediaWiki parsing, and content is
|
||||||
|
HTML-unescaped so inline markup written in the fence (e.g. <b>…</b>) renders
|
||||||
|
rather than appearing as literal text.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def render(match: re.Match) -> str:
|
||||||
|
body = html.unescape(match.group(1))
|
||||||
|
columns = _COLUMN_SEP_RE.split(body)
|
||||||
|
poems = "".join("<poem>\n" + col.strip("\n") + "\n</poem>" for col in columns)
|
||||||
|
return '<div style="display:flex; gap:3em; align-items:flex-start">' + poems + "</div>"
|
||||||
|
|
||||||
|
return _COLUMNS_BLOCK_RE.sub(render, wikitext)
|
||||||
|
|
||||||
|
|
||||||
def markdown_to_wikitext(body: str) -> str:
|
def markdown_to_wikitext(body: str) -> str:
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
["pandoc", "-f", "markdown", "-t", "mediawiki"],
|
["pandoc", "-f", "markdown", "-t", "mediawiki"],
|
||||||
@@ -30,7 +58,7 @@ def markdown_to_wikitext(body: str) -> str:
|
|||||||
)
|
)
|
||||||
if result.returncode != 0:
|
if result.returncode != 0:
|
||||||
raise RuntimeError(f"pandoc failed: {result.stderr.strip()}")
|
raise RuntimeError(f"pandoc failed: {result.stderr.strip()}")
|
||||||
return result.stdout
|
return expand_columns(result.stdout)
|
||||||
|
|
||||||
|
|
||||||
def connect_wiki() -> mwclient.Site:
|
def connect_wiki() -> mwclient.Site:
|
||||||
|
|||||||
@@ -89,6 +89,7 @@ The pipeline always runs `--all`: every file with `wiki.publish: true` (not unde
|
|||||||
| Publish mode | Always `--all`; no diff detection — small repo, simpler than fragile git-diff gating | 2026-06-09 |
|
| Publish mode | Always `--all`; no diff detection — small repo, simpler than fragile git-diff gating | 2026-06-09 |
|
||||||
| Manual-page protection | Bot skips pages without the `{{Auto-generated` banner to avoid overwriting hand-written content | 2026-06-09 |
|
| Manual-page protection | Bot skips pages without the `{{Auto-generated` banner to avoid overwriting hand-written content | 2026-06-09 |
|
||||||
| `workflow_call` abandoned | Cross-repo `workflow_call` fails — run token scoped to triggering repo; cannot clone private callee | 2026-06-09 |
|
| `workflow_call` abandoned | Cross-repo `workflow_call` fails — run token scoped to triggering repo; cannot clone private callee | 2026-06-09 |
|
||||||
|
| Side-by-side columns | Authored as a ` ```columns ` fenced block (separator `===`); Pandoc emits `<pre class="columns">`, expanded post-Pandoc into a flex `<div>` of `<poem>` columns. No wiki template or extension — all Pi-side before the API write. See [SCHEMA.md](SCHEMA.md#columns-side-by-side-shorthand) | 2026-06-14 |
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
|
|||||||
@@ -122,20 +122,46 @@ The siblings list is a bill of materials — declaring a sibling has no effect u
|
|||||||
|
|
||||||
## Body
|
## Body
|
||||||
|
|
||||||
Plain markdown. The renderer applies two transformations before and after Pandoc:
|
Plain markdown. The renderer applies these transformations before and after Pandoc:
|
||||||
|
|
||||||
1. **Pre-Pandoc**: strip the first-line `# Heading` if present.
|
1. **Pre-Pandoc**: strip the first-line `# Heading` if present.
|
||||||
2. **Post-Pandoc**: substitute `TAG` placeholders declared in `wiki.siblings`.
|
2. **Post-Pandoc**: expand `columns` fenced blocks into side-by-side wikitext (see below).
|
||||||
|
3. **Post-Pandoc**: substitute `TAG` placeholders declared in `wiki.siblings`.
|
||||||
|
|
||||||
Do not embed raw wikitext-specific syntax (`{{Template}}`, raw `[[Wikilink]]` not declared via `siblings`, etc.) in the body unless you intend the literal output. Use markdown idioms; the renderer adds the metadata-derived bits (banner, LRC link, categories, sibling resolution) around Pandoc's output.
|
Do not embed raw wikitext-specific syntax (`{{Template}}`, raw `[[Wikilink]]` not declared via `siblings`, etc.) in the body unless you intend the literal output. Use markdown idioms; the renderer adds the metadata-derived bits (banner, LRC link, categories, sibling resolution) around Pandoc's output.
|
||||||
|
|
||||||
|
### Columns (side-by-side) shorthand
|
||||||
|
|
||||||
|
For parallel content — e.g. an original and its translation — write a fenced code block tagged `columns` and separate the columns with a line containing only `===`:
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
```columns
|
||||||
|
<b>原題</b>
|
||||||
|
|
||||||
|
一行目
|
||||||
|
二行目
|
||||||
|
===
|
||||||
|
<b>Title</b>
|
||||||
|
|
||||||
|
first line
|
||||||
|
second line
|
||||||
|
```
|
||||||
|
````
|
||||||
|
|
||||||
|
The renderer turns this into a flexbox row of `<poem>` columns (one per `===`-delimited section). Notes:
|
||||||
|
|
||||||
|
- **Line breaks and blank lines are preserved verbatim** — that's the point of using a fenced block; Pandoc passes the body through untouched, and each column is wrapped in `<poem>` so MediaWiki keeps the line breaks.
|
||||||
|
- **N columns**: use N−1 `===` separators. Two is the common case (original / translation).
|
||||||
|
- **Inline markup is allowed**: HTML such as `<b>…</b>` or `<br>` written inside the block is HTML-unescaped on the way out, so it renders rather than showing as literal text.
|
||||||
|
- This is a shared transform (`lib/wiki.py`), so it works for any pipeline, not just songs. Column width/gap styling currently lives in that transform.
|
||||||
|
|
||||||
## Renderer behaviour
|
## Renderer behaviour
|
||||||
|
|
||||||
For each `.md` with `wiki.publish: true` and not under `wip/`:
|
For each `.md` with `wiki.publish: true` and not under `wip/`:
|
||||||
|
|
||||||
1. Parse and validate frontmatter.
|
1. Parse and validate frontmatter.
|
||||||
2. Strip the leading `# Heading` from the body if present.
|
2. Strip the leading `# Heading` from the body if present.
|
||||||
3. Pipe the body through `pandoc -f markdown -t mediawiki`.
|
3. Pipe the body through `pandoc -f markdown -t mediawiki`, then expand any `columns` fenced blocks into side-by-side `<poem>` columns.
|
||||||
4. Substitute `TAG` placeholders with resolved `[[Page|Display]]` wikilinks (or fallback text).
|
4. Substitute `TAG` placeholders with resolved `[[Page|Display]]` wikilinks (or fallback text).
|
||||||
5. Prepend the auto-generated banner: `{{Auto-generated|source=<source URL>|commit=<sha>}}`.
|
5. Prepend the auto-generated banner: `{{Auto-generated|source=<source URL>|commit=<sha>}}`.
|
||||||
6. If `lrc` is declared: upload the LRC file if its SHA1 has changed; append a `[[Media:…]]` link.
|
6. If `lrc` is declared: upload the LRC file if its SHA1 has changed; append a `[[Media:…]]` link.
|
||||||
|
|||||||
Reference in New Issue
Block a user