docs: update all docs to reflect v1 live state
- template_auto_generated.wikitext: generated_at → commit parameter - SCHEMA.md: same banner fix; clarify that frontmatter-less files are silently skipped, not errors - pipelines/songs/README.md: full rewrite — v1 live status, self- contained workflow pattern, correct env table, modes, one-off setup marked done, full decisions log - README.md: architecture reflects actual pattern (no workflow_call), URL_TO_GITEA variable added, setup checklist marked complete, host network mode documented Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+31
-29
@@ -12,8 +12,10 @@ Implemented:
|
||||
- YAML frontmatter parsing per [SCHEMA.md](SCHEMA.md)
|
||||
- Pandoc-based markdown → wikitext body rendering
|
||||
- First-line h1 stripping
|
||||
- Auto-generated banner + album category injection
|
||||
- MediaWiki bot API write with idempotency (no-op edits skipped)
|
||||
- Auto-generated banner (`{{Auto-generated|source=...|commit=<sha>}}`) + album category injection
|
||||
- MediaWiki bot API write with idempotency — no-op if wiki content matches generated output
|
||||
- Files without `wiki.publish: true` silently skipped
|
||||
- Pages with existing non-auto-generated content skipped (logged to stderr) — delete the page or add the banner to hand ownership to the bot
|
||||
|
||||
Deferred (schema reserves the fields; renderer doesn't yet act on them):
|
||||
|
||||
@@ -35,41 +37,39 @@ Source repo: `mikkeli/ncmr-songs`. The pipeline triggers on push to `master`.
|
||||
|
||||
## Invocation
|
||||
|
||||
The pipeline is invoked via a reusable Gitea Actions workflow (`.gitea/workflows/publish-songs.yml` in this repo). The `ncmr-songs` repo holds a thin stub workflow (`.gitea/workflows/publish.yml`) that calls into this one on push to `master`.
|
||||
The pipeline runs as a self-contained Gitea Actions workflow in `mikkeli/ncmr-songs` (`.gitea/workflows/publish.yml`). It clones this repo at runtime to get the renderer.
|
||||
|
||||
### Workflow inputs and secrets
|
||||
Triggers:
|
||||
- `push` to `master` (path-filtered as above)
|
||||
- `workflow_dispatch` — manual trigger from the Gitea Actions UI; useful after renderer or template changes without needing a dummy commit
|
||||
|
||||
### Environment (secrets and variables)
|
||||
|
||||
| Name | Kind | Source | Purpose |
|
||||
|---|---|---|---|
|
||||
| `WIKI_API_URL` | variable | Gitea (cross-cutting) | MediaWiki action API endpoint |
|
||||
| `WIKI_BASE_URL` | variable | Gitea (cross-cutting) | Wiki base URL (used in the banner link) |
|
||||
| `WIKI_BOT_USER` | secret | Gitea (cross-cutting) | Bot login |
|
||||
| `WIKI_BASE_URL` | variable | Gitea (cross-cutting) | Wiki base URL |
|
||||
| `WIKI_BOT_USER` | secret | Gitea (cross-cutting) | Bot login (`Dubrowski@giteaAutomaton`) |
|
||||
| `WIKI_BOT_PASSWORD` | secret | Gitea (cross-cutting) | BotPasswords value |
|
||||
| `SOURCE_DIR` | workflow input | from the stub | Path to checked-out `ncmr-songs` working tree |
|
||||
| `SOURCE_REF` | workflow input | from the stub | The pushed-to commit SHA |
|
||||
| `SOURCE_BASE_REF` | workflow input | from the stub | The SHA prior to the push (for diff-based change detection) |
|
||||
| `URL_TO_GITEA` | variable | Gitea (cross-cutting) | Gitea instance base URL for cloning |
|
||||
| `FAPAT` | secret | Gitea (cross-cutting) | Full-Access PAT for cloning private repos |
|
||||
| `SOURCE_REF` | env (set in workflow) | `github.sha` | Pushed commit SHA, used in banner and edit summary |
|
||||
| `GITEA_REPO_URL` | env (set in workflow) | composed from vars | Source-file URL base for the banner link |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Python 3.12+
|
||||
- Pandoc 2.x or newer (apt-installable in the runner container)
|
||||
- `mwclient` or `requests` for the MediaWiki API
|
||||
- `PyYAML` for frontmatter parsing
|
||||
- `python-frontmatter` (convenience wrapper around PyYAML for markdown frontmatter)
|
||||
|
||||
`requirements.txt` will be added when the implementation lands.
|
||||
|
||||
## One-off wiki setup
|
||||
|
||||
Before the first run, two things must exist on the wiki:
|
||||
|
||||
1. **`Template:Auto-generated`** — the banner injected at the top of every auto-published page. Wikitext for this template is included in the implementation step (not yet written).
|
||||
2. **The bot user has edit rights** for the namespace(s) the pipeline writes to. The default main namespace is fine; verify by attempting a manual edit via the bot account before relying on the pipeline.
|
||||
- Python 3.12+ (job container: `python:3.12-slim`)
|
||||
- Pandoc (apt-installed in the job container)
|
||||
- `python-frontmatter`, `mwclient`, `PyYAML` — see `requirements.txt`
|
||||
|
||||
## Modes
|
||||
|
||||
- **Incremental** (default, triggered by push to `master`): publishes only files changed between `SOURCE_BASE_REF` and `SOURCE_REF`.
|
||||
- **Full** (`--all` flag, triggered by `workflow_dispatch`): re-publishes every publishable file in the source tree. Use after template or renderer changes.
|
||||
The pipeline always runs `--all`: every file with `wiki.publish: true` (not under `wip/`) is checked against the current wiki content. Writes only happen when content has changed (commit SHA in the banner drives idempotency). This is appropriate for the current repo size.
|
||||
|
||||
## One-off wiki setup
|
||||
|
||||
- [x] **`Template:Auto-generated`** — exists on the wiki. Wikitext source: `template_auto_generated.wikitext` in this directory.
|
||||
- [x] **Bot user in `bot` group** — grants `skipcaptcha`; verified working.
|
||||
|
||||
## Decisions log (songs pipeline)
|
||||
|
||||
@@ -79,11 +79,13 @@ Before the first run, two things must exist on the wiki:
|
||||
| Body rendering | Pandoc-based, with a thin Python pre/post-processor | 2026-06-09 |
|
||||
| Excluded paths | `wip/**`, plus repo-meta files | 2026-06-09 |
|
||||
| Page template | Designed from scratch (no existing wiki pages to mirror) | 2026-06-09 |
|
||||
| Album landing pages | Auto-injected `[[Category:<album>]]`; the wiki category page is user-written and not overwritten by the pipeline | 2026-06-09 |
|
||||
| Album landing pages | Auto-injected `[[Category:<album>]]`; the wiki category page is user-written and not overwritten | 2026-06-09 |
|
||||
| Source-repo branch convention | `autowiki/<song-slug>` short-lived branches in `ncmr-songs` | 2026-06-09 |
|
||||
| Idempotency | Commit SHA in banner (`commit=<sha>`), not timestamp — stable across identical re-runs | 2026-06-09 |
|
||||
| Publish mode | Always `--all`; no diff detection — small repo, simpler than fragile git-diff gating | 2026-06-09 |
|
||||
| Manual-page protection | Bot skips pages without the `{{Auto-generated` banner to avoid overwriting hand-written content | 2026-06-09 |
|
||||
| `workflow_call` abandoned | Cross-repo `workflow_call` fails — run token scoped to triggering repo; cannot clone private callee | 2026-06-09 |
|
||||
|
||||
## Status
|
||||
|
||||
Schema and pipeline overview committed. No executable code yet.
|
||||
|
||||
**Next**: implement renderer (Python module), workflow YAMLs (this repo + `ncmr-songs` stub), `Template:Auto-generated` wikitext, and end-to-end dry-run on `ses/pulse.md`.
|
||||
v1 live. Pipeline active and publishing to https://wiki.novoyuuparosk.org.
|
||||
|
||||
Reference in New Issue
Block a user