mikkeli
905a1051a8
Build destination, both builder-image references and the three publish pins now point at novoyuuparosk-wiki/*. The publish pins name a tag that does not exist in the new namespace until this run's build job pushes it and the pin job rewrites them, so publishes are briefly broken mid-run.
121 lines
8.9 KiB
Markdown
121 lines
8.9 KiB
Markdown
# novoyuuparosk-auto-wiki
|
|
|
|
CI/CD pipelines that auto-apply commits to https://wiki.novoyuuparosk.org from upstream content repos.
|
|
|
|
## Pipelines
|
|
|
|
| Path | Source repo | Purpose | Status |
|
|
|---|---|---|---|
|
|
| [`pipelines/songs/`](pipelines/songs/) | `novoyuuparosk-wiki/ncmr-songs` | Song lyric pages | v1 live |
|
|
| [`pipelines/ses/`](pipelines/ses/) | `novoyuuparosk-wiki/ses-light-novel` | SES light novel pages | v1 in development |
|
|
| [`pipelines/tech/`](pipelines/tech/) | `novoyuuparosk-wiki/tech-blogs` | Tech blog posts | v1 live |
|
|
|
|
Per-pipeline READMEs cover everything specific to that pipeline (source schema, renderer, runtime, decisions). This root README covers only what's cross-cutting.
|
|
|
|
## Architecture
|
|
|
|
Hybrid layout. The Gitea Actions trigger must live in the source repo (Gitea only fires workflows from `.gitea/workflows/` of the pushed-to repo); the rendering logic lives here. The source-repo workflow is self-contained but clones this repo at runtime to get the renderer.
|
|
|
|
```
|
|
<source-repo>/
|
|
.gitea/workflows/<name>.yml <- workflow: clones this repo, runs renderer (deps pre-baked in job image)
|
|
|
|
novoyuuparosk-auto-wiki/ <- this repo
|
|
.gitea/workflows/<pipeline>.yml <- reusable workflow stubs (kept for reference; not actively called; may be stale)
|
|
pipelines/<pipeline>/ <- per-pipeline code, schema, templates
|
|
lib/ <- shared modules (MediaWiki client, etc.)
|
|
```
|
|
|
|
Note: `workflow_call` across private repos was abandoned — the auto-generated run token is scoped to the triggering repo only and cannot clone a private callee. The source-repo workflow clones this repo directly using `FAPAT`.
|
|
|
|
## Wiki
|
|
|
|
- Base URL: https://wiki.novoyuuparosk.org
|
|
- MediaWiki API: https://wiki.novoyuuparosk.org/api.php *(confirmed)*
|
|
|
|
## Bot identity
|
|
|
|
MediaWiki BotPassword issued for user `Mikkeli`, bot name `giteaBot`. Login form: `Mikkeli@giteaBot`.
|
|
|
|
`Mikkeli` holds the `bot` right, so pipeline edits are flagged as bot edits and stay out of default Recent Changes — `mwclient`'s `page.save()` requests the bot flag and the wiki honours it.
|
|
|
|
This section previously named `Dubrowski@giteaAutomaton`, but every bot-flagged revision in the wiki's history is attributed to `Mikkeli`, so that had been stale for some time — the live secret never matched the doc.
|
|
|
|
Credentials are stored in the Gitea org-scope secret vault under `novoyuuparosk-wiki` (`WIKI_BOT_USER`, `WIKI_BOT_PASSWORD`). Not stored in this repo.
|
|
|
|
## Runner infrastructure
|
|
|
|
One `act_runner` instance serves all pipelines. Runs on a Pi 5 (Raspberry Pi OS Bookworm, `aarch64`) inside the same `docker-compose` stack that hosts the Gitea instance. Job execution is via the host Docker socket — runner is a container, jobs spawn as sibling containers.
|
|
|
|
`act_runner` build: `linux-arm64`, from the `gitea/act_runner` Docker image.
|
|
|
|
Container network mode: `host` — required so job containers can reach `localhost:3005` (Gitea) and resolve mDNS hostnames.
|
|
|
|
### Job container image
|
|
|
|
All pipelines share a single pre-built Docker image, served from the Gitea registry at `pi5-16.local:3005/novoyuuparosk-wiki/novoyuuparosk-wiki-runner`. The `Dockerfile` is at the repo root. It bakes in system deps (git, pandoc, ca-certificates) and all pipeline Python packages so job containers start instantly with no install steps.
|
|
|
|
The image builds automatically via [`.gitea/workflows/build-image.yml`](.gitea/workflows/build-image.yml), which triggers on pushes that touch the `Dockerfile`, any pipeline `requirements.txt`, or that workflow itself. It uses kaniko (daemonless, unprivileged) to build and push two tags: an immutable `:<short-sha>` and a moving `:latest`.
|
|
|
|
A follow-up `pin` job then rewrites the `image:` pin in each `publish-*.yml` to the new `:<short-sha>` and commits it back to `master` (using `FAPAT` for contents write). The publish workflows therefore always reference an immutable tag, kept current automatically — no manual bump. The pin commit only touches `workflow_call` files, so it triggers no further runs.
|
|
|
|
## Gitea Actions setup (cross-cutting)
|
|
|
|
Secrets and variables are scoped to the `novoyuuparosk-wiki` org, inherited by all repos under it. Runs belong to the *caller* repo, so a source repo calling a reusable workflow here resolves secrets and variables from its own owner — which is why they live at org scope rather than on this repo.
|
|
|
|
**Secrets:**
|
|
- `WIKI_BOT_USER` = `Mikkeli@giteaBot`
|
|
- `WIKI_BOT_PASSWORD` = the value from *Bot identity* above
|
|
- `FAPAT` = Full-Access PAT owned by `mikkeli`, used by source-repo workflows to clone this repo at runtime. The PAT stays personal; only its storage scope moved to the org — hence the `mikkeli:` basic-auth username in the clone URLs.
|
|
- `PKGRW_PAT` = package read/write PAT owned by `mikkeli`, used by `build-image.yml` to push to the container registry
|
|
|
|
**Variables:**
|
|
- `WIKI_BASE_URL` = `https://wiki.novoyuuparosk.org`
|
|
- `WIKI_API_URL` = `https://wiki.novoyuuparosk.org/api.php`
|
|
- `URL_TO_GITEA` = Gitea instance base URL (e.g. `http://localhost:3005`). Named with `URL_TO_` prefix — Gitea blocks variable names starting with `GITEA_` or `GITHUB_`.
|
|
|
|
## Branch naming
|
|
|
|
- This repo: `automation/<pipeline-name>` for pipeline-development branches (e.g., `automation/songs`).
|
|
- Source repos: each pipeline's README defines the source-side branch convention (e.g., `autowiki/<song-slug>` in `ncmr-songs`).
|
|
|
|
## Decisions log (cross-cutting)
|
|
|
|
| Decision | Value | Date |
|
|
|---|---|---|
|
|
| Architecture | Hybrid: source-repo workflow clones this repo at runtime for the renderer | 2026-06-09 |
|
|
| Workflow pattern | Self-contained (not `workflow_call`) — cross-repo `workflow_call` blocked by token scoping on private repos | 2026-06-09 |
|
|
| Runner execution | Docker, added as a service to the existing Gitea docker-compose | 2026-06-09 |
|
|
| Runner network mode | `host` — job containers need to reach Gitea on localhost | 2026-06-09 |
|
|
| Secret/runner scope | User-level on `mikkeli` (no orgs on this instance) | 2026-06-09 |
|
|
| Ownership | This repo and all three source repos moved to the `novoyuuparosk-wiki` org. Secrets/variables re-created at org scope; runner re-registered instance-level so it serves org-owned runs | 2026-08-11 |
|
|
| **This repo must stay public** | `act_runner` resolves a cross-repo `uses:` by cloning the callee **anonymously** — the job token is not applied. A private callee therefore 404s with `repository not found`, regardless of the caller sharing its owner. Shared ownership does **not** satisfy the read requirement. Alternative if it must be private again: Settings → Actions → General → collaborative owners (Gitea 1.26+), untested here | 2026-08-11 |
|
|
| Runner cache masks this | The resolved callee is cached at `/root/.cache/act/<owner>-<repo>@<ref>`. Changing owner changes the key, so a working pipeline can break on a clone that had been served from cache for months. Suspect the cache before suspecting permissions | 2026-08-11 |
|
|
| Container registry | Gitea cannot transfer packages, so images were *rebuilt* into `pi5-16.local:3005/novoyuuparosk-wiki/*` rather than moved. `kaniko-act` bootstrapped via the new dispatch-only `build-kaniko-act.yml`, running in the old `mikkeli` copy; `novoyuuparosk-wiki-runner` came from a normal `build-image.yml` run. Registry auth is still the `mikkeli`-owned `PKGRW_PAT`, hence the `mikkeli:` username in the auth blob | 2026-08-11 |
|
|
| Old images left in place | The `mikkeli/*` package versions are orphaned but retained — nothing references them, and deleting a container version is irreversible. Safe to purge once the org images have proven themselves | 2026-08-11 |
|
|
| MediaWiki API path | `api.php` (classic action API) | 2026-06-09 |
|
|
| Branch naming (this repo) | `automation/<pipeline>` for pipeline-development branches | 2026-06-09 |
|
|
| Variable naming | `URL_TO_GITEA` not `GITEA_URL` — Gitea blocks `GITEA_`/`GITHUB_` prefixes | 2026-06-09 |
|
|
| Columns shorthand | Side-by-side columns authored as a ` ```columns ` fenced block, expanded post-Pandoc in shared `lib/wiki.py` (universal across pipelines). No wiki template or PHP extension — runs Pi-side before the API call | 2026-06-14 |
|
|
|
|
Per-pipeline decisions live in each pipeline's README.
|
|
|
|
## Setup checklist (cross-cutting)
|
|
|
|
Via the Gitea web UI logged in as `mikkeli` (an owner of `novoyuuparosk-wiki`):
|
|
|
|
- [x] Org-scoped secrets and variables set per *Gitea Actions setup* above
|
|
- [x] `WIKI_BOT_USER`
|
|
- [x] `WIKI_BOT_PASSWORD`
|
|
- [x] `FAPAT` (Full-Access PAT — value not stored in this README; saved directly into the Gitea secret. Regenerate if lost.)
|
|
- [x] `WIKI_BASE_URL`
|
|
- [x] `WIKI_API_URL`
|
|
- [x] `URL_TO_GITEA`
|
|
|
|
With Pi access:
|
|
|
|
- [x] Add `act_runner` service to the existing Gitea docker-compose
|
|
- [x] Generate a runner registration token at `/-/admin/actions/runners`, bake into the compose env, `docker compose up -d act_runner`, confirm "online" in the Gitea UI
|
|
|
|
Per-pipeline setup lives in each pipeline's README. Start with [`pipelines/songs/`](pipelines/songs/).
|