# novoyuuparosk-auto-wiki

CI/CD pipelines that auto-apply commits to https://wiki.novoyuuparosk.org from upstream content repos.

## Pipelines

| Path | Source repo | Purpose | Status |
|---|---|---|---|
| [`pipelines/songs/`](pipelines/songs/) | `novoyuuparosk-wiki/ncmr-songs` | Song lyric pages | v1 live |
| [`pipelines/ses/`](pipelines/ses/) | `novoyuuparosk-wiki/ses-light-novel` | SES light novel pages | v1 in development |
| [`pipelines/tech/`](pipelines/tech/) | `novoyuuparosk-wiki/tech-blogs` | Tech blog posts | v1 live |

Per-pipeline READMEs cover everything specific to that pipeline (source schema, renderer, runtime). This root README covers only what's cross-cutting.

## Architecture

Split across two repos, because Gitea only fires workflows from `.gitea/workflows/` of the repo that was pushed to, while the rendering logic belongs here.

```
<source-repo>/
  .gitea/workflows/publish.yml      <- thin caller: on: push, uses: the reusable workflow below

novoyuuparosk-auto-wiki/            <- this repo
  .gitea/workflows/publish-<p>.yml  <- on: workflow_call. Clones both repos, runs the renderer
  .gitea/workflows/test.yml         <- on: push. Runs tests/ in the publish runner image
  .gitea/workflows/build-image.yml  <- builds and pins the shared job container image
  pipelines/<pipeline>/             <- per-pipeline renderer, schema, templates
  lib/                              <- shared render and MediaWiki helpers
  tests/                            <- pytest suite for lib/ and the pipelines
```

The two halves reach the runner by different routes, which matters when a change appears not to take effect:

- **Renderer code** (`pipelines/`, `lib/`) is a plain `git clone` of this repo's **default branch** on every run. Always current, never cached.
- **Workflow YAML** (`publish-*.yml`) is resolved through the caller's `uses:` and **is cached** — see *Callee workflow cache* below.

**This repo must stay public.** `act_runner` resolves a cross-repo `uses:` by cloning the callee **anonymously** — the job token is not applied, and shared ownership does not satisfy the read requirement. A private callee 404s with `repository not found`. The alternative, untested here, is Settings → Actions → General → collaborative owners (Gitea 1.26+).

## Wiki

- Base URL: https://wiki.novoyuuparosk.org
- MediaWiki API: https://wiki.novoyuuparosk.org/api.php (classic action API)

## Bot identity

MediaWiki BotPassword issued for user `Mikkeli`, bot name `giteaBot`. Login form: `Mikkeli@giteaBot`.

`Mikkeli` holds the `bot` right, so pipeline edits are flagged as bot edits and stay out of default Recent Changes — `mwclient`'s `page.save()` requests the bot flag and the wiki honours it.

Credentials live in the Gitea org-scope secret vault under `novoyuuparosk-wiki` (`WIKI_BOT_USER`, `WIKI_BOT_PASSWORD`). Not stored in this repo.

## Runner infrastructure

One `act_runner` instance serves all pipelines. It runs on a Pi 5 (Raspberry Pi OS Bookworm, `aarch64`) as a service in the same `docker-compose` stack as Gitea, built `linux-arm64` from the `gitea/act_runner` image. Job execution is via the host Docker socket — the runner is a container and jobs spawn as sibling containers.

Container network mode is `host`, so job containers can reach Gitea on `localhost:3005` and resolve mDNS hostnames.

### Callee workflow cache

`act_runner` caches each resolved cross-repo callee at `/root/.cache/act/<owner>-<repo>@<ref>` inside the runner container, and **never refreshes it per run**. The key contains no commit SHA, so pushing to `master` does not invalidate it: edits to `publish-*.yml` — including the pin job's own commits — keep executing the stale cached copy indefinitely.

Two consequences. When a pipeline misbehaves after a workflow edit, suspect the cache before suspecting permissions. And after any `publish-*.yml` change, flush it:

```
docker compose up -d --no-deps --force-recreate act_runner
```

`--no-deps` is required: `act_runner` declares `depends_on: gitea`, so without it the recreate cascades and bounces the Gitea instance. Pass no `--pull`, or the flush silently adopts whatever `act_runner:latest` has become. The cache sits in the container layer while `.runner` is in the `/data` volume, so registration survives the recreate.

Changing a repo's owner also changes the cache key, which flushes it as a side effect.

### Job container image

All pipelines share one pre-built image, served from the Gitea registry at `pi5-16.local:3005/novoyuuparosk-wiki/novoyuuparosk-wiki-runner`. The root `Dockerfile` bakes in system deps (git, pandoc, ca-certificates) and every pipeline's Python packages, so job containers start with no install step.

[`build-image.yml`](.gitea/workflows/build-image.yml) rebuilds it on pushes touching the `Dockerfile`, any pipeline `requirements.txt`, or itself. It uses kaniko (daemonless, unprivileged) to push an immutable `:<short-sha>` and a moving `:latest`. A follow-up `pin` job rewrites the `image:` pin in each `publish-*.yml` to the new short SHA and commits it back to `master` using `FAPAT`, so the publish workflows always reference an immutable tag with no manual bump. That pin commit touches only `workflow_call` files, so it triggers no further runs — but it does need a cache flush to take effect.

The builder image `kaniko-act` lives under `mikkeli`, not the org, and **cannot be rebuilt by `build-image.yml`**: kaniko does not persist the empty dir from `mkdir -p -m 1777 /tmp`, so the resulting image has no `/tmp` and `wget -O /tmp/...` fails. Bumping it needs a host `docker build` and push, or a Dockerfile that forces the dir to materialise by writing a file inside it. [`build-kaniko-act.yml`](.gitea/workflows/build-kaniko-act.yml) is dispatch-only and runs in the old `mikkeli` copy.

## Gitea Actions setup (cross-cutting)

Secrets and variables are scoped to the `novoyuuparosk-wiki` org and inherited by every repo under it. A run belongs to the *caller* repo, so a source repo resolves secrets and variables from its own owner — which is why they live at org scope rather than on this repo.

**Secrets:**

| Name | Value / purpose |
|---|---|
| `WIKI_BOT_USER` | `Mikkeli@giteaBot` |
| `WIKI_BOT_PASSWORD` | the BotPassword from *Bot identity* above |
| `FAPAT` | Full-Access PAT owned by `mikkeli`, used by the publish workflows to clone repos at runtime. The PAT is personal; only its storage scope is the org — hence the `mikkeli:` basic-auth username in clone URLs |
| `PKGRW_PAT` | package read/write PAT owned by `mikkeli`, used by `build-image.yml` to push to the container registry |

**Variables:**

| Name | Value | Used for |
|---|---|---|
| `WIKI_BASE_URL` | `https://wiki.novoyuuparosk.org` | wiki base |
| `WIKI_API_URL` | `https://wiki.novoyuuparosk.org/api.php` | MediaWiki action API |
| `URL_TO_GITEA` | `http://localhost:3005` | **runner-internal** address, cloning only |
| `PUBLIC_URL_TO_GITEA` | `https://git.novoyuuparosk.org` | **public** address, page banner source links only |

Names avoid a `GITEA_`/`GITHUB_` prefix because Gitea blocks those, hence `URL_TO_GITEA` rather than `GITEA_URL`.

The two URL variables must stay separate. `URL_TO_GITEA` has to be an address the runner container can clone from, and host networking makes that a loopback. `PUBLIC_URL_TO_GITEA` is baked into the `{{Auto-generated}}` banner of every published page and has to resolve for a reader on the internet. `lib/wiki.public_source_base()` validates the banner value and aborts the run on an unset, relative, loopback, mDNS, or private-range address, so collapsing them back into one fails loudly instead of publishing dead links.

## Branch naming

- This repo: `automation/<pipeline-name>` for pipeline-development branches (e.g. `automation/songs`).
- Source repos: each pipeline's README defines its own convention (e.g. `autowiki/<song-slug>` in `ncmr-songs`).

## Tests

```
pip install -r requirements-dev.txt
python -m pytest tests -q
```

Pure string-in/string-out coverage of `lib/` and the pipelines' discovery, validation and publish-scoping logic — no wiki, no network. The `TestPandocContract` cases shell out to real pandoc and are skipped if it isn't installed; they pin the pandoc output shape the link resolver depends on, so a pandoc upgrade fails a test instead of publishing broken links. [`test.yml`](.gitea/workflows/test.yml) runs the suite on push in the same image the pipelines publish from.