diff --git a/docs/README.md b/docs/README.md index b7218e9..385aa9b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -196,10 +196,31 @@ arm64-host `gen_snapshot` ([flutter#189724](https://github.com/flutter/flutter/i An x86_64 runner is a requirement, not a preference. The runner is **host mode**: jobs use the machine's own Flutter, JDK and Android -SDK/NDK. That means CI builds with the same toolchain that produced the verified -release, and no NDK download per run. The trade is isolation — workflows run as -that user on that machine — so the runner is only for repos whose workflows are -trusted. +SDK/NDK. CI therefore builds with the same toolchain that produced the verified +release, and no NDK is downloaded per run. The cost is that there is no +container boundary, which is what decides the next paragraph. + +Jobs run as a dedicated unprivileged account, `gitea-runner`, in the +`flutterdev` group. In host mode the service account *is* the sandbox — there +is no other — and a workflow file is code that runs as whoever the service runs +as. A human account would hand every workflow that user's home directory and +SSH keys; root would be worse and would also miss the warm caches in that home, +re-downloading dependencies every run. The dedicated account keeps its own +Gradle and pub caches under `/var/lib/gitea-runner`, so the first Android build +is slow by design and later ones are not. + +`flutterdev` is what grants write access to `/opt/flutter` and +`/opt/android-sdk`; both carry ACLs with `group:flutterdev:rwx` and matching +`default:` entries, so new files inherit it. The setup script proves that access +as the service account rather than assuming group membership is sufficient. + +It installs a **system** service, not a user service: root-managed lifecycle, +starts at boot, no lingering session. That the service is root-managed is +unrelated to the jobs being unprivileged — `User=` in the unit settles the +second, and conflating the two is easy to do. + +`sudo ./tool/setup_ci_runner.sh --check` validates this machine's toolchain +without creating anything. Two jobs: diff --git a/tool/setup_ci_runner.sh b/tool/setup_ci_runner.sh index aefd9a4..382aa49 100755 --- a/tool/setup_ci_runner.sh +++ b/tool/setup_ci_runner.sh @@ -1,82 +1,160 @@ #!/usr/bin/env bash -# Register this machine as a Gitea Actions runner for the android-stuff org. +# Provision this machine as a Gitea Actions runner for the android-stuff org. # -# WHY THIS MACHINE, AND NOT THE GITEA HOST -# Gitea runs on pi5-16 (aarch64). Flutter cannot build an Android APK on a -# Linux arm64 host at all — the SDK ships no arm64-host gen_snapshot -# (flutter/flutter#189724, #75864, open as of July 2026). So an Android build -# runner has to be x86_64 regardless of how the Gitea host is configured. +# sudo ./tool/setup_ci_runner.sh --check # pre-flight only, changes nothing +# sudo RUNNER_TOKEN=xxxx ./tool/setup_ci_runner.sh # -# WHY HOST MODE, AND NOT A CONTAINER -# This machine already has a proven Flutter + Android SDK + NDK toolchain. In -# host mode the job uses it directly: no SDK download, no ~1GB NDK pull per -# run, and CI builds with the exact toolchain that produced the verified -# release. The trade is isolation — a workflow in this org runs commands as -# this user on this machine. That is the normal self-hosted-runner bargain, but -# it is a real one: only grant it for repos whose workflows you trust. +# Get an ORG-scoped token from +# /org/android-stuff/settings/actions/runners → "Create new runner" +# Org scope, not repo scope, so every android-stuff repo shares one runner. # -# USAGE -# 1. Get an ORG-scoped registration token: -# http://pi5-16.local:3005/org/android-stuff/settings/actions/runners -# → "Create new runner" → copy the token -# Org scope, not repo scope, so every android-stuff repo can use it. -# 2. RUNNER_TOKEN=xxxx ./tool/setup_ci_runner.sh +# ── WHY THIS MACHINE, NOT THE GITEA HOST ──────────────────────────────────── +# Gitea runs on an aarch64 box. Flutter cannot build an Android APK on a Linux +# arm64 host at all: the SDK ships no arm64-host gen_snapshot +# (flutter/flutter#189724, #75864, open as of July 2026). An x86_64 runner is a +# requirement here, not a preference. # -# Re-running is safe: it skips the download if the binary is current and -# refuses to clobber an existing registration. +# ── WHY HOST MODE, NOT A CONTAINER ────────────────────────────────────────── +# This machine has a working Flutter + Android SDK + NDK. In host mode jobs use +# it directly, so CI builds with the same toolchain that produced the verified +# release and skips a per-run NDK download. The cost is that there is no +# container boundary — which is exactly why the service account below matters. +# +# ── WHY A DEDICATED USER, AND A SYSTEM SERVICE ────────────────────────────── +# Host mode means the service account IS the sandbox; there is no other. A +# workflow file is code, and it runs as whoever this service runs as. Running +# as a human user would give every workflow read access to that user's home, +# SSH keys included. Running as root would be worse, and would also miss the +# warm Gradle/pub caches in that home (HOME=/root), re-downloading dependencies +# every run for no benefit. +# +# So: a dedicated unprivileged account, in the flutterdev group for SDK access, +# with its own HOME so its caches are its own. A *system* service rather than a +# user service, so it starts at boot under plain systemctl with no lingering +# session — the service being root-managed is unrelated to the jobs being +# unprivileged, and conflating those two was the earlier mistake here. set -euo pipefail VERSION="${RUNNER_VERSION:-3.3.0}" INSTANCE="${GITEA_INSTANCE:-http://pi5-16.local:3005}" +RUNNER_USER="${RUNNER_USER:-gitea-runner}" +SDK_GROUP="${SDK_GROUP:-flutterdev}" +RUNNER_HOME="${RUNNER_HOME:-/var/lib/gitea-runner}" +BIN="${RUNNER_BIN:-/usr/local/bin/gitea-runner}" +UNIT="/etc/systemd/system/gitea-runner.service" RUNNER_NAME="${RUNNER_NAME:-$(hostname)-android}" -RUNNER_HOME="${RUNNER_HOME:-$HOME/.local/share/gitea-runner}" -BIN="${RUNNER_BIN:-$HOME/.local/bin/gitea-runner}" -# Capability-based label, deliberately not host-based: it says what the runner -# can do, so the workflow does not encode which machine happens to provide it. +# Capability label, deliberately not a hostname: it says what the runner can +# do, so a workflow does not encode which machine happens to provide it. LABEL="${RUNNER_LABEL:-flutter-android}" -: "${RUNNER_TOKEN:?set RUNNER_TOKEN to an org registration token (see header)}" +CHECK_ONLY=0 +[[ "${1:-}" == "--check" ]] && CHECK_ONLY=1 -echo "==> Checking the toolchain this runner promises to provide" +[[ $EUID -eq 0 ]] || { echo "Run with sudo: sudo RUNNER_TOKEN=... $0" >&2; exit 1; } +# --check is for validating this machine before committing to anything, so it +# deliberately does not require a token. +if [[ $CHECK_ONLY -eq 0 ]]; then + : "${RUNNER_TOKEN:?set RUNNER_TOKEN to an org registration token (see header)}" +fi + +say() { printf '\n==> %s\n' "$1"; } + +# ── Pre-flight, before anything is created ───────────────────────────────── +say "Pre-flight" missing=0 -for cmd in flutter dart java cmake ninja node git unzip; do - if command -v "$cmd" >/dev/null; then - printf ' ok %s\n' "$cmd" +for cmd in flutter dart java cmake ninja node git unzip curl; do + # Resolved from systemd's own PATH, not an interactive shell's, since that + # is what the service will actually have. + if PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \ + command -v "$cmd" >/dev/null; then + printf ' ok %s\n' "$cmd" else - printf ' MISSING %s\n' "$cmd" + printf ' MISSING %s (not on systemd default PATH)\n' "$cmd" missing=1 fi done -SDK="${ANDROID_HOME:-${ANDROID_SDK_ROOT:-}}" -if [[ -n "$SDK" && -d "$SDK/ndk" && -d "$SDK/build-tools" ]]; then - echo " ok Android SDK at $SDK (ndk: $(ls "$SDK/ndk" | tr '\n' ' '))" -else - echo " MISSING Android SDK with ndk/ and build-tools/ (ANDROID_HOME=${SDK:-unset})" - missing=1 -fi + +SDK="${ANDROID_HOME:-${ANDROID_SDK_ROOT:-/opt/android-sdk}}" +for d in "$SDK/ndk" "$SDK/build-tools" "$SDK/platforms"; do + [[ -d "$d" ]] && printf ' ok %s\n' "$d" \ + || { printf ' MISSING %s\n' "$d"; missing=1; } +done +getent group "$SDK_GROUP" >/dev/null \ + && printf ' ok group %s\n' "$SDK_GROUP" \ + || { printf ' MISSING group %s\n' "$SDK_GROUP"; missing=1; } + if [[ $missing -ne 0 ]]; then echo echo "Refusing to register: a runner advertising '$LABEL' must actually" >&2 - echo "provide these, or every job will fail confusingly instead of here." >&2 + echo "provide these, or every job fails confusingly instead of here." >&2 exit 1 fi -echo "==> Installing gitea-runner $VERSION" -mkdir -p "$(dirname "$BIN")" "$RUNNER_HOME" +if [[ $CHECK_ONLY -eq 1 ]]; then + say "Pre-flight passed" + echo " Nothing was created. Re-run with RUNNER_TOKEN set to provision." + echo " Note: SDK write access is verified as $RUNNER_USER, which can only" + echo " be checked once that account exists — so it is not covered here." + exit 0 +fi + +# ── Service account ──────────────────────────────────────────────────────── +say "Service account: $RUNNER_USER" +if id "$RUNNER_USER" &>/dev/null; then + echo " exists; ensuring membership of $SDK_GROUP" + usermod -aG "$SDK_GROUP" "$RUNNER_USER" +else + # --shell /bin/bash, not nologin: host-mode steps are executed as shell + # scripts, and a login shell costs nothing here — the account has no + # password, so it is only reachable via root anyway. + useradd --system \ + --create-home --home-dir "$RUNNER_HOME" \ + --shell /bin/bash \ + --groups "$SDK_GROUP" \ + --comment "Gitea Actions runner ($LABEL)" \ + "$RUNNER_USER" + echo " created" +fi +install -d -o "$RUNNER_USER" -g "$SDK_GROUP" -m 0750 \ + "$RUNNER_HOME" "$RUNNER_HOME/workspace" +id "$RUNNER_USER" + +# Group membership is worthless if the ACLs disagree, so prove it rather than +# assume it — as the service user, against the paths a build actually writes. +say "Verifying $RUNNER_USER can write the SDKs" +acl_ok=1 +for d in "/opt/flutter/bin/cache" "$SDK"; do + if sudo -u "$RUNNER_USER" test -w "$d"; then + printf ' ok writable: %s\n' "$d" + else + printf ' FAIL not writable: %s\n' "$d" + acl_ok=0 + fi +done +if [[ $acl_ok -ne 1 ]]; then + echo + echo "$RUNNER_USER is in $SDK_GROUP but still cannot write. Check the ACLs:" >&2 + echo " getfacl /opt/flutter /opt/android-sdk" >&2 + exit 1 +fi + +# ── Binary ───────────────────────────────────────────────────────────────── +say "Installing gitea-runner $VERSION" if [[ -x "$BIN" ]] && "$BIN" --version 2>/dev/null | grep -qF "$VERSION"; then echo " already at $VERSION" else url="https://dl.gitea.com/gitea-runner/${VERSION}/gitea-runner-${VERSION}-linux-amd64" - echo " downloading $url" + echo " $url" curl -fSL --retry 3 -o "$BIN.tmp" "$url" - chmod +x "$BIN.tmp" + chmod 0755 "$BIN.tmp" mv "$BIN.tmp" "$BIN" fi "$BIN" --version -echo "==> Writing config" +# ── Config ───────────────────────────────────────────────────────────────── +say "Config" CONFIG="$RUNNER_HOME/config.yaml" if [[ -f "$CONFIG" ]]; then echo " $CONFIG exists, leaving it alone" @@ -87,74 +165,103 @@ log: runner: file: .runner - # One Android build at a time: gradle + a 2-ABI NDK compile will happily - # saturate this box, and concurrent runs would just contend. + # One at a time: a 2-ABI NDK compile plus Gradle will saturate this box, and + # concurrent jobs would only contend for it. capacity: 1 timeout: 45m - # ':host' runs the job directly on this machine using the toolchain above, - # rather than in a container that would have to fetch its own. + # ':host' runs steps directly on this machine as ${RUNNER_USER}, using the + # toolchain verified in pre-flight, rather than in a container that would + # have to fetch its own. labels: - "${LABEL}:host" container: - # Unused in host mode, kept so switching to container mode later is a - # one-line change rather than a rediscovery. + # Unused in host mode. Kept so switching to container mode later is an edit + # rather than a rediscovery — it would need the Docker network that the + # Gitea host's own runner also needs. network: "" privileged: false host: workdir_parent: ${RUNNER_HOME}/workspace YAML + chown "$RUNNER_USER:$SDK_GROUP" "$CONFIG" + chmod 0640 "$CONFIG" echo " wrote $CONFIG" fi -echo "==> Registering with $INSTANCE" +# ── Registration ─────────────────────────────────────────────────────────── +say "Registering with $INSTANCE" if [[ -f "$RUNNER_HOME/.runner" ]]; then - echo " already registered (found $RUNNER_HOME/.runner) — skipping." - echo " To re-register: remove that file and the runner in the Gitea UI first." + echo " already registered ($RUNNER_HOME/.runner) — skipping." + echo " To re-register: delete that file AND remove the runner in Gitea." else - (cd "$RUNNER_HOME" && "$BIN" register \ - --no-interactive \ - --instance "$INSTANCE" \ - --token "$RUNNER_TOKEN" \ - --name "$RUNNER_NAME" \ - --labels "${LABEL}:host" \ - --config "$CONFIG") + sudo -u "$RUNNER_USER" HOME="$RUNNER_HOME" \ + "$BIN" register \ + --no-interactive \ + --instance "$INSTANCE" \ + --token "$RUNNER_TOKEN" \ + --name "$RUNNER_NAME" \ + --labels "${LABEL}:host" \ + --config "$CONFIG" + # register writes .runner into the cwd it was given; make sure it landed. + [[ -f "$RUNNER_HOME/.runner" ]] || { + echo " registration produced no .runner in $RUNNER_HOME" >&2 + exit 1 + } fi -cat < Done. Start it with: - - cd $RUNNER_HOME && $BIN daemon --config $CONFIG - - Or install a user service that survives logout: - - systemctl --user enable --now gitea-runner - loginctl enable-linger $USER # so it runs without an active session - - A unit file is written alongside this script's output at: - $RUNNER_HOME/gitea-runner.service - Copy it to ~/.config/systemd/user/ and 'systemctl --user daemon-reload'. -EOF - -cat > "$RUNNER_HOME/gitea-runner.service" < "$UNIT" < Done. + + Logs: journalctl -u gitea-runner -f + Stop: systemctl stop gitea-runner + Remove: systemctl disable --now gitea-runner + rm $UNIT $BIN && rm -rf $RUNNER_HOME + userdel $RUNNER_USER + ...and delete the runner in the Gitea org settings. + + The runner should now appear under + ${INSTANCE}/org/android-stuff/settings/actions/runners + with label '$LABEL'. The queued run for this repo should start on its own. + + First Android build will be slow: this account's Gradle and pub caches + start empty by design. Subsequent runs reuse them. +EOF