diff --git a/.gitea/workflows/test.yml b/.gitea/workflows/test.yml index 3ee7dc6..bf9bc43 100644 --- a/.gitea/workflows/test.yml +++ b/.gitea/workflows/test.yml @@ -13,6 +13,8 @@ on: push: branches: [master] pull_request: + # So a new runner can be proved out from the UI without inventing a commit. + workflow_dispatch: env: DEBIAN_FRONTEND: noninteractive diff --git a/Makefile b/Makefile index ce706c6..29760c2 100644 --- a/Makefile +++ b/Makefile @@ -73,7 +73,7 @@ android-sdk: ## Install the Android SDK packages this app builds against android-release: ## Build the release APK (set KADENCE_KEYSTORE* to sign it) cd android && ./gradlew :app:assembleRelease -ci-runner: ## Start the Gitea Actions runner (needs GITEA_RUNNER_REGISTRATION_TOKEN) +ci-runner: ## Start the Gitea Actions runner here (see docs/gitea-runner.md) docker compose -f docker-compose.ci.yml up -d report: ## Build and send today's report right now diff --git a/README.md b/README.md index f9e76e6..35afb37 100644 --- a/README.md +++ b/README.md @@ -275,20 +275,25 @@ Gitea Actions, in `.gitea/workflows/`: | `docker.yml` | push to master, `v*` tags | builds the image, starts it against SQLite and waits for `/api/health`, then pushes `git.timka.su/timka123/kadence:latest` and `:` | | `android-release.yml` | push to master, `v*` tags | `:app`'s suite (81, on the JVM), then the release APK — signed from secrets, with `apksigner verify` printing the certificate it actually used | -**Nothing runs until a runner is registered.** `docker-compose.ci.yml` is the whole -of that, and its header is the setup checklist: a registration token from -*Settings → Actions → Runners*, then `make ci-runner`. Gitea itself needs -`[actions] ENABLED = true` and `DEFAULT_ACTIONS_URL = github` in `app.ini`, the -second so `actions/checkout@v4` can be resolved. +**Nothing runs until a docker-mode runner is registered** — +[`docs/gitea-runner.md`](docs/gitea-runner.md) is the guide, and +`docker-compose.ci.yml` + `deploy/act-runner.yaml` are the two files it installs. +Short version, on the runner host: -Two details decide whether the jobs work at all: +```bash +mkdir -p /opt/gitea-runner && cd /opt/gitea-runner +cp …/docker-compose.ci.yml docker-compose.yml && cp …/deploy/act-runner.yaml config.yaml +printf 'GITEA_RUNNER_REGISTRATION_TOKEN=%s\n' "" > .env && chmod 600 .env +docker compose up -d +``` -* **The label map.** `runs-on: ubuntu-latest` is pinned to - `catthehacker/ubuntu:act-24.04` by the runner's own config, because the jobs need - two things from the image: a `node` (Gitea's javascript actions run *inside* the - job container) and Python 3.12, which the backend's `requires-python` insists on. -* **The docker socket**, mounted into job containers so the image can be built. It - needs both the mount and `valid_volumes` naming it — see `deploy/act-runner.yaml`. +`runs-on: ubuntu-latest` means what it means on GitHub: a disposable container, +here `catthehacker/ubuntu:act-24.04` — chosen because the jobs need a `node` in the +image (Gitea's javascript actions run *inside* the job container) and Python 3.12, +which the backend's `requires-python` insists on. A **host-mode** runner answering +the same label is the one thing that breaks all of this: the job then runs on the +machine as the runner's user, where `apt-get install` either fails or installs +packages on your server. The guide covers removing one. Secrets, all under *Repository → Settings → Actions → Secrets*: diff --git a/deploy/act-runner.yaml b/deploy/act-runner.yaml index 9ae0628..e9224f9 100644 --- a/deploy/act-runner.yaml +++ b/deploy/act-runner.yaml @@ -1,26 +1,48 @@ -# act_runner's own configuration. See docker-compose.ci.yml for what it is for. +# act_runner's own configuration, mounted into the container as +# /config/config.yaml. See docker-compose.ci.yml and docs/gitea-runner.md. log: level: info runner: + # Two jobs at a time. The test workflow has four, so a push runs them in two + # waves on an 8-core box — raise it if the box is idle anyway. capacity: 2 timeout: 1h - # Jobs pull their own tooling; nothing here needs to be fetched per job. - fetch_timeout: 10s - fetch_interval: 5s + shutdown_timeout: 3h + insecure: false + fetch_timeout: 5s + fetch_interval: 2s + # Deliberately empty: the labels come from GITEA_RUNNER_LABELS at registration + # and live in .runner afterwards. Setting them in both places is how they end + # up disagreeing. + labels: [] cache: - # Gitea's built-in cache server is what makes actions/cache@v4 work, and the - # Android SDK download is worth caching: it is most of that job's wall clock. + # What makes `actions/cache@v4` work — the Android SDK download and the gradle + # caches are most of the APK job's wall clock. The server is run by the runner + # itself, inside the container, and reached by job containers over the shared + # `gitea-runner` network, so there is no port to publish or firewall. enabled: true + dir: "" + host: "" + port: 0 container: - network: bridge + # The same network the runner is on. Not `bridge`: on the default bridge the + # job containers cannot reach the runner's cache server. + network: gitea-runner privileged: false - # The host's docker daemon, for the image build. Both lines are needed: the - # mount itself, and the allow-list that permits it. - options: -v /var/run/docker.sock:/var/run/docker.sock + options: + workdir_parent: workspace + # Mount the host's docker socket into job containers that ask for docker, so + # `docker.yml` can build and push. `valid_volumes` is the allow-list that + # makes any such mount legal at all. + docker_host: automount valid_volumes: - /var/run/docker.sock - # Let the job's own `docker` CLI find it. - docker_host: unix:///var/run/docker.sock + force_pull: false + force_rebuild: false + +host: + # Only used by `:host` labels, which this runner deliberately does not offer. + workdir_parent: /tmp/act_runner diff --git a/docker-compose.ci.yml b/docker-compose.ci.yml index a5f704e..8568450 100644 --- a/docker-compose.ci.yml +++ b/docker-compose.ci.yml @@ -1,48 +1,72 @@ -# The Gitea Actions runner, for a single-machine Gitea. +# The Gitea Actions runner, in Docker. # -# Nothing in .gitea/workflows runs until a runner is registered, and this is the -# whole of it. Two settings do the work: +# This is the runner that executes everything in `.gitea/workflows`. It claims +# the label **`ubuntu-latest`** and maps it to a container image, which is what +# the workflows expect: GitHub's own name, GitHub's own semantics — the job runs +# in a disposable container, not on the machine. # -# * the **label map** decides what `runs-on: ubuntu-latest` means. It is pinned -# to catthehacker/ubuntu:act-24.04 because the workflows need two things from -# the image: a node (Gitea's javascript actions — checkout, cache, -# upload-artifact — run *inside* the job container) and Python 3.12, which -# the backend's `requires-python` insists on. Ubuntu 24.04 ships 3.12. -# * the **docker socket**, mounted into job containers so the `docker` workflow -# can build and push an image. That is a real privilege handed to CI; it is -# also the normal arrangement when Gitea and its runner are the same box as -# the deployment. +# It replaced a *host*-mode runner on the same box (`ubuntu-latest:host`), where +# every job ran directly on the server as the `act_runner` user. That is why the +# workflows' `apt-get install` steps failed: nothing could be installed, and had +# it succeeded it would have been installing packages on the host. # -# Setting it up: +# Setting it up, and what each piece is for, is in +# [`docs/gitea-runner.md`](docs/gitea-runner.md). The short version: # -# 1. Gitea → Settings → Actions → Runners → "Create new runner", copy the token. -# 2. GITEA_RUNNER_REGISTRATION_TOKEN= docker compose -f docker-compose.ci.yml up -d -# 3. Repository → Settings → Actions → Secrets, add: -# REGISTRY_USER, REGISTRY_TOKEN — a Gitea user and token with -# package:write, for the registry -# KADENCE_KEYSTORE_BASE64 — base64 -w0 of your keystore -# KADENCE_KEYSTORE_PASSWORD, KADENCE_KEY_ALIAS, KADENCE_KEY_PASSWORD -# Without the keystore secrets the APK job still runs and produces an -# unsigned-for-release (debug-signed) artifact rather than failing. +# mkdir -p /opt/gitea-runner && cd /opt/gitea-runner +# # copy this file as docker-compose.yml and deploy/act-runner.yaml as config.yaml +# printf 'GITEA_RUNNER_REGISTRATION_TOKEN=%s\n' "" > .env +# chmod 600 .env +# docker compose up -d # -# Gitea itself needs `[actions] ENABLED = true` in app.ini, and -# `DEFAULT_ACTIONS_URL = github` so `actions/checkout@v4` can be resolved. +# The registration token is needed **once**: after the first start the runner has +# its own permanent token in `./data/.runner` and the one in `.env` is dead. services: runner: - image: gitea/act_runner:0.2.11 + image: gitea/act_runner:0.3.1 + container_name: gitea-runner restart: unless-stopped + environment: GITEA_INSTANCE_URL: ${GITEA_INSTANCE_URL:-https://git.timka.su} - GITEA_RUNNER_REGISTRATION_TOKEN: ${GITEA_RUNNER_REGISTRATION_TOKEN:?set this to the token from Gitea} - GITEA_RUNNER_NAME: ${GITEA_RUNNER_NAME:-kadence-runner} - GITEA_RUNNER_LABELS: ubuntu-latest:docker://catthehacker/ubuntu:act-24.04 - # Mounted into every job container, which is what lets the docker workflow - # build an image. `valid_volumes` has to allow it explicitly. + # Only read at first registration; harmless (and ignored) afterwards. + GITEA_RUNNER_REGISTRATION_TOKEN: ${GITEA_RUNNER_REGISTRATION_TOKEN:-} + GITEA_RUNNER_NAME: ${GITEA_RUNNER_NAME:-dockerized} + # The label map: what `runs-on` may ask for, and what it gets. + # + # `act-24.04` and not something slimmer because the jobs need two things + # from the image: a **node**, since Gitea's javascript actions (checkout, + # cache, upload-artifact) run *inside* the job container, and **Python + # 3.12**, which the backend's `requires-python` insists on and Ubuntu + # 24.04 ships. A plain `ubuntu:24.04` has neither and every job would + # fail at the first `uses:`. + # + # Labels are baked into `./data/.runner` at registration — changing this + # line later does nothing on its own. See the guide. + GITEA_RUNNER_LABELS: >- + ubuntu-latest:docker://catthehacker/ubuntu:act-24.04, + ubuntu-24.04:docker://catthehacker/ubuntu:act-24.04, + ubuntu-22.04:docker://catthehacker/ubuntu:act-22.04 CONFIG_FILE: /config/config.yaml + volumes: - - runner-data:/data - - ./deploy/act-runner.yaml:/config/config.yaml:ro + - ./config.yaml:/config/config.yaml:ro + # The registration lives here. Delete `./data/.runner` and the runner + # re-registers as a *new* one, leaving a stale entry in Gitea. + - ./data:/data + # The host's docker daemon, so a job can build and push an image. This is + # a real privilege — root on the host, effectively — and the reason the + # runner should not be registered at instance level on a box you do not + # want CI to own. `config.yaml` passes it through to job containers. - /var/run/docker.sock:/var/run/docker.sock -volumes: - runner-data: + networks: + - runner + +networks: + # Job containers join this same network (`container.network` in config.yaml), + # which is what makes the runner's built-in cache server reachable from them: + # it advertises its own address, and on a separate network that address would + # route nowhere. It is also why nothing here needs a published port. + runner: + name: gitea-runner diff --git a/docs/gitea-runner.md b/docs/gitea-runner.md new file mode 100644 index 0000000..79e2fbb --- /dev/null +++ b/docs/gitea-runner.md @@ -0,0 +1,187 @@ +# A dockerized Gitea Actions runner + +Everything in `.gitea/workflows` needs a runner, and it has to be a **docker-mode** +one. This is how to set one up, what each setting is for, and what goes wrong if +it is set up the other way. + +* [Why docker mode and not host mode](#why-docker-mode-and-not-host-mode) +* [Prerequisites](#prerequisites) +* [Install](#install) +* [Removing a host-mode runner](#removing-a-host-mode-runner) +* [Verifying it](#verifying-it) +* [Changing the labels later](#changing-the-labels-later) +* [What can go wrong](#what-can-go-wrong) + +--- + +## Why docker mode and not host mode + +A Gitea runner advertises **labels**, and each label says *where* a job runs: + +| Label | Means | +| --- | --- | +| `ubuntu-latest:docker://catthehacker/ubuntu:act-24.04` | run the job in a fresh container from that image | +| `ubuntu-latest:host` | run the job **directly on the machine**, as the runner's own user | + +A workflow only says `runs-on: ubuntu-latest`. It cannot tell which of those it +got, and the two behave nothing alike: + +* **Host mode has no job isolation and no job toolchain.** `apt-get install` either + fails (the runner user is not root — the common case) or succeeds, and then you + have CI installing packages on a server. `actions/checkout` clones into a temp + directory that persists between runs, so one job's leftovers are the next job's + surprise. Every workflow written against GitHub's runners assumes a clean + container and does not get one. +* **Docker mode is what `ubuntu-latest` means everywhere else.** A disposable + container per job, the tools the image ships, and nothing left behind. + +Host mode is for the narrow case where a job *must* touch the machine — a deploy +script using that host's ssh keys, say. Everything else wants docker mode. + +**One label, one meaning.** If two runners both claim `ubuntu-latest`, jobs land on +whichever answers first, and a workflow that passes on one fails on the other for +reasons nobody can see from the logs. Pick one, or give the special-purpose runner +a label of its own (`deploy:host`) and leave `ubuntu-latest` to the docker one. + +## Prerequisites + +* Docker and the compose plugin on the host (`docker --version`, `docker compose version`). +* Gitea with Actions on. In `app.ini`: + + ```ini + [actions] + ENABLED = true + ; so `uses: actions/checkout@v4` can be resolved from GitHub + DEFAULT_ACTIONS_URL = github + ``` + +* Disk. The job image is ~1–2 GB; the Android SDK cache another ~3 GB, and gradle + caches grow. Budget 20 GB and it will not surprise you. +* A **registration token**, from whichever scope the runner should serve: + + | Scope | Where | Serves | + | --- | --- | --- | + | Repository | repo → *Settings → Actions → Runners → Create new runner* | that repo only | + | Organisation | org → *Settings → Actions → Runners* | every repo in the org | + | Instance | *Site Administration → Actions → Runners* | everything on the instance | + + Prefer the narrowest scope that covers what you need — the runner's docker socket + is effectively root on its host, and an instance-level runner hands that to every + repository on the server. + +## Install + +Two files, both in this repo: `docker-compose.ci.yml` (the service) and +`deploy/act-runner.yaml` (act_runner's own config). On the runner host: + +```bash +mkdir -p /opt/gitea-runner && cd /opt/gitea-runner + +# From a checkout, or scp them across: +cp /path/to/kadence/docker-compose.ci.yml docker-compose.yml +cp /path/to/kadence/deploy/act-runner.yaml config.yaml + +# The token is needed once, at registration. Keep it out of the compose file. +printf 'GITEA_INSTANCE_URL=https://git.timka.su\n' > .env +printf 'GITEA_RUNNER_REGISTRATION_TOKEN=%s\n' "" >> .env +printf 'GITEA_RUNNER_NAME=%s\n' "$(hostname)-docker" >> .env +chmod 600 .env + +docker compose up -d +docker compose logs -f # expect: "runner registered successfully" +``` + +That is it. The runner appears in Gitea under the scope you took the token from, +with its labels listed beside it. + +### What the pieces do + +* **`GITEA_RUNNER_LABELS`** is the label map, and it is read **only at first + registration** — see [Changing the labels later](#changing-the-labels-later). +* **`./data`** holds `.runner`: the runner's identity and its permanent token. + Back it up or do not delete it; removing it makes the runner register again as a + *new* one and leaves a dead entry behind in Gitea. +* **`/var/run/docker.sock`** is mounted so a job can build images. With + `docker_host: automount` in `config.yaml`, act_runner passes it into the job + container too, which is what lets `docker build` work inside a workflow. Treat it + as what it is: any workflow on this runner can do anything to this host's docker. +* **The shared `gitea-runner` network.** Job containers join the runner's own + network (`container.network`), because act_runner's built-in cache server — the + thing that makes `actions/cache@v4` work — advertises its own address, and from + the default bridge that address routes nowhere. Symptom if you get it wrong: + cache steps hang, then time out. +* **`catthehacker/ubuntu:act-24.04`** rather than `ubuntu:24.04`, because Gitea's + javascript actions (`checkout`, `cache`, `upload-artifact`) execute **inside the + job container** and need a `node` there. This image also ships Python 3.12, which + Kadence's backend requires. + +## Removing a host-mode runner + +If the box already has one (the usual sign: jobs fail at `apt-get` with permission +errors, or succeed and change the host): + +```bash +systemctl disable --now act_runner.service +mv /var/lib/act_runner/.runner /var/lib/act_runner/.runner.disabled # reversible +rm -f /etc/systemd/system/act_runner.service && systemctl daemon-reload +rm -f /usr/local/bin/act_runner +``` + +Then delete its (now offline) entry in Gitea, or it sits there looking available. + +**Check what else used it first.** A runner is shared: look at its recent tasks +(`journalctl -u act_runner | grep "task .* repo is"`) and at the other repos' +workflows. Jobs that ran on the host because they *needed* the host — deploying +with that machine's ssh keys, writing to a host path — will not work in a +container as they are. The docker socket is mounted, so `docker compose` steps +generally still do. + +## Verifying it + +```bash +docker compose logs --tail 20 # "runner registered" / "daemon started" +``` + +In Gitea: the runner is listed, online, with its labels. Then run something — a +push, or the *Run workflow* button if the workflow has `workflow_dispatch`. The +first job pulls the image, so give it a few minutes; later ones start in seconds. + +Job logs should show the job running *as root inside a container*: + +``` +whoami -> root +cat /etc/os-release -> Ubuntu 24.04 +``` + +If it says your runner's user and the host's distribution, the job landed on a +host-mode runner and the label map is wrong. + +## Changing the labels later + +Labels live in `./data/.runner` after registration; the environment variable is +only read when that file does not exist. To change them, re-register: + +```bash +cd /opt/gitea-runner +docker compose down +rm data/.runner # the old registration goes stale in Gitea +# get a fresh token, put it in .env, adjust GITEA_RUNNER_LABELS +docker compose up -d +``` + +Then delete the stale runner in Gitea. (Editing `.runner`'s `labels` array by hand +works too, but Gitea keeps its own copy, so the UI and the runner can disagree — +re-registering is the honest route.) + +## What can go wrong + +| Symptom | Cause | +| --- | --- | +| Jobs stay queued forever | No runner claims that `runs-on` label, or the runner is registered at a scope that does not include this repo | +| `Cannot find: node in PATH` | A job image without node — use a `catthehacker/*` image, not bare `ubuntu` | +| `apt-get: Permission denied` | The job is on a **host**-mode runner | +| `uses: actions/checkout@v4` fails to resolve | `DEFAULT_ACTIONS_URL` is not `github` | +| Cache steps hang | Job containers are not on the runner's network — see `container.network` | +| `docker: command not found` in a job | That image has no docker CLI; use the `catthehacker` one, which does | +| `Cannot connect to the Docker daemon` in a job | `docker_host: automount` missing from `config.yaml`, or the socket is not mounted into the runner | +| Everything is slow on the first run | It is pulling the job image. It is cached afterwards |