diff --git a/.gitignore b/.gitignore index 4c49bd7..ba509d4 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ .env +runner-config.yml diff --git a/Caddyfile b/Caddyfile index a8c3617..be97de4 100644 --- a/Caddyfile +++ b/Caddyfile @@ -28,3 +28,4 @@ n8n.{$DOMAIN} { vpn.{$DOMAIN} { reverse_proxy wg-easy:51821 } + diff --git a/HARDENING.md b/HARDENING.md index 5636eb4..f2b3358 100644 --- a/HARDENING.md +++ b/HARDENING.md @@ -92,3 +92,35 @@ Plus: enable your provider's server snapshots. Two clicks, saves your company. Not covered here, on purpose (own episodes): SSO everywhere, 2FA enforcement per service, Netbird zero-trust mesh, central logging, CrowdSec, offsite borg. Baseline first. Perfect later. + +## CI: the two ways to hand your server away + +Both of these look like shortcuts and both give anyone who can push a branch full +control of the machine. + +**1. Mounting the host Docker socket into the runner.** Every forum answer suggests +it, because it works instantly: + +```yaml +volumes: + - /var/run/docker.sock:/var/run/docker.sock # do not do this +``` + +A job can then start a privileged container on the host, mount `/` and read every +volume on the box, including the password manager sitting in the next container. +Use the docker-in-docker sidecar in `compose.yml` instead. It costs six lines and +the jobs talk to a daemon that has nothing of yours in it. + +**2. An unrestricted SSH key for the deploy step.** If the key in your CI secrets +can run any command, the pipeline is a root shell with extra steps. Lock it to one +command in `authorized_keys`: + +``` +command="/usr/local/bin/deploy-myapp",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-ed25519 AAAA... +``` + +The script receives the argument from the job in `$SSH_ORIGINAL_COMMAND`. Sanitise +it — the example in `examples/deploy-hello-ops.sh` strips everything that is not a +hex character, because it only ever expects an image tag. The deploy user needs +Docker access, so the forced command is the thing containing it; without the forced +command, membership in the `docker` group is equivalent to root. diff --git a/README.md b/README.md index d81db3b..be145e7 100644 --- a/README.md +++ b/README.md @@ -90,6 +90,56 @@ Outline → its postgres/redis. **Is `latest` okay for images?** For a homelab: pragmatic. For production: pin versions and read HARDENING.md. +## CI: your own runners (Episode 2) + +Forgejo ships Actions. Turn it on, give it a runner, and you have GitHub Actions +with no minute counter. + +```bash +# 1. Enable it (already set in compose.yml) +# FORGEJO__actions__ENABLED and repo.actions in DEFAULT_REPO_UNITS. +# Without repo.actions in the default units, the Actions tab stays hidden +# and it looks like the feature is broken. Existing repos: flip it per repo. + +# 2. One shared secret for declarative registration +SECRET=$(openssl rand -hex 20) +UUID=$(python3 -c 'import uuid,sys;print(uuid.UUID(bytes=sys.argv[1][:16].encode()))' "$SECRET") +printf 'RUNNER_SECRET=%s\nRUNNER_UUID=%s\n' "$SECRET" "$UUID" >> .env + +# 3. Tell Forgejo about it +docker compose exec -u git forgejo forgejo forgejo-cli actions register \ + --name box-runner --labels docker --secret "$SECRET" + +# 4. Fill in the runner config and start the CI profile +cp runner-config.example.yml runner-config.yml +sed -i "s|https://git.example.com|https://git.$DOMAIN|; s|\${RUNNER_UUID}|$UUID|; s|\${RUNNER_SECRET}|$SECRET|" runner-config.yml +docker compose --profile ci up -d +``` + +Then check Site Administration → Runners: the runner should be online with the +label `docker`. + +**Four things that cost people an evening:** + +- A registration **token from the web UI is not a shared secret**. The declarative + path above only accepts the secret you generated. Feed it the web token and the + runner connects and never receives a job, with no error anywhere. +- The `connections:` block lives under a top-level **`server:`** key in runner + 13.x. Older examples put it under `runner:`, which fails with + `0 server connections configured`. +- **An empty `labels:` list means every job waits forever.** No error, just a + queued job and a healthy-looking runner. +- The image behind a label **must contain node** — Forgejo actions are JavaScript. + Pointing `docker` at a docker-cli image gives you + `node: executable file not found`. Use a node image and let the build job fetch + a static docker client (see `examples/ci.yml`). + +`examples/ci.yml` is a complete pipeline: test, build a container, push it to +Forgejo's built-in registry, and deploy it over an SSH key that can only run one +command (`examples/deploy-hello-ops.sh`). Read the CI section of +[HARDENING.md](HARDENING.md) before you wire the deploy step — there are two +comfortable ways to hand your whole server to anyone who can push a branch. + ## Hardening Before you put real company data on this box, read **[HARDENING.md](HARDENING.md)**. diff --git a/compose.yml b/compose.yml index db6cafd..935ce7c 100644 --- a/compose.yml +++ b/compose.yml @@ -22,6 +22,7 @@ name: selfhostedops-stack networks: proxy: + ci: internal: volumes: @@ -34,6 +35,9 @@ volumes: kuma_data: n8n_data: wg_data: + docker_certs: + dind_data: + runner_data: services: caddy: @@ -53,7 +57,7 @@ services: networks: [proxy] forgejo: - image: codeberg.org/forgejo/forgejo:11 + image: codeberg.org/forgejo/forgejo:16 restart: unless-stopped environment: USER_UID: "1000" @@ -63,6 +67,8 @@ services: FORGEJO__server__SSH_PORT: "2222" FORGEJO__service__DISABLE_REGISTRATION: "true" FORGEJO__openid__ENABLE_OPENID_SIGNIN: "false" + FORGEJO__actions__ENABLED: "true" + FORGEJO__repository__DEFAULT_REPO_UNITS: "repo.code,repo.issues,repo.pulls,repo.releases,repo.packages,repo.actions" ports: - "2222:22" # git over SSH volumes: @@ -163,3 +169,38 @@ services: volumes: - wg_data:/etc/wireguard networks: [proxy] + + # --- CI (episode 2) -- start with: docker compose --profile ci up -d --- + + docker-in-docker: + image: data.forgejo.org/oci/docker:dind + # The hostname MUST be "docker": the TLS certificates dind generates are + # only valid for that name. Rename it and you get a handshake error that + # reads like a network problem and is not one. + hostname: docker + profiles: [ci] + privileged: true + restart: unless-stopped + environment: + DOCKER_TLS_CERTDIR: /certs + volumes: + - docker_certs:/certs + - dind_data:/var/lib/docker + networks: [ci] + + forgejo-runner: + image: data.forgejo.org/forgejo/runner:13.0.0 + profiles: [ci] + restart: unless-stopped + depends_on: [docker-in-docker] + environment: + DOCKER_HOST: tcp://docker:2376 + DOCKER_CERT_PATH: /certs/client + DOCKER_TLS_VERIFY: "1" + volumes: + - runner_data:/data + - docker_certs:/certs + - ./runner-config.yml:/data/config.yml:ro + networks: [ci, proxy] + command: forgejo-runner --config /data/config.yml daemon + diff --git a/examples/ci.yml b/examples/ci.yml new file mode 100644 index 0000000..70e5766 --- /dev/null +++ b/examples/ci.yml @@ -0,0 +1,52 @@ +name: ci + +on: + push: + branches: [main] + +# Two pushes in a row must not deploy over each other. +concurrency: + group: deploy-${{ github.ref }} + cancel-in-progress: false + +env: + IMAGE: git.example.com/youruser/hello-ops + +jobs: + test: + runs-on: docker + steps: + - uses: actions/checkout@v4 + - run: npm test + + build: + needs: test + runs-on: docker + steps: + - uses: actions/checkout@v4 + # The job image has node (it must -- actions are JavaScript) but no docker + # client. One static binary, no package manager, no daemon. + - name: Get a docker client + run: | + curl -fsSL https://download.docker.com/linux/static/stable/x86_64/docker-29.7.2.tgz \ + | tar xz -C /usr/local/bin --strip-components=1 docker/docker + docker version --format "{{.Client.Version}}" + - name: Log in to our own registry + run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login git.example.com -u youruser --password-stdin + - name: Build and push + run: | + TAG=${GITHUB_SHA::7} + docker build --build-arg APP_VERSION=$TAG -t $IMAGE:$TAG -t $IMAGE:latest . + docker push $IMAGE:$TAG + docker push $IMAGE:latest + + deploy: + needs: build + runs-on: docker + steps: + - name: Deploy over a key that can only do one thing + run: | + install -m 700 -d ~/.ssh + echo "${{ secrets.DEPLOY_KEY }}" > ~/.ssh/id_ed25519 + chmod 600 ~/.ssh/id_ed25519 + ssh -o StrictHostKeyChecking=no deploy@${{ secrets.DEPLOY_HOST }} ${GITHUB_SHA::7} diff --git a/examples/deploy-hello-ops.sh b/examples/deploy-hello-ops.sh new file mode 100755 index 0000000..3d58a72 --- /dev/null +++ b/examples/deploy-hello-ops.sh @@ -0,0 +1,11 @@ +#!/bin/sh +# The only thing the CI key is allowed to do. Argument comes from the CI job. +set -eu +TAG=$(printf "%s" "${SSH_ORIGINAL_COMMAND:-latest}" | tr -cd "a-f0-9") +[ -n "$TAG" ] || TAG=latest +IMAGE=git.example.com/youruser/hello-ops:$TAG +docker pull "$IMAGE" +docker rm -f hello-ops 2>/dev/null || true +docker run -d --name hello-ops --restart unless-stopped \ + --network selfhostedops-stack_proxy -e APP_VERSION="$TAG" "$IMAGE" +echo "deployed $TAG" diff --git a/runner-config.example.yml b/runner-config.example.yml new file mode 100644 index 0000000..6faa053 --- /dev/null +++ b/runner-config.example.yml @@ -0,0 +1,42 @@ +log: + level: info + +runner: + file: /data/.runner + # How many jobs may run at the same time. On a small box keep this low -- + # your builds share CPU with everything else on the server. + capacity: 2 + timeout: 1h + # Jobs get a docker client pointed at the sidecar, never at the host daemon. + envs: + DOCKER_HOST: tcp://docker:2376 + DOCKER_TLS_VERIFY: "1" + DOCKER_CERT_PATH: /certs/client + # A label maps the name you write in "runs-on:" to the image the job runs in. + # This list is EMPTY by default -- and an empty list means every job you push + # waits forever with no error anywhere. + labels: + # The image behind a label must contain node -- Forgejo actions like + # checkout are JavaScript. A plain docker-cli image has no node and every + # JS action dies with "node: executable file not found". + - "docker:docker://code.forgejo.org/oci/node:22-bookworm" + +cache: + enabled: true + +container: + network: host + # Jobs talk to the docker-in-docker sidecar over TLS, never to the host daemon. + options: -v /certs/client:/certs/client + valid_volumes: + - /certs/client + +server: + # Declarative registration: same secret you passed to + # "forgejo forgejo-cli actions register --secret ...". + # A registration token from the web UI does NOT work here. + connections: + default: + url: https://git.example.com + uuid: ${RUNNER_UUID} + token: ${RUNNER_SECRET}