GitLab CI runners in Kubernetes using spot/preemptible instances
Running your GitLab CI jobs using their shared runners is really simple and straightforward, but you may want to use your own runners for several reasons:
- Build minutes limits (currently 400 per month)
- Build performance: ability to use high-end CPU performant instances (I observed near x2 improvement on CPU-intensive builds duration)
- And also network performance (ie. when using your cloud provider's registry), configuration fine tuning, etc.
However, using your own runners may become quickly expensive when dedicating instances for this purpose, which will probably only be used partially (only when building, probably not at night, etc.).
If you are running Kubernetes in the cloud, on AWS, GCP or similar, you can leverage the usage of spot or preemptible instances to run your builds at low cost (up to the tenth of regular price). When coupled with cluster autoscaling, Nodes (instances) will be added to your cluster only when required, and stopped automatically.
In your existing Kubernetes cluster, add a Node Pool with the following settings:
- Type: Spot (AWS) or Preemptible (GCP)
We'll be using Helm to deploy Gitlab Runner using official chart. To configure it, create a
gitlabUrl: https://gitlab.com/ runnerRegistrationToken: "<your-gitlab-runner-token>" concurrent: 10 checkInterval: 5 rbac: create: true clusterWideAccess: false # serviceAccountName: default runners: # Wait 10 minutes for runners availability, ie. when a node needs to be added pollTimeout: 600 image: ubuntu:20.04 # Required when using Docker-in-Docker privileged: true namespace: gitlab tags: "docker,kubernetes" builds: # cpuLimit: 200m # Be careful when setting memory limits, as runners will be killed when exceed it # memoryLimit: 256Mi cpuRequests: "1" memoryRequests: 1Gi services: cpuRequests: 100m memoryRequests: 128Mi helpers: cpuRequests: 100m memoryRequests: 128Mi # Force runners to run on specific nodes with this label nodeSelector: worlad-type: gitlab-runner # Allow runners to run on specific nodes with this taint nodeTolerations: - key: "dedicated" operator: "Equal" value: "gitlab-runner" effect: "NoSchedule"
Then deploy the chart to your cluster:
# Using Helm 3 helm repo add gitlab https://charts.gitlab.io helm upgrade --install --namespace gitlab gitlab-runner -f values.yaml gitlab/gitlab-runner
You're all set, you should now see your runner in your CI/CD GitLab settings, Runners section.