Last updated on August 11, 2024

Pre-Check

I will guide you use Github self-hosted runner to build a CICD pipeline on your private environment. We will deploy the index.html on node1 /var/www/html.
In this sample, we need two servers as below:

Node(Hostname)	IP	Network Privilege	Description
cicd	192.168.70.4	Github reachable	self-host server
node1	192.168.70.5	cicd node reachable	will deploy a static html on this server

## OS version
➜  ~ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04 LTS
Release:    24.04
Codename:   noble

## kernel version
➜  ~ uname -r
6.8.0-40-generic

## local DNS resolve
➜  ~ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 cicd
192.168.70.4 cicd
192.168.70.5 node1

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

I will use my private sample Github repo https://github.com/kylechenoO/webpage in this sample(only an index.html included).

Preparation

Before we start, please setup the static IP and NTP as below links:

Ubuntu 24.04 Static IP Configuration

Ubuntu 24.04 NTP Configuration

Install docker on cicd server

## run on cicd server
## install docker.io
➜  ~ apt install docker.io -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  bridge-utils containerd dns-root-data dnsmasq-base pigz runc ubuntu-fan
Suggested packages:
  ifupdown aufs-tools cgroupfs-mount | cgroup-lite debootstrap docker-buildx docker-compose-v2 docker-doc rinse zfs-fuse | zfsutils
The following NEW packages will be installed:
  bridge-utils containerd dns-root-data dnsmasq-base docker.io pigz runc ubuntu-fan
0 upgraded, 8 newly installed, 0 to remove and 4 not upgraded.
Need to get 57.6 MB of archives.
After this operation, 237 MB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports noble/universe arm64 pigz arm64 2.8-1 [60.7 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 bridge-utils arm64 1.7.1-1ubuntu2 [34.2 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 runc arm64 1.1.12-0ubuntu3 [7,913 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 containerd arm64 1.7.12-0ubuntu4 [27.8 MB]
Get:5 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 dns-root-data all 2023112702~willsync1 [4,450 B]
Get:6 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 dnsmasq-base arm64 2.90-2build2 [366 kB]
Get:7 http://ports.ubuntu.com/ubuntu-ports noble/universe arm64 docker.io arm64 24.0.7-0ubuntu4 [21.4 MB]
Get:8 http://ports.ubuntu.com/ubuntu-ports noble/universe arm64 ubuntu-fan all 0.12.16 [35.2 kB]
Fetched 57.6 MB in 2min 54s (332 kB/s)
Preconfiguring packages ...
Selecting previously unselected package pigz.
(Reading database ... 132092 files and directories currently installed.)
Preparing to unpack .../0-pigz_2.8-1_arm64.deb ...
Unpacking pigz (2.8-1) ...
Selecting previously unselected package bridge-utils.
Preparing to unpack .../1-bridge-utils_1.7.1-1ubuntu2_arm64.deb ...
Unpacking bridge-utils (1.7.1-1ubuntu2) ...
Selecting previously unselected package runc.
Preparing to unpack .../2-runc_1.1.12-0ubuntu3_arm64.deb ...
Unpacking runc (1.1.12-0ubuntu3) ...
Selecting previously unselected package containerd.
Preparing to unpack .../3-containerd_1.7.12-0ubuntu4_arm64.deb ...
Unpacking containerd (1.7.12-0ubuntu4) ...
Selecting previously unselected package dns-root-data.
Preparing to unpack .../4-dns-root-data_2023112702~willsync1_all.deb ...
Unpacking dns-root-data (2023112702~willsync1) ...
Selecting previously unselected package dnsmasq-base.
Preparing to unpack .../5-dnsmasq-base_2.90-2build2_arm64.deb ...
Unpacking dnsmasq-base (2.90-2build2) ...
Selecting previously unselected package docker.io.
Preparing to unpack .../6-docker.io_24.0.7-0ubuntu4_arm64.deb ...
Unpacking docker.io (24.0.7-0ubuntu4) ...
Selecting previously unselected package ubuntu-fan.
Preparing to unpack .../7-ubuntu-fan_0.12.16_all.deb ...
Unpacking ubuntu-fan (0.12.16) ...
Setting up dnsmasq-base (2.90-2build2) ...
Setting up runc (1.1.12-0ubuntu3) ...
Setting up dns-root-data (2023112702~willsync1) ...
Setting up bridge-utils (1.7.1-1ubuntu2) ...
Setting up pigz (2.8-1) ...
Setting up containerd (1.7.12-0ubuntu4) ...
Setting up ubuntu-fan (0.12.16) ...
Setting up docker.io (24.0.7-0ubuntu4) ...
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /usr/lib/systemd/system/docker.socket.
Processing triggers for dbus (1.14.10-4ubuntu4) ...
Processing triggers for man-db (2.12.0-4build2) ...
Scanning processes...
Scanning linux images...

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

## enable service
➜  ~ systemctl enable docker

## start service
➜  ~ systemctl start docker

## check service status
➜  ~ systemctl status docker
● docker.service - Docker Application Container Engine
     Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; preset: enabled)
     Active: active (running) since Sat 2024-08-10 17:07:38 HKT; 29s ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
   Main PID: 1954 (dockerd)
      Tasks: 9
     Memory: 25.7M (peak: 26.1M)
        CPU: 79ms
     CGroup: /system.slice/docker.service
             └─1954 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Aug 10 17:07:38 cicd systemd[1]: Starting docker.service - Docker Application Container Engine...
Aug 10 17:07:38 cicd dockerd[1954]: time="2024-08-10T17:07:38.125195169+08:00" level=info msg="Starting up"
Aug 10 17:07:38 cicd dockerd[1954]: time="2024-08-10T17:07:38.125536896+08:00" level=info msg="detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf"
Aug 10 17:07:38 cicd dockerd[1954]: time="2024-08-10T17:07:38.149085792+08:00" level=info msg="Loading containers: start."
Aug 10 17:07:38 cicd dockerd[1954]: time="2024-08-10T17:07:38.209419887+08:00" level=info msg="Loading containers: done."
Aug 10 17:07:38 cicd dockerd[1954]: time="2024-08-10T17:07:38.214255129+08:00" level=info msg="Docker daemon" commit=24.0.7-0ubuntu4 graphdriver=overlay2 version=24.0.7
Aug 10 17:07:38 cicd dockerd[1954]: time="2024-08-10T17:07:38.214334527+08:00" level=info msg="Daemon has completed initialization"
Aug 10 17:07:38 cicd dockerd[1954]: time="2024-08-10T17:07:38.222271533+08:00" level=info msg="API listen on /run/docker.sock"
Aug 10 17:07:38 cicd systemd[1]: Started docker.service - Docker Application Container Engine.

Create a user(cicdusr) to run cicd pipeline(can’t run with root user directly)

## run on cicd server
## add user
➜  ~ useradd cicdusr

## create home dir
➜  ~ mkdir /home/cicdusr

## grant privileges
➜  ~ chown -R cicdusr:cicdusr /home/cicdusr

## set permission for /var/run/docker.sock (if no, will occor 'permission denied')
➜  ~ setfacl -m u:cicdusr:rwx /var/run/docker.sock

## evaluate the permission setting
➜  ~ getfacl /var/run/docker.sock
getfacl: Removing leading '/' from absolute path names
# file: var/run/docker.sock
# owner: root
# group: docker
user::rw-
user:cicdusr:rwx
group::rw-
mask::rwx
other::---

Create a directory to config and run self-host runner for cicdusr

## run on cicd server
➜  ~ mkdir /develop/github/runners
➜  ~ chown -R cicdusr:cicdusr /develop/github/runners

Install apache2 on node1(for this sample, we will deploy an static html on node1 with apache2)

## run on node1
## install apache2
root@node1:~# apt install apache2 -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  apache2-bin apache2-data apache2-utils libapr1t64 libaprutil1-dbd-sqlite3 libaprutil1-ldap libaprutil1t64 liblua5.4-0 ssl-cert
Suggested packages:
  apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser
The following NEW packages will be installed:
  apache2 apache2-bin apache2-data apache2-utils libapr1t64 libaprutil1-dbd-sqlite3 libaprutil1-ldap libaprutil1t64 liblua5.4-0 ssl-cert
0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded.
Need to get 2,064 kB of archives.
After this operation, 14.0 MB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 libapr1t64 arm64 1.7.2-3.1build2 [105 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 libaprutil1t64 arm64 1.6.3-1.1ubuntu7 [93.9 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 libaprutil1-dbd-sqlite3 arm64 1.6.3-1.1ubuntu7 [11.2 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 libaprutil1-ldap arm64 1.6.3-1.1ubuntu7 [9,046 B]
Get:5 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 liblua5.4-0 arm64 5.4.6-3build2 [158 kB]
Get:6 http://ports.ubuntu.com/ubuntu-ports noble-updates/main arm64 apache2-bin arm64 2.4.58-1ubuntu8.4 [1,319 kB]
Get:7 http://ports.ubuntu.com/ubuntu-ports noble-updates/main arm64 apache2-data all 2.4.58-1ubuntu8.4 [163 kB]
Get:8 http://ports.ubuntu.com/ubuntu-ports noble-updates/main arm64 apache2-utils arm64 2.4.58-1ubuntu8.4 [96.3 kB]
Get:9 http://ports.ubuntu.com/ubuntu-ports noble-updates/main arm64 apache2 arm64 2.4.58-1ubuntu8.4 [90.2 kB]
Get:10 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 ssl-cert all 1.1.2ubuntu1 [17.8 kB]
Fetched 2,064 kB in 7s (277 kB/s)
Preconfiguring packages ...
Selecting previously unselected package libapr1t64:arm64.
(Reading database ... 129721 files and directories currently installed.)
Preparing to unpack .../0-libapr1t64_1.7.2-3.1build2_arm64.deb ...
Unpacking libapr1t64:arm64 (1.7.2-3.1build2) ...
Selecting previously unselected package libaprutil1t64:arm64.
Preparing to unpack .../1-libaprutil1t64_1.6.3-1.1ubuntu7_arm64.deb ...
Unpacking libaprutil1t64:arm64 (1.6.3-1.1ubuntu7) ...
Selecting previously unselected package libaprutil1-dbd-sqlite3:arm64.
Preparing to unpack .../2-libaprutil1-dbd-sqlite3_1.6.3-1.1ubuntu7_arm64.deb ...
Unpacking libaprutil1-dbd-sqlite3:arm64 (1.6.3-1.1ubuntu7) ...
Selecting previously unselected package libaprutil1-ldap:arm64.
Preparing to unpack .../3-libaprutil1-ldap_1.6.3-1.1ubuntu7_arm64.deb ...
Unpacking libaprutil1-ldap:arm64 (1.6.3-1.1ubuntu7) ...
Selecting previously unselected package liblua5.4-0:arm64.
Preparing to unpack .../4-liblua5.4-0_5.4.6-3build2_arm64.deb ...
Unpacking liblua5.4-0:arm64 (5.4.6-3build2) ...
Selecting previously unselected package apache2-bin.
Preparing to unpack .../5-apache2-bin_2.4.58-1ubuntu8.4_arm64.deb ...
Unpacking apache2-bin (2.4.58-1ubuntu8.4) ...
Selecting previously unselected package apache2-data.
Preparing to unpack .../6-apache2-data_2.4.58-1ubuntu8.4_all.deb ...
Unpacking apache2-data (2.4.58-1ubuntu8.4) ...
Selecting previously unselected package apache2-utils.
Preparing to unpack .../7-apache2-utils_2.4.58-1ubuntu8.4_arm64.deb ...
Unpacking apache2-utils (2.4.58-1ubuntu8.4) ...
Selecting previously unselected package apache2.
Preparing to unpack .../8-apache2_2.4.58-1ubuntu8.4_arm64.deb ...
Unpacking apache2 (2.4.58-1ubuntu8.4) ...
Selecting previously unselected package ssl-cert.
Preparing to unpack .../9-ssl-cert_1.1.2ubuntu1_all.deb ...
Unpacking ssl-cert (1.1.2ubuntu1) ...
Setting up ssl-cert (1.1.2ubuntu1) ...
Setting up libapr1t64:arm64 (1.7.2-3.1build2) ...
Setting up liblua5.4-0:arm64 (5.4.6-3build2) ...
Setting up apache2-data (2.4.58-1ubuntu8.4) ...
Setting up libaprutil1t64:arm64 (1.6.3-1.1ubuntu7) ...
Setting up libaprutil1-ldap:arm64 (1.6.3-1.1ubuntu7) ...
Setting up libaprutil1-dbd-sqlite3:arm64 (1.6.3-1.1ubuntu7) ...
Setting up apache2-utils (2.4.58-1ubuntu8.4) ...
Setting up apache2-bin (2.4.58-1ubuntu8.4) ...
Setting up apache2 (2.4.58-1ubuntu8.4) ...
Enabling module mpm_event.
Enabling module authz_core.
Enabling module authz_host.
Enabling module authn_core.
Enabling module auth_basic.
Enabling module access_compat.
Enabling module authn_file.
Enabling module authz_user.
Enabling module alias.
Enabling module dir.
Enabling module autoindex.
Enabling module env.
Enabling module mime.
Enabling module negotiation.
Enabling module setenvif.
Enabling module filter.
Enabling module deflate.
Enabling module status.
Enabling module reqtimeout.
Enabling conf charset.
Enabling conf localized-error-pages.
Enabling conf other-vhosts-access-log.
Enabling conf security.
Enabling conf serve-cgi-bin.
Enabling site 000-default.
Created symlink /etc/systemd/system/multi-user.target.wants/apache2.service → /usr/lib/systemd/system/apache2.service.
Created symlink /etc/systemd/system/multi-user.target.wants/apache-htcacheclean.service → /usr/lib/systemd/system/apache-htcacheclean.service.
Processing triggers for ufw (0.36.2-6) ...
Processing triggers for man-db (2.12.0-4build2) ...
Processing triggers for libc-bin (2.39-0ubuntu8.2) ...
Scanning processes...
Scanning linux images...

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

## enable and start apache2
root@node1:~# systemctl enable apache2
Synchronizing state of apache2.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable apache2
root@node1:~# systemctl start apache2

## check service status
root@node1:~# systemctl status apache2
● apache2.service - The Apache HTTP Server
     Loaded: loaded (/usr/lib/systemd/system/apache2.service; enabled; preset: enabled)
     Active: active (running) since Sat 2024-08-10 17:21:53 HKT; 29s ago
       Docs: https://httpd.apache.org/docs/2.4/
   Main PID: 5168 (apache2)
      Tasks: 55 (limit: 4550)
     Memory: 5.0M (peak: 5.2M)
        CPU: 11ms
     CGroup: /system.slice/apache2.service
             ├─5168 /usr/sbin/apache2 -k start
             ├─5169 /usr/sbin/apache2 -k start
             └─5170 /usr/sbin/apache2 -k start

Aug 10 17:21:53 node1 systemd[1]: Starting apache2.service - The Apache HTTP Server...
Aug 10 17:21:53 node1 apachectl[5167]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.70.5. Set the 'ServerName' directive globally to suppress this message
Aug 10 17:21:53 node1 systemd[1]: Started apache2.service - The Apache HTTP Server.

## check port avaliablity
root@node1:~# netstat -luntp | grep -i 80
tcp6       0      0 :::80                   :::*                    LISTEN      5168/apache2
udp6       0      0 fe80::20c:29ff:fee6:123 :::*                                2011/ntpd

We can also open browser to evaluate the service is up or not.

Create service user and grant privileges to deployment

## run on node1
## add user
root@node1:~# useradd appusr

## create home directory
root@node1:~# mkdir /home/appusr

## grant privileges
root@node1:~# chown -R appusr:appusr /home/appusr

## set passwd for appusr(please remember it, need to save it on Github Actions)
root@node1:~# passwd appusr
New password:
Retype new password:
passwd: password updated successfully

## grant privileges on service folder
root@node1:~# setfacl -m u:appusr:rwx /var/www/html/

## review the permission
root@node1:~# getfacl /var/www/html/
getfacl: Removing leading '/' from absolute path names
# file: var/www/html/
# owner: root
# group: root
user::rwx
user:appusr:rwx
group::r-x
mask::rwx
other::r-x

Let’s do it!

Open sample Github repo https://github.com/kylechenoO/webpage. Actions->Runners->Self-hosted runners->New runner

New Self-hosted runner

Select the OS version for your server for self-host

Configuration and setup self-host runner on cicd server

## run on cicd server
## switch to cicdusr
➜  ~ su - cicdusr

## change to runners directory
cicdusr@cicd:~$ cd /develop/github/runners/

## create a runner directory for agent
cicdusr@cicd:/develop/github/runners$ mkdir actions-runner && cd actions-runner

## download runner agent
cicdusr@cicd:/develop/github/runners/actions-runner$ curl -o actions-runner-linux-arm64-2.317.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.317.0/actions-runner-linux-arm64-2.317.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  107M  100  107M    0     0  10.2M      0  0:00:10  0:00:10 --:--:-- 5564k

## check md5 for tarball
cicdusr@cicd:/develop/github/runners/actions-runner$ echo "xxx  actions-runner-linux-arm64-2.317.0.tar.gz" | shasum -a 256 -c
actions-runner-linux-arm64-2.317.0.tar.gz: OK
cicdusr@cicd:/develop/github/runners/actions-runner$ tar xzf ./actions-runner-linux-arm64-2.317.0.tar.gz

## configure runner with your repo token
cicdusr@cicd:/develop/github/runners/actions-runner$ ./config.sh --url https://github.com/kylechenoO/webpage --token xxx

--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------

# Authentication

√ Connected to GitHub

# Runner Registration

Enter the name of the runner group to add this runner to: [press Enter for Default]

Enter the name of runner: [press Enter for cicd]

This runner will have the following labels: 'self-hosted', 'Linux', 'ARM64'
Enter any additional labels (ex. label-1,label-2): [press Enter to skip]

√ Runner successfully added
√ Runner connection is good

# Runner settings

Enter name of work folder: [press Enter for _work]

√ Settings Saved.

## start service
cicdusr@cicd:/develop/github/runners/actions-runner$ ./run.sh

√ Connected to GitHub

Current runner version: '2.317.0'
2024-08-10 09:44:05Z: Listening for Jobs