Deploy a Virtualized Environment¶
This guide demonstrates how to deploy the virtualized airshipctl and treasuremap gating environments. While both environments provide an under-the-hood demonstration of how Airship works, they are not required for development. We recommend that developers testing changes consider if using Airship-in-a-Pod (AIAP) or using Kubectl to apply rendered manifests to external Kubernetes clusters better suits their needs before proceeding.
Pre-requisites¶
The following are pre-requisites for deploying virtualized Airship environments:
Minimum 20 GB RAM
Minimum 8 vCPUs
Minimum 100 GB storage
Ubuntu 18.04
Nested virtualization (if your host is a virtual machine)
Select an environment¶
This guide supports the airshipctl test-site and treasuremap test-site.
Clone repositories¶
Clone airshipctl:
git clone https://opendev.org/airship/airshipctl.git
If you are deploying a Treasuremap site, clone Treasuremap to the same parent directory as airshipctl:
git clone https://opendev.org/airship/treasuremap.git
Proxy Setup¶
If your organization requires development behind a proxy server, you will need to define the following environment variables with your organization’s information:
HTTP_PROXY=http://username:password@host:port
HTTPS_PROXY=http://username:password@host:port
NO_PROXY="localhost,127.0.0.1,10.23.0.0/16,10.96.0.0/12"
PROXY=http://username:password@host:port
USE_PROXY=true
10.23.0.0/16 encapsulates the range of addresses used by airshipctl
development environment virtual machines, and 10.96.0.0/12 is the Kubernetes
service CIDR.
Configure DNS servers¶
If you cannot reach the Google DNS servers from your local environment, you will
need to replace the Google DNS servers with your DNS servers in your site’s
NetworkCatalogue.
For the airshipctl test-site, update
airshipctl/manifests/type/gating/shared/catalogues/networking.yaml. For
the treasuremap test-site, update
treasuremap/manifests/site/test-site/target/catalogues/networking.yaml.
Configure test encryption key¶
Execute the following to download and export the test encryption key and fingerprint.
curl -fsSL -o /tmp/key.asc https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc
export SOPS_IMPORT_PGP="$(cat /tmp/key.asc)"
export SOPS_PGP_FP="FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4"
Run the setup scripts¶
Install required packages and configure Ansible¶
From the root of the airshipctl repository, run:
./tools/gate/00_setup.sh
Create virsh VMs¶
From the root of the airshipctl repository, run:
./tools/gate/10_build_gate.sh
Generate an airshipctl configuration file¶
For the airshipctl test-site, execute the following from the root of the airshipctl repository:
./tools/deployment/22_test_configs.sh
For the treasuremap test-site, execute the following from the root of the treasuremap repository:
./tools/deployment/airship-core/22_test_configs.sh
Download deployment manifests (documents)¶
For the airshipctl test-site, execute the following from the root of the airshipctl repository:
./tools/deployment/23_pull_documents.sh
For the treasuremap test-site, execute the following from the root of the treasuremap repository:
./tools/deployment/airship-core/23_pull_documents.sh
Generate site secrets¶
For the airshipctl test-site, execute the following from the root of the airshipctl repository:
./tools/deployment/23_generate_secrets.sh
For the treasuremap test-site, execute the following from the root of the treasuremap repository:
./tools/deployment/airship-core/23_generate_secrets.sh
Build ephemeral node ISO and target cluster control plane and data plane images¶
For the airshipctl test-site, execute the following from the root of the airshipctl repository:
./tools/deployment/24_build_images.sh
For the treasuremap test-site, execute the following from the root of the treasuremap repository:
./tools/deployment/airship-core/24_build_images.sh
Deploy the ephemeral and target clusters¶
For the airshipctl test-site, execute the following from the root of the airshipctl repository:
./tools/deployment/25_deploy_gating.sh
For the treasuremap test-site, execute the following from the root of the treasuremap repository:
./tools/deployment/airship-core/25_deploy_gating.sh
Troubleshooting¶
Validate Ephemeral Cluster is Operational¶
If the 25_deploy_gating.sh script fails with:
19: Retrying to reach the apiserver
+ sleep 60
+ '[' 19 -ge 30 ]
+ timeout 20 kubectl --context ephemeral-cluster get node -o name
+ wc -l
The connection to the server 10.23.25.101:6443 was refused - did you specify the right host or port?
or a similar error, validate that the ephemeral cluster is reachable:
kubectl --kubeconfig ~/.airship/kubeconfig --context ephemeral-cluster get pods --all-namespaces
The command should yield output that looks like this:
NAMESPACE NAME READY STATUS RESTARTS AGE
capi-kubeadm-bootstrap-system capi-kubeadm-bootstrap-controller-manager-556678c94-hngzj 2/2 Running 0 50s
capi-kubeadm-control-plane-system capi-kubeadm-control-plane-controller-manager-556d47dffd-qljht 2/2 Running 0 47s
capi-system capi-controller-manager-67859f6b78-2tgcx 2/2 Running 0 54s
capi-webhook-system capi-controller-manager-5c785c685c-fds47 2/2 Running 0 55s
capi-webhook-system capi-kubeadm-bootstrap-controller-manager-77658d7745-5bb7z 2/2 Running 0 52s
capi-webhook-system capi-kubeadm-control-plane-controller-manager-74dcf8b9c-ds4l7 2/2 Running 0 49s
capi-webhook-system capm3-controller-manager-568747bbbb-zld5v 2/2 Running 0 45s
capm3-system capm3-controller-manager-698c6d6df9-n72cf 2/2 Running 0 42s
cert-manager cert-manager-578cd6d964-lznfq 1/1 Running 0 76s
cert-manager cert-manager-cainjector-5ffff9dd7c-h9v6l 1/1 Running 0 76s
cert-manager cert-manager-webhook-556b9d7dfd-hvvfs 1/1 Running 0 75s
hardware-classification hardware-classification-controller-manager-776b5f66f8-6z9xl 2/2 Running 0 10m
kube-system calico-kube-controllers-94b8f9766-6cl6l 1/1 Running 0 10m
kube-system calico-node-dw6c8 1/1 Running 0 10m
kube-system coredns-66bff467f8-57wpm 1/1 Running 0 13m
kube-system coredns-66bff467f8-lbfw2 1/1 Running 0 13m
kube-system etcd-ephemeral 1/1 Running 0 13m
kube-system kube-apiserver-ephemeral 1/1 Running 0 13m
kube-system kube-controller-manager-ephemeral 1/1 Running 0 13m
kube-system kube-proxy-whdhw 1/1 Running 0 13m
kube-system kube-scheduler-ephemeral 1/1 Running 0 13m
metal3 ironic-5d95b49d6c-lr6b2 4/4 Running 0 10m
metal3 metal3-baremetal-operator-84f9df77fb-zq4qv 3/3 Running 0 10m
One of the most common reasons for a failed ephemeral cluster deployment is because a user is behind a corporate firewall and has not configured the proxy and DNS settings required for the virtual machines to reach the internet. If the ephemeral cluster is not reachable, we recommend validating that you have configured your environment’s proxy and DNS settings above.
Validate Target Cluster is Operational¶
Similarly, you can validate that your target cluster is operational using the context target-cluster:
kubectl --kubeconfig ~/.airship/kubeconfig --context target-cluster get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
capi-kubeadm-bootstrap-system capi-kubeadm-bootstrap-controller-manager-556678c94-svqmn 2/2 Running 0 56s
capi-kubeadm-control-plane-system capi-kubeadm-control-plane-controller-manager-556d47dffd-z28lq 2/2 Running 0 46s
capi-system capi-controller-manager-67859f6b78-x4k25 2/2 Running 0 64s
capi-webhook-system capi-controller-manager-5c785c685c-9t58p 2/2 Running 0 69s
capi-webhook-system capi-kubeadm-bootstrap-controller-manager-77658d7745-wv8bt 2/2 Running 0 62s
capi-webhook-system capi-kubeadm-control-plane-controller-manager-74dcf8b9c-rskqk 2/2 Running 0 51s
capi-webhook-system capm3-controller-manager-568747bbbb-gpvqc 2/2 Running 0 35s
capm3-system capm3-controller-manager-698c6d6df9-n6pfm 2/2 Running 0 27s
cert-manager cert-manager-578cd6d964-nkgj7 1/1 Running 0 99s
cert-manager cert-manager-cainjector-5ffff9dd7c-ps62z 1/1 Running 0 99s
cert-manager cert-manager-webhook-556b9d7dfd-2spgg 1/1 Running 0 99s
flux-system helm-controller-cbb96fc8d-7vh96 1/1 Running 0 11m
flux-system source-controller-64f4b85496-zfj6w 1/1 Running 0 11m
hardware-classification hardware-classification-controller-manager-776b5f66f8-zd5rt 2/2 Running 0 11m
kube-system calico-kube-controllers-94b8f9766-9r2cn 1/1 Running 0 11m
kube-system calico-node-6gfpc 1/1 Running 0 11m
kube-system coredns-66bff467f8-4gggz 1/1 Running 0 16m
kube-system coredns-66bff467f8-qgbhj 1/1 Running 0 16m
kube-system etcd-node01 1/1 Running 0 16m
kube-system kube-apiserver-node01 1/1 Running 0 16m
kube-system kube-controller-manager-node01 1/1 Running 0 16m
kube-system kube-proxy-ch6z9 1/1 Running 0 16m
kube-system kube-scheduler-node01 1/1 Running 0 16m
metal3 ironic-5d95b49d6c-8xwcx 4/4 Running 0 11m
metal3 metal3-baremetal-operator-84f9df77fb-25h4w 3/3 Running 0 11m
Restart VMs¶
In case a restart of your host causes the Airship VMs to not restart, execute the commands below to restart your VMs.
$ sudo virsh list --all
Id Name State
----------------------------------------------------
- air-ephemeral shut off
- air-target-1 shut off
- air-worker-1 shut off
$ virsh net-start air_nat
Network air_nat started
$ virsh net-start air_prov
Network air_prov started
$ virsh start air-target-1
Domain air-target-1 started
$ virsh start air-worker-1
Domain air-worker-1 started
$ sudo virsh list --all
Id Name State
----------------------------------------------------
3 air-target-1 running
4 air-worker-1 running
Re-deploying¶
In case you need to re-run the deployment from a clean state, we recommend running the script below from the root of the airshipctl repository beforehand.
sudo ./tools/deployment/clean.sh