Upgrading a Helm-deployed Concourse
Introduction
Helm is the package manager for Kubernetes, a tool that streamlines installing and managing Kubernetes applications. It creates Kubernetes objects that can be submitted to Kubernetes clusters, and materialized into a Concourse deployment using Kubernetes constructs (Deployments, StatefulSets, PersistentVolumeClaims, etc).
A Helm Chart is a versioned package of pre-configured Kubernetes resources. Deploying Concourse via Helm Chart makes it easy to deploy, scale, maintain, or upgrade your deployment in the future. This guide aims to walk an operator through the step by step process of deploying with Helm.
Once you've deployed Concourse to a Kubernetes cluster, you can use the following process to keep it up to date as new releases arrive on VMware Tanzu Network.
Backup
In the case of a Helm-deployed Concourse pointed to a Postgres database that has been deployed and managed with BOSH, you can use BBR to perform a backup.
Otherwise, use the pg_dump
utility to backup your Postgres database. For more information on the tool, click here to view a guide on postgresql.org.
Download, Tag, and Push Images to Internal Registry
Download Concourse Helm Chart and load images into Docker
-
If you have not already done so, visit VMware Tanzu Network and download the Concourse Helm Chart.
-
Unarchive the Helm Chart tarball to a local directory. For example, with version v5.5.8, the tarball will be called
concourse-5.5.8-helm.tgz
.1
mkdir concourse-helm
1
tar xvzf ./concourse-5.5.8-helm.tgz -C ./concourse-helm
1
cd ./concourse-helm
-
Load the container images into a local Docker client by running the following
docker load
commands one at a time:1
docker load -i ./images/concourse.tar
1
docker load -i ./images/postgres.tar
1
docker load -i ./images/helm.tar
These images are quite large, and there will be no output until Docker is done loading.
Success
Once the loading finishes, you'll see:
1
Loaded image: IMAGE-NAME
Tag and push the loaded images to internal registry
Registry Authentication
This step assumes that the current docker
client has already authenticated against the internal registry through a regular docker login
.
In addition to logging in, if you're using a registry with self-signed certificates, you should also make sure your registry has been added to the 'Insecure Registries' section of the Daemon tab in the Docker settings ui for your current workstation.
For more information about certificates and secure registry concerns, see this article: Test an insecure registry
-
Begin by exporting a pair of variables to your shell to be reused throughout this process. In your terminal, run the following two commands:
1
export INTERNAL_REGISTRY=INTERNAL-REGISTRY
1
export PROJECT=PROJECT-NAME
Where:
INTERNAL-REGISTRY
is the domain of your internal registry - if you are using Harbor, this must correspond to the URL (without scheme)PROJECT-NAME
is the name of the project in your registry. If the project does not exist already you will need to make it.
-
The
.tar
file you downloaded contains a directory calledimages
. You need to extract the tag of each image so that you can appropriately tag the images with the internal registry and project name details from the last step.To do this, run the following command for each of the images in the
/images/
directory:1
cat ./images/IMAGE.tar.name | cut -d ':' -f 2
Where:
IMAGE
is one of the*.tar.name
files inside theimages
folder
Take note of the result for each image - this is corresponding
IMAGE-TAG
that will be used with each image in the next steps. -
Tag the images so their names include the internal registry address:
1 2 3
docker tag concourse/concourse:IMAGE-TAG $INTERNAL_REGISTRY/$PROJECT/concourse:IMAGE-TAG docker tag concourse/postgres:IMAGE-TAG $INTERNAL_REGISTRY/$PROJECT/postgres:IMAGE-TAG docker tag concourse/helm:IMAGE-TAG $INTERNAL_REGISTRY/$PROJECT/helm:IMAGE-TAG
1 2 3
docker tag concourse/concourse:IMAGE-TAG USERNAME/concourse:IMAGE-TAG docker tag concourse/postgres:IMAGE-TAG USERNAME/postgres:IMAGE-TAG docker tag concourse/helm:IMAGE-TAG USERNAME/helm:IMAGE-TAG
Where:
IMAGE-TAG
is the corresponding result from step 2.- (in the Docker Hub example)
USERNAME
is your Docker Hub username
-
Push the images to the internal registry by running the following commands in your terminal:
1 2 3
docker push $INTERNAL_REGISTRY/$PROJECT/concourse:IMAGE-TAG docker push $INTERNAL_REGISTRY/$PROJECT/postgres:IMAGE-TAG docker push $INTERNAL_REGISTRY/$PROJECT/helm:IMAGE-TAG
1 2 3
docker push USERNAME/concourse:IMAGE-TAG docker push USERNAME/postgres:IMAGE-TAG docker push USERNAME/helm:IMAGE-TAG
Where:
IMAGE-TAG
is the corresponding result from step 2.- (in the Docker Hub example)
USERNAME
is your Docker Hub username
You must have the necessary credentials (and authorization) to push to the targetted project.
Get previous deployment values
Helm uses a set of configuration values to template the different Kubernetes objects that Concourse requires into a release. In order to ensure you maintain the exact same configuration between the current version of Concourse and the upgraded version you are about to deploy, it is important to know what configuration is already deployed.
Version Control
Pivotal recommends checking in these configuration values to a source control management (SCM) system like git
so that you can track changes over time. If you have your configuration values in to an SCM system, check them out to your local machine so that they can be updated now.
If you're unsure of what has been deployed, and don't have access to the values files locally, you can fetch the deployment values that your existing release is using by running the helm get values
command. This will allow you to recreate the exact configuration with your upgraded release.
-
Get your previous deployment-values.yml file from SCM, or download it using this command:
1
helm get values RELEASE-NAME > deployment-values.yml
Where:
- RELEASE-NAME is the name of your existing Concourse release. Discover this by running the
helm list
command and looking for a chart that starts withconcourse
.
- RELEASE-NAME is the name of your existing Concourse release. Discover this by running the
-
Open the file in your favorite text editor. For example, with vim:
1
vim deployment-values.yml
-
Change the
imageTag
value to match the new version of Concourse. For example, using the v5.5.8 Helm Chart, this image tag will be5.5.8-ubuntu
.The following is a sample snippet; make sure the rest of the file you edit reflects your existing Concourse deployment:
1 2 3 4 5 6 7 8 9 10
--- image: INTERNAL_REGISTRY/PROJECT/concourse imageTag: 5.5.8-ubuntu imagePullSecrets: ["regcred"] #remove this line for public image registry postgresql: image: registry: INTERNAL_REGISTRY repository: PROJECT/postgres tag: POSTGRES-IMAGE-TAG pullSecrets: ["regcred"] # remove for public image registry
Where:
- INTERNAL-REGISTRY/PROJECT is your registry address and project.
-
imagePullSecrets is the pull secret listed under the key
spec.template.spec
in the following file:1
helm init --tiller-image $INTERNAL_REGISTRY/$PROJECT/helm:IMAGE-TAG --service-account tiller --dry-run --debug > tiller.yml
Remove if using a public repository.
-
POSTGRES-IMAGE-TAG is the output of
1
cat ./images/postgres.tar.name | cut -d ':' -f 2
-
pullSecrets is the pull secret created in step 6a. Remove if using a public repository.
Upgrade
Now that the deployment values are updated, use the helm upgrade
command to upgrade your existing release using the new values.
-
Upgrade your release:
1
helm upgrade RELEASE-NAME ./charts/ --values ./deployment-values.yml
Where:
- RELEASE-NAME is the name of your existing Concourse release. Discover this by running the
helm list
command and looking for a chart that starts withconcourse
.
Successful Deployment
If Kubernetes has accepted the manifest correctly, then you will see the following response followed by more information about your cluster:
1 2 3 4 5
NAME: DEPLOYMENT-NAME LAST DEPLOYED: DEPLOYMENT-DATE NAMESPACE: default STATUS: DEPLOYED ...
To determine whether your pods are up and running correctly, use the following command:
1
kubectl get pods
This will show you a list of deployed pods as well as their current status. It should look something like this:
1 2 3 4 5 6
$ kubectl get pods NAME READY STATUS RESTARTS AGE concourse-postgresql-0 1/1 Running 0 15m concourse-web-57ccf57f46-zsd25 1/1 Running 0 9m52s concourse-worker-0 1/1 Running 0 8m45s concourse-worker-1 1/1 Running 0 9m45s
- RELEASE-NAME is the name of your existing Concourse release. Discover this by running the