Restoring from a BBR Backup
ivotal recommends that you create and restore Concourse backups with BOSH Backup and Restore (BBR). The process for restoring involves the following prerequisites:
- A valid BBR backup of Concourse. If you have not used BBR before, see the Performing a backup with BBR guide to get started.
bbr
, the BOSH Backup and Restore (BBR) CLI toolfly
, Concourse's CLI tool
With those conditions met, you can use this guide to perform a restore operation.
Reasons to restore Concourse from a backup include:
- Something went wrong with your Concourse pipelines
- You want to validate that a backup works
- An upgrade/downgrade failed
Creating and then validating backups can even be automated into a Concourse pipeline.
Note about restoring Concourse
Currently only restoring Concourse to a fresh deployment is supported.
If there is existing data, errors can arise when restoring Concourse.
Restore Procedure
This guide assumes you have done a bbr backup on some Concourse instance and you want to restore the backup to a new deployment.
-
Go to install docs to create a new deployment and use the
backup-atc.yml
ops-file. This ops-file will ensure thebbr-atcdb
anddatabase-backup-restorer
jobs are on the same VM instance as PostgreSQL.At this point, you should have a BOSH deployment of Concourse and PostgreSQL that you can target.
If your PostgreSQL instance where the Concourse atc
database is located is managed separatly from your Concourse BOSH deployment you will have to ensure that the bbr-atcdb
and database-backup-restorer
jobs are on the same VM instance as PostgreSQL.
Configure the database with superuser
capabilities for Concourse to enable pg_statements
There are two methods that can be used to address this step. Please read both methods below and choose the one that seems most appropriate.
Method 1
-
Log into Concourse's postgres instance using
psql
or your preferred database client
Where:1 2
# ssh onto the vm where postgres is running bosh -d DEPLOYMENT ssh INSTANCE_NAME
DEPLOYMENT
is the BOSH deployment where PostgreSQL is runningINSTANCE_NAME
is the VM instance where PostgreSQL is running
INSTANCE_NAME
To figure out what
INSTANCE_NAME
should be you can runbosh vms
to see what instance names are part of your BOSH deployment. For example:1
bosh -d DEPLOYMENT vms
-
Once you're ssh'd on the VM and you have access to a user (DB_USER) with the superuser permission, you can login using
psql
.1 2 3 4 5 6
$ /var/vcap/packages/postgres-11.5/bin/psql -d atc -U DB_USER -W Password: <input password> psql (11.5) Type "help" for help. atc=>
-
Run
ALTER USER <concourse_db_user> WITH SUPERUSER;
Method 2
Modify the PostgreSQL manifest to elevate the Concourse role to SUPERUSER
. Fill out the databases.roles[n].permissions
field in the postgres
job.
For example:
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Deploy Concourse database instance
After updating the BOSH manifest to give the database user superuser permissions, the database will have to be redeployed.
- If part of Concourse deployment, use BOSH to redeploy the db vm.
- If separate from the Concourse deployment, follow infrastructure specific instructions.
bbr restore
-
Run the
bbr deployment restore
command:1 2 3 4
bbr deployment \ --deployment TARGET-CONCOURSE \ restore \ --artifact-path PATH-TO-BACKUP
Where:
TARGET-CONCOURSE
is your target deployment, andPATH-TO-BACKUP
is the path to the backup directory you wish to restore from.
-
Make sure that you are running the same version of Concourse that your backup was created from.
Verify your pipelines have been successfully restored
-
Run the following command to verify that the state of your pipelines has been restored:
1
fly -t TARGET-CONCOURSE pipelines
Where
TARGET-CONCOURSE
is your target deployment.
Unpause pipelines
Troubleshooting
Broken git Resources
Symptom: After a restore, git repository resources appear broken with an error like this:
1 |
|
Solution: Recreate the pipelines by running fly set-pipeline
.
Cannot Drop Table
Symptom: There's an error when restoring Concourse like this:
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Solution: Restore Concourse to a fresh deployment. This happens when restoring Concourse to a deployment with existing data.
Unregistered Redirect URI
Symptom: When logging in after BBR restoration, there's an error like this:
1 2 |
|
Solution: Stop and restart the Concourse web instance(s).
1 |
|
1 |
|