Concourse Release Notes
June 4, 2018
groundcrew job has been renamed to
worker in this release of Concourse;
please update your BOSH manifest accordingly.
For additional usage information and sample manifests you can refer to the ops file in the Cluster Concourse deployment section of the concourse-bosh-deployment repository.
Warning A bug in Chrome 67 causes it to crash when loading the Concourse UI. At the time of this notice (June 4, 2018) the dev/canary versions of Google Chrome should work, as well as other browsers like Firefox and Safari. You can follow along the issue in GitHub issue #2236
- This release involves a worker protocol version bump, from 1.3 to 2.0, and
also switches the default
BaggageClaimdriver back to
btrfs. We recommend spinning up a new pool of worker nodes, upgrading your web nodes, and then removing the old workers. Otherwise your workers may get swarmed with containers, if only one 2.0 worker is added at a time with the web nodes already upgraded.
- Concourse now defaults the worker
direct. If your Concourse installation uses external workers, please verify that your worker manifest explicitly specifies
registration_mode:forward. For more information on this new parameter, please reference the documentation on bosh.io.
- Added a new authentication provider for teams using OpenID Connect (OIDC) (PR #2)
- Concourse can now emit to Datadog using
semverresource now supports an optional
- The dashboard now supports the “not” operator for searches. This can be used on pipeline name searches, team searches, and status searches. Here are some examples:
-maingives you every pipeline other than the one called main
team:-maingives you every team’s pipeline other than the ones belonging to main
status:-pausedgives you all pipelines that are not paused
- The ATC will now batch-delete containers and volumes, rather than making individual calls out to the worker
- The ATC can now be configured with a global
default build_logs_to_retain. This is useful for operators who want more control over their database usage. A maximum value can also be configured to ensure users don’t set an unreasonable value. The flags are
fly set-pipelinecommand will no longer prompt
apply configuration?if there are no changes to apply.
- There’s a
fly order-pipelinescommand now, which allows you to set the order of pipelines in the web UI.
fly statuscommand has been added for checking whether or not you’re logged in to the given target.
fly check-resourcefails, it’ll bubble up the error message rather than just saying
error code 500.
- When a previously-created volume disappears from a worker and the ATC tries to use it, the error message will now include the worker name and the volume handle
git-resourcecan be configured to disable the fetching of tags; you can do this by configuring
/dashboardpage is mobile responsive and looks better on small screens
/dashboardpage makes way fewer requests now, so it’s a lot faster to load and more efficient to periodically refresh.
fly buildscommand can now filter by team (
-t) or pipeline (
- Concourse now supports AWS Secrets Manager for credential management. Please refer to our docs on Using Amazon SSM
- The Vault credential manager can now be configured with a
--vault-auth-backend-max-ttl, after which it will force a re-login
- The Vault credential manager will now retry with exponential backoff when logging in, rather than retrying every second.
- The Git resource will now make the commit message accessible under
- The web node can now be configured with
secure: trueon its cookies.
- The Github Release resource now supports a
tag_filterconfiguration for matching arbitrary semver tags
- The Docker Image resource now supports configuring
- The Docker Image resource now has a new param,
cache_from. This new param is like
load_basesexcept everything loaded will also be used as a cache during the build.
--strictwill now be more strict with your YAML
- The CF resource now supports
verbose: true, which will tell the CLI to dump trace logs to the output.
- The Docker Image resource now supports a
target_nameparam for specifying the target to build in a multi-stage
- The BOSH release now bakes in the glue code for use with BOSH Backup and Restore. Caveats apply, see PR #1975
fly set-pipelinecommand can now be given
--no-colorflag to strip out the color from the diffs. Instead of using color,
-will be at the start of added and removed lines.
- Now that we’re building with Go 1.10+,
flywill respect socks5 proxies configured via the “standard”
- The concourse worker commands can now be pointed at multiple TSA addresses,
rather than one, so that it can retry against a random node each time.
As part of this, we’ve removed the
--tsa-portflag and changed
--tsa-hostto instead take a
- We’ve revamped how
fly executegets its inputs and outputs to/from the build, so that configuring the ATC with an external URL is no longer required. See #2069 for the nitty-gritty.
- We’ve switched back to
btrfsas the default driver. This resolves a long-standing performance issue when using privileged tasks or resource types (like the Docker Image resource). For more information, see #1404 and #1966. Be sure to use the latest supported stemcell version so that you have a
btrfswith the latest fixes. We suspect that this will still be an occasional issue, though far less frequent.
- The Docker Image resource now supports pushing multiple tags
- When the ATC is streaming data between workers, the stream will now be
gzip-compressed, which should speed things up quite a bit.
- The ATC now requires TLS v1.2+ and a stricter set of cipher suites.
- Work around an apparent regression/behavior change in recent versions of Chrome that prevented the pipeline UI jobs from being clickable.
- Fixed a corner case in error handling that could cause a lock to be held forever when detecting new versions of resource types. This could lead to things like builds stuck in “pending” state.
- When directed to the login page from the resource page, you will now be redirected back to where you were
concourse webcommand is now capable of running the migration flags (
fly check-resourcecommand will now fail more clearly when the resource’s type is not found.
flywill once again helpfully instruct you to log in rather than just saying
- The Time resource will now correctly handle a tricky configurations that span multiple days (e.g. 10AM - 5AM),
- Added a missing property to the BOSH release for configuring the Generic oAuth provider’s CA cert.
- The Git resource will now recover from a deleted tag when configured with
- Previously, tags on a resource type didn’t get respected, that’s been addressed now.
- Fixed an ATC crash that would occurn when a
taskstep errored with the next step using an
- Concourse now supports the newer
umask-hardened BOSH stemcells (v3541.x).
- Fixed a botched
bashism that led to the Docker Image resource to exit early on certain environments (more info here).
- Cleaned up a noisy PostgreSQL error that would occur on start when checking
for the legacy
- Fixed a UI glitch that caused the last line to be misaligned with the timestamps if it had no trailing linebreak.
- Fixed a couple migrations that assumed a
flywill no longer repeatedly error when given an invalid token during
May 8, 2018
- Certificate propogation breaks Alpine OpenJDK images. See GitHub issue #2042
- Certificates can now be automatically propagated from the worker machine into resource containers. This resolves GitHub issue #1027. This feature is enabled when you install Concourse v3.9.2. Please refer to the OSS documentation on Certificate Propagation for more details.
- Concourse Dashboard has been graduated out of beta and can now be found under
- Tasks now support
- CredHub credential manager integration can now be configured with mutal TLS based Authentication
- Teams can now be renamed via the
flytarget will no longer be deted when running
fly logout, just the token
- Resource page on the web UI will now show when it last checked
Docker Imageresource supports loading multiple images at the start via
load_basesfor use in multi-part Dockerfiles
- When using
--url, the appropriate target will now be auto-detected based on the URL
- The Prometheus metrics endpoint now includes scheduling and database metrics
- Added support for NewRelic Insights metric emitter
- Added support for using AWS SSM for credential management
CF resourcenow has a
show_app_logconfig for tailing the app logs while starting it up
Docker Imageresource will now propagate
https_proxywhen building docker images
Docker Imageresource can now be configured with
Github Releaseresource will now produce a
commit_shafile containing the commit sha that the release’s tag points to
- Thet build page on the web UI can now render exotic ANSI text modes (e.g. faint text, framed text, Fraktur)
- Modified a configuration on the
btrfsvolume driver, making it much more stable to use
Docker Imageresource now correctly handles complicated build arguments
s3-resourcewill now auto-adjust the part size so it can upload files over 50GB
- Multi-part Dockerfiles with multiple ECR images will now correctly pull each with ECR login support
- Reduced the throttling when talking to k8s for credential management
- The Prometheus metrics endpoint no longer breaks HTTP metrics down by path
- When contacting CredHub, the configured CA cert is now respected. It was ignored in previous releases
- Fixed HTTP 500 errors when running
fly volumesas a result of volumes disappearing while the API walks through and gets their info.
- Fixed missing validation for
ensurewhen configured on a job
- Fixed a subtle timing issue that could result in
fly watchnot finding any builds to watch when given a job.
- We’ve optimized the rendering of the build page, which got quite a bit slower with the introduction of timestamps in v3.6.0
- Fixed a crash that would occur when a
flyCLI will now buffer output when rendering tables, which should make things a bit faster on Windows.
- Removed a database constraint
- Fixed an issue where builds would occasionally fail with
http2: no cached connection was availablewhen interacting with Vault
- Certain ANSI cursor movement escape sequences would wreak havoc on the Concourse build output page because there was no window size set on the TTY, thus defaulting it to 80x24. We’ve set it to 500x500
- Fixed an issue where Firefox users couldn’t click around on the build page.
February 1, 2018
If you are currently running a version of Concourse that is older than v3.6.0 and are planning to upgrade to v3.8.0, you must first upgrade to v3.6.0 before upgrading past it!
- When configuring CredHub to Concourse, you may encounter a
certificate signed by unkown authorityerror. Please see GH Issue #1873 for more details
- Concourse now support both up and down migrations. In the future, this will allow you to back out of an upgrade and revert back to a previous version of Concourse. However, this work required us to squash our migrations, so you will need to first upgrade to Concourse v3.6.0 before upgrading to v3.8.0
- If you are upgrading from v3.6.0 you will be required to execute certain changes to your Concourse BOSH manifest.:
- New required
- New required
- New required
tsa.private_key, which was just the private key portion
host_key, which is now of type
ssh, containing both the public and private portion.
host_public_key; superseded by the above property
authorize_generated_worker_key; no longer means anything
- The tsa
authorized_keysproperty must now be specified. No workers are automatically authorized anymore.
- New required
You can consult our canges to
manifests/single-vm.yml as a reference
Please also refer to Concourse BOSH release documentation on
- The ATC can now be configured with an idle
timeoutfor intercept sessions
- The Generic oAuth provider can now be configured with a CA certificate
- The Concourse Dashboard has been updated and has a new home under
beta/dashboard. Tell us what you think about the new dashboard by commenting on our GitHub issue #1829
executecommand will now default to
-x, which has been replaced with a new flag,
--include-ignored, to rever to the old behavior. In addition,
flywill gracefully handle executions with an input that doesn’t have a
.gitignore. It will also gracefully handle inputs that are files and not directories.
- The ATC will now use a separate database connection pool for the API and the pipeline scheduling work. This will make it so that slow API requests won’t starve critical functionality.
- Pipeline-provided resource types will no longer fail for the first two mintues after configuration
- Jobs and steps now support
- ATC can be configured with a pure-random worker selection strategy. This can be configured by passing
jobscommand now has a column indicating whether any builds are pending or started for each job
- The S3 resource now supports being configured with a session token
- Git repos encrypted with
git-cryptwill now be automatically decrypted by the Git resource
- The ATC can now be configured to serve a metrics endpoint for Prometheus
- Teams now support BitBucked-based auth
- The Git resource can now be configured with a HTTPS proxy
- Inline task configs are now validated as part of pipeline validation
- The CF resource can now be configured with a Docker username/password for pushing an app using a private repository
- The Github Release resource now supports being configured with insecure: true to support private GitHub Enterprise installations
- The Semver resource now supports being configured with skip_ssl_verification: true to support private S3-compatible blobstores
- ATC now has a flag for using k8s secrets when running in a cluster. This change makes using the k8s credential manager an explicit choice when running inside k8s, and also allows you to use a different credential manager when running in a cluster
- When the ATC is configured with multiple metrics emitters, it will now error, rather than silently picking one
- Fixed an issue where selecting/copying the build output would also select the timestamp on the left.
fly loginwill now error if arguments are mistakenly given to it
- Turns out you could easily spam the build page by holding
Tto trigger multiple builds. We’ve fixed that now so it only triggers a build once.
- Fixed the web UI so that it appropriately shows that you are logged out when your session expires.
- The deprecated Bosh Deployment resource resource contains the bosh CLI again
- Fixed an issue with the CredHub integration that made it necessary to configure
November 9, 2017
Concourse 3.6.0 now requires you to install and manage an external PostgreSQL database (v9.5+). We have enabled an upgrade path to the CloudFoundry Postgres BOSH release for your convenience.
Note: You do not need to follow these instructions if your Concourse deployment already connects to an external PostgreSQL database (v9.5+).
If you have not done so already, upgrade your Concourse to 3.5.0. Concourse 3.5.0 includes a change to the postgres job that moves its data to a new location where the Cloud Foundry Postgres release will detect and upgrade from.
Upload the Cloud Foundry Postgres release to your BOSH Director. Pivotal tested this upgrade path with version 20, currently available on bosh.io
Once the release is uploaded, add a reference to the job in your Concourse deployment manifest. You can do this by swapping out the
In the same Concourse deployment manifest, update the ATC properties to explicilty configure the database name and role. These values will vary based on your deployment preferences. You can refer to our changes on the single-vm Concourse manifest as a reference point.
Note that the Postgres DB upgrade must not be combined in the same deployment operation as a stemcell update
Concourse 3.6.0 now requires Garden runC 1.9.0. Make sure you download the appropriate version of Garden runC and recreate your workers.
fly validate-pipelinewill now validate the
configfield on embedded tasks. As a part of this change we have removed support for configuring both
file, which has been depracated.
- Build logs now have timestamps. You can find more about it on the feature post here
- Build page now supports keyboard shortcuts. You can find more about it on the feature post here. There is a known issue where keyboard shortcuts are non-functional on Firefox browsers. This should be fixed in a subsequent release.
- Fixed an issue with pipeline scheduling that would result in high database connection usage.
- Fixed an issue where clicking and dragging on the pipeline view would send you to the job details page.
September 25, 2017
- Support for CredHub for external credential management
- BaggageClaim volume creation APIs are now asynchronous
- Parallelized garbage collection. This should make things more durable to a slow worker, and make it harder for containers and volumes to “pile up” when the ATC is out of service briefly (i.e. during a deploy)
- BaggageClaim’s response header timeout is now configurable, which should help those with large images that they’re using for privileged tasks.
- When using groups in pipelines,
flywill now let you know when you forgot to assign a job to a group
flynow prints a URL to your build page when you run
- The fly command for
destroy-teamnow lets you supply the flag
- Jobs with a pending build now have a static halo to better represent its waiting state
flyCLI can now format a pipeline configuration into its ‘canonical form’ using the new
abort-buildcommand can now abort by build ID
- The Semver resource now supports Google Cloud Storage
- The Bosh Deployment resource now uses the latest BOSH CLI
- The Semver resource now supports Server Side Encryption
- The Git resource will now save the committer email to
- The legend on the pipeline page will now auto-hide after 10 seconds.
- When switiching between pipelines, the UI will now fit the pipeline in view.
- You can also press 'F’ center a pipeline in view.
- Jobs and resources with a forward slash in their name no longer error out when loading their details.
- Fixed a leak with goroutines that happens from
- Check containers will no longer be brutally destroyed if they’re used too close to their expiration time.
- Previously, if a resource or resource type was parameterized via a credential manager, its check containers and caches would be mistakenly garbage-collected. They will now be kept around.
- Fixed an issue where the pipeline view will reset after a state change on the pipeline.
- Added the appropriate headers to stop GitHub from caching badges
- Fixed an issue with the garbage collector that happens when deleting teams
- Files with the
setgidpermissions set on them will no longer have them removed. This used to be lost with the
chownperformed for namespacing the files. We’ll now restore it after the
- The flags for configuring GitLab oAuth are now present in
- Fixed an underflow in BaggageClaim’s volume size detection
unpackparameter in S3 resource will no longer load the entire archive into memory, so it can be used for larger archives
- A migration introduced in
v3.3.0would load all the builds into memory and then process them, causing a lot of issues when upgrading. We optimized this migration to migrate build plans in batches, rather than all at once.
July 31, 2017
Concourse for PCF is the first version of Concourse that is eligible for Pivotal Support. Concourse v3.3.3 was selected for this release because it addresses crucial issues from the Concourse for PCF tile beta program. Some of the new features in this version are:
- Major changes to the lifecycle management of workers, containers and volumes. For more details please refer to issue #629
- Support for web hooks
- Pipeline config and team auth settings can now be encrypted in the database. See Encrypting Concourse Databases
- Workers now use
btrfsfor their filesystems. For more details please refer to issue #1045
- New templating syntax for pipeline parameterization. See Using ((parameters))
- Performance and stability enhancements from schema optimizations, and parallelized ATC garbage collection
- Credential Management with Vault
- Support for GitLab oAuth configurations
- …and so much more! For a full history of features please visit the official Concourse Release Notes
- Misc bug fixes and stability improvements
- For a full history of bug fixes and known issues please visit the official Concourse Release Notes
- Operators may encounter memory issues on the ATC while upgrading from a version of Concourse older than 3.3.0.