Concourse for PCF v4.x

Concourse Release Notes

Page last updated:


Release Date: May 21, 2019

Warning: To protect against clickjacking, set the x_frame_options property for the ATC job to deny or sameorigin instead of allow-from.
For more information, see X-Frame-Options in the Mozilla documentation.

Security Fix

This release contains the following security fix:

  • SSH MAC algorithms have been restricted to a smaller set to fix a vulnerability with the Concourse web instance VM. By default, Golang allows for some weak algorithms that can lead to security vulnerabilities on port 2222, which is used for worker communication.

    For more information, see SSH Weak MAC Algorithms Enabled in the Tenable documentation.

Known Issue

This release has the following issue:


Release Date: February 15, 2019

Warning: To protect against clickjacking, set the x_frame_options property for the ATC job to deny or sameorigin instead of allow-from.
For more information, see X-Frame-Options in the Mozilla documentation.

Security Fixes

This release contains the following security fix:

Fixed Issues

This release fixes the following issue:

  • The cf auth connector has been updated to use the authorization_endpoint so that the authentication flow can be completed successfully.

    Previously, authentication flows with the cf auth connector failed whenever a third-party SAML redirect is required.


Release Date: December 12, 2018

Security Fixes

This release contains the following security fix:

  • An issue related to authentication that is present in Concourse for PCF v4.x.


Release Date: October 5, 2018

Concourse for PCF v4.2.1 introduces the concept of user-based team authentication.

Previously, Concourse required you to log in under a specific team without an association to a user. In this version of Concourse, users can authenticate into teams as specific users. Users can be added to a team by configuring the team’s whitelist as described in Configuring Team Authentication.

Breaking Changes

This release has the following breaking changes:


  • The BOSH deployment for Concourse for PCF v4.0.0 requires manifest changes. For examples, see the operations directory in the Concourse BOSH deployment GitHub repository.

  • The concourse/bosh-deployment-resource has been removed from this release. Use the updated cloudfoundry/bosh-deployment-resource instead.

  • If you are currently on v3.6.0 and are looking to upgrade to v4.2.1, you must first upgrade to one of the following Concourse versions before upgrading to 4.2.1:

    • v3.8.0
    • v3.9.2
    • v3.13.0

    This is because v4.2.1 requires a code change for migration that is available in v3.8 and later.


  • If you have multiple teams configured with the same basic authentication username, the migration will fail. This is because “basic auth” is gone and in its place is local user configuration. Logging in with basic authentication now means logging in as the configured user.

  • There is no support for configuring the same provider multiple times, such as for multiple GitHub Enterprise instances. The migration fails when trying to upgrade an instance with teams that have different configurations for the same provider. The workaround to this limitation is to deploy multiple Concourse instances.

  • BitBucket authentication is no longer supported because the underlying authentication library does not support it. However, Concourse for PCF v4.0.0 supports GitHub, generic LDAP, OAuth, and OIDC connectors.


  • The team authentication flags for fly set-team have been split between set-team and the Concourse web module. For the specific changes, see Configuring Team Authentication. For further examples, see the operations directory in the Concourse BOSH deployment GitHub repository.

  • fly sync might not be able to update a v3.x version of fly to v4.0.0 due to the new authentication mechanisms. The workaround is to download and install the latest v4.0.0 fly CLI binary.


Features and changes in this release:


  • The fly teams command only lists teams of which you are a member (or all teams, if you are a member of the admin team main).
  • fly teams has a new flag, -d or --details, that displays each team’s authentication configuration.
  • The fly execute command with -j uses the job’s pipeline’s resource_types.
  • fly login can be invoked with -b to auto-launch a browser and execute OAuth.
  • fly supports --json on most commands to dump info in JSON format, rather than in table format.
  • fly validate-pipeline can be instructed to print the interpolated pipeline configuration using the --output flag.
  • Users can force an immediate check of a resource type using fly check-resource-type.


  • git-resource
    • Has git-crypt v0.6.0, which enables you to pin resources across the pipeline as part of the pipeline configuration by specifying version on the resource definition in the pipeline
    • Supports two new parameters:
      • submodule_recursive: false for disabling the default recursive fetching
      • submodule_remote: true for fetching submodules with --remote
    • Emits a short SHA to .git/short_ref, which can be useful for dynamic tagging
    • Uses the latest version of the Git Large File Storage (Git LFS) extension
    • Supports shallow clones by configuring depth: 1
    • Has the webhook_token property, which can be interpolated using a credential manager
  • s3-resource
    • Supports skip_download: true in parameters
    • Supports configuration of an initial version and content, which can be useful for bootstrapping state
  • docker-image-resource
    • Replaces the tag parameter with tag_file for the sake of clearer naming.
    • Supports interpolating the Concourse-provided environment variables in build arguments
  • cf-resource
    • Now supports client credentials-based authentication


  • The interval on which resource types are checked for new versions can now be:
    • Set globally using default_resource_type_check_interval
    • Set per-resource-type in a pipeline using check_every
      For more information, see default_resource_type_check_interval in the BOSH documentation.
  • Task caches can now be cleared using fly clear-task-cache.
  • The ATC can now be configured to periodically emit build logs to a syslog endpoint. This is configured using syslog parameters on the ATC job. When enabled, build logs ship off in a batch as builds complete.
  • The ATC now exposes an API endpoint for performing a health check against the configured credential manager at /api/v1/info/creds. This propagates useful information, depending on your credential manager backend.

    Note: This endpoint is only accessible by admin users (members of the main team).

  • Runtime has been improved to reduce the load on the database, improving garbage collection efficiency and web UI response time.
  • Container and volume garbage collection are now performed in parallel across the worker cluster. The ATC still displays when containers and volumes are to be removed, but is no longer responsible for performing the actual “destroy” API calls. This can make large-scale Concourse deployments much more efficient due to the removal of network and CPU overhead from the ATC.
  • Concourse workers can now be registered with the ephemeral parameter. When specified, the worker will be removed immediately after it stalls.
  • The ATC no longer fails to start if it is configured with CredHub and CredHub is not running. It will try to reach CredHub later instead.

Core Functionality

  • Concourse now supports user authentication into teams. Supported authorization providers include Basic Auth, GitHub Auth, CF Auth, OIDC, and OAuth.
  • Concourse emits warnings in the task logs when it detects that parameters are declared but not configured.
  • Pipeline credentials can now be verified using a new --check-creds flag available on fly set-pipeline. This command fetches values from the configured credential manager and tells you which values could not be interpolated.

Web UI

  • The dashboard page now has “Dashboard” in the title.
  • The Concourse pipeline view now has breadcrumbs to indicate which pipeline, job, or resource you are looking at.
  • Pipeline groups navigation has been redesigned to better display lots of groups and long group names.
  • Dashboard searches update the URL, making it easier to bookmark and share specific dashboard views.
  • The main page (/) now shows the dashboard instead of a pipeline configured by the first team on the instance.
  • Concourse Dashboard allows you to pause and reorder pipelines.
  • The sidebar has been removed from the pipeline view.
  • The dashboard now displays an orange triangle on a pipeline that has a resource that is failing to check.
  • Build page load performance is improved.
  • The dashboard view now indicates whether you are a member of a team or whether you are only seeing the dashboard view because it has exposed pipelines.


  • The Prometheus metrics now automatically prune stale workers.
  • The Prometheus metrics for pipeline scheduling are now counters instead of gauges.
  • There are now metrics emitted for periodic resource checking.
  • The Prometheus metric emitter has been generally improved.

Fixed Issues

This release fixes the following issues:


  • The fly intercept command no longer lists containers that are still being created and are not yet interceptable. Previously, this led to a websocket: bad handshake error.


  • Previously, if a resource was only ever used as an explicit output of a job, it would always show up as black even if it were erroring. It now shows up as orange, like the other resources.
  • s3-resource can now be used with Dell’s EMC ECS object store.
  • Publishing draft releases with the github-release-resource no longer causes errors.
  • Recent versions of Docker introduced an issue where dockerd could fail to start if the worker was under load. This resulted in an infinite loop in the docker-image-resource. Resources are now more resilient to this: they detect a failure to start and repeatedly attempt try to resuscitate dockerd until either two minutes elapse or dockerd starts.
  • docker-image-resource
    • Skips starting the Docker daemon if skip_download: true is set
    • Includes support for fetching and extracting XZ packages in ADD commands
    • Fails gracefully when build_args_file cannot be parsed
    • Fails with a clearer error when your ECR credentials are incorrect


  • Fixed handling of no_proxy on Concourse workers
  • The ATC now fails gracefully and early if no session signing key is specified, rather than ungracefully and late. In addition, a session signing key generates automatically if not given to Concourse Web.

    Warning: Do not rely on this behavior of automatic key generation as a long-term solution. If you use this behavior and continue to grow and onboard your Concourse users on to the system, a system restart will require new keys and might cause trouble for you down the line.

  • An artificial limit to the garbage collector, which was created to prevent excessive work for a single worker, is removed. Now that workers garbage-collect themselves, this limit is no longer necessary and only slows down the database side of the garbage collection lifecycle.
  • Fixed a container failure mode that occurs when check containers fail to create.
  • Tables are now cleaned up through database triggers on pipeline/team deletion. Previously, repeated team and pipeline creation and destruction would leave a few tables around, such as team_build_events_XXX and pipeline_build_events_XXX. This would cause the database to increase in CPU usage over time. If you see symptoms of this problem, it is likely safe to manually drop the tables that have no corresponding pipeline or team.


  • The BOSH release now has properties for configuring the DataDog metrics emitter.
  • The BOSH release now uses the configured postgresql.client_cert property.

Web UI

  • Messaging in the UI is now clearer. When viewing a build that has not been made public, it now says you are not authorized. Previously, it would tell you to log in, only to tell you to log in again because that did not change anything.
  • The build number in the <title> when viewing a one-off build in your browser is now consistent with the number reflected on the page.
  • Jobs and pipelines with spaces in their names now render correctly. Previously they rendered incorrectly in a couple of places. In general, Pivotal does not recommend whitespaces in pipeline names.
  • Fixed an “Aw, snap!” browser crash that affected some versions of Chrome when viewing the pipeline page

Core Functionality

  • A CredHub integration that caused very high CPU usage on the ATC is fixed. In addition, the CredHub client has been bumped to include a crucial fix.
  • When running on Windows, Concourse no longer shells out to TAR for performing volume streaming operations as doing so is unreliable. A native Go implementation will be used instead.
  • A potential problem in the delete worker API endpoint is fixed. This endpoint is used internally as part of the worker draining lifecycle.
  • The TSA now recognizes the configured log level for worker heartbeating logs.
  • Fixed a few API endpoints so that they correctly return Content-Type: application/json
  • Fixed an issue that caused the Vault login retry logic to go into a fast loop if retrying failed for long enough to exceed the maximum retry backoff
  • Removed unnecessary log messages from the TSA
  • Any errors when checking for new versions of a resource type are now reported as resource-checking errors. This includes failures to fetch credentials.

Known Issue

This release has the following issue:

Upgrading from Concourse for PCF v3.x to v4.2.1 can result in the following JSON exception when attempting to start the web node:

json: cannot unmarshal object into Go value of type []string
This is caused by a failing migration during the upgrade process. To recover from this error state, do the following:

  1. Access the Concourse database.
  2. Go to the teams table.
  3. Set the auth field on the main team to the empty string ''.
  4. Attempt the upgrade again.

This forces the ATC component of the web node to set the main team’s authentication to the parameters supplied in your manifest. For more information about this issue, see Upgrading concourse from 3.10 to 4.10 results in JSON exception on starting web nodes #2595 in GitHub.

User Authentication flows against UAA with external identity providers, such as SAML, do not complete when using the Concourse cf connector command. This is due to a mismatch in service URLs.

View Release Notes for Another Version

To view the release notes for another product version, select the version from the dropdown at the top of this page.

Create a pull request or raise an issue on the source for this page in GitHub