Skip to content

Concourse Release Notes

v5.5.11

Release Date: April 24, 2020

Maintenace

This release contains the following maintenance features:

  • Operators can now limit the number of concurrent requests made to an API endpoint. Currently this only supports the ListAllJobs endpoint.

    • This is controllable through the new concurrent_request_limits property. The value is set in a action:limit format. An example would be ListAllJobs:5.
  • Add loading indicator on dashboard while awaiting initial API response.

Resolved Issues

  • The dashboard page refreshes its data every 5 seconds. Until now, it was possible (especially for admin users) for the dashboard to initiate an ever-growing number of API calls, unnecessarily consuming browser, network and API resources. Now the dashboard will not initiate a request for more data until the previous request finishes.

v5.5.10

Release Date: April 30, 2020

Security Fixes

This release contains the following security fixes:

  • Made a revision to the original fix for CVE-2018-15798 to protect against a recently discovered edge case:
    • CVE-2018-15798:
      • Pivotal Concourse Release login flow allows redirects to untrusted websites. A remote unauthenticated attacker could convince a user to click on a link using the oAuth redirect link with an untrusted website and gain access to that user's access token in Concourse.

Resolved Issues

This release fixes the following issues:

Core

These are the fixes to the core functionality of Concourse:

  • Added a flag called --disable-list-all-jobs. When this flag is passed, the /api/v1/jobs endpoint will always return an empty JSON array instead of making complex and expensive database operations. The most significant impact of this change is that the dashboard will no longer display pipeline previews.

v5.5.8

Release Date: March 10, 2020

Security Fixes

This release contains the following security fixes:

  • Updates the golang.org/x/crypto module from v0.0.0-20190313024323-a1f597ede03a to v0.0.0-20200220183623-bac4c82f6975 to address a recently reported security vulnerability in the ssh package:
    • CVE-2020-9283:
      • golang.org/x/crypto before v0.0.0-20200220183623-bac4c82f6975 for Go allows a panic during signature verification in the golang.org/x/crypto/ssh package. A client can attack an SSH server that accepts public keys. Also, a server can attack any SSH client.

v5.5.7

Release Date: December 18, 2019

Security Fixes

This release contains the following security fixes:

  • Updates the git resource to v1.6.3 to address a recently reported security vulnerability:
    • CVE-2019-19604:
      • Arbitrary command execution is possible in Git before 2.20.2, 2.21.x before 2.21.1, 2.22.x before 2.22.2, 2.23.x before 2.23.1, and 2.24.x before 2.24.1 because a "git submodule update" operation can run commands found in the .gitmodules file of a malicious repository.

v5.5.6

Release Date: November 19th, 2019

Features

New features and changes in this release:

Core

These are the new features and changes to the core functionality of Concourse:

  • API endpoints have been changed to use a single transaction per request, so that they become "all or nothing" instead of holding data in memory while waiting for another connection from the pool. In earlier versions, this could lead to snowballing and increased memory usage as requests from the web UI (polling every 5 seconds) piled up.
  • Batching has been added to the NewRelic emitter as well as logging info for non 2xx responses from NewRelic.

Resolved Issues

This release fixes the following issues:

Runtime

These are the fixes to the Concourse runtime:

  • Concourse now garbage-collects worker containers and volumes that are not tracked in the database. In some niche cases, it is possible for containers and/or volumes to be created on the worker, but the database (via the web) assumes their creation had failed. If this occurs, these untracked containers can pile up on the worker and use resources. This fix ensures that they get cleaned appropriately.

  • failed state containers have been transitioned to destroying, resulting in them being garbage-collected. This fix ensures that if the web's call to garden to create a container times out, the container is subsequently deleted from garden prior to being deleted from the db. This will keep the state of both the web and the worker consistent.

  • A five minute timeout for worker's garden client http calls has been added. This fix primarily addresses issues where destroy may hang indefinitely, causing garbage collection to stop working.

  • A five minute timeout for baggageclaim destroy calls has been added.


v5.5.3

Release Date: October 4, 2019

Upgrading

Pivotal Concourse provides upgrade guides describing the step-by-step process for upgrading a BOSH-deployed Concourse.

If you are currently using Pivotal Concourse v3.13.0, Pivotal recommends first upgrading to v4.2.4 before upgrading to v5.5.3 as the most reliable upgrade path.

Choose your guide based on the version you want to upgrade from:

Security Fixes

This release contains the following security fixes:

  • The 'password' form field on the login page now has autocomplete="off" set. This will be ignored by most browsers, but helps Concourse pass some automated security scans.

Breaking Changes

This release has the following breaking changes:

Runtime

These breaking changes affect the Concourse runtime:

  • Switched to a more efficient algorithm for streaming volumes between various components of the system: the web and worker nodes will now transmit volumes compressed with the Zstandard algorithm rather than Gzip - fly is affected too (when uploading/downloading artifacts during fly execute). This should resolve some general complaints of steps being slow to start or appearing to "hang" in clustered environments.
    • Because of the efficiency gained from employing Zstandard compression to volume streaming, the CPU on the web and worker nodes is no longer a limiting factor. Now volumes will be streamed over multiple parallel connections.
  • There is a new container placement strategy, limit-active-tasks: If this strategy is selected, the cluster will maintain a counter of the number of task containers currently running on each worker. When this strategy is in use, the worker with the fewest active task containers will be chosen to run a new container.
    • There is also an optional max_active_tasks_per_worker configuration. If this is set to a positive integer, the following behaviour will occur:
      • If all workers are at their active task limit, a "All workers are busy at the moment, please stand-by" message will occur and the scheduler will retry a minute later. This pattern will repeat each minute indefinitely, until a worker is available.
    • Note: This is an experimental feature. As such, this feature isn't yet recommended for production use.
  • Requests from the web nodes to the garden servers inside workers will time out after 5 minutes. This means that if a user has a lot of latency or their garden servers are slow, their builds and/or resource checks will start to fail.

Web UI

This breaking change affects the Concourse web UI:

  • On the build page all steps are now collapsed by default. This will result in the build page loading faster. However, this now means that if users want to watch the logs in real time, they will need to click on the header of the running step.

Features

New features and changes in this release:

Fly Commands

These are the new features and changes to fly commands:

  • Concourse admins can now run fly active-users and get a summary of all the users on the cluster, filtering by their last login time (the last 2 months by default).
  • A fly completion subcommand has been added that allows fly to output autocompletion for the bash and zsh shells.
  • fly has the ability to autocomplete certain configurations for certain shells. This release now allows team and worker names to be autocompleted in fly commands.
  • The output of fly targets now shows an error message in the table if your token for a given target is invalid.

Resources

These are the new features and changes to Concourse resources:

  • Git Resource:
    • Added the ability for put steps to git resources with the parameter merge: true to be further configured with the parameter returning: unmerged. This should ensure that concurrent changes to the same remote branch do not affect downstream jobs.
  • BOSH Backup and Restore:
    • Added the option for the BOSH Backup and Restore database backup job in the BOSH release to be configured via a BOSH link.
  • Pool Resource:
    • Optimized the pool resource: check and in scripts will run faster by using shallow cloning.
    • A depth parameter was added to the get steps associated with the pool resource: This will create the ability to get a specified number of commits when fetching a repository.
  • Semver Resource:
    • Added an option to skip SSL verification for the git driver on the semver-resource.
    • Added a pre_without_version param to the get and put steps associated to the semver resource. This allows for notating pre-releases without an associated "RC number".
  • Github Release Resource:
    • Added proxy support to the Github release resource.
    • The number of assets listed per release was raised by from 30 to 100 on the Github release resource.
    • Added annotated tag support to the Github release resource: Previously only lightweight tags were supported.
  • Mercurial Resource:
    • The evolve extension has been added to the Mercurial resource.

Core Functionality

These are the new features and changes to the core functionality of Concourse:

  • Updated admin privileges: In previous releases, admins (owners of the main team) had permission to modify the auth configuration for other teams in the same cluster. Now, admins also have full control over pipelines, jobs, resources, builds, etc for all teams. Using fly, they can log in to any team on the cluster as though they are an owner.
  • Web nodes can now accept encrypted connections over HTTPS: Users now have the option of connecting to an ACME server to automatically retrieve a certificate. By default, it will reach the free public ACME service provided by Let's Encrypt or from an ACME server of the user's choice.
    • Users using LetsEncrypt are recommended to upgrade to this version because it includes a fix for a bug where updating existing acme/autocert certificates fail.
  • Improvements to the info returned by the api endpoint (/api/v1/info): The endpoint now returns both the external URL and cluster name.
  • Concourse is now enabled to authenticate with Vault as a credential manager via an LDAP backend: This can be accomplished by setting CONCOURSE_VAULT_AUTH_PARAM="username:<your ldap username>,password:<your ldap password>.
  • Added a configurable buffer size for metrics emission (regardless of configured emitter): On particularly busy clusters, users have observed metrics events being dropped due to a full queue. This update should allow operators to trade memory pressure on the web nodes for reliability of metric transmission.
  • Added features for InfluxDB metrics: To decrease the request load on InfluxDB, users can configure the number of events to batch into a single InfluxDB request, or can specify a hardcoded interval at which to emit events, regardless of how many have accumulated.
  • Updated the worker containers and worker volumes metrics to include metadata for the tag(s) on the worker and/or the team the worker belongs to (if any).

Web UI

These are the new features and changes to the Concourse web UI:

  • Updated the 'prep' section at the top of the build page: When a build is manually triggered through the web UI, it will now show a spinner on the entries marked "discovering new versions of..." while the resource checks are in progress. As the checks resolve, the spinners will become checkmarks, giving a better sense of what might be holding up a build.
  • When a pending build is aborted before it gets scheduled, the build page will now report it as 'cancelled' instead of indefinitely displaying a loading spinner.
  • The build process has been tuned so that the size of the compressed frontend code is reduced by a factor of 3: This should result in faster page loads.
  • The CSS optimizer used in building the web UI has been upgraded: The resulting CSS is slightly smaller, which should result in some faster page loads.
  • Made design improvements to failing steps: Now a failing step has a red border around its header, an erroring step has an orange border, and a running step has a yellow border.
  • Build step headers are now sticky: The headers will stick to the top of the body on the build page instead of scrolling out of sight as users navigate long logs.
  • When a resource is pinned through the web UI, the user’s username and the time they pinned the resource will now be automatically added as the pin's comment.
  • The highlighted lines on the build page are now slightly darker.
  • Improved the appearance of the sidebar: Teams and pipelines are now easier to distinguish, and there are nicely styled tooltips for long items.

Resolved Issues

This release fixes the following issues:

Fly Commands

These are the fixes to fly commands:

  • fly execute makes sure only to upload user artifacts to workers matching the platform of the task being executed: Previously, when a local machine was running Mac or Linux and a user ran fly execute with --input against a cluster that had a mix of Linux and Windows workers, files could get uploaded to a Windows worker, where the permission metadata would be stripped off.
  • Updated fly so that it prints the usage text on stdout and returns a successful status code when passing the -h or --help flag.
  • Addressed a bug where in rare circumstances, fly intercept -c would find multiple check containers for the same resource.
  • Fixed an issue with the build process that caused fly to stop working for some users making use of a VPN with split DNS.
  • Fixed an issue where the -j or --json flag was not honored when running fly get-team.
  • When outputting sample commands to the terminal, fly is aware of the path from which it is being executed.
  • The get-team command for fly will now take team-name as an argument for the team name instead of just team.

Resources

Thse are the fixes to Concourse resources:

  • Git Resource:
    • Fixed a bug in the git resource where commits from extraneous branches could appear in the version history if a tag filter was specified.
  • BOSH Backup and Restore:
    • The BOSH Backup and Restore job for the database now works properly if the database configuration is provided via a BOSH link. Before, it would always use a hardcoded default if a user didn't explicitly specify, causing BOSH links to be ignored.
  • s3 Resource:
    • Fixed an issue where the s3 resource would break when disable_multipart: true was set in the source configuration, and a tracked file was smaller than the minimum upload part size for the configured s3-compatible blobstore.
  • Mercurial Resource:
    • Fixed hg.pull by passing the sourceUri in the Mercurial resource. This could fix some issues where Mercurial wasn't saving the full uri in the hgrc file.

Core Functionality

These are the fixes to the core functionality of Concourse:

  • Configured the web node's tolerance for idle connections to be more lenient: If a node has been using more than 32 of its available connections, up to 32 connections will be allowed to stay idly open.
    • Furthermore, the total max connection pool size has been made configurable - this should allow operators to avoid overloading the max connection limit on the database side.
  • Fixed a bug where operators were prevented from retiring team-scoped workers with the concourse retire-worker command.
  • Fixed a problem with one of the database migrations which could cause the web node startup to fail in some cases.
    • If you're upgrading from a version prior to v5.0.0 the upgrade should be a little more resilient.
  • Fixed a problem where timestamps weren't being properly returned when a get step or a put step finished.

Runtime

These are the fixes to the Concourse runtime:

  • Concourse worker process is now systemd-aware: This fixed a bug that could, on rare occasions, cause container limits not to be enforced or prevent containers from being killed.
  • Task caches are now decoupled from the worker(s) they are stored on: As a result, in multi-worker deployments, pipelines making use of task caches may consume duplicate storage across workers.
  • Fixed an issue where, when a pipeline was paused for an extended period of time, many of its resources' versions would disappear.
  • Fixed an issue where re-checking a resource would cause the metadata from its latest version to be erased.
  • When using the overlay volume driver In past releases, users would have to be cautious about preserving their mount table. If it had been cleared volumes would appear empty. Baggageclaim now has the capability to recover missing mounts when the process restarts.
  • Fixed a bug where if a user had a try step and they aborted while the hooked step was running, their whole web node would crash.
  • Fixed an issue where the logs for a Windows or Darwin worker get populated with irrelevant error messages.

Web UI

These are the fixes to the web UI of Concourse:

  • The web UI used to silently break when a user’s token (which includes a potentially-long JSON-encoded string detailing all the teams a user is a part of and what roles they have on them) was longer than the size of a single cookie (4096 bytes on most browsers). This limit has been increased.
  • Fixed a long-standing bug on the dashboard, where the existence of circular pipelines would cause the browser to crash. This was fixed in v5.2.0, but the fix caused a performance regression, which is now also fixed.
  • Fixed a bug, where, on a build step with attempts: specified, only one attempt would be possible to view on the build page.
  • Fixed a bug on the reaped build logs screen where a link was pointing to deprecated documentation. This screen is visible when you visit a build that is older than its job's configured retention policy.
  • Fixed a bug where the dashboard preview of a paused job with a pending build would show the color of the most recent build's status, when it should be blue.
  • Fixed a regression involving momentum based scrolling on build pages on iOS browsers. The fix was also applied to the sidebar and the dashboard.
  • Fixed improper formatting on the tag for the pipeline-operator role.
  • Fixed an issue where the sidebar becomes unclickable when viewing a build for which a user is unauthorized to see the logs.
  • Improved the styling on scrollbars so that they better match Concourse design language. As a result, they should no longer use browser defaults.
  • Fixed an issue where the pipelines in the sidebar could randomly rearrange.

v5.2.7

Release Date: March 10, 2019

Security Fixes

This release contains the following security fixes:

  • Updates the golang.org/x/crypto module from v0.0.0-20190313024323-a1f597ede03a to v0.0.0-20200220183623-bac4c82f6975 to address a recently reported security vulnerability in the ssh package:
    • CVE-2020-9283:
      • golang.org/x/crypto before v0.0.0-20200220183623-bac4c82f6975 for Go allows a panic during signature verification in the golang.org/x/crypto/ssh package. A client can attack an SSH server that accepts public keys. Also, a server can attack any SSH client.

v5.2.6

Release Date: December 18, 2019

Security Fixes

This release contains the following security fixes:

  • Updates the git resource to v1.6.3 to address a recently reported security vulnerability:
    • CVE-2019-19604:
      • Arbitrary command execution is possible in Git before 2.20.2, 2.21.x before 2.21.1, 2.22.x before 2.22.2, 2.23.x before 2.23.1, and 2.24.x before 2.24.1 because a "git submodule update" operation can run commands found in the .gitmodules file of a malicious repository.

v5.2.5

Release Date: November 19, 2019

Features

New features and changes in this release:

Core

These are the new features and changes to the core functionality of Concourse:

  • API endpoints have been changed to use a single transaction per request, so that they become "all or nothing" instead of holding data in memory while waiting for another connection from the pool. In earlier versions, this could lead to snowballing and increased memory usage as requests from the web UI (polling every 5 seconds) piled up.

v5.2.4

Release Date: September 30, 2019

Security Fixes

This release contains the following security fix:

  • Updates golang to v1.13.1 to address a recently reported issue with Go net/http:
    • CVE-2019-16276:
      • GoLang's net/http (through net/textproto) used to accept and normalize invalid HTTP/1.1 headers with a space before the colon, in violation of RFC 7230. If a Go server is used behind an uncommon reverse proxy that accepts and forwards but doesn't normalize such invalid headers, the reverse proxy and the server can interpret the headers differently. This can lead to filter bypasses or request smuggling.

v5.2.3

Release Date: September 1, 2019

Security Fixes

This release contains the following security fix:

  • Updates golang to v1.12.9 to address two CVEs:
    • High CVE-2019-9512, also known as Ping Flood:
      • The attacker sends continual pings to an HTTP/2 peer, causing the peer to build an internal queue of responses. Depending on how efficiently this data is queued, this can consume excess CPU, memory, or both, potentially leading to a denial of service.
    • High CVE-2019-9514, also known as Reset Flood:
      • The attacker opens a number of streams and sends an invalid request over each stream that should solicit a stream of RST_STREAM frames from the peer. Depending on how the peer queues the RST_STREAM frames, this can consume excess memory, CPU, or both, potentially leading to a denial of service.

v5.2.1

Release Date: August 20, 2019

Resolved Issues

This release has the following fix:

  • Fixes an issue where the web UI used to crash when your JSON Web token (JWT) was longer than the size of a single cookie. In most browsers, this is 4096 bytes.
    • If a user was a member of many teams, then their JWT was very long because it encoded all teams and roles for that user.
    • Now, this JWT is broken into multiple cookies to accommodate users who belong to hundreds of teams.

v5.2.0

Release Date: July 2, 2019

Pivotal Concourse v5.2.0 includes Role Based Access Control (RBAC).

Currently, there are five defined roles:

  • main team
  • team owner
  • team member
  • pipeline operator
  • team viewer

These roles evolved out of the common request of having a read-only team member. Also, a less powerful team member role was added. For more information about RBAC roles and how they impact authentication, see Set User Roles and Permissions.

Security Fixes

This release contains the following security fixes:

  • Fixed an information leak that potentially permits unauthenticated users to fetch the step names and structure for a build that has a non-public job. This scenario now returns a 401 Unauthorized error.
  • Resource metadata is no longer shown by default in exposed pipelines. The resource must be set to public to show metadata, similar to jobs. Build output is an exception. If a job is public, any get step or put steps still show in the build's metadata.
  • SSH MAC algorithms have been restricted to a smaller set to fix a vulnerability with the Concourse web instance VM. By default, golang allows for some weak algorithms that can lead to security vulnerabilities on port 2222, which is used for worker communication.

    For more information, see SSH Weak MAC Algorithms Enabled in the Tenable documentation. * To safeguard against clickjacking attacks, the web node now defaults X-Frame-Options to deny. Running Concourse in an iframe no longer works.

Upgrading

Warning

If you are currently using Pivotal Concourse v3.13.0, you must first upgrade to v4.2.4 before upgrading to v5.2.0.

This release contains the following changes to the manifest:

  • atc and tsa jobs are consolidated under web:

    • atc variables:
      • These are still at the root level of job, but are now under web (for instance, atc.bind_port is now web.bind_port).
      • Vault's app-role configuration now uses : instead of = (for example, CONCOURSE_VAULT_AUTH_PARAM uses role_id:some-id instead of role_id=some-id).
    • tsa properties:
      • These are no longer at the root level of a job (previously, tsa):
        • They are now all after worker_gateway (for instance, tsa.host_key is now web.worker_gateway.host_key)
      • web.worker_gateway.team_authorized_keys now takes a map rather than a list of objects.

    For more information on the new web properties used in v5.2.0, see web job in the BOSH documentation.
    For more information on the previous atc variables and tsa jobs used in v4.2.4, see atc job and tsa job in the BOSH documentation.

  • baggageclaim,worker, and garden jobs are consolidated under worker.

    • baggageclaim properties are now under worker.baggageclaim.
    • garden properties are now under worker.garden.

      Also, only a subset of the previously-exposed flags are able to be set. * worker.tsa properties are under worker.worker_gateway * baggageclaim-windows, worker-windows, and houdini-windows jobs are consolidated under worker-windows. * baggageclaim properties are now under worker.baggageclaim * worker.tsa properties are under worker.worker_gateway

Breaking Changes

This release has the following breaking changes:

Fly Commands

This breaking change affects fly commands:

  • The short flag for fly builds --team has been updated from -t to -n to make it consistent across fly.

Release

These breaking changes affect the Concourse release:

Warning

You must re-create your workers during or after deploying. The location where the worker stores volumes has changed and the old volume directory is not cleaned up. Failing to do so causes disk usage to leak.

Warning

The additional\_resource\_types property can no longer be configured.

  • The Concourse BOSH release has been redesigned and is now centered around the Concourse binary.

  • The Concourse release no longer needs to be deployed alongside a Garden-runC BOSH release. Instead, it ships alongside the gdn binary.

  • Concourse now uses bpm for deploying the web node. For more information about bpm, see the bpm-release repository in GitHub.
  • blackbox is no longer included. For more information about blackbox, see blackbox job in the BOSH documentation.
  • In v5.2.0, Concourse workers will not run on machines that do not have systemd installed. This is the case in single-machine test environments.

    For more information, see Common Pivotal Concourse Issues.

Binary

These breaking changes affect the Concourse binary:

  • The Concourse binary distribution has been reformatted. Instead of a self-contained binary, it is now shipped as a TGZ file containing the binary and its prepackaged dependencies. Garden flags are no longer supported because they rely on directly embedding their code. If you have been passing specific flags to Garden, you must switch to using a config file through --garden-config, or pass them as env vars, for example, CONCOURSE_GARDEN_FOO_BAR. The effects of this reformatting include:
    • Simpler and faster startup. The Concourse worker command no longer needs to extract resource types, amongst other things, on start.
    • The Concourse binary no longer directly embeds Garden-runC code. Instead, it ships alongside the gdn binary, copied from their releases. This simplifies the interface for configuring Garden and enables Concourse to leverage Garden's build process rather than risking deviation.
  • Generic credential caching has been implemented for all credential managers. This replaces Vault-only caching. As a result of this change, credential managers now implement a simpler interface that makes it easier to look up secrets in multiple paths. To make this transition, the following two flags must be updated:

    • --vault-cache is now --secret-cache-enabled
    • --vault-max-lease is now --secret-cache-duration

Core Functionality

These breaking changes affect the core functionality:

  • The concourse web --peer-url flag has been removed. Web nodes no longer need to stream user artifacts to one another as a result of internal refactoring and decoupling between various components. The --peer-address flag has been added because the SSH gateways that run on the web node need their address for the forwarded worker connections to be advertised to other web nodes.

This value used to be inferred by ‘--peer-url’.

  • Two flags have been modified to be more consistent with other flag syntax:
    • concourse web --vault-auth-param foo=bar is now concourse web --vault-auth-param foo:bar
    • concourse web --tsa-team-authorized-keys team=path/to/key is now concourse web --tsa-team-authorized-keys team:path/to/key

Resources

This breaking change affects Concourse resources:

  • The version history for each resource across your pipelines is reset when you upgrade to Pivotal Concourse v5.2.0.

For more information about the changes to how resource versions are stored, see Resources in the Features section below.

Teams

This breaking change affects team functionality:

  • The --allow-all-users flag has been removed because it was often misused. You must now configure users explicitly. After upgrading, any teams that have --allow-all-users configured will continue to allow all users. The next time a team is configured, you must specify another flag because --allow-all-users is no longer available.

Runtime

This breaking change affects the Concourse runtime:

  • The --peer-ip flag is no longer available on concourse workers. Support for direct worker registration has been removed. When updating to v5.2.0, remove this flag because the worker registers through forwarding instead. Forwarding is more secure because it does not require you to open your workers to inbound traffic.

Features

New features and changes in this release:

Fly Commands

These are the new features and changes to fly commands:

  • fly login has an improved landing page and workflow. When you log in to a remote session, if your token transfer fails, you can now copy the token. The auto-login prompt also no longer asks for your token.
  • fly login with the -u and -p flags no longer prompts you for your authentication method. The command assumes that you are using local authentication.
  • fly set-pipeline prints a fly unpause-pipeline command after creating a pipeline. This enables you to unpause new pipelines. New pipelines are paused by default.
  • fly set-pipeline prints changes in the order of Grouping Jobs in the diff.
  • fly intercept with the --handle flag inspects the given Garden container.
  • fly prune-worker with the --all-stalled or -a flag prunes all the stalled workers. It also shows a warning when no stalled workers are found.
  • fly execute uploads inputs and downloads outputs in parallel.
  • fly execute with the -m flag enables you to specify input mappings. You can use this flag when a job renames inputs with the --inputs-from-job flag.
  • fly watch with the --timestamps flag shows timestamps in the build output.
  • fly get-pipeline prints resources and resource types in a deterministic order.
  • fly status prints the expiration status of your token.
  • Adds the following fly commands:
  • fly curl enables manual API requests to Concourse.
  • fly userinfo prints the teams you are logged in to and what roles you have in each team.
  • fly-land-worker lands workers remotely. This starts the landing process through the API and exits the worker process.
  • fly-edit-target edits the target’s name, team or URL.
  • fly-delete-target removes targets from the ~/.flyrc file.
  • fly get-team retrieves a team’s configuration.

Resources

These are the new features and changes to Concourse resources:

  • Adds a registry-image resource: The registry-image resource has been added to the core. This resource replaces the docker-image resource for image pulling and pushing. registry-image does not replace image building.

    This resource improves on the docker-image resource in the following ways:

    • It does not run the Docker daemon to fetch the image because it is written in pure Go.
    • Because it does not run the Docker daemon, it does not need a privileged container.

These improvements result in faster, more efficient, and more resilient image fetching. Pivotal recommends that users switch their image_resources and Resource Types to this new resource type. In most cases, you can do this by replacing type: docker-image with type: registry-image.

  • Global resources beta feature: Global resources share detected resource versions between all resources that have the same type and source configuration.

    To enable this feature, set enable_global_resource to true in your web job.

    For more information about global resources, see Global Resources in the open source Concourse Documentation.

    For more information about enable_global_resource, see web.enable_global_resource in the BOSH documentation.

  • Concourse BOSH Release Ubuntu image: The BOSH release for Concourse packages Ubuntu images for each core resource type. Before, the BOSH release packaged Alpine images. This change has been made for compliance reasons.

Runtime

These are the new features and changes to the Councourse runtime:

  • Containers and volumes are garbage-collected in parallel: Workers used to garbage-collect containers in volumes sequentially and destroyed containers before volumes. This meant that if a worker had many volumes to remove, the worker destroyed them one by one and containers were not garbage-collected during this time.

    Running garbage collection in parallel speeds up garbage collection and prevents an imbalance between volume and container counts from slowing each other down. This is important as workers are typically capped at 250 containers, but might have more volumes and a slow disk.

    The default max-in-flight for garbage collection is 3 containers and 3 volumes at a time.

  • You can configure web nodes with a fewest-build-containers strategy: This places containers on workers that have the fewest build containers.

  • You can configure workers to rebalance: This prevents workers from being forwarded through a single web node. To enable this, set the --rebalance-interval flag on the Concourse worker.

    Rebalancing workers drains in-flight connections and does not disrupt in-flight builds. This results in a breaking change.

    For more information, see Breaking Changes - Runtime above. * Volumes and containers that disappear from their worker are automatically removed from the database: This makes it easier for Concourse to recover from this situation instead of erroring with file not found or unknown handle errors. * Tasks can have inputs, outputs, and caches with overlapping paths. * Adds an in_parallel step: The in_parallel step can run steps in parallel using the following properties: * limit: This property limits the number of parallel steps. * fail_fast: This property interrupts running steps and prevents scheduling pending steps.

    Note

    This release deprecates the aggregate step.

    Pivotal recommends that users use in_parallel.

  • Improved error messages for when a task file property specifies an unknown artifact source.

Core Functionality

These are the new features and changes to the core functionality of Concourse:

  • Web nodes can be configured to enable audit logs: Auditing logs API calls to the default logger using flags to enable specific auditing groups. For more information about audit logs, see Enabling audit logs in the open source Concourse documentation.
  • Steps can have an on_error step hook: The value for the on_error step hook is a second step to execute only if the parent step errors.
  • Improvements for task ((vars)):
  • You can provide values for task ((vars)) when using the fly execute command.
  • You can provide values to a task step in a pipeline using vars.
  • You can add task ((vars)) in the task config in properties besides image_resource.
  • Pipeline ((params)) are now referred to as pipeline ((vars)).
  • You can give the put step a list of inputs to use: You can use this rather than having the put step use all of the inputs. This can speed up builds that have a large amount of artifacts before a put.
  • Concourse automatically retries fetching credentials when the request to the credential manager fails: By default, Concourse retries five times and waits one second between each attempt. You can adjust this with the --secret-retry-attempts and --secret-retry-interval flags on Concourse web.
  • Concourse is compatible with CredHub v2.x, except v2.1.
  • You can configure Vault credential manager with a global shared path for credential lookup: This makes sharing credentials between teams easier to manage. Pivotal recommends that you use caution when using this feature because it enables all teams to have access to the credentials.
  • Concourse keeps build history when you rename a job: If you rename a job, Concourse updates the job name and specifies the old name as old_name within the job config.
  • Build logs are kept for a specific, configured time duration.
  • Adds a --external-garden-url flag: This flag enables you to use a separately managed Garden server as a worker.
  • The Concourse worker command can be given a --garden-use-houdini flag on Linux: This enables the use of a "no-op" Houdini Garden backend. You can use this if you do not want containerization.
  • Improved Concourse components logs: If you use a tool to parse your logs, you must update your tool.

Web UI

These are the new features and changes to the Concourse web UI:

  • Resource pinning has replaced resource pausing: When you upgrade to Concourse v5.2.0, paused resources automatically change to their pinned equivalent. Concourse pins the paused resource to the most recent available version. A comment is left on any migrated resources so that changes are visible to pipeline users. Versions can be individually pinned using the web UI or the pipeline config. Pinned resources skip resource checking. If resource checking does happen, the resource still stays pinned to the desired version. Comments can be left on pinned versions to explain why the resource is pinned.
  • Re-added the pipeline navigation sidebar.
  • Resources have an icon property: This enables you to add icons to your pipeline. You can use this property to distinguish between different resource types.
  • You can add a Concourse cluster name to the dashboard page: You can set a name for a Concourse cluster with the --cluster-name flag. This name is displayed on the dashboard page.
  • Concourse uses Material Design icons everywhere in the UI.
  • You can click URLs in resource metadata.
  • Pipelines have a play and pause button at the top bar: You do not have to go back to the dashboard page to use the play and pause button. The play and pause button is only enabled for authorized users. A tooltip appears for users who do not have access.
  • Relative timestamps in the build header convert to absolute timestamps after 24 hours.
  • Adds a tooltip for yellow get steps: The web UI now explains why some get steps have a yellow icon in a tooltip. The dashboard page uses an “eye” icon to indicate that you are seeing a pipeline because it is exposed.
  • Adds duration information for steps in the build log: You can hover over the right side of a step in the build log to view how long it took to initialize and run.
  • Improved animation for running builds in the build number list: This animation helps to differentiate between errored and running builds.

Teams

These are the new features and changes to team functionality:

  • Generic OAuth can be configured with different user ID and name keys.
  • Generic OIDC authentication can be configured with a different username key.
  • BitBucket authentication support has been re-introduced.

Resolved Issues

This release fixes the following issues:

Fly Commands

These are the fixes to fly commands:

  • fly get-pipeline now throws an error if the given pipeline does not exist. Before, the command returned an empty pipeline config.
  • fly execute with the -j flag now prints the correct build URL.
  • fly execute with the --include-ignored flag no longer fails when files are removed locally.
  • fly execute with the --input flag no longer hangs if the web node stops.
  • fly login now creates the ~/.flyrc file with chmod 0600 file system permissions.
  • fly intercept now prints clearer error messages when it fails to execute.
  • fly set-pipeline with the --check-creds flag no longer fails.

Core Functionality

These are the fixes to the core functionality of Concourse:

  • Multiple groups in the same pipeline can no longer use the same name. If you use the same name, an error occurs.
  • Aborting a started build before a web node re-attaches to it no longer results in an orphaned build that continued to run.
  • The Concourse API now returns a 401 Unauthorized when an expired or invalid token is used to access an endpoint which supports authenticated and unauthenticated views.
  • concourse web ensures that --session-signing-key is specified. Before, concourse web crashed if --session-signing-key was not specified.
  • Credential managers no longer instantiate twice. This resulted in two authentication loops.
  • Builds no longer crash when a resource is removed from the pipeline configuration while the build is running. This happened for builds that produced outputs for the removed resource.
  • UNIQUE constraints for resource_configs are no longer ineffective. For more information, see the `resource_configs` unique constraint is ineffective #2509 issue in GitHub.
  • The /api/v1/resources and /api/v1/jobs endpoints now return [] instead of null when there are no resources or jobs.
  • Auth configs set from empty environment variables no longer result in incorrect Dex configurations.
  • Concourse now sends TCP keepalives for connections to the database. This enables Concourse to detect when the connection has been interrupted ungracefully.
  • The --tsa-authorized-keys flag is now optional. You can use this flag when all authorized keys are associated to teams through --tsa-team-authorized-keys.
  • When configured to drain build logs to syslog, the web node no longer leaks a connection and goroutine for each build.
  • version parameters in a get step now take precedence over version parameters pinned through the web UI and version parameters in a resource definition. For information about version in get steps, see get step in the open source Concourse documentation.

    For information about version in resource definitions, see Resources in the open source Concourse documentation.

Resources

Fixed a resource error that caused the following error:

worker_resource_config_check__resource_config_check_sessio_fkey references unreticulated spline

Runtime

These are the fixes to the Concourse runtime:

  • The web node now retries on unexpected end-of-file errors. This occurred when a worker was restarted while a build was running a container on the worker. For information about web nodes, see Running a web node in the Concourse documentation.
  • The scheduler no longer starts a manually-triggered build until each resource last checked timestamp is after the build created at timestamp. Before, manually-triggered builds caused resource checking to happen in the job scheduling loop. This ensured that manually-triggered builds ran with the latest versions available. However, it also slowed down scheduling for every other job in the pipeline because they are all scheduled in succession.
    • The above refactoring also fixed a race condition that resulted in inputs configured with version: having every version skipped when a build is manually triggered.
  • Task caches are now supported on Windows.

Web UI

These are the fixes to the web UI of Concourse:

  • Dashboard search has been improved in the following ways:
    • Team name autocomplete now works even if you are not logged in.
    • Fixed the unstyled autosuggest menu in Chrome.
    • Hitting the escape key now un-focuses the search field.
    • Search autocomplete now only appears if you press a key with the search field focused.
    • Typing ? into the search field no longer brings up the hotkey help pane.
  • The dashboard no longer crashes when a pipeline is configured with a circular dependency.
  • Improved the HD dashboard view.
  • Improved rendering for pipeline groups.
  • Tall pipelines no longer are cut off by the top bar.
  • Fixed the status:running search functionality on the dashboard view.
  • When viewing a pipeline build by ID, the top bar now shows the breadcrumb for its pipeline and job.
  • The breadcrumb in the top bar now uses hyperlinks. You can middle-click and right-click the hyperlinks. This results in predictable browser behavior.
  • The groups bar on the pipeline view now has a hover state for each group.
  • When viewing a one-off build in the web UI, the build now renders. Before, it was sending errors.
  • When the dashboard is more than one page, using the vertical scroll no longer produces a horizontal scroll that continually grows.
  • When viewed on a mobile browser, the black header bar now is sticky.
  • The text legibility and anti-aliasing in the web UI has been improved.
  • Dashboard behavior has been improved when there are no pipelines in the following ways:
    • You can now view which team you are a member of. Before, you could only view a no pipelines set page.
    • The bar along the bottom now shows up.
    • The search function no longer is shown.
    • The HD view has been disabled and now redirects.
  • When viewed on a mobile browser, the username in the header now displays correctly.
  • When the width of resource metadata is large, the text is cordoned off to the side. Before, resource metadata was compressed into the resource get output.
  • The URL link to the web app manifest no longer is relative to the URL. Before, the link was broken on all pages except root URL.
  • The position of the no results text has been fixed when searching on the dashboard.
  • When viewing a build, only the first batch of builds are rendered. Older builds are automatically rendered if the build currently being viewed is old or if the user scrolls to view them.

Teams

These are the fixes to team functionality:

  • fly login checks to ensure you have successfully logged in to the target team and returns an error if the team is not in your token.
  • Users who are members of many teams can now access all of their teams. Before, the Cloud Foundry Dex connector did not iterate over all results when requesting a user's org and space for team authorization.
  • If an OAuth provider is configured for Concourse, a user that is not a member of any team can no longer log in to Concourse. However, when these users could log in, they could not do anything.

Credentials

These are the fixes that relate to credentials in Concourse:

  • The AWS SSM credential manager and the AWS Secrets Manager credential manager no longer declare the AWS_REGION environment variable as their own. Before, if you set these environment variables, both credential managers tried to configure them and failed. The AWS SSM and AWS Secrets Manager credential managers now have AWS_REGION environment variables in different namespaces.
  • The Vault login retry logic no longer causes Vault to go into a fast loop after reaching the maximum interval. It now stays at the maximum interval.

Installation

These are the fixes that relate to Concourse installation:

  • The BOSH release now sets file system permissions for its config values to chmod 0600. This fixes PostgreSQL certificate configuration.
  • The BOSH release now correctly handles array values for authorized worker keys.