Frequently Asked Questions

Page last updated:

How do Cloud Native Buildpacks (CNBs), kpack, and Tanzu Build Service overlap and differ?

CNBs are build tools that adhere to the CNB v3 Specification and transform source code into an OCI compliant runnable image. The v3 specification, lifecycle, and local CLI (pack) are governed by the open source Cloud Native Buildpacks project.

kpack is a collection of open source resource controllers that together function as a Kubernetes native build service. The product provides a declarative image type that builds an image and schedules image rebuilds when dependencies of the image change. kpack is a platform implementation of CNBs in that it utilizes CNBs and the v3 lifecycle to execute image builds.

Tanzu Build Service is a commercial product owned and operated by VMware that utilizes kpack and CNBs. Build Service provides additional abstractions intended to ease the use of the above technologies in Enterprise settings. These abstractions are covered in detail throughout the documentation on this site. Additionally, customers of Build Service are entitled to support and VMware Tanzu buildpacks.

Why do I see two images in the image registry after a successful build?

By default Build Service will tag each built image twice. The first tag will be the configured image tag. The second tag will be a unique tag with the build number and build timestamp. The second tag is added to ensure that previous images are not deleted on registries that garbage collect untagged images.

How does TBS work in air gapped environments?

Build Service is installed and deployed using Carvel tools. Therefore, the imgpkg copy command can create a .tar file composed of the kubernetes config and images required to successfully install Build Service. The imgpkg copy command also ensures that all the images can be relocated to air-gapped registries, and by providing the credentials to the air-gapped registry when executing the kapp install command, Build Service can then use that secret to pull images from said registry, hence working in air-gapped environments.

Currently, kbld package and kbld unpackage must be used to import dependencies to an air-gapped environment.

For more details on air-gapped installation, see Installation to Air-Gapped Environment.

For more details on air-gapped builds, see Offline Builds.

Is there documentation on supported Tanzu Buildpacks?

Yes, documentation is available on Tanzu Buildpacks Documenation.

Why do I get an X509 error from Build Service when trying to create an image in my registry?

When interacting with a registry or a git repo that has been deployed using a self signed certificate, Build Service must be provided with the certificate during install time. Unfortunately, you will either need to target a registry that does not have self signed certificates or re-install Build Service to work with this registry.

How do I configure a secret to publish images to Dockerhub?

  1. Create a dockerhub secret with the kp cli:

    kp secret create my-dockerhub-creds --dockerhub DOCKERHUB-USERNAME

    Where DOCKERHUB-USERNAME is your dockerhub username You will be prompted for your dockerhub password

How can I configure an image to pull from a private GitHub repository?

  1. Create a github secret with the kp cli:

    Using a git ssh key

    kp secret create my-git-ssh-cred --git --git-ssh-key PATH-TO-GITHUB-PRIVATE-KEY

    Where PATH-TO-GITHUB-PRIVATE-KEY is the absolute local path to the github ssh private key

    Or with a basic auth github username and password

    kp secret create my-git-cred --git --git-user GITHUB-USERNAME

    Where GITHUB-USERNAME is your github username You will be prompted for your github password

Why do some builds fail with “Error: could not read run image: *”?

The run image must be publicly readable or readable with the registry credentials configured in a project/namespace.

To see where the build service run image is located run: kp stack status STACK-NAME.

If you cannot make the run image publicly readable, you must kp to create a registry secret within the namespace where your builds reside. This can be accomplished using kp secret create.

Why don’t my image builds appear in my Harbor v1.X.X registry?

There is a known bug in Harbor that, at times, prevents the UI from showing images. If you are unable to see a recently built image in the Harbor UI, try pulling it using the docker CLI to verify that it exists.

How do I fix “unsupported status code 500” when creating a builder on my Harbor v2.X.X registry?

Some builders are very large and can overwhelm Harbor’s default database connection. You can remediate this issue by increasing the database.maxOpenConns setting in the helm values.yaml file. Increase this value from 100 to 300. The exact setting can be found here.

How do I configure credentials for using gcr as my installation registry?

You can use Google Container Registry for your Tanzu Build Service installation registry.

If you have trouble configuring the registry credentials for gcr when following the install docs, use the following to set the gcr credentials:

registry_password="$(cat /path/to/gcp/service/account/key.json)"
ytt -f /tmp/bundle/values.yaml \
    -f /tmp/bundle/config/ \
    -v docker_repository='<IMAGE-REPOSITORY>' \
    -v docker_username="$registry_name" \
    -v docker_password="$registry_password" \
    | kbld -f /tmp/bundle/.imgpkg/images.yml -f- \
    | kapp deploy -a tanzu-build-service -f- -y

Can I configure a proxy for my Tanzu Build Service?

TBS can be configured with a proxy at installation time by specifying additional parameters:

  • http_proxy: The HTTP proxy to use for network traffic.
  • https_proxy: The HTTPS proxy to use for network traffic.
  • no_proxy: A comma-separated list of hostnames, IP addresses, or IP ranges in CIDR format that should not use a proxy.

Note: When proxy server is enabled using http_proxy and/or https_proxy, traffic to the kubernetes API server will also flow through the proxy server. This is a known limitation and can be circumvented by using no_proxy to specify the kubernetes API server.

ytt -f /tmp/bundle/values.yaml \
    -f /tmp/bundle/config/ \
    -v docker_repository='<IMAGE-REPOSITORY>' \
    -v docker_username='<REGISTRY-USERNAME>' \
    -v docker_password='<REGISTRY-PASSWORD>' \
    -v http_proxy='<HTTP-PROXY-URL>' \
    -v https_proxy='<HTTPS-PROXY-URL>' \
    -v no_proxy='<KUBERNETES-API-SERVER-URL>' \
    | kbld -f /tmp/bundle/.imgpkg/images.yml -f- \
    | kapp deploy -a tanzu-build-service -f- -y

How do I build my app locally using kpack builders?

You can use the pack cli with your kpack builders to test them locally before checking in your code. By using your kpack builder locally, you can guarantee that the buildpacks, stacks, and lifecycle used to build the image config will also be used by the pack CLI, resulting in a container image that is the exact same, whether it is built by kpack or pack.

Note: Make sure that you `docker login` to the image repository containing your kpack builder.

pack build my-app --path ~/workspace/my-app --builder --trust-builder

What can I do with the kp –dry-run and –output flags?

From kp CLI v1.0.3+ the --dry-run and --output flags are made available to kp commands that create or update any kpack Kubernetes resources.

The --dry-run flag lets you perform a quick validation with no side-effects as no objects are sent to the server. And the --output flag lets you view the resource in yaml or json format.

The --dry-run-with-image-upload flag is similar to the --dry-run flag in that no kpack Kubernetes resources are updated. This flag is provided as a convenience for kp commands that can output Kubernetes resource with generated container image references.

For example, consider the command below

$ kp clusterstack create test-stack \
 --dry-run \
 --output yaml \
 --build-image \

Creating ClusterStack... (dry run)
Uploading to ''... (dry run)
    Skipping ''
    Skipping ''
kind: ClusterStack
  creationTimestamp: null
  name: test-stack
  id: io.buildpacks.stacks.bionic
  buildImage: {}
  runImage: {}

The resource yaml output above has the relocated build and run image urls. However, the images were never uploaded.

If you now apply the resource output using kubectl apply -f as shown below, then the resource will be created but will be faulty since the referenced images do not exist.

$ kp clusterstack create test-stack \
 --dry-run \
 --output yaml \
 --build-image \
 --run-image \
 | kubectl apply -f -
Creating ClusterStack... (dry run)
Uploading to ''... (dry run)
    Skipping ''
    Skipping '' created

Running the same command above with the --dry-run-with-image-upload flag (instead of --dry-run) ensures the created resource refers to images exist.

Does TBS support Azure Devops for git repositories

Yes! Azure DevOps Git is fully supported as of TBS 1.2

Why do I get a “repository does not exist” error when I use ECR Registry?

ECR is supported but requires manually creating each repository that TBS will use. With other registries, the repositories will be created automatically.

How do I troubleshoot a failed build?

Like many Kubernetes native products, operating TBS involves orchestrating resources that depend on each other to function. If a resource is in a “not ready” state it is likely that there is a problem with one of the resources it depends on.

If you are encountering a not ready Image, check and see which builder it uses and then check the status of that builder for additional information that could help you troubleshoot the problem. “` $ kp image status

$ kp clusterbuilder status ”`

Similarly, if a builder resource is in a “not ready” state, it is possible that there is a problem with the clusterstack or clusterstore resources it is referencing.

$ kp clusterstack status <clusterstack-name> --verbose

$ kp clusterstore status <clusterstore-name> --verbose

All Build Service concepts are also Kubernetes resources. Therefore, customers can interact with them using the kubectl CLI to see all the information that can be provided by the Kubernetes API.

$ kubectl describe image <image-name>

$ kubectl describe clusterbuilder <clusterbuilder-name>

How do I troubleshoot an UNAUTHENTICATED error?

During imgpkg copy

  1. Ensure you are logged in locally to both registries with:
docker logout && docker login
docker logout <tbs-registry> && docker login <tbs-registry>
  1. On linux, if you have installed docker with snap you will need to copy /root/snap/docker/471/.docker/config.json to ~/.docker/config.json which is where imgpkg is looking for the docker credentials
  2. Ensure your credentials have write access to your registry with docker push <registry>/<build-service-repository> this is the same repository used during install with the ytt/kapp command

During kp import

  1. Ensure you are logged in locally to both registries with:
docker logout && docker login
docker logout <tbs-registry> && docker login <tbs-registry>
  1. Ensure the credentials used to install TBS have write access to your registry as they sometimes differ from local credentials
    • Use docker login <tbs-registry> using the credentials used to install TBS with ytt/kapp
    • Try to docker push <tbs-registry>/<build-service-repository> this is the same repository used during install with the ytt/kapp command

Why does TBS leave behind pods after builds on my Cluster?

All TBS builds happen in pods. By default, TBS will not delete the last ten successful builds and the last ten failed builds for the purpose of providing historical logging and debugging. If this behavior is not desired, users can configure the number of stored build pods by modifying the failedBuildHistoryLimit and successBuildHistoryLimit on the Image resource. This is not currently supported in the kp CLI, but users can apply yaml configuration using kubectl to update these fields. Follow this link for documentation.

How do I check what version of TBS I am using?

After successfully installing tanzu-build-service In terminal run the command kubectl describe configMap build-service-version -n build-service

Under the data field you will see the version of TBS you are currently using. EX: data: version: 1.2.0

Note: This will only work for TBS versions 1.2 and above

How does TBS use windows-based images?

When running imgpkg copy, the command will output the following message:

Skipped layer due to it being non-distributable. If you would like to include non-distributable layers, use the --include-non-distributable flag

This is because TBS ships with windows images to support windows builds. Windows images contain “foreign layers” that are references to proprietary windows layers that cannot be distributed without proper Microsoft licensing.

By default, imgpkg will not relocate the proprietary windows layers to your registry. TBS also will not pull any windows layers to the cluster unless windows builds are being run so if you do not need windows this message can be ignored.