Using kpack
Page last updated:
This topic describes kpack, a collection of open source resource controllers and custom resources that contain declarative build logic. It also describes how to set up and use kpack.
Overview
The kpack integration is a declarative build system that allows you to have more granular control over how Build Service builds app images.
A declarative build system is a system to which you declare the desired result of a build, which the system then builds on its own without direction. For example, when you give the system the names of the libraries a project depends upon, the system fetches the libraries and finds the source classes on its own. This is different from an imperative build system, which requires you to run commands to tell it how to do what you want to accomplish.
The kpack integration can be used with Build Service, with other services, or on its own to build app images. For more information on kpack-only installations, see the Install Build Service as a kpack-Only Installation section of the Installing and Configuring Build Service topic.
Prerequisites
This documentation assumes you have:
An installation of kpack. To install kpack, see Installing kpack on GitHub.
A Kubernetes cluster. To create a Kubernetes cluster, see Using Minikube to Create a Cluster in the Kubernetes documentation.
The
kubectl
CLI. To download thekubectl
CLI, see Install and Set Up kubectl in the Kubernetes documentation.A Docker v2 registry. To create and run a Docker registry, see Docker Registry in the Docker documentation.
An installation of the kpack log utility. To install and use the kpack log utility, see kpack logs on GitHub.
Understand the Image Resource
The image
resource is what kpack uses to declaratively build app images. It is contained within a YAML file.
The image
resource YAML file contains these fields:
tag
: The image tag.builder
: Configuration of thebuilder
resource the image builds use. For more information, see Builder Configuration.serviceAccount
: The service account name that is used for credential lookup.source
: The source code that is monitored and built into images. For more information, see Source Configuration.cacheSize
: The size of the volume claim that the build cache uses.failedBuildHistoryLimit
: The maximum number of failed builds for an image that is retained.successBuildHistoryLimit
: The maximum number of successful builds for an image that is retained.imageTaggingStrategy
: Allow for builds to be additionally tagged with the build number. Valid options areNone
andBuildNumber
.build
: Configuration that is passed to every image build. For more information, see Build Configuration.
Configure the Builder
The builder
field in the image resource describes the builder that builds the OCI images for a provided image configuration.
To configure the builder
field:
Define the
builder
field in your image resource YAML file in one of these ways:Cluster builder
builder: name: CLUSTER-BUILDER-NAME kind: ClusterBuilder
Where:
CLUSTER-BUILDER-NAME
is the name of the ClusterBuilder resource in Kubernetes.ClusterBuilder
is the type as defined in Kubernetes. This value is alwaysClusterBuilder
.
Namespaced builder
builder: name: BUILDER-NAME kind: Builder
Where:
BUILDER-NAME
is the name of the Builder resource in Kubernetes.Builder
is the type as defined in Kubernetes. This value is alwaysBuilder
.
Note: This image can only reference builders defined in the same namespace. This is not true for cluster builders because they are not namespace-scoped.
Configure the Source
The source
field in the image resource is a composition of a source code location and a subpath
.
To configure the source
field:
Configure the
source
field in your image resource YAML file in one of these ways:To use a Git repository, configure the
source
field with these variables:source: git: GIT-REPOSITORY url: GIT-REPOSITORY-URL revision: GIT-REVISION subPath: SUBDIRECTORY
Where:
GIT-REPOSITORY
is the Git repository that contains the source code.GIT-REPOSITORY-URL
is the Git repository URL. Currently, only HTTPS repositories are supported.GIT-REVISION
is the Git revision you want to use. This value may be a commit SHA, branch name, or tag.SUBDIRECTORY
is a subdirectory within the source folder where the app code resides. This can be ignored if the source code resides at theroot
level.
To use a blob, configure the
source
field with these variables:source: blob: BLOB url: BLOB-URL subPath: SUBDIRECTORY
Where:
BLOB
is the blob in a blobstore that contains the source code.BLOB-URL
is the URL of the source code blob. Either this blob must be publicly accessible, or the access token must be contained in the URL.SUBDIRECTORY
is the subdirectory within the source folder where the app code resides. This can be ignored if the source code resides at theroot
level.
To use a registry, configure the
source
field with these variables:source: registry: OCI-IMAGE image: IMAGE-LOCATION imagePullSecrets: - name: SECRET-NAME-LIST subPath: SUBDIRECTORY
Where:
OCI-IMAGE
is the OCI image in a registry that contains the source code.IMAGE-LOCATION
is the location of the source image.SECRET-NAME-LIST
is a list ofdockercfg
ordockerconfigjson
secret names. This is required if the source image is private.SUBDIRECTORY
is a subdirectory within the source folder where the app code resides. This can be ignored if the source code resides at theroot
level.
Configure the Build
You can use the build
field in the image
resource to configure environment variables required during the build process and resource limits on CPU
and memory
.
To configure the build
field:
Configure these variables in your image resource YAML file:
build: env: - name: ENV-VAR-NAME value: ENV-VAR-VALUE resources: limits: cpu: CPU-LIMIT memory: MEMORY-LIMIT requests: cpu: CPU-REQUEST memory: MEMORY-REQUEST
Where:
ENV-VAR-NAME
is the name of the environment variable.ENV-VAR-VALUE
is the value of the environment variable.CPU-LIMIT
is the amount of CPU the build is allowed to use.MEMORY-LIMIT
is the amount of memory the build is allowed to use.CPU-REQUEST
is the maximum CPU capacity of the build.MEMORY-REQUEST
is the maximum memory capacity of the build.
For more information about setting environment variables, see Define Environment Variables for a Container and the Resource requests and limits of Pod and Container section of the Managing Compute Resources for Containers topic in the Kubernetes documentation.
Create a Builder Resource
There are two types of builder resources that kpack supports. You must create these builder resources before you create any image resource, because they define which builder is used to create these images.
To create a builder resource:
Configure the builder resource in your image resource YAML file in one of these ways:
Namespaced builder: This is a builder that exists inside a namespace and cannot be shared between namespaces. This property allows you to push the image of the builder to a private registry by giving the option to set
ImagePullSecret
. For example:apiVersion: build.pivotal.io/v1alpha1 kind: Builder metadata: name: BUILDER-NAME namespace: NAMESPACE spec: image: BUILDER-IMAGE-TAG updatePolicy: UPDATE-POLICY imagePullSecrets: - name: BUILDER-SECRET
Where:
BUILDER-NAME
is the name of the builder that the image references.NAMESPACE
is the namespace where the builder is created.BUILDER-IMAGE-TAG
is the builder image tag.UPDATE-POLICY
is the update policy of the builder. Valid options arepolling
andexternal
.external
requires you to update the resource by applying a new configuration, whereaspolling
automatically checks every five minutes to see if a new version of the builder image exists.BUILDER-SECRET
is an optional parameter that you should use only if the builder image is in a private builder that require credentials to access. To create this secret, see the Create a Secret based on existing Docker credentials section of the Pull an Image from a Private Registry topic in the Kubernetes documentation.
Cluster builder: This type of builder can be used inside any namespace. The image must exist in a publicly accessible registry. For example:
apiVersion: build.pivotal.io/v1alpha1 kind: ClusterBuilder metadata: name: CLUSTER-BUILDER-NAME spec: image: BUILDER-IMAGE-TAG updatePolicy: UPDATE-POLICY
Where:
CLUSTER-BUILDER-NAME
is the name of the builder that the image references.BUILDER-IMAGE-TAG
is the builder image tag.UPDATE-POLICY
is the update policy of the builder. Valid options arepolling
andexternal
.external
requires you to update the resource by applying a new configuration, whereaspolling
automatically checks every five minutes to see if a new version of the builder image exists.
For more information about builders, see Builders in the kpack repository on GitHub.
Create an Image Resource
There are several resource types that work together to produce an image resource. This section describes how to configure and create each resource in your image resource YAML file.
The YAML resources below each contain a name
field. You should reference these names when configuring other resource types.
To apply a particular resource type:
Run:
kubectl apply -f RESOURCE-PATH.yml
To configure and create a resource:
Create a namespace inside which all of the builds are to run:
apiVersion: v1 kind: Namespace metadata: name: NAMESPACE
Where
NAMESPACE
is the name you give the namespace.Create a secret so kpack can push image builds to your desired registry. The registry uses the value in the
name
field to create a service account.Note: The
annotations
field must include the registry prefix for its corresponding registry. For Docker Hub, this should beindex.docker.io
. For GCR, this should begcr.io
.For GCR:
apiVersion: v1 kind: Secret metadata: name: GCR-BUILD-NAME namespace: NAMESPACE annotations: build.pivotal.io/docker: gcr.io type: kubernetes.io/basic-auth stringData: username: USERNAME password: PASSWORD
Where:
GCR-BUILD-NAME
is the build name.NAMESPACE
is your namespace.USERNAME
is the username you define for the build.PASSWORD
is the password you define for the build.
For Docker Hub:
apiVersion: v1 kind: Secret metadata: name: DOCKER-BUILD-NAME namespace: NAMESPACE annotations: build.pivotal.io/docker: index.docker.io type: kubernetes.io/basic-auth stringData: username: USERNAME password: PASSWORD
Where:
DOCKER-BUILD-NAME
is the build name.NAMESPACE
is your namespace.USERNAME
is the username you define for the build.PASSWORD
is the password you define for the build.
For more information about Docker registry secrets, see Secrets in the kpack repository on GitHub.
Create a secret to give the build pull access from the desired Git repository. The example below is for a GitHub repository. You can also specify a personal access token. If you are building against source code that is stored in a public registry, you do not need to configure a Git secret.
apiVersion: v1 kind: Secret metadata: name: GIT-BUILD-NAME namespace: NAMESPACE annotations: build.pivotal.io/git: https://github.com type: kubernetes.io/basic-auth stringData: username: USERNAME password: PASSWORD
Where:
GIT-BUILD-NAME
is the build name.NAMESPACE
is your namespace.USERNAME
is the username you define for the build.PASSWORD
is the password you define for the build.
For more information about Git registry secrets, see Secrets in the kpack repository on GitHub.
Apply each of these credentials to your Kubernetes cluster by running:
kubectl apply -f SECRET-FILE.yml
Create a service account that uses the Docker registry secret and the Git registry secret. When configuring the image resource, include the
name
of your Docker registry secret and thename
of your Git repository secret.apiVersion: v1 kind: ServiceAccount metadata: name: SERVICE-ACCOUNT namespace: NAMESPACE secrets: - name: DOCKER-BUILD-NAME - name: GIT-BUILD-NAME
Where:
SERVICE-ACCOUNT
is the name of the service account.NAMESPACE
is your namespace.DOCKER-BUILD-NAME
is the name of your Docker registry secret.GIT-BUILD-NAME
is the name of your Git repository secret.
Apply this service account to your Kubernetes cluster by running:
kubectl apply -f SERVICE-ACCOUNT.yml
Go to the sample-java-app repository on GitHub and fork it to your GitHub account. You can use this fork to build an app with kpack and watch it update when pushes are made to your fork.
Apply an image configuration to the cluster. In addition to specifying the source code URL and any build time environment variable names and values your app needs, you can also exercise additional control over how kpack executes builds. You can do this by specifying cache size, build history limit, and resources used.
To build an image from a Git repository:
apiVersion: build.pivotal.io/v1alpha1 kind: Image metadata: name: IMAGE namespace: NAMESPACE spec: tag: gcr.io/PROJECT-NAME/app serviceAccount: SERVICE-ACCOUNT builder: name: BUILDER-NAME kind: ClusterBuilder cacheSize: "CACHE-SIZE" failedBuildHistoryLimit: FAILED-BUILD-HISTORY-LIMIT successBuildHistoryLimit: SUCCESSFUL-BUILD-HISTORY-LIMIT source: git: url: GIT-REPOSITORY-URL revision: GIT-REVISION build: env: - name: ENV-VAR-NAME value: ENV-VAR-VALUE resources: limits: cpu: CPU-LIMIT memory: MEMORY-LIMIT requests: cpu: CPU-REQUEST memory: MEMORY-REQUEST
Where:
IMAGE
is the name of your image.NAMESPACE
is your namespace.PROJECT-NAME
is the name of your project.SERVICE-ACCOUNT
is the name of the service account.BUILDER-NAME
is the name of the builder that the image references.CACHE-SIZE
is the size of the volume claim that the build cache uses. If you do not specify a size, the caching feature is disabled.FAILED-BUILD-HISTORY-LIMIT
is the maximum number of failed builds for an image that is retained. If you do not specify a limit, the default limit of10
is used.SUCCESSFUL-BUILD-HISTORY-LIMIT
is the maximum number of successful builds for an image that is retained. If you do not specify a limit, the default limit of10
is used.GIT-REPOSITORY-URL
is the Git repository URL. Currently, only HTTPS repositories are supported.GIT-REVISION
is the Git revision you want to use. This value may be a commit SHA, branch name, or tag.ENV-VAR-NAME
is the name of the environment variable.ENV-VAR-VALUE
is the value of the environment variable.CPU-LIMIT
is the amount of CPU the build is allowed to use.MEMORY-LIMIT
is the amount of memory the build is allowed to use.CPU-REQUEST
is the maximum CPU capacity of the build.MEMORY-REQUEST
is the maximum memory capacity of the build.
If you would like to build an image from a hosted ZIP or JAR:
apiVersion: build.pivotal.io/v1alpha1 kind: Image metadata: name: IMAGE namespace: NAMESPACE spec: tag: gcr.io/PROJECT-NAME/app serviceAccount: SERVICE-ACCOUNT builder: name: BUILDER-NAME kind: ClusterBuilder cacheSize: "CACHE-SIZE" failedBuildHistoryLimit: FAILED-BUILD-HISTORY-LIMIT successBuildHistoryLimit: SUCCESSFUL-BUILD-HISTORY-LIMIT source: blob: url: BLOB-URL build: env: - name: ENV-VAR-NAME value: ENV-VAR-VALUE resources: limits: cpu: CPU-LIMIT memory: MEMORY-LIMIT requests: cpu: CPU-REQUEST memory: MEMORY-REQUEST
Where:
IMAGE
is the name of your image.NAMESPACE
is your namespace.PROJECT-NAME
is the name of your project.SERVICE-ACCOUNT
is the name of your service account.BUILDER-NAME
is the name of the builder that the image references.CACHE-SIZE
is the size of the volume claim that the build cache uses. If you do not specify a size, the caching feature is disabled.FAILED-BUILD-HISTORY-LIMIT
is the maximum number of failed builds for an image that is retained. If you do not specify a limit, the default limit of10
is used.SUCCESSFUL-BUILD-HISTORY-LIMIT
is the maximum number of successful builds for an image that is retained. If you do not specify a limit, the default limit of10
is used.BLOB-URL
is the URL of the source code blob ZIP or JAR. Either this blob must be publicly accessible, or the access token must be contained in the URL.ENV-VAR-NAME
is the name of the environment variable.ENV-VAR-VALUE
is the value of the environment variable.CPU-LIMIT
is the amount of CPU the build is allowed to use.MEMORY-LIMIT
is the amount of memory the build is allowed to use.CPU-REQUEST
is the maximum CPU capacity of the build.MEMORY-REQUEST
is the maximum memory capacity of the build.
Apply the image to your Kubernetes cluster by running:
kubectl apply -f IMAGE.yaml
Check the status of the image by running:
kubectl get images
You should see that the image has an unknown
READY
status as it builds:NAME LATESTIMAGE READY tutorial-image Unknown
You can tail the logs for the image that is currently building by running:
logs -image tutorial-image
Note: The log utility does not exit when the build finishes. You must exit it manually.
Once the image finishes building, get the image by running:
kubectl get image tutorial-image
You should see the following output:
NAMESPACE NAME LATESTIMAGE READY test tutorial-image gcr.io/PROJECT-NAME/app@SHA-NUMBER True
You can now use the latest image locally in a Kubernetes deployment. To run the latest image locally, run:
docker run -p 8080:8080 LATEST-IMAGE
Where
LATEST-IMAGE
is the latest image.When you push any update to the forked sample app repository you configured, kpack should recognize the update and automatically rebuild your image. To see the new builds, run:
kubectl get builds
When
cloudfoundry/cnb:bionic
is updated, kpack detects if it contains buildpack updates to any of the buildpacks used by the tutorial image. If there is a buildpack update, kpack automatically creates a new build to rebuild your image.