Installing PFS on PKS on vSphere

This topic describes how to install Pivotal Function Service (PFS) on Pivotal Container Service (PKS) installed on VMware vSphere.

PKS can be deployed on other IaaS platforms including Azure and AWS. These platforms will also be supported in future releases of PFS.

Requirements

  • The pks CLI has been installed.
  • The kubectl CLI has been installed at version 1.10 or later.
  • The duffle CNAB runtime CLI has been downloaded and installed.
  • The PFS thick bundle has been downloaded.
  • The PFS thick bundle has been been relocated to a container registry configured as below.

Container Registry Configuration

PFS installations require a container registry. If your organization already has a Docker registry then you can use that as long as it meets the following criteria:

  • The server has a DNS entry that can be resolved from all systems in your network environment.
  • The server is configured with a TLS certificate signed by a trusted Certificate Authority.
  • The registry uses basic authentication (OAuth will be available as an option in a future PFS release.)
  • You can push images that are public so no authentication is required to pull them.
Additional authentication options and support for private projects/repos will be available in future PFS releases. Using a self-signed certificate, as described in the PKS Harbor documentation, is not currently supported.

The registry service provided by Docker Hub does meet these criteria but you might prefer to use a registry inside your corporate firewall for security or other reasons.

If you decide to install your own container registry you can follow these steps:

  1. Choose your registry software. There are several options available including VMware Harbor, JFrog Artifactory and Sonatype Nexus Repository. Install the softare according to your needs.

  2. Create a DNS entry for your registry’s host name.

  3. Install a matching TLS certificate signed by a trusted Certificate Authority like Let’s Encrypt.

  4. Configure the registry server for basic authentication.

  5. Create necessary user accounts and registry projects. The images used for PFS installation must be available as public images.

PKS Login

Log into the PKS environment using your usual credentials. E.g. To log in targeting the PKS API server pks-api.example.com as user admin with password adminpassword the following command would be run.

pks login -a pks-api.example.com -u admin -p adminpassword

Create PKS cluster

Create a new PKS cluster with a plan intended for large workloads. E.g. A master node VM with 2 CPUs and 8GB of memory and four worker nodes each with 2 CPUs and 8GB of memory. To create a new cluster called mycluster using the large plan, and an external hostname of myhostname.example.com run:

pks create-cluster mycluster --external-hostname myhostname.example.com --plan large

Track the progress of the create using the pks cluster command. For example, to check on the status of a cluster named mycluster run pks cluster mycluster

It can take up to 30 minutes for cluster creation to complete.

Configure Load Balancer and DNS

Configure load balancer and DNS for the created cluster. - See the PKS documentation - Configure Cluster Access

It can take several minutes for the DNS record information to propagate around the network. During this time “unable to connect” or “no such host” errors may occur when attempting to use kubectl with the cluster.

Retrieve cluster credentials

Use the pks CLI to retrieve credentials and change your kubectl context to your PKS cluster. - To change context to a PKS cluster named mycluster run the following:

pks get-credentials mycluster
  • Verify that the current context is as expected using kubectl:
kubectl config current-context
mycluster

Verify the storageClass

Verify that you have a default StorageClass for the cluster.

kubectl get storageclass

If that doesn’t show a StorageClass that is labelled as default create a storageclass.yaml file, and apply it using kubectl. E.g.

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: pfs
  annotations:
    storageclass.kubernetes.io/is-default-class: 'true'
provisioner: kubernetes.io/vsphere-volume
parameters:
  diskformat: thin
  fstype: ext3
kubectl apply -f storageclass.yaml

Configure duffle

Set the environment variables required by the duffle Kubernetes driver, create a namespace for duffle, create a service account for duffle and give it cluster-admin permissions.

export SERVICE_ACCOUNT=duffle-runtime
export KUBE_NAMESPACE=duffle
kubectl create namespace $KUBE_NAMESPACE
kubectl create serviceaccount "${SERVICE_ACCOUNT}" -n "${KUBE_NAMESPACE}"
kubectl create clusterrolebinding "${SERVICE_ACCOUNT}-cluster-admin" --clusterrole cluster-admin --serviceaccount "${KUBE_NAMESPACE}:${SERVICE_ACCOUNT}"

Install PFS

Change to the directory with the downloaded PFS thick bundle and run duffle install with the relocation mapping file created during image relocation.

duffle install my-pfs pfs-bundle-thick.tgz --bundle-is-file \
  --relocation-mapping pfs-relmap.json \
  --driver k8s
Executing install action...
time="2019-08-15T10:00:56Z" level=info msg="Installing bundle components"
time="2019-08-15T10:00:56Z" level=info
time="2019-08-15T10:00:56Z" level=info msg="installing istio..."
time="2019-08-15T10:00:57Z" level=info msg="done installing istio"
time="2019-08-15T10:00:57Z" level=info msg="installing knative-build..."
time="2019-08-15T10:00:59Z" level=info msg="done installing knative-build"
time="2019-08-15T10:00:59Z" level=info msg="installing knative-serving..."
time="2019-08-15T10:01:14Z" level=info msg="done installing knative-serving"
time="2019-08-15T10:01:14Z" level=info msg="installing riff-system..."
time="2019-08-15T10:01:15Z" level=info msg="done installing riff-system"
time="2019-08-15T10:01:15Z" level=info msg="installing riff-application-build-template..."
time="2019-08-15T10:01:15Z" level=info msg="done installing riff-application-build-template"
time="2019-08-15T10:01:15Z" level=info msg="installing riff-function-build-template..."
time="2019-08-15T10:01:16Z" level=info msg="done installing riff-function-build-template"
time="2019-08-15T10:01:16Z" level=info msg="Kubernetes Application Bundle installed\n\n"

After the command completes pods should be successfully running in the istio-system, knative-build, knative-serving, and kube-system namespaces similar to the output from kubectl get pods shown below.

kubectl get pods --all-namespaces
NAMESPACE         NAME                                                 READY   STATUS    RESTARTS   AGE
istio-system      cluster-local-gateway-7c46bdbc48-kgdq9               1/1     Running   0          5m9s
istio-system      istio-ingressgateway-5c879898cd-rqvgz                2/2     Running   0          5m9s
istio-system      istio-pilot-96844b8bc-td7mw                          1/1     Running   0          5m9s
knative-build     build-controller-54bc4d89b8-mckjd                    1/1     Running   0          5m8s
knative-build     build-webhook-69cb7d8685-5prkt                       1/1     Running   0          5m8s
knative-serving   activator-6c76ff6dcd-j7z7d                           1/1     Running   0          5m6s
knative-serving   autoscaler-5b58449d8d-h2hh6                          1/1     Running   0          5m6s
knative-serving   controller-5bf877dcc5-xmm4r                          1/1     Running   0          5m6s
knative-serving   networking-certmanager-85ddd75579-d75sl              1/1     Running   0          5m6s
knative-serving   networking-istio-55f6f5c9c5-s5x5x                    1/1     Running   0          5m6s
knative-serving   webhook-6fdbf9ff8f-9v684                             1/1     Running   0          5m6s
kube-system        heapster-6d5f964dbd-fgrlk                     1/1       Running   0          1h
kube-system        kube-dns-6b697fcdbd-76p4d                     3/3       Running   0          1h
kube-system        kubernetes-dashboard-785584f46b-bd68t         1/1       Running   0          1h
kube-system        metrics-server-5f68584c5b-n9wh4               1/1       Running   0          1h
kube-system        monitoring-influxdb-54759946d4-wv29g          1/1       Running   0          1h
kube-system        telemetry-agent-68c6647967-886bd              1/1       Running   0          1h
pks-system         fluent-bit-dmrcv                              1/1       Running   0          1h
pks-system         fluent-bit-fcvhk                              1/1       Running   0          1h
pks-system         fluent-bit-lbwns                              1/1       Running   0          1h
pks-system         fluent-bit-s8j7r                              1/1       Running   0          1h
pks-system         sink-controller-7c85744bd6-4lbwq              1/1       Running   0          1h

PFS is now installed. Next you need configure a container registry.

Configure Registry Credentials for a Namespace

These instructions assume that your registry supports basic authentication with a username and password.

Use the pfs CLI to apply your registry credentials to a Kubernetes namespace. The commands below initialize the default namespace.

The values of --registry and --registry-user will be used to create a basic authentication secret, prompting for the password when the command is run. The --image-prefix string is used to auto-generate image names for pfs function create and should match the registry.

For example, to use your own Docker Hub account, replace ??? with your Docker ID below.

export DOCKER_ID=???
pfs credential apply my-creds \
  --registry https://index.docker.io/v1/ \
  --registry-user $DOCKER_ID \
  --default-image-prefix: index.docker.io/$DOCKER_ID

output

Apply credentials "my-creds"
Set default image prefix to "index.docker.io/???"

To use a dev project in say a Harbor registry, with a user account called pfs, deployed at harbor.example.com:

pfs credential apply my-creds \
  --registry https://harbor.example.com \
  --registry-user pfs \
  --default-image-prefix harbor.example.com/dev

NOTE: In Windows PowerShell use a backtick instead of \ for line continuation.

Optional: Enable Outbound Network Access

Knative blocks all outbound traffic by default. For PFS functions to call services outside the cluster, it is necessary to enable outbound network access. Details on how to do that are given in the Knative guide for configuring outbound network access. See Troubleshooting PFS for details on how to verify the outbound traffic configuration.

You can now create your first function.