Using Compute Profiles (vSphere Only)

Page last updated:

Warning: This feature is a beta component and is intended for evaluation and test purposes only. Do not use this feature in a production environment. Product support and future availability are not guaranteed for beta components.

This topic describes how to use compute profiles for Kubernetes clusters provisioned with Enterprise Pivotal Container Service (Enterprise PKS) on vSphere with or without NSX-T integration.

The procedures in this topic use the PKS API.

Overview

Compute profiles allow you to specify which vSphere resources are used when deploying Kubernetes clusters in a PKS deployment. Use this feature to control where your Kubernetes clusters are created within your virtual infrastructure.

With compute profiles, you can define Availability Zones (AZs) dynamically instead of adding new AZs in the BOSH Director tile. As a result, you do not need to make AZ changes to each plan and the overall impact to the PKS control plane is smaller.

Note: Compute profiles only override the AZ values specified in Enterprise PKS plans. The rest of the cluster’s configuration is inherited from the plan.

You define a compute profile within a JSON file, and then create clusters that use the compute profile using the PKS API.

You can customize any number of compute profiles.

Prerequisites

Before you define a compute profile, you must have the following:

Define a Compute Profile

Define a compute profile in a JSON file.

  1. In a new text file, enter the parameters for the compute profile in the following format:

    {
        "name" : "PROFILE-NAME",
        "description" : "PROFILE-DESCRIPTION",
        "parameters" : {
            "azs": [{
                "name": "AZ-NAME",
                "cpi": "CPI-NAME",
                "cloud_properties": {
                    "datacenters": [{
                        "name": "DC-NAME",
                        "clusters": [{
                            "CLUSTER-NAME": {
                                "resource_pool": "RESOURCE-POOL-NAME"
                            }
                        }]     
                    }]
                }
            }],
            "master_azs": ["AZ-NAME"],
            "worker_azs": ["AZ-NAME"]
        }
    }
    

    Where:

    • PROFILE-NAME is the name of the compute profile that you want to define. Enter a string value up to 26 characters long. For example, dev.
    • (Optional) PROFILE-DESCRIPTION describes the compute profile.
    • AZ-NAME is a name for the availability zone (AZ) where you want to deploy cluster VMs. For example, z1.

      Note: If you define multiple AZs in the azs array, you can specify different AZs for worker node VMs and master node VMs.

    • CPI-NAME is the BOSH CPI ID of your Enterprise PKS deployment. For instructions on how to obtain the CPI-NAME, see Retrieve the BOSH CPI ID. For example, abc012abc345abc567de.
    • DC-NAME is the name of your data center as it appears in Ops Manager and your cloud provider console. For example, dc-east. For additional cloud properties related to AZs, see the vSphere CPI AZs section of the BOSH documentation.
    • CLUSTER-NAME is the name that you want to give your cluster.
    • RESOURCE-POOL-NAME is the name of the resource pool where you want to deploy your cluster.

    For example, this compute profile shows one AZ for one cluster assigned to the resource_pool named my-res-pool:

    {
        "name": "dev",
        "description": "For development clusters",
        "parameters": {
            "azs": [{
                "cpi": "abc012abc345abc567des",
                "name": "z1",
                "cloud_properties": {
                    "datacenters": [{
                        "name": "my-dc",
                        "clusters": [{
                            "my-vsphere-cluster": {
                                "resource_pool": "my-res-pool1"
                            }
                        }]
                    }]
                }
            }],
            "master_azs": ["z1"],
            "worker_azs": ["z1"]
        }
    }
    
  2. Save the file with a JSON extension. For example, dev-clusters.json.

For more example compute profiles, see Example Compute Profiles.

Retrieve the BOSH CPI ID

Use the following procedure to retrieve the BOSH CPI ID for your Enterprise PKS deployment.

  1. Locate the credentials that were used to import the PCF .ova or .ovf file into your virtualization system. You configured these credentials when you installed Ops Manager.

    Note: If you lose your credentials, you must shut down the Ops Manager VM in the vSphere UI and reset the password. See vCenter Password Requirements and Lockout Behavior in the vSphere documentation for more information.

  2. From a command line, run the following command to SSH into the Ops Manager VM:

    ssh ubuntu@OPS-MANAGER-FQDN
    

    Where OPS-MANAGER-FQDN is the fully qualified domain name (FQDN) of Ops Manager.

  3. When prompted, enter the password that you configured during the .ova deployment into vCenter. For example:

    $ ssh ubuntu@my-opsmanager-fqdn.example.com
    Password: ***********
    

  4. Run bosh cpi-config to locate the Cloud Provider Interface (CPI) name for your deployment. For example:

    $ bosh cpi-config
    Using environment 'BOSH-DIRECTOR-IP' as client 'ops_manager'
    cpis:
    - migrated_from:
      - name: ""
      name: YOUR-CPI-NAME
    

    For more information about running BOSH commands in your Enterprise PKS deployment, see Managing Enterprise PKS Deployments with BOSH.

Create a Compute Profile

Use the JSON file that contains your compute profile parameters to make a request to the PKS API. The PKS API applies the parameters to your PKS deployment as a compute profile.

To create the compute profile, run the following command:

curl -s -k -X POST "https://PKS-API:9021/v1beta1/compute-profiles" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json" -d "@PATH-TO-YOUR-COMPUTE-PROFILE.json" 

Where:

  • PKS-API is the FQDN of your PKS API endpoint. For example, api.pks.example.com.
  • YOUR-ACCESS-TOKEN is your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.
  • @PATH-TO-YOUR-COMPUTE-PROFILE.json is the path and filename for the file that contains your compute profile parameters. For example, @../compute-profiles/dev-clusters.json.

List Compute Profiles

You can list compute profiles in your PKS deployment by making a request to the PKS API.

To list compute profiles, run the following command:

curl -s -k  "https://PKS-API:9021/v1beta1/compute-profiles" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json"

Where:

  • PKS-API is the FQDN of your PKS API endpoint. For example, api.pks.example.com.
  • YOUR-ACCESS-TOKEN is your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.

For example:

$ curl -s -k  "https://api.pks.example.com:9021/v1beta1/compute-profiles" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json"
  [
    {
        "name": "cluster-1",
        "plan_name": "Plan-1",
        "last_action": "UPDATE",
        "last_action_state": "succeeded",
        "last_action_description": "Instance update completed",
        "uuid": "123a45bc-de6f-7891-gh23-45hi678912j3",
        "kubernetes_master_ips": [
            "10.0.8.6"
        ],
        "network_profile_name": "my-net-profile",
        "compute_profile_name": "my-compute-profile",
        "parameters": {
            "kubernetes_master_host": "cluster-1.virtmerlin.io",
            "kubernetes_master_port": 8443,
            "kubernetes_worker_instances": 1
        }
    },
    {
        "name": "cluster-2",
        "plan_name": "Plan-1",
        "last_action": "UPDATE",
        "last_action_state": "succeeded",
        "last_action_description": "Instance update completed",
        "uuid": "a9876b54-3c21-9def-87g6-efgh1543219i",
        "kubernetes_master_ips": [
            "10.0.8.4"
        ],
        "network_profile_name": "my-net-profile-2",
        "compute_profile_name": "my-compute-profile-2",
        "parameters": {
            "kubernetes_master_host": "cluster-2.virtmerlin.io",
            "kubernetes_master_port": 8443,
            "kubernetes_worker_instances": 2
        }
    }
]

Create a Cluster with a Compute Profile

Define cluster parameters in a JSON file. Include the name of the compute profile and plan in this file. Then make a request to the PKS API to create the cluster.

To create a cluster with a compute profiles, do the following:

  1. Create a new JSON file that describes the cluster that you want to create. Include the compute profile names in the file. For example:

    {
      "name": "my-cluster",
      "plan_name": "small",
      "parameters": {
        "kubernetes_master_host": "master.host"
      },
      "compute_profile_name": "dev-clusters",
      "network_profile_name": "my-network-profile"
    }
    

    If you are using Enterprise PKS with NSX-T integration, you can optionally specify a network profile. For more information on network profiles, see Using Network Profiles.

  2. To create the cluster using the compute profile, run the following command:

    curl -k -X POST "https://PKS-API:9021/v1beta1/clusters/" \
    -H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS=TOKEN" \
    -H "Content-Type: application/json" -d "@PATH-TO-CLUSTER.json"
    

    Where:

    • PKS-API is the FQDN of your PKS API endpoint. For example, api.pks.example.com.
    • YOUR-ACCESS-TOKEN is your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.
    • @PATH-TO-CLUSTER.json is the path and filename for the JSON file that you created in Define a Compute Profile.

    If successful, the cluster creation command outputs something similar to the following:

    {
        "name": "k8s1",
        "plan_name": "small",
        "last_action": "CREATE",
        "last_action_state": "in progress",
        "last_action_description": "Creating cluster",
        "uuid": "abcdefg-a123-b456-c789-1011121314",
        "kubernetes_master_ips": ["In Progress"],
        "network_profile_name": "np-tenant-a",
        "compute_profile_name": "",
        "parameters": {
            "kubernetes_master_host": "k8s1.example.com",
            "kubernetes_master_port": 8443,
            "worker_haproxy_ip_addresses": null,
            "kubernetes_worker_instances": 2,
            "authorization_mode": null,
            "nsxt_network_profile": "{\"fip_pool_ids\":[\"abcdefg-b234-c567-d891-hijklmnop\"],\"lb_size\":\"small\",
            \"master_vms_nsgroup_id\":\"0e96f87d-891c-4b32-9757-54051a7f2b91\",\"pod_ip_block_ids\":
            [\"abcdefg-b234-c567-d891-hijklmnop\"],\"pod_routable\":true,\"pod_subnet_prefix\":24,\"t0_router_id\":
            \"zyxwvu-b234-c567-d891-hijklmnop\"}",
            "compute_profile": "{\"azs\":[{\"name\":\"az-1\",\"cpi\":\"abc012abc345abc567des\",\"cloud_properties\":{\"datacenters\":[{\"name\":\"my-dc\",\"clusters\":[{\"cluster1\":{\"resource_pool\":\"pks-respool-1\"}}]}]}},{\"name\":\"z2\",\"cpi\":\"abc012abc345abc567des\",\"cloud_properties\":{\"datacenters\":[{\"name\":\"my-dc\",\"clusters\":[{\"cluster2\":{\"resource_pool\":\"pks-respool-2\"}}]}]}}],\"worker_azs\":[\"az-2\"],\"master_azs\":[\"az-1\"]}",
            "k8s_customization_parameters": {
                "vsphere_cloud_provider_config_key": null,
                "unset_http_proxy": null,
                "insecure_registries": null
            }
        }
    }
    

Resize a Cluster with a Compute Profile

Currently, you cannot use the PKS API to resize an existing cluster that uses a compute profile.

To resize an existing cluster, perform the following steps:

  1. Delete the cluster by running the pks delete cluster command.

  2. Recreate the cluster with a compute profile as described in Create a Cluster with a Compute Profile, and specify a larger value for the kubernetes_worker_instances parameter in your cluster JSON file.

List Clusters with Compute Profile

When you execute the pks clusters command, the output does not include compute profile information for clusters that use compute profiles.

To view compute profile information for all clusters, run the following command:

curl -s -k -H "authorization: Bearer $YOUR-ACCESS-TOKEN" "https://PKS-API:9021/v1beta1/clusters"

Where:

  • PKS-API is the FQDN of your PKS API endpoint. For example, api.pks.example.com.
  • $YOUR-ACCESS-TOKEN is your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.

The output includes compute profile information.

For example:

$ curl -s -k  "https://api.pks.example.com:9021/v1beta1/clusters" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json"
  [
    {
        "name": "cluster-1",
        "plan_name": "Plan-1",
        "last_action": "UPDATE",
        "last_action_state": "succeeded",
        "last_action_description": "Instance update completed",
        "uuid": "123a45bc-de6f-7891-gh23-45hi678912j3",
        "kubernetes_master_ips": [
            "10.0.8.6"
        ],
        "network_profile_name": "my-net-profile",
        "compute_profile_name": "my-compute-profile",
        "parameters": {
            "kubernetes_master_host": "cluster-1.virtmerlin.io",
            "kubernetes_master_port": 8443,
            "kubernetes_worker_instances": 1
        }
    },
    {
        "name": "cluster-2",
        "plan_name": "Plan-1",
        "last_action": "UPDATE",
        "last_action_state": "succeeded",
        "last_action_description": "Instance update completed",
        "uuid": "a9876b54-3c21-9def-87g6-efgh1543219i",
        "kubernetes_master_ips": [
            "10.0.8.4"
        ],
        "network_profile_name": "my-net-profile-2",
        "compute_profile_name": "my-compute-profile-2",
        "parameters": {
            "kubernetes_master_host": "cluster-2.virtmerlin.io",
            "kubernetes_master_port": 8443,
            "kubernetes_worker_instances": 2
        }
    }
]

Delete Compute Profiles

You can delete compute profiles in your PKS deployment by making a request to the PKS API.

You cannot delete a compute profile that is applied to an existing cluster. First delete any clusters that use the compute profile. To delete a cluster using a compute profile, use the pks delete cluster command.

To delete a compute profile, run the following command:

curl -k -X DELETE  "https://PKS-API:9021/v1beta1/compute-profiles" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json"

Where:

  • PKS-API is the FQDN of your PKS API endpoint. For example, api.pks.example.com.
  • YOUR-ACCESS-TOKEN is your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.

Example Compute Profiles

This section includes example compute profiles that define different sets of vSphere resources.

dev-clusters.json

The following example compute profile for a single cluster uses one AZ and one resource pool:

{
    "name": "dev",
    "description": "For development clusters",
    "parameters": {
        "azs": [{
            "cpi": "abc012abc345abc567des",
            "name": "z1",
            "cloud_properties": {
                "datacenters": [{
                    "name": "my-dc",
                    "clusters": [{
                        "my-vsphere-cluster": {
                            "resource_pool": "my-res-pool1"
                            }
                    }]
                }]
            }
        }],
       "master_azs": ["z1"],
       "worker_azs": ["z1"]
    }
}

prod-clusters.json

The following example compute profile uses two AZs, with one cluster in the first AZ and three clusters in the second AZ:

{
    "name": "prod",
    "description": "For production clusters",
    "parameters": {
        "azs": [{
                "name": "z1",
                "cpi": "abc012abc345abc567des",
                "cloud_properties": {
                    "datacenters": [{
                        "name": "my-dc",
                        "clusters": [{
                            "cluster1": {
                                "resource_pool": "rp3"
                            }
                        }]
                    }]
                }
            },
            {
                "name": "z2",
                "cpi": "abc012abc345abc567des",
                "cloud_properties": {
                    "datacenters": [{
                        "name": "my-dc",
                        "clusters": [{
                            "cluster2": {
                                "resource_pool": "rp1"
                            }
                        },
                            {
                            "cluster3": {
                                "resource_pool": "rp1"
                            }
                        },
                            {
                            "cluster4": {
                                "resource_pool": "rp2"
                            }
                        }]
                    }]
                }
            }
       ],
      "master_azs": ["z1","z2"],
      "worker_azs": ["z1","z2"]
    }
}

Please send any feedback you have to pks-feedback@pivotal.io.