Using Compute Profiles (vSphere Only)
Page last updated:
Warning: This feature is a beta component and is intended for evaluation and test purposes only. Do not use this feature in a production environment. Product support and future availability are not guaranteed for beta components.
This topic describes how to use compute profiles for Kubernetes clusters provisioned with VMware Enterprise PKS on vSphere with NSX-T integration.
The procedures in this topic use the PKS API.
Overview
Compute profiles allow you to specify which vSphere resources are used when deploying Kubernetes clusters in a PKS deployment. Use this feature to control where your Kubernetes clusters are created within your virtual infrastructure.
With compute profiles, you can define Availability Zones (AZs) dynamically instead of adding new AZs in the BOSH Director tile. As a result, you do not need to make AZ changes to each plan and the overall impact to the PKS control plane is smaller.
Note: Compute profiles only override the AZ values specified in Enterprise PKS plans. The rest of the cluster’s configuration is inherited from the plan.
You define a compute profile within a JSON file, and then create clusters that use the compute profile using the PKS API.
You can customize any number of compute profiles.
Prerequisites
Before you define a compute profile, you must have the following:
- A configured Enterprise PKS installation on vSphere with NSX-T. See Installing Enterprise PKS on vSphere with NSX-T.
- A PKS UAA admin access token to provide when calling the PKS API. For instructions that explain how to export your access token to an environment variable, see Export PKS API Token.
Define a Compute Profile
Define a compute profile in a JSON file.
In a new text file, enter the parameters for the compute profile in the following format:
{ "name" : "PROFILE-NAME", "description" : "PROFILE-DESCRIPTION", "parameters" : { "azs": [{ "name": "AZ-NAME", "cpi": "CPI-NAME", "cloud_properties": { "datacenters": [{ "name": "DC-NAME", "clusters": [{ "CLUSTER-NAME": { "resource_pool": "RESOURCE-POOL-NAME" } }] }] } }], "master_azs": ["AZ-NAME"], "worker_azs": ["AZ-NAME"] } }Where:
PROFILE-NAMEis the name of the compute profile that you want to define. Enter a string value up to 26 characters long. For example,dev.- (Optional)
PROFILE-DESCRIPTIONdescribes the compute profile. AZ-NAMEis a name for the availability zone (AZ) where you want to deploy cluster VMs. For example,z1.
Note: If you define multiple AZs in the
azsarray, you can specify different AZs for worker node VMs and master node VMs.CPI-NAMEis the BOSH CPI ID of your Enterprise PKS deployment. For instructions on how to obtain theCPI-NAME, see Retrieve the BOSH CPI ID. For example,abc012abc345abc567de.DC-NAMEis the name of your data center as it appears in Ops Manager and your cloud provider console. For example,dc-east. For additional cloud properties related to AZs, see the vSphere CPI AZs section of the BOSH documentation.CLUSTER-NAMEis the name that you want to give your cluster.RESOURCE-POOL-NAMEis the name of the resource pool where you want to deploy your cluster.
For example, this compute profile shows one AZ for one cluster assigned to the
resource_poolnamedmy-res-pool:{ "name": "dev", "description": "For development clusters", "parameters": { "azs": [{ "cpi": "abc012abc345abc567des", "name": "z1", "cloud_properties": { "datacenters": [{ "name": "my-dc", "clusters": [{ "my-vsphere-cluster": { "resource_pool": "my-res-pool1" } }] }] } }], "master_azs": ["z1"], "worker_azs": ["z1"] } }Save the file with a JSON extension. For example,
dev-clusters.json.
For more example compute profiles, see Example Compute Profiles.
Retrieve the BOSH CPI ID
Use the following procedure to retrieve the BOSH CPI ID for your Enterprise PKS deployment.
Locate the credentials that were used to import the PCF .ova or .ovf file into your virtualization system. You configured these credentials when you installed Ops Manager.
Note: If you lose your credentials, you must shut down the Ops Manager VM in the vSphere UI and reset the password. See vCenter Password Requirements and Lockout Behavior in the vSphere documentation for more information.
From a command line, run the following command to SSH into the Ops Manager VM:
ssh ubuntu@OPS-MANAGER-FQDNWhere
OPS-MANAGER-FQDNis the fully qualified domain name (FQDN) of Ops Manager.When prompted, enter the password that you configured during the .ova deployment into vCenter. For example:
$ ssh ubuntu@my-opsmanager-fqdn.example.com Password: ***********
Run
bosh cpi-configto locate the Cloud Provider Interface (CPI) name for your deployment. For example:$ bosh cpi-config Using environment 'BOSH-DIRECTOR-IP' as client 'ops_manager' cpis: - migrated_from: - name: "" name: YOUR-CPI-NAME
For more information about running BOSH commands in your Enterprise PKS deployment, see Using BOSH Diagnostic Commands in Enterprise PKS.
Create a Compute Profile
Use the JSON file that contains your compute profile parameters to make a request to the PKS API. The PKS API applies the parameters to your PKS deployment as a compute profile.
To create the compute profile, run the following command:
curl -s -k -X POST "https://PKS-API:9021/v1/compute-profiles" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json" -d "@PATH-TO-YOUR-COMPUTE-PROFILE.json"
Where:
PKS-APIis the FQDN of your PKS API endpoint. For example,api.pks.example.com.YOUR-ACCESS-TOKENis your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.@PATH-TO-YOUR-COMPUTE-PROFILE.jsonis the path and filename for the file that contains your compute profile parameters. For example,@../compute-profiles/dev-clusters.json.
List Compute Profiles
You can list compute profiles in your PKS deployment by making a request to the PKS API.
To list compute profiles, run the following command:
curl -s -k "https://PKS-API:9021/v1/compute-profiles" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json"
Where:
PKS-APIis the FQDN of your PKS API endpoint. For example,api.pks.example.com.YOUR-ACCESS-TOKENis your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.
For example:
$ curl -s -k "https://api.pks.example.com:9021/v1/compute-profiles" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json"
[
{
"description": "For development clusters",
"name": "dev",
"parameters": {
"azs": [
{
"cloud_properties": {
"datacenters": [
{
"clusters": [
{
"my-vsphere-cluster": {
"drs_rules": null,
"host_group": null,
"resource_pool": "my-res-pool1"
}
}
],
"name": "my-dc"
}
]
},
"cpi": "abc012abc345abc567des",
"name": "z1"
}
],
"master_azs": [
"z1"
],
"worker_azs": [
"z1"
]
},
"uuid": "618abbcc-4dc5-4080-a24a-56c788b5ed44"
}
]
Create a Cluster with a Compute Profile
Define cluster parameters in a JSON file. Include the name of the compute profile and plan in this file. Then make a request to the PKS API to create the cluster.
To create a cluster with a compute profiles, do the following:
Create a new JSON file that describes the cluster that you want to create. Include the compute profile names in the file. For example:
{ "name": "my-cluster", "plan_name": "small", "parameters": { "kubernetes_master_host": "master.host" }, "compute_profile_name": "dev-clusters", "network_profile_name": "my-network-profile" }If you are using Enterprise PKS with NSX-T integration, you can optionally specify a network profile. For more information on network profiles, see Using Network Profiles.
To create the cluster using the compute profile, run the following command:
curl -k -X POST "https://PKS-API:9021/v1/clusters/" \ -H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS=TOKEN" \ -H "Content-Type: application/json" -d "@PATH-TO-CLUSTER.json"Where:
PKS-APIis the FQDN of your PKS API endpoint. For example,api.pks.example.com.YOUR-ACCESS-TOKENis your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.@PATH-TO-CLUSTER.jsonis the path and filename for the JSON file that you created in Define a Compute Profile.
If successful, the cluster creation command outputs something similar to the following:
{ "name": "k8s1", "plan_name": "small", "last_action": "CREATE", "last_action_state": "in progress", "last_action_description": "Creating cluster", "uuid": "abcdefg-a123-b456-c789-1011121314", "kubernetes_master_ips": ["In Progress"], "network_profile_name": "np-tenant-a", "compute_profile_name": "", "parameters": { "kubernetes_master_host": "k8s1.example.com", "kubernetes_master_port": 8443, "worker_haproxy_ip_addresses": null, "kubernetes_worker_instances": 2, "authorization_mode": null, "nsxt_network_profile": "{\"fip_pool_ids\":[\"abcdefg-b234-c567-d891-hijklmnop\"],\"lb_size\":\"small\", \"master_vms_nsgroup_id\":\"0e96f87d-891c-4b32-9757-54051a7f2b91\",\"pod_ip_block_ids\": [\"abcdefg-b234-c567-d891-hijklmnop\"],\"pod_routable\":true,\"pod_subnet_prefix\":24,\"t0_router_id\": \"zyxwvu-b234-c567-d891-hijklmnop\"}", "compute_profile": "{\"azs\":[{\"name\":\"az-1\",\"cpi\":\"abc012abc345abc567des\",\"cloud_properties\":{\"datacenters\":[{\"name\":\"my-dc\",\"clusters\":[{\"cluster1\":{\"resource_pool\":\"pks-respool-1\"}}]}]}},{\"name\":\"z2\",\"cpi\":\"abc012abc345abc567des\",\"cloud_properties\":{\"datacenters\":[{\"name\":\"my-dc\",\"clusters\":[{\"cluster2\":{\"resource_pool\":\"pks-respool-2\"}}]}]}}],\"worker_azs\":[\"az-2\"],\"master_azs\":[\"az-1\"]}", "k8s_customization_parameters": { "vsphere_cloud_provider_config_key": null, "unset_http_proxy": null, "insecure_registries": null } } }
Resize a Cluster with a Compute Profile
Currently, you cannot use the PKS API to resize an existing cluster that uses a compute profile.
To resize an existing cluster, perform the following steps:
Delete the cluster by running the
pks delete clustercommand.Recreate the cluster with a compute profile as described in Create a Cluster with a Compute Profile, and specify a larger value for the
kubernetes_worker_instancesparameter in your cluster JSON file.
List Clusters with Compute Profile
When you execute the pks clusters command, the output does not include
compute profile information for clusters that use compute profiles.
To view compute profile information for all clusters, run the following command:
curl -s -k -H "authorization: Bearer $YOUR-ACCESS-TOKEN" "https://PKS-API:9021/v1/clusters"
Where:
PKS-APIis the FQDN of your PKS API endpoint. For example,api.pks.example.com.$YOUR-ACCESS-TOKENis your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.
The output includes compute profile information.
For example:
$ curl -s -k "https://api.pks.example.com:9021/v1/clusters" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json"
[
{
"name": "cluster-1",
"plan_name": "Plan-1",
"last_action": "UPDATE",
"last_action_state": "succeeded",
"last_action_description": "Instance update completed",
"uuid": "123a45bc-de6f-7891-gh23-45hi678912j3",
"kubernetes_master_ips": [
"10.0.8.6"
],
"network_profile_name": "my-net-profile",
"compute_profile_name": "my-compute-profile",
"parameters": {
"kubernetes_master_host": "cluster-1.virtmerlin.io",
"kubernetes_master_port": 8443,
"kubernetes_worker_instances": 1
}
},
{
"name": "cluster-2",
"plan_name": "Plan-1",
"last_action": "UPDATE",
"last_action_state": "succeeded",
"last_action_description": "Instance update completed",
"uuid": "a9876b54-3c21-9def-87g6-efgh1543219i",
"kubernetes_master_ips": [
"10.0.8.4"
],
"network_profile_name": "my-net-profile-2",
"compute_profile_name": "my-compute-profile-2",
"parameters": {
"kubernetes_master_host": "cluster-2.virtmerlin.io",
"kubernetes_master_port": 8443,
"kubernetes_worker_instances": 2
}
}
]
Delete Compute Profiles
You can delete compute profiles in your PKS deployment by making a request to the PKS API.
You cannot delete a compute profile that is applied to an existing cluster.
First delete any clusters that use the compute profile.
To delete a cluster using a compute profile, use the pks delete cluster command.
To delete a compute profile, run the following command:
curl -k -X DELETE "https://PKS-API:9021/v1/compute-profiles" \
-H "cache-control: no-cache" -H "authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json"
Where:
PKS-APIis the FQDN of your PKS API endpoint. For example,api.pks.example.com.YOUR-ACCESS-TOKENis your PKS UAA admin access token specified as an environment variable. For information on how to export this token as an environment variable, see Export PKS API Token.
Example Compute Profiles
This section includes example compute profiles that define different sets of vSphere resources.
dev-clusters.json
The following example compute profile for a single cluster uses one AZ and one resource pool:
{
"name": "dev",
"description": "For development clusters",
"parameters": {
"azs": [{
"cpi": "abc012abc345abc567des",
"name": "z1",
"cloud_properties": {
"datacenters": [{
"name": "my-dc",
"clusters": [{
"my-vsphere-cluster": {
"resource_pool": "my-res-pool1"
}
}]
}]
}
}],
"master_azs": ["z1"],
"worker_azs": ["z1"]
}
}
prod-clusters.json
The following example compute profile uses two AZs, with one cluster in the first AZ and three clusters in the second AZ:
{
"name": "prod",
"description": "For production clusters",
"parameters": {
"azs": [{
"name": "z1",
"cpi": "abc012abc345abc567des",
"cloud_properties": {
"datacenters": [{
"name": "my-dc",
"clusters": [{
"cluster1": {
"resource_pool": "rp3"
}
}]
}]
}
},
{
"name": "z2",
"cpi": "abc012abc345abc567des",
"cloud_properties": {
"datacenters": [{
"name": "my-dc",
"clusters": [{
"cluster2": {
"resource_pool": "rp1"
}
},
{
"cluster3": {
"resource_pool": "rp1"
}
},
{
"cluster4": {
"resource_pool": "rp2"
}
}]
}]
}
}
],
"master_azs": ["z1","z2"],
"worker_azs": ["z1","z2"]
}
}
Please send any feedback you have to pks-feedback@pivotal.io.