Creating and Deleting Network Profiles (NSX-T Only)

Page last updated:

This topic describes how VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) administrators can create and delete network profiles for Kubernetes clusters provisioned by TKGI on vSphere with NSX‑T integration.

This topic also describes the use cases for when a TKGI administrator should use a network profile.

Prerequisite

TKGI supports network profiles on TKGI on vSphere with NSX‑T only.

To manage TKGI network profiles you must be either a cluster manager or cluster administrator:

• To create or delete a network profile, you must be a cluster administrator, pks.clusters.admin.

• To use or manage network profiles, you must be a cluster manager, pks.clusters.manage or a cluster administrator, pks.clusters.admin.

Note: If a cluster manager, pks.clusters.manage, attempts to create or delete a network profile, the following error occurs: “You do not have enough privileges to perform this action. Please contact the TKGI administrator.”

Overview

You can use network profiles to customize your TKGI Kubernetes clusters on vSphere with NSX‑T.

Cluster administrators can administer network profiles in the following ways:

• Create a Network Profile
• Update an Existing Network Profile
• Delete a Network Profile

Cluster administrators can also use and manage network profiles in all the ways that a cluster manager can:

• List Network Profiles
• Create a Cluster with a Network Profile
• Assign a Network Profile to an Existing Cluster
  
  For information on managing network profiles, see Using and Managing Network Profiles.
  

For information on when to use network profiles, see Network Profile Use Cases below.

Create a Network Profile

The following is the basic structure of a network profile JSON configuration:

{
    "name": "PROFILE-NAME",
    "description": "PROFILE-DESCRIP",
    "parameters": {
        TOP-LEVEL-PARAMETERS,
        "cni_configurations": {
            "type": "nsxt",
            "parameters": {
                CNI-CONFIGURATIONS-PARAMETERS
            }
        }
    }
}

Where:

• PROFILE-NAME is the internal name of the network profile.
• PROFILE-DESCRIP is an internal description of the network profile.
• TOP-LEVEL-PARAMETERS are one or more comma-delimited top-level parameters in a Network Profile. For more information, see Top-Level Parameters below.
• CNI-CONFIGURATIONS-PARAMETERS are one or more comma-delimited cni_configurations parameters in a Network Profile. For more information, see cni_configurations Parameters below.

To create a network profile in TKGI:

1. Create a network profile configuration JSON file with the following content:

   {
       "name": "PROFILE-NAME",
       "description": "PROFILE-DESCRIP",
       "parameters": {
   
           "cni_configurations": {
               "type": "nsxt",
               "parameters": {
   
               }
           }
       }
   }
   

   Where:

   • PROFILE-NAME is the internal name for your network profile.
   • PROFILE-DESCRIP is an internal description for your network profile.

2. Edit the file to specify your network parameters. For information about the available network parameters, see Network Profile Parameters below.

3. Review your network profile configuration carefully. You can modify only a cluster’s pod_ip_block_ids network profile parameter after applying a network profile to a cluster.

4. To create a network profile from your network profile configuration, run the following TKGI CLI command:

   tkgi create-network-profile PATH-TO-YOUR-NETWORK-PROFILE-CONFIGURATION
   

   Where PATH-TO-YOUR-NETWORK-PROFILE-CONFIGURATION is the path to your network profile configuration file.
   
   For example:

   $ tkgi create-network-profile np-routable-pods.json
   
   Network profile example-network-profile successfully created
   

5. Store a copy of your network profile configuration in case you need to modify the network profile in the future.

Cluster managers can create new clusters with your network profile and assign your network profile to existing clusters. For information on managing network profiles, see Using and Managing Network Profiles.

Network Profile Example

The following is an example of a complete network profile JSON configuration:

{
    "name": "example-network-profile",
    "description": "Example Network Profile with All Available Parameters -- FOR ILLUSTRATION PURPOSES ONLY",
    "parameters": {
        "lb_size": "large",
        "pod_ip_block_ids": [
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee55",
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee56" ],
        "pod_subnet_prefix": 27,
        "pod_routable": true,
        "fip_pool_ids": [
            "e50e8f6e-1a7a-45dc-ad49-3a607baa7fa0",
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee55" ],
        "t0_router_id": "5a7a82b2-37e2-4d73-9cb1-97a8329e1a90",
        "master_vms_nsgroup_id": "9b8d535a-d3b6-4735-9fd0-56305c4a5293",
        "node_ip_block_ids": [
            "2250dc43-63c8-4bb8-b8cf-c6e12ccfb7de", "3d577e5c-dcaf-4921-9458-d12b0e1318e6" ],
        "node_routable": true,
        "node_subnet_prefix": 20,
        "nodes_dns": [
            "8.8.8.8", "192.168.115.1", "192.168.116.1" ],      
        "dns_lookup_mode": "API_INGRESS",
        "ingress_prefix": "ingress",
        "single_tier_topology": true,
        "infrastructure_networks": [
            "30.0.0.0/24",
            "192.168.111.0/24",
            "192.168.115.1" ],
        "cni_configurations": {
            "type": "nsxt",
            "parameters": {
                "nsx_lb": false,
                "nsx_ingress_controller": true,         
                "x_forwarded_for": "insert",
                "ingress_ip": "192.168.160.212",
                "log_settings": {
                    "log_level": "DEBUG",
                "ingress_persistence_settings": {
                    "persistence_type": "cookie",
                    "persistence_timeout": 1 },
                "max_l4_lb_service": 10,
                "l4_persistence_type": "source_ip",
                "l4_lb_algorithm": "weighted_round_robin",
                "top_firewall_section_marker":"section-id",
                "bottom_firewall_section_marker":"section-id"
                "bottom_firewall_section_marker":"section-id",
                "log_firewall_traffic":"ALL",
                "lb_http_request_header_size":60,
                "lb_http_response_header_size":45,
                "lb_http_response_timeout":30,
                "connect_retry_timeout":30,
                "enable_hostport": true, 
                "enable_nodelocaldns": true,
                "failover_mode": "PREEMPTIVE",
                "client_ssl_profile": "example_ssl_profile_name",
                "lb_connection_multiplexing_enabled": true,
                "lb_connection_multiplexing_number": 80  
            }
        }
    }
}

Note: This example network profile is for illustration purposes only. It is not intended to be used as a template for a network profile configuration.

Update an Existing Network Profile

To update an existing cluster’s network profile:

1. Confirm the Network Profile Property Supports Updates
2. Create a Modified Network Profile Configuration
3. Create a Modified Network Profile
4. Update the Cluster With a Modified Network Profile

Confirm the Network Profile Property Supports Updates

After you apply a network profile to a cluster, you can update only the following parameters:

• client_ssl_profile
  
• enable_hostport
  
• enable_nodelocaldns
  
• lb_connection_multiplexing_enabled
  
• lb_connection_multiplexing_number
  
• nodes_dns
  
• pod_ip_block_ids
  

You cannot update your existing clusters with any other network profile changes. For more information, see Update-Cluster Network Profile Validation Rules below.

Create a Modified Network Profile Configuration

To create a modified network profile configuration file:

1. Make a copy of your original network profile configuration file.
   
   If it is not possible to obtain the original network profile, create a new network profile with the original values in all of the fields.
   
2. Change the name field to a unique name.
   

3. If you are updating the pod_ip_block_ids field, reorder the IP Block IDs or add additional IP Block IDs.
   
   For example, the following network profile has two pod_ip_block_ids, the first is the original IP block used when creating the cluster, and the second is the new IP block to use for pods.

   {
       "description": "Example network profile for adding pod IP addresses to an existing cluster",
       "name": "pod-ips-add",
       "parameters": {
         "pod_ip_block_ids": [
           "ebe78a74-a5d5-4dde-ba76-9cf4067eee55",
           "ebe78a74-a5d5-4dde-ba76-9cf4067eee56"
         ]
       }
   }
   

   Note: Update only network profile properties that support being updated.

   For more information on configuring a network profile, see Network Profile Parameters above.

4. Review and save the network profile configuration file.

5. Store a copy of your network profile configuration in case you need to modify the network profile in the future.

Create a Modified Network Profile

To create a network profile from a configuration file:

1. Run the following TKGI CLI command:

   tkgi create-network-profile PATH-TO-YOUR-NETWORK-PROFILE-CONFIGURATION
   

   Where PATH-TO-YOUR-NETWORK-PROFILE-CONFIGURATION is the path to your network profile configuration file.

Update the Cluster with a Modified Network Profile

To update a cluster with a modified network profile:

1. To apply the network profile created above to your cluster, run the following command::

   tkgi update-cluster CLUSTER-NAME --network-profile NETWORK-PROFILE-NAME
   

   Where:

   • CLUSTER-NAME is the unique name of your cluster.
   • NETWORK-PROFILE-NAME is the name of the network profile you want to use for your cluster.

TKGI validates the network profile before updating the cluster with the new network profile. For more information, see Update-Cluster Network Profile Validation Rules below.

Update-Cluster Network Profile Validation Rules

TKGI uses strict validation rules before applying a network profile to a cluster with an existing network profile:

• If a field in the original network profile is empty, the system ignores the empty field even if the field is included in the new network profile.
  
• If a field in the new network profile is empty, the system ignores the field even if the field is not empty in the original network profile.
  
• If the pod_ip_block_ids field in the new network profile contains the same entries as the existing network profile, the entry passes validation.
  
• If a field in the new network profile conflicts with the field in the existing network profile, the system reports the conflict and fails the validation.
  

Delete a Network Profile

TKGI administrators can delete a network profile that is not in use.

To delete a network profile:

1. Run the following TKGI CLI command:

   tkgi delete-network-profile NETWORK-PROFILE-NAME
   

   Where NETWORK-PROFILE-NAME is the name of the network profile you want to delete.

Note: You cannot delete a network profile that is in use.

Network Profile Parameters

The Top-Level Parameters and cni_configurations Parameters sections below describe the parameters you can add to a Network Profile.

Top-Level Parameters

TKGI supports the following top-level network profile parameters:

Note: You can update only the parameters labeled as Updatable after applying a network profile to a cluster.

cni_configurations Parameters

TKGI supports the following cni_configurations parameters:

Note: You can update only the parameters labeled as Updatable after applying a network profile to a cluster.

Parameter Descriptions

The following describes commonly used network profile parameters:

client_ssl_profile

The primary use case for updating the network profile client_ssl_profile field is to configure which NSX-T client-side ssl profile is to be used by NCP.

You can change the client_ssl_profile parameter to a valid NSX-T client-side ssl profile name. If client_ssl_profile is blank or incorrect, nsx-default-client-ssl-profile the default NCP client_ssl_profile value is used instead.

Configure client_ssl_profile in the cni_configurations parameters section of your network profile.

enable_hostport

The primary use case for updating the network profile enable_hostport field is to enable or disable NCP support for Kubernetes Host Port.

You can change the enable_hostport parameter to either true or false. Configure enable_hostport in the cni_configurations parameters section of your network profile.

The Kubernetes Host Port feature allows you to expose an application to be externally accessible through a single port from outside of your cluster. For more information on Host Port, see Pod Security Policies in the Kubernetes documentation.

enable_nodelocaldns

The primary use case for updating the network profile enable_nodelocaldns field is to enable or disable NCP support for Kubernetes NodeLocal DNSCache.

You can change the enable_nodelocaldns parameter to either true or false. Configure enable_nodelocaldns in the cni_configurations parameters section of your network profile.

NodeLocal DNSCache improves cluster DNS performance by running a dns caching agent on cluster nodes as a DaemonSet. For more information on NodeLocal DNSCache, see Using NodeLocal DNSCache in Kubernetes clusters in the Kubernetes documentation.

lb_connection_multiplexing_enabled

The primary use case for updating the network profile lb_connection_multiplexing_enabled field is to enable NSX-T load balancer TCP multiplexing.

You can change the lb_connection_multiplexing_enabled parameter to either true or false.

Configure lb_connection_multiplexing_enabled in the cni_configurations parameters section of your network profile.

lb_connection_multiplexing_number

The primary use case for updating the network profile lb_connection_multiplexing_number field is to configure the maximum number of NSX-T load balancer TCP multiplexing connections.

You can change the lb_connection_multiplexing_number parameter to an integer value.

Configure lb_connection_multiplexing_number in the cni_configurations parameters section of your network profile.

nodes_dns

The primary use case for updating the network profile nodes_dns field is to update the DNS server configuration for a cluster.

You can configure the network profile nodes_dns field to add, modify or remove IP addresses from a cluster DNS server configuration. For more information on the network profile nodes_dns field and an example of a nodes_dns configuration, see Specify Nodes DNS Servers.

Note: If you modify a DNS server configuration, do not exceed the maximum of three DNS server IP addresses.

pod_ip_block_ids

The primary use case for updating the network profile pod_ip_block_ids field is to add additional IP addresses for pods when a cluster is at or near the point exhausting all available public IP addresses for pods.

You can change the pod_ip_block_ids parameter array as follows:

• Add more IP Block IDs to the array.
• Reorder the IP Block IDs in the array.

You cannot remove any of the IP Block IDs from an existing pod_ip_block_ids parameter array. Do not create a copy of a network profile, remove pod_ip_block_ids array values, and assign the new profile to a cluster that has the original profile assigned.

For more information on modifying a network profile pod_ip_block_ids field, see Add Pod IPs in Customizing Pod Networks. For more information on the pod_ip_block_ids field, see Network Profile Parameters above.

Network Profile Use Cases

Network profiles let you customize configuration parameters for Kubernetes clusters provisioned by TKGI on vSphere with NSX‑T.

You can apply a network profile to a Kubernetes cluster for the following scenarios: