Creating and Managing Network Profiles with the CLI

Page last updated:

This topic describes how VMware Tanzu Kubernetes Grid Integrated Edition administrators on vSphere with NSX-T can create and delete network profiles for Kubernetes clusters provisioned by Tanzu Kubernetes Grid Integrated Edition. It also lists verified use cases for network profiles on Tanzu Kubernetes Grid Integrated Edition.

For more information on how to use network profiles, see Using Network Profiles.

Create a Network Profile

To create a network profile in Tanzu Kubernetes Grid Integrated Edition, you:

  1. Define a network profile in a JSON configuration file, following the Network Profile Format below.

  2. Use the TKGI CLI to define the network profile within Tanzu Kubernetes Grid Integrated Edition, as described in The create-network-profile Command, below.

Network Profile Format

To create a network profile, you must first define it as a JSON file that specifies network parameters, listed in Network Profile Parameters below.

Here is an example network profile that includes all available parameters.

Note: This example network profile is for illustration purposes only. It is not intended to be used as a template for network profile definition.

{
    "name": "example-network-profile",
    "description": "Example Network Profile with All Available Parameters -- FOR ILLUSTRATION PURPOSES ONLY",
    "parameters": {
        "lb_size": "large",
        "pod_ip_block_ids": [
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee55",
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee56" ],
        "pod_subnet_prefix": 27,
        "pod_routable": true,
        "fip_pool_ids": [
            "e50e8f6e-1a7a-45dc-ad49-3a607baa7fa0",
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee55" ],
        "t0_router_id": "5a7a82b2-37e2-4d73-9cb1-97a8329e1a90",
        "master_vms_nsgroup_id": "9b8d535a-d3b6-4735-9fd0-56305c4a5293",
        "node_ip_block_ids": [
            "2250dc43-63c8-4bb8-b8cf-c6e12ccfb7de", "3d577e5c-dcaf-4921-9458-d12b0e1318e6" ],
        "node_routable": true,
        "node_subnet_prefix": 20,
        "nodes_dns": [
            "8.8.8.8", "192.168.115.1", "192.168.116.1" ],      
        "dns_lookup_mode": "API_INGRESS",
        "ingress_prefix": "ingress",
        "single_tier_topology": true,
        "infrastructure_networks": [
            "30.0.0.0/24",
            "192.168.111.0/24",
            "192.168.115.1" ],
        "cni_configurations": {
            "type": "nsxt",
            "parameters": {
                "nsx_lb": false,
                "nsx_ingress_controller": true,         
                "x_forwarded_for": "insert",
                "ingress_ip": "192.168.160.212",
                "log_settings": {
                    "log_level": "DEBUG",
                "ingress_persistence_settings": {
                    "persistence_type": "cookie",
                    "persistence_timeout": 1 },
                "max_l4_lb_service": 10,
                "l4_persistence_type": "source_ip",
                "l4_lb_algorithm": "weighted_round_robin",
                "top_firewall_section_marker":"section-id",
                "bottom_firewall_section_marker":"section-id",
                "log_firewall_traffic":"ALL",
                "lb_http_request_header_size":60,
                "lb_http_response_header_size":45,
                "lb_http_response_timeout":30,
                "connect_retry_timeout":30
            }
        }
    }
}

WARNING: None of the parameters defined under `cni_configurations` can be updated.

Network Profile Parameters

The network profile JSON can include the following top-level parameters:

Parameter Type Description
name String Name of the network profile.
description String Description of the network profile.
parameters Map One or more name-value pairs.
lb_size String Size of the NSX-T load balancer service: small (default), medium, and large.
pod_ip_block_ids String Array of Pod IP Block UUIDs.
pod_subnet_prefix Integer Size of the Pods IP Block subnet.
pod_routable Boolean Make routable the Pods subnet. Default is false.
fip_pool_ids String Array of floating IP pool UUIDs defined in NSX-T.
t0_router_id String Tenant Tier-0 Router UUID defined in NSX-T.
master_vms_nsgroup_id String Namespace Group UUID as defined in NSX-T.
nodes_dns String Array (up to 3) of DNS server IP addresses for lookup of Kubernetes nodes and pods.
dns_lookup_mode String DNS lookup mode for Kubernetes API load balancer (API) and ingress controller (API_INGRESS).
ingress_prefix String Ingress controller hostname prefix for DNS lookup.
single_tier_topology Boolean Use a single Tier-1 Router per cluster (shared). Default is true.
infrastructure_networks String Array of IP addresses and subnets for Node Networks for use with a Shared Tier-1 topology in a Multi-Tier-0 environment.
cni_configurations Map Map containing key-value pairs for configuring NCP (see table below).

The network profile JSON accepts the following parameters for cni_configurations:

Parameter Type Description
type String Only the constant nsxt is accepted.
parameters Map One or more name-value pairs for NCP settings.
x_forwarded_for String Use the same source IP for calling clients. Accepts "insert" and "replace".
nsx_lb Boolean Use NSX-T layer 4 virtual server for each Kubernetes service of type LoadBalancer. Default is true.
nsx_ingress_controller Boolean Use NSX-T layer 7 virtual server as the ingress controller for the Kubernetes cluster. Default is true.
ingress_ip String IP address to use for the ingress controller.
log_settings Map Parameters for configuring NCP logging.
log_level String Accepted values: “INFO”, “WARNING”, “DEBUG”, “ERROR”, and “CRITICAL”.
log_dropped_traffic Boolean Log dropped firewall traffic. Default is false.
ingress_persistence_settings String Parameters for customizing Layer 7 persistence.
persistence_type String Specify the ingress persistence type: none, cookie, or source_ip.
persistence_timeout Integer Persistence timeout interval in seconds.
max_l4_lb_service Integer Limit the maximum number of layer 4 virtual servers per cluster. Minimum is 1.
l4_persistence_type String Connection stickiness based on source_ip.
l4_lb_algorithm String Layer 4 load balancer behavior: round_robin (default), least_connection, ip_hash, weighted_round_robin.
top_firewall_section_marker String UUID of the top section-id for the distributed firewall (DFW) section as defined in NSX-T.
bottom_firewall_section_marker String UUID of the bottom section-id for the distributed firewall (DFW) section as defined in NSX-T.
log_firewall_traffic String Log firewall traffic, such as “ALL”.
lb_http_request_header_size Integer Configure HTTP LoadBalancer request header size, such as 60.
lb_http_response_header_size Integer Configure HTTP LoadBalancer response header size, such as 45.
lb_http_response_timeout Integer Configure HTTP LoadBalancer response timeout, such as 30.
connect_retry_timeout Integer Configure HTTP LoadBalancer connection retry timeout, such as 30.

The create-network-profile Command

After a network profile is defined in a JSON file as described in Network Profile Format, an Tanzu Kubernetes Grid Integrated Edition administrator can create the network profile by running the following TKGI CLI command:

tkgi create-network-profile PATH-TO-YOUR-NETWORK-PROFILE-CONFIGURATION

Where PATH-TO-YOUR-NETWORK-PROFILE-CONFIGURATION is the path to the JSON file you created when defining the network profile.

For example:

$ tkgi create-network-profile np-routable-pods.json

Network profile small-routable-pod successfully created

Only cluster administrators, pks.clusters.admin, can create network profiles. If a cluster manager, pks.clusters.manage, attempts to create a network profile, the following error occurs:

You do not have enough privileges to perform this action. Please contact the TKGI administrator.

After an administrator creates a network profile, cluster managers can create clusters with it or assign it to existing clusters. For more information, see the Using Network Profiles topic.

Manage Network Profiles

Tanzu Kubernetes Grid Integrated Edition administrators can delete network profiles. Administrators can also perform the same operations that cluster managers use to list network profiles and manage how clusters use them.

Delete a Network Profile

To delete a network profile, run the following command:

tkgi delete-network-profile NETWORK-PROFILE-NAME

Where NETWORK-PROFILE-NAME is the name of the network profile you want to delete.

Note: You cannot delete a network profile that is in use.

Only cluster administrators, pks.clusters.admin, can delete network profiles. If a cluster manager, pks.clusters.manage, attempts to delete a network profile, the following error occurs:

You do not have enough privileges to perform this action. Please contact the TKGI administrator.

Cluster Manager Operations

The following sections link to operations that both Tanzu Kubernetes Grid Integrated Edition administrators and cluster managers can perform on network profiles, documented in the Using Network Profiles (NSX-T Only) topic.

Update Network Profile

If a cluster has network profile assigned, the only field you can update is the pod_ip_block_ids array parameter. Any other change will fail strict validation rules and not be applied. For more information on the pod_ip_block_ids field, see Network Profile Parameters.

You cannot remove a cluster’s pod IP block IDs by creating and assigning a new network profile with pod_ip_block_ids array values removed. You only can change the pod_ip_block_ids field as follows:

  • Add more IP Block IDs in the array
  • Reorder the IP Block IDs in the array

The primary network profile use case for updating the pod_ip_block_ids field is to add additional IP addresses for pods when a cluster is at or near the point exhausting all available public IP addresses for pods. For an example, see Adding Pods IPs.

To update the pod_ip_block_ids network profile field for a cluster, do the following:

  1. Define a new network profile. In the pod_ip_block_ids field, reorder the IP Block IDs or add additional IP Block IDs. For more information on defining the network profile JSON file, see Defining Network Profiles.

    Note: If possible, start with the exact contents of the original network profile and update the pod_ip_block_ids field, as well as the name field. If it is not possible to obtain the original network profile, create a new network profile with a unique name and the original values in the pod_ip_block_ids field. Then reorder or add IPs as desired.

  2. Apply the network profile using the command tkgi update-cluster --network-profile.

Network Profile Use Cases

Network profiles let you customize NSX-T configuration parameters at the time of cluster creation. Use cases for network profiles include:

Topic Description
Size a Load Balancer Customize the size of the NSX-T load balancer service that is created when a Kubernetes cluster is provisioned.
Customize Pod Networks Customize Kubernetes Pod Networks, including adding pod IP addresses, subnet size, and routability.
Customize Node Networks Customize Kubernetes Node Networks, including the IP addresses, subnet size, and routability.
Customize Floating IP Pools Specify a custom floating IP pool.
Configure Bootstrap NSGroups Specify an NSX-T Namespace Group where Kubernetes master nodes will be added to during cluster creation.
Configure Edge Router Selection Specify the NSX-T Tier-0 router where Kubernetes node and Pod networks will be connected to.
Specify Nodes DNS Servers Specify one or more DNS servers for Kubernetes clusters.
Configure DNS for Pre-Provisioned IPs Configure DNS lookup of the Kubernetes API load balancer or ingress controller.
Configure the TCP Layer 4 Load Balancer Configure layer 4 TCP load balancer settings; use third-party load balancer.
Configure the HTTP/S Layer 7 Ingress Controller Configure layer 7 HTTP/S ingress controller settings; use third-party ingress controller.
Define DFW Section Markers Configure top or bottom section markers for explicit DFW rule placement.
Configure NCP Logging Configure NCP logging.
Dedicated Tier-1 Topology Use dedicated Tier-1 routers, rather than a shared router, for each cluster’s Kube node, Namespace, and NSX-T load balancer.

Please send any feedback you have to pks-feedback@pivotal.io.