Defining Network Profiles

Page last updated:

This topic describes how to define network profiles for Kubernetes clusters provisioned with VMware Enterprise PKS on vSphere with NSX-T.

About Network Profiles

Network profiles let you customize NSX-T configuration parameters at the time of cluster creation. Use cases for network profiles include the following:

Profile Type Description
Load Balancer Sizing Customize the size of the NSX-T load balancer service that is created when a Kubernetes cluster is provisioned.
Custom Node Networks Customize Kubernetes Node Networks, including the IP addresses, subnet size, and routability.
Custom Pod Networks Customize Kubernetes Pod Networks, including IP addresses, subnet size, and routability.
Custom Floating IP Pool Specify a custom floating IP pool.
Security Groups Namespace Specify an NSX-T Namespace Group where Kubernetes master nodes will be added to during cluster creation.
Edge Router Selection Specify the NSX-T Tier-0 router where Kubernetes node and Pod networks will be connected to.
Nodes DNS Configuration Specify one or more DNS servers for Kubernetes clusters.
DNS Lookup of Pre-Provisioned IPs Configure DNS lookup of the Kubernetes API load balancer or ingress controller.
Load Balancer Configuration Configure layer 4 TCP load balancer settings; use third-party load balancer.
Ingress Configuration Configure layer 7 HTTP/S ingress controller settings; use third-party ingress controller.
FW Section Marking Configure top or bottom section markers for explicit DFW rule placement.
Shared Tier-1 Router Use a single Shared Tier-1 Router for each Kubernetes cluster.
NCP Logging Configure NCP logging.

Network Profile Format

Network profiles are defined using JSON. Here is an example network profile that includes all available parameters.

Note: This example network profile is for illustration purposes only. It is not intended to be used as a template for network profile definition.

{
    "name": "example-network-profile",
    "description": "Example Network Profile with All Available Parameters -- FOR ILLUSTRATION PURPOSES ONLY",
    "parameters": {
        "lb_size": "large",
        "pod_ip_block_ids": [
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee55",
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee56" ],
        "pod_subnet_prefix": 27,
        "pod_routable": true,
        "fip_pool_ids": [
            "e50e8f6e-1a7a-45dc-ad49-3a607baa7fa0",
            "ebe78a74-a5d5-4dde-ba76-9cf4067eee55" ],
        "t0_router_id": "5a7a82b2-37e2-4d73-9cb1-97a8329e1a90",
        "master_vms_nsgroup_id": "9b8d535a-d3b6-4735-9fd0-56305c4a5293",
        "node_ip_block_ids": [
            "2250dc43-63c8-4bb8-b8cf-c6e12ccfb7de", "3d577e5c-dcaf-4921-9458-d12b0e1318e6" ],
        "node_routable": true,
        "node_subnet_prefix": 20,
        "nodes_dns": [
            "8.8.8.8", "192.168.115.1", "192.168.116.1" ],      
        "dns_lookup_mode": "API_INGRESS",
        "ingress_prefix": "ingress",
        "single_tier_topology": true,
        "infrastructure_networks": [
            "30.0.0.0/24",
            "192.168.111.0/24",
            "192.168.115.1" ],
        "cni_configurations": {
            "type": "nsxt",
            "parameters": {
                "nsx_lb": false,
                "nsx_ingress_controller": true,         
                "x_forwarded_for": "insert",
                "ingress_ip": "192.168.160.212",
                "log_settings": {
                    "log_level": "DEBUG",
                "ingress_persistence_settings": {
                    "persistence_type": "cookie",
                    "persistence_timeout": 1 },
                "max_l4_lb_service": 10,
                "l4_persistence_type": "source_ip",
                "l4_lb_algorithm": "weighted_round_robin",
                "top_firewall_section_marker":"section-id",
                "bottom_firewall_section_marker":"section-id"                         
            }
        }
    }
}

WARNING: None of the parameters defined under cni_configurations can be updated.

Network Profile Parameters

The network profile JSON accepts the following top-level parameters:

Parameter Type Description
name String Name of the network profile.
description String Description of the network profile.
parameters Map One or more name-value pairs.
lb_size String Size of the NSX-T load balancer service: small (default), medium, and large.
pod_ip_block_ids String Array of Pod IP Block UUIDs.
pod_subnet_prefix Integer Size of the Pods IP Block subnet.
pod_routable Boolean Make routable the Pods subnet. Default is false.
fip_pool_ids String Array of floating IP pool UUIDs defined in NSX-T.
t0_router_id String Tenant Tier-0 Router UUID defined in NSX-T.
master_vms_nsgroup_id String Namespace Group UUID as defined in NSX-T.
nodes_dns String Array (up to 3) of DNS server IP addresses for lookup of Kubernetes nodes and pods.
dns_lookup_mode String DNS lookup mode for Kubernetes API load balancer (API) and ingress controller (API_INGRESS).
ingress_prefix String Ingress controller hostname prefix for DNS lookup.
single_tier_topology Boolean Use a single Tier-1 Router per cluster (shared). Default is false.
infrastructure_networks String Array of IP addresses and subnets for Node Networks for use with a Shared Tier-1 topology in a Multi-Tier-0 environment.
cni_configurations Map Map containing key-value pairs for configuring NCP (see table below).

The network profile JSON accepts the following parameters for cni_configurations:

Parameter Type Description
type String Only the constant nsxt is accepted.
parameters Map One or more name-value pairs for NCP settings.
x_forwarded_for String Use the same source IP for calling clients. Accepts "insert" and "replace".
nsx_lb Boolean Use NSX-T layer 4 virtual server for each Kubernetes service of type LoadBalancer. Default is true.
nsx_ingress_controller Boolean Use NSX-T layer 7 virtual server as the ingress controller for the Kubernetes cluster. Default is true.
ingress_ip String IP address to use for the ingress controller.
log_settings Map Parameters for configuring NCP logging.
log_level String Accepted values: “INFO”, “WARNING”, “DEBUG”, “ERROR”, and “CRITICAL”.
log_dropped_traffic Boolean Log dropped firewall traffic. Default is false.
ingress_persistence_settings String Parameters for customizing Layer 7 persistence.
persistence_type String Specify the ingress persistence type: none, cookie, or source_ip.
persistence_timeout Integer Persistence timeout interval in seconds.
max_l4_lb_service Integer Limit the maximum number of layer 4 virtual servers per cluster. Minimum is 1.
l4_persistence_type String Connection stickiness based on source_ip.
l4_lb_algorithm String Layer 4 load balancer behavior: round_robin (default), least_connection, ip_hash, weighted_round_robin.
top_firewall_section_marker String UUID of the top section-id for the distributed firewall (DFW) section as defined in NSX-T.
bottom_firewall_section_marker String UUID of the bottom section-id for the distributed firewall (DFW) section as defined in NSX-T.

Network Profile Creation

After the network profile is defined in a JSON file, an Enterprise PKS administrator can create the network profile using the PKS CLI. The Kubernetes administrator can use the network profile when creating a cluster.

After you define your network profile configuration as described in Defining Network Profiles, run the following command:

pks create-network-profile PATH-TO-YOUR-NETWORK-PROFILE-CONFIGURATION

Where PATH-TO-YOUR-NETWORK-PROFILE-CONFIGURATION is the path to the JSON file you created when defining the network profile.

For example:

$ pks create-network-profile np-routable-pods.json

Network profile small-routable-pod successfully created

Only cluster administrators, pks.clusters.admin, can create network profiles. If a cluster manager, pks.clusters.manage, attempts to create a network profile, the following error occurs:

You do not have enough privileges to perform this action. Please contact the PKS administrator.

For more information, see the Create and Use Network Profiles section of Using Network Profiles (NSX-T Only).

Please send any feedback you have to pks-feedback@pivotal.io.