Defining Network Profile for DNS Lookup of Pre-Provisioned IP Addresses

Page last updated:

This topic describes how to define network profile for performing DNS lookup of the pre-provisioned IP addresses for the Kubernetes API load balancer and ingress controller.

About DNS Lookup of Pre-Provisioned IP Addresses

In an Enterprise PKS environment on NSX-T, when you provision a Kubernetes cluster using the command pks create-cluster, NSX-T creates a layer 4 load balancer that fronts the Kubernetes API server running on the master node(s). In addition, NCP creates two layer 7 virtual servers (HTTP and HTTPS) as front-end load balancers for the the ingress resources in Kubernetes servers.

The IP addresses that are assigned to the API load balancer and ingress controller are derived from the floating IP pool in NSX-T. These IP addresses are not known in advance, and you have to wait for the IP addresses to be allocated to know what they are so you can update your DNS records.

If you want to pre-provision these IP addresses, you define a network profile to lookup the IP addresses for these components from your DNS server. In this way you can tell PKS what IP addresses to use for these resources when the cluster is created, and be able to have DNS records for them so FQDNs can be used.

Workflow for Implementing DNS Lookup

Refer to the following workflow for implementing DNS lookup of the Kubernetes API server, the ingress controller, or both.

  1. Choose a valid IP address for the Kubernetes API server load balancer and for the ingress controller. Each IP is allocated from the floating IP pool configured for Enterprise PKS, either in the PKS Tile or a custom floating IP pool using a network profile. Each IP address must not be currently in use.
  2. Create a DNS record (A-Record) with the IP addresses and FQDNs for the Kubernetes API server load balancer and ingress controller.
  3. Create a network profile with dns_lookup_mode (API or API_INGRESS) and the FQDN specified for the API Server in the CLI with -e option, or the ingress controller prefix set in the network profile. See the following examples for details:
  4. Provision a Kubernetes cluster using the network profile defined with dns_lookup_mode (API or API_INGRESS).
  5. During cluster creation Enterprise PKS performs a DNS lookup of the IP address for the Kubernetes API server load balancer and for the ingress controller based on records in the Node DNS server specified in the PKS Tile or configured in a network profile. IP address assignment occurs only when the allocation from the floating IP Pool succeeds. If the IP address cannot be allocated from the floating IP pool, cluster creation fails.

DNS Lookup Parameters

Using the dns_lookup_mode parameter, you can define a network profile to specify the lookup mode: API or API_INGRESS. If the mode is API, PKS will perform a lookup of the pre-provisioned IP address for the Kubernetes API load balancer. If the mode is API_INGRESS, PKS will perform a lookup of the pre-provisioned IP addresses for the Kubernetes API load balancer and the ingress controller.

The IP addresses used must come from the floating IP pool. The floating IP pool, if not specified in the network profile, will come from the PKS tile configuration.

The DNS lookup, whether for the Kubernetes master(s) load balancer or the ingress controller, is performed in the Kubernetes master VM using the DNS server(s) configured in the PKS tile or the nodes_dns field in the network profile.

Note: Enterprise PKS does not support “Name based virtual hosting” where name-based virtual hosts can support routing HTTP traffic with different host names in the URL to the same IP address. Enterprise PKS only supports Path Mode where HTTP traffic is routed with a single IP address or a single hostname to more than one service based on the HTTP URI being provided. For more information, see Types of Ingress in the Kubernetes documentation.

Example Network Profile for Performing a DNS Lookup of the Kubernetes API Load Balancer IP Address

The following network profile, api.json, triggers a DNS lookup for the Kubernetes master node(s) IP address. In this example, a custom floating IP pool is specified, as well as Node DNS servers. If these parameters are not specified, the values in the PKS tile are used.

{
    "name": "example-network-profile",
    "description": "Network profile using API lookup mode",
    "parameters": {
      "nodes_dns": [
        "8.8.8.8", "192.168.115.1", "192.168.116.1"
        ],          
      "fip_pool_ids": [
        "ENTER-FIP-POOL-ID1",
        "ENTER-FIP-POOL-ID2" 
        ],
      "dns_lookup_mode": "API"
    }
}

Example Network Profile for Performing a DNS Lookup of the Ingress Controller IP Address

The following example network profile, api_ingress.json, triggers a DNS lookup for the Kubernetes master node(s) IP address and the ingress controller IP address. In this example, the Node DNS server entered in the PKS Tile is used to perform the lookup.

{
    "name": "api_ingress",
    "description": "Network profile using API_INGRESS dns lookup mode",
    "parameters": {
        "fip_pool_ids": [
            "ENTER-FIP-POOL-ID1",
            "ENTER-FIP-POOL-ID2"
        ],
        "dns_lookup_mode": "API_INGRESS",
        "ingress_prefix": "ingress"
    }
}

Example CLI Command for Setting the Hostname for the Kubernetes API Load Balancer

As an alternative to DNS lookup, you can specify a fixed IP address in the command line so that it will be used for the Kubernetes master node(s) load balancer.

Previously, to create a cluster, you were required to specify an external hostname for the cluster. For example:

$ pks create-cluster my-cluster --external-hostname example.hostname --plan small

Now you can specify the IP address for the load balancer that fronts the Kubernetes master node(s) using the --external-hostname or -e flag. For example:

$ pks create-cluster my-cluster -e 192.168.160.20 -p small

The IP address that you use must belong to a valid floating IP pool created in NSX-T.


Please send any feedback you have to pks-feedback@pivotal.io.