LATEST VERSION: v1.4 - RELEASE NOTES
Pivotal Container Service v1.2

Using Proxies with PKS on NSX-T

This topic describes how to use proxies with Pivotal Container Service (PKS) with NSX-T.

Overview

If your environment includes HTTP proxies, you can configure PKS with NSX-T to use these proxies so that PKS-deployed Kubernetes master and worker nodes access public Internet services and other internal services through a proxy.

In addition, PKS proxy settings apply to the PKS API instance. When a PKS operator creates a Kubernetes cluster, the PKS API instance VM behind a proxy is able to manage NSX-T objects on the standard network.

You can also proxy outgoing HTTP/HTTPS traffic from Ops Manager and the BOSH Director so that all PKS components use the same proxy service.

The following diagram illustrates the network architecture:

PKS Proxy Architecture

Enable PKS API and Kubernetes Proxy

To configure a global HTTP proxy for all outgoing HTTP/HTTPS traffic from the Kubernetes cluster nodes and the PKS API server, perform the following steps:

  1. Navigate to Ops Manager and log in.

  2. Click the PKS tile.

  3. Click Networking.

    PKS-NSX-T Proxy Settings

  4. Under HTTP/HTTPS proxy, select Enabled. When this option is enabled, you can proxy HTTP traffic, HTTPS traffic, or both.

  5. To proxy outgoing HTTP traffic, under HTTP Proxy URL, enter the HTTP URL of your proxy endpoint. For example, http://myproxy.com:80.

  6. If the proxy for outgoing HTTP traffic uses basic authentication, enter the user name and password in the HTTP Proxy Credentials fields.

  7. To proxy outgoing HTTPS traffic, under HTTPS Proxy URL, enter the HTTP URL of your proxy endpoint. For example, http://myproxy.com:80.

    Note: Using an HTTPS connection to the proxy server is not supported. HTTP and HTTPS proxy options can only be configured with an HTTP connection to the proxy server. You cannot populate either of the proxy URL fields with an HTTPS URL. The proxy host and port can be different for HTTP and HTTPS traffic, but the proxy protocol must be HTTP.

  8. If the proxy for outgoing HTTPS traffic uses basic authentication, enter the user name and password in the HTTPS Proxy Credentials fields.

  9. Under No Proxy, enter the comma-separated list of IP addresses that must bypass the proxy to allow for internal PKS communication.

    In addition to 127.0.0.1 and localhost, you must include your deployment network CIDR, your node network IP block, and your pod network IP block CIDR:

    127.0.0.1,localhost,
    DEPLOYMENT-NETWORK-CIDR,
    NODE-NETWORK-IP-BLOCK-CIDR,
    POD-NETWORK-IP-BLOCK-CIDR
    

    The No Proxy field in the PKS tile does not accept wildcard domain notation, such as *.docker.io and *.docker.com. You must specify the exact IP or FQDN to bypass the proxy. Typical FQDNs to include in the No Proxy field include the following common Docker repositories:

    • registry-1.docker.io
    • auth.docker.io
    • production.cloudflare.docker.com
    • gcr.io
    • storage.googleapis.com

    If you are upgrading and have an existing proxy configuration for reaching a Docker registry or other external services, add the following IP addresses to the No Proxy field to prevent the PKS to IaaS traffic from going through the proxy: NSX Manager, vCenter Server, and all ESXi hosts.

    If a component is communicating with PKS or Harbor using a hostname instead of an IP address, you will need to add the corresponding FQDN to the No Proxy list. For example:

    127.0.0.1,localhost,
    DEPLOYMENT-NETWORK-CIDR,
    NODE-NETWORK-IP-BLOCK-CIDR,
    POD-NETWORK-IP-BLOCK-CIDR,
    PKS-API-FQDN,HARBOR-API-FQDN
    

    Note: By default, the .internal, 10.100.0.0/8, and 10.200.0.0/8 IP address ranges are not proxied. This allows internal PKS communication.

  10. Save the changes to the PKS tile.

  11. Proceed with any remaining PKS tile configurations and deploy PKS. See Installing PKS on vSphere with NSX-T.

Enable Ops Manager and BOSH Proxy

To enable an HTTP proxy for outgoing HTTP/HTTPS traffic from Ops Manager and the BOSH Director, perform the following steps:

  1. Navigate to Ops Manager and log in.

  2. Select User Name > Settings in the upper right.

  3. Click Proxy Settings.

    PKS Proxy Settings

  4. Under HTTP Proxy, enter the FQDN or IP address of the HTTP proxy endpoint. For example, http://myproxy.com:80.

  5. Under HTTPS Proxy, enter the FQDN or IP address of the HTTPS proxy endpoint. For example, http://myproxy.com:80.

    Note: Using an HTTPS connection to the proxy server is not supported. Ops Manager and BOSH HTTP and HTTPS proxy options can be only configured with an HTTP connection to the proxy.

  6. Under No Proxy, include the hosts that must bypass the proxy. This is required.

    In addition to 127.0.0.1 and localhost, include the BOSH Director IP and the PKS VM IP. The BOSH Director IP is typically the first IP address in the deployment network CIDR, and the PKS VM IP is the second IP address in the deployment network CIDR. In addition, be sure to include the Ops Manager IP address in the No Proxy field as well.

    127.0.0.1,localhost,BOSH-DIRECTOR-IP,PKS-VM-IP,OPS-MANAGER-IP
    

    Note: Ops Manager does not allow the use of a CIDR range in the No Proxy field. You must specify each individual IP address to bypass the proxy.

    The No Proxy field does not accept wildcard domain notation, such as *.docker.io and *.docker.com. You must specify the exact IP or FQDN to bypass the proxy, such as registry-1.docker.io.

  7. Click Save.

  8. Return to the Ops Manager Installation Dashboard and click Review Pending Changes.

  9. Click Apply Changes to deploy Ops Manager and the BOSH Director with the updated proxy settings.


Please send any feedback you have to pks-feedback@pivotal.io.

Create a pull request or raise an issue on the source for this page in GitHub