LATEST VERSION: v1.4 - RELEASE NOTES
Pivotal Container Service v1.3

Implementing a Multi-Foundation PKS Deployment

Page last updated:

This topic describes how to deploy multiple instances of PKS on vSphere with NSX-T infrastructure.

About Multi-Foundation PKS

A multi-foundation deployment of PKS lets you install and run multiple instances of PKS. The purpose of a multi-foundation deployment of PKS is to share a common vSphere and NSX-T infrastructure across multiple foundations, while providing complete networking isolation across foundations.

As shown in the diagram, with a multi-foundation PKS topology, each PKS instance is deployed to a dedicated NSX-T Tier-0 router. Foundation A T0 router with Management CIDR 10.0.0.0/16 connects to the vSphere and NSX-T infrastructure. Similarly, Foundation B T0 router with Management CIDR 20.0.0.0/16 connects to the same vSphere and NSX-T components.

As with a single instance deployment, PKS management components are deployed to a dedicated network, for example, 10.0.0.0/24 for PKS Foundation A; 20.0.0.0/24 for PKS Foundation B. When PKS is deployed, networks are defined for nodes, pods, and load balancers. Because of the dedicated Tier-0 router, there is complete networking isolation between each PKS instance.

Architecture for Multi-Foundational PKS Deployment

Requirements

To implement a multi-foundation PKS topology, adhere to the following requirements:

  • One Tier-0 router for each PKS instance. For more information, see Configuring Multiple Tier-0 Routers for Tenant Isolation.
  • The Floating IP pool must not overlap. The CIDR range for each Floating IP Pool must be unique and not overlapping across foundations. For more information, see Create Floating IP Pool.
  • PKS instances can be deployed in NAT and no-NAT mode. If more than one PKS instance is deployed in no-NAT mode, the Nodes IP Block networks cannot overlap.
  • For any Pods IP Block used to deploy Kubernetes clusters in no-NAT (routable) mode, the Pods IP Block cannot overlap across foundations.
  • The NSX-T Super User Principal Identity Certificate should be unique per PKS instance.

The image below shows three PKS installations across three Tier-0 foundations. Key considerations to keep in mind with a multi-foundation PKS topology include the following:

  • Each foundation must rely on a dedicated Tier-0 router
  • You can mix-and-match NAT and no-NAT mode across foundations for Node and Pod networks
  • If you are using non-routable Pods IP Block networks, the Pods IP Block addresses can overlap across foundations
  • Because Kubernetes nodes are behind a dedicated Tier-0 router, if clusters are deployed in NAT mode the Nodes IP Block addresses can also overlap across foundations
  • For each foundation you must define a unique Floating ID Pool with non-overlapping IPs

Configuring Multi-Foundational PKS Deployment


Please send any feedback you have to pks-feedback@pivotal.io.

Create a pull request or raise an issue on the source for this page in GitHub