Firewall Ports and Protocols Requirements for vSphere with NSX-T

Page last updated:

This topic describes the firewall ports and protocols requirements for using Enterprise Pivotal Container Service (Enterprise PKS) on vSphere with NSX-T integration.

Firewalls and security policies are used to filter traffic and limit access in environments with strict inter-network access control policies.

Apps frequently require the ability to pass internal communication between system components on different networks and require one or more conduits through the environment’s firewalls. Firewall rules are also required to enable interfacing with external systems such as with enterprise apps or apps and data on the public Internet.

For Enterprise PKS, Pivotal recommends that you disable security policies that filter traffic between the networks supporting the system. To secure the environment and grant access between system components with Enterprise PKS, use one of the following methods:

  • Enable access to apps through standard Kubernetes load-balancers and ingress controller types. This enables you to designate specific ports and protocols as a firewall conduit.
  • Enable access using the NSX-T load balancer and ingress. This enables you to configure external addresses and ports that are automatically mapped and resolved to internal/local addresses and ports.

For information on ports and protocol requirements for vSphere without NSX-T, see Firewall Ports and Protocols Requirements for vSphere without NSX-T

If you are unable to implement your security policy using the methods described above, refer to the following table, which identifies the flows between system components in a typical Enterprise PKS deployment.

Note: To control which groups access deploying and scaling your organization’s Enterprise PKS-deployed Kubernetes clusters, configure your firewall settings as described on the Operator –> PKS API server lines below.

Enterprise PKS Ports and Protocols

The following tables list ports and protocols required for network communications between Enterprise PKS v1.5.0 and later, and vSphere 6.7 and NSX-T 2.4.0.1 and later.

Enterprise PKS Users Ports and Protocols

The following table lists ports and protocols used for network communication between Enterprise PKS user interface components.

Source Component Destination Component Destination Protocol Destination Port Service
Admin/Operator Console All System Components TCP 22 ssh
Admin/Operator Console All System Components TCP 80 http
Admin/Operator Console All System Components TCP 443 https
Admin/Operator Console Cloud Foundry BOSH Director TCP 25555 bosh director rest api
Admin/Operator Console NSX-T API VIP TCP 443 https
Admin/Operator Console Pivotal Cloud Foundry Operations Manager TCP 22 ssh
Admin/Operator Console Pivotal Cloud Foundry Operations Manager TCP 443 https
Admin/Operator Console PKS Controller TCP 9021 pks api server
Admin/Operator Console vCenter Server TCP 443 https
Admin/Operator Console vCenter Server TCP 5480 vami
Admin/Operator Console vSphere ESXI Hosts Mgmt. vmknic TCP 902 ideafarm-door
Admin/Operator and Developer Consoles Harbor Private Image Registry TCP 80 http
Admin/Operator and Developer Consoles Harbor Private Image Registry TCP 443 https
Admin/Operator and Developer Consoles Harbor Private Image Registry TCP 4443 notary
Admin/Operator and Developer Consoles Kubernetes App Load-Balancer Svc TCP/UDP Varies varies with apps
Admin/Operator and Developer Consoles Kubernetes Cluster API Server -LB VIP TCP 8443 httpsca
Admin/Operator and Developer Consoles Kubernetes Cluster Ingress Controller TCP 80 http
Admin/Operator and Developer Consoles Kubernetes Cluster Ingress Controller TCP 443 https
Admin/Operator and Developer Consoles Kubernetes Cluster Worker Node TCP/UDP 30000-32767 kubernetes nodeport
Admin/Operator and Developer Consoles PKS Controller TCP 8443 httpsca
All User Consoles (Operator, Developer, Consumer) Kubernetes App Load-Balancer Svc TCP/UDP Varies varies with apps
All User Consoles (Operator, Developer, Consumer) Kubernetes Cluster Ingress Controller TCP 80 http
All User Consoles (Operator, Developer, Consumer) Kubernetes Cluster Ingress Controller TCP 443 https
All User Consoles (Operator, Developer, Consumer) Kubernetes Cluster Worker Node TCP/UDP 30000-32767 kubernetes nodeport

Note: The type:NodePort Service type is not supported for Enterprise PKS deployments on vSphere with NSX-T. Only type:LoadBalancer and Services associated with Ingress rules are supported on vSphere with NSX-T.

Enterprise PKS Core Ports and Protocols

The following table lists ports and protocols used for network communication between core Enterprise PKS components.

Source Component Destination Component Destination Protocol Destination Port Service
All System Components Corporate Domain Name Server TCP/UDP 53 dns
All System Components Network Time Server UDP 123 ntp
All System Components vRealize LogInsight TCP/UDP 514/1514 syslog/tls syslog
All System Control Plane Components AD/LDAP Directory Server TCP/UDP 389/636 ldap/ldaps
Pivotal Cloud Foundry Operations Manager Admin/Operator Console TCP 22 ssh
Pivotal Cloud Foundry Operations Manager Cloud Foundry BOSH Director TCP 6868 bosh agent http
Pivotal Cloud Foundry Operations Manager Cloud Foundry BOSH Director TCP 8443 httpsca
Pivotal Cloud Foundry Operations Manager Cloud Foundry BOSH Director TCP 8844 credhub
Pivotal Cloud Foundry Operations Manager Cloud Foundry BOSH Director TCP 25555 bosh director rest api
Pivotal Cloud Foundry Operations Manager Harbor Private Image Registry TCP 22 ssh
Pivotal Cloud Foundry Operations Manager Kubernetes Cluster Master/Etcd Node TCP 22 ssh
Pivotal Cloud Foundry Operations Manager Kubernetes Cluster Worker Node TCP 22 ssh
Pivotal Cloud Foundry Operations Manager NSX-T API VIP TCP 443 https
Pivotal Cloud Foundry Operations Manager NSX-T Manager/Controller Node TCP 22 ssh
Pivotal Cloud Foundry Operations Manager NSX-T Manager/Controller Node TCP 443 https
Pivotal Cloud Foundry Operations Manager PKS Controller TCP 22 ssh
Pivotal Cloud Foundry Operations Manager PKS Controller TCP 8443 httpsca
Pivotal Cloud Foundry Operations Manager vCenter Server TCP 443 https
Pivotal Cloud Foundry Operations Manager vSphere ESXI Hosts Mgmt. vmknic TCP 443 https
Cloud Foundry BOSH Director NSX-T API VIP TCP 443 https
Cloud Foundry BOSH Director vCenter Server TCP 443 https
Cloud Foundry BOSH Director vSphere ESXI Hosts Mgmt. vmknic TCP 443 https
BOSH Compilation Job VM Cloud Foundry BOSH Director TCP 4222 bosh nats server
BOSH Compilation Job VM Cloud Foundry BOSH Director TCP 25250 bosh blobstore
BOSH Compilation Job VM Cloud Foundry BOSH Director TCP 25923 health monitor daemon
BOSH Compilation Job VM Harbor Private Image Registry TCP 443 https
BOSH Compilation Job VM Harbor Private Image Registry TCP 8853 bosh dns health
PKS Controller Cloud Foundry BOSH Director TCP 4222 bosh nats server
PKS Controller Cloud Foundry BOSH Director TCP 8443 httpsca
PKS Controller Cloud Foundry BOSH Director TCP 25250 bosh blobstore
PKS Controller Cloud Foundry BOSH Director TCP 25555 bosh director rest api
PKS Controller Cloud Foundry BOSH Director TCP 25923 health monitor daemon
PKS Controller Kubernetes Cluster Master/Etcd Node TCP 8443 httpsca
PKS Controller NSX-T API VIP TCP 443 https
PKS Controller vCenter Server TCP 443 https
Harbor Private Image Registry Cloud Foundry BOSH Director TCP 4222 bosh nats server
Harbor Private Image Registry Cloud Foundry BOSH Director TCP 25250 bosh blobstore
Harbor Private Image Registry Cloud Foundry BOSH Director TCP 25923 health monitor daemon
Harbor Private Image Registry IP NAS Storage Array TCP 111 nfs rpc portmapper
Harbor Private Image Registry IP NAS Storage Array TCP 2049 nfs
Harbor Private Image Registry Public CVE Source Database TCP 443 https
kube-system pod/telemetry-agent PKS Controller TCP 24224 fluentd out_forward
Kubernetes Cluster Ingress Controller NSX-T API VIP TCP 443 https
Kubernetes Cluster Ingress Controller vCenter Server TCP 443 https
Kubernetes Cluster Master/Etcd Node Cloud Foundry BOSH Director TCP 4222 bosh nats server
Kubernetes Cluster Master/Etcd Node Cloud Foundry BOSH Director TCP 25250 bosh blobstore
Kubernetes Cluster Master/Etcd Node Cloud Foundry BOSH Director TCP 25923 health monitor daemon
Kubernetes Cluster Master/Etcd Node Kubernetes Cluster Master/Etcd Node TCP 2379 etcd clent
Kubernetes Cluster Master/Etcd Node Kubernetes Cluster Master/Etcd Node TCP 2380 etcd server
Kubernetes Cluster Master/Etcd Node Kubernetes Cluster Master/Etcd Node TCP 8443 httpsca
Kubernetes Cluster Master/Etcd Node Kubernetes Cluster Master/Etcd Node TCP 8853 bosh dns health
Kubernetes Cluster Master/Etcd Node Kubernetes Cluster Worker Node TCP 4194 cadvisor
Kubernetes Cluster Master/Etcd Node Kubernetes Cluster Worker Node TCP 10250 kubelet api
Kubernetes Cluster Master/Etcd Node Kubernetes Cluster Worker Node TCP 31194 cadvisor
Kubernetes Cluster Master/Etcd Node NSX-T API VIP TCP 443 https
Kubernetes Cluster Master/Etcd Node PKS Controller TCP 8443 httpsca
Kubernetes Cluster Master/Etcd Node PKS Controller TCP 8853 bosh dns health
Kubernetes Cluster Master/Etcd Node vCenter Server TCP 443 https
Kubernetes Cluster Worker Node Cloud Foundry BOSH Director TCP 4222 bosh nats server
Kubernetes Cluster Worker Node Cloud Foundry BOSH Director TCP 25250 bosh blobstore
Kubernetes Cluster Worker Node Cloud Foundry BOSH Director TCP 25923 health monitor daemon
Kubernetes Cluster Worker Node Harbor Private Image Registry TCP 443 https
Kubernetes Cluster Worker Node Harbor Private Image Registry TCP 8853 bosh dns health
Kubernetes Cluster Worker Node IP NAS Storage Array TCP 111 nfs rpc portmapper
Kubernetes Cluster Worker Node IP NAS Storage Array TCP 2049 nfs
Kubernetes Cluster Worker Node Kubernetes Cluster Master/Etcd Node TCP 8443 httpsca
Kubernetes Cluster Worker Node Kubernetes Cluster Master/Etcd Node TCP 8853 bosh dns health
Kubernetes Cluster Worker Node Kubernetes Cluster Master/Etcd Node TCP 10250 kubelet api
pks-system pod/cert-generator PKS Controller TCP 24224 fluentd out_forward
pks-system pod/fluent-bit PKS Controller TCP 24224 fluentd out_forward

VMware Ports and Protocols

The following tables list ports and protocols required for network communication between VMware components.

VMware Virtual Infrastructure Ports and Protocols

The following table lists ports and protocols used for network communication between VMware virtual infrastructure components.

Source Component Destination Component Destination Protocol Destination Port Service
vCenter Server NSX-T Manager/Controller Node TCP 8080 http alt
vCenter Server vSphere ESXI Hosts Mgmt. vmknic TCP 443 https
vCenter Server vSphere ESXI Hosts Mgmt. vmknic TCP 8080 http alt
vCenter Server vSphere ESXI Hosts Mgmt. vmknic TCP 9080 io filter storage
vSphere ESXI Hosts Mgmt. vmknic NSX-T Manager/Controller Node TCP 443 https
vSphere ESXI Hosts Mgmt. vmknic NSX-T Manager/Controller Node TCP 1235 netcpa
vSphere ESXI Hosts Mgmt. vmknic NSX-T Manager/Controller Node TCP 5671 amqp traffic
vSphere ESXI Hosts Mgmt. vmknic NSX-T Manager/Controller Node TCP 8080 http alt
vSphere ESXI Hosts Mgmt. vmknic vCenter Server UDP 902 ideafarm-door
vSphere ESXI Hosts Mgmt. vmknic vCenter Server TCP 9084 update manager
vSphere ESXI Hosts Mgmt. vmknic vSphere ESXI Hosts Mgmt. vmknic TCP 8182 vsphere ha
vSphere ESXI Hosts Mgmt. vmknic vSphere ESXI Hosts Mgmt. vmknic UDP 8182 vsphere ha
vSphere ESXI Hosts vMotion vmknic vSphere ESXI Hosts vMotion vmknic TCP 8000 vmotion
vSphere ESXI Hosts IP Storage vmknic IP NAS Storage Array TCP 111 nfs rpc portmapper
vSphere ESXI Hosts IP Storage vmknic IP NAS Storage Array TCP 2049 nfs
vSphere ESXI Hosts IP Storage vmknic IP NAS Storage Array TCP 3260 iscsi
vSphere ESXI Hosts vSAN vmknic vSphere ESXI Hosts vSAN vmknic TCP 2233 vsan transport
vSphere ESXI Hosts vSAN vmknic vSphere ESXI Hosts vSAN vmknic UDP 12321 unicast agent
vSphere ESXI Hosts vSAN vmknic vSphere ESXI Hosts vSAN vmknic UDP 12345 vsan cluster svc
vSphere ESXI Hosts vSAN vmknic vSphere ESXI Hosts vSAN vmknic UDP 23451 vsan cluster svc
vSphere ESXI Hosts TEP vmknic vSphere ESXI Hosts TEP vmknic UDP 3784 bfd
vSphere ESXI Hosts TEP vmknic vSphere ESXI Hosts TEP vmknic UDP 3785 bfd
vSphere ESXI Hosts TEP vmknic vSphere ESXI Hosts TEP vmknic UDP 6081 geneve
vSphere ESXI Hosts TEP vmknic NSX-T Edge TEP vNIC UDP 3784 bfd
vSphere ESXI Hosts TEP vmknic NSX-T Edge TEP vNIC UDP 3785 bfd
vSphere ESXI Hosts TEP vmknic NSX-T Edge TEP vNIC UDP 6081 geneve
NSX-T Manager/Controller Node NSX-T API VIP TCP 443 https
NSX-T Manager/Controller Node NSX-T Manager/Controller Node TCP 443 https
NSX-T Manager/Controller Node NSX-T Manager/Controller Node TCP 5671 amqp traffic
NSX-T Manager/Controller Node NSX-T Manager/Controller Node TCP 8080 http alt
NSX-T Manager/Controller Node NSX-T Manager/Controller Node TCP 9000 loginsight ingestion api
NSX-T Manager/Controller Node Traceroute Destination UDP 33434-33523 traceroute
NSX-T Manager/Controller Node vCenter Server TCP 80 http
NSX-T Manager/Controller Node vCenter Server TCP 443 https
NSX-T Manager/Controller Node vSphere ESXI Hosts Mgmt. vmknic TCP 443 https
NSX-T Edge Management NSX-T Edge Management TCP 1167 DHCP backend
NSX-T Edge Management NSX-T Edge Management TCP 2480 Nestdb
NSX-T Edge Management NSX-T Edge Management UDP 3784 bfd
NSX-T Edge Management NSX-T Edge Management UDP 50263 high-availability
NSX-T Edge Management NSX-T Manager/Controller Node TCP 443 https
NSX-T Edge Management NSX-T Manager/Controller Node TCP 1235 netcpa
NSX-T Edge Management NSX-T Manager/Controller Node TCP 5671 amqp traffic
NSX-T Edge Management NSX-T Manager/Controller Node TCP 8080 http alt
NSX-T Edge Management Traceroute Destination UDP 33434-33523 traceroute
NSX-T Edge TEP vNIC NSX-T Edge TEP vNIC UDP 3784 bfd
NSX-T Edge TEP vNIC NSX-T Edge TEP vNIC UDP 3785 bfd
NSX-T Edge TEP vNIC NSX-T Edge TEP vNIC UDP 6081 geneve
NSX-T Edge TEP vNIC NSX-T Edge TEP vNIC UDP 50263 high-availability
NSX-T Edge TEP vNIC vSphere ESXI Hosts TEP vmknic UDP 3784 bfd
NSX-T Edge TEP vNIC vSphere ESXI Hosts TEP vmknic UDP 3785 bfd
NSX-T Edge TEP vNIC vSphere ESXI Hosts TEP vmknic UDP 6081 geneve
NSX-T Edge Tier-0 Uplink IP(s) / HA VIP Physical Network Router TCP 179 bgp
Physical Network Router NSX-T Edge Tier-0 Uplink IP(s) / HA VIP TCP 179 bgp

VMware Optional Integration Ports and Protocols

The following table lists ports and protocols used for network communication between optional VMware integrations.

Source Component Destination Component Destination Protocol Destination Port Service
Admin/Operator Console vRealize Operations Manager TCP 443 https
vRealize Operations Manager Kubernetes Cluster API Server -LB VIP TCP 8443 httpsca
vRealize Operations Manager NSX-T API VIP TCP 443 https
vRealize Operations Manager PKS Controller TCP 8443 httpsca
vRealize Operations Manager Kubernetes Cluster API Server -LB VIP TCP 8443 httpsca
Admin/Operator Console vRealize LogInsight TCP 443 https
Kubernetes Cluster Ingress Controller vRealize LogInsight TCP 9000 ingestion api
Kubernetes Cluster Master/Etcd Node vRealize LogInsight TCP 9000 ingestion api
Kubernetes Cluster Master/Etcd Node vRealize LogInsight TCP 9543 ingestion api -tls
Kubernetes Cluster Worker Node vRealize LogInsight TCP 9000 ingestion api
Kubernetes Cluster Worker Node vRealize LogInsight TCP 9543 ingestion api -tls
NSX-T Manager/Controller Node vRealize LogInsight TCP 9000 ingestion api
PKS Controller vRealize LogInsight TCP 9000 ingestion api
Admin/Operator and Developer Consoles Wavefront SaaS APM TCP 443 https
kube-system pod/wavefront-proxy Wavefront SaaS APM TCP 443 https
kube-system pod/wavefront-proxy Wavefront SaaS APM TCP 8443 httpsca
pks-system pod/wavefront-collector PKS Controller TCP 24224 fluentd out_forward
Admin/Operator Console vRealize Network Insight Platform TCP 443 https
Admin/Operator Console vRealize Network Insight Proxy TCP 22 ssh
vRealize Network Insight Proxy Kubernetes Cluster API Server -LB VIP TCP 8443 httpsca
vRealize Network Insight Proxy NSX-T API VIP TCP 22 ssh
vRealize Network Insight Proxy NSX-T API VIP TCP 443 https
vRealize Network Insight Proxy PKS Controller TCP 8443 httpsca
vRealize Network Insight Proxy PKS Controller TCP 9021 pks api server

Please send any feedback you have to pks-feedback@pivotal.io.