Configuring BOSH Director with NSX-T for PKS
Page last updated:
Warning: Pivotal Container Service (PKS)
v1.3 is no longer supported because it has reached the End
of General Support (EOGS) phase as defined by the
Support Lifecycle Policy.
To stay up to date with the latest software and security updates, upgrade to a supported version.
This topic describes how to configure BOSH Director for vSphere with NSX-T integration for PKS.
Before you begin this procedure, ensure that you have successfully completed all preceding steps for installing PKS on vSphere with NSX-T, including:
- Deploying NSX-T for PKS
- Creating the PKS Management Plane
- Creating the PKS Compute Plane
- Deploying Ops Manager with NSX-T for PKS
- Generating and Registering the NSX Manager Certificate for PKS
Log in to Ops Manager with the Admin username and password credentials.
Click the BOSH Director for vSphere tile.
Select vCenter Config.
Enter the following information:
- vCenter Host: The hostname of the vCenter that manages ESXi/vSphere.
- vCenter Username: A vCenter username with create and delete privileges for virtual machines (VMs) and folders.
- vCenter Password: The password for the vCenter user specified above.
- Datacenter Name: The name of the datacenter as it appears in vCenter.
- Virtual Disk Type: The Virtual Disk Type to provision for all VMs. For guidance on selecting a virtual disk type, see Provisioning a Virtual Disk in vSphere.
- Ephemeral Datastore Names (comma delimited): The names of the datastores that store ephemeral VM disks deployed by Ops Manager.
- Persistent Datastore Names (comma delimited): The names of the datastores that store persistent VM disks deployed by Ops Manager.
Select NSX Networking, then select NSX-T.
Configure NSX-T networking as follows:
- NSX Address: Enter the IP address of the NSX Manager host.
- NSX Username and NSX Password: Enter the NSX Manager username and password.
- NSX CA Cert: Provide the CA certificate in PEM format that authenticates to the NSX server. Open the [NSX CA Certificate that you generate in Generating and Registering the NSX Manager Certificate for PKS, then copy the content into this field.
Configure the following folder names:
- VM Folder: The vSphere datacenter folder where Ops Manager places VMs. Enter
- Template Folder: The vSphere datacenter folder where Ops Manager places VMs. Enter
- Disk path Folder: The vSphere datastore folder where Ops Manager creates attached disk images. You must not nest this folder. Enter
Note: After your initial deployment, you cannot edit the VM Folder, Template Folder, and Disk path Folder names.
- VM Folder: The vSphere datacenter folder where Ops Manager places VMs. Enter
Select Director Config.
In the NTP Servers (comma delimited) field, enter your NTP server addresses.
Note: The NTP server configuration only updates after VM recreation. Ensure that you select the Recreate all VMs checkbox if you modify the value of this field.
Leave the JMX Provider IP Address field blank.
Note: Starting from PCF v2.0, BOSH-reported system metrics are available in the Loggregator Firehose by default. If you continue to use PCF JMX Bridge for consuming them outside of the Firehose, you may receive duplicate data. To prevent this duplicate data, leave the JMX Provider IP Address field blank.
Leave the Bosh HM Forwarder IP Address field blank.
Note: Starting in PCF v2.0, BOSH-reported component metrics are available in the Loggregator Firehose by default. If you continue to use the BOSH HM Forwarder to consume these component metrics, you may receive duplicate data. To prevent this, leave the Bosh HM Forwarder IP Address field blank. For additional guidance, see BOSH System Metrics Available in Loggregator Firehose in the PCF v2.0 Release Notes.
Select the Enable VM Resurrector Plugin to enable BOSH Resurrector functionality.
Select Enable Post Deploy Scripts to run a post-deploy script after deployment. This script allows the job to execute additional commands against a deployment.
Note: You must enable post-deploy scripts to install PKS.
Select Recreate all VMs to force BOSH to recreate all VMs on the next deploy. This process does not destroy any persistent disk data.
For typical PKS deployments, the default settings for all other BOSH Director configuration parameters are suitable. Optionally you can apply additional configurations to BOSH Director. See Director Config Page in Configuring BOSH Director on vSphere in the PCF documentation for details.
Note: If you need to be able to remotely access the BOSH Director VM using the BOSH CLI, and you are deploying PKS with NSX-T in a NAT topology, you must provide the Director Hostname for BOSH at the time of installation. See Director Config Page in Configuring BOSH Director on vSphere in the PCF documentation for details.
Ops Manager Availability Zones correspond to your vCenter clusters and resource pools. Multiple Availability Zones allow you to provide high-availability and load balancing to your applications. When you run more than one instance of an application, Ops Manager balances those instances across all of the Availability Zones assigned to the application. At least three availability zones are recommended for a highly available installation of your chosen runtime.
Note: For more information about using availability zones in vSphere, see Understanding Availability Zones in VMware Installations in the PCF documentation.
Select Create Availability Zones.
Use the following steps to create one or more Availability Zones for PKS to use:
- Click Add and create the PKS Management AZ.
- Enter a unique Name for the Availability Zone, such as
- Select the IaaS configuration (vSphere/vCenter).
- Enter the name of an existing vCenter Cluster to use as an Availability Zone, such as
- Enter the name of the PKS Management Resource Pool in the vCenter cluster that you specified above, such as
RP-MGMT-PKS. The jobs running in this Availability Zone share the CPU and memory resources defined by the pool.
- Click Add Cluster and create at least one PKS Compute AZ.
- Specify the Cluster and the Resource Pool, such as
- Add additional clusters as necessary. Click the trash icon to delete a cluster. The first cluster cannot be deleted.
Select Create Networks.
Select Enable ICMP checks to enable ICMP on your networks. Ops Manager uses ICMP checks to confirm that components within your network are reachable.
Click Add Network.
Create the following network:
NET-MGMT-PKS: Network for Ops Manager, BOSH Director, and the PKS API. This network maps to the NSX logical switch created for the PKS Management Network. See Creating PKS Management Plane.
Note: NSX-T automatically creates the service network to be used by the master and worker nodes (VMs) for Kubernetes clusters managed by PKS. You should not manually create this network.
Use the following values as a guide when you define the network in BOSH. Replace the IP addresses with ranges you defined for the PKS Management Network. Reserve any IP addresses from the subnet that are already in use, such as the IP for Ops Manager and subnet gateway.
Field Configuration Name
vSphere Network Name
Reserved IP Ranges
Select the AZ-MGMT Availability Zone to use with the NET-MGMT-PKS network.
Note: Do not select the COMPUTE network at this point in the configuration. It will be performed at the end of the procedure.
Select Assign AZs and Networks.
Use the drop-down menu to select a Singleton Availability Zone. The Ops Manager Director installs in this Availability Zone. For PKS, this will be the
Use the drop-down menu to select a Network for BOSH Director. BOSH Director runs on the PKS Management Plane network. Select the
In Trusted Certificates, enter a custom certificate authority (CA) certificate to insert into your organization’s certificate trust chain. This allows all BOSH-deployed components in your deployment to trust a custom root certificate. If you are using a private Docker registry, such as VMware Harbor, use this field to enter the certificate for the registry. See Integrating Harbor Registry with PKS for details.
Choose Generate passwords or Use default BOSH password. Pivotal recommends that you use the Generate passwords option for increased security.
Click Save. To view your saved Director password, click the Credentials tab.
(Optional) To send BOSH Director system logs to a remote server, select Yes.
In the Address field, enter the IP address or DNS name for the remote server.
In the Port field, enter the port number that the remote server listens on.
In the Transport Protocol dropdown menu, select TCP or UDP. This selection determines which transport protocol is used to send the logs to the remote server.
(Optional) Mark the Enable TLS checkbox to use TLS encryption when sending logs to the remote server.
- In the Permitted Peer field, enter either the name or SHA1 fingerprint of the remote peer.
- In the SSL Certificate field, enter the SSL certificate for the remote server.
Select Resource Config.
Adjust any values as necessary for your deployment. Under the Instances, Persistent Disk Type, and VM Type fields, choose Automatic from the drop-down menu to allocate the recommended resources for the job. If the Persistent Disk Type field reads None, the job does not require persistent disk space.
Note: Ops Manager requires a Director VM with at least 8 GB memory.
Note: If you set a field to Automatic and the recommended resource allocation changes in a future version, Ops Manager automatically uses the updated recommended allocation.
Follow the steps below to deploy BOSH:
Go to the Ops Manager Installation Dashboard.
Click Review Pending Changes.
Click Apply Changes.
Confirm changes applied successfully.
Check BOSH VM. Log in to vCenter and check for the
p-boshVM deployment in the PKS Management resource pool.
Ater BOSH is successfully deployed, update the network you defined above (
NET-MGMT-PKS) to include each of the COMPUTE AZs you defined. This will ensure that both the Management AZ and the Compute AZ(s) appear in the PKS tile for the Plans.
Return to the BOSH tile and select Create Networks.
Edit the network (
NET-MGMT-PKS) and each COMPUTE AZ.
Review pending changes and apply them to deploy BOSH.
Please send any feedback you have to firstname.lastname@example.org.