Generating and Registering the NSX Manager Certificate for Enterprise PKS

Page last updated:

This topic describes how to generate and register the NSX Manager certificate authority (CA) certificate in preparation for installing VMware Enterprise PKS on vSphere with NSX-T.

Overview

The NSX Manager CA certificate is an IP-based, self-signed certificate that you create and register with the NSX Manager. The NSX Manager CA certificate is used to authenticate Enterprise PKS with NSX Manager during Enterprise PKS installation on vSphere with NSX-T.

Both the BOSH Director and Enterprise PKS tiles require the NSX Manager CA certificate when on vSphere with NSX-T. Ops Manager requires strict certificate validation and requires the subject and issuer of a self-signed certificate to be either the IP address or fully qualified domain name (FQDN) of the NSX Manager.

Note: By default, the NSX Manager includes a self-signed API certificate with both the subject and issuer populated with hostname instead of IP address or FQDN. You must generate a valid self-signed certificate.

You can use either of the following methods to generate your NSX Manager CA certificate:

You can then register the generated certificate with the NSX Manager using the NSX API:

Note: The following instructions are specific to NSX-T v2.4.1.

Prerequisites

Before you generate and register an NSX Manager CA certificate, ensure that you have successfully completed all of the following steps:

Before configuring your BOSH and PKS tiles with your NSX Manager CA certificate, ensure that you have successfully completed the steps in Install Enterprise PKS in Installing Enterprise PKS on vSphere with NSX-T.

For configuration information, see the Networking section of Installing Enterprise PKS on vSphere with NSX-T.

Generate the NSX-T Root CA Certificate using the Cluster Virtual IP Address

The NSX-T Management Cluster is comprised of three NSX-T Manager nodes. You can configure the NSX-T Manager UI and API to support a single access point by assigning a virtual IP address (VIP) to the NSX-T Management Cluster.

NSX-T Root CA Cert Configuration Using VIP

To generate an NSX-T Management Cluster root CA certificate using your NSX Management Cluster VIP, complete the following steps:

Configure a VIP for the NSX-T Management Cluster

The NSX-T Management Cluster VIP IP address is used to configure the NSX-T Management Cluster certificate.

  1. To support a single access point for the NSX Manager API, complete the steps in Assign a Virtual IP Address to the NSX-T Management Cluster.

Define the Certificate Signing Request Using the VIP

To define the Certificate Signing Request, complete the following steps:

  1. Create a new Certificate Signing Request (CSR) file named nsx-cert.cnf.

  2. Complete the CSR file using the following template:

    [ req ]
default_bits = 2048
distinguished_name = req_distinguished_name
req_extensions = req_ext
prompt = no
[ req_distinguished_name ]
countryName = COUNTRY-INIT
stateOrProvinceName = STATE
localityName = STATE-INIT
organizationName = NSX
commonName = IP-ADDRESS
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1 = IP-ADDRESS

    Where:

    • COUNTRY-INIT are the initials for your corporation’s country of origin.
    • STATE is your corporation’s state or province.
    • STATE-INIT are the initials for your corporation’s state or province.
    • IP-ADDRESS is your Management Cluster VIP IP address.

    For example:

    [ req ]
default_bits = 2048
distinguished_name = req_distinguished_name
req_extensions = req_ext
prompt = no
[ req_distinguished_name ]
countryName = US
stateOrProvinceName = California
localityName = CA
organizationName = NSX
commonName = 10.40.206.5
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1 = IP-ADDRESS

    Note: The Cluster VIP IP address must be used as the commonName attribute value because the certificate will be registered as a Cluster CA certificate, not as a Node CA certificate.

  3. Copy the completed nsx-cert.cnf file to a Linux-based VM that is on the same network as the PKS Management Plane.

Generate the VIP Certificate and Private Key

To use your new CSR to generate a certificate and private key, complete the following commands:

  1. To export the NSX_MANAGER_IP_ADDRESS and NSX_MANAGER_COMMONNAME environment variables, run the following command:

    export NSX_MANAGER_IP_ADDRESS=IP-ADDRESS
export NSX_MANAGER_COMMONNAME=IP-ADDRESS

    Where IP-ADDRESS is your Management Cluster VIP IP Address.

    For example: 

    $ export NSX_MANAGER_IP_ADDRESS=10.40.206.5
$ export NSX_MANAGER_COMMONNAME=10.40.206.5

  2. To use the above CSR to generate the certificate and private key files, nsx.crt and nsx.key, run the following command:

    openssl req -newkey rsa:2048 -x509 -nodes \
> -keyout nsx.key -new -out nsx.crt -subj /CN=$NSX_MANAGER_COMMONNAME \
> -reqexts SAN -extensions SAN -config <(cat ./nsx-cert.cnf \
>  <(printf "[SAN]\nsubjectAltName=DNS:$NSX_MANAGER_COMMONNAME,IP:$NSX_MANAGER_IP_ADDRESS")) -sha256 -days 365

  3. To verify the certificate, run the following command:

    openssl x509 -in nsx.crt -text -noout

Register the Cluster Virtual IP Certificate

To register the Cluster Virtual IP NSX-T Root CA certificate, complete the following:

Import the VIP Certificate to NSX Manager

To import the certificate to the NSX Manager, complete the following steps:

  1. Log in to the NSX Manager UI using the VIP IP address.

  2. Navigate to System > Certificates.

    Import the NSX Manager CA certificate to the NSX Manager

  3. Click Import > Import Certificate. The Import Certificate screen is displayed.

    Note: Ensure that you select Import Certificate and not Import CA Certificate.

  4. In the Name field, enter a unique name for the certificate, such as NSX-VIP-CERT.

    Note: The certificate name must be unique. The default NSX Manager CA certificate is typically named NSX-API-CERT.

    Import the NSX Manager CA certificate to the NSX Manager

  5. In the Certificate Contents field, enter the certificate string. You can locate the certificate string in the nsx.crt certificate file that you previously generated. Copy and paste the certificate contents of the certificate file into the field.

  6. In the Private Key field, enter your certificate’s private key. You can locate the key string in the nsx.key private key file that you previously generated. Copy and paste the key content of the key file into the field.

  7. Verify that the Service Certificate option is set to No. The Service Certificate setting should be “off” because you are using the certificate with NSX Manager appliance nodes.

  8. Click Import.

  9. To verify that your configuration has replicated to all NSX-T Manager instances, perform the following steps:

    1. Log in to each individual NSX Manager node.
    2. On each node, navigate to the System > Certificates screen.
    3. Confirm that the certificate has replicated to the NSX-T Manager instance.

Register the VIP Certificate Using the Cluster Certificate API

To register the imported VIP certificate with the NSX Management Cluster Certificate API, complete the following steps:

  1. To retrieve the certificate UUID, open the NSX Manager System > Certificates screen that you used to import the certificate and copy the UUID.

    Retrieve the UUID

  2. To create NSX_MANAGER_IP_ADDRESS and CERTIFICATE_ID environment variables, run the following commands:

    export NSX_MANAGER_IP_ADDRESS=IP-ADDRESS
export CERTIFICATE_ID="CERTIFICATE-ID"

    Where:

    • IP-ADDRESS is the VIP IP address.
    • CERTIFICATE-ID is the certificate UUID.

    For example: 

    $ export NSX_MANAGER_IP_ADDRESS=10.40.206.5
$ export CERTIFICATE_ID="63cd6646-057b-48bf-b603-64d7e5bdb5bf"

  3. To register the NSX-T Manager CA certificate, run the following cURL request to the Cluster Certificate API:

    curl --insecure -u admin:'PASSWORD' -X POST "https://$NSX_MANAGER_IP_ADDRESS/api/v1/cluster/api-certificate?action=set_cluster_certificate&certificate_id=$CERTIFICATE_ID"

    Where PASSWORD is the password for NSX Manager.

    For example:

    $ curl --insecure -u admin:'P1A2S3S4W56O7R8D!' -X POST "https://$NSX_MANAGER_IP_ADDRESS/api/v1/cluster/api-certificate?action=set_cluster_certificate&certificate_id=$CERTIFICATE_ID"
{
  "certificate_id": "63bb6646-052c-49df-b603-64d7e5bdb5bf"
}

    Note: The certificate chain must be in the industry standard order of certificate - intermediate - root.

  4. Verify that the API request response contains the certificate UUID.

Verify the VIP Certificate

To verify that the VIP certificate has been registered with the NSX Manager, perform the following steps:

  1. Open a browser to the VIP IP address of the NSX Manager.
  2. Log in to the NSX Manager.

  3. Confirm that the new certificate is used by the site.

    Retrieve the UUID

  4. SSH to each NSX Manager host and run the following commands:

    get certificate api
get certificate cluster

  5. Confirm that all returned certificates are the certificates that you generated when performing the steps above.

Add the VIP Certificate to BOSH and PKS Tiles

To add the VIP Certificate to BOSH and PKS Tiles, complete the steps in Add or Update the Certificate for the BOSH and PKS tiles, below.

Generate the NSX-T Root CA Certificate using the Load Balancer IP Address

The NSX-T Management Cluster is comprised of three NSX-T Manager Nodes.

NSX-T Root CA Cert Configuration Using LB IP

The IP address used to configure the NSX-T Management Cluster certificate above is the IP address to use for configuring the load balancer.

To generate the NSX-T root CA certificate using the load balancer IP, complete the following:

Configure a Load Balancer for the NSX Management Cluster

To scale your load balancer, complete the following steps:

  1. Scale your load balancer by completing the steps in Provision a Load Balancer for the NSX-T Management Cluster.

Define a CSR Using the Load Balancer IP Address

To define the Certificate Signing Request for a load balancer in front of the management cluster, complete the following steps:

  1. Create a new Certificate Signing Request (CSR) file named nsx-cert.cnf.

  2. Complete the CSR file using the following template:

    [ req ]
default_bits = 2048
distinguished_name = req_distinguished_name
req_extensions = req_ext
prompt = no
[ req_distinguished_name ]
countryName = COUNTRY-INIT
stateOrProvinceName = STATE
localityName = STATE-INIT
organizationName = NSX
commonName = IP-ADDRESS
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1 = IP-ADDRESS

    Where:

    • COUNTRY-INIT are the initials for your corporation’s country of origin.
    • STATE is your corporation’s state or province.
    • STATE-INIT are the initials for your corporation’s state or province.
    • IP-ADDRESS is your load balancer’s IP address.

    For example:

    [ req ]
default_bits = 2048
distinguished_name = req_distinguished_name
req_extensions = req_ext
prompt = no
[ req_distinguished_name ]
countryName = US
stateOrProvinceName = California
localityName = CA
organizationName = NSX
commonName = 10.40.14.250
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1 = 10.40.14.250

Generate NSX Root CA Certificate and Private Key Using the Load Balancer IP Address

  1. To create NSX_MANAGER_IP_ADDRESS and CERTIFICATE_ID environment variables, run the following commands:

    export NSX_MANAGER_IP_ADDRESS=IP-ADDRESS
export CERTIFICATE_ID="CERTIFICATE-ID"

    Where:

    • IP-ADDRESS is the VIP IP address.
    • CERTIFICATE-ID is the certificate UUID.

    For example: 

    $ export NSX_MANAGER_IP_ADDRESS=10.40.14.250
$ export NSX_MANAGER_COMMONNAME=10.40.14.250

  2. To use the above CSR to generate the certificate and private key files, nsx.crt and nsx.key, run the following commands:

    openssl req -newkey rsa:2048 -x509 -nodes \
> -keyout nsx.key -new -out nsx.crt -subj /CN=$NSX_MANAGER_COMMONNAME \
> -reqexts SAN -extensions SAN -config <(cat ./nsx-cert.cnf \
>  <(printf "[SAN]\nsubjectAltName=DNS:$NSX_MANAGER_COMMONNAME,IP:$NSX_MANAGER_IP_ADDRESS")) -sha256 -days 365

  3. To verify the certificate, run the following command:

    openssl x509 -in nsx.crt -text -noout

Register the Load Balancer IP NSX-T Root CA Certificate

To register the load balancer IP NSX-T root CA certificate, complete the following:

Import CA Certificate to All Three NSX Managers

Complete the following steps to import the certificate to the NSX Manager:

  1. Log in to the NSX Manager UI using the VIP IP address.

  2. Navigate to System > Certificates.

    Import the NSX Manager CA certificate to the NSX Manager

  3. Click Import > Import Certificate. The Import Certificate screen is displayed.

    Note: Ensure that you select Import Certificate and not Import CA Certificate.

  4. In the Name field, enter a unique name for the certificate, such as NSX-VIP-CERT.

    Note: The certificate name must be unique. The default NSX Manager CA certificate is typically named NSX-API-CERT.

  5. In the Certificate Contents field, enter the certificate string. You can locate the certificate string in the nsx.crt certificate file that you previously generated. Copy and paste the certificate contents of the certificate file into the field.

  6. In the Private Key field, enter your certificate’s private key. You can find the key string in the nsx.key private key file you previously generated. Copy and paste the key contents of the key key into the field.

  7. Verify that the Service Certificate option is set to No. The Service Certificate setting should be “off” because you are using the certificate with NSX Manager appliance nodes.

  8. Click Import.

  9. To verify that your configuration has replicated to all NSX-T Manager instances, perform the following:

    1. Log in to each individual NSX Manager node.
    2. On each node, navigate to the System > Certificates screen.
    3. Confirm that the certificate has replicated to the NSX-T Manager instance.

Register Load Balancer Certificate with All Three NSX-T Manager Appliances

To register the imported load balancer certificate with the NSX Management Cluster Certificate API, complete the following steps:

  1. To retrieve the certificate UUID, open the NSX Manager System > Certificates screen that you used to import the certificate and copy the UUID.

    Retrieve the UUID

  2. To create NSX_MANAGER_IP_ADDRESS and CERTIFICATE_ID environment variables, run the following commands:

    export NSX_MANAGER_IP_ADDRESS=IP-ADDRESS
export CERTIFICATE_ID="CERTIFICATE-ID"

    Where:

    • IP-ADDRESS is the IP address of one of the NSX-Manager nodes.
    • CERTIFICATE-ID is the certificate UUID.

    For example: 

    $ export NSX_MANAGER_IP_ADDRESS=10.40.206.1
$ export CERTIFICATE_ID="63bb6646-052c-49df-b603-64d7e5bdb5bf"

    In this example, there are three NSX Manager nodes, 10.40.206.1, 10.40.206.2 and 10.40.206.3, where 10.40.206.1 is the IP address of NSX Manager 1.

  3. To register the NSX-T Manager CA certificate, run the following cURL request to the NSX API:

    curl --insecure -u admin:'PASSWORD' -X POST "https://$NSX_MANAGER_IP_ADDRESS/api/v1/node/services/http?action=apply_certificate&certificate_id=$CERTIFICATE_ID"

    Where PASSWORD is the password for NSX Manager.

    For example: 

    $ curl --insecure -u admin:'P1A2S3S4W5O6R7D!' -X POST "https://$NSX_MANAGER_IP_ADDRESS/api/v1/node/services/http?action=apply_certificate&certificate_id=$CERTIFICATE_ID"
{
  "certificate_id": "63bb6646-052c-49df-b603-64d7e5bdb5bf"
}

    Note: The certificate chain must be in the industry standard order of certificate - intermediate - root.

  4. Verify that the API request response contains the certificate UUID.

  5. Repeat the preceding registration steps for the other two NSX Manager nodes.

    Note: When repeating the steps for the other NSX Manager nodes, NSX_MANAGER_IP_ADDRESS should be different for each node. The CERTIFICATE_ID should be identical for all of the NSX Manager nodes.

Verify Load Balancer Certificate

  1. Log in to each NSX Manager node.

    1. For each site, confirm that the site’s load balancer IP address is used on the site’s certificate Common Name field.

      Load Balancer Certificate

  2. SSH to each NSX Manager host.

    1. For each host, run the following command:

      get certificate cluster

    2. Confirm that the returned certificate is the certificate that you generated when performing the steps above.

  3. Access the NSX-T Manager using the NSX-T Load Balancer IP address.

  4. Confirm that the returned certificate is the certificate that you generated when performing the steps above.

    Load Balancer Certificate

Add the VIP Certificate to BOSH and PKS Tiles

To add the VIP Certificate to BOSH and PKS Tiles, complete the steps in Add or Update the Certificate for the BOSH and PKS tiles.

Add or Update the Certificate for the BOSH and PKS Tiles

To create, delete, and modify NSX-T networking resources, Pivotal Platform tiles and APIs use a VMware NSX Manager account with the Enterprise Administrator role and permissions.

Users configure Pivotal Platform to authenticate to NSX Manager for different purposes in different tiles:

  • Enterprise PKS tile
    The Enterprise PKS tile uses NSX Manager to create load balancers, providing a Kubernetes service described in the Create an External Load Balancer section of the Kubernetes documentation.

  • BOSH Director for vSphere tile
    The BOSH Director for vSphere tile uses NSX Manager to configure networking and security for external-facing Pivotal Platform component VMs, such as Pivotal Application Service routers.

Both the BOSH Director and PKS tiles must be configured with the NSX Manager CA certificate during Enterprise PKS installation on vSphere with NSX-T. The following are examples of properly configured BOSH Director and PKS tile certificate settings.

  • BOSH Director tile NSX Manager CA certificate configuration:

    NSX-T Root CA certificate configuration in BOSH tile

  • PKS tile NSX Manager CA certificate configuration:

    NSX-T Root CA certificate configuration in PKS tile

    Note: The Disable SSL certificate verification option disables NSX Manager CA certificate validation. If you disable TLS certificate verification, unsecured mode takes precedence regardless of whether the NSX Manager CA Cert field is populated. Select the Disable SSL certificate verification option for testing purposes only.

Update the NSX-T Manager IP and Certificate for BOSH

  1. Log in to Ops Manager.
  2. Select the BOSH Tile.

  3. Select the vCenter config.

    Update BOSH

  4. Update the NSX Address field with the load balancer VIP IP address.

  5. Update the NSX CA Cert field with the newly generated certificate.

  6. Click Save.

    Save BOSH Edits

Update the NSX-T Manager IP and Certificate for PKS

  1. Log in to Ops Manager.
  2. Select the PKS tile.
  3. Select the Networking. Update PKS Tile
  4. Update the NSX Manager hostname field with the load balancer VIP IP address.
  5. Update the NSX Manager CA cert field with the newly generated certificate.

  6. Click Save.

    Save PKS Tile edits

Deploy Enterprise PKS

To update the certificate IP address while upgrading Enterprise PKS, complete the following steps:

  1. At the Ops Manager Installation Dashboard, click Review Pending Changes.

    Review Changes

  2. Verify that the Update all clusters errand is enabled for Enterprise PKS.

    Apply Changes

  3. Click Apply Changes.

    Changes Applied

Deploying Enterprise PKS with the Upgrade all clusters errand selected allows NCP configurations on all Kubernetes clusters to be updated with the NSX-T Management Cluster VIP, instead of only the individual NSX-T Manager node.

Next Step

Configure BOSH Director with NSX-T for Enterprise PKS.

Please send any feedback you have to pks-feedback@pivotal.io.