Installing IPsec

Note: Pivotal Platform is now part of VMware Tanzu. In v1.9 and later, Pivotal IPsec is named IPsec for VMware Tanzu.

Page last updated:

This topic describes how to prepare your network for IPsec for VMware Tanzu, create an IPsec runtime config, and add IPsec to your deployment.

Overview

To install IPsec, you must do the following:

  1. Configure Network Security
  2. Create the IPsec Manifest
  3. Download and Deploy IPsec
  4. Verify Your IPsec Installation

Prerequisites

Before you install IPsec, you must have the following:

  • Google Cloud Platform (GCP), vSphere, Azure, Amazon Web Services (AWS), or OpenStack as your IaaS.

  • In your IaaS, external load balancers that are not Network Load Balancers (NLB). IPsec does not work with NLBs. For more information, see IPsec Load Balancer Issues.

  • Ops Manager operator administration rights

  • BOSH deployed through Ops Manager v1.8 or later.

  • The Maximum Transmission Unit (MTU) for your IaaS set in the VMware Tanzu Application Service for VMs tile. You can find the Applications Network Maximum Transmission Unit (MTU) (in bytes) setting by navigating to Ops Manager > TAS for VMs > Networking.

    VMware recommends the following MTU values:

    IaaS Recommended MTU Value
    GCP 1354
    Azure 1438
    AWS Default value
    vSphere Default value
    OpenStack Follow the recommendations of your Neutron ML2 plugin provider, or empirically test the correct MTU for your environment.

Best Practices

  • IPsec might affect the functionality of other service tiles. As a result, VMware recommends deploying TAS for VMs and each service tile to different isolated subnets. Alternatively, you can minimally deploy all service tiles to a single isolated subnet, apart from the TAS for VMs subnet. Some service tiles do not support IPsec and must be placed in a non-IPsec subnet.
  • For IPsec on Linux VMs, VMware recommends any Ubuntu stemcells for vSphere, OpenStack, and HVM stemcells for AWS. These stemcells are available on VMware Tanzu Network. If you use PV stemcells obtained from bosh.io, see the Packet Loss section of the Troubleshooting IPsec for VMware Tanzu topic to adjust MTU values.

Configure Network Security

To configure network security, do one of the following procedures for your IaaS:

Configure Network Security for Google Cloud Platform

To configure your Google Cloud Platform (GCP) environment for IPsec, do the following:

  1. Navigate to the Networking section of the GCP Console.
  2. Click Firewall rules.
  3. Click Create Firewall Rule.
  4. For Name, enter ipsec.
  5. For Network, select the network where Ops Manager is deployed. For example, opsmgr.
  6. For Source filter, select Allow from any source (0.0.0.0/0).
  7. For Allowed protocols and ports, enter udp:500; ah; esp.
  8. Click Create.
  9. Adjust the MTU value to 1354 by performing the procedure in the Packet Loss section of the Troubleshooting IPsec for VMware Tanzu topic.

Configure Network Security for vSphere

Note: IPsec is not compatible with VMware NSX-T Container Plug-in for VMware Tanzu Application Service for VMs.

To configure your vSphere environment for IPsec, confirm that your network allows the protocols listed in the table below:

Protocol Name Protocol Number Port(s)
AH 51 Any
ESP 50 Any
UDP 17 500

Configure Network Security for Azure

To configure your Azure environment for IPsec, do the following:

  1. Confirm that your network allows the protocols listed in the table below.

    Protocol Name Protocol Number Port(s)
    AH 51 Any
    ESP 50 Any
    UDP 17 500

  2. Adjust the MTU value to 1438. For instructions, see Explanation: Packet Loss.

Configure Network Security for AWS

To configure your AWS environment for IPsec, do the following:

  1. Navigate to EC2 Dashboard > Security Groups.

  2. Select the Security Group with the description TAS for VMs VMs Security Group and click Edit.

  3. Create the following Inbound Rules.

    Type Protocol Name Protocol Number Port Range Source
    Custom Protocol AH 51 All 10.0.0.0/16
    Custom Protocol ESP 50 All 10.0.0.0/16
    Custom UDP Rule UDP 17 500 10.0.0.0/16

Note: The default TAS for VMs VMs Security Group is typically specified with a subnet of 10.0.0.0/16. If your TAS for VMs subnet is deployed to a different CIDR block, adjust the source as needed.

Configure Network Security for OpenStack

Note: The following network configuration is optimized for Mirantis OpenStack, but other OpenStack distributions have a similar workflow.

To configure your Mirantis OpenStack environment for IPsec, perform the following steps:

  1. Navigate to Project / Access & Security.

  2. Select the security group and click Manage Rules.

  3. Create the following Ingress and Egress Rules. Adjust the source CIDR as needed for your environment.

    Protocol Name Protocol Number Port Range Source
    ESP 50 Any 0.0.0.0/0
    AH 51 Any 0.0.0.0/0
    UDP 17 500 0.0.0.0/0

Create the IPsec Manifest

To add IPsec to VMs in your deployment, you must create a runtime config file named ipsec-addon.yml. The ipsec-addon.yml configures IPsec add-on properties for Linux VMs.

To create the IPsec manifest, add IPsec to Linux VMs in your deployment:

  1. Create an IPsec runtime config file called ipsec-addon.yml and add the following YAML:

    releases:
    - name: ipsec
      version: 1.x.x
    addons:
    - name: ipsec-addon
      jobs:
      - name: ipsec
        release: ipsec
        properties:
          ipsec:
            optional: false
            ipsec_subnets:
            - 10.0.1.1/20
            no_ipsec_subnets:
            - 10.0.1.10/32  # bosh director
            - 10.0.1.4/32 # ops managers
            instance_certificate: |
              -----BEGIN CERTIFICATE-----
              EXAMPLExCERTIFICATExEXAMPLExCERTIFICATExEXAMPLExCERTIFICATE
              ...
              -----END CERTIFICATE-----
            instance_private_key: |
              -----BEGIN EXAMPLE RSA PRIVATE KEY-----
              EXAMPLExRSAxPRIVATExKEYxDATAxEXAMPLExRSAxPRIVATExKEYxDATA
              ...
              -----END EXAMPLE RSA PRIVATE KEY-----
            ca_certificates:
              - |
                -----BEGIN CERTIFICATE-----
                EXAMPLExCERTIFICATExEXAMPLExCERTIFICATExEXAMPLExCERTIFICATE
                ...
                -----END CERTIFICATE-----
              - |
                -----BEGIN CERTIFICATE-----
                EXAMPLExCERTIFICATExEXAMPLExCERTIFICATExEXAMPLExCERTIFICATE
                ...
                -----END CERTIFICATE-----
            prestart_timeout: 30
            stop_timeout: 30
            esp_proposals: aes128gcm16!
            ike_proposals: aes128-sha256-modp2048!
            log_level: 1
            ike_version: ike
            optional_warn_interval: 1
            force_udp_encapsulation: false
            instance_certificate_info_period: 30
            instance_certificate_warn_period: 14
            instance_certificate_error_period: 7
            instance_certificate_critical_period: 3
      include:
        stemcell:
        - os: ubuntu-trusty
        - os: ubuntu-xenial
    

  2. Replace the values listed in the template above as follows:

    Property Instructions
    version Specify the version number of your IPsec download from VMware Tanzu Network.
    optional This value makes IPsec enforcement optional. To add IPsec to an existing deployment, set this flag to true. After IPsec has been successfully installed, set this flag back to false and redeploy.

    Warning: Communication between existing components fails if you try to add IPsec to an existing deployment without setting optional to true.

    ipsec_subnets List the subnets that you want to be encrypted. You can include the entire deployment to encrypt all traffic or a portion of the network. Encrypt any network that handles business-sensitive data.
    no_ipsec_subnets List the IP address of your BOSH Director and Ops Manager VM, along with any other IP addresses in your deployment to which you want to communicate without encryption. VMware recommends that you list the subnets that are used for TAS for VMs managed services. Subnets for TAS for VMs managed services that do not support IPsec, such as Ops Manager or service tiles, must be listed under no_ipsec_subnets and installed in a separate network.

    Note: If you have an external load balancer such as F5, add it to the no_ipsec_subnets property. If you want to include it in the ipsec_subnet, you must configure it manually.

    Warning: IPs that are not in ipsec_subnets or no_ipsec_subnets have no default behavior and cannot communicate with other internal VMs. You must specify internal IPs in ipsec_subnets or no_ipsec_subnets.

    Warning: In GCP, if you use the default router for DNS instead of the Google public DNS at 8.8.8.8, you must add the IP address of the default router in your subnet to no_ipsec_subnets. For example, 10.0.0.1/32.

    instance_certificate Copy in the signed certificate to be used by all your instance VMs. You must use one of the CAs in the ca_certificates property to sign this certificate.

    VMware recommends that you use a self-signed certificate. For more information, see Generate a Self-Signed Certificate below.
    instance_private_key Copy in the private key that corresponds to the instance_certificate above. This key must not use a pass phrase.

    ca_certificates Copy in CA certificates for the instance VM to trust during the validation process. In most cases, you only need the CA certificate used to sign the instance certificate. During CA credential rotation, you need two CA certificates.

    IPsec v1.8.12 and later supports the CA certificate chain. Concatenate the contents of the root and the intermediate certificates as one of the list items in ca_certificates, with the root CA at the top:
    ca_certificates:
         - |
           -----BEGIN CERTIFICATE-----
           ... <ROOT>
           -----END CERTIFICATE-----
           -----BEGIN CERTIFICATE-----
           ... <INTERMEDIATE 1 ISSUED BY
           THE ROOT CERT ABOVE>
           -----END CERTIFICATE-----
           -----BEGIN CERTIFICATE-----
           ... <INTERMEDIATE 2 ISSUED BY
           THE INTERMEDIATE 1 ABOVE
           ... AND SIGNS THE INSTANCE CERT>
           -----END CERTIFICATE-----
         - |
           -----BEGIN CERTIFICATE-----
           ... <SECOND ROOT>
           -----END CERTIFICATE-----
        

    Note: The root and the intermediate certificates cannot have the same subjectName, also called the common name and set with CN=. The root must be the first certificate of the chain.

    prestart_timeout Use this to modify the 30-second default prestart timeout value. This value limits the number of seconds allowed for IPsec to start before failing the attempt.
    stop_timeout Use this to modify the 30-second default post stop timeout value. When IPsec stops, communication between VMs is not encrypted and might cause errors. By default, IPsec waits 30 seconds for all other running jobs to stop before stopping itself.
    log_level You can specify the IKE daemon numerical log level, ranging from -1 to 4. For more information, see Logger Configuration in the strongSwan documentation.
    optional_warn_interval The interval in hours of warning when optional property is set to true. It prints the warning message {Date} - IPsec is set to "Optional" in the file /var/vcap/sys/log/ipsec/ipsec.stdout.log for Linux.
    force_udp_encapsulation Available on Linux-only deployments. If set to true it forces UDP encapsulation for ESP packets.

    Warning: Setting this property to true in mixed deployments causes the deployment to fail. If you do not have a Linux-only deployment, you must set force_udp_encapsulation to false.

    instance_certificate
    _info_period
    If the instance certificate expires during the set period in days, the IPsec release writes an [INFO] message in the logs.
    instance_certificate
    _warn_period
    If the instance certificate expires during the set period in days, the IPsec release writes a [WARN] message in the logs.
    instance_certificate
    _error_period
    If the instance certificate expires during the set period in days, the IPsec release writes an [ERROR] message in the logs.
    instance_certificate
    _critical_period
    If the instance certificate expires during the set period in days, the IPsec release writes a [CRITICAL] message in the logs.

(Optional) Configure Custom Deployment Proposals

The manifest templates given above configure a default proposal for the ipsec-addon.yml. If you want to use different proposals, modify the ipsec-addon.yml using the following table:

  1. Select the encryption type from the first row.

    Encryption Type Linux (ipsec-addon)
    ike_proposals esp_proposals
    128 Bit Encryption (default)
    aes128-sha256-modp2048!
    aes128gcm16!
    256 Bit Encryption
    aes256-sha256-modp2048!
    aes256gcm16!

  2. Copy the properties from that row into ipsec-addon.yml accordingly. See the ipsec-addon.yml file example above.

    Property Instructions
    ike_proposals You can modify the IKE (Main Mode) encryption and integrity algorithms, and the Diffie-Hellman group. The default, aes128-sha256-modp2048!, is 128 bit AES-CBC for encryption, SHA2_256_128 HMAC for integrity, and Group 14 for Diffie-Hellman.
    esp_proposals You can modify the ESP (Quick Mode) encryption and integrity algorithms. The default, aes128gcm16!, is 128 bit AES-GCM with 128 bit ICV for both encryption and integrity.

Download and Deploy IPsec

To download the IPsec binary, add your IPsec runtime config to your BOSH manifest, and deploy IPsec, follow the steps below:

Breaking Change: If you are using v1.12 or later, you must use named runtime configs. If you have not already split your runtime config into multiple named files, do so before upgrading IPsec for VMware Tanzu. For general information about named runtime config files, see Generic Configs in the BOSH documentation.

  1. Download the IPsec software binary from the VMware Tanzu Network to your local machine.

  2. Copy the software binary to your Ops Manager instance.

    $ scp -i PATH-TO-PRIVATE-KEY ipsec-release.tar.gz ubuntu@YOUR-OPS-MANAGER-VM-IP:

  3. Copy the IPsec runtime config file to your Ops Manager instance.

    $ scp -i PATH-TO-PRIVATE-KEY ipsec-addon.yml ubuntu@YOUR-OPS-MANAGER-VM-IP:

  4. SSH into Ops Manager.

    $ ssh -i PATH-TO-PRIVATE-KEY ubuntu@YOUR-OPS-MANAGER-VM-IP

  5. On the Ops Manager VM, navigate to the software binary location in your working directory.

    $ cd PATH-TO-BINARY

  6. Log in to the BOSH Director.

    1. On the Ops Manager VM, create an alias in the BOSH CLI for your BOSH Director IP address. For example:
      $ bosh alias-env my-env -e 10.0.0.3
    2. Log in to the BOSH Director, specifying the newly created alias. For example:
      $ bosh -e my-env log-in
  7. Upload your release, specifying the path to the tarballed IPsec binary, by running the following command:

    $ bosh -e my-env upload-release PATH-TO-BINARY/BINARY-NAME.tar

  8. List the releases by running the following command, and confirm that the IPsec binary file appears:

    $ bosh -e my-env releases

  9. Download your current runtime config and save as bosh-manifest.yml by running the following command:

    $ bosh -e my-env runtime-config > bosh-manifest.yml

  10. Update your runtime configuration to include IPsec.

    $ bosh -e my-env update-runtime-config --name=ipsec PATH/bosh-manifest.yml

  11. Verify your runtime configuration changes match what you specified in the IPsec manifest file.

     $ bosh -e my-env runtime-config --name=ipsec

    For example:

    $ bosh -e my-env runtime-config --name=ipsec
    Acting as user 'admin' on 'micro'
    
    releases:
     - {name: ipsec, version: 1.9.9}
    
    addons:
     name: ipsec-addon
      jobs:
      - name: ipsec
        release: ipsec
        ...
    
  12. If you have already deployed TAS for VMs or are adding IPsec to an existing deployment:

    1. Set the optional flag to true.
    2. Navigate to your Installation Dashboard in Ops Manager.
    3. If you are using Ops Manager v2.3 or later, click Review Pending Changes. For more information about this Ops Manager page, see Reviewing Pending Product Changes.
    4. Click Apply Changes
    5. Wait for the installation to complete.
    6. Set the optional flag to false.
    7. Update the runtime config.
      $ bosh -e my-env update-runtime-config --name=ipsec PATH/bosh-manifest.yml
  13. Navigate to your Installation Dashboard.

  14. If you are using Ops Manager v2.3 or later, click Review Pending Changes.

  15. Click Apply Changes.

  16. If the TAS for VMs tile is not yet installed:

    1. Navigate to your Installation Dashboard in Ops Manager.
    2. If you are using Ops Manager v2.3 or later, click Review Pending Changes.
    3. Click Apply Changes
    4. Deploy TAS for VMs by following the installation instructions for your IaaS. For more information, see Architecture and Installation Overview.
  17. The bosh-manifest.yml and ipsec-addon.yml files contain sensitive information. When the deployment process is completed, remove any unneeded copies of these files from the local workstation. VMware recommends that you use encryption or logical access controls to secure any archival copies of manifest files you want to retain.

Verify Your IPsec Installation

After installing IPsec and deploying TAS for VMs, perform the following steps to verify your IPsec installation:

  1. List the job VMs in your deployment by running the following command:

    bosh -e BOSH-ENVIRONMENT vms
    
  2. Open an SSH connection into the VM, using the job name and index of any VM found above, by running the following command:

    bosh -e BOSH-ENVIRONMENT -d DEPLOYMENT-NAME ssh JOB-NAME/INDEX
    

    Note: The exact VM does not matter, because installing IPsec loads IPsec on all VMs deployed by Ops Manager.

  3. Enter the root environment with root privileges by running:

    sudo su -
    
  4. Confirm that your ipsec job is listed as a bosh job by running:

    monit summary
    

    For example:

    $ monit summary
    The Monit daemon 5.2.5 uptime: 18h 32m
    ...
    Process 'ipsec'                     running
    System 'system_localhost'           running
    

  5. Confirm that IPsec is running. If IPsec is not running, this command produces no output.

    PATH-TO-IPSEC/ipsec statusall
    

    For example:

    $ /var/vcap/packages/strongswan-5.6.3/sbin/ipsec statusall
    Status of IKE charon daemon (strongSwan 5.6.3, Linux 3.19.0-56-generic, x86_64):
      uptime: 18 hours, since Mar 16 23:58:50 2016
      malloc: sbrk 2314240, mmap 0, used 1182400, free 1131840
      worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 206
      loaded plugins: charon aes sha1 sha2 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pem gmp xcbc cmac hmac attr kernel-netlink socket-default stroke
    Listening IP addresses:
      10.10.5.66
    Connections:
    ipsec-10.10.4.0/24:  %any...%any  IKEv1/2
    ipsec-10.10.4.0/24:   local:  [CN=test-cert-1-ca-1] uses public key authentication
    ipsec-10.10.4.0/24:    cert:  "CN=test-cert-1-ca-1"
    ipsec-10.10.4.0/24:   remote: uses public key authentication
    ipsec-10.10.9.0/24:   child:  10.10.5.66/32 === 10.10.9.0/24 TRANSPORT
    no-ipsec-10.10.4.1/32:  %any...%any  IKEv1/2
    no-ipsec-10.10.4.1/32:   local:  uses public key authentication
    no-ipsec-10.10.4.1/32:   remote: uses public key authentication
    no-ipsec-10.10.4.1/32:   child:  dynamic === 10.10.4.1/32 PASS
    Shunted Connections:
    no-ipsec-10.10.4.1/32:  dynamic === 10.10.4.1/32 PASS
    no-ipsec-10.10.5.1/32:  dynamic === 10.10.5.1/32 PASS
    no-ipsec-10.10.6.1/32:  dynamic === 10.10.6.1/32 PASS
    Routed Connections:
    ipsec-10.10.9.0/24{6}:  ROUTED, TRANSPORT, reqid 6
    ipsec-10.10.9.0/24{6}:   10.10.5.66/32 === 10.10.9.0/24
    ipsec-10.10.8.0/24{5}:  ROUTED, TRANSPORT, reqid 5
    ipsec-10.10.4.0/24{1}:   10.10.5.66/32 === 10.10.4.0/24
    Security Associations (45 up, 0 connecting):
    ipsec-10.10.4.0/24[459]: ESTABLISHED 13 seconds ago, 10.10.5.66[CN=test-cert-1-ca-1]...10.10.4.38[CN=test-cert-1-ca-1]
    ipsec-10.10.4.0/24{1527}:   10.10.5.66/32 === 10.10.4.38/32
    ...
    

Generate a Self-Signed Certificate with OpenSSL

Follow these steps to generate a self-signed certificate for your IPsec manifest.

  1. Download the openssl-create-ipsec-certs.sh bash script.
  2. Navigate to the directory where you downloaded the script:
    $ cd ~/workspace
  3. Change the permissions of the script:
    $ chmod u+x openssl-create-ipsec-certs.sh
  4. Run the script:
    $ ./openssl-create-ipsec-certs.sh
  5. This generates four files in a new certs directory where the script is run:
    • pcf-ipsec-ca-cert.pem — this value can be used as the CA Cert in the ca_certificates manifest field.
    • pcf-ipsec-ca-key.pem — the key used to sign the generated CA Cert.
    • pcf-ipsec-peer-key.pem — this value can be used as the instance private key in the instance_private_key manifest field.
    • pcf-ipsec-peer-cert.pem — this value can be used as the instance certificate in the instance_certificate manifest field.
  6. Because this certificate expires in 365 days, set a calendar reminder to rotate the certificate within the year. For instructions on changing certificates, see Rotating IPsec Certificates.