LATEST VERSION: v1.1 - RELEASE NOTES
Pivotal Container Service v1.1

Preparing GCP Before Deploying PKS

Page last updated:

This topic describes the preparation steps required to install Pivotal Container Service (PKS) on Google Cloud Platform (GCP).

In addition to fulfilling the prerequisites listed in the GCP Prerequisites and Resource Requirements topic, you must create resources in GCP such as a new network, firewall rules, load balancers, and a service account before deploying PKS. Follow these procedures to prepare your GCP environment.

Step 1: Enable Google Cloud APIs

Ops Manager manages GCP resources using the Google Compute Engine and Cloud Resource Manager APIs. To enable these APIs, perform the following steps:

  1. Log in to the Google Developers console at https://console.developers.google.com.
  2. In the console, navigate to the GCP project where you want to install PKS.
  3. Select Enable APIs & Services to access the API Library.
  4. In the search field, enter Compute Engine API and press Enter.
  5. On the Google Compute Engine API page, click Enable.
  6. In the search field, enter Cloud Resource Manager API and press Enter.
  7. On the Google Cloud Resource Manager API page, click Enable.
  8. To verify that the APIs have been enabled, perform the following steps:
    1. Log in to GCP:
      $ gcloud auth login
      
    2. List your projects:
      $ gcloud projects list
      PROJECT_ID       NAME                 PROJECT_NUMBER
      my-project-id    my-project-name      ##############
      
      This command lists the projects where you enabled Google Cloud APIs.

Step 2: Create Service Accounts

In order for Kubernetes to create load balancers and attach persistent disks to pods, you must create service accounts with sufficient permissions.

You need separate service accounts for Kubernetes cluster master and worker node VMs, and a third account for BOSH and Ops Manager. Pivotal recommends configuring each service account with the least permissive privileges and unique credentials.

Create the Master Node Service Account

  1. From the GCP Console, select IAM & admin > Service accounts.
  2. Click Create Service Account.
  3. Enter a name for the service account, and add the following roles:
    • Compute Engine
      • Compute Instance Admin (v1)
      • Compute Network Admin
      • Compute Security Admin
      • Compute Storage Admin
      • Compute Viewer
    • Service Accounts
      • Service Account User
  4. Click Create.

Create the Worker Node Service Account

  1. From the GCP Console, select IAM & admin > Service accounts.
  2. Click Create Service Account.
  3. Enter a name for the service account, and add the Compute Engine > Compute Viewer role.
  4. Click Create.

Create the BOSH/Ops Manager Service Account

  1. From the GCP Console, select IAM & admin > Service accounts.
  2. Click Create Service Account.
  3. Enter a name for the service account, and add the following roles:
    • Service Accounts
      • Service Account User
      • Service Account Token Creator
    • Compute Engine
      • Compute Instance Admin (v1)
      • Compute Network Admin
      • Compute Storage Admin
    • Storage
      • Storage Admin
  4. Select Furnish a new private key and select JSON.
  5. Click Create. Your browser automatically downloads a JSON file with a private key for this account. Save this file in a secure location.

Note: Pivotal recommends confirming the permissions of your Master Node Service Account, Worker Node Service Account, and BOSH/Ops Manager Service Account after you create them. To verify these account permissions, run the gcloud auth list command. For more information, see gcloud auth in the Google Cloud documentation.

Step 3: Create a GCP Network with Subnets

You must provide infrastructure subnets for Ops Manager, the BOSH Director, and NAT instances. These subnets may already exist in environments with an existing Ops Manager deployment.

If these infrastructure subnets do not exist in your environment, follow the steps below to create them.

  1. Log in to the GCP Console.

  2. Navigate to the GCP project where you want to install PKS.

  3. Select VPC network, then CREATE VPC NETWORK.

  4. In the Name field, enter your-pks-virt-net. your-pks is a lower-case prefix to help you identify resources for this PKS deployment in the GCP console. Network names must be lower-case. Use the values from the following tables as a guide when you create each network, replacing the IP addresses with ranges that are available in your GCP environment.

    Note: Pivotal recommends using all three networks in production environments. You can combine pks-infrastructure and pks-main into a single network in non-production environments. pks-services always requires its own network.

    1. Under Subnets, complete the form as follows to create an infrastructure subnet for Ops Manager, the BOSH Director, and NAT instances:
      Name MY-PKS-subnet-infrastructure-GCP-REGION
      Region A region that supports three availability zones (AZs). For help selecting the correct region for your deployment, see Regions and Zones in the Google documentation.
      IP address range A CIDR ending in /26
      Example: 192.168.101.0/26

    2. Click Add subnet to add a second subnet for the PKS control plane with the following details:
      Name MY-PKS-subnet-pks-GCP-REGION
      Region The same region you selected for the infrastructure subnet
      IP address range A CIDR ending in /26
      Example: 192.168.16.0/26

    3. Click Add subnet to add a third subnet for the Kubernetes clusters with the following details:
      Name MY-PKS-subnet-services-GCP-REGION
      Region The same region you selected for the previous subnets
      IP address range A CIDR in /22
      Example: 192.168.20.0/22

  5. Under Dynamic routing mode, leave Regional selected.

  6. Click Create.

Step 4: Create NAT Instances

Use NAT instances when you want to expose only a minimal number of public IP addresses.

Creating NAT instances permits Internet access from cluster VMs. You might, for example, need this Internet access for pulling Docker images or enabling Internet access for your workloads.

  1. In the console, navigate to Compute Engine > VM instances.
  2. Click CREATE INSTANCE.
  3. Complete the following fields:

    • Name: Enter MY-PKS-nat-gateway-pri. This is the first, or primary, of three NAT instances you need. If you are using a single AZ, you need only one NAT instance.
    • Zone: Select the first zone from your region. Example: For region us-west1, select zone us-west1-a.
    • Machine type: Select n1-standard-4.
    • Boot disk: Click Change and select Ubuntu 14.04 LTS.
  4. Expand the additional configuration fields by clicking Management, disks, networking, SSH keys.

    1. In the Startup script field under Automation, enter the following text: #! /bin/bash
      sudo sh -c 'echo 1 > /proc/sys/net/ipv4/ip_forward'
      sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
  5. Click Networking to open additional network configuration fields:

    1. In the Network tags field, add the following: nat-traverse and MY-PKS-nat-instance.
    2. Click the pencil icon to edit the Network interface.
    3. For Network, select your-pks-virt-net. You created this network in Step 3: Create a GCP Network with Subnets.
    4. For Subnetwork, select MY-PKS-subnet-infrastructure-GCP-REGION.
    5. For Primary internal IP, select Ephemeral (Custom).
    6. Enter an IP address in the Custom ephemeral IP address field. Example: 192.168.101.2. The IP address must meet the following requirements:
      • The IP address must exist in the CIDR range you set for the MY-PKS-subnet-infrastructure-GCP-REGION subnet.
      • The IP address must exist in a reserved IP range set later in Ops Manager Director. The reserved range is typically the first .1 through .9 addresses in the CIDR range you set for the MY-PKS-subnet-infrastructure-GCP-REGION subnet.
      • The IP address cannot be the same as the Gateway IP address set later in Ops Manager. The Gateway IP address is typically the first .1 address in the CIDR range you set for the MY-PKS-subnet-infrastructure-GCP-REGION subnet.
    7. For External IP, select Ephemeral.
    8. Set IP forwarding to On.
    9. Click Done.
  6. Click Create to finish creating the NAT instance.

  7. To create additional NAT instances, repeat steps 2-6 using the names and zones specified in the table below.

    Instance 2 Name MY-PKS-nat-gateway-sec
    Zone Select the second zone from your region.
    Example: For region us-west1, select zone us-west1-b.
    Internal IP Select Custom and enter an IP address in the Internal IP address field. Example: 192.168.101.3.

    As described above, this address must in the CIDR range you set for the MY-PKS-subnet-infrastructure-GCP-REGION subnet, must exist in a reserved IP range set later in Ops Manager Director, and cannot be the same as the Gateway IP address set later in Ops Manager.
    Instance 3 Name MY-PKS-nat-gateway-ter
    Zone Select the third zone from your region.
    Example: For region us-west1, select zone us-west1-c.
    Internal IP Select Custom and enter an IP address in the Internal IP address field. Example: 192.168.101.4.

    As described above, this address must in the CIDR range you set for the MY-PKS-subnet-infrastructure-GCP-REGION subnet, must exist in a reserved IP range set later in Ops Manager Director, and cannot be the same as the Gateway IP address set later in Ops Manager.

Create Routes for NAT Instances

  1. In the GCP console, navigate to VPC Networks > Routes.

  2. Click CREATE ROUTE.

  3. Complete the form as follows:

    • Name: MY-PKS-nat-pri
    • Network: your-pks-virt-net
    • Destination IP range: 0.0.0.0/0
    • Priority: 800
    • Instance tags: MY-PKS
    • Next hop: Specify an instance
    • Next hop instance: MY-PKS-nat-gateway-pri
  4. Click Create to finish creating the route.

  5. Repeat steps 2-4 to create two additional routes with the names and next hop instances specified in the table below. The rest of the configuration remains the same.

    Route 2 Name: MY-PKS-nat-sec
    Next hop instance: MY-PKS-nat-gateway-sec
    Route 3 Name: MY-PKS-nat-ter
    Next hop instance: MY-PKS-nat-gateway-ter

Step 5: Create Firewall Rules for the Network

GCP lets you assign tags to virtual machine (VM) instances and create firewall rules that apply to VMs based on their tags. This step assigns tags and firewall rules to Ops Manager components and VMs that handle incoming traffic.

  1. Follow the instructions in the GCP documentation to create firewall rules according to the table below. For more information, see Firewall Rules Overview in the GCP documentation.

    Note: If you want your firewalls rules to only allow traffic within your private network, modify the Source IP Ranges from the table accordingly.

    Firewall Rules
    Rule 1 This rule allows SSH from public networks.

    Name: MY-PKS-allow-ssh
    Network: your-pks-virt-net
    Allowed protocols and ports: tcp:22
    Source filter: IP ranges
    Source IP ranges: 0.0.0.0/0
    Target tags: allow-ssh
    Rule 2 This rule allows HTTP from public networks.

    Name: MY-PKS-allow-http
    Network: your-pks-virt-net
    Allowed protocols and ports: tcp:80
    Source filter: IP ranges
    Source IP ranges: 0.0.0.0/0
    Target tags: allow-http, router
    Rule 3 This rule allows HTTPS from public networks.

    Name: MY-PKS-allow-https
    Network: your-pks-virt-net
    Allowed protocols and ports: tcp:443
    Source filter: IP ranges
    Source IP ranges: 0.0.0.0/0
    Target tags: allow-https, router
    Rule 4 This rule allows communication between BOSH-deployed jobs.

    Name: MY-PKS-allow-pks-all
    Network: your-pks-virt-net
    Allowed protocols and ports: tcp;udp;icmp
    Source filter: Source tags
    Target tags: MY-PKS, MY-PKS-opsman, nat-traverse
    Source tags: MY-PKS, MY-PKS-opsman, nat-traverse

  2. (Optional) If you use your GCP project only to deploy PKS, you can delete the following default firewall rules:

    • default-allow-http
    • default-allow-https
    • default-allow-icmp
    • default-allow-internal
    • default-allow-rdp
    • default-allow-ssh

Next Steps

To install PKS on GCP, follow the procedures in Deploying Ops Manager on GCP.


Please send any feedback you have to pks-feedback@pivotal.io.

Create a pull request or raise an issue on the source for this page in GitHub