Deploying Ops Manager on Azure

Page last updated:

This topic describes how to deploy BOSH and Pivotal Operations Manager for Pivotal Platform on Azure by using individual commands to create resources. Pivotal recommends this manual procedure for deploying to Azure China, Azure Germany, and Azure Government Cloud.

Before you perform the procedures in this topic, you must have completed the procedures in Preparing to Deploy Ops Manager on Azure. After you complete the procedures in this topic, follow the instructions in Configuring BOSH Director on Azure.

Note: If you are deploying BOSH and Ops Manager on Azure Stack, complete the procedures in Install and configure CLI for use with Azure Stack in the Microsoft documentation before following the procedures in this topic.

Note: The Azure portal sometimes displays the names of resources with incorrect capitalization. Always use the Azure CLI to retrieve the correctly capitalized name of a resource.

Step 1: Create Network Resources

To create network resources for your deployment, do the following:

1. Navigate to the Azure portal, click Resource groups, and click Add to create a new resource group for your deployment.

2. Enter a Resource group name, select your Subscription, and select a Resource group location. Click Create.

3. Export the name of your resource group as the environment variable $RESOURCE_GROUP.

   $ export RESOURCE_GROUP="YOUR-RESOURCE-GROUP-NAME"
   

   Note: If you are on a Windows machine, you can use set instead of export.

4. Export your location. For example, westus.

   $ export LOCATION=westus
   

   For a list of available locations, run az account list-locations.

5. Create a network security group named pcf-nsg.

   $ az network nsg create --name pcf-nsg \
   --resource-group $RESOURCE_GROUP \
   --location $LOCATION
   

6. Add network security group rules to the pcf-nsg group to allow traffic to known ports from the public Internet.

   $ az network nsg rule create --name ssh \
   --nsg-name pcf-nsg --resource-group $RESOURCE_GROUP \
   --protocol Tcp --priority 100 \
   --destination-port-range '22'
   

   $ az network nsg rule create --name http \
   --nsg-name pcf-nsg --resource-group $RESOURCE_GROUP \
   --protocol Tcp --priority 200 \
   --destination-port-range '80'
   

   $ az network nsg rule create --name https \
   --nsg-name pcf-nsg --resource-group $RESOURCE_GROUP \
   --protocol Tcp --priority 300 \
   --destination-port-range '443'
   

   $ az network nsg rule create --name diego-ssh \
   --nsg-name pcf-nsg --resource-group $RESOURCE_GROUP \
   --protocol Tcp --priority 400 \
   --destination-port-range '2222'
   

   • To block traffic from the public Internet, append --source-address-prefixes AzureLoadBalancer to allow traffic from only the Azure load balancer or --source-address-prefixes VirtualNetwork to only allow traffic from the virtual network.
   • To allow traffic from both the Azure load balancer and the virtual network, create duplicates of each rule, one specifying --source-address-prefixes AzureLoadBalancer and one specifying --source-address-prefixes VirtualNetwork.

7. Create a network security group named opsmgr-nsg.

   $ az network nsg create --name opsmgr-nsg \
   --resource-group $RESOURCE_GROUP \
   --location $LOCATION
   

8. Add a network security group rule to the opsmgr-nsg group to allow HTTP traffic to the Ops Manager VM.

   $ az network nsg rule create --name http \
   --nsg-name opsmgr-nsg --resource-group $RESOURCE_GROUP \
   --protocol Tcp --priority 100 \
   --destination-port-range 80
   

9. Add a network security group rule to the opsmgr-nsg group to allow HTTPS traffic to the Ops Manager VM.

   $ az network nsg rule create --name https \
   --nsg-name opsmgr-nsg --resource-group $RESOURCE_GROUP \
   --protocol Tcp --priority 200 \
   --destination-port-range 443
   

10. Add a network security group rule to the opsmgr-nsg group to allow SSH traffic to the Ops Manager VM.

    $ az network nsg rule create --name ssh \
    --nsg-name opsmgr-nsg --resource-group $RESOURCE_GROUP \
    --protocol Tcp --priority 300 \
    --destination-port-range 22
    

    • To block traffic from the public Internet, append --source-address-prefixes AzureLoadBalancer to allow traffic from only the Azure load balancer or --source-address-prefixes VirtualNetwork to only allow traffic from the virtual network.
    • To allow traffic from both the Azure load balancer and the virtual network, create duplicates of each rule, one specifying --source-address-prefixes AzureLoadBalancer and one specifying --source-address-prefixes VirtualNetwork.
    • Optionally, if you want to use private IP ranges with Ops Manager and allow all internal traffic, you can create the Network Security Groups to allow all internal traffic.

      $ az network nsg rule create --name internal-virtual-network \
      --nsg-name internal-traffic --resource-group $RESOURCE_GROUP \
      --protocol Tcp --priority 100 \
      --destination-port-range * \
      --source-address-prefixes VirtualNetwork
      $ az network nsg rule create --name internal-from-lb \
      --nsg-name internal-traffic --resource-group $RESOURCE_GROUP \
      --protocol Tcp --priority 110 \
      --destination-port-range * \
      --source-address-prefixes AzureLoadBalancer
      

11. Create a virtual network named pcf-virtual-network.

    $ az network vnet create --name pcf-virtual-network \
    --resource-group $RESOURCE_GROUP --location $LOCATION \
    --address-prefixes 10.0.0.0/16
    

12. Add subnets to the network for Ops Manager, BOSH director, and the VMs for your runtime, and attach the Network Security Group.

    $ az network vnet subnet create --name pcf-infrastructure-subnet \
    --vnet-name pcf-virtual-network \
    --resource-group $RESOURCE_GROUP \
    --address-prefix 10.0.4.0/26 \
    --network-security-group pcf-nsg
    $ az network vnet subnet create --name pcf-pas-subnet \
    --vnet-name pcf-virtual-network \
    --resource-group $RESOURCE_GROUP \
    --address-prefix 10.0.12.0/22 \
    --network-security-group pcf-nsg
    $ az network vnet subnet create --name pcf-services-subnet \
    --vnet-name pcf-virtual-network \
    --resource-group $RESOURCE_GROUP \
    --address-prefix 10.0.8.0/22 \
    --network-security-group pcf-nsg
    

Step 2: Create BOSH and Deployment Storage Accounts

Pivotal Platform on Azure uses multiple general-purpose Azure storage accounts. The BOSH and Ops Manager VMs use one main BOSH account, and the other components share five or more deployment storage accounts.

To create storage accounts for BOSH and your deployment, do the following:

1. Choose a name for your BOSH storage account, and export it as the environment variable $STORAGE_NAME. Storage account names must be globally unique across Azure, between 3 and 24 characters in length, and contain only lowercase letters and numbers.

   $ export STORAGE_NAME="YOUR-BOSH-STORAGE-ACCOUNT-NAME"
   

2. Create a Standard storage account for BOSH with the following command. This account will be used for BOSH bookkeeping and running the Ops Manager VM itself, but does not have to be used for running any other VMs.

   $ az storage account create --name $STORAGE_NAME \
   --resource-group $RESOURCE_GROUP \
   --sku Standard_LRS \
   --location $LOCATION
   

   Note: Standard_LRS refers to a Standard Azure storage account. The BOSH Director requires table storage to store stemcell information. Azure Premium storage does not support table storage and cannot be used for the BOSH storage account.

   If the command fails, ensure you have followed the rules for naming your storage account. Export another new storage account name if necessary.

3. Configure the Azure CLI to use the BOSH storage account as its default.

   1. Retrieve the connection string for the account.

      $ az storage account show-connection-string \
      --name $STORAGE_NAME --resource-group $RESOURCE_GROUP
      

      The command returns output similar to the following:

      {
      "connectionString": "DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=cfdocsboshstorage;AccountKey=EXAMPLEaaaaabbbrnc5igFxYWsgq016Tu9uGwseOl8bqNBEL/2tp7wX92QMUM19Pz9BYTXt8aq4A=="
      }
      

   2. Record the full value of connectionString from the output above, starting with and including DefaultEndpointsProtocol=.
   3. Export the value of connectionString as the environment variable $CONNECTION_STRING.

      $ export CONNECTION_STRING="YOUR-ACCOUNT-KEY-STRING"
      

4. Create three blob containers in the BOSH storage account, named opsmanager, bosh, and stemcell.

   $ az storage container create --name opsmanager \
   --connection-string $CONNECTION_STRING
   $ az storage container create --name bosh \
   --connection-string $CONNECTION_STRING
   $ az storage container create --name stemcell --public-access blob \
   --connection-string $CONNECTION_STRING
   

5. Create a table named stemcells.

   $ az storage table create --name stemcells \
   --connection-string $CONNECTION_STRING
   

6. Choose a set of unique names for five or more deployment storage accounts. As with the BOSH storage account above, the names must be unique, alphanumeric, lowercase, and 3-24 characters long. The account names must also be sequential or otherwise identical except for the last character. For example: xyzdeploystorage1, xyzdeploystorage2, xyzdeploystorage3, xyzdeploystorage4, and xyzdeploystorage5.

7. Decide which type of storage to use and run the corresponding command below.

   Note: Pivotal recommends five Premium storage accounts, which provides a reasonable amount of initial storage capacity. You can use either Premium or Standard storage accounts, but they have very different scalability metrics. Pivotal recommends creating 1 Standard storage account for every 30 VMs, or 1 Premium storage account for every 150 VMs. You can increase the number of storage accounts later by provisioning more and following the naming sequence.

   • To use Premium storage (recommended):

     $ export STORAGE_TYPE="Premium_LRS"
     

   • To use Standard storage:

     $ export STORAGE_TYPE="Standard_LRS"
     

8. For each deployment storage account you create, do the following:

   1. Create the storage account with the following command, replacing MY_DEPLOYMENT_STORAGE_X with one of your deployment storage account names.

      $ az storage account create --name MY_DEPLOYMENT_STORAGE_X \
      --resource-group $RESOURCE_GROUP --sku $STORAGE_TYPE \
      --kind Storage --location $LOCATION
      

      If the command fails, try a different set of account names.
   2. Retrieve the connection string for the account.

      $ az storage account show-connection-string \
      --name MY_DEPLOYMENT_STORAGE_X --resource-group $RESOURCE_GROUP
      

      The command returns output similar to the following:

      {
      "connectionString": "DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=cfdocsdeploystorage1;AccountKey=EXAMPLEaaaaaaaQiSAmqj1OocsGhKBwnMf8wEwdeJMvvonrbmNk27bfkSL8ZFzAhs3Kb78si5CTPHhjHHiK4qPcYzn/8OmFg=="
      }
      

   3. Record the full value of connectionString from the output above, starting with and including DefaultEndpointsProtocol=.
   4. Create two blob containers named bosh and stemcell in the account.

      $ az storage container create --name bosh \
      --connection-string $CONNECTION_STRING
      

      $ az storage container create --name stemcell \
      --connection-string $CONNECTION_STRING
      

Step 3: Create Load Balancers

Your load balancer configuration depends on whether you want apps to be available from public IP addresses, private IP addresses, or both.

To create load balancers, do the following:

1. Required: PAS Load Balancer

   1. Create a load balancer named pcf-lb. A static IP address will be automatically created for Standard SKU load balancers unless specified otherwise with --public-ip-address-allocation Dynamic.

      $ az network lb create --name pcf-lb \
      --resource-group $RESOURCE_GROUP --location $LOCATION \
      --backend-pool-name pcf-lb-be-pool --frontend-ip-name pcf-lb-fe-ip \
      --public-ip-address pcf-lb-ip --public-ip-address-allocation Static \
      --sku Standard
      

      Note: If the Standard SKU is not available in Azure China, you can use the Basic SKU.

      This back end pool is empty when you create it.
   2. Add a probe to the load balancer.

      $ az network lb probe create --lb-name pcf-lb \
      --name http8080 --resource-group $RESOURCE_GROUP \
      --protocol Http --port 8080 --path health
      

   3. Add a load balancing rule for HTTP.

      $ az network lb rule create --lb-name pcf-lb \
      --name http --resource-group $RESOURCE_GROUP \
      --protocol Tcp --frontend-port 80 \
      --backend-port 80 --frontend-ip-name pcf-lb-fe-ip \
      --backend-pool-name pcf-lb-be-pool \
      --probe-name http8080
      

   4. Add a load balancing rule for HTTPS.

      $ az network lb rule create --lb-name pcf-lb \
      --name https --resource-group $RESOURCE_GROUP \
      --protocol Tcp --frontend-port 443 \
      --backend-port 443 --frontend-ip-name pcf-lb-fe-ip \
      --backend-pool-name pcf-lb-be-pool \
      --probe-name http8080
      

   5. (Optional) For private IPs, do the following:
      1. Create a load balancer named pcf-lb.

         $ az network lb create --name pcf-lb \
         --resource-group $RESOURCE_GROUP --location $LOCATION \
         --backend-pool-name pcf-lb-be-pool --frontend-ip-name pcf-lb-fe-ip \
         --private-ip-address 10.0.12.6 --sku Standard \
         --vnet-name pcf-virtual-network \
         --subnet pcf-pas-subnet
         

      2. Add a probe to the load balancer.

         $ az network lb probe create --lb-name pcf-lb \
         --name http8080 --resource-group $RESOURCE_GROUP \
         --protocol Http --port 8080 --path health
         

      3. Add a load balancing rule for HTTP.

         $ az network lb rule create --lb-name pcf-lb \
         --name http --resource-group $RESOURCE_GROUP \
         --protocol Tcp --frontend-port 80 \
         --backend-port 80 --frontend-ip-name pcf-lb-fe-ip \
         --backend-pool-name pcf-lb-be-pool \
         --probe-name http8080
         

      4. Add a load balancing rule for HTTPS.

         $ az network lb rule create --lb-name pcf-lb \
         --name https --resource-group $RESOURCE_GROUP \
         --protocol Tcp --frontend-port 443 \
         --backend-port 443 --frontend-ip-name pcf-lb-fe-ip \
         --backend-pool-name pcf-lb-be-pool \
         --probe-name http8080
         

         Note: If the Standard SKU is not available in Azure China, you can use the Basic SKU.

         This back end pool is empty when you create it.
   6. Navigate to your DNS provider and create an entry that points *.YOUR-SUBDOMAIN to the public IP address of your load balancer. For example, create an entry that points azure.example.com to 198.51.100.1. You can retrieve it by running az network public-ip show --name pcf-lb-ip --resource-group $RESOURCE_GROUP.

2. Optional: Diego SSH Load Balancer

   1. Create a load balancer named pcf-ssh-lb.

      $ az network lb create --name pcf-ssh-lb \
      --resource-group $RESOURCE_GROUP --location $LOCATION \
      --backend-pool-name pcf-ssh-lb-be-pool --frontend-ip-name pcf-ssh-lb-fe-ip \
      --public-ip-address pcf-ssh-lb-ip --public-ip-address-allocation Static \
      --sku Standard
      

      Note: If the Standard SKU is not available in Azure China, you can use the Basic SKU.

      This back end pool is empty when you create it.
   2. (Optional) For private IPs, create a load balancer named pcf-ssh-lb.

      $ az network lb create --name pcf-ssh-lb \
      --resource-group $RESOURCE_GROUP --location $LOCATION \
      --backend-pool-name pcf-ssh-lb-be-pool --frontend-ip-name pcf-ssh-lb-fe-ip \
      --private-ip-address 10.0.12.7 \
      --sku Standard \
      --vnet-name pcf-virtual-network \
      --subnet pcf-pas-subnet
      

      Note: If the Standard SKU is not available in Azure China, you can change to use the Basic SKU.

      This back end pool is empty when you create it.
   3. Add a probe to the load balancer.

      $ az network lb probe create --lb-name pcf-ssh-lb \
      --name tcp2222 --resource-group $RESOURCE_GROUP \
      --protocol Tcp --port 2222
      

   4. Add a load balancing rule for SSH.

      $ az network lb rule create --lb-name pcf-ssh-lb \
      --name diego-ssh --resource-group $RESOURCE_GROUP \
      --protocol Tcp --frontend-port 2222 \
      --backend-port 2222 --frontend-ip-name pcf-ssh-lb-fe-ip \
      --backend-pool-name pcf-ssh-lb-be-pool \
      --probe-name tcp2222
      

   5. Navigate to your DNS provider, and create an entry that points ssh.sys.YOUR-SUBDOMAIN to the public IP address of your load balancer. For example, create an entry that points azure.example.com to 198.51.100.1. You can retrieve it by running az network public-ip show --name pcf-ssh-lb-ip --resource-group $RESOURCE_GROUP.

Step 4: Boot Ops Manager

To boot Ops Manager, do the following:

1. Navigate to Pivotal Network and download the latest release of Pivotal Ops Manager for Azure.

2. View the downloaded PDF and locate the Ops Manager image URL appropriate for your region.

3. Export the Ops Manager image URL as an environment variable.

   $ export OPS_MAN_IMAGE_URL="YOUR-OPS-MAN-IMAGE-URL"

4. Download the Ops Manager image. For compatibility when upgrading to future versions of Ops Manager, choose a unique name for the image that includes the Ops Manager version number. For example, replace opsman-image-2.6.x in the following examples with opsman-image-2.6.1.

   • If you use unmanaged disks, perform the following steps:

     Note: Azure Stack requires unmanaged disks.

     1. Download the Ops Manager image to your local machine. The image size is 10 GB.

        $ wget $OPS_MAN_IMAGE_URL -O opsman-image-2.6.x.vhd

     2. Upload the image to your storage account using the Azure CLI.

        $ az storage blob upload --name opsman-image-2.6.x.vhd \
        --connection-string $CONNECTION_STRING \
        --container-name opsmanager \
        --type page \
        --file opsman-image-2.6.x.vhd
        

   • If you use managed disks, do the following:
     1. Copy the Ops Manager image into your storage account using the Azure CLI.

        $ az storage blob copy start --source-uri $OPS_MAN_IMAGE_URL \
        --connection-string $CONNECTION_STRING \
        --destination-container opsmanager \
        --destination-blob opsman-image-2.6.x.vhd
        

     2. Copying the image may take several minutes. Run the following command and examine the output under "copy":

        $ az storage blob show --name opsman-image-2.6.x.vhd \
        --container-name opsmanager \
        --connection-string $CONNECTION_STRING
        ...
        "copy": {
        "completionTime": "2017-06-26T22:24:11+00:00",
        "id": "b9c8b272-a562-4574-baa6-f1a04afcefdf",
        "progress": "53687091712/53687091712",
        "source": "https://opsmanagerwestus.blob.core.windows.net/images/opsman-image-2.6.x.vhd",
        "status": "success",
        "statusDescription": null
        },
        

     3. Wait a few moments and re-run the command above if status is pending. When status reads success, continue to the next step.

5. Create a public IP address named ops-manager-ip.

   $ az network public-ip create --name ops-manager-ip \
   --resource-group $RESOURCE_GROUP --location $LOCATION \
   --allocation-method Static
   {
     "publicIp": {
       "dnsSettings": null,
       "etag": "W/\"4450ebe2-9e97-4b17-9cf2-44838339c661\"",
       "id": "/subscriptions/995b7eed-77ef-45ff-a5c9-1a405ffb8243/resourceGroups/cf-docs/providers/Microsoft.Network/publicIPAddresses/ops-manager-ip",
       "idleTimeoutInMinutes": 4,
       "ipAddress": "40.83.148.183",
       "ipConfiguration": null,
       "location": "westus",
       "name": "ops-manager-ip",
       "provisioningState": "Succeeded",
       "publicIpAddressVersion": "IPv4",
       "publicIpAllocationMethod": "Static",
       "resourceGroup": "cf-docs",
       "resourceGuid": "950d4831-1bec-42da-8a79-959bcddea9dd",
       "tags": null,
       "type": "Microsoft.Network/publicIPAddresses"
     }
   }
   

   If you do not want to use a public IP for Ops Manager, skip this step.

6. Record the ipAddress from the output above. This is the public IP address of Ops Manager.

7. Create a network interface for Ops Manager.

   $ az network nic create --vnet-name pcf-virtual-network \
   --subnet pcf-infrastructure-subnet --network-security-group opsmgr-nsg \
   --private-ip-address 10.0.4.4 \
   --public-ip-address ops-manager-ip \
   --resource-group $RESOURCE_GROUP \
   --name opsman-nic --location $LOCATION
   

   If you do not want use a public IP address for Ops Manager, remove the --public-ip-address ops-manager-ip flag and value.

8. Create a keypair on your local machine with the username ubuntu. For example, enter the following command:

   $ ssh-keygen -t rsa -f opsman -C ubuntu
   

   
   When prompted for a passphrase, press the enter key to provide an empty passphrase.

9. Create the Ops Manager VM.

   • If you are using unmanaged disks, run the following command to create your Ops Manager VM, replacing PATH-TO-PUBLIC-KEY with the path to your public key .pub file:

     $ az vm create --name opsman-2.6.x --resource-group $RESOURCE_GROUP \
     --location $LOCATION \
     --nics opsman-nic \
     --image https://$STORAGE_NAME.my-azure-instance.com/opsmanager/opsman-image-2.6.x.vhd \
     --os-disk-name opsman-2.6.x-osdisk \
     --os-disk-size-gb 128 \
     --os-type Linux \
     --use-unmanaged-disk \
     --storage-account $STORAGE_NAME \
     --storage-container-name opsmanager \
     --admin-username ubuntu \
     --ssh-key-value PATH-TO-PUBLIC-KEY
     

     Replace my-azure-instance.com with the URL of your Azure instance. Find the complete source URL in the Azure UI by viewing the Blob properties of the Ops Manager image you created earlier in this procedure.
     
     
   • If you are using Azure managed disks, do the following:
     1. Create a managed image from the Ops Manager VHD file:

        $ az image create --resource-group $RESOURCE_GROUP \
        --name opsman-image-2.6.x \
        --source https://$STORAGE_NAME.blob.core.windows.net/opsmanager/image-2.6.x.vhd \
        --location $LOCATION \
        --os-type Linux
        

        If you are using Azure China, Azure Government Cloud, or Azure Germany, replace blob.core.windows.net with the following:
        • For Azure China, use blob.core.chinacloudapi.cn. For more information, see the Azure documentation.
        • For Azure Government Cloud, use blob.core.usgovcloudapi.net. For more information, see the Azure documentation.
        • For Azure Germany, use blob.core.cloudapi.de. For more information, see the Azure documentation.
     2. Create your Ops Manager VM, replacing PATH-TO-PUBLIC-KEY with the path to your public key .pub file.

         $ az vm create --name opsman-2.6.x --resource-group $RESOURCE_GROUP \
         --location $LOCATION \
         --nics opsman-nic \
         --image opsman-image-2.6.x \
         --os-disk-size-gb 128 \
         --os-disk-name opsman-2.6.x-osdisk \
         --admin-username ubuntu \
         --size Standard_DS2_v2 \
         --storage-sku Standard_LRS \
         --ssh-key-value PATH-TO-PUBLIC-KEY
        

10. If you plan to install more than one tile in this Ops Manager installation, do the following to increase the size of the Ops Manager VM disk. You can repeat this process and increase the disk again at a later time if necessary.

    Note: If you use Azure Stack, you must increase the Ops Manager VM disk size using the Azure Stack UI.

    1. Run the following command to stop the VM and detach the disk:

       $ az vm deallocate --name opsman-2.6.x \
       --resource-group $RESOURCE_GROUP
       

    2. Run the following command to resize the disk to 128 GB:

       $ az disk update --size-gb 128 --name opsman-2.6.x-osdisk \
       --resource-group $RESOURCE_GROUP
       

    3. Run the following command to start the VM:

       $ az vm start --name opsman-2.6.x --resource-group $RESOURCE_GROUP
       

Step 5: Complete BOSH Director Configuration

To finish configuring BOSH Director, do the following:

1. Navigate to your DNS provider, and create an entry that points a fully qualified domain name (FQDN) to the public IP address of Ops Manager. As a best practice, always use the FQDN to access Ops Manager.

2. Continue to Configuring BOSH Director on Azure.