LATEST VERSION: v1.0 - CHANGELOG
Pivotal Container Service v1.0

Preparing to Deploy PKS on vSphere

Page last updated:

Before you install Pivotal Container Service (PKS) on vSphere without NSX-T integration, you must prepare your vSphere environment. In addition to fulfilling the prerequisites specified in vSphere Prerequisites and Resource Requirements, you must create the following two service accounts:

  • Master Node Service Account: You must create a service account for Kubernetes cluster master VMs and grant it sufficient storage permissions to create load balancers and attach persistent disks to pods.

  • BOSH/Ops Manager Service Account: You must create a service account for BOSH and Ops Manager.

Note: If your Kubernetes clusters span multiple vCenters, you must set the service account privileges correctly in each vCenter.

vSphere Cloud Provider service accounts require the following:

  • Read access to the folder, host, and datacenter of the cluster node VMs
  • Permission to create and delete VMs within the resource pool where PKS is deployed

Step 1: Create the Master Node Service Account

  1. From the vCenter console, create a service account for Kubernetes cluster master VMs.

  2. Grant the following Virtual Machine Object permissions to the service account:

    Privilege (UI)Privilege (API)
    AdvancedVirtualMachine.Configuration.Advanced
    SettingsVirtualMachine.Configuration.Settings

Step 2: Grant Additional Storage Permissions

Grant additional permissions to the master node service account based on your storage configuration.

The tables in Storage Permissions for Service Accounts describe the following:

See vSphere Storage for Kubernetes in the VMware documentation for more information.

Storage Permissions for Service Accounts

The following tables describe the minimum permissions required by the master node service account based on your storage configuration.

Static Only Persistent Volume Provisioning

Roles Privileges Entities Propagate to Children
manage-k8s-node-vms
  • VirtualMachine.Config.AddExistingDisk
  • VirtualMachine.Config.AddNewDisk
  • VirtualMachine.Config.AddRemoveDevice
  • VirtualMachine.Config.RemoveDisk
VM Folder Yes
manage-k8s-volumes Datastore.FileManagement (Low level file operations) Datastore No
Read-only (pre-existing default role)
  • System.Anonymous
  • System.Read
  • System.View
  • vCenter
  • Datacenter
  • Datastore Cluster
  • Datastore Storage Folder
No

Note: Datastore.FileManagement is only required for the role manage-k8s-volumes if a Persistent Volume Claim (PVC) is created to bind with a statically provisioned Persistent Volume (PV), and the reclaim policy set to delete. When the PVC is deleted, the statically provisioned PV is also deleted.

Dynamic Persistent Volume Provisioning (with Storage Policy-Based Volume Placement)

Roles Privileges Entities Propagate to Children
manage-k8s-node-vms
  • Resource.AssignVMToPool
  • VirtualMachine.Config.AddExistingDisk
  • VirtualMachine.Config.AddNewDisk
  • VirtualMachine.Config.AddRemoveDevice
  • VirtualMachine.Config.RemoveDisk
  • VirtualMachine.Inventory.Create
  • VirtualMachine.Inventory.Delete
  • Cluster
  • Hosts
  • VM Folder
Yes
manage-k8s-volumes
  • Datastore.AllocateSpace
  • Datastore.FileManagement (Low level file operations)
Datastore No
k8s-system-read-and-spbm-profile-view StorageProfile.View (Profile-driven storage view) vCenter No
Read-only (pre-existing default role)
  • System.Anonymous
  • System.Read
  • System.View
  • Datacenter
  • Datastore Cluster
  • Datastore Storage Folder
No

Dynamic Volume Provisioning (without Storage Policy-Based Volume Placement)

Roles Privileges Entities Propagate to Children
manage-k8s-node-vms
  • VirtualMachine.Config.AddExistingDisk
  • VirtualMachine.Config.AddNewDisk
  • VirtualMachine.Config.AddRemoveDevice
  • VirtualMachine.Config.RemoveDisk
VM Folder Yes
manage-k8s-volumes
  • Datastore.AllocateSpace
  • Datastore.FileManagement (Low level file operations)
Datastore No
Read-only (pre-existing default role)
  • System.Anonymous
  • System.Read
  • System.View
  • vCenter
  • Datacenter
  • Datastore Cluster
  • Datastore Storage Folder
No

Step 3: Create the BOSH/Ops Manager Service Account

  1. From the vCenter console, create a service account for BOSH and Ops Manager.

  2. Grant the permissions below to the BOSH and Ops Manager service account.

vCenter Root Privileges

Grant the following privileges on the root vCenter server entity to the service account:

Privilege (UI)Privilege (API)
Read-onlySystem.Anonymous
System.Read
System.View
Manage custom attributesGlobal.ManageCustomFields

vCenter Datacenter Privileges

Grant the following privileges on any entities in a datacenter where you deploy PKS:

Role Object

Privilege (UI)Privilege (API)
Users inherit the Read-Only role from the vCenter root levelSystem.Anonymous
System.Read
System.View

Datastore Object

Grant the following privileges must at the datacenter level to upload and delete virtual machine files:

Privilege (UI)Privilege (API)
Allocate spaceDatastore.AllocateSpace
Browse datastoreDatastore.Browse
Low level file operationsDatastore.FileManagement
Remove fileDatastore.DeleteFile
Update virtual machine filesDatastore.UpdateVirtualMachineFiles

Folder Object

Privilege (UI)Privilege (API)
Delete folderFolder.Delete
Create folderFolder.Create
Move folderFolder.Move
Rename folderFolder.Rename

Global Object

Privilege (UI)Privilege (API)
Set custom attributeGlobal.SetCustomField
Host Object
Privilege (UI)Privilege (API)
Modify clusterHost.Inventory.EditCluster

Inventory Service Object

Privilege (UI)Privilege (API)
vSphere Tagging > Create vSphere TagInventoryService.Tagging.CreateTag
vSphere Tagging > Delete vSphere TagInventoryService.Tagging.EditTag
vSphere Tagging > Edit vSphere TagInventoryService.Tagging.DeleteTag

Network Object

Privilege (UI)Privilege (API)
Assign networkNetwork.Assign

Resource Object

Privilege (UI)Privilege (API)
Assign virtual machine to resource poolResource.AssignVMToPool
Migrate powered off virtual machineResource.ColdMigrate
Migrate powered on virtual machineResource.HotMigrate

vApp Object

Grant these privileges at the resource pool level.

Privilege (UI)Privilege (API)
ImportVApp.Import
vApp application configurationVApp.ApplicationConfig

Virtual Machine Object

Configuration

Privilege (UI)Privilege (API)
Add existing diskVirtualMachine.Config.AddExistingDisk
Add new diskVirtualMachine.Config.AddNewDisk
Add or remove deviceVirtualMachine.Config.AddRemoveDevice
AdvancedVirtualMachine.Config.AdvancedConfig
Change CPU countVirtualMachine.Config.CPUCount
Change resourceVirtualMachine.Config.Resource
Configure managedByVirtualMachine.Config.ManagedBy
Disk change trackingVirtualMachine.Config.ChangeTracking
Disk leaseVirtualMachine.Config.DiskLease
Display connection settingsVirtualMachine.Config.MksControl
Extend virtual diskVirtualMachine.Config.DiskExtend
MemoryVirtualMachine.Config.Memory
Modify device settingsVirtualMachine.Config.EditDevice
Raw deviceVirtualMachine.Config.RawDevice
Reload from pathVirtualMachine.Config.ReloadFromPath
Remove diskVirtualMachine.Config.RemoveDisk
RenameVirtualMachine.Config.Rename
Reset guest informationVirtualMachine.Config.ResetGuestInfo
Set annotationVirtualMachine.Config.Annotation
SettingsVirtualMachine.Config.Settings
Swapfile placementVirtualMachine.Config.SwapPlacement
Unlock virtual machineVirtualMachine.Config.Unlock

Guest Operations

Privilege (UI)Privilege (API)
Guest Operation Program ExecutionVirtualMachine.GuestOperations.Execute
Guest Operation ModificationsVirtualMachine.GuestOperations.Modify
Guest Operation QueriesVirtualMachine.GuestOperations.Query

Interaction

Privilege (UI)Privilege (API)
Answer questionVirtualMachine.Interact.AnswerQuestion
Configure CD mediaVirtualMachine.Interact.SetCDMedia
Console interactionVirtualMachine.Interact.ConsoleInteract
Defragment all disksVirtualMachine.Interact.DefragmentAllDisks
Device connectionVirtualMachine.Interact.DeviceConnection
Guest operating system management by VIX APIVirtualMachine.Interact.GuestControl
Power offVirtualMachine.Interact.PowerOff
Power onVirtualMachine.Interact.PowerOn
ResetVirtualMachine.Interact.Reset
SuspendVirtualMachine.Interact.Suspend
VMware Tools installVirtualMachine.Interact.ToolsInstall

Inventory

Privilege (UI)Privilege (API)
Create from existingVirtualMachine.Inventory.CreateFromExisting
Create newVirtualMachine.Inventory.Create
MoveVirtualMachine.Inventory.Move
RegisterVirtualMachine.Inventory.Register
RemoveVirtualMachine.Inventory.Delete
UnregisterVirtualMachine.Inventory.Unregister

Provisioning

Privilege (UI)Privilege (API)
Allow disk accessVirtualMachine.Provisioning.DiskRandomAccess
Allow read-only disk accessVirtualMachine.Provisioning.DiskRandomRead
Allow virtual machine downloadVirtualMachine.Provisioning.GetVmFiles
Allow virtual machine files uploadVirtualMachine.Provisioning.PutVmFiles
Clone templateVirtualMachine.Provisioning.CloneTemplate
Clone virtual machineVirtualMachine.Provisioning.Clone
CustomizeVirtualMachine.Provisioning.Customize
Deploy templateVirtualMachine.Provisioning.DeployTemplate
Mark as templateVirtualMachine.Provisioning.MarkAsTemplate
Mark as virtual machineVirtualMachine.Provisioning.MarkAsVM
Modify customization specificationVirtualMachine.Provisioning.ModifyCustSpecs
Promote disksVirtualMachine.Provisioning.PromoteDisks
Read customization specificationsVirtualMachine.Provisioning.ReadCustSpecs

Snapshot Management

Privilege (UI)Privilege (API)
Create snapshotVirtualMachine.State.CreateSnapshot
Remove snapshotVirtualMachine.State.RemoveSnapshot
Rename snapshotVirtualMachine.State.RenameSnapshot
Revert snapshotVirtualMachine.State.RevertToSnapshot

Next Steps

To install PKS on vSphere, follow the procedures in Deploying Ops Manager to vSphere.


Please send any feedback you have to pks-feedback@pivotal.io.

Create a pull request or raise an issue on the source for this page in GitHub