Diego Components and Architecture
Page last updated:
Warning: Pivotal Cloud Foundry (PCF) v2.3 is no longer supported because it has reached the End of General Support (EOGS) phase as defined by the Support Lifecycle Policy. To stay up to date with the latest software and security updates, upgrade to a supported version.
Page last updated:
This topic provides an overview of the structure and components of Diego, the container management system for Pivotal Cloud Foundry versions 1.6 and newer.
This topic includes the following sections:
- Architecture Diagram
- Diego Components
- Platform-Specific Components
Cloud Foundry uses the Diego architecture to manage app containers. Diego components assume app scheduling and management responsibility from the Cloud Controller.
See the following diagram and descriptions for information about the way Diego handles app requests.
The Cloud Controller passes requests to stage and run apps to several components on the Diego Brain.
The BBS tracks desired LRPs, running LRP instances, and in-flight Tasks. It also periodically analyzes this information and corrects discrepancies to ensure consistency between
Diego components run and monitor Tasks and LRPs.
Diego Brain components distribute Tasks and LRPs to Diego Cells, and correct discrepancies between
DesiredLRP counts to ensure fault-tolerance and long-term consistency.
The Diego Brain consists of the following:
Uses the auction package to run Diego Auctions for Tasks and LRPs
Communicates with Cell Reps over SSL/TLS
Maintains a lock in the BBS that restricts auctions to one Auctioneer at a time
See the Auctioneer repository on GitHub for more information.
Mediates uploads from the Executor to the Cloud Controller
Translates simple HTTP POST requests from the Executor into complex multipart-form uploads for the Cloud Controller
See the CC-Uploader repository on GitHub for more information.
- This “blobstore” serves static assets that can include general-purpose App Lifecycle binaries and app-specific droplets and build artifacts.
See the File Server repository on GitHub for more information.
- Brokers connections between SSH clients and SSH servers running inside instance containers
Provides the Cloud Controller with information about currently running LRPs to respond to
cf app APP_NAMErequests
ActualLRPactivity for crashes and reports them the Cloud Controller
See the TPS repository on GitHub for more information.
ActualLRPstates, emitting TCP route registration and unregistration messages to the Cloud Foundry routing API when it detects changes
Periodically emits TCP routes to the Cloud Foundry routing API
Listens for app requests to update the
DesiredLRPscount and updates
DesiredLRPsthrough the BBS
Periodically polls the Cloud Controller for each app to ensure that Diego maintains accurate
See the Nsync repository on GitHub for more information.
Translates staging requests from the Cloud Controller into generic Tasks and LRPs
Sends a response to the Cloud Controller when a Task completes
See the Stager repository on GitHub for more information.
Diego Cell components manage and maintain Tasks and LRPs.
The Diego Cell consists of the following:
Represents a Cell in Diego Auctions for Tasks and LRPs
Mediates all communication between the Cell and the BBS
Ensures synchronization between the set of Tasks and LRPs in the BBS with the containers present on the Cell
Maintains the presence of the Cell in the BBS
Runs Tasks and LRPs by asking the in-process Executor to create a container and
See the Rep repository on GitHub for more information.
Runs as a logical process inside the Rep
Implements the generic Executor actions detailed in the API documentation
STDERRto the Metron agent running on the Cell
See the Executor repository on GitHub for more information.
Provides a platform-independent server and clients to manage Garden containers
Defines the Garden-runC interface for container implementation
Forwards app logs, errors, and app and Diego metrics to the Loggregator Doppler component
See the Metron repository on GitHub for more information.
ActualLRPstates, emitting route registration and unregistration messages to the Cloud Foundry Gorouter when it detects changes
Periodically emits the entire routing table to the Cloud Foundry Gorouter
See the Route-Emitter repository on GitHub for more information.
The Diego database VM consists of the following components.
Maintains a real-time representation of the state of the Diego cluster, including all desired LRPs, running LRP instances, and in-flight Tasks
Ensure consistency and fault tolerance for Tasks and LRPs by comparing desired state (stored in the database) with actual state (from running instances)
Acts to keep
ActualLRPcount synchronized in the following ways:
- If the
DesiredLRPcount exceeds the
ActualLRPcount, requests a start auction from the Auctioneer
- If the
ActualLRPcount exceeds the
DesiredLRPcount, sends a stop message to the Rep on the Cell hosting an instance
- If the
Monitors for potentially missed messages, resending them if necessary
See the Bulletin Board System repository on GitHub for more information.
- Provides a consistent key-value data store to Diego
- Provides a consistent key-value store for maintenance of distributed locks and component presence
The Diego BBS stores data in MySQL. Diego uses the Go MySQL Driver to communicate with MySQL.
See the Go MySQL Driver repository on GitHub for more information.
Provides service discovery through colocated DNS servers on all BOSH-deployed VMs
Provides client-side load-balancing by randomly selecting a healthy VM when multiple VMs are available
See the BOSH DNS documentation for more information.
The following three platform-specific binaries deploy apps and govern their lifecycle:
The Builder, which stages a CF app. The Builder runs as a Task on every staging request. It performs static analysis on the app code and does any necessary pre-processing before the app is first run.
The Launcher, which runs a CF app. The Launcher is set as the Action on the
DesiredLRPfor the app. It executes the start command with the correct system context, including working directory and environment variables.
The Healthcheck, which performs a status check on running CF app from inside the container. The Healthcheck is set as the Monitor action on the
DesiredLRPfor the app.