Droplet Execution Agent
Page last updated:
Note: Pivotal Cloud Foundry (PCF) versions 1.6 and above no longer use Droplet Execution Agents (DEAs). Diego Cells replace their function. This topic is published here to help people who are running older versions of PCF.
A Droplet Execution Agent (DEA) performs the following key functions:
Manage Warden containers: The DEA stages applications and runs applications in Warden containers.
Stage applications: When a new application or a new version of an application is pushed to Cloud Foundry, the Cloud Controller selects a DEA from the pool of available DEAs to stage the application. The DEA uses the appropriate buildpack to stage the application. The result of this process is a droplet.
Run droplets: A DEA manages the lifecycle of each application instance running in it, starting and stopping droplets upon request of the Cloud Controller. The DEA monitors the state of a started application instance, and periodically broadcasts application state messages over NATS for consumption by the HM9000.
When the DEA receives requests for directories and files, it redirects them to the Directory Server URL. The URL is signed by the DEA, and the Directory Server checks the validity of the URL with the DEA before serving it.
Configurable Directory Server behaviors are controlled by keys in the DEA configuration file,
dea.yml, described below in DEA Configuration.
The Directory Server is written in Go and can be found in the
go/ directory of the DEA source code repository.
It is a replacement for the older directory server that was embedded in the DEA itself.
A DEA periodically checks the health of the applications running in it.
If a URL is mapped to an application, the DEA attempts to connect to the port assigned to the application. If the application port is accepting connections, the DEA considers that application state to be “Running.” If there is no URL mapped to the application, the DEA checks the system process table for the application process PID. If the PID exists, the DEA considers that application state to be “Running.”
The DEA also checks for a
AppState object for the application.
You can run the
dea executable at the command line by passing the path to the DEA configuration file:
$ bin/dea config/dea.yml
DEA behavior is configured in the
The following is a partial list of the keys that the DEA reads from the YAML file:
logging: A Steno configuration
nats_uri: A URI of the form
nats://host:portthat the DEA uses to connect to NATS
warden_socket: The path to a UNIX domain socket that the DEA uses to communicate to a Warden server.
dea.yml file follows:
Note: See https://github.com/cloudfoundry/dea_ng/blob/master/lib/dea/config.rb for optional configuration keys.
# See src/lib/dea/config.rb for optional config values. # Base directory for dea. Application directories, DEA temp files, etc. are all relative to this. base_dir: /tmp/dea_ng logging: level: debug resources: memory_mb: 2048 memory_overcommit_factor: 2 disk_mb: 2048 disk_overcommit_factor: 2 nats_uri: nats://localhost:4222/ pid_filename: /tmp/dea_ng.pid warden_socket: /tmp/warden.sock evacuation_delay_secs: 10 index: 0 staging: enabled: true platform_config: cache: /var/vcap/data/stager/package_cache/ruby environment: PATH: /usr/local/ruby/bin BUILDPACK_CACHE: /var/vcap/packages/buildpack_cache memory_limit_mb: 1024 disk_limit_mb: 2048 max_staging_duration: 900 # 15 minutes dea_ruby: /usr/bin/ruby # For Go-based directory server directory_server: v1_port: 4385 v2_port: 5678 file_api_port: 1234 streaming_timeout: 10 logging: level: info stacks: - lucid64 # Hook scripts for droplet start/stop # hooks: # before_start: path/to/script # after_start: path/to/script # before_stop: path/to/script # after_stop: path/to/script
When you contribute to the DEA, we recommend that you run it as a standalone component. Follow the instructions below for running the DEA as a standalone component on Vagrant 1.1x.
Refer to the Vagrant documentation for instructions on installing Vagrant.
$ git clone http://github.com/cloudfoundry/dea_ng $ bundle install # check that your version of vagrant is 1.1 or greater $ vagrant --version # create your test VM $ rake test_vm
Failure of the
rake test_vm step followed by an error similar to “undefined method ‘configure’ for Vagrant” or “found character that cannot start any token while scanning for the next token” can result from using an unsupported version of Vagrant. Ensure you have installed Vagrant version 1.1 or later.
# initialize the test VM $ vagrant up # shell into the VM $ vagrant ssh # start warden $ cd /warden/warden $ bundle install $ rvmsudo bundle exec rake warden:start[config/test_vm.yml] > /tmp/warden.log & # start the DEA's dependencies $ cd /vagrant $ bundle install $ git submodule update --init $ foreman start > /tmp/foreman.log & # run the dea tests $ bundle exec rspec
This section describes how the DEA architecture stages applications.
At the command line, the developer enters the directory containing her application and uses the Cloud Foundry Command Line Interface (cf CLI) tool to issue a push command.
The cf CLI tool tells the Cloud Controller to create a record for the application.
The Cloud Controller stores the application metadata (e.g. the app name, number of instances the user specified, and the buildpack).
The cf CLI tool uploads the application files.
The Cloud Controller stores the raw application files in the blobstore.
The cf CLI tool issues an app start command.
Because the app has not already been staged, the Cloud Controller chooses a DEA instance from the DEA pool to stage the application. The staging DEA uses the instructions in the buildpack to stage the application.
The staging DEA streams the output of the staging process so the developer can troubleshoot application staging problems.
The staging DEA packages the resulting staged application into a tarball called a “droplet” and stores it in the blobstore. The results are cached and used next time the application is staged.
The staging DEA reports to the Cloud Controller that staging is complete.
The Cloud Controller chooses one or more DEAs from the pool to run the staged application.
The running DEAs report the status of the application to the Cloud Controller.
A DEA publishes the following staging messages on NATS:
staging.advertise: Stagers broadcast their capacity/capability.
staging.locate: Stagers respond to any message on this subject with a
staging.advertisemessage. CC uses this to bootstrap.
staging.UUID.start: Stagers respond to requests on this subject to stage applications.
staging: Stagers, in a queue group, respond to requests to stage an application. Note: This is an old protocol.
For more information about how the Cloud Controller uses an algorithm to schedule apps on DEAs, see the DEA Placement Algorithm topic.
Steno handles the DEA logging.
DEA logs to STDOUT by default. You can configure the DEA to log to a file, a
syslog server, or both.
The logging level specifies the verbosity of the logs, for example