Viewing Service Instance Logs

Pivotal Spring Cloud Data Flow provides access to the logs generated by each Data Flow server service instance, including logs for each of the two backing applications (Data Flow server application and Skipper application) for each instance. You can view these logs either using the Service Instance Logs cf CLI plugin or by visiting the dashboard of the Spring Cloud Data Flow service broker.

Using the cf CLI Plugin

After installing the Service Instance Logs cf CLI plugin (see the instructions in the Installing section of the plugin’s README), you can use the service-logs command to tail logs or dump recent logs for a service instance.

To tail logs for a Data Flow service instance, run cf service-logs SERVICE_NAME, where SERVICE_NAME is the name of the service instance:

$ cf service-logs data-flow

To dump recent logs for the instance, use the --recent flag:

$ cf service-logs --recent data-flow

If your Pivotal Cloud Foundry deployment uses a self-signed certificate, you must use the --skip-ssl-validation flag to disable the default validation of the platform’s SSL certificate:

$ cf service-logs --skip-ssl-validation data-flow

Using the Service Broker Dashboard

Note: To access the service broker dashboard, you must be a Space Developer in the broker application’s space (this is typically the system org and p-dataflow space).

Visit the Spring Cloud Data Flow service broker’s dashboard. You can access it at the following URL, where apps.example.com is the application domain of your PCF deployment:

https://p-dataflow.apps.example.com/

The dashboard shows the name, org, and space of each service instance, as well as a link to view logs for the instance.

Broker dashboard

Click the Logs link to view logs for a particular service instance’s backing application.

Recent logs

You can stream current logs for the instance by clicking the button.

Stream logs

Reading Aggregated Logs

The logs retrieved by the Service Instance Logs cf CLI plugin aggregate logs from two backing applications: a Spring Cloud Data Flow server application and a Spring Cloud Skipper application. The following excerpt shows logs after deploying a stream:

2019-03-18T14:19:20.04-0500 [APP/PROC/WEB/skipper 0] OUT 2019-03-18 19:19:20.048  INFO 22 --- [ry-client-nio-5] o.s.c.d.s.c.CloudFoundryAppDeployer      : Successfully computed status [deployed] for 2NkvRKK-this-is-a-stream-http-v1
2019-03-18T14:19:20.28-0500 [RTR/skipper 0] OUT skipper-18b33fb7-0e3e-4051-b5b0-bd02987e3049.apps.example.com - [2019-03-18T19:19:19.366+0000] "GET /api/release/status/this-is-a-stream HTTP/1.1" 200 0 2410 "-" "Apache-HttpClient/4.5.6 (Java/1.8.0_192)" "35.222.34.90:37146" "10.0.4.8:61002" x_forwarded_for:"35.222.34.90" x_forwarded_proto:"https" vcap_request_id:"ad32d730-cacb-49bd-465a-3675d1cf322c" response_time:0.917901116 app_id:"d62a5dc9-db5d-446c-845a-7226eaabf2f3" app_index:"0" x_b3_traceid:"12fef2178ba803ba" x_b3_spanid:"12fef2178ba803ba" x_b3_parentspanid:"-"

2019-03-18T14:19:20.26-0500 [APP/PROC/WEB/skipper 0] OUT 2019-03-18 19:19:20.268  INFO 22 --- [ry-client-nio-6] o.s.c.d.s.c.CloudFoundryAppDeployer      : Successfully computed status [deployed] for 2NkvRKK-this-is-a-stream-splitter-v1
2019-03-18T14:19:20.29-0500 [RTR/dataflow 0] OUT dataflow-18b33fb7-0e3e-4051-b5b0-bd02987e3049.apps.example.com - [2019-03-18T19:19:19.323+0000] "GET /runtime/streams?names=this-is-a-stream HTTP/1.1" 200 0 681 "https://dataflow-18b33fb7-0e3e-4051-b5b0-bd02987e3049.apps.example.com/dashboard/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" "97.85.165.2:41744" "10.0.4.22:61006" x_forwarded_for:"97.85.165.2" x_forwarded_proto:"https" vcap_request_id:"710909ca-eb8a-4dcb-74c8-4bfce93aa1bd" response_time:0.969019946 app_id:"8affbeaf-442b-4a42-a6f9-8e9c6d1abeaa" app_index:"0" x_b3_traceid:"4d98bfaa0267e110" x_b3_spanid:"4d98bfaa0267e110" x_b3_parentspanid:"-"

The Data Flow server application’s logs are identified as belonging to the dataflow application. The Spring Cloud Skipper application is called skipper.