Post on 16-Aug-2015
TECHNOLOGY SOLUTIONS DELIVERED
TECHNOLOGY SOLUTIONS DELIVERED
Cloud Foundry Logging & Metrics“Understanding the new Architecture and integrating with ELK”
TECHNOLOGY SOLUTIONS DELIVERED
Agenda
• The new architecture• How To: Ship logs to ELK• Demo: Subscribing to the Firehose• Demo: Firehose-to-syslog• Demo: Sending Metrics to Doppler• Things to consider…• Q & A
TECHNOLOGY SOLUTIONS DELIVERED
The new architecture – 5 Basic Components
Sources
1
Cloud Foundry components and
apps that emit logs & metrics
Metron
2
Agents running on Cloud Foundry
VM’s that collect emitted
logs/metrics and forward them on to
Doppler
Doppler
3
Server(s) that collect (aggregate) logs & metric data from the Metron
agents
Traffic Controller
4
Handle subscription requests and stream logs/metrics to clients
Sinks
5
Clients – subscribe to receive
logs/metrics from Doppler.
Such as =>syslog drains, cf cli, …
TECHNOLOGY SOLUTIONS DELIVERED
Dropsonde EventsEvent Description
HttpStart Emitted when a client sends a request (or immediately when a server receives a request)
HttpStop Emitted when a client receives a response to its request (or when a server completes its handling and returns a response).
HttpStartStop An event representing the entire lifecycle of an http request
LogMessage An event containing a log entry (this is the normal log data available from cf cmd line or via the syslog daemon)
ValueMetric An event representing the value of a metric at a particular point in time
CounterEvent Represents the increment of a counter. It contains only the change in the value; it is the responsibility of downstream consumers to maintain the value of the counter.
Error Represents an error in the process that originated the event.
ContainerMetric An event representing resource usage (cpu/mem/disk) of an app in a container
TECHNOLOGY SOLUTIONS DELIVERED
How to send CF logs to ELK• Update your deployment manifest or find the appropriate template for your
release/deployment.• You need to populate the following in one of your templates or in your manifest.
properties: syslog_daemon_config: address: <<ip of your ELK Ingestor>> port: <<correct port of chosen transport. Usually 5514 for syslog 5515 for relp transport: <<syslog or relp>>
• You could put this in cf-properties.yml or create or use cf-syslog.yml
• Using bosh…target your deployment then perform a bosh deploy.• Open Kibana and see the wonder of CF logs steaming into your ELK
environment!
TECHNOLOGY SOLUTIONS DELIVERED
Demos
• Use NOAA client to subscribe for logs/metrics.
• Use Firehose-to-syslog to get metrics data from the Firehose into your ELK environment.
• Emit some new metrics and see them flow through to ELK.
TECHNOLOGY SOLUTIONS DELIVERED
Helpful links…• Main Loggregator Repository
• https://github.com/cloudfoundry/loggregator
• Main NOAA Client (GO) Repository• https://github.com/cloudfoundry/noaa
• Dropsonde• https://github.com/cloudfoundry/dropsonde• https://github.com/cloudfoundry/sonde-go
• Firehose-to-Syslog• https://github.com/cloudfoundry-community/firehose-to-syslog
• Cloud Credo – Great background and how to(s) for Firehose and ELK/Graphite integration• http://www.cloudcredo.com/cloud-foundry-logging-and-monitoring/• http://www.cloudcredo.com/the-basics/• http://www.cloudcredo.com/cloud-foundry-firehose-and-friends/• http://www.cloudcredo.com/how-to-integrate-elasticsearch-logstash-and-kibana-elk-with-cloud-foundry/• http://cloudcredo.com/how-to-integrate-graphite-with-cloud-foundry/
• ELK – bosh release(s)/deployments• https://github.com/cloudfoundry-community/logsearch-boshworkspace• https://github.com/logsearch/logsearch-boshrelease
• Stark & Wayne • https://blog.starkandwayne.com/2014/11/19/running-logsearch-on-bosh-lite-in-10-minutes/
TECHNOLOGY SOLUTIONS DELIVERED
About Us
Founded in 1999
Over 130 Consultants
HQ in Greenwood Village, CO
Delivering Technology Solutions
Solving Strategic Business
Challenges
TECHNOLOGY SOLUTIONS DELIVERED
Our Solutions
Cloud-NativeDevelopment
Collaboration & Portals
Custom Software Development
Enterprise Architecture
Enterprise Mobility Integration & API Management
Interactive Design UI/UX
Project Management & Governance
TECHNOLOGY SOLUTIONS DELIVERED
Stage Focus
Project Management & Governance
Ensure long-term project success through governance.
Discover Validate business challenges and develop a plan to address these challenges with a new solution.
Design Map the business challenges defined in discover to a technology solution.
Build Convert the design into a working, deployable solution. At the end of build, the solution is ready for the organization to use.
Launch Deploy the developed and tested solution.
Adopt Achieve client success and realize solution business value.
ECS Team ensures project success through its process of discover, design, build, launch and adopt,while offering project management & governance at the same time.
ECS Team Solution Framework