Sunday, May 3, 2015

StatsD and Microservices Monitoring

StatsD is a network daemon for aggregating statistics, such as counters and timers, and shipping over UDP to backend services, such as Graphite or DatadogStatsD has many small clients libs for Java, Python, Ruby, Node, etc.  StatsD server collects stats from clients using a published wire protocol.  StatsD is the de facto standard. Although the Etsy StatsD Server is the reference implementation (the first implementation was written in Perl), there are other implementations like Go Stats Daemon, Data Dog and many moreStatsD captures different metrics, Gauges, Counters, Timing Summary Statistics, and Sets. You decorate your code to capture this type of data and report it. Although StatsD collects runtime statistics data over time and does periodic “flushes” of the data to analysis and monitoring engines you choose, StatsD was originally written with Graphite in mind. Graphite is used to visualize the state of microservices. Graphite is made up of Graphite-Web (graph and dashboard rendering), Carbon (metric processing daemons), and Whisper (time-series database) library.

StatsD seems to be the current champion of mind space. Mainly due to its simplicity and fire-and-forget protocol. StatsD can’t cause as cascading failure, and its client libs are very small. There are other alternatives that QBit can integrate with as well like Coda Hale’s Metrics library which uses a Go Daemon.

StatsD can also dump its feed to Kibana or Banana via a Logstash plugin. You can use Kibana and Banana in place of Graphite. There is even commercial support of StatsD via DataDog which allows monitoring, graphing, alerting, and event correlation. DataDog embedded the StatsD daemon within the Datadog Agent so it is a drop in replacement for StatsD. Datadog is a monitoring service for IT, Operations, Development and DevOps. It attempts to take input from many vendors, cloud providers, open source tools, servers, and aggregate their data into reactive actionable metrics.

Read more at Microservices Monitoring.
Kafka and Cassandra support, training for AWS EC2 Cassandra 3.0 Training