Getting Started With Graphite

Graphite is a highly scalable free open-source software tool that monitors, stores, and graphs numeric time-series data in real-time or on demand

Overview of Graphite

If you want to track values of anything that could potentially change over time, and you want to represent those values over time on a graph, then Graphite is just the right tool you need. Whether it’s a few time-series data or thousands of them from numerous sources, Graphite will meet your needs. Graphite is ideal for monitoring the performance of IT infrastructure, cloud, and on-prem business applications, the operating temperature of a machine, flight arrival, and departure statistics, stock price changes over time, road traffic fluctuations, number of customers who walk in and out of a retail store, and more.

A practical example of a real-life application of Graphite would be the case of a retail store that uses Graphite to determine the number of customers that walk out of a store due to long checkout queues. The retail store uses motion detectors to count people as they enter and exit the store, and uploads that count to the Graphite server once per minute. Another detector keeps track of how many people are waiting in the checkout line. Because the store manager knows how many people enter the store, leave the store, and are queued up at check-out, it can potentially extrapolate how many people walk out of the store because the queue at the check-out point is too long. This data would enable a store manager to know when to open an additional point of sale to prevent customers from leaving due to long queues.

System Architecture

Graphite consists of three main software components: carbon, whisper, and graphite webapp.

• Carbon A network service installed on the client machine that listens for inbound time-series metric submissions, data, or TCP/UDP packets sent over the network, and stores them in Graphite’s specialized database.
• Whisper A simple database that stores time-series data collected from machines and other sources.
• Graphite webapp A Django web app that renders graphs on demand. It enables data visualization through graphite’s web interfaces.

Graphite does not in itself collect data. Data collection is done via a vast array of tools such as Diamond, Bucky, Collectd, and others used to automate the process of the metric collection with minimal code changes. The collected metrics get fed into the stack via the Carbon service, which writes the data to the Whisper database for storage. Users interact with the Graphite web UI or API, which in turn queries Carbon and Whisper for the data needed to generate the requested graphs. Graphite’s web platform offers several ways, output styles, and formats to create and display graphs, including raw images, CSV, XML, and JSON, allowing anyone to easily embed custom graphs in other web pages or dashboards.

The ability to swiftly store and retrieve time-series data at high volume is key to the success of Graphite. Without the ability to scale, it would be impractical to use Graphite for anything besides small teams and hobby projects. The scalability of Graphite enables it to support the processing of millions of data points per second, making it a suitable visualization tool for virtually any scenario where time-series analysis is needed.

Installing and Configuring Graphite

Graphite is designed to run efficiently on almost any type of environment you need to run it in including cloud, on-premise, and hybrid environments. Several installation options exist for Graphite such as installing from Source, Pip, Virtualenv, and others depending on your skill and experience. These options can be quite complex, especially for a newbie. Thankfully, other methods simplify the installation process.

One such is the use of the open-source containerization platform known as Docker. You can use Docker to install the Graphite image to get it up and running in a matter of minutes. This makes for a fast and extremely portable installation. To install Graphite using Docker, simply run the following command:

docker run -d\

 --name graphite\

 --restart=always\

 -p 80:80\

 -p 2003-2004:2003-2004\

 -p 2023-2024:2023-2024\

 -p 8125:8125/udp\

 -p 8126:8126\

 graphiteapp/graphite-statsd

This command would generate a container named graphite and this container would have the following components:

• Graphite: Front-end dashboard
• Grafana: A more modern front-end dashboard
• Nginx: Reverse proxies the graphite dashboard
• Carbon: A daemon that listens for inbound time-series data
• Statsd: A simple daemon for easy stats aggregation

For detailed instructions, please refer to the official Graphite docker image repo on Github.

Another method that simplifies the installation process is by installing from Synthesize. Synthesize is a script that allows the installation and configuration of a range of components to automate the configuration of Graphite. Synthesize is built to run on Ubuntu 18.04 LTS only. It will not run on other Ubuntu releases or Linux distributions.  For other Linux distributions such as CentOS (8.2 and older), use the Resynthesize script—a fork of Synthesize designed to make Graphite easy to install in CentOS distributions.

Synthesize and Resynthesize scripts make it as easy as possible for newbies to get started with Graphite with a minimal learning curve. The resulting Graphite web interface according to the official documentation “listens only on https port 443 and has been configured to collect metrics specifically for helping profile the performance of your Graphite and Carbon services”. It comes with a Grafana dashboard—a modern alternative to Graphite’s built-in Composer, and a default dashboard for monitoring Carbon’s internal statistics. For detailed instructions, please refer to the official Synthesize and Resynthesize project documentation on Github.

Graphite does not directly run on Windows, but you can run it on Windows in Docker or installing via Synthesize. To leverage this capability, you will need to install Vagrant—a tool for building and managing virtual machine environments. If you run into any installation issues with Graphite, you can submit your issue to the Questions forum on Launchpad for assistance.

Getting Metrics into Graphite

Carbon is the software component of Graphite that listens for inbound time-series metric submissions, and sends them to the Graphite database. There are three main methods for sending data to Graphite: Plaintext, Pickle, and AMQP.

For a singular script, the plaintext protocol is the most straightforward method supported by Carbon. The data sent must be in the following format:

 <metric path> <metric value> <metric timestamp>

All that’s required is a dot-delimited metric name, a numeric value, and a metric timestamp. Using this format, you can issue a Unix command in your shell terminal to your application to send a data string to Graphite such as shown in the example below from Graphite documentation:

$ echo "foo.bar 41 `date +%s`" | nc graphite-server.example.com 2003

In the above example, we’re using the Unix echo command to print the metric string “foo.bar” along with the metric value and the metric timestamp generated by the date command. The output is piped into netcat (nc)—a network utility that connects to the Carbon service at graphite-server.example.com TCP port 2003 (by default, ‘plaintext’ runs on port 2003), and sends the data string. On Unix, the netcat (nc) program can be used to create a socket and send data to Carbon. Carbon will then translate this line of text into a metric which is then stored in a Whisper database on your Graphite server’s filesystem that the web interface understands.

The process described above is entirely manual. But in situations where you don’t want to manually feed metrics into Graphite, you can use several methods and tools such as Cron and Shell scripts, StatsD, and Collectd to automate the entire process. For example, you can use the shell script below from Graphite website, make it executable, and add it to your crontab to run at routine intervals:

#!/bin/bash

CARBON_HOST="10.10.10.5"

CARBON_PORT="2003"

METRIC_NAME="debug.jdixon.procs.bad_java_app"

TIMESTAMP=`date +%s`

NUMBER_OF_WAYWARD_PROCS=`ps auwx | grep MY_BAD_JAVA_APP | wc -l`

echo ${METRIC_NAME} ${NUMBER_OF_WAYWARD_PROCS} ${TIMESTAMP} | nc ${CARBON_HOST} ${CARBON_PORT}

When it comes to sending large amounts of data, the plaintext protocol fails woefully. For that, you’ll need a much more efficient protocol like Pickle which supports sending batches of metrics to Carbon in one go. Graphite also supports Advanced Message Queuing Protocol (AMQP), an open standard for passing business messages between applications or organizations. Through AMQP, Carbon can listen to a message bus.

Getting Metrics out of Graphite

There are several ways and tools you can use to create graphs from Graphite. One such is the use of the in-built Graphite Composer. The Graphite Composer provides the fastest means to familiarize yourself with Graphite’s visualization capabilities.

The main Composer window shown below shows a blank graph that acts as a canvas for your graph. On the left pane, you’ll find a navigation tree containing all of your metrics and saved graphs in a hierarchy mapping to their dot-delimited namespace. Clicking on any of the metrics allows you to add or remove it to the graph as shown in figure 3.0 below. The vertical axis tracks the values recorded, and the horizontal axis displays the time range.

Selecting the other memory metrics will add them to the graph too. This is a great way to correlate different metrics at the same time.

The Graphite render API is also a great way to retrieve graphs or metrics from Graphite. The graphite webapp provides a /render endpoint for generating graphs and retrieving raw data. It accepts various arguments via query string parameters which are separated by an ampersand (&). These parameters are supplied in the format: &name=value

To visualize specific metrics, you’d have to pass one or more target parameters (which specify a path identifying one or several metrics), and specify a time window for the graph via from/until. For details on the Graphite render API capabilities, please check out the official render API documentation as well as the huge library of Graphite’s transformative and statistical functions.

Similarly, the Grafana project offers more modern visualization for Graphite. But since it is not native, you will be required to integrate with Graphite. Thankfully, Grafana comes with a built-in plugin that can be directly used to connect and fetch data from graphite instances located on the same or different servers. It also offers an interactive sandbox that you can use to try out the tool. The Grafana documentation provides detailed information to help you get started.

Tools that Work with Graphite

The table below lists some examples of popular tools that can be used to extend the capabilities of Graphite:

Table 1.0 | List of some of the popular tools that work with Graphite