Set up Telegraf, InfluxDB and Grafana with Docker Compose

15263 VIEWS

5 months ago

In this tutorial, we have three Docker containers for Telegraf, InfluxDB, and Grafana. We will write a `docker-compose.yml` file and add all three containers as services.

Docker Compose is a powerful tool for building and running multiple containers on the Docker Engine. With Docker Compose, you will write a YML file and configure the Docker containers services. For example, you may have a Web application that requires a React.js container for the front end, Node.js for the backend, and MongoDB for the database. If you use Docker compose to create this web application, you will write one YML file that would start all three containers as a service.

We’ll also add Docker images for creating the three Docker containers (Telegraf, InfluxDB, and Grafana). Docker Compose will pull the specified Docker images from Docker Hub and build the Docker containers.

The Docker Compose YML file will also have all the configurations for the three containers (Telegraf, InfluxDB, and Grafana). It will have the port mapping, the persistent volumes for our application, and the container names. Docker Compose speeds up building multiple Docker containers.

Telegraf, InfluxDB and Grafana (TIG) Stack

These three platforms (Telegraf, InfluxDB, and Grafana) provide a DevOps continuous monitoring solution. Each of these platforms plays a different role in tracking and measuring the performance of machines and DevOps systems.

Telegraf

Telegraf is an open-source agent written in the Go programming language. It collects, manages, processes, and aggregates metrics of a DevOps system or machine. It then sends the aggregated metrics data to various storage systems such as Apache Kafka, Graphite, and InfluxDB. The most common metrics that Telegraf collects from machines are CPU utilization, memory (RAM) usage, disk (ROM) usage, and computer processors.

InfluxDB

InfluxDB is an open-source time series database. It is written in the Go programming language. It gets the metrics data that agents such as Telegraf collect and stores it as time series data.

Grafana

Grafana is an open-source DevOps continuous monitoring and interactive data visualization platform. It allows developers to create interactive graphs, charts, and dashboards using a connected data source. There are various data sources that Grafana can use, like Prometheus and InfluxDB. We will use InfluxDB as the data source. Grafana will use the metrics data from InfluxDB and create a visualization dashboard.

In this tutorial, we will cover the following:

How to Set up Telegraf, InfluxDB, and Grafana using Docker Compose.
Configure Telegraf using telegraf.conf file. Telegraf will send the metrics data (CPU utilization, memory (RAM) usage, ROM usage, and computer processor utilization) to InfluxDB.
Configure InfluxDB as a Data Source in the Grafana Platform.
Create a Visualization Dashboard in Grafana using the InfluxDB data source.

How to Set up Telegraf, InfluxDB, and Grafana using Docker Compose

We will start by creating a `docker-compose.yml`. In your computer, create a folder named `docker-compose-application`. Open the folder in your code editor and create a `docker-compose.yml` file. Let’s start configuring the container services.

Configuring the Telegraf Service

You will configure Telegraf Service using the following code:

version: '3.6'
services:
  telegraf:
    image: telegraf
    container_name: telegraf
    restart: always
    volumes:
    - ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
    depends_on:
      - influxdb
    links:
      - influxdb
    ports:
    - '8125:8125'

The YML code will create a service named `telegraf`:

This service will use the `telegraf` Docker image to build the Telegraf Docker container.
It will assign `telegraf` as the container name. This `telegraf` service depends on the `influxdb` service.
The YML code will also create a volume and map the `telegraf.conf` file to the Telegraf container. The container port mapping is `8125:8125`.

Configuring the InfluxDB Service

Add the following YAML code to configure the InfluxDB Service:

influxdb:
    image: influxdb:1.8-alpine
    container_name: influxdb
    restart: always
    environment:
      - INFLUXDB_DB=influx
      - INFLUXDB_ADMIN_USER=admin
      - INFLUXDB_ADMIN_PASSWORD=admin
    ports:
      - '8086:8086'
    volumes:
      - influxdb_data:/var/lib/influxdb

The YML code will create a service named `influxdb`:

This service will use the `influxdb:1.8-alpine` Docker image to build the InfluxDB Docker Container.
It will assign `influxdb` as the container name.
We have set the InfluxDB admin user as `admin` and the admin password as `admin`. We use these credentials to login into InfluxDb.
We have also created an `influxdb_data` persistent volume. The container port mapping is `8086:8086`.

Configuring the Grafana Service

Add the following YAML code to configure the InfluxDB Service:

grafana:
   image: grafana/grafana
   container_name: grafana-server
   restart: always
   depends_on:
     - influxdb
   environment:
     - GF_SECURITY_ADMIN_USER=admin
     - GF_SECURITY_ADMIN_PASSWORD=admin
     - GF_INSTALL_PLUGINS=
   links:
     - influxdb
   ports:
     - '3000:3000'
   volumes:
     - grafana_data:/var/lib/grafana

The YML code will create a service named `grafana`:

This service will use the `grafana/grafana` Docker image to build the Telegraf Docker Container.
It will assign `grafana-server` as the container name. This `grafana` service depends on the `influxdb` service.
We have set the Grafana admin user as `admin` and the admin password as `admin`. We use these credentials to login into the Grafana platform.
We have also created a `grafana_data` persistent volume. The container port mapping is `3000:3000`.

Your final `docker-compose.yml` file after configuring all the container services should look like this:

version: '3.6'
services:
  telegraf:
    image: telegraf
    container_name: telegraf
    restart: always
    volumes:
    - ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
    depends_on:
      - influxdb
    links:
      - influxdb
    ports:
    - '8125:8125'

  influxdb:
    image: influxdb:1.8-alpine
    container_name: influxdb
    restart: always
    environment:
      - INFLUXDB_DB=influx
      - INFLUXDB_ADMIN_USER=admin
      - INFLUXDB_ADMIN_PASSWORD=admin
    ports:
      - '8086:8086'
    volumes:
      - influxdb_data:/var/lib/influxdb

  grafana:
    image: grafana/grafana
    container_name: grafana-server
    restart: always
    depends_on:
      - influxdb
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=admin
      - GF_INSTALL_PLUGINS=
    links:
      - influxdb
    ports:
      - '3000:3000'
    volumes:
      - grafana_data:/var/lib/grafana
      
volumes:
  grafana_data: {}
  influxdb_data: {}

Before we build the Docker image using Docker Compose, let’s create a `telegraf.conf` file to configure Telegraf.

Configure Telegraf using telegraf.conf File

In the `docker-compose-application` folder, create a `telegraf.conf` file. Open the file and add the following configurations:

[global_tags]

[agent]
  interval = "60s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  hostname = "192.xxx.0.xxx"
  omit_hostname = false

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  database = "influx"
  timeout = "5s"
  username = "telegraf"
  password = "metricsmetricsmetricsmetrics"


[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false


[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]

[[inputs.mem]]

[[inputs.processes]]

The telegraf.conf file will configure Telegraf to send the metrics data to InfluxDB. InfluxDb will be running on “http://influxdb:8086”. It will use `username = “telegraf”` and `password = “metricsmetricsmetricsmetrics”` to authenticate Telegraf (telegraf service) requests to InfluxDB (influxdb service). The Telegraf agent will collect CPU usage, memory (RAM) usage, disk usage, and computer processor utilization from the host machine.

To build three Docker containers using Docker Compose, run this `docker compose` command:

docker compose up

The command will build all three Docker containers as shown below:

To check if containers are running, run this docker command.

docker ps

It outputs our three Docker containers as shown below:

Our three containers are running, let’s configure InfluxDB as a Data Source in the Grafana Platform.

Configure InfluxDB as a Data Source in the Grafana Platform

You will access the Grafana Docker Container and configure InfluxDB as a data source. Type localhost:3000 in your web browser to access Grafana. It will launch a Grafana `Welcome to Grafana` login page:

You will enter the admin user as `admin` and the admin password as `admin` to login into Grafana. After login, it will launch a `Welcome to Grafana` home page:

To configure InfluxDB as a Data Source, click `Data Sources`:

It will open another page. You will then click InfluxDB:

In the InfluxDB data source page, you will add the InfluxDB URL as http://influxdb:8086:

You will also InfluxDB database name as `influx`, the admin user as `admin`, and the admin password as `admin`:

Grafana will use these details to configure InfluxDB as the data source. You then click the `Save & Test` button. Grafana will validate all the configuration details provided and test the InfluxDB data source:

From the image above, the data source is working. Grafana has also found four metrics. These metrics are the ones we configured using the telegraf.conf file. Let’s use the InfluxDB data source to create a visualization dashboard.

Create a Visualization Dashboard in the Grafana Platform using the InfluxDB Data Source

To create a visualization dashboard, click `DASHBOARDS` on the `Welcome to Grafana` home page:

You will then click `Add a new panel`:

After clicking `Add a new panel`, it will generate a new Dashboard using the InfluxDB data source:

From the image above, we have successfully created a visualization dashboard in Grafana using the InfluxDB data source.

Conclusion

In this tutorial, you have learned how to set up Telegraf, InfluxDB, and Grafana using Docker Compose. We started by creating a `docker-compose.yml` and added all three containers as services. Then we added Docker images to the YAML file for creating the three Docker containers (Telegraf, InfluxDB, and Grafana). We then configured Telegraf using the telegraf.conf file.

Telegraf can send the metrics data (CPU utilization, memory (RAM) usage, ROM usage, and computer processor utilization) to InfluxDB. We configured InfluxDB as a Data Source in the Grafana Platform. Finally, we created a visualization dashboard in Grafana using the InfluxDB data source.