Browse the Repo

file-type-icon.circleci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconchronograf-security-group-rules
file-type-iconchronograf-server
file-type-iconinfluxdb-cluster
file-type-iconREADME.md
file-type-iconmain.tf
file-type-iconoutputs.tf
file-type-iconvariables.tf
file-type-iconinfluxdb-commons
file-type-iconinfluxdb-iam-policies
file-type-iconinfluxdb-security-group-rules
file-type-iconinstall-chronograf
file-type-iconinstall-influxdb
file-type-iconinstall-kapacitor
file-type-iconinstall-telegraf
file-type-iconkapacitor-security-group-rules
file-type-iconkapacitor-server
file-type-iconload-balancer-target-group
file-type-iconload-balancer
file-type-iconrun-chronograf
file-type-iconrun-influxdb
file-type-iconrun-kapacitor
file-type-iconrun-telegraf
file-type-icontest
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconCONTRIBUTING.md
file-type-iconLICENSE
file-type-iconNOTICE
file-type-iconREADME.md
file-type-iconmain.tf
file-type-iconoutputs.tf
file-type-iconvariables.tf

Browse the Repo

file-type-icon.circleci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconchronograf-security-group-rules
file-type-iconchronograf-server
file-type-iconinfluxdb-cluster
file-type-iconREADME.md
file-type-iconmain.tf
file-type-iconoutputs.tf
file-type-iconvariables.tf
file-type-iconinfluxdb-commons
file-type-iconinfluxdb-iam-policies
file-type-iconinfluxdb-security-group-rules
file-type-iconinstall-chronograf
file-type-iconinstall-influxdb
file-type-iconinstall-kapacitor
file-type-iconinstall-telegraf
file-type-iconkapacitor-security-group-rules
file-type-iconkapacitor-server
file-type-iconload-balancer-target-group
file-type-iconload-balancer
file-type-iconrun-chronograf
file-type-iconrun-influxdb
file-type-iconrun-kapacitor
file-type-iconrun-telegraf
file-type-icontest
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconCONTRIBUTING.md
file-type-iconLICENSE
file-type-iconNOTICE
file-type-iconREADME.md
file-type-iconmain.tf
file-type-iconoutputs.tf
file-type-iconvariables.tf
InfluxDB

InfluxDB

Deploy an InfluxDB cluster. Supports automatic bootstrapping and auto healing.

Code Preview

Preview the Code

mobile file icon

README.md

down

InfluxDB Cluster

This folder contains a Terraform module to deploy an InfluxDB Enterprise cluster in AWS on top of an Auto Scaling Group. The idea is to create an Amazon Machine Image (AMI) that has InfluxDB meta and data binaries installed using the install-influxdb module.

How do you use this module?

This folder defines a Terraform module, which you can use in your code by adding a module configuration and setting its source parameter to URL of this folder:

module "influxdb_data_cluster" {
  # TODO: replace <VERSION> with the latest version from the releases page: https://github.com/gruntwork-io/terraform-aws-influx/releases
  source = "github.com/gruntwork-io/terraform-aws-influx//modules/influxdb-cluster?ref=<VERSION>"

  # Specify the ID of the InfluxDB AMI. You should build this using the scripts in the install-influxdb module.
  ami_id = "ami-abcd1234"
  
  # Configure and start InfluxDB during boot. 
  user_data = <<-EOF
              #!/bin/bash
              sudo systemctl start influxdb
              EOF
  
  # ... See variables.tf for the other parameters you must define for the influxdb-cluster module
}

Note the following parameters:

  • source: Use this parameter to specify the URL of the influxdb-cluster module. The double slash (//) is intentional and required. Terraform uses it to specify subfolders within a Git repo (see module sources). The ref parameter specifies a specific Git tag in this repo. That way, instead of using the latest version of this module from the master branch, which will change every time you run Terraform, you're using a fixed version of the repo.

  • ami_id: Use this parameter to specify the ID of an InfluxDB Amazon Machine Image (AMI) to deploy on each server in the cluster. You should install InfluxDB the scripts in the install-influxdb module.

  • user_data: Use this parameter to specify a User Data script that each server will run during boot. This is where you can use the run-influxdb script to configure and run InfluxDB as either a meta node or a data node.

You can find the other parameters in variables.tf.

Check out the examples folder for fully-working sample code.

Why InfluxDB enterprise?

Unlike the Enterprise edition which distributes the entire InfluxDB deployment across multiple meta and data nodes in a cluster, the OSS edition is a single binary that can simply be installed on a single instance. This makes a robust module like this one quite unnecessary for single instance InfluxDB OSS deployments.

How do you connect to the InfluxDB cluster?

While InfluxDB doesn't use a load balancer for intra-cluster communication, one can be optionally setup to communicate with a data cluster from the outside world. An Application Load Balancer is deployed in front of the cluster and you can use the InfluxDB CLI to connect to the load balancer on port 8086. Alternatively, you can use the InfluxDB SDK for your programming language of choice.

What's included in this module?

This module creates the following:

Auto Scaling Group

This module runs InfluxDB on top of an Auto Scaling Group (ASG). Typically, you should run the ASG with multiple Instances spread across multiple Availability Zones. Each of the EC2 Instances should be running an AMI that has InfluxDB installed via the install-influxdb module. You pass in the ID of the AMI to run using the ami_id input parameter.

The run-influxdb script ensures that a new node added by an ASG scale-up event automatically joins the existing cluster. However, you're responsible for de-registering any dead nodes (terminated EC2 instance) from the cluster when the ASG scales down. See the cluster commands guide for instructions.

EBS Volumes

This module can optionally create an EBS volume for each EC2 Instance in the ASG. You can use these volume to store InfluxDB data and are mandotory for InfluxDB data nodes.

We recommend a single EBS volume for the meta nodes for storing its Raft database. And one for data nodes, to store the /data, /meta, /wal, and /hh direcotries.

Backup and Replication

Setting up an adequate backup and recovery mechanism fo your cluster is hugely important. InfluxDB when used with some other components of the TICK stack has a very capable multi-datacenter and cross-cluster replication system. You can find more information of InfluxDB disaster recovery here.

Security Group

Each EC2 Instance in the ASG has a Security Group that allows minimal connectivity:

  • All outbound requests
  • Inbound SSH access from the CIDR blocks and security groups you specify

The Security Group ID is exported as an output variable which you can use with the influxdb-security-group-rules module to open up all the ports necessary for InfluxDB.

IAM Role and Permissions

Each EC2 Instance in the ASG has an IAM Role attached. The IAM Role ARN and ID are exported as output variables if you need to add additional permissions.

How do you roll out updates?

This module currently doesn't support updating the version of InfluxDB installed across the cluster. You can however follow this guide to manually perform the upgrade.

Dedicated instances

If you wish to use dedicated instances, you can set the tenancy parameter to "dedicated" in this module.

Encryption

This module does not currently support specifying encryption information. The official documentation contains a guide for enabling SSL.

Security groups

This module attaches a security group to each EC2 Instance that allows inbound requests as follows:

  • SSH: For the SSH port (default: 22), you can use the allowed_ssh_cidr_blocks parameter to control the list of
    CIDR blocks that will be allowed access. You can use the allowed_inbound_ssh_security_group_ids parameter to control the list of source Security Groups that will be allowed access.

The ID of the security group is exported as an output variable, which you can use with the influxdb-security-group-rules module to open up all the ports necessary for InfluxDB.

SSH access

You can associate an EC2 Key Pair with each of the EC2 Instances in this cluster by specifying the Key Pair's name in the ssh_key_name variable. If you don't want to associate a Key Pair with these servers, set ssh_key_name to an empty string.

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?