Browse the Repo

file-type-icon.circleci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconchronograf-security-group-rules
file-type-iconchronograf-server
file-type-iconinfluxdb-cluster
file-type-iconinfluxdb-commons
file-type-iconinfluxdb-iam-policies
file-type-iconinfluxdb-security-group-rules
file-type-iconinstall-chronograf
file-type-iconinstall-influxdb
file-type-iconinstall-kapacitor
file-type-iconinstall-telegraf
file-type-iconkapacitor-security-group-rules
file-type-iconkapacitor-server
file-type-iconload-balancer-target-group
file-type-iconload-balancer
file-type-iconrun-chronograf
file-type-iconrun-influxdb
file-type-iconREADME.md
file-type-iconrun-influxdb
file-type-iconrun-kapacitor
file-type-iconrun-telegraf
file-type-icontest
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconCONTRIBUTING.md
file-type-iconLICENSE
file-type-iconNOTICE
file-type-iconREADME.md
file-type-iconmain.tf
file-type-iconoutputs.tf
file-type-iconvariables.tf

Browse the Repo

file-type-icon.circleci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconchronograf-security-group-rules
file-type-iconchronograf-server
file-type-iconinfluxdb-cluster
file-type-iconinfluxdb-commons
file-type-iconinfluxdb-iam-policies
file-type-iconinfluxdb-security-group-rules
file-type-iconinstall-chronograf
file-type-iconinstall-influxdb
file-type-iconinstall-kapacitor
file-type-iconinstall-telegraf
file-type-iconkapacitor-security-group-rules
file-type-iconkapacitor-server
file-type-iconload-balancer-target-group
file-type-iconload-balancer
file-type-iconrun-chronograf
file-type-iconrun-influxdb
file-type-iconREADME.md
file-type-iconrun-influxdb
file-type-iconrun-kapacitor
file-type-iconrun-telegraf
file-type-icontest
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconCONTRIBUTING.md
file-type-iconLICENSE
file-type-iconNOTICE
file-type-iconREADME.md
file-type-iconmain.tf
file-type-iconoutputs.tf
file-type-iconvariables.tf
InfluxDB

InfluxDB

Deploy an InfluxDB cluster. Supports automatic bootstrapping and auto healing.

Code Preview

Preview the Code

mobile file icon

README.md

down

InfluxDB Run Script

This folder contains a script for configuring and initializing InfluxDB on an AWS server. This script has been tested on the following operating systems:

  • Ubuntu 18.04
  • Amazon Linux 2

There is a good chance it will work on other flavors of Debian, CentOS, and RHEL as well.

Quick start

This script assumes you installed it, plus all of its dependencies (including InfluxDB itself), using the install-influxdb module.

This will:

  1. Fill out the templated configuration file with user supplied values.

  2. Start InfluxDB on the local node.

  3. Wait for the Meta and Data ASGs to spin up all desired instances then update /etc/hosts with the IPs of all instances. the value of the instances' Name tag is used as the hostname entry.

  4. Figure out a rally point for your InfluxDB cluster. This is a "leader" Meta node that will be responsible for initializing the cluster. See Picking a rally point for more info.

  5. On the rally point, initialize the cluster, including adding all Meta and Data nodes to the cluster

We recommend using the run-influxdb command as part of User Data, so that it executes when the EC2 Instance is first booting.

See the examples folder for fully-working sample code.

Command line Arguments

Run run-influxdb --help to see all available arguments.

Usage: run-influxdb [options]

This script can be used to configure and initialize InfluxDB. This script has been tested with Ubuntu 18.04 and Amazon Linux 2.

Options:

  --hostname         The hostname of the current node.
  --node-type        Specifies whether the instance will be a Meta or Data node. Must be one of 'meta' or 'data'.
  --meta-asg-name    The name of the ASG that contains meta nodes.
  --data-asg-name    The name of the ASG that contains data nodes.
  --region           The AWS region the Auto Scaling Groups are deployed in.
  --auto-fill        Search the InfluxDB config file for KEY and replace it with VALUE. May be repeated.

Example:

  run-influxdb --node-type meta --meta-asg-name asg-meta --data-asg-name asg-data --region us-east-1 --auto-fill '<__LICENSE_KEY__>=******'

Picking a rally point

The Influx cluster needs a "rally point", which is a single Meta node that is responsible for:

  1. Initializing the cluster.
  2. Adding/removing nodes to the cluster.

We need a way to unambiguously and reliably select exactly one rally point. If there's more than one node, you may end up with multiple separate clusters instead of just one!

The run-influxdb script can automatically pick a rally point automatically by:

  1. Looking up all the servers in the Auto Scaling Group specified via the --cluster-name parameter.

  2. Pick the meta node with the oldest Launch Time as the rally point. If multiple nodes have identical launch times, use the one with the earliest Instance ID, alphabetically.

Passing credentials securely

The run-influxdb script requires that you pass in your license key and shared secret. You should make sure to never store these credentials in plaintext! You should use a secrets management tool to store the credentials in an encrypted format and only decrypt them, in memory, just before calling run-influxdb. Here are some tools to consider:

Moreover, if you're ever calling run-influxdb interactively (i.e., you're manually running CLI commands rather than executing a script), be careful of passing credentials directly on the command line, or they will be stored, in plaintext, in Bash history! You can either use a CLI tool to set the credentials as environment variables or you can temporarily disable Bash history.

Required permissions

The run-influxdb script assumes it is running on an EC2 Instance with an IAM Role that has the following permissions:

  • ec2:DescribeInstances
  • ec2:DescribeTags
  • autoscaling:DescribeAutoScalingGroups

These permissions are automatically added by the influxdb-cluster module.

Debugging tips and tricks

Some tips and tricks for debugging issues with your InfluxDB cluster:

  • Log file locations: https://docs.influxdata.com/enterprise_influxdb/v1.6/administration/logs/.
  • Use systemctl status influxdb to see if systemd thinks the InfluxDB process is running.

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?