Browse the Repo

file-type-icon.circleci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconauto-discovery
file-type-iconbeats-iam-policies
file-type-iconelastalert-iam-policies
file-type-iconelastalert-security-group-rules
file-type-iconelastalert
file-type-iconelasticsearch-cluster-backup
file-type-iconelasticsearch-cluster-restore
file-type-iconelasticsearch-cluster
file-type-iconelasticsearch-iam-policies
file-type-iconelasticsearch-security-group-rules
file-type-iconinstall-collectd
file-type-iconinstall-elastalert
file-type-iconinstall-elasticsearch
file-type-iconinstall-filebeat
file-type-iconinstall-kibana
file-type-iconinstall-logstash
file-type-iconkibana-cluster
file-type-iconkibana-security-group-rules
file-type-iconload-balancer-alb-target-group
file-type-iconlogstash-cluster
file-type-iconlogstash-iam-policies
file-type-iconlogstash-security-group-rules
file-type-iconrun-collectd
file-type-iconrun-elastalert
file-type-iconrun-elasticsearch
file-type-iconrun-filebeat
file-type-iconrun-kibana
file-type-iconrun-logstash
file-type-iconbin
file-type-iconREADME.md
file-type-iconinstall.sh
file-type-icontest
file-type-icon.gitignore
file-type-iconCODEOWNERS
file-type-iconCONTRIBUTING.md
file-type-iconLICENSE.md
file-type-iconREADME.md

Browse the Repo

file-type-icon.circleci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconauto-discovery
file-type-iconbeats-iam-policies
file-type-iconelastalert-iam-policies
file-type-iconelastalert-security-group-rules
file-type-iconelastalert
file-type-iconelasticsearch-cluster-backup
file-type-iconelasticsearch-cluster-restore
file-type-iconelasticsearch-cluster
file-type-iconelasticsearch-iam-policies
file-type-iconelasticsearch-security-group-rules
file-type-iconinstall-collectd
file-type-iconinstall-elastalert
file-type-iconinstall-elasticsearch
file-type-iconinstall-filebeat
file-type-iconinstall-kibana
file-type-iconinstall-logstash
file-type-iconkibana-cluster
file-type-iconkibana-security-group-rules
file-type-iconload-balancer-alb-target-group
file-type-iconlogstash-cluster
file-type-iconlogstash-iam-policies
file-type-iconlogstash-security-group-rules
file-type-iconrun-collectd
file-type-iconrun-elastalert
file-type-iconrun-elasticsearch
file-type-iconrun-filebeat
file-type-iconrun-kibana
file-type-iconrun-logstash
file-type-iconbin
file-type-iconREADME.md
file-type-iconinstall.sh
file-type-icontest
file-type-icon.gitignore
file-type-iconCODEOWNERS
file-type-iconCONTRIBUTING.md
file-type-iconLICENSE.md
file-type-iconREADME.md
Elasticsearch (self-hosted)

Elasticsearch (self-hosted)

Deploy a self-hosted Elasticsearch cluster. Supports automatic bootstrap, zero-downtime rolling deployment, auto healing, backup, and recovery.

Code Preview

Preview the Code

mobile file icon

README.md

down

Run Logstash Script

This folder contains a script for configuring and running Logstash on an AWS EC2 instance. This script has been tested on the following operating systems:

  • Ubuntu 16.04
  • Ubuntu 18.04
  • CentOS 7
  • Amazon Linux 2

Quick start

This module depends on bash-commons, so you must install that project first as documented in its README.

The easiest way to use this module is with the Gruntwork Installer:

gruntwork-install \
  --module-name "run-logstash" \
  --repo "https://github.com/gruntwork-io/package-elk" \
  --tag "<VERSION>"

Checkout the releases to find the latest version.

We recommend using the run-logstash command as part of User Data, so that it executes when the EC2 Instance is first booting.

See the examples folder for fully working sample code.

Command line Arguments

Run run-logstash --help to see all available arguments.

Usage: run-logstash [options]

This script can be used to configure and run Logstash. This script has been tested with Ubuntu 16.04 + 18.04, CentOS 7 and Amazon Linux 2.

Options:

  --config-file                         The path to a YAML config file for Logstash. Default: /etc/logstash/logstash.yml.
  --auto-fill KEY=VALUE                 Search the Logstash config file for KEY and replace it with VALUE. May be repeated.
  --auto-fill-jvm KEY=VALUE   Search the Logstash JVM config file for KEY and replace it with VALUE. May be repeated.
  --auto-fill-pipeline KEY=VALUE        Search the Logstash pipeline config file for KEY and replace it with VALUE. May be repeated."
  --help                                Show this help text and exit.

Example:

  run-logstash --auto-fill '<__PATH__>=/var/lib/logstash' --auto-fill-pipeline '<__PORT__>=5044' --auto-fill-jvm '<__XMS__>=4g'

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?