Browse the Repo

file-type-icon.circleci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-icontest
file-type-icon.gitignore
file-type-iconCODEOWNERS
file-type-iconCONTRIBUTING.md
file-type-iconLICENSE.md
file-type-iconREADME.md

Browse the Repo

file-type-icon.circleci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-icontest
file-type-icon.gitignore
file-type-iconCODEOWNERS
file-type-iconCONTRIBUTING.md
file-type-iconLICENSE.md
file-type-iconREADME.md
Elasticsearch (self-hosted)

Elasticsearch (self-hosted)

Deploy a self-hosted Elasticsearch cluster. Supports automatic bootstrap, zero-downtime rolling deployment, auto healing, backup, and recovery.

Preview the Code

mobile file icon

README.md

down

Terraform Version

ELK AWS Module

This repo contains modules for deploying and managing the ELK Stack.

Elasticsearch architecture

Quick start

If you want to quickly spin up the entire ELK stack installed on a single machine, you can run the single server example. This is a simple example that shows you how all the components in this module work together. If you want a more production-like deployment with separate clusters for each component of ELK, look at ELK Deployment example.

What's in this repo

This repo has the following folder structure:

  • modules: This folder contains the main implementation code for this Module, broken down into multiple standalone submodules.
  • examples: This folder contains examples of how to use the submodules.
  • test: Automated tests for the submodules and examples.

The main modules are:

    • install-kibana: Install Kibana, a web-based data visualizer for Elasticsearch.

    • run-kibana: Start Kibana.

    • kibana-cluster: A Terraform module to run a cluster of Kibana nodes with ENIs and EBS Volumes attached, zero-downtime deployment, and auto-recovery of failed nodes.

    • install-logstash: Install Logstash, a server based data collection and processing engine.

    • run-logstash: Start Logstash.

    • logstash-cluster: A Terraform module to run a cluster of Logstash nodes with ENIs and EBS Volumes attached, zero-downtime deployment, and auto-recovery of failed nodes.

The supporting modules are:

  • elasticsearch-cluster-backup: A Terraform module to deploy a Lambda function that takes snapshots of the Elasticsearch cluster on a configurable schedule and stores those snapshots in S3.

  • elasticsearch-cluster-restore: A Terraform module to deploy a Lambda function that restores a cluster from saved snapshots.

Click on each module above to see its documentation.

What's a Module?

A Module is a canonical, reusable, best-practices definition for how to run a single piece of infrastructure, such as a database or server cluster. Each Module is written using a combination of Terraform and scripts (mostly bash) and include automated tests, documentation, and examples. It is maintained both by the open source community and companies that provide commercial support.

Instead of figuring out the details of how to run a piece of infrastructure from scratch, you can reuse existing code that has been proven in production. And instead of maintaining all that infrastructure code yourself, you can leverage the work of the Module community to pick up infrastructure improvements through a version number bump.

Who maintains this Module?

This Module is maintained by Gruntwork. If you're looking for help or commercial support, send an email to modules@gruntwork.io. Gruntwork can help with:

  • Setup, customization, and support for this Module.
  • Modules for other types of infrastructure, such as VPCs, Docker clusters, databases, and continuous integration.
  • Modules that meet compliance requirements, such as HIPAA.
  • Consulting & Training on AWS, Terraform, and DevOps.

How do I contribute to this Module?

Contributions are very welcome! Check out the Contribution Guidelines for instructions.

How is this Module versioned?

This Module follows the principles of Semantic Versioning. You can find each new release, along with the changelog, in the Releases Page.

During initial development, the major version will be 0 (e.g., 0.x.y), which indicates the code does not yet have a stable API. Once we hit 1.0.0, we will make every effort to maintain a backwards compatible API and use the MAJOR, MINOR, and PATCH versions on each release to indicate any incompatibilities.

License

Please see LICENSE for how the code in this repo is licensed.

Copyright © 2018 Gruntwork, Inc.

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?