Browse the Repo
Browse the Repo
Deploy a self-hosted Elasticsearch cluster. Supports automatic bootstrap, zero-downtime rolling deployment, auto healing, backup, and recovery.
This script assumes that you already have Filebeat installed. If you don't, we recommend using the install-filebeat module. As part of the installation process, we recommend that you create a Filebeat YAML config file and install it using the
--config-file option of the
You may want some of the configs, such as the IPs of the Logstash/Elasticsearch servers, to be filled in dynamically, when the server is booting up. You can do this using the
run-filebeat script! Simply leave placeholders in your Filebeat config file like this (see the filebeat-ami folder for a full example):
filebeat.prospectors: - type: log enabled: true paths: - <__PATH__> # /var/log/source.log #----------------------------- File output -------------------------------- # This is purely for testing purposes, real world usage will most likely send logs to Logstash or Elasticsearch output.file: path: "/var/log" filename: destination.log
Now you can fill in those placeholders and start Filebeat by executing the
run-filebeat script as follows:
run-filebeat --auto-fill "<__PATH__>=/path/to/source.log"
<__PATH__>in the Filebeat config file with the path to the log file that Filebeat will read from
We recommend using the
run-filebeat command as part of User Data, so that it executes when the EC2 Instance is first booting.
See the examples folder for fully working sample code.
run-filebeat --help to see all available arguments.
Usage: run-filebeat [options] This script can be used to configure and run Filebeat. This script has been tested with Ubuntu 16.04 + 18.04, CentOS 7 and Amazon Linux 2. Options: --config-file The path to a YAML config file for Filebeat. Default: /etc/filebeat/filebeat.yml. --skip-auto-discovery Specifies whether this script should skip checking for Logstash nodes. Default: false. --auto-fill KEY=VALUE Search the Filebeat config file for KEY and replace it with VALUE. May be repeated. --help Show this help text and exit. Required arguments only if auto discovery is enabled --tag The tag key and value on the instances. --port The port on which the Logstash nodes listen for beats connections. Default: 5044 --aws-region The AWS region where the ASG is deployed. Default: us-east-1 --use-public-ips Sets whether to get the public or private IPs from the ASG. Default: true --schedule A cron expression specifying the schedule this script will run with. Default: */5 * * * * Example: run-filebeat --auto-fill '<__PATH__>=/var/log/*.log'
run-filebeat script automatically discovers Logstash nodes by using the auto-discovery module. You can disable this feature by setting the
true which is useful if you have your own in-house auto discovery mechanism.
We're here to talk about our services, answer any questions, give advice, or just to chat.