This folder contains a script for configuring and running ElastAlert on an AWS server. This script has been tested on the following operating systems:
Ubuntu 16.04
Ubuntu 18.04
Amazon Linux 2
CentOS 7
There is a good chance it will work on other flavors of Debian, CentOS, and RHEL as well.
What is ElastAlert?
ElastAlert is a simple framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch. Note that it's not an Elasticsearch plugin, just a Python service that connects to the ElasticSearch API. However, there is a separate ElastAlert Kibana Plugin that exposes ElastAlert functionality in Kibana.
The key idea is that you define an Elasticsearch query as part of a rule, which is of some rule type that specifies when that query should trigger an alert, which is of some alert type.
For example, you might use the frequency rule type ("Match where there are at least X events in Y time") to define a rule that triggers an alert of type Slack so that a Slack message appears when the rule is triggered.
Quick start
This script depends on bash-commons, so you must install that project
first as documented in its README.
Review the config.yaml file in /opt/elastalert/config/config.yaml and make sure the rules_folder property specifies the folder that contains all the ElastAlert rule definitions.
Review the example rules. As a quick start, use the sns alert type by making sure the example rule contains the following and no other alert definitions:
This will begin querying data in Elasticsearch starting from right this moment.
Finally, send some data into the Elasticsearch cluster and confirm that you've received an email (TODO: How best to send data to cluster?)
Command line Arguments
Run run-elastalsert --help to see all available arguments, or just check out the run-elastalert source code.
Alternatives
Elastalert vs. X-Pack
X-Pack is a full-featured, commercial plugin maintained by
elastic.co that includes the following features:
Security. Lock down elasticsearch nodes to certain IPs or require authentication via LDAP, Active Directory, or
basic authentication.
Monitoring. Monitor the components of the Elasticsearch cluster: Elasticsearch, Logstash, and Kibana.
Alerting and Notifications. Define a query and a condition under which its results should trigger a notification.
Reports. You can download Kibana reports as PDFs optimized for printing.
Query Relationships Between Data. Explore how different attributes of a table connecto to each other with visual graph renderings.
Machine Learning. Perform "Time Series Anomaly Detection" to automatically alert on unusual changes.
By contrast, Elastalert is limited to alerting and notifications. We chose Elastalert over X-Package because, as an alerting plugin, Elastalert compares favorably to X-Pack, and it's open source. However, if you require other features of X-Pack, you may prefer to use X-Pack for alerting and notifications as well.
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"config.yml","path":".circleci/config.yml","sha":"9fec455dfc15ca063a578654ad29591ffd2252e1"}]},{"name":".gitignore","path":".gitignore","sha":"437200d1e2f9ac489a6f8febd3f3fa255a7fdfe9"},{"name":".pre-commit-config.yaml","path":".pre-commit-config.yaml","sha":"71195b5cefdf89443b15adbdfdc7735b11833a37"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"19e775a4b681f94468d8fd2a6a2780cfbe91ecff"},{"name":"CONTRIBUTING.md","path":"CONTRIBUTING.md","sha":"492bef006a63f2c3189fd171116eb3f900c6cd02"},{"name":"LICENSE.md","path":"LICENSE.md","sha":"a2cf01ecdd725fddd718ab91c80c115882c94f3c"},{"name":"README.md","path":"README.md","sha":"887c739ec9651db0f4827445b56f975345b2820f"},{"name":"_docs","children":[{"name":"elk-architecture.png","path":"_docs/elk-architecture.png","sha":"a7585c6f40eb0a7740b34f00d412bc2e37373266"}]},{"name":"examples","children":[{"name":"elasticsearch-docker","children":[{"name":"local-mocks","children":[{"name":"entrypoint.sh","path":"examples/elasticsearch-docker/local-mocks/entrypoint.sh","sha":"a7405f93f4482217729e3694d14a59a331471750"}]},{"name":"non-ssl","children":[{"name":".env","path":"examples/elasticsearch-docker/non-ssl/.env","sha":"bc82d24eea229df2ba77f0fdf1f6e1c6f6a8e532"},{"name":"README.md","path":"examples/elasticsearch-docker/non-ssl/README.md","sha":"ed4bee52f5093a864164067b57d54f41668e55f5"},{"name":"docker-compose.yml","path":"examples/elasticsearch-docker/non-ssl/docker-compose.yml","sha":"e7547917f574679857f800b24f86bd573671931e"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/elasticsearch-docker/non-ssl/user-data/user-data.sh","sha":"81df178b3080a40a9e03ff7e94f1ef5db7d57f98"}]}]},{"name":"ssl","children":[{"name":".env","path":"examples/elasticsearch-docker/ssl/.env","sha":"bc82d24eea229df2ba77f0fdf1f6e1c6f6a8e532"},{"name":"README.md","path":"examples/elasticsearch-docker/ssl/README.md","sha":"ed4bee52f5093a864164067b57d54f41668e55f5"},{"name":"docker-compose.yml","path":"examples/elasticsearch-docker/ssl/docker-compose.yml","sha":"25e2b036c7e672a8ffc4b9559ab53f8d77441f99"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/elasticsearch-docker/ssl/user-data/user-data.sh","sha":"fc95c5a93069b251aca3ec2ae70a2be90231fcd9"}]}]}]},{"name":"elasticsearch-only-cluster","children":[{"name":"README.md","path":"examples/elasticsearch-only-cluster/README.md","sha":"e830acc97cf711f5087d03db0c61dc6ed82b1b8c"},{"name":"main.tf","path":"examples/elasticsearch-only-cluster/main.tf","sha":"16e304e12274a2c4cbc6c5b4c1d1758144ce259b"},{"name":"outputs.tf","path":"examples/elasticsearch-only-cluster/outputs.tf","sha":"382f2cda2c87d824b274f986f5107fd90477544a"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/elasticsearch-only-cluster/user-data/user-data.sh","sha":"1248e6d8692154f3d024ad95dab49a7a8646bc31"}]},{"name":"vars.tf","path":"examples/elasticsearch-only-cluster/vars.tf","sha":"6c3db8f2b043c53d04f7278a076230b4ce06308a"}]},{"name":"elk-amis","children":[{"name":"README.md","path":"examples/elk-amis/README.md","sha":"2d26c99b7a9a0d3a2124a771f0335eb5251f3b4c"},{"name":"all-in-one","children":[{"name":"README.md","path":"examples/elk-amis/all-in-one/README.md","sha":"42799bc50b4f41eb9e27ed79f1b5624b812b81b2"},{"name":"all-in-one.json","path":"examples/elk-amis/all-in-one/all-in-one.json","sha":"ecd2d44afe1d6040f90e304bc6cf042ec3668bc9"}]},{"name":"app-server","children":[{"name":"README.md","path":"examples/elk-amis/app-server/README.md","sha":"577966ff3d1aa65d69aa43d6b7e1e86968099335"},{"name":"app-server.json","path":"examples/elk-amis/app-server/app-server.json","sha":"6098834ef5ea26e93099f5131f4bd7c8ad77e82a"}]},{"name":"collectd","children":[{"name":"README.md","path":"examples/elk-amis/collectd/README.md","sha":"a9fa0d44e853022868368cab446e4aaeb24997ab"},{"name":"collectd-install-steps.sh","path":"examples/elk-amis/collectd/collectd-install-steps.sh","sha":"bfe1b037dcdee47751c588c4e431bfefec4661ee"},{"name":"collectd.json","path":"examples/elk-amis/collectd/collectd.json","sha":"236b08e2e60af5d34a9efabca7f49cf90772201f"},{"name":"config","children":[{"name":"collectd-ssl.conf","path":"examples/elk-amis/collectd/config/collectd-ssl.conf","sha":"0ccfc0f2aee65c0e17f5cd530c5edaccf247c3d8"},{"name":"collectd.conf","path":"examples/elk-amis/collectd/config/collectd.conf","sha":"74c1b46937039070cd95f64269eef63aabd72805"}]}]},{"name":"elastalert","children":[{"name":"README.md","path":"examples/elk-amis/elastalert/README.md","sha":"9dd33dfd29ddafa1040213466560ffcf8a66944d"},{"name":"aws","children":[{"name":"elastalert-config","children":[{"name":"config-ssl.yml","path":"examples/elk-amis/elastalert/aws/elastalert-config/config-ssl.yml","sha":"f26f8404e40736c5b0b17bc23e6a5de119d0c80f"},{"name":"config.yml","path":"examples/elk-amis/elastalert/aws/elastalert-config/config.yml","sha":"6054d91b8e86df9102fd5fd1fa2c545f7a981d31"}]}]},{"name":"docker","children":[{"name":"elastalert-config","children":[{"name":"config-ssl.yml","path":"examples/elk-amis/elastalert/docker/elastalert-config/config-ssl.yml","sha":"f26f8404e40736c5b0b17bc23e6a5de119d0c80f"},{"name":"config.yml","path":"examples/elk-amis/elastalert/docker/elastalert-config/config.yml","sha":"6054d91b8e86df9102fd5fd1fa2c545f7a981d31"}]}]},{"name":"elastalert-install-steps.sh","path":"examples/elk-amis/elastalert/elastalert-install-steps.sh","sha":"7da5b70623c85a0d61762a2ac5a2d91afd58d969"},{"name":"elastalert-rules","children":[{"name":"example_change.yml","path":"examples/elk-amis/elastalert/elastalert-rules/example_change.yml","sha":"4020886483786ff9a8847159b5750014c5d2b0fb"}]},{"name":"elastalert.json","path":"examples/elk-amis/elastalert/elastalert.json","sha":"762483ee0d35fdc4179cea051779f1076991fee5"}]},{"name":"elasticsearch","children":[{"name":"README.md","path":"examples/elk-amis/elasticsearch/README.md","sha":"f77a938e72424815565ffe16b9ae7787cb430cc7"},{"name":"aws","children":[{"name":"config","children":[{"name":"elasticsearch-ssl.yml","path":"examples/elk-amis/elasticsearch/aws/config/elasticsearch-ssl.yml","sha":"e6b0e5d3efd07f195aa18ae567789548a0eebc68"},{"name":"elasticsearch.yml","path":"examples/elk-amis/elasticsearch/aws/config/elasticsearch.yml","sha":"4918d1d6ec52dafa7741bda8212240d75b162f37"},{"name":"jvm.options","path":"examples/elk-amis/elasticsearch/aws/config/jvm.options","sha":"28dcf519ee339aba5dd15059c1acc4b97846d6ce"}]}]},{"name":"docker","children":[{"name":"config","children":[{"name":"elasticsearch-ssl.yml","path":"examples/elk-amis/elasticsearch/docker/config/elasticsearch-ssl.yml","sha":"705af52bc1e914df93faa919de8a23b973184938"},{"name":"elasticsearch.yml","path":"examples/elk-amis/elasticsearch/docker/config/elasticsearch.yml","sha":"2ac38b9c875dd6352b7a3d7d0fc2c02662d0e9cb"},{"name":"jvm.options","path":"examples/elk-amis/elasticsearch/docker/config/jvm.options","sha":"28dcf519ee339aba5dd15059c1acc4b97846d6ce"}]}]},{"name":"elasticsearch-install-steps.sh","path":"examples/elk-amis/elasticsearch/elasticsearch-install-steps.sh","sha":"a0de2d98730e3b1d0e3c5060dc80d925cb4043d3"},{"name":"elasticsearch.json","path":"examples/elk-amis/elasticsearch/elasticsearch.json","sha":"c8bfafebb715d5e1305f7c758274ce63de9f73ec"},{"name":"plugins","children":[{"name":"readonlyrest-1.22.1_es6.8.12.zip","path":"examples/elk-amis/elasticsearch/plugins/readonlyrest-1.22.1_es6.8.12.zip","sha":"93bdf77044e9ac16fb6eb8e091a1700419425ea9"}]},{"name":"readonlyrest-config","children":[{"name":"readonlyrest.yml","path":"examples/elk-amis/elasticsearch/readonlyrest-config/readonlyrest.yml","sha":"e056fad877791d455e337ff867845c26f514874a"}]}]},{"name":"filebeat","children":[{"name":"README.md","path":"examples/elk-amis/filebeat/README.md","sha":"ce76c3cbb426dc4d5e82f04a85eb0efd043fe671"},{"name":"config","children":[{"name":"filebeat-ssl.yml","path":"examples/elk-amis/filebeat/config/filebeat-ssl.yml","sha":"24b163e671b4cc49af08726867d8869f105a8ecb"},{"name":"filebeat.yml","path":"examples/elk-amis/filebeat/config/filebeat.yml","sha":"ae4641ee77c89e389728836fca3cac1428214fe2"}]},{"name":"filebeat-install-steps.sh","path":"examples/elk-amis/filebeat/filebeat-install-steps.sh","sha":"e7374df095835dddd4d89555d5529c7c3fdff175"},{"name":"filebeat.json","path":"examples/elk-amis/filebeat/filebeat.json","sha":"50b118a2452670dc1291f8e302a7045cba0c958a"}]},{"name":"kibana","children":[{"name":"README.md","path":"examples/elk-amis/kibana/README.md","sha":"b8af363345ba445cb9fbc2b04ec0e7d6b0b08149"},{"name":"config","children":[{"name":"kibana-ssl.yml","path":"examples/elk-amis/kibana/config/kibana-ssl.yml","sha":"7bcab515eb6cba2026899fb711541ebdecc97e3d"},{"name":"kibana.yml","path":"examples/elk-amis/kibana/config/kibana.yml","sha":"852b146720a4ab3fc1380beacce6f77851017b51"}]},{"name":"kibana-install-steps.sh","path":"examples/elk-amis/kibana/kibana-install-steps.sh","sha":"3be2e4579c93962ad8da899728ac3b496a5cd78e"},{"name":"kibana.json","path":"examples/elk-amis/kibana/kibana.json","sha":"6293086359621b813fc384d580598de122f29db0"}]},{"name":"logstash","children":[{"name":"README.md","path":"examples/elk-amis/logstash/README.md","sha":"b1035b8b3d34f28000e329171ba65275b1fac258"},{"name":"config","children":[{"name":"jvm.options","path":"examples/elk-amis/logstash/config/jvm.options","sha":"6017dd218501163070f950077ce6a1e9cff8f3a4"},{"name":"logstash-ssl.yml","path":"examples/elk-amis/logstash/config/logstash-ssl.yml","sha":"46e72285a2074ceba2313f726db5eb0db6dfa414"},{"name":"logstash.yml","path":"examples/elk-amis/logstash/config/logstash.yml","sha":"46e72285a2074ceba2313f726db5eb0db6dfa414"},{"name":"pipeline-ssl.conf","path":"examples/elk-amis/logstash/config/pipeline-ssl.conf","sha":"723780826865edbc2274837265a9059f9c7c1397"},{"name":"pipeline.conf","path":"examples/elk-amis/logstash/config/pipeline.conf","sha":"4a68c41fc5e86106915cb1c91ec39bd4c0cd2b55"}]},{"name":"logstash-install-steps.sh","path":"examples/elk-amis/logstash/logstash-install-steps.sh","sha":"e44ef50df6ac8989bfae91012cf04bbe386121b2"},{"name":"logstash.json","path":"examples/elk-amis/logstash/logstash.json","sha":"5ebdff0ca36a3bbe30043c3ef9b88959d1cdf3a8"}]}]},{"name":"elk-multi-cluster","children":[{"name":"README.md","path":"examples/elk-multi-cluster/README.md","sha":"8fd9aa87b62930d731636f72f6c0d22f900cb087"},{"name":"main.tf","path":"examples/elk-multi-cluster/main.tf","sha":"41e03942f41795d52e001ad11c5ea974abbf0dea"},{"name":"outputs.tf","path":"examples/elk-multi-cluster/outputs.tf","sha":"8bc4cde32815078921bc75375d963b533f6c50db"},{"name":"user-data","children":[{"name":"app-server","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/app-server/user-data.sh","sha":"e3ab01683b870b0bd96ad620d4b25860a7d7a833"}]},{"name":"elastalert","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/elastalert/user-data.sh","sha":"48181b3b07a3aec5e5b808b4101a9ef447eed37b"}]},{"name":"elasticsearch","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/elasticsearch/user-data.sh","sha":"7c7f36d79d716ff7e7618455097513aa936919bf"}]},{"name":"kibana","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/kibana/user-data.sh","sha":"b7f5e2b2d8c7091f1aa3e036a723913d9feca37b"}]},{"name":"logstash","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/logstash/user-data.sh","sha":"9e1af5d687be9c44149560d2a022b62b88743b44"}]}]},{"name":"vars.tf","path":"examples/elk-multi-cluster/vars.tf","sha":"43f00586a4b0fb616a2904f6bb11112ab334f36e"}]},{"name":"elk-single-cluster","children":[{"name":"README.md","path":"examples/elk-single-cluster/README.md","sha":"33320b782aca0aea4d298599e5d3d706c93a12cf"},{"name":"main.tf","path":"examples/elk-single-cluster/main.tf","sha":"1c58be21a641bc4c0bdb683fcc5afd37258a8649"},{"name":"outputs.tf","path":"examples/elk-single-cluster/outputs.tf","sha":"c12a0b236e6fcbf5881cb60276d62a432dadb87a"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/elk-single-cluster/user-data/user-data.sh","sha":"446a992b93c4a3a53a86377546b0cc520bc941d3"}]},{"name":"vars.tf","path":"examples/elk-single-cluster/vars.tf","sha":"a67bfbaabbca1a1f25865570d9cc1a70d369485b"}]}]},{"name":"modules","children":[{"name":"auto-discovery","children":[{"name":"README.md","path":"modules/auto-discovery/README.md","sha":"f4de11c428f8fefd962220e7fbc4b976b4f838fe"},{"name":"bin","children":[{"name":"auto-discovery","path":"modules/auto-discovery/bin/auto-discovery","sha":"fa1f2368e4ff71c550151ed9bd67107515d04873"}]},{"name":"install.sh","path":"modules/auto-discovery/install.sh","sha":"99a2f2fb2cbfaa31e752cbcc6495e5b755f034f2"}]},{"name":"beats-iam-policies","children":[{"name":"main.tf","path":"modules/beats-iam-policies/main.tf","sha":"e9a8c04747f61ad5aa049421312e955773f656e6"},{"name":"vars.tf","path":"modules/beats-iam-policies/vars.tf","sha":"52873644e20b4cafdcc0f33dc11db99f1fa9b586"}]},{"name":"elastalert-iam-policies","children":[{"name":"README.md","path":"modules/elastalert-iam-policies/README.md","sha":"012578675e314c2985c4c4c2ef7e58bd8eb37956"},{"name":"main.tf","path":"modules/elastalert-iam-policies/main.tf","sha":"e4f7ef96c2a388c60e6a6fb315f8cdac3256e280"},{"name":"variables.tf","path":"modules/elastalert-iam-policies/variables.tf","sha":"bc55d045eba3b8df1b08f78a4657cae25ee0ea73"}]},{"name":"elastalert-security-group-rules","children":[{"name":"README.md","path":"modules/elastalert-security-group-rules/README.md","sha":"050c72419243011fbcbdd5359764a2f9a22d0aaa"},{"name":"main.tf","path":"modules/elastalert-security-group-rules/main.tf","sha":"dae2fbeebd5a32a9dcd12194e7a433ea9762f52d"},{"name":"variables.tf","path":"modules/elastalert-security-group-rules/variables.tf","sha":"0d87d8de7864335c546406e1047a680f14e6ec73"}]},{"name":"elastalert","children":[{"name":"README.md","path":"modules/elastalert/README.md","sha":"ebc03e5af2cd6e03520a8e9ab126b34c9782acc6"},{"name":"main.tf","path":"modules/elastalert/main.tf","sha":"241f707360fec4cf509098fffe13f7f2bede372d"},{"name":"outputs.tf","path":"modules/elastalert/outputs.tf","sha":"aa5817acc77a75b75f4ea4acc686332c5b865618"},{"name":"vars.tf","path":"modules/elastalert/vars.tf","sha":"bd053dc8923e14a406bae848f75c92e4f0e8a822"}]},{"name":"elasticsearch-cluster-backup","children":[{"name":"README.md","path":"modules/elasticsearch-cluster-backup/README.md","sha":"2c3ca6928de82cfbdb8b96a4eaf5a2b86a0d39c7"},{"name":"backup","children":[{"name":"index.js","path":"modules/elasticsearch-cluster-backup/backup/index.js","sha":"6d3ff6a2a58d70dccd55399ba8bf0e0270140279"}]},{"name":"main.tf","path":"modules/elasticsearch-cluster-backup/main.tf","sha":"f6ed9528ec7b0f77513c638ad6410c118474353b"},{"name":"notification","children":[{"name":"index.js","path":"modules/elasticsearch-cluster-backup/notification/index.js","sha":"8e92e6b387974c4f0c6f007b8e3555d64e984202"}]},{"name":"outputs.tf","path":"modules/elasticsearch-cluster-backup/outputs.tf","sha":"ce7c4ee206984e74253ad6bb0bf25cd4a3965053"},{"name":"vars.tf","path":"modules/elasticsearch-cluster-backup/vars.tf","sha":"93c34c8c22ecfa0c9498e450df4886996b4bcfa1"}]},{"name":"elasticsearch-cluster-restore","children":[{"name":"README.md","path":"modules/elasticsearch-cluster-restore/README.md","sha":"754fa372fff69c4726499744ff41a83fe5c40446"},{"name":"main.tf","path":"modules/elasticsearch-cluster-restore/main.tf","sha":"f960c5c7e8298f62557bee5afef162721e7029f0"},{"name":"notification","children":[{"name":"index.js","path":"modules/elasticsearch-cluster-restore/notification/index.js","sha":"4e441662f92edf384c0c92a96ea5e1a7e1b0b24d"}]},{"name":"outputs.tf","path":"modules/elasticsearch-cluster-restore/outputs.tf","sha":"92454e1cf45fcc57f60c09eb8b31064efb460726"},{"name":"restore","children":[{"name":"index.js","path":"modules/elasticsearch-cluster-restore/restore/index.js","sha":"7d84ea278f01b64310fbc33d18c86a2a73b60573"}]},{"name":"vars.tf","path":"modules/elasticsearch-cluster-restore/vars.tf","sha":"b1c5309ac8d80d86b7ad12912fbf003d7177028f"}]},{"name":"elasticsearch-cluster","children":[{"name":"README.md","path":"modules/elasticsearch-cluster/README.md","sha":"96381fd14e010c475343d0eee79641c13ffe3e4e"},{"name":"main.tf","path":"modules/elasticsearch-cluster/main.tf","sha":"13d41726d7cf5dc768c0414390d922bdd9da450f"},{"name":"outputs.tf","path":"modules/elasticsearch-cluster/outputs.tf","sha":"0231e24415e13f62f29d8abefe0398ddb6d0fa04"},{"name":"variables.tf","path":"modules/elasticsearch-cluster/variables.tf","sha":"8b680c1329382792bee8415ff36bd1314675a4c9"}]},{"name":"elasticsearch-iam-policies","children":[{"name":"README.md","path":"modules/elasticsearch-iam-policies/README.md","sha":"efe4837043dbce44ce15e2b9959d2cfa11983485"},{"name":"main.tf","path":"modules/elasticsearch-iam-policies/main.tf","sha":"55bf1f01c743d84523b3d24ab9e41787a1e28d39"},{"name":"variables.tf","path":"modules/elasticsearch-iam-policies/variables.tf","sha":"c612a588acea6049cfdb930209854e253b549729"}]},{"name":"elasticsearch-security-group-rules","children":[{"name":"README.md","path":"modules/elasticsearch-security-group-rules/README.md","sha":"3292b4b51529305c57cccd4687c64bfa2a2dd6ec"},{"name":"main.tf","path":"modules/elasticsearch-security-group-rules/main.tf","sha":"cdb770582d24b7a9fc7502dd395c6f98a54acbac"},{"name":"outputs.tf","path":"modules/elasticsearch-security-group-rules/outputs.tf","sha":"e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"},{"name":"variables.tf","path":"modules/elasticsearch-security-group-rules/variables.tf","sha":"7f055aa865ca315861e7a69f2d7741ab7883fddf"}]},{"name":"install-collectd","children":[{"name":"README.md","path":"modules/install-collectd/README.md","sha":"ed7f82bbf5f6a3cd5042b8cf78a8b69292f6a929"},{"name":"install.sh","path":"modules/install-collectd/install.sh","sha":"4090c1e8b94dd1a4d30ce68da03b73c281e53a3a"}]},{"name":"install-elastalert","children":[{"name":"README.md","path":"modules/install-elastalert/README.md","sha":"ab3850634329401b2734c5ed46064f93afbb989e"},{"name":"install.sh","path":"modules/install-elastalert/install.sh","sha":"2428d52ac73bd555044c5848546aac8b564c3a5f"}]},{"name":"install-elasticsearch","children":[{"name":"README.md","path":"modules/install-elasticsearch/README.md","sha":"ca54645f1ea89c80bb6953a2e5d70abec68a5346"},{"name":"install.sh","path":"modules/install-elasticsearch/install.sh","sha":"89c6d78790ed08a49af5821e04a459ec4c6b89d6"}]},{"name":"install-filebeat","children":[{"name":"README.md","path":"modules/install-filebeat/README.md","sha":"e8aea474de93cfb6421e038bdfd0459bf9936055"},{"name":"install.sh","path":"modules/install-filebeat/install.sh","sha":"b7537d09a916f93ded0214c5386d897e2f38bf56"}]},{"name":"install-kibana","children":[{"name":"README.md","path":"modules/install-kibana/README.md","sha":"9517d9c1b2f73823a00f8c1b190bb90e5b90312e"},{"name":"install.sh","path":"modules/install-kibana/install.sh","sha":"35eeef5d8aa75b4a1dd54d2913acf6688429c780"}]},{"name":"install-logstash","children":[{"name":"README.md","path":"modules/install-logstash/README.md","sha":"e1a8bde78e9bae25b852d7e9e44d0c6f372c3294"},{"name":"install.sh","path":"modules/install-logstash/install.sh","sha":"a689e67980aa28d7d2f04f693ccb59138ccfe3e2"}]},{"name":"kibana-cluster","children":[{"name":"README.md","path":"modules/kibana-cluster/README.md","sha":"87b31b9825999d66d6d334b48dc73de56ba647f7"},{"name":"main.tf","path":"modules/kibana-cluster/main.tf","sha":"7eb4359660f7951c4551e44babd9649568f2be31"},{"name":"outputs.tf","path":"modules/kibana-cluster/outputs.tf","sha":"f32e707371481d34b2b84d4ef7f205afaa04f2d4"},{"name":"variables.tf","path":"modules/kibana-cluster/variables.tf","sha":"58d03200bff5a18c52347d42af2a6ac1a6452802"}]},{"name":"kibana-security-group-rules","children":[{"name":"README.md","path":"modules/kibana-security-group-rules/README.md","sha":"f8d904f11c6a2238c5be9100c7e0255de28e5e2d"},{"name":"main.tf","path":"modules/kibana-security-group-rules/main.tf","sha":"f890ae74fae825d5db87f5011ba6638af874fda2"},{"name":"outputs.tf","path":"modules/kibana-security-group-rules/outputs.tf","sha":"e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"},{"name":"variables.tf","path":"modules/kibana-security-group-rules/variables.tf","sha":"b89d5c7f5f1d1b0e9b5e1d239a1dec9a4ea34486"}]},{"name":"load-balancer-alb-target-group","children":[{"name":"README.md","path":"modules/load-balancer-alb-target-group/README.md","sha":"0a95fccf4f76af1149a6ab10d2854156e28f03f3"},{"name":"main.tf","path":"modules/load-balancer-alb-target-group/main.tf","sha":"e96c609cc002f2a9de68147aeb5ea60b6d1eb108"},{"name":"outputs.tf","path":"modules/load-balancer-alb-target-group/outputs.tf","sha":"cdf904445cf5099ccea5cc8820d198a1ffba4283"},{"name":"variables.tf","path":"modules/load-balancer-alb-target-group/variables.tf","sha":"4a262b0abde2721f9e3a5edeb6a98046009b9df3"}]},{"name":"logstash-cluster","children":[{"name":"README.md","path":"modules/logstash-cluster/README.md","sha":"78e4b47484d9f68ad16ffe06fb1af15991282cfe"},{"name":"main.tf","path":"modules/logstash-cluster/main.tf","sha":"55219e71d6f49ec878fafebad547e2573e4449be"},{"name":"outputs.tf","path":"modules/logstash-cluster/outputs.tf","sha":"218a4f0042d52ea1673de0c4f78857a4831143a6"},{"name":"vars.tf","path":"modules/logstash-cluster/vars.tf","sha":"bff728633e0f853502c3a15b4f9311b5b9d2d41f"}]},{"name":"logstash-iam-policies","children":[{"name":"README.md","path":"modules/logstash-iam-policies/README.md","sha":"3e354879298dbe3456cf3bae79594ee815487ac4"},{"name":"main.tf","path":"modules/logstash-iam-policies/main.tf","sha":"0ce69e25d9c2f964dc35f41e10813af86f2a5a7c"},{"name":"vars.tf","path":"modules/logstash-iam-policies/vars.tf","sha":"865c74386084c6eff1f46cb103aa847a6ad0da68"}]},{"name":"logstash-security-group-rules","children":[{"name":"README.md","path":"modules/logstash-security-group-rules/README.md","sha":"7ca6878f6b4415297d21c96ca48a4d8aa0314095"},{"name":"main.tf","path":"modules/logstash-security-group-rules/main.tf","sha":"dd54760fec89fa4a83ed37835d9901256af24bf5"},{"name":"outputs.tf","path":"modules/logstash-security-group-rules/outputs.tf","sha":"3641c561d0e23ef02fa83319ddafa8d1f9f813d2"},{"name":"vars.tf","path":"modules/logstash-security-group-rules/vars.tf","sha":"9e26c219ddad783caf509581fae84953eae60fe9"}]},{"name":"run-collectd","children":[{"name":"README.md","path":"modules/run-collectd/README.md","sha":"2242bcaca7f50717b3d6154c0576847d7b08c12c"},{"name":"bin","children":[{"name":"run-collectd","path":"modules/run-collectd/bin/run-collectd","sha":"7f7971d8b564a84a251d4c253c42f9025b5a1cf7"}]},{"name":"install.sh","path":"modules/run-collectd/install.sh","sha":"d2d8126f6fc5ba95a4e12242aa6ffe9bdc945ba6"}]},{"name":"run-elastalert","children":[{"name":"README.md","path":"modules/run-elastalert/README.md","sha":"e52cf808ab755b2d56a392ee8f525a0651963472","toggled":true},{"name":"bin","children":[{"name":"run-elastalert","path":"modules/run-elastalert/bin/run-elastalert","sha":"cccab0f5cf41600c1b8ad48da5173db97965440f"}]},{"name":"install.sh","path":"modules/run-elastalert/install.sh","sha":"308f895a2366fc0cd8e6669229f6928a3d888f13"}],"toggled":true},{"name":"run-elasticsearch","children":[{"name":"README.md","path":"modules/run-elasticsearch/README.md","sha":"598ecd294a3177b5f9791793f75cec95559ad886"},{"name":"bin","children":[{"name":"run-elasticsearch","path":"modules/run-elasticsearch/bin/run-elasticsearch","sha":"e8e21b99ca07e03ce807ad537fbd9c3858912713"}]},{"name":"install.sh","path":"modules/run-elasticsearch/install.sh","sha":"c66e6da3d46195ac94d68eed18ed301ca7e0eb51"}]},{"name":"run-filebeat","children":[{"name":"README.md","path":"modules/run-filebeat/README.md","sha":"7b20727ebd231d78a39f30eeb2bad79619447f0a"},{"name":"bin","children":[{"name":"run-filebeat","path":"modules/run-filebeat/bin/run-filebeat","sha":"2e323cd6b7550db8470085911fa372c4edad8ae6"}]},{"name":"install.sh","path":"modules/run-filebeat/install.sh","sha":"cd890712803c3961ada86acf1e13b4c299402fb6"}]},{"name":"run-kibana","children":[{"name":"README.md","path":"modules/run-kibana/README.md","sha":"f6aaf8c3bd3901d753551ab4c28c0772d73ed143"},{"name":"bin","children":[{"name":"run-kibana","path":"modules/run-kibana/bin/run-kibana","sha":"0f60bca1be5eebc5594de9e8eab162aa979d4a17"}]},{"name":"install.sh","path":"modules/run-kibana/install.sh","sha":"e23b447832fce06ad32ce654f94a182a521e2e38"}]},{"name":"run-logstash","children":[{"name":"README.md","path":"modules/run-logstash/README.md","sha":"e1b6d3752f9f2f917c4bb17283caabd163f4d589"},{"name":"bin","children":[{"name":"run-logstash","path":"modules/run-logstash/bin/run-logstash","sha":"60dc8d906d8181e5a94c3ad18b5b27e53d513cf1"}]},{"name":"install.sh","path":"modules/run-logstash/install.sh","sha":"0a30f2adc2d0765b5581274263e12ce2e688b623"}]}],"toggled":true},{"name":"terraform-cloud-enterprise-private-module-registry-placeholder.tf","path":"terraform-cloud-enterprise-private-module-registry-placeholder.tf","sha":"ae586c0fe830819580e1009d41a9074f16e65bed"},{"name":"test","children":[{"name":"README.md","path":"test/README.md","sha":"819e790d15767c851bfc438f0a1413ed2bde7d3b"},{"name":"elasticsearch_aws_simple_test.go","path":"test/elasticsearch_aws_simple_test.go","sha":"cccda6ac30573bc7fea6772f56f68a96ab1b0581"},{"name":"elasticsearch_docker_test.go","path":"test/elasticsearch_docker_test.go","sha":"9aab286b753db9127f7425acea12bcc93e5abdd1"},{"name":"elk_aio_test.go","path":"test/elk_aio_test.go","sha":"b1ca766cbee9d5b69e1028ef48e971f4d333232b"},{"name":"elk_end_to_end_test.go","path":"test/elk_end_to_end_test.go","sha":"fb74457130157625a2d817e0d54d6be55aa6a3c2"},{"name":"go.mod","path":"test/go.mod","sha":"b6d375ede5dfd840285c97e9e8c1e95a6b8cfb52"},{"name":"go.sum","path":"test/go.sum","sha":"c64c51d793c7c3ec7848f7998ddb9494a979ea59"},{"name":"notes.md","path":"test/notes.md","sha":"664915d4b948ccd194ffda80854f47c191904e3d"},{"name":"test_helpers.go","path":"test/test_helpers.go","sha":"0ba1e792e7a26419f43c55971e4910cbc242a009"},{"name":"test_helpers_keystore.go","path":"test/test_helpers_keystore.go","sha":"e3b1ed8963088a2bcbdc2863287e60d19d23cd47"},{"name":"tls_helpers.go","path":"test/tls_helpers.go","sha":"828f04c10b9ca6f4b6f8f82ddcd498bdb1c7dbb4"}]}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"run-elastalert-script\">Run Elastalert Script</h1><div class=\"preview__body--border\"></div><p>This folder contains a script for configuring and running <a href=\"https://github.com/Yelp/elastalert\" class=\"preview__body--description--blue\" target=\"_blank\">ElastAlert</a> on an AWS server. This script has been tested on the following operating systems:</p>\n<ul>\n<li>Ubuntu 16.04</li>\n<li>Ubuntu 18.04</li>\n<li>Amazon Linux 2</li>\n<li>CentOS 7</li>\n</ul>\n<p>There is a good chance it will work on other flavors of Debian, CentOS, and RHEL as well.</p>\n<h2 class=\"preview__body--subtitle\" id=\"what-is-elast-alert\">What is ElastAlert?</h2>\n<p>ElastAlert is a simple framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch. Note that it's not an <a href=\"https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-plugins.html\" class=\"preview__body--description--blue\" target=\"_blank\">Elasticsearch plugin</a>, just a Python service that connects to the ElasticSearch API. However, there is a separate <a href=\"https://github.com/bitsensor/elastalert-kibana-plugin\" class=\"preview__body--description--blue\" target=\"_blank\">ElastAlert Kibana Plugin</a> that exposes ElastAlert functionality in Kibana.</p>\n<p>The key idea is that you define an Elasticsearch query as part of a <strong>rule</strong>, which is of some <strong>rule type</strong> that specifies when that query should trigger an <strong>alert,</strong> which is of some <strong>alert type</strong>.</p>\n<p>For example, you might use the <code>frequency</code> rule type ("Match where there are at least X events in Y time") to define a rule that triggers an alert of type <code>Slack</code> so that a Slack message appears when the rule is triggered.</p>\n<h2 class=\"preview__body--subtitle\" id=\"quick-start\">Quick start</h2>\n<p>This script depends on <a href=\"/repos/bash-commons\" class=\"preview__body--description--blue\">bash-commons</a>, so you must install that project\nfirst as documented in its README.</p>\n<p>This script also assumes you installed ElastAlert using the <a href=\"/repos/v0.7.1/package-elk/modules/install-elastalert\" class=\"preview__body--description--blue\">install-elastalert module</a>, and are already running a live Elasticsearch cluster (most likely from using the <a href=\"/repos/v0.7.1/package-elk/modules/run-elasticsearch\" class=\"preview__body--description--blue\">run-elasticsearch module</a>).</p>\n<ol>\n<li>\n<p>Review the <code>config.yaml</code> file in <code>/opt/elastalert/config/config.yaml</code> and make sure the <code>rules_folder</code> property specifies the folder that contains all the ElastAlert rule definitions.</p>\n</li>\n<li>\n<p>Review the <a href=\"/repos/v0.7.1/package-elk/examples/elk-amis/elastalert/elastalert-rules\" class=\"preview__body--description--blue\">example rules</a>. As a quick start, use the <code>sns</code> alert type by making sure the example rule contains the following and no other <code>alert</code> definitions:</p>\n</li>\n</ol>\n<pre><span class=\"hljs-symbol\">alert:</span>\n- <span class=\"hljs-string\">\"sns\"</span>\n\n<span class=\"hljs-symbol\">aws_region:</span> <span class=\"hljs-string\">\"fill-this-in\"</span>\n<span class=\"hljs-symbol\">sns_topic_arn:</span> <span class=\"hljs-string\">\"fill-this-in\"</span>\n</pre>\n<ol>\n<li>\n<p>Now start ElastAlert by executing the <code>run-elastalsert</code> script as follows:</p>\n<pre>/opt/elastalert/bin/<span class=\"hljs-built_in\">run</span>-elastalert --<span class=\"hljs-built_in\">config</span> /opt/elastalert/<span class=\"hljs-built_in\">config</span>/<span class=\"hljs-built_in\">config</span>.yaml\n</pre>\n<p>This will begin querying data in Elasticsearch starting from right this moment.</p>\n</li>\n<li>\n<p>Finally, send some data into the Elasticsearch cluster and confirm that you've received an email (TODO: How best to send data to cluster?)</p>\n</li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"command-line-arguments\">Command line Arguments</h2>\n<p>Run <code>run-elastalsert --help</code> to see all available arguments, or just check out the <a href=\"/repos/v0.7.1/package-elk/modules/run-elastalert/bin/run-elastalert\" class=\"preview__body--description--blue\">run-elastalert source code</a>.</p>\n<h2 class=\"preview__body--subtitle\" id=\"alternatives\">Alternatives</h2>\n<h3 class=\"preview__body--subtitle\" id=\"elastalert-vs-x-pack\">Elastalert vs. X-Pack</h3>\n<p><a href=\"https://www.elastic.co/guide/en/x-pack/6.2/xpack-introduction.html\" class=\"preview__body--description--blue\" target=\"_blank\">X-Pack</a> is a full-featured, commercial plugin maintained by\nelastic.co that includes the following features:</p>\n<ul>\n<li><strong>Security.</strong> Lock down elasticsearch nodes to certain IPs or require authentication via LDAP, Active Directory, or\nbasic authentication.</li>\n<li><strong>Monitoring.</strong> Monitor the components of the Elasticsearch cluster: Elasticsearch, Logstash, and Kibana.</li>\n<li><strong>Alerting and Notifications.</strong> Define a query and a condition under which its results should trigger a notification.</li>\n<li><strong>Reports.</strong> You can download Kibana reports as PDFs optimized for printing.</li>\n<li><strong>Query Relationships Between Data.</strong> Explore how different attributes of a table connecto to each other with visual graph renderings.</li>\n<li><strong>Machine Learning.</strong> Perform "Time Series Anomaly Detection" to automatically alert on unusual changes.</li>\n</ul>\n<p>By contrast, Elastalert is limited to alerting and notifications. We chose Elastalert over X-Package because, as an alerting plugin, Elastalert compares favorably to X-Pack, and it's open source. However, if you require other features of X-Pack, you may prefer to use X-Pack for alerting and notifications as well.</p>\n","repoName":"package-elk","repoRef":"v0.9.0","serviceDescriptor":{"serviceName":"Elasticsearch (self-hosted)","serviceRepoName":"package-elk","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws"],"description":"Deploy a self-hosted Elasticsearch cluster. Supports automatic bootstrap, zero-downtime rolling deployment, auto healing, backup, and recovery.","imageUrl":"elk.png","licenseType":"subscriber","technologies":["Terraform","Bash","JavaScript"],"compliance":[],"tags":[""]},"serviceCategoryName":"NoSQL","fileName":"README.md","filePath":"/modules/run-elastalert","title":"Repo Browser: Elasticsearch (self-hosted)","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}