Run run-elasticsearch --help to see all available arguments.
Usage: run-elasticsearch [OPTIONS]
This script can be used to configure andrun Elasticsearch.
Optional arguments:
--auto-fill-endpoint VALUE should be aws_region. Value will be converted to aws_endpoint. KEY in Elasticsearch config file will be replaced with aws_endpoint. Only needed if running with ec2 discovery plugin.
--auto-fill KEY=VALUE Search the Elasticsearch config file for KEY and replace it with VALUE. May be repeated.
--auto-fill-jvm KEY=VALUE Search the Elasticsearch JVM config file for KEY and replace it with VALUE. May be repeated.
Example:
install.sh
--auto-fill "<__CLUSTER_NAME__>=the-cluster-name" --auto-fill-jvm "<__XMS__>=4g"
How it works
The run-elasticsearch script:
Replace <__KEY_NAME__> with VALUE that you specify as part of your --auto-fill arguments in the Elasticsearch config file located in: /etc/elasticsearch/elasticsearch.yml
Replace <__KEY_NAME__> with VALUE that you specify as part of your --auto-fill-jvm arguments in the JVM config file located in: /etc/elasticsearch/jvm.options
Enabling SSL
In order to use Elasticsearch with SSL we use the ReadonlyREST plugin.
A complete setup will require the following steps:
We use the generate-key-stores
script in order to create the Java Keystore that ReadonlyREST will use to store certificates.
Under the hood, we are using keytool to create the Key Store and Trust Store and openssl to sign the certificate.
Note: It is important to generate the Java Key Store first because we will then include this keystore in the AMI/Docker Image
that we create in the next step. See this Packer Template for a concrete
example of how we are including the Java Key Store in the AMI/Docker Image.
Create ReadonlyREST config file
You will need to create a file called readonlyrest.yml. It should look something like this:
readonlyrest:
ssl:
# put the keystore in the same dir with elasticsearch.yml
keystore_file: "elasticsearch.server.keystore.jks"
keystore_pass: <__KEYSTORE_PASS__>
key_pass: <__KEY_PASS__>
key_alias: localhost
See this Packer Template for a concrete
example of where we create this file and how we then make sure this file is included in the AMI/Docker Image we create.
Note: You should not hard code Java Key Store passwords or key passwords in this (or any) config file!
Instead, encrypt the password using something like KMS or Vault and at run time (e.g., in User Data)
decrypt it and use the --auto-fill-ror param of run-elasticsearch script to fill it into this file,
which can only be accessed by the root or the elasticsearch user.
Complete documentation for the readonlyrest.yml can be found
here
Creating Elasticsearch AMI with ReadonlyREST plugin
The first thing we will need to do is build an AMI that has Elasticsearch installed and also has the ReadonlyREST plugin installed.
See this example that goes through generating either an Elasticsearch docker image or AMI
with ReadonlyREST plugin installed.
Note: There's currently no way to easily download the plugin from https://readonlyrest.com as the site
requires you to submit your email in order to get a custom download link. We are currently hosting one particular version of the plugin
that we are using in our examples. If you need a different/new version then you will have to download it and host it locally/provide it to the
elasticsearch-install module.
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"config.yml","path":".circleci/config.yml","sha":"c2cff2a8764e0fabad7a8408ba9e42113cc23487"}]},{"name":".gitignore","path":".gitignore","sha":"f2438833ea6b1bc45e125b598cb1e08cbd274e1b"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"cef671f8c0aee235f6d8cf886e421cfab3b51729"},{"name":"CONTRIBUTING.md","path":"CONTRIBUTING.md","sha":"492bef006a63f2c3189fd171116eb3f900c6cd02"},{"name":"LICENSE.md","path":"LICENSE.md","sha":"a2cf01ecdd725fddd718ab91c80c115882c94f3c"},{"name":"README.md","path":"README.md","sha":"a528d869a6d4e60a4639921169d18041b0ce1a1e"},{"name":"_docs","children":[{"name":"elk-architecture.png","path":"_docs/elk-architecture.png","sha":"a7585c6f40eb0a7740b34f00d412bc2e37373266"}]},{"name":"examples","children":[{"name":"elasticsearch-docker","children":[{"name":"local-mocks","children":[{"name":"entrypoint.sh","path":"examples/elasticsearch-docker/local-mocks/entrypoint.sh","sha":"8e298671493441b6d7548f619a6155a408677d99"}]},{"name":"non-ssl","children":[{"name":".env","path":"examples/elasticsearch-docker/non-ssl/.env","sha":"bc82d24eea229df2ba77f0fdf1f6e1c6f6a8e532"},{"name":"README.md","path":"examples/elasticsearch-docker/non-ssl/README.md","sha":"ba7a1c035faf9791430ace13275d2d6b9b616b75"},{"name":"docker-compose.yml","path":"examples/elasticsearch-docker/non-ssl/docker-compose.yml","sha":"e7547917f574679857f800b24f86bd573671931e"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/elasticsearch-docker/non-ssl/user-data/user-data.sh","sha":"81df178b3080a40a9e03ff7e94f1ef5db7d57f98"}]}]},{"name":"ssl","children":[{"name":".env","path":"examples/elasticsearch-docker/ssl/.env","sha":"bc82d24eea229df2ba77f0fdf1f6e1c6f6a8e532"},{"name":"README.md","path":"examples/elasticsearch-docker/ssl/README.md","sha":"ba7a1c035faf9791430ace13275d2d6b9b616b75"},{"name":"docker-compose.yml","path":"examples/elasticsearch-docker/ssl/docker-compose.yml","sha":"25e2b036c7e672a8ffc4b9559ab53f8d77441f99"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/elasticsearch-docker/ssl/user-data/user-data.sh","sha":"fc95c5a93069b251aca3ec2ae70a2be90231fcd9"}]}]}]},{"name":"elasticsearch-only-cluster","children":[{"name":"README.md","path":"examples/elasticsearch-only-cluster/README.md","sha":"5bdbc120e354d55895febb193390b8ba4440e9ac"},{"name":"main.tf","path":"examples/elasticsearch-only-cluster/main.tf","sha":"491b09a2683ba577be29cd1ca226c3b75ed55183"},{"name":"outputs.tf","path":"examples/elasticsearch-only-cluster/outputs.tf","sha":"989bd7f7dfdb7fa4356d9926731f2849195ed345"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/elasticsearch-only-cluster/user-data/user-data.sh","sha":"55678f9678e93453686c629e364f82ebacdb83df"}]},{"name":"vars.tf","path":"examples/elasticsearch-only-cluster/vars.tf","sha":"6c3db8f2b043c53d04f7278a076230b4ce06308a"}]},{"name":"elk-amis","children":[{"name":"README.md","path":"examples/elk-amis/README.md","sha":"4e58b3a2e6be086188621bee400ff750f02436e8"},{"name":"all-in-one","children":[{"name":"README.md","path":"examples/elk-amis/all-in-one/README.md","sha":"42799bc50b4f41eb9e27ed79f1b5624b812b81b2"},{"name":"all-in-one.json","path":"examples/elk-amis/all-in-one/all-in-one.json","sha":"e0b06562c958eae8d91c371c9ebd757446ed79cd"}]},{"name":"app-server","children":[{"name":"README.md","path":"examples/elk-amis/app-server/README.md","sha":"577966ff3d1aa65d69aa43d6b7e1e86968099335"},{"name":"app-server.json","path":"examples/elk-amis/app-server/app-server.json","sha":"d9f6b7c6009f24c60fe210b77f580b58b22562aa"}]},{"name":"collectd","children":[{"name":"README.md","path":"examples/elk-amis/collectd/README.md","sha":"a9fa0d44e853022868368cab446e4aaeb24997ab"},{"name":"collectd-install-steps.sh","path":"examples/elk-amis/collectd/collectd-install-steps.sh","sha":"1fa3c68c3a1318caab275a7eb4bbaf9533b75319"},{"name":"collectd.json","path":"examples/elk-amis/collectd/collectd.json","sha":"b3f883f1cfd1866f08f411ac302ba5db7bf2fde6"},{"name":"config","children":[{"name":"collectd-ssl.conf","path":"examples/elk-amis/collectd/config/collectd-ssl.conf","sha":"ca0469d729e643644f90118e5a639671a3f60d08"},{"name":"collectd.conf","path":"examples/elk-amis/collectd/config/collectd.conf","sha":"74c1b46937039070cd95f64269eef63aabd72805"}]}]},{"name":"elastalert","children":[{"name":"README.md","path":"examples/elk-amis/elastalert/README.md","sha":"9dd33dfd29ddafa1040213466560ffcf8a66944d"},{"name":"aws","children":[{"name":"elastalert-config","children":[{"name":"config-ssl.yml","path":"examples/elk-amis/elastalert/aws/elastalert-config/config-ssl.yml","sha":"f26f8404e40736c5b0b17bc23e6a5de119d0c80f"},{"name":"config.yml","path":"examples/elk-amis/elastalert/aws/elastalert-config/config.yml","sha":"6054d91b8e86df9102fd5fd1fa2c545f7a981d31"}]}]},{"name":"docker","children":[{"name":"elastalert-config","children":[{"name":"config-ssl.yml","path":"examples/elk-amis/elastalert/docker/elastalert-config/config-ssl.yml","sha":"f26f8404e40736c5b0b17bc23e6a5de119d0c80f"},{"name":"config.yml","path":"examples/elk-amis/elastalert/docker/elastalert-config/config.yml","sha":"6054d91b8e86df9102fd5fd1fa2c545f7a981d31"}]}]},{"name":"elastalert-install-steps.sh","path":"examples/elk-amis/elastalert/elastalert-install-steps.sh","sha":"0e5cca36e97e25543d28d2d4d346b80110ceef8d"},{"name":"elastalert-rules","children":[{"name":"example_change.yml","path":"examples/elk-amis/elastalert/elastalert-rules/example_change.yml","sha":"4020886483786ff9a8847159b5750014c5d2b0fb"}]},{"name":"elastalert.json","path":"examples/elk-amis/elastalert/elastalert.json","sha":"419a67dc9d900c05d480b60452dba7aea7a3ee92"}]},{"name":"elasticsearch","children":[{"name":"README.md","path":"examples/elk-amis/elasticsearch/README.md","sha":"f77a938e72424815565ffe16b9ae7787cb430cc7"},{"name":"aws","children":[{"name":"config","children":[{"name":"elasticsearch-ssl.yml","path":"examples/elk-amis/elasticsearch/aws/config/elasticsearch-ssl.yml","sha":"aab4150ebef375ddb962e17c679642f728df538d"},{"name":"elasticsearch.yml","path":"examples/elk-amis/elasticsearch/aws/config/elasticsearch.yml","sha":"0bf97273dce0664ca96ef87151f856162b1a2411"},{"name":"jvm.options","path":"examples/elk-amis/elasticsearch/aws/config/jvm.options","sha":"28dcf519ee339aba5dd15059c1acc4b97846d6ce"}]}]},{"name":"docker","children":[{"name":"config","children":[{"name":"elasticsearch-ssl.yml","path":"examples/elk-amis/elasticsearch/docker/config/elasticsearch-ssl.yml","sha":"705af52bc1e914df93faa919de8a23b973184938"},{"name":"elasticsearch.yml","path":"examples/elk-amis/elasticsearch/docker/config/elasticsearch.yml","sha":"2ac38b9c875dd6352b7a3d7d0fc2c02662d0e9cb"},{"name":"jvm.options","path":"examples/elk-amis/elasticsearch/docker/config/jvm.options","sha":"28dcf519ee339aba5dd15059c1acc4b97846d6ce"}]}]},{"name":"elasticsearch-install-steps.sh","path":"examples/elk-amis/elasticsearch/elasticsearch-install-steps.sh","sha":"7d766638a6a8368b56ae5b81c65f3d202c907811"},{"name":"elasticsearch.json","path":"examples/elk-amis/elasticsearch/elasticsearch.json","sha":"9c91629c2ba2b14d7647bf380532e442343228e9"},{"name":"plugins","children":[{"name":"readonlyrest-1.18.5_es6.8.3.zip","path":"examples/elk-amis/elasticsearch/plugins/readonlyrest-1.18.5_es6.8.3.zip","sha":"92de49ac4f2a7e90b58dd0d04effe068483a5002"}]},{"name":"readonlyrest-config","children":[{"name":"readonlyrest.yml","path":"examples/elk-amis/elasticsearch/readonlyrest-config/readonlyrest.yml","sha":"a5efa565eca562bcc9a33b959c668e5fd7087165"}]}]},{"name":"filebeat","children":[{"name":"README.md","path":"examples/elk-amis/filebeat/README.md","sha":"6b4f4ff0ec66a8031cf4d5dcd86c26cb7b19bb50"},{"name":"config","children":[{"name":"filebeat-ssl.yml","path":"examples/elk-amis/filebeat/config/filebeat-ssl.yml","sha":"24b163e671b4cc49af08726867d8869f105a8ecb"},{"name":"filebeat.yml","path":"examples/elk-amis/filebeat/config/filebeat.yml","sha":"ae4641ee77c89e389728836fca3cac1428214fe2"}]},{"name":"filebeat-install-steps.sh","path":"examples/elk-amis/filebeat/filebeat-install-steps.sh","sha":"71192c99d9e5a1207a7a0f6f0dfa107532a21fe0"},{"name":"filebeat.json","path":"examples/elk-amis/filebeat/filebeat.json","sha":"3d87c3b35089e41f432ff160593e8e2f7ad0854f"}]},{"name":"kibana","children":[{"name":"README.md","path":"examples/elk-amis/kibana/README.md","sha":"b8af363345ba445cb9fbc2b04ec0e7d6b0b08149"},{"name":"config","children":[{"name":"kibana-ssl.yml","path":"examples/elk-amis/kibana/config/kibana-ssl.yml","sha":"9d0761e628a703eec051b31e6cce9d39e62f5ade"},{"name":"kibana.yml","path":"examples/elk-amis/kibana/config/kibana.yml","sha":"852b146720a4ab3fc1380beacce6f77851017b51"}]},{"name":"kibana-install-steps.sh","path":"examples/elk-amis/kibana/kibana-install-steps.sh","sha":"1dc5d1dd094d0533947c53da510cb5d2139556bf"},{"name":"kibana.json","path":"examples/elk-amis/kibana/kibana.json","sha":"568c21c4d7c3eb056710155a3ab50a12b4d16378"}]},{"name":"logstash","children":[{"name":"README.md","path":"examples/elk-amis/logstash/README.md","sha":"227bc8cdc3f55c5ae071bb3c2b5b544aacd1dcc5"},{"name":"config","children":[{"name":"jvm.options","path":"examples/elk-amis/logstash/config/jvm.options","sha":"6017dd218501163070f950077ce6a1e9cff8f3a4"},{"name":"logstash-ssl.yml","path":"examples/elk-amis/logstash/config/logstash-ssl.yml","sha":"46e72285a2074ceba2313f726db5eb0db6dfa414"},{"name":"logstash.yml","path":"examples/elk-amis/logstash/config/logstash.yml","sha":"46e72285a2074ceba2313f726db5eb0db6dfa414"},{"name":"pipeline-ssl.conf","path":"examples/elk-amis/logstash/config/pipeline-ssl.conf","sha":"10c09944749f51350709b628eda2f84e8345e4c5"},{"name":"pipeline.conf","path":"examples/elk-amis/logstash/config/pipeline.conf","sha":"4a68c41fc5e86106915cb1c91ec39bd4c0cd2b55"}]},{"name":"logstash-install-steps.sh","path":"examples/elk-amis/logstash/logstash-install-steps.sh","sha":"7fd979a06f44c76a70745fd014895ead31f6b693"},{"name":"logstash.json","path":"examples/elk-amis/logstash/logstash.json","sha":"558130f7b83282583a5e32ce1b0dfe820f2caf18"}]}]},{"name":"elk-multi-cluster","children":[{"name":"README.md","path":"examples/elk-multi-cluster/README.md","sha":"ca696b69e95a04bbee5ea51c1b411131d0a47b33"},{"name":"main.tf","path":"examples/elk-multi-cluster/main.tf","sha":"bea044afb4be911d51617264221f5a5cb46d449c"},{"name":"outputs.tf","path":"examples/elk-multi-cluster/outputs.tf","sha":"e0516b88a1243a674feadd292225318a2d865f05"},{"name":"user-data","children":[{"name":"app-server","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/app-server/user-data.sh","sha":"b9361ec30397e7e392a53af77ba7b8d1b50f451c"}]},{"name":"elastalert","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/elastalert/user-data.sh","sha":"48181b3b07a3aec5e5b808b4101a9ef447eed37b"}]},{"name":"elasticsearch","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/elasticsearch/user-data.sh","sha":"55678f9678e93453686c629e364f82ebacdb83df"}]},{"name":"kibana","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/kibana/user-data.sh","sha":"cb5337f5e4c5a8029554a1af33982f963e4b741b"}]},{"name":"logstash","children":[{"name":"user-data.sh","path":"examples/elk-multi-cluster/user-data/logstash/user-data.sh","sha":"4a74a7ca4c180c4e53fa03e8d1d56184adedfff1"}]}]},{"name":"vars.tf","path":"examples/elk-multi-cluster/vars.tf","sha":"4e28eb820aa3fbbd19ce8958361e9a039a155439"}]},{"name":"elk-single-cluster","children":[{"name":"README.md","path":"examples/elk-single-cluster/README.md","sha":"f0842ece87a00338900f35f17b199604d4e46e0c"},{"name":"main.tf","path":"examples/elk-single-cluster/main.tf","sha":"5b6c0e2bfd58c7db42912edfb8eca7b95bb6dafa"},{"name":"outputs.tf","path":"examples/elk-single-cluster/outputs.tf","sha":"c12a0b236e6fcbf5881cb60276d62a432dadb87a"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/elk-single-cluster/user-data/user-data.sh","sha":"936dd8fb14577667d5aa4f1625a26ad7debf0ef5"}]},{"name":"vars.tf","path":"examples/elk-single-cluster/vars.tf","sha":"a67bfbaabbca1a1f25865570d9cc1a70d369485b"}]}]},{"name":"modules","children":[{"name":"auto-discovery","children":[{"name":"README.md","path":"modules/auto-discovery/README.md","sha":"f4de11c428f8fefd962220e7fbc4b976b4f838fe"},{"name":"bin","children":[{"name":"auto-discovery","path":"modules/auto-discovery/bin/auto-discovery","sha":"b8d119c933db64ee2fb5f960bb398316c53bf0eb"}]},{"name":"install.sh","path":"modules/auto-discovery/install.sh","sha":"99a2f2fb2cbfaa31e752cbcc6495e5b755f034f2"}]},{"name":"beats-iam-policies","children":[{"name":"main.tf","path":"modules/beats-iam-policies/main.tf","sha":"2b3b4462aa4b3ed81068f4cea15e686d5a4b322c"},{"name":"vars.tf","path":"modules/beats-iam-policies/vars.tf","sha":"52873644e20b4cafdcc0f33dc11db99f1fa9b586"}]},{"name":"elastalert-iam-policies","children":[{"name":"README.md","path":"modules/elastalert-iam-policies/README.md","sha":"4dc09bec085bc821efa8052578b54cc4c321ea67"},{"name":"main.tf","path":"modules/elastalert-iam-policies/main.tf","sha":"989f18bad613464d4086df3305087650cf3a18bb"},{"name":"variables.tf","path":"modules/elastalert-iam-policies/variables.tf","sha":"bc55d045eba3b8df1b08f78a4657cae25ee0ea73"}]},{"name":"elastalert-security-group-rules","children":[{"name":"README.md","path":"modules/elastalert-security-group-rules/README.md","sha":"dc33e8391d9958ef1915f6a4e2104df391e2d320"},{"name":"main.tf","path":"modules/elastalert-security-group-rules/main.tf","sha":"519b25d67eb73a1587b8ad0556bba585230d1440"},{"name":"variables.tf","path":"modules/elastalert-security-group-rules/variables.tf","sha":"0d87d8de7864335c546406e1047a680f14e6ec73"}]},{"name":"elastalert","children":[{"name":"README.md","path":"modules/elastalert/README.md","sha":"8ee7b607def1754891d8ca65dde5a22552787ebb"},{"name":"main.tf","path":"modules/elastalert/main.tf","sha":"18ac92569874242356c83986ebab5f36df6f62bc"},{"name":"outputs.tf","path":"modules/elastalert/outputs.tf","sha":"aa5817acc77a75b75f4ea4acc686332c5b865618"},{"name":"vars.tf","path":"modules/elastalert/vars.tf","sha":"b51689b7b7dd8df9def896e3fb0ba1ad239a7fcc"}]},{"name":"elasticsearch-cluster-backup","children":[{"name":"README.md","path":"modules/elasticsearch-cluster-backup/README.md","sha":"6b3689afcf27cc5cde7a4048e125e97c7d82189b"},{"name":"backup","children":[{"name":"index.js","path":"modules/elasticsearch-cluster-backup/backup/index.js","sha":"6d3ff6a2a58d70dccd55399ba8bf0e0270140279"}]},{"name":"main.tf","path":"modules/elasticsearch-cluster-backup/main.tf","sha":"f0baf650b3eca0ed959b51eb2aba759da30b1ef0"},{"name":"notification","children":[{"name":"index.js","path":"modules/elasticsearch-cluster-backup/notification/index.js","sha":"8e92e6b387974c4f0c6f007b8e3555d64e984202"}]},{"name":"outputs.tf","path":"modules/elasticsearch-cluster-backup/outputs.tf","sha":"ce7c4ee206984e74253ad6bb0bf25cd4a3965053"},{"name":"vars.tf","path":"modules/elasticsearch-cluster-backup/vars.tf","sha":"9c742c1e44a3c1c9ea7f7883a6da8a56dca3a217"}]},{"name":"elasticsearch-cluster-restore","children":[{"name":"README.md","path":"modules/elasticsearch-cluster-restore/README.md","sha":"11f4f63105657b323a60828ff155c22aaa0ebfbf"},{"name":"main.tf","path":"modules/elasticsearch-cluster-restore/main.tf","sha":"d74982003b28e2dca1ef09d48a1acb612031b63b"},{"name":"notification","children":[{"name":"index.js","path":"modules/elasticsearch-cluster-restore/notification/index.js","sha":"4e441662f92edf384c0c92a96ea5e1a7e1b0b24d"}]},{"name":"outputs.tf","path":"modules/elasticsearch-cluster-restore/outputs.tf","sha":"92454e1cf45fcc57f60c09eb8b31064efb460726"},{"name":"restore","children":[{"name":"index.js","path":"modules/elasticsearch-cluster-restore/restore/index.js","sha":"7d84ea278f01b64310fbc33d18c86a2a73b60573"}]},{"name":"vars.tf","path":"modules/elasticsearch-cluster-restore/vars.tf","sha":"2a3504c6b8f86e28a1659b336796bee04f48e2fa"}]},{"name":"elasticsearch-cluster","children":[{"name":"README.md","path":"modules/elasticsearch-cluster/README.md","sha":"f0dc0ff8093bd99055e2c6753e631337e0df2ec9"},{"name":"main.tf","path":"modules/elasticsearch-cluster/main.tf","sha":"6cce0fa70166851bf8da3af72a04fa38f5f5b989"},{"name":"outputs.tf","path":"modules/elasticsearch-cluster/outputs.tf","sha":"0231e24415e13f62f29d8abefe0398ddb6d0fa04"},{"name":"variables.tf","path":"modules/elasticsearch-cluster/variables.tf","sha":"8b680c1329382792bee8415ff36bd1314675a4c9"}]},{"name":"elasticsearch-iam-policies","children":[{"name":"README.md","path":"modules/elasticsearch-iam-policies/README.md","sha":"57a7117828cff0d9d5bdf7c9fe3af050e55e4aa0"},{"name":"main.tf","path":"modules/elasticsearch-iam-policies/main.tf","sha":"cdb51d7b73acb7e3689e0fa5ab554abc8d576081"},{"name":"variables.tf","path":"modules/elasticsearch-iam-policies/variables.tf","sha":"c612a588acea6049cfdb930209854e253b549729"}]},{"name":"elasticsearch-security-group-rules","children":[{"name":"README.md","path":"modules/elasticsearch-security-group-rules/README.md","sha":"5f0b25abeaa80973e4839626ac96ff306291e759"},{"name":"main.tf","path":"modules/elasticsearch-security-group-rules/main.tf","sha":"fd06486758030a63372610631b3f88d25b7f0cd9"},{"name":"outputs.tf","path":"modules/elasticsearch-security-group-rules/outputs.tf","sha":"e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"},{"name":"variables.tf","path":"modules/elasticsearch-security-group-rules/variables.tf","sha":"7f055aa865ca315861e7a69f2d7741ab7883fddf"}]},{"name":"install-collectd","children":[{"name":"README.md","path":"modules/install-collectd/README.md","sha":"dea15e26259636922a06968d852635faa3d05024"},{"name":"install.sh","path":"modules/install-collectd/install.sh","sha":"4090c1e8b94dd1a4d30ce68da03b73c281e53a3a"}]},{"name":"install-elastalert","children":[{"name":"README.md","path":"modules/install-elastalert/README.md","sha":"8ff13a7658035994828103785d70cf4071b41e15"},{"name":"install.sh","path":"modules/install-elastalert/install.sh","sha":"99f9f170e2c9f3fc31a0f5a92a6a83da5c2fe431"}]},{"name":"install-elasticsearch","children":[{"name":"README.md","path":"modules/install-elasticsearch/README.md","sha":"ff4e35e43bb505354640bca43a12ef4e6da886f0"},{"name":"install.sh","path":"modules/install-elasticsearch/install.sh","sha":"54db7090509f2e7eb56659de74480e2f20422870"}]},{"name":"install-filebeat","children":[{"name":"README.md","path":"modules/install-filebeat/README.md","sha":"d9e46730117d2a71b4646f35267327270a7c5d56"},{"name":"install.sh","path":"modules/install-filebeat/install.sh","sha":"b38f01c2caff489023c1f9a29fa8ea062e17ee58"}]},{"name":"install-kibana","children":[{"name":"README.md","path":"modules/install-kibana/README.md","sha":"9489572678aabb69b2921e7a8b6be320b9ad1b5d"},{"name":"install.sh","path":"modules/install-kibana/install.sh","sha":"2109aefeccc5c1028a881da3bf9b5fa36c62f6d8"}]},{"name":"install-logstash","children":[{"name":"README.md","path":"modules/install-logstash/README.md","sha":"522ce13991cbac0f0d72073e913a7c81aabaf3a1"},{"name":"install.sh","path":"modules/install-logstash/install.sh","sha":"a6a44552a2ec903558d494bf4151f32f3374f786"}]},{"name":"kibana-cluster","children":[{"name":"README.md","path":"modules/kibana-cluster/README.md","sha":"6f95050e9979037eb28177d7aebd38ff908b385f"},{"name":"main.tf","path":"modules/kibana-cluster/main.tf","sha":"7f9a9ecd3b4b4aaf8efcf92108fc123f4fb7ee5d"},{"name":"outputs.tf","path":"modules/kibana-cluster/outputs.tf","sha":"f32e707371481d34b2b84d4ef7f205afaa04f2d4"},{"name":"variables.tf","path":"modules/kibana-cluster/variables.tf","sha":"8306891e82fd960d9587851d3e44c6c1579685c3"}]},{"name":"kibana-security-group-rules","children":[{"name":"README.md","path":"modules/kibana-security-group-rules/README.md","sha":"c35fc72e41001065470a412a0ad9756eb2e84310"},{"name":"main.tf","path":"modules/kibana-security-group-rules/main.tf","sha":"59a0df1a9f5ceb464efa6ee73b39ecaf11240c2c"},{"name":"outputs.tf","path":"modules/kibana-security-group-rules/outputs.tf","sha":"e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"},{"name":"variables.tf","path":"modules/kibana-security-group-rules/variables.tf","sha":"b89d5c7f5f1d1b0e9b5e1d239a1dec9a4ea34486"}]},{"name":"load-balancer-alb-target-group","children":[{"name":"README.md","path":"modules/load-balancer-alb-target-group/README.md","sha":"5918b13e284a75e76bc34792789a44150bf0a12f"},{"name":"main.tf","path":"modules/load-balancer-alb-target-group/main.tf","sha":"7cb9fede7e1bb3c06907f90a81c64b02d78900f4"},{"name":"outputs.tf","path":"modules/load-balancer-alb-target-group/outputs.tf","sha":"cdf904445cf5099ccea5cc8820d198a1ffba4283"},{"name":"variables.tf","path":"modules/load-balancer-alb-target-group/variables.tf","sha":"4a262b0abde2721f9e3a5edeb6a98046009b9df3"}]},{"name":"logstash-cluster","children":[{"name":"README.md","path":"modules/logstash-cluster/README.md","sha":"dbc17e8fc3c4d9a61661ca59b934295ce0bcc6a5"},{"name":"main.tf","path":"modules/logstash-cluster/main.tf","sha":"f138bbca6e0bc6858f8d323f134e95b71031747d"},{"name":"outputs.tf","path":"modules/logstash-cluster/outputs.tf","sha":"218a4f0042d52ea1673de0c4f78857a4831143a6"},{"name":"vars.tf","path":"modules/logstash-cluster/vars.tf","sha":"bff728633e0f853502c3a15b4f9311b5b9d2d41f"}]},{"name":"logstash-iam-policies","children":[{"name":"README.md","path":"modules/logstash-iam-policies/README.md","sha":"3714f39713885c9698daa2973a7891d0a6fb982f"},{"name":"main.tf","path":"modules/logstash-iam-policies/main.tf","sha":"7b553a45925c6ea587b95e8de3d05f546d03e278"},{"name":"vars.tf","path":"modules/logstash-iam-policies/vars.tf","sha":"865c74386084c6eff1f46cb103aa847a6ad0da68"}]},{"name":"logstash-security-group-rules","children":[{"name":"README.md","path":"modules/logstash-security-group-rules/README.md","sha":"e96766f09754cf166f53debb653ea278613a9c8e"},{"name":"main.tf","path":"modules/logstash-security-group-rules/main.tf","sha":"5ecab3d1c9d60cf246a98f9e36296bc518caa9bb"},{"name":"outputs.tf","path":"modules/logstash-security-group-rules/outputs.tf","sha":"3641c561d0e23ef02fa83319ddafa8d1f9f813d2"},{"name":"vars.tf","path":"modules/logstash-security-group-rules/vars.tf","sha":"9e26c219ddad783caf509581fae84953eae60fe9"}]},{"name":"run-collectd","children":[{"name":"README.md","path":"modules/run-collectd/README.md","sha":"783453e215c5907681e77ab4b48e64cad42ab989"},{"name":"bin","children":[{"name":"run-collectd","path":"modules/run-collectd/bin/run-collectd","sha":"7f7971d8b564a84a251d4c253c42f9025b5a1cf7"}]},{"name":"install.sh","path":"modules/run-collectd/install.sh","sha":"d2d8126f6fc5ba95a4e12242aa6ffe9bdc945ba6"}]},{"name":"run-elastalert","children":[{"name":"README.md","path":"modules/run-elastalert/README.md","sha":"e52cf808ab755b2d56a392ee8f525a0651963472"},{"name":"bin","children":[{"name":"run-elastalert","path":"modules/run-elastalert/bin/run-elastalert","sha":"cccab0f5cf41600c1b8ad48da5173db97965440f"}]},{"name":"install.sh","path":"modules/run-elastalert/install.sh","sha":"308f895a2366fc0cd8e6669229f6928a3d888f13"}]},{"name":"run-elasticsearch","children":[{"name":"README.md","path":"modules/run-elasticsearch/README.md","sha":"aed66a5572147d69f0633d20ba3131fd1e19f7c3","toggled":true},{"name":"bin","children":[{"name":"run-elasticsearch","path":"modules/run-elasticsearch/bin/run-elasticsearch","sha":"e8e21b99ca07e03ce807ad537fbd9c3858912713"}]},{"name":"install.sh","path":"modules/run-elasticsearch/install.sh","sha":"c66e6da3d46195ac94d68eed18ed301ca7e0eb51"}],"toggled":true},{"name":"run-filebeat","children":[{"name":"README.md","path":"modules/run-filebeat/README.md","sha":"9ce2f4e4cba993bd49967d00338c985e2b4a188a"},{"name":"bin","children":[{"name":"run-filebeat","path":"modules/run-filebeat/bin/run-filebeat","sha":"61451fdb984b815a6747857d46e10c21b3e22939"}]},{"name":"install.sh","path":"modules/run-filebeat/install.sh","sha":"cd890712803c3961ada86acf1e13b4c299402fb6"}]},{"name":"run-kibana","children":[{"name":"README.md","path":"modules/run-kibana/README.md","sha":"57d636cd26e4af0b51364b4ffb70b382f76314e4"},{"name":"bin","children":[{"name":"run-kibana","path":"modules/run-kibana/bin/run-kibana","sha":"0f60bca1be5eebc5594de9e8eab162aa979d4a17"}]},{"name":"install.sh","path":"modules/run-kibana/install.sh","sha":"e23b447832fce06ad32ce654f94a182a521e2e38"}]},{"name":"run-logstash","children":[{"name":"README.md","path":"modules/run-logstash/README.md","sha":"04cd9e728caad40929b74d0f2f1e5d397a90ffe0"},{"name":"bin","children":[{"name":"run-logstash","path":"modules/run-logstash/bin/run-logstash","sha":"60dc8d906d8181e5a94c3ad18b5b27e53d513cf1"}]},{"name":"install.sh","path":"modules/run-logstash/install.sh","sha":"0a30f2adc2d0765b5581274263e12ce2e688b623"}]}],"toggled":true},{"name":"test","children":[{"name":"Gopkg.lock","path":"test/Gopkg.lock","sha":"d726b0d9bf25eae861d500289fb65ec2ad356105"},{"name":"Gopkg.toml","path":"test/Gopkg.toml","sha":"630429bc686d90b570c81062c7b9c0c925c5fdd3"},{"name":"README.md","path":"test/README.md","sha":"819e790d15767c851bfc438f0a1413ed2bde7d3b"},{"name":"elasticsearch_aws_simple_test.go","path":"test/elasticsearch_aws_simple_test.go","sha":"0362c99a0c22300c5723259b48acd75656acf280"},{"name":"elasticsearch_docker_test.go","path":"test/elasticsearch_docker_test.go","sha":"dcccc141f2d210b9f8c1a2df465a82f8396f16eb"},{"name":"elk_aio_test.go","path":"test/elk_aio_test.go","sha":"ef12f5aa440b29507c695c4e95a96df42e07cacf"},{"name":"elk_end_to_end_test.go","path":"test/elk_end_to_end_test.go","sha":"bf2f5c16ebdab4f83abf6cc84153e881c8a06580"},{"name":"notes.md","path":"test/notes.md","sha":"664915d4b948ccd194ffda80854f47c191904e3d"},{"name":"test_helpers.go","path":"test/test_helpers.go","sha":"227ab02ba9ad534bbd9106059f83a450e777cf52"},{"name":"test_helpers_keystore.go","path":"test/test_helpers_keystore.go","sha":"1e189342859a6561ec9e75009ecb2e55dc42fc23"},{"name":"tls_helpers.go","path":"test/tls_helpers.go","sha":"bf18a8f21521a483f1825c23b2d1a16718bf3565"}]}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"run-elasticsearch-script\">Run Elasticsearch Script</h1><div class=\"preview__body--border\"></div><p>This folder contains a script for configuring and running <a href=\"https://www.elastic.co/products/elasticsearch\" class=\"preview__body--description--blue\" target=\"_blank\">Elasticsearch</a></p>\n<p>This script has been tested on the following operating systems:</p>\n<ul>\n<li>Ubuntu 16.04</li>\n<li>Ubuntu 18.04</li>\n<li>Amazon Linux 2</li>\n<li>CentOS 7</li>\n</ul>\n<h2 class=\"preview__body--subtitle\" id=\"quick-start\">Quick start</h2>\n<p>This module depends on <a href=\"/repos/bash-commons\" class=\"preview__body--description--blue\">bash-commons</a>, so you must install that project\nfirst as documented in its README.</p>\n<p>The easiest way to use this module is with the <a href=\"/repos/gruntwork-installer\" class=\"preview__body--description--blue\">Gruntwork Installer</a>:</p>\n<pre>gruntwork-install \\\n -<span class=\"ruby\">-<span class=\"hljs-class\"><span class=\"hljs-keyword\">module</span>-<span class=\"hljs-title\">name</span> \"<span class=\"hljs-title\">run</span>-<span class=\"hljs-title\">elasticsearch</span>\" \\</span>\n</span> -<span class=\"ruby\">-repo <span class=\"hljs-string\">\"https://github.com/gruntwork-io/package-elk\"</span> \\\n</span> -<span class=\"ruby\">-tag <span class=\"hljs-string\">\"<VERSION>\"</span>\n</span></pre>\n<p>Checkout the <a href=\"#open_modal\" class=\"preview__body--description--blue\">releases</a> to find the latest version.</p>\n<h2 class=\"preview__body--subtitle\" id=\"command-line-arguments\">Command line Arguments</h2>\n<p>Run <code>run-elasticsearch --help</code> to see all available arguments.</p>\n<pre> Usage: run-elasticsearch [OPTIONS]\n\n This<span class=\"hljs-built_in\"> script </span>can be used <span class=\"hljs-keyword\">to</span> configure <span class=\"hljs-keyword\">and</span> <span class=\"hljs-builtin-name\">run</span> Elasticsearch.\n \n Optional arguments:\n \n --auto-fill-endpoint VALUE should be aws_region. Value will be converted <span class=\"hljs-keyword\">to</span> aws_endpoint. KEY <span class=\"hljs-keyword\">in</span> Elasticsearch<span class=\"hljs-built_in\"> config </span>file will be replaced with aws_endpoint. Only needed <span class=\"hljs-keyword\">if</span> running with ec2<span class=\"hljs-built_in\"> discovery </span>plugin.\n --auto-fill <span class=\"hljs-attribute\">KEY</span>=VALUE Search the Elasticsearch<span class=\"hljs-built_in\"> config </span>file <span class=\"hljs-keyword\">for</span> KEY <span class=\"hljs-keyword\">and</span> replace it with VALUE. May be repeated.\n --auto-fill-jvm <span class=\"hljs-attribute\">KEY</span>=VALUE Search the Elasticsearch JVM<span class=\"hljs-built_in\"> config </span>file <span class=\"hljs-keyword\">for</span> KEY <span class=\"hljs-keyword\">and</span> replace it with VALUE. May be repeated.\n \n Example:\n \n install.sh\n --auto-fill <span class=\"hljs-string\">\"<__CLUSTER_NAME__>=the-cluster-name\"</span> --auto-fill-jvm <span class=\"hljs-string\">\"<__XMS__>=4g\"</span>\n</pre>\n<h2 class=\"preview__body--subtitle\" id=\"how-it-works\">How it works</h2>\n<p>The <code>run-elasticsearch</code> script:</p>\n<ul>\n<li>Replace <code><__KEY_NAME__></code> with <code>VALUE</code> that you specify as part of your <code>--auto-fill</code> arguments in the Elasticsearch config file located in: <code>/etc/elasticsearch/elasticsearch.yml</code></li>\n<li>Replace <code><__KEY_NAME__></code> with <code>VALUE</code> that you specify as part of your <code>--auto-fill-jvm</code> arguments in the JVM config file located in: <code>/etc/elasticsearch/jvm.options</code></li>\n</ul>\n<h2 class=\"preview__body--subtitle\" id=\"enabling-ssl\">Enabling SSL</h2>\n<p>In order to use Elasticsearch with SSL we use the <a href=\"https://github.com/sscarduzio/elasticsearch-readonlyrest-plugin\" class=\"preview__body--description--blue\" target=\"_blank\">ReadonlyREST</a> plugin.\nA complete setup will require the following steps:</p>\n<ol>\n<li><a href=\"#generating-java-key-store\" class=\"preview__body--description--blue\">Generating Java Keystore</a></li>\n<li><a href=\"#create-readonlyrest-config-file\" class=\"preview__body--description--blue\">Create a <code>readonlyrest.yml</code> config file</a></li>\n<li><a href=\"#creating-elasticsearch-ami-with-readonlyrest-plugin\" class=\"preview__body--description--blue\">Create an Elasticsearch AMI with ReadonlyREST plugin installed</a></li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"generating-java-key-store\">Generating Java Key Store</h3>\n<p>We use the <a href=\"/repos/package-kafka/modules/generate-key-stores#quick-start\" class=\"preview__body--description--blue\">generate-key-stores</a>\nscript in order to create the Java Keystore that ReadonlyREST will use to store certificates.</p>\n<p>Under the hood, we are using keytool to create the Key Store and Trust Store and openssl to sign the certificate.</p>\n<p><em>Note:</em> It is important to generate the Java Key Store first because we will then include this keystore in the AMI/Docker Image\nthat we create in the next step. See <a href=\"/repos/v0.5.0/package-elk/examples/elasticsearch-ami/elasticsearch-ami-ssl.json\" class=\"preview__body--description--blue\">this Packer Template</a> for a concrete\nexample of how we are including the Java Key Store in the AMI/Docker Image.</p>\n<h3 class=\"preview__body--subtitle\" id=\"create-readonly-rest-config-file\">Create ReadonlyREST config file</h3>\n<p>You will need to create a file called <code>readonlyrest.yml</code>. It should look something like this:</p>\n<pre>readonlyrest:\n ssl:\n <span class=\"hljs-comment\"># put the keystore in the same dir with elasticsearch.yml</span>\n keystore_file: <span class=\"hljs-string\">\"elasticsearch.server.keystore.jks\"</span>\n keystore_pass: <__KEYSTORE_PASS__>\n key_pass: <__KEY_PASS__>\n key_alias: localhost\n</pre>\n<p>See <a href=\"/repos/v0.5.0/package-elk/examples/elasticsearch-ami/elasticsearch-ami-ssl.json\" class=\"preview__body--description--blue\">this Packer Template</a> for a concrete\nexample of where we create this file and how we then make sure this file is included in the AMI/Docker Image we create.</p>\n<p><em>Note:</em> You should <em>not</em> hard code Java Key Store passwords or key passwords in this (or any) config file!\nInstead, encrypt the password using something like KMS or Vault and at run time (e.g., in User Data)\ndecrypt it and use the <code>--auto-fill-ror</code> param of <code>run-elasticsearch</code> script to fill it into this file,\nwhich can only be accessed by the root or the <code>elasticsearch</code> user.</p>\n<p>Complete documentation for the <code>readonlyrest.yml</code> can be found\n<a href=\"https://github.com/beshu-tech/readonlyrest-docs/blob/master/elasticsearch.md#overview\" class=\"preview__body--description--blue\" target=\"_blank\">here</a></p>\n<h3 class=\"preview__body--subtitle\" id=\"creating-elasticsearch-ami-with-readonly-rest-plugin\">Creating Elasticsearch AMI with ReadonlyREST plugin</h3>\n<p>The first thing we will need to do is build an AMI that has Elasticsearch installed and also has the ReadonlyREST plugin installed.\nSee <a href=\"/repos/v0.5.0/package-elk/examples/elk-amis/elasticsearch#local-ssl-testing\" class=\"preview__body--description--blue\">this example</a> that goes through generating either an Elasticsearch docker image or AMI\nwith ReadonlyREST plugin installed.</p>\n<p><em>Note:</em> There's currently no way to easily download the plugin from <a href=\"https://readonlyrest.com\" class=\"preview__body--description--blue\" target=\"_blank\">https://readonlyrest.com</a> as the site\nrequires you to submit your email in order to get a custom download link. We are currently hosting one particular version of the plugin\nthat we are using in our examples. If you need a different/new version then you will have to download it and host it locally/provide it to the\n<code>elasticsearch-install</code> module.</p>\n","repoName":"package-elk","repoRef":"v0.5.0","serviceDescriptor":{"serviceName":"Elasticsearch (self-hosted)","serviceRepoName":"package-elk","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws"],"description":"Deploy a self-hosted Elasticsearch cluster. Supports automatic bootstrap, zero-downtime rolling deployment, auto healing, backup, and recovery.","imageUrl":"elk.png","licenseType":"subscriber","technologies":["Terraform","Bash","JavaScript"],"compliance":[],"tags":[""]},"serviceCategoryName":"NoSQL","fileName":"README.md","filePath":"/modules/run-elasticsearch","title":"Repo Browser: Elasticsearch (self-hosted)","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}
.circleci
install.sh
