This folder contains the tests for the modules in this repo.
Running the tests locally
Running Services With Docker
Build AMIs and launching EC2 Instances can take a long time, making local development cycles painfully slow. For that
reason, this repo has enabled Docker-based tests that will launch a service using Docker and docker-compose to support
much faster cycles times. See Terratest Best Practices for
additional detail.
How To Run Docker Tests
To run your "unit tests" with Docker manually:
cd examples/zookeeper-cluster
# Pick one of the followingexportOS_NAME="ubuntu"exportOS_NAME="centos"exportOS_NAME="amazon-linux"
docker-compose up
To run "unit tests" with Docker automatically:
cd test
# To run all Linux distros
go test -run TestUnitZooKeeper*
# To run a single Linux disro
go test -run TestUnitZooKeeperUbuntu
go test -run TestUnitZooKeeperCentos
go test -run TestUnitZooKeeperAmazonLinux
Run the main tests
Now we're ready to run the main tests. Note that we'll tell Terratest to skip building an AMI, and calling terraform apply
by setting the appropriate env vars:
# To run just the Unbuntu test
env SKIP_setup_ami=trueSKIP_setup_deploy=trueSKIP_validate_redeploy=trueSKIP_teardown_terraform=trueSKIP_teardown_test_data=true go test -run TestZooKeeperClusterUbuntu
# To run all tests
env SKIP_setup_ami=trueSKIP_setup_deploy=trueSKIP_validate_redeploy=trueSKIP_teardown_terraform=trueSKIP_teardown_test_data=true go test
Gotchas
When tests are run repeatedly on the same Zookeeper cluster, they cause the following error Connect to ZooKeeper returned an error: zk: node already exists.
I did not invest time to fix this because you can easily launch a new test cluster with docker-compose stop && docker-compose rm,
and it seemed likely to be a test code error rather than a Zookeeper error.
Running Full Tests Locally
Note #1: Many of these tests create real resources in an AWS account. That means they cost money to run, especially
if you don't clean up after yourself. Please be considerate of the resources you create and take extra care to clean
everything up when you're done!
Note #2: Never hit CTRL + C or cancel a build once tests are running or the cleanup tasks won't run!
Note #3: We set -timeout 45m on all tests not because they necessarily take 45 minutes, but because Go has a
default test timeout of 10 minutes, after which it does a SIGQUIT, preventing the tests from properly cleaning up
after themselves. Therefore, we set a timeout of 45 minutes to make sure all tests have enough time to finish and
cleanup.
Add your AWS credentials as environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
For some of the tests, you also need to set the GITHUB_OAUTH_TOKEN environment variable to a valid GitHub
auth token with "repo" access. You can generate one here: https://github.com/settings/tokens
Setup
Download Go dependencies using Godep:
cdtest
dep ensure
Run all the tests
cdtest
go test -v -timeout 45m -parallel 128
Run a specific test
To run a specific test called TestFoo:
cdtest
go test -v -timeout 45m -parallel 128 -run TestFoo
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"config.yml","path":".circleci/config.yml","sha":"fd3d9d38ea65a464451a744754077f19efc584b0"}]},{"name":".gitignore","path":".gitignore","sha":"9de2b4892a4b8331b381f1a81ffeadfd151353f4"},{"name":".pre-commit-config.yaml","path":".pre-commit-config.yaml","sha":"19d10bbef4ca8fa40036edf1ca4f1db464e28a2f"},{"name":".pre-commit-hooks.yaml","path":".pre-commit-hooks.yaml","sha":"92462242c5c4a3aa0f5d5f5b19a81811c71e4b49"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"98ce69cba614ca3100ef584efdddc020568d295b"},{"name":"LICENSE.txt","path":"LICENSE.txt","sha":"689cf10ec98e3297a75bdd9b9fb5da10b7a675f8"},{"name":"README.md","path":"README.md","sha":"940f96053554ddd69d7ed70e5e9f76357e4a57d6"},{"name":"examples","children":[{"name":"install-open-jdk","children":[{"name":"main.tf","path":"examples/install-open-jdk/main.tf","sha":"7cf188bc79577db1f730726a29d9c0261dd73302"},{"name":"outputs.tf","path":"examples/install-open-jdk/outputs.tf","sha":"6d1b9b915cdb10c8ba114d61d0c47f0babd2ebe8"},{"name":"packer","children":[{"name":"build.json","path":"examples/install-open-jdk/packer/build.json","sha":"742d0a47d40515ef5c3c162d9798aecf03986eea"}]},{"name":"vars.tf","path":"examples/install-open-jdk/vars.tf","sha":"d1eba93458ba1cfdf6b588d968e558f756087e9f"}]},{"name":"zookeeper-ami","children":[{"name":"README.md","path":"examples/zookeeper-ami/README.md","sha":"05c95c33022491de1a986d0aca53019913a1f44a"},{"name":"configure-image.sh","path":"examples/zookeeper-ami/configure-image.sh","sha":"908f15f17dad7254fc27d6b3b4f8f3a85c3c1619"},{"name":"docker-compose.yml","path":"examples/zookeeper-ami/docker-compose.yml","sha":"feaa66af7fa5879e41b88d09214f94a8e7dfb767"},{"name":"mock","children":[{"name":"README.md","path":"examples/zookeeper-ami/mock/README.md","sha":"fcb885c49c095432b8c800525bef76903968f2a0"},{"name":"bash-commons","children":[{"name":"aws.sh","path":"examples/zookeeper-ami/mock/bash-commons/aws.sh","sha":"c3d5b6b6c112ad9296f3207404e5aa183a6346e6"}]},{"name":"modules","children":[{"name":"attach-eni","path":"examples/zookeeper-ami/mock/modules/attach-eni","sha":"da052caea4586b27c2dc13e521092e9403fcc327"},{"name":"mount-ebs-volume","path":"examples/zookeeper-ami/mock/modules/mount-ebs-volume","sha":"e0171fef05dbd120dc8668ceeca176cc5c0dfce4"}]},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/zookeeper-ami/mock/user-data/user-data.sh","sha":"43a641aa2937ba6f7c07c8391c947de2f65b7920"}]}]},{"name":"zookeeper.json","path":"examples/zookeeper-ami/zookeeper.json","sha":"b9ef3b6bc03fbf91f8b1fe534cc8700c6eb3f8c6"}]},{"name":"zookeeper-cluster","children":[{"name":"README.md","path":"examples/zookeeper-cluster/README.md","sha":"eda7e80c820a57b327129083f52b73e0713ae6b3"},{"name":"main.tf","path":"examples/zookeeper-cluster/main.tf","sha":"951e5861164f297b7babbe86be32fb6de0504132"},{"name":"outputs.tf","path":"examples/zookeeper-cluster/outputs.tf","sha":"b975f1a72a486b619e310e8140110b84dfc29085"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/zookeeper-cluster/user-data/user-data.sh","sha":"1f29df534a7a71456ea56de596059025a8fcd12d"}]},{"name":"vars.tf","path":"examples/zookeeper-cluster/vars.tf","sha":"ad3d6ba8d303e025fa8afd2a618510deeafd2e20"}]}]},{"name":"modules","children":[{"name":"bash-commons","children":[{"name":"README.md","path":"modules/bash-commons/README.md","sha":"0b7594de5380fe493e62f91965b5f8d283d76c55"},{"name":"install.sh","path":"modules/bash-commons/install.sh","sha":"d531a722d55a8a82d312725aa19ba2c1ddebcf02"},{"name":"lib","children":[{"name":"assert.sh","path":"modules/bash-commons/lib/assert.sh","sha":"46db8af49acb3e8d707447827dc614740add6a63"},{"name":"aws.sh","path":"modules/bash-commons/lib/aws.sh","sha":"3f14d36e2c7b57c91be34bdf30fff4f400a7d1aa"},{"name":"log.sh","path":"modules/bash-commons/lib/log.sh","sha":"66f936873d4d104c94693d3e68790cc0b9d36c63"},{"name":"os.sh","path":"modules/bash-commons/lib/os.sh","sha":"435e592c0e992f2266247a2da797a0eefc429f35"},{"name":"strings.sh","path":"modules/bash-commons/lib/strings.sh","sha":"19dbfe630edfbe71e7c8c1beee3d50fffdd5b1e6"}]}]},{"name":"exhibitor-shared-config","children":[{"name":"README.md","path":"modules/exhibitor-shared-config/README.md","sha":"ec9415af7d44ddfdf4cdfc58b89f4a5df2f0721e"},{"name":"main.tf","path":"modules/exhibitor-shared-config/main.tf","sha":"9637bf1851ec6ec30efd93a681d3d3a0dd7310be"},{"name":"outputs.tf","path":"modules/exhibitor-shared-config/outputs.tf","sha":"6cae4a73a186ba26e732a5d77af9b5d8707759e1"},{"name":"vars.tf","path":"modules/exhibitor-shared-config/vars.tf","sha":"72f9f93e425ee4ec00f3464aa3c0d41be5629775"}]},{"name":"install-exhibitor","children":[{"name":"README.md","path":"modules/install-exhibitor/README.md","sha":"98e9e15f48b87ae76e30ec6150301bb16d7acb3e"},{"name":"install.sh","path":"modules/install-exhibitor/install.sh","sha":"8c56b759a036ff1e8c0130ee644469f4fb11f74e"},{"name":"pom.xml","path":"modules/install-exhibitor/pom.xml","sha":"e5cca292196cdaa093cc484465eb5bc2d48ed618"},{"name":"zookeeper-log4j.properties","path":"modules/install-exhibitor/zookeeper-log4j.properties","sha":"80e3ae28555ed63b5a74d127796ff3f2adfce223"}]},{"name":"install-open-jdk","children":[{"name":"README.md","path":"modules/install-open-jdk/README.md","sha":"a048ab6e2cc74020657a68b7e32584f6ed78f76d"},{"name":"install.sh","path":"modules/install-open-jdk/install.sh","sha":"9982cf730afc5d58b05b888054138ffbb8725795"}]},{"name":"install-oracle-jdk","children":[{"name":"README.md","path":"modules/install-oracle-jdk/README.md","sha":"b46c3ab123f53f6a32ba2026bc18f3d870617bc1"},{"name":"install.sh","path":"modules/install-oracle-jdk/install.sh","sha":"53b798bd0984de5fd27048d3dfcd437d0687c117"}]},{"name":"install-supervisord","children":[{"name":"README.md","path":"modules/install-supervisord/README.md","sha":"eb07bdb5928882c73efcb5a069257f1356bd5533"},{"name":"install.sh","path":"modules/install-supervisord/install.sh","sha":"2b765667b825b94791e715b0dcbcdbd8ec4e95a2"},{"name":"supervisor-initd-script.sh","path":"modules/install-supervisord/supervisor-initd-script.sh","sha":"171b91613e98ab2bd10282025caff1707918c95a"},{"name":"supervisor-systemd-unit.service","path":"modules/install-supervisord/supervisor-systemd-unit.service","sha":"995a1a95863c21abcd245e45313e99b7cd98d98a"},{"name":"supervisord.conf","path":"modules/install-supervisord/supervisord.conf","sha":"d96beb0ca9a16279ed1bdf74cbb6516275d85085"}]},{"name":"install-zookeeper","children":[{"name":"README.md","path":"modules/install-zookeeper/README.md","sha":"9f23978ae6dfd0124fa8a9d8327b6256aa654b2f"},{"name":"install.sh","path":"modules/install-zookeeper/install.sh","sha":"7d25699e7835711dfc4c9e1fa78565fd578ccfb1"},{"name":"security","children":[{"name":"zookeeper.KEYS","path":"modules/install-zookeeper/security/zookeeper.KEYS","sha":"f23552bd9489de0f229bb4bb27dfb6a0376904f4"},{"name":"zookeeper.asc","path":"modules/install-zookeeper/security/zookeeper.asc","sha":"de701fc5a9b18af3095d32de998b95d7f25667ef"}]}]},{"name":"precommit-hooks","children":[{"name":"README.md","path":"modules/precommit-hooks/README.md","sha":"f5fa146415c58421be1653d09d95fc97b65bf5aa"},{"name":"gofmt.sh","path":"modules/precommit-hooks/gofmt.sh","sha":"6df4a06074bfc2eea33864e0c00e282caf4fb213"},{"name":"golint.sh","path":"modules/precommit-hooks/golint.sh","sha":"e74ea4da285fee9d45b4cac6426657cfc504de6d"},{"name":"shellcheck.sh","path":"modules/precommit-hooks/shellcheck.sh","sha":"5e2b2ab86606d80419237cf47f40762baa5f1c0d"},{"name":"terraform-fmt.sh","path":"modules/precommit-hooks/terraform-fmt.sh","sha":"048ec70ba42772675c1e7d2438b531fd572ad59e"}]},{"name":"run-exhibitor","children":[{"name":"README.md","path":"modules/run-exhibitor/README.md","sha":"35608381a0756187d1bf77f1d71775edfff0858a"},{"name":"bin","children":[{"name":"run-exhibitor","path":"modules/run-exhibitor/bin/run-exhibitor","sha":"e7d039d097d060e6b7abcc6eb4622af7a1b010fe"}]},{"name":"install.sh","path":"modules/run-exhibitor/install.sh","sha":"f8f5a2961f42fc23b4a8fe06544a38ad28bac386"}]},{"name":"run-health-checker","children":[{"name":"README.md","path":"modules/run-health-checker/README.md","sha":"2b68f92659844150a106fa2e55972848e6373135"},{"name":"bin","children":[{"name":"check-zookeeper","path":"modules/run-health-checker/bin/check-zookeeper","sha":"c952376818aa1b3c08e6ce54c6b7b44811911696"},{"name":"run-health-checker","path":"modules/run-health-checker/bin/run-health-checker","sha":"0409f239d7078fd1a34d311790b0648e8173b102"}]},{"name":"install.sh","path":"modules/run-health-checker/install.sh","sha":"5241e81822039d9644d2a02f7f10003e64907de4"}]},{"name":"zookeeper-cluster","children":[{"name":"README.md","path":"modules/zookeeper-cluster/README.md","sha":"c604ec0b8326a2496b116da1579996aefa0ac04a"},{"name":"main.tf","path":"modules/zookeeper-cluster/main.tf","sha":"8d9c97dc13125ccaebb296d25c198063ea524db7"},{"name":"outputs.tf","path":"modules/zookeeper-cluster/outputs.tf","sha":"424dada690378a4bd7aa8e903c61ad77746ebf99"},{"name":"vars.tf","path":"modules/zookeeper-cluster/vars.tf","sha":"831792934ee50e50856973450f44e9e8f99518a6"}]},{"name":"zookeeper-iam-permissions","children":[{"name":"README.md","path":"modules/zookeeper-iam-permissions/README.md","sha":"2a8f49855ac55e5bd083e8466ac5a2576fb62878"},{"name":"main.tf","path":"modules/zookeeper-iam-permissions/main.tf","sha":"a78e1cd57e7ccc13d35a3dfec5ad5a2fd1fd7276"},{"name":"vars.tf","path":"modules/zookeeper-iam-permissions/vars.tf","sha":"9c3a5c00d0591fd79c9e63b6c8bf753cc14cda61"}]},{"name":"zookeeper-security-group-rules","children":[{"name":"README.md","path":"modules/zookeeper-security-group-rules/README.md","sha":"c2681bc0df96de4e97ea63c138f625d7bacd2439"},{"name":"main.tf","path":"modules/zookeeper-security-group-rules/main.tf","sha":"512acb4af29f05a4eae2d1ebd74b7377fcf8cffd"},{"name":"vars.tf","path":"modules/zookeeper-security-group-rules/vars.tf","sha":"6319feab9b2eb4407d562436d716e64ef57b17fd"}]}]},{"name":"test","children":[{"name":"Gopkg.lock","path":"test/Gopkg.lock","sha":"fd0e206ff06d8477115df4e530fd4910b57e15de"},{"name":"Gopkg.toml","path":"test/Gopkg.toml","sha":"7bc1f94cd3e197e77bbf369ae74205a4d5f2f3ba"},{"name":"README.md","path":"test/README.md","sha":"67c59943d09b170960acbafeb4a4c78683873a66","toggled":true},{"name":"open_jdk_test.go","path":"test/open_jdk_test.go","sha":"f02e087559c57221223139bc6d66ecb602cf38e8"},{"name":"zookeeper_cluster_test.go","path":"test/zookeeper_cluster_test.go","sha":"d4d9cef0f13e90575dd67fdbf8bd9213070cca71"}],"toggled":true}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"tests\">Tests</h1><div class=\"preview__body--border\"></div><p>This folder contains the tests for the modules in this repo.</p>\n<h2 class=\"preview__body--subtitle\" id=\"running-the-tests-locally\">Running the tests locally</h2>\n<h3 class=\"preview__body--subtitle\" id=\"running-services-with-docker\">Running Services With Docker</h3>\n<p>Build AMIs and launching EC2 Instances can take a long time, making local development cycles painfully slow. For that\nreason, this repo has enabled Docker-based tests that will launch a service using Docker and <code>docker-compose</code> to support\nmuch faster cycles times. See <a href=\"/repos/terratest#best-practices\" class=\"preview__body--description--blue\">Terratest Best Practices</a> for\nadditional detail.</p>\n<h4 id=\"how-to-run-docker-tests\">How To Run Docker Tests</h4>\n<p>To run your "unit tests" with Docker manually:</p>\n<pre>cd examples/zookeeper-cluster\n\n<span class=\"hljs-comment\"># Pick one of the following</span>\n<span class=\"hljs-builtin-name\">export</span> <span class=\"hljs-attribute\">OS_NAME</span>=<span class=\"hljs-string\">\"ubuntu\"</span>\n<span class=\"hljs-builtin-name\">export</span> <span class=\"hljs-attribute\">OS_NAME</span>=<span class=\"hljs-string\">\"centos\"</span>\n<span class=\"hljs-builtin-name\">export</span> <span class=\"hljs-attribute\">OS_NAME</span>=<span class=\"hljs-string\">\"amazon-linux\"</span>\n\ndocker-compose up\n</pre>\n<p>To run "unit tests" with Docker automatically:</p>\n<pre>cd test\n\n<span class=\"hljs-comment\"># To run all Linux distros</span>\ngo test -run TestUnitZooKeeper*\n\n<span class=\"hljs-comment\"># To run a single Linux disro</span>\ngo test -run TestUnitZooKeeperUbuntu\ngo test -run TestUnitZooKeeperCentos\ngo test -run TestUnitZooKeeperAmazonLinux\n</pre>\n<ol>\n<li>\n<p>Run the main tests</p>\n<p>Now we're ready to run the main tests. Note that we'll tell Terratest to skip building an AMI, and calling <code>terraform apply</code>\nby setting the appropriate env vars:</p>\n<pre><span class=\"hljs-comment\"># To run just the Unbuntu test</span>\nenv <span class=\"hljs-attribute\">SKIP_setup_ami</span>=<span class=\"hljs-literal\">true</span> <span class=\"hljs-attribute\">SKIP_setup_deploy</span>=<span class=\"hljs-literal\">true</span> <span class=\"hljs-attribute\">SKIP_validate_redeploy</span>=<span class=\"hljs-literal\">true</span> <span class=\"hljs-attribute\">SKIP_teardown_terraform</span>=<span class=\"hljs-literal\">true</span> <span class=\"hljs-attribute\">SKIP_teardown_test_data</span>=<span class=\"hljs-literal\">true</span> go test -<span class=\"hljs-builtin-name\">run</span> TestZooKeeperClusterUbuntu\n\n<span class=\"hljs-comment\"># To run all tests</span>\nenv <span class=\"hljs-attribute\">SKIP_setup_ami</span>=<span class=\"hljs-literal\">true</span> <span class=\"hljs-attribute\">SKIP_setup_deploy</span>=<span class=\"hljs-literal\">true</span> <span class=\"hljs-attribute\">SKIP_validate_redeploy</span>=<span class=\"hljs-literal\">true</span> <span class=\"hljs-attribute\">SKIP_teardown_terraform</span>=<span class=\"hljs-literal\">true</span> <span class=\"hljs-attribute\">SKIP_teardown_test_data</span>=<span class=\"hljs-literal\">true</span> go test\n</pre>\n</li>\n</ol>\n<h4 id=\"gotchas\">Gotchas</h4>\n<ol>\n<li>When tests are run repeatedly on the same Zookeeper cluster, they cause the following error <code>Connect to ZooKeeper returned an error: zk: node already exists</code>.\nI did not invest time to fix this because you can easily launch a new test cluster with <code>docker-compose stop && docker-compose rm</code>,\nand it seemed likely to be a test code error rather than a Zookeeper error.</li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"running-full-tests-locally\">Running Full Tests Locally</h3>\n<p><strong>Note #1</strong>: Many of these tests create real resources in an AWS account. That means they cost money to run, especially\nif you don't clean up after yourself. Please be considerate of the resources you create and take extra care to clean\neverything up when you're done!</p>\n<p><strong>Note #2</strong>: Never hit <code>CTRL + C</code> or cancel a build once tests are running or the cleanup tasks won't run!</p>\n<p><strong>Note #3</strong>: We set <code>-timeout 45m</code> on all tests not because they necessarily take 45 minutes, but because Go has a\ndefault test timeout of 10 minutes, after which it does a <code>SIGQUIT</code>, preventing the tests from properly cleaning up\nafter themselves. Therefore, we set a timeout of 45 minutes to make sure all tests have enough time to finish and\ncleanup.</p>\n<h2 class=\"preview__body--subtitle\" id=\"prerequisites\">Prerequisites</h2>\n<ul>\n<li>Install the latest version of <a href=\"https://golang.org/\" class=\"preview__body--description--blue\" target=\"_blank\">Go</a>.</li>\n<li>Install <a href=\"https://glide.sh/\" class=\"preview__body--description--blue\" target=\"_blank\">glide</a> for Go dependency management. On OSX, the simplest way to install is\n<code>brew update; brew install glide</code>.</li>\n<li>Install <a href=\"https://www.terraform.io/downloads.html\" class=\"preview__body--description--blue\" target=\"_blank\">Terraform</a>.</li>\n<li>Add your AWS credentials as environment variables: <code>AWS_ACCESS_KEY_ID</code>, <code>AWS_SECRET_ACCESS_KEY</code></li>\n<li>For some of the tests, you also need to set the <code>GITHUB_OAUTH_TOKEN</code> environment variable to a valid GitHub\nauth token with "repo" access. You can generate one here: https://github.com/settings/tokens</li>\n</ul>\n<h3 class=\"preview__body--subtitle\" id=\"setup\">Setup</h3>\n<p>Download Go dependencies using Godep:</p>\n<pre><span class=\"hljs-built_in\">cd</span> <span class=\"hljs-built_in\">test</span>\ndep ensure\n</pre>\n<h3 class=\"preview__body--subtitle\" id=\"run-all-the-tests\">Run all the tests</h3>\n<pre><span class=\"hljs-built_in\">cd</span> <span class=\"hljs-built_in\">test</span>\ngo <span class=\"hljs-built_in\">test</span> -v -timeout 45m -parallel 128\n</pre>\n<h3 class=\"preview__body--subtitle\" id=\"run-a-specific-test\">Run a specific test</h3>\n<p>To run a specific test called <code>TestFoo</code>:</p>\n<pre><span class=\"hljs-keyword\">cd</span> <span class=\"hljs-keyword\">test</span>\ngo <span class=\"hljs-keyword\">test</span> -v -timeout 45m -parallel 128 -<span class=\"hljs-keyword\">run</span> TestFoo\n</pre>\n","repoName":"package-zookeeper","repoRef":"v0.6.6","serviceDescriptor":{"serviceName":"Apache ZooKeeper","serviceRepoName":"package-zookeeper","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws"],"description":"Deploy an Apache ZooKeeper cluster. Supports automatic bootstrap, Exhibitor, zero-downtime rolling deployment, and auto healing.","imageUrl":"zookeeper.png","licenseType":"subscriber","technologies":["Terraform","Bash"],"compliance":[],"tags":[""]},"serviceCategoryName":"NoSQL","fileName":"README.md","filePath":"/test","title":"Repo Browser: Apache ZooKeeper","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}