It builds two AMIs, one on top of Amazon Linux and one on top of Ubuntu. Pick whichever distro you prefer.
This AMI includes examples of how to configure Schema Registry and REST Proxy with SSL support. It expects you to use
the generate-key-stores module to generate a Key Store and Trust Store before building
the AMI.
This AMI only includes Schema Registry and REST Proxy. It does not include ZooKeeper, Kafka, or Kafka Connect, which
are expected to be running in one or more separate clusters, launched from separate AMIs (see the zookeeper-ami
example and kafka-ami example). For an example of all services
co-located in the same cluster, see kafka-zookeeper-confluent-ami.
Run the generate-key-stores module to create a Key Store, Trust Store, and
self-signed SSL certificate for each of Schema Registry, REST Proxy, and the Kafka Connect workers. You can run the
check-for-xxx-key-store.sh scripts to print out example usage for the generate-key-stores.sh script. For additional
information on this step, see [Generating SSL Certificates](#generating-ssl-certificates] below.
Copy the Trust Store you created for the kafka-ami to the confluent-oss-ami/ssl/kafka folder.
We'll use this to enable Schema Registry, REST Proxy, and Kafka Connect to connect via SSL to Kafka.
Set the GITHUB_OAUTH_TOKEN environment variable to a valid GitHub auth token with "repo" access. You can generate
one here: https://github.com/settings/tokens
Run packer build -var "package_kafka_branch=<branch>" -only <target os> confluent-oss.json to create a new AMI in your AWS account. Note down the ID of this new AMI.
The number of possible interactions between all the tools in
the Confluent Platform makes understanding the various SSL certificates needed for a "secure" Confluent setup potentially
confusing. This section aims to clear that up.
Terminology
Key Store: A file that contains SSL certificates a given service presents to others. It contains both a public
certificate that the server presents to clients and a corresponding private key that the server uses to prove that it
is the identity presented by the public certificate. This file is generated by the keytool program that
comes bundled with the JDK. TL;DR: The Key Store contains the certs a given service presents to others.
Trust Store: A file that contains the public certificate of the Certificate Authority (CA) used to sign public
certificates used by other servers. Services use the Trust Store to know which public certificates they trust when
making an SSL connection. This file is generated by the keytool program that
comes bundled with the JDK. TL;DR: The Trust Store contains the certs a given server will trust from others.
Instructions
The steps to generate the right SSL certificate are straightforward. Here are some guidelines to follow:
All SSL certificates should be generated using the generate-key-stores module.
Kafka brokers should receive their own Key Store and Trust STore. Schema Registry, REST Proxy, and Kafka Connect may
share the same Key Store and Trust Store, although some users may wish to create a separate Key Store/Trust Store for
each of these services. Check out the check-for-xxx-key-store.sh scripts in this folder to see additional details
about generating SSL certificates for each service.
Note that you will use the same Kafka broker SSL certificates in both the Kafka broker AMI and other AMIs. That's
because supporting tools like Schema Registry and REST Proxy need the Trust Store file you generated when you created
the Kafka SSL certificates.
Now place all your SSL files in the following folder structure in this Packer template folder:
If you are re-using the same set of SSL files, feel free to copy the same file into different file paths. This makes
it explicit exactly where each Key Store and/or Trust Store file comes from.
Inspecting SSL Certificate Files
If you want to examine an existing SSL certificate file to see what information it contains, run the following command:
openssl x509 -in /path/to/cert/file -text
Note that this will only work on the cert and ca-cert files, not on the Key Store and Trust Store files, which are
unique to Java. To view those, you can use an open source tool like Keystore Explorer.
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"config.yml","path":".circleci/config.yml","sha":"f6a1d049f1988183bd05a90c285533dfcdeaf48a"}]},{"name":".gitignore","path":".gitignore","sha":"e68eece82c5bbdddf63121f38b66cdea255b5567"},{"name":".pre-commit-config.yaml","path":".pre-commit-config.yaml","sha":"8f0a49e6e74c419dd55216b6397d21c6cc2e1029"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"4be01a6334d39aa5bf6abe6baae701f5e2a8c5ac"},{"name":"LICENSE.txt","path":"LICENSE.txt","sha":"689cf10ec98e3297a75bdd9b9fb5da10b7a675f8"},{"name":"README.md","path":"README.md","sha":"3b28f35855c132ef18878116e94b623ef2831d0d"},{"name":"examples","children":[{"name":"confluent-oss-ami","children":[{"name":"README.md","path":"examples/confluent-oss-ami/README.md","sha":"675ce60c000b84facb441e96fa16e89614fe2b6b","toggled":true},{"name":"check-for-kafka-trust-store.sh","path":"examples/confluent-oss-ami/check-for-kafka-trust-store.sh","sha":"e2e7d323e5153471809af13b4434c9edfad1b85b"},{"name":"check-for-key-store.sh","path":"examples/confluent-oss-ami/check-for-key-store.sh","sha":"120539a6d63dfd2ed208e92c0a25a32b6b86a2bd"},{"name":"config","children":[{"name":"README.md","path":"examples/confluent-oss-ami/config/README.md","sha":"50e3000eda90f23880d9f34da435f1b170e213f5"},{"name":"kafka-connect","children":[{"name":"config","children":[{"name":"dev.worker-4.0.x.properties","path":"examples/confluent-oss-ami/config/kafka-connect/config/dev.worker-4.0.x.properties","sha":"5b4760ba1b5805ed9ec7eb0f0be06a3597f15001"},{"name":"prod.worker-4.0.x.properties","path":"examples/confluent-oss-ami/config/kafka-connect/config/prod.worker-4.0.x.properties","sha":"5b4760ba1b5805ed9ec7eb0f0be06a3597f15001"},{"name":"stage.worker-4.0.x.properties","path":"examples/confluent-oss-ami/config/kafka-connect/config/stage.worker-4.0.x.properties","sha":"5b4760ba1b5805ed9ec7eb0f0be06a3597f15001"}]},{"name":"log4j","children":[{"name":"dev.log4j.properties","path":"examples/confluent-oss-ami/config/kafka-connect/log4j/dev.log4j.properties","sha":"a23dfdbf369c5a8cba498d9016ab239f3c1c18a8"},{"name":"prod.log4j.properties","path":"examples/confluent-oss-ami/config/kafka-connect/log4j/prod.log4j.properties","sha":"a23dfdbf369c5a8cba498d9016ab239f3c1c18a8"},{"name":"stage.log4j.properties","path":"examples/confluent-oss-ami/config/kafka-connect/log4j/stage.log4j.properties","sha":"a23dfdbf369c5a8cba498d9016ab239f3c1c18a8"}]}]},{"name":"kafka-rest","children":[{"name":"config","children":[{"name":"dev.kafka-rest-4.0.x.properties","path":"examples/confluent-oss-ami/config/kafka-rest/config/dev.kafka-rest-4.0.x.properties","sha":"29c9ca3bd784637597c683b1585bfd52fd0035db"},{"name":"prod.kafka-rest-4.0.x.properties","path":"examples/confluent-oss-ami/config/kafka-rest/config/prod.kafka-rest-4.0.x.properties","sha":"29c9ca3bd784637597c683b1585bfd52fd0035db"},{"name":"stage.kafka-rest-4.0.x.properties","path":"examples/confluent-oss-ami/config/kafka-rest/config/stage.kafka-rest-4.0.x.properties","sha":"29c9ca3bd784637597c683b1585bfd52fd0035db"}]},{"name":"log4j","children":[{"name":"dev.log4j.properties","path":"examples/confluent-oss-ami/config/kafka-rest/log4j/dev.log4j.properties","sha":"43c18e3a2eb5bdf7a49c0336919aac1acf5f6b6d"},{"name":"prod.log4j.properties","path":"examples/confluent-oss-ami/config/kafka-rest/log4j/prod.log4j.properties","sha":"43c18e3a2eb5bdf7a49c0336919aac1acf5f6b6d"},{"name":"stage.log4j.properties","path":"examples/confluent-oss-ami/config/kafka-rest/log4j/stage.log4j.properties","sha":"43c18e3a2eb5bdf7a49c0336919aac1acf5f6b6d"}]}]},{"name":"schema-registry","children":[{"name":"config","children":[{"name":"dev.schema-registry-4.0.x.properties","path":"examples/confluent-oss-ami/config/schema-registry/config/dev.schema-registry-4.0.x.properties","sha":"e6541005171b9f0de27e7f177f915b08399f9404"},{"name":"prod.schema-registry-4.0.x.properties","path":"examples/confluent-oss-ami/config/schema-registry/config/prod.schema-registry-4.0.x.properties","sha":"e6541005171b9f0de27e7f177f915b08399f9404"},{"name":"stage.schema-registry-4.0.x.properties","path":"examples/confluent-oss-ami/config/schema-registry/config/stage.schema-registry-4.0.x.properties","sha":"e6541005171b9f0de27e7f177f915b08399f9404"}]},{"name":"log4j","children":[{"name":"dev.log4j.properties","path":"examples/confluent-oss-ami/config/schema-registry/log4j/dev.log4j.properties","sha":"28fa60645b6ba0ab402433aebbedec8a8a9533e3"},{"name":"prod.log4j.properties","path":"examples/confluent-oss-ami/config/schema-registry/log4j/prod.log4j.properties","sha":"28fa60645b6ba0ab402433aebbedec8a8a9533e3"},{"name":"stage.log4j.properties","path":"examples/confluent-oss-ami/config/schema-registry/log4j/stage.log4j.properties","sha":"28fa60645b6ba0ab402433aebbedec8a8a9533e3"}]}]}]},{"name":"configure-common-dependencies.sh","path":"examples/confluent-oss-ami/configure-common-dependencies.sh","sha":"a5b7c815d15309c39f3aff9e9ad004a029057de8"},{"name":"configure-kafka-connect.sh","path":"examples/confluent-oss-ami/configure-kafka-connect.sh","sha":"effeed2d32e0ea2878b1d5ad726452e21af72647"},{"name":"configure-kafka-rest.sh","path":"examples/confluent-oss-ami/configure-kafka-rest.sh","sha":"aa9b04bb10946d50a22157a351ba859935b0350c"},{"name":"configure-schema-registry.sh","path":"examples/confluent-oss-ami/configure-schema-registry.sh","sha":"6ecc2c129f17a28e81ff45edade8a57e56bc9e07"},{"name":"confluent-oss.json","path":"examples/confluent-oss-ami/confluent-oss.json","sha":"8d9a8100d0c6823d0e670b0b9b1b32bf43640922"},{"name":"ssl","children":[{"name":"README.md","path":"examples/confluent-oss-ami/ssl/README.md","sha":"2b7b50749a90c78e026f597f677e44ece9f2458d"},{"name":"ca-cert","path":"examples/confluent-oss-ami/ssl/ca-cert","sha":"fb02e172efcdc4ad4c660e137059be86926108f4"},{"name":"cert","path":"examples/confluent-oss-ami/ssl/cert","sha":"0f486b16f80eebe97d9135542d229404a6b48ddc"},{"name":"kafka-connect","children":[{"name":"keystore","children":[{"name":"dev.keystore.jks","path":"examples/confluent-oss-ami/ssl/kafka-connect/keystore/dev.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"},{"name":"prod.keystore.jks","path":"examples/confluent-oss-ami/ssl/kafka-connect/keystore/prod.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"},{"name":"stage.keystore.jks","path":"examples/confluent-oss-ami/ssl/kafka-connect/keystore/stage.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"}]},{"name":"truststore","children":[{"name":"dev.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka-connect/truststore/dev.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"prod.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka-connect/truststore/prod.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"stage.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka-connect/truststore/stage.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"}]}]},{"name":"kafka-rest","children":[{"name":"keystore","children":[{"name":"dev.keystore.jks","path":"examples/confluent-oss-ami/ssl/kafka-rest/keystore/dev.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"},{"name":"prod.keystore.jks","path":"examples/confluent-oss-ami/ssl/kafka-rest/keystore/prod.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"},{"name":"stage.keystore.jks","path":"examples/confluent-oss-ami/ssl/kafka-rest/keystore/stage.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"}]},{"name":"truststore","children":[{"name":"dev.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka-rest/truststore/dev.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"prod.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka-rest/truststore/prod.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"stage.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka-rest/truststore/stage.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"}]}]},{"name":"kafka","children":[{"name":"truststore","children":[{"name":"dev.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka/truststore/dev.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"prod.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka/truststore/prod.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"stage.truststore.jks","path":"examples/confluent-oss-ami/ssl/kafka/truststore/stage.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"}]}]},{"name":"schema-registry","children":[{"name":"keystore","children":[{"name":"dev.keystore.jks","path":"examples/confluent-oss-ami/ssl/schema-registry/keystore/dev.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"},{"name":"prod.keystore.jks","path":"examples/confluent-oss-ami/ssl/schema-registry/keystore/prod.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"},{"name":"stage.keystore.jks","path":"examples/confluent-oss-ami/ssl/schema-registry/keystore/stage.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"}]},{"name":"truststore","children":[{"name":"dev.truststore.jks","path":"examples/confluent-oss-ami/ssl/schema-registry/truststore/dev.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"prod.truststore.jks","path":"examples/confluent-oss-ami/ssl/schema-registry/truststore/prod.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"stage.truststore.jks","path":"examples/confluent-oss-ami/ssl/schema-registry/truststore/stage.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"}]}]}]}],"toggled":true},{"name":"kafka-ami","children":[{"name":"README.md","path":"examples/kafka-ami/README.md","sha":"692e94969d0352ab15fb3f87a67b44085a1285a5"},{"name":"check-for-kafka-key-store.sh","path":"examples/kafka-ami/check-for-kafka-key-store.sh","sha":"27141a3a4a2ffe07fae4e83503d833ef3e3ec36b"},{"name":"config","children":[{"name":"README.md","path":"examples/kafka-ami/config/README.md","sha":"ec0ff5b551e31f4783712d3ed169dbbb757ca9f9"},{"name":"kafka","children":[{"name":"config","children":[{"name":"dev.server-4.0.x.properties","path":"examples/kafka-ami/config/kafka/config/dev.server-4.0.x.properties","sha":"5ea1bae91e95a50333444d0ffffc94924cfd0483"},{"name":"prod.server-4.0.x.properties","path":"examples/kafka-ami/config/kafka/config/prod.server-4.0.x.properties","sha":"5ea1bae91e95a50333444d0ffffc94924cfd0483"},{"name":"stage.server-4.0.x.properties","path":"examples/kafka-ami/config/kafka/config/stage.server-4.0.x.properties","sha":"5ea1bae91e95a50333444d0ffffc94924cfd0483"}]},{"name":"log4j","children":[{"name":"dev.log4j.properties","path":"examples/kafka-ami/config/kafka/log4j/dev.log4j.properties","sha":"394c539d46d5922b33ba1e8b3a50db2fbed7e6ef"},{"name":"prod.log4j.properties","path":"examples/kafka-ami/config/kafka/log4j/prod.log4j.properties","sha":"394c539d46d5922b33ba1e8b3a50db2fbed7e6ef"},{"name":"stage.log4j.properties","path":"examples/kafka-ami/config/kafka/log4j/stage.log4j.properties","sha":"394c539d46d5922b33ba1e8b3a50db2fbed7e6ef"}]}]}]},{"name":"configure-kafka-server.sh","path":"examples/kafka-ami/configure-kafka-server.sh","sha":"e4206f8df32e89810d8eff1be13be1e90df733b4"},{"name":"kafka.json","path":"examples/kafka-ami/kafka.json","sha":"98b106f115513800b58723a8d97f632cdda65c11"},{"name":"ssl","children":[{"name":"README.md","path":"examples/kafka-ami/ssl/README.md","sha":"51859e48ac5ba48f1278f479d38112e69e761fa3"},{"name":"kafka","children":[{"name":"ca-cert","path":"examples/kafka-ami/ssl/kafka/ca-cert","sha":"fb02e172efcdc4ad4c660e137059be86926108f4"},{"name":"cert","path":"examples/kafka-ami/ssl/kafka/cert","sha":"0f486b16f80eebe97d9135542d229404a6b48ddc"},{"name":"keystore","children":[{"name":"dev.keystore.jks","path":"examples/kafka-ami/ssl/kafka/keystore/dev.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"},{"name":"prod.keystore.jks","path":"examples/kafka-ami/ssl/kafka/keystore/prod.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"},{"name":"stage.keystore.jks","path":"examples/kafka-ami/ssl/kafka/keystore/stage.keystore.jks","sha":"6283b3e9b655c2a987192e81b3a6172e6c9ea487"}]},{"name":"truststore","children":[{"name":"dev.truststore.jks","path":"examples/kafka-ami/ssl/kafka/truststore/dev.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"prod.truststore.jks","path":"examples/kafka-ami/ssl/kafka/truststore/prod.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"},{"name":"stage.truststore.jks","path":"examples/kafka-ami/ssl/kafka/truststore/stage.truststore.jks","sha":"9545e6ac795144d714c23f252abe79f4811d4d89"}]}]}]}]},{"name":"kafka-zookeeper-confluent-oss-ami","children":[{"name":"README.md","path":"examples/kafka-zookeeper-confluent-oss-ami/README.md","sha":"1626f60062f2b2ba3d8352a459067b5fae812dbf"},{"name":"config","children":[{"name":"README.md","path":"examples/kafka-zookeeper-confluent-oss-ami/config/README.md","sha":"c14625eae0097f9e3428ce11d5a0a39580bde664"},{"name":"kafka-connect","children":[{"name":"config","children":[{"name":"worker-4.0.x.properties","path":"examples/kafka-zookeeper-confluent-oss-ami/config/kafka-connect/config/worker-4.0.x.properties","sha":"7cde3283393b1b511a79b87936a0430c4607641f"}]},{"name":"log4j","children":[{"name":"log4j.properties","path":"examples/kafka-zookeeper-confluent-oss-ami/config/kafka-connect/log4j/log4j.properties","sha":"a23dfdbf369c5a8cba498d9016ab239f3c1c18a8"}]}]},{"name":"kafka-rest","children":[{"name":"config","children":[{"name":"kafka-rest-4.0.x.properties","path":"examples/kafka-zookeeper-confluent-oss-ami/config/kafka-rest/config/kafka-rest-4.0.x.properties","sha":"29c9ca3bd784637597c683b1585bfd52fd0035db"}]},{"name":"log4j","children":[{"name":"log4j.properties","path":"examples/kafka-zookeeper-confluent-oss-ami/config/kafka-rest/log4j/log4j.properties","sha":"43c18e3a2eb5bdf7a49c0336919aac1acf5f6b6d"}]}]},{"name":"kafka","children":[{"name":"config","children":[{"name":"server-4.0.x.properties","path":"examples/kafka-zookeeper-confluent-oss-ami/config/kafka/config/server-4.0.x.properties","sha":"f9ae2462af98e6e98f4b64df21be5bbdd4df56a9"}]},{"name":"log4j","children":[{"name":"log4j.properties","path":"examples/kafka-zookeeper-confluent-oss-ami/config/kafka/log4j/log4j.properties","sha":"394c539d46d5922b33ba1e8b3a50db2fbed7e6ef"}]}]},{"name":"schema-registry","children":[{"name":"config","children":[{"name":"schema-registry-4.0.x.properties","path":"examples/kafka-zookeeper-confluent-oss-ami/config/schema-registry/config/schema-registry-4.0.x.properties","sha":"6d499d60b09982b45424f4060e04c586dd39287c"}]},{"name":"log4j","children":[{"name":"log4j.properties","path":"examples/kafka-zookeeper-confluent-oss-ami/config/schema-registry/log4j/log4j.properties","sha":"28fa60645b6ba0ab402433aebbedec8a8a9533e3"}]}]}]},{"name":"configure-kafka-zk-confluent-server.sh","path":"examples/kafka-zookeeper-confluent-oss-ami/configure-kafka-zk-confluent-server.sh","sha":"eaa35dd030e655d16b6e0e170e569aa5aba0a550"},{"name":"docker-compose.yml","path":"examples/kafka-zookeeper-confluent-oss-ami/docker-compose.yml","sha":"8beedfce4e773f452dee733f4f26cf5e8b0cb763"},{"name":"kafka-zookeeper-confluent-oss.json","path":"examples/kafka-zookeeper-confluent-oss-ami/kafka-zookeeper-confluent-oss.json","sha":"996c854d2065434180500881778d184615bbcb61"},{"name":"mock","children":[{"name":"README.md","path":"examples/kafka-zookeeper-confluent-oss-ami/mock/README.md","sha":"d373af41223ad92e574a958e4e17b78da61fe725"},{"name":"bash-commons","children":[{"name":"aws.sh","path":"examples/kafka-zookeeper-confluent-oss-ami/mock/bash-commons/aws.sh","sha":"ce067be902c8c7c49b85bb395f8eb50b87a535e6"},{"name":"docker.sh","path":"examples/kafka-zookeeper-confluent-oss-ami/mock/bash-commons/docker.sh","sha":"7827db443288057e5f2df9f43a955bc2afa464a4"}]},{"name":"modules","children":[{"name":"attach-eni","path":"examples/kafka-zookeeper-confluent-oss-ami/mock/modules/attach-eni","sha":"da052caea4586b27c2dc13e521092e9403fcc327"},{"name":"mount-ebs-volume","path":"examples/kafka-zookeeper-confluent-oss-ami/mock/modules/mount-ebs-volume","sha":"9b81549efc7c94e5baf609918e4831dd780eee2f"}]},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/kafka-zookeeper-confluent-oss-ami/mock/user-data/user-data.sh","sha":"31964f5f291ce525a04c0aedd3a9c8c84557944a"}]}]},{"name":"wait_for_zk.sh","path":"examples/kafka-zookeeper-confluent-oss-ami/wait_for_zk.sh","sha":"0ac5e9e1bb712d727f14c8777b35c0ccfbfbf59e"}]},{"name":"kafka-zookeeper-confluent-oss-colocated-cluster","children":[{"name":"README.md","path":"examples/kafka-zookeeper-confluent-oss-colocated-cluster/README.md","sha":"1ab7d9295b492aee332af4c8e507320eef2351ee"},{"name":"main.tf","path":"examples/kafka-zookeeper-confluent-oss-colocated-cluster/main.tf","sha":"906ff574edf90d0fe06522099ea3166eb762ea36"},{"name":"outputs.tf","path":"examples/kafka-zookeeper-confluent-oss-colocated-cluster/outputs.tf","sha":"331d0632825a6f349d01751717ecf84bb08581c7"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/kafka-zookeeper-confluent-oss-colocated-cluster/user-data/user-data.sh","sha":"a63ab90111ce199b179f3db24f04e00e6cfeaf69"}]},{"name":"vars.tf","path":"examples/kafka-zookeeper-confluent-oss-colocated-cluster/vars.tf","sha":"b671f2f0d2f1ac9888ec630f42275ccd7e3d6a4e"}]},{"name":"kafka-zookeeper-confluent-oss-standalone-clusters","children":[{"name":"README.md","path":"examples/kafka-zookeeper-confluent-oss-standalone-clusters/README.md","sha":"c951818155d03b44f3335a54da07f2ba43360bbf"},{"name":"main.tf","path":"examples/kafka-zookeeper-confluent-oss-standalone-clusters/main.tf","sha":"a6b34d7b2150b7ba907470c6f612eee2f68f5e59"},{"name":"outputs.tf","path":"examples/kafka-zookeeper-confluent-oss-standalone-clusters/outputs.tf","sha":"3c12deec6a6f70912b60bd14f20ac5f40152046e"},{"name":"user-data","children":[{"name":"confluent-tools-cluster-user-data.sh","path":"examples/kafka-zookeeper-confluent-oss-standalone-clusters/user-data/confluent-tools-cluster-user-data.sh","sha":"01b77ddab8cebc5367c05af25f361113d1d34a5f"},{"name":"kafka-cluster-user-data.sh","path":"examples/kafka-zookeeper-confluent-oss-standalone-clusters/user-data/kafka-cluster-user-data.sh","sha":"4216a3d67c3fab1db94c78d290eb685952268b46"},{"name":"zookeeper-cluster-user-data.sh","path":"examples/kafka-zookeeper-confluent-oss-standalone-clusters/user-data/zookeeper-cluster-user-data.sh","sha":"e5edb3e727f377436ae9c991cd2c7b01fc52baba"}]},{"name":"vars.tf","path":"examples/kafka-zookeeper-confluent-oss-standalone-clusters/vars.tf","sha":"80ed87cfc775f8e81415d9c6d0d636fa4eec4fed"}]},{"name":"kafka-zookeeper-standalone-clusters","children":[{"name":"README.md","path":"examples/kafka-zookeeper-standalone-clusters/README.md","sha":"9780235b75ceb497d3fd299c4794eb0f2601ef15"},{"name":"main.tf","path":"examples/kafka-zookeeper-standalone-clusters/main.tf","sha":"d4a90c87dc0062972e31385dbc792123ed63a6c2"},{"name":"outputs.tf","path":"examples/kafka-zookeeper-standalone-clusters/outputs.tf","sha":"051d6e5ddda7cef04fd9f5031a57694124762eca"},{"name":"user-data","children":[{"name":"kafka-user-data.sh","path":"examples/kafka-zookeeper-standalone-clusters/user-data/kafka-user-data.sh","sha":"ccb4d9324cabce27d4617c7e32ea10a10014385a"},{"name":"zookeeper-user-data.sh","path":"examples/kafka-zookeeper-standalone-clusters/user-data/zookeeper-user-data.sh","sha":"2e144c5dc55c4e721ecf595ae60df4553024191f"}]},{"name":"vars.tf","path":"examples/kafka-zookeeper-standalone-clusters/vars.tf","sha":"fc70b12e7d2c784e3a491ad08d10005f44ed71a7"}]},{"name":"zookeeper-ami","children":[{"name":"README.md","path":"examples/zookeeper-ami/README.md","sha":"6ebd1619152754561ae480c0ec00945ff7d2df43"},{"name":"configure-zookeeper-server.sh","path":"examples/zookeeper-ami/configure-zookeeper-server.sh","sha":"4faecc54c6dae27a79256f8346b1195de0ebb74b"},{"name":"zookeeper.json","path":"examples/zookeeper-ami/zookeeper.json","sha":"cbe0d0d35c0353dd3d7d6203a8e678e3c9def2c5"}]}],"toggled":true},{"name":"modules","children":[{"name":"bash-commons","children":[{"name":"README.md","path":"modules/bash-commons/README.md","sha":"0b7b7bf23db870999e14ee833d05488ba44a136c"},{"name":"install.sh","path":"modules/bash-commons/install.sh","sha":"bedb09f6eaa00a323ae1dd814afec954ca3efeeb"},{"name":"lib","children":[{"name":"array.sh","path":"modules/bash-commons/lib/array.sh","sha":"2d4e0ef22dc608392e99522e8ff0eb68ed1f708c"},{"name":"assert.sh","path":"modules/bash-commons/lib/assert.sh","sha":"bfaf1740050694ed05d03bcff7dbc99724c4fc43"},{"name":"aws.sh","path":"modules/bash-commons/lib/aws.sh","sha":"e6986c813e1fef28dfd5881b0193c4925e8dc66b"},{"name":"file.sh","path":"modules/bash-commons/lib/file.sh","sha":"196b04006ff622844d6d198f78130b2d2fd7c0c6"},{"name":"java.sh","path":"modules/bash-commons/lib/java.sh","sha":"3cc8614fd91c2d9e0816555e558ee12ba0a8c95b"},{"name":"log.sh","path":"modules/bash-commons/lib/log.sh","sha":"1b5887a63f9e7de613707866753e2cfe910da4d1"},{"name":"os.sh","path":"modules/bash-commons/lib/os.sh","sha":"3371306dc7959874cf6b8935d9454e5c1c942c4d"},{"name":"strings.sh","path":"modules/bash-commons/lib/strings.sh","sha":"67a96995df1886ff0d4ce528b5fb26cfbe7b044d"}]}]},{"name":"confluent-tools-cluster","children":[{"name":"README.md","path":"modules/confluent-tools-cluster/README.md","sha":"658308333cc07a35f46a8ec04ff882c74db52bfe"},{"name":"main.tf","path":"modules/confluent-tools-cluster/main.tf","sha":"ac1bdaf4f075ba6ddfbf5d7462564d8d7f9a025a"},{"name":"outputs.tf","path":"modules/confluent-tools-cluster/outputs.tf","sha":"12f380d0aff454ad6d556c116d84c344a62bb999"},{"name":"vars.tf","path":"modules/confluent-tools-cluster/vars.tf","sha":"8234819f003e0566e96cb3a5fa14f71e6020139d"}]},{"name":"confluent-tools-iam-permissions","children":[{"name":"README.md","path":"modules/confluent-tools-iam-permissions/README.md","sha":"06057008bc666088f319da89baa9d16b68fdce30"},{"name":"main.tf","path":"modules/confluent-tools-iam-permissions/main.tf","sha":"b9d177f5ae0a29b9b57f79452b5093253e279d75"},{"name":"vars.tf","path":"modules/confluent-tools-iam-permissions/vars.tf","sha":"9803fc14dc414f11fc9f236685dbb0db8f5273e6"}]},{"name":"confluent-tools-security-group-rules","children":[{"name":"README.md","path":"modules/confluent-tools-security-group-rules/README.md","sha":"5ee75e53e2ba7c2893e147fe86a2ba48fc8feab0"},{"name":"main.tf","path":"modules/confluent-tools-security-group-rules/main.tf","sha":"552b6ef4bfa9da912df5a834b659c268db65e998"},{"name":"vars.tf","path":"modules/confluent-tools-security-group-rules/vars.tf","sha":"3faf1ad90e4c918aefbec1545235ecf97ecc08cf"}]},{"name":"generate-key-stores","children":[{"name":"README.md","path":"modules/generate-key-stores/README.md","sha":"e4cacd66c8857a552810dc5a8d328a5395659392"},{"name":"generate-key-stores.sh","path":"modules/generate-key-stores/generate-key-stores.sh","sha":"b2076744af89591375d36974d4c3a84ba25f32bb"},{"name":"install.sh","path":"modules/generate-key-stores/install.sh","sha":"33c6e02e94425b4d58050ca7719e70c0292e1ba6"}]},{"name":"install-confluent-tools","children":[{"name":"README.md","path":"modules/install-confluent-tools/README.md","sha":"ca24e1b76b9a0299b4c36e07bff2941e6960e0c1"},{"name":"install.sh","path":"modules/install-confluent-tools/install.sh","sha":"2a0b67e853dfa3f9e6fa760f05fea02a3abe9f87"},{"name":"security","children":[{"name":"confluent.key","path":"modules/install-confluent-tools/security/confluent.key","sha":"1025a2c6dfa66f224c0c45ac172fd8d3efce1744"}]}]},{"name":"install-kafka","children":[{"name":"README.md","path":"modules/install-kafka/README.md","sha":"c94e75bda5a0c753d35a67d785af65d46b551e5b"},{"name":"install.sh","path":"modules/install-kafka/install.sh","sha":"4b0c29e5515b4e94ed4f56c1685faebccbd22471"}]},{"name":"kafka-cluster","children":[{"name":"README.md","path":"modules/kafka-cluster/README.md","sha":"a054f23f1a0aebcc5be1f1ec0368bb67f17e20f6"},{"name":"main.tf","path":"modules/kafka-cluster/main.tf","sha":"27d089f3d898bb01ade5125fcd94d8136435864d"},{"name":"outputs.tf","path":"modules/kafka-cluster/outputs.tf","sha":"d53790ae73bc413f3a21acb52064bddf30d0d517"},{"name":"vars.tf","path":"modules/kafka-cluster/vars.tf","sha":"9bf8a4f71cf0c382192eb5a7fee70fbe333a8f7e"}]},{"name":"kafka-iam-permissions","children":[{"name":"README.md","path":"modules/kafka-iam-permissions/README.md","sha":"3763dd58a4cd52e71b5ffe0d3e5e9fe0cf48053f"},{"name":"main.tf","path":"modules/kafka-iam-permissions/main.tf","sha":"ecfc7111106491e318ca9eed66d258f3b08df09d"},{"name":"vars.tf","path":"modules/kafka-iam-permissions/vars.tf","sha":"d29e8e5a07834701c3050e6369607747f85e43d8"}]},{"name":"kafka-security-group-rules","children":[{"name":"README.md","path":"modules/kafka-security-group-rules/README.md","sha":"63627f3af8842e4af90d0c31337004ef2503fc9c"},{"name":"main.tf","path":"modules/kafka-security-group-rules/main.tf","sha":"a5946f1e5cf4dfa784be01c332c058672ce2fbfe"},{"name":"vars.tf","path":"modules/kafka-security-group-rules/vars.tf","sha":"9c27d84eb7bfd6291acafc3e844f0ab7e1ec970f"}]},{"name":"run-health-checker","children":[{"name":"README.md","path":"modules/run-health-checker/README.md","sha":"8bc3cab46b5eecb25e642364dbd57ea35a0be71c"},{"name":"bin","children":[{"name":"run-health-checker","path":"modules/run-health-checker/bin/run-health-checker","sha":"ee3357d5fcc32957115b32538c00702c20b36a97"}]},{"name":"install.sh","path":"modules/run-health-checker/install.sh","sha":"af927c79f7df2b1d57204e1207a4104cbf0f63a2"}]},{"name":"run-kafka-connect","children":[{"name":"README.md","path":"modules/run-kafka-connect/README.md","sha":"0f180c7e494588f218cfe2ace99c671c8dfcadcf"},{"name":"bin","children":[{"name":"run-kafka-connect","path":"modules/run-kafka-connect/bin/run-kafka-connect","sha":"fb4145cc36ab2f8efc39ad7fc94444fa866ccef1"}]},{"name":"config","children":[{"name":"README.md","path":"modules/run-kafka-connect/config/README.md","sha":"5c6f2b6e63f1eba41957dd083c0787ac139c9622"},{"name":"kafka-connect","children":[{"name":"worker-3.3.x.properties","path":"modules/run-kafka-connect/config/kafka-connect/worker-3.3.x.properties","sha":"6bd29bb369f3aaea951e0a00d157ba167e943813"},{"name":"worker-4.0.x.properties","path":"modules/run-kafka-connect/config/kafka-connect/worker-4.0.x.properties","sha":"5b4760ba1b5805ed9ec7eb0f0be06a3597f15001"}]},{"name":"log4j","children":[{"name":"log4j.properties","path":"modules/run-kafka-connect/config/log4j/log4j.properties","sha":"a23dfdbf369c5a8cba498d9016ab239f3c1c18a8"}]}]},{"name":"install.sh","path":"modules/run-kafka-connect/install.sh","sha":"ba03f49f90a63a0e683c7be1b016b14db41bd71c"},{"name":"security","children":[{"name":"README.md","path":"modules/run-kafka-connect/security/README.md","sha":"5242a8435552c50055c926e1ec704545ca2c1b24"},{"name":"confluent-3.3.1-2.11.tar.gz.checksum","path":"modules/run-kafka-connect/security/confluent-3.3.1-2.11.tar.gz.checksum","sha":"c7aed490972e7b1565795221488d18449bd0bae1"},{"name":"confluent-4.0.0-2.11.tar.gz.checksum","path":"modules/run-kafka-connect/security/confluent-4.0.0-2.11.tar.gz.checksum","sha":"27b7a13f188475b4157386aa2761915633b72aa3"}]}]},{"name":"run-kafka-rest","children":[{"name":"README.md","path":"modules/run-kafka-rest/README.md","sha":"93c501f797818e3fee1e078cfbb93cc35577cc91"},{"name":"bin","children":[{"name":"run-kafka-rest","path":"modules/run-kafka-rest/bin/run-kafka-rest","sha":"2d1ff513fef3134b9c066f1e80ef04986ec6d5a2"}]},{"name":"config","children":[{"name":"README.md","path":"modules/run-kafka-rest/config/README.md","sha":"772b1a95a54aadd3c0f0dce34eac10f3d1967634"},{"name":"kafka-rest","children":[{"name":"kafka-rest-3.3.x.properties","path":"modules/run-kafka-rest/config/kafka-rest/kafka-rest-3.3.x.properties","sha":"21263ea344efbfa26e1a792b8bb29c33606c0d7a"},{"name":"kafka-rest-4.0.x.properties","path":"modules/run-kafka-rest/config/kafka-rest/kafka-rest-4.0.x.properties","sha":"29c9ca3bd784637597c683b1585bfd52fd0035db"}]},{"name":"log4j","children":[{"name":"log4j.properties","path":"modules/run-kafka-rest/config/log4j/log4j.properties","sha":"43c18e3a2eb5bdf7a49c0336919aac1acf5f6b6d"}]}]},{"name":"install.sh","path":"modules/run-kafka-rest/install.sh","sha":"97c543dd2a175ea3866f92e835494723645c6edc"}]},{"name":"run-kafka","children":[{"name":"README.md","path":"modules/run-kafka/README.md","sha":"b1fcb424860e462141f9a17423b0c3ecc3f2de08"},{"name":"bin","children":[{"name":"run-kafka","path":"modules/run-kafka/bin/run-kafka","sha":"2301427f95fbfb42ac5be080993d5cdd8699e172"}]},{"name":"config","children":[{"name":"README.md","path":"modules/run-kafka/config/README.md","sha":"e08702423b137b254ad0a0a070f7300f094ba046"},{"name":"kafka","children":[{"name":"server-3.3.x.properties","path":"modules/run-kafka/config/kafka/server-3.3.x.properties","sha":"b5aa3757d41f0e6fef81799b58c222774a0da63d"},{"name":"server-4.0.x.properties","path":"modules/run-kafka/config/kafka/server-4.0.x.properties","sha":"5ea1bae91e95a50333444d0ffffc94924cfd0483"}]},{"name":"log4j","children":[{"name":"log4j.properties","path":"modules/run-kafka/config/log4j/log4j.properties","sha":"394c539d46d5922b33ba1e8b3a50db2fbed7e6ef"}]}]},{"name":"install.sh","path":"modules/run-kafka/install.sh","sha":"d538e667d2a66004ceacc149b1d8c2f54e639ec7"}]},{"name":"run-schema-registry","children":[{"name":"README.md","path":"modules/run-schema-registry/README.md","sha":"343ddd7b1e9b24054c05722b3cb6ea4e63d88419"},{"name":"bin","children":[{"name":"run-schema-registry","path":"modules/run-schema-registry/bin/run-schema-registry","sha":"07a457889d54a9385de0bcc4d4d5dc51b3636b19"}]},{"name":"config","children":[{"name":"README.md","path":"modules/run-schema-registry/config/README.md","sha":"93a30f8adf3f778682463bbc8715a38f70908866"},{"name":"log4j","children":[{"name":"log4j.properties","path":"modules/run-schema-registry/config/log4j/log4j.properties","sha":"28fa60645b6ba0ab402433aebbedec8a8a9533e3"}]},{"name":"schema-registry","children":[{"name":"schema-registry.properties","path":"modules/run-schema-registry/config/schema-registry/schema-registry.properties","sha":"e6541005171b9f0de27e7f177f915b08399f9404"}]}]},{"name":"install.sh","path":"modules/run-schema-registry/install.sh","sha":"81b4a8b7c8b26d6b4a60e524bcda30e95a2777b7"}]}]},{"name":"terraform-cloud-enterprise-private-module-registry-placeholder.tf","path":"terraform-cloud-enterprise-private-module-registry-placeholder.tf","sha":"ae586c0fe830819580e1009d41a9074f16e65bed"},{"name":"test","children":[{"name":"README.md","path":"test/README.md","sha":"cf72a9f58a3b36aee8053b37dfc6e1a617f93f7b"},{"name":"generate_key_stores_test.go","path":"test/generate_key_stores_test.go","sha":"fceb03a8da4583eff59b8c4da1c070b9ef0c980a"},{"name":"go.mod","path":"test/go.mod","sha":"73214126337cfa12cfd00919689fb51ea7493b3a"},{"name":"go.sum","path":"test/go.sum","sha":"93a629ddbce948e976f8b110943947df31c0975e"},{"name":"kafka_zookeeper_confluent_colocated_cluster_test.go","path":"test/kafka_zookeeper_confluent_colocated_cluster_test.go","sha":"d5ac28bbeef65dfaf784f7339e516510c9c22ba4"},{"name":"kafka_zookeeper_confluent_standalone_clusters_test.go","path":"test/kafka_zookeeper_confluent_standalone_clusters_test.go","sha":"55445cf91cd5534aa0fdb148b96bd5d1013b4915"},{"name":"kafka_zookeeper_standalone_clusters_test.go","path":"test/kafka_zookeeper_standalone_clusters_test.go","sha":"55abe10b05b0cba29f54df6f83b02d849fe19c83"},{"name":"test_helpers.go","path":"test/test_helpers.go","sha":"f01924f4c18c7595d58621e5e3dcc2128996f44e"},{"name":"test_helpers_kafka.go","path":"test/test_helpers_kafka.go","sha":"474caf79574b27637d376fe5ab65abea9af7eb3c"},{"name":"test_helpers_kafka_connect.go","path":"test/test_helpers_kafka_connect.go","sha":"1ecef92e9a45501fb4bf834579d02ddcc05e7103"},{"name":"test_helpers_keystore.go","path":"test/test_helpers_keystore.go","sha":"02d88327f4021955dca74c5311eb916bed5c7afa"},{"name":"test_helpers_rest_proxy.go","path":"test/test_helpers_rest_proxy.go","sha":"39a5c7d2f96a873615c856cd56861ad5e7920e1c"},{"name":"test_helpers_schema_registry.go","path":"test/test_helpers_schema_registry.go","sha":"b15a39916cd41441705265ccabeb23e0460b260f"}]}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"confluent-open-source-tools-example-ami\">Confluent Open Source Tools Example AMI</h1><div class=\"preview__body--border\"></div><p>This folder contains an example <a href=\"https://www.packer.io/\" class=\"preview__body--description--blue\" target=\"_blank\">Packer</a> template that can be used to create an <a href=\"http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html\" class=\"preview__body--description--blue\" target=\"_blank\">Amazon\nMachine Image (AMI)</a> with open source Confluent Tools,\nincluding <a href=\"https://docs.confluent.io/4.0.0/schema-registry/docs/index.html\" class=\"preview__body--description--blue\" target=\"_blank\">Schema Registry</a> and <a href=\"https://docs.confluent.io/4.0.0/kafka-rest/docs/index.html\" class=\"preview__body--description--blue\" target=\"_blank\">REST Proxy</a>. A few notes about this Packer template:</p>\n<ul>\n<li>\n<p>It builds two AMIs, one on top of Amazon Linux and one on top of Ubuntu. Pick whichever distro you prefer.</p>\n</li>\n<li>\n<p>This AMI includes examples of how to configure Schema Registry and REST Proxy with SSL support. It expects you to use\nthe <a href=\"/repos/v0.11.0/package-kafka/modules/generate-key-stores\" class=\"preview__body--description--blue\">generate-key-stores module</a> to generate a Key Store and Trust Store before building\nthe AMI.</p>\n</li>\n<li>\n<p>This AMI <em>only</em> includes Schema Registry and REST Proxy. It does not include ZooKeeper, Kafka, or Kafka Connect, which\nare expected to be running in one or more separate clusters, launched from separate AMIs (see the <a href=\"/repos/v0.11.0/package-kafka/examples/zookeeper-ami\" class=\"preview__body--description--blue\">zookeeper-ami\nexample</a> and <a href=\"/repos/v0.11.0/package-kafka/examples/kafka-ami\" class=\"preview__body--description--blue\">kafka-ami example</a>). For an example of all services\nco-located in the same cluster, see <a href=\"/repos/v0.11.0/package-kafka/examples/kafka-zookeeper-confluent-oss-ami\" class=\"preview__body--description--blue\">kafka-zookeeper-confluent-ami</a>.</p>\n</li>\n</ul>\n<p>To deploy this AMI, see the <a href=\"/repos/v0.11.0/package-kafka/examples/kafka-zookeeper-standalone-clusters\" class=\"preview__body--description--blue\">kafka-zookeeper-standalone-clusters example</a>.</p>\n<h2 class=\"preview__body--subtitle\" id=\"quick-start\">Quick start</h2>\n<p>To build the AMI:</p>\n<ol>\n<li>\n<p>Run the <a href=\"/repos/v0.11.0/package-kafka/modules/generate-key-stores\" class=\"preview__body--description--blue\">generate-key-stores module</a> to create a Key Store, Trust Store, and\nself-signed SSL certificate for each of Schema Registry, REST Proxy, and the Kafka Connect workers. You can run the\n<code>check-for-xxx-key-store.sh</code> scripts to print out example usage for the <code>generate-key-stores.sh</code> script. For additional\ninformation on this step, see [Generating SSL Certificates](#generating-ssl-certificates] below.</p>\n</li>\n<li>\n<p>Copy the Trust Store you created for the <a href=\"/repos/v0.11.0/package-kafka/examples/kafka-ami\" class=\"preview__body--description--blue\">kafka-ami</a> to the <code>confluent-oss-ami/ssl/kafka</code> folder.\nWe'll use this to enable Schema Registry, REST Proxy, and Kafka Connect to connect via SSL to Kafka.</p>\n</li>\n<li>\n<p>Install <a href=\"https://www.packer.io/\" class=\"preview__body--description--blue\" target=\"_blank\">Packer</a>.</p>\n</li>\n<li>\n<p>Set up your <a href=\"https://www.packer.io/docs/builders/amazon.html\" class=\"preview__body--description--blue\" target=\"_blank\">AWS credentials as environment variables</a>.</p>\n</li>\n<li>\n<p>Set the <code>GITHUB_OAUTH_TOKEN</code> environment variable to a valid GitHub auth token with "repo" access. You can generate\none here: https://github.com/settings/tokens</p>\n</li>\n<li>\n<p>Run <code>packer build -var "package_kafka_branch=<branch>" -only <target os> confluent-oss.json</code> to create a new AMI in your AWS account. Note down the ID of this new AMI.</p>\n</li>\n</ol>\n<p>To deploy the new AMI, plug its ID into the <a href=\"/repos/v0.11.0/package-kafka/examples/kafka-zookeeper-standalone-clusters\" class=\"preview__body--description--blue\">kafka-zookeeper-standalone-clusters\nexample</a>.</p>\n<h3 class=\"preview__body--subtitle\" id=\"generating-ssl-certificates\">Generating SSL Certificates</h3>\n<p>The number of <a href=\"/repos/v0.11.0/package-kafka/modules/confluent-tools-cluster/README.md#tls-ssl-and-security\" class=\"preview__body--description--blue\">possible interactions between all the tools</a> in\nthe Confluent Platform makes understanding the various SSL certificates needed for a "secure" Confluent setup potentially\nconfusing. This section aims to clear that up.</p>\n<h4 id=\"terminology\">Terminology</h4>\n<ul>\n<li>\n<p><strong>Key Store:</strong> A file that contains SSL certificates a given service presents to others. It contains both a public\ncertificate that the server presents to clients and a corresponding private key that the server uses to prove that it\nis the identity presented by the public certificate. This file is generated by the <a href=\"https://docs.oracle.com/javase/9/tools/keytool.htm#JSWOR-GUID-5990A2E4-78E3-47B7-AE75-6D1826259549\" class=\"preview__body--description--blue\" target=\"_blank\">keytool</a> program that\ncomes bundled with the JDK. <em>TL;DR: The Key Store contains the certs a given service presents to others.</em></p>\n</li>\n<li>\n<p><strong>Trust Store:</strong> A file that contains the public certificate of the Certificate Authority (CA) used to sign public\ncertificates used by other servers. Services use the Trust Store to know which public certificates they trust when\nmaking an SSL connection. This file is generated by the <a href=\"https://docs.oracle.com/javase/9/tools/keytool.htm#JSWOR-GUID-5990A2E4-78E3-47B7-AE75-6D1826259549\" class=\"preview__body--description--blue\" target=\"_blank\">keytool</a> program that\ncomes bundled with the JDK. <em>TL;DR: The Trust Store contains the certs a given server will trust from others.</em></p>\n</li>\n</ul>\n<h4 id=\"instructions\">Instructions</h4>\n<p>The steps to generate the right SSL certificate are straightforward. Here are some guidelines to follow:</p>\n<ol>\n<li>\n<p>All SSL certificates should be generated using the <a href=\"/repos/v0.11.0/package-kafka/modules/generate-key-stores\" class=\"preview__body--description--blue\">generate-key-stores</a> module.</p>\n</li>\n<li>\n<p>Kafka brokers should receive their own Key Store and Trust STore. Schema Registry, REST Proxy, and Kafka Connect may\nshare the same Key Store and Trust Store, although some users may wish to create a separate Key Store/Trust Store for\neach of these services. Check out the <code>check-for-xxx-key-store.sh</code> scripts in this folder to see additional details\nabout generating SSL certificates for each service.</p>\n<p>Note that you will use the <em>same</em> Kafka broker SSL certificates in both the Kafka broker AMI and other AMIs. That's\nbecause supporting tools like Schema Registry and REST Proxy need the Trust Store file you generated when you created\nthe Kafka SSL certificates.</p>\n</li>\n<li>\n<p>Now place all your SSL files in the following folder structure in this Packer template folder:</p>\n<pre><span class=\"hljs-bullet\">- </span>confluent-oss-ami\n<span class=\"hljs-bullet\"> - </span>ssl\n<span class=\"hljs-bullet\"> - </span>kafka\n<span class=\"hljs-bullet\"> - </span>truststore.jks\n<span class=\"hljs-bullet\"> - </span>kafka-connect\n<span class=\"hljs-bullet\"> - </span>keystore.jks\n<span class=\"hljs-bullet\"> - </span>truststore.jks (optional)\n<span class=\"hljs-bullet\"> - </span>kafka-rest\n<span class=\"hljs-bullet\"> - </span>keystore.jks\n<span class=\"hljs-bullet\"> - </span>truststore.jks (optional)\n<span class=\"hljs-bullet\"> - </span>kafka-rest\n<span class=\"hljs-bullet\"> - </span>keystore.jks\n<span class=\"hljs-bullet\"> - </span>truststore.jks (optional) \n</pre>\n<p>If you are re-using the same set of SSL files, feel free to copy the same file into different file paths. This makes\nit explicit exactly where each Key Store and/or Trust Store file comes from.</p>\n</li>\n</ol>\n<h4 id=\"inspecting-ssl-certificate-files\">Inspecting SSL Certificate Files</h4>\n<p>If you want to examine an existing SSL certificate file to see what information it contains, run the following command:</p>\n<pre>openssl x509 -<span class=\"hljs-keyword\">in</span> /path/<span class=\"hljs-keyword\">to</span>/cert/<span class=\"hljs-built_in\">file</span> -<span class=\"hljs-built_in\">text</span>\n</pre>\n<p>Note that this will only work on the <code>cert</code> and <code>ca-cert</code> files, not on the Key Store and Trust Store files, which are\nunique to Java. To view those, you can use an open source tool like <a href=\"http://keystore-explorer.org/\" class=\"preview__body--description--blue\" target=\"_blank\">Keystore Explorer</a>.</p>\n","repoName":"package-kafka","repoRef":"v0.8.0","serviceDescriptor":{"serviceName":"Apache Kafka and Confluent Tools","serviceRepoName":"package-kafka","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws"],"description":"Deploy a cluster of Kafka brokers. Optionally deploy Confluent tools such as Schema Registry, REST Proxy, and Kafka Connect.","imageUrl":"kafka.png","licenseType":"subscriber","technologies":["Terraform","Bash"],"compliance":[],"tags":[""]},"serviceCategoryName":"Messaging & streaming","fileName":"README.md","filePath":"/examples/confluent-oss-ami","title":"Repo Browser: Apache Kafka and Confluent Tools","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}
.circleci
check-for-kafka-trust-store.sh
