Browse the Repo

file-type-icon.circleci
file-type-iconexamples
file-type-iconmodules
file-type-iconbash-commons
file-type-iconconfluent-tools-cluster
file-type-iconconfluent-tools-iam-permissions
file-type-iconconfluent-tools-security-group-rules
file-type-icongenerate-key-stores
file-type-iconinstall-confluent-tools
file-type-iconinstall-kafka
file-type-iconREADME.md
file-type-iconinstall.sh
file-type-iconkafka-cluster
file-type-iconkafka-iam-permissions
file-type-iconkafka-security-group-rules
file-type-iconrun-health-checker
file-type-iconrun-kafka-connect
file-type-iconrun-kafka-rest
file-type-iconrun-kafka
file-type-iconrun-schema-registry
file-type-icontest
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconLICENSE.txt
file-type-iconREADME.md
file-type-iconterraform-cloud-enterprise-private-module-...

Browse the Repo

file-type-icon.circleci
file-type-iconexamples
file-type-iconmodules
file-type-iconbash-commons
file-type-iconconfluent-tools-cluster
file-type-iconconfluent-tools-iam-permissions
file-type-iconconfluent-tools-security-group-rules
file-type-icongenerate-key-stores
file-type-iconinstall-confluent-tools
file-type-iconinstall-kafka
file-type-iconREADME.md
file-type-iconinstall.sh
file-type-iconkafka-cluster
file-type-iconkafka-iam-permissions
file-type-iconkafka-security-group-rules
file-type-iconrun-health-checker
file-type-iconrun-kafka-connect
file-type-iconrun-kafka-rest
file-type-iconrun-kafka
file-type-iconrun-schema-registry
file-type-icontest
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconLICENSE.txt
file-type-iconREADME.md
file-type-iconterraform-cloud-enterprise-private-module-...
Apache Kafka and Confluent Tools

Apache Kafka and Confluent Tools

Deploy a cluster of Kafka brokers. Optionally deploy Confluent tools such as Schema Registry, REST Proxy, and Kafka Connect.

Code Preview

Preview the Code

mobile file icon

README.md

down

Install Kafka

This folder contains a script for installing Apache Kafka.

This script has been tested on the following operating systems:

  • Amazon Linux
  • Ubuntu

There is a good chance it will work on Debian, CentOS, and RHEL as well, but our automated testing for this module does not cover these other distros at the moment.

Quick start

The easiest way to use this module is with the Gruntwork Installer:

gruntwork-install \
  --module-name "install-kafka" \
  --repo "https://github.com/gruntwork-io/terraform-aws-kafka" \
  --tag "v0.0.4"

We recommend running this module as part of a Packer template to create an Amazon Machine Image (AMI) (see kafka-ami and kafka-zookeeper-ami for fully-working sample code). You can then deploy the AMI using the kafka-cluster module (see the kafka-zookeeper-standalone-clusters and kafka-zookeeper-confluent-oss-colocated-cluster examples for fully-working sample code).

Command line Arguments

The install-kafka module accepts the following arguments, all optional:

  • --kafka-version: The version of Kafka to install.
  • --scala-version: The version of Scala that --kafka-version is compiled with.
  • --install-dir: The directory where Kafka should be installed. Default: /opt/kafka.
  • --user: The OS user who should own --install-dir. Default: kafka.
  • --key-store-path: The path to a Key Store to use for SSL. Default: (none). See the generate-key-stores for how to generate a Key Store.
  • --trust-store-path: The path to a Trust Store to use for SSL. Default: (none). See the generate-key-stores for how to generate a Key Store.

If you're using gruntwork-install to run this module, you can pass these arguments using --module-param arguments. Example:

gruntwork-install \
  --module-name "install-kafka" \
  --repo "https://github.com/gruntwork-io/terraform-aws-kafka" \
  --tag "v0.0.4" \
  --module-param "version=0.11.0.0"

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?