kafka #0

Supports: trusty

Add to new model


A single Kafka broker can handle hundreds of megabytes of reads and writes per
second from thousands of clients.

Kafka is designed to allow a single cluster to serve as the central data
backbone for a large organization. It can be elastically and transparently
expanded without downtime. Data streams are partitioned and spread over a
cluster of machines to allow data streams larger than the capability of any
single machine and to allow clusters of co-ordinated consumers

Messages are persisted on disk and replicated within the cluster to prevent
data loss. Each broker can handle terabytes of messages without performance

Distributed by Design
Kafka has a modern cluster-centric design that offers strong durability and
fault-tolerance guarantees.


Scalable distributed publish/subscribe system built ontop of a distributed log.


$ juju deploy zookeeper $ juju deploy kafka $ juju add-relation kafka zookeeper


First we need to get a hold of the zookeeper servers that are kafka brokers
are connected to:

$ juju run --unit=kafka/0 ./hooks/utils.py

We can list all the cli commands available to us via

$ juju run --unit=kafka/0 "ls bin"

kafka-console-consumer.sh kafka-console-producer.sh kafka-consumer-perf-test.sh kafka-preferred-replica-election.sh kafka-producer-perf-test.sh kafka-reassign-partitions.sh kafka-replay-log-producer.sh kafka-simple-consumer-perf-test.sh kafka-simple-consumer-shell.sh kafka-topics.sh

We can create a topic with

$ juju run --unit=kafka/0 "./bin/kafka-topics.sh --zookeeper --create --topic my_topic_name --partitions 5 --replication-factor 1"

We can list topics with

$ juju run --unit=kafka/0 "./bin/kafka-topics.sh --zookeeper --list