Apache Kafka — Digital Wristband (Create Kafka producer and consumer from Linux Terminal)

First, Download Apache Kafka : https://kafka.apache.org/downloads. For example : https://www.apache.org/dyn/closer.cgi?path=/kafka/2.6.0/kafka_2.13-2.6.0.tgz

unzip files

tar -xzf kafka_2.13-2.6.0.tgz
cd kafka_2.13-2.6.0

need Java 8+

# for debian, ubuntu, mint
sudo apt-get install openjdk-8-jdk

Start the Kafka Environment

# Open new terminal 
bin/zookeeper-server-start.sh config/zookeeper.properties
---------------------------------------------------------
# Open another terminal
bin/kafka-server-start.sh config/server.properties

Create digital-wristband topic

for example: 1 broker, 5 partitions, 1 replication-factor

# Open another terminal
bin/kafka-topics.sh \
--create --zookeeper localhost:2181 \
--replication-factor 1 \
--partitions 5 \
--topic digital-wristband

Now, the zookeeper is running, a kafka broker is running, and a kafka topic has been created.

# Open another terminal 
➜  kafka_2.13-2.6.0 bin/kafka-topics.sh --describe --topic digital-wristband --bootstrap-server localhost:9092
Topic: digital-wristband PartitionCount: 5 ReplicationFactor: 1 Configs: segment.bytes=1073741824
 Topic: digital-wristband Partition: 0 Leader: 0 Replicas: 0 Isr: 0
 Topic: digital-wristband Partition: 1 Leader: 0 Replicas: 0 Isr: 0
 Topic: digital-wristband Partition: 2 Leader: 0 Replicas: 0 Isr: 0
 Topic: digital-wristband Partition: 3 Leader: 0 Replicas: 0 Isr: 0
 Topic: digital-wristband Partition: 4 Leader: 0 Replicas: 0 Isr: 0

Now let’s create 1 producer and a consumer from cli and show the relationship between each other.

# Open another ""PRODUCER"" terminal
➜  kafka_2.13-2.6.0 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic digital-wristband 
>Fedora
>
# Open another ""CONSUMER"" terminal
➜  kafka_2.13-2.6.0 bin/kafka-console-consumer.sh --topic digital-wristband --from-beginning --bootstrap-server localhost:9092
Ubuntu
Manjaro Linux
Linux Mint
Ubuntu 20
Debian
Fedora

Conclusion