In this tutorial we are going to install Kafka on Ubuntu 18.04 TLS
- Ubuntu OS
- Java JDK 8
- 4GB of RAM on the server.
1- First check the java version :
$ java –version
2- Go to Kafka website to download
selected “Scala 2.12 version”
create downlaod directory
$ mkdir ~/kafka && cd ~/kafka
and download the file into kafka directory
open tar file kafka_2.12-2,1,0.tgz
$ tar -xvf kafka_2.12-2.1.0.tgz
you should be able to see below files under your bin directory.
Test your installation using below command:
Zookeeper : Which is used by Kafka to maintain state between the nodes of the cluster.
Kafka brokers : The “pipes” in our pipeline, which store and emit data.
Producers : That insert data into the cluster.
Consumers : That read data from the cluster.
First we need to start zookeeper: lets check zookeeper config file where we specify client port
Run the command
$ bin/zookeeper-server-start.sh ../config/zookeeper.properties
You should see a confirmation that the server has started.
$ bin/kafka-server-start.sh ../config/server.properties
Its time to create our first topic :
Created topic “firstTopic”
We can list the topics which we created using below command:
$ /kafka-topics.sh –list –zookeeper localhost:2181
Now check how producer and consumer works briefly:
I started my consumer : (we already created the topic)
$ ./kafka-console-consumer.sh –bootstrap-server localhost:9092 –topic firstTopic –from-beginning
I started my consumer:
$ ./kafka-console-consumer.sh –bootstrap-server localhost:9092 –topic FirstTopic –from-beginning