Kafka Basic Operator

In these tutorial, kafka cmd tool is direct and effective.

before operator the cmd, we should enter kafka container by next bash

   docker exec -it 066d6fbce8cb sh

create topic

The next cmd will create a topic name test.

kafka-topics.sh --zookeeper zookeeper:2181 --create --topic test --partitions 2 --replication-factor 1

List Topic

kafka-topics.sh --zookeeper zookeeper:2181 --list

this cmd will show all the topic names.

also, if we want to see the topic more detail, we can exec next cmd:

kafka-topics.sh --zookeeper zookeeper:2181 --describe --topic test

will show info

Topic: test     TopicId: VlyyjZkrTcO54GHlK5HRyw PartitionCount: 2       ReplicationFactor: 1    Configs:
        Topic: test     Partition: 0    Leader: 1       Replicas: 1     Isr: 1
        Topic: test     Partition: 1    Leader: 1       Replicas: 1     Isr: 1

Message Product And Consumer

client listener

./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test

create another client to product message

kafka-console-producer.sh --broker-list localhost:9092 --topic test

now, we can test product and consumer message

Consumer offset detail

kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --describe --group aaa

aaa is means group name, then we can see the detail

Kafka Meta Data

if we want see kafka Meta Data, we can enter zookeeper container

docker exec -it e6feccc5166b bash
./bin/zkCli.sh
ls /
ls /brokers

detail

[seqid, topics, ids]

ls /brokers/ids
[1]

broker info

get /brokers/ids/1

{"features":{},"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT"},"endpoints":["PLAINTEXT://10.32.1.200:9092"],"jmx_port":-1,"port":9092,"host":"10.32.1.200","version":5,"timestamp":"1720082796829"}
cZxid = 0x19
ctime = Thu Jul 04 08:46:36 UTC 2024
mZxid = 0x19
mtime = Thu Jul 04 08:46:36 UTC 2024
pZxid = 0x19
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x1000087a5a10000
dataLength = 206
numChildren = 0

topic and partitions info

[zk: localhost:2181(CONNECTED) 5] ls /brokers/topics
[test, __consumer_offsets]
[zk: localhost:2181(CONNECTED) 6] ls /brokers/topics/test
[partitions]
[zk: localhost:2181(CONNECTED) 7] ls /brokers/topics/test/partitions
[0, 1]
[zk: localhost:2181(CONNECTED) 8] ls /brokers/topics/test/partitions/0
[state]

[zk: localhost:2181(CONNECTED) 18] get /brokers/topics/test/partitions/0/state
{"controller_epoch":1,"leader":1,"version":1,"leader_epoch":0,"isr":[1]}
cZxid = 0xb0
ctime = Tue Jan 05 05:56:06 GMT 2021
mZxid = 0xb0
mtime = Tue Jan 05 05:56:06 GMT 2021
pZxid = 0xb0
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 72
numChildren = 0