安裝Zookeeper和Kafka集群
本文介紹如何安裝Zookeeper和Kafka集群机打。為了方便妒貌，介紹的是在一臺服務(wù)器上的安裝谐算，實(shí)際應(yīng)該安裝在多臺服務(wù)器上卓舵，但步驟是一樣的叫确。
安裝Zookeeper集群
下載安裝包
從官網(wǎng)上下載安裝包：
curl https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz -o apache-zookeeper-3.7.1-bin.tar.gz
解壓：
tar xvf apache-zookeeper-3.7.1-bin.tar.gz
配置
創(chuàng)建目錄 zk1跳芳，然后添加如下配置：
zk1/myid:
1
zk1/zk.config:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk1
clientPort=2181
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
對于zk2和zk3也重復(fù)同樣的步驟，并修改相應(yīng)的配置：
zk2/myid:
2
zk2/zk.config:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk2
clientPort=2182
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
zk3/myid:
3
zk3/zk.config:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk3
clientPort=2183
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
啟動(dòng)集群
啟動(dòng)三個(gè)服務(wù)如下：
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk1/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk1/zk.config
Starting zookeeper ... STARTED
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk2/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk2/zk.config
Starting zookeeper ... STARTED
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk3/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk3/zk.config
Starting zookeeper ... STARTED
查看狀態(tài)
通過status命令查看：
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk1/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk1/zk.config
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk2/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk2/zk.config
Client port found: 2182. Client address: localhost. Client SSL: false.
Mode: leader
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk3/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk3/zk.config
Client port found: 2183. Client address: localhost. Client SSL: false.
Mode: follower
連接其中一個(gè)服務(wù)并添加數(shù)據(jù)：
$ ./apache-zookeeper-3.7.1-bin/bin/zkCli.sh -server localhost:2181
[zk: localhost:2181(CONNECTED) 0] create /pkslow
Created /pkslow
[zk: localhost:2181(CONNECTED) 1] create /pkslow/website www.pkslow.com
Created /pkslow/website
連接另外一個(gè)服務(wù)竹勉，并查看數(shù)據(jù)飞盆，發(fā)現(xiàn)與之前創(chuàng)建的是一樣的：
$ ./apache-zookeeper-3.7.1-bin/bin/zkCli.sh -server localhost:2182
[zk: localhost:2182(CONNECTED) 1] get /pkslow/website
www.pkslow.com
安裝Kafka集群
下載安裝包
通過官網(wǎng)下載如下：
curl https://downloads.apache.org/kafka/3.4.0/kafka_2.13-3.4.0.tgz -o kafka_2.13-3.4.0.tgz
解壓安裝包：
tar -xzf kafka_2.13-3.4.0.tgz
配置
Broker 1的配置如下：
broker.id=1
port=9091
listeners=PLAINTEXT://:9091
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka1/kafka-logs
Broker 2的配置如下：
broker.id=2
port=9092
listeners=PLAINTEXT://:9092
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka2/kafka-logs
Broker 3的配置如下：
broker.id=3
port=9093
listeners=PLAINTEXT://:9093
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka3/kafka-logs
目錄結(jié)構(gòu)如下：
啟動(dòng)集群
啟動(dòng)kafka服務(wù)如下：
./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka1/server.properties
./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka2/server.properties
./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka3/server.properties
檢查如測試
創(chuàng)建topic：
$ kafka_2.13-3.4.0/bin/kafka-topics.sh --create --topic pkslow-topic --bootstrap-server localhost:9091,localhost:9092,localhost:9093 --partitions 3 --replication-factor 3
Created topic pkslow-topic.
列出topic：
$ kafka_2.13-3.4.0/bin/kafka-topics.sh --list --bootstrap-server localhost:9091,localhost:9092,localhost:9093
pkslow-topic
查看topic：
$ kafka_2.13-3.4.0/bin/kafka-topics.sh --describe --topic pkslow-topic --bootstrap-server localhost:9091,localhost:9092,localhost:9093
Topic: pkslow-topic TopicId: 7CLy7iZeRvm8rCrn8Dw_mA PartitionCount: 3 ReplicationFactor: 3 Configs:
Topic: pkslow-topic Partition: 0 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2
Topic: pkslow-topic Partition: 1 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: pkslow-topic Partition: 2 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1
生產(chǎn)者發(fā)消息到brokers：
$ kafka_2.13-3.4.0/bin/kafka-console-producer.sh --broker-list localhost:9091,localhost:9092,localhost:9093 --topic pkslow-topic
>My name is Larry Deng.
>My website is www.pkslow.com.
>
消費(fèi)者從brokers收消息：
$ kafka_2.13-3.4.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9091,localhost:9092,localhost:9093 --topic pkslow-topic --from-beginning
My name is Larry Deng.
My website is www.pkslow.com.
代碼
配置可以參考 GitHub pkslow-samples
