轉(zhuǎn)載自：https://www.cnblogs.com/saneri/p/8762168.html

主機(jī)規(guī)劃：

10.200.3.85 　　Kafka+ZooKeeper

10.200.3.86 　　Kafka+ZooKeeper

10.200.3.87 　　 Kafka+ZooKeeper

軟件下載地址：

wget http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.10/zookeeper-3.4.10.tar.gz
wget http://mirror.bit.edu.cn/apache/kafka/1.1.0/kafka_2.12-1.1.0.tgz

三臺主機(jī)hosts文件一致：
cat /etc/hosts
10.200.3.85 ZooKeeper-Kafka-01
10.200.3.86 ZooKeeper-Kafka-02
10.200.3.87 ZooKeeper-Kafka-03

一、安裝zookeeper

1.在master節(jié)點(diǎn)上操作：
[root@ZooKeeper-Kafka-01 src]# tar -zxvf zookeeper-3.4.10.tar.gz -C /usr/local/ [root@ZooKeeper-Kafka-01 src]# cd ..
[root@ZooKeeper-Kafka-01 local]# ln -s zookeeper-3.4.10 zookeeper
[root@ZooKeeper-Kafka-01 local]# cd zookeeper/conf/
[root@ZooKeeper-Kafka-01 conf]# cp zoo_sample.cfg zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper
clientPort=2181
server.1=ZooKeeper-Kafka-01:2888:3888
server.2=ZooKeeper-Kafka-02:2888:3888
server.3=ZooKeeper-Kafka-03:2888:3888

2.創(chuàng)建dataDir目錄創(chuàng)建/tmp/zookeeper

在master節(jié)點(diǎn)上

mkdir /tmp/zookeeper
[root@ZooKeeper-Kafka-01 conf]# touch /tmp/zookeeper/myid
[root@ZooKeeper-Kafka-01 conf]# echo 1 > /tmp/zookeeper/myid</pre>

3.將zookeeper文件復(fù)制到另外兩個節(jié)點(diǎn)：

[root@ZooKeeper-Kafka-01 local]# scp -r zookeeper-3.4.10/ 10.200.3.86:/usr/local/ [root@ZooKeeper-Kafka-01 local]# scp -r zookeeper-3.4.10/ 10.200.3.87:/usr/local/</pre>

4.在兩個slave節(jié)點(diǎn)創(chuàng)建目錄和文件

ZooKeeper-Kafka-02節(jié)點(diǎn):

[root@ZooKeeper-Kafka-02 local]# ln -s zookeeper-3.4.10 zookeeper
[root@ZooKeeper-Kafka-02 local]# mkdir /tmp/zookeeper
[root@ZooKeeper-Kafka-02 local]# touch /tmp/zookeeper/myid
[root@ZooKeeper-Kafka-02 local]# echo 2 > /tmp/zookeeper/myid</pre>

ZooKeeper-Kafka-03節(jié)點(diǎn)

[root@ZooKeeper-Kafka-03 local]# ln -s zookeeper-3.4.10 zookeeper
[root@ZooKeeper-Kafka-03 local]# mkdir /tmp/zookeeper
[root@ZooKeeper-Kafka-03 local]# touch /tmp/zookeeper/myid
[root@ZooKeeper-Kafka-03 local]# echo 3 > /tmp/zookeeper/myid</pre>

5.分別在每個節(jié)點(diǎn)上啟動 zookeeper測試：

[root@ZooKeeper-Kafka-01 zookeeper]# ./bin/zkServer.sh start
[root@ZooKeeper-Kafka-02 zookeeper]# ./bin/zkServer.sh start
[root@ZooKeeper-Kafka-03 zookeeper]# ./bin/zkServer.sh start</pre>

6.查看狀態(tài)：

[root@ZooKeeper-Kafka-01 zookeeper]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@ZooKeeper-Kafka-02 zookeeper]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@ZooKeeper-Kafka-03 zookeeper]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower</pre>

至此zookeeper集群安裝成功７氚肌！坡脐！

Kafka集群安裝配置

1.安裝：

[root@ZooKeeper-Kafka-01 src]# tar -zxvf kafka_2.12-1.1.0.tgz -C /usr/local/ [root@ZooKeeper-Kafka-01 src]# cd ..
[root@ZooKeeper-Kafka-01 local]# ln -s kafka_2.12-1.1.0 kafka</pre>

2.修改server.properties文件

master為0
broker.id=0 listeners=PLAINTEXT://ZooKeeper-Kafka-01:9092
advertised.listeners=PLAINTEXT://ZooKeeper-Kafka-01:9092
num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/tmp/kafka-logs
num.partitions=5 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=24 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 # 連接
zookeeper.connect=ZooKeeper-Kafka-01:2181,ZooKeeper-Kafka-02:2181,ZooKeeper-Kafka-03:2181 zookeeper.connection.timeout.ms=6000 group.initial.rebalance.delay.ms=0 # 可刪除topic
delete.topic.enable=true

3.將 kafka_2.12-1.1.0 文件夾復(fù)制到另外兩個節(jié)點(diǎn)下

[root@ZooKeeper-Kafka-01 local]# scp -r kafka_2.12-1.1.0/ 10.200.3.86:/usr/local/ [root@ZooKeeper-Kafka-01 local]# scp -r kafka_2.12-1.1.0/ 10.200.3.87:/usr/local/

并修改每個節(jié)點(diǎn)對應(yīng)的 server.properties 文件的 broker.id和listenrs：

[root@ZooKeeper-Kafka-02 config]# cat server.properties
broker.id=1 listeners=PLAINTEXT://ZooKeeper-Kafka-02:9092
advertised.listeners=PLAINTEXT://ZooKeeper-Kafka-02:9092
[root@ZooKeeper-Kafka-03 config]# cat server.properties
broker.id=2 listeners=PLAINTEXT://ZooKeeper-Kafka-03:9092
advertised.listeners=PLAINTEXT://ZooKeeper-Kafka-03:9092

4.啟動服務(wù)

bin/kafka-server-start.sh config/server.properties &

Zookeeper+Kafka集群測試

創(chuàng)建topic：

[root@ZooKeeper-Kafka-01 kafka]# bin/kafka-topics.sh --create --zookeeper ZooKeeper-Kafka-01:2181, ZooKeeper-Kafka-02:2181, ZooKeeper-Kafka-03:2181 --replication-factor 3 --partitions 3 --topic test

顯示topic：

[root@ZooKeeper-Kafka-01 kafka]# bin/kafka-topics.sh --describe --zookeeper ZooKeeper-Kafka-01:2181, ZooKeeper-Kafka-02:2181, ZooKeeper-Kafka-03:2181 --topic test

列出topic：
[root@ZooKeeper-Kafka-01 kafka]# bin/kafka-topics.sh --list --zookeeper ZooKeeper-Kafka-01:2181, ZooKeeper-Kafka-02:2181, ZooKeeper-Kafka-03:2181 test

創(chuàng)建 producer(生產(chǎn)者);

在master節(jié)點(diǎn)上 測試生產(chǎn)消息

[2018-04-03 12:20:53,141] INFO Updated PartitionLeaderEpoch. New: {epoch:0, offset:0}, Current: {epoch:-1, offset:-1} for Partition: test-1. Cache now contains 0 entries. (kafka.server.epoch.LeaderEpochFileCache)

創(chuàng)建 consumer(消費(fèi)者):

[root@ZooKeeper-Kafka-02 kafka]# bin/kafka-console-consumer.sh --bootstrap-server ZooKeeper-Kafka-01:9092, ZooKeeper-Kafka-02:9092, ZooKeeper-Kafka-03:9092 -topic test --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper]. this is example ...
hello world
[2018-04-03 12:20:53,145] INFO Updated PartitionLeaderEpoch. New: {epoch:0, offset:0}, Current: {epoch:-1, offset:-1} for Partition: test-1. Cache now contains 0 entries. (kafka.server.epoch.LeaderEpochFileCache)
welcome to china

在ZooKeeper-Kafka-03節(jié)點(diǎn)上 測試消費(fèi)

[root@ZooKeeper-Kafka-03 kafka]# bin/kafka-console-consumer.sh --bootstrap-server ZooKeeper-Kafka-01:9092, ZooKeeper-Kafka-02:9092, ZooKeeper-Kafka-03:9092 -topic test --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
welcome to china
hello world this is example ...</pre>

然后在 producer 里輸入消息肋坚，consumer 中就會顯示出同樣的內(nèi)容，表示消費(fèi)成功

刪除 topic 和關(guān)閉服務(wù)

[root@ZooKeeper-Kafka-01 kafka]# bin/kafka-topics.sh --delete --zookeeper ZooKeeper-Kafka-01:2181, ZooKeeper-Kafka-02:2181, ZooKeeper-Kafka-03:2181 --topic test</pre>

啟動服務(wù):

bin/kafka-server-start.sh config/server.properties &</pre>

停止服務(wù)：

bin/kafka-server-stop.sh</pre>

至此Zookeeper+Kafka集群配置成功.