zk集群的一些基本概念
zookeeper集群搭建:
- zk集群,主從節(jié)點哲身,心跳機(jī)制(選舉模式)
- 配置數(shù)據(jù)文件 myid 1/2/3 對應(yīng) server.1/2/3
- 通過 zkCli.sh -server [ip]:[port] 命令檢測集群是否配置成功
和其他大多數(shù)集群結(jié)構(gòu)一樣抚芦,zookeeper集群也是主從結(jié)構(gòu)。搭建集群時琴锭,機(jī)器數(shù)量最低也是三臺蝌借,因為小于三臺就無法進(jìn)行選舉。選舉就是當(dāng)集群中的master節(jié)點掛掉之后吟吝,剩余的兩臺機(jī)器會進(jìn)行選舉菱父,在這兩臺機(jī)器中選舉出一臺來做master節(jié)點。而當(dāng)原本掛掉的master恢復(fù)正常后剑逃,也會重新加入集群當(dāng)中浙宜。但是不會再作為master節(jié)點,而是作為slave節(jié)點蛹磺。如下:
單機(jī)偽分布式搭建zookeeper集群
本節(jié)介紹單機(jī)偽分布式的zookeeper安裝粟瞬,官方下載地址如下:
我這里使用的是3.4.11版本,所以找到相應(yīng)的版本點擊進(jìn)去称开,復(fù)制到.tar.gz的下載鏈接到Linux上進(jìn)行下載亩钟。命令如下:
[root@study-01 ~]# cd /usr/local/src/
[root@study-01 /usr/local/src]# wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.11/zookeeper-3.4.11.tar.gz
下載完成之后將其解壓到/usr/local/目錄下:
[root@study-01 /usr/local/src]# tar -zxvf zookeeper-3.4.11.tar.gz -C /usr/local/
[root@study-01 /usr/local/src]# cd ../zookeeper-3.4.11/
[root@study-01 /usr/local/zookeeper-3.4.11]# ls
bin dist-maven lib README_packaging.txt zookeeper-3.4.11.jar.asc
build.xml docs LICENSE.txt recipes zookeeper-3.4.11.jar.md5
conf ivysettings.xml NOTICE.txt src zookeeper-3.4.11.jar.sha1
contrib ivy.xml README.md zookeeper-3.4.11.jar
[root@study-01 /usr/local/zookeeper-3.4.11]#
然后給目錄重命名一下:
[root@study-01 ~]# cd /usr/local/
[root@study-01 /usr/local]# mv zookeeper-3.4.11/ zookeeper00
接著進(jìn)行一系列的配置:
[root@study-01 /usr/local]# cd zookeeper00/
[root@study-01 /usr/local/zookeeper00]# cd conf/
[root@study-01 /usr/local/zookeeper00/conf]# cp zoo_sample.cfg zoo.cfg # 拷貝官方提供的模板配置文件
[root@study-01 /usr/local/zookeeper00/conf]# vim zoo.cfg # 增加或修改成如下內(nèi)容
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper00/dataDir
dataLogDir=/usr/local/zookeeper00/dataLogDir
clientPort=2181
4lw.commands.whitelist=*
server.1=192.168.190.129:2888:3888 # master節(jié)點,ip后面跟的是集群通信的端口
server.2=192.168.190.129:2889:3889
server.3=192.168.190.129:2890:3890
[root@study-01 /usr/local/zookeeper00/conf]# cd ../
[root@study-01 /usr/local/zookeeper00]# mkdir {dataDir,dataLogDir}
[root@study-01 /usr/local/zookeeper00]# cd dataDir/
[root@study-01 /usr/local/zookeeper00/dataDir]# vim myid # 配置該節(jié)點的id
1
[root@study-01 /usr/local/zookeeper00/dataDir]#
配置完之后鳖轰,拷貝多個目錄出來清酥,因為是單機(jī)的偽分布式所以需要在一臺機(jī)器上安裝多個zookeeper:
[root@study-01 /usr/local]# cp zookeeper00 zookeeper01 -rf
[root@study-01 /usr/local]# cp zookeeper00 zookeeper02 -rf
配置 zookeeper01:
[root@study-01 /usr/local]# cd zookeeper01/conf/
[root@study-01 /usr/local/zookeeper01/conf]# vim zoo.cfg # 修改內(nèi)容如下
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper01/dataDir
dataLogDir=/usr/local/zookeeper01/dataLogDir
clientPort=2182 # 端口號必須要修改
4lw.commands.whitelist=*
server.1=192.168.190.129:2888:3888
server.2=192.168.190.129:2889:3889
server.3=192.168.190.129:2890:3890
[root@study-01 /usr/local/zookeeper01/conf]# cd ../dataDir/
[root@study-01 /usr/local/zookeeper01/dataDir]# vim myid
2
[root@study-01 /usr/local/zookeeper01/dataDir]#
配置 zookeeper02:
[root@study-01 /usr/local]# cd zookeeper02/conf/
[root@study-01 /usr/local/zookeeper02/conf]# vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper02/dataDir
dataLogDir=/usr/local/zookeeper02/dataLogDir
clientPort=2183 # 端口號必須要修改
4lw.commands.whitelist=*
server.1=192.168.190.129:2888:3888
server.2=192.168.190.129:2889:3889
server.3=192.168.190.129:2890:3890
[root@study-01 /usr/local/zookeeper02/conf]# cd ../dataDir/
[root@study-01 /usr/local/zookeeper02/dataDir]# vim myid
3
[root@study-01 /usr/local/zookeeper02/dataDir]#
以上就在單機(jī)上配置好三個zookeeper集群節(jié)點了,現(xiàn)在我們來測試一下蕴侣,這個偽分布式的zookeeper集群能否正常運作起來:
[root@study-01 ~]# cd /usr/local/zookeeper00/bin/
[root@study-01 /usr/local/zookeeper00/bin]# ./zkServer.sh start # 啟動第一個節(jié)點
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper00/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@study-01 /usr/local/zookeeper00/bin]# netstat -lntp |grep java # 查看監(jiān)聽的端口
tcp6 0 0 192.168.190.129:3888 :::* LISTEN 3191/java # 集群通信的端口
tcp6 0 0 :::44793 :::* LISTEN 3191/java
tcp6 0 0 :::2181 :::* LISTEN 3191/java
[root@study-01 /usr/local/zookeeper00/bin]# cd ../../zookeeper01/bin/
[root@study-01 /usr/local/zookeeper01/bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper01/bin/../conf/zoo.cfg # 啟動第二個節(jié)點
Starting zookeeper ... STARTED
[root@study-01 /usr/local/zookeeper01/bin]# cd ../../zookeeper02/bin/
[root@study-01 /usr/local/zookeeper02/bin]# ./zkServer.sh start # 啟動第三個節(jié)點
[root@study-01 /usr/local/zookeeper02/bin]# netstat -lntp |grep java # 查看監(jiān)聽的端口
tcp6 0 0 192.168.190.129:2889 :::* LISTEN 3232/java
tcp6 0 0 :::48463 :::* LISTEN 3232/java
tcp6 0 0 192.168.190.129:3888 :::* LISTEN 3191/java
tcp6 0 0 192.168.190.129:3889 :::* LISTEN 3232/java
tcp6 0 0 192.168.190.129:3890 :::* LISTEN 3286/java
tcp6 0 0 :::44793 :::* LISTEN 3191/java
tcp6 0 0 :::60356 :::* LISTEN 3286/java
tcp6 0 0 :::2181 :::* LISTEN 3191/java
tcp6 0 0 :::2182 :::* LISTEN 3232/java
tcp6 0 0 :::2183 :::* LISTEN 3286/java
[root@study-01 /usr/local/zookeeper02/bin]# jps # 查看進(jìn)程
3232 QuorumPeerMain
3286 QuorumPeerMain
3191 QuorumPeerMain
3497 Jps
[root@study-01 /usr/local/zookeeper02/bin]#
如上焰轻,可以看到,三個節(jié)點都正常啟動成功了昆雀,接下來我們進(jìn)入客戶端辱志,創(chuàng)建一些znode,看看是否會同步到集群中的其他節(jié)點上去:
[root@study-01 /usr/local/zookeeper02/bin]# ./zkCli.sh -server localhost:2181 # 登錄第一個節(jié)點的客戶端
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 1] create /data test-data
Created /data
[zk: localhost:2181(CONNECTED) 2] ls /
[zookeeper, data]
[zk: localhost:2181(CONNECTED) 3] quit
[root@study-01 /usr/local/zookeeper02/bin]# ./zkCli.sh -server localhost:2182 # 登錄第二個節(jié)點的客戶端
[zk: localhost:2182(CONNECTED) 0] ls / # 可以查看到我們在第一個節(jié)點上創(chuàng)建的znode狞膘,代表集群中的節(jié)點能夠正常同步數(shù)據(jù)
[zookeeper, data]
[zk: localhost:2182(CONNECTED) 1] get /data # 數(shù)據(jù)也是一致的
test-data
cZxid = 0x100000002
ctime = Tue Apr 24 18:35:56 CST 2018
mZxid = 0x100000002
mtime = Tue Apr 24 18:35:56 CST 2018
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: localhost:2182(CONNECTED) 2] quit
[root@study-01 /usr/local/zookeeper02/bin]# ./zkCli.sh -server localhost:2183 # 登錄第三個節(jié)點的客戶端
[zk: localhost:2183(CONNECTED) 0] ls / # 第三個節(jié)點也能查看到我們在第一個節(jié)點上創(chuàng)建的znode
[zookeeper, data]
[zk: localhost:2183(CONNECTED) 1] get /data # 數(shù)據(jù)也是一致的
test-data
cZxid = 0x100000002
ctime = Tue Apr 24 18:35:56 CST 2018
mZxid = 0x100000002
mtime = Tue Apr 24 18:35:56 CST 2018
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: localhost:2183(CONNECTED) 2] quit
[root@study-01 /usr/local/zookeeper02/bin]#
查看集群的狀態(tài)揩懒、主從信息需要使用 ./zkServer.sh status
命令,但是多個節(jié)點的話挽封,逐個查看有些費勁已球,所以我們寫一個簡單的shell腳本來批量執(zhí)行命令。如下:
[root@study-01 ~]# vim checked.sh # 腳本內(nèi)容如下
#!/bin/bash
/usr/local/zookeeper00/bin/zkServer.sh status
/usr/local/zookeeper01/bin/zkServer.sh status
/usr/local/zookeeper02/bin/zkServer.sh status
[root@study-01 ~]# sh ./checked.sh # 執(zhí)行腳本
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper00/bin/../conf/zoo.cfg
Mode: follower # 從節(jié)點
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper01/bin/../conf/zoo.cfg
Mode: leader # 主節(jié)點
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper02/bin/../conf/zoo.cfg
Mode: follower
[root@study-01 ~]#
到此為止辅愿,我們就成功完成了單機(jī)zookeeper偽分布式集群的搭建智亮,并且也測試成功了。
搭建zookeeper分布式集群
接下來点待,我們使用三臺虛擬機(jī)來搭建zookeeper真實分布式集群阔蛉,機(jī)器的ip地址如下:
- 192.168.190.128
- 192.168.190.129
- 192.168.190.130
注:三臺機(jī)器都必須具備java的運行環(huán)境,并且關(guān)閉或清空防火墻規(guī)則癞埠,不想關(guān)閉防火墻的話状原,就需要去配置相應(yīng)的防火墻規(guī)則
首先配置一下系統(tǒng)的hosts文件:
[root@localhost ~]# vim /etc/hosts
192.168.190.128 zk000
192.168.190.129 zk001
192.168.190.130 zk002
把之前做偽分布式實驗機(jī)器上的其他zookeeper目錄刪除掉聋呢,并把zookeeper目錄使用rsync同步到其他機(jī)器上。如下:
[root@zk001 ~]# cd /usr/local/
[root@zk001 /usr/local]# rm -rf zookeeper01
[root@zk001 /usr/local]# rm -rf zookeeper02
[root@zk001 /usr/local]# mv zookeeper00/ zookeeper
[root@zk001 /usr/local]# rsync -av /usr/local/zookeeper/ 192.168.190.128:/usr/local/zookeeper/
[root@zk001 /usr/local]# rsync -av /usr/local/zookeeper/ 192.168.190.130:/usr/local/zookeeper/
然后逐個在三臺機(jī)器上都配置一下環(huán)境變量遭笋,如下:
[root@zk001 ~]# vim .bash_profile # 增加如下內(nèi)容
export ZOOKEEPER_HOME=/usr/local/zookeeper
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin
export PATH
[root@zk001 ~]# source .bash_profile
逐個修改配置文件坝冕,zk000:
[root@zk000 ~]# cd /usr/local/zookeeper/conf/
[root@zk000 /usr/local/zookeeper/conf]# vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/dataDir
dataLogDir=/usr/local/zookeeper/dataLogDir
clientPort=2181
4lw.commands.whitelist=*
server.1=192.168.190.128:2888:3888 # 默認(rèn)server.1為master節(jié)點
server.2=192.168.190.129:2888:3888
server.3=192.168.190.130:2888:3888
[root@zk000 /usr/local/zookeeper/conf]# cd ../dataDir/
[root@zk000 /usr/local/zookeeper/dataDir]# vim myid
1
[root@zk000 /usr/local/zookeeper/dataDir]#
zk001:
[root@zk001 ~]# cd /usr/local/zookeeper/conf/
[root@zk001 /usr/local/zookeeper/conf]# vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/dataDir
dataLogDir=/usr/local/zookeeper/dataLogDir
clientPort=2181
4lw.commands.whitelist=*
server.1=192.168.190.128:2888:3888 # 默認(rèn)server.1為master節(jié)點
server.2=192.168.190.129:2888:3888
server.3=192.168.190.130:2888:3888
[root@zk001 /usr/local/zookeeper/conf]# cd ../dataDir/
[root@zk001 /usr/local/zookeeper/dataDir]# vim myid
2
[root@zk001 /usr/local/zookeeper/dataDir]#
zk002:
[root@zk002 ~]# cd /usr/local/zookeeper/conf/
[root@zk002 /usr/local/zookeeper/conf]# vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/dataDir
dataLogDir=/usr/local/zookeeper/dataLogDir
clientPort=2181
4lw.commands.whitelist=*
server.1=192.168.190.128:2888:3888 # 默認(rèn)server.1為master節(jié)點
server.2=192.168.190.129:2888:3888
server.3=192.168.190.130:2888:3888
[root@zk002 /usr/local/zookeeper/conf]# cd ../dataDir/
[root@zk002 /usr/local/zookeeper/dataDir]# vim myid
3
[root@zk002 /usr/local/zookeeper/dataDir]#
配置完成之后徒探,啟動三臺機(jī)器的zookeeper服務(wù):
[root@zk000 ~]# zkServer.sh start
[root@zk001 ~]# zkServer.sh start
[root@zk002 ~]# zkServer.sh start
啟動成功后瓦呼,查看三個機(jī)器的集群狀態(tài)信息:
[root@zk000 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@zk000 ~]#
[root@zk001 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@zk001 ~]#
[root@zk002 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@zk002 ~]#
然后我們來測試創(chuàng)建znode是否會同步,進(jìn)入192.168.190.128機(jī)器的客戶端:
[root@zk000 ~]# zkCli.sh -server 192.168.190.128:2181
[zk: 192.168.190.128:2181(CONNECTED) 0] ls /
[zookeeper, data]
[zk: 192.168.190.128:2181(CONNECTED) 1] create /real-culster real-data
Created /real-culster
[zk: 192.168.190.128:2181(CONNECTED) 2] ls /
[zookeeper, data, real-culster]
[zk: 192.168.190.128:2181(CONNECTED) 3] get /real-culster
real-data
cZxid = 0x300000002
ctime = Tue Apr 24 20:48:32 CST 2018
mZxid = 0x300000002
mtime = Tue Apr 24 20:48:32 CST 2018
pZxid = 0x300000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: 192.168.190.128:2181(CONNECTED) 4] quit
進(jìn)入192.168.190.129機(jī)器的客戶端
[root@zk000 ~]# zkCli.sh -server 192.168.190.129:2181
[zk: 192.168.190.129:2181(CONNECTED) 0] ls /
[zookeeper, data, real-culster]
[zk: 192.168.190.129:2181(CONNECTED) 1] get /real-culster
real-data
cZxid = 0x300000002
ctime = Tue Apr 24 20:48:32 CST 2018
mZxid = 0x300000002
mtime = Tue Apr 24 20:48:32 CST 2018
pZxid = 0x300000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: 192.168.190.129:2181(CONNECTED) 2] quit
進(jìn)入192.168.190.130機(jī)器的客戶端
[root@zk000 ~]# zkCli.sh -server 192.168.190.130:2181
[zk: 192.168.190.130:2181(CONNECTED) 0] ls /
[zookeeper, data, real-culster]
[zk: 192.168.190.130:2181(CONNECTED) 1] get /real-culster
real-data
cZxid = 0x300000002
ctime = Tue Apr 24 20:48:32 CST 2018
mZxid = 0x300000002
mtime = Tue Apr 24 20:48:32 CST 2018
pZxid = 0x300000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: 192.168.190.130:2181(CONNECTED) 2] quit
從以上的測試可以看到测暗,在zookeeper分布式集群中央串,我們在任意一個節(jié)點創(chuàng)建的znode都會被同步的集群中的其他節(jié)點上,數(shù)據(jù)也會被一并同步碗啄。所以到此為止质和,我們的zookeeper分布式集群就搭建成功了。
測試集群角色以及選舉
以上我們只是測試了znode的同步稚字,還沒有測試集群中的節(jié)點選舉饲宿,所以本節(jié)就來測試一下,當(dāng)master節(jié)點掛掉之后看看slave節(jié)點會不會通過選舉坐上master的位置胆描。首先我們來把master節(jié)點的zookeeper服務(wù)給停掉:
[root@zk001 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@zk001 ~]# zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@zk001 ~]#
這時到其他兩臺機(jī)器上進(jìn)行查看瘫想,可以看到有一臺已經(jīng)成為master節(jié)點了:
[root@zk002 ~]# zkServer.sh status # 可以看到zk002這個節(jié)點成為了master節(jié)點
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@zk002 ~]#
[root@zk000 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@zk000 ~]#
然后再把停掉的節(jié)點啟動起來,可以看到昌讲,該節(jié)點重新加入了集群国夜,但是此時是以slave角色存在,而不會以master角色存在:
[root@zk001 ~]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@zk001 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@zk001 ~]#
可以看到短绸,zk002這個節(jié)點依舊是master角色车吹,不會被取代,所以只有在選舉的時候集群中的節(jié)點才會切換角色:
[root@zk002 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@zk002 ~]#