ZooKeeper是Hadoop的正式子項(xiàng)目祟滴,它是一個(gè)針對(duì)大型分布式系統(tǒng)的可靠協(xié)調(diào)系統(tǒng)麦到,提供的功能包括:配置維護(hù)、名字服務(wù)欠肾、分布式同步瓶颠、組服務(wù)等。ZooKeeper的目標(biāo)就是封裝好復(fù)雜易出錯(cuò)的關(guān)鍵服務(wù)刺桃,將簡(jiǎn)單易用的接口和性能高效粹淋、功能穩(wěn)定的系統(tǒng)提供給用戶。
官網(wǎng) http://zookeeper.apache.org/
下載 http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.6/
單機(jī)模式配置
windows為例
解壓后重命名為 zookeeper 瑟慈,修改zoo.cfg廓啊,修改如下
tickTime=2000
initLimit=10
syncLimit=5
dataDir=D:\\Java\\zookeeper\\data
dataLogDir=D:\\Java\\zookeeper\\logs
clientPort=7011
需要手動(dòng)創(chuàng)建兩個(gè)目錄 dataDir 和 dataLogDir,設(shè)置端口為7011
啟動(dòng)客戶端連接
D:\Java\zookeeper\bin>zkCli.cmd -server localhost:7011
查看節(jié)點(diǎn)
[zk: localhost:7011(CONNECTED) 9] ls /
[zkclitest, zookeeper]
創(chuàng)建節(jié)點(diǎn)
[zk: localhost:7011(CONNECTED) 11] create /testcli hello
Created /testcli
再查看
[zk: localhost:7011(CONNECTED) 12] ls /
[testcli, zkclitest, zookeeper]
退出客戶端
[zk: localhost:7011(CONNECTED) 13] quit
Quitting...
2015-10-30 14:59:57,525 [myid:] - INFO [main:ZooKeeper@684] - Session: 0x150b78
055c70000 closed
2015-10-30 14:59:57,525 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread
@512] - EventThread shut down
集群部署
zookeeper不僅可以單擊提供服務(wù)封豪,同時(shí)也支持多機(jī)組成集群來(lái)提供服務(wù)。集群模式有兩種
- 偽集群:一臺(tái)物理機(jī)器運(yùn)行多個(gè)zookeeper實(shí)例
- 集群:在多個(gè)物理機(jī)上運(yùn)行炒瘟,每個(gè)物理機(jī)只運(yùn)行一個(gè)zookeeper實(shí)例
簡(jiǎn)單說(shuō)下集群配置方式
環(huán)境:
- linux
- 172.28.20.101吹埠,172.28.20.102 兩臺(tái)機(jī)器
測(cè)試資源有限,在上述兩臺(tái)機(jī)器各部署兩個(gè)zookeeper 實(shí)例疮装。
下載解壓
將之前本機(jī)(windows)下載好的zookeeper上傳到服務(wù)器缘琅,并解壓重命名為zk2
tar -xzvf zookeeper-3.4.6.tar.gz
mv zookeeper-3.4.6 zk2
配置 zoo.cfg
打開bin目錄,將zoo_sample.cfg 備份并改名為zoo.cfg
cp zoo_sample.cfg zoo.cfg
zoo.cfg 是zookeeper的核心配置文件廓推,zk啟動(dòng)時(shí)候是通過(guò)配置文件的參數(shù)來(lái)初始化的刷袍。下面是本文的配置,
[hadoop@dc1 conf]$ vi zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/hadoop/zk2/data
dataLogDir=/home/hadoop/zk2/logs
clientPort=7012
server.1=dc1:7021:7031
server.2=dc2:7021:7031
server.3=dc2:7022:7032
server.4=dc1:7022:7032
- dataDir:數(shù)據(jù)存儲(chǔ)目錄
- dataLogDir:日志存儲(chǔ)目錄
這兩個(gè)目錄需要手動(dòng)先創(chuàng)建好樊展,否則會(huì)使用默認(rèn)的目錄呻纹,可能會(huì)與其他實(shí)例重復(fù) - clientPort:服務(wù)端口,同一臺(tái)機(jī)器 端口不能重復(fù)
- server.A=B:C:D
A: zk服務(wù)編號(hào)专缠,數(shù)字
B:服務(wù)所在IP地址
C:zk leader選舉端口
D:zk 各實(shí)例之間通信端口
除了上述coo.cfg配置文件外雷酪,集群模式需要在dataDir目錄下配置一個(gè)文件myid,如果沒有涝婉,需要新建哥力。文件內(nèi)容就是上面server.A中A的數(shù)值,表示當(dāng)前啟動(dòng)的server編號(hào)墩弯。
這樣101上的一個(gè)zk實(shí)例就配置好了吩跋,參照這個(gè)配置,將另外3個(gè)實(shí)例也配置好渔工。
啟動(dòng)server
分別啟動(dòng)四個(gè)實(shí)例
啟動(dòng)101機(jī)器上兩個(gè)zk實(shí)例
啟動(dòng)zk1
[hadoop@dc1 bin]$ ./zkServer.sh start
JMX enabled by default
Using config: /home/hadoop/zk1/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
啟動(dòng)zk2
[hadoop@dc1 bin]$ ./zkServer.sh start
JMX enabled by default
Using config: /home/hadoop/zk2/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
jps 看下進(jìn)程
[hadoop@dc1 bin]$ jps
9969 QuorumPeerMain
10508 QuorumPeerMain
10542 Jps
用客戶端連到server
[hadoop@dc1 bin]$ ./zkCli.sh -server localhost:7012
Connecting to localhost:7012
2015-10-29 10:00:54,496 [myid:] - INFO [Main Thread:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-10-29 10:00:54,515 [myid:] - INFO [Main Thread:Environment@100] - Client environment:host.name=dc1
2015-10-29 10:00:54,515 [myid:] - INFO [Main Thread:Environment@100] - Client environment:java.version=1.6.0_29
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:java.vendor=Oracle Corporation
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:java.home=/opt/app/cargo/jrockit-jdk1.6.0/jrockit-jdk1.6.0_29-R28.1.5-4.0.1/jre
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:java.class.path=/home/hadoop/zk2/bin/../build/classes:/home/hadoop/zk2/bin/../build/lib/*.jar:/home/hadoop/zk2/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/hadoop/zk2/bin/../lib/slf4j-api-1.6.1.jar:/home/hadoop/zk2/bin/../lib/netty-3.7.0.Final.jar:/home/hadoop/zk2/bin/../lib/log4j-1.2.16.jar:/home/hadoop/zk2/bin/../lib/jline-0.9.94.jar:/home/hadoop/zk2/bin/../zookeeper-3.4.6.jar:/home/hadoop/zk2/bin/../src/java/lib/*.jar:/home/hadoop/zk2/bin/../conf:.:/opt/app/cargo/jrockit-jdk1.6.0/jrockit-jdk1.6.0_29-R28.1.5-4.0.1/lib/dt.jar:/opt/app/cargo/jrockit-jdk1.6.0/jrockit-jdk1.6.0_29-R28.1.5-4.0.1/lib/tools.jar
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:java.library.path=/opt/app/cargo/jrockit-jdk1.6.0/jrockit-jdk1.6.0_29-R28.1.5-4.0.1/jre/lib/amd64/jrockit:/opt/app/cargo/jrockit-jdk1.6.0/jrockit-jdk1.6.0_29-R28.1.5-4.0.1/jre/lib/amd64:/opt/app/cargo/jrockit-jdk1.6.0/jrockit-jdk1.6.0_29-R28.1.5-4.0.1/jre/../lib/amd64
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:java.io.tmpdir=/tmp
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:java.compiler=<NA>
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:os.name=Linux
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:os.arch=amd64
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:os.version=2.6.32-431.23.3.el6.x86_64
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:user.name=hadoop
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:user.home=/home/hadoop
2015-10-29 10:00:54,533 [myid:] - INFO [Main Thread:Environment@100] - Client environment:user.dir=/home/hadoop/zk2/bin
2015-10-29 10:00:54,536 [myid:] - INFO [Main Thread:ZooKeeper@438] - Initiating client connection, connectString=localhost:7012 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@9a031d8
Welcome to ZooKeeper!
JLine support is enabled
2015-10-29 10:00:54,649 [myid:] - INFO [Main Thread-SendThread(localhost:7012):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:7012. Will not attempt to authenticate using SASL (java.lang.SecurityException: 無(wú)法定位登錄配置)
[zk: localhost:7012(CONNECTING) 0] 2015-10-29 10:00:54,701 [myid:] - INFO [Main Thread-SendThread(localhost:7012):ClientCnxn$SendThread@852] - Socket connection established to localhost/0:0:0:0:0:0:0:1:7012, initiating session
2015-10-29 10:00:54,754 [myid:] - INFO [Main Thread-SendThread(localhost:7012):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:7012, sessionid = 0x450b15327460000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
連接成功锌钮,可以測(cè)試一些基本命令,ls/get/create 等等
Java應(yīng)用中連接zk Server
添加依賴
maven項(xiàng)目中 在pom.xml 添加依賴
<!--zookeeper-->
<dependency>
<groupId>com.github.sgroschupf</groupId>
<artifactId>zkclient</artifactId>
<version>0.1</version>
<scope>${jar-scope}</scope>
</dependency>
示例代碼
@Test
public void testZkClient() {
ZkClient zkClient = new ZkClient("172.28.20.101:7011,172.28.20.101:7012,172.28.20.102:7011");
String node = "/app/local/zkclitest";
if (!zkClient.exists(node)) {
zkClient.createPersistent(node, "hello zk");
}
System.out.println(zkClient.readData(node));
}
在zk服務(wù)器通過(guò)zkCli連接可以看到剛才創(chuàng)建的節(jié)點(diǎn):
[zk: localhost:7011(CONNECTED) 2] ls /app/local
[ciq-web, mfoc-web, zkclitest, ctas-web, ccsp-bill]
[zk: localhost:7011(CONNECTED) 3]
參考資料
http://www.cnblogs.com/sunddenly/p/4033574.html
http://www.cnblogs.com/shanyou/p/3221990.html