個(gè)人專(zhuān)題目錄

4.1 分布式安裝部署

1．集群規(guī)劃
在hadoop102、hadoop103和hadoop104三個(gè)節(jié)點(diǎn)上部署Zookeeper。
2．解壓安裝
（1）解壓Zookeeper安裝包到/opt/module/目錄下
$ tar -zxvf zookeeper-3.4.10.tar.gz -C /opt/module/
（2）同步/opt/module/zookeeper-3.4.10目錄內(nèi)容到hadoop103玖院、hadoop104
$ xsync zookeeper-3.4.10/
3．配置服務(wù)器編號(hào)
（1）在/opt/module/zookeeper-3.4.10/這個(gè)目錄下創(chuàng)建zkData
$ mkdir -p zkData
（2）在/opt/module/zookeeper-3.4.10/zkData目錄下創(chuàng)建一個(gè)myid的文件
$ touch myid
添加myid文件，注意一定要在linux里面創(chuàng)建第岖，在notepad++里面很可能亂碼
（3）編輯myid文件
$ vi myid
    在文件中添加與server對(duì)應(yīng)的編號(hào)：2
（4）拷貝配置好的zookeeper到其他機(jī)器上
$ xsync myid
并分別在hadoop102难菌、hadoop103上修改myid文件中內(nèi)容為3、4
4．配置zoo.cfg文件
（1）重命名/opt/module/zookeeper-3.4.10/conf這個(gè)目錄下的zoo_sample.cfg為zoo.cfg
$ mv zoo_sample.cfg zoo.cfg
（2）打開(kāi)zoo.cfg文件
$ vim zoo.cfg
修改數(shù)據(jù)存儲(chǔ)路徑配置
dataDir=/opt/module/zookeeper-3.4.10/zkData
增加如下配置
#######################cluster##########################
server.2=hadoop102:2888:3888
server.3=hadoop103:2888:3888
server.4=hadoop104:2888:3888
（3）同步zoo.cfg配置文件
$ xsync zoo.cfg
（4）配置參數(shù)解讀
server.A=B:C:D蔑滓。
A是一個(gè)數(shù)字郊酒，表示這個(gè)是第幾號(hào)服務(wù)器；
集群模式下配置一個(gè)文件myid键袱，這個(gè)文件在dataDir目錄下燎窘，這個(gè)文件里面有一個(gè)數(shù)據(jù)就是A的值，Zookeeper啟動(dòng)時(shí)讀取此文件杠纵，拿到里面的數(shù)據(jù)與zoo.cfg里面的配置信息比較從而判斷到底是哪個(gè)server荠耽。
B是這個(gè)服務(wù)器的ip地址；
C是這個(gè)服務(wù)器與集群中的Leader服務(wù)器交換信息的端口比藻；
D是萬(wàn)一集群中的Leader服務(wù)器掛了铝量，需要一個(gè)端口來(lái)重新進(jìn)行選舉，選出一個(gè)新的Leader银亲，而這個(gè)端口就是用來(lái)執(zhí)行選舉時(shí)服務(wù)器相互通信的端口慢叨。
4．集群操作
（1）分別啟動(dòng)Zookeeper
$ bin/zkServer.sh start
（2）查看狀態(tài)
bin/zkServer.sh status
JMX enabled by default
Using config: /opt/module/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower
bin/zkServer.sh status
JMX enabled by default
Using config: /opt/module/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader
bin/zkServer.sh status
JMX enabled by default
Using config: /opt/module/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

4.2 客戶(hù)端命令行操作

Zookeeper表現(xiàn)為一個(gè)分層的文件系統(tǒng)目錄樹(shù)結(jié)構(gòu)
不同于文件系統(tǒng)之處在于：zk節(jié)點(diǎn)可以有自己的數(shù)據(jù)，而unix文件系統(tǒng)中的目錄節(jié)點(diǎn)只有子節(jié)點(diǎn)

一個(gè)節(jié)點(diǎn)對(duì)應(yīng)一個(gè)應(yīng)用/服務(wù)务蝠，節(jié)點(diǎn)存儲(chǔ)的數(shù)據(jù)就是應(yīng)用需要的配置信息拍谐。

1．啟動(dòng)客戶(hù)端
$ bin/zkCli.sh
2．顯示所有操作命令
[zk: localhost:2181(CONNECTED) 1] help
3．查看當(dāng)前znode中所包含的內(nèi)容
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
4．查看當(dāng)前節(jié)點(diǎn)詳細(xì)數(shù)據(jù)
[zk: localhost:2181(CONNECTED) 1] ls2 /
[zookeeper]
cZxid = 0x0
ctime = Thu Jan 01 08:00:00 CST 1970
mZxid = 0x0
mtime = Thu Jan 01 08:00:00 CST 1970
pZxid = 0x0
cversion = -1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 1
5．分別創(chuàng)建2個(gè)普通節(jié)點(diǎn)
[zk: localhost:2181(CONNECTED) 3] create /sanguo "jinlian"
Created /sanguo
[zk: localhost:2181(CONNECTED) 4] create /sanguo/shuguo "liubei"
Created /sanguo/shuguo
6．獲得節(jié)點(diǎn)的值
[zk: localhost:2181(CONNECTED) 5] get /sanguo
jinlian
cZxid = 0x100000003
ctime = Wed Aug 29 00:03:23 CST 2018
mZxid = 0x100000003
mtime = Wed Aug 29 00:03:23 CST 2018
pZxid = 0x100000004
cversion = 1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 7
numChildren = 1
[zk: localhost:2181(CONNECTED) 6]
[zk: localhost:2181(CONNECTED) 6] get /sanguo/shuguo
liubei
cZxid = 0x100000004
ctime = Wed Aug 29 00:04:35 CST 2018
mZxid = 0x100000004
mtime = Wed Aug 29 00:04:35 CST 2018
pZxid = 0x100000004
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0
7．創(chuàng)建短暫節(jié)點(diǎn)
[zk: localhost:2181(CONNECTED) 7] create -e /sanguo/wuguo "zhouyu"
Created /sanguo/wuguo
（1）在當(dāng)前客戶(hù)端是能查看到的
[zk: localhost:2181(CONNECTED) 3] ls /sanguo 
[wuguo, shuguo]
（2）退出當(dāng)前客戶(hù)端然后再重啟客戶(hù)端
[zk: localhost:2181(CONNECTED) 12] quit
$ bin/zkCli.sh
（3）再次查看根目錄下短暫節(jié)點(diǎn)已經(jīng)刪除
[zk: localhost:2181(CONNECTED) 0] ls /sanguo
[shuguo]
8．創(chuàng)建帶序號(hào)的節(jié)點(diǎn)
    （1）先創(chuàng)建一個(gè)普通的根節(jié)點(diǎn)/sanguo/weiguo
[zk: localhost:2181(CONNECTED) 1] create /sanguo/weiguo "caocao"
Created /sanguo/weiguo
    （2）創(chuàng)建帶序號(hào)的節(jié)點(diǎn)
[zk: localhost:2181(CONNECTED) 2] create -s /sanguo/weiguo/xiaoqiao "jinlian"
Created /sanguo/weiguo/xiaoqiao0000000000
[zk: localhost:2181(CONNECTED) 3] create -s /sanguo/weiguo/daqiao "jinlian"
Created /sanguo/weiguo/daqiao0000000001
[zk: localhost:2181(CONNECTED) 4] create -s /sanguo/weiguo/diaocan "jinlian"
Created /sanguo/weiguo/diaocan0000000002
如果原來(lái)沒(méi)有序號(hào)節(jié)點(diǎn)，序號(hào)從0開(kāi)始依次遞增馏段。如果原節(jié)點(diǎn)下已有2個(gè)節(jié)點(diǎn)轩拨，則再排序時(shí)從2開(kāi)始，以此類(lèi)推院喜。
9．修改節(jié)點(diǎn)數(shù)據(jù)值
[zk: localhost:2181(CONNECTED) 6] set /sanguo/weiguo "simayi"
10．節(jié)點(diǎn)的值變化監(jiān)聽(tīng)
    （1）在hadoop104主機(jī)上注冊(cè)監(jiān)聽(tīng)/sanguo節(jié)點(diǎn)數(shù)據(jù)變化
[zk: localhost:2181(CONNECTED) 26] [zk: localhost:2181(CONNECTED) 8] get /sanguo watch
    （2）在hadoop103主機(jī)上修改/sanguo節(jié)點(diǎn)的數(shù)據(jù)
[zk: localhost:2181(CONNECTED) 1] set /sanguo "xisi"
    （3）觀(guān)察hadoop104主機(jī)收到數(shù)據(jù)變化的監(jiān)聽(tīng)
WATCHER::
WatchedEvent state:SyncConnected type:NodeDataChanged path:/sanguo
11．節(jié)點(diǎn)的子節(jié)點(diǎn)變化監(jiān)聽(tīng)（路徑變化）
    （1）在hadoop104主機(jī)上注冊(cè)監(jiān)聽(tīng)/sanguo節(jié)點(diǎn)的子節(jié)點(diǎn)變化
[zk: localhost:2181(CONNECTED) 1] ls /sanguo watch
[aa0000000001, server101]
    （2）在hadoop103主機(jī)/sanguo節(jié)點(diǎn)上創(chuàng)建子節(jié)點(diǎn)
[zk: localhost:2181(CONNECTED) 2] create /sanguo/jin "simayi"
Created /sanguo/jin
    （3）觀(guān)察hadoop104主機(jī)收到子節(jié)點(diǎn)變化的監(jiān)聽(tīng)
WATCHER::
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/sanguo
12．刪除節(jié)點(diǎn)
[zk: localhost:2181(CONNECTED) 4] delete /sanguo/jin
13．遞歸刪除節(jié)點(diǎn)
[zk: localhost:2181(CONNECTED) 15] rmr /sanguo/shuguo
14．查看節(jié)點(diǎn)狀態(tài)
[zk: localhost:2181(CONNECTED) 17] stat /sanguo
cZxid = 0x100000003
ctime = Wed Aug 29 00:03:23 CST 2018
mZxid = 0x100000011
mtime = Wed Aug 29 00:21:23 CST 2018
pZxid = 0x100000014
cversion = 9
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 4
numChildren = 1

zookeeper支持某些特定的四字命令亡蓉，他們大多是用來(lái)查詢(xún)ZK服務(wù)的當(dāng)前狀態(tài)及相關(guān)信息的，
通過(guò)telnet或nc向zookeeper提交相應(yīng)命令喷舀，如：echo ruok | nc 127.0.0.1 2181

運(yùn)行公式：echo 四字命令 | nc 主機(jī)IP zookeeper端口

ruok：測(cè)試服務(wù)是否處于正確狀態(tài)砍濒。如果確實(shí)如此淋肾，那么服務(wù)返回“imok ”，否則不做任何相應(yīng)
stat：輸出關(guān)于性能和連接的客戶(hù)端的列表
conf：輸出相關(guān)服務(wù)配置的詳細(xì)信息
cons：列出所有連接到服務(wù)器的客戶(hù)端的完全的連接 /會(huì)話(huà)的詳細(xì)信息爸邢。包括“接受 / 發(fā)送”的包數(shù)量樊卓、會(huì)話(huà)id 、操作延遲杠河、最后的操作執(zhí)行等等信息
dump：列出未經(jīng)處理的會(huì)話(huà)和臨時(shí)節(jié)點(diǎn)
envi：輸出關(guān)于服務(wù)環(huán)境的詳細(xì)信息（區(qū)別于conf命令）
reqs：列出未經(jīng)處理的請(qǐng)求
wchs：列出服務(wù)器watch的詳細(xì)信息
wchc：通過(guò)session列出服務(wù)器watch的詳細(xì)信息碌尔，它的輸出是一個(gè)與watch相關(guān)的會(huì)話(huà)的列表
wchp：通過(guò)路徑列出服務(wù)器 watch的詳細(xì)信息。它輸出一個(gè)與 session相關(guān)的路徑

4.3 API應(yīng)用

4.3.1 環(huán)境搭建

1．創(chuàng)建一個(gè)Maven工程
2．添加pom文件
<dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.8.2</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper -->
        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.4.10</version>
        </dependency>
</dependencies>
3．拷貝log4j.properties文件到項(xiàng)目根目錄
需要在項(xiàng)目的src/main/resources目錄下感猛，新建一個(gè)文件七扰，命名為“l(fā)og4j.properties”，在文件中填入陪白。
log4j.rootLogger=INFO, stdout  
log4j.appender.stdout=org.apache.log4j.ConsoleAppender  
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout  
log4j.appender.stdout.layout.ConversionPattern=%d %p [%c] - %m%n  
log4j.appender.logfile=org.apache.log4j.FileAppender  
log4j.appender.logfile.File=target/spring.log  
log4j.appender.logfile.layout=org.apache.log4j.PatternLayout  
log4j.appender.logfile.layout.ConversionPattern=%d %p [%c] - %m%n  

4.3.2 創(chuàng)建ZooKeeper客戶(hù)端

private static String connectString =
 "hadoop102:2181,hadoop103:2181,hadoop104:2181";
    private static int sessionTimeout = 2000;
    private ZooKeeper zkClient = null;

    @Before
    public void init() throws Exception {

    zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher() {

            @Override
            public void process(WatchedEvent event) {

                // 收到事件通知后的回調(diào)函數(shù)（用戶(hù)的業(yè)務(wù)邏輯）
                System.out.println(event.getType() + "--" + event.getPath());

                // 再次啟動(dòng)監(jiān)聽(tīng)
                try {
                    zkClient.getChildren("/", true);
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        });
    }

4.3.3 創(chuàng)建子節(jié)點(diǎn)

// 創(chuàng)建子節(jié)點(diǎn)
@Test
public void create() throws Exception {
        // 參數(shù)1：要?jiǎng)?chuàng)建的節(jié)點(diǎn)的路徑颈走； 參數(shù)2：節(jié)點(diǎn)數(shù)據(jù) ； 參數(shù)3：節(jié)點(diǎn)權(quán)限 咱士；參數(shù)4：節(jié)點(diǎn)的類(lèi)型
        String nodeCreated = zkClient.create("/test", "jinlian".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
}

4.3.4 獲取子節(jié)點(diǎn)并監(jiān)聽(tīng)節(jié)點(diǎn)變化

// 獲取子節(jié)點(diǎn)
@Test
public void getChildren() throws Exception {

     List<String> children = zkClient.getChildren("/", true);

     for (String child : children) {
         System.out.println(child);
     }

     // 延時(shí)阻塞
     Thread.sleep(Long.MAX_VALUE);
}

4.3.5 判斷Znode是否存在

// 判斷znode是否存在
@Test
public void exist() throws Exception {

  Stat stat = zkClient.exists("/eclipse", false);

  System.out.println(stat == null ? "not exist" : "exist");
}

4.4 監(jiān)聽(tīng)服務(wù)器節(jié)點(diǎn)動(dòng)態(tài)上下線(xiàn)案例

具體實(shí)現(xiàn)
（0）先在集群上創(chuàng)建/servers節(jié)點(diǎn)
[zk: localhost:2181(CONNECTED) 10] create /servers "servers"
Created /servers
（1）服務(wù)器端向Zookeeper注冊(cè)代碼
package com.test.zkcase;
import java.io.IOException;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.ZooDefs.Ids;

public class DistributeServer {

    private static String connectString = "hadoop102:2181,hadoop103:2181,hadoop104:2181";
    private static int sessionTimeout = 2000;
    private ZooKeeper zk = null;
    private String parentNode = "/servers";
    
    // 創(chuàng)建到zk的客戶(hù)端連接
    public void getConnect() throws IOException{
        
        zk = new ZooKeeper(connectString, sessionTimeout, new Watcher() {

            @Override
            public void process(WatchedEvent event) {

            }
        });
    }
    
    // 注冊(cè)服務(wù)器
    public void registServer(String hostname) throws Exception{

        String create = zk.create(parentNode + "/server", hostname.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
        
        System.out.println(hostname +" is online "+ create);
    }
    
    // 業(yè)務(wù)功能
    public void business(String hostname) throws Exception{
        System.out.println(hostname+" is working ...");
        
        Thread.sleep(Long.MAX_VALUE);
    }
    
    public static void main(String[] args) throws Exception {
        
// 1獲取zk連接
        DistributeServer server = new DistributeServer();
        server.getConnect();
        
        // 2 利用zk連接注冊(cè)服務(wù)器信息
        server.registServer(args[0]);
        
        // 3 啟動(dòng)業(yè)務(wù)功能
        server.business(args[0]);
    }
}
（2）客戶(hù)端代碼
package com.test.zkcase;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;

public class DistributeClient {

    private static String connectString = "hadoop102:2181,hadoop103:2181,hadoop104:2181";
    private static int sessionTimeout = 2000;
    private ZooKeeper zk = null;
    private String parentNode = "/servers";

    // 創(chuàng)建到zk的客戶(hù)端連接
    public void getConnect() throws IOException {
        zk = new ZooKeeper(connectString, sessionTimeout, new Watcher() {

            @Override
            public void process(WatchedEvent event) {

                // 再次啟動(dòng)監(jiān)聽(tīng)
                try {
                    getServerList();
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        });
    }

    // 獲取服務(wù)器列表信息
    public void getServerList() throws Exception {
        
        // 1獲取服務(wù)器子節(jié)點(diǎn)信息立由，并且對(duì)父節(jié)點(diǎn)進(jìn)行監(jiān)聽(tīng)
        List<String> children = zk.getChildren(parentNode, true);

        // 2存儲(chǔ)服務(wù)器信息列表
        ArrayList<String> servers = new ArrayList<>();
        
        // 3遍歷所有節(jié)點(diǎn)，獲取節(jié)點(diǎn)中的主機(jī)名稱(chēng)信息
        for (String child : children) {
            byte[] data = zk.getData(parentNode + "/" + child, false, null);

            servers.add(new String(data));
        }

        // 4打印服務(wù)器列表信息
        System.out.println(servers);
    }

    // 業(yè)務(wù)功能
    public void business() throws Exception{

        System.out.println("client is working ...");
Thread.sleep(Long.MAX_VALUE);
    }

    public static void main(String[] args) throws Exception {

        // 1獲取zk連接
        DistributeClient client = new DistributeClient();
        client.getConnect();

        // 2獲取servers的子節(jié)點(diǎn)信息序厉，從中獲取服務(wù)器信息列表
        client.getServerList();

        // 3業(yè)務(wù)進(jìn)程啟動(dòng)
        client.business();
    }
}