背景簡(jiǎn)介
Redis 提供的如下技術(shù)「Redis Sentinel『主從切換』、Redis Cluster『分片』」春寿，有效實(shí)現(xiàn)了 Redis 的高可用朗涩、高性能、高可伸縮性绑改，本文對(duì)以上技術(shù)進(jìn)行親自動(dòng)手實(shí)踐谢床。
1. Redis Sentinel「主從切換」
- 監(jiān)控主從節(jié)點(diǎn)的在線狀態(tài)，并根據(jù)配置自行完成切換「基于raft協(xié)議」厘线。
- 主從復(fù)制從容量角度來說识腿，還是單機(jī)。
2. Redis Cluster「分片」
- 通過一致性 hash 的方式造壮，將數(shù)據(jù)分散到多個(gè)服務(wù)器節(jié)點(diǎn)：設(shè)計(jì)了 16384 個(gè)哈希槽渡讼，并分配到多臺(tái) redis-server。
- 當(dāng)需要在 Redis Cluster 中存取一個(gè) key 時(shí)耳璧，Redis 客戶端先對(duì) key 使用 CRC16 算法計(jì)算一個(gè)數(shù)值成箫，然后對(duì) 16384 取模，這樣每個(gè) key 都會(huì)對(duì)應(yīng)一個(gè)編號(hào)在 0-16383 之間的哈希槽旨枯，然后在此槽對(duì)應(yīng)的節(jié)點(diǎn)上操作蹬昌。
一、主從復(fù)制
設(shè)置詳情
# 已知網(wǎng)關(guān) IP 為：172.17.0.1
# 啟動(dòng) master 節(jié)點(diǎn)
docker run -it --name redis-6380 -p 6380:6379 redis
docker exec -it redis-6380 /bin/bash
redis-cli -h 172.17.0.1 -p 6380
# 啟動(dòng)slave節(jié)點(diǎn) 1
docker run -it --name redis-6381 -p 6381:6379 redis
docker exec -it redis-6381 /bin/bash
redis-cli -h 172.17.0.1 -p 6381
replicaof 172.17.0.1 6380
# 啟動(dòng)slave節(jié)點(diǎn) 2
docker run -it --name redis-6382 -p 6382:6379 redis
docker exec -it redis-6382 /bin/bash
redis-cli -h 172.17.0.1 -p 6382
replicaof 172.17.0.1 6380
之后可查看 master 節(jié)點(diǎn)的信息召廷，在 master-redis 下凳厢，執(zhí)行：
> info Replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.1,port=6379,state=online,offset=686,lag=0
slave1:ip=172.17.0.1,port=6379,state=online,offset=686,lag=1
master_replid:79187e2241015c2f8ed98ce68caafa765796dff2
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:686
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:686
之后操作 master 節(jié)點(diǎn)，slave 節(jié)點(diǎn)會(huì)自動(dòng)同步竞慢。
slave-redis 下執(zhí)行 replicaof no one 可重新改為主節(jié)點(diǎn)先紫。
關(guān)鍵點(diǎn)
- 查看網(wǎng)絡(luò)相關(guān)信息：
docker network ls
docker network inspect bridge
容器之間互訪問，可使用內(nèi)部端口號(hào)筹煮，也可使用外部映射端口號(hào)遮精；
執(zhí)行
docker network inspect bridge后，可查看到網(wǎng)關(guān) IP 以及各容器 IP，可使用網(wǎng)關(guān) IP : 外部映射端口本冲，或容器 IP : 6379訪問 Redis准脂；
參考資料
二、Sentinel 高可用
當(dāng)前狀態(tài)：
- 網(wǎng)關(guān)IP：172.17.0.1
- master端口：6390
- slave端口：6391檬洞，6392
操作步驟
1. 重新創(chuàng)建 redis 的 docker 容器:
redis.conf 配置內(nèi)容如下：
# 默認(rèn)端口6379
port 6390
# 綁定ip狸膏，如果是內(nèi)網(wǎng)可以直接綁定 127.0.0.1, 或者忽略, 0.0.0.0 是外網(wǎng)
bind 0.0.0.0
# 守護(hù)進(jìn)程啟動(dòng)
daemonize no
變更監(jiān)聽端口號(hào)，并重新創(chuàng)建 redis 容器：
docker run -p 6390:6390 -v D:\develop\shell\docker\redis\conf6390:/usr/local/etc/redis --name redis-conf-6390 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6390 /bin/bash
redis-cli -h 172.17.0.1 -p 6390
docker run -p 6391:6391 -v D:\develop\shell\docker\redis\conf6391:/usr/local/etc/redis --name redis-conf-6391 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6391 /bin/bash
redis-cli -h 172.17.0.1 -p 6391
slaveof 172.17.0.1 6390
docker run -p 6392:6392 -v D:\develop\shell\docker\redis\conf6392:/usr/local/etc/redis --name redis-conf-6392 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6392 /bin/bash
redis-cli -h 172.17.0.1 -p 6392
slaveof 172.17.0.1 6390
之后可查看 master 節(jié)點(diǎn)的信息添怔，可看到 master 獲取到的 slave 的端口號(hào)恢復(fù)了正常湾戳。在 master-redis 下，執(zhí)行：
> info Replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.1,port=6391,state=online,offset=84,lag=0
slave1:ip=172.17.0.1,port=6392,state=online,offset=84,lag=0
master_replid:ed2e513ceed2b48a272b97c674c99d82284342a1
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:84
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:84
2. 創(chuàng)建配置文件
創(chuàng)建 sentinel.conf广料，文件中寫入如下內(nèi)容：
sentinel monitor bitkylin-master 172.17.0.1 6390 2
sentinel down-after-milliseconds bitkylin-master 5000
sentinel failover-timeout bitkylin-master 10000
sentinel parallel-syncs bitkylin-master 1
命令詳解：指示 Sentinel 去監(jiān)視一個(gè)名為 bitkylin-master 的主服務(wù)器砾脑，將這個(gè)主服務(wù)器標(biāo)記為客觀下線至少需要 2 個(gè) Sentinel 同意；
響應(yīng)超時(shí) 5 秒標(biāo)記為主觀下線艾杏，主觀下線后就開始了遷移流程韧衣，超時(shí) 10 秒為遷移超時(shí)，暫不知用途购桑。
3. 再創(chuàng)建兩個(gè) redis-docker 容器
將配置文件復(fù)制到 docker 容器內(nèi)畅铭，共兩個(gè)容器需要復(fù)制該文件：
docker run -it --name redis-6490 redis
docker run -it --name redis-6491 redis
docker cp ./sentinel.conf dcbd015dbc0e:/data/sentinel.conf
docker cp ./sentinel.conf 7c8307730bcc:/data/sentinel.conf
4. 執(zhí)行 redis-sentinel 命令
redis-sentinel sentinel.conf
5. 最終效果
此時(shí)任意啟停 redis 容器，可以看到 sentinel 自動(dòng)完成 redis 的主從切換其兴，主從配置等不需要人工操作顶瞒。
參考資料
- Redis 的 Sentinel 文檔
- Docker 容器的文件操作
> 覆蓋寫入; >> 追加寫入
三、Cluster 集群
操作步驟
1. 更新 redis 配置文件
主要追加集群配置信息元旬，示例配置文件如下：
# 默認(rèn)端口6379
port 6390
# 綁定 ip榴徐，如果是內(nèi)網(wǎng)可以直接綁定 127.0.0.1, 或者忽略, 0.0.0.0 是外網(wǎng)
bind 0.0.0.0
# 守護(hù)進(jìn)程啟動(dòng)
daemonize no
# 集群配置
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
2. 創(chuàng)建 6 個(gè)容器
以第二節(jié)作為基礎(chǔ)，基于最新的配置文件匀归，創(chuàng)建 6 個(gè)容器坑资，注意新增集群總線端口映射：
docker run -p 6390:6390 -p 16390:16390 -v D:\develop\shell\docker\redis\conf6390:/usr/local/etc/redis --name redis-conf-6390 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6390 /bin/bash
redis-cli -h 172.17.0.1 -p 6390
docker run -p 6391:6391 -p 16391:16391 -v D:\develop\shell\docker\redis\conf6391:/usr/local/etc/redis --name redis-conf-6391 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6391 /bin/bash
redis-cli -h 172.17.0.1 -p 6391
docker run -p 6392:6392 -p 16392:16392 -v D:\develop\shell\docker\redis\conf6392:/usr/local/etc/redis --name redis-conf-6392 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6392 /bin/bash
redis-cli -h 172.17.0.1 -p 6392
docker run -p 6393:6393 -p 16393:16393 -v D:\develop\shell\docker\redis\conf6393:/usr/local/etc/redis --name redis-conf-6393 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6393 /bin/bash
redis-cli -h 172.17.0.1 -p 6393
docker run -p 6394:6394 -p 16394:16394 -v D:\develop\shell\docker\redis\conf6394:/usr/local/etc/redis --name redis-conf-6394 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6394 /bin/bash
redis-cli -h 172.17.0.1 -p 6394
docker run -p 6395:6395 -p 16395:16395 -v D:\develop\shell\docker\redis\conf6395:/usr/local/etc/redis --name redis-conf-6395 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6395 /bin/bash
redis-cli -h 172.17.0.1 -p 6395
3. 直接通過命令創(chuàng)建集群
> redis-cli --cluster create 172.17.0.1:6390 172.17.0.1:6391 172.17.0.1:6392 172.17.0.1:6393 172.17.0.1:6394 172.17.0.1:6395 --cluster-replicas 1
# 以下是命令執(zhí)行結(jié)果：
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.17.0.1:6394 to 172.17.0.1:6390
Adding replica 172.17.0.1:6395 to 172.17.0.1:6391
Adding replica 172.17.0.1:6393 to 172.17.0.1:6392
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: a9678b062663957e59bc3b4beb7be4366fa24adc 172.17.0.1:6390
slots:[0-5460] (5461 slots) master
M: 41a4976431713cce936220fba8a230627d28d40c 172.17.0.1:6391
slots:[5461-10922] (5462 slots) master
M: 1bf83414a12bad8f2e25dcea19ccea1c881d28c5 172.17.0.1:6392
slots:[10923-16383] (5461 slots) master
S: 3d65eadd3321ef34c9413ae8f75d610c4228eda7 172.17.0.1:6393
replicates 41a4976431713cce936220fba8a230627d28d40c
S: b604356698a5f211823ada4b45a97939744b1d57 172.17.0.1:6394
replicates 1bf83414a12bad8f2e25dcea19ccea1c881d28c5
S: 2c1cc93221dc3830aa1eb28601ac27e22a6801cc 172.17.0.1:6395
replicates a9678b062663957e59bc3b4beb7be4366fa24adc
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.17.0.1:6390)
M: a9678b062663957e59bc3b4beb7be4366fa24adc 172.17.0.1:6390
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: b604356698a5f211823ada4b45a97939744b1d57 172.17.0.1:6394
slots: (0 slots) slave
replicates 1bf83414a12bad8f2e25dcea19ccea1c881d28c5
M: 41a4976431713cce936220fba8a230627d28d40c 172.17.0.1:6391
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 3d65eadd3321ef34c9413ae8f75d610c4228eda7 172.17.0.1:6393
slots: (0 slots) slave
replicates 41a4976431713cce936220fba8a230627d28d40c
M: 1bf83414a12bad8f2e25dcea19ccea1c881d28c5 172.17.0.1:6392
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 2c1cc93221dc3830aa1eb28601ac27e22a6801cc 172.17.0.1:6395
slots: (0 slots) slave
replicates a9678b062663957e59bc3b4beb7be4366fa24adc
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
集群創(chuàng)建成功
注意點(diǎn)
- 需要開放集群總線端口號(hào)，默認(rèn)為
業(yè)務(wù)端口號(hào) + 10000 -
cluster reset命令可以將當(dāng)前節(jié)點(diǎn)從集群中移除
