6.2 實(shí)現(xiàn)Nginx TCP負(fù)載均衡
Nginx在1.9.0版本開始支持tcp模式的負(fù)載均衡, 在1.9.13版本開始支持udp協(xié)議的負(fù)載均衡, udp主要用于DNS的域名解析, 其配置方式和指令和http代理類似, 其基于ngx_stream_proxy_module模塊實(shí)現(xiàn)tcp負(fù)載, 另外基于模塊ngx_http_upstream_module實(shí)現(xiàn)后端服務(wù)器分組轉(zhuǎn)發(fā), 權(quán)重分配, 狀態(tài)監(jiān)測(cè), 調(diào)度算法等高級(jí)功能
如果是編譯安裝, 需要指定--with-stream選項(xiàng)才能支持ngx_stream_proxy_module模塊
6.2.1 tcp 負(fù)載均衡配置參數(shù)
stream { # 定義stream相關(guān)的服務(wù), 定義在main語(yǔ)句塊和http平級(jí)
upstream backend { # 定義后端服務(wù)器
hash $remote_addr consistent; # 定義調(diào)度算法
server backend1.example.com:12345 weight=5; # 定義具體server
server 127.0.0.1:12345 max_fails=3 fail_timeout=30s;
server unix:/tmp/backend3;
}
upstream dns { # 定義后端服務(wù)器
server 10.0.0.1:53535; # 定義具體server
server dns.example.com:53;
}
server { # 定義server, 定義nginx如何接收和轉(zhuǎn)發(fā)遠(yuǎn)程用戶的請(qǐng)求
listen 12345; # 監(jiān)聽ip:port
proxy_connect_timeout 1s; # 連接超時(shí)時(shí)間
proxy_timeout 3s; # 轉(zhuǎn)發(fā)超時(shí)時(shí)間
proxy_pass backend; # 轉(zhuǎn)發(fā)到具體服務(wù)器組
}
server {
listen 127.0.0.1:53 udp reuseport;
proxy_timeout 20s;
proxy_pass dns;
}
server {
listen [::1]:12345;
proxy_pass unix:/tmp/stream.socket;
}
}
6.2.2 負(fù)載均衡案例: Redis
6.2.2.1 準(zhǔn)備環(huán)境
準(zhǔn)備兩臺(tái)后端服務(wù)器, 安裝redis
可以基于每臺(tái)服務(wù)器的run_id測(cè)試調(diào)度結(jié)果
yum -y install redis
vim /etc/redis.conf
bind 0.0.0.0
systemctl enable --now redis
6.2.2.2 nginx配置負(fù)載均衡
# 先在nginx主配置文件定義tcp負(fù)載均衡的子配置文件路徑
[13:50:26 root@nginx ~]#vim /apps/nginx/conf/nginx.conf
include /apps/nginx/conf/tcp/tcp.conf; # 注意: tcp負(fù)載均衡的子配置文件路徑指令要和http語(yǔ)句塊平級(jí), 一般寫在主配置文件最后即可, 由于stream是一級(jí)語(yǔ)句塊, 整個(gè)nginx只能有一個(gè), 因此, 所有的tcp/udp負(fù)載均衡配置, 都寫在同一個(gè)文件里
[00:30:17 root@nginx /apps/nginx/conf/tcp]#vim redis.conf
stream {
upstream redis-server {
server 10.0.0.85:6379 weight=2;
server 10.0.0.84:6379 weight=1;
}
server{
# 配置負(fù)載均衡時(shí), server語(yǔ)句塊不支持server_name, 客戶端直接訪問nginx的ip地址和端口
listen 6379;
proxy_pass redis-server;
}
}
[00:35:51 root@nginx /apps/nginx/conf/tcp]#nginx -s reload
[13:51:16 root@nginx ~]#ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 0.0.0.0:6379 # 驗(yàn)證監(jiān)聽6379端口 0.0.0.0:*
LISTEN 0 128 0.0.0.0:80 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
6.2.2.3 客戶端連接nginx測(cè)試
# 可以看到1:2調(diào)度
[00:38:02 root@client ~]#redis-cli -h 10.0.0.86 info server | grep run_id
run_id:6c86d33bf1002c7839f43a6ac9055f0406d61b7a
[00:38:25 root@client ~]#redis-cli -h 10.0.0.86 info server | grep run_id
run_id:580343970ae2d5078ff064260e170a8932a48338
[00:38:27 root@client ~]#redis-cli -h 10.0.0.86 info server | grep run_id
run_id:6c86d33bf1002c7839f43a6ac9055f0406d61b7a
[00:38:28 root@client ~]#redis-cli -h 10.0.0.86 info server | grep run_id
run_id:6c86d33bf1002c7839f43a6ac9055f0406d61b7a
6.2.3 負(fù)載均衡案例: MySQL
圖片.png
6.2.3.1 準(zhǔn)備環(huán)境
為了測(cè)試效果, 后端分別使用mariadb和mysql
數(shù)據(jù)庫(kù)上創(chuàng)建用戶, 需要對(duì)nginx的ip地址進(jìn)行授權(quán)
nginx基于四層的調(diào)度是偽四層, 因此nginx會(huì)拆分請(qǐng)求和響應(yīng)報(bào)文, 重新封裝
# 10.0.0.85 - MySQL
[23:34:16 root@mysql ~]#yum -y install mysql-server
[23:35:30 root@mysql ~]#systemctl enable --now mysqld
Created symlink /etc/systemd/system/multi-user.target.wants/mysqld.service → /usr/lib/systemd/system/mysqld.service.
[23:42:31 root@mysql ~]#mysql -e 'create user admin@"10.0.0.%" identified by "000000"'
# 10.0.0.84 - MariaDB
[23:34:41 root@mariadb ~]#yum -y install mariadb-server
[23:35:13 root@mariadb ~]#systemctl enable --now mariadb
Created symlink /etc/systemd/system/mysql.service → /usr/lib/systemd/system/mariadb.service.
Created symlink /etc/systemd/system/mysqld.service → /usr/lib/systemd/system/mariadb.service.
Created symlink /etc/systemd/system/multi-user.target.wants/mariadb.service → /usr/lib/systemd/system/mariadb.service.
[23:44:03 root@mariadb ~]#mysql -e 'create user admin@"10.0.0.%" identified by "000000"'
6.2.3.2 nginx配置負(fù)載均衡
# 單獨(dú)指定存放tcp負(fù)載均衡的配置文件, 關(guān)于tcp的負(fù)載均衡配置, 都可以寫到tcp目錄下某個(gè).conf文件里, 因?yàn)閟tream是一級(jí)語(yǔ)句塊, 一個(gè)服務(wù)器只能有一個(gè), 因此只能存在一個(gè).conf配置文件, 專門來(lái)配置tcp負(fù)載均衡, 如果多個(gè)文件都有stream模塊, 那么重啟nginx會(huì)報(bào)錯(cuò)
include /apps/nginx/conf/tcp/*.conf;
[23:47:05 root@nginx /apps/nginx/conf]#mkdir /apps/nginx/conf/tcp
[23:47:44 root@nginx /apps/nginx/conf]#cd /apps/nginx/conf/tcp
[23:47:45 root@nginx /apps/nginx/conf/tcp]#
[23:47:46 root@nginx /apps/nginx/conf/tcp]#
[23:47:46 root@nginx /apps/nginx/conf/tcp]#vim mysql.conf
[23:47:46 root@nginx /apps/nginx/conf/tcp]#vim mysql.conf
stream {
upstream mysql-server { # upstream定義后端服務(wù)器組
server 10.0.0.85:3306 max_fails=3 fail_timeout=30s;
server 10.0.0.84:3306; # 后端服務(wù)器監(jiān)聽的端口
}
server { # server定義nginx如何接收和處理客戶端請(qǐng)求, 包括監(jiān)聽的端口, 和轉(zhuǎn)發(fā)到的后端群組
listen 3306; # 面對(duì)客戶端的端口
proxy_pass mysql-server;
}
}
[23:59:02 root@nginx /apps/nginx/conf/tcp]#nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[23:59:05 root@nginx /apps/nginx/conf/tcp]#nginx -s reload
[23:59:10 root@nginx /apps/nginx/conf/tcp]#ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 0.0.0.0:3306 0.0.0.0:*
LISTEN 0 128 0.0.0.0:80 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
6.2.3.3 客戶端連接nginx測(cè)試
# 10.0.0.187安裝mysql客戶端
[00:00:27 root@client ~]#yum -y install mysql
# 連接nginx
[00:00:44 root@client ~]#mysql -h 10.0.0.86 -uadmin -p000000
MySQL [(none)]>
# 驗(yàn)證版本
圖片.png
此時(shí)后端服務(wù)器會(huì)認(rèn)為是nginx在訪問, 因此數(shù)據(jù)庫(kù)授權(quán)也是針對(duì)nginx進(jìn)行授權(quán)
圖片.png
通過nginx的tcp負(fù)載均衡, 可以實(shí)現(xiàn)對(duì)后端的PXC集群進(jìn)行調(diào)度
圖片.png
補(bǔ)充: lvs和nginx四層負(fù)載均衡的區(qū)別
nginx對(duì)于tcp的負(fù)載均衡實(shí)際是偽四層, 因?yàn)閚ginx會(huì)代替客戶端向后端服務(wù)器發(fā)起請(qǐng)求, 而不是單獨(dú)的轉(zhuǎn)發(fā), 后端服務(wù)器看到的是nginx服務(wù)器的ip地址
lvs對(duì)于tcp的負(fù)載均衡是只轉(zhuǎn)發(fā), 后端服務(wù)器看到的是客戶端的源ip地址
