keepalived是定一個沒有機器再用的ip(也就是ping不通的地址)托修,做虛擬ip来破,這個ip下面綁定幾個真實node的ip徙歼,當(dāng)請求這個虛擬ip的時候就會被轉(zhuǎn)發(fā)到某個子ip機器上蜂大,來實現(xiàn)高可用健提。
Haproxy是類似于nginx的轉(zhuǎn)發(fā)的軟件扎狱。當(dāng)你請求我這里一個端口侧到,我把他轉(zhuǎn)發(fā)到幾個其他的機器的端口勃教。
上面兩個配合使用,keepalived定義一個虛擬ip匠抗,當(dāng)請求過來后到某一個當(dāng)選的節(jié)點故源,然后這個節(jié)點把請求再轉(zhuǎn)發(fā)到這幾個節(jié)點上,來實現(xiàn)高可用汞贸。
yum install -ykeepalived
vim /etc/keepalived/keepalived.conf
修改方法(參考文件在最下面):
-------------------------------------------------------------------
關(guān)鍵的是virtual_ipaddress? 這里需要寫一個虛擬ip地址 找一個你ping 不同的也就是不用的,建議在同一網(wǎng)段绳军,也就是差距不大的比較靠近的如:我的節(jié)點是172.20.0.68,我找了一個這個范圍靠后的172.20.0.251
易錯點interface? 目前ip地址所在的網(wǎng)口
Priority??????? 權(quán)限依次排列不能超過250
State?? ??????? 一個MASTER 其他都是BACKUP
K8s的每個master節(jié)點都要部署
---------------------------------------------------------------
啟動語句:
systemctl daemon-reload && systemctl enable keepalived && systemctl start keepalived && systemctl status keepalived
啟動后矢腻,在master節(jié)點輸入ip address? 查看是否有172.20.0.251/32(注意只有一個節(jié)點能競爭到门驾,其他的節(jié)點就沒有,你可以重啟該節(jié)點多柑,你就會在別的節(jié)點ip address 下看到)
重啟語句:
systemctldaemon-reload && systemctl enable keepalived && systemctlrestart keepalived && systemctl status keepalived
yum install -yhaproxy
vim/etc/haproxy/haproxy.cfg
參考修改:
-------------------------------------------------------------
bind 后面的填入代表了匹配所有發(fā)送到該端口的數(shù)據(jù)包?*:19900?? 但是我們需要它來轉(zhuǎn)發(fā)虛擬ip的數(shù)據(jù)包時奶是,發(fā)往真實地址的數(shù)據(jù)包就不適合被接受所以需要寫成172.20.0.251:19900? 但是這樣需要添加一個配置?讓節(jié)點忽略不需要的數(shù)據(jù)包
cat <? /etc/sysctl.d/k8s.conf
net.ipv4.ip_nonlocal_bind = 1
EOF
sysctl --system
易錯點
Frontend下面的default_backend 需要和backend 一一對應(yīng)同名
我這里配置了兩個端口轉(zhuǎn)發(fā)規(guī)則,一般要一個16443就好顷蟆,19900不需要
對應(yīng)的6443是自己的master節(jié)點
K8s的每個master節(jié)點都要部署
------------------------------------------------------------
啟動語句:
systemctl daemon-reload && systemctl enable haproxy && systemctl start haproxy && systemctl status haproxy
重啟語句
systemctl daemon-reload && systemctl enable haproxy && systemctl restart haproxy && systemctl status haproxy
keepalived.conf
=胗纭!U寿恕V鹞场!O鞣;砩!B辍5橄洹!Q钙辍I种场!G疵铩M憧ァ!RАG远恪!G账5僦稀!!H髯痢Q肀!N撤铩8U辍!MJ俊M诹薄!
! Configuration File for keepalived
global_defs {
? router_id LVS_k8s
}
vrrp_script CheckK8sMaster {
? ? script "killall -0 haproxy"
? ? interval 2
}
vrrp_instance VI_1 {
? ? state MASTER
? ? interface macvlan
? ? virtual_router_id 51
? ? # 主節(jié)點權(quán)重最高 依次減少
? ? priority 150
? ? advert_int 1
? ? #修改為本地IP
? ? #mcast_src_ip 172.20.0.68
? ? authentication {
? ? ? ? auth_type PASS
? ? ? ? auth_pass 1111
? ? }
? ? virtual_ipaddress {
? ? ? ? 172.20.0.251
? ? }
? ? track_script {
? ? ? ? CheckK8sMaster
? ? }
}
A导肌D匆ā!r叩住=颈馈!1「āR鳌!U境M讯琛!A骸@弧!>善颉N等蟆!3咂堋5站馈!Q佣摹;踽恪!Fず!I统佟B藕亍!!Kφ弧P合伞!A棵弧S褡!E固恪>孔ァ!O啤刺下!
haproxy.cfg
!;i佘浴!R陶伞3┳俊!s瘛N膛恕!=钕帧L拼 !7伞R慌颉!H髀佟1鳌!I暄邸B鹘颉!@ㄊO矧健!1舴Fò亍@材ぁ!L视鳌I摇!B闵尽0斯啊!Q乃<〉尽!I怂5破肌!每聪!
global
? ? log? ? ? ? 127.0.0.1 local2
? ? chroot? ? ? /var/lib/haproxy
? ? pidfile? ? /var/run/haproxy.pid
? ? maxconn? ? 4000
? ? user? ? ? ? haproxy
? ? group? ? ? haproxy
? ? daemon
? ? # turn on stats unix socket
? ? stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
? ? mode? ? ? ? ? ? ? ? ? ? http
? ? log? ? ? ? ? ? ? ? ? ? global
? ? option? ? ? ? ? ? ? ? ? httplog
? ? option? ? ? ? ? ? ? ? ? dontlognull
? ? option http-server-close
? ? option forwardfor? ? ? except 127.0.0.0/8
? ? option? ? ? ? ? ? ? ? ? redispatch
? ? retries? ? ? ? ? ? ? ? 3
? ? timeout connect? ? ? ? 10s
? ? timeout client? ? ? ? ? 1m
? ? timeout server? ? ? ? ? 1m
? ? timeout http-keep-alive 10s
? ? timeout check? ? ? ? ? 10s
? ? maxconn? ? ? ? ? ? ? ? 3000
#---------------------------------------------------------------------
# kubernetes apiserver frontend which proxys to the backend
#---------------------------------------------------------------------
frontend kubernetes-apiserver
? ? mode? ? ? ? ? ? ? ? tcp
? ? bind? ? ? ? ? ? ? ? 172.20.0.251:16443
? ? option? ? ? ? ? ? ? tcplog
? ? default_backend? ? ? kubernetes-apiserver
#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend kubernetes-apiserver
? ? mode? ? ? ? tcp
? ? balance? ? roundrobin
? ? server? sz-pg-oam-k8snode-010 172.20.0.71:6443 check
#? ? server? sz-pg-oam-k8snode-009 172.20.0.70:6443 check
? ? server? sz-pg-oam-k8snode-007 172.20.0.68:6443 check
#---------------------------------------------------------------------
# kubernetes apiserver frontend which proxys to the backends
#---------------------------------------------------------------------
frontend ipallocator-server
? ? mode? ? ? ? ? ? ? ? tcp
? ? bind? ? ? ? ? ? ? ? *:19900
? ? option? ? ? ? ? ? ? tcplog
? ? default_backend? ? ? ipallocator-server
#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend ipallocator-server
? ? mode? ? ? ? tcp
? ? balance? ? roundrobin
? ? server? k8snode-010 172.20.0.71:9900 check
#? ? server? k8snode-009 172.20.0.70:9900 check
? ? server? k8snode-007 172.20.0.68:9900 check
#---------------------------------------------------------------------
# collection haproxy statistics message
#---------------------------------------------------------------------
listen stats
? ? bind? ? ? ? ? ? ? ? *:1080
? ? stats auth? ? ? ? ? admin:awesomePassword
? ? stats refresh? ? ? ? 5s
? ? stats realm? ? ? ? ? HAProxy\ Statistics
? ? stats uri? ? ? ? ? ? /admin?statseck