云服務(wù)器-異地部署集群服務(wù)-Kubernetes(K8S)-網(wǎng)絡(luò)篇

一币励、環(huán)境

之前在騰訊云買了臺(tái)學(xué)生機(jī)慷蠕，后面又在阿里買了一臺(tái)，主要是便宜食呻，然后由于網(wǎng)絡(luò)不好流炕，又買了臺(tái)境外的搶占式實(shí)例。后來(lái)部署k8s集群的時(shí)候發(fā)現(xiàn)仅胞，網(wǎng)卡上綁定的地址不是公網(wǎng)IP每辟，而應(yīng)用只能綁定網(wǎng)卡上的地址。但是私網(wǎng)IP之間又不通干旧，這就和網(wǎng)上的教程有些出入了渠欺，一時(shí)有些懵逼，最后去看文檔發(fā)現(xiàn)其實(shí)官方是支持這種方式部署的椎眯，只要改下配置就行了

1.1 k8s版本列表
軟件 版本
CentOS 8.0
Kubernetes v1.19.0-alpha.1
Docker 19.03.8
Etcd v3.4.7
Flannel v0.12.0
云服務(wù)商 主機(jī)名 公網(wǎng)ip/私網(wǎng)ip 組件 推薦配置
阿里云 master 47.241.67.61 172.21.221.58 kube-apiserver kube-controller-manager kube-scheduler etcd flannel docker 2C2G
阿里云 work02 39.100.145.150 172.26.95.163 kube-proxy kube-proxy docker flannel etcd 2C1G
騰訊云 work01 175.24.19.25 172.17.0.13 kube-proxy kube-proxy docker flannel etcd 1C2G
image-20200402204842088.png

由于是個(gè)人練習(xí)使用挠将，所以用的都是最新版的，部署方式是下載二進(jìn)制部署编整。

部署過(guò)程中參考了很多人的教程

1.1 修改主機(jī)名

  • master

    hostnamectl set-hostname master

  • work01

    hostnamectl set-hostname work01

  • work02

    hostnamectl set-hostname work02

最后重新登錄或直接使用

1.2 修改hosts

master work01 work02都執(zhí)行

cat  >> /etc/hosts <<EOF
47.241.67.61  master
175.24.19.25 work01
39.100.145.150 work02
EOF
1.3 關(guān)閉SElinux

修改/etc/selinux/config文件

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
1.4 關(guān)閉防火墻

由于云服務(wù)器默認(rèn)已經(jīng)關(guān)閉防火墻捐名，這里就不再重復(fù)

二、ETCD集群部署

2.1 創(chuàng)建ETCD證書

對(duì)證書這塊我還是有點(diǎn)不太清楚闹击，反正按教程來(lái)镶蹋，里面步驟講的很詳細(xì)

kubernetes1.13.1+etcd3.3.10+flanneld0.10集群部署

1）cfssl安裝

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo

2）存放目錄創(chuàng)建

mkdir /k8s/etcd/{bin,cfg,ssl} -p
mkdir /k8s/kubernetes/{bin,cfg,ssl} -p
cd /k8s/etcd/ssl/

3）ETCD CA配置

cat << EOF | tee ca-config.json
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "etcd": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF

4）ETCD CA證書

cat << EOF | tee ca-csr.json
{
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}
EOF

5）ETCD Server證書

不太懂，所以只能把所有的IP都放進(jìn)去了，省得麻煩

cat << EOF | tee server-csr.json
{
    "CN": "etcd",
    "hosts": [
    "47.241.67.61",
    "39.100.145.150",
    "175.24.19.25",
    "172.21.221.58",
    "172.17.0.13",
    "172.26.95.163",
    "127.0.0.1"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}
EOF

6） 生成ETCD CA證書和私鑰

cfssl gencert -initca ca-csr.json | cfssljson -bare ca

7）生成ETCD Server證書

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd server-csr.json | cfssljson -bare server

8）將生成的證書分發(fā)到其它節(jié)點(diǎn)

2.2 ETCD 下載

1）下載

[root@master ~]# wget https://github.com/etcd-io/etcd/releases/download/v3.4.7/etcd-v3.4.7-linux-amd64.tar.gz

強(qiáng)烈建議買個(gè)境外的服務(wù)器贺归，下載速度賊快

2）解壓淆两，復(fù)制到指定目錄，可以自定義

[root@master ~]# tar -xvf etcd-v3.4.7-linux-amd64.tar.gz
[root@master ~]# cd etcd-v3.4.7-linux-amd64/
[root@master etcd-v3.4.7-linux-amd64]# cp etcd etcdctl /k8s/etcd/bin/

3）可執(zhí)行文件發(fā)送到其它節(jié)點(diǎn)（先在各節(jié)點(diǎn)把目錄創(chuàng)建好）

[root@master ~]# scp -r /k8s/etcd/bin/ root@work01:/k8s/etcd/bin/
[root@master ~]# scp -r /k8s/etcd/bin/ root@work02:/k8s/etcd/bin/
2.3 ETCD 配置（重點(diǎn)）

這里我使用ETCD給的公共發(fā)現(xiàn)服務(wù)拂酣，可以節(jié)省一點(diǎn)配置

1）獲取private discovery URL

[root@master ~]# curl https://discovery.etcd.io/new?size=3
https://discovery.etcd.io/4636d0525ea552bb567fa3f8c59312f8

$ curl https://discovery.etcd.io/new?size=3
https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de

后面的size=3表示創(chuàng)建集群的初始大小為3

3

2）添加ETCD配置文件(三個(gè)節(jié)點(diǎn)都要做)

[root@master ~]# mkdir /data1/etcd
[root@master ~]# vim /k8s/etcd/cfg/etcd.conf

master配置


#[Member]
ETCD_NAME="etcd01"
ETCD_DATA_DIR="/data1/etcd"

ETCDCTL_API="2"
ETCD_ENABLE_V2="true"

ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://47.241.67.61:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://47.241.67.61:2379,https://127.0.0.1:2379"

#[discovery]
ETCD_DISCOVERY="https://discovery.etcd.io/4636d0525ea552bb567fa3f8c59312f8"

ETCD_CERT_FILE="/k8s/etcd/ssl/server.pem"
ETCD_KEY_FILE="/k8s/etcd/ssl/server-key.pem"
ETCD_TRUSTED_CA_FILE="/k8s/etcd/ssl/ca.pem"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_PEER_CERT_FILE="/k8s/etcd/ssl/server.pem"
ETCD_PEER_KEY_FILE="/k8s/etcd/ssl/server-key.pem"
ETCD_PEER_TRUSTED_CA_FILE="/k8s/etcd/ssl/ca.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"

注意 ETCDCTL_API="2",ETCD_ENABLE_V2="true"這兩條加上秋冰，因?yàn)閒lannel不支持API3，而新版ETCD貌似默認(rèn)不啟用API2婶熬，所以要在配置處打開(kāi)

因?yàn)?code>ETCD

work01配置

#[Member]
ETCD_NAME="etcd02"
ETCD_DATA_DIR="/data1/etcd"
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"

ETCDCTL_API="2"
ETCD_ENABLE_V2="true"
 
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://175.24.19.25:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://175.24.19.25:2379,https://127.0.0.1:2379"

#[discovery]
ETCD_DISCOVERY="https://discovery.etcd.io/4636d0525ea552bb567fa3f8c59312f8"

ETCD_CERT_FILE="/k8s/etcd/ssl/server.pem"
ETCD_KEY_FILE="/k8s/etcd/ssl/server-key.pem"
ETCD_TRUSTED_CA_FILE="/k8s/etcd/ssl/ca.pem"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_PEER_CERT_FILE="/k8s/etcd/ssl/server.pem"
ETCD_PEER_KEY_FILE="/k8s/etcd/ssl/server-key.pem"
ETCD_PEER_TRUSTED_CA_FILE="/k8s/etcd/ssl/ca.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"

work02配置

#[Member]
ETCD_NAME="etcd03"
ETCD_DATA_DIR="/data1/etcd"
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"

ETCDCTL_API="2"
ETCD_ENABLE_V2="true"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://39.100.145.150:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://39.100.145.150:2379,https://127.0.0.1:2379"

#[discovery]
ETCD_DISCOVERY="https://discovery.etcd.io/4636d0525ea552bb567fa3f8c59312f8"

#[Security]
ETCD_CERT_FILE="/k8s/etcd/ssl/server.pem"
ETCD_KEY_FILE="/k8s/etcd/ssl/server-key.pem"
ETCD_TRUSTED_CA_FILE="/k8s/etcd/ssl/ca.pem"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_PEER_CERT_FILE="/k8s/etcd/ssl/server.pem"
ETCD_PEER_KEY_FILE="/k8s/etcd/ssl/server-key.pem"
ETCD_PEER_TRUSTED_CA_FILE="/k8s/etcd/ssl/ca.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"

3）設(shè)置系統(tǒng)控制啟動(dòng)（三個(gè)節(jié)點(diǎn)配置一樣）

此處master,work01,work02配置相同

由于新版本直接從EnvironmentFile指定的文件中讀取配置剑勾，所以也無(wú)需在向ExecStart中添加參數(shù)了

[root@master ~]# vim /usr/lib/systemd/system/etcd.service

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/data1/etcd/
EnvironmentFile=-/k8s/etcd/cfg/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /k8s/etcd/bin/etcd"
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

4）到控制臺(tái)開(kāi)放2379和2380端口（重點(diǎn)）

5）啟動(dòng)ETCD

需要三臺(tái)同時(shí)打開(kāi)，因?yàn)榈谝淮未蜷_(kāi)時(shí)要建立集群信息赵颅，時(shí)間長(zhǎng)了會(huì)超時(shí)

systemctl daemon-reload
systemctl enable etcd
systemctl start etcd

6）查看集群狀態(tài)

[root@master ~]# /k8s/etcd/bin/etcdctl --ca-file=/k8s/etcd/ssl/ca.pem --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem --endpoints="https://47.241.67.61:2379,https://175.24.19.25:2379,https://39.100.145.150:2379" cluster-health
member 55fbdb6e3ad20da8 is healthy: got healthy result from https://127.0.0.1:2379
member bdadcd6be126f0f2 is healthy: got healthy result from https://127.0.0.1:2379
member ca87f0191f2c8efa is healthy: got healthy result from https://127.0.0.1:2379
cluster is healthy

顯示cluster is healthy即可

三虽另、部署Flannel

3.1 下載安裝

1）下載文件

[root@master ~]# wget https://github.com/coreos/flannel/releases/download/v0.12.0/flannel-v0.12.0-linux-amd64.tar.gz

2）解壓并將文件移動(dòng)到指定位置(其它節(jié)點(diǎn)也需要)

[root@master ~]# tar -zxvf flannel-v0.12.0-linux-amd64.tar.gz
[root@master ~]# mkdir -p /k8s/flannel/{bin,cfg}
[root@master ~]# mv flanneld mk-docker-opts.sh /k8s/flannel/bin/

3）添加flanneld配置

[root@master ~]# vim /k8s/flannel/cfg/flannel.conf

master配置

#[flannel config]
FLANNELD_PUBLIC_IP="47.241.67.61"
FLANNELD_IFACE="eth0"

#[etcd]
FLANNELD_ETCD_ENDPOINTS="https://47.241.67.61:2379,https://175.24.19.25:2379,https://39.100.145.150:2379"
FLANNELD_ETCD_KEYFILE="/k8s/etcd/ssl/server-key.pem"
FLANNELD_ETCD_CERTFILE="/k8s/etcd/ssl/server.pem"
FLANNELD_ETCD_CAFILE="/k8s/etcd/ssl/ca.pem"
FLANNELD_IP_MASQ=true

其中下面兩行重點(diǎn)（官方支持的，FLANNELD_PUBLIC_IP填公網(wǎng)饺谬，FLANNELD_IFACE直接填私網(wǎng)網(wǎng)卡名捂刺，或者私網(wǎng)IP）

FLANNELD_PUBLIC_IP="47.241.67.61"
FLANNELD_IFACE="eth0"

work01配置

#[flannel config]
FLANNELD_PUBLIC_IP="175.24.19.25"
FLANNELD_IFACE="eth0"

#[etcd]
FLANNELD_ETCD_ENDPOINTS="https://47.241.67.61:2379,https://175.24.19.25:2379,https://39.100.145.150:2379"
FLANNELD_ETCD_KEYFILE="/k8s/etcd/ssl/server-key.pem"
FLANNELD_ETCD_CERTFILE="/k8s/etcd/ssl/server.pem"
FLANNELD_ETCD_CAFILE="/k8s/etcd/ssl/ca.pem"
FLANNELD_IP_MASQ=true

work02配置

#[flannel config]
FLANNELD_PUBLIC_IP="39.100.145.150"
FLANNELD_IFACE="eth0"

#[etcd]
FLANNELD_ETCD_ENDPOINTS="https://47.241.67.61:2379,https://175.24.19.25:2379,https://39.100.145.150:2379"
FLANNELD_ETCD_KEYFILE="/k8s/etcd/ssl/server-key.pem"
FLANNELD_ETCD_CERTFILE="/k8s/etcd/ssl/server.pem"
FLANNELD_ETCD_CAFILE="/k8s/etcd/ssl/ca.pem"
FLANNELD_IP_MASQ=true
3.3 添加到系統(tǒng)啟動(dòng)

三個(gè)節(jié)點(diǎn)都一樣，可以直接復(fù)制

[root@master ~]# vim /usr/lib/systemd/system/flanneld.service

#/k8s/flannel/cfg/flannel.conf
[Unit]
Description=Flanneld overlay address etc agent
After=network-online.target network.target
#Before=docker.service


[Service]
Type=notify
EnvironmentFile=-/k8s/flannel/cfg/flannel.conf
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /k8s/flannel/bin/flanneld"
ExecStartPost=/k8s/flannel/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env


Restart=on-failure


[Install]
WantedBy=multi-user.target

同樣募寨，由于族展，新版本的flannel也支持直接讀取環(huán)境變量導(dǎo)入的配置，所以無(wú)需再后面添加參數(shù)

注意設(shè)置先于docker啟動(dòng)的參數(shù)已經(jīng)注釋了拔鹰，后面再修改仪缸，此處只是為了盡快打通flannel網(wǎng)絡(luò)方便測(cè)試

3.4 開(kāi)放8472端口（重點(diǎn)）

使用了vxlan，默認(rèn)走的是8472列肢，UDP端口腹殿。

一定要到控制臺(tái)打開(kāi)端口，我之前就卡這了

3.5 向ETCD集群中添加網(wǎng)絡(luò)配置
[root@master ~]# /k8s/etcd/bin/etcdctl --ca-file=/k8s/etcd/ssl/ca.pem --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem --endpoints="https://47.241.67.61:2379,https://175.24.19.25:2379,https://39.100.145.150:2379" set  /coreos.com/network/config '{"Network":"10.254.0.0/16","Backend":{"Type":"vxlan"}}'
3.6 啟動(dòng)flannel 
[root@master ~]# systemctl daemon-reload
[root@master ~]# systemctl start flanneld
3.7 驗(yàn)證

flannel啟動(dòng)之后例书，使用ifconfig就可以看到新建的網(wǎng)卡名了

master

[root@master ~]# ifconfig

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.21.221.58  netmask 255.255.240.0  broadcast 172.21.223.255
        inet6 fe80::216:3eff:fe02:c141  prefixlen 64  scopeid 0x20<link>
        ether 00:16:3e:02:c1:41  txqueuelen 1000  (Ethernet)
        RX packets 17773699  bytes 3481620865 (3.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 18485980  bytes 3378770561 (3.1 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.254.5.0  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 fe80::e416:57ff:fe05:4590  prefixlen 64  scopeid 0x20<link>
        ether e6:16:57:05:45:90  txqueuelen 0  (Ethernet)
        RX packets 184  bytes 20342 (19.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 271  bytes 21777 (21.2 KiB)
        TX errors 0  dropped 76 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 5551843  bytes 1164398442 (1.0 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5551843  bytes 1164398442 (1.0 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

work01

[root@work01 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.13  netmask 255.255.240.0  broadcast 172.17.15.255
        inet6 fe80::5054:ff:fe74:1c2c  prefixlen 64  scopeid 0x20<link>
        ether 52:54:00:74:1c:2c  txqueuelen 1000  (Ethernet)
        RX packets 11260582  bytes 1955580984 (1.8 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10765978  bytes 1595473013 (1.4 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.254.20.0  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 fe80::ecad:57ff:febf:9ca6  prefixlen 64  scopeid 0x20<link>
        ether ee:ad:57:bf:9c:a6  txqueuelen 0  (Ethernet)
        RX packets 177  bytes 13893 (13.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 216  bytes 23030 (22.4 KiB)
        TX errors 0  dropped 8 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 34578  bytes 1838675 (1.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 34578  bytes 1838675 (1.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

確認(rèn)8472 UDP端口開(kāi)放后

直接ping一下其它節(jié)點(diǎn)的flannel網(wǎng)關(guān)即可

[root@master ~]# ping 10.254.20.0
PING 10.254.20.0 (10.254.20.0) 56(84) bytes of data.
64 bytes from 10.254.20.0: icmp_seq=1 ttl=64 time=68.2 ms
64 bytes from 10.254.20.0: icmp_seq=2 ttl=64 time=68.1 ms
64 bytes from 10.254.20.0: icmp_seq=3 ttl=64 time=68.1 ms
3.8 安裝docker

1）安裝

部署flannel后锣尉，稍加配置，docker 間即可相互訪問(wèn)了

裝docker很簡(jiǎn)單决采，這里直接用阿里的鏡像倉(cāng)庫(kù)提供的安裝步驟來(lái)

具體見(jiàn)https://developer.aliyun.com/mirror/docker-ce?spm=a2c6h.13651102.0.0.69bb1b11v9sJ7l

# step 1: 安裝必要的一些系統(tǒng)工具
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
# Step 2: 添加軟件源信息
sudo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# Step 3: 更新并安裝Docker-CE
sudo yum makecache fast
sudo yum -y install docker-ce

如果出現(xiàn)CentOS 8.0 安裝docker 報(bào)錯(cuò)：Problem: package docker-ce_xxx_64 requires containerd.io >= 1.2.2-3
請(qǐng)參考這篇文章https://www.backendcloud.cn/自沧，安裝或更新 containerd.io版本即可
2 ）修改flannel配置

vim /usr/lib/systemd/system/flanneld.service

取消之前注釋的這行

Before=docker.service

#/k8s/flannel/cfg
[Unit]
Description=Flanneld overlay address etc agent
After=network-online.target network.target
Before=docker.service


[Service]
Type=notify
EnvironmentFile=-/k8s/flannel/cfg/flannel.conf
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /k8s/flannel/bin/flanneld"
ExecStartPost=/k8s/flannel/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env


Restart=on-failure


[Install]
WantedBy=multi-user.target

3）修改docker配置

修改其中兩行

# 導(dǎo)入flannel網(wǎng)絡(luò)配置
EnvironmentFile=-/run/flannel/subnet.env
# 添加$DOCKER_NETWORK_OPTIONS參數(shù)
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/dockerd  -H fd:// --containerd=/run/containerd/containerd.sock $DOCKER_NETWORK_OPTIONS"

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker

# 導(dǎo)入flannel網(wǎng)絡(luò)配置
EnvironmentFile=-/run/flannel/subnet.env
# 添加$DOCKER_NETWORK_OPTIONS參數(shù)
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/dockerd  -H fd:// --containerd=/run/containerd/containerd.sock $DOCKER_NETWORK_OPTIONS"

ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3

# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

重啟docker，輸入ifconfig树瞭，可以看到網(wǎng)段已經(jīng)生效拇厢，docker也可以相互ping`通了

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.254.5.1  netmask 255.255.255.0  broadcast 10.254.5.255
        inet6 fe80::42:d6ff:fe13:a5b4  prefixlen 64  scopeid 0x20<link>
        ether 02:42:d6:13:a5:b4  txqueuelen 0  (Ethernet)
        RX packets 5  bytes 308 (308.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 32  bytes 2436 (2.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

至此，網(wǎng)絡(luò)配置已經(jīng)完成晒喷，不能ping通的孝偎，先檢查，ETCD使用的是2379和2380凉敲，TCP協(xié)議端口衣盾，flannel如果使用的是vxlan寺旺，默認(rèn)是使用8472，UDP協(xié)議端口势决，請(qǐng)?jiān)诳刂婆_(tái)打開(kāi)阻塑。

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
  • 人面猴
    序言：七十年代末，一起剝皮案震驚了整個(gè)濱河市果复，隨后出現(xiàn)的幾起案子陈莽，更是在濱河造成了極大的恐慌，老刑警劉巖虽抄，帶你破解...
    沈念sama閱讀 212,718評(píng)論 6贊 492
  • 死咒
    序言：濱河連續(xù)發(fā)生了三起死亡事件走搁，死亡現(xiàn)場(chǎng)離奇詭異，居然都是意外死亡迈窟，警方通過(guò)查閱死者的電腦和手機(jī)私植，發(fā)現(xiàn)死者居然都...
    沈念sama閱讀 90,683評(píng)論 3贊 385
  • 救了他兩次的神仙讓他今天三更去死
    文/潘曉璐 我一進(jìn)店門，熙熙樓的掌柜王于貴愁眉苦臉地迎上來(lái)菠隆，“玉大人兵琳，你說(shuō)我怎么就攤上這事狂秘『Ь叮” “怎么了？”我有些...
    開(kāi)封第一講書人閱讀 158,207評(píng)論 0贊 348
  • 道士緝兇錄：失蹤的賣姜人
    文/不壞的土叔 我叫張陵者春，是天一觀的道長(zhǎng)破衔。 經(jīng)常有香客問(wèn)我，道長(zhǎng)钱烟，這世上最難降的妖魔是什么晰筛？ 我笑而不...
    開(kāi)封第一講書人閱讀 56,755評(píng)論 1贊 284
  • ?港島之戀（遺憾婚禮）
    正文 為了忘掉前任，我火速辦了婚禮拴袭，結(jié)果婚禮上读第，老公的妹妹穿的比我還像新娘。我一直安慰自己拥刻，他們只是感情好怜瞒，可當(dāng)我...
    茶點(diǎn)故事閱讀 65,862評(píng)論 6贊 386
  • 惡毒庶女頂嫁案：這布局不是一般人想出來(lái)的
    文/花漫 我一把揭開(kāi)白布。 她就那樣靜靜地躺著般哼，像睡著了一般吴汪。 火紅的嫁衣襯著肌膚如雪。 梳的紋絲不亂的頭發(fā)上蒸眠，一...
    開(kāi)封第一講書人閱讀 50,050評(píng)論 1贊 291
  • 城市分裂傳說(shuō)
    那天漾橙，我揣著相機(jī)與錄音，去河邊找鬼楞卡。 笑死霜运，一個(gè)胖子當(dāng)著我的面吹牛脾歇，可吹牛的內(nèi)容都是我干的。 我是一名探鬼主播觉渴，決...
    沈念sama閱讀 39,136評(píng)論 3贊 410
  • 雙鴛鴦連環(huán)套：你想象不到人心有多黑
    文/蒼蘭香墨 我猛地睜開(kāi)眼介劫，長(zhǎng)吁一口氣：“原來(lái)是場(chǎng)噩夢(mèng)啊……” “哼！你這毒婦竟也來(lái)了案淋？” 一聲冷哼從身側(cè)響起座韵，我...
    開(kāi)封第一講書人閱讀 37,882評(píng)論 0贊 268
  • 萬(wàn)榮殺人案實(shí)錄
    序言：老撾萬(wàn)榮一對(duì)情侶失蹤，失蹤者是張志新（化名）和其女友劉穎踢京，沒(méi)想到半個(gè)月后誉碴，有當(dāng)?shù)厝嗽跇?shù)林里發(fā)現(xiàn)了一具尸體，經(jīng)...
    沈念sama閱讀 44,330評(píng)論 1贊 303
  • ?護(hù)林員之死
    正文 獨(dú)居荒郊野嶺守林人離奇死亡瓣距，尸身上長(zhǎng)有42處帶血的膿包…… 初始之章·張勛 以下內(nèi)容為張勛視角 年9月15日...
    茶點(diǎn)故事閱讀 36,651評(píng)論 2贊 327
  • ?白月光啟示錄
    正文 我和宋清朗相戀三年黔帕，在試婚紗的時(shí)候發(fā)現(xiàn)自己被綠了。 大學(xué)時(shí)的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片蹈丸。...
    茶點(diǎn)故事閱讀 38,789評(píng)論 1贊 341
  • 活死人
    序言：一個(gè)原本活蹦亂跳的男人離奇死亡成黄，死狀恐怖，靈堂內(nèi)的尸體忽然破棺而出逻杖，到底是詐尸還是另有隱情奋岁，我是刑警寧澤，帶...
    沈念sama閱讀 34,477評(píng)論 4贊 333
  • ?日本核電站爆炸內(nèi)幕
    正文 年R本政府宣布荸百，位于F島的核電站闻伶，受9級(jí)特大地震影響，放射性物質(zhì)發(fā)生泄漏够话。R本人自食惡果不足惜蓝翰，卻給世界環(huán)境...
    茶點(diǎn)故事閱讀 40,135評(píng)論 3贊 317
  • 男人毒藥：我在死后第九天來(lái)索命
    文/蒙蒙 一、第九天 我趴在偏房一處隱蔽的房頂上張望女嘲。 院中可真熱鬧畜份，春花似錦、人聲如沸欣尼。這莊子的主人今日做“春日...
    開(kāi)封第一講書人閱讀 30,864評(píng)論 0贊 21
  • 一樁弒父案，背后竟有這般陰謀
    文/蒼蘭香墨 我抬頭看了看天上的太陽(yáng)媒至。三九已至顶别，卻和暖如春，著一層夾襖步出監(jiān)牢的瞬間拒啰，已是汗流浹背驯绎。 一陣腳步聲響...
    開(kāi)封第一講書人閱讀 32,099評(píng)論 1贊 267
  • 情欲美人皮
    我被黑心中介騙來(lái)泰國(guó)打工， 沒(méi)想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留谋旦，地道東北人剩失。 一個(gè)月前我還...
    沈念sama閱讀 46,598評(píng)論 2贊 362
  • 代替公主和親
    正文 我出身青樓屈尼，卻偏偏與公主長(zhǎng)得像，于是被迫代替她去往敵國(guó)和親拴孤。 傳聞我的和親對(duì)象是個(gè)殘疾皇子脾歧，可洞房花燭夜當(dāng)晚...
    茶點(diǎn)故事閱讀 43,697評(píng)論 2贊 351

推薦閱讀更多精彩內(nèi)容