Kubernetes 0-1 二進(jìn)制搭建K8S（二）部署etcd集群

Run Rsyslog server in Kubernetes - ITNEXT

寫在前面

記錄和分享使用二進(jìn)制搭建K8S集群的詳細(xì)過程窜觉，由于操作比較冗長夏块，大概會分四篇寫完：

etcd作為k8s的數(shù)據(jù)庫，需要首先安裝扬舒，為其他組件做服務(wù)基礎(chǔ)阐肤。

etcd是一個分布式的數(shù)據(jù)庫系統(tǒng)，為了模擬etcd的高可用讲坎，我們將etcd部署在三臺虛擬機(jī)上孕惜，正好就部署在K8S集群所使用的三臺機(jī)器上吧。

etcd集群晨炕，K8S組件之間通信衫画，為了安全可靠，我們最好啟用HTTPS安全機(jī)制府瞄。K8S提供了基于CA簽名的雙向數(shù)字證書認(rèn)證方式和簡單的基于HTTP Base或Token的認(rèn)證方式碧磅，其中CA證書方式的安全性最高。我們使用cfssl為我們的K8S集群配置CA證書遵馆，此外也可以使用openssl鲸郊。

安裝cfssl

在Master機(jī)器執(zhí)行：

cd /root/kubernetes/resources
cp cfssl_linux-amd64 /usr/bin/cfssl
cp cfssljson_linux-amd64 /usr/bin/cfssljson
cp cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
chmod +x /usr/bin/cfssl /usr/bin/cfssljson /usr/bin/cfssl-certinfo

在所有機(jī)器執(zhí)行：

mkdir /etc/etcd/ssl -p

制作etcd證書

在Master機(jī)器執(zhí)行：

mkdir /root/kubernetes/resources/cert/etcd -p
cd /root/kubernetes/resources/cert/etcd

編輯ca-config.json

vim ca-config.json

寫入文件內(nèi)容如下：

{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "etcd": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}

編輯ca-csr.json：

vim ca-csr.json

寫入文件內(nèi)容如下：

{
    "CN": "etcd ca",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Hunan",
            "ST": "Changsha"
        }
    ]
}

生成ca證書和密鑰：

cfssl gencert -initca ca-csr.json | cfssljson -bare ca

編輯server-csr.json：

vim server-csr.json

寫入文件內(nèi)容如下：

{
    "CN": "etcd",
    "hosts": [
        "192.168.115.131",
        "192.168.115.132",
        "192.168.115.133"
        ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Hunan",
            "ST": "Changsha"
        }
    ]
}

hosts中配置所有Master和Node的IP列表。

生成etcd證書和密鑰

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd server-csr.json | cfssljson -bare server
# 此時目錄下會生成7個文件
ls
ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  server.csr  server-csr.json  server-key.pem  server.pem

拷貝證書

cp ca.pem server-key.pem  server.pem /etc/etcd/ssl
scp ca.pem server-key.pem  server.pem 192.168.115.132:/etc/etcd/ssl
scp ca.pem server-key.pem  server.pem 192.168.115.133:/etc/etcd/ssl

安裝etcd集群

在所有機(jī)器執(zhí)行：

cd /root/kubernetes/resources
tar -zxvf /root/kubernetes/resources/etcd-v3.4.9-linux-amd64.tar.gz
cp ./etcd-v3.4.9-linux-amd64/etcd ./etcd-v3.4.9-linux-amd64/etcdctl /usr/bin

配置etcd

這里開始命令需要分別在Master和Node機(jī)器執(zhí)行货邓，配置etcd.conf

vim /etc/etcd/etcd.conf

k8s-master01寫入文件內(nèi)容如下：

[Member]
ETCD_NAME="etcd01"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.115.131:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.115.131:2379,https://127.0.0.1:2379"

[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.115.131:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.115.131:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.115.131:2380,etcd02=https://192.168.115.132:2380,etcd03=https://192.168.115.133:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

k8s-node01寫入文件內(nèi)容如下：

[Member]
ETCD_NAME="etcd02"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.115.132:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.115.132:2379,https://127.0.0.1:2379"

[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.115.132:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.115.132:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.115.131:2380,etcd02=https://192.168.115.132:2380,etcd03=https://192.168.115.133:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

k8s-node02寫入文件內(nèi)容如下：

[Member]
ETCD_NAME="etcd03"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.115.133:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.115.133:2379,https://127.0.0.1:2379"

[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.115.133:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.115.133:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.115.131:2380,etcd02=https://192.168.115.132:2380,etcd03=https://192.168.115.133:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

這里開始在所有機(jī)器執(zhí)行秆撮，設(shè)置etcd服務(wù)配置文件

mkdir -p /var/lib/etcd
vim /usr/lib/systemd/system/etcd.service

執(zhí)行上行命令，寫入文件內(nèi)容如下：

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd \
        --cert-file=/etc/etcd/ssl/server.pem \
        --key-file=/etc/etcd/ssl/server-key.pem \
        --peer-cert-file=/etc/etcd/ssl/server.pem \
        --peer-key-file=/etc/etcd/ssl/server-key.pem \
        --trusted-ca-file=/etc/etcd/ssl/ca.pem \
        --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

etcd3.4版本會自動EnvironmentFile文件中的環(huán)境變量换况，不需要再ExecStart的命令參數(shù)重復(fù)設(shè)置职辨，否則會報："xxx" is shadowed by corresponding command-line flag的錯誤信息。

啟動etcd戈二，并且設(shè)置開機(jī)自動運(yùn)行etcd

systemctl daemon-reload
systemctl start etcd.service
systemctl enable etcd.service

檢查etcd集群的健康狀態(tài)

etcdctl endpoint health --cacert=/etc/etcd/ssl/ca.pem --cert=/etc/etcd/ssl/server.pem --key=/etc/etcd/ssl/server-key.pem --endpoints="https://192.168.115.131:2379,https://192.168.115.132:2379,https://192.168.115.133:2379"

輸出如下舒裤，說明etcd集群已經(jīng)部署成功。

https://192.168.115.133:2379 is healthy: successfully committed proposal: took = 15.805605ms
https://192.168.115.132:2379 is healthy: successfully committed proposal: took = 22.127986ms
https://192.168.115.131:2379 is healthy: successfully committed proposal: took = 24.829669ms

第二段落部署etcd集群愉快結(jié)束觉吭。