公眾號(hào)關(guān)注「WeiyiGeek」

設(shè)為「特別關(guān)注」，每天帶你玩轉(zhuǎn)網(wǎng)絡(luò)安全運(yùn)維、應(yīng)用開(kāi)發(fā)、物聯(lián)網(wǎng)IOT學(xué)習(xí)兽埃！

[圖片上傳失敗...(image-1b0721-1652954316553)]

本章目錄：

• 0x00 前言簡(jiǎn)述

• 0x01 環(huán)境準(zhǔn)備

• 主機(jī)規(guī)劃

• 軟件版本

• 網(wǎng)絡(luò)規(guī)劃

• 0x02 安裝部署

• 1.基礎(chǔ)主機(jī)環(huán)境準(zhǔn)備配置

• 2.負(fù)載均衡管理工具安裝與內(nèi)核加載

• 3.高可用HAproxy與Keepalived軟件安裝配置

• 4.配置部署etcd集群與etcd證書(shū)簽發(fā)

• 5.Containerd 運(yùn)行時(shí)安裝部署

溫馨提示: 由于實(shí)踐篇幅太長(zhǎng)，此處分為上下兩節(jié)進(jìn)行發(fā)布倔撞。

0x00 前言簡(jiǎn)述

描述: 在我博客以及前面的文章之中講解Kubernetes相關(guān)集群環(huán)境的搭建, 隨著K8S及其相關(guān)組件的迭代, 與讀者當(dāng)前接觸的版本有所不同讲仰，所以在當(dāng)前【2022年4月26日 10:08:29】時(shí)間節(jié)點(diǎn)，博主使用ubuntu 20.04 、haproxy鄙陡、keepalive冕房、containerd、etcd趁矾、kubeadm种柑、kubectl 等相關(guān)工具插件【最新或者穩(wěn)定的版本】進(jìn)行實(shí)踐高可用的kubernetes集群的搭建竭缝，這里不再對(duì)k8s等相關(guān)基礎(chǔ)知識(shí)做介紹，如有新入門的童鞋岁疼，請(qǐng)?jiān)L問(wèn)如下【博客文章】(https://blog.weiyigeek.top/tags/k8s/) 或者【B站專欄】(https://www.bilibili.com/read/readlist/rl520875?spm_id_from=333.999.0.0) 按照順序?qū)W習(xí)传睹。

簡(jiǎn)述
Kubernetes(后續(xù)簡(jiǎn)稱k8s)是 Google(2014年6月) 開(kāi)源的一個(gè)容器編排引擎斑粱，使用Go語(yǔ)言開(kāi)發(fā)则北，它支持自動(dòng)化部署尚揣、大規(guī)模可伸縮娜庇、以及云平臺(tái)中多個(gè)主機(jī)上的容器化應(yīng)用進(jìn)行管理思灌。其目標(biāo)是讓部署容器化的應(yīng)用更加簡(jiǎn)單并且高效，提供了資源調(diào)度熄守、部署管理蜈垮、服務(wù)發(fā)現(xiàn)、擴(kuò)容縮容裕照、狀態(tài) 監(jiān)控攒发、維護(hù)等一整套功能, 努力成為跨主機(jī)集群的自動(dòng)化部署、自動(dòng)化擴(kuò)展以及運(yùn)行應(yīng)用程序容器的平臺(tái)晋南，它支持一些列CNCF畢業(yè)項(xiàng)目惠猿，包括 Containerd、calico 等 负间。

0x01 環(huán)境準(zhǔn)備

主機(jī)規(guī)劃

溫馨提示: 此處使用的是 Ubuntu 20.04 操作系統(tǒng), 該系統(tǒng)已做安全加固和內(nèi)核優(yōu)化符合等保2.0要求【SecOpsDev/Ubuntu-InitializeSecurity.sh at master · WeiyiGeek/SecOpsDev (github.com)】, 如你的Linux未進(jìn)行相應(yīng)配置環(huán)境可能與讀者有些許差異, 如需要進(jìn)行(windows server 偶妖、Ubuntu、CentOS)安全加固請(qǐng)參照如下加固腳本進(jìn)行加固, 請(qǐng)大家瘋狂的 star 趾访。

加固腳本地址:【 https://github.com/WeiyiGeek/SecOpsDev/blob/master/OS-操作系統(tǒng)/Linux/Ubuntu/Ubuntu-InitializeSecurity.sh 】

軟件版本

操作系統(tǒng)

• Ubuntu 20.04 LTS - 5.4.0-107-generic

TLS證書(shū)簽發(fā)

• cfssl - v1.6.1

• cfssl-certinfo - v1.6.1

• cfssljson - v1.6.1

高可用軟件

• ipvsadm - 1:1.31-1

• haproxy - 2.0.13-2

• keepalived - 1:2.0.19-2

ETCD數(shù)據(jù)庫(kù)

• etcd - v3.5.4

容器運(yùn)行時(shí)

• containerd.io - 1.6.4

Kubernetes

• kubeadm - v1.23.6

• kube-apiserver - v1.23.6

• kube-controller-manager - v1.23.6

• kubectl - v1.23.6

• kubelet - v1.23.6

• kube-proxy - v1.23.6

• kube-scheduler - v1.23.6

網(wǎng)絡(luò)插件&輔助軟件
calico - v3.22
coredns - v1.9.1
kubernetes-dashboard - v2.5.1
k9s - v0.25.18

網(wǎng)絡(luò)規(guī)劃

溫馨提示: 上述環(huán)境所使用的到相關(guān)軟件及插件我已打包, 方便大家進(jìn)行下載扼鞋，可訪問(wèn)如下鏈接（訪問(wèn)密碼請(qǐng)?jiān)L問(wèn) WeiyiGeek 公眾號(hào)回復(fù)【k8s二進(jìn)制】獲取）云头。

下載地址: http://share.weiyigeek.top/f/36158960-578443238-a1a5fa （訪問(wèn)密碼：點(diǎn)擊訪問(wèn) WeiyiGeek 公眾號(hào)回復(fù)【k8s二進(jìn)制】）

[圖片上傳失敗...(image-8424b3-1652954316553)]

/kubernetes-cluster-binary-install# tree ..├── calico│   └── calico-v3.22.yaml├── certificate│   ├── admin-csr.json│   ├── apiserver-csr.json│   ├── ca-config.json│   ├── ca-csr.json│   ├── cfssl│   ├── cfssl-certinfo│   ├── cfssljson│   ├── controller-manager-csr.json│   ├── etcd-csr.json│   ├── kube-scheduler-csr.json│   ├── proxy-csr.json│   └── scheduler-csr.json├── containerd.io│   └── config.toml├── coredns│   ├── coredns.yaml│   ├── coredns.yaml.sed│   └── deploy.sh├── cri-containerd-cni-1.6.4-linux-amd64.tar.gz├── etcd-v3.5.4-linux-amd64.tar.gz├── k9s├── kubernetes-dashboard│   ├── kubernetes-dashboard.yaml│   └── rbac-dashboard-admin.yaml├── kubernetes-server-linux-amd64.tar.gz└── nginx.yaml

0x02 安裝部署

1.基礎(chǔ)主機(jī)環(huán)境準(zhǔn)備配置

步驟 01.【所有主機(jī)】主機(jī)名設(shè)置按照上述主機(jī)規(guī)劃進(jìn)行設(shè)置捐友。

# 例如, 在10.10.107.223主機(jī)中運(yùn)行。hostnamectl set-hostname master-223# 例如, 在10.10.107.227主機(jī)中運(yùn)行溃槐。hostnamectl set-hostname node-2

步驟 02.【所有主機(jī)】將規(guī)劃中的主機(jī)名稱與IP地址進(jìn)行硬解析脆粥。

sudo tee -a /etc/hosts <<'EOF'10.10.107.223 master-22310.10.107.224 master-22410.10.107.225 master-22510.10.107.226 node-110.10.107.227 node-210.10.107.222 weiyigeek.cluster.k8sEOF

步驟 03.驗(yàn)證每個(gè)節(jié)點(diǎn)上IP、MAC 地址和 product_uuid 的唯一性,保證其能相互正常通信

# 使用命令 ip link 或 ifconfig -a 來(lái)獲取網(wǎng)絡(luò)接口的 MAC 地址ifconfig -a# 使用命令 查看 product_uuid 校驗(yàn)sudo cat /sys/class/dmi/id/product_uuid

步驟 04.【所有主機(jī)】系統(tǒng)時(shí)間同步與時(shí)區(qū)設(shè)置

date -Rsudo ntpdate ntp.aliyun.comsudo timedatectl set-timezone Asia/Shanghai# 或者# sudo dpkg-reconfigure tzdatasudo timedatectl set-local-rtc 0timedatectl

步驟 05.【所有主機(jī)】禁用系統(tǒng)交換分區(qū)

swapoff -a && sed -i 's|^/swap.img|#/swap.ing|g' /etc/fstab# 驗(yàn)證交換分區(qū)是否被禁用free | grep "Swap:"

步驟 07.【所有主機(jī)】系統(tǒng)內(nèi)核參數(shù)調(diào)整

# 禁用 swap 分區(qū)egrep -q "^(#)?vm.swappiness.*" /etc/sysctl.conf && sed -ri "s|^(#)?vm.swappiness.*|vm.swappiness = 0|g"  /etc/sysctl.conf || echo "vm.swappiness = 0" >> /etc/sysctl.conf# 允許轉(zhuǎn)發(fā)egrep -q "^(#)?net.ipv4.ip_forward.*" /etc/sysctl.conf && sed -ri "s|^(#)?net.ipv4.ip_forward.*|net.ipv4.ip_forward = 1|g"  /etc/sysctl.conf || echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf# - 允許 iptables 檢查橋接流量egrep -q "^(#)?net.bridge.bridge-nf-call-iptables.*" /etc/sysctl.conf && sed -ri "s|^(#)?net.bridge.bridge-nf-call-iptables.*|net.bridge.bridge-nf-call-iptables = 1|g" /etc/sysctl.conf || echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.confegrep -q "^(#)?net.bridge.bridge-nf-call-ip6tables.*" /etc/sysctl.conf && sed -ri "s|^(#)?net.bridge.bridge-nf-call-ip6tables.*|net.bridge.bridge-nf-call-ip6tables = 1|g" /etc/sysctl.conf || echo "net.bridge.bridge-nf-call-ip6tables = 1" >> /etc/sysctl.conf

步驟 07.【所有主機(jī)】禁用系統(tǒng)防火墻

ufw disable && systemctl disable ufw && systemctl stop ufw

步驟 08.【master-225 主機(jī)】使用 master-225 主機(jī)的公鑰免賬號(hào)密碼登陸其它主機(jī)（可選）方便文件在各主機(jī)上傳下載肌厨。

# 生成ed25519格式的公密鑰sh-keygen -t ed25519# 例如,在master-225 主機(jī)上使用密鑰登錄到 master-223 設(shè)置 (其它主機(jī)同樣)ssh-copy-id -p 20211 weiyigeek@10.10.107.223  # /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_ed25519.pub"  # Are you sure you want to continue connecting (yes/no/[fingerprint])? yes # 輸入yes  # weiyigeek@10.10.107.223s password: # 輸入主機(jī)密碼  # Number of key(s) added: 1  # Now try logging into the machine, with:   "ssh -p '20211' 'weiyigeek@10.10.107.223'"  # and check to make sure that only the key(s) you wanted were added.ssh-copy-id -p 20211 weiyigeek@10.10.107.224ssh-copy-id -p 20211 weiyigeek@10.10.107.226ssh-copy-id -p 20211 weiyigeek@10.10.107.227

2.負(fù)載均衡管理工具安裝與內(nèi)核加載

步驟 01.安裝ipvs模塊以及負(fù)載均衡相關(guān)依賴。

# 查看可用版本sudo apt-cache madison ipvsadm  # ipvsadm |   1:1.31-1 | http://mirrors.aliyun.com/ubuntu focal/main amd64 Packages# 安裝sudo apt -y install ipvsadm ipset sysstat conntrack# 鎖定版本 apt-mark hold ipvsadm  # ipvsadm set on hold.

步驟 02.將模塊加載到內(nèi)核中(開(kāi)機(jī)自動(dòng)設(shè)置-需要重啟機(jī)器生效)

tee /etc/modules-load.d/k8s.conf <<'EOF'# netfilterbr_netfilter# containerdoverlay# nf_conntracknf_conntrack# ipvsip_vsip_vs_lcip_vs_lblcip_vs_lblcrip_vs_rrip_vs_wrrip_vs_ship_vs_dhip_vs_foip_vs_nqip_vs_sedip_vs_ftpip_tablesip_setipt_setipt_rpfilteript_REJECTipipxt_setEOF

步驟 03.手動(dòng)加載模塊到內(nèi)核中

mkdir -vp /etc/modules.d/tee /etc/modules.d/k8s.modules <<'EOF'#!/bin/bash# netfilter 模塊 允許 iptables 檢查橋接流量modprobe -- br_netfilter# containerdmodprobe -- overlay# nf_conntrackmodprobe -- nf_conntrack# ipvsmodprobe -- ip_vsmodprobe -- ip_vs_lcmodprobe -- ip_vs_lblcmodprobe -- ip_vs_lblcrmodprobe -- ip_vs_rrmodprobe -- ip_vs_wrrmodprobe -- ip_vs_shmodprobe -- ip_vs_dhmodprobe -- ip_vs_fomodprobe -- ip_vs_nqmodprobe -- ip_vs_sedmodprobe -- ip_vs_ftpmodprobe -- ip_tablesmodprobe -- ip_setmodprobe -- ipt_setmodprobe -- ipt_rpfiltermodprobe -- ipt_REJECTmodprobe -- ipipmodprobe -- xt_setEOFchmod 755 /etc/modules.d/k8s.modules && bash /etc/modules.d/k8s.modules && lsmod | grep -e ip_vs -e nf_conntrack  # ip_vs_sh               16384  0  # ip_vs_wrr              16384  0  # ip_vs_rr               16384  0  # ip_vs                 155648  6 ip_vs_rr,ip_vs_sh,ip_vs_wrr  # nf_conntrack          139264  1 ip_vs  # nf_defrag_ipv6         24576  2 nf_conntrack,ip_vs  # nf_defrag_ipv4         16384  1 nf_conntrack  # libcrc32c              16384  5 nf_conntrack,btrfs,xfs,raid456,ip_vssysctl --system

溫馨提示: 在 kernel 4.19 版本及以上將使用 nf_conntrack 模塊, 則在 4.18 版本以下則需使用nf_conntrack_ipv4 模塊祥诽。

3.高可用HAproxy與Keepalived軟件安裝配置

描述: 由于是測(cè)試學(xué)習(xí)環(huán)境, 此處我未專門準(zhǔn)備兩臺(tái)HA服務(wù)器, 而是直接采用master節(jié)點(diǎn)機(jī)器，如果是正式環(huán)境建議獨(dú)立出來(lái)绳姨。

步驟 01.【Master節(jié)點(diǎn)機(jī)器】安裝下載 haproxy (HA代理健康檢測(cè)) 與 keepalived (虛擬路由協(xié)議-主從)。

# 查看可用版本sudo apt-cache madison haproxy keepalived  #  haproxy | 2.0.13-2ubuntu0.5 | http://mirrors.aliyun.com/ubuntu focal-security/main amd64 Packages  # keepalived | 1:2.0.19-2ubuntu0.2 | http://mirrors.aliyun.com/ubuntu focal-updates/main amd64 Packages# 安裝sudo apt -y install haproxy keepalived# 鎖定版本 apt-mark hold haproxy keepalived

步驟 02.【Master節(jié)點(diǎn)機(jī)器】進(jìn)行 HAProxy 配置，其配置目錄為 /etc/haproxy/狞甚，所有節(jié)點(diǎn)配置是一致的孕豹。

sudo cp /etc/haproxy/haproxy.cfg{,.bak}tee /etc/haproxy/haproxy.cfg<<'EOF'global  user haproxy  group haproxy  maxconn 2000  daemon  log /dev/log local0  log /dev/log local1 err  chroot /var/lib/haproxy  stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners  stats timeout 30s  # Default SSL material locations  ca-base /etc/ssl/certs  crt-base /etc/ssl/private  # See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate  ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384  ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256  ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-ticketsdefaults  log     global  mode    http  option  httplog  option  dontlognull  timeout connect 5000  timeout client  50000  timeout server  50000  timeout http-request 15s  timeout http-keep-alive 15s  # errorfile 400 /etc/haproxy/errors/400.http  # errorfile 403 /etc/haproxy/errors/403.http  # errorfile 408 /etc/haproxy/errors/408.http  # errorfile 500 /etc/haproxy/errors/500.http  # errorfile 502 /etc/haproxy/errors/502.http  # errorfile 503 /etc/haproxy/errors/503.http  # errorfile 504 /etc/haproxy/errors/504.http# 注意: 管理HAproxy (可選)# frontend monitor-in#   bind *:33305#   mode http#   option httplog#   monitor-uri /monitor# 注意: 基于四層代理, 1644 3為VIP的 ApiServer 控制平面端口, 由于是與master節(jié)點(diǎn)在一起所以不能使用6443端口.frontend k8s-master  bind 0.0.0.0:16443  bind 127.0.0.1:16443  mode tcp  option tcplog  tcp-request inspect-delay 5s  default_backend k8s-master# 注意: Master 節(jié)點(diǎn)的默認(rèn) Apiserver 是6443端口backend k8s-master  mode tcp  option tcplog  option tcp-check  balance roundrobin  default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100  server master-223 10.10.107.223:6443 check  server master-224 10.10.107.224:6443 check  server master-225 10.10.107.225:6443 checkEOF

步驟 03.【Master節(jié)點(diǎn)機(jī)器】進(jìn)行 置KeepAlived 配置 春霍，其配置目錄為 /etc/haproxy/

# 創(chuàng)建配置目錄，分別在各個(gè)master節(jié)點(diǎn)執(zhí)行叶眉。mkdir -vp /etc/keepalived# __ROLE__ 角色: MASTER 或者 BACKUP# __NETINTERFACE__ 宿主機(jī)物理網(wǎng)卡名稱 例如我的ens32# __IP__ 宿主機(jī)物理IP地址# __VIP__ 虛擬VIP地址sudo tee /etc/keepalived/keepalived.conf <<'EOF'! Configuration File for keepalivedglobal_defs {  router_id LVS_DEVELscript_user root  enable_script_security}vrrp_script chk_apiserver {  script "/etc/keepalived/check_apiserver.sh"  interval 5  weight -5  fall 2    rise 1}vrrp_instance VI_1 {  state __ROLE__  interface __NETINTERFACE__  mcast_src_ip __IP__  virtual_router_id 51  priority 101  advert_int 2  authentication {    auth_type PASS    auth_pass K8SHA_KA_AUTH  }  virtual_ipaddress {    __VIP__  }  # HA 健康檢查  # track_script {  #   chk_apiserver  # }}EOF# 此處 master-225 性能較好所以配置為Master (master-225 主機(jī)上執(zhí)行)# master-225 10.10.107.225 => MASTERsed -i -e 's#__ROLE__#MASTER#g' \  -e 's#__NETINTERFACE__#ens32#g' \  -e 's#__IP__#10.10.107.225#g' \  -e 's#__VIP__#10.10.107.222#g' /etc/keepalived/keepalived.conf # master-224 10.10.107.224 => BACKUP  (master-224 主機(jī)上執(zhí)行)sed -i -e 's#__ROLE__#BACKUP#g' \  -e 's#__NETINTERFACE__#ens32#g' \  -e 's#__IP__#10.10.107.224#g' \  -e 's#__VIP__#10.10.107.222#g' /etc/keepalived/keepalived.conf # master-223 10.10.107.223 => BACKUP  (master-223 主機(jī)上執(zhí)行)sed -i -e 's#__ROLE__#BACKUP#g' \  -e 's#__NETINTERFACE__#ens32#g' \  -e 's#__IP__#10.10.107.223#g' \  -e 's#__VIP__#10.10.107.222#g' /etc/keepalived/keepalived.conf

溫馨提示: 注意上述的健康檢查是關(guān)閉注釋了的址儒，你需要將K8S集群建立完成后再開(kāi)啟。

track_script {  chk_apiserver}

步驟 04.【Master節(jié)點(diǎn)機(jī)器】進(jìn)行配置 KeepAlived 健康檢查文件衅疙。

sudo tee /etc/keepalived/check_apiserver.sh <<'EOF'#!/bin/basherr=0for k in $(seq 1 3)do  check_code=$(pgrep haproxy)  if [[ $check_code == "" ]]; then    err=$(expr $err + 1)    sleep 1    continue  else    err=0    break  fidoneif [[ $err != "0" ]]; then  echo "systemctl stop keepalived"  /usr/bin/systemctl stop keepalived  exit 1else  exit 0fiEOFsudo chmod +x /etc/keepalived/check_apiserver.sh

步驟 05.【Master節(jié)點(diǎn)機(jī)器】啟動(dòng) haproxy 莲趣、keepalived 相關(guān)服務(wù)及測(cè)試VIP漂移。

# 重載 Systemd 設(shè)置 haproxy 饱溢、keepalived 開(kāi)機(jī)自啟以及立即啟動(dòng)sudo systemctl daemon-reloadsudo systemctl enable --now haproxy && sudo systemctl enable --now keepalived# Synchronizing state of haproxy.service with SysV service script with /lib/systemd/systemd-sysv-install.# Executing: /lib/systemd/systemd-sysv-install enable haproxy# Synchronizing state of keepalived.service with SysV service script with /lib/systemd/systemd-sysv-install.# Executing: /lib/systemd/systemd-sysv-install enable keepalived# 在 master-223 主機(jī)中發(fā)現(xiàn)vip地址在其主機(jī)上喧伞。root@master-223:~$ ip addr  # 2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000  #     link/ether 00:0c:29:00:0f:8f brd ff:ff:ff:ff:ff:ff  #     inet 10.10.107.223/24 brd 10.10.107.255 scope global ens32  #        valid_lft forever preferred_lft forever  #     inet 10.10.107.222/32 scope global ens32  #        valid_lft forever preferred_lft forever# 其它兩臺(tái)主機(jī)上通信驗(yàn)證。root@master-224:~$ ping 10.10.107.222root@master-225:~$ ping 10.10.107.222

# 手動(dòng)驗(yàn)證VIP漂移,我們將該服務(wù)器上keepalived停止掉理朋。root@master-223:~$ pgrep haproxy  # 6320  # 6321root@master-223:~$ /usr/bin/systemctl stop keepalived# 此時(shí),發(fā)現(xiàn)VIP已經(jīng)飄到master-225主機(jī)中root@master-225:~$ ip addr show ens32  # 2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000  #     link/ether 00:0c:29:93:28:61 brd ff:ff:ff:ff:ff:ff  #     inet 10.10.107.225/24 brd 10.10.107.255 scope global ens32  #       valid_lft forever preferred_lft forever  #     inet 10.10.107.222/32 scope global ens32  #       valid_lft forever preferred_lft forever

至此絮识，HAProxy 與 Keepalived 配置就告一段落了,下面將學(xué)習(xí) ETCD 集群配置與證書(shū)簽發(fā)。

4.配置部署etcd集群與etcd證書(shū)簽發(fā)

描述: 創(chuàng)建一個(gè)高可用的ETCD集群嗽上，此處我們?cè)凇緈aster-225】機(jī)器中操作。

步驟 01.【master-225】創(chuàng)建一個(gè)配置與相關(guān)文件存放的目錄, 以及下載獲取cfssl工具進(jìn)行CA證書(shū)制作與簽發(fā)(cfssl工具往期文章參考地址: https://blog.weiyigeek.top/2019/10-21-12.html#3-CFSSL-生成 )熄攘。

# 工作目錄創(chuàng)建mkdir -vp /app/k8s-init-work && cd /app/k8s-init-work# cfssl 最新下載地址: https://github.com/cloudflare/cfssl/releases# cfssl 相關(guān)工具拉取 (如果拉取較慢兽愤，建議使用某雷下載，然后上傳到服務(wù)器里)curl -L https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl_1.6.1_linux_amd64 -o /usr/local/bin/cfsslcurl -L https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssljson_1.6.1_linux_amd64 -o /usr/local/bin/cfssljsoncurl -L https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl-certinfo_1.6.1_linux_amd64 -o /usr/local/bin/cfssl-certinfo# 賦予執(zhí)行權(quán)限chmod +x /usr/local/bin/cfssl*/app# cfssl version# Version: 1.2.0# Revision: dev# Runtime: go1.6

溫馨提示:

• cfssl : CFSSL 命令行工具

• cfssljson : 用于從cfssl程序中獲取JSON輸出并將證書(shū)、密鑰浅萧、證書(shū)簽名請(qǐng)求文件CSR和Bundle寫(xiě)入到文件中逐沙，

步驟 02.利用上述 cfssl 工具創(chuàng)建 CA 證書(shū)。

# - CA 證書(shū)簽名請(qǐng)求配置文件fssl print-defaults csr > ca-csr.jsontee ca-csr.json <<'EOF'{  "CN": "kubernetes",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "L": "ChongQing",      "ST": "ChongQing",      "O": "k8s",      "OU": "System"    }  ],  "ca": {    "expiry": "87600h"  }}EOF# 關(guān)鍵參數(shù)解析:CN: Common Name洼畅，瀏覽器使用該字段驗(yàn)證網(wǎng)站是否合法吩案，一般寫(xiě)的是域名，非常重要帝簇。瀏覽器使用該字段驗(yàn)證網(wǎng)站是否合法key：生成證書(shū)的算法hosts：表示哪些主機(jī)名(域名)或者IP可以使用此csr申請(qǐng)的證書(shū)徘郭，為空或者""表示所有的都可以使用(本例中沒(méi)有`"hosts": [""]`字段)names：常見(jiàn)屬性設(shè)置  * C: Country， 國(guó)家  * ST: State丧肴，州或者是省份  * L: Locality Name残揉，地區(qū)，城市  * O: Organization Name芋浮，組織名稱抱环，公司名稱(在k8s中常用于指定Group，進(jìn)行RBAC綁定)  * OU: Organization Unit Name纸巷，組織單位名稱镇草，公司部門# - CA 證書(shū)策略配置文件cfssl print-defaults config > ca-config.jsontee ca-config.json <<'EOF'{  "signing": {    "default": {      "expiry": "87600h"    },    "profiles": {      "kubernetes": {        "expiry": "87600h",        "usages": [            "signing",            "key encipherment",            "server auth",            "client auth"        ]      },      "etcd": {        "expiry": "87600h",        "usages": [            "signing",            "key encipherment",            "server auth",            "client auth"        ]      }    }  }}EOF# 關(guān)鍵參數(shù)解析:default 默認(rèn)策略，指定了證書(shū)的默認(rèn)有效期是10年(87600h)profile 自定義策略配置  * kubernetes：表示該配置(profile)的用途是為kubernetes生成證書(shū)及相關(guān)的校驗(yàn)工作  * signing：表示該證書(shū)可用于簽名其它證書(shū)瘤旨；生成的 ca.pem 證書(shū)中 CA=TRUE  * server auth：表示可以該CA 對(duì) server 提供的證書(shū)進(jìn)行驗(yàn)證  * client auth：表示可以用該 CA 對(duì) client 提供的證書(shū)進(jìn)行驗(yàn)證  * expiry：也表示過(guò)期時(shí)間陶夜，如果不寫(xiě)以default中的為準(zhǔn)# - 執(zhí)行cfssl gencert 命令生成CA證書(shū)# 利用CA證書(shū)簽名請(qǐng)求配置文件 ca-csr.json 生成CA證書(shū)和CA私鑰和CSR(證書(shū)簽名請(qǐng)求):cfssl gencert -initca ca-csr.json | cfssljson -bare ca  # 2022/04/27 16:49:37 [INFO] generating a new CA key and certificate from CSR  # 2022/04/27 16:49:37 [INFO] generate received request  # 2022/04/27 16:49:37 [INFO] received CSR  # 2022/04/27 16:49:37 [INFO] generating key: rsa-2048  # 2022/04/27 16:49:37 [INFO] encoded CSR  # 2022/04/27 16:49:37 [INFO] signed certificate with serial number 245643466964695827922023924375276493244980966303$ ls  # ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem$ openssl x509 -in ca.pem -text -noout | grep "Not"  # Not Before: Apr 27 08:45:00 2022 GMT  # Not After : Apr 24 08:45:00 2032 GMT

溫馨提示: 如果將 expiry 設(shè)置為87600h 表示證書(shū)過(guò)期時(shí)間為十年。

步驟 03.配置ETCD證書(shū)相關(guān)文件以及生成其證書(shū),

# etcd 證書(shū)請(qǐng)求文件tee etcd-csr.json <<'EOF'{  "CN": "etcd",  "hosts": [    "127.0.0.1",    "10.10.107.223",    "10.10.107.224",    "10.10.107.225",    "etcd1",    "etcd2",    "etcd3"  ],  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "L": "ChongQing",      "ST": "ChongQing",      "O": "etcd",      "OU": "System"    }  ]}EOF# 利用ca證書(shū)簽發(fā)生成etcd證書(shū)cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd etcd-csr.json | cfssljson -bare etcd$ ls etcd*etcd.csr  etcd-csr.json  etcd-key.pem  etcd.pem$ openssl x509 -in etcd.pem -text -noout | grep  "X509v3 Subject Alternative Name" -A 1  # X509v3 Subject Alternative Name:  #   DNS:etcd1, DNS:etcd2, DNS:etcd3, IP Address:127.0.0.1, IP Address:10.10.107.223, IP Address:10.10.107.224, IP Address:10.10.107.225

步驟 04.【所有Master節(jié)點(diǎn)主機(jī)】下載部署ETCD集群, 首先我們需要下載etcd軟件包, 可以 Github release 找到最新版本的etcd下載路徑(https://github.com/etcd-io/etcd/releases/)裆站。

# 下載wget -L https://github.com/etcd-io/etcd/releases/download/v3.5.4/etcd-v3.5.4-linux-amd64.tar.gztar -zxvf etcd-v3.5.4-linux-amd64.tar.gzcp -a etcd* /usr/local/bin/# 版本 etcd --version  # etcd Version: 3.5.4  # Git SHA: 08407ff76  # Go Version: go1.16.15  # Go OS/Arch: linux/amd64# 復(fù)制到其它master主機(jī)上scp -P 20211 ./etcd-v3.5.4-linux-amd64.tar.gz weiyigeek@master-223:~scp -P 20211 ./etcd-v3.5.4-linux-amd64.tar.gz weiyigeek@master-224:~# 分別在master-223與master-224執(zhí)行, 解壓到 /usr/local/ 目錄同樣復(fù)制二進(jìn)制文件到 /usr/local/bin/tar -zxvf /home/weiyigeek/etcd-v3.5.4-linux-amd64.tar.gz -C /usr/local/cp -a /usr/local/etcd-v3.5.4-linux-amd64/etcd* /usr/local/bin/

溫馨提示: etcd 官網(wǎng)地址 ( https://etcd.io/)

步驟 05.創(chuàng)建etcd集群所需的配置文件条辟。

# 證書(shū)準(zhǔn)備mkdir -vp /etc/etcd/pki/cp *.pem /etc/etcd/pki/ls /etc/etcd/pki/  # ca-key.pem  ca.pem  etcd-key.pem  etcd.pem# 上傳到~家目錄,并需要將其復(fù)制到 /etc/etcd/pki/ 目錄中scp -P 20211 *.pem weiyigeek@master-224:~scp -P 20211 *.pem weiyigeek@master-223:~  # ****************** [ 安全登陸 (Security Login) ] *****************  # Authorized only. All activity will be monitored and reported.By Security Center.  # ca-key.pem             100% 1675     3.5MB/s   00:00  # ca.pem                 100% 1375     5.2MB/s   00:00  # etcd-key.pem           100% 1679     7.0MB/s   00:00  # etcd.pem               100% 1399     5.8MB/s   00:00# master-225 執(zhí)行tee /etc/etcd/etcd.conf <<'EOF'# [成員配置]# member 名稱ETCD_NAME=etcd1# 存儲(chǔ)數(shù)據(jù)的目錄(注意需要建立)ETCD_DATA_DIR="/var/lib/etcd/data"# 用于監(jiān)聽(tīng)客戶端etcdctl或者curl連接ETCD_LISTEN_CLIENT_URLS="https://10.10.107.225:2379,https://127.0.0.1:2379"# 用于監(jiān)聽(tīng)集群中其它member的連接ETCD_LISTEN_PEER_URLS="https://10.10.107.225:2380"# [證書(shū)配置]# ETCD_CERT_FILE=/etc/etcd/pki/etcd.pem# ETCD_KEY_FILE=/etc/etcd/pki/etcd-key.pem# ETCD_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.pem# ETCD_CLIENT_CERT_AUTH=true# ETCD_PEER_CLIENT_CERT_AUTH=true# ETCD_PEER_CERT_FILE=/etc/etcd/pki/etcd.pem# ETCD_PEER_KEY_FILE=/etc/etcd/pki/etcd-key.pem# ETCD_PEER_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.pem# [集群配置]# 本機(jī)地址用于通知客戶端，客戶端通過(guò)此IPs與集群通信;ETCD_ADVERTISE_CLIENT_URLS="https://10.10.107.225:2379"# 本機(jī)地址用于通知集群member與member通信ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.10.107.225:2380"# 描述集群中所有節(jié)點(diǎn)的信息宏胯，本member根據(jù)此信息去聯(lián)系其他memberETCD_INITIAL_CLUSTER="etcd1=https://10.10.107.225:2380,etcd2=https://10.10.107.224:2380,etcd3=https://10.10.107.223:2380"# 集群狀態(tài)新建集群時(shí)候設(shè)置為new,若是想加入某個(gè)已經(jīng)存在的集群設(shè)置為existingETCD_INITIAL_CLUSTER_STATE=newEOF# master-224 執(zhí)行tee /etc/etcd/etcd.conf <<'EOF'# [成員配置]# member 名稱ETCD_NAME=etcd2# 存儲(chǔ)數(shù)據(jù)的目錄(注意需要建立)ETCD_DATA_DIR="/var/lib/etcd/data"# 用于監(jiān)聽(tīng)客戶端etcdctl或者curl連接ETCD_LISTEN_CLIENT_URLS="https://10.10.107.224:2379,https://127.0.0.1:2379"# 用于監(jiān)聽(tīng)集群中其它member的連接ETCD_LISTEN_PEER_URLS="https://10.10.107.224:2380"# [集群配置]# 本機(jī)地址用于通知客戶端羽嫡，客戶端通過(guò)此IPs與集群通信;ETCD_ADVERTISE_CLIENT_URLS="https://10.10.107.224:2379"# 本機(jī)地址用于通知集群member與member通信ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.10.107.224:2380"# 描述集群中所有節(jié)點(diǎn)的信息，本member根據(jù)此信息去聯(lián)系其他memberETCD_INITIAL_CLUSTER="etcd1=https://10.10.107.225:2380,etcd2=https://10.10.107.224:2380,etcd3=https://10.10.107.223:2380"# 集群狀態(tài)新建集群時(shí)候設(shè)置為new,若是想加入某個(gè)已經(jīng)存在的集群設(shè)置為existingETCD_INITIAL_CLUSTER_STATE=newEOF# master-223 執(zhí)行tee /etc/etcd/etcd.conf <<'EOF'# [成員配置]# member 名稱ETCD_NAME=etcd3# 存儲(chǔ)數(shù)據(jù)的目錄(注意需要建立)ETCD_DATA_DIR="/var/lib/etcd/data"# 用于監(jiān)聽(tīng)客戶端etcdctl或者curl連接ETCD_LISTEN_CLIENT_URLS="https://10.10.107.223:2379,https://127.0.0.1:2379"# 用于監(jiān)聽(tīng)集群中其它member的連接ETCD_LISTEN_PEER_URLS="https://10.10.107.223:2380"# [集群配置]# 本機(jī)地址用于通知客戶端肩袍，客戶端通過(guò)此IPs與集群通信;ETCD_ADVERTISE_CLIENT_URLS="https://10.10.107.223:2379"# 本機(jī)地址用于通知集群member與member通信ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.10.107.223:2380"# 描述集群中所有節(jié)點(diǎn)的信息杭棵，本member根據(jù)此信息去聯(lián)系其他memberETCD_INITIAL_CLUSTER="etcd1=https://10.10.107.225:2380,etcd2=https://10.10.107.224:2380,etcd3=https://10.10.107.223:2380"# 集群狀態(tài)新建集群時(shí)候設(shè)置為new,若是想加入某個(gè)已經(jīng)存在的集群設(shè)置為existingETCD_INITIAL_CLUSTER_STATE=newEOF

步驟 06.【所有Master節(jié)點(diǎn)主機(jī)】創(chuàng)建配置 etcd 的 systemd 管理配置文件，并啟動(dòng)其服務(wù)氛赐。

mkdir -vp /var/lib/etcd/cat > /usr/lib/systemd/system/etcd.service <<EOF[Unit]Description=Etcd ServerDocumentation=https://github.com/etcd-io/etcdAfter=network.targetAfter=network-online.targetwants=network-online.target[Service]Type=notifyWorkingDirectory=/var/lib/etcd/EnvironmentFile=-/etc/etcd/etcd.confExecStart=/usr/local/bin/etcd \  --client-cert-auth \  --trusted-ca-file /etc/etcd/pki/ca.pem \  --cert-file /etc/etcd/pki/etcd.pem \  --key-file /etc/etcd/pki/etcd-key.pem \  --peer-client-cert-auth \  --peer-trusted-ca-file /etc/etcd/pki/ca.pem \  --peer-cert-file /etc/etcd/pki/etcd.pem \  --peer-key-file /etc/etcd/pki/etcd-key.pemRestart=on-failureRestartSec=5LimitNOFILE=65535LimitNPROC=65535[Install]WantedBy=multi-user.targetEOF# 重載 systemd && 開(kāi)機(jī)啟動(dòng)與手動(dòng)啟用etcd服務(wù)systemctl daemon-reload && systemctl enable --now etcd.service

步驟 07.【所有Master節(jié)點(diǎn)主機(jī)】查看各個(gè)master節(jié)點(diǎn)的etcd集群服務(wù)是否正常及其健康狀態(tài)魂爪。

# 服務(wù)查看systemctl status etcd.service# 利用 etcdctl 工具查看集群成員信息export ETCDCTL_API=3etcdctl --endpoints=https://10.10.107.225:2379,https://10.10.107.224:2379,https://10.10.107.223:2379 \--cacert="/etc/etcd/pki/ca.pem" --cert="/etc/etcd/pki/etcd.pem" --key="/etc/etcd/pki/etcd-key.pem" \--write-out=table member list  # +------------------+---------+-------+----------------------------+----------------------------+------------+  # |        ID        | STATUS  | NAME  |         PEER ADDRS         |        CLIENT ADDRS        | IS LEARNER |  # +------------------+---------+-------+----------------------------+----------------------------+------------+  # | 144934d02ad45ec7 | started | etcd3 | https://10.10.107.223:2380 | https://10.10.107.223:2379 |      false |  # | 2480d95a2df867a4 | started | etcd2 | https://10.10.107.224:2380 | https://10.10.107.224:2379 |      false |  # | 2e8fddd3366a3d88 | started | etcd1 | https://10.10.107.225:2380 | https://10.10.107.225:2379 |      false |  # +------------------+---------+-------+----------------------------+----------------------------+------------+# 集群節(jié)點(diǎn)信息etcdctl --endpoints=https://10.10.107.225:2379,https://10.10.107.224:2379,https://10.10.107.223:2379 \--cacert="/etc/etcd/pki/ca.pem" --cert="/etc/etcd/pki/etcd.pem" --key="/etc/etcd/pki/etcd-key.pem"  \--write-out=table endpoint status  # +----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+  # |          ENDPOINT          |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |  # +----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+  # | https://10.10.107.225:2379 | 2e8fddd3366a3d88 |   3.5.4 |   20 kB |     false |      false |         3 |         12 |                 12 |        |  # | https://10.10.107.224:2379 | 2480d95a2df867a4 |   3.5.4 |   20 kB |      true |      false |         3 |         12 |                 12 |        |  # | https://10.10.107.223:2379 | 144934d02ad45ec7 |   3.5.4 |   20 kB |     false |      false |         3 |         12 |                 12 |        |  # +----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+# 集群節(jié)點(diǎn)健康狀態(tài)etcdctl --endpoints=https://10.10.107.225:2379,https://10.10.107.224:2379,https://10.10.107.223:2379 \--cacert="/etc/etcd/pki/ca.pem" --cert="/etc/etcd/pki/etcd.pem" --key="/etc/etcd/pki/etcd-key.pem"  \--write-out=table endpoint health  # +----------------------------+--------+-------------+-------+  # |          ENDPOINT          | HEALTH |    TOOK     | ERROR |  # +----------------------------+--------+-------------+-------+  # | https://10.10.107.225:2379 |   true |  9.151813ms |       |  # | https://10.10.107.224:2379 |   true | 10.965914ms |       |  # | https://10.10.107.223:2379 |   true | 11.165228ms |       |  # +----------------------------+--------+-------------+-------+# 集群節(jié)點(diǎn)性能測(cè)試etcdctl --endpoints=https://10.10.107.225:2379,https://10.10.107.224:2379,https://10.10.107.223:2379 \--cacert="/etc/etcd/pki/ca.pem" --cert="/etc/etcd/pki/etcd.pem" --key="/etc/etcd/pki/etcd-key.pem"  \--write-out=tableendpoint check perf# 59 / 60 Boooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooom   !  98.33%PASS: Throughput is 148 writes/s# Slowest request took too long: 1.344053s# Stddev too high: 0.143059s# FAIL

5.Containerd 運(yùn)行時(shí)安裝部署

步驟 01.【所有節(jié)點(diǎn)】在各主機(jī)中安裝二進(jìn)制版本的 containerd.io 運(yùn)行時(shí)服務(wù)，Kubernertes 通過(guò) CRI 插件來(lái)連接 containerd 服務(wù)中, 控制容器的生命周期艰管。

# 從 Github 中下載最新的版本的 cri-containerd-cni wget -L https://github.com/containerd/containerd/releases/download/v1.6.4/cri-containerd-cni-1.6.4-linux-amd64.tar.gz# 解壓到當(dāng)前cri-containerd-cni目錄中滓侍。mkdir -vp cri-containerd-cnitar -zxvf cri-containerd-cni-1.6.4-linux-amd64.tar.gz -C cri-containerd-cni

步驟 02.查看其文件以及配置文件路徑信息。

$ tree ./cri-containerd-cni/.├── etc│   ├── cni│   │   └── net.d│   │       └── 10-containerd-net.conflist│   ├── crictl.yaml│   └── systemd│       └── system│           └── containerd.service├── opt│   ├── cni│   │   └── bin│   │       ├── bandwidth│   │       ├── bridge│   │       ├── dhcp│   │       ├── firewall│   │       ├── host-device│   │       ├── host-local│   │       ├── ipvlan│   │       ├── loopback│   │       ├── macvlan│   │       ├── portmap│   │       ├── ptp│   │       ├── sbr│   │       ├── static│   │       ├── tuning│   │       ├── vlan│   │       └── vrf│   └── containerd│       └── cluster│           ├── gce│           │   ├── cloud-init│           │   │   ├── master.yaml│           │   │   └── node.yaml│           │   ├── cni.template│           │   ├── configure.sh│           │   └── env│           └── version└── usr    └── local        ├── bin        │   ├── containerd        │   ├── containerd-shim        │   ├── containerd-shim-runc-v1        │   ├── containerd-shim-runc-v2        │   ├── containerd-stress        │   ├── crictl        │   ├── critest        │   ├── ctd-decoder        │   └── ctr        └── sbin            └── runc# 然后在所有節(jié)點(diǎn)上復(fù)制到上述文件夾到對(duì)應(yīng)目錄中cd ./cri-containerd-cni/cp -r etc/ /cp -r opt/ /cp -r usr/ /

步驟 03.【所有節(jié)點(diǎn)】進(jìn)行containerd 配置創(chuàng)建并修改 config.toml .

mkdir -vp /etc/containerd# 默認(rèn)配置生成containerd config default >/etc/containerd/config.tomlls /etc/containerd/config.toml  # /etc/containerd/config.toml# pause 鏡像源sed -i "s#k8s.gcr.io/pause#registry.cn-hangzhou.aliyuncs.com/google_containers/pause#g"  /etc/containerd/config.toml# 使用 SystemdCgroupsed -i 's#SystemdCgroup = false#SystemdCgroup = true#g' /etc/containerd/config.toml# docker.io mirrorsed -i '/registry.mirrors]/a\ \ \ \ \ \ \ \ [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]' /etc/containerd/config.tomlsed -i '/registry.mirrors."docker.io"]/a\ \ \ \ \ \ \ \ \ \ endpoint = ["https://xlx9erfu.mirror.aliyuncs.com","https://docker.mirrors.ustc.edu.cn"]' /etc/containerd/config.toml# gcr.io mirrorsed -i '/registry.mirrors]/a\ \ \ \ \ \ \ \ [plugins."io.containerd.grpc.v1.cri".registry.mirrors."gcr.io"]' /etc/containerd/config.tomlsed -i '/registry.mirrors."gcr.io"]/a\ \ \ \ \ \ \ \ \ \ endpoint = ["https://gcr.mirrors.ustc.edu.cn"]' /etc/containerd/config.toml# k8s.gcr.io mirrorsed -i '/registry.mirrors]/a\ \ \ \ \ \ \ \ [plugins."io.containerd.grpc.v1.cri".registry.mirrors."k8s.gcr.io"]' /etc/containerd/config.tomlsed -i '/registry.mirrors."k8s.gcr.io"]/a\ \ \ \ \ \ \ \ \ \ endpoint = ["https://gcr.mirrors.ustc.edu.cn/google-containers/","https://registry.cn-hangzhou.aliyuncs.com/google_containers/"]' /etc/containerd/config.toml# quay.io mirrorsed -i '/registry.mirrors]/a\ \ \ \ \ \ \ \ [plugins."io.containerd.grpc.v1.cri".registry.mirrors."quay.io"]' /etc/containerd/config.tomlsed -i '/registry.mirrors."quay.io"]/a\ \ \ \ \ \ \ \ \ \ endpoint = ["https://quay.mirrors.ustc.edu.cn"]' /etc/containerd/config.toml

步驟 04.客戶端工具 runtime 與 鏡像 端點(diǎn)配置:

# 手動(dòng)設(shè)置臨時(shí)生效# crictl config runtime-endpoint /run/containerd/containerd.sock# /run/containerd/containerd.sock # 配置文件設(shè)置永久生效cat <<EOF > /etc/crictl.yamlruntime-endpoint: unix:///run/containerd/containerd.sockimage-endpoint: unix:///run/containerd/containerd.socktimeout: 10debug: falseEOF

步驟 05.重載 systemd自啟和啟動(dòng)containerd.io服務(wù)牲芋。

systemctl daemon-reload && systemctl enable --now containerd.servicesystemctl status containerd.servicectr version  # Client:  #   Version:  1.5.11  #   Revision: 3df54a852345ae127d1fa3092b95168e4a88e2f8  #   Go version: go1.17.8  # Server:  #   Version:  1.5.11  #   Revision: 3df54a852345ae127d1fa3092b95168e4a88e2f8  #   UUID: 71a28bbb-6ed6-408d-a873-e394d48b35d8

步驟 06.用于根據(jù)OCI規(guī)范生成和運(yùn)行容器的CLI工具 runc 版本查看

runc -v  # runc version 1.1.1  # commit: v1.1.1-0-g52de29d7  # spec: 1.0.2-dev  # go: go1.17.9  # libseccomp: 2.5.1

溫馨提示: 當(dāng)默認(rèn) runc 執(zhí)行提示 runc: symbol lookup error: runc: undefined symbol: seccomp_notify_respond 時(shí)撩笆，由于上述軟件包中包含的runc對(duì)系統(tǒng)依賴過(guò)多捺球，所以建議單獨(dú)下載安裝 runc 二進(jìn)制項(xiàng)目(https://github.com/opencontainers/runc/)

wget https://github.com/opencontainers/runc/releases/download/v1.1.1/runc.amd64# 執(zhí)行權(quán)限賦予chmod +x runc.amd64# 替換掉 /usr/local/sbin/ 路徑原軟件包中的 runcmv runc.amd64 /usr/local/sbin/runc

本文至此完畢，更多技術(shù)文章夕冲，盡情期待下一章節(jié)氮兵！

歡迎各位志同道合的朋友一起學(xué)習(xí)交流，如文章有誤請(qǐng)?jiān)谙路搅粝履鷮氋F的經(jīng)驗(yàn)知識(shí)歹鱼，個(gè)人郵箱地址【master#weiyigeek.top】或者個(gè)人公眾號(hào)【W(wǎng)eiyiGeek】聯(lián)系我泣栈。

更多文章來(lái)源于【WeiyiGeek Blog 個(gè)人博客 - 為了能到遠(yuǎn)方，腳下的每一步都不能少 】

個(gè)人主頁(yè): 【 https://weiyigeek.top】

博客地址: 【 https://blog.weiyigeek.top 】

專欄書(shū)寫(xiě)不易弥姻，如果您覺(jué)得這個(gè)專欄還不錯(cuò)的南片，請(qǐng)給這篇專欄 【點(diǎn)個(gè)贊、投個(gè)幣蚁阳、收個(gè)藏铃绒、關(guān)個(gè)注，轉(zhuǎn)個(gè)發(fā)螺捐，留個(gè)言】(人間六大情)颠悬，這將對(duì)我的肯定，謝謝定血！赔癌。

• echo "【點(diǎn)個(gè)贊】，動(dòng)動(dòng)你那粗壯的拇指或者芊芊玉手澜沟，親灾票！"

• printf("%s", "【投個(gè)幣】，萬(wàn)水千山總是情茫虽，投個(gè)硬幣行不行刊苍，親！")

• fmt.Printf("【收個(gè)藏】濒析，閱后即焚不吃灰正什，親！")

• console.info("【轉(zhuǎn)個(gè)發(fā)】号杏，讓更多的志同道合的朋友一起學(xué)習(xí)交流婴氮，親！")

• System.out.println("【關(guān)個(gè)注】盾致，后續(xù)瀏覽查看不迷路喲主经，親！")

• cout << "【留個(gè)言】庭惜，文章寫(xiě)得好不好罩驻、有沒(méi)有錯(cuò)誤，一定要留言喲蜈块，親! " << endl;

往期相關(guān)文章

記一次在k8s集群搭建的Harbor私有倉(cāng)庫(kù)無(wú)法提供服務(wù)之鏡像遷移恢復(fù)實(shí)踐

K9s之Kubernetes集群管理交互工具實(shí)踐

K9s之Kuberntes集群管理交互工具實(shí)踐

3.Containerd容器運(yùn)行時(shí)的配置淺析與知識(shí)擴(kuò)充實(shí)踐

4.如何使用nerdctl工具并配合Containerd容器運(yùn)行時(shí)來(lái)替代Docker容器環(huán)境

WeiyiGeek

Always keep a beginner's mind, don't forget the beginner's mind. Blog :【https://weiyigeek.top】

174篇原創(chuàng)內(nèi)容

更多網(wǎng)絡(luò)安全鉴腻、系統(tǒng)運(yùn)維迷扇、應(yīng)用開(kāi)發(fā)百揭、全棧文章爽哎，盡在【個(gè)人博客 - https://blog.weiyigeek.top】站點(diǎn)，謝謝支持器一！

↓↓↓ 更多文章课锌，請(qǐng)點(diǎn)擊下方閱讀原文。