一、Containerd簡介和安裝
1.apt安裝containerd
(1)驗證倉庫版本
root@45b5bc5f4a38:/# apt update
root@45b5bc5f4a38:/# apt-cache madison containerd
(2)安裝containerd
root@45b5bc5f4a38:/# apt install containerd
(3)查看Service文件
root@ecs-67093:~# cat /lib/systemd/system/containerd.service
# Copyright The containerd Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#? ? http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
(4)驗證runc環(huán)境
(5)配置文件
root@45b5bc5f4a38:/# containerd config default? #查看默認(rèn)配置
root@45b5bc5f4a38:/# mkdir /etc/containerd
root@45b5bc5f4a38:/# containerd config default > /etc/containerd/config.toml
root@45b5bc5f4a38:/#?systemctl restart containerd.service
(6)下載鏡像
root@ecs-67093:~# ctr images pull docker.io/library/alpine:latest
(7)驗證鏡像
root@ecs-67093:~# ctr images ls
(8)ctr客戶端創(chuàng)建測試容器
root@ecs-67093:~# ctr run -t --net-host docker.io/library/alpine:latest test-container sh
2.二進(jìn)制安裝containerd
(1)下載二進(jìn)制包
root@ecs-67093:~# tar xvf containerd-1.6.6-linux-amd64.tar.gz
root@ecs-67093:~# cp bin/* /usr/local/bin/
(2)創(chuàng)建service文件
root@ecs-67093:~# cat /lib/systemd/system/containerd.service
# Copyright The containerd Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#? ? http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
(3)配置文件
root@ecs-67093:~# mkdir /etc/containerd
root@ecs-67093:/usr/local/src# containerd config default > /etc/containerd/config.toml
root@ecs-67093:~# cat /etc/containerd/config.toml
61? ? ? ? sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.7"
153? ? ? [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
154? ? ? ? [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
155? ? ? ? ? endpoint = ["https://e1bqjzsj.mirror.aliyuncs.com"]
root@ecs-67093:/usr/local/src# systemctl restart containerd
root@ecs-67093:/usr/local/src# systemctl enable containerd
root@ecs-67093:/usr/local/src# systemctl status containerd
(4)部署runc（v1.1.3）
root@ecs-67093:/usr/local/src# chmod +x runc.amd64
root@ecs-67093:/usr/local/src# cp runc.amd64 /usr/bin/runc
(5)下載鏡像并運行容器
root@ecs-67093:/usr/local/src# ctr images pull docker.io/library/alpine:latest
root@ecs-67093:/usr/local/src# ctr run -t --net-host docker.io/library/alpine:latest container1 sh
/ # ping www.baidu.com
PING www.baidu.com (110.242.68.3): 56 data bytes
64 bytes from 110.242.68.3: seq=0 ttl=47 time=12.022 ms
64 bytes from 110.242.68.3: seq=1 ttl=47 time=11.948 ms
64 bytes from 110.242.68.3: seq=2 ttl=47 time=11.914 ms
二、Containerd客戶端工具擴(kuò)展
1.crictl
(1)下載crictl
root@ecs-67093:/usr/local/src# tar xvf crictl-v1.24.2-linux-amd64.tar.gz
root@ecs-67093:/usr/local/src# cp crictl /usr/local/bin/
(2)配置crictl運行工具
默認(rèn)連接unix:///var/run/dockershim.sock
但是containerd在以下路徑：
root@ecs-67093:/usr/local/src# ls /run/containerd/containerd.sock
/run/containerd/containerd.sock
所以，修改配置文件
root@ecs-67093:/usr/local/src# cat /etc/crictl.yaml
runtime-endpoint: "unix:///run/containerd/containerd.sock"
image-endpoint: "unix:///run/containerd/containerd.sock"
timeout: 10
debug: false
(3)下載并驗證鏡像
root@ecs-67093:/usr/local/src# crictl pull nginx:1.20.2
Image is up to date for sha256:50fe74b50e0d0258922495297efbb9ebc3cbd5742103df1ca54dc21c07d24575
root@ecs-67093:/usr/local/src# crictl images ls
IMAGE? ? ? ? ? ? ? ? ? ? TAG? ? ? ? ? ? ? ? IMAGE ID? ? ? ? ? ? SIZE
docker.io/library/nginx? 1.20.2? ? ? ? ? ? ? 50fe74b50e0d0? ? ? 56.7MB
2.nerdctl-推薦使用
(1)安裝nerdctl
root@ecs-67093:/usr/local/src# tar xvf nerdctl-0.18.0-linux-amd64.tar.gz
nerdctl
containerd-rootless-setuptool.sh
containerd-rootless.sh
root@ecs-67093:/usr/local/src# cp nerdctl /usr/local/bin/
root@ecs-67093:/usr/local/src# nerdctl version
(2)安裝cni
root@ecs-67093:/usr/local/src# mkdir /opt/cni/bin -p
root@ecs-67093:/usr/local/src# tar xvf cni-plugins-linux-amd64-v1.1.1.tgz -C /opt/cni/bin/
(3)創(chuàng)建容器并指定端口
nginx：
root@ecs-67093:/usr/local/src# nerdctl run -d -p 80:80 --name=nginx-web1 --restart=always nginx:1.20.2
root@ecs-67093:/usr/local/src# nerdctl ps
root@ecs-67093:/usr/local/src# nerdctl exec -it c1b91b522807 bash
tomcat:
root@ecs-67093:/usr/local/src# nerdctl run -d -p 8080:8080 --name=tomcat-web1 --restart=always tomcat:7.0.88-alpine
root@ecs-67093:/usr/local/src# nerdctl ps
三、基于kubeadm和containerd部署k8s
1.服務(wù)器環(huán)境準(zhǔn)備
(1)開機(jī)啟動br_netfilter模塊
root@ecs-67093:~# cat << EOF >> /etc/rc.local
> #!/bin/bash
> modprobe br_netfilter
> EOF
root@ecs-67093:~# chmod a+x /etc/rc.local
root@ecs-67093:~# systemctl restart rc-local
root@ecs-67093:~# lsmod | grep br_netfilter
(2)優(yōu)化內(nèi)核參數(shù)
root@ecs-67093:~# cat << EOF >> /etc/sysctl.conf
> net.bridge.bridge-nf-call-iptables = 1? #容器啟動后會創(chuàng)建網(wǎng)橋坛猪，內(nèi)核監(jiān)聽網(wǎng)橋上有過的報文，以實現(xiàn)對報文的安全控制、規(guī)則檢查毒费，ingress、egress就是通過對報文的檢查來決定允許通行或禁止通行
> net.ipv4.ip_forward = 1? #把Linux當(dāng)作路由器使用愈魏，使其具備路由功能觅玻，基于路由表做地址轉(zhuǎn)發(fā)、報文轉(zhuǎn)發(fā)培漏、源地址替換等溪厘，否則無法跨主機(jī)通信
> EOF
root@ecs-67093:~# sysctl -p
2.安裝kubeadm基礎(chǔ)環(huán)境
(1)安裝kubeadm、kubectl牌柄、kubelet
master上安裝kubeadm畸悬、kubectl、kubelet珊佣；node上安裝kubeadm蹋宦、kubelet，kubectl可以不安裝
root@ecs-67093:~# apt-get update && apt-get install -y apt-transport-https? #支持https源
root@ecs-67093:~# curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -? #導(dǎo)入key
root@ecs-67093:~# cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
> deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
> EOF? #配置源
root@ecs-67093:~# apt-get update
root@ecs-67093:~# apt-cache madison kubeadm
root@ecs-67093:~# apt-get install -y kubeadm=1.24.3-00 kubectl=1.24.3-00 kubelet=1.24.3-00
(2)安裝nerdctl
(3)安裝cni
3.初始化kubernetes
(1)下載鏡像
root@ecs-67093:~# kubeadm config images list --kubernetes-version v1.24.3
k8s.gcr.io/kube-apiserver:v1.24.3
k8s.gcr.io/kube-controller-manager:v1.24.3
k8s.gcr.io/kube-scheduler:v1.24.3
k8s.gcr.io/kube-proxy:v1.24.3
k8s.gcr.io/pause:3.7
k8s.gcr.io/etcd:3.5.3-0
k8s.gcr.io/coredns/coredns:v1.8.6
root@ecs-67093:~# cat image-down.sh
%s/k8s.gcr.io/nerdctl -n k8s.io pull registry.cn-hangzhou.aliyuncs.com\/google_containers/g
(2)初始化k8s集群
root@ecs-67093:~# kubeadm init --apiserver-advertise-address=192.168.0.75 --apiserver-bind-port=6443 --kubernetes-version=v1.24.3 --pod-network-cidr=10.100.0.0/16 --service-cidr=10.200.0.0/16 --service-dns-domain=cluster.local --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers --ignore-preflight-errors=swap
......
Your Kubernetes control-plane has initialized successfully!
......
(3)配置認(rèn)證文件
root@ecs-67093:~# mkdir -p $HOME/.kube
root@ecs-67093:~# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@ecs-67093:~# sudo chown $(id -u):$(id -g) $HOME/.kube/config
root@ecs-67093:~# kubectl get node
(4)安裝網(wǎng)絡(luò)組件
root@ecs-67093:~# kubectl apply -f calico-ipip.yaml
root@k8s-master01:~# nerdctl -n k8s.io images
(5)添加node節(jié)點
root@k8s-node01:/usr/local/src# kubeadm join 192.168.0.173:6443 --token pa2l6x.6mp1hbewr1hr5rwu --discovery-token-ca-cert-hash sha256:d71b81c437de1790cc3aed44b2dd65669d08175bd518b947af28351fe5b5e337? #在node節(jié)點執(zhí)行
root@k8s-node01:/usr/local/src# mkdir /root/.kube
root@k8s-master01:~# scp /root/.kube/config 192.168.0.42:/root/.kube/
四咒锻、部署時遇到的問題
1.添加節(jié)點時冷冗，calico容器報錯
root@ecs-67093:~# find / -name *ubeadm*
