場景:kubernetes(1.1.2版本)默認證書有效期是一年是目,到期后需要重新生成
方案一:集群處于不可用狀態(tài)渠欺,手動更新證書和配置文件
重新生成證書
在所有控制平面執(zhí)行以下命令,備份或刪除過期證書
mkdir -p /etc/kubernetes/pki.bak/etcd
cd /etc/kubernetes/pki
mv apiserver* front-proxy-client* /etc/kubernetes/pki.bak
cd /etc/kubernetes/pki/etcd/
mv healthcheck-client* peer* server* /etc/kubernetes/pki.bak/etcd
可以選擇保留ca證書跳纳,ca證書有效期一般為10年面氓,可以重用,不需要刪除
如果選擇刪除所有證書霹疫,這樣的話就需要手動將第一個控制平面新生成的ca證書copy到其它控制平面的對應(yīng)目錄中
ca證書如下:
/etc/kubernetes/pki/ca.crt
/etc/kubernetes/pki/ca.key
/etc/kubernetes/pki/front-proxy-ca.crt
/etc/kubernetes/pki/front-proxy-ca.key
/etc/kubernetes/pki/sa.key
/etc/kubernetes/pki/sa.pub
/etc/kubernetes/pki/etcd/ca.crt
/etc/kubernetes/pki/etcd/ca.key
執(zhí)行以下命令生成新證書
kubeadm alpha phase certs all --config=kubeadm.yaml
其中kubeadm.yaml為初始化集群時使用的配置文件,具體配置如下
(kubeadm config view)
kind: ClusterConfiguration
apiVersion: kubeadm.k8s.io/v1beta1
controlPlaneEndpoint: "192.168.100.170:6443"
apiServer:
certSANS:
- master1
- master2
- master3
- 192.168.100.170
- 192.168.100.171
- 192.168.100.181
- 192.168.100.191
重新生成配置文件
在每個控制平面節(jié)點執(zhí)行以下命令综芥,備份并重新生成配置文件丽蝎,包括admin.conf、controller-manager.conf膀藐、kubelet.conf屠阻、scheduler.conf
cd /etc/kubernetes
mkdir conf.bak
mv *.conf conf.bak/
kubeadm alpha phase kubeconfig all --config kubeadm.yaml
在每個worker節(jié)點執(zhí)行以下命令,備份重新生成kubelet的配置文件
cd /etc/kubernetes
mkdir conf.bak
mv *.conf conf.bak/
kubeadm alpha kubeconfig user --org system:nodes --client-name system:node:$(hostname) --config kubeadm.yaml > /etc/kubernetes/kubelet.conf
注意额各,一般來說worker節(jié)點上只有ca.crt證書国觉,沒有ca.key,生成配置文件時會報錯虾啦,這時需要從控制平面將ca.key拷貝過來麻诀,配置文件生成完畢后即可刪除。上述命令在某版本執(zhí)行時報錯 可以改寫為:kubeadm alpha kubeconfig user --org system:nodes --client-name system:node:$(hostname) --apiserver-advertise-address 192.168.100.170 > /etc/kubernetes/kubelet.conf
在每個worker節(jié)點執(zhí)行以下命令傲醉,備份或刪除過期的證書(kubelet啟動時會自動生成pki目錄及證書)蝇闭,然后重啟kubelet。
mv /var/lib/kubelet/pki /var/lib/kubelet/pki.bak
systemctl daemon-reload && systemctl restart kubelet
注意:命令執(zhí)行過程中可能有配置文件版本與kubeadm不配套的情況硬毕,這時候按提示升級配置文件即可呻引,如下
kubeadm config migrate --old-config old.yaml --new-config new.yaml
最后重啟管理部件pod,包括etcd吐咳,apiserver逻悠,controller-manager,scheduler韭脊。如果集群不可用蹂风,可以到對應(yīng)節(jié)點上通過docker rm 的方式刪除容器,kubelet會自動拉起新的pod
踩坑:
etcd證書使用本地外部yaml生成
kubeadm init phase certs all --config=/home/kubernetes/config_1.13_local_etcd.yaml
當出現(xiàn) Unable to connect to the server: x509: certificate has expired or is not yet valid
mv ~/.kube/config ~/.kube/config.bak && cp /etc/kubernetes/admin.conf ~/.kube/config
方案二:(未實踐)如果集群正常使用中乾蓬,則可以使用kubeadm進行證書輪換
檢查證書是否過期
root@master1:~/.kube# kubeadm alpha certs check-expiration
CERTIFICATE EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
admin.conf Dec 30, 2020 02:43 UTC 364d no
apiserver Dec 30, 2020 02:25 UTC 364d no
apiserver-etcd-client Dec 30, 2020 02:25 UTC 364d no
apiserver-kubelet-client Dec 30, 2020 02:25 UTC 364d no
controller-manager.conf Jul 07, 2020 08:16 UTC 189d no
etcd-healthcheck-client Jul 04, 2020 09:26 UTC 186d no
etcd-peer Jul 04, 2020 09:26 UTC 186d no
etcd-server Jul 04, 2020 09:26 UTC 186d no
front-proxy-client Dec 30, 2020 02:25 UTC 364d no
scheduler.conf Jul 07, 2020 08:16 UTC 189d no
證書輪換
# Step 1): Backup old certs and kubeconfigs
mkdir /etc/kubernetes.bak
cp -r /etc/kubernetes/pki/ /etc/kubernetes.bak
cp /etc/kubernetes/*.conf /etc/kubernetes.bak
# Step 2): Renew all certs
kubeadm alpha certs renew all --config kubeadm.yaml
# Step 3): Renew all kubeconfigs
kubeadm alpha kubeconfig user --client-name=admin
kubeadm alpha kubeconfig user --org system:masters --client-name kubernetes-admin > /etc/kubernetes/admin.conf
kubeadm alpha kubeconfig user --client-name system:kube-controller-manager > /etc/kubernetes/controller-manager.conf
kubeadm alpha kubeconfig user --org system:nodes --client-name system:node:$(hostname) > /etc/kubernetes/kubelet.conf
kubeadm alpha kubeconfig user --client-name system:kube-scheduler > /etc/kubernetes/scheduler.conf
# Another way to renew kubeconfigs
# kubeadm init phase kubeconfig all --config kubeadm.yaml
# Step 4): Copy certs/kubeconfigs and restart Kubernetes services