## 機器準備
3臺機器费韭,一臺做master著蟹,兩臺做slave
配置:16G 16核
Docker版本(18.06.01)?
k8s版本(1.15.0)
ansible是一個很強大的運維自動化運維工具棒假,基于ssh缩挑,非常方便钠绍。
## 一冀偶、環(huán)境準備(三個節(jié)點都需要執(zhí)行)
```
1.配置免密
三臺互做免密 執(zhí)行以下命令
ssh-copy-id -i ~/.ssh/id_rsa.pub root@<hostname> -p 22022
2.安裝ansible 并配置ansible inventory文件
yum install -y ansible
[k8s-master]
172.18.0.171 ansible_ssh_user=root ansible_ssh_port=22022//端口默認是22 如果沒有更改則使用默認
[k8s-slave]
172.18.0.172 ansible_ssh_user=root ansible_ssh_port=22022
172.18.0.173 ansible_ssh_user=root ansible_ssh_port=22022
ansible使用示例:
在所有節(jié)點執(zhí)行命令:ansible all -m shell -a "systemctl start docker "
在master上執(zhí)行:ansible k8s-master -m shell -a "systemctl start docker"
3.關(guān)閉防火墻和selinux
systemctl stop firewalld && systemctl disable firewalld?
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config? && setenforce 0
4.關(guān)閉swap分區(qū)
swapoff -a # 臨時
$ sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab #永久
5.修改主機名
hostnamectl set-hostname master
hostnamectl set-hostname slave1
hostnamectl set-hostname slave2
6.修改hosts文件
172.18.0.171 master
172.18.0.172 slave1
172.18.0.173 slave2
7.內(nèi)核調(diào)整醒第,將橋接的IPV4流量傳遞到iptable的鏈
cat > /etc/sysctl.d/k8s.conf << EOFnet.bridge.bridge-nf-call-ip6tables = 1net.bridge.bridge-nf-call-iptables = 1EOF$ sysctl --system
8.三臺機器同步時鐘
yum install -y ntpdate
ntpdate time.windows.com
```
## 二、安裝docker(三個節(jié)點都需要執(zhí)行)
```
yum install -y yum-utils? device-mapper-persistent-data? lvm2
配置docker源
yum-config-manager? ? --add-repo? ? https://download.docker.com/linux/centos/docker-ce.repo
查看docker版本
yum list docker-ce --showduplicates | sort -r
安裝docker
yum install docker-ce-18.09.6 docker-ce-cli-18.09.6 containerd.io(指定版本安裝)
yum install -y docker-ce docker-ce-cli containerd.io(最新版本安裝)
啟動docker
systemctl start docker
systemctl enable docker
安裝命令補全
ansible all -m shell -a "yum -y install bash-completion"
ansible all -m shell -a "source /etc/profile.d/bash_completion.sh"
```
## 三进鸠、安裝k8s(三個節(jié)點都需要執(zhí)行)
```
1.申請阿里源docker加速器
https://cr.console.aliyun.com/cn-hangzhou/instances/mirrors
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
? "registry-mirrors": ["https://wv8lwzcp.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
2.添加k8s yum源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
更新緩存
yum clean all& yum -y makecache"
3.安裝kubeadm kubelet kubectl
yum install -y kubelet-1.15.0 kubeadm-1.15.0 kubectl-1.15.0
systemctl enable kubelet
```
## 四稠曼、配置master節(jié)點(只在master執(zhí)行)
```
kubeadm init --apiserver-advertise-address=172.18.0.171 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.15.0 --service-cidr=10.1.0.0/16 --pod-network-cidr=10.244.0.0/16
踩坑:這地方如果執(zhí)行報超時,則有可能是swap分區(qū)沒關(guān)堤如,執(zhí)行下面腳本
#!/bin/bash
rm -rf /etc/kubernetes/*
rm -rf ~/.kube/*
rm -rf /var/lib/etcd/*
lsof -i :6443|grep -v "PID"|awk '{print "kill -9",$2}'|sh
lsof -i :10251|grep -v "PID"|awk '{print "kill -9",$2}'|sh
lsof -i :10252|grep -v "PID"|awk '{print "kill -9",$2}'|sh
lsof -i :10250|grep -v "PID"|awk '{print "kill -9",$2}'|sh
lsof -i :2379|grep -v "PID"|awk '{print "kill -9",$2}'|sh
lsof -i :2380|grep -v "PID"|awk '{print "kill -9",$2}'|sh
swapoff -a && kubeadm reset? && systemctl daemon-reload && systemctl restart kubelet? && iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
重新初始化蒲列,成功后出現(xiàn)下面命令
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
? mkdir -p $HOME/.kube
? sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
? sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
? https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 172.18.0.171:6443 --token mi2ip9.629f41c46tvh79g1 \
? ? --discovery-token-ca-cert-hash sha256:649afe0a5c0f9599a0b4a6e4baa6aac3e3e6007adf98d215f495182d31d2dfac
按照要求執(zhí)行上述命令
[root@master ~]# cat kube_preinstall.sh
#!/bin/bash
? mkdir -p $HOME/.kube
? sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
? sudo chown $(id -u):$(id -g) $HOME/.kube/config
記住上面的命令窒朋,用來把子節(jié)點接入到master中。
```
## 五蝗岖、子節(jié)點加入到mater中(兩個子節(jié)點都需要執(zhí)行)
```
kubeadm join 172.18.0.171:6443 --token mi2ip9.629f41c46tvh79g1 \
? ? --discovery-token-ca-cert-hash sha256:649afe0a5c0f9599a0b4a6e4baa6aac3e3e6007adf98d215f495182d31d2dfac
```
**踩坑:**如果上面的token忘記了侥猩,則執(zhí)行下面的命令重新生成token的sha256編碼
```
[root@master ~]# kubeadm token list
TOKEN? ? ? ? ? ? ? ? ? ? TTL? ? ? ? EXPIRES? ? ? ? ? ? ? ? ? ? USAGES? ? ? ? ? ? ? ? ? DESCRIPTION? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? EXTRA GROUPS
zxjr2d.ecnowzegec34w8vj? <invalid>? 2021-02-04T13:37:09+08:00? authentication,signing? The default bootstrap token generated by 'kubeadm init'.? system:bootstrappers:kubeadm:default-node-token
[root@master ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
d7d8d27c50c1ef63cd56e8894e154d6e2861693b8f554460df4eb6fc14ce84aa
```
## 六、安裝網(wǎng)絡(luò)插件抵赢,使用flannel網(wǎng)絡(luò)
```
wget https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml
```
**踩坑:** :可能會遇到 https://raw.githubusercontent.com 打不開的情況欺劳,這是由于dns域名解析失敗的原因。進入到如下網(wǎng)站:
https://www.ipaddress.com/
輸入域名:https://raw.githubusercontent.com
解析出域名的ip地址铅鲤,并添加到主機的hosts文件中
```
[root@master ~]# cat /etc/hosts
127.0.0.1? localhost localhost.localdomain localhost4 localhost4.localdomain4
::1? ? ? ? localhost localhost.localdomain localhost6 localhost6.localdomain6
172.18.0.171 master
172.18.0.172 slave1
172.18.0.173 slave2
199.232.96.133 raw.githubusercontent.com
```
下載成功后划提,開始修改鏡像
```
由于默認的可能不能拉取,如果確保能夠訪問到quay.io這個registery,則不用修改邢享,否則修改如下內(nèi)容:
169 serviceAccountName: flannel
170? ? ? initContainers:
171? ? ? - name: install-cni
172? ? ? ? image: easzlab/flannel:v0.11.0-amd64
173? ? ? ? command:
174? ? ? ? - cp
175? ? ? ? args:
176? ? ? ? - -f
177? ? ? ? - /etc/kube-flannel/cni-conf.json
178? ? ? ? - /etc/cni/net.d/10-flannel.conflist
179? ? ? ? volumeMounts:
180? ? ? ? - name: cni
181? ? ? ? ? mountPath: /etc/cni/net.d
182? ? ? ? - name: flannel-cfg
183? ? ? ? ? mountPath: /etc/kube-flannel/
184? ? ? containers:
185? ? ? - name: kube-flannel
186? ? ? ? image: easzlab/flannel:v0.11.0-amd64
187? ? ? ? command:
188? ? ? ? - /opt/bin/flanneld
修改完成后鹏往,開始拉起flannel鏡像
kubectl apply -f kube-flannel.yml
查看是否被拉起
ps -ef|grep flannel
```
查看集群的網(wǎng)絡(luò)狀態(tài),只有在如下狀態(tài)骇塘,說明所有的節(jié)點都已經(jīng)ready
```
[root@master ~]# kubectl get nodes
NAME? ? STATUS? ROLES? ? AGE? VERSION
master? Ready? ? master? 25h? v1.15.0
slave1? Ready? ? <none>? 25h? v1.15.0
slave2? Ready? ? <none>? 25h? v1.15.0
[root@master ~]# kubectl get pod -n kube-system
NAME? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? READY? STATUS? ? RESTARTS? AGE
coredns-bccdc95cf-cpc96? ? ? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
coredns-bccdc95cf-d5fs2? ? ? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
etcd-master? ? ? ? ? ? ? ? ? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-apiserver-master? ? ? ? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-controller-manager-master? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-flannel-ds-amd64-25ztw? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-flannel-ds-amd64-cqmx8? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-flannel-ds-amd64-f6mxw? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-proxy-mz2rb? ? ? ? ? ? ? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-proxy-nd9zp? ? ? ? ? ? ? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-proxy-s4xfh? ? ? ? ? ? ? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kube-scheduler-master? ? ? ? ? ? ? 1/1? ? Running? 0? ? ? ? ? 25h
kubernetes-dashboard-79ddd5-nchbb? 1/1? ? Running? 0? ? ? ? ? 21h
```
## 七伊履、測試功能
```
創(chuàng)建一個pod,并暴露端口,驗證是否能訪問:
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=NodePort
[root@master ~]# kubectl get pods,svc
NAME? ? ? ? ? ? ? ? ? ? ? ? READY? STATUS? ? RESTARTS? AGE
pod/nginx-554b9c67f9-gsgsm? 1/1? ? Running? 0? ? ? ? ? 25h
pod/redis-686d55dddd-lhhl8? 1/1? ? Running? 0? ? ? ? ? 25h
NAME? ? ? ? ? ? ? ? TYPE? ? ? ? CLUSTER-IP? ? EXTERNAL-IP? PORT(S)? ? ? ? ? AGE
service/kubernetes? ClusterIP? 10.1.0.1? ? ? <none>? ? ? ? 443/TCP? ? ? ? ? 25h
service/nginx? ? ? ? NodePort? ? 10.1.6.189? ? <none>? ? ? ? 80:30551/TCP? ? 25h
service/redis? ? ? ? NodePort? ? 10.1.228.85? <none>? ? ? ? 2379:30642/TCP? 25h
```
## 八.配置kubernetes-dashboard
```
wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
修改yaml文件:
[root@k8s-master ~]# vim kubernetes-dashboard.yaml
修改內(nèi)容:
109? ? spec:
110? ? ? containers:
111? ? ? - name: kubernetes-dashboard
112? ? ? ? image: easzlab/kubernetes-dashboard-amd64:v1.10.1? # 修改此行
......
157 spec:
158? type: NodePort? ? # 增加此行
159? ports:
160? ? - port: 443
161? ? ? targetPort: 8443
162? ? ? nodePort: 30001? # 增加此行
163? selector:
164? ? k8s-app: kubernetes-dashboard
k8s-dashboard只允許30000以上端口訪問
[root@k8s-master ~]# kubectl apply -f kubernetes-dashboard.yaml
訪問頁面 https://172.18.0.171:30001
可能會不能訪問款违,因為原yaml文件的token有問題唐瀑。
此時需要我們手動生成一個token
進入到目錄 cd /etc/kubernetes/pki/
1.創(chuàng)建一個證書
[root@master pki]# (umask 077; openssl genrsa -out dashboard.key 2048)
2.簽署證書
openssl req -new -key dashboard.key -out dashboard.csr -subj "/O=zkxy/CN=kubernetes-dashboard"
3.使用集群的ca簽署證書
openssl x509 -req -in dashboard.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out dashboard.crt -days 5000
4.把創(chuàng)建好的證書給集群使用
完全刪除:dashboard:sudo kubectl -n kube-system delete $(sudo kubectl -n kube-system get pod -o name | grep dashboard)
kubectl create secret generic kubernetes-dashboard-certs --from-file=dashboard.crt=./dashboard.crt --from-file=dashboard.key=./dashboard.key? -n kube-system? #我們需要把我們創(chuàng)建的證書創(chuàng)建為secret給k8s使用
5.注釋掉dashboad.yaml的關(guān)于secret的配置
#apiVersion: v1
#kind: Secret
#metadata:
#? labels:
#? ? k8s-app: kubernetes-dashboard
#? name: kubernetes-dashboard-certs
#? namespace: kube-system
#type: Opaque
6.重新應用yaml文件
kubectl create -f kubernetes-dashboard.yaml
7.重新登錄即可彈出kubernetes儀表盤
此時需要創(chuàng)建默認的集群管理員用戶,并申請token訪問集群
kubectl create serviceaccount zkxy-admin -n kube-system
## 綁定集群資源
kubectl create clusterrolebinding zkxy-cluster-admin --clusterrole=zkxy-cluster-admin --serviceaccount=kube-system:zkxy-admin
## 獲取命名空間中的token
[root@master ~]# kubectl get secret -n kube-system
zkxy-admin-token-4dpbz? ? ? ? ? ? ? ? ? ? ? ? ? kubernetes.io/service-account-token? 3? ? ? 22h
## 使用該secret 獲取token
[root@master ~]#? kubectl describe secret zkxy-admin-token-4dpbz -n kube-system
Name:? ? ? ? zkxy-admin-token-4dpbz
Namespace:? ? kube-system
Labels:? ? ? <none>
Annotations:? kubernetes.io/service-account.name: zkxy-admin
? ? ? ? ? ? ? kubernetes.io/service-account.uid: 3a169baf-55f9-4cc4-abb5-950962b2315c
Type:? kubernetes.io/service-account-token
Data
====
ca.crt:? ? 1025 bytes
namespace:? 11 bytes
token:? ? ? eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJ6a3h5LWFkbWluLXRva2VuLTRkcGJ6Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6InpreHktYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIzYTE2OWJhZi01NWY5LTRjYzQtYWJiNS05NTA5NjJiMjMxNWMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06emt4eS1hZG1pbiJ9.am-UOQPoSEYWWp-CKLu5k9q3Ysh5GksRQBG9zOqNsJ2O_5zWUChdKrPPTlSTGJfz1ZiHtYRuKeRloSKem65IbuSHSfKfI_bKqTioqpzfDQSBMh2Hz4gvmiyJw3sk2g2DRCynjFjjSWB0QDgVemMn7vEPdcnPD0AwFxW0pwSPJI--hkdSbCTfm5ZXtHsvDt4avQGP1BAVw1IWeke9XsRouHurJU9I19-14LXzUWmY7nBceMCf7pWiho68gyea3kIar0JmCMtRJHAWOyWOxojocsfIb2iDsq9eK6SqhgJjXCrDMABUMErjZ-ACIA94e3q1gbwFPBGIhEXrDFUPK1z-dQ
復制上述token到頁面插爹,就可以訪問集群了哄辣。
```
## 九、總結(jié)
? 至此赠尾,一個一主二從的k8s集群就已經(jīng)搭建好力穗,期間踩了很多的坑,遇到錯誤不要緊萍虽,追根溯源去解決就好了睛廊。下一篇將詳細闡述如何將已有的微服務(wù)架構(gòu)(spring cloud)遷移到容器云上。