01 搭建K8s集群
GitHub：https://github.com/kubernetes/kubeadm
課程中：使用kubeadm搭建一個(gè)3臺(tái)機(jī)器組成的k8s集群澎剥，1臺(tái)master節(jié)點(diǎn)嗽元，2臺(tái)worker節(jié)點(diǎn)如果大家機(jī)器配置不夠，也可以使用在線的鞠呈，或者minikube的方式或者1個(gè)master和1個(gè)worker
配置要求：
- One or more machines running one of:
- Ubuntu 16.04+
- Debian 9+
- CentOS 7
- Red Hat Enterprise Linux (RHEL) 7
- Fedora 25+
- HypriotOS v1.0.1+
- Container Linux (tested with 1800.6.0)
- 2 GB or more of RAM per machine (any less will leave little room for your apps)
- 2 CPUs or more
- Full network connectivity between all machines in the cluster (public or private network is fine)
- Unique hostname, MAC address, and product_uuid for every node. See here for more details.
- Certain ports are open on your machines. See here for more details.
- Swap disabled. You MUST disable swap in order for the kubelet to work properly.
1.1 版本統(tǒng)一
Docker 18.09.0
---
kubeadm-1.14.0-0
kubelet-1.14.0-0
kubectl-1.14.0-0
---
k8s.gcr.io/kube-apiserver:v1.14.0
k8s.gcr.io/kube-controller-manager:v1.14.0
k8s.gcr.io/kube-scheduler:v1.14.0
k8s.gcr.io/kube-proxy:v1.14.0
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.3.10
k8s.gcr.io/coredns:1.3.1
---
calico:v3.9
1.2 準(zhǔn)備3臺(tái)centos
大家根據(jù)自己的情況來準(zhǔn)備centos7的虛擬機(jī)。
要保證彼此之間能夠ping通杭抠，也就是處于同一個(gè)網(wǎng)絡(luò)中离咐，虛擬機(jī)的配置要求上面也描述咯旱幼。
1.3 更新并安裝依賴
3臺(tái)機(jī)器都需要執(zhí)行
yum -y update
yum install -y conntrack ipvsadm ipset jq sysstat curl iptables libseccomp
1.4 安裝Docker
根據(jù)之前學(xué)習(xí)的Docker方式[Docker第一節(jié)課的筆記中也有這塊的說明]
在每一臺(tái)機(jī)器上都安裝好Docker笙纤，版本為18.09.0
01 安裝必要的依賴 sudo yum install -y yum-utils \ device-mapper-persistent-data \ lvm2 02 設(shè)置docker倉庫 sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo 【設(shè)置要設(shè)置一下阿里云鏡像加速器】 sudo mkdir -p /etc/docker sudo tee /etc/docker/daemon.json <<-'EOF' { "registry-mirrors": ["這邊替換成自己的實(shí)際地址"] } EOF sudo systemctl daemon-reload 03 安裝docker yum install -y docker-ce-18.09.0 docker-ce-cli-18.09.0 containerd.io 04 啟動(dòng)docker sudo systemctl start docker && sudo systemctl enable docker
1.5 修改hosts文件
(1)master
# 設(shè)置master的hostname耗溜，并且修改hosts文件
sudo hostnamectl set-hostname m
vi /etc/hosts
192.168.8.51 m
192.168.8.61 w1
192.168.8.62 w2
(2)兩個(gè)worker
# 設(shè)置worker01/02的hostname，并且修改hosts文件
sudo hostnamectl set-hostname w1
sudo hostnamectl set-hostname w2
vi /etc/hosts
192.168.8.51 m
192.168.8.61 w1
192.168.8.62 w2
(3)使用ping測(cè)試一下
1.6 系統(tǒng)基礎(chǔ)前提配置
# (1)關(guān)閉防火墻
systemctl stop firewalld && systemctl disable firewalld
# (2)關(guān)閉selinux
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
# (3)關(guān)閉swap
swapoff -a
sed -i '/swap/s/^\(.*\)$/#\1/g' /etc/fstab
# (4)配置iptables的ACCEPT規(guī)則
iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat && iptables -P FORWARD ACCEPT
# (5)設(shè)置系統(tǒng)參數(shù)
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
1.7 Installing kubeadm, kubelet and kubectl
(1)配置yum源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
(2)安裝kubeadm&kubelet&kubectl
yum install -y kubeadm-1.14.0-0 kubelet-1.14.0-0 kubectl-1.14.0-0
(3)docker和k8s設(shè)置同一個(gè)cgroup
# docker
vi /etc/docker/daemon.json
"exec-opts": ["native.cgroupdriver=systemd"],
systemctl restart docker
# kubelet省容，這邊如果發(fā)現(xiàn)輸出directory not exist抖拴，也說明是沒問題的，大家繼續(xù)往下進(jìn)行即可
sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/g" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
systemctl enable kubelet && systemctl start kubelet
1.8 proxy/pause/scheduler等國(guó)內(nèi)鏡像
(1)查看kubeadm使用的鏡像
kubeadm config images list
可以發(fā)現(xiàn)這里都是國(guó)外的鏡像
k8s.gcr.io/kube-apiserver:v1.14.0
k8s.gcr.io/kube-controller-manager:v1.14.0
k8s.gcr.io/kube-scheduler:v1.14.0
k8s.gcr.io/kube-proxy:v1.14.0
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.3.10
k8s.gcr.io/coredns:1.3.1
(2)解決國(guó)外鏡像不能訪問的問題
- 創(chuàng)建kubeadm.sh腳本腥椒，用于拉取鏡像/打tag/刪除原有鏡像
#!/bin/bash
set -e
KUBE_VERSION=v1.14.0
KUBE_PAUSE_VERSION=3.1
ETCD_VERSION=3.3.10
CORE_DNS_VERSION=1.3.1
GCR_URL=k8s.gcr.io
ALIYUN_URL=registry.cn-hangzhou.aliyuncs.com/google_containers
images=(kube-proxy:${KUBE_VERSION}
kube-scheduler:${KUBE_VERSION}
kube-controller-manager:${KUBE_VERSION}
kube-apiserver:${KUBE_VERSION}
pause:${KUBE_PAUSE_VERSION}
etcd:${ETCD_VERSION}
coredns:${CORE_DNS_VERSION})
for imageName in ${images[@]} ; do
docker pull $ALIYUN_URL/$imageName
docker tag $ALIYUN_URL/$imageName $GCR_URL/$imageName
docker rmi $ALIYUN_URL/$imageName
done
- 運(yùn)行腳本和查看鏡像
# 運(yùn)行腳本
sh ./kubeadm.sh
# 查看鏡像
docker images
- 將這些鏡像推送到自己的阿里云倉庫【可選阿宅，根據(jù)自己實(shí)際的情況】
# 登錄自己的阿里云倉庫
docker login --username=xxx registry.cn-hangzhou.aliyuncs.com
#!/bin/bash
set -e
KUBE_VERSION=v1.14.0
KUBE_PAUSE_VERSION=3.1
ETCD_VERSION=3.3.10
CORE_DNS_VERSION=1.3.1
GCR_URL=k8s.gcr.io
ALIYUN_URL=xxx
images=(kube-proxy:${KUBE_VERSION}
kube-scheduler:${KUBE_VERSION}
kube-controller-manager:${KUBE_VERSION}
kube-apiserver:${KUBE_VERSION}
pause:${KUBE_PAUSE_VERSION}
etcd:${ETCD_VERSION}
coredns:${CORE_DNS_VERSION})
for imageName in ${images[@]} ; do
docker tag $GCR_URL/$imageName $ALIYUN_URL/$imageName
docker push $ALIYUN_URL/$imageName
docker rmi $ALIYUN_URL/$imageName
done
運(yùn)行腳本 sh ./kubeadm-push-aliyun.sh
1.9 kube init初始化master
(1)kube init流程
01-進(jìn)行一系列檢查，以確定這臺(tái)機(jī)器可以部署kubernetes
02-生成kubernetes對(duì)外提供服務(wù)所需要的各種證書可對(duì)應(yīng)目錄
/etc/kubernetes/pki/*
03-為其他組件生成訪問kube-ApiServer所需的配置文件
ls /etc/kubernetes/
admin.conf controller-manager.conf kubelet.conf scheduler.conf
04-為 Master組件生成Pod配置文件笼蛛。
ls /etc/kubernetes/manifests/*.yaml
kube-apiserver.yaml
kube-controller-manager.yaml
kube-scheduler.yaml
05-生成etcd的Pod YAML文件家夺。
ls /etc/kubernetes/manifests/*.yaml
kube-apiserver.yaml
kube-controller-manager.yaml
kube-scheduler.yaml
etcd.yaml
06-一旦這些 YAML 文件出現(xiàn)在被 kubelet 監(jiān)視的/etc/kubernetes/manifests/目錄下，kubelet就會(huì)自動(dòng)創(chuàng)建這些yaml文件定義的pod伐弹，即master組件的容器拉馋。master容器啟動(dòng)后，kubeadm會(huì)通過檢查localhost：6443/healthz這個(gè)master組件的健康狀態(tài)檢查URL惨好，等待master組件完全運(yùn)行起來
07-為集群生成一個(gè)bootstrap token
08-將ca.crt等 Master節(jié)點(diǎn)的重要信息煌茴，通過ConfigMap的方式保存在etcd中，工后續(xù)部署node節(jié)點(diǎn)使用
09-最后一步是安裝默認(rèn)插件日川，kubernetes默認(rèn)kube-proxy和DNS兩個(gè)插件是必須安裝的
(2)初始化master節(jié)點(diǎn)
官網(wǎng)：https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
注意：此操作是在主節(jié)點(diǎn)上進(jìn)行
# 本地有鏡像
kubeadm init --kubernetes-version=1.14.0 --apiserver-advertise-address=192.168.8.51 --pod-network-cidr=10.244.0.0/16
【若要重新初始化集群狀態(tài)：kubeadm reset蔓腐，然后再進(jìn)行上述操作】
記得保存好最后kubeadm join的信息
(3)根據(jù)日志提示
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
此時(shí)kubectl cluster-info查看一下是否成功
(4)查看pod驗(yàn)證一下
等待一會(huì)兒，同時(shí)可以發(fā)現(xiàn)像etc龄句，controller回论，scheduler等組件都以pod的方式安裝成功了
注意：coredns沒有啟動(dòng)，需要安裝網(wǎng)絡(luò)插件
kubectl get pods -n kube-system
(5)健康檢查
curl -k https://localhost:6443/healthz
1.10 部署calico網(wǎng)絡(luò)插件
選擇網(wǎng)絡(luò)插件：https://kubernetes.io/docs/concepts/cluster-administration/addons/
calico網(wǎng)絡(luò)插件：https://docs.projectcalico.org/v3.9/getting-started/kubernetes/
calico分歇，同樣在master節(jié)點(diǎn)上操作
# 在k8s中安裝calico
kubectl apply -f https://docs.projectcalico.org/v3.9/manifests/calico.yaml
# 確認(rèn)一下calico是否安裝成功
kubectl get pods --all-namespaces -w
1.11 kube join
記得保存初始化master節(jié)點(diǎn)的最后打印信息【注意這邊大家要自己的傀蓉，下面我的只是一個(gè)參考】
kubeadm join 192.168.0.51:6443 --token yu1ak0.2dcecvmpozsy8loh \
--discovery-token-ca-cert-hash sha256:5c4a69b3bb05b81b675db5559b0e4d7972f1d0a61195f217161522f464c307b0
(1)在woker01和worker02上執(zhí)行上述命令
(2)在master節(jié)點(diǎn)上檢查集群信息
kubectl get nodes
NAME STATUS ROLES AGE VERSION
master-kubeadm-k8s Ready master 19m v1.14.0
worker01-kubeadm-k8s Ready <none> 3m6s v1.14.0
worker02-kubeadm-k8s Ready <none> 2m41s v1.14.0
1.12 再次體驗(yàn)Pod
(1)定義pod.yml文件，比如pod_nginx_rs.yaml
cat > pod_nginx_rs.yaml <<EOF
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: nginx
labels:
tier: frontend
spec:
replicas: 3
selector:
matchLabels:
tier: frontend
template:
metadata:
name: nginx
labels:
tier: frontend
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
EOF
(2)根據(jù)pod_nginx_rs.yml文件創(chuàng)建pod
kubectl apply -f pod_nginx_rs.yaml
(3)查看pod
kubectl get pods
kubectl get pods -o wide
kubectl describe pod nginx
(4)感受通過rs將pod擴(kuò)容
kubectl scale rs nginx --replicas=5
kubectl get pods -o wide
(5)刪除pod
kubectl delete -f pod_nginx_rs.yaml
02 Basic
2.1 yaml文件
2.1.1 簡(jiǎn)介
YAML（IPA: /?j?m?l/）是一個(gè)可讀性高的語言职抡，參考了XML葬燎、C、Python等缚甩。
理解：Yet Another Markup Language
后綴：可以是.yml或者是.yaml谱净，更加推薦.yaml，其實(shí)用任意后綴都可以擅威，只是閱讀性不強(qiáng)
2.1.2 基礎(chǔ)
- 區(qū)分大小寫
- 縮進(jìn)表示層級(jí)關(guān)系壕探，相同層級(jí)的元素左對(duì)齊
- 縮進(jìn)只能使用空格，不能使用TAB
- "#"表示當(dāng)前行的注釋
- 是JSON文件的超級(jí)郊丛，兩個(gè)可以轉(zhuǎn)換
- ---表示分隔符李请，可以在一個(gè)文件中定義多個(gè)結(jié)構(gòu)
- 使用key: value派继，其中":"和value之間要有一個(gè)英文空格
2.1.3 Maps
2.1.3.1 簡(jiǎn)單
apiVersion: v1
kind: Pod
---表示分隔符，可選捻艳。要定義多個(gè)結(jié)構(gòu)一定要分隔
apiVersion表示key，v1表示value庆猫，英文":"后面要有一個(gè)空格
kind表示key认轨，Pod表示value
也可以這樣寫apiVersion: "v1"
轉(zhuǎn)換為JSON格式{ "apiVersion": "v1", "kind": "Pod" }
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
2.1.2.2 復(fù)雜
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
metadata表示key，下面的內(nèi)容表示value月培，該value中包含兩個(gè)直接的key：name和labels
name表示key嘁字，nginx-deployment表示value
labels表示key，下面的表示value杉畜，這個(gè)值又是一個(gè)map
app表示key纪蜒，nginx表示value
相同層級(jí)的記得使用空間縮進(jìn)，左對(duì)齊
轉(zhuǎn)換為JSON格式{ "apiVersion": "apps/v1", "kind": "Deployment", "metadata": { "name": "nginx-deployment", "labels": { "app": "nginx" } } }
2.1.4 Lists
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container01
image: busybox:1.28
- name: myapp-container02
image: busybox:1.28
containers表示key此叠，下面的表示value纯续，其中value是一個(gè)數(shù)組
數(shù)組中有兩個(gè)元素，每個(gè)元素里面包含name和image
image表示key灭袁，myapp-container表示value
轉(zhuǎn)換成JSON格式{ "apiVersion": "v1", "kind": "Pod", "metadata": { "name": "myapp", "labels": { "app": "myapp" } }, "spec": { "containers": [{ "name": "myapp-container01", "image": "busybox:1.28", }, { "name": "myapp-container02", "image": "busybox:1.28", }] } }
2.1.5 找個(gè)k8s的yaml文件
# yaml格式對(duì)于Pod的定義：
apiVersion: v1 #必寫猬错，版本號(hào)，比如v1
kind: Pod #必寫茸歧，類型倦炒，比如Pod
metadata: #必寫，元數(shù)據(jù)
name: nginx #必寫软瞎，表示pod名稱
namespace: default #表示pod名稱屬于的命名空間
labels:
app: nginx #自定義標(biāo)簽名字
spec: #必寫逢唤，pod中容器的詳細(xì)定義
containers: #必寫，pod中容器列表
- name: nginx #必寫涤浇，容器名稱
image: nginx #必寫鳖藕，容器的鏡像名稱
ports:
- containerPort: 80 #表示容器的端口
2.2 Container
2.2.1 Docker世界中
可以通過docker run運(yùn)行一個(gè)容器
或者定義一個(gè)yml文件，本機(jī)使用docker-compose只锭，多機(jī)通過docker swarm創(chuàng)建
2.2.2 K8S世界中
同樣以一個(gè)yaml文件維護(hù)吊奢，container運(yùn)行在pod中
2.3 Pod
官網(wǎng)：https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/
2.3.1 What is Pod
A Pod is the basic execution unit of a Kubernetes application
A Pod encapsulates an application’s container (or, in some cases, multiple containers), storage resources, a unique network IP, and options that govern how the container(s) should run
2.3.2 Pod初體驗(yàn)
(1)創(chuàng)建一個(gè)pod的yaml文件，名稱為nginx_pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
labels:
app: nginx
spec:
containers:
- name: nginx-container
image: nginx
ports:
- containerPort: 80
(2)根據(jù)該nginx_pod.yaml文件創(chuàng)建pod
kubectl apply -f nginx_pod.yaml
(3)查看pod
01 kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-pod 1/1 Running 0 29s
02 kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-pod 1/1 Running 0 40m 192.168.80.194 w2
03 kubectl describe pod nginx-pod
Name: nginx-pod
Namespace: default
Priority: 0
PriorityClassName: <none>
Node: w2/192.168.0.62
Start Time: Sun, 06 Oct 2019 20:45:35 +0000
Labels: app=nginx
Annotations: cni.projectcalico.org/podIP: 192.168.80.194/32
kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-pod","namespace":"default"},"spec":{"c...
Status: Running
IP: 192.168.80.194
Containers:
nginx-container:
Container ID: docker://eb2fd0b2906f53e9892e22a6fd791c9ac68fb8e5efce3bbf94ec12bae96e1984
Image: nginx
Image ID: docker-pullable:/
(4)可以發(fā)現(xiàn)該pod運(yùn)行在worker02節(jié)點(diǎn)上
于是來到worker02節(jié)點(diǎn)纹烹，docker ps一下
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eb2fd0b2906f nginx "nginx -g 'daemon of…" 6 minutes ago Up 6 minutes k8s_nginx-container_nginx-pod_default_3ee0706d-e87a-11e9-a904-5254008afee6_0
不妨進(jìn)入該容器試試[可以發(fā)現(xiàn)只有在worker02上有該容器页滚，因?yàn)閜od運(yùn)行在worker02上]：
docker exec -it k8s_nginx-container_nginx-pod_default_3ee0706d-e87a-11e9-a904-5254008afee6_0 bash
root@nginx-pod:/#
(5)訪問nginx容器
curl 192.168.80.194 OK，并且在任何一個(gè)集群中的Node上訪問都成功
(6)刪除Pod
kubectl delete -f nginx_pod.yaml
kubectl get pods
2.3.3 Storage and Networking
官網(wǎng)：https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/#networking
- Networking
Each Pod is assigned a unique IP address. Every container in a Pod shares the network namespace, including the IP address and network ports.
官網(wǎng)：https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/#storage
- Storage
A Pod can specify a set of shared storage Volumes. All containers in the Pod can access the shared volumes, allowing those containers to share data.
