1.Docker 安裝

1.1CentOS Docker安裝

https://www.runoob.com/docker/centos-docker-install.html

1.2Docker 鏡像加速

https://www.runoob.com/docker/docker-mirror-acceleration.html

1.3Docker Cgroup Driver改為systemd

命令：sudo vi /etc/docker/daemon.json

配置如下：

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

# 如果使用了加速器配置格式如下
{
  "registry-mirrors": ["https://sv2jerob.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"]
}

1.4啟動(dòng)Docker

sudo systemctl restart docker

可以查看docker的Cgroup Driver已修改為systemd煤裙，未修改前的值為cgroupfs,命令：sudo docker info

2.kubeadm 安裝

2.1使用阿里鏡像安裝

# 配置源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

# 安裝
yum install -y kubeadm

3.k8s master init

sudo kubeadm init

如果沒有翻墻，上面的命令會(huì)報(bào)錯(cuò):

[preflight] Some fatal errors occurred:
    [ERROR ImagePull]: failed to pull image [k8s.gcr.io/kube-apiserver:v1.18.1]: exit status 1
    [ERROR ImagePull]: failed to pull image [k8s.gcr.io/kube-controller-manager:v1.18.1]: exit status 1
    [ERROR ImagePull]: failed to pull image [k8s.gcr.io/kube-scheduler:v1.18.1]: exit status 1
    [ERROR ImagePull]: failed to pull image [k8s.gcr.io/kube-proxy:v1.18.1]: exit status 1
    [ERROR ImagePull]: failed to pull image [k8s.gcr.io/pause:3.2]: exit status 1
    [ERROR ImagePull]: failed to pull image [k8s.gcr.io/etcd:3.4.3-0]: exit status 1
    [ERROR ImagePull]: failed to pull image [k8s.gcr.io/coredns:1.6.7]: exit status 1

如果不能翻墻，可以手動(dòng)下載這些鏡像：

curl -s https://www.zhangguanzhang.com/pull | bash -s k8s.gcr.io/kube-apiserver:v1.18.1
curl -s https://www.zhangguanzhang.com/pull | bash -s k8s.gcr.io/kube-controller-manager:v1.18.1
curl -s https://www.zhangguanzhang.com/pull | bash -s k8s.gcr.io/kube-scheduler:v1.18.1
curl -s https://www.zhangguanzhang.com/pull | bash -s k8s.gcr.io/kube-proxy:v1.18.1
curl -s https://www.zhangguanzhang.com/pull | bash -s k8s.gcr.io/pause:3.2
curl -s https://www.zhangguanzhang.com/pull | bash -s k8s.gcr.io/etcd:3.4.3-0
curl -s https://www.zhangguanzhang.com/pull | bash -s k8s.gcr.io/coredns:1.6.7

4.k8s node join

重復(fù)1组去、2兩步，安裝好Docker、kubeadm后，執(zhí)行命令：

sudo kubeadm join 10.0.0.6:6443 --token sh3723.182lsru545rta8qc --discovery-token-ca-cert-hash sha256:d079daadd145a5f5badfcfc8cfed0cb1fa47f734f7e05edb985243aeda202b75

5.第一次使用 Kubernetes 集群所需要的配置命令

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

將剛剛部署生成的 Kubernetes 集群的安全配置文件暑塑，保存到當(dāng)前用戶的.kube 目錄下，kubectl 默認(rèn)會(huì)使用這個(gè)目錄下的授權(quán)信息訪問 Kubernetes 集群锅必。

如果不這么做的話事格，我們每次都需要通過 export KUBECONFIG 環(huán)境變量告訴 kubectl 這個(gè)安全配置文件的位置。

6.部署網(wǎng)絡(luò)插件

以Weave Net為例：

export kubever=$(kubectl version | base64 | tr -d '\n')
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever"

7.部署 Dashboard 可視化插件

7.1安裝

還是墻的問題况毅，使用私人的鏡像image: kubernetesui/dashboard:v2.0.0-rc7
sudo vi dashboard.yaml

# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: Namespace
metadata:
  name: kubernetes-dashboard

---

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30001
  selector:
    k8s-app: kubernetes-dashboard

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque

---

kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
    # Allow Dashboard to get metrics.
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  # Allow Metrics Scraper to get metrics from the Metrics server
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.0.0-rc7
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
            # Uncomment the following line to manually specify Kubernetes API server Host
            # If not specified, Dashboard will attempt to auto discover the API server and connect
            # to it. Uncomment only if the default does not work.
            # - --apiserver-host=http://my-address:port
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              # Create on-disk volume to store exec logs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
    spec:
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.4
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
          - mountPath: /tmp
            name: tmp-volume
      serviceAccountName: kubernetes-dashboard
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}

運(yùn)行pod

kubectl apply -f dashboard.yaml

查看pod的運(yùn)行情況

kubectl get pods -n kubernetes-dashboard

如果處于running狀態(tài)分蓖，查看一下dashboard的log日志

kubectl logs  -f  kubernetes-dashboard-cdbc9547c-7sb2n -n kubernetes-dashboard

kubectl logs  -f  dashboard-metrics-scraper-fb986f88d-nd9d6  -n kubernetes-dashboard

7.2允許外部訪問

kubectl proxy --address='0.0.0.0' --accept-hosts='^*$'

7.3瀏覽器中訪問

瀏覽器輸入https://IP:30001，通過token訪問

7.3獲取token

Dashboard支持Kubeconfig和Token兩種認(rèn)證方式尔许，為了簡(jiǎn)化配置，我們通過配置文件 dashboard-admin.yaml為Dashboard默認(rèn)用戶賦予admin權(quán)限

sudo vi dashboard-admin.yaml

文件內(nèi)容：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: dashboard-admin
  namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: dashboard-admin
subjects:
  - kind: ServiceAccount
    name: dashboard-admin
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

執(zhí)行kubectl apply使之生效

kubectl apply -f dashboard-admin.yaml

獲取token终娃，其中dashboard-admin-token-nhrx9通過命令kubectl get secret -n kube-system |grep dashboard-admin獲取

kubectl get secret -n kube-system |grep dashboard-admin
kubectl describe secret dashboard-admin-token-nhrx9 -n kube-system

8.安裝存儲(chǔ)插件

cd /usr/local/src
yum -y install git
git clone https://github.com/rook/rook.git
cd /usr/local/src/rook/cluster/examples/kubernetes/ceph
kubectl apply -f common.yaml
kubectl apply -f operator.yaml
kubectl apply -f cluster.yaml 

配置系統(tǒng)相關(guān)參數(shù)

# 臨時(shí)禁用selinux
# 永久關(guān)閉 修改/etc/sysconfig/selinux文件設(shè)置
sed -i 's/SELINUX=permissive/SELINUX=disabled/' /etc/sysconfig/selinux
setenforce 0

# 臨時(shí)關(guān)閉swap
# 永久關(guān)閉 注釋/etc/fstab文件里swap相關(guān)的行
swapoff -a



# 開啟forward
# Docker從1.13版本開始調(diào)整了默認(rèn)的防火墻規(guī)則
# 禁用了iptables filter表中FOWARD鏈
# 這樣會(huì)引起Kubernetes集群中跨Node的Pod無法通信

iptables -P FORWARD ACCEPT

# 配置轉(zhuǎn)發(fā)相關(guān)參數(shù)味廊，否則可能會(huì)出錯(cuò)
cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.swappiness=0
EOF
sysctl --system