準(zhǔn)備
所有k8s節(jié)點需要安裝nfs服務(wù)
一臺nfs服務(wù)器提供存儲
根據(jù)需求創(chuàng)建數(shù)據(jù)共享目錄
image.png
配置對所有k8s節(jié)點的 授權(quán)
檢查是否已經(jīng)授權(quán)
showmount -e 192.168.100.204
image.png
之后不需要掛載，直接開始使用nfs的存儲創(chuàng)建PV和PVC。
PersistentVolume（PV）是指由集群管理員配置提供的某存儲系統(tǒng)上的段存儲空間伤靠，它是對底層共享存儲的抽象，將共享存儲作為種可由用戶申請使的資源查吊，實現(xiàn)了“存儲消費”機制。通過存儲插件機制湖蜕，PV支持使用多種網(wǎng)絡(luò)存儲系統(tǒng)或云端存儲等多種后端存儲系統(tǒng)，例如宋列，NFS昭抒、RBD和Cinder等。PV是集群級別的資源炼杖，不屬于任何名稱空間灭返，用戶對PV資源的使需要通過PersistentVolumeClaim（PVC）提出的使申請（或稱為聲明）來完成綁定，是PV資源的消費者坤邪，它向PV申請?zhí)囟ù笮〉目臻g及訪問模式（如rw或ro）熙含，從創(chuàng)建出PVC存儲卷，后再由Pod資源通過PersistentVolumeClaim存儲卷關(guān)聯(lián)使艇纺，如下圖：
image
image.png
二 下載NFS插件
GitHub地址：https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client/deploy
1怎静、下載所需文件
for file in class.yaml deployment.yaml rbac.yaml; do wget https://raw.githubusercontent.com/kubernetes-incubator/external-storage/master/nfs-client/deploy/$file; done
2、創(chuàng)建RBAC授權(quán)
# cat rbac.yaml
kind: ServiceAccount
apiVersion: v1
metadata:
name: nfs-client-provisioner
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
roleRef:
kind: Role
name: leader-locking-nfs-client-provisioner
apiGroup: rbac.authorization.k8s.io
3黔衡、創(chuàng)建Storageclass類
# cat class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
archiveOnDelete: "false"
4蚓聘、創(chuàng)建NFS的deployment，修改相應(yīng)的NFS服務(wù)器IP及掛載路徑
# cat deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: nfs-client-provisioner
spec:
replicas: 1
selector:
matchLabels:
app: nfs-client-provisioner
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: quay.io/external_storage/nfs-client-provisioner:v2.0.0
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value: 192.168.1.100
- name: NFS_PATH
value: /huoban/k8s
volumes:
- name: nfs-client-root
nfs:
server: 192.168.1.100
path: /huoban/k8s
image.png
三盟劫、創(chuàng)建一個PV動態(tài)供給應(yīng)用實例
例如：創(chuàng)建一個nginx動態(tài)獲取PV
# cat nginx.yaml
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
selector:
matchLabels:
app: nginx
serviceName: "nginx"
replicas: 3
template:
metadata:
labels:
app: nginx
spec:
imagePullSecrets:
- name: nginx
terminationGracePeriodSeconds: 10
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
name: web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: www
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "managed-nfs-storage"
resources:
requests:
storage: 1Gi
啟動之后我們可以看到一下信息
# kubectl get pod,pv,pvc
pod/coredns-68c964545b-mszkt 0/1 Evicted 0 11d
pod/coredns-68c964545b-xzszx 1/1 Running 14 11d
pod/kube-apiserver-ha-proxy-k8s-master-1 1/1 Running 19 23d
pod/kube-apiserver-ha-proxy-k8s-node-1 1/1 Running 17 23d
pod/kube-apiserver-ha-proxy-k8s-node-2 1/1 Running 8 23d
pod/kube-flannel-ds-amd64-588x5 1/1 Running 1 3d19h
pod/kube-flannel-ds-amd64-jfv4h 1/1 Running 36 23d
pod/kube-flannel-ds-amd64-mkjqk 1/1 Running 5 10d
pod/metrics-server-8459f8db8c-h8bn7 2/2 Running 55 13d
pod/my-web-0 1/1 Running 1 2d18h
pod/my-web-1 1/1 Running 1 2d18h
pod/my-web-2 1/1 Running 1 2d18h
pod/my-web-3 1/1 Running 1 2d18h
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-122aef78-c633-4eb0-8edf-4d9a5fa633ac 1Gi RWO Delete Bound kube-system/www-my-web-0 managed-nfs-storage 2d18h
persistentvolume/pvc-6d3698d5-5e4d-48f8-bb86-c0a300c25b2a 1Gi RWO Delete Bound kube-system/www-my-web-2 managed-nfs-storage 2d18h
persistentvolume/pvc-77c8a432-7ee5-4b0d-afae-94a88e0a6113 5Gi RWX Delete Bound kube-system/managed-nfs-storage managed-nfs-storage 2d18h
persistentvolume/pvc-77e21151-70cc-4185-ad4b-12cf39c4128c 1Gi RWO Delete Bound kube-system/www-my-web-1 managed-nfs-storage 2d18h
persistentvolume/pvc-cf0c4a85-4d09-4918-b63c-a40c11c1b658 1Gi RWO Delete Bound kube-system/www-my-web-3 managed-nfs-storage 2d18h
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/managed-nfs-storage Bound pvc-77c8a432-7ee5-4b0d-afae-94a88e0a6113 5Gi RWX managed-nfs-storage 2d18h
persistentvolumeclaim/www-my-web-0 Bound pvc-122aef78-c633-4eb0-8edf-4d9a5fa633ac 1Gi RWO managed-nfs-storage 2d18h
persistentvolumeclaim/www-my-web-1 Bound pvc-77e21151-70cc-4185-ad4b-12cf39c4128c 1Gi RWO managed-nfs-storage 2d18h
persistentvolumeclaim/www-my-web-2 Bound pvc-6d3698d5-5e4d-48f8-bb86-c0a300c25b2a 1Gi RWO managed-nfs-storage 2d18h
persistentvolumeclaim/www-my-web-3 Bound pvc-cf0c4a85-4d09-4918-b63c-a40c11c1b658 1Gi RWO managed-nfs-storage 2d18h
現(xiàn)在夜牡，我們在NFS服務(wù)器上也可以看到自動生成了3個掛載目錄，單pod刪除之后數(shù)據(jù)還會存在
[10:41:55root@k8s-master-1 /nfs/data]#ll
total 0
drwxrwxrwx 2 root root 6 Mar 26 16:30 kube-system-managed-nfs-storage-pvc-77c8a432-7ee5-4b0d-afae-94a88e0a6113
drwxrwxrwx 2 root root 24 Mar 26 16:33 kube-system-www-my-web-0-pvc-122aef78-c633-4eb0-8edf-4d9a5fa633ac
drwxrwxrwx 2 root root 24 Mar 26 16:33 kube-system-www-my-web-1-pvc-77e21151-70cc-4185-ad4b-12cf39c4128c
drwxrwxrwx 2 root root 24 Mar 26 16:33 kube-system-www-my-web-2-pvc-6d3698d5-5e4d-48f8-bb86-c0a300c25b2a
drwxrwxrwx 2 root root 24 Mar 26 16:33 kube-system-www-my-web-3-pvc-cf0c4a85-4d09-4918-b63c-a40c11c1b658
StatefulSet應(yīng)用有以下特點：
1.唯一的網(wǎng)絡(luò)標(biāo)識
2.域名訪問(<statefulsetName-index>.<service-name>.svc.cluster.local) 如：web-0.nginx.default.svc.cluster.local
3.獨立的持久存儲
4.有序的部署和刪除
[13:45:46root@k8s-master-1 ~/nfs/lk]#cat nfs-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: managed-nfs-storage
spec:
storageClassName: "managed-nfs-storage"
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
[13:46:18root@k8s-master-1 ~/nfs/lk]#cat nginx_deployment.yaml
apiVersion: v1
kind: Service
metadata:
name: my-nginx-service #服務(wù)名
labels:
app: my-nginx #標(biāo)簽名
spec:
ports:
- port: 80
name: web
clusterIP: None #Headless 無頭服務(wù)配置
selector:
app: my-nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: my-web
spec:
selector:
matchLabels:
app: my-nginx
serviceName: "my-nginx-service"
replicas: 5 #部署4個副本
template:
metadata:
labels:
app: my-nginx
spec:
containers:
- name: nginx
image: registry.aliyuncs.com/google_containers/nginx-slim:0.8
ports:
- containerPort: 80
name: my-web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: www
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "managed-nfs-storage"
resources:
requests:
storage: 1Gi
大部分內(nèi)容來自：http://www.reibang.com/p/8c2c6946767e
