一. 持續(xù)集成/部署/交付概述

持續(xù)集成(Continuous Integration又固，CI): 代碼合并、構建须教、部署皿渗、測試都在一起斩芭，不斷地執(zhí)行這個過程，并對結果反饋乐疆。
持續(xù)部署(Continuous Deployment划乖，CD): 部署到測試環(huán)境、預生產(chǎn)環(huán)境挤土、生產(chǎn)環(huán)境琴庵。
持續(xù)交付(Continuous Delivery，CD): 將最終產(chǎn)品發(fā)布到生產(chǎn)環(huán)境仰美，給用戶使用迷殿。

image.png

image.png

image.png

二. 準備工作

1. 對項目的理解

? 單體架構、微服務
? 怎么部署
? 啟動是否有依賴

2. 部署到k8s平臺流程

(1). 制作鏡像
(2). 容器放到Pod
(3). 控制器管理Pod
(4). 暴露應用
(5). 對外發(fā)布應用
(6). 日志管理/監(jiān)控

3. 不同環(huán)境區(qū)分配置文件

? configmap
? entrypoint.sh
? 統(tǒng)一配置中心咖杂，例如 Apollo庆寺，Disconf

4. Harbor鏡像倉庫

部署參考: http://www.reibang.com/p/7ca6c59f9882

5. Git代碼版本倉庫

(1). 安裝Git

# yum install git

(2). 創(chuàng)建Git用戶密碼

# useradd git
# passwd git

(3). 創(chuàng)建倉庫

# su - git
$ mkdir java-demo.git
$ cd java-demo.git
$ git --bare init

(4). Git服務器SSH免交互認證

配置客戶端(10.40.6.213)與Git服務器SSH免交互認證

# mkdir git && cd git
# git clone git@10.40.6.165:/home/git/java-demo.git
# touch 123
# git add .
# git commit -m 'test'
# git push origin master

(5). 密鑰免交互測試

# ssh-keygen  
# ssh-copy-id git@10.40.6.165
# ssh git@10.40.6.165
# cd .. && rm -rf java-demo 
# git clone git@10.40.6.165:/home/git/java-demo.git

三. Kubernetes中部署Jenkins

參考:https://github.com/jenkinsci/kubernetes-plugin/tree/fc40c869edfd9e3904a9a56b0f80c5a25e988fa1/src/main/kubernetes

image.png

NFS服務部署：http://www.reibang.com/p/26003390626e
創(chuàng)建NFS 動態(tài)供給參考：http://www.reibang.com/p/092eb3aacefc

1. 部署有狀態(tài)的jenkins Pod

# cat statefulset-jenkins.yml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: jenkins
  labels:
    name: jenkins
spec:
  serviceName: jenkins
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      name: jenkins
      labels:
        name: jenkins
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: jenkins
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts-alpine
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
            - containerPort: 50000
          resources:
            limits:
              cpu: 1
              memory: 1Gi
            requests:
              cpu: 0.5
              memory: 500Mi
          env:
            - name: LIMITS_MEMORY
              valueFrom:
                resourceFieldRef:
                  resource: limits.memory
                  divisor: 1Mi
            - name: JAVA_OPTS
              value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=GMT+08
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
          livenessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12
          readinessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12
      securityContext:
        fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
      name: jenkins-home
    spec:
      storageClassName: "managed-nfs-storage"
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

  securityContext:
    fsGroup: 1000  這個配置可以去掉，但得先給后端存儲切換組诉字，如果用這個配置止邮，后面jenkins 使用久了，jenkins 重啟后會需要很久參能啟動成功奏窑，這樣就會影響健康檢測失敗导披，pod一直啟動失敗

2. 創(chuàng)建jenkins service

# cat service-jenkins.yml
apiVersion: v1
kind: Service
metadata:
  name: jenkins
spec:
  selector:
    name: jenkins
  type: NodePort
  ports:
    -
      name: http
      port: 80
      targetPort: 8080
      protocol: TCP
      nodePort: 30006
    -
      name: agent
      port: 50000
      protocol: TCP

3. 創(chuàng)建jenkins ingress

# cat ingress-jenkins.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jenkins
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/tls-acme: "true"
    # 如果上傳插件超出默認會報"413 Request Entity Too Large", 增加 client_max_body_size
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    # nginx-ingress controller版本小于 0.9.0.beta-18 的配置
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/proxy-body-size: 50m
    ingress.kubernetes.io/proxy-request-buffering: "off"
spec:
  rules:
  - host: jenkins.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: jenkins
          servicePort: 80

4. jenkins認證授權

# cat service-account.yml
---
# 創(chuàng)建名為jenkins的ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins

---
# 創(chuàng)建名為jenkins的Role，授予允許管理API組的資源Pod
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
# 將名為jenkins的Role綁定到名為jenkins的ServiceAccount
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins

5. 獲取jenkins密碼并登錄

# kubectl create -f .
登錄容器獲取jenkins登錄密碼：
# kubectl exec -it jenkins-0 bash
bash-4.4$ cat /var/jenkins_home/secrets/initialAdminPassword
0f8c730f4b7a426098283cf94aa57231

jenkins地址：http://10.40.6.213:30006
jenkins 安裝插件：pipeline埃唯、git撩匕、kubernetes

四. Jenkins在K8S中動態(tài)創(chuàng)建代理

1. 傳統(tǒng)架構與K8S 架構的區(qū)別

(1). 傳統(tǒng)Master/Slave架構，Master收到Job后墨叛，將請求轉發(fā)到Slave節(jié)點處理止毕。Slave節(jié)點數(shù)固定，Slave節(jié)點未能自動申縮容漠趁。

Jenkins Master/Slave架構

(2). K8S中Jenkins Master/Slave架構扁凛，Master收到Job后，會自動創(chuàng)建Slave節(jié)點處理此Job闯传，根據(jù)客戶端的Job自動申縮容谨朝。

K8S中Jenkins Master/Slave架構

2. jenkins插件連接K8S配置

需要先安裝pipeline插件, 建議不在UI上配置pod創(chuàng)建模版，免得以后每來一個項目都要創(chuàng)建甥绿，管理不方便字币，建議使用pipeline統(tǒng)一 配置。
Kubernetes插件介紹:https://github.com/jenkinsci/kubernetes-plugin

image.png

五. 構建Jenkins Slave鏡像

參考:https://github.com/jenkinsci/docker-jnlp-slave

image.png

1. 構建Jenkins Slave鏡像環(huán)境準備

構建Jenkins Slave鏡像環(huán)境準備：
代碼拉裙猜啤：git洗出，安裝git命令
單元測試：忽略，這不是我們擅長的,如果公司有可以寫進來
代碼編譯：maven图谷，安裝maven包
構建鏡像：Dockerfile文件翩活、docker命令(通過掛載宿主機docker)
推送鏡像：docker命令(通過掛載宿主機docker)
鏡像啟動后支持slave: 下載官方slave.jar包（獲融搴椤：http://10.40.6.213:30006/jnlpJars/slave.jar）
啟動 slave.ja包：jenkins-slave啟動腳步(通過參考文檔URL)
maven配置文件：settings.xml （這里配置阿里云的倉庫源）

獲取相關文件：
Dockerfile
jenkins-slave 啟動腳步
settings.xml
slave.jar

創(chuàng)建目錄并進入：
mkdir jenkins-slave && cd jenkins-slave

2. Dockerfile配置文件

# cat Dockerfile
FROM centos:7
LABEL maintainer liuzhousheng

RUN yum install -y java-1.8.0-openjdk maven curl git libtool-ltdl-devel && \
    yum clean all && \
    rm -rf /var/cache/yum/* && \
    mkdir -p /usr/share/jenkins

COPY slave.jar /usr/share/jenkins/slave.jar
COPY jenkins-slave /usr/bin/jenkins-slave
COPY settings.xml /etc/maven/settings.xml
RUN chmod +x /usr/bin/jenkins-slave

ENTRYPOINT ["jenkins-slave"]

3. jenkins-slave啟動腳步

# cat jenkins-slave
#!/usr/bin/env sh

if [ $# -eq 1 ]; then

    # if `docker run` only has one arguments, we assume user is running alternate command like `bash` to inspect the image
    exec "$@"

else

    # if -tunnel is not provided try env vars
    case "$@" in
        *"-tunnel "*) ;;
        *)
        if [ ! -z "$JENKINS_TUNNEL" ]; then
            TUNNEL="-tunnel $JENKINS_TUNNEL"
        fi ;;
    esac

    # if -workDir is not provided try env vars
    if [ ! -z "$JENKINS_AGENT_WORKDIR" ]; then
        case "$@" in
            *"-workDir"*) echo "Warning: Work directory is defined twice in command-line arguments and the environment variable" ;;
            *)
            WORKDIR="-workDir $JENKINS_AGENT_WORKDIR" ;;
        esac
    fi

    if [ -n "$JENKINS_URL" ]; then
        URL="-url $JENKINS_URL"
    fi

    if [ -n "$JENKINS_NAME" ]; then
        JENKINS_AGENT_NAME="$JENKINS_NAME"
    fi

    if [ -z "$JNLP_PROTOCOL_OPTS" ]; then
        echo "Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior"
        JNLP_PROTOCOL_OPTS="-Dorg.jenkinsci.remoting.engine.JnlpProtocol3.disabled=true"
    fi

    # If both required options are defined, do not pass the parameters
    OPT_JENKINS_SECRET=""
    if [ -n "$JENKINS_SECRET" ]; then
        case "$@" in
            *"${JENKINS_SECRET}"*) echo "Warning: SECRET is defined twice in command-line arguments and the environment variable" ;;
            *)
            OPT_JENKINS_SECRET="${JENKINS_SECRET}" ;;
        esac
    fi

    OPT_JENKINS_AGENT_NAME=""
    if [ -n "$JENKINS_AGENT_NAME" ]; then
        case "$@" in
            *"${JENKINS_AGENT_NAME}"*) echo "Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable" ;;
            *)
            OPT_JENKINS_AGENT_NAME="${JENKINS_AGENT_NAME}" ;;
        esac
    fi

    #TODO: Handle the case when the command-line and Environment variable contain different values.
    #It is fine it blows up for now since it should lead to an error anyway.

    exec java $JAVA_OPTS $JNLP_PROTOCOL_OPTS -cp /usr/share/jenkins/slave.jar hudson.remoting.jnlp.Main -headless $TUNNEL $URL $WORKDIR $OPT_JENKINS_SECRET $OPT_JENKINS_AGENT_NAME "$@"
fi

4. 獲取slave.jar包

wget http://10.40.6.213:30006/jnlpJars/slave.jar

5. maven源配置文件settings.xml

maven源配置文件settings.xml，這里配置阿里云的源菠镇。

# cat settings.xml
<?xml version="1.0" encoding="UTF-8"?>

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <pluginGroups>
  </pluginGroups>

  <proxies>
  </proxies>

  <servers>
  </servers>

  <mirrors>
    <mirror>
      <id>central</id>
      <mirrorOf>central</mirrorOf>
      <name>aliyun maven</name>
      <url>https://maven.aliyun.com/repository/public</url>
    </mirror>
  </mirrors>

  <profiles>
  </profiles>

</settings>

6. 構建鏡像, 并推送至私有鏡像倉庫

# docker build -t 10.40.6.165/library/jenkins-slave-jdk:1.8 .
     ....
Successfully built 3aaa12e391fc
Successfully tagged 10.40.6.165/library/jenkins-slave-jdk:1.8

# docker login 10.40.6.165   ###先登錄鏡像倉庫再推送
Username: admin
Password:

# docker push 10.40.6.165/library/jenkins-slave-jdk:1.8

注意： 私有鏡像倉庫用的是http訪問澄峰，kubernetes是不信任的，得每個node節(jié)點配置信任辟犀。
參考：http://www.reibang.com/p/7ca6c59f9882

六. Jenkins Pipeline構建流水線發(fā)布

1. Jenkins Pipeline 核心概念

? Jenkins Pipeline是一套插件俏竞，支持在Jenkins中實現(xiàn)持續(xù)集成、交付管道;
? Pipeline通過特定語法從簡單到復雜的傳輸管道進行建模;
① 聲明式:遵循與Groovy相同語法堂竟。pipeline { }
② 腳本式:支持Groovy大部分功能魂毁，也是非常表達和靈活的工具。node { }
? Jenkins Pipeline的定義被寫入一個文本文件出嘹，稱為Jenkinsfile席楚。

Jenkins Pipeline 核心概念：
Node： 節(jié)點，一個 Node 就是一個 Jenkins 節(jié)點税稼，Master 或者 Agent烦秩，是執(zhí)行 Step 的具體運行環(huán)境，比如我們之前動態(tài)運行的 Jenkins Slave 就是一個 Node 節(jié)點
Stage： 階段郎仆，一個 Pipeline 可以劃分為若干個 Stage只祠，每個 Stage 代表一組操作，比如：Build、Test、Deploy等项滑，Stage 是一個邏輯分組的概念，可以跨多個 Node
Step： 步驟盗舰，Step 是最基本的操作單元，可以是打印一句話桂躏，也可以是構建一個 Docker 鏡像钻趋，由各類 Jenkins 插件提供，比如命令：sh ‘make’剂习，就相當于我們平時 shell 終端中執(zhí)行 make 命令一樣蛮位。
參考:https://jenkins.io/doc/book/pipeline/syntax/

image.png

image.png

2. 拉取代碼配置

(1). Pipeline語法使用

image.png

(2). 生成Pipeline語句

image.png

(3). 拉取Git代碼秘鑰配置

image.png

(4). Pileline語句

image.png

然后構建拉取代碼測試

七. 編寫Pipeline腳本完成CI階段

參考文檔：https://github.com/jenkinsci/kubernetes-plugin
pipeline script注意：
podTemplate 中l(wèi)abel和node括號里的字符串要一致
podTemplate 中cloud為之前配置jenkins Cloud 名稱
containerTemplate中name最好保持為"jnlp"
pipeline script變量docker_registry_auth、git_auth进倍、k8s_auth通過保存在jenkins憑據(jù)中相應的憑據(jù)ID土至。

Pipeline Script如下：

// 公共
def registry = "10.40.6.165"
// 項目
def project = "project"
def app_name = "java-demo"
def image_name = "${registry}/${project}/${app_name}:${BUILD_NUMBER}"
def git_address = "git@10.40.6.165:/home/git/java-demo.git"
// 認證
def secret_name = "registry-pull-secret"
def docker_registry_auth = "eba0f763-747e-47e3-a7ed-20eaf5cbab31"  //Harbor login auth
def git_auth = "fec843cc-3cc4-4c97-969f-df0b0f1bcc50" //git login auth
def k8s_auth = "80e66a86-d189-4555-b1ef-054285031b7a"  // k8s auth,CI階段不用到, CD 部署時才用到

podTemplate(label: 'jenkins-slave', cloud: 'kubernetes', containers: [
    containerTemplate(
        name: 'jnlp',
        image: "${registry}/library/jenkins-slave-jdk:1.8"
    ),
  ],
  volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
    hostPathVolume(mountPath: '/usr/bin/docker', hostPath: '/usr/bin/docker'),
    hostPathVolume(mountPath: '/etc/localtime', hostPath: '/etc/localtime')
  ],
)
{
  node("jenkins-slave"){
      // 第一步
      stage('拉取代碼'){
         checkout([$class: 'GitSCM', branches: [[name: '${Branch}']], userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_address}"]]])
      }
      // 第二步
      stage('代碼編譯'){
          sh "mvn clean package -Dmaven.test.skip=true"
      }
      // 第三步
      stage('構建鏡像'){
          //Harbor鏡像倉庫登錄驗證购对，
          withCredentials([usernamePassword(credentialsId: "${docker_registry_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
            sh """
              echo '
                FROM 10.40.6.165/project/java-demo:v1
                RUN rm -rf /usr/local/tomcat/webapps/*
                ADD target/*.war /usr/local/tomcat/webapps/ROOT.war
              ' > Dockerfile
              ls
              ls target
              docker build -t ${image_name} .
              docker login -u ${username} -p '${password}' ${registry}
              docker push ${image_name}
            """
            }
      }
  }
}

pipeline script代碼可選擇保存在git代碼首目錄的代碼中
測試剛構建的鏡像：
docker run -d -p 666:8080 10.40.6.165/project/java-demo:17
訪問地址：http://10.40.6.213:666

八. Jenkins在K8s中持續(xù)部署

Kubernetes Continuous Deploy插件: 用于將資源配置部署到Kubernetes
插件介紹: https://plugins.jenkins.io/kubernetes-cd
支持以下資源類型:
? Deployment
? Replica Set
? Daemon Set
? Pod
? Job
? Service
? Ingress
? Secret

Jenkins將鏡像部署到kubernetes需要配置:
①. 要連接kubernetes 需要用到kubeconfig配置文件猾昆，要在pipeline 中讀取此配置文件，得配置到jenkins到憑據(jù)中骡苞，使用jenkins Kubernetes Continuous Deploy插件讀取此憑據(jù)的ID(kubeconfigID)即可垂蜗；
②. 指定部署哪個資源文件楷扬，deplayment yaml文件；
③. 資源是否使用到secret驗證信息贴见，得指定烘苹。

1. 生成kubeconfig

(1). 主體角色綁定

liuzhousheng 用戶主體綁定deployment-secret角色：
mkdir deployment-client && cd deployment-client

創(chuàng)建角色權限
# cat rbac-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: deployment-secret
rules:
- apiGroups: [""]
  resources: ["pods","services"]
  verbs: ["get", "list", "watch", "create", "delete","update","patch"]
- apiGroups: ["extensions", "apps"]
  resources: ["deployments","ingresses"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

user liuzhousheng主體綁定deployment-secret角色
# cat rbac-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: deployment-rolebinding
subjects:
- kind: User
  name: liuzhousheng # Name is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role #this must be Role or ClusterRole
  name: deployment-secret # this must match the name of the Role or ClusterRole you wish to bind to
  apiGroup: rbac.authorization.k8s.io

# kubectl create -f rbac-role.yaml
# kubectl create -f rbac-rolebinding.yaml

(2). 生成kubeconfig配置文件

根據(jù)集群根證書頒發(fā)客戶端證書，然后生成連接集群配置文件liuzhousheng-kubeconfig配置文件片部。
mkdir liuzhousheng && cd liuzhousheng

# cat rbac-user.sh
#!/bin/bash
cat > liuzhousheng-csr.json <<EOF
{
  "CN": "liuzhousheng",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing"
    }
  ]
}
EOF
###簽發(fā)一個客戶端證書,注意要指定根證書
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes liuzhousheng-csr.json | cfssljson -bare liuzhousheng

#-----------------------------------
##生成配置文件镣衡，使用配置文件連接集群

#配置集群，這里的ca.pem為集群ca證書
kubectl config set-cluster kubernetes \
  --certificate-authority=ca.pem \
  --embed-certs=true \
  --server=https://10.40.6.175:6443 \
  --kubeconfig=liuzhousheng-kubeconfig

#客戶端證書配置
kubectl config set-credentials liuzhousheng \
  --client-key=liuzhousheng-key.pem \
  --client-certificate=liuzhousheng.pem \
  --embed-certs=true \
  --kubeconfig=liuzhousheng-kubeconfig

#配置上下文
kubectl config set-context default \
  --cluster=kubernetes \
  --user=liuzhousheng \
  --kubeconfig=liuzhousheng-kubeconfig

kubectl config use-context default --kubeconfig=liuzhousheng-kubeconfig

# cp  /usr/local/src/k8s/kube-apiserver/{ca.pem,ca-key.pem,ca-config.json} ./
# bash rbac-user.sh
# cat liuzhousheng-kubeconfig

2. 配置kuberconfig憑據(jù)

將liuzhousheng-kubeconfig配置文件保存到jenkins憑據(jù)中

image.png

image.png

3. 編寫deploy.yaml文件

將deploy.yaml文件提交到代碼倉庫中：

# cat deploy.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: java-demo
  template:
    metadata:
      labels:
        app: java-demo
    spec:
      imagePullSecrets:
      - name: $SECRET_NAME
      containers:
      - name: tomcat
        image: $IMAGE_NAME
        ports:
        - containerPort: 8080
          name: web
        livenessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 5
          failureThreshold: 12
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 5
          failureThreshold: 12


---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  type: NodePort
  selector:
    app: java-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web
spec:
  rules:
  - host: java.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: web
          servicePort: 80

# git add .
# git commit -m 'deploy.yaml'
# git push origin master

4. 編寫pipeline 腳步

pipeline腳本：

// 公共
def registry = "10.40.6.165"
// 項目
def project = "project"
def app_name = "java-demo"
def image_name = "${registry}/${project}/${app_name}:${BUILD_NUMBER}"
def git_address = "git@10.40.6.165:/home/git/java-demo.git"
// 認證
def secret_name = "registry-pull-secret"
def docker_registry_auth = "eba0f763-747e-47e3-a7ed-20eaf5cbab31"  //Harbor login auth
def git_auth = "fec843cc-3cc4-4c97-969f-df0b0f1bcc50" //git login auth
def k8s_auth = "5a9f013d-b01e-43b6-b7d7-aef2b54c64c7"   //k8s login auth

podTemplate(label: 'jenkins-slave', cloud: 'kubernetes', containers: [
    containerTemplate(
        name: 'jnlp',
        image: "${registry}/library/jenkins-slave-jdk:1.8"
    ),
  ],
  volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
    hostPathVolume(mountPath: '/usr/bin/docker', hostPath: '/usr/bin/docker'),
    hostPathVolume(mountPath: '/etc/localtime', hostPath: '/etc/localtime')
  ],
)
{
  node("jenkins-slave"){
      // 第一步
      stage('拉取代碼'){
         checkout([$class: 'GitSCM', branches: [[name: '${Branch}']], userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_address}"]]])
      }
      // 第二步
      stage('代碼編譯'){
          sh "mvn clean package -Dmaven.test.skip=true"
      }
      // 第三步
      stage('構建鏡像'){
          //Harbor鏡像倉庫登錄驗證档悠，
          withCredentials([usernamePassword(credentialsId: "${docker_registry_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
            sh """
              echo '
                FROM 10.40.6.165/project/java-demo:v1
                RUN rm -rf /usr/local/tomcat/webapps/*
                ADD target/*.war /usr/local/tomcat/webapps/ROOT.war
              ' > Dockerfile
              ls
              ls target
              docker build -t ${image_name} .
              docker login -u ${username} -p '${password}' ${registry}
              docker push ${image_name}
            """
            }
      }
    // 第四步
      stage('部署到K8S平臺'){
          sh """
          sed -i 's#\$IMAGE_NAME#${image_name}#' deploy.yaml
          sed -i 's#\$SECRET_NAME#${secret_name}#' deploy.yaml
          """
          kubernetesDeploy configs: 'deploy.yaml', kubeconfigId: "${k8s_auth}"
      }
  }
}

腳本注意變量：
k8s_auth：jenkins保存kubeconfig配置憑據(jù)的ID

5. 創(chuàng)建harbor鏡像倉庫secret驗證

pipeline腳步中的secret_name變量為：訪問harbor鏡像倉庫私有庫secret驗證廊鸥，在kubernetes master 創(chuàng)建，如下：

# kubectl create secret --help
# kubectl create secret docker-registry --help
# kubectl create secret docker-registry registry-pull-secret --docker-username=admin --docker-password=Harbor12345 --docker-email=888888@qq.com --docker-server=10.40.6.165

可選擇通過yaml文件定義創(chuàng)建：

首先在其中一個node上登錄私有倉庫（docker 訪問私庫的配置辖所，這里不做描述）
# docker login my.registry

登錄成功后會在/root/.docker目錄下生產(chǎn)config.json文件惰说，然后執(zhí)行如下命令：
# cat /root/.docker/config.json | base64 -w 0

該命令會將你的認證信息通過base64編碼，生成一個編碼之后的字符串缘回。
在kubernetes中的master節(jié)點中創(chuàng)建registry-pull-secret-vpc.yaml：
# cat registry-pull-secret-vpc.yaml
apiVersion: v1
kind: Secret
metadata:
  name: registry-pull-secret-vpc
  namespace: test
data:
  .dockerconfigjson: xxxxxxxxxx
type: kubernetes.io/dockerconfigjson

# kubectl apply -f registry-pull-secret-vpc.yaml

6. 配置代碼拉取分支變量

Branch：代碼拉取分支選擇

image.png

7.構建測試

jenkins構建吆视，然后綁定hosts訪問 10.40.6.213 java.example.com

九. Pipeline腳本與源代碼版本管理

1. Pipeline 腳本版本管理

將Pipeline 腳本內容保存到同源代碼git代碼倉庫中的Jenkinsfile文件中，同代碼版本控制

# ll
drwxr-xr-x 2 root root    34 6月  29 18:12 db
-rw-r--r-- 1 root root  1155 7月   1 22:09 deploy.yaml
-rw-r--r-- 1 root root  2255 7月   1 23:16 Jenkinsfile
-rw-r--r-- 1 root root 11357 6月  29 18:12 LICENSE
-rw-r--r-- 1 root root  1930 6月  29 18:12 pom.xml
-rw-r--r-- 1 root root    88 6月  29 18:12 README.md
drwxr-xr-x 3 root root    18 6月  29 18:12 src

# git add .
# git commit -m 'add Jenkinsfile'
# git push origin master

2. 修改jenkins Pipeline定義

修改jenkins UI配置

image.png

回滾：
# kubectl get deployment
# kubectl rollout history deployment web

回滾到上一個版本
# kubectl rollout history deployment web

回滾到指定版本
# kubectl rollout undo deployment web --to-revision=5

擴縮容：
# kubectl scale deployment web --replicas=5

十. K8s滾動發(fā)布實現(xiàn)原理

1. 滾動發(fā)布原理

每次只升級一個或多個服務酥宴，升級完成后加入
生產(chǎn)環(huán)境啦吧，不斷執(zhí)行這個過程，直到集群中的
全部舊版升級新版本拙寡。
特點:
? 用戶無感知丰滑，平滑過渡 缺點:
? 部署周期長
? 發(fā)布策略較復雜
? 不易回滾

image.png

2. K8s滾動發(fā)布架構

1個Deployment
2個ReplicaSet

image.png

十一. 在K8s中實現(xiàn)灰度發(fā)布方案

1. 灰度發(fā)布原理

只升級部分服務，即讓一部分用戶繼續(xù)用老版本倒庵，一 部分用戶開始用新版本褒墨，如果用戶對新版本沒有什么 意見，那么逐步擴大范圍擎宝，把所有用戶都遷移到新版 本上面來郁妈。
特點:
? 保證整體系統(tǒng)穩(wěn)定性
? 用戶無感知，平滑過渡 缺點:
? 自動化要求高

image.png

2. K8s灰度發(fā)布架構

結合LB绍申，2個Deployment

image.png

十二. 小結

? 使用Jenkins三個插件
? Kubernetes
? Pipeline
? Kubernetes Continuous Deploy

? CI/CD環(huán)境特點
? Slave彈性伸縮
? 基于鏡像隔離構建環(huán)境 ? 流水線發(fā)布噩咪，易維護

? Jenkins參數(shù)化構建可幫助你完成更復雜環(huán)境CI/CD