環(huán)境

• kubernetes 1.15
• gitlab 12.6.0-ee
• gitlab runner 12.6.0
• helm 2.14.3

kubernetes骏全、helm和gitlab已提前安裝配置好。helm配置可參考http://www.reibang.com/p/8aec2776f3f1，為gitlab添加kubernetes可參考http://www.reibang.com/p/1208c132b84c屡贺。

安裝配置

下載chart

下載gitlab runner helm chart
https://gitlab.com/gitlab-org/charts/gitlab-runner/tree/master

添加訪問(wèn)gitlab的證書(shū)

添加gitlab的證書(shū)：

kubectl --namespace <NAMESPACE> create secret generic <SECRET_NAME> --from-file=<CERTFICATE_FILENAME>

#xxx為你的證書(shū)名
/home/admin(master) # kubectl create secret -n default generic gitlab-cert --from-file=./xxx.crt 

查看

/home/admin(master) # kubectl get secret                                                         
NAME                                 TYPE                                  DATA   AGE
default-token-4w6nw                  kubernetes.io/service-account-token   3      58d
gangly-zorse-gitlab-runner           Opaque                                2      23h
gitlab-cert                          Opaque                                1      14d

修改values.yaml

根據(jù)個(gè)人要求修改對(duì)應(yīng)配置，以下為我個(gè)人修改了的配置

# image: gitlab/gitlab-runner:alpine-v11.6.0 
image: ip:port/gitlab/gitlab-runner:alpine-bleeding-1 #這里鏡像我用的自己私有倉(cāng)庫(kù)的鏡像
gitlabUrl: https://gitlab.example.com/ #這里輸入你自己的gitlab的地址。
runnerRegistrationToken: " " #輸入你的注冊(cè)token

你可以把runner安裝為共享的或者特定的，token的查看分別在：Admin area > runner > Set up a shared Runner manually

shared.png

Project > Setting > CI / CD Settings > Runners > Expand

specific.png

certsSecretName: gitlab-cert #輸入你剛才創(chuàng)建的secret的名字
runners:
  ## Default container image to use for builds when none is specified
  ##
  image: ip:port/ubuntu:16.04 #這里我用的自己私有倉(cāng)庫(kù)里的鏡像
  imagePullSecrets: [regsecret] #輸入私有倉(cāng)庫(kù)的secret几睛，可參考http://www.reibang.com/p/de030582cd75
  helpers:
    # cpuLimit: 200m
    # memoryLimit: 256Mi
    cpuRequests: 100m
    memoryRequests: 128Mi
    image: ip:port/gitlab-runner-helper:x86_64-577f813d #這里也是用的我自己私有倉(cāng)庫(kù)里的鏡像
    # image: gitlab/gitlab-runner-helper:x86_64-latest 

如遇到couldn't execute POST against https://xxx/api/v4/runners: Post https://xxx/api/v4/runners: x509: certificate signed by unknown authority，修改envVars:

envVars:
  - name: RUNNER_EXECUTOR
    value: kubernetes
  - name: CI_SERVER_TLS_CA_FILE
    value: /home/gitlab-runner/.gitlab-runner/certs/<CERTFICATE_FILENAME>.crt
  - name: CONFIG_FILE
    value: /home/gitlab-runner/.gitlab-runner/config.toml

測(cè)試的時(shí)候ci/cd 遇到unable to get local issuer certificate的問(wèn)題粤攒，不知道怎么在helm的chart改所森，所以gitlab-runner-helper的鏡像是我自己關(guān)了git的ssl再打包的鏡像，知道怎么改的可以留言一下夯接，感謝焕济。
關(guān)閉git ssl認(rèn)證：git config --global http.sslVerify false

綁定本地 Docker 守護(hù)進(jìn)程

用 volume 綁定的形式把本地 docker.sock 通過(guò) host_path 的方式掛載到 runner 中，修改configmap.yaml文件

# git-lab/runner/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "gitlab-runner.fullname" . }}
  labels:
    app: {{ include "gitlab-runner.fullname" . }}
    chart: {{ include "gitlab-runner.chart" . }}
    release: "{{ .Release.Name }}"
    heritage: "{{ .Release.Service }}"
data:
  entrypoint: |
    #!/bin/bash
    set -e
    mkdir -p /home/gitlab-runner/.gitlab-runner/
    cp /scripts/config.toml /home/gitlab-runner/.gitlab-runner/

    # Register the runner
    if [[ -f /secrets/accesskey && -f /secrets/secretkey ]]; then
      export CACHE_S3_ACCESS_KEY=$(cat /secrets/accesskey)
      export CACHE_S3_SECRET_KEY=$(cat /secrets/secretkey)
    fi

    if [[ -f /secrets/gcs-applicaton-credentials-file ]]; then
      export GOOGLE_APPLICATION_CREDENTIALS="/secrets/gcs-applicaton-credentials-file"
    else
      if [[ -f /secrets/gcs-access-id && -f /secrets/gcs-private-key ]]; then
        export CACHE_GCS_ACCESS_ID=$(cat /secrets/gcs-access-id)
        # echo -e used to make private key multiline (in google json auth key private key is oneline with \n)
        export CACHE_GCS_PRIVATE_KEY=$(echo -e $(cat /secrets/gcs-private-key))
      fi
    fi

    if [[ -f /secrets/runner-registration-token ]]; then
      export REGISTRATION_TOKEN=$(cat /secrets/runner-registration-token)
    fi

    if [[ -f /secrets/runner-token ]]; then
      export CI_SERVER_TOKEN=$(cat /secrets/runner-token)
    fi

    if ! sh /scripts/register-the-runner; then
      exit 1
    fi

    # add volume config
    cat >>/home/gitlab-runner/.gitlab-runner/config.toml <<EOF
      [[runners.kubernetes.volumes.host_path]]
            name = "docker"
            mount_path = "/var/run/docker.sock"
    EOF
    # Start the runner
    exec /entrypoint run --user=gitlab-runner \
      --working-directory=/home/gitlab-runner

  config.toml: |
    concurrent = {{ .Values.concurrent }}
    check_interval = {{ .Values.checkInterval }}
    log_level = {{ default "info" .Values.logLevel | quote }}
    {{- if .Values.logFormat }}
    log_format = {{ .Values.logFormat | quote }}
    {{- end }}
    {{- if .Values.metrics.enabled }}
    listen_address = '[::]:9252'
    {{- end }}
  configure: |
    set -e
    cp /init-secrets/* /secrets
  register-the-runner: |
    #!/bin/bash
    MAX_REGISTER_ATTEMPTS=30

    for i in $(seq 1 "${MAX_REGISTER_ATTEMPTS}"); do
      echo "Registration attempt ${i} of ${MAX_REGISTER_ATTEMPTS}"
      /entrypoint register \
        {{- range .Values.runners.imagePullSecrets }}
        --kubernetes-image-pull-secrets {{ . | quote }} \
        {{- end }}
        {{- range $key, $val := .Values.runners.nodeSelector }}
        --kubernetes-node-selector {{ $key | quote }}:{{ $val | quote }} \
        {{- end }}
        {{- range $key, $value := .Values.runners.podLabels }}
        --kubernetes-pod-labels {{ $key | quote }}:{{ $value | quote }} \
        {{- end }}
        {{- range $key, $val := .Values.runners.podAnnotations }}
        --kubernetes-pod-annotations {{ $key | quote }}:{{ $val | quote }} \
        {{- end }}
        {{- range $key, $value := .Values.runners.env }}
        --env {{ $key | quote -}} = {{- $value | quote }} \
        {{- end }}
        {{- if and (hasKey .Values.runners "runUntagged") .Values.runners.runUntagged }}
        --run-untagged=true \
        {{- end }}
        {{- if and (hasKey .Values.runners "protected") .Values.runners.protected }}
        --access-level="ref_protected" \
        {{- end }}
        --non-interactive

      retval=$?

      if [ ${retval} = 0 ]; then
        break
      elif [ ${i} = ${MAX_REGISTER_ATTEMPTS} ]; then
        exit 1
      fi

      sleep 5 
    done

    exit 0

  check-live: |
    #!/bin/bash
    if /usr/bin/pgrep -f .*register-the-runner; then
      exit 0
    elif /usr/bin/pgrep gitlab.*runner; then
      exit 0
    else
      exit 1
    fi

使用helm安裝的gitlab runner

安裝

/home/admin/git-lab/runner(master) # ls 
CHANGELOG.md  Chart.yaml  CONTRIBUTING.md  LICENSE  NOTICE  README.md  scripts  templates  values.yaml
-------------------------------------------------------------------------------------
/home/admin/git-lab/runner(master) # helm install ./  
/home/admin/git-lab/runner(master) # helm list  
NAME            REVISION    UPDATED                     STATUS      CHART                         APP VERSION   NAMESPACE
gangly-zorse    1           Thu Jan  2 17:32:23 2020    DEPLOYED    gitlab-runner-0.12.0-beta     bleeding      default  
/home/admin/git-lab/runner(master) # kubectl get pods
NAME                                          READY   STATUS    RESTARTS   AGE
gangly-zorse-gitlab-runner-7d4f5f9bbb-th4m6   1/1     Running   0          24h

helm 刪除命令helm delete --purge <NAME>

在gitlab上查看runner是否部署成功

runner.png

簡(jiǎn)單測(cè)試

創(chuàng)建Dockerfile文件

FROM ip:port/tomcat:7  #我用的自己私有倉(cāng)庫(kù)的jing
RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN echo "Asia/Shanghai" > /etc/timezone
EXPOSE 8080

創(chuàng)建一個(gè)deployment.yaml文件

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: tomcat-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: tomcat-demo
  template:
    metadata:
      labels:
        app: tomcat-demo
    spec:
      imagePullSecrets:
        - name: regsecret
      containers:
      - name: tomcat-demo
        image: REGISTRY/TAG:7
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: tomcat-demo
spec:
  ports:
  - port: 80
    targetPort: 8080
    name: tomcat-demo
  selector:
    app: tomcat-demo
  type: NodePort

創(chuàng)建.gitlab-ci.yml文件

創(chuàng)建一個(gè)project盔几，并在里面創(chuàng)建一個(gè)名為.gitlab-ci.yml的文件吼蚁，內(nèi)容如下：

#just a test
image: ip:port/docker:stable-dind
variables:
  DOCKER_DRIVER: overlay2
  REGISTRY: "ip:port" #私有倉(cāng)庫(kù)地址
  TAG: "tomcat" 
stages:
  - build
  - deploy
docker-build:
  stage: build
    script:
      - echo "Building Dockerfile-based application..."
      - docker login https://ip:port -u <USER> -p <PASSWORD> #填寫(xiě)自己私有倉(cāng)庫(kù)的用戶和密碼
      - docker pull ip:port/tomcat:7 #拉取tomcat7的鏡像
      - docker build -t $REGISTRY/$TAG:$CI_COMMIT_SHORT_SHA .
      - docker push $REGISTRY/$TAG:$CI_COMMIT_SHORT_SHA
k8s-deploy:
  image: ip:port/bitnami/kubectl:latest #使用的自己倉(cāng)庫(kù)里的
  stage: deploy
  script:
    - echo "deploy to k8s cluster "
    - sed -i "s/REGISTRY/$REGISTRY/g" deployment.yaml
    - sed -i "s|TAG|$TAG|g" deployment.yaml
    - kubectl apply -f deployment.yaml

保存.gitlab-ci.yml文件并push到已經(jīng)配置好了runner的project，這里我創(chuàng)建了個(gè)空的project進(jìn)行的runner部署測(cè)試问欠。
運(yùn)行git push -u origin master后查看你的project的ci/cd。

demo.png

如果failed粒蜈，也可點(diǎn)進(jìn)build里去查看失敗原因顺献。

參考

• https://docs.gitlab.com/runner/executors/kubernetes.html
• https://docs.gitlab.com/runner/install/kubernetes.html
• https://docs.gitlab.com/ee/ci/runners/
• https://gitlab.com/gitlab-org/gitlab-runner/issues/3968
• https://gitlab.com/gitlab-org/gitlab-foss/tree/master/lib/gitlab/ci/templates
• https://www.cnblogs.com/Sinte-Beuve/p/11739196.html