前言

最近一直在從事云原生相關(guān)的開發(fā)蚂四，開發(fā)的時候難免要進行 CRD 的控制器和 webhook 的開發(fā)歼疮。從最開始的磕磕絆絆，到現(xiàn)在還算順暢近顷，所以打算將這個過程總結(jié)了一下生音，寫成文檔，讓后來者少走些彎路窒升。

CRD 是什么缀遍，大家可以自行查閱，以后有時間精力饱须，我也會寫一篇文檔總結(jié)一下域醇。

這篇文檔，主要分為以下幾個部分：

• kubebuilder 安裝
• 項目創(chuàng)建
• 項目實現(xiàn)
• 項目部署

在 kubebuilder 安裝部分中蓉媳，會和大家如何進行安裝 kubebuilder譬挚。

在項目創(chuàng)建部分會和大家一起使用 kubebuilder 創(chuàng)建我們的項目。

在項目實現(xiàn)部分會和大家完成我們的項目代碼酪呻。

在項目部署會部分和大家一起將我們的代碼部署到k8s集群减宣。

kubebuilder 安裝

介紹

Kubebuilder 是一個使用自定義資源定義 (CRD) 構(gòu)建 Kubernetes API 的框架。它可以提升我們開發(fā)CRD的效率玩荠，降低我們的開發(fā)成本漆腌，使我們低成本的進行 k8s Operator 開發(fā)。

和大多數(shù)開源項目一樣阶冈，kubebuilder 的安裝非常簡單闷尿。 只需要下載好我們需要的對應(yīng)的版本的安裝包就可以安裝了。

需要注意的是女坑，版本兼容問題：不同 kubebuilder 版本創(chuàng)建出來 controller 可能與 k8s 集群存在一定的兼容性問題填具，開發(fā)前，需要先明確自己的集群版本堂飞，然后再去選擇 kubebuilder 的版本灌旧。

安裝

在這里我選擇的是 v3.3.0 的 kubebuilder 進行安裝。

kubebuilder 開源倉庫

kubebuilder 發(fā)版地址

# 下載kubebuilder 二進制文件 注意 需要根據(jù)自己的系統(tǒng)內(nèi)核和CPU架構(gòu)進行下載  
sudo curl -L -o kubebuilder https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.3.0/kubebuilder_linux_amd64  
# 修改權(quán)限  
chmod +x kubebuilder  
# 移動到bin目錄  
mv kubebuilder /usr/local/bin/  
# 查看版本  
kubebuilder version  
Version: main.version{KubeBuilderVersion:"3.3.0", KubernetesVendor:"1.23.1", GitCommit:"47859bf2ebf96a64db69a2f7074ffdec7f15c1ec", BuildDate:"2022-01-18T17:03:29Z", GoOs:"linux", GoArch:"amd64"}

這樣绰筛，我們就完成了 kubebuilder 的安裝枢泰，是不是非常簡便。

接下來铝噩，我們就可以使用它進行項目創(chuàng)建了衡蚂。

項目創(chuàng)建

項目創(chuàng)建中，我們會使用到一些 kubebuilder 常用的命令來幫助我們快速創(chuàng)建項目。

命令介紹

我將在實戰(zhàn)中使用下面3個命令來創(chuàng)建我們的項目：

# 初始化項目
# 使用 --domain 可以指定<域>, 我們在這個項目中所創(chuàng)建的所有的API組都將是<group>.<domain>
# 使用 --projiect-name 可以指定我們項目的名稱毛甲。
kubebuilder init --domain test.crd --project-name test.crd

# 創(chuàng)建API 這個命令可以幫助我們快速創(chuàng)建CRD資源以及CRD控制器
# 使用 --group 可以指定資源組年叮。
# 使用 --version 可以指定資源的版本。
# 使用  --kind 可以指定資源的類型玻募，這個類型就是你自定義的CRD的名字只损。
kubebuilder create api --group test --version v1 --kind Test

# 創(chuàng)建WebHook
# --group  --kind  --version 和上述一致
# 使用 --defaulting 會為我們生成一個 webhook 的 Deafult 的接口實現(xiàn)，需要我們自己完成七咧，用來補全CRD的默認值
# 使用--programmatic-validation 會為我們生成一個 webhook 的 Validation 的接口實現(xiàn)跃惫，要我們自己完成，用來校驗CRD資源在創(chuàng)建艾栋，更新爆存，刪除時數(shù)據(jù)是否正確。
kubebuilder create webhook --group test --version v1 --kind Test --defaulting --programmatic-validation

實戰(zhàn)

在這里蝗砾，我們創(chuàng)建一個CRD先较，這個CRD是用來送外賣訂單：

• kind: Order
• group: demo
• domain: sumeng.com
• version: v1

初始化項目

創(chuàng)建項目

# 創(chuàng)建項目文件夾
mkdir crd-demo
cd crd-demo
# 初始化項目
kubebuilder init --domain sumeng.com --project-name crd-demo
# 查看項目結(jié)構(gòu)
tree
.
├── config # 配置文件目錄
│   ├── default
│   │   ├── kustomization.yaml
│   │   ├── manager_auth_proxy_patch.yaml
│   │   └── manager_config_patch.yaml
│   ├── manager
│   │   ├── controller_manager_config.yaml
│   │   ├── kustomization.yaml
│   │   └── manager.yaml
│   ├── prometheus
│   │   ├── kustomization.yaml
│   │   └── monitor.yaml
│   └── rbac
│       ├── auth_proxy_client_clusterrole.yaml
│       ├── auth_proxy_role_binding.yaml
│       ├── auth_proxy_role.yaml
│       ├── auth_proxy_service.yaml
│       ├── kustomization.yaml
│       ├── leader_election_role_binding.yaml
│       ├── leader_election_role.yaml
│       ├── role_binding.yaml
│       └── service_account.yaml
├── Dockerfile # 構(gòu)建鏡像的 Dockerfile
├── go.mod
├── go.sum
├── hack
│   └── boilerplate.go.txt # 代碼頭文件
├── LICENSE
├── main.go 
├── Makefile
├── PROJECT
└── README.md

修改頭文件

# 修改頭文件
vim hack/boilerplate.go.txt

boilerplate.go.txt

/*
Copyright 2022 The CRD-Demo Authors. # 修改了這里

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

修改 Makefile

使用golang版本為1.18時 直接使用 kubebuilder create api命令會出現(xiàn) controller-gen:no such file or directory 的錯誤。我們需要修改一下 Makefile 來避免錯誤:

修改 go-get-tool 使用go install 來替代 go get

# go-get-tool will 'go get' any package $2 and install it to $1.
PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))
define go-get-tool
@[ -f $(1) ] || { \
set -e ;\
TMP_DIR=$$(mktemp -d) ;\
cd $$TMP_DIR ;\
go mod init tmp ;\
echo "Downloading $(2)" ;\
GOBIN=$(PROJECT_DIR)/bin go install $(2) ;\ # 修改這里 go get --> go install
rm -rf $$TMP_DIR ;\
}
endef

創(chuàng)建API

# 創(chuàng)建API
# kubebuilder 會詢問你是否創(chuàng)建資源和控制器悼粮，我們都選擇是就可以了闲勺。
kubebuilder create api --group demo --version v1 --kind Order

# 查看結(jié)構(gòu) 我們會發(fā)現(xiàn)多出來了以下內(nèi)容。
tree
.
├── api # 我們資源的資源結(jié)構(gòu)存放在這里矮锈。
│   └── v1 # 我們定義的版本在結(jié)構(gòu)中以目錄的形式呈現(xiàn)出來霉翔。
│       ├── groupversion_info.go # 我們定義的CRD的 Group 和 version 信息
│       ├── order_types.go # CRD 結(jié)構(gòu)體定義，我們需要根據(jù)自己的需求修改。
│       └── zz_generated.deepcopy.go
├── bin
│   └── controller-gen # 用來生成控制器的工具
├── config # 該目錄下的大多數(shù)配置文件都不需要手動編寫，可以使用工具生成
│   ├── crd # crd 相關(guān)配置
│   │   ├── kustomization.yaml
│   │   ├── kustomizeconfig.yaml
│   │   └── patches
│   │       ├── cainjection_in_orders.yaml
│   │       └── webhook_in_orders.yam
│   ├── rbac # 權(quán)限相關(guān)
│   │   ├── order_editor_role.yaml
│   │   ├── order_viewer_role.yaml
│   └── samples # 使用demo
│       └── demo_v1_order.yaml
├── controllers # 控制器代碼目錄
│   ├── order_controller.go
│   └── suite_test.go

創(chuàng)建WebHook

# 創(chuàng)建WebHook
kubebuilder create webhook --group demo --version v1 --kind Order --defaulting --programmatic-validation
# 查看結(jié)構(gòu)
tree
.
├── api
│   └── v1
│       ├── order_webhook.go # webhook 代碼文件
│       └── webhook_suite_test.go
├── config
│   ├── certmanager #認證密鑰相關(guān)
│   │   ├── certificate.yaml
│   │   ├── kustomization.yaml
│   │   └── kustomizeconfig.yaml
│   ├── default
│   │   ├── manager_webhook_patch.yaml
│   │   └── webhookcainjection_patch.yaml
│   └── webhook
│       ├── kustomization.yaml
│       ├── kustomizeconfig.yaml
│       └── service.yaml

項目實現(xiàn)

經(jīng)過上述步驟畸陡，我們已經(jīng)完成了項目的創(chuàng)建坦刀，接下來我們就開始項目實現(xiàn)的過程，主要分為以下幾步：

• 項目介紹
• CRD 編寫
• 控制器實現(xiàn)
• WebHook實現(xiàn)

項目介紹

外賣訂單臭杰，訂單里有客戶的點商品信息粤咪，商家信息，以及配送信息渴杆。

訂單狀態(tài)變更為：

• "" ---> 未接單
• 未接單 ---> 已接單
• 已接單 ---> 制作中
• 制作中 ---> 制作完成
• 制作完成 ---> 待配送
• 待配送 ---> 配送中
• 配送中 ---> 訂單完成

因為我們沒有實際的商家和騎手來更新狀態(tài)寥枝，所以我們隨機一個時間，來推動訂單進程磁奖。

接下來囊拜，我們就來實現(xiàn)這樣一個簡單的demo吧！

CRD 編寫

./api/v1/order_types.go

Spec

Spec 部分主要用于描述我們的自定義資源信息比搭。在這里冠跷，我們用來編寫訂單的信息，如商品信息，商家信息蜜托。

// OrderSpec defines the desired state of Order  
type OrderSpec struct {  
   // the information for the Shop  
   // +kubebuilder:validation:Required  
   Shop *ShopInfo `json:"shop"`  
  
   // Commodities is a list of CommodityInfo  
   // +kubebuilder:validation:Required  
   Commodities []CommodityInfo `json:"commodity"`  
  
   // TotalPrice is the total price of the Order  
   // +kubebuilder:validation:Required  
   TotalPrice int64 `json:"totalPrice"`  
  
   // Remark of Order  
   // +optional  
   Remark string `json:"remark,omitempty"`  
}  
  
type ShopInfo struct {  
   // Name of the shop   
   Name string `json:"name"`  
}  
  
type CommodityInfo struct {  
   // Name of the commodity   
   Name string `json:"name"`  
  
   // Price of the commodity   
   Price int64 `json:"price"`  
  
   // Quantity of commodity   
   Quantity int64 `json:"quantity"`  
}

上述代碼中抄囚，我們可以看到一些帶 ‘+’ 的注釋，這些是我們在生成 crd yaml 文件時的一些標(biāo)識橄务，他可以幫我做一些簡單的功能幔托，節(jié)約我們的開發(fā)時間。

• +kubebuilder:validation:Required 用于標(biāo)記該字段是必填項蜂挪。
• +optional 用于標(biāo)記該字段是可選項

更多的標(biāo)記我們可以查閱：Kubebuilder Book

Status

status 用來記錄資源的狀態(tài)柑司，這樣我們用來記錄訂單的狀態(tài)。

// OrderStatus defines the observed state of Order  
type OrderStatus struct {  
   // Conditions a list of conditions an order can have.   
   // +optional   
   Conditions []OrderCondition `json:"conditions,omitempty"`  
   // +optional  
   Phase OrderPhase `json:"phase,omitempty"`  
   // +optional  
   Message string `json:"message,omitempty"`  
}

type OrderCondition struct {  
   // Type of order condition.   Type OrderConditionType `json:"type"`  
   // Phase of the condition, one of True, False, Unknown.  
   Status corev1.ConditionStatus `json:"status"`  
   // The last time this condition was updated.  
   LastUpdateTime metav1.Time `json:"lastUpdateTime,omitempty"`  
   // Last time the condition transitioned from one status to another.  
   LastTransitionTime metav1.Time `json:"lastTransitionTime,omitempty"`  
   // The reason for the condition's last transition.  
   Reason string `json:"reason,omitempty"`  
   // A human-readable message indicating details about the transition.  
   Message string `json:"message,omitempty"`  
}  
  
type OrderConditionType string  
  
const (  
   ConditionShop     OrderConditionType = "Shop"  
   ConditionDelivery OrderConditionType = "Delivery"  
)

type OrderPhase string  
  
const (  
   OrderNotAccepted  OrderPhase = "未接單"  
   OrderAccepted     OrderPhase = "已接單"  
   OrderInMaking     OrderPhase = "制作中"  
   OrderMakeComplete OrderPhase = "制作完成"  
   OrderWaiting      OrderPhase = "待配送"  
   OrderDelivery     OrderPhase = "配送中"  
   OrderFinish       OrderPhase = "訂單完成"  
)

Order

主體部分我們不需要修改锅劝，在此處攒驰，僅僅只是加上打印標(biāo)識，方便我們后續(xù)觀察故爵。

//+genclient  
//+kubebuilder:object:root=true  
//+kubebuilder:subresource:status  
//+kubebuilder:printcolumn:name="STATUS",type="string",JSONPath=".status.phase",description="The order status phase"  
//+kubebuilder:printcolumn:name="MESSAGE",type="string",JSONPath=".status.message",description="The order status message"  
  
// Order is the Schema for the orders APItype Order struct {  
   metav1.TypeMeta   `json:",inline"`  
   metav1.ObjectMeta `json:"metadata,omitempty"`  
  
   Spec   OrderSpec   `json:"spec,omitempty"`  
   Status OrderStatus `json:"status,omitempty"`  
}

• kubebuilder:printcolumn 打印列

上述我們就完成了我們代碼CRD的編寫玻粪，接下來，我們就開始我們代碼的控制器的實現(xiàn)诬垂。

控制器實現(xiàn)

./controllers/order_controller.go

const MaxSpeedTime int = 60
func (r *OrderReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {  
   // Fetch the order instance  
   order := &demov1.Order{}  
   err := r.Get(context.TODO(), req.NamespacedName, order)  
   if err != nil {  
      if errors.IsNotFound(err) {  
         return ctrl.Result{}, nil  
      }  
      // Error reading the object - requeue the request.  
      return ctrl.Result{}, err  
   }  
     
   status := order.Status.DeepCopy()  
   defer func() {  
      err := r.updateScaleStatusInternal(order, *status)  
      if err != nil {  
         klog.Errorf("update order(%s/%s) status failed: %s",  
            order.Namespace, order.Name, err.Error())  
         return  
      }  
   }()  
  
   //Simulate the time spent in each phase  
   speedTime := rand.Int() % MaxSpeedTime  
   switch status.Phase {  
   case "":  
      status.Phase = demov1.OrderNotAccepted  
      status.Message = "Order not accepted"  
   case demov1.OrderNotAccepted:  
      status.Phase = demov1.OrderAccepted  
      status.Message = "Order accepted"  
      cond := NewOrderCondition(demov1.ConditionShop, corev1.ConditionFalse, status.Message, status.Message)  
      SetOrderCondition(status, *cond)  
   case demov1.OrderAccepted:  
      status.Phase = demov1.OrderInMaking  
      status.Message = "Order in making"  
   case demov1.OrderInMaking:  
      status.Phase = demov1.OrderMakeComplete  
      status.Message = "Order make complete"  
      cond := NewOrderCondition(demov1.ConditionShop, corev1.ConditionTrue, status.Message, status.Message)  
      SetOrderCondition(status, *cond)  
   case demov1.OrderMakeComplete:  
      status.Phase = demov1.OrderWaiting  
      status.Message = "Order wait delivery"  
      cond := NewOrderCondition(demov1.ConditionDelivery, corev1.ConditionFalse, status.Message, status.Message)  
      SetOrderCondition(status, *cond)  
   case demov1.OrderWaiting:  
      status.Phase = demov1.OrderDelivery  
      status.Message = "Order delivery"  
   case demov1.OrderDelivery:  
      status.Phase = demov1.OrderFinish  
      status.Message = "Order finished,customer has signed"  
   case demov1.OrderFinish:  
      cond := NewOrderCondition(demov1.ConditionDelivery, corev1.ConditionTrue, "Success", status.Message)  
      SetOrderCondition(status, *cond)  
      return ctrl.Result{}, nil  
   }  
  
   return ctrl.Result{RequeueAfter: time.Duration(speedTime) * time.Second}, nil  
}  
  
func (r *OrderReconciler) updateScaleStatusInternal(scale *demov1.Order, newStatus demov1.OrderStatus) error {  
   if reflect.DeepEqual(scale.Status, newStatus) {  
      return nil  
   }  
   clone := scale.DeepCopy()  
   if err := retry.RetryOnConflict(retry.DefaultBackoff, func() error {  
      if err := r.Client.Get(context.TODO(),  
         types.NamespacedName{Name: scale.Name, Namespace: scale.Namespace},  
         clone); err != nil {  
         klog.Errorf("error getting updated scale(%s/%s) from client",  
            scale.Namespace, scale.Name)  
         return err  
      }  
      clone.Status = newStatus  
      if err := r.Client.Status().Update(context.TODO(), clone); err != nil {  
         return err  
      }  
      return nil  
   }); err != nil {  
      return err  
   }  
   oldBy, _ := json.Marshal(scale.Status)  
   newBy, _ := json.Marshal(newStatus)  
   klog.V(5).Infof("order(%s/%s) status from(%s) -> to(%s)", scale.Namespace, scale.Name, string(oldBy), string(newBy))  
   return nil  
}  
  
// SetupWithManager sets up the controller with the Manager.
func (r *OrderReconciler) SetupWithManager(mgr ctrl.Manager) error {  
   //Order status changes do not trigger the reconcile process  
   predicates := builder.WithPredicates(predicate.Funcs{  
      UpdateFunc: func(e event.UpdateEvent) bool {  
         oldObject := e.ObjectOld.(*demov1.Order)  
         newObject := e.ObjectNew.(*demov1.Order)  
         if oldObject.Generation != newObject.Generation || newObject.DeletionTimestamp != nil {  
            klog.V(3).Infof("Observed updated for order: %s/%s", newObject.Namespace, newObject.Name)  
            return true  
         }  
         return false  
      },  
   })  
   return ctrl.NewControllerManagedBy(mgr).  
      For(&demov1.Order{}, predicates).  
      Complete(r)  
}

上面便是控制器的邏輯劲室，其實很多時候也和crud沒太多區(qū)別，我們就在 Reconcile 里實現(xiàn)我們的業(yè)務(wù)邏輯就可以了结窘。

WebHook實現(xiàn)

./api/v1/order_webhook.go

// Default implements webhook.Defaulter so a webhook will be registered for the type
func (in *Order) Default() {  
   // Set the default value.  
   // However, we have noting to do in this crd resources.
}  
  

var _ webhook.Validator = &Order{}  
  
// ValidateCreate implements webhook.Validator so a webhook will be registered for the type
func (in *Order) ValidateCreate() error {  
   orderlog.Info("validate create", "name", in.Name)  
   return in.Spec.validate()  
}  
  
// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type
func (in *Order) ValidateUpdate(old runtime.Object) error {  
   orderlog.Info("validate update", "name", in.Name)  
   return in.Spec.validate()  
}  
  
// ValidateDelete implements webhook.Validator so a webhook will be registered for the type
func (in *Order) ValidateDelete() error {  
   return nil  
}  
  
func (in *OrderSpec) validate() error {  
   if in.TotalPrice <= 0 {  
      return fmt.Errorf("total price must be greater than 0")  
   }  
  
   var totalPrice int64 = 0  
   for i := range in.Commodities {  
      err := in.Commodities[i].validate()  
      if err != nil {  
         return err  
      }  
      totalPrice += in.Commodities[i].Price * in.Commodities[i].Quantity  
   }  
  
   if totalPrice != in.TotalPrice {  
      return fmt.Errorf("the total price of the item is incorrect")  
   }  
   return nil  
}  
  
func (in *CommodityInfo) validate() error {  
   if in.Quantity <= 0 {  
      return fmt.Errorf("commodity %s quantity must be greater than 0", in.Name)  
   }  
   if in.Price <= 0 {  
      return fmt.Errorf("commodity %s price must be greater than 0", in.Name)  
   }  
   return nil  
}

在上面的代碼中很洋，我們只是簡單的校驗的訂單數(shù)據(jù)，具體的開發(fā)中隧枫，我們可以根據(jù)我們的需求去實現(xiàn)喉磁。

項目部署

經(jīng)過上面的步驟，我們項目算是編碼完成了官脓，接下來协怒，我們就開始部署我們的服務(wù)。

主要分為以下幾步：

• 修改 Makefile
• 修改 Dockerfile
• 生成部署文件
• 部署服務(wù)

修改 Makefile

大多數(shù)情況下卑笨，kubebuilder 生產(chǎn)的 makefile 文件已經(jīng)夠我們使用了孕暇，但為了更加方便我們的開發(fā)與部署，我們可以進行以下改造：

# ===remove===  
# Image URL to use all building/pushing image targets  
# IMG ?= controller:latest  

# ===add=== 
# your docker repositories  
REPO ?=  
  
# your project name  
PROJ ?=order  
  
# your project tag or version  
TAG ?=latest

.PHONY: deploy-yaml  
# Generate deploy yaml.  
deploy-yaml: kustomize ## Generate deploy yaml.  
   $(call gen-yamls)

define gen-yamls  
{\  
set -e ;\  
[ -f ${PROJECT_DIR}/_output/yamls/build ] || mkdir -p ${PROJECT_DIR}/_output/yamls/build; \  
rm -rf ${PROJECT_DIR}/_output/yamls/build/${PROJ}; \  
cp -rf ${PROJECT_DIR}/config/* ${PROJECT_DIR}/_output/yamls/build/; \  
cd ${PROJECT_DIR}/_output/yamls/build/manager; \  
${KUSTOMIZE} edit set image controller=${REPO}/${PROJ}:${TAG}; \  
set +x ;\  
echo "==== create deploy.yaml in ${PROJECT_DIR}/_output/yamls/ ====";\  
${KUSTOMIZE} build ${PROJECT_DIR}/_output/yamls/build/default > ${PROJECT_DIR}/_output/yamls/deploy.yaml;\  
}  
endef

# ===modify===
.PHONY: docker-build  
docker-build: ## Build docker image with the manager.  
   docker build --no-cache . -t ${REPO}/${PROJ}:${TAG}  
   docker rmi `docker images | grep none | awk '{print $$3}'` # clean the assets of docker images when build endings  
  
.PHONY: docker-push  
docker-push: ## Push docker image with the manager.  
   docker push ${REPO}/${PROJ}:${TAG}

## Location to install dependencies to  
LOCALBIN ?= $(shell pwd)/bin  
$(LOCALBIN):  
   mkdir -p $(LOCALBIN)  
  
KUSTOMIZE = $(LOCALBIN)/kustomize  
KUSTOMIZE_VERSION ?= v3.8.7  
KUSTOMIZE_INSTALL_SCRIPT ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  
.PHONY: kustomize  
kustomize: $(KUSTOMIZE) ## Download kustomize locally if necessary.  
$(KUSTOMIZE): $(LOCALBIN)  
ifeq ($(wildcard $(KUSTOMIZE)),)  
   curl -s $(KUSTOMIZE_INSTALL_SCRIPT) | bash -s -- $(subst v,,$(KUSTOMIZE_VERSION)) $(LOCALBIN)  
endif

上述文件中：

• 將 docker 鏡像名稱拆分成了3部分：REPO/PROJ/TAG,方便我們構(gòu)建鏡像赤兴。
• 新增了開始生成 deploy.yaml 的代碼妖滔。
• 修改了獲取 kustomize 的方式

修改 Dockerfile

原來的 Dockerfile 中，每次構(gòu)建都需要重新拉去依賴桶良，我們可以使用 go vendor 的形式減少省下拉取依賴的時間座舍。

但是這就要求我們在構(gòu)建代碼前，現(xiàn)將依賴拉取到本地艺普。

# ===remove===
RUN go mod download
# ===add===
COPY vendor/ vendor/

生成部署文件

使用 makefile 可以幫我們快速生成相關(guān)配置文件

生成CRD & RBAC

# 生成 crd 和 rbac yaml 文件
make manifests
# 查看結(jié)構(gòu)
tree
.
├── config
│   ├── crd
│   │   ├── bases
│   │   │   └── demo.sumeng.com_orders.yaml # crd 文件

生成部署文件

修改文件

修改 ./config/crd/kustomization.yaml

resources:  
- bases/demo.sumeng.com_orders.yaml  
#+kubebuilder:scaffold:crdkustomizeresource  
  
patchesStrategicMerge:  
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix.  
# patches here are for enabling the conversion webhook for each CRD  
- patches/webhook_in_orders.yaml  
#+kubebuilder:scaffold:crdkustomizewebhookpatch  
  
# [CERTMANAGER] To enable cert-manager, uncomment all the sections with [CERTMANAGER] prefix.  
# patches here are for enabling the CA injection for each CRD  
- patches/cainjection_in_orders.yaml  
#+kubebuilder:scaffold:crdkustomizecainjectionpatch  
  
# the following config is for teaching kustomize how to do kustomization for CRDs.  
configurations:  
- kustomizeconfig.yaml

修改 ./config/default/kustomization.yaml

# Adds namespace to all resources.  
namespace: crd-demo-system  
  
# Value of this field is prepended to the  
# names of all resources, e.g. a deployment named  
# "wordpress" becomes "alices-wordpress".  
# Note that it should also match with the prefix (text before '-') of the namespace  
# field above.  
namePrefix: crd-demo-  
  
# Labels to add to all resources and selectors.  
#commonLabels:  
#  someName: someValue  
  
bases:  
- ../crd  
- ../rbac  
- ../manager  
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix including the one in  
# crd/kustomization.yaml  
- ../webhook  
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER'. 'WEBHOOK' components are required.  
- ../certmanager  
# [PROMETHEUS] To enable prometheus monitor, uncomment all sections with 'PROMETHEUS'.  
#- ../prometheus  
  
patchesStrategicMerge:  
# Protect the /metrics endpoint by putting it behind auth.  
# If you want your controller-manager to expose the /metrics  
# endpoint w/o any authn/z, please comment the following line.  
- manager_auth_proxy_patch.yaml  
  
# Mount the controller config file for loading manager configurations  
# through a ComponentConfig type  
#- manager_config_patch.yaml  
  
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix including the one in  
# crd/kustomization.yaml  
- manager_webhook_patch.yaml  
  
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER'.  
# Uncomment 'CERTMANAGER' sections in crd/kustomization.yaml to enable the CA injection in the admission webhooks.  
# 'CERTMANAGER' needs to be enabled to use ca injection  
- webhookcainjection_patch.yaml  
  
# the following config is for teaching kustomize how to do var substitution  
vars:  
# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER' prefix.  
- name: CERTIFICATE_NAMESPACE # namespace of the certificate CR  
  objref:  
    kind: Certificate  
    group: cert-manager.io  
    version: v1  
    name: serving-cert # this name should match the one in certificate.yaml  
  fieldref:  
    fieldpath: metadata.namespace  
- name: CERTIFICATE_NAME  
  objref:  
    kind: Certificate  
    group: cert-manager.io  
    version: v1  
    name: serving-cert # this name should match the one in certificate.yaml  
- name: SERVICE_NAMESPACE # namespace of the service  
  objref:  
    kind: Service  
    version: v1  
    name: webhook-service  
  fieldref:  
    fieldpath: metadata.namespace  
- name: SERVICE_NAME  
  objref:  
    kind: Service  
    version: v1  
    name: webhook-service

修改 ./config/default/manager_auth_proxy_patch.yaml

apiVersion: apps/v1  
kind: Deployment  
metadata:  
  name: controller-manager  
  namespace: system  
spec:  
  template:  
    spec:  
      containers:  
      - name: manager  
        args:  
        - "--health-probe-bind-address=:8081"  
        - "--metrics-bind-address=127.0.0.1:8080"  
        - "--leader-elect"

修改 ./config/default/webhookcainjection_patch.yaml

apiVersion: admissionregistration.k8s.io/v1  
kind: ValidatingWebhookConfiguration  
metadata:  
  name: validating-webhook-configuration  
  annotations:  
    cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME)

修改 ./config/webhook/manifests.yaml

---  
apiVersion: admissionregistration.k8s.io/v1  
kind: ValidatingWebhookConfiguration  
metadata:  
  creationTimestamp: null  
  name: validating-webhook-configuration  
webhooks:  
- admissionReviewVersions:  
  - v1  
  clientConfig:  
    service:  
      name: webhook-service  
      namespace: system  
      path: /validate-demo-sumeng-com-v1-order  
  failurePolicy: Fail  
  name: vorder.kb.io  
  rules:  
  - apiGroups:  
    - demo.sumeng.com  
    apiVersions:  
    - v1  
    operations:  
    - CREATE  
    - UPDATE  
    resources:  
    - orders  
  sideEffects: None

生成文件

# 安裝 kustomize
make kustomize
# 生成部署文件
make deploy-yaml REPO=<xxx> TAG=<xxx>
# 生成的部署文件為 ./_output/yamls/deploy.yaml, 我們拿著這個文件去k8s集群部署就可以了

部署服務(wù)

經(jīng)歷了這么多簸州，終于要開始部署服務(wù)了鉴竭。開始之前，我們要先構(gòu)建我們的服務(wù)岸浑。

# 拉取依賴
go mod tidy
go mod vendor
# 生成 deepcopy 文件
make generate
# 制作鏡像
make docker-build REPO=<xxx> TAG=<xxx>
# 上傳鏡像
make docker-push REPO=<xxx> TAG=<xxx>

# 進行制作完成后搏存，為了防止我們的指定的鏡像不對，我們可以重新生成部署文件
make deploy-yaml REPO=<xxx> TAG=<xxx>

# 部署服務(wù)
# 這個是時候矢洲，我們就可以拿著這個文件去我們的集群部署服務(wù)了璧眠。
kubectl apply -f ./_output/yamls/deploy.yaml

# 我們查看pod，會發(fā)現(xiàn)我們的控制器一致起不來读虏，是因為我們添加了webhook服務(wù)责静，需要安裝certmanger
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml

kubectl apply -f ./_output/yamls/deploy.yaml
namespace/crd-demo-system created
customresourcedefinition.apiextensions.k8s.io/orders.demo.sumeng.com created
serviceaccount/crd-demo-controller-manager created
role.rbac.authorization.k8s.io/crd-demo-leader-election-role created
clusterrole.rbac.authorization.k8s.io/crd-demo-manager-role created
clusterrole.rbac.authorization.k8s.io/crd-demo-metrics-reader created
clusterrole.rbac.authorization.k8s.io/crd-demo-proxy-role created
rolebinding.rbac.authorization.k8s.io/crd-demo-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/crd-demo-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/crd-demo-proxy-rolebinding created
configmap/crd-demo-manager-config created
service/crd-demo-controller-manager-metrics-service created
service/crd-demo-webhook-service created
deployment.apps/crd-demo-controller-manager created
certificate.cert-manager.io/crd-demo-serving-cert created
issuer.cert-manager.io/crd-demo-selfsigned-issuer created
validatingwebhookconfiguration.admissionregistration.k8s.io/crd-demo-validating-webhook-configuration created
?  crd-demo k get pod -n crd-demo-system 
NAME                                           READY   STATUS    RESTARTS   AGE
crd-demo-controller-manager-68dc8c65bc-nlkwp   1/1     Running   0          4m5s

經(jīng)過上述步驟，我們的服務(wù)就跑起來了盖桥。

我們可編寫一個資源去驗證一下灾螃。

./config/samples/demo_v1_order.yaml

apiVersion: demo.sumeng.com/v1  
kind: Order  
metadata:  
  name: order-sample  
spec:  
  shop:  
    name: 鮮果市場  
  commodity:  
    - name: 白菜  
      price: 1  
      quantity: 5  
    - name: 黃瓜  
      price: 2  
      quantity: 5  
  totalPrice: 15  
  remark: 送貨上門

觀察她的狀態(tài)變化

 k get orders.demo.sumeng.com order-sample -w
NAME           STATUS   MESSAGE  
order-sample   未接單      Order not accepted
order-sample   已接單      Order accepted
order-sample   制作中      Order in making
order-sample   制作完成     Order make complete
order-sample   待配送      Order wait delivery

這樣我們便完整的進行了一個 CRD 控制器和 webhook 開發(fā)與部署。

附錄

github 項目地址

kububuilder book

cretmanager