1. 前言

轉(zhuǎn)載請說明原文出處, 尊重他人勞動成果!

源碼位置: https://github.com/nicktming/kubernetes/tree/tming-v1.13/staging/src/k8s.io/code-generator
分支: tming-v1.13 (基于v1.13版本)

https://github.com/nicktming/k8s-crd-controller

由于crd是需要生成代碼的, 當(dāng)然如果自己把那段代碼自己寫上也不會有任何問題, 只是這些代碼重復(fù)性很高, 所以規(guī)律性就很強(qiáng), 只需要把某些字段改成自己的字段即可. 所以本文將分析如何使用生成代碼并分析其原理.

2. 例子

2.1 準(zhǔn)備crd

參考 Kubernetes Deep Dive: Code Generation for CustomResources 以及 [極客時間深入剖析Kubernetes] (推薦學(xué)習(xí))

文件內(nèi)容以及結(jié)構(gòu)如下:

crd.png

doc.go 和 types.go

===> pkg/apis/example.com/v1/doc.go
// +k8s:deepcopy-gen=package,register
// +groupName=nicktming.example.com
package v1

===> pkg/apis/example.com/v1/types.go
package v1
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
// +genclient
// +genclient:noStatus
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// Database describes a database.
type Database struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`
    Spec DatabaseSpec `json:"spec"`
}
// DatabaseSpec is the spec for a Foo resource
type DatabaseSpec struct {
    User     string `json:"user"`
    Password string `json:"password"`
    Encoding string `json:"encoding,omitempty"`
}
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// DatabaseList is a list of Database resources
type DatabaseList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata"`
    Items []Database `json:"items"`
}

Global Tags

1. // +k8s:deepcopy-gen=package,register 表示為該package下面所有的Type創(chuàng)建DeepCopy方法. (關(guān)于什么是Type后面源碼分析部分會解釋)

It tells deepcopy-gen to create deepcopy methods by default for every 
type in that package. 

If you have types where deepcopy is not necessary or not desired, you 
can opt-out for such a type with a local tag // +k8s:deepcopy-gen=false. 

If you do not enable package-wide deepcopy, you have to opt-in 
to deepcopy for each desired type via // +k8s:deepcopy-gen=true

2. // +groupName=nicktming.example.com 表示group的名字是nicktming.example.com, 后面在注冊schema的時候會用到.

Local Tags

3. // +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object 表示該Type需要實(shí)現(xiàn)k8s.io/apimachinery/pkg/runtime.Object這個借口的兩個方法GetObjectKind和DeepCopyObject. (因?yàn)樵谡{(diào)用clientset的時候會有decode, 這個時候會用到)

type Object interface {
    GetObjectKind() schema.ObjectKind
    DeepCopyObject() Object
}

Client-gen Tags

4.
// +genclient 表示為該Type創(chuàng)建client. (clientset會為Type創(chuàng)建client)
// +genclient:noStatus

this type is not using spec-status separation via the /status subresource. 

The resulting client will not have the UpdateStatus method 
(client-gen would generate that blindly otherwise as soon as
 it finds a Status field in your struct)

register.go

package v1

import (
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/runtime"
    "k8s.io/apimachinery/pkg/runtime/schema"

)

// SchemeGroupVersion is group version used to register these objects
var SchemeGroupVersion = schema.GroupVersion{Group: "nicktming.example.com", Version: "v1"}

// Kind takes an unqualified kind and returns back a Group qualified GroupKind
func Kind(kind string) schema.GroupKind {
    return SchemeGroupVersion.WithKind(kind).GroupKind()
}

// Resource takes an unqualified resource and returns a Group qualified GroupResource
func Resource(resource string) schema.GroupResource {
    return SchemeGroupVersion.WithResource(resource).GroupResource()
}

var (
    // SchemeBuilder initializes a scheme builder
    SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)
    // AddToScheme is a global function that registers this API group & version to a scheme
    AddToScheme = SchemeBuilder.AddToScheme
)

// Adds the list of known types to Scheme.
func addKnownTypes(scheme *runtime.Scheme) error {
    scheme.AddKnownTypes(SchemeGroupVersion,
        &Database{},
        &DatabaseList{},
    )
    metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
    return nil
}

該文件主要為了向schema注冊apiGroupVersion用的, 在生成客戶端中decode階段會用到.
說白了就是在解析的時候GroupVersion=nicktming.example.com/v1并且Kind=Database的時候可以知道將結(jié)果解析成Database結(jié)構(gòu), 所以需要提前注冊信息.

2.2 運(yùn)行

執(zhí)行參數(shù)如下:
./generate-groups.sh all github.com/nicktming/k8s-crd-controller/pkg/client github.com/nicktming/k8s-crd-controller/pkg/apis example.com:v1
all: 代表deepcopy-gen, client-gen, 和 lister-gen和informer-gen都要執(zhí)行. 如果只想執(zhí)行deepcopy-gen, 把all換成deepcopy-gen即可.

[root@master k8s-crd-controller]# pwd
/root/go/src/github.com/nicktming/k8s-crd-controller
[root@master k8s-crd-controller]# tree
.
├── pkg
│   └── apis
│       └── example.com
│           └── v1
│               ├── doc.go
│               └── types.go
└── README.md
[root@master code-generator]# pwd
/root/go/src/k8s.io/kubernetes/staging/src/k8s.io/code-generator
[root@master code-generator]# ./generate-groups.sh all github.com/nicktming/k8s-crd-controller/pkg/client github.com/nicktming/k8s-crd-controller/pkg/apis example.com:v1
Generating deepcopy funcs
Generating clientset for example.com:v1 at github.com/nicktming/k8s-crd-controller/pkg/client/clientset
Generating listers for example.com:v1 at github.com/nicktming/k8s-crd-controller/pkg/client/listers
Generating informers for example.com:v1 at github.com/nicktming/k8s-crd-controller/pkg/client/informers

[root@master k8s-crd-controller]# pwd
/root/go/src/github.com/nicktming/k8s-crd-controller
 [root@master k8s-crd-controller]# tree
.
├── pkg
│   ├── apis
│   │   └── example.com
│   │       └── v1
│   │           ├── doc.go
│   │           ├── register.go
│   │           ├── types.go
│   │           └── zz_generated.deepcopy.go
│   └── client
│       ├── clientset
│       │   └── versioned
│       │       ├── clientset.go
│       │       ├── doc.go
│       │       ├── fake
│       │       │   ├── clientset_generated.go
│       │       │   ├── doc.go
│       │       │   └── register.go
│       │       ├── scheme
│       │       │   ├── doc.go
│       │       │   └── register.go
│       │       └── typed
│       │           └── example.com
│       │               └── v1
│       │                   ├── database.go
│       │                   ├── doc.go
│       │                   ├── example.com_client.go
│       │                   ├── fake
│       │                   │   ├── doc.go
│       │                   │   ├── fake_database.go
│       │                   │   └── fake_example.com_client.go
│       │                   └── generated_expansion.go
│       ├── informers
│       │   └── externalversions
│       │       ├── example.com
│       │       │   ├── interface.go
│       │       │   └── v1
│       │       │       ├── database.go
│       │       │       └── interface.go
│       │       ├── factory.go
│       │       ├── generic.go
│       │       └── internalinterfaces
│       │           └── factory_interfaces.go
│       └── listers
│           └── example.com
│               └── v1
│                   ├── database.go
│                   └── expansion_generated.go
└── README.md

21 directories, 27 files

deepcopy-gen: 生成了github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1/zz_generated.deepcopy.go文件.
client-gen: 生成了nicktming/k8s-crd-controller/pkg/client/clientset文件夾及其下屬所有文件. (主要與api-server打交道代碼)
lister-gen: 生成了nicktming/k8s-crd-controller/pkg/client/listers文件夾及其下屬所有文件. (informer體系相關(guān)代碼)
informers: 生成了nicktming/k8s-crd-controller/pkg/client/informers文件夾及其下屬所有文件.(從本地緩存中list該元素)

client文件下生成的這些文件在 [k8s源碼分析][client-go] informer之SharedInformerFactory 和 [k8s源碼分析][client-go] client之clientset 中已經(jīng)分析過了.

關(guān)于zz_generated.deepcopy.go文件中的內(nèi)容主要是生成對象拷貝方法.

3. 創(chuàng)建crd

crd.yaml

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: databases.nicktming.example.com
spec:
  group: nicktming.example.com
  version: v1
  names:
    kind: Database
    plural: databases
  scope: Namespaced

my-database.yaml

apiVersion: nicktming.example.com/v1
kind: Database
metadata:
  name: my-database
spec:
  username: "nicktming"
  password: "123456"
  encoding: "json"

運(yùn)行結(jié)果如下:

[root@master kubectl]# ./kubectl apply -f crd/crd.yaml 
customresourcedefinition.apiextensions.k8s.io/databases.nicktming.example.com created
[root@master kubectl]# ./kubectl get crd 
NAME                              CREATED AT
databases.nicktming.example.com   2019-10-29T13:30:04Z
[root@master kubectl]# ./kubectl apply -f crd/my-database.yaml 
database.nicktming.example.com/my-database created
[root@master kubectl]# ./kubectl get database
NAME          AGE
my-database   58s
[root@master kubectl]# ./kubectl describe database my-database
Name:         my-database
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"nicktming.example.com/v1","kind":"Database","metadata":{"annotations":{},"name":"my-database","namespace":"default"},"spec"...
API Version:  nicktming.example.com/v1
Kind:         Database
Metadata:
  Creation Timestamp:  2019-10-29T13:30:51Z
  Generation:          1
  Resource Version:    100743
  Self Link:           /apis/nicktming.example.com/v1/namespaces/default/databases/my-database
  UID:                 53787ca0-fa50-11e9-8739-525400d54f7e
Spec:
  Encoding:  json
  Password:  123456
  Username:  nicktming
Events:      <none>

通過selflink訪問 curl http://localhost:8080/apis/nicktming.example.com/v1/namespaces/default/databases/my-database

browser.png

可以看到看到可以通過api-server訪問所創(chuàng)建的資源. 那么接下來看一下如何使用代碼進(jìn)行訪問.

3.1 代碼訪問

前面生成的代碼就是為了用代碼可以訪問, 之所有可以用代碼訪問pod, service這些kubernetes的資源, 這是因?yàn)?code>client-go已經(jīng)實(shí)現(xiàn)了這部分代碼, 然而對于自定義的資源, kubernetes無法提前寫好, 但是由于基本結(jié)構(gòu)差不多, 所以用代碼直接生成了.

package main

import (
    "fmt"
    nicktmingv1 "github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1"
    "k8s.io/apimachinery/pkg/labels"
    "k8s.io/client-go/rest"
    nicktmingclientset "github.com/nicktming/k8s-crd-controller/pkg/client/clientset/versioned"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    nicktminginformers "github.com/nicktming/k8s-crd-controller/pkg/client/informers/externalversions"
    "k8s.io/client-go/tools/cache"
)

func main()  {
    config := &rest.Config{
        Host: "http://172.21.0.16:8080",
    }
    // 生成client
    databaseClient, _ := nicktmingclientset.NewForConfig(config)

    // 從api-server中獲取
    myDatabase, _ := databaseClient.NicktmingV1().Databases("default").Get("my-database", metav1.GetOptions{})
    fmt.Printf("===>Database Name:%v(%v,%v,%v)\n", myDatabase.Name, myDatabase.Spec.User, myDatabase.Spec.Password, myDatabase.Spec.Encoding)


    factory := nicktminginformers.NewSharedInformerFactory(databaseClient, 10)

    // 添加event handler
    databaseInformer := factory.Nicktming().V1().Databases()
    databaseInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
        AddFunc:    func(obj interface{}) {fmt.Printf("add: %v\n", obj.(*nicktmingv1.Database).Name)},
        UpdateFunc: func(oldObj, newObj interface{}) {fmt.Printf("update: %v\n", newObj.(*nicktmingv1.Database).Name)},
        DeleteFunc: func(obj interface{}){fmt.Printf("delete: %v\n", obj.(*nicktmingv1.Database).Name)},
    })

    // 啟動
    stopCh := make(chan struct{})
    factory.Start(stopCh)
    factory.WaitForCacheSync(stopCh)


    // 從本地緩存中獲取元素
    databaseLister := databaseInformer.Lister()

    allDatabases, _ := databaseLister.List(labels.Everything())
    for _, p := range allDatabases {
        fmt.Printf("list database: %v\n", p.Name)
    }
    <- stopCh

}

如果報錯的話把config.NegotiatedSerializer = serializer.DirectCodecFactory{CodecFactory: scheme.Codecs} 改成 config.NegotiatedSerializer = scheme.Codecs, 因?yàn)榘姹締栴}, apimachinery中的版本不存在DirectCodecFactory.