k8s裸機(jī)LB支持床三，ingress服務(wù)

metallb簡介

官方網(wǎng)站：https://metallb.universe.tf/

MetalLB是使用標(biāo)準(zhǔn)路由協(xié)議的裸機(jī)Kubernetes集群的軟負(fù)載均衡器，目前處于測試版本階段排宰。

私有云裸金屬架構(gòu)的kubernetes集群不支持LoadBalance

Kubernetes沒有為裸機(jī)群集提供網(wǎng)絡(luò)負(fù)載均衡器（類型為LoadBalancer的服務(wù)）的實(shí)現(xiàn)，如果你的kubernetes集群沒有在公有云的IaaS平臺（GCP那婉，AWS板甘，Azure …）上運(yùn)行，則LoadBalancers將在創(chuàng)建時無限期地保持“掛起”狀態(tài)详炬，也就是說只有公有云廠商自家的kubernetes支持LoadBalancer盐类。

裸機(jī)群集運(yùn)營商留下了兩個較小的工具來將用戶流量帶入其集群，“NodePort”和“externalIPs”服務(wù)呛谜。這兩種選擇都對生產(chǎn)使用產(chǎn)生了重大影響在跳，這使得裸露的金屬集群成為Kubernetes生態(tài)系統(tǒng)中的二等公民。

部署metallb

metallb 目前有兩種部署方式隐岛，支持 Helm 和 YAML 兩種安裝方法


kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml

第一次安裝還需要運(yùn)行以下：


kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"

查看：


# kubectl get pod -n metallb-system   

NAME                          READY  STATUS    RESTARTS  AGE

controller-684f5d9b49-x859t  1/1    Running  0          6h3m

speaker-lmrgx                1/1    Running  0          6h3m

speaker-x9jk5                1/1    Running  0          6h3m

創(chuàng)建ConfigMap

下載 config.yaml 模版


wget https://raw.githubusercontent.com/google/metallb/v0.9.3/manifests/example-layer2-config.yaml

這里直接生成


cat >> config.yaml << EOF

apiVersion: v1

kind: ConfigMap

metadata:

  namespace: metallb-system

  name: config

data:

  config: |

    address-pools:

    - name: my-ip-space

      protocol: layer2

      addresses:

      - 192.168.200.130-192.168.200.140

EOF

創(chuàng)建：


# kubectl apply -f config.yaml

查看 configMap 情況


# kubectl describe ConfigMap config -n metallb-system

Name:        config

Namespace:    metallb-system

Labels:      <none>

Annotations: 

Data

====

config:

----

address-pools:

- name: my-ip-space

  protocol: layer2

  addresses:

  - 192.168.200.130-192.168.200.140

Events:  <none>

至此猫妙，LoadBalancer的配置工作就完成

創(chuàng)建service測試 LoadBalancer

這里創(chuàng)建一個 nginx 服務(wù)，包含deployment和一個LoadBalancer類型的service


# vim nginx-test.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  name: nginx

spec:

  selector:

    matchLabels:

      app: nginx

  template:

    metadata:

      labels:

        app: nginx

    spec:

      containers:

      - name: nginx

        image: nginx:1

        ports:

        - name: http

          containerPort: 80

---

apiVersion: v1

kind: Service

metadata:

  name: nginx

spec:

  ports:

  - name: http

    port: 80

    protocol: TCP

    targetPort: 80

  selector:

    app: nginx

  type: LoadBalancer

查看service分配的EXTERNAL-IP：


# kubectl get svc

NAME        TYPE          CLUSTER-IP  EXTERNAL-IP      PORT(S)        AGE

kubernetes  ClusterIP      10.0.0.1    <none>            443/TCP        4d5h

nginx        LoadBalancer  10.0.0.86    192.168.200.130  80:32295/TCP  6h1m

集群外的機(jī)器訪問EXTERNAL-IP 192.168.200.130 成功即可

部署 ingress-nginx 控制器

1聚凹、安裝

官方文檔 https://kubernetes.github.io/ingress-nginx/deploy/

下載


# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml

# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml

# kubctl apply -f mandatory.yaml

創(chuàng)建 NodePort 模式的 service


# kubctl apply -f service-nodeport.yaml

集群有LB的可以創(chuàng)建 LoadBalarcer 模式 service


# cp service-nodeport.yaml service-lb.yaml

# vim service-lb.yaml

spec:

  #type: NodePort

  type: LoadBalancer

...

# kubctl apply -f service-lb.yaml

檢查是否安裝：


# kubectl get pods -n ingress-nginx

NAME                                        READY  STATUS      RESTARTS  AGE

ingress-nginx-controller-69fb496d7d-t9n6g  1/1    Running    0          18m

2割坠、部署 Ingress

這里創(chuàng)建一個nginx deployment 和 ingress


# vim ingress-test.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  name: nginx

spec:

  selector:

    matchLabels:

      app: nginx

  template:

    metadata:

      labels:

        app: nginx

    spec:

      containers:

      - name: nginx

        image: nginx:1

        ports:

        - name: http

          containerPort: 80

---

apiVersion: extensions/v1beta1

kind: Ingress

metadata:

  name: test-ingress

spec:

  rules:

  - host: k8s.alwooo.cn

    http:

      paths:

      - path: /test

        backend:

          serviceName: nginx

          servicePort: 80

創(chuàng)建：


# kubectl apply -f ingress-test.yaml

Error from server (InternalError): error when creating "ingress-test.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post https://ingress-nginx-controller-admission.ingress-nginx.svc:443/extensions/v1beta1/ingresses?timeout=30s: dial tcp 10.0.0.171:443: connect: connection timed out

這里發(fā)現(xiàn)報(bào)錯了

錯誤解決：

查看 node 節(jié)點(diǎn)的 kube-proxy


# systemctl status kube-proxy

6月 09 10:17:57 kub-node2 kube-proxy[28434]: E0609 10:17:57.325032  28434 node.go:125] Failed to retrieve node info: nodes "kube-node2" not found

發(fā)現(xiàn)其中一臺節(jié)點(diǎn) kube-proxy 報(bào)錯齐帚，是配置文件 hostnameOverride 錯誤導(dǎo)致，修改即可


# vim /opt/kubernetes/cfg/kube-porxy-config.yaml

...

hostnameOverride: kub-node2  # 修改為正確的主機(jī)名

...

# systemctl restart kube-proxy

重新部署 ingress 還是報(bào)錯彼哼。

判斷是 master 節(jié)點(diǎn)無法訪問 10.0.0.171:443

查看ingress 服務(wù)


# kubectl -n ingress-nginx get svc

NAME                                TYPE        CLUSTER-IP  EXTERNAL-IP  PORT(S)                      AGE

ingress-nginx-controller            NodePort    10.0.0.196  <none>        80:31217/TCP,443:30348/TCP  12s

ingress-nginx-controller-admission  ClusterIP  10.0.0.171  <none>        443/TCP                      12s

svervice 為 NodePort 方式对妄，修改為 LoadBalancer


# vim deploy.yaml

...

apiVersion: v1

kind: Service

metadata:

  labels:

    helm.sh/chart: ingress-nginx-2.0.3

    app.kubernetes.io/name: ingress-nginx

    app.kubernetes.io/instance: ingress-nginx

    app.kubernetes.io/version: 0.32.0

    app.kubernetes.io/managed-by: Helm

    app.kubernetes.io/component: controller

  name: ingress-nginx-controller-admission

  namespace: ingress-nginx

spec:

  type:  LoadBalancer    # 修改此字段

  ports:

    - name: https-webhook

      port: 443

      targetPort: webhook

  selector:

    app.kubernetes.io/name: ingress-nginx

    app.kubernetes.io/instance: ingress-nginx

    app.kubernetes.io/component: controller

---

...

重新部署后，還是一樣沪羔。

經(jīng)過多番查閱后發(fā)現(xiàn)饥伊，是master節(jié)點(diǎn)沒有部署 kubelet, kube-proxy 導(dǎo)致。必須把master節(jié)點(diǎn)當(dāng)成worker節(jié)點(diǎn)蔫饰。

重新部署kubelet 和 kube-proxy 到master節(jié)點(diǎn)后琅豆。部署ingerss服務(wù)成功


# kubectl apply -f ingress-test.yaml

deployment.apps/nginx created

ingress.extensions/test-ingress created

# kubectl get ingress

NAME          CLASS    HOSTS          ADDRESS          PORTS  AGE

test-ingress  <none>  k8s.alwooo.cn  192.168.200.120  80      41s

如果 service 為 NodePort 模式，則客戶端域名指向的地址應(yīng)為 ADDRESS
如使用是 LoadBalancer 則使用 LB 地址作為 VIP篓吁，映射到 ASSRESS


# kubectl get svc -n ingress-nginx

NAME                                TYPE          CLUSTER-IP  EXTERNAL-IP      PORT(S)                      AGE

ingress-nginx-controller            LoadBalancer  10.0.0.207  192.168.200.131  80:30859/TCP,443:30756/TCP  57m

ingress-nginx-controller-admission  ClusterIP      10.0.0.242  <none>            443/TCP                      57m

其他坑 https://www.cnblogs.com/tchua/p/10844013.html

配置 ingress 處理 TLS 傳輸

創(chuàng)建私鑰和證書


# openssl genrsa -out tls.key 2048

# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=GuangDong/L=ZhongShan/O=Anlewo/CN=tomcat.alwooo.cn

創(chuàng)建 k8s Secret


kubectl create secret tls ingress-secret --cert=tls.crt --key=tls.key

添加部署文件


# cat tomcat-ingress-tls.yaml

apiVersion: extensions/v1beta1

kind: Ingress

metadata:

  name: ingress-tomcat-tls

  namespace: default

  annotations:

    kubernetes.io/ingress.class: "nginx"

spec:

  tls:

  - hosts:

    - tomcat.alwooo.cn

    secretName: tomcat-ingress-secret

  rules:

  - host: tomcat.alwooo.cn

    http:

      paths:

      - path:

        backend:

          serviceName: tomcat

          servicePort: 8080

部署完后就可以用 https 訪問了

k8s裸機(jī)LB支持，ingress服務(wù)