Kubernetes 默認 HPA 功能可實現(xiàn)基于Metric CPU 與 MEM 的使用率來進行動態(tài)擴縮容败去，在實現(xiàn)工作中針對如果已知的突峰流量的場景（如指定時間段的業(yè)務推廣等）可能存在擴容置后不及時的問題，為了實現(xiàn)此類場景可以通過類似 Crontab 定時機制來動態(tài)擴縮容爷绘。本次實踐是使用了一個開源的 FinOps 項目Crane ，項目在 k8s 默認的 HPA基礎上封裝了更智能的 ehpa、 Analytics政鼠、tsp 等功能逮栅，如果有興趣可以進一步了解。今天主要實踐了基于 cron 方式的ehpa 功能胸私。
一、 安裝
k8s 集群內(nèi)安裝 Crane (推薦采用自定義安裝方式)
- clone 最新代碼并切到最新分支內(nèi)容
git clone https://github.com/gocrane/crane.git
cd crane
CRANE_LATEST_VERSION=$(curl -s https://api.github.com/repos/gocrane/crane/releases/latest | grep -oP '"tag_name": "\K(.*)(?=")')
git checkout $CRANE_LATEST_VERSION
- 應用 CRD 資源對象的創(chuàng)建
kubectl apply -f deploy/manifests
- 修改 craned 連接的 prometheus server 的地址配置
vi deploy/craned/deployment.yaml
command:
- /craned
- --prometheus-address=http://<ipaddr>:<port>
- 應用核心服務部署
kubectl apply -f deploy/craned
kubectl apply -f deploy/metric-adapter
二鳖谈、 配置
定義定時擴縮調(diào)度策略配置
#配置實例岁疼，創(chuàng)建 cron-scale.yaml
apiVersion: autoscaling.crane.io/v1alpha1
kind: EffectiveHorizontalPodAutoscaler
metadata:
name: ehpa-cron-test
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: nginx #指定scale deployment對象名稱
minReplicas: 1 #最大復本數(shù)
maxReplicas: 5 #最小復本數(shù)
scaleStrategy: Auto
crons:
- name: "biz-push-cron-1"
timezone: "Asia/Shanghai" #時區(qū)定義
description: "scale up" #擴容至3個POD復本
start: "36 17 ? * *" ##17:36~17:37 執(zhí)行scale up復本數(shù)為3
end: "37 17 ? * *"
targetReplicas: 3
- name: "biz-push-cron-2" #縮容至1個POD復本
timezone: "Asia/Shanghai"
description: "scale down"
start: "37 17 ? * *"
end: "45 17 ? * *" ##scale down復本數(shù)為1，注意start-end須>5m
targetReplicas: 1
# 應用策略
kubectl apply -f cron-scale.yaml
三缆娃、測試驗證
- 查看應用策略狀態(tài)
[crane]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
ehpa-ehpa-cron-test Deployment/nginx 1/1 (avg) 1 5 1 3m
[root@itg-k8s-master01 crane]# kubectl get ehpa
NAME STRATEGY MINPODS MAXPODS SPECIFICPODS REPLICAS AGE
ehpa-cron-test Auto 1 5 1 3m3s
- 時間到達觸發(fā)后狀態(tài)查看捷绒，擴容到達三個
[crane]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
ehpa-ehpa-cron-test Deployment/nginx 1/1 (avg) 1 5 3 5m29s
[crane]# kubectl get ehpa
NAME STRATEGY MINPODS MAXPODS SPECIFICPODS REPLICAS AGE
ehpa-cron-test Auto 1 5 3 5m58s
- 查看 pod craned 日志
I0624 1 effective_hpa_controller.go:40] Got ehpa default/ehpa-cron-test
I0624 1 effective_hpa_controller.go:139] Update EffectiveHorizontalPodAutoscaler status successful, ehpa default/ehpa-cron-test
- 時間到達觸發(fā)后查看 ehpa 實例的狀態(tài)信息
# 第一次cron調(diào)度(biz-push-cron-1)
# kubectl get ehpa ehpa-cron-test -oyaml
Status:
Conditions:
Last Transition Time: 2022-06-29T09:36:07Z
Message: Effective HPA is ready
Reason: EffectiveHorizontalPodAutoscalerReady
Status: True
Type: Ready
Last Transition Time: 2022-06-29T09:36:07Z
Message: the HPA controller was able to update the target scale to 3
Reason: SucceededRescale
Status: True
Type: AbleToScale
Last Transition Time: 2022-06-29T09:36:07Z
Message: the HPA was able to successfully calculate a replica count from external metric ehpa-cron-test(&LabelSelector{MatchLabels:map[string]string{autoscaling.crane.io/effective-hpa-uid: f2525179-a3bf-44b7-babe-11a71c73a252,},MatchExpressions:[]LabelSelectorRequirement{},})
Reason: ValidMetricFound
Status: True
Type: ScalingActive
Last Transition Time: 2022-06-29T09:36:07Z
Message: the desired count is within the acceptable range
Reason: DesiredWithinRange
Status: False
Type: ScalingLimited
Current Replicas: 1
Expect Replicas: 3
---------------
Status:
Conditions:
Last Transition Time: 2022-06-29T09:37:07Z
Message: Effective HPA is ready
Reason: EffectiveHorizontalPodAutoscalerReady
Status: True
Type: Ready
Last Transition Time: 2022-06-29T09:37:07Z
Message: recent recommendations were higher than current one, applying the highest recent recommendation
Reason: ScaleDownStabilized
Status: True
Type: AbleToScale
Last Transition Time: 2022-06-29T09:37:07Z
Message: the HPA was able to successfully calculate a replica count from external metric ehpa-cron-test(&LabelSelector{MatchLabels:map[string]string{autoscaling.crane.io/effective-hpa-uid: f2525179-a3bf-44b7-babe-11a71c73a252,},MatchExpressions:[]LabelSelectorRequirement{},})
Reason: ValidMetricFound
Status: True
Type: ScalingActive
Last Transition Time: 2022-06-29T09:37:07Z
Message: the desired count is within the acceptable range
Reason: DesiredWithinRange
Status: False
Type: ScalingLimited
Current Replicas: 3
Expect Replicas: 3
# (上次結(jié)束時間+5分鐘)第二次 cron 調(diào)度 (biz-push-cron-2)
Status:
Conditions:
Last Transition Time: 2022-06-29T09:41:53Z
Message: Effective HPA is ready
Reason: EffectiveHorizontalPodAutoscalerReady
Status: True
Type: Ready
Last Transition Time: 2022-06-29T09:41:53Z
Message: the HPA controller was able to update the target scale to 1
Reason: SucceededRescale
Status: True
Type: AbleToScale
Last Transition Time: 2022-06-29T09:41:53Z
Message: the HPA was able to successfully calculate a replica count from external metric ehpa-cron-test(&LabelSelector{MatchLabels:map[string]string{autoscaling.crane.io/effective-hpa-uid: f2525179-a3bf-44b7-babe-11a71c73a252,},MatchExpressions:[]LabelSelectorRequirement{},})
Reason: ValidMetricFound
Status: True
Type: ScalingActive
Last Transition Time: 2022-06-29T09:41:53Z
Message: the desired count is within the acceptable range
Reason: DesiredWithinRange
Status: False
Type: ScalingLimited
Current Replicas: 3
Expect Replicas: 1 #觸發(fā)期望復本數(shù)
---------------
Status:
Conditions:
Last Transition Time: 2022-06-29T09:42:08Z
Message: Effective HPA is ready
Reason: EffectiveHorizontalPodAutoscalerReady
Status: True
Type: Ready
Last Transition Time: 2022-06-29T09:42:08Z
Message: recommended size matches current size
Reason: ReadyForNewScale
Status: True
Type: AbleToScale
Last Transition Time: 2022-06-29T09:42:08Z
Message: the HPA was able to successfully calculate a replica count from external metric ehpa-cron-test(&LabelSelector{MatchLabels:map[string]string{autoscaling.crane.io/effective-hpa-uid: f2525179-a3bf-44b7-babe-11a71c73a252,},MatchExpressions:[]LabelSelectorRequirement{},})
Reason: ValidMetricFound
Status: True
Type: ScalingActive
Last Transition Time: 2022-06-29T09:42:08Z
Message: the desired count is within the acceptable range
Reason: DesiredWithinRange
Status: False
Type: ScalingLimited
Current Replicas: 1 # HPA 達成所期望復本數(shù)
Expect Replicas: 1
～～～ Finish ～～～
