metrics-server 是用來取代heapster狼速,負責從kubelet中采集數(shù)據(jù), 并通過Metrics API在Kubernetes Apiserver中暴露它們酝蜒。
metrics-server 采集node 和pod 的cpu/mem誊辉,數(shù)據(jù)存在容器本地,不做持久化亡脑。這些數(shù)據(jù)的使用場景有 kubectl top 和scheduler 調度堕澄、hpa 彈性伸縮,以及原生的dashboard 監(jiān)控數(shù)據(jù)展示霉咨。
metrics-server 和prometheus 沒有半毛錢關系蛙紫。 也沒有任何數(shù)據(jù)或者接口互相依賴關系。
prometheus 能力更強途戒,也更重惊来,擁有更多的監(jiān)控指標以及自定義監(jiān)控指標,可以配合grafana 面板更好的展示數(shù)據(jù)棺滞,配合alertmanager 實現(xiàn)告警裁蚁。介紹見之前內容
https://github.com/cai11745/k8s-ocp-yaml/blob/master/prometheus/2019-10-22-prometheus-1-install-and-metricsIngress.md
metrics-server 指標少矢渊,但是更輕量,適用于簡單場景的容器與節(jié)點數(shù)據(jù)監(jiān)控枉证。
安裝 metrics-server
官網(wǎng) https://github.com/kubernetes-sigs/metrics-server
修改下鏡像地址矮男,默認的k8s.grc.io 國內拉取不到,換成阿里云
registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6
[root@master ~]# wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml
[root@master ~]# vim components.yaml
# 修改image
# registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6
[root@master ~]# kubectl apply -f components.yaml
使用kubectl top 查看pod 和node 的資源使用情況室谚,獲取不到數(shù)據(jù)
[root@master ~]# kubectl top node
error: metrics not available yet
[root@master ~]# kubectl top pod
W0414 13:34:35.193688 13720 top_pod.go:266] Metrics not available for pod default/centos-758b7556f5-542wl, age: 299h19m20.193680607s
error: Metrics not available for pod default/centos-758b7556f5-542wl, age: 299h19m20.193680607s
查看metrics-server pod 日志
[root@master ~]# kubectl -n kube-system logs metrics-server-58c885686f-nlp25
...
E0414 05:34:32.752194 1 reststorage.go:135] unable to fetch node metrics for node "node1": no metrics known for node
E0414 05:34:32.752208 1 reststorage.go:135] unable to fetch node metrics for node "master": no metrics known for node
E0414 05:34:35.186237 1 reststorage.go:160] unable to fetch pod metrics for pod default/tomtest-86f7667d85-hxnzl: no metrics known for pod
E0414 05:34:35.186247 1 reststorage.go:160] unable to fetch pod metrics for pod default/centos-758b7556f5-542wl: no metrics known for pod
提示 無法解析節(jié)點的主機名毡鉴,是metrics-server 這個容器不能通過CoreDNS 解析各Node的主機名,metrics-server 連節(jié)點時默認是連接節(jié)點的主機名秒赤,需要加個參數(shù)猪瞬,讓它連接節(jié)點的IP,而不是使用主機名:
- --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP
修改yaml 文件入篮,增加這兩行
image: k8s.gcr.io/metrics-server-amd64:v0.3.6
imagePullPolicy: IfNotPresent
args:
- --cert-dir=/tmp
- --secure-port=4443
- --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP
導入文件陈瘦,過一會就正常了
[root@master ~]# kubectl apply -f components.yaml
[root@master ~]# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
master 101m 5% 971Mi 17%
node1 72m 1% 914Mi 3%
[root@master ~]# kubectl top pod --all-namespaces
NAMESPACE NAME CPU(cores) MEMORY(bytes)
ingress-nginx nginx-ingress-controller-c8848f54b-z2fwd 4m 181Mi
kube-system calico-kube-controllers-77c4b7448-n92dt 1m 14Mi
kube-system calico-node-nqltv 12m 53Mi
kube-system calico-node-rf9gh 10m 49Mi
kube-system coredns-6955765f44-579bp 2m 13Mi
kube-system coredns-6955765f44-7h4vh 1m 13Mi
kube-system etcd-master 12m 108Mi
kube-system kube-apiserver-master 19m 336Mi
kube-system kube-controller-manager-master 5m 42Mi
kube-system kube-proxy-24k9w 1m 22Mi
kube-system kube-proxy-w48qf 1m 17Mi
kube-system kube-scheduler-master 2m 18Mi
kube-system metrics-server-6ffdb54684-lg77c 1m 14Mi
接口測試
metrics-server 將node 和pod 的監(jiān)控數(shù)據(jù)通過k8s 標準api 暴露出來。
All endpoints are GET endpoints, rooted at /apis/metrics/v1alpha1/. There won't be support for the other REST methods.
The list of supported endpoints:
/nodes - all node metrics; type []NodeMetrics
/nodes/{node} - metrics for a specified node; type NodeMetrics
/namespaces/{namespace}/pods - all pod metrics within namespace with support for all-namespaces; type []PodMetrics
/namespaces/{namespace}/pods/{pod} - metrics for a specified pod; type PodMetrics
The following query parameters are supported:
labelSelector - restrict the list of returned objects by labels (list endpoints only)
通過kubectl proxy 命令暴露api 端口潮售,默認8080痊项,非加密端口。如果端口被用了就加上-p 參數(shù)指定一個端口
kubectl proxy -p 8002
# 再開一個終端酥诽,測試訪問
[root@master ~]# kubectl api-resources |grep metrics
nodes metrics.k8s.io false NodeMetrics
pods metrics.k8s.io true PodMetrics
[root@master ~]# curl 127.0.0.1:8002/ |grep metrics
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4963 0 4963 0 0 4822k 0 --:--:-- --:--:-- --:--:-- 4846k
"/apis/metrics.k8s.io",
"/apis/metrics.k8s.io/v1beta1",
"/metrics",
# 獲取所有node
[root@master ~]# curl 127.0.0.1:8002/apis/metrics.k8s.io/v1beta1/nodes
# 某個pod
[root@master ~]# curl 127.0.0.1:8002/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-apiserver-master
{
"kind": "PodMetrics",
"apiVersion": "metrics.k8s.io/v1beta1",
"metadata": {
"name": "kube-apiserver-master",
"namespace": "kube-system",
"selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-apiserver-master",
"creationTimestamp": "2020-04-14T15:25:04Z"
},
"timestamp": "2020-04-14T15:24:10Z",
"window": "30s",
"containers": [
{
"name": "kube-apiserver",
"usage": {
"cpu": "19383740n",
"memory": "344704Ki"
}
}
]