grafana啟用插件：grafana-kubernetes-app
grafana有一個(gè)專門針對(duì)Kubernetes集群監(jiān)控的插件：grafana-kubernetes-app
安裝這個(gè)插件
- 可以在部署grafana的時(shí)候邀层，直接把插件裝上：
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: monitoring-grafana
namespace: janny
spec:
replicas: 1
template:
metadata:
labels:
task: monitoring
k8s-app: grafana
spec:
containers:
- name: grafana
image: registry.cn-shanghai.aliyuncs.com/grafana_cluster/grafana:latest
ports:
- containerPort: 3000
protocol: TCP
env:
- name: INFLUXDB_HOST
value: monitoring-influxdb
- name: GF_INSTALL_PLUGINS
value: grafana-kubernetes-app
---
apiVersion: v1
kind: Service
metadata:
name: monitoring-grafana
namespace: kube-system
spec:
ports:
- port: 80
targetPort: 3000
type: LoadBalancer
selector:
k8s-app: grafana
- 在grafana的pod中執(zhí)行安裝命令：
kubectl get pods -n <namespace>
kubectl exec -it <pod name> /bin/bash -n <namespace>
grafana-cli plugins install grafana-kubernetes-app
-
裝好插件后，需要在grafana中配置，才會(huì)生效
在grafana頁(yè)面彬犯，點(diǎn)擊plugins
點(diǎn)擊kubernets-enable
image.png
配置集群訪問(wèn)地址以及訪問(wèn)證書：
image.png -
集群訪問(wèn)證書脸爱，用幾圈配置文件中的證書信息即可
image.png
其中屬性certificate-authority-data如孝、client-certificate-data操禀、client-key-data對(duì)應(yīng) CA 證書君躺、Client 證書寇蚊、Client 私鑰笔时， config 文件里面的內(nèi)容是base64編碼過(guò)后的，這里填寫內(nèi)容需要做base64解碼（百度搜索base64解碼）幔荒，保存糊闽。
grafana dashboard中自動(dòng)出現(xiàn)下圖中的dashboard
image.png
image.png
從圖中看到dashboard中都沒(méi)有數(shù)據(jù)。
Edit 圖表爹梁，修改監(jiān)控的數(shù)據(jù)項(xiàng)
image.png
prometheus alertmanager配置釘釘告警
Prometheus alertmanager支持告警發(fā)送到釘釘右犹，但是需要部署prometheus-webhook-dingtalk
apiVersion: v1
kind: Service
metadata:
name: prometheus-webhook-dingtalk
namespace: kube-ops
labels:
app: prometheus-webhook-dingtalk
spec:
type: ClusterIP
selector:
app: prometheus-webhook-dingtalk
ports:
- name: http
port: 5358
targetPort: 5358
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: prometheus-webhook-dingtalk
namespace: kube-ops
spec:
template:
metadata:
labels:
app: prometheus-webhook-dingtalk
spec:
containers:
- name: prometheus-webhook-dingtalk
image: timonwong/prometheus-webhook-dingtalk:latest
args:
- '--web.listen-address=:5358'
- '--ding.profile=webhook=https://oapi.dingtalk.com/robot/send?access_token=c83f3a98adf27544d6c1b01cbf30674cbb18c5de63784d62ccd3a42c2c06bb2c'
- '--ding.timeout=5s'
- '--log.level=info'
ports:
- containerPort: 5358
prometheus-alert-configuration 配置如下：
---
kind: ConfigMap
apiVersion: v1
metadata:
name: alertmanager
namespace: kube-ops
data:
config.yml: |-
global:
resolve_timeout: 5m
smtp_smarthost: 'localhost:25'
smtp_from: 'alertmanager@example.org'
smtp_auth_username: 'alertmanager'
smtp_auth_password: 'password'
route:
receiver: webhook
group_wait: 30s
group_interval: 5m
repeat_interval: 4h
group_by: [alertname]
routes:
- receiver: webhook
group_wait: 10s
match:
team: node
receivers:
- name: webhook
webhook_configs:
- url: 'http://prometheus-webhook-dingtalk:5358/dingtalk/webhook/send'
send_resolved: true
pagerduty_configs:
- service_key: 84c023a8c96f4339aa9716dcd213f421
