PromQL

空閑的cpu指標(biāo)

node_cpu_seconds_total{instance="192.168.153.21:9100",mode="idle"}

除空閑之外的其他指標(biāo)

node_cpu_seconds_total{instance="192.168.153.21:9100",mode!="idle"}

192.168.153.21 的cpu使用率

100-(avg(irate(node_cpu_seconds_total{instance="192.168.153.21:9100",mode="idle"}[5m]))*100)

標(biāo)簽管理

自定義標(biāo)簽

  - job_name: 'Linux Server'
    static_configs:
    - targets: ['192.168.153.21:9100']
      labels:
        project: k8s
        type: monitor

1635734900648.png

重命名標(biāo)簽

  - job_name: 'BJLinux Server'
    static_configs:
    - targets: ['192.168.153.22:9100']
      labels:
        add: bj
    metric_relabel_configs:
    - action: replace                                    
      source_labels: ['instance']
      regex: (.*):([0-9]+)
      replacement: $1
      target_label: "ip"

1635733160764.png

過濾Target

 

  - job_name: 'Linux Server'
    static_configs:
    - targets: ['192.168.153.21:9100','192.168.153.22:9100']
      labels:
        project: k8s
        type: monitor
    relabel_configs:
      - action: drop
        regex: "192.168.153.22:9100"
        source_labels: ["__address__"]
        
-------------------------------------------------------------------
#查詢驗(yàn)證，192.168.153.22:9100信息將不再顯示
node_cpu_seconds_total{project="k8s"}

1635734900648.png

刪除標(biāo)簽

  - job_name: 'SHLinux Server'
    static_configs:
    - targets: ['192.168.153.21:9100']
      labels:
        add: sh
    relabel_configs:
    - action: labeldrop
      regex: "job"

-------------------------------------------------------------------
#查詢驗(yàn)證正勒，job信息信息將不再顯示
node_cpu_seconds_total{add="sh"}

1635735782180.png

1635735834447.png

Grafana

儀表盤

1635821014977.png

導(dǎo)航欄（job）

1635821510036.png

label_values(up,job)

Preview of values：
BJLinux Server
kubernetes-service-endpoints
kubernetes-nodes-cadvisor
Linux Server
prometheus

導(dǎo)航欄（節(jié)點(diǎn)）

1635821894512.png

1635822164638.png

(up{job="$job"},instance)
#每個(gè)job有不同節(jié)點(diǎn)

cpu核數(shù)

1635822689574.png

count(node_cpu_seconds_total{instance="$node",job="$job",mode="idle"})
組件：stat

內(nèi)存總量

1635823252179.png

node_memory_MemTotal_bytes{instance="$node",job="$job"}
組件：stat

內(nèi)存使用率

1635823654918.png

1635838490518.png

100 - (avg(irate(node_cpu_seconds_total{instance="$node",mode="idle"}[5m])) by (instance) * 100)

內(nèi)存使用率

1635838734374.png

100 - (node_memory_MemFree_bytes+node_memory_Cached_bytes+node_memory_Buffers_bytes) / node_memory_MemTotal_bytes * 100 

100 - (node_memory_MemFree_bytes{instance="$node",job="$job"}+node_memory_Cached_bytes{instance="$node",job="$job"}+node_memory_Buffers_bytes{instance="$node",job="$job"}) / node_memory_MemTotal_bytes{instance="$node",job="$job"} * 100 

根分區(qū)使用率

1635839249336.png

100 - (node_filesystem_free_bytes{fstype=~"ext4|xfs",mountpoint="/"} / node_filesystem_size_bytes{fstype=~"ext4|xfs",mountpoint="/"} * 100)


100 - (node_filesystem_free_bytes{fstype=~"ext4|xfs",mountpoint="/",instance="$node",job="$job"} / node_filesystem_size_bytes{fstype=~"ext4|xfs",mountpoint="/",instance="$node",job="$job"} * 100)