說明

Prometheus負責收集數(shù)據(jù)董朝，Grafana負責展示數(shù)據(jù)变勇。其中采用Prometheus 中的 Exporter含：

1. Node Exporter，負責收集 host 硬件和操作系統(tǒng)數(shù)據(jù)魁瞪。它將以容器方式運行在所有 host 上
2. 2cAdvisor卿叽，負責收集容器數(shù)據(jù)。它將以容器方式運行在所有 host 上
3. Alertmanager滋早，負責告警榄审。它將以容器方式運行在所有 host 上

docker安裝

環(huán)境準備

使用清華鏡像

curl -fsSL https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/debian/gpg | sudo apt-key add -

配置docker apt

echo 'deb https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/debian/ buster stable' | sudo tee /etc/apt/sources.list.d/docker.list

更新apt

apt-get update

安裝docker

如果已經(jīng)安裝過的需要先卸載就版本，沒有安裝過輸入命令顯示無效操作

apt-get docker docker-engine docker.io

進行docker安裝

apt-get install docker-ce

查看docker版本

docker -v

查看docker運行狀態(tài)

systemctl status docker 或者 service docker status

安裝docker-compose

apt install docker-compose

添加配置文件

mkdir -p /usr/local/src/config
cd /usr/local/src/config

添加prometheus.yml配置文件

global:
  scrape_interval:     15s
  evaluation_interval: 15s

alerting:
  alertmanagers:
  - static_configs:
    - targets: ['192.168.127.130:9093']

rule_files:
  - "node_down.yml"

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
    - targets: ['192.168.127.130:9090']

  - job_name: 'cadvisor'
    static_configs:
    - targets: ['192.168.127.130:8080']

  - job_name: 'node'
    scrape_interval: 8s
    static_configs:
      - targets: ['192.168.127.130:9100']

添加郵件告警配置文件

global:
  smtp_smarthost: 'smtp.qq.com:25'
  smtp_from: ''
  smtp_auth_username: ''
  smtp_auth_password: ''
  smtp_require_tls: false

route:
  group_by: ['alertname']
  group_wait: 10s
  group_interval: 10s
  repeat_interval: 10m
  receiver: live-monitoring

receivers:
- name: 'live-monitoring'
  email_configs:
  - to: ''

添加報警規(guī)則

groups:
- name: node_down
  rules:
  - alert: InstanceDown
    expr: up == 0
    for: 1m
    labels:
      user: test
    annotations:
      summary: "Instance {{ $labels.instance }} down"
      description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 1 minutes."

編寫docker-compose

version: '2'

networks:
    monitor:
        driver: bridge

services:
    prometheus:
        image: prom/prometheus:latest
        container_name: prometheus
        hostname: prometheus
        restart: always
        volumes:
            - /usr/local/src/config/prometheus.yml:/etc/prometheus/prometheus.yml
            - /usr/local/src/config/node_down.yml:/etc/prometheus/node_down.yml
        ports:
            - "9090:9090"
        networks:
            - monitor

    alertmanager:
        image: quay.io/prometheus/alertmanager:v0.12.0
        container_name: alertmanager
        hostname: alertmanager
        restart: always
        volumes:
            - /usr/local/src/config/alertmanager.yml:/etc/alertmanager/alertmanager.yml
        ports:
            - "9093:9093"
        networks:
            - monitor

    grafana:
        image: grafana/grafana:6.3.5
        container_name: grafana
        hostname: grafana
        restart: always
        ports:
            - "3000:3000"
        networks:
            - monitor

    node-exporter:
        image: quay.io/prometheus/node-exporter
        container_name: node-exporter
        hostname: node-exporter
        restart: always
        ports:
            - "9100:9100"
        networks:
            - monitor

    cadvisor:
        image: google/cadvisor:latest
        container_name: cadvisor
        hostname: cadvisor
        restart: always
        volumes:
            - /:/rootfs:ro
            - /var/run:/var/run:rw
            - /sys:/sys:ro
            - /var/lib/docker/:/var/lib/docker:ro
        ports:
            - "8080:8080"
        networks:
            - monitor

啟動docker-compose

啟動容器：
docker-compose -f /usr/local/src/config/docker-compose-monitor.yml up -d
刪除容器：
docker-compose -f /usr/local/src/config/docker-compose-monitor.yml down
重啟容器：
docker restart id

image.png

9100：收集到數(shù)據(jù)杆麸，有了它可以做數(shù)據(jù)展示
9093：配置文件的數(shù)據(jù)
9090：prometheus的web界面
8080：用于收集搁进，聚合，處理和導出有關正在運行的容器的信息
3000：grafana的web界面

附錄：單獨命令啟動各容器

#啟動prometheus
docker run -d -p 9090:9090 --name=prometheus \
-v /usr/local/src/config/prometheus.yml:/etc/prometheus/prometheus.yml \
-v /usr/local/src/config/node_down.yml:/etc/prometheus/node_down.yml \
prom/prometheus

# 啟動grafana
docker run -d -p 3000:3000 --name=grafana grafana/grafana

#啟動alertmanager容器
docker run -d -p 9093:9093 -v /usr/local/src/config/config.yml:/etc/alertmanager/config.yml --name alertmanager prom/alertmanager

#啟動node exporter
docker run -d \
  -p 9100:9100 \
  -v "/:/host:ro,rslave" \
  --name=node_exporter \
  quay.io/prometheus/node-exporter \
  --path.rootfs /host

#啟動cadvisor
docker run                                    \
--volume=/:/rootfs:ro                         \
--volume=/var/run:/var/run:rw                 \
--volume=/sys:/sys:ro                         \
--volume=/var/lib/docker/:/var/lib/docker:ro  \
--publish=8080:8080                           \
--detach=true                                 \
--name=cadvisor                               \
google/cadvisor:latest