docker swarm部署minio集群

安裝系統(tǒng)

略

更換軟件源

首先備份原來的軟件源并另存：

sudo cp -v /etc/apt/sources.list /etc/apt/sources.list.backup

編輯軟件源文件

sudo vim /etc/apt/sources.list

注釋所有軟件源

阿里云源：

deb http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse

清華源

deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan-updates main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan-backports main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan-security main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan-security main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan-proposed main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ eoan-proposed main restricted universe multiverse

修改完軟件源后框仔，更新軟件列表和軟件：

sudo apt update
sudo apt upgrade

安裝docker

sudo apt install docker.io

設(shè)置docker鏡像加速

編輯daemon.json文件：

sudo vim /etc/docker/daemon.json

添加阿里云加速鏡像地址內(nèi)容：

{
"registry-mirrors": ["https://******.mirror.aliyuncs.com"]
}

重啟服務(wù)

sudo systemctl daemon-reload
sudo systemctl restart docker
sudo systemctl status docker

創(chuàng)建Swarm

在管理節(jié)點上創(chuàng)建一個swarm,請運行下面的命令

docker swarm init --advertise-addr <MANAGER-IP>

一旦swarm初使化了机打，你可以看到下面的響應(yīng)信息

docker swarm join \
  --token  SWMTKN-1-49nj1cmql0jkz5s954yi3oex3nedyz0fb0xx14ie39trti4wxv-8vxv8rssmk743ojnwacrr2e7c \
  192.168.99.100:2377

你現(xiàn)在可以運行上述命令添加worker節(jié)點到swarm。

添加`label`標(biāo)簽

# 查看節(jié)點信息
sudo docker node ls
# 為節(jié)點添加標(biāo)簽
sudo docker node update --label-add minio1=true (node_name)

minio1=true標(biāo)簽是minio在stack文件中定義的猿棉，用來確定節(jié)點部署的磅叛，所以必須添加，不然節(jié)點將不會部署萨赁。

為MinIO創(chuàng)建Docker secret

sudo echo "AKIAIOSFODNN7EXAMPLE" | sudo docker secret create access_key -
sudo echo "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" | sudo docker secret create secret_key -

access_key和secret_key可以自定義弊琴，盡量不要相同。

部署分布式minio服務(wù)

在你的Swarm master上下載Docker Compose file 杖爽，然后運行下面的命令

sudo docker stack deploy --compose-file=docker-compose-secrets.yaml minio_stack

查看節(jié)點部署信息

sudo docker service ls

服務(wù)列表

確認(rèn)每個節(jié)點都起來了敲董。

查看節(jié)點日志

sudo docker service logs (節(jié)點ID)

確認(rèn)每個節(jié)點都運行正常

配置nginx負(fù)載均衡

創(chuàng)建nginx配置文件minio.conf

內(nèi)容如下

upstream  minio-server {
       server    10.168.1.136:9003;
       server    10.168.1.136:9004;
       server    10.168.1.131:9001;
       server    10.168.1.131:9002;
}


server {
    listen              80;
    server_name 10.168.1.136;
    location / {
        proxy_buffering off;
        proxy_set_header Host $http_host;
        proxy_pass http://minio-server;
        proxy_redirect off;
    }
}

創(chuàng)建nginx相關(guān)文件夾

sudo mkdir -p /home/nginx/logs /home/nginx/conf

部署nginx

sudo docker run -d -p 80:80 --name nginx \
  -v /home/nginx/conf/:/etc/nginx/conf.d/ \
  -v /home/nginx/logs:/var/log/nginx \
  nginx

在瀏覽器中輸入服務(wù)器ip進行訪問

登錄界面

如果現(xiàn)在為初始化，請重試慰安，請檢查每個節(jié)點是否正常運行腋寨，以及每個節(jié)點是否能連通。
檢查每個節(jié)點的磁盤空間是否大于1G泻帮。

附：docker-compose-secrets.yaml文件內(nèi)容

version: '3.7'

services:
  minio1:
    image: minio/minio:RELEASE.2020-07-02T00-15-09Z
    hostname: minio1
    volumes:
      - minio1-data:/export
    ports:
      - "9001:9000"
    networks:
      - minio_distributed
    deploy:
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
      placement:
        constraints:
          - node.labels.minio1==true #和節(jié)點的label對應(yīng)
    command: server http://minio{1...4}/export
    secrets:
      - secret_key
      - access_key
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio2:
    image: minio/minio:RELEASE.2020-07-02T00-15-09Z
    hostname: minio2
    volumes:
      - minio2-data:/export
    ports:
      - "9002:9000"
    networks:
      - minio_distributed
    deploy:
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
      placement:
        constraints:
          - node.labels.minio2==true #和節(jié)點的label對應(yīng)
    command: server http://minio{1...4}/export
    secrets:
      - secret_key
      - access_key
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio3:
    image: minio/minio:RELEASE.2020-07-02T00-15-09Z
    hostname: minio3
    volumes:
      - minio3-data:/export
    ports:
      - "9003:9000"
    networks:
      - minio_distributed
    deploy:
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
      placement:
        constraints:
          - node.labels.minio3==true #和節(jié)點的label對應(yīng)
    command: server http://minio{1...4}/export
    secrets:
      - secret_key
      - access_key
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio4:
    image: minio/minio:RELEASE.2020-07-02T00-15-09Z
    hostname: minio4
    volumes:
      - minio4-data:/export
    ports:
      - "9004:9000"
    networks:
      - minio_distributed
    deploy:
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
      placement:
        constraints:
          - node.labels.minio4==true #和節(jié)點的label對應(yīng)
    command: server http://minio{1...4}/export
    secrets:
      - secret_key
      - access_key
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

volumes:
  minio1-data:
  minio2-data:
  minio3-data:
  minio4-data:

networks:
  minio_distributed:
    driver: overlay

secrets:
  secret_key:
    external: true
  access_key:
    external: true