Rancher2.0ha安裝
環(huán)境
- CentOS 7.5
- Docker-ce 17.03
- rke v0.1.11
- kubectl Client v1.12.2, kubectl Server v1.11.3
- helm Client v2.11.0, helm Server v2.11.0
| 主機(jī)名 | IP | 備注 |
|---|---|---|
| k8s-master | 10.176.56.232 | 負(fù)載均衡器铃岔，rancher url請求 |
| k8s-node00 | 10.176.56.240 | rancher node汪疮，etcd controlplane |
| k8s-node01 | 10.176.57.151 | rancher node，worker |
| k8s-node01 | 10.176.57.152 | rancher node毁习，worker |
注：ip隨意智嚷，保證互通就行
1. 基礎(chǔ)環(huán)境配置(所有節(jié)點)
1.1 hostname和hosts配置
配置每臺主機(jī)的hosts(/etc/hosts),添加host_ip $hostname到/etc/hosts文件中。
[admin@k8s-master home]$ cat /etc/hostname
k8s-master
[admin@k8s-master home]$ cat /etc/hosts
10.176.57.152 k8s-node02
10.176.57.151 k8s-node01
10.176.56.240 k8s-node00
10.176.56.232 k8s-master
1.2 CentOS關(guān)閉selinux
sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
1.3 關(guān)閉防火墻(可選)或者放行相應(yīng)端口
systemctl stop firewalld.service && systemctl disable firewalld.service
注：為了避免出現(xiàn)網(wǎng)絡(luò)通信問題纺且，此例我將防火墻直接關(guān)閉盏道。如不選擇關(guān)閉防火墻，可配置端口放行载碌。端口放行可參考：https://www.cnrancher.com/docs/rancher/v2.x/cn/installation/references/
1.4 配置主機(jī)時間猜嘱、時區(qū)、系統(tǒng)語言
修改時區(qū)
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
修改系統(tǒng)語言環(huán)境
sudo echo 'LANG="en_US.UTF-8"' >> /etc/profile;source /etc/profile
安裝ntp服務(wù)
yum install ntp ntpdate -y
修改/etc/ntp.conf文件嫁艇，將server0~server4全部注釋然后添加自己或者其他的ntp服務(wù)器地址
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
server 10.176.56.9 iburst
重啟服務(wù)
systemctl restart ntpd.service
1.5 Kernel性能調(diào)優(yōu)
cat >> /etc/sysctl.conf<<EOF
net.ipv4.ip_forward=1
net.bridge.bridge-nf-call-iptables=1
net.ipv4.neigh.default.gc_thresh1=4096
net.ipv4.neigh.default.gc_thresh2=6144
net.ipv4.neigh.default.gc_thresh3=8192
EOF
保存配置
sysctl –p
2. Docker的安裝與配置（所有節(jié)點）
2.1 修改系統(tǒng)源
sudo cp /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bak
cat > /etc/yum.repos.d/CentOS-Base.repo << EOF
[base]
name=CentOS-$releasever - Base - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/os/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/os/$basearch/
http://mirrors.cloud.aliyuncs.com/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
#released updates
[updates]
name=CentOS-$releasever - Updates - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/updates/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/updates/$basearch/
http://mirrors.cloud.aliyuncs.com/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/extras/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/extras/$basearch/
http://mirrors.cloud.aliyuncs.com/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/centosplus/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/centosplus/$basearch/
http://mirrors.cloud.aliyuncs.com/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
#contrib - packages by Centos Users
[contrib]
name=CentOS-$releasever - Contrib - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/contrib/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/contrib/$basearch/
http://mirrors.cloud.aliyuncs.com/centos/$releasever/contrib/$basearch/
gpgcheck=1
enabled=0
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
EOF
生成緩存
sudo yum makecache
2.2 docker-ce 安裝
yum install https://mirrors.aliyun.com/docker-ce/linux/centos/7/x86_64/stable/Packages/docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch.rpm -y
yum install https://mirrors.aliyun.com/docker-ce/linux/centos/7/x86_64/stable/Packages/docker-ce-17.03.2.ce-1.el7.centos.x86_64.rpm -y
把當(dāng)前用戶加入docker用戶組
sudo usermod -aG dockerroot USERNAME
sudo systemctl restart docker
sudo chmod a+rw /var/run/docker.sock
注：USERNAME填寫自己的用戶名
注：因為CentOS的安全限制朗伶，通過RKE安裝K8S集群時候無法使用root賬戶。所以步咪，建議CentOS用戶使用非root用戶來運行docker
設(shè)置開機(jī)啟動
sudo systemctl enable docker
2.3 docker-ce 配置
編輯/etc/docker/daemon.json文件
{
"max-concurrent-downloads": 3,
"max-concurrent-uploads": 5,
"registry-mirrors": ["https://7bezldxe.mirror.aliyuncs.com/"],
"storage-driver": "overlay2",
"storage-opts": ["overlay2.override_kernel_check=true"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m",
"max-file": "3"
}
}
有私有倉庫的可加上自己的私有倉庫 "insecure-registries": ["IP:PORT"]
3. 創(chuàng)建四層負(fù)載均衡（56.232節(jié)點）
3.1 推薦架構(gòu)
3.2 必備工具
- kubectl - Kubernetes命令行工具
- rke - Rancher Kubernetes Engine用于構(gòu)建Kubernetes集群
- helm - Kubernetes的包管理
https://www.cnrancher.com/docs/rancher/v2.x/cn/installation/download/#kubectl
3.3 配置負(fù)載均衡器（56.232節(jié)點）
添加nginx源
vim /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
下載nginx
yum install -y nginx
配置/etc/nginx/nginx.conf文件
worker_processes 4;
worker_rlimit_nofile 40000;
events {
worker_connections 8192;
}
http {
server {
listen 80;
return 301 https://$host$request_uri;
}
}
stream {
upstream rancher_servers {
least_conn;
server 10.176.56.240:443 max_fails=3 fail_timeout=5s;
server 10.176.57.151:443 max_fails=3 fail_timeout=5s;
server 10.176.57.152:443 max_fails=3 fail_timeout=5s;
}
server {
listen 443;
proxy_pass rancher_servers;
}
}
重新加載nginx服務(wù)
nginx -s reload
4. RKE安裝kubernetes（56.232節(jié)點）
4.1 創(chuàng)建rancher-cluster.yml文件
[admin@k8s-master home]$ cat rancher-cluster.yml
nodes:
- address: 10.176.56.240
user: admin
role: [controlplane,etcd]
- address: 10.176.57.151
user: admin
role: [worker]
- address: 10.176.57.152
user: admin
role: [worker]
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
如果您的節(jié)點有public and internal地址论皆，建議設(shè)置internal_address:以便Kubernetes將其用于集群內(nèi)通信
4.2 運行rke命令
下載rke_linux-amd64并放在和rancher-cluster.yml同目錄下，修改rke_linux-amd64名稱為rke
mv rke_linux-amd64 rke
chmod +x ./rke
使用rke安裝kubernetes
./rke up --config ./rancher-cluster.yml
注：如果顯示沒有權(quán)限創(chuàng)建kube_config_rancher-cluster.yml文件猾漫，sudo也不能創(chuàng)建点晴。可先手動創(chuàng)建一個kube_config_rancher-cluster.yml文件悯周，并將其權(quán)限修改為普通用戶可讀寫即可`
將kube_config_rancher-cluster.yml文件復(fù)制到HOME/.kube/config
cp kube_config_rancher-cluster.yml $HOME/.kube/config
將KUBECONFIG環(huán)境變量設(shè)置為kube_config_rancher-cluster.yml文件路徑
export KUBECONFIG=$(pwd)/kube_config_rancher-cluster.yml
4.3 安裝kubectl
下載kubectl_linux-amd64
確保kubectl二進(jìn)制文件是可執(zhí)行文件
chmod +x ./kubectl
將kubectl二進(jìn)制文件移動到PATH路徑下
sudo mv ./kubectl /usr/local/bin/kubectl
4.4 測試是否安裝成功
保存kube_config_rancher-cluster.yml和rancher-cluster.yml文件的副本,您將需要這些文件來維護(hù)和升級Rancher實例
5. 安裝配置helm（56.232節(jié)點）
5.1 配置helm客戶端訪問權(quán)限
kubectl -n kube-system create serviceaccount tiller
kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
Helm在集群上安裝tiller服務(wù)以管理charts. 由于RKE默認(rèn)啟用RBAC, 因此我們需要使用kubectl來創(chuàng)建一個serviceaccount粒督，clusterrolebinding才能讓tiller具有部署到集群的權(quán)限
5.2 安裝helm客戶端
下載你需要的版本：https://github.com/helm/helm/releases
解壓縮
tar -zxvf helm-v2.11.0-linux-amd64.tgz
使helm可執(zhí)行
mv linux-amd64/helm /usr/local/bin/helm
在kube-system命名空間中創(chuàng)建ServiceAccount
kubectl -n kube-system create serviceaccount tiller
創(chuàng)建ClusterRoleBinding以授予tiller帳戶對集群的訪問權(quán)限
kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
5.3 安裝Helm Server(Tiller)
helm init --service-account tiller --tiller-image registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.11.0 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
查看tiller是否運行成功
kubectl get pods --namespace kube-system
6. helm安裝rancher
6.1 添加chart倉庫地址
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
6.2 安裝證書管理器
helm install stable/cert-manager \
--name cert-manager \
--namespace kube-system
只有Rancher自動生成的證書和LetsEncrypt頒發(fā)的證書才需要cert-manager。如果是你自己的證書禽翼，可使用ingress.tls.source=secret參數(shù)指定證書屠橄，并跳過此步驟∪虻玻可參考https://www.cnrancher.com/docs/rancher/v2.x/cn/installation/server-installation/ha-install/helm-rancher/rancher-install/
6.3 選擇SSL配置方式并安裝Rancher server(rancher自動生成證書)
helm install rancher-stable/rancher \
--name rancher \
--namespace cattle-system \
--set hostname=k8s-master
內(nèi)網(wǎng)環(huán)境可添加 --set proxy=" " 和 --set noProxy=" "來設(shè)置代理
6.4 為Agent Pod添加主機(jī)別名(/etc/hosts)
如果你沒有內(nèi)部DNS服務(wù)器而是通過添加/etc/hosts主機(jī)別名的方式指定的Rancher server域名锐墙，那么不管通過哪種方式(自定義、導(dǎo)入解总、Host驅(qū)動等)創(chuàng)建K8S集群贮匕，K8S集群運行起來之后，因為cattle-cluster-agent Pod和cattle-node-agent無法通過DNS記錄找到Rancher server,最終導(dǎo)致無法通信花枫。
export KUBECONFIG=xxx/xxx/xx.kubeconfig.yaml #指定kubectl配置文件
kubectl -n cattle-system patch deployments cattle-cluster-agent --patch '{
"spec": {
"template": {
"spec": {
"hostAliases": [
{
"hostnames":
[
"k8s-master"
],
"ip": "10.176.56.232"
}
]
}
}
}
}'
export KUBECONFIG=xxx/xxx/xx.kubeconfig.yaml #指定kubectl配置文件
kubectl -n cattle-system patch daemonsets cattle-node-agent --patch '{
"spec": {
"template": {
"spec": {
"hostAliases": [
{
"hostnames":
[
"k8s-master"
],
"ip": "10.176.56.232"
}
]
}
}
}
}'
我沒有域名服務(wù)器所有直接將ip加到了訪問rancher的機(jī)器的hosts里面刻盐，出現(xiàn)不是安全連接掏膏，點擊高級??繼續(xù)前往
訪問https://k8s-master,創(chuàng)建admin密碼即可
