一酸纲、準(zhǔn)備環(huán)境

1.1 準(zhǔn)備機(jī)器

準(zhǔn)備三臺(tái)CentOS7.2機(jī)器蚓土，一臺(tái)作為master節(jié)點(diǎn)庞溜，其他作為node節(jié)點(diǎn)

修改主機(jī)名稱

hostnamectl set-hostname k8s-mst

Role          IP                   Hostname

Master        192.168.0.87         k8s-cns1-mst

Node          192.168.0.88         k8s-cns1-nod1

Node          192.168.0.89         k8s-cns1-nod2

修改master節(jié)點(diǎn)/etc/hosts，添加以下內(nèi)容（不設(shè)置的話丛晦，在master上kubectl相關(guān)命令無(wú)法操作對(duì)應(yīng)主機(jī)上對(duì)象）

192.168.0.87     k8s-cns1-mst

192.168.0.88     k8s-cns1-nod1

192.168.0.89     k8s-cns1-nod2

為了避免和Docker的iptables產(chǎn)生沖突，關(guān)閉Node節(jié)點(diǎn)上的防火墻

systemctl stop firewalld

systemctl disable firewalld

了讓各個(gè)節(jié)點(diǎn)的時(shí)間保持一致提陶，所有節(jié)點(diǎn)安裝NTP

yum -y install ntp

systemctl start ntpd

systemctl enable ntpd

1.2 安裝docker（此處安裝的是docker-ce版本）

[root@k8s-cns1-nod2 home]# cat installdocker.sh

#!/bin/bash

yum install -y yum-utils device-mapper-persistent-data lvm2

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

yum-config-manager --enable docker-ce-edge

yum-config-manager --enable docker-ce-testing

yum makecache fast

yum install -y docker-ce

運(yùn)行Docker Daemon

systemctl start docker

二烫沙、源碼編譯

2.1 準(zhǔn)備golang環(huán)境

參考https://golang.org/doc/install，下載對(duì)應(yīng)版本并解壓到/usr/local隙笆，例如

tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz

安裝git斧吐，下載kubenates源碼并切換到需要分支

yum install git

go get -d k8s.io/kubernetes

cd /root/go/src/k8s.io/kubernetes

git checkout release-1.6.3      //使用release-1.6.3版本

make

編譯成功后又固，可執(zhí)行文件在

/root/go/src/k8s.io/kubernetes/_output/bin

三、Master配置

3.1 安裝ectd（可選煤率，如已有etcd集群略過(guò)）

3.1.1 軟件安裝

yum -y install etcd

3.1.2 修改etcd配置/etc/etcd/etcd.conf

ETCD_NAME=default

ETCD_DATA_DIR="/var/lib/etcd/default.etcd"

ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"

3.1.3 運(yùn)行etcd

systemctl enable etcd

systemctl start etcd

3.1.4 配置etcd子網(wǎng)

etcdctl mk /atomic.io/network/config '{"Network":"172.17.0.0/16"}'

3.2 kubernates配置

3.2.1 復(fù)制二進(jìn)制

將位于/root/go/src/k8s.io/kubernetes/_output/bin/目錄下的kube-apiserver仰冠、kube-controller-manager、kube-scheduler蝶糯、kubectl復(fù)制到Master節(jié)點(diǎn)的/usr/bin/目錄下

3.2.2 創(chuàng)建service配置腳本(shell)

[root@k8s-cns1-mst home]# cat configmaster.sh 
#!/bin/bash
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

MASTER_ADDRESS=${1:-"192.168.0.87"}
ETCD_SERVERS=${2:-"http://192.168.0.87:2379"}
SERVICE_CLUSTER_IP_RANGE=${3:-"10.254.0.0/16"}
ADMISSION_CONTROL=${4:-"NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"}

cat <<EOF >/etc/kubernetes/config
# --logtostderr=true: log to standard error instead of files
KUBE_LOGTOSTDERR="--logtostderr=false"

# --v=0: log level for V logs
KUBE_LOG_LEVEL="--v=0"

# --allow-privileged=false: If true, allow privileged containers.
KUBE_ALLOW_PRIV="--allow-privileged=true"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=${MASTER_ADDRESS}:8080"
EOF

cat <<EOF >/etc/kubernetes/apiserver
# --insecure-bind-address=127.0.0.1: The IP address on which to serve the --insecure-port.
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"

# --insecure-port=8080: The port on which to serve unsecured, unauthenticated access.
KUBE_API_PORT="--insecure-port=8080"

# --kubelet-port=10250: Kubelet port
NODE_PORT="--kubelet-port=10250"

# --etcd-servers=[]: List of etcd servers to watch (http://ip:port), 
# comma separated. Mutually exclusive with -etcd-config
KUBE_ETCD_SERVERS="--etcd-servers=${ETCD_SERVERS}"

# --advertise-address=<nil>: The IP address on which to advertise 
# the apiserver to members of the cluster.
KUBE_ADVERTISE_ADDR="--advertise-address=${MASTER_ADDRESS}"

# --service-cluster-ip-range=<nil>: A CIDR notation IP range from which to assign service cluster IPs. 
# This must not overlap with any IP ranges assigned to nodes for pods.
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=${SERVICE_CLUSTER_IP_RANGE}"

# --admission-control="AlwaysAdmit": Ordered list of plug-ins 
# to do admission control of resources into cluster. 
# Comma-delimited list of: 
#   LimitRanger, AlwaysDeny, SecurityContextDeny, NamespaceExists, 
#   NamespaceLifecycle, NamespaceAutoProvision,
#   AlwaysAdmit, ServiceAccount, ResourceQuota, DefaultStorageClass
KUBE_ADMISSION_CONTROL="--admission-control=${ADMISSION_CONTROL}"

# Add your own!
KUBE_API_ARGS="--log-dir=/var/log/kubenates/"
EOF

KUBE_APISERVER_OPTS="   \${KUBE_LOGTOSTDERR}         \\
                        \${KUBE_LOG_LEVEL}           \\
                        \${KUBE_ETCD_SERVERS}        \\
                        \${KUBE_API_ADDRESS}         \\
                        \${KUBE_API_PORT}            \\
                        \${NODE_PORT}                \\
                        \${KUBE_ADVERTISE_ADDR}      \\
                        \${KUBE_ALLOW_PRIV}          \\
                        \${KUBE_SERVICE_ADDRESSES}   \\
                        \${KUBE_ADMISSION_CONTROL}   \\
                        \${KUBE_API_ARGS}"


cat <<EOF >/usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target
After=etcd.service

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver ${KUBE_APISERVER_OPTS}
Restart=on-failure
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/kubernetes/controller-manager
###
# The following values are used to configure the kubernetes controller-manager

# defaults from config and apiserver should be adequate

# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS=""
EOF

KUBE_CONTROLLER_MANAGER_OPTS="  \${KUBE_LOGTOSTDERR} \\
                                \${KUBE_LOG_LEVEL}   \\
                                \${KUBE_MASTER}      \\
                                \${KUBE_CONTROLLER_MANAGER_ARGS}"

cat <<EOF >/usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager ${KUBE_CONTROLLER_MANAGER_OPTS}
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/kubernetes/scheduler
###
# kubernetes scheduler config

# Add your own!
KUBE_SCHEDULER_ARGS=""
EOF

KUBE_SCHEDULER_OPTS="   \${KUBE_LOGTOSTDERR}     \\
                        \${KUBE_LOG_LEVEL}       \\
                        \${KUBE_MASTER}          \\
                        \${KUBE_SCHEDULER_ARGS}"

cat <<EOF >/usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler ${KUBE_SCHEDULER_OPTS}
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload

3.2.3 創(chuàng)建service啟動(dòng)腳本

for svc in kube-apiserver kube-controller-manager kube-scheduler; do 
    systemctl restart $svc
    systemctl enable $svc
    systemctl status $svc 
done

四洋只、Node配置

4.1 安裝flanneld

4.1.1 安裝軟件

yum -y install flannel

4.1.2 修改flannel的配置文件/etc/sysconfig/flanneld

FLANNEL_ETCD="http://192.168.0.87:2379"
FLANNEL_ETCD_KEY="/atomic.io/network"

4.1.3 運(yùn)行flannel

systemctl restart flanneld
systemctl enable flanneld
systemctl status flanneld

4.1.4 上傳網(wǎng)絡(luò)配置
創(chuàng)建一個(gè)config.json文件，內(nèi)容如下：

{
"Network": "172.17.0.0/16",
"SubnetLen": 24,
"Backend": {
     "Type": "vxlan",
     "VNI": 7890
     }
 }

將配置上傳到etcd服務(wù)器上

curl -L http://192.168.0.87:2379/v2/keys/atomic.io/network/config -XPUT --data-urlencode value@config.json

查看etcd分配的子網(wǎng)信息

[root@k8s-sz-0002 ~]# cat /run/flannel/subnet.env 
FLANNEL_NETWORK=172.17.0.0/16
FLANNEL_SUBNET=172.17.79.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=false

4.2 kubernates配置

4.2.1 復(fù)制二進(jìn)制

將位于/root/go/src/k8s.io/kubernetes/_output/bin/目錄下的kube-proxy昼捍、kubelet 復(fù)制到Node節(jié)點(diǎn)的/usr/bin/目錄下

4.2.2 創(chuàng)建service配置腳本

[root@k8s-cns1-nod1 home]# cat configslave.sh 
#!/bin/bash
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

MASTER_ADDRESS=${1:-"192.168.0.87"}
NODE_HOSTNAME=${2:-"k8s-cns1-nod1"}

cat <<EOF >/etc/kubernetes/config
# --logtostderr=true: log to standard error instead of files
KUBE_LOGTOSTDERR="--logtostderr=true"

# --v=0: log level for V logs
KUBE_LOG_LEVEL="--v=0"

# --allow-privileged=false: If true, allow privileged containers.
KUBE_ALLOW_PRIV="--allow-privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=${MASTER_ADDRESS}:8080"
EOF

cat <<EOF >/etc/kubernetes/proxy
###
# kubernetes proxy config

# default config should be adequate

# Add your own!
KUBE_PROXY_ARGS="--log-dir=/var/log/kubenates/"
EOF

KUBE_PROXY_OPTS="   \${KUBE_LOGTOSTDERR} \\
                    \${KUBE_LOG_LEVEL}   \\
                    \${KUBE_MASTER}    \\
                    \${KUBE_PROXY_ARGS}"

cat <<EOF >/usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Proxy
After=network.target

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kube-proxy
ExecStart=/usr/bin/kube-proxy ${KUBE_PROXY_OPTS}
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/kubernetes/kubelet
# --address=0.0.0.0: The IP address for the Kubelet to serve on (set to 0.0.0.0 for all interfaces)
KUBELET__ADDRESS="--address=0.0.0.0"

# --port=10250: The port for the Kubelet to serve on. Note that "kubectl logs" will not work if you set this flag.
KUBELET_PORT="--port=10250"

# --hostname-override="": If non-empty, will use this string as identification instead of the actual hostname.
KUBELET_HOSTNAME="--hostname-override=${NODE_HOSTNAME}"

# --api-servers=[]: List of Kubernetes API servers for publishing events, 
# and reading pods and services. (ip:port), comma separated.
KUBELET_API_SERVER="--api-servers=http://${MASTER_ADDRESS}:8080"

# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

# Add your own!
KUBELET_ARGS=""
EOF

KUBE_PROXY_OPTS="   \${KUBE_LOGTOSTDERR}     \\
                    \${KUBE_LOG_LEVEL}       \\
                    \${KUBELET__ADDRESS}         \\
                    \${KUBELET_PORT}            \\
                    \${KUBELET_HOSTNAME}        \\
                    \${KUBELET_API_SERVER}   \\
                    \${KUBE_ALLOW_PRIV}      \\
                    \${KUBELET_POD_INFRA_CONTAINER}\\
                    \${KUBELET_ARGS}"

cat <<EOF >/usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet ${KUBE_PROXY_OPTS}
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload

4.2.3 創(chuàng)建service啟動(dòng)腳本

# cat /home/startslave.sh 
mkdir -p /etc/kubernetes/
mkdir -p /var/logs/kubernetes/
mkdir -p /var/lib/kubelet

source /run/flannel/subnet.env

sed -i "s|--bip=.*|--bip=${FLANNEL_SUBNET} --ip-masq=true --mtu=1472|" /usr/lib/systemd/system/docker.service

systemctl daemon-reload

for svc in docker kubelet kube-proxy; do 
    systemctl restart $svc
    systemctl enable $svc
    systemctl status $svc 
done

必須注意识虚，/usr/lib/systemd/system/docker.service是docker 的配置文件，因?yàn)閚ode和etcd之間是有租約時(shí)間的妒茬，如果node長(zhǎng)時(shí)間不在線的話担锤，etcd會(huì)認(rèn)為租約超期，清除子網(wǎng)信息乍钻，當(dāng)node起來(lái)后肛循，flannel重新獲取新的子網(wǎng)，所以docker容器網(wǎng)段(--bip=)建議每次都從新的flannel獲取银择，保證一致

編輯/etc/rc.local多糠，將腳本執(zhí)行加入開(kāi)機(jī)啟動(dòng)項(xiàng)，保證node重啟后自動(dòng)執(zhí)行腳本

驗(yàn)證環(huán)境配置

在Master節(jié)點(diǎn)運(yùn)行命令kubectl get nodes浩考，輸出信息如下：

[root@k8s-cns1-mst home]# kubectl get node
NAME            STATUS    AGE       VERSION
k8s-cns1-nod1   Ready     1d        v1.6.4
k8s-cns1-nod2   Ready     1d        v1.6.4

常見(jiàn)問(wèn)題

1夹孔、使用etcdctl時(shí)連接本地報(bào)錯(cuò)

/usr/lib # etcdctl ls /
Error:  client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
; error #1: dial tcp 127.0.0.1:4001: getsockopt: connection refused

error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
error #1: dial tcp 127.0.0.1:4001: getsockopt: connection refused

這是因?yàn)镋TCD_LISTEN_CLIENT_URLS參數(shù)沒(méi)有配置http://127.0.0.1:2379而導(dǎo)致，使用etcdctl時(shí)加上endpoints選項(xiàng)

/usr/lib # etcdctl --endpoints=192.168.0.87:2379 ls /