Centos7.6 使用kubeadm部署Kubernetes1.15集羣

1、環境準備

172.16.8.101    k8s-master01
172.16.8.102    k8s-master02
172.16.8.103    k8s-master03
172.16.8.104    k8s-master04
172.16.8.105    k8s-master05
172.16.8.106    k8s-master06
#harbor
172.16.8.201    k8s-node01

一、升級內核

CentOS 7.x 系統自帶的 3.10.x 內核存在一些 Bugs，致使運行的 Docker、Kubernetes 不穩定，例如：
高版本的 docker(1.13 之後) 啓用了 3.10 kernel 實驗支持的 kernel memory account 功能(沒法關閉)，當節點壓力大如頻繁啓動和中止容器時會致使 cgroup memory leak；
網絡設備引用計數泄漏，會致使相似於報錯："kernel:unregister_netdevice: waiting for eth0 to become free. Usage count = 1";
https://blog.51cto.com/10880347/2420861
https://blog.csdn.net/xiegh2014/article/details/96192233

二、設置/etc/hosts

cat >> /etc/hosts<<EOF
172.16.8.101    k8s-master01
172.16.8.102    k8s-master02
172.16.8.103    k8s-master03
172.16.8.104    k8s-master04
172.16.8.105    k8s-master05
172.16.8.106    k8s-master06
#harbor
172.16.8.201    k8s-node01
EOF

三、關閉防火牆及SELINUX

https://devops51.blog.csdn.net/article/details/53031781

四、關閉swap分區

swapoff -a
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

五、Kernel性能調優

必須關閉 tcp_tw_recycle，不然和NAT 衝突，會致使服務不通；
關閉 IPV6，防止觸發 docker BUG；

cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0 
vm.overcommit_memory=1 
vm.panic_on_oom=0 
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF

sysctl -p /etc/sysctl.d/k8s.conf

六、加載內核模塊

modprobe ip_vs_rr
modprobe br_netfilter

七、設置系統時區

調整系統 TimeZone
timedatectl set-timezone Asia/Shanghai

將當前的 UTC 時間寫入硬件時鐘
timedatectl set-local-rtc 0

重啓依賴於系統時間的服務
systemctl restart rsyslog 
systemctl restart crond

yum install -y conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget
yum install -y yum-utils device-mapper-persistent-data lvm2

使用阿里雲Docker Yum源
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

選擇安裝的版本

yum list docker-ce --showduplicates | sort -r

已加載插件：fastestmirror, langpacks
可安裝的軟件包
Loading mirror speeds from cached hostfile

• epel: mirror01.idc.hinet.net
• elrepo: mirror.rackspace.com
  docker-ce.x86_64 3:19.03.1-3.el7 docker-ce-stable
  docker-ce.x86_64 3:19.03.0-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.8-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.7-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.6-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.5-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.4-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.3-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.2-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.1-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.0-3.el7 docker-ce-stable
  docker-ce.x86_64 18.06.3.ce-3.el7 docker-ce-stable
  docker-ce.x86_64 18.06.2.ce-3.el7 docker-ce-stable
  docker-ce.x86_64 18.06.1.ce-3.el7 docker-ce-stable
  docker-ce.x86_64 18.06.0.ce-3.el7 docker-ce-stable
  docker-ce.x86_64 18.03.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 18.03.0.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.12.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.12.0.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.09.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.09.0.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.06.2.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.06.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.06.0.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.03.3.ce-1.el7 docker-ce-stable
  docker-ce.x86_64 17.03.2.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.03.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.03.0.ce-1.el7.centos docker-ce-stable

yum install docker-ce-18.09.8-3.el7 -y

啓動DOcker
systemctl start docker
systemctl enable docker

cat > /etc/docker/daemon.json<<EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors" : [
"https://ot2k4d59.mirror.aliyuncs.com/"
]
}
EOF
配置Docker鏡像倉庫必須重啓
systemctl restart docker

查看Docker信息
[root@k8s-master01 ~]# docker info
Client:
Debug Mode: false

Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.09.8
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.4.183-1.el7.elrepo.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.936GiB
Name: k8s-master01
ID: YZ5Q:2IUX:THW4:Z7DF:GK43:QG4O:D2M2:DVUL:3YI3:G3Y3:J4QE:5A64
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://ot2k4d59.mirror.aliyuncs.com/
Live Restore Enabled: false
Product License: Community Engine

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes