Kubernetes集羣之安裝flannel組件

Kubernetes集羣之安裝flannel組件

• flannel網絡組件，還有一個是calico，calico支持bgp
• overlay network：覆蓋網絡，在基礎網絡上疊加的一種虛擬網絡技術模式，該網絡中的主機經過虛擬鏈路tunnmel鏈接起來
• vxlan：將原數據包封裝到UDP協議中，並使用基礎網絡的IP/mac做爲外層報文頭進行封裝，而後在以太網二層鏈路上傳輸，到達目的地後由隧道端點解封裝並將數據發送給目標地址
• flannel：是overlay網絡中的一種，也是將源數據包封裝在另外一種網絡包裏面進行路由轉發和通訊，目前已經支持UDP、VXLAN、aws VPS和gce路由等數據轉發方式
  1.vxlan網絡拓撲
  
  vtep能夠當成docker 0 端口理解，vtep與物理網卡之間進行nat地址轉換，像這種信息也會寫入到etcd中
  2.集羣內不一樣節點間容器通信流程
  
  3.寫入分配的子網段到etcd中，給flannel使用
  在master節點寫入

  k8s/etcd/bin/etcdctl \
  --ca-file=/k8s/etcd/ssl/ca.pem \
  --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem \
  --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" \
  set /coreos.com/network/config '{ "network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
  [root@master1 /]# k8s/etcd/bin/etcdctl \
  > --ca-file=/k8s/etcd/ssl/ca.pem \
  > --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem \
  > --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" \
  > set /coreos.com/network/config '{ "network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
  { "network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}
  [root@master1 /]#

  查看寫入的信息，在其餘節點也能夠查看到

  /k8s/etcd/bin/etcdctl \
  --ca-file=/k8s/etcd/ssl/ca.pem \
  --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem \
  --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" \
  get /coreos.com/network/config

  4.導入二進制包，flannel安裝在node節點上
  哪一個節點須要跑業務，哪一個節點就要安裝fannel組件

  [root@master1 /]# cp /abc/k8s/flannel-v0.10.0-linux-amd64.tar.gz /root/k8s/
  [root@master1 /]# cp /abc/k8s/flannel-v0.10.0-linux-amd64.tar.gz /root/k8s/
  [root@master1 /]# cd /root/k8s
  [root@master1 k8s]# scp flannel-v0.10.0-linux-amd64.tar.gz root@192.168.191.131:/opt/
  root@192.168.191.131's password: 
  flannel-v0.10.0-linux-amd64.tar.gz                                                    100% 9479KB  53.4MB/s   00:00    
  [root@master1 k8s]# scp flannel-v0.10.0-linux-amd64.tar.gz root@192.168.191.132:/opt/
  root@192.168.191.132's password: 
  flannel-v0.10.0-linux-amd64.tar.gz

  5.部署與配置flannel，編輯flannel啓動腳本，加入到systemd中
  以node1節點爲例

  [root@node01 yum.repos.d]# cd /opt
  [root@node01 opt]# tar xf flannel-v0.10.0-linux-amd64.tar.gz 
  [root@node01 opt]# ls
  containerd  flanneld  flannel-v0.10.0-linux-amd64.tar.gz  mk-docker-opts.sh  README.md  rh

  建立fannel工做目錄

  [root@node01 opt]# mkdir /k8s/flannel/{cfg,bin,ssl} -p
  [root@node01 opt]# mv mk-docker-opts.sh /k8s/flannel/bin/
  [root@node01 opt]# mv flanneld /k8s/flannel/bin/

  fannel組件啓動腳本

  [root@node01 opt]# vim flannel.sh
  #!/bin/bash
  ETCD_ENDPOINTS=${1:-"http://127.0.0.1:2379"}
  cat <<EOF >/k8s/flannel/cfg/flanneld
  FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
  -etcd-cafile=/k8s/etcd/ssl/ca.pem \
  -etcd-certfile=/k8s/etcd/ssl/server.pem \
  -etcd-keyfile=/k8s/etcd/ssl/server-key.pem"
  EOF
  cat <<EOF >/usr/lib/systemd/system/flanneld.service
  [Unit]
  Description=Flanneld overlay address etcd agent
  After=network-online.target network.target
  Before=docker.service
  [Service]
  Type=notify
  EnvironmentFile=/k8s/flannel/cfg/flanneld
  ExecStart=/k8s/flannel/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
  ExecStartPost=/k8s/flannel/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
  Restart=on-failure
  [Install]
  WantedBy=multi-user.target
  EOF
  systemctl daemon-reload
  systemctl enable flanneld
  systemctl restart flanneld

  開啓flannel網絡功能，指定etcdIP：端口

  [root@node01 flannel]# bash flannel.sh https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379

  兩個node節點都須要
  6.配置docker，以使用flannel生成的子網
  以node1爲例
  讓docker鏈接flannel的網段

  [root@node01 flannel]# vim /usr/lib/systemd/system/docker.service 
  #在第十三行註釋下添加
  14 EnvironmentFile=/run/flannel/subnet.env
  #在ExecStart中添加
  15 ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS -H fd:// --containerd=/run/containerd/containerd.sock

  重啓docker服務

  [root@node01 flannel]# systemctl daemon-reload
  [root@node01 flannel]# systemctl restart docker

  7.啓動flannel
  查看node01節點分配的flannelIP地址，爲172.17.45.0/24

  [root@node01 flannel]# cat /run/flannel/subnet.env 
  DOCKER_OPT_BIP="--bip=172.17.45.1/24"
  DOCKER_OPT_IPMASQ="--ip-masq=false"
  DOCKER_OPT_MTU="--mtu=1450"
  DOCKER_NETWORK_OPTIONS=" --bip=172.17.45.1/24 --ip-masq=false --mtu=1450"

  查看fannel網絡

  [root@node01 flannel]# ifconfig
  docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
      inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
  ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
      inet 192.168.191.131  netmask 255.255.255.0  broadcast 192.168.247.255
  flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
      inet 172.17.45.0  netmask 255.255.255.255  broadcast 0.0.0.0
  lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
      inet 127.0.0.1  netmask 255.0.0.0
  virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
      inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255

  此時node1與node2中的容器就實現互通了
  在 兩個node節點分別測試

  [root@node01 flannel]# docker run -it centos:7 /bin/bash
  Unable to find image 'centos:7' locally
  7: Pulling from library/centos
  ab5ef0e58194: Pull complete 
  Digest: sha256:4a701376d03f6b39b8c2a8f4a8e499441b0d567f9ab9d58e4991de4472fb813c
  Status: Downloaded newer image for centos:7
  [root@39f034a2f24e /]# yum install net-tools -y

  [root@node02 opt]# docker run -it centos:7 /bin/bash
  Unable to find image 'centos:7' locally
  7: Pulling from library/centos
  ab5ef0e58194: Pull complete 
  Digest: sha256:4a701376d03f6b39b8c2a8f4a8e499441b0d567f9ab9d58e4991de4472fb813c
  Status: Downloaded newer image for centos:7
  [root@fea29d0ff39b /]# yum install net-tools -y

  node1容器ip

  [root@39f034a2f24e /]# ifconfig
  eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
      inet 172.17.45.2  netmask 255.255.255.0  broadcast 172.17.45.255

  node2節點ping node1 容器

  [root@fea29d0ff39b /]# ifconfig
  eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
      inet 172.17.42.2  netmask 255.255.255.0  broadcast 172.17.42.255
  [root@fea29d0ff39b /]# ping 172.17.45.2
  PING 172.17.45.2 (172.17.45.2) 56(84) bytes of data.
  64 bytes from 172.17.45.2: icmp_seq=1 ttl=62 time=0.792 ms
  64 bytes from 172.17.45.2: icmp_seq=2 ttl=62 time=0.762 ms
  64 bytes from 172.17.45.2: icmp_seq=3 ttl=62 time=0.483 ms
  64 bytes from 172.17.45.2: icmp_seq=4 ttl=62 time=1.38 ms
  ^C
  --- 172.17.45.2 ping statistics ---
  4 packets transmitted, 4 received, 0% packet loss, time 3003ms
  rtt min/avg/max/mdev = 0.483/0.855/1.384/0.328 ms
  [root@fea29d0ff39b /]#

  成功ping通。兩個node節點容器互通，fannel組件安裝部署成功！