主機名html |
ipnode |
備註python |
k8s_masternginx |
192.168.98.18docker |
Master&etcdvim |
k8s_node1centos |
192.168.98.19api |
Node1安全 |
k8s_node2bash |
192.168.98.20 |
Node2 |
Kubernetes 是goole開源的大規模容器集羣管理系統,使用centos7 自帶的Kubernetes 組件、分佈式鍵值存儲系統etcd 以及flannel 實現Docker容器中跨容器訪問。
(集羣環境須要ntp時鐘一致,由於是雲的機器,系統默認有時鐘覈對)
第一步組件安裝
Master節點:
systemctl stop firewalld && sudo systemctl disable firewalld yum install -y kubernetes etcd docker flannel
Node節點:
systemctl stop firewalld && sudo systemctl disable firewalld yum install -y kubernetes docker flannel
第二步配置
節點 |
運行服務 |
Master |
etcd kube-apiserver kube-controller-manager kube-scheduler kube-proxy kubelet docker flanneld |
node |
flanneld docker kube-proxy kubelet |
Master:
etcd配置
vi /etc/etcd/etcd.conf
ETCD_NAME=default ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379" ETCD_ADVERTISE_CLIENT_URLS=http://localhost:2379
apiserver 配置
vi /etc/kubernetes/apiserver
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" (apiserver綁定主機的非安全IP地址) KUBE_API_PORT="--port=8080" (apiserver綁定主機的非安全端口號) KUBELET_PORT="--kubelet-port=10250" KUBE_ETCD_SERVERS="--etcd-servers=http://192.168.98.18:2379" KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=192.168.98.0/24" (虛機同一網段) KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota" KUBE_API_ARGS=""
Kubelet配置
vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=0.0.0.0" KUBELET_PORT="--port=10250" KUBELET_HOSTNAME="--hostname-override=192.168.98.18" KUBELET_API_SERVER="--api-servers=http://192.168.98.18:8080" KUBELET_POD_INFRA_CONTAINER="--pod-infra-Container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest" KUBELET_ARGS=""
config配置
vi /etc/kubernetes/config
KUBE_LOGTOSTDERR="--logtostderr=true" KUBE_LOG_LEVEL="--v=0" KUBE_ALLOW_PRIV="--allow-privileged=false" KUBE_MASTER="--master=http://192.168.98.18:8080"
scheduler和 proxy 暫時沒有用到,就不須要配置
flannel 配置
vi /etc/sysconfig/flanneld
FLANNEL_ETCD="http://192.168.98.18:2379" FLANNEL_ETCD_KEY="/atomic.io/network"
添加網絡:
systemctl enable etcd.service systemctl start etcd.service etcdctl mk //atomic.io/network/config '{"Network":"172.17.0.0/16"}' 建立 etcdctl rm //atomic.io/network/config '{"Network":"172.17.0.0/16"}' 刪除
Master啓動:
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler kube-proxy kubelet docker flanneld ; do systemctl restart $SERVICES; systemctl enable $SERVICES; systemctl status $SERVICES; done;
node配置:
hostnamectl set-hostname k8s_node1/2
Kubelet配置
vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=0.0.0.0" KUBELET_PORT="--port=10250" KUBELET_HOSTNAME="--hostname-override=192.168.98.19" (相應節點IP) KUBELET_API_SERVER="--api-servers=http://192.168.98.18:8080" (master節點IP) KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest" KUBELET_ARGS=" "
config配置
vi /etc/kubernetes/config
KUBE_LOGTOSTDERR="--logtostderr=true" KUBE_LOG_LEVEL="--v=0" KUBE_ALLOW_PRIV="--allow-privileged=false" KUBE_MASTER="--master=http://192.168.98.18:8080"
flannel 配置
vi /etc/sysconfig/flanneld FLANNEL_ETCD="http://192.168.98.18:2379" FLANNEL_ETCD_KEY="/atomic.io/network"
node啓動
for SERVICES in kube-proxy kubelet docker flanneld; do
systemctl restart $SERVICES
systemctl enable $SERVICES
systemctl status $SERVICES
done;
查看全部NODE是否正常
kubectl -s 192.168.98.18:8080 get no kubectl get nodes
訪問http://kube-apiserver:port
http://192.168.98.18:8080/ 查看全部請求url
http://192.168.98.18:8080/healthz/ping 查看健康情況
###################################################以上搭建完畢
開始排錯:
1,—————–部署nginx測試——————-
nginx-pod.yaml (請注意語法)
apiVersion: v1 kind: Pod metadata: name: nginx-pod labels: name: nginx-pod spec: containers: - name: nginx image: nginx ports: - containerPort: 80
1、開始建立pod
[root@localhost ~]# kubectl create -f /opt/dockerconfig/nginx-pod.yaml Error from server (ServerTimeout): error when creating "/opt/dockerconfig/nginx-pod.yaml": No API token found for service account "default",retry after the token is automatically created and added to the service account
報錯是驗證產生的
[root@localhost ~]# vim /etc/kubernetes/apiserver
去掉相應配置
#KUBE_ADMISSION_CONTROL="–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota" KUBE_ADMISSION_CONTROL="–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota" [root@localhost ~]# systemctl restart kube-apiserver
問題解決
[root@localhost ~]# kubectl create -f /opt/dockerconfig/nginx-pod.yaml pod "nginx-pod" created [root@localhost ~]#
可是一直卡着
[root@localhost ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE nginx-pod 0/1 ContainerCreating 0 12m
[root@localhost ~]# kubectl get service
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes 192.168.98.1 <none> 443/TCP 1h
[root@localhost ~]#
主要是經過「kubectl describe pod PodName」指令查看pod發生的事件,從事件列表中能夠查找到錯誤信息。
查狀態
[root@master ~]# kubectl describe pod nginx
這個報錯,你們都懂的,哈哈。
手動下載:
在工做節點(node)上執行
docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest Trying to pull repository registry.access.redhat.com/rhel7/pod-infrastructure ... open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory
解決方案
經過提示的路徑查找該文件,是個軟鏈接,連接目標是/etc/rhsm,查看沒有rhsm
[root@MyCentos7 ca]# cd /etc/docker/certs.d/registry.access.redhat.com/ [root@MyCentos7 registry.access.redhat.com]# ll 總用量 0 lrwxrwxrwx. 1 root root 27 5月 11 14:30 redhat-ca.crt -> /etc/rhsm/ca/redhat-uep.pem [root@MyCentos7 ca]# cd /etc/rhsm -bash: cd: /etc/rhsm: 沒有那個文件或目redhat-ca.crt -> /etc/rhsm/ca/redhat-uep.pem
安裝rhsm(node上):
yum install *rhsm*
已加載插件:fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirror.lzu.edu.cn
* extras: mirror.lzu.edu.cn
* updates: ftp.sjtu.edu.cn
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
軟件包 python-rhsm-1.19.10-1.el7_4.x86_64 被已安裝的 subscription-manager-rhsm-1.20.11-1.el7.centos.x86_64 取代
軟件包 subscription-manager-rhsm-1.20.11-1.el7.centos.x86_64 已安裝而且是最新版本
軟件包 python-rhsm-certificates-1.19.10-1.el7_4.x86_64 被已安裝的 subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64 取代
軟件包 subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64 已安裝而且是最新版本
技術分享圖片
可是在/etc/rhsm/ca/目錄下依舊沒有證書文件,因而反覆卸載與安裝都不靠譜,後來發現你們所謂yum install *rhsm*其實安裝的的是python-rhsm-1.19.10-1.el7_4.x86_64和python-rhsm-certificates-1.19.10-1.el7_4.x86_64,可是在實際安裝過程當中會有以下提示:
軟件包 python-rhsm-1.19.10-1.el7_4.x86_64 被已安裝的 subscription-manager-rhsm-1.20.11-1.el7.centos.x86_64 取代
軟件包 subscription-manager-rhsm-1.20.11-1.el7.centos.x86_64 已安裝而且是最新版本
軟件包 python-rhsm-certificates-1.19.10-1.el7_4.x86_64 被已安裝的 subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64 取代
軟件包 subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64 已安裝而且是最新版本
罪魁禍首在這裏。原來咱們想要安裝的rpm包被取代了。而取代後的rpm包在安裝完成後之建立了目錄,並無證書文件redhat-uep.pem。因而乎,手動下載並生成文件
wget http://mirror.centos.org/centos/7/os/x86_64/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm
rpm2cpio python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm | cpio -iv --to-stdout ./etc/rhsm/ca/redhat-uep.pem | tee /etc/rhsm/ca/redhat-uep.pem
node上手動下載鏡像
至此查看狀態變爲pulling
說明ca 找不到問題解決。
可是發現又出現一個新的DNS問題,解決方案以下:
node執行:
yum remove subscription-manager-rhsm-certificates -y wget http://mirror.centos.org/centos/7/os/x86_64/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm rpm -ivh python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-pod 1/1 Running 0 11h
apiVersion: v1
kind: ReplicationController
metadata:
name: nginx-rc
spec:
replicas: 2
selector:
name: nginx-pod
template:
metadata:
labels:
name: nginx-pod
spec:
containers:
- name: nginx-pod
image: nginx
ports:
- containerPort: 80
kubectl create -f nginx-rc.yaml
3、新建 service.
nginx-service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: NodePort
ports:
- port: 80
nodePort: 30001
selector:
name: nginx-pod
[root@localdockerconfig]# kubectl create -f nginx-service.yaml service "nginx-service" created
訪問 node 機器的 30001端口測試成功