linux運維、架構之路-Kubernetes本地鏡像倉庫+dashboard部署

1、部署docker registry

           生產環境中咱們通常經過搭建本地的私有鏡像倉庫(docker registry)來拉取鏡像。

一、拉取registry鏡像

[root@k8s-master ~]#docker pull docker.io/registry 
Using default tag: latest
Trying to pull repository docker.io/library/registry ... 
sha256:0e40793ad06ac099ba63b5a8fae7a83288e64b50fe2eafa2b59741de85fd3b97: Pulling from docker.io/library/registry
b7f33cc0b48e: Pull complete 
46730e1e05c9: Pull complete 
458210699647: Pull complete 
0cf045fea0fd: Pull complete 
b78a03aa98b7: Pull complete 
Digest: sha256:0e40793ad06ac099ba63b5a8fae7a83288e64b50fe2eafa2b59741de85fd3b97
Status: Downloaded newer image for docker.io/registry:latest

二、啓動registry

docker run -d -p 5000:5000 --name=registry --restart=always --privileged=true  --log-driver=none -v /home/data/registrydata:/tmp/registry registry

注：/home/data/registrydata是一個比較大的系統分區，從此鏡像倉庫中的所有數據都會保存在這個外掛目錄下

三、Node節點更名並推送鏡像

①以部署dashboard的鏡像爲例，後面也會用到

百度雲下載連接：https://pan.baidu.com/s/1geKEADt#list/path=%2F    密碼：lbyp

②上傳到Node節點並推送到鏡像倉庫

docker load < dashboard.tar
docker load < podinfrastructure.tar
docker tag gcr.io/google_containers/kubernetes-dashboard-amd64:v1.7.1 10.0.0.211:5000/google_containers/kubernetes-dashboard-amd64:latest
docker tag registry.access.redhat.com/rhel7/pod-infrastructure:latest 10.0.0.211:5000/rhel7/pod-infrastructure:latest

docker push 10.0.0.211:5000/google_containers/kubernetes-dashboard-amd64:latest
docker push 10.0.0.211:5000/rhel7/pod-infrastructure:latest

推送失敗報錯

Get https://10.0.0.211:5000/v1/_ping: http: server gave HTTP response to HTTPS client

解決辦法

①種方法：vim /etc/sysconfig/docker加入
OPTIONS='--insecure-registry 10.0.0.211:5000' ②種方法
echo '{ "insecure-registries":["10.0.0.211:5000"] }' > /etc/docker/daemon.json
systemctl restart docker.service

四、Master節點從本地倉庫拉取鏡像

docker pull 10.0.0.211:5000/google_containers/kubernetes-dashboard-amd64:latest
docker pull 10.0.0.211:5000/rhel7/pod-infrastructure

查看：

2、部署dashboard

一、編輯dashboard.yaml文件

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
# Keep the name in sync with image version and
# gce/coreos/kube-manifests/addons/dashboard counterparts
  name: kubernetes-dashboard-latest
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
        version: latest
        kubernetes.io/cluster-service: "true"
    spec:
      containers:
      - name: kubernetes-dashboard
        image: 10.0.0.211:5000/google_containers/kubernetes-dashboard-amd64:latest
        resources:
          # keep request = limit to keep this container in guaranteed class
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 100m
            memory: 50Mi
        ports:
        - containerPort: 9090
        args:
         -  --apiserver-host=http://10.0.0.211:8080
        livenessProbe:
          httpGet:
            path: /
            port: 9090
          initialDelaySeconds: 30
          timeoutSeconds: 30

注：Dashboard是在yaml中定義的，要更改dashboard.yaml中對應的「image: gcr.io/google_containers/kubernetes-dashboard-amd64:v1.5.1」爲「image: 10.0.0.211:5000/kubernetes-dashboard-amd64:latest」

 二、編輯dashboardsvc.yaml文件

apiVersion: v1
kind: Service
metadata:
  name: kubernetes-dashboard
  namespace: kube-system
  labels:
    k8s-app: kubernetes-dashboard
    kubernetes.io/cluster-service: "true"
spec:
  selector:
    k8s-app: kubernetes-dashboard
  ports:
  - port: 80
    targetPort: 9090

三、Master節點建立啓動命令

kubectl create -f dashboard.yaml
kubectl create -f dashboardsvc.yaml

四、Master執行命令驗證

[root@k8s-master ~]# kubectl get deployment --all-namespaces
NAMESPACE     NAME                          DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
kube-system   kubernetes-dashboard-latest   1         1         1            1           3h
[root@k8s-master ~]# kubectl get svc  --all-namespaces
NAMESPACE     NAME                   CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
default       kubernetes             10.254.0.1      <none>        443/TCP   2d
kube-system   kubernetes-dashboard   10.254.233.11   <none>        80/TCP    3h
[root@k8s-master ~]# kubectl get pod  -o wide  --all-namespaces
NAMESPACE     NAME                                           READY     STATUS    RESTARTS   AGE       IP           NODE
kube-system   kubernetes-dashboard-latest-1294433048-09p76   1/1       Running   0          3h        172.16.6.2   k8s-node-1

五、瀏覽器訪問

http://10.0.0.211:8080/ui

六、銷燬應用

Master節點執行

kubectl delete deployment kubernetes-dashboard-latest --namespace=kube-system
kubectl delete svc  kubernetes-dashboard --namespace=kube-system

3、部署遇到的問題

問題1：

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "no endpoints available for service \"kubernetes-dashboard\"",
  "reason": "ServiceUnavailable",
  "code": 503
}

解決：須要pause-amd64這個鏡像

docker pull googlecontainer/pause-amd64:3.0
docker tag googlecontainer/pause-amd64:3.0 gcr.io/google_containers/pause-amd64:3.0

kubectl delete -f dashboard.yaml
kubectl delete -f dashboardsvc.yaml

kubectl create -f dashboard.yaml
kubectl create -f dashboardsvc.yaml

詳細查看報錯問題

kubectl describe pod kubernetes-dashboard-latest-bf59c4df4-xcblq --namespace kube-system

問題2：部署完成以後訪問瀏覽器報錯

Error: 'dial tcp 172.16.6.2:9090: getsockopt: connection refused'
Trying to reach: 'http://172.16.6.2:9090/'

解決：iptables攔截

iptables -P FORWARD ACCEPT   或者  echo "net.ipv4.ip_forward = 1" >>/usr/lib/sysctl.d/50-default.conf

 若是永久生效的話，能夠修改docker服務啓動腳本

vim /etc/systemd/system/docker.service #增長一行
[Service]
ExecStartPost=/sbin/iptables -I FORWARD -s 0.0.0.0/0 -j ACCEPT

總結排查方法：

①檢查apiserver的地址設置的是否正確,而後就是flannel是否配置啓動,確保docker0和flannel0處於同一網段

②查看master和nodes上的flannel配置是否一致

# Flanneld configuration options  

# etcd url location.  Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://10.0.0.211:2379"

# etcd config key.  This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/atomic.io/network"

# Any additional options that you want to pass
#FLANNEL_OPTIONS=""

③檢查iptables -L,檢查node節點上的FORWARD 查看轉發是不是drop，若是是drop，則開啓

iptables -P FORWARD ACCEPT 
或者
echo "net.ipv4.ip_forward = 1" >>/usr/lib/sysctl.d/50-default.conf