一,理解Kubedns原理git
通俗理解,首先明白k8s dns是爲服務的發現而生,即service的發現,爲了可以讓其它服務可以直接經過service 名字找到它們,因而就須要dns將service名轉換爲它的VIP, 那麼service的變化如何知道? 如何知道目前有哪些service 及知道他們的vip呢,service的增長減小又如何知道了, github
因此要有一個組件(1.3中是Kube2sky,1.3後是kubedns)來時刻監控它的變化,監控到了什麼就把它記錄下來,記錄的是service和ip之間的映射關係,稱爲DNS解析記錄,可是記錄在某個位置呢,監控的組件不一樣,記錄的位置也不一樣, Kube2sky是記錄到etcd中,kubedns是記錄到哪了呢, 記錄在內存當中,使用樹形結構在內存中保存監控到的記錄。express
因爲一切對集羣的操做都是經過其API來的,因此它監控service資源變化也是藉助於k8s APIapache
這個組件僅僅是記錄下來之後,並不提供查詢。因此查詢就是另一個組件的做用了,1.3以前是由Skydns來查詢的, 全部服務(pod)都找skydns查詢,1.3之後都是由dnsmasq查詢的,它們從前面記錄的地方查詢相關service的解析記錄,之因此用dnsmasq來查詢,由於它提供DNS查詢緩存,在內存中完成查詢,查詢速度很是快!api
兩個組件一個負責監控並記錄,另一個負責幫其它服務作查詢, 兩個組件必需要同時運行,若是其中一個出問題了,那麼dns系統將沒法解析,因而須要有另外一個組件專門來監控它們的健康狀態,這個組件就是exechealthz,Exechealthz是兩個版本中惟一保留的容器,依然提供健康檢查。緩存
在K8s中,這三個組件都是用容器運行的,並且是在同一個pod中,bash
官方理解,上圖:數據結構
● dnsmasq簡介app
Dnsmasq是一款小巧的DNS配置工具less
在kube-dns插件中的做用:
● 經過kubedns容器獲取DNS規則,在集羣中提供DNS查詢服務,至關於dns的server端。
● 提供DNS緩存,提升查詢性能
● 下降kubedns容器的壓力、提升穩定性
● Dockerfile在GitHub上Kubernetes組織的contrib倉庫中,位於dnsmasq目錄下
● exechealthz簡介
● 在kube-dns插件中提供健康檢查功能
● 源碼一樣在contrib倉庫中,位於exec-healthz目錄下。
● 新版中會對兩個容器都進行健康檢查,更加完善。
● 總結
kube-dns插件的三個容器的功能以下:
● kubedns容器
● 監視k8s Service資源並更新DNS記錄
● 替換etcd,使用TreeCache數據結構保存DNS記錄並實現SkyDNS的Backend接口
● 接入SkyDNS,對dnsmasq提供DNS查詢服務
● dnsmasq容器
● 對集羣提供DNS查詢服務
● 設置kubedns爲upstream
● 提供DNS緩存,下降kubedns負載,提升性能
● exechealthz容器
● 按期檢查kubedns和dnsmasq的健康狀態
● 爲k8s活性檢測提供HTTP API
2、部署kubedns
1,上官方網址下載須要的yaml部署文件:https://github.com/kubernetes/kubernetes/tree/release-1.8/cluster/addons/dns
$ ls *.yaml *.base kubedns-cm.yaml kubedns-sa.yaml kubedns-controller.yaml.base kubedns-svc.yaml.base
2,主要是修改這四個部署文件
cat kubedns-cm.yaml #此文件不需修改 apiVersion: v1 kind: ConfigMap metadata: name: kube-dns namespace: kube-system labels: addonmanager.kubernetes.io/mode: EnsureExists
# cat kubedns-sa.yaml #此文件不需修改 apiVersion: v1 kind: ServiceAccount metadata: name: kube-dns namespace: kube-system labels: kubernetes.io/cluster-service: "true" addonmanager.kubernetes.io/mode: Reconcile
接下來主要修改的是 kubedns-controller.yaml 和 kubedns-svc.yaml
主要diff來比較下原文件與修改過的文件區別:
diff kubedns-controller.yaml.base /root/kubernetes/k8s-deploy/mainifest/dns/kubedns-controller.yaml 58c58 < image: gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.5 --- > image: gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.4 88c88 < - --domain=$DNS_DOMAIN. --- > - --domain=cluster.local. 109c109 < image: gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.5 --- > image: gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.4 128c128 < - --server=/$DNS_DOMAIN/127.0.0.1#10053 --- > - --server=/cluster.local/127.0.0.1#10053 147c147 < image: gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.5 --- > image: gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.4 160,161c160,161 < - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.$DNS_DOMAIN,5,A < - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.$DNS_DOMAIN,5,A --- > - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,A > - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,A 無非就是將有 k8s-dns-sidecar-amd64:1.14.5 的改成 k8s-dns-sidecar-amd64:1.14.4,將變量集羣域名的環境變量 $DNS_DOMAIN 替換咱們以前設置好的域名,如cluster.local。
修改事後的完整文件內容以下:
# cat kubedns-controller.yaml # Copyright 2016 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # Should keep target in cluster/addons/dns-horizontal-autoscaler/dns-horizontal-autoscaler.yaml # in sync with this file. # __MACHINE_GENERATED_WARNING__ apiVersion: extensions/v1beta1 kind: Deployment metadata: name: kube-dns namespace: kube-system labels: k8s-app: kube-dns kubernetes.io/cluster-service: "true" addonmanager.kubernetes.io/mode: Reconcile spec: # replicas: not specified here: # 1. In order to make Addon Manager do not reconcile this replicas parameter. # 2. Default is 1. # 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on. strategy: rollingUpdate: maxSurge: 10% maxUnavailable: 0 selector: matchLabels: k8s-app: kube-dns template: metadata: labels: k8s-app: kube-dns annotations: scheduler.alpha.kubernetes.io/critical-pod: '' spec: tolerations: - key: "CriticalAddonsOnly" operator: "Exists" volumes: - name: kube-dns-config configMap: name: kube-dns optional: true containers: - name: kubedns image: gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.4 resources: # TODO: Set memory limits when we've profiled the container for large # clusters, then set request = limit to keep this container in # guaranteed class. Currently, this container falls into the # "burstable" category so the kubelet doesn't backoff from restarting it. limits: memory: 170Mi requests: cpu: 100m memory: 70Mi livenessProbe: httpGet: path: /healthcheck/kubedns port: 10054 scheme: HTTP initialDelaySeconds: 60 timeoutSeconds: 5 successThreshold: 1 failureThreshold: 5 readinessProbe: httpGet: path: /readiness port: 8081 scheme: HTTP # we poll on pod startup for the Kubernetes master service and # only setup the /readiness HTTP server once that's available. initialDelaySeconds: 3 timeoutSeconds: 5 args: - --domain=cluster.local. - --dns-port=10053 - --config-dir=/kube-dns-config - --v=2 env: - name: PROMETHEUS_PORT value: "10055" ports: - containerPort: 10053 name: dns-local protocol: UDP - containerPort: 10053 name: dns-tcp-local protocol: TCP - containerPort: 10055 name: metrics protocol: TCP volumeMounts: - name: kube-dns-config mountPath: /kube-dns-config - name: dnsmasq image: gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.4 livenessProbe: httpGet: path: /healthcheck/dnsmasq port: 10054 scheme: HTTP initialDelaySeconds: 60 timeoutSeconds: 5 successThreshold: 1 failureThreshold: 5 args: - -v=2 - -logtostderr - -configDir=/etc/k8s/dns/dnsmasq-nanny - -restartDnsmasq=true - -- - -k - --cache-size=1000 - --log-facility=- - --server=/cluster.local/127.0.0.1#10053 - --server=/in-addr.arpa/127.0.0.1#10053 - --server=/ip6.arpa/127.0.0.1#10053 ports: - containerPort: 53 name: dns protocol: UDP - containerPort: 53 name: dns-tcp protocol: TCP # see: https://github.com/kubernetes/kubernetes/issues/29055 for details resources: requests: cpu: 150m memory: 20Mi volumeMounts: - name: kube-dns-config mountPath: /etc/k8s/dns/dnsmasq-nanny - name: sidecar image: gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.4 livenessProbe: httpGet: path: /metrics port: 10054 scheme: HTTP initialDelaySeconds: 60 timeoutSeconds: 5 successThreshold: 1 failureThreshold: 5 args: - --v=2 - --logtostderr - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,A - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,A ports: - containerPort: 10054 name: metrics protocol: TCP resources: requests: memory: 20Mi cpu: 10m dnsPolicy: Default # Don't use cluster DNS. serviceAccountName: kube-dns
kubedns-controller 運行了 三個容器:kubedns dnsmasq sidecar,sidecar 是一個監控健康模塊,同時向外暴露metrics 記錄。
cat kubedns-svc.yaml # Copyright 2016 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # __MACHINE_GENERATED_WARNING__ apiVersion: v1 kind: Service metadata: name: kube-dns namespace: kube-system labels: k8s-app: kube-dns kubernetes.io/cluster-service: "true" addonmanager.kubernetes.io/mode: Reconcile kubernetes.io/name: "KubeDNS" spec: selector: k8s-app: kube-dns clusterIP: 10.254.0.2 #修改成咱們設定的cluster的IP,其它默認。 ports: - name: dns port: 53 protocol: UDP - name: dns-tcp port: 53 protocol: TCP
部署文件設置好後,一塊兒來建立:
# kubectl create -f kubedns-cm.yaml -f kubedns-sa.yaml -f kubedns-controller.yaml -f kubedns-svc.yaml
查看建立後的pod,svc:
# kubectl get pod,svc -n kube-system NAME READY STATUS RESTARTS AGE po/kube-dns-7797cb8758-dlhqs 3/3 Running 0 3h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/kube-dns ClusterIP 10.254.0.2 <none> 53/UDP,53/TCP 3h
三,驗證kubedns功能
若是咱們此前已經有了部署了許多pod和服務,而且在Kubelet 啓動配置文件加入了 --cluster-dns=10.254.0.2 --cluster-domain=cluster.local 參數,則能夠進入某個pod中的容器內查看其/etc/resolv.conf 文件
root@jekins:~# cat /etc/resolv.conf nameserver 10.254.0.2 search default.svc.cluster.local svc.cluster.local cluster.local options ndots:5 如上所示,根據kubelet的啓動參數,kubelet會在每一個pod中設置DNS域名解析文件/etc/resolv.conf,加入了nameser和search搜索域。最後應用程序就可以像訪問網站同樣,僅僅經過服務的名字就能訪問到服務了。 root@jek:~# ping jenkinsservice PING jenkinsservice.default.svc.cluster.local (10.254.145.97) 56(84) bytes of data. 能夠看到已經解析出來了
一般能夠啓動一個帶有nslookup工具的pod,好比busybox,鏡像從gcr.io/google_containers/busybox下載,來驗證DNS服務是否可以正常工做。
四,kubedns自動水平擴展
在上面部署的Kubedns,它只有一個pod,若是這個pod沒有了 又或一個pod由於解析量大沒法撐起解析時,會致使dns沒法工做,因此官方給了一個自動水平伸縮的方案,根據當前pod的負載來決定是否增長或減小pod數量,
稱爲,dns-horizontal-autoscaler ,部署文件位於kubernetes/cluster/addons/dns-horizontal-autoscale,下載下來可直接部署。
$ /data/k8sdns# ls dns-horizontal-autoscaler* dns-horizontal-autoscaler-rbac.yaml dns-horizontal-autoscaler.yaml
dns-horizontal-autoscaler-rbac.yaml文件解析:
實際它就建立了三個資源:ServiceAccount、ClusterRole、ClusterRoleBinding ,建立賬戶,建立角色,賦予權限,將賬戶綁定到角色上面。
而dns-horizontal-autoscaler.yaml 文件它就建立一個Deployment資源,裏面跑了一個autoscaler的容器,直接部署便可。
如今來部署: 先部署RBAC文件,再部署服務文件:
# kubectl create -f dns-horizontal-autoscaler-rbac.yaml serviceaccount "kube-dns-autoscaler" created clusterrole "system:kube-dns-autoscaler" created clusterrolebinding "system:kube-dns-autoscaler" created # kubectl create -f dns-horizontal-autoscaler.yaml deployment "kube-dns-autoscaler" created
查看建立狀態 ,而且能夠看到在建立以前dns只有一個pod:
# kubectl get pod -n kube-system NAME READY STATUS RESTARTS AGE kube-dns-7797cb8758-dlhqs 3/3 Running 3 19h kube-dns-autoscaler-7db47cb9b7-gq5wk 0/1 ContainerCreating 0 21s
過幾分鐘後查看,就會自動建立出來一個dns pod:
# kubectl get pod -n kube-system NAME READY STATUS RESTARTS AGE kube-dns-7797cb8758-dlhqs 3/3 Running 3 19h kube-dns-7797cb8758-kwmqr 0/3 ContainerCreating 0 48s kube-dns-autoscaler-7db47cb9b7-gq5wk 1/1 Running 0 1m
至此,一個完整的kubedns部署完成了。