Kubernetes K8S之Pod生命週期與探針檢測

 

K8S中Pod的生命週期與ExecAction、TCPSocketAction和HTTPGetAction探針檢測html

 

主機配置規劃

服務器名稱(hostname) 系統版本 配置 內網IP 外網IP(模擬)
k8s-master CentOS7.7 2C/4G/20G 172.16.1.110 10.0.0.110
k8s-node01 CentOS7.7 2C/4G/20G 172.16.1.111 10.0.0.111
k8s-node02 CentOS7.7 2C/4G/20G 172.16.1.112 10.0.0.112

 

Pod容器生命週期

 

Pause容器說明

每一個Pod裏運行着一個特殊的被稱之爲Pause的容器,其餘容器則爲業務容器,這些業務容器共享Pause容器的網絡棧和Volume掛載卷,所以他們之間通訊和數據交換更爲高效。在設計時能夠充分利用這一特性,將一組密切相關的服務進程放入同一個Pod中;同一個Pod裏的容器之間僅需經過localhost就能互相通訊。node

kubernetes中的pause容器主要爲每一個業務容器提供如下功能:

PID命名空間:Pod中的不一樣應用程序能夠看到其餘應用程序的進程ID。nginx

網絡命名空間:Pod中的多個容器可以訪問同一個IP和端口範圍。web

IPC命名空間:Pod中的多個容器可以使用SystemV IPC或POSIX消息隊列進行通訊。docker

UTS命名空間:Pod中的多個容器共享一個主機名;Volumes(共享存儲卷)。api

Pod中的各個容器能夠訪問在Pod級別定義的Volumes。tomcat

 

容器探針

探針是由 kubelet 對容器執行的按期診斷。要執行診斷,則需kubelet 調用由容器實現的 Handler。探針有三種類型的處理程序:bash

  • ExecAction:在容器內執行指定命令。若是命令退出時返回碼爲 0 則認爲診斷成功。
  • CPSocketAction:對指定端口上的容器的 IP 地址進行 TCP 檢查。若是端口打開,則診斷被認爲是成功的。
  • HTTPGetAction:對指定的端口和路徑上的容器的 IP 地址執行 HTTP Get 請求。若是響應的狀態碼大於等於200 且小於 400,則診斷被認爲是成功的。

每次探測都將得到如下三種結果之一:服務器

  • 成功:容器經過了診斷。
  • 失敗:容器未經過診斷。
  • 未知:診斷失敗,所以不會採起任何行動。

Kubelet 能夠選擇是否在容器上運行三種探針執行和作出反應:網絡

  • livenessProbe:指示容器是否正在運行。若是存活探測失敗,則 kubelet 會殺死容器,而且容器將受到其重啓策略的影響。若是容器不提供存活探針,則默認狀態爲 Success。
  • readinessProbe:指示容器是否準備好服務請求【對外接受請求訪問】。若是就緒探測失敗,端點控制器將從與 Pod 匹配的全部 Service 的端點中刪除該 Pod 的 IP 地址。初始延遲以前的就緒狀態默認爲 Failure。若是容器不提供就緒探針,則默認狀態爲 Success。
  • startupProbe: 指示容器中的應用是否已經啓動。若是提供了啓動探測(startup probe),則禁用全部其餘探測,直到它成功爲止。若是啓動探測失敗,kubelet 將殺死容器,容器服從其重啓策略進行重啓。若是容器沒有提供啓動探測,則默認狀態爲成功Success。

備註:能夠以Tomcat web服務爲例。

 

容器重啓策略

PodSpec 中有一個 restartPolicy 字段,可能的值爲 Always、OnFailure 和 Never。默認爲 Always。

Always表示一旦無論以何種方式終止運行,kubelet都將重啓;OnFailure表示只有Pod以非0退出碼退出才重啓;Nerver表示再也不重啓該Pod。

restartPolicy 適用於 Pod 中的全部容器。restartPolicy 僅指經過同一節點上的 kubelet 從新啓動容器。失敗的容器由 kubelet 以五分鐘爲上限的指數退避延遲(10秒,20秒,40秒…)從新啓動,並在成功執行十分鐘後重置。如 Pod 文檔中所述,一旦pod綁定到一個節點,Pod 將永遠不會從新綁定到另外一個節點。

 

存活(liveness)和就緒(readiness)探針的使用場景

若是容器中的進程可以在遇到問題或不健康的狀況下自行崩潰,則不必定須要存活探針;kubelet 將根據 Pod 的restartPolicy 自動執行正確的操做。

若是你但願容器在探測失敗時被殺死並從新啓動,那麼請指定一個存活探針,並指定restartPolicy 爲 Always 或 OnFailure。

若是要僅在探測成功時纔開始向 Pod 發送流量,請指定就緒探針。在這種狀況下,就緒探針可能與存活探針相同,可是 spec 中的就緒探針的存在乎味着 Pod 將在沒有接收到任何流量的狀況下啓動,而且只有在探針探測成功後纔開始接收流量。

 

Pod phase(階段)

Pod 的 status 定義在 PodStatus 對象中,其中有一個 phase 字段。

Pod 的運行階段(phase)是 Pod 在其生命週期中的簡單宏觀概述。該階段並非對容器或 Pod 的綜合彙總,也不是爲了作爲綜合狀態機。

Pod 相位的數量和含義是嚴格指定的。除了本文檔中列舉的內容外,不該該再假定 Pod 有其餘的 phase 值。

 

下面是 phase 可能的值:

  • 掛起(Pending):Pod 已被 Kubernetes 系統接受,但有一個或者多個容器鏡像還沒有建立。等待時間包括調度 Pod 的時間和經過網絡下載鏡像的時間,這可能須要花點時間。
  • 運行中(Running):該 Pod 已經綁定到了一個節點上,Pod 中全部的容器都已被建立。至少有一個容器正在運行,或者正處於啓動或重啓狀態。
  • 成功(Succeeded):Pod 中的全部容器都被成功終止,而且不會再重啓。
  • 失敗(Failed):Pod 中的全部容器都已終止了,而且至少有一個容器是由於失敗終止。也就是說,容器以非0狀態退出或者被系統終止。
  • 未知(Unknown):由於某些緣由沒法取得 Pod 的狀態,一般是由於與 Pod 所在主機通訊失敗。

 

檢測探針-就緒檢測

pod yaml腳本

 1 [root@k8s-master lifecycle]# pwd
 2 /root/k8s_practice/lifecycle
 3 [root@k8s-master lifecycle]# cat readinessProbe-httpget.yaml 
 4 apiVersion: v1
 5 kind: Pod
 6 metadata:
 7   name: readiness-httpdget-pod
 8   namespace: default
 9   labels:
10     test: readiness-httpdget
11 spec:
12   containers:
13   - name: readiness-httpget
14     image: registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17
15     imagePullPolicy: IfNotPresent
16     readinessProbe:
17       httpGet:
18         path: /index1.html
19         port: 80
20       initialDelaySeconds: 5  #容器啓動完成後,kubelet在執行第一次探測前應該等待 5 秒。默認是 0 秒,最小值是 021       periodSeconds: 3  #指定 kubelet 每隔 3 秒執行一次存活探測。默認是 10 秒。最小值是 1

 

建立 Pod,並查看pod狀態

1 [root@k8s-master lifecycle]# kubectl apply -f readinessProbe-httpget.yaml 
2 pod/readiness-httpdget-pod created
3 [root@k8s-master lifecycle]# kubectl get pod -n default -o wide
4 NAME                     READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
5 readiness-httpdget-pod   0/1     Running   0          5s    10.244.2.25   k8s-node02   <none>           <none>

 

查看pod詳情

 1 [root@k8s-master lifecycle]# kubectl describe pod readiness-httpdget-pod
 2 Name:         readiness-httpdget-pod
 3 Namespace:    default
 4 Priority:     0
 5 Node:         k8s-node02/172.16.1.112
 6 Start Time:   Sat, 23 May 2020 16:10:04 +0800
 7 Labels:       test=readiness-httpdget
 8 Annotations:  kubectl.kubernetes.io/last-applied-configuration:
 9                 {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"readiness-httpdget"},"name":"readiness-httpdget-pod","names...
10 Status:       Running
11 IP:           10.244.2.25
12 IPs:
13   IP:  10.244.2.25
14 Containers:
15   readiness-httpget:
16     Container ID:   docker://066d66aaef191b1db08e1b3efba6a9be75378d2fe70e99400fc513b91242089c
17 ………………
18     Port:           <none>
19     Host Port:      <none>
20     State:          Running
21       Started:      Sat, 23 May 2020 16:10:05 +0800
22     Ready:          False   ##### 狀態爲False
23     Restart Count:  0
24     Readiness:      http-get http://:80/index1.html delay=5s timeout=1s period=3s #success=1 #failure=3
25     Environment:    <none>
26     Mounts:
27       /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
28 Conditions:
29   Type              Status
30   Initialized       True 
31   Ready             False   ##### 爲False
32   ContainersReady   False   ##### 爲False
33   PodScheduled      True 
34 Volumes:
35   default-token-v48g4:
36     Type:        Secret (a volume populated by a Secret)
37     SecretName:  default-token-v48g4
38     Optional:    false
39 QoS Class:       BestEffort
40 Node-Selectors:  <none>
41 Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
42                  node.kubernetes.io/unreachable:NoExecute for 300s
43 Events:
44   Type     Reason     Age                From                 Message
45   ----     ------     ----               ----                 -------
46   Normal   Scheduled  <unknown>          default-scheduler    Successfully assigned default/readiness-httpdget-pod to k8s-node02
47   Normal   Pulled     49s                kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
48   Normal   Created    49s                kubelet, k8s-node02  Created container readiness-httpget
49   Normal   Started    49s                kubelet, k8s-node02  Started container readiness-httpget
50   Warning  Unhealthy  2s (x15 over 44s)  kubelet, k8s-node02  Readiness probe failed: HTTP probe failed with statuscode: 404

由上可見,容器未就緒。

咱們進入pod的第一個容器,而後建立對應的文件

1 [root@k8s-master lifecycle]# kubectl exec -it readiness-httpdget-pod -c readiness-httpget bash
2 root@readiness-httpdget-pod:/# cd /usr/share/nginx/html
3 root@readiness-httpdget-pod:/usr/share/nginx/html# ls
4 50x.html  index.html
5 root@readiness-httpdget-pod:/usr/share/nginx/html# echo "readiness-httpdget info" > index1.html
6 root@readiness-httpdget-pod:/usr/share/nginx/html# ls
7 50x.html  index.html  index1.html

 

以後看pod狀態與詳情

 1 [root@k8s-master lifecycle]# kubectl get pod -n default -o wide
 2 NAME                     READY   STATUS    RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
 3 readiness-httpdget-pod   1/1     Running   0          2m30s   10.244.2.25   k8s-node02   <none>           <none>
 4 [root@k8s-master lifecycle]# kubectl describe pod readiness-httpdget-pod
 5 Name:         readiness-httpdget-pod
 6 Namespace:    default
 7 Priority:     0
 8 Node:         k8s-node02/172.16.1.112
 9 Start Time:   Sat, 23 May 2020 16:10:04 +0800
10 Labels:       test=readiness-httpdget
11 Annotations:  kubectl.kubernetes.io/last-applied-configuration:
12                 {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"readiness-httpdget"},"name":"readiness-httpdget-pod","names...
13 Status:       Running
14 IP:           10.244.2.25
15 IPs:
16   IP:  10.244.2.25
17 Containers:
18   readiness-httpget:
19     Container ID:   docker://066d66aaef191b1db08e1b3efba6a9be75378d2fe70e99400fc513b91242089c
20 ………………
21     Port:           <none>
22     Host Port:      <none>
23     State:          Running
24       Started:      Sat, 23 May 2020 16:10:05 +0800
25     Ready:          True     ##### 狀態爲True
26     Restart Count:  0
27     Readiness:      http-get http://:80/index1.html delay=5s timeout=1s period=3s #success=1 #failure=3
28     Environment:    <none>
29     Mounts:
30       /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
31 Conditions:
32   Type              Status
33   Initialized       True 
34   Ready             True   ##### 爲True
35   ContainersReady   True   ##### 爲True
36   PodScheduled      True 
37 Volumes:
38   default-token-v48g4:
39     Type:        Secret (a volume populated by a Secret)
40     SecretName:  default-token-v48g4
41     Optional:    false
42 QoS Class:       BestEffort
43 Node-Selectors:  <none>
44 Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
45                  node.kubernetes.io/unreachable:NoExecute for 300s
46 Events:
47   Type     Reason     Age                   From                 Message
48   ----     ------     ----                  ----                 -------
49   Normal   Scheduled  <unknown>             default-scheduler    Successfully assigned default/readiness-httpdget-pod to k8s-node02
50   Normal   Pulled     2m33s                 kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
51   Normal   Created    2m33s                 kubelet, k8s-node02  Created container readiness-httpget
52   Normal   Started    2m33s                 kubelet, k8s-node02  Started container readiness-httpget
53   Warning  Unhealthy  85s (x22 over 2m28s)  kubelet, k8s-node02  Readiness probe failed: HTTP probe failed with statuscode: 404

由上可見,容器已就緒。

 

檢測探針-存活檢測

存活檢測-執行命令

pod yaml腳本

 1 [root@k8s-master lifecycle]# pwd
 2 /root/k8s_practice/lifecycle
 3 [root@k8s-master lifecycle]# cat livenessProbe-exec.yaml 
 4 apiVersion: v1
 5 kind: Pod
 6 metadata:
 7   name: liveness-exec-pod
 8   labels:
 9     test: liveness
10 spec:
11   containers:
12   - name: liveness-exec
13     image: registry.cn-beijing.aliyuncs.com/google_registry/busybox:1.24
14     imagePullPolicy: IfNotPresent
15     args:
16     - /bin/sh
17     - -c
18     - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
19     livenessProbe:
20       exec:
21         command:
22         - cat
23         - /tmp/healthy
24       initialDelaySeconds: 5   # 第一次檢測前等待5秒
25       periodSeconds: 3   # 檢測週期3秒一次

這個容器生命的前 30 秒,/tmp/healthy 文件是存在的。因此在這最開始的 30 秒內,執行命令 cat /tmp/healthy 會返回成功碼。30 秒以後,執行命令 cat /tmp/healthy 就會返回失敗狀態碼。

 

建立 Pod

1 [root@k8s-master lifecycle]# kubectl apply -f livenessProbe-exec.yaml 
2 pod/liveness-exec-pod created

 

在 30 秒內,查看 Pod 的描述:

 1 [root@k8s-master lifecycle]# kubectl get pod -o wide
 2 NAME                READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
 3 liveness-exec-pod   1/1     Running   0          17s   10.244.2.21   k8s-node02   <none>           <none>
 4 [root@k8s-master lifecycle]# kubectl describe pod liveness-exec-pod
 5 Name:         liveness-exec-pod
 6 Namespace:    default
 7 Priority:     0
 8 Node:         k8s-node02/172.16.1.112
 9 ………………
10 Events:
11   Type    Reason     Age   From                 Message
12   ----    ------     ----  ----                 -------
13   Normal  Scheduled  25s   default-scheduler    Successfully assigned default/liveness-exec-pod to k8s-node02
14   Normal  Pulled     24s   kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/busybox:1.24" already present on machine
15   Normal  Created    24s   kubelet, k8s-node02  Created container liveness-exec
16   Normal  Started    24s   kubelet, k8s-node02  Started container liveness-exec

輸出結果顯示:存活探測器成功。

 

35 秒以後,再來看 Pod 的描述:

 1 [root@k8s-master lifecycle]# kubectl get pod -o wide   # 顯示 RESTARTS 的值增長了 1
 2 NAME                READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
 3 liveness-exec-pod   1/1     Running   1          89s   10.244.2.22   k8s-node02   <none>           <none>
 4 [root@k8s-master lifecycle]# kubectl describe pod liveness-exec-pod
 5 ………………
 6 Events:
 7   Type     Reason     Age              From                 Message
 8   ----     ------     ----             ----                 -------
 9   Normal   Scheduled  42s              default-scheduler    Successfully assigned default/liveness-exec-pod to k8s-node02
10   Normal   Pulled     41s              kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/busybox:1.24" already present on machine
11   Normal   Created    41s              kubelet, k8s-node02  Created container liveness-exec
12   Normal   Started    41s              kubelet, k8s-node02  Started container liveness-exec
13   Warning  Unhealthy  2s (x3 over 8s)  kubelet, k8s-node02  Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
14   Normal   Killing    2s               kubelet, k8s-node02  Container liveness-exec failed liveness probe, will be restarted

由上可見,在輸出結果的最下面,有信息顯示存活探測器失敗了,所以這個容器被殺死而且被重建了。

 

存活檢測-HTTP請求

pod yaml腳本

 1 [root@k8s-master lifecycle]# pwd
 2 /root/k8s_practice/lifecycle
 3 [root@k8s-master lifecycle]# cat livenessProbe-httpget.yaml 
 4 apiVersion: v1
 5 kind: Pod
 6 metadata:
 7   name: liveness-httpget-pod
 8   labels:
 9     test: liveness
10 spec:
11   containers:
12   - name: liveness-httpget
13     image: registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17
14     imagePullPolicy: IfNotPresent
15     ports:
16     - name: http
17       containerPort: 80
18     livenessProbe:
19       httpGet:   # 任何大於或等於 200 而且小於 400 的返回碼錶示成功,其它返回碼都表示失敗。
20         path: /index.html
21         port: 80
22         httpHeaders:  #請求中自定義的 HTTP 頭。HTTP 頭字段容許重複。
23         - name: Custom-Header
24           value: Awesome
25       initialDelaySeconds: 5
26       periodSeconds: 3

 

建立 Pod,查看pod狀態

1 [root@k8s-master lifecycle]# kubectl apply -f livenessProbe-httpget.yaml 
2 pod/liveness-httpget-pod created
3 [root@k8s-master lifecycle]# kubectl get pod -n default -o wide
4 NAME                   READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
5 liveness-httpget-pod   1/1     Running   0          3s    10.244.2.27   k8s-node02   <none>           <none>

 

查看pod詳情

 1 [root@k8s-master lifecycle]# kubectl describe pod liveness-httpget-pod
 2 Name:         liveness-httpget-pod
 3 Namespace:    default
 4 Priority:     0
 5 Node:         k8s-node02/172.16.1.112
 6 Start Time:   Sat, 23 May 2020 16:45:25 +0800
 7 Labels:       test=liveness
 8 Annotations:  kubectl.kubernetes.io/last-applied-configuration:
 9                 {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"liveness"},"name":"liveness-httpget-pod","namespace":"defau...
10 Status:       Running
11 IP:           10.244.2.27
12 IPs:
13   IP:  10.244.2.27
14 Containers:
15   liveness-httpget:
16     Container ID:   docker://4b42a351414667000fe94d4f3166d75e72a3401e549fed723126d2297124ea1a
17 ………………
18     Port:           80/TCP
19     Host Port:      8080/TCP
20     State:          Running
21       Started:      Sat, 23 May 2020 16:45:26 +0800
22     Ready:          True
23     Restart Count:  0
24     Liveness:       http-get http://:80/index.html delay=5s timeout=1s period=3s #success=1 #failure=3
25     Environment:    <none>
26     Mounts:
27       /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
28 Conditions:
29   Type              Status
30   Initialized       True 
31   Ready             True 
32   ContainersReady   True 
33   PodScheduled      True 
34 ………………
35 Events:
36   Type    Reason     Age        From                 Message
37   ----    ------     ----       ----                 -------
38   Normal  Scheduled  <unknown>  default-scheduler    Successfully assigned default/liveness-httpget-pod to k8s-node02
39   Normal  Pulled     5m52s      kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
40   Normal  Created    5m52s      kubelet, k8s-node02  Created container liveness-httpget
41   Normal  Started    5m52s      kubelet, k8s-node02  Started container liveness-httpget

由上可見,pod存活檢測正常

 

咱們進入pod的第一個容器,而後刪除對應的文件

1 [root@k8s-master lifecycle]# kubectl exec -it liveness-httpget-pod -c liveness-httpget bash
2 root@liveness-httpget-pod:/# cd /usr/share/nginx/html/
3 root@liveness-httpget-pod:/usr/share/nginx/html# ls
4 50x.html  index.html
5 root@liveness-httpget-pod:/usr/share/nginx/html# rm -f index.html 
6 root@liveness-httpget-pod:/usr/share/nginx/html# ls
7 50x.html

 

再次看pod狀態和詳情,可見Pod的RESTARTS從0變爲了1。

 1 [root@k8s-master lifecycle]# kubectl get pod -n default -o wide   # RESTARTS 從0變爲了1
 2 NAME                   READY   STATUS    RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
 3 liveness-httpget-pod   1/1     Running   1          8m16s   10.244.2.27   k8s-node02   <none>           <none>
 4 [root@k8s-master lifecycle]# kubectl describe pod liveness-httpget-pod
 5 Name:         liveness-httpget-pod
 6 Namespace:    default
 7 Priority:     0
 8 Node:         k8s-node02/172.16.1.112
 9 Start Time:   Sat, 23 May 2020 16:45:25 +0800
10 Labels:       test=liveness
11 Annotations:  kubectl.kubernetes.io/last-applied-configuration:
12                 {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"liveness"},"name":"liveness-httpget-pod","namespace":"defau...
13 Status:       Running
14 IP:           10.244.2.27
15 IPs:
16   IP:  10.244.2.27
17 Containers:
18   liveness-httpget:
19     Container ID:   docker://5d0962d383b1df5e59cd3d1100b259ff0415ac37c8293b17944034f530fb51c8
20 ………………
21     Port:           80/TCP
22     Host Port:      8080/TCP
23     State:          Running
24       Started:      Sat, 23 May 2020 16:53:38 +0800
25     Last State:     Terminated
26       Reason:       Completed
27       Exit Code:    0
28       Started:      Sat, 23 May 2020 16:45:26 +0800
29       Finished:     Sat, 23 May 2020 16:53:38 +0800
30     Ready:          True
31     Restart Count:  1
32     Liveness:       http-get http://:80/index.html delay=5s timeout=1s period=3s #success=1 #failure=3
33     Environment:    <none>
34     Mounts:
35       /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
36 Conditions:
37   Type              Status
38   Initialized       True 
39   Ready             True 
40   ContainersReady   True 
41   PodScheduled      True 
42 Volumes:
43   default-token-v48g4:
44     Type:        Secret (a volume populated by a Secret)
45     SecretName:  default-token-v48g4
46     Optional:    false
47 QoS Class:       BestEffort
48 Node-Selectors:  <none>
49 Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
50                  node.kubernetes.io/unreachable:NoExecute for 300s
51 Events:
52   Type     Reason     Age                 From                 Message
53   ----     ------     ----                ----                 -------
54   Normal   Scheduled  <unknown>           default-scheduler    Successfully assigned default/liveness-httpget-pod to k8s-node02
55   Normal   Pulled     7s (x2 over 8m19s)  kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
56   Normal   Created    7s (x2 over 8m19s)  kubelet, k8s-node02  Created container liveness-httpget
57   Normal   Started    7s (x2 over 8m19s)  kubelet, k8s-node02  Started container liveness-httpget
58   Warning  Unhealthy  7s (x3 over 13s)    kubelet, k8s-node02  Liveness probe failed: HTTP probe failed with statuscode: 404
59   Normal   Killing    7s                  kubelet, k8s-node02  Container liveness-httpget failed liveness probe, will be restarted

由上可見,當liveness-httpget檢測失敗,重建了Pod容器

 

存活檢測-TCP端口

pod yaml腳本

 1 [root@k8s-master lifecycle]# pwd
 2 /root/k8s_practice/lifecycle
 3 [root@k8s-master lifecycle]# cat livenessProbe-tcp.yaml 
 4 apiVersion: v1
 5 kind: Pod
 6 metadata:
 7   name: liveness-tcp-pod
 8   labels:
 9     test: liveness
10 spec:
11   containers:
12   - name: liveness-tcp
13     image: registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17
14     imagePullPolicy: IfNotPresent
15     ports:
16     - name: http
17       containerPort: 80
18     livenessProbe:
19       tcpSocket:
20         port: 80
21       initialDelaySeconds: 5
22       periodSeconds: 3

 

TCP探測正常狀況

建立 Pod,查看pod狀態

1 [root@k8s-master lifecycle]# kubectl apply -f livenessProbe-tcp.yaml
2 pod/liveness-tcp-pod created
3 [root@k8s-master lifecycle]# kubectl get pod -o wide
4 NAME               READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
5 liveness-tcp-pod   1/1     Running   0          50s   10.244.4.23   k8s-node01   <none>           <none>

 

查看pod詳情

 1 [root@k8s-master lifecycle]# kubectl describe pod liveness-tcp-pod
 2 Name:         liveness-tcp-pod
 3 Namespace:    default
 4 Priority:     0
 5 Node:         k8s-node01/172.16.1.111
 6 Start Time:   Sat, 23 May 2020 18:02:46 +0800
 7 Labels:       test=liveness
 8 Annotations:  kubectl.kubernetes.io/last-applied-configuration:
 9                 {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"test":"liveness"},"name":"liveness-tcp-pod","namespace":"default"}...
10 Status:       Running
11 IP:           10.244.4.23
12 IPs:
13   IP:  10.244.4.23
14 Containers:
15   liveness-tcp:
16     Container ID:   docker://4de13e7c2e36c028b2094bf9dcf8e2824bfd15b8c45a0b963e301b91ee1a926d
17 ………………
18     Port:           80/TCP
19     Host Port:      8080/TCP
20     State:          Running
21       Started:      Sat, 23 May 2020 18:03:04 +0800
22     Ready:          True
23     Restart Count:  0
24     Liveness:       tcp-socket :80 delay=5s timeout=1s period=3s #success=1 #failure=3
25     Environment:    <none>
26     Mounts:
27       /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)
28 Conditions:
29   Type              Status
30   Initialized       True 
31   Ready             True 
32   ContainersReady   True 
33   PodScheduled      True 
34 Volumes:
35   default-token-v48g4:
36     Type:        Secret (a volume populated by a Secret)
37     SecretName:  default-token-v48g4
38     Optional:    false
39 QoS Class:       BestEffort
40 Node-Selectors:  <none>
41 Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
42                  node.kubernetes.io/unreachable:NoExecute for 300s
43 Events:
44   Type    Reason     Age        From                 Message
45   ----    ------     ----       ----                 -------
46   Normal  Scheduled  <unknown>  default-scheduler    Successfully assigned default/liveness-tcp-pod to k8s-node01
47   Normal  Pulling    74s        kubelet, k8s-node01  Pulling image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17"
48   Normal  Pulled     58s        kubelet, k8s-node01  Successfully pulled image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17"
49   Normal  Created    57s        kubelet, k8s-node01  Created container liveness-tcp
50   Normal  Started    57s        kubelet, k8s-node01  Started container liveness-tcp

以上是正常狀況,可見存活探測成功。

 

模擬TCP探測失敗狀況

將上面yaml文件中的探測TCP端口進行以下修改:

1 livenessProbe:
2   tcpSocket:
3     port: 8090  # 以前是80

 

刪除以前的pod並從新建立,並過一下子看pod狀態

1 [root@k8s-master lifecycle]# kubectl apply -f livenessProbe-tcp.yaml 
2 pod/liveness-tcp-pod created
3 [root@k8s-master lifecycle]# kubectl get pod -o wide   # 可見RESTARTS變爲了1,再過一下子會變爲2,以後依次疊加
4 NAME               READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
5 liveness-tcp-pod   1/1     Running   1          25s   10.244.2.28   k8s-node02   <none>           <none>

 

pod詳情

 1 [root@k8s-master lifecycle]# kubectl describe pod liveness-tcp-pod
 2 Name:         liveness-tcp-pod
 3 Namespace:    default
 4 Priority:     0
 5 Node:         k8s-node02/172.16.1.112
 6 Start Time:   Sat, 23 May 2020 18:08:32 +0800
 7 Labels:       test=liveness
 8 ………………
 9 Events:
10   Type     Reason     Age                From                 Message
11   ----     ------     ----               ----                 -------
12   Normal   Scheduled  <unknown>          default-scheduler    Successfully assigned default/liveness-tcp-pod to k8s-node02
13   Normal   Pulled     12s (x2 over 29s)  kubelet, k8s-node02  Container image "registry.cn-beijing.aliyuncs.com/google_registry/nginx:1.17" already present on machine
14   Normal   Created    12s (x2 over 29s)  kubelet, k8s-node02  Created container liveness-tcp
15   Normal   Started    12s (x2 over 28s)  kubelet, k8s-node02  Started container liveness-tcp
16   Normal   Killing    12s                kubelet, k8s-node02  Container liveness-tcp failed liveness probe, will be restarted
17   Warning  Unhealthy  0s (x4 over 18s)   kubelet, k8s-node02  Liveness probe failed: dial tcp 10.244.2.28:8090: connect: connection refused

由上可見,liveness-tcp檢測失敗,重建了Pod容器。

 

檢測探針-啓動檢測

有時候,會有一些現有的應用程序在啓動時須要較多的初始化時間【如:Tomcat服務】。這種狀況下,在不影響對觸發這種探測的死鎖的快速響應的狀況下,設置存活探測參數是要有技巧的。

技巧就是使用一個命令來設置啓動探測。針對HTTP 或者 TCP 檢測,能夠經過設置 failureThreshold * periodSeconds 參數來保證有足夠長的時間應對糟糕狀況下的啓動時間。

示例以下:

pod yaml文件

 1 [root@k8s-master lifecycle]# pwd
 2 /root/k8s_practice/lifecycle
 3 [root@k8s-master lifecycle]# cat startupProbe-httpget.yaml
 4 apiVersion: v1
 5 kind: Pod
 6 metadata:
 7   name: startup-pod
 8   labels:
 9     test: startup
10 spec:
11   containers:
12   - name: startup
13     image: registry.cn-beijing.aliyuncs.com/google_registry/tomcat:7.0.94-jdk8-openjdk
14     imagePullPolicy: IfNotPresent
15     ports:
16     - name: web-port
17       containerPort: 8080
18       hostPort: 8080
19     livenessProbe:
20       httpGet:
21         path: /index.jsp
22         port: web-port
23       initialDelaySeconds: 5
24       periodSeconds: 10
25       failureThreshold: 1
26     startupProbe:
27       httpGet:
28         path: /index.jsp
29         port: web-port
30       periodSeconds: 10      #指定 kubelet 每隔 10 秒執行一次存活探測。默認是 10 秒。最小值是 1
31       failureThreshold: 30   #最大的失敗次數

 

啓動pod,並查看狀態

1 [root@k8s-master lifecycle]# kubectl apply -f startupProbe-httpget.yaml 
2 pod/startup-pod created
3 [root@k8s-master lifecycle]# kubectl get pod -o wide
4 NAME          READY   STATUS    RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
5 startup-pod   1/1     Running   0          8m46s   10.244.4.26   k8s-node01   <none>           <none>

 

查看pod詳情

[root@k8s-master ~]# kubectl describe pod startup-pod

有啓動探測,應用程序將會有最多 5 分鐘(30 * 10 = 300s) 的時間來完成它的啓動。一旦啓動探測成功一次,存活探測任務就會接管對容器的探測,對容器死鎖能夠快速響應。 若是啓動探測一直沒有成功,容器會在 300 秒後被殺死,而且根據 restartPolicy 來設置 Pod 狀態。

 

探測器配置詳解

使用以下這些字段能夠精確的控制存活和就緒檢測行爲:

  • initialDelaySeconds:容器啓動後要等待多少秒後存活和就緒探測器才被初始化,默認是 0 秒,最小值是 0。
  • periodSeconds:執行探測的時間間隔(單位是秒)。默認是 10 秒。最小值是 1。
  • timeoutSeconds:探測的超時時間。默認值是 1 秒。最小值是 1。
  • successThreshold:探測器在失敗後,被視爲成功的最小連續成功數。默認值是 1。存活探測的這個值必須是 1。最小值是 1。
  • failureThreshold:當探測失敗時,Kubernetes 的重試次數。存活探測狀況下的放棄就意味着從新啓動容器。就緒探測狀況下的放棄 Pod 會被打上未就緒的標籤。默認值是 3。最小值是 1。

HTTP 探測器能夠在 httpGet 上配置額外的字段:

  • host:鏈接使用的主機名,默認是 Pod 的 IP。也能夠在 HTTP 頭中設置 「Host」 來代替。
  • scheme :用於設置鏈接主機的方式(HTTP 仍是 HTTPS)。默認是 HTTP。
  • path:訪問 HTTP 服務的路徑。
  • httpHeaders:請求中自定義的 HTTP 頭。HTTP 頭字段容許重複。
  • port:訪問容器的端口號或者端口名。若是數字必須在 1 ~ 65535 之間。

 

相關閱讀

一、官網:配置存活、就緒和啓動探測器

二、Kubernetes K8S之Pod 生命週期與init container初始化容器

完畢!

 


———END———
若是以爲不錯就關注下唄 (-^O^-) !

相關文章
相關標籤/搜索