【原創】Pod和容器的LimitRange原理和實踐總結
1、背景介紹
一般狀況下，Pod中的容器能夠無限制的使用節點上的CPU和內存資源，在共享資源和資源有限的狀況下，若不加以限制，某個集羣或命名空間的資源可能會消耗殆盡，致使其節點上優先級低的Pod發生驅逐、從新調度、重啓等問題，所以針對資源限額須要有針對命名空間級別的使用限制，以及針對Pod和容器級別的使用限制，LimitRange就是在後者的背景下誕生的。html
2、應用場景
一、適用對集羣的某個命名空間中的容器設置默認的統一的request和limit值，配合Resource Quotas更好的實現命名空間級別的資源限額。nginx
二、適用對單個Pod的CPU和內存資源限額控制，如實現Pod中全部容器CPUlimit之和不得超過某值。web
三、適用對單個容器的CPU和內存資源限額控制，如實現容器的CPUlimit值不得超過某值。json
3、實踐
1、LimitRange設置默認request和limitapi
適用場景：適用對集羣的某個命名空間中的容器設置默認的統一的request和limit值app
【新建limitrange】curl
LimitRange針對某個namespace的Pod或容器的CPU和內存的默認值、request和limit值實現管控，所以限制的的範圍是在namespace下，在建立LimitRange以前，先建立1個用於測試的namespaceide
kubectl create namespace limitrange-test-ns
[root@k8s-master limitrange]# kubectl get ns |grep limitrange-test-ns limitrange-test-ns Active 78m
namespace建立成功後，在此命名空間下建立limitrange測試
[root@k8s-master limitrange]# ls limitrange.yaml [root@k8s-master limitrange]# cat limitrange.yaml apiVersion: v1 kind: LimitRange metadata: name: mem-limit-range-test namespace: limitrange-test-ns #指定命名空間 spec: limits: - default: memory: 256Mi defaultRequest: memory: 128Mi type: Container
kubectl apply -f limitrange.yaml
[root@k8s-master limitrange]# kubectl describe limitrange mem-limit-range-test -n limitrange-test-ns Name: mem-limit-range-test Namespace: limitrange-test-ns Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- Container memory - - 128Mi 256Mi -
[root@k8s-master limitrange]# kubectl describe ns limitrange-test-ns Name: limitrange-test-ns Labels: <none> Annotations: <none> Status: Active No resource quota. Resource Limits Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- Container memory - - 128Mi 256Mi -
能夠看到已爲limitrange-test-ns的命名空間下全部的容器設置了默認的request值和limit值，接下來看下在容器中是如何生效的ui
【容器limitrange生效驗證】
一、容器未設置limit，且未設置request
[root@k8s-master limitrange]# cat container-limitrange.yaml apiVersion: v1 kind: Pod metadata: name: pod-container-men-limit-test spec: containers: - name: container-limlitrange-test image: nginx:latest imagePullPolicy: IfNotPresent
[root@k8s-master limitrange]# kubectl get pod -o wide -n limitrange-test-ns NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod-container-men-limit-test 1/1 Running 0 28s 10.122.235.216 k8s-master <none> <none> [root@k8s-master limitrange]# curl 10.122.235.216:80 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
能夠看到此Pod已經running，且業務正常，在建立此Pod容器部分並未設置默認值，可是Pod所在的命名空間有配置過默認的limitrange策略，驗證下此策略是否生效到Pod的容器部分
spec: containers: - image: nginx:latest imagePullPolicy: IfNotPresent name: container-limlitrange-test resources: limits: memory: 256Mi requests: memory: 128Mi
總結：容器未設置limit，且未設置request，則容器運行起來後的request和limit值默認與limitrange type爲container設置的request和limit的保持一致。
二、容器中設置limit，可是未設置request
[root@k8s-master limitrange]# cat container-limitrange.yaml apiVersion: v1 kind: Pod metadata: name: pod-container-men-limit-test namespace: limitrange-test-ns spec: containers: - name: container-limlitrange-test image: nginx:latest imagePullPolicy: IfNotPresent resources: limits: memory: "500Mi"
只設置內存的limit值且大於limitrange設置的limit值
[root@k8s-master limitrange]# kubectl apply -f container-limitrange.yaml pod/pod-container-men-limit-test created [root@k8s-master limitrange]# kubectl get pod -n limitrange-test-ns NAME READY STATUS RESTARTS AGE pod-container-men-limit-test 1/1 Running 0 9s
[root@k8s-master limitrange]# kubectl get pod pod-container-men-limit-test -o json -n limitrange-test-ns "spec": { "containers": [ { "image": "nginx:latest", "imagePullPolicy": "IfNotPresent", "name": "container-limlitrange-test", "resources": { "limits": { "memory": "500Mi" }, "requests": { "memory": "500Mi" } }, "terminationMessagePath": "/dev/termination-log", "terminationMessagePolicy": "File"
總結：容器中設置實際limit，可是未設置request，容器運行成功後request=實際limit
三、容器中未設置limit，可是設置request
[root@k8s-master limitrange]# cat container-limitrange.yaml apiVersion: v1 kind: Pod metadata: name: pod-container-men-limit-test namespace: limitrange-test-ns spec: containers: - name: container-limlitrange-test image: nginx:latest imagePullPolicy: IfNotPresent resources: requests: memory: "150Mi"
[root@k8s-master limitrange]# kubectl apply -f container-limitrange.yaml
pod/pod-container-men-limit-test created
[root@k8s-master limitrange]# kubectl get pod pod-container-men-limit-test -o json -n limitrange-test-ns "spec": { "containers": [ { "image": "nginx:latest", "imagePullPolicy": "IfNotPresent", "name": "container-limlitrange-test", "resources": { "limits": { "memory": "256Mi" }, "requests": { "memory": "150Mi" } }, "terminationMessagePath": "/dev/termination-log", "terminationMessagePolicy": "File",
總結：容器中未設置limit，可是設置實際request，則容器運行起來後request=實際request，limit=limitrange.limit
2、Pod和容器的max、min、maxLimitRequestRatio
適用場景：控制容器或Pod的CPU或內存資源使用，如容器的CPU的request不得小於多少，limit不得大於多少，不知足條件則建立失敗，先介紹下參數：
容器部分：
一、defaultRequest：容器默認的request值，若不指定request值，則默認爲此值
二、max：容器的實際設置的limit值應小於等於此值
三、min：容器實際設置的request值應大於等於此值
四、maxLimitRequestRatio：Max Limit/Request Ratio，爲容器CPU或內存的Limit/Request值應小於等於此值
Pod部分：
一、max：Pod中全部容器的實際設置的limit之和值應小於等於此值
二、min：Pod中全部容器實際設置的request值之和應大於等於此值
三、maxLimitRequestRatio：Max Limit/Request Ratio，爲Pod中全部容器CPU或內存的Limit之和/Request之和值應小於等於此值
演示以下：
kubectl create namespace limitrange-02-test-ns
[root@k8s-master limitrange]# cat limitrange.yaml apiVersion: v1 kind: LimitRange metadata: name: limitrange-test namespace: limitrange-02-test-ns spec: limits: - max: cpu: 2000m memory: 1000Mi min: cpu: 200m memory: 6Mi maxLimitRequestRatio: cpu: 3 memory: 2 type: Pod - default: #容器的默認limit值 cpu: 300m memory: 200Mi defaultRequest: #容器的默認request值 cpu: 200m memory: 100Mi max: #容器的limit值不得大於max cpu: 1000m memory: 500Mi min: #容器的request的值不得小於min cpu: 100m memory: 3Mi maxLimitRequestRatio: #容器的limit/request不得大於，如容器的CPU的limit/request不得大於4 cpu: 5 memory: 4 type: Container
kubectl apply -f container-02-limitrange.yaml
[root@k8s-master limitrange]# kubectl describe ns limitrange-02-test-ns Name: limitrange-02-test-ns Labels: <none> Annotations: <none> Status: Active No resource quota. Resource Limits Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- Pod cpu 200m 2 - - 3 Pod memory 6Mi 1000Mi - - 2 Container cpu 100m 1 200m 300m 5 Container memory 3Mi 500Mi 100Mi 200Mi 4
能夠看到設置的Pod和容器設置的max和min已經生效了，接下來驗證下設置的策略是否生效，建立1個Pod的容器的內存limit值大於500Mi
[root@k8s-master limitrange]# cat container-02-limitrange.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-limitrange-test
namespace: limitrange-02-test-ns
spec:
containers:
- name: container-limlitrange-test
image: nginx:latest
imagePullPolicy: IfNotPresent
resources:
requests:
memory: "100Mi"
limits:
memory: "600Mi"
[root@k8s-master limitrange]# kubectl apply -f container-02-limitrange.yaml
Error from server (Forbidden): error when creating "container-02-limitrange.yaml": pods "pod-limitrange-test" is forbidden: [memory max limit to request ratio per Pod is 2, but provided ratio is 6.000000, maximum memory usage per Container is 500Mi, but limit is 600Mi, memory max limit to request ratio per Container is 4, but provided ratio is 6.000000]
提示容器的limit/request值爲6大於4，且limit爲600Mi大於設置最大max值500Mi，所以建立失敗，在Pod只有1個容器的狀況下podlimit/request值爲6大於2，一樣作下request值小於min值的測試
[root@k8s-master limitrange]# cat container-02-limitrange.yaml apiVersion: v1 kind: Pod metadata: name: pod-limitrange-test namespace: limitrange-02-test-ns spec: containers: - name: container-limlitrange-test image: nginx:latest imagePullPolicy: IfNotPresent resources: requests: memory: "1Mi" #小於min值 limits: memory: "200Mi
[root@k8s-master limitrange]# kubectl apply -f container-02-limitrange.yaml Error from server (Forbidden): error when creating "container-02-limitrange.yaml": pods "pod-limitrange-test" is forbidden: [minimum memory usage per Pod is 6Mi, but request is 1048576, memory max limit to request ratio per Pod is 2, but provided ratio is 200.000000, minimum memory usage per Container is 3Mi, but request is 1Mi, memory max limit to request ratio per Container is 4, but provided ratio is 200.000000]
提示容器request的值爲1Mi小於min值3Mi，且limit/request=200大於4，一樣不知足Pod的min爲6和ratio，所以建立失敗。
4、總結
本文描述了LimitRange的誕生背景、適用場景、以及具體如何實現和控制Pod、容器的資源限額，產品設計上主要考慮以下幾點：
一、產品設計上須要方便用戶低門檻快速設置默認request、limit值，配合Resource Quotas提供強大而穩定的K8S原生資源限額能力。
二、max、min、Limit/Request Ratio等功能若是開放給對K8S不瞭解的用戶，門檻較高，若無心配置完成且生效，可能會致使容器的建立異常，且不知如何解決，是否直接開放須要認真考慮。
- 1. kafka原理和實踐（一）原理
- 2. Docker容器的原理與實踐 （下）
- 3. Docker容器的原理與實踐（上）
- 4. MyBatis的原理與實踐的總結
- 5. 瀏覽器工做原理和實踐
- 6. 監控神器-Prometheus(原理和實踐)
- 7. keepalived原理和實踐
- 8. HTTP3原理和實踐
- 9. Yii rbac原理和實踐
- 10. Spring中的IoC容器原理總結
- 更多相關文章...
- • MyBatis的工作原理 - MyBatis教程
- • 使用Redis和Lua的原子性實現搶紅包功能 - 紅包項目實戰
- • TiDB 在摩拜單車在線數據業務的應用和實踐
- • Docker容器實戰(七) - 容器眼光下的文件系統
-
每一个你不满意的现在，都有一个你没有努力的曾经。
- 1. 外部其他進程嵌入到qt FindWindow獲得窗口句柄 報錯無法鏈接的外部符號 [email protected] 無法被([email protected]@[email protected]@@引用
- 2. UVa 11524 - InCircle
- 3. The Monocycle（bfs）
- 4. VEC-C滑窗
- 5. 堆排序的應用-TOPK問題
- 6. 實例演示ElasticSearch索引查詢term,match,match_phase,query_string之間的區別
- 7. 數學基礎知識 集合
- 8. amazeUI 復擇框問題解決
- 9. 揹包問題理解
- 10. 算數平均-幾何平均不等式的證明,從麥克勞林到柯西
- 1. kafka原理和實踐（一）原理
- 2. Docker容器的原理與實踐 （下）
- 3. Docker容器的原理與實踐（上）
- 4. MyBatis的原理與實踐的總結
- 5. 瀏覽器工做原理和實踐
- 6. 監控神器-Prometheus(原理和實踐)
- 7. keepalived原理和實踐
- 8. HTTP3原理和實踐
- 9. Yii rbac原理和實踐
- 10. Spring中的IoC容器原理總結