Kubernetes資源管理

時間 2019-11-18

標籤 kubernetes 資源管理简体版

原文原文鏈接

目錄貼：Kubernetes學習系列

一、資源模型

　　虛擬化技術是雲計算平臺的基礎，其目標是對計算資源進行整合或劃分，這是雲計算管理平臺中的關鍵技術。虛擬化技術爲雲計算管理乎臺的資源管理提供了資源調配上的靈活性，從而使得雲計算管理平臺能夠經過虛擬化層整合或劃分計算資源。html

　　相比於虛擬機，新出現的容器技術使用了一系列的系統級別的機制，諸如利用Linux Namespace進行空間隔離，經過文件系統的掛載點決定容器能夠訪問哪些文件，經過Cgroup肯定每一個容器能夠利用多少資源。此外，容器之間共享同一個系統內核，這樣當同一個內核被多個容器使用時，內存的使用效率會獲得提高。node

　　容器和虛擬機兩大虛擬化技術，雖然實現方式徹底不一樣，可是它們的資源需求和模型實際上是相似的。容器像虛擬機同樣須要內存、CPU、硬盤空間和網絡帶寬，宿主機系統能夠將虛擬機和容器都視做一個總體，爲這個總體分配其所需的資源，並進行管理。固然，虛擬機提供了專用操做系統的安全性和更牢固的邏輯邊界，而容器在資源邊界上比較鬆散，這帶來了靈活性以及不肯定性。docker

　　Kubernetes是一個容器集羣管理平臺，Kubernetes須要統計總體平臺的資源使用狀況，合理地將資源分配給容器使用，而且要保證容器生命週期內有足夠的資源來保證其運行。更進一步，若是資源發放是獨佔的，即資源已發放給了個容器，一樣的資源不會發放給另一個容器，對於空閒的容器來講佔用着沒有使用的資源好比CPU是很是浪費的，Kubernetes須要考慮如何在優先度和公平性的前提下提升資源的利用率。json

二、資源請求與限制

　　建立Pod的時候，能夠指定計算資源（目前支持的資源類型有CPU和內存），即指定每一個容器的資源請求（Request）和資源限制（Limit），資源請求是容器所需的最小資源需求，資源限制則是容器不能超過的資源上限。它們的大小關係是：api

0<=request<=limit<=infinity安全

　　Pod的資源請求就是Pod中容器資源請求之和。Kubernetes在調度Pod時，會根據Node中的資源總量（經過cAdvisor接口得到），以及該Node上已使用的計算資源，來判斷該Node是否知足需求。網絡

　　資源請求可以保證Pod有足夠的資源來運行，而資源限制則是防止某個Pod無限制地使用資源，致使其餘Pod崩潰。特別是在公有云場景，每每會有惡意軟件經過搶佔內存來攻擊平臺。架構

　　原理：Docker 經過使用Linux Cgroup來實現對容器資源的控制，具體到啓動參數上是--memory和--cpu-shares。Kubernetes中是經過控制這兩個參數來實現對容器資源的控制。app

如下給出某個Pod申請內存及CPU的示例：tcp

[root@k8s-master demon2]# cat test-limit.yaml
apiVersion: v1
kind: Pod
metadata:
  labels:
    name: test-limit
    role: master
  name: test-limit
spec:
  containers:
    - name: test-limit
      image: registry:5000/back_demon:1.0
      resources:
        requests:
          memory: "256Mi"
          cpu: "500m"
        limits:
          memory: "512Mi"
          cpu: "1000m"
      command:
      - /run.sh

待Pod調度到具體某個機器上以後，在該機器上查詢對應容器的詳情，以下：

[root@k8s-node-4 home]# docker inspect 1fdbd6f1b39b
[
    {
        "Id": "1fdbd6f1b39b561d09084adafb382b721959e5edd0ee9538472313ed0a39162a",
        "Created": "2017-03-20T05:40:30.756006226Z",
        "Path": "/run.sh",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 18603,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2017-03-20T05:40:31.113657323Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:9369911131d30b12759074e5b72356345446996bf6044950c2def787471e9b4c",
        "ResolvConfPath": "/var/lib/docker/containers/8fdb38a4d0074b07ff2f07e21fd8602fdbf4267eafc76179e931d4f5d9265940/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/8fdb38a4d0074b07ff2f07e21fd8602fdbf4267eafc76179e931d4f5d9265940/hostname",
        "HostsPath": "/var/lib/kubelet/pods/ba75e7a9-0d2f-11e7-b3d5-fa163ebba51b/etc-hosts",
        "LogPath": "/var/lib/docker/containers/1fdbd6f1b39b561d09084adafb382b721959e5edd0ee9538472313ed0a39162a/1fdbd6f1b39b561d09084adafb382b721959e5edd0ee9538472313ed0a39162a-json.log",
        "Name": "/k8s_test-limit.79cbd53f_test-limit_default_ba75e7a9-0d2f-11e7-b3d5-fa163ebba51b_efc94078",
        "RestartCount": 0,
        "Driver": "devicemapper",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/var/lib/kubelet/pods/ba75e7a9-0d2f-11e7-b3d5-fa163ebba51b/etc-hosts:/etc/hosts:Z",
                "/var/lib/kubelet/pods/ba75e7a9-0d2f-11e7-b3d5-fa163ebba51b/containers/test-limit/efc94078:/dev/termination-log:Z"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "container:8fdb38a4d0074b07ff2f07e21fd8602fdbf4267eafc76179e931d4f5d9265940",
            "PortBindings": null,
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "container:8fdb38a4d0074b07ff2f07e21fd8602fdbf4267eafc76179e931d4f5d9265940",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 984,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "seccomp=unconfined"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "docker-runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 1024,
            "Memory": 536870912,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 100000,
            "CpuQuota": 200000,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": -1,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Name": "devicemapper",
            "Data": {
                "DeviceId": "210",
                "DeviceName": "docker-253:0-100693626-cb08877222f483f043fc45c5c4b024de8da9c393c3c06c6252d3c59d330dd4d4",
                "DeviceSize": "10737418240"
            }
        },
        "Mounts": [
            {
                "Source": "/var/lib/kubelet/pods/ba75e7a9-0d2f-11e7-b3d5-fa163ebba51b/etc-hosts",
                "Destination": "/etc/hosts",
                "Mode": "Z",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Source": "/var/lib/kubelet/pods/ba75e7a9-0d2f-11e7-b3d5-fa163ebba51b/containers/test-limit/efc94078",
                "Destination": "/dev/termination-log",
                "Mode": "Z",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "test-limit",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "222/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "KUBERNETES_PORT_443_TCP_PROTO=tcp",
                "KUBERNETES_PORT_443_TCP_PORT=443",
                "FRONTEND_SERVICE_PORT=tcp://10.254.232.119:8080",
                "REDIS_SERVICE_SERVICE_PORT=6379",
                "REDIS_SERVICE_PORT_6379_TCP_ADDR=10.254.71.136",
                "KUBERNETES_SERVICE_PORT_HTTPS=443",
                "KUBERNETES_SERVICE_HOST=10.254.0.1",
                "KUBERNETES_PORT_443_TCP=tcp://10.254.0.1:443",
                "BACK_SERVICE_PORT_8080_TCP_PORT=8080",
                "FRONTEND_SERVICE_PORT_8080_TCP=tcp://10.254.232.119:8080",
                "FRONTEND_SERVICE_PORT_8080_TCP_ADDR=10.254.232.119",
                "REDIS_SERVICE_PORT_6379_TCP=tcp://10.254.71.136:6379",
                "REDIS_MASTER_PORT_6379_TCP_PORT=6379",
                "FRONTEND_PORT_80_TCP_PORT=80",
                "REDIS_MASTER_PORT_6379_TCP_ADDR=10.254.132.210",
                "REDIS_SLAVE_PORT_6379_TCP=tcp://10.254.104.23:6379",
                "REDIS_SLAVE_PORT_6379_TCP_PORT=6379",
                "BACK_SERVICE_SERVICE_HOST=10.254.246.51",
                "BACK_SERVICE_PORT=tcp://10.254.246.51:8080",
                "BACK_SERVICE_PORT_8080_TCP_PROTO=tcp",
                "FRONTEND_PORT=tcp://10.254.93.91:80",
                "REDIS_MASTER_SERVICE_HOST=10.254.132.210",
                "REDIS_MASTER_PORT_6379_TCP_PROTO=tcp",
                "KUBERNETES_SERVICE_PORT=443",
                "FRONTEND_SERVICE_PORT_8080_TCP_PROTO=tcp",
                "REDIS_MASTER_SERVICE_PORT=6379",
                "REDIS_SLAVE_SERVICE_HOST=10.254.104.23",
                "REDIS_SLAVE_PORT=tcp://10.254.104.23:6379",
                "REDIS_SERVICE_PORT_6379_TCP_PROTO=tcp",
                "REDIS_MASTER_PORT=tcp://10.254.132.210:6379",
                "KUBERNETES_PORT_443_TCP_ADDR=10.254.0.1",
                "BACK_SERVICE_SERVICE_PORT=8080",
                "FRONTEND_SERVICE_HOST=10.254.93.91",
                "FRONTEND_SERVICE_SERVICE_PORT=8080",
                "REDIS_SERVICE_SERVICE_HOST=10.254.71.136",
                "REDIS_SERVICE_PORT=tcp://10.254.71.136:6379",
                "BACK_SERVICE_PORT_8080_TCP=tcp://10.254.246.51:8080",
                "REDIS_SLAVE_SERVICE_PORT=6379",
                "REDIS_SLAVE_PORT_6379_TCP_PROTO=tcp",
                "REDIS_SLAVE_PORT_6379_TCP_ADDR=10.254.104.23",
                "KUBERNETES_PORT=tcp://10.254.0.1:443",
                "BACK_SERVICE_PORT_8080_TCP_ADDR=10.254.246.51",
                "FRONTEND_PORT_80_TCP=tcp://10.254.93.91:80",
                "FRONTEND_PORT_80_TCP_ADDR=10.254.93.91",
                "FRONTEND_SERVICE_PORT_8080_TCP_PORT=8080",
                "REDIS_MASTER_PORT_6379_TCP=tcp://10.254.132.210:6379",
                "FRONTEND_PORT_80_TCP_PROTO=tcp",
                "FRONTEND_SERVICE_SERVICE_HOST=10.254.232.119",
                "REDIS_SERVICE_PORT_6379_TCP_PORT=6379"
            ],
            "Cmd": null,
            "Image": "registry:5000/back_demon:1.0",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "/run.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "io.kubernetes.container.hash": "79cbd53f",
                "io.kubernetes.container.name": "test-limit",
                "io.kubernetes.container.restartCount": "0",
                "io.kubernetes.container.terminationMessagePath": "/dev/termination-log",
                "io.kubernetes.pod.name": "test-limit",
                "io.kubernetes.pod.namespace": "default",
                "io.kubernetes.pod.terminationGracePeriod": "30",
                "io.kubernetes.pod.uid": "ba75e7a9-0d2f-11e7-b3d5-fa163ebba51b"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": null,
            "SandboxKey": "",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": null
        }
    }
]

三、LimitRange

　　LimitRange設計的初衷是爲了知足如下場景：

　　　　可以約束租戶的資源需求。

　　　　可以約束容器的資源請求範圍。

　　　　可以約束Pod的資源請求範圍。

　　　　可以指定容器的默認資源限制。

　　　　可以指定Pod的默認資源限制。

　　　　可以約束資源請求和限制之間的比例。

四、Resource Quota

　　Kubernetes是一個多租戶架構，當多用戶或者團隊共享一個Kubernetes系統的時候，系統管理員須要防止租戶的資源搶佔，定義好資源分配策略。好比Kubernetes系統共有20 核CPU和32GB內存，分配給A租戶5核CPU和16GB，分配給B租戶5核CPU 和8GB，預留10核CPU和8GB內存。這樣，租戶中所使用的CPU和內存的總和不能超過指定的資源配額，促使其更合理地使用資源。

　　Kubernetes提供API對象Resource Quota（資源配額）來實現資源配額，Resource Quota不只能夠做用於CPU和內存，另外還能夠限制好比建立Pod的總數目、Service總數目、RC總數目等。

　　默認狀況下，Namespace是沒有Resource Quota的，須要另外建立Resource Quota。一旦Namespace中有了Resource Quota，那麼建立Pod的時候就必須制定資源請求，不然Pod就會建立失敗。