Kubernetes -- Horizontal Pod Autoscaler

時間 2019-11-12

標籤 kubernetes horizontal pod autoscaler 简体版

原文原文鏈接

前言

在kubernetes中，咱們使用pod對外提供服務。這時候，咱們須要如下兩種情形須要關注：php

pod由於不明緣由掛掉，致使服務不可用css
Pod在高負荷的狀況下，不能支撐咱們的服務node

若是咱們人工監控pods，人工進行調整副本那麼這個工做量無疑是巨大的，但kubernetes已經有了相應的機制來應對了。docker

那麼今天就來介紹一下在k8s 1.6中的彈性伸縮的實施shell

k8s是kubernetes的官方簡稱
HPA全稱Horizontal Pod Autoscaler數據庫

HPA的原理

Kubernetes有一個HPA(Horizontal Pod Autoscaler)的資源，能夠實現基於CPU使用率的Pod自動伸縮的功能。HPA基於Master Node上的kube-controller-manager服務啓動參數–horizontal-pod-autoscaler-sync-period定義的時長(默認爲30秒)，週期性的檢測Pod的CPU使用率(須要事先安裝heapster)。若是須要設置–horizontal-pod-autoscaler-sync-period能夠在Master Node上的/etc/default/kube-controller-manager中修改。vim

安裝Heapster

K8S從1.8版本開始，CPU、內存等資源的metrics信息能夠經過 Metrics API來獲取，用戶能夠直接獲取這些metrics信息（例如經過執行kubect top命令），HPA使用這些metics信息來實現動態伸縮，可是在以前咱們使用Heapster來收集節點的相關數據api

導入相關鏡像

咱們在實施的時候通常會建立/data目錄，把全部的deployment放在此目錄下，所以在k8s master建立kube-system目錄ruby

[root@master data]# mkdir kube-system

上傳相鏡像，並導入

# 導入heasper [root@master kube-system]# docker load < heapster_3.tar 38ac8d0f5bb3: Loading layer [==================================================>] 1.312MB/1.312MB 388f58c4d5b0: Loading layer [==================================================>] 99.87MB/99.87MB c6772246bc46: Loading layer [==================================================>] 281.1kB/281.1kB Loaded image: registry.cn-hangzhou.aliyuncs.com/lczean/heapster-amd64-v1.3.0-beta.1:v1.3.0-beta.1 # 導入influxdb數據庫 [root@master kube-system]# docker load < influxdb13.tar 7da815924651: Loading layer [==================================================>] 10.48MB/10.48MB 2d447b9e914f: Loading layer [==================================================>] 5.12kB/5.12kB Loaded image: registry.cn-hangzhou.aliyuncs.com/golden/heapster-influxdb-amd64:latest

查看導入imagesapp

[root@master kube-system]# docker images |grep heapster registry.cn-hangzhou.aliyuncs.com/lczean/heapster-amd64-v1.3.0-beta.1 v1.3.0-beta.1 6393b81e2220 17 months ago 101MB registry.cn-hangzhou.aliyuncs.com/golden/heapster-influxdb-amd64 latest d3fccbedd180 22 months ago 11.6MB

修改images tag以便咱們能夠導入到私有registry中

[root@master kube-system]# docker tag registry.cn-hangzhou.aliyuncs.com/lczean/heapster-amd64-v1.3.0-beta.1:v1.3.0-beta.1 registry.k8s.osc:5000/heapster:v1.3.0 [root@master kube-system]# docker tag registry.cn-hangzhou.aliyuncs.com/golden/heapster-influxdb-amd64 registry.k8s.osc:5000/heapster-influxdb # 查看修改後的images [root@master kube-system]# docker images |grep heapster registry.cn-hangzhou.aliyuncs.com/lczean/heapster-amd64-v1.3.0-beta.1 v1.3.0-beta.1 6393b81e2220 17 months ago 101MB registry.k8s.osc:5000/heapster v1.3.0 6393b81e2220 17 months ago 101MB registry.cn-hangzhou.aliyuncs.com/golden/heapster-influxdb-amd64 latest d3fccbedd180 22 months ago 11.6MB registry.k8s.osc:5000/heapster-influxdb latest d3fccbedd180 22 months ago 11.6MB

推送到私有倉庫

[root@master kube-system]# docker push registry.k8s.osc:5000/heapster:v1.3.0
The push refers to repository [registry.k8s.osc:5000/heapster]
c6772246bc46: Pushed 
388f58c4d5b0: Pushed 
38ac8d0f5bb3: Pushed 
v1.3.0: digest: sha256:e23b30d2e131e042eec9b5fdc30af905b63e454d140dc335246e74a4e8b4c857 size: 949
[root@master kube-system]# docker push registry.k8s.osc:5000/heapster-influxdb 
The push refers to repository [registry.k8s.osc:5000/heapster-influxdb]
2d447b9e914f: Pushed 
7da815924651: Pushed 
38ac8d0f5bb3: Mounted from heapster 
latest: digest: sha256:d2ecd285eb6585d56e8853da7b9fd8f4a57de4a3006f6720173a3f3942c0e7c9 size: 945

influxdb時間序列庫介紹

建立deployment

[root@master kube-system]# vim influxdb-deployment.yaml
[root@master kube-system]# vim influxdb-service.yaml
[root@master kube-system]# vim heapster-deployment.yaml
[root@master kube-system]# vim heapster-service.yaml

分別看一下yaml：
influxdb-deployment.yaml
修改image

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: monitoring-influxdb
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        task: monitoring
        k8s-app: influxdb
    spec:
      volumes:
      - name: influxdb-storage
        emptyDir: {}
      containers:
      - name: influxdb
        image: registry.k8s.osc:5000/heapster-influxdb
        volumeMounts:
        - mountPath: /data
          name: influxdb-storage

influxdb-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    task: monitoring
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: monitoring-influxdb
  name: monitoring-influxdb
  namespace: kube-system
spec:
  ports:
  - name: http
    port: 8083
    targetPort: 8083
  - name: api
    port: 8086
    targetPort: 8086
  selector:
    k8s-app: influxdb

建立deployment、service

[root@master kube-system]# kubectl create -f influxdb-deployment.yaml 
[root@master kube-system]# kubectl create -f influxdb-service.yaml

安裝這兩個後查看influxdb坐在的pod ip

[root@master kube-system]# kubectl get pods -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE monitoring-influxdb-3696415694-q9tds 1/1 Running 0 16m 172.99.39.6 172.16.187.158

測試安裝正常，再安裝flanneld的node訪問如下連接，若是無報錯說明安裝成功

[root@node0 ~]# curl http://172.99.39.6:8086/ping

建立heapster-deployment.yaml
修改image、--source、--sink

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: heapster
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        task: monitoring
        k8s-app: heapster
        version: v6
    spec:
      containers:
      - name: heapster
        image: registry.k8s.osc:5000/heapster:v1.3.0
        imagePullPolicy: Always
        command:
        - /heapster
        - --source=kubernetes:http://172.16.187.162:8080
        - --sink=influxdb:http://172.99.39.6:8086

建立heapster-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    task: monitoring  
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: Heapster
  name: heapster
  namespace: kube-system
spec:
  ports:
  - port: 80
    targetPort: 8082
  selector:
    k8s-app: heapster

建立heapster的deployment、service

[root@master kube-system]# kubectl create -f heapster-deployment.yaml 
deployment "heapster" created
[root@master kube-system]# kubectl create -f heapster-service.yaml 
service "heapster" created

所有安裝後能夠查看日誌是不是正常啓動的

[root@master kube-system]# kubectl get pods -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE heapster-1258036176-sjg7s 1/1 Running 0 1m 172.99.93.13 172.16.187.160 monitoring-influxdb-3696415694-q9tds 1/1 Running 0 26m 172.99.39.6 172.16.187.158 [root@master kube-system]# kubectl logs -f monitoring-influxdb-3696415694-q9tds -n kube-system 8888888 .d888 888 8888888b. 888888b. 888 d88P" 888 888 "Y88b 888 "88b 888 888 888 888 888 888 .88P 888 88888b. 888888 888 888 888 888 888 888 888 8888888K. 888 888 "88b 888 888 888 888 Y8bd8P' 888 888 888 "Y88b 888 888 888 888 888 888 888 X88K 888 888 888 888 888 888 888 888 888 Y88b 888 .d8""8b. 888 .d88P 888 d88P 8888888 888 888 888 888 "Y88888 888 888 8888888P" 8888888P" [run] 2018/12/07 05:27:33 InfluxDB starting, version unknown, branch unknown, commit unknown [run] 2018/12/07 05:27:33 Go version go1.7.4, GOMAXPROCS set to 16 [run] 2018/12/07 05:27:33 Using configuration at: /etc/config.toml [store] 2018/12/07 05:27:33 Using data dir: /data/data [subscriber] 2018/12/07 05:27:33 opened service [monitor] 2018/12/07 05:27:33 Starting monitor system [monitor] 2018/12/07 05:27:33 'build' registered for diagnostics monitoring [monitor] 2018/12/07 05:27:33 'runtime' registered for diagnostics monitoring [monitor] 2018/12/07 05:27:33 'network' registered for diagnostics monitoring [monitor] 2018/12/07 05:27:33 'system' registered for diagnostics monitoring [shard-precreation] 2018/12/07 05:27:33 Starting precreation service with check interval of 10m0s, advance period of 30m0s [snapshot] 2018/12/07 05:27:33 Starting snapshot service [continuous_querier] 2018/12/07 05:27:33 Starting continuous query service [httpd] 2018/12/07 05:27:33 Starting HTTP service [httpd] 2018/12/07 05:27:33 Authentication enabled: false ## heapster [root@master kube-system]# kubectl logs -f heapster-1258036176-sjg7s -n kube-system I1207 05:53:00.275512 1 heapster.go:71] /heapster --source=kubernetes:http://172.16.187.162:8080 --sink=influxdb:http://172.99.39.6:8086 I1207 05:53:00.275568 1 heapster.go:72] Heapster version v1.3.0-beta.1 I1207 05:53:00.275794 1 configs.go:61] Using Kubernetes client with master "http://172.16.187.162:8080" and version v1 I1207 05:53:00.275816 1 configs.go:62] Using kubelet port 10255 I1207 05:53:00.283647 1 influxdb.go:252] created influxdb sink with options: host:172.99.39.6:8086 user:root db:k8s I1207 05:53:00.283680 1 heapster.go:193] Starting with InfluxDB Sink I1207 05:53:00.283687 1 heapster.go:193] Starting with Metric Sink I1207 05:53:00.294214 1 heapster.go:105] Starting heapster on port 8082 I1207 05:54:05.082812 1 influxdb.go:215] Created database "k8s" on influxDB server at "172.99.39.6:8086"

最後查看heapster，因爲收集數據須要時間，過一段時間後，查看節點的node的監控數據

[root@master ~]# kubectl top node
NAME             CPU(cores) CPU% MEMORY(bytes) MEMORY% 172.16.187.158 121m 0% 19721Mi 30% 172.16.187.159 112m 0% 15805Mi 24% 172.16.187.160 172m 1% 28090Mi 43%

建立HPA

以上步驟都成功的時候，咱們能夠建立HorizontalPodAutoscaler來管理，下面就用ms-wechat來進行測試

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: ms-wechat  # 名稱
  namespace: default #k8s命名空間
spec:
  maxReplicas: 10  # 最大副本數
  minReplicas: 3   # 最小副本數
  scaleTargetRef:   
    apiVersion: apps/v1beta1
    kind: Deployment  
    name: ms-wechat   # 監控名爲ms-wechat的Deployment
  targetCPUUtilizationPercentage: 80  # cpu 閾值

查看hpa

[root@master ~]# kubectl get hpa 
NAME        REFERENCE              TARGETS           MINPODS   MAXPODS   REPLICAS   AGE
ms-wechat   Deployment/ms-wechat   <unknown> / 80%   3         10        3          10m

你們看到 targets爲unknown有兩種緣由

查看原始deployment的resource有沒有設置cpu的限制若是沒有：kubectl set resources deployment/ms-wechat --limits=cpu=2000m動態設置
等一段時間再查看

查看結果

[root@master ~]# kubectl get hpa NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE ms-wechat Deployment/ms-wechat 47% / 80% 3 10 3 11m

能夠進行壓力測試，觀察REPLICAS變化

轉載：https://www.jianshu.com/p/31ed5c98648e

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。