Prometheus 監控K8S 資源狀態對象

Prometheus 監控K8S 資源狀態對象

• 官方文檔：https://github.com/kubernetes/kube-state-metrics

kube-state-metrics是一個簡單的服務，它監聽Kubernetes API服務器並生成有關對象狀態的指標。它不關注單個Kubernetes組件的運行情況，而是關注內部各類對象的運行情況，例如部署，節點和容器。

採集了k8s中各類資源對象的狀態信息：

kube-state-metrics
kube_daemonset_*
kube_deployment_*
kube_job_*
kube_namespace_*
kube_node_*
kube_persistentvolumeclaim_*
kube_pod_container_*
kube_pod_*
kube_replicaset_*
kube_service_*
kube_statefulset_*

View Code

配置文件

下列是已經修改好的配置文件

• # 主程序
• kube-state-metrics-deployment.yaml

• apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: kube-state-metrics
    namespace: kube-system
    labels:
      k8s-app: kube-state-metrics
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
      version: v1.3.0
  spec:
    selector:
      matchLabels:
        k8s-app: kube-state-metrics
        version: v1.3.0
    replicas: 1
    template:
      metadata:
        labels:
          k8s-app: kube-state-metrics
          version: v1.3.0
        annotations:
          scheduler.alpha.kubernetes.io/critical-pod: ''
      spec:
        priorityClassName: system-cluster-critical
        serviceAccountName: kube-state-metrics
        containers:
        - name: kube-state-metrics
          # 修改成鏡像地址
          image: lizhenliang/kube-state-metrics:v1.3.0
          ports:
          - name: http-metrics
            containerPort: 8080
          - name: telemetry
            containerPort: 8081
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 5
        - name: addon-resizer
          # 修改成鏡像地址
          image: lizhenliang/addon-resizer:1.8.3
          resources:
            limits:
              cpu: 100m
              memory: 30Mi
            requests:
              cpu: 100m
              memory: 30Mi
          env:
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: MY_POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          volumeMounts:
            - name: config-volume
              mountPath: /etc/config
          command:
            - /pod_nanny
            - --config-dir=/etc/config
            - --container=kube-state-metrics
            - --cpu=100m
            - --extra-cpu=1m
            - --memory=100Mi
            - --extra-memory=2Mi
            - --threshold=5
            - --deployment=kube-state-metrics
        volumes:
          - name: config-volume
            configMap:
              name: kube-state-metrics-config
  ---
  # Config map for resource configuration.
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: kube-state-metrics-config
    namespace: kube-system
    labels:
      k8s-app: kube-state-metrics
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
  data:
    NannyConfiguration: |-
      apiVersion: nannyconfig/v1alpha1
      kind: NannyConfiguration

  配置文件
• # APIServer受權
• kube-state-metrics-rbac.yaml

• apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: kube-state-metrics
    namespace: kube-system
    labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: kube-state-metrics
    labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
  rules:
  - apiGroups: [""]
    resources:
    - configmaps
    - secrets
    - nodes
    - pods
    - services
    - resourcequotas
    - replicationcontrollers
    - limitranges
    - persistentvolumeclaims
    - persistentvolumes
    - namespaces
    - endpoints
    verbs: ["list", "watch"]
  - apiGroups: ["extensions"]
    resources:
    - daemonsets
    - deployments
    - replicasets
    verbs: ["list", "watch"]
  - apiGroups: ["apps"]
    resources:
    - statefulsets
    verbs: ["list", "watch"]
  - apiGroups: ["batch"]
    resources:
    - cronjobs
    - jobs
    verbs: ["list", "watch"]
  - apiGroups: ["autoscaling"]
    resources:
    - horizontalpodautoscalers
    verbs: ["list", "watch"]
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    name: kube-state-metrics-resizer
    namespace: kube-system
    labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
  rules:
  - apiGroups: [""]
    resources:
    - pods
    verbs: ["get"]
  - apiGroups: ["extensions"]
    resources:
    - deployments
    resourceNames: ["kube-state-metrics"]
    verbs: ["get", "update"]
  ---
  apiVersion: rbac.authorization.k8s.io/v1 
  kind: ClusterRoleBinding
  metadata:
    name: kube-state-metrics
    labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: kube-state-metrics
  subjects:
  - kind: ServiceAccount
    name: kube-state-metrics
    namespace: kube-system
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    name: kube-state-metrics
    namespace: kube-system
    labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: kube-state-metrics-resizer
  subjects:
  - kind: ServiceAccount
    name: kube-state-metrics
    namespace: kube-system

  配置文件
• # 暴露端口
• kube-state-metrics-service.yaml

• apiVersion: v1
  kind: Service
  metadata:
    name: kube-state-metrics
    namespace: kube-system
    labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
      kubernetes.io/name: "kube-state-metrics"
    annotations:
      prometheus.io/scrape: 'true'
  spec:
    ports:
    - name: http-metrics
      port: 8080
      targetPort: http-metrics
      protocol: TCP
    - name: telemetry
      port: 8081
      targetPort: telemetry
      protocol: TCP
    selector:
      k8s-app: kube-state-metrics

  配置文件

部署

一、建立容器

kubectl apply -f kube-state-metrics-rbac.yaml
kubectl apply -f kube-state-metrics-deployment.yaml
kubectl apply -f kube-state-metrics-service.yaml

二、查看建立pod

kubectl get pods -n kube-system

NAME                                 READY   STATUS    RESTARTS   AGE
coredns-64479cf49b-lsqqn             1/1     Running   0          5h26m
kube-state-metrics-6459c9f7f-wbwwt   2/2     Running   0          58s
prometheus-0                         2/2     Running   0          4h12m

三、測試訪問

 四、Grafana可視化導入模板：6417

 五、查看監控（其餘資源可根據主機狀況進行微調）