kubernetes 部署 Traefik2.4.13

說明

關於Traefik介紹，網上有不少資料能夠參考，本文主要講解traefik基本使用。

核心概念

首先，當啓動Traefik時，須要定義entrypoints（入口點），而後，根據鏈接到這些entrypoints的路由來分析傳入的請求，來查看他們是否與一組規則相匹配，若是匹配，則路由可能會將請求經過一系列中間件轉換事後再轉發到你的服務上去。在瞭解Traefik以前有幾個核心概念咱們必需要了解：

• Providers: 用來自動發現平臺上的服務，能夠是編排工具、容器引擎或者 key-value 存儲等，好比 Docker、Kubernetes、File

• Entrypoints: 監聽傳入的流量（端口等…），是網絡入口點，它們定義了接收請求的端口（HTTP 或者 TCP）。

• Routers: 分析請求（host, path, headers, SSL, …），負責將傳入請求鏈接到能夠處理這些請求的服務上去。

• Services: 將請求轉發給你的應用（load balancing, …），負責配置如何獲取最終將處理傳入請求的實際服務。

• Middlewares: 中間件，用來修改請求或者根據請求來作出一些判斷（authentication, rate limiting, headers, …），中間件被附件到路由上，是一種在請求發送到你的服務以前（或者在服務的響應發送到客戶端以前）調整請求的一種方法。

部署Traefik

系統環境

Traefik version: v2.4.13 
Kubernetes version: v1.17.9

建立 CRD 資源

在Traefik v2.0版本後，開始使用CRD（Custom Resource Definition）來完成路由配置等，因此須要提早建立CRD資源。

# cat traefik-crd.yaml

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingre***outes.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Ingre***oute
    plural: ingre***outes
    singular: ingre***oute
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingre***outetcps.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Ingre***outeTCP
    plural: ingre***outetcps
    singular: ingre***outetcp
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingre***outeudps.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Ingre***outeUDP
    plural: ingre***outeudps
    singular: ingre***outeudp
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsstores.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSStore
    plural: tlsstores
    singular: tlsstore
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: serverstransports.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: ServersTransport
    plural: serverstransports
    singular: serverstransport
  scope: Namespaced

# 建立 Traefik CRD 資源
kubectl apply -f traefik-crd.yaml

建立 RBAC 權限

Kubernetes 在 1.6 版本中引入了基於角色的訪問控制（RBAC）策略，方便對 Kubernetes 資源和 API 進行細粒度控制。Traefik 須要必定的權限，因此，這裏提早建立好Traefik ServiceAccount並分配必定的權限。

# cat traefik-rbac.yaml

## ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: kube-system
  name: traefik-ingress-controller
---
## ClusterRole
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io
    resources:
      - ingresses
      - ingressclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - ingre***outes
      - ingre***outetcps
      - ingre***outeudps
      - middlewares
      - tlsoptions
      - tlsstores
      - traefikservices
      - serverstransports
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.x-k8s.io
    resources:
      - gatewayclasses
      - gatewayclasses/status
      - gateways
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.x-k8s.io
    resources:
      - gatewayclasses/status
    verbs:
      - get
      - patch
      - update
  - apiGroups:
      - networking.x-k8s.io
    resources:
      - gateways/status
    verbs:
      - get
      - patch
      - update
  - apiGroups:
      - networking.x-k8s.io
    resources:
      - httproutes
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
  - apiGroups:
      - networking.x-k8s.io
    resources:
      - httproutes/status
    verbs:
      - get
      - patch
      - update
---
## ClusterRoleBinding
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: kube-system

# 建立 Traefik RBAC 資源
kubectl apply -f traefik-rbac.yaml

建立 Traefik 配置文件

因爲 Traefik 配置不少，經過 CLI 定義不是很方便，通常時候都會經過配置文件配置 Traefik 參數，而後存入ConfigMap，將其掛入 Traefik 中。

下面配置中能夠經過配置kubernetesCRD與kubernetesIngress和kubernetesGateway三項參數，讓 Traefik 支持CRD、Ingress與kubernetesGateway三種路由配置方式。

# cat traefik-config.yaml 

kind: ConfigMap
apiVersion: v1
metadata:
  name: traefik-config
  namespace: kube-system
data:
  traefik.yaml: |-
    ping: ""                    ## 啓用 Ping
    serversTransport:
      insecureSkipVerify: true  ## Traefik 忽略驗證代理服務的 TLS 證書
    api:
      insecure: true            ## 容許 HTTP 方式訪問 API
      dashboard: true           ## 啓用 Dashboard
      debug: false              ## 啓用 Debug 調試模式
    metrics:
      prometheus: ""            ## 配置 Prometheus 監控指標數據，並使用默認配置
    entryPoints:
      web:
        address: ":80"          ## 配置 80 端口，並設置入口名稱爲 web
      websecure:
        address: ":443"         ## 配置 443 端口，並設置入口名稱爲 websecure
    providers:
      kubernetesCRD: ""         ## 啓用 Kubernetes CRD 方式來配置路由規則
      kubernetesIngress: ""     ## 啓用 Kubernetes Ingress 方式來配置路由規則
      kubernetesGateway: ""     ## 啓用 Kubernetes Gateway API
    experimental:               
      kubernetesGateway: true   ## 容許使用 Kubernetes Gateway API
    log:
      filePath: ""              ## 設置調試日誌文件存儲路徑，若是爲空則輸出到控制檯
      level: error              ## 設置調試日誌級別
      format: json              ## 設置調試日誌格式
    accessLog:
      filePath: ""              ## 設置訪問日誌文件存儲路徑，若是爲空則輸出到控制檯
      format: json              ## 設置訪問調試日誌格式
      bufferingSize: 0          ## 設置訪問日誌緩存行數
      filters:
        #statusCodes: ["200"]   ## 設置只保留指定狀態碼範圍內的訪問日誌
        retryAttempts: true     ## 設置代理訪問重試失敗時，保留訪問日誌
        minDuration: 20         ## 設置保留請求時間超過指定持續時間的訪問日誌
      fields:                   ## 設置訪問日誌中的字段是否保留（keep 保留、drop 不保留）
        defaultMode: keep       ## 設置默認保留訪問日誌字段
        names:                  ## 針對訪問日誌特別字段特別配置保留模式
          ClientUsername: drop  
        headers:                ## 設置 Header 中字段是否保留
          defaultMode: keep     ## 設置默認保留 Header 中字段
          names:                ## 針對 Header 中特別字段特別配置保留模式
            User-Agent: redact
            Authorization: drop
            Content-Type: keep
    #tracing:                     ## 鏈路追蹤配置,支持 zipkin、datadog、jaeger、instana、haystack 等 
    #  serviceName:               ## 設置服務名稱（在鏈路追蹤端收集後顯示的服務名）
    #  zipkin:                    ## zipkin配置
    #    sameSpan: true           ## 是否啓用 Zipkin SameSpan RPC 類型追蹤方式
    #    id128Bit: true           ## 是否啓用 Zipkin 128bit 的跟蹤 ID
    #    sampleRate: 0.1          ## 設置鏈路日誌採樣率（能夠配置0.0到1.0之間的值）
    #    httpEndpoint: http://localhost:9411/api/v2/spans     ## 配置 Zipkin Server 端點

# 建立Traefik configmap 資源
kubectl apply -f traefik-config.yaml

設置節點 Label 標籤

# 節點設置 Label 標籤
kubectl label nodes fsyy IngressProxy=true
# 查看節點是否設置 Label 成功
kubectl get nodes --show-labels
# 刪除標籤
kubectl label nodes fsyy IngressProxy-

建立 Traefik

下面將用DaemonSet方式部署 Traefik，便於在多服務器間擴展，用 hostport 方式綁定服務器 80、443 端口，方便流量經過物理機進入 Kubernetes 內部。

# cat traefik-deploy.yaml

apiVersion: v1
kind: Service
metadata:
  name: traefik
  namespace: kube-system
  labels:
    app: traefik
spec:
  ports:
    - name: web
      port: 80
    - name: websecure
      port: 443
    - name: admin
      port: 8080
  selector:
    app: traefik
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    app: traefik
spec:
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      name: traefik
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 1
      containers:
        - image: traefik:v2.4.13
          name: traefik-ingress-lb
          ports:
            - name: web
              containerPort: 80
              hostPort: 80         ## 將容器端口綁定所在服務器的 80 端口
            - name: websecure
              containerPort: 443
              hostPort: 443        ## 將容器端口綁定所在服務器的 443 端口
            - name: admin
              containerPort: 8080  ## Traefik Dashboard 端口
          resources:
            limits:
              cpu: 2000m
              memory: 1024Mi
            requests:
              cpu: 1000m
              memory: 1024Mi
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
          args:
            - --configfile=/config/traefik.yaml
          volumeMounts:
            - mountPath: "/config"
              name: "config"
          readinessProbe:
            httpGet:
              path: /ping
              port: 8080
            failureThreshold: 3
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          livenessProbe:
            httpGet:
              path: /ping
              port: 8080
            failureThreshold: 3
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5    
      volumes:
        - name: config
          configMap:
            name: traefik-config 
      tolerations:              ## 設置容忍全部污點，防止節點被設置污點
        - operator: "Exists"
      nodeSelector:             ## 設置node篩選器，在特定label的節點上啓動
        IngressProxy: "true"

# 建立 Traefik
kubectl apply -f traefik-deploy.yaml

配置路由規則

使用 CRD 方式配置 Traefik 路由規則 -- Traefik Dashboard 爲例

# cat traefik-dashboard-route.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: Ingre***oute
metadata:
  name: traefik-dashboard-route
  namespace: kube-system
spec:
  entryPoints:
  - web
  routes:
  - match: Host(`traefik.mydlq.club`)
    kind: Rule
    services:
      - name: traefik
        port: 8080

接下來配置 Hosts，客戶端想經過域名訪問服務，必需要進行 DNS 解析，這裏能夠經過 DNS 服務器進行域名解析，也能夠修改 hosts 文件將 Traefik 指定節點的 IP 和自定義 host 綁定

# 瀏覽器訪問
http://traefik.mydlq.club/

使用 CRD 方式配置 Traefik 路由規則 -- ArgoCD 爲例

我在以前的文章中講過部署 Argo CD，當時argocd-server服務暴露的方式採用的是NodePort，如今咱們使用今天的方法。

首先，咱們須要將暴露服務類型由NodePort更改成ClusterIP。而後，編輯argocd-server Deployment以將--insecure標誌添加到argocd-server命令中：

spec:
  template:
    spec:
      containers:
      - name: argocd-server
        command:
        - argocd-server
        - --staticassets
        - /shared/app
        - --repo-server
        - argocd-repo-server:8081
        - --insecure  # 須要禁用 tls，不然會 `redirected you too many times`

而後建立以下Ingre***oute資源對象便可，咱們建立了一個redirect-https的中間件，可讓 http 服務強制跳轉到 https 服務去：

# cat argocd-dashboard-https.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: redirect-https
  namespace: argocd
spec:
  redirectScheme:
    scheme: https
---
apiVersion: traefik.containo.us/v1alpha1
kind: Ingre***oute
metadata:
  name: argocd-server-http
  namespace: argocd
spec:
  entryPoints:
    - web
  routes:
    - kind: Rule
      match: Host(`argocd.k8s.local`)
      priority: 10
      middlewares:
        - name: redirect-https
      services:
        - name: argocd-server
          port: 80
    - kind: Rule
      match: Host(`argocd.k8s.local`) && Headers(`Content-Type`, `application/grpc`)
      priority: 11
      middlewares:
        - name: redirect-https
      services:
        - name: argocd-server
          port: 80
          scheme: h2c
---
apiVersion: traefik.containo.us/v1alpha1
kind: Ingre***oute
metadata:
  name: argocd-server
  namespace: argocd
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`argocd.k8s.local`)
      priority: 10
      services:
        - name: argocd-server
          port: 80
    - kind: Rule
      match: Host(`argocd.k8s.local`) && Headers(`Content-Type`, `application/grpc`)
      priority: 11
      services:
        - name: argocd-server
          port: 80
          scheme: h2c
  tls:
    certResolver: default
    options: {}

# 瀏覽器訪問
# 證書是自簽名的，因此在第一次訪問的時候會提示不安全，強制跳轉便可
http://argocd.k8s.local/

CLI命令行登陸：

# 系統 /etc/hosts 作下解析
argocd login argocd.k8s.local --username admin --password Transsion#123

參考連接

Kubernetes 部署 Ingress 控制器 Traefik v2.4