helm 持久化部署ingres

Ingress 是一種 Kubernetes 資源,也是將 Kubernetes 集羣內服務暴露到外部的一種方式。本文將講一講如何用 Helm 在 Kubernetes 集羣中部署 Ingress,並部署兩個應用來演示 Ingress 的具體使用。php

閱讀本文前你須要先掌握 Helm 和一些 Kubernetes 服務暴露的相關知識點,若是你還不瞭解能夠先讀一讀我以前寫的 「Helm 入門指南」和「淺析從外部訪問 Kubernetes 集羣中應用的幾種方式」這兩篇文章。html

部署 Ingress Controller

Ingress 只是一個統稱,其由 Ingress 和 Ingress Controller 兩部分組成。Ingress 用做將原來須要手動配置的規則抽象成一個 Ingress 對象,使用 YAML 格式的文件來建立和管理。Ingress Controller 用做經過與 Kubernetes API 交互,動態的去感知集羣中 Ingress 規則變化。linux

目前可用的 Ingress Controller 類型有不少,好比:Nginx、HAProxy、Traefik 等,咱們將演示如何部署一個基於 Nginx 的 Ingress Controller。nginx

這裏咱們使用 Helm 來部署,在開始部署前,請確認您已經安裝和配置好 Helm 相關環境。git

查找軟件倉庫中是否有 Nginx Ingress 包

1
2
3
$ helm search nginx-ingress
NAME CHART VERSION APP VERSION DESCRIPTION
stable/nginx-ingress 0.9.5 0.10.2 An nginx Ingress controller that uses ConfigMap...

注:這裏咱們使用的是在阿里雲 Helm 鏡像倉庫。若是你還不知道如何增長三方倉庫,可先閱讀 「Helm 入門指南」一文。github

阿里雲 Helm 鏡像倉庫裏的 nginx-ingress 軟件包已經將要用到的相關容器鏡像地址改爲了國內可訪問的地址。安裝時須要用到的容器鏡像有:chrome

1
2
3
repository: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.10.2
repository: registry.cn-hangzhou.aliyuncs.com/google_containers/defaultbackend:1.3
repository: sophos/nginx-vts-exporter:0.6

使用 Helm 部署 Nginx Ingress Controller

Ingress Controller 自己對外暴露的方式有幾種,好比:hostNetwork、externalIP 等。這裏咱們採用 externalIP 的方式進行,若是你要使用 hostNetwork 方式,可使用 controller.hostNetwork=true參數進行設置。數據庫

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
# 啓用 RBAC 支持
$ helm install --name nginx-ingress --set "rbac.create=true,controller.service.externalIPs[0]=192.168.100.211,controller.service.externalIPs[1]=192.168.100.212,controller.service.externalIPs[2]=192.168.100.213" stable/nginx-ingress

NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
nginx-ingress-controller 1 2m

==> v1beta1/ClusterRole
NAME AGE
nginx-ingress 2m

==> v1beta1/ClusterRoleBinding
NAME AGE
nginx-ingress 2m

==> v1beta1/RoleBinding
NAME AGE
nginx-ingress 2m

==> v1/ServiceAccount
NAME SECRETS AGE
nginx-ingress 1 2m

==> v1beta1/Role
NAME AGE
nginx-ingress 2m

==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress-controller LoadBalancer 10.254.84.72 192.168.100.211,192.168.100.212,192.168.100.213 80:8410/TCP,443:8948/TCP 2m
nginx-ingress-default-backend ClusterIP 10.254.206.175 <none> 80/TCP 2m

==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
nginx-ingress-controller 1 1 1 1 2m
nginx-ingress-default-backend 1 1 1 1 2m

==> v1beta1/PodDisruptionBudget
NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE
nginx-ingress-controller 1 N/A 0 2m
nginx-ingress-default-backend 1 N/A 0 2m

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-665cd897fc-n5rq4 1/1 Running 0 2m
nginx-ingress-default-backend-df594cfc6-pbcxk 1/1 Running 0 2m


NOTES:
The nginx-ingress controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w nginx-ingress-controller'

An example Ingress that makes use of the controller:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: example
namespace: foo
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
serviceName: exampleService
servicePort: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls

部署完成後咱們能夠看到 Kubernetes 服務中增長了 nginx-ingress-controller 和 nginx-ingress-default-backend 兩個服務。nginx-ingress-controller 爲 Ingress Controller,主要作爲一個七層的負載均衡器來提供 HTTP 路由、粘性會話、SSL 終止、SSL直通、TCP 和 UDP 負載平衡等功能。nginx-ingress-default-backend 爲默認的後端,當集羣外部的請求經過 Ingress 進入到集羣內部時,若是沒法負載到相應後端的 Service 上時,這種未知的請求將會被負載到這個默認的後端上。json

因爲咱們採用了 externalIP 方式對外暴露服務, 因此 nginx-ingress-controller 會在 192.168.100.2十一、192.168.100.2十二、192.168.100.213 三臺節點宿主機上的 暴露 80/443 端口。後端

1
2
3
4
5
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 18d
nginx-ingress-controller LoadBalancer 10.254.84.72 192.168.100.211,192.168.100.212,192.168.100.213 80:8410/TCP,443:8948/TCP 46s
nginx-ingress-default-backend ClusterIP 10.254.206.175 <none> 80/TCP 46s

訪問 Nginx Ingress Controller

咱們可使用如下命令來獲取 Nginx 的 HTTP 和 HTTPS 地址。

1
2
3
$ kubectl --namespace default get services -o wide -w nginx-ingress-controller
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nginx-ingress-controller LoadBalancer 10.254.84.72 192.168.100.211,192.168.100.212,192.168.100.213 80:8410/TCP,443:8948/TCP 4h app=nginx-ingress,component=controller,release=nginx-ingress

由於咱們尚未在 Kubernetes 集羣中建立 Ingress資源,因此直接對 ExternalIP 的請求被負載到了 nginx-ingress-default-backend 上。nginx-ingress-default-backend 默認提供了兩個 URL 進行訪問,其中的 /healthz 用做健康檢查返回 200,而 / 返回 404 錯誤。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# 返回 200
$ curl -I http://192.168.100.212/healthz/
HTTP/1.1 200 OK
Server: nginx/1.13.8
Date: Tue, 24 Jul 2018 06:25:58 GMT
Content-Type: text/html
Content-Length: 0
Connection: keep-alive
Strict-Transport-Security: max-age=15724800; includeSubDomains;

# 返回 404
$ curl -I http://192.168.100.212/
HTTP/1.1 404 Not Found
Server: nginx/1.13.8
Date: Tue, 24 Jul 2018 06:26:39 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 21
Connection: keep-alive
Strict-Transport-Security: max-age=15724800; includeSubDomains;

# 返回 200
$ curl -I --insecure https://192.168.100.212/healthz/
HTTP/2 200
server: nginx/1.13.8
date: Tue, 24 Jul 2018 06:27:41 GMT
content-type: text/html
content-length: 0
strict-transport-security: max-age=15724800; includeSubDomains;

# 返回 404
$ curl -I --insecure https://192.168.100.212/
HTTP/2 404
server: nginx/1.13.8
date: Tue, 24 Jul 2018 06:28:25 GMT
content-type: text/plain; charset=utf-8
content-length: 21
strict-transport-security: max-age=15724800; includeSubDomains;

在幾臺節點宿主機上查看,咱們能夠看到 ExternalIP 的 Service 是經過 Kube-Proxy對外暴露的,這裏的 192.168.100.2十一、192.168.100.2十二、192.168.100.213 是三個內網 IP。 實際生產應用中是須要經過邊緣路由器或全局統一接入層的負載均衡器將到達公網 IP 的外網流量轉發到這幾個內網 IP 上,外部用戶再經過域名訪問集羣中以 Ingress 暴露的全部服務。

1
2
3
4
5
6
7
8
9
10
11
$ sudo netstat -tlunp|grep kube-proxy|grep -E '80|443'
tcp 0 0 192.168.100.211:80 0.0.0.0:* LISTEN 714/kube-proxy
tcp 0 0 192.168.100.211:443 0.0.0.0:* LISTEN 714/kube-proxy

$ sudo netstat -tlunp|grep kube-proxy|grep -E '80|443'
tcp 0 0 192.168.100.212:80 0.0.0.0:* LISTEN 690/kube-proxy
tcp 0 0 192.168.100.212:443 0.0.0.0:* LISTEN 690/kube-proxy

$ sudo netstat -tlunp|grep kube-proxy|grep -E '80|443'
tcp 0 0 192.168.100.213:80 0.0.0.0:* LISTEN 748/kube-proxy
tcp 0 0 192.168.100.213:443 0.0.0.0:* LISTEN 748/kube-proxy

卸載 Nginx Ingress Controller

使用 Helm 卸載 Nginx Ingress Controller 很是的簡單,只需下面一條指令就搞定了。

1
$ helm delete --purge nginx-ingress

使用 --purge 參數能夠完全刪除 Release 不留下任何記錄,不然下一次部署的時候不能使用重名的 Release。

部署 Ingress

接下來,咱們經過 Helm 以 Ingress 方式在 Kubernetes 集羣中部署兩個應用。因爲測試環境沒有使用 PersistentVolume(持久卷,簡稱 PV),下面兩個例子中都暫時將其關閉。有關於 PersistentVolume 的知識點,我將在後面的文章來說一講,敬請期待。

部署 DokuWiki

DokuWiki 是一個針對小公司文件需求而開發的 Wiki 引擎,用 PHP 語言開發。DokuWiki 基於文本存儲,不須要數據庫。由於 DokuWiki 很是的輕量,因此我選擇它來作演示。

這裏咱們使用 Helm 官方倉庫裏的 Chart 包來進行,由於阿里雲鏡像倉庫中的不少 Chart 都不是最新的版本而且不支持以 Ingress 方式部署。

1
2
# 從 Helm 官方 Chart 倉庫遷出全部軟件包
$ git clone https://github.com/helm/charts.git

使用 helm install 進行一鍵部署,並經過 ingress.enabled=true 和 ingress.hosts[0].name=wiki.hi-linux.com 參數啓用 Ingress 特性和設置對應的主機名。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
$ cd /home/k8s/charts/stable
$ helm install --name dokuwiki --set "ingress.enabled=true,ingress.hosts[0].name=wiki.hi-linux.com,persistence.enabled=false" dokuwiki

NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
dokuwiki-dokuwiki wiki.hi-linux.com 80 53s

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
dokuwiki-dokuwiki-747b45cddb-qt8l2 1/1 Running 0 53s

==> v1/Secret
NAME TYPE DATA AGE
dokuwiki-dokuwiki Opaque 1 54s

==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dokuwiki-dokuwiki LoadBalancer 10.254.235.137 <pending> 80:8430/TCP,443:8848/TCP 54s

==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
dokuwiki-dokuwiki 1 1 1 1 54s


NOTES:

** Please be patient while the chart is being deployed **

1. Get the DokuWiki URL indicated on the Ingress Rule and associate it to your cluster external IP:

export CLUSTER_IP=$(minikube ip) # On Minikube. Use: `kubectl cluster-info` on others K8s clusters
export HOSTNAME=$(kubectl get ingress --namespace default dokuwiki-dokuwiki -o jsonpath='{.spec.rules[0].host}')
echo "Dokuwiki URL: http://$HOSTNAME/"
echo "$CLUSTER_IP $HOSTNAME" | sudo tee -a /etc/hosts

2. Login with the following credentials

echo Username: user
echo Password: $(kubectl get secret --namespace default dokuwiki-dokuwiki -o jsonpath="{.data.dokuwiki-password}" | base64 --decode)

部署完成後,根據提示生成相應的登錄用戶名和密碼。

1
2
3
4
5
$ echo Username: user
Username: user

$ echo Password: $(kubectl get secret --namespace default dokuwiki-dokuwiki -o jsonpath="{.data.dokuwiki-password}" | base64 --decode)
Password: e2GrABBkwF

測試從各節點的宿主機 IP 訪問應用,這裏咱們直接使用 Curl 命令進行訪問。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
$ curl -I  http://wiki.hi-linux.com/doku.php -x 192.168.100.211:80
HTTP/1.1 200 OK
Server: nginx/1.13.8
Date: Wed, 25 Jul 2018 05:16:02 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/7.0.31
Vary: Cookie,Accept-Encoding
Set-Cookie: DokuWiki=k2clt6f2qe472ehsq6tcmh6v20; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Set-Cookie: DW68700bfd16c2027de7de74a5a8202a6f=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; HttpOnly
X-UA-Compatible: IE=edge,chrome=1


$ curl -I http://wiki.hi-linux.com/doku.php -x 192.168.100.212:80
HTTP/1.1 200 OK
Server: nginx/1.13.8
Date: Wed, 25 Jul 2018 05:18:13 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/7.0.31
Vary: Cookie,Accept-Encoding
Set-Cookie: DokuWiki=ork8sv8qpurteblasuq3eb3nt2; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Set-Cookie: DW68700bfd16c2027de7de74a5a8202a6f=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; HttpOnly


$ curl -I http://wiki.hi-linux.com/doku.php -x 192.168.100.213:80
HTTP/1.1 200 OK
Server: nginx/1.13.8
Date: Wed, 25 Jul 2018 05:18:30 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/7.0.31
Vary: Cookie,Accept-Encoding
Set-Cookie: DokuWiki=6ulgtsddqq3rlo0mriavj64jc4; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Set-Cookie: DW68700bfd16c2027de7de74a5a8202a6f=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; HttpOnly
X-UA-Compatible: IE=edge,chrome=1

Curl 用法不少,你也可使用下面方式來達到相同的效果。

1
$ curl -H "Host:wiki.hi-linux.com"  "http://192.168.100.211/doku.php"

固然你也能夠在本地 hosts 文件中對 IP 和域名進行綁定後,經過瀏覽器訪問該應用。效果圖以下:

部署 Minio

Minio 是一個基於 Apache License v2.0 開源協議的對象存儲服務,Minio 使用 Go 語言開發,具備良好的跨平臺性,一樣是一個很是輕量的服務。它兼容亞馬遜 S3 雲存儲服務接口,很是適合於存儲大容量非結構化的數據,例如:圖片、視頻、日誌文件、備份數據和容器/虛擬機鏡像等。

1
2
# 從 Helm 官方 Chart 倉庫遷出全部軟件包
$ git clone https://github.com/helm/charts.git

若是你須要修改主機名,請修改 values.yaml 文件中的 hosts 的值。我想你必定以爲很奇怪,爲何在這個例子我沒用使用傳遞參數的方式來動態修改模板中對應的值?真相只有一個,哪就是我沒有找到能成功修改模板中對應的變量,驚不驚喜,意不意外呢?哈哈哈。若是你知道能夠留言告訴我喲!

1
2
3
4
5
$ cd /home/k8s/charts/stable
$ cat minio/values.yaml
hosts:
- minio.hi-linux.com
#- chart-example.local

使用 helm install 進行一鍵部署,並經過 ingress.enabled=true 參數啓用 Ingress 特性。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
$ helm install --name minio  --set "ingress.enabled=true,persistence.enabled=false" minio

NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
minio minio.hi-linux.com 80 1m

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
minio-7c7cf49d4-gqf8p 1/1 Running 0 1m

==> v1/Secret
NAME TYPE DATA AGE
minio Opaque 2 1m

==> v1/ConfigMap
NAME DATA AGE
minio 2 1m

==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
minio ClusterIP None <none> 9000/TCP 1m

==> v1beta2/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
minio 1 1 1 1 1m


NOTES:

Minio can be accessed via port 9000 on the following DNS name from within your cluster:
minio-svc.default.svc.cluster.local

To access Minio from localhost, run the below commands:

1. export POD_NAME=$(kubectl get pods --namespace default -l "release=minio" -o jsonpath="{.items[0].metadata.name}")

2. kubectl port-forward $POD_NAME 9000 --namespace default

Read more about port forwarding here: http://kubernetes.io/docs/user-guide/kubectl/kubectl_port-forward/

You can now access Minio server on http://localhost:9000. Follow the below steps to connect to Minio server with mc client:

1. Download the Minio mc client - https://docs.minio.io/docs/minio-client-quickstart-guide

2. mc config host add minio-local http://localhost:9000 AKIAIOSFODNN7EXAMPLE wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY S3v4

3. mc ls minio-local

Alternately, you can use your browser or the Minio SDK to access the server - https://docs.minio.io/categories/17

部署完成後,咱們在本地 hosts 文件中對 IP 和域名進行綁定,並經過瀏覽器訪問該應用。效果圖以下:

登錄用戶名和密碼在部署完成後的提示信息中。

最後咱們在 Kubernetes 上來查看下部署成功後的 Ingress 信息。

1
2
3
4
5
$ kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
dokuwiki-dokuwiki wiki.hi-linux.com 80 44m
minio minio.hi-linux.com 80 50s
`

參考文檔

相關文章
相關標籤/搜索