對於Kubernetes的service,不管是cluster-ip和nodeport均是四層的負載,集羣內的服務如何實現七層的負載均衡,這就須要藉助於ingress,ingress控制器實現的方式有不少,好比nginx,contour,haproxy,trafik,lstio。幾種經常使用的ingress功能對比和選型能夠參考這裏www.kubernetes.org.cn/5948.htmlhtml
ingress-nginx是七層的負載均衡器,負責統一管理外部對k8s cluster中的service的請求。主要包含前端
總結用ingress好處:
一、同臺服務器不一樣業務不須要再給每一個業務映射端口(Nodeport),只須要每臺機安裝一個ingress,利用ingress反代CluserIP,前端機訪問Ingress固定端口
二、添加新業務只須要再建立一個ingress反代新業務的service,再去前端Nginx反代配置servername裏面添加一個域名便可以訪問新業務,經過不一樣的域名訪問不一樣的業務,不須要再配反代node
hdss7-200機主機上操做nginx
docker pull traefik:v1.7-alpine docker tag c36f69007d98 harbor.od.com/k8s/traefik:v1.7 docker push harbor.od.com/k8s/traefik:v1.7
清單下載地址:https://github.com/traefik/traefik/tree/v1.7/examples/k8sgit
mkdir -p /data/k8s-yaml/traefik && cd /data/k8s-yaml/traefik
cat > /data/k8s-yaml/traefik/rbac.yaml <<'eof' apiVersion: v1 kind: ServiceAccount metadata: name: traefik-ingress-controller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: traefik-ingress-controller rules: - apiGroups: - "" resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: kube-system eof
cat > /data/k8s-yaml/traefik/daemonset.yaml <<'eof' apiVersion: apps/v1 kind: DaemonSet metadata: name: traefik-ingress-controller namespace: kube-system labels: k8s-app: traefik-ingress-lb spec: selector: matchLabels: k8s-app: traefik-ingress-lb template: metadata: labels: k8s-app: traefik-ingress-lb name: traefik-ingress-lb spec: serviceAccountName: traefik-ingress-controller terminationGracePeriodSeconds: 60 containers: - image: harbor.od.com/k8s/traefik:v1.7 name: traefik-ingress-lb ports: - name: http containerPort: 80 hostPort: 81 securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE args: - --api - --kubernetes - --logLevel=INFO - --insecureskipverify=true - --kubernetes.endpoint=https://10.4.7.10:7443 - --accesslog - --accesslog.filepath=/var/log/traefik_access.log - --traefiklog - --traefiklog.filepath=/var/log/traefik.log - --metrics.prometheus imagePullSecrets: - name: harbor eof
hostPort: 81 爲ingress的程序80端口映射到宿主機供提供訪問的端口github
kubectl apply -f http://k8s-yaml.od.com/traefik/rbac.yaml kubectl apply -f http://k8s-yaml.od.com/traefik/daemonset.yaml
mkdir /data/k8s-yaml/nginxtest
cat > /data/k8s-yaml/nginxtest/ingress.yml <<'eof' apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx-web namespace: default annotations: kubernetes.io/ingress.class: traefik spec: rules: - host: nginxtest.od.com http: paths: - path: / backend: serviceName: nginx-test servicePort: 80 eof
主機名爲nginxtest.od.com,反代到svc的name爲nginx-test,路徑爲/,端口80web
cat > /data/k8s-yaml/nginxtest/svc.yml <<'eof' apiVersion: v1 kind: Service metadata: labels: k8s-app: nginx-test name: nginx-test namespace: default spec: ports: - port: 80 protocol: TCP selector: app: nginx-test sessionAffinity: None eof
svc標籤選擇器app: nginx-test,反代pod爲app:nginx-testdocker
cat > /data/k8s-yaml/nginxtest/deploy.yml <<'eof' apiVersion: apps/v1 kind: Deployment metadata: name: nginx-test labels: app: nginx-test spec: replicas: 2 selector: matchLabels: app: nginx-test template: metadata: labels: app: nginx-test spec: containers: - name: nginx-test image: harbor.od.com/public/nginx:v1.7.9 ports: - name: web containerPort: 80
hdss7-11.host.com上操做shell
cat >> /var/named/od.com.zone <<'eof' nginxtest A 10.4.7.10 eof vi /var/named/od.com.zone 2020100504 ; serial # 日期加1 systemctl restart named
在hdss7-11.host.com和hdss7-12.host.com上操做api
cat >/etc/nginx/conf.d/nginxtest.com.conf <<'eof' upstream default_backend_traefik { server 10.4.7.21:81 max_fails=3 fail_timeout=10s; server 10.4.7.22:81 max_fails=3 fail_timeout=10s; } server { server_name nginxtest.od.com; location / { proxy_pass http://default_backend_traefik; proxy_set_header Host $http_host; proxy_set_header x-forwarded-for $proxy_add_x_forwarded_for; } } eof nginx -s reload
如下都在hdss7-21.host.com或hdss7-22上操做
[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/nginxtest/deploy.yml [root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/nginxtest/svc.yml [root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/nginxtest/ingress.yml [root@hdss7-22 ~]# kubectl get ing Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress NAME CLASS HOSTS ADDRESS PORTS AGE nginx-web <none> nginxtest.od.com 80 18h [root@hdss7-22 ~]# kubectl get pods -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES coredns-57c78bdbcd-lsf5z 1/1 Running 4 30h 172.7.21.3 hdss7-21.host.com <none> <none> traefik-ingress-controller-9n8zb 1/1 Running 0 11h 172.7.21.5 hdss7-21.host.com <none> <none> traefik-ingress-controller-wxnqw 1/1 Running 0 11h 172.7.22.4 hdss7-22.host.com <none> <none>
[root@hdss7-22 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-test-558df79dc9-d95rp 1/1 Running 0 9h 172.7.21.2 hdss7-21.host.com <none> <none> nginx-test-558df79dc9-qw2fj 1/1 Running 0 9h 172.7.22.2 hdss7-22.host.com <none> <none> [root@hdss7-22 ~]# kubectl exec -it nginx-test-558df79dc9-d95rp -- /bin/bash root@nginx-test-558df79dc9-d95rp:/# echo WEB1 > /usr/share/nginx/html/index.html root@nginx-test-558df79dc9-d95rp:/# exit exit [root@hdss7-22 ~]# kubectl exec -it nginx-test-558df79dc9-qw2fj -- /bin/bash root@nginx-test-558df79dc9-qw2fj:/# echo WEB2 > /usr/share/nginx/html/index.html