05 . k8s實戰之部署PHP/JAVA網站

時間 2020-07-02

標籤 k8s 實戰部署 php java 網站欄目 PHP 简体版

原文原文鏈接

傳統部署和k8s部署區別

一般使用傳統的部署的時候，咱們一個web項目，網站的搭建，每每使用的以下的一種總體架構，可能有的公司在某一環節使用的東西是不同，可是大致的框架流程是都是差很少的php

使用k8s部署,便於彈性伸縮,節約資源,發佈週期快,總體框架以下html

環境

節點名	IP	軟件版本	硬件	網絡	說明
K8s-master	192.168.43.190	list 裏面都有	2C4G	Nat，內網	測試環境
K8s-node1	192.168.43.120	list 裏面都有	2C4G	Nat，內網	測試環境
K8s-node2	192.168.43.9	list 裏面都有	2C4G	Nat，內網	測試環境
K8s-harbor	192.168.43.129	list 裏面都有	2C4G	Nat，內網	測試環境

安裝運行harbor（http方式）

安裝docker

# 安裝一些必要的系統工具
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
# 添加軟件源信息
# docker 官方源
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

# 阿里雲源
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

  
# 安裝前能夠先更新 yum 緩存：
sudo yum makecache fast

# CentOS7安裝 Docker-ce
yum -y install docker-ce        # CentOS 中安裝
apt-get install docker-ce       # Ubuntu 中安裝
pacman -S docker                # Arch 中安裝
emerge --ask docker             # Gentoo 中安裝

# 若是想安裝特定版本的Docker-ce版本，先列出repo中可用版本，而後選擇安裝
yum list docker-ce --showduplicates |sort -r
Loading mirror speeds from cached hostfile
Loaded plugins: fastestmirror
Installed Packages
docker-ce.x86_64            3:19.03.4-3.el7                    docker-ce-stable
docker-ce.x86_64            3:19.03.4-3.el7                    @docker-ce-stable
docker-ce.x86_64            3:19.03.3-3.el7                    docker-ce-stable
docker-ce.x86_64            3:19.03.2-3.el7                    docker-ce-stable
docker-ce.x86_64            3:19.03.1-3.el7                    docker-ce-stable

yum install docker-ce-<VERSION STRING>
# 選擇安裝 docker-ce-18.06.1.ce
yum install docker-ce-18.06.1.ce -y

# Docker鏡像加速
# 沒有啓動/etc/docker目錄不存在，須要本身建立，docker啓動也會本身建立
# 爲了指望咱們的鏡像下載快一點，應該定義一個鏡像加速器，加速器在國內
mkdir /etc/docker
vim /etc/docker/daemon.json
{
"registry-mirrors": ["https://registry.docker-cn.com"]
}

# 啓動Docker後臺服務
systemctl start docker && systemctl enable docker
systemctl daemon-reload                 # 守護進程重啓

# 經過運行hello-world鏡像，驗證是否正確安裝了docker，或者經過查看版本
docker run hello-world
docker version
Client: Docker Engine - Community
Version:           19.03.4
API version:       1.40
Go version:        go1.12.10
Git commit:        9013bf583a
Built:            Fri Oct 18 15:52:22 2019
OS/Arch:           linux/amd64
Experimental:      false

https請看我下面專門寫的文章java

https://www.cnblogs.com/you-men/p/13121835.htmlnode

Harbor 可幫助用戶迅速搭建企業級的 Registry 服務, 它提供了管理圖形界面, 基於角色的訪問控制 ( Role Based Access Control), 鏡像遠程複製 (同步), AD/LDAP 集成, 以及審計日誌等企業用戶需求的功能, 同時還原生支持中文, 深受中國用戶的喜好;python

安裝harbor

注意mysql

安裝harbor以前須要安裝dockerlinux

是 VMware 公司開源了企業級 Registry 項目, 其的目標是幫助用戶迅速搭建一個企業級的 Docker registry 服務。nginx

因爲 Harbor 是基於 Docker Registry V2 版本，因此 docker 版本必須 >=1.10.0 docker-compose >=1.6.0git

下載最新版 Docker Compose
curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-composegithub

# 對二進制文件應用可執行權限：
sudo chmod +x /usr/local/bin/docker-compose
# 測試是否安裝成功
docker-compose --version
# 按照上面給的docker harbor地址，下載離線安裝包
tar xvf harbor-offline-installer-v1.8.1.tgz -C  /usr/local/
vim  /usr/local/harbor/harbor.yml
hostname: 47.92.24.137

# 運行安裝腳本
./install.sh 
[Step 0]: checking installation environment ...
Note: docker version: 19.03.4
Note: docker-compose version: 1.22.0
[Step 1]: loading Harbor images ...
Loaded image: goharbor/harbor-core:v1.8.1
Loaded image: goharbor/harbor-registryctl:v1.8.1
Loaded image: goharbor/redis-photon:v1.8.1
Loaded image: goharbor/notary-server-photon:v0.6.1-v1.8.1
Loaded image: goharbor/chartmuseum-photon:v0.8.1-v1.8.1
Loaded image: goharbor/harbor-db:v1.8.1
Loaded image: goharbor/harbor-jobservice:v1.8.1
Loaded image: goharbor/nginx-photon:v1.8.1
Loaded image: goharbor/registry-photon:v2.7.1-patch-2819-v1.8.1
Loaded image: goharbor/harbor-migrator:v1.8.1
Loaded image: goharbor/prepare:v1.8.1
Loaded image: goharbor/harbor-portal:v1.8.1
Loaded image: goharbor/harbor-log:v1.8.1
Loaded image: goharbor/notary-signer-photon:v0.6.1-v1.8.1
Loaded image: goharbor/clair-photon:v2.0.8-v1.8.1
[Step 2]: preparing environment ...
prepare base dir is set to /usr/local/harbor
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file:/secret/core/private_key.pem, cert file:/secret/registry/root.crt
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[Step 3]: starting Harbor ...
✔ ----Harbor has been installed and started successfully.----
Now you should be able to visit the admin portal at http://47.92.24.137. 
For more details, please visit https://github.com/goharbor/harbor

接下來咱們能夠直接瀏覽器訪問配置文件定義的IP或者域名加端口

默認用戶密碼: admin/Harbor12345

修改harbor端口

# 由於harbor默認端口是80，而大多數時候是不但願使用80端口，修改方法以下
# vim harbor.yml
# 找到port選項修改端口，而後執行./install 就會使用配置文件端口

# 還有一種狀況就是更改已有harbor的配置
vim docker-compose.yml
    dns_search: .
    ports:
      - 99:80

auth:
  token:
    issuer: harbor-token-issuer
    realm: http://47.92.24.137:99/service/token
    rootcertbundle: /etc/registry/root.crt
    service: harbor-registry

docker-compose down -v
docker-compose up -d

使用harbor

爲了體現出效果，建議使用非harbor的另外一臺機器

# 鏡像推送
docker login 47.92.24.137:99 -u admin -p Harbor12345
vim  /etc/docker/daemon.json
{
  "insecure-registries":["192.168.43.129"]
}
systemctl daemon-reload
systemctl restart docker
# 由於docker默認使用的是https協議，而搭建harbor是http提供服務的，
# 因此要配置可信任,或者強制docker login和docker push 走http的80端口，而不是443端口.
docker tag daocloud.io/library/nginx:latest 192.168.43.129/library/nginx:latest

docker push 192.168.43.129/library/nginx:latest

PHP部署項目流程

當咱們把項目遷移到K8S平臺上時，首先咱們須要瞭解的是整個部署的流程，按照這個流程部署，才能避免出現問題，也方便你們理解

製做鏡像

使用Dockerfile製做鏡像，把應用程序、運行環境、文件系統一塊兒打包成一個鏡像，而後推送到Harbor鏡像倉庫中首先在k8s的master節點進行操做

[root@k8s-master ]# git clone https://github.com/zhangdongdong7/php-demo.git
[root@k8s-master ]# cd php-demo
[root@k8s-master php-demo]# ls
deployment.yaml  ingress.yaml  mysql.yaml  namespace.yaml  README.md  service.yaml  wordpress

使用wordpress建立一個博客網站，打開wordpress，編寫Dockerfile構建鏡像，而後推送到一個harbor鏡像倉庫中，能夠看前面章節，harbor鏡像的搭建，這裏是使用的harbor鏡像倉庫地址爲192.168.43.129

[root@k8s-master php-demo]cd wordpress
[root@k8s-master wordpress]# vim Dockerfile 
FROM lizhenliang/nginx-php:latest
MAINTAINER www.ctnrs.com
ADD . /usr/local/nginx/html

[root@k8s-master wordpress]docker login 192.168.43.129

[root@k8s-master wordpress] docker build -t 192.168.43.129/library/php-demo:latest .
  
[root@k8s-master wordpress] docker push 192.168.43.129/library/php-demo:latest

建立控制器管理Pod

回到php-demo目錄編寫yaml，首先部署一個test的命令空間

[root@k8s-master wordpress]# cd ../
[root@k8s-master php-demo]# vim namespace.yaml 
apiVersion: v1
kind: Namespace
metadata:
  name: test
[root@k8s-master php-demo]# kubectl apply -f  namespace.yaml

kubectl get ns
NAME                   STATUS   AGE
default                Active   5h59m
kube-node-lease        Active   5h59m
kube-public            Active   5h59m
kube-system            Active   5h59m
kubernetes-dashboard   Active   5h18m
test                   Active   3s

建立認證

kubectl create secret docker-registry regsecret --docker-server=192.168.43.129 --docker-username=admin --docker-password=Harbor12345 -n test

配置deployment控制器

編寫deployment.yaml控制器,這裏須要把image進行修改爲剛纔推送到Harbor鏡像倉庫中的地址

[root@k8s-master php-demo]# vim deployment.yaml 
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: php-demo
  namespace: test
spec:
  replicas: 2
  selector:
    matchLabels:
      project: www
      app: php-demo
  template:
    metadata:
      labels:
        project: www
        app: php-demo
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: nginx
        image: 192.168.43.129/library/php-demo:latest 
        imagePullPolicy: Always
        ports:
        - containerPort: 80
          name: web
          protocol: TCP
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        resources:
          requests:
            cpu: 0.5
            memory: 256Mi
          limits:
            cpu: 1
            memory: 1Gi
        livenessProbe:
          httpGet:
            path: /status.php
            port: 80

Pod數據持久化

由於是一個靜態的網站,基本不須要作持久化,直接把代碼打包到鏡像中

暴露應用

建立一個service來暴露應用，直接使用的了ingress控制器的方式暴露應用了

[root@k8s-master php-demo]# vim service.yaml
apiVersion: v1
kind: Service
metadata:
  name: php-demo
  namespace: test
spec:
  selector:
    project: www
    app: php-demo
  ports:
  - name: web
    port: 80
    targetPort: 80
      
      
[root@k8s-master php-demo]# kubectl apply -f service.yaml

[root@k8s-master php-demo]# kubectl apply -f deployment.yaml 


[root@master php-demo]# kubectl get pods,svc -n test -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP            NODE     NOMINATED NODE   READINESS GATES
pod/php-demo-65bc56fdb8-grklk   1/1     Running   0          15s   10.244.0.11   master   <none>           <none>
pod/php-demo-65bc56fdb8-td6nv   0/1     Running   0          15s   10.244.2.7    node2    <none>           <none>

NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE     SELECTOR
service/php-demo   ClusterIP   10.0.0.221   <none>        80/TCP    4m16s   app=php-demo,project=www

建立ingress對外發布應用

編寫yaml，首先建立ingress控制器，建立ingress，最後能夠查看pod，svc，ingress的狀態，所有都正常能夠開始下一步，若是有異常可使用kubectl describe命令查看日誌進行排錯

[root@k8s-master java-demo]# kubectl apply -f mandatory.yaml
[root@k8s-master php-demo]# vim ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: php-demo
  namespace: test
spec:
  rules:
    - host: php.ctnrs.com
      http:
        paths:
        - path: /
          backend:
            serviceName: php-demo
            servicePort: 80
[root@k8s-master php-demo]# kubectl apply -f ingress.yaml 

kubectl get pods,svc,ingress -n test -o wide
NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE     NOMINATED NODE   READINESS GATES
pod/php-demo-65bc56fdb8-grklk   1/1     Running   0          2m31s   10.244.0.11   master   <none>           <none>
pod/php-demo-65bc56fdb8-td6nv   1/1     Running   0          2m31s   10.244.2.7    node2    <none>           <none>

NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE     SELECTOR
service/php-demo   ClusterIP   10.0.0.221   <none>        80/TCP    6m32s   app=php-demo,project=www

NAME                          CLASS    HOSTS           ADDRESS   PORTS   AGE
ingress.extensions/php-demo   <none>   php.ctnrs.com             80      60s

能夠在集羣以外找一個數據庫,也能夠在harbor鏡像倉庫安裝

docker run -d -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 daocloud.io/library/mysql:5.7.5
    
docker exec -it mysql:5.7 /bin/bash
mysql -uroot -p$MYSQL_ROOT_PASSWORD
create database wp;
grant all on youmen.* TO 'youmen'@'%' IDENTIFIED BY 'zhoujian20';

# 咱們能夠進入已經運行的pod修改下數據庫ip,最好構建鏡像時候就修改好
# 此處就不演示了,咱們能訪問到錯誤頁面說明服務訪問暴露是沒有問題的

綁定hosts,訪問域名驗證

windows系統，hosts文件地址：C:\Windows\System32\drivers\etc

Mac系統sudo vi /private/etc/hosts 編輯hosts文件，在底部加入域名和ip，用於解析

這個ip地址爲node節點ip地址加入以下命令，而後保存

在瀏覽器中，輸入php.ctnrs.com，會跳轉到初始化設置界面，設置對應的帳號，而後安裝，登陸，而後就能夠編輯文章發佈了，一個簡單的WordPress的php網站搭建完成

Java項目部署流程

製做鏡像

使用Dockerfile製做鏡像，把應用程序、運行環境、文件系統一塊兒打包成一個鏡像，而後推送到Harbor鏡像倉庫中

首先在k8s的master節點進行操做

[root@k8s-master ]# git clone https://github.com/zhangdongdong7/java-demo.git
[root@k8s-master java-demo]# cd java-demo
[root@k8s-master java-demo]# ls
deployment.yaml  ingress.yaml    mysql.yaml      README.md     tomcat-java-demo-master.zipdeploy.yml       mandatory.yaml  namespace.yaml  service.yaml
[root@k8s-master java-demo]# unzip tomcat-java-demo-master.zip
[root@k8s-master java-demo]# cd tomcat-java-demo-master/

安裝環境

[root@k8s-master tomcat-java-demo-master]# yum install java-1.8.0-openjdk maven -y

編譯構建

若是maven構建慢可使用阿里源

vim /etc/maven/settings.xml，大概在（159-164行），更換爲以下代碼

[root@k8s-master tomcat-java-demo-master]# vim /etc/maven/settings.xml    
    ...
    <mirror>
      <id>central</id>
      <mirrorOf>central</mirrorOf>
      <name>aliyun maven</name>
      <url>https://maven.aliyun.com/repository/public</url>
    </mirror>
    ...
    

[root@k8s-master tomcat-java-demo-master]# ls
db  Dockerfile  LICENSE  pom.xml  README.md  src  target

# 這一次咱們提早修改好數據庫配置再生成鏡像
vim src/main/resources/application.yml

在tomcat目錄下建立鏡像

[root@k8s-master tomcat-java-demo-master]# docker login 192.168.73.136
Authenticating with existing credentials…WARNING! 
Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. 
Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin 
Succeeded

[root@k8s-master tomcat-java-demo-master]# docker build -t 192.168.43.129/library/java-demo:latest .
  
[root@k8s-master tomcat-java-demo-master]# docker push 192.168.73.136/test/java-demo:latest

回到上一級java-demo目錄中

[root@k8s-master tomcat-java-demo-master]# cd ../
[root@k8s-master java-demo]# ls
db               deploy.yml    mandatory.yaml  namespace.yaml  service.yaml             tomcat-java-demo-master.zip
deployment.yaml  ingress.yaml  mysql.yaml      README.md       tomcat-java-demo-master
[root@k8s-master java-demo]#

建立一個test的命令空間

[root@k8s-master java-demo]# cat namespace.yaml 
apiVersion: v1
kind: Namespace
metadata:
  name: test namespace.yaml
[root@k8s-master java-demo]# kubectl apply -f namespace.yaml

建立控制器管理Pod

編寫deployment.yaml，建立pods，這裏須要把image進行修改爲剛纔推送到Harbor鏡像倉庫中的地址

[root@k8s-master java-demo]# vim deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-java-demo
  namespace: test
spec:
  replicas: 2
  selector:
    matchLabels:
      project: www
      app: java-demo
  template:
    metadata:
      labels:
        project: www
        app: java-demo
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: tomcat
        image: 192.168.73.136/test/java-demo:latest
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        resources:
          requests:
            cpu: 0.25
            memory: 1Gi
          limits:
            cpu: 1
            memory: 2Gi
        livenessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
[root@k8s-master java-demo]# kubectl adpply -f deployment.yaml

Pod數據持久化

這裏演示的是一個靜態的web網站，基本不須要作持久化，直接把代碼打包到了鏡像中

暴露應用

建立一個service來暴露應用，直接使用的了ingress控制器的方式暴露應用了

[root@k8s-master java-demo]# cat service.yaml
apiVersion: v1
kind: Service
metadata:
  name: tomcat-java-demo
  namespace: test
spec:
  selector:
    project: www
    app: java-demo
  ports:
  - name: web
    port: 80
    targetPort: 8080
[root@k8s-master java-demo]# kubectl apply -f service.yaml

建立ingress對外發布應用

編寫yaml，由於剛纔php項目建立過ingress控制器，所以能夠不用建立，直接建立ingress，最後能夠查看pod，svc，ingress的狀態，所有都正常能夠開始下一步，若是有異常可使用kubectl describe命令查看日誌進行排錯

[root@k8s-master java-demo]# cat ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: tomcat-java-demo 
  namespace: test
spec:
  rules:
    - host: java.ctnrs.com
      http:
        paths:
        - path: /
          backend:
            serviceName: tomcat-java-demo 
            servicePort: 80
[root@k8s-master java-demo]# kubectl apply -f ingress.yaml
[root@k8s-master java-demo]# kubectl get pod,svc,ingress -n test -o wide
NAME                                    READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
pod/php-demo-66d9c64968-4r4vn           1/1     Running   0          24h   10.244.1.73   k8s-node01   <none>           <none>
pod/php-demo-66d9c64968-8zw9s           1/1     Running   0          24h   10.244.2.43   k8s-node02   <none>           <none>
pod/tomcat-java-demo-5f4f64dd4b-tcmtv   1/1     Running   0          24h   10.244.2.42   k8s-node02   <none>           <none>
pod/tomcat-java-demo-5f4f64dd4b-vvx5x   1/1     Running   0          24h   10.244.1.72   k8s-node01   <none>           <none>

NAME                       TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE   SELECTOR
service/php-demo           NodePort    10.1.136.96   <none>        80:32625/TCP   24h   app=php-demo,project=www
service/tomcat-java-demo   ClusterIP   10.1.198.15   <none>        80/TCP         24h   app=java-demo,project=www

NAME                                  HOSTS            ADDRESS   PORTS   AGE
ingress.extensions/php-demo           php.ctnrs.com              80      24h
ingress.extensions/tomcat-java-demo   java.ctnrs.com             80      24h