apollo須要使用數據庫,若是是mysql,須要版本在5.6以上:
本次環境mysql部署在10.4.7.11上,使用mysql5.7,爲測試簡單起見,各環境數據庫使用同一個,不作隔離node
rpm -Uvh https://dev.mysql.com/get/mysql80-community-release-el7-1.noarch.rpm yum -y install yum-utils yum-config-manager --disable mysql80-community yum-config-manager --enable mysql57-community yum install mysql-server -y
cat >/etc/my.cnf <<'EOF' [mysqld] character_set_server = utf8mb4 collation_server = utf8mb4_general_ci init_connect = "SET NAMES 'utf8mb4'" [mysql] default-character-set = utf8mb4 EOF
systemctl start mysqld systemctl enable mysqld mysql -u root -p`grep password /var/log/messages|awk '{print $NF}'` # 修改密碼 > set global validate_password_policy=0; > set global validate_password_length=1; > set password=password('123456'); > flush privileges; # 檢查字符集:須要四個都是utf8mb4 > \s
configdb初始化腳本
portal初始化腳本mysql
wget -O apolloconfig.sql https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/configdb/V1.0.0__initialization.sql # 導入sql文件 mysql -uroot -p123456 < apolloconfig.sql # 檢查是否導入成功 mysql -uroot -p123456 -e "show databases;"|grep ApolloConfigDB
mysql -uroot -p123456 > grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigDB.* to 'apollo'@'10.4.7.%' identified by "123456"; # 修改數據 > use ApolloConfigDB > update ServerConfig set Value='http://apollo-config.zq.com/eureka' where Id=1;
vi /var/named/zq.com.zone mysql A 10.4.7.11 apollo-config A 10.4.7.10 apollo-admin A 10.4.7.10 apollo-portal A 10.4.7.10 # 重啓並驗證 systemctl restart named dig -t A apollo-config.zq.com @10.4.7.11 +short
因爲portal使用的是另外一個portaldb,咱們須要在數據庫中新建portdb,並初始化git
wget -O apollo-portal.sql https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/portaldb/V1.0.0__initialization.sql # 導入sql文件 mysql -uroot -p123456 < apollo-portal.sql # 檢查是否導入成功 mysql -uroot -p123456 -e "show databases;"|grep ApolloPortalDB
都使用apollo
用戶來管理數據庫是爲了方便,若是有相關的安全考慮能夠給config和portal分別使用不一樣的數據庫帳號github
mysql -uroot -p123456 > grant INSERT,DELETE,UPDATE,SELECT on ApolloPortalDB.* to "apollo"@"10.4.7.%" identified by "123456"; # 更新部門名 > update ApolloPortalDB.ServerConfig set Value='[{"orgId":"zq01","orgName":"研發部"},{"orgId":"zq02","orgName":"運維部"}]' where Id=2;
操做在7.200
上完成web
wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-configservice-1.5.1-github.zip mkdir /data/dockerfile/apollo-configservice unzip -o apollo-configservice-1.5.1-github.zip -d /data/dockerfile/apollo-configservice/
cd /data/dockerfile/apollo-configservice/config # 修改數據庫鏈接地址 sed -i 's#fill-in-the-correct-server#mysql.zq.com#g' application-github.properties # 修改數據庫鏈接用戶和密碼 sed -i 's#FillInCorrectUser#apollo#g' application-github.properties sed -i 's#FillInCorrectPassword#123456#g' application-github.properties # 查看結果 config]# egrep -v "^#|$^" application-github.properties spring.datasource.url = jdbc:mysql://mysql.zq.com:3306/ApolloConfigDB?characterEncoding=utf8 spring.datasource.username = apollo spring.datasource.password = 123456
程序中自帶的start.sh
啓動腳本時不適用與K8S運行,所以須要專門下載他們提供的K8S內使用的腳本spring
# 1.從官網下載啓動腳本 cd /data/dockerfile/apollo-configservice/scripts/ wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/apollo-on-kubernetes/apollo-config-server/scripts/startup-kubernetes.sh # 2. 添加一行使用主機名的變量 sed -i '5i APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)' startup-kubernetes.sh # 3.根據須要修改下jvm限制
cd .. cat >Dockerfile <<'EOF' FROM harbor.zq.com/base/jre8:8u112 ENV VERSION 1.5.1 RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\ echo "Asia/Shanghai" > /etc/timezone ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar ADD config/ /apollo-configservice/config ADD scripts/ /apollo-configservice/scripts CMD ["sh","/apollo-configservice/scripts/startup-kubernetes.sh"] EOF
docker build . -t harbor.zq.com/infra/apollo-configservice:v1.5.1 docker push harbor.zq.com/infra/apollo-configservice:v1.5.1
mkdir /data/k8s-yaml/apollo-configservice cd /data/k8s-yaml/apollo-configservice
給configservice建立cm資源的清單的目的是方便修改
其實裏面的內容就是前面修改的application-github.properties
文件
若是肯定不會修改,能夠不建立此cm,直接寫死配置到docker鏡像中docker
cat >cm.yaml <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: apollo-configservice-cm namespace: infra data: application-github.properties: | # DataSource spring.datasource.url = jdbc:mysql://mysql.zq.com:3306/ApolloConfigDB?characterEncoding=utf8 spring.datasource.username = apollo spring.datasource.password = 123456 eureka.service.url = http://apollo-config.zq.com/eureka app.properties: | appId=100003171 EOF
在同一個configmap資源中,能夠添加多個配置文件,上述配置就有兩個,分別是:
application-github.properties
和app.properties
數據庫
cat >dp.yaml <<'EOF' kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-configservice namespace: infra labels: name: apollo-configservice spec: replicas: 1 selector: matchLabels: name: apollo-configservice template: metadata: labels: app: apollo-configservice name: apollo-configservice spec: volumes: - name: configmap-volume configMap: name: apollo-configservice-cm containers: - name: apollo-configservice image: harbor.zq.com/infra/apollo-configservice:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-configservice/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 EOF
cat >svc.yaml <<'EOF' kind: Service apiVersion: v1 metadata: name: apollo-configservice namespace: infra spec: ports: - protocol: TCP port: 8080 targetPort: 8080 selector: app: apollo-configservice EOF
cat >ingress.yaml <<'EOF' kind: Ingress apiVersion: extensions/v1beta1 metadata: name: apollo-configservice namespace: infra spec: rules: - host: apollo-config.zq.com http: paths: - path: / backend: serviceName: apollo-configservice servicePort: 8080 EOF
service中不必定必須暴露8080,分配的clusterIP中全部的端口均可以
但ingress中的servicePort必定要與service中暴露的端口匹配
kubectl create -f http://k8s-yaml.zq.com/apollo-configservice/cm.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-configservice/dp.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-configservice/svc.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-configservice/ingress.yaml
kubectl -n infra get pod|grep apollo-config # 檢查命令 kubectl -n infra logs apollo-configservice-64fc749978-9nz5h --tail=4
須要等到eureka啓動之後才能夠,接下來使用瀏覽器訪問apollo-config.zq.com
操做在7.200
上完成
wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-adminservice-1.5.1-github.zip mkdir /data/dockerfile/apollo-adminservice unzip -o apollo-adminservice-1.5.1-github.zip -d /data/dockerfile/apollo-adminservice/
因爲使用了configmap資源將配置文件掛載出來了,因此不在修改配置文件,如需修改配置文件,請參考部署apollo-configservice時候的修改方法:
程序中自帶的start.sh
啓動腳本時不適用與K8S運行,所以須要專門下載他們提供的K8S內使用的腳本
# 1.從官網下載啓動腳本 cd /data/dockerfile/apollo-adminservice/scripts/ wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/apollo-on-kubernetes/apollo-admin-server/scripts/startup-kubernetes.sh # 2. 添加一行使用主機名的變量 sed -i '5i APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)' startup-kubernetes.sh # 3.修改端口爲8080 sed -i 's#8090#8080#g' startup-kubernetes.sh
官方配置文件端口改成8090的目的是虛擬機部署的時候端口不衝突
但咱們用K8S部署,會給他單獨的clusterIP,因此不用擔憂端口重複
cd .. cat >Dockerfile <<'EOF' FROM stanleyws/jre8:8u112 ENV VERSION 1.5.1 RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\ echo "Asia/Shanghai" > /etc/timezone ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar ADD config/ /apollo-adminservice/config ADD scripts/ /apollo-adminservice/scripts CMD ["/bin/bash","/apollo-adminservice/scripts/startup-kubernetes.sh"] EOF
因爲要使用cm配置資源,所以就不改config中的配置了
docker build . -t harbor.zq.com/infra/apollo-adminservice:v1.5.1 docker push harbor.zq.com/infra/apollo-adminservice:v1.5.1
adminservice向註冊中心註冊服務,不直接對外提供服務,所以不須要暴露端口,只須要cm資源和dp資源
mkdir /data/k8s-yaml/apollo-adminservice cd /data/k8s-yaml/apollo-adminservice
cat >cm.yaml <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: apollo-adminservice-cm namespace: infra data: application-github.properties: | # DataSource spring.datasource.url = jdbc:mysql://mysql.zq.com:3306/ApolloConfigDB?characterEncoding=utf8 spring.datasource.username = apollo spring.datasource.password = 123456 eureka.service.url = http://apollo-config.zq.com/eureka app.properties: | appId=100003172 EOF
注意每一個服務的appId都不會同樣哦
cat >dp.yaml <<'EOF' kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-adminservice namespace: infra labels: name: apollo-adminservice spec: replicas: 1 selector: matchLabels: name: apollo-adminservice template: metadata: labels: app: apollo-adminservice name: apollo-adminservice spec: volumes: - name: configmap-volume configMap: name: apollo-adminservice-cm containers: - name: apollo-adminservice image: harbor.zq.com/infra/apollo-adminservice:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-adminservice/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 EOF
kubectl create -f http://k8s-yaml.zq.com/apollo-adminservice/cm.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-adminservice/dp.yaml
~]# kubectl -n infra get pod|grep apollo-admin apollo-adminservice-6cd4fcfdc8-2drnq 1/1 Running 0 9s # 檢查命令 kubectl -n infra logs apollo-configservice-6cd4fcfdc8-2drnq --tail=4
經過 apollo-config.zq.com 檢查是否註冊到了eureka:
已經順利的註冊到了註冊中心中。
wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-portal-1.5.1-github.zip mkdir /data/dockerfile/apollo-portal unzip -o apollo-portal-1.5.1-github.zip -d /data/dockerfile/apollo-portal/
因爲使用concigmap資源,故不在這裏修改
注意若是要修改的話,要分別修改兩個文件
apollo-env.properties
修改數據庫配置apollo-env.properties
修改支持的環境列表# 1.從官網下載啓動腳本 cd /data/dockerfile/apollo-portal/scripts/ wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/apollo-on-kubernetes/apollo-portal-server/scripts/startup-kubernetes.sh # 2. 添加一行使用主機名的變量 sed -i '5i APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)' startup-kubernetes.sh # 3.修改端口爲8080 sed -i 's#8070#8080#g' startup-kubernetes.sh
cd /data/dockerfile/apollo-portal/ cat >Dockerfile <<'EOF' FROM stanleyws/jre8:8u112 ENV VERSION 1.5.1 RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\ echo "Asia/Shanghai" > /etc/timezone ADD apollo-portal-${VERSION}.jar /apollo-portal/apollo-portal.jar ADD config/ /apollo-portal/config ADD scripts/ /apollo-portal/scripts CMD ["/bin/bash","/apollo-portal/scripts/startup-kubernetes.sh"] EOF
docker build . -t harbor.zq.com/infra/apollo-portal:v1.5.1 docker push harbor.zq.com/infra/apollo-portal:v1.5.1
mkdir /data/k8s-yaml/apollo-portal cd /data/k8s-yaml/apollo-portal
cat >cm.yaml <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: apollo-portal-cm namespace: infra data: application-github.properties: | # DataSource spring.datasource.url = jdbc:mysql://mysql.zq.com:3306/ApolloPortalDB?characterEncoding=utf8 spring.datasource.username = apollo spring.datasource.password = 123456 app.properties: | appId=100003173 apollo-env.properties: | dev.meta=http://apollo-config.zq.com EOF
這裏暫時只管理一個環境,等跑通了之後,再演示多環境問題
cat >dp.yaml <<'EOF' kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-portal namespace: infra labels: name: apollo-portal spec: replicas: 1 selector: matchLabels: name: apollo-portal template: metadata: labels: app: apollo-portal name: apollo-portal spec: volumes: - name: configmap-volume configMap: name: apollo-portal-cm containers: - name: apollo-portal image: harbor.zq.com/infra/apollo-portal:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-portal/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 EOF
cat >svc.yaml <<'EOF' kind: Service apiVersion: v1 metadata: name: apollo-portal namespace: infra spec: ports: - protocol: TCP port: 8080 targetPort: 8080 selector: app: apollo-portal EOF
cat >ingress.yaml <<'EOF' kind: Ingress apiVersion: extensions/v1beta1 metadata: name: apollo-portal namespace: infra spec: rules: - host: apollo-portal.zq.com http: paths: - path: / backend: serviceName: apollo-portal servicePort: 8080 EOF
kubectl create -f http://k8s-yaml.zq.com/apollo-portal/cm.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-portal/dp.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-portal/svc.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-portal/ingress.yaml
因爲前面已經一塊兒添加了域名解析,所以portal建立好後不須要在添加域名解析,直接瀏覽器登陸驗證
網頁:apollo-portal.zq.com
默認用戶名:apollo
默認密碼:admin
登陸成功後,立馬修改密碼爲apollo123
到此,apollo的三個組件都已經交付到k8s裏了。
使用配置中心,須要開發對代碼進行調整,將一些配置,經過變量的形式配置到apollo中,服務經過配置中心來獲取具體的配置
項目屬性:
AppId:dubbo-demo-service
應用名稱:dubbo服務提供者
部門:研發部
爲新項目添加配置以下:
key | value | 備註 |
---|---|---|
dubbo.registry | zookeeper://zk1.zq.com:2181 | 註冊中心地址 |
dubbo.port | 20880 | dubbo服務監聽端口 |
發佈後效果圖以下:
仍是使用以前的流水線,可是使用分支爲apollo的代碼進行打包,參數以下:
參數名 | 參數值 |
---|---|
app_name | dubbo-demo-service |
image_name | app/dubbo-demo-service |
git_repo | https://gitee.com/noah-luo/dubbo-demo-service.git |
git_ver | apollo |
add_tag | 200512_0746 |
mvn_dir | ./ |
target_dir | ./dubbo-server/target |
mvn_cmd | mvn clean package -Dmaven.test.skip=true |
base_image | base/jre8:8u112 |
maven | 3.6.1 |
修改dp.yaml資源配置清單
C_OPTS
,以便指定配置中心vim /data/k8s-yaml/dubbo-server/dp.yaml #----------原內容---------- spec: containers: - name: dubbo-demo-service image: harbor.zq.com/app/dubbo-demo-service:master_200509_0800 ports: - containerPort: 20880 protocol: TCP env: - name: JAR_BALL value: dubbo-server.jar #----------新內容---------- spec: containers: - name: dubbo-demo-service image: harbor.zq.com/app/dubbo-demo-service:apollo_200512_0746 ports: - containerPort: 20880 protocol: TCP env: - name: JAR_BALL value: dubbo-server.jar - name: C_OPTS value: -Denv=dev -Dapollo.meta=http://apollo-config.zq.com
應用資源配置清單:
kubectl apply -f http://k8s-yaml.zq.com/dubbo-server/dp.yaml
項目屬性:
AppId:dubbo-demo-web
應用名稱:dubbo服務消費者
部門:運維部
爲新項目添加配置以下:
key | value | 備註 |
---|---|---|
dubbo.registry | zookeeper://zk1.zq.com:2181 | 註冊中心地址 |
發佈後效果圖以下:
略
仍是使用以前的流水線,可是使用分支爲apollo的代碼進行打包,參數以下:
參數名 | 參數值 |
---|---|
app_name | dubbo-demo-consumer |
image_name | app/dubbo-demo-consumer |
git_repo | git@gitee.com:noah-luo/dubbo-demo-web.git |
git_ver | apollo |
add_tag | 200512_0801 |
mvn_dir | ./ |
target_dir | ./dubbo-client/target |
mvn_cmd | mvn clean package -Dmaven.test.skip=true |
base_image | base/jre8:8u112 |
maven | 3.6.1 |
構建完成後,修改資源配置清單並應用:
修改dp.yaml資源配置清單
C_OPTS
,以便指定配置中心vim /data/k8s-yaml/dubbo-consumer/dp.yaml #----------原內容---------- spec: containers: - name: dubbo-demo-consumer image: harbor.zq.com/app/dubbo-demo-consumer:master_200506_1430 ports: - containerPort: 8080 protocol: TCP - containerPort: 20880 protocol: TCP env: - name: JAR_BALL value: dubbo-client.jar #----------新內容---------- spec: containers: - name: dubbo-demo-consumer image: harbor.zq.com/app/dubbo-demo-consumer:apollo_200512_0801 ports: - containerPort: 8080 protocol: TCP - containerPort: 20880 protocol: TCP env: - name: JAR_BALL value: dubbo-client.jar - name: C_OPTS value: -Denv=dev -Dapollo.meta=http://apollo-config.zq.com
應用資源配置清單:
kubectl apply -f http://k8s-yaml.zq.com/dubbo-consumer/dp.yaml
管理機上,修改dubbo-monitor的dp資源的使用的cm資源
set -i 's#dubbo-monitor-cm-pro#dubbo-monitor-cm#g' /data/k8s-yaml/dubbo-monitor/dp-cm.yaml
任意node節點應用資源
kubectl apply -f http://k8s-yaml.zq.com/dubbo-monitor/dp.yaml
登陸dubbo-monitor查看
訪問http://dubbo-monitor.zq.com/
瀏覽器查看
訪問http://dubbo-demo.zq.com/hello?name=lg
apollo中看實例列表