k8s Pipline CI/CD
1、Pipeline介紹
pipeline是一套jenkins官方提供的插件，它能夠用來在jenkins中實現和集成連續交付java
用戶能夠利用Pipeline的許多功能：node
代碼：pipeline在代碼中實現，一般檢查到源代碼控制，使團隊可以編輯，審查和迭代其交付管道。git
持久：pipeline能夠在Jenkins master的計劃內和計劃外重啓中存活。web
Pausable：在繼續pipeline運行以前，pipeline能夠選擇中止並等待人工輸入或批准。spring
多功能：pipeline支持複雜的實際CD要求，包括並行分叉/鏈接，循環和執行工做的能力。docker
可擴展：Pipeline插件支持其DSL的自定義擴展 和多個與其餘插件集成的選項shell
2、k8s實現集成/部署/交付
3、建立一個gitlab的測試項目（網上找的簡單代碼測試）
一、測試代碼
代碼結構apache
[root@node2 test1]# tree
.
├── pom.xml
└── src
└── main
└── java
└── hello
├── Greeter.java
└── HelloWorld.java
package hello; public class HelloWorld { public static void main(String[] args) { Greeter greeter = new Greeter(); System.out.println(greeter.sayHello()); } }
package hello; public class Greeter { public String sayHello() { return "Hello world!"; } }
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.springframework</groupId> <artifactId>gs-maven</artifactId> <packaging>jar</packaging> <version>0.1.0</version> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.1</version> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> <mainClass>hello.HelloWorld</mainClass> </transformer> </transformers> </configuration> </execution> </executions> </plugin> </plugins> </build> </project>
二、git的測試項目
4、jenkins配置
一、插件安裝
安裝好git、GitLab Plugin、Gitlab Hook Plugin、Gitlab API Plugin、Pipeline、Kubernetes plugin插件。api
二、在jenkins中配置kubernetes
三、簡單測試（建立一個Pipline項目）
測試的Pipline腳本tomcat
def label = "mypod-${UUID.randomUUID().toString()}"
podTemplate(label: 'label', cloud: 'kubernetes') {
node('label') {
stage('Run shell') {
sh 'echo hello world'
}
}
}
四、pipline中配置gi憑證
建立憑證，username是gitlab的帳號，password是gitlab帳號對應的密碼。
五、Pipline中視圖git憑證
六、從新構建jnlp-slave鏡像（添加maven）
mkdir myjenkins cd myjenkins/ wget https://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz tar xf apache-maven-3.5.4-bin.tar.gz [root@node2 myjenkins]# cat Dockerfile FROM jenkins/jnlp-slave:3.27-1 LABEL maintainer="maven@qq.com" ENV MAVEN_HOME /usr/local/maven ENV PATH=$PATH:$MAVEN_HOME/bin COPY apache-maven-3.5.4 /usr/local/maven [root@node2 myjenkins]# docker build -t myjnlp-slave:1.0 ./
七、配置好鏡像倉庫（我使用的aliyun）
八、把dokcer組的gid改爲10000（容器中jenkins帳號的ugid）
groupmod -g10000 docker
chown root.docker /var/run/docker.sock
九、配置gitlab和jenkins的交互（我使用的webhook方式，並設置提交tag就觸發構建）
十、從新更改Pipline腳本，使執行jenkins構建後能直接建立鏡像並提交到鏡像倉庫
Pipline腳本
def label = "mypod-${UUID.randomUUID().toString()}"
def registry = "registry-vpc.cn-hongkong.aliyuncs.com"
def app_name = "javatest"
def namespace = "aliyun-zhang"
def username = "xxxxxxxxxxxxxxxxxx"
def regpass = "xxxxxxxxxxxxxxxxxxx"
podTemplate(label: 'label', cloud: 'kubernetes', containers: [
containerTemplate(
name: 'jnlp',
image: 'myjnlp-slave:1.0'
),
],
volumes: [
hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
hostPathVolume(mountPath: '/usr/bin/docker', hostPath: '/usr/bin/docker')
],)
{
node('label') {
stage('Task') {
stage('拉取代碼') {
git credentialsId: 'gitlab-auth', url: 'http://mygitlab-gitlab-ce.default.svc.cluster.local/root/test1.git'
def mytag = sh returnStdout: true, script: 'git describe --always --tag'
sh "git checkout -b $mytag"
echo "mytag $mytag ${mytag} ----"
}
stage('編譯打包') {
echo "mvn clean package -Dmaven.test.skip=true"
}
stage('構建上傳鏡像') {
def mytag = sh returnStdout: true, script: 'git describe --always --tag'
def image_name = "${app_name}:${mytag}".minus("\n")
echo "image_name $image_name"
sh label: '', script: ''' echo \'
FROM tomcat:latest
ADD pom.xml /usr/local/tomcat/webapps/
\' > Dockerfile
'''
sh """
docker build -t "${registry}/${namespace}/${image_name}" ./
docker login -u ${username} -p \"${regpass}\" ${registry}
docker push ${registry}/${namespace}/${image_name}
"""
}
}
}
}
十一、jenkins構建測試與查看
5、自動部署到k8s集羣
一、jenkins下載kubernetes continuous deploy插件（官方文檔：https://plugins.jenkins.io/kubernetes-cd）
二、配置kubeconfig憑證
添加 kubeconfig 文件，我選擇的是直接粘貼文件內容（master端的.kube/config文件）
三、記錄憑證ID值
四、建立一個deploy.yaml文件，並把文件上傳到項目的git中
[root@k8s-m ~]# cat java-deploy.yaml apiVersion: apps/v1 kind: Deployment metadata: name: mydeploy namespace: default spec: replicas: 2 selector: matchLabels: test_node: k8s-node template: metadata: labels: test_node: k8s-node spec: containers: - name: myjava-server image: tomcat:latest ports: - name: http containerPort: 8080
五、更改Pipline腳本內容
def label = "mypod-${UUID.randomUUID().toString()}"
def registry = "registry-vpc.cn-hongkong.aliyuncs.com"
def app_name = "javatest"
def namespace = "aliyun-zhang"
def username = "xxxxxxxxxxxx"
def regpass = "xxxxxxxxx"
def k8s_auth = "5ce0993e-e2e9-4126-a910-2acd0a77fefb"
podTemplate(label: 'label', cloud: 'kubernetes', containers: [
containerTemplate(
name: 'jnlp',
image: 'myjnlp-slave:1.0'
),
],
volumes: [
hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
hostPathVolume(mountPath: '/usr/bin/docker', hostPath: '/usr/bin/docker')
],)
{
node('label') {
stage('Task') {
stage('拉取代碼') {
git credentialsId: 'gitlab-auth', url: 'http://mygitlab-gitlab-ce.default.svc.cluster.local/root/test1.git'
def mytag = sh returnStdout: true, script: 'git describe --always --tag'
sh "git checkout -b $mytag"
echo "mytag $mytag ${mytag} ----"
}
stage('編譯打包') {
sh "mvn clean package -Dmaven.test.skip=true"
}
stage('構建上傳鏡像') {
def mytag = sh returnStdout: true, script: 'git describe --always --tag'
def image_name = "${app_name}:${mytag}".minus("\n")
echo "image_name $image_name"
sh label: '', script: ''' echo \'
FROM tomcat:latest
ADD target/*.jar /usr/local/tomcat/webapps/
\' > Dockerfile
'''
sh """
docker build -t "${registry}/${namespace}/${image_name}" ./
docker login -u ${username} -p \"${regpass}\" ${registry}
docker push ${registry}/${namespace}/${image_name}
"""
}
stage('部署到K8S'){
def mytag = sh returnStdout: true, script: 'git describe --always --tag'
def image_name = "${app_name}:${mytag}".minus("\n")
sh """
sed -i 's#tomcat:latest#${registry}/${namespace}/${image_name}#' java-deploy.yaml
"""
kubernetesDeploy configs: 'java-deploy.yaml', kubeconfigId: "${k8s_auth}"
}
}
}
}
六、構建測試
我配置了jenkins和gitlab的webhook配置,提交tag後會自動構建到部署到k8s集羣
測試
[root@node2 test1]# echo 123 >> README.md [root@node2 test1]# git commit -a -m '123' [master bda4c6f] 123 1 file changed, 1 insertion(+), 1 deletion(-) [root@node2 test1]# git tag 10.0 [root@node2 test1]# git push origin master 10.0 Username for 'http://10.101.58.237': root Password for 'http://root@10.101.58.237': Counting objects: 5, done. Delta compression using up to 2 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 347 bytes | 0 bytes/s, done. Total 3 (delta 0), reused 0 (delta 0) To http://10.101.58.237/root/test1.git da0fbc3..bda4c6f master -> master * [new tag] 10.0 -> 10.0
jenkins構建過程
k8s集羣查看
[root@k8s-m ~]# kubectl get deploy mydeploy NAME READY UP-TO-DATE AVAILABLE AGE mydeploy 2/2 2 2 64s [root@k8s-m ~]# kubectl get pod NAME READY STATUS RESTARTS AGE mydeploy-746bb8db64-4fpvz 1/1 Running 0 69s mydeploy-746bb8db64-pp7tw 1/1 Running 0 69s [root@k8s-m ~]# kubectl get pod mydeploy-746bb8db64-4fpvz -o yaml|grep image: - image: registry-vpc.cn-hongkong.aliyuncs.com/aliyun-zhang/javatest:10.0 image: registry-vpc.cn-hongkong.aliyuncs.com/aliyun-zhang/javatest:10.0
6、版本回滾
基於k8s鏡像的回滾仍是比較簡單的
回滾上一個版本
kubectl rollout undo deployment/deploy-name
查看歷史版本信息
#查看 deployment 升級歷史 [root@k8s-m ~]# kubectl rollout history deployment/mydeploy deployment.extensions/mydeploy REVISION CHANGE-CAUSE 3 <none> 4 <none> #看歷史版本更加詳細的升級信息 [root@k8s-m ~]# kubectl rollout history deployment/mydeploy --revision=3 deployment.extensions/mydeploy with revision #3 Pod Template: Labels: pod-template-hash=746bb8db64 test_node=k8s-node Containers: myjava-server: Image: registry-vpc.cn-hongkong.aliyuncs.com/aliyun-zhang/javatest:10.0 Port: 8080/TCP Host Port: 0/TCP Environment: <none> Mounts: <none> Volumes: <none>
回滾指定版本
kubectl rollout undo deployment/mydeploy --to-revision=3
回滾指定版本2
直接修改delpoy中的鏡像
7、Deployment升級策略
一、介紹
deployment 有 2 種策略，分別是Recreate和RollingUpdate，RollingUpdate是默認的策略
RollingUpdate也有相對應的升級策略，若是策略設置的不合理，那麼升級的過程就有可能致使服務中斷
Max Unavailable
最多有幾個 pod 處於沒法工做的狀態，默認值是25%
Max Surge
升級過程當中能夠比預設的 pod 的數量多出的個數，默認值是25%
minReadySeconds
等待容器啓動的時間，默認值是 0，單位是：秒，容器運行成功以後直接執行下一步
根據應用啓動時間，設定相應的minReadySeconds，保證應用不中斷
二、查看deploy的更新策略
[root@k8s-m recreate]# kubectl explain deploy.spec.strategy.type KIND: Deployment VERSION: extensions/v1beta1 FIELD: type <string> DESCRIPTION: Type of deployment. Can be "Recreate" or "RollingUpdate". Default is RollingUpdate.
三、recreate策略測試
結論：recreate方式會先中止全部就版本，中止完後才部署新版本
[root@k8s-m ~]# kubectl get pod -w NAME READY STATUS RESTARTS AGE my-app-bb9cc7597-dn5bv 1/1 Running 0 3m36s my-app-bb9cc7597-gqfx4 1/1 Running 0 3m36s my-app-bb9cc7597-p4dfb 1/1 Running 0 3m36s my-app-bb9cc7597-p4dfb 1/1 Terminating 0 3m44s my-app-bb9cc7597-dn5bv 1/1 Terminating 0 3m44s my-app-bb9cc7597-gqfx4 1/1 Terminating 0 3m44s my-app-bb9cc7597-p4dfb 0/1 Terminating 0 3m45s my-app-bb9cc7597-gqfx4 0/1 Terminating 0 3m45s my-app-bb9cc7597-dn5bv 0/1 Terminating 0 3m45s my-app-bb9cc7597-p4dfb 0/1 Terminating 0 3m57s my-app-bb9cc7597-p4dfb 0/1 Terminating 0 3m57s my-app-db47b56bf-kwd2c 0/1 Pending 0 0s my-app-db47b56bf-kwd2c 0/1 Pending 0 0s my-app-db47b56bf-xkqdz 0/1 Pending 0 0s my-app-db47b56bf-5v4r5 0/1 Pending 0 0s my-app-db47b56bf-xkqdz 0/1 Pending 0 0s my-app-db47b56bf-kwd2c 0/1 ContainerCreating 0 0s
四、RollingUpdate策略測試
結論: 一個接一個地以滾動更新方式發佈新版本，滾動的方式能夠定義，即先刪除多少pod在添加多少pod，或者是閒添加在刪除。
[root@k8s-m ~]# kubectl get pod -w NAME READY STATUS RESTARTS AGE my-app-db47b56bf-2c2c6 1/1 Running 0 19s my-app-db47b56bf-k7k5z 1/1 Running 0 19s my-app-db47b56bf-mclj8 1/1 Running 0 19s my-app-bb9cc7597-hgz6r 0/1 Pending 0 0s my-app-bb9cc7597-hgz6r 0/1 Pending 0 0s my-app-bb9cc7597-hgz6r 0/1 ContainerCreating 0 1s my-app-bb9cc7597-hgz6r 0/1 ContainerCreating 0 1s my-app-bb9cc7597-hgz6r 0/1 Running 0 5s my-app-bb9cc7597-hgz6r 1/1 Running 0 6s my-app-db47b56bf-mclj8 1/1 Terminating 0 29s my-app-bb9cc7597-84ngs 0/1 Pending 0 0s my-app-bb9cc7597-84ngs 0/1 Pending 0 0s my-app-bb9cc7597-84ngs 0/1 ContainerCreating 0 1s my-app-bb9cc7597-84ngs 0/1 ContainerCreating 0 2s my-app-db47b56bf-mclj8 0/1 Terminating 0 31s my-app-db47b56bf-mclj8 0/1 Terminating 0 34s my-app-db47b56bf-mclj8 0/1 Terminating 0 34s my-app-bb9cc7597-84ngs 0/1 Running 0 6s my-app-bb9cc7597-84ngs 1/1 Running 0 7s
[root@k8s-m recreate]# kubectl explain deploy.spec.strategy.rollingUpdate KIND: Deployment VERSION: extensions/v1beta1 RESOURCE: rollingUpdate <Object> DESCRIPTION: Rolling update config params. Present only if DeploymentStrategyType = RollingUpdate. Spec to control the desired behavior of rolling update. FIELDS: maxSurge <string> The maximum number of pods that can be scheduled above the desired number of pods. Value can be an absolute number (ex: 5) or a percentage of desired pods (ex: 10%). This can not be 0 if MaxUnavailable is 0. Absolute number is calculated from percentage by rounding up. By default, a value of 1 is used. Example: when this is set to 30%, the new RC can be scaled up immediately when the rolling update starts, such that the total number of old and new pods do not exceed 130% of desired pods. Once old pods have been killed, new RC can be scaled up further, ensuring that total number of pods running at any time during the update is at most 130% of desired pods. maxUnavailable <string> The maximum number of pods that can be unavailable during the update. Value can be an absolute number (ex: 5) or a percentage of desired pods (ex: 10%). Absolute number is calculated from percentage by rounding down. This can not be 0 if MaxSurge is 0. By default, a fixed value of 1 is used. Example: when this is set to 30%, the old RC can be scaled down to 70% of desired pods immediately when the rolling update starts. Once new pods are ready, old RC can be scaled down further, followed by scaling up the new RC, ensuring that the total number of pods available at all times during the update is at least 70% of desired pods.
五、其它更新策略（金絲雀等）
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
