DEVOPS技術實踐_12:建立持續集成的管道

時間  2019-11-12
標籤 devops 技術 實踐 建立 持續 集成 管道 简体版
原文   原文鏈接

持續集成不只包含了Jenkins或者相關其它的CI工具，也包含了包含代碼如何控制，採用的什麼分支策略等。
不一樣的組織可能採用不一樣的類型的策略來完成CI，策略類型和項目的類型的有很大的關係。node

一 分支策略

1.1 本實驗分支

分支可以有效的對代碼較好的管理，也是可以把工做的代碼和開發環境的代碼隔離的有效方式。主要有三種的分支策略類型
– master分支
– integration 分支
– feature 分支git

1.master分支web

master分支也叫作生產分支，該分支的代碼所有是通過測試OK的代碼。任何沒有通過測試OK的代碼都不會出如今該分支上。正則表達式

2. Integration分支shell

集成分支也叫作mainline分支，該分支存放着全部功能的集成，並進行構建和測試。開發人員不能在該分支上直接開發，然而開發人員能夠在集成分支上建立feaure分支，並在上面進行開發。app

3. feature分支webapp

最後是feature分支，這也是開發人員所在的分支，能夠有多個feature分支。maven

下圖展示了經常使用的分支策略。ide

在feature分支或者集成分支上進行構建、靜態代碼分析、集成測試等，假如代碼測試經過，那麼把打好的包上傳到Artifactory(二進制庫)工具

1.2 CI的管道

在Jenkins中建立多分支管道有如下步驟：
（1）從版本控制系統中拉取代碼，並觸發一個事件（CI管道的初始化）

（2）執行靜態分析並把結果上傳到SonarQube,假如發現的bug超過在quality gate的閥值，那麼管道構建將會失敗。

（3）執行集成測試和在Jenkins上公佈單元測試報告。

（4）上傳構建的artifacts，並附加一些相關的屬性信息。

持續集成的目的就是把持續構建、測試（單元測試和集成測試），靜態代碼性能分析，和上傳構建的artifacts到二進制庫，這個過程是徹底自動化的。 而且報告每一步是否成功或者失敗。

1.3 持續集成工具集下面是持續集成的要用到的工具

Technology Characteristic
Java Primary programming language used for coding
Maven Build tool
Junit Unit testing and integration testing tools
Jenkins Continuous Integration tool
GitHub Version control system
SoanrQube Static code analysis tool
Artifactory Binary repository mananger

 

二 構建一個集成管道

建立執行集成管道，將會執行下面步驟：

  1. 在GitHub上建立源代碼庫
  2. 建立Jenkinsfile去描述構建、單元測試、靜態代碼分析、集成測試、和上分佈到構建的二進制文件到Artifactory.
  3. 利用Docker去產生構建的agents去運行咱們的持續集成管道。
  4. 在jenkins中建立多分支的管道。

2.1 node塊

本實驗依然使用master構建

node('master'){

}

而且將會在master的節點上執行如下步驟：

  • 執行構建
  • 執行單元測試並推送單元測試報告
  • 執行代碼分析並把它的結果上傳到SonarQube
  • 執行集成測試並把發佈集成測試報告
  • 推送artifacts到Artifactory

2.2 獲取源代碼

從版本控制器中下載最新的源代碼，代碼以下

stage(‘Poll’) {
scm checkout
}

2.3 管道代碼去執行構建和單元測試

sh ‘mvn clean verify -DskipITs=true’;
junit ‘**/target/surefire-reports/TEST-.xml’
archive ‘target/.jar’

-DskipITs=ture表明跳過集成測試僅執行構建和單元測試,junit ‘/target/surefire-reports/TEST-*.xml’ 命令是開啓Jenkins去公佈單元測試報告,‘/target/surefire-reports/TEST-*.xml’ 此處存放的是單元測試報告所存在的目錄位置。

管道代碼去執行靜態代碼分析

經過運行Maven的命令，就是一個簡單的shell腳本去執行靜態代碼的分析，這是使用針對maven的SonarQube scanner工具來進行配置的。

sh ‘mvn clean verify sonar:sonar -Dsonar.projectName=example-project

-Dsonar.projectKey=example-project -Dsonar.projectVersion=$BUILD_NUMBER’;

-Dsonar.projectName=example-project 這是傳送給SonarQube項目名稱的一個選項，測試的全部結果將會顯示在projectName=example-project下

類似的-Dsonar.projectKey=example-project，這個選項是針對Maven的SonarQube Sanner去確認projectkey=example-project

-Dsonar.projectVersion=$BUILD_NUMBER這個選項是指關聯到Jenkins的構建分析的號碼，並把它上傳到SonarQube。$BUILD_NUMBER是針對構建序列的環境變量。

 2.4 靜態代碼分析

stage('Static Code Analysis'){
    sh 'mvn clean verify sonar:sonar -Dsonar.host.url=http://192.168.132.133:9000 -Dsonar.login=278c0c5ddadca63754f0fa9ce50ba99c20214fb5';
  }

管道代碼去執行集成測試

-Dsurefire.skip=true是跳過單元測試，僅執行集成測試的junit ‘/target/failsafe-reports/TEST-*.xml’ 命令是開啓jenkins去發佈Junit unit 測試報告到Jenkins管道頁面。/target/failsafe-reports/TEST-*.xml 這是產生的集成報告所在的目錄。

集成測試階段所包含的代碼以下：

stage (‘Integration Test’){
sh ‘mvn clean verify -Dsurefire.skip=true’;
junit ‘**/target/failsafe-reports/TEST-*.xml’
archive ‘target/*.jar’
}

注意：針對單元測試和集成測試必須安裝Jenkins Junit Plugin.

2.5  管道代碼去發佈構建的artifacts到Artifactory

去上傳構建的artifacts到Artifactory，使用是File Specs。代碼以下

「files」: [
{
「pattern」: 「[Mandatory]」,
「target」: 「[Mandatory]」,
「props」: 「[Optional]」,
「recursive」: 「[Optional, Default: ‘true’]」,
「flat」 : 「[Optional, Default: ‘true’]」,
「regexp」: 「[Optional, Default: ‘false’]」
}
]

 上面各參數的代碼解釋：

Parameters Condition Description
pattern [Mandatory] 指定應該上傳到Artifactory的本地artifacts的的路徑，能夠經過反掩碼或者正則表達式指定多個artifacts，假如使用正則表達式，須要使用\逃義保留的字符（好比.,？等，從版本2.9.0以後全部的路徑分隔符均以/進行分隔，包含winows系統
target [Mandatory] 指定artifactory目錄的路徑，格式：[repository_name]/[repository_path]，假如模式的如何是以/結尾，那麼b就會被認爲是目錄。假如repo-name/a/b，那麼上傳的文件就會以b在artifactory命名，上傳路徑建議使用｛1｝，｛2｝，｛3｝…這樣代替，具體參考(https://www.jfrog.com/confluence/display/RTF/Using+File+Specs#UsingFileSpecs-UsingPlaceholders).
Props [Optional] 是以Key=value的方式指定上傳屬性的值，若是有多個值，用，分隔。如 key1=value1;key2=value21,value22;key3=value3
flat [Default:true] 假如指定爲true，那麼上傳的artifactory裏面的源系統文件層級就會被忽略，設置爲true，纔會保留源系統的文件層級
recursive [Default:true] ｜若是爲true，那麼artifacts將會收集上傳源目錄的子目錄，若是爲faluse，僅會指定的源目錄會被上傳  
regexp [Default:false] 命令將會依據模式進行解釋，就是正則表達式不會生效。若是爲false，將會依正則表達式進行解釋

下面是File Specs代碼示例：

def server = Artifactory.server 'Default Artifactory server'
    def uploadSpec = """{
      "files": [
        {
          "pattern": "target/hello-0.0.1.war",
          "target": "example-project/${BUILD_NUMBER}/",
          "props": "Integration-Tested=Yes;Performance-Tested=No"
        }
      ]
    }"""

下面是對代碼的解釋：

Parameters Description
def server = Artifactory.server ‘Default Artifactory Server’ This line tells Jenkins to use the existing Artifactory server configured in Jenkins. In our example, it is the default Artifactory server.
Default Artifactory Server This is the name of the Artifactory server configured inside Jenkins
「pattern」: 「target/hello-0.0.1.war」 This line of code will look at a file named hello-0.0.1.war inside the directory target, which is again inside the Jenkins workspace directory
「target」: 「example-project/${BUILD_NUMBER}/」, This line of code will try to upload the build artifacts to the Artifactory repository named helloworld-greeting-project. It will place the inside a folder named after the build number inside the Artifactory repository.
${BUILD_NUMBER} The Jenkins environment variable for the build number.
「props」: 「Integration-Tested=Yes;Performance-Testd=No」 This code creates two key-value paris and assigns them to the uploaded artifacts. These key-value paris can be used as labels for code promotion in Artifactory

2.6 合併pipeline代碼

node('master') {
  stage('Poll') {
    checkout scm
  }
  stage('Build & Unit test'){
    sh 'mvn clean verify -DskipITs=true';
    junit '**/target/surefire-reports/TEST-*.xml'
    archive 'target/*.jar'
  }
  stage('Static Code Analysis'){
    sh 'mvn clean verify sonar:sonar -Dsonar.host.url=http://192.168.132.133:9000 -Dsonar.login=278c0c5ddadca63754f0fa9ce50ba99c20214fb5';
  }
  stage ('Integration Test'){
    sh 'mvn clean verify -Dsurefire.skip=true';
    junit '**/target/failsafe-reports/TEST-*.xml'
    archive 'target/*.jar'
  }
  stage ('Publish'){
    def server = Artifactory.server 'Default Artifactory server'
    def uploadSpec = """{
      "files": [
        {
          "pattern": "target/hello-0.0.1.war",
          "target": "example-project/${BUILD_NUMBER}/",
          "props": "Integration-Tested=Yes;Performance-Tested=No"
        }
      ]
    }"""
    server.upload(uploadSpec)
  }
}

三 jenkins流水線任務

3.1 添加一個pipeline的任務

3.2 流水線配置

3.3 嘗試構建

第一次構建失敗

沒有mvn命令，是由於master節點上，沒有安裝mvn工具，同時在jenkinsfile也沒有使用參數調用mvn

直接安裝一個mvn

3.4 再次構建

成功

3.5 查看sonar

3.7 查看artifcatory

構建的管道基本知足實驗要求

3.8 查看日誌

Started by user darren ning
Obtained Jenkinsfile from git http://192.168.132.132/root/hello-world-greeting.git
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /root/.jenkins/workspace/pipelin-jenkinsfile
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Poll)
[Pipeline] checkout
using credential e3e48ed7-dbce-4642-bb18-28e0c71ab962
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url http://192.168.132.132/root/hello-world-greeting.git # timeout=10
Fetching upstream changes from http://192.168.132.132/root/hello-world-greeting.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 
 > git fetch --tags --progress http://192.168.132.132/root/hello-world-greeting.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision d8a642e6869e108fcada44e0dd2de7b5e882fe1d (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f d8a642e6869e108fcada44e0dd2de7b5e882fe1d
Commit message: "Update Jenkinsfile"
 > git rev-list --no-walk d8a642e6869e108fcada44e0dd2de7b5e882fe1d # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build & Unit test)
[Pipeline] sh
+ mvn clean verify -DskipITs=true
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Hello Maven Webapp 0.0.1
[INFO] ------------------------------------------------------------------------
。。。。。。
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 3 source files to /root/.jenkins/workspace/pipelin-jenkinsfile/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ hello ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /root/.jenkins/workspace/pipelin-jenkinsfile/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ hello ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 3 source files to /root/.jenkins/workspace/pipelin-jenkinsfile/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.19:test (default-test) @ hello ---
。。。。。。。
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running hello.DateTimeTest
3
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec - in hello.DateTimeTest
Running hello.MessageTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in hello.MessageTest

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-war-plugin:2.1.1:war (default-war) @ hello ---

[INFO] Packaging webapp
[INFO] Assembling webapp [hello] in [/root/.jenkins/workspace/pipelin-jenkinsfile/target/hello-0.0.1]
[INFO] Processing war project
[INFO] Copying webapp resources [/root/.jenkins/workspace/pipelin-jenkinsfile/src/main/webapp]
[INFO] Webapp assembled in [19 msecs]
[INFO] Building war: /root/.jenkins/workspace/pipelin-jenkinsfile/target/hello-0.0.1.war
[INFO] WEB-INF/web.xml already added, skipping
[INFO] 
[INFO] --- maven-failsafe-plugin:2.19:integration-test (default) @ hello ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-failsafe-plugin:2.19:verify (default) @ hello ---
[INFO] Tests are skipped.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 47.640s
[INFO] Finished at: Sun Oct 27 03:54:45 EDT 2019
[INFO] Final Memory: 18M/159M
[INFO] ------------------------------------------------------------------------
[Pipeline] junit
Recording test results
[Pipeline] archive
The archive step is deprecated, please use archiveArtifacts instead.
No files found to archive for pattern "target/*.jar"; continuing.
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Static Code Analysis)
[Pipeline] sh
+ mvn clean verify sonar:sonar -Dsonar.host.url=http://192.168.132.133:9000 -Dsonar.login=278c0c5ddadca63754f0fa9ce50ba99c20214fb5
[INFO] Scanning for projects...、
。。。。。。
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Hello Maven Webapp 0.0.1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ hello ---
[INFO] Deleting /root/.jenkins/workspace/pipelin-jenkinsfile/target
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ hello ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /root/.jenkins/workspace/pipelin-jenkinsfile/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ hello ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 3 source files to /root/.jenkins/workspace/pipelin-jenkinsfile/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ hello ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /root/.jenkins/workspace/pipelin-jenkinsfile/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ hello ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 3 source files to /root/.jenkins/workspace/pipelin-jenkinsfile/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.19:test (default-test) @ hello ---
。。。。。。。
[INFO] User cache: /root/.sonar/cache
[INFO] SonarQube version: 6.7.7
[INFO] Default locale: "en_US", source code encoding: "UTF-8"
[INFO] Publish mode
[INFO] Load global settings
[INFO] Load global settings (done) | time=420ms
[INFO] Server id: 8AB2C9A6-AWyV7Ii4gK-saCFcgAZW
[INFO] User cache: /root/.sonar/cache
[INFO] Load plugins index
[INFO] Load plugins index (done) | time=123ms
。。。。。
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:11.733s
[INFO] Finished at: Sun Oct 27 03:55:58 EDT 2019
[INFO] Final Memory: 34M/218M
[INFO] ------------------------------------------------------------------------
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Integration Test)
[Pipeline] sh
+ mvn clean verify -Dsurefire.skip=true
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Hello Maven Webapp 0.0.1
。。。。。。。
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

 

參考：https://edu.csdn.net/course/play/9051/188016  臧雪園老師視頻課程

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程