Jenkins工做相關

Jenkins比較消耗資源，bug也比較多，公司後期會轉向其餘系統，暫時有些東東記錄在這裏。

輪詢沒法檢測新提交的問題

現象是同時選中「Build when a change is pushed to GitHub」和 「Poll SCM", git有提交時，不會觸發構建。

一般這種狀況使用hook便可，在代碼目錄的hooks子目錄配置post-commit便可。可是我這邊不能徹底控制git，因此只能用輪詢的方法，Jenkins的輪詢比較低效，常常出現內存溢出等不穩定狀況，也須要外來腳本進行干預，爲此用python腳本：

#!/usr/bin/python
# -*- coding: utf-8 -*-

import logging
import logging.handlers
import time
import re
import os.path
import subprocess32
import sys
sys.path.append('../lib/')


from requests.auth import HTTPDigestAuth

dial_logger = logging.getLogger('MyLogger')
dial_logger.setLevel(logging.DEBUG)
fh = logging.handlers.RotatingFileHandler(
    '../logs/dial_log_server.log',
    maxBytes=10000000,
    backupCount=5)
fh.setLevel(logging.DEBUG)
formatter = logging.Formatter(u'%(asctime)s [%(levelname)s] %(message)s')
fh.setFormatter(formatter)
dial_logger.addHandler(fh)
dial_logger.info("Application starting! \n")


while True:

    for line in open("../conf/url.csv"):
        
        try:
        
            now = time.strftime("%Y-%m-%d_%H:%M:%S", time.gmtime())
            print(now)
    
            name, git_url, branch, jenkins_url = line.strip().split(',')
            # print(name, git_url, branch, jenkins_url)
            dial_logger.info("Beginning Scan {0}! \n".format(name))
            dial_logger.info("{0} {1} {2}! \n".format(git_url, branch, jenkins_url))
            cmd = "git ls-remote -h " + git_url
            result = subprocess32.check_output(cmd,shell=True)
            #print(result)
            version = re.findall(r'(\S+)\s+{0}'.format(branch), result, re.MULTILINE)[0]
            dial_logger.info("current version: {0}! \n".format(version))
            file_name = '/tmp/{0}'.format(name)
    
            if os.path.exists(file_name):
                old_version = open(file_name).read().strip()
                # print(old_version, version)
                dial_logger.info("old version: {0}! \n".format(old_version))
                if old_version == version:
                    dial_logger.info("Don not call {0} \n".format(name))
                    print("Don not call {0} \n".format(name))
                    continue
    
            f = open(file_name,'w')
            f.write(version)
            f.close()
            
            dial_logger.info("Call {0} \n".format(jenkins_url))
            cmd = "curl --user xurongzhong:123456 -s {0} &".format(jenkins_url)
            # print(cmd)
            dial_logger.info("command: {0}! \n".format(cmd))
            subprocess32.call(cmd, shell=True)
            time.sleep(1.5)
            
        except Exception as e:
            dial_logger.error(str(e), exc_info=True)        

    dial_logger.info("End Scan! \n")
    time.sleep(3 * 60)

注意點：

* curl後面的接的地址不要有換行。

* subprocess32是爲了適應centos 6上面的python2.6.6

使用說明：

1，腳本dial_log_server.py放置在bin目錄，同時創建同級目錄：conf和logs。

2，bin目錄中增長啓停腳本：

shutdown.sh

#!/bin/sh

APP_MAIN=dial_log_server.py
PID=0

getPID(){
    pythonps=`ps aux | grep $APP_MAIN | grep -v grep`
    if [ -n "$pythonps" ]; then
        PID=`echo $pythonps | awk '{print $2}'`
    else
        PID=0
    fi
}

shutdown(){
    getPID
    echo "================================================================================================================"
    if [ $PID -ne 0 ]; then
        echo -n "Stopping $APP_MAIN(PID=$PID)..."
        kill -9 $PID
        if [ $? -eq 0 ]; then
            echo "[Success]"
            echo "================================================================================================================"
        else
            echo "[Failed]"
            echo "================================================================================================================"
        fi
        getPID
        if [ $PID -ne 0 ]; then
            shutdown
        fi
    else
        echo "$APP_MAIN is not running"
        echo "================================================================================================================"
    fi
}

shutdown
exit 0

startup.sh

#!/bin/sh

APP_MAIN=dial_log_server.py
APP_LOG=logs
PID=0

getPID(){
    pythonps=`ps aux | grep $APP_MAIN | grep -v grep`
    if [ -n "$pythonps" ]; then
        PID=`echo $pythonps | awk '{print $2}'`
    else
        PID=0
    fi
}


startup(){
    getPID
    echo "================================================================================================================"
    if [ $PID -ne 0 ]; then
        echo "$APP_MAIN already started(PID=$PID)"
        echo "================================================================================================================"
    else
        echo -n "Starting $APP_MAIN"
         if [ ! -d "../$APP_LOG" ]; then
            mkdir "../$APP_LOG"
         fi
        nohup python $APP_MAIN > ../$APP_LOG/nohup.log 2>&1 &
        sleep 2
        getPID
        if [ $PID -ne 0 ]; then
            echo "(PID=$PID)...[Success]"
            echo "================================================================================================================"
        else
            echo "[Failed]"
            echo "================================================================================================================"
        fi
    fi
}

startup

3，conf目錄生成配置文件：

Advertisement_Manage,ssh://root@172.17.100.19/srv/repos/Advertisement_Manage,refs/heads/master,http://172.17.100.18:8080/job/Advertisement_Web_New/build?token=123456
ads-ad-local,ssh://root@172.17.100.19/srv/repos/ads-ad,refs/heads/master,http://172.17.100.18:8080/job/ads-ad-local/build?token=123457
ads-ad-client-local,ssh://root@172.17.100.19/srv/repos/ads-ad,refs/heads/master,http://172.17.100.18:8080/job/ads-ad-client-local/build?token=123458

4，Jenkins對應項目"觸發遠程構建"的「身份驗證令牌」。

5，後續規劃：

* 基於ConfigParser的配置文件。

* 增長Jenkins無反應時重啓。

* 其餘輔助功能。

Jenkins動輒內存不足

修改/etc/sysconfig/jenkins,  設置以下或者更高的值：

JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Xmx4096m -XX:MaxPermSize=2048m"

jenkins的系統配置中，全局屬性，環境變量 MAVEN_OPTS

-Xmx4096m -XX:MaxPermSize=2048m

Maven項目配置，全局MAVEN_OPTS

-Xmx4096m -XX:MaxPermSize=2048m

/etc/profile

export JAVA_OPTS="-XX:MaxPermSize=2048m -Xms4096m -Xmx4096m"
export GRADLE_OPTS="-XX:MaxPermSize=2048m"
export SONAR_RUNNER_OPTS="-Xmx4096m -XX:MaxPermSize=2048m"

增長Findbugs靜態分析

在pom.xml的build部分增長以下內容：

                                <plugin>
                                    <groupId>org.codehaus.mojo</groupId>
                                    <artifactId>findbugs-maven-plugin</artifactId>
                                    <version>3.0.1</version>
                                    <configuration>
                                        <findbugsXmlOutput>true</findbugsXmlOutput>
                                        <findbugsXmlWithMessages>true</findbugsXmlWithMessages>
                                        <xmlOutput>true</xmlOutput>
                                        <onlyAnalyze>nl.berg.packt.FindBugs_all.candles.*</onlyAnalyze>
                                        <effort>Max</effort>
                                   </configuration>
                                </plugin>

注意：

* Findbugs分析很消耗內存，須要配置好jvm參數。

* findbugsXmlOutput等標籤所有要爲小寫，一些文檔有坑，部分爲大寫，會有以下報錯：

[ERROR] No plugin found for prefix 'findBugs' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/var/lib/jenkins/.m2/repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]

* Jenkins的上面選中」Publish FindBugs analysis results「，編譯參數增長：" findbugs:findbugs "

好比：

clean package findbugs:findbugs cobertura:cobertura -Denv=dev -Ddubbo.protocol.port=28999

Jenkins 基於Cobertura的代碼覆蓋率支持

在pom.xml的build部分增長以下內容：

                                <plugin>
                                        <groupId>org.codehaus.mojo</groupId>
                                        <artifactId>cobertura-maven-plugin</artifactId>
                                        <version>2.6</version>
                                        <configuration>
                                        <formats>
                                                <format>xml</format>
                                                <format>html</format>
                                        </formats>
                                        </configuration>
                                </plugin>

注意Jenkins不能在頁面上展現覆蓋率，須要下載html文件。好比：/var/lib/jenkins/workspace/ads-ad-local/ads-ad/target/site/cobertura目錄。

展現結果仍是挺漂亮的，好比：