K8S集羣中部署jenkins

時間  2021-01-02
標籤 java node git docker apache centos api 架構 ssh maven 欄目 負載均衡 简体版
原文   原文鏈接

本文介紹在k8s環境中進行jenkins server的部署和配置。Jenkins是一個開源的、功能強大的持續集成和持續構建工具，採用master和salve架構，咱們經過將jenkins集成環境部署在k8s集羣中，能夠實現jenkins slave按需建立、動態的伸縮。同時也提供了在k8s環境中應用的持續部署解決方案。java

1、準備docker鏡像文件

一、編譯jenkins server docker鏡像，默認的jenkis鏡像已包含jdk，版本爲1.8.0_171node

# cat dockerfile
FROM jenkins
MAINTAINER ylw@fjhb.cn
ENV MAVEN_HOME /usr/local/maven
ENV JAVA_HOME  /usr/local/java
ENV CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
ENV PATH ${JAVA_HOME}/bin:${MAVEN_HOME}/bin:${PATH}

COPY apache-maven-3.5.4 /usr/local/maven
USER root
RUN mkdir -p /usr/local/maven/repository  &&  ln -s /usr/java/jdk1.8.0_171 /usr/local/java
# docker build -t harbor.59iedu.com/fjhb/jenkins:2018-08-12-v1 . 
# docker push harbor.59iedu.com/fjhb/jenkins:2018-08-12-v1

K8S集羣中部署jenkins
二、編譯jenkins slave鏡像
能夠根據實際狀況配置maven內網私服nexus，私服能夠避免編譯過程當中經過公網下載依賴的jar包，配置私服須要把對應的setting.xml文件打包到apache-maven-3.5.4/conf目錄下；
libltdl.so.7文件的獲取路徑爲操做系統路徑/usr/lib64/libltdl.so.7（其實是個軟連接，須要copy出來重命名）
slave.jar文件的獲取路徑爲http://jenkins-server/jnlpJars/slave.jargit

# cat Dockerfile
FROM openshift/base-centos7
MAINTAINER ylw@fjhb.cn
COPY apache-maven-3.5.4 /usr/local/maven
COPY jdk1.8.0_171       /usr/local/java
COPY kubectl            /usr/local/bin/kubectl
COPY libltdl.so.7 /usr/lib64/libltdl.so.7
COPY slave.jar /usr/share/jenkins/slave.jar 
COPY jenkins-slave /usr/local/bin/jenkins-slave

ENV HOME /home/jenkins
ENV AGENT_WORKDIR=/home/jenkins/agent
ENV JAVA_HOME /usr/local/java
ENV MAVEN_HOME /usr/local/maven/
ENV CLASSPATH .:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
ENV PATH ${JAVA_HOME}/bin:${MAVEN_HOME}/bin:${PATH} 
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"

RUN  chmod 755 /usr/share/jenkins \
  && chmod 644 /usr/share/jenkins/slave.jar 

RUN mkdir -p /home/jenkins/.jenkins \
 && mkdir -p ${AGENT_WORKDIR} \
 && yum -y install git subversion sshpass

VOLUME /home/jenkins/.jenkins
VOLUME ${AGENT_WORKDIR}
WORKDIR /home/jenkins

USER root
ENTRYPOINT ["jenkins-slave"]
# docker build -t harbor.59iedu.com/fjhb/jenkins-slave-toolkit:2018-08-10-v1 . 
# docker push harbor.59iedu.com/fjhb/jenkins-slave-toolkit:2018-08-10-v1

K8S集羣中部署jenkins

2、建立jenkins server

一、建立pv和pvcdocker

# cat pv.yaml 
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-master-vol
spec:
  capacity:
    storage: 5Gi 
  accessModes:
  - ReadWriteMany 
  nfs: 
    path: /home/jenkins
    server: 192.168.115.6
  persistentVolumeReclaimPolicy: Recycle 
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: maven-repository
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteMany
  nfs:
    path: /home/maven
    server: 192.168.115.6
  persistentVolumeReclaimPolicy: Recycle

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: jenkins-master-claim
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: maven-repository-claim
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi

二、建立deployment和serviceapache

# cat deploy.yaml 
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins-master
spec:
  template:
    metadata:
      labels:
        name: jenkins-master
    spec:
      securityContext:
        fsGroup: 1000
      containers:
        - name: jenkins-master
          image: harbor.59iedu.com/fjhb/jenkins:2018-08-12-v1
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
              name: http
            - containerPort: 50000
              name: agent
          volumeMounts:
            - name: jenkins-master-vol
              mountPath: /var/jenkins_home
            - name: maven-repository
              mountPath: /opt/maven/repository
            - name: docker
              mountPath: /usr/bin/docker
            - name: docker-sock
              mountPath: /var/run/docker.sock
      volumes:
        - name: jenkins-master-vol
          persistentVolumeClaim:
            claimName: jenkins-master-claim
        - name: maven-repository
          persistentVolumeClaim:
            claimName: maven-repository-claim
        - name: docker
          hostPath:
            path: /usr/bin/docker
        - name: docker-sock
          hostPath:
            path: /var/run/docker.sock
      serviceAccount: "jenkins-master"
      imagePullSecrets:
        - name: harborsecret

---
apiVersion: v1
kind: Service
metadata:
  name: jenkins-master
spec:
  type: NodePort
  ports:
    - port: 8080
      name: http
      targetPort: 8080
      nodePort: 8452
    - port: 50000
      name: agent
      nodePort: 50000
      targetPort: 50000 
  selector:
    name: jenkins-master

三、rbac受權centos

# cat sa.yaml 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-master
  namespace: default

---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins-master

rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins-master

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins-master
subjects:
- kind: ServiceAccount
  name: jenkins-master
  namespace: default

四、default sa的rbac受權api

# cat default-sa.yaml 
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: default-role

rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: default-rolebinding

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: default-role 
subjects:
- kind: ServiceAccount
  name: default
  namespace: default

K8S集羣中部署jenkins
K8S集羣中部署jenkins

3、初始化jenkins server

一、經過祕鑰解鎖jenkins（本文爲了方便局域網其餘主機訪問，在vmware上配置了nat規則）
K8S集羣中部署jenkins
二、配置代理
K8S集羣中部署jenkins
K8S集羣中部署jenkins
三、安裝插件
K8S集羣中部署jenkins
K8S集羣中部署jenkins
四、建立管理員帳號
K8S集羣中部署jenkins架構

4、配置jenkins server

一、系統管理 —— 系統設置 —— 新增一個雲」kubernetes」
K8S集羣中部署jenkins
Kubernetes URL: 輸入api-server的地址
Jenkins URL: 輸入jenkins server的服務名,端口8080
Jenkins Tunnel: 指的是slave鏈接master的端口，默認是50000
K8S集羣中部署jenkins
上圖pod的模板名稱爲jenkins-slave，Container的模板名稱爲jnlp。這裏有很是重要的兩點要注意：
當Container的模板名稱爲jnlp的時候，jenkins-slave纔會使用下面配置的docker鏡像來啓動pod，若是不爲jnlp，則會使用默認的鏡像jenkins/jnlp-slave:alpinessh

當使用自定義的docker鏡像來啓動jenkins slave pod的時候，下面的command to run（默認值是 sh -c）和arguments to pass to the command(默認值是cat)兩個值須要清空。不然會出現jenkins slave jnlp鏈接不上master的狀況，嘗試100次鏈接以後銷燬pod，而後再建立一個pod繼續嘗試鏈接，無限循環。maven

二、系統管理 —— Configure Global Security
確認jnlp agent的端口默認爲50000，若是有修改，要保障這裏的配置及前面部署deployment、service的端口配置、前文的雲環境Jenkins Tunnel設置保持一致
K8S集羣中部署jenkins

三、系統管理 —— Global Tool Configuration
在這裏設置對應的工具及環境變量，爲了不沒必要要的問題，前面經過dockerfile把jenkins server 和jenkins slave的環境變量調整成一致, java目錄經過軟鏈接的方式實現。
K8S集羣中部署jenkins
K8S集羣中部署jenkins

四、系統管理 —— 管理插件
推薦安裝的幾個插件：maven、 gitlab 、subversion、pipeline、Kubernetes Continuous Deploy、Publish Over SSH
K8S集羣中部署jenkins完成插件安裝後須要對jenkins server進行重啓操做，能夠點擊「系統管理 」——「準備關機」來完成重啓操做，至此咱們就完成了jenkins server在k8s環境中的部署和配置工做，下文開始介紹使用jenkins完成項目構建和發佈。

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程