使用Velero Restic快速完成雲原生應用遷移至ACK集羣

本文記錄使用Velero Restic快速完成雲原生應用遷移至ACK集羣的實踐過程。

0. 實踐步驟概覽

（1）建立GKE集羣（或自建Kubernetes集羣）
（2）在GKE集羣上部署示例應用Jenkins Application並執行一個構建任務
（3）建立ACK集羣
（4）在ACK集羣中部署Minio Service用於應用遷移時數據中轉服務
（5）在GKE上部署Velero並備份整個Jenkins Application
（6）把Jenkins Application使用的容器鏡像同步到阿里雲容器鏡像倉庫
（7）在ACK上建立Jenkins Application所使用的StorageClass
（8）在ACK上部署Velero並恢復整個Jenkins Application
（9）在ACK上替換Jenkins Application所使用的容器鏡像
（10）驗證Jenkins Application可否正常提供服務及其構建任務是否存在

1. 環境物料清單

GKE集羣（或自建Kubernetes集羣）
ACK集羣

2. 建立GKE集羣並部署示例應用Jenkins Application

(1)在MarketPlace中找到Jenkins應用並配置部署到GKE集羣的jenkins命名空間下：

(2)應用正在完成部署：

(3)查看Ingress並訪問jenkins服務：

(4)Jenkins應用的初始化配置：

(5)Jenkin應用上建立一個名爲gke-to-ack的任務並執行構建：

（6）Jenkins應用的k8s資源列表以下

$ kubectl -n jenkins get deploy
NAME                         DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
jenkins-jenkins-deployment   1         1         1            1           177m
$ kubectl -n jenkins get svc
NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
jenkins-jenkins-agents-connector   ClusterIP   10.39.241.75    <none>        50000/TCP        177m
jenkins-jenkins-ui                 NodePort    10.39.253.141   <none>        8080:31759/TCP   177m
$ kubectl -n jenkins get ing
NAME                 HOSTS   ADDRESS        PORTS     AGE
jenkins-jenkins-ui   *       xx.xx.xx.xx   80, 443   177m
$ kubectl -n jenkins get pvc
NAME                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
jenkins-jenkins-pvc   Bound    pvc-e64eacc4-a482-11e9-bfa9-42010a8000da   8Gi        RWO            standard       177m
$ kubectl -n jenkins get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                       STORAGECLASS   REASON   AGE
pvc-e64eacc4-a482-11e9-bfa9-42010a8000da   8Gi        RWO            Delete           Bound    jenkins/jenkins-jenkins-pvc                                 standard                177m

3. 建立ACK集羣並部署Minio對象存儲應用

(1) 自定義替換minio-deploy.yaml中的 MINIO_ACCESS_KEY 和 MINIO_SECRET_KEY值並部署：

kubectl apply -f minio-deploy.yaml

---
apiVersion: v1
kind: Namespace
metadata:
  name: minio

---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  namespace: minio
  name: minio
  labels:
    component: minio
spec:
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        component: minio
    spec:
      volumes:
      - name: storage
        emptyDir: {}
      - name: config
        emptyDir: {}
      containers:
      - name: minio
        image: minio/minio:latest
        imagePullPolicy: IfNotPresent
        args:
        - server
        - /storage
        - --config-dir=/config
        env:
        - name: MINIO_ACCESS_KEY
          value: "<your MINIO_ACCESS_KEY>"
        - name: MINIO_SECRET_KEY
          value: "your MINIO_SECRET_KEY"
        ports:
        - containerPort: 9000
        volumeMounts:
        - name: storage
          mountPath: "/storage"
        - name: config
          mountPath: "/config"

---
apiVersion: v1
kind: Service
metadata:
  namespace: minio
  name: minio
  labels:
    component: minio
spec:
  # ClusterIP is recommended for production environments.
  # Change to NodePort if needed per documentation,
  # but only if you run Minio in a test/trial environment, for example with Minikube.
  type: LoadBalancer
  ports:
    - port: 9000
      targetPort: 9000
      protocol: TCP
  selector:
    component: minio

(2) 查看minio LoadBalancer信息並登錄應用：

$ kubectl -n minio get svc
NAME    TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)          AGE
minio   LoadBalancer   172.21.2.216   xxx.xxx.xxx.xx   9000:30912/TCP   2d4h

(3) 替換minio-job.yaml中的minio_server_url minino_access_key minio_secret_key並運行job在minio中建立名爲velero的bucket：

kubectl apply -f minio-job.yaml

apiVersion: batch/v1
kind: Job
metadata:
  namespace: minio1
  name: minio1-setup
  labels:
    component: minio1
spec:
  template:
    metadata:
      name: minio1-setup
    spec:
      restartPolicy: OnFailure
      volumes:
      - name: config
        emptyDir: {}
      containers:
      - name: mc
        image: minio/mc:latest
        imagePullPolicy: IfNotPresent
        command:
        - /bin/sh
        - -c
        - "mc --config-dir=/config config host add velero <your minio_server_url> <your minino_access_key> <your minio_secret_key> && mc --config-dir=/config mb -p velero/velero"
        volumeMounts:
        - name: config
          mountPath: "/config"

(4) 查看bucket是否建立成功：

4. 在GKE和ACK上部署velero

(1) 安裝velero client：
請從official release下載最新版本的velero客戶端：

(2) 安裝velero server：
建立credentials-velero文件並設置aws_access_key_id 與 aws_secret_access_key的值

[default]
aws_access_key_id = <your minio_access_key>
aws_secret_access_key = <your minio_secret_key>

ACK集羣中請替換minio_server_url的值並指定image參數部署velero server

velero install     --provider aws    --image registry.cn-hangzhou.aliyuncs.com/acs/velero:latest  --bucket velero     --secret-file ./credentials-velero     --use-volume-snapshots=false     --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=<your minio_server_url> --use-restic     --wait

GKE中請替換minio_server_url的值並部署velero server

velero install     --provider aws    --bucket velero     --secret-file ./credentials-velero     --use-volume-snapshots=false     --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=<your minio_server_url> --use-restic     --wait

5. 在GKE集羣中備份Jenkins Application

(1)在備份帶volume信息的pod以前，咱們要經過給pod加annotation來告訴velero哪些pod須要包含volume數據， 查看jenkins應用的Deployment資源：

$ kubectl -n jenkins get deploy jenkins-jenkins-deployment -oyaml

(2)爲pod加annotation:

$ kubectl -n jenkins get po
NAME                                          READY   STATUS      RESTARTS   AGE
jenkins-deployer-zh5p4                        0/1     Completed   0          109m
jenkins-jenkins-deployment-7df86c64d4-tqqlr   1/1     Running     0          109m
$ kubectl -n jenkins annotate pod/jenkins-jenkins-deployment-7df86c64d4-tqqlr backup.velero.io/backup-volumes=jenkins-jenkins-pvc
pod/jenkins-jenkins-deployment-7df86c64d4-tqqlr annotated

(3)建立備份

$ velero backup create gcloud-jenkins-backup-restic --include-namespaces jenkins --wait
Backup request "gcloud-jenkins-backup-restic" submitted successfully.
Waiting for backup to complete. You may safely press ctrl-c to stop waiting - your backup will continue in the background.
.......................
Backup completed with status: Completed. You may check for more information using the commands `velero backup describe gcloud-jenkins-backup-restic` and `velero backup logs gcloud-jenkins-backup-restic`.

(4) 查看備份

$ ./velero backup get
NAME                           STATUS      CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
gcloud-jenkins-backup-restic   Completed   2019-07-12 18:48:48 +0800 +08   29d       default            <none>

（5）登陸Minio Server查看

5. 同步（或批量）遷移容器鏡像

把Jenkins應用使用的容器鏡像導入到阿里雲容器鏡像倉庫，導入到的地址爲：

registry.cn-hangzhou.aliyuncs.com/haoshuwei/jenkins:2.150.3

批量遷移可參考：https://github.com/AliyunContainerService/sync-repo.git

6. 在ACK集羣中建立Jenkins應用所使用的StorageClass standard

$ kubectl apply -f storageclass.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: standard
provisioner: alicloud/disk
parameters:
  type: cloud
reclaimPolicy: Delete

7. 在ACK集羣中恢復Jenkins Application

查看備份信息：

$ velero  backup get
NAME                           STATUS      CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
gcloud-jenkins-backup-restic   Completed   2019-07-12 18:48:48 +0800 CST   29d       default            <none>

恢復應用：

$ velero restore create --from-backup gcloud-jenkins-backup-restic

此處須要編輯deployment修改image爲registry.cn-hangzhou.aliyuncs.com/haoshuwei/jenkins:2.150.3

查看restore的狀態：

$ velero restore describe gcloud-jenkins-backup-restic-20190712190536
$ velero restore logs gcloud-jenkins-backup-restic-20190712190536

restore完成後的狀態爲：

$ velero restore get
NAME                                          BACKUP                         STATUS      WARNINGS   ERRORS   CREATED                         SELECTOR
gcloud-jenkins-backup-restic-20190712190536   gcloud-jenkins-backup-restic   Completed   0          0        2019-07-12 19:05:36 +0800 CST   <none>

8. 查看ACK集羣上jenkins應用的ingress並訪問服務進行驗證

$ kubectl -n jenkins get ing
NAME                 HOSTS   ADDRESS          PORTS     AGE
jenkins-jenkins-ui   *       xx.xx.xx.xx   80, 443   56m

至此，一個帶pv存儲的jenkins應用被完整地從GKE遷移到ACK上。

原文連接 本文爲雲棲社區原創內容，未經容許不得轉載。