kubernetes實踐：helm安裝harbor

時間 2021-01-31

標籤 node git github redis docker shell json api 瀏覽器服務器欄目 Git 简体版

原文原文鏈接

GitHub:Helm Chart for Harbornode

1. 添加harbor的helm庫

helm repo add harbor https://helm.goharbor.io

2. 將harbor下載到本地

helm fetch harbor/harbor
tar xf harbor-1.1.1.tgz

3. 自定義配置

默認的values.yaml基本不須要多大改動，只有極個別的須要自定義修改
具體改什麼須要根據自動需求，能夠查看GitHub上面的配置列表configurationgit

根據本身的特殊需求建立一個新的values文件，覆蓋掉以前的值github

cat new-values.yaml

expose:
  type: ingress
  tls:
    enabled: true
  ingress:
    hosts:
      core: harbor.mytest.io
      notary: notary.mytest.io

externalURL: https://harbor.mytest.io

persistence:
  enabled: true
  resourcePolicy: "keep"
  persistentVolumeClaim:
    registry:
      storageClass: "harbor-data"
    chartmuseum:
      storageClass: "harbor-data"
    jobservice:
      storageClass: "harbor-data"
    database:
      storageClass: "harbor-data"
    redis:
      storageClass: "harbor-data"

4. 建立一個harbor的存儲類

存儲類見《kubernetes集羣使用nfs-client實現storageclass》redis

cat harbor-data-sc.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: harbor-data
provisioner: fuseim.pri/ifs

5. 建立harbor

kubectl create -f harbor-data-sc.yaml
helm install --name harbor -f new-values.yaml --namespace kube-ops ./harbor

6. 查看ingress

kubectl get ingresses. -n kube-ops
NAME                    HOSTS                               ADDRESS                                                                                                 PORTS     AGE
harbor-harbor-ingress   harbor.mytest.io,notary.mytest.io   172.18.1.14,.....,172.18.1.9   80, 443   101m

7. 本地訪問

在k8s任意一節點，配置本地/etc/hosts文件docker

kube-ip harbor.mytest.io

使用瀏覽器訪問https://harbor.mytest.ioshell

帳號：adminjson

密碼：Harbor12345api

8. docker本地訪問配置

使用 docker cli 來進行 pull/push 鏡像，因爲上面咱們安裝的時候經過 Ingress 來暴露的 Harbor 的服務，並且強制使用了 https，因此若是咱們要在終端中使用咱們這裏的私有倉庫的話，就須要配置上相應的證書：瀏覽器

docker login harbor.mytest.io
Username: admin
Password: 
Error response from daemon: Get https://harbor.mytest.io/v2/: x509: certificate signed by unknown authority

這是由於咱們沒有提供證書文件，咱們將使用到的ca.crt文件複製到/etc/docker/certs.d/harbor.mytest.io目錄下面，若是該目錄不存在，則建立它。ca.crt 這個證書文件咱們能夠經過 Ingress 中使用的 Secret 資源對象來提供：服務器

kubectl get secret harbor-harbor-ingress -n kube-ops -o yaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJ...
  tls.crt: LS0tLS1CRdJ...
  tls.key: LS0tLS1CRUd...
kind: Secret
metadata:
  creationTimestamp: "2019-07-09T07:30:49Z"
  labels:
    app: harbor
    chart: harbor
    heritage: Tiller
    release: harbor
  name: harbor-harbor-ingress
  namespace: kube-ops
  resourceVersion: "1805594"
  selfLink: /api/v1/namespaces/kube-ops/secrets/harbor-harbor-ingress
  uid: 79634a0e-a21b-11e9-984b-0017fa037437
type: kubernetes.io/tls

其中 data 區域中 ca.crt 對應的值就是咱們須要證書，不過須要注意還須要作一個 base64 的解碼，這樣證書配置上之後就能夠正常訪問了。

kubectl get secret harbor-harbor-ingress -n kube-ops -o jsonpath="{.data.ca\.crt}"|base64 --decode

-----BEGIN CERTIFICATE-----
MIIC9DCCAdygAwIBAgIQEZIgi3AhzJ8htXYR3fzC+jANBgkqhkiG9w0BAQsFADAU
MRIwEAYDVQQDEwloYXJib3ItY2EwHhcNMTkwNzA5MDc1NTI5WhcNMjAwNzA4MDc1
NTI5WjAUMRIwEAYDVQQDEwloYXJib3ItY2EwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQCyG22aCWdqcsZd39t6O/qbIG/DUGQhC3VxBsTvR6XIXMUu+8vl
WTKk8qMO0bWI8cQwilthJDA50h5POsAUYBBwWNtlE4lR/1FIx2tKSy42Fnd61GVr
ThG6xh4mjp1v5OlerCqCXSVum695xFAi65Pdg2qsw4bCOCeaG5wynDQ3W0+T9pPJ
z8/Xp59rRP+Y3ulTYv5bdY6rczKNGL432SsutyI+TrQvo2NApOFbrZwMg66WTuL2
+n3YPUoQYHzT5RumURowSbrORS7Tuls3fp3rjIrkZ3Z5rY2YHikpkj03vOEdYV0P
7Jxah7gzuVqpSHRzr8yG9aJ+arzsG6BxaTnTAgMBAAGjQjBAMA4GA1UdDwEB/wQE
AwICpDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDwYDVR0TAQH/BAUw
AwEB/zANBgkqhkiG9w0BAQsFAAOCAQEArBtwin0wswSWA1lCshYszq4+n8acv1id
NMhPzql3iqs+dFWNLAnVYKrT6acOPTOJ2O/pYcpap0/6EMMErSiXpKattjB6b2Sg
+X/ExJYJHust8cgx8rtfjbNWdLgrTIL7tMRQ8dd3NZ/WpiYK3A33wRi4Zjm0JY4e
JhAukw8j6h0QxBaedXhnzNd3i+5w92R/QC616vhjP0TPgeosVnQB02R7fTg5BXwl
vxHazvNMmn3BfReRCXiJrfiqAxzjNN25yyMlnQiGsBMbxbw19Q/sWAy9k/PXgOT5
PJ0pgCZTF0cxXP3d7mI2Lld53z+J6ojqtvg/oyaLzM3m1Af0bC0Kaw==
-----END CERTIFICATE-----

手動建立docker下面的harbor專用的證書文件夾

sudo mkdir -pv /etc/docker/certs.d/harbor.mytest.io

將上面生成的密鑰寫入到文件夾裏面的ca.crt中

sudo -i
cat >> /etc/docker/certs.d/harbor.mytest.io/ca.crt<< EOF
-----BEGIN CERTIFICATE-----
MIIC9DCCAdygAwIBAgIQEZIgi3AhzJ8htXYR3fzC+jANBgkqhkiG9w0BAQsFADAU
MRIwEAYDVQQDEwloYXJib3ItY2EwHhcNMTkwNzA5MDc1NTI5WhcNMjAwNzA4MDc1
NTI5WjAUMRIwEAYDVQQDEwloYXJib3ItY2EwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQCyG22aCWdqcsZd39t6O/qbIG/DUGQhC3VxBsTvR6XIXMUu+8vl
WTKk8qMO0bWI8cQwilthJDA50h5POsAUYBBwWNtlE4lR/1FIx2tKSy42Fnd61GVr
ThG6xh4mjp1v5OlerCqCXSVum695xFAi65Pdg2qsw4bCOCeaG5wynDQ3W0+T9pPJ
z8/Xp59rRP+Y3ulTYv5bdY6rczKNGL432SsutyI+TrQvo2NApOFbrZwMg66WTuL2
+n3YPUoQYHzT5RumURowSbrORS7Tuls3fp3rjIrkZ3Z5rY2YHikpkj03vOEdYV0P
7Jxah7gzuVqpSHRzr8yG9aJ+arzsG6BxaTnTAgMBAAGjQjBAMA4GA1UdDwEB/wQE
AwICpDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDwYDVR0TAQH/BAUw
AwEB/zANBgkqhkiG9w0BAQsFAAOCAQEArBtwin0wswSWA1lCshYszq4+n8acv1id
NMhPzql3iqs+dFWNLAnVYKrT6acOPTOJ2O/pYcpap0/6EMMErSiXpKattjB6b2Sg
+X/ExJYJHust8cgx8rtfjbNWdLgrTIL7tMRQ8dd3NZ/WpiYK3A33wRi4Zjm0JY4e
JhAukw8j6h0QxBaedXhnzNd3i+5w92R/QC616vhjP0TPgeosVnQB02R7fTg5BXwl
vxHazvNMmn3BfReRCXiJrfiqAxzjNN25yyMlnQiGsBMbxbw19Q/sWAy9k/PXgOT5
PJ0pgCZTF0cxXP3d7mI2Lld53z+J6ojqtvg/oyaLzM3m1Af0bC0Kaw==
-----END CERTIFICATE-----
EOF

重啓docker

systemctl restart docker

9. docker login登陸

使用docker-cli登陸測試harbor.mytest.io

docker login harbor.mytest.io
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

10. 上傳個鏡像試試

拉取個busybox鏡像到本地

docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
8e674ad76dce: Pull complete 
Digest: sha256:c94cf1b87ccb80f2e6414ef913c748b105060debda482058d2b8d0fce39f11b9
Status: Downloaded newer image for busybox:latest

將busybox的tags修改爲harbor.mytest.io/library/busybox:latest，library是harbor默認的庫

docker tag busybox:latest harbor.mytest.io/library/busybox:latest

使用docker push將修改tag後的鏡像推送到harbor

docker push harbor.mytest.io/library/busybox:latest
The push refers to repository [harbor.mytest.io/library/busybox]
6194458b07fc: Pushed 
latest: digest: sha256:bf510723d2cd2d4e3f5ce7e93bf1e52c8fd76831995ac3bd3f90ecc866643aff size: 527

圖圖圖示：

11. 注意

在結合jenkins使用cicd過程當中須要注意：

每臺node節點須要配置dns解析，不然docker沒法訪問
須要將剛纔上述的密鑰推送到所有node節點上，否則會報錯，cicd構建過程會失敗
能夠在pipeline的過程當中制定nodeselector，定義其中一臺node專門運行jenkins，這臺node節點須要打上labels，jenkins經過nodeseletor=labels，能夠在一臺服務器上報錯images了