kubernetes環境下私有倉庫搭建

前期在客戶那裏搭建了基本運行環境，鑑於不少企業的環境都是內部網沒法鏈接外部，所以搭建私有倉庫是逃避不開的問題，按照網上的步驟搭建，雖然遇到一些問題，但還好都算容易解決了，下面大體把步驟記錄一下便於下次去客戶那裏更新。

首先在須要在啓動registry Pod的機器上把registry images獲取下來

docker pull registry

而後在生成幾個構建persistence volumn(pv), persistence volumn claim(pvc),以及registry rc和service的文件

[root@k8s-master registry]# cat pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv01
  release: stable
spec:
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  nfs:
    path: /k8s/test
    server: 10.182.168.99

 

[root@k8s-master registry]# cat pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclaim2
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 20G

 

[root@k8s-master registry]# cat registry-rc.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: registry
  labels:
    name: registry
spec:
  replicas: 1
  selector:
    name: registry
  template:
    metadata:
      labels:
        name: registry
    spec:
      containers:
      - name: registry
        image: registry
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5000
        volumeMounts:
        - mountPath: "/var/lib/registry"
          name: mypd
      volumes:
      - name: mypd
        persistentVolumeClaim:
          claimName: myclaim2

[root@k8s-master registry]# cat registry-srv.yaml 
apiVersion: v1
kind: Service
metadata:
    name: registry
    labels:
      name: registry
spec:
    type: NodePort
    ports:
    - port: 5000
      nodePort: 30002
    selector:
      name: registry

10.182.168.99是k8s-node-1的地址,非flannel集羣地址

一個一個創建起來，固然須要在k8s-node-1上打個標籤

kubectl label node k8s-node-1 name=registry

在創建registry-rc的時候遇到一些問題。

registry pod老是處於container creating的狀態。

• 首先須要在啓動registry pod的機器上建立相關的目錄，個人是在k8s-node-1上建立/k8s/test

經過describe pods 一看，基本都是nfs mount的問題，解決方式以下：

• 啓動nfs service

systemctl start nfs

• 遇到

Output: mount.nfs: access denied by server while mounting 10.182.168.99:/k8s/test錯誤時,修改配置文件/etc/exports，加入 insecure 選項

/k8s/test  *(insecure,rw,async,no_root_squash)

 

啓動完成，一切順利，

[root@k8s-master registry]# kubectl get pods
NAME                       READY     STATUS    RESTARTS   AGE
helloworld-service-62wl1   1/1       Running   6          88d
helloworld-service-8cbt2   1/1       Running   6          88d
registry-7nj8q             1/1       Running   2          1h
[root@k8s-master registry]# kubectl get services
NAME            CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
helloworldsvc   10.254.247.84   <nodes>       7001:30001/TCP   88d
kubernetes      10.254.0.1      <none>        443/TCP          120d
registry        10.254.174.54   <nodes>       5000:30002/TCP   1h

 

=====================================================================

接下來驗證。

在k8s-node-1上有一大堆的本地images

[root@k8s-node-1 v2]# docker images
REPOSITORY                                             TAG                      IMAGE ID            CREATED             SIZE
docker.io/registry                                     latest                   3ebefe7c539b        5 days ago          33.19 MB
1213-domain                                            v2                       326bf14bb29f        3 months ago        2.055 GB
oracle/coherence                                       12.2.1.0.0-cacheserver   57a90e86e1d2        3 months ago        625 MB
oracle/coherence                                       12.2.1.0.0-proxy         238c85d61468        3 months ago        625 MB
gcr.io/google_containers/nginx-ingress-controller      0.9.0-beta.7             2c3d45bb8cb9        3 months ago        130.6 MB
gcr.io/google_containers/k8s-dns-sidecar-amd64         1.14.2                   7c4034e4ffa4        4 months ago        44.5 MB
gcr.io/google_containers/k8s-dns-kube-dns-amd64        1.14.2                   ca8759c215c9        4 months ago        52.36 MB
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64   1.14.2                   e5c335701995        4 months ago        44.84 MB
1213-helloworld                                        v1                       351691157b77        4 months ago        2.064 GB

找一個小的，而後從新tag一下,這裏我認爲應該指向node的外部地址和端口而不是集羣地址

docker tag docker.io/nginx  k8s-node-1:30002/nginx

 

而後修改/etc/sysconfig/docker文件，主要是修改兩行（在須要用到registry的節點上都須要修改)

ADD_REGISTRY='--add-registry k8s-node-1'

INSECURE_REGISTRY='--insecure-registry k8s-node-1:30002'

而後重啓docker

service docker restart

將images push到本地的registry,而後咱們查看/k8s/test目錄下就有內容了.

docker push k8s-node-1:30002/nginx

 

在其餘機器運行docker pull,看到已經從本地拉回images.

[root@k8s-master registry]# docker pull k8s-node-1:30002/nginx
Using default tag: latest
Trying to pull repository k8s-node-1:30002/nginx ... 
sha256:c15f1fb8fd55c60c72f940a76da76a5fccce2fefa0dd9b17967b9e40b0355316: Pulling from k8s-node-1:30002/nginx
36a46ebd5019: Pull complete 
57168433389f: Pull complete 
332ec8285c50: Pull complete 
Digest: sha256:c15f1fb8fd55c60c72f940a76da76a5fccce2fefa0dd9b17967b9e40b0355316
Status: Downloaded newer image for k8s-node-1:30002/nginx:latest
[root@k8s-master registry]# docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
k8s-node-1:30002/nginx   latest              46102226f2fd        4 months ago        109.4 MB