docker (2) 私有倉庫的創建

時間 2019-12-14

原文原文鏈接

安裝部署一個私有的Docker Registry是引入、學習和使用Docker這門技術的必經之路之一。尤爲是當Docker被所在組織接受，更多人、項目和產品開始接觸和使用Docker時，存儲和分發自制的Docker image便成了剛需。Docker Registry一如既往的繼承了「Docker坑多」的特色，爲此這裏將本身搭建」各種」Registry過程當中執行的步驟、遇到的問題記錄下來，爲己備忘，爲他參考。node

Docker在2015年推出了distribution項目，即Docker Registry 2。相比於old registry，Registry 2使用Go實現，在安全性、性能方面均有大幅改進。Registry設計了全新的Rest API，而且在image存儲格式等方面再也不兼容於old Registry。去年8月份，docker官方hub使用Registriy 2.1替代了原先的old Registry。若是你要與Registry2交互，你的Docker版本至少要是Docker 1.6。
Docker的開發者也一直在致力於改善Registry安裝和使用的體驗，經過提供官方Registry Image以及Docker Compose工具等來簡化Registry的配置。不過在本文中，咱們只是利用Docker以及Registry的官方Image來部署Registry，這樣更便於全面瞭解Registry的部署配置細節。git

Registry2在鏡像存儲方面不只支持本地盤，還支持諸多主流第三方存儲方案。經過分佈式存儲系統你還能夠實現一個分佈式Docker Registry服務。這裏僅以本地盤以及single node registry2爲例。docker

1、環境

這裏仍是複用以往文章中的Docker環境：
Docker Registry Server: 10.10.105.71 Ubuntu 14.04 3.16.0-57-generic；docker 1.9.1
其餘兩個工做Server：
10.10.105.72 Ubuntu 14.04 3.19.0-25-generic; docker 1.9.1
10.10.126.101 Ubuntu 12.04 3.16.7-013607-generic; docker 1.9.1
本次Registry使用當前最新stable版本:Registry 2.3.0。因爲鏡像採用本地磁盤存儲，root分區較小，須要映射使用其餘volume。json

2、初次搭建

本覺得Docker Registry的搭建是何其簡單的，甚至簡單到經過一行命令就能夠完成的。好比咱們在Registry Server上執行：ubuntu

在~/dockerregistry下，執行：安全

## 啓動docker鏡像倉庫鏡像
$sudo docker run -d -p 5000:5000 -v `pwd`/data:/var/lib/registry --restart=always --name registry registry:2
Unable to find image 'registry:2' locally
2: Pulling from library/registry
f32095d4ba8a: Pull complete
9b607719a62a: Pull complete
973de4038269: Pull complete
2867140211c1: Pull complete
8da16446f5ca: Pull complete
fd8c38b8b68d: Pull complete
136640b01f02: Pull complete
e039ba1c0008: Pull complete
c457c689c328: Pull complete
Digest: sha256:339d702cf9a4b0aa665269cc36255ee7ce424412d56bee9ad8a247afe8c49ef1
Status: Downloaded newer image for registry:2
e9088ef901cb00546c59f89defa4625230f4b36b0a44b3713f38ab3d2a5a2b44

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
registry            2                   c457c689c328        9 days ago          165.7 MB

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                    NAMES
e9088ef901cb        registry:2          "/bin/registry /etc/d"   About a minute ago   Up About a minute   0.0.0.0:5000->5000/tcp   registry

Registry container已經跑起來了，其啓動日誌能夠經過：docker logs registry查看。網絡

咱們在71本地給busybox:latest打一個tag，並嘗試將新tag下的image push到Registry中去：dom

$ docker tag busybox:latest 10.10.105.71:5000/tonybai/busybox:latest
$ docker images
REPOSITORY                          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
registry                            2                   c457c689c328        9 days ago          165.7 MB
busybox                             latest              65e4158d9625        9 days ago          1.114 MB
10.10.105.71:5000/tonybai/busybox   latest              65e4158d9625        9 days ago          1.114 MB
... ...

push到Registry中：curl

$ docker push 10.10.105.71:5000/tonybai/busybox
The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1)
unable to ping registry endpoint https://10.10.105.71:5000/v0/
v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: Tunnel or SSL Forbidden
 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: Tunnel or SSL Forbidden

出錯了！簡單分析了一下，多是71上docker daemon配置中加了http代理的緣故，致使沒法ping通registry endpoint。因而在/etc/default/docker中註釋掉export http_proxy=」xxx」的設置，並重啓docker daemon。tcp

再次嘗試push：

$ docker push 10.10.105.71:5000/tonybai/busybox
The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1)
unable to ping registry endpoint https://10.10.105.71:5000/v0/
v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527
 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527

雖然仍是失敗，但錯誤信息已有所不一樣了。此次看來鏈接是能夠創建的，但client端經過https訪問server端，彷佛想tls通訊，但這一過程並未完成。

在其餘機器上嘗試push image到registry也遇到了一樣的錯誤輸出，以下：

10.10.105.72:

$ docker push 10.10.105.71:5000/tonybai/ubuntu
The push refers to a repository [10.10.105.71:5000/tonybai/ubuntu] (len: 1)
unable to ping registry endpoint https://10.10.105.71:5000/v0/
v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527
 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527

從錯誤信息來看，client與Registry交互，默認將採用https訪問，但咱們在install Registry時並未配置指定任何tls相關的key和crt文件，https訪問定然失敗。要想弄清這個問題，只能查看Registry Manual。

3、Insecure Registry

Registry的文檔仍是相對詳盡的。在文檔中，咱們找到了Insecure Registry，即接收plain http訪問的Registry的配置和使用方法，雖然這不是官方推薦的。
實際上對於咱們內部網絡而言，Insecure Registry基本能知足需求，部署過程也避免了secure registry的那些繁瑣步驟，好比製做和部署證書等。

爲了搭建一個Insecure Registry，咱們須要先清理一下上面已經啓動的Registry容器。

$ docker stop registry
registry
$ docker rm registry
registry

修改Registry server上的Docker daemon的配置，爲DOCKER_OPTS增長–insecure-registry：

DOCKER_OPTS="--insecure-registry 10.10.105.71:5000 ....

重啓Docker Daemon，啓動Registry容器：

$ sudo service docker restart
docker stop/waiting
docker start/running, process 6712
$ sudo docker run -d -p 5000:5000 -v `pwd`/data:/var/lib/registry --restart=always --name registry registry:2
5966e92fce9c34705050e19368d19574e021a272ede1575385ef35ecf5cea019

嘗試再次Push image:

$ docker push 10.10.105.71:5000/tonybai/busybox
The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1)
65e4158d9625: Pushed
5506dda26018: Pushed
latest: digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739

這回push ok！

咱們將本地的tag作untag處理，再從Registry pull相關image：

$ docker images
REPOSITORY                          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
registry                            2                   c457c689c328        9 days ago          165.7 MB
10.10.105.71:5000/tonybai/busybox   latest              65e4158d9625        9 days ago          1.114 MB
busybox                             latest              65e4158d9625        9 days ago          1.114 MB
ubuntu                              14.04               6cc0fc2a5ee3        5 weeks ago         187.9 MB

$ docker rmi 10.10.105.71:5000/tonybai/busybox
Untagged: 10.10.105.71:5000/tonybai/busybox:latest

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
registry            2                   c457c689c328        9 days ago          165.7 MB
busybox             latest              65e4158d9625        9 days ago          1.114 MB
ubuntu              14.04               6cc0fc2a5ee3        5 weeks ago         187.9 MB

$ docker pull 10.10.105.71:5000/tonybai/busybox
Using default tag: latest
latest: Pulling from tonybai/busybox
Digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892
Status: Downloaded newer image for 10.10.105.71:5000/tonybai/busybox:latest

$ docker images
REPOSITORY                          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
registry                            2                   c457c689c328        9 days ago          165.7 MB
10.10.105.71:5000/tonybai/busybox   latest              65e4158d9625        9 days ago          1.114 MB
busybox                             latest              65e4158d9625        9 days ago          1.114 MB
ubuntu                              14.04               6cc0fc2a5ee3        5 weeks ago         187.9 MB

能夠看到：Pull過程也很順利。

在Private Registry2中查看或檢索Repository或images，將不能用docker search：

$ docker search 10.10.105.71:5000/tonybai/busybox/
Error response from daemon: Unexpected status code 404
但經過v2版本的API，咱們能夠實現相同目的：

$curl  http://10.10.105.71:5000/v2/_catalog
{"repositories":["tonybai/busybox"]}

$ curl  http://10.10.105.71:5000/v2/tonybai/busybox/tags/list
{"name":"tonybai/busybox","tags":["latest"]}
在其餘主機上，咱們嘗試pull busybox：

10.10.105.72:

$docker pull 10.10.105.71:5000/tonybai/busybox
Using default tag: latest
Error response from daemon: unable to ping registry endpoint https://10.10.105.71:5000/v0/
v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527
 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527

咱們發現依舊不能pull和push！在Registry手冊中講到，若是採用insecure registry的模式，那麼全部與Registry交互的主機上的Docker Daemon都要配置：–insecure-registry選項。

咱們按照上面的配置方法，修改105.72上的/etc/default/docker，重啓Docker daemon，再執行pull/push就會獲得正確的結果：

$ sudo vi /etc/default/docker
$ sudo service docker restart
docker stop/waiting
docker start/running, process 10614
$ docker pull 10.10.105.71:5000/tonybai/busybox
Using default tag: latest
latest: Pulling from tonybai/busybox
5506dda26018: Pull complete
65e4158d9625: Pull complete
Digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892
Status: Downloaded newer image for 10.10.105.71:5000/tonybai/busybox:latest

$ docker images
REPOSITORY                          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
ubuntu                              14.04               36248ae4a9ac        8 days ago          187.9 MB
10.10.105.71:5000/tonybai/ubuntu    14.04               36248ae4a9ac        8 days ago          187.9 MB
10.10.105.71:5000/tonybai/busybox   latest              65e4158d9625        9 days ago          1.114 MB

$ docker push 10.10.105.71:5000/tonybai/ubuntu
The push refers to a repository [10.10.105.71:5000/tonybai/ubuntu] (len: 1)
36248ae4a9ac: Pushed
8ea5373bf5a6: Pushed
2e0188208e83: Pushed
e3c70beaa378: Pushed
14.04: digest: sha256:72e56686cb9fb38438f0fd68fecf02ef592ce2ef7069bbf97802d959d568c5cc size: 6781

4、Secure Registry

Docker官方是推薦你採用Secure Registry的工做模式的，即transport採用tls。這樣咱們就須要爲Registry配置tls所需的key和crt文件了。
咱們首先清理一下環境，將上面的Insecure Registry停掉並rm掉；將各臺主機上Docker Daemon的DOCKER_OPTS配置中的–insecure-registry去掉，並重啓Docker Daemon。
若是你擁有一個域名，域名下主機提供Registry服務，而且你擁有某知名CA簽署的證書文件，那麼你能夠創建起一個Secure Registry。不過我這裏沒有現成的證書，只能使用自簽署的證書。嚴格來說，使用自簽署的證書在Docker官方眼中依舊屬於Insecure，不過這裏只是藉助自簽署的證書來講明一下Secure Registry的部署步驟罷了。

製做自簽署證書
若是你有知名CA簽署的證書，那麼這步可直接忽略。

$ openssl req -newkey rsa:2048 -nodes -sha256 -keyout certs/domain.key -x509 -days 365 -out certs/domain.crt
Generating a 2048 bit RSA private key
..............+++
............................................+++
writing new private key to 'certs/domain.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:Liaoning
Locality Name (eg, city) []:shenyang
Organization Name (eg, company) [Internet Widgits Pty Ltd]:foo
Organizational Unit Name (eg, section) []:bar
Common Name (e.g. server FQDN or YOUR name) []:mydockerhub.com
Email Address []:bigwhite.cn@gmail.com
``
2. 啓動Secure Registry
啓動帶證書的Registry：

$ docker run -d -p 5000:5000 --restart=always --name registry -v pwd/data:/var/lib/registry -v pwd/certs:/certs -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key registry:2
35e8ce77dd455f2bd50854e4581cd52be8a137f4aaea717239b6d676c5ea5777

因爲證書的CN是mydockerhub.com，咱們須要修改一下/etc/hosts文件:

10.10.105.71 mydockerhub.com
從新爲busybox製做一個tag:

$docker tag busybox:latest mydockerhub.com:5000/tonybai/busybox:latest
Push到Registry:

$ docker push mydockerhub.com:5000/tonybai/busybox
The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1)
unable to ping registry endpoint https://mydockerhub.com:5000/v0/
v2 ping attempt failed with error: Get https://mydockerhub.com:5000/v2/: x509: certificate signed by unknown authority
v1 ping attempt failed with error: Get https://mydockerhub.com:5000/v1/_ping: x509: certificate signed by unknown authority

push失敗了！從錯誤日誌來看，docker client認爲server傳輸過來的證書的簽署方是一個unknown authority（未知的CA），所以驗證失敗。咱們須要讓docker client安裝咱們的CA證書：

$ sudo mkdir -p /etc/docker/certs.d/mydockerhub.com:5000
$ sudo cp certs/domain.crt /etc/docker/certs.d/mydockerhub.com:5000/ca.crt
$ sudo service docker restart //安裝證書後，重啓Docker Daemon
``再執行Push，咱們看到了成功的輸出日誌。因爲data目錄下以前已經被push了tonybai/busybox repository，所以提示「已存在」：
$docker push mydockerhub.com:5000/tonybai/busybox
The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1)
65e4158d9625: Image already exists
5506dda26018: Image already exists
latest: digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739

3. 外部訪問Registry
咱們換其餘機器試試訪問這個secure registry。根據以前的要求，咱們照貓畫虎的修改一下hosts文件，安裝ca.cert，去除–insecure-registry選項，並重啓Docker daemon。以後嘗試從registry pull image：

$ docker pull mydockerhub.com:5000/tonybai/busybox
Using default tag: latest
latest: Pulling from tonybai/busybox

Digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892
Status: Downloaded newer image for mydockerhub.com:5000/tonybai/busybox:latest

$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
10.10.105.71:5000/tonybai/ubuntu 14.04 36248ae4a9ac 9 days ago 187.9 MB
ubuntu 14.04 36248ae4a9ac 9 days ago 187.9 MB
10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB
mydockerhub.com:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB

這樣來看，若是使用自簽署的證書，那麼全部要與Registry交互的Docker主機都須要安裝mydockerhub.com的ca.crt(domain.crt)。但若是你使用知名CA，這一步也就能夠忽略。

### 5、Registry的鑑權管理
Registry提供了一種基礎的鑑權方式。咱們經過下面步驟便可爲Registry加上基礎鑑權：

在Register server上，爲Registry增長foo用戶，密碼foo123：（以前須要停掉已有的Registry，並刪除之）

//生成鑑權密碼文件
$ mkdir auth
$ docker run --entrypoint htpasswd registry:2 -Bbn foo foo123 > auth/htpasswd
$ ls auth
htpasswd

//啓動帶鑑權功能的Registry：
$ docker run -d -p 5000:5000 --restart=always --name registry -v pwd/auth:/auth -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd -v pwd/data:/var/lib/registry -v pwd/certs:/certs -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key registry:2
199ad0b3591fb9613b21b1c96f017267f3c39661a7025d30df636c6805e7ab50

在105.72上，咱們嘗試push image到Registry：

$ docker push mydockerhub.com:5000/tonybai/busybox
The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1)
65e4158d9625: Image push failed
Head https://mydockerhub.com:5000/v2/tonybai/busybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4: no basic auth credentials

錯誤信息提示：鑑權失敗。

在72上執行docker login:

$docker login mydockerhub.com:5000
Username: foo
Password:
Email: bigwhite.cn@gmail.com
WARNING: login credentials saved in /home/baiming/.docker/config.json
Login Succeeded

login成功後，再行Push：

$ docker push mydockerhub.com:5000/tonybai/busybox
The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1)
65e4158d9625: Image already exists
5506dda26018: Image already exists
latest: digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739
Push ok！
```

6、Registry中images的管理

前面提到過，經過V2版Rest API能夠查詢Repository和images：

$ curl --cacert domain.crt --basic --user foo:foo123 https://mydockerhub.com:5000/v2/_catalog
{"repositories":["tonybai/busybox","tonybai/ubuntu"]}
但若是要刪除Registry中的Repository或某個tag的Image，目前v2還不支持，緣由見Registry的roadmap中的說明。

不過若是你的Registry的存儲引擎使用的是本地盤，卻是有一些第三方腳本可供使用，好比：delete-docker-registry-image。