harbor私有鏡像倉庫的搭建與使用與主從複製

harbor私有鏡像倉庫，
私有倉庫有兩種，一種是harbor，一種是小型的私有倉庫，harbor有兩種模式，一種是主
從，一種是高可用倉庫，
項目需求，須要兩臺服務器，都有docker。
ldap權限統一管理，
harbor能夠基於角色的訪問控制來管理，
harbor能夠圖形化用戶界面，用戶能夠經過瀏覽器，檢索當前docker鏡像倉庫，管理項目
和命名空間
harbor能夠審計管理，全部針對鏡像倉庫的操做，均可以被記錄追溯，用於審計管理
基於英語與中文語言進行了本地化。能夠最佳更多的語言支持。
harbor能夠自動啓動clair漏洞掃描
harbor最低配置2cpu4gmem
操做系統7.5

首先須要作一個ca認證的問題，
經過openssl沒有的yum安裝一個
步驟
mkdir -p /data/ssl
cd /data/ssl
openssl req -newkey rsa:4096 -nodes -sha256 -keyout ca.key -x509 -days 365 -out ca.crt
會出現一些的字樣
[root@bogon ssl]# openssl req -newkey rsa:4096 -nodes -sha256 -keyout ca.key -x509 -days 365 -out ca.crt
Generating a 4096 bit RSA private key
.............................................................................. 
...........................++
.............................................................................. 
............................++
writing new private key to 'ca.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN（國家的名字，輸入CN）
State or Province Name (full name) []:Beijing（輸入城市）
Locality Name (eg, city) [Default City]:Beijing
Organization Name (eg, company) [Default Company Ltd]:yunjisuan（公司的名） 
Organizational Unit Name (eg, section) []:yunjisuan（公司的名字）
Common Name (eg, your name or your server's hostname) []:www.yunjisuan.com（域 
名，綁定地址）
Email Address []:（email能夠不寫）
以上的操做完成ca證書就出來了
而後生成域名證書
openssl req -newkey rsa:4096 -nodes -sha256 -keyout www.yunjisuan.com.key -out 
www.yunjisuan.com.csr

Generating a 4096 bit RSA private key
......++
..................................................................++
writing new private key to 'www.yunjisuan.com.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN(國家)
State or Province Name (full name) []:Beijing（地區）
Locality Name (eg, city) [Default City]:Beijing（城市）
Organization Name (eg, company) [Default Company Ltd]:yunjisuan（公司名）
Organizational Unit Name (eg, section) []:yunjisuan（公司名）
Common Name (eg, your name or your server's hostname) []:www.yunjisuan.com（域 
名）
Email Address []:（email）

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:（登錄密碼，能夠不寫）
An optional company name []:

完成以上就完成了域名證書

[root@bogon ssl]# ls 
ca.crt ca.key www.yunjisuan.com.csr www.yunjisuan.com.key

而後生成域名的crt證書
openssl x509 -req -days 365 -in www.yunjisuan.com.csr -CA ca.crt -CAkey ca.key 
-CAcreateserial -out www.yunjisuan.com.crt 
會出現一些的結果 
Signature ok
subject=/C=CN/ST=Beijing/L=Beijing/O=yunjisuan/OU=yunjisuan/CN=www.yunjisuan.c 
om
Getting CA Private Key
[root@bogon ssl]# ls 
ca.crt ca.key ca.srl www.yunjisuan.com.crt www.yunjisuan.com.csr 
www.yunjisuan.com.key
這樣全部的證書就簽發完了，咱們開始分發證書
先讓本機信任證書，將證書複製到證書的目錄裏
[root@bogon ssl]# cp www.yunjisuan.com.crt /etc/pki/ca-trust/source/anchors/
而後讓它當即生效
[root@bogon ssl]# update-ca-trust enable
[root@bogon ssl]# update-ca-trust extract

 

而後看下selinux關沒有

sestatus

 

關閉防火牆和selinux命令

[root@bogon ssl]# setenforce 0
[root@bogon ssl]# systemctl stop firewalld

 

重啓docker

[root@bogon ssl]# systemctl restart docker 

 

接下了下載並安裝harbor

先建立一個harbor的目錄，等下要鏈接到軟件
[root@bogon ssl]# mkdir -p /etc/ssl/harbor
將證書複製過去
[root@bogon ssl]# cp www.yunjisuan.com.crt www.yunjisuan.com.key 
/etc/ssl/harbor/
建立安裝目錄
[root@bogon ~]# mkdir -p /data/install
[root@bogon ~]# cd /data/install/
下載安裝包
wget http://harbor.orientsoft.cn/harbor-v1.5.0/harbor-offline-installer- 
v1.5.0.tgz
若是有直接解壓過去就好
[root@bogon ~]# tar xf harbor-offline-installer-v1.5.0.tgz -C /data/install/
解壓完了看一下
[root@bogon install]# ll harbor/
total 854960
drwxr-xr-x. 3 root root 23 Dec 11 17:26 common （模板目錄）
-rw-r--r--. 1 root root 1185 May 2 2018 docker-compose.clair.yml（漏洞 
掃描）
-rw-r--r--. 1 root root 1725 May 2 2018 docker-compose.notary.yml（鏡像 
簽名）
-rw-r--r--. 1 root root 3596 May 2 2018 docker-compose.yml（接口文件）
drwxr-xr-x. 3 root root 156 May 2 2018 ha（高可用模式）
-rw-r--r--. 1 root root 6687 May 2 2018 harbor.cfg（harbor配置文件）
-rw-r--r--. 1 root root 875401338 May 2 2018 harbor.v1.5.0.tar.gz
-rwxr-xr-x. 1 root root 5773 May 2 2018 install.sh（安裝腳本）
-rw-r--r--. 1 root root 10771 May 2 2018 LICENSE
-rw-r--r--. 1 root root 482 May 2 2018 NOTICE
-rwxr-xr-x. 1 root root 27379 May 2 2018 prepare
而後備份配置文件，修改配置文件
[root@bogon harbor]# pwd
/data/install/harbor
[root@bogon harbor]# cp harbor.cfg{,.bak}
而後修改配置文件
（7行） hostname = www.yunjisuan.com（以前註冊的域名）
（11行） ui_url_protocol = https（改爲https）
（23行） ssl_cert = /etc/ssl/harbor/www.yunjisuan.com.crt（以前複製過去證書的 
位置）
（24行） ssl_cert_key = /etc/ssl/harbor/www.yunjisuan.com.key（以前複製過去證 
書的位置）
其他的不用下載
而後下載安裝docker-compose
若是有就直接給x權限移動到/usr/bin下就能夠了
[root@bogon ~]# chmod +x docker-compose 
[root@bogon ~]# ls 
anaconda-ks.cfg docker-compose harbor-offline-installer-v1.5.0.tgz
[root@bogon ~]# mv docker-compose /usr/bin/
[root@bogon ~]# which docker-compose
/usr/bin/docker-compose
[root@bogon ~]# docker-compose --version
docker-compose version 1.21.2, build a133471
接下來就能夠去離線安裝harbor了
[root@bogon ~]# cd /data/install/harbor/
[root@bogon harbor]# ls 
common docker-compose.notary.yml ha 
harbor.cfg.bak install.sh NOTICE
docker-compose.clair.yml docker-compose.yml harbor.cfg 
harbor.v1.5.0.tar.gz LICENSE prepare
而後啓動安裝腳本
[root@bogon harbor]# ./install.sh --with-notary --with-clair
（--with-notary啓用鏡像簽名，--with-clair啓用漏洞掃描）
安裝完了是一下的樣子

✔ ----Harbor has been installed and started successfully.----

Now you should be able to visit the admin portal at https://www.yunjisuan.com. 
For more details, please visit https://github.com/vmware/harbor .

[root@bogon harbor]# echo $?
0
[root@bogon harbor]# docker images
REPOSITORY TAG IMAGE ID CREATED 
SIZE
centos 7 1e1148e4cc2c 5 days 
ago 202MB
vmware/redis-photon v1.5.0 7c03076402d9 7 months 
ago 207MB
vmware/clair-photon v2.0.1-v1.5.0 7ae4e0670a3f 7 months 
ago 301MB
vmware/notary-server-photon v0.5.1-v1.5.0 0b2b23300552 7 months 
ago 211MB
vmware/notary-signer-photon v0.5.1-v1.5.0 67c41b4a1283 7 months 
ago 209MB
vmware/registry-photon v2.6.2-v1.5.0 3059f44f4b9a 7 months 
ago 198MB
vmware/nginx-photon v1.5.0 e100456182fc 7 months 
ago 135MB
vmware/harbor-log v1.5.0 62bb6b8350d9 7 months 
ago 200MB
vmware/harbor-jobservice v1.5.0 aca9fd2e867f 7 months 
ago 194MB
vmware/harbor-ui v1.5.0 1055166068d0 7 months 
ago 212MB
vmware/harbor-adminserver v1.5.0 019bc4544829 7 months 
ago 183MB
vmware/harbor-db v1.5.0 82354dcf564f 7 months 
ago 526MB
vmware/mariadb-photon v1.5.0 403e7e656499 7 months 
ago 526MB
vmware/postgresql-photon v1.5.0 35c891dea9cf 7 months 
ago 221MB
vmware/harbor-migrator v1.5.0 466c57ab0dc3 7 months 
ago 1.16GB
vmware/photon 1.0 4b481ecbef2a 7 months 
ago 130MB

 

 編譯成功後使用瀏覽器登錄，惟一能夠用的瀏覽器是火狐瀏覽器，

 

 點擊添加例外直接確認就能夠進去了

安全管理

點擊項目進入如圖勾選保存

 

 在服務器上作映射、

[root@bogon harbor]# 
[root@bogon harbor]# echo "192.168.200.132 www.yunjisuan.com" >> /etc/hosts

服務器登錄了

root@bogon harbor]# docker login -uadmin -pHarbor12345 www.yunjisuan.com
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

警告不用理會

上傳鏡像到私有倉庫，必須前綴帶公司名例如：www.yunjisuan.com/library/centos:v1

[root@bogon harbor]# docker tag centos:7 www.yunjisuan.com/library/centos:v1 (實驗是拿公網下載的鏡像的，因此要先改下名字)
[root@bogon harbor]# docker push www.yunjisuan.com/library/centos:v1（上傳）

The push refers to repository [www.yunjisuan.com/library/centos]
071d8bd76517: Pushed
v1: digest: sha256:365fc7f33107869dfcf2b3ba220ce0aa42e16d3f8e8b3c21d72af1ee622f0cf0 size: 529

在瀏覽器上查看

 

遠程分發證書到別的服務器

[root@bogon ssl]# ls 
ca.crt  ca.key  ca.srl  www.yunjisuan.com.crt  www.yunjisuan.com.csr  www.yunjisuan.com.key
[root@bogon ssl]# pwd 
/data/ssl

[root@bogon ssl]# scp www.yunjisuan.com.crt 192.168.200.146:/etc/pki/ca-trust/source/anchors/
The authenticity of host '192.168.200.146 (192.168.200.146)' can't be established.
ECDSA key fingerprint is SHA256:gm/RhqGrfDo5Rgcr/LmBAaqPv6tmni7cRpXjGEWZQpg.
ECDSA key fingerprint is MD5:ae:f6:0b:6e:80:96:67:cf:bd:e8:f5:b5:c4:e0:da:11.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.200.146' (ECDSA) to the list of known hosts.
root@192.168.200.146's password:
www.yunjisuan.com.crt 100% 1931 201.7KB/s 00:00

而後在146服務器上當即生效並關閉selinux和防火牆，重啓docker

[root@bogon ~]# update-ca-trust enable
[root@bogon ~]# update-ca-trust extract
[root@bogon ~]# setenforce 0
[root@bogon ~]# systemctl stop firewalld
[root@bogon ~]# systemctl restart docker

這樣就146的服務器就能夠推鏡像了，146也須要作映射，

推的方法跟132的同樣

[root@bogon ~]# echo "192.168.200.132 www.yunjisuan.com" >> /etc/hosts

[root@bogon ~]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
centos              7                   75835a67d134        2 months ago        200MB
[root@bogon ~]# docker login -uadmin -pHarbor12345 www.yunjisuan.com
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@bogon ~]# docker tag centos:7 www.yunjisuan.com/library/centos:v2
[root@bogon ~]# docker push www.yunjisuan.com/library/centos:v2
The push refers to repository [www.yunjisuan.com/library/centos]
f972d139738d: Pushed 
v2: digest: sha256:dc29e2bcceac52af0f01300402f5e756cc8c44a310867f6b94f5f7271d4f3fec size: 529

 

主從複製，要將證書籤發在從庫上重新作一遍，將域名修改便可，別的都不用變，還要將本身的信任證書發個主一份，讓其當即生效
而且從起docker，而後在從的在安裝一遍harbor，在兩邊都要映射

在主的裏邊要在/etc/ssl/harbor/目錄裏用docker-compose down 關如下，在從起docker
在/etc/ssl/harbor/目錄裏輸入./prepare 重新認證下文件。在重啓./install.sh --with-clair

 

 

而後在dns服務器上搭建dns

yum -y install bind bind-chroot bind-utils
cd /etc
cp named.conf{,.bak}
修改配置文件
[root@bogon etc]# vim named.conf

//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
// See the BIND Administrator's Reference Manual (ARM) for details about the
// configuration located in /usr/share/doc/bind-{version}/Bv9ARM.html

options {
listen-on port 53 { 192.168.200.147; };
// listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
recursing-file "/var/named/data/named.recursing";
secroots-file "/var/named/data/named.secroots";
allow-query { any; };
forwarders { 192.168.200.2; };

/* 
- If you are building an AUTHORITATIVE DNS server, do NOT enable 
recursion.
- If you are building a RECURSIVE (caching) DNS server, you need to 
enable 
recursion. 
- If your recursive DNS server has a public IP address, you MUST 
enable access 
control to limit queries to your legitimate users. Failing to do so 
will
cause your server to become part of large scale DNS amplification 
attacks. Implementing BCP38 within your network would greatly
reduce such attack surface 
*/
recursion yes;

dnssec-enable no;
dnssec-validation no;

/* Path to ISC DLV key */
bindkeys-file "/etc/named.iscdlv.key";

managed-keys-directory "/var/named/dynamic";

pid-file "/run/named/named.pid";
session-keyfile "/run/named/session.key";
};

logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};

zone "." IN {
type hint;
file "named.ca";
};

zone "yunjisuan.com" IN {
type master;
file "yunjisuan.com.zone";
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

檢查配置文件是否正確
[root@bogon etc]# named-checkconf /etc/named.conf
什麼都不出就ok

在去/var/named/複製一份空的點的解析文件修改
[root@bogon named]# cp -p named.empty yunjisuan.com.zone
[root@bogon named]# vim yunjisuan.com.zone 
修改爲一下的樣子
$TTL 3H
@ IN SOA yunjisuan.com. root.ns1.yunjisuan.com. (
0 ; serial
1D ; refresh
1H ; retry
1W ; expire
3H ) ; minimum
NS ns1.yunjisuan.com.
ns1 A 192.168.200.147
www A 192.168.200.132
www2 A 192.168.200.146
檢查配置文件是否正確
[root@bogon named]# named-checkzone yunjisuan.com yunjisuan.com.zone 
zone yunjisuan.com/IN: loaded serial 0
OK
而後啓動域名解析服務
[root@bogon etc]# systemctl start named
[root@bogon etc]# ss -antup | grep named
udp UNCONN 0 0 *:21253 *:* 
users:(("named",pid=5898,fd=514))
tcp LISTEN 0 128 127.0.0.1:953 *:* 
users:(("named",pid=5898,fd=21))
tcp LISTEN 0 128 ::1:953 :::* 
users:(("named",pid=5898,fd=22))
而後零時修改網關，修改爲本身
[root@bogon etc]# vim /etc/resolv.conf

把另外的兩個服務器都修改了零時網關

 

檢查dns解析

[root@bogon etc]# nslookup www.yunjisuan.com
Server:        192.168.200.147
Address:    192.168.200.147#53

Name:    www.yunjisuan.com
Address: 192.168.200.132

 

 

在瀏覽器上作主從複製就能夠了