https://docs.gitlab.com/ee/administration/high_availability/gitlab.htmlhtml
https://about.gitlab.com/high-availability/前端
https://www.oschina.net/translate/gitlab-high-availability?lang=chsvue
gitlab目錄結構：python
find / -name gitlab |grep -v "^/opt/"
/run/gitlab # pid目錄
/etc/gitlab # 配置目錄
/var/log/gitlab # 日誌目錄
/var/opt/gitlab # 應用數據保存目錄，要同步的主要是這裏的部分數據
/opt/gitlab # bin程序目錄nginx
架構圖：git
git1web
git2:redis
（gitlab 容許多個節點，多活）sql
postgresql集羣：數據庫
redis集羣（哨兵模式）
MFS網絡文件存儲
# MFS集羣部署見 http://www.cnblogs.com/linkenpark/p/7416998.html
redis集羣（哨兵模式）：http://blog.51cto.com/tianshili/1759289
# PostgreSQL 主從集羣部署 http://www.cnblogs.com/linkenpark/p/8339936.html
安裝postgresql擴展pg_trgm
yum -y install postgresql96-contrib-9.6.8 # 默認的 yum -y install postgresql-contrib
建立用戶及庫
su - postgres psql create role gitlab login encrypted password 'pass'; \du ;顯示用戶 create database gitlabhq_production owner=gitlab ENCODING = 'UTF8'; \l ;列出數據庫 ;添加pg_trgm擴展 CREATE EXTENSION pg_trgm;
在MFS服務端根目錄下建立一個項目給gitlab用
mkdir gitlab
cd gitlab; mkdir -p .ssh gitlab-rails/uploads gitlab-rails/shared gitlab-ci/builds git-data
git一、git2 掛載MFS目錄
mkdir -p /var/opt/gitlab/.ssh /var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/git-data
cat /etc/fstab
mfsmount /var/opt/gitlab/.ssh fuse mfsmaster=mfsmaster,mfsport=9421,mfssubfolder=/gitlab/.ssh,mfspassword=passwd,_netdev 0 0 mfsmount /var/opt/gitlab/gitlab-rails/uploads fuse mfsmaster=mfsmaster,mfsport=9421,mfssubfolder=/gitlab/gitlab-rails/uploads,mfspassword=passwd,_netdev 0 0 mfsmount /var/opt/gitlab/gitlab-rails/shared fuse mfsmaster=mfsmaster,mfsport=9421,mfssubfolder=/gitlab/gitlab-rails/shared,mfspassword=passwd,_netdev 0 0 mfsmount /var/opt/gitlab/gitlab-ci/builds fuse mfsmaster=mfsmaster,mfsport=9421,mfssubfolder=/gitlab/gitlab-ci/builds,mfspassword=passwd,_netdev 0 0 mfsmount /var/opt/gitlab/git-data fuse mfsmaster=mfsmaster,mfsport=9421,mfssubfolder=/gitlab/git-data,mfspassword=passwd,_netdev 0 0
git一、git2安裝gitlab
添加gitlab yum源
cat /etc/yum.repos.d/gitlab-ce.repo
[gitlab-ce] name=Gitlab CE Repository baseurl=https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el$releasever/ gpgcheck=0 enabled=1
yum install -y curl policycoreutils-python openssh-server sudo systemctl enable sshd
systemctl start sshd
firewall-cmd --permanent --add-service=http
systemctl reload firewalld
yum -y install postfix
systemctl enable postfix
systemctl start postfix
yum install -y gitlab-ce
gitlab啓動腳本
cat /etc/systemd/system/gitlab.service
[Unit] Description=gitlab [Service] Type=oneshot RemainAfterExit=yes ExecStart=/bin/gitlab-ctl start ExecStop=/bin/gitlab-ctl stop [Install] WantedBy=multi-user.target
systemctl enable gitlab
git1配置
grep -Ev "^#|^$" /etc/gitlab/gitlab.rb
external_url 'http://git.conf.com'
gitlab_rails['time_zone'] = 'Asia/Shanghai'
# Disable components that will not be on the GitLab application server roles ['application_role'] # Prevent GitLab from starting if MFS data mounts are not available high_availability['mountpoint'] = '/var/opt/gitlab/git-data' # PostgreSQL connection details postgresql['enable'] = false gitlab_rails['db_adapter'] = "postgresql" gitlab_rails['db_encoding'] = "utf8" gitlab_rails['db_database'] = "gitlabhq_production" gitlab_rails['db_username'] = "gitlab" gitlab_rails['db_password'] = "db_password" gitlab_rails['db_host'] = "172.16.3.203" gitlab_rails['db_port'] = 5432 # Redis connection details(單實例、或主從集羣) redis['enable'] = false gitlab_rails['redis_host'] = "172.16.3.203" gitlab_rails['redis_port'] = 6379 gitlab_rails['redis_password'] = "redis_password" gitlab_rails['redis_database'] = 0 # Redis connection details(哨兵集羣) # redis鏈接方式只設置一種就能夠，CE版也能夠鏈接redis sentinel集羣 redis['enable'] = false gitlab_rails['redis_sentinels'] = [ {'host' => 'sentinel-0.conf.com', 'port' => 26379}, {'host' => 'sentinel-1.conf.com', 'port' => 26379}, {'host' => 'sentinel-2.conf.com', 'port' => 26379}, {'host' => 'sentinel-3.conf.com', 'port' => 26379}, ] redis['master_name'] = 'mymaster' redis['master_password'] = 'redis_auth_pass' gitlab_rails['redis_database'] = 0 # nginx enable nginx['enable'] = true
使配置生效
gitlab-ctl reconfigure
如要再次初始化數據，運行 gitlab-rake gitlab:setup（通常前面執行了gitlab-ctl reconfigure已經初始化發數據）
gitlab-ctl start
git2配置
從git1把/etc/gitlab/gitlab-secrets.json 複製到 git2的/etc/gitlab目錄下
touch /etc/gitlab/skip-auto-migrations
grep -v "^#" /etc/gitlab/gitlab.rb
external_url 'http://git.conf.com'
gitlab_rails['time_zone'] = 'Asia/Shanghai'
# Disable components that will not be on the GitLab application server roles ['application_role'] # Prevent GitLab from starting if MFS data mounts are not available high_availability['mountpoint'] = '/var/opt/gitlab/git-data' # disable automatic database migrations gitlab_rails['auto_migrate'] = false # PostgreSQL connection details postgresql['enable'] = false gitlab_rails['db_adapter'] = "postgresql" gitlab_rails['db_encoding'] = "utf8" gitlab_rails['db_database'] = "gitlabhq_production" gitlab_rails['db_username'] = "gitlab" gitlab_rails['db_password'] = "db_password" gitlab_rails['db_host'] = "172.16.3.203" gitlab_rails['db_port'] = 5432 # Redis connection details(單實例、或主從集羣) redis['enable'] = false gitlab_rails['redis_host'] = "172.16.3.203" gitlab_rails['redis_port'] = 6379 gitlab_rails['redis_password'] = "redis_password" gitlab_rails['redis_database'] = 0 # Redis connection details(哨兵集羣) # redis鏈接方式只設置一種就能夠，CE版也能夠鏈接redis sentinel集羣 redis['enable'] = false gitlab_rails['redis_sentinels'] = [ {'host' => 'sentinel-0.conf.com', 'port' => 26379}, {'host' => 'sentinel-1.conf.com', 'port' => 26379}, {'host' => 'sentinel-2.conf.com', 'port' => 26379}, {'host' => 'sentinel-3.conf.com', 'port' => 26379}, ] redis['master_name'] = 'mymaster' redis['master_password'] = 'redis_auth_pass' gitlab_rails['redis_database'] = 0 # nginx enable nginx['enable'] = true
gitlab-ctl reconfigure
gitlab-ctl start
若有更多的 second app 請參照git2配置
# 前端負載均衡
前端再分別對http負載（可用nginx反向代理）
ssh（可用HAproxy代理，或nginx TCP代理）
由其餘gitlab服務器數據遷移到此gitlab集羣
原gitlab服務器數據備份
能夠經過/etc/gitlab/gitlab.rb配置文件來修改默認存放備份文件的目錄
gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"
/var/opt/gitlab/backups修改成你想存放備份的目錄便可, 修改完成以後使用gitlab-ctl reconfigure命令重載配置文件便可.
/opt/gitlab/bin/gitlab-rake gitlab:backup:create #執行備份命令
如 /var/opt/gitlab/backups/1524449406_2018_04_23_10.6.4_gitlab_backup.tar
把 備份的數據 1524449406_2018_04_23_10.6.4_gitlab_backup.tar 複製到 gitlab集羣的第一個節點的/var/opt/gitlab/backups目錄下
chmod 777 /var/opt/gitlab/backups/1524449406_2018_04_23_10.6.4_gitlab_backup.tar
# 中止相關數據鏈接服務
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
umount /var/opt/gitlab/gitlab-rails/uploads # 因爲恢復時會把該目錄重啓命令成 /var/opt/gitlab/gitlab-rails/uploads.時間戳 ，生產新的/var/opt/gitlab/gitlab-rails/uploads目錄，但該目錄被掛載了會提示設備busy
umount /var/opt/gitlab/gitlab-ci/builds # 因爲恢復時會把該目錄重啓命令成/var/opt/gitlab/gitlab-ci/builds.時間戳 ，生產新的/var/opt/gitlab/gitlab-ci/builds目錄，但該目錄被掛載了會提示設備busy
gitlab-rake gitlab:backup:restore BACKUP=1524449406_2018_04_23_10.6.4 #執行恢復備份
mv /var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/uploads_ok
mkdir -p /var/opt/gitlab/gitlab-rails/uploads
mount /var/opt/gitlab/gitlab-rails/uploads
cp -ap /var/opt/gitlab/gitlab-rails/uploads_ok/* /var/opt/gitlab/gitlab-rails/uploads
mv /var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/gitlab-ci/builds_ok
mkdir -p /var/opt/gitlab/gitlab-ci/builds
mount /var/opt/gitlab/gitlab-ci/builds
cp -rp /var/opt/gitlab/gitlab-ci/builds_ok/* /var/opt/gitlab/gitlab-ci/builds
gitlab-ctl start unicorn
gitlab-ctl start sidekiq
完成恢復
# key認證問題
可能報錯：
報錯狀況1：（key沒有從master複製過來）
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:9DdK4jo9LPLg7snd/vueT3wI2dy0hb7CVYRRGOTU8TY.
Please contact your system administrator.
Add correct host key in /c/Users/cd/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /c/Users/cd/.ssh/known_hosts:4
ECDSA host key for git.tuandai888.com has changed and you have requested strict checking.
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
報錯狀況2：（從master複製過來的key權限沒設置好）
ssh_exchange_identification: Connection closed by remote host
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
解決方法：
把master的 /etc/ssh/*key* 文件所有複製到從節點來，並注意權限與master的設置同樣，
scp root@[master_IP]:/etc/ssh/*key* /etc/ssh
chown root:ssh_keys ssh_host_ecdsa_key ssh_host_ed25519_key ssh_host_rsa_key
# 注意
只有鏈接Postgresql master庫, gitlab才能正常使用。由於gitlab web登陸時就要求寫數據庫。鏈接到postgresql從庫，gitlab web登陸失敗，報500錯。ssh鏈接倉庫正常。