【Linux】【gitlab】gitlab安裝、備份、恢復、升級、內存消耗問題

時間 2019-11-20

標籤 Linux gitlab 安裝備份恢復升級內存消耗問題欄目 Linux 简体版

原文原文鏈接

前言

GitLab：GitLab 是一個用於倉庫管理系統的開源項目，使用Git做爲代碼管理工具，並在此基礎上搭建起來的web服務。
功能：Gitlab 是一個提供代碼託管、提交審覈和問題跟蹤的代碼管理平臺。對於軟件工程質量管理很是重要。
版本：GitLab 分爲社區版（CE）和企業版（EE）。
配置：建議CPU2核，內存2G以上。node

安裝

1、gitlab安裝

一、安裝gitlab yum庫

1.安裝最新版gitlab-ee（企業版）nginx

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.rpm.sh | sudo bash

2.安裝最新版gilab-ce（社區版）git

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash

二、安裝gitlab

其中ip爲gitlab的訪問地址。web

1.企業最新版：redis

EXTERNAL_URL="http://ip" yum install -y gitlab-ee

2.社區最新版：sql

EXTERNAL_URL="http://ip" yum install -y gitlab-ce

3.安裝指定版本：shell

EXTERNAL_URL="http://ip" yum install -y gitlab-ee-12.1.9-ee.0.el7.x86_64
EXTERNAL_URL="http://ip" yum install -y gitlab-ce-12.1.9-ce.0.el7.x86_64

4.安裝完成基本指令：數據庫

查看運行狀態json

gitlab-ctl status

啓動瀏覽器

gitlab-ctl start

中止

gitlab-ctl stop

重啓

gitlab-ctl restart

查看版本

cat /opt/gitlab/embedded/service/gitlab-rails/VERSION

重載配置

gitlab-ctl reconfigure

三、訪問gitlab

http://ip 瀏覽器打開

設置root管理員初始密碼。

設置郵箱及https等請參考原文。

本文參考CSDN「一縷南風」博主，原文連接：https://blog.csdn.net/weixin_43767602/article/details/84568858

2、gitlab備份、恢復

1.gitlab備份

1.設置gitlab備份目錄、權限、有限期

1 [root@localhost /]# cat /etc/gitlab/gitlab.rb | grep -v "#"
2 external_url 'http://192.168.8.127'   //安裝設定的訪問域名
3 gitlab_rails['manage_backup_path'] = true
4 gitlab_rails['backup_path'] = "/data/gitlab/backups"    //備份路徑
5 gitlab_rails['backup_archive_permissions'] = 0644     //備份文件權限
6 gitlab_rails['backup_keep_time'] = 7776000    //備份文件有效期 30天

2.建立備份路徑

1 [root@localhost /]# mkdir -vp /data/gitlab/backups
2 mkdir: 已建立目錄 "/data"
3 mkdir: 已建立目錄 "/data/gitlab"
4 mkdir: 已建立目錄 "/data/gitlab/backups"

3.重載配置

 1 [root@localhost /]# gitlab-ctl reconfigure
 2 Starting Chef Client, version 14.13.11
 3 resolving cookbooks for run list: ["gitlab-ee"]
 4 Synchronizing Cookbooks:
 5   - gitlab-ee (0.0.1)
 6   - package (0.1.0)
 7   - gitlab (0.0.1)
 8   - consul (0.1.0)
 9   - runit (4.3.0)
10   - redis (0.1.0)
11   - repmgr (0.1.0)
12   - postgresql (0.1.0)
13   - gitaly (0.1.0)
14   - letsencrypt (0.1.0)
15   - monitoring (0.1.0)
16   - registry (0.1.0)
17   - mattermost (0.1.0)
18   - nginx (0.1.0)
19   - acme (4.0.0)
20   - crond (0.1.0)
21 Installing Cookbook Gems:
22 Compiling Cookbooks...
23 Recipe: gitlab::default
24 ……
25 ……
26 ……
27      (up to date)
28 Recipe: <Dynamically Defined Resource>
29   * service[repmgrd] action nothing (skipped due to action :nothing)
30 Recipe: repmgr::repmgrd_disable
31   * runit_service[repmgrd] action disable
32     * ruby_block[disable repmgrd] action run (skipped due to only_if)
33      (up to date)
34 Recipe: gitlab-ee::geo-secondary_disable
35   * templatesymlink[Removes database_geo.yml symlink] action delete
36     * file[/var/opt/gitlab/gitlab-rails/etc/database_geo.yml] action delete (up to date)
37     * link[/opt/gitlab/embedded/service/gitlab-rails/config/database_geo.yml] action delete (up to date)
38      (up to date)
39 Recipe: <Dynamically Defined Resource>
40   * service[unicorn] action restart
41     - restart service service[unicorn]
42   * service[sidekiq] action restart
43     - restart service service[sidekiq]
44 Recipe: gitlab::gitlab-rails
45   * execute[clear the gitlab-rails cache] action run
46     - execute /opt/gitlab/bin/gitlab-rake cache:clear
47 Running handlers:
48 Running handlers complete
49 Chef Client finished, 10/708 resources updated in 29 seconds

4.開始備份

 1 [root@localhost /]# gitlab-rake gitlab:backup:create
 2 2019-09-16 11:00:23 +0800 -- Dumping database ... 
 3 Dumping PostgreSQL database gitlabhq_production ... [DONE]
 4 2019-09-16 11:00:24 +0800 -- done
 5 2019-09-16 11:00:24 +0800 -- Dumping repositories ...
 6 2019-09-16 11:00:24 +0800 -- done
 7 2019-09-16 11:00:24 +0800 -- Dumping uploads ... 
 8 2019-09-16 11:00:24 +0800 -- done
 9 2019-09-16 11:00:24 +0800 -- Dumping builds ... 
10 2019-09-16 11:00:24 +0800 -- done
11 2019-09-16 11:00:24 +0800 -- Dumping artifacts ... 
12 2019-09-16 11:00:24 +0800 -- done
13 2019-09-16 11:00:24 +0800 -- Dumping pages ... 
14 2019-09-16 11:00:24 +0800 -- done
15 2019-09-16 11:00:24 +0800 -- Dumping lfs objects ... 
16 2019-09-16 11:00:24 +0800 -- done
17 2019-09-16 11:00:24 +0800 -- Dumping container registry images ... 
18 2019-09-16 11:00:24 +0800 -- [DISABLED]
19 Creating backup archive: 1568602824_2019_09_16_12.2.5-ee_gitlab_backup.tar ... done
20 Uploading backup archive to remote storage  ... skipped
21 Deleting tmp directories ... done
22 done
23 done
24 done
25 done
26 done
27 done
28 done
29 Deleting old backups ... done. (0 removed)
30 Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
31 and are not included in this backup. You will need these files to restore a backup.
32 Please back them up manually.
33 Backup task is done.

紅字部分表示 gitlab.rb 和 gitlab-secrets.json 兩個文件包含敏感信息。未被備份到備份文件中。須要手動備份。

1 [root@localhost backups]# pwd
2 /data/gitlab/backups
3 [root@localhost backups]# ls -l
4 總用量 132
5 -rw-r--r-- 1 git git 133120 9月  16 11:00 1568602824_2019_09_16_12.2.5-ee_gitlab_backup.tar
6 [root@localhost backups]#

執行完成後在 /data/gitlab/backups 中生成了備份文件。

2.gitlab恢復備份

1.首先停用gitlab的數據鏈接部分服務

 1 [root@localhost backups]# gitlab-ctl stop unicorn
 2 ok: down: unicorn: 0s, normally up
 3 [root@localhost backups]# gitlab-ctl stop sidekiq
 4 ok: down: sidekiq: 0s, normally up
 5 [root@localhost backups]# gitlab-ctl stop nginx
 6 ok: down: nginx: 0s, normally up
 7 [root@localhost backups]# gitlab-ctl status
 8 run: alertmanager: (pid 30960) 3683s; run: log: (pid 30623) 3735s
 9 run: gitaly: (pid 30846) 3685s; run: log: (pid 30081) 3828s
10 run: gitlab-monitor: (pid 30843) 3685s; run: log: (pid 30498) 3753s
11 run: gitlab-workhorse: (pid 30820) 3686s; run: log: (pid 30349) 3784s
12 run: grafana: (pid 30980) 3682s; run: log: (pid 30750) 3698s
13 run: logrotate: (pid 37253) 178s; run: log: (pid 30384) 3777s
14 down: nginx: 4s, normally up; run: log: (pid 30368) 3781s
15 run: node-exporter: (pid 30830) 3685s; run: log: (pid 30418) 3764s
16 run: postgres-exporter: (pid 30971) 3683s; run: log: (pid 30650) 3730s
17 run: postgresql: (pid 30127) 3825s; run: log: (pid 30139) 3821s
18 run: prometheus: (pid 30943) 3684s; run: log: (pid 30588) 3741s
19 run: redis: (pid 29960) 3838s; run: log: (pid 29972) 3835s
20 run: redis-exporter: (pid 30854) 3684s; run: log: (pid 30522) 3747s
21 down: sidekiq: 13s, normally up; run: log: (pid 30324) 3788s
22 down: unicorn: 17s, normally up; run: log: (pid 30307) 3792s

2.執行恢復

遠程複製的備份須要 chmod 777 1568602824_2019_09_16_12.2.5-ee..... 賦予權限

 1 [root@localhost backups]# gitlab-rake gitlab:backup:restore BACKUP=1568602824_2019_09_16_12.2.5-ee
 2 Unpacking backup ... done
 3 Before restoring the database, we will remove all existing
 4 tables to avoid future upgrade problems. Be aware that if you have
 5 custom tables in the GitLab database these tables and all data will be
 6 removed.
 7 ……………………………………
 8 Do you want to continue (yes/no)? yes
 9 This task will now rebuild the authorized_keys file.
10 You will lose any data stored in the authorized_keys file.
11 Do you want to continue (yes/no)? yes
12 
13 Deleting tmp directories ... done
14 done
15 done
16 done
17 done
18 done
19 done
20 done
21 Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
22 and are not included in this backup. You will need to restore these files manually.
23 Restore task is done.

備份tar包必定要放到備份路徑下。恢復是刪除原有數據，恢復備份tar包中的數據。

若是是在其餘服務器恢復備份，必定要記得將 gitlab.rb 和 gitlab-secrets.json 手動複製到相應路徑下。

gitlab.rb路徑：/etc/gitlab/gitlab.rb

gitlab-secrets.json路徑：/etc/gitlab/gitlab-secrets.json

3.重載配置

gitlab-ctl reconfigure

4.重啓gitlab，check檢查

 1 [root@localhost backups]# gitlab-ctl restart
 2 ok: run: alertmanager: (pid 30960) 3785s
 3 ok: run: gitaly: (pid 30846) 3787s
 4 ok: run: gitlab-monitor: (pid 30843) 3787s
 5 ok: run: gitlab-workhorse: (pid 30820) 3788s
 6 ok: run: grafana: (pid 30980) 3784s
 7 ok: run: logrotate: (pid 37253) 280s
 8 ok: run: nginx: (pid 37535) 0s
 9 ok: run: node-exporter: (pid 30830) 3787s
10 ok: run: postgres-exporter: (pid 30971) 3785s
11 ok: run: postgresql: (pid 30127) 3927s
12 ok: run: prometheus: (pid 30943) 3786s
13 ok: run: redis: (pid 29960) 3940s
14 ok: run: redis-exporter: (pid 30854) 3786s
15 ok: run: sidekiq: (pid 37547) 1s
16 ok: run: unicorn: (pid 37553) 0s
17 [root@localhost backups]# gitlab-rake gitlab:check SANITIZE=true
18 Checking GitLab subtasks ...
19 
20 Checking GitLab Shell ...
21 
22 GitLab Shell: ... GitLab Shell version >= 9.3.0 ? ... OK (9.3.0)
23 Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
24 Check GitLab API access: OK
25 Redis available via internal API: OK
26 
27 Access to /var/opt/gitlab/.ssh/authorized_keys: OK
28 gitlab-shell self-check successful
29 
30 Checking GitLab Shell ... Finished
31 
32 Checking Gitaly ...
33 
34 Gitaly: ... default ... OK
35 
36 Checking Gitaly ... Finished
37 
38 Checking Sidekiq ...
39 
40 Sidekiq: ... Running? ... yes
41 Number of Sidekiq processes ... 1
42 
43 Checking Sidekiq ... Finished
44 
45 Checking Incoming Email ...
46 
47 Incoming Email: ... Reply by email is disabled in config/gitlab.yml
48 
49 Checking Incoming Email ... Finished
50 
51 Checking LDAP ...
52 
53 LDAP: ... LDAP is disabled in config/gitlab.yml
54 
55 Checking LDAP ... Finished
56 
57 Checking GitLab App ...
58 
59 Git configured correctly? ... yes
60 Database config exists? ... yes
61 All migrations up? ... yes
62 Database contains orphaned GroupMembers? ... no
63 GitLab config exists? ... yes
64 GitLab config up to date? ... yes
65 Log directory writable? ... yes
66 Tmp directory writable? ... yes
67 Uploads directory exists? ... yes
68 Uploads directory has correct permissions? ... yes
69 Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet)
70 Init script exists? ... skipped (omnibus-gitlab has no init script)
71 Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
72 Projects have namespace: ... can't check, you have no projects
73 Redis version >= 2.8.0? ... yes
74 Ruby version >= 2.5.3 ? ... yes (2.6.3)
75 Git version >= 2.22.0 ? ... yes (2.22.0)
76 Git user has default SSH configuration? ... yes
77 Active users: ... 1
78 Elasticsearch version 5.6 - 6.x? ... skipped (elasticsearch is disabled)
79 
80 Checking GitLab App ... Finished
81 
82 
83 Checking GitLab subtasks ... Finished

ok，至此恢復完成

3、gitlab版本升級

1.中止數據傳輸服務

 1 [root@localhost backups]# gitlab-ctl stop unicorn
 2 ok: down: unicorn: 0s, normally up
 3 [root@localhost backups]# gitlab-ctl stop sidekiq
 4 ok: down: sidekiq: 0s, normally up
 5 [root@localhost backups]# gitlab-ctl stop nginx
 6 ok: down: nginx: 1s, normally up
 7 [root@localhost backups]# gitlab-ctl status
 8 run: gitaly: (pid 73164) 939s; run: log: (pid 72843) 968s
 9 run: gitlab-monitor: (pid 73232) 938s; run: log: (pid 73021) 957s
10 run: gitlab-workhorse: (pid 75742) 136s; run: log: (pid 72919) 962s
11 run: logrotate: (pid 72985) 960s; run: log: (pid 72984) 960s
12 down: nginx: 5s, normally up; run: log: (pid 72960) 961s
13 run: node-exporter: (pid 73003) 959s; run: log: (pid 73002) 959s
14 run: postgres-exporter: (pid 73253) 937s; run: log: (pid 73086) 948s
15 run: postgresql: (pid 72644) 997s; run: log: (pid 72643) 997s
16 run: prometheus: (pid 73241) 938s; run: log: (pid 73064) 950s
17 run: redis: (pid 72598) 1003s; run: log: (pid 72597) 1003s
18 run: redis-exporter: (pid 73048) 951s; run: log: (pid 73047) 951s
19 down: sidekiq: 7s, normally up; run: log: (pid 72818) 974s
20 down: unicorn: 10s, normally up; run: log: (pid 72778) 976s

2.下載最新的ce/ee rpm包（清華源https://mirrors.tuna.tsinghua.edu.cn/gitlab-ee/yum/el7/ 或者https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/）

wget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ee/yum/el7/gitlab-ee-12.2.5-ee.0.el7.x86_64.rpm

gitlab10.0.0版本升級最新版本須要先升級到11.11.0，再升級12.2.5！

3.升級ce/ee

rpm -Uvh gitlab-ee-12.2.5-ee.0.el7.x86_64.rpm

4.重載配置

gitlab-ctl reconfigure

5.重啓gitlab

gitlab-ctl restart

4、gitlab解決內存消耗問題

 1 [root@storage100 backups]# cat /etc/gitlab/gitlab.rb | grep -v "#"
 2 #進程超時時間
 3 unicorn['worker_timeout'] = 60
 4 #進程數
 5 unicorn['worker_processes'] = 10
 6 #進程最小內存
 7 unicorn['worker_memory_limit_min'] = "200 * 1 << 20"
 8 #進程最大內存
 9 unicorn['worker_memory_limit_max'] = "300 * 1 << 20"
10 #併發數
11 sidekiq['concurrency'] = 16
12 #數據庫緩存
13 postgresql['shared_buffers'] = "256MB"
14 #數據庫併發數
15 postgresql['max_worker_processes'] = 8

參考CSDN博主"歐陽鵬",原文連接：https://blog.csdn.net/ouyang_peng/article/details/84066417

附gitlab備份腳本：

#!/bin/bash

#Gitlab 備份地址
LocalBackDir=/home/gitlab/data/backups

#服務機gitlab配置文件地址
#正式環境地址：
ConfigDir=/etc/gitlab/gitlab.rb

#nginx配置文件地址
#正式環境地址：NginxDir=/home/dgd/gitlab/data/nginx/conf

#郵件配置地址
#MailDir=/etc/postfix/main.cf

#Backup server 存儲路徑
RemoteBackDir=/home/gitlab/data/backups

#遠程備份使用用戶及端口
RemoteUser=root
RemotePort=22

#備份服務器IP
RemoteIP=192.168.8.200

#以當前時間戳建立備份目錄
bakname=$(date -d "today" +"%Y%m%d_%H%M%S")
BakDir=$LocalBackDir/$bakname
mkdir $BakDir

#備份日誌文件
LogFile=$LocalBackDir/remote_backup.log

#新建備份日誌文件
touch $LogFile

#記錄配置文件備份日誌
echo "Gitlab configure file auto backup at local server, start at  $(date +"%Y-%m-%d %H:%M:%S")" >>  $LogFile
echo "--------------------------------------------------------------------------" >> $LogFile

#拷貝配置文件
cp $ConfigDir $BakDir >> $LogFile 2>&1
#cp -r $NginxDir $BakDir >> $LogFile 2>&1
#cp $MailDir $BakDir >> $LogFile 2>&1

#記錄本地生成gitlab備份日誌
echo "Gitlab auto backup at local server, start at  $(date +"%Y-%m-%d %H:%M:%S")" >>  $LogFile
echo "--------------------------------------------------------------------------" >> $LogFile

#執行gitlab本地備份
gitlab-rake gitlab:backup:create >> $LogFile 2>&1

# $?符號顯示上一條命令的返回值，若是爲0則表明執行成功，其餘表示失敗
if [ $? -eq 0 ];then
   #追加日誌到日誌文件
   echo "--------------------------------Success!-------------------------------" >> $LogFile
   echo "Gitlab auto backup at local server, end at $(date +"%Y-%m-%d %H:%M:%S")" >> $LogFile
else
   #追加日誌到日誌文件
   echo "--------------------------------Failed!----------------------------------" >> $LogFile
   echo "Gitlab auto backup at local server failed at $(date +"%Y-%m-%d %H:%M:%S")" >> $LogFile
fi


#查找本地備份目錄修改時間爲10分鐘之內且後綴爲.tar的Gitlab備份文件
Backfile_Send_To_Remote=`find $LocalBackDir -type f  -mmin -10 -name '*.tar'` >> $LogFile 2>&1

#移動生成的備份文件到配置文件備份地址
mv -bfu $Backfile_Send_To_Remote $BakDir 


#記錄備份日誌
echo "$(date +"%Y-%m-%d %H:%M:%S") Gitlab auto backup to remote server." >> $LogFile
echo "--------------------------------------------" >> $LogFile

#打印每次備份的檔案名
echo "The files need send to remote server is: $Backfile_Send_To_Remote" >> $LogFile

# 本地傳輸Gitlab備份檔案到遠程
scp -r  $BakDir $RemoteUser@$RemoteIP:$RemoteBackDir

# 備份結果追加到備份日誌
if [ $? -eq 0 ];then
  echo ""
  echo "$(date +"%Y-%m-%d %H:%M:%S") Gitlab Remote Backup Succeed!" >> $LogFile
else
  echo "$(date +"%Y-%m-%d %H:%M:%S") Gitlab Remote Backup Failed!" >> $LogFile
fi