分佈式文件系統之MogileFS的安裝使用

時間 2020-09-21

原文原文鏈接

　　1、簡介html

　　MogileFS是一個開源的分佈式文件存儲系統，由LiveJournal旗下的Danga Interactive公司開發；它主要由三部分組成，第一部分是server端，server端主要包括mogilefsd和mogstored兩個應用程序。mogilefsd實現的是tracker，它經過數據庫來保存元數據信息，包括站點domain、class、host等；mogstored是存儲節點(store node)，它實際上是個WebDAV服務，默認監聽在7500端口，接受客戶端的文件存儲請求。在MogileFS安裝完後，要運行mogadm工具將全部的store node註冊到mogilefsd的數據庫裏，mogilefsd會對這些節點進行管理和監控。第二部是mogilefs的utils,這部分主要是一些工具，好比mogadm、mogupload、mogfileinfo 等等；第三部分是客戶端API，mogilefs支持衆多編程語言的客戶端API接口，使用對應的客戶端API接口，咱們能夠編寫mogilefs的客戶端，從而實現來管理mogilefs上的文件；前端

　　2、Mogilefs架構圖node

　　提示：tracker的主要做用是代理前端用戶的請求到數據庫查詢相關文件的元信息，同時它還要監控註冊到tracker的storagenode的健康狀態，一旦後端storage掛掉，它會馬上把對應的storage節點上的device從集羣信息中刪除（對應數據還在，只是在集羣的狀態信息看不到對應設備），上線時再加入到集羣信息；客戶端經過向tracker發送請求文件的信息，tracker經過去數據庫中找到對應的文件所在storage節點的ip地址、端口、設備id以及文件fid，而後經過http連接的形式從tracker返回給客戶端，客戶端拿到這tracker的返回的文件地址，而後在去對應的storage請求文件，從而在客戶端就訪問到對應的文件內容信息；客戶端和tracker，storage之間通訊都是http協議，從本質上講storage就是一個web服務器；因此對於客戶端咱們可使用web服務器的反向代理，也能夠是調用mogilefsAPI寫的客戶端程序；mysql

　　3、Mogilefs安裝部署nginx

　　環境說明web

名稱	角色	ip地址	端口
node01	tracker+mysql+storage	192.168.0.41	7001/7500/7501/3306
node02	tracker+storage	192.168.0.42	7001/7500/7501
node03	storage	192.168.0.43	7500/7501

　　各server時間同步sql

　　提示：若是有必要，能夠配置各節點用hosts文件解析；數據庫

　　安裝環境依賴包編程

[root@node01 ~]# yum install perl-Net-Netmask perl-IO-stringy perl-Sys-Syslog perl-IO-AIO -y 
Loaded plugins: fastestmirror
base                                                                                                            | 3.6 kB  00:00:00     
epel                                                                                                            | 4.7 kB  00:00:00     
extras                                                                                                          | 2.9 kB  00:00:00     
updates                                                                                                         | 2.9 kB  00:00:00     
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package perl-IO-AIO.x86_64 0:4.20-1.el7 will be installed
--> Processing Dependency: perl(common::sense) for package: perl-IO-AIO-4.20-1.el7.x86_64
---> Package perl-IO-stringy.noarch 0:2.110-22.el7 will be installed
---> Package perl-Net-Netmask.noarch 0:1.9015-13.el7 will be installed
---> Package perl-Sys-Syslog.x86_64 0:0.33-3.el7 will be installed
--> Running transaction check
---> Package perl-common-sense.noarch 0:3.6-4.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================================
 Package                                Arch                        Version                            Repository                 Size
=======================================================================================================================================
Installing:
 perl-IO-AIO                            x86_64                      4.20-1.el7                         epel                      117 k
 perl-IO-stringy                        noarch                      2.110-22.el7                       base                       71 k
 perl-Net-Netmask                       noarch                      1.9015-13.el7                      epel                       25 k
 perl-Sys-Syslog                        x86_64                      0.33-3.el7                         base                       42 k
Installing for dependencies:
 perl-common-sense                      noarch                      3.6-4.el7                          epel                       28 k

Transaction Summary
=======================================================================================================================================
Install  4 Packages (+1 Dependent package)

Total download size: 283 k
Installed size: 636 k
Downloading packages:
(1/5): perl-Sys-Syslog-0.33-3.el7.x86_64.rpm                                                                    |  42 kB  00:00:00     
(2/5): perl-Net-Netmask-1.9015-13.el7.noarch.rpm                                                                |  25 kB  00:00:00     
(3/5): perl-IO-AIO-4.20-1.el7.x86_64.rpm                                                                        | 117 kB  00:00:00     
(4/5): perl-common-sense-3.6-4.el7.noarch.rpm                                                                   |  28 kB  00:00:00     
(5/5): perl-IO-stringy-2.110-22.el7.noarch.rpm                                                                  |  71 kB  00:00:00     
---------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                  642 kB/s | 283 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : perl-common-sense-3.6-4.el7.noarch                                                                                  1/5 
  Installing : perl-IO-AIO-4.20-1.el7.x86_64                                                                                       2/5 
  Installing : perl-Net-Netmask-1.9015-13.el7.noarch                                                                               3/5 
  Installing : perl-Sys-Syslog-0.33-3.el7.x86_64                                                                                   4/5 
  Installing : perl-IO-stringy-2.110-22.el7.noarch                                                                                 5/5 
  Verifying  : perl-IO-AIO-4.20-1.el7.x86_64                                                                                       1/5 
  Verifying  : perl-common-sense-3.6-4.el7.noarch                                                                                  2/5 
  Verifying  : perl-IO-stringy-2.110-22.el7.noarch                                                                                 3/5 
  Verifying  : perl-Sys-Syslog-0.33-3.el7.x86_64                                                                                   4/5 
  Verifying  : perl-Net-Netmask-1.9015-13.el7.noarch                                                                               5/5 

Installed:
  perl-IO-AIO.x86_64 0:4.20-1.el7           perl-IO-stringy.noarch 0:2.110-22.el7       perl-Net-Netmask.noarch 0:1.9015-13.el7      
  perl-Sys-Syslog.x86_64 0:0.33-3.el7      

Dependency Installed:
  perl-common-sense.noarch 0:3.6-4.el7                                                                                                 

Complete!
[root@node01 ~]#

　　提示：三個節點都須要安裝環境依賴的包；後端

　　上傳rpm包到服務器並安裝

[root@node01 ~]# rz
rz waiting to receive.
 zmodem trl+C ȡ

  100%       1 KB    1 KB/s 00:00:01       0 Errors.el7.centos.noarch.rpm...
  100%     173 KB  173 KB/s 00:00:01       0 Errorsentos.noarch.rpm...
  100%      26 KB   26 KB/s 00:00:01       0 Errorsentos.noarch.rpm...
  100%      75 KB   75 KB/s 00:00:01       0 Errorsh.rpm...
  100%      27 KB   27 KB/s 00:00:01       0 Errors.rpm...
  100%      29 KB   29 KB/s 00:00:01       0 Errors.noarch.rpm...
  100%     262 KB  262 KB/s 00:00:01       0 Errors

[root@node01 ~]# ls
MogileFS-Server-2.46-2.el7.centos.noarch.rpm            perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm
MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm  perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm
MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm  perl-Perlbal-1.78-1.el6.noarch.rpm
MogileFS-Utils-2.19-1.el7.centos.noarch.rpm
[root@node01 ~]# yum install ./*.rpm -y
Loaded plugins: fastestmirror
Examining ./MogileFS-Server-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-2.46-2.el7.centos.noarch
Marking ./MogileFS-Server-2.46-2.el7.centos.noarch.rpm to be installed
Examining ./MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch
Marking ./MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm to be installed
Examining ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogstored-2.46-2.el7.centos.noarch
Marking ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm to be installed
Examining ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm: MogileFS-Utils-2.19-1.el7.centos.noarch
Marking ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm to be installed
Examining ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm: perl-Danga-Socket-1.61-1.el6.rf.noarch
Marking ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm to be installed
Examining ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm: perl-MogileFS-Client-1.14-1.el7.centos.noarch
Marking ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm to be installed
Examining ./perl-Perlbal-1.78-1.el6.noarch.rpm: perl-Perlbal-1.78-1.el6.noarch
Marking ./perl-Perlbal-1.78-1.el6.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package MogileFS-Server.noarch 0:2.46-2.el7.centos will be installed
---> Package MogileFS-Server-mogilefsd.noarch 0:2.46-2.el7.centos will be installed
--> Processing Dependency: perl(DBI) >= 1.44 for package: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
--> Processing Dependency: perl(DBD::mysql) >= 3 for package: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch
---> Package MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos will be installed
……省略部份內容……
Installed:
  MogileFS-Server.noarch 0:2.46-2.el7.centos                         MogileFS-Server-mogilefsd.noarch 0:2.46-2.el7.centos              
  MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos               MogileFS-Utils.noarch 0:2.19-1.el7.centos                         
  perl-Danga-Socket.noarch 0:1.61-1.el6.rf                           perl-MogileFS-Client.noarch 0:1.14-1.el7.centos                   
  perl-Perlbal.noarch 0:1.78-1.el6                                  

Dependency Installed:
  mailcap.noarch 0:2.1.41-2.el7                                  perl-BSD-Resource.x86_64 0:1.29.07-1.el7                             
  perl-Business-ISBN.noarch 0:2.06-2.el7                         perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7                  
  perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7                   perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7                          
  perl-DBD-MySQL.x86_64 0:4.023-6.el7                            perl-DBI.x86_64 0:1.627-4.el7                                        
  perl-Data-Dumper.x86_64 0:2.145-3.el7                          perl-Digest.noarch 0:1.17-245.el7                                    
  perl-Digest-MD5.x86_64 0:2.52-3.el7                            perl-Encode-Locale.noarch 0:1.03-5.el7                               
  perl-File-Listing.noarch 0:6.04-7.el7                          perl-HTML-Parser.x86_64 0:3.71-4.el7                                 
  perl-HTML-Tagset.noarch 0:3.20-15.el7                          perl-HTTP-Cookies.noarch 0:6.01-5.el7                                
  perl-HTTP-Daemon.noarch 0:6.01-8.el7                           perl-HTTP-Date.noarch 0:6.02-8.el7                                   
  perl-HTTP-Message.noarch 0:6.06-6.el7                          perl-HTTP-Negotiate.noarch 0:6.01-5.el7                              
  perl-IO-Compress.noarch 0:2.061-2.el7                          perl-IO-HTML.noarch 0:1.00-2.el7                                     
  perl-IO-Socket-IP.noarch 0:0.21-5.el7                          perl-IO-Socket-SSL.noarch 0:1.94-7.el7                               
  perl-LWP-MediaTypes.noarch 0:6.02-2.el7                        perl-Mozilla-CA.noarch 0:20130114-5.el7                              
  perl-Net-Daemon.noarch 0:0.48-5.el7                            perl-Net-HTTP.noarch 0:6.06-2.el7                                    
  perl-Net-LibIDN.x86_64 0:0.12-15.el7                           perl-Net-SSLeay.x86_64 0:1.55-6.el7                                  
  perl-PlRPC.noarch 0:0.2020-14.el7                              perl-Sys-Syscall.noarch 0:0.25-10.el7                                
  perl-TimeDate.noarch 1:2.30-2.el7                              perl-URI.noarch 0:1.60-9.el7                                         
  perl-WWW-RobotRules.noarch 0:6.02-5.el7                        perl-libwww-perl.noarch 0:6.05-2.el7                                 

Complete!
[root@node01 ~]#

　　提示：安裝環境依賴和以上rpm包，首先要配置好epel源；由於有些依賴包，它來自epel源；相關包我已經上傳至個人網盤http://www.qiuhom.com/d/383f34f57cf847beb296/；

　　安裝mariadb-server

　　配置yum源

[root@node01 ~]# cat /etc/yum.repos.d/mairadb.repo 
[mariadb]
name=mariadb repo
baseurl=https://mirrors.tuna.tsinghua.edu.cn/mariadb//mariadb-10.1.46/yum/centos/7/x86_64/
gpgcheck=0
[root@node01 ~]#

　　安裝mariadb-server

[root@node01 ~]# yum install MariaDB-server -y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package MariaDB-server.x86_64 0:10.1.46-1.el7.centos will be installed
--> Processing Dependency: MariaDB-client for package: MariaDB-server-10.1.46-1.el7.centos.x86_64
--> Processing Dependency: rsync for package: MariaDB-server-10.1.46-1.el7.centos.x86_64
--> Processing Dependency: MariaDB-common for package: MariaDB-server-10.1.46-1.el7.centos.x86_64
--> Processing Dependency: galera for package: MariaDB-server-10.1.46-1.el7.centos.x86_64
--> Processing Dependency: libjemalloc.so.1()(64bit) for package: MariaDB-server-10.1.46-1.el7.centos.x86_64
--> Running transaction check
……省略部份內容……

Installed:
  MariaDB-server.x86_64 0:10.1.46-1.el7.centos                       MariaDB-shared.x86_64 0:10.1.46-1.el7.centos                      

Dependency Installed:
  MariaDB-client.x86_64 0:10.1.46-1.el7.centos                       MariaDB-common.x86_64 0:10.1.46-1.el7.centos                      
  boost-program-options.x86_64 0:1.53.0-28.el7                       galera.x86_64 0:25.3.29-1.rhel7.el7.centos                        
  jemalloc.x86_64 0:3.6.0-1.el7                                      rsync.x86_64 0:3.1.2-10.el7                                       

Replaced:
  mariadb-libs.x86_64 1:5.5.56-2.el7                                                                                                   

Complete!
[root@node01 ~]#

　　提示：mariadb-server包名是大寫的MariaDB-server；

　　配置mariadb忽略名稱解析

　　設置mariadb開機自動啓動，並啓動mariadb

　　添加用於mogilefs訪問數據庫的帳號

　　初始化數據庫

[root@node01 ~]# mogdbsetup --help
Usage: mogdbsetup [opts]

Options:

                  Default      Description
                  ============ ===========================================
 --verbose        <off>        Be verbose about what's happening.

 --dbhost=        localhost    hostname or IP to database server.

 --dbport=        dbd default  port number to database server.

 --dbname=        mogilefs     database name to create/upgrade.

 --dbrootuser=    root         Database administrator username.  Only needed
                               for initial setup, not subsequent upgrades.

 --dbrootpass=    <blank>      Database administrator password.  Only needed
                               for initial setup, not subsequent upgrades.

 --dbuser=        mogile       Regular database user to create and/or use
                               for MogileFS database.  This is what the
                               mogilefsd trackers connect as.

 --dbpass=        <blank>      You should change this, especially if your
                               database servers are accessible to other users
                               on the network.  But they shouldn't be
                               if you're running MogileFS, because MogileFS
                               assumes your network is closed.

  --type=         MySQL        Which MogileFS::Store implementation to use.
                               Available: MySQL, Postgres

  --yes                        Run without questions.

[root@node01 ~]# mogdbsetup --dbhost=192.168.0.41 --dbport=3306 --dbname=mogilefs --dbrootpass=admin --dbuser=mogilefs --dbpass=mogilefs

This will attempt to setup or upgrade your MogileFS database.
It won't destroy existing data.
Run with --help for more information.  Run with --yes to shut up these prompts.

Continue? [N/y]: y

Create/Upgrade database name 'mogilefs'? [Y/n]: y

Grant all privileges to user 'mogilefs', connecting from anywhere, to the mogilefs database 'mogilefs'? [Y/n]: n
Stopped.
[root@node01 ~]# mogdbsetup --dbhost=192.168.0.41 --dbport=3306 --dbname=mogilefs --dbrootpass=admin --dbuser=mogilefs --dbpass=mogilefs

This will attempt to setup or upgrade your MogileFS database.
It won't destroy existing data.
Run with --help for more information.  Run with --yes to shut up these prompts.

Continue? [N/y]: y
[root@node01 ~]#

　　提示：若是不想讓mogdbsetup在數據庫中建立一個任何地址均可訪問的mogilefs的用戶，在最後的選項中使用n便可，而後在重複執行上述命令便可；若是不想交互式輸入y，也能夠直接在命令後面用--yes；

　　驗證：在數據庫中查看對應的庫是否都已建立，對應庫中是否都有表？

[root@node01 ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 7
Server version: 10.1.46-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mogilefs           |
| mysql              |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.01 sec)

MariaDB [(none)]> use mogilefs
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [mogilefs]> show tables;
+----------------------+
| Tables_in_mogilefs   |
+----------------------+
| checksum             |
| class                |
| device               |
| domain               |
| file                 |
| file_on              |
| file_on_corrupt      |
| file_to_delete       |
| file_to_delete2      |
| file_to_delete_later |
| file_to_queue        |
| file_to_replicate    |
| fsck_log             |
| host                 |
| server_settings      |
| tempfile             |
| unreachable_fids     |
+----------------------+
17 rows in set (0.00 sec)

MariaDB [mogilefs]>

　　到此數據庫就準備好了

　　配置mogilefsd鏈接數據庫

　　提示：咱們只須要修改紅框中的內容便可；db_dsn用於指定鏈接數據庫的驅動，以及數據庫庫名和主機地址；db_user用於指定鏈接數據庫所使用的用戶，db_pass用於指定鏈接數據庫的密碼；listen用於指定mogilefsd監聽的ip地址和端口；後面的配置就是和啓動的進程數量的配置，這個能夠根據業務的規模來設置；

　　啓動mogilefsd

　　提示：在centos7上啓動mogilefsd時，會報錯，咱們能夠不用管這個錯誤，實際上mogilefsd是啓動成功的，只要看到對應的端口和進程起來，表示mogilefsd啓動就沒有問題；

　　配置node01上的mogstored

[root@node01 ~]# cat /etc/mogilefs/mogstored.conf
maxconns = 10000
httplisten = 0.0.0.0:7500
mgmtlisten = 0.0.0.0:7501
docroot = /data/mogilefs
[root@node01 ~]# mkdir /data/mogilefs/ -pv
mkdir: created directory ‘/data’
mkdir: created directory ‘/data/mogilefs/’
[root@node01 ~]# chown -R mogilefs.mogilefs /data/mogilefs/
[root@node01 ~]# ll -d /data/mogilefs/
drwxr-xr-x 2 mogilefs mogilefs 6 Sep 16 13:37 /data/mogilefs/
[root@node01 ~]#

　　啓動mogstored

　　提示：在centos7上啓動mogstored也會同啓動mogilefsd同樣報錯，這個主要是由於systemd的緣故，在centos6不會報錯；到此，node01上的mariadb、mogilefsd和mogstored都已經安裝配置好了；

　　複製安裝包到node02和node03

　　在node02上安裝

[root@node02 ~]# yum install perl-Net-Netmask perl-IO-stringy perl-Sys-Syslog perl-IO-AIO ./*.rpm -y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Package perl-Net-Netmask-1.9015-13.el7.noarch already installed and latest version
Package perl-IO-stringy-2.110-22.el7.noarch already installed and latest version
Package perl-Sys-Syslog-0.33-3.el7.x86_64 already installed and latest version
Package perl-IO-AIO-4.20-1.el7.x86_64 already installed and latest version
Examining ./MogileFS-Server-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-2.46-2.el7.centos.noarch
Marking ./MogileFS-Server-2.46-2.el7.centos.noarch.rpm to be installed
Examining ./MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch
Marking ./MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm to be installed
Examining ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogstored-2.46-2.el7.centos.noarch
Marking ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm to be installed
Examining ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm: MogileFS-Utils-2.19-1.el7.centos.noarch
Marking ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm to be installed
Examining ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm: perl-Danga-Socket-1.61-1.el6.rf.noarch
Marking ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm to be installed
Examining ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm: perl-MogileFS-Client-1.14-1.el7.centos.noarch
Marking ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm to be installed
Examining ./perl-Perlbal-1.78-1.el6.noarch.rpm: perl-Perlbal-1.78-1.el6.noarch
Marking ./perl-Perlbal-1.78-1.el6.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package MogileFS-Server.noarch 0:2.46-2.el7.centos will be installed
---> Package MogileFS-Server-mogilefsd.noarch 0:2.46-2.el7.centos will be installed
--> Processing Dependency: perl(DBI) >= 1.44 for package: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch
……省略部份內容……
Installed:
  MogileFS-Server.noarch 0:2.46-2.el7.centos                                                              
  MogileFS-Server-mogilefsd.noarch 0:2.46-2.el7.centos                                                    
  MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos                                                    
  MogileFS-Utils.noarch 0:2.19-1.el7.centos                                                               
  perl-Danga-Socket.noarch 0:1.61-1.el6.rf                                                                
  perl-MogileFS-Client.noarch 0:1.14-1.el7.centos                                                         
  perl-Perlbal.noarch 0:1.78-1.el6                                                                        

Dependency Installed:
  mailcap.noarch 0:2.1.41-2.el7                    perl-BSD-Resource.x86_64 0:1.29.07-1.el7               
  perl-Business-ISBN.noarch 0:2.06-2.el7           perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7    
  perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7     perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7            
  perl-DBD-MySQL.x86_64 0:4.023-6.el7              perl-DBI.x86_64 0:1.627-4.el7                          
  perl-Data-Dumper.x86_64 0:2.145-3.el7            perl-Digest.noarch 0:1.17-245.el7                      
  perl-Digest-MD5.x86_64 0:2.52-3.el7              perl-Encode-Locale.noarch 0:1.03-5.el7                 
  perl-File-Listing.noarch 0:6.04-7.el7            perl-HTML-Parser.x86_64 0:3.71-4.el7                   
  perl-HTML-Tagset.noarch 0:3.20-15.el7            perl-HTTP-Cookies.noarch 0:6.01-5.el7                  
  perl-HTTP-Daemon.noarch 0:6.01-8.el7             perl-HTTP-Date.noarch 0:6.02-8.el7                     
  perl-HTTP-Message.noarch 0:6.06-6.el7            perl-HTTP-Negotiate.noarch 0:6.01-5.el7                
  perl-IO-Compress.noarch 0:2.061-2.el7            perl-IO-HTML.noarch 0:1.00-2.el7                       
  perl-IO-Socket-IP.noarch 0:0.21-5.el7            perl-IO-Socket-SSL.noarch 0:1.94-7.el7                 
  perl-LWP-MediaTypes.noarch 0:6.02-2.el7          perl-Mozilla-CA.noarch 0:20130114-5.el7                
  perl-Net-Daemon.noarch 0:0.48-5.el7              perl-Net-HTTP.noarch 0:6.06-2.el7                      
  perl-Net-LibIDN.x86_64 0:0.12-15.el7             perl-Net-SSLeay.x86_64 0:1.55-6.el7                    
  perl-PlRPC.noarch 0:0.2020-14.el7                perl-Sys-Syscall.noarch 0:0.25-10.el7                  
  perl-TimeDate.noarch 1:2.30-2.el7                perl-URI.noarch 0:1.60-9.el7                           
  perl-WWW-RobotRules.noarch 0:6.02-5.el7          perl-libwww-perl.noarch 0:6.05-2.el7                   

Complete!
[root@node02 ~]#

　　把node01上的mogilefsd的配置文件複製到node02上，而後啓動mogilefsd

　　把node01上的mogstored配置文件複製到node02上，而後在node02上建立對應的目錄並修改屬主和屬組爲mogilefs，而後啓動mogstored

　　到此node02上的mogilefsd和mogstored就安裝配置啓動好了；

　　在node03上安裝配置mogstored

[root@node03 ~]# ls
MogileFS-Server-2.46-2.el7.centos.noarch.rpm            perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm
MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm  perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm
MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm  perl-Perlbal-1.78-1.el6.noarch.rpm
MogileFS-Utils-2.19-1.el7.centos.noarch.rpm
[root@node03 ~]# yum install ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm ./perl-*.rpm perl-Net-Netmask perl-IO-stringy perl-Sys-Syslog perl-IO-AIO -y    
Loaded plugins: fastestmirror
Examining ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogstored-2.46-2.el7.centos.noarch
Marking ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm to be installed
Examining ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm: MogileFS-Utils-2.19-1.el7.centos.noarch
Marking ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm to be installed
Examining ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm: perl-Danga-Socket-1.61-1.el6.rf.noarch
Marking ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm to be installed
Examining ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm: perl-MogileFS-Client-1.14-1.el7.centos.noarch
Marking ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm to be installed
Examining ./perl-Perlbal-1.78-1.el6.noarch.rpm: perl-Perlbal-1.78-1.el6.noarch
Marking ./perl-Perlbal-1.78-1.el6.noarch.rpm to be installed
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Package perl-Net-Netmask-1.9015-13.el7.noarch already installed and latest version
Package perl-IO-stringy-2.110-22.el7.noarch already installed and latest version
Package perl-Sys-Syslog-0.33-3.el7.x86_64 already installed and latest version
Package perl-IO-AIO-4.20-1.el7.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos will be installed
---> Package MogileFS-Utils.noarch 0:2.19-1.el7.centos will be installed
--> Processing Dependency: perl(Compress::Zlib) for package: MogileFS-Utils-2.19-1.el7.centos.noarch
……省略部份內容……
Installed:
  MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos   MogileFS-Utils.noarch 0:2.19-1.el7.centos        
  perl-Danga-Socket.noarch 0:1.61-1.el6.rf               perl-MogileFS-Client.noarch 0:1.14-1.el7.centos  
  perl-Perlbal.noarch 0:1.78-1.el6                      

Dependency Installed:
  mailcap.noarch 0:2.1.41-2.el7                    perl-BSD-Resource.x86_64 0:1.29.07-1.el7               
  perl-Business-ISBN.noarch 0:2.06-2.el7           perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7    
  perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7     perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7            
  perl-Data-Dumper.x86_64 0:2.145-3.el7            perl-Digest.noarch 0:1.17-245.el7                      
  perl-Digest-MD5.x86_64 0:2.52-3.el7              perl-Encode-Locale.noarch 0:1.03-5.el7                 
  perl-File-Listing.noarch 0:6.04-7.el7            perl-HTML-Parser.x86_64 0:3.71-4.el7                   
  perl-HTML-Tagset.noarch 0:3.20-15.el7            perl-HTTP-Cookies.noarch 0:6.01-5.el7                  
  perl-HTTP-Daemon.noarch 0:6.01-8.el7             perl-HTTP-Date.noarch 0:6.02-8.el7                     
  perl-HTTP-Message.noarch 0:6.06-6.el7            perl-HTTP-Negotiate.noarch 0:6.01-5.el7                
  perl-IO-Compress.noarch 0:2.061-2.el7            perl-IO-HTML.noarch 0:1.00-2.el7                       
  perl-IO-Socket-IP.noarch 0:0.21-5.el7            perl-IO-Socket-SSL.noarch 0:1.94-7.el7                 
  perl-LWP-MediaTypes.noarch 0:6.02-2.el7          perl-Mozilla-CA.noarch 0:20130114-5.el7                
  perl-Net-HTTP.noarch 0:6.06-2.el7                perl-Net-LibIDN.x86_64 0:0.12-15.el7                   
  perl-Net-SSLeay.x86_64 0:1.55-6.el7              perl-Sys-Syscall.noarch 0:0.25-10.el7                  
  perl-TimeDate.noarch 1:2.30-2.el7                perl-URI.noarch 0:1.60-9.el7                           
  perl-WWW-RobotRules.noarch 0:6.02-5.el7          perl-libwww-perl.noarch 0:6.05-2.el7                   

Complete!
[root@node03 ~]#

　　複製node01上的mogstored配置文件到node03上，而後在node03上建立對應的目錄並修改屬主和屬組爲mogilefs，而後啓動mogstored

　　提示：在node03上沒有安裝mogilefs-server這個包，因此在對應系統上沒有mogilefs這個帳號，咱們須要手動建立這個帳號；

　　啓動mogstored

　　到此三個節點的環境配置，以及服務都已經正常啓動了；

　　4、使用mogadm管理mogilefs

　　把三個storage節點註冊到tracker上

　　提示：若是集羣各節點沒有作hosts主機名解析,那麼對應的的節點就須要寫清楚ip地址便可，若是端口不是7500，咱們還須要用--port去指定端口；

　　檢查集羣

　　提示：在tracker節點監聽在本機全部地址的7001，執行上述命令不須要加--trackers選項來指明tracker是誰，它默認會去找本機的7001；不然執行上述命令須要加上--trackers來指定tracker的地址和端口；

　　在非tracker節點執行檢查集羣的命令

　　建立devices

　　分別在3個mogstored節點的docroot說指定的目錄下建立以dev+數字的目錄，並將屬主和屬組修改爲mogilefs

[root@node03 ~]# mkdir /data/mogilefs/dev3
[root@node03 ~]# chown -R mogilefs.mogilefs /data/mogilefs/dev3
[root@node03 ~]# ll -d /data/mogilefs/dev3
drwxr-xr-x 3 mogilefs mogilefs 37 Sep 16 21:02 /data/mogilefs/dev3
[root@node03 ~]#

　　添加device

　　查看device

　　提示：若是在指定節點的docroot所指定的目錄下沒有建立dev+設備編號的目錄，上面是看不到對應設備的空間大小的；

　　建立domain

　　建立class

　　好了，到此咱們就能夠上傳文件了

　　上傳文件，把/etc/passwd上傳到files域中，使用html class

　　提示：mogupload這個工具是mogilefs的一個上傳文件的工具，其中--domain用於指定域，就是把文件放到那個名稱空間中；--class用於指定使用那個class策略來同步文件，默認不指定使用default；--key用於指定客戶端訪問文件的路徑；--file用於指定要上傳的文件路徑；

　　查看上傳文件的信息

　　提示：查看上傳文件的信息須要用mogfileinfo這個工具，咱們只須要指定trancker，domain以及文件的key就能夠看到，該文件的class，devcount,domain,fid,key,length,以及訪問http地址；這裏須要說明一下，咱們剛纔在建立files域中的html這個class 咱們指定了mindevcount=3，表示只要使用這個class上傳文件，文件至少有3個副本；而咱們如今看到的只有一個，這是由於在centos7上，Sys-syscall這個包的版本太高，致使mogilefs異常；

　　降級Sys-Syscall，解決centos7上安裝mogilefs 後端mogstored 同步問題

　　安裝perl編譯環境

[root@node01 ~]# yum -y install make gcc unzip perl-DBD-MySQL perl perl-CPAN perl-YAML perl-Time-HiRes
Loaded plugins: fastestmirror
base                                                                                           | 3.6 kB  00:00:00     
epel                                                                                           | 4.7 kB  00:00:00     
extras                                                                                         | 2.9 kB  00:00:00     
mariadb                                                                                        | 2.9 kB  00:00:00     
updates                                                                                        | 2.9 kB  00:00:00     
(1/2): epel/x86_64/updateinfo                                                                  | 1.0 MB  00:00:00     
(2/2): epel/x86_64/primary_db                                                                  | 6.9 MB  00:00:00     
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Package gcc-4.8.5-39.el7.x86_64 already installed and latest version
Package unzip-6.0-21.el7.x86_64 already installed and latest version
Package perl-DBD-MySQL-4.023-6.el7.x86_64 already installed and latest version
Package 4:perl-5.16.3-295.el7.x86_64 already installed and latest version
Package 4:perl-Time-HiRes-1.9725-3.el7.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package make.x86_64 1:3.82-23.el7 will be updated
---> Package make.x86_64 1:3.82-24.el7 will be an update
---> Package perl-CPAN.noarch 0:1.9800-295.el7 will be installed
--> Processing Dependency: perl(local::lib) for package: perl-CPAN-1.9800-295.el7.noarch
……省略部份內容……
Installed:
  perl-CPAN.noarch 0:1.9800-295.el7                           perl-YAML.noarch 0:0.84-5.el7                          

Dependency Installed:
  gdbm-devel.x86_64 0:1.10-8.el7                            libdb-devel.x86_64 0:5.3.21-25.el7                        
  perl-Digest-SHA.x86_64 1:5.85-4.el7                       perl-ExtUtils-Install.noarch 0:1.58-295.el7               
  perl-ExtUtils-MakeMaker.noarch 0:6.68-3.el7               perl-ExtUtils-Manifest.noarch 0:1.61-244.el7              
  perl-ExtUtils-ParseXS.noarch 1:3.18-3.el7                 perl-Test-Harness.noarch 0:3.28-3.el7                     
  perl-devel.x86_64 4:5.16.3-295.el7                        perl-local-lib.noarch 0:1.008010-4.el7                    
  pyparsing.noarch 0:1.5.6-9.el7                            systemtap-sdt-devel.x86_64 0:4.0-11.el7                   

Updated:
  make.x86_64 1:3.82-24.el7                                                                                           

Dependency Updated:
  libdb.x86_64 0:5.3.21-25.el7                           libdb-utils.x86_64 0:5.3.21-25.el7                          

Complete!
[root@node01 ~]#

　　提示：在mogstored節點上都要安裝好perl環境依賴包；而後按照下面的步驟從新編譯安裝Sys-Syscall；而後重啓mogstored便可生效；

　　下載低版本的Sys-Syscall源碼包，下載地址：https://cpan.metacpan.org/authors/id/B/BR/BRADFITZ/Sys-Syscall-0.23.tar.gz；並解壓，進入到源碼目錄

　　編譯安裝Sys-Syscall

　　驗證

　　重啓mogstored，mogilefsd

　　提示：在某一臺tracker節點從新編譯安裝Sys-Syscall，而後重啓mogilefsd和mogstored便可；我這裏是在全部節點都編譯安裝了，而後再把全部節點的mogstored和全部的mogilefsd都重啓了一遍，其實用不着；

　　上傳文件看看對應的副本數量是否仍是隻有一個？

　　提示：能夠看到對應的devcount和咱們以前指定的class上的mindevcount就同樣了；

　　驗證：訪問指定上述任一一個地址，看看訪問到的文件內容是不是咱們上傳到文件內容？

　　提示：能夠看到咱們上傳到文件內容在tracker給咱們返回的地址上可以訪問到，而且裏面的內容也是同樣的；

　　驗證：停掉node02上的mogstored，看看對應的tracker是否可以及時發現對應的storage不在線了呢？

　　提示：能夠看到當node02上的mogstored掛掉之後，在檢查集羣，tracker就能及時的發現對應的設備不在線了，這個有點相似nginx對後端web服務器作健康狀態監測的感受；也就說tracker在監視着註冊在tracker上的storage節點；

　　到此，mogilefs在centos7上的安裝部署，以及使用mogadm工具管理集羣就完成了，對於mogilefs一般咱們上傳文件是經過業務程序調用api實現，mogupload只是一個測試工具；對於查看文件，一般用戶也不會使用mogfieinfo返回的連接去查看文件，一般會使用一個web反向代理（nginx）去代理用戶的訪問；好比用戶要查看/images/test.jpg，而在tracker返回的對應資源的連接中，必定不是/images/test.jpg，這個時候咱們就須要用到web反向代理，讓其反向代理可以幫助咱們去tracker取到對應uri的資源連接，從而幫助咱們去storage節點獲取到對應的文件，而後再返回給用戶；最後仍是建議，若是用到mogilefs，最好跑在centos6上，centos7系統環境較高，有些依賴可能成問題；在centos6上，我測試過，徹底沒有上述的問題；咱們只須要安裝perl-IO-AIO這一個依賴包就能夠解決mogstored啓動問題；centos6上須要用到的rpm包能夠去個人網盤下載http://www.qiuhom.com/d/47e8f7ac56a445f1bfc6/；

相關標籤/搜索