CDH構建大數據平臺-使用自建的鏡像地址安裝Cloudera Manager

　　　　　　　　　CDH構建大數據平臺-使用自建的鏡像地址安裝Cloudera Manager

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　做者：尹正傑

版權聲明：原創做品，謝絕轉載！不然將追究法律責任。 

 

 

 一.搭建CM私有倉庫

　　詳情請參考個人筆記： https://www.cnblogs.com/yinzhengjie/p/10412455.html。

1>.安裝web服務器並啓動 

[root@node105 ~]# yum -y install httpd Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn base | 3.6 kB  00:00:00 extras | 3.4 kB  00:00:00 updates | 3.4 kB  00:00:00 (1/2): extras/7/x86_64/primary_db                                                                                                                | 180 kB  00:00:04 (2/2): updates/7/x86_64/primary_db                                                                                                               | 2.4 MB  00:01:01 Resolving Dependencies --> Running transaction check ---> Package httpd.x86_64 0:2.4.6-88.el7.centos will be installed --> Processing Dependency: httpd-tools = 2.4.6-88.el7.centos for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-88.el7.centos.x86_64 --> Running transaction check ---> Package apr.x86_64 0:1.4.8-3.el7_4.1 will be installed ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed ---> Package httpd-tools.x86_64 0:2.4.6-88.el7.centos will be installed ---> Package mailcap.noarch 0:2.1.41-2.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================== Package Arch Version Repository Size ======================================================================================================================================================================== Installing: httpd x86_64 2.4.6-88.el7.centos                               base                              2.7 M Installing for dependencies: apr x86_64 1.4.8-3.el7_4.1                                   base                              103 k apr-util                                 x86_64                              1.5.2-6.el7                                       base                               92 k httpd-tools                              x86_64                              2.4.6-88.el7.centos                               base                               90 k mailcap noarch 2.1.41-2.el7                                      base                               31 k Transaction Summary ======================================================================================================================================================================== Install 1 Package (+4 Dependent packages) Total download size: 3.0 M Installed size: 10 M Downloading packages: (1/5): mailcap-2.1.41-2.el7.noarch.rpm                                                                                                           |  31 kB  00:00:00 (2/5): apr-1.4.8-3.el7_4.1.x86_64.rpm                                                                                                            | 103 kB  00:00:00 (3/5): apr-util-1.5.2-6.el7.x86_64.rpm                                                                                                           |  92 kB  00:00:00 (4/5): httpd-tools-2.4.6-88.el7.centos.x86_64.rpm                                                                                                |  90 kB  00:00:00 (5/5): httpd-2.4.6-88.el7.centos.x86_64.rpm                                                                                                      | 2.7 MB  00:00:21     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 143 kB/s | 3.0 MB  00:00:21 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : apr-1.4.8-3.el7_4.1.x86_64                                                                                                                           1/5 Installing : apr-util-1.5.2-6.el7.x86_64                                                                                                                          2/5 Installing : httpd-tools-2.4.6-88.el7.centos.x86_64                                                                                                               3/5 Installing : mailcap-2.1.41-2.el7.noarch                                                                                                                          4/5 Installing : httpd-2.4.6-88.el7.centos.x86_64                                                                                                                     5/5 Verifying : httpd-tools-2.4.6-88.el7.centos.x86_64                                                                                                               1/5 Verifying : apr-1.4.8-3.el7_4.1.x86_64                                                                                                                           2/5 Verifying : mailcap-2.1.41-2.el7.noarch                                                                                                                          3/5 Verifying : httpd-2.4.6-88.el7.centos.x86_64                                                                                                                     4/5 Verifying : apr-util-1.5.2-6.el7.x86_64                                                                                                                          5/5 Installed: httpd.x86_64 0:2.4.6-88.el7.centos Dependency Installed: apr.x86_64 0:1.4.8-3.el7_4.1          apr-util.x86_64 0:1.5.2-6.el7          httpd-tools.x86_64 0:2.4.6-88.el7.centos          mailcap.noarch 0:2.1.41-2.el7 Complete! [root@node105 ~]#

[root@node105 ~]# yum -y install httpd　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　#安裝httpd服務

[root@node105 ~]# cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.`date +%F` [root@node105 ~]# [root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #' AddType application/x-gzip .gz .tgz [root@node105 ~]# [root@node105 ~]# sed -i s'#.tgz#.tgz .parcel#' /etc/httpd/conf/httpd.conf [root@node105 ~]# [root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #' AddType application/x-gzip .gz .tgz .parcel　　　　　　　　　　　　　　　　　　　　　　　　#其實我就是在.tgz後面加了一個「.parcel」的後綴，這是CM官方強調要作的一個操做！ [root@node105 ~]# [root@node105 ~]#

[root@node105 ~]# systemctl enable httpd Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service. [root@node105 ~]# [root@node105 ~]# systemctl start httpd [root@node105 ~]# [root@node105 ~]# [root@node105 ~]# systemctl status httpd ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2019-02-22 11:02:22 CST; 3s ago Docs: man:httpd(8) man:apachectl(8) Main PID: 4674 (httpd) Status: "Processing requests..." CGroup: /system.slice/httpd.service ├─4674 /usr/sbin/httpd -DFOREGROUND ├─4675 /usr/sbin/httpd -DFOREGROUND ├─4676 /usr/sbin/httpd -DFOREGROUND ├─4677 /usr/sbin/httpd -DFOREGROUND ├─4678 /usr/sbin/httpd -DFOREGROUND └─4679 /usr/sbin/httpd -DFOREGROUND Feb 22 11:02:22 node105.yinzhengjie.org.cn systemd[1]: Starting The Apache HTTP Server... Feb 22 11:02:22 node105.yinzhengjie.org.cn systemd[1]: Started The Apache HTTP Server. [root@node105 ~]# [root@node105 ~]#

[root@node105 ~]# systemctl start httpd　　　　　　　　　　　　　　#啓動httpd服務

2>.下載CM對應到版本到對應到自建到web路徑中（https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.16.1/RPMS/x86_64/）

[root@node105 ~]# mkdir /var/www/html/cm-5.16.1　　　　　　　　　　　　　　　　　　 　　　　　　#建立CM安裝包存放目錄 [root@node105 ~]# [root@node105 ~]# ll /var/www/html/cm-5.16.1/　　　　　　　　　　　　　　　　　　　　　　　　　　#去官方網站把你須要的rpm包都下載到你到安裝目錄便可！ total 953108
-rwxr-xr-x 1 root root   9864584 Feb 21 18:22 cloudera-manager-agent-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 789872988 Feb 21 12:21 cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root      8704 Feb 21 12:21 cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root     10612 Feb 21 12:21 cloudera-manager-server-db-2-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 176209195 Feb 21 18:22 jdk-8u201-linux-x64.rpm　　　　　　　　　　　　#這個軟件是我從Oracle官方上download下來的，CM默認的JDK是6或者7，而Oracle官方已經不支持使用JDK7啦！
-rw-r--r-- 1 root root      1690 Feb 21  2018 RPM-GPG-KEY-cloudera [root@node105 ~]# [root@node105 ~]#

3>.製做本地鏡像

[root@node105 ~]# yum -y install yum-utils createrepo yum-plugin-priorities Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package createrepo.noarch 0:0.9.9-28.el7 will be installed --> Processing Dependency: python-deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: libxml2-python for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: deltarpm for package: createrepo-0.9.9-28.el7.noarch ---> Package yum-plugin-priorities.noarch 0:1.1.31-50.el7 will be installed ---> Package yum-utils.noarch 0:1.1.31-50.el7 will be installed --> Processing Dependency: python-kitchen for package: yum-utils-1.1.31-50.el7.noarch --> Running transaction check ---> Package deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package libxml2-python.x86_64 0:2.9.1-6.el7_2.3 will be installed ---> Package python-deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package python-kitchen.noarch 0:1.1.1-5.el7 will be installed --> Processing Dependency: python-chardet for package: python-kitchen-1.1.1-5.el7.noarch --> Running transaction check ---> Package python-chardet.noarch 0:2.2.1-1.el7_1 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================== Package Arch Version Repository Size ======================================================================================================================================================================== Installing: createrepo noarch 0.9.9-28.el7                                  base                             94 k yum-plugin-priorities                            noarch                            1.1.31-50.el7                                 base                             29 k yum-utils                                        noarch                            1.1.31-50.el7                                 base                            121 k Installing for dependencies: deltarpm x86_64 3.6-3.el7                                     base                             82 k libxml2-python                                   x86_64                            2.9.1-6.el7_2.3                               base                            247 k python-chardet                                   noarch                            2.2.1-1.el7_1                                 base                            227 k python-deltarpm                                  x86_64                            3.6-3.el7                                     base                             31 k python-kitchen                                   noarch                            1.1.1-5.el7                                   base                            267 k Transaction Summary ======================================================================================================================================================================== Install 3 Packages (+5 Dependent packages) Total download size: 1.1 M Installed size: 4.8 M Downloading packages: (1/8): libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm                                                                                                 | 247 kB  00:00:00 (2/8): createrepo-0.9.9-28.el7.noarch.rpm                                                                                                        |  94 kB  00:00:00 (3/8): python-deltarpm-3.6-3.el7.x86_64.rpm                                                                                                      |  31 kB  00:00:00 (4/8): deltarpm-3.6-3.el7.x86_64.rpm                                                                                                             |  82 kB  00:00:00 (5/8): yum-utils-1.1.31-50.el7.noarch.rpm                                                                                                        | 121 kB  00:00:00 (6/8): python-chardet-2.2.1-1.el7_1.noarch.rpm                                                                                                   | 227 kB  00:00:00 (7/8): yum-plugin-priorities-1.1.31-50.el7.noarch.rpm                                                                                            |  29 kB  00:00:00 (8/8): python-kitchen-1.1.1-5.el7.noarch.rpm                                                                                                     | 267 kB  00:00:03     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 332 kB/s | 1.1 MB  00:00:03 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : deltarpm-3.6-3.el7.x86_64                                                                                                                            1/8 Installing : libxml2-python-2.9.1-6.el7_2.3.x86_64                                                                                                                2/8 Installing : python-deltarpm-3.6-3.el7.x86_64                                                                                                                     3/8 Installing : python-chardet-2.2.1-1.el7_1.noarch                                                                                                                  4/8 Installing : python-kitchen-1.1.1-5.el7.noarch                                                                                                                    5/8 Installing : yum-utils-1.1.31-50.el7.noarch                                                                                                                       6/8 Installing : createrepo-0.9.9-28.el7.noarch                                                                                                                       7/8 Installing : yum-plugin-priorities-1.1.31-50.el7.noarch                                                                                                           8/8 Verifying : yum-utils-1.1.31-50.el7.noarch                                                                                                                       1/8 Verifying : python-deltarpm-3.6-3.el7.x86_64                                                                                                                     2/8 Verifying : yum-plugin-priorities-1.1.31-50.el7.noarch                                                                                                           3/8 Verifying : libxml2-python-2.9.1-6.el7_2.3.x86_64                                                                                                                4/8 Verifying : deltarpm-3.6-3.el7.x86_64                                                                                                                            5/8 Verifying : createrepo-0.9.9-28.el7.noarch                                                                                                                       6/8 Verifying : python-chardet-2.2.1-1.el7_1.noarch                                                                                                                  7/8 Verifying : python-kitchen-1.1.1-5.el7.noarch                                                                                                                    8/8 Installed: createrepo.noarch 0:0.9.9-28.el7                   yum-plugin-priorities.noarch 0:1.1.31-50.el7                   yum-utils.noarch 0:1.1.31-50.el7 Dependency Installed: deltarpm.x86_64 0:3.6-3.el7             libxml2-python.x86_64 0:2.9.1-6.el7_2.3     python-chardet.noarch 0:2.2.1-1.el7_1     python-deltarpm.x86_64 0:3.6-3.el7 python-kitchen.noarch 0:1.1.1-5.el7 Complete! [root@node105 ~]#

[root@node105 ~]# yum -y install yum-utils createrepo yum-plugin-priorities　　　　　　　　#安裝本地源製做相關工具

[root@node105 ~]# cd /var/www/html/cm-5.16.1/ [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]# ll total 953108
-rwxr-xr-x 1 root root   9864584 Feb 21 18:22 cloudera-manager-agent-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 789872988 Feb 21 12:21 cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root      8704 Feb 21 12:21 cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root     10612 Feb 21 12:21 cloudera-manager-server-db-2-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 176209195 Feb 21 18:22 jdk-8u201-linux-x64.rpm -rw-r--r-- 1 root root      1690 Feb 21  2018 RPM-GPG-KEY-cloudera [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]# createrepo .　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　#建立yum源的元數據文件！ Spawning worker 0 with 3 pkgs Spawning worker 1 with 2 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]# ll total 953112
-rwxr-xr-x 1 root root 　 9864584 Feb 21 18:22 cloudera-manager-agent-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 789872988 Feb 21 12:21 cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root      8704 Feb 21 12:21 cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root     10612 Feb 21 12:21 cloudera-manager-server-db-2-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 176209195 Feb 21 18:22 jdk-8u201-linux-x64.rpm drwxr-xr-x 2 root root      4096 Feb 22 11:12 repodata　　　　　　　　　　　　　　　　　　#這個目錄就是咱們執行上面的命令後生成的目錄,該文件不可被刪除！不然你配置的yum的使用者依舊沒法使用yum命令進行安裝 -rw-r--r-- 1 root root      1690 Feb 21  2018 RPM-GPG-KEY-cloudera [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]#

4>.下載CDH的版本存放到指定路徑（https://archive.cloudera.com/cdh5/parcels/5.15.1/）

[root@node105 ~]# mkdir /var/www/html/cdh-5.15.1 [root@node105 ~]# [root@node105 ~]# cd /var/www/html/cdh-5.15.1/ [root@node105 cdh-5.15.1]# [root@node105 cdh-5.15.1]# ll total 2070488
-rw-r--r-- 1 root root 2120090032 Feb 22 11:19 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root         41 Feb 22 11:19 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha -rw-r--r-- 1 root root      73767 Feb 22 11:19 manifest.json [root@node105 cdh-5.15.1]# 

 5>.各個主機配置CM自建的yum倉庫

[root@node101 ~]# cat /etc/yum.repos.d/cloudera-manager.repo [cloudera-manager] # Packages for Cloudera Manager, Version 5, on RedHat or CentOS 7 x86_64 name=Cloudera Manager baseurl=http://node105.yinzhengjie.org.cn/cm-5.16.1/
gpgkey =http://node105.yinzhengjie.org.cn/cm-5.16.1//RPM-GPG-KEY-cloudera 
enabled = 1 gpgcheck = 0 [root@node101 ~]#

 

二.安裝和配置元數據庫

 　　詳情請參考個人筆記：安裝和配置元數據庫。

 

三.Cloudera Manager 部署

　　安裝服務前請確保你磁盤的可用空間大於50G！

1>.安裝Cloudera Manager Server端（因爲咱們使用自建的鏡像地址安裝，並且也配置好了yum源，咱們這裏直接安裝便可！）

[root@node101 ~]# yum -y install cloudera-manager-daemons cloudera-manager-server Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.163.com * extras: mirror01.idc.hinet.net * updates: centos.cs.nctu.edu.tw Resolving Dependencies --> Running transaction check ---> Package cloudera-manager-daemons.x86_64 0:5.16.1-1.cm5161.p0.1.el7 will be installed ---> Package cloudera-manager-server.x86_64 0:5.16.1-1.cm5161.p0.1.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================== Package Arch Version Repository Size ======================================================================================================================================================================== Installing: cloudera-manager-daemons                      x86_64                      5.16.1-1.cm5161.p0.1.el7                         cloudera-manager                      753 M cloudera-manager-server                       x86_64                      5.16.1-1.cm5161.p0.1.el7                         cloudera-manager                      8.5 k Transaction Summary ======================================================================================================================================================================== Install 2 Packages Total download size: 753 M Installed size: 935 M Downloading packages: (1/2): cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm                                                                               | 8.5 kB  00:00:00 (2/2): cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm                                                                              | 753 MB  00:00:14     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 51 MB/s | 753 MB  00:00:14 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64                                                                                             1/2 Installing : cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64                                                                                              2/2 Verifying : cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64                                                                                              1/2 Verifying : cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64                                                                                             2/2 Installed: cloudera-manager-daemons.x86_64 0:5.16.1-1.cm5161.p0.1.el7                          cloudera-manager-server.x86_64 0:5.16.1-1.cm5161.p0.1.el7 Complete! [root@node101 ~]#

[root@node101 ~]# yum -y install cloudera-manager-daemons cloudera-manager-server

2>.配置Cloudera Manager Server的默認堆內存大小 

[root@node101 ~]# grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server 　　　　　　　　　　　　　#Cloudera Manager默認分配的JVM堆內存是2G，某些狀況下會出現OOM的狀況，咱們能夠適當將其調大。 export CMF_JAVA_OPTS="-Xmx2G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp" [root@node101 ~]# [root@node101 ~]# sed -i s'#Xmx2G#Xmx4G#' /etc/default/cloudera-scm-server 　　　　　　　　　　#咱們將Cloudera Manager的堆內存調大到4G。 [root@node101 ~]# [root@node101 ~]# grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server export CMF_JAVA_OPTS="-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp" [root@node101 ~]# [root@node101 ~]#

3>.初始化數據庫配置 

　　Cloudera官網已經提供了一個sh腳本用戶配置Cloudera Manager鏈接元數據庫，官方文檔參考：

　　　　https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_ig_installing_configuring_dbs.html#concept_i2r_m3m_hn。

　　咱們的安裝規劃下，元數據庫和Cloudera Manager運行於同一個節點，且已經預先建立了scm庫和用戶，所以，命令很簡單，具體操做以下： 

[root@node101 ~]# /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm yinzhengjie JAVA_HOME=/usr/java/jdk1.8 Verifying that we can write to /etc/cloudera-scm-server Creating SCM configuration file in /etc/cloudera-scm-server Executing: /usr/java/jdk1.8/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. [ main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly!　　　　　　　　　　　　　　　　　　　　　　#注意，若是你看到了這一行提示，說明配置完成，不然須要根據提示解決相應的錯誤！ [root@node101 ~]# 

4>.啓動Cloudera Manager Server服務

[root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: inactive (dead) Docs: man:systemd-sysv-generator(8) Feb 21 12:29:08 node101.yinzhengjie.org.cn systemd[1]: [/run/systemd/generator.late/cloudera-scm-server.service:13] Failed to add dependency on +cloudera-scm-server-db.service, ignoring: Invalid argument Feb 21 12:29:08 node101.yinzhengjie.org.cn systemd[1]: [/run/systemd/generator.late/cloudera-scm-server.service:13] Failed to add dependency on +cloudera-scm-server-db.service, ignoring: Invalid argument [root@node101 ~]# [root@node101 ~]# systemctl start cloudera-scm-server 　　　　　　　　　　　　　　　　　　　　　　#啓動Cloudera Manager，所須要的時間可能在5～10秒左右！ [root@node101 ~]# [root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: active (exited) since Thu 2019-02-21 12:53:21 CST; 2min 38s ago Docs: man:systemd-sysv-generator(8) Process: 7253 ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=0/SUCCESS) Feb 21 12:53:16 node101.yinzhengjie.org.cn systemd[1]: Starting LSB: Cloudera SCM Server... Feb 21 12:53:16 node101.yinzhengjie.org.cn su[7277]: PAM unable to dlopen(/usr/lib64/security/pam_fprintd.so): /usr/lib64/security/pam_fprintd.so: cannot open shared object file: No such file or directory Feb 21 12:53:16 node101.yinzhengjie.org.cn su[7277]: PAM adding faulty module: /usr/lib64/security/pam_fprintd.so Feb 21 12:53:16 node101.yinzhengjie.org.cn su[7277]: (to cloudera-scm) root on none Feb 21 12:53:21 node101.yinzhengjie.org.cn cloudera-scm-server[7253]: Starting cloudera-scm-server: [ OK ] Feb 21 12:53:21 node101.yinzhengjie.org.cn systemd[1]: Started LSB: Cloudera SCM Server. [root@node101 ~]#

[root@node101 ~]# systemctl start cloudera-scm-server 　　　　　　　　　　　　　　　　　　　　　　#啓動Cloudera Manager，所須要的時間可能在5～10秒左右！ 

5>.查看Cloudera Manager Server的日誌並訪問WebUI

[root@node101 ~]# tail -10f /var/log/cloudera-scm-server/cloudera-scm-server.log 　　　　　　　　　　　　　　　　　　　　#查看Cloudera Manager的服務的日誌 2019-02-21 12:54:38,541 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Initializing SearchTemplateManager:2019-02-21T04:54:38.540Z 2019-02-21 12:54:38,673 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating entities:2019-02-21T04:54:38.673Z 2019-02-21 12:54:38,721 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num entities:208
2019-02-21 12:54:38,721 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating documents:2019-02-21T04:54:38.721Z 2019-02-21 12:54:38,795 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num docs:221
2019-02-21 12:54:38,823 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Constructing repo:2019-02-21T04:54:38.823Z 2019-02-21 12:54:39,541 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4
2019-02-21 12:54:39,631 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180　　　　　　　　#注意，若是出現這一行提示，說明你的服務啓動成功啦！默認端口爲7180，下面有WebServerImpl：Started Jetty server提示！ 2019-02-21 12:54:39,631 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server. 2019-02-21 12:54:39,827 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2019-02-21T04:54:39.827Z

[root@node101 ~]# tail -10f /var/log/cloudera-scm-server/cloudera-scm-server.log 　　　　　　　　　　　　　　　　　　　　#查看Cloudera Manager的服務的日誌

　　打開瀏覽器，訪問：http://node101.yinzhengjie.org.cn:7180/cmf/login，以下圖所示：

 

四.安裝CDH5.15.1

1>.登錄Cloudera Manager Server 的WebUI

2>.接受協議，點擊繼續

3>.選擇企業版（該版本的CDH是免費的） 

4>.選擇CDH 

5>.爲 CDH 羣集安裝指定主機。

6>.點擊更多選項

 

7>.選擇自建的CDH Parcels包鏡像地址

8>.配置Cloudera Manager Agent的地址

9>.不要選中安裝oracle Java，由於咱們已經安裝過JDK8，選中的話會裝會JDK7，直接點擊繼續便可！ 

10>.不啓用單用戶模式安裝，直接點擊繼續便可 

11>.提供SSH登錄憑據 

12>.等待Cloudera Manager Agent 安裝完成 

13>.查看安裝過程的日誌詳情 

14>.安裝 Cloudera Manager Agent成功（根據集羣性而定，我使用的是MAC，該過程大概5～10分鐘左右！） 

15>.下載CDH（從咱們自建的鏡像地址下載）

16>.CDH安裝完成，點擊繼續

 

17>.等待主機檢查

18>.點擊完成

19>.完成上述步驟就已經完成了CDH的安裝，還有一種方法安裝CDH，就是下載CDH並存放到指定目錄（瞭解便可！）

　　第一步：下載CDH 5.15.1的parcels包（https://archive.cloudera.com/cdh5/parcels/5.15.1/）

　　第二步：將下載的rpm包存放到CM默認到目錄中（/opt/cloudera/parcel-repo）

[root@node101 ~]# ll /opt/cloudera/parcel-repo/ total 2070488
-rw-r--r-- 1 root root 2120090032 Feb 21 17:12 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root         41 Feb 21 17:12 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 -rw-r--r-- 1 root root      73767 Feb 21 17:12 manifest.json [root@node101 ~]# [root@node101 ~]# mv /opt/cloudera/parcel-repo/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha 　　　　#咱們這裏須要作一個重命名的操做！ [root@node101 ~]# [root@node101 ~]# ll /opt/cloudera/parcel-repo/ total 2070488
-rw-r--r-- 1 root root 2120090032 Feb 21 17:12 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root         41 Feb 21 17:12 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha -rw-r--r-- 1 root root      73767 Feb 21 17:12 manifest.json [root@node101 ~]#

 

五.安裝Hadoop初始化的服務

1>.選擇自定義服務

2>.選擇須要安裝的服務，點擊繼續（爲了減小虛擬機的開銷和安裝時間，我這裏只演示安裝zookeeper集羣和hdfs集羣便可！其餘服務的安裝方法相似） 

3>.自定義角色分配

4>.審覈更改（這裏我們能夠修改一些默認的配置，好比hdfs的塊大小，數據存儲的路徑等等，實驗環境咱們默認便可，生產環境須要根據實際狀況稍做修改便可！）

5>.安裝初始化服務（安裝過程時間可能須要3～5分鐘）

6>.服務安裝完成 

7>.查看Cloudera Manager的監控界面

8>.安裝服務完成後，咱們能夠查看到剩餘的可用空間