大數據之---CDH集羣版本部署全網終極篇---更新中

一、軟件環境和IP規劃

RHEL6 角色

jdk-8u45apache-maven-3.3.9

hive-1.1.0-cdh5.7.1-src.tar.gz 

hadoop-2.8.1.tar.gz

mysql-connector-java-6.0.6.tar.gz

apache-maven-3.3.9

cloudera-manager-el6-cm5.9.3_x86_64.tar

mysql-5.7

CDH-5.9.3-1.cdh5.9.3.p0.4-el6

172.16.18.133 NN && SN && Jobtrack hadoop01

172.16.18.134 DN && tasktrack hadoop02

172.16.18.136 DN && tasktrack hadoop03

172.16.18.143 DN && tasktrack hadoop04

172.16.18.145 DN && tasktrack hadoop05

NN =namenode    SN=secondarynamenode  DN=datanode

 

集羣介紹：

不收費的Hadoop版本主要有三個（均是國外廠商），分別是：Apache（最原始的版本，全部發行版均基於這個版本進行改進）、Cloudera版本（Cloudera’s Distribution Including Apache Hadoop，簡稱CDH）、Hortonworks版本(Hortonworks Data Platform，簡稱「HDP」），對於國內而言，絕大多數選擇CDH版本。

CDH (Cloudera's Distribution, including Apache Hadoop)，是Hadoop衆多分支中的一種，由Cloudera維護，基於穩定版本的Apache Hadoop構建，並集成了不少補丁，可直接用於生產環境。 

Cloudera Manager則是爲了便於在集羣中進行Hadoop等大數據處理相關的服務安裝和監控管理的組件，對集羣中主機、Hadoop、Hive、Spark等服務的安裝配置管理作了極大簡化。 

 

 

集羣安裝，本文采用選擇  離線安裝CDH

https://www.cloudera.com/downloads/cdh/5-9-0.html 

官網對CDH的描述，CHD對system JDK database 等版本支持列表

 

官網看支持jdk1.8可是部分1.8版本會報錯，因此咱們選擇jdk1.7

二、軟件包裝備

Cloudera Manager軟件包

http://archive.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.9.3_x86_64.tar.gz

CDH軟件包  （下載對應Linux版本包）

http://archive.cloudera.com/cdh5/parcels/5.9.3/CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel

http://archive.cloudera.com/cdh5/parcels/5.9.3/CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel.sha1

mysql jdbc驅動版本是：

http://download.softagency.net/MySQL/Downloads/Connector-J/mysql-connector-java-6.0.6.tar.gz

 

 

三、系統相關配置

全部主機相同   安裝JDK 關閉selinux  iptables  配置/etc/hosts 配置yum

 [root@hadoop01 ~]# vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.7.0_79

export PATH=$JAVA_HOME/bin:$ORACLE_HOME/bin:$R_HOME/bin:$PATH

[root@hadoop01 ~]# getenforce 

Disabled

[root@hadoop01 ~]# iptables -L

Chain INPUT (policy ACCEPT)

target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)

target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)

target     prot opt source               destination         

[root@hadoop01 ~]# cat /etc/hosts

127.0.0.1   localhost

172.16.18.133   hadoop01  

172.16.18.134    hadoop02

172.16.18.136    hadoop03

172.16.18.143    hadoop04

172.16.18.145    hadoop05

[root@hadoop01 ~]# hostname

hadoop01

四、配置ssh自動登陸互信

參考僞分佈式ssh互信配置

每一個節點驗證不須要進行交互輸入yes

useradd hadoop   ----創建用戶

ssh hadoop01 date

ssh hadoop02 date

ssh hadoop03 date

ssh hadoop04 date

ssh hadoop05 date

 

五、修改swap空間的swappiness=0

cat  /proc/sys/vm/swappiness 

sysctl vm.swappiness=0

echo 0 > /proc/sys/vm/swappiness

關閉告警：echo never > /sys/kernel/mm/transparent_hugepage/defrag

 

六、配置NTP服務器

先選定主服務器,其餘服務器都同步這臺主服務器的時間

# hwclock -w

配置開機啓動

[root@hadoop01 ~]# chkconfig ntpd on

[root@hadoop01 ~]# chkconfig --list ntpd

[root@hadoop01 ~]#vi /etc/ntp.conf

    (找到這一行,放開restrict的註釋,而且修改ip地址)

    # Hosts on local network are less restricted.

    restrict 192.168.128.1 mask 255.255.255.0 nomodify notrap

    (找到這一行,註釋下面的server)

    # Please consider joining the pool (http://www.pool.ntp.org/join.html).

    #server 0.rhel.pool.ntp.org iburst

    #server 1.rhel.pool.ntp.org iburst

    #server 2.rhel.pool.ntp.org iburst

    #server 3.rhel.pool.ntp.org iburst

    添加下面兩行

    server  127.0.1.0     # local clock

    fudge   127.0.1.0 stratum 10

配置其餘服務器

 vi /etc/ntp.conf

    # Hosts on local network are less restricted.

    restrict 192.168.128.1 nomodify notrap noquery

    (註釋下面server)

    # Please consider joining the pool (http://www.pool.ntp.org/join.html).

    #server 0.rhel.pool.ntp.org iburst

    #server 1.rhel.pool.ntp.org iburst

    #server 2.rhel.pool.ntp.org iburst

    #server 3.rhel.pool.ntp.org iburst

    指定時間服務

    server 192.168.128.51

全部重啓ntp服務

[root@hadoop01 ~]# service ntpd restart

[root@hadoop01 ~]# ntpstat  

synchronised to NTP server (172.16.18.33) at stratum 12 

time correct to within 18 ms

polling server every 64 s

[root@hadoop01 ~]# date

 

七、禁用ipv6和「透明大頁面」

[root@hadoop01 ~]# echo "alias ipv6 off" >> /etc/modprobe.d/dist.conf  

[root@hadoop01 ~]# echo "alias net-pf-10 off" >> /etc/modprobe.d/dist.conf

[root@hadoop01 ~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag

[root@hadoop01 ~]# echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local

[root@hadoop01 ~]# 

 

八、準備好mysql數據庫

修改 mysql 權限： 

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;

flush privileges;

delete from user where host !='%';

[root@hadoop01 software]# mysql -h 172.16.18.133   -uroot -p

 

 

###############################準備工做########################################

hadoop01 Server || Agent

hadoop02 Agent

hadoop03 Agent

CDH採用3臺服務器，剩下2臺作集羣添加節點使用

########################################################################

 

 

 

10.CM安裝

安裝cloudera Manager Server、Agent

cdh集羣節點都要安裝 軟件準備 帳號創建

主節點：

[root@hadoop01 software]# ls cloudera-manager-el6-cm5.9.3_x86_64.tar.gz 

cloudera-manager-el6-cm5.9.3_x86_64.tar.gz

[root@hadoop01 software]# pwd

/opt/software

[root@hadoop01 software]# ls cloudera-manager-el6-cm5.9.3_x86_64.tar.gz 

cloudera-manager-el6-cm5.9.3_x86_64.tar.gz

[root@hadoop01 software]# mkdir /opt/cloudera-manager

[root@hadoop01 software]# tar zxvf cloudera-manager-el6-cm5.9.3_x86_64.tar.gz -C /opt/cloudera-manager/

客戶端配置

/opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-agent/config.ini

server_host=hadoop01   ---在cm server主機名

[root@hadoop01 software]# useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

[root@hadoop01 software]# id cloudera-scm

uid=495(cloudera-scm) gid=492(cloudera-scm) 組=492(cloudera-scm)

 

haoop02 hadoop03全部從節點

useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

mkdir /opt/cloudera-manager

[root@hadoop01 opt]# scp -r /opt/cloudera-manager/cm-5.9.3  hadoop02:/opt/cloudera-manager/

[root@hadoop01 opt]# scp -r /opt/cloudera-manager/cm-5.9.3  hadoop03:/opt/cloudera-manager/

 

十一、配置CM Server數據庫

咱們開始準備mysql數據庫創建

[root@hadoop01 ~]# mysql -h172.16.18.133 -uroot -p

mysql> 

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;

flush privileges;

mysql> flush privileges;

[root@hadoop01 schema]# pwd

/opt/cloudera-manager/cm-5.9.3/share/cmf/schema

[root@hadoop01 schema]# ./scm_prepare_database.sh  mysql -hhadoop01 -uroot -p123 --scm-host hadoop01 cmdb root 123

JAVA_HOME=/usr/java/jdk1.7.0_79

Verifying that we can write to /opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-server

Sat Apr 28 14:20:38 CST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

Creating SCM configuration file in /opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-server

Executing:  /usr/java/jdk1.7.0_79/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cloudera-manager/cm-5.9.3/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.

Sat Apr 28 14:20:39 CST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

[                          main] DbCommandExecutor              INFO  Successfully connected to database.

All done, your SCM database is configured correctly!

說明：這個腳本就是用來建立和配置CMS須要的數據庫的腳本。各參數是指：

mysql：數據庫用的是mysql，若是安裝過程當中用的oracle，那麼該參數就應該改成oracle。

-hadoop01：數據庫創建在hadoop01主機上面,也就是主節點上面。

-uroot：root身份運行mysql。-123：mysql的root密碼是

--scm-host hadoop01 ：CMS的主機，通常是和mysql安裝的主機是在同一個主機上,

最後三個參數是：數據庫名，數據庫用戶名，數據庫密碼。

 

十二、製做CDH本地源

Server節點

mkdir -p /opt/cloudera/parcel-repo

chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo

Agent節點

mkdir -p /opt/cloudera/parcels

chown cloudera-scm:cloudera-scm /opt/cloudera/parcels

 

上傳到CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel   manifest.json主節點/opt/cloudera/parcel-repo/路徑

[root@hadoop01 CDH]# cd /opt/cloudera/parcel-repo/

[root@hadoop01 parcel-repo]# ls

CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel   manifest.json

[root@hadoop01 parcel-repo]# ls

CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel  manifest.json

[root@hadoop01 parcel-repo]# mv manifest.json CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel.sha

manifest.json更名文件名與你的 parel包名一致，並加上.sha後綴

 

 

1三、啓動

保障mysql先啓動

server：hadoop01

[root@hadoop01 init.d]# pwd

/opt/cloudera-manager/cm-5.9.3/etc/init.d

[root@hadoop01 init.d]# ./cloudera-scm-server start

Starting cloudera-scm-server:

agent:hadoop01 hadoop02 hadoop02

/opt/cloudera-manager/cm-5.9.3/etc/init.d

./cloudera-scm-agent  start

正在啓動 cloudera-scm-agent：                              [肯定]

 

2018-04-28 14:43:37,022 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4

2018-04-28 14:43:37,024 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180

2018-04-28 14:43:37,024 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

出現下面內容表示啓動成功

 

1四、圖形訪問

 

 

錯誤大全：

問題1：JDBC driver驅動

[root@hadoop01 schema]# ./scm_prepare_database.sh mysql cmdb -h hadoop01 -uroot -p123456 --scm-host hadoop01 scm scm scm

JAVA_HOME=/usr/java/jdk1.7.0_79

Verifying that we can write to /opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-server

[                          main] DbProvisioner                  ERROR Unable to find the MySQL JDBC driver. Please make sure that you have installed it as per instruction in the installation guide.

[                          main] DbProvisioner                  ERROR Stack Trace:

java.lang.ClassNotFoundException: com.mysql.jdbc.Driver

    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)[:1.7.0_79]

    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)[:1.7.0_79]

    at java.security.AccessController.doPrivileged(Native Method)[:1.7.0_79]

    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)[:1.7.0_79]

解決方法

[root@hadoop01 software]# ls mysql-connector-java-5.1.46.zip 

mysql-connector-java-5.1.46.zip

[root@hadoop01 software]# unzip  mysql-connector-java-5.1.46.zip ^C

[root@hadoop01 software]# cp mysql-connector-java-5.1.46/

build.xml                            mysql-connector-java-5.1.46-bin.jar  README.txt

CHANGES                              mysql-connector-java-5.1.46.jar      src/

COPYING                              README                               

[root@hadoop01 software]# cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46.jar  /usr/share/java/

[root@hadoop01 software]# mv /usr/share/java/mysql-connector-java-5.1.46.jar /usr/share/java/mysql-connector-java.jar

 

問題2：

dbc url 'jdbc:mysql://hadoop01/?useUnicode=true&characterEncoding=UTF-8'

java.sql.SQLException: Access denied for user 'root'@'hadoop01' (using password: YES)

    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)[mysql-connector-java.jar:5.1.46]