CentOS 7.3 CDH 5.10.0 Druid0.12.4安裝記錄

CentOS 7.3 CDH 5.10.0安裝記錄

0. 集羣規劃
192.167.1.247 realtime247 realtime+hadoopdata
192.167.1.248 broker248 broker
192.167.1.249 history249 middleware+history+hadoopdata
192.167.1.250 history250 middleware+history+hadoopdata
192.167.1.251 master251 Mysql+coordinator+overlord+hadoopMaster

1. 環境準備

安裝第三方依賴包：（全部節點，注意必定要安裝)
yum install -y chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb

echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/defrag

修改swappiness參數:
vi /etc/sysctl.conf 文件裏添加以下參數：
vm.swappiness=0
或者：
echo 'vm.swappiness=0'>> /etc/sysctl.conf

JDK 1.8 RPM包:
http://download.oracle.com/otn-pub/java/jdk/8u172-b11/a58eab1ec242421181065cdc37240b08/jdk-8u172-linux-x64.rpm?AuthParam=1527227488_e083c615628d8b91527508f00008dd6d

下載cloudera-manager，parcel：
https://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz

http://archive.cloudera.com/cdh5/parcels/5.10.0/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel
http://archive.cloudera.com/cdh5/parcels/5.10.0/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.10.0/manifest.json

MYSQL安裝包：
https://cdn.mysql.com/archives/mysql-5.6/MySQL-server-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-client-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-shared-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-shared-compat-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-embedded-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-devel-5.6.30-1.el7.x86_64.rpm
或者
https://cdn.mysql.com/archives/mysql-5.6/MySQL-5.6.30-1.el7.x86_64.rpm-bundle.tar

jdbc鏈接包mysql-connector-java.jar:
http://dev.mysql.com/downloads/connector/j/

2. 每臺機器安裝JDK
2.1 檢查是否安裝了JDK： rpm -qa | grep jdk
2.2 若是已經安裝了JDK，則卸載: yum -y remove [查詢出來的包名]
yum -y remove java-1.7.0-openjdk-headless-1.7.0.111-2.6.7.8.el7.x86_64
yum -y remove java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64
yum -y remove java-1.7.0-openjdk-1.7.0.111-2.6.7.8.el7.x86_64
yum -y remove copy-jdk-configs-1.2-1.el7.noarch
yum -y remove java-1.8.0-openjdk-headless-1.8.0.102-4.b14.el7.x86_64

2.3 安裝下載下來的JDK RPM包:
rpm -ivh /home/workspace/zipped/jdk-8u172-linux-x64.rpm

2.4 設置JDK環境變量
vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_172-amd64
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

source /etc/profile #環境變量生效

3. 配置網絡，修改機器名，關閉防火牆，關閉selinux，安裝NTP時間同步
3.1 配置固定IP地址（每臺機器，IP每臺本身選擇變化)
vi /etc/sysconfig/network-scripts/ifcfg-eth0

TYPE=Ethernet
BOOTPROTO=static
DEFROUTE=yes
NAME=eth0
UUID=7ac09286-c35b-4f15-a9ba-701c093832bf
DEVICE=eth0
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_PRIVACY=no
ONBOOT=yes
DNS1=192.168.1.1
IPADDR=192.168.1.246
PREFIX=24
GATEWAY=192.168.1.1

3.2 修改機器名（在各臺機器上執行相應的腳本)
192.167.1.247
hostnamectl set-hostname realtime247
hostnamectl --static set-hostname realtime247

192.167.1.248
hostnamectl set-hostname broker248
hostnamectl --static set-hostname broker248

192.167.1.249
hostnamectl set-hostname history249
hostnamectl --static set-hostname history249

192.167.1.250
hostnamectl set-hostname history250
hostnamectl --static set-hostname history250

192.167.1.251
hostnamectl set-hostname master251
hostnamectl --static set-hostname master251

修改hosts文件
vi /etc/hosts

127.0.0.1 localhost
::1 localhost

192.168.1.247 realtime247
192.168.1.248 broker248
192.168.1.249 history249
192.168.1.250 history250
192.168.1.251 master251

3.3 關閉防火牆
systemctl stop firewalld #關閉防火牆
systemctl disable firewalld #禁用防火牆
systemctl status firewalld #查看防火牆狀態

3.4 selinux關閉(全部節點官方文檔要求)
vi /etc/selinux/config
SELINUX=disabled
#注意： 重啓才能生效
#重啓後檢查
sestatus -v
#SELinux status: disabled

3.5 NTP服務器設置(用於不一樣節點間實現時間同步)
3.5.1 yum -y install ntp #全部節點安裝時間同步服務
3.5.2 master節點更改時間
更改master的節點的時間同步服務器
vi /etc/ntp.conf
# 註釋掉全部server *.*.*的指向，新添加一條可鏈接的ntp服務器(百度一下ntp服務器，我選的是上海交大的)
server ntp.sjtu.edu.cn iburst
restrict default nomodify notrap
driftfile /var/lib/ntp/drift
restrict 127.0.0.1
restrict -6 ::1
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys

3.5.3 在其餘節點上把ntp指向master服務器地址便可(/etc/ntp.conf下)
vi /etc/ntp.conf
server master251 iburst

3.5.4 啓動並設置開機啓動
systemctl start ntpd.service #啓動時間同步服務
systemctl enable ntpd.service #設置開機啓動

3.5.5 時間同步(主節點不作）
ntpdate -u master251

4. 設置SSH免密登錄
A） 每臺機器生成訪問祕鑰，複製到192.168.1.251:/home/workspace目錄下
192.168.1.247：
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.1.251:/home/workspace/authorized_keys247
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件

192.168.1.248：
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.1.251:/home/workspace/authorized_keys248
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件

192.168.1.249：
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.1.251:/home/workspace/authorized_keys249
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件

192.168.1.250：
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.1.251:/home/workspace/authorized_keys250
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件

192.168.1.251：
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
cp ~/.ssh/authorized_keys /home/workspace/authorized_keys251
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件

B） 在192.168.1.251上將全部的共鑰合併成一個公鑰文件
cat /home/workspace/authorized_keys247 >> /home/workspace/authorized_keys
cat /home/workspace/authorized_keys248 >> /home/workspace/authorized_keys
cat /home/workspace/authorized_keys249 >> /home/workspace/authorized_keys
cat /home/workspace/authorized_keys250 >> /home/workspace/authorized_keys
cat /home/workspace/authorized_keys251 >> /home/workspace/authorized_keys

C） 將合併後的公鑰文件複製到集羣中的各個主機中
scp /home/workspace/authorized_keys 192.168.1.247:~/.ssh/
scp /home/workspace/authorized_keys 192.168.1.248:~/.ssh/
scp /home/workspace/authorized_keys 192.168.1.249:~/.ssh/
scp /home/workspace/authorized_keys 192.168.1.250:~/.ssh/
cp /home/workspace/authorized_keys ~/.ssh/ #由於目前在251主機中，因此使用的命令爲cp而不是scp

D） 每臺機器：
chmod 600 ~/.ssh/authorized_keys #authorized_keys的訪問權限
chmod 600 ~/.ssh/

5. 安裝MYSQL
A) 查詢MYSQL是否安裝，若是安裝了，則卸載
rpm -qa | grep mariadb
rpm -qa | grep mysql

卸載命令：
# rpm -e [查詢出來的MYSQL包名] --nodeps
注意：若是卸載不了就強制卸載，--nodeps --force

yum -y remove [查詢出來的MYSQL包名]

rm -rf /var/lib/mysql
rm /etc/my.cnf

B） 安裝MYSQL
rpm -ivh MySQL-devel-5.6.30-1.el7.x86_64.rpm
rpm -ivh MySQL-shared-5.6.30-1.el7.x86_64.rpm
rpm -ivh MySQL-server-5.6.30-1.el7.x86_64.rpm
rpm -ivh MySQL-client-5.6.30-1.el7.x86_64.rpm

C) 設置MYSQL
C-1) systemctl start mysql #啓動MYSQL ##>>>MYSQL啓動後，默認其實己經生成了默認密碼，密碼保存在/root/.mysql_secret文件中
C-2) chkconfig mysql on #設置開機啓動
C-3) 修改MYSQL默認密碼
方法1：
mysql -uroot -p[初始化密碼] #cat /root/.mysql_secret 便可看到初始化密碼
SET PASSWORD=PASSWORD('root'); （重置密碼）
方法2：
service mysql stop
mysqld_safe --user=mysql --skip-grant-tables --skip-networking&
mysql -u root mysql
mysql> use mysql;
mysql> UPDATE user SET Password=PASSWORD('root')where USER='root';
mysql> FLUSH PRIVILEGES;
mysql> quit
# service mysql restart
# mysql -u root –p
Enter password: root
mysql>SET PASSWORD=PASSWORD('root'); （重置密碼）
mysql> use mysql; (此DB存放MySQL的各類配置信息)
Database changed
mysql> select host,user from user; (查看是否可以查詢)

mysqladmin -uroot password '新密碼' #設置密碼

若是數據庫如法啓動，使用下面的腳本進行初始化：
mysql_install_db --user=mysql --ldata=/var/lib/mysql/

C-4) 設置遠程訪問
GRANT ALL PRIVILEGES ON *.* TO '新用戶'@'%' IDENTIFIED BY '新用戶的密碼' WITH GRANT OPTION;
FLUSH PRIVILEGES;

C-5) 複製jdbc mysql 訪問包到java lib目錄下
若是/usr/share/java/目錄不存在，執行
mkdir /usr/share/java/
cp mysql-connector-java-5.1.40-bin.jar /usr/share/java/mysql-connector-java.jar
或
cp mysql/mysql-connector-java-5.1.40-bin.jar /usr/java/jdk1.8.0_172-amd64/lib/mysql-connector-java.jar

C-6） 建立數據庫
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database monitor DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
grant all on *.* to root@"%" Identified by "root";

6. 安裝Cloudera-Manager
A） 解壓cm tar包到指定目錄全部服務器都要(或者在主節點解壓好，而後經過scp到各個節點同一目錄下)
scp /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz 192.168.1.247:/home/workspace/zipped/
scp /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz 192.168.1.248:/home/workspace/zipped/
scp /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz 192.168.1.249:/home/workspace/zipped/
scp /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz 192.168.1.250:/home/workspace/zipped/

mkdir /opt/cloudera-manager
tar -axvf /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz -C /opt/cloudera-manager

 

B) 建立cloudera-scm用戶（全部節點）
useradd --system --home=/opt/cloudera-manager/cm-5.10.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

C) 在主節點建立cloudera-manager-server的本地元數據保存目錄
mkdir /var/cloudera-scm-server
chown cloudera-scm:cloudera-scm /var/cloudera-scm-server
chown cloudera-scm:cloudera-scm /opt/cloudera-manager

D) 配置從節點cloudera-manger-agent指向主節點服務器(master251節點不作此操做)
vi /opt/cloudera-manager/cm-5.10.0/etc/cloudera-scm-agent/config.ini
將server_host改成CMS所在的主機名即master251

E) 主節點中建立parcel-repo倉庫目錄(只在master251節點操做)
mkdir -p /opt/cloudera/parcel-repo
ln -s /var/www/html/cdh/510/parcels/ parcel-repo
chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
#cp CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha1 manifest.json /opt/cloudera/parcel-repo
#mv CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha1 CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha #把CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha1 後綴要把1去掉

F) 全部節點建立parcels目錄(必定要在全部節點操做）
#解釋：Clouder-Manager將CDHs從主節點的/opt/cloudera/parcel-repo目錄中抽取出來，分發解壓激活到各個節點的/opt/cloudera/parcels目錄中】
mkdir -p /opt/cloudera/parcels #建立parcels目錄
chown cloudera-scm:cloudera-scm /opt/cloudera/parcels #給cloudera-scm受權目錄讀寫權限

G）初始腳本配置數據庫scm_prepare_database.sh(在主節點master251上操做)

#mysql -u root -proot #使用root用戶登錄mysql
#create database cloudera DEFAULT CHARSET utf8 COLLATE utf8_general_ci; #建立cloudera數據庫
grant all on *.* to root@"%" Identified by "root"; #受權

/opt/cloudera-manager/cm-5.10.0/share/cmf/schema/scm_prepare_database.sh mysql -hlocalhost -uroot -proot --scm-host master251 cloudera root root
說明：這個腳本就是用來建立和配置CMS須要的數據庫的腳本。各參數是指：
mysql：數據庫用的是mysql，若是安裝過程當中用的oracle，那麼該參數就應該改成oracle。
-hlocalhost：數據庫創建在localhost主機上面。也就是主節點上面。
-uroot：root身份運行mysql。-123456：mysql的root密碼是***。
--scm-host hadoop1：CMS的主機，通常是和mysql安裝的主機是在同一個主機上。
最後三個參數是：數據庫名，數據庫用戶名，數據庫密碼。

H） 主節點上啓動cloudera-scm-server
cp /opt/cloudera-manager/cm-5.10.0/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server #將cloudera-scm-server複製到開機啓動目錄
chkconfig cloudera-scm-server on #設置cloudera-scm-server開機啓動

vi /etc/init.d/cloudera-scm-server #修改cloudera-scm-server服務啓動位置
#CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default}改成
CMF_DEFAULTS=/opt/cloudera-manager/cm-5.10.0/etc/default

service cloudera-scm-server start #啓動cloudera-scm-server服務
#同時爲了保證在每次服務器重啓的時候都能啓動cloudera-scm-server，應該在開機啓動腳本vi /etc/rc.local中加入命令：
service cloudera-scm-server restart

I） 啓動cloudera-scm-agent全部節點
mkdir /opt/cloudera-manager/cm-5.10.0/run/cloudera-scm-agent #建立cloudera-scm-agent運行目錄
cp /opt/cloudera-manager/cm-5.10.0/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent #將cloudera-scm-agent複製到開機啓動目錄
chkconfig cloudera-scm-agent on #設置cloudera-scm-agent開機啓動

vi /etc/init.d/cloudera-scm-agent #修改cloudera-scm-agent服務啓動位置
#CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default}改成
CMF_DEFAULTS=/opt/cloudera-manager/cm-5.10.0/etc/default

service cloudera-scm-agent start #啓動cloudera-scm-agent服務
#同時爲了保證在每次服務器重啓的時候都能啓動cloudera-scm-agent，應該在開機啓動腳本vi /etc/rc.local中加入命令：
service cloudera-scm-agent restart

Master:
mkdir /var/log/cloudera-scm-server
chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-server
chmod g+w /var/log/cloudera-scm-server

Agent:
mkdir /var/log/cloudera-scm-agent
chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-agent
chmod g+w /var/log/cloudera-scm-agent

 

7. 搭建本地yum源

A) 安裝軟件
yum install -y yum-utils createrepo #master機器執行
yum install -y httpd # master機器執行 安裝apache2
systemctl start httpd.service #master機器執行 啓動apache服務
systemctl enable httpd.service #master機器執行 設置開機啓動

systemctl start httpd.service #啓動
systemctl stop httpd.service #中止
systemctl restart httpd.service #重啓
systemctl enable httpd.service #開機啓動
systemctl disable httpd.service #開機不啓動

B) 下載CDH rpm包和Parcel包
下載地址：
http://archive.cloudera.com/cm5/repo-as-tarball/5.10.0/cm5.10.0-centos7.tar.gz #CM
http://archive.cloudera.com/cdh5/repo-as-tarball/5.10.0/ #CDH
mkdir -p /var/www/html/cdh/510

tar -zxvf /home/workspace/zipped/cm5.10.0-centos7.tar.gz -C /var/www/html/cdh/510 #解壓到本地www站點地址 rpms
mv /opt/cloudera/parcel-repo/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel /var/www/html/cdh/510
mv /opt/cloudera/parcel-repo/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha /var/www/html/cdh/510
mv /opt/cloudera/parcel-repo/manifest.json /var/www/html/cdh/510

chmod -R 755 /var/www/html/* #受權文件夾權限

C) 本地yum源
yum install yum-utils createrepo #安裝repo util
mv /home/lenmom/cdh5/ /home/lenmom/cdh5/ # /home/lenmom/cdh5/ contains parcels and rpms.
createrepo /var/www/html/cdh/510/rpms # /var/www/html/cdh/510/rpms rpms.

chmod -R 755 /var/www/html/* #受權文件夾權限

vi /etc/yum.repos.d/cdh510.repo 寫入以下內容
[cdh510]
name=repo
baseurl=http://master251/cdh/510/rpms/
enabled=true
gpgcheck=false

分發到全部agent節點:
scp /etc/yum.repos.d/cdh510.repo realtime247:/etc/yum.repos.d/
scp /etc/yum.repos.d/cdh510.repo broker248:/etc/yum.repos.d/
scp /etc/yum.repos.d/cdh510.repo history249:/etc/yum.repos.d/
scp /etc/yum.repos.d/cdh510.repo history250:/etc/yum.repos.d/

保存後，執行如下兩條命令
yum clean all #清除yum緩存
rm -Rf /var/cache/yum/x86_64
yum update #更新yum緩存
yum makecache #全部節點執行

8. 安裝：

查看端口占用：
netstat -anp|grep 7180 #7180是cloudera-server host的端口號，查看某個端口是否被佔用
lsof -i:7180 #7180是cloudera-server host的端口號，查看某個端口是否被佔用

瀏覽器
http://192.168.1.251:7180/ 開始安裝過程（等待服務器啓動須要一段時間，耐心等待）

遠程pacel所有刪除，只留一個，地址爲：
http://192.168.1.251/cdh/510/parcels/

注意：安裝過程當中，若是出現「主機運行情況不良」，是由於節點上次安裝沒有成功，須要刪除cm_guid文件才能再次安裝。其次要注意master分配的資源是否足夠，有多是由於master資源不夠，心跳沒法及時響應，致使報告主機運行情況不良。解決辦法以下：1. rm -f /opt/cloudera-manager/cm-5.10.0/lib/cloudera-scm-agent/cm_guid service cloudera-scm-agent restart2. 增長master節點的內存和cpu資源，讓他可以及時響應心跳檢測服務。