CDH6安裝文檔

一、準備工做

1.1 環境

centos七、jdk八、mysql5.七、python2.七、CDH6

1.2文件下載

1.2.1 cloudmanger地址

https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/

• cloudera-manager-agent-6.0.0-530873.el7.x86_64.rpm
• cloudera-manager-daemons-6.0.0-530873.el7.x86_64.rpm
  

• cloudera-manager-server-6.0.0-530873.el7.x86_64.rpm

  1.2.2 資源下載地址

  https://archive.cloudera.com/cdh6/6.0.0/parcels/
• CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel
• CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256

• manifest.json

二、開始安裝

2.1 服務器配置

2.1.1 修改HOST和主機名

1. vim /etc/hosts修改host，例如
   
   198.168.12.1 cdh1
   
   192.168.12.2 cdh2
   
   192.168.12.3 cdh3
   
   不能有大寫字母和下劃線

2. vim /etc/hostname修改主機名,修改完後三臺機器分別執行hostname ***(當前節點的主機名)

   2.1.2 配置免密登陸

   1. ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa b、ssh-copy-id 每臺服務器地址

      2.1.3 關閉防火牆

   2. 臨時關閉 systemctl stop firewalld

   3. 禁止開機啓動 systemctl disable firewalld

      2.1.4 安裝jdk，配置環境變量

      這個你們都會，若是不會能夠參考這個教程
      
      https://www.cnblogs.com/ocean-sky/p/8392444.html

      2.1.5 其餘依賴

      每臺服務器yum安裝依賴包：redhat-lsb、portmap、httpd、mod_ssl、openssl-devel、python-psycopg二、MySQL-python、chkconfig、python、chkconfig、bind-utils、libxslt、zlib、sqlite、cyrus-sasl-plain、cyrus-sasl-gssapi、fuse、fuse-libs

      2.2 Mysql5.7安裝(在主節點安裝)

      2.2.1 安裝

   4. wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
   5. yum -y install mysql57-community-release-el7-10.noarch.rpm
   6. yum -y install mysql-community-server
   7. 啓動 systemctl start mysqld.service
   8. 查看運行狀態 systemctl status mysqld.service
   9. 查看初始密碼 grep "password" /var/log/mysqld.log
   10. 登陸後修改密碼（儘可能複雜一點，太簡單了驗證不經過，也能夠查看如何關閉mysql校驗規則） ALTER USER 'root'@'localhost' IDENTIFIED BY 'new password';
   11. 卸載yum源防止自動更新 yum -y remove mysql57-community-release-el7-10.noarch
       

   12. 安裝MySQL-shared-compat或MySQL-shared軟件包(這是Cloudera Manager Agent軟件包安裝所必需的)yum install MySQL-shared-advanced

       2.2.2 建立CDH相關數據庫

• create database scm default character set utf8 default collate utf8_general_ci;
• grant all on scm.* to 'scm'@'%' identified by 'password';
• create database amon default character set utf8 default collate utf8_general_ci;
• grant all on amon.* to 'amon'@'%' identified by 'password';
• create database rman default character set utf8 default collate utf8_general_ci;
• grant all on rman.* to 'rman'@'%' identified by 'password';
• create database hue default character set utf8 default collate utf8_general_ci;
• grant all on hue.* to 'hue'@'%' identified by 'password';
• create database metastore default character set utf8 default collate utf8_general_ci;
• grant all on metastore.* to 'hive'@'%' identified by 'password';
• create database sentry default character set utf8 default collate utf8_general_ci;
• grant all on sentry.* to 'sentry'@'%' identified by 'password';
• create database nav default character set utf8 default collate utf8_general_ci;
• grant all on nav.* to 'nav'@'%' identified by 'password';
• create database oozie default character set utf8 default collate utf8_general_ci;

• grant all on oozie.* to 'oozie'@'%' identified by 'password';

  2.3 CDH安裝

  1. mkdir /opt/cloudera-manager
  2. 將cloudera-manager-agent-6.0.0-530873.el7.x86_64.rpm、cloudera-manager-daemons-6.0.0-530873.el7.x86_64.rpm 、cloudera-manager-server-6.0.0-530873.el7.x86_64.rpm
     複製到/opt/cloudera-manager文件夾中
  3. 按 daemons、server、agent順序解壓
  4. 修改文件/etc/cloudera-scm-agent/config.ini中serverhost指定server節點
  5. 下載mysql-connector-java-5.1.47.jar包並複製爲/usr/share/java/mysql-connector-java.jar
  6. 在主節點執行
     /opt/cloudera/cm/schema/scmpreparedatabase.sh mysql -h 主節點ip --scm-host 主節點ip scm scm
  7. mkdir -p /opt/cloudera/parcel-repo
  8. 將CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel、CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha25六、manifest.json複製到/opt/cloudera/parcel-repo
  9. 將CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256 重命名爲 CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha
  10. 在manifest.json文件中，找到對應版本的祕鑰，複製到CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha文件中。

  11. 啓動systemctl start cloudera-scm-agent、systemctl start cloudera-scm-server

      2.4 集羣配置

      2.4.1 禁用透明大頁面壓縮

      echo never > /sys/kernel/mm/transparent_hugepage/defrag
      
      echo never > /sys/kernel/mm/transparent_hugepage/enabled

      2.4.1 分配組件

      訪問http://192.168.1.1:7180開始配置集羣組件 由於是先安裝集羣后編寫文檔，沒有截圖就沒有詳細寫。