基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫01 —— 準備環境，搭建本地倉庫，安裝ambari
1、集羣軟硬件環境準備：html
操做系統： centos 7 x86_64.1804java
Ambari版本：2.7.0python
HDP版本：3.0.0mysql
HAWQ版本：2.3.0
5臺PC做爲工做站：linux
| ep-bd01 | ep-bd02 | ep-bd03 | ep-bd04 | ep-bd05 |
其中ep-bd01做爲主節點，用於安裝ambari-server。sql
2、配置操做系統，安裝必備軟件數據庫
1，安裝CentOS 7操做系統：環境配置，安裝必備軟件。json
2，安裝配置NTP服務，保證集羣時間保持同步，以防止因爲時間不一樣而形成掉線故障。vim
詳細看隨筆：基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫之——安裝配置NTP服務，保證集羣時間保持同步
見《安裝配置NTP服務》centos
3，安裝MariaDB Server用於Ambari server以及Hue和Hive
詳細過程，參見：基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫之一 —— MariaDB 安裝配置
4，安裝yum priorities plugin
yum install yum-plugin-priorities -y
3、搭建本地倉庫：
1，下載軟件包：
cd /root mkdir downloads cd downloads wget http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.0.0.0/HDP-GPL-3.0.0.0-centos7-gpl.tar.gz wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0/hdp.repo wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0/HDP-3.0.0.0-1634.xml wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.0.0/ambari.repo wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0/HDP-3.0.0.0-centos7-rpm.tar.gz wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.0.0/ambari-2.7.0.0-centos7.tar.gz
二、搭建本地倉庫：
安裝並開啓Apache HTTP服務
yum install httpd -y systemctl enable httpd systemctl start httpd
確保/var/www/html目錄存在，沒有的話建立。
mkdir -p /var/www/html
建立HDP，HDF子目錄
cd /var/www/html mkdir hdp hdf
解開下載的軟件包：
cd /var/www/html tar -zxvf /root/downloads/ambari-2.7.0.0-centos7.tar.gz -C . tar -zxvf /root/downloads/HDP-3.0.0.0-centos7-rpm.tar.gz -C ./hdp tar -zxvf /root/downloads/HDP-GPL-3.0.0.0-centos7-gpl.tar.gz -C ./hdp tar -zxvf /root/downloads/HDP-UTILS-1.1.0.22-centos7.tar.gz -C ./hdp
修改下載的ambari.repo，
vim ambari.repo
安裝以下內容修改，[注意版本號，須要根據具體下載的版本不一樣修改，解壓後本身查看一下]：
#VERSION_NUMBER=2.7.0.0-897
[ambari-2.7.0.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.0.0
baseurl=http://ep-bd01/ambari/centos7/2.7.0.0-897
gpgcheck=1
gpgkey=http://ep-bd01/ambari/centos7/2.7.0.0-897/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
複製到/etc/yum.repos.d
cp ambari.repo /etc/yum.repos.d/ambari.repo
修改下載的hdp.repo，
vim hdp.repo
安裝以下內容修改，[注意版本號，須要根據具體下載的版本不一樣修改，解壓後本身查看一下]：
#VERSION_NUMBER=3.0.0.0-1634
[HDP-3.0]
name=HDP Version - HDP-3.0.0.0
baseurl=http://ep-bd01/hdp/HDP/centos7/3.0.0.0-1634
gpgcheck=1
gpgkey=http://ep-bd01/hdp/HDP/centos7/3.0.0.0-1634/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[HDP-3.0-GPL]
name=HDP GPL Version - HDP-GPL-3.0.0.0
baseurl=http://ep-bd01/hdp/HDP-GPL/centos7/3.0.0.0-1634
gpgcheck=1
gpgkey=http://ep-bd01/hdp/HDP-GPL/centos7/3.0.0.0-1634/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://ep-bd01/hdp/HDP-UTILS/centos7/1.1.0.22
gpgcheck=1
gpgkey=http://ep-bd01/hdp/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
保存退出，複製到/etc/yum.repos.d/
cp hdp.repo /etc/yum.repos.d/hdp.repo
4、主節點安裝ambari server
1，使用剛纔配置好的本地倉庫，直接yum命令安裝。
yum install ambari-server -y
2，查看ambari server 狀態
systemctl status ambari-server ● ambari-server.service - LSB: ambari-server daemon Loaded: loaded (/etc/rc.d/init.d/ambari-server; bad; vendor preset: disabled) Active: inactive (dead) Docs: man:systemd-sysv-generator(8)
看到ambari server已成功安裝了。
3，配置mariadb，創建用戶和數據庫供ambari使用
創建數據庫用戶ambari
mysql -uroot -p
MariaDB [(none)]> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [mysql]> grant all privileges on *.* to 'ambari'@'%' identified by 'ambari';
Query OK, 0 rows affected (0.06 sec)
MariaDB [mysql]> flush privileges;
Query OK, 0 rows affected (0.00 sec)
MariaDB [mysql]>
創建數據庫ambari，並運行ambari數據庫建表sql命令文件。
MariaDB [mysql]> create database ambari; Query OK, 1 row affected (0.01 sec) MariaDB [mysql]> use ambari; Database changed MariaDB [ambari]> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql; Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.06 sec) Query OK, 0 rows affected (0.00 sec) Statement prepared ... ...
4，配置ambari server
執行命令
ambari-server setup
回答選擇項，其中JDK選擇「」Custom「」，給出系統安裝目錄，數據庫必定要選擇高級配置，指定mariadb數據庫和用戶，本人系統中詳細過程以下：
Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 1 To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually. Do you accept the Oracle Binary Code License Agreement [y/n] (y)? ^C Aborting ... Keyboard Interrupt. [root@ep-bd01 downloads]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 2 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: ^C Aborting ... Keyboard Interrupt. [root@ep-bd01 downloads]# echo $JAVA_HOME /usr/java/jdk1.8.0_181-amd64 [root@ep-bd01 downloads]# $JAVA_HOME/bin/java -version java version "1.8.0_181" Java(TM) SE Runtime Environment (build 1.8.0_181-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode) [root@ep-bd01 downloads]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 2 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: ^C Aborting ... Keyboard Interrupt. [root@ep-bd01 downloads]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 1 To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually. Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-8u112-linux-x64.tar.gz ERROR: Exiting with exit code 1. REASON: Downloading or installing JDK failed: 'Fatal exception: Failed to download JDK: <urlopen error [Errno -2] Name or service not known>. Please check that the JDK is available at http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz. Also you may specify JDK file location in local filesystem using --jdk-location command line argument., exit code 1'. Exiting. [root@ep-bd01 downloads]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 2 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /usr/java/jdk1.8.0_181-amd64 Validating JDK on Ambari Server...done. Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? n Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... ============================================================================== Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB ============================================================================== Enter choice (1): 3 Hostname (localhost): ep-bd01 Port (3306): Database name (ambari): ambari Username (ambari): ambari Enter Database Password (bigdata): Re-enter password: Configuring ambari database... Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? y Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ambari-admin-2.7.0.0.897.jar .... Ambari repo file doesn't contain latest json url, skipping repoinfos modification Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully.
5，命令行方式設置mysql數據庫鏈接庫，用於oozie和ranger鏈接mariadb時使用（上面設置後不起做用，必須以下操做）：
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
6，配置ambari-server自啓動，啓動ambari-server
[root@ep-bd01 downloads]# systemctl enable ambari-server [root@ep-bd01 downloads]# systemctl start ambari-server [root@ep-bd01 downloads]# systemctl status ambari-server ● ambari-server.service - LSB: ambari-server daemon Loaded: loaded (/etc/rc.d/init.d/ambari-server; bad; vendor preset: disabled) Active: active (running) since Tue 2018-08-14 11:06:16 CST; 2min 36s ago Docs: man:systemd-sysv-generator(8) Process: 323056 ExecStart=/etc/rc.d/init.d/ambari-server start (code=exited, status=0/SUCCESS) CGroup: /system.slice/ambari-server.service └─323080 /usr/java/jdk1.8.0_181-amd64/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -Dsun.zip.disableMemoryMapping=true -... Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Organizing resource files at /var/lib/ambari-server/resources... Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Ambari database consistency check started... Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Server PID at: /var/run/ambari-server/ambari-server.pid Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Server out at: /var/log/ambari-server/ambari-server.out Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Server log at: /var/log/ambari-server/ambari-server.log Aug 14 11:06:16 ep-bd01 ambari-server[323056]: Waiting for server start..................... Aug 14 11:06:16 ep-bd01 ambari-server[323056]: Server started listening on 8080 Aug 14 11:06:16 ep-bd01 ambari-server[323056]: DB configs consistency check: no errors and warnings were found. Aug 14 11:06:16 ep-bd01 ambari-server[323056]: Ambari Server 'start' completed successfully. Aug 14 11:06:16 ep-bd01 systemd[1]: Started LSB: ambari-server daemon.systemctl status ambari-server ● ambari-server.service - LSB: ambari-server daemon Loaded: loaded (/etc/rc.d/init.d/ambari-server; bad; vendor preset: disabled) Active: active (running) since Tue 2018-08-14 11:06:16 CST; 2min 36s ago Docs: man:systemd-sysv-generator(8) Process: 323056 ExecStart=/etc/rc.d/init.d/ambari-server start (code=exited, status=0/SUCCESS) CGroup: /system.slice/ambari-server.service └─323080 /usr/java/jdk1.8.0_181-amd64/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -Dsun.zip.disableMemoryMapping=true -... Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Organizing resource files at /var/lib/ambari-server/resources... Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Ambari database consistency check started... Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Server PID at: /var/run/ambari-server/ambari-server.pid Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Server out at: /var/log/ambari-server/ambari-server.out Aug 14 11:05:58 ep-bd01 ambari-server[323056]: Server log at: /var/log/ambari-server/ambari-server.log Aug 14 11:06:16 ep-bd01 ambari-server[323056]: Waiting for server start..................... Aug 14 11:06:16 ep-bd01 ambari-server[323056]: Server started listening on 8080 Aug 14 11:06:16 ep-bd01 ambari-server[323056]: DB configs consistency check: no errors and warnings were found. Aug 14 11:06:16 ep-bd01 ambari-server[323056]: Ambari Server 'start' completed successfully. Aug 14 11:06:16 ep-bd01 systemd[1]: Started LSB: ambari-server daemon.
ok，Ambari-server安裝完成。瀏覽器訪問ambari-server服務：
http://ep-bd01:8080
默認用戶名/密碼爲：admin/admin，登錄後界面以下：
5、全部主機節點，安裝ambari-agent，並配置自啓動
yum install ambari-agent -y systemctl enable ambari-agent systemctl restart ambari-agent && systemctl status ambari-agent
相關文章
- 1. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫03 —— 安裝HAWQ 2.3.0.0
- 2. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫 —— MariaDB 安裝配置
- 3. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫02 ——使用ambari-server安裝HDP
- 4. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫04 —— 安裝HAWQ插件PXF3.3.0.0
- 5. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫——操做系統配置，安裝必備軟件
- 6. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】HAWQ數據倉庫 使用之 gpfdist協議
- 7. CentOS 7使用本地倉庫安裝ambari 2.4，搭建hadoop集羣環境（一）
- 8. Ambari倉庫安裝教程
- 9. Ambari環境搭建（離線搭建）
- 10. 建立本地倉庫centos7
- 更多相關文章...
- • Swift 環境搭建 - Swift 教程
- • Rust 環境搭建 - RUST 教程
- • 適用於PHP初學者的學習線路和建議
- • Composer 安裝與使用
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
- 1. Window下Ribbit MQ安裝
- 2. Linux下Redis安裝及集羣搭建
- 3. shiny搭建網站填坑戰略
- 4. Mysql8.0.22安裝與配置詳細教程
- 5. Hadoop安裝及配置
- 6. Python爬蟲初學筆記
- 7. 部署LVS-Keepalived高可用集羣
- 8. keepalived+mysql高可用集羣
- 9. jenkins 公鑰配置
- 10. HA實用詳解
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫03 —— 安裝HAWQ 2.3.0.0
- 2. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫 —— MariaDB 安裝配置
- 3. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫02 ——使用ambari-server安裝HDP
- 4. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫04 —— 安裝HAWQ插件PXF3.3.0.0
- 5. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ數據倉庫——操做系統配置，安裝必備軟件
- 6. 基於【CentOS-7+ Ambari 2.7.0 + HDP 3.0】HAWQ數據倉庫 使用之 gpfdist協議
- 7. CentOS 7使用本地倉庫安裝ambari 2.4，搭建hadoop集羣環境（一）
- 8. Ambari倉庫安裝教程
- 9. Ambari環境搭建（離線搭建）
- 10. 建立本地倉庫centos7