手把手教你經過Ambari新建Hadoop集羣圖解案例
手把手教你經過Ambari新建Hadoop集羣圖解案例html
做者：尹正傑node
版權聲明：原創做品，謝絕轉載！不然將追究法律責任。python
登錄系統以後，會看到Ambari空空如也的歡迎界面，接下來咱們就須要介紹如何經過Ambari新建Hadoop集羣。關於如何部署Ambari是很簡單的，我這裏就不囉嗦了，感興趣的小夥伴能夠參考我以前的筆記：離線方式部署Ambari2.6.0.0。linux
1>.登錄Ambari的界面git
2>.點擊「Launc Install Wizard」按鈕進入新建集羣嚮導redis
3>.自定義集羣名稱shell
4>.使用本地的yum倉庫（須要選擇「Use Local Repository」，我這裏默認就使用的這個選項。） express
5>.在redhat7按照上圖輸入本地源的路徑後，點擊下一步 apache
6>.將私鑰上傳到Ambari到管理控制檯上（這個步驟應該是你在部署Ambari服務到時候就已經生成過私鑰啦！）vim
7>.註冊服務器時拍錯處理（附有解決問題的詳細過程）
下圖正在安裝軟件：
註冊服務器，以下圖：
在上面的那個過程若是你卡頓的時間比較長的話，估計會失敗，以下圖：
查看錯誤日誌，以下圖：
根據上面的報錯信息，你們都知道是沒有這個目錄，纔會報這個錯誤。多是Ambari公司的開發人員在對這個目錄進行判斷時邏輯出現問題了，若是這個目錄不存在他們並無自動建立，
解決方案：
所以咱們須要登錄對應的服務器手動將這個目錄建立出來便可。
咱們看報錯必定要看全，咱們須要把進度條往下拉，看是否還有其餘的報錯，你會發現有如下報錯：
如上圖所示，報錯「SSLError: Failed to connect. Please check openssl library versions. 」，根據報錯提示，說是個人openssl的版本有問題。百度一下緣由，說有多是openssl版本不匹配，由於python 2.7.5有的版本（或更高版本）增長了certificate verification，正是由於這個特性致使ambari-agent沒法鏈接server。（呵呵～是否是感受跟沒說同樣！）
解決步驟
既然他說個人版本有問題，那我就先看卡本身的openssl究竟是啥版本
[root@node101 ~]# rpm -qa | grep openssl openssl-libs-1.0.2k-16.el7.x86_64 openssl-1.0.2k-16.el7.x86_64 [root@node101 ~]#
好，接下里，我們就說幹就幹，開始升級版本：
[root@node101 ~]# yum update openssl Loaded plugins: fastestmirror, priorities ambari-2.6.0.0 | 2.9 kB 00:00:00 Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn 17 packages excluded due to repository priority protections No packages marked for update [root@node101 ~]#
根據上面的提示，說沒有能夠更新的安裝包對於我服務器而言，所以我得像個方法，讓他能夠支持更新，那就得從新更換yum源了。
[root@node101 ~]# mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.`date +%F` #先備份一份yum源 [root@node101 ~]# [root@node101 ~]# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo --2018-12-06 17:01:48-- http://mirrors.aliyun.com/repo/Centos-7.repo Resolving mirrors.aliyun.com (mirrors.aliyun.com)... 42.81.12.221, 42.81.12.217, 42.81.12.218, ... Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|42.81.12.221|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2523 (2.5K) [application/octet-stream] Saving to: ‘/etc/yum.repos.d/CentOS-Base.repo’ 100%[===========================================================================>] 2,523 --.-K/s in 0.001s 2018-12-06 17:01:48 (2.68 MB/s) - ‘/etc/yum.repos.d/CentOS-Base.repo’ saved [2523/2523] [root@node101 ~]#
[root@node101 ~]# yum clean all Loaded plugins: fastestmirror, priorities Cleaning repos: HDP-2.6.3.0 HDP-UTILS-1.1.0.21 ambari-2.6.0.0 base extras updates Cleaning up list of fastest mirrors [root@node101 ~]#
[root@node101 ~]# yum makecache Loaded plugins: fastestmirror, priorities HDP-2.6.3.0 | 2.9 kB 00:00:00 HDP-UTILS-1.1.0.21 | 2.9 kB 00:00:00 ambari-2.6.0.0 | 2.9 kB 00:00:00 base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/21): HDP-2.6.3.0/filelists_db | 418 kB 00:00:00 (2/21): HDP-2.6.3.0/primary_db | 100 kB 00:00:00 (3/21): HDP-2.6.3.0/other_db | 14 kB 00:00:00 (4/21): HDP-UTILS-1.1.0.21/filelists_db | 95 kB 00:00:00 (5/21): HDP-UTILS-1.1.0.21/other_db | 20 kB 00:00:00 (6/21): HDP-UTILS-1.1.0.21/primary_db | 38 kB 00:00:00 (7/21): ambari-2.6.0.0/filelists_db | 167 kB 00:00:00 (8/21): ambari-2.6.0.0/other_db | 1.3 kB 00:00:00 (9/21): ambari-2.6.0.0/primary_db | 8.6 kB 00:00:00 (10/21): base/7/x86_64/group_gz | 166 kB 00:00:00 (11/21): base/7/x86_64/filelists_db | 7.1 MB 00:00:11 (12/21): extras/7/x86_64/prestodelta | 33 kB 00:00:00 (13/21): extras/7/x86_64/filelists_db | 330 kB 00:00:00 (14/21): extras/7/x86_64/other_db | 106 kB 00:00:01 (15/21): updates/7/x86_64/prestodelta | 173 kB 00:00:00 (16/21): updates/7/x86_64/filelists_db | 1.3 MB 00:00:02 (17/21): extras/7/x86_64/primary_db | 153 kB 00:00:03 (18/21): updates/7/x86_64/other_db | 182 kB 00:00:01 (19/21): updates/7/x86_64/primary_db | 1.3 MB 00:00:02 (20/21): base/7/x86_64/other_db | 2.6 MB 00:00:07 (21/21): base/7/x86_64/primary_db | 6.0 MB 00:00:21 Determining fastest mirrors * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Metadata Cache Created [root@node101 ~]#
[root@node101 ~]# yum -y update ......(內容太多了我就懶得複製了~) libdrm.x86_64 0:2.4.91-3.el7 libffi.x86_64 0:3.0.13-18.el7 libgcc.x86_64 0:4.8.5-36.el7 libgcrypt.x86_64 0:1.5.3-14.el7 libgomp.x86_64 0:4.8.5-36.el7 libgudev1.x86_64 0:219-62.el7 libmount.x86_64 0:2.23.2-59.el7 libndp.x86_64 0:1.2-7.el7 libnetfilter_conntrack.x86_64 0:1.0.6-1.el7_3 libnl3.x86_64 0:3.2.28-4.el7 libpcap.x86_64 14:1.5.3-11.el7 libpciaccess.x86_64 0:0.14-1.el7 libproxy.x86_64 0:0.4.11-11.el7 libpwquality.x86_64 0:1.2.3-5.el7 libselinux.x86_64 0:2.5-14.1.el7 libselinux-python.x86_64 0:2.5-14.1.el7 libselinux-utils.x86_64 0:2.5-14.1.el7 libsemanage.x86_64 0:2.5-14.el7 libsepol.x86_64 0:2.5-10.el7 libsoup.x86_64 0:2.62.2-2.el7 libss.x86_64 0:1.42.9-13.el7 libssh2.x86_64 0:1.4.3-12.el7 libstdc++.x86_64 0:4.8.5-36.el7 libtasn1.x86_64 0:4.10-1.el7 libuser.x86_64 0:0.60-9.el7 libuuid.x86_64 0:2.23.2-59.el7 linux-firmware.noarch 0:20180911-69.git85c5d90.el7 logrotate.x86_64 0:3.8.6-17.el7 lsscsi.x86_64 0:0.27-6.el7 lua.x86_64 0:5.1.4-15.el7 lvm2.x86_64 7:2.02.180-10.el7_6.2 lvm2-libs.x86_64 7:2.02.180-10.el7_6.2 make.x86_64 1:3.82-23.el7 man-db.x86_64 0:2.6.3-11.el7 mariadb-libs.x86_64 1:5.5.60-1.el7_5 microcode_ctl.x86_64 2:2.1-47.el7 mozjs17.x86_64 0:17.0.0-20.el7 ncurses.x86_64 0:5.9-14.20130511.el7_4 ncurses-base.noarch 0:5.9-14.20130511.el7_4 ncurses-libs.x86_64 0:5.9-14.20130511.el7_4 net-tools.x86_64 0:2.0-0.24.20131004git.el7 nettle.x86_64 0:2.7.1-8.el7 nspr.x86_64 0:4.19.0-1.el7_5 nss.x86_64 0:3.36.0-7.el7_5 nss-softokn.x86_64 0:3.36.0-5.el7_5 nss-softokn-freebl.x86_64 0:3.36.0-5.el7_5 nss-sysinit.x86_64 0:3.36.0-7.el7_5 nss-tools.x86_64 0:3.36.0-7.el7_5 nss-util.x86_64 0:3.36.0-1.el7_5 numactl-libs.x86_64 0:2.0.9-7.el7 openldap.x86_64 0:2.4.44-20.el7 openssh.x86_64 0:7.4p1-16.el7 openssh-clients.x86_64 0:7.4p1-16.el7 openssh-server.x86_64 0:7.4p1-16.el7 os-prober.x86_64 0:1.58-9.el7 p11-kit.x86_64 0:0.23.5-3.el7 p11-kit-trust.x86_64 0:0.23.5-3.el7 pam.x86_64 0:1.1.8-22.el7 parted.x86_64 0:3.1-29.el7 pciutils-libs.x86_64 0:3.5.1-3.el7 pcre.x86_64 0:8.32-17.el7 perl.x86_64 4:5.16.3-293.el7 perl-Pod-Escapes.noarch 1:1.04-293.el7 perl-libs.x86_64 4:5.16.3-293.el7 perl-macros.x86_64 4:5.16.3-293.el7 pinentry.x86_64 0:0.8.1-17.el7 plymouth.x86_64 0:0.8.9-0.31.20140113.el7.centos plymouth-core-libs.x86_64 0:0.8.9-0.31.20140113.el7.centos plymouth-scripts.x86_64 0:0.8.9-0.31.20140113.el7.centos policycoreutils.x86_64 0:2.5-29.el7 polkit.x86_64 0:0.112-18.el7 postfix.x86_64 2:2.10.1-7.el7 procps-ng.x86_64 0:3.3.10-23.el7 python.x86_64 0:2.7.5-76.el7 python-libs.x86_64 0:2.7.5-76.el7 python-perf.x86_64 0:3.10.0-957.1.3.el7 python-pycurl.x86_64 0:7.19.0-19.el7 python-pyudev.noarch 0:0.15-9.el7 python-slip.noarch 0:0.4.0-4.el7 python-slip-dbus.noarch 0:0.4.0-4.el7 readline.x86_64 0:6.2-10.el7 rsyslog.x86_64 0:8.24.0-34.el7 selinux-policy.noarch 0:3.13.1-229.el7_6.6 selinux-policy-targeted.noarch 0:3.13.1-229.el7_6.6 setup.noarch 0:2.8.71-10.el7 shadow-utils.x86_64 2:4.1.5.1-25.el7 shared-mime-info.x86_64 0:1.8-4.el7 sudo.x86_64 0:1.8.23-3.el7 systemd.x86_64 0:219-62.el7 systemd-libs.x86_64 0:219-62.el7 systemd-sysv.x86_64 0:219-62.el7 tar.x86_64 2:1.26-35.el7 trousers.x86_64 0:0.3.14-2.el7 tuned.noarch 0:2.10.0-6.el7 tzdata.noarch 0:2018g-1.el7 util-linux.x86_64 0:2.23.2-59.el7 vim-minimal.x86_64 2:7.4.160-5.el7 virt-what.x86_64 0:1.18-4.el7 wpa_supplicant.x86_64 1:2.6-12.el7 xfsprogs.x86_64 0:4.5.0-18.el7 xz.x86_64 0:5.2.2-1.el7 xz-libs.x86_64 0:5.2.2-1.el7 yum-plugin-fastestmirror.noarch 0:1.1.31-50.el7 zlib.x86_64 0:1.2.7-18.el7 Replaced: NetworkManager.x86_64 1:1.0.6-27.el7 grub2.x86_64 1:2.02-0.29.el7.centos grub2-tools.x86_64 1:2.02-0.29.el7.centos pygobject3-base.x86_64 0:3.14.0-3.el7 Complete! [root@node101 ~]#
[root@node101 ~]# yum update openssl Loaded plugins: fastestmirror, priorities Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com 17 packages excluded due to repository priority protections No packages marked for update [root@node101 ~]#
問題依舊仍是僵持着，參考社區版本連接：https://community.hortonworks.com/questions/145/openssl-error-upon-host-registration.html
修改/etc/python/cert-verification.cfg配置文件
[root@node101 ~]# cat /etc/python/cert-verification.cfg # Possible values are: # 'enable' to ensure HTTPS certificate verification is enabled by default # 'disable' to ensure HTTPS certificate verification is disabled by default # 'platform_default' to delegate the decision to the redistributor providing this particular Python version # For more info refer to https://www.python.org/dev/peps/pep-0493/ [https] #verify=platform_default #這是我操做系統默認的設置，我將它的值改成「disable」，建議複製一行進行修改，這樣你知道上修改前的屬性值是多少，便於你恢復。 verify=disable [root@node101 ~]#
修改/etc/ambari-agent/conf/ambari-agent.ini，在[security]添加下面內容：
[root@node101 ~]# cat /etc/ambari-agent/conf/ambari-agent.ini # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific [server] hostname=node101.yinzhengjie.org.cn url_port=8440 secured_url_port=8441 connect_retry_delay=10 max_reconnect_retry_delay=30 [agent] logdir=/var/log/ambari-agent piddir=/var/run/ambari-agent prefix=/var/lib/ambari-agent/data ;loglevel=(DEBUG/INFO) loglevel=INFO data_cleanup_interval=86400 data_cleanup_max_age=2592000 data_cleanup_max_size_MB = 100 ping_port=8670 cache_dir=/var/lib/ambari-agent/cache tolerate_download_failures=true run_as_user=root parallel_execution=0 alert_grace_period=5 status_command_timeout=5 alert_kinit_timeout=14400000 system_resource_overrides=/etc/resource_overrides ; memory_threshold_soft_mb=400 ; memory_threshold_hard_mb=1000 ; ignore_mount_points=/mnt/custom1,/mnt/custom2 [security] keysdir=/var/lib/ambari-agent/keys server_crt=ca.crt passphrase_env_var_name=AMBARI_PASSPHRASE ssl_verify_cert=0 credential_lib_dir=/var/lib/ambari-agent/cred/lib credential_conf_dir=/var/lib/ambari-agent/cred/conf credential_shell_cmd=org.apache.hadoop.security.alias.CredentialShell force_https_protocol=PROTOCOL_TLSv1_2 #沒錯，我修改的就是它！ [network] ; this option apply only for Agent communication use_system_proxy_settings=true [services] pidLookupPath=/var/run/ [heartbeat] state_interval_seconds=60 dirs=/etc/hadoop,/etc/hadoop/conf,/etc/hbase,/etc/hcatalog,/etc/hive,/etc/oozie, /etc/sqoop, /var/run/hadoop,/var/run/zookeeper,/var/run/hbase,/var/run/templeton,/var/run/oozie, /var/log/hadoop,/var/log/zookeeper,/var/log/hbase,/var/run/templeton,/var/log/hive ; 0 - unlimited log_lines_count=300 idle_interval_min=1 idle_interval_max=10 [logging] syslog_enabled=0 [root@node101 ~]#
修改上述的配置文件後，咱們須要重啓重啓ambari-agent 服務：
[root@node101 ~]# xrsync.sh /etc/ambari-agent/conf/ambari-agent.ini =========== node102.yinzhengjie.org.cn : /etc/ambari-agent/conf/ambari-agent.ini =========== 命令執行成功 =========== node103.yinzhengjie.org.cn : /etc/ambari-agent/conf/ambari-agent.ini =========== 命令執行成功 [root@node101 ~]#
[root@node101 ~]# ambari-agent restart Restarting ambari-agent Verifying Python version compatibility... Using python /usr/bin/python Found ambari-agent PID: 5248 Stopping ambari-agent Removing PID file at /run/ambari-agent/ambari-agent.pid ambari-agent successfully stopped Verifying Python version compatibility... Using python /usr/bin/python Checking for previously running Ambari Agent... Checking ambari-common dir... Starting ambari-agent Verifying ambari-agent process status... Ambari Agent successfully started Agent PID at: /run/ambari-agent/ambari-agent.pid Agent out at: /var/log/ambari-agent/ambari-agent.out Agent log at: /var/log/ambari-agent/ambari-agent.log [root@node101 ~]#
Duang～，你會發現服務器註冊成功啦～問題獲得解決：（看來HDP社區版的論壇的確是有很多大佬啊～哈哈哈）
8>.進入服務選擇頁面，自定義須要安裝的服務
9>.自定義須要安裝的服務後，點擊下一步
10>.分配主控服務，分配完畢後點擊下一步
11>.分配子服務和客戶端
12>.配置NameNode和DataNode在服務器上的存儲路徑
輸入Grafana Admin的密碼，以下圖：
點擊途中的有警告的按鈕，我們依然還須要輸入一個密碼，以下圖：
13>.查看安裝預覽頁面，顯示了安裝組件的概覽信息（若是你察覺到哪裏配置的不合理能夠點擊「back」返回去修改），如無異常，點擊「Deploy」
14>.等待組件安裝過程
15>.安裝完畢，點擊下一步
16>.安裝完畢後，點擊「Complete」，進入主界面
17>.進入Ambari的主界面
相關文章
- 1. 手把手教你搭建Hadoop集羣
- 2. Hadoop（三）手把手教你搭建Hadoop全分佈式集羣
- 3. 手把手教你搭建hadoop集羣（CentOS6.X）
- 4. 手把手教你搭建hadoop+hive測試環境(新手向)
- 5. 手把手帶你搭建redis集羣
- 6. 手把手帶你搭建 Elasticsearch 集羣
- 7. redis集羣搭建，手把手教學
- 8. 【手把手教你Maven】構建過程
- 9. 手把手教你新建SSM項目
- 10. 手把手教你搭建一個 Elasticsearch 集羣
- 更多相關文章...
- • Swarm 集羣管理 - Docker教程
- • XQuery 參考手冊 - XQuery 教程
- • 再有人問你分佈式事務，把這篇扔給他
- • PHP Ajax 跨域問題最佳解決方案
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
