概念瞭解html
Ambair介紹node
Apache Ambari是一個用於支持大數據軟件供應 管理與監控軟件.它也是一個分佈式軟件,分爲Ambair-Server與Ambari-Client兩個部分.在生產環境下通常單獨用一臺服務器安裝ambari-server來確保服務的穩定性.而後須要安裝大數據服務組件的服務器上均安裝ambari-client組件,server發送命令與client進行交互完成任務.python
HDP介紹linux
HDP是hortonworks公司的Hadoop發行版,其中包括大多數Hadoop生態下的重要組件,能夠做爲Ambari進行部署與安裝.從官網上能夠查看HDP中版本與其餘組件的版本對應關係:https://zh.hortonworks.com/products/data-platforms/hdp/數據庫
準備工做json
硬件準備ubuntu
這裏使用的是阿里雲中Ubuntu16.06版本服務器,預裝有Python2.7環境,機器三臺,配置爲一臺2H8G與兩臺2H4G服務器做爲測試搭建環境.系統準備好後防火牆及端口是全開放的,直接啓用的root用戶.若環境與之不符做爲測試環境能夠先匹配一下.選用配置最好的做爲master,剩下的兩臺做爲兩個slave節點.服務器
準備安裝包dom
這邊由於是要測試Spark的一些新特性,因此所準備的安裝包均爲當前時間點最新的,直接從hortonworks官網進行下載;ssh
其中要安裝的組件有JDK-1.8 確認一下Python版本爲2.7 而後下載Ambari2.7.3對應的組件,其中包括:ambari-2.7.3.0 HDP HDP-UTILS HDP-GPL
首先是Ambari安裝包位置: 選擇下載最新版,這樣子在版本控制方面選擇餘地更大 主頁: https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/ambari_repositories.html https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/hdp_31_repositories.html 下載連接: ambari-2.7.3.0 http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu16.tar.gz HDP http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu16-deb.tar.gz HDP-UTILS http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16/HDP-UTILS-1.1.0.22-ubuntu16.tar.gz HDP-GPL http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu16-gpl.tar.gz
查看配置
首先查看機器硬件配置信息,包括內存及CPU配置信息,根據配置分配不一樣的角色:
cat /proc/meminfo |grep MemTotal
cat /proc/cpuinfo |grep 「model name」 && cat /proc/cpuinfo |grep 「physical id」
主機命名 master.example.com node1.example.com node2.example.com(生產環境下建議使用域名服務器進行解析)
三步操做:1 )hostname 主機名 2)修改/etc/hosts 文件對應DNS解析 3)修改/etc/hostname文件 最後使用hostname hostname -A對結果進行檢測,要求與設置值一致
修改/etc/hosts 添加對應的域名配置解析,將主機名與IP地址相對應(注意IP地址與DNS要作到一一對應,會存在反解析操做,不然有坑)
ssh免密碼登陸 ssh-keygen -t rsa 而後id_rsa.pub 導入到authorized_keys文件中 scp將文件發送到遠程服務器對應目錄上
配置環境
配置全部機器的JDK環境 首先是安裝Ambari Server服務器,這邊由於牆的緣由須要本身配置本地源 : mkdir -p /var/www/html 將上述安裝包解壓到該目錄下,經過Python服務器在該目錄啓動訪問連接:( 以靜默的方式啓動Python服務器 nohup python -m SimpleHTTPServer 1>out.log 2>&1 &)
mkdir -p /var/www/html
ls /var/www/html
ambari HDP HDP-GPL HDP-UTILS
將安裝包解壓後放入上述目錄
配置Ubuntu啓動源
指定倉庫地址: https://www.jianshu.com/p/e33e6d9a0655 /etc/apt/sources.list.d cat /etc/apt/sources.lost.d/ambari.list deb http://10.1.51.17:8000/ambari/ubuntu16/2.7.3.0-139/ Ambari main etc/apt/sources.list.d/ambari-hdp.list deb http://10.1.51.17:8000/HDP/ubuntu16/3.1.0.0-78/ HDP main deb http://10.1.51.17:8000/HDP-GPL/ubuntu16/3.1.0.0-78/ HDP-GPL main deb http://10.1.51.17:8000/HDP-UTILS/ubuntu16/1.1.0.22/ HDP-UTILS main apt-key adv --recv-keys --keyserver keyserver.ubuntu.com B9733A7A07513CAD apt-get update apt-cache showpkg ambari-server
最後在apt-server服務器上執行:
apt-get install ambari-server
能夠順利安裝ambari-server;
以後執行ambari-server setup對其進行安裝配置操做,包括設置jdk與存儲數據庫等配置
最後ambari-server start啓動,啓動後能夠經過該機器的8080端口訪問Ambari頁面
當打開如下頁面後,Ambari的安裝流程就告一段落,能夠先打一個鏡像了;
當打開以上界面後證實Ambari Server已經能夠成功啓動了,後邊如今須要利用它進行Ambari agent配置工做,同時包括大數據的一些服務組件配置:
首先在以上界面對該集羣進行命名;
選擇對應的HDP版本,同時配置一些源服務.這個地方若是不須要修改源碼能夠初略處理,後邊我想要更換HDP源還須要進行一些替換操做;
填好須要安裝Ambari agent服務器的hostname,同時下邊貼上ambari server服務器的id_rsa文件,注意沒有.pub後綴,不然在下一步安裝的時候會提示ssh鏈接失敗的錯誤.(以下圖所示)
填好以後下一步就會進行Ambari agent安裝工做了.
安裝完成後選擇本身所須要的服務組件,進行安裝,注意要將服務分配均勻 同時中間若是有出錯須要及時解決錯誤;
出坑1:
Ubuntu16.04安裝Ambari2.6.2.0配置HDP2.5.5.0-292碰見大坑,安裝Ambari Client有個坎一直過不去.
[code=csharp]
==========================
Creating target directory...
==========================
Command start time 2019-01-04 12:38:12
chmod: cannot access '/var/lib/ambari-agent/data': No such file or directory
Warning: Permanently added 'master.example.com,10.10.80.131' (ECDSA) to the list of known hosts.
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:12
==========================
Copying ambari sudo script...
==========================
Command start time 2019-01-04 12:38:12
scp /var/lib/ambari-server/ambari-sudo.sh
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying common functions script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_commons
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
scp /var/lib/ambari-server/create-python-wrap.sh
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying OS type check script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_server/os_check_type.py
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Running create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running OS type check...
==========================
Command start time 2019-01-04 12:38:14
Cluster primary/cluster OS family is ubuntu16 and local/current OS family is ubuntu16
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Checking 'sudo' package on remote host...
==========================
Command start time 2019-01-04 12:38:14
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Copying repo file to 'tmp' folder...
==========================
Command start time 2019-01-04 12:38:14
scp /etc/apt/sources.list.d/ambari.list
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Moving file to repo dir...
==========================
Command start time 2019-01-04 12:38:15
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Changing permissions for ambari.repo...
==========================
Command start time 2019-01-04 12:38:15
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Update apt cache of repository...
==========================
Command start time 2019-01-04 12:38:15
0% [Working]
Get:1 http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155 Ambari InRelease [7,394 B]
0% [1 InRelease 7,394 B/7,394 B 100%]
0% [Working]
Get:2 http://10.10.80.131:8000/HDP/ubuntu16/2.6.5.0-292 HDP InRelease [7,376 B]
Get:3 http://10.10.80.131:8000/HDP-GPL/ubuntu16/2.6.5.0-292 HDP-GPL InRelease [7,395 B]
Get:4 http://10.10.80.131:8000/HDP-UTILS/ubuntu16/1.1.0.22 HDP-UTILS InRelease [7,405 B]
0% [Working]
0% [1 InRelease gpgv 7,394 B]
0% [Working]
0% [2 InRelease gpgv 7,376 B]
0% [Working]
0% [3 InRelease gpgv 7,395 B]
0% [Working]
0% [4 InRelease gpgv 7,405 B]
100% [Working]
Fetched 29.6 kB in 0s (148 kB/s)
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 8%
Reading package lists... 19%
Reading package lists... 19%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... Done
W: http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155/dists/Ambari/InRelease: Signature by key DF52ED4F7A3A5882C0994C66B9733A7A07513CAD uses weak digest algorithm (SHA1)
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Copying setup script file...
==========================
Command start time 2019-01-04 12:38:16
scp /usr/lib/ambari-server/lib/ambari_server/setupAgent.py
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Running setup agent script...
==========================
Command start time 2019-01-04 12:38:16
dpkg-query: no packages found matching ambari-agent
('WARNING 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7f0e12cad490>; currently running: False
INFO 2019-01-04 12:38:30,685 hostname.py:106 - Read public hostname \'master.example.com\' using socket.getfqdn()
INFO 2019-01-04 12:38:30,686 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:30,737 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:30,748 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'master.example.com\' using socket.getfqdn().
INFO 2019-01-04 12:38:30,750 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources.
INFO 2019-01-04 12:38:30,752 Hardware.py:73 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'example.com\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.4.0-117-generic\', \'uptime_days\': \'0\', \'memorytotal\': 8174812, \'swapfree\': \'0.00 GB\', \'memorysize\': 8174812, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': \'255.255.255.0\', \'timezone\': \'CST\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 241164, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.4\', \'kernelversion\': \'4.4.0\', \'macaddress\': \'00:16:3E:10:7E:5C\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'10.10.80.131\', \'hostname\': \'master\', \'uptime_hours\': \'1\', \'fqdn\': \'master.example.com\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'4068960\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'4068960\'}, {\'available\': \'24108320\', \'used\': \'14930060\', \'percent\': \'39%\', \'device\': \'/dev/vda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'41151808\'}, {\'available\': \'88747784\', \'used\': \'9072256\', \'percent\': \'10%\', \'device\': \'/dev/vdb1\', \'mountpoint\': \'/mnt/disk1\', \'type\': \'ext4\', \'size\': \'103079200\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'7017\', \'interfaces\': \'eth0,lo\'}
INFO 2019-01-04 12:38:30,953 Controller.py:170 - Registering with master.example.com (10.10.80.131) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "example.com", "physicalprocessorcount": 4, "kernelrelease": "4.4.0-117-generic", "uptime_days": "0", "memorytotal": 8174812, "swapfree": "0.00 GB", "memorysize": 8174812, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": "255.255.255.0", "timezone": "CST", "hardwareisa": "x86_64", "memoryfree": 241164, "operatingsystem": "ubuntu", "kernelmajversion": "4.4", "kernelversion": "4.4.0", "macaddress": "00:16:3E:10:7E:5C", "operatingsystemrelease": "16.04", "ipaddress": "10.10.80.131", "hostname": "master", "uptime_hours": "1", "fqdn": "master.example.com", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "4068960", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "4068960"}, {"available": "24108320", "used": "14930060", "percent": "39%", "device": "/dev/vda1", "mountpoint": "/", "type": "ext4", "size": "41151808"}, {"available": "88747784", "used": "9072256", "percent": "10%", "device": "/dev/vdb1", "mountpoint": "/mnt/disk1", "type": "ext4", "size": "103079200"}], "hardwaremodel": "x86_64", "uptime_seconds": "7017", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.6.2.0", "agentEnv": {"transparentHugePage": "madvise", "hostHealth": {"agentTimeStampAtReporting": 1546576710951, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1546576710753, "hostname": "master.example.com", "responseId": -1, "publicHostname": "master.example.com"}\')
INFO 2019-01-04 12:38:30,953 NetUtil.py:70 - Connecting to https://master.example.com:8440/connection_info
INFO 2019-01-04 12:38:30,981 security.py:93 - SSL Connect being called.. connecting to the server
ERROR 2019-01-04 12:38:30,984 Controller.py:226 - Unable to connect to: https://master.example.com:8441/agent/v1/register/master.example.com
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 175, in registerWithServer
ret = self.sendRequest(self.registerUrl, data)
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 549, in sendRequest
raise IOError(\'Request to {0} failed due to {1}\'.format(url, str(exception)))
IOError: Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
ERROR 2019-01-04 12:38:30,984 Controller.py:227 - Error:Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
WARNING 2019-01-04 12:38:30,984 Controller.py:228 - Sleeping for 27 seconds and then trying again
', None)
('WARNING 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7f0e12cad490>; currently running: False
INFO 2019-01-04 12:38:30,685 hostname.py:106 - Read public hostname \'master.example.com\' using socket.getfqdn()
INFO 2019-01-04 12:38:30,686 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:30,737 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:30,748 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'master.example.com\' using socket.getfqdn().
INFO 2019-01-04 12:38:30,750 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources.
INFO 2019-01-04 12:38:30,752 Hardware.py:73 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'example.com\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.4.0-117-generic\', \'uptime_days\': \'0\', \'memorytotal\': 8174812, \'swapfree\': \'0.00 GB\', \'memorysize\': 8174812, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': \'255.255.255.0\', \'timezone\': \'CST\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 241164, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.4\', \'kernelversion\': \'4.4.0\', \'macaddress\': \'00:16:3E:10:7E:5C\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'10.10.80.131\', \'hostname\': \'master\', \'uptime_hours\': \'1\', \'fqdn\': \'master.example.com\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'4068960\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'4068960\'}, {\'available\': \'24108320\', \'used\': \'14930060\', \'percent\': \'39%\', \'device\': \'/dev/vda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'41151808\'}, {\'available\': \'88747784\', \'used\': \'9072256\', \'percent\': \'10%\', \'device\': \'/dev/vdb1\', \'mountpoint\': \'/mnt/disk1\', \'type\': \'ext4\', \'size\': \'103079200\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'7017\', \'interfaces\': \'eth0,lo\'}
INFO 2019-01-04 12:38:30,953 Controller.py:170 - Registering with master.example.com (10.10.80.131) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "example.com", "physicalprocessorcount": 4, "kernelrelease": "4.4.0-117-generic", "uptime_days": "0", "memorytotal": 8174812, "swapfree": "0.00 GB", "memorysize": 8174812, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": "255.255.255.0", "timezone": "CST", "hardwareisa": "x86_64", "memoryfree": 241164, "operatingsystem": "ubuntu", "kernelmajversion": "4.4", "kernelversion": "4.4.0", "macaddress": "00:16:3E:10:7E:5C", "operatingsystemrelease": "16.04", "ipaddress": "10.10.80.131", "hostname": "master", "uptime_hours": "1", "fqdn": "master.example.com", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "4068960", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "4068960"}, {"available": "24108320", "used": "14930060", "percent": "39%", "device": "/dev/vda1", "mountpoint": "/", "type": "ext4", "size": "41151808"}, {"available": "88747784", "used": "9072256", "percent": "10%", "device": "/dev/vdb1", "mountpoint": "/mnt/disk1", "type": "ext4", "size": "103079200"}], "hardwaremodel": "x86_64", "uptime_seconds": "7017", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.6.2.0", "agentEnv": {"transparentHugePage": "madvise", "hostHealth": {"agentTimeStampAtReporting": 1546576710951, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1546576710753, "hostname": "master.example.com", "responseId": -1, "publicHostname": "master.example.com"}\')
INFO 2019-01-04 12:38:30,953 NetUtil.py:70 - Connecting to https://master.example.com:8440/connection_info
INFO 2019-01-04 12:38:30,981 security.py:93 - SSL Connect being called.. connecting to the server
ERROR 2019-01-04 12:38:30,984 Controller.py:226 - Unable to connect to: https://master.example.com:8441/agent/v1/register/master.example.com
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 175, in registerWithServer
ret = self.sendRequest(self.registerUrl, data)
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 549, in sendRequest
raise IOError(\'Request to {0} failed due to {1}\'.format(url, str(exception)))
IOError: Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
ERROR 2019-01-04 12:38:30,984 Controller.py:227 - Error:Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
WARNING 2019-01-04 12:38:30,984 Controller.py:228 - Sleeping for 27 seconds and then trying again
', None)
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:31
Registering with the server...
Registration with the server failed.
==========================
Creating target directory...
==========================
Command start time 2019-01-04 12:38:12
chmod: cannot access '/var/lib/ambari-agent/data': No such file or directory
Warning: Permanently added 'node1.example.com,10.10.80.130' (ECDSA) to the list of known hosts.
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:12
==========================
Copying ambari sudo script...
==========================
Command start time 2019-01-04 12:38:12
scp /var/lib/ambari-server/ambari-sudo.sh
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying common functions script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_commons
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
scp /var/lib/ambari-server/create-python-wrap.sh
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying OS type check script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_server/os_check_type.py
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:14
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running OS type check...
==========================
Command start time 2019-01-04 12:38:14
Cluster primary/cluster OS family is ubuntu16 and local/current OS family is ubuntu16
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Checking 'sudo' package on remote host...
==========================
Command start time 2019-01-04 12:38:14
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Copying repo file to 'tmp' folder...
==========================
Command start time 2019-01-04 12:38:14
scp /etc/apt/sources.list.d/ambari.list
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Moving file to repo dir...
==========================
Command start time 2019-01-04 12:38:15
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Changing permissions for ambari.repo...
==========================
Command start time 2019-01-04 12:38:15
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Update apt cache of repository...
==========================
Command start time 2019-01-04 12:38:15
0% [Working]
Get:1 http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155 Ambari InRelease [7,394 B]
0% [1 InRelease 7,394 B/7,394 B 100%]
0% [Working]
Get:2 http://10.10.80.131:8000/HDP/ubuntu16/2.6.5.0-292 HDP InRelease [7,376 B]
Get:3 http://10.10.80.131:8000/HDP-GPL/ubuntu16/2.6.5.0-292 HDP-GPL InRelease [7,395 B]
Get:4 http://10.10.80.131:8000/HDP-UTILS/ubuntu16/1.1.0.22 HDP-UTILS InRelease [7,405 B]
0% [Working]
0% [1 InRelease gpgv 7,394 B]
0% [Working]
0% [2 InRelease gpgv 7,376 B]
0% [Working]
0% [3 InRelease gpgv 7,395 B]
0% [Working]
0% [4 InRelease gpgv 7,405 B]
100% [Working]
Fetched 29.6 kB in 0s (158 kB/s)
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 8%
Reading package lists... 19%
Reading package lists... 19%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... Done
W: http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155/dists/Ambari/InRelease: Signature by key DF52ED4F7A3A5882C0994C66B9733A7A07513CAD uses weak digest algorithm (SHA1)
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Copying setup script file...
==========================
Command start time 2019-01-04 12:38:16
scp /usr/lib/ambari-server/lib/ambari_server/setupAgent.py
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Running setup agent script...
==========================
Command start time 2019-01-04 12:38:16
dpkg-query: no packages found matching ambari-agent
("INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,808 DataCleaner.py:39 - Data cleanup thread started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:120 - Data cleanup started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:122 - Data cleanup finished
INFO 2019-01-04 12:38:25,809 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:25,834 PingPortListener.py:50 - Ping port listener started on port: 8670
INFO 2019-01-04 12:38:25,834 main.py:439 - Connecting to Ambari server at https://master.example.com:8440 (10.10.80.131)
INFO 2019-01-04 12:38:25,834 NetUtil.py:70 - Connecting to https://master.example.com:8440/ca
INFO 2019-01-04 12:38:26,256 main.py:449 - Connected to Ambari server master.example.com
INFO 2019-01-04 12:38:26,257 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads
WARNING 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7ff4bfa91490>; currently running: False
INFO 2019-01-04 12:38:28,265 hostname.py:106 - Read public hostname 'node1.example.com' using socket.getfqdn()
INFO 2019-01-04 12:38:28,266 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:28,310 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:28,319 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:28,321 Facter.py:202 - Directory: '/etc/resource_overrides' does not exist - it won't be used for gathering system resources.
INFO 2019-01-04 12:38:28,323 Hardware.py:73 - Host system information: {'kernel': 'Linux', 'domain': 'example.com', 'physicalprocessorcount': 4, 'kernelrelease': '4.4.0-117-generic', 'uptime_days': '0', 'memorytotal': 4046044, 'swapfree': '0.00 GB', 'memorysize': 4046044, 'osfamily': 'ubuntu', 'swapsize': '0.00 GB', 'processorcount': 4, 'netmask': '255.255.255.0', 'timezone': 'CST', 'hardwareisa': 'x86_64', 'memoryfree': 1011008, 'operatingsystem': 'ubuntu', 'kernelmajversion': '4.4', 'kernelversion': '4.4.0', 'macaddress': '00:16:3E:0E:F0:53', 'operatingsystemrelease': '16.04', 'ipaddress': '10.10.80.130', 'hostname': 'node1', 'uptime_hours': '1', 'fqdn': 'node1.example.com', 'id': 'root', 'architecture': 'x86_64', 'selinux': False, 'mounts': [{'available': '2004576', 'used': '0', 'percent': '0%', 'device': 'udev', 'mountpoint': '/dev', 'type': 'devtmpfs', 'size': '2004576'}, {'available': '35189292', 'used': '3849088', 'percent': '10%', 'device': '/dev/vda1', 'mountpoint': '/', 'type': 'ext4', 'size': '41151808'}, {'available': '97758996', 'used': '61044', 'percent': '1%', 'device': '/dev/vdb1', 'mountpoint': '/mnt/disk1', 'type': 'ext4', 'size': '103079200'}], 'hardwaremodel': 'x86_64', 'uptime_seconds': '7180', 'interfaces': 'eth0,lo'}
", None)
("INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,808 DataCleaner.py:39 - Data cleanup thread started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:120 - Data cleanup started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:122 - Data cleanup finished
INFO 2019-01-04 12:38:25,809 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:25,834 PingPortListener.py:50 - Ping port listener started on port: 8670
INFO 2019-01-04 12:38:25,834 main.py:439 - Connecting to Ambari server at https://master.example.com:8440 (10.10.80.131)
INFO 2019-01-04 12:38:25,834 NetUtil.py:70 - Connecting to https://master.example.com:8440/ca
INFO 2019-01-04 12:38:26,256 main.py:449 - Connected to Ambari server master.example.com
INFO 2019-01-04 12:38:26,257 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads
WARNING 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7ff4bfa91490>; currently running: False
INFO 2019-01-04 12:38:28,265 hostname.py:106 - Read public hostname 'node1.example.com' using socket.getfqdn()
INFO 2019-01-04 12:38:28,266 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:28,310 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:28,319 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:28,321 Facter.py:202 - Directory: '/etc/resource_overrides' does not exist - it won't be used for gathering system resources.
INFO 2019-01-04 12:38:28,323 Hardware.py:73 - Host system information: {'kernel': 'Linux', 'domain': 'example.com', 'physicalprocessorcount': 4, 'kernelrelease': '4.4.0-117-generic', 'uptime_days': '0', 'memorytotal': 4046044, 'swapfree': '0.00 GB', 'memorysize': 4046044, 'osfamily': 'ubuntu', 'swapsize': '0.00 GB', 'processorcount': 4, 'netmask': '255.255.255.0', 'timezone': 'CST', 'hardwareisa': 'x86_64', 'memoryfree': 1011008, 'operatingsystem': 'ubuntu', 'kernelmajversion': '4.4', 'kernelversion': '4.4.0', 'macaddress': '00:16:3E:0E:F0:53', 'operatingsystemrelease': '16.04', 'ipaddress': '10.10.80.130', 'hostname': 'node1', 'uptime_hours': '1', 'fqdn': 'node1.example.com', 'id': 'root', 'architecture': 'x86_64', 'selinux': False, 'mounts': [{'available': '2004576', 'used': '0', 'percent': '0%', 'device': 'udev', 'mountpoint': '/dev', 'type': 'devtmpfs', 'size': '2004576'}, {'available': '35189292', 'used': '3849088', 'percent': '10%', 'device': '/dev/vda1', 'mountpoint': '/', 'type': 'ext4', 'size': '41151808'}, {'available': '97758996', 'used': '61044', 'percent': '1%', 'device': '/dev/vdb1', 'mountpoint': '/mnt/disk1', 'type': 'ext4', 'size': '103079200'}], 'hardwaremodel': 'x86_64', 'uptime_seconds': '7180', 'interfaces': 'eth0,lo'}
", None)
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:28
Registering with the server...
Registration with the server failed.
[/code]
解決方案(谷歌比百度好用)
https://community.hortonworks.com/questions/97831/registration-to-ambari-server-via-the-ambari-agent.html
/etc/ambari-agent/conf修改ambari.ini文件
We have over come the problem by adding
following option to security section in ambari-agent.ini in all the hosts in the cluster:
[security] force_https_protocol=PROTOCOL_TLSv1_2