Hadoop搭建_01

時間 2019-12-05

標籤 hadoop 搭建欄目 Hadoop 简体版

原文原文鏈接

安裝vmware

安裝linux

配置linux

配置IP

192.168.1.100 masterjava

192.168.1.102 slave2linux

192.168.1.103 slave1sql

配置主機名

[root@localhost ~]# vim /etc/sysconfig/networkshell

NETWORKING=yesvim

NETWORKING_IPV6=nodom

HOSTNAME=masterssh

[root@localhost ~]# hostname masteride

三臺主機都配置主機名工具

[root@localhost ~]# vim /etc/sysconfig/networkoop

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=slave1

[root@localhost ~]# hostnameslave1

[root@localhost ~]# vim /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=slave2

[root@localhost ~]# hostnameslave2

重啓才能生效

Reboot

而後使用遠程鏈接工具，這裏使用xshell

關閉防火牆和selinux

[root@master ~]# vim /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

# enforcing - SELinux security policy is enforced.

# permissive - SELinux prints warnings instead of enforcing.

# disabled - SELinux is fully disabled.

SELINUX=disabled

# SELINUXTYPE= type of policy in use. Possible values are:

# targeted - Only targeted network daemons are protected.

# strict - Full SELinux protection.

SELINUXTYPE=targeted

[root@master ~]# serviceiptables status

表格：filter

Chain INPUT (policy ACCEPT)

num targetprot opt source destination

Chain FORWARD (policy ACCEPT)

num targetprot opt source destination

Chain OUTPUT (policy ACCEPT)

num targetprot opt source destination

域名解析

使得master和slave1 slave2 鏈接對方

三臺都要作

[root@master ~]# vim /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomainlocalhost

::1 localhost6.localdomain6 localhost6

192.168.7.100 master

192.168.7.101 slave1

192.168.7.102 slave2

配置java

啓動ftp

[root@master ~]# /etc/init.d/vsftpd restart

關閉vsftpd： [失敗]

爲vsftpd啓動vsftpd： [肯定]

默認狀況下 root不容許使用ftp

[root@mastervsftpd]# pwd

/etc/vsftpd

[root@mastervsftpd]# ls

ftpusersuser_list

將這兩個文件中的root註釋掉。

而後重啓ftp

在三個機器上建立文件夾

[root@master ~]# mkdir installer

[root@master ~]#

上傳jdk

E:\開發工具

jdk-6u27-linux-i586-rpm.bin

到installer目錄

[root@master installer]# ll

總計 78876

-rw-r--r-- 1 root root 80680219 12-01 12:50 jdk-6u27-linux-i586-rpm.bin

[root@master installer]# chmoda+x jdk-6u27-linux-i586-rpm.bin

[root@master installer]#

[root@master installer]# ./jdk-6u27-linux-i586-rpm.bin

Unpacking...

Checksumming...

Extracting...

UnZipSFX 5.50 of 17 February 2002, by Info-ZIP (Zip-Bugs@lists.wku.edu).

inflating: jdk-6u27-linux-i586.rpm

這時候能夠再開一個鏈接進行遠程拷貝

[root@master installer]# scp jdk-6u27-linux-i586-rpm.bin slave1:/root/installer

root@slave1's password:

jdk-6u27-linux-i586-rpm.bin

拷貝完在三個機器上都要安裝jdk。

[root@master installer]# java -version

java version "1.6.0_27"

添加用戶add

三臺機器都作

[root@master ~]# useraddhadoop

[root@master ~]# passwdhadoop

Changing password for user hadoop.

New UNIX password:

BAD PASSWORD: it is based on a dictionary word

Retype new UNIX password:

passwd: all authentication tokens updated successfully.

[root@master ~]#

而且在slave1和slave2使用hadoop用戶建立installers目錄

[root@slave1 ~]# su - hadoop

[hadoop@slave1 ~]$ mkdir installer

配置ssh等效性

[hadoop@master ~]# ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

5b:d6:30:95:6f:37:b1:d8:e0:ec:b4:cb:94:cc:3f:ccroot@master

[hadoop @master ~]#

一路回車

這一個動做在三臺機器上都執行

[hadoop@master ~]$ cd .ssh/

[hadoop@master .ssh]$ ls

id_rsa id_rsa.pub

[hadoop@master .ssh]$

[hadoop@master .ssh]$ cat id_rsa.pub >authorized_keys

[hadoop@master .ssh]$ ls

authorized_keysid_rsa id_rsa.pub

[hadoop@master .ssh]$

將生成的authorized_keys文件拷貝到slave1和slave2

[hadoop@master .ssh]$ scpauthorized_keys slave1:~/.ssh/

The authenticity of host 'slave1 (192.168.1.103)' can't be established.

RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'slave1,192.168.1.103' (RSA) to the list of known hosts.

hadoop@slave1's password:

authorized_keys 100% 395 0.4KB/s 00:00

[hadoop@master .ssh]$

而後進入slave1

[hadoop@slave1 .ssh]$ cat id_rsa.pub >>authorized_keys

[hadoop@slave1 .ssh]$ catauthorized_keys

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw8taarZ+/ndWV04MqGsnT5cKcYs5LqMmtocWSsIxfUttYpMjwcgktjEPSByb/SFPE3alx0/Te7bjG8nFu2HHV4v++2jNfraqoBjIrO3/ITzHOSGduYmM4xbvBcXCAX5BSawwbpKn8RifPM5M1ZbExFhdZ0njsYSBlq6ZAMV+2F77enfwCI6jB/WhtfClj4QpWuMTQ8O/gqaMbM0OMrIuY84ssoYfDSpl2uUtGBBGY3cyyTDEbQukRH5doapSNPwZQs6lJSVIO7JWLGMfOQbvsqlS0r1nly57I1b7hAMZcGdVWZy2CGclQX3s8a7vjpJ8+iTFtwiAdydFsP+aQ9ldUw== hadoop@master

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAqhiMNhNlBZ1+aC+tU9O8HKTd7lSMmqhi7FcBKue/q/H37hyMp+PqS/BVYStvEhtHzcy+1/SJWKqSV0ut1Qh8zUo42w81KW/g1xCt5fAJLe61/XtC2WyTrwfVQbFVXCPTpAarYJTlgy+ZgarD8Qg4hS642dmXKbSUQf/Mjbxd7PpcAZx1GCVOX3wck+7LIQJuLInlAFIXhyP0rq+I80CX9u40utkgJQd6ZVvsqJdnB+eeXr08w16GEOSY8ER2Vksbw69PGJjjKz1eMFpCUNatlf3bgmLp+JBOnlbgEizc21ogwcnyTXKCP9j3ZHTO2pDxAaHJ2hYJnOjr2+GSALzeOw== hadoop@slave1

[hadoop@slave1 .ssh]$

而後在slave1 傳輸到slave2

[hadoop@slave1 .ssh]$ scpauthorized_keys slave2:~/.ssh

The authenticity of host 'slave2 (192.168.1.102)' can't be established.

RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'slave2,192.168.1.102' (RSA) to the list of known hosts.

hadoop@slave2's password:

authorized_keys 100% 790 0.8KB/s 00:00

[hadoop@slave1 .ssh]$

到slave2上

[hadoop@slave2 .ssh]$ cat id_rsa.pub >>authorized_keys

[hadoop@slave2 .ssh]$ catauthorized_keys

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzyFZKYRXh1HIm+p//kh/P268u6CHQJ88M+vEcb0fEjpXhNoDaVDceuYhQZxc0E/3dJRd86jaRNWnV+G+IPN00ykV2+UJhE2yjsdMa+Yqwy6XU14H25lMaImJGtxpoXO+3kWKJZ1uGB0E2TU2nS+Epb8EI+6ezZ0ilQhgwpc0kQR/jN6d6hUKKK5yTxKZg4agn4QsOZhyBNQZX7tLofHELR970T5n7to19UejB1j09AVdME+TYf7q3reLYHtVA1NsD7+wQcPB3WOKCRhHU5Uas+Rd3ukIP2/H8h13mJ5NHhq5FzxdVa62OPw9BKZVVO2vXp7SvxJG0MW0Aw8fO+AuRQ== hadoop@slave2

[hadoop@slave2 .ssh]$

而後將這個文件傳回slave1和master

[hadoop@slave2 .ssh]$ scpauthorized_keys master:~/.ssh/

The authenticity of host 'master (192.168.1.100)' can't be established.

RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'master,192.168.1.100' (RSA) to the list of known hosts.

hadoop@master's password:

authorized_keys 100% 1185 1.2KB/s 00:00

[hadoop@slave2 .ssh]$

[hadoop@slave2 .ssh]$ scpauthorized_keys slave1:~/.ssh/

The authenticity of host 'slave1 (192.168.1.103)' can't be established.