Hadoop0.21.0部署安裝以及mapreduce測試
鑑於hadoop的須要。。。可是並不限於此。。。有時候閒輸入密碼麻煩，也能夠用這種辦法從一個節點經過ssh進入另外一個節點。。。java
設要使master進入slave是免密碼的，則能夠在master（ip爲192.168.169.9）中以下操做：node
命令：ssh-keygen -t rsa 而後一路回車（該命令不須要進入特定目錄）python
cd進入/root/.ssh/能夠看見以下（橙色是新生成的）linux
id_rsa id_rsa.pub known_hostsweb
而後用scp命令將id_rsa遠程傳輸到slave（ip爲192.168.169.10）節點中：scp id_rsa.pub 192.168.169.10:/root/.ssh/192.168.169.9apache
則能夠在slave的/root/.ssh/目錄下看到名爲192.168.169.9的文件：bash
| 主機名app |
IP 框架 |
功能 dom |
| namenode |
192.168.169.9 |
NameNode,jobtraker |
| datanode1 |
192.168.169.10 |
DataNode,tasktraker |
| datanode2 |
192.168.169.20 |
DataNode,tasktraker |
同時在三臺機子的/etc/hosts中添加如下內容以修改機器名：
192.168.169.9 namenode
192.168.169.10 datanode1
192.168.169.20 datanode2
修改後注意解析測驗下：（datanode測驗同理）
-bash-3.1# ping -c 4 namenode
PING namenode (192.168.169.9) 56(84) bytes of data.
64 bytes from namenode (192.168.169.9): icmp_seq=1 ttl=64 time=0.020 ms
64 bytes from namenode (192.168.169.9): icmp_seq=2 ttl=64 time=0.009 ms
64 bytes from namenode (192.168.169.9): icmp_seq=3 ttl=64 time=0.009 ms
64 bytes from namenode (192.168.169.9): icmp_seq=4 ttl=64 time=0.010 ms
--- namenode ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2997ms
rtt min/avg/max/mdev = 0.009/0.012/0.020/0.004 ms
而後修改/etc/sysconfig/network中的域名（由於必需要域名和機器名同樣）如圖（這裏只列出namenode的，datanode的相似，只修改HOSTNAME）
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=namenode
2.ssh免密碼認證
具體作法上一篇博文已經講過。。這裏再也不贅述，可是要注意，這裏須要作到的是：
- master到slave的免密碼認證；
- slave到master的免密碼認證；
- master到master的免密碼認證。
例如我對第一條測驗一下：
-bash-3.1# ssh datanode1
Last login: Fri Feb 17 08:32:34 2012 from namenode
-bash-3.1# exit
logout
Connection to datanode1 closed.
-bash-3.1# ssh datanode2
Last login: Fri Feb 17 08:32:42 2012 from namenode
-bash-3.1# exit
logout
Connection to datanode2 closed.
3.安裝JDK並配置環境變量，由於個人已經安裝過了，因此我只須要配置環境變量了，修改/etc/profile內容以下：（注意修改後要source /etc/profile）
export JAVA_HOME=/usr/java/jdk1.6.0_29
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:/usr/java/jdk1.6.0_29/bin
export CLASSPATH=:/usr/java/jdk1.6.0_29/lib:/usr/java/jdk1.6.0_29/jre/lib
查看下版本：
-bash-3.1# java -version
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02, mixed mode)
寫個具體例子跑一下（test.java）：
class test
{
public static void main(String[] args)
{
System.out.println("Hello,World!");
}
}
測試一下：
-bash-3.1# javac test.java
-bash-3.1# java test
Hello,World!
說明成功了。。。。。
4.Hadoop安裝配置
下載hadoop-0.21.0.tar.gz壓縮包下來以後解壓縮到必定目錄，而後把它放到個固定的地方，我把它放在了/usr/local/hadoop目錄下，進去看下
-bash-3.1# ll
drwxrwxr-x 5 huyajun huyajun 4096 2010-08-17 c++
drwxr-xr-x 8 huyajun huyajun 4096 2010-08-17 common
drwxrwxr-x 2 huyajun huyajun 4096 02-16 15:54 conf
-rw-rw-r-- 1 huyajun huyajun 1289953 2010-08-17 hadoop-common-0.21.0.jar
-rw-rw-r-- 1 huyajun huyajun 622276 2010-08-17 hadoop-common-test-0.21.0.jar
-rw-rw-r-- 1 huyajun huyajun 934881 2010-08-17 hadoop-hdfs-0.21.0.jar
-rw-rw-r-- 1 huyajun huyajun 613332 2010-08-17 hadoop-hdfs-0.21.0-sources.jar
-rw-rw-r-- 1 huyajun huyajun 6956 2010-08-17 hadoop-hdfs-ant-0.21.0.jar
-rw-rw-r-- 1 huyajun huyajun 688026 2010-08-17 hadoop-hdfs-test-0.21.0.jar
-rw-rw-r-- 1 huyajun huyajun 419671 2010-08-17 hadoop-hdfs-test-0.21.0-sources.jar
-rw-rw