搭建hadoop

1.下載hadoop

2.增長帳戶
sudo adduser --system --no-create-home --disabled-login --disabled-password --group hadoop
添加到管理員組
sudo usermod -aG root hadoop

3.安裝ssh
sudo apt-get install openssh-server #安裝
sudo /etc/init.d/ssh start #啓動
ps -e | grep ssh #測試
ssh-keygen -t rsa -P "" #生成祕鑰
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys #導入公鑰
ssh localhost #登入測試
exit #登出
ssh localhost #再次登入

4.安裝hadoop
cd /opt/hadoop/
tar -zxvf hadoop-1.2.1.tar.gz
受權:sudo chown -R hadoop:hadoop /opt/hadoop/hadoop-1.2.1
設定hadoop-env.sh(Java 安裝路徑):
進入hadoop目錄，打開conf目錄下到hadoop-env.sh，添加如下信息：
export JAVA_HOME=/opt/java/jdk1.6.0_29
export HADOOP_HOME=/opt/hadoop/hadoop-1.2.1
export PATH=$PATH:/opt/hadoop/hadoop-1.2.1/bin
export HADOOP_HOME_WARN_SUPPRESS="TRUE"
source /opt/hadoop/hadoop-1.2.1/conf/hadoop-env.sh #更新配置
hadoop version #測試

5.#####################僞分佈模式配置#####################
cd /opt/hadoop/hadoop-1.2.1
mkdir tmp
mkdir -p hdfs/name
mkdir -p hdfs/data
sudo chmod g-w /opt/hadoop/hadoop-1.2.1/hdfs/data #sudo chmod 775 /opt/hadoop/hadoop-1.2.1/hdfs/data
gedit /opt/hadoop/hadoop-1.2.1/conf/core-site.xml:
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop/hadoop-1.2.1/tmp</value>
</property>
</configuration>
gedit /opt/hadoop/hadoop-1.2.1/conf/hdfs-site.xml:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/opt/hadoop/hadoop-1.2.1/hdfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/opt/hadoop/hadoop-1.2.1/hdfs/data</value>
</property>
</configuration>
gedit /opt/hadoop/hadoop-1.2.1/conf/mapred-site.xml:
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
6.格式化HDFS
source /opt/hadoop/hadoop-1.2.1/conf/hadoop-env.sh
hadoop namenode -format
7.啓動Hadoop
cd /opt/hadoop/hadoop-1.2.1/bin
start-all.sh #啓動全部服務(namenode,datanode) 中止stop-all.sh
8.驗證安裝成功
jps
9.檢查運行狀態
經過下面的操做來查看服務是否正常，在Hadoop中用於監控集羣健康狀態的Web界面：
http://localhost:50030/ #Hadoop 管理介面
http://localhost:50060/ #Hadoop Task Tracker 狀態
http://localhost:50070/ #Hadoop DFS 狀態
10.僞分佈模式下運行Hadoop自帶的wordcount: /opt/hadoop/hadoop-1.2.1$ ./bin/start-all.sh
/home/works$ echo "welcome to hadoop world" >test1.txt
                     echo "big data is popular" >test2.txt
/opt/hadoop/hadoop-1.2.1$ ./bin/hadoop dfs -put /home/works/ ./tmp/test
                                         ./bin/hadoop dfs -ls ./tmp/test/*
                                         [http://localhost.localdomain:50075/browseDirectory.jsp?dir=%2Fuser%2Ftmp%2Ftest&namenodeInfoPort=50070]                                          
/opt/hadoop/hadoop-1.2.1$ ./bin/hadoop jar hadoop-examples-1.2.1.jar wordcount /user/tmp/test /user/tmp/out
                                         ./bin/hadoop dfs -ls
參考:
升級成2.2.0 http://bigdatahandler.com/hadoop-hdfs/installing-single-node-hadoop-2-2-0-on-ubuntu/
                http://www.iteblog.com/archives/856
http://blog.csdn.net/hitwengqi/article/details/8008203
http://www.cnblogs.com/welbeckxu/category/346329.html