hadoop2.7ubuntu僞分佈式搭建

一.安裝Java

下載java的壓縮包，解壓

sudo tar -zxvf 8.tar.gz

配置java環境變量

sudo vim ~/.bashrc

export JAVA_HOME=/usr/local/java-8-openjdk-amd64
export JRE_HOME=/usr/local/java-8-openjdk-amd64/jre
export CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

source ~/.bashrc

二.ssh免密碼登陸

先判斷是否安裝ssh，執行ssh localhost，若須要輸入密碼，則已經安裝

未安裝，執行如下命令安裝ssh

sudo apt-get install openssh-server

登陸ssh localhost須要輸入密碼

退出exit

查看/home/hadoop家目錄下的/.ssh，沒有就建立

ls
cd  #進入用戶目錄
mkdir ./.ssh
cd ./.ssh
ssh-keygen -t rsa
cat ./id_rsa.pub >> ./authorized_keys  #將生成的公鑰加入到key中
ssh -version  #驗證是否安裝成功
ssh localhost  #不須要輸入密碼便可登陸
exit  #登出

三。添加IP映射

爲了方便，咱們在hosts添加IP地址映射

sudo vim /etc/hosts

127.0.0.1 localhost
192.168.153.133  hadoop
#IP地址           主機名

四.安裝hadoop

1.將壓縮包放到/usr/local/目錄下，解壓hadoop

cd /usr/local
sudo tar -zxvf hadoop-2.7.5.tar.gz  #解壓
sudo mv hadoop-2.7.5 hadoop  #重命名

2.賦予用戶權限

sudo chown -R hadoop ./hadoop

3.查看hadoop版本

cd ./hadoop 
./bin/hadoop version

4.添加hadoop環境變量

sudo vim /etc/profile

export HADOOP_HOME=/usr/local/hadoop
export PATH=$HADOOP_HOME/bin:$PATH

使環境變量生效

source /etc/profile

5.修改hadoop配置文件

cd /usr/local/hadoop/etc/hadoop
sudo cp mapred-site.xml.template mapred-site.xml
sudo vim core-site.xml

core-site.xml

sudo vim core-site.xml

<configuration>

    <property>
    <name>hadoop.tmp.dir</name>
    <value>/home/hadoop/hadoop/tmp</value> <!--設置臨時文件夾，只要在home下便可  -->
    </property>
    <property>
    <name>fs.defaultFS</name>
    <value>hdfs://hadoop:9000</value><!-- 修改成本機IP地址  -->
    </property>

​</configuration>
​

 hdfs-site.xml

sudo vim hdfs-site.xml

<configuration>

        <property>
             <name>dfs.replication</name>
             <value>1</value>
        </property>
        <property>
             <name>dfs.namenode.name.dir</name>
             <value>/home/hadoop/hadoop/dfs/name</value>
        </property>
        <property>
             <name>dfs.datanode.data.dir</name>
             <value>/home/hadoop/hadoop/dfs/data</value>
        </property>
        <property>
            <name>dfs.permissions</name>
           <value>false</value>
        </property>

</configuration>

mapred-site.xml

sudo vim mapred-site.xml

<configuration>

<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

</configuration>

​

yarn-site.xml

sudo vim yarn-site.xml

<configuration>

    <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>hadoop</value>
    </property>
    <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
    </property>

​</configuration>
​

slaves中添加主機名或者IP地址

sudo vim slaves

hadoop

hadoop-env.sh 添加Javahome路徑

sudo vim hadoop-env.sh

export JAVA_HOME=/usr/local/java-8-openjdk-amd64
​

 

6.格式化namenode

cd /usr/lcoal/hadoop/bin
./hadoop namenode -format

7. 啓動hadoop

cd /usr/local/hadoop/sbin
./start-all.sh

8. 查看進程jps

5、hadoop網址

http://hadoop:8088   8088端口查看yarn運行狀況

http://hadoop:50070    50070查看HDFS狀況