Hadoop部署方式-本地模式(Local (Standalone) Mode)

　　　　　　　　　　　　Hadoop部署方式-本地模式(Local (Standalone) Mode)

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　做者：尹正傑

版權聲明：原創做品，謝絕轉載！不然將追究法律責任。

 

 

　　Hadoop總共有三種運行方式。本地模式（Local (Standalone) Mode），僞分佈式（Pseudo-Distributed Mode），分佈式（Fully-Distributed Mode）。本篇博客主要講解第一種運行方式，即本地運行方式，想要查看僞分佈式的小夥伴請參考：http://www.javashuo.com/article/p-ugidapgo-cy.html，想要查看徹底分佈式的小夥伴請參考：http://www.javashuo.com/article/p-uxdqgqbb-dh.html。

　　須要注意的是，本地運行模式，實際上就是用Linux操做系統，會直接把文件存儲在Linux路徑上，咱們在上傳文件的時候要注意權限喲，好比你在搭建好Hadoop本地模式後，直接就提交一個文件到根下的話，可能會報權限錯誤喲，可是，若是你是root用戶的話就另當別論了。

 

 

一.下載相應的jdk和Hadoop安裝包

　　  jdk：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

　　Hadoop：http://hadoop.apache.org/releases.html

　　注意，Hadoop官方雖然沒有windows版本，可是可用經過「visual studio」編譯工具將Linux的運行僞分佈式遷移到windows中，建議安裝2015版本，網上的教程有不少。遺憾的是，這個工具的使用並非本篇博客的重點。

 

二.安裝Java環境

1>.建立軟件解壓目錄並受權給普通用戶

[yinzhengjie@yinzhengjie ~]$ ll
total 390268
-rw-rw-r--. 1 yinzhengjie yinzhengjie 214092195 Aug 26 2016 hadoop-2.7.3.tar.gz
-rw-rw-r--. 1 yinzhengjie yinzhengjie 185540433 May 17 2017 jdk-8u131-linux-x64.tar.gz
[yinzhengjie@yinzhengjie ~]$ sudo mkdir /soft
[sudo] password for yinzhengjie: 
[yinzhengjie@yinzhengjie ~]$ sudo chown yinzhengjie:yinzhengjie /soft/
[yinzhengjie@yinzhengjie ~]$ ll /soft/ -d
drwxr-xr-x. 2 yinzhengjie yinzhengjie 6 May 24 02:30 /soft/
[yinzhengjie@yinzhengjie ~]$ 

2>.解壓jdk並配置軟鏈接路徑

[yinzhengjie@yinzhengjie ~]$ tar zxf jdk-8u131-linux-x64.tar.gz -C /soft/
[yinzhengjie@yinzhengjie ~]$ cd /soft/
[yinzhengjie@yinzhengjie soft]$ ll
total 4
drwxr-xr-x. 8 yinzhengjie yinzhengjie 4096 Mar 15 2017 jdk1.8.0_131
[yinzhengjie@yinzhengjie soft]$ ln -s jdk1.8.0_131/ jdk
[yinzhengjie@yinzhengjie soft]$ ll
total 4
lrwxrwxrwx. 1 yinzhengjie yinzhengjie 13 May 24 02:33 jdk -> jdk1.8.0_131/
drwxr-xr-x. 8 yinzhengjie yinzhengjie 4096 Mar 15 2017 jdk1.8.0_131
[yinzhengjie@yinzhengjie soft]$ 

3>.設置Java環境變量

[yinzhengjie@yinzhengjie soft]$ sudo vi /etc/profile
[yinzhengjie@yinzhengjie soft]$ tail -3 /etc/profile
#Add by yinzhengjie
JAVA_HOME=/soft/jdk/
PATH=$PATH:$JAVA_HOME/bin
[yinzhengjie@yinzhengjie soft]$ . /etc/profile
[yinzhengjie@yinzhengjie soft]$ 
[yinzhengjie@yinzhengjie soft]$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[yinzhengjie@yinzhengjie soft]$ 

 

 

三.安裝Hadoop

1>.解壓jdk並配置軟鏈接路徑

[yinzhengjie@yinzhengjie ~]$ ll
total 390268
-rw-rw-r--. 1 yinzhengjie yinzhengjie 214092195 Aug 26 2016 hadoop-2.7.3.tar.gz
-rw-rw-r--. 1 yinzhengjie yinzhengjie 185540433 May 17 2017 jdk-8u131-linux-x64.tar.gz
[yinzhengjie@yinzhengjie ~]$ tar zxf hadoop-2.7.3.tar.gz -C /soft/
[yinzhengjie@yinzhengjie ~]$ ln -s /soft/hadoop-2.7.3/ /soft/hadoop
[yinzhengjie@yinzhengjie ~]$ ll /soft/
total 8
lrwxrwxrwx. 1 yinzhengjie yinzhengjie 19 May 24 02:40 hadoop -> /soft/hadoop-2.7.3/
drwxr-xr-x. 9 yinzhengjie yinzhengjie 4096 Aug 17 2016 hadoop-2.7.3
lrwxrwxrwx. 1 yinzhengjie yinzhengjie 13 May 24 02:33 jdk -> jdk1.8.0_131/
drwxr-xr-x. 8 yinzhengjie yinzhengjie 4096 Mar 15 2017 jdk1.8.0_131
[yinzhengjie@yinzhengjie ~]$ 

2>.設置Hadoop的環境變量

[yinzhengjie@yinzhengjie ~]$ sudo vi /etc/profile
[sudo] password for yinzhengjie: 
[yinzhengjie@yinzhengjie ~]$ tail -7 /etc/profile
#Add by yinzhengjie
JAVA_HOME=/soft/jdk/
PATH=$PATH:$JAVA_HOME/bin

#Add HADOOP_HOME
HADOOP_HOME=/soft/hadoop/
PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
[yinzhengjie@yinzhengjie ~]$ 
[yinzhengjie@yinzhengjie ~]$ source /etc/profile
[yinzhengjie@yinzhengjie ~]$ grep JAVA_HOME /soft/hadoop/etc/hadoop/hadoop-env.sh | grep -v ^#
export JAVA_HOME=/soft/jdk/
[yinzhengjie@yinzhengjie ~]$ 

3>.驗證是否安裝完畢（注意，提交的目錄當前用戶須要有權限，由於本地部署不須要啓動服務，它用的就是Linux操做系統，若是普通用戶把文件直接提交到根的話確定會報異常的喲！）

[yinzhengjie@yinzhengjie ~]$ ll
total 599344
-rw-rw-r--. 1 yinzhengjie yinzhengjie 214092195 Aug 26 2016 hadoop-2.7.3.tar.gz
-rw-rw-r--. 1 yinzhengjie yinzhengjie 185540433 May 17 2017 jdk-8u131-linux-x64.tar.gz
-rw-r--r--. 1 yinzhengjie yinzhengjie 214092195 May 24 03:24 test
[yinzhengjie@yinzhengjie ~]$ rm -rf test 
[yinzhengjie@yinzhengjie ~]$ 
[yinzhengjie@yinzhengjie ~]$ 
[yinzhengjie@yinzhengjie ~]$ ll
total 390268
-rw-rw-r--. 1 yinzhengjie yinzhengjie 214092195 Aug 26 2016 hadoop-2.7.3.tar.gz
-rw-rw-r--. 1 yinzhengjie yinzhengjie 185540433 May 17 2017 jdk-8u131-linux-x64.tar.gz
[yinzhengjie@yinzhengjie ~]$ 
[yinzhengjie@yinzhengjie ~]$ hdfs dfs -put hadoop-2.7.3.tar.gz /home/yinzhengjie/test
[yinzhengjie@yinzhengjie ~]$ tar zxf test 
[yinzhengjie@yinzhengjie ~]$ ll
total 599348
drwxr-xr-x. 9 yinzhengjie yinzhengjie 4096 Aug 17 2016 hadoop-2.7.3
-rw-rw-r--. 1 yinzhengjie yinzhengjie 214092195 Aug 26 2016 hadoop-2.7.3.tar.gz
-rw-rw-r--. 1 yinzhengjie yinzhengjie 185540433 May 17 2017 jdk-8u131-linux-x64.tar.gz
-rw-r--r--. 1 yinzhengjie yinzhengjie 214092195 May 24 03:24 test
[yinzhengjie@yinzhengjie ~]$