Mac搭建Hadoop源碼閱讀環境

一、本次Hadoop源碼閱讀環境使用的閱讀工具是idea，Hadoop版本是2.7.3。須要安裝的工具包括idea、jdk、maven、protobuf等

二、jdk，使用的版本是1.8版，在jdk官網下載jdk-8u111-macosx-x64.dmg，點擊安裝，一路next。

三、idea安裝，略

四、maven，使用的版本是3.3.9,下載apache-maven-3.3.9-bin.tar,解壓：

tar -zxvf  apache-maven-3.3.9-bin.tar

進入 Maven 安裝根目錄 cd apache-maven-3.3.9

進入 Maven配置文件目錄 cd conf

編輯settings.xml文件 vi  settings.xml

設置Maven本地庫的路徑：這個路徑要建立好。

 <localRepository>/opt/PRG/apache-maven-3.3.9/repository</localRepository>

同時在加入阿里雲的Maven庫，參考：maven使用阿里雲倉庫

<mirror>
          <id>nexus-aliyun</id>
          <mirrorOf>*</mirrorOf>
          <name>Nexus aliyun</name>
          <url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>

設置Maven環境變量：

vi ~/.bash_profile

export MAVEN_HOME=/opt/PRG/apache-maven-3.3.9
export PATH=${PATH}:${MAVEN_HOME}/bin:
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m"
export PATH

更新環境變量 source ~/.bash_profile

mvn -version 能看到相應的版本，則Maven安裝成功。

五、安裝protobuf，下載protobuf-2.5.0.tar.gz，下載地址： http://pan.baidu.com/s/1pJlZubT （網友提供）

解壓：tar -zxvf protobuf-2.5.0.tar.gz

cd protobuf-2.5.0

依次執行，參考：http://blog.sina.com.cn/s/blog_7d1531ed0101fmc5.html

 1 ./configure
 2 make
 3 make check
 4 make install

若是報沒有權限的錯誤，su root 切換到root執行。

最後輸入  protoc --version命令，如顯示libprotoc 2.5.0則安裝成功

 六、編譯Hadoop源碼

在Hadoop官網下載hadoop-2.7.3-src.tar.gz源碼

解壓：tar -zxvf  hadoop-2.7.3-src.tar.gz

cd hadoop-2.7.3-src

執行：

mvn package -Pdist -DskipTests -Dtar

最後是漫長的編譯過程。。。

下載依賴jar和相關文件。。。

最後輸出：[INFO] Reactor Summary：

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop Main ................................. SUCCESS [  5.429 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [  2.148 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  1.061 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  3.297 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.200 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  1.627 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  3.751 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  4.644 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [  6.750 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  3.126 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [01:10 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [  5.191 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [01:12 min]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.037 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [01:38 min]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [ 34.932 s]
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [ 13.482 s]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  3.823 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.039 s]
[INFO] hadoop-yarn ........................................ SUCCESS [  0.041 s]
[INFO] hadoop-yarn-api .................................... SUCCESS [ 28.981 s]
[INFO] hadoop-yarn-common ................................. SUCCESS [ 22.892 s]
[INFO] hadoop-yarn-server ................................. SUCCESS [  0.028 s]
[INFO] hadoop-yarn-server-common .......................... SUCCESS [  7.887 s]
[INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [  9.785 s]
[INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [  2.782 s]
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [  5.300 s]
[INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 14.558 s]
[INFO] hadoop-yarn-server-tests ........................... SUCCESS [  4.048 s]
[INFO] hadoop-yarn-client ................................. SUCCESS [  4.785 s]
[INFO] hadoop-yarn-server-sharedcachemanager .............. SUCCESS [  3.260 s]
[INFO] hadoop-yarn-applications ........................... SUCCESS [  0.024 s]
[INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [  2.129 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [  1.734 s]
[INFO] hadoop-yarn-site ................................... SUCCESS [  0.026 s]
[INFO] hadoop-yarn-registry ............................... SUCCESS [  4.279 s]
[INFO] hadoop-yarn-project ................................ SUCCESS [  3.812 s]
[INFO] hadoop-mapreduce-client ............................ SUCCESS [  0.124 s]
[INFO] hadoop-mapreduce-client-core ....................... SUCCESS [ 14.922 s]
[INFO] hadoop-mapreduce-client-common ..................... SUCCESS [ 11.181 s]
[INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [  2.904 s]
[INFO] hadoop-mapreduce-client-app ........................ SUCCESS [  7.072 s]
[INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [  4.399 s]
[INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [  4.986 s]
[INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [  1.767 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [  4.552 s]
[INFO] hadoop-mapreduce ................................... SUCCESS [  2.718 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [  5.164 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [  8.356 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [  2.022 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [  4.104 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  3.251 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [  1.897 s]
[INFO] Apache Hadoop Ant Tasks ............................ SUCCESS [  1.855 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [  2.406 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [  0.023 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [  3.771 s]
[INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [ 17.460 s]
[INFO] Apache Hadoop Azure support ........................ SUCCESS [  4.841 s]
[INFO] Apache Hadoop Client ............................... SUCCESS [  6.743 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  1.029 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [  4.379 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [  6.696 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.022 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [ 29.114 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:08 min
[INFO] Finished at: 2017-01-02T11:41:11+08:00
[INFO] Final Memory: 229M/959M
[INFO] ------------------------------------------------------------------------

沒有報錯，則說明編譯成功。

七、把編譯好的Hadoop源碼導入Idea

源碼路徑

找到編譯好的Hadoop，選擇便可，next。

選擇Maven工程 

 

以後一路next便可，以後就能夠開始閱讀源碼了。