hadoop項目使用

該引用適合hadoop使用1.2.1，提供參考

 

maven安裝好以後便可使用

 

 D:\workspace\java>mvn archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId=org.conan.myhadoop.mr
-DartifactId=myHadoop -DpackageName=org.conan.myhadoop.mr -Dversion=1.0-SNAPSHOT -DinteractiveMode=false
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> maven-archetype-plugin:2.2:generate (default-cli) @ standalone-pom >>>
[INFO]
[INFO] <<< maven-archetype-plugin:2.2:generate (default-cli) @ standalone-pom <<<
[INFO]

.......

期間會出現須要填寫的groupId，便是你的項目名稱，其餘輸入直接回車便可。最後點Y

 

成功以後：

D:\workspace\java>cd myHadoop
~ D:\workspace\java\myHadoop>mvn clean install
此時在你的路徑下就用你敲的groupId命名的項目，導入到eclipse中便可

 

最後編輯pom.xml  追加

<dependency>
   <groupId>org.apache.hadoop</groupId>
   <artifactId>hadoop-core</artifactId>
   <version>1.0.3</version>
  </dependency>

最後執行命令下載依賴mvn clean install  

 

此時你能夠直接運行main方法了， 針對這個注意期間出現的問題

嚴重: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator1702422322\.staging to 0700
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp

修改下源碼文件FileUtil.java從新編譯便可

 

運行時可能還會出現

嚴重: PriviledgedActionException as:xuanxy cause:org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
Exception in thread "main" org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
 at org.apache.hadoop.ipc.Client.call(Client.java:1070)
 at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
 at com.sun.proxy.$Proxy1.getProtocolVersion(Unknown Source)
 at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
 at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)

 

注意hadoop的版本