apache的hadoop升級到CDH hadoop2.0時遇到的問題及解決

1：引入的jar包

1.X版本有hadoop-core包；而2.x沒有

若是你須要hdfs就引入\share\hadoop\common\lib + 

hadoop-common-2.0.0-cdh4.6.0.jar +

hadoop-hdfs-2.0.0-cdh4.6.0.jar +

\share\hadoop\yarn\*

 

若是你要MR則加上

\share\hadoop\mapreduce1\* 或

\share\hadoop\mapreduce2\* 

 

2：遇到權限問題的時候最好是把權限賦給當前的執行用戶

chown hadoop.hadoop ./logs

 

3：No FileSystem for scheme: hdfs,No FileSystem for scheme: file

http://blog.newitfarmer.com/big_data/big-data-platform/hadoop/13953/repost-no-filesystem-for-scheme-hdfsno-filesystem-for-scheme-file

個人辦法是加上

Configuration config = new Configuration();config.set("fs.hdfs.impl",org.apache.hadoop.hdfs.DistributedFileSystem.class.getName()); config.set("fs.file.impl",org.apache.hadoop.fs.LocalFileSystem.class.getName());