/** * 上傳代碼 */ @Test public void putFileToHDFS() throws URISyntaxException, IOException, InterruptedException { //1.建立配置信息 //F2能夠快速的定位錯誤 Configuration conf = new Configuration(); //2.設置部分參數 conf.set("dfs.replication","2"); //3.找到HDFS的地址 FileSystem fileSystem = FileSystem.get(new URI("hdfs://bigdata111:9000"), conf, "root"); //4.上傳本地Windows文件的路徑 Path src = new Path("D:\\data\\demo1.txt"); //Path src = new Path("D:\\tools\\null\\null.sys"); //Path src = new Path("D:\\tools\\Linux所用Jar包\\hadoop-2.7.2.rar"); //5.要上傳到HDFS的路徑 Path dst = new Path("hdfs://bigdata111:9000/"); //6.以拷貝的方式上傳從src到dst fileSystem.copyFromLocalFile(src,dst); //7.關閉 fileSystem.close(); System.out.print("Ok,上傳完成了"); }
@Test public void getFileFromHDFS() throws URISyntaxException, IOException, InterruptedException { //1.獲取配置 Configuration conf = new Configuration(); //2.獲取hdfs鏈接 FileSystem fs = FileSystem.get(new URI("hdfs://bigdata111:9000/"),conf,"root"); //3.文件下載 // Path dst = new Path("D:\\tools\\demoFile\\null.sys"); // Path src = new Path("hdfs://bigdata111:9000/null.sys"); Path dst = new Path("D:\\tools\\demoFile\\hadoop-2.7.2.rar"); Path src = new Path("hdfs://bigdata111:9000/hadoop-2.7.2.rar"); fs.copyToLocalFile(src,dst); fs.close(); System.out.print("OK,下載完成"); }
上傳文件到HDFS的時候報鏈接超時java
解決辦法,添加以下代碼:oop
fileSystem.setTimes(dst,0,3000);
下載的時候,下載對應的目錄回多餘crc文件(目前還不知道緣由,求大神指點):spa