Java IO與NIO實現文件拷貝

作這個實驗是由於最近要作傳輸，雖然方向上定的是用Mina的IoBuffer來進行傳值，可是在系統對接和文件備份上都要用到拷貝，原生的IO和NIO也是一個不錯的選擇。作這個實驗主要的目的仍是熟悉一下各類寫法，並比較一下性能。

其實，說內心話，我一直是但願NIO有一個很不錯的效率，可以大大的領先「舊」IO，但是實驗的結果卻並不如我所願。我採用了以下幾種文件傳輸方式：FileInputStream、BufferedInputStream、BufferedReader、RandomAccessFile以及FileChannel。因爲這裏是單線程的測試，因此沒有用到管道流。

第一輪對300MB的壓縮文件作複製和粘貼，結果BufferedInputStream完勝其餘幾個方式。速度幾乎接近了系統自身的複製和粘貼。而NIO並無想象的那麼出色，和其餘幾種方式時間相差並不大。因而我不死心，開始第二輪的測試。文件大小爲3個GB。直接致使了FileChannel中自帶的transferFrom方法報錯。而其餘方法時間也大體相同，誰都沒有特別優點。

下面貼出實驗結果：

/**
	 * a.rar 336MB
	 * 採用傳統IO FileInputStream 讀取，耗時：2372 
	 * 採用傳統IO BufferedInputStream 讀取，耗時：699
	 * 採用傳統IO RandomAccessFile 讀取，耗時：2302
	 * 採用NIO FileChannel 自帶方法 讀取，耗時：2724
	 * 採用NIO FileChannel 循環 讀取，耗時：2077
	 * 
	 * Red Hat Enterprise Linux 5 64-bit (2).vmdk 3GB
	 * 採用傳統IO FileInputStream 讀取，耗時：83519
	 * 採用傳統IO BufferedInputStream 讀取，耗時：89459
	 * 採用傳統IO RandomAccessFile 讀取，耗時：97847
	 * 採用NIO FileChannel 循環 讀取，耗時：88136
	 * 
	 */

出現這樣的結果也很正常，JDK已經優化了傳統IO的實現方式，大部分底層都採用了NIO的方式來實現，因此差距並不會很明顯。而第一次試驗BufferedInputStream之因此能完勝，主要我想仍是由於操做系統內存管理的問題，頁的頻繁切換也會產生極大地消耗。因此其實在日常的應用中舊IO也足夠用了。

可是對於高併發的系統，IO始終是瓶頸，而這筆開銷主要仍是在於複製上。這裏雖然也用到了transferTo這樣的API，可是因爲測試環境在WIN8上，因此並無體現其優點，在Kafka項目的介紹中有一段關於文件Copy的說明：

Maintaining this common format allows optimization of the most important operation: network transfer of persistent log chunks. Modern unix operating systems offer a highly optimized code path for transferring data out of pagecache to a socket; in Linux this is done with the sendfile system call. Java provides access to this system call with the FileChannel.transferTo api.
To understand the impact of sendfile, it is important to understand the common data path for transfer of data from file to socket:
1.	The operating system reads data from the disk into pagecache in kernel space
2.	The application reads the data from kernel space into a user-space buffer
3.	The application writes the data back into kernel space into a socket buffer
4.	The operating system copies the data from the socket buffer to the NIC buffer where it is sent over the network
This is clearly inefficient, there are four copies, two system calls. Using sendfile, this re-copying is avoided by allowing the OS to send the data from pagecache to the network directly. So in this optimized path, only the final copy to the NIC buffer is needed.
We expect a common use case to be multiple consumers on a topic. Using the zero-copy optimization above, data is copied into pagecache exactly once and reused on each consumption instead of being stored in memory and copied out to kernel space every time it is read. This allows messages to be consumed at a rate that approaches the limit of the network connection.
For more background on the sendfile and zero-copy support in Java, see this article on IBM developerworks.

在linux系統下能夠採用epoll技術區實現，具體的內容能夠參考IBM的這篇論文https://www.ibm.com/developerworks/linux/library/j-zerocopy/。文件的順序寫基本上能知足性能上的要求，可是一旦涉及到隨即讀寫，或者有了併發性的讀寫，這些都須要不斷的改進和測試了。

下面貼出測試代碼：

package com.a2.desktop.example3.fileread;

import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.RandomAccessFile;
import java.io.Reader;
import java.io.Writer;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

/**
 * 文件拷貝各類方式的比較
 * 
 * @author Chen.Hui
 * 
 */
public class TestFileTransport {

	 public final static String FILE_PATH =
	 "F:\\RedHetServer\\Red Hat Enterprise Linux 5 64-bit (2).vmdk";

	//public final static String FILE_PATH = "F:\\apache-tomcat-7.0.11\\webapps\\ROOT\\a.rar";

	public final static String FILE_PATH_OUT = "C:\\Red Hat Enterprise Linux 5 64-bit (2).vmdk";

	public static void TransByCommonIoStream() throws Exception {

		long beginTime = System.currentTimeMillis();

		FileInputStream fis = new FileInputStream(new File(FILE_PATH));

		FileOutputStream fos = new FileOutputStream(new File(FILE_PATH_OUT));

		byte[] b = new byte[1024];

		int len = 0;

		while ((len = fis.read(b)) != -1) {
			fos.write(b, 0, len);
		}

		fos.flush();

		fis.close();
		fos.close();

		long endTime = System.currentTimeMillis();

		System.out.println("採用傳統IO FileInputStream 讀取，耗時："
				+ (endTime - beginTime));

	}

	public static void TransByCommonIoBufferedStream() throws Exception {

		long beginTime = System.currentTimeMillis();

		FileInputStream fis = new FileInputStream(new File(FILE_PATH));

		FileOutputStream fos = new FileOutputStream(new File(FILE_PATH_OUT));

		BufferedInputStream bis = new BufferedInputStream(fis);

		BufferedOutputStream bos = new BufferedOutputStream(fos);

		byte[] b = new byte[1024];

		int len = 0;

		while ((len = bis.read(b)) != -1) {
			bos.write(b, 0, len);
		}

		bos.flush();

		fis.close();
		fos.close();
		bis.close();
		bos.close();

		long endTime = System.currentTimeMillis();

		System.out.println("採用傳統IO BufferedInputStream 讀取，耗時："
				+ (endTime - beginTime));

	}

	public static void TransByCommonIoBuffered() throws Exception {

		long beginTime = System.currentTimeMillis();

		Reader br = new BufferedReader(new FileReader(new File(FILE_PATH)));
		Writer bw = new BufferedWriter(new FileWriter(new File(FILE_PATH_OUT)));

		char[] c = new char[1024];

		int len = 0;

		while ((len = br.read(c)) != -1) {
			bw.write(c, 0, len);
		}

		bw.flush();
		br.close();
		bw.close();

		long endTime = System.currentTimeMillis();

		System.out.println("採用傳統IO  BufferedReader 讀取，耗時："
				+ (endTime - beginTime));
	}

	public static void TransByRandomAccFile() throws Exception {

		long beginTime = System.currentTimeMillis();

		FileInputStream fis = new FileInputStream(new File(FILE_PATH));

		RandomAccessFile raf = new RandomAccessFile(new File(FILE_PATH_OUT),
				"rw");

		byte[] b = new byte[1024];

		int len = 0;

		while ((len = fis.read(b)) != -1) {
			raf.write(b, 0, len);
		}

		long endTime = System.currentTimeMillis();

		System.out.println("採用傳統IO RandomAccessFile 讀取，耗時："
				+ (endTime - beginTime));

	}

	/**
	 * 採用FileChannel 自帶方法測試 public abstract long
	 * transferFrom(ReadableByteChannel src, long position, long count) throws
	 * IOException;
	 */
	public static void TransByNioFileChannel() throws Exception {

		long beginTime = System.currentTimeMillis();

		FileChannel fc = new FileInputStream(new File(FILE_PATH)).getChannel();

		FileChannel fco = new RandomAccessFile(new File(FILE_PATH_OUT), "rw")
				.getChannel();

		fco.transferFrom(fc, 0, fc.size());

		long endTime = System.currentTimeMillis();

		System.out.println("採用NIO FileChannel 自帶方法  讀取，耗時："
				+ (endTime - beginTime));
	}

	public static void TransByNioFileChannelCommon() throws Exception {

		long beginTime = System.currentTimeMillis();

		FileChannel fc = new FileInputStream(new File(FILE_PATH)).getChannel();

		FileChannel fco = new RandomAccessFile(new File(FILE_PATH_OUT), "rw")
				.getChannel();

		ByteBuffer buf = ByteBuffer.allocate(1024);

		while (fc.read(buf) != -1) {
			buf.flip();
			fco.write(buf);
			buf.clear();
		}

		long endTime = System.currentTimeMillis();

		System.out.println("採用NIO FileChannel 循環 讀取，耗時："
				+ (endTime - beginTime));
	}

	public static void deleteFile() {
		File f = new File(FILE_PATH_OUT);
		if (f.exists())
			f.delete();
	}

	public static void main(String[] args) throws Exception {

		TransByCommonIoStream();
		deleteFile();
		TransByCommonIoBufferedStream();
		deleteFile();
		TransByRandomAccFile();
//		deleteFile();
//		TransByNioFileChannel();
		deleteFile();
		TransByNioFileChannelCommon();
		deleteFile();
	}

	/**
	 * a.rar 336MB
	 * 採用傳統IO FileInputStream 讀取，耗時：2372 
	 * 採用傳統IO BufferedInputStream 讀取，耗時：699
	 * 採用傳統IO RandomAccessFile 讀取，耗時：2302
	 * 採用NIO FileChannel 自帶方法 讀取，耗時：2724
	 * 採用NIO FileChannel 循環 讀取，耗時：2077
	 * 
	 * Red Hat Enterprise Linux 5 64-bit (2).vmdk 3GB
	 * 採用傳統IO FileInputStream 讀取，耗時：83519
	 * 採用傳統IO BufferedInputStream 讀取，耗時：89459
	 * 採用傳統IO RandomAccessFile 讀取，耗時：97847
	 * 採用NIO FileChannel 循環 讀取，耗時：88136
	 * 
	 */
}

測試主要是爲了追求更高的性能，最近研究性的東西作的比較多，今天作完了Mina和C的對接，明天要作協議的解析，希望能有新的發現。

謝謝觀賞。