零拷貝( Zore Copy )

零拷貝是實現高速數據傳輸的一種協議。數據從源節點傳送到目的節點的過程當中不通過中間緩存。java

具體提升了哪些須要咱們先來了解傳統IO的方式,經過與傳統IO方式對比來看。緩存

傳統IO方式

在java開發中,從某臺機器將一份數據經過網絡傳輸到另一臺機器,大體的代碼以下:網絡

Socket socket = new Socket(HOST, PORT);
InputStream inputStream = new FileInputStream(FILE_PATH);
OutputStream outputStream = new DataOutputStream(socket.getOutputStream());

byte[] buffer = new byte[4096];
while (inputStream.read(buffer) >= 0) {
    outputStream.write(buffer);
}

outputStream.close();
socket.close();
inputStream.close();

看起來代碼很簡單,但若是咱們深刻到操做系統層面,就會發現實際的微觀操做更復雜。具體操做以下圖:架構

1. 用戶進程向OS發出read()系統調用,觸發上下文切換,從用戶態轉換到內核態。
2. CPU發起IO請求,經過直接內存訪問(DMA)從磁盤讀取文件內容,複製到內核緩衝區PageCache中
3. 將內核緩衝區數據,拷貝到用戶空間緩衝區,觸發上下文切換,從內核態轉換到用戶態。
4. 用戶進程向OS發起write系統調用,觸發上下文切換,從用戶態切換到內核態。
5. 將數據從用戶緩衝區拷貝到內核中與目的地Socket關聯的緩衝區。
6. 數據最終經由Socket經過DMA傳送到硬件(網卡)緩衝區,write()系統調用返回,並從內核態切換回用戶態。

零拷貝(Zero-copy)

如下使用FileChannel.transferTo方法,實現zero-copy:框架

SocketAddress socketAddress = new InetSocketAddress(HOST, PORT);
SocketChannel socketChannel = SocketChannel.open();
socketChannel.connect(socketAddress);

File file = new File(FILE_PATH);
FileChannel fileChannel = new FileInputStream(file).getChannel();
fileChannel.transferTo(0, file.length(), socketChannel);

fileChannel.close();
socketChannel.close();

相比傳統方式,零拷貝的執行流程以下圖:socket

能夠看到,相比傳統方式,零拷貝不走數據緩衝區減小了一些沒必要要的操做。ide

零拷貝的應用

零拷貝在不少框架中獲得了普遍使用,常見的好比Netty、Kafka等等。ui

在kafka中使用了不少設計思想,好比分區並行、順序寫入、頁緩存、高效序列化、零拷貝等等。this

上邊博客分析了Kafka的大概架構,知道了kafka中的文件都是以.log文件存儲,每一個日誌文件對應兩個索引文件.index與.timeindex。操作系統

kafka在傳輸數據時利用索引,使用fileChannel.transferTo(position, count, socketChannel)指定數據位置與大小實現零拷貝。

kafka底層傳輸源碼:(TransportLayer)

/**
     * Transfers bytes from `fileChannel` to this `TransportLayer`.
     *
     * This method will delegate to {@link FileChannel#transferTo(long, long, java.nio.channels.WritableByteChannel)},
     * but it will unwrap the destination channel, if possible, in order to benefit from zero copy. This is required
     * because the fast path of `transferTo` is only executed if the destination buffer inherits from an internal JDK
     * class.
     *
     * @param fileChannel The source channel
     * @param position The position within the file at which the transfer is to begin; must be non-negative
     * @param count The maximum number of bytes to be transferred; must be non-negative
     * @return The number of bytes, possibly zero, that were actually transferred
     * @see FileChannel#transferTo(long, long, java.nio.channels.WritableByteChannel)
     */
    long transferFrom(FileChannel fileChannel, long position, long count) throws IOException;

實現類(PlaintextTransportLayer):

@Override
 public long transferFrom(FileChannel fileChannel, long position, long count) throws IOException {
      return fileChannel.transferTo(position, count, socketChannel);
 }

該方法的功能是將FileChannel中的數據傳輸到TransportLayer,也就是SocketChannel。在實現類PlaintextTransportLayer的對應方法中,就是直接調用了FileChannel.transferTo()方法。

相關文章
相關標籤/搜索