標準的IO是基於字節流和字符流進行操做的,而JAVA中的NIO是基於Channel和Buffer進行操做的。react
傳統IOnginx
NIO數組
NIO主要有三個核心部分:Selector、Channel、Buffer服務器
數據老是從Channel讀取到Buffer或者從Buffer寫入到Channel中。網絡
Selector能夠監聽多個Channel的多個事件。多線程
1.傳統的IO是BIO的,而Channel是NIO的。併發
*當流調用了read()、write()方法後會一直阻塞線程直到數據被讀取或寫入完畢。框架
2.傳統IO流是單向的,而Channel是雙向的。dom
FileChannel:從文件中進行讀取 DatagramChannel:能夠經過UDP協議在網絡中進行數據的傳輸 SocketChannel:能夠經過TCP協議在網絡中進行數據的傳輸 ServerSocketChannel:能夠做爲一個服務器監聽鏈接
Channel通用API:異步
read(buffer):將數據從Channel讀取到Buffer中,讀取完畢返回-1。 read(buffer []):將數據從Channel讀取到多個Buffer中,僅當第一個Buffer被寫滿後往第二個Buffer中進行寫入。 write(buffer):將Buffer中的數據寫入到Channel中。 write(buffer[]):將多個Buffer中的數據寫入到Channel中,僅當第一個Buffer中的數據被讀取完畢後再從第二個Buffer中進行讀取。 register(selector,interest):將Channel註冊到Selector中,同時須要向Selector傳遞要監聽此Channel的事件類型(註冊到Selector中的Channel必定要非阻塞的) configureBlocking(boolean):設置Channel是否爲阻塞。 transferFrom(position,count,channel):將其餘Channel中的數據傳輸到當前Channel中。 transferTo(position,count,channel):將當前Channel中的數據傳輸到其餘Channel中。
SocketChannel API
open()靜態方法:建立SocketChannel。 connect(new InetSocketAddress(port))方法:鏈接服務器。 finishConnect()方法:判斷是否已經與服務器創建鏈接。
ServerSocketChannel API
open()靜態方法:建立ServerSocketChannel。 accept()方法:該方法會一直阻塞線程直到有新鏈接到達。
阻塞式與非阻塞式Channel
正常狀況下Channel都是阻塞的,只有當調用了configureBlocking(false)方法時Channel才爲非阻塞。
阻塞式Channel的connect()、accept()、read()、write()方法都會阻塞線程,直處處理完畢。
非阻塞式Channel的connect()、accept()、read()、write()方法都是異步的。
*當調用了非阻塞式Channel的connect()方法後,須要使用finishConnect()方法判斷是否已經與服務器創建鏈接。
*當調用了非阻塞式Channel的accept()方法後,須要根據方法的返回值是否爲NULL判斷是否接收到新的鏈接。
*當調用了非阻塞式Channel的read()方法後,須要根據方法的返回值是否大於0判斷是否有讀取到數據。
*在使用非阻塞式Channel的write()方法時,須要藉助while循環與hasRemaining()方法保證buffer中的內容被所有寫入。
*FileChannel必定是阻塞的。
示例
public void testFileChannel() throws IOException { RandomAccessFile randomAccessFile = new RandomAccessFile(new File("F:\\筆記\\nginx.txt"), "rw"); FileChannel fileChannel = randomAccessFile.getChannel(); ByteBuffer byteBuffer = ByteBuffer.allocate(64); int count = fileChannel.read(byteBuffer); while (count != -1) { byteBuffer.flip(); System.out.println(new String(Arrays.copyOfRange(byteBuffer.array(),0,byteBuffer.limit()),Charset.forName("UTF-8"))); byteBuffer.clear(); count = fileChannel.read(byteBuffer); } }
Buffer是一塊能夠進行讀寫操做的內存(順序存儲結構)
ByteBuffer:基於Byte類型進行存儲 CharBuffer:基於Char類型進行存儲 DoubleBuffer:基於Double類型進行存儲 FloatBuffer:基於Float類型進行存儲 IntBuffer:基於Int類型進行存儲 LongBuffer:基於Long類型進行存儲 ShortBuffer:基於Short類型進行存儲
Buffer的內部結構
1.capacity:表示buffer的容量
2.position:表示當前的位置(從0開始,最大值爲capacity-1)
3.limit:在寫模式中表示能夠寫入的個數(與capacity同樣),在讀模式中表示能夠讀取的個數。
從寫模式轉換成讀模式
limit設置爲position+1,position設置爲0。
從讀模式轉換成寫模式
limit設置爲capacity,position設置爲0。
往Buffer中寫數據
1.將數據從Channel讀取到Buffer中。
2.使用Buffer的put()方法。
從Buffer中讀數據
1.將Buffer中的數據寫入到Channel中。
2.使用Buffer的get()方法
Buffer通用API:
allocate(size)靜態靜態:初始化一個Buffer。 flip():將buffer從寫模式轉換成讀模式。 array():將Buffer中的內容轉換成數組(不受limit控制) get():獲取Buffer中的內容。 hasRemaining():判斷Buffer中是否還有未讀的元素(limit - (postion+1) ) rewind():將positon設置爲0。 clear():將limit設置爲capacity,position設置爲0。 compact():將全部未讀的元素移動到Buffer的起始處,position指向最後一個未讀的元素的下一位,limit設置爲capacity。 *clear()和compact()方法均可以理解成將Buffer從讀模式轉換成寫模式,區別在於compact()方法會保留未讀取的元素。 mark():在當前position處打一個標記。 reset():將position恢復到標記處。
Selector用於監聽多個Channel的多個事件(單線程)
Channel的事件類型
1.鏈接就緒:當SocketChannel、DatagramChannel成功與服務器創建鏈接時將會觸發鏈接就緒事件。
2.接收就緒:當有鏈接到達服務器時將會觸發接收就緒事件。
3.讀就緒:當SocketChannel、DatagramChannel有數據可讀時將會觸發讀就緒事件。
4.寫就緒:當SocketChannel、DatagramChannel能夠進行數據寫入時將會觸發寫就緒事件。
SelectionKey
SelectionKey用於存儲Selector與Channel之間的相關信息。
SelectionKey中提供了四個常量分別表明Channel的事件類型。
SelectionKey.OP_CONNECT SelectionKey.OP_ACCEPT SelectionKey.OP_READ SelectionKey.OP_WRITE
SelectableChannel提供的register(selector,interest)方法用於將Channel註冊到Selector中,同時須要向Selector傳遞要監聽此Channel的事件類型,當要監聽的事件類型不止一個時可使用或運算,當將Channel註冊到Selector後會返回SelectionKey實例,用於存儲Selector與此Channel之間的相關信息。
SelectionKey API:
interestOps()方法:返回Selector監聽此Channel的事件類型。 readyOps()方法:返回此Channel目前就緒的事件。 isAcceptable():判斷Channel是否接收就緒。 isConnectable():判斷Channel是否鏈接就緒。 isReadable():判斷Channel是否讀就緒。 isWriteable():判斷Channel是否寫就緒。 channel():返回具體的Channel實例。 selector():返回Selector實例。 attach():往SelectionKey中添加一個附加對象。 attachment():返回保存在SelectionKey中的附加對象。
Selector API:
open()靜態方法:建立一個Selector。 select()方法:該方法會一直阻塞線程直到所監聽的Channel有事件就緒,返回就緒的Channel個數(只會返回新就緒的Channel個數) selectedKeys()方法:返回就緒的Channel對應的SelectionKey。 *當Channel就緒的事件處理完畢後,須要手動刪除SelectionKey集合中該Channel對應的SelectionKey,當該Channel再次有事件就緒時會自動加入到Selectionkey集合中。
非阻塞式Channel與Selector
非阻塞式Channel通常與Selector配合使用
當Selector監聽到ServerSocketChannel接收就緒時,那麼此時能夠當即調用ServerSocketChannel的accept()方法獲取新鏈接。
當Selector監聽到SocketChannel讀就緒時,那麼此時能夠當即調用SocketChannel的read()方法進行數據的讀取。
非阻塞式服務器
/** * @Author: Zhuang HaoTang * @Date: 2019/10/26 16:35 * @Description: */ public class Server { public void start() throws IOException { Selector selector = Selector.open(); ServerSocketChannel serverSocketChannel = createNIOServerSocketChannel(); System.out.println("start nio server and bind port 8888"); serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); int ready = selector.select(); while (ready > 0) { System.out.println("ready channel count " + ready); Set<SelectionKey> selectionKeySet = selector.selectedKeys(); for (Iterator<SelectionKey> iterator = selectionKeySet.iterator(); iterator.hasNext(); ) { SelectionKey selectionKey = iterator.next(); if (selectionKey.isAcceptable()) { System.out.println("acceptable"); acceptHandler(selectionKey); } else if (selectionKey.isReadable()) { System.out.println("readable"); readHandler(selectionKey); } iterator.remove(); } ready = selector.select(); } } private ServerSocketChannel createNIOServerSocketChannel() throws IOException { ServerSocketChannel serverSocketChannel = ServerSocketChannel.open(); serverSocketChannel.bind(new InetSocketAddress(InetAddress.getLocalHost(), 8888)); serverSocketChannel.configureBlocking(false); return serverSocketChannel; } private void acceptHandler(SelectionKey selectionKey) throws IOException { Selector selector = selectionKey.selector(); ServerSocketChannel serverSocketChannel = (ServerSocketChannel) selectionKey.channel(); SocketChannel socketChannel = serverSocketChannel.accept(); socketChannel.configureBlocking(false); socketChannel.register(selector, SelectionKey.OP_READ); System.out.println("accept client connection " + socketChannel.getLocalAddress()); } private void readHandler(SelectionKey selectionKey) throws IOException { SocketChannel socketChannel = (SocketChannel) selectionKey.channel(); ByteBuffer byteBuffer = ByteBuffer.allocate(100); int num = socketChannel.read(byteBuffer); if(num == -1){ // 鏈接已斷開 System.out.println("client "+socketChannel.getLocalAddress() + " disconnection"); socketChannel.close(); return; } byteBuffer.flip(); while (byteBuffer.hasRemaining()) { byte b = byteBuffer.get(); System.out.println((char) b); } } public static void main(String[] args) throws IOException { Server server = new Server(); server.start(); } }
*一個Channel不會同時有多個事件就緒,以事件爲單位。
*當客戶端斷開鏈接,那麼將會觸發讀就緒,而且channel的read()方法返回-1,表示鏈接已斷開,服務器應該要作出處理,關閉這個鏈接。
客戶端
/** * @Auther: Zhuang HaoTang * @Date: 2019/10/26 16:36 * @Description: */ public class Client { public static void main(String[] args) throws IOException, InterruptedException { SocketChannel socketChannel = SocketChannel.open(); socketChannel.connect(new InetSocketAddress(InetAddress.getLocalHost(),8888)); String message = "today is sunday"; ByteBuffer byteBuffer = ByteBuffer.allocate(message.getBytes().length); byteBuffer.put(message.getBytes()); byteBuffer.flip(); socketChannel.write(byteBuffer); Thread.sleep(5000); } }
運行結果
Reactor有三種模式
1.Reactor單線程模式 2.Reactor多線程模式 3.主從Reactor多線程模式
*Reactor模式是在NIO下實現的。
1.單線程的事件分化器,同時這個線程須要處理接收、讀、寫就緒事件。
/** * @Author: Zhuang HaoTang * @Date: 2019/10/26 16:35 * @Description: */ public class ReactorSingleThreadServer { private void start() throws IOException { Selector selector = Selector.open(); ServerSocketChannel serverSocketChannel = createNIOServerSocketChannel(); System.out.println("start nio server and bind port 8888"); serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); int ready = selector.select(); while (ready > 0) { System.out.println("ready channel count " + ready); Set<SelectionKey> selectionKeySet = selector.selectedKeys(); for (Iterator<SelectionKey> iterator = selectionKeySet.iterator(); iterator.hasNext(); ) { SelectionKey selectionKey = iterator.next(); if (selectionKey.isAcceptable()) { System.out.println("acceptable"); acceptHandler(selectionKey); } else if (selectionKey.isReadable()) { System.out.println("readable"); readHandler(selectionKey); } iterator.remove(); } ready = selector.select(); } } private ServerSocketChannel createNIOServerSocketChannel() throws IOException { ServerSocketChannel serverSocketChannel = ServerSocketChannel.open(); serverSocketChannel.bind(new InetSocketAddress(InetAddress.getLocalHost(), 8888)); serverSocketChannel.configureBlocking(false); return serverSocketChannel; } private void acceptHandler(SelectionKey selectionKey) throws IOException { Selector selector = selectionKey.selector(); ServerSocketChannel serverSocketChannel = (ServerSocketChannel) selectionKey.channel(); SocketChannel socketChannel = serverSocketChannel.accept(); socketChannel.configureBlocking(false); socketChannel.register(selector, SelectionKey.OP_READ); System.out.println("accept client connection " + socketChannel.getLocalAddress()); } private void readHandler(SelectionKey selectionKey) throws IOException { SocketChannel socketChannel = (SocketChannel) selectionKey.channel(); ByteBuffer byteBuffer = ByteBuffer.allocate(100); int num = socketChannel.read(byteBuffer); if (num == -1) { System.out.println("client " + socketChannel.getLocalAddress() + " disconnection"); socketChannel.close(); return; } byteBuffer.flip(); while (byteBuffer.hasRemaining()) { byte b = byteBuffer.get(); System.out.println((char) b); } } public static void main(String[] args) throws IOException { ReactorSingleThreadServer server = new ReactorSingleThreadServer(); server.start(); } }
1.單線程的事件分發器。
2.具體事件類型的Handler線程池。
3.業務線程池。
/** * @Author: Zhuang HaoTang * @Date: 2019-10-28 17:00 * @Description: */ public class ReactorMultiThreadServer { private ThreadPoolExecutor eventHandlerPool = new ThreadPoolExecutor(10, 50, 2, TimeUnit.MINUTES, new ArrayBlockingQueue<Runnable>(200), new ThreadPoolExecutor.CallerRunsPolicy()); private void start() throws IOException { Selector selector = Selector.open(); ServerSocketChannel serverSocketChannel = createNIOServerSocketChannel(); System.out.println("start nio server and bind port 8888"); serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); selector.select(); for (;;) { Set<SelectionKey> selectionKeySet = selector.selectedKeys(); for (Iterator<SelectionKey> iterator = selectionKeySet.iterator(); iterator.hasNext(); ) { final SelectionKey selectionKey = iterator.next(); if (selectionKey.isAcceptable()) { System.out.println("acceptable"); eventHandlerPool.submit(new Runnable() { @Override public void run() { try { acceptHandler(selectionKey); } catch (IOException e) { e.printStackTrace(); } } }); } else if (selectionKey.isReadable()) { System.out.println("readable"); eventHandlerPool.submit(new Runnable() { @Override public void run() { readHandler(selectionKey); } }); } iterator.remove(); } // Thread.sleep(10); // 沒找到好方案,留一些時間給register selector.select(); } } private ServerSocketChannel createNIOServerSocketChannel() throws IOException { ServerSocketChannel serverSocketChannel = ServerSocketChannel.open(); serverSocketChannel.bind(new InetSocketAddress(InetAddress.getLocalHost(), 8888)); serverSocketChannel.configureBlocking(false); return serverSocketChannel; } private void acceptHandler(SelectionKey selectionKey) throws IOException { Selector selector = selectionKey.selector(); ServerSocketChannel serverSocketChannel = (ServerSocketChannel) selectionKey.channel(); SocketChannel socketChannel = serverSocketChannel.accept(); if (socketChannel != null) { socketChannel.configureBlocking(false); selector.wakeup(); // 往Selector註冊Channel時,Selector要處於非阻塞狀態 socketChannel.register(selector, SelectionKey.OP_READ); System.out.println("accept client connection " + socketChannel.getLocalAddress()); } } private void readHandler(SelectionKey selectionKey) { SocketChannel socketChannel = (SocketChannel) selectionKey.channel(); ByteBuffer byteBuffer = ByteBuffer.allocate(100); try { int num = socketChannel.read(byteBuffer); if (num == -1) { System.out.println("client " + socketChannel.getLocalAddress() + " disconnection"); socketChannel.close(); // 底層有些邏輯 return; } byteBuffer.flip(); while (byteBuffer.hasRemaining()) { byte b = byteBuffer.get(); System.out.println((char) b); } } catch (Exception e) { System.out.println("因爲鏈接關閉致使併發線程讀取異常"); } } public static void main(String[] args) throws IOException { ReactorMultiThreadServer reactorServer = new ReactorMultiThreadServer(); reactorServer.start(); } }
1.使用兩個單線程的事件分發器。
第一個事件分發器只負責監聽ServerSocketChannel的接收就緒事件,同時ServerSocketChannel接收到的鏈接要註冊到第二個事件分發器中。 第二個事件分發器只負責監聽SocketChannel的讀、寫就緒事件。
2.具體事件類型的Handler線程池。
3.業務線程池。
/** * @Author: Zhuang HaoTang * @Date: 2019-10-28 17:00 * @Description: */ public class MainSubReactorMultiThreadServer { private ThreadPoolExecutor eventHandlerPool = new ThreadPoolExecutor(10, 50, 2, TimeUnit.MINUTES, new ArrayBlockingQueue<Runnable>(200), new ThreadPoolExecutor.CallerRunsPolicy()); private void start() throws IOException { final Selector mainSelector = Selector.open(); final Selector subSelector = Selector.open(); new Thread(new Runnable() { @Override public void run() { try { startMainSelector(mainSelector, subSelector); } catch (IOException e) { e.printStackTrace(); } } }).start(); new Thread(new Runnable() { @Override public void run() { try { startSubSelector(subSelector); } catch (IOException e) { e.printStackTrace(); } } }).start(); } /** * 第一個事件分發器,用於監聽ServerSocketChannel的接收就緒事件 */ private void startMainSelector(Selector mainSelector, final Selector subSelector) throws IOException { ServerSocketChannel serverSocketChannel = createNIOServerSocketChannel(); System.out.println("start nio server and bind port 8888"); serverSocketChannel.register(mainSelector, SelectionKey.OP_ACCEPT); mainSelector.select(); for (; ; ) { final Set<SelectionKey> selectionKeySet = mainSelector.selectedKeys(); final SelectionKey selectionKey = Iterables.getOnlyElement(selectionKeySet); if (selectionKey.isAcceptable()) { System.out.println("acceptable"); eventHandlerPool.submit(new Runnable() { @Override public void run() { try { acceptHandler(selectionKey, subSelector); } catch (IOException e) { e.printStackTrace(); } } }); selectionKeySet.clear(); } mainSelector.select(); } } /** * 第二個事件分發器,用於監聽SockChannel的讀寫就緒事件 */ private void startSubSelector(Selector subSelector) throws IOException { subSelector.select(); for (; ; ) { Set<SelectionKey> selectionKeySet = subSelector.selectedKeys(); for (Iterator<SelectionKey> iterator = selectionKeySet.iterator(); iterator.hasNext(); ) { final SelectionKey selectionKey = iterator.next(); if (selectionKey.isReadable()) { System.out.println("readable"); eventHandlerPool.submit(new Runnable() { @Override public void run() { readHandler(selectionKey); } }); iterator.remove(); } } // Thread.sleep(10); // 沒找到好方案,留一些時間給register subSelector.select(); } } private ServerSocketChannel createNIOServerSocketChannel() throws IOException { ServerSocketChannel serverSocketChannel = ServerSocketChannel.open(); serverSocketChannel.bind(new InetSocketAddress(InetAddress.getLocalHost(), 8888)); serverSocketChannel.configureBlocking(false); return serverSocketChannel; } private void acceptHandler(SelectionKey selectionKey, Selector subSelector) throws IOException { ServerSocketChannel serverSocketChannel = (ServerSocketChannel) selectionKey.channel(); SocketChannel socketChannel = serverSocketChannel.accept(); if (socketChannel != null) { socketChannel.configureBlocking(false); subSelector.wakeup(); // 往Selector註冊Channel時,Selector要處於非阻塞狀態 socketChannel.register(subSelector, SelectionKey.OP_READ); System.out.println("accept client connection " + socketChannel.getLocalAddress() + " and register to subSelector"); } } private void readHandler(SelectionKey selectionKey) { SocketChannel socketChannel = (SocketChannel) selectionKey.channel(); ByteBuffer byteBuffer = ByteBuffer.allocate(100); try { int num = socketChannel.read(byteBuffer); if (num == -1) { System.out.println("client " + socketChannel.getLocalAddress() + " disconnection"); socketChannel.close(); // 底層有些邏輯 return; } byteBuffer.flip(); while (byteBuffer.hasRemaining()) { byte b = byteBuffer.get(); System.out.println((char) b); } } catch (Exception e) { System.out.println("因爲鏈接關閉致使併發線程讀取異常"); } } public static void main(String[] args) throws IOException { MainSubReactorMultiThreadServer reactorServer = new MainSubReactorMultiThreadServer(); reactorServer.start(); } }
通用客戶端
/** * @Author: Zhuang HaoTang * @Date: 2019/10/26 16:36 * @Description: */ public class Client { public static void main(String[] args) throws IOException, InterruptedException { SocketChannel socketChannel = SocketChannel.open(); socketChannel.connect(new InetSocketAddress(InetAddress.getLocalHost(), 8888)); String message = "today is sunday"; ByteBuffer byteBuffer = ByteBuffer.allocate(message.getBytes().length); byteBuffer.put(message.getBytes()); byteBuffer.flip(); socketChannel.write(byteBuffer); Thread.sleep(5000); ByteBuffer byteBuffer1 = ByteBuffer.allocate("wo".getBytes().length).put("wo".getBytes()); byteBuffer1.flip(); socketChannel.write(byteBuffer1); ByteBuffer receiveBuffer = ByteBuffer.allocate(1024); while (true) { socketChannel.read(receiveBuffer); receiveBuffer.flip(); while (receiveBuffer.hasRemaining()) { System.out.println((char)receiveBuffer.get()); } receiveBuffer.clear(); } } }
*主線程不須要等待具體事件類型的Handler處理完畢,直接異步返回,那麼將會致使事件重複就緒,程序作出相應的控制便可。
*當有鏈接到達服務器時,將會觸發接收就緒事件,那麼主線程將會不停的向線程池中提交任務,直到某個線程接收了鏈接,此時將會中止接收就緒,其餘線程接收到的鏈接爲NULL。
*當channel有數據可讀時,將會觸發讀就緒,那麼主線程將會不停的向線程池提交任務,直到某個線程讀取完畢,此時將會中止讀就緒,其餘線程讀取到的個數爲0。
*當客戶端斷開鏈接時,將會觸發讀就緒,那麼主線程將會不停的向線程池提交任務,直到某個線程關閉鏈接,此時將會中止讀就緒
通常不會直接去使用JAVA NIO,只是經過JAVA NIO學習他的設計思想,若是要想搭建NIO服務器那麼應該使用Netty等NIO框架。
BIO即同步並阻塞,線程會進入阻塞狀態,若是併發鏈接數只有幾百,那麼建立幾百個線程去處理是沒有任何問題的,這種方式更加簡單高效。
可是若是併發鏈接數達到幾萬,那麼顯然建立幾萬個線程去處理是不可行的,系統承受不了這個負荷,此時應該使用NIO,即同步非阻塞,利用更少的線程去作更多的事情。
JAVA NIO就是使用NIO(同步非阻塞),使用IO多路複用的Select模型。
*無論客戶端有多少個併發鏈接和請求,服務端老是能夠利用更少的線程去處理(單線程事件分發器 和 具體事件類型的Handler線程池)