由dubbo服務禁用system.gc而引發的思考

我一直都有一個疑問，豐巢業務服務的生產環境jvm參數設置是禁止system.gc的，也就是開啓設置：-XX:+DisableExplicitGC，可是生產環境卻歷來沒有出現過堆外內存溢出的狀況。說明一下，豐巢使用了阿里開源的dubbo，而dubbo底層通訊默認狀況下使用了3.2.5.Final版本的netty，而咱們對於netty的常規認知裏，netty必定是使用了堆外內存，而且堆外內存在禁止了system.gc這個函數調用的話，在服務沒有主動回收分配的堆外內存的狀況下，必定會出現堆外內存的泄露。帶着這個問題，恰好前天晚上有些時間，研究了一下3.2.5版本的netty源碼，又是在科興科興園等饅頭媽媽時候，發現了祕密之所在，我只能說，科興科學園真是個人寶地啊。

 

涉及到的netty類：NioWorker、HeapChannelBufferFactory、BigEndianHeapChannelBuffer、SocketReceiveBufferPool

 

核心的祕密在SocketReceiveBufferPool中

 1 final class SocketReceiveBufferPool {
 2 
 3     private static final int POOL_SIZE = 8;
 4 
 5     @SuppressWarnings("unchecked")
 6     private final SoftReference<ByteBuffer>[] pool = new SoftReference[POOL_SIZE];
 7 
 8     SocketReceiveBufferPool() {
 9         super();
10     }
11 
12     final ByteBuffer acquire(int size) {
13         final SoftReference<ByteBuffer>[] pool = this.pool;
14         for (int i = 0; i < POOL_SIZE; i ++) {
15             SoftReference<ByteBuffer> ref = pool[i];
16             if (ref == null) {
17                 continue;
18             }
19 
20             ByteBuffer buf = ref.get();
21             if (buf == null) {
22                 pool[i] = null;
23                 continue;
24             }
25 
26             if (buf.capacity() < size) {
27                 continue;
28             }
29 
30             pool[i] = null;
31 
32             buf.clear();
33             return buf;
34         }
35 
36         ByteBuffer buf = ByteBuffer.allocateDirect(normalizeCapacity(size));
37         buf.clear();
38         return buf;
39     }
40 
41     final void release(ByteBuffer buffer) {
42         final SoftReference<ByteBuffer>[] pool = this.pool;
43         for (int i = 0; i < POOL_SIZE; i ++) {
44             SoftReference<ByteBuffer> ref = pool[i];
45             if (ref == null || ref.get() == null) {
46                 pool[i] = new SoftReference<ByteBuffer>(buffer);
47                 return;
48             }
49         }
50 
51         // pool is full - replace one
52         final int capacity = buffer.capacity();
53         for (int i = 0; i< POOL_SIZE; i ++) {
54             SoftReference<ByteBuffer> ref = pool[i];
55             ByteBuffer pooled = ref.get();
56             if (pooled == null) {
57                 pool[i] = null;
58                 continue;
59             }
60 
61             if (pooled.capacity() < capacity) {
62                 pool[i] = new SoftReference<ByteBuffer>(buffer);
63                 return;
64             }
65         }
66     }
67 
68     private static final int normalizeCapacity(int capacity) {
69         // Normalize to multiple of 1024
70         int q = capacity >>> 10;
71         int r = capacity & 1023;
72         if (r != 0) {
73             q ++;
74         }
75         return q << 10;
76     }
77 }

SocketReceiveBufferPool中維護了一個SoftReference<ByteBuffer>類型的數組，關於java的SoftReference，你們能夠自行搜索。其實就是在此類中維護了一個directbuffer的內存池，此部分的內存是能夠重複利用的。那麼問題來了，若是咱們把netty用於接收網絡信息的directbuffer直接傳給dubbo的業務代碼，那麼這個內存池的做用是什麼呢，內存如何被release回內存池？帶着這個疑問，繼續分析調用了SocketReceiveBufferPool的NioWorker代碼。

 1     private boolean read(SelectionKey k) {
 2         final SocketChannel ch = (SocketChannel) k.channel();
 3         final NioSocketChannel channel = (NioSocketChannel) k.attachment();
 4 
 5         final ReceiveBufferSizePredictor predictor =
 6             channel.getConfig().getReceiveBufferSizePredictor();
 7         final int predictedRecvBufSize = predictor.nextReceiveBufferSize();
 8 
 9         int ret = 0;
10         int readBytes = 0;
11         boolean failure = true;
12 
13         ByteBuffer bb = recvBufferPool.acquire(predictedRecvBufSize);
14         15         try {
16             while ((ret = ch.read(bb)) > 0) {
17                 readBytes += ret;
18                 if (!bb.hasRemaining()) {
19                     break;
20                 }
21             }
22             failure = false;
23         } catch (ClosedChannelException e) {
24             // Can happen, and does not need a user attention.
25         } catch (Throwable t) {
26             fireExceptionCaught(channel, t);
27         }
28 
29         if (readBytes > 0) {
30             bb.flip();
31 
32             final ChannelBufferFactory bufferFactory =
33                 channel.getConfig().getBufferFactory();
34             final ChannelBuffer buffer = bufferFactory.getBuffer(readBytes);
35             buffer.setBytes(0, bb);
36             buffer.writerIndex(readBytes);
37             //if(buffer instanceof BigEndianHeapChannelBuffer){
38             //    logger2.info("buffer instanceof BigEndianHeapChannelBuffer.");
39             //}
40             recvBufferPool.release(bb);
41 
42             // Update the predi||\\|||||
43             predictor.previousReceiveBufferSize(readBytes);
44 
45             // Fire the event.
46             fireMessageReceived(channel, buffer);
47         } else {
48             recvBufferPool.release(bb);
49         }
50 
51         if (ret < 0 || failure) {
52             k.cancel(); // Some JDK implementations run into an infinite loop without this.
53             close(channel, succeededFuture(channel));
54             return false;
55         }
56 
57         return true;
58     }

在代碼裏發現了netty會再創造一個chanelbuffer對象，而後將directbuffer裏的內容複製到chanelbuffer裏面，而這個chanelbuffer對象其實是一個堆內內存，而後netty會真對這塊內存進行解碼及返回給上層調用服務等，也就是說沒有直接將directbuffer返回給dubbo服務，這樣也就解釋了，咱們在提供dubbo服務的jvm裏，禁止掉了system.gc的狀況下，沒有發生過堆外內存泄漏的緣由。後面我會找時間詳細的分析一下netty4和kafka使用directbuffer的狀況。