android: 使用okhttp可能引起OOM的一個點

遇到一個問題: 須要給全部的請求加簽名校驗以防刷接口;傳入請求url及body生成一個文本串做爲一個header傳給服務端;已經有現成的簽名檢驗方法String doSignature(String url, byte[] body);當前網絡庫基於com.squareup.okhttp3:okhttp:3.14.2.

這很簡單了,固然是寫一個interceptor而後將request對象的url及body傳入就好.因而有:

public class SignInterceptor implements Interceptor {
    @NonNull
    @Override
    public Response intercept(@NonNull Chain chain) throws IOException {
        Request request = chain.request();
        RequestBody body = request.body();
        byte[] bodyBytes = null;
        if (body != null) {
            final Buffer buffer = new Buffer();
            body.writeTo(buffer);
            bodyBytes = buffer.readByteArray();
        }

        Request.Builder builder = request.newBuilder();
        HttpUrl oldUrl = request.url();
        final String url = oldUrl.toString();
        final String signed = doSignature(url, bodyBytes));
        if (!TextUtils.isEmpty(signed)) {
            builder.addHeader(SIGN_KEY_NAME, signed);
        }
        return chain.proceed(builder.build());
    }
}

okhttp的ReqeustBody是一個抽象類,內容輸出只有writeTo方法,將內容寫入到一個BufferedSink接口實現體裏,而後再將數據轉成byte[]也就是內存數組.能達到目的的類只有Buffer,它實現了BufferedSink接口並能提供轉成內存數組的方法readByteArray. 這貌似沒啥問題呀,能形成OOM?

是的,要看請求類型,若是是一個上傳文件的接口呢?若是這個文件比較大呢?上傳接口有可能會用到public static RequestBody create(final @Nullable MediaType contentType, final File file)方法,若是是針對文件的實現體它的writeTo方法是sink.writeAll(source);而咱們傳給簽名方法時用到的Buffer.readByteArray是將緩衝中的全部內容轉成了內存數組, 這意味着文件中的全部內容被轉成了內存數組, 就是在這個時機容易形成OOM! RequestBody.create源碼以下:

public static RequestBody create(final @Nullable MediaType contentType, final File file) {
    if (file == null) throw new NullPointerException("file == null");

    return new RequestBody() {
      @Override public @Nullable MediaType contentType() {
        return contentType;
      }

      @Override public long contentLength() {
        return file.length();
      }

      @Override public void writeTo(BufferedSink sink) throws IOException {
        try (Source source = Okio.source(file)) {
          sink.writeAll(source);
        }
      }
    };
  }

能夠看到實現體持有了文件,Content-Length返回了文件的大小, 內容所有轉給了Source對象。

這確實是之前很是容易忽略的一個點,不多有對請求體做額外處理的操做,而一旦這個操做變成一次性的大內存分配, 很是容易形成OOM. 因此要如何解決呢? 簽名方法又是如何處理的呢? 原來這個簽名方法在這裏偷了個懶——它只讀取傳入body的前4K內容,而後只針對這部份內容進行了加密,至於傳入的這個內存數組自己多大並不考慮,徹底把風險和麻煩丟給了外部(優秀的SDK!).

快速的方法固然是羅列白名單,針對上傳接口服務端不進行加簽驗證, 但這容易掛一漏萬,並且增長維護成本, 要簽名方法sdk的人另寫合適的接口等於要他們的命, 因此仍是得從根本解決. 既然簽名方法只讀取前4K內容,咱們便只將內容的前4K部分讀取再轉成方法所需的內存數組不就可了? 因此咱們的目的是: 指望RequestBody可以讀取一部分而不是所有的內容. 可否繼承RequestBody重寫它的writeTo? 能夠,但不現實,不可能所有替代現有的RequestBody實現類, 同時ok框架也有可能建立私有的實現類. 因此只能針對writeTo的參數BufferedSink做文章, 先得了解BufferedSink又是如何被okhttp框架調用的.

BufferedSink相關的類包括Buffer, Source,都屬於okio框架,okhttp只是基於okio的一坨, okio沒有直接用java的io操做,而是另行寫了一套io操做,具體是數據緩衝的操做.接上面的描述, Source是怎麼建立, 同時又是如何操做BufferedSink的? 在Okio.java中：

public static Source source(File file) throws FileNotFoundException {
    if (file == null) throw new IllegalArgumentException("file == null");
    return source(new FileInputStream(file));
  }
 
  public static Source source(InputStream in) {
    return source(in, new Timeout());
  }

  private static Source source(final InputStream in, final Timeout timeout) {
    return new Source() {
      @Override public long read(Buffer sink, long byteCount) throws IOException {
        try {
          timeout.throwIfReached();
          Segment tail = sink.writableSegment(1);
          int maxToCopy = (int) Math.min(byteCount, Segment.SIZE - tail.limit);
          int bytesRead = in.read(tail.data, tail.limit, maxToCopy);
          if (bytesRead == -1) return -1;
          tail.limit += bytesRead;
          sink.size += bytesRead;
          return bytesRead;
        } catch (AssertionError e) {
          if (isAndroidGetsocknameError(e)) throw new IOException(e);
          throw e;
        }
      }

      @Override public void close() throws IOException {
        in.close();
      }

      @Override public Timeout timeout() {
        return timeout;
      }
    };
  }

Source把文件做爲輸入流inputstream進行了各類讀操做, 可是它的read方法參數倒是個Buffer實例，它又是從哪來的，又怎麼和BufferedSink關聯的？ 只好再繼續看BufferedSink.writeAll的實現體。

BufferedSink的實現類就是Buffer， 而後它的writeAll方法:

@Override public long writeAll(Source source) throws IOException {
    if (source == null) throw new IllegalArgumentException("source == null");
    long totalBytesRead = 0;
    for (long readCount; (readCount = source.read(this, Segment.SIZE)) != -1; ) {
      totalBytesRead += readCount;
    }
    return totalBytesRead;
  }

原來是顯式的調用了Source.read(Buffer,long)方法，這樣就串起來了，那個Buffer參數原來就是自身。

基本能夠肯定只要實現BufferedSink接口類, 而後判斷讀入的內容超過指定大小就中止寫入就返回就可知足目的, 能夠名之FixedSizeSink.

然而麻煩的是BufferedSink的接口很是多, 將近30個方法， 不知道框架會在什麼時機調用哪一個方法，只能所有都實現! 其次是接口方法的參數有不少okio的類, 這些類的用法須要瞭解, 不然一旦用錯了效果拔苗助長. 因而對一個類的瞭解變成對多個類的瞭解, 沒辦法只能硬着頭皮寫.

第一個接口就有點蛋疼: Buffer buffer(); BufferedSink返回一個Buffer實例供外部調用, BufferedSink的實現體便是Buffer, 而後再返回一個Buffer?! 看了半天猜想BufferedSink是爲了提供一個可寫入的緩衝對象, 但框架做者也懶的再搞接口解耦的那一套了(唉,你們都是怎麼簡單怎麼來). 因而FixedSizeSink至少須要持有一個Buffer對象, 它做實際的數據緩存，同時能夠在須要Source.read(Buffer ,long)的地方做爲參數傳過去.

同時能夠看到RequestBody的一個實現類FormBody, 用這個Buffer對象直接寫入一些數據:

private long writeOrCountBytes(@Nullable BufferedSink sink, boolean countBytes) {
    long byteCount = 0L;

    Buffer buffer;
    if (countBytes) {
      buffer = new Buffer();
    } else {
      buffer = sink.buffer();
    }

    for (int i = 0, size = encodedNames.size(); i < size; i++) {
      if (i > 0) buffer.writeByte('&');
      buffer.writeUtf8(encodedNames.get(i));
      buffer.writeByte('=');
      buffer.writeUtf8(encodedValues.get(i));
    }

    if (countBytes) {
      byteCount = buffer.size();
      buffer.clear();
    }

    return byteCount;
  }

有這樣的操做就有可能限制不了緩衝區大小變化！不過數據量應該相對小一些並且這種用法場景相對少，咱們指定的大小應該能覆蓋的了這種狀況。

接着還有一個接口BufferedSink write(ByteString byteString), 又得了解ByteString怎麼使用, 真是心力交瘁啊...

@Override public Buffer write(ByteString byteString) {
    byteString.write(this);
    return this;
  }

Buffer實現體裏能夠直接調用ByteString.write(Buffer)由於是包名訪問，本身實現的FixedSizeSink聲明在和同一包名package okio;也能夠這樣使用，若是是其它包名只能先轉成byte[]了, ByteString應該不大否則也不能這麼搞(沒有找到ByteString讀取一段數據的方法)：

@Override
    public BufferedSink write(@NotNull ByteString byteString) throws IOException {
        byte[] bytes = byteString.toByteArray();
        this.write(bytes);
        return this;
    }

總之就是把這些對象轉成內存數組或者Buffer可以接受的參數持有起來!

重點關心的writeAll反而相對好實現一點, 咱們連續讀取指定長度的內容直到內容長度達到咱們的閾值就行.

還有一個蛋疼的點是各類對象的read/write數據流方向:
Caller.read(Callee)/Caller.write(Callee), 有的是從Caller到Callee, 有的是相反，被一個小類整的有點頭疼……

最後上完整代碼, 若是發現什麼潛在的問題也能夠交流下~:

public class FixedSizeSink implements BufferedSink {
    private static final int SEGMENT_SIZE = 4096;
    private final Buffer mBuffer = new Buffer();
    private final int mLimitSize;

    private FixedSizeSink(int size) {
        this.mLimitSize = size;
    }

    @Override
    public Buffer buffer() {
        return mBuffer;
    }

    @Override
    public BufferedSink write(@NotNull ByteString byteString) throws IOException {
        byte[] bytes = byteString.toByteArray();
        this.write(bytes);
        return this;
    }

    @Override
    public BufferedSink write(@NotNull byte[] source) throws IOException {
        this.write(source, 0, source.length);
        return this;
    }

    @Override
    public BufferedSink write(@NotNull byte[] source, int offset,
            int byteCount) throws IOException {
        long available = mLimitSize - mBuffer.size();
        int count = Math.min(byteCount, (int) available);
        android.util.Log.d(TAG, String.format("FixedSizeSink.offset=%d,"
                         "count=%d,limit=%d,size=%d",
                offset, byteCount, mLimitSize, mBuffer.size()));
        if (count > 0) {
            mBuffer.write(source, offset, count);
        }
        return this;
    }

    @Override
    public long writeAll(@NotNull Source source) throws IOException {
        this.write(source, mLimitSize);
        return mBuffer.size();
    }

    @Override
    public BufferedSink write(@NotNull Source source, long byteCount) throws IOException {
        final long count = Math.min(byteCount, mLimitSize - mBuffer.size());
        final long BUFFER_SIZE = Math.min(count, SEGMENT_SIZE);
        android.util.Log.d(TAG, String.format("FixedSizeSink.count=%d,limit=%d"
                         ",size=%d,segment=%d",
                byteCount, mLimitSize, mBuffer.size(), BUFFER_SIZE));
        long totalBytesRead = 0;
        long readCount;
        while (totalBytesRead < count && (readCount = source.read(mBuffer, BUFFER_SIZE)) != -1) {
            totalBytesRead = readCount;
        }
        return this;
    }

    @Override
    public int write(ByteBuffer src) throws IOException {
        final int available = mLimitSize - (int) mBuffer.size();
        if (available < src.remaining()) {
            byte[] bytes = new byte[available];
            src.get(bytes);
            this.write(bytes);
            return bytes.length;
        } else {
            return mBuffer.write(src);
        }
    }

    @Override
    public void write(@NotNull Buffer source, long byteCount) throws IOException {
        mBuffer.write(source, Math.min(byteCount, mLimitSize - mBuffer.size()));
    }

    @Override
    public BufferedSink writeUtf8(@NotNull String string) throws IOException {
        mBuffer.writeUtf8(string);
        return this;
    }

    @Override
    public BufferedSink writeUtf8(@NotNull String string, int beginIndex, int endIndex)
            throws IOException {
        mBuffer.writeUtf8(string, beginIndex, endIndex);
        return this;
    }

    @Override
    public BufferedSink writeUtf8CodePoint(int codePoint) throws IOException {
        mBuffer.writeUtf8CodePoint(codePoint);
        return this;
    }

    @Override
    public BufferedSink writeString(@NotNull String string,
            @NotNull Charset charset) throws IOException {
        mBuffer.writeString(string, charset);
        return this;
    }

    @Override
    public BufferedSink writeString(@NotNull String string, int beginIndex, int endIndex,
            @NotNull Charset charset) throws IOException {
        mBuffer.writeString(string, beginIndex, endIndex, charset);
        return this;
    }

    @Override
    public BufferedSink writeByte(int b) throws IOException {
        mBuffer.writeByte(b);
        return this;
    }

    @Override
    public BufferedSink writeShort(int s) throws IOException {
        mBuffer.writeShort(s);
        return this;
    }

    @Override
    public BufferedSink writeShortLe(int s) throws IOException {
        mBuffer.writeShortLe(s);
        return this;
    }

    @Override
    public BufferedSink writeInt(int i) throws IOException {
        mBuffer.writeInt(i);
        return this;
    }

    @Override
    public BufferedSink writeIntLe(int i) throws IOException {
        mBuffer.writeIntLe(i);
        return this;
    }

    @Override
    public BufferedSink writeLong(long v) throws IOException {
        mBuffer.writeLong(v);
        return this;
    }

    @Override
    public BufferedSink writeLongLe(long v) throws IOException {
        mBuffer.writeLongLe(v);
        return this;
    }

    @Override
    public BufferedSink writeDecimalLong(long v) throws IOException {
        mBuffer.writeDecimalLong(v);
        return this;
    }

    @Override
    public BufferedSink writeHexadecimalUnsignedLong(long v) throws IOException {
        mBuffer.writeHexadecimalUnsignedLong(v);
        return this;
    }

    @Override
    public void flush() throws IOException {
        mBuffer.flush();
    }

    @Override
    public BufferedSink emit() throws IOException {
        mBuffer.emit();
        return this;
    }

    @Override
    public BufferedSink emitCompleteSegments() throws IOException {
        mBuffer.emitCompleteSegments();
        return this;
    }

    @Override
    public OutputStream outputStream() {
        return mBuffer.outputStream();
    }

    @Override
    public boolean isOpen() {
        return mBuffer.isOpen();
    }

    @Override
    public Timeout timeout() {
        return mBuffer.timeout();
    }

    @Override
    public void close() throws IOException {
        mBuffer.close();
    }
}