Android Camera原理之setRepeatingRequest與capture模塊

時間  2019-11-06
標籤 android camera 原理 setrepeatingrequest capture 模塊 欄目 Android 简体版
原文   原文鏈接

Camera操做過程當中最重要的四個步驟：android

  • CameraManager-->openCamera ---> 打開相機
  • CameraDeviceImpl-->createCaptureSession ---> 建立捕獲會話
  • CameraCaptureSession-->setRepeatingRequest ---> 設置預覽界面
  • CameraDeviceImpl-->capture ---> 開始捕獲圖片

以前咱們介紹了openCamera流程和createCaptureSession流程，以下：
《Android Camera原理之openCamera模塊(一)》
《Android Camera原理之openCamera模塊(二)》
《Android Camera原理之createCaptureSession模塊》
至此，Camera 會話已經建立成功，接下來咱們能夠開始預覽了，預覽回調onCaptureCompleted以後就能夠拍照(回調到onCaptureCompleted，說明capture 完整frame數據已經返回了，能夠捕捉其中的數據了。)，因爲預覽和拍照的不少流程很類似，拍照只是預覽過程當中的一個節點，因此咱們把預覽和拍照放在一文裏講解。緩存

1.預覽

預覽發起的函數就是CameraCaptureSession-->setRepeatingRequest，本文咱們就談一下Camera 是如何發起預覽操做的。
CameraCaptureSession-->setRepeatingRequest是createCaptureSession(List<Surface> outputs, CameraCaptureSession.StateCallback callback, Handler handler)中輸出流配置成功以後執行CameraCaptureSession.StateCallback.onConfigured(@NonNull CameraCaptureSession session)函數中執行的。session

mCameraDevice.createCaptureSession(Arrays.asList(surface, mImageReader.getSurface()),
                    new CameraCaptureSession.StateCallback() {

                        @Override
                        public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                            // The camera is already closed
                            if (null == mCameraDevice) {
                                return;
                            }

                            // When the session is ready, we start displaying the preview.
                            mCaptureSession = cameraCaptureSession;
                            try {
                                // Auto focus should be continuous for camera preview.
                                mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE,
                                        CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
                                // Flash is automatically enabled when necessary.
                                setAutoFlash(mPreviewRequestBuilder);

                                // Finally, we start displaying the camera preview.
                                mPreviewRequest = mPreviewRequestBuilder.build();
                                mCaptureSession.setRepeatingRequest(mPreviewRequest,
                                        mCaptureCallback, mBackgroundHandler);
                            } catch (CameraAccessException e) {
                                e.printStackTrace();
                            }
                        }

                        @Override
                        public void onConfigureFailed(
                                @NonNull CameraCaptureSession cameraCaptureSession) {
                            showToast("Failed");
                        }
                    }, null
            );

最終執行了
mCaptureSession.setRepeatingRequest(mPreviewRequest, mCaptureCallback, mBackgroundHandler);
來執行camera preview操做。像對焦等操做就能夠在這個onConfigured回調中完成。架構

  • onConfigured回調錶示當前的配置流已經完成，相機已經顯示出來了，能夠預覽了。
  • onConfigureFailed配置流失敗，相機黑屏。
public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
            Handler handler) throws CameraAccessException {
        checkRepeatingRequest(request);

        synchronized (mDeviceImpl.mInterfaceLock) {
            checkNotClosed();

            handler = checkHandler(handler, callback);

            return addPendingSequence(mDeviceImpl.setRepeatingRequest(request,
                    createCaptureCallbackProxy(handler, callback), mDeviceExecutor));
        }
    }
  • 第一個參數CaptureRequest 標識當前capture 請求的屬性，是請求一個camera仍是多個camera，是否複用以前的請求等等。
  • 第二個參數CaptureCallback 是捕捉回調，這是開發者直接接觸的回調。
public interface CaptureCallback {
        public static final int NO_FRAMES_CAPTURED = -1;
        public void onCaptureStarted(CameraDevice camera,
                CaptureRequest request, long timestamp, long frameNumber);
        public void onCapturePartial(CameraDevice camera,
                CaptureRequest request, CaptureResult result);
        public void onCaptureProgressed(CameraDevice camera,
                CaptureRequest request, CaptureResult partialResult);
        public void onCaptureCompleted(CameraDevice camera,
                CaptureRequest request, TotalCaptureResult result);
        public void onCaptureFailed(CameraDevice camera,
                CaptureRequest request, CaptureFailure failure);
        public void onCaptureSequenceCompleted(CameraDevice camera,
                int sequenceId, long frameNumber);
        public void onCaptureSequenceAborted(CameraDevice camera,
                int sequenceId);
        public void onCaptureBufferLost(CameraDevice camera,
                CaptureRequest request, Surface target, long frameNumber);
    }

這須要開發者本身實現，這些回調是如何調用到上層的，請看《Android Camera原理之CameraDeviceCallbacks回調模塊》，都是經過CameraDeviceCallbacks回調調上來的。
下面咱們從camera 調用原理的角度分析一下
mCaptureSession.setRepeatingRequest
--->CameraDeviceImpl.setRepeatingRequest
--->CameraDeviceImpl.submitCaptureRequest
其中CameraDeviceImpl.setRepeatingRequest中第3個參數傳入的是true。之因此這個強調一點，由於接下來執行CameraDeviceImpl.capture的時候也會執行setRepeatingRequest，這裏第3個參數傳入的就是false。第3個參數boolean repeating若是爲true，表示當前捕獲的是一個過程，camera frame不斷在填充；若是爲false，表示當前捕獲的是一個瞬間，就是拍照。ide

public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
            Executor executor) throws CameraAccessException {
        List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
        requestList.add(request);
        return submitCaptureRequest(requestList, callback, executor, /*streaming*/true);
    }
    private int submitCaptureRequest(List<CaptureRequest> requestList, CaptureCallback callback,
            Executor executor, boolean repeating)  {
//......
    }

CameraDeviceImpl.submitCaptureRequest核心工做就是3步：函數

  • 1.驗證當前CaptureRequest列表中的request是否合理：核心就是驗證與request綁定的Surface是否存在。
  • 2.向底層發送請求信息。
  • 3.將底層返回的請求信息和傳入的CaptureCallback 綁定，以便後續正確回調。

而這三步中，第二步倒是核心工做。ui

1.1 向底層發送captureRequest請求this

SubmitInfo requestInfo;

            CaptureRequest[] requestArray = requestList.toArray(new CaptureRequest[requestList.size()]);
            // Convert Surface to streamIdx and surfaceIdx
            for (CaptureRequest request : requestArray) {
                request.convertSurfaceToStreamId(mConfiguredOutputs);
            }

            requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);
            if (DEBUG) {
                Log.v(TAG, "last frame number " + requestInfo.getLastFrameNumber());
            }

            for (CaptureRequest request : requestArray) {
                request.recoverStreamIdToSurface();
            }
  • 執行request.convertSurfaceToStreamId(mConfiguredOutputs);將本地已經緩存的surface和stream記錄在內存中，並binder傳輸到camera service層中，防止camera service端重複請求。
  • requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);這兒直接調用到camera service端。這兒須要重點講解一下的。
  • request.recoverStreamIdToSurface();回調成功，清除以前在內存中的數據。

CameraDeviceClient::submitRequest
--->CameraDeviceClient::submitRequestList
這個函數代碼不少，前面不少執行都是在複用檢索以前的緩存是否可用，咱們關注一下核心的執行：預覽的狀況下傳入的streaming是true，執行上面；若是是拍照的話，那就執行下面的else。
err = mDevice->setStreamingRequestList(metadataRequestList, surfaceMapList, &(submitInfo->mLastFrameNumber));
傳入的submitInfo就是要返回上層的回調參數，若是是預覽狀態，須要不斷更新當前的的frame數據，因此每次更新最新的frame number。spa

if (streaming) {
        err = mDevice->setStreamingRequestList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
        if (err != OK) {
            String8 msg = String8::format(
                "Camera %s:  Got error %s (%d) after trying to set streaming request",
                mCameraIdStr.string(), strerror(-err), err);
            ALOGE("%s: %s", __FUNCTION__, msg.string());
            res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
                    msg.string());
        } else {
            Mutex::Autolock idLock(mStreamingRequestIdLock);
            mStreamingRequestId = submitInfo->mRequestId;
        }
    } else {
        err = mDevice->captureList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
        if (err != OK) {
            String8 msg = String8::format(
                "Camera %s: Got error %s (%d) after trying to submit capture request",
                mCameraIdStr.string(), strerror(-err), err);
            ALOGE("%s: %s", __FUNCTION__, msg.string());
            res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
                    msg.string());
        }
        ALOGV("%s: requestId = %d ", __FUNCTION__, submitInfo->mRequestId);
    }

Camera3Device::setStreamingRequestList
--->Camera3Device::submitRequestsHelper.net

status_t Camera3Device::submitRequestsHelper(
        const List<const PhysicalCameraSettingsList> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        bool repeating,
        /*out*/
        int64_t *lastFrameNumber) {
    ATRACE_CALL();
    Mutex::Autolock il(mInterfaceLock);
    Mutex::Autolock l(mLock);

    status_t res = checkStatusOkToCaptureLocked();
    if (res != OK) {
        // error logged by previous call
        return res;
    }

    RequestList requestList;

    res = convertMetadataListToRequestListLocked(requests, surfaceMaps,
            repeating, /*out*/&requestList);
    if (res != OK) {
        // error logged by previous call
        return res;
    }

    if (repeating) {
        res = mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
    } else {
        res = mRequestThread->queueRequestList(requestList, lastFrameNumber);
    }
//......
    return res;
}

預覽的時候會執行mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
拍照的時候執行mRequestThread->queueRequestList(requestList, lastFrameNumber);

mRequestThread->setRepeatingRequests

status_t Camera3Device::RequestThread::setRepeatingRequests(
        const RequestList &requests,
        /*out*/
        int64_t *lastFrameNumber) {
    ATRACE_CALL();
    Mutex::Autolock l(mRequestLock);
    if (lastFrameNumber != NULL) {
        *lastFrameNumber = mRepeatingLastFrameNumber;
    }
    mRepeatingRequests.clear();
    mRepeatingRequests.insert(mRepeatingRequests.begin(),
            requests.begin(), requests.end());

    unpauseForNewRequests();

    mRepeatingLastFrameNumber = hardware::camera2::ICameraDeviceUser::NO_IN_FLIGHT_REPEATING_FRAMES;
    return OK;
}

將當前提交的CaptureRequest請求放入以前的預覽請求隊列中，告知HAL層有新的request請求，HAL層鏈接請求開始工做，源源不斷地輸出信息到上層。這兒是跑在Camera3Device中定義的RequestThread線程中，能夠保證在預覽的時候不斷地捕獲信息流，camera就不斷處於預覽的狀態了。

1.2 將返回請求信息和 CaptureCallback 綁定

if (callback != null) {
                mCaptureCallbackMap.put(requestInfo.getRequestId(),
                        new CaptureCallbackHolder(
                            callback, requestList, executor, repeating, mNextSessionId - 1));
            } else {
                if (DEBUG) {
                    Log.d(TAG, "Listen for request " + requestInfo.getRequestId() + " is null");
                }
            }
/** map request IDs to callback/request data */
    private final SparseArray<CaptureCallbackHolder> mCaptureCallbackMap =
            new SparseArray<CaptureCallbackHolder>();

1.向底層發送captureRequest請求--->回調的requestIinfo表示當前capture request的結果，將requestInfo.getRequestId()與CaptureCallbackHolder綁定，由於Camera 2架構支持發送屢次CaptureRequest請求，若是不使用這種綁定機制，後續的回調會形成嚴重的錯亂，甚至回調不上來，那麼開發者沒法繼續使用了。
咱們看看使用這些回調的地方的代碼：
《Android Camera原理之CameraDeviceCallbacks回調模塊》已經說明了CameraDeviceCallbacks.aidl纔是camera service進程與用戶進程通訊的回調，到這個回調裏面，再取出CaptureRequest綁定的CaptureCallback回調，調用到CaptureCallback回調函數，這樣開發者能夠直接使用。
下面看看CameraDeviceCallbacks的onCaptureStarted回調---->

public void onCaptureStarted(final CaptureResultExtras resultExtras, final long timestamp) {
            int requestId = resultExtras.getRequestId();
            final long frameNumber = resultExtras.getFrameNumber();

            if (DEBUG) {
                Log.d(TAG, "Capture started for id " + requestId + " frame number " + frameNumber);
            }
            final CaptureCallbackHolder holder;

            synchronized(mInterfaceLock) {
                if (mRemoteDevice == null) return; // Camera already closed

                // Get the callback for this frame ID, if there is one
                holder = CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);

                if (holder == null) {
                    return;
                }

                if (isClosed()) return;

                // Dispatch capture start notice
                final long ident = Binder.clearCallingIdentity();
                try {
                    holder.getExecutor().execute(
                        new Runnable() {
                            @Override
                            public void run() {
                                if (!CameraDeviceImpl.this.isClosed()) {
                                    final int subsequenceId = resultExtras.getSubsequenceId();
                                    final CaptureRequest request = holder.getRequest(subsequenceId);

                                    if (holder.hasBatchedOutputs()) {
                                        // Send derived onCaptureStarted for requests within the
                                        // batch
                                        final Range<Integer> fpsRange =
                                            request.get(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE);
                                        for (int i = 0; i < holder.getRequestCount(); i++) {
                                            holder.getCallback().onCaptureStarted(
                                                CameraDeviceImpl.this,
                                                holder.getRequest(i),
                                                timestamp - (subsequenceId - i) *
                                                NANO_PER_SECOND/fpsRange.getUpper(),
                                                frameNumber - (subsequenceId - i));
                                        }
                                    } else {
                                        holder.getCallback().onCaptureStarted(
                                            CameraDeviceImpl.this,
                                            holder.getRequest(resultExtras.getSubsequenceId()),
                                            timestamp, frameNumber);
                                    }
                                }
                            }
                        });
                } finally {
                    Binder.restoreCallingIdentity(ident);
                }
            }
        }

holder = CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);而後直接調用

holder.getCallback().onCaptureStarted(
                                                CameraDeviceImpl.this,
                                                holder.getRequest(i),
                                                timestamp - (subsequenceId - i) *
                                                NANO_PER_SECOND/fpsRange.getUpper(),
                                                frameNumber - (subsequenceId - i));

簡單明瞭，脈絡清楚。

2.拍照

開發者若是想要拍照的話，直接調用
mCaptureSession.capture(mPreviewRequestBuilder.build(), mCaptureCallback, mBackgroundHandler);
拍照的調用流程和預覽很類似，只是在調用函數中個傳入的參數不一樣。

public int capture(CaptureRequest request, CaptureCallback callback, Executor executor)
            throws CameraAccessException {
        if (DEBUG) {
            Log.d(TAG, "calling capture");
        }
        List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
        requestList.add(request);
        return submitCaptureRequest(requestList, callback, executor, /*streaming*/false);
    }

拍照的時候也是調用submitCaptureRequest，只不過第3個參數傳入的是false，表示不用循環獲取HAL調用上來的幀數據，只獲取瞬間的幀數據就能夠。
拍照和預覽調用的區分在：CameraDeviceClient::submitRequestList

if (streaming) {
//......
    } else {
        err = mDevice->captureList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
        if (err != OK) {
            String8 msg = String8::format(
                "Camera %s: Got error %s (%d) after trying to submit capture request",
                mCameraIdStr.string(), strerror(-err), err);
            ALOGE("%s: %s", __FUNCTION__, msg.string());
            res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
                    msg.string());
        }
        ALOGV("%s: requestId = %d ", __FUNCTION__, submitInfo->mRequestId);
    }

接下里調用到
mDevice->captureList
--->Camera3Device::submitRequestsHelper

status_t Camera3Device::submitRequestsHelper(
        const List<const PhysicalCameraSettingsList> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        bool repeating,
        /*out*/
        int64_t *lastFrameNumber) {
//......
    RequestList requestList;
//......
    if (repeating) {
        res = mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
    } else {
        res = mRequestThread->queueRequestList(requestList, lastFrameNumber);
    }
//......
    return res;
}

執行Camera3Device::RequestThread線程中的queueRequestList。

status_t Camera3Device::RequestThread::queueRequestList(
        List<sp<CaptureRequest> > &requests,
        /*out*/
        int64_t *lastFrameNumber) {
    ATRACE_CALL();
    Mutex::Autolock l(mRequestLock);
    for (List<sp<CaptureRequest> >::iterator it = requests.begin(); it != requests.end();
            ++it) {
        mRequestQueue.push_back(*it);
    }

    if (lastFrameNumber != NULL) {
        *lastFrameNumber = mFrameNumber + mRequestQueue.size() - 1;
        ALOGV("%s: requestId %d, mFrameNumber %" PRId32 ", lastFrameNumber %" PRId64 ".",
              __FUNCTION__, (*(requests.begin()))->mResultExtras.requestId, mFrameNumber,
              *lastFrameNumber);
    }

    unpauseForNewRequests();

    return OK;
}

*lastFrameNumber = mFrameNumber + mRequestQueue.size() - 1;
這裏有關鍵的執行代碼，表示當前取最新的capture frame數據。

拍照的時候在什麼地方捕捉image？

camera1的時候提供了PictureCallback回調方式來提供實時預覽回調，能夠在這裏獲取image數據回調。
camera2沒有這個接口，可是提供了ImageReader.OnImageAvailableListener來實現回調。

public interface OnImageAvailableListener {
        /**
         * Callback that is called when a new image is available from ImageReader.
         *
         * @param reader the ImageReader the callback is associated with.
         * @see ImageReader
         * @see Image
         */
        void onImageAvailable(ImageReader reader);
    }

還記得《Android Camera模塊解析之拍照》中提到openCamera以前要設置

mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(),
                        ImageFormat.JPEG, /*maxImages*/2);
                mImageReader.setOnImageAvailableListener(
                        mOnImageAvailableListener, mBackgroundHandler);

ImageReader中有一個getSurface()函數，這是ImageReader的拍照輸出流，咱們拍照的時候通常有兩個輸出流(outputSurface對象)，一個是預覽流，還有一個是拍照流。不記得能夠參考《Android Camera原理之createCaptureSession模塊》，ImageReader設置的拍照流會設置到camera service端。

public Surface getSurface() {
        return mSurface;
    }

咱們看看在何時回調這個接口。

ImageReader回調接口.jpg


看上面的調用流程，調用到ImageReader.OnImageAvailableListener->onImageAvailable中，咱們獲取ImageReader->acquireNextImage能夠獲取採集的image圖片。其實ImageReader中也能夠獲取預覽的流式數據。SurfacePlane 封裝了返回的ByteBuffer數據，可供開發者實時獲取。

 

private class SurfacePlane extends android.media.Image.Plane {
            private SurfacePlane(int rowStride, int pixelStride, ByteBuffer buffer) {
                mRowStride = rowStride;
                mPixelStride = pixelStride;
                mBuffer = buffer;
                /**
                 * Set the byteBuffer order according to host endianness (native
                 * order), otherwise, the byteBuffer order defaults to
                 * ByteOrder.BIG_ENDIAN.
                 */
                mBuffer.order(ByteOrder.nativeOrder());
            }

            @Override
            public ByteBuffer getBuffer() {
                throwISEIfImageIsInvalid();
                return mBuffer;
            }
            final private int mPixelStride;
            final private int mRowStride;

            private ByteBuffer mBuffer;
}

Note：不少開發者在camera1使用Camera.PreviewCallback的
void onPreviewFrame(byte[] data, Camera camera)
能夠獲取實時數據，可是在camera2中沒有這個接口了，雖然camera1的接口方法也能用，camera2替代的接口就是ImageReader.OnImageAvailableListener->onImageAvailable

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程