FFmpeg編解碼處理2-編解碼API詳解

本文爲做者原創，轉載請註明出處：http://www.javashuo.com/article/p-bfosweqg-kv.html

FFmpeg編解碼處理系列筆記：
[0]. FFmpeg時間戳詳解
[1]. FFmpeg編解碼處理1-轉碼全流程簡介
[2]. FFmpeg編解碼處理2-編解碼API詳解
[3]. FFmpeg編解碼處理3-視頻編碼
[4]. FFmpeg編解碼處理4-音頻編碼

基於FFmpeg 4.1版本。

4. 編解碼API詳解

解碼使用avcodec_send_packet()和avcodec_receive_frame()兩個函數。
編碼使用avcodec_send_frame()和avcodec_receive_packet()兩個函數。

4.1 API定義

4.1.1 avcodec_send_packet()

/**
 * Supply raw packet data as input to a decoder.
 *
 * Internally, this call will copy relevant AVCodecContext fields, which can
 * influence decoding per-packet, and apply them when the packet is actually
 * decoded. (For example AVCodecContext.skip_frame, which might direct the
 * decoder to drop the frame contained by the packet sent with this function.)
 *
 * @warning The input buffer, avpkt->data must be AV_INPUT_BUFFER_PADDING_SIZE
 *          larger than the actual read bytes because some optimized bitstream
 *          readers read 32 or 64 bits at once and could read over the end.
 *
 * @warning Do not mix this API with the legacy API (like avcodec_decode_video2())
 *          on the same AVCodecContext. It will return unexpected results now
 *          or in future libavcodec versions.
 *
 * @note The AVCodecContext MUST have been opened with @ref avcodec_open2()
 *       before packets may be fed to the decoder.
 *
 * @param avctx codec context
 * @param[in] avpkt The input AVPacket. Usually, this will be a single video
 *                  frame, or several complete audio frames.
 *                  Ownership of the packet remains with the caller, and the
 *                  decoder will not write to the packet. The decoder may create
 *                  a reference to the packet data (or copy it if the packet is
 *                  not reference-counted).
 *                  Unlike with older APIs, the packet is always fully consumed,
 *                  and if it contains multiple frames (e.g. some audio codecs),
 *                  will require you to call avcodec_receive_frame() multiple
 *                  times afterwards before you can send a new packet.
 *                  It can be NULL (or an AVPacket with data set to NULL and
 *                  size set to 0); in this case, it is considered a flush
 *                  packet, which signals the end of the stream. Sending the
 *                  first flush packet will return success. Subsequent ones are
 *                  unnecessary and will return AVERROR_EOF. If the decoder
 *                  still has frames buffered, it will return them after sending
 *                  a flush packet.
 *
 * @return 0 on success, otherwise negative error code:
 *      AVERROR(EAGAIN):   input is not accepted in the current state - user
 *                         must read output with avcodec_receive_frame() (once
 *                         all output is read, the packet should be resent, and
 *                         the call will not fail with EAGAIN).
 *      AVERROR_EOF:       the decoder has been flushed, and no new packets can
 *                         be sent to it (also returned if more than 1 flush
 *                         packet is sent)
 *      AVERROR(EINVAL):   codec not opened, it is an encoder, or requires flush
 *      AVERROR(ENOMEM):   failed to add packet to internal queue, or similar
 *      other errors: legitimate decoding errors
 */
int avcodec_send_packet(AVCodecContext *avctx, const AVPacket *avpkt);

4.1.2 avcodec_receive_frame()

/**
 * Return decoded output data from a decoder.
 *
 * @param avctx codec context
 * @param frame This will be set to a reference-counted video or audio
 *              frame (depending on the decoder type) allocated by the
 *              decoder. Note that the function will always call
 *              av_frame_unref(frame) before doing anything else.
 *
 * @return
 *      0:                 success, a frame was returned
 *      AVERROR(EAGAIN):   output is not available in this state - user must try
 *                         to send new input
 *      AVERROR_EOF:       the decoder has been fully flushed, and there will be
 *                         no more output frames
 *      AVERROR(EINVAL):   codec not opened, or it is an encoder
 *      other negative values: legitimate decoding errors
 */
int avcodec_receive_frame(AVCodecContext *avctx, AVFrame *frame);

4.1.3 avcodec_send_frame()

/**
 * Supply a raw video or audio frame to the encoder. Use avcodec_receive_packet()
 * to retrieve buffered output packets.
 *
 * @param avctx     codec context
 * @param[in] frame AVFrame containing the raw audio or video frame to be encoded.
 *                  Ownership of the frame remains with the caller, and the
 *                  encoder will not write to the frame. The encoder may create
 *                  a reference to the frame data (or copy it if the frame is
 *                  not reference-counted).
 *                  It can be NULL, in which case it is considered a flush
 *                  packet.  This signals the end of the stream. If the encoder
 *                  still has packets buffered, it will return them after this
 *                  call. Once flushing mode has been entered, additional flush
 *                  packets are ignored, and sending frames will return
 *                  AVERROR_EOF.
 *
 *                  For audio:
 *                  If AV_CODEC_CAP_VARIABLE_FRAME_SIZE is set, then each frame
 *                  can have any number of samples.
 *                  If it is not set, frame->nb_samples must be equal to
 *                  avctx->frame_size for all frames except the last.
 *                  The final frame may be smaller than avctx->frame_size.
 * @return 0 on success, otherwise negative error code:
 *      AVERROR(EAGAIN):   input is not accepted in the current state - user
 *                         must read output with avcodec_receive_packet() (once
 *                         all output is read, the packet should be resent, and
 *                         the call will not fail with EAGAIN).
 *      AVERROR_EOF:       the encoder has been flushed, and no new frames can
 *                         be sent to it
 *      AVERROR(EINVAL):   codec not opened, refcounted_frames not set, it is a
 *                         decoder, or requires flush
 *      AVERROR(ENOMEM):   failed to add packet to internal queue, or similar
 *      other errors: legitimate decoding errors
 */
int avcodec_send_frame(AVCodecContext *avctx, const AVFrame *frame);

4.1.4 avcodec_receive_packet()

/**
 * Read encoded data from the encoder.
 *
 * @param avctx codec context
 * @param avpkt This will be set to a reference-counted packet allocated by the
 *              encoder. Note that the function will always call
 *              av_frame_unref(frame) before doing anything else.
 * @return 0 on success, otherwise negative error code:
 *      AVERROR(EAGAIN):   output is not available in the current state - user
 *                         must try to send input
 *      AVERROR_EOF:       the encoder has been fully flushed, and there will be
 *                         no more output packets
 *      AVERROR(EINVAL):   codec not opened, or it is an encoder
 *      other errors: legitimate decoding errors
 */
int avcodec_receive_packet(AVCodecContext *avctx, AVPacket *avpkt);

4.2 API使用說明

4.2.1 解碼API使用詳解

關於avcodec_send_packet()與avcodec_receive_frame()的使用說明：

1. 按dts遞增的順序向解碼器送入編碼幀packet，解碼器按pts遞增的順序輸出原始幀frame，實際上解碼器不關注輸入packet的dts(錯值都不要緊)，它只管依次處理收到的packet，按需緩衝和解碼
2. avcodec_receive_frame()輸出frame時，會根據各類因素設置好frame->best_effort_timestamp(文檔明確說明)，實測frame->pts也會被設置(一般直接拷貝自對應的packet.pts，文檔未明確說明)用戶應確保avcodec_send_packet()發送的packet具備正確的pts，編碼幀packet與原始幀frame間的對應關係經過pts肯定
3. avcodec_receive_frame()輸出frame時，frame->pkt_dts拷貝自當前avcodec_send_packet()發送的packet中的dts，若是當前packet爲NULL(flush packet)，解碼器進入flush模式，當前及剩餘的frame->pkt_dts值總爲AV_NOPTS_VALUE。由於解碼器中有緩存幀，當前輸出的frame並非由當前輸入的packet解碼獲得的，因此這個frame->pkt_dts沒什麼實際意義，能夠沒必要關注
4. avcodec_send_packet()發送第一個NULL會返回成功，後續的NULL會返回AVERROR_EOF。
5. avcodec_send_packet()屢次發送NULL並不會致使解碼器中緩存的幀丟失，使用avcodec_flush_buffers()能夠當即丟掉解碼器中緩存幀。所以播放完畢時應avcodec_send_packet(NULL)來取完緩存的幀，而SEEK操做或切換流時應調用avcodec_flush_buffers()來直接丟棄緩存幀。
6. 解碼器一般的沖洗方法：調用一次avcodec_send_packet(NULL)(返回成功)，而後不停調用avcodec_receive_frame()直到其返回AVERROR_EOF，取出全部緩存幀，avcodec_receive_frame()返回AVERROR_EOF這一次是沒有有效數據的，僅僅獲取到一個結束標誌。

4.2.2 編碼API使用詳解

關於avcodec_send_frame()與avcodec_receive_packet()的使用說明：

1. 按pts遞增的順序向編碼器送入原始幀frame，編碼器按dts遞增的順序輸出編碼幀packet，實際上編碼器關注輸入frame的pts不關注其dts，它只管依次處理收到的frame，按需緩衝和編碼
2. avcodec_receive_packet()輸出packet時，會設置packet.dts，從0開始，每次輸出的packet的dts加1，這是視頻層的dts，用戶寫輸出前應將其轉換爲容器層的dts
3. avcodec_receive_packet()輸出packet時，packet.pts拷貝自對應的frame.pts，這是視頻層的pts，用戶寫輸出前應將其轉換爲容器層的pts
4. avcodec_send_frame()發送NULL frame時，編碼器進入flush模式
5. avcodec_send_frame()發送第一個NULL會返回成功，後續的NULL會返回AVERROR_EOF
6. avcodec_send_frame()屢次發送NULL並不會致使編碼器中緩存的幀丟失，使用avcodec_flush_buffers()能夠當即丟掉編碼器中緩存幀。所以編碼完畢時應使用avcodec_send_frame(NULL)來取完緩存的幀，而SEEK操做或切換流時應調用avcodec_flush_buffers()來直接丟棄緩存幀。
7. 編碼器一般的沖洗方法：調用一次avcodec_send_frame(NULL)(返回成功)，而後不停調用avcodec_receive_packet()直到其返回AVERROR_EOF，取出全部緩存幀，avcodec_receive_packet()返回AVERROR_EOF這一次是沒有有效數據的，僅僅獲取到一個結束標誌。
8. 對音頻來講，若是AV_CODEC_CAP_VARIABLE_FRAME_SIZE(在AVCodecContext.codec.capabilities變量中，只讀)標誌有效，表示編碼器支持可變尺寸音頻幀，送入編碼器的音頻幀能夠包含任意數量的採樣點。若是此標誌無效，則每個音頻幀的採樣點數目(frame->nb_samples)必須等於編碼器設定的音頻幀尺寸(avctx->frame_size)，最後一幀除外，最後一幀音頻幀採樣點數能夠小於avctx->frame_size

4.3 API使用例程

4.3.1 解碼API例程

// retrun 0:                got a frame success
//        AVERROR(EAGAIN):  need more packet
//        AVERROR_EOF:      end of file, decoder has been flushed
//        <0:               error
int av_decode_frame(AVCodecContext *dec_ctx, AVPacket *packet, bool *new_packet, AVFrame *frame)
{
    int ret = AVERROR(EAGAIN);

    while (1)
    {
        // 2. 從解碼器接收frame
        if (dec_ctx->codec_type == AVMEDIA_TYPE_VIDEO)
        {
            // 2.1 一個視頻packet含一個視頻frame
            //     解碼器緩存必定數量的packet後，纔有解碼後的frame輸出
            //     frame輸出順序是按pts的順序，如IBBPBBP
            //     frame->pkt_pos變量是此frame對應的packet在視頻文件中的偏移地址，值同pkt.pos
            ret = avcodec_receive_frame(dec_ctx, frame);
            if (ret >= 0)
            {
                if (frame->pts == AV_NOPTS_VALUE)
                {
                    frame->pts = frame->best_effort_timestamp;
                    printf("set video pts %d\n", frame->pts);
                }
            }
        }
        else if (dec_ctx->codec_type ==  AVMEDIA_TYPE_AUDIO)
        {
            // 2.2 一個音頻packet含一至多個音頻frame，每次avcodec_receive_frame()返回一個frame，此函數返回。
            //     下次進來此函數，繼續獲取一個frame，直到avcodec_receive_frame()返回AVERROR(EAGAIN)，
            //     表示解碼器須要填入新的音頻packet
            ret = avcodec_receive_frame(dec_ctx, frame);
            if (ret >= 0)
            {
                if (frame->pts == AV_NOPTS_VALUE)
                {
                    frame->pts = frame->best_effort_timestamp;
                    printf("set audio pts %d\n", frame->pts);
                }
            }
        }

        if (ret >= 0)                   // 成功解碼獲得一個視頻幀或一個音頻幀，則返回
        {
            return ret;   
        }
        else if (ret == AVERROR_EOF)    // 解碼器已沖洗，解碼中全部幀已取出
        {
            avcodec_flush_buffers(dec_ctx);
            return ret;
        }
        else if (ret == AVERROR(EAGAIN))// 解碼器須要喂數據
        {
            if (!(*new_packet))         // 本函數中已向解碼器餵過數據，所以須要從文件讀取新數據
            {
                //av_log(NULL, AV_LOG_INFO, "decoder need more packet\n");
                return ret;
            }
        }
        else                            // 錯誤
        {
            av_log(NULL, AV_LOG_ERROR, "decoder error %d\n", ret);
            return ret;
        }

        /*
        if (packet == NULL || (packet->data == NULL && packet->size == 0))
        {
            // 復位解碼器內部狀態/刷新內部緩衝區。當seek操做或切換流時應調用此函數。
            avcodec_flush_buffers(dec_ctx);
        }
        */

        // 1. 將packet發送給解碼器
        //    發送packet的順序是按dts遞增的順序，如IPBBPBB
        //    pkt.pos變量能夠標識當前packet在視頻文件中的地址偏移
        //    發送第一個 flush packet 會返回成功，後續的 flush packet 會返回AVERROR_EOF
        ret = avcodec_send_packet(dec_ctx, packet);
        *new_packet = false;
        
        if (ret != 0)
        {
            av_log(NULL, AV_LOG_ERROR, "avcodec_send_packet() error, return %d\n", ret);
            return ret;
        }
    }

    return -1;
}

4.3.2 編碼API例程

int av_encode_frame(AVCodecContext *enc_ctx, AVFrame *frame, AVPacket *packet)
{
    int ret = -1;
    
    // 第一次發送flush packet會返回成功，進入沖洗模式，可調用avcodec_receive_packet()
    // 將編碼器中緩存的幀(可能不止一個)取出來
    // 後續再發送flush packet將返回AVERROR_EOF
    ret = avcodec_send_frame(enc_ctx, frame);
    if (ret == AVERROR_EOF)
    {
        //av_log(NULL, AV_LOG_INFO, "avcodec_send_frame() encoder flushed\n");
    }
    else if (ret == AVERROR(EAGAIN))
    {
        //av_log(NULL, AV_LOG_INFO, "avcodec_send_frame() need output read out\n");
    }
    else if (ret < 0)
    {
        //av_log(NULL, AV_LOG_INFO, "avcodec_send_frame() error %d\n", ret);
        return ret;
    }

    ret = avcodec_receive_packet(enc_ctx, packet);
    if (ret == AVERROR_EOF)
    {
        av_log(NULL, AV_LOG_INFO, "avcodec_recieve_packet() encoder flushed\n");
    }
    else if (ret == AVERROR(EAGAIN))
    {
        //av_log(NULL, AV_LOG_INFO, "avcodec_recieve_packet() need more input\n");
    }
    
    return ret;
}