H264VideoToolBox硬件解碼

這套硬解碼API是幾個純C函數，在任何OC或者 C++代碼裏均可以使用。

首先要把 VideoToolbox.framework 添加到工程裏，而且包含如下頭文件。 #include <VideoToolbox/VideoToolbox.h>

解碼主要須要如下三個函數

VTDecompressionSessionCreate 建立解碼 session

VTDecompressionSessionDecodeFrame 解碼一個frame

VTDecompressionSessionInvalidate 銷燬解碼 session

首先要建立 decode session，方法以下：

OSStatus status = VTDecompressionSessionCreate(kCFAllocatorDefault,
                                              decoderFormatDescription,
                                              NULL, attrs,
                                              &callBackRecord,
                                              &deocderSession);

其中 decoderFormatDescription 是 CMVideoFormatDescriptionRef 類型的視頻格式描述，這個須要用H.264的 sps 和 pps數據來建立，調用如下函數建立 decoderFormatDescription

CMVideoFormatDescriptionCreateFromH264ParameterSets

須要注意的是，這裏用的 sps和pps數據是不包含「00 00 00 01」的start code的。

attr是傳遞給decode session的屬性詞典

CFDictionaryRef attrs = NULL;
        const void *keys[] = { kCVPixelBufferPixelFormatTypeKey };
// kCVPixelFormatType_420YpCbCr8Planar is YUV420
// kCVPixelFormatType_420YpCbCr8BiPlanarFullRange is NV12
        uint32_t v = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange;
        const void *values[] = { CFNumberCreate(NULL, kCFNumberSInt32Type, &v) };
        attrs = CFDictionaryCreate(NULL, keys, values, 1, NULL, NULL);

其中重要的屬性就一個，kCVPixelBufferPixelFormatTypeKey，指定解碼後的圖像格式，必須指定成NV12，蘋果的硬解碼器只支持NV12。

callBackRecord 是用來指定回調函數的，解碼器支持異步模式，解碼後會調用這裏的回調函數。

若是 decoderSession建立成功就能夠開始解碼了。

VTDecodeFrameFlags flags = 0;
            //kVTDecodeFrame_EnableTemporalProcessing | kVTDecodeFrame_EnableAsynchronousDecompression;
            VTDecodeInfoFlags flagOut = 0;
            CVPixelBufferRef outputPixelBuffer = NULL;
            OSStatus decodeStatus = VTDecompressionSessionDecodeFrame(deocderSession,
                                                                      sampleBuffer,
                                                                      flags,
                                                                      &outputPixelBuffer,
                                                                      &flagOut);

其中 flags 用0 表示使用同步解碼，這樣比較簡單。 其中 sampleBuffer是輸入的H.264視頻數據，每次輸入一個frame。 先用CMBlockBufferCreateWithMemoryBlock 從H.264數據建立一個CMBlockBufferRef實例。 而後用 CMSampleBufferCreateReady建立CMSampleBufferRef實例。 這裏要注意的是，傳入的H.264數據須要Mp4風格的，就是開始的四個字節是數據的長度而不是「00 00 00 01」的start code，四個字節的長度是big-endian的。 通常來講從 視頻裏讀出的數據都是 「00 00 00 01」開頭的，這裏須要本身轉換下。

解碼成功以後，outputPixelBuffer裏就是一幀 NV12格式的YUV圖像了。 若是想獲取YUV的數據能夠經過

CVPixelBufferLockBaseAddress(outputPixelBuffer, 0);
    void *baseAddress = CVPixelBufferGetBaseAddress(outputPixelBuffer);

得到圖像數據的指針，須要說明baseAddress並非指向YUV數據，而是指向一個CVPlanarPixelBufferInfo_YCbCrBiPlanar結構體，結構體裏記錄了兩個plane的offset和pitch。

可是若是想把視頻播放出來是不須要去讀取YUV數據的，由於CVPixelBufferRef是能夠直接轉換成OpenGL的Texture或者UIImage的。

調用CVOpenGLESTextureCacheCreateTextureFromImage，能夠直接建立OpenGL Texture

從 CVPixelBufferRef 建立 UIImage

CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pixelBuffer];
    UIImage *uiImage = [UIImage imageWithCIImage:ciImage];

解碼完成後銷燬 decoder session

VTDecompressionSessionInvalidate(deocderSession)

源碼

@implementation WXVideoDecoder{
    //幀
    NALUnit frame_buffer;
    
    //sps
    NALUnit sps_buffer;
    
    //pps
    NALUnit pps_buffer;
    
    uint8_t *_buffer;
    long _bufferSize;
    long _maxSize;
    
    //解碼會話
    VTDecompressionSessionRef wx_decodeSession;
    //描述
    CMFormatDescriptionRef  wx_formatDescription;
    
}

- (WXResult)decodeWithPath:(NSString *)path complete:(VideoDecodeCompleteBlock)complete{
    self.completeBlock = [complete copy];
    self.inputStream = [NSInputStream inputStreamWithFileAtPath:path];
    [self.inputStream open];
    
    _bufferSize = 0;
    _maxSize = 10000*1000;
    _buffer = malloc(_maxSize);
    
    //循環讀取
    while (true) {
        //讀數據
        if ([self readStream] == NO) {
            NSLog(@"播放結束");
            break;
        }
        
        //轉換
        uint32_t nalSize = (uint32_t)(frame_buffer.size - 4);
        uint32_t *pNalSize = (uint32_t *)frame_buffer.data;
        *pNalSize = CFSwapInt32HostToBig(nalSize);
        
        //存放像素信息
        CVPixelBufferRef pixelBuffer = NULL;
        //NAL的類型(startCode後的第一個字節的後5位)
        int NAL_type = frame_buffer.data[4] & 0x1f;
        switch (NAL_type) {
            case 0x5:
                NSLog(@"Nal type is IDR frame");
                if (!wx_decodeSession){
                    [self setupDecodeSession];
                }
                pixelBuffer = [self decode];
                break;
            case 0x7:
                NSLog(@"Nal type is SPS");
                //從幀中獲取sps信息
                sps_buffer.size = frame_buffer.size-4;
                if (!sps_buffer.data){
                    sps_buffer.data = malloc(sps_buffer.size);
                }
                memcpy(sps_buffer.data, frame_buffer.data+4, sps_buffer.size);
                break;
            case 0x8:
                NSLog(@"Nal type is PPS");
                //從幀中獲取sps信息
                pps_buffer.size = frame_buffer.size-4;
                if (!pps_buffer.data){
                    pps_buffer.data = malloc(pps_buffer.size);
                }
                memcpy(pps_buffer.data, frame_buffer.data+4, pps_buffer.size);
                break;
            default:
                //圖像信息
                NSLog(@"Nal type is B/P frame or another");
                pixelBuffer = [self decode];
                break;
        }
        if (pixelBuffer) {
            NSLog(@">>> %@",pixelBuffer);

            //同步保證數據信息不釋放
            dispatch_sync(dispatch_get_main_queue(), ^{
                if (self.completeBlock){
                    self.completeBlock(pixelBuffer);
                }
            });
            CVPixelBufferRelease(pixelBuffer);
        }
    }
    return WXResultNoErr;
}

//解碼會話
- (void)setupDecodeSession{
    const uint8_t * const paramSetPointers[2] = {sps_buffer.data,pps_buffer.data};
    const size_t paramSetSize[2] = {sps_buffer.size,pps_buffer.size};
    OSStatus status = CMVideoFormatDescriptionCreateFromH264ParameterSets(kCFAllocatorDefault, 2, paramSetPointers, paramSetSize, 4, &wx_formatDescription);
    
    if (status == noErr){
        CFDictionaryRef attrs = NULL;
        const void *keys[] = { kCVPixelBufferPixelFormatTypeKey };
        //      kCVPixelFormatType_420YpCbCr8Planar is YUV420
        //      kCVPixelFormatType_420YpCbCr8BiPlanarFullRange is NV12
        uint32_t v = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange;
        const void *values[] = { CFNumberCreate(NULL, kCFNumberSInt32Type, &v) };
        attrs = CFDictionaryCreate(NULL, keys, values, 1, NULL, NULL);
        
        //結束後的回調
        VTDecompressionOutputCallbackRecord callBackRecord;
        callBackRecord.decompressionOutputCallback = didDecompress;
        callBackRecord.decompressionOutputRefCon = NULL;
        //填入null，videottoolbox選擇解碼器
        status = VTDecompressionSessionCreate(kCFAllocatorDefault, wx_formatDescription, NULL, attrs, &callBackRecord, &wx_decodeSession);
        
        if (status!=noErr){
            NSLog(@"解碼會話建立失敗");
        }
        CFRelease(attrs);
    }else {
        NSLog(@"建立FormatDescription失敗");
    }
}

- (BOOL)readStream{
    
    if (_bufferSize<_maxSize && self.inputStream.hasBytesAvailable) {
        //正數：讀取的字節數，0：讀取到尾部，-1：讀取錯誤
        NSInteger readSize = [self.inputStream read:_buffer+_bufferSize maxLength:_maxSize-_bufferSize];
        _bufferSize += readSize;
    }
    //對比buffer的前四位是不是startCode(每一幀前都有startCode)，而且數據長度須要大於startCode
    if (memcmp(_buffer, startCode, 4) == 0 && _bufferSize > 4){
        //buffer的起始和結束位置
        uint8_t *startPoint = _buffer + 4;
        uint8_t *endPoint = _buffer + _bufferSize;
        while (startPoint != endPoint) {
            //獲取當前幀長度（經過獲取到下一個0x00000001,來肯定）
            if (memcmp(startPoint, startCode, 4) == 0){
                //找到下一幀，計算幀長
                frame_buffer.size = (unsigned int)(startPoint - _buffer);
                //置空幀
                if (frame_buffer.data){
                    free(frame_buffer.data);
                    frame_buffer.data = NULL;
                }
                frame_buffer.data = malloc(frame_buffer.size);
                //從緩衝區內複製當前幀長度的信息賦值給幀
                memcpy(frame_buffer.data, _buffer, frame_buffer.size);
                //緩衝區中數據去掉幀數據（長度減小，地址移動）
                memmove(_buffer, _buffer+frame_buffer.size, _bufferSize-frame_buffer.size);
                _bufferSize -= frame_buffer.size;
                
                return YES;
            }else{
                //若是不是，移動指針
                startPoint++;
            }
        }
    }
    return NO;
}

//解碼
- (CVPixelBufferRef)decode {
    CVPixelBufferRef outputPixelBuffer = NULL;
    //視頻圖像數據
    CMBlockBufferRef blockBuffer = NULL;
    OSStatus status  = CMBlockBufferCreateWithMemoryBlock(kCFAllocatorDefault,
                                                          (void*)frame_buffer.data,
                                                          frame_buffer.size,
                                                          kCFAllocatorNull,
                                                          NULL,
                                                          0,
                                                          frame_buffer.size,
                                                          0,
                                                          &blockBuffer);
    if(status == kCMBlockBufferNoErr) {
        CMSampleBufferRef sampleBuffer = NULL;
        const size_t sampleSizeArray[] = {frame_buffer.size};
        status = CMSampleBufferCreateReady(kCFAllocatorDefault,
                                           blockBuffer,
                                           wx_formatDescription ,
                                           1,
                                           0,
                                           NULL,
                                           1,
                                           sampleSizeArray,
                                           &sampleBuffer);
        if (status == kCMBlockBufferNoErr && sampleBuffer) {
            VTDecodeFrameFlags flags = 0;
            VTDecodeInfoFlags flagOut = 0;
            OSStatus decodeStatus = VTDecompressionSessionDecodeFrame(wx_decodeSession,
                                                                      sampleBuffer,
                                                                      flags,
                                                                      &outputPixelBuffer,
                                                                      &flagOut);
            
            if(decodeStatus == kVTInvalidSessionErr) {
                NSLog(@"IOS8VT: Invalid session, reset decoder session");
            } else if(decodeStatus == kVTVideoDecoderBadDataErr) {
                NSLog(@"IOS8VT: decode failed status=%d(Bad data)", (int)decodeStatus);
            } else if(decodeStatus != noErr) {
                NSLog(@"IOS8VT: decode failed status=%d", (int)decodeStatus);
            }
            
            CFRelease(sampleBuffer);
        }
        CFRelease(blockBuffer);
    }
    free(frame_buffer.data);
    frame_buffer.data = NULL;
    return outputPixelBuffer;
}

- (WXResult)destroy {
    
    if (wx_decodeSession) {
        VTDecompressionSessionInvalidate(wx_decodeSession);
        CFRelease(wx_decodeSession);
        wx_decodeSession = NULL;
    }
    if (wx_formatDescription) {
        CFRelease(wx_formatDescription);
        wx_formatDescription = NULL;
    }
    
    return WXResultNoErr;
}


//解碼回調結束 （使用VTDecompressionSessionDecodeFrameWithOutputHandler，直接接受處理結果）
static void didDecompress(void *decompressionOutputRefCon, void *sourceFrameRefCon, OSStatus status, VTDecodeInfoFlags infoFlags, CVImageBufferRef imageBuffer, CMTime presentationTimeStamp, CMTime presentationDuration ){
    CVPixelBufferRef *outputPixelBuffer = (CVPixelBufferRef*)sourceFrameRefCon;
    *outputPixelBuffer = CVPixelBufferRetain(imageBuffer);
}

硬解碼的基本流程就是這樣了，若是須要成功解碼播放視頻還須要一些H.264視頻格式，YUV圖像格式，OpenGL等基礎知識。 具體流程能夠參考github上的例子，https://github.com/adison/-VideoToolboxDemo、https://github.com/simonwxun/WXH264AndAACDemo.git