關(guān)于VideoToolBox 解碼 H264 用押，這次我們通過(guò) ffmpeg 提取一個(gè)視頻流的 的視頻流汞舱，也就是 h264 編碼格式的視頻流（沒(méi)有音頻）；

命令如下：

ffmpeg -i /Users/pengchao/Downloads/download.mp4 -codec copy  -f h264 output.h264

1. 獲取 NALU 單元

demo中，我們首先把h264 文件讀到內(nèi)存中柠偶，通過(guò)創(chuàng)建定時(shí)器，來(lái)讀取 一個(gè)NALU單元睬关；該步驟重點(diǎn)是如何在文件流中找到 NALU 單元诱担，眾所周知 ，每個(gè) NALU單元前面都有起始碼 0x00 0x00 0x00 0x01 或 0x00 0x00 0x01來(lái)分割 NALU 單元共螺； 這里 我們畫(huà)圖來(lái)解釋 如何通過(guò)指針移動(dòng)來(lái)找到 一個(gè)NALU 單元该肴，并拿到NALU 單元的長(zhǎng)度情竹，從而獲取到 一個(gè)完整NALU藐不；

image.png

源碼邏輯可參考如下代碼：

- (void)tick {
    
    dispatch_sync(_decodeQueue, ^{
        //1.獲取packetBuffer和packetSize
        packetSize = 0;
        if (packetBuffer) {
            free(packetBuffer);
            packetBuffer = NULL;
        }
        if (_inputSize < _inputMaxSize && _inputStream.hasBytesAvailable) { //一般情況下只會(huì)執(zhí)行一次,使得inputMaxSize等于inputSize
            _inputSize += [_inputStream read:_inputBuffer + _inputSize maxLength:_inputMaxSize - _inputSize];
        }
        if ((memcmp(_inputBuffer, startCode, 4) == 0) && (_inputSize > 4)) {
            
            uint8_t *pStart = _inputBuffer + 4;         //pStart 表示 NALU 的起始指針
            uint8_t *pEnd = _inputBuffer + _inputSize;  //pEnd 表示 NALU 的末尾指針
            while (pStart != pEnd) {                    //這里使用一種簡(jiǎn)略的方式來(lái)獲取這一幀的長(zhǎng)度：通過(guò)查找下一個(gè)0x00000001來(lái)確定。
                if(memcmp(pStart - 3, startCode, 4) == 0 ) {
                    packetSize = pStart - _inputBuffer - 3;
                    if (packetBuffer) {
                        free(packetBuffer);
                        packetBuffer = NULL;
                    }
                    packetBuffer = malloc(packetSize);
                    memcpy(packetBuffer, _inputBuffer, packetSize); //復(fù)制packet內(nèi)容到新的緩沖區(qū)
                    memmove(_inputBuffer, _inputBuffer + packetSize, _inputSize - packetSize); //把緩沖區(qū)前移
                    _inputSize -= packetSize;
                    break;
                }
                else {
                    ++pStart;
                }
            }
        }
        if (packetBuffer == NULL || packetSize == 0) {
            [self endDecode];
            return;
        }
        /// 拿到NALU 的首地址和 長(zhǎng)度后秦效，解析該NALU
}

2. 獲取SPS 和PPS

在上一篇文章中雏蛮，我們首先保存的是SPS和PPS 數(shù)據(jù)，所以在文件流的讀取中阱州，我們應(yīng)該曉得第一個(gè)和第二個(gè)NALU分別是SPS和PPS挑秉，這正是我們創(chuàng)建VideoToolBox所需要的參數(shù)；
在解析NALU的時(shí)候苔货，還是要再講一下 H264 碼流的結(jié)構(gòu)犀概。H264碼流是由一個(gè)個(gè)的NAL單元組成，其中SPS夜惭、PPS姻灶、IDR和SLICE是NAL單元某一類型的數(shù)據(jù)。

如下圖所示：

image.png

所以在找到 start code后诈茧，第一個(gè)字節(jié)為NALU Header 产喉，通過(guò)NALU Header判斷這是一個(gè)什么類型的NALU；
關(guān)于 NALU Header 的結(jié)構(gòu)：

• 第 0位 F
• 第1-2 位 NRI
• 第3-7位:TYPE

image.png

關(guān)于NALU 類型的定義我們可以參考下圖：

image.png

解析NALU 的代碼如所示：

        //2.將packet的前4個(gè)字節(jié)換成大端的長(zhǎng)度
        //大端：高字節(jié)保存在低地址
        //小端：高字節(jié)保存在高地址
        //大小端的轉(zhuǎn)換實(shí)際上及時(shí)將字節(jié)順序換一下即可
        uint32_t nalSize = (uint32_t)(packetSize - 4);
        uint8_t *pNalSize = (uint8_t*)(&nalSize);
        packetBuffer[0] = pNalSize[3];
        packetBuffer[1] = pNalSize[2];
        packetBuffer[2] = pNalSize[1];
        packetBuffer[3] = pNalSize[0];
        
        //3.判斷幀類型（根據(jù)碼流結(jié)構(gòu)可知，startcode后面緊跟著就是碼流的類型）
        int nalType = packetBuffer[4] & 0x1f;
        switch (nalType) {
            case 0x05:
                //IDR frame
                [self initDecodeSession];
                [self decodePacket];
                break;
            case 0x07:
                //sps
                if (_sps) { _sps = nil;}
                size_t spsSize = (size_t) packetSize - 4;
                uint8_t *sps = malloc(spsSize);
                memcpy(sps, packetBuffer+4, spsSize);
                _sps = [NSData dataWithBytes:sps length:spsSize];
                break;
            case 0x08:
                //pps
                if (_pps) { _pps = nil; }
                size_t ppsSize = (size_t) packetSize - 4;
                uint8_t *pps = malloc(ppsSize);
                memcpy(pps, packetBuffer+4, ppsSize);
                _pps = [NSData dataWithBytes:pps length:ppsSize];
                break;
            default:
                // B/P frame
                [self decodePacket];
                break;
        }
    });

3. 創(chuàng)建 VideoToolBox

在拿到 sps 和pps后曾沈，創(chuàng)建videoTooBox这嚣；
如果沒(méi)有sps和pps我們 需要 xxx 來(lái)創(chuàng)建 videoToolBox；

-(void)initVideoToolBox {
    
    if (_decodeSession) {
        return;
    }
    
    CMFormatDescriptionRef formatDescriptionOut;
    const uint8_t * const param[2] = {_sps.bytes,_pps.bytes};
    const size_t paramSize[2] = {_sps.length,_pps.length};
    OSStatus formateStatus =
    CMVideoFormatDescriptionCreateFromH264ParameterSets(NULL,
                                                        2,
                                                        param,
                                                        paramSize,
                                                        4,
                                                        &formatDescriptionOut);
    _formatDescriptionOut = formatDescriptionOut;
    
    if (formateStatus!=noErr) {
        NSLog(@"FormatDescriptionCreate fail");
        return;
    }
    //2. 創(chuàng)建VTDecompressionSessionRef
    //確定編碼格式
    const void *keys[] = {kCVPixelBufferPixelFormatTypeKey};
    
    uint32_t t = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange;
    const void *values[] = {CFNumberCreate(NULL, kCFNumberSInt32Type, &t)};
    
    CFDictionaryRef att = CFDictionaryCreate(NULL, keys, values, 1, NULL, NULL);
    
    VTDecompressionOutputCallbackRecord VTDecompressionOutputCallbackRecord;
    VTDecompressionOutputCallbackRecord.decompressionOutputCallback = decodeCompressionOutputCallback;
    VTDecompressionOutputCallbackRecord.decompressionOutputRefCon = (__bridge void * _Nullable)(self);
    
    OSStatus sessionStatus = VTDecompressionSessionCreate(NULL,
                                 formatDescriptionOut,
                                 NULL,
                                 att,
                                 &VTDecompressionOutputCallbackRecord,
                                 &_decodeSession);
    CFRelease(att);
    if (sessionStatus != noErr) {
        NSLog(@"SessionCreate fail");
        [self endDecode];
    }
}

4.解碼NALU 單元

再拿到 關(guān)鍵關(guān)鍵幀后塞俱，我們 通過(guò)NSData 構(gòu)造videoToolBox 需要的sampleBuffe 姐帚；并送入編碼器；

關(guān)于解碼的源碼如下：

- (void)encoderWithData:(NSData *)data{
    if (!_decodeSession) {
        return;
    }
    //1.創(chuàng)建CMBlockBufferRef
    CMBlockBufferRef blockBuffer = NULL;
    OSStatus blockBufferStatus =
    CMBlockBufferCreateWithMemoryBlock(kCFAllocatorDefault,
                                       data.bytes,
                                       data.length,
                                       NULL,
                                       NULL,
                                       0,
                                       data.length,
                                       0,
                                       &blockBuffer);
    if (blockBufferStatus!=noErr) {
        NSLog(@"BolkBufferCreate fail");
        return;
    }
    //2.創(chuàng)建CMSampleBufferRef
    CMSampleBufferRef sampleBuffer = NULL;
    const size_t sampleSizeArray[] = {data.length};
    OSStatus sampleBufferStatus =
    CMSampleBufferCreateReady(kCFAllocatorDefault,
                              blockBuffer,
                              _formatDescriptionOut,
                              1, //sample 的數(shù)量
                              0, //sampleTimingArray 的長(zhǎng)度
                              NULL, //sampleTimingArray 對(duì)每一個(gè)設(shè)置一些屬性障涯，這些我們并不需要
                              1, //sampleSizeArray 的長(zhǎng)度
                              sampleSizeArray,
                              &sampleBuffer);
    
    if (blockBuffer && sampleBufferStatus == kCMBlockBufferNoErr) {
        //3.編碼生成
        VTDecodeFrameFlags flags = 0;
        VTDecodeInfoFlags flagOut = 0;
        OSStatus decodeStatus = VTDecompressionSessionDecodeFrame(_decodeSession,
                                          sampleBuffer,flags,
                                          NULL,
                                          &flagOut); //receive information about the decode operation
        if (decodeStatus!= noErr) {
            NSLog(@"DecodeFrame fail %d",(int)decodeStatus);
            return;
        }
    }
    if (sampleBufferStatus != noErr) {
        NSLog(@"SampleBufferCreate fail");
        return;
    }
}

5.獲取解碼后的pixelBuffer圖像信息

解碼成功后的回調(diào)

static void decodeCompressionOutputCallback(void * CM_NULLABLE decompressionOutputRefCon,
                                      void * CM_NULLABLE sourceFrameRefCon,
                                      OSStatus status,
                                      VTDecodeInfoFlags infoFlags,
                                      CM_NULLABLE CVImageBufferRef imageBuffer,
                                      CMTime presentationTimeStamp,
                                      CMTime presentationDuration ){
    
    VideoDecoder *self = (__bridge VideoDecoder *)(decompressionOutputRefCon);
    dispatch_queue_t callbackQuque = self ->_decodeCallbackQueue;
    
    CIImage *ciimage = [CIImage imageWithCVPixelBuffer:imageBuffer];
    UIImage *image = [UIImage imageWithCIImage:ciimage];
    if (imageBuffer && [self.delegate respondsToSelector:@selector(videoDecoderCallbackPixelBuffer:)]) {
        CIImage *ciimage = [CIImage imageWithCVPixelBuffer:imageBuffer];
        UIImage *image = [UIImage imageWithCIImage:ciimage];
        dispatch_async(callbackQuque, ^{
            [self.delegate videoDecoderCallbackPixelBuffer:image];
        });
    }
}

6. 總結(jié)

源碼地址： https://github.com/hunter858/OpenGL_Study/AVFoundation/VideoToolBox-decoder