"本文轉(zhuǎn)載自：[yanbixing123]的Android MultiMedia框架完全解析 - MediaCodec解析"

1.概述

??MediaCodec是一個(gè)Codec掂为，通過(guò)硬件加速解碼和編碼煌寇。它為芯片廠商和應(yīng)用開(kāi)發(fā)者搭建了一個(gè)統(tǒng)一接口缸逃。MediaCodec幾乎是所有安卓播放器硬解的標(biāo)配催首，要深入分析一個(gè)播放器的源碼雅任，如NuPlayer姐仅，ijkplayer属铁，有必要了解其基礎(chǔ)的使用方法。

??先來(lái)看看MediaCodec在NuPlayer中的位置：

01.png

同樣莽鸿，想要深入了解MediaCodec，首先需要知道它在Android App中是如何使用的拾给。

1.1 MediaCodecList

??MediaCodec是一個(gè)統(tǒng)一API祥得，支持不同編碼格式，在創(chuàng)建MediaCodec的時(shí)候需要根據(jù)視頻編碼選擇合適的解碼器蒋得。這是通過(guò) MediaCodec.createByCodecName 完成的级及。

??然而不同廠商提供的解碼器名稱有所不同，編寫(xiě)通用播放器的時(shí)候额衙，無(wú)法預(yù)見(jiàn)饮焦。所以Android API中提供了一個(gè)MediaCodecList用于枚舉設(shè)備支持的編解碼器的名字怕吴、能力，以查找合適的編解碼器县踢。

??我們先枚舉下設(shè)備支持的解碼器：

private void displayDecoders() {                                            
    MediaCodecList list = new MediaCodecList(MediaCodecList.REGULAR_CODECS);
    MediaCodecInfo[] codecs = list.getCodecInfos();                         
    for (MediaCodecInfo codec : codecs) {                                   
        if (codec.isEncoder())                                              
            continue;                                                       
        Log.i(TAG, "displayDecoders: "+codec.getName());                    
    }                                                                       
}

1.2 創(chuàng)建MediaCodec

??所以转绷，一種比較合理創(chuàng)建MediaCodec的方法是：

MediaFormat selTrackFmt = chooseVideoTrack(extractor);
codec = createCodec(selTrackFmt, surface);

private MediaFormat chooseVideoTrack(MediaExtractor extractor) {         
    int count = extractor.getTrackCount();                               
    for (int i = 0; i < count; i++) {                                    
        MediaFormat format = extractor.getTrackFormat(i);                
        if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")){
            extractor.selectTrack(i);//選擇軌道                                    
            return format;                                               
        }                                                                
    }                                                                    
    return null;                                                         
}  

private MediaCodec createCodec(MediaFormat format, Surface surface) throws IOException{     
    MediaCodecList codecList = new MediaCodecList(MediaCodecList.REGULAR_CODECS);           
    MediaCodec codec = MediaCodec.createByCodecName(codecList.findDecoderForFormat(format));
    codec.configure(format, surface, null, 0);                                              
    return codec;                                                                           
}

先使用MediaExtractor來(lái)選擇需要的軌道，這里我們簡(jiǎn)單選擇第一條視頻軌道硼啤。軌道選擇后议经，從MediaExtractor那里取到了目標(biāo)軌道的MediaFormat，然后就可以通過(guò)codecList.findDecoderForFormat(format)獲取到最合適的解碼器了谴返。

1.3 MediaCodec的使用方式

??MediaCodec有兩種使用方式 —— 同步和異步煞肾。MediaCodec可以處理具體的視頻流，主要有這幾個(gè)方法：

• getInputBuffers：獲取需要編碼數(shù)據(jù)的輸入流隊(duì)列嗓袱，返回的是一個(gè)ByteBuffer數(shù)組籍救；

• queueInputBuffer：輸入流入隊(duì)列；

• dequeueInputBuffer：從輸入流隊(duì)列中取數(shù)據(jù)進(jìn)行編碼操作索抓；

• getOutputBuffers：獲取編解碼之后的數(shù)據(jù)輸出流隊(duì)列钧忽，返回的是一個(gè)ByteBuffer數(shù)組；

• dequeueOutputBuffer：從輸出隊(duì)列中取出編碼操作之后的數(shù)據(jù)逼肯；

• releaseOutputBuffer：處理完成耸黑，釋放ByteBuffer數(shù)據(jù)。

1.3.1 同步方法

??同步的主流程是在一個(gè)循環(huán)內(nèi)不斷調(diào)用dequeueInputBuffer -> queueInputBuffer填充數(shù)據(jù) -> dequeueOutputBuffer -> releaseOutputBuffer顯示畫(huà)面篮幢。與MediaExtractor配合大刊，給MediaCodec喂數(shù)據(jù)的一般流程是這樣的：

if (inIndex >= 0) {                                                                    
    ByteBuffer buffer = codec.getInputBuffer(inIndex);                                 
    int sampleSize = extractor.readSampleData(buffer, 0);                              
    if (sampleSize < 0) {                                                              
        codec.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
        isEOS = true;                                                                  
    } else {                                                                           
        long sampleTime = extractor.getSampleTime();                                   
        codec.queueInputBuffer(inIndex, 0, sampleSize, sampleTime, 0);                 
        extractor.advance();                                                           
    }                                                                                  
}

通過(guò)MediaExtractor的readSampleData和advance我們可以從視頻文件中不斷取到所選軌道的未解碼數(shù)據(jù)。當(dāng)sampleSize大于等于0三椿，說(shuō)明取到了數(shù)據(jù)缺菌，通過(guò)queueInputBuffer通知MediaCodec inIndex的input buffer已經(jīng)準(zhǔn)備好了。如果返回值小于0搜锰，說(shuō)明已經(jīng)到的文件末尾伴郁，此時(shí)可以通過(guò)BUFFER_FLAG_END_OF_STREAM標(biāo)記通知MediaCodec已經(jīng)達(dá)到文件末尾。

??從MediaCodec取解碼后的數(shù)據(jù)（實(shí)際拿到的是一個(gè)Index）蛋叼，一般流程是這樣的：

int outIndex = codec.dequeueOutputBuffer(info, 10000);          
Log.i(TAG, "run: outIndex="+outIndex);                          
switch (outIndex) {                                             
    case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:                 
        Log.i(TAG, "run: new format: "+codec.getOutputFormat());
        break;                                                  
    case MediaCodec.INFO_TRY_AGAIN_LATER:                       
        Log.i(TAG, "run: try later");                           
        break;                                                  
    case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:                
        Log.i(TAG, "run: output buffer changed");               
        break;                                                  
    default:                                                    
        codec.releaseOutputBuffer(outIndex, true);              
        break;                                                  
}                                                               

if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) { 
    Log.i(TAG, "OutputBuffer BUFFER_FLAG_END_OF_STREAM");       
    break;                                                      
}

dequeueOutputBuffer從MediaCodec中獲取已經(jīng)解碼好的一幀的索引焊傅，再通過(guò)releaseOutputBuffer(outIndex, true)就可以讓MediaCodec將這一幀輸出到Surface上。(如果想控制播放速率狈涮，或丟幀狐胎，可以在這里控制)

??dequeueOutputBuffer還會(huì)在特殊情況返回一些輔助值，如視頻格式變化歌馍、或解碼數(shù)據(jù)未準(zhǔn)備好等握巢。

1.3.2 異步方法

codec.setCallback(new MediaCodec.Callback() {                                                     
    @Override                                                                                     
    public void onInputBufferAvailable(MediaCodec codec, int index) {                             
        ByteBuffer buffer = codec.getInputBuffer(index);                                          
        int sampleSize = extractor.readSampleData(buffer, 0);                                     
        if (sampleSize < 0) {                                                                     
            codec.queueInputBuffer(index, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);         
        } else {                                                                                  
            long sampleTime = extractor.getSampleTime();                                          
            codec.queueInputBuffer(index, 0, sampleSize, sampleTime, 0);                          
            extractor.advance();                                                                  
        }                                                                                         
    }                                                                                             

    @Override                                                                                     
    public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo info) {
        codec.releaseOutputBuffer(index, true);                                                   
    }                                                                                             

    @Override                                                                                     
    public void onError(MediaCodec codec, MediaCodec.CodecException e) {                          
        Log.e(TAG, "onError: "+e.getMessage());                                                   
    }                                                                                             

    @Override                                                                                     
    public void onOutputFormatChanged(MediaCodec codec, MediaFormat format) {                     
        Log.i(TAG, "onOutputFormatChanged: "+format);                                             
    }                                                                                             
});                                                                                               
codec.start();

異步播放的方式是給MediaCodec注冊(cè)了一個(gè)回調(diào)，由MediaCodec通知何時(shí)input buffer可用松却，何時(shí)output buffer可用暴浦，何時(shí)format changed等溅话。

??我們需要做的就是，當(dāng)inuput buffer可用時(shí)肉渴，就把MediaExtractor提取的數(shù)據(jù)填充給指定buffer公荧；當(dāng)output buffer可用時(shí)，決定是否顯示該幀（實(shí)際應(yīng)用中同规，不會(huì)立即顯示它循狰，而是要根據(jù)fps控制顯示速度）。

2.MediaCodec實(shí)現(xiàn)

??根據(jù)上面的MediaCodec的使用方式券勺，提煉出來(lái)的幾行重要代碼：

MediaCodec codec = MediaCodec.createByCodecName("video/avc");
MediaFormat format = MediaFormat.createVideoFormat("video/avc", 320, 480);
// format配置的代碼...
codec.configure(format, surface, null, 0);
codec.start()

2.1 createByCodecName()

??先來(lái)看下MediaCodec.createByCodecName()方法绪钥，這個(gè)方法事實(shí)上包裝一些參數(shù)后來(lái)到：

• MediaCodec.java

private MediaCodec(
            @NonNull String name, boolean nameIsType, boolean encoder) {
        Looper looper;
        if ((looper = Looper.myLooper()) != null) {
            mEventHandler = new EventHandler(this, looper);
        } else if ((looper = Looper.getMainLooper()) != null) {
            mEventHandler = new EventHandler(this, looper);
        } else {
            mEventHandler = null;
        }
        mCallbackHandler = mEventHandler;
        mOnFrameRenderedHandler = mEventHandler;

        mBufferLock = new Object();

        native_setup(name, nameIsType, encoder);
    }

初始化了一個(gè)Handler和一個(gè)緩沖對(duì)象的對(duì)象鎖。接著來(lái)看native_setup()方法：

static void android_media_MediaCodec_native_setup(
        JNIEnv *env, jobject thiz,
        jstring name, jboolean nameIsType, jboolean encoder) {
    // ... 省略部分代碼

    const char *tmp = env->GetStringUTFChars(name, NULL);

    if (tmp == NULL) {
        return;
    }

    sp codec = new JMediaCodec(env, thiz, tmp, nameIsType, encoder);

    const status_t err = codec->initCheck();
    // ... 省略部分代碼
    codec->registerSelf();

    setMediaCodec(env,thiz, codec);
}

看到JMediaCodec的構(gòu)造方法：

JMediaCodec::JMediaCodec(
        JNIEnv *env, jobject thiz,
        const char *name, bool nameIsType, bool encoder)
    : mClass(NULL),
      mObject(NULL) {
    jclass clazz = env->GetObjectClass(thiz);
    CHECK(clazz != NULL);

    mClass = (jclass)env->NewGlobalRef(clazz);
    mObject = env->NewWeakGlobalRef(thiz);

    cacheJavaObjects(env);

    mLooper = new ALooper;
    mLooper->setName("MediaCodec_looper");

    mLooper->start(
            false,      // runOnCallingThread
            true,       // canCallJava
            PRIORITY_FOREGROUND);

    if (nameIsType) {
        mCodec = MediaCodec::CreateByType(mLooper, name, encoder, &mInitStatus);
    } else {
        mCodec = MediaCodec::CreateByComponentName(mLooper, name, &mInitStatus);
    }
    CHECK((mCodec != NULL) != (mInitStatus != OK));
}

JMediaCodec同樣有一個(gè)mLooper成員變量关炼，這個(gè)變量是一個(gè)ALooper對(duì)象程腹，初始化mLooper后開(kāi)始start()，start()方法內(nèi)部會(huì)開(kāi)啟一個(gè)線程并run儒拂。nameIsType此時(shí)為true寸潦，來(lái)到MediaCodec::CreateByType()方法：

sp MediaCodec::CreateByType(
        const sp &looper, const AString &mime, bool encoder, status_t *err, pid_t pid) {
    // 創(chuàng)建MediaCodec對(duì)象
    sp codec = new MediaCodec(looper, pid);

    // MediaCodec初始化
    const status_t ret = codec->init(mime, true /* nameIsType */, encoder);
    if (err != NULL) {
        *err = ret;
    }
    return ret == OK ? codec : NULL; // NULL deallocates codec.
}

此時(shí)傳入的Looper對(duì)象是android_media_MediaCodec中JMediaCodec對(duì)象的mLooper成員變量，在Android消息隊(duì)列機(jī)制中一個(gè)線程只能和一個(gè)Looper實(shí)例對(duì)應(yīng)社痛。接著看MediaCodec的構(gòu)造函數(shù)：

MediaCodec::MediaCodec(const sp &looper, pid_t pid)
    : mState(UNINITIALIZED),
      mReleasedByResourceManager(false),
      mLooper(looper),
      mCodec(NULL),
      mReplyID(0),
      mFlags(0),
      mStickyError(OK),
      mSoftRenderer(NULL),
      mResourceManagerClient(new ResourceManagerClient(this)),
      mResourceManagerService(new ResourceManagerServiceProxy(pid)),
      mBatteryStatNotified(false),
      mIsVideo(false),
      mVideoWidth(0),
      mVideoHeight(0),
      mRotationDegrees(0),
      mDequeueInputTimeoutGeneration(0),
      mDequeueInputReplyID(0),
      mDequeueOutputTimeoutGeneration(0),
      mDequeueOutputReplyID(0),
      mHaveInputSurface(false),
      mHavePendingInputBuffers(false) {
}

就是參數(shù)列表见转，創(chuàng)建MediaCodec對(duì)象后，會(huì)執(zhí)行MediaCodec:init()方法：

status_t MediaCodec::init(const AString &name, bool nameIsType, bool encoder) {
    mResourceManagerService->init();

    // save init parameters for reset
    mInitName = name;
    mInitNameIsType = nameIsType;
    mInitIsEncoder = encoder;

    // Current video decoders do not return from OMX_FillThisBuffer
    // quickly, violating the OpenMAX specs, until that is remedied
    // we need to invest in an extra looper to free the main event
    // queue.

    // 這里獲取到的是ACodec
    mCodec = GetCodecBase(name, nameIsType);
    if (mCodec == NULL) {
        return NAME_NOT_FOUND;
    }

    bool secureCodec = false;
    if (nameIsType && !strncasecmp(name.c_str(), "video/", 6)) {
        mIsVideo = true;
    } else {
        AString tmp = name;
        if (tmp.endsWith(".secure")) {
            secureCodec = true;
            tmp.erase(tmp.size() - 7, 7);
        }
        // 獲取MediaCodecList實(shí)例
        const sp mcl = MediaCodecList::getInstance();
        if (mcl == NULL) {
            mCodec = NULL;  // remove the codec.
            return NO_INIT; // if called from Java should raise IOException
        }
        // 根據(jù)name查找解碼器
        ssize_t codecIdx = mcl->findCodecByName(tmp.c_str());
        if (codecIdx >= 0) {
            const sp info = mcl->getCodecInfo(codecIdx);
            Vector mimes;
            info->getSupportedMimes(&mimes);
            for (size_t i = 0; i < mimes.size(); i++) {
                if (mimes[i].startsWith("video/")) {
                    mIsVideo = true;
                    break;
                }
            }
        }
    }

    if (mIsVideo) {
        // video codec needs dedicated looper
        if (mCodecLooper == NULL) {
            mCodecLooper = new ALooper;
            mCodecLooper->setName("CodecLooper");
            mCodecLooper->start(false, false, ANDROID_PRIORITY_AUDIO);
        }

        mCodecLooper->registerHandler(mCodec);
    } else {
        mLooper->registerHandler(mCodec);
    }

    mLooper->registerHandler(this);

    mCodec->setNotificationMessage(new AMessage(kWhatCodecNotify, this));

    sp msg = new AMessage(kWhatInit, this);
    msg->setString("name", name);
    msg->setInt32("nameIsType", nameIsType);

    if (nameIsType) {
        msg->setInt32("encoder", encoder);
    }

    status_t err;
    Vector resources;
    MediaResource::Type type =
            secureCodec ? MediaResource::kSecureCodec : MediaResource::kNonSecureCodec;
    MediaResource::SubType subtype =
            mIsVideo ? MediaResource::kVideoCodec : MediaResource::kAudioCodec;
    resources.push_back(MediaResource(type, subtype, 1));
    for (int i = 0; i <= kMaxRetry; ++i) {
        if (i > 0) {
            // Don't try to reclaim resource for the first time.
            if (!mResourceManagerService->reclaimResource(resources)) {
                break;
            }
        }

        sp response;
        // 發(fā)送kWhatInit消息
        err = PostAndAwaitResponse(msg, &response);
        if (!isResourceError(err)) {
            break;
        }
    }
    return err;
}

這里用GetCodecBase()方法初始化了mCodec成員變量：

sp MediaCodec::GetCodecBase(const AString &name, bool nameIsType) {
    // at this time only ACodec specifies a mime type.
    if (nameIsType || name.startsWithIgnoreCase("omx.")) {
        return new ACodec;
    } else if (name.startsWithIgnoreCase("android.filter.")) {
        return new MediaFilter;
    } else {
        return NULL;
    }
}

這里nameIsType為true蒜哀，就會(huì)構(gòu)建一個(gè)ACodec實(shí)例：

ACodec::ACodec()
    : mQuirks(0),
      mNode(0),
      mUsingNativeWindow(false),
      mNativeWindowUsageBits(0),
      mLastNativeWindowDataSpace(HAL_DATASPACE_UNKNOWN),
      mIsVideo(false),
      mIsEncoder(false),
      mFatalError(false),
      mShutdownInProgress(false),
      mExplicitShutdown(false),
      mIsLegacyVP9Decoder(false),
      mEncoderDelay(0),
      mEncoderPadding(0),
      mRotationDegrees(0),
      mChannelMaskPresent(false),
      mChannelMask(0),
      mDequeueCounter(0),
      mInputMetadataType(kMetadataBufferTypeInvalid),
      mOutputMetadataType(kMetadataBufferTypeInvalid),
      mLegacyAdaptiveExperiment(false),
      mMetadataBuffersToSubmit(0),
      mNumUndequeuedBuffers(0),
      mRepeatFrameDelayUs(-1ll),
      mMaxPtsGapUs(-1ll),
      mMaxFps(-1),
      mTimePerFrameUs(-1ll),
      mTimePerCaptureUs(-1ll),
      mCreateInputBuffersSuspended(false),
      mTunneled(false),
      mDescribeColorAspectsIndex((OMX_INDEXTYPE)0),
      mDescribeHDRStaticInfoIndex((OMX_INDEXTYPE)0) {
    mUninitializedState = new UninitializedState(this);
    mLoadedState = new LoadedState(this);
    mLoadedToIdleState = new LoadedToIdleState(this);
    mIdleToExecutingState = new IdleToExecutingState(this);
    mExecutingState = new ExecutingState(this);

    mOutputPortSettingsChangedState =
        new OutputPortSettingsChangedState(this);

    mExecutingToIdleState = new ExecutingToIdleState(this);
    mIdleToLoadedState = new IdleToLoadedState(this);
    mFlushingState = new FlushingState(this);

    mPortEOS[kPortIndexInput] = mPortEOS[kPortIndexOutput] = false;
    mInputEOSResult = OK;

    memset(&mLastNativeWindowCrop, 0, sizeof(mLastNativeWindowCrop));

    changeState(mUninitializedState);
}

先回到剛才的MediaCodec::init()方法來(lái)斩箫，看到后面的一段代碼：

if (mIsVideo) {
        // video codec needs dedicated looper
        if (mCodecLooper == NULL) {
            mCodecLooper = new ALooper;
            mCodecLooper->setName("CodecLooper");
            mCodecLooper->start(false, false, ANDROID_PRIORITY_AUDIO);
        }

        mCodecLooper->registerHandler(mCodec);
    } else {
        mLooper->registerHandler(mCodec);
    }

    mLooper->registerHandler(this);

    mCodec->setNotificationMessage(new AMessage(kWhatCodecNotify, this));

    sp msg = new AMessage(kWhatInit, this);
    msg->setString("name", name);
    msg->setInt32("nameIsType", nameIsType);

    if (nameIsType) {
        msg->setInt32("encoder", encoder);
    }

    status_t err;
    Vector resources;
    MediaResource::Type type =
            secureCodec ? MediaResource::kSecureCodec : MediaResource::kNonSecureCodec;
    MediaResource::SubType subtype =
            mIsVideo ? MediaResource::kVideoCodec : MediaResource::kAudioCodec;
    resources.push_back(MediaResource(type, subtype, 1));
    for (int i = 0; i <= kMaxRetry; ++i) {
        if (i > 0) {
            // Don't try to reclaim resource for the first time.
            if (!mResourceManagerService->reclaimResource(resources)) {
                break;
            }
        }

        sp response;
        // 發(fā)送kWhatInit消息
        err = PostAndAwaitResponse(msg, &response);
        if (!isResourceError(err)) {
            break;
        }
    }

在這里init()方法發(fā)送了一條kWhatInit的Msg，對(duì)應(yīng)的回調(diào)MediaCodec::onMessageReceived()方法：

case kWhatInit:
        {
            sp replyID;
            CHECK(msg->senderAwaitsResponse(&replyID));

            if (mState != UNINITIALIZED) {
                PostReplyWithError(replyID, INVALID_OPERATION);
                break;
            }

            mReplyID = replyID;
            // 設(shè)置MedaiCodec的狀態(tài)
            setState(INITIALIZING);

            AString name;
            CHECK(msg->findString("name", &name));

            int32_t nameIsType;
            int32_t encoder = false;
            CHECK(msg->findInt32("nameIsType", &nameIsType));
            if (nameIsType) {
                CHECK(msg->findInt32("encoder", &encoder));
            }

            sp format = new AMessage;

            if (nameIsType) {
                format->setString("mime", name.c_str());
                format->setInt32("encoder", encoder);
            } else {
                format->setString("componentName", name.c_str());
            }

            mCodec->initiateAllocateComponent(format);
            break;
        }

主要是改變內(nèi)部狀態(tài)為INITIALIZING撵儿，同時(shí)調(diào)用了mCodec(此時(shí)為ACodec)的initiateAllocateComponent()方法：

void ACodec::initiateAllocateComponent(const sp &msg) {
    msg->setWhat(kWhatAllocateComponent);
    msg->setTarget(this);
    msg->post();
}

同樣的按照消息隊(duì)列機(jī)制找到回調(diào)：

ACodec::UninitializedState::onMessageReceived(const sp &msg)
 case ACodec::kWhatAllocateComponent:
        {
            onAllocateComponent(msg);
            handled = true;
            break;
        }

MediaCodec::UninitializedState::onAllocateComponent()

bool ACodec::UninitializedState::onAllocateComponent(const sp &msg) {
    ALOGV("onAllocateComponent");

    CHECK(mCodec->mNode == 0);

    OMXClient client;
    // 客戶端鏈接OMX服務(wù)
    if (client.connect() != OK) {
        mCodec->signalError(OMX_ErrorUndefined, NO_INIT);
        return false;
    }

    sp omx = client.interface();

    sp notify = new AMessage(kWhatOMXDied, mCodec);

    Vector matchingCodecs;

    AString mime;

    AString componentName;
    uint32_t quirks = 0;
    int32_t encoder = false;
    if (msg->findString("componentName", &componentName)) {
        sp list = MediaCodecList::getInstance();
        if (list != NULL && list->findCodecByName(componentName.c_str()) >= 0) {
            matchingCodecs.add(componentName);
        }
    } else {
        CHECK(msg->findString("mime", &mime));

        if (!msg->findInt32("encoder", &encoder)) {
            encoder = false;
        }

        // 找出符合要求的解碼器
        MediaCodecList::findMatchingCodecs(
                mime.c_str(),
                encoder, // createEncoder
                0,       // flags
                &matchingCodecs);
    }

    sp observer = new CodecObserver;
    IOMX::node_id node = 0;

    status_t err = NAME_NOT_FOUND;
    for (size_t matchIndex = 0; matchIndex < matchingCodecs.size();
            ++matchIndex) {
        componentName = matchingCodecs[matchIndex];
        quirks = MediaCodecList::getQuirksFor(componentName.c_str());

        pid_t tid = gettid();
        int prevPriority = androidGetThreadPriority(tid);
        androidSetThreadPriority(tid, ANDROID_PRIORITY_FOREGROUND);
        // 創(chuàng)建真正的解碼器實(shí)例
        err = omx->allocateNode(componentName.c_str(), observer, &mCodec->mNodeBinder, &node);
        androidSetThreadPriority(tid, prevPriority);

        if (err == OK) {
            break;
        } else {
            ALOGW("Allocating component '%s' failed, try next one.", componentName.c_str());
        }

        node = 0;
    }

    if (node == 0) {
        if (!mime.empty()) {
            ALOGE("Unable to instantiate a %scoder for type '%s' with err %#x.",
                    encoder ? "en" : "de", mime.c_str(), err);
        } else {
            ALOGE("Unable to instantiate codec '%s' with err %#x.", componentName.c_str(), err);
        }

        mCodec->signalError((OMX_ERRORTYPE)err, makeNoSideEffectStatus(err));
        return false;
    }

    mDeathNotifier = new DeathNotifier(notify);
    if (mCodec->mNodeBinder == NULL ||
            mCodec->mNodeBinder->linkToDeath(mDeathNotifier) != OK) {
        // This was a local binder, if it dies so do we, we won't care
        // about any notifications in the afterlife.
        mDeathNotifier.clear();
    }

    notify = new AMessage(kWhatOMXMessageList, mCodec);
    observer->setNotificationMessage(notify);

    mCodec->mComponentName = componentName;
    mCodec->mRenderTracker.setComponentName(componentName);
    mCodec->mFlags = 0;

    if (componentName.endsWith(".secure")) {
        mCodec->mFlags |= kFlagIsSecure;
        mCodec->mFlags |= kFlagIsGrallocUsageProtected;
        mCodec->mFlags |= kFlagPushBlankBuffersToNativeWindowOnShutdown;
    }

    mCodec->mQuirks = quirks;
    mCodec->mOMX = omx;
    mCodec->mNode = node;

    {
        sp notify = mCodec->mNotify->dup();
        notify->setInt32("what", CodecBase::kWhatComponentAllocated);
        notify->setString("componentName", mCodec->mComponentName.c_str());
        notify->post();
    }

    // 這里設(shè)置的是ACodec的狀態(tài)
    mCodec->changeState(mCodec->mLoadedState);

    return true;
}

這段代碼中乘客，mCodec的mOMX被初始化，而omx對(duì)象的初始化過(guò)程非常像是C/S架構(gòu)淀歇，那么這里插入一段易核，介紹下Android 中OMX的實(shí)現(xiàn)手段了：

02.png

??（1）先看OMXClient::connect()方法

status_t OMXClient::connect() {
    sp sm = defaultServiceManager();
    sp playerbinder = sm->getService(String16("media.player"));
    sp mediaservice = interface_cast(playerbinder);

    if (mediaservice.get() == NULL) {
        ALOGE("Cannot obtain IMediaPlayerService");
        return NO_INIT;
    }

    sp mediaServerOMX = mediaservice->getOMX();
    if (mediaServerOMX.get() == NULL) {
        ALOGE("Cannot obtain mediaserver IOMX");
        return NO_INIT;
    }

    // If we don't want to use the codec process, and the media server OMX
    // is local, use it directly instead of going through MuxOMX
    if (!sCodecProcessEnabled &&
            mediaServerOMX->livesLocally(0 /* node */, getpid())) {
        mOMX = mediaServerOMX;
        return OK;
    }

    sp codecbinder = sm->getService(String16("media.codec"));
    sp codecservice = interface_cast(codecbinder);

    if (codecservice.get() == NULL) {
        ALOGE("Cannot obtain IMediaCodecService");
        return NO_INIT;
    }

    sp mediaCodecOMX = codecservice->getOMX();
    if (mediaCodecOMX.get() == NULL) {
        ALOGE("Cannot obtain mediacodec IOMX");
        return NO_INIT;
    }

    mOMX = new MuxOMX(mediaServerOMX, mediaCodecOMX);

    return OK;
}

這里做了一次IPC工作，函數(shù)通過(guò)binder機(jī)制獲得到MediaPlayerService浪默，然后通過(guò)MediaPlayerService來(lái)創(chuàng)建OMX的實(shí)例牡直。這樣OMXClient就獲得到了OMX的入口，接下來(lái)就可以通過(guò)binder機(jī)制來(lái)獲得OMX提供的服務(wù)浴鸿。也就是說(shuō)OMXClient 是Android中 openmax 的入口井氢。

??可以看到這里的mediaServerOMX弦追，mediaCodecOMX都是Binder通信的代理對(duì)象岳链，在AOSP官網(wǎng)上對(duì)與OMX的支持描述如下：

Application Framework At the application framework level is application code that utilizesandroid.media APIs to interact with the multimedia hardware. Binder IPC The Binder IPC proxies facilitate communication over process boundaries. They are located in the frameworks/av/media/libmedia directory and begin with the letter "I". Native Multimedia Framework At the native level, Android provides a multimedia framework that utilizes the Stagefright engine for audio and video recording and playback. Stagefright comes with a default list of supported software codecs and you can implement your own hardware codec by using the OpenMax integration layer standard. For more implementation details, see the MediaPlayer and Stagefright components located in frameworks/av/media. OpenMAX Integration Layer (IL) The OpenMAX IL provides a standardized way for Stagefright to recognize and use custom hardware-based multimedia codecs called components. You must provide an OpenMAX plugin in the form of a shared library namedlibstagefrighthw.so. This plugin links Stagefright with your custom codec components, which must be implemented according to the OpenMAX IL component standard.

??可以猜測(cè)，在Server端真正實(shí)現(xiàn)功能時(shí)劲件，肯定是和這個(gè)libstagefrighthw.so動(dòng)態(tài)鏈接庫(kù)有關(guān)掸哑。往后看的時(shí)候再來(lái)分析這個(gè)東西约急。這里獲取OMX對(duì)象是整個(gè)MediaCodec功能實(shí)現(xiàn)的核心點(diǎn)，后面在configure方法的分析時(shí)會(huì)重點(diǎn)看這個(gè)對(duì)象的獲取過(guò)程苗分。

??分析完這個(gè)函數(shù)后厌蔽，我們?cè)偃タ碅Codec的構(gòu)造函數(shù)中的一個(gè)函數(shù)：changeState()，這個(gè)函數(shù)的實(shí)現(xiàn)在frameworks/av/media/libstagefright/foundation/AHierarchicalStateMachine.cpp中

void AHierarchicalStateMachine::changeState(const sp &state) {
    if (state == mState) {
        // Quick exit for the easy case.
        return;
    }

    Vector > A;
    sp cur = mState;
    for (;;) {
        A.push(cur);
        if (cur == NULL) {
            break;
        }
        cur = cur->parentState();
    }

    Vector > B;
    cur = state;
    for (;;) {
        B.push(cur);
        if (cur == NULL) {
            break;
        }
        cur = cur->parentState();
    }

    // Remove the common tail.
    while (A.size() > 0 && B.size() > 0 && A.top() == B.top()) {
        A.pop();
        B.pop();
    }

    mState = state;

    for (size_t i = 0; i < A.size(); ++i) {
        A.editItemAt(i)->stateExited();
    }

    for (size_t i = B.size(); i > 0;) {
        i--;
        B.editItemAt(i)->stateEntered();
    }
}

同時(shí)看到ACodec.h中：

struct ACodec : public AHierarchicalStateMachine, public CodecBase {
    // .....
     // AHierarchicalStateMachine implements the message handling
    virtual void onMessageReceived(const sp &msg) {
        handleMessage(msg);
    }
}

CodecBase.h中：

struct CodecBase : public AHandler, /* static */ ColorUtils {
}

AHierarchicalStateMachine.cpp中

void AHierarchicalStateMachine::handleMessage(const sp &msg) {
    sp save = mState;

    sp cur = mState;
    while (cur != NULL && !cur->onMessageReceived(msg)) {
        // If you claim not to have handled the message you shouldn't
        // have called setState...
        CHECK(save == mState);

        cur = cur->parentState();
    }

    if (cur != NULL) {
        return;
    }

    ALOGW("Warning message %s unhandled in root state.",
         msg->debugString().c_str());
}

也就是說(shuō)摔癣，從Handler傳給ACodec的Message經(jīng)過(guò)上面的邏輯都會(huì)轉(zhuǎn)發(fā)到當(dāng)前ACodec狀態(tài)機(jī)上狀態(tài)鏈上每一個(gè)AState::onMessageReceived()方法上奴饮，這個(gè)很關(guān)鍵，會(huì)對(duì)后面的MediaCodec#start(), stop(), release()等都有關(guān)系择浊。

也就是說(shuō)戴卜，Acodec內(nèi)部事實(shí)上是一個(gè)維護(hù)了由AState子類構(gòu)建的狀態(tài)機(jī)，我們現(xiàn)在已經(jīng)在構(gòu)造函數(shù)中看到了它c(diǎn)hangeState()到了一個(gè)mUninitializedState狀態(tài)琢岩。

??從剛才的changeState()方法可以看到投剥，如果狀態(tài)相同的會(huì)被直接移去，而狀態(tài)不同的狀態(tài)鏈担孔，原有的狀態(tài)鏈會(huì)被逐個(gè)調(diào)用stateExited()江锨，現(xiàn)在新加入的狀態(tài)鏈會(huì)翻轉(zhuǎn)過(guò)來(lái)調(diào)用stateEntered()方法，接著就來(lái)看這個(gè)UninitializedState::stateEntered()做了什么：

void ACodec::UninitializedState::stateEntered() {
    ALOGV("Now uninitialized");

    if (mDeathNotifier != NULL) {
        mCodec->mNodeBinder->unlinkToDeath(mDeathNotifier);
        mDeathNotifier.clear();
    }

    mCodec->mUsingNativeWindow = false;
    mCodec->mNativeWindow.clear();
    mCodec->mNativeWindowUsageBits = 0;
    mCodec->mNode = 0;
    mCodec->mOMX.clear();
    mCodec->mQuirks = 0;
    mCodec->mFlags = 0;
    mCodec->mInputMetadataType = kMetadataBufferTypeInvalid;
    mCodec->mOutputMetadataType = kMetadataBufferTypeInvalid;
    mCodec->mConverter[0].clear();
    mCodec->mConverter[1].clear();
    mCodec->mComponentName.clear();
}

其實(shí)就是初始化工作糕篇，到這里MediaCodec的構(gòu)造過(guò)程基本上就分析完了啄育，其中省略了大量關(guān)于Android消息機(jī)制native的邏輯，這一塊可以參考一下網(wǎng)上很多的文章娩缰，而且這也不是這篇文章的重點(diǎn)灸撰。

2.2 configure()

??再來(lái)看一下MediaCodec.configure()方法，由于代碼太多拼坎，分析下大致流程：

（1）MediaCodec.java  
    configure(MediaFormat format,Surface surface, @Nullable MediaCrypto crypto,
                                    int flags)
    native native_configure()
-------
（2）android_medai_MediaCodec.cpp         
    android_media_MediaCodec_native_configure()
    JMediaCodec::configure()
-------
（3）MediaCodec.cpp
    configure()
    onMessageReceived()  case kWhatConfigure:
-------
（4）ACodec.cpp
    void ACodec::initiateConfigureComponent(const sp<AMessage> &msg)
    ACodec::LoadedState::onMessageReceived() case ACodec::kWhatConfigureComponent
    ACodec::LoadedState::onConfigureComponent()
    ACodec::configureCodec(const char *mime, const sp<AMessage> &msg)

ACodec::configureCodec函數(shù)代碼太長(zhǎng)浮毯，大概有600多行，就不貼出來(lái)了泰鸡，主要就是通過(guò)之前保存的IOMXNode债蓝，設(shè)置編解碼的相關(guān)參數(shù)，最終會(huì)設(shè)置到編碼器中盛龄，如：SoftAACEncoder2的internalSetParameter函數(shù)饰迹。

2.3 start()

??接下來(lái)是MediaCodec.start()，也是先看一下大致流程：

（1）MediaCodec.java  
    start()
    native native_start()
-------
（2）android_medai_MediaCodec.cpp         
    android_media_MediaCodec_start()
    JMediaCodec::start()
-------
（3）MediaCodec.cpp
    start()
    onMessageReceived()  case kWhatStart:
-------
（4）ACodec.cpp
    void ACodec::initiateStart()
    ACodec::LoadedState::onMessageReceived() case ACodec::kWhatStart
    ACodec::LoadedState::onStart()
    
-------
（5）void ACodec::LoadedState::onStart() {
    ALOGV("onStart");

    status_t err = mCodec->mOMXNode->sendCommand(OMX_CommandStateSet, OMX_StateIdle);
    if (err != OK) {
        mCodec->signalError(OMX_ErrorUndefined, makeNoSideEffectStatus(err));
    } else {
        mCodec->changeState(mCodec->mLoadedToIdleState);
    }
}

在最后的ACodec::LoadedState::onStart()方法中余舶，mOMNode發(fā)送command啊鸭，omx處理狀態(tài)。并且ACodec狀態(tài)改為mLoadedToIdleState匿值，allocateBuffers分配buffer赠制。

void ACodec::LoadedToIdleState::stateEntered() {
    status_t err;
    if ((err = allocateBuffers()) != OK) {
        //。挟憔。钟些。烟号。。政恍。
    }
}

??OMX處理信息 ：OMX_CommandStateSet 汪拥，OMX_StateIdle，會(huì)發(fā)送event通知ACodec篙耗，看看ACodec是如何處理的：

bool ACodec::LoadedToIdleState::onOMXEvent(
        OMX_EVENTTYPE event, OMX_U32 data1, OMX_U32 data2) {
    switch (event) {
        case OMX_EventCmdComplete:
        {
            //....

            if (err == OK) {
                err = mCodec->mOMXNode->sendCommand(
                    OMX_CommandStateSet, OMX_StateExecuting);
            }

            if (err != OK) {
                mCodec->signalError(OMX_ErrorUndefined, makeNoSideEffectStatus(err));
            } else {
                mCodec->changeState(mCodec->mIdleToExecutingState);
            }

            return true;
        }

    }
}

LoadedToIdleState接收到event：OMX_EventCmdComplete后迫筑，向OMX發(fā)送command OMX_StateExecuting，并且ACodec將狀態(tài)切換為IdleToExecutingState宗弯。再繼續(xù)下去铣焊，最終ACodec state切換為ExecutingState。

3.MediaCodec輸入緩沖區(qū)

??這節(jié)分析一下MeidaCodec獲取可用輸入buffer罕伯，buffer加入隊(duì)列相關(guān)流程曲伊。輸出和輸入差不多的流程就不分析了。

3.1 相關(guān)數(shù)據(jù)結(jié)構(gòu)

• MediaCodec.cpp

List<size_t> mAvailPortBuffers[2];
std::vector<BufferInfo> mPortBuffers[2];

• mAvailPortBuffers：可用buffer對(duì)應(yīng)的index追他。mAvailPortBuffers[0]為輸入坟募，mAvailPortBuffers[1]為輸出。
• mPortBuffers：所有buffer緩沖區(qū)邑狸，包括可用和已占用懈糯。mPortBuffers[0]為輸入buffer，mPortBuffers[1]為輸出buffer单雾。
  這些數(shù)據(jù)緩沖區(qū)是在Mediacodec configure的時(shí)候進(jìn)行的分配赚哗。

3.2 dequeueInputBuffer()

??查看MediaCodec 輸入緩沖區(qū)使用流程，這里就不在貼java層代碼了硅堆，直接看MediaCodec.cpp dequeueInputBuffer屿储。

status_t MediaCodec::dequeueInputBuffer(size_t *index, int64_t timeoutUs) {
    sp<AMessage> msg = new AMessage(kWhatDequeueInputBuffer, this);
    msg->setInt64("timeoutUs", timeoutUs);

    sp<AMessage> response;
    status_t err;
    if ((err = PostAndAwaitResponse(msg, &response)) != OK) {
        return err;
    }

    CHECK(response->findSize("index", index));

    return OK;
}

這里發(fā)送AMessage消息，MediaCodec繼承AHandler渐逃，通過(guò)onMessageReceived處理消息：

void MediaCodec::onMessageReceived(const sp<AMessage> &msg) {
    switch (msg->what()) {
            case kWhatDequeueInputBuffer:
        {
            //.....
            if (handleDequeueInputBuffer(replyID, true /* new request */)) {
                break;
            }
        }
        //.......
}

接著看handleDequeueInputBuffer：

bool MediaCodec::handleDequeueInputBuffer(const sp<AReplyToken> &replyID, bool newRequest) {
    if (!isExecuting() || (mFlags & kFlagIsAsync)
            || (newRequest && (mFlags & kFlagDequeueInputPending))) {
        PostReplyWithError(replyID, INVALID_OPERATION);
        return true;
    } else if (mFlags & kFlagStickyError) {
        PostReplyWithError(replyID, getStickyError());
        return true;
    }

    ssize_t index = dequeuePortBuffer(kPortIndexInput);

    if (index < 0) {
        CHECK_EQ(index, -EAGAIN);
        return false;
    }

    sp<AMessage> response = new AMessage;
    response->setSize("index", index);
    response->postReply(replyID);

    return true;
}

• 檢查狀態(tài)及flag够掠；
• dequeuePortBuffer從mAvailPortBuffers獲取可用輸入buffer下標(biāo)；
• 將index通過(guò)response返回茄菊。

3.3 getInputBuffer()

??MediaCodec.java 中g(shù)etInputBuffer直接看jni的方法：

static jobject android_media_MediaCodec_getBuffer(
        JNIEnv *env, jobject thiz, jboolean input, jint index) {

    jobject buffer;
    status_t err = codec->getBuffer(env, input, index, &buffer);
        //獲取buffer,并return;
    if (err == OK) {
        return buffer;
    }

    return NULL;
}
---------------
status_t JMediaCodec::getBuffer(
        JNIEnv *env, bool input, size_t index, jobject *buf) const {
    sp<MediaCodecBuffer> buffer;

    status_t err =
        input
            ? mCodec->getInputBuffer(index, &buffer)
            : mCodec->getOutputBuffer(index, &buffer);

    if (err != OK) {
        return err;
    }

    return createByteBufferFromABuffer(
            env, !input /* readOnly */, input /* clearBuffer */, buffer, buf);
}

• 調(diào)用MediaCodec.cpp中g(shù)etInputBuffer函數(shù)疯潭，獲取id 對(duì)應(yīng)buffer；
• 通過(guò)MediaCodecBuffer創(chuàng)建ByteBuffer面殖，實(shí)現(xiàn)java和native共享內(nèi)存竖哩；
• 返回ByteBuffer。

接著看mCodec->getInputBuffer

status_t MediaCodec::getInputBuffer(size_t index, sp<MediaCodecBuffer> *buffer) {
    sp<AMessage> format;
    return getBufferAndFormat(kPortIndexInput, index, buffer, &format);
}
-------------
status_t MediaCodec::getBufferAndFormat(
        size_t portIndex, size_t index,
        sp<MediaCodecBuffer> *buffer, sp<AMessage> *format) {
        //...
    std::vector<BufferInfo> &buffers = mPortBuffers[portIndex];
    if (index >= buffers.size()) {
        return INVALID_OPERATION;
    }

    const BufferInfo &info = buffers[index];
    if (!info.mOwnedByClient) {
        return INVALID_OPERATION;
    }

    *buffer = info.mData;
    *format = info.mData->format();

    return OK;
}

• mPortBuffers通過(guò)index區(qū)分輸入輸出脊僚，這里獲取所有輸入BufferInfo相叁；
• 通過(guò)下標(biāo)獲取指定的BufferInfo；
• 將buffer指針指向BufferInfo的mData。

3.4 queueInputBuffer()

static void android_media_MediaCodec_queueInputBuffer(
        JNIEnv *env,
        jobject thiz,
        jint index,
        jint offset,
        jint size,
        jlong timestampUs,
        jint flags) {
    sp<JMediaCodec> codec = getMediaCodec(env, thiz);

    if (codec == NULL) {
        throwExceptionAsNecessary(env, INVALID_OPERATION);
        return;
    }

    AString errorDetailMsg;
    status_t err = codec->queueInputBuffer(
            index, offset, size, timestampUs, flags, &errorDetailMsg);
    throwExceptionAsNecessary(
            env, err, ACTION_CODE_FATAL, errorDetailMsg.empty() ? NULL : errorDetailMsg.c_str());
}
----------------
status_t JMediaCodec::queueInputBuffer(
        size_t index,
        size_t offset, size_t size, int64_t timeUs, uint32_t flags,
        AString *errorDetailMsg) {
    return mCodec->queueInputBuffer(
            index, offset, size, timeUs, flags, errorDetailMsg);
}

jni方法android_media_MediaCodec_queueInputBuffer獲取JMediaCodec钝荡，調(diào)用其queueInputBuffer，然后調(diào)用MediaCodec的queueInputBuffer函數(shù)：

status_t MediaCodec::queueInputBuffer(
        size_t index,
        size_t offset,
        size_t size,
        int64_t presentationTimeUs,
        uint32_t flags,
        AString *errorDetailMsg) {
    if (errorDetailMsg != NULL) {
        errorDetailMsg->clear();
    }

    sp<AMessage> msg = new AMessage(kWhatQueueInputBuffer, this);
    msg->setSize("index", index);
    msg->setSize("offset", offset);
    msg->setSize("size", size);
    msg->setInt64("timeUs", presentationTimeUs);
    msg->setInt32("flags", flags);
    msg->setPointer("errorDetailMsg", errorDetailMsg);

    sp<AMessage> response;
    return PostAndAwaitResponse(msg, &response);
}
------------------
void MediaCodec::onMessageReceived(const sp<AMessage> &msg) {
    switch (msg->what()) {
        case kWhatQueueInputBuffer:{
                //......
            status_t err = onQueueInputBuffer(msg);
            PostReplyWithError(replyID, err);
            break;
        }
    }
}

onMessageReceived接收消息kWhatQueueInputBuffer舶衬，調(diào)用onQueueInputBuffer函數(shù)：

status_t MediaCodec::onQueueInputBuffer(const sp<AMessage> &msg) {
   //......
    sp<MediaCodecBuffer> buffer = info->mData;
    status_t err = OK;
    if (hasCryptoOrDescrambler()) {
        
    } else {
        err = mBufferChannel->queueInputBuffer(buffer);
    }

    return err;
}

調(diào)用ACodecBufferChannel queueInputBuffer函數(shù)：

status_t ACodecBufferChannel::queueInputBuffer(const sp<MediaCodecBuffer> &buffer) {
    if (mDealer != nullptr) {
        return -ENOSYS;
    }
    std::shared_ptr<const std::vector<const BufferInfo>> array(
            std::atomic_load(&mInputBuffers));
    BufferInfoIterator it = findClientBuffer(array, buffer);
    if (it == array->end()) {
        return -ENOENT;
    }
    sp<AMessage> msg = mInputBufferFilled->dup();
    msg->setObject("buffer", it->mCodecBuffer);
    msg->setInt32("buffer-id", it->mBufferId);
    msg->post();
    return OK;
}

暫不清楚ACodecBufferChannel 的用處是什么埠通。 發(fā)送kWhatInputBufferFilled message。

ACodec接收到kWhatInputBufferFilled信息后續(xù)：
void ACodec::BaseState::onInputBufferFilled(const sp<AMessage> &msg){
    //.......
         switch (mode) {
        case KEEP_BUFFERS:{}
        case RESUBMIT_BUFFERS:{
            status_t err2 = mCodec->mOMXNode->emptyBuffer(
                        bufferID, OMXBuffer::sPreset, OMX_BUFFERFLAG_EOS, 0, info->mFenceFd);
            
                 break;
        }
        case FREE_BUFFERS:
            break;
}

ACodec通知OMX逛犹，寫(xiě)入數(shù)據(jù)端辱。