需求
iOS中將壓縮音頻數(shù)據(jù)(如AAC)進行解碼以得到原始音頻數(shù)據(jù)類型:線性PCM.
本例最終實現(xiàn)的是通過Audio Queue采集到AAC壓縮數(shù)據(jù),將其解碼為PCM數(shù)據(jù),并將解碼后的PCM數(shù)據(jù)以錄制的形式保存在沙盒中.可調(diào)整解碼后采樣率,解碼器類型等參數(shù).
本例可拓展,不僅僅解碼AAC音頻數(shù)據(jù)流,還可以是音頻文件,視頻文件中的音頻等等.
實現(xiàn)原理
利用Audio Toolbox Framework中的Audio Converter可以實現(xiàn)音頻數(shù)據(jù)解碼,即將AAC數(shù)據(jù)轉(zhuǎn)為原始音頻數(shù)據(jù)PCM.
閱讀前提:
- Core Audio基本原理:簡書,掘金,博客
- Audio Queue解析:掘金,簡書,博客
- Audio Queue實戰(zhàn):簡書,博客, 掘金
- 音頻文件錄制:簡書,掘金,博客
- 音視頻基礎知識
- C,C++基本知識
GitHub地址(附代碼) : 音頻解碼
簡書地址 : 音頻解碼
掘金地址 : 音頻解碼
博客地址 : 音頻解碼
1.初始化
1.1. 初始化解碼器
初始化解碼器實例, 通過指定原始數(shù)據(jù)格式,最終解碼后的格式,采樣率,以及使用硬編還是軟編,以下是具體步驟.
- (instancetype)initWithSourceFormat:(AudioStreamBasicDescription)sourceFormat destFormatID:(AudioFormatID)destFormatID sampleRate:(float)sampleRate isUseHardwareDecode:(BOOL)isUseHardwareDecode {
if (self = [super init]) {
mSourceFormat = sourceFormat;
mAudioConverter = [self configureDecoderBySourceFormat:sourceFormat
destFormat:&mDestinationFormat
destFormatID:destFormatID
sampleRate:sampleRate
isUseHardwareDecode:isUseHardwareDecode];
}
return self;
}
1.2. 配置解碼后ASBD音頻流信息
AudioStreamBasicDescription destinationFormat = {0};
destinationFormat.mSampleRate = sampleRate;
if (destFormatID != kAudioFormatLinearPCM) {
NSLog(@"Not get compression format after decoding !");
return NULL;
} else {
destinationFormat.mFormatID = destFormatID;
destinationFormat.mChannelsPerFrame = sourceFormat.mChannelsPerFrame;
destinationFormat.mFormatID = kAudioFormatLinearPCM;
destinationFormat.mFormatFlags = (kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked);
destinationFormat.mFramesPerPacket = kXDXAudioPCMFramesPerPacket;
destinationFormat.mBitsPerChannel = KXDXAudioBitsPerChannel;
destinationFormat.mBytesPerFrame = destinationFormat.mBitsPerChannel / 8 *destinationFormat.mChannelsPerFrame;
destinationFormat.mBytesPerPacket = destinationFormat.mBytesPerFrame * destinationFormat.mFramesPerPacket;
destinationFormat.mReserved = 0;
}
memcpy(destFormat, &destinationFormat, sizeof(AudioStreamBasicDescription));
對音頻做解碼操作,實際就是將壓縮數(shù)據(jù)格式如AAC格式轉(zhuǎn)為線性PCM原始音頻數(shù)據(jù),通過kAudioFormatProperty_FormatInfo屬性可以自動獲取指定音頻格式的參數(shù)信息.
1.3. 選擇解碼器類型
AudioClassDescription結(jié)構(gòu)體描述了系統(tǒng)使用音頻解碼器信息,其中最重要的就是使用硬編或軟編。然后解碼器的數(shù)量导狡，即數(shù)組的個數(shù)，由當前的聲道數(shù)決定弦叶。
//獲取解碼器的描述信息
AudioClassDescription *audioClassDesc = [self getAudioCalssDescriptionWithType:destFormatID fromManufacture:kAppleHardwareAudioCodecManufacturer];
...
- (AudioClassDescription *)getAudioCalssDescriptionWithType:(AudioFormatID)type fromManufacture:(uint32_t)manufacture {
static AudioClassDescription desc;
UInt32 decoderSpecific = type;
UInt32 size;
OSStatus status = AudioFormatGetPropertyInfo(kAudioFormatProperty_Decoders,
sizeof(decoderSpecific),
&decoderSpecific,
&size);
if (status != noErr) {
NSLog(@"Error３室啊：硬解碼AAC get info 失敗, status= %d", (int)status);
return nil;
}
//計算aac解碼器的個數(shù)
unsigned int count = size / sizeof(AudioClassDescription);
//創(chuàng)建一個包含count個解碼器的數(shù)組
AudioClassDescription description[count];
//將滿足aac解碼的解碼器的信息寫入數(shù)組
status = AudioFormatGetProperty(kAudioFormatProperty_Encoders,
sizeof(decoderSpecific),
&decoderSpecific,
&size,
&description);
if (status != noErr) {
NSLog(@"Error！：硬解碼AAC get propery 失敗, status= %d", (int)status);
return nil;
}
for (unsigned int i = 0; i < count; i++) {
if (type == description[i].mSubType && manufacture == description[i].mManufacturer) {
desc = description[i];
return &desc;
}
}
return nil;
}
注意：硬解即利用設備GPU硬件完成高效解碼，降低CPU消耗. 軟解就是傳統(tǒng)的通過CPU計算京腥。
1.4. 創(chuàng)建解碼器
AudioConverterNewSpecific: 通過指定解碼器來創(chuàng)建audio converter實例對象.第3,4個
分別是解碼器的數(shù)量與解碼器描述,同上,與聲道數(shù)保持一致.
// Create the AudioConverterRef.
AudioConverterRef converter = NULL;
if (![self checkError:AudioConverterNewSpecific(&sourceFormat, &destinationFormat, destinationFormat.mChannelsPerFrame, audioClassDesc, &converter) withErrorString:@"Audio Converter New failed"]) {
return NULL;
}else {
printf("Audio converter create successful \n");
}
2.解碼
2.1. 計算解碼數(shù)據(jù)大小
注意,當使用Audio Convert無論做編解碼,每次都需要1024個采樣點才能完成一次轉(zhuǎn)換,此值是固定的.
根據(jù)解碼器的采樣點,計算解碼出音頻數(shù)據(jù)的大小.因為線性PCM的數(shù)據(jù)可以通過公式算出,即數(shù)據(jù)包數(shù)量*聲道數(shù)*每個數(shù)據(jù)包中字節(jié)數(shù).
// Note: audio convert must set 1024.
UInt32 ioOutputDataPackets = kIOOutputDataPackets;
UInt32 outputBufferSize = (UInt32)(ioOutputDataPackets * destFormat.mChannelsPerFrame * destFormat.mBytesPerFrame);
2.2. 為解碼后音頻數(shù)據(jù)預分配內(nèi)存
我們可以將2.1中算出的size為這個Buffer list分配內(nèi)存.
// Set up output buffer list.
// Set up output buffer list.
AudioBufferList fillBufferList = {0};
fillBufferList.mNumberBuffers = 1;
fillBufferList.mBuffers[0].mNumberChannels = destFormat.mChannelsPerFrame;
fillBufferList.mBuffers[0].mDataByteSize = outputBufferSize;
fillBufferList.mBuffers[0].mData = malloc(outputBufferSize * sizeof(char));
2.3. 解碼音頻數(shù)據(jù)
解析AudioConverterFillComplexBuffer:用來解碼音頻數(shù)據(jù).同時需要指定回調(diào)函數(shù)(C語言函數(shù)),
第二個參數(shù)即指定回調(diào)函數(shù),此回調(diào)函數(shù)中主要做的是為即將解碼的數(shù)據(jù)進行賦值,即我們要把原始音頻數(shù)據(jù)賦值給回調(diào)函數(shù)中的ioData參數(shù),這是我們在解碼前最后一次控制原始音頻數(shù)據(jù),此回調(diào)函數(shù)執(zhí)行后即完成了解碼的過程,新的數(shù)據(jù)會填充到第五個參數(shù)中,也就是我們上面預定義的fillBufferList.
-
userInfo: 自定義一個結(jié)構(gòu)體,用來與解碼回調(diào)函數(shù)間交互以傳遞數(shù)據(jù).在這里是將原始音頻數(shù)據(jù)信息傳給解碼回調(diào)函數(shù)中. -
ioOutputDataPackets: 填入函數(shù)中時表示原始音頻數(shù)據(jù)包的數(shù)量,而函數(shù)調(diào)用完成時表示轉(zhuǎn)換后輸出的音頻數(shù)據(jù)包總數(shù),注意,當我們做解碼時,輸出肯定為PCM類型數(shù)據(jù),所以需要提供1024個AAC采樣點.而做編碼時會將PCM數(shù)據(jù)壓縮成很多音頻數(shù)據(jù)包,僅僅需要1個完整的PCM數(shù)據(jù)包即可. -
outputPacketDescriptions: 轉(zhuǎn)換完成后,如果此參數(shù)非空,表示轉(zhuǎn)換器輸出使用的音頻數(shù)據(jù)包描述,它必須提前分配好內(nèi)存,以讓轉(zhuǎn)換器賦值到其中.
最終,我們將轉(zhuǎn)換后得到的AAC數(shù)據(jù)以回調(diào)函數(shù)的形式傳給調(diào)用者.
OSStatus DecodeConverterComplexInputDataProc(AudioConverterRef inAudioConverter,
UInt32 *ioNumberDataPackets,
AudioBufferList *ioData,
AudioStreamPacketDescription **outDataPacketDescription,
void *inUserData) {
XDXConverterInfoType *info = (XDXConverterInfoType *)inUserData;
if (info->sourceDataSize <= 0) {
ioNumberDataPackets = 0;
return -1;
}
*outDataPacketDescription = &info->packetDesc;
(*outDataPacketDescription)[0].mStartOffset = 0;
(*outDataPacketDescription)[0].mDataByteSize = info->sourceDataSize;
(*outDataPacketDescription)[0].mVariableFramesInPacket = 0;
ioData->mNumberBuffers = 1;
ioData->mBuffers[0].mData = info->sourceBuffer;
ioData->mBuffers[0].mNumberChannels = info->sourceChannelsPerFrame;
ioData->mBuffers[0].mDataByteSize = info->sourceDataSize;
return noErr;
}
- (void)decodeFormatByConverter:(AudioConverterRef)audioConverter sourceBuffer:(void *)sourceBuffer sourceBufferSize:(UInt32)sourceBufferSize sourceFormat:(AudioStreamBasicDescription)sourceFormat dest:(AudioStreamBasicDescription)destFormat completeHandler:(void(^)(AudioBufferList *destBufferList, UInt32 outputPackets, AudioStreamPacketDescription *outputPacketDescriptions))completeHandler {
...
XDXConverterInfoType userInfo = {0};
userInfo.sourceBuffer = sourceBuffer;
userInfo.sourceDataSize = sourceBufferSize;
userInfo.sourceChannelsPerFrame = sourceFormat.mChannelsPerFrame;
userInfo.packetDesc.mDataByteSize = (UInt32)sourceBufferSize;
userInfo.packetDesc.mStartOffset = 0;
userInfo.packetDesc.mVariableFramesInPacket = 0;
AudioStreamPacketDescription outputPacketDesc;
OSStatus status = AudioConverterFillComplexBuffer(audioConverter,
DecodeConverterComplexInputDataProc,
&userInfo,
&ioOutputDataPackets,
&fillBufferList,
&outputPacketDesc);
// if interrupted in the process of the conversion call, we must handle the error appropriately
if (status != noErr) {
if (status == kAudioConverterErr_HardwareInUse) {
printf("Audio Converter returned kAudioConverterErr_HardwareInUse!\n");
} else {
if (![self checkError:status withErrorString:@"AudioConverterFillComplexBuffer error!"]) {
return;
}
}
} else {
if (ioOutputDataPackets == 0) {
// This is the EOF condition.
status = noErr;
}
if (completeHandler) {
completeHandler(&fillBufferList, ioOutputDataPackets, &outputPacketDesc);
}
}
}
3. 模塊對接
因為音頻解碼要依賴音頻采集,所以我們這里以audio unit采集為例作示范,即使用audio unit采集pcm數(shù)據(jù)然后使用此模塊解碼得到aac數(shù)據(jù).如需了解請參考如下鏈接
- GitHub地址(附代碼) : Audio Unit Capture
- 簡書地址 : Audio Unit Capture
- 掘金地址 : Audio Unit Capture
- 博客地址 : Audio Unit Capture
3.1. 初始化解碼器
如下,在音頻采集的類中聲明一個解碼器實例變量,然后初始化它. 僅僅需要設置原始數(shù)據(jù)格式,解碼后的格式,采樣率,使用硬編,軟編即可.
@property (nonatomic, strong) XDXAduioDecoder *audioDecoder;
...
// audio decode: aac->pcm
self.audioDecoder = [[XDXAduioDecoder alloc] initWithSourceFormat:m_audioInfo->mDataFormat
destFormatID:kAudioFormatLinearPCM
sampleRate:48000
isUseHardwareDecode:YES];
3.2. 解碼音頻數(shù)據(jù)
在Audio Queue采集AAC音頻數(shù)據(jù)的回調(diào)中將AAC數(shù)據(jù)送入解碼器,然后在回調(diào)函數(shù)中將得到的PCM數(shù)據(jù)其寫入文件.
注意: 直接用Audio Queue采集AAC類型音頻數(shù)據(jù),實際系統(tǒng)在其內(nèi)部做了一次轉(zhuǎn)換,即直接采集其實只能采原始PCM數(shù)據(jù),直接用Audio Queue設置采集AAC相當于系統(tǒng)在內(nèi)部為我們做了一次轉(zhuǎn)換.
static void CaptureAudioDataCallback(void * inUserData,
AudioQueueRef inAQ,
AudioQueueBufferRef inBuffer,
const AudioTimeStamp * inStartTime,
UInt32 inNumPackets,
const AudioStreamPacketDescription* inPacketDesc) {
XDXAudioQueueCaptureManager *instance = (__bridge XDXAudioQueueCaptureManager *)inUserData;
[instance.audioDecoder decodeAudioWithSourceBuffer:inBuffer->mAudioData
sourceBufferSize:inBuffer->mAudioDataByteSize
completeHandler:^(AudioBufferList * _Nonnull destBufferList, UInt32 outputPackets, AudioStreamPacketDescription * _Nonnull outputPacketDescriptions) {
if (instance.isRecordVoice) {
[[XDXAudioFileHandler getInstance] writeFileWithInNumBytes:destBufferList->mBuffers->mDataByteSize
ioNumPackets:outputPackets
inBuffer:destBufferList->mBuffers->mData
inPacketDesc:outputPacketDescriptions];
}
free(destBufferList->mBuffers->mData);
}];
if (instance.isRunning) {
AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL);
}
}
4. 文件錄制
此部分可參考另一篇文章: 音頻文件錄制
- 簡書地址 : Audio File Record
- 掘金地址 : Audio File Record
- 博客地址 : Audio File Record
5. 釋放解碼器資源
如需釋放內(nèi)存,請保證解碼器工作徹底結(jié)束后再釋放內(nèi)存.
- (void)freeEncoder {
if (mAudioConverter) {
AudioConverterDispose(mAudioConverter);
mAudioConverter = NULL;
}
}
