1.需求來(lái)源逆瑞。
最近有一個(gè)用戶反饋,發(fā)出去的視頻有點(diǎn)不清楚伙单。由于視頻壓縮模塊是在幾年前寫(xiě)的获高,當(dāng)時(shí)的已經(jīng)滿足不了現(xiàn)在的需求了,所以需要重新設(shè)計(jì)壓縮的實(shí)現(xiàn)吻育。
2.現(xiàn)狀
使用AVAssetExportSession
作為導(dǎo)出工具念秧,指定壓縮質(zhì)量AVAssetExportPresetMediumQuality
,這樣能有效的減少視頻體積布疼,但是視頻畫(huà)面清晰度比較差摊趾,舉個(gè)例子:一個(gè)25秒的1080p視頻,經(jīng)過(guò)壓縮后從1080p變?yōu)?20p游两,大小從34m變成2.6m砾层。
AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:avAsset presetName:AVAssetExportPresetMediumQuality];
exportSession.outputURL= url;
exportSession.shouldOptimizeForNetworkUse = YES;
exportSession.outputFileType = AVFileTypeMPEG4;
[exportSessionexportAsynchronouslyWithCompletionHandler:^{
switch([exportSessionstatus]) {
case AVAssetExportSessionStatusFailed:
NSLog(@"Export canceled");
break;
case AVAssetExportSessionStatusCancelled:
NSLog(@"Export canceled");
break;
case AVAssetExportSessionStatusCompleted:{
NSLog(@"Successful!");
break;
}
default:break;
}
重新梳理下我們的需求,我們的場(chǎng)景對(duì)視頻質(zhì)量要求稍高贱案,對(duì)視頻的大小容忍比較高肛炮,所以將最大分辨率設(shè)為720p。
所以我們的壓縮設(shè)置改為AVAssetExportPreset1280x720
宝踪,壓縮后大小幾乎沒(méi)變侨糟,從34m變成32.5m。我們可以用mideaInfo來(lái)查看下兩個(gè)視頻文件到底有什么區(qū)別瘩燥,上圖為1080p秕重,下圖為720p:
由上圖可以看到,兩個(gè)分辨率差別巨大的視頻厉膀,大小居然差不多溶耘,要分析其中的原因首先要了解H264編碼。
3.H264編碼
關(guān)于H264編碼的原理可以參考(這篇文章),本文不詳細(xì)展開(kāi)服鹅,只說(shuō)明幾個(gè)參數(shù)凳兵。
Bit Rate
:
比特率是指每秒傳送的比特(bit)數(shù)。單位為 bps(Bit Per Second)菱魔,比特率越高留荔,每秒傳送數(shù)據(jù)就越多,畫(huà)質(zhì)就越清晰澜倦。聲音中的比特率是指將模擬聲音信號(hào)轉(zhuǎn)換成數(shù)字聲音信號(hào)后聚蝶,單位時(shí)間內(nèi)的二進(jìn)制數(shù)據(jù)量,是間接衡量音頻質(zhì)量的一個(gè)指標(biāo)藻治。 視頻中的比特率(碼率)原理與聲音中的相同碘勉,都是指由模擬信號(hào)轉(zhuǎn)換為數(shù)字信號(hào)后,單位時(shí)間內(nèi)的二進(jìn)制數(shù)據(jù)量桩卵。
所以選擇適合的比特率是壓縮視頻大小的關(guān)鍵验靡,比特率設(shè)置太小的話,視頻會(huì)變得模糊雏节,失真胜嗓。比特率太高的話,視頻數(shù)據(jù)太大钩乍,又達(dá)不到我們壓縮的要求辞州。
Format profile
:
作為行業(yè)標(biāo)準(zhǔn),H.264編碼體系定義了4種不同的Profile(類):Baseline(基線類),Main(主要類), Extended(擴(kuò)展類)和High Profile(高端類)(它們各自下分成許多個(gè)層):
Baseline Profile 提供I/P幀寥粹,僅支持progressive(逐行掃描)和CAVLC变过;
Extended Profile 提供I/P/B/SP/SI幀,僅支持progressive(逐行掃描)和CAVLC涝涤;
Main Profile 提供I/P/B幀媚狰,支持progressive(逐行掃描)和interlaced(隔行掃描),提供CAVLC或CABAC阔拳;
High Profile (也就是FRExt)在Main Profile基礎(chǔ)上新增:8x8 intra prediction(8x8 幀內(nèi)預(yù)測(cè)), custom quant(自定義量化), lossless video coding(無(wú)損視頻編碼), 更多的yuv格式(4:4:4...)崭孤;
從壓縮比例來(lái)說(shuō) 從壓縮比例來(lái)說(shuō),baseline< main < high糊肠,由于上圖中720p是Main@L3.1
裳瘪,1080p是High@L4
,這就是明明分辨率不一樣罪针,但是壓縮后的大小卻差不多的原因彭羹。
關(guān)于iPhone設(shè)備對(duì)的支持
iPhone 3GS 和更早的設(shè)備支持 Baseline Profile level 3.0 及更低的級(jí)別
iPhone 4S 支持 High Profile level 4.1 及更低的級(jí)別
iPhone 5C 支持 High Profile level 4.1 及更低的級(jí)別
iPhone 5S 支持 High Profile level 4.1 及更低的級(jí)別
iPad 1 支持 Main Profile level 3.1 及更低的級(jí)別
iPad 2 支持 Main Profile level 3.1 及更低的級(jí)別
iPad with Retina display 支持 High Profile level 4.1 及更低的級(jí)別
iPad mini 支持 High Profile level 4.1 及更低的級(jí)別
GOP
:
GOP 指的就是兩個(gè)I幀之間的間隔。
在視頻編碼序列中泪酱,主要有三種編碼幀:I幀派殷、P幀、B幀墓阀。
- I幀即Intra-coded picture(幀內(nèi)編碼圖像幀)毡惜,不參考其他圖像幀,只利用本幀的信息進(jìn)行編碼
- P幀即Predictive-codedPicture(預(yù)測(cè)編碼圖像幀)斯撮,利用之前的I幀或P幀经伙,采用運(yùn)動(dòng)預(yù)測(cè)的方式進(jìn)行幀間預(yù)測(cè)編碼
- B幀即Bidirectionallypredicted picture(雙向預(yù)測(cè)編碼圖像幀),提供最高的壓縮比,它既需要之前的圖
像幀(I幀或P幀)帕膜,也需要后來(lái)的圖像幀(P幀)枣氧,采用運(yùn)動(dòng)預(yù)測(cè)的方式進(jìn)行幀間雙向預(yù)測(cè)編碼
在視頻編碼序列中,GOP即Group of picture(圖像組)垮刹,指兩個(gè)I幀之間的距離达吞,Reference(參考周期)指兩個(gè)P幀之間的距離。一個(gè)I幀所占用的字節(jié)數(shù)大于一個(gè)P幀荒典,一個(gè)P幀所占用的字節(jié)數(shù)大于一個(gè)B幀酪劫。
所以在碼率不變的前提下,GOP值越大寺董,P覆糟、B幀的數(shù)量會(huì)越多卧斟,平均每個(gè)I窜护、P、B幀所占用的字節(jié)數(shù)就越多描扯,也就更容易獲取較好的圖像質(zhì)量盯滚;Reference越大踢械,B幀的數(shù)量越多,同理也更容易獲得較好的圖像質(zhì)量魄藕。
需要說(shuō)明的是内列,通過(guò)提高GOP值來(lái)提高圖像質(zhì)量是有限度的,在遇到場(chǎng)景切換的情況時(shí)背率,H.264編碼器會(huì)自動(dòng)強(qiáng)制插入一個(gè)I幀话瞧,此時(shí)實(shí)際的GOP值被縮短了。另一方面寝姿,在一個(gè)GOP中交排,P、B幀是由I幀預(yù)測(cè)得到的饵筑,當(dāng)I幀的圖像質(zhì)量比較差時(shí)埃篓,會(huì)影響到一個(gè)GOP中后續(xù)P、B幀的圖像質(zhì)量根资,直到下一個(gè)GOP開(kāi)始才有可能得以恢復(fù)架专,所以GOP值也不宜設(shè)置過(guò)大。
同時(shí)玄帕,由于P部脚、B幀的復(fù)雜度大于I幀,所以過(guò)多的P裤纹、B幀會(huì)影響編碼效率委刘,使編碼效率降低。另外,過(guò)長(zhǎng)的GOP還會(huì)影響Seek操作的響應(yīng)速度锡移,由于P呕童、B幀是由前面的I或P幀預(yù)測(cè)得到的,所以Seek操作需要直接定位罩抗,解碼某一個(gè)P或B幀時(shí)拉庵,需要先解碼得到本GOP內(nèi)的I幀及之前的N個(gè)預(yù)測(cè)幀才可以灿椅,GOP值越長(zhǎng)套蒂,需要解碼的預(yù)測(cè)幀就越多,seek響應(yīng)的時(shí)間也越長(zhǎng)茫蛹。
M 和 N :M值表示I幀或者P幀之間的幀數(shù)目操刀,N值表示GOP的長(zhǎng)度。N的至越大婴洼,代表壓縮率越大骨坑。因?yàn)閳D2中N=15遠(yuǎn)小于圖一中N=30。這也是720p尺寸壓縮不理想的原因柬采。
4.解決思路
由上可知壓縮視頻主要可以采用以下幾種手段:
- 降低分辨率
- 降低碼率
- 指定高的
Format profile
由于業(yè)務(wù)指定分辨率為720p欢唾,所以我們只能?chē)L試另外兩種方法。
降低碼率
根據(jù)這篇文章Video Encoding Settings for H.264 Excellence粉捻,推薦了適合720p的推薦碼率為2400~3700
之間礁遣。之前壓縮的文件碼率為9979
,所以碼率還是有很大的優(yōu)化空間的。
指定高的 Format profile
由于現(xiàn)在大部分的設(shè)備都支持High Profile level
,所以我們可以把Format profile
從Main Profile level
改為High Profile level
肩刃。
現(xiàn)在我們已經(jīng)知道要做什么了祟霍,那么怎么做呢?
5.解決方法
由于之前的AVAssetExportSession
不能指定碼率和Format profile
,我們這里需要使用AVAssetReader
和AVAssetWriter
盈包。
AVAssetReader
負(fù)責(zé)將數(shù)據(jù)從asset里拿出來(lái)沸呐,AVAssetWriter
負(fù)責(zé)將得到的數(shù)據(jù)存成文件。
核心代碼如下:
//生成reader 和 writer
self.reader = [AVAssetReader.alloc initWithAsset:self.asset error:&readerError];
self.writer = [AVAssetWriter assetWriterWithURL:self.outputURL fileType:self.outputFileType error:&writerError];
//視頻
if (videoTracks.count > 0) {
self.videoOutput = [AVAssetReaderVideoCompositionOutput assetReaderVideoCompositionOutputWithVideoTracks:videoTracks videoSettings:self.videoInputSettings];
self.videoOutput.alwaysCopiesSampleData = NO;
if (self.videoComposition)
{
self.videoOutput.videoComposition = self.videoComposition;
}
else
{
self.videoOutput.videoComposition = [self buildDefaultVideoComposition];
}
if ([self.reader canAddOutput:self.videoOutput])
{
[self.reader addOutput:self.videoOutput];
}
//
// Video input
//
self.videoInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:self.videoSettings];
self.videoInput.expectsMediaDataInRealTime = NO;
if ([self.writer canAddInput:self.videoInput])
{
[self.writer addInput:self.videoInput];
}
NSDictionary *pixelBufferAttributes = @
{
(id)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_32BGRA),
(id)kCVPixelBufferWidthKey: @(self.videoOutput.videoComposition.renderSize.width),
(id)kCVPixelBufferHeightKey: @(self.videoOutput.videoComposition.renderSize.height),
@"IOSurfaceOpenGLESTextureCompatibility": @YES,
@"IOSurfaceOpenGLESFBOCompatibility": @YES,
};
self.videoPixelBufferAdaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:self.videoInput sourcePixelBufferAttributes:pixelBufferAttributes];
}
//音頻
NSArray *audioTracks = [self.asset tracksWithMediaType:AVMediaTypeAudio];
if (audioTracks.count > 0) {
self.audioOutput = [AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks:audioTracks audioSettings:nil];
self.audioOutput.alwaysCopiesSampleData = NO;
self.audioOutput.audioMix = self.audioMix;
if ([self.reader canAddOutput:self.audioOutput])
{
[self.reader addOutput:self.audioOutput];
}
} else {
// Just in case this gets reused
self.audioOutput = nil;
}
//
// Audio input
//
if (self.audioOutput) {
self.audioInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeAudio outputSettings:self.audioSettings];
self.audioInput.expectsMediaDataInRealTime = NO;
if ([self.writer canAddInput:self.audioInput])
{
[self.writer addInput:self.audioInput];
}
}
//開(kāi)始讀寫(xiě)
[self.writer startWriting];
[self.reader startReading];
[self.writer startSessionAtSourceTime:self.timeRange.start];
//壓縮完成的回調(diào)
__block BOOL videoCompleted = NO;
__block BOOL audioCompleted = NO;
__weak typeof(self) wself = self;
self.inputQueue = dispatch_queue_create("VideoEncoderInputQueue", DISPATCH_QUEUE_SERIAL);
if (videoTracks.count > 0) {
[self.videoInput requestMediaDataWhenReadyOnQueue:self.inputQueue usingBlock:^
{
if (![wself encodeReadySamplesFromOutput:wself.videoOutput toInput:wself.videoInput])
{
@synchronized(wself)
{
videoCompleted = YES;
if (audioCompleted)
{
[wself finish];
}
}
}
}];
}
else {
videoCompleted = YES;
}
if (!self.audioOutput) {
audioCompleted = YES;
} else {
[self.audioInput requestMediaDataWhenReadyOnQueue:self.inputQueue usingBlock:^
{
if (![wself encodeReadySamplesFromOutput:wself.audioOutput toInput:wself.audioInput])
{
@synchronized(wself)
{
audioCompleted = YES;
if (videoCompleted)
{
[wself finish];
}
}
}
}];
}
其中self.videoInput
里的self.videoSettings
我們需要對(duì)視頻壓縮參數(shù)做設(shè)置
self.videoSettings = @
{
AVVideoCodecKey: AVVideoCodecH264,
AVVideoWidthKey: @1280,
AVVideoHeightKey: @720,
AVVideoCompressionPropertiesKey: @
{
AVVideoAverageBitRateKey: @3000000,
AVVideoProfileLevelKey: AVVideoProfileLevelH264High40,
},
};
封裝好的控件可以參考https://github.com/rs/SDAVAssetExportSession呢燥。
6.最終效果
通過(guò)下圖我們可以看到崭添,視頻已經(jīng)成功被壓縮成10m左右。結(jié)合視頻效果叛氨,這個(gè)壓縮成果我們還是很滿意的呼渣。
下面是視頻部分畫(huà)面截圖的效果:
7.視頻轉(zhuǎn)碼時(shí)遇到的坑
使用 SDAVAssetExportSession
時(shí)遇到一個(gè)坑畸裳,大部分視頻轉(zhuǎn)碼沒(méi)問(wèn)題缰犁,部分視頻轉(zhuǎn)碼會(huì)有黑屏問(wèn)題,最后定位出現(xiàn)問(wèn)題的代碼如下:
- (AVMutableVideoComposition *)buildDefaultVideoComposition
{
AVMutableVideoComposition *videoComposition = [AVMutableVideoComposition videoComposition];
AVAssetTrack *videoTrack = [[self.asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
// get the frame rate from videoSettings, if not set then try to get it from the video track,
// if not set (mainly when asset is AVComposition) then use the default frame rate of 30
float trackFrameRate = 0;
if (self.videoSettings)
{
NSDictionary *videoCompressionProperties = [self.videoSettings objectForKey:AVVideoCompressionPropertiesKey];
if (videoCompressionProperties)
{
NSNumber *frameRate = [videoCompressionProperties objectForKey:AVVideoAverageNonDroppableFrameRateKey];
if (frameRate)
{
trackFrameRate = frameRate.floatValue;
}
}
}
else
{
trackFrameRate = [videoTrack nominalFrameRate];
}
if (trackFrameRate == 0)
{
trackFrameRate = 30;
}
videoComposition.frameDuration = CMTimeMake(1, trackFrameRate);
CGSize targetSize = CGSizeMake([self.videoSettings[AVVideoWidthKey] floatValue], [self.videoSettings[AVVideoHeightKey] floatValue]);
CGSize naturalSize = [videoTrack naturalSize];
CGAffineTransform transform = videoTrack.preferredTransform;
// Workaround radar 31928389, see https://github.com/rs/SDAVAssetExportSession/pull/70 for more info
if (transform.ty == -560) {
transform.ty = 0;
}
if (transform.tx == -560) {
transform.tx = 0;
}
CGFloat videoAngleInDegree = atan2(transform.b, transform.a) * 180 / M_PI;
if (videoAngleInDegree == 90 || videoAngleInDegree == -90) {
CGFloat width = naturalSize.width;
naturalSize.width = naturalSize.height;
naturalSize.height = width;
}
videoComposition.renderSize = naturalSize;
// center inside
{
float ratio;
float xratio = targetSize.width / naturalSize.width;
float yratio = targetSize.height / naturalSize.height;
ratio = MIN(xratio, yratio);
float postWidth = naturalSize.width * ratio;
float postHeight = naturalSize.height * ratio;
float transx = (targetSize.width - postWidth) / 2;
float transy = (targetSize.height - postHeight) / 2;
CGAffineTransform matrix = CGAffineTransformMakeTranslation(transx / xratio, transy / yratio);
matrix = CGAffineTransformScale(matrix, ratio / xratio, ratio / yratio);
transform = CGAffineTransformConcat(transform, matrix);
}
// Make a "pass through video track" video composition.
AVMutableVideoCompositionInstruction *passThroughInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
passThroughInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, self.asset.duration);
AVMutableVideoCompositionLayerInstruction *passThroughLayer = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoTrack];
[passThroughLayer setTransform:transform atTime:kCMTimeZero];
passThroughInstruction.layerInstructions = @[passThroughLayer];
videoComposition.instructions = @[passThroughInstruction];
return videoComposition;
}
1. transform 不正確引起的黑屏
CGAffineTransform transform = videoTrack.preferredTransform;
1.參考評(píng)論區(qū) @baopanpan同學(xué)的說(shuō)法,TZImagePickerController
可以解決帅容,找到代碼試了一下颇象,確實(shí)不會(huì)出現(xiàn)黑屏的問(wèn)題了,代碼如下
/// 獲取優(yōu)化后的視頻轉(zhuǎn)向信息
- (AVMutableVideoComposition *)fixedCompositionWithAsset:(AVAsset *)videoAsset {
AVMutableVideoComposition *videoComposition = [AVMutableVideoComposition videoComposition];
// 視頻轉(zhuǎn)向
int degrees = [self degressFromVideoFileWithAsset:videoAsset];
if (degrees != 0) {
CGAffineTransform translateToCenter;
CGAffineTransform mixedTransform;
videoComposition.frameDuration = CMTimeMake(1, 30);
NSArray *tracks = [videoAsset tracksWithMediaType:AVMediaTypeVideo];
AVAssetTrack *videoTrack = [tracks objectAtIndex:0];
AVMutableVideoCompositionInstruction *roateInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
roateInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, [videoAsset duration]);
AVMutableVideoCompositionLayerInstruction *roateLayerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoTrack];
if (degrees == 90) {
// 順時(shí)針旋轉(zhuǎn)90°
translateToCenter = CGAffineTransformMakeTranslation(videoTrack.naturalSize.height, 0.0);
mixedTransform = CGAffineTransformRotate(translateToCenter,M_PI_2);
videoComposition.renderSize = CGSizeMake(videoTrack.naturalSize.height,videoTrack.naturalSize.width);
[roateLayerInstruction setTransform:mixedTransform atTime:kCMTimeZero];
} else if(degrees == 180){
// 順時(shí)針旋轉(zhuǎn)180°
translateToCenter = CGAffineTransformMakeTranslation(videoTrack.naturalSize.width, videoTrack.naturalSize.height);
mixedTransform = CGAffineTransformRotate(translateToCenter,M_PI);
videoComposition.renderSize = CGSizeMake(videoTrack.naturalSize.width,videoTrack.naturalSize.height);
[roateLayerInstruction setTransform:mixedTransform atTime:kCMTimeZero];
} else if(degrees == 270){
// 順時(shí)針旋轉(zhuǎn)270°
translateToCenter = CGAffineTransformMakeTranslation(0.0, videoTrack.naturalSize.width);
mixedTransform = CGAffineTransformRotate(translateToCenter,M_PI_2*3.0);
videoComposition.renderSize = CGSizeMake(videoTrack.naturalSize.height,videoTrack.naturalSize.width);
[roateLayerInstruction setTransform:mixedTransform atTime:kCMTimeZero];
}else {//增加異常處理
videoComposition.renderSize = CGSizeMake(videoTrack.naturalSize.width,videoTrack.naturalSize.height);
}
roateInstruction.layerInstructions = @[roateLayerInstruction];
// 加入視頻方向信息
videoComposition.instructions = @[roateInstruction];
}
return videoComposition;
}
/// 獲取視頻角度
- (int)degressFromVideoFileWithAsset:(AVAsset *)asset {
int degress = 0;
NSArray *tracks = [asset tracksWithMediaType:AVMediaTypeVideo];
if([tracks count] > 0) {
AVAssetTrack *videoTrack = [tracks objectAtIndex:0];
CGAffineTransform t = videoTrack.preferredTransform;
if(t.a == 0 && t.b == 1.0 && t.c == -1.0 && t.d == 0){
// Portrait
degress = 90;
} else if(t.a == 0 && t.b == -1.0 && t.c == 1.0 && t.d == 0){
// PortraitUpsideDown
degress = 270;
} else if(t.a == 1.0 && t.b == 0 && t.c == 0 && t.d == 1.0){
// LandscapeRight
degress = 0;
} else if(t.a == -1.0 && t.b == 0 && t.c == 0 && t.d == -1.0){
// LandscapeLeft
degress = 180;
}
}
return degress;
}
2. naturalSize 不正確的坑
之前用模擬器錄屏得到一個(gè)視頻并徘,調(diào)用[videoTrack naturalSize]
的時(shí)候遣钳,得到的size為 (CGSize) naturalSize = (width = 828, height = 0.02734375)
,明顯是不正確的,這個(gè)暫時(shí)沒(méi)有找到解決辦法麦乞,知道的同學(xué)可以在下面評(píng)論一下蕴茴。
3. 視頻黑邊問(wèn)題
視頻黑邊應(yīng)該是視頻源尺寸和目標(biāo)尺寸比例不一致造成的,需要根據(jù)原尺寸的比例算出目標(biāo)尺寸
CGSize targetSize = CGSizeMake(videoAsset.pixelWidth, videoAsset.pixelHeight);
//尺寸過(guò)大才壓縮姐直,否則不更改targetSize
if (targetSize.width * targetSize.height > 1280 * 720) {
int width = 0,height = 0;
if (targetSize.width > targetSize.height) {
width = 1280;
height = 1280 * targetSize.height/targetSize.width;
}else {
width = 720;
height = 720 * targetSize.height/targetSize.width;
}
targetSize = CGSizeMake(width, height);
}else if (targetSize.width == 0 || targetSize.height == 0) {//異常情況處理
targetSize = CGSizeMake(720, 1280);
}