前言
上一篇介紹UIImagePickerController 和AVCaptureSession+AVCaptureMovieFileOutput（音視頻采集學(xué)習(xí)筆記（一）), 這篇介紹AVCaptureSession + AVCaptureVideoDataOutput +AVAssetWriter的實(shí)現(xiàn)盈咳，效果圖如下所示 (采集實(shí)現(xiàn)Demo)
AVCaptureSession + AVCaptureVideoDataOutput+AVAssetWriter
音視頻輸入和輸出都是通過(guò)AVCaptureSession來(lái)管理碰凶，通過(guò)AVCaptureVideoDataOutput可以實(shí)時(shí)的獲取采集的每一幀的數(shù)據(jù)進(jìn)行一系列的操作(增加濾鏡，根據(jù)設(shè)備方向轉(zhuǎn)動(dòng)視圖等) 炕檩，最后通過(guò)AVAssetWriter進(jìn)行編碼處理窖剑，存儲(chǔ)
流程
1.創(chuàng)建捕捉會(huì)話
2.設(shè)置視頻的輸入
3.設(shè)置音頻的輸入
4.設(shè)置視頻的輸出
5.設(shè)置音頻的輸出
6.添加視頻預(yù)覽層
7.添加濾鏡
8.寫(xiě)入音視頻數(shù)據(jù)
- 創(chuàng)建捕捉會(huì)話
fileprivate lazy var session : AVCaptureSession = {
let session = AVCaptureSession()
session.sessionPreset = .vga640x480
return session
}()
- 視頻的輸入
@available(iOS 10.2, *)
func setUpVideo(position: AVCaptureDevice.Position) {
currentDevicePosition = position
// 視頻輸入設(shè)備
let videoCaptureDevice = AVCaptureDevice.default(.builtInWideAngleCamera,
for: .video,
position: position)
// 視頻輸入源
do {
videoInput = try AVCaptureDeviceInput.init(device: videoCaptureDevice!)
} catch {
}
if self.session.canAddInput(videoInput) {
self.session.addInput(videoInput)
}
if let dataOutPut = dataOutPut {
let connection = dataOutPut.connection(with: .video)
// 相機(jī)前置設(shè)置鏡像
connection?.isVideoMirrored = currentDevicePosition == .front
}
}
- 音頻的輸入
func setUpAudio() {
let audioCaptureDevice = AVCaptureDevice.default(for: .audio)
do {
audioInput = try AVCaptureDeviceInput(device: audioCaptureDevice!)
} catch {
}
if session.canAddInput(audioInput) {
self.session.addInput(audioInput)
}
}
- 視頻的輸出
func setUpDataOutPut() {
dataOutPut = AVCaptureVideoDataOutput()
dataOutPut.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String : Int(kCVPixelFormatType_32BGRA)]
dataOutPut.alwaysDiscardsLateVideoFrames = true
if session.canAddOutput(dataOutPut) {
session.addOutput(dataOutPut)
}
dataOutPut.setSampleBufferDelegate(self, queue: DispatchQueue(label: "VideoQueue"))
}
- 音頻的輸出
func setUpAudioOutPut() {
let audioOutPut = AVCaptureAudioDataOutput()
if session.canAddOutput(audioOutPut) {
session.addOutput(audioOutPut)
}
audioOutPut.setSampleBufferDelegate(self, queue: DispatchQueue(label: "audioQueue"))
}
- 視頻預(yù)覽層
func setUpLayerView() {
previewLayer = CALayer()
previewLayer.anchorPoint = CGPoint.zero
// 默認(rèn)為手機(jī)水平坚洽，home在右邊的方向， 所以previewLayer的寬對(duì)應(yīng)sessionPreset的高西土，高度同理
previewLayer.frame = CGRect.init(x: 0, y: 75, width: 480, height: 640)
self.view.contentMode = .scaleAspectFit
self.view.layer.insertSublayer(previewLayer, at: 0)
}
- 添加濾鏡 (UIImage, CGImageRef, CIImage)
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
// 采集后的圖像是未編碼的CMSampleBuffer形式讶舰，利用給定的接口函數(shù)CMSampleBufferGetImageBuffer從中提取出CVPixelBufferRef
let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)CMSampleBufferGetImageBuffer(sampleBuffer)
// 實(shí)例CIImage
var outImage = CIImage(cvPixelBuffer: imageBuffer!)
// CIFilter濾鏡并給濾鏡設(shè)置屬性（KVC）
if filter != nil {
filter.setValue(outImage, forKey: kCIInputImageKey)
outImage = filter.outputImage!
}
// 根據(jù)設(shè)備方向轉(zhuǎn)動(dòng)視圖
if orientation == UIDeviceOrientation.portrait {
let rotationAngle = currentDevicePosition == .front ? CGFloat.pi / 2.0 : -CGFloat.pi / 2.0
t = CGAffineTransform(rotationAngle: rotationAngle)
} else if orientation == UIDeviceOrientation.portraitUpsideDown {
t = CGAffineTransform(rotationAngle: CGFloat.pi / 2.0)
} else if (orientation == UIDeviceOrientation.landscapeRight) {
t = CGAffineTransform(rotationAngle: CGFloat.pi)
} else {
t = CGAffineTransform(rotationAngle: 0)
}
outImage = outImage.transformed(by: t)
// context 轉(zhuǎn)化為CGImage (非位圖圖片，不能保存到相冊(cè)需了，不能轉(zhuǎn)換為NSData (jpeg png)）
let cgImage = context.createCGImage(outImage, from: outImage.extent)
// 賦給previewLayer進(jìn)行顯示
DispatchQueue.main.async {
self.previewLayer.contents = cgImage
}
}
CIContext的創(chuàng)建
lazy var context: CIContext = {
let eaglContext = EAGLContext(api: EAGLRenderingAPI.openGLES2)
let options = [CIContextOption.workingColorSpace : NSNull()]
return CIContext(eaglContext: eaglContext!, options: options)
}()
處理前置鏡像問(wèn)題（呈像相反）：
默認(rèn)系統(tǒng)前置相機(jī)是非鏡像的跳昼，就是呈現(xiàn)的圖像為鏡子相反的，因?yàn)橹髁饔脩舳剂?xí)慣這個(gè)鏡像功能肋乍，所以我們?cè)谝曨l的輸入的時(shí)候設(shè)置了前置isVideoMirrored為true鹅颊，這時(shí)候前置就為鏡像了，由此也有一個(gè)問(wèn)題墓造，這時(shí)候當(dāng)設(shè)備方向?yàn)榇怪保╤ome鍵在底部）堪伍，前置呈像為右锚烦，后置為左涮俄，所以通過(guò)設(shè)備方向轉(zhuǎn)動(dòng)視圖需要根據(jù)相機(jī)方向來(lái)逆時(shí)針轉(zhuǎn)動(dòng)或順時(shí)針轉(zhuǎn)動(dòng)
關(guān)于轉(zhuǎn)動(dòng)視圖的問(wèn)題 ：
這里寫(xiě)的是根據(jù)設(shè)備方向從而去轉(zhuǎn)動(dòng)視圖顯示刘离，這樣實(shí)現(xiàn)也可以硫惕，但是有一個(gè)弊端踪旷，錄制中無(wú)法轉(zhuǎn)動(dòng)相機(jī)方向令野，否則其中部分畫(huà)面反的，而且實(shí)現(xiàn)更麻煩餐抢，通過(guò)下一章GpuImage2的學(xué)習(xí)碳锈，發(fā)現(xiàn)通過(guò)AVCaptureConnection的videoOrientation屬性設(shè)置portrait就可以自動(dòng)實(shí)現(xiàn)根據(jù)設(shè)備方向而轉(zhuǎn)動(dòng) (AVCaptureConnection)
if let dataOutPut = dataOutPut {
let connection = dataOutPut.connection(with: .video)
connection?.videoOrientation = .portrait
}
- 寫(xiě)入音視頻數(shù)據(jù)
流程
1.創(chuàng)建AssetWriter
2.創(chuàng)建AssetWriterVideoInput
3.創(chuàng)建AssetWriterPixelBufferInput
4.創(chuàng)建AssetWriterAudioInput
5.添加音視頻寫(xiě)入的輸出到AssetWriter里绞呈，開(kāi)始寫(xiě)入
6.AVCaptureVideoDataOutputSampleBufferDelegate實(shí)現(xiàn)音視頻數(shù)據(jù)寫(xiě)入
(1) 創(chuàng)建AssetWriter
// 移除之前存儲(chǔ)的數(shù)據(jù)
if fileManager.fileExists(atPath: self.videoStoragePath()) {
do {
try fileManager.removeItem(atPath: self.videoStoragePath())
} catch {
}
}
let path = self.videoStoragePath()
videoUrl = URL.init(fileURLWithPath: path)
do {
assetWriter = try AVAssetWriter.init(outputURL: videoUrl as URL, fileType: .mov)
} catch {
}
(2) 創(chuàng)建AssetWriterVideoInput （音視頻開(kāi)發(fā)概念篇）
let numPixels = SCREEN_WIDTH * SCREEN_HEIGHT
let bitsPerPixel = 6.0 as CGFloat
let bitsPerSecond = numPixels * bitsPerPixel
let compressionProperties = [AVVideoAverageBitRateKey : bitsPerSecond,// 比特率(碼率)
AVVideoExpectedSourceFrameRateKey : 30,// 幀率
AVVideoMaxKeyFrameIntervalKey : 30,
AVVideoProfileLevelKey : AVVideoProfileLevelH264BaselineAutoLevel
] as [String : Any]
//視頻屬性
let videoCompressionSettings = [AVVideoCodecKey : AVVideoCodecType.h264, // 編碼格式，一般選h264,硬件編碼
AVVideoScalingModeKey : AVVideoScalingModeResizeAspectFill, // 填充模式
AVVideoWidthKey : 1920, // 視頻寬度碗誉，以手機(jī)水平，home 在右邊的方向
AVVideoHeightKey : 1080, // 視頻高度尝苇，以手機(jī)水平，home 在右邊的方向
AVVideoCompressionPropertiesKey : compressionProperties] as [String : Any]
assetWriterVideoInput = AVAssetWriterInput(mediaType: .video,
outputSettings: videoCompressionSettings)
//expectsMediaDataInRealTime 必須設(shè)為true非竿，需要從capture session 實(shí)時(shí)獲取數(shù)據(jù)
assetWriterVideoInput.expectsMediaDataInRealTime = true;
let rotationAngle = currentDevicePosition == .front ? CGFloat.pi / 2.0 : -CGFloat.pi / 2.0
// 視頻寫(xiě)入輸出轉(zhuǎn)動(dòng)方向
assetWriterVideoInput.transform = CGAffineTransform(rotationAngle: rotationAngle);
(3) 創(chuàng)建AssetWriterPixelBufferInput
let sourcePixelBufferAttributesDictionary = [
String(kCVPixelBufferPixelFormatTypeKey) : Int(kCVPixelFormatType_32BGRA), // 像素格式類型
String(kCVPixelBufferWidthKey) : 1920,
String(kCVPixelBufferHeightKey) : 1080,
// 允許在 OpenGL 的上下文中直接繪制解碼后的圖像蓖乘，而不是從總線和 CPU 之間復(fù)制數(shù)據(jù)嘉抒。這有時(shí)候被稱為零拷貝通道握牧，因?yàn)樵诶L制過(guò)程中沒(méi)有解碼的圖像被拷貝
String(kCVPixelFormatOpenGLESCompatibility) : kCFBooleanTrue
] as [String : Any]
assetWriterPixelBufferInput = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: assetWriterVideoInput,
sourcePixelBufferAttributes: sourcePixelBufferAttributesDictionary)
(4) 創(chuàng)建AssetWriterAudioInput
let audioCompressionSettings = [AVEncoderBitRatePerChannelKey : 28000,// 編碼時(shí)每個(gè)通道的比特率
AVFormatIDKey : kAudioFormatMPEG4AAC,// 音頻格式
AVNumberOfChannelsKey : 1,// 通道數(shù)1為單通道2為立體通道
AVSampleRateKey : 22050]// 采樣率輸入的模擬音頻信號(hào)每一秒的采樣數(shù)是影響音頻質(zhì)量和音頻文件大小非常重要的一個(gè)因素采樣率越小文件越小質(zhì)量越低如44.1kHz
assetWriterAudioInput = AVAssetWriterInput.init(mediaType: .audio,
outputSettings: audioCompressionSettings)
(5) 添加音視頻寫(xiě)入輸出到AssetWriter
if assetWriter.canAdd(assetWriterVideoInput) {
assetWriter.add(assetWriterVideoInput)
}
if assetWriter.canAdd(assetWriterAudioInput) {
assetWriter.add(assetWriterAudioInput)
}
assetWriter.startWriting()
self.assetWriter.startSession(atSourceTime: self.currentSampleTime)
self.isStart = true
(6) AVCaptureVideoDataOutputSampleBufferDelegate實(shí)現(xiàn)音視頻數(shù)據(jù)寫(xiě)入, 通過(guò)媒體類型判斷分別寫(xiě)入
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
// 寫(xiě)入音頻
if mediaType == kCMMediaType_Audio {
guard isStart else {
return
}
self.assetWriterAudioInput.append(sampleBuffer)
return
}
// 寫(xiě)入視頻數(shù)據(jù)
if isStart {
// 緩沖區(qū)中的數(shù)據(jù)是否已經(jīng)處理完成
if (self.assetWriterPixelBufferInput?.assetWriterInput.isReadyForMoreMediaData)! {
var newPixelBuffer: CVPixelBuffer? = nil
CVPixelBufferPoolCreatePixelBuffer(nil, self.assetWriterPixelBufferInput!.pixelBufferPool!, &newPixelBuffer)
self.context.render(outImage, to: newPixelBuffer!, bounds: outImage.extent, colorSpace: nil)
let success = self.assetWriterPixelBufferInput?.append(newPixelBuffer!, withPresentationTime: self.currentSampleTime!)
if success == false {
print("Pixel Buffer沒(méi)有附加成功")
}
}
}
}
擴(kuò)展:
AssetWriterPixelBufferInput （用法及理解）
H264視頻硬件編解碼說(shuō)明 (CMSampleBuffer結(jié)構(gòu)以及編解碼使用)
特別感謝以上分享文章的朋友
