1.Metal是蘋果的圖形圖像渲染框架凉泄，也可以實(shí)現(xiàn)普通的GPU高并發(fā)計(jì)算荤胁，流程與OpenGL ES非常類似缚甩，且其里面也有封裝好的濾鏡，可以直接使用無需自己寫.metal渲染的代碼實(shí)現(xiàn)弊决，視情況而定
2.主要思路
Metal視頻渲染和圖片渲染流程和思路完全一致噪舀，只不過圖片讀取的是圖片紋理魁淳，視頻讀取的是Y和UV兩個(gè)紋理，外加需要傳遞YUV到RGB的顏色轉(zhuǎn)換矩陣和偏移量
1.OC代碼段編寫Metal渲染所需的相關(guān)流程代碼
2.OC和metal文件之間的橋接文件提供數(shù)據(jù)類型和端口索引
3.編寫metal的 vertex頂點(diǎn)函數(shù) fragment片元函數(shù)
4.在MTKView的代理方法里面渲染每一幀的圖片紋理數(shù)據(jù)
5.視頻紋理的獲取和顏色轉(zhuǎn)換矩陣設(shè)置
6.視頻流捕獲AVFoundation与倡，具體在前面的文字里有展開分析
3.具體步驟
1. OC代碼段編寫Metal渲染所需的相關(guān)流程代碼
1.創(chuàng)建MTKView界逛，設(shè)置代理，這是Metal渲染的目標(biāo)view纺座，這里的代碼都是面向協(xié)議的編碼息拜，MTKView的 id <MTLDevice> device，device非常重要比驻，device可以理解成GPU该溯，其他的很多代碼都是跟device相關(guān)，里面的渲染管道别惦，命令隊(duì)列，命令緩存區(qū)夫椭，渲染命令編碼器掸掸，紋理，各種緩存區(qū)蹭秋，MTLLibrary都需要device生成
2.創(chuàng)建紋理
3.創(chuàng)建頂點(diǎn)
4.創(chuàng)建渲染管道扰付，加載頂點(diǎn)函數(shù)和片元函數(shù)
2.OC和metal文件之間的橋接文件提供數(shù)據(jù)類型和端口索引
1.具體的數(shù)據(jù)結(jié)構(gòu)和索引視情況而定，頂點(diǎn)坐標(biāo)仁讨，紋理坐標(biāo)羽莺，圖片紋理，頂點(diǎn)索引洞豁，紋理索引盐固，其他需要的參數(shù)，比如時(shí)間丈挟，數(shù)組刁卜，矩陣，需要的都可以
2.視頻的紋理是Y紋理 和 UV紋理兩個(gè)部分曙咽，需要增加YUV到RGB的顏色轉(zhuǎn)換矩陣
3.編寫metal的 vertex頂點(diǎn)函數(shù) fragment片元函數(shù)
1.編寫metal函數(shù)蛔趴，語法看起來有些復(fù)雜，其實(shí)還好例朱，要注意與橋接對(duì)象的數(shù)據(jù)類型的對(duì)應(yīng)關(guān)系孝情，主要有數(shù)據(jù)結(jié)構(gòu)和端口索引值，類型關(guān)鍵字比如 [[position]] [[buffer]] [[stage_in]] 等等洒嗤，函數(shù)修飾符箫荡，參數(shù)地址空間修飾符等，其余編碼的思路與OpenGL ES基本一致烁竭，只是語法有些差異
3.編寫metal的 vertex頂點(diǎn)函數(shù) fragment片元函數(shù)
1.編寫metal函數(shù)菲茬，語法看起來有些復(fù)雜，其實(shí)還好，要注意與橋接對(duì)象的數(shù)據(jù)類型的對(duì)應(yīng)關(guān)系婉弹，主要有數(shù)據(jù)結(jié)構(gòu)和端口索引值睬魂，類型關(guān)鍵字比如 [[position]] [[buffer]] [[stage_in]] 等等，函數(shù)修飾符镀赌，參數(shù)地址空間修飾符等氯哮，其余編碼的思路與OpenGL ES基本一致，只是語法有些差異
4.在MTKView的代理方法里面渲染每一幀的圖片紋理數(shù)據(jù)
MTKView的兩個(gè)代理方法
// 設(shè)置渲染范圍
- (void)mtkView:(nonnull MTKView *)view drawableSizeWillChange:(CGSize)size {
self.viewportSize = (vector_int2){size.width, size.height};
}
// MTKViewDelegate
// 每一幀渲染命令的具體實(shí)現(xiàn)
- (void)drawInMTKView:(nonnull MTKView *)view
5.視頻紋理的獲取和顏色轉(zhuǎn)換矩陣設(shè)置
1.視頻流數(shù)據(jù)CMSampleBuffer或者是CVPixelBuffer 解析成Metal的紋理CVMetalTextureRef 類型商佛，分為Y紋理和UV紋理喉钢，需要注意的是采集的顏色輸出格式和Y紋理和UV紋理的MTLPixelFormat格式，否則顏色會(huì)異常良姆，偏紅偏綠偏藍(lán)肠虽，這部分獲取的Y和UV紋理用來設(shè)置MTKView渲染代理的紋理數(shù)據(jù)
2.設(shè)置視頻顏色格式Y(jié)UV和渲染紋理的RGB顏色轉(zhuǎn)換矩陣，這也需要傳遞的參數(shù)
4.代碼實(shí)現(xiàn)
1.OC類->客戶端代碼玛追，一般針對(duì)于Metal函數(shù) 或者OpenGL ES的shader文件税课，我們寫的CPU執(zhí)行的代碼相對(duì)于以上兩種是GPU執(zhí)行的代碼統(tǒng)稱為客戶端
1.頭文件 變量 屬性
#import <MetalKit/MetalKit.h>
#import <Metal/Metal.h>
#import <AVFoundation/AVFoundation.h>
#import "VideoCaptureView.h"
#import "YYVideoShaderTypes.h"
@interface MetalVideoFilterView ()
<MTKViewDelegate, VideoCaptureViewDelegate>
{
CGRect m_frame;
BOOL isChangeFillMode;
}
// 渲染范圍
@property (nonatomic, assign) vector_int2 viewportSize;
// MTKView Metal渲染的view
@property (nonatomic, strong) MTKView * mtkView;
// 用來渲染的設(shè)備(GPU)
@property (nonatomic, strong) id <MTLDevice> device;
// 渲染管道，管理頂點(diǎn)函數(shù)和片元函數(shù)
@property (nonatomic, strong) id <MTLRenderPipelineState> renderPipelineState;
// 渲染指令隊(duì)列
@property (nonatomic, strong) id <MTLCommandQueue> commondQueue;
// 頂點(diǎn)緩存對(duì)象
@property (nonatomic, strong) id <MTLBuffer> vertexBuffer;
// 顏色轉(zhuǎn)換矩陣
@property (nonatomic, strong) id <MTLBuffer> convertMatrix;
// 紋理對(duì)象
@property (nonatomic, strong) id <MTLTexture> texture;
// 頂點(diǎn)數(shù)量
@property (nonatomic, assign) NSUInteger vertexCount;
// 獲取視頻數(shù)據(jù)
@property (nonatomic, strong) VideoCaptureView * m_videoCaptureView;
// 視頻數(shù)據(jù)
@property (atomic, assign) CVPixelBufferRef pixelBuffer;
// 紋理緩存 用于使用CVPixelBuffer格式的數(shù)據(jù)生成Metal的紋理
@property (nonatomic, assign) CVMetalTextureCacheRef textureCache;
// Y紋理
@property (atomic, strong) id <MTLTexture> textureY;
// UV紋理
@property (atomic, strong) id <MTLTexture> textureUV;
2.初始化調(diào)用
- (id)initWithFrame:(CGRect)frame {
self = [super initWithFrame:frame];
if (self) {
m_frame = frame;
[self createVideoCaptureView];
// 1.創(chuàng)建 MTKView
[self createMTKView];
// 2.設(shè)置頂點(diǎn)
[self setupVertexsWithWidthScaling:1.0f heightScaling:1.0f];
// 3. 設(shè)置顏色轉(zhuǎn)換矩陣 YUV->RGB
[self setupMatrix];
// 4.創(chuàng)建渲染管道
[self createPipeLineState];
}
return self;
}
3.創(chuàng)建MTKView
// 創(chuàng)建 MTKView
- (void)createMTKView {
MTKView * mtkView = [[MTKView alloc] initWithFrame:CGRectMake(0, 0, m_frame.size.width, m_frame.size.height)];
mtkView.delegate = self;
// 創(chuàng)建Device
mtkView.device = MTLCreateSystemDefaultDevice();
// 設(shè)置device
self.device = mtkView.device;
self.viewportSize = (vector_int2){mtkView.drawableSize.width, mtkView.drawableSize.height};
self.mtkView = mtkView;
[self addSubview:mtkView];
//3._textureCache的創(chuàng)建(通過CoreVideo提供給CPU/GPU高速緩存通道讀取紋理數(shù)據(jù))
CVMetalTextureCacheCreate(NULL, NULL, self.mtkView.device, NULL, &_textureCache);
}
4.設(shè)置頂點(diǎn)數(shù)據(jù) 頂點(diǎn)緩存
- (void)setupVertexsWithWidthScaling:(CGFloat)widthScaling heightScaling:(CGFloat)heightScaling {
// 1.頂點(diǎn)紋理數(shù)組
// 頂點(diǎn)x,y,z,w 紋理x,y
// 因?yàn)閳D片和視頻的默認(rèn)紋理是反的 左上 00 右上10 左下 01 右下11
// // 左下 右下
YYVideoVertex vertexArray[] = {
{{-1.0 * widthScaling, -1.0 * heightScaling, 0.0, 1.0}, {0.0, 1.0}},
{{1.0 * widthScaling, -1.0 * heightScaling, 0.0, 1.0}, {1.0, 1.0}},
{{-1.0 * widthScaling, 1.0 * heightScaling, 0.0, 1.0}, {0.0, 0.0}}, //左上
{{1.0 * widthScaling, 1.0 * heightScaling, 0.0, 1.0}, {1.0, 0.0}}, // 右上
};
// 2.生成頂點(diǎn)緩存
// MTLResourceStorageModeShared 屬性可共享的痊剖，表示可以被頂點(diǎn)或者片元函數(shù)或者其他函數(shù)使用
self.vertexBuffer = [self.device newBufferWithBytes:vertexArray length:sizeof(vertexArray) options:MTLResourceStorageModeShared];
// 3.獲取頂點(diǎn)數(shù)量
self.vertexCount = sizeof(vertexArray) / sizeof(YYVideoVertex);
}
5.設(shè)置渲染管道
// 4.創(chuàng)建渲染管道
// 根據(jù).metal里的函數(shù)名韩玩，使用MTLLibrary創(chuàng)建頂點(diǎn)函數(shù)和片元函數(shù)
// 從這里可以看出來，MTLLibrary里面包含所有.metal的文件陆馁，所以找颓，不同的.metal里面的函數(shù)名不能相同
// id <MTLDevice> 創(chuàng)建library、MTLRenderPipelineState叮贩、MTLCommandQueue
- (void)createPipeLineState {
// 1.從項(xiàng)目中加載.metal文件击狮，創(chuàng)建一個(gè)library
id <MTLLibrary> library = [self.device newDefaultLibrary];
// id <MTLDevice> -> id <MTLLibrary>
// 2.從庫中MTLLibrary，加載頂點(diǎn)函數(shù)
id <MTLFunction> vertexFunction = [library newFunctionWithName:@"vertexVideoShader"];
// 3.從庫中MTLLibrary妇汗，加載頂點(diǎn)函數(shù)
id <MTLFunction> fragmentFunction = [library newFunctionWithName:@"fragmentVideoShader"];
// 4.創(chuàng)建管道渲染管道描述符
MTLRenderPipelineDescriptor * renderPipeDescriptor = [[MTLRenderPipelineDescriptor alloc] init];
// 5.設(shè)置管道頂點(diǎn)函數(shù)和片元函數(shù)
renderPipeDescriptor.vertexFunction = vertexFunction;
renderPipeDescriptor.fragmentFunction = fragmentFunction;
// 6.設(shè)置管道描述的關(guān)聯(lián)顏色存儲(chǔ)方式
renderPipeDescriptor.colorAttachments[0].pixelFormat = self.mtkView.colorPixelFormat;
NSError * error = nil;
// 7.根據(jù)渲染管道描述符 創(chuàng)建渲染管道
id <MTLRenderPipelineState> renderPipelineState = [self.device newRenderPipelineStateWithDescriptor:renderPipeDescriptor error:&error];
self.renderPipelineState = renderPipelineState;
// id <MTLDevice> -> id <MTLRenderPipelineState>
// 8. 創(chuàng)建渲染指令隊(duì)列
id <MTLCommandQueue> commondQueue = [self.device newCommandQueue];
self.commondQueue = commondQueue;
// id <MTLDevice> -> id <MTLCommandQueue>
}
6.設(shè)置顏色轉(zhuǎn)換矩陣和偏移量的 數(shù)據(jù)結(jié)構(gòu)
// 設(shè)置YUV->RGB轉(zhuǎn)換的矩陣
- (void)setupMatrix {
//1.轉(zhuǎn)化矩陣
// BT.601, which is the standard for SDTV.
matrix_float3x3 kColorConversion601DefaultMatrix = (matrix_float3x3){
(simd_float3){1.164, 1.164, 1.164},
(simd_float3){0.0, -0.392, 2.017},
(simd_float3){1.596, -0.813, 0.0},
};
// BT.601 full range
matrix_float3x3 kColorConversion601FullRangeMatrix = (matrix_float3x3){
(simd_float3){1.0, 1.0, 1.0},
(simd_float3){0.0, -0.343, 1.765},
(simd_float3){1.4, -0.711, 0.0},
};
// BT.709, which is the standard for HDTV.
matrix_float3x3 kColorConversion709DefaultMatrix = (matrix_float3x3){
(simd_float3){1.164, 1.164, 1.164},
(simd_float3){0.0, -0.213, 2.112},
(simd_float3){1.793, -0.533, 0.0},
};
//2.偏移量
vector_float3 kColorConversion601FullRangeOffset = (vector_float3){ -(16.0/255.0), -0.5, -0.5};
//3.創(chuàng)建轉(zhuǎn)化矩陣結(jié)構(gòu)體.
YYVideoYUVToRGBConvertMatrix matrix;
//設(shè)置轉(zhuǎn)化矩陣
/*
kColorConversion601DefaultMatrix帘不；
kColorConversion601FullRangeMatrix；
kColorConversion709DefaultMatrix杨箭；
*/
matrix.matrix = kColorConversion601FullRangeMatrix;
//設(shè)置offset偏移量
matrix.offset = kColorConversion601FullRangeOffset;
//4.創(chuàng)建轉(zhuǎn)換矩陣緩存區(qū).
self.convertMatrix = [self.mtkView.device newBufferWithBytes:&matrix length:sizeof(YYVideoYUVToRGBConvertMatrix) options:MTLResourceStorageModeShared];
}
7.或者視頻流轉(zhuǎn)換為Y和UV紋理
// 視頻捕捉的View
- (void)createVideoCaptureView {
VideoCaptureView * videoCaptureView = [[VideoCaptureView alloc] initWithFrame:CGRectMake(0, 0, m_frame.size.width, m_frame.size.height)];
videoCaptureView.m_delegate = self;
self.m_videoCaptureView = videoCaptureView;
}
// 捕獲視頻的代理回調(diào)方法
- (void)outputCVPixelBuffer:(CVPixelBufferRef)pixelBuffer {
// 視頻對(duì)外輸出一般要CMSampleBufferRef從中提取出CVPixelBufferRef
// CVPixelBufferRetain 使用這個(gè)強(qiáng)引用
if (_pixelBuffer) {
CFRelease(_pixelBuffer);
}
_pixelBuffer = CVPixelBufferRetain(pixelBuffer);
[self setupTextureWithPixelBuffer:_pixelBuffer];
// NSLog(@"outputSampleBuffer");
// AVFoundation采集的CMSampleBufferRef不需要釋放寞焙，因?yàn)橹皇菂?shù)，代理方法內(nèi)部有釋放
// CFRelease(sampleBuffer);
}
// 生成Y紋理和UV紋理互婿，提供給MTKView的代理方法使用
- (void)setupTextureWithPixelBuffer:(CVPixelBufferRef)pixelBuffer {
if (!pixelBuffer) {
return;
}
// Y紋理
{
// 0表示Y紋理
size_t width = CVPixelBufferGetWidthOfPlane(pixelBuffer, 0);
size_t height = CVPixelBufferGetHeightOfPlane(pixelBuffer, 0);
if (isChangeFillMode) {
isChangeFillMode = NO;
[self resetVertexWithWidth:width height:height];
}
// 像素顏色格式 MTLPixelFormatR8Unorm 表示只取R一個(gè)顏色分支
MTLPixelFormat pixelFormat = MTLPixelFormatR8Unorm;
CVMetalTextureRef texture = NULL;
CVMetalTextureCacheRef textureCache = NULL;
// 開辟紋理緩存區(qū)
CVReturn TextureCacheCreateStatus =CVMetalTextureCacheCreate(NULL, NULL, self.device, NULL, &textureCache);
if(TextureCacheCreateStatus == kCVReturnSuccess) {
// NSLog(@"CVMetalTextureCacheCreate is success");
}
// 根據(jù)CVPixelBufferRef數(shù)據(jù)捣郊，使用CVMetalTextureCacheRef，創(chuàng)建CVMetalTextureRef
// 0表示Y紋理
CVReturn status = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, self.textureCache, pixelBuffer, NULL, pixelFormat, width, height, 0, &texture);
if(status == kCVReturnSuccess) {
// 根據(jù)紋理CVMetalTextureRef 創(chuàng)建id <MTLTexture>
self.textureY = CVMetalTextureGetTexture(texture);
// 使用完畢釋放資源
CFRelease(texture);
// NSLog(@"create Y texture is Success");
} else {
// NSLog(@"create Y texture is failed");
// NSLog(@"status == %d", status);
}
CFRelease(textureCache);
}
// UV紋理
{
// 1表示UV紋理
size_t width = CVPixelBufferGetWidthOfPlane(pixelBuffer, 1);
size_t height = CVPixelBufferGetHeightOfPlane(pixelBuffer, 1);
// 像素顏色格式 MTLPixelFormatRG8Unorm 表示只取RG兩個(gè)顏色分支
MTLPixelFormat pixelFormat = MTLPixelFormatRG8Unorm;
CVMetalTextureRef texture = NULL;
CVMetalTextureCacheRef textureCache = NULL;
// 開辟紋理緩存區(qū)
CVReturn TextureCacheCreateStatus =CVMetalTextureCacheCreate(NULL, NULL, self.device, NULL, &textureCache);
if(TextureCacheCreateStatus == kCVReturnSuccess) {
// NSLog(@"CVMetalTextureCacheCreate is success");
}
// 根據(jù)CVPixelBufferRef數(shù)據(jù)慈参，使用CVMetalTextureCacheRef呛牲，創(chuàng)建CVMetalTextureRef
// 1表示UV紋理
CVReturn status = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, self.textureCache, pixelBuffer, NULL, pixelFormat, width, height, 1, &texture);
if(status == kCVReturnSuccess) {
// 根據(jù)紋理CVMetalTextureRef 創(chuàng)建id <MTLTexture>
self.textureUV = CVMetalTextureGetTexture(texture);
// 使用完畢釋放資源
CFRelease(texture);
// NSLog(@"create UV texture is Success");
} else {
// NSLog(@"create UV texture is failed");
// NSLog(@"status == %d", status);
}
CFRelease(textureCache);
}
}
8.MTKView代理方法
// MTKViewDelegate 設(shè)置渲染區(qū)域或者是區(qū)域變動(dòng)時(shí)執(zhí)行
- (void)mtkView:(nonnull MTKView *)view drawableSizeWillChange:(CGSize)size {
self.viewportSize = (vector_int2){size.width, size.height};
}
// MTKViewDelegate 每一幀的渲染流程
- (void)drawInMTKView:(nonnull MTKView *)view {
if (self.textureY && self.textureUV) {
// 1.為當(dāng)前渲染的每個(gè)渲染傳遞創(chuàng)建一個(gè)新的命令緩沖區(qū)
id <MTLCommandBuffer> commandBuffer = [self.commondQueue commandBuffer];
//指定緩存區(qū)名稱
commandBuffer.label = @"EachCommand";
// 2.獲取渲染命令編碼器 MTLRenderCommandEncoder的描述符
// currentRenderPassDescriptor描述符包含currentDrawable's的紋理、視圖的深度驮配、模板和sample緩沖區(qū)和清晰的值娘扩。
// MTLRenderPassDescriptor描述一系列attachments的值着茸，類似GL的FrameBuffer；同時(shí)也用來創(chuàng)建MTLRenderCommandEncoder
MTLRenderPassDescriptor * renderPassDescriptor = view.currentRenderPassDescriptor;
if (renderPassDescriptor) {
// 設(shè)置默認(rèn)顏色 背景色
renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(1.0, 1.0, 1.0, 1.0f);
// 3.根據(jù)描述創(chuàng)建x 渲染命令編碼器
id <MTLRenderCommandEncoder> renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor];
// typedef struct {
// double originX, originY, width, height, znear, zfar;
// } MTLViewport;
// 4.設(shè)置繪制區(qū)域
[renderEncoder setViewport:(MTLViewport){0, 0, self.viewportSize.x, self.viewportSize.y, -1.0, 1.0}];
// 5.設(shè)置渲染管道
[renderEncoder setRenderPipelineState:self.renderPipelineState];
// 6.傳遞頂點(diǎn)緩存
[renderEncoder setVertexBuffer:self.vertexBuffer offset:0 atIndex:YYVideoVertexInputIndexVertexs];
// 7.傳遞顏色轉(zhuǎn)換矩陣
[renderEncoder setFragmentBuffer:self.convertMatrix offset:0 atIndex:YYVideoConvertMatrixIndexYUVToRGB];
// 設(shè)置Y UV紋理
if (self.textureY) {
// NSLog(@"setFragmentTexture textureY");
[renderEncoder setFragmentTexture:self.textureY atIndex:YYVidoTextureIndexYTexture];
self.textureY = nil;
}
if (self.textureUV) {
// NSLog(@"setFragmentTexture textureUV");
[renderEncoder setFragmentTexture:self.textureUV atIndex:YYVidoTextureIndexUVTexture];
self.textureUV = nil;
}
// 8.繪制
[renderEncoder drawPrimitives:MTLPrimitiveTypeTriangleStrip vertexStart:0 vertexCount:self.vertexCount];
// 9.命令結(jié)束
[renderEncoder endEncoding];
// 10.顯示
[commandBuffer presentDrawable:view.currentDrawable];
}
// 11. 提交
[commandBuffer commit];
}
}
9.視頻顯示模式設(shè)置琐旁，同Image顯示模式設(shè)置
- (void)resetVertexWithWidth:(CGFloat)width height:(CGFloat)height {
dispatch_async(dispatch_get_main_queue(), ^{
CGSize inputImageSize = CGSizeMake(width, height);
CGFloat heightScaling = 1.0, widthScaling = 1.0;
CGSize currentViewSize = self.bounds.size;
CGRect insetRect = AVMakeRectWithAspectRatioInsideRect(inputImageSize, self.bounds);
switch(self.fillMode)
{
case kMetalVideoFilterViewFillModeStretch:
{
widthScaling = 1.0;
heightScaling = 1.0;
};
break;
case kMetalVideoFilterViewFillModePreserveAspectRatio:
{
widthScaling = insetRect.size.width / currentViewSize.width;
heightScaling = insetRect.size.height / currentViewSize.height;
};
break;
case kMetalVideoFilterViewFillModePreserveAspectRatioAndFill:
{
widthScaling = currentViewSize.height / insetRect.size.height;
heightScaling = currentViewSize.width / insetRect.size.width;
};
break;
}
// NSLog(@"widthScaling == %lf", widthScaling);
// NSLog(@"heightScaling == %lf", heightScaling);
[self setupVertexsWithWidthScaling:widthScaling heightScaling:heightScaling];
});
}
- (void)setFillMode:(kMetalVideoFilterViewFillModeType)fillMode {
isChangeFillMode = YES;
_fillMode = fillMode;
}
2.橋接類 .h文件涮阔，里面有數(shù)據(jù)結(jié)構(gòu)和端口索引值
simd這個(gè)類提供一些公共的數(shù)據(jù)類型，客戶端和Metal都可以使用的類型
#include <simd/simd.h>
// 頂點(diǎn)坐標(biāo)和紋理坐標(biāo)數(shù)據(jù)結(jié)構(gòu)
typedef struct {
// 頂點(diǎn)坐標(biāo) 4維向量
vector_float4 position;
// 紋理坐標(biāo)
vector_float2 textureCoordinate;
} YYVideoVertex;
//顏色轉(zhuǎn)換數(shù)據(jù)結(jié)構(gòu) YUV轉(zhuǎn)RGBA 轉(zhuǎn)換矩陣
typedef struct {
//三維矩陣
matrix_float3x3 matrix;
//偏移量
vector_float3 offset;
} YYVideoYUVToRGBConvertMatrix;
// 頂點(diǎn)index
typedef enum {
YYVideoVertexInputIndexVertexs = 0,
} YYVideoVertexInputIndex;
// 紋理 index
typedef enum {
// Y紋理索引 index
YYVidoTextureIndexYTexture = 0,
// UV紋理索引 index
YYVidoTextureIndexUVTexture = 1,
} YYVideoTextureIndex;
// 顏色轉(zhuǎn)換結(jié)構(gòu)體 index
typedef enum {
YYVideoConvertMatrixIndexYUVToRGB = 0,
} YYVideoConvertMatrixIndex;
#endif /* YYVideoShaderTypes_h */
3.Metal代碼實(shí)現(xiàn) 頂點(diǎn)片元函數(shù)實(shí)現(xiàn)
1.需要注意灰殴，函數(shù)修飾符敬特，變量修飾符，地址空間修飾符牺陶，橋接或者輸出的數(shù)據(jù)結(jié)構(gòu)類型伟阔，索引值，內(nèi)部的邏輯編碼與OpenGL ES非常類似掰伸，邏輯一致
#include <metal_stdlib>
#import "YYVideoShaderTypes.h"
using namespace metal;
// 定義了一個(gè)類型為RasterizerData的結(jié)構(gòu)體皱炉，里面有一個(gè)float4向量和float2向量，其中float4被[[position]]修飾碱工，其表示的變量為頂點(diǎn)
typedef struct {
// float4 4維向量 clipSpacePosition參數(shù)名
// position 修飾符的表示頂點(diǎn) 語法是[[position]]娃承，這是蘋果內(nèi)置的語法和position關(guān)鍵字不能改變
float4 clipSpacePosition [[position]];
// float2 2維向量 表示紋理
float2 textureCoordinate;
} RasterizerData;
// 頂點(diǎn)函數(shù)通過一個(gè)自定義的結(jié)構(gòu)體，返回對(duì)應(yīng)的數(shù)據(jù)怕篷，頂點(diǎn)函數(shù)的輸入?yún)?shù)也可以是自定義結(jié)構(gòu)體
// 頂點(diǎn)函數(shù)
// vertex 函數(shù)修飾符表示頂點(diǎn)函數(shù)，
// RasterizerData返回值類型酗昼，
// vertexImageShader函數(shù)名
// vertex_id 頂點(diǎn)id修飾符廊谓，蘋果內(nèi)置不可變，[[vertex_id]]
// buffer 緩存數(shù)據(jù)修飾符麻削，蘋果內(nèi)置不可變蒸痹，YYImageVertexInputIndexVertexs是索引
// [[buffer(YYImageVertexInputIndexVertexs)]]
// constant 變量類型修飾符，表示存儲(chǔ)在device區(qū)域
vertex RasterizerData vertexVideoShader(uint vertexID [[vertex_id]], constant YYVideoVertex * vertexArray [[buffer(YYVideoVertexInputIndexVertexs)]]) {
RasterizerData outData;
// 獲取YYVertex里面的頂點(diǎn)坐標(biāo)和紋理坐標(biāo)
outData.clipSpacePosition = vertexArray[vertexID].position;
outData.textureCoordinate = vertexArray[vertexID].textureCoordinate;
return outData;
}
// 片元函數(shù)
// fragment 函數(shù)修飾符表示片元函數(shù) float4 返回值類型->顏色RGBA fragmentImageShader 函數(shù)名
// RasterizerData 參數(shù)類型 input 變量名
// [[stage_in] stage_in表示這個(gè)數(shù)據(jù)來自光柵化呛哟。（光柵化是頂點(diǎn)處理之后的步驟叠荠，業(yè)務(wù)層無法修改）
// texture2d 類型表示紋理 baseTexture 變量名
// [[ texture(index)]] 紋理修飾符
// 可以加索引 [[ texture(0)]]紋理0， [[ texture(1)]]紋理1
// YYImageTextureIndexBaseTexture表示紋理索引
fragment float4 fragmentVideoShader(RasterizerData input [[stage_in]], texture2d<float> textureY [[texture (YYVidoTextureIndexYTexture)]], texture2d<float> textureUV [[texture(YYVidoTextureIndexUVTexture)]], constant YYVideoYUVToRGBConvertMatrix * convertMatrix [[buffer(YYVideoConvertMatrixIndexYUVToRGB)]]) {
// constexpr 修飾符
// sampler 采樣器
// textureSampler 采樣器變量名
// mag_filter:: linear, min_filter:: linear 設(shè)置放大縮小過濾方式
constexpr sampler textureSampler(mag_filter:: linear, min_filter:: linear);
// 得到紋理對(duì)應(yīng)位置的顏色
// float YColor = textureY.sample(textureSampler, input.textureCoordinate).r;
// float2 UVColor = textureUV.sample(textureSampler, input.textureCoordinate).rg;
// float3 color = float3(YColor, UVColor);
// float3 outputColor = matrix->matrix * (color + matrix->offset);
float3 yuv = float3(textureY.sample(textureSampler, input.textureCoordinate).r,
textureUV.sample(textureSampler, input.textureCoordinate).rg);
//3.將YUV 轉(zhuǎn)化為 RGB值.convertMatrix->matrix * (YUV + convertMatrix->offset)
float3 rgb = convertMatrix->matrix * (yuv + convertMatrix->offset);
// 返回顏色值
return float4(rgb, 1.0);
}
