上一個(gè)章節(jié)偎窘，介紹了YUV類型的視頻數(shù)據(jù)渲染乌助，這一篇文章，介紹一下RGB格式的視頻幀的繪制陌知；其實(shí)按難易程度應(yīng)該是 RGB的繪制比較簡(jiǎn)單他托，這篇文章順帶著優(yōu)化一下demo，上個(gè)demo 采用的是頂點(diǎn)數(shù)據(jù)繪制仆葡，這個(gè)demo 采用索引繪圖的方法赏参，減少頂點(diǎn)的數(shù)量；
首先呢沿盅，還是放上效果圖
思路：
這里我們的思路和繪制YUV視頻的思路一樣把篓，拿到一個(gè)mp4的文件，用AVPlayer對(duì)mp4文件資源加載 并播放嗡呼，然后對(duì) palyerItem 設(shè)置 AVPlayerItemVideoOutput纸俭，獲取到每秒30幀的視頻幀畫面，即pixeBuffer數(shù)據(jù) 南窗，然后將 視頻幀數(shù)據(jù) pixeBuffer通過(guò)Opengl ES 繪制出來(lái)揍很；
重點(diǎn)
不一樣的地方是，這里 kCVPixelBufferPixelFormatTypeKey 的配置 改成 kCVPixelFormatType_32BGRA, 因?yàn)檫@次我們要拿到的數(shù)據(jù)是RGBA的數(shù)據(jù) 不是YUV420f万伤；
1.設(shè)置AVplayer 和 AVPlayerItemVideoOutput 獲取到視頻幀
- (void)initParams {
/// 設(shè)置ItemVideoOutput 用于從AVPlayerItem 獲取實(shí)時(shí) 的視頻幀數(shù)據(jù)
/// 這里視頻幀的格式設(shè)置成 kCVPixelFormatType_32BGRA
NSDictionary *pixelBufferAttribute = @{(id)kCVPixelBufferPixelFormatTypeKey:@(kCVPixelFormatType_32BGRA)};
AVPlayerItemVideoOutput *videoOutput = [[AVPlayerItemVideoOutput alloc]initWithPixelBufferAttributes:pixelBufferAttribute];
_output = videoOutput;
///加載視頻資源
NSString *path = [[NSBundle mainBundle] pathForResource:@"download" ofType:@"mp4"];
NSURL *pathURL = [NSURL fileURLWithPath:path];
AVPlayerItem *item = [AVPlayerItem playerItemWithURL:pathURL];
[item addOutput:_output];
_resourceURL = pathURL;
/// 初始化播放器
[self playWithItem:item];
/// 開(kāi)始播放窒悔、并起一個(gè)定時(shí)器用于獲取當(dāng)前視頻幀
[self playPlayer];
}
創(chuàng)建一個(gè)每秒 30FPS 的定時(shí)器，用于在播放器成功播放后敌买，獲取32BGRA類型的CVPixelBufferRef简珠，這里可以用 dispatch_source_t 或CADisplayLink 等
- (void)startTimer {
[self stoptimer];
/// 每秒30幀
NSUInteger FPS = 30;
dispatch_queue_t _queue = dispatch_queue_create("com.render.statistics", NULL);
dispatch_source_t timer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, 0, _queue);
dispatch_time_t start = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(0 * NSEC_PER_SEC));
uint64_t interval = (uint64_t)(1.0/FPS * NSEC_PER_SEC);
dispatch_source_set_timer(timer, start, interval, 0);
__weak typeof(self) weakSelf = self;
dispatch_source_set_event_handler(timer, ^{
[weakSelf _tick];
});
dispatch_resume(timer);
_timer = timer;
}
- (void)stoptimer {
if (_timer) dispatch_source_cancel(_timer);
_timer = nil;
}
- (CVPixelBufferRef)_copyTextureFromPlayItem:(AVPlayerItem *)item {
AVPlayerItemVideoOutput *output = _output;
AVAsset *asset = item.asset;
CMTime time = item.currentTime;
float offset = time.value * 1.0f / time.timescale;
float frames = asset.duration.value * 1.0f / asset.duration.timescale;
if (offset == frames) {
[self pausePlayer];
return NULL;
}
CVPixelBufferRef pixelBuffer = [output copyPixelBufferForItemTime:time itemTimeForDisplay:nil];
return pixelBuffer;
}
創(chuàng)建一個(gè)每秒 30FPS 的定時(shí)器，用于在播放器成功播放后虹钮，獲取視頻幀聋庵，這里可以用 dispatch_source_t 或CADisplayLink 等
2. 設(shè)置CAEAGLLayer 并初始化需要的配置
前面的代碼是幫我們拿到一個(gè)BGRG 的 pixeBuffer 數(shù)據(jù)，下面的代碼才是真正的繪制過(guò)程芙粱；
和之前的代碼一樣祭玉，我們需要自定 UIView 或者一個(gè)CALayer 遵守 CAEAGLLayer 協(xié)議；
- (instancetype)initWithFrame:(CGRect)frame{
self = [super initWithFrame:frame];
if (self) {
//1.設(shè)置圖層
[self setupLayer];
//2.設(shè)置圖形上下文
[self setupContext];
//3. 加載shader
[self loadShaders];
//4.設(shè)置FrameBuffer
[self setupFrameBuffer];
}
return self;
}
+(Class)layerClass
{
return [CAEAGLLayer class];
}
3. 頂點(diǎn)坐標(biāo) 和 索引數(shù)據(jù)
GLfloat Vertex[] = {
-1.0f, 1.0f, 0.0f, 1.0f, //左上角A
1.0f, 1.0f, 1.0f, 1.0f, //右上角B
1.0f, -1.0f, 1.0f, 0.0f, //右下角C
-1.0f, -1.0f, 0.0f, 0.0f, //左下角D
};
GLuint elementIndex[] =
{
0, 3, 2,
0, 2, 1,
};
這里要介紹一下索引繪圖
原先的頂點(diǎn) 表示一個(gè)矩形 需要 6 個(gè)頂點(diǎn)數(shù)據(jù)繪制2 個(gè)三角形春畔；
如下圖所示的頂點(diǎn)中ABCD四個(gè)頂點(diǎn)的所以分別代表的索引為 0 脱货、1岛都、2、3
現(xiàn)在頂點(diǎn)數(shù)組只需要4個(gè)振峻，然后通過(guò)索引繪制的方式臼疫， 繪制2個(gè)三角形，繪制順序是 ADC 和CBA 扣孟；
如下圖所示幫助理解
3. 頂點(diǎn)著色器講解
position 頂點(diǎn)坐標(biāo)烫堤；
texCoord 紋理坐標(biāo);
preferredRotation 旋轉(zhuǎn)弧度;
texCoordVarying 傳給片元著色器的紋理坐標(biāo);
const NSString *vertexShader = @" \
attribute vec4 position; \
attribute vec4 texCoord; \
uniform float preferredRotation; \
varying vec2 texCoordVarying; \
void main() \
{ \
mat4 rotationMatrix = mat4(cos(preferredRotation), -sin(preferredRotation), 0.0, 0.0, \
sin(preferredRotation), cos(preferredRotation), 0.0, 0.0, \
0.0, 0.0, 1.0, 0.0, \
0.0, 0.0, 0.0, 1.0); \
gl_Position = position * rotationMatrix; \
texCoordVarying = texCoord.xy; \
} \
";
4.片元著色器講解
texture RGB類型的紋理 （圖片數(shù)據(jù)）;
texCoordVarying 紋理坐標(biāo);
const NSString *fragmentShader = @" \
varying highp vec2 texCoordVarying; \
uniform sampler2D texture; \
void main() \
{ \
gl_FragColor = texture2D(texture, texCoordVarying); \
} \
";
5.pixelBuffer 處理
較為重要的是 通過(guò) CVOpenGLESTextureCacheCreateTextureFromImage() 函數(shù) 拿到 RGB
圖像數(shù)據(jù)；
從上面上的頂點(diǎn)著色器 和片元著色器的講解中哈打，我們已經(jīng)知道了塔逃，現(xiàn)在需要對(duì) Y紋理 和 UV紋理 和相應(yīng)的圖層進(jìn)行綁定；
GLuint textureUniform = glGetUniformLocation(_program, "texture");
glUniform1i(textureUniform, 0);
float radius = 180 * 3.14159f / 180.0f;旋轉(zhuǎn)180度料仗，轉(zhuǎn)換成弧度湾盗，然后賦值 給 變量preferredRotation；
`GLint rotation = glGetUniformLocation(_program, "preferredRotation");`
`float radius = 180 * 3.14159f / 180.0f;`
`glUniform1i(textureUniform, 0);`
索引繪制用 glDrawElements()方法,傳入需要繪制的圖元類型立轧，索引個(gè)數(shù)格粪、數(shù)據(jù)類型、索引數(shù)據(jù)的首地址氛改；
glDrawElements(GL_TRIANGLES, sizeof(elementIndex)/sizeof(elementIndex[0]), GL_UNSIGNED_INT, elementIndex);
- (void)setPixelBuffer:(CVPixelBufferRef)pixelBuffer{
if (!pixelBuffer) {
return;
}
if (_pixelBuffer) {
CFRelease(_pixelBuffer);
}
_pixelBuffer = CVPixelBufferRetain(pixelBuffer);
[self ensureCurentContext];
uint32_t width = (int)CVPixelBufferGetWidth(_pixelBuffer);
uint32_t height = (int)CVPixelBufferGetHeight(_pixelBuffer);
CVOpenGLESTextureCacheRef _videoTextureCache;
CVReturn error = CVOpenGLESTextureCacheCreate(kCFAllocatorDefault, NULL, _eaglContext, NULL, &_videoTextureCache);
if (error != noErr) {
NSLog(@"CVOpenGLESTextureCacheCreate error %d",error);
return;
}
glActiveTexture(GL_TEXTURE0);
/// 獲取RGB紋理
error = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
_videoTextureCache,
_pixelBuffer,
NULL,
GL_TEXTURE_2D,
GL_RGBA,
width,
height,
GL_BGRA,
GL_UNSIGNED_BYTE,
0,
&_rgbTexture);
if (error) {
NSLog(@"error for reateTextureFromImage %d",error);
}
glBindTexture(CVOpenGLESTextureGetTarget(_rgbTexture), CVOpenGLESTextureGetName(_rgbTexture));
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glDisable(GL_DEPTH_TEST);
glBindFramebuffer(GL_FRAMEBUFFER, _frameBuffer);
glViewport(0, 0, _width, _height);
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
glUseProgram(_program);
GLuint textureUniform = glGetUniformLocation(_program, "texture");
/// uniform
GLint rotation = glGetUniformLocation(_program, "preferredRotation");
/// 旋轉(zhuǎn)角度
float radius = 180 * 3.14159f / 180.0f;
/// 定義uniform 采樣器對(duì)應(yīng)紋理 0 也就是Y 紋理
glUniform1i(textureUniform, 0);
/// 為當(dāng)前程序?qū)ο笾付║niform變量的值
glUniform1f(rotation, radius);
/// 開(kāi)始繪制
glDrawElements(GL_TRIANGLES, sizeof(elementIndex)/sizeof(elementIndex[0]), GL_UNSIGNED_INT, elementIndex);
[_eaglContext presentRenderbuffer:GL_RENDERBUFFER];
/// 清除紋理帐萎、釋放內(nèi)存
[self cleanUpTextures];
CVOpenGLESTextureCacheFlush(_videoTextureCache, 0);
if(_videoTextureCache) {
CFRelease(_videoTextureCache);
}
}
由于篇幅原因，本文不能將全部代碼貼出來(lái)胜卤，只是帖了一些核心的關(guān)鍵代碼和編碼思路疆导，如果有不明白的同學(xué)需要可以看源碼
源碼地址： https://github.com/hunter858/OpenGL_Study/OpenGL015-RGB視頻繪制
