實(shí)際sOpenGL ES除了可以結(jié)合MediaCodec添加濾鏡處理陨帆，進(jìn)行視頻錄制素跺。還可以結(jié)合MediaPlayer進(jìn)行視頻的播放齐苛，在視頻播放的時(shí)候添加濾鏡怎披。

核心原理

主要的原理就是使用SurfaceTexture來(lái)設(shè)置MediaPlayer的輸出胸嘁，使用SurfaceTexture創(chuàng)建一個(gè)Surface，將這個(gè)Surface作為MediaPlayer的輸出顯示凉逛。
和相機(jī)預(yù)覽類似性宏，通過SurfaceTexture的updateTexImage來(lái)刷新一幀的圖像數(shù)據(jù)，然后創(chuàng)建的紋理textureId來(lái)交給OpenGL進(jìn)行渲染状飞。

SurfaceTexture

SurfaceTexture是從Android3.0（API 11）加入的一個(gè)新類衔沼。這個(gè)類跟SurfaceView很像，可以從camera preview或者video decode里面獲取圖像流（image stream）昔瞧。但是指蚁，和SurfaceView不同的是，SurfaceTexture在接收?qǐng)D像流之后自晰，不需要顯示出來(lái)凝化，而是轉(zhuǎn)為GL外部紋理，因此可用于圖像流數(shù)據(jù)的二次處理（如Camera濾鏡酬荞，桌面特效等）搓劫。
SurfaceTexture從圖像流（來(lái)自Camera預(yù)覽瞧哟，視頻解碼，GL繪制場(chǎng)景等）中獲得幀數(shù)據(jù)枪向，當(dāng)調(diào)用updateTexImage()時(shí)勤揩，根據(jù)內(nèi)容流中最近的圖像更新SurfaceTexture對(duì)應(yīng)的GL紋理對(duì)象。

主要實(shí)現(xiàn)邏輯代碼

class MediaGLRenderer(ctx:Context?,listener: SurfaceTexture.OnFrameAvailableListener?):GLSurfaceView.Renderer {
    private var mContext: Context? = null
    //透視矩陣秘蛔、相機(jī)矩陣定義放在基類中陨亡，方便傳給其他繪制對(duì)象
    private val mMVPMatrix = FloatArray(16)
    private val mTempMatrix = FloatArray(16)
    protected var mProjectMatrix = FloatArray(16)
    protected var mCameraMatrix = FloatArray(16)
    private var mProgram = 0

    // 原來(lái)的方向不對(duì)

    private val mPosCoordinate = floatArrayOf(
            -1f, -1f,1f,
            -1f, 1f,1f,
            1f, -1f,1f,
            1f, 1f,1f)

    private val mTexCoordinate = floatArrayOf(0f, 0f, 0f, 1f, 1f, 0f, 1f, 1f)

    private var mPosBuffer: FloatBuffer? = null
    private var mTexBuffer: FloatBuffer? = null

    lateinit var mPlayer: MediaPlayer
    //!!! 此路徑需根據(jù)自己情況，改為自己手機(jī)里的視頻路徑
    private val videoUrl:String = "/storage/emulated/0/Android/data/aom.example.dj.appgl/files/big_buck_bunny.mp4"
//    private val videoUrl:String = "http://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4"
    private var textureId = 0
    private lateinit var surfaceTexture:SurfaceTexture
    private var listener: SurfaceTexture.OnFrameAvailableListener? = null

    private var uPosHandle = 0
    private var aTexHandle = 0
    private var mMVPMatrixHandle = 0
    private var mTexRotateMatrixHandle = 0
    // 旋轉(zhuǎn)矩陣
    private val rotateOriMatrix = FloatArray(16)

    init {
        this.mContext = ctx
        Matrix.setIdentityM(mProjectMatrix, 0)
        Matrix.setIdentityM(mCameraMatrix, 0)
        Matrix.setIdentityM(mMVPMatrix, 0)
        Matrix.setIdentityM(mTempMatrix, 0)
        mPlayer = MediaPlayer()
        mPosBuffer = GLDataUtil.createFloatBuffer(mPosCoordinate)
        mTexBuffer = GLDataUtil.createFloatBuffer(mTexCoordinate)
    }

    fun initMediaPlayer(){
        mPlayer.reset()
        mPlayer.setDataSource(videoUrl)


        val texture = IntArray(1)
        GLES30.glGenTextures(1, texture, 0) //生成一個(gè)OpenGl紋理
        textureId = texture[0]
        GLES30.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, texture[0]) //申請(qǐng)紋理存儲(chǔ)區(qū)域并設(shè)置相關(guān)參數(shù)
        GLES30.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_LINEAR.toFloat())
        GLES30.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR.toFloat())
        GLES30.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE)
        GLES30.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_WRAP_T, GL10.GL_CLAMP_TO_EDGE)
        GLES30.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES,0)
        surfaceTexture = SurfaceTexture(textureId)
        surfaceTexture.setOnFrameAvailableListener(listener)


        val surface = Surface(surfaceTexture)
        mPlayer.setSurface(surface)
        surface.release()
        try {
            mPlayer.prepare()
            mPlayer.start()
        }catch (e:IOException){
            Log.e(Companion.TAG, "initMediaPlayer: $e")
        }
    }

    override fun onSurfaceCreated(gl: GL10?, config: EGLConfig?) {
        //編譯頂點(diǎn)著色程序
        val vertexShaderStr = ResReadUtils.readResource(R.raw.vertex_media_player_shade)
        val vertexShaderId = ShaderUtils.compileVertexShader(vertexShaderStr)
        //編譯片段著色程序
        // fragment_media_player_normal_shade  --正常
        // fragment_media_player_nostalgia_shade  -- 懷舊濾鏡
        // fragment_media_player_negative_shade  -- 負(fù)面濾鏡
        val fragmentShaderStr = ResReadUtils.readResource(R.raw.fragment_media_player_normal_shade)
        val fragmentShaderId = ShaderUtils.compileFragmentShader(fragmentShaderStr)
        //連接程序
        mProgram = ShaderUtils.linkProgram(vertexShaderId, fragmentShaderId)
        initMediaPlayer()
    }

    override fun onSurfaceChanged(gl: GL10?, width: Int, height: Int) {

    }

    override fun onDrawFrame(gl: GL10?) {
        GLES30.glEnable(GLES20.GL_DEPTH_TEST)
        GLES30.glClear(GLES20.GL_COLOR_BUFFER_BIT or GLES20.GL_DEPTH_BUFFER_BIT)
        GLES30.glUseProgram(mProgram)
        uPosHandle = GLES20.glGetAttribLocation(mProgram, "position")
        aTexHandle = GLES20.glGetAttribLocation(mProgram, "inputTextureCoordinate")
        mMVPMatrixHandle = GLES20.glGetUniformLocation(mProgram, "textureTransform")
        mTexRotateMatrixHandle = GLES20.glGetUniformLocation(mProgram,"uTextRotateMatrix")
        // 將前面計(jì)算得到的mMVPMatrix(frustumM setLookAtM 通過multiplyMM 相乘得到的矩陣) 傳入vMatrix中深员，與頂點(diǎn)矩陣進(jìn)行相乘
        GLES30.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, mMVPMatrix, 0)
        surfaceTexture!!.updateTexImage()

        //?? 為何要取矩陣负蠕？？!!!
        surfaceTexture.getTransformMatrix(rotateOriMatrix)

        GLES30.glVertexAttribPointer(uPosHandle, 3, GLES30.GL_FLOAT, false, 0, mPosBuffer)
        GLES30.glVertexAttribPointer(aTexHandle, 2, GLES30.GL_FLOAT, false, 8, mTexBuffer)
        GLES30.glUniformMatrix4fv(mTexRotateMatrixHandle, 1, false, rotateOriMatrix, 0)

        GLES30.glEnableVertexAttribArray(uPosHandle)
        GLES30.glEnableVertexAttribArray(aTexHandle)
        //頂點(diǎn)個(gè)數(shù)是4個(gè) mPosCoordinate.length/2每個(gè)定點(diǎn)x倦畅、y2個(gè)坐標(biāo)遮糖，所以得到頂點(diǎn)個(gè)數(shù)。
        GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, mPosCoordinate.size / 2)
        GLES30.glDisableVertexAttribArray(uPosHandle)
        GLES30.glDisableVertexAttribArray(aTexHandle)
        GLES30.glUseProgram(0)
    }

    companion object {
        private const val TAG = "MediaGLRenderer"
    }


}

本文主要闡述OpenGL ES + MediaPlayer播放原理叠赐，所以代碼做了簡(jiǎn)化欲账，也沒做播放控制。
需要注意的是MediaPlayer的輸出往往不是RGB格式（一般是YUV）芭概，而GLSurfaceView需要RGB格式才能正常顯示赛不。所以生成紋理代碼如下：

        val texture = IntArray(1)
        GLES30.glGenTextures(1, texture, 0) //生成一個(gè)OpenGl紋理
        textureId = texture[0]
        GLES30.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, texture[0]) //申請(qǐng)紋理存儲(chǔ)區(qū)域并設(shè)置相關(guān)參數(shù)
        GLES30.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_LINEAR.toFloat())
        GLES30.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR.toFloat())
        GLES30.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE)
        GLES30.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_WRAP_T, GL10.GL_CLAMP_TO_EDGE)

GLES11Ext.GL_TEXTURE_EXTERNAL_OES的用處:視頻解碼的輸出格式是YUV，這個(gè)擴(kuò)展紋理的作用就是實(shí)現(xiàn)YUV格式到RGB的自動(dòng)轉(zhuǎn)化谈山。

設(shè)置矩陣
surfaceTexture.getTransformMatrix(rotateOriMatrix)
這句代碼調(diào)試時(shí)得到的矩陣為：

image.png

頂點(diǎn)著色器代碼

uniform mat4 textureTransform;
attribute vec4 inputTextureCoordinate;
attribute vec3 position;            //NDK坐標(biāo)點(diǎn)
varying   vec2 textureCoordinate; //紋理坐標(biāo)點(diǎn)變換后輸出
uniform mat4 uTextRotateMatrix;

 void main() {
     vec4 pos = vec4(position, 1.0);
     gl_Position = pos.xyww;
     textureCoordinate = (uTextRotateMatrix * inputTextureCoordinate).xy;
 }

片元著色器代碼

#extension GL_OES_EGL_image_external : require
precision mediump float;
uniform samplerExternalOES videoTex;
varying vec2 textureCoordinate;

void main() {
    vec4 tc = texture2D(videoTex, textureCoordinate);
    gl_FragColor = tc;
}

samplerExternalOES：Android端用于繪制視頻或者相機(jī)預(yù)覽臊诊，使用samplerExternalOES需要在shader頭部添加

 #extension GL_OES_EGL_image_external : require

代碼：
https://github.com/godtrace12/DOpenglTest

參考：
https://zhuanlan.zhihu.com/p/164722231
https://blog.csdn.net/King1425/article/details/72773331?spm=1001.2014.3001.5501
https://juejin.cn/post/6903718814467883021#heading-9
https://www.cnblogs.com/hrlnw/p/3277300.html