說到濾鏡問題,市面上所有美顏類的相機(jī)都存在各式各樣的濾鏡。那么我們?cè)趺磳?shí)現(xiàn)濾鏡呢弧腥?我們首先想到,是否有相關(guān)開源項(xiàng)目可以參考的憔杨。iOS 下有比較著名的GPUImage是用來(lái)做濾鏡渲染的鸟赫,Android下面也有類似的項(xiàng)目。其中,美顏類開源相機(jī)比較出名的是程序員杠把子(CSDN博客:http://my.csdn.net/oShunz)的MagicCamera(github地址:https://github.com/wuhaoyu1990/MagicCamera)了抛蚤。但是我發(fā)現(xiàn)MagicCamera的幀率實(shí)在是太低了台谢,尤其是在使用了濾鏡之后,在紅米Note2上岁经,發(fā)現(xiàn)跑起來(lái)的幀率只有13幀朋沮,這還是只做了磨皮和濾鏡的情況下,如下圖的log所示:
相比市面上的美顏類相機(jī)來(lái)說缀壤,這樣的幀率太低了樊拓。因?yàn)閷?duì)商業(yè)相機(jī)來(lái)說,不僅僅需要做磨皮塘慕,還要做人臉關(guān)鍵點(diǎn)檢測(cè)筋夏,檢測(cè)完成后需要做瘦臉大眼等比較耗時(shí)的渲染處理,這么低的幀率肯定是不能滿足需要的图呢。為此条篷,本人開發(fā)了自己的相機(jī):CainCamera(github地址:CainCamera)。在開發(fā)CainCamera相機(jī)過程中蛤织,在寫濾鏡的過程中赴叹,很大程度上參考了MagicCamera,對(duì)此指蚜,表示非常感謝程序員杠把子所做的努力和提供了這么棒的開源相機(jī)和濾鏡乞巧。
廢話不多說,下面開始進(jìn)入正題摊鸡。本人開發(fā)的相機(jī)幀率究竟如何呢绽媒?同樣是做了磨皮和濾鏡渲染,CainCamera在紅米Note2上做和MagicCamera同樣的磨皮和濾鏡時(shí)柱宦,實(shí)時(shí)預(yù)覽的幀率穩(wěn)定在19幀以上些椒,如下圖所示:
關(guān)于市面上主流的美顏類相機(jī)的幀率表現(xiàn)播瞳,可以參考本人的文章:Android預(yù)覽實(shí)時(shí)渲染的幀率優(yōu)化相關(guān)掸刊,里面有跟市面上主流美顏類相機(jī)做過比較
首先是所有濾鏡的基類構(gòu)建問題∮遥基類寫得好忧侧,不僅可以節(jié)省具體濾鏡的代碼量,甚至能夠提高渲染效率牌芋。下面來(lái)看看本人寫的濾鏡基類是如何構(gòu)成的:
CainCamera的濾鏡基類由BaseImageFilter 和 BaseImageFilterGroup構(gòu)成的蚓炬。BaseImageFilter是所有濾鏡和濾鏡組的基類,BaseImageFilterGroup則是所有濾鏡組的基類躺屁,繼承于BaseImageFilter肯夏。濾鏡組是用于管理渲染多個(gè)濾鏡的渲染。下面我們來(lái)看看兩個(gè)基類的寫法:
首先是BaseImageFilter, BaseImageFilter的核心方法如下:
/**
* 繪制Frame
* @param textureId
*/
public boolean drawFrame(int textureId) {
return drawFrame(textureId, mVertexArray, mTexCoordArray);
}
/**
* 繪制Frame
* @param textureId
* @param vertexBuffer
* @param textureBuffer
*/
public boolean drawFrame(int textureId, FloatBuffer vertexBuffer,
FloatBuffer textureBuffer) {
if (textureId == GlUtil.GL_NOT_INIT) {
return false;
}
GLES30.glUseProgram(mProgramHandle);
runPendingOnDrawTasks();
vertexBuffer.position(0);
GLES30.glVertexAttribPointer(maPositionLoc, mCoordsPerVertex,
GLES30.GL_FLOAT, false, 0, vertexBuffer);
GLES30.glEnableVertexAttribArray(maPositionLoc);
textureBuffer.position(0);
GLES30.glVertexAttribPointer(maTextureCoordLoc, 2,
GLES30.GL_FLOAT, false, 0, textureBuffer);
GLES30.glEnableVertexAttribArray(maTextureCoordLoc);
GLES30.glUniformMatrix4fv(muMVPMatrixLoc, 1, false, mMVPMatrix, 0);
GLES30.glUniformMatrix4fv(mTexMatrixLoc, 1, false, mTexMatrix, 0);
GLES30.glActiveTexture(GLES30.GL_TEXTURE0);
GLES30.glBindTexture(getTextureType(), textureId);
GLES30.glUniform1i(mInputTextureLoc, 0);
onDrawArraysBegin();
GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, mVertexCount);
onDrawArraysAfter();
GLES30.glDisableVertexAttribArray(maPositionLoc);
GLES30.glDisableVertexAttribArray(maTextureCoordLoc);
GLES30.glBindTexture(getTextureType(), 0);
GLES30.glUseProgram(0);
return true;
}
/**
* 繪制到FBO
* @param textureId
* @return FBO綁定的Texture
*/
public int drawFrameBuffer(int textureId) {
return drawFrameBuffer(textureId, mVertexArray, mTexCoordArray);
}
/**
* 繪制到FBO
* @param textureId
* @param vertexBuffer
* @param textureBuffer
* @return FBO綁定的Texture
*/
public int drawFrameBuffer(int textureId, FloatBuffer vertexBuffer, FloatBuffer textureBuffer) {
if (mFramebuffers == null) {
return GlUtil.GL_NOT_INIT;
}
runPendingOnDrawTasks();
GLES30.glViewport(0, 0, mFrameWidth, mFrameHeight);
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, mFramebuffers[0]);
GLES30.glUseProgram(mProgramHandle);
vertexBuffer.position(0);
GLES30.glVertexAttribPointer(maPositionLoc, mCoordsPerVertex,
GLES30.GL_FLOAT, false, 0, vertexBuffer);
GLES30.glEnableVertexAttribArray(maPositionLoc);
textureBuffer.position(0);
GLES30.glVertexAttribPointer(maTextureCoordLoc, 2,
GLES30.GL_FLOAT, false, 0, textureBuffer);
GLES30.glEnableVertexAttribArray(maTextureCoordLoc);
GLES30.glUniformMatrix4fv(muMVPMatrixLoc, 1, false, mMVPMatrix, 0);
GLES30.glActiveTexture(GLES30.GL_TEXTURE0);
GLES30.glBindTexture(getTextureType(), textureId);
GLES30.glUniform1i(mInputTextureLoc, 0);
onDrawArraysBegin();
GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, mVertexCount);
onDrawArraysAfter();
GLES30.glDisableVertexAttribArray(maPositionLoc);
GLES30.glDisableVertexAttribArray(maTextureCoordLoc);
GLES30.glBindTexture(getTextureType(), 0);
GLES30.glUseProgram(0);
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, 0);
GLES30.glViewport(0, 0, mDisplayWidth, mDisplayHeight);
return mFramebufferTextures[0];
}
基類的核心方法就兩個(gè):drawFrame 和 drawFrameBuffer驯击。drawFrame方法直接繪制渲染輸入烁兰。drawFrameBuffer方法,則將texture渲染到FBO中徊都,該方法用于多重濾鏡的渲染上沪斟。2017年12月08日更新: 這里的drawFrame和drawFrameBuffer方法都做了更新,因?yàn)楸救藢?shí)現(xiàn)了多段視頻錄制的功能之后暇矫,發(fā)現(xiàn)這里存在一個(gè)小Bug主之,就是處于錄制視頻狀態(tài)下,切換濾鏡會(huì)產(chǎn)生一幀的黑屏現(xiàn)象李根,由于本人使用OpenGLES 的 multi thread multi rendering context做錄制操作槽奕,所以在這里存在一個(gè)線程同步的問題,多線程環(huán)境下切換濾鏡房轿,需要考慮FBO是否正確綁定texture了史翘。不過這里也很好改動(dòng),如果沒有綁定冀续,則返回一個(gè)標(biāo)志琼讽,讓FBO綁定跳過就好,因?yàn)榇藭r(shí)并沒有當(dāng)前濾鏡層的渲染操作洪唐,否則會(huì)將FBO綁定到一個(gè)空的Texture 中钻蹬,導(dǎo)致錄制時(shí)切換濾鏡瞬間存在黑屏的現(xiàn)象,該問題已修復(fù)凭需。
接下來(lái)我們來(lái)看看BaseImageFilterGroup的核心:
@Override
public void onInputSizeChanged(int width, int height) {
super.onInputSizeChanged(width, height);
if (mFilters.size() <= 0) {
return;
}
int size = mFilters.size();
for (int i = 0; i < size; i++) {
mFilters.get(i).onInputSizeChanged(width, height);
}
// 先銷毀原來(lái)的Framebuffers
if(mFramebuffers != null && (mImageWidth != width
|| mImageHeight != height || mFramebuffers.length != size-1)) {
destroyFramebuffer();
mImageWidth = width;
mImageWidth = height;
}
initFramebuffer(width, height);
}
@Override
public void onDisplayChanged(int width, int height) {
super.onDisplayChanged(width, height);
// 更新顯示的的視圖大小
if (mFilters.size() <= 0) {
return;
}
int size = mFilters.size();
for (int i = 0; i < size; i++) {
mFilters.get(i).onDisplayChanged(width, height);
}
}
@Override
public boolean drawFrame(int textureId) {
if (mFramebuffers == null || mFrameBufferTextures == null || mFilters.size() <= 0) {
return false;
}
int size = mFilters.size();
mCurrentTextureId = textureId;
for (int i = 0; i < size; i++) {
BaseImageFilter filter = mFilters.get(i);
if (i < size - 1) {
GLES30.glViewport(0, 0, mImageWidth, mImageHeight);
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, mFramebuffers[i]);
GLES30.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
if (filter.drawFrame(mCurrentTextureId)) {
mCurrentTextureId = mFrameBufferTextures[i];
}
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, 0);
} else {
GLES30.glViewport(0, 0, mDisplayWidth, mDisplayHeight);
filter.drawFrame(mCurrentTextureId);
}
}
return true;
}
@Override
public boolean drawFrame(int textureId, FloatBuffer vertexBuffer, FloatBuffer textureBuffer) {
if (mFramebuffers == null || mFrameBufferTextures == null || mFilters.size() <= 0) {
return false;
}
int size = mFilters.size();
mCurrentTextureId = textureId;
for (int i = 0; i < size; i++) {
BaseImageFilter filter = mFilters.get(i);
if (i < size - 1) {
GLES30.glViewport(0, 0, mImageWidth, mImageHeight);
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, mFramebuffers[i]);
GLES30.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
filter.drawFrame(mCurrentTextureId, vertexBuffer, textureBuffer);
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, 0);
mCurrentTextureId = mFrameBufferTextures[i];
} else {
GLES30.glViewport(0, 0, mDisplayWidth, mDisplayHeight);
filter.drawFrame(mCurrentTextureId, vertexBuffer, textureBuffer);
}
}
return true;
}
@Override
public int drawFrameBuffer(int textureId) {
if (mFramebuffers == null || mFrameBufferTextures == null || mFilters.size() <= 0) {
return textureId;
}
int size = mFilters.size();
mCurrentTextureId = textureId;
GLES30.glViewport(0, 0, mImageWidth, mImageHeight);
for (int i = 0; i < size; i++) {
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, mFramebuffers[i]);
GLES30.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
if (mFilters.get(i).drawFrame(mCurrentTextureId)) {
mCurrentTextureId = mFrameBufferTextures[i];
}
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, 0);
}
return mCurrentTextureId;
}
@Override
public int drawFrameBuffer(int textureId, FloatBuffer vertexBuffer, FloatBuffer textureBuffer) {
if (mFramebuffers == null || mFrameBufferTextures == null || mFilters.size() <= 0) {
return textureId;
}
int size = mFilters.size();
mCurrentTextureId = textureId;
GLES30.glViewport(0, 0, mImageWidth, mImageHeight);
for (int i = 0; i < size; i++) {
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, mFramebuffers[i]);
GLES30.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
if (mFilters.get(i).drawFrame(mCurrentTextureId, vertexBuffer, textureBuffer)) {
mCurrentTextureId = mFrameBufferTextures[i];
}
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, 0);
}
return mCurrentTextureId;
}
@Override
public void release() {
if (mFilters != null) {
for (BaseImageFilter filter : mFilters) {
filter.release();
}
mFilters.clear();
}
destroyFramebuffer();
}
/**
* 初始化framebuffer问欠,這里在調(diào)用drawFrame時(shí),會(huì)多一個(gè)FBO粒蜈,這里為了方便后面錄制視頻縮放處理
*/
public void initFramebuffer(int width, int height) {
int size = mFilters.size();
// 創(chuàng)建Framebuffers 和 Textures
if (mFramebuffers == null) {
mFramebuffers = new int[size];
mFrameBufferTextures = new int[size];
createFramebuffer(0, size);
}
}
/**
* 創(chuàng)建Framebuffer
* @param start
* @param size
*/
private void createFramebuffer(int start, int size) {
for (int i = start; i < size; i++) {
GLES30.glGenFramebuffers(1, mFramebuffers, i);
GLES30.glGenTextures(1, mFrameBufferTextures, i);
GLES30.glBindTexture(GLES30.GL_TEXTURE_2D, mFrameBufferTextures[i]);
GLES30.glTexImage2D(GLES30.GL_TEXTURE_2D, 0, GLES30.GL_RGBA,
mImageWidth, mImageHeight, 0, GLES30.GL_RGBA, GLES30.GL_UNSIGNED_BYTE, null);
GLES30.glTexParameterf(GLES30.GL_TEXTURE_2D,
GLES30.GL_TEXTURE_MAG_FILTER, GLES30.GL_LINEAR);
GLES30.glTexParameterf(GLES30.GL_TEXTURE_2D,
GLES30.GL_TEXTURE_MIN_FILTER, GLES30.GL_LINEAR);
GLES30.glTexParameterf(GLES30.GL_TEXTURE_2D,
GLES30.GL_TEXTURE_WRAP_S, GLES30.GL_CLAMP_TO_EDGE);
GLES30.glTexParameterf(GLES30.GL_TEXTURE_2D,
GLES30.GL_TEXTURE_WRAP_T, GLES30.GL_CLAMP_TO_EDGE);
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, mFramebuffers[i]);
GLES30.glFramebufferTexture2D(GLES30.GL_FRAMEBUFFER, GLES30.GL_COLOR_ATTACHMENT0,
GLES30.GL_TEXTURE_2D, mFrameBufferTextures[i], 0);
GLES30.glBindTexture(GLES30.GL_TEXTURE_2D, 0);
GLES30.glBindFramebuffer(GLES30.GL_FRAMEBUFFER, 0);
}
}
/**
* 銷毀Framebuffers
*/
public void destroyFramebuffer() {
if (mFrameBufferTextures != null) {
GLES30.glDeleteTextures(mFrameBufferTextures.length, mFrameBufferTextures, 0);
mFrameBufferTextures = null;
}
if (mFramebuffers != null) {
GLES30.glDeleteFramebuffers(mFramebuffers.length, mFramebuffers, 0);
mFramebuffers = null;
}
}
BaseImageFilterGroup 主要用于管理FBO的創(chuàng)建和銷毀顺献。其中,onInputSizeChanged 方法主要用于相機(jī)預(yù)覽大小(PreviewSize)改變時(shí)調(diào)用枯怖,onDisplayChanged 方法則用于SurfaceView大小發(fā)生改變時(shí)調(diào)用注整,initFramebuffer 方法用于為每個(gè)濾鏡創(chuàng)建一個(gè)FBO,destroyFramebuffer 方法則用于銷毀FBO度硝。drawFrame 方法繼承于基類BaseImageFilter肿轨,需要重寫該方法,逐個(gè)濾鏡綁定FBO并繪制蕊程。
這樣椒袍,所有濾鏡和濾鏡組寫起來(lái)就方便很多了。我們來(lái)看下其中一個(gè)濾鏡和濾鏡組的寫法藻茂,
比如相機(jī)輸入流繪制可以這么寫:
public class CameraFilter extends BaseImageFilter {
private static final String VERTEX_SHADER =
"uniform mat4 uMVPMatrix; \n" +
"uniform mat4 uTexMatrix; \n" +
"attribute vec4 aPosition; \n" +
"attribute vec4 aTextureCoord; \n" +
"varying vec2 textureCoordinate; \n" +
"void main() { \n" +
" gl_Position = uMVPMatrix * aPosition; \n" +
" textureCoordinate = (uTexMatrix * aTextureCoord).xy; \n" +
"} \n";
private static final String FRAGMENT_SHADER_OES =
"#extension GL_OES_EGL_image_external : require \n" +
"precision mediump float; \n" +
"varying vec2 textureCoordinate; \n" +
"uniform samplerExternalOES inputTexture; \n" +
"void main() { \n" +
" gl_FragColor = texture2D(inputTexture, textureCoordinate); \n" +
"} \n";
private int muTexMatrixLoc;
private float[] mTextureMatrix;
public CameraFilter() {
this(VERTEX_SHADER, FRAGMENT_SHADER_OES);
}
public CameraFilter(String vertexShader, String fragmentShader) {
super(vertexShader, fragmentShader);
muTexMatrixLoc = GLES30.glGetUniformLocation(mProgramHandle, "uTexMatrix");
// 視圖矩陣
Matrix.setLookAtM(mViewMatrix, 0, 0, 0, -1, 0f, 0f, 0f, 0f, 1f, 0f);
}
@Override
public void onInputSizeChanged(int width, int height) {
super.onInputSizeChanged(width, height);
float aspect = (float) width / height; // 計(jì)算寬高比
Matrix.perspectiveM(mProjectionMatrix, 0, 60, aspect, 2, 10);
}
@Override
public int getTextureType() {
return GLES11Ext.GL_TEXTURE_EXTERNAL_OES;
}
@Override
public void onDrawArraysBegin() {
GLES30.glUniformMatrix4fv(muTexMatrixLoc, 1, false, mTextureMatrix, 0);
}
public void updateTextureBuffer() {
mTexCoordArray = TextureRotationUtils.getTextureBuffer();
}
/**
* 設(shè)置SurfaceTexture的變換矩陣
* @param texMatrix
*/
public void setTextureTransformMatirx(float[] texMatrix) {
mTextureMatrix = texMatrix;
}
/**
* 鏡像翻轉(zhuǎn)
* @param coords
* @param matrix
* @return
*/
private float[] transformTextureCoordinates(float[] coords, float[] matrix) {
float[] result = new float[coords.length];
float[] vt = new float[4];
for (int i = 0; i < coords.length; i += 2) {
float[] v = { coords[i], coords[i + 1], 0, 1 };
Matrix.multiplyMV(vt, 0, matrix, 0, v, 0);
result[i] = vt[0];// x軸鏡像
// result[i + 1] = vt[1];y軸鏡像
result[i + 1] = coords[i + 1];
}
return result;
}
}
而濾鏡組則更加簡(jiǎn)單:
public class DefaultFilterGroup extends BaseImageFilterGroup {
// 實(shí)時(shí)美顏層
private static final int BeautyfyIndex = 0;
// 顏色層
private static final int ColorIndex = 1;
// 瘦臉大眼層
private static final int FaceStretchIndex = 2;
// 貼紙
private static final int StickersIndex = 3;
public DefaultFilterGroup() {
this(initFilters());
}
private DefaultFilterGroup(List<BaseImageFilter> filters) {
mFilters = filters;
}
private static List<BaseImageFilter> initFilters() {
List<BaseImageFilter> filters = new ArrayList<BaseImageFilter>();
filters.add(BeautyfyIndex, FilterManager.getFilter(FilterType.REALTIMEBEAUTY));
filters.add(ColorIndex, FilterManager.getFilter(FilterType.SOURCE));
filters.add(FaceStretchIndex, FilterManager.getFilter(FilterType.FACESTRETCH));
filters.add(StickersIndex, FilterManager.getFilter(FilterType.STICKER));
return filters;
}
@Override
public void changeFilter(FilterType type) {
FilterIndex index = FilterManager.getIndex(type);
if (index == FilterIndex.BeautyIndex) {
changeBeautyFilter(type);
} else if (index == FilterIndex.ColorIndex) {
changeColorFilter(type);
} else if (index == FilterIndex.FaceStretchIndex) {
changeFaceStretchFilter(type);
} else if (index == FilterIndex.MakeUpIndex) {
changeMakeupFilter(type);
} else if (index == FilterIndex.StickerIndex) {
changeStickerFilter(type);
}
}
/**
* 切換美顏濾鏡
* @param type
*/
private void changeBeautyFilter(FilterType type) {
if (mFilters != null) {
mFilters.get(BeautyfyIndex).release();
mFilters.set(BeautyfyIndex, FilterManager.getFilter(type));
// 設(shè)置寬高
mFilters.get(BeautyfyIndex).onInputSizeChanged(mImageWidth, mImageHeight);
mFilters.get(BeautyfyIndex).onDisplayChanged(mDisplayWidth, mDisplayHeight);
}
}
/**
* 切換顏色濾鏡
* @param type
*/
private void changeColorFilter(FilterType type) {
if (mFilters != null) {
mFilters.get(ColorIndex).release();
mFilters.set(ColorIndex, FilterManager.getFilter(type));
// 設(shè)置寬高
mFilters.get(ColorIndex).onInputSizeChanged(mImageWidth, mImageHeight);
mFilters.get(ColorIndex).onDisplayChanged(mDisplayWidth, mDisplayHeight);
}
}
/**
* 切換瘦臉大眼濾鏡
* @param type
*/
private void changeFaceStretchFilter(FilterType type) {
if (mFilters != null) {
mFilters.get(FaceStretchIndex).release();
mFilters.set(FaceStretchIndex, FilterManager.getFilter(type));
// 設(shè)置寬高
mFilters.get(FaceStretchIndex).onInputSizeChanged(mImageWidth, mImageHeight);
mFilters.get(FaceStretchIndex).onDisplayChanged(mDisplayWidth, mDisplayHeight);
}
}
/**
* 切換貼紙濾鏡
* @param type
*/
private void changeStickerFilter(FilterType type) {
if (mFilters != null) {
mFilters.get(StickersIndex).release();
mFilters.set(StickersIndex, FilterManager.getFilter(type));
// 設(shè)置寬高
mFilters.get(StickersIndex).onInputSizeChanged(mImageWidth, mImageHeight);
mFilters.get(StickersIndex).onDisplayChanged(mDisplayWidth, mDisplayHeight);
}
}
/**
* 切換彩妝濾鏡
* @param type
*/
private void changeMakeupFilter(FilterType type) {
// Do nothing, 彩妝濾鏡放在彩妝濾鏡組里面
}
}
上面是默認(rèn)的實(shí)時(shí)渲染濾鏡組驹暑,可以看到玫恳,本人在處理多個(gè)濾鏡的時(shí)候,使用了分層的形式优俘。這么寫的原因很簡(jiǎn)單纽窟,為了節(jié)省渲染前的計(jì)算時(shí)間。一般實(shí)時(shí)性不夠強(qiáng)的情況下兼吓,通常都是在調(diào)用某個(gè)方法的時(shí)候再逐個(gè)計(jì)算有多少個(gè)濾鏡臂港,然后在繪制的時(shí)候再動(dòng)態(tài)綁定FBO。但這么做對(duì)實(shí)時(shí)性要求非常高的情況下并不是非常好的方式视搏。因?yàn)槊看握{(diào)用drawFrame/drawFrameBuffer繪制的時(shí)候都需要綁定和解綁FBO审孽,效率肯定會(huì)有所影響。但實(shí)際上浑娜,我們可以在繪制之前就已經(jīng)知道需要多少個(gè)濾鏡佑力,需要多少個(gè)FBO了,繪制方法因?yàn)檎{(diào)用的次數(shù)非常非常多筋遭,所以任何耗時(shí)操作如非必須打颤,都不能放在繪制方法里面,新建對(duì)象就更加不建議了漓滔,不僅新建對(duì)象比較耗時(shí)编饺,而且可能會(huì)產(chǎn)生內(nèi)存抖動(dòng)問題。
現(xiàn)在我們得到了濾鏡和濾鏡組的基類响驴,也得到了默認(rèn)實(shí)時(shí)渲染的濾鏡組透且,接下來(lái)就是實(shí)現(xiàn)具體濾鏡了。具體的濾鏡實(shí)現(xiàn)就是glsl的事情了豁鲤。如果濾鏡有多個(gè)Texture如何處理呢秽誊?沒關(guān)系,我們?cè)贐aseImageFilter的drawFrame 方法中加入了一個(gè)空方法onDrawArraysBegin()琳骡,當(dāng)濾鏡需要綁定除了inputTexture外的其他Texture锅论,則可以在這里進(jìn)行綁定,比如像LOMO濾鏡一樣楣号,綁定需要混合的Texture:
public class LomoFilter extends BaseImageFilter {
private static final String FRAGMENT_SHADER =
"precision mediump float;\n" +
" \n" +
" varying mediump vec2 textureCoordinate;\n" +
" \n" +
" uniform sampler2D inputTexture;\n" +
" uniform sampler2D mapTexture;\n" +
" uniform sampler2D vignetteTexture;\n" +
" \n" +
" uniform float strength;\n" +
"\n" +
" void main()\n" +
" {\n" +
" vec4 originColor = texture2D(inputTexture, textureCoordinate);\n" +
" vec3 texel = texture2D(inputTexture, textureCoordinate).rgb;\n" +
"\n" +
" vec2 red = vec2(texel.r, 0.16666);\n" +
" vec2 green = vec2(texel.g, 0.5);\n" +
" vec2 blue = vec2(texel.b, 0.83333);\n" +
"\n" +
" texel.rgb = vec3(\n" +
" texture2D(mapTexture, red).r,\n" +
" texture2D(mapTexture, green).g,\n" +
" texture2D(mapTexture, blue).b);\n" +
"\n" +
" vec2 tc = (2.0 * textureCoordinate) - 1.0;\n" +
" float d = dot(tc, tc);\n" +
" vec2 lookup = vec2(d, texel.r);\n" +
" texel.r = texture2D(vignetteTexture, lookup).r;\n" +
" lookup.y = texel.g;\n" +
" texel.g = texture2D(vignetteTexture, lookup).g;\n" +
" lookup.y = texel.b;\n" +
" texel.b\t= texture2D(vignetteTexture, lookup).b;\n" +
"\n" +
" texel.rgb = mix(originColor.rgb, texel.rgb, strength);\n" +
"\n" +
" gl_FragColor = vec4(texel,1.0);\n" +
" }";
private int mMapTexture;
private int mMapTextureLoc;
private int mVignetteTexture;
private int mVignetteTextureLoc;
private int mStrengthLoc;
public LomoFilter() {
this(VERTEX_SHADER, FRAGMENT_SHADER);
}
public LomoFilter(String vertexShader, String fragmentShader) {
super(vertexShader, fragmentShader);
mMapTextureLoc = GLES30.glGetUniformLocation(mProgramHandle, "mapTexture");
mVignetteTextureLoc = GLES30.glGetUniformLocation(mProgramHandle, "vignetteTexture");
mStrengthLoc = GLES30.glGetUniformLocation(mProgramHandle, "strength");
createTexture();
setFloat(mStrengthLoc, 1.0f);
}
private void createTexture() {
mMapTexture = GlUtil.createTextureFromAssets(ParamsManager.context,
"filters/lomo_map.png");
mVignetteTexture = GlUtil.createTextureFromAssets(ParamsManager.context,
"filters/lomo_vignette.png");
}
@Override
public void onDrawArraysBegin() {
super.onDrawArraysBegin();
GLES30.glActiveTexture(GLES30.GL_TEXTURE1);
GLES30.glBindTexture(getTextureType(), mMapTexture);
GLES30.glUniform1i(mMapTextureLoc, 1);
GLES30.glActiveTexture(GLES30.GL_TEXTURE2);
GLES30.glBindTexture(getTextureType(), mVignetteTexture);
GLES30.glUniform1i(mVignetteTextureLoc, 2);
}
@Override
public void release() {
super.release();
GLES30.glDeleteTextures(2, new int[]{mMapTexture, mVignetteTexture}, 0);
}
}
濾鏡之所以沒有持有Context最易,這樣做可以讓RenderThread也不需要持有相應(yīng)的Context,方便移植到不同的系統(tǒng)中竖席,如果有需要耘纱,也可以在基類添加Context上下文敬肚。這就看個(gè)人喜好了毕荐。
好了,濾鏡的實(shí)現(xiàn)我們也知道怎么做了艳馒。那么憎亚,像我這樣將濾鏡組分層又怎么做濾鏡切換呢员寇?為了方便,我寫了幾個(gè)管理類第美,用于管理濾鏡和不同濾鏡組的切換蝶锋,其中FilterIndex用于指定濾鏡層,F(xiàn)ilterType用于指定具體的濾鏡什往,F(xiàn)ilterManager用于管理切換濾鏡以及切換濾鏡組扳缕,ColorFilterManager則用于管理顏色濾鏡的切換,方便Activity調(diào)用别威。具體實(shí)現(xiàn)如下:
FilterIndex:
public enum FilterIndex {
// 無(wú)
NoneIndex,
// 美顏
BeautyIndex,
// 顏色
ColorIndex,
// 瘦臉大眼
FaceStretchIndex,
// 貼紙
StickerIndex,
// 彩妝
MakeUpIndex,
// 水印
WaterMaskIndex,
// 圖片編輯
ImageEditIndex
}
``
FilterType:
public enum FilterType {
NONE, // 沒有濾鏡
// 圖片編輯濾鏡
BRIGHTNESS, // 亮度
CONTRAST, // 對(duì)比度
EXPOSURE, // 曝光
GUASS, // 高斯模糊
HUE, // 色調(diào)
MIRROR, // 鏡像
SATURATION, // 飽和度
SHARPNESS, // 銳度
WATERMASK, // 水印
// 人臉美顏美妝貼紙
REALTIMEBEAUTY, // 實(shí)時(shí)美顏
FACESTRETCH, // 人臉變形(瘦臉大眼等)
STICKER, // 貼紙
MAKEUP, // 彩妝
// 顏色濾鏡
SOURCE, // 原圖
AMARO, // 阿馬羅
ANTIQUE, // 古董
BLACKCAT, // 黑貓
BLACKWHITE, // 黑白
BROOKLYN, // 布魯克林
CALM, // 冷靜
COOL, // 冷色調(diào)
EARLYBIRD, // 晨鳥
EMERALD, // 翡翠
EVERGREEN, // 常綠
FAIRYTALE, // 童話
FREUD, // 佛洛伊特
HEALTHY, // 健康
HEFE, // 酵母
HUDSON, // 哈德森
KEVIN, // 凱文
LATTE, // 拿鐵
LOMO, // LOMO
NOSTALGIA, // 懷舊之情
ROMANCE, // 浪漫
SAKURA, // 櫻花
SKETCH, // 素描
SUNSET, // 日落
WHITECAT, // 白貓
WHITENORREDDEN, // 白皙還是紅潤(rùn)
}
FilterManager:
public final class FilterManager {
private static HashMap<FilterType, FilterIndex> mIndexMap = new HashMap<FilterType, FilterIndex>();
static {
mIndexMap.put(FilterType.NONE, FilterIndex.NoneIndex);
// 圖片編輯
mIndexMap.put(FilterType.BRIGHTNESS, FilterIndex.ImageEditIndex);
mIndexMap.put(FilterType.CONTRAST, FilterIndex.ImageEditIndex);
mIndexMap.put(FilterType.EXPOSURE, FilterIndex.ImageEditIndex);
mIndexMap.put(FilterType.GUASS, FilterIndex.ImageEditIndex);
mIndexMap.put(FilterType.HUE, FilterIndex.ImageEditIndex);
mIndexMap.put(FilterType.MIRROR, FilterIndex.ImageEditIndex);
mIndexMap.put(FilterType.SATURATION, FilterIndex.ImageEditIndex);
mIndexMap.put(FilterType.SHARPNESS, FilterIndex.ImageEditIndex);
// 水印
mIndexMap.put(FilterType.WATERMASK, FilterIndex.WaterMaskIndex);
// 美顏
mIndexMap.put(FilterType.REALTIMEBEAUTY, FilterIndex.BeautyIndex);
// 瘦臉大眼
mIndexMap.put(FilterType.FACESTRETCH, FilterIndex.FaceStretchIndex);
// 貼紙
mIndexMap.put(FilterType.STICKER, FilterIndex.StickerIndex);
// 彩妝
mIndexMap.put(FilterType.MAKEUP, FilterIndex.MakeUpIndex);
// 顏色濾鏡
mIndexMap.put(FilterType.AMARO, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.ANTIQUE, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.BLACKCAT, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.BLACKWHITE, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.BROOKLYN, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.CALM, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.COOL, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.EARLYBIRD, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.EMERALD, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.EVERGREEN, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.FAIRYTALE, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.FREUD, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.HEALTHY, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.HEFE, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.HUDSON, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.KEVIN, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.LATTE, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.LOMO, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.NOSTALGIA, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.ROMANCE, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.SAKURA, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.SKETCH, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.SOURCE, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.SUNSET, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.WHITECAT, FilterIndex.ColorIndex);
mIndexMap.put(FilterType.WHITENORREDDEN, FilterIndex.ColorIndex);
}
private FilterManager() {}
public static BaseImageFilter getFilter(FilterType type) {
switch (type) {
// 圖片基本屬性編輯濾鏡
// 飽和度
case SATURATION:
return new SaturationFilter();
// 鏡像翻轉(zhuǎn)
case MIRROR:
return new MirrorFilter();
// 高斯模糊
case GUASS:
return new GuassFilter();
// 亮度
case BRIGHTNESS:
return new BrightnessFilter();
// 對(duì)比度
case CONTRAST:
return new ContrastFilter();
// 曝光
case EXPOSURE:
return new ExposureFilter();
// 色調(diào)
case HUE:
return new HueFilter();
// 銳度
case SHARPNESS:
return new SharpnessFilter();
// TODO 貼紙濾鏡需要人臉關(guān)鍵點(diǎn)計(jì)算得到
case STICKER:
return new DisplayFilter();
// return new StickerFilter();
// 白皙還是紅潤(rùn)
case WHITENORREDDEN:
return new WhitenOrReddenFilter();
// 實(shí)時(shí)磨皮
case REALTIMEBEAUTY:
return new RealtimeBeautify();
// AMARO
case AMARO:
return new AmaroFilter();
// 古董
case ANTIQUE:
return new AnitqueFilter();
// 黑貓
case BLACKCAT:
return new BlackCatFilter();
// 黑白
case BLACKWHITE:
return new BlackWhiteFilter();
// 布魯克林
case BROOKLYN:
return new BrooklynFilter();
// 冷靜
case CALM:
return new CalmFilter();
// 冷色調(diào)
case COOL:
return new CoolFilter();
// 晨鳥
case EARLYBIRD:
return new EarlyBirdFilter();
// 翡翠
case EMERALD:
return new EmeraldFilter();
// 常綠
case EVERGREEN:
return new EvergreenFilter();
// 童話
case FAIRYTALE:
return new FairyTaleFilter();
// 佛洛伊特
case FREUD:
return new FreudFilter();
// 健康
case HEALTHY:
return new HealthyFilter();
// 酵母
case HEFE:
return new HefeFilter();
// 哈德森
case HUDSON:
return new HudsonFilter();
// 凱文
case KEVIN:
return new KevinFilter();
// 拿鐵
case LATTE:
return new LatteFilter();
// LOMO
case LOMO:
return new LomoFilter();
// 懷舊之情
case NOSTALGIA:
return new NostalgiaFilter();
// 浪漫
case ROMANCE:
return new RomanceFilter();
// 櫻花
case SAKURA:
return new SakuraFilter();
// 素描
case SKETCH:
return new SketchFilter();
// 日落
case SUNSET:
return new SunsetFilter();
// 白貓
case WHITECAT:
return new WhiteCatFilter();
case NONE: // 沒有濾鏡
case SOURCE: // 原圖
default:
return new DisplayFilter();
}
}
/**
* 獲取濾鏡組
* @return
*/
public static BaseImageFilterGroup getFilterGroup() {
return new DefaultFilterGroup();
}
public static BaseImageFilterGroup getFilterGroup(FilterGroupType type) {
switch (type) {
// 彩妝濾鏡組
case MAKEUP:
return new MakeUpFilterGroup();
// 默認(rèn)濾鏡組
case DEFAULT:
default:
return new DefaultFilterGroup();
}
}
/**
* 獲取層級(jí)
* @param Type
* @return
*/
public static FilterIndex getIndex(FilterType Type) {
FilterIndex index = mIndexMap.get(Type);
if (index != null) {
return index;
}
return FilterIndex.NoneIndex;
}
}
ColorFilterManager:
public final class ColorFilterManager {
private static ColorFilterManager mInstance;
private ArrayList<FilterType> mFilterType;
private ArrayList<String> mFilterName;
public static ColorFilterManager getInstance() {
if (mInstance == null) {
mInstance = new ColorFilterManager();
}
return mInstance;
}
private ColorFilterManager() {
initColorFilters();
}
/**
* 初始化顏色濾鏡
*/
public void initColorFilters() {
mFilterType = new ArrayList<FilterType>();
mFilterType.add(FilterType.SOURCE); // 原圖
mFilterType.add(FilterType.AMARO);
mFilterType.add(FilterType.ANTIQUE);
mFilterType.add(FilterType.BLACKCAT);
mFilterType.add(FilterType.BLACKWHITE);
mFilterType.add(FilterType.BROOKLYN);
mFilterType.add(FilterType.CALM);
mFilterType.add(FilterType.COOL);
mFilterType.add(FilterType.EARLYBIRD);
mFilterType.add(FilterType.EMERALD);
mFilterType.add(FilterType.EVERGREEN);
mFilterType.add(FilterType.FAIRYTALE);
mFilterType.add(FilterType.FREUD);
mFilterType.add(FilterType.HEALTHY);
mFilterType.add(FilterType.HEFE);
mFilterType.add(FilterType.HUDSON);
mFilterType.add(FilterType.KEVIN);
mFilterType.add(FilterType.LATTE);
mFilterType.add(FilterType.LOMO);
mFilterType.add(FilterType.NOSTALGIA);
mFilterType.add(FilterType.ROMANCE);
mFilterType.add(FilterType.SAKURA);
mFilterType.add(FilterType.SKETCH);
mFilterType.add(FilterType.SUNSET);
mFilterType.add(FilterType.WHITECAT);
mFilterName = new ArrayList<String>();
mFilterName.add("原圖");
mFilterName.add("阿馬羅");
mFilterName.add("古董");
mFilterName.add("黑貓");
mFilterName.add("黑白");
mFilterName.add("布魯克林");
mFilterName.add("冷靜");
mFilterName.add("冷色調(diào)");
mFilterName.add("晨鳥");
mFilterName.add("翡翠");
mFilterName.add("常綠");
mFilterName.add("童話");
mFilterName.add("佛洛伊特");
mFilterName.add("健康");
mFilterName.add("酵母");
mFilterName.add("哈德森");
mFilterName.add("凱文");
mFilterName.add("拿鐵");
mFilterName.add("LOMO");
mFilterName.add("懷舊之情");
mFilterName.add("浪漫");
mFilterName.add("櫻花");
mFilterName.add("素描");
mFilterName.add("日落");
mFilterName.add("白貓");
}
/**
* 獲取顏色濾鏡類型
* @param index
* @return
*/
public FilterType getColorFilterType(int index) {
if (mFilterType == null || mFilterType.isEmpty()) {
return FilterType.SOURCE;
}
int i = index % mFilterType.size();
return mFilterType.get(i);
}
/**
* 獲取顏色濾鏡的名稱
* @param index
* @return
*/
public String getColorFilterName(int index) {
if (mFilterName == null || mFilterName.isEmpty()) {
return "原圖";
}
int i = index % mFilterName.size();
return mFilterName.get(i);
}
/**
* 獲取顏色濾鏡數(shù)目
* @return
*/
public int getColorFilterCount() {
return mFilterType == null ? 0 : mFilterType.size();
}
}
這樣躯舔,在外層的Activity中,切換濾鏡只需要這么使用:
@Override
public void swipeBack() {
mColorIndex++;
if (mColorIndex >= ColorFilterManager.getInstance().getColorFilterCount()) {
mColorIndex = 0;
}
DrawerManager.getInstance()
.changeFilterType(ColorFilterManager.getInstance().getColorFilterType(mColorIndex));
if (isDebug) {
Log.d("changeFilter", "index = " + mColorIndex + ", filter name = "
+ ColorFilterManager.getInstance().getColorFilterName(mColorIndex));
}
}
@Override
public void swipeFrontal() {
mColorIndex--;
if (mColorIndex < 0) {
int count = ColorFilterManager.getInstance().getColorFilterCount();
mColorIndex = count > 0 ? count - 1 : 0;
}
DrawerManager.getInstance()
.changeFilterType(ColorFilterManager.getInstance().getColorFilterType(mColorIndex));
if (isDebug) {
Log.d("changeFilter", "index = " + mColorIndex + ", filter name = "
+ ColorFilterManager.getInstance().getColorFilterName(mColorIndex));
}
}
``
其中DrawerManager是用于管理相機(jī)預(yù)覽渲染線程的省古。這樣的方案可以使得我們?cè)诓僮黜?yè)面邏輯的時(shí)候就不需要考慮濾鏡切換的細(xì)節(jié)是如何處理的粥庄,頁(yè)面需要大改的時(shí)候,對(duì)濾鏡來(lái)說并沒有任何影響豺妓。
至此惜互,你就會(huì)得到一個(gè)耦合度較低、渲染效率較高琳拭、預(yù)覽幀率足夠的美顏類相機(jī)了训堆。
具體實(shí)現(xiàn)過程可以參考本人的項(xiàng)目CainCamera,github地址:CainCamera
備注:截止這篇文章發(fā)布時(shí)白嘁,本人還沒有實(shí)現(xiàn)貼紙功能蔫慧,但人臉關(guān)鍵點(diǎn)檢測(cè)已經(jīng)成功接入Face++ 的SDK,關(guān)鍵點(diǎn)已經(jīng)取得权薯,但貼紙濾鏡的具體實(shí)現(xiàn)還沒有實(shí)現(xiàn)姑躲。本人的實(shí)現(xiàn)思路如下:
貼紙功能分成JSON解析器、zip解壓器盟蚣、貼紙管理器等功能構(gòu)成黍析,用于解壓貼紙的zip包,解析json屎开,生成texture等流程阐枣,后續(xù)功能等實(shí)現(xiàn)了,本人將會(huì)發(fā)布新的文章介紹如何實(shí)現(xiàn)奄抽。當(dāng)然蔼两,這是在幀率得到保證、不產(chǎn)生內(nèi)存抖動(dòng)逞度、內(nèi)存溢出的前提下實(shí)現(xiàn)额划。本人對(duì)CainCamera項(xiàng)目的要求能夠達(dá)到商業(yè)相機(jī)的水平,直接可以商用的程度档泽。
關(guān)于相機(jī)控制和渲染具體實(shí)現(xiàn)俊戳,可以參考本人的文章:
Android預(yù)覽實(shí)時(shí)渲染的幀率優(yōu)化相關(guān)
關(guān)于磨皮算法的渲染效率優(yōu)化揖赴,可以參考本人的文章:
Android OpenGLES 實(shí)時(shí)美顏的優(yōu)化