前言

從Android4.0開始，Android源碼開始合入BlurFilter(模糊濾波)相關(guān)算法扼鞋，提供模糊（毛玻璃）效果支持申鱼，幫助上層應(yīng)用在使用窗口、視圖或圖片時(shí)更有層次感云头。圖像處理的模糊濾波算法有很多捐友，包括常見的均值模糊和高斯模糊，Android 12采用的模糊濾波為Kawase濾波算法盘寡。

卷積

提及圖像濾波一定需要講到卷積楚殿。在圖像處理的基礎(chǔ)原理中，卷積是圖像處理的基本操作竿痰。通過 圖像源矩陣 乘 卷積核 得到 結(jié)果矩陣 脆粥，這里的結(jié)果矩陣即單次圖像處理的結(jié)果。

卷積.png

均值/高斯/Kawase濾波器

不同的濾波器采用的卷積核各不相同影涉，以下列三種為例：

• 均值濾波器

  均值濾波器的卷積核通常是一個(gè)m*m的矩陣变隔，且每個(gè)元素為均等的1/（m*m)；

• 高斯濾波器

  高斯濾波器的卷積核中的元素遵循高斯分布蟹倾，卷積核中心的值相對(duì)邊緣較大匣缘，因此在做卷積處理時(shí)中心點(diǎn)的權(quán)重最大猖闪，權(quán)重逐步向邊緣遞減。

• Kawase濾波器

  Kawase濾波器采用的是取中心點(diǎn)向外的四個(gè)點(diǎn)進(jìn)行采樣肌厨，且隨著迭代次數(shù)增加向外擴(kuò)張更遠(yuǎn)的采樣距離

Kawase_1.png

Kawase_2.png

通過對(duì)模糊品質(zhì)培慌、穩(wěn)定性和性能三個(gè)標(biāo)準(zhǔn)的比較（引用十種模糊算法比較）

• 模糊品質(zhì)：模糊品質(zhì)的好壞是模糊算法是否優(yōu)秀的主要指標(biāo)
• 模糊穩(wěn)定性：模糊的穩(wěn)定性決定了在畫面變化過程中，模糊是否穩(wěn)定柑爸，不會(huì)出現(xiàn)跳變或者閃爍吵护。
• 性能：性能的好壞是模糊算法是否能被廣泛使用的關(guān)鍵所在。

十種模糊算法比較.jpg

RenderEngine

在圖像顯示系統(tǒng)中表鳍，需要使用特定的渲染引擎通過GPU繪制到暫存緩沖區(qū)（采用Client合成方式的圖層）馅而，因此在設(shè)備啟動(dòng)時(shí)會(huì)進(jìn)行渲染引擎的創(chuàng)建。

渲染引擎的選擇是根據(jù)renderEngineType來確定的譬圣。在Android 12 中默認(rèn)選擇SKIA_GL_THREADED瓮恭，Android 11 默認(rèn)選擇GLES。前者是采用異步的SKIA_GL厘熟，底層實(shí)現(xiàn)是在OpenGL基礎(chǔ)上做的封裝屯蹦， 后者是直接調(diào)用OpenGL接口并使用GLES語言進(jìn)行著色器編輯。

private:
    // 1 means RGBA_8888
    int pixelFormat = 1;
    uint32_t imageCacheSize = 0;
    bool useColorManagement = true;
    bool enableProtectedContext = false;
    bool precacheToneMapperShaderOnly = false;
    bool supportsBackgroundBlur = false;
    RenderEngine::ContextPriority contextPriority = RenderEngine::ContextPriority::MEDIUM;
    RenderEngine::RenderEngineType renderEngineType =
            RenderEngine::RenderEngineType::SKIA_GL_THREADED;
};

開發(fā)人員可根據(jù)設(shè)置DEBUG_RENDERENGINE_BACKEND參數(shù)為“gles/threaded/skiagl/skiaglthreaded"選擇渲染引擎盯漂。

[\framework\native\libs\renderengine\RenderEngine.cpp]
RenderEngineType renderEngineType = args.renderEngineType;

    // Keep the ability to override by PROPERTIES:
    char prop[PROPERTY_VALUE_MAX];
    property_get(PROPERTY_DEBUG_RENDERENGINE_BACKEND, prop, "");
    if (strcmp(prop, "gles") == 0) {
        renderEngineType = RenderEngineType::GLES;
    }
    if (strcmp(prop, "threaded") == 0) {
        renderEngineType = RenderEngineType::THREADED;
    }
    if (strcmp(prop, "skiagl") == 0) {
        renderEngineType = RenderEngineType::SKIA_GL;
    }
    if (strcmp(prop, "skiaglthreaded") == 0) {
        renderEngineType = RenderEngineType::SKIA_GL_THREADED;
    }
switch (renderEngineType) {
        case RenderEngineType::THREADED:
            ALOGD("Threaded RenderEngine with GLES Backend");
            return renderengine::threaded::RenderEngineThreaded::create(
                    [args]() { return android::renderengine::gl::GLESRenderEngine::create(args); },
                    renderEngineType);
        case RenderEngineType::SKIA_GL:
            ALOGD("RenderEngine with SkiaGL Backend");
            return renderengine::skia::SkiaGLRenderEngine::create(args);
        case RenderEngineType::SKIA_GL_THREADED: {
            // These need to be recreated, since they are a constant reference, and we need to
            // let SkiaRE know that it's running as threaded, and all GL operation will happen on
            // the same thread.
            RenderEngineCreationArgs skiaArgs =
                    RenderEngineCreationArgs::Builder()
                            .setPixelFormat(args.pixelFormat)
                            .setImageCacheSize(args.imageCacheSize)
                            .setUseColorManagerment(args.useColorManagement)
                            .setEnableProtectedContext(args.enableProtectedContext)
                            .setPrecacheToneMapperShaderOnly(args.precacheToneMapperShaderOnly)
                            .setSupportsBackgroundBlur(args.supportsBackgroundBlur)
                            .setContextPriority(args.contextPriority)
                            .setRenderEngineType(renderEngineType)
                            .build();
            ALOGD("Threaded RenderEngine with SkiaGL Backend");
            return renderengine::threaded::RenderEngineThreaded::create(
                    [skiaArgs]() {
                        return android::renderengine::skia::SkiaGLRenderEngine::create(skiaArgs);
                    },
                    renderEngineType);
        }
        case RenderEngineType::GLES:
        default:
            ALOGD("RenderEngine with GLES Backend");
            return renderengine::gl::GLESRenderEngine::create(args);
    }

GLESRenderEngine

以Android 11 為例颇玷，因?yàn)閞enderEngineType默認(rèn)為GLES，因此在渲染引擎初始化時(shí)會(huì)選擇GLESRenderEngine進(jìn)行創(chuàng)建就缆。在創(chuàng)建過程中如果設(shè)備支持Blur則創(chuàng)建BlurFilter對(duì)象帖渠。

[\framework\native\libs\renderengine\gl\GLESRenderEngine.cpp]
GLESRenderEngine::GLESRenderEngine(const RenderEngineCreationArgs& args, EGLDisplay display,
                                   EGLConfig config, EGLContext ctxt, EGLSurface stub,
                                   EGLContext protectedContext, EGLSurface protectedStub)
      : RenderEngine(args.renderEngineType),
        mEGLDisplay(display),
        mEGLConfig(config),
        mEGLContext(ctxt),
        mStubSurface(stub),
        mProtectedEGLContext(protectedContext),
        mProtectedStubSurface(protectedStub),
        mVpWidth(0),
        mVpHeight(0),
        mFramebufferImageCacheSize(args.imageCacheSize),
        mUseColorManagement(args.useColorManagement),
        mPrecacheToneMapperShaderOnly(args.precacheToneMapperShaderOnly) {
    ...
    if (args.supportsBackgroundBlur) {
        mBlurFilter = new BlurFilter(*this);
        checkErrors("BlurFilter creation");
    }
    ...
}

BlurFilter初始化時(shí)，標(biāo)識(shí)統(tǒng)一變量鏈接著色器中的屬性值部分會(huì)有一定耗時(shí)竭宰，在初始化中完成能夠減少因代碼邏輯問題的反復(fù)鏈接導(dǎo)致的渲染性能降低空郊。

[\framework\native\libs\renderengine\gl\filters\BlurFilter.cpp]
BlurFilter::BlurFilter(GLESRenderEngine& engine)
      : mEngine(engine),
        mCompositionFbo(engine),
        mPingFbo(engine),
        mPongFbo(engine),
        mMixProgram(engine),
        mBlurProgram(engine) {
    mMixProgram.compile(getVertexShader(), getMixFragShader());
    mMPosLoc = mMixProgram.getAttributeLocation("aPosition");
    mMUvLoc = mMixProgram.getAttributeLocation("aUV");
    mMTextureLoc = mMixProgram.getUniformLocation("uTexture");
    mMCompositionTextureLoc = mMixProgram.getUniformLocation("uCompositionTexture");
    mMMixLoc = mMixProgram.getUniformLocation("uMix");

    mBlurProgram.compile(getVertexShader(), getFragmentShader());
    mBPosLoc = mBlurProgram.getAttributeLocation("aPosition");
    mBUvLoc = mBlurProgram.getAttributeLocation("aUV");
    mBTextureLoc = mBlurProgram.getUniformLocation("uTexture");
    mBOffsetLoc = mBlurProgram.getUniformLocation("uOffset");

    static constexpr auto size = 2.0f;
    static constexpr auto translation = 1.0f;
    const GLfloat vboData[] = {
        // Vertex data
        translation - size, -translation - size,
        translation - size, -translation + size,
        translation + size, -translation + size,
        // UV data
        0.0f, 0.0f - translation,
        0.0f, size - translation,
        size, size - translation
    };
    mMeshBuffer.allocateBuffers(vboData, 12 /* size */);
}

在GLESRenderEngine初始化創(chuàng)建完成后，如果SurfaceFlinger采用Client合成方式切揭，那么各個(gè)Layer的內(nèi)容會(huì)用GPU渲染到暫存緩沖區(qū)狞甚，最后將暫存緩沖區(qū)傳送到顯示組件。具體使用GLESRenderEngine::drawLayers將所有圖層繪制到該緩沖區(qū)廓旬。

[\framework\native\libs\renderengine\gl\GLESRenderEngine.cpp]
status_t GLESRenderEngine::drawLayers(const DisplaySettings& display,
                                      const std::vector<const LayerSettings*>& layers,
                                      const std::shared_ptr<ExternalTexture>& buffer,
                                      const bool useFramebufferCache, base::unique_fd&& bufferFence,
                                      base::unique_fd* drawFence) {
    ATRACE_CALL();
    if (layers.empty()) {
        ALOGV("Drawing empty layer stack");
        return NO_ERROR;
    }

    if (bufferFence.get() >= 0) {
        // Duplicate the fence for passing to waitFence.
        base::unique_fd bufferFenceDup(dup(bufferFence.get()));
        if (bufferFenceDup < 0 || !waitFence(std::move(bufferFenceDup))) {
            ATRACE_NAME("Waiting before draw");
            sync_wait(bufferFence.get(), -1);
        }
    }
    ...

在針對(duì)每個(gè)Layer的渲染過程中哼审，如果該圖層被設(shè)置進(jìn)行模糊濾波處理，則通過BlurFilter::prepare進(jìn)行Kawase濾波算法的圖像渲染孕豹。這里的做法是先縮小整個(gè)圖像進(jìn)行高效率的模糊處理涩盾，再通過BlurFilter::render放大模糊效果，將使用較大的合成紋理對(duì)其進(jìn)行插值得到一幀畫面励背，以隱藏縮小的瑕疵春霍。

[\framework\native\libs\renderengine\gl\GLESRenderEngine.cpp]
status_t GLESRenderEngine::drawLayers(const DisplaySettings& display,
                                      const std::vector<const LayerSettings*>& layers,
                                      const std::shared_ptr<ExternalTexture>& buffer,
                                      const bool useFramebufferCache, base::unique_fd&& bufferFence,
                                      base::unique_fd* drawFence) {
    ...
    for (auto const layer : layers) {
        if (blurLayers.size() > 0 && blurLayers.front() == layer) {
            blurLayers.pop_front();

            auto status = mBlurFilter->prepare();
            ...
            if (blurLayers.size() == 0) {
                // Done blurring, time to bind the native FBO and render our blur onto it.
                fbo = std::make_unique<BindNativeBufferAsFramebuffer>(*this,
                                                                      buffer.get()
                                                                              ->getBuffer()
                                                                              ->getNativeBuffer(),
                                                                      useFramebufferCache);
                status = fbo->getStatus();
                setViewportAndProjection(display.physicalDisplay, display.clip);
            } else {
                // There's still something else to blur, so let's keep rendering to our FBO
                // instead of to the display.
                status = mBlurFilter->setAsDrawTarget(display,
                                                      blurLayers.front()->backgroundBlurRadius);
            }
            ...
            status = mBlurFilter->render(blurLayersSize > 1);
            ...
        }
    ...
}

BlurFilter——GL

具體到模糊濾波的代碼，在BlurFilter::prepare中叶眉，通過傳入的模糊半徑參數(shù)mRadius計(jì)算模糊限制步長stepX/stepY和kawase模糊迭代次數(shù)passes址儒。Android 12中限定的最多迭代次數(shù)為6次芹枷。

[\frameworks\native\libs\renderengine\gl\filters\BlurFilter.cpp]
status_t BlurFilter::prepare() {
    ATRACE_NAME("BlurFilter::prepare");

    const auto radius = mRadius / 6.0f;
    // Calculate how many passes we'll do, based on the radius.
    // Too many passes will make the operation expensive.
    const auto passes = min(kMaxPasses, (uint32_t)ceil(radius));

    const float radiusByPasses = radius / (float)passes;
    const float stepX = radiusByPasses / (float)mCompositionFbo.getBufferWidth();
    const float stepY = radiusByPasses / (float)mCompositionFbo.getBufferHeight();
   ...
}

傳入紋理信息mCompositionFbo.getTextureName()、步長stepX莲趣，stepY進(jìn)行片段渲染鸳慈。

status_t BlurFilter::prepare() {
    ...
    // Let's start by downsampling and blurring the composited frame simultaneously.
    mBlurProgram.useProgram();
    glActiveTexture(GL_TEXTURE0);
    glUniform1i(mBTextureLoc, 0);
    glBindTexture(GL_TEXTURE_2D, mCompositionFbo.getTextureName());
    glUniform2f(mBOffsetLoc, stepX, stepY);
    glViewport(0, 0, mPingFbo.getBufferWidth(), mPingFbo.getBufferHeight());
    mPingFbo.bind();
    drawMesh(mBUvLoc, mBPosLoc);
    ...
}

調(diào)用OpenGL API 執(zhí)行單次模糊渲染。

void BlurFilter::drawMesh(GLuint uv, GLuint position) {

    glEnableVertexAttribArray(uv);
    glEnableVertexAttribArray(position);
    mMeshBuffer.bind();
    glVertexAttribPointer(position, 2 /* size */, GL_FLOAT, GL_FALSE,
                          2 * sizeof(GLfloat) /* stride */, 0 /* offset */);
    glVertexAttribPointer(uv, 2 /* size */, GL_FLOAT, GL_FALSE, 0 /* stride */,
                          (GLvoid*)(6 * sizeof(GLfloat)) /* offset */);
    mMeshBuffer.unbind();

    // draw mesh
    glDrawArrays(GL_TRIANGLES, 0 /* first */, 3 /* count */);
}

原生限制最多6次的迭代模糊渲染妖爷，通過兩個(gè)幀緩沖區(qū)read和draw做離屏讀寫緩沖蝶涩，使用幀緩沖區(qū)渲染到紋理的方式將結(jié)果紋理作為下次渲染所需要的紋理圖片理朋，迭代反復(fù)渲染絮识。

status_t BlurFilter::prepare() {

    ...
    // And now we'll ping pong between our textures, to accumulate the result of various offsets.
    GLFramebuffer* read = &mPingFbo;
    GLFramebuffer* draw = &mPongFbo;
    glViewport(0, 0, draw->getBufferWidth(), draw->getBufferHeight());
    for (auto i = 1; i < passes; i++) {
        ATRACE_NAME("BlurFilter::renderPass");
        draw->bind();
j
        glBindTexture(GL_TEXTURE_2D, read->getTextureName());
        glUniform2f(mBOffsetLoc, stepX * i, stepY * i);

        drawMesh(mBUvLoc, mBPosLoc);

        // Swap buffers for next iteration
        auto tmp = draw;
        draw = read;
        read = tmp;
    }
    mLastDrawTarget = read;

    return NO_ERROR;
}

原生GL渲染引擎的著色器采用Kawase算法，著色器的修改和自定義可以在該部分完成嗽上。需要注意的是修改著色器中的濾波算法可能由于算法限制導(dǎo)致渲染效率低次舌，畫面顯示幀率降低。

string BlurFilter::getFragmentShader() const {
    return R"SHADER(#version 300 es
        precision mediump float;

        uniform sampler2D uTexture;
        uniform vec2 uOffset;

        in highp vec2 vUV;
        out vec4 fragColor;

        void main() {
            fragColor  = texture(uTexture, vUV, 0.0);
            fragColor += texture(uTexture, vUV + vec2( uOffset.x,  uOffset.y), 0.0);
            fragColor += texture(uTexture, vUV + vec2( uOffset.x, -uOffset.y), 0.0);
            fragColor += texture(uTexture, vUV + vec2(-uOffset.x,  uOffset.y), 0.0);
            fragColor += texture(uTexture, vUV + vec2(-uOffset.x, -uOffset.y), 0.0);

            fragColor = vec4(fragColor.rgb * 0.2, 1.0);
        }
    )SHADER";
}

BlurFilter——Skia

Android 12 上針對(duì)模糊濾波的相關(guān)內(nèi)容同Android 11兽愤，不過是由原來的GL換成了Skia彼念。內(nèi)部的模糊濾波算法和執(zhí)行流程近似相同。

以下簡(jiǎn)單列出Skia中BlurFilter源碼浅萧，其中BlurFilter::generate類比GL中的BlurFilter::prepare

sk_sp<SkImage> BlurFilter::generate(GrRecordingContext* context, const uint32_t blurRadius,
                                    const sk_sp<SkImage> input, const SkRect& blurRect) const {
    ...
    // And now we'll build our chain of scaled blur stages
    for (auto i = 1; i < numberOfPasses; i++) {
        const float stepScale = (float)i * kInputScale;
        blurBuilder.child("input") =
                tmpBlur->makeShader(SkTileMode::kClamp, SkTileMode::kClamp, linear);
        blurBuilder.uniform("in_blurOffset") = SkV2{stepX * stepScale, stepY * stepScale};
        blurBuilder.uniform("in_maxSizeXY") =
                SkV2{blurRect.width() * kInputScale, blurRect.height() * kInputScale};
        tmpBlur = blurBuilder.makeImage(context, nullptr, scaledInfo, false);
    }

    return tmpBlur;
}

Skia中的BlurFilter::drawBlurRegion類比GL中的BlurFilter::render逐沙。

void BlurFilter::drawBlurRegion(SkCanvas* canvas, const SkRRect& effectRegion,
                                const uint32_t blurRadius, const float blurAlpha,
                                const SkRect& blurRect, sk_sp<SkImage> blurredImage,
                                sk_sp<SkImage> input) {
    ATRACE_CALL();

    SkPaint paint;
    paint.setAlphaf(blurAlpha);

    const auto blurMatrix = getShaderTransform(canvas, blurRect, kInverseInputScale);
    SkSamplingOptions linearSampling(SkFilterMode::kLinear, SkMipmapMode::kNone);
    const auto blurShader = blurredImage->makeShader(SkTileMode::kClamp, SkTileMode::kClamp,
                                                     linearSampling, &blurMatrix);

    if (blurRadius < kMaxCrossFadeRadius) {
        // For sampling Skia's API expects the inverse of what logically seems appropriate. In this
        // case you might expect the matrix to simply be the canvas matrix.
        SkMatrix inputMatrix;
        if (!canvas->getTotalMatrix().invert(&inputMatrix)) {
            ALOGE("matrix was unable to be inverted");
        }

        SkRuntimeShaderBuilder blurBuilder(mMixEffect);
        blurBuilder.child("blurredInput") = blurShader;
        blurBuilder.child("originalInput") =
                input->makeShader(SkTileMode::kClamp, SkTileMode::kClamp, linearSampling,
                                  inputMatrix);
        blurBuilder.uniform("mixFactor") = blurRadius / kMaxCrossFadeRadius;

        paint.setShader(blurBuilder.makeShader(nullptr, true));
    } else {
        paint.setShader(blurShader);
    }
    ...
}

注意事項(xiàng)

• 不同于gl傳入片段著色器的是紋理映射坐標(biāo)，范圍是（0洼畅，1）吩案；skia中傳入片段著色器的是屏幕頂點(diǎn)坐標(biāo)，范圍是（screenW帝簇，screenH）徘郭。

• 本文介紹的BlurFilter處理流程都是SurfaceFlinger采用Client合成方式的圖層，因此會(huì)通過renderEngine渲染后傳給HWC的Layer list丧肴，如果采用Device合成方式的圖層不會(huì)經(jīng)過renderEngine渲染残揉，而是直接將該圖層對(duì)應(yīng)的GraphicBuffer的buffer handle放入HWC的Layer list。