前言

在這個(gè)美即真理井佑、全民娛樂的時(shí)代草巡，可愛有趣的人臉貼紙?jiān)诟鞔竺李佨浖械玫搅藦V泛的應(yīng)用守呜，現(xiàn)在已經(jīng)不僅局限于相機(jī)美顏類軟件中，在社交山憨、娛樂類的app中對(duì)人臉貼紙查乒、AR貼紙的需求也非常廣泛。本文詳細(xì)介紹了集成華為HMS ML kit人臉識(shí)別實(shí)現(xiàn)2d貼紙的集成過程郁竟，在后面的文章中我們還會(huì)介紹3D貼紙的開發(fā)過程玛迄，歡迎大家關(guān)注哦

場(chǎng)景

在美顏相機(jī)、美圖app以及社交類app(如抖音棚亩、微博蓖议、微信)等需要對(duì)拍照，或者對(duì)照片進(jìn)行處理的app都會(huì)構(gòu)建自己特有的貼紙的需求讥蟆。

開發(fā)前準(zhǔn)備

在項(xiàng)目級(jí)gradle里添加華為maven倉

打開AndroidStudio項(xiàng)目級(jí)build.gradle文件

增量添加如下maven地址：

buildscript {
     {        
        maven {url 'http://developer.huawei.com/repo/'}
    }    
}
allprojects {
    repositories {       
        maven { url 'http://developer.huawei.com/repo/'}
    }
}

在應(yīng)用級(jí)的build.gradle里面加上SDK依賴

// Face detection SDK.
implementation 'com.huawei.hms:ml-computer-vision-face:2.0.1.300'
// Face detection model.
implementation 'com.huawei.hms:ml-computer-vision-face-shape-point-model:2.0.1.300'

在AndroidManifest.xml文件里面申請(qǐng)相機(jī)勒虾、訪問網(wǎng)絡(luò)和存儲(chǔ)權(quán)限

<!--相機(jī)權(quán)限--> 
<uses-feature android:name="android.hardware.camera" />
<uses-permission android:name="android.permission.CAMERA" />
<!--寫權(quán)限--> 
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!--讀權(quán)限--> 
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

代碼開發(fā)關(guān)鍵步驟

設(shè)置人臉檢測(cè)器

MLFaceAnalyzerSetting detectorOptions;
detectorOptions = new MLFaceAnalyzerSetting.Factory()
        .setFeatureType(MLFaceAnalyzerSetting.TYPE_UNSUPPORT_FEATURES)
        .setShapeType(MLFaceAnalyzerSetting.TYPE_SHAPES)
        .allowTracing(MLFaceAnalyzerSetting.MODE_TRACING_FAST)
        .create();
detector = MLAnalyzerFactory.getInstance().getFaceAnalyzer(detectorOptions);

這里我們通過相機(jī)回調(diào)拿到相機(jī)幀數(shù)據(jù),并通過調(diào)用人臉檢測(cè)器拿到人臉輪廓點(diǎn)后寫入FacePointEngine供貼紙濾鏡使用.

@Override
public void onPreviewFrame(final byte[] imgData, final Camera camera) {
    int width = mPreviewWidth;
    int height = mPreviewHeight;

    long startTime = System.currentTimeMillis();
    //設(shè)置前后攝方向一致
    if (isFrontCamera()){
        mOrientation = 0;
    }else {
        mOrientation = 2;
    }
    MLFrame.Property property =
            new MLFrame.Property.Creator()
                    .setFormatType(ImageFormat.NV21)
                    .setWidth(width)
                    .setHeight(height)
                    .setQuadrant(mOrientation)
                    .create();

    ByteBuffer data = ByteBuffer.wrap(imgData);
    // 調(diào)用人臉檢測(cè)接口
    SparseArray<MLFace> faces = detector.analyseFrame(MLFrame.fromByteBuffer(data,property));
    //判斷是否獲取到人臉信息
    if(faces.size()>0){
        MLFace mLFace = faces.get(0);
        EGLFace EGLFace = FacePointEngine.getInstance().getOneFace(0);
        EGLFace.pitch = mLFace.getRotationAngleX();
        EGLFace.yaw = mLFace.getRotationAngleY();
        EGLFace.roll = mLFace.getRotationAngleZ() - 90;
        if (isFrontCamera())
            EGLFace.roll = -EGLFace.roll;
        if (EGLFace.vertexPoints == null) {
            EGLFace.vertexPoints = new PointF[131];
        }
        int index = 0;
        // 獲取一個(gè)人的輪廓點(diǎn)坐標(biāo)并轉(zhuǎn)化到openGL歸一化坐標(biāo)系下的浮點(diǎn)值
        for (MLFaceShape contour : mLFace.getFaceShapeList()) {
            if (contour == null) {
                continue;
            }
            List<MLPosition> points = contour.getPoints();

            for (int i = 0; i < points.size(); i++) {
                MLPosition point = points.get(i);
                float x = ( point.getY() / height) * 2 - 1;
                float y = ( point.getX() / width ) * 2 - 1;
                if (isFrontCamera())
                    x = -x;
                PointF Point = new PointF(x,y);
                EGLFace.vertexPoints[index] = Point;
                index++;
            }
        }
        // 插入人臉對(duì)象
        FacePointEngine.getInstance().putOneFace(0, EGLFace);
        // 設(shè)置人臉個(gè)數(shù)
        FacePointEngine.getInstance().setFaceSize(faces!= null ? faces.size() : 0);
    }else{
        FacePointEngine.getInstance().clearAll();
    }
    long endTime = System.currentTimeMillis();
    Log.d("TAG","Face detect time: " + String.valueOf(endTime - startTime));
}

ML kit接口返回的人臉輪廓點(diǎn)情況如圖所示：

介紹如何設(shè)計(jì)貼紙,首先看一下貼紙數(shù)JSON數(shù)據(jù)定義

public class FaceStickerJson {

    public int[] centerIndexList;   // 中心坐標(biāo)索引列表，有可能是多個(gè)關(guān)鍵點(diǎn)計(jì)算中心點(diǎn)
    public float offsetX;           // 相對(duì)于貼紙中心坐標(biāo)的x軸偏移像素
    public float offsetY;           // 相對(duì)于貼紙中心坐標(biāo)的y軸偏移像素
    public float baseScale;         // 貼紙基準(zhǔn)縮放倍數(shù)
    public int startIndex;          // 人臉起始索引瘸彤，用于計(jì)算人臉的寬度
    public int endIndex;            // 人臉結(jié)束索引从撼，用于計(jì)算人臉的寬度
    public int width;               // 貼紙寬度
    public int height;              // 貼紙高度
    public int frames;              // 貼紙幀數(shù)
    public int action;              // 動(dòng)作，0表示默認(rèn)顯示，這里用來處理貼紙動(dòng)作等
    public String stickerName;      // 貼紙名稱低零，用于標(biāo)記貼紙所在文件夾以及png文件的
    public int duration;            // 貼紙幀顯示間隔
    public boolean stickerLooping;  // 貼紙是否循環(huán)渲染
    public int maxCount;            // 最大貼紙渲染次數(shù)
    ...
    }

我們制作貓耳貼紙JSON文件,通過人臉?biāo)饕业矫夹?4號(hào)點(diǎn)和鼻尖85號(hào)點(diǎn)分別貼上耳朵和鼻子,然后把它和圖片都放在assets目錄下

{
    "stickerList": [{
        "type": "sticker",
        "centerIndexList": [84],
        "offsetX": 0.0,
        "offsetY": 0.0,
        "baseScale": 1.3024,
        "startIndex": 11,
        "endIndex": 28,
        "width": 495,
        "height": 120,
        "frames": 2,
        "action": 0,
        "stickerName": "nose",
        "duration": 100,
        "stickerLooping": 1,
        "maxcount": 5
    }, {
        "type": "sticker",
        "centerIndexList": [83],
        "offsetX": 0.0,
        "offsetY": -1.1834,
        "baseScale": 1.3453,
        "startIndex": 11,
        "endIndex": 28,
        "width": 454,
        "height": 150,
        "frames": 2,
        "action": 0,
        "stickerName": "ear",
        "duration": 100,
        "stickerLooping": 1,
        "maxcount": 5
    }]
}

這里渲染貼紙紋理我們使用GLSurfaceView,使用起來比TextureView簡單, 首先在onSurfaceChanged實(shí)例化貼紙濾鏡,傳入貼紙路徑并開啟相機(jī)

@Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {

    GLES30.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
    mTextures = new int[1];
    mTextures[0] = OpenGLUtils.createOESTexture();
    mSurfaceTexture = new SurfaceTexture(mTextures[0]);
    mSurfaceTexture.setOnFrameAvailableListener(this);

    //將samplerExternalOES 輸入到紋理中
    cameraFilter = new CameraFilter(this.context);

    //設(shè)置assets目錄下人臉貼紙路徑
    String folderPath ="cat";
    stickerFilter = new FaceStickerFilter(this.context,folderPath);

    //創(chuàng)建屏幕濾鏡對(duì)象
    screenFilter = new BaseFilter(this.context);

    facePointsFilter = new FacePointsFilter(this.context);
    mEGLCamera.openCamera();
}

然后在onSurfaceChanged初始化貼紙濾鏡

@Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
    Log.d(TAG, "onSurfaceChanged. width: " + width + ", height: " + height);
    int previewWidth = mEGLCamera.getPreviewWidth();
    int previewHeight = mEGLCamera.getPreviewHeight();
    if (width > height) {
        setAspectRatio(previewWidth, previewHeight);
    } else {
        setAspectRatio(previewHeight, previewWidth);
    }
    // 設(shè)置畫面的大小,創(chuàng)建FrameBuffer,設(shè)置顯示尺寸
    cameraFilter.onInputSizeChanged(previewWidth, previewHeight);
    cameraFilter.initFrameBuffer(previewWidth, previewHeight);
    cameraFilter.onDisplaySizeChanged(width, height);

    stickerFilter.onInputSizeChanged(previewHeight, previewWidth);
    stickerFilter.initFrameBuffer(previewHeight, previewWidth);
    stickerFilter.onDisplaySizeChanged(width, height);

    screenFilter.onInputSizeChanged(previewWidth, previewHeight);
    screenFilter.initFrameBuffer(previewWidth, previewHeight);
    screenFilter.onDisplaySizeChanged(width, height);

    facePointsFilter.onInputSizeChanged(previewHeight, previewWidth);
    facePointsFilter.onDisplaySizeChanged(width, height);
    mEGLCamera.startPreview(mSurfaceTexture);
}

最后通過onDrawFrame把貼紙繪制到屏幕

@Override
public void onDrawFrame(GL10 gl) {
    int textureId;
    // 清除屏幕和深度緩存
    GLES30.glClear(GLES30.GL_COLOR_BUFFER_BIT | GLES30.GL_DEPTH_BUFFER_BIT);
    //更新獲取一張圖
    mSurfaceTexture.updateTexImage();
    //獲取SurfaceTexture轉(zhuǎn)化矩陣
    mSurfaceTexture.getTransformMatrix(mMatrix);
    //設(shè)置相機(jī)顯示轉(zhuǎn)化矩陣
    cameraFilter.setTextureTransformMatrix(mMatrix);

    //繪制相機(jī)紋理
    textureId = cameraFilter.drawFrameBuffer(mTextures[0],mVertexBuffer,mTextureBuffer);
    //繪制貼紙紋理
    textureId = stickerFilter.drawFrameBuffer(textureId,mVertexBuffer,mTextureBuffer);
    //繪制到屏幕
    screenFilter.drawFrame(textureId , mDisplayVertexBuffer, mDisplayTextureBuffer);
    if(drawFacePoints){
        facePointsFilter.drawFrame(textureId, mDisplayVertexBuffer, mDisplayTextureBuffer);
    }
}

這樣我們的貼紙就畫到人臉上了.

Demo效果