上篇文章已經(jīng)簡單了說了一下人臉識別的流程桥氏，那接下來的篇幅效诅，我將詳細的寫出整個流程中用到的Library 和 具體的代碼输吏，讓大家更直觀的感受到整個人臉識別的檢測過程
回顧一下人臉識別的過程
這里我將標注出用到的 Library
- 1.發(fā)現(xiàn)人臉 (ios-AVFoundation+MTCNN Android-MTCNN)
- 2.關鍵點檢測 （MTCNN）
- 3.確定人臉姿態(tài) （關鍵點計算乡数，這里可以不用管织盼，因為我用了9個方向的特征，比如說左轉的姿態(tài)就用底庫中左臉的特征點來比較盗似，這里只做正臉檢測）
- 4.特征抽取 （MobileFaceNet模型）
- 5.根據(jù)姿態(tài)對比相應的特征 （特征余弦相似度）
單張圖片特征抽取的流程
image.png
1.發(fā)現(xiàn)人臉 && 關鍵點抽取
即 找到人臉的位置哩陕，這里iOS端多用了一個 用AVFoundation來檢測人臉，為什么這么做呢赫舒？
因為MTCNN里面是3個網(wǎng)絡悍及，跑起來CPU很高，手機發(fā)熱接癌，耗電量也高心赶，為了不一直去跑MTCNN，所以先用原生方法檢測人臉缺猛，因為原生檢測人臉效率更高缨叫，也不是特別發(fā)熱，當檢測到人臉后再丟給MTCNN檢測荔燎，拿出關鍵點耻姥；
Android 因為系統(tǒng)版本差異太大，沒有用原生湖雹，直接丟進MTCNN咏闪，檢測出人臉框和關鍵點
- iOS代碼 檢測最大人臉，返回格式[關鍵點(array),人臉Rect(value)]
- (NSArray *)detectMaxFace:(UIImage *)image
{
int w = image.size.width;
int h = image.size.height;
unsigned char* rgba = new unsigned char[w*h*4];
{
CGColorSpaceRef colorSpace = CGImageGetColorSpace(image.CGImage);
CGContextRef contextRef = CGBitmapContextCreate(rgba, w, h, 8, w*4,
colorSpace,
kCGImageAlphaNoneSkipLast | kCGBitmapByteOrderDefault);
CGContextDrawImage(contextRef, CGRectMake(0, 0, w, h), image.CGImage);
CGContextRelease(contextRef);
}
ncnn::Mat ncnn_img;
ncnn_img = ncnn::Mat::from_pixels(rgba, ncnn::Mat::PIXEL_RGBA2RGB, w, h);
std::vector<Bbox> finalBbox;
// new_mtcnn->detect(ncnn_img, finalBbox);
cv::Mat temp;
UIImageToMat(image, temp);
// float cost;
new_mtcnn->detectMaxFace(ncnn_img, finalBbox);
int32_t num_face = static_cast<int32_t>(finalBbox.size());
int out_size = 1+num_face*14;
NSMutableArray *faceInfoArr = [NSMutableArray arrayWithCapacity:0];
//
int *faceInfo = new int[out_size];
faceInfo[0] = num_face;
for(int i=0;i<num_face;i++){
NSMutableArray *points = [NSMutableArray arrayWithCapacity:0];
CGRect rect = CGRectMake(finalBbox[i].x1, finalBbox[i].y1, finalBbox[i].x2 - finalBbox[i].x1, finalBbox[i].y2 - finalBbox[i].y1);
for (int j =0;j<5;j++){
CGPoint point = CGPointMake(finalBbox[i].ppoint[j], finalBbox[i].ppoint[j + 5]);
[points addObject:[NSValue valueWithCGPoint:point]];
}
[faceInfoArr addObject:points];
[faceInfoArr addObject:[NSValue valueWithCGRect:rect]];
}
delete [] rgba;
delete [] faceInfo;
finalBbox.clear();
return faceInfoArr;
}
2.判斷人臉姿態(tài)是否為正臉
因為我們要抽取特征的臉 一定要是正臉 所抽取的特征才有意義
image.png
上圖為MTCNN抽取出來的關鍵點摔吏，同時在數(shù)組里的位置我也有標注出來,主要比的就是眼睛到鼻子的距離
- (BOOL)faceFront:(NSArray *)shape
{
int mStartNumRightTemp = 0;
int mStartNumLeftTemp = 0;
CGPoint right_pupil;
CGPoint left_pupil;
left_pupil.x = [shape[0] CGPointValue].x ; //左眼瞳孔坐標x
left_pupil.y = [shape[0] CGPointValue].y ; //左眼瞳孔坐標y
right_pupil.x = [shape[1] CGPointValue].x ; //右眼瞳孔坐標x
right_pupil.y = [shape[1] CGPointValue].y ; //右眼瞳孔坐標y
mStartNumRightTemp = abs(right_pupil.x - [shape[2] CGPointValue].x); //到鼻子中間的距離
mStartNumLeftTemp = abs([shape[2] CGPointValue].x - left_pupil.x);
float tempp1 = mStartNumRightTemp / (float)mStartNumLeftTemp;
float tempp2 = mStartNumLeftTemp / (float)mStartNumRightTemp;
if (tempp1 > 0.7 && tempp2>0.7)
return YES;
else
return NO;
}
3.人臉對齊 （openCV）
所謂人臉對齊鸽嫂，看下面的圖就知道什么是人臉對齊了
image.png
左邊如果是原圖的話，對齊后的效果就是右邊這樣征讲，旋轉了一下圖像
下面的方法包括了 旋轉與摳出人臉圖像据某，
輸入整張圖片，人臉的5個關鍵點诗箍，返回對齊后 并截取了臉部的圖
- (Mat)getAlignMat:(Mat)bgrmat landmarks:(NSArray *)landmarks
{
//left eye
float left_eye_x = [landmarks[0] CGPointValue].x;
float left_eye_y = [landmarks[0] CGPointValue].y;
//right eye
float right_eye_x = [landmarks[1] CGPointValue].x;
float right_eye_y = [landmarks[1] CGPointValue].y;
//nose
float nose_x = [landmarks[2] CGPointValue].x;
float nose_y = [landmarks[2] CGPointValue].y;
//mouth left
float mouth_left_x = [landmarks[3] CGPointValue].x;
float mouth_left_y = [landmarks[3] CGPointValue].y;
//mouth right
float mouth_right_x = [landmarks[4] CGPointValue].x;
float mouth_right_y = [landmarks[4] CGPointValue].y;
//mouth center
float mouth_center_x = (mouth_left_x + mouth_right_x) / 2;
float mouth_center_y = (mouth_left_y + mouth_right_y) / 2;
cv::Mat affineMat;
std::vector<cv::Point2f> src_pts ;
src_pts.push_back(cv::Point2f(left_eye_x, left_eye_y));
src_pts.push_back(cv::Point2f(right_eye_x, right_eye_y));
src_pts.push_back(cv::Point2f(mouth_center_x, mouth_center_y));
cv::Point2f left_eye(38, 52);
cv::Point2f right_eye(74, 52);
cv::Point2f mouth_center(56, 92);
cv::Size dsize(112, 112);
std::vector<cv::Point2f> dst_pts;
dst_pts.push_back(left_eye);
dst_pts.push_back(right_eye);
dst_pts.push_back(mouth_center);
affineMat = cv::getAffineTransform(src_pts, dst_pts);
cv::Mat alignedImg;
cv::warpAffine(bgrmat, alignedImg, affineMat, dsize, cv::INTER_CUBIC, cv::BORDER_REPLICATE);
return alignedImg;
}
4.抽取特征 （MobileFaceNet）
ncnn::Mat NCNNNet::getFaceFeatures(cv::Mat face)
{
ncnn::Mat in = ncnn::Mat::from_pixels(face.data, ncnn::Mat::PIXEL_BGR, face.cols, face.rows);
const float mean_vals[3] = {127.5f, 127.5f, 127.5f};
const float std_vals[3] = {0.0078125f, 0.0078125f, 0.0078125f};
in.substract_mean_normalize(mean_vals, std_vals);
ncnn::Extractor ex = features.create_extractor();
ex.set_light_mode(true);
ex.set_num_threads(4);
ex.input(mobilefacenet_proto_id::BLOB_data, in);
ncnn::Mat out;
ex.extract(mobilefacenet_proto_id::BLOB_fc1_fc1_scale, out);
return out;
}
抽取出來的為128個float的數(shù)據(jù)癣籽，我們需要將ncnn:Mat 轉換成你要的Array，然后再進行相似度計算,這里以iOS端為??
- (NSArray <NSNumber *>*)turnNCNNMat:(ncnn::Mat)feature
{
NSMutableArray *features = [NSMutableArray arrayWithCapacity:0];
for (int i = 0; i < feature.w; i++) {
[features addObject:@((float)feature[i])];
}
return features;
}
同時為了提高運算精度，我們需要將抽取的特征數(shù)據(jù)歸一化筷狼，這里得到的就是我們需要的人臉特征了
- (NSArray *)normalizeFeature:(NSArray <NSNumber *>*)feature
{
NSMutableArray *fixFArr = [NSMutableArray arrayWithCapacity:0];
if (!feature.count) return nil;
float norm = 0;
for (int i = 0; i < feature.count; i++) {
norm += [feature[i] floatValue] * [feature[i] floatValue];
}
norm = sqrt(norm);
if (norm == 0) return feature;
for (int i = 0; i < feature.count; i++) {
float newFeature = [feature[i] floatValue] / norm;
[fixFArr addObject:@(newFeature)];
}
return fixFArr;
}
5.人臉相似度比對（余弦相似度）
通過以上步驟瓶籽，我們已經(jīng)可以抽取一張人臉的特征了，如果我們抽取兩張埂材，就可以得到兩張人臉的信息塑顺，這時候，我們就可以進行對比的操作了
- (CGFloat)getFeatureDistanceWithFirstFeatures:(NSArray *)firstFeature second:(NSArray *)secondFeature
{
if (!firstFeature.count || !secondFeature.count) return 0;
float distance = 0;
for (int i = 0; i < firstFeature.count && i < firstFeature.count; ++i) {
distance += ([firstFeature[i] floatValue] - [secondFeature[i] floatValue]) * ([firstFeature[i] floatValue]- [secondFeature[i] floatValue]);
}
distance = sqrt(distance);
CGFloat similarity = (1 - pow(distance / 2, 2)) * 100;
return similarity;
}
