虹軟人臉識別3.0 - 圖像數據結構介紹（C++）

時間 2019-12-23

標籤識別 3.0 圖像數據結構介紹 c++ 欄目 C&C++ 简体版

原文原文鏈接

從虹軟開放了2.0版本SDK以來，因爲具備免費、離線使用的特色，咱們公司在人臉識別門禁應用中使用了虹軟SDK，識別效果還不錯，所以比較關注虹軟SDK的官方動態。近期上線了ArcFace 3.0 SDK版本，確實作了比較大的更新。html

特徵比對支持比對模型選擇，有生活照比對模型和人證比對模型c++
識別率、防攻擊效果顯著提高算法
特徵值更新，升級後人臉庫需從新註冊數據結構
人臉檢測同時支持全角度及單一角度指針
新增了一種圖像數據傳入方式code

在V3.0版本接入過程當中，發現使用新的圖像數據結構仍是具備必定難度的，本文將從如下幾點對該圖像數據結構及使用方式進行介紹orm

SDK接口變更htm
圖像數據結構blog
步長的做用接口
OpenCV圖像數據結構轉換爲虹軟圖像數據結構

1、SDK 接口變更

在接入ArcFace 3.0 SDK時，發現新增了ASFDetectFacesEx、ASFFaceFeatureExtractEx、ASFProcessEx、ASFProcessEx_IR一組接口，該組接口使用LPASF_ImageData結構體指針的方式傳入圖像數據，以人臉檢測接口爲例，具體接口比對以下：

原始接口：

MRESULT ASFDetectFaces(
		MHandle				hEngine,							// [in] 引擎handle
		MInt32				width,								// [in] 圖片寬度
		MInt32				height,								// [in] 圖片高度
		MInt32				format,								// [in] 顏色空間格式
		MUInt8*				imgData,							// [in] 圖片數據
		LPASF_MultiFaceInfo	detectedFaces,						// [out]檢測到的人臉信息 
		ASF_DetectModel		detectModel = ASF_DETECT_MODEL_RGB	// [in] 預留字段，當前版本使用默認參數便可
		);

新增接口：

MRESULT ASFDetectFacesEx(
		MHandle				hEngine,							// [in] 引擎handle
		LPASF_ImageData		imgData,							// [in] 圖片數據
		LPASF_MultiFaceInfo	detectedFaces,						// [out] 檢測到的人臉信息
		ASF_DetectModel		detectModel = ASF_DETECT_MODEL_RGB	// [in]	預留字段，當前版本使用默認參數便可
		);

相對於原始接口，新增接口經過傳入LPASF_ImageData圖像數據結構指針替代原始接口傳入圖像數據的方式。

2、圖像數據結構

新增的圖像數據結構引入了步長pi32Pitch的概念。

步長定義：圖像對齊後一行的字節數。

2.1 虹軟圖像數據結構

圖像結構定義：

typedef LPASVLOFFSCREEN LPASF_ImageData;

typedef struct __tag_ASVL_OFFSCREEN
{
	MUInt32	u32PixelArrayFormat;
	MInt32	i32Width;
	MInt32	i32Height;
	MUInt8*	ppu8Plane[4];
	MInt32	pi32Pitch[4];
}ASVLOFFSCREEN, *LPASVLOFFSCREEN;

虹軟官方文檔中對該圖像數據結構的介紹：

類型	變量名	描述
MUInt32	u32PixelArrayFormat	顏色格式
MInt32	i32Width	圖像寬度
MInt32	i32Height	圖像高度
MUInt8*	ppu8Plane	圖像數據
MInt32	pi32Pitch	圖像步長

2.2 OpenCV 圖像數據結構

OpenCV提供了IplImage和Mat兩種比較經常使用的圖像數據結構。

IplImage 圖像數據結構

typedef struct _IplImage
{
    int  width;             /* Image width in pixels.                           */
    int  height;            /* Image height in pixels.                          */
    char *imageData;        /* Pointer to aligned image data.         */
    int  widthStep;         /* Size of aligned image row in bytes.    */
    ...  //其餘字段這裏不作展現，感興趣的小夥伴能夠查看下opencv中的頭文件
}
IplImage;

Mat 圖像數據結構

屬性	說明
cols	矩陣的列數（圖像寬度）
rows	矩陣的行數（圖像高度）
data	uchar型的指針。Mat類分爲了兩個部分:矩陣頭和指向矩陣數據部分的指針，data就是指向矩陣數據的指針。
step	圖像對齊以後一行的字節數

3、步長的做用

經過以上描述咱們看到OpenCV和虹軟算法庫針對圖像數據結構都引入了圖像步長的概念，這裏咱們瞭解一下圖像步長。

OpenCV 讀圖會作圖像對齊

以下圖，一張尺寸爲998x520的圖像，使用OpenCV讀取圖像數據後，圖像尺寸仍爲998x520，顏色格式爲BGR24，可是圖像步長並非998 * 3，而是1000 * 3，右邊填充了2個像素，OpenCV對圖像作了四字節對齊，虹軟SDK內部算法再經過傳入的圖像寬度去計算步長則會出現誤差，圖像數據錯亂，基本不可能檢測到人臉。

步長的重要性 只是差了這幾個像素，爲何就致使人臉檢測不到了呢？以前說到過，步長能夠理解爲圖像對齊後一行的字節數。若是第一行像素的讀取有誤差，那後續像素的讀取也會受到影響。

如下是對一張大小爲1000x554的圖片，以不一樣步長進行解析的結果：

以1000爲步長解析	以996爲步長解析

能夠看到，對於一張圖像，若是使用了錯誤的步長去解析，咱們可能就沒法看到正確的圖像內容。

結論：經過引入圖像步長可以有效的避免高字節對齊的問題。

4、 OpenCV圖像數據結構轉換爲虹軟圖像數據結構

當前C/C++開發者對圖像進行編解碼處理通常都會用到OpenCV庫，這裏咱們介紹一下如何將OpenCV轉換爲虹軟的圖像數據結構。虹軟官方文檔中說明支持七種顏色格式，咱們就列出七種顏色格式的轉換方法。

OpenCV 讀取過來的圖像通常爲BGR24格式，可以使用下述方法進行圖像數據結構轉換。
若原圖爲紅外圖像，需將圖像轉換爲ASVL_PAF_GRAY格式（官網文檔中也有示例），再使用下述方法進行轉換。

IplImage 轉 ASVLOFFSCREEN

int ColorSpaceConversion(MInt32 format, IplImage* img, ASVLOFFSCREEN& offscreen)
{
	switch (format)		//原始圖像顏色格式
	{
	case ASVL_PAF_I420:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img->width;
		offscreen.i32Height = img->height;
		offscreen.pi32Pitch[0] = img->widthStep;
		offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0] >> 1;
		offscreen.pi32Pitch[2] = offscreen.pi32Pitch[0] >> 1;
		offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
		offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0];
		offscreen.ppu8Plane[2] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0] * 5 / 4;
		break;
	case ASVL_PAF_YUYV:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img->width;
		offscreen.i32Height = img->height;
		offscreen.pi32Pitch[0] = img->widthStep;
		offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
		break;
	case ASVL_PAF_NV12:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img->width;
		offscreen.i32Height = img->height;
		offscreen.pi32Pitch[0] = img->widthStep;
		offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0];
		offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
		offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height;
		break;
	case ASVL_PAF_NV21:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img->width;
		offscreen.i32Height = img->height;
		offscreen.pi32Pitch[0] = img->widthStep;
		offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0];
		offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
		offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height;
		break;
	case ASVL_PAF_RGB24_B8G8R8:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img->width;
		offscreen.i32Height = img->height;
		offscreen.pi32Pitch[0] = img->widthStep;
		offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
		break;
	case ASVL_PAF_DEPTH_U16:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img->width;
		offscreen.i32Height = img->height;
		offscreen.pi32Pitch[0] = img->widthStep;
		offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
		break;
	case ASVL_PAF_GRAY:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img->width;
		offscreen.i32Height = img->height;
		offscreen.pi32Pitch[0] = img->widthStep;
		offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
		break;
	default:
		return 0;
	}
	return 1;
}

Mat 轉 ASVLOFFSCREEN

int ColorSpaceConversion(MInt32 format, cv::Mat img, ASVLOFFSCREEN& offscreen)
{
	switch (format)   //原始圖像顏色格式
	{
	case ASVL_PAF_I420:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img.cols;
		offscreen.i32Height = img.rows;
		offscreen.pi32Pitch[0] = img.step;
		offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0] >> 1;
		offscreen.pi32Pitch[2] = offscreen.pi32Pitch[0] >> 1;
		offscreen.ppu8Plane[0] = img.data;
		offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0];
		offscreen.ppu8Plane[2] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0] * 5 / 4;
		break;
	case ASVL_PAF_YUYV:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img.cols;
		offscreen.i32Height = img.rows;
		offscreen.pi32Pitch[0] = img.step;
		offscreen.ppu8Plane[0] = img.data;;
		break;
	case ASVL_PAF_NV12:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img.cols;
		offscreen.i32Height = img.rows;
		offscreen.pi32Pitch[0] = img.step;
		offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0];
		offscreen.ppu8Plane[0] = img.data;
		offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height;
		break;
	case ASVL_PAF_NV21:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img.cols;
		offscreen.i32Height = img.rows;
		offscreen.pi32Pitch[0] = img.step;
		offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0];
		offscreen.ppu8Plane[0] = img.data;
		offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height;
		break;
	case ASVL_PAF_RGB24_B8G8R8:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img.cols;
		offscreen.i32Height = img.rows;
		offscreen.pi32Pitch[0] = img.step;
		offscreen.ppu8Plane[0] = img.data;
		break;
	case ASVL_PAF_DEPTH_U16:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img.cols;
		offscreen.i32Height = img.rows;
		offscreen.pi32Pitch[0] = img.step;
		offscreen.ppu8Plane[0] = img.data;
		break;
	case ASVL_PAF_GRAY:
		offscreen.u32PixelArrayFormat = (unsigned int)format;
		offscreen.i32Width = img.cols;
		offscreen.i32Height = img.rows;
		offscreen.pi32Pitch[0] = img.step;
		offscreen.ppu8Plane[0] = img.data;
		break;
	default:
		return 0;
	}
	return 1;
}

舉例說明

這裏引用了虹軟官網文檔中的示例，但使用了上述的圖像格式轉換方法。

//opencv方式裁剪圖片
void CutIplImage(IplImage* src, IplImage* dst, int x, int y)
{
	CvSize size = cvSize(dst->width, dst->height);//區域大小
	cvSetImageROI(src, cvRect(x, y, size.width, size.height));//設置源圖像ROI
	cvCopy(src, dst); //複製圖像
	cvResetImageROI(src);//源圖像用完後，清空ROI
}

IplImage* originalImg = cvLoadImage("1280 x 720.jpg");	

//圖像裁剪，寬度作四字節對齊，若能保證圖像是四字節對齊這步能夠不用作
IplImage* img = cvCreateImage(cvSize(originalImg->width - originalImg->width % 4, originalImg->height), IPL_DEPTH_8U, originalImg->nChannels);
CutIplImage(originalImg, img, 0, 0);

//圖像數據以結構體形式傳入，對更高精度的圖像兼容性更好
ASF_MultiFaceInfo detectedFaces = { 0 };
ASVLOFFSCREEN offscreen = { 0 };
//IplImage 轉 ASVLOFFSCREEN
ColorSpaceConversion(ASVL_PAF_RGB24_B8G8R8, img, offscreen);
if (img)
{
    MRESULT res = ASFDetectFacesEx(handle, &offscreen, &detectedFaces);
    if (MOK != res)
    {
        printf("ASFDetectFacesEx failed: %d\n", res);
    }
    else
    {
        // 打印人臉檢測結果
        for (int i = 0; i < detectedFaces.faceNum; i++)
		{
			printf("Face Id: %d\n", detectedFaces.faceID[i]);
			printf("Face Orient: %d\n", detectedFaces.faceOrient[i]);
			printf("Face Rect: (%d %d %d %d)\n", 
				detectedFaces.faceRect[i].left, detectedFaces.faceRect[i].top, 
				detectedFaces.faceRect[i].right, detectedFaces.faceRect[i].bottom);
		}
    }
    
    //釋放圖像內存，這裏只是作人臉檢測，若還須要作特徵提取等處理，圖像數據不必釋放這麼早
    cvReleaseImage(&img);
}
cvReleaseImage(&originalImg);

我的總結：經過研究發現V3.0 版本SDK使用老接口也是能夠正常使用的，新接口對更高字節對齊的圖像兼容性更好。

Demo可在虹軟人臉識別開放平臺下載