CN105608449B

CN105608449B - Phase and reflectivity based face recognition device and method based on phase profilometry

Info

Publication number: CN105608449B
Application number: CN201610103659.9A
Authority: CN
Inventors: 刘凯; 王前慧; 余祥
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2016-02-25
Filing date: 2016-02-25
Publication date: 2021-06-22
Anticipated expiration: 2036-02-25
Also published as: CN105608449A

Abstract

The invention discloses a phase and reflectivity face recognition device and method based on phase profilometry, relates to the field of face recognition, and aims to solve the technical problems of high cost, low precision, poor universality, huge calculation amount, insufficient algorithm structure optimization and the like of the device in the prior art. The method mainly comprises the steps of scanning the face by using a PMP method and calculating phase data and reflectivity data of the face without acquiring three-dimensional data; carrying out normalization processing on the phase data of the face area; and (3) performing dimensionality reduction treatment on the phase image and the reflectivity image of the human face by using a PCA (principal component analysis) method, and classifying and identifying the human face by using a nearest neighbor classifier. The invention is used for simplifying the face recognition operation and reducing the implementation cost.

Description

Phase and reflectivity face recognition device and method based on phase profilometry

Technical Field

The invention relates to the field of face recognition, in particular to phase and reflectivity face recognition based on phase profilometry.

Background

The research of the face recognition technology has primarily focused on the field of two-dimensional face recognition, and many recognition methods have been proposed in succession. The classical two-dimensional face recognition methods mainly include methods based on geometric features, algebraic features and machine learning. The Eigenface (Eigenface) method is a face recognition method based on statistical features and is relatively successful. An Eigenface (Eigenface) is a face recognition method derived from Principal Component Analysis (PCA), and the key steps of the method are that K-L transformation is performed on a face image to obtain Eigenface matrix data, then an original face image is projected onto an Eigenface matrix to realize dimension reduction and feature extraction, and finally a nearest neighbor classifier is used for face classification and recognition. In recent decades, with the development of computer technology and measuring instruments, three-dimensional data is more and more easily obtained, and many researchers have turned from two-dimensional face recognition research to three-dimensional face recognition research. At present, two technologies of passive stereo measurement and active stereo measurement are mainly used for measuring three-dimensional data in the field of machine vision. Phase Measurement Profiling (PMP) is a structured light measurement, belongs to a very successful three-dimensional measurement technology, and has high data accuracy. The existing three-dimensional face recognition method mainly comprises a recognition algorithm based on spatial domain matching, integral feature matching and 3D +2D double modes.

Two-dimensional face recognition technology is mature and enters commercial application, but two-dimensional face recognition has defects in many aspects, and is easily affected by factors such as ambient light, posture, expression and the like, and the recognition effect is difficult to further improve. With the development of computer technology and measuring instruments, three-dimensional data is more and more easily obtained, and thus many studies are beginning to shift from two-dimensional face recognition studies to three-dimensional face recognition studies. Phase Measurement Profiling (PMP) is a structured light measurement, belongs to a very successful three-dimensional measurement technology, and has high data accuracy. Because the three-dimensional face data is less influenced by illumination compared with the two-dimensional face data, a better recognition effect than two-dimensional face recognition is obtained. The three-dimensional face recognition method using the depth map belongs to a method based on integral feature matching, and converts three-dimensional face recognition into two-dimensional face recognition, so that the algorithm is greatly simplified. However, in the structured light measurement, the acquisition of three-dimensional data needs to depend on calibration data, the calculation amount required by calibration is large or the technical requirement is high, and the calculation of three-dimensional point cloud is required before the three-dimensional face recognition, so that the calculation amount is large.

Disclosure of Invention

Aiming at the prior art, the invention aims to provide a phase and reflectivity face recognition device and method based on phase profilometry, and aims to solve the technical problems of high cost, low precision, poor universality, huge calculated amount, insufficient algorithm structure optimization and the like of the device in the prior art; in addition, both the apparatus and the method are susceptible to significant effects of environmental variables, further reducing system identification accuracy and efficiency.

In order to achieve the purpose, the technical scheme adopted by the invention is as follows:

the phase and reflectivity face recognition device based on the phase profilometry comprises a projection unit: the device comprises a light source capable of projecting a pattern, a first lens, a sinusoidal grating template, a phase shifter and a second lens; the first lens: for converging the output light of the light source; the sine grating template comprises the following components: receiving the transmitted light of the first lens and transmitting parallel light having a first sinusoidal phase; the phase shifter comprises: the device comprises a sine grating template, a first sine phase and a second sine phase, wherein the sine grating template is used for adjusting and transmitting parallel light with the second sine phase; the second lens: receiving the parallel light of a first sinusoidal phase or the parallel light of a second sinusoidal phase transmitted by a sinusoidal grating template, and focusing and projecting the parallel light to the surface of the identified object; further comprising a processing unit: the phase shifter and the light source are used for calculation and control and are controlled; further comprising an image sensing unit: receiving surface modulated light of the identified object and feeding back an illumination signal to the processing unit.

In the above scheme, the image sensing unit includes a charge coupled array device, a filter circuit, an integration circuit, a video processing module and an amplification output circuit, which are connected in sequence.

In the above scheme, the video processing module comprises a CPLD control chip and a VSP video processing chip; the CPLD control chip generates a driving time sequence of the VSP video processing chip, and the VSP video processing chip performs logic conversion operation on an output signal of the integrating circuit and generates a digital clock signal. Fast speed and easy programming.

A face recognition method based on phase and reflectivity of phase profilometry comprises the following steps:

adjusting the sine grating template for N times through a phase shifter, transmitting a group of N frames of sine patterns to the surface of an identified object by a projection unit, and transmitting the light intensity distribution I of the image by the projection unit^pComprises the following steps:

(1) in the formula (x)^p，y^p) Representing projection unit coordinates, A^pAnd B^pIs a fixed parameter for the projection unit, N is the phase shift coefficient (N ═ 1,2, …, N), f is the spatial frequency at which the projection unit transmits the sinusoidal pattern;

(II) synchronously capturing a corresponding frame sinusoidal pattern modulated by the surface of the identified object by the image sensing unit while the projection unit projects the sinusoidal pattern to obtain an illumination signal;

(III) the processing unit is used for processing the light intensity distribution I of the transmission image of the projection unit^pMapping relation with light intensity distribution at image sensing unit position to obtain light intensity distribution I at image sensing unit^c；

(IV) the image sensing unit feeds back the illumination signal to the processing unit;

(V) according to the light intensity distribution I at the image sensing unit^cThe processing unit calculates the feedback illumination signal and obtains the phase

And brightness modulation B^cThe calculation formula is as follows:

(2) in the formulae (1) and (3), I^cIs the light intensity distribution of the image captured by the image sensing unit, and (2) in the formula (x)^c,y^c) Representing image sensing unit coordinates;

(VI) modulation of the luminance B^cIn proportion to the reflectivity, the processing unit pairs the phase according to a principal component analysis algorithm

And brightness modulation B^cAfter the normalization dimension reduction operation is carried out, the normalized phase P and the normalized reflectivity are respectively and correspondingly obtained;

and (VII) generating a reflectivity image and a normalized phase P image of the recognition result of the recognized object through a processing unit, and then classifying and recognizing the human face features in the generated image by using a nearest neighbor classifier.

In the method, the normalization dimension reduction operation comprises the operation of formula (4),

(4) in the formula, P (x)^c，y^c) For the purpose of the normalized phase data,

is the original phase data of the phase data,

is the minimum value in the original phase map,

is the maximum value in the original phase map.

In the method, in the step (VII), before the recognition, each image to be recognized is projected into the characteristic face subspace, and then the nearest neighbor classifier is used for face recognition.

Compared with the prior art, the invention has the beneficial effects that:

the method utilizes the phase information and reflectivity calculated by the phase measurement profilometry and uses the principal component analysis method and the nearest neighbor classifier to carry out face recognition, reduces the calculation amount of calibration data and three-dimensional data and has less calculation amount and better recognition effect than three-dimensional depth face recognition.

And respectively calculating a phase image and a reflectivity image by using a PMP algorithm, then reducing the dimension by using a PCA algorithm, and finally performing face recognition by using a nearest neighbor classifier. The phase diagram and the reflectivity diagram are less influenced by ambient light, so that a better recognition effect than that of the two-dimensional gray level image face recognition is obtained. In addition, the method does not need to obtain three-dimensional data, so that calibration data and the three-dimensional data do not need to be calculated, and a better recognition effect than that of the three-dimensional depth map face recognition can be obtained.

Drawings

FIG. 1 is a schematic structural view of the present invention;

fig. 2 is a comparison graph of the recognition rates of the four methods.

Detailed Description

All of the features disclosed in this specification, or all of the steps in any method or process so disclosed, may be combined in any combination, except combinations of features and/or steps that are mutually exclusive.

The invention is further described below with reference to the accompanying drawings:

example 1

In order to further increase the universality and reduce the cost, the projection unit can be a projector, and the processing unit can be a computer or an embedded single chip microcomputer.

Phase profilometry is a structured light three-dimensional measurement technique that uses a sinusoidal projection pattern with phase shift to calculate phase information, which is combined with calibration data to calculate three-dimensional data. PMP rationale is as follows:

firstly, a projector projects a group of N frames of sinusoidal patterns containing multiple phase shifts to the surface of a measured object, and an image I projected by the projector^pCan be expressed as:

in the formula (x)^p，y^p) Representing projector coordinates; a is a^pAnd B^pIs a fixed parameter of the projector, A, for ensuring the gray value of the pattern is 0-255 ″^pAnd B^pThe value is generally 127.5; n is the phase shift coefficient (N ═ 1,2, …, N); f is the spatial frequency of the projected pattern.

When the projector projects the sine pattern, the camera synchronously captures each frame of sine pattern modulated by the object surface, and calculates the captured image to obtain the phase

And brightness modulation B^cThe calculation formula of the phase and brightness modulation is as follows:

in the formula I^cIs the light intensity distribution of the image captured by the image sensing unit, (x)^c,y^c) Representing the image sensing unit coordinates. Due to brightness modulation B^cIs proportional to the reflectivity and thus becomes reflectivity data after normalization. At present, the phase information and the reflectivity information are directly used as the face data, and calibration data and three-dimensional data do not need to be acquired and calculated, so the process of calculating three-dimensional coordinates subsequently by the PMP is not repeated here.

Example 2

An M × N face image is set, and each column is sequentially connected to form a column vector with a size of D being M × N, where D is the dimension of the data space. Let n be the number of training samples, X_iAnd (3) representing a column vector corresponding to the ith human face image, wherein u is an average human face image column vector of the training sample, and the calculation formula is as follows:

let A be [ X ═ X₁-u,X₂-u,…,X_n-u]Then the covariance matrix Sr of the sample is AAT, whose dimension is D × D. According to the Karhunen-Loeve orthogonal transformation principle, the eigenface space is composed of orthogonal eigenvectors of the covariance matrix Sr, but since the dimensionality is too high and the calculation amount is too large, it is very difficult to directly solve the eigenvalues and eigenvectors. Therefore, it is necessary to reduce the amount of calculation, and therefore, a Singular Value Decomposition (SVD) method is used. According to the singular value decomposition theorem, the characteristic value and the characteristic vector of ATA can be obtained by solvingAnd obtaining the characteristic value and the characteristic vector of Sr. Let k_jIs r non-zero characteristic values, v, of the ATA matrix_jIs ATA corresponding to k_jThe feature vector of (A) is then the ATA orthonormal feature vector w_jComprises the following steps:

w_jthe singular value decomposition principle is to calculate the eigenvalue and eigenvector of AAT indirectly by calculating the eigenvalue and eigenvector of an n × n low-dimensional matrix (n is much smaller than D) ATA.

Then, k is added_jArranged from big to small, the corresponding feature vector is w_j. In order to further reduce the dimension, only the eigenvectors corresponding to the maximum d eigenvalues in the front can be selected as the subspace, and thus the eigenface subspace is obtained as: w ═ W₁,w₂,…,w_d). Finally, the training sample is projected to the characteristic face subspace, a group of projection vectors are obtained through calculation to form a face database, and the projection vectors Q_iIs calculated as follows:

Q_i＝W^T(X_i-u)(i＝1,2,…,n)，

in the identification process, each face image to be identified is projected into a characteristic face subspace, and then a nearest neighbor classifier is used for face identification.

The face phase map and the reflectivity map calculated by PMP are used. Before identification, the phase map data needs to be normalized. The specific normalization procedure is shown below:

in the formula, P (x)^c，y^c) For the purpose of the normalized phase data,

is the number of original phaseAccording to the above-mentioned technical scheme,

is the minimum value in the original phase map,

is the maximum value in the original phase map. This allows the phase information values to be normalized to the range 0 to 255, and the depth map and the gray map are similarly normalized.

The above-mentioned embodiments only express the specific embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the spirit of the invention, which falls within the scope of the invention.

Claims

1. the face recognition method based on the phase of phase profilometry and reflectivity, is characterized in that, step is as follows:

(I) The ^sinusoidal grating template is adjusted N times by the phase shifter, the projection unit transmits a group of N frames of sinusoidal patterns to the surface of the recognized object, and the image light intensity distribution IP transmitted by the projection unit is:

In formula (1), (x ^P , y ^P ) represent the coordinates of the projection unit, A ^P and ^BP are fixed parameters about the projection unit, n is the phase shift coefficient (n=1, 2, . . . , N), and f is the the spatial frequency of the projection unit transmission sinusoidal pattern;

(II) While the projection unit projects the sinusoidal pattern, the image sensing unit simultaneously captures the sinusoidal pattern of the corresponding frame modulated by the surface of the recognized object to obtain the illumination signal;

(III) The processing unit ^obtains the light intensity distribution ^IC at the image sensing unit according to the mapping relationship between the light intensity distribution IP of the transmitted image of the projection unit and the light intensity distribution at the position of the image sensing unit;

(V) According to the light intensity distribution I ^C at the image sensing unit, the processing unit calculates the feedback illumination signal and obtains the phase φ and the brightness modulation B ^C . The calculation formula is as follows:

In formulas (2) and (3), I ^C is the image light intensity distribution captured by the image sensing unit, and in formula (2), (x ^c , y ^c ) represents the coordinates of the image sensing unit;

(Ⅵ) Since the brightness modulation B ^C is proportional to the reflectivity, according to the principal component analysis algorithm, the processing unit performs a normalized dimension reduction operation on the phase φ and the brightness modulation B ^C , and obtains the normalized phase P correspondingly. and reflectivity;

(VII) Generate a reflectance image and a normalized phase P image about the recognition result of the recognized object by the processing unit, and then use the nearest neighbor classifier to classify and recognize the facial features in the generated image.

2. the face recognition method based on the phase of phase profilometry and reflectivity according to claim 1, is characterized in that, described normalized dimension reduction operation comprises carrying out (4) formula operation,

In formula (4), P(x ^c , y ^c ) is the normalized phase data, φ(x ^c , y ^c ) is the original phase data, φ _min is the minimum value in the original phase diagram, and φ _min is The maximum value in the original phase diagram.

3. the face recognition method based on the phase of phase profilometry and reflectivity according to claim 1, is characterized in that, described step (VII), before identifying, first each image to be recognized is projected to feature In the face subspace, the nearest neighbor classifier is used for face recognition.

4. the face recognition method based on the phase and reflectivity of phase profilometry according to claim 1, is characterized in that, also comprises following device:

Projection unit: which includes a light source capable of projecting patterns, a first lens, a sinusoidal grating template, a phase shifter and a second lens;

The first lens: used for condensing the output light of the light source;

The sinusoidal grating template: receives the transmitted light of the first lens and transmits the parallel light with the first sinusoidal phase;

The phase shifter: used to adjust the sinusoidal grating template and make it transmit parallel light with a second sinusoidal phase;

The second lens: receiving the sine grating template and transmitting the parallel light of the first sine phase or the parallel light of the second sine phase and focusing and projecting it to the surface of the recognized object;

Further includes a processing unit: used for calculation and control, to control the phase shifter and the light source;

It also includes an image sensing unit: receiving the surface modulated light of the recognized object and feeding back the illumination signal to the processing unit.

5. The face recognition method based on the phase and reflectivity of phase profilometry according to claim 4, wherein the image sensing unit comprises a charge-coupled array device, a filter circuit, an integrator circuit connected in sequence , video processing module and amplifying output circuit.