CN106169075A - Auth method and device - Google Patents

Abstract

The disclosure provides an identity verification method and device, which belong to the technical field of face recognition. The method includes: acquiring at least two face images collected by a camera in real time; performing image processing on each of the collected face images to obtain a plurality of facial elements and face features in each face image; according to the Multiple facial elements in each face image, determine whether there is a designated image in the at least two face images; when there are at least a preset number of the designated images in the at least two face images, verify pass. This disclosure uses a verification model containing multiple loss functions to process multiple face images collected in real time, which can not only output the location information of facial feature points, but also extract multiple facial elements at the same time, avoiding the use of different classifiers The problem of large amount of calculation caused by extracting different facial elements, in addition, can also ensure the accuracy of the extracted multiple facial elements and facial features.

Description

Identity verification method and device

technical field

The present disclosure relates to the technical field of face recognition, and in particular to an identity verification method and device.

Background technique

Face recognition technology refers to computer technology that uses the analysis and comparison of facial visual feature information for identity identification. Face recognition in a broad sense actually includes a series of related technologies for building a face recognition system, including face image acquisition, face positioning, face recognition preprocessing, identity verification, and identity search; A technology or system for face authentication or identity lookup. Specifically, face recognition technology is based on human facial features. For an input face image or video stream, first judge whether there is a human face in the image or video stream, and if there is a human face, then further detect each face. Position, size, and location information of each major facial organ in each face. Based on this information, the identity features corresponding to each face are further extracted, and compared with known faces, so as to identify the identity corresponding to each face.

With the maturity of face recognition technology, there are more and more application scenarios of this technology, such as access control system and online identity verification. Verification can ensure that only legitimate users can perform corresponding online operations. Identity verification through face recognition technology has the characteristics of high security, which can ensure the safety of user information and property.

Contents of the invention

In order to overcome the problems existing in related technologies, the present disclosure provides an identity verification method and device.

According to a first aspect of an embodiment of the present disclosure, an identity verification method is provided, including:

Obtain at least two face images captured by the camera in real time;

Image processing is performed on each face image collected to obtain a plurality of facial elements and face features in each face image;

According to the plurality of facial elements in each of the facial images, determine whether there is a specified image in the at least two facial images, and the specified image is a specified facial element in the plurality of facial elements that meets a preset standard face image;

When the specified image exists in the at least two face images, the verification is passed.

By using at least two face images collected in real time by a verification model containing multiple loss functions for processing, not only can output the location information of facial feature points, but also extract multiple facial elements at the same time, avoiding the use of different classifiers The problem of large amount of calculation caused by extracting different facial elements. In addition, since the model to be trained corresponding to the verification model contains multiple loss functions, the verification model obtained by training can have higher precision and accuracy, thereby ensuring The accuracy of the extracted multiple facial elements and facial features.

In a first possible implementation of the first aspect of the present disclosure, image processing is performed on each collected face image to obtain multiple facial elements and face feature information in each face image, including :

Each face image is input into a verification model to obtain a plurality of face elements and face features of each face image, and the verification model is used to extract face elements in the face image and analyze face features, The verification model is composed of a convolutional neural network and multiple loss functions.

By using a verification model containing multiple loss functions to process at least two face images collected, not only can output the location information of facial feature points, but also extract multiple facial elements at the same time, avoiding the use of different classifiers The problem of large amount of calculation caused by extracting different facial elements.

In a second possible implementation of the first aspect of the present disclosure, after detecting whether a specified image exists in the at least two face images according to the plurality of facial elements in each face image, the method Also includes:

Acquiring the number of images of the specified image in the at least two face images;

Detecting whether the number of images is greater than a preset number;

When the number of images is greater than the preset number, the verification is passed;

When the number of images is not greater than the preset number, the verification fails.

In a third possible implementation manner of the first aspect of the present disclosure, the plurality of facial elements at least include eye state information, mouth state information, head state information, and nose state information.

By acquiring multiple facial elements, the problem of inaccurate or low verification results caused by using a single facial element can be avoided, and at the same time, the applicable scenarios of the identity verification method provided by the present disclosure can be improved.

In a fourth possible implementation manner of the first aspect of the present disclosure, the plurality of loss functions include at least a first loss function for detecting the state of the eyes, a second loss function for detecting the state of the mouth, and a second loss function for detecting the state of the mouth. A third loss function to detect the state of the head.

By using different loss functions to detect the state of different parts of the face, it is possible to avoid the problem of using different classifiers for classification to obtain the state of different parts of the face, resulting in a large amount of calculation; and it is also possible to obtain the state information of multiple parts of the face at the same time , that is, get multiple facial elements at the same time.

In a fifth possible implementation manner of the first aspect of the present disclosure, the first loss function and the second loss function are binary classification functions, and the third loss function is a three-classification function.

Different classification results are obtained through different loss functions, and different parts of the face can be detected in a targeted manner.

In a sixth possible implementation manner of the first aspect of the present disclosure, when at least a preset number of the specified images exist in the at least two face images, the verification is passed, including:

When there are at least a preset number of face images with eyes open in the at least two face images, the verification is passed; or,

When there are at least a preset number of face images in a mouth-opening state in the at least two face images, the verification is passed; or,

When there are at least a preset number of face images whose heads are turned left or right in the at least two face images, the verification is passed.

Liveness verification is realized by detecting whether the designated image exists in the at least two face images, and when the designated image exists in the at least two face images, it can be quickly determined that the face is a live face, and then the verification is passed, so that The terminal performs a corresponding operation or displays a corresponding operation interface according to the verification result.

In a seventh possible implementation of the first aspect of the present disclosure, the plurality of facial elements and facial features are represented by a 197-dimensional vector, and the 197-dimensional vector includes two-dimensional coordinates of 95 facial feature points, two-dimensional Eye state information, 2D mouth state information and 3D head state information.

By using a multi-dimensional vector to output the multiple facial elements and facial features, the purpose of obtaining multiple facial elements and facial features at the same time can be achieved, and the neatness and clarity of the results can be improved, and the confusion of multiple results can be avoided.

According to a second aspect of an embodiment of the present disclosure, an identity verification device is provided, the device comprising:

An image acquisition module, configured to acquire at least two face images collected by the camera in real time;

An image processing module, configured to perform image processing on each face image collected to obtain multiple facial elements and face features in each face image;

A determining module, configured to determine whether there is a designated image in the at least two face images according to the plurality of facial elements in each of the facial images, and the designated image is a designated facial element in the plurality of facial elements Face images that meet preset standards;

A verification module, configured to pass the verification when the specified image exists in the at least two face images.

By using at least two face images collected in real time by a verification model containing multiple loss functions for processing, not only can output the location information of facial feature points, but also extract multiple facial elements at the same time, avoiding the use of different classifiers The problem of large amount of calculation caused by extracting different facial elements. In addition, since the model to be trained corresponding to the verification model contains multiple loss functions, the verification model obtained by training can have higher precision and accuracy, thereby ensuring The accuracy of the extracted multiple facial elements and facial features.

In a first possible implementation manner of the second aspect of the present disclosure, the image processing module is configured to:

Each face image is input into a verification model to obtain a plurality of face elements and face features of each face image, and the verification model is used to extract face elements in the face image and analyze face features, The verification model is composed of a convolutional neural network and multiple loss functions.

By using a verification model containing multiple loss functions to process at least two face images collected, not only can output the location information of facial feature points, but also extract multiple facial elements at the same time, avoiding the use of different classifiers The problem of large amount of calculation caused by extracting different facial elements.

In a second possible implementation manner of the second aspect of the present disclosure, the device further includes:

An image quantity acquisition module, configured to acquire the image quantity of the specified image in the at least two face images;

The detection module is also used to detect whether the number of images is greater than a preset number;

The verification module is further configured to pass the verification when the number of images is greater than the preset number; fail the verification when the number of images is not greater than the preset number.

In a third possible implementation manner of the second aspect of the present disclosure, the plurality of facial elements at least include eye state information, mouth state information, head state information, and nose state information.

By acquiring multiple facial elements, the problem of inaccurate or low verification results caused by using a single facial element can be avoided, and at the same time, the applicable scenarios of the identity verification method provided by the present disclosure can be improved.

In a fourth possible implementation manner of the second aspect of the present disclosure, the multiple loss functions include at least a first loss function for detecting the state of the eyes, a second loss function for detecting the state of the mouth, and a second loss function for detecting the state of the mouth. A third loss function to detect the state of the head.

By using different loss functions to detect the state of different parts of the face, it is possible to avoid the problem of using different classifiers for classification to obtain the state of different parts of the face, resulting in a large amount of calculation; and it is also possible to obtain the state information of multiple parts of the face at the same time , that is, get multiple facial elements at the same time.

In a fifth possible implementation manner of the second aspect of the present disclosure, the first loss function and the second loss function are binary classification functions, and the third loss function is a three-classification function.

Different classification results are obtained through different loss functions, and different parts of the face can be detected in a targeted manner.

In a sixth possible implementation manner of the second aspect of the present disclosure, the verification module is configured to:

When there are at least a preset number of face images with eyes open in the at least two face images, the verification is passed; or,

When there are at least a preset number of face images in a mouth-opening state in the at least two face images, the verification is passed; or,

When there are at least a preset number of face images whose heads are turned left or right in the at least two face images, the verification is passed.

Liveness verification is realized by detecting whether there is a preset number of the designated images in the at least two face images, and when the designated image exists in the at least two face images, it can be quickly determined that the face is a live face, and then If the verification is passed, the terminal executes a corresponding operation or displays a corresponding operation interface according to the verification result.

In a seventh possible implementation of the second aspect of the present disclosure, the plurality of facial elements and facial features are represented by a 197-dimensional vector, and the 197-dimensional vector includes two-dimensional coordinates of 95 facial feature points, two-dimensional Eye state information, 2D mouth state information and 3D head state information.

By using a multi-dimensional vector to output the multiple facial elements and facial features, the purpose of obtaining multiple facial elements and facial features at the same time can be achieved, and the neatness and clarity of the results can be improved, and the confusion of multiple results can be avoided.

In a third aspect, an identity verification device is also provided, including:

processor;

memory for storing processor-executable instructions;

Among them, the processor is configured as:

Obtain at least two face images captured by the camera in real time;

Image processing is performed on each face image collected to obtain a plurality of facial elements and face features in each face image;

According to the plurality of facial elements in each of the facial images, determine whether there is a specified image in the at least two facial images, and the specified image is a specified facial element in the plurality of facial elements that meets a preset standard face image;

When the specified image exists in the at least two face images, the verification is passed.

The beneficial effects brought by the technical solutions provided by the embodiments of the present disclosure are:

This disclosure processes at least two face images collected in real time by using a verification model containing multiple loss functions, not only can output the location information of face feature points, but also can extract multiple facial elements at the same time, avoiding the use of different The problem of large amount of calculation caused by the extraction of different facial elements by the classifier. In addition, since the model to be trained corresponding to the verification model contains multiple loss functions, the verification model obtained by training can have higher precision and accuracy, and then The accuracy of the extracted multiple facial elements and facial features can be guaranteed.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

Fig. 1 is a flowchart of an identity verification method shown according to an exemplary embodiment;

Fig. 2A is a flowchart of an identity verification method according to an exemplary embodiment;

FIG. 2B is a CNN (Convolutional Neural Networks, Convolutional Neural Networks) network design diagram shown according to an exemplary embodiment;

Fig. 2C is a schematic diagram of a CNN model to be trained according to an exemplary embodiment;

Fig. 3 is a block diagram of an identity verification device according to an exemplary embodiment;

Fig. 4 is a block diagram of an identity verification device 400 according to an exemplary embodiment.

detailed description

In order to make the purpose, technical solution and advantages of the present disclosure clearer, the implementation manners of the present disclosure will be further described in detail below in conjunction with the accompanying drawings.

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

Fig. 1 is a flowchart showing an identity verification method according to an exemplary embodiment. As shown in Fig. 1 , the identity verification method is used in a terminal and includes the following steps.

In step 101, at least two face images collected by a camera in real time are obtained.

In step 102, image processing is performed on each collected face image to obtain a plurality of facial elements and face features in each face image.

In step 103, according to the plurality of facial elements in each of the human face images, it is detected whether there is a specified image in the at least two human face images, and the specified image is a specified facial element in the plurality of facial elements Face images that meet preset standards.

In step 104, when at least a preset number of the specified images exist in the at least two face images, the verification is passed.

The method provided by the embodiment of the present disclosure processes at least two face images collected in real time by using a verification model containing multiple loss functions, not only can output the location information of facial feature points, but also can extract multiple facial elements at the same time , which can avoid the problem of large amount of calculation caused by using different classifiers to extract different facial elements. In addition, since the model to be trained corresponding to the verification model contains multiple loss functions, the verification model obtained by training can have a higher Accuracy and accuracy, and thus can ensure the accuracy of the extracted multiple facial elements and facial features.

In a first possible implementation of the present disclosure, image processing is performed on each of the collected face images to obtain multiple facial elements and face feature information in each of the face images, including:

Each face image is input into a verification model to obtain a plurality of face elements and face features of each face image, and the verification model is used to extract face elements in the face image and analyze face features, The verification model is composed of a convolutional neural network and multiple loss functions.

In a second possible implementation of the present disclosure, after detecting whether a specified image exists in the at least two face images according to the plurality of facial elements in each face image, the method further includes:

Acquiring the number of images of the specified image in the at least two face images;

Detecting whether the number of images is greater than a preset number;

When the number of images is greater than the preset number, the verification is passed;

When the number of images is not greater than the preset number, the verification fails.

In a third possible implementation manner of the present disclosure, the plurality of facial elements at least include eye state information, mouth state information, head state information, and nose state information. .

In a fourth possible implementation manner of the present disclosure, the multiple loss functions include at least a first loss function for detecting the state of the eyes, a second loss function for detecting the state of the mouth, and a second loss function for detecting the state of the head. The third loss function of .

In a fifth possible implementation manner of the present disclosure, the first loss function and the second loss function are binary classification functions, and the third loss function is a three-classification function.

In a sixth possible implementation manner of the present disclosure, when at least a preset number of the specified images exist in the at least two face images, the verification is passed, including:

When there are at least a preset number of face images with eyes open in the at least two face images, the verification is passed; or,

When there are at least a preset number of face images in a mouth-opening state in the at least two face images, the verification is passed; or,

When there are at least a preset number of face images whose heads are turned left or right in the at least two face images, the verification is passed.

In the seventh possible implementation of the present disclosure, the multiple facial elements and facial features are represented by a 197-dimensional vector, and the 197-dimensional vector includes two-dimensional coordinates of 95 facial feature points and two-dimensional eye state information , two-dimensional mouth state information and three-dimensional head state information.

All the above optional technical solutions may be combined in any way to form optional embodiments of the present disclosure, which will not be repeated here.

Fig. 2 is a flowchart of an identity verification method according to an exemplary embodiment. The execution subject of this embodiment may be a terminal. Referring to FIG. 2, this embodiment specifically includes:

In step 201, at least two face images collected by a camera in real time are acquired.

When the terminal detects that the user is performing or is about to perform a sensitive operation, the camera is automatically turned on, and the camera collects the face image in real time to achieve the purpose of verifying the user's identity; the sensitive operation can include online payment, online transfer And user information modification and other operations. For example, when the terminal detects the user's trigger operation on the payment option, it automatically turns on the camera and reminds the user to aim the camera at the face area, so that the camera can completely capture the entire area of the face.

It should be noted that the identity verification method provided by the embodiment of the present disclosure is used to perform live body verification on the face. Therefore, when the camera is turned on, the terminal can also remind the user to make different expressions, so that the terminal can perform different facial expressions when the user makes different facial expressions. Image acquisition is performed during the expression process, and then it is verified whether the collected face is a living body. The way the terminal reminds the user can be a voice reminder, for example, the voice plays "Please blink", "Please open your mouth" or "Please shake your head", etc., or the user can be reminded by displaying text on the terminal screen. The user may be reminded in other ways, which are not specifically limited in the embodiments of the present disclosure.

By collecting at least two face images in real time through the camera, it is possible to perform image processing on the at least two collected face images respectively, so as to achieve the purpose of living body verification.

In step 202, image processing is performed on each collected face image to obtain a plurality of facial elements and face features in each face image.

The plurality of facial elements refers to the status information of the five sense organs of the human face, for example, the eyes are in the closed state, the mouth is in the open state, etc.; the facial feature refers to the feature point position information of the human face. In the embodiment of the present disclosure, the plurality of facial elements at least include eye state information, mouth state information, head state information, nose state information, and may also include other information such as nasolabial fold information; the facial features may be The location information of 95 feature points, the location information is expressed in the form of coordinates.

The specific method for performing image processing on each face image in the at least two face images may be: input the at least two face images into the verification model to obtain each face in the at least two face images Multiple facial elements and facial features of the image. The verification model is used to extract facial elements in the face image and analyze facial features. The verification model is composed of a convolutional neural network and multiple loss functions. Specifically, for each face image in the at least two face images, the face position is first determined from the face image by a face detection algorithm, and the face feature point position in the face image is determined by CNN , and obtain the position information of the face feature points, and input the face features output by the deep learning algorithm into the multiple loss functions to obtain multiple facial elements in the face image.

Wherein, the face detection algorithm may be an Adaboost algorithm, that is, the number of faces in each image is obtained through an Adaboost classifier. Specifically, the Adaboost classifier divides any image to obtain multiple regions of the image, and for each region obtained by division, extracts the features of the region according to the preset feature extraction algorithm, and inputs the features of the region to The classifier calculates the features of the region based on the classifier, and obtains an output result of the classifier, and then obtains a classification result of the region, and the classification result is a face region or a non-face region.

The Adaboost classifier is composed of multiple weak classifiers, and the multiple weak classifiers are trained based on the same training sample set. For example, weak features of multiple sample images, such as rectangular features, can be obtained, and the weak features of each sample image can be used as training samples, and multiple training samples can be used to form a training sample set. Select a number of training samples from the training sample set to form the first training set. According to the first training set, train the first weak classifier, and then select a number of new training samples from the training sample set. The selected training samples and the training samples misclassified by the first weak classifier constitute the second training set. According to the second training set, the second weak classifier is trained, and then several new training samples are selected from the training sample set. Samples, the training samples selected this time and the training samples of the first weak classifier and the second weak classifier are equally divided to form the third training set, and according to the third training set, the third weak classifier is trained , and so on, until the error rate is less than the preset minimum error rate, the trained multiple weak classifiers are combined into a strong classifier, which can be used to classify images.

As a kind of artificial neural network, CNN has become a research hotspot in the field of image recognition. CNN is a multi-layer perceptron specially designed to recognize two-dimensional shapes. This network structure is highly invariant to translation, tilt or other forms of deformation. CNN's weight sharing network structure makes it more similar to biological neural networks. By learning a large number of mapping relationships between input and output, it can output processing results without any precise mathematical expressions between input and output, thus effectively It greatly reduces the complexity of the network model and reduces the number of weights. Especially when the network input is a multi-dimensional image, the advantages of CNN are more obvious, avoiding the complex feature extraction and data reconstruction process in traditional image recognition algorithms.

Fig. 2B is a design diagram of a CNN network according to an exemplary embodiment. As can be seen from Figure 2B, the CNN network consists of an input layer and 7 training layers, and the 7 training layers are C1 layer, S2 layer, C3 layer, S4 layer, C5 layer, F6 layer and output layer, among them, C1, Layers C3 and C5 are convolution layers, which are used to enhance the characteristics of the original image and reduce noise through convolution operations; S2 and S4 are down-sampling layers, which are used to sub-sample the image using the principle of local correlation of the image to reduce Data processing volume and retain valid features. Wherein, the image size input by the input layer is 32*32. Each training layer has a plurality of feature images, each feature image is a feature extracted from an input through a convolution filter, and each feature image has a plurality of neurons. In addition, each training layer contains multiple parameters to be trained.

Among them, the C1 layer is the first-level convolutional layer, which is composed of 6 feature images with a size of 28*28. Each neuron in each feature image is connected to a 5×5 neighborhood in the input image. Each filter in the C1 layer has 5*5=25 filter parameters and 1 bias parameter, and the 6 filters have (5*5+1)*6=156 parameters to be trained in total. For 156 parameters to be trained, there are 156*(28*28)=122304 connections in total.

The S2 layer is the first-level downsampling layer, consisting of six 14*14 feature images. Each unit in each feature image is connected to a 2×2 neighborhood of the corresponding feature image in layer C1. In addition, the S2 layer has 12 parameters to be trained and 5880 connections.

The C3 layer is the second-level convolutional layer, which consists of 16 feature images of 10*10. Among them, the feature image of 10*10 is obtained by convoluting the downsampling layer S2 with a convolution kernel of 5*5. Each feature image in the C3 layer is connected to all or part of the feature images in the S2 layer, that is, the feature image in the C3 layer is a combination of the feature images extracted in the S2 layer.

The S4 layer is the second downsampling layer, which consists of 16 5*5 feature images. Each unit in each feature map is connected to a 2×2 neighborhood of the corresponding feature image in layer C3. In addition, the S4 layer has 32 parameters to be trained and 2000 connections.

Layer C5 is the second-level convolutional layer, consisting of 120 feature images. Each unit in each feature image is connected to a 5*5 neighborhood of 16 feature images in the S4 layer. Since the size of the feature image of the S4 layer is also 5*5, the ratio of the feature image of the C5 layer to the feature image of the S4 layer should be 1*1, that is, there is a full connection between the S4 layer and the C5 layer. In addition, the C5 layer has 48120 connections to be trained.

The F6 layer consists of 84 feature images and is fully connected with the C5 layer. In addition, the F6 layer has 10164 parameters to be trained.

The output layer is composed of RBF (Radial Basis Function, Radial Basis Function) units. Each RBF unit is used to calculate the Euclidean distance between the input vector and the output vector. The greater the Euclidean distance between the input vector and the output vector, the RBF The output of the unit is also larger. The output of the RBF unit is used as a penalty term that measures how well the input vector matches a model of the class associated with the RBF unit. Probabilistically, the output of the RBF unit can be considered as the negative log-likelihood of the Gaussian distribution of the configuration space of the F6 layer. Given any input vector, the loss function should make the configuration of the F6 layer close enough to the RBF output vector.

In another embodiment of the present disclosure, the multiple loss functions include at least a first loss function for detecting the state of the eyes, a second loss function for detecting the state of the mouth, and a third loss for detecting the state of the head functions; wherein, the first loss function, the second loss function and the third loss function may be Softmax functions or other functions with multi-classification functions, which are not specifically limited in the embodiments of the present disclosure.

In the embodiment of the present disclosure, the first loss function and the second loss function are two-category functions, and the third loss function is three-category functions. Specifically, the output of the first loss function for detecting the eye state includes the probability of being in the eye-open state and the probability of being in the eye-closed state, and the output of the second loss function for detecting the mouth state includes the probability of being in the mouth-open state and the probability of being in the closed state, the output of the third loss function used to detect the head state includes the probability that the face is facing the camera, the probability that the head is turned left, and the probability that the head is turned right.

Of course, the first loss function, the second loss function and the third loss function can also obtain two or more classification results, for example, the output of the first classification function used to detect the state of the mouth can include The probability of the eye state, the probability of the half-open state, and the probability of the open state. In addition, the first loss function, the second loss function, and the third loss function can also be used to extract other facial elements, and the embodiment of the present disclosure does not make any feature extraction targets and detection results of the above three loss functions. Specific limits.

It should be noted that the multiple loss functions also include a fourth loss function for regressing the coordinates of the 95 feature points, and the fourth loss function is used to make the coordinates of the 95 feature points finally obtained and the calibrated 95 features The error between the coordinates of the points is the smallest, and the fourth loss function may be a Euclidean distance function.

Of course, the plurality of loss functions may also include loss functions for detecting other facial elements, for example, a fifth loss function and a sixth loss function for detecting the state of the nose and nasolabial folds, and the fifth loss function may be two Classification function, the output includes the probability that the nose is in a flat state and the probability that it is in a wrinkled state; the sixth loss function can be a multi-classification function with more than three classifications, and the output includes the probability that the nasolabial folds are at different depths, when the user laughs , the nasolabial folds are darker, and when the user is in a normal expressionless state, the nasolabial folds are lighter. The embodiments of the present disclosure do not limit the specific detection areas and output types of the multiple loss functions.

In yet another embodiment of the present disclosure, the plurality of facial elements and facial features are represented by a 197-dimensional vector, and the 197-dimensional vector includes two-dimensional coordinates of 95 facial feature points, two-dimensional eye state information, two-dimensional mouth State information and 3D head state information. Specifically, the first to 190th dimensions of the 197-dimensional vector represent the two-dimensional coordinates of the 95 facial feature points, the 191st to 192nd dimensions represent two-dimensional eye state information, and the 193rd to 194th dimensions represent For two-dimensional mouth state information, the 195th to 197th dimensions represent three-dimensional head state information; of course, the specific content corresponding to the 197-dimensional vector can be set by the developer, which is not specifically limited in the embodiments of the present disclosure.

It should be noted that, according to the different classification results corresponding to each of the multiple loss functions, the obtained vector dimensions corresponding to the multiple facial elements and human face features are also different.

In yet another embodiment of the present disclosure, the above-mentioned process is a process of using a verification model to perform image processing on a face image to obtain multiple facial elements and face features in the face image. Before realizing the above-mentioned process, it is necessary to The initial model composed of CNN and multiple loss functions is trained to obtain the verification model. It should be noted that the training process of the initial model is the process of determining the model parameters of the CNN in the initial model. The specific training method can be : You can first initialize an initial model parameter for the CNN model to be trained, use a large number of face images with different facial elements as training samples, divide the training samples into multiple groups, and for each group of training samples, input the group of training samples into the initial Model, compare the obtained result with the calibration result to obtain the error value, if the error value is greater than the preset threshold, adjust the model parameters of CNN through backpropagation until the obtained error value is less than or equal to the preset threshold; if the error If the value is less than or equal to the preset threshold, then input the next set of training samples to continue training until all training samples are trained to obtain a verification model. Wherein, each group of training samples can contain a preset number of face images, and the specific value of the preset number can be set by the developer according to needs, and the embodiment of the present disclosure does not specifically limit the value and setting method of the preset number .

Among them, the CNN model to be trained usually includes at least two levels of convolutional layers and at least one level of fully connected layers, and each level of convolutional layers includes multiple convolution kernels and multiple bias matrices, and each level of fully connected layers includes multiple weights Matrix and multiple bias vectors. Therefore, the obtained model parameters include the initial convolution kernel of each level of convolutional layer, the initial bias matrix of each level of convolutional layer, the initial weight matrix of the fully connected layer, and the initial weight matrix of the fully connected layer. Initial bias vector. The embodiments of the present disclosure do not specifically limit the number of convolutional layers and the number of fully connected layers included in the CNN model to be trained. During specific implementation, it can be set as required. For example, as shown in FIG. 2C , this figure shows a schematic diagram of a CNN model to be trained. The CNN model to be trained shown in FIG. 2C includes five convolutional layers and two fully connected layers.

Further, the embodiments of the present disclosure do not specifically limit the number of convolution kernels and offset matrices included in each level of convolutional layer, and the number of weight matrices and offset vectors included in each level of fully connected layer. In addition, the embodiments of the present disclosure also do not limit the dimensions of each convolution kernel and offset matrix, and the dimensions of each weight matrix and each offset vector. During specific implementation, the number and dimensions of convolution kernels and offset matrices included in each level of convolutional layer, and the number and dimensions of fully connected weight matrices and offset vectors at each level can be empirical values.

In combination with the above content, when obtaining the initial model parameters of the CNN model to be trained, a value can be randomly selected within the specified value range as the value of each element in the initial model parameters. For example, for each element in each initial convolution kernel, initial weight matrix, initial bias matrix, and initial bias vector, a random number can be taken in the interval [-r, r]. Here, r is a threshold for initializing model parameters, which may be an empirical value. For example, r can take 0.001.

In another embodiment of the present disclosure, the image size parameter input by the first layer of CNN in the verification model can be set to 64*64, and when the image to be trained or the image to be verified is input into the initial model or the verification model, the The image is resized to 64*64.

By setting the input image size parameter of the first layer of CNN in the verification model to 64*64, it can be avoided that when the parameter is 224*224, the verification model can only verify part of the image, or can only guarantee part of the image. The verification accuracy of the image can be improved, thereby improving the scope of application of the verification model.

By using a verification model containing multiple loss functions to process at least two face images obtained in real time in step 201, not only can output the position information of face feature points, but also extract multiple facial elements at the same time, which can avoid using Different classifiers extract different facial elements, resulting in a large amount of calculation; in addition, since the initial model corresponding to the verification model contains multiple loss functions, it can make the trained verification model have higher precision and accuracy, Furthermore, the accuracy of the extracted facial elements and facial features can be guaranteed.

In step 203, according to the plurality of face elements in each face image, it is determined whether there is a specified image in the at least two face images, and the specified image is a specified face element in the plurality of face elements that meets a preset standard face images. When the specified image exists in the at least two human face images, perform step 204; when the specified image does not exist in the at least two human face images, perform step 205.

The specified image can be determined according to the prompt information given to the user during the process of capturing the face image by the camera in real time in step 201; for example, when the prompt information prompts the user to perform an eye blinking action, the specified image can be a human face in a closed-eye state The image, that is, the specified facial element is the eye state information, and accordingly, the preset standard is that the eyes are closed; when the prompt information prompts the user to perform an action of opening the mouth, the specified image can be a face image in the state of opening the mouth, That is, the specified facial element is mouth state information, and correspondingly, the preset standard is that the mouth is in an open state; when the prompt information prompts the user to perform a head shaking action, the specified image can be a person whose head is turned left or right The face image, that is, the designated facial element is head state information, and correspondingly, the preset standard is that the head is turned left or right.

If the terminal does not play or display the prompt information, the specified image may be a face image of any of the multiple facial elements in a corresponding state, for example, when the obtained facial elements include eye state information, mouth state information , head state information and nose state information, the specified image can be any of the face images in the state of eyes closed or mouth open, the head in the state of turning left or right, or the state of the nose in a wrinkled state One kind of image, that is, the designated facial element can be any state information in eye state information, mouth state information, head state information or nose state information, and correspondingly, the preset standard is corresponding to the designated facial element any status information.

The preset standard may be determined according to the prompt information, or may be determined according to multiple extracted facial elements, which is not specifically limited in this embodiment of the present disclosure.

It should be noted that when the multiple facial elements and facial features are represented by a 197-dimensional vector, the sum of the two-dimensional values representing the eye state information in the 197-dimensional vector is 1, and the corresponding state with a higher probability is the The state of the eyes in the face image; for example, in the extraction result of the facial elements of any face image, when the two-dimensional value corresponding to the eye state is (0.7,0.3), and the position corresponding to 0.7 indicates that the eye is closed Probability, the position corresponding to 0.3 represents the probability of being in the eye-open state, then the human eyes in the face image are in the eye-closed state; the extraction results of other facial elements are the same.

In step 204, when the specified image exists in the at least two face images, the verification is passed.

When the specified image exists in the at least two face images, it means that the face is a live face, and the verification is passed, and the terminal continues to perform corresponding operations or display a corresponding operation interface.

Specifically, in the embodiment of the present disclosure, when the specified image exists in the at least two human face images, the verification is passed, including: when there is a human face image in an open-eyed state in the at least two human face images, The verification is passed; or, when there is a face image in the state of opening the mouth in the at least two face images, the verification is passed; or, when there is a person whose head is turned left or right in the at least two face images When the face image is displayed, the verification is passed.

In another embodiment of the present disclosure, when the designated image exists in the at least two face images, the number of images of the designated image in the at least two face images is obtained; and whether the number of images is greater than a preset number is detected ; When the number of images is greater than the preset number, the verification is passed; when the number of images is not greater than the preset number, the verification is not passed. Wherein, the preset number can be set to any value smaller than the number of the at least two face images, and the embodiment of the present disclosure does not limit the setting method and specific value of the preset number.

By passing the verification of the current user when the number of images of the specified image is greater than the preset number, it is possible to avoid the situation that a verification error occurs when an error occurs in the facial element extraction process, thereby improving the security of the verification method provided by the embodiment of the present disclosure. sex.

In step 205, when the specified image does not exist in the at least two face images, the verification fails.

When the specified image does not exist in the at least two face images, it means that the face is not a live face, and the verification fails, and the terminal continues to terminate the execution of the corresponding operation, and displays a prompt message of verification failure, or re-executes steps 201 and 201. Its subsequent steps are to avoid the inaccurate extraction of facial elements and facial features caused by errors in the image acquisition process.

In another embodiment of the present disclosure, when step 201 and subsequent steps are repeated for more than a preset number of times, the terminal or the current application is locked for a preset duration, so as to further protect user privacy and property security. Wherein, the preset number of times and the preset duration can be set by the developer, or can be set by the user according to the needs, which is not limited in the embodiment of the present disclosure; similarly, the specific value of the preset number of times in the embodiment of the present disclosure Neither the specific value nor the preset duration is limited.

The method provided by the embodiment of the present disclosure processes at least two face images collected in real time by using a verification model containing multiple loss functions, not only can output the location information of facial feature points, but also can extract multiple facial elements at the same time , which can avoid the problem of large amount of calculation caused by using different classifiers to extract different facial elements. In addition, since the model to be trained corresponding to the verification model contains multiple loss functions, the verification model obtained by training can have a higher Accuracy and accuracy, thereby ensuring the accuracy of the extracted multiple facial elements and facial features; further, by passing the verification of the current user when the number of images of the specified image is greater than the preset number, it is possible to avoid facial Errors in the element extraction process lead to verification errors, thereby improving the security of the verification method.

Fig. 3 is a block diagram of an identity verification device according to an exemplary embodiment. Referring to FIG. 3 , the device includes an image acquisition module 301 , an image processing module 302 , a determination module 303 and a verification module 304 .

An image acquisition module 301, configured to acquire at least two face images collected by the camera in real time;

An image processing module 302, configured to perform image processing on each face image collected to obtain a plurality of facial elements and face features in each face image;

A determining module 303, configured to determine whether there is a designated image in the at least two face images according to the plurality of facial elements in each of the facial images, and the designated image is a designated face in the plurality of facial elements Face images whose elements meet the preset standards;

The verification module 304 is configured to pass the verification when there are at least a preset number of the specified images in the at least two face images.

In the first possible implementation manner provided in the present disclosure, the image processing module 302 is configured to:

Each face image is input into a verification model to obtain a plurality of face elements and face features of each face image, and the verification model is used to extract face elements in the face image and analyze face features, The verification model is composed of a convolutional neural network and multiple loss functions.

In a second possible implementation manner provided in the present disclosure, the device further includes:

An image quantity acquisition module, configured to acquire the image quantity of the specified image in the at least two face images;

The determination module 303 is also used to determine whether the number of images is greater than a preset number;

The verification module 304 is further configured to pass the verification when the number of images is greater than the preset number; fail the verification when the number of images is not greater than the preset number.

In a third possible implementation manner provided in the present disclosure, the plurality of facial elements at least include eye state information, mouth state information, head state information, and nose state information.

In the fourth possible implementation manner provided by the present disclosure, the multiple loss functions include at least a first loss function for detecting the state of the eyes, a second loss function for detecting the state of the mouth, and a second loss function for detecting the state of the head The third loss function for the state.

In a fifth possible implementation manner provided in the present disclosure, the first loss function and the second loss function are binary classification functions, and the third loss function is a three-classification function.

In a sixth possible implementation manner provided in the present disclosure, the verification module 304 is configured to:

When there are at least a preset number of face images with eyes open in the at least two face images, the verification is passed; or,

When there are at least a preset number of face images in a mouth-opening state in the at least two face images, the verification is passed; or,

When there are at least a preset number of face images whose heads are turned left or right in the at least two face images, the verification is passed.

In the seventh possible implementation mode provided by the present disclosure, the multiple facial elements and facial features are represented by a 197-dimensional vector, and the 197-dimensional vector includes the two-dimensional coordinates of 95 facial feature points, the two-dimensional eye state information, two-dimensional mouth state information, and three-dimensional head state information.

Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

Fig. 4 is a block diagram of an identity verification device 400 according to an exemplary embodiment. For example, the apparatus 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

4, apparatus 400 may include one or more of the following components: processing component 402, memory 404, power supply component 406, multimedia component 408, audio component 410, input/output (I/O) interface 412, sensor component 414, and communication component 416 .

The processing component 402 generally controls the overall operations of the device 400, such as those associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 402 may include one or more modules that facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402 .

The memory 404 is configured to store various types of data to support operations at the device 400 . Examples of such data include instructions for any application or method operating on device 400, contact data, phonebook data, messages, pictures, videos, and the like. The memory 404 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

The power supply component 406 provides power to various components of the device 400 . Power components 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for device 400 .

The multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 408 includes a front camera and/or a rear camera. When the device 400 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 410 is configured to output and/or input audio signals. For example, the audio component 410 includes a microphone (MIC), which is configured to receive external audio signals when the device 400 is in operation modes, such as call mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 404 or sent via communication component 416 . In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

The I/O interface 412 provides an interface between the processing component 402 and a peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 414 includes one or more sensors for providing status assessments of various aspects of device 400 . For example, the sensor component 414 can detect the open/closed state of the device 400, the relative positioning of components, such as the display and keypad of the device 400, and the sensor component 414 can also detect a change in the position of the device 400 or a component of the device 400 , the presence or absence of user contact with the device 400 , the device 400 orientation or acceleration/deceleration and the temperature change of the device 400 . The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 416 is configured to facilitate wired or wireless communication between the apparatus 400 and other devices. The device 400 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, apparatus 400 may be programmed by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the authentication method described above.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as the memory 404 including instructions, which can be executed by the processor 420 of the device 400 to implement the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium, and when the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal can execute the above identity verification method.

Other embodiments of the present disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, and these modifications, uses or adaptations follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure . The specification and examples are to be considered exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It should be understood that the present disclosure is not limited to the precise constructions which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (15)

1. an auth method, it is characterised in that described method includes:

Obtain at least two facial images of photographic head Real-time Collection；

Every the facial image collected is carried out image procossing, obtain multiple facial elements in described every facial image and Face characteristic；

According to the multiple facial elements in described every facial image, determine described in whether at least two facial images exist refer to Determining image, described appointment image is to specify facial elements to meet the facial image of preset standard in the plurality of facial elements；

When described at least two facial images at least exist the described appointment image of predetermined number, it is verified.

Method the most according to claim 1, it is characterised in that every the facial image collected is carried out image procossing, Obtain the multiple facial elements in described every facial image and face characteristic information, including:

By described every facial image input validation model, the multiple facial elements and the face that obtain described every facial image are special Levying, described checking model is for extracting the facial elements in facial image and analyzing face characteristic, and described checking model is by convolution Neutral net and multiple loss function are constituted.

Method the most according to claim 1, it is characterised in that the plurality of facial elements at least includes that eye state is believed Breath, mouth status information, head state information, nose status information.

Method the most according to claim 2, it is characterised in that the plurality of loss function at least includes for detecting eye The first-loss function of state, for detecting the second loss function of mouth state and for detecting the 3rd loss of head state Function.

Method the most according to claim 4, it is characterised in that described first-loss function and described second loss function are Two classification functions, described 3rd loss function is three classification functions.

Method the most according to claim 1, it is characterised in that preset when at least existing in described at least two facial images During the described appointment image of quantity, it is verified, including:

When at least there is the facial image being in eyes-open state of predetermined number in described at least two facial images, checking is logical Cross；Or,

When at least there is the facial image being in the state of opening one's mouth of predetermined number in described at least two facial images, checking is logical Cross；Or,

When the head that at least there is predetermined number in described at least two facial images is in left-hand rotation or the face figure of right turn state During picture, it is verified.

Method the most according to claim 1, it is characterised in that the plurality of facial elements and face characteristic with 197 dimensions to Amount represents, described 197 dimensional vectors include the two-dimensional coordinate of 95 human face characteristic points, two dimension eye status information, two dimension mouth shape State information and three-dimensional head status information.

8. an authentication means, it is characterised in that described device includes:

Image collection module, for obtaining at least two facial images of photographic head Real-time Collection；

Image processing module, for every the facial image collected is carried out image procossing, obtains described every facial image In multiple facial elements and face characteristic；

Determine module, for according to the multiple facial elements in described every facial image, determine described at least two face figures Whether there is appointment image in Xiang, described appointment image is to specify facial elements to meet preset standard in the plurality of facial elements Facial image；

Authentication module, for when at least there is the described appointment image of predetermined number in described at least two facial images, tests Card passes through.

Device the most according to claim 8, it is characterised in that described image processing module is used for:

By described every facial image input validation model, the multiple facial elements and the face that obtain described every facial image are special Levying, described checking model is for extracting the facial elements in facial image and analyzing face characteristic, and described checking model is by convolution Neutral net and multiple loss function are constituted.

Device the most according to claim 8, it is characterised in that the plurality of facial elements at least includes that eye state is believed Breath, mouth status information, head state information, nose status information.

11. devices according to claim 9, it is characterised in that the plurality of loss function at least includes for detecting eye The first-loss function of portion's state, for detecting the second loss function of mouth state and for detecting the 3rd damage of head state Lose function.

12. devices according to claim 11, it is characterised in that described first-loss function and described second loss function Being two classification functions, described 3rd loss function is three classification functions.

13. devices according to claim 8, it is characterised in that described authentication module is used for:

When at least there is the facial image being in eyes-open state of predetermined number in described at least two facial images, checking is logical Cross；Or,

When at least there is the facial image being in the state of opening one's mouth of predetermined number in described at least two facial images, checking is logical Cross；Or,

When the head that at least there is predetermined number in described at least two facial images is in left-hand rotation or the face figure of right turn state During picture, it is verified.

14. devices according to claim 8, it is characterised in that the plurality of facial elements and face characteristic with 197 dimensions to Amount represents, described 197 dimensional vectors include the two-dimensional coordinate of 95 human face characteristic points, two dimension eye status information, two dimension mouth shape State information and three-dimensional head status information.

15. 1 kinds of authentication means, it is characterised in that including:

Processor；

For storing the memorizer of the executable instruction of processor；

Wherein, described processor is configured to:

Obtain at least two facial images of photographic head Real-time Collection；

Every the facial image collected is carried out image procossing, obtain multiple facial elements in described every facial image and Face characteristic；

According to the multiple facial elements in described every facial image, determine described in whether at least two facial images exist refer to Determining image, described appointment image is to specify facial elements to meet the facial image of preset standard in the plurality of facial elements；

When described at least two facial images at least exist the described appointment image of predetermined number, it is verified.