CN112084816A - A cross-domain facial recognition method - Google Patents

Abstract

The invention provides a cross-domain face recognition method, which comprises the following steps: preparing source sample data and target sample data; a fixed feature extractor E, which maximizes a domain discriminator G and discriminates the difference between the source sample data and the target sample data; a fixed domain discriminator G for minimizing the feature extractor E and discriminating the difference between the source sample data and the target sample data; and the maximization of the domain discriminator G and the minimization of the feature extractor E reach a balance point, and a feature value without inter-domain difference between the source sample data and the target sample data is extracted from the feature extractor E. On the basis of facenet, the feature extractor E adopts an increment-Resnet-V1 network structure, and the source classification loss function and the target classification loss function adopt triple loss functions, and then a domain confusion loss function is added, so that the problem of cross-domain facial recognition under the condition that the image to be recognized and the training sample image have different statistical distribution characteristics is solved, the facial features in the same domain are closer, and the cross-domain recognition accuracy is improved.

Description

A cross-domain facial recognition method

【Technical field】

The invention relates to the technical field of facial recognition, in particular to a cross-domain facial recognition method.

【Background technique】

Facial recognition is a biometric technology for identifying people based on facial feature information. A series of related technologies that use cameras or cameras to collect images or video streams containing faces, and automatically detect and track faces in the images, and then perform face recognition on the detected faces. .

Most of the existing face recognition algorithms can solve the problem of single-domain face recognition (that is, the image to be recognized and the training sample image have the same statistical distribution characteristics). For example, algorithms such as FaceNet and DeepId have achieved good results in single-domain face recognition tasks.

However, in practical application scenarios, such as the task of identifying key monitoring objects under surveillance cameras, the face photos extracted from the videos captured by the surveillance cameras are compared with the ID photos of a group of objects to determine whether it is one of them. The difficulty is that the image of the camera is very different from the ID card photo. The ID card photo is in the restricted domain and requires the cooperation of the object. It has strict requirements on resolution, angle, light, expression, etc., while the surveillance camera is an open domain. The resolution, angle, light, and expression of the captured face photos are not subject to any restrictions. Therefore, the statistical laws of the face photos captured by the camera and the ID card photos are very different, and can be defined as cross-domain Facial recognition problem. For cross-domain recognition, FaceNet, DeepId and other algorithms based on CNN (Convolutional Neural Network) cannot achieve good results.

[Content of the invention]

The technical problem to be solved by the present invention is to provide a cross-domain face recognition method, which can solve the problem of face cross-domain recognition under the condition that the image to be recognized and the training sample image have different statistical distribution characteristics, so that the facial features in the same domain are more accurate. close, improving the accuracy of cross-domain recognition.

In order to solve the above technical problems, an embodiment of the present invention provides a cross-domain facial recognition method, which is characterized by comprising:

Prepare source sample data and target sample data;

Fix the feature extractor E, maximize the domain discriminator G, and discriminate the difference between the source sample data and the target sample data;

Fix the domain discriminator G, minimize the feature extractor E, and discriminate the difference between the source sample data and the target sample data;

The domain discriminator G is maximized and the feature extractor E is minimized to reach a balance point, and feature values that have no inter-domain difference between the source sample data and the target sample data are extracted from the feature extractor E.

Preferably, preparing the source sample data and the target sample data includes:

The source sample data and the target sample data are respectively aligned with facial features.

Preferably, the domain discriminator G and feature extractor E share parameters for different domains.

Preferably, preparing the source sample data and the target sample data includes:

The source sample data and the target sample data are processed to increase the diversity of the sample data.

Preferably, the feature extractor E is fixed, the domain discriminator G is maximized, and before discriminating the difference between the source sample data and the target sample data, the method includes: determining a loss function.

Preferably, processing the source sample data and the target sample data includes: randomly cropping, flipping, adjusting brightness, contrast, hue, and saturation on the source sample data and the target sample data.

Preferably, the loss function includes a classification loss function and a domain confusion loss function.

Preferably, the loss function is a weighted sum of a classification loss function and a domain confusion loss function.

Preferably, total loss=source classification loss+target classification loss−KL divergence, where KL divergence is used to measure the difference between the feature probability distributions of the source sample data and the target sample data.

Preferably, the KL divergence is also called relative entropy, which is a measure between the feature probability distributions of the source sample data and the target sample data.

Compared with the prior art, the above technical solution has the following advantages: on the basis of the original technical solution of facenet, the Inception-Resnet-V1 network structure is adopted through the feature extractor E, and the source classification loss function and the target classification loss function adopt ternary On the basis of the group loss function triplet loss, the domain confusion loss function is added to solve the problem of face cross-domain recognition under the condition that the image to be recognized and the training sample image have different statistical distribution characteristics, so that the facial features in the same domain are closer. The false recognition rate is reduced and the cross-domain recognition accuracy is improved.

【Description of drawings】

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a flow chart of a cross-domain facial recognition method according to the present invention.

FIG. 2 is a flow chart of a preferred embodiment of a cross-domain facial recognition method according to the present invention.

FIG. 3 is a network architecture diagram of a cross-domain facial recognition method according to the present invention.

FIG. 4 is a schematic diagram of t-SNE after training 20 epochs in a cross-domain face recognition method of the present invention.

FIG. 5 is a schematic diagram of t-SNE after training 50 epochs in a cross-domain face recognition method of the present invention.

【Detailed ways】

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

FIG. 1 is a flow chart of a cross-domain facial recognition method according to the present invention. As shown in Figure 1, a cross-domain facial recognition method includes steps:

S11. Prepare source sample data and target sample data. The preparation of the source sample data and the target sample data includes: aligning the facial features of the source sample data and the target sample data respectively, processing the source sample data and the target sample data, and increasing the diversity of the sample data. The processing of the source sample data and the target sample data includes: randomly cropping, flipping, adjusting the brightness, contrast, hue and saturation of the source sample data and the target sample data.

S12. Fix the feature extractor E, and let the domain discriminator G maximize the difference between the source sample data and the target sample data.

S13, fix the domain discriminator G, and let the feature extractor E minimize the difference between the source sample data and the target sample data;

S14, the domain discriminator G is maximized and the feature extractor E is minimized to reach a balance point, and feature values with no inter-domain difference between the source sample data and the target sample data are extracted from the feature extractor E.

The domain discriminator G and feature extractor E share parameters for different domains.

During specific implementation, the feature extractor E is fixed, the domain discriminator G is maximized, and before discriminating the difference between the source sample data and the target sample data, the method includes: determining a loss function.

The loss functions include a classification loss function and a domain confusion loss function. The loss function is the weighted sum of the classification loss function and the domain confusion loss function. Total loss = source classification loss + target classification loss - KL divergence, where KL divergence is used to measure the difference between the feature probability distributions of source sample data and target sample data. The KL divergence, also called relative entropy, is a measure between the feature probability distributions of the source sample data and the target sample data.

FIG. 2 is a flow chart of a preferred embodiment of a cross-domain facial recognition method according to the present invention. A cross-domain facial recognition method, comprising steps:

Step 21: Prepare the data. Prepare the source domain, which is the ID card photo, and the target domian, which is the camera photo, and align the photo with the face. The faces can be aligned using the MTCNN algorithm.

Step 22: Process the data. Randomly crop, flip, and adjust brightness, contrast, hue, saturation, etc. for photos in two different domains to increase the diversity of samples.

Step 23: Determine the network structure. FIG. 3 is a network architecture diagram of a cross-domain facial recognition method according to the present invention. The source sample data input and the target sample data input are the same specified pixel image, such as but not limited to an image of 160 pixels × 160 pixels, the feature extractor E is a multi-layer convolutional neural network, and the domain discriminator G is a multi-layer fully connected neural network. network. Feature extractor E and domain discriminator G share parameters for different domains, and there is actually only one network. This algorithm can be used for various classical network structures, such as the network structure of the feature extractor E, which can be Inception, ResNet, etc., such as the Inception-Resnet-V1 network structure. Of course, the original input images may vary in size, but the input images preferably have little difference in length and width. Otherwise, if they are uniformly scaled to the same size, some output images will be distorted too much, and the meaning of such image processing will be lost.

Step 24: Determine the loss function.

A good domain adaptation method should be based on features that are as similar as possible in the source and target domains while minimizing the prediction error in the source domain. So our loss consists of 2 parts.

Step 241: Determine the classification loss function. Classification loss is a classification loss function, which enables the model to have the ability to classify, so that the feature distance within the class (intra-class) is as small as possible, and the feature distance between the classes (inter-class) is as large as possible. This algorithm is universal to the loss function, and the classification loss function of source and target can be softmax, triplet, etc. Triplet loss is a loss function in deep learning, used to train samples with small differences, such as faces, etc. Feed data includes anchor (Anchor) examples, positive (Positive) examples, negative (Negative) examples, through optimization The distance between the anchor example and the positive example is smaller than the distance between the anchor example and the negative example, and the similarity calculation of the samples is realized.

Step 242: Determine the domain obfuscation function. domain confusionloss, the domain confusion function, exploits the confrontation between the feature extractor E and the domain discriminator G.

Step 2421: Fix the feature extractor E, and let the domain discriminator G discriminate the difference between the samples of the source domain and the target domain to the maximum extent.

Step 2422: Fix the domain discriminator G, and let the feature extractor E discriminate the difference between the samples of the source domain and the target domain in a minimal manner.

The above differences can be represented by the commonly used KL divergence. The training goal is to allow the domain discriminator G to maximize the difference between the samples from the source domain or the target domain, and to let the feature extractor E minimize the source domain samples and The difference between the target domain samples is min _G max _E KL.

Step 25: Feature extractor E and domain discriminator G are optimized. During training, the feature extractor E is initialized with the industry pre-training model of facenet, and it is divided into two steps to alternate.

Step 251 optimizes the feature extractor E. The total loss is the weighted sum of the above two domain classification loss functions classificationloss and domain confusionloss, namely total loss=source classification loss+target classification loss-KL, where alpha is the relative weight of the target classification loss function targetclassification loss. With the goal of minimizing this loss, a feature extractor E is trained with an SGD (stochastic gradient descent) optimizer. The negative sign in front of KL is because the goal of feature extractor E is to maximize KL, and the optimizer is to minimize loss.

Step 252 optimizes the domain discriminator G. The total loss is KL. With the goal of minimizing this loss, the domain discriminator G is trained with an SGD (stochastic gradient descent) optimizer.

Step S26, acquiring the target. In this way, the dual goal of the feature distribution of different people being far away from the feature distribution of the same person in different domains is achieved.

FIG. 4 is a schematic diagram of t-SNE after training 20 epochs in a cross-domain face recognition method of the present invention. As shown in Figure 4, during the training process, the effect of training can be verified in an intuitive way, and the multi-dimensional output of the feature extractor E is reduced to two-dimensional display through T-sne to display the intra-class distance and inter-class distance of features .

The feature extractor E network is set to Inception-ResNet V1, the output features are 128 dimensions, the domain discriminator G network is set to 2 fully connected layers and sigmoid, and the classification loss is set to triplet loss. At the beginning, we loaded the industry image webface The trained pretrained model.

Each point on Figure 4 represents the feature of a face image, the odd number represents the source domain, the even number represents the target domain, and the odd and even numbers of adjacent serial numbers (such as 1 and 2, 3 and 4, etc.) are respectively Source domain and target domain images representing the same person. The short distance between points means that the features are similar. The ideal effect of cross-domain recognition is that all points of the same person are highly clustered, and points of different people are far apart.

FIG. 5 is a schematic diagram of t-SNE after training 50 epochs in a cross-domain face recognition method of the present invention. Each point on Figure 5 represents the feature of a face image, the odd number represents the source domain, the even number represents the target domain, and the odd and even numbers of adjacent serial numbers (such as 1 and 2, 3 and 4, etc.) represent respectively Source domain sourcedomain and target domain target domain images of the same person. The short distance between points means that the features are similar. The ideal effect of cross-domain recognition is that all points of the same person are highly clustered, and points of different people are far apart.

Through the comparison between Figure 4 and Figure 5, as the number of training epochs increases, we can find that as the training progresses, not only the features within the same human domain are more aggregated, but also the features between the same human domain are closer.

It can be seen from the above description that using the cross-domain face recognition method according to the present invention, the beneficial technical effects are: on the basis of the original technical solution of facenet, the feature extractor E adopts the Inception-Resnet-V1 network structure, and the source classification loss function On the basis of the triplet loss function and the target classification loss function, the domain confusion loss function is added to solve the problem of face cross-domain recognition under the condition that the image to be recognized and the training sample image have different statistical distribution characteristics, so that the same domain The facial features between them are closer, which reduces the misrecognition rate and improves the accuracy of cross-domain recognition.

The embodiments of the present invention have been introduced in detail above, and specific examples are used to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present invention; at the same time, for Persons of ordinary skill in the art, according to the idea of the present invention, will have changes in the specific embodiments and application scope. To sum up, the contents of this specification should not be construed as limiting the present invention.

Claims (10)

1. a cross-domain facial recognition method, is characterized in that, comprises: Prepare source sample data and target sample data; Fix the feature extractor E, let the domain discriminator G maximize the difference between the source sample data and the target sample data; Fix the domain discriminator G, and let the feature extractor E minimize the difference between the source sample data and the target sample data; The domain discriminator G is maximized and the feature extractor E is minimized to reach a balance point, and feature values that have no inter-domain difference between the source sample data and the target sample data are extracted from the feature extractor E. 2. The cross-domain face recognition method according to claim 1, wherein preparing the source sample data and the target sample data comprises: The source sample data and the target sample data are respectively aligned with facial features. 3 . The cross-domain facial recognition method according to claim 1 , wherein the domain discriminator G and the feature extractor E share parameters for different domains. 4 . 4. cross-domain facial recognition method according to claim 1, is characterized in that, preparing source sample data and target sample data comprises: The source sample data and the target sample data are processed to increase the diversity of the sample data. 5. The cross-domain facial recognition method according to claim 1, wherein the feature extractor E is fixed, the domain discriminator G is maximized, and before discriminating the difference between the source sample data and the target sample data, comprising: determining a loss function . 6. The cross-domain facial recognition method according to claim 4, wherein processing the source sample data and the target sample data comprises: randomly cropping, flipping, adjusting brightness, contrast, Hue and saturation processing. 7. The cross-domain facial recognition method according to claim 5, wherein the loss function comprises a classification loss function and a domain confusion loss function. 8. The cross-domain facial recognition method according to claim 7, wherein the loss function is a weighted sum of a classification loss function and a domain confusion loss function. 9. The cross-domain facial recognition method according to claim 8, wherein the total loss=source classification loss+target classification loss−KL divergence, wherein KL divergence is used to measure the feature probability of source sample data and target sample data difference between distributions. 10 . The cross-domain facial recognition method according to claim 9 , wherein the KL divergence, also called relative entropy, is a measure between the feature probability distributions of the source sample data and the target sample data. 11 .

Images