CN107038400A - Face identification device and method and utilize its target person tracks of device and method - Google Patents

Abstract

The invention provides a face recognition device and method, and a target person tracking device and method using the same. The face recognition device includes a face set generation module configured to generate a face set based on multiple video images; a feature metric normalization module configured to perform metric space transformation on the same face image under multiple different cameras, To eliminate the difference between the same face images under the multiple different cameras; and the feature envelope forming module is configured to perform envelope processing on the multiple face images transformed in the metric space, so as to combine the face sets Multiple different face set feature spaces are transformed into one same face set feature space. The present invention normalizes the feature metric space according to the differences of different cameras, and performs recognition in combination with the feature metric space and the face image set, thereby greatly improving the accuracy of face recognition in the monitoring scene.

Description

Face recognition device and method, and target person tracking device and method using the same

technical field

The present invention generally relates to the technical field of computer video processing, and more specifically, to a face recognition device, a face recognition method, a target person tracking device using a face recognition device, and a target person tracking method using a face recognition method. The device and method are suitable for urban security management, public areas such as airports, prisons and libraries.

Background technique

Existing face recognition technology has a high degree of recognition for faces with better image quality. For databases such as LFW (which are completed by the Technical Vision Laboratory of the University of Massachusetts, Amherst), the face recognition accuracy can reach more than 99%. However, if such algorithms are directly applied to monitoring scenarios, their effects will be greatly reduced. Taking the LFW database as an example, although it is generally believed that this database is an image taken under uncontrolled conditions, it is much better than the face image of the monitoring scene in terms of resolution, color fidelity, and face posture. Therefore, if the general face recognition algorithm is directly transplanted into the face recognition of the surveillance scene, its effect will be greatly reduced.

The main problems of using the above LFW database are the following two points: 1. Conventional face recognition generally uses a single image, but in the monitoring scene, due to the poor image quality, the information of a single image is limited, and the accuracy of recognition cannot be guaranteed. ; 2. The imaging differences faced by conventional face recognition are not large, so generally, the difference reduction processing will not be performed for different camera images. In the monitoring scene, due to the face resolution, color, There will be large differences in posture and so on, so it is necessary to reduce the difference first to ensure the accuracy of recognition. Therefore, the LFW database cannot be applied to field monitoring scenarios.

Compared with ordinary face recognition, face recognition in surveillance scenes has some new features. 1. A single target object can be described by multiple continuous images, and the features extracted from multiple images can describe the characteristics of the target more completely. At present, the image-set method can describe a single target by forming a space similar to a high-dimensional envelope through multiple images. For two different targets, the minimum distance between the two envelopes is calculated as the difference between the two targets. At present, related algorithms include image set distance AHISD (Affine Hullbased Image Set Distance) based on affine package, sparse approximate nearest neighbor point SANP, CRNP (Collaboratively Regularized Nearest Points) and binary linear regression classification DLRC (Dual Linear Regression Classification), etc. . Although this type of algorithm can describe the feature space of the target more abundantly, the envelopes formed by this type of algorithm for different targets tend to overlap because of the similarity of their features. Therefore, for different targets, the degree of difference in the extracted features must be Larger so that the envelopes formed based on features do not overlap. 2. Surveillance scenes across cameras have different lighting, viewing angles/poses, and image resolutions, which will greatly affect the accuracy of recognition. Currently, metric-learning (ie, metric learning) methods can solve this problem. This type of algorithm can project the features of different feature spaces into the same metric space, reduce the difference of the same target due to the difference of camera and pose/illumination/expression, and increase the feature difference of different targets. This feature of the metric-learning method can make up for the shortcomings of the image-set (ie, image set, also known as atlas) method. At present, related algorithms include large margin nearest neighbor LMNN (Large margin nearest neighbor) and so on. 3. Since the surveillance camera needs to continuously record data, the amount of data it stores is astonishing. Finding the target image in a large amount of data requires an extremely fast algorithm, and the facial features are compressed and then recognized , which can greatly improve the algorithm efficiency.

Therefore, there is a problem of multi-camera face recognition in the monitoring scene using multiple cameras. Due to the large difference from the conventional scene, the processing method is quite different from the conventional face recognition. The objects tracked across cameras are generally multi-source images. Different cameras (manufacturers and models) lead to large differences in image resolution, color, and deformation; in addition, differences in environmental light sources, differences in human focus, and differences in camera installation angles, etc. It will also make the same target in the images generated by different cameras have greater differences; secondly, in the monitoring scene, the distance between the moving target and the camera varies greatly, which will lead to large differences in imaging clarity and color. These differences are unique problems faced by face recognition in surveillance scenarios. In addition, cross-camera tracking is to process massive data from multiple cameras, so fast processing of massive data is also a key issue that needs to be solved for cross-camera tracking.

Contents of the invention

Aiming at the differences in multi-camera face recognition in the monitoring scene in the prior art and the technical problems of massive data processing for multiple cameras, the present invention provides a face recognition device and a face recognition device that can solve the above technical problems. method, and target person tracking apparatus and method.

In order to solve the above-mentioned defects, the present invention proposes a cross-camera recognition method applicable to large-scale data monitoring scenarios. Specifically, the advantages are provided in the following three ways: 1. For multiple cameras, in order to reduce their differences, use feature metric space normalization; 2. For the continuity of video frames, extract and merge multiple faces Features to describe the same target, so that the target feature space is more complete; and 3. For large-scale data set operations, use feature compression and two-stage matching methods.

According to an aspect of the present invention, a face recognition device is provided, including: a face set generation module configured to generate a face set based on multiple video images; a feature metric normalization module configured to perform multiple different The same face image under the camera is transformed into a metric space to eliminate the difference between the same face images under different cameras; and the feature envelope forming module is configured to wrap the multiple face images transformed in the metric space Networking processing is used to transform multiple different feature spaces of the face set into a feature space of the same face set.

Preferably, the differences between the multiple images of the same face under different cameras include differences in background light sources and differences in shooting angles.

Preferably, performing metric space transformation on the same face images from multiple different cameras to eliminate the differences between the multiple human face images further includes: performing metric space transformation on multiple human face images to make multiple different cameras The same face images under the same background light source; and the metric space transformation is performed on multiple face images, so that the same face images under multiple different cameras have the same shooting angle.

Preferably, the face recognition device further includes: a recognition module configured to, based on the set target person, select a target face set with the highest matching degree with the set target person from multiple face sets.

Preferably, the face set generation module includes: a face detection module configured to perform face detection on video images acquired at predetermined frame intervals, wherein the predetermined frame interval is determined according to the video frame rate.

Preferably, the face set generation module also includes a face alignment module configured to perform the following image processing on the video image for face detection: it is configured to extract key points from the video image, and to align people according to the distance between the pupils of the eyes Scale transformation of the face; and according to the position of the extracted key points, the yaw angle of the face is estimated, and the face is rotated into a frontal face according to the yaw angle of the face, and multiple face images including the same target person are generated. A set of human faces, where keypoints include eyes, noses, and mouths.

Preferably, the feature metric normalization module also includes: a feature extraction module, which extracts features from the face set to form an original feature face set; and a compression module, configured to compress the original features to form compressed features Human faces set.

Preferably, the identification module further includes: a rough matching identification module, configured to determine an orderly predetermined number of face sets based on the set target person; and a fine matching identification module, configured to, based on the set target person, A target face set is quickly determined from an ordered predetermined number of face sets.

Preferably, based on the set target person, a predetermined number of face sets are determined from a plurality of compressed feature face sets; according to the matching degree with the set target person, the predetermined number of face sets are determined in order of matching degree from high to low. Sorting the face sets to generate an ordered predetermined number of face sets; and based on the set target person, quickly determine Target face set.

According to another aspect of the present invention, a face recognition method is provided, comprising the following steps: generating a face set based on multiple video images; performing metric space transformation on multiple face images in the face set to eliminate multiple The difference between the same face images under different cameras; and enveloping the multiple face images of the metric space transformation, so as to transform the feature space of multiple different face sets into one same The feature space of face sets.

Preferably, the differences between the multiple images of the same face under different cameras include differences in background light sources and differences in shooting angles.

Preferably, performing metric space transformation on the same face images under multiple different cameras to eliminate the difference between the same face images under multiple different cameras further includes: performing metric space transformation on the same face images under multiple different cameras Transformation, so that multiple images of the same face under different cameras have the same background light source; and perform metric space transformation on multiple facial images, so that multiple facial images have the same shooting angle.

Preferably, the face recognition method further includes: based on the set target person, selecting the target face set with the highest matching degree with the set target person from multiple face sets.

Preferably, the face recognition method further includes: before generating the face set, performing face detection on video images acquired at predetermined frame intervals, wherein the predetermined frame interval is determined according to the video frame rate.

Preferably, before carrying out metric space transformation to a plurality of face images, carry out following image processing to the video image of face detection: extract key point from video image, and carry out scale transformation to face according to binocular pupil distance; And according to the position of the extracted key point, estimate the yaw angle of the face, and rotate the face into a frontal face according to the yaw angle of the face, generate a face set including multiple face images of the same target person, wherein , the key points include eyes, nose and mouth.

Preferably, the following steps are also included before performing metric space transformation on multiple face images in the face set: performing feature extraction on the face set to form an original feature face set; and being configured to compress the original features , to form a compressed feature face set.

Preferably, the face recognition method further includes: based on the set target person, determining an ordered and predetermined number of human face sets; and based on the set target person, quickly determining the target person from the ordered and ordered number of human faces face set.

Preferably, the face recognition method further includes: based on the set target person, determining a predetermined number of face sets from a plurality of compressed feature face sets; according to the matching degree with the set target person, the matching degree is from high to Sorting the predetermined number of human face sets in a low order to generate an ordered predetermined number of human face sets; The target face set is quickly determined from the original feature face set.

According to another aspect of the present invention, a target person tracking device is provided, including: a human face set generation module configured to generate a human face set based on multiple video images; a feature metric normalization module configured to The metric space transformation is performed on the face images to eliminate the difference between the multiple face images; the feature envelope forming module is configured to perform envelope processing on the multiple face images transformed by the metric space, so as to The feature space of multiple different face sets in the face set is transformed into the feature space of the same set of faces; the face recognition module, based on the set target person, selects the target person from multiple face sets that matches the set target person. The target face set with the highest matching degree of face images; the tracking module determines the route of the target person according to the shooting location and time of each face image in the target face set.

According to yet another aspect of the present invention, a target person tracking method is provided, including the following steps: generating a face set based on multiple video images; performing metric space transformation on multiple face images in the face set to eliminate multiple The difference between the face images; and enveloping the multiple face images of the metric space transformation, so as to transform the feature space of multiple different face sets of the face set into the same face set Feature space; based on the set target person, select the target face set that matches the face image of the set target person from multiple face sets; according to the shooting of each face image in the target face set Location and time, determine the route of the target person.

The present invention is specially aimed at the face recognition device and method of the monitoring scene. The feature metric space is normalized according to the differences of different cameras, and combined with the feature metric space and face image set for recognition, which greatly improves the accuracy of face recognition in surveillance scenarios.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some of the present invention. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1 is a block diagram of a face recognition device according to an embodiment of the present invention;

Fig. 2 is the flow chart of the face recognition method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of feature extraction and compression steps further included in the face recognition method in FIG. 2 according to an embodiment of the present invention.

4 is a schematic diagram of a feature envelope generated by enveloping processing in a face recognition method according to an embodiment of the present invention;

5 is a block diagram of a target person tracking device according to an embodiment of the present invention;

Fig. 6 is a flow chart of the target person tracking method according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of two-stage matching performed in the target person tracking process according to an embodiment of the present invention;

8A, 8B and 8C respectively show that the target person is tracked in one camera, the target person is continued to be tracked in multiple cameras, and the trajectory tracked on the map is displayed in the process of target person tracking according to an embodiment of the present invention. system diagram.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention belong to the protection scope of the present invention.

FIG. 1 is a block diagram of a face recognition device according to an embodiment of the present invention. Fig. 2 is a flowchart of a face recognition method according to an embodiment of the present invention.

Referring to FIG. 1 , the face recognition device includes a face set generation module 102 , a feature metric normalization module 104 and a feature envelope formation module 106 . Specifically, the face set generation module 102 is configured to generate a face set based on multiple video images. The feature metric normalization module 104 is configured to perform metric space transformation on multiple same face images from different cameras, so as to eliminate differences among multiple same face images from different cameras. The feature envelope forming module 106 is configured to perform enveloping processing on the multiple face images of the metric space transformation, so as to transform the feature space of a plurality of different face sets of the face set into features of the same face set space.

In one embodiment, the face set generation module 102 further includes: a face detection module configured to perform face detection on video images acquired at predetermined frame intervals, wherein the predetermined frame interval is determined according to the video frame rate. In addition, the human face set generation module 102 also includes a human face alignment module, which is configured to perform the following image processing on the video image for face detection: it is configured to extract key points from the video image, and align the human faces according to the distance between the pupils of the eyes Scale transformation of the face; and according to the position of the extracted key points, the yaw angle of the face is estimated, and the face is rotated into a frontal face according to the yaw angle of the face, and multiple face images including the same target person are generated. A set of human faces, where keypoints include eyes, noses, and mouths.

Specifically, the differences between multiple images of the same face under different cameras include differences in background light sources and differences in shooting angles. Using the feature metric normalization module 104 to perform metric space transformation on the same face images under different cameras to eliminate the difference between the multiple face images. Make the same face images under different cameras have the same background light source; and perform metric space transformation on multiple face images, so that the same face images under different cameras have the same shooting angle. In addition, the feature metric normalization module 104 also includes a feature extraction module, which extracts features from the face set to form an original feature face set; and a compression module, configured to compress the original features to form a compressed feature face set. face set.

In addition, the face recognition device further includes a recognition module configured to, based on the set target person, select a target face set with the highest matching degree with the set target person from multiple face sets. The recognition module also includes: a rough matching recognition module configured to determine an ordered predetermined number of face sets based on the set target person; A predetermined number of face sets can be used to quickly determine the target face set. Specifically, based on the set target person, determine a predetermined number of face sets from multiple compressed feature face sets; Sorting the face sets to generate an ordered predetermined number of face sets; and based on the set target person, quickly determine Target face set.

With reference to Fig. 2, the face recognition method comprises the following steps: in step 210, generate a human face set based on multiple video images; in step 220, carry out metric space transformation to the multiple pieces of human face images in the human face set, to eliminate multiple difference between the same face images under different cameras; and in step 230, enveloping the multiple face images of the metric space transformation, so as to combine the features of multiple different face sets in the face set The space is transformed into a feature space of the same face set.

The face recognition method further includes: before generating the face set, performing face detection on video images acquired at predetermined frame intervals, wherein the predetermined frame interval is determined according to the video frame rate. In addition, before performing metric space transformation on multiple face images, the following image processing is performed on the video image for face detection: extracting key points from the video image, and performing scale transformation on the face according to the distance between the pupils of the eyes; and According to the positions of the extracted key points, the yaw angle of the face is estimated, and the face is rotated into a frontal face according to the yaw angle of the face, and a face set including multiple face images of the same target person is generated, wherein, Key points include the eyes, nose and mouth.

The differences among the same face images under different cameras include differences in background light sources and differences in shooting angles. Before performing metric space transformation on multiple face images in the face set, it also includes: performing feature extraction on the face set to form an original feature face set; and being configured to compress the original features to form compressed features Human faces set. Perform metric space transformation on the same face images from different cameras to eliminate the difference between the same face images from different cameras. Make the same face images under different cameras have the same background light source; and perform metric space transformation on the multiple face images so that the multiple face images have the same shooting angle.

In addition, the face recognition method further includes: based on the set target person, selecting the target face set with the highest matching degree with the set target person from multiple face sets. The face recognition method further includes: determining an ordered and predetermined number of face sets based on the set target person; and quickly determining the target face set from the ordered and predetermined number of face sets based on the set target person. Specifically, the face recognition method further includes: based on the set target person, determining a predetermined number of face sets from a plurality of compressed feature face sets; Sorting the predetermined number of human face sets in a low order to generate an ordered predetermined number of human face sets; The target face set is quickly determined from the original feature face set.

According to the above-mentioned embodiments of the present invention, using the face recognition device and method of the monitoring scene, wherein the feature metric space is normalized for the differences of different cameras, and the feature metric space and the face image set are combined for recognition, which can Greatly improve the accuracy of face recognition.

The target person tracking device and the target person tracking method will be described below with reference to Figures 3 to 8C, and then the face recognition device included in the target person tracking device will be described in detail and the face used in the target person tracking method The identification method is described in detail.

FIG. 5 is a block diagram of a target person tracking device according to an embodiment of the present invention. First, the target person tracking device will be described in detail with reference to FIG. 5 . Referring to FIG. 5 , according to another embodiment of the present invention, the target person tracking device includes a face set generation module 502 , a feature metric normalization module 504 , a feature envelope formation module 506 , a face recognition module 510 and a tracking module 512 . The face set generation module 502, the feature measure normalization module 504, and the feature envelope forming module 506 are included in the face recognition device. In a preferred embodiment, the face set generation module 502 shown in FIG. 5 , the feature metric normalization module 504 , and the feature envelope formation module 506 can be respectively combined with the face set generation module 102 , feature metric normalization module 506 shown in FIG. 1 . The normalization module 104 is the same as or similar to the feature envelope forming module 106 .

In the following, each module in the target person tracking device will be described in detail respectively.

The face set generation module 502 is configured to generate a face set based on multiple video images. In one embodiment, the face set generation module 502 further includes: a face detection module configured to perform face detection on video images acquired at predetermined frame intervals, wherein the predetermined frame interval is determined according to the video frame rate. In addition, the human face set generation module 502 also includes a human face alignment module, which is configured to perform the following image processing on the video image for face detection: it is configured to extract key points from the video image, and to align the human face according to the distance between the pupils of the eyes Scale transformation of the face; and according to the position of the extracted key points, the yaw angle of the face is estimated, and the face is rotated into a frontal face according to the yaw angle of the face, and multiple face images including the same target person are generated. A set of human faces, where keypoints include eyes, noses, and mouths.

Hereinafter, an example of the face generation module 502 will be described in detail. In a specific example, face detection and tracking are first performed by the face detection module, and then face alignment is performed by the face alignment module. Specifically, the surveillance video is decomposed into pictures, and face detection is performed on the video pictures every few frames. The interval frames are determined according to the actual video frame rate. The detected face area contains as little background information as possible to reduce the influence of background noise on recognition. In order to obtain multiple pictures of the same person, face images need to be tracked. There may be two kinds of errors in face tracking: 1. The tracked target does not belong to the same person; the solution is to detect the similarity of adjacent frames of face images in real time, and if the similarity is lower than the threshold, set A new ID, where the threshold can be set according to user needs; 2. The tracked target belongs to the same person, but the tracked image cannot accurately cover the face area. The solution is: use the closest face detection image of the face tracking frame to correct the face tracking result.

In a specific example, face alignment is performed next. Face alignment has a great influence on the recognition results. First, extract the four key points of the eyes, nose, and mouth from the face, and then perform scale transformation on the face according to the distance between the pupils of the two eyes; then rotate the face image based on the angle between the line connecting the two pupils and the horizontal line is horizontal; according to the positions of these four key points, the yaw angle of the face is estimated, and the face is rotated to the front face. If the four key points of the face cannot be extracted, then these face images are not processed. When the number of images in a single image set is relatively large, the images whose key points cannot be extracted can be discarded, but it is necessary to ensure that the number of images in a single image set is greater than 15.

The target person tracking device also includes a feature metric normalization module 504, which is configured to perform metric space transformation on multiple face images, so as to eliminate the differences between multiple face images, specifically, the same Differences among face images include differences in background light sources and differences in shooting angles. Specifically, performing metric space transformation on the same face images from multiple different cameras to eliminate the differences between the multiple face images further includes: performing metric space transformation on multiple face images so that multiple different cameras The same face images under the same background light source; and the metric space transformation is performed on multiple face images, so that the same face images under multiple different cameras have the same shooting angle. In addition, the feature metric normalization module 504 also includes a feature extraction module, which extracts features from the face set to form an original feature face set; and a compression module, configured to compress the original features to form a compressed feature face set. face set. Utilizing the feature metric normalization module 504 can greatly reduce differences in background light sources and shooting angles.

Hereinafter, an example of the feature metric normalization module 504 will be described in detail. In a specific example, when normalizing the feature metric space of a multi-camera based on a face set, the face detection and tracking are first performed through the feature extraction module and the compression module, the face detection module, and then through the feature metric normalization module 504 Perform multi-camera feature metric space normalization.

FIG. 3 is a schematic diagram of feature extraction and compression steps further included in the face recognition method in FIG. 2 according to an embodiment of the present invention. Referring to Fig. 3, the face detection and tracking are performed by the feature extraction module and the compression module face detection module. In a specific example, feature extraction is performed on the aligned face image set, and the extracted features may be LBP (ie, local binary pattern), Gabor (ie, Gabor) or DCT (ie, discrete cosine transform) features. In general, for a 32×32 or 64×64 image, its feature dimension can reach more than 1000. Although the processing time of a single image is not much, if the number of images reaches 100,000 levels, the time-consuming length will be extremely Therefore, it is necessary to perform feature compression on the extracted features. Sampling the extracted features; then using WTA hash compression technology to compress the sampled features, the compression result is: the local maximum eigenvalue is 1, and the remaining eigenvalues are compressed to 0; finally, the compression of all local areas is connected in series , the result of which is the compressed result of this eigenvector. In the feature compression process, the feature sampling interval and local region length are determined according to actual needs. The compressed feature vector is binary compressed, and the compressed feature dimension will be reduced.

In a specific example, the normalization of the multi-camera feature metric space is described in detail. This process requires training before use. First, use the samples between different cameras for transformation matrix training, and then apply this transformation matrix to the corresponding camera during use. The purpose of feature metric space normalization is to reduce the difference caused by the same person under different cameras due to different lighting and angles. For example, the feature metric normalization module 504 transforms the left illumination into the right illumination or transforms the right illumination into the left illumination. In addition, for example, the different angles at which different cameras shoot the target person are converted into shooting the target person at the same angle. This process transforms the compressed feature vector into a metric space, and changes the features of different metric spaces into the same metric space, that is, has the same metric. This process uses the metric-learning method. Metric-learning can transform the input feature space into a scale-meaning space: L is the transformation matrix. When projecting feature metrics from different spaces to the same space, the following two conditions need to be met: 1. Make the feature space distance of different ID objects larger, and the distance exceeds the safety distance; 2. Reduce the feature space distance of the same ID object. In this way, misclassification can be avoided. Before describing its error function, the following symbols are defined:

input feature vector,

y _ij : if with have the same label as 1, otherwise 0,

η _ij : if The adjacent target of is 1 when , otherwise 0.

Its error function is as follows:

ε(L)＝ε _pull (L)+ε _push (L)

in

ε _pull is an error function that brings the same target closer in the neighborhood, and ε _push is an error function that pushes different targets in the neighborhood farther away. Since solving L directly may lead to a local optimal solution, it is transformed: M=L ^T L, so that ε(M) will have a global optimal solution, and the distance form after the corresponding feature transformation is transformed into: The final error function form is:

Minimize:

ε(M)＝ε _pull (M)+ε _push (M)

Subject to:

M≥0

The optimal value solution uses tree regression for gradient approximation. In complex cases such as linear inseparability, kernel techniques can be introduced.

Next, the feature envelope forming module 506 is configured to perform envelope processing on the multiple face images transformed in the metric space, so as to transform a plurality of different face set feature spaces of the face set into one same face The feature space 508 of the set. Specifically, performing metric space transformation on the same face images from multiple different cameras to eliminate the differences between the multiple face images further includes: performing metric space transformation on multiple face images so that multiple different cameras The same face images under the same background light source; and the metric space transformation is performed on multiple face images, so that the same face images under multiple different cameras have the same shooting angle.

Fig. 4 is a schematic diagram of a feature envelope generated by envelope processing in the face recognition method according to an embodiment of the present invention. Hereinafter, the envelope processing performed by the feature envelope forming module will be described in detail with reference to FIG. 4 .

The purpose of using several images, that is, image sets, to describe human faces is to expand the feature space of human faces and describe human faces more comprehensively. For the same target, on the basis of extracting multiple face images, after performing metric space transformation on the compressed face features, use the image-set method to envelop them, and the final result is to express multiple images of the same target for different weighted combinations. The so-called envelope refers to the feature space formed by an image set formed by different pictures of the same person. Suppose X _c is a feature set including several image features, X _c,i is one of the samples, where c=1,...,C, i=1,...,n _c . For example, n _c is 5-50 or any other positive integer, preferably 15, 20, and 50. More preferably, when n _c is 5, the efficiency is the highest. The envelope is expressed as H _c , and the specific form is:

Among them, α _c,i is the weight of each sample. There are different acquisition methods for the weight, and in the present invention, the method of DLRC weight calculation is adopted.

The face recognition module (also known as the recognition module) 510 selects the target face set with the highest degree of matching with the face image of the set target person from multiple face sets based on the set target person. The recognition module also includes: a rough matching recognition module configured to determine an ordered predetermined number of face sets based on the set target person; A predetermined number of face sets can be used to quickly determine the target face set. More specifically, based on the set target person, determine a predetermined number of face sets from multiple compressed feature face sets; The face sets are sorted to generate ordered predetermined number of face sets; Determine the target face set.

In the following examples, the cross-camera face set envelope recognition will be described in detail with reference to FIGS. 7 and 8A to 8C.

After the feature envelope of each face set is formed, it is necessary to measure the difference of the features of different face sets. The difference is determined by calculating the minimum distance of the envelope of the face set. Bad to be sure. Figure 3 is a schematic diagram of feature envelopes of different face sets. There are three face sets in the figure, and each face set consists of several face images. In the actual recognition process, each face set is a set of face map features, and each face map feature includes two features: original features and compressed features. The original feature is the image set feature in Figure 3 and the compressed feature is the image set hash code feature in Figure 3. The characteristics of the original features and the compressed features are: the recognition rate of the original features is higher, but it takes a long time; the recognition rate of the compressed features is lower than that of the original features, but the time required is short. The identification process is divided into two phases:

1. Rough matching stage: use the compressed features, and the object is the set of all recognized faces; speed is the priority at this stage;

2. Accurate matching stage: use the original features, and the object is the N face set that is relatively high in the rough matching result; the accuracy is prioritized in this stage.

Figure 4 is a schematic diagram of two-stage matching. In the rough matching stage, all test samples are used for matching, and the compressed features are used to calculate the distance of the face image set, and then the test samples are sorted according to the distance to the face to be matched. False matching problems may occur in the rough matching stage, such as more similar faces are ranked behind, so exact matching is needed to adjust the results of rough matching. Exact matching only operates on the top image sets sorted by distance, and the features used are original features. For example, in the rough matching stage, 2-50 or any other number of ordered face sets can be quickly selected from the compressed set of all faces to be recognized; The obtained ordered face set determines the target face set. The number of ordered face sets can be determined according to user needs, for example, 5, 10, and 15.

The tracking module 512 determines the route of the target person according to the shooting location and time of each face image in the target face set. Specifically, after the face recognition module selects the target face set with the highest matching degree with the set target person's face image, the tracking module 512 according to each face image in the obtained target face set The location and time of the capture are shown on a map showing the person's trajectory.

In the scene of cross-camera tracking, the present invention can effectively track and identify the same person in different cameras, and finally display the person's trajectory on the map. Figures 8A to 8C respectively show the following results: (a) tracking the target person in one camera; (b) continuing to track the target in multiple cameras; (c) showing the trajectory of the tracked person in the map .

Fig. 6 is a flowchart of a method for tracking a target person according to an embodiment of the present invention. The target person tracking method will be described in detail below with reference to FIG. 6 .

The target person tracking method includes the following steps:

In step 610, a face set is generated based on multiple video images; before the face set is generated, face detection is performed on video images acquired at predetermined frame intervals, wherein the predetermined frame interval is determined according to the video frame rate. Before the metric space transformation is performed on multiple face images, the following image processing is performed on the video image for face detection: extract key points from the video image, and perform scale transformation on the face according to the distance between the pupils of the eyes; and according to the extracted The position of the key points, estimate the yaw angle of the face, and rotate the face into a frontal face according to the yaw angle of the face, and generate a face set including multiple face images of the same target person, where the key points Includes eyes, nose and mouth.

In step 620, perform metric space transformation on multiple face images in the face set to eliminate the differences between multiple face images; before performing metric space transformation, perform feature extraction on the face set to form an original feature face set; and being configured to compress the original features to form a compressed feature face set. Among them, the differences between multiple images of the same face under different cameras include differences in background light sources and differences in shooting angles. Specifically, performing metric space transformation on the same face images from multiple different cameras to eliminate the differences between the same face images from different cameras further includes: performing metric space transformation on multiple same face images from different cameras Transformation, so that multiple images of the same face under different cameras have the same background light source; and perform metric space transformation on multiple facial images, so that multiple facial images have the same shooting angle.

In step 630, envelope processing is performed on the plurality of face images transformed by the metric space, so as to transform a plurality of different face set feature spaces of the face set into a feature space of the same face set;

In step 640, based on the set target person, a target face set with the highest matching degree to the set target person's face image is selected from multiple face sets. Specifically, based on the set target person, determine an ordered and predetermined number of human face sets; and based on the set target person, quickly determine the ordered target human face set from the ordered and predetermined number of human faces. More specifically, based on the set target person, determine a predetermined number of face sets from multiple compressed feature face sets; The face sets are sorted to generate ordered predetermined number of face sets; Determine the target face set.

In step 650, the route of the target person is determined according to the shooting location and time of each face image in the target face set.

Therefore, the embodiments of the present invention first compress the extracted invariant features, and then propose a cross-camera recognition method based on the image-set method and metric-learning to solve the problem of cross-camera tracking in surveillance scenarios.

Therefore, using the face recognition device and the face recognition method of the embodiments of the present invention can improve the face recognition accuracy in a monitoring scene. In addition, the target person tracking device using the face recognition device and the target person tracking method using the face recognition method can not only provide face recognition accuracy, but also can find the target person through two-stage matching in a large amount of data Find the target image at an extremely fast speed. That is to say, in practical application, the present invention can assist users to find target persons from massive databases and extract their time and route information, greatly improving work efficiency.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of the present invention. within the scope of protection.

Claims (20)

1. A face recognition device, characterized in that, comprising: A human face set generation module is configured to generate a human face set based on multiple video images; The feature metric normalization module is configured to perform metric space transformation on the multiple identical face images under different cameras, so as to eliminate the difference between the multiple identical facial images under different cameras; and The feature envelope forming module is configured to perform enveloping processing on the plurality of face images transformed in the metric space, so as to transform the feature space of a plurality of different face sets of the face set into one same person Feature space for face sets. 2 . The face recognition device according to claim 1 , wherein the differences between the plurality of images of the same face under different cameras include differences in background light sources and differences in shooting angles. 3. face recognition device according to claim 2, is characterized in that, carries out metric space transformation to described multiple identical facial images under different cameras, to eliminate the difference between described multiple facial images further include: Performing metric space transformation on the multiple face images, so that the multiple same face images under different cameras have the same background light source; and The multiple face images are subjected to metric space transformation, so that the multiple same face images under different cameras have the same shooting angle. 4. The face recognition device according to claim 1, further comprising: a recognition module configured to, based on the set target person, select from a plurality of faces to match the set target person The target face set with the highest degree. 5. The face recognition device according to claim 1, wherein the face set generation module comprises: a face detection module configured to perform face detection with video images acquired at predetermined frame intervals, wherein, The predetermined frame interval is determined according to the video frame rate. 6. face recognition device according to claim 5, is characterized in that, described face collection generation module also comprises face alignment module, is configured to carry out following image processing to the described video image that carries out face detection: configured to extract key points from the video image, and perform scale transformation on the face according to the distance between the pupils of the eyes; and Estimating the yaw angle of the human face according to the extracted position of the key point, and rotating the human face into a frontal face according to the yaw angle of the human face, generating a plurality of human face images including the same target person A human face set, wherein the key points include eyes, nose and mouth. 7. face recognition device according to claim 1, is characterized in that, described feature measure normalization module also comprises: A feature extraction module, which extracts features from the face set to form an original feature face set; and The compression module is configured to perform compression processing on the original features to form a compressed feature face set. 8. The face recognition device according to claim 4, wherein the recognition module further comprises: The rough matching recognition module is configured to determine an orderly predetermined number of face sets based on the set target person; and The fine matching recognition module is configured to quickly determine the target face set from the ordered predetermined number of face sets based on the set target person. 9. The face recognition device according to any one of claims 7 and 8, characterized in that, Based on the set target person, determine a predetermined number of face sets from a plurality of compressed feature face sets; According to the degree of matching with the set target person, sort the predetermined number of human face sets in descending order of matching degree, so as to generate the ordered predetermined number of human face sets; and Based on the set target person, the target human face set is quickly determined from a predetermined number of original feature human face sets corresponding to the ordered predetermined number of human face sets. 10. A face recognition method, characterized in that, comprising the following steps: Generate a face set based on multiple video images; Performing metric space transformation on multiple face images in the face set to eliminate the differences between the multiple same face images under different cameras; and Envelope processing is performed on the plurality of face images transformed by metric space, so as to transform a plurality of different feature spaces of the face set into a same feature space of the face set. 11. The face recognition method according to claim 10, characterized in that the differences between the plurality of images of the same face under different cameras include differences in background light sources and differences in shooting angles. 12. The face recognition method according to claim 11, characterized in that, performing metric space transformation on the same face images under the multiple different cameras, so as to eliminate the gap between the same face images under the multiple different cameras. The differences further include: Performing a metric space transformation on the multiple identical face images under different cameras, so that the multiple identical facial images under different cameras have the same background light source; and The multiple human face images are subjected to metric space transformation, so that the multiple human face images have the same shooting angle. 13. The face recognition method according to claim 10, further comprising: based on the set target person, selecting the target face with the highest matching degree with the set target person from a plurality of face sets set. 14. The face recognition method according to claim 10, further comprising: before generating the set of faces, performing face detection on video images acquired at predetermined frame intervals, wherein, performing face detection according to the frame rate of the video The predetermined frame interval is determined. 15. the face recognition method according to claim 14, is characterized in that, before carrying out metric space transformation to described multiple face images, carry out following image processing to the described video image that carries out face detection: Extracting key points from the video image, and performing scale transformation on the face according to the distance between the pupils of the eyes; and Estimating the yaw angle of the human face according to the extracted position of the key point, and rotating the human face into a frontal face according to the yaw angle of the human face, generating a plurality of human face images including the same target person A human face set, wherein the key points include eyes, nose and mouth. 16. face recognition method according to claim 10, is characterized in that, carrying out metric space transformation to a plurality of pieces of face images in described face collection also comprises the following steps: performing feature extraction on the face set to form an original feature face set; and It is configured to compress the original features to form a compressed feature face set. 17. The face recognition method according to claim 13, further comprising: Based on the set target person, determine an ordered and predetermined number of face sets; and Based on the set target person, quickly determine the target human face set from the ordered predetermined number of human face sets. 18. The face recognition method according to any one of claims 16 and 17, further comprising: Based on the set target person, determine a predetermined number of face sets from a plurality of compressed feature face sets; According to the degree of matching with the set target person, sort the predetermined number of human face sets in descending order of matching degree, so as to generate the ordered predetermined number of human face sets; and Based on the set target person, the target human face set is quickly determined from a predetermined number of original feature human face sets corresponding to the ordered predetermined number of human face sets. 19. A target person tracking device, comprising: A human face set generation module is configured to generate a human face set based on multiple video images; A feature metric normalization module configured to perform metric space transformation on the multiple face images to eliminate differences between the multiple face images; The feature envelope forming module is configured to perform enveloping processing on the plurality of face images transformed in the metric space, so as to transform the feature space of a plurality of different face sets of the face set into one same person The feature space of the face set; The face recognition module, based on the set target person, selects the target face set with the highest matching degree with the face image of the set target person from multiple face sets; The tracking module determines the route of the target person according to the shooting location and time of each face image in the target face set. 20. A target person tracking method, comprising the following steps: Generate a face set based on multiple video images; performing a metric space transformation on multiple face images in the face set to eliminate differences among the multiple face images; and performing enveloping processing on the plurality of face images transformed by metric space, so as to transform a plurality of different face set feature spaces of the face set into a feature space of the same face set; Based on the set target person, select the target face set with the highest matching degree with the face image of the set target person from multiple face sets; The route of the target person is determined according to the shooting location and time of each face image in the target face set.