CN115810207A - Method and apparatus for training a deep learning network for face recognition - Google Patents

Abstract

Methods and apparatus are provided for training a deep learning network for face recognition. The method comprises the following steps: performing face alignment processing on the at least one captured image using a face coordinate detector, thereby outputting at least one aligned image; inputting the at least one alignment image into a teacher model to obtain a first output vector; inputting the at least one captured image into a student model corresponding to the teacher model to obtain a second output vector; and adjusting the parameter setting of the student model according to the first output vector and the second output vector.

Description

Method and apparatus for training a deep learning network for face recognition

technical field

The present disclosure is about machine learning, especially a training method and a related device for realizing a face recognition network model with face alignment effect based on knowledge distillation techniques.

Background technique

Today's facial recognition algorithms mainly perform identity recognition on facial images. In order to make the deep learning network of face recognition try to obtain facial images in the same environment, a face coordinate detector (landmark detector) is usually added before the face recognition network model. Coordinates of features (eg, eyes, ears, mouth, nose, etc.) are processed for face alignment. In the architecture shown in FIG. 1 , the source image will first pass through the face detector 10, and the face detector 10 will find out the face pattern from the source image and extract it from the source image. Then, the extracted face pattern is input to the face coordinate detector 20, and the face coordinate detector 20 performs face alignment on the face pattern. Among them, the face coordinate detector 20 performs geometric processing such as translation, scaling, or two-dimensional/three-dimensional rotation on the facial figure according to the coordinates of important facial features. Only the images processed by face alignment will be input into the face recognition network model 30 for face recognition. The purpose of face alignment is to avoid problems such as graphic skew or wrong proportions, which will have a negative impact on the face recognition network model 30, thereby improving the recognition accuracy. However, if the face coordinate detector 20 is to be implemented, it is necessary to configure computing resources from the system to perform: the calculation of the deep learning model of the facial facial features coordinates, the calculation of the angle of rotation of the facial figure based on the facial features coordinates, and the use of The calculated angle performs operations such as rotation on the image. For embedded platforms with relatively limited computing resources, adding additional modules to achieve face alignment will significantly reduce the overall computing efficiency of the system.

public content

In view of this, the purpose of the present disclosure is to provide a training method of a deep learning network model for face recognition. Through the training method of the present disclosure, the need for face alignment processing in the face recognition algorithm can be omitted. Among them, the present disclosure adopts Knowledge Distillation to pre-train a teacher model by using aligned face images. Next, train a student model by using the trained teacher model and the unaligned face images. Since the face images without face alignment processing are used for training, the adaptability of the student model to face images with skewed angles or wrong proportions is improved. Later, when using the student model for face recognition, the same good recognition ability can be achieved under the premise of omitting the face coordinate detector (landmark detector) in the known architecture.

An embodiment of the present disclosure provides a method for training a deep learning network for face recognition, the method comprising: using a face coordinate detector (landmark detector) to perform face alignment processing on at least one captured image, thereby outputting at least one aligned image; inputting the at least one aligned image into a teacher model to obtain a first output vector; inputting the at least one captured image into a student model corresponding to the teacher model to obtain a second output vector; and adjusting parameter settings of the student model according to the first output vector and the second output vector.

An embodiment of the present disclosure provides a device for training a deep learning network for face recognition, the device includes: a storage unit and a processing unit. The storage unit is used for storing a program code. The processing unit is used to execute the program code, so that the processing unit can perform the following operations: perform face alignment processing on at least one captured image, thereby outputting at least one aligned image; input the at least one aligned image into a teacher model, obtaining a first output vector; inputting the at least one captured image into a student model corresponding to the teacher model to obtain a second output vector; and adjusting the output vector according to the first output vector and the second output vector Parameter settings for the student model.

Description of drawings

Figure 1 shows a simplified architecture of a known deep learning network for face recognition.

FIG. 2 shows how to train a teacher model using images processed by face alignment in an embodiment of the present disclosure.

FIG. 3 shows how to use the trained teacher model and the unaligned images to train the student model in an embodiment of the present disclosure.

FIG. 4 illustrates a method for training a deep learning network for face recognition according to an embodiment of the present disclosure.

FIG. 5 shows an apparatus for training a deep learning network for face recognition according to an embodiment of the present disclosure.

Detailed ways

In the following description, numerous specific details are described in order to provide the reader with a thorough understanding of the embodiments of the present disclosure. However, one skilled in the art will understand how to practice the present disclosure without one or more of the specific details, or with other methods or components or materials, or the like. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring core concepts of the disclosure.

Reference in the specification to "an embodiment" means that a specific feature, structure or characteristic described in the embodiment may be included in at least one embodiment of the present disclosure. Therefore, appearances of "in one embodiment" in various places in this specification do not necessarily mean the same embodiment. Furthermore, the particular features, structures, or characteristics described above may be combined in any suitable form in one or more embodiments.

Please refer to FIG. 2 and FIG. 3 , which illustrate how to use the knowledge distillation technique to train a deep learning network for face recognition according to an embodiment of the present disclosure. Among them, the deep learning network for face recognition trained by the present disclosure can be used for identity recognition, which can generate a one-dimensional output vector according to an input face image, and compare the output vector with all registered vectors in the database Comparison. When the L2 distance between the output vector and a registered vector is smaller than a preset threshold, it can be determined that the input facial image matches the identity associated with the registered vector.

As shown in FIG. 2 , the embodiment of the present disclosure first trains a teacher model 110 . During the training process, one or more source images IMG_S will be input to the face detector 120, and the face detector 120 will find the part including human face features from the source image IMG_S, extract it, and output the extracted Fetch image IMG_C to face coordinate detector 130 . The facial coordinate detector 130 will identify the coordinates of important facial features (such as eyes, ears, mouth, nose, etc.) in the captured image IMG_C, and perform face alignment as required. For example, when the face image in the captured image IMG_C has problems such as skewed angles or incorrect proportions, the face coordinate detector 130 will perform geometric processing such as translation, scaling, or 2D/3D rotation on the captured image IMG_C. . Accordingly, the face coordinate detector 130 inputs the aligned image IMG_A after the face alignment process to the teacher model 110 . After the aligned image IMG_A is input to the teacher model 110 , the teacher model 110 will generate an output vector 140 . The output vector 140 will be compared with the label (label) information corresponding to the source image IMG_S (ie, the identity category that the source image IMG_S essentially corresponds to), thereby generating a loss function 150 (ie, identification loss (identification loss)) . The parameter setting of the teacher model 110 will be adjusted according to the current loss function 150 , so as to realize the training of the teacher model 110 . After training the teacher model 110 using a large number of different source images IMG_S such that the loss function 150 is lower than a predetermined value, the training of the teacher model 110 can be completed. Next, a simplified student model (student model) 210 is extracted from the trained teacher model 110 based on knowledge distillation techniques. Compared with the teacher model 110, the student model 210 has a simpler structure and lower computational complexity, and occupies a lower proportion of the overall computing resources of the system. Since the student model 210 is distilled from the teacher model 110 , it has a recognition capability substantially similar to that of the teacher model 110 .

Please refer to FIG. 3, which illustrates how the present disclosure trains a student model. Wherein, the face detector 120 extracts facial images from one or more source images IMG_S, so as to generate the captured image IMG_C. The captured image IMG_C is directly input to the student model 210 without face alignment processing. The student model 210 generates an output vector 240 according to the captured image IMG_C. At the same time, the captured image IMG_C is also processed by the facial coordinate detector 130 to generate an aligned image IMG_A. The aligned image IMG_A is input to the teacher model 110 to generate a corresponding output vector 145 . Based on the difference between the output vector 145 and the output vector 240 (eg, L2 distance), a corresponding loss function 250 can be obtained. According to the loss function 250, the parameter settings of the student model 210 can be adjusted. On the other hand, the output vector 240 is also compared with the tag information associated with the source image IMG_S. Based on the difference between the two (eg, recognition loss), another loss function 260 will be generated. According to the loss function 260, the parameters of the student model 210 may also be adjusted. Through the loss functions 250 and 260, the training of the student model 210 can be realized. When the student model 210 is trained by using a large number of different source images IMG_S such that the loss functions 250 and 260 are lower than respective predetermined values, the training of the student model 210 can be completed. Please note that during the process of training the student model 210, the teacher model 110 is only used as an inference (Inference only), and its parameter settings will not be adjusted during this period.

FIG. 4 shows a method for training a deep learning network for face recognition according to an embodiment of the present disclosure. As shown in the figure, the disclosed training method includes the following simplified process:

S310: Use a face coordinate detector to perform face alignment processing on at least one captured image, so as to output at least one aligned image;

S320: Input the at least one aligned image into a teacher model to obtain a first output vector; input the at least one captured image into a student model corresponding to the teacher model to obtain a second output vector; and

S330: Adjust parameter settings of the student model according to the first output vector and the second output vector.

Since the principles and specific details of the above steps have been described in detail in the previous embodiments, they will not be repeated here. It should be noted that the above process may be able to better realize the training of the face recognition network model and further improve its recognition ability by adding other additional steps or making appropriate changes and adjustments. Furthermore, all the operations in the aforementioned embodiments of the present disclosure can be implemented by the apparatus 400 shown in FIG. 5 . Wherein, the storage unit 410 in the device 400 can be used to store program codes, instructions, variables or data. The hardware processing unit 420 in the device 400 can execute the program codes and instructions stored in the storage unit 410 , and perform all the operations in the foregoing embodiments with reference to the variables or data therein.

In summary, the present disclosure provides a method for training a deep learning network model for face recognition. Through the training method of the present disclosure, the need for face alignment processing in the face recognition algorithm can be omitted. Among them, the present disclosure uses face images that have been processed for face alignment to pre-train the teacher model. Then, a student model is trained by using the trained teacher model and the face images that have not been processed for face alignment. Since the face images without face alignment processing are used for training, the adaptability of the student model to face images with skewed angles or wrong proportions is improved. Later, when the student model performs face recognition, the same good recognition ability can be achieved under the premise of omitting the face coordinate detector in the known architecture. In this way, the present disclosure effectively reduces the occupancy ratio of the facial recognition network model to system computing resources.

Embodiments of the present disclosure can be implemented using hardware, software, firmware, and related combinations thereof. With an appropriate instruction execution system, the embodiments of the present disclosure may be implemented using software or firmware stored in a memory. As far as hardware is concerned, it can be implemented by applying any of the following technologies or related combinations: an individual arithmetic logic with logic gates that can perform logic functions according to data signals, an application-specific integrated circuit with suitable combinational logic gates ( application specific integrated circuit (ASIC), programmable gate array (programmable gate array, PGA) or a field programmable gate array (field programmable gate array, FPGA), etc.

The procedures and blocks in the flowcharts in the specification show the architecture, functions and operations that can be realized by the systems, methods and computer software products based on various embodiments of the present disclosure. In this regard, each block in the flowchart or functional block diagram may represent a module, section, or portion of program code, which includes one or more executable instructions for implementing the specified logical function. In addition, each block in the functional block diagrams and/or flowcharts, as well as combinations of blocks, can basically be realized by a dedicated hardware system for performing specified functions or actions, or a combination of dedicated hardware and computer program instructions. These computer program instructions can also be stored in a computer-readable medium, and the medium can make a computer or other programmable data processing device work in a specific manner, so that the instructions stored in the computer-readable medium can realize the flowchart and/or the functional block The function/action specified by the block in the figure.

The above descriptions are only preferred embodiments of the present disclosure, and all equivalent changes and modifications made according to the claims of the present disclosure shall fall within the scope of the present disclosure.

【Symbol Description】

10, 120: face detector

20, 130: Face coordinate detector

30: Facial Recognition Network Model

110: Teacher Model

210: Student Model

140, 145, 240: output vector

150, 250, 260: loss function

IMG_S: Source image

IMG_C: capture image

IMG_A: Align the image

S310～S330: steps

400: device

410: storage unit

420: a hardware processing unit.

Claims (10)

1. A method for training a deep learning network for face recognition, comprising:

performing face alignment processing on the at least one captured image using a face coordinate detector, thereby outputting at least one aligned image;

inputting the at least one alignment image into a teacher model to obtain a first output vector;

inputting the at least one captured image into a student model corresponding to the teacher model to obtain a second output vector; and

and adjusting the parameter setting of the student model according to the first output vector and the second output vector.

2. The method of claim 1, wherein the step of performing face alignment on the at least one captured image comprises:

performing two-dimensional/three-dimensional rotation, scaling or translation on the at least one captured image to obtain the at least one aligned image.

3. The method of claim 1, wherein the method further comprises:

the at least one captured image is captured from the source image using a face detector.

4. The method of claim 1, wherein the method further comprises:

using the face coordinate detector to perform face alignment processing on the plurality of captured images, thereby outputting a plurality of aligned images;

inputting the plurality of alignment images into the teacher model to obtain a plurality of third output vectors;

calculating a first loss function between the plurality of third output vectors and the label information associated with the plurality of captured images respectively; and

and adjusting the parameter setting of the teacher model according to the first loss function.

5. The method of claim 4, wherein the step of inputting the at least one alignment image into the teacher model to obtain the first output vector comprises:

inputting the at least one alignment image into the teacher model adjusted according to the first loss function to obtain the first output vector.

6. The method of claim 1, wherein the step of adjusting the parameter settings of the student model comprises:

calculating a second penalty function between the first output vector and the second output vector; and

and adjusting the parameter setting of the student model according to the second loss function.

7. The method of claim 1, wherein the method further comprises:

calculating a third loss function of the second output vector and label information associated with the at least one captured image; and

and adjusting the parameter setting of the student model according to the third loss function.

8. An apparatus for training a deep learning network for face recognition, comprising:

a storage unit for storing a program code;

a processing unit, configured to execute the program code, such that the processing unit can perform the following operations:

performing face alignment processing on at least one captured image, thereby outputting at least one aligned image;

inputting the at least one alignment image into a teacher model to obtain a first output vector;

inputting the at least one captured image into a student model corresponding to the teacher model to obtain a second output vector; and

and adjusting the parameter setting of the student model according to the first output vector and the second output vector.

9. The apparatus of claim 8, wherein the processing unit executes the program code to:

performing two-dimensional/three-dimensional rotation, scaling or translation on the at least one captured image to obtain the at least one aligned image.

10. The apparatus of claim 8, wherein the processing unit executes the program code to:

performing face alignment processing on the plurality of captured images, thereby outputting a plurality of aligned images;

inputting the plurality of alignment images into the teacher model to obtain a plurality of third output vectors;

calculating a first loss function between the plurality of third output vectors and the label information associated with the plurality of captured images respectively; and

and adjusting the parameter setting of the teacher model according to the first loss function.

Images