HK1246921B - Face shielding detection method, device and storage medium - Google Patents

Description

Face occlusion detection method, device and storage medium

Technical Field

The present invention relates to the field of computer vision processing technology, and in particular to a face occlusion detection method, device and computer-readable storage medium.

Background Art

Face recognition is a biometric technology that authenticates people based on their facial features. It collects images or video streams containing faces, detects and tracks faces in the images, and then matches and identifies the detected faces. At present, face recognition has a wide range of applications, playing a very important role in many fields such as financial payment, access control and attendance, and identity recognition, bringing great convenience to people's lives. However, it is crucial to ensure that the face is not blocked, so before performing face recognition, it is necessary to detect whether the face in the image is blocked.

Products in the industry generally determine face occlusion through deep learning training, but this judgment method has high requirements on sample size, and if deep learning is used to predict occlusion, the amount of calculation is large and the speed is relatively slow.

Summary of the invention

The present invention provides a face occlusion detection method, device and computer-readable storage medium, the main purpose of which is to quickly detect face occlusion in real-time facial images.

To achieve the above object, the present invention provides an electronic device, which includes: a memory, a processor and a camera device, wherein the memory includes a face occlusion detection program, and when the face occlusion detection program is executed by the processor, the following steps are implemented:

Image acquisition step: acquiring a real-time image captured by a camera device, and extracting a real-time facial image from the real-time image using a face recognition algorithm;

Feature point recognition step: input the real-time facial image into a pre-trained facial average model, and use the facial average model to recognize t facial feature points from the real-time facial image; and

Feature area judgment step: determine the eye area and the lip area according to the position information of the t facial feature points, input the eye area and the lip area into a pre-trained face eye classification model and a face lip classification model, judge the authenticity of the eye area and the lip area, and judge whether the face in the real-time image is occluded based on the judgment result.

Optionally, when the face occlusion detection program is executed by the processor, the following steps are further implemented:

Judgment step: judging whether the judgment results of the eye area and the lip area of the face by the eye classification model and the lip classification model are both true.

Optionally, when the face occlusion detection program is executed by the processor, the following steps are further implemented:

When the judgment results of the eye classification model and the lip classification model for the eye region and the lip region are both true, it is judged that the face in the real-time facial image is not blocked; and

When the judgment results of the eye area and the lip area by the eye classification model and the lip classification model of the face contain untrue information, it is prompted that the face in the real-time facial image is occluded.

In addition, to achieve the above purpose, the present invention also provides a face occlusion detection method, the method comprising:

Image acquisition step: acquiring a real-time image captured by a camera device, and extracting a real-time facial image from the real-time image using a face recognition algorithm;

Feature point recognition step: input the real-time facial image into a pre-trained facial average model, and use the facial average model to recognize t facial feature points from the real-time facial image; and

Feature area judgment step: determine the eye area and the lip area according to the position information of the t facial feature points, input the eye area and the lip area into a pre-trained face eye classification model and a face lip classification model, judge the authenticity of the eye area and the lip area, and judge whether the face in the real-time image is occluded based on the judgment result.

Optionally, the method further comprises:

Judgment step: judging whether the judgment results of the eye area and the lip area of the face by the eye classification model and the lip classification model are both true.

Optionally, the method further comprises:

When the judgment results of the eye classification model and the lip classification model for the eye region and the lip region are both true, it is judged that the face in the real-time facial image is not blocked; and

When the judgment results of the eye area and the lip area by the eye classification model and the lip classification model of the face contain untrue information, it is prompted that the face in the real-time facial image is occluded.

In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, which includes a face occlusion detection program. When the face occlusion detection program is executed by a processor, any step in the face occlusion detection method as described above is implemented.

The face occlusion detection method, electronic device and computer-readable storage medium proposed in the present invention input a real-time face image into a face average model, identify facial feature points in the real-time face image, use a face eye classification model and a face lip classification model to judge the authenticity of the eye area and the lip area determined by the facial feature points, and judge whether the face in the real-time face image is occluded based on the authenticity of the eye area and the lip area.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of an application environment of a preferred embodiment of a face occlusion detection method of the present invention;

FIG2 is a functional module diagram of the face occlusion detection program in FIG1 ;

FIG. 3 is a flow chart of a preferred embodiment of a face occlusion detection method of the present invention.

The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

The present invention provides a method for detecting face occlusion, which is applied to an electronic device 1. Referring to Fig. 1, it is a schematic diagram of an application environment of a preferred embodiment of the method for detecting face occlusion of the present invention.

In this embodiment, the electronic device 1 may be a terminal device with computing functions, such as a server, a smart phone, a tablet computer, a portable computer, or a desktop computer.

The electronic device 1 includes: a processor 12, a memory 11, a camera 13, a network interface 14 and a communication bus 15. The camera 13 is installed in a specific place, such as an office place or a monitoring area, and a real-time image is obtained by shooting a target entering the specific place in real time, and the real-time image is transmitted to the processor 12 through a network. The network interface 14 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The communication bus 15 is used to realize the connection and communication between these components.

The memory 11 includes at least one type of readable storage medium. The at least one type of readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card-type memory, etc. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 1, such as a hard disk of the electronic device 1. In other embodiments, the readable storage medium may also be an external memory of the electronic device 1, such as a plug-in hard disk equipped on the electronic device 1, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc.

In this embodiment, the readable storage medium of the memory 11 is generally used to store the face occlusion detection program 10 installed in the electronic device 1, the face image sample library, the eye sample library, the lip sample library, the constructed and trained face average model of facial feature points, the eye classification model and the lip classification model of the face, etc. The memory 11 can also be used to temporarily store data that has been output or is to be output.

In some embodiments, the processor 12 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 11, such as executing the face occlusion detection program 10.

FIG. 1 only shows an electronic device 1 having components 11 - 15 , but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

Optionally, the electronic device 1 may further include a user interface, which may include an input unit such as a keyboard, a voice input device such as a microphone and other devices with voice recognition function, a voice output device such as a speaker, headphones, etc. Optionally, the user interface may also include a standard wired interface or a wireless interface.

Optionally, the electronic device 1 may further include a display, which may also be appropriately referred to as a display screen or a display unit. In some embodiments, it may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display is used to display information processed in the electronic device 1 and to display a visual user interface.

Optionally, the electronic device 1 further includes a touch sensor. The area provided by the touch sensor for the user to perform a touch operation is called a touch control area. In addition, the touch sensor described here may be a resistive touch sensor, a capacitive touch sensor, etc. Moreover, the touch sensor includes not only a contact touch sensor, but also a proximity touch sensor, etc. In addition, the touch sensor may be a single sensor or a plurality of sensors arranged in an array, for example.

In addition, the area of the display of the electronic device 1 can be the same as or different from the area of the touch sensor. Optionally, the display and the touch sensor are stacked to form a touch display screen. The device detects touch operations triggered by the user based on the touch display screen.

Optionally, the electronic device 1 may further include an RF (Radio Frequency) circuit, a sensor, an audio circuit, etc., which will not be described in detail here.

In the device embodiment shown in FIG1 , the memory 11 as a computer storage medium may include an operating system and a face occlusion detection program 10; when the processor 12 executes the face occlusion detection program 10 stored in the memory 11, the following steps are implemented:

A real-time image captured by the camera device 13 is acquired, and the processor 12 extracts a real-time facial image from the real-time image by using a facial recognition algorithm, calls a facial average model, an eye classification model of a face, and a lip classification model of a face from the memory 11, inputs the real-time facial image into the facial average model, identifies facial feature points in the real-time facial image, inputs the eye area and the lip area determined by the facial feature points into the eye classification model and the lip classification model of the face, and determines whether the face in the real-time facial image is occluded by judging the authenticity of the eye area and the lip area.

In other embodiments, the face occlusion detection program 10 may be divided into one or more modules, one or more modules are stored in the memory 11, and executed by the processor 12 to complete the present invention. The module referred to in the present invention refers to a series of computer program instruction segments that can complete specific functions.

2 , which is a functional module diagram of the face occlusion detection program 10 in FIG. 1 .

The face occlusion detection program 10 can be divided into: an acquisition module 110 , a recognition module 120 , a determination module 130 and a prompt module 140 .

The acquisition module 110 is used to acquire the real-time image captured by the camera device 13 and extract a real-time facial image from the real-time image using a facial recognition algorithm. When the camera device 13 captures a real-time image, the camera device 13 sends the real-time image to the processor 12. When the processor 12 receives the real-time image, the acquisition module 110 extracts the real-time facial image using a facial recognition algorithm.

Specifically, the face recognition algorithm for extracting the real-time facial image from the real-time image may be a method based on geometric features, a local feature analysis method, a feature face method, a method based on an elastic model, a neural network method, and the like.

The recognition module 120 is used to input the real-time facial image into a pre-trained facial average model, and use the facial average model to identify t facial feature points from the real-time facial image. Assume that t=34, among the 27 facial feature points in the facial average model, there are 12 eye socket feature points, 2 eyeball feature points, and 20 lip feature points. After the acquisition module 110 extracts the real-time facial image, the recognition module 120 calls the trained facial average model of facial feature points from the memory 11, aligns the real-time facial image with the facial average model, and then uses the feature extraction algorithm to search for 12 eye socket feature points, 2 eyeball feature points, and 20 lip feature points that match the 27 facial feature points of the facial average model in the real-time facial image. Among them, the facial average model of facial feature points is pre-constructed and trained, and the specific implementation method will be described in the following face occlusion detection method.

In this embodiment, the feature extraction algorithm is a SIFT (scale-invariant feature transform) algorithm. The SIFT algorithm extracts the local features of each facial feature point from the average facial model of the facial feature points, such as 12 orbital feature points, 2 eyeball feature points, and 20 lip feature points, and selects an eye feature point or a lip feature point as a reference feature point, and searches for feature points that are the same or similar to the local features of the reference feature point in the real-time facial image. For example, whether the difference between the local features of the two feature points is within a preset range, if so, it indicates that the feature point is the same or similar to the local features of the reference feature point, and it is used as an eye feature point or a lip feature point. This principle is followed until all facial feature points are found in the real-time facial image. In other embodiments, the feature extraction algorithm can also be a SURF (Speeded Up Robust Features) algorithm, an LBP (Local Binary Patterns) algorithm, a HOG (Histogram of Oriented Gridients) algorithm, etc.

The judgment module 130 is used to determine the eye area and the lip area according to the position information of the t facial feature points, input the eye area and the lip area into the pre-trained eye classification model and the lip classification model of the face, judge the authenticity of the eye area and the lip area, and judge whether the face in the real-time image is blocked according to the judgment result. When the recognition module 120 recognizes 12 eye socket feature points, 2 eyeball feature points, and 20 lip feature points from the real-time facial image, an eye area can be determined according to the 12 eye socket feature points and the 2 eyeball feature points, and a lip area can be determined according to the 20 lip feature points, and then the determined eye area and lip area are input into the trained eye classification model and the lip classification model of the face, and the authenticity of the determined eye area and lip area is judged according to the results obtained by the model, that is, the results output by the model may be all false, all true, or both true and false. When the results output by the eye classification model and the lip classification model are both false, it means that the eye area and the lip area are not the eye area and the lip area of a person; when the results output by the eye classification model and the lip classification model are both true, it means that the eye area and the lip area are the eye area and the lip area of a person. The eye classification model and the lip classification model are pre-built and trained, and the specific implementation method will be described in the following face occlusion detection method.

Specifically, the judgment module 130 is further used to judge whether the judgment results of the eye classification model and the lip classification model of the face are both true for the eye area and the lip area. After the eye classification model and the lip classification model of the face output the results, it is judged whether the results only contain true.

The judgment module 130 is further configured to judge that the face in the real-time facial image is not blocked when the judgment results of the eye classification model and the lip classification model for the face are both true for the eye region and the lip region. In other words, when the eye region and the lip region determined according to the facial feature points are both the eye region or the lip region of a person, it is considered that the face in the real-time facial image is not blocked.

The prompt module 140 is used to prompt that the face in the real-time facial image is occluded when the judgment results of the eye classification model and the lip classification model of the face include untrue results for the eye area and the lip area. When any one of the eye area and the lip area determined according to the facial feature points is not the eye area or the lip area of the person, it is considered that the face in the real-time facial image is occluded, and the prompt module 140 prompts that the face in the real-time facial image is occluded.

Furthermore, when the output result of the eye classification model of the face is false, it is considered that the eye area in the image is occluded; when the output result of the lip classification model of the face is false, it is considered that the lip area in the image is occluded, and a corresponding prompt is given.

In other embodiments, if subsequent face recognition is performed after detecting whether the face is obscured, then when the face in the real-time facial image is obscured, the prompt module 140 is also used to prompt that the face in the current facial image is obscured, and the acquisition module re-acquires the real-time image captured by the camera device and performs subsequent steps.

The electronic device 1 proposed in this embodiment extracts a real-time facial image from a real-time image, identifies facial feature points in the real-time facial image using a facial average model, analyzes the eye area and lip area determined by the facial feature points using a face eye classification model and a face lip classification model, and quickly determines whether the face in the current image is occluded based on the authenticity of the eye area and the lip area.

In addition, the present invention also provides a face occlusion detection method. Referring to Figure 3, it is a flow chart of a first embodiment of the face occlusion detection method of the present invention. The method can be executed by a device, and the device can be implemented by software and/or hardware.

In this embodiment, the face occlusion detection method includes:

Step S10, obtaining a real-time image captured by the camera device, and extracting a real-time facial image from the real-time image using a face recognition algorithm. When the camera device captures a real-time image, the camera device sends the real-time image to the processor, and when the processor receives the real-time image, the real-time facial image is extracted using a face recognition algorithm.

Specifically, the face recognition algorithm for extracting the real-time facial image from the real-time image may be a method based on geometric features, a local feature analysis method, a feature face method, a method based on an elastic model, a neural network method, and the like.

Step S20: input the real-time facial image into a pre-trained facial average model, and use the facial average model to identify t facial feature points from the real-time facial image.

A first sample library with n face images is established, and t facial feature points are marked in each face image, wherein the t facial feature points include: t ₁ eye socket feature points representing eye positions, t ₂ eyeball feature points, and t ₃ lip feature points representing lip positions. In each face image in the first sample library, t ₁ eye socket feature points, t ₂ eyeball feature points, and t ₃ lip feature points are manually marked, and the (t ₁ +t ₂ +t ₃ ) feature points in each face image form a shape feature vector S, thereby obtaining n shape feature vectors S of the face.

The face feature recognition model is trained using the t facial feature points to obtain a facial average model. The face feature recognition model is an Ensemble of Regression Tress (ERT) algorithm. The ERT algorithm is expressed in the following formula:

Where t represents the cascade number, and τ _t (·,·) represents the regressor of the current level. Each regressor consists of many regression trees, and the purpose of training is to obtain these regression trees.

Where is the shape estimate of the current model; each regressor τ _t (·,·) predicts an increment based on the input image I and adds this increment to the current shape estimate to improve the current model. Each level of regressor makes predictions based on feature points. The training data set is: (I1,S1),...,(In,Sn) where I is the input sample image and S is the shape feature vector composed of feature points in the sample image.

In the process of model training, the number of face images in the first sample library is t. Assuming that each sample image has 34 feature points, feature vectors _x1 to _x12 represent the horizontal coordinates of the eye socket feature points, _x13 to _x14 represent the horizontal coordinates of the eyeball feature points, and _x15 to _x34 represent the horizontal coordinates of the lip feature points. The feature vector S formed by taking some feature points of all sample images (for example, randomly taking 25 feature points from the 34 feature points of each sample image) is trained to train the first regression tree, and the residual between the predicted value of the first regression tree and the true value of the partial feature points (the weighted average of the 25 feature points taken from each sample image) is used to train the second tree... and so on, until the residual between the predicted value of the Nth tree and the true value of the partial feature points is close to 0, and all regression trees of the ERT algorithm are obtained, and the facial average model (mean shape) is obtained according to these regression trees, and the model file and the sample library are saved in the memory. Because the samples of the training model are marked with 12 eye socket feature points, 2 eyeball feature points and 20 lip feature points, the trained facial average model of the face can be used to identify 12 eye socket feature points, 2 eyeball feature points and 20 lip feature points from the face image.

After acquiring the real-time facial image, the trained average facial model is called from the memory, the real-time facial image is aligned with the average facial model, and the feature extraction algorithm is used to search for 12 orbital feature points, 2 eyeball feature points and 20 lip feature points that match the 12 orbital feature points, 2 eyeball feature points and 20 lip feature points of the average facial model in the real-time facial image. Among them, the 20 lip feature points are evenly distributed on the lips.

Step S30, determining the eye area and the lip area according to the position information of the t facial feature points, inputting the eye area and the lip area into a pre-trained face eye classification model and a face lip classification model, judging the authenticity of the eye area and the lip area, and judging whether the face in the real-time image is occluded according to the judgment result.

A first number of positive sample images of human eyes and a second number of negative sample images of human eyes are collected, and local features of each positive sample image of human eyes and negative sample image of human eyes are extracted. Positive sample images of human eyes refer to eye samples containing human eyes, and both eyes can be cut out from the face image sample library as eye samples. Negative eye sample images of human eyes refer to images with incomplete eye areas. Multiple positive sample images of human eyes and negative sample images form the second sample library.

A third number of lip positive sample images and a fourth number of lip negative sample images are collected, and local features of each lip positive sample image and each lip negative sample image are extracted. A lip positive sample image refers to an image containing human lips, and the lip part can be cut out from the face image sample library as a lip positive sample image. A lip negative sample image refers to an image in which the lip area of a person is incomplete, or the lips in the image are not the lips of a human (such as an animal). Multiple lip positive sample images and negative sample images form a third sample library.

Specifically, the local feature is a Histogram of Oriented Gradient (HOG) feature, which is extracted from the human eye sample image and the lip sample image by a feature extraction algorithm. Since the color information in the sample image is not very useful, it is usually converted into a grayscale image, and the entire image is normalized, the gradients in the horizontal and vertical directions of the image are calculated, and the gradient direction value of each pixel position is calculated accordingly to capture the contour, human shadow and some texture information, and further weaken the influence of illumination. Then the entire image is divided into individual Cell cells (8*8 pixels), and a gradient direction histogram is constructed for each Cell cell to count the local image gradient information and quantize it to obtain the feature description vector of the local image area. Then the cell cells are combined into large blocks. Due to the change of local illumination and the change of foreground-background contrast, the range of gradient intensity is very large, which requires normalization of the gradient intensity, and further compression of illumination, shadow and edge. Finally, all the HOG descriptors of the "block" are combined together to form the final HOG feature description vector.

The positive and negative sample images in the second sample library and the third sample library and the extracted HOG features are used to train a support vector machine classifier (SVM) to obtain the eye classification model and the lip classification model of the human face.

After identifying 12 eye socket feature points, 2 eyeball feature points, and 20 lip feature points from the real-time facial image, an eye region can be determined based on the 12 eye socket feature points and the 2 eyeball feature points, and a lip region can be determined based on the 20 lip feature points, and then the determined eye region and lip region are input into the trained eye classification model and the lip classification model of the face, and the authenticity of the determined eye region and lip region is judged based on the results obtained by the model, that is, the results output by the model may be all false, all true, or both true and false. When the results output by the eye classification model and the lip classification model of the face are both false, it means that the eye region and the lip region are not the eye region and the lip region of a person; when the results output by the eye classification model and the lip classification model of the face are both true, it means that the eye region and the lip region are the eye region and the lip region of a person.

Step S40, judging whether the judgment results of the eye classification model and the lip classification model for the eye area and the lip area are both true. After the eye classification model and the lip classification model for the face output the results, judging whether the results only contain true.

Step S50, when the eye classification model and the lip classification model of the face have judgment results that the eye region and the lip region are both true, it is determined that the face in the real-time facial image is not blocked. In other words, when the eye region and the lip region determined based on the facial feature points are both the eye region or the lip region of a person, it is considered that the face in the real-time facial image is not blocked.

Step S60: When the eye classification model and the lip classification model of the face contain untrue judgment results for the eye area and the lip area, it is prompted that the face in the real-time facial image is occluded. When any one of the eye area and the lip area determined according to the facial feature points is not the eye area or the lip area of the person, it is considered that the face in the real-time facial image is occluded, and it is prompted that the face in the real-time facial image is occluded.

Furthermore, when the output result of the eye classification model of the face is false, it is considered that the eye area in the image is occluded; when the output result of the lip classification model of the face is false, it is considered that the lip area in the image is occluded, and a corresponding prompt is given.

In other embodiments, if subsequent face recognition is performed after detecting whether the face is blocked, when the face in the real-time facial image is blocked, step S50 further includes:

It is prompted that the face in the current facial image is blocked, the acquisition module reacquires the real-time image captured by the camera device, and performs subsequent steps.

The face occlusion detection method proposed in this embodiment uses the facial average model of facial feature points to identify the key facial feature points in the real-time facial image, uses the eye classification model of the face and the lip classification model of the face to analyze the eye area and the lip area determined by the feature points, and judges whether the face in the current image is occluded based on the authenticity of the eye area and the lip area, so as to quickly detect the occlusion of the face in the real-time facial image.

In addition, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a face occlusion detection program, and when the face occlusion detection program is executed by a processor, the following operations are implemented:

Image acquisition step: acquiring a real-time image captured by a camera device, and extracting a real-time facial image from the real-time image using a face recognition algorithm;

Feature point recognition step: input the real-time facial image into a pre-trained facial average model, and use the facial average model to recognize t facial feature points from the real-time facial image; and

Feature area judgment step: determine the eye area and the lip area according to the position information of the t facial feature points, input the eye area and the lip area into a pre-trained face eye classification model and a face lip classification model, judge the authenticity of the eye area and the lip area, and judge whether the face in the real-time image is occluded based on the judgment result.

Optionally, when the face occlusion detection program is executed by the processor, the following operations are also implemented:

Judgment step: judging whether the judgment results of the eye area and the lip area of the face by the eye classification model and the lip classification model are both true.

Optionally, when the face occlusion detection program is executed by the processor, the following operations are also implemented:

When the judgment results of the eye classification model and the lip classification model for the eye region and the lip region are both true, it is judged that the face in the real-time facial image is not blocked; and

When the judgment results of the eye area and the lip area by the eye classification model and the lip classification model of the face contain untrue information, it is prompted that the face in the real-time facial image is occluded.

Optionally, the training step of the facial average model includes:

Establishing a first sample library with n face images, marking t facial feature points in each face image, wherein the t facial feature points include: _t1 eye socket feature points representing eye positions, _t2 eyeball feature points, and _t3 lip feature points representing lip positions; and

Using the t facial feature points to train a facial feature recognition model to obtain a facial average model;

Optionally, the steps of training the eye classification model and the lip classification model of the face include:

Collecting a first number of positive sample images of human eyes and a second number of negative sample images of human eyes, and extracting local features of each positive sample image of human eyes and each negative sample image of human eyes;

The support vector classifier (SVM) is trained using positive sample images of human eyes, negative sample images of human eyes and their local features to obtain a face eye classification model;

Collecting a third number of lip positive sample images and a fourth number of lip negative sample images, and extracting local features of each lip positive sample image and each lip negative sample image; and

The lip positive sample images, lip negative sample images and their local features are used to train a support vector classifier (SVM) to obtain a lip classification model for the face.

The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the specific implementation of the above-mentioned face occlusion detection method, and will not be described in detail here.

It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. Through the description of the above implementation modes, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims (9)

1. An electronic device, characterized in that the device comprises: a memory, a processor and a camera device, the memory comprises a face occlusion detection program, and the face occlusion detection program, when executed by the processor, implements the following steps: Image acquisition step: acquiring a real-time image captured by a camera device, and extracting a real-time facial image from the real-time image using a face recognition algorithm; Feature point recognition step: input the real-time facial image into a pre-trained facial average model, and use the facial average model to recognize t facial feature points from the real-time facial image. The training steps of the facial average model include: Establish a first sample library with n face images, mark t facial feature points in each face image, the t facial feature points include: t ₁ orbital feature points representing eye positions, t ₂ eyeball feature points and t ₃ lip feature points representing lip positions, the t feature points in each face image form a shape feature vector S, and obtain a training data set (I1, S1), ..., (In, Sn), where I is a face image, and S is a shape feature vector composed of feature points in the face image; and The training data set is used to train the face feature recognition model to obtain the face average model. The face feature recognition model is the ERT algorithm, and the formula is: where t represents the cascade number, τ _t (·, ·) represents the regressor of the current level, each regressor is composed of multiple regression trees, and is the shape estimation of the current model; each regressor τ _t (·, ·) predicts an increment according to the input image I and In the process of model training, a first regression tree is trained according to the feature vector S composed of the t facial feature points in the face image in the sample library, and the residual between the predicted value of the first regression tree and the true value of the t facial feature points is used to train the second tree; and so on, until the residual between the predicted value of the trained Nth tree and the true value of the t facial feature points is close to 0, all regression trees of the ERT algorithm are obtained, and the facial average model is obtained according to the regression trees; and Feature area judgment step: determine the eye area and the lip area according to the position information of the t facial feature points, input the eye area and the lip area into a pre-trained face eye classification model and a face lip classification model, judge the authenticity of the eye area and the lip area, and judge whether the face in the real-time image is occluded based on the judgment result. 2. The electronic device according to claim 1, wherein when the face occlusion detection program is executed by the processor, the following steps are further implemented: Judgment step: judging whether the judgment results of the eye area and the lip area of the face by the eye classification model and the lip classification model are both true. 3. The electronic device according to claim 1 or 2, characterized in that when the face occlusion detection program is executed by the processor, the following steps are further implemented: When the judgment results of the eye classification model and the lip classification model for the eye region and the lip region are both true, it is judged that the face in the real-time facial image is not blocked; and When the judgment results of the eye area and the lip area by the eye classification model and the lip classification model of the face contain untrue information, it is prompted that the face in the real-time facial image is occluded. 4. The electronic device according to claim 1, wherein the training steps of the eye classification model and the lip classification model of the face include: Collecting a first number of positive sample images of human eyes and a second number of negative sample images of human eyes, and extracting local features of each positive sample image of human eyes and each negative sample image of human eyes; The support vector classifier (SVM) is trained using positive sample images of human eyes, negative sample images of human eyes and their local features to obtain a face eye classification model; Collecting a third number of lip positive sample images and a fourth number of lip negative sample images, and extracting local features of each lip positive sample image and each lip negative sample image; and The lip positive sample images, lip negative sample images and their local features are used to train a support vector classifier (SVM) to obtain a lip classification model for the face. 5. A face occlusion detection method, characterized in that the method comprises: Image acquisition step: acquiring a real-time image captured by a camera device, and extracting a real-time facial image from the real-time image using a face recognition algorithm; Feature point recognition step: input the real-time facial image into a pre-trained facial average model, and use the facial average model to identify t facial feature points from the real-time facial image. The training step of the facial average model includes: establishing a first sample library with n facial images, marking t facial feature points in each facial image, the t facial feature points including: _t1 orbital feature points representing eye positions, _t2 eyeball feature points and _t3 lip feature points representing lip positions, the t feature points in each facial image form a shape feature vector S, and obtain a training data set (I1, S1), ..., (In, Sn), where I is a facial image and S is a shape feature vector composed of feature points in the facial image; and The training data set is used to train the face feature recognition model to obtain the face average model. The face feature recognition model is the ERT algorithm, and the formula is: where t represents the cascade number, τ _t (·, ·) represents the regressor of the current level, each regressor is composed of multiple regression trees, and is the shape estimation of the current model; each regressor τ _t (·, ·) predicts an increment according to the input image I and In the process of model training, a first regression tree is trained according to the feature vector S composed of the t facial feature points in the face image in the sample library, and the residual between the predicted value of the first regression tree and the true value of the t facial feature points is used to train the second tree; and so on, until the residual between the predicted value of the trained Nth tree and the true value of the t facial feature points is close to 0, all regression trees of the ERT algorithm are obtained, and the facial average model is obtained according to the regression trees; and Feature area judgment step: determine the eye area and the lip area according to the position information of the t facial feature points, input the eye area and the lip area into a pre-trained face eye classification model and a face lip classification model, judge the authenticity of the eye area and the lip area, and judge whether the face in the real-time image is occluded based on the judgment result. 6. The face occlusion detection method according to claim 5, characterized in that the method further comprises: Judgment step: judging whether the judgment results of the eye area and the lip area of the face by the eye classification model and the lip classification model are both true. 7. The face occlusion detection method according to claim 5 or 6, characterized in that the method further comprises: When the judgment results of the eye classification model and the lip classification model for the eye region and the lip region are both true, it is judged that the face in the real-time facial image is not blocked; and When the judgment results of the eye area and the lip area by the eye classification model and the lip classification model of the face contain untrue information, it is prompted that the face in the real-time facial image is occluded. 8. The face occlusion detection method according to claim 5, wherein the training steps of the eye classification model and the lip classification model of the face include: Collecting a first number of positive sample images of human eyes and a second number of negative sample images of human eyes, and extracting local features of each positive sample image of human eyes and each negative sample image of human eyes; The support vector classifier (SVM) is trained using positive sample images of human eyes, negative sample images of human eyes and their local features to obtain a face eye classification model; Collecting a third number of lip positive sample images and a fourth number of lip negative sample images, and extracting local features of each lip positive sample image and each lip negative sample image; and The lip positive sample images, lip negative sample images and their local features are used to train a support vector classifier (SVM) to obtain a lip classification model for the face. 9. A computer-readable storage medium, characterized in that the computer-readable storage medium includes a face occlusion detection program, and when the face occlusion detection program is executed by a processor, the steps of the face occlusion detection method as described in any one of claims 5 to 8 are implemented.