CN111027504A - Face key point detection method, device, equipment and storage medium - Google Patents
Face key point detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111027504A CN111027504A CN201911310807.4A CN201911310807A CN111027504A CN 111027504 A CN111027504 A CN 111027504A CN 201911310807 A CN201911310807 A CN 201911310807A CN 111027504 A CN111027504 A CN 111027504A
- Authority
- CN
- China
- Prior art keywords
- face
- face image
- image
- key point
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
本申请提供一种人脸关键点检测方法、装置、设备及存储介质。该方法包括：获取包含人脸的人脸图像；通过神经网络对所述人脸图像进行关键点检测，得到所述人脸图像中人脸关键点的位置信息，其中，所述神经网络经过训练数据集训练，所述训练数据集包括多张第一人脸图像样本及其相应的标记信息，每张第一人脸图像样本中所包含的人脸存在被遮挡的部分，所述标记信息包括人脸上被遮挡部分和未被遮挡部分的关键点的位置信息。本申请能够通过标记出被遮挡部分的人脸关键点的人脸图像样本训练神经网络，使得神经网络对被部分遮挡的人脸的人脸图像也能准确检测出人脸关键点，提高人脸关键点检测的检测准确度。
The present application provides a method, device, device and storage medium for detecting key points of a face. The method includes: acquiring a face image including a face; performing key point detection on the face image through a neural network to obtain position information of the key points of the face in the face image, wherein the neural network is trained Data set training, the training data set includes a plurality of first face image samples and their corresponding marking information, the face contained in each first face image sample has an occluded part, and the marking information includes The location information of the key points of the occluded and unoccluded parts of the face. The present application can train the neural network through face image samples that mark the face key points of the occluded part, so that the neural network can accurately detect the face key points of the face image of the partially occluded face, and improve the performance of the face. Detection accuracy of keypoint detection.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting a face key point.
Background
The face key point detection refers to a technology for detecting key points such as eyes, a nose, face edges and the like on a face in a face image. The face key point detection can be applied to scenes such as face local positioning, expression recognition, intelligent driving test judgment, driving assistance and the like.
Generally, a pre-constructed face key point detection network is trained by a plurality of face image samples with key point positions marked, and then the key point positions of the face in the image to be processed are detected by the trained face key point detection network.
However, there may be occlusion in the image to be processed, for example, the turning angle of the face in the image is too large, or there is an occlusion on the face. The existing face key point detection method is adopted to detect the face image with the face sheltered, and the accuracy of the detection result is poor.
Disclosure of Invention
The embodiment of the application provides a face key point detection method, a face key point detection device, face key point detection equipment and a storage medium, and aims to solve the problem that the detection accuracy of the existing face key point detection method is poor.
In a first aspect, an embodiment of the present application provides a method for detecting a face key point, including:
acquiring a face image containing a face;
the method comprises the steps of detecting key points of a face image through a neural network to obtain position information of key points of the face in the face image, wherein the neural network is trained through a training data set, the training data set comprises a plurality of first face image samples and corresponding mark information of the first face image samples, the face contained in each first face image sample has a blocked part, and the mark information comprises the position information of the key points of the blocked part and the unblocked part on the face.
In one possible embodiment, the neural network comprises a convolutional neural network;
carrying out key point detection on the face image through a neural network to obtain the position information of the key points of the face in the face image, wherein the position information comprises the following steps:
inputting the face image into the convolutional neural network to obtain a preset number of feature maps, wherein the feature maps correspond to key points on the face in the face image one to one;
and for each feature map, determining the coordinates of the positions of the pixel points with the maximum pixel values in the feature map as the position coordinates of the key points corresponding to the feature map.
In a possible implementation, before acquiring a face image including a face, the method further includes:
acquiring a plurality of first human face image samples and marking coordinates of human face key points marked in advance by the first human face image samples;
generating a thermodynamic diagram corresponding to each key point on the face of each first face image sample according to the mark coordinate of each key point on the face of the first face image sample, wherein the pixel value of the position corresponding to the mark coordinate of the key point in the thermodynamic diagram is larger than the pixel values of the rest positions, and the pixel values of the rest positions are reduced along with the increase of the distance from the position corresponding to the mark coordinate of the key point;
and forming the training data set by each first face image and a group of corresponding thermodynamic diagrams.
In one possible embodiment, after the training data set is composed of each first face image sample and a corresponding set of thermodynamic diagrams, the method further includes:
and aiming at each first face image sample in the training data set, inputting the first face image sample into a pre-constructed neural network to obtain a plurality of predicted characteristic graphs output by the neural network, wherein the predicted characteristic graphs correspond to thermodynamic diagrams of the first face image sample one by one, and network parameters of the neural network are adjusted according to a group of thermodynamic diagrams and predicted characteristic graphs of the first face image sample.
In a possible implementation manner, a ratio of the number of key points of the portion of the first face image sample where the face is occluded to the total number of key points of the face is less than or equal to a preset threshold.
In a possible implementation manner, the training data set further includes a plurality of second face image samples and corresponding label information thereof, and each second face image sample includes a face that is not occluded.
In one possible implementation, acquiring a face image including a face includes:
acquiring an image acquired by a camera;
identifying the position information of the area where the face is located in the image;
and extracting the image of the area corresponding to the position information from the image as the face image.
In a possible implementation, identifying the location information of the region where the face is located in the image includes:
and inputting the image into a single-shot multi-frame detection SSD network to obtain the position information of the area where the face is located in the image.
In a possible implementation manner, after performing key point detection on the face image through a neural network to obtain position information of key points of the face in the face image, the method further includes:
and marking each face key point in the face image according to the position information of the face key points in the face image, and displaying the marked face image.
In a second aspect, an embodiment of the present application provides a face keypoint detection apparatus, including:
the acquisition module is used for acquiring a face image containing a face;
the processing module is used for detecting key points of the face image through a neural network to obtain position information of key points of the face in the face image, wherein the neural network is trained through a training data set, the training data set comprises a plurality of first face image samples and corresponding marking information of the first face image samples, a blocked part exists in a face contained in each first face image sample, and the marking information comprises position information of key points of the blocked part and the unblocked part on the face.
In a third aspect, an embodiment of the present application provides a face keypoint detection apparatus, including: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executes the computer-executable instructions stored by the memory, so that the at least one processor performs the face keypoint detection method as described in the first aspect and various possible implementations of the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer executing instruction is stored in the computer-readable storage medium, and when a processor executes the computer executing instruction, the method for detecting a face keypoint is implemented as described in the first aspect and various possible implementations of the first aspect.
According to the method, the device, the equipment and the storage medium for detecting the key points of the face, a face image containing the face is obtained firstly; and then carrying out key point detection on the face image through a neural network to obtain the position information of face key points in the face image, wherein the neural network is trained through a training data set, the training data set comprises a plurality of first face image samples and corresponding marking information thereof, the face contained in each first face image sample has a shielded part, the marking information comprises the position information of key points of the shielded part and the non-shielded part on the face, and the neural network can be trained through the face image samples for marking the face key points of the shielded part, so that the neural network can also accurately detect the face key points on the face image of the face shielded by the part, and the detection accuracy of the face key point detection is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a schematic diagram illustrating an architecture of a face keypoint detection system according to an embodiment of the present application;
fig. 2 is a schematic flow chart of a face key point detection method according to an embodiment of the present application;
fig. 3 is a schematic flow chart of a face key point detection method according to another embodiment of the present application;
fig. 4 is a schematic flow chart of a face keypoint detection method according to another embodiment of the present application;
fig. 5 is a schematic diagram of marking positions of key points of a face according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a face keypoint detection apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a face keypoint detection apparatus according to yet another embodiment of the present application;
fig. 8 is a schematic diagram of a hardware structure of a face keypoint detection apparatus according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a schematic structural diagram of a face keypoint detection system according to an embodiment of the present application. As shown in fig. 1, the face keypoint detection system provided by the present embodiment includes a camera 11 and a face keypoint detection apparatus 12. The camera 11 is used for acquiring a face image including a human face. The camera 11 may send the face image to the face keypoint detection device 12. The face key point detection device 12 is used for performing key point detection on the face image. The face key point detection device 12 may be a mobile phone, a tablet computer, a vehicle-mounted terminal, a desktop computer, a portable computer, a server, and the like, and is not limited herein.
The face key point detection method provided by the embodiment can be applied to scenes such as face local positioning, expression recognition, intelligent driving test judgment, driving assistance and the like, and is not limited herein. Taking the assisted driving as an example, in the assisted driving process, it is often necessary to determine the state of the driver, such as the sight line direction and fatigue condition of the driver, and generally, the face and the positions of key points of the face are first located from the acquired image, and then the state of the driver is determined according to the positions of the key points. Therefore, the positioning accuracy of the key points of the human face plays an important role in accurately judging the state of the driver. In this application scenario, the camera may be mounted in a fixed position inside the vehicle for capturing images of the driver. The face key point detection equipment can be a vehicle-mounted terminal or a server in communication connection with the camera and the like. After the camera collects the image of the face of the driver, the image is sent to face key point detection equipment, the face key point detection equipment detects the key point position of the face in the image, then the face key point detection equipment identifies the state of the driver in the image according to the key point position of the face, or the face key point detection equipment sends the key point position of the face to other identification equipment, and the other identification equipment identifies the state of the driver in the image according to the key point position of the face.
The existing human face key point detection method mostly focuses on the situation that the human face is more positive and is not shielded, and the accuracy of the detection result is poor under the condition that the key point is shielded. If the steering angle of the driver is too large or a shielding object exists between the driver and the camera, the positions of key points on the face of the driver cannot be accurately detected, and further the state of the driver cannot be identified or is identified wrongly. According to the face key point detection method provided by the embodiment of the application, the neural network can be trained by the face image sample of the face key point marked out by the shielded part, so that the neural network can accurately detect the face key point for the face image of the face shielded by the part, the detection accuracy of face key point detection is improved, and the accuracy of subsequent driver state identification is further improved.
Fig. 2 is a schematic flow chart of a face key point detection method according to an embodiment of the present application. As shown in fig. 2, the method includes:
s201, obtaining a face image containing a face.
In this embodiment, the face keypoint detection apparatus may acquire a face image. For example, the face key point detection device may acquire a face image collected by a camera inside a vehicle, a road-mounted snapshot camera, a camera of a mobile phone, or the like, or may acquire the face image from a server and a database, which is not limited herein.
S202, carrying out key point detection on the face image through a neural network to obtain position information of key points of the face in the face image, wherein the neural network is trained through a training data set, the training data set comprises a plurality of first face image samples and corresponding marking information thereof, a blocked part exists in the face contained in each first face image sample, and the marking information comprises position information of key points of the blocked part and the non-blocked part on the face.
In this embodiment, the face key point detection device may input the face image into a neural network for face key point detection, and the neural network may detect the positions of the key points of the face in the face image and output position information of the face key points. The neural network is pre-constructed and the training of the network parameters is performed through a training data set.
The training data set comprises a plurality of face image samples with partially shielded faces and corresponding marking information. In this embodiment, a face image sample whose face is partially occluded is referred to as a first face image sample, and a face image sample whose face is not occluded is referred to as a second face image sample. The marking information of each first face image sample is position information of key points of the face in the image sample marked by manually marking the image sample. The marked key point position information comprises key point position information of an unoccluded part of the face in the image sample and key point position information of an occluded part of the face in the image sample.
For example, assuming that 68 keypoints are marked on each first face image sample, for a certain first face image sample, 56 keypoints of the non-occluded part of the face in the image sample are marked, and 12 keypoints of the occluded part of the face are marked, 56 keypoints of the non-occluded part of the face can be directly marked in the image sample during manual marking, and 12 keypoint positions of the occluded part of the face can be marked according to manual experience.
The method comprises the steps of firstly, obtaining a face image containing a face; and then carrying out key point detection on the face image through a neural network to obtain the position information of face key points in the face image, wherein the neural network is trained through a training data set, the training data set comprises a plurality of first face image samples and corresponding marking information thereof, the face contained in each first face image sample has a shielded part, the marking information comprises the position information of key points of the shielded part and the non-shielded part on the face, and the neural network can be trained through the face image samples for marking the face key points of the shielded part, so that the neural network can also accurately detect the face key points on the face image of the face shielded by the part, and the detection accuracy of the face key point detection is improved.
Optionally, S202 may include:
inputting the face image into the convolutional neural network to obtain a preset number of feature maps, wherein the feature maps correspond to key points on the face in the face image one to one;
and for each feature map, determining the coordinates of the positions of the pixel points with the maximum pixel values in the feature map as the position coordinates of the key points corresponding to the feature map.
In this embodiment, the neural network may be a convolutional neural network for performing keypoint detection on a face image. The face key point detection equipment can input the face image into the convolutional neural network, and the convolutional neural network outputs the feature images of the preset number corresponding to the face image. The value of the predetermined number of sheets is not limited herein, and may be, for example, 68 or 136. The preset number is the same as the number of key points on the human face. For example, assuming that 68 key point positions are recognized on the face in each face image, after the face image is input by the neural network, 68 feature maps are output, and each feature map corresponds to one key point. And the position of the pixel point with the maximum pixel value in each feature map is the position of the key point corresponding to the feature map. The face key point detection equipment can select the coordinates of the positions of the pixel points with the maximum pixel values from each feature image as the position coordinates of the key points corresponding to the feature image, so that the position coordinates of each key point of the face in the face image are obtained.
The structure of the convolutional neural network is not limited herein, for example, the convolutional neural network may be constructed by 18 convolutional layers, 3 Relu (Rectified Linear Unit) active layers, 5 pooling layers, 4 deconvolution layers, 5 eltwise layers, and 3 batcnorm layers.
Optionally, a ratio of the number of key points of the portion, in which the face is occluded, of the first face image sample to the total number of key points of the face is less than or equal to a preset threshold.
In this embodiment, if too many parts of the face in the first face image sample are blocked, the neural network cannot be trained effectively, for example, more than 90% of the face area in the image sample is blocked, the position of the key point of the neural network cannot be determined by the neural network or the artificial marker, and the network parameters cannot be improved by training the neural network through the image sample. Therefore, the shielded part in the first face image sample is limited by the preset threshold value, so that effective training of the neural network is guaranteed. The value of the preset threshold is not limited herein, for example, the preset threshold may be 30%, 40%, etc. The ratio of the number of key points of the part, in which the face is shielded, of each first face image sample to the total number of key points of the face is less than or equal to the preset threshold. For example, if the preset threshold is 40%, the number of the key points of the portion of the first face image sample where the face is occluded is 12, and the total number of the face key points is 68, the proportion of the number of the key points of the portion of the first face image sample where the face is occluded to the total number of the face key points is 12/68-17.6%, and thus the first face image sample meets the requirement and can be used for training the neural network model.
Optionally, the training data set further includes a plurality of second face image samples and corresponding label information thereof, and each second face image sample includes an unobstructed face.
In this embodiment, the label information of each second face image sample includes position information of a key point on the face in the second face image sample. In this embodiment, besides training the neural network by using a plurality of first face image samples, a plurality of second face image samples are also used for training the neural network, so that the training data set includes both image samples in which the face is partially occluded and image samples in which the face is not occluded. The neural network is trained through the training data set, so that the neural network can detect key points of images of which the faces are not shielded and images of which the faces are partially shielded, and the accuracy is high.
Optionally, after S202, the method may further include:
and marking each face key point in the face image according to the position information of the face key points in the face image, and displaying the marked face image.
In this embodiment, the face key point detection device may mark each face key point in the face image according to the position information of the face key point in the face image, for example, the face key point in the face image may be marked by a preset color or symbol, and then the marked face image is displayed, so that the user can view the detected key point.
Alternatively, the face key point detection device may store the face image and the position information of the face key points in the detected face image in a designated file or a designated database in an associated manner.
Fig. 3 is a schematic flow chart of a face keypoint detection method according to another embodiment of the present application. In this embodiment, before the face image is detected by the neural network, a training data set is first constructed, and then the neural network is trained by the training data. The embodiment describes a specific implementation process for constructing the training data set in detail. As shown in fig. 3, the method may further include:
s301, obtaining a plurality of first face image samples and the marked coordinates of the face key points marked in advance.
In this embodiment, a plurality of first face image samples may be acquired, and the mark coordinates of the face key points in each first face image sample are marked in an artificial mark manner. The marking coordinates are the position coordinates of the manually marked key points.
And S302, generating a thermodynamic diagram corresponding to each key point on the face of each first face image sample according to the mark coordinate of each key point on the face of the first face image sample, wherein the pixel value of the position corresponding to the mark coordinate of the key point in the thermodynamic diagram is larger than the pixel values of the rest positions, and the pixel values of the rest positions are reduced along with the increase of the distance from the position corresponding to the mark coordinate of the key point.
In this embodiment, a first face image sample may be selected each time, and then, for each key point on a face in the first face image sample, a corresponding thermodynamic diagram is generated according to the mark coordinates of the key point. The thermodynamic diagram is the same size as the first face image sample. The pixel value of the position corresponding to the mark coordinate of the key point in the thermodynamic diagram is larger than the pixel values of the rest positions, and the pixel values of the rest positions are reduced along with the increase of the distance from the position corresponding to the mark coordinate of the key point. The farther a position is from the corresponding position of the mark coordinate of the key point, the smaller the pixel value is.
Optionally, the [ x, y ] pixel value of the position where the corresponding key point in each thermodynamic diagram is located is 1, and the pixel values of the rest positions conform to the gaussian distribution according to the distance from the [ x, y ] position. Thus, for each first face image sample, a set of thermodynamic diagrams corresponding to the first face image sample is obtained. For example, if a face has 68 keypoints in each first face image sample, then each first face image sample results in 68 thermodynamic diagrams.
And S303, forming the training data set by each first face image and a group of thermodynamic diagrams corresponding to the first face images.
In this embodiment, a set of thermodynamic diagram coordinates corresponding to each first face image may be used as the label information corresponding to the first face image, so as to obtain a training data set.
Optionally, after S303, the method may further include: and aiming at each first face image sample in the training data set, inputting the first face image sample into a pre-constructed neural network to obtain a plurality of predicted characteristic graphs output by the neural network, wherein the predicted characteristic graphs correspond to thermodynamic diagrams of the first face image sample one by one, and network parameters of the neural network are adjusted according to a group of thermodynamic diagrams and predicted characteristic graphs of the first face image sample.
In this embodiment, the predicted feature map and the thermodynamic map have the same size. For a certain first face image sample, calculating a loss function value corresponding to the first face image sample according to a group of thermodynamic diagrams of the first face image sample and a corresponding prediction characteristic diagram, then calculating back propagation according to the loss function value, iteratively updating network parameters, and considering that the model converges when the loss function value tends to be stabilized to a value of about 0.4 or less, thereby completing training. The loss function may be selected according to actual requirements, and is not limited herein, for example, the loss function may be a mean-square loss function, and is expressed as:
wherein L represents a loss function, xiI-th pixel value, y, of a predicted feature map representing the output of a neural networkiAnd the ith pixel value in the thermodynamic diagram is represented, and N is the number of pixel points in the prediction characteristic diagram and the thermodynamic diagram.
In the embodiment, the neural network is trained through a plurality of first face image samples and corresponding thermodynamic diagrams, the trained neural network identifies the face image to be processed, and each feature diagram corresponds to one key point position, so that the accuracy of face key point detection is improved.
Fig. 4 is a schematic flow chart of a face keypoint detection method according to another embodiment of the present application. The embodiment describes in detail a specific implementation process for acquiring a face image. As shown in fig. 4, the method includes:
s401, obtaining an image collected by a camera.
In this embodiment, the face key point detection device may acquire an image acquired by the camera. The image collected by the camera comprises a human face and a background image except the human face.
S402, identifying the position information of the area where the face in the image is located.
In this embodiment, the face key point detection device may identify the position information of the region where the face is located by using a face detection method. For example, the position information of the region where the face is located may be represented as [ x, y, w, h ], where x, y, w, h respectively represent the vertex coordinate x value and y value of the minimum bounding rectangle of the region where the face is located, and the width and height of the rectangle.
Optionally, the image is input to a Single Shot multi box Detector (SSD) network, so as to obtain position information of a region where the face is located in the image.
And S403, extracting the image of the area corresponding to the position information from the image as the face image.
In this embodiment, the face key point detection device may identify position information of an area where a face in the image is located through the SSD network, extract an image of the position information from the image as a face image, and detect a position of a face key point in the face image through the neural network.
Optionally, the structure of the SSD network is not limited herein, for example, the SSD network may be built up by 10 convolution layers, 7 Relu active layers, 2 pooling layers, 1 deconvolution layer, 1 clipping layer, 1 eltwise layer, 1 flatten layer, and 3 splicing layers.
S404, carrying out key point detection on the face image through a neural network to obtain position information of key points of the face in the face image, wherein the neural network is trained through a training data set, the training data set comprises a plurality of first face image samples and corresponding marking information thereof, a blocked part exists in the face contained in each first face image sample, and the marking information comprises position information of key points of the blocked part and the non-blocked part on the face.
In this embodiment, S404 is similar to S202 in the embodiment of fig. 2, and is not described here again.
In the embodiment, the position information of the region where the face is located in the image collected by the camera is identified, then the image of the corresponding region is extracted to be used as the face image, and then the key point identification is carried out on the face image, so that the interference of the image information outside the region where the face is located can be eliminated, and the key point identification accuracy is further improved.
In a possible implementation example, the face key point detection method provided by this embodiment may apply face detection to a driver, and may include the following steps:
step 1, installing a camera and program running equipment in a vehicle, and training an SSD network in advance. The method mainly comprises the following steps:
step 1.1, data is prepared. The method comprises the steps of collecting RGB image data of a scene in the car under different illumination conditions of different people through a color camera installed at a fixed position in the car, carrying out normalization processing on all collected images, carrying out normalization through subtracting 127.5 from a pixel value on each pixel point and dividing the pixel value by 127.5, and scaling the normalized images to 286 multiplied by 286 pixel size.
And step 1.2, manually marking the face position [ x, y, width, height ] of the driver in each image, recording the face position [ x, y, width, height ] of the driver in a label file, and making the face position [ x, y, width, height ] of the driver in each image and the image into a training data set of the SSD network.
And step 1.3, constructing an SSD detection network structure by adopting 10 convolution layers, 7 Relu activation layers, 2 pooling layers, 1 deconvolution layer, 1 clipping layer, 1 eltwise layer, 1 flatten layer and 3 splicing layers.
Step 1.4, when the SSD detection network is trained, inputting the normalized image data and the corresponding label data to the SSD detection network at the same time, manually setting K rectangles with different sizes and aspect ratios as preset position frames in advance, determining whether the overlap degree between each preset position frame and the face position area of the manual mark is large, if so, regarding the preset position frame as the face area, otherwise, regarding the preset position frame as the background area, respectively calculating a classification loss function value loss (two types of background and face area) and a position regression loss on each preset position frame, and adding the two to obtain a final loss of the network:
wherein x represents a preset position frame set, c represents a category set of the preset position frame, l represents an [ x, y, width, height ] parameter set of the preset position frame, g represents an [ x, y, width, height ] parameter set of the artificial mark position, and N represents the number of the preset position frames matched with the artificial mark position; lconf (x, c) represents calculating the cross entropy loss of the category of the preset position frame and the actual category; lloc (x, l, g) represents a smoothL1 loss to calculate the position information of the preset position frame and the actual position information. And calculating back propagation according to the loss, iteratively updating network parameters, and considering the model to be converged when the loss tends to be stable to a value of about 1.0 or less.
Step 2, training a face key point network model, which mainly comprises the following steps:
and 2.1, preparing data, intercepting and storing a corresponding position in the image as a new picture according to the marked face region information [ x, y, width, height ] during the SSD network training, and zooming the picture to 256 multiplied by 256 pixels. The image is the face image sample, and may include an image in which the face is partially occluded and an image in which the face is not occluded.
And 2.2, marking key point information [ x, y ] which can confirm positions of eyebrows, eyes, a nose, a mouth, the lower half part edge of the face and the like of the face in the acquired picture, wherein the x and the y respectively represent a horizontal coordinate and a vertical coordinate of the key point, as shown in fig. 5. A total of 68 sets of keypoint information represent 68 keypoints; and drawing 68 pairs of thermodynamic diagrams of 256 x 256 pixels according to the positions of the key points, wherein each key point corresponds to one thermodynamic diagram, the value of the [ x, y ] pixel at the position where the key point is arranged in the thermodynamic diagram is set to be 1, the values of the rest positions conform to Gaussian distribution according to the distance from the [ x, y ] position, and the 68 thermodynamic diagrams are used as labels.
And 2.3, constructing a human face key point detection network structure by adopting 18 convolution layers, 3 Relu activation layers, 5 pooling layers, 4 deconvolution layers, 5 eltwise layers and 3 Batchnorm layers.
And 2.4, when the face key point detection network is trained, inputting the scaled image data and the corresponding thermodynamic diagram data into the network, returning a 68 x 256 feature diagram to the last convolutional layer of the network, calculating the mean square loss function mse loss between the feature diagram output by the network and the thermodynamic diagram, calculating back propagation according to the loss, iteratively updating network parameters, and considering the model to be converged when the loss tends to be stabilized to a value of about 0.4 or less.
And 3, inputting the real-time in-car picture acquired by the camera into the SSD network acquired in the step 1, and acquiring face position information [ x, y, w, h ], wherein x, y, w, h respectively represent the vertex coordinate x value and y value of the minimum rectangular frame containing the whole face and the width and height of the rectangular frame. The method mainly comprises the following steps:
and 3.1, acquiring RGB image data of a scene in the vehicle through the camera, normalizing the image data, subtracting 127.5 from a pixel value of each point, dividing by 127.5, and zooming to 286 × 286 pixels.
And 3.2, inputting the normalized data into a network, obtaining a classification result and a position regression result in each preset position frame, and obtaining K groups of [ score, x, y, width and height ] arrays by corresponding the classification result and the position regression result, wherein the score represents the confidence coefficient that the current preset position frame belongs to the face, and the x, y, width and height represent the detailed position of the face. Since there is only one driver at most in the car and the driver is closest to the camera, the array with the largest score can be selected as the detection result.
Step 4, intercepting the image of the face area through the acquired face position information, scaling the image to 256 multiplied by 256 and then inputting the image into the face key point detection network, and the main steps comprise:
and 4.1, acquiring RGB image data of a scene in the vehicle through the camera, intercepting an image of a face area according to the face position information acquired in the step 3, and scaling the face image to 256 multiplied by 256 pixels.
And 4.2, inputting the zoomed image into a face key point detection network to obtain 68 predicted feature images with the size of 256 multiplied by 256, wherein the position of the maximum pixel value of the k-th feature image with the size of 256 multiplied by 256 is obtained and is regarded as the position of the k-th key point, so that the positions of the 68 key points can be obtained.
According to the embodiment, the neural network can be trained by marking the face image samples of the face key points of the shielded part, so that the neural network can accurately detect the face key points of the face image of the partially shielded face, and the detection accuracy of the face key point detection is improved.
Fig. 6 is a schematic structural diagram of a face keypoint detection apparatus according to an embodiment of the present application. As shown in fig. 6, the face key point detecting device 60 includes: .
The acquiring module 601 is configured to acquire a face image including a face.
The processing module 602 is configured to perform key point detection on the face image through a neural network to obtain position information of key points of a face in the face image, where the neural network is trained through a training data set, the training data set includes a plurality of first face image samples and corresponding label information thereof, a face included in each first face image sample has a blocked portion, and the label information includes position information of key points of a blocked portion and an unblocked portion on the face.
The method comprises the steps of firstly, obtaining a face image containing a face; and then carrying out key point detection on the face image through a neural network to obtain the position information of face key points in the face image, wherein the neural network is trained through a training data set, the training data set comprises a plurality of first face image samples and corresponding marking information thereof, the face contained in each first face image sample has a shielded part, the marking information comprises the position information of key points of the shielded part and the non-shielded part on the face, and the neural network can be trained through the face image samples for marking the face key points of the shielded part, so that the neural network can also accurately detect the face key points on the face image of the face shielded by the part, and the detection accuracy of the face key point detection is improved.
Fig. 7 is a schematic structural diagram of a face keypoint detection apparatus according to yet another embodiment of the present application. As shown in fig. 7, the face keypoint detection apparatus 60 provided in this embodiment may further include, on the basis of the face keypoint detection apparatus provided in the embodiment shown in fig. 6: a training module 603 and a display module 604.
Optionally, the neural network comprises a convolutional neural network;
the processing module 602 is specifically configured to:
inputting the face image into the convolutional neural network to obtain a preset number of feature maps, wherein the feature maps correspond to key points on the face in the face image one to one;
and for each feature map, determining the coordinates of the positions of the pixel points with the maximum pixel values in the feature map as the position coordinates of the key points corresponding to the feature map.
Optionally, the training module 603 is configured to:
acquiring a plurality of first human face image samples and marking coordinates of human face key points marked in advance by the first human face image samples;
generating a thermodynamic diagram corresponding to each key point on the face of each first face image sample according to the mark coordinate of each key point on the face of the first face image sample, wherein the pixel value of the position corresponding to the mark coordinate of the key point in the thermodynamic diagram is larger than the pixel values of the rest positions, and the pixel values of the rest positions are reduced along with the increase of the distance from the position corresponding to the mark coordinate of the key point;
and forming the training data set by each first face image and a group of corresponding thermodynamic diagrams.
Optionally, the training module 603 is further configured to:
and aiming at each first face image sample in the training data set, inputting the first face image sample into a pre-constructed neural network to obtain a plurality of predicted characteristic graphs output by the neural network, wherein the predicted characteristic graphs correspond to thermodynamic diagrams of the first face image sample one by one, and network parameters of the neural network are adjusted according to a group of thermodynamic diagrams and predicted characteristic graphs of the first face image sample.
Optionally, a ratio of the number of key points of the portion, in which the face is occluded, of the first face image sample to the total number of key points of the face is less than or equal to a preset threshold.
Optionally, the training data set further includes a plurality of second face image samples and corresponding label information thereof, and each second face image sample includes an unobstructed face.
Optionally, the obtaining module 601 is specifically configured to:
acquiring an image acquired by a camera;
identifying the position information of the area where the face is located in the image;
and extracting the image of the area corresponding to the position information from the image as the face image.
Optionally, the obtaining module 601 is specifically configured to:
and inputting the image into a single-shot multi-frame detection SSD network to obtain the position information of the area where the face is located in the image.
Optionally, a display module 604, configured to:
and marking each face key point in the face image according to the position information of the face key points in the face image, and displaying the marked face image.
The face key point detection device provided by the embodiment of the application can be used for executing the method embodiment, the implementation principle and the technical effect are similar, and the embodiment is not repeated herein.
Fig. 8 is a schematic diagram of a hardware structure of a face keypoint detection apparatus according to an embodiment of the present application. As shown in fig. 8, the face keypoint detection apparatus 80 provided in this embodiment includes: at least one processor 801 and a memory 802. The face keypoint detection apparatus 80 further comprises a communication component 803. The processor 801, the memory 802, and the communication unit 803 are connected by a bus 804.
In a specific implementation, the at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that the at least one processor 801 executes the above face keypoint detection method.
For a specific implementation process of the processor 801, reference may be made to the above method embodiments, which have similar implementation principles and technical effects, and details of this embodiment are not described herein again.
In the embodiment shown in fig. 8, it should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in the incorporated application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.
The memory may comprise high speed RAM memory and may also include non-volatile storage NVM, such as at least one disk memory.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.
The application also provides a computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, and when a processor executes the computer-executable instructions, the above human face key point detection method is realized.
The computer-readable storage medium may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk. Readable storage media can be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary readable storage medium is coupled to the processor such the processor can read information from, and write information to, the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the readable storage medium may also reside as discrete components in the apparatus.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (12)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911310807.4A CN111027504A (en) | 2019-12-18 | 2019-12-18 | Face key point detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911310807.4A CN111027504A (en) | 2019-12-18 | 2019-12-18 | Face key point detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111027504A true CN111027504A (en) | 2020-04-17 |
Family
ID=70210653
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911310807.4A Pending CN111027504A (en) | 2019-12-18 | 2019-12-18 | Face key point detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111027504A (en) |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111491180A (en) * | 2020-06-24 | 2020-08-04 | 腾讯科技（深圳）有限公司 | Method and device for determining key frame |
| CN111598038A (en) * | 2020-05-22 | 2020-08-28 | 深圳市瑞立视多媒体科技有限公司 | Face feature point detection method, device, equipment and storage medium |
| CN111695495A (en) * | 2020-06-10 | 2020-09-22 | 杭州萤石软件有限公司 | Face recognition method, electronic device and storage medium |
| CN111695516A (en) * | 2020-06-12 | 2020-09-22 | 百度在线网络技术（北京）有限公司 | Thermodynamic diagram generation method, device and equipment |
| CN111860343A (en) * | 2020-07-22 | 2020-10-30 | 杭州海康威视数字技术股份有限公司 | Method and device for determining face comparison result |
| CN111860199A (en) * | 2020-06-28 | 2020-10-30 | 上海芯翌智能科技有限公司 | Method and equipment for detecting key points in image |
| CN111881813A (en) * | 2020-07-24 | 2020-11-03 | 深圳市卡联科技股份有限公司 | Data storage method and system of face recognition terminal |
| CN111967406A (en) * | 2020-08-20 | 2020-11-20 | 高新兴科技集团股份有限公司 | Method, system, equipment and storage medium for generating human body key point detection model |
| CN111985374A (en) * | 2020-08-12 | 2020-11-24 | 汉王科技股份有限公司 | Face positioning method and device, electronic equipment and storage medium |
| CN112287802A (en) * | 2020-10-26 | 2021-01-29 | 汇纳科技股份有限公司 | Face image detection method, system, storage medium and equipment |
| CN112380981A (en) * | 2020-11-12 | 2021-02-19 | 平安科技（深圳）有限公司 | Face key point detection method and device, storage medium and electronic equipment |
| CN112419170A (en) * | 2020-10-16 | 2021-02-26 | 上海哔哩哔哩科技有限公司 | Method for training occlusion detection model and method for beautifying face image |
| CN112560584A (en) * | 2020-11-27 | 2021-03-26 | 北京芯翌智能信息技术有限公司 | Face detection method and device, storage medium and terminal |
| CN112784775A (en) * | 2021-01-27 | 2021-05-11 | 北京洛塔信息技术有限公司 | Key point marking method, system and device |
| CN112861678A (en) * | 2021-01-29 | 2021-05-28 | 上海依图网络科技有限公司 | Image identification method and device |
| CN112926424A (en) * | 2021-02-10 | 2021-06-08 | 北京爱笔科技有限公司 | Face occlusion recognition method and device, readable medium and equipment |
| CN112990057A (en) * | 2021-03-26 | 2021-06-18 | 北京易华录信息技术股份有限公司 | Human body posture recognition method and device and electronic equipment |
| CN113486701A (en) * | 2021-05-12 | 2021-10-08 | 李美满 | Training method of face recognition model and face recognition method |
| CN113505763A (en) * | 2021-09-09 | 2021-10-15 | 北京爱笔科技有限公司 | Key point detection method and device, electronic equipment and storage medium |
| CN113689527A (en) * | 2020-05-15 | 2021-11-23 | 武汉Tcl集团工业研究院有限公司 | Training method of face conversion model and face image conversion method |
| CN113688664A (en) * | 2021-07-08 | 2021-11-23 | 三星（中国）半导体有限公司 | Face key point detection method and face key point detection device |
| CN114140836A (en) * | 2020-08-12 | 2022-03-04 | 广州久邦世纪科技有限公司 | Method and system for identifying and detecting key points of human face |
| CN114462495A (en) * | 2021-12-30 | 2022-05-10 | 浙江大华技术股份有限公司 | Training method of face shielding detection model and related device |
| CN114758368A (en) * | 2020-12-28 | 2022-07-15 | 重庆中星微人工智能芯片技术有限公司 | Face key point information generation method, device, equipment and computer readable medium |
| CN114764925A (en) * | 2020-12-30 | 2022-07-19 | 北京眼神智能科技有限公司 | Mask wearing detection method and device, computer readable storage medium and equipment |
| CN116206287A (en) * | 2021-11-29 | 2023-06-02 | 魔门塔(苏州)科技有限公司 | Driver state judging method and device and driver monitoring system |
| CN116740783A (en) * | 2023-06-02 | 2023-09-12 | 展讯通信(天津)有限公司 | Face detection model training method and device |
| US20240193970A1 (en) * | 2022-12-07 | 2024-06-13 | Idemia Identity & Security USA LLC | Automatic system and method for document authentication using portrait fraud detection |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104992148A (en) * | 2015-06-18 | 2015-10-21 | 江南大学 | ATM terminal human face key points partially shielding detection method based on random forest |
| CN107679490A (en) * | 2017-09-29 | 2018-02-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detection image quality |
| CN108805040A (en) * | 2018-05-24 | 2018-11-13 | 复旦大学 | It is a kind of that face recognition algorithms are blocked based on piecemeal |
| CN109063695A (en) * | 2018-09-18 | 2018-12-21 | 图普科技（广州）有限公司 | A kind of face critical point detection method, apparatus and its computer storage medium |
| CN109508681A (en) * | 2018-11-20 | 2019-03-22 | 北京京东尚科信息技术有限公司 | The method and apparatus for generating human body critical point detection model |
| CN109685013A (en) * | 2018-12-25 | 2019-04-26 | 上海智臻智能网络科技股份有限公司 | The detection method and device of header key point in human body attitude identification |
| CN109960974A (en) * | 2017-12-22 | 2019-07-02 | 北京市商汤科技开发有限公司 | Face critical point detection method, apparatus, electronic equipment and storage medium |
| CN110163080A (en) * | 2019-04-02 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Face critical point detection method and device, storage medium and electronic equipment |
| CN110276316A (en) * | 2019-06-26 | 2019-09-24 | 电子科技大学 | A human key point detection method based on deep learning |
| CN110287760A (en) * | 2019-03-28 | 2019-09-27 | 电子科技大学 | A method for occlusion detection of facial facial features based on deep learning |
| CN110459301A (en) * | 2019-07-29 | 2019-11-15 | 清华大学 | Brain neurosurgery navigation registration method based on heat map and facial key points |
-
2019
- 2019-12-18 CN CN201911310807.4A patent/CN111027504A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104992148A (en) * | 2015-06-18 | 2015-10-21 | 江南大学 | ATM terminal human face key points partially shielding detection method based on random forest |
| CN107679490A (en) * | 2017-09-29 | 2018-02-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detection image quality |
| CN109960974A (en) * | 2017-12-22 | 2019-07-02 | 北京市商汤科技开发有限公司 | Face critical point detection method, apparatus, electronic equipment and storage medium |
| CN108805040A (en) * | 2018-05-24 | 2018-11-13 | 复旦大学 | It is a kind of that face recognition algorithms are blocked based on piecemeal |
| CN109063695A (en) * | 2018-09-18 | 2018-12-21 | 图普科技（广州）有限公司 | A kind of face critical point detection method, apparatus and its computer storage medium |
| CN109508681A (en) * | 2018-11-20 | 2019-03-22 | 北京京东尚科信息技术有限公司 | The method and apparatus for generating human body critical point detection model |
| CN109685013A (en) * | 2018-12-25 | 2019-04-26 | 上海智臻智能网络科技股份有限公司 | The detection method and device of header key point in human body attitude identification |
| CN110287760A (en) * | 2019-03-28 | 2019-09-27 | 电子科技大学 | A method for occlusion detection of facial facial features based on deep learning |
| CN110163080A (en) * | 2019-04-02 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Face critical point detection method and device, storage medium and electronic equipment |
| CN110276316A (en) * | 2019-06-26 | 2019-09-24 | 电子科技大学 | A human key point detection method based on deep learning |
| CN110459301A (en) * | 2019-07-29 | 2019-11-15 | 清华大学 | Brain neurosurgery navigation registration method based on heat map and facial key points |
Cited By (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113689527A (en) * | 2020-05-15 | 2021-11-23 | 武汉Tcl集团工业研究院有限公司 | Training method of face conversion model and face image conversion method |
| CN111598038A (en) * | 2020-05-22 | 2020-08-28 | 深圳市瑞立视多媒体科技有限公司 | Face feature point detection method, device, equipment and storage medium |
| CN111695495A (en) * | 2020-06-10 | 2020-09-22 | 杭州萤石软件有限公司 | Face recognition method, electronic device and storage medium |
| CN111695495B (en) * | 2020-06-10 | 2023-11-14 | 杭州萤石软件有限公司 | Face recognition method, electronic device and storage medium |
| CN111695516A (en) * | 2020-06-12 | 2020-09-22 | 百度在线网络技术（北京）有限公司 | Thermodynamic diagram generation method, device and equipment |
| CN111695516B (en) * | 2020-06-12 | 2023-11-07 | 百度在线网络技术（北京）有限公司 | Thermodynamic diagram generation method, device and equipment |
| CN111491180A (en) * | 2020-06-24 | 2020-08-04 | 腾讯科技（深圳）有限公司 | Method and device for determining key frame |
| CN111860199A (en) * | 2020-06-28 | 2020-10-30 | 上海芯翌智能科技有限公司 | Method and equipment for detecting key points in image |
| CN111860199B (en) * | 2020-06-28 | 2022-09-27 | 上海芯翌智能科技有限公司 | Method and equipment for detecting key points in image |
| CN111860343B (en) * | 2020-07-22 | 2023-04-28 | 杭州海康威视数字技术股份有限公司 | Method and device for determining face comparison result |
| CN111860343A (en) * | 2020-07-22 | 2020-10-30 | 杭州海康威视数字技术股份有限公司 | Method and device for determining face comparison result |
| CN111881813A (en) * | 2020-07-24 | 2020-11-03 | 深圳市卡联科技股份有限公司 | Data storage method and system of face recognition terminal |
| CN114140836A (en) * | 2020-08-12 | 2022-03-04 | 广州久邦世纪科技有限公司 | Method and system for identifying and detecting key points of human face |
| CN111985374B (en) * | 2020-08-12 | 2022-11-15 | 汉王科技股份有限公司 | Face positioning method and device, electronic equipment and storage medium |
| CN111985374A (en) * | 2020-08-12 | 2020-11-24 | 汉王科技股份有限公司 | Face positioning method and device, electronic equipment and storage medium |
| CN111967406A (en) * | 2020-08-20 | 2020-11-20 | 高新兴科技集团股份有限公司 | Method, system, equipment and storage medium for generating human body key point detection model |
| CN112419170A (en) * | 2020-10-16 | 2021-02-26 | 上海哔哩哔哩科技有限公司 | Method for training occlusion detection model and method for beautifying face image |
| US12367705B2 (en) | 2020-10-16 | 2025-07-22 | Shanghai Bilibili Technology Co., Ltd. | Occlusion detection |
| CN112419170B (en) * | 2020-10-16 | 2023-09-22 | 上海哔哩哔哩科技有限公司 | Training method of shielding detection model and beautifying processing method of face image |
| WO2022078041A1 (en) * | 2020-10-16 | 2022-04-21 | 上海哔哩哔哩科技有限公司 | Occlusion detection model training method and facial image beautification method |
| CN112287802A (en) * | 2020-10-26 | 2021-01-29 | 汇纳科技股份有限公司 | Face image detection method, system, storage medium and equipment |
| WO2021204037A1 (en) * | 2020-11-12 | 2021-10-14 | 平安科技（深圳）有限公司 | Detection method and apparatus for facial key point, and storage medium and electronic device |
| CN112380981A (en) * | 2020-11-12 | 2021-02-19 | 平安科技（深圳）有限公司 | Face key point detection method and device, storage medium and electronic equipment |
| CN112560584A (en) * | 2020-11-27 | 2021-03-26 | 北京芯翌智能信息技术有限公司 | Face detection method and device, storage medium and terminal |
| CN114758368A (en) * | 2020-12-28 | 2022-07-15 | 重庆中星微人工智能芯片技术有限公司 | Face key point information generation method, device, equipment and computer readable medium |
| CN114764925A (en) * | 2020-12-30 | 2022-07-19 | 北京眼神智能科技有限公司 | Mask wearing detection method and device, computer readable storage medium and equipment |
| CN112784775A (en) * | 2021-01-27 | 2021-05-11 | 北京洛塔信息技术有限公司 | Key point marking method, system and device |
| CN112861678B (en) * | 2021-01-29 | 2024-04-19 | 上海依图网络科技有限公司 | Image recognition method and device |
| CN112861678A (en) * | 2021-01-29 | 2021-05-28 | 上海依图网络科技有限公司 | Image identification method and device |
| CN112926424A (en) * | 2021-02-10 | 2021-06-08 | 北京爱笔科技有限公司 | Face occlusion recognition method and device, readable medium and equipment |
| CN112926424B (en) * | 2021-02-10 | 2024-05-31 | 北京爱笔科技有限公司 | Face shielding recognition method, device, readable medium and equipment |
| CN112990057A (en) * | 2021-03-26 | 2021-06-18 | 北京易华录信息技术股份有限公司 | Human body posture recognition method and device and electronic equipment |
| CN113486701A (en) * | 2021-05-12 | 2021-10-08 | 李美满 | Training method of face recognition model and face recognition method |
| CN113688664B (en) * | 2021-07-08 | 2024-04-26 | 三星（中国）半导体有限公司 | Human face key point detection method and human face key point detection device |
| CN113688664A (en) * | 2021-07-08 | 2021-11-23 | 三星（中国）半导体有限公司 | Face key point detection method and face key point detection device |
| CN113505763B (en) * | 2021-09-09 | 2022-02-01 | 北京爱笔科技有限公司 | Key point detection method and device, electronic equipment and storage medium |
| CN113505763A (en) * | 2021-09-09 | 2021-10-15 | 北京爱笔科技有限公司 | Key point detection method and device, electronic equipment and storage medium |
| CN116206287A (en) * | 2021-11-29 | 2023-06-02 | 魔门塔(苏州)科技有限公司 | Driver state judging method and device and driver monitoring system |
| CN114462495A (en) * | 2021-12-30 | 2022-05-10 | 浙江大华技术股份有限公司 | Training method of face shielding detection model and related device |
| US20240193970A1 (en) * | 2022-12-07 | 2024-06-13 | Idemia Identity & Security USA LLC | Automatic system and method for document authentication using portrait fraud detection |
| US12387511B2 (en) * | 2022-12-07 | 2025-08-12 | Idemia Public Security France | Automatic system and method for document authentication using portrait fraud detection |
| CN116740783A (en) * | 2023-06-02 | 2023-09-12 | 展讯通信(天津)有限公司 | Face detection model training method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111027504A (en) | Face key point detection method, device, equipment and storage medium | |
| US9619708B2 (en) | Method of detecting a main subject in an image | |
| CN114120163B (en) | Video frame processing method, device and related equipment and storage medium | |
| WO2021098796A1 (en) | Image processing method and apparatus, device, and computer readable storage medium | |
| CN112330601A (en) | Parking detection method, device, equipment and medium based on fisheye camera | |
| CN108446694B (en) | A target detection method and device | |
| WO2017054314A1 (en) | Building height calculation method and apparatus, and storage medium | |
| CN106682620A (en) | Human face image acquisition method and device | |
| CN111292334B (en) | A panoramic image segmentation method, device and electronic equipment | |
| CN113490947A (en) | Detection model training method and device, detection model using method and storage medium | |
| CN109299658B (en) | Face detection method, face image rendering device and storage medium | |
| CN111832561B (en) | Character sequence recognition method, device, equipment and medium based on computer vision | |
| CN111667504B (en) | Face tracking method, device and equipment | |
| CN112991159B (en) | Face illumination quality assessment method, system, server and computer readable medium | |
| CN113313626A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN110443245B (en) | A method, device and equipment for locating license plate area in unrestricted scene | |
| CN114399729B (en) | Monitoring object movement identification method, system, terminal and storage medium | |
| CN112333468B (en) | Image processing method, device, equipment and storage medium | |
| CN111325107A (en) | Detection model training method and device, electronic equipment and readable storage medium | |
| CN111709377B (en) | Feature extraction method, target re-identification method and device and electronic equipment | |
| CN109919128A (en) | Acquisition methods, device and the electronic equipment of control instruction | |
| CN112183431A (en) | Real-time pedestrian number statistical method and device, camera and server | |
| CN117542005A (en) | Depth recognition model training method, image depth recognition method and related equipment | |
| WO2022206679A1 (en) | Image processing method and apparatus, computer device and storage medium | |
| CN117770774A (en) | Nailfold microcirculation image processing system, method and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200417 |
|
| RJ01 | Rejection of invention patent application after publication |