TWI768641B - Monitoring method, electronic equipment and storage medium - Google Patents

Abstract

The embodiments of the present disclosure relate to a monitoring method, electronic equipment and storage medium, the method includes: performing target area recognition on the visible light image to obtain a first area where the target object is located and a second area where a preset part is located; performing live detection on the target object according to the size of the first area and the third area corresponding to the first area in the infrared image to obtain a live detection result; when the result of the living body detection is a living body, performing identity recognition on the first area to obtain identity information, and performing temperature detection on the fourth area corresponding to the second area in the infrared image to obtain temperature information; generating first warning information according to the identity information when the temperature information is greater than or equal to a preset threshold.

Description

Monitoring method, electronic device and storage medium

The present invention relates to the technical field of computer vision, and relates to a monitoring method, an electronic device and a storage medium.

In the prevention and control of respiratory virus epidemics, early detection and early isolation are the most primitive and extremely effective measures. Early detection refers to the early detection of suspicious persons by means of medical diagnosis and testing, which may include temperature detection. Early isolation refers to reducing the contact between virus-carrying patients and others and reducing the risk of cross-infection, which can include measures such as non-contact temperature detection and wearing masks.

In related technologies, body temperature can be detected by means of thermometers, thermal imaging, etc., but the physical examination and detection process is cumbersome and difficult to adapt to places with large traffic, and when a suspected case with fever symptoms is detected, it is difficult to verify the identity of the suspected case News.

Embodiments of the present invention provide a monitoring method, an electronic device, and a storage medium.

According to an aspect of the embodiments of the present invention, a monitoring method is provided, including: performing target area identification on a visible light image of a monitoring area, and obtaining a first area where a target object is located in the visible light image and a pre-determined target area of the target object. Set the second area where the part is located; according to the size information of the first area and the third area corresponding to the first area in the infrared image of the monitoring area, perform live detection on the target object, and obtain The living body detection result; in the case that the living body detection result is a living body, the identification processing is performed on the first area, the identity information of the target object is obtained, and the corresponding information of the second area in the infrared image is obtained. The fourth area performs temperature detection processing to obtain the temperature information of the target object; when the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object.

The monitoring method according to the embodiment of the present invention can perform live detection and temperature detection based on infrared images, can quickly detect the body temperature of the target object in the image, improve the efficiency of body temperature detection, and is suitable for places with large traffic. In addition, the identity information of the target object can be identified through the visible light image, which is helpful to determine the identity information of the suspected case and improve the efficiency of epidemic prevention.

In a possible implementation manner, according to the size information of the first area and the third area corresponding to the first area in the infrared image, the target object is subjected to living body detection to obtain a living body detection result , comprising: determining the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first area; determining a living body detection strategy according to the distance; The position of the region in the visible light image is determined, and the position of the third region is determined in the infrared image; according to the living body detection strategy, the third region in the infrared image is subjected to living body detection processing to obtain the obtained The results of the live test.

In this way, the living body detection strategy can be determined by the size of the first region, and the living body detection can be performed based on the living body detection strategy, and the isothermal analysis can be used to improve the living body detection accuracy in the far-infrared image.

In a possible implementation manner, determining the living body detection strategy according to the distance, including one of the following: when the distance is greater than or equal to a first distance threshold, determining the living body detection strategy as based on human body shape the living body detection strategy; when the distance is greater than or equal to the second distance threshold and less than the first distance threshold, the living body detection strategy is determined as a living body detection strategy based on head and neck morphology; In the case of the second distance threshold, the living body detection strategy is determined as a facial morphology-based living body detection strategy.

In a possible implementation manner, according to the living body detection strategy, performing a living body detection process on the third area to obtain the living body detection result, including: according to the living body detection strategy, performing a morphology on the third area detection processing to obtain a shape detection result; if the shape detection result is a living body shape, perform isothermal analysis processing on the face region in the third area to obtain an isothermal analysis result; according to the living body detection strategy, determining a first weight of the morphological detection result and a second weight of the isothermal analysis result; determining the living body according to the first weight, the second weight, the morphological detection result and the isothermal analysis result Test results.

In a possible implementation manner, the method further includes, in the case that the living body detection result is a living body, detecting whether a preset target is included in the first area, and obtaining the first detection result, wherein the preset target is obtained. Suppose that the target includes an item that blocks a part of the face, and in the case of a living body detection result being a living body, performing identification processing on the first area to obtain the identity information of the target object, including: when the living body detection result is In the case of a living body, an identification process is performed on the first area according to the first detection result to obtain the identity information of the target object.

In this way, the identification method can be selected according to whether there is a preset target, and the accuracy of identification can be improved.

In a possible implementation manner, detecting whether the first area includes a preset target, and obtaining the first detection result, includes: performing detection processing on the facial area in the first area, and determining the facial area If the feature missing result is a preset feature missing, detect whether a preset target is included in the face area, and obtain the first detection result.

In a possible implementation manner, performing identification processing on the first area according to the first detection result to obtain the identity information of the target object, including one of the following: when the first detection result is: In the case where the preset target does not exist, perform a first identification process on the face area in the first area to obtain the identity information of the target object; if the first detection result is that the preset exists In the case of a target, a second identity recognition process is performed on the face area in the first area to obtain the identity information of the target object, wherein the weight of the feature of the unoccluded area of the face in the second identity recognition process Greater than the weight of the feature of the corresponding area in the first identification process.

In a possible implementation manner, the method further includes: when the first detection result is that the preset target does not exist, generating second early warning information.

In a possible implementation manner, the method further includes: acquiring gender information of the target object; and determining the preset temperature threshold according to the gender information.

In a possible implementation manner, the method further includes: combining the position information of the first area or the third area, the temperature information of the target object and the identity information of the target object with the visible light The images and/or infrared images are superimposed to obtain detection images.

According to an aspect of the embodiments of the present invention, a monitoring system is provided, comprising: a visible light image acquisition part, an infrared image acquisition part, and a processing part, wherein the processing part is configured to: monitor a monitoring area acquired by the visible light image acquisition part The visible light image is used to identify the target area, and the first area where the target object is located in the visible light image and the second area where the preset part of the target object is located is obtained; according to the size information of the first area, and the The first region corresponds to the third region in the infrared image obtained by the infrared image acquisition part, and the target object is subjected to living body detection to obtain a living body detection result; in the case that the living body detection result is a living body, the third Perform identification processing on an area to obtain the identity information of the target object, and perform temperature detection processing on the fourth area corresponding to the second area in the infrared image to obtain the temperature information of the target object; When the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object.

In a possible implementation manner, the processing part is further configured to: determine the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first area; According to the distance, a living body detection strategy is determined; according to the position of the first region in the visible light image, the position of the third region is determined in the infrared image; according to the living body detection strategy, the infrared image is determined. The third region in the in vivo detection process is performed to obtain the in vivo detection result.

In a possible implementation manner, the processing part is further configured to: when the distance is greater than or equal to a first distance threshold, determine the living body detection strategy as a human body shape-based living body detection strategy; or When the distance is greater than or equal to the second distance threshold and less than the first distance threshold, the living body detection strategy is determined as a living body detection strategy based on head and neck morphology; or when the distance is less than the second distance threshold In the case of , the living body detection strategy is determined as a facial morphology-based living body detection strategy.

In a possible implementation manner, the processing part is further configured to: perform shape detection processing on the third region according to the living body detection strategy to obtain a shape detection result; when the shape detection result is a living body shape In this case, perform isothermal analysis processing on the face area in the third area to obtain an isothermal analysis result; according to the living body detection strategy, determine the first weight of the morphological detection result and the second weight of the isothermal analysis result. Weight; according to the first weight, the second weight, the morphological detection result and the isothermal analysis result, determine the living body detection result.

In a possible implementation manner, the processing part is further configured to: in the case that the living body detection result is a living body, detect whether a preset target is included in the first area, and obtain the first detection result, wherein, The preset target includes an item that shields a partial area of the face, and the processing part is further configured to: in the case that the living body detection result is a living body, according to the first detection result, perform an operation on the first area. The identity recognition process obtains the identity information of the target object.

In a possible implementation manner, the processing part is further configured to: perform detection processing on the facial region in the first region, and determine a feature missing result of the facial region; when the feature missing result is a preset In the case of missing features, it is detected whether the face area includes a preset target, and the first detection result is obtained.

In a possible implementation manner, the processing part is further configured to: in the case that the first detection result is that the preset target does not exist, perform a first identification on the face area in the first area identification processing to obtain the identity information of the target object; or in the case that the first detection result is that the preset target exists, perform a second identification processing on the facial area in the first area to obtain the The identity information of the target object, wherein the weight of the feature of the unoccluded area of the face in the second identity recognition process is greater than the weight of the feature of the corresponding area in the first identity recognition process.

In a possible implementation manner, the processing part is further configured to: combine the location information of the first area or the third area, the temperature information of the target object and the identity information of the target object with The visible light image and/or the infrared image are superimposed to obtain a detection image.

In a possible implementation manner, the processing part is further configured to: acquire gender information of the target object; and determine the preset temperature threshold according to the gender information.

In a possible implementation manner, the processing part is further configured to generate second warning information when the first detection result is that the preset target does not exist.

According to an aspect of the embodiments of the present invention, an electronic device is provided, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: execute the above monitoring method.

According to an aspect of the embodiments of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above monitoring method when executed by a processor.

According to an aspect of the present invention, a computer program is provided, including computer-readable codes, when the computer-readable codes are executed in an electronic device, the processor in the electronic device executes the above-mentioned monitoring method.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of embodiments of the present invention.

Other features and aspects of the present invention will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Various exemplary embodiments, features and aspects of the present invention will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote components that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" in this article is only an association relationship to describe associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, which can mean that A exists alone, A and B exist at the same time, and B exists alone. three conditions. In addition, the term "at least one" herein refers to any combination of any one of a plurality or at least two of a plurality, for example, including at least one of A, B, and C, and may mean including those composed of A, B, and C. Any one or more elements selected in the collection.

In addition, in order to better illustrate the embodiments of the present invention, numerous details are given in the following embodiments. It should be understood by those skilled in the art that the embodiments of the present invention may also be practiced without certain details. In some instances, methods, means, components and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the embodiments of the present invention.

FIG. 1 shows a flowchart of an image processing method according to an embodiment of the present invention. As shown in FIG. 1 , the method includes: In step S11, the target area identification is performed on the visible light image of the monitoring area, and the first area where the target object is located in the visible light image and the second area where the preset part of the target object is located is obtained; In step S12, according to the size information of the first area and the third area corresponding to the first area in the infrared image of the monitoring area, the target object is subjected to living body detection to obtain a living body detection result ; In step S13, in the case that the living body detection result is a living body, perform identification processing on the first area, obtain the identity information of the target object, and correspond to the second area in the infrared image The temperature detection process is performed in the fourth area of to obtain the temperature information of the target object; In step S14, when the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object.

The monitoring method according to the embodiment of the present invention can perform live detection and temperature detection based on infrared images, can quickly detect the body temperature of the target object in the image, improve the efficiency of body temperature detection, and is suitable for places with large traffic. In addition, the identity information of the target object can be identified through the visible light image, which is helpful to determine the identity information of the suspected case and improve the efficiency of epidemic prevention.

In a possible implementation manner, the monitoring method may be performed by a terminal device or other processing device, where the terminal device may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, Wireless phones, Personal Digital Assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. Other processing devices may be servers, cloud servers, or the like. In some possible implementations, the monitoring method may be implemented by the processor calling computer-readable instructions stored in the memory.

In a possible implementation manner, the monitoring method can be used in front-end monitoring equipment. For example, the front-end monitoring equipment can be a device that integrates parts such as a processor and a camera head, and the parts such as the camera head can be controlled by the processor, An image of the monitoring area is acquired, and the monitoring method is executed by the processor. In a possible implementation manner, the monitoring method can also be used in a server (for example, the processor that executes the monitoring method is located in the server, and the processor and the camera head are not packaged into one body, but are distributed in the monitoring area and In the back-end processor, which constitutes a monitoring system), the server may receive an image captured by a camera in the monitoring area, and execute the monitoring method on the image.

In a possible implementation manner, the processor may be a System on a Chip (SoC), and the camera may include an infrared camera for acquiring infrared images and an infrared camera for acquiring visible light images camera. In a possible implementation, special sensors such as vanadium oxide uncooled infrared focal plane detectors can be used to capture spatial light signals, which can be used in far-infrared cameras to obtain life rays and monitor temperature. .

In a possible implementation manner, the camera can capture a video of the monitoring area, and the video frame of the video is a visible light image and an infrared image. The infrared camera and the visible light camera can be arranged in close positions. For example, the infrared camera and the visible light camera can be arranged adjacently, side by side, etc., or the two cameras can be arranged as a whole, etc. The method is not limited. Therefore, in the images captured by the two cameras at the same time, the positions of each target object are approximate, that is, the position deviation of the same target object in the light image and the corresponding infrared image is small, and The deviation can be corrected according to the positional relationship of the two cameras. For example, the third region of the target object in the infrared image can be determined according to the position of the first region in the visible light image (that is, the region where the target object is located). s position.

In a possible implementation manner, the image captured by the camera may have poor image quality, for example, image blur, focus error, and the camera is polluted or blocked. The image quality can be detected first. For example, image quality detection can be performed on the visible light image, for example, the texture and boundary clarity of the visible light image can be detected. When the sharpness is greater than or equal to the sharpness threshold, it can be considered that the image quality is good, and processing such as the next position detection can be performed. When the sharpness is less than the sharpness threshold, it may be considered that the image quality is poor, and the visible light image and the infrared image corresponding to the visible light image may be deleted at the same time.

In a possible implementation manner, in step S11, target area recognition may be performed on the visible light image, and a first area where the target object is located may be obtained, and the first area may be the face area of the target object and/or the target The body area of the subject. In this process, the position coordinates of the first region where the target object is located, or the position coordinates of the detection frame including the first region can be obtained.

In a possible implementation, in the process of temperature detection, the temperature of the forehead and other regions can be measured. For example, the temperature of the forehead region can be determined through the primitive value of the infrared image, and then the body temperature of the target object can be obtained. Therefore, the second area where the preset part (for example, the forehead and other areas) is located can be detected in the visible light image, and then the fourth area of the preset part in the infrared image can be determined, and the primitives of the infrared image can be pre-established The corresponding relationship between the value and the temperature, and then the body temperature of the target object can be determined according to the primitive value in the fourth area.

In a possible implementation manner, a living body is a biologically active and real biological mode exhibited by a human body with life characteristics. A prosthesis is a model that is similar to a living body (eg, photos, masks, headgear, etc.) Usually, in the process of living body detection, a near-infrared image of a human face is used for living body detection. It is mainly based on the difference in imaging characteristics between the living body and the prosthesis under near-infrared light, for example, the difference in characteristics such as optical flow, texture, and color, so as to realize the distinction between the living body and the prosthesis. However, in the process of temperature monitoring, the infrared images obtained by far-infrared photography are far-infrared images (that is, images based on life rays), not the commonly used near-infrared images. The size of the first region determines the liveness detection strategy and improves the accuracy of liveness detection.

In a possible implementation manner, step S12 may include: determining the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first area; according to the distance, determining a living body detection strategy; determining the position of the third region in the infrared image according to the position of the first region in the visible light image; The third region performs a living body detection process to obtain the living body detection result.

In a possible implementation manner, the first region is the region where the target object is located in the visible light image, for example, it may be the face region of the target object, and the size of the region may be the same as that of the target object and the image obtained by acquiring the visible light image. The distance between image acquisition devices (visible light cameras) is negatively correlated, that is, the larger the area size, the smaller the distance, and the smaller the area size, the larger the distance. Further, living bodies are characterized in infrared images with unique life characteristics (eg, far-infrared images based on life rays). Human blood vessels are all over the body (for example, the human body, face, neck and shoulders, etc.), but in response to the target object being close to the position in the visible light image, it only shows part of the body in the visible light image, for example, Only the face, head and neck are shown, but the facial features are clearer. The farther the distance is, the smaller the facial feature amount and the more human body feature amount. Based on different distances, different live detection strategies are combined to improve the accuracy of live detection and are suitable for use at different distances.

The correspondence between the reference size and the reference distance in the image can be established in advance, and the distance between the target object and the camera can be determined according to the proportional relationship between the size of the area where the target object is located and the reference size, and the reference distance.

2 shows a schematic diagram of living body detection according to an embodiment of the present invention. As shown in FIG. 2 , the distance between the target object and the camera can be determined according to the size information of the first area, and the living body detection strategy can be selected according to the distance. The corresponding third area in the infrared image may be determined according to the first area in the visible light image. For example, the first area is the face area of the target object A in the visible light image (for example, the distance between the target object A and the camera is relatively close, and only the face area of the target object A can be photographed), then the third area is The face area of target object A in the infrared image. The first area is the head and neck area of the target object B in the visible light image (for example, the face, shoulder and neck area of the target object B, or the upper body area, etc. can be photographed), and the third area is the target object B in the infrared image. the head and neck area. The first area is the human body area of the target object C in the visible light image (for example, the target object C is far away from the camera, and the human body area of the target object C can be photographed), then the third area is the target object C in the infrared image. body area.

In a possible implementation manner, determining a living body detection strategy according to the distance of the target object, including one of the following: when the distance is greater than or equal to a first distance threshold, determining the living body detection strategy as A living body detection strategy based on human body shape; when the distance is greater than or equal to the second distance threshold and less than the first distance threshold, the living body detection strategy is determined as a living body detection strategy based on head and neck morphology; in the When the distance is less than the second distance threshold, the living body detection strategy is determined as a facial morphology-based living body detection strategy.

In a possible implementation manner, the first distance threshold and the second distance threshold may be determined according to parameters such as the range and focal length of the detection area of the camera. For example, when the distance between the target object and the camera is greater than or equal to the first distance threshold, the picture captured by the camera may include the entire body, or most of the body of the target object, and a living body based on human body shape may be selected. detection strategy. In the case where the distance between the target object and the camera is less than the first distance threshold and greater than or equal to the second distance threshold, the picture captured by the camera cannot include the whole body of the target object, but can include the face and face of the target object. For the shoulder and neck area, or the upper body area, you can choose a living detection strategy based on the shape of the head and neck. When the distance between the target object and the camera is less than the second distance threshold, the picture captured by the camera only includes the face area of the target object, and a facial morphology-based living detection strategy can be selected.

In a possible implementation manner, the position of the third region is determined in the infrared image according to the position of the first region in the visible light image. In an example, the visible light image may include one or more target objects, that is, include one or more first regions. According to the positional relationship between the infrared camera and the visible light camera for quality inspection, and the position of the first region, Determine the location of the third region in the infrared image.

In a possible implementation, since the infrared camera can obtain far-infrared images for detecting life rays, not near-infrared images for detecting image features, the location of the target object is directly detected in the far-infrared images. The accuracy of the third region may be lower. In addition, when there are multiple target objects, if the third region where the target object is located is directly detected in the infrared image, the correspondence between each third region and each first region in the visible light image is difficult to clarify. Therefore, the position of the third region corresponding to the first region can be obtained according to the position of the first region in the visible light image and the positional relationship between the two cameras. Similarly, the position of the corresponding fourth region in the infrared image can be obtained through the position of the second region in the visible light image.

In a possible implementation manner, after the living body detection strategy is determined, the selected living body detection strategy may be used to perform living body detection processing on the third region in the infrared image. According to the living body detection strategy, performing a living body detection process on the third area to obtain the living body detection result includes: according to the living body detection strategy, performing a shape detection process on the third area to obtain a shape detection result; In the case that the shape detection result is a living body shape, perform isothermal analysis processing on the face region in the third area to obtain an isothermal analysis result; according to the living body detection strategy, determine the first weight of the shape detection result and the second weight of the isothermal analysis result; according to the first weight, the second weight, the morphological detection result and the isothermal analysis result, the living body detection result is determined.

In a possible implementation manner, morphology detection may be performed on the third region based on a living body detection strategy.

In a possible implementation manner, in the living body detection strategy based on human body shape, the third area may include the face area and the human body area of the target object, and there are many human body features, which can be extracted by neural networks and other methods. The processing is to perform shape detection processing by obtaining the human body feature amount, and determine whether the human body shape of the target object is a living body shape (for example, it can be judged whether the shape of the target object is a living body shape according to the characteristics of the target object's posture, movement, shape, etc.).

In a possible implementation, in the head and neck morphology-based living detection strategy, the third region may include the face, shoulder and neck region of the target object, or the upper body region, and feature extraction processing may be performed by means of neural networks and the like , to obtain the feature quantities of the face, shoulder and neck region, and perform shape detection processing to determine whether the shape of the head and neck of the target object is a living body (for example, it can be judged whether the target object is a living body according to the characteristics of the target object's posture, movement, shape, etc. form).

In a possible implementation manner, in the face shape-based living detection strategy, the third region may include the face region of the target object, with fewer human body features and more face features, which can be obtained through neural networks, etc. The feature extraction process is carried out in order to obtain the feature amount of the face area and perform the shape detection process to determine whether the facial shape of the target object is a living shape (for example, whether the target object is a living shape can be judged according to the facial expression, texture and other features of the target object) ).

In a possible implementation manner, the shape detection result of the shape detection process described above may be a result in the form of a fraction. For example, in the case where the score of the morphology detection is greater than or equal to the first score threshold, the morphology detection result may be regarded as a living body morphology. The embodiment of the present invention does not limit the form of the morphological detection result.

In a possible implementation manner, in the case that the shape detection result is a living body shape, isothermal analysis processing is performed on the face region in the third region to obtain an isothermal analysis result.

In a possible implementation, isothermal analysis can be used to determine whether the face area is a living body. For example, the real face of the target object has a uniform distribution of blood vessels, and if the target object wears a mask, headgear and other prostheses, due to false There is no blood vessel distribution in the body, and its isotherm is different from that of the real face. It can be determined whether the face area is a living body according to the isotherm analysis.

In a possible implementation manner, the isothermal analysis result may be a result in the form of a fraction. For example, if the isothermal analysis fraction is greater than or equal to the second fractional threshold, the isothermal analysis result may be considered as a living body. The embodiment of the present invention does not limit the form of the isothermal analysis result.

In a possible implementation, since in different living detection strategies, the proportions of the features of the face area are different (for example, in the living detection strategy based on human shape, the third area includes the face area and the human body area. , the proportion of the face area is small, and in the face shape-based living detection strategy, the third area only includes the face area, and the proportion of the face area is large). The first weight of the morphological detection result and the second weight of the isothermal analysis result may be determined according to the living body detection strategy.

For example, in the living body detection strategy based on human shape, since the proportion of the face area is small, the second weight of the isothermal analysis result based on the face area may be smaller, and the first weight of the shape detection result may be larger. For example, in the face shape-based living detection strategy, since the face area occupies a large proportion, the second weight of the isothermal analysis result based on the face area may be larger, and the first weight of the shape detection result may be smaller. For example, in a head and neck morphology-based in vivo detection strategy, the second weight and the first weight may be close or equal. The embodiments of the present invention do not limit the values of the first weight and the second weight.

In a possible implementation manner, the morphological detection result and the isothermal analysis result may be weighted and summed according to the first weight and the second weight, respectively, to obtain the living body detection result. In a possible implementation manner, the living body detection result may be a result in the form of a score. In the case that the living body detection result is greater than or equal to the living body score threshold, the target object may be considered as a living body; in the case where the living body detection result is less than the living body score threshold value , it can be considered that the target object is a prosthesis, or a non-human body, and no subsequent processing is required.

In this way, the living body detection strategy can be determined by the size of the first region, and the living body detection can be performed based on the living body detection strategy, and the isothermal analysis can be used to improve the living body detection accuracy in the far-infrared image.

In a possible implementation manner, in step S13, when the target object is a living body, an identity recognition process may be performed on the target object. In a possible implementation, in medical and health places (such as hospitals, operating rooms, etc.) or public places during the epidemic, it is usually necessary to wear protective equipment (such as masks, goggles, etc.). The facial area causes a certain degree of occlusion. Therefore, it is necessary to monitor whether the target object wears preset items such as masks, and determine the identification method according to the detection results.

Figure 3 shows a schematic diagram of an identity recognition process according to an embodiment of the present invention. The method further includes: in the case that the living body detection result is a living body, detecting whether a preset target is included in the first area, and obtaining the first detection result, wherein the preset target includes a partial area of the face For the item to be covered, the first detection result is whether the target object wears a mask or other item that can block the face area. Further, the identification may be performed according to the first detection result, and step S13 may include: in the case that the living body detection result is a living body, performing identification processing on the first area according to the first detection result, and obtaining the The identity information of the target object.

In a possible implementation, it may be first determined whether there is a preset target (for example, an item such as a mask) on the face of the target object. Detecting whether a preset target is included in the first area, and obtaining the first detection result, includes: performing detection processing on the facial area in the first area, and determining a feature missing result of the facial area; When the feature missing result is that the preset feature is missing, it is detected whether a preset target is included in the face region, and the first detection result is obtained.

In a possible implementation, the facial region in the first region can be detected and processed through a neural network. For example, a feature map of the facial region can be obtained through a convolutional neural network, and wearing a mask on the target object can cover the face. In the case of the object (preset target), there is a lack of features in the feature map of the face area. For example, if the target object wears a mask, the features of the target object's nose and mouth cannot be detected, that is, the nose and mouth features are missing, When the target object wears sunglasses, eye features cannot be detected, ie, eye features are missing.

In a possible implementation manner, in the case where the preset features (for example, the features of the nose and mouth or the features of the eyes, etc.) are missing, the target object may be wearing masks, sunglasses and other items, and further detection can be performed, for example, Whether there are preset targets such as masks on the face of the target object can be monitored by methods such as convolutional neural networks. For example, the shape, texture and other characteristics of the mask can be detected to identify items such as masks, and exclude the target object from covering the face with his hands or due to occlusion by others, etc.

In a possible implementation, in the case where the preset features (for example, the features of the nose and mouth or the eyes, etc.) are not missing (that is, the facial features are complete), there is no preset target such as a mask. If the monitoring area is a medical and health place such as a hospital, or a public place during the epidemic, not wearing a mask may be a violation. The method further includes: when the first detection result is that the preset target does not exist, generating second warning information. Further, a second warning message can also be output through a warning device or a display to remind the target object to wear a mask. For example, the warning device may include a device such as a sound, an alarm light, etc., and the second warning information may be output in the form of sound or light, or the information that the target object is not wearing a mask may be displayed on the display. The embodiment of the present invention does not limit the output mode of the second early warning information.

In a possible implementation manner, the identity recognition method may also be determined according to the first detection result (that is, whether the target object wears an item that can cover the face). According to the first detection result, performing identification processing on the first area to obtain the identity information of the target object, including one of the following: when the first detection result is that the preset target does not exist In this case, perform a first identification process on the face area in the first area to obtain the identity information of the target object; in the case that the first detection result is that the preset target exists, perform the first identification process on the target object. The facial area in the first area is subjected to a second identity recognition process to obtain the identity information of the target object, wherein the weight of the feature of the unoccluded area of the face in the second identity recognition process is greater than the corresponding weight in the first identity recognition process. The weights of the features of the region.

In a possible implementation manner, a neural network may be used to perform the identity recognition process, wherein the neural network used for the first identity recognition process is different from the neural network used for the second identity recognition process. In the example, in the case that the face area of the target object is partially occluded, and only the unoccluded area can be used for the facial area of identification, in the process of neural network training, the weight of the features of the unoccluded area is relatively high. Large, for example, focuses the attention mechanism of the neural network on the features of the unoccluded regions, and facilitates the identification of the features of the unoccluded regions.

In a possible implementation, in the case that the facial region of the target object is not blocked, and the features of the facial region can be used for identification processing, the neural network can perform identification based on all the features of the facial region.

For example, when the target object wears a mask, the neural network can only obtain the features of the eyes and eyebrows (the features of the nose and mouth are missing), the neural network can increase the weight of the features of the eyes and eyebrows during training, for identification based on eye and brow features. In the case that the target object does not wear a mask, the neural network can obtain all the features of the face area, and the neural network can perform identification based on all the features of the face area.

For example, the neural network can determine the similarity between eye and brow features and reference features in the database (reference features for eyes and brows), and assigns the reference features whose similarity is greater than or equal to a similarity threshold. The corresponding identity information is determined as the identity information of the target object. Similarly, the neural network can determine the similarity between the facial features and the reference features in the database (face reference features), and determine the identity information corresponding to the reference features whose similarity is greater than or equal to the similarity threshold as the target object. identity information.

For example, if there is no reference feature in the database whose feature similarity with the target object is greater than or equal to the similarity threshold (that is, the target object does not match the existing identity information in the database), the Features are stored in the database and add an identity (eg, identity code, etc.) to the target object. The embodiments of the present invention do not limit the method for identification.

In this way, the identification method can be selected according to whether there is a preset target, and the accuracy of identification can be improved.

In one possible implementation, the gender of the target object may also be identified. In a possible implementation manner, there is a certain difference in the average body temperature of men and women. Therefore, the preset temperature thresholds can be set differently according to gender. For example, the average body temperature of women is 0.3°C higher than that of men. Set preset temperature thresholds, such as 37 degrees for men and 37.3 degrees for women. The method further includes: acquiring gender information of the target object; and determining the preset temperature threshold according to the gender information.

In a possible implementation manner, the gender information of the target object can be determined according to the facial features of the target object. For example, when the target object does not wear a mask, the gender of the target object can be determined according to all the features of the target object's face. When the target object wears a mask, the gender of the target object can be determined according to the characteristics of the target object's eyes and eyebrows. Alternatively, when the facial features or the eye and eyebrow features of the target object match the reference features in the database, the gender of the target object can be determined according to the gender recorded in the identity information corresponding to the reference features in the database. The embodiments of the present invention do not limit the method for determining gender.

In a possible implementation manner, the preset temperature threshold may be determined according to gender information. For example, if the average body temperature of women is higher than the average body temperature of men, the preset temperature threshold for women may be higher than the preset temperature threshold for men. This embodiment of the present invention does not limit the method for determining the preset temperature threshold.

In a possible implementation manner, in step S13, the fourth area (corresponding to the second area where the preset part is located in the visible light image) in the infrared image where the preset part (for example, the forehead) is located may also be performed. The temperature detection process determines the temperature information (eg, body temperature) of the target object. For example, the primitive value of the primitive point in the infrared image can represent the color temperature value, and the color temperature value can be used to determine the temperature of the forehead area of the target object. The embodiment of the present invention does not limit the method of determining the temperature information.

In a possible implementation manner, in step S14, when the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object. For example, during the epidemic, fever is a suspected symptom, and the target objects with symptoms can be monitored. After detection, when the body temperature of the target object is greater than or equal to the preset temperature threshold, it is considered that the target object has fever symptoms, and the first warning information can be generated according to the identity information of the target object. Identity information and body temperature, in order to focus on monitoring the target object, and for example, the first warning information can be issued through audio and other equipment to remind the epidemic prevention personnel to focus on the target object, or to isolate the target object. The embodiment of the present invention does not limit the output mode of the first warning information.

In a possible implementation manner, the method further includes: combining the position information of the first area or the third area, the temperature information of the target object and the identity information of the target object with the visible light The images and/or infrared images are superimposed to obtain detection images.

In a possible implementation manner, the visible light image and/or the infrared image may be displayed on the display, and the target object's identity information and/or the position of the target object in the visible light image and/or the infrared image may be displayed. temperature information. For example, the identity information and/or temperature information of the target object may be superimposed on the first area in the visible light image where the target object is located, that is, the identity information and/or temperature information of the target object is displayed in the first area. Similarly, the identity information and/or temperature information of the target object may be displayed in the third area of the infrared image. The embodiment of the present invention does not limit the display manner.

According to the monitoring method of the embodiment of the present invention, the living body detection strategy can be determined by the size of the first region, and the living body detection can be performed based on the living body detection strategy, and the isothermal analysis can be used to improve the living body detection accuracy in the far-infrared image, and The identification method can be selected according to whether there is a preset target to improve the accuracy of identification. Further, the temperature detection can be performed on a preset part, the body temperature of the target object in the image can be quickly detected, the efficiency of body temperature detection can be improved, and it is suitable for places with a large flow of people. In addition, the identity information of the target object can be identified through the visible light image, which is helpful to determine the identity information of the suspected case and improve the efficiency of epidemic prevention.

4A and 4B show schematic diagrams of application of a monitoring method according to an embodiment of the present invention. As shown in FIG. 4A , the shown monitoring method can be used in a system for monitoring a monitoring area, and the monitoring system may include a system for collecting vital signs. Infrared camera 1 for acquiring infrared images, visible light camera 2 for acquiring visible light images, and processor 3 (eg, a system-on-a-chip SoC). The system may also include a supplementary light unit 4 for use in environments with poor lighting conditions, a display 5 for displaying visible light images, infrared images and/or warning information, and a communication for transmitting warning information, temperature information and other information Unit 6, warning unit 7 for outputting warning information, and interactive unit 8 for inputting instructions, etc.

In a possible implementation, as shown in FIG. 4B , the infrared camera 1 and the visible light camera 2 can capture video frames of the monitoring area, for example, the visible light image AF (AF1-AF25) and the infrared image BF (BF1-AF25) BF25). The visible light image AF and the infrared image BF can be subjected to positional deviation correction, that is, the position of the third region in the infrared image BF can be determined according to the position of the first region in the visible light image AF.

In a possible implementation manner, the processor 3 can perform image quality detection on the visible light image AF, for example, the detection can improve the texture of the light image, the sharpness of the boundary, and delete the visible light image with poor image quality and corresponding infrared images. Further, when there is no target object in the visible light image, the visible light image and the corresponding infrared image are deleted. For example, if there is no one in the monitoring area, the visible light image and the corresponding infrared image can also be deleted.

In a possible implementation manner, the processor 3 may perform position detection processing on the visible light image AF to obtain the first area where the target object is located, and also obtain the second area where the target object's forehead is located. According to the positions of the first area and the second area in the visible light image, the position of the third area where the target object is located in the infrared image and the location of the fourth area where the target object's forehead is located in the infrared image can be determined.

In a possible implementation manner, the processor 3 may determine the distance between the target object and the camera according to the size information of the first area, and determine the living body detection strategy based on the distance.

In a possible implementation manner, when the distance of the target object is greater than or equal to the first distance threshold, the living body detection strategy is determined as a living body detection strategy based on human body shape. Morphological detection can be performed on the third area in the infrared image to obtain a morphological detection score. And perform isothermal analysis on the face area in the infrared image to obtain isothermal analysis scores. Further, according to the living body detection strategy based on human body shape, a larger first weight can be assigned to the shape detection score, and a smaller second weight can be assigned to the isothermal analysis score. After the weighted summation, the living body detection result can be obtained.

In a possible implementation manner, when the distance of the target object is less than the first distance and greater than or equal to the second distance threshold, the living body detection strategy is determined as a living body detection strategy based on the shape of the head and neck. Morphological detection can be performed on the third area in the infrared image to obtain a morphological detection score. And perform isothermal analysis on the face area in the infrared image to obtain isothermal analysis scores. Further, according to the in vivo detection strategy based on head and neck morphology, similar weights can be assigned to the morphology detection score and the isothermal analysis score, and after the weighted summation, the in vivo detection result can be obtained.

In a possible implementation manner, when the distance of the target object is less than the second distance threshold, the living body detection strategy is determined as a facial morphology-based living body detection strategy. Morphological detection can be performed on the third area in the infrared image to obtain a morphological detection score. And perform isothermal analysis on the face area in the infrared image to obtain isothermal analysis scores. Further, according to the facial morphology-based living detection strategy, a smaller first weight can be assigned to the morphology detection score, and a larger second weight can be assigned to the isothermal analysis score. After the weighted summation, the living body detection result can be obtained.

In a possible implementation manner, the processor 3 can detect whether the target object wears a mask in the first area of the visible light image, and when the target object does not wear a mask, can generate second warning information and display it on the display 5 The second early warning information, or the second early warning information is played by the warning unit 7 .

In a possible implementation manner, the processor 3 may determine the identification method according to whether the target object wears a mask. The reference features are compared to determine the identity information of the target object. When the target object wears a mask, the identity information of the target object can be determined by comparing the eye and eyebrow features of the target object with the reference features in the database. Further, the second early warning information may also include the identity information of the target object, for example, it may be displayed on the display 5 that XXX (name) does not wear a mask, etc.

In a possible implementation manner, the processor 3 can identify the gender of the target object, for example, can identify the gender of the target object according to the facial features of the target object, or obtain the gender of the target object according to the identity information of the target object. Further, the preset temperature threshold for male objects and the preset temperature threshold for female objects can be determined respectively according to the gender of the target object. Alternatively, the preset temperature threshold may also be input through the interaction unit 8, and the embodiment of the present invention does not limit the setting manner of the preset temperature threshold.

In a possible implementation manner, the temperature of the fourth area (ie, the area where the forehead is located in the infrared image) may be monitored to obtain the body temperature of the target object. When the body temperature exceeds the preset temperature threshold, the processor 3 may generate first warning information. And display the first warning information through the display 5 , or play the first warning information through the warning unit 7 . Further, the identity information, body temperature, and location information of the camera (that is, the location where the fever target appears) can be transmitted to the background database or server through the communication unit 6 to facilitate the tracking of suspected cases. And the determination of the activity trajectories of suspected cases.

In a possible implementation, the identity information and body temperature of the target object can be superimposed on the first area in the visible light image or the third area in the infrared image, and displayed on the display 5 for intuitive observation The temperature information and identity information of each target object in the monitoring area can be stored, and the superimposed images can be stored.

In a possible implementation manner, the monitoring method can use visible light images and infrared images to perform temperature monitoring, which can reduce the hardware cost of the camera and reduce the data processing pressure. It can also monitor whether each target object wears a mask. It can be used in fields such as monitoring of health care places or monitoring of epidemic prevention. The embodiments of the present invention do not limit the application field of the monitoring method.

FIG. 5 shows a block diagram of a monitoring system according to an embodiment of the present invention. As shown in FIG. 5 , the system includes: an infrared image acquisition part 11 , a visible light image acquisition part 12 , and a processing part 13 .

The processing section 13 is configured to: Perform target area identification on the visible light image of the monitoring area acquired by the visible light image acquisition part 12, and obtain the first area where the target object is located in the visible light image and the second area where the preset part of the target object is located; According to the size information of the first area, and the corresponding third area of the first area in the infrared image acquired by the infrared image acquisition part 11 , perform living body detection on the target object, and obtain a living body detection result; In the case that the living body detection result is a living body, perform identification processing on the first area to obtain the identity information of the target object, and perform an identification process on the fourth area corresponding to the second area in the infrared image. temperature detection processing to obtain temperature information of the target object; When the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object.

In a possible implementation, the processing part 13 may include a processor (eg, the processor 3 in FIG. 4A ), and the infrared image acquisition part 11 may include an infrared camera (eg, the infrared camera in FIG. 4A ) 1), the visible light image acquisition part 12 may include a visible light camera (eg, the visible light camera 2 in FIG. 4A ).

In a possible implementation, the processing part is further configured to: determining the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first area; According to the distance, determine the living body detection strategy; determining the position of the third region in the infrared image according to the position of the first region in the visible light image; According to the living body detection strategy, a living body detection process is performed on the third region in the infrared image to obtain the living body detection result.

In a possible implementation, the processing part is further configured to: In the case that the distance is greater than or equal to the first distance threshold, determine the living body detection strategy as a body shape-based living body detection strategy; or In the case that the distance is greater than or equal to the second distance threshold and less than the first distance threshold, determining the living body detection strategy as a head and neck morphology-based living body detection strategy; or In the case that the distance is smaller than the second distance threshold, the living body detection strategy is determined as a facial morphology-based living body detection strategy.

In a possible implementation, the processing part is further configured to: According to the living body detection strategy, perform shape detection processing on the third region to obtain a shape detection result; In the case that the shape detection result is a living body shape, performing isothermal analysis processing on the face region in the third region to obtain an isothermal analysis result; determining a first weight of the morphological detection result and a second weight of the isothermal analysis result according to the in vivo detection strategy; The living body detection result is determined according to the first weight, the second weight, the morphology detection result, and the isothermal analysis result.

In a possible implementation manner, the processing part is further configured to: In the case that the living body detection result is a living body, detecting whether the first area includes a preset target, and obtaining the first detection result, wherein the preset target includes an item that blocks a part of the face; The processing section is further configured to: In the case that the living body detection result is a living body, an identity recognition process is performed on the first area according to the first detection result, and the identity information of the target object is obtained.

In a possible implementation, the processing part is further configured to: Perform detection processing on the facial area in the first area, and determine the feature missing result of the facial area; In the case that the feature missing result is a preset feature missing, it is detected whether a preset target is included in the face region, and the first detection result is obtained.

In a possible implementation, the processing part is further configured to: If the first detection result is that the preset target does not exist, perform a first identification process on the face area in the first area to obtain the identity information of the target object; or In the case where the first detection result is that the preset target exists, a second identity recognition process is performed on the face area in the first area to obtain the identity information of the target object, wherein the second The weights of the features of the unoccluded regions of the face in the identification process are greater than the weights of the features of the corresponding regions in the first identification process.

In a possible implementation manner, the processing part is further configured to: Superimpose the position information of the first area or the third area, the temperature information of the target object and the identity information of the target object with the visible light image and/or infrared image to obtain detection image.

In a possible implementation manner, the processing part is further configured to: obtain the gender information of the target object; The predetermined temperature threshold is determined according to the gender information.

In a possible implementation manner, the processing part is further configured to: When the first detection result is that the preset target does not exist, second warning information is generated.

It can be understood that the foregoing method embodiments mentioned in the embodiments of the present invention can be combined with each other to form a combined embodiment without violating the principle and logic.

In addition, the embodiments of the present invention also provide monitoring systems, electronic devices, computer-readable storage media, and programs, all of which can be used to implement any monitoring method provided by the embodiments of the present invention. record, without further elaboration.

Those skilled in the art can understand that in the above method of the embodiments, the writing order of each step does not mean a strict execution order but constitutes any limitation to the implementation process, and the execution order of each step should be based on its function and possible internal logic. Sure.

In some embodiments, the functions or included parts of the apparatus provided in the embodiments of the present invention may be configured to execute the methods described in the above method embodiments. For implementation, reference may be made to the above method embodiments. For brevity, here No longer.

An embodiment of the present invention also provides a computer-readable storage medium, which stores computer program instructions, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

An embodiment of the present invention further provides an electronic device, including: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to perform the above method.

Embodiments of the present invention also provide a computer program product, including computer-readable codes. When the computer-readable codes are run on a device, a processor in the device executes instructions for implementing the monitoring method provided in any of the above embodiments.

Embodiments of the present invention further provide another computer program product for storing computer-readable instructions, and when the instructions are executed, the computer executes the operations of the monitoring method provided by any of the above embodiments.

The electronic device may be provided as a terminal, server or other form of device.

FIG. 6 is a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

6, the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812 , sensor component 814 , and communication component 816 .

The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 802 may include one or more subsections that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia subsection to facilitate interaction between multimedia component 808 and processing component 802.

The memory 804 is configured to store various types of data to support the operation of the electronic device 800 . Examples of such data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. Memory 804 may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Disk or CD.

Power supply assembly 806 provides power to various components of electronic device 800 . Power supply components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800 .

Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). Where the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a calling mode, a recording mode, and a voice recognition mode. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, audio component 810 also includes a speaker configured to output audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface portions, which may be keyboards, click wheels, buttons, and the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for electronic device 800 . For example, the sensor assembly 814 can detect the open/closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, the sensor assembly 814 can also detect the electronic device 800 or Changes in the position of a component of the electronic device 800 , presence or absence of user contact with the electronic device 800 , orientation or acceleration/deceleration of the electronic device 800 and changes in the temperature of the electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 814 may also include a light sensor, such as a Complementary Metal Oxide Semiconductor (Complementary Metal Oxide Semiconductor) or a CCD (Charge-coupled Device, Charge Coupled Device) image sensor, configured for imaging applications used in. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 2G (2-Generation wireless telephone technology, second generation mobile communication technology) or 3G (3-Generation wireless telephone technology, third generation mobile communication technology) technology), or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a Near Field Communication (NFC) portion to facilitate short-range communication. For example, the NFC part can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (Bluetooth, BT) technology and other technology to achieve.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processing (DSP), a Digital Signal Processing Device (Digital Signal Processing) device, DSPD), programmable logic device (PLD), field programmable gate array (Field Programmable Gate Array, FPGA), controller, microcontroller, microprocessor or other electronic components to achieve, with to execute the above method.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions executable by the processor 820 of the electronic device 800 to accomplish the above method.

FIG. 7 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. 7, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions executable by the processing component 1922, such as applications. An application program stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Additionally, the processing component 1922 is configured to execute instructions to perform the above-described methods.

The electronic device 1900 may also include a power supply assembly 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input output (I/O) Interface 1958. Electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions executable by the processing component 1922 of the electronic device 1900 to accomplish the above method.

Embodiments of the present invention may be systems, methods and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the embodiments of the present invention.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. Examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (Electrical Programmable Read Only Memory, EPROM or flash memory), static random access memory (Static Random Access Memory, SRAM), portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices, such as punch cards or recessed protrusions on which instructions are stored structure, and any suitable combination of the above. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or Electrical signals carried by wires.

The computer-readable program instructions described herein may be downloaded from computer-readable storage media to various computing/processing devices, or downloaded to external computers or external storage over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network device. Networks may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. A network interface card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for computer-readable storage stored in each computing/processing device in the medium.

The computer program instructions for carrying out the operations of the embodiments of the present invention may be assembly instructions, Industry Standard Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or in one or Source or object code written in any combination of a variety of programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" or Similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely remotely. run on a client computer or server. In the case of a remote computer, the remote computer can be connected to the user computer via any kind of network—including a local area network (LAN) or a wide area network (WAN)—or, can be connected to to an external computer (eg using an Internet service provider to connect via the Internet). In some embodiments, electronic circuits are personalized by utilizing state information of computer readable program instructions, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays, PLA), the electronic circuit can execute computer-readable program instructions to implement various aspects of the embodiments of the present invention.

Aspects of embodiments of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to the processor of a general purpose computer, special purpose computer or other programmable data processing device to produce a machine for execution of the instructions by the processor of the computer or other programmable data processing device When, means are created that implement the functions/acts specified in one or more of the blocks in the flowchart and/or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium, the instructions causing the computer, programmable data processing device and/or other equipment to operate in a particular manner, so that the computer readable medium storing the instructions Included is an article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to generate a computer Processes of implementation such that instructions executing on a computer, other programmable data processing apparatus, or other device implement the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions that contains one or more logic for implementing the specified logic Executable instructions for the function. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by dedicated hardware-based systems that perform the specified functions or actions. implementation, or may be implemented in a combination of special purpose hardware and computer instructions.

The computer program product can be implemented in hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

Various embodiments of the present invention have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

1: Infrared camera 2: Visible light camera 3: Processor 4: fill light unit 5: Display 6: Communication unit 7: Warning unit 8: Interactive unit 11: Infrared image acquisition part 12: Visible light image acquisition part 13: Processing part 800: Electronics 802: Process component 804: memory 806: Power Components 808: Multimedia Components 810: Audio Components 812: Input/Output Interface 814: Sensor Assembly 816: Communication Components 820: Processor 1900: Electronic equipment 1922: Processing components 1926: Power Components 1932: Memory 1950: Web Interface 1958: Input and output interface S11~S14: Steps

The accompanying drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the embodiments of the present invention, and together with the description, serve to explain the technical solutions of the embodiments of the present invention. 1 shows a flowchart of an image processing method according to an embodiment of the present invention; FIG. 2 shows a schematic diagram of living body detection according to an embodiment of the present invention; FIG. 3 shows a schematic diagram of an identity recognition process according to an embodiment of the present invention; 4A and 4B show schematic diagrams of application of the monitoring method according to an embodiment of the present invention; Figure 5 shows a block diagram of a monitoring system according to an embodiment of the present invention; 6 shows a block diagram of an electronic device according to an embodiment of the present invention; 7 shows a block diagram of an electronic device according to an embodiment of the present invention.

S11~S14: Steps

Claims (10)

A monitoring method, the method is applied to an electronic device, wherein the method comprises: performing target area identification on a visible light image of a monitoring area, and obtaining a first area where a target object is located in the visible light image and the target object The second area where the preset part of the target object is located; wherein, the preset part of the target object includes at least the forehead of the target object; according to the size information of the first area, the target object is determined and the visible light map is obtained. The distance between the image acquisition devices of the images; when the distance is greater than or equal to the first distance threshold, the living body detection strategy is determined to be a living body detection strategy based on human body shape; when the distance is greater than or equal to the second distance threshold value and less than the first distance threshold, determine that the living body detection strategy is a living body detection strategy based on head and neck morphology; when the distance is less than the second distance threshold, determine that the living body detection strategy is determined to be based on facial morphology. a living body detection strategy; according to the living body detection strategy and the third area corresponding to the first area in the infrared image of the monitoring area, perform living body detection on the target object, and obtain a living body detection result; In the case that the living body detection result is a living body, the identification processing is performed on the first area, the identity information of the target object is obtained, and the temperature of the fourth area corresponding to the second area in the infrared image is obtained. In the detection process, temperature information of the target object is obtained; in the case that the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object. The method according to claim 1, wherein the target object is subjected to living body detection according to the living body detection strategy and a third area corresponding to the first area in the infrared image of the monitoring area , obtaining a living body detection result, comprising: determining the position of the third region in the infrared image according to the position of the first region in the visible light image; The third region in the infrared image is subjected to living body detection processing to obtain the living body detection result. The method according to claim 2, wherein, according to the living body detection strategy, performing a living body detection process on the third region in the infrared image to obtain the living body detection result comprises: according to the The living body detection strategy is to perform morphological detection processing on the third area to obtain a morphological detection result; if the morphological detection result is a living body form, perform isothermal analysis processing on the face area in the third area to obtain isothermal analysis result; according to the living detection strategy, determine the first weight of the morphology detection result and the second weight of the isothermal analysis result; according to the first weight, the second weight, the morphology The detection result and the isothermal analysis result determine the in vivo detection result. The method according to claim 1, further comprising: in the case that the living body detection result is a living body, detecting whether a preset target is included in the first area, and obtaining the first detection result, wherein the preset target is obtained. Set the target to include items that occlude part of the face; The performing identity recognition processing on the first area to obtain the identity information of the target object includes: performing identity recognition processing on the first area according to the first detection result to obtain the identity of the target object News. The method according to claim 4, wherein the detecting whether a preset target is included in the first area and obtaining the first detection result includes: performing detection processing on the face area in the first area, Determine a feature missing result of the facial region; in the case that the feature missing result is a preset feature missing, detect whether the preset target is included in the facial region, and obtain the first detection result. The method according to claim 4, wherein the performing identification processing on the first area according to the first detection result to obtain the identity information of the target object includes one of the following: When the first detection result is that the preset target does not exist, perform a first identity recognition process on the face area in the first area to obtain the identity information of the target object; when the first detection result is In the presence of the preset target, a second identity recognition process is performed on the face area in the first area to obtain the identity information of the target object, wherein the face is not occluded in the second identity recognition process The weight of the feature of the region is greater than the weight of the feature of the corresponding region in the first identification process. The method according to claim 4, further comprising: When the first detection result is that the preset target does not exist, second warning information is generated. The method according to any one of claims 1 to 7, further comprising: combining the location information of the first area or the third area, the temperature information of the target object and the identity information of the target object with The visible light image and/or the infrared image are superimposed to obtain a detection image. An electronic device, comprising: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to: execute the method described in any one of request items 1 to 8. A computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implement the method described in any one of claim 1 to 8.

Images