CN110738821A - A method and system for alarming by remote camera - Google Patents

Abstract

The application provides remote camera shooting warning method which comprises the steps that a camera device shoots a target person to obtain a target image and sends the target image to a cloud, the cloud receives the target image sent by the camera device to generate high-level semantic description of the target image, wherein the high-level semantic description is high-level description of the target person in a text mode, the cloud generates a warning prompt and sends the warning prompt to a terminal device under the condition that the high-level semantic description comprises preset keywords, and the terminal device receives the warning prompt sent by the cloud.

Description

A method and system for alarming by remote camera

technical field

The present application relates to the field of monitoring and management technology methods, and in particular, to a method and system for alarming through remote cameras.

Background technique

Remote camera equipment plays a very important role in security monitoring. At present, the existing remote camera equipment uploads the captured image information to the cloud, and the cloud transmits the image information to the terminal device, so as to realize the purpose of remote monitoring. However, the camera equipment usually only has the function of collecting image information, and cannot describe the image information in text by means of image recognition and semantic description. The user can only obtain the image information stored in the cloud through the terminal device to subjectively judge what happened in the image. The accuracy of the judgment is not high, and the existing camera equipment does not have the function of alarm and remote call, and the user experience is poor.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a remote camera alarm method and system, which are used to improve the user's accuracy in judging events that occur in image files captured by a camera device, and improve the user's interaction experience with the camera device through a terminal device.

In a first aspect, an embodiment of the present application provides a method for alarming through a remote camera, including:

The camera device captures the target person to obtain the target image, and sends it to the cloud;

The cloud receives the target image sent by the camera device, and generates a high-level semantic description of the target image, wherein the high-level semantic description is a high-level description of the target person by means of text;

In the case that the high-level semantic description includes a preset keyword, the cloud generates an alarm prompt and sends it to the terminal device;

The terminal device receives the alarm prompt sent by the cloud.

In some possible embodiments, the cloud generates a high-level semantic description of the target image, including:

The cloud extracts, from the target image, position vectors of multiple target subjects including the target person;

The cloud extracts, from the target image, gesture vectors of multiple target subjects including the target person;

The cloud inputs the position vectors of the multiple target subjects and the pose vectors of the multiple target subjects into a semantic description model, thereby generating a high-level semantic description of the target image.

In some possible embodiments, the semantic description model includes a low-level semantic unit and a high-level semantic unit, and the position vectors of the multiple target subjects and the pose vectors of the multiple target subjects are input into the semantic description model, thereby Generate a high-level semantic description of the target image, including:

inputting the position vectors of the plurality of target subjects into the low-level semantic unit to obtain a low-level semantic description;

The high-level semantic description is obtained by inputting the gesture vectors of the plurality of target subjects and the low-level semantic description into the high-level semantic unit.

It can be seen that the embodiment of the present application can extract the position vectors and posture vectors of multiple target subjects including the target person in the target image, and can generate high-level semantic descriptions according to the extracted position vectors and posture vectors, so as to improve the performance of the target image. Describe the state between multiple target subjects in the target image well, and get a more accurate image semantic description.

In some possible embodiments, the method further includes:

The terminal device sends the call request to the camera device when receiving the alarm prompt sent by the cloud;

In the case that the terminal device accesses the network through the wireless local area network (WLAN), the terminal device establishes a video call with the camera device;

In the case that the terminal device does not access the network through the wireless local area network (WLAN), the terminal device establishes a voice call with the camera device.

In some possible embodiments, when the terminal device establishes a voice call with the camera device, the method further includes:

Displaying a video call shortcut key on the interface of the voice call of the terminal device;

When the video call shortcut key is triggered, the terminal device stops the voice call with the camera device and establishes a video call with the camera device.

It can be seen that, in the embodiment of the present application, a video call shortcut key can be set on the voice call interface of the terminal device, and the user can switch the voice call with the camera device to a video call during the voice call process, which improves the interaction between the user and the camera device. experience. In addition, the embodiment of the present application does not require the user to install third-party application software on the terminal device to interact with the camera device, which can prevent image files captured by the camera device from being acquired by third-party software and leak user privacy.

In a second aspect, an embodiment of the present application provides a remote camera warning system, including:

The camera device is used for photographing the target person to obtain the target image, and sending it to the cloud;

The cloud is configured to receive the target image sent by the camera device, and generate a high-level semantic description of the target image, where the high-level semantic description is a high-level description of the target person by means of text;

When the high-level semantic description includes a preset keyword, the cloud is further configured to generate an alarm prompt and send it to the terminal device;

The terminal device is configured to receive the alarm prompt sent by the cloud.

In some possible embodiments, the cloud is configured to generate a high-level semantic description of the target image, including:

The cloud is used to extract the position vectors of multiple target subjects including the target person from the target image;

The cloud is used for extracting gesture vectors of multiple target subjects including the target person from the target image;

The cloud is configured to input the position vectors of the multiple target subjects and the pose vectors of the multiple target subjects into a semantic description model, so as to generate a high-level semantic description of the target image.

In some possible embodiments, the semantic description model includes a low-level semantic unit and a high-level semantic unit, and the position vectors of the multiple target subjects and the pose vectors of the multiple target subjects are input into the semantic description model, thereby Generate a high-level semantic description of the target image, including:

inputting the position vectors of the plurality of target subjects into the low-level semantic unit to obtain a low-level semantic description;

The high-level semantic description is obtained by inputting the gesture vectors of the plurality of target subjects and the low-level semantic description into the high-level semantic unit.

In some possible embodiments, the system further includes:

The terminal device is configured to send the call request to the camera device when receiving the alarm prompt sent by the cloud;

In the case that the terminal device accesses the network through the wireless local area network (WLAN), the terminal device establishes a video call with the camera device;

In the case that the terminal device does not access the network through the wireless local area network (WLAN), the terminal device establishes a voice call with the camera device.

In some possible embodiments, when the terminal device establishes a voice call with the camera device, the system further includes:

Displaying a video call shortcut key on the interface of the voice call of the terminal device;

When the video call shortcut key is triggered, the terminal device stops the voice call with the camera device and establishes a video call with the camera device.

In the above solution, the cloud can generate a high-level semantic description of the target image according to the position vectors and attitude vectors of multiple target subjects including the target person in the target image. If the high-level semantic description includes preset keywords, the cloud can generate an alarm prompt. Sending it to the terminal device can let the user know the occurrence of the alarm event in time, improve the accuracy of the alarm event judgment in the target image, and the user can make a video call or voice call with the camera device as needed, which improves the interaction between the user and the camera device. experience.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic structural diagram of a remote camera alarm system provided by an embodiment of the present application;

2 is a schematic flowchart of a remote camera alarm method provided by an embodiment of the present application;

3 is a schematic diagram of a possible target image for generating high-level semantic description provided by an embodiment of the present application;

4 is a schematic diagram of a semantic description model provided by an embodiment of the present application;

5 is a schematic diagram of a video call interface between a terminal device and a camera device provided by an embodiment of the present application;

6 is a schematic diagram of a voice call interface between a terminal device and a camera device provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a cloud provided by an embodiment of the present application;

8 is a schematic structural diagram of a terminal device provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a camera device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

It is to be understood that, when used in this specification and the appended claims, the terms "comprising" and "comprising" indicate the presence of the described features, integers, steps, operations, elements and/or components, but do not exclude one or The presence or addition of a number of other features, integers, steps, operations, elements, components, and/or sets thereof.

It should also be understood that the terminology used in the specification of the application herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise.

It should also be further understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items .

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

The remote camera alarm method and system according to the embodiments of the present application use the cloud to perform image recognition on images captured by the camera device in real time and generate a high-level semantic description, so as to determine whether an alarm event has occurred in the monitoring area of the camera device, and send an alarm prompt to the terminal device. This enables users to be informed of the occurrence of an alarm event in time and to take corresponding measures as soon as possible. The remote camera alarming method and system provided by the embodiments of the present application can not only be applied to regional monitoring, such as residential area monitoring, office buildings, banks, shopping malls and other traditional monitoring, but also be applied to remote child and elderly care, hospital patient care, etc. , which is not specifically limited here.

First, please refer to FIG. 1 . FIG. 1 is a schematic structural diagram of a remote camera warning system provided by an embodiment of the present application.

As shown in FIG. 1 , the remote camera alarm system according to the embodiment of the present application includes: a cloud 110 , a terminal device 120 , and a camera device 130 .

In a specific embodiment of the present application, the above-mentioned cloud 110 can be used to establish a communication connection between the terminal device 120 and the camera device 130. For example, the terminal device 120 can send a call request to the cloud 110, and the cloud 110 forwards the call request to the camera device 130. After receiving the call request, the camera device 130 may make a call with the terminal device 120, and the call may be a voice call or a video call, which is not specifically limited here. It can be understood that the communication connection between the terminal device 120 and the camera device 130 may not be established through the cloud 110 , which is not specifically limited here. The communication connection may be implemented in a wireless manner, such as 3G, 4G, or a wireless local area network (wireless local area network, WLAN).

In the specific embodiment of this application, taking the cloud 110 for establishing the communication connection between the terminal device 120 and the camera device 130 as an example, when the cloud 110 establishes the first communication connection between the terminal device 120 and the camera device 130 , it is necessary to identify the first identification information used to represent the terminal device 120 and the second identification information used to represent the camera device 130, the cloud 110 can identify the terminal device 120 through the first identification information, and make the terminal device 120 join the cloud 110, The cloud 110 can identify the camera device 130 through the second identification information, and make the camera device 130 join the cloud 110 . The cloud 110 may also store the first identification information, the second identification information, and the binding relationship between the first identification information and the second identification information.

After the communication connection between the terminal device 120 and the camera device 130 is established through the above steps, there is no need to re-establish the communication connection between the terminal device 120 and the camera device 130, but the first identification information, the second identification information and the The binding relationship between the first identification information and the second identification information is updated or deleted.

It can be seen from this that the cloud 110 establishes a communication connection between the first identification information of the terminal device 120 and the second identification information of the camera device 130 according to the binding relationship between the two, which can avoid the unbound relationship with the camera device 130. After the terminal device 120 acquires the second identification information of the camera device 130, it establishes a communication connection with the camera device 130, which can also prevent the camera device 130 that is not bound to the terminal device 120 from acquiring the first identification information of the terminal device 120. Afterwards, a communication connection is established with the terminal device 120 .

In a specific embodiment of the present application, the cloud 110 may further include a cloud storage space and a cloud storage space. When the cloud 110 stores an image file uploaded by the camera device 130 for the first time, it can create a corresponding image file according to the second identification information of the camera device 130 for the first time. When the cloud 110 transmits an image file to the terminal device 120 for the first time, a corresponding cloud storage space can be created according to the second identification information of the terminal device 120 . The cloud storage subspace is used to store image files captured by a single camera device 130, and the cloud storage space is used to store one or more cloud storage subspaces that are bound to it. The cloud storage space is created according to the first identification information of the terminal device 120 , that is, the cloud storage space has a one-to-one correspondence with the first identification information, and the cloud storage subspace is created according to the second identification information of the camera device 130 Yes, in the same way, there is a one-to-one correspondence between the cloud storage subspace and the second identification information. It can be understood that if there is a binding relationship between the first identification information and the second identification information, there is a corresponding binding relationship between the cloud storage space and the cloud storage subspace.

After the cloud storage space corresponding to the terminal device 120 and the cloud storage subspace corresponding to the camera device 130 are established through the above steps, it is not necessary to re-establish the cloud storage space and the cloud storage subspace. You can also update and delete the image files stored in the cloud storage space and the cloud storage subspace as needed.

It can be seen that, the camera device 130 sends the captured target image to the cloud 110 for storage, without the need for the user to use the terminal device 120 for storage, thereby reducing the memory occupation of the terminal device 120 . And the cloud 110 establishes the cloud storage space and the cloud storage subspace according to the first identification information of the terminal device 120, the second identification information of the camera device 130, and the binding relationship between the first identification information and the second identification information, which can avoid the After acquiring the second identification information of the camera device 130, the terminal device 120 whose camera device 130 has no binding relationship can view the image files in the cloud storage subspace corresponding to the camera device 130, so as to avoid no binding relationship with the terminal device 120. After acquiring the first identification information of the terminal device 120, the camera device 130 uploads the captured image file to the cloud storage space corresponding to the terminal device 120.

For example, the first identification information representing the terminal device 120 may be a dial-up number or ID, etc., or may be a user-defined name of the terminal device 120, such as "terminal 1", "terminal 2", etc. The identification information may be a dial-up number or an IP address, etc., or may be a user-defined name of the camera device 130, such as "camera 1", "camera 2", and so on.

In the specific embodiment of the present application, after the cloud 110 receives the image file uploaded by the camera device 130, it can intercept a single frame or multiple frames of target images in the image file at preset time intervals for image recognition, and the cloud 110 extracts the target image in the target image. After the position vectors (ie, position features) and attitude vectors (ie, pose features) of multiple target subjects, input the position vectors of the multiple target subjects into the low-level semantic units in the semantic description model to obtain low-level semantic descriptions. The high-level semantic description can be obtained by the high-level semantic unit in the input semantic description model of the pose vector of the target subject and the above-mentioned low-level semantic description. If the high-level semantic description includes preset keywords, such as keywords such as "fall", "fall", "struggle", "crying", "twitch", "blood", etc., the cloud 110 determines that the occurrence of the occurrence in the monitoring area of the camera device 130 If an alarm event is detected, an alarm prompt is generated and sent to the terminal device 120 . The above-mentioned low-level semantic description refers to a low-level description of the target person through text, and the high-level semantic description refers to a high-level description of the target person through text. The preset time can be 0.001 seconds, 0.03 seconds, 0.5 seconds, 1 seconds, etc., the image file uploaded by the camera device 130 received by the cloud 110 may be picture information (such as multiple pictures consecutively at the acquisition time), or video information (such as a video of a certain duration, such as a video of 10s), etc. If The cloud 110 intercepts multiple frames of target images, and the time interval between two adjacent frames of target images may be equal or unequal, which is not specifically limited here.

It can be seen that the cloud 110 intercepts the target image in the image file every preset time to perform image recognition and generate a semantic description, and sends an alarm prompt to the terminal device 120 when it is determined that an alarm event has occurred, which not only improves the judgment of the alarm event in the target image It can also make users know that an alarm event has occurred in time without the need for the user to view the monitoring area in real time, and take corresponding measures as soon as possible.

In the specific embodiment of the present application, after the above-mentioned terminal device 120 receives the alarm prompt sent by the cloud 110 , the terminal device 120 may send a call request to the camera device 130 . When the terminal device 120 accesses the network through the wireless local area network WLAN, the terminal device 120 establishes a video call with the camera device 130 (ie, calls up the video screen of the camera device 130). When the terminal device 120 does not access the network through the wireless local area network (WLAN), the terminal device 120 establishes a voice call with the camera device 130, wherein a video call shortcut key is set on the voice call interface of the terminal device 120. When the user clicks the video call shortcut key during the voice call with the camera device 130 , the voice call with the camera device 130 is terminated, and a video call with the camera device 130 is established.

For example, when the terminal device 120 is connected to the network through the WLAN, the terminal device 120 receives the alarm prompt sent by the cloud 110, and the call interface of the terminal device 120 can display the "start" shortcut key or the "end" shortcut key, etc., and the user clicks "Start", the video call with the camera device 130 can be established to view the screen of the monitoring area of the camera device 130, and the user can click "End" to terminate the video call with the camera device 130. In the case where the terminal device 120 is not connected to the network through WLAN, the call interface of the terminal device 120 may be provided with a "start" shortcut key, a "video call" shortcut key, an icon of a built-in application (such as a camera or photo album), or a "end" shortcut key. ” shortcut key, etc., the user clicks “Start” to establish a voice call with the camera device 130 and send a voice message to the camera device 130; the user clicks the “Video call” shortcut key to terminate the voice call with the camera device 130, establish Make a video call with the camera device 130 to view the screen of the monitoring area of the camera device 130; the user clicks the icon of the built-in application (such as a camera or photo album), and can enter the built-in application to see the information of all the camera devices 130 bound to the terminal device 120. icon, a video call or a voice call can be established with any camera device 130, and the camera device 130 can be switched during the video call or voice call; the user can click "End" to terminate the voice call with the camera device 130.

In a specific embodiment of the present application, the terminal device 120 can also view the cloud storage space of the cloud 110 and the image files in the cloud storage subspace and download, update or delete as needed, wherein the cloud storage space is based on the first The identification information is created, and the cloud storage subspace is created according to the second identification information. The first identification information represents the terminal device 120, the second identification information indicates the camera device 130, and the cloud storage space and the cloud storage subspace have a binding relationship. The binding relationship may be established according to the binding relationship between the first identification information and the second identification information.

It can be seen that, in the embodiment of the present application, the terminal device 120 receives the alarm prompt sent by the cloud 110, and the user can make a video call or a voice call with the camera device 130 as needed, which improves the interaction experience between the user and the camera device 130, and the terminal The device 120 is provided with a preset application icon shortcut key and a “start” shortcut key, etc., the user does not need to install third-party application software on the terminal device 120 to interact with the camera device 130, which can prevent the image captured by the camera device 130 installed by the user. Files are obtained by third-party software, revealing user privacy.

In a specific embodiment of the present application, the above-mentioned camera device 130 is rotatable, and can collect image information of its monitoring area in real time to obtain an image file. It is video information (such as a video of a certain length, such as a 10s-long video), etc., to achieve the purpose of monitoring a certain area. The camera device 130 may upload the captured image files to the cloud 110 at preset time intervals. In addition, the camera device 130 can establish a video call or a voice call with the terminal device 120 after receiving the call request sent by the terminal device 120 . In this application, the imaging device 130 may be a camera or a camera, etc., and the imaging device 130 may be installed in a home, in a corridor, on a building in a residential area, on a street, or the like.

It can be understood that the remote camera warning system described in FIG. 1 may further include an audio playback device, and the audio playback device is connected to the camera device 130 . The audio playback device includes, for example, a smart speaker, a Bluetooth speaker, or other devices that can play audio files.

In addition, the remote camera warning system described in FIG. 1 may further include a video playback device, and the video playback device is connected to the camera device 130 . Wherein, the video playback device is, for example, a smart TV, or other devices that can play video files.

In addition, it should be noted that the numbers of the terminal device 120, the camera device 130, and the cloud 110 in FIG. 1 are only illustrative. According to implementation requirements, there may be any number of terminal devices 120 , camera devices 130 and cloud 110 .

In the embodiment of the present application, the cloud 110 can generate a high-level semantic description of the target image according to the position vectors and posture vectors of multiple target subjects including the target person in the target image. If the high-level semantic description includes a preset keyword, the cloud 110 An alarm prompt can be generated and sent to the terminal device 120, so that the user can be informed of the occurrence of the alarm event in time, and the accuracy of the alarm event judgment in the target image can be improved. The user's interactive experience with the camera device 130 .

The present application also provides a remote camera alarm method, in which the cloud performs image recognition on real-time images captured by the camera device and generates a high-level semantic description, and sends an alarm prompt to the terminal device when the high-level semantic description includes preset keywords.

Please refer to FIG. 2 . FIG. 2 is a schematic flowchart of a method for alarming through a remote camera provided by an embodiment of the present application. The method for alarming by remote camera provided by the embodiment of the present application includes the following steps:

S101: The imaging device 130 captures the target person to obtain a target image, and sends the target image to the cloud 110.

In the specific embodiment of the present application, the camera device 130 can be rotated, can be monitored in real time, and can photograph the monitoring area at preset time intervals, such as 0.003s, 0.01s, 0.06s, 0.1s, etc., which are not described here. limited. The image file captured by the camera device 130 may be picture information (eg, multiple pictures that are continuously collected at the time of acquisition), or video information (eg, a video of a certain duration, such as a video of 10s long), and so on. After the camera device 130 captures the image file, it can upload the image file to the cloud storage subspace of the cloud 110 corresponding to the second identification information of the camera device 130 at preset time intervals, and the cloud 110 can store the image files in the cloud storage subspace. Perform operations such as merging or forwarding. The sending of the image file by the camera device 130 to the cloud 110 can be realized by wireless, such as 3G, 4G or WLAN, etc. The image file shot by the camera device 130 contains the target image, and the target image is obtained by the camera device 130 shooting the target person In the image, the target person is a human appearing in the area monitored by the camera device 130 . In addition, for the definition of the second identification information and the cloud storage subspace, reference may be made to the descriptions in the foregoing embodiments, and details are not repeated here.

It can be seen that the camera device 130 monitors the monitoring area in real time, and can take pictures of the monitoring area at preset time intervals, and send the captured image files to the cloud 110 for storage, without the need for the user to use the terminal device 120 for storage, reducing the number of terminals The occupation of the memory of the device 120 can also achieve the purpose of real-time monitoring.

S102: The cloud 110 receives the target image sent by the camera device 130, and generates a high-level semantic description of the target image.

In the specific embodiment of the present application, after the cloud storage subspace of the cloud 110 receives the target image sent by the camera device 130, the cloud 110 first performs image recognition on the target image, and extracts multiple targets including the target person in the target image. The position vector (that is, the position feature) and the pose vector (that is, the pose feature) of the subject, and then the position vectors of the multiple target subjects are input into the low-level semantic unit in the semantic description model to generate a low-level semantic description, and the poses of the multiple target subjects are The vectors and the high-level semantic units in the input semantic description model above low-level semantic descriptions generate high-level semantic descriptions. Wherein, the above-mentioned low-level semantic description refers to the low-level description of the target person by means of text, and the high-level semantic description refers to the high-level description of the target person by means of text.

In a specific embodiment of the present application, the target image used to generate the high-level semantic description may be a single frame or multiple frames, the target subject in the target image may be one or more, and the target subject may include one or more targets People, target subjects can also be animals or objects, etc. Different target characters can have the same or different actions, and different target subjects can have the same or different states. As shown in FIG. 3 , FIG. 3 shows a schematic diagram of a possible target image for generating high-level semantic description applicable to an embodiment of the present application. In FIG. 3 , the target image is n frames, where n is greater than or A natural number equal to 1. The target subjects include the elderly, shoes, steps and baskets. Among them, the status of the elderly is sitting on the steps, the status of the basket is lying beside the steps, and the status of the shoes is that one is on the feet of the elderly, and the other is on the steps. On the ground, the state of several target subjects in the comprehensive picture shows that the state of the target person, the old man, in the picture is that he fell on the steps. It can be understood that the state of the target person can be determined according to the state of the multiple target subjects in the image. It should be noted that the above Figure 3 is only an example. In practical applications, the target person may be other people, such as a man, a child or a baby, and the target subject may include other objects, such as a car, a tree, a house, etc. , the target subject can also include animals, such as cats, dogs, fish, etc., the number of target subjects can also be more, the state of the target subject can also be other actions, etc. TV, basketball, etc.

Next, the specific process of generating the high-level semantic description of the target image by the cloud 110 is described in detail, and the process may include the following steps:

A1: The cloud 110 extracts the position vectors of multiple target subjects including the target person from the target image.

In a specific embodiment of the present application, the cloud 110 extracting the position vectors of multiple target subjects including the target person from the target image may be implemented by a convolutional neural network. Continuing to take the target image shown in FIG. 3 as an example, FIG. 3 includes n frames of target images P ₁ , P ₂ , . . . , P _n .

The first step is to input _n frames of target images P ₁ , P ₂ , . Take the target image Pi as an example, where 1≤i≤n, _n is a natural number, input the target image _Pi into the convolution layer for convolution operation to obtain the feature image of the target image, and obtain the specific formula of the feature image of the target image. for:

W'=conv2(W,X,'valid')+b

表示激活函数，P_i'表示目标图像的特征图像。Among them, X represents the convolution kernel, valid represents the padding method, b is the bias value, and conv2() represents the convolution operation on the target image using the convolution kernel X,

represents the activation function, and P _i ' represents the feature image of the target image.

等参数或函数可以是人为根据需要提取的特征以及目标图像P_i的大小设置的。以卷积核X的步长为例，当目标图像P_i比较大时，步长也可以比较大，当目标图像P_i比较小时，步长也可以比较小，此处不作具体限定。在本申请实施例中，在对目标图像P_i进行卷积操作之前，还可以根据需要对目标图像P_i进行去均值处理、归一化处理或者白化处理等操作。In practical applications, among the above parameters of obtaining the feature image P _i ' of the target image P _i , the convolution kernel X (number, size, step size, etc.), bias value b, activation function

The parameters or functions can be artificially set according to the features to be extracted and the size of the target image _Pi . Taking the step size of the convolution kernel X as an example, when the target image _Pi is relatively large, the step size can also be relatively large, and when the target image _Pi is relatively small, the step size can also be relatively small, which is not specifically limited here. In this embodiment of the present application, before the convolution operation is performed on the target image _Pi , operations such as de-averaging, normalization, or whitening processing may also be performed on the target image _Pi as required.

For the sake of simplicity, only the extraction of the feature images P _i ' of the target image _{Pi is} described above. _{In fact, the respective feature images P 1 ', P 2 ' , ...} , The extraction method of P _n ' is similar to the extraction method of the feature image P _i ' of the target image P _i , and will not be repeated here.

The second step, usually, after obtaining the feature image of the target image, it is necessary to input the feature image into the downsampling layer for pooling operation to obtain the pooled image. The purpose of the pooling operation is to reduce the amount of data of the feature image. The process can be:

A _i =AveragePooling(P' _i )

Among them, A _i represents the pooled image, P _i ' represents the feature image, and AveragePooling() represents the mean pooling.

In practical applications, the feature image may also be subjected to maximum pooling A _i =MaxPooling(P' _i ) to obtain a pooled image, which is not specifically limited here. Before performing the convolution operation, operations such as de-average processing, normalization processing, or whitening processing can also be performed on the n-frame target images. For the sake of simplicity, only the pooling process of the feature images P _i ' is described above. In fact, the pooling methods of the feature images P ₁ ', P ₂ ', ..., P _n ' are all the same as the pooling methods of the feature images P _i ' The method is similar, and will not be repeated here.

The third step is to expand the _n pooled images A ₁ , A ₂ , . The output is the initial feature vectors V ₁ ', V ₂ ', ..., V _n ' corresponding to the target images P ₁ , P ₂ , ..., P _n .

可以通过将初始特征向量V_i'输入位置向量层进行提取得到，其中，m为自然数，并且1≤i≤n，位置向量可以是通过大量的已知目标图像以及已知目标主体的位置向量进行训练得到。本申请实施例为了简便起见，上面只陈述了目标图像P_i中m个目标主体的位置向量

的提取，实际上，目标图像P₁、P₂、…、P_n各自的m个目标主体的位置向量的提取方式均与目标图像P_i的m个目标主体的位置向量提取方式相类似，此处不再展开赘述。 _The fourth step _is to input the initial feature vectors V _{1 '} , V ₂ ', . _{' , . _ _ _ _} The initial feature vectors V ₁ ', V ₂ ', ..., V _n ' can be spliced to obtain the target feature vectors V ₁ , V ₂ , ..., V _n of the target subject, and the target feature vectors V ₁ , V ₂ , ... , ... Inverse convolution processing, the position vector of the target subject can be obtained. Here, taking the target image P _i as an example, assuming that the target image P _i includes m target subjects, then the position vectors of the m target subjects in the target image P _i

It can be obtained by inputting the initial feature vector V _i ' into the position vector layer for extraction, where m is a natural number, and 1≤i≤n, and the position vector can be obtained through a large number of known target images and the position vector of the known target subject. Trained to get. For the sake of simplicity in the embodiments of the present application, only the position vectors of m target subjects in the target image P _i are described above.

In fact, the extraction methods of the position vectors of the m target subjects in the target images P ₁ , P ₂ , ..., P _n are all similar to the extraction methods of the position vectors of the m target subjects in the target image P _i . No further elaboration here.

In a specific embodiment of the present application, the position vector is used to represent the position of the corresponding target subject in the target image. The target subject 2 uses different target subjects 1 as a reference, and the positions of the target subjects 2 are different. The vector may include the abscissa, ordinate, width and height of the center point of the target body in the target image. The pose vector is used to represent the action of the target subject. The action of the target subject is usually different when the target subject is different. It can be understood that the pose vector is usually different. Taking the target subject as a child as an example, the actions of the target subject child can be sleeping, jumping, cheering, clapping, crying, and the like.

A2: The cloud 110 extracts pose vectors of multiple target subjects including the target person from the target image.

In a specific embodiment of the present application, the cloud 110 may also use a convolutional neural network to extract the pose vectors of multiple target subjects including the target person from the target image. Continuing to take the multi-frame image shown in FIG. 3 as an example, FIG. 3 includes n frames of target images P ₁ , P ₂ , . . . , P _n , where n is a natural number greater than or equal to 1.

The first step, the second step and the third step are the same as the first step, the second step and the third step in the above-mentioned A1 step, and will not be repeated here.

的提取，实际上，目标图像P₁、P₂、…、P_n各自的m个目标主体的姿态向量的提取方式均与目标图像P_i的m个目标主体的姿态向量提取方式相类似，此处不再展开赘述。The fourth step is to input the initial feature vectors V ₁ ', V ₂ ', ..., V _n ' output by the above-mentioned fully connected layer into the attitude vector layer, and the attitude vector layer can use the filtering and tracking algorithm to analyze the initial feature vectors V ₁ ', V 2 ', V ₂ . _{' , . _ _ _ _} The target feature vectors V ₁ , V ₂ , ..., V n of the target subject can be obtained by splicing the feature vectors V ₁ ', V ₂ ', ..., V _n ', and the target feature vectors V ₁ , V ₂ , ..., V _n can be obtained by using the Kmeans algorithm. , V _n can be calculated to obtain the attitude vector of the target subject. Taking the target image P _i as an example, assuming that the target image P _i includes m target subjects, the pose vectors Z ₁ ⁱ , Z ₂ ⁱ , . . . , Z ⁱ _m of the m target subjects in the target image _Pi can be determined by The initial feature vector V _i ' is extracted from the input pose vector layer, where m is a natural number, and 1≤i≤n, and the pose vector can be obtained by training a large number of known target images and known target subject pose vectors. For the sake of simplicity, only the pose _vectors of m target subjects in the target image Pi are stated above.

In fact, the way of extracting the pose vectors of m target subjects in each of the target images P ₁ , P ₂ , ..., P _n is similar to that of the m target subjects in the target image P _i . No further elaboration here.

A3: The cloud 110 inputs the position vectors of the multiple target subjects and the pose vectors of the multiple target subjects into the semantic description model, thereby generating a high-level semantic description of the target image.

In a specific embodiment of the present application, the semantic description model can be expressed as:

y=f(x)

Among them, x is the influence factor of the advanced semantic description, y is the advanced semantic description, and f() is the mapping relationship between the influence factor of the advanced semantic description and the advanced semantic description. f() can be obtained by training a large number of known high-level semantic description influence factors and known high-level semantic descriptions.

In a specific embodiment, the semantic description model may be as shown in FIG. 4 , and the low-level semantic description can be obtained by inputting the position vectors of multiple target subjects including the target person extracted in the above-mentioned embodiment into the low-level semantic unit; The high-level semantic description can be obtained by inputting the gesture vectors of multiple target subjects including the target person extracted in the above embodiment into the high-level semantic unit in combination with the above-mentioned low-level semantic description. Among them, the low-level semantic description refers to the low-level description of the target person through text, and the high-level semantic description refers to the high-level description of the target person through text. For different target people, the generated low-level semantic description and high-level semantic description are usually different. same.

For example, continuing to take the image shown in Figure 3 as an example, suppose that the extracted low-level semantic description of the target person, the old man, is "the old man is above the steps", and the low-level semantic description of the target subject's shoes is "a shoe is on the old man's foot, a Only the shoes are in front of the old man's feet", and then these two low-level semantic descriptions are combined with the pose vectors of the target person, the old man and the target subject's shoes, extracted from the above embodiment, and input into the high-level semantic description unit to generate the high-level semantic description "The old man fell from the steps. fall".

In the specific embodiment of the present application, the feature vector extraction, the position vector extraction and the attitude vector extraction in the above embodiments may be implemented in the same convolutional neural network, or may be implemented in different convolutional neural networks respectively. Wherein, the convolutional neural network may include VGGNet, ResNet, FPNet, etc., which is not specifically limited here.

In the specific embodiment of the present application, the generation of the low-level semantic description and the generation of the high-level semantic description in the above-mentioned embodiments may be integrated in the same recurrent neural network, or may be generated by different recurrent neural networks, which are not described here. limited. The recurrent neural network may include a long short-term memory model (Long short-term memory, LSTM), a bidirectional long short-term memory model (Bi Long short-term memory, BiLSTM), etc., which are not specifically limited here.

It can be seen that the embodiment of the present application can use the convolutional neural network to extract the position vector and attitude vector of multiple target subjects including the target person in the target image, and use the recurrent neural network to generate high-level semantic description, so as to better It can accurately describe the state between multiple target subjects in the target image, and obtain a more accurate image semantic description.

S103: In the case that the high-level semantic description includes a preset keyword, the cloud 110 generates an alarm prompt and sends it to the terminal device 120.

In a specific embodiment of the present application, if the high-level semantic description of the target image generated by the cloud 110 includes preset keywords, such as "fall", "fall", "crying", "struggle", "twitch", "blood" and other keywords, the cloud 110 generates an alarm prompt and sends it to the terminal device 120, wherein the alarm prompt may include a high-level semantic description. Continuing to take the image shown in Figure 3 as an example, it is assumed that the generated high-level semantic description of the target person, the old man, is "the old man fell from the steps", and the high-level semantic description includes the keyword "fall", so the cloud 110 generates An alarm prompt of "the old man fell from the steps" is sent to the terminal device 120 .

It can be seen that, in the embodiment of the present application, when the high-level semantic description generated by the cloud 110 includes preset keywords, an alarm prompt is generated and sent to the terminal device 120, so that the user can know the location and type of the alarm event as soon as possible, without the need for The user downloads and views the image file or activates the camera device 130 to view the monitoring area so as to subjectively judge the event in the image, which improves the accuracy of the event judgment and saves the user's time.

S104: The terminal device 120 receives the alarm prompt sent by the cloud 110.

In the specific embodiment of the present application, the terminal device 120 sends a call request to the camera device 130 in the case of receiving the alarm prompt sent by the cloud 110 . Wherein, if the terminal device 120 accesses the network through the wireless local area network (WLAN), the terminal device 120 sends a call request to the camera device 130 to establish a video call with the camera device 130; When 3G, 4G, etc. are connected to the network, the terminal device 120 sends a call request to the camera device 130 to establish a voice call with the camera device 130 . In addition, when the terminal device 120 establishes a voice call with the camera device 130, the voice call interface of the terminal device 120 displays a video call shortcut key. If the video call shortcut key is triggered, the terminal device 120 stops the voice call with the camera device 130. , switch to the video call with the camera device 130 .

Continuing to take the image shown in FIG. 3 above as an example, the terminal device 120 receives the alarm prompt "the old man fell from the steps" sent by the cloud 110, and the user can know the alarm event of "the old man fell from the steps". In the case where the device 120 is connected to the network through WLAN, as shown in FIG. 5 , FIG. 5 is a schematic diagram of a possible video call interface between the terminal device 120 and the camera device 130 , and the “Start” shortcut is displayed on the video call interface of the terminal device 120 key or the "end" shortcut key, etc., the user clicks "start" to establish a video call with the camera device 130 to view the screen of the monitoring area of the camera device 130, and to check the specific situation of the elderly falling from the steps. The user clicks "End" to terminate the video call with the camera device 130 . In the case where the terminal device 120 is not connected to the network through WLAN, as shown in FIG. 6, FIG. 6 is a schematic diagram of a possible voice call interface between the terminal device 120 and the camera device 130, and the voice call interface of the terminal device 120 displays “ Start” shortcut key, “video call” shortcut key, built-in application icon (such as camera or photo album) or “end” shortcut key, etc., the user can click “Start” to establish a voice call with the camera device 130 to the camera device 130 Send a voice message; the user clicks the "video call" shortcut key to terminate the voice call with the camera device 130, establish a video call with the camera device 130 to view the screen of the monitoring area of the camera device 130; the user clicks the built-in application icon (such as the camera or photo album), you can enter the built-in application to see the icons of all the camera devices 130 bound to the terminal device 120, and can establish a video call or a voice call with any camera device 130. Switch the camera device 130 in the middle; the user can click "End" to terminate the voice call with the camera device 130 .

In the specific embodiment of the present application, the terminal device 120 can also view the image files in the cloud storage space and the cloud storage subspace on the cloud 110 and download, update or delete them as needed.

In the above solution, the cloud 110 can generate a high-level semantic description of the target image according to the position vectors and posture vectors of multiple target subjects including the target person in the target image. If the high-level semantic description includes a preset keyword, the cloud 110 can generate a high-level semantic description. The alarm prompt is sent to the terminal device 120, so that the user can be informed of the occurrence of the alarm event in time, and the terminal device 120 is provided with a "start" shortcut key, a "video call shortcut key" and a built-in application icon, etc. The device 130 conducts a video call or a voice call, which improves the user's interactive experience with the camera device 130 . In addition, the embodiments of the present application are deeply combined with the terminal device 120, the cloud 110 and the camera device 130, and the built-in application (such as a camera or photo album application) of the terminal device 120 has a built-in entrance of the camera device 130. When the camera device 130 interacts, the user does not need to install third-party application software on the terminal device 120 to interact with the camera device 130, which can prevent the image files captured by the camera device 130 from being acquired by third-party software and leak user privacy.

The embodiment of the present application relates to the cloud 110, as shown in FIG. 7, which is a schematic diagram of a possible cloud 100 involved in the present application. The cloud computing infrastructure of the cloud 100 deployed by the owner of the cloud itself, ie, deploying computing resources 111 (eg, servers), deploying storage resources 112 (eg, memory), and deploying network resources 113 (eg, network cards), and so on. Then, the owner of the public cloud (eg, an operator) virtualizes the computing resources 111 , storage resources 112 , and network resources 113 of the cloud computing infrastructure, and provides corresponding services for the users (eg, users) of the cloud to use . Among them, operators can provide the following three services to users: cloud computing infrastructure as a service (IaaS), platform as a service (PaaS) and software as a service (SaaS) .

The service provided by IaaS to the user is the utilization of cloud computing infrastructure, including processing, storage, network and other basic computing resources 111 , and the user can deploy and run any software, including operating systems and applications, in the cloud 100 . Users do not manage or control any cloud computing infrastructure, but can control the choice of operating system, storage space, deploy applications, and may also gain control of limited network components (eg, firewalls, load balancers, etc.).

The service provided by PaaS to users is to deploy applications developed or acquired by users using development languages and tools (such as Java, python, Net, etc.) provided by suppliers to cloud computing infrastructure. Users do not need to manage or control the underlying cloud computing infrastructure, including networks, servers, operating systems, storage, etc., but users can control the deployed applications and possibly the configuration of the hosting environment in which the applications run.

The service provided by SaaS to the user is the application program that the operator runs on the cloud computing infrastructure, and the user can access the application program on the cloud computing infrastructure through the client interface, such as a browser, on various terminal devices. Users do not need to manage or control any cloud computing infrastructure, including networks, servers, operating systems, storage, and more.

It can be understood that operators provide leasing services for different tenants through any one of IaaS, PaaS, and SaaS, and the data and configurations between different tenants are isolated from each other, thereby ensuring the security and privacy of each tenant's data.

Those skilled in the art can understand that the cloud shown in FIG. 7 does not constitute a limitation on the cloud, and may include more or less services or facilities than the one shown, or combine some services or facilities, or split some Services or facilities, or different service assignments or facility arrangements.

The embodiments of the present application relate to a terminal device 120, and the terminal device 120 may be a mobile phone, a tablet computer, a personal digital assistant (PDA), a mobile internet device (MID), a notebook computer, or a smart wearable device (such as Various terminal devices 120 such as smart watches and smart bracelets) are not limited in the embodiments of the present application.

Taking the terminal device 120 as a mobile phone as an example, FIG. 8 is a block diagram showing a partial structure of the mobile phone 200 related to the embodiment of the present application. 8 , the mobile phone 200 includes a memory 211 , a processor 212 , an input/output (I/O) subsystem 213 , other input device controllers 214 , other input devices 215 , a display controller 216 , and a display screen 217 And components such as sensor controller 218. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 8 does not constitute a limitation on the mobile phone, and may include more or less components than the one shown in the figure, or combine some components, or disassemble some components, or Different component arrangements.

Below in conjunction with FIG. 8, each constituent component of the mobile phone 200 will be specifically introduced:

The memory 211 can be used to store software programs and modules, and the processor 212 executes various functional applications and data processing of the mobile phone 200 by running the software programs and modules stored in the memory 211 . The memory 211 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playback function, an image playback function, etc.) required for at least one function, and the like; The use of the cell phone 200 creates data (such as audio data, phone book, etc.) and the like. In addition, memory 211 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The display screen 217 may be used to display information input by or provided to the user and various menus of the cell phone 200, and may also accept user input. Optionally, the display screen 217 may include a display panel and a touch panel. The display panel can be configured in the form of a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (organic light-emitting diode, OLED) or the like. A touch panel, also known as a touch screen, touch-sensitive screen, etc., collects the user's contact or non-contact operations on or near it (such as the user's finger, stylus, etc., any suitable object or accessory on the touch panel or in the The operation near the touch panel may also include somatosensory operation; the operation includes operation types such as single-point control operation, multi-point control operation, etc.), and the corresponding connection device is driven according to a preset program.

The I/O subsystem 213 is used to control input and output external devices, which may include other input device controllers 214 , sensor controllers 218 , and display controllers 216 . Optionally, one or more other input device controllers 214 receive signals from and/or send signals to other input devices 215, which may include physical buttons (push buttons, rocker buttons, etc.), Dials, slide switches, joysticks, click wheels, light mice (light mice are touch-sensitive surfaces that do not display visual output, or are extensions of touch-sensitive surfaces formed by a touch screen). It should be noted that other input device controllers 214 may be connected to any one or more of the above-mentioned devices. The display controller 216 in the I/O subsystem 213 receives signals from and/or sends signals to the display screen 217 . After the display screen 217 detects the user input, the display controller 216 converts the detected user input into interaction with the user interface objects displayed on the display screen 217 , that is, to realize human-computer interaction. Sensor controller 218 may receive signals from and/or send signals to one or more sensors.

The processor 212 is the control center of the mobile phone 200, using various interfaces and lines to connect various parts of the entire mobile phone, by running or executing the software programs and/or modules stored in the memory 211, and calling the data stored in the memory 211, Execute various functions of the mobile phone 200 and process data, so as to monitor the mobile phone as a whole. Optionally, the processor 212 may include one or more processing units; preferably, the processor 212 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc. , the modem processor mainly deals with wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 212 .

Although not shown, the mobile phone 200 may further include a power supply (such as a battery) for supplying power to various components, an RF (Radio Frequency) circuit, an audio circuit, a sensor, a camera, a Bluetooth module, etc., which will not be repeated here.

The embodiment of the present application relates to a camera device 130. The camera device 130 may be a variety of camera devices 130 such as an analog camera, a network camera, or a smart camera. The present application uses a network camera as an example, as shown in FIG. 9 . A block diagram of a partial structure of a possible network camera 300 involved in this application. IP camera 300, also known as IP camera (IP camera, IPC), adopts embedded architecture and integrates various functions such as video and audio acquisition, signal processing, encoding and compression, front-end storage and network transmission, etc., combined with network video storage and recording system and The management platform software can form a large-scale and distributed network video surveillance system. Referring to FIG. 9 , the network camera 300 includes a lens and a sensor 311 , an encoding processor 312 , and a network camera main control board 313 and other components. Those skilled in the art can understand that the structure of the network camera shown in FIG. 9 does not constitute a limitation on the camera, and may include more or less components than those shown in the figure, or combine some components, or split some components, Or a different component arrangement.

In the following, each component of the network camera 300 will be introduced in detail with reference to FIG. 9 :

The lens in the lens and the sensor 311 is the key equipment of the video surveillance system, and its quality directly affects the overall quality of the network camera 300 . The lens can be used to image the outside world on the sensor. At present, the lens of the IP camera 300 is threaded and usually consists of a group of lenses and a diaphragm. The lens has a manual iris (MI) and an automatic iris (auto iris). , AI), the manual aperture lens is suitable for occasions where the brightness is unchanged, and the automatic aperture lens will automatically adjust its aperture when the brightness changes, so it is suitable for occasions where the brightness changes. Optionally, the lens may be a standard lens, a telephoto lens, a zoom lens, or a variable focus lens, etc., and the material of the lens may be glass or plastic.

The sensor in the lens and sensor 311 can be an image sensor, such as a charge-coupled device (CCD) sensor or a complementary metal oxide semiconductor (complementary metal oxide semiconductor, CMOS) sensor, etc., for converting the light signal ( The image of the object) is converted into an electrical signal, and output to the encoding processor 312 through the driving circuit. The encoding processor 312 optimizes the digital image signal collected by the lens and the sensor 311, such as color, sharpness or white balance, etc., Then it is input to the network camera main control board 313 in the form of network video signal. The network camera main control board 313 has bayonet nut connector (BNC) video output, network communication interface, audio input, audio output, alarm output, alarm Input, serial communication interface and other functions. The encoding processor 312 is used to optimize the digital image signals transmitted from the lens and the sensor 311. The encoding processor 312 may include an image signal processor (ISP) or an image decoder, etc., which are not specifically described here. limited.

Although not shown, the IP camera 300 may also include a power source (such as a battery) for supplying power to various components, an optical filter or a Bluetooth module, etc., which will not be repeated here.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. Interchangeability, the above description has generally described the components and steps of each example in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of the description, for the specific working process of the cloud 110, the terminal device 120, the camera device 130 and the unit described above, reference may be made to the corresponding process in the foregoing method embodiments. This will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application are essentially or part of contributions to the prior art, or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in the present application. Modifications or substitutions shall be covered by the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1, A method for alarming by remote camera shooting, which is characterized in that the method comprises:

the camera shooting device shoots a target person to obtain a target image and sends the target image to the cloud;

the cloud receives the target image sent by the camera equipment and generates high-level semantic description of the target image, wherein the high-level semantic description is used for performing high-level description on the target person in a text mode;

under the condition that the high-level semantic description comprises preset keywords, the cloud generates an alarm prompt and sends the alarm prompt to the terminal equipment;

and the terminal equipment receives the alarm prompt sent by the cloud.

2. The method of claim 1, wherein the cloud generates a high-level semantic description of the target image, comprising:

the cloud extracts position vectors of a plurality of target subjects including target characters from the target image;

the cloud extracts attitude vectors of a plurality of target subjects including target characters from the target image;

and the cloud end inputs the position vectors of the target subjects and the posture vectors of the target subjects into a semantic description model so as to generate high-level semantic description of the target image.

3. The method of claim 2, wherein the semantic description model comprises a low-level semantic unit and a high-level semantic unit, and wherein inputting the position vectors of the plurality of target subjects and the pose vectors of the plurality of target subjects into the semantic description model to generate a high-level semantic description of the target image comprises:

inputting the position vectors of the plurality of target subjects into the low-level semantic unit to obtain a low-level semantic description;

inputting the pose vectors of the plurality of target subjects and the low-level semantic description into the high-level semantic unit to obtain the high-level semantic description.

4. The method of any one of claims 1 to 3, , wherein the method further comprises:

the terminal equipment sends the call request to the camera equipment under the condition of receiving the alarm prompt sent by the cloud end;

under the condition that the terminal equipment is accessed to a network through a Wireless Local Area Network (WLAN), the terminal equipment establishes a video call with the camera equipment;

and under the condition that the terminal equipment is not accessed into the network through a Wireless Local Area Network (WLAN), the terminal equipment establishes a voice call with the camera equipment.

5. The method according to claim 4, wherein in a case where the terminal apparatus establishes a voice call with the image pickup apparatus, the method further comprises:

displaying a video call shortcut key on a voice call interface of the terminal equipment;

and under the condition that the video call shortcut key is triggered, the terminal equipment stops the voice call with the camera equipment and establishes the video call with the camera equipment.

6, A warning system by remote camera shooting, comprising:

the camera device is used for shooting a target person to obtain a target image and sending the target image to the cloud;

the cloud is used for receiving the target image sent by the camera equipment and generating high-level semantic description of the target image, wherein the high-level semantic description is used for performing high-level description on the target person in a text mode;

under the condition that the high-level semantic description comprises preset keywords, the cloud is further used for generating an alarm prompt and sending the alarm prompt to the terminal equipment;

the terminal equipment is used for receiving the alarm prompt sent by the cloud.

7. The system of claim 6, wherein the cloud is configured to generate a high-level semantic description of the target image, comprising:

the cloud is used for extracting position vectors of a plurality of target subjects including target characters from the target image;

the cloud is used for extracting attitude vectors of a plurality of target subjects including target characters from the target image;

the cloud is used for inputting the position vectors of the target subjects and the posture vectors of the target subjects into a semantic description model so as to generate high-level semantic description of the target image.

8. The system of claim 7, wherein the semantic description model comprises a low-level semantic unit and a high-level semantic unit, and wherein inputting the position vectors of the plurality of target subjects and the pose vectors of the plurality of target subjects into the semantic description model to generate a high-level semantic description of the target image comprises:

inputting the position vectors of the plurality of target subjects into the low-level semantic unit to obtain a low-level semantic description;

inputting the pose vectors of the plurality of target subjects and the low-level semantic description into the high-level semantic unit to obtain the high-level semantic description.

9. The system of any one of claims 6 to 8, , further comprising:

the terminal equipment is used for sending the call request to the camera equipment under the condition of receiving the alarm prompt sent by the cloud end;

under the condition that the terminal equipment is accessed to a network through a Wireless Local Area Network (WLAN), the terminal equipment establishes a video call with the camera equipment;

and under the condition that the terminal equipment is not accessed into the network through a Wireless Local Area Network (WLAN), the terminal equipment establishes a voice call with the camera equipment.

10. The system according to claim 9, wherein in a case where the terminal apparatus establishes a voice call with the image pickup apparatus, the system further comprises:

displaying a video call shortcut key on a voice call interface of the terminal equipment;

and under the condition that the video call shortcut key is triggered, the terminal equipment stops the voice call with the camera equipment and establishes the video call with the camera equipment.

Images