WO2019242222A1 - Method and device for use in generating information - Google Patents

Abstract

Disclosed are a method and device for use in generating information. One embodiment of the method comprises: acquiring a target video; inputting the target video into a pretrained video recognition model to acquire a recognition result pair corresponding to the target video, where the video recognition model is used for expressing correlations between videos and recognition result pairs, the recognition result pairs comprise first recognition results and second recognition results, the first recognition results are used for expressing foregrounds that the videos comprise, and the second recognition results are used for expressing backgrounds that the videos comprise. The embodiment increases the diversity of information generation.

Description

Method and device for generating information

This patent application claims the priority of a Chinese patent application filed on June 21, 2018, with application number 201810644175.4, the applicant being Beijing BYTE Network Technology Co., Ltd., and the invention name "Methods and Devices for Generating Information" , The entire application of which is incorporated herein by reference.

Technical field

Embodiments of the present application relate to the field of computer technology, and in particular, to a method and an apparatus for generating information.

Background technique

Video usually includes foreground and background. The foreground may include the shooting content (such as people, animals, behaviors, etc.) corresponding to the captured video; the background may include the shooting scene (such as the sky, stadium, forest, etc.) corresponding to the captured video.

Currently, video recognition usually only identifies the foreground of a video.

Summary of the Invention

The embodiments of the present application provide a method and device for generating information.

In a first aspect, an embodiment of the present application provides a method for generating information. The method includes: obtaining a target video; inputting the target video into a pre-trained video recognition model to obtain a pair of recognition results corresponding to the target video, wherein, The video recognition model is used to characterize the correspondence between the video and the recognition result pair. The recognition result pair includes a first recognition result and a second recognition result. The first recognition result is used to characterize the foreground included in the video, and the second recognition result is used to characterize the video. Included background.

In some embodiments, the video recognition model includes a first video recognition sub-model, a second video recognition sub-model, and a feature extraction network; and inputting a target video into a pre-trained video recognition model to obtain a pair of recognition results corresponding to the target video, The method includes: extracting a target video from a feature extraction network to obtain video features of the target video; and inputting the obtained video features to a first video recognition sub-model and a second video recognition sub-model, respectively, to obtain a target video corresponding to the first video recognition The result is a pair of recognition results with the second recognition result.

In some embodiments, the video recognition model is obtained by training as follows: obtaining a training sample set, wherein the training sample includes a sample video and a pair of sample recognition results pre-labeled for the sample video; and using the sample video of the training sample in the training sample set as As input, the pair of sample recognition results corresponding to the input sample video is taken as the desired output, and a video recognition model is obtained by training using a machine learning method.

In some embodiments, the sample video of the training samples in the training sample set is used as an input, the sample recognition result pair corresponding to the input sample video is used as an output, and a video recognition model is trained by using a machine learning method, including: training samples The set is divided into a preset number of training sample groups; a training sample group is selected as a candidate training sample group from the preset number of training sample groups, and based on the candidate training sample group and a predetermined initial model, the following training steps are performed: for the candidate The training sample group takes the sample video of the training sample as an input, and uses the pair of sample recognition results corresponding to the input sample video as an output. The initial model is trained using the machine learning method to obtain the initial video recognition model. Determining whether the completion condition of the training is satisfied; in response to determining that the completion condition is satisfied, generating a video recognition model based on the obtained initial video recognition model; in response to determining that the completion condition is not satisfied, selecting a training from a group of training samples that have not been selected Sample group as new Selected training sample set, and the initial video recognition model is obtained most recently as a new initial model, continuing its training procedures.

In some embodiments, the completion condition includes, but is not limited to, at least one of the following: the preset number of training sample groups does not include unselected training sample groups; and inputting a sample video of training samples in candidate training sample groups into the initial model A loss value of the obtained actual recognition result pair with respect to the sample recognition result pair corresponding to the input sample video is less than a preset loss threshold.

In a second aspect, an embodiment of the present application provides an apparatus for generating information. The apparatus includes: an acquiring unit configured to acquire a target video; an input unit configured to input the target video into a pre-trained video recognition model, Obtain a pair of recognition results corresponding to the target video, where the video recognition model is used to characterize the correspondence between the video and the pair of recognition results, the pair of recognition results includes a first recognition result and a second recognition result, and the first recognition result is used to represent the The included foreground and the second recognition result are used to characterize the background included in the video.

In some embodiments, the video recognition model includes a first video recognition sub-model, a second video recognition sub-model, and a feature extraction network; and the input unit includes: a first input module configured to input a target video into the feature extraction network to obtain Video features of the target video; a second input module configured to input the obtained video features into the first video recognition sub-model and the second video recognition sub-model, respectively, to obtain the corresponding target video including the first recognition result and the first video recognition sub-model. The recognition result pairs of the two recognition results.

In some embodiments, the video recognition model is obtained by training as follows: obtaining a training sample set, wherein the training sample includes a sample video and a pair of sample recognition results pre-labeled for the sample video; and using the sample video of the training sample in the training sample set as As input, the pair of sample recognition results corresponding to the input sample video is taken as the desired output, and a video recognition model is obtained by training using a machine learning method.

In some embodiments, the sample video of the training samples in the training sample set is used as an input, the sample recognition result pair corresponding to the input sample video is used as an output, and a video recognition model is trained by using a machine learning method, including: training samples The set is divided into a preset number of training sample groups; a training sample group is selected as a candidate training sample group from the preset number of training sample groups, and based on the candidate training sample group and a predetermined initial model, the following training steps are performed: for the candidate The training sample group takes the sample video of the training sample as an input, and uses the pair of sample recognition results corresponding to the input sample video as an output. The initial model is trained using the machine learning method to obtain the initial video recognition model. Determining whether the completion condition of the training is satisfied; in response to determining that the completion condition is satisfied, generating a video recognition model based on the obtained initial video recognition model; in response to determining that the completion condition is not satisfied, selecting a training from a group of training samples that have not been selected Sample group as new Selected training sample set, and the initial video recognition model is obtained most recently as a new initial model, continuing its training procedures.

In some embodiments, the completion condition includes, but is not limited to, at least one of the following: the preset number of training sample groups does not include unselected training sample groups; and inputting a sample video of training samples in candidate training sample groups into the initial model A loss value of the obtained actual recognition result pair with respect to the sample recognition result pair corresponding to the input sample video is less than a preset loss threshold.

According to a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors; a storage device storing one or more programs thereon; when one or more programs are processed by one or more processors Execution causes one or more processors to implement the method of any one of the foregoing methods for generating information.

In a fourth aspect, an embodiment of the present application provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the method of any one of the foregoing methods for generating information is implemented.

The method and device for generating information provided by the embodiments of the present application obtain a target video corresponding to the target video by acquiring the target video and then inputting the target video into a pre-trained video recognition model, where the video recognition model is used for characterization Correspondence between the video and the recognition result pair. The recognition result pair includes a first recognition result and a second recognition result. The first recognition result is used to characterize the foreground included in the video, and the second recognition result is used to characterize the background included in the video. The pre-trained video recognition model can be used to identify the foreground and background of the target video at the same time, which improves the diversity of information generation.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features, objects, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present application can be applied; FIG.

2 is a flowchart of an embodiment of a method for generating information according to the present application;

3 is a schematic diagram of an application scenario of a method for generating information according to the present application;

4 is a flowchart of another embodiment of a method for generating information according to the present application;

5 is a schematic structural diagram of an embodiment of an apparatus for generating information according to the present application;

FIG. 6 is a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.

detailed description

The following describes the present application in detail with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. It should also be noted that, for convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The application will be described in detail below with reference to the drawings and embodiments.

FIG. 1 illustrates an exemplary system architecture 100 of an embodiment of a method for generating information or an apparatus for generating information to which the present application can be applied.

As shown in FIG. 1, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages and the like. Various communication client applications, such as model training applications, video recognition applications, web browser applications, social platform software, and the like can be installed on the terminal devices 101, 102, and 103.

The terminal devices 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with a display screen, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Pictures Experts Group Audio Audio Layer III, Motion picture expert compression standard audio layer 3), MP4 (Moving Picture Experts Group Audio Layer 4), player, laptop portable computer and desktop computer, etc. When the terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, multiple software or software modules used to provide distributed services), or it can be implemented as a single software or software module. It is not specifically limited here.

When the terminals 101, 102, and 103 are hardware, a video capture device may also be installed on the terminals. The video capture device can be a variety of devices that can capture video, such as cameras, sensors, and so on. Users can use the video capture devices on the terminals 101, 102, 103 to capture video.

The server 105 may be a server that provides various services, such as a background server that processes videos displayed on the terminal devices 101, 102, and 103. The background server can perform analysis and other processing on the received target video and other data, and can feed back the processing results (such as the recognition result pair) to the terminal device.

It should be noted that the server may be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers or as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, multiple software or software modules used to provide distributed services), or it can be implemented as a single software or software module. It is not specifically limited here.

It should be understood that the numbers of terminal devices, networks, and servers in FIG. 1 are merely exemplary. Depending on the implementation needs, there can be any number of terminal devices, networks, and servers. In particular, in a case where the target face video or the data used in the process of generating the recognition result does not need to be obtained from a remote place, the above-mentioned system architecture may not include a network but only a terminal device or a server.

With continued reference to FIG. 2, a flowchart 200 of one embodiment of a method for generating information according to the present application is shown. The method for generating information includes the following steps:

Step 201: Obtain a target video.

In this embodiment, an execution subject (for example, a server shown in FIG. 1) of the method for generating information may obtain a target video by using a wired connection method or a wireless connection method. The target video may be a video to be identified.

It should be noted that the above-mentioned execution subject may obtain the target video sent by the electronic device (for example, the terminal device shown in FIG. 1) in communication connection therewith, or may obtain the target video stored in the local area in advance.

Step 202: Input a target video into a pre-trained video recognition model to obtain a pair of recognition results corresponding to the target video.

In this embodiment, based on the target video obtained in step 201, the execution subject may input the target video into a pre-trained video recognition model to obtain a pair of recognition results corresponding to the target video. Among them, the video recognition model can be used to characterize the correspondence between the video and the recognition result pair. The recognition result pair includes a first recognition result and a second recognition result. The first recognition result may be used to characterize the foreground included in the video. The second recognition result can be used to characterize the background included in the video. The recognition results (the first recognition result and the second recognition result) in the recognition result pair may include, but are not limited to, at least one of the following: text, numbers, symbols, images, videos.

Here, it can be understood that the foreground included in the video usually refers to the shooting content (such as people, animals, behaviors, etc.) corresponding to the video, and the background included in the video usually refers to the shooting scene (such as Sky, stadium, forest, etc.).

In some optional implementations of this embodiment, the video recognition model may be obtained by training as follows: First, a training sample set is obtained, where the training sample includes a sample video and a sample recognition result pair that is pre-labeled for the sample video. Then, the sample video of the training samples in the training sample set is used as an input, and the pair of sample recognition results corresponding to the input sample video is used as the desired output, and a video recognition model is obtained by training using a machine learning method.

Specifically, as an example, training samples can be selected from the training sample set and the following steps are performed: the sample video of the selected training samples is input to an initial model (such as a Convolutional Neural Network (CNN), a residual network ( ResNet), etc.) to obtain the recognition result pair; use the sample recognition result pair corresponding to the input sample video as the expected output of the initial model, and adjust the parameters of the initial model based on the obtained recognition result pair and sample recognition result pair; determine Whether there are unselected training samples in the training sample set; and in response to the absence of unselected training samples, the adjusted initial model is determined as a video recognition model. It should be noted that the manner of selecting training samples is not limited in this application. For example, it may be randomly selected, or training samples with better sharpness of the sample video may be preferentially selected.

In some optional implementations of this embodiment, the video recognition model described above may also be obtained by training as follows:

First, obtain a training sample set, and divide the training sample set into a preset number of training sample groups.

Here, the training sample set can be divided into a preset number of training sample groups in various ways. For example, the training sample set can be divided into a preset number of training sample groups in an equal division manner, or the training sample set can be divided such that the training included in each training sample group in the preset number of training sample groups The number of samples is greater than or equal to a preset threshold. It should be noted that the preset number can be set in advance by a technician.

Then, a training sample group can be selected as a candidate training sample group from a preset number of training sample groups, and based on the candidate training sample group and a predetermined initial model, the following training steps are performed: For the candidate training sample group, The sample video is used as an input, and the sample recognition result pair corresponding to the input sample video is used as an output. The initial model is trained by using a machine learning method to obtain an initial video recognition model. It is determined whether a preset completion condition is used to indicate the completion of training. Yes; in response to determining that the completion conditions are met, a video recognition model is generated based on the obtained initial video recognition model.

Here, an initial video recognition model may be selected from the obtained initial video recognition models as a video recognition model, or the obtained initial video recognition model may be processed (fused) to obtain a video recognition model.

It should be noted that the selection method of the training sample group is not limited in this application. For example, it may be randomly selected, or a training sample group with more training samples may be preferentially selected.

In addition, in response to determining that the completion conditions are not satisfied, a training sample group is selected from the unselected training sample group as a new candidate training sample group, and the newly obtained initial video recognition model is used as a new initial model, and the process continues. Perform the training steps described above.

It should be noted that the execution subject of the steps for obtaining the video recognition model may be the same as or different from the execution subject of the method for generating information. If they are the same, the executing subject of the step for obtaining the video recognition model may store the trained video recognition model locally after training to obtain the video recognition model. If they are different, the execution subject of the step for obtaining the video recognition model may send the trained video recognition model to the execution subject of the method for generating information after training to obtain the video recognition model.

In some optional implementations of this embodiment, the above completion conditions may include, but are not limited to, at least one of the following: a preset number of training sample groups does not include an unselected training sample group; and a candidate training sample group The loss value of the actual recognition result pair obtained from the input of the initial model of the training sample of the training sample relative to the sample recognition result pair corresponding to the input sample video is less than the preset loss threshold.

With continued reference to FIG. 3, FIG. 3 is a schematic diagram of an application scenario of the method for generating information according to this embodiment. In the application scenario of FIG. 3, the terminal 301 first sends a target video (video obtained by shooting a kite) 302 to the server 303. Then, the server 303 obtains the target video 302, and inputs the target video 302 into a pre-trained video recognition model 304 to obtain a recognition result pair 305 corresponding to the target video 302. Among them, the video recognition model can be used to characterize the correspondence between the video and the recognition result pair. The recognition result pair 305 includes a first recognition result (kite) 3051 and a second recognition result (sky) 3052. The first recognition result 3051 can be used to characterize the foreground included in the video, and the second recognition result 3052 can be used to characterize the video. Background.

The method provided by the above embodiments of the present application obtains a target video and then inputs the target video into a pre-trained video recognition model to obtain a pair of recognition results corresponding to the target video. The video recognition model is used to characterize the relationship between the video and the recognition result pair. Correspondence, the recognition result pair includes a first recognition result and a second recognition result. The first recognition result is used to characterize the foreground included in the video, and the second recognition result is used to characterize the background included in the video, so that a pre-trained video can be used. Recognition model, while identifying the foreground and background of the target video, improves the diversity of information generation.

With further reference to FIG. 4, a flowchart 400 of still another embodiment of a method for generating information is shown. The process 400 of the method for generating information includes the following steps:

Step 401: Obtain a target video.

In this embodiment, an execution subject (for example, a server shown in FIG. 1) of the method for generating information may obtain a target video by using a wired connection method or a wireless connection method.

It should be noted that step 401 may be implemented in a manner similar to step 201 in the foregoing embodiment. Accordingly, the above description of step 201 is also applicable to step 401 of this embodiment, and details are not described herein again.

Step 402: Input the target video into a feature extraction network of a pre-trained video recognition model to obtain video features of the target video.

In this embodiment, the video recognition model may include a feature extraction network. Further, based on the target video obtained in step 401, the execution subject may input the target video into the feature extraction network of the video recognition model to obtain the video features of the target video.

It can be understood that the target video is essentially a target image sequence arranged in chronological order. Therefore, the video characteristics of the target video can be reflected by the image characteristics of the target image in the target image sequence.

In this embodiment, the feature extraction network may be used to extract the image features of the target image corresponding to the target video, and based on the image features, generate the video features and output corresponding to the target video.

Specifically, the execution subject may directly determine the obtained image feature as the video feature corresponding to the target video, or may process the obtained image feature and determine the processed image feature as the video corresponding to the target video. feature. As an example, the above-mentioned execution subject may fuse the obtained image features to obtain the fused features, and then determine the fused features as the video features corresponding to the target video.

Here, the feature extraction network may include a structure (such as a convolution layer) for extracting image features, and of course, may also include other structures (such as a pooling layer), which is not limited herein.

Step 403: Input the obtained video features into the first video recognition sub-model and the second video recognition sub-model of the video recognition model, and obtain the recognition result pair corresponding to the target video including the first recognition result and the second recognition result. .

In this embodiment, the video recognition model may further include a first video recognition sub-model and a second video recognition sub-model. Further, the above-mentioned execution body may input the obtained video features into the first video recognition sub-model of the video recognition model, respectively. And the second video recognition sub-model to obtain a pair of recognition results corresponding to the target video and including the first recognition result and the second recognition result. The first recognition result may be used to characterize the foreground included in the video. The second recognition result can be used to characterize the background included in the video. The recognition results (the first recognition result and the second recognition result) in the recognition result pair may include, but are not limited to, at least one of the following: text, numbers, symbols, images, videos.

In this embodiment, the first video recognition sub-model is connected to a feature extraction network for generating a first recognition result based on the input video features. The second video sub-model is connected to a feature extraction network, and is configured to generate a second recognition result based on the input video features. Here, the first video recognition sub-model and the second video sub-model may include a structure for generating results (for example, a classifier, a fully connected layer), and of course, other structures (for example, an output layer) may be included, which is not limited here. .

As can be seen from FIG. 4, compared with the embodiment corresponding to FIG. 2, the process 400 of the method for generating information in this embodiment highlights the target video input feature extraction network, obtains the video features of the target video, and And using the obtained video features as shared features to input a first video recognition sub-model and a second video recognition sub-model, respectively, and then obtain a pair of recognition results. Therefore, the solution described in this embodiment can use the overall features (including foreground features and background features) of the target video to generate a first recognition result and a second recognition result. For the first recognition result, the background feature is added. For reference data, for the second recognition result, reference data of foreground features is added, which can realize more accurate video recognition and improve the accuracy of information generation.

With further reference to FIG. 5, as an implementation of the methods shown in the foregoing figures, this application provides an embodiment of an apparatus for generating information. The apparatus embodiment corresponds to the method embodiment shown in FIG. 2. The device can be specifically applied to various electronic devices.

As shown in FIG. 5, the apparatus 500 for generating information in this embodiment includes: an obtaining unit 501 and an input unit 502. The obtaining unit 501 is configured to obtain the target video; the input unit 502 is configured to input the target video into a pre-trained video recognition model to obtain a pair of recognition results corresponding to the target video, wherein the video recognition model is used to characterize the video and recognition Correspondence between result pairs. The recognition result pair includes a first recognition result and a second recognition result. The first recognition result is used to characterize the foreground included in the video, and the second recognition result is used to characterize the background included in the video.

In this embodiment, the obtaining unit 501 of the apparatus 500 for generating information may obtain a target video by using a wired connection method or a wireless connection method. The target video may be a video to be identified.

It should be noted that the obtaining unit 501 may obtain a target video sent by an electronic device (such as the terminal device shown in FIG. 1) that is communicatively connected to the target unit, and may also obtain a target video stored in advance locally.

In this embodiment, based on the target video obtained in the obtaining unit 501, the input unit 502 may input the target video into a pre-trained video recognition model to obtain a pair of recognition results corresponding to the target video. Among them, the video recognition model can be used to characterize the correspondence between the video and the recognition result pair. The recognition result pair includes a first recognition result and a second recognition result. The first recognition result may be used to characterize the foreground included in the video. The second recognition result can be used to characterize the background included in the video. The recognition results (the first recognition result and the second recognition result) in the recognition result pair may include, but are not limited to, at least one of the following: text, numbers, symbols, images, videos.

Here, it can be understood that the foreground included in the video usually refers to the shooting content (such as people, animals, behaviors, etc.) corresponding to the video, and the background included in the video usually refers to the shooting scene (such as Sky, stadium, forest, etc.).

In some optional implementations of this embodiment, the video recognition model may include a first video recognition sub-model, a second video recognition sub-model, and a feature extraction network; and the input unit 502 may include: a first input module (in the figure) (Not shown), configured to extract a target video into a feature extraction network to obtain video features of the target video; a second input module (not shown in the figure), configured to input the obtained video features into a first video recognition, respectively The sub-model and the second video recognition sub-model obtain a recognition result pair corresponding to the target video and including the first recognition result and the second recognition result.

In some optional implementations of this embodiment, the video recognition model may be obtained by training as follows: obtaining a training sample set, where the training sample includes a sample video and a pair of sample recognition results pre-labeled for the sample video; and training samples The sample video of the concentrated training samples is used as input, and the pair of sample recognition results corresponding to the input sample video is used as the desired output, and a video recognition model is obtained by training using a machine learning method.

In some optional implementations of this embodiment, a sample video of training samples in a training sample set is used as an input, and a pair of sample recognition results corresponding to the input sample video is used as an output. Video recognition is obtained by training using a machine learning method. The model includes: dividing a training sample set into a preset number of training sample groups; selecting a training sample group as a candidate training sample group from the preset number of training sample groups, and based on the candidate training sample group and a predetermined initial model, The following training steps are performed: for the candidate training sample group, the sample video of the training sample is used as an input, and the sample recognition result pair corresponding to the input sample video is used as an output, and the initial model is trained using the machine learning method to obtain the initial video recognition Model; determine whether the pre-set completion conditions used to indicate the completion of training are met; in response to determining that the completion conditions are met, generate a video recognition model based on the obtained initial video recognition model; in response to determining that the completion conditions are not met, never selected From the training sample group The training sample group is used as a new candidate training sample group, and the newly obtained initial video recognition model is used as a new initial model, and the training step is continued.

In some optional implementations of this embodiment, the completion condition may include, but is not limited to, at least one of the following: a preset number of training sample groups does not include an unselected training sample group; The loss value of the actual recognition result pair obtained by inputting the sample video of the training sample into the initial model with respect to the sample recognition result pair corresponding to the input sample video is less than a preset loss threshold.

The apparatus 500 provided by the foregoing embodiment of the present application obtains a target video through the obtaining unit 501, and then the input unit 502 inputs the target video into a pre-trained video recognition model to obtain a pair of recognition result corresponding to the target video, where the video recognition model is used for Characterizing the correspondence between the video and the recognition result pair, the recognition result pair includes a first recognition result and a second recognition result, the first recognition result is used to characterize the foreground included in the video, and the second recognition result is used to characterize the background included in the video, Therefore, the pre-trained video recognition model can be used to identify the foreground and background of the target video at the same time, which improves the diversity of information generation.

Reference is now made to FIG. 6, which illustrates a schematic structural diagram of a computer system 600 suitable for implementing an electronic device according to an embodiment of the present application. The electronic device shown in FIG. 6 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present application.

As shown in FIG. 6, the computer system 600 includes a central processing unit (CPU) 601, which can be loaded into a random access memory (RAM) 603 from a program stored in a read-only memory (ROM) 602 or from a storage portion 608. Instead, perform various appropriate actions and processes. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

The following components are connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage portion 608 including a hard disk and the like And a communication section 609 including a network interface card such as a LAN card, a modem, and the like. The communication section 609 performs communication processing via a network such as the Internet. The driver 610 is also connected to the I / O interface 605 as necessary. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing a method shown in a flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programming read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal that is included in baseband or propagated as part of a carrier wave, and which carries computer-readable program code. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. . The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, which contains one or more functions to implement a specified logical function Executable instructions. It should also be noted that in some alternative implementations, the functions labeled in the blocks may also occur in a different order than those labeled in the drawings. For example, two blocks represented one after the other may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified function or operation , Or it can be implemented with a combination of dedicated hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software or hardware. The described unit may also be provided in a processor, for example, it may be described as: a processor includes an obtaining unit and an input unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the obtaining unit may also be described as a “unit obtaining a target video”.

As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device in. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: obtains a target video; inputs the target video into a pre-trained video recognition model to obtain the target video Corresponding recognition result pair, where the video recognition model is used to characterize the correspondence between the video and the recognition result pair, the recognition result pair includes the first recognition result and the second recognition result, and the first recognition result is used to represent the foreground included in the video The second recognition result is used to characterize the background included in the video.

The above description is only a preferred embodiment of the present application and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solution of the specific combination of the above technical features, but should also cover the above technical features or Other technical solutions formed by arbitrarily combining their equivalent features. For example, a technical solution formed by replacing the above features with technical features disclosed in the present application (but not limited to) having similar functions.

Claims (12)

A method for generating information, including: Get the target video; Inputting the target video into a pre-trained video recognition model to obtain a recognition result pair corresponding to the target video, wherein the video recognition model is used to characterize the correspondence between the video and the recognition result pair, and the recognition result pair includes a first A recognition result and a second recognition result. The first recognition result is used to characterize the foreground included in the video, and the second recognition result is used to characterize the background included in the video. The method of claim 1, wherein the video recognition model comprises a first video recognition sub-model, a second video recognition sub-model, and a feature extraction network; and The step of inputting the target video into a pre-trained video recognition model and obtaining a pair of recognition results corresponding to the target video includes: Inputting the target video into the feature extraction network to obtain video features of the target video; The obtained video features are respectively input into the first video recognition sub-model and the second video recognition sub-model, and a pair of recognition results including the first recognition result and the second recognition result corresponding to the target video is obtained. The method according to claim 1 or 2, wherein the video recognition model is obtained by training as follows: Obtaining a training sample set, where the training sample includes a sample video and a pair of sample recognition results pre-labeled for the sample video; Taking the sample video of the training samples in the training sample set as an input, using the pair of sample recognition results corresponding to the input sample video as the desired output, and using a machine learning method to train to obtain a video recognition model. The method according to claim 3, wherein the sample video of the training samples in the training sample set is used as an input, and the sample recognition result pair corresponding to the input sample video is used as an output, and is obtained by training using a machine learning method. Video recognition models, including: Dividing the training sample set into a preset number of training sample groups; Select a training sample group as a candidate training sample group from a preset number of training sample groups, and perform the following training steps based on the candidate training sample group and a predetermined initial model: For the candidate training sample group, use the sample video of the training sample as Input, taking the pair of sample recognition results corresponding to the input sample video as the output, and using the machine learning method to train the initial model to obtain the initial video recognition model; determine whether the pre-set completion conditions that indicate the completion of training are met; response When it is determined that the completion conditions are met, a video recognition model is generated based on the obtained initial video recognition model; In response to determining that the completion conditions are not satisfied, the training sample group is selected from the selected training sample group as a new candidate training sample group, and the newly obtained initial video recognition model is used as the new initial model, and the training is continuously performed. step. The method according to claim 4, wherein the completion condition includes but is not limited to at least one of the following: the preset number of training sample groups does not include unselected training sample groups; the training in the candidate training sample group The loss value of the actual recognition result pair obtained by inputting the sample video of the sample into the initial model relative to the sample recognition result pair corresponding to the input sample video is less than a preset loss threshold. An apparatus for generating information includes: An acquisition unit configured to acquire a target video; The input unit is configured to input the target video into a pre-trained video recognition model to obtain a recognition result pair corresponding to the target video, wherein the video recognition model is used to characterize a correspondence relationship between the video and the recognition result pair, The recognition result pair includes a first recognition result and a second recognition result. The first recognition result is used to characterize the foreground included in the video, and the second recognition result is used to characterize the background included in the video. The apparatus according to claim 1, wherein the video recognition model comprises a first video recognition sub-model, a second video recognition sub-model, and a feature extraction network; and The input unit includes: A first input module configured to input the target video into the feature extraction network to obtain video features of the target video; A second input module is configured to input the obtained video features into the first video recognition sub-model and the second video recognition sub-model, respectively, and obtain the first video recognition result and the first video recognition sub-model corresponding to the target video. The recognition result pairs of the two recognition results. The device according to claim 6 or 7, wherein the video recognition model is obtained by training as follows: Obtaining a training sample set, where the training sample includes a sample video and a pair of sample recognition results pre-labeled for the sample video; Taking the sample video of the training samples in the training sample set as an input, using the pair of sample recognition results corresponding to the input sample video as the desired output, and using a machine learning method to train to obtain a video recognition model. The device according to claim 8, wherein the sample video of the training samples in the training sample set is used as an input, and the sample recognition result pair corresponding to the input sample video is used as an output, and is obtained by training using a machine learning method. Video recognition models, including: Dividing the training sample set into a preset number of training sample groups; Select a training sample group as a candidate training sample group from a preset number of training sample groups, and perform the following training steps based on the candidate training sample group and a predetermined initial model: For the candidate training sample group, use the sample video of the training sample as Input, taking the pair of sample recognition results corresponding to the input sample video as the output, and using the machine learning method to train the initial model to obtain the initial video recognition model; determine whether the pre-set completion conditions that indicate the completion of training are met; response When it is determined that the completion conditions are met, a video recognition model is generated based on the obtained initial video recognition model; In response to determining that the completion conditions are not satisfied, the training sample group is selected from the selected training sample group as a new candidate training sample group, and the newly obtained initial video recognition model is used as the new initial model, and the training is continuously performed. step. The device according to claim 9, wherein the completion condition includes but is not limited to at least one of the following: a preset number of training sample groups does not include an unselected training sample group; and the training in the candidate training sample group The loss value of the actual recognition result pair obtained by inputting the sample video of the sample into the initial model relative to the sample recognition result pair corresponding to the input sample video is less than a preset loss threshold. An electronic device includes: One or more processors; A storage device on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-5. A computer-readable medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method of any one of claims 1-5.

Images