CN112115299B - Video search method, device, recommendation method, electronic device and storage medium - Google Patents

Abstract

The embodiments of the present application disclose a video search method, device, recommendation method, electronic device and storage medium, which relate to computer vision and video analysis technology, including: obtaining a search request carrying search information, wherein the search request is used to request a search for a target video corresponding to the search information, if the search information includes text information, determining a first similarity between the text information and pre-set label information of each video, wherein the label information includes a text label and a video frame label, selecting and outputting a target video from each video according to the first similarity, and combining the video frame label to describe the video in more detail and accurately, thereby improving the accuracy of the similarity calculation, thereby improving the accuracy and reliability of the video search, and improving the user's search experience.

Description

Video search method, device, recommendation method, electronic device and storage medium

Technical Field

The present application relates to computer vision, image technology and video analysis technology in artificial intelligence and computing technology, and in particular to a video search method, device, recommendation method, electronic device and storage medium.

Background Art

With the development of the Internet, video applications are becoming more and more popular, especially short video applications, which bring speed and convenience to people. However, with the increase in the amount of videos, improving the accuracy of search has become an urgent problem to be solved.

In the prior art, when a user uploads a video to a server of a video application, the user can annotate the uploaded video, or the staff of the video application can annotate the video uploaded by the user and generate text tags corresponding to the video. When the server receives a search request sent by the user, it can search from a large number of videos based on the text tags and provide feedback to the user.

Summary of the invention

Provided are a video search method, device, video recommendation method, electronic device, and storage medium for improving video accuracy.

According to a first aspect, a video search method is provided, which obtains a search request carrying search information, wherein the search request is used to request a search for a target video corresponding to the search information;

If the search information includes text information, determining a first similarity between the text information and preset tag information of each video, wherein the tag information includes a text tag and a video frame tag;

The target video is selected from each of the videos according to the first similarity and outputted.

In this embodiment, by calculating the similarity between text information and tag information including text tags and video frame tags, the accuracy and reliability of search results can be improved.

According to a second aspect, an embodiment of the present application provides a video search device, including:

An acquisition module, used to acquire a search request carrying search information, wherein the search request is used to request a search for a target video corresponding to the search information;

A first determination module, configured to determine a first similarity between the text information and preset tag information of each video if the search information includes text information, wherein the tag information includes a text tag and a video frame tag;

A selection module, configured to select the target video from the videos according to the first similarity;

An output module is used to output the target video.

According to a third aspect, an embodiment of the present application provides an electronic device, including:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in any of the above embodiments.

According to a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

According to a fifth aspect, an embodiment of the present application provides a video recommendation method, including:

Get the user's video access history;

Determine text information corresponding to the historical record;

Determine a third similarity between the text information corresponding to the historical record and the preset tag information of each video, wherein the tag information includes a text tag and a video frame tag;

A video is selected from each of the videos according to the third similarity and recommended to the user.

According to the sixth aspect of the present application, a computer program product is provided, the program product comprising: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.

The present application provides a video search method, device, recommendation method, electronic device and storage medium, including: obtaining a search request carrying search information, wherein the search request is used to request a search for a target video corresponding to the search information, if the search information includes text information, determining a first similarity between the text information and pre-set tag information of each video, wherein the tag information includes a text tag and a video frame tag, selecting and outputting a target video from each video according to the first similarity, in this embodiment, the tag information includes two dimensions of content, one dimension of content is a text tag, and the other dimension of content is a video frame tag, compared with the solution of determining the target video based only on the text tag in the related art, the embodiment of the present application combines the video frame tag to describe the video in more detail and accurately, thereby improving the accuracy of similarity calculation, and further improving the accuracy and reliability of video search, especially when there is a large difference between the text tag and the content of the video, using the tag information including the video frame tag of the present embodiment to perform video search, can accurately determine the target video corresponding to the user's search intention, improve the accuracy and reliability of video search, and improve the user's search experience.

It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

FIG1 is a schematic diagram of an application scenario of a video search method according to an embodiment of the present application;

FIG2 is a schematic diagram of a video search interface according to an embodiment of the present application;

FIG3 is a schematic diagram of a flow chart of a video search method according to an embodiment of the present application;

FIG4 is a schematic diagram of a flow chart of a video search method according to another embodiment of the present application;

FIG5 is a schematic diagram of a video frame image according to an embodiment of the present application;

FIG6 is a schematic flow chart of a video search method according to another embodiment of the present application;

FIG7 is a schematic diagram of a video search device according to an embodiment of the present application;

FIG8 is a schematic diagram of a video search device according to another embodiment of the present application;

FIG9 is a block diagram of an electronic device according to an embodiment of the present application;

FIG. 10 is a flow chart of a video recommendation method according to an embodiment of the present application.

DETAILED DESCRIPTION

The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the embodiments of the present application. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

Please refer to FIG. 1 , which is a schematic diagram of an application scenario of a video search method according to an embodiment of the present application.

In the application scenario shown in FIG1 , an application with a video playback function is installed on the user device 100, which may be Douyin, Kuaishou, Weishi, etc. The user sends a search request to the server 200 through the above application installed in the user device 100, and the search request is used to request to search for a target video corresponding to the search information.

For a schematic diagram of the search interface of the user equipment 100 , please refer to FIG. 2 .

As shown in Figure 2, the user can enter relevant information about the video (which can be called the target video) that the user expects to search in the input box (such as the "Enter search content" as shown in Figure 2), that is, the information can be used to characterize the user's search intention, such as the time when the target video was uploaded to the server 200, the uploader of the target video, the content of the target video, and so on.

When the user clicks “Search” as shown in FIG. 2 , the user device 100 is triggered to send a search request to the server.

After receiving the search request, the server 200 may obtain a target video corresponding to the search information from the database based on the search request, and send the target video to the user device 100, which then displays the target video.

Among them, the user device 100 is a device that can install various applications and display objects provided in the installed applications. The electronic device can be mobile or fixed, for example, a mobile phone, a tablet computer, various wearable devices, vehicle-mounted devices, a personal digital assistant (PDA), a point of sales (POS), a device that can recommend short videos, or other electronic devices that can achieve the above functions.

The server 100 may be any device capable of providing Internet services.

The user equipment 100 and the server 200 are communicatively connected via a network, and the network may be a local area network or a wide area network.

It is worth noting that the above embodiment is only used to exemplarily illustrate the application scenarios to which the video search method of this embodiment can be applied, and cannot be understood as a limitation on the application scenarios.

Based on the description of the above application scenario, it can be known that the user device can generate a search request carrying search information based on the relevant information about the target video input by the user, and transmit the search request to the server through the communication network. In the related art, the search information is generally text information, such as the text information input by the user based on the input box in Figure 2, etc. The server matches the text information with the pre-set text tag and feeds back the target video based on the matching result.

However, there may be deviations between text information and video content, resulting in low accuracy of matching results and reducing the user's search experience.

The inventor of the present application has obtained the inventive concept of the present application through creative work: matching text information with tag information including text tags and video frame tags, and determining the target video based on the matching result.

The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

According to one aspect of an embodiment of the present application, an embodiment of the present application provides a video search method, which is applied to artificial intelligence and computer technology, specifically to computer vision, image technology and video analysis technology, to achieve reliability and accuracy of video search.

Please refer to FIG. 3 , which is a flow chart of a video search method according to an embodiment of the present application.

As shown in FIG3 , the method includes:

S101: Obtain a search request carrying search information, wherein the search request is used to request a search for a target video corresponding to the search information.

Among them, the executor of the embodiment of the present application can be a video search device, and the video search device can specifically be a server (including a cloud server and a local server). The description of the server can be found in the above example and will not be repeated here.

In conjunction with the application scenario shown in FIG. 1 , a search request may be generated by a user device based on search information input by a user (ie, the user's search intent).

S102: If the search information includes text information, determine a first similarity between the text information and preset tag information of each video, wherein the tag information includes a text tag and a video frame tag.

Among them, text information can be understood as information related to text or voice. For example, the text information can be the user entering "food" in the input box as shown in Figure 2; for another example, a voice component can be set on the user device, and the user can input voice information through the voice component, then the text information can be the text information generated by the user device analyzing the voice information.

The tag information can be understood as the description information of each video, that is, based on the tag information, the relevant content of each video can be determined. For example, if the tag information is food, the corresponding video should be a food-related video, such as a cooking-related video.

The “first” in the first similarity is used to distinguish it from the second similarity in the following text, and should not be understood as a limitation on the content of the similarity.

In this embodiment, the first similarity can be understood as the similarity between the text information and the tag information.

It is worth noting that, in this embodiment, the tag information may include two dimensions of content, one dimension of content is a text tag, and the other dimension of content is a video frame tag.

The description of the text label is as follows:

In one possible implementation, when a user uploads a video to a server, the user can annotate the name, general content, and type (such as story, comedy, and music, etc.) of the video. The server can generate a text label for the video based on the user's annotation information.

In another possible implementation, the server can set partial tags (such as the type of video, etc.). When a user uploads a video, he or she can select the partial tags set by the server and annotate the video. The server generates text tags based on the partial tags selected by the user and the annotated partial tags.

That is, the tag generated by the server for describing the relevant information of the video based on the text description of the uploaded video by the user and/or the text description preset by the server is called a text tag.

The description of the video frame label is as follows:

The video includes multiple frames of images, and the video frame label can be used to represent the labels of the multiple frames of images, that is, the video frame can understand the label determined based on the multiple frames of images in the video and used to describe the video.

It is worth noting that, in the present embodiment, since the tag information includes text tags and video frame tags, the tag information is relatively richer and can describe the video in more detail and clarity. When similarity calculation is performed based on the tag information and the target video is determined, the accuracy and reliability of the similarity calculation can be improved, thereby achieving improved accuracy of video search.

S103: Select and output a target video from each video according to the first similarity.

In a possible technical solution, a video with a relatively high similarity can be selected from each video as a target video based on the first similarity. For example, a similarity greater than a preset threshold value in the first similarity can be determined, and the video corresponding to the similarity greater than the preset threshold value can be determined as the target video. For another example, the first n similarities with the largest similarity in the first similarity can be selected, and the video corresponding to the selected similarity can be determined as the target video.

Based on the above analysis, it can be seen that an embodiment of the present application provides a video search method, which includes: obtaining a search request carrying search information, wherein the search request is used to request a search for a target video corresponding to the search information, if the search information includes text information, then determining a first similarity between the text information and the pre-set tag information of each video, wherein the tag information includes a text tag and a video frame tag, and selecting and outputting a target video from each video according to the first similarity. In this embodiment, the tag information includes two dimensions of content, one dimension of content is a text tag, and the other dimension of content is a video frame tag. Compared with the solution of determining the target video based only on text tags in the related art, the embodiment of the present application combines the video frame tag to describe the video in more detail and accurately, thereby improving the accuracy of similarity calculation, thereby improving the accuracy and reliability of video search, and achieving the technical effect of, especially when there is a large difference between the text tag and the content of the video, the tag information including the video frame tag of this embodiment is used to perform video search, which can accurately determine the target video corresponding to the user's search intention, improve the accuracy and reliability of video search, and improve the user's search experience.

Now, in combination with the principle of determining the video frame label, the video search method of the embodiment of the present application is described in more detail.

Please refer to FIG. 4 for details, which is a flowchart of a video search method according to another embodiment of the present application.

As shown in FIG4 , the method includes:

S201: Slice any video to obtain a slice set corresponding to any video.

It should be understood that the server can receive videos uploaded by various users, and each user can upload multiple videos. Therefore, multiple videos are stored in the server, and the server can slice any video it receives.

In some embodiments, the video stored in the server is obtained by screening based on the video information and audio information of the video.

For example, when the server receives a video uploaded by a user, the video may be reviewed from the dimensions of video information and audio information of the video, and if the review is passed, the video may be stored.

It is worth noting that by reviewing videos from two dimensions, video information and audio information, it is possible to avoid the problem of low review accuracy caused by reviewing videos from a single dimension of video information in related technologies, avoid missed reviews and misreviews, and thus achieve the technical effect of reliability and accuracy of video review.

For example, a video classification algorithm (learnable pooling with ContextGating, LPCG) may be set in the server. When the server receives a video uploaded by a user, the video classification algorithm may be called to perform classification calculations on the video, thereby implementing an audit of the video.

Based on the above analysis, it can be known that a slice can be understood as cutting a video into multiple frames of images. Accordingly, a slice set can be understood as a set including multiple frames of images.

In some embodiments, S201 may include:

S2011: Slice any video using time as a slicing unit to obtain an image corresponding to any video.

That is to say, the video may be sliced based on time, such as slicing the video in units of 0.1 seconds to obtain images corresponding to the video.

It is worth noting that, generally speaking, if the video is a color video, in a possible implementation, the video may be grayscaled before slicing.

By gray-scaling the video, color interference can be avoided and the reliability and accuracy of slicing and subsequent aggregation can be improved.

S2012: Slice the image using the object as a slice unit to obtain a slice set.

The object may be a living person or animal, or a building, etc., which is not limited in this embodiment.

That is to say, in this embodiment, the video can be sliced based on time first, and on this basis, the video can be sliced based on objects. By slicing the video based on both time and object levels, it is equivalent to first slicing the video from a large range, and then slicing the video again from a small focus point. This can improve the technical effect of the accuracy and reliability of the slicing, so that the video frame labels generated based on the slice set have higher reliability.

S202: Perform clustering processing on the slice set to obtain a target key frame image of any video.

In this embodiment, by processing the slice set in a clustering manner, it is possible to reduce the number of images with high similarity in the slice set being divided into the same category, avoid repeated labeling of the same image, and avoid being affected by image noise, thereby reducing the accuracy of video frame labels, that is, it is possible to improve the determination of video frame labels with a high degree of fit for describing the video.

In some embodiments, S202 may include:

S2021: Clustering the slice set based on the object as the category unit.

That is to say, during clustering, each image in the slice set can be divided into categories based on objects, such as people, animals, and buildings.

In a possible technical solution, time-discontinuous images in a slice set are determined to be different categories, thereby avoiding omission of video frame labels and improving the comprehensiveness and completeness of video frame labels.

S2022: For any category in the clustering results, an image with the largest image information entropy in any category is determined as a candidate key frame image.

Among them, image information entropy can be used to characterize the amount of information in an image. Relatively speaking, the larger the image information entropy is, the greater the amount of information in the image is. Conversely, the smaller the image information entropy is, the less the amount of information in the image is.

In this embodiment, the image with the largest image information entropy is determined as the candidate key frame image. Since the image with the largest image information entropy is the image with the largest amount of image information, the candidate key frame image is a representative image in the category, so that the video frame label has a strong representativeness for the video.

S2023: Perform redundancy processing on the candidate key frame images to obtain a target key frame image.

In order to reduce the amount of annotation, after obtaining the candidate key frame images, the candidate key frame images may be subjected to redundancy processing.

In some embodiments, S2023 may include:

S20231: For any two temporally adjacent candidate key frame images, determine the edge histograms corresponding to any two temporally adjacent candidate key frame images.

Among them, the edge histogram is used to reflect the edge and texture features of the image. The method for determining the edge histogram of a candidate key frame image may include: performing edge operator operation on the candidate key frame image; calculating the edge direction of each pixel of the candidate key frame image; quantizing the edge direction to obtain the edge direction value; performing histogram statistics and normalization processing on the edge direction value to obtain the edge histogram of the candidate key frame image.

S20232: Determine the difference information between the corresponding edge histograms.

S20233: Perform redundancy processing on the candidate key frame images based on the difference information to obtain a target key frame image.

The difference information may be a difference value. For example, if the candidate key frame image A and the candidate key frame image B are adjacent in time, the difference information may be a difference between a normalized value of an edge histogram of the key frame image A and a normalized value of an edge histogram of the candidate key frame image B.

Correspondingly, the difference can be compared with a preset threshold. If the difference is greater than or equal to the threshold, it means that the difference between candidate key frame image A and candidate key frame image B is large. In order to ensure the integrity and comprehensiveness of the video frame label, both candidate key frame image A and key image frame B are determined as target key frame images; if the difference is less than the threshold, it means that the difference between candidate key frame image A and candidate key frame image B is small. In order to reduce the amount of annotation and save storage space on the server, candidate key frame image A or candidate key frame image B can be determined as the target key frame image, or, the image with a larger image information entropy between candidate key frame image A and candidate key frame image B can be determined as the target key frame image.

S203: Generate a video frame label according to the target key frame image.

Based on the above analysis, it can be seen that since the target key frame image is an image from which redundant images are eliminated, and on this basis, images with a large amount of image information are retained, the video frame labels generated according to the target key frame image have high integrity, accuracy and reliability.

In some embodiments, S203 may include:

S2031: Generate description information for describing the target key frame image.

The description information may be used to represent information describing the content of the key frame image.

In some embodiments, S2031 may include:

S20311: Determine a phrase used to describe an object in a target key frame image.

S20312: Connect the phrases based on pre-set connecting words to obtain description information.

For example, the server may perform semantic analysis and syntactic analysis on the target key frame image to obtain description information.

Specifically, semantic analysis and syntactic analysis may be performed on the target key frame image based on a preset network model, such as a neural network model for image captioning (NCPIC), so as to obtain description information.

Now, the generation of description information is described as follows in conjunction with FIG5:

The server can identify the object of the target key frame image and form a phrase for describing the object: puppy, ball and grass. The server can be pre-set with a corpus for storing connectives. When a phrase for describing an object is determined, the server can select each connective from the corpus so that each phrase and each connective can form a reasonable sentence. If there are multiple sentences, each sentence can be semantically compared with the target key frame image to obtain the final sentence (which may be one or more), such as: the puppy is playing with a ball on the grass.

It is worth noting that, in this embodiment, by first forming a phrase and then combining the phrase with a conjunction to obtain description information, the description information can appropriately describe the target key frame image, thereby achieving the technical effect of reliability and accuracy of video frame labels.

S2032: Generate a video frame label according to the description information.

In this embodiment, the description information is used to describe the target key video frame image. Therefore, the video frame label generated according to the description information can describe the video more comprehensively, completely and accurately, thereby improving the reliability and accuracy of the similarity calculation, and further achieving the technical effect of reliability and accuracy of video search.

In this step, the video frame label can be directly generated according to the description information. For example, in combination with the above embodiment, "a puppy playing with a ball on the grass" can be determined as a video frame label. The description information can also be expanded and synonymous substitution can be performed to enrich the video frame label. For example, "a puppy playing with a ball on the grass" can be expanded to "a dog playing with a ball on the grass" and "a puppy playing with a ball on the lawn".

In some embodiments, the target key frame image includes multiple frame images, and the description information includes description information corresponding to each of the multiple frame images, then S2032 includes:

S20321: Determine the number of occurrences of the description information corresponding to each of the multiple image frames.

S20322: Selecting a video frame label from the description information corresponding to each of the plurality of frames of images based on pre-set selection parameters and the number of occurrences.

The selection parameter may be a percentage or a threshold, and the percentage and the selection threshold may be set by the server based on demand, historical records, and experiments, etc., and this embodiment does not limit this.

For example, the description information with the top 5% of occurrence times may be selected to determine the video frame label; or the description information with the occurrence times greater than a preset threshold (ie, the selection threshold) may be selected to determine the video frame label.

In some embodiments, the number of occurrences of each description information may be calculated and ranked based on a label ranking (EmbedRank) algorithm, and the video frame label may be determined based on the percentage and the number of occurrences.

In this embodiment, by selecting parameters and the number of occurrences to select video frame labels, it is possible to reduce the storage space occupied by the video frame labels, reduce the computing resources during similarity calculation, and other technical effects.

S204: Obtain a search request carrying the search information, wherein the search request is used to request a search for a target video corresponding to the search information.

The description of S204 can be found in S101 and will not be repeated here.

S205: If the search information includes text information, determine a first similarity between the text information and preset tag information of each video, wherein the tag information includes a text tag and a video frame tag.

Among them, the description of S205 can be found in S102 and will not be repeated here.

In some embodiments, the server pre-sets a database for storing tag information, and determines a first similarity between the text information and the tag information in the database.

It is worth noting that, by pre-building a database for storing tag information, each tag information can be stored uniformly, thereby achieving the technical effect of convenience in calculating similarity and improving the efficiency of determining the first similarity.

Furthermore, in this embodiment, the text tag may be expanded by the server based on the original text tag (such as text information provided by the user when uploading the video, or text tags annotated by the server (or staff) based on the video uploaded by the user).

For example, the server can perform semantic analysis on the original text tags to obtain similar tags to supplement and improve the original text tags, and can sort the supplemented original text tags (such as based on the tag sorting (EmbedRank) algorithm), and filter the sorted original text tags through the Maximum Marginal Relevance (MMR) model to obtain text tags.

S206: Select and output a target video from each video according to the first similarity.

The description of S206 can be found in S103 and will not be repeated here.

It is worth noting that, in order to support diversified video searches, this embodiment can also support image search for videos, which is now exemplarily described in conjunction with FIG. 6 .

FIG6 is a flow chart of a video search method according to another embodiment of the present application.

As shown in FIG6 , the method includes:

S301: Obtain a search request carrying search information, wherein the search request is used to request a search for a target video corresponding to the search information.

The description of S301 can be found in S101 and will not be repeated here.

S302: Determine whether the search information includes text information and/or pictures. If the search information includes text information, execute S303 to S304; if the search information includes pictures, execute S305 to S306; if the search information includes text information and pictures, execute S307 to S309.

S303: Determine a first similarity between the text information and preset tag information of each video, wherein the tag information includes a text tag and a video frame tag.

The description of S303 can be found in S102 and will not be repeated here.

S304: Select and output a target video from each video according to the first similarity.

The description of S304 can be found in S103 and will not be repeated here.

In some embodiments, the target video may be screened again based on the user's historical search history, historical viewing history, and historical comment information, and the screened target video may be output.

S305: Determine a second similarity between the picture and the images of each video.

The image in this step may be a target key frame image of each video.

S306: Select and output a target video from each video according to the second similarity.

The principle of determining the target video according to the second similarity can refer to the principle of determining the target video according to the first similarity in the above embodiment, which will not be described in detail here.

S307: Determine a first similarity between the text information and preset tag information of each video, wherein the tag information includes a text tag and a video frame tag.

The description of S303 can be found in S102 and will not be repeated here.

S308: Determine a second similarity between the picture and the images of each video.

S309: Determine and output a target video according to the intersection of the first similarity and the second similarity.

That is to say, the server can respectively determine the videos corresponding to the first similarity (hereinafter referred to as the first video set) and the videos corresponding to the second similarity (hereinafter referred to as the second video set), and then can determine the videos included in the first video set and the videos included in the second video set, and determine this part of the video as the target video.

Similarly, some videos may be selected from the intersection to be determined as target videos. The implementation principle thereof may be referred to the above embodiment and will not be described in detail here.

In some other embodiments, weight coefficients may be respectively allocated in advance to the first similarity and the second similarity, and the target video may be determined based on the weight coefficients.

In this embodiment, the server can support both text-based video search and image-based video search, which can improve the technical effect of flexibility and diversity of video search, and by determining the target video in combination with the first similarity and the second similarity, the technical effect of accuracy of the target video can be improved.

According to another aspect of the embodiments of the present application, the embodiments of the present application also provide a video search device for executing the video search method described in any of the above embodiments, such as executing the method shown in any of the embodiments in Figures 3, 4 and 6.

Please refer to FIG. 7 , which is a schematic diagram of a video search device according to an embodiment of the present application.

As shown in FIG7 , the device comprises:

An acquisition module 11 is used to acquire a search request carrying search information, wherein the search request is used to request a search for a target video corresponding to the search information;

A first determination module 12, configured to determine a first similarity between the text information and preset tag information of each video if the search information includes text information, wherein the tag information includes a text tag and a video frame tag;

A selection module 13, configured to select the target video from the videos according to the first similarity;

The output module 14 is used to output the target video.

In conjunction with FIG8 , it can be seen that in some embodiments, the method further includes:

A slicing module 15 is used to perform slicing processing on any video to obtain a slice set corresponding to any video;

A clustering module 16, configured to perform clustering processing on the slice set to obtain a target key frame image of any video;

The generating module 17 is used to generate the video frame label according to the target key frame image.

In some embodiments, the slicing module 15 is used to slice any of the videos using time as a slicing unit to obtain an image corresponding to any of the videos, and to slice the image using objects as a slicing unit to obtain the slice set.

In some embodiments, the clustering module 16 is used to perform clustering processing on the slice set based on the object as a category unit, and for any category in the clustering results, an image with the largest image information entropy in any category is determined as a candidate key frame image, and redundancy processing is performed on the candidate key frame image to obtain the target key frame image.

In some embodiments, the clustering module 16 is used to determine, for any two temporally adjacent candidate key frame images, the edge histograms corresponding to each of the two temporally adjacent candidate key frame images, determine the difference information between the corresponding edge histograms, perform redundant processing on the candidate key frame images based on the difference information, and obtain the target key frame image.

In some embodiments, the generating module 17 is used to generate description information for describing the target key frame image, and generate the video frame label according to the description information.

In some embodiments, the target key frame image includes multiple frame images, and the description information includes description information corresponding to each of the multiple frame images. The generation module 17 is used to determine the number of occurrences of the description information corresponding to each of the multiple frame images, and select the video frame label from the description information corresponding to each of the multiple frame images based on pre-set selection parameters and the number of occurrences.

In some embodiments, the generating module 17 is used to determine phrases for describing the object in the target key frame image, and connect the phrases based on preset connecting words to obtain the description information.

As can be seen from FIG8 , in some embodiments, if the search information also includes pictures, it also includes:

A second determination module 18, configured to determine a second similarity between the picture and the images of each of the videos;

Furthermore, the selection module 13 is used to determine the intersection of the first similarity and the second similarity as the target video.

In some embodiments, each of the videos is obtained by screening based on video information and audio information of each of the videos.

In conjunction with FIG8 , it can be seen that in some embodiments, the method further includes:

A processing module 19, configured to perform grayscale processing on any of the videos;

Furthermore, the slicing module is used to perform slicing processing on any video after grayscale processing to obtain the slicing set.

According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

According to an embodiment of the present application, the present application also provides a computer program product, the program product includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

As shown in Figure 9, it is a block diagram of an electronic device according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the embodiments of the present application described and/or required herein.

As shown in Figure 9, the electronic device includes: one or more processors 101, a memory 102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input/output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and/or multiple buses can be used together with multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). In Figure 9, a processor 101 is taken as an example.

The memory 102 is a non-transitory computer-readable storage medium provided in the embodiment of the present application. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the video search method provided in the embodiment of the present application. The non-transitory computer-readable storage medium of the embodiment of the present application stores computer instructions, which are used to enable a computer to perform the video search method provided in the embodiment of the present application.

The memory 102 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions/modules in the embodiments of the present application. The processor 101 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 102, that is, implements the video search method in the above method embodiment.

The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 102 may optionally include a memory remotely arranged relative to the processor 101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a Blockchain Service Network (BSN), a mobile communication network, and a combination thereof.

The electronic device may further include: an input device 103 and an output device 104. The processor 101, the memory 102, the input device 103 and the output device 104 may be connected via a bus or other means, and FIG9 takes the connection via a bus as an example.

The input device 103 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 104 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and/or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and/or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and/or data to a programmable processor.

To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a Block-chain-based Service Network (BSN), a wide area network (WAN), and the Internet.

A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

According to another aspect of the embodiment of the present application, the embodiment of the present application also provides a video recommendation method.

Please refer to FIG. 10 , which is a flowchart of a video recommendation method according to an embodiment of the present application.

As shown in FIG. 10 , the method includes:

S401: Obtain the user's video access history.

S402: Determine text information corresponding to the historical record.

Among them, the text information corresponding to the historical records can be understood from two dimensions. One dimension is from the historical records themselves, such as the text tags of the videos watched by the user, the authors of the videos watched by the user, the upload time of the videos watched by the user, and the comment information of the videos watched by the user (including the comment information of the user and other users), etc.; the other dimension is other information extended based on the historical records, such as relevant information of similar videos that the user is interested in based on the historical records.

S403: Determine a third similarity between text information corresponding to the historical record and preset tag information of each video, wherein the tag information includes a text tag and a video frame tag.

The description of the tag information can be found in the above embodiments and will not be repeated here.

That is to say, in this embodiment, the server can recommend videos to users based on historical records and tag information. Since the videos recommended by the server to users fully consider the content of two dimensions, namely, text tags and video frame tags, the accuracy and reliability of video recommendations can be improved, thereby improving the user's video viewing experience.

S404: Select a video from each video according to the third similarity and recommend it to the user.

It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of this application can be achieved, and this document is not limited here.

The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims (18)

1. A video search method, comprising:

acquiring a search request carrying search information, wherein the search request is used for requesting to search a target video corresponding to the search information;

if the search information comprises text information, determining a first similarity between the text information and preset label information of each video, wherein the label information comprises a text label and a video frame label;

selecting and outputting the target video from the videos according to the first similarity;

further comprises:

slicing any video to obtain a slice set corresponding to the any video;

Clustering the slice set to obtain a target key frame image of any video;

Generating description information for describing the target key frame image;

generating the video frame tag according to the description information;

The target key frame image comprises a plurality of frame images, the description information comprises description information corresponding to the plurality of frame images, and the generating the video frame tag according to the description information comprises the following steps:

determining the occurrence times of the descriptive information corresponding to the multi-frame images respectively;

selecting the video frame tag from the descriptive information corresponding to the multi-frame images based on preset selection parameters and the occurrence times;

the generating description information for describing the target key frame image comprises the following steps:

Determining a phrase for describing an object in the target key frame image;

and connecting the phrases based on preset connecting words to obtain the description information.

2. The method of claim 1, wherein slicing any video to obtain a set of slices corresponding to the any video comprises:

slicing any video by taking time as a slicing unit to obtain an image corresponding to any video;

and slicing the image by taking the object as a slice unit to obtain the slice set.

3. The method of claim 2, wherein clustering the set of slices comprises:

clustering the slice set by taking the object as a category unit;

for any one of the categories of the clustering result, determining the image with the maximum image information entropy in the any one of the categories as a candidate key frame image;

and carrying out redundancy processing on the candidate key frame images to obtain the target key frame images.

4. The method of claim 3, wherein redundantly processing the candidate key frame images to obtain the target key frame image comprises:

determining edge histograms corresponding to any two candidate key frame images adjacent to each other in time according to any two candidate key frame images adjacent to each other in time;

determining difference information between the respective corresponding edge histograms;

And carrying out redundancy processing on the candidate key frame images based on the difference information to obtain the target key frame images.

5. The method according to any one of claims 1 to 4, wherein if the search information further includes a picture, further comprising:

Determining a second similarity of the picture to an image of each of the videos;

And selecting and outputting the target video from the videos according to the first similarity, wherein the selecting and outputting the target video comprises: and determining and outputting the target video by the intersection of the first similarity and the second similarity.

6. The method according to any one of claims 1 to 4, wherein each of the videos is obtained by filtering based on video information and audio information of each of the videos.

7. The method of any of claims 2 to 4, further comprising: gray processing is carried out on any video;

And performing slice processing on any video, wherein obtaining a slice set corresponding to any video comprises: and slicing any video subjected to gray level processing to obtain the slice set.

8. A video search apparatus comprising:

The acquisition module is used for acquiring a search request carrying search information, wherein the search request is used for requesting to search a target video corresponding to the search information;

The first determining module is used for determining first similarity between the text information and preset label information of each video if the search information comprises the text information, wherein the label information comprises a text label and a video frame label;

The selecting module is used for selecting the target video from the videos according to the first similarity;

the output module is used for outputting the target video;

further comprises:

The slicing module is used for slicing any video to obtain a slicing set corresponding to any video;

the clustering module is used for carrying out clustering processing on the slice set to obtain a target key frame image of any video;

The generation module is used for generating the video frame tag according to the target key frame image;

the generation module is used for generating description information for describing the target key frame image and generating the video frame tag according to the description information;

the generation module is used for determining the occurrence times of the descriptive information corresponding to the multi-frame images, and selecting the video frame tag from the descriptive information corresponding to the multi-frame images based on preset selection parameters and the occurrence times;

the generation module is used for determining a phrase for describing the object in the target key frame image, and connecting the phrase based on preset connecting words to obtain the description information.

9. The apparatus of claim 8, wherein the slicing module is configured to slice the any video in units of time to obtain an image corresponding to the any video, and slice the image in units of slices to obtain the slice set.

10. The apparatus of claim 9, wherein the clustering module is configured to perform clustering on the slice set with the object as a category unit, determine, for any one of the categories of the clustering result, an image with a maximum image information entropy in the any one of the categories as a candidate key frame image, and perform redundancy processing on the candidate key frame image to obtain the target key frame image.

11. The apparatus of claim 10, wherein the clustering module is configured to determine, for any two temporally adjacent candidate key frame images, edge histograms corresponding to the any two temporally adjacent candidate key frame images, determine difference information between the respective corresponding edge histograms, and perform redundancy processing on the candidate key frame images based on the difference information to obtain the target key frame image.

12. The apparatus according to any one of claims 8 to 11, wherein if the search information further includes a picture, further comprising:

a second determining module, configured to determine a second similarity between the picture and an image of each video;

and the selecting module is used for determining the intersection of the first similarity and the second similarity as the target video.

13. The apparatus according to any one of claims 8 to 11, wherein each of the videos is obtained by filtering based on video information and audio information of each of the videos.

14. The apparatus of any of claims 8 to 11, further comprising:

the processing module is used for carrying out gray processing on any video;

And the slicing module is used for slicing any video after gray level processing to obtain the slicing set.

15. An electronic device, comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-7.

17. A video recommendation method, comprising:

Acquiring a history record of video access of a user;

determining text information corresponding to the history record;

determining a third similarity between text information corresponding to the history record and preset label information of each video, wherein the label information comprises a text label and a video frame label;

selecting videos from the videos according to the third similarity and recommending the videos to the user;

further comprises:

slicing any video to obtain a slice set corresponding to the any video;

Clustering the slice set to obtain a target key frame image of any video;

Generating description information for describing the target key frame image;

generating the video frame tag according to the description information;

The target key frame image comprises a plurality of frame images, the description information comprises description information corresponding to the plurality of frame images, and the generating the video frame tag according to the description information comprises the following steps:

determining the occurrence times of the descriptive information corresponding to the multi-frame images respectively;

selecting the video frame tag from the descriptive information corresponding to the multi-frame images based on preset selection parameters and the occurrence times;

the generating description information for describing the target key frame image comprises the following steps:

Determining a phrase for describing an object in the target key frame image;

and connecting the phrases based on preset connecting words to obtain the description information.

18. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-7.