CN107657269A - A kind of method and apparatus for being used to train picture purification model - Google Patents

Abstract

The object of the present invention is to provide a method and device for training a picture refinement model. Compared with the prior art, the present invention expands the size of the existing pictures, obtains the expanded pictures, clusters the expanded pictures, obtains corresponding clustering results, and selects a predetermined number of clustering results from at least one clustering result. The picture is presented to the user as a sample picture, and the positive and negative samples obtained by the user based on the relevant operations on the clustering results are obtained. According to the positive and negative samples selected by the user, the corresponding picture purification model is trained, and then the picture purification model can be used Purification of image quality has achieved low-cost acquisition of high-quality data; different from the previous tens of thousands or hundreds of thousands of manual annotations, users only need to spend a few minutes to complete the labeling task of small samples, start model training, and get used for A picture refinement model for mass picture quality purification, and then use the picture refinement model to dig out more high-quality pictures from massive picture data.

Description

A method and device for training an image refinement model

technical field

The invention relates to the technical field of image processing, in particular to a technique for training a picture purification model.

Background technique

The quality purification of image data is a critical step in obtaining training data. Especially in the field of deep learning, the vast majority of methods are data-driven, resulting in the quality of image data directly related to the performance of algorithm models. Therefore, obtaining high-quality training data is an extremely important step in algorithm research.

Currently, image data purification methods mainly include algorithm-based automatic mining methods and manual labeling methods. The method based on algorithmic mining is low-cost, but the effect cannot be guaranteed. Although the manual labeling method is of high quality, it is costly and takes a long time, especially for massive data, which often has tens of millions or even hundreds of millions of records. The manual labeling method cannot well meet the business needs of big data. .

Therefore, how to provide an efficient and accurate method for training a picture refinement model, so as to use the model to perform picture quality refinement, has become one of the problems to be solved urgently by those skilled in the art.

Contents of the invention

The object of the present invention is to provide a method and device for training a picture refinement model.

According to one aspect of the present invention, a method for training a picture refinement model is provided, wherein the method includes:

a. Expand the image size of the existing image to obtain the expanded image;

b. Clustering the expanded pictures to obtain corresponding clustering results;

c selecting a predetermined number of pictures from at least one clustering result as sample pictures and presenting them to the user;

d obtaining positive and negative samples obtained by the user based on related operations on the clustering results;

e According to the positive and negative samples selected by the user, train the corresponding image purification model.

Preferably, step a comprises:

According to the keyword input by the user, use a picture search engine to obtain a picture matching the keyword as the existing picture.

Preferably, step a comprises:

Obtain the picture uploaded by the user as the existing picture.

Preferably, step a comprises:

According to the keywords input by the user and the pictures selected from the existing pictures, the pictures are expanded by using the keywords and searching for pictures by pictures to obtain the pictures after expansion.

Preferably, the method also includes:

Obtain the target number of image expansion set by the user;

Wherein, step a includes:

According to the target quantity, image expansion is performed on the existing image to obtain an expanded image.

Preferably, step d also includes:

Re-clustering is performed on at least one clustering result, and positive and negative samples are selected from the clustering results obtained after re-clustering.

Preferably, the method also includes:

According to the image purification model, image quality is purified for a large-scale image collection.

According to another aspect of the present invention, there is also provided a device for training a picture refinement model, wherein the device includes:

The expansion device is used to expand the image size of the existing image to obtain the expanded image;

A clustering device, configured to cluster the expanded pictures to obtain corresponding clustering results;

Presenting means, configured to select a predetermined number of pictures from at least one clustering result as sample pictures and present them to the user;

A first acquiring device, configured to acquire positive and negative samples obtained by the user based on related operations on the clustering results;

The training device is used to train a corresponding image refinement model according to the positive and negative samples selected by the user.

Preferably, the expansion device is used for:

According to the keyword input by the user, use a picture search engine to obtain a picture matching the keyword as the existing picture.

Preferably, the expansion device is used for:

Obtain the picture uploaded by the user as the existing picture.

Preferably, the expansion device is used for:

According to the keywords input by the user and the pictures selected from the existing pictures, the pictures are expanded by using the keywords and searching for pictures by pictures to obtain the pictures after expansion.

Preferably, the device also includes:

The second obtaining means is used to obtain the target quantity of image expansion set by the user;

Wherein, the expansion device is used for:

According to the target quantity, image expansion is performed on the existing image to obtain an expanded image.

Preferably, the first acquiring device is also used for:

Re-clustering is performed on at least one clustering result, and positive and negative samples are selected from the clustering results obtained after re-clustering.

Preferably, the device also includes:

The purification device is used for purifying the image quality of a large-scale image collection according to the image purification model.

According to still another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium stores computer code, when the computer code is executed, the method described in any one of the above is executed .

According to still another aspect of the present invention, a computer program product is also provided. When the computer program product is executed by a computer device, the method described in any one of the above items is executed.

According to still another aspect of the present invention, a computer device is also provided, and the computer device includes:

one or more processors;

memory for storing one or more computer programs;

When the one or more computer programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

Compared with the prior art, the present invention expands the size of the existing pictures, obtains the expanded pictures, clusters the expanded pictures, obtains corresponding clustering results, and selects a predetermined clustering result from at least one clustering result. A large number of pictures are presented to the user as sample pictures, the positive and negative samples obtained by the user based on the related operations on the clustering results are obtained, and the corresponding picture purification model is trained according to the positive and negative samples selected by the user , and then you can use the image purification model to purify the image quality, and achieve low-cost acquisition of high-quality data; different from the previous tens of thousands or hundreds of thousands of manual labeling, users only need to spend a few minutes to complete the small sample After labeling tasks, model training can be started, and the image purification model used for mass image quality purification can be directly obtained. After that, the image purification model can be used to mine more high-quality images from the massive image data.

Description of drawings

Other characteristics, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

Fig. 1 shows a schematic diagram of a device for training a picture refinement model according to one aspect of the present invention;

Fig. 2 shows a schematic diagram for training a picture purification model according to a preferred embodiment of the present invention;

Fig. 3 shows a schematic diagram for training a picture purification model according to a preferred embodiment of the present invention;

Fig. 4 shows a schematic diagram for training a picture purification model according to a preferred embodiment of the present invention;

Fig. 5 shows a schematic diagram for training a picture purification model according to a preferred embodiment of the present invention;

Fig. 6 shows a schematic flowchart of a method for training a picture refinement model according to another aspect of the present invention.

The same or similar reference numerals in the drawings represent the same or similar components.

Detailed ways

Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations as sequential processing, many of the operations may be performed in parallel, concurrently, or simultaneously. In addition, the order of operations can be rearranged. The process may be terminated when its operations are complete, but may also have additional steps not included in the figure. The processing may correspond to a method, function, procedure, subroutine, subroutine, or the like.

The term "computer equipment" in this context, also referred to as "computer", refers to an intelligent electronic device that can perform predetermined processing procedures such as numerical calculations and/or logic calculations by running predetermined programs or instructions, which may include a processor and The memory is realized by the processor executing the pre-stored surviving instructions in the memory to execute the predetermined processing procedure, or by hardware such as ASIC, FPGA, DSP to execute the predetermined processing procedure, or by a combination of the above two. Computer equipment includes, but is not limited to, servers, personal computers, laptops, tablets, smartphones, etc.

The computer equipment includes user equipment and network equipment. Wherein, the user equipment includes, but is not limited to, computers, smart phones, PDAs, etc.; Or a cloud composed of network servers, among them, cloud computing is a kind of distributed computing, a super virtual computer composed of a group of loosely coupled computer sets. Wherein, the computer device can operate independently to realize the present invention, and can also be connected to a network and realize the present invention by interacting with other computer devices in the network. Wherein, the network where the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like.

It should be noted that the user equipment, network equipment, and network are only examples, and other existing or future computer equipment or networks that are applicable to the present invention should also be included in the protection scope of the present invention, and Included herein by reference.

The methods discussed below, some of which are illustrated by flowcharts, can be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine or computer readable medium such as a storage medium. The processor(s) can perform the necessary tasks.

Specific structural and functional details disclosed herein are representative only and for purposes of describing example embodiments of the present invention. This invention may, however, be embodied in many alternative forms and should not be construed as limited to only the embodiments set forth herein.

It will be understood that although the terms "first", "second", etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present. Other words used to describe the relationship between elements should be interpreted in a similar fashion (e.g., "between" as opposed to "directly between", "adjacent to" as opposed to than "directly adjacent to" etc.).

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a", "an" and "an" are intended to include the plural unless the context clearly dictates otherwise. It should also be understood that the terms "comprising" and/or "comprising" as used herein specify the presence of stated features, integers, steps, operations, units and/or components, but do not exclude the presence or addition of one or more Other features, integers, steps, operations, units, components and/or combinations thereof.

It should also be noted that in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed substantially concurrently or may sometimes be executed in the reverse order, depending upon the functionality/acts involved.

The present invention will be described in further detail below in conjunction with the accompanying drawings.

Fig. 1 shows a schematic diagram of an apparatus for training a picture refinement model according to one aspect of the present invention.

The device 1 includes a capacity expansion device 101 , a clustering device 102 , a presentation device 103 , a first acquisition device 104 and a training device 105 . The apparatus 1 is located in computer equipment, for example, and the computer equipment includes user equipment and network equipment. The following describes in detail by taking the apparatus 1 located in a network device as an example.

Wherein, the capacity expansion device 101 performs picture expansion on an existing picture to obtain a picture after capacity expansion.

Specifically, for an existing picture, the capacity expansion device 101 can directly search for a picture by picture based on the existing picture, and perform picture matching in a local picture library, or search online for pictures, etc. Existing pictures are expanded to obtain expanded pictures. Here, the method of image expansion directly based on all existing images can be understood as that the user has selected all existing images. Alternatively, the user may select existing pictures, and select a part of pictures that meet his needs, and then, according to the existing pictures selected by the user, the expansion device 101 adopts the method of searching pictures by pictures, and searches through the local picture library. Perform image matching, or through online image search, etc., to expand the image size of the existing image, so as to obtain the expanded image. Alternatively, the user may input a certain keyword. For example, the user inputs a certain keyword at the keyword input port through interaction with the user equipment, and the capacity expansion device 101 interacts with the user equipment, for example, through one or more calls The application program interface (API) provided by the user equipment or through other agreed communication methods obtains the keyword entered by the user, and according to the keyword, obtains the keywords related to the keyword by searching in the local picture library or search engine, etc. Matching pictures, so as to expand the picture size of the existing picture, and obtain the picture after expansion. Alternatively, the capacity expansion device 101 may combine the above methods of image search and keyword search. words, by searching in a local image library or a search engine, etc., to obtain pictures that match the keyword and all or part of the existing pictures, so as to expand the size of the existing pictures and obtain the expanded pictures. Here, the existing picture may be obtained through matching with a picture search engine according to keywords input by the user, or may be a picture directly uploaded by the user. Here, there is no specific requirement for the target number of pictures after expansion, that is, as long as they match all or part of the existing pictures selected by these users, or further match the keywords entered by the user, they can be used as The image after expansion; of course, the target number of the image after expansion can also be specifically set, which can be preset by the system or adjusted by the user.

Preferably, according to the keyword input by the user, the expansion device 101 uses a picture search engine to obtain a picture matching the keyword as the existing picture.

Specifically, the user inputs a certain keyword at a predetermined entry or through a keyword input port of a specific application, for example, through interaction with the user equipment. The corresponding application program interface (API) or other agreed communication methods are called multiple times to obtain the keywords input by the user; then, the capacity expansion device 101 uses the image search engine to search according to the keywords input by the user, Thereby, a picture matching the keyword is obtained as the aforementioned existing picture.

Preferably, the expansion device 101 obtains the picture uploaded by the user as the existing picture.

Specifically, the user can also directly upload pictures. For example, the user uploads a certain number of pictures at a predetermined entry or through the picture upload entry of a specific application through interaction with the user equipment, and the capacity expansion device 101 communicates with the user equipment. Interaction, such as obtaining the picture uploaded by the user as the aforementioned existing picture by calling the corresponding application program interface (API) one or more times, or other agreed communication methods.

Those skilled in the art should be able to understand that the above methods of obtaining existing pictures are only examples, and other existing or future ways of obtaining existing pictures, if applicable to the present invention, should also be included within the protection scope of the present invention , and is hereby incorporated by reference.

Preferably, the capacity expansion device 101 expands the capacity of the pictures by using the key words input by the user and the pictures selected from the existing pictures by using the key words and searching pictures by pictures, and obtains the pictures after capacity expansion.

Specifically, for example, the user inputs certain keywords through the keyword input port and selects among these existing pictures, and the capacity expansion device 101 can , using keywords and image search methods, through existing image search engines, search for corresponding images, these images correspond to the keywords, and at the same time match with the existing images selected by these users, so as to search for images Expand to obtain the expanded image.

For example, as shown in Figure 2, for the existing pictures related to "sheep", the user selects some pictures that meet his needs in these existing pictures by checking. In addition, the user also enters keywords , if the keyword "sheep" is input in the keyword input port in the upper left corner, then the expansion device 101 will combine the keyword "sheep" input by the user according to the existing pictures of "sheep" selected by these users, Use the existing image search engine to search for corresponding images by using keywords and image search methods, so as to expand the image size of the existing images and obtain the expanded images. Here, FIG. 2 only shows part of the interface, and the actual number of existing pictures is not limited to the part shown in FIG. 2 .

Those skilled in the art should be able to understand that the above-mentioned way of expanding the size of an existing picture is only an example, and other existing or future ways of expanding the size of an existing picture, if applicable to the present invention, should also include within the scope of the present invention and are incorporated herein by reference.

Preferably, the device 1 also includes a second acquisition device (not shown). The second acquiring means acquires the target number of picture expansion set by the user; wherein, the expansion means 101 performs picture expansion on the existing picture according to the target number, and obtains the picture after expansion.

Specifically, the user can also set the target number of picture expansion. For example, the user sets the target number of picture expansion at the setting entry to 50,000 through interaction with the user equipment, and the second obtaining means interacts with the user equipment, By calling the application programming interface (API) provided by the user equipment one or more times or through other agreed communication methods, the target number of image expansion set by the user is obtained; then, the capacity expansion device 101 then uses the aforementioned target number according to the target number. The various methods listed are used to expand the size of the existing picture, so as to obtain the expanded picture.

For example, as shown in Figure 3, if the target number of image expansion set by the user is 50,000, the "Download Order of magnitude" displayed in the "New Task Submission" window shown in Figure 3 is 50,000. Task name: goat" indicates that pictures about "sheep" need to be downloaded, and "number of selection: 300" indicates that the user has selected 300 pictures from existing pictures. Then, the user clicks the button of "OK to submit the task" to start expanding the size of these existing pictures.

The clustering unit 102 clusters the expanded pictures to obtain a corresponding clustering result.

Specifically, the clustering device 102 performs clustering processing on the pictures expanded by the expansion device 101, so that pictures with the same or similar characteristics are clustered into one category, thereby obtaining corresponding clustering results. For example, for the 50,000 pictures related to "sheep" after the expansion in the above example, the clustering device 102 performs picture clustering on them. Although these 50,000 pictures are all related to "sheep", they include various "Sheep" with different picture features, such as pictures including "sheep" of various varieties, and even "sheep" in comics and paper-cut "sheep". Therefore, the clustering device 102 can classify The 50,000 pictures are clustered, so that the pictures with the same or similar features related to "sheep" are clustered into one category, so as to obtain the corresponding clustering results.

The presenting means 103 selects a predetermined number of pictures from at least one clustering result as sample pictures and presents them to the user.

Specifically, after the clustering means 102 clusters the expanded pictures to obtain the corresponding clustering results, the presentation means 103 selects a predetermined number of pictures from at least one of the clustering results as sample pictures, and uses the agreed presentation method , presenting these sample images to the user. Here, the specific value of the predetermined number may be preset by the system, or may be adjusted by the user.

As shown in Figure 4, following the above example, the clustering device 102 clustered the 50,000 pictures of "sheep" and obtained a plurality of clustering results, and the presenting device 103 selected from among each of the clustering results The 6 pictures are used as sample pictures, and are presented to the user in the interface shown in Figure 4. Here, FIG. 4 only shows a part of the interface, and the number of actual clustering results is not limited to the part shown in FIG. 4 . It can be seen from Figure 4 that each row is a clustering result, and each clustering result includes 6 sample pictures, and these 6 sample pictures have the same or similar picture features.

The first obtaining means 104 obtains positive and negative samples obtained by the user based on related operations on the clustering results.

Specifically, for each clustering result, the user can select one of the clustering results to perform related operations, and the user's related operations on the clustering results include but are not limited to:

Select one of the clustering results as a positive sample;

Select one of the clustering results as a negative sample;

Continue to cluster one of the clustering results and select positive and negative samples for the clustering results obtained after clustering again;

Of course, the user can also not perform any operation on a certain clustering result, or if the user is still not satisfied with the clustering result obtained after re-clustering, the clustering can be continued until the user is satisfied with the obtained clustering result , the user can select positive and negative samples from the satisfactory clustering results. Therefore, the first obtaining means 104 obtains positive and negative samples obtained by the user based on related operations on the clustering results.

Those skilled in the art should be able to understand that the related operations of the above-mentioned users on the clustering results are only examples, and other existing or future related operations of users on the clustering results, if applicable to the present invention, should also be included in this document. within the scope of the invention and is incorporated herein by reference.

Preferably, the first obtaining means 104 also re-clusters at least one clustering result, and selects positive and negative samples from the clustering result obtained after re-clustering.

Specifically, if the user is dissatisfied with at least one clustering result obtained by the clustering means 102, he may choose to cluster the at least one clustering result again, and the first obtaining means 104, for example, based on the user's instruction, Re-clustering the at least one clustering result respectively, if after one re-clustering, the user is satisfied with the clustering result obtained after re-clustering, then can select positive and negative samples from the clustering results; if the If the user is still dissatisfied, he can continue to cluster until he is satisfied with the obtained clustering results, and the user can select positive and negative samples from the satisfactory clustering results. Therefore, the first obtaining means 104 obtains the positive and negative samples finally selected by the user.

The training device 105 trains a corresponding image refinement model according to the positive and negative samples selected by the user.

Specifically, according to the positive and negative samples selected by the user, for example, all pairs of image clustering results are used as positive samples, and all wrong image clustering results are used as negative samples, the training device 105 trains the corresponding image Purification model, obtain a picture purification model of a small data set.

Here, device 1 is used for filtering massive image data by means of small-sample model training to extract high-quality image data, and uses existing image search engines, such as Baidu or Google, to obtain original image data, Use small samples (such as less than 100,000) for clustering of image data, such as clustering into hundreds of categories, select a few images for each category as example images of this category, and present them to the user for selection, and the user selects positive and negative samples from them After that, model training can be carried out, and a model with good classification performance can be obtained by using a low-cost method and a small number of user annotations; different from the previous tens of thousands or hundreds of thousands of manual annotations, users only need to spend a few minutes The small sample labeling task can be completed within a short time, and then the model training can be started to directly obtain the image purification model for mass image quality purification. After that, the image purification model can be used to mine more high-quality images from the massive image data .

Preferably, the device 1 also includes a purification device (not shown). The purifying device purifies the picture quality of a large-scale picture collection according to the picture purifying model.

Specifically, for the image purification model trained by the training device 105, the purification device performs image quality purification on a large-scale image collection according to the image purification model, for example, performs image quality purification on the aforementioned 50,000 pictures after capacity expansion, or Perform image quality purification on large-scale image collections obtained by other methods to obtain purified images.

For example, as shown in Figure 5, the training device 105 has trained a picture purification model according to the positive and negative samples of the pictures about "sheep" selected by the user, and the purification device performs the above-mentioned 50,000 pictures according to the picture purification model. The large-scale picture collection of "sheep" is purified for picture quality, and the pictures about "sheep" as shown in Figure 5 are obtained, and these pictures about "sheep" are more in line with the quality requirements of users.

Here, device 1 is used for quality purification of massive picture data according to the picture purification model obtained from the aforementioned training, so as to dig out more high-quality pictures from these massive picture data, so that these quality-purified picture data can be further used for model training.

Fig. 6 shows a schematic flowchart of a method for training a picture refinement model according to another aspect of the present invention.

In step S601, the device 1 expands the size of the existing picture to obtain the expanded picture.

Specifically, for an existing picture, in step S601, the device 1 can directly search for a picture based on the existing picture, by performing picture matching in the local picture library, or by searching for pictures online. etc. to expand the image size of the existing image, so as to obtain the expanded image. Here, the method of image expansion directly based on all existing images can be understood as that the user has selected all existing images. Alternatively, the user can select existing pictures and select some pictures that meet his needs. Then, in step S601, the device 1 will search for pictures by pictures according to the existing pictures selected by the user. Image matching is performed in the local image library, or through online image search, etc., and the image size of the existing image is expanded to obtain the expanded image. Alternatively, the user may input a certain keyword, for example, the user inputs a certain keyword in the keyword input port through interaction with the user equipment, then in step S601, the device 1 interacts with the user equipment, for example, once Or call the application programming interface (API) provided by the user's device multiple times or through other agreed communication methods to obtain the keywords entered by the user, and according to the keywords, obtain The image matching the keyword is used to expand the image size of the existing image and obtain the expanded image. Alternatively, in step S601, device 1 may combine the above methods of image search and keyword search, that is, in step S601, device 1 may select all or part of the existing pictures according to the , combined with the keyword entered by the user, by searching in the local image library or search engine, etc., to obtain pictures that match the keyword and all or part of the existing pictures, so as to expand the size of the existing pictures and obtain expansion after the picture. Here, the existing picture may be obtained through matching with a picture search engine according to keywords input by the user, or may be a picture directly uploaded by the user. Here, there is no specific requirement for the target number of pictures after expansion, that is, as long as they match all or part of the existing pictures selected by these users, or further match the keywords entered by the user, they can be used as The image after expansion; of course, the target number of the image after expansion can also be specifically set, which can be preset by the system or adjusted by the user.

Preferably, in step S601, the device 1 uses a picture search engine to obtain a picture matching the keyword according to the keyword input by the user as the existing picture.

Specifically, the user, for example, through interaction with the user equipment, enters a certain keyword at a predetermined entry or through a keyword input port of a specific application, and then, in step S601, the device 1 interacts with the user equipment, For example, by calling the corresponding application program interface (API) one or more times, or other agreed communication methods, the keyword input by the user is obtained; then, in step S601, the device 1 uses the image search engine again, according to the user The input keyword is searched, so as to obtain the picture matching the keyword as the aforementioned existing picture.

Preferably, in step S601, the device 1 obtains the picture uploaded by the user as the existing picture.

Specifically, the user can also directly upload pictures. For example, the user uploads a certain number of pictures at a predetermined entry or through the picture upload entry of a specific application through interaction with the user equipment, then in step S601, the device 1 passes through the The interaction of the user equipment, such as calling the corresponding application program interface (API) one or more times, or other agreed communication methods, obtains the picture uploaded by the user as the aforementioned existing picture.

Those skilled in the art should be able to understand that the above methods of obtaining existing pictures are only examples, and other existing or future ways of obtaining existing pictures, if applicable to the present invention, should also be included within the protection scope of the present invention , and is hereby incorporated by reference.

Preferably, in step S601, the device 1 expands the size of the image by using keywords and searching images by image according to the keyword input by the user and the image selected from the existing images, and obtains the enlarged image.

Specifically, for example, the user inputs certain keywords through the keyword input port and selects among these existing pictures. For the pictures selected in the website, use keywords and image search methods to search for corresponding pictures through existing picture search engines. These pictures correspond to the keywords and also match the existing pictures selected by these users. , so as to expand the image size and obtain the expanded image.

For example, as shown in Figure 2, for the existing pictures related to "sheep", the user selects some pictures that meet his needs in these existing pictures by checking. In addition, the user also enters keywords , if the keyword "sheep" is input in the keyword input port in the upper left corner, then in step S601, the device 1 combines the keywords input by the user with the existing pictures of "sheep" selected by these users "Sheep" uses the existing picture search engine to search for corresponding pictures by using keywords and image search methods, so as to expand the size of existing pictures and obtain pictures after expansion. Here, FIG. 2 only shows part of the interface, and the actual number of existing pictures is not limited to the part shown in FIG. 2 .

Those skilled in the art should be able to understand that the above-mentioned way of expanding the size of an existing picture is only an example, and other existing or future ways of expanding the size of an existing picture, if applicable to the present invention, should also include within the scope of the present invention and are incorporated herein by reference.

Preferably, the method further includes step S606 (not shown). In step S606, device 1 obtains the target number of picture expansion set by the user; wherein, in step S601, device 1 performs picture expansion on the existing picture according to the target number, and obtains an expanded picture.

Specifically, the user can also set the target number of image expansion. For example, the user sets the target number of image expansion to 50,000 through the interaction with the user equipment at the setting entry. Then in step S606, the device 1 communicates with the user equipment interaction, by calling the application programming interface (API) provided by the user equipment one or more times or through other agreed communication methods, to obtain the target number of picture expansion set by the user; then, in step S601, the device 1 then according to For the target quantity, through the above-mentioned various methods, the image size of the existing image is expanded, so as to obtain the expanded image.

For example, as shown in Figure 3, if the target number of image expansion set by the user is 50,000, the "Download Order of magnitude" displayed in the "New Task Submission" window shown in Figure 3 is 50,000. Task name: goat" indicates that pictures about "sheep" need to be downloaded, and "number of selection: 300" indicates that the user has selected 300 pictures from existing pictures. Then, the user clicks the button of "OK to submit the task" to start expanding the size of these existing pictures.

In step S602, the device 1 clusters the expanded pictures to obtain a corresponding clustering result.

Specifically, for the picture expanded in step S601, in step S602, the device 1 performs clustering processing on it, so that pictures with the same or similar characteristics are clustered into one category, so as to obtain the corresponding clustering result. For example, for the 50,000 pictures related to "sheep" after expansion in the above example, in step S602, device 1 performs picture clustering on them. Although these 50,000 pictures are all related to "sheep", they include "Sheep" with various picture features, such as pictures including "sheep" of various varieties, and even "sheep" in comics and paper-cut "sheep". Therefore, in step S602, the device 1 can follow the Different image features are used to cluster the 50,000 images, so that the images with the same or similar features related to "sheep" are clustered into one category to obtain corresponding clustering results.

In step S603, the device 1 selects a predetermined number of pictures from at least one clustering result as sample pictures and presents them to the user.

Specifically, in step S602, after device 1 clusters the expanded pictures to obtain corresponding clustering results, in step S603, device 1 selects a predetermined number of pictures from at least one of the clustering results as sample pictures , and present these sample images to the user through the agreed presentation method. Here, the specific value of the predetermined number may be preset by the system, or may be adjusted by the user.

As shown in Figure 4, following the example above, in step S602, device 1 clustered the 50,000 pictures of "sheep" and obtained multiple clustering results, and in step S603, device 1 clustered each clustering results, select 6 pictures as sample pictures, and present them to the user in the interface shown in Figure 4. Here, FIG. 4 only shows a part of the interface, and the number of actual clustering results is not limited to the part shown in FIG. 4 . It can be seen from Figure 4 that each row is a clustering result, and each clustering result includes 6 sample pictures, and these 6 sample pictures have the same or similar picture features.

In step S604, the device 1 acquires positive and negative samples obtained by the user based on related operations on the clustering results.

Specifically, for each clustering result, the user can select one of the clustering results to perform related operations, and the user's related operations on the clustering results include but are not limited to:

Select one of the clustering results as a positive sample;

Select one of the clustering results as a negative sample;

Continue to cluster one of the clustering results and select positive and negative samples for the clustering results obtained after clustering again;

Of course, the user can also not perform any operation on a certain clustering result, or if the user is still not satisfied with the clustering result obtained after re-clustering, the clustering can be continued until the user is satisfied with the obtained clustering result , the user can select positive and negative samples from the satisfactory clustering results. Therefore, in step S604, the device 1 acquires the positive and negative samples obtained by the user based on related operations on the clustering results.

Those skilled in the art should be able to understand that the related operations of the above-mentioned users on the clustering results are only examples, and other existing or future related operations of users on the clustering results, if applicable to the present invention, should also be included in this document. within the scope of the invention and is incorporated herein by reference.

Preferably, in step S604, the device 1 further performs re-clustering on at least one clustering result, and selects positive and negative samples from the clustering result obtained after re-clustering.

Specifically, if the user is dissatisfied with at least one clustering result obtained after clustering in step S602, he may choose to cluster the at least one clustering result again. In step S604, the device 1, for example, based on the user's indicates that the at least one clustering result is re-clustered respectively, and if the user is satisfied with the clustering result obtained after re-clustering after one re-clustering, the positive and negative samples can be selected from the clustering results; If the user is still not satisfied, he can continue to cluster until he is satisfied with the obtained clustering results, and the user can select positive and negative samples from the satisfied clustering results. Therefore, in step S604, the device 1 acquires the positive and negative samples finally selected by the user.

In step S605, the device 1 trains a corresponding image refinement model according to the positive and negative samples selected by the user.

Specifically, according to the positive and negative samples selected by the user, for example, all pairs of image clustering results are used as positive samples, and all wrong image clustering results are used as negative samples. In step S605, the device 1 according to these positive and negative samples, Train the corresponding image refinement model to obtain a picture refinement model for a small data set.

Here, device 1 is used for filtering massive image data by means of small-sample model training to extract high-quality image data, and uses existing image search engines, such as Baidu or Google, to obtain original image data, Use small samples (such as less than 100,000) for clustering of image data, such as clustering into hundreds of categories, select a few images for each category as example images of this category, and present them to the user for selection, and the user selects positive and negative samples from them After that, model training can be carried out, and a model with good classification performance can be obtained by using a low-cost method and a small number of user annotations; different from the previous tens of thousands or hundreds of thousands of manual annotations, users only need to spend a few minutes The small sample labeling task can be completed within a short time, and then the model training can be started to directly obtain the image purification model for mass image quality purification. After that, the image purification model can be used to mine more high-quality images from the massive image data .

Preferably, the method further includes step S607 (not shown). In step S607, the device 1 performs picture quality refinement on a large-scale picture collection according to the picture refinement model.

Specifically, for the image purification model trained and obtained in step S605, in step S607, device 1 performs image quality purification on a large-scale image collection according to the image purification model, for example, for the aforementioned 50,000 images after the expansion Perform picture quality refinement, or perform picture quality refinement on a large-scale picture collection obtained in other ways, to obtain the refined picture.

For example, as shown in Figure 5, in step S605, device 1 has trained a picture purification model according to the positive and negative samples of the pictures about "sheep" selected by the user. Image refinement model, the image quality of the aforementioned 50,000 large-scale image collections related to "sheep" is purified, and the images related to "sheep" are obtained as shown in Figure 5. These images related to "sheep" are more in line with the user's quality need.

Here, device 1 is used for quality purification of massive picture data according to the picture purification model obtained from the aforementioned training, so as to dig out more high-quality pictures from these massive picture data, so that these quality-purified picture data can be further used for model training.

The present invention also provides a computer-readable storage medium, the computer-readable storage medium stores computer codes, and when the computer codes are executed, the method described in any one of the preceding items is executed.

The present invention also provides a computer program product, when the computer program product is executed by a computer device, the method described in any one of the preceding items is executed.

The present invention also provides a kind of computer equipment, and described computer equipment comprises:

one or more processors;

memory for storing one or more computer programs;

When the one or more computer programs are executed by the one or more processors, the one or more processors are made to implement the method as described in any one of the preceding items.

It should be noted that the present invention can be implemented in software and/or a combination of software and hardware. For example, each device of the present invention can be implemented by using an application specific integrated circuit (ASIC) or any other similar hardware devices. In one embodiment, the software program of the present invention can be executed by a processor to realize the steps or functions described above. Likewise, the software program (including associated data structures) of the present invention can be stored in a computer-readable recording medium such as RAM memory, magnetic or optical drive or floppy disk and the like. In addition, some steps or functions of the present invention may be implemented by hardware, for example, as a circuit that cooperates with a processor to execute each step or function.

It will be apparent to those skilled in the art that the invention is not limited to the details of the above-described exemplary embodiments, but that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Accordingly, the embodiments should be regarded in all points of view as exemplary and not restrictive, the scope of the invention being defined by the appended claims rather than the foregoing description, and it is therefore intended that the scope of the invention be defined by the appended claims rather than by the foregoing description. All changes within the meaning and range of equivalents of the elements are embraced in the present invention. Any reference sign in a claim should not be construed as limiting the claim concerned. In addition, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or devices stated in the system claims may also be realized by one unit or device through software or hardware. The words first, second, etc. are used to denote names and do not imply any particular order.

Claims (17)

1. a kind of method for being used to train picture purification model, wherein, this method includes：

A carries out picture dilatation to existing picture, obtains the picture after dilatation；

B clusters to the picture after the dilatation, cluster result corresponding to acquisition；

C selects the picture of predetermined quantity to be presented to user as sample picture at least one cluster result；

D obtains the user based on the positive and negative samples obtained by the associative operation to the cluster result；

Positive negative samples of the e according to selected by the user, picture corresponding to training purify model.

2. according to the method for claim 1, wherein, step a includes：

The keyword inputted according to the user, the picture with the Keywords matching is obtained using photographic search engine, as The existing picture.

3. according to the method for claim 1, wherein, step a includes：

The picture that the user uploads is obtained, as the existing picture.

4. according to the method in any one of claims 1 to 3, wherein, step a includes：

The keyword inputted according to the user and the picture selected from the existing picture, use keyword and to scheme to search figure Mode carry out picture dilatation, obtain the picture after dilatation.

5. method according to any one of claim 1 to 4, wherein, this method also includes：

Obtain the destination number for the picture dilatation that the user is set；

Wherein, step a includes：

According to the destination number, picture dilatation is carried out to the existing picture, obtains the picture after dilatation.

6. method according to any one of claim 1 to 5, wherein, step d also includes：

At least one cluster result is clustered again, and the cluster result to being obtained after clustering again select to obtain it is positive and negative Sample.

7. method according to any one of claim 1 to 6, wherein, this method also includes：

Model is purified according to the picture, picture quality purification is carried out to extensive picture set.

8. a kind of device for being used to train picture purification model, wherein, the device includes：

Flash chamber, for carrying out picture dilatation to existing picture, obtain the picture after dilatation；

Clustering apparatus, for being clustered to the picture after the dilatation, cluster result corresponding to acquisition；

Device is presented, for selecting the picture of predetermined quantity to be presented to use as sample picture at least one cluster result Family；

First acquisition device, for obtaining the user based on the positive and negative sample obtained by the associative operation to the cluster result This；

Trainer, for the positive negative sample according to selected by the user, picture corresponding to training purifies model.

9. device according to claim 8, wherein, the flash chamber is used for：

The keyword inputted according to the user, the picture with the Keywords matching is obtained using photographic search engine, as The existing picture.

10. device according to claim 8, wherein, the flash chamber is used for：

The picture that the user uploads is obtained, as the existing picture.

11. the device according to any one of claim 8 to 10, wherein, the flash chamber is used for：

The keyword inputted according to the user and the picture selected from the existing picture, use keyword and to scheme to search figure Mode carry out picture dilatation, obtain the picture after dilatation.

12. the device according to any one of claim 8 to 11, wherein, the device also includes：

Second acquisition device, the destination number of the picture dilatation set for obtaining the user；

Wherein, the flash chamber is used for：

According to the destination number, picture dilatation is carried out to the existing picture, obtains the picture after dilatation.

13. the device according to any one of claim 8 to 12, wherein, first acquisition device is additionally operable to：

At least one cluster result is clustered again, and the cluster result to being obtained after clustering again select to obtain it is positive and negative Sample.

14. the device according to any one of claim 8 to 13, wherein, the device also includes：

Purifying plant, for purifying model according to the picture, picture quality purification is carried out to extensive picture set.

15. a kind of computer-readable recording medium, the computer-readable recording medium storage has computer code, when the meter When calculation machine code is performed, the method as any one of claim 1 to 7 is performed.

16. a kind of computer program product, when the computer program product is performed by computer equipment, such as claim 1 It is performed to the method any one of 7.

17. a kind of computer equipment, the computer equipment includes：

One or more processors；

Memory, for storing one or more computer programs；

When one or more of computer programs are by one or more of computing devices so that one or more of Processor realizes the method as any one of claim 1 to 7.