CN103778146B - Image clustering device and method - Google Patents

Abstract

Embodiments of the present invention provide an image clustering device and method, and the image clustering method includes: clustering multiple images based on visual features to obtain a first set; clustering multiple images with a link structure to obtain Obtaining the second set; fusing the first set and the second set by visual feature information and link structure information to obtain an image clustering result. Through the embodiments of the present invention, the accuracy of clustering results can be further improved, and classes with more consistent semantics can be generated.

Description

Image clustering device and method

technical field

The present invention relates to the field of image processing, in particular to an image clustering device and method.

Background technique

With the popularization of digital cameras and mobile phones with camera functions, the acquisition of images has become easier and easier. In addition, with the rapid development of the Internet and the increasing popularity of image sharing sites on the web, the number of images is increasing explosively, and it becomes time-consuming and laborious to quickly browse and search for desired images. At present, image tags are mainly relied on to assist quick browsing, but the tags themselves have limitations such as ambiguity, ambiguity, and inaccuracy, so they cannot solve this problem well.

Image self-organization based on image content is very important, and it can effectively assist image browsing. Image clustering (Image Clustering) is an effective method to achieve image self-organization based on image content, which quickly combines similar images together in a certain way. ClustTour is a recently proposed method for clustering and organizing images of scenic spots in cities. It first constructs two similarity graphs using image label information and visual information respectively, and then applies a graph clustering method on these two similarity graphs. Get the final clustering result.

However, the inventors found that the prior art (such as ClustTour) only considers the link structure between images, and it only uses the graph-based clustering method, so it needs the label information of the image, and cannot further improve the clustering result.

Documents useful to the understanding of the present invention and conventional art are listed below and are incorporated by reference as if fully set forth herein.

[Reference 1] S. Papadopoulos, C. Zigkolis, S. Kapiris, Y. Kompatsiaris and A. Vakali. Clust Tour: City Exploration by use of Hybrid Photo Clustering, In Proceedings of ACM Multimedia, 1617-1620, 2010.

[Reference 2] X.W.Xu, N.Yuruk, Z.D.Feng and T.A.J.Schweiger. SCAN: AStructural Clustering Algorithm for Networks, Proceedings of the 13th ACMSIGKDD international conference on Knowledge discovery and data mining, 824-833, 2007.

Contents of the invention

Embodiments of the present invention provide an image clustering device and method, with the purpose of further improving the accuracy of clustering results and generating clusters with more consistent semantics.

According to an aspect of an embodiment of the present invention, an image clustering device is provided, and the image clustering device includes:

The first clustering unit performs visual feature-based clustering on multiple images to obtain a first set;

a second clustering unit, performing clustering of link structures on the plurality of images to obtain a second set;

The fusion unit is configured to fuse the first set and the second set by using visual feature information and link structure information to obtain an image clustering result.

According to another aspect of the embodiments of the present invention, an image aggregation method is provided, and the image clustering method includes:

performing visual feature-based clustering on the plurality of images to obtain a first set;

performing link-structured clustering on the plurality of images to obtain a second set;

The result of image clustering is obtained by fusing the first set and the second set by using visual feature information and link structure information.

The beneficial effect of the present invention is that the accuracy of clustering results can be further improved and more consistent semantic classes can be generated by fusing the clustering based on visual features and the clustering based on link structure information.

With reference to the following description and accompanying drawings, there are disclosed in detail specific embodiments of the invention, indicating the manner in which the principles of the invention may be employed. It should be understood that embodiments of the invention are not limited thereby in scope. Embodiments of the invention encompass many changes, modifications and equivalents within the spirit and scope of the appended claims.

Features described and/or illustrated with respect to one embodiment can be used in the same or similar manner in one or more other embodiments, in combination with, or instead of features in other embodiments .

It should be emphasized that the term "comprising/comprising" when used herein refers to the presence of a feature, integer, step or component, but does not exclude the presence or addition of one or more other features, integers, steps or components.

Description of drawings

FIG. 1 is a schematic diagram of the composition of an image clustering device according to Embodiment 1 of the present invention;

FIG. 2 is a schematic diagram of the composition of an image clustering device according to Embodiment 2 of the present invention;

3 is a schematic diagram of the composition of the fusion unit in Embodiment 2 of the present invention;

FIG. 4 is a schematic diagram of the composition of the first update unit in Embodiment 2 of the present invention;

Fig. 5 is a flow chart of the image clustering method of Embodiment 3 of the present invention;

Fig. 6 is a flow chart of the image clustering method of Embodiment 4 of the present invention;

Fig. 7 is another schematic diagram of the image clustering method in Embodiment 4 of the present invention;

Fig. 8 is a flow chart of updating the second subset according to Embodiment 4 of the present invention.

detailed description

The foregoing and other features of the invention will become apparent from the following description, taken with reference to the accompanying drawings. In the specification and drawings, specific embodiments of the invention are disclosed, which illustrate some embodiments in which the principles of the invention may be employed. It is to be understood that the invention is not limited to the described embodiments, but rather, the invention The invention includes all modifications, variations and equivalents that come within the scope of the appended claims.

Example 1

An embodiment of the present invention provides an image clustering device, and FIG. 1 is a schematic diagram of the composition of the image clustering device according to the embodiment of the present invention. As shown in FIG. 1 , the image clustering apparatus 100 includes: a first clustering unit 101 , a second clustering unit 102 and a fusion unit 103 .

Wherein, the first clustering unit 101 performs clustering based on visual features to multiple images to obtain the first set; the second clustering unit 102 performs clustering of the link structure to multiple images to obtain the second set; the fusion unit 103 The result of image clustering is obtained by fusing the first set and the second set by visual feature information and link structure information.

In this embodiment, a plurality of images may be given first, and the images may be geotagged images. For example, given N images I={(x ₁ ,g ₁ ),(x ₂ ,g ₂ ),...,(x _n ,g _n )}, I _i =(x _i ,g _i ), where x _i is a d-dimensional feature vector, which represents the original feature of the i-th image, g _i is an e-dimensional feature vector, which represents the additional information of the image, such as GPS information. The purpose of the present invention is to divide the N images into m categories, so that the images in each category are as similar as possible.

In this embodiment, the first clustering unit 101 may perform clustering based on visual features. You can use traditional clustering methods, such as k-means, merge clustering, etc., to cluster images on the visual features of images, and obtain clustering results based on visual features, in which images with greater visual similarity will be classified as one type.

In this embodiment, the second clustering unit 102 can construct a K-Nearest Neighbor (KNN, K-Nearest Neighbor) graph for multiple images based on image visual features, and perform structured clustering on the KNN graph to obtain link-based structural information clustering results.

For example, KNN graphs can be constructed for all images based on image visual features first, and the specific construction process can be as follows: Calculate the distance between each image and all other images according to image visual features, and select the top k ₁ images with the smallest distance as the k ₁ of the image Neighboring images, each image is regarded as a node in the graph, and each image is connected with its k ₁ neighboring images to form the edges of the graph, and the visual similarity between images determines the weight of the edges.

Then, use a structured clustering algorithm on the KNN graph, such as SCAN ² , to cluster the images to obtain a clustering result based on link structure information. If two images have enough common link images, then their will be classified as one.

In this embodiment, the fusion unit 103 may fuse the first set and the second set to obtain an image clustering result. Since the visual features of images and the link structure between images are considered, different clustering methods can be used for different modal information, thus obtaining good clustering results.

It is worth noting that the above is only a schematic illustration of visual feature-based clustering and link structure-based clustering. But the present invention is not limited thereto, for example, other clustering algorithms can also be used. Alternatively, the clustering based on the link structure is not limited to the KNN graph, and other structures can also be constructed. The specific implementation manner can be determined according to specific conditions.

It can be known from the above embodiments that by fusing the clustering based on visual features and the clustering based on link structure information, the accuracy of the clustering results can be further improved and more semantically consistent clusters can be generated.

Example 2

On the basis of Embodiment 1, the embodiment of the present invention provides an image clustering device, and the same content as Embodiment 1 will not be repeated.

Fig. 2 is another schematic diagram of the structure of the image clustering device according to the embodiment of the present invention. As shown in FIG. 2 , the image clustering apparatus 200 includes: a first clustering unit 201 , a second clustering unit 202 and a fusion unit 203 . As shown in FIG. 2 , the image clustering apparatus 200 may further include: a classification unit 204 , which clusters a plurality of images according to classification information, so as to screen the plurality of images.

In this embodiment, the classification information may be image additional information, such as GPS information, but the present invention is not limited thereto, and may also be other classification information. The classification unit 204 can cluster the images based on the additional information of the images, and use the clustering results to filter the images.

For example, traditional clustering methods such as k-means, meanshift, etc. can be used to cluster images on image additional information such as GPS information, and then filter each category of the clustering results according to a predetermined principle, such as Classes with fewer images or classes with more GPS locations are filtered and deleted. The remaining images after filtering will be used for subsequent clustering processing, and can be input into the first clustering unit 201 and the second clustering unit 202 .

In this embodiment, the entire image clustering device can be mainly divided into three parts: (1) cluster images based on image additional information such as GPS information and perform image filtering based on the clustering results; (2) based on visual features and links (3) Fusion of visual feature-based clustering results and link structure-based clustering results. The following is a schematic illustration of how to specifically integrate.

FIG. 3 is a schematic structural diagram of a fusion unit according to an embodiment of the present invention. As shown in FIG. 3 , the fusion unit 203 may include a selection unit 301 and a processing unit 302 . Wherein, the selection unit 301 takes one of the first set and the second set as the target set, and uses the other set as the source set; the processing unit 302 adds the elements in the source set to the target set, or according to the elements in the source set Element updates the elements in the target collection.

Specifically, the processing unit 302 may include: a first calculating unit 3021 , a combining unit 3022 and a first updating unit 3023 . Wherein, the first calculation unit 3021 calculates the degree of overlap between the first subset and each subset of the target set for a first subset in the source set; the merging unit 3022 does not have an overlap degree greater than a preset threshold in the target set When there are subsets, the first subset is added to the target set; the first updating unit 3023 updates the second subset when there is a second subset whose degree of overlap is greater than a preset threshold in the target set.

During specific implementation, one of the clustering results from the first set and the second set may be selected as the target set C _d , and the other clustering result may be used as the source set C _s . Each element c _si in the source set C _s (that is, a class in the clustering result) can be analyzed to determine whether the element is directly added to the target set C _d or used to update certain elements in the target set C _d . The specific analysis process can be as follows:

Calculate the degree of overlap between c _si and each element c _dj in the target set C _d , where the calculation formula for the overlap of two elements Overlap(c _si ,c _dj ) can be as follows (1):

If there is no element whose overlapping degree is greater than the preset threshold thr in the target set, then add c _si to the target set; if there is an element c _dj in the target set, its overlap with element c _si is greater than the preset threshold thr, Then use the element c _si to update the element c _dj in the target set.

FIG. 4 is a schematic diagram of the composition of the first update unit according to an embodiment of the present invention. As shown in FIG. 4 , the first update unit 3023 may include: a first generation unit 401 , a second update unit 402 and a first replacement unit 403 . Among them, the first generating unit 401 uses the intersection of the first subset and the second subset as the third subset; the second updating unit 402 updates the third subset based on the cluster measurement value; the first replacing unit 403 uses the updated Then the third subset replaces the second subset.

Specifically, the second update unit 402 may include: a second generation unit 4021 , a second calculation unit 4022 and a second replacement unit 4023 . Among them, the second generation unit 4021, for each element that does not belong to the third subset but belongs to the first subset or the second subset, adds to the third subset and forms a new fourth subset; the second calculation The unit 4022 calculates the clustering measurement value for each fourth subset, so as to obtain a fourth subset with the optimal clustering measurement value; the clustering measurement value of the second replacement unit 4023 in the fourth subset is better than the first When clustering measurements of three subsets, replace the third subset with the fourth subset.

In a specific implementation, the second generation unit 4021, the second calculation unit 4022 and the second replacement unit 4023 can be executed multiple times, and the above process is repeated until no clustering measurement value of the new set is better than the clustering of the third subset Measurements.

In this embodiment, the image clustering apparatus 200 may further include: a third calculation unit (not shown in the figure), for a certain set of multiple images, the third calculation unit calculates the clustering measurement value of the set. Wherein, the cluster measurement value may include: one of global visual correlation value, local visual correlation value, global link correlation value, local link correlation value or a combination thereof.

In the following, the set is any type c for schematic illustration. The visual feature similarity of two images I _i and I _j can be defined as

where σ=mean(‖x _i −x _j ‖ ² , 1≤i≠j≤n) is the average visual feature distance computed over all pairwise image pairs.

Each image is used as a graph node. For each image i, its feature similarity with other images can be calculated by (2), and the top k ₂ images with the largest feature similarity can be found, denoted as N _vk2 (i ), and connect the image with these k ₂ images to form the edges of the graph, thereby constructing a k-nearest neighbor graph, and the weight of the edges of the graph is calculated by formula (3).

In one embodiment, the global visual correlation value can be calculated as follows: according to the visual features of the image, the visual similarity between any two images in class c is calculated, and the average of all visual similarities is the global visual correlation value, such as Formula (4).

Dist _gv (c)=mean(S _v,ij ),i,j∈c (4)

In another embodiment, the local visual correlation value can be calculated as follows: according to the visual characteristics of the image, for each image in class c, find the k2 images most similar to it from c and calculate its relationship with the k2 images Visual similarity, the average of all visual similarities is the local visual correlation value, such as formula (5).

Dist _lv (c)=mean(S _v,ij ),i∈c, _{j∈N Nvk2} (i) (5)

In another embodiment, the correlation value for the global link can be calculated as follows: construct a k-nearest neighbor graph based on visual features on all images, find such edges on the k-nearest neighbor graph, whose connected images are all in class c, and for these edges The weights are summed to obtain the first sum value; find such edges on the k-nearest neighbor graph, and the connected images have at least one image in class c, and the weights of these edges are summed to obtain the second sum value; through the first sum value Dividing by the second sum gives the global link correlation value, as in Equation (6).

During specific implementation, the third computing unit may include: a fourth computing unit, a fifth computing unit, and a sixth computing unit. Wherein, the fourth calculation unit finds one or more edges in which the connected images are all in the set on the constructed K-nearest neighbor graph, and sums the weights of these edges to obtain the first sum value; Find out that the connected image has at least one image in the set of one or more edges, and sum the weights of these edges to obtain the second sum value; the sixth calculation unit divides the first sum value by the second sum value to obtain Get the global link related value.

In another embodiment, the local link correlation value can be calculated as follows: find the link weights of any two images in class c, and average all the link weights to get the local link correlation value. Among them, the k-nearest neighbor graph can be constructed based on visual features on all images. The link weight of two images is the weight of the edge connecting the two images on the graph. If there is no edge connecting the two images on the graph, the link weight is 0. It can be as formula (7).

Dist _ll (c)=mean(W _ij ),i,j∈c (7)

During specific implementation, the third computing unit may include: a seventh computing unit and an eighth computing unit. Wherein, the seventh calculation unit calculates the link weights of any two images in the set; the eighth calculation unit averages all link weights to obtain local link correlation values.

In this embodiment, two types of clustering results can be fused in a consistent manner: clustering based on visual features and clustering based on link information, so two types of information can be considered at the same time to generate more semantically consistent clusters . Moreover, four different clustering measurements can be considered at the same time, which can evaluate clustering more effectively and provide a good basis for consistent fusion of two clustering results. Furthermore, the combined cluster measures can also be used in other applications such as ranking classes in the cluster results, etc.

Example 3

An embodiment of the present invention provides an image aggregation method, which corresponds to the image aggregation method in Embodiment 1, and the same content will not be repeated here. Fig. 5 is a flow chart of the image clustering method of the embodiment of the present invention, as shown in Fig. 5, this image clustering method comprises:

Step 501, performing clustering based on visual features on multiple images to obtain a first set;

Step 502, performing link structure clustering on multiple images to obtain a second set;

Step 503, merging the first set and the second set by using visual feature information and link structure information to obtain an image clustering result.

In this embodiment, step 502 performing link structure clustering on multiple images to obtain the second set may specifically include: constructing a KNN graph for multiple images based on image visual features, and performing structured clustering on the KNN graph to A clustering result based on link structure information is obtained. But the present invention is not limited thereto.

It can be known from the above embodiments that by fusing the clustering based on visual features and the clustering based on link structure information, the accuracy of the clustering results can be further improved and more semantically consistent clusters can be generated.

Example 4

An embodiment of the present invention provides an image aggregation method, which corresponds to the image aggregation method in Embodiment 2, and the same content will not be repeated here. Fig. 6 is a flow chart of the image clustering method of the embodiment of the present invention, as shown in Fig. 6, the image clustering method includes:

Step 601, clustering multiple images according to classification information for screening;

Step 602, performing clustering based on visual features on the multiple screened images to obtain a first set;

Step 603, performing link structure clustering on the screened multiple images to obtain a second set;

Step 604, merging the first set and the second set by using visual feature information and link structure information to obtain an image clustering result.

FIG. 7 is another schematic diagram of the image clustering method of the embodiment of the present invention. As shown in FIG. 7 , additional image information (such as GPS information) can be used to filter out noisy images, thereby making the clustering results more accurate.

In this embodiment, fusing the first set and the second set by using visual feature information and link structure information may specifically include: taking one of the first set and the second set as a target set, and using the other set as a source set; Add the elements in the source collection to the target collection, or update the elements in the target collection according to the elements in the source collection.

In this embodiment, adding the elements in the source set to the target set, or updating the elements in the target set according to the elements in the source set may specifically include: for each first subset in the source set, calculating the first The degree of overlap between the subset and each subset of the target set; when there is no subset with an overlap greater than the preset threshold in the target set, the first subset is added to the target set; if there is an overlap greater than the preset threshold in the target set When the second subset of the threshold is set, the second subset is updated.

During specific implementation, updating the second subset may include: taking the intersection of the first subset and the second subset as the third subset; updating the third subset based on the cluster measurement value; using the updated The third subset replaces the second subset.

Fig. 8 is a flow chart of updating the second subset according to the embodiment of the present invention. As shown in FIG. 8, updating the second subset may specifically include:

Step 801, taking the intersection of the first subset and the second subset as the third subset;

Step 802, for each element not belonging to the third subset but belonging to the first subset or the second subset, adding to the third subset to form a new fourth subset;

In this embodiment, if a new fourth subset is generated, step 803 is performed; if no new fourth subset is generated, step 805 is performed.

Step 803, calculating the clustering measurement value for each new fourth subset to obtain a fourth subset with the optimal clustering measurement value; and judging whether the clustering measurement value of the fourth subset is better than Clustering measures for the third subset;

In this embodiment, if the clustering measurement value of the fourth subset is better than the clustering measurement value of the third subset, then step 804 is performed; otherwise, step 805 is performed.

Step 804, replace the third subset with the fourth subset; then execute step 802.

Step 805, replace the second subset with the updated third subset.

That is to say, during specific implementation, the images shared by c _si and c _dj can form a class c _com . For each image that belongs to c _si ∪ c _d and does not belong to c _com , it can be added to c _com to generate a new class. If no new class is generated, c _dj can be replaced by class c _com ; otherwise for each new class, its clustering measure is calculated and a new class with the best clustering measure is selected. Judging whether the clustering measurement value of the new class is better than that of c _com , if it is better than c _com , replace and update the class c _com with the new class, and repeat the previous steps to regenerate the new class, otherwise it can be Replace c _dj with class c _com .

In this embodiment, the image clustering method may further include: for a set of multiple images, calculating the clustering measurement value of the set; wherein the clustering measurement value includes: global visual correlation value, local visual correlation value, global One or a combination of link-related values, partial link-related values.

In one embodiment, calculating the global link correlation value specifically includes: finding one or more edges in which the connected images are all in the set on the constructed K-nearest neighbor graph, and summing the weights of these edges to obtain the first sum value; On the K-nearest neighbor graph, find out that the connected images have at least one image in the set of one or more edges, and sum the weights of these edges to obtain the second sum value; divide the first sum value by the second sum value to obtain Get the global link related value.

In another embodiment, calculating the local link correlation value specifically includes: calculating the link weights of any two images in the set; and calculating the average value of all link weights to obtain the local link correlation value.

It can be seen from the above examples that two types of clustering results can be fused in a consistent manner: clustering based on visual features and clustering based on link information, so two types of information can be considered at the same time to generate more semantically consistent clusters . Moreover, four different clustering measurements can be considered at the same time, which can evaluate clustering more effectively and provide a good basis for consistent fusion of two clustering results. Furthermore, the combined cluster measures can also be used in other applications such as ranking classes in the cluster results, etc.

The above devices and methods of the present invention can be implemented by hardware, or by combining hardware and software. The present invention relates to such a computer-readable program that, when the program is executed by a logic component, enables the logic component to realize the above-mentioned device or constituent component, or enables the logic component to realize the above-mentioned various methods or steps. The present invention also relates to a storage medium for storing the above program, such as hard disk, magnetic disk, optical disk, DVD, flash memory and the like.

The present invention has been described above in conjunction with specific embodiments, but those skilled in the art should be clear that these descriptions are all exemplary and not limiting the protection scope of the present invention. Those skilled in the art can make various variations and modifications to the present invention according to the spirit and principle of the present invention, and these variations and modifications are also within the scope of the present invention.

Regarding the implementation manner comprising the above embodiments, the following additional notes are also disclosed:

(Supplementary note 1) An image clustering device, the image clustering device comprising:

The first clustering unit performs visual feature-based clustering on multiple images to obtain a first set;

a second clustering unit, performing clustering of link structures on the plurality of images to obtain a second set;

The fusion unit is configured to fuse the first set and the second set by using visual feature information and link structure information to obtain an image clustering result.

(Supplement 2) The image clustering device according to Supplement 1, wherein the image clustering device further includes:

The classification unit is configured to cluster the multiple images according to the classification information, so as to filter the multiple images.

(Supplementary Note 3) The image clustering device according to Supplementary Note 1 or 2, wherein the second clustering unit constructs a KNN graph for the plurality of images based on image visual features, and performs Structured clustering to obtain clustering results based on link structure information.

(Supplement 4) The image clustering device according to any one of Supplements 1 to 3, wherein the fusion unit includes:

A selection unit, using one of the first set and the second set as a target set, and using the other set as a source set;

A processing unit, adding elements in the source collection to the target collection, or updating elements in the target collection according to the elements in the source collection.

(Supplementary Note 5) The image clustering device according to Supplementary Note 4, wherein the processing unit includes:

A first calculation unit, for a first subset in the source set, calculates the degree of overlap between the first subset and each subset of the target set;

The merging unit is configured to add the first subset to the target set when there is no subset with an overlapping degree greater than a preset threshold in the target set;

The first updating unit is configured to update the second subset when there is a second subset whose degree of overlap is greater than a preset threshold in the target set.

(Supplementary Note 6) The image clustering device according to Supplementary Note 5, wherein the first updating unit includes:

a first generation unit, using the intersection of the first subset and the second subset as a third subset;

a second updating unit, configured to update the third subset based on the clustering measurement value;

The first replacing unit is configured to replace the second subset with the updated third subset.

(Supplement 7) The image clustering device according to Supplement 6, wherein the second update unit includes:

The second generating unit, for each element that does not belong to the third subset but belongs to the first subset or the second subset, is added to the third subset to form a new fourth subset ;

a second computing unit, computing clustering measures for each of said fourth subsets to obtain a fourth subset with an optimal clustering measure;

The second replacement unit is configured to replace the third subset with the fourth subset when the cluster measurement value of the fourth subset is better than the cluster measurement value of the third subset.

(Supplement 8) The image clustering device according to any one of Supplements 1 to 7, wherein the image clustering device further includes:

A third calculation unit, for a set of the plurality of images, calculates the clustering measurement value of the set; the clustering measurement value includes: global visual correlation value, local visual correlation value, global link correlation value, local link One or a combination of related values.

(Supplement 9) The image clustering device according to Supplement 8, wherein the third calculation unit includes:

The fourth calculation unit finds one or more edges in which the connected images are all in the set on the constructed K-nearest neighbor graph, and sums the weights of the one or more edges to obtain the first sum;

The fifth calculation unit finds out one or more edges in which at least one of the connected images is in the set on the K-nearest neighbor graph, and sums the weights of the one or more edges to obtain a second sum value ;

A sixth calculation unit, dividing the first sum by the second sum to obtain the global link correlation value.

(Supplementary Note 10) The image clustering device according to Supplementary Note 8, wherein the third calculation unit includes:

The seventh calculation unit is used to calculate the link weight of any two images in the set;

The eighth calculation unit calculates the average value of all link weights to obtain the local link correlation value.

(Appendix 11) An image aggregation method, the image clustering method comprising:

performing visual feature-based clustering on the plurality of images to obtain a first set;

performing link-structured clustering on the plurality of images to obtain a second set;

The result of image clustering is obtained by fusing the first set and the second set by using visual feature information and link structure information.

(Supplementary Note 12) The image clustering method according to Supplementary Note 11, wherein the image clustering method further includes:

clustering the plurality of images according to classification information, so as to filter the plurality of images;

In addition, visual feature-based clustering is performed on the multiple screened images to obtain a first set, and link structure clustering is performed on the multiple screened images to obtain a second set.

(Supplementary Note 13) The image clustering method according to Supplementary Note 11 or 12, wherein performing link structure clustering on the plurality of images to obtain the second set specifically includes:

Constructing a KNN graph for the plurality of images based on image visual features, and performing structured clustering on the KNN graph to obtain a clustering result based on link structure information.

(Supplementary Note 14) The image clustering method according to any one of Supplementary Notes 11 to 13, wherein fusing the first set and the second set through visual feature information and link structure information specifically includes:

using one of the first set and the second set as a target set, and using the other set as a source set;

Add the elements in the source collection to the target collection, or update the elements in the target collection according to the elements in the source collection.

(Supplementary Note 15) The image clustering method according to Supplementary Note 14, wherein the elements in the source set are added to the target set, or the target set is updated according to the elements in the source set The elements specifically include:

For a first subset of the source set, calculating the degree of overlap between the first subset and each subset of the target set;

When there is no subset with an overlap greater than a preset threshold in the target set, adding the first subset to the target set; there is a second subset with an overlap greater than a preset threshold in the target set , update the second subset.

(Supplementary Note 16) The image clustering method according to Supplementary Note 15, wherein updating the second subset specifically includes:

taking the intersection of the first subset and the second subset as a third subset;

updating the third subset based on clustering measures;

The second subset is replaced with the updated third subset.

(Supplementary Note 17) The image clustering method according to Supplementary Note 16, wherein updating the third subset based on the clustering measurement value specifically includes:

For each element not belonging to the third subset but belonging to the first subset or the second subset, adding to the third subset forms a new fourth subset;

calculating a clustering measure for each of said fourth subsets to obtain a fourth subset having an optimal clustering measure;

The third subset is replaced with the fourth subset when the cluster measure of the fourth subset is better than the cluster measure of the third subset.

(Supplementary Note 18) The image clustering method according to any one of Supplementary Notes 11 to 17, wherein the image clustering method further includes:

For a set of the plurality of images, calculate the clustering measure value of the set; the clustering measure value includes: global visual correlation value, local visual correlation value, global link correlation value, local link correlation value among them one or a combination thereof.

(Supplementary Note 19) The image clustering method according to Supplementary Note 18, wherein calculating the global link correlation value specifically includes:

On the constructed K-nearest neighbor graph, find one or more edges whose connected images are all in the set, and sum the weights of the one or more edges to obtain the first sum;

Finding on the K-nearest neighbor graph that the connected images have at least one image in the set of one or more edges, and summing the weights of the one or more edges to obtain a second sum;

The first sum is divided by the second sum to obtain the global linkage correlation value.

(Supplementary Note 20) The image clustering method according to Supplementary Note 18, wherein calculating the local link correlation value specifically includes:

find the link weights of any two images in the set;

All link weights are averaged to obtain the local link correlation value.

(Supplementary Note 21) A computer-readable program, wherein when the program is executed in a computer, the program causes the computer to execute the image clustering method described in any one of Supplementary Notes 11 to 20.

(Supplementary Note 22) A storage medium storing a computer-readable program, wherein the computer-readable program causes a computer to execute the image clustering method described in any one of Supplementary Notes 11 to 20.

Claims (8)

1. a kind of image clustering device, described image clustering apparatus include：

Multiple images are carried out the cluster of view-based access control model feature to obtain first set by the first cluster cell；

Second cluster cell, carries out the cluster of link structure to obtain second set to the plurality of image；And

Integrated unit, by visual signature information and first set and second set described in link structure information fusion, to obtain The result of image clustering,

Wherein, described integrated unit includes：

Select unit, one of described first set and described second set is gathered as goal set, another is collected Cooperate to gather for source；And

Processing unit, the element during described source is gathered is added in described goal set, or according in the set of described source Element updates the element in described goal set,

And, described processing unit includes：

First computing unit, for one of described source set the first subset, calculates described first subset and described object set The degree of overlapping of each subset closed；

Combining unit, when there is not the subset that degree of overlapping is more than predetermined threshold value in described goal set, by described first subset It is added in goal set；And

First updating block, when there is the yield in the second subset that degree of overlapping is more than predetermined threshold value in described goal set, to described the Two subsets are updated.

2. image clustering device according to claim 1, wherein, described image clustering apparatus also include：

Taxon, clusters to the plurality of image according to classification information, to screen to the plurality of image.

3. image clustering device according to claim 1, wherein, described first updating block includes：

First signal generating unit, using the common factor of described first subset and described yield in the second subset as the 3rd subset；

Second updating block, is updated to described 3rd subset based on cluster measured value；

First replacement unit, replaces described yield in the second subset with described 3rd subset after updating.

4. image clustering device according to claim 3, wherein, described second updating block includes：

Second signal generating unit, for being not belonging to described 3rd subset and belong to each unit of described first subset or yield in the second subset Element, forms a 4th new subset after increasing to described 3rd subset；

Second computing unit, calculates cluster measured value for the 4th subset each described, to obtain the cluster measurement with optimum One the 4th subset of value；

Second replacement unit, when the cluster measured value of described 4th subset is better than the cluster measured value of described 3rd subset, uses Described 4th subset replaces described 3rd subset.

5. the image clustering device according to any one of Claims 1-4, wherein, described image clustering apparatus also include：

3rd computing unit, for a set of the plurality of image, calculates the cluster measured value of described set；Described cluster Measured value includes：Overall Vision correlation, local visual correlation, overall situation link correlation, its in local link correlation One of or a combination thereof.

6. image clustering device according to claim 5, wherein, described 3rd computing unit includes：

4th computing unit, finds the image of connection one or more side all in described set on the k nearest neighbor figure building, Weight summation to described one or more side obtains the first value preset；

5th computing unit, finds out at least one image of image of connection in described set on described k nearest neighbor figure Bar or multiple summits, the weight summation to described one or more side obtains the second value preset；

6th computing unit, by described first value preset divided by described second value preset to obtain described overall situation link correlation.

7. image clustering device according to claim 5, wherein, described 3rd computing unit includes：

7th computing unit, obtains the link weight of any two image in described set；

8th computing unit, averages to all-links weight to obtain described local link correlation.

8. a kind of image clustering method, described image clustering method includes：

Multiple images are carried out with the cluster of view-based access control model feature to obtain first set；

The plurality of image is carried out with the cluster of link structure to obtain second set；And

By visual signature information and first set and second set described in link structure information fusion, to obtain image clustering As a result,

Wherein, included by visual signature information and first set and second set described in link structure information fusion：

Using the set of one of described first set and described second set as goal set, using another set as source collection Close；And

Element during described source is gathered is added in described goal set, or updates institute according to the element in the set of described source State the element in goal set,

And, the element in the described set by described source is added in described goal set, or according in the set of described source The process that element updates the element in described goal set includes：

For one of described source set the first subset, calculate each subset of described first subset and described goal set Degree of overlapping；

When there is not the subset that degree of overlapping is more than predetermined threshold value in described goal set, described first subset is added to target In set；And

When there is the yield in the second subset that degree of overlapping is more than predetermined threshold value in described goal set, described yield in the second subset is carried out more Newly.