CN108846404B - A method and device for image saliency detection based on correlation constraint graph ranking - Google Patents

Abstract

The invention discloses an image saliency detection method and device based on related constraint graph sorting. The method includes: performing superpixel segmentation on a to-be-detected image, establishing a closed-loop graph model, and then calculating the prior information of each superpixel node; extracting Input the color, texture, position and other information of the image; obtain the foreground probability value of each superpixel node; take the set of nodes whose foreground probability value is greater than the first preset threshold as the foreground seed point set ind_fore; set the foreground probability value less than the second The set of nodes with the preset threshold is used as the set of background seed points ind_back; the first preset threshold is greater than the second preset threshold; the foreground probability S_f of each superpixel node is obtained by using the model for sorting the relevant constraint graph, and the foreground probability value is used. S_f as the final saliency estimate S_final. By applying the embodiments of the present invention, the saliency detection result can be made more accurate.

Description

A method and device for image saliency detection based on correlation constraint graph ranking

technical field

The present invention relates to a saliency detection method and device, and more particularly to an image saliency detection method and device based on correlation constraint graph ranking.

Background technique

With the rapid development of computer and network communication technology, there are more and more image data. Massive multimedia image data brings great challenges to information processing. In recent years, the main research focus is how to efficiently store, analyze and process these image information. Saliency detection is an important preprocessing step used to reduce computational complexity in the field of computer vision. The task of saliency detection is to locate and segment the most salient foreground objects from the scene. The application fields of this technology are particularly wide, such as: object detection and recognition, content-based image retrieval, context-aware image resizing, video object detection, etc. How to quickly and accurately find the salient regions of an image has not yet formed a complete theoretical system and is closely related to specific applications, which is still a challenging topic for researchers.

Currently, bottom-up approaches are commonly used for visual information processing. Bottom-up methods are usually based on the underlying visual information, so they can effectively detect the detailed information of the image rather than the global shape information, and the detected salient regions may only contain part of the object or easily blend with the background. In recent years, many bottom-up saliency detection models have emerged: Initially, Itti et al. proposed a neural network-based saliency detection model, which combines three feature channels in multiple scales to achieve fast scene analysis, Although the model was able to identify some salient pixels, the results also contained a large number of false positives. Harel et al. proposed a graph-based saliency detection method, which is a bottom-up model that obtains the final saliency result by computing dissimilarity. Chang et al. constructed a graphical model that combines similarity and regional saliency to obtain better saliency estimates. Wang et al. proposed a saliency detection model combining local graph structure and background priors, and proposed an optimized framework for saliency detection, and the final experimental results showed good performance in most scenarios. Jiang et al. proposed an absorption Markov chain model for image saliency detection. Tu et al. proposed the use of a minimum spanning tree model for image saliency detection. Li et al. proposed a ranking model using regularized random walks to estimate significance values. Yang et al. proposed a graph-based saliency detection algorithm for manifold sorting (hereinafter referred to as the MR algorithm). The algorithm filters out some foreground seed points and background seed points, and then uses the manifold sorting model to calculate the relationship between the remaining nodes and the correlation between these seed points to get the final saliency value.

However, the MR algorithm is divided into two stages. First, the correlation between the remaining nodes and the obtained background seed points is calculated, and the preliminary significant results are obtained after inversion. Then, on the basis of the first stage, the foreground seed points are obtained. , and then calculate the correlation of the remaining nodes with these foreground seed points to obtain the final result. The two sorting processes of this method are completed independently, which will lead to the technical problem that the accuracy of image saliency detection is not high.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide an image saliency detection method and device based on related constraint graph sorting, so as to solve the deficiencies existing in the existing graph-based manifold sorting model.

The present invention solves the above-mentioned technical problems through the following technical solutions:

An embodiment of the present invention provides an image saliency detection method based on correlation constraint graph ranking, and the method includes:

A: For each image to be detected, use the simple linear iterative clustering SLIC algorithm to perform superpixel segmentation on the image to be detected to obtain non-overlapping superpixel blocks, and then use each non-overlapping superpixel block as The node establishes a closed-loop graph model, and then calculates the center prior information of each node;

B: Extract the color, texture, position and other information of the input image;

C: Use the MR algorithm to obtain the foreground probability value of each node;

D: The set of nodes whose foreground probability value is greater than the first preset threshold is used as the foreground seed point set ind_fore; the set of nodes whose foreground probability value is less than the second preset threshold is used as the background seed point set ind_back; the first preset threshold is greater than the second preset threshold;

E: Calculate the foreground probability S_f of each superpixel node using the model sorted by the relevant constraint graph, and use the foreground probability value S_f as the final saliency estimate value S_final.

Optionally, the step A includes:

A1: For each image to be detected, use the SLIC algorithm to divide the image into N superpixel blocks, and each superpixel is used as a node in the set V; then obtain the undirected edge corresponding to each node , and then construct an undirected graph model G ¹ =(V, E);

计算每一个节点的中心先验信息，其中，A2: Using the formula,

Calculate the center prior information of each node, where,

c _i is the center prior information of the ith node; _xi is the abscissa of the center position of the ith node; y _i is the ordinate of the center position of the ith node; (x ₀ , y ₀ ) represents the integer is the coordinate of the center position of the image; σ ₁ is the balance parameter, which is used to control the discrete degree of the calculated position distance; exp() is the exponential function with the natural base as the base; i is the number of nodes.

Optionally, the undirected edge is obtained through the following steps:

For the adjacent node l of the second neighbor k of each node i, using the formula, dist(k,l)=||x _k -x _l || ² , calculate the color Euclidean distance dist(k,l ); if the color Euclidean distance is less than the threshold θ, connect an undirected edge between node l and node i, and continue searching after finding the connected nodes until all nodes are connected, where dist(k,l) is the color Euclidean distance between the lth node and the kth node; x _k is the color value of the kth node; x _l is the color value of the lth node; |||| is the modulo function.

计算每个无向边的权重，构建第一关联矩阵

其中，Optionally, the step B includes: according to the color, texture, and position information of the extracted image; using a formula,

Calculate the weight of each undirected edge and construct the first correlation matrix

in,

为第i个节点与第j个节点之间的无向边的权重；i和j为节点的序号，且0≤i,j≤N；v_i为第i个节点的特征描述符，且v_i∈R⁶⁵，v_i＝[x_i,y_i,L_i,a_i,b_i,c_i,ω_i]；(x_i,y_i)表示的是每个超像素节点的中心位置坐标；(L_i,a_i,b_i)表示的是每个超像素节点的在CIE LAB颜色空间中包含的所有像素点的颜色均值；c_i为第i个节点的中心先验信息；ω_i为第i个节点的LBP值；v_j为第j个节点的特征描述符；σ为预设的控制权重平衡的常数；n为超像素块的数量。

is the weight of the undirected edge between the ith node and the jth node; i and j are the serial numbers of the nodes, and 0≤i,j≤N; vi is the feature descriptor of the _ith node, and v _i ∈R ⁶⁵ , v _i =[x _i ,y _i ,L _i ,a _i , _bi , _ci ,ω _i ]; (x _i ,y _i ) represents the center position coordinates of each superpixel node ; (L _i , a _i , b _i ) represents the color mean of all pixels contained in the CIE LAB color space of each superpixel node; c _i is the center prior information of the ith node; ω _i is the LBP value of the i-th node; v _j is the feature descriptor of the j-th node; σ is the preset constant to control the weight balance; n is the number of superpixel blocks.

Optionally, the step C includes:

C1: Obtain the weight of each undirected edge of each node in the MR algorithm;

其中，C2: According to the weight of each of the undirected edges, construct the second correlation matrix of the MR algorithm

in,

为所述第i个超像素和第j个超像素之间边的权重，且

W²为第二关联矩阵；i,j∈V，i为第i个节点的序号；j为第j个节点的序号；c_i为第i个节点在CIE LAB颜色空间中所有像素点的颜色均值；c_j为第j个节点在CIELAB颜色空间中所有像素点的颜色均值；σ为控制权重平衡的常数；

is the weight of the edge between the ith superpixel and the jth superpixel, and

W ² is the second association matrix; i,j∈V, i is the serial number of the ith node; j is the serial number of the jth node; c _i is the color of all pixels of the ith node in the CIE LAB color space Mean; c _j is the color mean of all pixels of the jth node in the CIELAB color space; σ is a constant that controls the weight balance;

C3: According to the formula, D=diag{d ₁₁ ,...,d _nn }, calculate the degree matrix, where,

为第二关联矩阵对应的无向边的权重；D is the degree matrix; diag{} is the diagonal matrix construction function; d _ii is the degree matrix element, and

is the weight of the undirected edge corresponding to the second association matrix;

C4: For each node on the boundary, mark the label value of the node according to the boundary prior;

C5: Using the formula, f:X→R ^m , calculate the sorting weight corresponding to the image to be detected, wherein,

f is the sorting function, and f=[f ₁ ,...,f _n ] ^T ; f ₁ is the sorting value of the first node; f _n is the sorting value of the n-th node; n is the number of nodes; let y =[y ₁ , y ₂ ,...y _n ] ^T represents the label vector, the label value of the seed point is 1, and the label value of the remaining nodes is 0; X is the feature matrix corresponding to the input image; R is the real number space; R ^m is the m-dimensional real number space; m is the space dimension; y is the vector composed of the label values of all seed nodes;

计算闭合解，其中，C6: Using the sorting function formula,

Compute the closed solution, where,

为求解函数最小值自变量函数；∑为求和函数；f_i为第i个节点的排序值；f_j为第j个节点的排序值；y_i为第i个节点的标签值；

为所述无向边的权重；d_ii为度矩阵中第i行i列的元素；d_jj为度矩阵中第j行j列的元素；μ为平衡参数；f* is the sorting function;

is the independent variable function of the minimum value of the solution function; ∑ is the summation function; f _i is the sorting value of the i-th node; f _j is the sorting value of the j-th node; y _i is the label value of the i-th node;

is the weight of the undirected edge; d _ii is the element in the i-th row and i-column in the degree matrix; d _jj is the element in the j-th row and j column in the degree matrix; μ is the balance parameter;

获取非归一化解，其中，C7: Using the formula according to the closed solution,

to obtain a non-normalized solution, where,

D is the degree matrix; W ² is the second correlation matrix; S is the normalization matrix of W ² ;

分别计算每个节点与四个边界上的背景种子点之间的相关性，得到四种情况下每个节点的背景概率值f，其中，λ为预设参数；C8: Using the formula,

Calculate the correlation between each node and the background seed points on the four boundaries respectively, and obtain the background probability value f of each node in the four cases, where λ is a preset parameter;

再进行取反得到每个节点的显著值；将四种情况下得到的显著值进行点乘获取初始结果S_MR，作为节点的前景概率值。C9: Normalize the correlation value between each node and the background seed points on the four boundaries to get

Then inversion is performed to obtain the salient value of each node; the salient value obtained in the four cases is dot-multiplied to obtain the initial result S_MR, which is used as the foreground probability value of the node.

Optionally, the D step includes:

获取第一预设阈值和第二预设阈值；其中，D1: Using the formula,

Obtain the first preset threshold and the second preset threshold; wherein,

h ₁ is the first preset threshold value; h ₂ is the second preset threshold value; mean is the averaging function; max is the maximum value evaluating function;

获取前景种子点集合ind_fore和背景种子点集合ind_back，其中，D2: Using the formula,

Get the foreground seed point set ind_fore and the background seed point set ind_back, where,

ind_fore is a set of foreground seed points; ind_back is a set of background seed points; θ is a preset parameter.

Optionally, the E step includes:

E1: Using the formula, F:X→R ⁿ , calculate the sorting weight corresponding to the image to be detected, wherein,

F is the sorting function; F _i represents the sorting value of the ith node, and F=(f, g); f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background;

获取每一个节点的标签值；再利用公式，Y＝(y¹,y²)∈R^m×2，获取每一个节点的标签向量，其中，E2: Using the formula,

Obtain the label value of each node; then use the formula, Y=(y ¹ , y ² )∈R ^m×2 , obtain the label vector of each node, where,

Y is the label vector of each node; y ¹ is the label value that the node belongs to the foreground; y ² is the label value that the node belongs to the background; R ^m×2 is a 2m-dimensional real number space;

E3: The model formula of the constructed related constraint graph ordering is,

其中，

in,

F ^* is the closed solution; W _ij is the first association matrix corresponding to the weight of the undirected edge between the i-th node and the j-th node; F _i is the sorting value of the i-th node; F _j is the j-th node d _i is the degree matrix element; f _i is the foreground probability of the _i -th node; gi is the background probability of the i-th node; w _i is the feature weight of the i-th node; x _i is the i-th node The characteristics of the node; b _i is the bias parameter; β ₁ is the linear constraint coefficient on the foreground probability; β ₂ is the linear constraint coefficient on the background probability;

E4: Obtain the significance value by taking the partial derivative of the foreground probability for the ranking model using the step E3.

The embodiment of the present invention also provides an image saliency detection device based on the correlation constraint graph ranking, and the device includes:

The first calculation module is used to perform superpixel segmentation on the image to be detected using a simple linear iterative clustering SLIC algorithm for each image to be detected, to obtain non-overlapping superpixel blocks, and then divide each of the non-overlapping superpixel blocks. The superpixel block is used as a node to establish a closed-loop graph model, and then the center prior information of each node is calculated;

The input module is used to extract the color, texture, position and other information of the input image;

The second calculation module is used to obtain the foreground probability value of each node by using the MR algorithm;

The first setting module is used to set the set of nodes whose foreground probability value is greater than the first preset threshold as the set of foreground seed points ind_fore; the set of nodes whose foreground probability value is less than the second preset threshold is set as the set of background seed points ind_back; a preset threshold is greater than the second preset threshold;

The second setting module is used to calculate the foreground probability S_f and the background probability S_g of each superpixel node by using the model of related constraint graph sorting, and use the foreground probability value S_f as the final significant estimation value S_final.

Optionally, the first computing module is also used for:

A1: For each image to be detected, use the SLIC algorithm to divide the image into N superpixel blocks, and each superpixel is used as a node in the set V; then obtain the undirected edge corresponding to each node , and then construct an undirected graph model G ¹ =(V, E);

计算每一个节点的中心先验信息，其中，A2: Using the formula,

Calculate the center prior information of each node, where,

c _i is the center prior information of the ith node; _xi is the abscissa of the center position of the ith node; y _i is the ordinate of the center position of the ith node; (x ₀ , y ₀ ) represents the integer is the coordinate of the center position of the image; σ ₁ is the balance parameter, which is used to control the discrete degree of the calculated position distance; exp() is the exponential function with the natural base as the base; i is the number of nodes.

Optionally, the second computing module is also used for:

C1: Obtain the weight of each undirected edge of each node in the MR algorithm;

其中，C2: According to the weight of each of the undirected edges, construct the second correlation matrix of the MR algorithm

in,

为所述第i个超像素和第j个超像素之间边的权重，且

W²为第二关联矩阵；i,j∈V，i为第i个节点的序号；j为第j个节点的序号；c_i为第i个节点在CIE LAB颜色空间中所有像素点的颜色均值；c_j为第j个节点在CIELAB颜色空间中所有像素点的颜色均值；σ为控制权重平衡的常数；

is the weight of the edge between the ith superpixel and the jth superpixel, and

W ² is the second association matrix; i,j∈V, i is the serial number of the ith node; j is the serial number of the jth node; c _i is the color of all pixels of the ith node in the CIE LAB color space Mean; c _j is the color mean of all pixels of the jth node in the CIELAB color space; σ is a constant that controls the weight balance;

C3: According to the formula, D=diag{d ₁₁ ,...,d _nn }, calculate the degree matrix, where,

为关联矩阵对应的无向边的权重；D is the degree matrix; diag{} is the diagonal matrix construction function; d _ii is the degree matrix element, and

is the weight of the undirected edge corresponding to the correlation matrix;

C4: For each node on the boundary, mark the label value of the node according to the boundary prior;

C5: Using the formula, f:X→R ^m , calculate the sorting weight corresponding to the image to be detected, wherein,

f is the sorting function, and f=[f ₁ ,...,f _n ] ^T ; f ₁ is the sorting value of the i-th node; f _n is the sorting value of the n-th node; n is the number of nodes; let y =[y ₁ , y ₂ ,...y _n ] ^T represents the label vector, the label value of the seed point is 1, and the label value of the remaining nodes is 0; X is the feature matrix corresponding to the input image; R is the real number space; R ^m is the m-dimensional real number space; m is the space dimension; y is the vector composed of the label values of all seed nodes;

计算闭合解，其中，C6: Using the sorting function formula,

Compute the closed solution, where,

为求解函数最小值自变量函数；∑为求和函数；f_i为第i个节点的排序值；f_j为第j个节点的排序值；y_i为第i个节点的标签值；

为所述无向边的权重；d_ii为度矩阵中第i行i列的元素；d_jj为度矩阵中第j行j列的元素；μ为平衡参数；f* is the sorting function;

is the independent variable function of the minimum value of the solution function; ∑ is the summation function; f _i is the sorting value of the i-th node; f _j is the sorting value of the j-th node; y _i is the label value of the i-th node;

is the weight of the undirected edge; d _ii is the element in the i-th row and i-column in the degree matrix; d _jj is the element in the j-th row and j column in the degree matrix; μ is the balance parameter;

获取非归一化解，其中，C7: Using the formula according to the closed solution,

to obtain a non-normalized solution, where,

D is the degree matrix; W ² is the second correlation matrix; S is the normalization matrix of W ² ;

分别计算每个节点与四个边界上的背景种子点之间的相关性，得到四种情况下每个节点的背景概率值f，其中，λ为预设参数；C8: Using the formula,

Calculate the correlation between each node and the background seed points on the four boundaries respectively, and obtain the background probability value f of each node in the four cases, where λ is a preset parameter;

再进行取反得到每个节点的显著值；将四种情况下得到的显著值进行点乘获取初始结果S_MR，作为节点的前景概率值。C9: Normalize the correlation value between each node and the background seed points on the four boundaries to get

Then inversion is performed to obtain the salient value of each node; the salient value obtained in the four cases is dot-multiplied to obtain the initial result S_MR, which is used as the foreground probability value of the node.

Optionally, the second computing module is also used for:

According to the color, texture and position information of the extracted image;

计算每个无向边的权重，构建第一关联矩阵

其中，Using the formula,

Calculate the weight of each undirected edge and construct the first correlation matrix

in,

为第i个节点与第j个节点之间的无向边的权重；i和j为节点的序号，且0≤i,j≤N；v_i为第i个节点的特征描述符，且v_i∈R⁶⁵，v_i＝[x_i,y_i，L_i,a_i,b_i,c_i,ω_i]；(x_i,y_i)表示的是每个超像素节点的中心位置坐标；(L_i,a_i,b_i)表示的是每个超像素节点的在CIE LAB颜色空间中包含的所有像素点的颜色均值；c_i为第i个节点的中心先验信息；ω_i为第i个节点的LBP值；v_j为第j个节点的特征描述符；σ为预设的控制权重平衡的常数。

is the weight of the undirected edge between the ith node and the jth node; i and j are the serial numbers of the nodes, and 0≤i,j≤N; vi is the feature descriptor of the _ith node, and v _i ∈R ⁶⁵ , vi = [x _i , y _i , _Li , a _i , b _i , c _i , ω _i ]; (x _i , y _i ) represents the center position coordinates of each _superpixel node ; (L _i , a _i , b _i ) represents the color mean of all pixels contained in the CIE LAB color space of each superpixel node; c _i is the center prior information of the ith node; ω _i is the LBP value of the i-th node; v _j is the feature descriptor of the j-th node; σ is a preset constant that controls the weight balance.

Compared with the prior art, the present invention has the following advantages:

Applying the embodiments of the present invention, when constructing a graph sorting function, an association parameter between foreground clues and background clues is introduced. Compared with the prior art, when constructing a graph sorting function, the association between foreground clues and background clues is not considered. , more influencing factors are considered, which makes the significance detection result more accurate. At the same time, since the traditional composition-based method ignores the role of image features in the saliency calculation, the linear learning of image features is used to constrain the final saliency value, so as to make full use of composition information and feature information. Further improve the final detection result.

Description of drawings

1 is a schematic flowchart of an image saliency detection method based on correlation constraint graph sorting according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of the principle of an image saliency detection method based on correlation constraint graph sorting provided by an embodiment of the present invention;

3 is a schematic structural diagram of a constructed closed-loop graph model provided by an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an image saliency detection apparatus based on correlation constraint graph ranking according to an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described in detail below. This embodiment is implemented on the premise of the technical solution of the present invention, and provides a detailed implementation manner and a specific operation process, but the protection scope of the present invention is not limited to the following implementation. example.

In order to solve the problems of the prior art, the embodiments of the present invention provide an image saliency detection method and apparatus based on the ordering of correlation constraint graphs. The following first describes the saliency detection of images based on the ordering of correlation constraint graphs provided by the embodiments of the present invention. method is introduced.

FIG. 1 is a schematic flowchart of an image saliency detection method based on correlation constraint graph sorting provided by an embodiment of the present invention; FIG. 2 is a principle of an image saliency detection method based on correlation constraint graph sorting provided by an embodiment of the present invention Schematic diagrams, as shown in Figure 1 and Figure 2, the method includes:

S101: For each image to be detected, use a simple linear iterative clustering SLIC algorithm to perform superpixel segmentation on the image to be detected to obtain non-overlapping superpixel blocks, and then use each non-overlapping superpixel block as The node establishes a closed-loop graph model, and then calculates the prior information of each node;

计算每一个节点的中心先验信息，其中，c_i为第i个节点的中心先验信息；x_i为第i个节点的中心位置横坐标；y_i为第i个节点的中心位置纵坐标；(x₀,y₀)表示的是整幅图像的中心位置坐标；σ₁为平衡参数，用于控制计算位置距离的离散程度；exp()为以自然底数为底数的指数函数；i为节点的个数。Specifically, step S101 may include: A1: for each image to be detected, use the SLIC algorithm to perform superpixel segmentation on the image into N superpixel blocks, and each superpixel is used as a node in the set V; There is an undirected edge corresponding to each node, and each node has an edge between its directly adjacent node and the adjacent adjacent (two-neighbor) node. formula, the connected edges are calculated, and the nodes on the four boundaries are connected to each other, and then the undirected graph model G ¹ =(V, E) is constructed; A2: Using the formula,

Calculate the center prior information of each node, where c _i is the center prior information of the ith node; x _i is the abscissa of the center position of the ith node; y _i is the ordinate of the center position of the ith node ; (x ₀ , y ₀ ) represents the center position coordinates of the entire image; σ ₁ is the balance parameter, used to control the discrete degree of the calculated position distance; exp() is the exponential function with the natural base as the base; i is the the number of nodes.

FIG. 3 is a schematic structural diagram of a constructed closed-loop graph model provided by an embodiment of the present invention. As shown in FIG. 3 , the construction process in FIG. 3 is: after the image to be detected is divided into superpixels, N superpixel blocks are obtained. , take each superpixel block as a node, connect each superpixel with those superpixels adjacent to it in the local area, construct edges, and finally establish a closed-loop graph model G ¹ .

In practical applications, the connection of edges between each node can be divided into the following four cases:

1. There is an edge between each node i and its directly adjacent node j.

2. There is an edge between each node i and its second neighbor (adjacent neighbor) node k.

3. For the adjacent node l of the second neighbor k of each node i, using the formula, dist(k, l)=||x _k -x _l || ² , calculate the color Euclidean distance dist(k, l), if the distance is less than the threshold θ, it is considered that there is also an edge between it and node i, and after finding the connected node, continue to search until all nodes are connected, where dist(k, l) is the lth node and The color Euclidean distance between the kth nodes; x _k is the color value of the kth node; x _l is the color value of the lth node; |||| is the modulo function.

4. Each node located on the four boundaries is connected to each other, forming a closed-loop connection around the image.

By applying the above-mentioned embodiments of the present invention, compared with the closed-loop graph model in the prior art, the third point is added to the constructed closed-loop graph model, which expands the local smoothing range of each superpixel region, and better interprets the model in a certain phase. The superpixel regions in adjacent regions have consistent features, which in turn improves the accuracy of image saliency detection.

S102: Extract the color, texture, position and other information of the input image.

Specifically, it can be based on the color, texture, and position information of the extracted image;

计算每个无向边的权重，构建第一关联矩阵

其中，Reuse the formula,

Calculate the weight of each undirected edge and construct the first correlation matrix

in,

为第i个节点与第j个节点之间的无向边的权重；W¹为第一关联矩阵；i和j为节点的序号，且0≤i,j≤N；v_i为第i个节点的特征描述符，且v_i∈R⁶⁵，v_i＝[x_i,y_i,L_i,a_i,b_i,c_i,ω_i]；(x_i,y_i)表示的是每个超像素节点的中心位置坐标；(L_i,a_i,b_i)表示的是每个超像素节点的在CIE LAB颜色空间中包含的所有像素点的颜色均值；c_i为第i个节点的中心先验信息；ω_i为第i个节点的LBP值；v_j为第j个节点的特征描述符；σ为预设的控制权重平衡的常数；n为超像素块的数量。

is the weight of the undirected edge between the i-th node and the j-th node; W ¹ is the first association matrix; i and j are the serial numbers of the nodes, and 0≤i,j≤N; v _i is the i-th The feature descriptor of the node, and v _i ∈ R ⁶⁵ , v _i =[x _i ,y _i ,L _i ,a _i ,b _i , _ci ,ω _i ]; (x _i ,y _i ) means that each The coordinates of the center position of each superpixel node; (L _i , a _i , b _i ) represents the color mean of all pixels contained in the CIE LAB color space of each superpixel node; c _i is the ith node ω _i is the LBP value of the i-th node; v _j is the feature descriptor of the j-th node; σ is a preset constant to control the weight balance; n is the number of superpixel blocks.

As shown in Figure 2, in practical applications, the color feature of the image uses the CIE LAB (LAB specified by the Commission International Eclairage light standard) color mean of each superpixel area containing pixels; the texture feature uses LBP (Local Binary Patterns, local binary pattern) feature. By calculating the difference in feature combination between two connected superpixel nodes, the weight W ² of the edge between them is calculated.

S103: Use the MR algorithm to obtain the foreground probability value of each node.

Specifically, the step S103 may include:

C1: Use the SLIC algorithm to perform superpixel segmentation on the image to be detected, and obtain n non-overlapping superpixel block sets X={x ₁ ,...x _q ,x _q+1 ,...x _n }, where the first q superpixel blocks Pixel blocks are the labeled query seed points, and the rest are unlabeled superpixel nodes. Then, a closed-loop graphical model G ² =(V,E) is constructed. V represents the set of all nodes, and E represents the set of all undirected edges. Among them, there is an edge between each node and its directly adjacent nodes and adjacent adjacent nodes, and the nodes on the four boundaries are connected to each other.

其中，

为所述第i个超像素和第j个超像素之间边的权重，且

W²为第二关联矩阵；i,j∈V，i为第i个节点的序号；j为第j个节点的序号；c_i为第i个节点在CIE LAB颜色空间中所有像素点的颜色均值；c_j为第j个节点在CIE LAB颜色空间中所有像素点的颜色均值；σ为控制权重平衡的常数，通常与两个节点之间颜色均值的欧氏距离正相关。C2: Construct the second correlation matrix

in,

is the weight of the edge between the ith superpixel and the jth superpixel, and

W ² is the second association matrix; i,j∈V, i is the serial number of the ith node; j is the serial number of the jth node; c _i is the color of all pixels of the ith node in the CIE LAB color space Mean; c _j is the color mean of all pixels in the CIE LAB color space of the jth node; σ is a constant that controls the weight balance, which is usually positively related to the Euclidean distance of the color mean between two nodes.

为关联矩阵对应的无向边的权重。C3: According to the formula, D=diag{d ₁₁ ,...,d _nn }, calculate the degree matrix, where D is the degree matrix; diag{} is the diagonal matrix construction function; d _ii is the degree matrix element, and

is the weight of the undirected edge corresponding to the correlation matrix.

C4: For each node on the boundary, according to the boundary prior, label the label value of the node.

C5: Calculate the sorting weight corresponding to the image to be detected by using the formula, f:X→R ^m , where f is a sorting function, and f=[f ₁ ,...,f _n ] ^T ; f ₁ is the first The sorting value of the node; f _n is the sorting value of the nth node; n is the number of nodes; let y=[y ₁ , y ₂ ,...y _n ] ^T represents the label vector, and the label value of the seed point is 1, The label value of the remaining nodes is 0; X is the feature matrix corresponding to the input image; R is the real number space; R ^m is the m-dimensional real number space; m is the space dimension; y is the vector composed of the label values of all seed nodes;

计算闭合解，其中，f^*为排序函数；

为求解函数最小值自变量函数；∑为求和函数；f_i为第i个节点的排序值；f_j为第j个节点的排序值；y₁为第i个节点的标签值；

为所述无向边的权重；d_ii为度矩阵中第i行i列的元素；d_jj为度矩阵中第j行j列的元素；μ为平衡参数。C6: Using the sorting function formula,

Calculate the closed solution, where f ^* is the sorting function;

is the independent variable function of the minimum value of the solution function; ∑ is the summation function; f _i is the sorting value of the i-th node; f _j is the sorting value of the j-th node; y ₁ is the label value of the i-th node;

is the weight of the undirected edge; d _ii is the element in the i-th row and i-column in the degree matrix; d _jj is the element in the j-th row and j column in the degree matrix; μ is the balance parameter.

获取非归一化解，其中，D为度矩阵；W²为所述第二关联矩阵；S是W²的归一化矩阵。C7: Using the formula according to the closed solution,

Obtain a non-normalized solution, where D is the degree matrix; W ² is the second correlation matrix; S is the normalized matrix of W ² .

The closed solution calculated in step C6 can be:

其中，λ为预设参数；f^*为所述闭合解；I是单位矩阵。

Among them, λ is a preset parameter; f ^* is the closed solution; I is the identity matrix.

According to the closed solution and the degree matrix, the non-normalized solution can be obtained

分别计算每个节点与四个边界上的背景种子点之间的相关性，得到四种情况下每个节点的背景概率值f，其中，λ为预设参数。C8: Using the formula,

The correlation between each node and the background seed points on the four boundaries is calculated separately, and the background probability value f of each node in the four cases is obtained, where λ is a preset parameter.

In practical applications, in image saliency detection, the degree matrix is usually used to replace the identity matrix in the closed solution, and then the above formula is obtained. For example, the calculated correlation value between each node and the background seed points on the four boundaries, that is, the background probability value of each node in the four cases is f.

再进行取反得到每个节点的显著值；将四种情况下得到的显著值进行点乘获取初始结果S_MR，作为前景概率值。C9: Normalize the correlation value between each node and the background seed points on the four boundaries to get

Then inversion is performed to obtain the salient value of each node; the salient value obtained in the four cases is dot-multiplied to obtain the initial result S_MR, which is used as the foreground probability value.

In practical applications, the inverse value of the normalized background probability value is:

Dot multiplication of the significant values obtained in the four cases to obtain the initial result can be:

S _bq (i)=S _t (i)×S _b (i)×S _l (i)×S _r (i).

It should be emphasized that in step S103, the MR algorithm is used to obtain the foreground probability value of each node, and the S_MR (foreground probability value) of the image to be detected obtained in the first stage of the classical graph-based manifold sorting algorithm is used.

Consider each superpixel as a node, connect each superpixel with those superpixels that it belongs to adjacent to in the local area, build edges, and then build a closed-loop graph model G ² . Then the LAB color feature of each superpixel region is extracted, and the edge weight W ² is calculated by calculating the difference in color feature between two connected superpixel nodes. The algorithm is mainly divided into two stages. According to the prior information calculated in step S101, the superpixels on the four boundaries of the image are selected as the background query points Query, and then the manifold sorting algorithm is used to calculate the relationship between each superpixel node and the background query. The correlation between points is calculated, and the probability that each superpixel belongs to the background is calculated, then normalized, and then negated to obtain the initial significant result. In the second stage, the initial results of the first stage are binarized, and then the foreground query point Query is filtered out, and then the manifold sorting algorithm is used to calculate the correlation between each superpixel node and the foreground query point, so as to obtain the pending query points. Detect the S_MR (foreground probability value) of the image.

S104: Use the set of nodes whose foreground probability value is greater than the first preset threshold as the foreground seed point set ind_fore; use the set of nodes whose foreground probability value is less than the second preset threshold as the background seed point set ind_back; the first preset threshold is greater than the second preset threshold.

Specifically, the step S104 may include:

获取第一预设阈值和第二预设阈值；h₁为第一预设阈值；h₂为第二预设阈值；mean为求平均值函数；max为最大值求值函数；D1: Using the formula,

Obtain the first preset threshold and the second preset threshold; _h1 is the first preset threshold; h2 is the _second preset threshold; mean is an average value function; max is a maximum value evaluation function;

获取前景种子点集合ind_fore和背景种子点集合ind_back，其中，ind_fore为前景种子点集合；ind_back为背景种子点集合；θ为预设参数。D2: Using the formula,

Obtain the foreground seed point set ind_fore and the background seed point set ind_back, where ind_fore is the foreground seed point set; ind_back is the background seed point set; θ is a preset parameter.

S105: Calculate the foreground probability S_f of each superpixel node using the model for sorting the relevant constraint graph, and use the foreground probability value S_f as the final significant estimation value S_final.

获取每一个节点的标签值；再利用公式，Y＝(y¹,y²)∈R^m×2，获取每一个节点的标签向量，其中，Specifically, step S105 may include: E1: Calculate the sorting weight corresponding to the image to be detected by using the formula, F:X→R ⁿ , where F is a sorting function; F _i represents the sorting value of the ith node , and F=(f, g); f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background; E2: Using the formula,

Obtain the label value of each node; then use the formula, Y=(y ¹ , y ² )∈R ^m×2 , obtain the label vector of each node, where,

Y is the label vector of each node; y ¹ is the label value of the node belonging to the foreground; y ² is the label value of the node belonging to the background; R ^m×2 is a 2m-dimensional real number space.

E3: The model formula of the constructed related constraint graph ordering is,

其中，F^*为闭合解；W_ij为第i个节点和第j个节点之间无向边的权重对应的第一关联矩阵；F_i为第i个节点的排序值；F_j为第j个节点的排序值；d_i为度矩阵元素；f_i为第i个节点的前景概率；g_i为第i个节点的背景概率；w_i为第i个节点的特征权重；x_i为第i个节点的特征；b_i为偏置参数；β₁为对前景概率的线性约束系数；β₂为对背景概率的线性约束系数；E4：对使用所述E3步骤的排序模型对前景概率求偏导得到显著性值。

Among them, F ^* is the closed solution; W _ij is the first association matrix corresponding to the weight of the undirected edge between the i-th node and the j-th node; F _i is the sorting value of the i-th node; F _j is the j-th node rank value of nodes; d _i is the degree matrix element; f _i is the foreground probability of the ith node; _gi is the background probability of the ith node; w _i is the feature weight of the ith node; x _i is the ith node Features of i nodes; b _i is the bias parameter; β ₁ is the linear constraint coefficient on the foreground probability; β ₂ is the linear constraint coefficient on the background probability; E4: For the ranking model using the E3 step, the foreground probability is calculated Partial derivative to get significance value.

In the formula in step E3, the first polynomial is a smooth term, because the surrounding area of the area with a certain characteristic in the image also has similar characteristics to it, that is, it is considered that the ordering between nodes in adjacent local areas The scores are as similar as possible, so a smoothing term can be added; the second term is a fitting term, so that the difference between our final calculated ranking value and the initial label value we give is as small as possible; the third term is a pair of The constraint items of f and g are designed to make the correlation between the calculated f and g as small as possible; the fourth and fifth items are linear constraints on f and g respectively, and the linear learning of image features is used to The final significant value is constrained.

In practice, the E4 steps can include:

1) Simplify the model formula of the constructed related constraint graph sorting, and obtain the optimized formula as,

2), the optimal solution of b and W can be obtained by fixing f, so there are:

和

and

1是全为1的向量。I∈R是单位矩阵，所以可得到

in,

1 is a vector of all ones. I∈R is the identity matrix, so we can get

in,

3), the solution problem of the model of the related constraint graph sorting is written as the following formula:

J=Tr[F ^T A ^* F-μF ^T Y]+λf ^T g+β ₁ ||X ^T W _f +b _f 1-f|| ² +β ₂ ||X ^T W _g +b _g 1- g|| ² , where,

A ^* =(1+μ)DW.

4), replace F in the above formula with F=(f, g), Y=(y ₁ , y ₂ ) and solve the simplification to get the formula,

5), derivation of f in step 4), obtain the following results:

6), to the derivation of g in 4) step, obtain the following results:

7), from 5) and 6) can be obtained:

8), can be calculated according to the formula of 7):

f ^* = μ(λ ² I-4(A ^* ) ² -2A ^* β ₁ B - 2β ₂ BA ^* -β ₂ β ₁ B ² ) ^-1 (λy ₂ -2A ^* y ₁ -β ₂ By ₁ ) ;

Take f ^* as the final saliency estimate S_final.

Applying the embodiment shown in FIG. 1 of the present invention, when constructing the graph sorting function, the correlation parameters between the foreground clues and the background clues are introduced, and the relationship between each superpixel node and a given foreground query point and background query point is calculated at the same time. When the correlation is determined, a correlation constraint is added to reduce the correlation between the obtained foreground probability value and the background probability value. Compared with the prior art, when constructing the graph sorting function, the foreground clues and background are not considered. The association between clues takes more influencing factors into account, which in turn makes the saliency detection results more accurate. At the same time, since the traditional composition-based method ignores the role of image features in the saliency calculation, the linear learning of image features is used to constrain the final saliency value, so as to make full use of composition information and feature information. Further improve the final detection result.

Corresponding to an image saliency detection method based on correlation constraint graph ranking provided by an embodiment of the present invention, an embodiment of the present invention further provides an image saliency detection device based on correlation constraint graph ranking.

FIG. 4 is a schematic structural diagram of an image saliency detection apparatus based on correlation constraint graph ranking according to an embodiment of the present invention. As shown in FIG. 4 , the apparatus includes:

The first calculation module 401 is used to perform superpixel segmentation on the image to be detected using a simple linear iterative clustering SLIC algorithm for each image to be detected, to obtain non-overlapping superpixel blocks, and then divide each of the non-overlapping superpixels. The overlapping superpixel blocks are used as nodes to establish a closed-loop graph model, and then the prior information of each node is calculated;

The input module 402 is used to extract information such as color, texture, and position of the input image;

The second calculation module 403 is used to obtain the foreground probability value of each node by using the MR algorithm;

The first setting module 404 is used to use the set of nodes whose foreground probability value is greater than the first preset threshold as the foreground seed point set ind_fore; the set of nodes whose foreground probability value is less than the second preset threshold is used as the background seed point set ind_back; The first preset threshold is greater than the second preset threshold;

The second setting module 405 is configured to calculate the foreground probability S_f and the background probability S_g of each superpixel node by using the model for sorting the related constraint graph, and use the foreground probability value S_f as the final significant estimated value S_final.

Applying the embodiment shown in FIG. 4 of the present invention, when constructing a graph sorting function, the correlation parameters between foreground clues and background clues are introduced, and the relationship between each superpixel node and a given foreground query point and background query is calculated simultaneously. When the correlation is determined, a correlation constraint is added to reduce the correlation between the obtained foreground probability value and the background probability value. Compared with the prior art, when constructing the graph sorting function, the foreground clues and background are not considered. The association between clues takes more influencing factors into account, which in turn makes the saliency detection results more accurate. At the same time, since the traditional composition-based method ignores the role of image features in the saliency calculation, the linear learning of image features is used to constrain the final saliency value, so as to make full use of composition information and feature information. Further improve the final detection result.

In a specific implementation manner of the embodiment of the present invention, the first computing module 404 is further configured to:

A1: For each image to be detected, use the SLIC algorithm to divide the image into N superpixel blocks, and each superpixel is used as a node in the set V; then obtain the undirected edge corresponding to each node , and then construct an undirected graph model G ¹ =(V, E);

计算每一个节点的中心先验信息，其中，A2: Using the formula,

Calculate the center prior information of each node, where,

c _i is the center prior information of the ith node; _xi is the abscissa of the center position of the ith node; y _i is the ordinate of the center position of the ith node; (x ₀ , y ₀ ) represents the integer is the coordinate of the center position of the image; σ ₁ is the balance parameter, which is used to control the discrete degree of the calculated position distance; exp() is the exponential function with the natural base as the base; i is the number of nodes.

In a specific implementation manner of the embodiment of the present invention, the second computing module 403 is further configured to:

C1: Obtain the weight of each undirected edge of each node in the MR algorithm;

其中，C2: According to the weight of each of the undirected edges, construct the second correlation matrix of the MR algorithm

in,

为所述第i个超像素和第j个超像素之间边的权重，且

W²为第二关联矩阵；i,j∈V，i为第i个节点的序号；j为第j个节点的序号；c_i为第i个节点在CIE LAB颜色空间中所有像素点的颜色均值；c_j为第j个节点在CIELAB颜色空间中所有像素点的颜色均值；σ为控制权重平衡的常数；

is the weight of the edge between the ith superpixel and the jth superpixel, and

W ² is the second association matrix; i,j∈V, i is the serial number of the ith node; j is the serial number of the jth node; c _i is the color of all pixels of the ith node in the CIE LAB color space Mean; c _j is the color mean of all pixels of the jth node in the CIELAB color space; σ is a constant that controls the weight balance;

C3: According to the formula, D=diag{d ₁₁ ,...,d _nn }, calculate the degree matrix, where,

为关联矩阵对应的无向边的权重；D is the degree matrix; diag{} is the diagonal matrix construction function; d _ii is the degree matrix element, and

is the weight of the undirected edge corresponding to the correlation matrix;

C4: For each node on the boundary, mark the label value of the node according to the boundary prior;

C5: Using the formula, f:X→R ^m , calculate the sorting weight corresponding to the image to be detected, wherein,

f is the sorting function, and f=[f ₁ ,...,f _n ] ^T ; f ₁ is the sorting value of the first node; f _n is the sorting value of the n-th node; n is the number of nodes; let y =[y ₁ , y ₂ ,...y _n ] ^T represents the label vector, the label value of the seed point is 1, and the label value of the other nodes is 0; X is the feature matrix corresponding to the input image; R is the real number space; R ^m is the m-dimensional real number space; m is the space dimension; y is the vector composed of the label values of all seed nodes;

计算闭合解，其中，C6: Using the sorting function formula,

Compute the closed solution, where,

为求解函数最小值自变量函数；∑为求和函数；f_i为第i个节点的排序值；f_j为第j个节点的排序值；y_i为第i个节点的标签值；

为所述无向边的权重；d_ii为度矩阵中第i行i列的元素；d_jj为度矩阵中第j行j列的元素；μ为平衡参数；f ^* is the sorting function;

is the independent variable function of the minimum value of the solution function; ∑ is the summation function; f _i is the sorting value of the i-th node; f _j is the sorting value of the j-th node; y _i is the label value of the i-th node;

is the weight of the undirected edge; d _ii is the element in the i-th row and i-column in the degree matrix; d _jj is the element in the j-th row and j column in the degree matrix; μ is the balance parameter;

获取非归一化解，其中，D为度矩阵；W²为所述第二关联矩阵；S是W²的归一化矩阵；C7: Using the formula according to the closed solution,

Obtain a non-normalized solution, wherein D is the degree matrix; W ² is the second correlation matrix; S is the normalized matrix of W ² ;

分别计算每个节点与四个边界上的背景种子点之间的相关性，得到四种情况下每个节点的背景概率值f，其中，λ为预设参数；C8: Using the formula,

Calculate the correlation between each node and the background seed points on the four boundaries respectively, and obtain the background probability value f of each node in the four cases, where λ is a preset parameter;

再进行取反得到每个节点的显著值；将四种情况下得到的显著值进行点乘获取初始结果S_MR，作为节点的前景概率值。C9: Normalize the correlation value between each node and the background seed points on the four boundaries to get

Then inversion is performed to obtain the salient value of each node; the salient value obtained in the four cases is dot-multiplied to obtain the initial result S_MR, which is used as the foreground probability value of the node.

In a specific implementation manner of the embodiment of the present invention, the second computing module 403 is further configured to:

计算每个无向边的权重，构建第一关联矩阵

其中，Using the formula,

Calculate the weight of each undirected edge and construct the first correlation matrix

in,

为第i个节点与第j个节点之间的无向边的权重；i和j为节点的序号，且0≤i,j≤N；v_i为第i个节点的特征描述符，且v_i∈R⁶⁵，v_i＝[x_i,y_i,L_i,a_i,b_i,c_i,ω_i]；(x_i,y_i)表示的是每个超像素节点的中心位置坐标；(L_i,a_i,b_i)表示的是每个超像素节点的在CIE LAB颜色空间中包含的所有像素点的颜色均值；c_i为第i个节点的中心先验信息；ω_i为第i个节点的LBP值；v_j为第j个节点的特征描述符；σ为预设的控制权重平衡的常数。

is the weight of the undirected edge between the ith node and the jth node; i and j are the serial numbers of the nodes, and 0≤i,j≤N; vi is the feature descriptor of the _ith node, and v _i ∈R ⁶⁵ , v _i =[x _i ,y _i ,L _i ,a _i , _bi , _ci ,ω _i ]; (x _i ,y _i ) represents the center position coordinates of each superpixel node ; (L _i , a _i , b _i ) represents the color mean of all pixels contained in the CIE LAB color space of each superpixel node; c _i is the center prior information of the ith node; ω _i is the LBP value of the i-th node; v _j is the feature descriptor of the j-th node; σ is a preset constant that controls the weight balance.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (8)

1. An image saliency detection method based on relevance constraint graph sorting is characterized by comprising the following steps:

a: aiming at each image to be detected, performing superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain non-overlapping superpixel blocks, then establishing a closed-loop graph model by taking each non-overlapping superpixel block as a node, and further calculating the central priori information of each node;

b: extracting color, texture and position information of an input image;

c: obtaining a foreground probability value of each node by using a significance detection algorithm based on manifold sorting of a graph;

d: taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;

e: calculating by using a model of related constraint graph sequencing to obtain a foreground probability S _ f of each super pixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final;

and step C, comprising:

c1: acquiring the weight of each undirected edge of each node in a significance detection algorithm based on manifold sorting of a graph;

c2: constructing a second incidence matrix of the significance detection algorithm based on manifold sorting of the graph according to the weight of each undirected edge

Wherein,

is the weight of the edge between the ith super pixel and the jth super pixel,and is

W²Is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. C_iThe color mean value of all pixel points of the ith node in the CIE LAB color space; c. C_jThe color mean value of all pixel points of the jth node in the CIE LAB color space is obtained; sigma is a constant for controlling weight balance;

c3: according to the formula, D ═ diag { D ═ D₁₁,…,d_nn-calculating a degree matrix, wherein,

d is a degree matrix; diag { } is a diagonal matrix construction function; d_iiIs a degree matrix element, and

weights of the non-directional edges corresponding to the incidence matrix;

c4: for each node on the boundary, marking the marking value of the node according to boundary prior;

c5: using the formula, f: X → R^mCalculating the corresponding sorting weight of the image to be detected, wherein,

f is the ranking function, and f ═ f₁,…,f_n]^T；f₁Is the ranking value of the 1 st node; f. of_nIs the ranking value of the nth node; n is the number of nodes; let y be [ y₁,y₂,…y_n]^TRepresenting a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; r^mIs m-dimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;

c6: by using the formula of the ranking function,

a closed solution is calculated in which, among other things,

f^*is a sorting function;

solving a function minimum independent variable function; sigma is a summation function; f. of_iIs the ranking value of the ith node; f. of_jIs the ranking value of the jth node; y is_iIs the label value of the ith node;

a weight of the undirected edge; d_iiThe element of the ith row and the ith column in the degree matrix; d_jjThe element of j row and j column in the degree matrix; mu is a balance parameter;

c7: using a formula in accordance with the closed solution,

obtaining a non-normalized solution, wherein,

d is a degree matrix; w²Is the second incidence matrix; s is W²The normalization matrix of (a);

c8: by means of the formula (I) and (II),

respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;

c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation values

Then carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.

2. The method for detecting the significance of the image based on the related constraint map sorting as claimed in claim 1, wherein the step A comprises:

a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a non-directional edge corresponding to each node is obtained, and a non-directional graph model G is further constructed¹＝(V,E)；

A2: by means of the formula (I) and (II),

central prior information is calculated for each node, wherein,

c_icentral prior information of the ith node; x is the number of_iThe abscissa of the central position of the ith node is; y is_iIs the longitudinal coordinate of the central position of the ith node; (x)₀,y₀) The coordinates of the center position of the whole image are shown; sigma₁The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.

3. The method for detecting the saliency of images based on the ordering of related constraint maps according to claim 2, characterized in that the undirected edges are obtained by:

for the neighboring node l of the two neighbors k of each node i, the formula is used, dist (k, l) | | x_k-x_l||²Calculating the Euclidean distance between the color of the color k and the color k; if the Euclidean distance of the color is smaller than the threshold value theta, connecting a non-directional edge between the node l and the node i, and continuing searching after finding the connected nodes until all the nodes are connected, wherein dist (k, l) is the Euclidean distance of the color between the ith node and the kth node; x is the number of_kIs the color value of the kth node; x is the number of_lIs the color value of the l node; and | | is a modulo function.

4. The method for detecting the saliency of images based on the ordering of correlation constraint maps according to claim 1, wherein the step B comprises:

according to the color, texture and position information of the extracted image; by means of the formula (I) and (II),

calculating the weight of each undirected edge to construct a first incidence matrix

Wherein,

a weight of an undirected edge between the ith node and the jth node; w¹Is a first incidence matrix; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. of_iIs a feature descriptor of the ith node, and v_i∈R⁶⁵，v_i＝[x_i,y_i,L_i,a_i,b_i,c_i,ω_i]；(x_i,y_i) Representing the center position coordinates of each superpixel node; (L)_i,a_i,b_i) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. C_iCentral prior information of the ith node; omega_iIs the LBP value of the ith node; v. of_jA feature descriptor for the jth node; sigma is a preset constant for controlling weight balance; n is the number of superpixel blocks.

5. The method for detecting the significance of the image based on the related constraint map sorting as claimed in claim 1, wherein the step D comprises:

d1: by means of the formula (I) and (II),

acquiring a first preset threshold and a second preset threshold; wherein,

h₁is a first preset threshold value;h₂is a second preset threshold; mean is an averaging function; max is a maximum evaluation function;

d2: by means of the formula (I) and (II),

obtaining a foreground seed point set ind _ form and a background seed point set ind _ back, wherein,

ind _ form is a foreground seed point set; ind _ back is a background seed point set; theta is a preset parameter.

6. The method for detecting the significance of the image based on the related constraint map ordering according to claim 1, wherein the step E comprises the following steps:

e1: using the formula, F: X → RⁿCalculating the corresponding sorting weight of the image to be detected, wherein,

f is a sorting function; f_iDenoted is the rank value of the ith node, and F ═ F, g; f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background;

e2: by means of the formula (I) and (II),

acquiring a label value of each node; using the formula, Y ═ Y¹,y²)∈R^m×2And obtaining a label vector of each node, wherein,

y is a label vector of each node; y is¹A label value for which the node belongs to the foreground; y is²A label value for which the node belongs to a background; r^m×2Is a real space with 2m dimension;

e3: the model formula of the constructed related constraint graph ordering is as follows,

wherein,

F^*is a closed solution; w_ijAs a non-directional edge between the ith node and the jth nodeA first incidence matrix corresponding to the weight of (a); f_iIs the ranking value of the ith node; f_jIs the ranking value of the jth node; d_iIs a degree matrix element; f. of_iForeground probability of the ith node; g_iThe background probability of the ith node; w is a_iThe characteristic weight of the ith node; x is the number of_iIs the characteristic of the ith node; b_iIs a bias parameter; beta is a₁Linear constraint coefficients to the foreground probability; beta is a₂Linear constraint coefficient to background probability;

e4: and (4) carrying out partial derivation on the foreground probability by using the sequencing model of the E3 step to obtain a significance value.

7. An apparatus for detecting image saliency based on relevance constraint map ordering, the apparatus comprising:

the first calculation module is used for carrying out superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm aiming at each image to be detected to obtain non-overlapping superpixel blocks, then establishing a closed-loop graph model by taking each non-overlapping superpixel block as a node, and further calculating the central prior information of each node;

the input module is used for extracting color, texture and position information of an input image;

the second calculation module is used for acquiring the foreground probability value of each node by utilizing a significance detection algorithm based on manifold sorting of the graph;

the first setting module is used for taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;

the second setting module is used for calculating a foreground probability S _ f and a background probability S _ g of each super-pixel node by using a model ordered by a related constraint graph, and using the foreground probability value S _ f as a final significant estimation value S _ final;

the second computing module is further configured to:

c1: acquiring the weight of each undirected edge of each node in a significance detection algorithm based on manifold sorting of a graph;

c2: constructing a second incidence matrix of the significance detection algorithm based on manifold sorting of the graph according to the weight of each undirected edge

Wherein,

is the weight of the edge between the ith and jth superpixels, and

W²is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. C_iThe color mean value of all pixel points of the ith node in the CIE LAB color space; c. C_jThe color mean value of all pixel points of the jth node in the CIE LAB color space is obtained; sigma is a constant for controlling weight balance;

c3: according to the formula, D ═ diag { D ═ D₁₁,…,d_nn-calculating a degree matrix, wherein,

d is a degree matrix; diag { } is a diagonal matrix construction function; d_iiIs a degree matrix element, and

the weight of the non-directional edge corresponding to the second incidence matrix;

c4: for each node on the boundary, marking the marking value of the node according to boundary prior;

c5: using the formula, f: X → R^mCalculating the corresponding sorting weight of the image to be detected, wherein,

f is the ranking function, and f ═ f₁,…,f_n]^T；f₁Is the ranking value of the 1 st node; f. of_nIs the ranking value of the nth node; n is the number of nodes; let y be [ y₁,y₂,…y_n]^TRepresenting a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; r^mIs m-dimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;

c6: by using the formula of the ranking function,

a closed solution is calculated in which, among other things,

f^*is a sorting function;

solving a function minimum independent variable function; sigma is a summation function; f. of_iIs the ranking value of the ith node; f. of_jIs the ranking value of the jth node; y is_iIs the label value of the ith node; y is_nThe weight of the nth node;

a weight of the undirected edge; d_iiThe element of the ith row and the ith column in the degree matrix; d_jjThe element of j row and j column in the degree matrix; mu is a balance parameter;

c7: using a formula in accordance with the closed solution,

obtaining a non-normalized solution, wherein,

d is a degree matrix; w²Is the second incidence matrix; s is W²The normalization matrix of (a);

c8: by means of the formula (I) and (II),

respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;

c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation values

Then carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.

8. The apparatus according to claim 7, wherein the first computing module is further configured to:

a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a non-directional edge corresponding to each node is obtained, and a non-directional graph model G is further constructed¹＝(V,E)；

A2: by means of the formula (I) and (II),

central prior information is calculated for each node, wherein,

c_icentral prior information of the ith node; x is the number of_iThe abscissa of the central position of the ith node is; y is_iIs the longitudinal coordinate of the central position of the ith node; (x)₀,y₀) The coordinates of the center position of the whole image are shown; sigma₁The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.

Images