CN106056579A - Saliency detection method based on background contrast - Google Patents

Abstract

The invention relates to a significance detection method based on background contrast. It includes six steps: input image I, over-segment image I, calculate background saliency, calculate image color contrast saliency, calculate image compactness saliency, and obtain final saliency. The over-segmentation in step 2 includes converting the input image I into a visually uniform CIELAB color space, and convolving the above-mentioned conversion result with a 17-dimensional filter. This filter consists of a Gaussian filter, a Gaussian derivative filter, The Gaussian filter of the Laplacian operator is used, and the Euclidean distance K-means clustering algorithm is used to perform unsupervised clustering of the image. Each pixel is assigned to the nearest cluster center, and finally an over-segmented image is generated. Compared with the method of calculating the contrast of the whole image, the method of the present invention is more robust, the generated saliency map is the closest to the result of human eye identification, and is very effective in natural scene image processing.

Description

A Saliency Detection Method Based on Background Contrast

technical field

The invention relates to an image analysis technology in the field of image data processing, in particular to a background contrast-based significance detection method.

Background technique

Saliency detection is a challenging problem in the field of computer vision research, and is also an important task in many applications, such as object recognition, image coding, image editing, image segmentation, and video tracking, etc. Through extensive efforts, many successful and notable assays have been developed,

They are roughly divided into two types: top-down (supervised) methods and bottom-up (unsupervised) methods. The former often describe salient information with the visual knowledge built from the training process, and then use this knowledge to perform saliency detection in the test image; the latter is usually based on the degree of difference from the surrounding neighborhood without any salient regions or object priorities Determine the saliency of a pixel.

Recent studies have shown that computational models based on bottom-up approaches are quite successful and well suited for scaling to large datasets. Perceptual studies and results from previous methods suggest that the most influential factor for bottom-up visual saliency is contrast. Niloy J. Mitra et al. proposed a salient region detection method based on global contrast, that is, histogram-based contrast (HC for short) and region contrast (RC for short) based on spatial information enhancement. HC methods are efficient and produce results with fine details; RC methods generate spatially enhanced high-quality saliency maps, but are less computationally efficient. And this method has weak processing ability for high-texture images. YANG et al. proposed a salient object detection method based on global color contrast. Firstly, the global color contrast feature is extracted, and a binary saliency mask is obtained by combining the saliency map with the salient object detection algorithm framework of global color contrast. Finally, The minimum circumscribed rectangle containing the salient objects is obtained through the calculation of the area delineation sub-calculation.

Both top-down and bottom-up methods tend to rely only on the saliency of the region center surround contrast or the global contrast. Algorithms that rely on the contrast around the center of the region are often imprecise, while global contrast saliency detection methods ignore the spatial relationship between parts of the image, and object detection at the edge of the image is incomplete.

Contents of the invention

The technical problem to be solved by the present invention is to solve the problem that the saliency algorithm is not refined or the saliency at the edge of the image is not refined because the saliency of the above-mentioned top-down and bottom-up methods only depends on the saliency of the area center surround contrast or the global contrast. The problem of incomplete object detection.

In view of this, the contrast based on the background region also plays an important role in this process, and the present invention proposes a method for saliency detection based on the background contrast, which includes the following steps:

Step 1: input image I;

Step 2: over-segment image I;

Step 3: Through the formula Calculate background saliency S _BS (r _i ), where Obtained by calculating the number of pixels on the x-axis and y-axis respectively;

Step 4: Through the formula Calculate the image color contrast saliency S _CCS (r _i ), c ^R ( ), c ^G ( ), c ^B ( ) are the average colors in the RGB channel;

Step 5: Through the formula Compute the image compactness saliency S _CS (r _i ), where L is the number of pixels in r _i , are the pixel number distribution of r _i on the x-axis and y-axis respectively, The parameter μ _i is the center of the kernel function, and σ _i is set to min{W,H};

Step 6: S _BS (r _i ), S _CCS (r _i ), and S _CS (r _i ) respectively obtained according to the above steps 3, 4, and 5, through the formula S(r _i )=S' _CCS (r _i ) S' _CS (r _i ) to get the final significance, where S' _CCS (r _i ) = S _CCS (r _i ) S _FS (r _i ), S'_CS' = S _CS (r _i ) S _FS (r _i ), S _FS (r _i )=exp{−α·S _BS (r _i )}.

Further, the above step 2 includes the following steps:

Step 2-1: Convert the input image I into a visually uniform CIELAB color space;

Step 2-2: Convolute the result of step 2-1 with a 17-dimensional filter, which consists of a Gaussian filter, a Gaussian derivative filter, and a Gaussian filter of the Laplacian operator;

Step 2-3: Unsupervised clustering of images using Euclidean distance K-means clustering algorithm;

Steps 2-4: Each pixel is assigned to the nearest cluster center to produce an over-segmented image.

Beneficial effect:

1. In the present invention, the candidate region can be automatically generated according to the spatial distribution of the background, and the salience of the background can be encoded by using the spatial distribution of the background region. Moreover, color contrast and compactness are adopted to encode foreground saliency.

2. The present invention uses the contrast relative to the background, which is more robust than the method of calculating the contrast relative to the entire image, and the generated saliency map is the closest to the result of human eye identification.

3. The method of the present invention is tested on the MSRA1000 data set, and the test results prove that it is very effective in natural scene image processing. In addition, according to the precision-recall rate curve and F value diagram obtained by the present invention, it is also concluded that the present invention is superior to other 12 kinds of significance detection methods in the field. At the same time, compared with the results expressed by human eyes, the effect of the present invention is also very effective.

Description of drawings

Fig. 1 is a flow chart of the method of the present invention.

Fig. 2 is the salience figure obtained by each step in the present invention.

Fig. 3 is a background saliency map in the present invention.

Fig. 4 is a color contrast saliency map in the present invention.

Fig. 5 is a saliency map of compactness in the present invention.

Figure 6 is the recall rate curve and F value diagram.

detailed description

The present invention will be further described in detail below in conjunction with the accompanying drawings.

The meanings of English abbreviations used in the present invention are as follows:

BS means background saliency (Background Saliency);

CCS stands for Color Contrast Saliency;

CS stands for Compactness Saliency;

FS stands for Foreground Saliency.

The technical solution adopted by the present invention to solve the technical problem is: a bottom-up saliency detection framework is introduced, and the comparison with the background candidate area is used. The input image is first over-segmented into many uniform segmentation blocks. Since the background area often occupies a relatively large area in the image and extends to the edge of the image, the saliency map of the background can be obtained, as shown in Figure 2(c), the background saliency value of the area far from the image center is high. In addition to encoding background salience by utilizing the spatial distribution of background regions, the present invention also uses color contrast and compactness to encode foreground saliency. After the background candidate area is abstracted according to the saliency map of the background, as shown in Figure 2(d), by calculating the contrast of each segmented block relative to its background area, the color contrast saliency value is finally obtained. The compactness saliency value is generated according to the Gestalt rule, indicating that regions with regular shapes are often more significant, and vice versa. Combining these three saliency maps, the final saliency map can be obtained, as shown in Figure 2(f).

The method flow consists of the following steps:

Step 1: input image I;

Step 2: over-segment image I;

Step 3: Through the formula Calculate background saliency S _BS (r _i ), where Obtained by calculating the number of pixels on the x-axis and y-axis respectively;

Step 4: Through the formula Calculate the image color contrast saliency S _CCS (r _i ), c ^R ( ), c ^G ( ), c ^B ( ) are the average colors in the RGB channel;

Step 5: Through the formula Compute the image compactness saliency S _CS (r _i ), where L is the number of pixels in r _i , are the pixel number distribution of r _i on the x-axis and y-axis respectively, The parameter μ _i is the center of the kernel function, and σ _i is set to min{W,H};

Step 6: S _BS (r _i ), S _CCS (r _i ), and S _CS (r _i ) respectively obtained according to the above steps 3, 4, and 5, through the formula S(r _i )=S' _CCS (r _i ) S' _CS (r _i ) to get the final significance, where S' _CCS (r _i ) = S _CCS (r _i ) S _FS (r _i ), S'_CS' = S _CS (r _i ) S _FS (r _i ), S _FS (r _i )=exp{−α·S _BS (r _i )}.

Further, the above step 2 includes the following steps:

Step 2-1: Convert the input image I into a visually uniform CIELAB color space;

Step 2-2: Convolute the result of step 2-1 with a 17-dimensional filter, which consists of a Gaussian filter, a Gaussian derivative filter, and a Gaussian filter of the Laplacian operator;

Step 2-3: Unsupervised clustering of images using Euclidean distance K-means clustering algorithm;

Steps 2-4: Each pixel is assigned to the nearest cluster center to produce an over-segmented image.

As shown in Figure 2(b), the first step is to generate a series of segmentation blocks based on the raw pixel intensities of the original image. Among them, smaller uniform segmentation blocks, regions with sharp boundaries in the image, are useful for detecting salient objects.

The input image is first converted into a visually uniform CIELAB color space, and then convolved with a 17-dimensional filter consisting of a Gaussian filter, a Gaussian derivative filter, and a Gaussian filter for the Laplacian operator. Then use the Euclidean distance K-means clustering algorithm for unsupervised clustering. Finally, each pixel is assigned to the nearest cluster center to produce the over-segmented image. Although these segmented regions are often highly irregular in size and shape, an advantage of the present invention is that it can aggregate larger homogeneous regions of similar appearance while subdividing heterogeneous regions into many smaller of blocks.

The specific implementation process of obtaining the saliency value of the image is as follows:

(1) Acquisition of background saliency map

a. Given an input image I of size W×H as shown in Figure 3(a), W and H are the width and height of the image, respectively. First establish a coordinate system, and establish a coordinate system with the upper left point of the image as the coordinate origin. The horizontal and vertical directions represent: x-axis and y-axis respectively, and the band-stop filter F ^h (x, W) is defined on the x-axis:

${\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; \&tau;</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&eta;</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; W</span>}}{\mathrm{<span\; class="notranslate">\; \&eta;</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}$

x is the x-axis coordinate, the parameter τ controls the upper limit, and η controls the shape of the bandstop filter.

b. Define Fv( ^y ,H) similarly on the y-axis.

c. As shown in FIG. 3(c), there are L pixels on (x, y) in the region _ri . _Compute the normalized spatial distribution of ri It is obtained by calculating the number of pixels on the x-axis and y-axis respectively.

d. Background saliency is defined as the weighted average filtered response of all pixels in _ri :

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}^{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}^{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>$

Therefore, those regions that are far away from the image center and are large and uniform will be assigned more saliency values than the central region, as shown in Fig. 3(d).

(2) Acquisition of color saliency map

It often happens that the salient object is not exactly in the center of the image, but the color of the target area is still very different relative to the whole scene. Therefore, in the present invention, the calculation is performed based on the contrast of the candidate background area, rather than the calculation of the saliency value based on the entire image. According to the BS in the above step 3, the one with higher saliency and larger size is selected as the candidate background region.

Select {B ₁ ,B ₂ ,...B _N } as the candidate background area. Then calculate the color difference significant (CCS) of the segment r _i in the RGB color space:

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\sqrt{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; G</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; R</span>}}}$

in, c ^R (·), c ^G (·), c ^B (·) are the average colors in the RGB channels, respectively. Figure 4 shows the calculation results of CCS.

(3) Acquisition of compact saliency map

Background regions are distributed throughout the image, exhibiting high spatial heterogeneity, while foreground objects are usually more compact and have regular shapes. According to such properties, the present invention defines compact saliency (CS):

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}^{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}^{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}^{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>$

Among them, L is the number of pixels in r _i , are the pixel number distribution of r _i on the x-axis and y-axis respectively, and N(·) represents the Gaussian kernel function, The parameter μ _i is the center of the kernel function, and σ _i is set to min{W,H}. Figure 5 shows the calculation results of CS

(4) Acquisition of comprehensive saliency map

In the present invention, it is assumed that the three measurements are independent. Initially normalize BS, CCS and CS to [0,1]. In practice, it is found that BS has higher discriminative power when representing the background. Therefore, the present invention uses an exponential function to emphasize the foreground saliency (FS) as:

S _FS (ri )=exp{-α·S _BS ( _ri ) _} , where α is a scaling factor.

Using only CCS and CS may highlight some background regions (as shown in Figure 2), resulting in wrong saliency assignments. In order to make up for this defect, it is modified by multiplying FS in the present invention, to eliminate the influence interference of background:

S' _CCS (r _i )＝S _CCS (r _i )·S _FS (r _i )

S'_CS' = S _CS (r _i ) · S _FS (r _i )

The final saliency map is defined as: S( _ri )=S' _CCS ( _ri )·S' _CS ( _ri )

The saliency map S( _ri ) is normalized to a fixed range [0, 255], and each image pixel of _ri is assigned a saliency value S( _ri ).

Recall and precision are two metrics used to evaluate the quality of the results. The recall rate is the ratio of the number of relevant documents retrieved to the number of all relevant documents in the document library, and measures the recall rate of the retrieval system. Accuracy is the ratio of the number of relevant documents retrieved to the total number of documents retrieved, and measures the precision of the retrieval system. These two measures can be fused into one measure - F-measure. A large number of tests are carried out in the MSRA database with the method of the present invention to obtain a saliency map. Draw the precision of the method of the present invention and other 12 kinds of methods--recall rate curve and F value figure, as shown in Figure 6, find that the method of the present invention is all higher than other 12 kinds of methods no matter aspect recall rate or precision, illustrate The effectiveness of the method of the present invention is superior to other 12 kinds of methods.

Claims (2)

1. A method for detecting significance based on background contrast, characterized in that it comprises the steps: Step 1: input image I; Step 2: over-segment image I; Step 3: Through the formula Calculate background saliency S _BS (r _i ), where Obtained by calculating the number of pixels on the x-axis and y-axis respectively; Step 4: Through the formula Calculate the image color contrast saliency S _CCS (r _i ), c ^G ( ), c ^B ( ) are the average colors in the RGB channel; Step 5: Through the formula Compute the image compactness saliency S _CS (r _i ), where L is the number of pixels in r _i , are the pixel number distribution of r _i on the x-axis and y-axis respectively, The parameter μ _i is the center of the kernel function, and σ _i is set to min{W,H}; Step 6: S _BS (r _i ), S _CCS (r _i ), and S _CS (r _i ) respectively obtained according to the above steps 3, 4, and 5, through the formula S(r _i )=S' _CCS (r _i ) S' _CS (r _i ) to get the final significance, where S' _CCS (r _i ) = S _CCS (r _i ) S _FS (r _i ), S'_CS' = S _CS (r _i ) S _FS (r _i ), S _FS (r _i )=exp{−α·S _BS (r _i )}. 2. a kind of method based on background contrast significance detection according to claim 1, is characterized in that described step 2 comprises the steps: Step 2-1: Convert the input image I into a visually uniform CIELAB color space; Step 2-2: Convolute the result of step 2-1 with a 17-dimensional filter, which consists of a Gaussian filter, a Gaussian derivative filter, and a Gaussian filter of the Laplacian operator; Step 2-3: Unsupervised clustering of images using Euclidean distance K-means clustering algorithm; Steps 2-4: Each pixel is assigned to the nearest cluster center to produce an over-segmented image.