CN114612815A - Improved ViBe indoor real-time foreground detection method and storage medium - Google Patents

Abstract

The invention provides an improved ViBe indoor real-time foreground detection method and a storage medium. On the basis of the ViBe algorithm, the inter-frame change rate calculation and the background update strategy of adjusting random factors based on the inter-frame change rate are added to suppress the band caused by the time-space propagation mechanism. The resulting "ghosting" and hole problems, and the use of light detection based on the rate of change between frames and the perceptual hash algorithm to determine whether there is a light mutation, if there is a light mutation, the background model is rebuilt, which effectively overcomes the ViBe algorithm. various problems. Beneficial effects of the present invention: the method effectively suppresses the "ghost image" and hole phenomenon without sacrificing the real-time performance of the original ViBe algorithm for foreground detection, improves the detection ability of the algorithm to a large extent, and enhances the detection ability of the algorithm. The robustness of illumination changes is more suitable for foreground detection in indoor environments than the ViBe algorithm.

Description

An improved ViBe indoor real-time foreground detection method and storage medium

technical field

The invention relates to the field of moving target detection, in particular to an improved ViBe indoor real-time foreground detection method and a storage medium.

Background technique

In recent years, video surveillance technology has been increasingly used in important fields such as national and social public security, aerospace, and many civil fields. Installing video surveillance systems in unattended areas can effectively eliminate potential safety hazards, give early warnings and avoid accidents. At present, the commonly used intelligent video surveillance algorithms are mainly foreground detection (moving target detection, foreground extraction) algorithms. Foreground detection is a technology for extracting foreground objects from image sequences.

The ViBe algorithm is a foreground detection algorithm that establishes a background model at the pixel level and makes a background difference with the current frame. The algorithm takes advantage of the fact that pixels in the same area have similar pixel values, and only selects the first frame of the video to initialize the background. The ViBe algorithm consists of three steps: background modeling, foreground detection and background model updating. There are also some challenges in the process of specific application: (1) There will be some common problems in the actual detection process, such as "ghosting", holes and other unfavorable phenomena caused by the spatiotemporal propagation mechanism used in the update of the background model; ( 2) Due to the particularity of indoor space, man-made operations such as switching lights, windows, and curtains will cause a large change in the indoor light intensity globally, so the algorithm must be robust to light changes. Therefore, a more comprehensive solution is required.

SUMMARY OF THE INVENTION

The main technical problem to be solved by the present invention is to solve the problem of "ghosting" and holes caused by the traditional ViBe algorithm and the problem of poor robustness to illumination changes. In order to solve the above problems, the present invention provides an improved ViBe indoor real-time foreground detection method, without sacrificing the real-time performance of the original ViBe algorithm for foreground detection, the random factor adjustment strategy based on the rate of change between frames is used to effectively suppress the "ghosting" and hole problems caused by the spatiotemporal propagation mechanism in the background update, and The robustness of the algorithm to light mutation is enhanced by the light mutation detection method based on the rate of change between frames and the perceptual hash algorithm, which is more suitable for foreground detection in indoor environments than the ViBe algorithm.

According to one aspect of the present invention, the present invention provides an improved ViBe indoor real-time foreground detection method, comprising the following steps:

S1: Receive the video stream and perform decoding and preprocessing to obtain the grayscale image of the frame-by-frame video image;

S2: monitor the number of frames of the current frame of video image, determine whether it is the first frame of the video, if so, establish a background model of the foreground detection algorithm and enter S1, otherwise, enter S3;

S3: Calculate the rate of change between frames through the grayscale image of the current frame and the grayscale image of the previous frame;

S4: determine whether a sudden change in illumination occurs, if so, re-create the background model in S2 and enter step S1, otherwise, enter S5;

S5: Obtain the foreground detection result of the video image of the current frame;

S6: Based on the inter-frame change rate obtained in S3, update the background model according to the inter-frame change rate adjustment strategy;

S7: Determine whether there is a next frame of video image, if so, enter S1, otherwise, the detection process ends.

Preferably, in S1, the steps of receiving the video stream, decoding and preprocessing include:

S11: Receive and decode the video stream to obtain a frame-by-frame video image queue;

S12: Perform grayscale conversion on the single-frame video image in the video image queue to obtain a grayscale image of the single-frame video image;

S13: Use Gaussian and median filtering to denoise the grayscale image to obtain a denoised grayscale image.

Preferably, in S2, the steps of establishing the background model include:

S21: Obtain the first frame grayscale image of the video, and construct a sample set with a maximum capacity of N for each pixel;

S22: Each pixel randomly selects the gray value of the pixels in its eight neighborhoods and adds it to the sample set, until the sample set capacity reaches the upper limit;

The expression for the sample set is as follows:

Sample(x,y)={Vi|i=1,2...N}

In the formula, Sample(x, y) is the sample set of pixels, (x, y) is the Cartesian coordinate value of any pixel, Vi is the sample, and N is the number of samples.

Preferably, in S3, the step of calculating the rate of change between frames includes:

S31: Select the grayscale image of the current frame and the grayscale image of the previous frame, perform a difference operation on the pixel points at the same position in the Cartesian coordinate system and take the absolute value to obtain a differential grayscale image. The formula is as follows:

D(x,y)=|fk(x,y)-fk-1(x,y)|

In the formula, D(x,y) is the obtained differential grayscale image, fk(x,y) is the current frame grayscale image, and fk-1(x,y) is the previous frame grayscale image;

S32: Determine whether the absolute value of the grayscale value of each pixel in the differential grayscale image is greater than the threshold T, and if so, modify its grayscale value to 255, otherwise, set it to 0;

S33: The number of pixels with a grayscale value of 255 in the statistical difference grayscale image is num1, the total number of pixels in the image is num2, and the inter-frame change rate P=num1/num2×100% is calculated.

Preferably, in S4, the method for judging whether a sudden change in illumination occurs includes:

S41: Determine whether the value of the inter-frame change rate P calculated in S33 is greater than the threshold Tlight, if so, enter S42, otherwise, determine that no sudden change in illumination occurs;

S42: Use the perceptual hash algorithm to compare the grayscale image of the current frame with the differential grayscale image obtained in S31, and determine whether the Hamming distance of the two images is greater than the threshold Tenter. Light mutation.

Preferably, in S5, the step of obtaining the foreground detection result includes:

S51: For the grayscale image of the current frame, compare the grayscale value of each pixel point x with the N grayscale values in the corresponding sample set Sample(x,y), and determine whether the sample set Sample(x,y) is in the Whether there are #min or more grayscale values and the two-dimensional Euclidean distance of the grayscale value of the pixel x less than the distance threshold R, if so, set the grayscale value of the pixel to 255, otherwise, set it to 0, # min represents the set foreground detection threshold, and N represents the number of samples;

S52: Perform morphological processing on the grayscale image processed in S51 to obtain a morphologically processed grayscale image.

Preferably, in S52, the step of morphological processing includes:

S521: Perform connectivity analysis on the pixels with a grayscale value of 255 and calculate the size of the connected region;

S522: For the pixel area with the connected area area less than 9, set the gray value of each pixel to 0;

S523: Perform erosion and expansion operations on the grayscale image to obtain a morphologically processed grayscale image.

Preferably, in S6, based on the inter-frame change rate obtained in S3, the step of updating the background model according to the inter-frame change rate adjustment strategy includes:

S61: Obtain a random factor θ according to the rate of change between frames obtained in S3;

S62: Update the background model for each pixel according to the random factor θ.

Preferably, S62 includes:

All pixels whose gray value is set to 0 have a probability of 1/θ to update the sample set corresponding to themselves and their eight-neighborhood pixels. The specific update method is:

Randomly replace the gray value of the current pixel with a gray value in the sample set corresponding to the current pixel;

The gray value of the current pixel is randomly replaced with one gray value in the sample set corresponding to the eight neighboring pixels of the current pixel.

According to another aspect of the present invention, a storage medium is also provided, the storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium, and the computer program is used to realize the The steps of the improved ViBe indoor real-time foreground detection method described above.

The technical scheme provided by the invention has the following beneficial effects:

On the basis of not losing the real-time performance of the original ViBe algorithm, the concept of inter-frame change rate is introduced. The random factor adjustment strategy based on the inter-frame change rate can effectively suppress the "ghosting" and hole caused by the space-time propagation mechanism. generation, which greatly enhances the detection ability of the algorithm. In addition, the illumination detection method based on the rate of change between frames and the perceptual hash algorithm enhances the algorithm's robustness to illumination changes to a certain extent, and is more suitable for foreground detection in indoor environments than the original ViBe algorithm.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

FIG. 1 is a flowchart of an improved ViBe indoor real-time foreground detection method provided by an embodiment of the present invention.

Detailed ways

In order to have a clearer understanding of the technical features, objects and effects of the present invention, the specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

Please refer to FIG. 1, an embodiment of the present invention provides an improved ViBe indoor real-time foreground detection method, comprising the following steps:

S1: Receive a video stream and perform decoding and preprocessing to obtain a grayscale image of frame-by-frame video images.

S1 specifically includes:

S11: The present embodiment uses the VideoCapture function of the OpenCV library to decode the input video, and obtains each frame of video image according to the time sequence;

S12: Convert the acquired video image to a grayscale image, and the specific formula is as follows:

Grey=0.299*R+0.587*G+0.114*B

Wherein R, G, B are the three primary colors of the color image respectively, and the present embodiment uses the cvtColor function of the OpenCV library to realize the conversion of the color image to the grayscale image;

S13: Use Gaussian filtering and median filtering to denoise the grayscale image in sequence. In this embodiment, Gaussian filtering uses a Gaussian kernel with a size of 5*5, and median filtering uses a filtering template with a size of 5. The specific implementation method GaussianBlur and medianBlur functions using OpenCV library.

S2: monitor the number of frames of the current frame of video image, determine whether it is the first frame of the video, if so, establish a background model of the foreground detection algorithm and enter S1, otherwise, enter S3;

S2 specifically includes:

S21: Obtain the first frame grayscale image of the video, and construct a sample set with a maximum capacity of N for each pixel;

S22: Each pixel randomly selects the gray value of the pixels in its eight neighborhoods and adds it to the sample set, until the sample set capacity reaches the upper limit;

The expression for the sample set is as follows:

Sample(x,y)={Vi|i=1,2...N}

In the formula, Sample(x, y) is the sample set of pixels, (x, y) is the Cartesian coordinate value of any pixel, Vi is the sample, N is the number of samples, and the upper limit N set in this embodiment is 20. .

S3: Calculate the rate of change between frames through the grayscale image of the current frame and the grayscale image of the previous frame;

S3 specifically includes:

S31: Select the grayscale images of the current frame and the previous frame, and perform a differential operation on any pixel point according to the following formula to obtain a differential grayscale image:

D(x,y)＝|f _k (x,y)-f _k-1 (x,y)|

where D(x,y) is the obtained differential grayscale image, fk( _x ,y) is the grayscale image of the current frame, and fk _-1 (x,y) is the grayscale image of the previous frame;

S32: Determine whether the absolute value of the grayscale value of each pixel in the differential grayscale image is greater than the threshold value T, if yes, modify its grayscale value to 255, if not, set it to 0; in this embodiment, the threshold value T is set to 5, can be set according to the actual situation;

S33 : the number of pixels with a grayscale value of 255 in the statistical difference grayscale image is num ₁ , the total number of pixels in the image is num ₂ , and the inter-frame change rate P=num ₁ /num ₂ ×100% is calculated.

S4: determine whether a sudden change in illumination occurs, if so, re-create the background model in S2 and enter step S1, otherwise, enter S5;

S4 specifically includes:

S41 : Determine whether the value of the inter-frame change rate P calculated in S33 is greater than the threshold value T _light , if so, proceed to S42 , otherwise, determine that no sudden change in illumination occurs. In this embodiment, the threshold T _light is set to 30%, which can be set according to the actual situation;

S42: Comparing the grayscale image of the current frame and the differential grayscale image obtained in S31 by using the perceptual hash algorithm, and calculating the _Hamming distance of the two images. The actual situation is set. If it is greater than the threshold, it is judged that no sudden change of light has occurred, otherwise, it is judged that sudden change of light has occurred.

S5: Obtain the foreground detection result of the video image of the current frame;

S5 specifically includes:

S51: For the grayscale image of the current frame, compare the grayscale value of each pixel point x with the N grayscale values in the corresponding sample set Sample(x), and determine whether the sample set Sample(x, y) exists The two-dimensional Euclidean distance between two or more grayscale values and the grayscale value of pixel x is less than the threshold value of 20. If yes, set the grayscale value of the pixel to 255. If not, set it to 0. The threshold here can be Set according to the actual situation;

S52: Perform morphological processing on the grayscale image processed in S51;

In this embodiment, use the findContours function of the OpenCV library to find all connected domains, traverse all the connected domains, and use the contourArea function of the OpenCV library to calculate the area of the connected domains. , regard it as noise, and set the gray value of each pixel to 0. After that, the grayscale image is first eroded and then expanded, specifically: using the dilate and erode functions of the OpenCV library, the size of the convolution kernel for the erosion and collision operations is set to 3*3.

S6: Based on the inter-frame change rate obtained in S3, update the background model according to the inter-frame change rate adjustment strategy;

S6 specifically includes:

S61: Obtain the random factor θ according to the inter-frame change rate obtained in S33, as shown in Table 1:

Table 1: Correspondence between the rate of change between frames and the random factor

The data in Table 1 are obtained from experiments and are suitable for most scenarios.

S62: All pixels with a gray value of 0 have a probability of 1/θ to update their corresponding sample set. The specific update method is: randomly replace the gray value of the current pixel with a gray value in the sample set corresponding to the current pixel.

S63: All pixels with a gray value of 0 have a probability of 1/θ to update the sample set corresponding to their own neighborhood pixels. The specific update method is: randomly replace the gray value of the current pixel with one gray value in the sample set corresponding to the eight neighboring pixels of the current pixel.

S7: Determine whether there is a next frame of video image, if so, enter S1, otherwise, the detection process ends. In this embodiment, the isOpened function of the OpenCV library is used to determine whether there is a next frame of video image.

In order to verify the beneficial effects of the present invention, this embodiment selects seven indoor subsets in the ChangeDetection2012 data set to carry out the same method as the commonly used foreground detection methods in 5 (Gaussian mixture model GMM, three frame difference method TFD, nearest neighbor node algorithm KNN, code This algorithm codebook and the visual background extractor algorithm ViBe) have carried out comparative experiments. The evaluation indicators are PWC (Percentage of Wrong Classifications) and F-Measure (F-Measure). The specific experimental results are shown in Tables 2 and 3:

Table 2 PWC comparison between the method of the present invention and 5 commonly used foreground detection methods

Table 3 Comparison of F values between the method of the present invention and 5 commonly used foreground detection methods

In some embodiments, a storage medium is also provided, the storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium, and the computer program is used to implement the described Steps of an improved ViBe indoor real-time foreground detection method.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order, and these words may be construed as identifications.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (10)

1. an improved ViBe indoor real-time foreground detection method, is characterized in that, comprises the following steps: S1: Receive the video stream and perform decoding and preprocessing to obtain the grayscale image of the frame-by-frame video image; S2: monitor the number of frames of the current frame of video image, determine whether it is the first frame of the video, if so, establish a background model of the foreground detection algorithm and enter S1, otherwise, enter S3; S3: Calculate the rate of change between frames through the grayscale image of the current frame and the grayscale image of the previous frame; S4: determine whether a sudden change in illumination occurs, if so, re-create the background model in S2 and enter step S1, otherwise, enter S5; S5: Obtain the foreground detection result of the video image of the current frame; S6: Based on the inter-frame change rate obtained in S3, update the background model according to the inter-frame change rate adjustment strategy; S7: Determine whether there is a next frame of video image, if so, enter S1, otherwise, the detection process ends. 2. the improved ViBe indoor real-time foreground detection method as claimed in claim 1, is characterized in that, in S1, the step of receiving video stream and decoding, preprocessing comprises: S11: Receive and decode the video stream to obtain a frame-by-frame video image queue; S12: Perform grayscale conversion on the single-frame video image in the video image queue to obtain a grayscale image of the single-frame video image; S13: Use Gaussian and median filtering to denoise the grayscale image to obtain a denoised grayscale image. 3. improved ViBe indoor real-time foreground detection method as claimed in claim 1, is characterized in that, in S2, the establishment step of background model comprises: S21: Obtain the first frame grayscale image of the video, and construct a sample set with a maximum capacity of N for each pixel; S22: Each pixel randomly selects the gray value of the pixels in its eight neighborhoods and adds it to the sample set, until the sample set capacity reaches the upper limit; The expression for the sample set is as follows: Sample(x,y)={V _i |i=1,2...N} In the formula, Sample(x, y) is the sample set of pixels, (x, y) is the Cartesian coordinate value of any pixel, V _i is the sample, and N is the number of samples. 4. improved ViBe indoor real-time foreground detection method as claimed in claim 1, is characterized in that, in S3, the step of calculating rate of change between frames comprises: S31: Select the grayscale image of the current frame and the grayscale image of the previous frame, perform a difference operation on the pixel points at the same position in the Cartesian coordinate system and take the absolute value to obtain a differential grayscale image. The formula is as follows: D(x,y)＝|f _k (x,y)-f _k-1 (x,y)| In the formula, D(x,y) is the obtained differential grayscale image, fk( _x ,y) is the current frame grayscale image, and fk _-1 (x,y) is the previous frame grayscale image; S32: Determine whether the absolute value of the grayscale value of each pixel in the differential grayscale image is greater than the threshold T, and if so, modify its grayscale value to 255, otherwise, set it to 0; S33 : the number of pixels with a grayscale value of 255 in the statistical difference grayscale image is num ₁ , the total number of pixels in the image is num ₂ , and the inter-frame change rate P=num ₁ /num ₂ ×100% is calculated. 5. the improved ViBe indoor real-time foreground detection method as claimed in claim 4, is characterized in that, in S4, judge whether the method for sudden change in illumination comprises: S41: Determine whether the value of the inter-frame change rate P calculated in S33 is greater than the threshold T _light , if so, enter S42, otherwise, determine that no sudden change in illumination occurs; S42: Use the perceptual hash algorithm to compare the grayscale image of the current frame and the differential grayscale image obtained in S31, and determine whether the Hamming distance of the two images is greater than the threshold T _enter , if so, determine that no sudden change in illumination has occurred, otherwise, determine that it has occurred light mutation. 6. improved ViBe indoor real-time foreground detection method as claimed in claim 1, is characterized in that, in S5, the step of obtaining foreground detection result comprises: S51: For the grayscale image of the current frame, compare the grayscale value of each pixel point x with the N grayscale values in the corresponding sample set Sample(x,y), and determine whether the sample set Sample(x,y) Whether there are #min or more grayscale values and the two-dimensional Euclidean distance of the grayscale value of the pixel x less than the distance threshold R, if so, set the grayscale value of the pixel to 255, otherwise, set it to 0, # min represents the set foreground detection threshold, and N represents the number of samples; S52: Perform morphological processing on the grayscale image processed in S51 to obtain a morphologically processed grayscale image. 7. improved ViBe indoor real-time foreground detection method according to claim 6 is characterized in that, in S52, the step of morphological processing comprises: S521: Perform connectivity analysis on the pixels with a grayscale value of 255 and calculate the size of the connected region; S522: for the pixel area with the connected area area less than 9, set the gray value of each pixel to 0; S523: Perform erosion and expansion operations on the grayscale image to obtain a morphologically processed grayscale image. 8. improved ViBe indoor real-time foreground detection method according to claim 1, is characterized in that, in S6, based on the inter-frame change rate that S3 obtains, the step of updating background model according to the inter-frame change rate adjustment strategy comprises: S61: Obtain a random factor θ according to the rate of change between frames obtained in S3; S62: Update the background model for each pixel according to the random factor θ. 9. improved ViBe indoor real-time foreground detection method according to claim 8 is characterized in that, S62 comprises: All pixels whose gray value is set to 0 have a probability of 1/θ to update the sample set corresponding to themselves and their eight-neighborhood pixels. The specific update method is: Randomly replace the gray value of the current pixel with a gray value in the sample set corresponding to the current pixel; The gray value of the current pixel is randomly replaced with one gray value in the sample set corresponding to the eight neighboring pixels of the current pixel. 10. A storage medium, characterized in that, the storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium, and the computer program is used to implement the methods of claims 1-9 when running. Any of the steps of the improved ViBe indoor real-time foreground detection method.

Images