CN107330372B - An Analysis Method for Video-Based Crowd Density and Abnormal Behavior Detection System - Google Patents

Abstract

The invention discloses an analysis method of a video-based crowd density and abnormal behavior detection system, and relates to the fields of intelligent video monitoring, target detection and the like. The methods include: using an improved method based on pixel statistics and texture statistics for density classification; using a combination of pixel statistics and foreground corner detection to estimate the number of people; introducing local optical flow, proposes and implements a method based on average A crowd abnormal behavior detection algorithm based on kinetic energy change magnification. This method is not only applicable to general surveillance video, but also applicable to crowd gathering videos in more prominent large-scale public places. The method system of the present invention is less time-consuming, has better effectiveness and practicability, and meets practical requirements.

Description

An Analysis Method of Video-Based Crowd Density and Abnormal Behavior Detection System

technical field

The invention relates to an analysis method of a video-based crowd density and abnormal behavior detection system in the field of public safety, in particular to an improved crowd density classification method based on the combination of pixel point statistics and texture statistics, pixel point statistics and corner point detection The combined population estimation method proposes and implements a crowd abnormal behavior detection algorithm based on the average kinetic energy change rate, which belongs to the field of machine vision and intelligent information processing.

Background technique

In recent years, with the rapid development of the economy and the continuous increase of people's social activities, the phenomenon of traffic congestion in public places such as transportation hubs, large-scale event sites and large shopping malls has become more and more frequent. serious. Therefore, in order to maintain public order and ensure the safety of the crowd, more and more monitoring systems have been put into use. The EU's ADVISOR system is a safety management system applied to public transportation; it covers individual behavior analysis, including group behavior analysis, and also has some involvement in human-computer interaction. With the explosive increase in the demand for video surveillance systems in the market, it is imminent to develop a stable, fast and multi-functional intelligent surveillance system.

The analysis method of the video-based crowd density and abnormal behavior detection system mainly includes three parts: crowd density classification, number estimation and abnormal behavior detection.

First, grade the crowd density in the video. Video-based crowd density classification methods can be roughly divided into the following categories: crowd density classification methods based on pixel statistics, crowd density classification methods based on texture features, and crowd density classification methods based on individual characteristics. The classification method based on pixel point statistics is simple to implement and requires less computation, but the final density classification result has a large error due to the mutual occlusion between crowds. This method is only suitable for scenarios with low crowd density. The classification method based on texture features can better deal with the overlapping and occlusion phenomenon between crowds, and can achieve better crowd density classification. The crowd density classification error in the scene is large, and this method is only suitable for scenes with high crowd flow density. The method based on individual characteristics is difficult to achieve real-time monitoring in practical application scenarios due to its high computational complexity.

Second, estimate the number of people in the video. The number of people detection algorithms are mainly divided into two categories, one is an algorithm that directly counts the number of people by detecting or identifying crowd targets, and the other is an algorithm that indirectly calculates the number of people by extracting the characteristics of the crowd. The direct calculation method can be divided into two types: model-based and trajectory-based clustering. The main idea of model-based is to segment people, and then use the model or shape of the person to match them for counting. The algorithm based on trajectory clustering is based on detection. The independent movement patterns in the crowd are counted separately according to the movement patterns. The indirect calculation method mainly includes the algorithm based on pixel features and the algorithm based on feature points. Davie et al. first proposed the use of image processing technology to detect the number of people in 1995. The main idea is to first extract the foreground edge image of the video image and obtain its total number of pixels, and then input the obtained number of pixels into training to get The linear regression equation calculates the number of people, the algorithm complexity of this method is low, but the algorithm accuracy is not high. In the case of high crowd density, the crowded crowd will lead to the overlapping or occlusion of targets. At this time, the accuracy of the pixel feature-based method will be greatly reduced. Therefore, the pixel feature-based algorithm can only be used when the crowd density is low. use. The main idea of the algorithm based on feature points is to first extract the feature points, such as Harris corner points, SIFT corner points, SURF corner points, etc., and then track the corner points. parameters and get the regression equation, and finally get the total number of people according to the number of foreground feature points and the trained regression equation.

Finally, abnormal behavior detection is performed on the crowd in the video. The detection of abnormal crowd behavior belongs to the category of group behavior recognition. At present, whether there is abnormal behavior is mainly judged by the movement speed and direction of the crowd. Many scholars have also proposed some algorithms for the detection of abnormal crowd behavior. Andrade et al. first defined a "normal group movement", and then used Hidden Markov Model (HMM) for crowd behavior analysis, but the performance of the crowd behavior description feature used by the algorithm was poor, so the detection accuracy lower. Wang et al. describe group behavior by using improved spatiotemporal volume features, but the computational complexity of this method is too high to meet real-time requirements. In addition, Hassner et al. proposed a crowd violence detection algorithm based on global optical flow information, which has high detection accuracy. Liu et al. proposed to use a series of subject motion-based models to discover group motion trajectories to identify group motions. This method has a good recognition effect, but the processing speed is too slow, which is not conducive to practical applications.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method for a video-based crowd density and abnormal behavior detection system in order to solve the above problems.

The present invention realizes above-mentioned purpose through following technical scheme:

(1) Crowd density classification:

Combining pixel point statistics and texture features, first, the pixel point statistics method is used to pre-estimate the crowd. When it is judged that the current crowd density is too high, the texture feature is used to estimate the crowd density. This method not only uses the pixel point statistics method to achieve It has the advantages of simplicity and high speed, and can use the texture feature method for further density classification when there are high-density crowds.

Crowd density classification based on pixel statistics:

(a) First convert the image to be processed into a grayscale image, and use the literature Canny edge detection algorithm to extract the edges of two adjacent grayscale images to obtain edge images;

(b) Differentiate the edge images obtained from the two frames, and remove the interference noise through morphological expansion and corrosion operations to obtain an optimized differential edge image;

(c) performing AND operation on the edge image of the current frame and the optimized differential edge image to obtain a moving foreground edge image;

(d) dividing the foreground edge image into a plurality of rectangular frames, calculating the number of edge pixels in each rectangular frame, and calculating the final total pixel number of the edge;

(e) Using the linear fitting method to fit the relationship between the number of crowds and the total number of pixels on the foreground edge of the crowd, to estimate the number of crowds, and then to estimate the approximate crowd density.

(f) Whether the current number of people is greater than the threshold Thr1, if not, output the density level, low density and medium density; if so, adopt the crowd density level classification based on texture features, and output high density and extremely high density.

Crowd density classification based on texture features, because the textures in the image are irregular, most of them obey the statistical distribution law, so this paper selects the texture features based on the gray level co-occurrence matrix (GLCM) to analyze the high-density crowd, and realize the classification of different crowd densities. . Let I(x,y) be the original image, M×N is the size of the image, M, N are the width and height of the image respectively, L is the gray level of the image, according to a certain spatial relationship, the obtained gray level co-occurrence matrix for:

f(i,j)=#{(x ₁ ,y ₁ ),(x ₂ ,y ₂ )∈M×N|I(x ₁ ,y ₁ )=i,I(x ₂ ,y ₂ )=j } (1)

In formula (1), i and j represent the gray value of the image, # represents the number of elements in the set, f is an L×L matrix, and the pixel points (x ₁ , y ₁ ) and (x ₂ , y ₂ ) are between The distance between them is d, and the angle between the connection line between the two pixels and the horizontal direction is θ, then the generated grayscale co-occurrence matrix is f(i,j|d,θ), which means that the gray value appears in the image. They are i and j respectively, the distance is d, and the direction is the number of two pixel points of θ, generally θ=0°, 45°, 90°, 135°. Five texture parameters are obtained from the gray level co-occurrence matrix: contrast, uniformity, energy, entropy, and correlation.

Contrast

uniformity

energy

entropy

Correlation

in,

(2) Estimated number of people:

Combining pixel statistics and foreground corner detection, first use pixel statistics to pre-estimate the number of people in the video. When it is judged that the current number of people is higher than the threshold Thr1, the foreground corner detection is used to estimate the number of people in the video. Specific steps are as follows:

The estimation of the number of people based on pixel counts is as described in (1) above.

Population estimation based on foreground point corner detection:

(a) Initialization parameters, mainly initializing the values of α and β, the minimum distance d _min between the corner points, etc.

(b) Corner detection, using the Shi-Tomasi algorithm to obtain all corners of the video image.

(c) Optical flow tracking, using the pyramid LK optical flow method to effectively track the acquired corners.

(d) Extract the foreground corner points, and remove the background points by calculating the optical flow amplitude at each corner point. When the optical flow amplitude at this point is greater than the threshold Thr2, it is the foreground point.

(e) Detect the number of people in the crowd, and according to the number of foreground corners, calculate the estimated number of crowds N _est according to the initial value and formula (11).

N _est =(αn+βn)/2 (11)

Among them, the number of foreground corner points detected in the video image is n, and the number of target crowds is N. In order to reduce the influence caused by the angle in the image, the lower limit of the number of crowds is defined as αn, the upper limit is βn, α, β It is an appropriate positive number, and α≤βAccording to a large number of data statistics, it is found that the number of people N is generally not greater than the number of foreground corners n, so β≤1. The size of α is related to the proportion of the image area occupied by individuals in the crowd, so generally 0.5<α≤1.

(3) Abnormal behavior detection

Generally speaking, when the crowd behaves abnormally, the change of human kinetic energy will be much more severe than normal. Therefore, the behavior state of the crowd can be judged by analyzing the change of human kinetic energy. The kinetic energy of human motion is generally positively correlated with the amplitude of its optical flow. Therefore, the amplitude of optical flow can be used to represent the kinetic energy of human motion. According to the optical flow at a certain point, the optical flow amplitude can be obtained. Combined with the motion information between frames, the The variation of the optical flow amplitude is obtained, so we can calculate the average optical flow amplitude of the crowd, and then according to the inter-frame information, we can obtain the change of the average optical flow amplitude.

Assume that the optical flow vector of a pixel p _x,y in the video image at time t is (u _x,y,t ,v _x,y,t ), where u _x,y,t ,v _{x,y , t} are the optical flow components of p _{x and y} in the x and y directions at time t, respectively, and the optical flow amplitude at time t at this point is expressed as follows:

Find the variation of the optical flow amplitude between adjacent frames at p _{x, y} , and then binarize it according to the set threshold mean, and set the variation of the optical flow amplitude after binarization as b _{x, y, t} , the binarization rule is expressed as follows:

The threshold mean is the average value of the variation of the optical flow amplitude between frames, and the mean is obtained according to formula (14).

Among them, row and cols represent the number of rows and columns of the video image, respectively.

依据式(15)得到。Then calculate the average change of the binarized optical flow amplitude of the same pixel at different times, assuming that the average change of the binarized optical flow amplitude of the pixel is

Obtained according to formula (15).

前一帧光流幅值平均值、动能平均值分别为

用式(16)表示动能变化与光流幅值量变化的关系。It can be seen from the above discussion that the optical flow amplitude value of each pixel is obtained by calculating the optical flow information, and then the average value of the optical flow amplitude value of each frame can be calculated. Here, the average value of the optical flow amplitude and the average value of kinetic energy of the current frame are set as

The average value of the optical flow amplitude and the average value of kinetic energy in the previous frame are respectively

Formula (16) is used to express the relationship between the change of kinetic energy and the change of the amplitude of the optical flow.

Among them, k is a general positive number, λ represents the change rate of the average kinetic energy, when λ exceeds the threshold Thr2, the current crowd movement has abnormal behavior.

In summary, we can calculate the average kinetic energy change rate of each group of people in the image, and then judge whether the current crowd has abnormal behavior according to the threshold Thr2. If there is abnormal behavior, save the average optical flow amplitude of the previous frame of the first abnormal frame, so as to find the average kinetic energy change ratio of the subsequent frame and this frame, so as to determine whether the subsequent frame is abnormal.

The beneficial effects of the present invention are:

The present invention designs a video-based crowd density and abnormal behavior detection system, which can detect crowd density levels, estimate the number of people, and detect whether the crowd is normal or not, aiming at the videos of crowds gathered in relatively prominent large-scale public places. It has the advantages of accurate detection and can implement alarm for abnormal crowds. The method of the present invention is easy to be implemented on a software platform, with small computation amount and high real-time performance.

Description of drawings

FIG. 1 is an overall flow chart of the video-based crowd density and abnormal behavior detection system according to the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings:

As shown in Figure 1, the flow in the figure is a brief description of the main process of the following method, and the following steps are used to analyze the video images used for monitoring:

(1) Crowd density classification:

Combining pixel point statistics and texture features, first, the pixel point statistics method is used to pre-estimate the crowd. When it is judged that the current crowd density is too high, the texture feature is used to estimate the crowd density. This method not only uses the pixel point statistics method to achieve It has the advantages of simplicity and high speed, and can use the texture feature method for further density classification when there are high-density crowds.

Crowd density classification based on pixel statistics:

(a) First convert the image to be processed into a grayscale image, and use the literature Canny edge detection algorithm to extract the edges of two adjacent grayscale images to obtain edge images;

(b) Differentiate the edge images obtained from the two frames, and remove the interference noise through morphological expansion and corrosion operations to obtain an optimized differential edge image;

(c) performing AND operation on the edge image of the current frame and the optimized differential edge image to obtain a moving foreground edge image;

(d) dividing the foreground edge image into a plurality of rectangular frames, calculating the number of edge pixels in each rectangular frame, and calculating the final total pixel number of the edge;

(e) Using the linear fitting method to fit the relationship between the number of crowds and the total number of pixels on the foreground edge of the crowd, to estimate the number of crowds, and then to estimate the approximate crowd density.

(f) Whether the current number of people is greater than the threshold Thr1, if not, output the density level, low density and medium density; if so, adopt the crowd density level classification based on texture features, and output high density and extremely high density.

Crowd density classification based on texture features, because the textures in the image are irregular, most of them obey the statistical distribution law, so this paper selects the texture features based on the gray level co-occurrence matrix (GLCM) to analyze the high-density crowd, and realize the classification of different crowd densities. . Let I(x,y) be the original image, M×N is the size of the image, M, N are the width and height of the image respectively, L is the gray level of the image, according to a certain spatial relationship, the obtained gray level co-occurrence matrix for:

f(i,j)=#{(x ₁ ,y ₁ ),(x ₂ ,y ₂ )∈M×N|I(x ₁ ,y ₁ )=i,I(x ₂ ,y ₂ )=j } (1)

In formula (1), i and j represent the gray value of the image, # represents the number of elements in the set, f is an L×L matrix, and the pixel points (x ₁ , y ₁ ) and (x ₂ , y ₂ ) are between The distance between them is d, and the angle between the connection line between the two pixels and the horizontal direction is θ, then the generated grayscale co-occurrence matrix is f(i,j|d,θ), which means that the gray value appears in the image. They are i and j respectively, the distance is d, and the direction is the number of two pixel points of θ, generally θ=0°, 45°, 90°, 135°. Five texture parameters are obtained from the gray level co-occurrence matrix: contrast, uniformity, energy, entropy, and correlation.

Contrast

uniformity

energy

entropy

Correlation

in,

(2) Estimated number of people:

Combining pixel statistics and foreground corner detection, first use pixel statistics to pre-estimate the number of people in the video. When it is judged that the current number of people is higher than the threshold Thr1, the foreground corner detection is used to estimate the number of people in the video. Specific steps are as follows:

The estimation of the number of people based on pixel counts is as described in (1) above.

Population estimation based on foreground point corner detection:

(a) Initialization parameters, mainly initializing the values of α and β, the minimum distance d _min between the corner points, etc.

(b) Corner detection, using the Shi-Tomasi algorithm to obtain all corners of the video image.

(c) Optical flow tracking, using the pyramid LK optical flow method to effectively track the acquired corners.

(d) Extract the foreground corner points, and remove the background points by calculating the optical flow amplitude at each corner point. When the optical flow amplitude at this point is greater than the threshold Thr2, it is the foreground point.

(e) Detect the number of people in the crowd, and according to the number of foreground corners, calculate the estimated number of crowds N _est according to the initial value and formula (11).

N _est =(αn+βn)/2 (11)

Among them, the number of foreground corner points detected in the video image is n, and the number of target crowds is N. In order to reduce the influence caused by the angle in the image, the lower limit of the number of crowds is defined as αn, the upper limit is βn, α, β It is an appropriate positive number, and α≤βAccording to a large number of data statistics, it is found that the number of people N is generally not greater than the number of foreground corners n, so β≤1. The size of α is related to the proportion of the image area occupied by individuals in the crowd, so generally 0.5<α≤1.

(3) Abnormal behavior detection

Assume that the optical flow vector of a pixel p _x,y in the video image at time t is (u _x,y,t ,v _x,y,t ), where u _x,y,t ,v _{x,y , t} are the optical flow components of p _{x and y} in the x and y directions at time t, respectively, and the optical flow amplitude at time t at this point is expressed as follows:

Find the variation of the optical flow amplitude between adjacent frames at p _{x, y} , and then binarize it according to the set threshold mean, and set the variation of the optical flow amplitude after binarization as b _{x, y, t} , the binarization rule is expressed as follows:

The threshold mean is the average value of the variation of the optical flow amplitude between frames, and the mean is obtained according to formula (14).

Among them, row and cols represent the number of rows and columns of the video image, respectively.

依据式(15)得到。Then calculate the average change of the binarized optical flow amplitude of the same pixel at different times, assuming that the average change of the binarized optical flow amplitude of the pixel is

Obtained according to formula (15).

前一帧光流幅值平均值、动能平均值分别为

用式(16)表示动能变化与光流幅值量变化的关系。It can be seen from the above discussion that the optical flow amplitude value of each pixel is obtained by calculating the optical flow information, and then the average value of the optical flow amplitude value of each frame can be calculated. Here, the average value of the optical flow amplitude and the average value of kinetic energy of the current frame are set as

The average value of the optical flow amplitude and the average value of kinetic energy in the previous frame are respectively

Formula (16) is used to express the relationship between the change of kinetic energy and the change of the amplitude of the optical flow.

Among them, k is a general positive number, λ represents the change rate of the average kinetic energy, when λ exceeds the threshold Thr2, the current crowd movement has abnormal behavior.

In summary, we can calculate the average kinetic energy change rate of each group of people in the image, and then judge whether the current crowd has abnormal behavior according to the threshold Thr2. If there is abnormal behavior, save the average optical flow amplitude of the previous frame of the first abnormal frame, so as to find the average kinetic energy change ratio of the subsequent frame and this frame, so as to determine whether the subsequent frame is abnormal.

In order to verify the validity and feasibility of the video-based crowd density and abnormal behavior detection system proposed by the present invention, detailed analysis and comparison are carried out through experiments below:

The relationship between population density and number of people is defined by Table 1. This experiment was tested on the public video Grand Central and the self-shot video. The accuracy of density grade classification is shown in Table 2, and the average relative error of population estimation is shown in Table 3. The positive detection rate and missed detection of abnormal behavior detection are shown in Table 2. The rate and false alarm rate are shown in Table 4, and the average time consumption of the video analysis algorithm of the present invention to process each frame of video is shown in Table 5. From Table 2, Table 3, Table 4, and Table 5, it can be seen that the proposed method of the present invention is adopted. The analysis method can obtain good results, and the experimental results also verify the validity and feasibility of the analysis method of the present invention.

Table 1 Definition of population density class

density class low density medium density high density very high density number of people 0-20 21-50 51-150 >150

Table 2 The accuracy rate of population density grade classification

density class The density classification accuracy rate of the algorithm of the present invention low density 91.1% medium density 90.2% high density 79.6% very high density 86.5%

Table 3 Average relative error of population estimates

algorithm mean relative error The number estimation algorithm of the present invention 16.82%

Table 4 Positive detection rate, missed detection rate and false alarm rate of abnormal behavior detection

algorithm Positive rate of abnormal behavior Abnormal behavior missed detection rate Abnormal behavior false alarm rate The abnormal behavior detection algorithm of the present invention 92.12% 7.88% 3.18%

Table 5 The average time-consuming of the system (processing each frame)

algorithm The average time of the system The video analysis method of the present invention 101.84ms/frame

Claims (3)

1. An analysis method of a video-based crowd density and abnormal behavior detection system comprises the following steps: the method comprises the following steps:

(1) grading the population density: classifying the crowd density in the video by a method combining pixel point statistics and textural features;

(2) estimating the number of people: estimating the number of people in the video by combining the estimation of the number of people based on the statistics of the pixel points and the estimation of the number of people based on the angular point detection, firstly, pre-estimating the number of people in the video by using a pixel point statistical method, and estimating the number of people in the video by using the foreground angular point detection when the current number of people is judged to be higher than a threshold Thr 1;

estimating the number of people based on foreground corner detection:

(a) initializing parameters, and initializing values of alpha and beta;

(b) detecting angular points, namely acquiring all angular points of the video image by adopting a Shi-Tomasi algorithm;

(c) optical flow tracking, namely effectively tracking the acquired corner points by adopting a pyramid LK optical flow method;

(d) extracting foreground angular points, and eliminating background points by calculating the optical flow amplitude of each angular point, wherein when the optical flow amplitude is greater than a threshold Thr3, the foreground points are obtained;

(e) detecting the number of people, and calculating according to the number of the foreground angular points and the initial value and the formula (1) to obtain an estimated value N of the number of people_est；

N_est＝(αn+βn)/2 (1)

The method comprises the steps that the number of foreground angular points detected in a video image is N, the number of people as a crowd target is N, in order to reduce the influence caused by angles in the image, the lower limit value of the number of people as alpha N is defined, the upper limit value as beta N, alpha and beta are proper positive numbers, alpha is less than or equal to beta, and according to statistics of a large amount of data, the number of people N is generally not greater than the number N of the foreground angular points, so that beta is less than or equal to 1; the size of alpha is related to the specific gravity of the image area occupied by the individual crowd, so that the alpha is more than 0.5 and less than or equal to 1;

(3) and (3) abnormal behavior detection: generally, when the crowd has abnormal behaviors, the change of the kinetic energy of the human body is much more drastic than that under normal conditions, so the behavior state of the crowd can be judged by analyzing the change of the kinetic energy of the human body; the motion kinetic energy of the person is generally in positive correlation with the amplitude quantity of the optical flow of the person, so that the motion kinetic energy of the person can be represented by the optical flow amplitude, the optical flow amplitude can be obtained according to the optical flow of a certain point, the variation quantity of the optical flow amplitude can be obtained by combining motion information among frames, the average optical flow amplitude of the crowd can be calculated, then the variation quantity of the average optical flow amplitude can be obtained according to the information among the frames, and then the variation multiplying power of the average kinetic energy is calculated according to the variation quantity of the average optical flow amplitude of the previous frame; a threshold Thr2 is set, and whether abnormal behavior occurs is judged according to the relation between the image average kinetic energy change multiplying power and Thr 2.

2. The method of claim 1, wherein the system further comprises a video-based crowd density and abnormal behavior detection system, wherein the video-based crowd density and abnormal behavior detection system comprises: in the step (1), the method for classifying the crowd density comprises the following steps:

combining pixel point statistics with textural features, firstly, carrying out pre-estimation on the crowd by using a pixel point statistical method, and carrying out crowd density estimation by using the textural features when the current crowd density is judged to be overhigh, wherein the method comprises the following specific steps:

classifying the crowd density based on pixel point statistics:

(a) firstly, converting an image to be processed into a gray image, and performing edge extraction on two adjacent frames of gray images by adopting a Canny edge detection algorithm to obtain an edge image;

(b) differentiating the edge images obtained by the two frames, and eliminating interference noise through morphological expansion and corrosion operation to obtain an optimized differential edge image;

(c) performing AND operation on the edge image of the current frame and the optimized differential edge image to obtain a moving foreground edge image;

(d) dividing the foreground edge image into a plurality of rectangular frames, calculating the number of edge pixel points in each rectangular frame, and calculating the final total pixel point number of the edge;

(e) fitting the relation between the number of the crowd and the total pixel points of the crowd foreground edge by adopting a linear fitting method, estimating the number of the crowd, and further estimating the approximate crowd density;

(f) whether the current number of people is greater than a threshold Thr1 or not, if not, outputting a density grade, a low density and a medium density; if so, classifying the crowd density grade based on the texture characteristics, and outputting high density and extremely high density;

grading the crowd density based on the textural features:

assuming that I (x, y) is an original image, M multiplied by N is an image size, M and N respectively represent the width and height of the image, L represents the gray level of the image, and the obtained gray co-occurrence matrix is as follows according to a certain spatial relationship:

f(i,j)＝#{(x₁,y₁),(x₂,y₂)∈M×N|I(x₁,y₁)＝i,I(x₂,y₂)＝j} (2)

in the formula (2), i and j represent image gray values, # represents the number of elements in the set, and f is an L × L matrix, when a pixel point (x)₁,y₁) And (x)₂,y₂) The distance between the two pixels is d, the included angle between the connecting line of the two pixels and the horizontal direction is theta, and the generated gray level co-occurrence matrix is f (i, j | d, theta), which means that the gray level values appearing in the image are i and j respectively, the distance between the two pixels is d, the direction is theta, the number of times of the two pixels is theta, theta is 0 degrees, 45 degrees, 90 degrees, 135 degrees, and 5 texture parameters are obtained by the gray level co-occurrence matrix: contrast, homogeneity, energy, entropy, correlation.

3. The method of claim 1, wherein the system further comprises a video-based crowd density and abnormal behavior detection system, wherein the video-based crowd density and abnormal behavior detection system comprises: in the step (3), the method for detecting the abnormal behavior comprises the following steps:

suppose a certain pixel point p in a video image_x,yThe optical flow vector at time t is (u)_x,y,t,v_x,y,t) Wherein u is_x,y,t、v_x,y,tRespectively at time t p_x,yThe optical flow component in the x and y directions, the optical flow magnitude at this point in time t, is represented as follows:

finding p_x,yAnd (3) processing the optical flow amplitude variation between adjacent frames, then binarizing the optical flow amplitude variation according to a set threshold mean, and setting the optical flow amplitude variation after binarization as b_x,y,tThe binarization rule is expressed as follows:

the threshold mean is the average value of the amplitude variation of the interframe optical flow, and is obtained according to the formula (5);

wherein row and cols respectively represent the number of rows and columns of the video image;

then calculating the average value of the optical flow amplitude variation values of the same pixel point after binaryzation at different moments, and assuming that the average variation value of the optical flow amplitude values of the pixel point after binaryzation is equal to

Obtained according to formula (6);

the conclusion deduced by the formula shows that the optical flow amplitude of each pixel point is calculated through the optical flow information, and then the average value of the optical flow amplitude of each frame can be calculated; the average value of the optical flow amplitude and the average value of the kinetic energy of the current frame are respectively set as

Average value of light stream amplitude and kinetic energy of previous frameAverage values are respectively

The relationship between the change in kinetic energy and the change in the magnitude of optical flow is expressed by equation (7):

wherein k is a positive number, λ represents the multiplying power of the average kinetic energy change, and when λ exceeds a threshold Thr2, the current crowd moves abnormally;

in conclusion, whether the current crowd has abnormal behaviors or not is judged by calculating the average kinetic energy change multiplying power of each crowd in the image and according to the threshold Thr 2; if the abnormal behavior exists, the average optical flow amplitude of the previous frame of the first frame abnormal frame is saved so as to obtain the average kinetic energy change multiplying power of the subsequent frame and the frame, and therefore whether the subsequent frame is abnormal or not is judged.

Images