JP2008191754A - Abnormality detection apparatus and abnormality detection method - Google Patents

Abstract

To provide an abnormality detection apparatus capable of detecting an abnormal state different from normal from arbitrary multidimensional feature data with high accuracy.
An anomaly detection device includes means for extracting multidimensional feature data, means for calculating an angle between a normal subspace obtained in advance by principal component analysis and feature data, and when the angle is greater than a predetermined value. First abnormality determining means for determining abnormality, means for projecting feature data not determined to be abnormal by the first abnormality determining means to normal subspace, and feature data projected by the projecting means are obtained in advance. Second abnormality determining means for determining whether or not the distribution range is within a normal sample range. An abnormality can be determined even for an event that has been determined to be normal even though it is abnormal in the past, and the accuracy of abnormality determination is improved.
[Selection] Figure 1

Description

  The present invention relates to an anomaly detection apparatus and an anomaly detection method, and in particular, an anomaly detection apparatus capable of detecting an abnormal state different from normal from any multidimensional feature data obtained from any object to be detected with high accuracy, and The present invention relates to an abnormality detection method.

  Conventionally, many monitoring systems using cameras such as video surveillance in the security field and monitoring systems for elderly care have been used, but these are systems for human monitoring. Thus, there is a need for an automatic detection system for abnormal states from moving images by a computer. Therefore, it is necessary to recognize motion from a moving image that extracts motion features for the target.

As a study of motion recognition, the following Patent Document 1 published by some of the present inventors discloses a technique for performing abnormal motion recognition using a cubic higher-order local autocorrelation feature (hereinafter also referred to as CHLAC data). Has been. The abnormal motion detection apparatus of Patent Document 1 generates inter-frame difference data from moving image data input from a video camera, and obtains feature data from three-dimensional data composed of a plurality of inter-frame difference data by three-dimensional higher-order local autocorrelation. Extracting and calculating the distance between the latest feature data and the partial space based on the principal component vector obtained from the past feature data by the principal component analysis method. And when this distance is larger than predetermined value, it determines with it being abnormal. By learning a normal operation as a partial space and detecting an abnormal operation as a deviation therefrom, it is possible to detect an abnormal operation of one person even when there are a plurality of persons on the screen, for example.
JP 2006-079272 A

  In the above-described conventional method for detecting an abnormal operation, whether or not there is an abnormality is determined based on the distance between the subspace based on the principal component vector obtained by the principal component analysis method and the latest feature data. However, there is a problem in that there is a possibility that feature data that should be determined as abnormal may exist in the vicinity of the partial space or in the partial space based on the principal component vector obtained by the principal component analysis method. The object of the present invention is to solve such problems and to detect an abnormal state with high accuracy from any multidimensional feature data (feature vector) obtained from any target to be detected with high accuracy. The object is to provide a detection device and an abnormality detection method.

The abnormality detection apparatus of the present invention calculates feature data extraction means for extracting feature data from input data, and an angle between a normal subspace obtained in advance and the feature data extracted by the feature data extraction means. An angle calculation unit, a first abnormality determination unit that determines an abnormality when the angle is greater than a predetermined value, and feature data that has not been determined to be abnormal by the first abnormality determination unit is projected onto the normal subspace A second projecting means for determining whether the feature data projected by the projecting means is within the distribution range of the normal sample based on the distribution range information of the normal sample in the normal subspace obtained in advance. The main feature is that an abnormality determining means is provided.
Moreover, in the above-described abnormality detection device, the normal part is further based on a principal component vector obtained by a principal component analysis method from a plurality of feature data extracted by the feature data extraction means based on input data for learning. There is also a feature in that learning means for calculating a space and further calculating distribution range information of normal samples in the normal subspace from the feature data projected onto the normal subspace by the projection means is provided.

Further, in the above-described abnormality detection device, the distribution range information of the normal sample in the normal subspace includes direction vector information indicating the center of the cone including the distribution range of the normal sample in the normal subspace and the spread of the cone. The point which is the angle information to show also has the characteristic.
Further, in the above-described abnormality detection device, the feature data extraction unit generates inter-frame difference data from moving image data including a plurality of input image frame data, and three-dimensional data including the plurality of inter-frame difference data. Another feature is that the motion feature data is generated by three-dimensional higher-order local autocorrelation.
Moreover, the above-described abnormality detection apparatus is characterized in that the projection means includes scale normalization means for normalizing the scale of each element of the feature data after projection.

The abnormality detecting apparatus further includes a cluster dividing unit that divides the distribution of the projected feature data in the normal space into a plurality of clusters, and the learning unit includes the normal part for each divided cluster. The distribution range information of normal samples in space is calculated, and the second abnormality determination means determines whether the feature data projected by the projection means for each cluster is within the distribution range of normal samples, Another feature is that the feature data is determined to be abnormal only when the feature data is determined to be abnormal in all clusters.
The abnormality detection apparatus further includes a feature data cluster dividing unit that divides the feature data into a plurality of clusters, and the learning unit includes the normal subspace, the direction vector, and the spread angle for each divided cluster. The angle calculation means calculates angles between the plurality of normal subspaces obtained in advance and the feature data extracted by the feature data extraction means, and the first abnormality determination means and the The second abnormality determination means is characterized in that it determines whether or not each cluster is normal and determines that the feature data is abnormal only when the determination result is determined to be abnormal in all clusters.

  The abnormality detection method of the present invention includes a step of extracting feature data from input data, a step of calculating an angle between a normal subspace obtained in advance and the feature data extracted by the feature data extraction unit, A first abnormality determination step for determining an abnormality when the angle is larger than a predetermined value; a step for projecting feature data that has not been determined to be abnormal in the first abnormality determination step to the normal subspace; And a second abnormality determination step for determining whether the projected feature data is within the distribution range of the normal sample based on the distribution range information of the normal sample in the normal subspace. To do.

The present invention has the following effects.
(1) It is possible to determine an abnormality even for an event that has been determined to be normal even though it is abnormal in the past, and the accuracy of the abnormality determination is improved.
(2) The amount of calculation for feature extraction and abnormality determination is small, the amount of calculation is constant regardless of the number of subjects, and real-time processing is possible.
(3) Corresponding to various monitoring targets, the feature vector is not limited to CHLAC data, and any feature vector can be adopted.
(4) When CHLAC is employed and abnormality determination is performed by angle, abnormality can be detected robustly on the target scale.
(5) The present invention can be applied not only to abnormality detection but also as a recognition means in general recognition tasks (face recognition, human recognition, etc.)

  First, as an example of the present invention, the detection of abnormal operation using CHLAC data is disclosed as an example, but the present invention is not limited to CHLAC data, and any multidimensional feature data obtained from any target to be detected. From (feature vector), not only abnormal operation but also an abnormal state different from normal, that is, an event having a small occurrence probability can be detected with high accuracy.

  Next, although “abnormal” is defined, the abnormality itself cannot be defined so that all abnormal events cannot be listed. Therefore, in this specification, an abnormality is defined as “not normal”. If normal is a region where the distribution is concentrated when considering the statistical distribution of features, for example, it is possible to learn from the statistical distribution. Then, an operation that greatly deviates from the distribution is regarded as abnormal.

  For example, in a security camera installed in a passage or road, general movements such as walking are learned and recognized as normal movements. However, it is an event that has a very low probability of appearing, so it is recognized as an abnormality.

  As a specific method of anomaly detection, first, for example, a partial space of normal motion features is generated in the motion feature space using, for example, a three-dimensional higher-order local autocorrelation feature, and an abnormality is detected using the angle from the subspace as an abnormal value To do. A norm-normalized principal component analysis method is used to generate a normal motion subspace, and a principal component subspace is configured by a principal component vector having a cumulative contribution rate of 0.99, for example (step 1).

  Here, the three-dimensional higher-order local autocorrelation feature has the property that object extraction is unnecessary and additiveity is established in the screen. Due to this additivity, when a normal motion subspace is configured, the feature vector will fit in the normal motion subspace no matter how many people perform normal operation on the screen, and one of them will also perform abnormal operation Then, it jumps out of the partial space and can be detected as an abnormal value. Since there is no need to track and calculate each person individually, the amount of calculation is constant regardless of the number of subjects, and can be calculated at high speed.

  Furthermore, in the present invention, it is found that normal feature vectors are distributed in a region obtained by combining one or a plurality of (hypersolid) conical regions in the normal subspace. Whether or not a feature vector that is determined to be normal at an angle from is projected into the normal subspace within one of the one or more cone-shaped regions. If it is present inside any one of the cone shapes, it is determined to be normal, and if not, it is determined to be abnormal (step 2).

  FIG. 11 is an explanatory diagram showing the determination method (step 1) of the present invention. In FIG. 11, to simplify the explanation, the CHLAC feature data space is three-dimensional (actually, for example, 251 dimensions), and the normal subspace is two-dimensional (in the embodiment, for example, a cumulative contribution ratio of 0.99 is 3-12. The normal motion subspace obtained by principal component analysis exists in a form including CHLAC feature data of normal motion. The CHLAC feature data of the abnormal operation has an angle θ with the normal operation partial space, and therefore, abnormality is determined based on this angle θ.

  FIG. 12 is an explanatory diagram showing the determination method (step 2) of the present invention. In FIG. 11, for the sake of simplicity, the subspace of normal operation is two-dimensional (FIG. 12 (a)) or three-dimensional (FIG. 12 (b)). About 12 dimensions). The two-dimensional (FIG. 12A) is a more detailed display of the normal subspace of FIG.

  In the following, the case of the two-dimensional (a) will be described. The normal operation CHLAC feature data is projected onto the normal subspace, and the normalized vector has an expansion angle from the center of the fan in the normal operation subspace. It is distributed in a circular arc shape that intersects the fan shape (φ (super solid) cone shape in multi-dimension) and unit circle (unit (hyper sphere) in multi-dimension). Therefore, the direction vector γ indicating the center direction of the fan indicating normal operation and φ indicating the fan spread angle are obtained by learning, and the angle from the center of the fan shape of the CHLAC feature data z to be determined (vector γ) is determined. The abnormality is determined by determining whether this angle is larger than φ.

If there are multiple fan-shaped distributions indicating normal operation, first, based on the feature vector of normal operation, decompose into individual fan-shaped groups (clusters) and then enter any fan-shaped region It is determined whether or not.
Hereinafter, detection of abnormal operation using CHLAC data will be described as an example of an embodiment of the present invention.

  FIG. 1 is a block diagram showing a configuration of an abnormality detection apparatus according to the present invention. The video camera 10 outputs moving image frame data of a target person or device in real time. The video camera 10 may be a monochrome camera or a color camera. The computer 11 may be a known personal computer (PC) provided with a video capture circuit for capturing a moving image, for example. The present invention is realized by creating and installing a program to be described later on an arbitrary known computer 11 such as a personal computer.

  The monitor device 12 is a well-known output device of the computer 11 and is used, for example, to display to the operator that an abnormal operation has been detected. The keyboard 13 and the mouse 14 are well-known input devices used for input by the operator. In the embodiment, for example, moving image data input from the video camera 10 may be processed in real time, or may be temporarily stored after being stored in a moving image file and processed.

  FIG. 2 is a flowchart showing the processing contents at the time of detection of the abnormality detection processing of the present invention. In S10, it waits until the input of the frame data from the video camera 10 is completed, and the frame data is input (read into the memory). The image data at this time is, for example, 256 gray scale data. In S11, a feature vector is generated from the input frame data by a method described later. The feature vector may be, for example, 251 dimensional CHLAC feature data described later. In S12, the feature vector is normalized by a vector norm. That is, the feature vector is divided by the vector norm (vector length).

  In S13, an angle θ with the normal subspace is calculated. The normal subspace is obtained in advance by a learning process described later. When the orthogonal basis vector in the normal subspace is P, the projector P⊥ to the complementary space orthogonal to the normal subspace is expressed by Equation 1, and the sine of the angle θ to the subspace is expressed by Equation 2. x is a feature vector.

  Accordingly, the square of the sine of θ can be calculated using the following formula 3. y is the feature vector normalized by the vector norm in S12.

  In S14, it is determined whether or not the calculated angle θ is less than the angle threshold for abnormality determination. If the determination result is negative, the process proceeds to S15, but if the determination is affirmative, the process proceeds to S23. The threshold is determined during learning. The above is the first step of determination, and the steps after S15 are the second step.

  In S15, first, the feature vector is projected onto the normal subspace. If the orthogonal vector in the subspace is P, the projected vector z ′ is obtained by Equation 4.

  In S16, the scale of each element is adjusted so that the projected vector z 'has an isotropic distribution on the (hyper) spherical surface. If Λ is an eigenvalue diagonal matrix of principal component analysis in the learning process described later, the vector z ″ after the scaling process is expressed by Equation 5.

  In S17, normalization by a vector norm is performed. The normalized vector z is obtained by Equation 6.

  In S18, an unprocessed cluster is selected. The number of clusters, the direction vector γ of each cluster, and φ indicating the spread are obtained in advance by a learning process described later. In S19, an angle ψ between the feature vector z and the direction vector γ as shown in Equation 7 is calculated.

  In S20, it is determined whether or not the obtained angle ψ is less than φ, which is an abnormality determination threshold in the cluster. If the determination result is negative, the process proceeds to S21, but if the determination is affirmative, the process proceeds to S22. . In S21, it is determined whether or not the determination process has been completed for all clusters. If the determination result is negative, the process proceeds to S18, but if the determination is affirmative, the process proceeds to S23.

  In S22, “normal” is output, and in S23, “abnormal” is output. In S24, it is determined whether or not the end of the process is instructed. If the determination result is negative, the process proceeds to S10 and the process is repeated. By the above two-step processing, it is possible to perform abnormality determination with high accuracy and low processing load.

  FIG. 3 is a flowchart showing the processing contents during learning of the abnormality detection processing of the present invention. This process is executed prior to the detection process of FIG. In S30, sample frame data for motion learning is input. In S31, a feature vector is generated as in S11. In S32, normalization by a vector norm is performed as in S12.

  In S33, a subspace of normal operation is calculated by principal component analysis, and eigenvectors, eigenvalues, and abnormal index threshold values are output. As the threshold value, for example, the angle to the normal space is calculated for all the learning samples, and the average of n times (n is an arbitrary real number) or the maximum value is set as the threshold value. Since the principal component analysis method itself is well known, an outline will be described. First, in order to construct a normal operation subspace, principal component vectors are obtained from the CHLAC feature data group by principal component analysis. The M-dimensional CHLAC feature vector x is expressed as follows.

  Note that M = 251. A matrix U (eigenvector) in which principal component vectors are arranged in columns is expressed as follows.

  Note that M = 251. The matrix U in which the principal component vectors are arranged in columns is obtained as follows. The covariance matrix R is shown in the following equation.

  The matrix U is obtained from the eigenvalue problem of the following equation 11 using this covariance matrix R.

The diagonal matrix Λ of eigenvalues is expressed by the following equation 12,

The cumulative contribution rate αK up to the Kth eigenvalue is expressed by the following Expression 13.

  Here, the space spanned by the eigenvectors u1,..., UK up to a dimension where the cumulative contribution rate αK becomes a predetermined value (for example, αK = 0.99) is applied as a subspace of normal operation. Note that the optimum value of the cumulative contribution rate αK is determined by an experiment or the like because it is considered to depend on the monitoring target and detection accuracy. By performing the above calculations, a partial space for normal operation is generated.

  In S34, similar to S15, the feature vector of the sample data is projected onto the partial space. In S35, the scale is adjusted as in S16. In S36, normalization by a vector norm is performed as in S17. In S37, the projected feature vectors are clustered by the Mean Shift method.

  FIG. 6 is a flowchart showing the contents of the clustering process in S37. The inventors invented the Mean Shift method using the well-known von Mises-Fisher distribution (normal distribution on a spherical surface). This will be described below. In S70, one unprocessed sample point is selected as the initial value of the reference point ξ. In S71, the reference point ξ is updated based on the update formula shown in Formula 14. κ is a parameter representing the degree of concentration in the von Mises-Fisher distribution, and is set by experiment.

  In S72, it is determined whether or not ξ has changed, that is, whether or not the absolute value of the difference between ξ after the update and ξ before the update is equal to or smaller than a predetermined value. If yes, the process proceeds to S73. In S73, it is determined whether or not processing has been completed for all sample points. If the determination result is negative, the process proceeds to S70, but if the determination is affirmative, the process proceeds to S74. Through the above processing, the reference point ξ corresponding to each sample point is obtained. In S74, the points at close positions are integrated. That is, a set of samples having reference points whose difference between reference points ξ is equal to or less than a predetermined value is regarded as one cluster.

  Returning to FIG. 3, in S38, one unprocessed cluster is selected. In S39, a direction vector and a threshold value are obtained by a 1-class SVM (1-class Support Vector Machine) method.

  FIG. 7 is a flowchart showing the contents of the direction vector calculation process in S39. Since the 1-class SVM method itself is well known, an outline will be described. In FIG. 12B, the circle on the (hyper) spherical surface defined by the direction vector γ and the spread angle φ can be expressed as an intersection of the (hyper) sphere and the (hyper) plane. Therefore, if the projection point of the sample (normalized to norm 1) and the origin are separated and the (hyper) plane having the maximum distance from the origin is obtained, the normal vector to this hyperplane and this ( From the distance from the origin of the hyper) plane, the direction vector γ and the angle range φ are obtained. That is, this is equivalent to the problem of 1-class SVM, and the problem of 1-class SVM expressed by the following formula 15 can be solved. Equation 15 is a quadratic programming problem to be finally solved in the problem of 1-class SVM.

  Note that ν is a parameter designated by the user, and may be, for example, ν = 0.01. Specifically, the above problem is solved as follows. In S80, a variable t = 1 is set, and α is initialized at random within a range satisfying the condition of Expression 16.

  In S81, reserve variables O, SV, and ρ are calculated as shown in Equation 17.

  In S82, iεSV satisfying the condition of Expression 18 is selected.

  In S83, it is determined whether or not i is present. If the determination result is negative, the process proceeds to S84, but if the determination is affirmative, the process proceeds to S86. In S84, i∈1,..., N satisfying the condition of Expression 18 is obtained. In S85, it is determined whether or not i is present. If the determination result is negative, the process ends. If the determination is affirmative, the process proceeds to S86. In S86, j is obtained based on the following Equation 19. In S87, α is updated based on Equation 20 below.

  In S88, it is determined whether or not α satisfies the condition shown in Expression 21. If the determination result is negative, the process proceeds to S89, but if the determination result is affirmative, the process proceeds to S90.

In S89, αj is projected so as to fall within a predetermined range [0, 1 / νN]. That is, if α> 1 / νN, α = 1 / νN, and if α <0, α = 0 and move to the nearest point (end point). Then, αi is recalculated. In S90, t = t + 1 and the process proceeds to S81.
The normal vectors γ and φ of the hyperplane are obtained by the following Expression 22 using α and ρ obtained by the above processing.

In order to solve the (convex) quadratic programming problem by a computer, for example, the quadprog function of Optimization Toolbox of the scientific calculation software MATLAB (registered trademark) system sold by Cybernet System Co., Ltd. can be used. Further, the algorithm of the convex quadratic programming problem is described in Non-Patent Documents 1 and 2 below, for example.
Coleman, TF and Y. Li, OA Reflective Newton Method for Minimizing a Quadratic Function Subject to Bounds on some of the Variables, O SIAM Journal on Optimization, Vol. 6, Number 4, pp. 1040-1058, 1996. Gill, PE and W. Murray, and MH Wright, Practical Optimization, Academic Press, London, UK, 1981.

  Returning to FIG. 3, in S <b> 40, it is determined whether or not the processing has been completed for all clusters. If the determination result is negative, the process proceeds to S <b> 38, but if the determination is positive, the process ends.

  FIG. 4 is a flowchart showing the contents of the angle determination process. This process corresponds to S11 to S14. In S50, difference data is generated and binarized. In S50, "motion" information is detected for the moving image data, and difference data is generated for the purpose of removing stationary objects such as the background. As a method of taking the difference, an inter-frame difference method that extracts a change in luminance of a pixel at the same position between adjacent frames is adopted, but an edge difference that extracts a luminance change portion in a frame or both are adopted. May be.

Further, binarization is performed by automatic threshold selection to remove color information and noise unrelated to “movement”. As a binarization method, for example, a fixed threshold, a discriminative least square automatic threshold method disclosed in Non-Patent Document 3 below, threshold 0 and a noise processing method (all differences except for 0 in a grayscale image are set to have motion = 1). A method of removing noise by a known noise removing method can be employed. By the above preprocessing, the input moving image data becomes a string of frame data having logical values of “moved (1)” and “not moved (0)” as pixel values.
Noriyuki Otsu, “Automatic threshold selection method based on discriminant and least-squares criteria” IEICE Transactions D, J63-D-4, P349-356, 1980.

  In S51, a correlation pattern count process for pixel data for one frame is performed to generate frame CHLAC data. Although details will be described later with reference to FIG. 5, 251-dimensional feature vector data is generated. For extraction of motion features from time-series binary difference data, cubic higher-order local autocorrelation (CHLAC) features are used. The Nth-order autocorrelation function is expressed by the following equation 1.

  Here, f is a time-series pixel value (difference value), and N displacements ai (i = 1,..., N) viewed from the reference point (target pixel) r and the reference point are two in the difference frame. It is a three-dimensional vector with time as a component in dimensional coordinates. Further, the integration range in the time direction is a parameter indicating how much correlation in the time direction is taken. However, the frame CHLAC data in S51 is data for each frame, and the data obtained by integrating (adding) this for a predetermined period in the time direction is CHLAC feature data.

  There are an infinite number of higher-order autocorrelation functions depending on the direction of displacement and the order, but the higher-order local autocorrelation function is limited to a local region. In the three-dimensional high-order local autocorrelation feature, the displacement direction is limited to a 3 × 3 × 3 pixel local area centered on the reference point r, that is, in the vicinity of 26 of the reference point r. In the calculation of the feature amount, the integral value of Expression 1 becomes one feature amount for one set of displacement directions. Therefore, feature amounts are generated as many as the number of combinations of displacement directions (= mask patterns).

  The number of feature amounts, that is, the dimension of the feature vector corresponds to the type of mask pattern. In the case of a binary image, since the pixel value 1 is multiplied by 1 regardless of how many times it is, a term of square or higher is deleted as an overlapping term of a square with a different multiplier only. In addition, patterns that overlap in the integration operation (parallel movement: scan) of Equation 12 are deleted while leaving one representative pattern. Since the expression on the right side of Expression 1 always includes a reference point (f (r): the center of the local area), the representative pattern includes the center point, and the entire pattern fits within the local area of 3 × 3 × 3 pixels. select.

  As a result, the number of types of mask patterns including the center point is 352 with a total number of selected pixels of one: one, two: 26, three: 26 × 25/2 = 325. However, there are 251 types of mask patterns except for overlapping patterns in the integration operation (parallel movement: scan) of Equation 1. That is, the three-dimensional higher-order local autocorrelation feature vector for one three-dimensional data is 251 dimensions.

  When the pixel value is a multi-value gray image, for example, when the pixel value is a, the correlation value is a (0th order) ≠ a × a (primary) ≠ a × a × a (secondary). Thus, even if the selected pixels are the same, those having different multipliers cannot be redundantly deleted. Therefore, in the case of multi-value, the number is 2 when the number of selected pixels is 1 and 26 when the number of selected pixels is 2, and the number of mask patterns is 279 in total.

  The nature of the cubic higher-order local autocorrelation feature is additive to the data because the displacement direction is limited to the local region, and position invariance in the data because it is integrated throughout the entire screen.

  In S52, the frame CHLAC data is stored in correspondence with the frame. In S53, the latest frame CHLAC data obtained in S51 is added to the current CHLAC data, and new CHLAC data is generated by subtracting the frame CHLAC data corresponding to the frame older than a predetermined period from the current CHLAC data. ,save.

  FIG. 10 is an explanatory diagram showing the contents of real-time processing of a moving image according to the present invention. The moving image data is a sequence of frames. Therefore, a time window having a certain width is set in the time direction, and a frame set in the window is set as one three-dimensional data. Each time a new frame is input, the time window is moved, and the old frame is deleted to obtain finite three-dimensional data. The length of this time window is desirably set equal to or longer than one cycle of the operation to be recognized.

  Actually, only one frame is stored in order to obtain the difference in the image frame data, and the frame CHLAC data corresponding to the frame is stored for the time window. That is, when a new frame is input at time t in FIG. 10, the frame CHLAC data corresponding to the immediately preceding time window (t−1, t−n−1) has already been calculated. However, in order to calculate the frame CHLAC data, the last three difference frames are required, but since the (t-1) frame is the end, the frame CHLAC data is calculated up to the one corresponding to the (t-2) frame. Yes.

  Therefore, using the newly input t frame, frame CHLAC data corresponding to the (t−1) frame is generated and added to the CHLAC data. Also, the frame CHLAC data corresponding to the oldest (t-n-1) frame is subtracted from the CHLAC data. Through such processing, the CHLAC feature data corresponding to the time window is updated.

In S54 (= S12), normalization by a vector norm is performed. In S55 (= S13), as described above, the angle θ between the CHLAC data obtained in S53 and the partial space is obtained using the equation shown in Equation 3.
In S56 (= S14), it is determined whether or not θ is larger than the threshold value. If the determination result is negative, the process proceeds to S15 in FIG. 2, but if the determination result is affirmative, the process proceeds to S23 in FIG. .

  FIG. 5 is a flowchart showing the contents of the three-dimensional higher-order local autocorrelation feature extraction process in S51. In S60, the correlation pattern counter is cleared. In S60, 251 correlation pattern counters are cleared. In S61, one unprocessed pixel is selected (the pixel of interest is scanned in sequence within the frame). In S62, one unprocessed mask pattern is selected.

  FIG. 8 is an explanatory diagram showing autocorrelation processing coordinates in a three-dimensional pixel space. In FIG. 8, the xy planes of three difference frames temporally adjacent to each of the t−1 frame, the t frame, and the t + 1 frame are shown side by side. In the present invention, correlation is performed for pixels inside a cube of 3 × 3 × 3 (= 27) pixels centered on the reference pixel of interest.

  The mask pattern is information indicating a combination of pixels to be correlated, and the pixel data selected by the mask pattern is used for calculating the correlation value, but the pixels not selected by the mask pattern are ignored. As described above, the target pixel (center pixel) is always selected in the mask pattern. Further, when considering correlation values from the 0th order to the 2nd order in a binary image, the number of patterns after eliminating the overlap in a 3 × 3 × 3 pixel cube is 251.

  FIG. 9 is an explanatory diagram showing an example of an autocorrelation mask pattern. FIG. 9A shows the simplest 0th-order mask pattern of only the target pixel. (2) is a primary mask pattern example in which two hatched pixels are selected, (3) and (4) are tertiary mask pattern examples in which three hatched pixels are selected, There are many other patterns.

  Returning to FIG. 5, in S63, the correlation value is calculated using the above-described equation (21). F (r) f (r + a1)... F (r + aN) in equation 21 is multiplied by the pixel value of the difference binarized three-dimensional data of coordinates corresponding to the mask pattern (= correlation value, 0 or 1). It corresponds to that. Further, the integration operation of Expression 21 corresponds to moving (scanning) the pixel of interest within the frame and adding the correlation values with a counter corresponding to the mask pattern (counting 1).

  In S64, it is determined whether or not the correlation value is 1. If the determination result is affirmative, the process proceeds to S65, but if not, the process proceeds to S66. In S65, the correlation pattern counter corresponding to the mask pattern is incremented by one. In S66, it is determined whether or not processing has been completed for all patterns. If the determination result is affirmative, the process proceeds to S67, but if not, the process proceeds to S62.

  In S67, it is determined whether or not processing has been completed for all pixels. If the determination result is affirmative, the process proceeds to S68, but if not, the process proceeds to S61. In S68, a set of pattern counter values is output as 251 dimensional frame CHLAC data.

  Next, a description will be given of a second embodiment in which clustering is performed in advance and then a partial space is obtained by principal component analysis in each cluster. In the first embodiment, first, an angle with the subspace is obtained (step 1). When this is less than the threshold value, the feature vector is projected onto the subspace and clustering is performed, and determination is made based on the angle with the cone in each cluster. A method of performing is disclosed. However, if the feature vector has multimodality, a method of performing determination by angle in each cluster after performing clustering in step 1 is also conceivable.

  FIG. 13 is a flowchart showing the processing contents at the time of detection of the abnormality detection processing according to the second embodiment of the present invention. Since the individual processing contents are substantially the same as the processing of each step of the first embodiment shown in FIG. 2, only the correspondence is shown. In S110 to S112, the same processing as S10 to S12 is performed.

  In S113, an unprocessed cluster is selected. The number of clusters, the normal subspace of each cluster, the direction vector γ, and φ indicating the spread are obtained in advance by a learning process described later. In S114, the angle θ with the normal subspace is calculated as in S13.

  In S115, it is determined whether or not the calculated angle θ is less than the abnormality determination threshold value. If the determination result is negative, the process proceeds to S121, but if the determination result is affirmative, the process proceeds to S116. The threshold is determined during learning. The above is the first step of determination, and the steps after S116 are the second step.

  In S116, the feature vector is projected onto the normal subspace as in S15. In S117, as in S16, the scale of each element is adjusted so that the projected vector z ′ has an isotropic distribution on the spherical surface. In S118, normalization by the vector norm is performed as in S17. In S119, as in S19, an angle ψ between the feature vector z and the direction vector γ as shown in Equation 7 is calculated.

  In S120, as in S20, it is determined whether or not the obtained angle ψ is less than φ, which is an abnormality determination threshold in the cluster. If the determination result is negative, the process proceeds to S121. Shifts to S122. In S121, it is determined whether or not the determination process has been completed for all clusters. If the determination result is negative, the process proceeds to S113, but if the determination is affirmative, the process proceeds to S123.

  In S122, “normal” is output, and in S123, “abnormal” is output. In S124, it is determined whether the end of the process is instructed. If the determination result is negative, the process proceeds to S110 and the process is repeated.

  FIG. 14 is a flowchart showing the processing contents during learning of the abnormality detection processing according to the second embodiment of the present invention. This process is executed prior to the detection process of FIG. In S130 to S132, the same processing as S30 to S32 is performed. In S133, the feature vectors are clustered by the Mean Shift method as in S37. In S134, one unprocessed cluster is selected. In S135, as in S33, a subspace of normal operation is calculated by principal component analysis, and eigenvectors, eigenvalues, and abnormal index threshold values are output.

  In S136, similar to S34, the feature vector of the sample data is projected onto the partial space. In S137, the scale is adjusted as in S35. In S138, normalization by the vector norm is performed as in S36. In S139, a direction vector and a threshold are obtained by a 1-class SVM (1-class Support Vector Machine) method. In S140, it is determined whether or not processing has been completed for all clusters. If the determination result is negative, the process proceeds to S134, but if the determination is positive, the process ends.

  In the second embodiment, clustering is performed first, and then a partial space is obtained for each individual cluster (sample group) to determine abnormality. Therefore, normal sample data is divided into a plurality of groups, and one partial There is a possibility that the determination can be performed more accurately than in the first embodiment when the space cannot be taken.

  As described above, the embodiment for detecting the abnormal operation has been described. However, the present invention may be modified as follows. In the embodiment, an example in which a partial space for normal operation is generated in advance by the learning phase has been disclosed. However, abnormal operation is detected while updating the partial space for normal operation, or for example, 1 minute, 1 hour or Update the partial space of normal operation using a part of the data at a predetermined cycle longer than the frame interval such as one day, and detect the abnormal operation using the fixed partial space until the next update. May be. In this way, the processing amount is further reduced.

Furthermore, as a learning method for updating the subspace in real time, the eigenvector is approximated from the input data sequentially without solving the eigenvalue problem by principal component analysis as disclosed in Non-Patent Document 4 below. Alternatively, a method for automatically obtaining may be employed.
Juyang Weng, Yilu Zhang and Wey-Shiuan Hwang, "Candid Covariance-Free Incremental Principal Component Analysis", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.25, No.8, pp.1034-1040, 2003

It is a block diagram which shows the structure of the abnormality detection apparatus by this invention. It is a flowchart which shows the processing content at the time of the detection of the abnormality detection process of this invention. It is a flowchart which shows the processing content at the time of learning of the abnormality detection process of this invention. It is a flowchart which shows the content of an angle determination process. It is a flowchart which shows the content of the solid high-order local autocorrelation feature extraction process of S51. It is a flowchart which shows the content of the clustering process of S37. It is a flowchart which shows the content of the normal vector calculation process of S39. It is explanatory drawing which shows the autocorrelation process coordinate in a three-dimensional pixel space. It is explanatory drawing which shows the example of an autocorrelation mask pattern. It is explanatory drawing which shows the content of the real-time process of the moving image by this invention. It is explanatory drawing which shows the determination method (step 1) of this invention. It is explanatory drawing which shows the determination method (step 2) of this invention. It is a flowchart which shows the processing content at the time of the detection of the abnormality detection process of Example 2 of this invention. It is a flowchart which shows the processing content at the time of learning of the abnormality detection process of Example 2 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Video camera 11 ... Computer 12 ... Monitor apparatus 13 ... Keyboard 14 ... Mouse

Claims (8)

Feature data extraction means for extracting feature data from input data;
An angle calculation means for calculating an angle between a normal subspace obtained in advance and the feature data extracted by the feature data extraction means;
First abnormality determining means for determining an abnormality when the angle is greater than a predetermined value;
Projecting means for projecting feature data that has not been determined to be abnormal by the first abnormality determining means to the normal subspace;
Second abnormality determining means for determining whether or not the feature data projected by the projecting means is within the distribution range of normal samples based on the distribution range information of normal samples in the normal subspace obtained in advance; An abnormality detection device characterized by comprising.   Further, based on the input data for learning, the normal subspace is calculated based on a principal component vector obtained by a principal component analysis method from a plurality of feature data extracted by the feature data extraction means, and further the projection means 2. The abnormality detection apparatus according to claim 1, further comprising learning means for calculating distribution range information of normal samples in the normal subspace from the feature data projected onto the normal subspace.   The distribution range information of the normal sample in the normal subspace is direction vector information indicating the center of a cone that includes or approximates the distribution range of the normal sample in the normal subspace and angle information indicating the spread of the cone. The abnormality detection device according to claim 1, wherein   The feature data extracting unit generates inter-frame difference data from moving image data composed of a plurality of input image frame data, and operates by three-dimensional high-order local autocorrelation from three-dimensional data composed of the plurality of inter-frame difference data The abnormality detection apparatus according to claim 1, wherein characteristic data is generated.   The abnormality detection apparatus according to claim 1, wherein the projection unit includes a scale normalization unit that normalizes a scale of each element of the feature data after the projection. Furthermore, a cluster dividing means for dividing the distribution of the projected feature data in the normal space into a plurality of clusters,
The learning means calculates distribution range information of normal samples in the normal subspace for each divided cluster,
The second abnormality determining means determines whether or not the feature data projected by the projecting means for each cluster is within the distribution range of normal samples, and the feature data is determined to be abnormal in all clusters The abnormality detection apparatus according to claim 1, wherein the characteristic data is determined to be abnormal only in the case. Furthermore, a feature data cluster dividing means for dividing the feature data into a plurality of clusters is provided,
The learning means calculates distribution range information of normal samples in the normal subspace and the normal subspace for each divided cluster,
The angle calculation means calculates angles between a plurality of normal subspaces obtained in advance and the feature data extracted by the feature data extraction means,
The first abnormality determination unit and the second abnormality determination unit determine whether each cluster is normal, and determine that the feature data is abnormal only when the determination result is determined to be abnormal in all clusters. The abnormality detection device according to claim 1, wherein: Extracting feature data from the input data;
Calculating an angle between a normal subspace obtained in advance and the feature data extracted by the feature data extraction unit;
A first abnormality determination step for determining an abnormality when the angle is greater than a predetermined value;
Projecting feature data not determined to be abnormal in the first abnormality determination step onto the normal subspace;
A second abnormality determination step for determining whether or not the projected feature data is within the distribution range of the normal sample based on the distribution range information of the normal sample in the normal subspace obtained in advance; Characteristic abnormality detection method.

Images