JP2018173914A - Image processing system, imaging apparatus, learning model creation method, information processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system capable of detecting an abnormality at an early stage after installation. An image processing system 100 that detects an abnormality from an image captured by an image pickup device (10), and image acquisition means (31, 32) for acquiring the image from a plurality of first image pickup devices installed at different installation locations. A classification means 33 that classifies the images having similar characteristics into clusters, and a learning means 34 that learns the images classified into the clusters and constructs an abnormality determination means for detecting an abnormality for each cluster. The abnormality determination means constructed from the cluster is determined by determining the cluster in which the image similar to the image captured by the second image pickup device installed at an installation location different from the first image pickup apparatus is classified. It has a determination means 35 for determining the above. [Selection diagram] Fig. 2

Description

  The present invention relates to an image processing system, an imaging apparatus, a learning model creation method, and an information processing apparatus.

  There is known a monitoring method in which a video is captured and recorded by an imaging device such as a monitoring camera, and a related person checks the video later to check whether there is an abnormality and what kind of abnormality has occurred. However, there are cases in which the person concerned cannot confirm the image unless he / she notices the necessity of confirming the image, and can only confirm the presence or absence of abnormality from the image after the fact. In addition, the presence or absence of an abnormality cannot be confirmed unless the person concerned sees all the images.

  For this reason, a technique for analyzing an image in real time to detect an abnormality has been devised (for example, see Patent Document 1). Abnormality detection refers to detecting a phenomenon that deviates from a normal state that can occur every day. Patent Document 1 discloses a method for suppressing congestion by performing suspicious person determination before collecting images captured by a camera and transmitting only the target image.

  However, Patent Document 1 is a technique for determining that an abnormality is detected when a person other than a registered resident is detected. If the registered content is not appropriate, the abnormality detection result may not be accurate.

  For such inconvenience, a technique has been devised in which an information processing apparatus learns various normal videos by machine learning and detects an abnormality from an image captured at a place where a camera is installed (for example, Patent Documents). 2). Patent Document 2 discloses a method of selecting information suitable for learning by further selecting information by a learner or learning agent from information narrowed down by an online learning unit.

JP 2013-222216 A Japanese Patent Laid-Open No. 2006-173682

  However, with the conventional technology, it is necessary to learn a normal image at the installation location in order for the newly installed camera to detect an abnormality, so it is difficult to start abnormality determination immediately after installation. was there. For example, when a suspicious person is detected using a learning model that has been machine-learned by an image processing system, a learning model that matches the scene captured by the camera is required, but an image that is used for learning is newly installed in the camera. Since there is no data, an abnormality cannot be determined until images used for learning are accumulated.

  In view of the above problems, an object of the present invention is to provide an image processing system capable of detecting an abnormality early after installation.

  The present invention is an image processing system that detects an abnormality from an image captured by an imaging device, and has features similar to those of an image acquisition unit that acquires the image from a plurality of first imaging devices installed at different installation locations. Classifying means for classifying the images into clusters, learning means for learning the images classified into the clusters and detecting abnormality for each cluster, learning means, and the first imaging device, Determining the cluster into which the image similar to the image captured by the second imaging device installed at a different installation location is classified, and determining means for determining the abnormality determination means constructed from the cluster; Have

  According to the present invention, it is possible to provide an image processing system capable of detecting an abnormality early after installation.

It is an example of the figure explaining the learning of the image by machine learning, and abnormality detection. It is an example of the figure explaining the outline of image processing. 1 is an example of a system configuration diagram of an image processing system. It is an example of the hardware block diagram of a camera apparatus. It is the block diagram which showed the schematic hardware constitutions of information processing apparatus. It is an example of the functional block diagram which illustrated each function with which an image processing system is provided. It is an example of the flowchart which shows the process sequence of clustering. The flowchart which shows the procedure in which a cluster setting part extracts a local feature-value and a global feature-value is shown. It is a figure which shows the classification | category by a K-means method typically. It is an example of the figure explaining clustering. It is an example of the figure explaining the isolation | separation degree R. FIG. It is an example of the figure explaining the learning method of SAE. It is a figure which shows typically the learning model which determines the presence or absence of abnormality from an image. It is an example of the figure explaining the procedure of learning of FIG. 13 in detail. It is an example of the functional block diagram of the information processing apparatus regarding abnormality determination. It is an example of the figure which illustrates the abnormality detection of the existing installation place typically. It is an example of the figure which illustrates the abnormality detection of a new installation place typically. It is an example of the figure which illustrates typically the processing which a judgment result processing part performs. It is an example of the figure explaining a structure in case a camera apparatus determines abnormality.

  Hereinafter, as an example of an embodiment for carrying out the present invention, an image processing system and a learning model creation method performed by the image processing system will be described with reference to the drawings.

<Specific examples of abnormality detection>
In describing this embodiment, a specific example of abnormality detection will be described. Abnormality is a phenomenon that deviates from normal that can occur on a daily basis.

FIG. 1A is an example of a diagram illustrating learning of a normal image 6 by machine learning.
(1) A large amount of images 6 (normal images 6) that do not include an abnormality are prepared. This image 6 is assumed to be a moving image. “Not including an abnormality” means that the contents shown are not subject to warning. Whether it is normal or not depends on the location where the camera is installed and the subject. For example, in the image 6 in which the pedestrian is shown, the image in which the pedestrian walks is a normal image, and the image 6 that gets over the wall is abnormal. In addition, in the image 6 showing the car, the image 6 in which a person gets on and off, the car stops, parks, and starts is a normal image 6, and the image 6 that shows the action to release the door lock is abnormal. . In the case of a bicycle, images different from normal and abnormal can be obtained as in a car.
(2) The learning unit 34 uses a machine learning such as deep learning or SVM (support vector machine) to learn a range in which the shape and motion of the target in the normal image 6 are normal, and constructs a learning model. . The learning model is an estimation of this normal range, and outputs whether the shape and movement of the input target are normal.
(3) An example of a learning model will be described. For example, in the image 6 showing a pedestrian, the pedestrian moves while the background is stationary, so the learning unit 34 automatically generates the pedestrian's posture, moving speed, and the like. And normal ranges, such as this pedestrian's attitude | position and moving speed, are estimated. Note that the learning unit 34 does not need a teacher signal such as a pedestrian, and does not need to recognize that the moving object is a pedestrian (person). The learning unit 34 merely quantifies a normal behavior pattern (moving speed here) of a certain shape (here, a pedestrian) as such.

FIG. 1B is an example of a diagram illustrating the abnormality determination performed by the identification device 5.
(4) An image 6 in which a person gets over the wall is captured as an abnormal image 6. A person rarely gets over the wall and is not included in the normal image 6.
(5) The identification device 5 performs abnormality determination by comparing with the learning model created in (3).
(6) For example, it is determined that the degree of coincidence of human postures is 20% and the degree of coincidence of moving speeds is 60%. When the degree of coincidence with the normal image 6 is low, the identification device 5 determines that the possibility of abnormality is high.

  As described above, the learning unit 34 learns a large amount of normal images 6, so that it is possible to construct an identification device that determines that the image 6 is abnormal as in the case where a person makes a determination. For this reason, the image processing system according to the present embodiment can appropriately respond such as detecting an abnormality in real time before the intruder enters the facility and dispatching a guard.

  However, on the other hand, the presence or absence of abnormality cannot be determined until a large amount of normal images 6 are learned. As described above, whether or not the image 6 is normal differs depending on the location where the camera is installed and the object that is captured, so that the normal image 6 used for learning by other cameras can be used for learning a newly installed camera. Even if diverted as it is, an appropriate learning model is not always constructed.

<Outline of Image Processing of this Embodiment>
Therefore, the image processing system according to the present embodiment learns by clustering the images 6 collected from a large number of existing installation locations with similar scenes, and a new learning model whose scenes are similar to the images 6 of the new installation locations. Used for the learning model of the installation location. Since the learning model of the existing installation location can be used, monitoring can be started immediately after the camera is installed at the new installation location.

  FIG. 2 is an example of a diagram for explaining the outline of the image processing of the present embodiment. The following description will be divided into a preparation phase before installation of the camera device and an abnormality determination phase based on the image 6 captured by the new camera device.

Preparation phase (1) There are already a plurality of existing installation locations 8a, 8b, 8c, 8d. When distinguishing a plurality of existing installation locations 8, an explanation will be given by adding an alphabetical character to the reference numeral 8 (the number of existing installation locations is not particularly limited). An image 6 captured by these camera devices 10 (an example of a first imaging device) is stored in the image storage unit 32. The image storage unit 32 stores the images 6 captured by the camera devices 10 at all the existing installation locations 8.
(2) The cluster setting unit 33 clusters the stored images 6 for each similar scene. By clustering, for example, images where the installation location of the camera device 10 and the captured object are similar are classified into the same cluster.
(3) Machine learning units 1 to n
The machine learning units 1 to n perform machine learning on the clustered normal images 6. In FIG. 2, there are as many machine learning units 1 to n as the number of clusters (n), but it is sufficient that there is at least one machine learning unit 1 to n. Each of the machine learning units 1 to n constructs a learning model. Each learning model is used in the abnormality determination unit 36.
(4) Next, the cluster determination unit 35 selects the cluster in which the image 6 captured by the camera device 10 (an example of the second imaging device) at the new installation location 7 is classified as the image 6 at the existing installation location 8. It is determined whether it is classified, and it is determined to apply the learning model of the cluster to be classified. Thus, it can be determined immediately after the appropriate learning model is installed in the newly installed camera device 10.

Abnormality determination phase (5) The abnormality determination unit 36 corresponding to the camera device 10 at the existing installation location 8 determines whether the image 6 is normal or abnormal using the learning model. For example, a learning model suitable for the existing installation location 8a is automatically determined by the clustering result of the image 6 of the existing installation location 8a. The fact that the learning model of the machine learning unit 1 is provided to the existing installation locations 8a and 8b in FIG. 2 indicates that the images 6 of the existing installation locations 8a and 8b are clustered into the same cluster.
(6) The abnormality determination unit 36 corresponding to the camera device 10 at the new installation location 7 also determines whether the image 6 is normal or abnormal using the learning model determined in (4).
(7) The determination result processing unit 37 acquires the determination result of the presence / absence of an abnormality from each abnormality determination unit 36, instructs the dispatch of guards, or transmits the image 6 to the customer.

  In this way, the learning model learned from the image 6 of the camera device 10 at the existing installation location 8 is obtained by clustering the image 6 of the newly installed camera device 10 into a cluster similar to the image of the existing installation location 8. Therefore, it is possible to determine whether there is an abnormality immediately after the new camera device 10 is installed. Also, for the existing installation location 8, there is a possibility that a more appropriate learning model can be created than the learning model learned from the image of one installation location.

<Terminology>
A cluster means a cluster. In this embodiment, the cluster is a collection of images having similar scenes. In addition to clusters, they may be referred to as classes, groups, categories, and the like.

  Learning refers to extracting useful rules, rules, knowledge expressions, criteria, or the like from data in order to realize the same functions as the learning ability performed by humans with a computer.

<System configuration example>
FIG. 3 shows an example of a system configuration diagram of the image processing system 100. The image processing system 100 mainly includes a camera device 10 and an information processing device 30 connected via a network N. One or more camera devices 10 are installed in one existing installation location 8. “Existing” means an installation location where a sufficient number of images 6 have already been captured. One or more camera devices 10 are also installed in the new installation place 7. The new installation place 7 is a place where the camera apparatus 10 is newly installed without the camera apparatus 10 in the past.

  The camera device 10 is an imaging device that captures several or more images 6 per second, and can capture a so-called moving image. An existing installation site or a new installation site is a place where intrusion of a third party can be mainly monitored. Examples include, but are not limited to, facilities, personal residences, parking lots, forests, coastal areas, and restricted areas. Moreover, it is not limited to the outdoors, but may be installed indoors such as facilities, offices, private residences, hotels, stores, research laboratories, warehouses, factories, and the like.

  The camera device 10 preferably has not only a function for capturing an image but also an image processing function for detecting a moving object, but is not essential in the present embodiment. The camera device 10 captures several images 6 per second and transmits them to the monitoring center.

  The network N is constructed by a telephone line, a LAN, a provider network of a provider that connects the LAN to the Internet, a line provided by a line operator, and the like that are installed at the installation location of the camera device 10. When the network N has a plurality of LANs, the network N is called WAN and may include the Internet. The network N may be constructed by either wired or wireless, and wired and wireless may be combined. Further, when the camera apparatus 10 is connected to a wireless public line network such as 3G, 4G, 5G, LTE (Long Term Revolution), etc., it can be connected to a provider network without going through a wired telephone line or LAN. .

  An information processing device 30 is installed in the monitoring center 9. The information processing apparatus 30 constructs a learning model or performs abnormality determination. The determination result that it is abnormal is transmitted together with the image 6 to the customer's mobile terminal or transmitted to the monitor's terminal. The customer or the monitor checks the image 6 determined to be abnormal, and if necessary, dispatches a guard to the installation location or reports to the police.

  The information processing apparatus 30 is an apparatus called a general computer, a PC (Personal Computer), a server, or the like. In FIG. 3, for convenience of explanation, one information processing apparatus 30 is shown. However, the information processing apparatus 30 is not limited to one, and a plurality of information processing apparatuses 30 share the functions described in this embodiment. You may have.

<Hardware configuration example>
<< Example hardware configuration of camera device >>
FIG. 4 is an example of a hardware configuration diagram of the camera apparatus 10. The camera device 10 includes an imaging unit 101, an image processing IC 103, a ROM 105, a CPU 106, a RAM 107, and a communication device 108.

  The imaging unit 101 is a camera having a lens, a diaphragm, a shutter (mechanical / electronic), a solid-state imaging device such as a CMOS or a CCD, and a memory. The image 6 may be monochrome or color. The imaging unit 101 periodically captures a predetermined range under the control of the CPU 106 and sends the image 6 to the image processing IC 103. The image processing IC 103 is an integrated circuit that performs image processing for detecting a moving object on the image 6.

  The CPU 106 controls the entire camera device 10 by executing a program stored in the ROM 105 using the RAM 107 as a working memory. That is, the imaging by the imaging unit 101 is controlled. Further, the communication device 108 is controlled to transmit the image 6 to the monitoring center 9.

  The communication device 108 is called a network interface or an Ethernet (registered trademark) card, and provides a function of connecting to a network. You may connect to the access point of a wireless LAN, or the base station of a mobile telephone network.

<< Hardware configuration example of information processing apparatus 30 >>
FIG. 5 is a block diagram illustrating a schematic hardware configuration of the information processing apparatus 30. The information processing apparatus 30 can be generally implemented as a personal computer, a workstation, or an appliance server. The information processing apparatus 30 includes a CPU 201 and a memory 202 that enables high-speed access to data used by the CPU 201. The CPU 201 and the memory 202 are connected to other devices or drivers of the information processing apparatus 30, for example, a graphics driver 204 and a network driver (NIC: Network Interface Card) 205 via a system bus 203. .

  An LCD (display device) 206 is connected to the graphics driver 204 and monitors the processing result of the CPU 201. A touch panel may be integrally disposed on the LCD 206. In this case, the user can operate the information processing apparatus 30 using a finger as an operation means.

  The network driver 205 connects the information processing apparatus 30 to the network N at the transport layer level and the physical layer level to establish a session with the camera apparatus 10 and the like.

  An I / O bus bridge 207 is further connected to the system bus 203. On the downstream side of the I / O bus bridge 207, a storage device such as an HDD 209 is connected by an IDE, ATA, ATAPI, serial ATA, SCSI, USB, etc. via an I / O bus 208 such as PCI. . Instead of the HDD 209 or together with the HDD 209, an SSD (Solid State Drive) may be included.

  The HDD 209 stores a program 209p for controlling the entire information processing apparatus 30. The information processing apparatus 30 executes the program 209p to accept or learn the operation of the supervisor. The program 209p may be distributed in a state of being stored in a portable storage medium such as a USB memory or an optical storage medium in addition to being distributed from a server that distributes the program.

  An input device 210 such as a keyboard and a mouse (referred to as a pointing device) is connected to the I / O bus 208 via a bus such as a USB, and receives inputs and commands from an operator.

  The information processing apparatus 30 may support cloud computing. Cloud computing refers to a usage pattern in which resources on a network are used without being aware of specific hardware resources. In this case, the hardware configuration shown in FIG. 5 does not need to be housed in a single housing or provided as a single device, and is preferably provided in the information processing device 30. Indicates an element.

<Functional Configuration Example of Image Processing System 100>
FIG. 6 is an example of a functional block diagram illustrating each function provided in the image processing system 100. In FIG. 6, functions related to the construction of the learning model are mainly described. Further, it is assumed that the function of the camera apparatus 10 is common in each installation place or that there is no problem in the description of the present embodiment even if the functions are different. For this reason, only one camera device 10 is shown in FIG.

<< Functional configuration of camera device >>
The camera device 10 includes an image acquisition unit 11, an image processing unit 12, and a communication unit 13. Each of these functions is a function or means realized by the CPU 106 shown in FIG. 4 executing a program and cooperating with the hardware of the camera apparatus 10. It is not necessary to clearly distinguish functions implemented in hardware or software, and some or all of these functions may be implemented by a hardware circuit such as an IC.

  Further, the camera device 10 has a storage unit 19 constructed by the ROM or RAM 107 shown in FIG. An image DB 191 is constructed in the storage unit 19. The image DB 191 does not have to be directly included in the camera apparatus 10 and may be in any place on the network that the camera apparatus 10 can access.

  The image acquisition unit 11 periodically acquires (images) the image 6 regardless of whether there is an abnormality. Moving images can be captured by capturing images at sufficiently short time intervals. Note that the imaging interval is not necessarily constant, and the imaging interval may be changed according to a time zone, a moving object detection result, or the like. The image acquisition unit 11 is realized by the CPU 106 in FIG. 4 executing a program and controlling the imaging unit 101.

  The image processing unit 12 performs image processing effective for clustering. For example, it is determined whether the imaging time is daytime or nighttime from the average luminance or the imaging time of the image 6 and attached to each image 6 as a label. Alternatively, processing such as noise removal, trimming, and higher resolution may be performed. The image processing unit 12 stores the captured image 6 in the image DB 191. In the image DB 191, an old image 6 is overwritten and a constant new image 6 is always held. The image processing unit 12 is realized by the CPU 106 in FIG. 4 executing a program or the like.

  The communication unit 13 transmits the image 6 stored in the image DB 191 to the information processing apparatus 30. In order to guarantee the real-time property of abnormality detection, it is preferable to immediately transmit the stored image 6 to the information processing apparatus 30. For this reason, the communication unit 13 continuously transmits the images 6 to the information processing device 30. However, the communication unit 13 may transmit the images 6 together after a certain amount of the images 6 is accumulated. The communication unit 13 is realized by the CPU 106 in FIG. 4 executing a program and controlling the communication device 108.

<< Functional configuration of information processing equipment >>
The information processing apparatus 30 includes a communication unit 31, an image storage unit 32, a cluster setting unit 33, a learning unit 34, and a cluster determination unit 35. These functions are functions or means realized by the CPU 201 shown in FIG. 5 executing the program 209p and cooperating with the hardware of the information processing apparatus 30. It is not necessary to clearly distinguish functions implemented in hardware or software, and some or all of these functions may be implemented by a hardware circuit such as an IC.

  The information processing apparatus 30 includes a storage unit 39 constructed from the memory 202 or the HDD 209 illustrated in FIG. In the storage unit 39, an image accumulation DB 391 and a learning model DB 392 are constructed. These DBs do not have to be directly held by the information processing apparatus 30 and may be located at any location on the network accessible by the information processing apparatus 30.

  The communication unit 31 receives the image 6 from the camera device 10. The communication unit 31 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p to control the network driver 205, and the like.

  The image storage unit 32 associates the image 6 received by the communication unit 31 with the identification information of the camera device 10 (or the identification information of the installation location when there is only one camera device 10 at the installation location). To remember. The image storage unit 32 is realized by the CPU 201 shown in FIG. 5 executing the program 209p.

  The cluster setting unit 33 extracts feature amounts from the images 6 stored in the image storage DB 391, and clusters the images 6 transmitted from the respective camera devices 10 into clusters having similar scenes using the feature amounts. The scene can be said to be a broad scene where the image 6 is captured. For example, the image of the camera device 10 installed in a parking lot is likely to be similar in the image 6 of a different parking lot. When clustering, the best number of clusters is determined. In addition, since there is a strong tendency for the features to be different between daytime and nighttime, there is a high possibility that even in the image 6 of the same camera device 10, clusters will be divided separately during the daytime and nighttime. Furthermore, if the above label is used, it can be surely clustered into clusters separately during the day and at night. The cluster setting unit 33 is realized by the CPU 201 shown in FIG. 5 executing the program 209p.

  The learning unit 34 further includes a machine learning unit 1, machine learning units 2,..., A machine learning unit n (n is a natural number). 1 to n of the machine learning units 1 to n correspond to the number (n) of clusters formed by clustering. However, one machine learning unit (for example, the machine learning unit 1) may learn the images 6 of all the clusters, and there may be one machine learning unit 1 to n. The machine learning unit 1 stores the learning model C1 constructed by learning in the learning model DB 392, the machine learning unit 2 stores the learning model C2 constructed by learning in the learning model DB 392, and the machine learning unit n performs learning constructed by learning. The model Cn is stored in the learning model DB 392. Hereinafter, for convenience of explanation, there may be a case where the “learning unit 34” simply constructs a learning model. The learning unit 34 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p.

  The cluster determination unit 35 determines a cluster on which the image 6 captured by the camera device 10 at the new installation location 7 is clustered from the clusters generated by the cluster setting unit 33. For example, the cluster having the closest feature quantity is selected. The correspondence between the camera device 10 and the learning model (cluster) is registered in the learning model DB 392. Even when the camera apparatus 10 does not attach a label for daytime or nighttime, it is preferable to determine a cluster corresponding to each of daytime and nighttime by using the fact that the feature amount is different between daytime and nighttime. Note that whether the image 6 was captured during the day or at night can also be determined by the information processing apparatus 30 side. The cluster determination unit 35 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p.

  Further, the cluster determination unit 35 determines a cluster in which the images 6 captured by the camera device 10 at the existing installation location 8a are clustered. Also in this case, it is preferable that the cluster determination unit 35 determines a corresponding cluster in the daytime and at night.

表１は、学習モデルＤＢ３９２に格納される情報を模式的に示す。学習モデルＤＢ３９２は、クラスタＩＤに対応付けて、学習モデル、カメラＩＤ、及び、日中／夜間の各項目を有する。クラスタＩＤは、最終的にクラスタリングされた各クラスタを特定する情報である。ＩＤはIdentificationの略であり識別子や識別情報という意味である。ＩＤは複数の対象から、ある特定の対象を一意的に区別するために用いられる名称、符号、文字列、数値又はこれらのうち１つ以上の組み合わせをいう。他のＩＤについても同様である。カメラＩＤは各カメラ装置１０を特定する情報であり、学習モデルと対応付けられている。日中／夜間の項目は各カメラ装置１０の日中の画像６と夜間の画像６のそれぞれがどの学習モデルと最も近いかを示す。これにより、既存設置場所８及び新規設置場所７の各カメラ装置１０が撮像した画像６をどの学習モデルで異常判定すればよいか決定される。

Table 1 schematically shows information stored in the learning model DB 392. The learning model DB 392 has items of learning model, camera ID, and daytime / nighttime in association with the cluster ID. The cluster ID is information for identifying each cluster finally clustered. ID is an abbreviation for Identification and means an identifier or identification information. ID refers to a name, a code, a character string, a numerical value, or a combination of one or more of these used to uniquely distinguish a specific target from a plurality of targets. The same applies to other IDs. The camera ID is information for specifying each camera device 10 and is associated with a learning model. The item of daytime / nighttime indicates which learning model is closest to each of the daytime image 6 and the nighttime image 6 of each camera device 10. Thereby, it is determined which learning model should be used to determine whether the image 6 captured by each camera device 10 at the existing installation location 8 and the new installation location 7 is abnormal.

<Necessity of clustering>
Explain the necessity of clustering. First, as a premise, the types of installation locations where the camera device 10 is installed are limited, and normal behavior (behavior patterns such as pedestrians) conceivable at the locations may also be limited. Table 2 shows an example of normal behavior at each installation location.

表２に示すように、設置場所が駐車場、家の玄関、建物外周のそれぞれで正常行動は幾つかに限定される。このため、画像６をクラスタリングすることで特徴が類似した画像を集めることができ、類似した画像から学習モデルを構築することで、異常判定の精度を向上できると期待できる。

As shown in Table 2, normal behavior is limited to several in each of the installation locations of the parking lot, the entrance of the house, and the outer periphery of the building. For this reason, it can be expected that images having similar features can be collected by clustering the images 6 and the accuracy of abnormality determination can be improved by constructing a learning model from the similar images.

<Clustering process>
Next, clustering will be described in detail with reference to FIG. FIG. 7 is an example of a flowchart showing a clustering processing procedure. Hereafter, each step of FIG. 7 is demonstrated in order.

(S1)
The cluster setting unit 33 acquires a background image from which noise has been removed from each image 6. For example, a known smoothing filter may be used to remove noise.

(S2)
The cluster setting unit 33 extracts local feature values and global feature values from the background image. The local feature value and the global feature value will be described with reference to Table 3.

表３は、局所的特徴量と大域的特徴量の利用目的と判定できないことがそれぞれまとめられたものである。局所的特徴量では画像６の一部（局所）の特徴が得られるが、シーンを分類することが得意でない。例えば、車が何台かあっても道路なのか駐車場なのかの特徴が得られない。大域的特徴量はベランダなのか駐車場なのかという画像全体の特徴が得られるが、駐車場と住宅のガレージを分類できない可能性がある。

Table 3 summarizes that the purpose of use of local feature values and global feature values cannot be determined. With the local feature amount, a part (local) feature of the image 6 can be obtained, but it is not good at classifying the scene. For example, even if there are several cars, the feature of whether it is a road or a parking lot cannot be obtained. Global features can be obtained from the overall image of whether it is a veranda or a parking lot, but there is a possibility that parking and residential garages cannot be classified.

  Thus, since the local feature quantity and the global feature quantity have a complementary relationship, it is effective to extract both the local feature quantity and the global feature quantity in order to appropriately classify the scene.

  FIG. 8 is a flowchart showing a procedure by which the cluster setting unit 33 extracts local feature values and global feature values.

  S2-1: First, the cluster setting unit 33 detects a local feature amount. In the present embodiment, a SIFT (Scale-Invariant Feature Transform) feature quantity is acquired as an example of a local feature quantity. The SIFT feature quantity is a feature quantity that is robust to illumination change, rotation, enlargement, and reduction using a scale space, and a plurality of 128-dimensional feature points can be acquired from one image 6. The cluster setting unit 33 first detects a key point. The key point is a point that appears characteristically on the image 6 even if the scale changes. For detection, an Gaussian filter with different resolutions is applied step by step to create an image 6 with different resolutions, and an extreme value of a DoG (Difference of Gaussian) image is searched. The central part of the local area obtained by the search is a key point.

  S2-2: Next, the cluster setting unit 33 detects the strength of the density gradient and the direction of the gradient with respect to the vicinity of the key point (extracts features). The gradient strength and gradient of each key point are SIFT feature values. The cluster setting unit 33 calculates SIFT feature values for all the images 6 to be classified.

  S2-3: Next, the cluster setting unit 33 classifies the local feature amounts for all the images 6 to be classified. All the images 6 are all the images 6 of the existing installation place 8.

FIG. 9A schematically shows classification by the K-means method. FIG. 9A shows each feature point in three dimensions, but actually has dimensions of 128 dimensions (SIFT feature value) × number of feature points obtained from one image × number of images. One point in FIG. 9A represents one feature point. In the K-means method, each feature point is classified into each cluster having an arbitrarily given number of clusters. First, the cluster number k is calculated by the following equation.
Number of clusters k = √ {128 dimensions (SIFT feature amount) × number of feature points obtained from one image × number of images}
Next, the cluster setting unit 33 classifies all the images 6 into clusters of this number of clusters. The initial center of gravity of the cluster is randomly assigned, each feature point is assigned to the cluster with the nearest center of gravity, the center of gravity is recalculated, and the process of assigning each feature point to the cluster with the nearest center of gravity is performed again. Repeat until there is no more. FIG. 9B schematically shows each feature point classified into three clusters 71.

S2-4: Next, the cluster setting unit 33
Quantize by determining words. That is, the center (center of gravity) of each cluster is determined as visual words 72. Thereby, k visual words which are the number of clusters are specified. FIG. 9C schematically shows three visual words 72 when k = 3 as an example. The number of visual words 72 is the same as the number of clusters, and the number of dimensions of each visual word 72 is 128 dimensions (SIFT feature amount) × number of feature points obtained from one image × number of images.

S2-5: Next, the cluster setting unit 33
All images 6 to be clustered are converted into feature vectors having words 72 as a dimension. The closest visual words 72 are searched for each local feature amount of one image 6 and voted for the visual words 72. Thereby, a histogram of one image 6 is obtained. FIG. 9D schematically shows a histogram. The horizontal axis of FIG. 9D is visual words, and the vertical axis is the number of local features (SIFT feature amounts) determined to be closest to each visual word in an image 6. This histogram becomes a feature vector of each image 6.

  S2-6: Next, the cluster setting unit 33 detects a global feature amount. In the present embodiment, a GIST feature value is acquired as an example of a global feature value. The GIST feature amount is a feature amount that describes scene information by dividing the image 6 into small regions and applying Gabor filters of various directions and frequencies to the small regions, and has 960-dimensional features. Since one GIST feature value is obtained from one image 6, it is not necessary to create a histogram.

  S2-7: Next, the cluster setting unit 33 combines the local feature quantity and the global feature quantity to generate a final feature quantity. Therefore, the number of dimensions is the number of visual words + 960 dimensions.

  S2-8: The cluster setting unit 33 outputs the feature amount.

(S3, S4)
Next, the cluster setting unit 33 clusters the feature amounts into n clusters. Here, n is not a fixed value, and the best number of clusters is dynamically determined.

  FIG. 10 is an example of a diagram illustrating clustering. FIG. 10A schematically shows a set of each image 6 converted into a feature amount in step S2. In order to facilitate the calculation, one image 6 is required for one camera device 10.

  For clustering, in addition to the K-means method used for the classification of local features, an EM algorithm using a Gaussian mixture model or a method such as LDA (Latent Dirichlet Allocation) may be used. Note that the number of clusters cannot be determined by the K-means method and the EM algorithm (need to be given as appropriate). On the other hand, with LDA, clustering and the number of clusters can be determined simultaneously. In the present embodiment, as an example of clustering by the K-means method or the EM algorithm, the optimum number of clusters is determined using a Stacked Auto Encoder described later.

  In order to determine the best number of clusters, the cluster setting unit 33 performs clustering by changing the number of clusters i. In FIG. 10, clusters of i = 5, 6, 100 are schematically shown. The cluster setting unit 33 performs clustering with the number of clusters i = 5 to 100. FIG. 10B schematically shows the results of clustering with the number of clusters i = 5, 6 and 100.

  When setting the number of clusters, the number of clusters i may be increased by one, or the number of clusters i may be increased by, for example, 10 and evaluated by changing the number of clusters i to be smaller around the number of clusters where the degree of separation is improved. May be.

  In order to evaluate the number of clusters, the cluster determination unit 35 calculates the degree of separation R every time clustering is performed. FIG. 11 is an example of a diagram illustrating the degree of separation R. In this embodiment, SAE (Stacked Auto Encoder) is used as an example for calculating the degree of separation R. In FIG. 11, a case where the number of clusters i = 4 will be described as an example. Assume that 2500 images 6 are classified into each cluster by clustering. A part of the cluster images 6 (for example, 2000) is used for learning, and the rest (for example, 500) is used for identification.

  FIG. 12 is an example of a diagram illustrating a SAE learning method. First, an auto encoder is a network having a feature expression capability that can be restored to the original pattern even if the features are compressed. Therefore, the auto encoder can learn by compressing the features, and if the learning is successful, it can be restored even if the data in the input layer is compressed. SAE is a network in which auto encoders are combined (stacked). 12A is an example of SAE, and FIGS. 12B, 12C, and 12D are examples of an auto encoder. Normally, if there are two or more intermediate layers, the solution converges to an inappropriate minimum solution, and error back-propagation does not work (this is called the gradient disappearance problem). In SAE, this is not calculated in two consecutive layers, but is piled up one layer at a time, making it difficult to fall into the gradient disappearance problem. If the SAE learning is successful, the SAE can restore the image 6. However, since the SAE cannot be restored when an abnormal image is input, it is assumed that the image 6 is only a normal image.

  As shown in FIG. 12A, the feature (node) decreases over the input layer x, the hidden layer y, the hidden layer z, and the output layer s, which corresponds to the compression of the feature, and the output layer s, the hidden layer z ′, Returning the number of nodes to the hidden layer y ′ and the input layer x ′ corresponds to restoration to the original pattern. When learning such a neural network parameter by SAE, a parameter (weight) between the input layer x and the hidden layer y is learned by the auto encoder shown in FIG.

  Next, in order to obtain the parameters (weights) of the hidden layer y and hidden layer z in FIG. 12A, the auto encoder shown in FIG. 12C hides the y in FIG. The parameters (weights) of the layer y and the hidden layer z are learned. Next, in order to obtain the parameters (weights) of the hidden layer z and the output layer s shown in FIG. 12A, the auto encoder shown in FIG. The parameters (weights) of the layer z and the output layer s are learned. Thus, the SAE parameters (weights) in FIG. 12A are learned.

  Returning to FIG. The 500 images for identification 6 are represented by x, the SAE is represented by a function f (), and the SAE output is represented by f (x). Score S is defined below. i is the number of the cluster. The score S is the square of the L2 norm (distance) in the feature amount space of the image 6.

この計算を各クラスタから抽出した１枚の画像６に対して行う。Ｓ_ciは学習がうまくいくほど小さくなる。

This calculation is performed on one image 6 extracted from each cluster. S _ci becomes smaller as learning is successful.

  Next, the separation degree R of all clusters is defined as follows.

学習がうまくいくこととクラスタリングがうまくいくことは相関するので、分離度Ｒが小さい方が好ましい。したがって、クラスタ設定部３３は幾つかクラスタ数ｉを変えて分離度Ｒを算出し、分離度Ｒが最も小さい時のクラスタ数ｉでクラスタリングすると判定する。つまり、各クラスタのスコアＳを合計した値が最も小さくなるようにクラスタ数を決定する
なお、ＳＡＥ以外でクラスタ数を決定する方法としては、ＡＩＣ（Akaike’s Information Criterion）、ＢＩＣ（Bayesian Information Criterion）などがある。

Since good learning and good clustering are correlated, it is preferable that the degree of separation R is small. Accordingly, the cluster setting unit 33 calculates the degree of separation R by changing the number of clusters i, and determines that clustering is performed with the number of clusters i when the degree of separation R is the smallest. In other words, the number of clusters is determined so that the total value of the scores S of each cluster becomes the smallest. Note that methods other than SAE for determining the number of clusters include AIC (Akaike's Information Criterion), BIC (Bayesian Information Criterion), etc. There is.

(S5)
Next, the learning unit 34 creates a learning model for determining an abnormality from the clustered image 6. In order to determine abnormality, the learning unit 34 learns only the normal image 6.

  FIG. 13 schematically shows a learning model for determining the presence / absence of an abnormality from the image 6. In this embodiment, a learning model that combines deep learning and SVM will be described as an example. As shown in FIG. 13, a patch region in which one image 6 is equally divided is input to an input layer 301 of a multilayer perceptron for deep learning. In the learning phase, a value (for example, 1) indicating that the image 6 is normal is input to the output layer 303. In the multilayer perceptron, the error back propagation method is generally used for learning the weights of the input layer 301, the intermediate layer 302, and the output layer 303. Note that the number of nodes in the output layer 303 is appropriately designed as a number that can appropriately extract the features of the image 6. The output value of each node of the output layer 303 is digitized into a value representing the characteristics of the moving object. Details will be described with reference to FIG.

  The SVM 305 is originally a supervised learning algorithm that performs 2-class discrimination, but the SVM can also be used as a 1-class SVM. In the 1-class SVM, the 2-class discrimination is replaced with an area discrimination problem for estimating a high-density area of data, so that a teacher signal is unnecessary. Lagrange's undetermined constant method is used to calculate the support vector, but the details are omitted. Since the normal feature vector region can be identified as shown in the figure, the abnormality determination unit 36 can determine that the possibility of abnormality is high as the region deviates from this region.

  Note that learning is performed for each image 6 of the same cluster clustered in step S3. Therefore, a learning model that learns that each scene is normal can be obtained.

  FIG. 14 is an example of a diagram for explaining the learning procedure of FIG. 13 in more detail. FIG. 14A schematically shows a plurality of images 6 of each cluster 71. Each of the clusters 71 in FIG. 14A is a cluster into which the parking lot images 6 are classified, and is clustered for each scene such as a car movement or walking. In addition, each is an image 6 of only normal behavior. As shown in FIG. 14B, the learning unit 34 divides one image 6 into patch regions (divided vertically and horizontally), and inputs it to the input layer 301 of the multilayer perceptron shown in FIG. Dividing into patch areas has the advantage that it is not necessary to consider the structure of the scene. For example, the car to be delivered has the same characteristics regardless of which direction it moves. In FIG. 14B, the patch area 304 in which the moving car is shown is indicated by a thick frame. When features are extracted for each patch area, the same car is detected in different patch areas of different images 6 when the car moves. Therefore, the learning unit 34 can track the same vehicle between different images based on the extracted feature amount.

  The learning unit 34 acquires the feature amount of each patch area of the image 6 at the time t from the feature amount obtained from the output layer 303 of the multilayer perceptron, and the past image 6 such as the times t-1, t-2,. Matches with feature quantity. Thereby, it can be determined to which patch area the vehicle has moved. For example, in the previous image 6 (one frame of the moving image), it can be seen that the car in the leftmost patch area has moved one patch area to the right. From the above, as shown in FIG. 14C, the shape (feature amount) and the movement (movement speed) of the moving car can be quantified. The feature quantity representing the shape of the car is extracted by deep learning. The moving speed may be calculated in an easy-to-handle unit such as n patch area / 1 image, n patch area / 1 second, n meter / 1 second, and the like. In this way, the shape (feature amount) and the movement (movement speed) are digitized and input to the SVM 305. If an abnormality occurs, there is some movement, so it is only necessary to extract features from the patch area where a moving object is detected (moving speed is greater than zero).

  The learning unit 34 learns all the images 6 for each cluster as described above, and creates learning models C1 to Cn.

<About functions when an abnormality is detected>
When the learning models C1 to Cn are obtained, the information processing apparatus 30 starts abnormality determination for the moving image. FIG. 15 is an example of a functional block diagram of the information processing apparatus 30 regarding abnormality determination. The information processing apparatus 30 of FIG. 15 includes a communication unit 31, an abnormality determination unit 36 (referred to as abnormality determination units 1 to n when distinguished), an abnormality determination unit x, and a determination result processing unit 37. First, the abnormality determination units 1 to n have learning models C1 to Cn, respectively. n is the number of clusters in steps S3 and S4 in FIG. The cluster determination unit 35 can determine the cluster having the closest image 6 of the existing installation location 8 by obtaining the local feature value and the global feature value from the image 6 of each existing installation location 8. Table 1 shows the correspondence between the cluster and the existing installation location 8 (camera ID). The association between the existing installation location 8 and the learning model may be performed once before the abnormality determination is started.

  The communication unit 31 allocates the images 6 transmitted from the respective existing installation locations 8 to the abnormality determination units 1 to n based on the correspondence between the learning models C1 to Cn and the camera ID. At this time, it is more preferable to refer to the label for daytime or nighttime and assign it to the daytime abnormality determination unit 36 or the nighttime abnormality determination unit 36. The association between the new installation location 7 and the learning model may be performed once before the abnormality determination is started.

  The abnormality determination unit x determines abnormality of the image 6 of the camera at the new installation location 7. The abnormality determination unit x has a learning model Cx. Similarly, the cluster determination unit 35 can determine the cluster having the closest image 6 of the new installation location 7 by obtaining the local feature value and the global feature value from the image 6 of the new installation location 7. The discrimination between daytime and nighttime may be the same as that of the abnormality determination units 1 to n.

  The abnormality determination units 1 to n and the abnormality determination unit x calculate the accuracy indicating how close to the normal image 6 using the assigned learning model. The accuracy takes a value of 0 to 1, for example, and the closer to normal, the higher the accuracy.

  The determination result processing unit 37 acquires the accuracy calculated by the abnormality determination units 1 to n and the abnormality determination unit x, and determines whether there is an abnormality. For example, what is necessary is just to determine with abnormality, when accuracy is less than a threshold value. If it is determined that there is an abnormality, a security guard is dispatched or the customer is notified that there is an abnormality.

<Abnormality detection of existing installation location>
FIG. 16 is an example of a diagram for schematically explaining the abnormality detection of the existing installation place 8. In FIG. 16, the existing installation location 8a is taken as an example, but the same is true for other existing installation locations 8.
(1) The cluster determination unit 35 determines a learning model suitable for the daytime and nighttime images 6 by clustering the images 6 captured by the camera device 10 at the existing installation location 8a.
(2) After the start of abnormality detection, the camera device 10 at the existing installation location 8a transmits a moving image to the information processing device 30. It is assumed that the movie has a day or night label. The information processing apparatus 30 may determine daytime or nighttime.
(3) The abnormality determination unit 36 corresponding to the existing installation location 8a determines whether there is an abnormality in the image 6 using a learning model assigned to the existing installation location 8 during the day or at night.
(4) The determination result (accuracy) by the abnormality determination unit 36 is sent to the determination result processing unit 37.

  As described above, the abnormality determination can be performed on the image 6 of the existing installation location 8a by using the learning model created from the clustered image 6, so that the abnormality determination can be performed with higher accuracy.

<Abnormality detection of new installation location>
FIG. 17 is an example of a diagram for schematically explaining the abnormality detection of the new installation place 7.
(1) The cluster determination unit 35 determines the learning model suitable for the daytime and nighttime images 6 by clustering the images 6 captured by the camera device 10 at the new installation location 7.
(2) After the start of abnormality detection, the camera device 10 at the new installation location 7 transmits a moving image to the information processing device 30. It is assumed that the movie has a day or night label. The information processing apparatus 30 may determine daytime or nighttime.
(3) The abnormality determination unit 36 of the new installation location 7 determines whether there is an abnormality in the image 6 using a learning model assigned to the new installation location 7 during the day or at night.
(4) The determination result (accuracy) by the abnormality determination unit 36 is sent to the determination result processing unit 37.

  As described above, when the camera device 10 is newly installed at the new installation location 7, an abnormality can be determined using the learning model obtained by clustering the images 6 of the existing installation location 8. It is possible to accurately determine an abnormality from the beginning when it is newly installed.

<Example of determination result processing>
The process of the determination result processing unit 37 will be described with reference to FIG. FIG. 18 is an example of a diagram schematically illustrating processing performed by the determination result processing unit 37.
(1) The image 6 is input to the abnormality determination unit 36 as described above. The abnormality determination unit 36 sends information regarding the installation location of the camera device 10 (for example, a customer ID and a customer name) and a moving image used for the determination of the abnormality to the determination result processing unit 37 together with the abnormality determination result.
(2) The determination result processing unit 37 compares the abnormality determination result with a threshold value to determine whether or not there is a high possibility of abnormality. When it is determined that the possibility of abnormality is high, at least a part of the moving image and the determination result are transmitted to the customer and the guard. More preferably, the monitoring person visually confirms the moving image before transmission. The transmission destination is a guard center serving as a control tower for dispatching security guards, a waiting area where security guards are stationed, guards around the site, customers, and the like.

FIG. 18 shows a screen example of the mobile terminals 74a and 74b possessed by a customer or a guard. On the screen of the portable terminal 74a, a message 401 “abnormality 80%”, a moving image display field 402, a moving image playback button 403, and the like are displayed. Customers and security guards can take appropriate actions by viewing messages and videos. One camera device 10 among the plurality of camera devices 10 is highlighted and displayed on the portable terminal 74b. In this way, when a plurality of camera devices 10 are installed at one installation location, by displaying which camera device 10 has detected the abnormality, it is easy for customers and security guards to identify the location where the abnormality has occurred. Become.
(3) Customers and security guards report to the police as necessary, or security guards visit the site to check. If the security guard has visited the site but is normal, the security guard notifies the judgment result processing section 37 (feedback) to that effect. At the time of notification, the moving image (or information that can identify the moving image) transmitted to the mobile terminals 74 a and 74 b is transmitted to the determination result processing unit 37. Since the learning unit 34 can learn from the teacher signal that the moving image is normal, the accuracy of abnormality detection can be improved.

<How to use a learning model in a camera device>
In this embodiment, the monitoring center determines an abnormality, but the camera device 10 can also determine an abnormality.

FIG. 19A shows a procedure in which the camera device 10 at the new installation location 7 determines an abnormality using the learning model Cx.
(1) The camera device 10 at the new installation location 7 captures the image 6. If there is at least one image 6, it does not have to be a moving image.
(2) The camera device 10 transmits the image 6 to the information processing device 30.
(3) The information processing apparatus 30 stores the image 6.
(4) The information processing apparatus 30 determines to which cluster the image 6 of the new installation location 7 is applicable.
(5) The information processing apparatus 30 transmits the learning model Cx determined to be applicable to the camera apparatus 10. Since the camera apparatus 10 determines an abnormality using the learning model Cx, it is not necessary to transmit a moving image to the information processing apparatus 30 in the abnormality determination phase.

FIG. 19B shows a procedure for determining an abnormality using the image 6 captured by the camera device 10 at the new installation location 7. When a certain amount of time has passed, the camera device 10 at the new installation location 7 can capture an image 6 sufficient for learning.
(1) The camera device 10 at the new installation location 7 captures the image 6. Since it is the image 6 for judging abnormality, it is preferable that it is the some image 6 (moving image).
(2) The camera device 10 transmits the image 6 to the information processing device 30.
(3) The information processing apparatus 30 stores a plurality of images 6.
(4) When a certain amount of image 6 is accumulated, the information processing apparatus 30 learns the normal image 6 and creates a learning model Cx (Ver 2.0).
(5) The information processing apparatus 30 transmits the created learning model Cx (Ver 2.0) to the camera apparatus 10 at the new installation location 7.
(6) The camera device 10 can determine the abnormality using the learning model Cx (Ver 2.0). You may use together with the original learning model Cx. When using together, when it is judged that one of the original learning model Cx and the learning model Cx (Ver 2.0) is abnormal, the abnormality is transmitted to the monitoring center. This can reduce false alarms.

  As described with reference to FIG. 19, the camera device 10 at the new installation location 7 can determine the abnormality immediately after installation, and can determine the abnormality with the learning model of the image 6 captured at the installation location after a lapse of time.

<Other application examples>
The best mode for carrying out the present invention has been described above with reference to the embodiments. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the scope of the present invention. And substitutions can be added.

  In the present embodiment, abnormality detection based on the image 6 is performed, but the information processing apparatus 30 can also detect abnormality by identifying sound. In this case, the camera device 10 has a microphone or the like that collects sound. The sound data collected by the camera device 10 is transmitted to the information processing device 30 together with the image 6. The information processing apparatus 30 learns normal sound data and determines whether or not it is abnormal. Also in this case, an abnormality can be detected immediately after the installation at the new installation place 7.

  In the system configuration diagram of FIG. 2, the camera device 10 and the information processing device 30 are separate bodies. However, as described with reference to FIG. 19, the camera device 10 and the information processing device 30 may be integrated.

  In addition, it has been described that the daytime and nighttime images can be classified into different clusters even in the camera device 10 in the same place, but may be more finely classified into different clusters for each time zone. Moreover, you may classify | categorize into a different cluster for every weather and a season. Thus, the cluster setting unit 33 can classify each image into an appropriate cluster according to the environment in which the camera device 10 captures an image.

  6 and 15 are divided according to main functions in order to facilitate understanding of processing by the camera apparatus 10 and the information processing apparatus 30. The present invention is not limited by the way of dividing the processing unit or the name. The processing of the camera device 10 and the information processing device 30 can be divided into more processing units according to the processing content. Moreover, it can also divide | segment so that one process unit may contain many processes.

  6 and 15, there is one information processing device 30, but there may be a plurality of the same information processing devices 30, and the functions of FIGS. 6 and 15 are distributed to the plurality of information processing devices 30. It may be.

  The communication unit 31 and the image storage unit 32 are an example of an image acquisition unit, the cluster setting unit 33 is an example of a classification unit, the learning unit 34 is an example of a learning unit, and the abnormality determination unit 36 is an abnormality determination unit. The abnormality determination unit 36 using the learning model Cx (Ver 2.0) is an example of a second abnormality determination unit, and the cluster determination unit 35 is an example of a determination unit.

6: Image 7: New installation location 8: Existing installation location 9: Monitoring center 10: Camera device 30: Information processing device 100: Image processing system

Claims (11)

An image processing system for detecting an abnormality from an image captured by an imaging device,
Image acquisition means for acquiring the image from a plurality of first imaging devices installed at different installation locations;
Classification means for classifying the images having similar features into clusters,
Learning means for learning, for each cluster, an abnormality determination means for learning the image classified into the clusters and detecting an abnormality;
The cluster in which the image of the first imaging device similar to the image captured by the second imaging device installed at a different installation location from the first imaging device is classified is determined and classified into the cluster Determining means for determining the abnormality determining means constructed from the image that has been made;
An image processing system.   The image processing system according to claim 1, wherein the abnormality determination unit determined by the determination unit performs abnormality detection from the image captured by the second imaging device.   The classification means extracts a local feature amount and a global feature amount from the image captured by the first imaging device, and the image having a similar local feature amount and global feature amount is used as the installation location. The image processing system according to claim 1, wherein the image processing system is classified into the same cluster regardless. The classification means classifies the images from which local feature values and global feature values have been extracted into several clusters, and calculates a degree of separation that decreases as the classification becomes appropriate.
The image processing system according to claim 3, wherein the number having the smallest degree of separation is determined, and the images captured by the first imaging device are classified into the number of the clusters. The classification means is Stacked
The difference between the image captured by the first imaging device input to the neural network constructed by the Auto Encoder and the output value output by the neural network is calculated for each cluster, and the sum of the differences of each cluster The image processing system according to claim 4, wherein the number is determined so as to be the smallest. The classification means classifies the images captured by the first imaging device at the same installation location into different clusters according to the environment in which the images are captured,
The image processing system according to claim 1, wherein the learning unit constructs the abnormality determination unit from the images classified into the different clusters according to the environment. The learning means extracts features of the image divided into regions by deep learning,
Quantify the shape and movement of the object in the image based on the characteristics of the region,
The image processing system according to claim 1, wherein the abnormality determination unit that classifies the shape and movement of the object as normal is constructed by an SVM (support vector machine). Image acquisition means for acquiring images from a plurality of other imaging devices installed in different installation locations;
Classification means for classifying the images having similar features into clusters,
A learning model that learns the image classified into the clusters and detects abnormality for each cluster to learn abnormality is acquired from the information processing apparatus,
An imaging device that detects an abnormality from a captured image,
Transmitting the image to the information processing device, obtaining the abnormality determination means constructed from the cluster into which the images with similar image characteristics are classified;
An imaging apparatus, wherein abnormality determination is performed on an image captured by the abnormality determination unit acquired from the information processing apparatus. Obtaining from the information processing device a second abnormality determination means constructed from an image captured by the imaging device;
The imaging apparatus according to claim 8, wherein the abnormality determination is performed by both the second abnormality determination unit and the abnormality determination unit. A learning model creation method performed by an image processing system that detects an abnormality from an image captured by an imaging device,
An image acquiring means acquiring the image from a plurality of first imaging devices installed at different installation locations;
A classifying unit classifying the images having similar features into clusters, respectively;
Learning means for learning the image classified into the cluster and constructing an abnormality determination means for detecting an abnormality for each cluster;
The determining means determines the cluster into which the image similar to the image captured by the second imaging device installed at a different installation location from the first imaging device is classified, and is constructed from the cluster Determining the abnormality determining means;
A learning model creation method. An information processing apparatus that creates a learning model for detecting an abnormality from an image captured by an imaging apparatus,
Image acquisition means for acquiring the image from a plurality of first imaging devices installed at different installation locations;
Classification means for classifying the images having similar features into clusters,
Learning means for learning, for each cluster, an abnormality determination means for learning the image classified into the clusters and detecting an abnormality;
The anomaly judging means constructed by judging the cluster into which the image similar to the image taken by the second imaging device installed at a different installation location from the first imaging device is classified, and constructed from the cluster A determination means for determining
An information processing apparatus.

Images