JP2011014059A - Behavior analysis system and behavior analysis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To analyze behavior of a person, without invading one's privacy.SOLUTION: A photographic device extracts numerical values representing features of persons in a photographed image, as feature quantity data and encrypts the data. Then, the photographic device transmits the encrypted feature quantity data, prescribed relevant information, and a photographic device identifier, specifically given to the photographic device, to a collation device. The collation device stores the encrypted feature quantity data, relevant information, and the photographic device identifier into a feature quantity database and performs matching of the feature quantity data encrypted, as it is, at an arbitrary time and a person identifier is put out for every person on the basis of the matching. Then, the collation device creates a behavior database, including a set of relevant information and the photographic identifier for each person identifier.

Description

  The present invention relates to a behavior analysis system and a behavior analysis method.

  2. Description of the Related Art In recent years, a technique for extracting, as feature amount data, a numerical value that represents the characteristics of a person shown in a video based on a video captured by a video camera has been considered. By generating a plurality of such feature amount data and comparing them, it is possible to determine whether or not the person photographed with the video camera is a specific person (for example, see Patent Document 1).

  In the technique disclosed in Patent Document 1, a person is photographed in advance in order to extract feature quantity data that serves as a reference for authenticating the person, and the feature quantity data extracted from the photographed video is managed by a management company. Store in the server in advance. Then, the video of a person photographed by a video camera installed at a fixed point on a door in an arbitrary facility such as a university or a hospital is transmitted to the management server. Then, the management server compares the feature amount data extracted from the transmitted video with the feature amount data stored in advance. As a result of this comparison, when these feature data match, the door is unlocked.

JP 2009-098814 A

  When a person is photographed, it is necessary to pay sufficient attention to the privacy of the person within a range that does not impair the photographing purpose. In the example of the technique disclosed in Patent Document 1, the management company only needs to be able to perform at least entrance management for visitors to the facility.

  However, in this technique, a person who belongs to a management company (for example, an administrator of a management server) has been photographed by a feature server stored in advance in the management server or captured by a video camera and transmitted to the management server. You can see the video.

  Therefore, there is a problem that the privacy of a person photographed with a video camera cannot be sufficiently protected within a range that does not impair the photographing purpose.

  An object of the present invention is to provide a behavior analysis system and a behavior analysis method that solve the above-described problems.

  In order to solve the above-described problem, the behavior analysis system of the present invention is a behavior analysis system including a photographing device for photographing a video and a collation device for identifying a person photographed by the photographing device. The apparatus extracts, as feature data, a numerical value representing a person's feature reflected in the video from the video, encrypts the feature data, and accompanies the encrypted feature data and the person's action. Predetermined related information and an imaging device identifier for uniquely identifying the imaging device are transmitted to the verification device, and the verification device transmits the encrypted feature amount data and the related information transmitted from the imaging device. The imaging device identifier is received, stored in the feature amount database of the matching device, and the feature amount data is matched with being encrypted at an arbitrary time, and based on the matching A unique identifier for identifying the person for each unique person, and creating or appending / updating an action database having a list of associated information and a set of photographing device identifiers for each person identifier. It is characterized by.

  In the behavior analysis system of the present invention, the photographing device may photograph a still image as the video at a predetermined time interval and extract the feature amount data from the still image.

  In the behavior analysis system of the present invention, when the imaging device extracts the feature amount data from the video, a person associated with the feature amount data is within the imaging range from outside the imaging range of the imaging device. A frame-in time that is a time when the person enters and a frame-out time that is a time when a person existing in the shooting range is out of the shooting range are extracted from the video, and the frame-in time and the frame-out time are extracted. May be transmitted to the verification device as the related information.

  In the behavior analysis system of the present invention, the imaging device may extract at least one of the frame-in time and the frame-out time from the video and transmit the extracted information to the verification device as the related information.

  Further, in the behavior analysis system of the present invention, the imaging device calculates a position where the person is reflected in the video, and calculates the position at the frame-in time and the position at the frame-out time. You may transmit to the said collation apparatus as related information.

  In the behavior analysis system of the present invention, the photographing device calculates a position where the person is reflected in the video, and the position at the frame-in time and the position immediately after the position are the related information. May be transmitted to the verification device.

  In the behavior analysis system of the present invention, the photographing device calculates a position where the person is reflected in the video, and the position at the frame-out time and the position immediately before the position are the related information. May be transmitted to the verification device.

  In order to solve the above-mentioned problem, a behavior analysis method of the present invention is a behavior analysis method in a behavior analysis system including a photographing device for photographing a video and a matching device for identifying a person photographed by the photographing device. The image capturing device extracts a feature value extraction process that extracts, as feature amount data, a numerical value representing a feature of a person reflected in the image from the image, and the image capturing device encrypts the feature amount data. Processing, and the imaging device transmits the encrypted feature amount data, predetermined related information associated with the action of the person, and an imaging device identifier for uniquely identifying the imaging device to the verification device In the transmission process, the collation device receives the encrypted feature amount data and the related information and the photographing device identifier transmitted from the photographing device, and stores them in the feature amount database of the collation device. A process for matching the feature amount data in an encrypted state at an arbitrary time, and the collation apparatus uniquely identifies the person for each unique person based on the matching. A process of paying out a person identifier for identification, and a process in which the collation device creates, adds, or updates a behavior database having a list of sets of related information and photographing device identifiers collected for each person identifier. It is characterized by.

  Further, in the behavior analysis method of the present invention, the photographing apparatus has a process of photographing a still image as the video at a predetermined time interval. In the feature amount extraction process, the feature amount data is extracted from the still image. May be extracted.

  In the behavior analysis method of the present invention, when the imaging device extracts the feature amount data from the video, a person associated with the feature amount data is within the imaging range from outside the imaging range of the imaging device. A process of extracting from the video a frame-in time that is an incoming time, and a frame-out time from which the person who is present in the shooting range is out of the shooting range is extracted from the video In the transmission process, the frame-in time and the frame-out time may be transmitted to the verification device as the related information.

  In the behavior analysis method of the present invention, in the transmission process, at least one of the frame-in time and the frame-out time may be extracted from the video and transmitted as the related information to the verification device.

  Further, in the behavior analysis method of the present invention, the photographing apparatus calculates a position where the person in the video at the frame-in time is reflected, and the photographing apparatus performs the process at the frame-out time. The position of the person in the video is calculated, and in the transmission process, the position at the frame-in time and the position at the frame-out time are used as the related information as the matching device. May be sent to.

  In the behavior analysis method of the present invention, the photographing device calculates a position where the person is reflected in the video, and the position at the frame-in time and the position immediately after the position are the related information. May be transmitted to the verification device.

  In the behavior analysis method of the present invention, the imaging device calculates a position where the person is reflected in the video, and the position at the frame-out time and the position immediately before the position are related information. May be transmitted to the verification device.

  According to the present invention, the behavior of the person can be analyzed without infringing on the privacy of the person.

It is a figure which shows the structure of the behavior analysis system according to embodiment of this invention. It is a figure which shows the structure of the imaging device shown in FIG. It is a figure which shows the structure of the collation apparatus shown in FIG. It is a figure which shows an example of the data structure of a feature-value database. It is a figure which shows an example of the data structure of an action database. It is a flowchart which shows operation | movement when an imaging device encrypts the feature-value data extracted from the image | photographed image | video, and transmits to a collation apparatus. It is a flowchart which shows operation | movement after a collation apparatus receives the encrypted feature-value data transmitted from the imaging device.

  Hereinafter, a behavior analysis system (including a behavior analysis method) according to an embodiment of the present invention will be described.

  First, the overall configuration of the behavior analysis system will be described.

  As shown in FIG. 1, the behavior analysis system includes a plurality of imaging devices 1-1 to 1-2, a verification device 2, a behavior database 231, and a single or a plurality of analysis devices 3. Yes. Each of the imaging devices 1-1, 1-2, the verification device 2, the behavior database 231, and the analysis device 3 has a communication function. In this example, each of the photographing devices 1-1 and 1-2 and the collation device 2 are communicably connected, and the collation device 2 and the behavior database 231, and the behavior database 231 and the analysis device 3 are communicably connected. Has been.

  Note that the number and location of the imaging devices included in the behavior analysis system may be arbitrary. For example, one photographing device may be installed at each entrance / exit at an arbitrary place where a plurality of entrances / exits are provided.

  In addition, each of the image capturing apparatuses 1-1 and 1-2 is given a “photographing apparatus identifier” for uniquely identifying the image capturing apparatus. In the present embodiment, the case where the photographing device identifier assigned to the photographing device 1-1 is ID1-1 and the photographing device identifier assigned to the photographing device 1-2 is ID1-2. explain.

  In addition, each of the image capturing apparatuses 1-1 and 1-2 extracts, as “feature data”, a numerical value representing the characteristics of a person shown in the image based on the captured image. The video captured by each of the imaging devices 1-1 and 1-2 may be either a moving image or a still image, but in this embodiment, each of the imaging devices 1-1 and 1-2 captures a video as a video. An example of the case will be described.

  Examples of the “feature data” include numerical values indicating the outline of the face of a person photographed by the photographing apparatuses 1-1 and 1-2, and the size of an arbitrary constituent element such as eyes, nose, and mouth in a predetermined direction. , Distances between components such as eyes, nose and mouth.

  Furthermore, each of the photographing apparatuses 1-1 and 1-2 encrypts the extracted feature data. Here, as the encryption algorithm of the present embodiment, an algorithm that can be operated while encrypted, such as a homomorphic encryption, is used.

  In addition, each of the image capturing apparatuses 1-1 and 1-2 extracts “related information” that is predetermined information associated with the action of the person captured by the image capturing apparatus from the captured image.

  This related information may be any information as long as it is information indicating the behavior and attributes of the person photographed by the photographing apparatuses 1-1 and 1-2. In the present embodiment, the case where the related information is “frame-in time” and “frame-out time” will be described as an example.

  Here, the “frame-in time” refers to the time when a certain person enters the shooting range from outside the shooting range of the shooting apparatuses 1-1 and 1-2.

  The “frame-out time” refers to a time when a person existing in the shooting range is out of the shooting range.

  Then, each of the imaging devices 1-1 and 1-2 transmits the encrypted feature amount data that is “encrypted feature amount data”, the imaging device identifier, and the related information to the verification device 2.

  The collation apparatus 2 has a “feature amount database 23” described with reference to FIG.

  The collation device 2 corresponds to the “encrypted feature data” transmitted from the photographing devices 1-1 and 1-2, the photographing device identifier, and related information (in this example, frame-in time and frame-out time). Then, it is stored in the feature amount database 23.

  Further, the collation device 2 performs encryption between an arbitrary data set in the feature amount database 23 and another data set at an “arbitrary time point” including a time point when data is accumulated in the feature amount database 23 to some extent. The matching of the feature data that has been performed may be performed while being encrypted.

  Here, as an example of “arbitrary time point”, the time point when the data amount stored in the feature amount database 23 becomes the same as a predetermined reference data amount, or transmitted from the photographing devices 1-1 to 1-2. For example, a point in time when the receiving unit 21 receives new encrypted feature data.

  Then, a common person identifier is paid out to the matched data set (that is, assignment of the same person identifier), and the photographing apparatus identifier associated with the action database 231 by one person identifier and a single related information Alternatively, the process of adding a plurality of sets is repeated while changing the target data set as long as there is a data set that does not match.

  When data (encrypted feature amount data, related information, and image capturing device identifier) is transmitted from the image capturing devices 1-1 and 1-2, encryption is performed between the data set and the data set in the feature amount database 23. It is also possible to perform matching as it is and add a set of data sets for one person to the behavior database.

  Regarding the payout of the person identifier, a new payout may be made for each matching, or the previous payout result (that is, the previously assigned person identifier) is stored in each data set in the feature amount database 23, It may be reused.

  In the present embodiment, the collation device 2 uses a plurality of data in the feature amount database 23 or “encrypted feature amount data” and feature amount database transmitted from the photographing devices 1-1 and 1-2. As a method of performing matching with the data set stored in 23, it is possible to execute a logical calculation on an encrypted numeric value or a digitized character string while the numeric value is still encrypted. An example is a secret circuit calculation.

  The analysis device 3 allows the verification device 2 to access the behavior database 231, and reads the association between the related information (in this example, frame-in time and frame-out time) and the imaging device identifier from the behavior database 231.

  Then, the analysis device 3 captures a person in the photographing apparatuses 1-1 and 1-2 based on the read related information and the photographing apparatus identifier, or the person is in front of the photographing apparatus. Analysis processing such as the time of appearance, the time of leaving, and the speed when the person passes the imaging device is executed.

  Hereinafter, the configuration of the imaging devices 1-1 to 1-2 will be described with reference to FIG. Since the photographing apparatuses 1-1 to 1-2 have the same configuration, the photographing apparatus 1-1 will be described below as an example.

  As shown in FIG. 2, the photographing apparatus 1-1 includes a photographing unit 11, a feature amount extraction unit 12, an encryption unit 13, a related information extraction unit 14, and a transmission unit 15.

  The photographing unit 11 shoots a video in which an arbitrary subject including a person is reflected, and assigns the shooting time of the video to the video. Then, the photographing unit 11 outputs the video with the photographing time to the feature amount extracting unit 12. An example of the photographing unit 11 is a photographing mechanism included in a general video camera.

  The feature amount extraction unit 12 identifies a person reflected in the video output from the imaging unit 11. Note that any method may be used for identifying a person shown in the video.

  For example, the feature amount extraction unit 12 may identify the person by detecting an edge indicating the person based on a luminance change in the video output from the imaging unit 11. The filter used for edge detection may be an arbitrary filter such as a Sobel filter, for example.

  Further, for example, the feature amount extraction unit 12 may identify a person reflected in the video by pattern matching that detects a pattern indicating a candidate for a person from the video output from the imaging unit 11.

  The feature amount extraction unit 12 extracts, as feature amount data, a numerical value representing the identified person's feature from the video at a timing depending on the employed person identification method, and outputs the feature amount data to the encryption unit 13.

  The encryption unit 13 encrypts the feature amount data output from the feature amount extraction unit 12. Then, the encryption unit 13 outputs the encrypted feature amount data to the transmission unit 15.

  Note that in the present embodiment, the collation device 2 compares a plurality of pieces of encrypted feature amount data while being encrypted. For this reason, the encryption unit 13 may use any method as long as it can encrypt the feature data in a format that can be compared with the verification device 2 in the encrypted state.

  For example, as disclosed in Document A, the encryption unit 13 may use a method of encrypting feature amount data for each digit expressed in binary numbers using Paillier encryption (Document A: Berry Schoenmakers). Pim Tuyls, “Practical Two-Party Computation Based on the Conditional Gate,” Proc. Of ASIACRYPT 2004, pp.119-136, Spring-Verlag, 2004.).

  When the related information extraction unit 14 identifies a person reflected in the video output from the imaging unit 11, it is determined whether the person has entered the imaging range from outside the imaging range of the imaging apparatus 1-1. Determine.

  As a result of this determination, when it is determined that the person has entered the shooting range from outside the shooting range of the shooting device 1-1, the time is extracted as the frame-in time.

  On the other hand, if the related information extraction unit 14 determines that the person has not entered the imaging range from outside the imaging range of the imaging device 1-1, the person existing in the imaging range is out of the imaging range. To determine whether

  As a result of this determination, when it is determined that a person existing in the shooting range is out of the shooting range, the time is extracted as a frame-out time.

  By the way, the timing for determining whether or not a person is in the shooting range and the timing for extracting feature amount data / related information are not necessarily real time, and the recorded data may be analyzed.

  Then, the related information extraction unit 14 outputs the extracted frame-in time and frame-out time to the transmission unit 15 as related information.

  The transmission unit 15 captures the related information output from the related information extraction unit 14 and the encrypted feature amount data output from the encryption unit 13 to uniquely identify the imaging device 1-1. It transmits to the collation apparatus 2 with an apparatus identifier (in this example, ID1-1).

  Next, the configuration of the verification device 2 will be described with reference to FIG.

  As shown in FIG. 3, the collation device 2 is provided with a receiving unit 21, a collation unit 22, and a feature amount database 23.

  The reception unit 21 encrypts feature data of a certain person transmitted from each of the photographing apparatuses 1-1 and 1-2, and predetermined related information (in this example, a frame) associated with the action of the person. In time and frame out time) and the imaging device identifier are output to the collation unit 22.

  The collation unit 22 stores the encrypted feature amount data output from the reception unit 21, the related information, and the imaging device identifier in the feature amount database 23 as one data set.

  Further, at any timing such as when a sufficient amount of data sets are stored in the feature amount database 23, the collation unit 22 identifies the data sets stored in the feature amount database 23 with encrypted feature amount data. The key is used as a key and is encrypted, and a person identifier is paid out in units of persons (that is, assignment of person identifiers), and a data set (related information and photographing apparatus identifier) excluding the person identifier and encrypted feature data is stored. The set is added to the behavior database 231.

  Note that this name identification (matching) is performed while the target feature data is encrypted.

  For example, as disclosed in Document B, the collation unit 22 converts an arbitrary logical operation on a numerical value expressed in binary numbers and encrypted for each digit into a format that allows the user to recognize the numerical value. By performing a secret circuit calculation that can be executed without conversion, it is possible to identify names as they are encrypted (reference B: G. Yamamoto, K. Chida, A. Nasciment, K. Suzuki, and S. Uchiyama, Efficient , non-optimistic secure circuit evaluation based on the ElGamal encryption, WISA 2005, Lecture Notes in Computer Science, pp.328-343, Spring-Verlag, 2005.).

  As illustrated in FIG. 4, the feature database 23 stores the encrypted feature data written by the collation unit 22, related information, and an imaging device identifier in association with each other.

  As shown in FIG. 5, the behavior database 231 stores the person identifier written by the collation unit 22 and a plurality of sets of related information and imaging device identifiers in association with each other.

  The reason why the analysis device 3 cannot communicate directly with the feature database 23 and can communicate with only the behavior database 231 is that although the feature data is encrypted, the imaging devices 1-1 and 1-2 From the viewpoint of avoiding personal information accompanying this person from being unnecessarily leaked to an analyst or the like for purposes other than shooting purposes. It is.

  However, as a special example, when the collation device 2 also serves as the analysis device 3, the analysis is performed by directly referring to the feature amount database 23 on the assumption that the owner of the imaging device 1-1 is obtained. It doesn't matter.

  Next, the operation of the behavior analysis system having the above configuration will be described.

  First, an operation in which the image capturing apparatuses 1-1 and 1-2 encrypt feature amount data extracted from captured images and transmit the encrypted data to the collation apparatus 2 will be described with reference to FIG. In addition, since the imaging devices 1-1 to 1-2 perform the same operation, the imaging device 1-1 will be described below as an example.

  First, in step 41 shown in FIG. 6, the photographing unit 11 included in the photographing apparatus 1-1 photographs an image in which an arbitrary subject including a person is reflected. In addition, the shooting unit 11 gives the shooting time of the video to the video.

  Then, in step 42, the related information extraction unit 14 determines whether a certain person has entered the imaging range from outside the imaging range of the imaging apparatus 1-1.

  As a result of the determination, if it is determined that the person has entered the shooting range from outside the shooting range of the shooting device 1-1, the shooting time is extracted as a frame-in time in step 43.

  Further, in step 44, the feature amount extraction unit 12 identifies a person reflected in the video output from the photographing unit 11 at an arbitrary timing, and in step 45, a numerical value representing the characteristic of the identified person is obtained. The feature amount data is extracted, and the feature amount data is output to the encryption unit 13.

  In step 46, the encryption unit 13 encrypts the feature amount data and outputs the encrypted feature amount data to the transmission unit 15.

  Note that the encryption unit 13 encrypts the feature amount data using, for example, encryption for the secret circuit calculation as disclosed in the above-described document B.

  If the target person is not within the shooting range, step 42 is repeated.

  Next, in step 47, the related information extraction unit 14 determines whether or not a person existing in the shooting range is out of the shooting range.

  As a result of the determination, if it is determined that the person existing in the shooting range is out of the shooting range, the shooting time is extracted as a frame-out time in step 48. Then, the related information extraction unit 14 outputs the extracted frame-in time and frame-out time to the transmission unit 15 as related information.

  Then, in step 49, the transmission unit 15 uniquely identifies the relevant information output from the related information extraction unit 14 and the feature amount data encrypted from the encryption unit 13 for the imaging device 1-1. It transmits to the collation apparatus 2 with the imaging device identifier (ID1-1 in this example) for identification.

  Thus, a series of operations in which the photographing apparatus 1-1 encrypts the feature amount data extracted from the photographed video and transmits it to the collation apparatus 2 ends.

  Next, the operation after the collation device 2 receives the encrypted feature amount data, the related information, and the photographing device identification device transmitted from the photographing devices 1-1 and 1-2, with reference to FIG. explain.

  First, the receiving unit 21 included in the collation device 2 includes encrypted feature data of a certain person transmitted from the photographing apparatuses 1-1 and 1-2, and a predetermined association associated with the action of the person. Information (in this example, frame-in time and frame-out time) and the imaging device identifier are output to the collation unit 22.

  Then, in step 51 shown in FIG. 7, the collation unit 22 associates the encrypted feature amount data output from the reception unit 21 with the related information and the imaging device identifier and stores them in the feature amount database 23. .

  After the process of step 51 is repeated any number of times one or more times, at “any time point”, the collation unit 22 performs encryption on any one data set in the feature amount database 23 as a step 52. Matching with other data set sets is performed with encryption using the converted feature data as a key.

  As a result, in step 53, the collation unit 22 pays out a person identifier for identifying the person in the behavior database 231 with respect to the matched data set set.

  Then, in step 54, the data set set matching the person identifier excluding the encrypted feature data is added to the action database.

  As step 55, if there remains a data set set excluding a data set targeted in the past and a data set matched therewith, steps 52 to 54 are repeated for the remaining data set.

  At the stage where name identification is completed for all data sets in the feature amount database 23, the collation apparatus 2 ends the processing.

  The analysis device 3 refers to the behavior database 231 accessible by the analysis device 3 and executes analysis processing such as a flow line that is a trajectory of the movement of the person.

  Thus, a series of operations after the collation device 2 receives the encrypted feature amount data, the related information, and the photographing device identification device transmitted from the photographing devices 1-1 and 1-2 are completed.

  As described above, according to the present invention, a person is photographed by a plurality of video cameras, and in performing a flow line analysis such as when the person appears before and when the person leaves, not the state of the image, Encryption is performed after conversion into feature data that can be numerically compared with the characteristics of other persons, and analysis can be performed after name identification is performed using the feature data as encrypted.

  This makes it possible to analyze the person's behavior without infringing on the person's privacy.

  Specifically, infringement of the privacy of the person by collecting names of the same person photographed by the plurality of photographing apparatuses 1-1 and 1-2, or leakage of personal information accompanying the photographed person Can be avoided.

  In addition, in any place (for example, exhibition hall) where a plurality of entrances / exits are provided, a photographing device is installed at each entrance / exit so that the personal information of the photographed person is protected and the place is visited. It is possible to specify from which entrance the person has entered and from which exit, and to specify the time spent at the place.

  Various modifications can be made without departing from the scope of the present invention.

  The photographing devices 1-1 and 1-2 are not limited to video cameras that shoot moving images as images, but may be still cameras that shoot still images as images. In this case, the imaging units 11 included in the imaging apparatuses 1-1 and 1-2 capture still images at predetermined time intervals and output them to the feature amount extraction unit 12. Then, the feature amount extraction unit 12 may calculate the feature amount of the person captured in the still image based on the still image output at a predetermined time interval.

  The related information extraction unit 14 may extract only one of the frame-in time and the frame-out time instead of both the frame-in time and the frame-out time.

  Further, the related information extraction unit 14 may be configured to calculate the position of a person reflected in the video. Then, the related information extraction unit 14 uses the position of the person reflected in the video at the frame-in time and the position of the person reflected in the video at the frame-out time as one piece of the related information as the matching device 2. Send to. And the feature-value database 23 which the collation apparatus 2 has associates and stores a locus | trajectory with frame-in time and frame-out time.

  Also, the photographing devices 1-1 and 1-2 may calculate the position where the person in the video is reflected, and transmit the position at the frame-in time and the position immediately after that to the matching device 2 as related information. Good.

  Also, the photographing apparatuses 1-1 and 1-2 may calculate the position where the person in the video is reflected, and transmit the position at the frame-out time and the position immediately before it to the matching apparatus 2 as related information. Good.

  Further, the related information extraction unit 14 may be configured to infer the action taken by the photographed person.

  In addition, the related information extracting unit 14 is obtained from video data, and when the photographing apparatuses 1-1 and 1-2 are provided with microphones, from voice data of a person input to the microphones, or from various sensor data. Using existing technology, attribute information such as gender and age of a person may be estimated, and the estimated attribute information may be transmitted as related information to the verification device 2 via the transmission unit 15. In this case, the collation device 2 stores the estimated attribute information together with the frame-in time and the frame-out time in the feature amount database 23 in association with each other. Then, the analysis apparatus 3 performs various analyzes in association with other related information using the estimated attribute information stored in the behavior database 231.

  Furthermore, in the present invention, it is assumed that the existing technology is used as a technology for identifying and tracking each person in a situation where a plurality of persons freely frame in and out, but in some cases Also, an embodiment in which the photographing apparatuses 1-1 and 1-2 extract feature amount data a plurality of times regularly or at special occasions such as immediately after frame-in or immediately before frame-out is also conceivable.

  The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention without departing from the gist of the present invention.

1-1, 1-2 Imaging device 11 Imaging unit 12 Feature amount extraction unit 13 Encryption unit 14 Related information extraction unit 15 Transmission unit 2 Verification unit 21 Reception unit 22 Verification unit 23 Feature amount database 231 Behavioral database 3 Analysis device

Claims (14)

In a behavior analysis system comprising a photographing device for photographing a video and a verification device for identifying a person photographed by the photographing device,
The photographing device extracts, as feature data, a numerical value representing a feature of a person reflected in the video from the video, encrypts the feature data, and adds the encrypted feature data and the action of the person. Transmitting accompanying predetermined related information and an imaging device identifier for uniquely identifying the imaging device to the verification device;
The collation device receives the encrypted feature amount data and the related information and the photographing device identifier transmitted from the photographing device, stores them in the feature amount database of the collation device, and encrypts them at an arbitrary time point The feature amount data is matched, a person identifier for uniquely identifying the person is paid out for each unique person based on the matching, and a set of related information and a photographing apparatus identifier collected for each person identifier A behavior analysis system characterized by creating, appending, and updating a behavior database having a list. The behavior analysis system according to claim 1,
The behavioral analysis system is characterized in that the photographing device photographs a still image as the video at a predetermined time interval and extracts the feature amount data from the still image. The behavior analysis system according to claim 1 or 2,
When the feature amount data is extracted from the video, the photographing device has a frame-in time that is a time when a person attached to the feature amount data enters the photographing range from outside the photographing range of the photographing device; A frame-out time that is a time when a person existing in the shooting range is out of the shooting range is extracted from the video, and the frame-in time and the frame-out time are transmitted to the verification device as the related information. A behavior analysis system characterized by The behavior analysis system according to claim 3,
The photographing apparatus extracts at least one of the frame-in time and the frame-out time from the video and transmits the extracted information as the related information to the collation device. The behavior analysis system according to claim 3,
The imaging device calculates a position where the person is reflected in the video, and transmits the position at the frame-in time and the position at the frame-out time as the related information to the verification device. Characteristic behavior analysis system. The behavior analysis system according to claim 4,
The photographing apparatus calculates a position where the person is reflected in the video, and transmits the position at the frame-in time and the position immediately after the position as the related information to the verification apparatus. Behavior analysis system. The behavior analysis system according to claim 4,
The photographing device calculates a position where the person is reflected in the video, and transmits the position at the frame-out time and the position immediately before the position as the related information to the matching device. Behavior analysis system. A behavior analysis method in a behavior analysis system comprising a photographing device for photographing a video and a matching device for identifying a person photographed by the photographing device,
A feature amount extraction process in which the photographing apparatus extracts, as feature amount data, a numerical value representing a feature of a person reflected in the image from the image;
A process in which the photographing apparatus encrypts the feature data;
A transmission process in which the imaging device transmits the encrypted feature value data, predetermined related information accompanying the action of the person, and an imaging device identifier for uniquely identifying the imaging device to the verification device; ,
The collation device receives encrypted feature amount data and related information transmitted from the photographing device and a photographing device identifier, and stores them in a feature amount database of the collation device;
Processing for matching the feature amount data while the encryption device is encrypted at an arbitrary time; and
A process in which the verification device pays out a person identifier for uniquely identifying the person for each unique person based on the matching;
The behavior analysis method, characterized in that the verification device includes processing for creating, appending, or updating a behavior database having a list of sets of related information and photographing device identifiers collected for each person identifier. The behavior analysis method according to claim 8,
The photographing device has a process of photographing a still image at a predetermined time interval as the video,
In the feature amount extraction process, the feature amount data is extracted from the still image. The behavior analysis method according to claim 8 or 9,
When the imaging device extracts the feature amount data from the video, a frame-in time that is a time when a person accompanying the feature amount data enters the imaging range from outside the imaging range of the imaging device is Processing to extract from
The imaging device has a process of extracting from the video a frame-out time that is a time when a person existing in the imaging range is out of the imaging range,
In the transmission process, the frame-in time and the frame-out time are transmitted as the related information to the verification device. The behavior analysis method according to claim 10,
In the transmission process, at least one of the frame-in time and the frame-out time is extracted from the video and transmitted to the verification device as the related information. The behavior analysis method according to claim 10,
A process in which the photographing device calculates a position where the person in the image is captured at the frame-in time;
The imaging device has a process of calculating a position where the person in the video is captured at the frame-out time,
In the transmission process, the position at the frame-in time and the position at the frame-out time are transmitted as the related information to the verification device. The behavior analysis method according to claim 11,
The photographing apparatus calculates a position where the person is reflected in the video, and transmits the position at the frame-in time and the position immediately after the position as the related information to the verification apparatus. To analyze behavior. The behavior analysis method according to claim 11,
The photographing device calculates a position where the person is reflected in the video, and transmits the position at the frame-out time and the position immediately before the position as the related information to the matching device. To analyze behavior.

Images