JP2016009896A - Video monitoring system, and video retrieval method of video monitoring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video monitoring system capable of enabling a recording device to retrieve a captured video image from which the presence of a monitoring target and features of the monitoring target are known, reproducing and displaying a monitoring video image including desired monitoring scene without requiring a high-cost device that performed complicated processing, the video monitoring system including the recording device which records information of captured video images or the like transmitted from a plurality of monitoring cameras installed in a monitoring area.SOLUTION: In a video monitoring system, a position coordinate detecting the presence of the monitoring target and a position coordinate detecting the features of the monitoring target are acquired from the captured video images of the monitoring cameras, and the recording device determines whether dates of the captured video images including the two position coordinates are close and whether positions of the two position coordinates are close on the monitoring area, thereby determining whether the monitoring targets relating to the two position coordinates including the captured video images from the same or different monitoring cameras are identical. A video image retrieval method to be implemented in the video monitoring system is also disclosed.

Description

The present invention relates to a video monitoring system that records video data captured by a plurality of monitoring cameras installed in a monitoring area, searches video data captured by a person to be monitored, and displays the video data, and the video monitoring thereof. The present invention relates to a video search method executed in the system.

In general, in a video surveillance system, a video of a surveillance area captured by a surveillance camera is recorded by a recording device, a video captured of a monitored person or the like is searched from the recorded video, and output to a surveillance monitor for playback display. . In such a video surveillance system, conventional methods for searching for a monitored person from video include a search method for specifying an imaging date and time, and information on a monitored person obtained by using sensors such as motion detection and face detection. There is a search method for searching for a video including the information in association with the video.

In addition, the video monitoring technique disclosed in Patent Document 1 installs a plurality of two types of cameras, a bird's-eye camera and a gaze camera, in a monitoring area, and the position information and events of the monitored person obtained from the captured images of the bird's-eye camera and the gaze camera. According to the information, the video surveillance device selects a plurality of gaze cameras and controls the direction and magnification of the imaging so as to optimally capture the monitored person. The monitoring video is generated and reproduced and displayed.

JP 2013-168757 A JP 2007-164718 A JP 2007-267294 A JP 2012-49774 A

In the conventional video surveillance system, the face of the monitored person is not always captured in the video of the monitoring camera that captures the desired surveillance scene. Therefore, even if a face detection sensor is used, the monitored person is not necessarily captured from the video. There is a possibility that it cannot be found. Further, since the motion detection sensor may detect a motion other than a person, it is considered that the number of videos including the motion detected information becomes enormous. Therefore, in order to search for a video in which a person to be monitored can be understood in a desired monitoring scene, it is necessary to reproduce and check one by one from videos that have been narrowed down to a certain extent, which requires a lot of time. is there.

Further, in the video surveillance technology of Patent Document 1, in order to capture a video that the monitored person can understand, the video surveillance device is based on information that changes every moment obtained from two types of cameras, an overhead camera and a gaze camera. It is necessary to perform a complicated process of selecting a plurality of gaze cameras and controlling the imaging direction, magnification, and the like. In addition, it is necessary to use a dedicated camera capable of inputting / outputting a control signal corresponding to the control from the video monitoring apparatus as the gaze camera. For this reason, there is a problem in that the cost required to install and operate a video surveillance apparatus with high arithmetic processing capability capable of executing complex processing and a plurality of dedicated cameras is increased.

The present invention has been made to solve the above-described problem, and a recording apparatus that records images fixedly picked up by a plurality of installed monitoring cameras can detect the presence of a monitoring target such as a monitored person, Video monitoring that can quickly search for captured images that show the characteristics of the monitoring target, such as the face of the person being monitored, and can reproduce and display the monitoring video including the desired monitoring scene without the need for expensive equipment that performs complex processing. The purpose is to provide a system.

The present invention includes a plurality of monitoring cameras that image a monitoring area, video data captured by the monitoring camera and a recording device that records the imaging date and time of the video data, an input device that inputs search conditions for the video data, A monitoring monitor that reproduces and displays the video data output by the recording device, detects the presence of a monitoring target from the video data, and outputs presence detection position coordinates that are position information in the monitoring area to the video data. A video monitoring system comprising: a presence detection unit for associating; and a feature detection unit for detecting a feature of a monitoring target from the video data and outputting a feature detection position coordinate as positional information in the monitoring area to associate with the video data The recording apparatus determines whether the presence detection position coordinates and the feature detection position coordinates associated with the video data are present. Position coordinate determination means for determining; first video search means for searching for first video data of a first monitoring camera satisfying the search condition among the video data captured by the plurality of monitoring cameras; and the plurality of monitoring Of the video data captured by the camera, a time difference between the imaging date and time and the imaging date and time of the first video data is a predetermined time or less, the feature detection position coordinates, and the presence detection of the first video data Second video search means for searching for the second video data of the second surveillance camera, wherein the distance between the position coordinates is equal to or less than a predetermined distance, video output means for outputting the searched video data to the monitor monitor, It is a video surveillance system characterized by having.

In addition, the present invention provides a plurality of monitoring cameras that capture an image of a monitoring area, video data captured by the monitoring camera, a recording device that records imaging date and time of the video data, and an input device that inputs search conditions for the video data And a monitoring monitor that reproduces and displays the video data output from the recording device, and a video search method that is executed in a video monitoring system that detects the presence of a monitoring target from the video data, and A presence detection step of outputting presence detection position coordinates, which is position information at the position, and associating it with the video data; detecting a feature to be monitored from the video data; and detecting feature detection position coordinates as position information in the monitoring area. A feature detection step of outputting and associating with the video data; and the presence detection position coordinates and the feature detection position associated with the video data A position coordinate determination step of determining whether or not there is a coordinate; and a first video search step of searching for the first video data of the first monitoring camera satisfying the search condition among the video data captured by the plurality of monitoring cameras And the time difference between the imaging date and time and the imaging date and time of the first video data among the video data captured by the plurality of surveillance cameras is a predetermined time or less, the feature detection position coordinates, and the first A second video search step of searching for second video data of a second monitoring camera, wherein the distance between the presence detection position coordinates of the video data is equal to or less than a predetermined distance, and the searched video data is output to the monitoring monitor An image output process, wherein the position coordinate determination process, the first image search process, the second image search process, and the image output process are executed in a recording apparatus. Temu is a video search method.

According to the present invention, the position coordinates at which the recording device that records information related to the image such as the imaged image and the imaged date and time transmitted from a plurality of monitoring cameras installed in the monitoring area detects the presence of the monitoring target such as a monitored person, for example. For example, the position coordinates at which the feature of the monitoring target such as the face of the monitored person is detected are acquired from the captured image, and the captured image including these two position coordinates is close to the date and time, and the two position coordinates are on the monitoring area. By determining whether or not they are close to each other, it is possible to determine whether or not the monitoring targets related to these two position coordinates included in the captured images from the same or different monitoring cameras are the same. . As a result, by detecting the presence of the monitoring target from the captured images captured by a plurality of monitoring cameras, the captured images from the same or different monitoring cameras at close dates and including the characteristics of the monitoring target Can be retrieved in a short time, and an effect of reproducing and displaying a surveillance video including a desired surveillance scene without requiring a high-cost apparatus for performing complicated processing can be obtained.

It is a block diagram which shows the outline of the video surveillance system based on Embodiment 1 of this invention. It is a block diagram which shows the flow of the data transmitted / received by the function part of a surveillance camera, and each function part. It is a block diagram which shows the flow of the data transmitted / received by the function part regarding recording of a recording device, and each function part. It is a block diagram which shows the function part regarding the reproduction | regeneration of a recording device, and the flow of the data transmitted / received by each function part. It is a figure which shows the structure of the data recorded on the video data recording part. It is a figure which shows the structure of the data recorded on the date data recording part. It is a figure which shows the structure of the data recorded on the characteristic image data recording part. It is a figure which shows the structure of the data recorded on the position coordinate data recording part. It is a figure which shows the structure of the data set to the camera cooperation table. It is a figure which shows the response | compatibility by which the setting in a camera cooperation table is reflected in each storage area of a recording medium. FIG. 11A is a schematic diagram in which two surveillance cameras are installed on the diagonal line of the surveillance area. FIG. 11B is a diagram showing video data obtained by imaging the monitored person in the monitoring area from the main monitoring camera. FIG. 11C is a diagram showing video data obtained by imaging the monitored person in the monitoring area from the sub-monitoring camera. It is a flowchart figure which shows a process until a recording device receives video data and metadata from a surveillance camera, and records them on a recording medium. It is a flowchart figure which shows the process until it searches video data based on the search conditions which the recording device input from the input device, and outputs it to a monitoring monitor. It is a figure of the example of a screen display which reproduced-displayed the reproduction | regeneration display video data which the recording device output on the monitoring monitor. It is a block diagram which shows the flow of the data transmitted / received by the function part of a surveillance camera based on Embodiment 2 of this invention, and each function part. It is a block diagram which shows the flow of the data transmitted / received by the function part regarding recording of a recording device, and each function part.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an outline of a video surveillance system. The video monitoring system includes a monitoring area 101, a monitoring room 102, a monitoring camera 103, an input device 104, a recording device 105, and a monitoring monitor 106.

The monitoring area 101 is an area to be monitored by the video monitoring system. The monitored person 108 in the monitoring area 101 is a monitoring target of the video monitoring system.

The monitoring room 102 is a room provided at a position away from the monitoring area 101 and includes a recording device 105, an input device 104, and a monitoring monitor 106. In the monitoring room 102, the monitoring worker 109 reproduces and displays the image captured by the monitoring camera 103 on the monitoring monitor 106 to monitor the status of the monitoring area 101.

A plurality of monitoring cameras 103 are installed in the monitoring area 101 and are imaging devices for imaging the monitored person 108 in the monitoring area 101. The monitoring camera 103 has a function of detecting the characteristics of the monitored person 108 from the captured video and a function of detecting the presence of the monitored person 108. The function of detecting the characteristics of the monitored person 108 is, for example, face detection. The function for detecting the presence of the monitored person 108 includes, for example, motion detection, sound detection, infrared detection, and the like.

Here, the face detection function includes, for example, a face recognition method using a unique face. In the motion detection function, a motion is detected by a partial change that occurs in a captured image with time, and position information in the monitoring area 101 is acquired. In the sound detection function, the position information where the sound is detected is acquired by using the directivity of the microphone. Further, in the infrared detection function, the position information of the monitoring target is acquired by detecting heat by an infrared detector in which the installation position in the monitoring area 101 is registered in advance. In the present invention, the position information is the position coordinates in the image obtained by imaging the monitoring area 101, and the position in the monitoring area 101 is specified by the position coordinates.

The input device 104 is a device that is connected to the recording device 105 and transmits a search condition input by the monitoring operator 109 to obtain a desired video as a reproduction request signal 410 to the recording device 105. The search conditions include, for example, the monitoring camera 103 that is capturing a desired monitoring scene among the plurality of monitoring cameras 103, the search order of the plurality of monitoring cameras 103, the imaging date and time, the location to be monitored in the monitoring area 101, and the like. Conceivable. One or more of these search conditions may be designated.

The recording device 105 is connected to a plurality of monitoring cameras 103, receives collectively the captured images transmitted by each monitoring camera 103, the imaging date and time, the position coordinates related to the monitored person 108, and records them on the recording medium 107. is there. The recording medium 107 is a storage device such as a hard disk or a memory card provided inside or outside the recording apparatus 105. Further, when the recording device 105 receives the reproduction request signal 410 from the input device 104, the recording device 105 searches the video recorded on the recording medium 107 for the video desired by the monitoring operator 109, and synthesizes the video as necessary. Output to the monitor 106.

The monitor 106 is a video / audio output device for reproducing and displaying the video output from the recording device 105.

FIG. 2 is a block diagram illustrating the functional units of the monitoring camera 103 and the flow of data exchanged between the functional units. The monitoring camera 103 includes a video imaging unit 201, a feature detection unit 202, a presence detection unit 203, a data management unit 204, and a data transmission unit 205.

Of the arrows connecting the functional units of the monitoring camera 103, the solid lines indicate the flow of the video data 206, and the dashed lines indicate the flow of the metadata 207. The metadata 207 indicates date and time data that is the date and time when the video data 206 was imaged, and position coordinates obtained from the video data 206.

The video imaging unit 201 is a functional unit for the surveillance camera 103 to capture a video. The video imaging unit 201 outputs the captured video together with the date / time data to the data management unit 204 as video data 206 for each frame.

The feature detection unit 202 is a functional unit that outputs, to the data management unit 204, the position coordinates where the feature of the monitored person 108 is detected from the video data 206. In the present invention, the position coordinates output by the feature detection unit 202 are referred to as feature detection position coordinates.

The presence detection unit 203 is a functional unit that outputs the position coordinates where the presence of the monitored person 108 is detected from the video data 206 to the data management unit 204. In the present invention, the position coordinates output by the presence detection unit 203 are referred to as presence detection position coordinates.

The data management unit 204 is a functional unit that manages the video data 206 and date / time data output from the video imaging unit 201 and the position coordinates output from the feature detection unit 202 and the presence detection unit 203. The data management unit 204 passes the video data 206 received from the video imaging unit 201 to the feature detection unit 202 and the presence detection unit 203, and acquires each position coordinate. Further, the data management unit 204 outputs the video data 206 and the metadata 207 including date / time data and each position coordinate to the data transmission unit 205 at a preset timing.

The data transmission unit 205 is a functional unit that transmits the video data 206 and metadata 207 output from the data management unit 204 to the recording apparatus 105.

FIG. 3 is a block diagram showing functional units related to recording of the recording apparatus 105 and the flow of data exchanged between the functional units.

3 includes a video data receiving unit 301, a metadata receiving unit 302, a data analyzing unit 303, a feature image clipping unit 304, a feature image data managing unit 305, and a position coordinate data managing unit 306. The recording medium 107 of the recording device 105 is divided into storage areas of a video data recording unit 307, a date / time data recording unit 308, a feature image data recording unit 309, and a position coordinate data recording unit 310, and information handled by each functional unit. It is recorded.

Of the arrows connecting the respective functional units and the storage areas of the recording apparatus 105, the solid line indicates the flow of the video data 206 or the characteristic image data, and the dotted line indicates the date and time. A flow of data and metadata 207 of each position coordinate is shown. The metadata 207 indicates date and time data that is the date and time when the video data 206 was imaged, and position coordinates obtained from the video data 206.

The video data receiving unit 301 is a functional unit that receives the video data 206 transmitted by the monitoring camera 103.

The metadata receiving unit 302 is a functional unit that receives the metadata 207 transmitted by the monitoring camera 103.

The data analysis unit 303 analyzes the video data 206 received by the video data reception unit 301 and the metadata 207 received by the metadata reception unit 302, and outputs various types of information to each storage unit of each functional unit or recording medium 107. It is a functional part. In addition, the data analysis unit 303 stores the video data 206 received from the video data reception unit 301 in the video data recording unit 307, and out of the metadata 207 received from the metadata reception unit 302, the date / time data is stored in the date / time data recording unit 308. Record.

The feature image cutout unit 304 is a functional unit that receives the video data 206 and the feature detection position coordinates from the data analysis unit 303 and cuts out feature image data from the video data 206 based on the feature detection position coordinates. The extracted feature image data is output to the feature image data management unit 305.

The feature image data management unit 305 is a functional unit that receives feature image data from the feature image cutout unit 304 and records it in the feature image data recording unit 309.

The position coordinate data management unit 306 is a functional unit that receives the feature detection position coordinates and the presence detection position coordinates from the data analysis unit 303 and records them in the position coordinate data recording unit 310.

FIG. 4 is a block diagram showing functional units related to reproduction of the recording apparatus 105 and the flow of data exchanged between the functional units.

4 includes a reproduction request receiving unit 401, a reproduction request analyzing unit 402, a search processing unit 403, a video searching unit 404, a position coordinate data collating unit 405, a feature image data searching unit 406, a video synthesizing unit 407, and a video. The display unit 408 is configured. Further, the recording apparatus 105 holds a camera cooperation table 409 that associates the position coordinates of the video data 206 captured by each monitoring camera 103.

Of the arrows connecting the respective functional units and the storage areas of the recording apparatus 105, the solid line indicates the flow of the video data 206 or the characteristic image data, and the dotted line indicates the date and time. A flow of data and metadata 207 of each position coordinate is shown. The metadata 207 indicates date and time data that is the date and time when the video data 206 was imaged, and position coordinates obtained from the video data 206.

The reproduction request receiving unit 401 is a functional unit that receives the reproduction request signal 410 from the input device 104.

The reproduction request analysis unit 402 is a functional unit that analyzes the search condition of the video data 206 that the monitoring operator 109 desires to reproduce and display from the content of the reproduction request signal 410 received by the reproduction request reception unit 401.

The search processing unit 403 is a functional unit that controls each functional unit according to the search condition analyzed by the reproduction request analyzing unit 402 and acquires the video data 206 and the feature image data from the recording medium 107. In addition, the search processing unit 403 sets initial values of the camera cooperation table 409 and refers to the set values to acquire offset information at the position coordinates of the video data 206 captured by each monitoring camera 103. The camera cooperation table 409 will be described later with reference to FIG.

The video search unit 404 is a functional unit that acquires the video data 206 corresponding to the video index designated by the search processing unit 403 from the video data recording unit 307 and outputs it. The video index will be described later with reference to FIG.

The position coordinate data collating unit 405 acquires a feature image index, a feature image position coordinate, and a presence detection position coordinate associated with the video index specified by the search processing unit 403 from the position coordinate data recording unit 310 and outputs the function image index. Part. Further, the position coordinate data collating unit 405 compares the presence detection position coordinate associated with one video data 206 with the feature detection position coordinate associated with another video data 206, and the presence detection position coordinate and the feature detection position. It is determined whether the distance between the coordinates is equal to or less than a predetermined distance. This determination will be described later with reference to FIG.

The feature image data search unit 406 is a functional unit that acquires and outputs the feature image data corresponding to the feature image index designated by the search processing unit 403 from the feature image data recording unit 309. The feature image index will be described later with reference to FIG.

The video composition unit 407 is a functional unit that generates the reproduction display video data 411 by superimposing the feature image data output by the feature image data search unit 406 on the video data 206 output by the video search unit 404 and combining the feature image data. . The generated reproduction display video data 411 is output to the video display unit 408. Further, the video composition unit 407 may output the video data 206 as it is without synthesizing the feature image data. Which of the video data 206 and the reproduction display video data 411 is output will be described later with reference to FIG.

The video display unit 408 is a functional unit that transmits the video data 206 or the reproduction display video data 411 received from the video synthesis unit 407 to the monitoring monitor 106.

5 to 8 are storage areas into which the recording medium 107 of the recording apparatus 105 is divided, which are configurations of a video data recording unit 307, a date / time data recording unit 308, a feature image data recording unit 309, and a position coordinate data recording unit 310, respectively. Indicates.

Data in each storage area of the recording medium 107 is recorded by being discriminated by the monitoring camera 103, and recording and searching of the video data 206 are performed in units of discrimination of the monitoring camera 103. 5 to 8, the n monitoring cameras 103 are represented as monitoring cameras C1 to Cn.

The search index common to each storage area of the recording medium 107 is an integer value of 1 or more that is assigned to specify the search order and specify the captured monitoring cameras C1 to Cn when searching the video data 206. It is. I is used as a variable indicating the search index. 5 to 8 show a state where no value is set in the search index yet. Setting of a value for the search index will be described with reference to FIG.

FIG. 5 is a diagram showing a configuration of data recorded in the video data recording unit 307.

In the video data recording unit 307, the data analysis unit 303 records a video index identifying video data 206 captured by the monitoring cameras C1 to Cn in association with each other. Here, the variable j is used for the video index for identifying the video data 206 captured by the same monitoring camera 103. A video index that is identified by the surveillance camera 103 and that identifies the video data 206 is represented as a video index (i, j) using a variable i of the search index and a variable j of the video index. Similarly, the video data 206 distinguished by the monitoring camera 103 is represented as video data (i, j) using the variable i of the search index and the variable j of the video index.

FIG. 6 is a diagram illustrating a configuration of data recorded in the date / time data recording unit 308.

The date / time data recording unit 308 records the date / time data, which is the date / time when each video data 206 is captured, in association with the video index corresponding to each video data 206 captured by the monitoring cameras C1 to Cn by the data analysis unit 303. Yes. The date / time data is represented as date / time data (i, j) using the variable i of the search index and the variable j of the video index.

FIG. 7 is a diagram illustrating a configuration of data recorded in the feature image data recording unit 309.

The feature image data recording unit 309 records the feature image data for distinguishing the feature image data in association with the feature image data extracted from the video data 206 by the feature image clipping unit 304 by the feature image data management unit 305. ing. Here, k is used as a variable for the feature image Index for identifying the video data 206 captured by the same monitoring camera 103. In addition, the feature image Index that is distinguished by the monitoring camera 103 and identifies the feature image data is represented as a feature image Index (i, k) by using a variable i of the search index and a variable k of the feature image index. Similarly, the feature image data distinguished by the monitoring camera 103 is represented as feature image data (i, k) using the variable i of the search index and the variable k of the feature image index.

FIG. 8 is a diagram illustrating a configuration of data recorded in the position coordinate data recording unit 310.

The position coordinate data recording unit 310 associates the feature image index, the feature detection position coordinate, and the presence detection position coordinate with the video index corresponding to each video data 206 captured by the monitoring cameras C1 to Cn by the position coordinate data management unit 306. Is recorded. Here, the position coordinates of the feature detection position coordinates are indicated by (fx, fy), and the position coordinates of the presence detection position coordinates are indicated by (ex, ey). Also, the feature detection position coordinates and the presence detection position coordinates are expressed as feature detection position coordinates (i, j) and presence detection position coordinates (i, j) using the variable i of the search index and the variable j of the video index.

When a plurality of monitored persons 108 are shown in the captured video, it is possible to associate a plurality of feature images, feature detection position coordinates, and presence detection position coordinates with one video data and the video index. In the present invention, description will be made assuming that one video image data is associated with one feature image index, feature detection position coordinates, and presence detection position coordinates.

FIG. 9 shows the configuration of data set in the camera cooperation table 409.

Taking the first line of the camera linkage table 409 as an example, of the monitoring cameras C1 to Cn, the monitoring camera C1 is the main monitoring camera, and the other monitoring cameras C2 to Cn are the sub monitoring cameras F1 to Fn-1. In the first row of the camera linkage table 409, offset information in position coordinates between the video data 206 captured by the main monitoring camera C1 and the video data 206 captured by the sub-monitoring cameras F1 to Fn-1 is set in advance. ing. In the other rows, offset information with the sub-monitoring cameras F1 to Fn-1 in which the monitoring cameras C2 to cn are the main monitoring cameras is set. By using this offset information, the video data 206 captured by each sub-surveillance camera can be coordinate-converted to the viewpoint of the main surveillance camera with an arbitrary main surveillance camera as a reference. The process of performing coordinate conversion using the offset information will be described later with reference to FIG.

Further, in the camera cooperation table 409, the selection of the primary monitoring camera that starts the search of the video data 206 first and the setting of the search index that is the order in which the secondary monitoring cameras F1 to Fn-1 are searched are performed. The monitoring camera C1 will be described below as a main monitoring camera.

First, from the camera cooperation table 409, the row corresponding to the main monitoring camera for the monitoring camera C1 to be searched first is selected and marked in the search start monitoring camera column. The first line corresponds to the camera cooperation table 409 in FIG. In the first row of the camera linkage table 409, monitoring cameras C2 to Cn corresponding to the sub monitoring cameras F1 to Fn-1 are arranged next to the monitoring camera C1 of the main monitoring camera.

Next, the search order is set in the search index at the bottom of the camera cooperation table 409 corresponding to the columns of the monitoring cameras C2 to Cn in the first row of the camera cooperation table 409. In FIG. 9, the search index corresponding to the surveillance cameras C2, C3... Cn is set to i = 2, 3,. Since the main monitoring camera is searched first, the search index of the monitoring camera C1 is i = 1.

By referring to the camera cooperation table 409 set as described above, the recording apparatus 105 can sequentially search the video data 206 captured by the monitoring cameras C1 to Cn.

The selection of the search start monitoring camera and the setting of the search index in the camera cooperation table 409 are determined by the contents of the search condition included in the playback request signal 410 from the input device 104, and the playback request signal 410 is received. Sometimes set by the search processing unit 403. Processing when the reproduction request signal 410 is received will be described later with reference to FIG.

FIG. 10 is a diagram illustrating the correspondence in which the settings in the camera cooperation table 409 are reflected in each storage area of the recording medium 107.

When the search start monitoring camera and the search index are set in the camera linkage table 409, the search processing unit 403 causes the video data recording unit 307, the date / time data recording unit 308, the feature image data recording unit 309, and the position coordinate data recording unit 310. The setting is reflected to the search index.

11A to 11C, the video data 206 captured by one monitoring camera 103 in the monitoring area 101 using the offset information set in the camera cooperation table 409 is displayed by the other monitoring camera 103. It is a figure which shows the process of performing coordinate conversion to the image | video data 206 imaged.

FIG. 11A is a schematic diagram in which two monitoring cameras 1101 and 1102 are installed on the diagonal line of the monitoring area 101. Here, it is assumed that the monitoring camera 1101 is a main monitoring camera and the monitoring camera 1102 is a sub-monitoring camera.

FIG. 11B is a diagram illustrating video data 1103 obtained by imaging the monitored person 108 in the monitoring area 101 from the main monitoring camera 1101.

The four corners of the monitoring area 101 shown in the video data 1103 are point A1, point B1, point C1, and point D1 clockwise from the upper left corner. The distance between the points A1 and B1 is X, and the distance between the points A1 and D1. Y. Here, when the point A1 is the origin of the position coordinates and the points A1 to D1 are represented by the coordinates (x, y), the origin A1 (x, y) = (0, 0) and the point B1 (x, y) = (X, 0), point C1 (x, y) = (X, Y) and point D1 (x, y) = (0, Y). In the video data 1103 in which the position coordinates are set in this way, if the position of the monitored person 108 is a point P, the function of the main monitoring camera 1101 for detecting the characteristics of the monitored person 108 and the presence of the monitored person 108 are shown. The feature detection position coordinates and the presence detection position coordinates, which are the position coordinates of the monitored person 108, are both specified as a point P (x, y) = (x1, y1) by the detection function.

FIG. 11C is a diagram showing video data 1104 obtained by imaging the monitored person 108 in the monitoring area 101 from the sub-monitoring camera 1102.

As with the video data 1103, the four corners of the monitoring area 101 shown in the video data 1104 are point A2, point B2, point C2, and point D2 clockwise from the upper left corner, and the intervals between the points A2 and B2 are X and point A2. And the distance between the point D2 and Y. Here, when the point A2 is the origin of the position coordinates and the points A2 to D2 are represented by the coordinates (x, y), the origin A2 (x, y) = (0, 0) and the point B2 (x, y) = (X, 0), point C2 (x, y) = (X, Y) and point D2 (x, y) = (0, Y). In the video data 1104 in which the position coordinates are set in this way, when the position of the monitored person 108 is a point S, the function of detecting the feature of the monitored person 108 and the presence of the monitored person 108 provided in the sub-monitoring camera 1102 are shown. By the detection function, both the feature detection position coordinates and the presence detection position coordinates, which are the position coordinates of the monitored person 108, are specified as a point S (x, y) = (x2, y2).

Incidentally, the point P (x1, y1) and the point S (x2, y2) of the position coordinates specified by the main monitoring camera 1101 and the sub-monitoring camera 1102 indicate the same place in the monitoring area 101. Therefore, an attempt is made to replace the point S (x2, y2) of the video data 1104 captured by the sub-monitoring camera 1102 with the point P (x1, y1) viewed from the viewpoint of the video data 1103 captured by the main monitoring camera 1101.

Here, the offset information of the sub-monitoring camera 1102 with respect to the main monitoring camera 1101 preset in the camera cooperation table 409 is x = X−x2 and y = y2. Using this offset information, when the point S_offset obtained by coordinate conversion of the point S (x2, y2) imaged by the sub-monitoring camera 1102 to the point P (x1, y1) imaged by the main monitoring camera 1101 is calculated, the point S_offset ( X-x2, y2).

As described above, the position coordinates of the video data 206 captured by an arbitrary monitoring camera 103 can be replaced with the position coordinates at the viewpoint from another monitoring camera 103 by using the offset information of the camera cooperation table 409.

FIG. 12 is a flowchart illustrating processing until the recording apparatus 105 receives the video data 206 and the metadata 207 from the monitoring camera 103 and records them on the recording medium 107.

In process S <b> 1201, the recording apparatus 105 receives video data 206 and metadata 207 from the monitoring camera 103. Then, it progresses to process S1202.

In process S1202, the data analysis unit 303 analyzes whether the feature detection position coordinates are included in the metadata 207 received from the monitoring camera 103. If the feature detection position coordinates are included, the process proceeds to step S1203. If the feature detection position coordinates are not included, the process proceeds to step S1205.

In step S1203, the position coordinate data management unit 306 records the feature detection position coordinates in the position coordinate data recording unit 310. Then, it progresses to process S1204.

In step S1204, the feature image data management unit 305 receives the feature image data from the feature image cutout unit 304 and records the feature image data in the feature image data recording unit 309. Then, it progresses to process S1206.

In step S1205, the position coordinate data management unit 306 sets the feature detection position coordinates as (fx, fy) = (0, 0) and records the position coordinate data because the metadata 207 does not include the feature detection position coordinates. Recorded in the unit 310. Then, it progresses to process S1206.

In step S1206, the data analysis unit 303 analyzes whether the presence detection position coordinates are included in the metadata 207 received from the monitoring camera 103. If the presence detection position coordinates are included, the process proceeds to step S1207. If the presence detection position coordinates are not included, the process proceeds to step S1208.

In step S1207, the position coordinate data management unit 306 records the presence detection position coordinates in the position coordinate data recording unit 310. Thereafter, the process ends.

In step S1208, the position coordinate data management unit 306 sets the presence detection position coordinate as (ex, ey) = (0, 0) because the metadata 207 received from the monitoring camera 103 does not include the presence detection position coordinate. And recorded in the position coordinate data recording unit 310. Thereafter, the process ends.

FIG. 13 is a flowchart illustrating processing until the recording device 105 searches the video data 206 based on the search condition input from the input device 104 and outputs the video data 206 to the monitoring monitor 106.

In FIG. 13, the number of the monitoring cameras 103 is n, and the monitoring camera 103 sets the search index value i in increments of 1 from the initial value 1 to the maximum n in order of the search indexes set in the camera linkage table 409. The captured video data 206 is searched. The surveillance camera 103 whose search index at this time is the value i is represented as surveillance camera (i).

In process S1301, the reproduction request receiving unit 401 receives the reproduction request signal 410 from the input device 104. Then, it progresses to process S1302.

In step S1302, the reproduction request analysis unit 402 analyzes the search condition for the video data 206 desired to be reproduced and displayed from the content of the received reproduction request signal 410. As the search conditions, for example, the monitoring camera 103 that first searches the video data 206, the search order of the plurality of monitoring cameras 103, the imaging date and time, information indicating a location such as position coordinates in the monitoring area 101, and a combination of these information Etc. are considered. In process S1302, it is assumed that the search order of the first monitoring camera 103 and the plurality of monitoring cameras 103 and the imaging date and time are specified as the search conditions. Then, it progresses to process S1303.

In process S1303, the variable i of the search index is set to the initial value 1. Then, it progresses to process S1304.

In process S1304, the search processing unit 403 selects the camera designated as the monitoring camera 103 to be searched first in the reproduction request signal 410 as the search start monitoring camera in the camera cooperation table 409, and sets i = 1 to the search index. Set. In addition, the search processing unit 403 sets the search order of the monitoring cameras 103 specified by the reproduction request signal 410 to the search index of each monitoring camera 103 in the camera cooperation table 409. If there is no designation of the search start monitoring camera or the search order in the search condition of the reproduction request signal 410, a value determined in advance by the video monitoring system is set. Then, it progresses to process S1305.

In process S1305, the search processing unit 403 directly or controls other functional units, and the correspondence between each monitoring camera 103 set in the camera cooperation table 409 and the search index is determined based on the video data recording unit 307 of the recording medium 107, The date and time data recording unit 308, the feature image data recording unit 309, and the position coordinate data recording unit 310 are reflected and set. Then, it progresses to process S1306.

In step S1306, the video index (1, j) associated with the monitoring camera (1) whose search index is i = 1 is searched for and acquired from the recording medium 107 that meets the search condition. Here, since the search condition includes the date and time, the date and time data recording unit 308 is searched. If the search condition includes position information in the monitoring area 101, the position coordinate data recording unit 310 is searched. Then, it progresses to process S1307.

In the process S1307, the feature detection position coordinate (1, j) and the presence detection position coordinate (1, j) associated with the video Index (1, j) acquired in the process S1306 are searched from the position coordinate data recording unit 310. And get. Then, it progresses to process S1308.

In process S1308, it is determined whether or not the acquired feature detection position coordinate (1, j) is (0, 0). The feature detection position coordinate is (0, 0). The feature of the monitored person 108 is not detected from the video data (1, j) associated with the video index (1, j) acquired in step S1306. It shows that. If the feature detection position coordinate (1, j) is (0, 0), the process proceeds to step S1309. If the feature detection position coordinate (1, j) is not (0, 0), the process proceeds to step S1319.

In process S1309, it is determined whether or not the acquired presence detection position coordinate (1, j) is (0, 0). The presence detection position coordinate (1, j) is (0, 0) is the presence of the monitored person 108 from the video data (1, j) associated with the video index (1, j) acquired in step S1306. Indicates that was not detected. If the presence detection position coordinate (1, j) is not (0, 0), the process proceeds to step S1310. If the presence detection position coordinate (1, j) is (0, 0), the process proceeds to step S1319.

The process S1310 is a loop process, and the processes from the process S1311 to the process S1314 are repeated as appropriate. The repetition condition at the beginning of the loop is that the search index variable i has a start value i = 2 and an end value i = number of surveillance cameras n-1, and is added to the variable i by 1 each time the loop returns to the start end. Indicates to do. According to this repetition condition, among the video data (i, j) captured by the monitoring camera (i) corresponding to the variable i, the feature of the monitored person 108 detected by the video index (1, j) is detected. Search for. When the variable i exceeds the end value, the video data (1, j) corresponding to the video index (1, j) is reproduced and displayed on the assumption that the video data 206 having detected the feature is not searched. It progresses to process S1319.

In process S1311, the video index (i, j) in which the time difference of the imaging date and time from the video index (1, j) acquired in process S1306 is equal to or less than a predetermined time is retrieved from the date data recording unit 308. The predetermined time here refers to 1 second, for example. Then, it progresses to process S1312.

In process S1312, the feature detection position coordinate (i, j) associated with the video Index (i, j) acquired in process S1311 is acquired from the position coordinate data recording unit 310. Then, it progresses to process S1313.

In process S1313, it is determined whether or not the feature detection position coordinates (i, j) acquired in process S1312 are (0, 0). If the feature detection position coordinate (i, j) is (0, 0), the process returns to the beginning of the loop processing in step S1310, and the video data 206 captured by another monitoring camera 103 is searched. If the feature detection position coordinate (i, j) is not (0, 0), the process proceeds to step S1314.

In process S1314, the camera cooperation table 409 is referred to, and the feature detection position coordinates (i, j) acquired in process S1312 are coordinate-converted with reference to the monitoring camera (1) of the main monitoring camera. Then, it progresses to process S1315.

In process S1315, it is determined whether or not the distance between the feature detection position coordinate (i, j) transformed in process S1314 and the presence detection position coordinate (1, j) acquired in process S1307 is equal to or smaller than a predetermined distance. To do. Here, the predetermined distance is arbitrarily set by the system. If the distance between the feature detection position coordinates (i, j) and the presence detection position coordinates (1, j) is equal to or less than the predetermined distance, the process proceeds to step S1316. If the distance between the feature detection position coordinates (i, j) and the presence detection position coordinates (1, j) is not less than or equal to the predetermined distance, the process returns to the start of the loop process in step S1310.

In process S1316, the feature image Index (i, k) associated with the video index (i, j) is acquired from the position coordinate data recording unit 310. Then, it progresses to process S1317.

In process S1317, the feature image data (i, k) associated with the feature image Index (i, k) acquired in process S1316 is retrieved and acquired from the feature image data recording unit 309. Then, it progresses to process S1318.

In the process S1318, the video data (1, j) associated with the video index (1, j) is searched and acquired from the video data recording unit 307, and the process S1317 is performed on the video data (1, j). Reproduced display video data 411 is generated by superimposing and synthesizing the feature image data (i, k) acquired in step (b). The position at which the feature image data (i, k) is superimposed and displayed on the video data (1, j) is, for example, a position that does not cover a region having a predetermined distance centered on the presence detection position coordinate (1, j). Then, it progresses to process S1319.

In the process S1319, the feature of the monitored person 108 is detected in the process S1308, the feature of the monitored person 108 is neither detected nor detected in the process S1309, or the loop processing repetition condition is exceeded in the process S1310. In such a case, the video data (1, j) is output to the monitor 106. In any case, if the process S1318 is performed, the reproduction display video data 411 generated in the process S1318 is output to the monitor 106. Thereafter, the process ends.

As described above, when a plurality of monitored persons 108 are shown in the video data 206 and a plurality of feature images Index are associated with the video Index, for example, the monitoring camera (1) searched first by the video Index. If so, the process advances to step S1319 to output video data 206 corresponding to the video index. If the video index is not the one of the surveillance camera (1) searched for first, a plurality of feature images Index is acquired in processing S1312, and feature image data corresponding to each feature image index is acquired in processing video S206 in processing S1318. In step S1319, reproduction display video data 411 obtained by combining a plurality of feature image data is output.

FIG. 14 is a diagram illustrating a screen display example in which the reproduction display video data 411 output from the recording device 105 is reproduced and displayed on the monitoring monitor 106.

The monitored person 1401 is the monitored person 108 shown in the presence detection position coordinate (1, j) in the video data (1, j) in the process S1318 of FIG. The feature image data 1402 is the feature image data (i, k) of the monitored person 1401 that is superimposed and displayed on the video data (1, j) in step S1318 of FIG.

According to the first embodiment, the recording device 105 that records information such as the captured video and the captured date and time transmitted from the plurality of monitoring cameras 103 installed in the monitoring area 101 detects the presence of the monitored person 108. The position coordinates and the feature detection position coordinates for detecting the characteristics of the monitored person 108 are acquired from the video data 206, and the video data 206 including these two position coordinates are close to each other, and the two position coordinates are the monitoring area 101. By determining whether or not they are close to each other, it is possible to determine whether or not the monitored person 108 related to these two position coordinates included in the captured images from the same or different monitoring cameras 103 is the same. It becomes possible. As a result, by detecting the presence of the monitored person 108 from the captured images captured by the plurality of monitoring cameras 103, the same or different monitoring including the characteristics of the monitored person 108 at a near date and time The video data 206 from the camera 103 can be searched in a short time, and the reproduction display video data 411 including a desired monitoring scene can be reproduced and displayed without requiring an expensive device that performs complicated processing.

Further, according to the first embodiment, the recording apparatus 105 cuts out the feature image data from the searched video data 206 from the video data 206 different from the desired monitoring scene, and the video data 206 of the desired monitoring scene. For example, it is possible to generate reproduction display video data 411 synthesized by superimposing and reproducing and displaying the data. As a result, when the recording device 105 searches for two pieces of video data 206, it is possible to reproduce and display as one piece of reproduced display video data 411 in which the presence and characteristics of the monitored person 108 are easily understood in a desired monitoring scene. Is obtained.

Further, according to the first embodiment, when the feature of the monitored person 108 is not detected from the video data 206 of the surveillance camera 103 searched for first, and only the presence is detected, the video data 206 of the other surveillance cameras 103 is detected. Thus, it is possible to acquire feature image data of the monitored person 108, generate playback display video data 411 in which the feature image data is synthesized with the video data 206 of the surveillance camera 103 searched first, and display the playback display video data 411. As a result, even if the video data 206 captured by another monitoring camera 103 in which the characteristics of the monitored person 108 are better understood than the video data 206 of the monitoring camera 103 searched first, the first monitoring camera 103 Thus, it is possible to reproduce and display the reproduction display video data 411 in which the presence and characteristics of the monitored person 108 can be recognized while maintaining the monitoring scene.

Further, according to the first embodiment, if the feature of the monitored person 108 can be detected in the video data 206 of the monitoring camera 103 searched first, it can be displayed on the monitoring monitor 106 as the reproduction display video data 411 as it is. . As a result, if the video data 206 of the surveillance camera 103 that is searched first can understand the presence and characteristics of the monitored person 108, an effect can be obtained that can be reproduced and displayed as it is without performing subsequent processing such as composition. .

Further, according to the first embodiment, the feature detection function and the presence detection function are implemented in the monitoring camera 103, and the feature detection position coordinates and the presence detection position coordinates of the monitored person 108 are obtained on the monitoring camera 103 side. Can be obtained. As a result, it is not necessary to install the feature detection function and the presence detection function on the recording apparatus 105 side, and to calculate each position coordinate of the monitored person 108, and the processing load of the recording apparatus 105 can be reduced. .

Further, according to the first embodiment, the recording device 105 records from the input device 104 by specifying at least one of the monitoring camera 103, the imaging date and time, and information indicating the position or area in the monitoring area 101 as a search condition. The retrieved video data 206 can be searched. As a result, it is possible to arbitrarily specify the scene, the date and time, and the location in the monitoring area to be monitored by the monitoring operator 109 as search conditions, and the video data 206 related to the conditions can be searched.

Embodiment 2. FIG.
In the first embodiment, the feature detection function and the presence detection function are implemented on the monitoring camera 103 side, and the feature detection position coordinates and the presence detection position coordinates of the monitored person 108 are obtained using these functions. It was transmitted to the recording device 105. In the second embodiment, the monitoring camera 1501 does not implement the feature detection function and the presence detection function, and obtains the feature detection position coordinates and the presence detection position coordinates of the monitored person 108 on the recording device 105 side.

FIG. 15 is a block diagram illustrating the functional units of the monitoring camera 1501 and the flow of data exchanged between the functional units. The components denoted by the same reference numerals as those in FIG. 2 are the same as those in the first embodiment, and thus the description thereof is omitted.

The monitoring camera 1501 includes a feature detection unit 202 that detects the characteristics of the monitored person 108 based on the captured video data 206 and a presence detection unit 203 that detects the presence of the monitored person 108 from the monitoring camera 103 of FIG. Excluded.

FIG. 16 is a block diagram showing functional units related to recording of the recording device 1601 and the flow of data exchanged between the functional units. The components denoted by the same reference numerals as those in FIG. 3 are the same as those in the first embodiment, and thus the description thereof is omitted.

The recording device 1601 includes a feature detection unit 1602 that detects the feature of the monitored person 108 based on the captured video data 206 and a presence detection unit 1603 that detects the presence of the monitored person 108 in the recording device 105 of FIG. It is implemented.

The feature detection unit 1602 receives the video data 206 from the data analysis unit 303 and outputs the feature detection position coordinates. Similar to the first embodiment, the data analysis unit 303 outputs the received feature detection position coordinates to the feature image cutout unit 304 and the position coordinate data management unit 306.

The presence detection unit 1603 receives the video data 206 from the data analysis unit 303 and outputs the monitoring target position coordinates. Similarly to the first embodiment, the data analysis unit 303 outputs the received monitoring target position coordinates to the position coordinate data management unit 306.

According to the second embodiment,
By installing the feature detection function and the presence detection function in the recording device 1601, it is possible to obtain the feature detection position coordinates and the presence detection position coordinates of the monitored person 108 on the recording device 1601 side. As a result, it is not necessary to install the feature detection function and the presence detection function in each monitoring camera 1501, and to calculate the position coordinates of the monitored person 108, thereby simplifying the configuration of the monitoring camera 1501 and reducing costs. An effect is obtained.

101 Monitoring Area 102 Monitoring Room 103, 1501 Monitoring Camera 104 Input Device 105, 1601 Recording Device 106 Monitoring Monitor 107 Recording Medium 108 Monitored Person 109 Monitoring Worker 201 Video Imaging Unit 202, 1602 Feature Detection Unit 203, 1603 Presence Detection Unit 204 Data management unit 205 Data transmission unit 206 Video data 207 Metadata 301 Video data reception unit 302 Metadata reception unit 303 Data analysis unit 304 Feature image clipping unit 305 Feature image data management unit 306 Position coordinate data management unit 307 Video data recording unit 308 Date / time data recording unit 309 Feature image data recording unit 310 Position coordinate data recording unit 401 Playback request receiving unit 402 Playback request analysis unit 403 Search processing unit 404 Video search unit 405 Position coordinate data matching unit 406 Feature image data detection Section 407 Video composition section 408 Video display section 409 Camera linkage table 410 Playback request signal 411 Playback display video data 1101 Primary monitoring camera 1102 Secondary monitoring camera 1103 Video data 1104 by primary monitoring camera Video data 1401 by secondary monitoring camera 1401 Playback display video data Monitored person 1402 to be reflected Feature image data to be displayed superimposed on playback display video data

Claims (8)

A plurality of surveillance cameras for imaging the surveillance area;
A recording device for recording the video data captured by the surveillance camera and the imaging date and time of the video data;
An input device for inputting search conditions for the video data;
A monitor for reproducing and displaying the video data output by the recording device;
Presence detection means for detecting the presence of a monitoring target from the video data, outputting presence detection position coordinates, which are position information in the monitoring area, and associating with the video data;
A feature detection unit that detects a feature of a monitoring target from the video data, outputs a feature detection position coordinate that is position information in the monitoring area, and associates the feature detection position coordinate with the video data;
A video surveillance system comprising:
The recording device comprises:
Position coordinate determination means for determining whether or not there is the presence detection position coordinate and the feature detection position coordinate associated with the video data;
First video search means for searching for first video data of a first surveillance camera that satisfies the search condition from among the video data captured by the plurality of surveillance cameras;
Among the video data captured by the plurality of monitoring cameras, a time difference between the imaging date and time and the imaging date and time of the first video data is equal to or less than a predetermined time, the feature detection position coordinates, and the first video data A second video search means for searching for second video data of the second surveillance camera, wherein a distance between the presence detection position coordinates is equal to or less than a predetermined distance;
Video output means for outputting the searched video data to the monitoring monitor;
Having
A video surveillance system characterized by that. The recording device comprises:
A feature image data cutout unit for cutting out the feature detected by the feature detection unit from the video data and outputting it as feature image data;
Video synthesizing means for generating third video data obtained by synthesizing the characteristic image data cut out from the second video data by the feature image data cutting out means with the first video data;
Further comprising
2. The video monitoring system according to claim 1, wherein the video output means outputs the first video data or the third video data to the monitoring monitor. The recording device comprises:
When it is determined by the position coordinate determining means that the first video data does not have the feature detection position coordinates and has the presence detection position coordinates,
Generating the third video data by the video synthesizing means;
Outputting the third video data to the monitoring monitor by the video output means;
The video surveillance system according to claim 2. The recording device comprises:
When it is determined by the position coordinate determination means that the first video data has the presence detection position coordinates and the feature detection position coordinates,
Outputting the first video data to the monitoring monitor by the video output means;
The video surveillance system according to claim 2, wherein the video surveillance system is a video surveillance system. The surveillance camera has the presence detection unit and the feature detection unit,
By the presence detection means and the feature detection means,
Outputting the presence detection position coordinates and the feature detection position coordinates of the imaged image data to the recording device;
The video surveillance system according to claim 1, wherein the video surveillance system is a video surveillance system. The recording apparatus includes the presence detection unit and the feature detection unit,
By the presence detection means and the feature detection means,
From the video data captured by the monitoring camera, the presence detection position coordinates and the feature detection position coordinates are output.
The video surveillance system according to claim 1, wherein the video surveillance system is a video surveillance system. The search condition is:
The video surveillance system according to claim 1, wherein the video surveillance system is a condition for designating at least one of information indicating the monitoring camera, the imaging date and time, and the position or area in the monitoring area. A plurality of surveillance cameras for imaging the surveillance area;
A recording device for recording the video data captured by the surveillance camera and the imaging date and time of the video data;
An input device for inputting search conditions for the video data;
A monitor for reproducing and displaying the video data output by the recording device;
A video search method executed in a video surveillance system having:
A presence detection step of detecting the presence of a monitoring target from the video data, outputting presence detection position coordinates, which are position information in the monitoring area, and associating with the video data;
A feature detection step of detecting a feature to be monitored from the video data, outputting feature detection position coordinates, which are location information in the monitoring area, and associating the feature with the video data;
A position coordinate determination step for determining whether or not there is the presence detection position coordinate and the feature detection position coordinate associated with the video data;
A first video search step of searching for the first video data of the first surveillance camera satisfying the search condition among the video data captured by the plurality of surveillance cameras;
Among the video data captured by the plurality of monitoring cameras, a time difference between the imaging date and time and the imaging date and time of the first video data is equal to or less than a predetermined time, the feature detection position coordinates, and the first video data A second video search step of searching for the second video data of the second surveillance camera, wherein a distance between the presence detection position coordinates is less than a predetermined distance;
A video output step of outputting the searched video data to the monitoring monitor;
Have
A video search method for a video surveillance system, wherein a position coordinate determination step, a first video search step, a second video search step, and a video output step are executed in a recording apparatus.

Images