JP2017033459A - Monitoring apparatus, monitoring system, and monitoring method - Google Patents

Abstract

[PROBLEMS] To improve the search efficiency of a dangerous person. A monitoring device includes an imaging unit that captures an image of a location to be monitored, a heartbeat detection unit that detects a heartbeat of a person based on a change in skin color included in the image captured by the imaging unit, And a warning target detection unit that detects a warning target person who is a dangerous person candidate based on the heart rate detected by the heart rate detection unit. [Selection] Figure 2

Description

  The present invention relates to a monitoring device, a monitoring system, and a monitoring method.

  In recent years, a monitoring device that detects a face from an image captured by a monitoring camera and classifies a person based on the detected feature amount of the face is known (for example, see Patent Document 1). In such a monitoring device, for example, the elapsed time is measured for each classified person, and when the elapsed time exceeds a predetermined time threshold, the person is determined to be a dangerous person candidate.

JP 2009-77064 A

  However, in the above-described monitoring apparatus, when the elapsed time exceeds a predetermined time threshold, the person is determined to be a dangerous person candidate. A person or a person who has only been there for a long time may be determined to be a dangerous person candidate. Therefore, in the above-described monitoring device, all persons who are determined to be dangerous person candidates including persons who have been in the store for a long time or who have only been in the store for a long time due to wondering about the purchase are subject to warning. There is a case where it is necessary to be wary as a person, and the process of searching for a dangerous person is inefficient.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a monitoring device, a monitoring system, and a monitoring method capable of improving the search efficiency of dangerous persons.

  In order to solve the above problem, an aspect of the present invention is based on an imaging unit that captures an image of a location to be monitored, and a change in skin color included in the image captured by the imaging unit. A monitoring apparatus comprising: a heartbeat detection unit that detects a heartbeat; and a warning target detection unit that detects a person to be warned based on the heartbeat detected by the heartbeat detection unit.

  According to the present invention, since the person to be warned who is a dangerous person candidate can be efficiently detected based on the heartbeat, the search efficiency of the dangerous person can be improved.

It is a hardware block diagram which shows an example of the monitoring apparatus by 1st Embodiment. It is a functional block diagram which shows an example of the monitoring apparatus by 1st Embodiment. It is a figure which shows the example of a detection of the change of the skin color by the color change detection part of 1st Embodiment. It is a figure which shows the example of a heartbeat detection by the same heartbeat detection part of 1st Embodiment. It is a figure which shows the example of a detection of the person who is a warning object by the warning object person detection part of 1st Embodiment. It is a flowchart which shows an example of operation | movement of the monitoring apparatus by 1st Embodiment. It is a flowchart which shows an example of operation | movement of the skin color detection part by 1st Embodiment. It is a flowchart which shows an example of operation | movement of the color change detection part and heart rate determination part by 1st Embodiment. It is a flowchart which shows an example of operation | movement of the same heartbeat detection part by 1st Embodiment, and a warning target person detection part. It is a functional block diagram which shows an example of the monitoring system by 2nd Embodiment.

Hereinafter, a monitoring device and a monitoring system according to an embodiment of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a hardware configuration diagram illustrating an example of a monitoring device 1 according to the first embodiment.
As shown in FIG. 1, the monitoring device 1 includes a CPU (Central Processing Unit), an input unit 102, an output unit 103, a ROM (Read Only Memory) 104, a RAM (Random Access Memory) 105, and a communication unit. 106. The CPU 101, the input unit 102, the output unit 103, the ROM 104, the RAM 105, and the communication unit 106 are connected to each other by an internal bus BS.

  The CPU 101 reads out various data stored in the ROM 104 and the RAM 105, executes various processes, and controls each unit of the monitoring device 1. The input unit 102 is an input device such as a camera equipped with a mouse, a keyboard, a touch panel, a CCD (Charge Coupled Device), and the like. The output unit 103 is an output device such as a display unit or a speaker. The ROM 104 stores a program for operating each functional unit of the monitoring device 1 and various data. The ROM 104 may include an electrically rewritable nonvolatile memory such as a flash memory, and may store data acquired via the input unit 102 or the communication unit 106. The RAM 105 temporarily stores various data. The communication unit 106 has a communication interface and is connected to a network by wire or wireless.

Next, the functional configuration of the monitoring apparatus 1 according to the present embodiment will be described with reference to FIG. FIG. 2 is a functional block diagram illustrating an example of the monitoring device 1 according to the present embodiment.
As shown in FIG. 2, the monitoring device 1 includes an imaging unit 11, a heartbeat detection unit 10, and a guard target person detection unit 16. Here, each part of the monitoring device 1 shown in FIG. 2 is realized using the hardware of the monitoring device 1 shown in FIG.

The imaging unit 11 is realized by the input unit 102 which is a camera, for example, and has a function of capturing a moving image (image). The imaging unit 11 captures a location to be monitored, and stores the captured moving image (image) in, for example, the RAM 105 or outputs it to an external storage device (not shown) via the communication unit 106.
The heartbeat detection unit 10 is realized by, for example, the CPU 101, the ROM 104, and the RAM 105, and detects the heartbeat of a person based on a skin color change included in a moving image (image) captured by the imaging unit 11. The heartbeat detection unit 10 includes a skin color detection unit 12, a color change detection unit 13, a heartbeat determination unit 14, and the same heartbeat detection unit 15.

  The skin color detection unit 12 detects the skin color for each frame of the moving image captured by the imaging unit 11 (hereinafter referred to as a moving image frame). For example, the skin color detection unit 12 acquires an image captured by the imaging unit 11, and detects an area within a predetermined color information range as an area of skin color among the images captured by the imaging unit 11. . That is, the skin color detection unit 12 has a predetermined range (for example, an H value (hue value) range, an S value (saturation value) range, and a V value (lightness value) in the HSV model color space). Are determined as skin colors, and whether the color is a skin color is detected according to whether image information (pixel information) is included in the range. Further, the skin color detection unit 12 stores (records) the detected region information (coordinate information in one frame), image information, and time information such as a moving image frame number in the RAM 105 in association with each other. In the present embodiment, an example in which the color space of the image information is an HSV model will be described as an example, but another model such as an RGB model may be used.

  The color change detection unit 13 detects changes in color components for each region in the regions and skin colors recorded over a plurality of moving image frames detected by the skin color detection unit 12. That is, the color change detection unit 13 detects a change in color information in the skin color region detected by the skin color detection unit 12 as a change in skin color. For example, the color change detection unit 13 detects a change in color information as shown in FIG.

FIG. 3 is a diagram illustrating an example of detection of a skin color change by the color change detection unit 13 of the present embodiment.
In the example illustrated in FIG. 3, the color change detection unit 13 calculates an average value for each image element as a color component (color information) in “region 1” detected in the moving image frame. The color change detection unit 13 is, for example, a region detected by the skin color detection unit 12 is 20 × 20 pixels, and the H component, S component, and V component of the HSV model are each a total of 20 × 20 pixels. The average value for 400 pixels is calculated. The color change detection unit 13 performs such calculation of the average value of the color components for each moving image frame, so that the waveforms (W1, W2) as shown in FIGS. 3A and 3B are obtained. can get. Note that the color change detection unit 13 similarly calculates an average value for each moving image frame for all the skin color regions detected by the skin color detection unit 12.

FIG. 3A shows a change in skin color in “region 1” detected in a certain moving image. In this figure, the vertical axis indicates the average value of the skin color, and the horizontal axis indicates the image acquisition time. Here, the waveform W1 indicates a change in the average value of the skin color in “region 1”.
FIG. 3B shows a change in skin color in a certain region different from that in FIG. In this figure, the vertical axis indicates the average value of the skin color, and the horizontal axis indicates the image acquisition time. Here, the waveform W2 indicates a change in the average value of the skin color in a certain region different from that in FIG. As shown in FIGS. 3A and 3B, the color change detection unit 13 detects a change in color information for each skin color region.

  Returning to the description of FIG. 2, the heartbeat determination unit 14 detects a heartbeat based on the change in the color information detected by the color change detection unit 13. That is, the heartbeat determination unit 14 examines the color change status of each region as shown in FIG. 3 obtained by detection by the color change detection unit 13 and determines whether or not the peak of the change is a heartbeat cycle. To do. For example, the heart rate determination unit 14 extracts color information peaks such as the peaks P1 to P5 shown in FIG. 3 and determines whether or not the peaks appear periodically. For example, in the case of the waveform W1 shown in FIG. 3A, since the peak P1, the peak P2, and the peak P3 are periodic, the heartbeat determining unit 14 determines that the color change in the “region 1” is due to the heartbeat. It is determined that there is a heartbeat (the heartbeat can be detected). Further, for example, in the case of the waveform W2 shown in FIG. 3B, since the peaks P4 and P5 are present but not periodic, the heartbeat determining unit 14 determines that the color change is not due to the heartbeat.

In addition, the heart rate determination unit 14 calculates a time interval between a plurality of extracted peaks (for example, time intervals t1, t2, and t3 shown in FIG. 3). The heartbeat determination unit 14 determines a heartbeat based on the calculated peak time interval (hereinafter, also referred to as a peak interval). Here, the heart rate per minute is, for example, about 60 to 200 times. In general, a heart rate of about 200 times is a considerably high heart rate, but the heart rate determination unit 14 also detects a large heart rate in order to detect a person who should be wary that may be a crime. To do. In addition, although it may be out of the range depending on the situation of the individual, in that case, the threshold value may be widened. The heart rate determination unit 14 determines whether or not the above-described peak time interval (for example, time intervals t1, t2, t3, etc.) is a heart rate depending on, for example, a range of 300 ms (milliseconds) to 1000 ms. (Or change due to heartbeat).
Since the color change (change in color information) is calculated based on the H value, S value, and V value of the image element, the heart rate determination unit 14 determines any of the H value, S value, and V value. In this change, if the above condition is satisfied, it is determined that the pulse is a heartbeat (or a change due to a heartbeat).

  The same heartbeat detection unit 15 detects whether or not the plurality of regions for which the change in the color information is determined to be a heartbeat by the heartbeat determination unit 14 is the same (including almost the same) change timing region. . For example, as shown in FIG. 4, the same heartbeat detection unit 15 compares the peak positions of the color information changes for each detected area, and determines that the areas where the peak positions (peak timings) match are the same person. It is determined that the skin color region. The same heart rate detection unit 15 detects the region as the heart rate of the same person because “region 1” and “region 3” shown in FIG. 4 have the same peak position. To do. The same heartbeat detection unit 15 does not completely match in time at each change timing, so a threshold value (for example, 20 ms) is provided, and the timing is matched if it is within the threshold value. You may make it determine.

FIG. 4 is a diagram illustrating a heartbeat detection example by the same heartbeat detection unit 15 of the present embodiment.
In FIG. 4, (a) to (e) represent the peak positions of heartbeats in “region 1” to “region 5” with bold lines. In FIG. 4, the horizontal axis indicates time.
In the example illustrated in FIG. 4, “Region 1” indicates a peak position at time T1, time T4, time T6, time T9, time T12, and time T15. In addition, since the heart rate of “region 3” coincides with the timing of “region 1”, the same heart beat detecting unit 15 detects “region 1” and “region 3” as the heart beat of the same person.

“Region 2” indicates that the peak position is at time T2, time T3, time T5, time T7, time T8, time T10, time T11, time T13, time T14, and time T16. In addition, since “Heart 5” has the same heartbeat timing as “Area 2”, the same heart beat detecting unit 15 detects “Area 2” and “Area 5” as heartbeats of the same person.
In this way, the same heartbeat detection unit 15 determines whether or not it is a skin color region of the same person based on the peak timing of the color information. Further, the same heartbeat detection unit 15 determines whether the region is a skin color region of the same person based on the change in the color information detected by the color change detection unit 13, and the skin color of the same person. The heartbeat of the same person is detected based on the change of the color information in the skin color region determined to be the region.

  Returning to the description of FIG. 2 again, the warning target person detection unit 16 (an example of the warning target detection unit) detects a warning target person who is a dangerous person candidate based on the heartbeat detected by the heartbeat detection unit 10. . In general, it is known that criminals are in an excited state and tend to have a higher heart rate than ordinary people. Therefore, the warning target person detection unit 16, for example, when the heartbeat interval detected by the heartbeat detection unit 10 is equal to or less than a predetermined interval, or the change in the detected heartbeat interval is equal to or greater than a predetermined range. In this case, the person corresponding to the heartbeat is detected as a person to be warned. The vigilance target person detection unit 16 monitors the area determined as the same person by the same heart rate detection unit 15 as the same person, and monitors the heart rate or a change in the heart rate. Further, the alert target person detection unit 16 similarly monitors an independent region as one person, and monitors the heart rate or a change in the heart rate.

  In the example illustrated in FIG. 4, the warning target person detection unit 16 monitors, for example, “area 1” and “area 3” as the same person, and monitors “area 2” and “area 5” as the same person. , “Region 4” is monitored as one person. For example, since the interval between the heartbeats of “region 2” and “region 5” is equal to or less than a predetermined interval (for example, 600 mS), the warning target person detection unit 16 is a person corresponding to “region 2” and “region 5”. Are detected as persons to be warned. In “area 4”, the heartbeat interval becomes shorter (the heart rate becomes higher) after time T10, and the change in the heartbeat interval is not less than a predetermined range (for example, ± 30%). The target person detection unit 16 detects a person corresponding to “region 4” as a person to be warned. In other words, in the example illustrated in FIG. 4, the warning target person detection unit 16 uses two persons, the person corresponding to “area 2” and “area 5”, and the person corresponding to “area 4” as the warning target persons. To detect.

As shown in FIG. 5, the same heart rate detection unit 15 and the warning target person detection unit 16 associate “area number”, “group identifier”, and “warning target” with each other and store them in, for example, the RAM 105. .
FIG. 5 is a diagram illustrating an example of detection of a person to be watched by the guard target person detecting unit 16 according to the present embodiment. In the example shown in FIG. 5, the same heartbeat detecting unit 15 detects “area 1” and “area 3” of “area number” as the same person, and assigns “G1” as “group identifier”. In addition, in the area where the “group identifier” is “G1”, the warning target person detection unit 16 has a heartbeat interval larger than a predetermined interval (for example, 600 mS), and a change in heartbeat interval is within a predetermined range ( For example, since it is less than ± 30%), the “warning target” is set to “unnecessary”.

  Further, the same heartbeat detecting unit 15 detects “area 2” and “area 5” of “area number” as the same person, and assigns “G2” as “group identifier”. In addition, in the area where the “group identifier” is “G2”, the warning target person detection unit 16 sets “warning target” to “necessary” because the heartbeat interval is equal to or less than a predetermined interval (for example, 600 mS).

Next, with reference to FIGS. 6-9, operation | movement of the monitoring apparatus 1 of this embodiment is demonstrated.
FIG. 6 is a flowchart illustrating an example of the operation of the monitoring device 1 of the present embodiment.
As shown in FIG. 6, first, the monitoring device 1 captures a moving image (step S101). That is, the imaging unit 11 of the monitoring device 1 captures a moving image of a location to be monitored, and stores the captured moving image in, for example, the RAM 105 for each moving image frame.
Next, the heartbeat detection unit 10 (skin color detection unit 12) of the monitoring device 1 detects the skin color for each moving image frame (step S102).
Next, the heartbeat detection unit 10 (color change detection unit 13) detects a change in the color component for each skin color region (step S103).

Next, the heartbeat detection unit 10 (heartbeat determination unit 14) determines a heartbeat for each skin color region (step S104).
Next, the heartbeat detection unit 10 (same heartbeat detection unit 15) detects a person's heartbeat (step S105). That is, the same heart rate detection unit 15 of the heart rate detection unit 10 determines whether or not it is a skin color region in the same person based on the timing of the peak of the color information. The same heart beat detecting unit 15 detects the heart beat of the same person based on the change of the color information in the skin color area determined to be the skin color area of the same person.

  Next, the warning target person detection unit 16 of the monitoring device 1 detects a warning target person based on the heartbeat of the person (step S106). After the processing in step S106, the monitoring apparatus 1 returns the processing to step S101, and repeats the processing in steps S101 to S106 at regular intervals (for example, 5 seconds) to detect a person to be warned.

Next, with reference to FIG. 7, the details of the processing in step S102 of FIG. 6 described above will be described. FIG. 7 is a flowchart showing an example of the operation of the skin color detection unit 12 of the present embodiment.
In FIG. 7, the skin color detection unit 12 first acquires an image (step S201). For example, the skin color detection unit 12 acquires a moving image stored in the RAM 105 for each moving image frame. That is, for example, the skin color detection unit 12 acquires image information for one frame of a moving image captured by the imaging unit 11 (for example, 640 × 480 pixels and an image element is an HSV model).

  Next, the skin color detection unit 12 detects a skin color region (step S202). The skin color detection unit 12 detects skin color in an image of 640 × 480 pixels, for example. The skin color detection unit 12 is, for example, a predetermined range (for example, an H value (hue value) range, an S value (saturation value) range, and a V value (lightness value) in the color space of the HSV model). Are determined as skin colors, and whether the color is a skin color is detected according to whether image information (pixel information) is included in the range.

Next, the skin color detection unit 12 records the color information of the detected skin color region (step S203). For example, the skin color detection unit 12 determines the average value of the detected skin color H value, S value, and V value, and the position coordinates (for example, upper left coordinates (10, 400), lower right coordinates ( Coordinate information such as 30, 380) is stored in the RAM 105. In addition, the skin color detection unit 12 manages a plurality of areas for each area and stores them in the RAM 105. For example, when the detection of the skin color is completed for the entire image of 640 × 480 pixels (after the process of step S203), the skin color detection unit 12 returns the process to step S201, and the next 1 of the moving image. Execute processing for the frame.
Thereby, in the monitoring apparatus 1, each average value and position coordinate of the skin color are managed for each moving image frame (= every time), and each region is set with a region number (for example, “region 1”, “ Area 2 ”etc.).

In addition, when the position of the skin color to detect moves, the skin color detection part 12 may perform a process as follows.
The skin color detection unit 12 calculates an average value and position coordinates of the skin color in the skin color region detected in a certain moving image frame. Then, the skin color detection unit 12 similarly calculates the average value and position coordinates of the skin color in the skin color region detected in the next moving image frame. In the skin color area of the two moving image frames, the skin color detection unit 12 has a color information difference within a predetermined threshold (for example, a value that takes into account the change in skin color). When the difference between the position coordinates is also within a preset threshold value, it is determined that the same skin color region has moved, thereby tracking the skin color region.
In addition, skin color detection may be affected by changes in the environment (for example, the color temperature of the lighting), but this can be dealt with with existing camera functions such as white balance and brightness adjustment. It is.

Next, details of the processing in step S104 in FIG. 6 will be described with reference to FIG. FIG. 8 is a flowchart illustrating an example of the operation of the heartbeat determination unit 14 of the present embodiment.
In FIG. 8, the heart rate determination unit 14 first calculates a peak interval from the peak of skin color change (step S301). For example, the color change detection unit 13 detects the peak position of the change from the change in skin color at each image acquisition time of each region as shown in FIG. A peak interval such as t2 is calculated.

Next, the heartbeat determining unit 14 determines whether or not the change is periodic and the peak interval is within a range of 300 ms to 1000 ms (step S302). The heart rate determination unit 14 determines that a plurality of peak positions are detected and each peak interval is within a range of 300 ms to 1000 ms and has periodicity, for example, as in the waveform W1 of FIG. If so (step S302: YES), the process proceeds to step S303. Moreover, the heartbeat determination part 14 advances a process to step S304, when there is no periodicity in a change or a peak space | interval is outside the range of 300 ms-1000 ms (step S302: NO).
As an example of detecting the skin color other than a person, a person image printed on a poster or the like can be considered. In such a case, for example, as indicated by the waveform W2 in FIG. 3B, the peak position is irregular and the peak interval is unlikely to be within the range of 300 ms to 1000 ms. The process proceeds to step S304.

In step S303, the heartbeat determination unit 14 determines that the color information change in the region is a heartbeat change (the color information change is a heartbeat). After the process of step S303, the heartbeat determination unit 14 ends the heartbeat determination process.
In step S304, the heartbeat determination unit 14 determines that the color information change in the region is other than the heartbeat (the color information change is not a heartbeat). After the process of step S304, the heartbeat determination unit 14 ends the heartbeat determination process.

Next, with reference to FIG. 9, the details of the processing in step S105 and step S106 in FIG. 6 will be described. FIG. 9 is a flowchart illustrating an example of the operations of the same heartbeat detection unit 15 and the guard target person detection unit 16 of the present embodiment.
In FIG. 9, the same heartbeat detection unit 15 first acquires change timings of M regions (“region 1” to “region M”) determined to be heartbeats (step S401). For example, in the example illustrated in FIG. 4, the same heartbeat detection unit 15 acquires change timings in five (= M) regions, “region 1” to “region 5”, which are determined to be heartbeats.

Next, the same heartbeat detection unit 15 treats the number of the target region as a variable N, sets “1” (N = 1), and initializes (step S402).
Next, the same heartbeat detection unit 15 determines whether or not a group identifier is added to “region N” (step S403). When the group identifier is added to “region N” (step S403: YES), the same heartbeat detecting unit 15 advances the process to step S409. In addition, when the group identifier is added to “region N” (step S403: NO), the same heartbeat detection unit 15 advances the process to step S404.

In step S <b> 404, the same heartbeat detection unit 15 compares the change timings of “region N” with other regions. Then, the same heartbeat detection unit 15 determines whether or not there is a region having the same change timing (step S405). If the same heartbeat detection unit 15 has a region with the same change timing (step S405: YES), the process proceeds to step S406. The same heart beat detecting unit 15 advances the process to step S407 when there is no region with the same change timing (step S405: NO).

In step S406, the same heartbeat detection unit 15 adds a group identifier (for example, “G1” shown in FIG. 5) as a group of “region N” with the region having the same change timing, and advances the processing to step S407.
In step S407, the warning target person detection unit 16 determines whether the heartbeat interval is 600 ms or less or the change in heartbeat interval is ± 30% or more with respect to the “region N” (or group identifier). Determine. If the heartbeat interval is 600 ms or less or the change in heartbeat interval is ± 30% or more (step S407: YES), the alert target person detection unit 16 advances the processing to step S408. Further, when the heartbeat interval is greater than 600 ms and the change in heartbeat interval is less than ± 30% (step S407: NO), the alert target person detection unit 16 advances the process to step S409.
For example, “Region 4” shown in FIG. 6 is a heartbeat interval of about 950 ms in the first half, but is shortened to about 630 ms in the second half. Although the alert target person detection unit 16 does not satisfy the condition that the heartbeat interval is 600 ms or less, the change in the heartbeat interval is (950−630) /950×100≈33.7%. Proceed to S408.

In step S <b> 408, the warning target person detection unit 16 detects “area N” (or group identifier) as a warning target person. That is, the warning target person detection unit 16 detects a person including the “region N” (or group identifier) as a warning target person, and records the behavior, face, and the like in the RAM 105 or the ROM 104, for example. Furthermore, the warning target person detection unit 16 can be used for easy monitoring by, for example, displaying the warning target person in different colors on the monitor screen of the display unit, which is the output unit 103, or generating a warning sound with a speaker or the like. Make it possible to confirm the target person. After the process of step S408, the warning target person detection unit 16 advances the process to step S409.
In step S409, the warning target person detection unit 16 updates the variable N (N = N + 1).
Next, the warning target person detection unit 16 determines whether or not the variable N is larger than the variable M (N> M) (step S410). The warning target person detection unit 16 ends the process when the variable N is larger than the variable M (step S410: YES). Moreover, the warning target person detection part 16 returns a process to step S403, when the variable N is the variable M or less (step S410: NO).

  In the example illustrated in FIG. 4, the same heartbeat detection unit 15 and the warning target person detection unit 16 perform “region 1” to “region” as illustrated in FIG. “5” is grouped, and the “G2” person in “Area 2” and “Area 5” and the person in “Area 4” are detected as persons to be monitored. In this way, the same heart rate detection unit 15 and the guard target person detection unit 16 identify the same heart rate from five regions as shown in FIG. 4, classify individuals, and monitor the state of each heart rate, It is possible to detect and identify two persons to be warned.

  As described above, the monitoring device 1 of the present embodiment includes the imaging unit 11, the heartbeat detection unit 10, and the warning target person detection unit 16 (warning target detection unit). The imaging unit 11 captures an image of a location to be monitored. The heartbeat detection unit 10 detects a person's heartbeat based on a change in skin color included in the image captured by the imaging unit 11. The warning target person detection unit 16 detects a warning target person based on the heartbeat detected by the heartbeat detection unit 10. Thereby, the monitoring apparatus 1 of this embodiment generally monitors the state of the heartbeat based on the grounds that the criminal's heart rate is likely to be high, so that the dangerous person is based on the heartbeat. The candidate person to be monitored can be detected efficiently. Therefore, the monitoring apparatus 1 of this embodiment can improve the search efficiency of a dangerous person.

  In the present embodiment, the heartbeat detection unit 10 includes a skin color detection unit 12, a color change detection unit 13, and the same heartbeat detection unit 15. The skin color detection unit 12 detects an area within a predetermined color information range in the image as a skin color area. The color change detection unit 13 detects a change in color information in the skin color region detected by the skin color detection unit 12 as a change in skin color. The same heartbeat detection unit 15 determines whether or not it is a skin color region in the same person based on the change in the color information detected by the color change detection unit 13, and the skin color region in the same person The heartbeat of the same person is detected based on the change in color information in the skin color region determined to be. Thereby, the monitoring apparatus 1 of this embodiment detects it as a skin color area | region by color information, and determines whether it is a skin color area | region in the same person based on the change of color information. And since the monitoring apparatus 1 of this embodiment detects the heartbeat of the same person, it can detect the person of the monitoring object who is a dangerous person candidate efficiently.

  The same heart beat detection unit 15 assigns a group identifier to the skin color region determined to be the same person. As a result, even if some of the plurality of regions determined to be the same person cannot be detected, if the heart beat of at least one region can be detected in the same person, A person can be detected. Therefore, the monitoring apparatus 1 of this embodiment can perform highly reliable monitoring that facilitates maintaining monitoring continuity.

  In the present embodiment, the heartbeat detection unit 10 detects a heartbeat based on the time interval of the peak of color information. The alert target person detection unit 16 is configured to detect the heartbeat when the heartbeat interval detected by the heartbeat detection unit 10 is equal to or less than a predetermined interval, or when the change in the detected heartbeat interval is equal to or greater than a predetermined range. A person corresponding to the heartbeat is detected as a person to be warned. Thereby, the monitoring device 1 of the present embodiment can detect the heartbeat by a simple method called the time interval of the peak of the color information, and the person to be alerted can be detected by the simple method using the heartbeat interval (heartbeat interval). Can be detected.

Further, in the present embodiment, the same heartbeat detection unit 15 determines whether or not it is a skin color region of the same person based on the timing of the peak of the color information.
Thereby, the monitoring apparatus 1 of this embodiment can determine whether it is the skin color area | region in the same person easily and reliably.
In the present embodiment, the monitoring device 1 describes that a person to be warned who may commit a crime is detected. However, since the timing of the heartbeat is observed, for example, the physical condition deteriorates. It is also possible to detect such a person, record the progress of heartbeat changes until the physical condition deteriorates when the physical condition changes suddenly. Therefore, there is a possibility that the monitoring device 1 can be applied to treatment, health management, and the like.

In addition, the monitoring method of the present embodiment includes an imaging step, a heartbeat detection step, and a guard target detection step. In the imaging step, the imaging unit 11 captures an image of the location to be monitored. In the heartbeat detection step, the heartbeat detection unit 10 detects the heartbeat of the person based on the change in skin color included in the image captured in the imaging step. In the warning target detection step, the warning target person detection unit 16 detects a warning target person based on the heartbeat detected in the heartbeat detection step.
Thereby, since the monitoring method of this embodiment can detect the person of the warning object who is a dangerous person candidate efficiently like the monitoring apparatus 1, it can improve the search efficiency of a dangerous person.

[Second Embodiment]
Next, the monitoring system 2 of the second embodiment will be described with reference to FIG. FIG. 10 is a functional block diagram illustrating an example of the monitoring system 2 of the present embodiment.
As shown in FIG. 10, the monitoring system 2 includes a plurality of heartbeat detection devices (100-1, 100-2, 100-3,...) And a warning management device 200. Note that each of the heartbeat detection devices (100-1, 100-2, 100-3,...) Has the same configuration, and indicates any heartbeat detection device provided in the monitoring system 2, or is not particularly distinguished. In this case, the heartbeat detection device 100 will be described. In addition, the plurality of heartbeat detection devices (100-1, 100-2, 100-3,...) And the alert management device 200 are connected via a network NW. The hardware configurations of the heartbeat detection device 100 and the alert management device 200 are the same as those in FIG.

The heartbeat detection device 100 includes an imaging unit 11 and a heartbeat detection unit 10a. The heartbeat detection unit 10a includes a skin color detection unit 12, a color change detection unit 13, and a heartbeat determination unit 14. The heartbeat detecting device 100 is configured by removing the same heartbeat detecting unit 15 from the monitoring device 1 shown in FIG. The heartbeat detection device 100 directs an image including a region detected as a heartbeat by the heartbeat determination unit 14, a color change timing, and a person detected as a warning target person via the communication unit 106 to the warning management device 200. Send.
The alert management device 200 processes the heartbeat detection information from the plurality of heartbeat detection devices 100 in a batch. The alert management device 200 includes the same heartbeat detection unit 15, an alert target person detection unit 16, and an alert target database unit 17.
10, the same components as those shown in FIG. 2 are denoted by the same reference numerals, and the description thereof is omitted.

The alert target database unit 17 registers, manages, and collates the person detected by the alert target person detection unit 16 in the database as the alert target person. The alarm target database unit 17 performs face detection (a known technique) from the video of the person detected by the alarm target person detection unit 16 when performing registration and collation in the database, and executes the detection based on the face data. In addition to the face, the alert target database unit 17 may extract feature quantities such as height, clothes color, skin color, and the like to improve the accuracy of matching.
The operation of the monitoring system 2 of the present embodiment is basically the same as that of the monitoring device 1 described above. The difference between the operation of the monitoring system 2 and the monitoring device 1 is that the output source of the skin color region is a plurality of heartbeat detection devices 100.

As described above, the monitoring system 2 of the present embodiment includes the heartbeat detection device 100 including the imaging unit 11, the heartbeat detection unit 10 a, and the alert management device 200 including the alert target person detection unit 16. . Thereby, since the monitoring system 2 of this embodiment can detect the person of the warning target who is a dangerous person candidate efficiently like the monitoring apparatus 1 mentioned above, it can improve the search efficiency of a dangerous person.
In the present embodiment, the monitoring system 2 includes a warning target database unit 17, and the warning target database unit 17 performs registration and verification based on face detection. Thereby, the monitoring system 2 of this embodiment can improve the identification precision of a warning subject person. In this embodiment, since the monitoring system 2 detects and registers a person to be alerted from detection of a tension state based on heartbeats, it is possible to construct a highly accurate database, and to identify the person to be alerted efficiently even in verification. it can. In addition, the monitoring system 2 can easily determine the person captured separately by each imaging unit 11 as the same person for the information acquired from the plurality of imaging units 11.

  The present invention is not limited to the above embodiments, and can be modified without departing from the spirit of the present invention. For example, the monitoring device 1 may include the alert target database unit 17. Moreover, you may comprise a part of each part of the monitoring apparatus 1 as a some apparatus connected via the network NW, for example.

  The monitoring device 1 and the monitoring system 2 described above have a computer system inside. The processing steps of the monitoring device 1 and the monitoring system 2 described above are stored in a computer-readable recording medium in the form of a program, and the above processing is performed by the computer reading and executing this program. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

  DESCRIPTION OF SYMBOLS 1 ... Monitoring apparatus, 2 ... Monitoring system, 10, 10a ... Heart rate detection part, 11 ... Imaging part, 12 ... Skin color detection part, 13 ... Color change detection part, 14 ... Heart rate determination part, 15 ... Same heart rate detection part, DESCRIPTION OF SYMBOLS 16 ... Alert object person detection part, 17 ... Alert object database part, 100, 100-1, 100-2, 100-3 ... Heart rate detection apparatus, 101 ... CPU, 102 ... Input part, 103 ... Output part, 104 ... ROM , 105 ... RAM, 106 ... communication unit, 200 ... warning management device, BS ... internal bus, NW ... network

Claims (5)

An imaging unit that captures an image of a location to be monitored;
A heartbeat detection unit that detects a heartbeat of a person based on a change in skin color included in the image captured by the imaging unit;
A monitoring target detection unit, comprising: a warning target detection unit that detects a warning target person based on the heart rate detected by the heart rate detection unit. The heartbeat detector
A skin color detection unit that detects a region within a predetermined color information range of the image as a skin color region;
A color change detection unit that detects a change in the color information in the skin color area detected by the skin color detection unit as a change in the skin color;
Based on the change in the color information detected by the color change detection unit, it is determined whether or not the skin color region is in the same person, and is the skin color region in the same person. The monitoring apparatus according to claim 1, further comprising: a same heart beat detecting unit that detects a heart beat of the same person based on a change in the color information in the determined skin color region. The heartbeat detection unit detects the heartbeat based on the time interval of the peak of the color information, and the alert target detection unit is when the interval of the heartbeat detected by the heartbeat detection unit is equal to or less than a predetermined interval The monitoring according to claim 2, wherein when the detected change in the interval of the heartbeats is equal to or greater than a predetermined range, a person corresponding to the heartbeat is detected as the person to be alerted. apparatus. An imaging unit that captures an image of a location to be monitored;
A heartbeat detection device comprising: a heartbeat detection unit that detects a heartbeat of a person based on a change in skin color included in the image captured by the imaging unit;
A monitoring system comprising: a warning management device that includes a warning target detection unit that detects a warning target person based on the heartbeat detected by the heartbeat detection unit. An imaging step in which an imaging unit captures an image of a location to be monitored;
A heartbeat detecting step for detecting a heartbeat of a person based on a change in a skin color included in the image captured by the imaging step;
A monitoring target detection unit comprising: a warning target detection step of detecting a person to be a warning target based on the heartbeat detected by the heartbeat detection step.

Images