HK1260746A1 - Monitoring system, image processing device, image processing method and program recording medium - Google Patents

Description

Monitoring system, image processing apparatus, image processing method, and program recording medium

Technical Field

The present invention relates to a monitoring system, an image processing apparatus configured to perform privacy processing of an image, an image processing method, and an image processing program recording medium.

Background

Monitoring systems have been developed to monitor congestion conditions at busy places such as train stations, crossroads and department stores, where camera devices are installed. The images taken by such camera devices of such surveillance systems may comprise many persons and therefore need to provide sufficient protection of the privacy of these persons.

In response to a demand for privacy protection of an image, privacy processing has been widely performed, for example, by reducing a spatial resolution in a person region (person region), which is a region in which a person appears in the image.

However, when the privacy processing is equally applied to all the person areas in the image, it may result in insufficient privacy protection for the near-side person due to insufficient blurring or a failure to find out a crowded situation from the image due to excessive blurring.

Examples of related techniques for applying privacy processing to an image include the techniques described in PTL 1 to PTL 3.

PTL 1 discloses a technique of privacy processing. The processing includes determining a privacy area of each user in advance, and in response to a user requesting distribution of an image, performing privacy processing on an area of the distribution image requested by the user at different image processing levels between the privacy area of the user and privacy areas of other persons.

PTL 2 discloses a technique including detecting a person from an image, acquiring position information about a person region, determining a region state indicating a state of the person region (crowd or individual person), and setting a mask image corresponding to the region state.

PTL 3 discloses a technique including: when the image processing apparatus projects a browsing-prohibited polyhedron on the image, prohibiting browsing of pixels determined to have depth information larger than that of the browsing-prohibited polyhedron; and allows browsing of pixels determined to have depth information smaller than that of the browsing-prohibited polygon. Thus, browsing of an object to be prevented is prevented while browsing of an object which is located on the near side with respect to the corresponding object and which does not require browsing inhibition is permitted.

As another technique related to the invention of the present application, PTL4 describes an example of a technique for detecting a crowd from an image. PTL 5 describes an example of privacy processing. PTLs 6 to 8 describe examples of techniques for sensing abnormal behavior of a person from an image.

[ list of references ]

[ patent document ]

PTL 1：WO No.2012/004907

PTL 2：JP 2015-222881 A

PTL 3：JP2012-23573 A

PTL 4：WO No.2014/207991

PTL 5：JP2012-503817A

PTL 6：WO No.2015/068854

PTL 7：WO No.2015/040929

PTL 8：WO No.2014/155922

Disclosure of Invention

[ problem ] to

When the privacy process is applied to the entire image, problems such as deterioration in visibility may occur.

It is therefore an object of the present invention to provide a monitoring system, an image processing apparatus, an image processing method, and a program recording medium, which achieve personal privacy protection while suppressing a decrease in image visibility.

[ means for solving the problems ]

A monitoring system according to an aspect of the present invention includes:

a camera device;

person region detection means for detecting a person region that is a region in which a person appears in an image captured by the camera device, and

privacy processing means for performing privacy processing on the person region, the intensity of which differs depending on the depth associated with the coordinates of the person region or a predetermined index relating to the depth.

An image processing apparatus according to an aspect of the present invention includes:

person region detection means for detecting a person region that is a region in which a person appears in an image captured by the camera device, and

privacy processing means for performing privacy processing on the person region, the intensity of which differs depending on the depth associated with the coordinates of the person region or a predetermined index relating to the depth.

An image processing method according to an aspect of the present invention includes:

detecting a person region, which is a region in which a person appears in an image captured by the camera device, and

privacy processing is performed on the person region, the intensity of which differs according to the depth associated with the coordinates of the person region or a predetermined index related to the depth.

A program storage medium according to an aspect of the present invention stores a program that causes a computer to execute:

a process of detecting a person's region, which is a region where a person appears in an image captured by a camera device, and

the process of performing privacy processing on the person region, the intensity of which differs according to the depth associated with the coordinates of the person region or a predetermined index related to the depth.

[ advantageous effects of the invention ]

According to the present invention, personal privacy protection is achieved while suppressing a decrease in image visibility.

Drawings

Fig. 1 is a configuration diagram showing an example of a monitoring system according to a first exemplary embodiment.

Fig. 2 is an explanatory diagram showing an example of a crowd patch (crowd patch).

Fig. 3 is an explanatory view showing an example of the detected original image and the detection result of the human figure region.

Fig. 4 is an explanatory diagram showing an example of image three-dimensional information.

Fig. 5 is an explanatory view showing another example of the detected original image and the detection result of the human figure region.

Fig. 6 is a flowchart showing an example of the operation of the monitoring system according to the first exemplary embodiment.

Fig. 7 is a configuration diagram showing an example of a monitoring system according to a second exemplary embodiment.

Fig. 8 is a configuration diagram showing an example of a monitoring system according to a third exemplary embodiment.

Fig. 9 is a schematic block diagram showing a configuration example of a computer according to an exemplary embodiment of the present invention.

Fig. 10 is a block diagram showing an outline of the present invention.

Fig. 11 is a block diagram showing another example of a monitoring system according to the present invention.

Detailed Description

First exemplary embodiment

Referring now to the drawings, exemplary embodiments of the present invention will be described. Fig. 1 is a configuration diagram showing an example of a monitoring system of the first exemplary embodiment. The monitoring system shown in fig. 1 includes a camera apparatus 10 and an image processing apparatus 20. The image processing apparatus 20 includes an image input unit 21, a person region detection unit 22, a depth information acquisition unit 23, a privacy processing unit 24, and an image output unit 25.

The camera device 10 is a camera device configured to capture an image of an area to be monitored. In the monitoring system of the present exemplary embodiment, the camera device 10 has an imaging range including an area in which an unspecified number of persons can be imaged, such as a train station, an intersection, and a department store.

The image input unit 21 receives an image from the camera apparatus 10. The image input unit 21 may receive an image together with camera parameters at the time of capturing the image, and also receive an output result, if any, from a human sensor directed to a range including at least a part of an imaging range of the image while capturing the image.

The human figure region detecting unit 22 detects a region including a human figure (particularly, a head) in the received image. The human region detection unit 22 may perform predetermined human group detection processing on the received image, and determine a detection region including a human group as a human region. As used in the present invention, the term "crowd (crowd)" includes a case where only one character is included.

Any method of detecting a person's region other than a method of separately recognizing the face of a person from an image and detecting a region may be employed. For example, a method widely known as a face recognition process is excluded. The face recognition processing detects a region including a face by detecting feature quantities indicating features of eyes or a nose, which are constituents of the face, from an image. Examples of the method of detecting the human figure region include the following methods: whether any person is present at a position in the real space corresponding to each pixel of the infrared image is determined using the infrared camera device interlocked with the camera device 10 as a human sensor and by using the infrared image of the same image range photographed by the infrared camera device at the same time and the same imaging range as the camera device 10. The method of detecting the human figure region may include the following methods: as the human sensor, an apparatus such as a measuring device provided with a light receiving element configured to emit light for inspection (inspection light) to sense whether the light is reflected or attenuated and detect whether a moving body (person) is present in a radiation range of the inspection light is used. In this case, for example, a plurality of measuring devices may be arranged in the imaging range at predetermined intervals, and whether any person is included in the imaging range may be detected based on the position information of the measuring devices.

The method of detecting a human figure region may include, for example, the method disclosed in PTL 4.

The method disclosed in PTL4 includes preparing a partial image of a crowd state generated by a method such as simulation (hereinafter referred to as a crowd patch), performing pattern matching of a randomly selected rectangular region in the image with respect to the crowd patch, and detecting a crowd region including a crowd in the image. The human figure region detection unit 22 can realize detection of a crowd region from an image by using a discriminator obtained by machine learning using an information set indicating a crowd patch and a crowd state (state of a human figure) of the crowd patch as training data.

As used herein, the term "local" is intended to include an area that is smaller than the area of the image to be detected. The partial image of the crowd state corresponds to an image including a set of reference parts (hereinafter, simply referred to as "reference parts") of persons constituting the crowd in such a region. Hereinafter, the partial image of the crowd state may be referred to as a crowd patch. The crowd plot may include portions of the person other than the reference location. Partial images including a set of various reference sites viewed from various angles may be prepared as a crowd patch. In particular, a partial image including reference portions of at least two persons may be prepared.

Although the local image includes reference portions of a plurality of persons in the method disclosed in PTL4, the crowd map block in the present exemplary embodiment may include a local image including a reference portion of one person. This enables detection of an area including only one person as a crowd area. Although the reference part is not particularly limited in the method disclosed in PTL4, the reference part may be extended to a range larger than a face, for example, a range including a head or an upper body, so that a feature such as an outline of a single person is also provided for detecting a crowd including only one person. Note that two or more types of reference sites may be specified.

The human figure region detection unit 22 may set a specific region (rectangular region group) so as to cover the entire region of the image to be detected. In the pattern matching the rectangular region, the human figure region detection unit 22 may change the size of the crowd patch according to the coordinates on the image based on the size of the reference portion corresponding to the crowd patch and camera parameters indicating the position, the posture focal length, the lens distortion, and the like of the camera apparatus 10. For example, the size of a reference part of a person included in an image may be derived from these camera parameters. The size of the crowd map block may be expanded or reduced according to the size of the reference location in the image. The size of the crowd graph block may be adjusted to the size of the rectangular area set by the coordinates. Other examples may include providing a larger crowd tile in a lower region of the image and providing a smaller crowd tile in an upper region of the image. The lower region of the image includes a region closer to the camera in real space and the upper region of the image includes a region farther from the camera. The setting of the rectangular area group is not limited to these methods and can be flexibly set. The rectangular area groups may be arranged in an overlapping manner.

By storing information indicating the crowd state of the partial image corresponding one-to-one to the crowd patch as described in PTL4, the human figure region detecting unit 22 can obtain the crowd state (the number of people, the orientation, and the like) from the region detected in the image by using the crowd patch.

For example, the human figure region detecting unit 22 may detect, as a result of matching, all rectangular regions in which determined human figures appear, collectively referred to as a crowd region. For example, the human figure region detection unit 22 may set a region including a predetermined amount of region added to the periphery of the rectangular region or a part of the rectangular region (e.g., the upper half most likely to include a face) as the crowd region.

The human figure region detection unit 22 may determine a human figure region detected as the human figure region in the above-described manner, and output representative coordinates and a size of the human figure region. The representative coordinates of the human figure region may be, for example, center coordinates of a specific region determined as a human figure region. The character region may have two or more representative coordinates. For example, when the crowd tile is provided with information indicating the head position in the corresponding crowd tile as the information indicating the crowd state, the coordinates of the head position in the specific area may be determined as the representative coordinates.

Fig. 2 is an explanatory diagram showing an example of a crowd map. Fig. 3 is an explanatory diagram showing an example of the detected original image and the detection result of the human figure region. Image (a) in fig. 3 is an explanatory view showing an example of an original detection image, and image (b) in fig. 3 is an explanatory view showing a detection result of a human figure region. In image (b) of fig. 3, a hatched rectangular region represents the detected human figure region. The image shown in fig. 3 is a binary image acquired by the camera apparatus 10 for emphasizing the outline of a person. However, any number of gradations is applicable to the image. The same applies to other image examples.

The human region detecting unit 22 may analyze each specific region (for example, each rectangular region in the figure) of the input image and determine whether at least one human is present in each specific region via a pattern matched with a crowd patch. In other words, the human figure region detecting unit 22 does not identify the respective faces of the human figures included in the image, but identifies a combination of reference parts of at least one human figure included in a specific region in the image as a batch through comparison and collation with partial images prepared in advance. Therefore, simple processing is realized and the processing load can be reduced.

The depth information acquisition unit 23 acquires the coordinates corresponding one-to-one to the pixels included in the image or the depth information (e.g., representative coordinates) at the coordinates of the human figure region in the image.

The depth information relates to the depth of a location in real space corresponding to the distance of the coordinates relative to the camera. The depth information may represent a depth of a position in the real space corresponding to the coordinates, or may represent a predetermined index related to the depth. Examples of the latter depth information include, for example, information related to depth or information that may lead to an estimate of the depth size. Examples of the depth information as described above include, for example: information indicating the size of the human figure region having the coordinates determined as the representative coordinates or the size of the reference part of the human figure included in the human figure region.

When information indicating the size of the human area and the size of the reference part is used as the depth information, the depth information may be acquired by the human area detecting unit 22. In this case, the depth information acquiring unit 23 may be omitted.

The depth information acquisition unit 23 may acquire depth information such as target coordinates from camera parameters of the camera device 10.

For example, the depth information acquisition unit 23 may estimate image three-dimensional information indicating the three-dimensional size of the reference object in the image acquired by the camera apparatus 10 shown in fig. 4 from camera parameters related to the position and posture of the camera such as the height, depression angle, horizontal view angle, diagonal view line of the camera apparatus 10.

In this case, the reference object may be, for example, a person having a predetermined height. The image three-dimensional information may be a pair of coordinates corresponding to the position (standing point) of a reference object (e.g., a person 168cm high) and the height (top) in the image. The image three-dimensional information may be any information indicating a position and a size (scale) at which a specific object having a known size and position appears in an image.

Fig. 4 is an explanatory diagram showing an example of three-dimensional information of an ab image. Image (a) in fig. 4 is an explanatory diagram showing an example of an image acquired by the camera apparatus 10. Image (b) in fig. 4 is an explanatory diagram showing an example of image three-dimensional information in an image. Image (b) in fig. 4 shows three pairs of coordinate sets { (X1, Y1), (X2, Y2) } corresponding to the stand point and the top of the reference object in the image.

For example, the depth information acquisition unit 23 may estimate image three-dimensional information as shown in image (b) in fig. 4 based on the camera parameters of the camera device 10. The depth information acquiring unit 23 may acquire an image by the user specifying three-dimensional information of image three-dimensional information as shown in image (b) in fig. 4 on the image acquired by the camera apparatus 10. Based on the image three-dimensional information acquired in this manner, the depth at the target coordinates of the image can be estimated. When the head of the person appears at the target pixel, the depth information obtaining unit 23 can obtain the depth by obtaining the position of the person in the real space.

The privacy processing unit 24 performs privacy processing on the person region in the image. The privacy processing unit 24 performs privacy processing whose intensity differs according to the depth indicated by the depth information associated with the coordinates of the human figure region or an index related to the depth. The privacy processing unit 24 may determine, for example, a position in the real space corresponding to the representative coordinates of the human figure region as a position in the real space of the human figure in the human figure region, and perform privacy processing whose intensity (high or low) differs depending on the depth at the position or an index related to the depth. In the following description, the depth information indicates the case of depth. However, when the depth information indicates an index related to depth, the description may be read by replacing the size relationship of the depth with the size relationship of the index.

Note that the greater the depth indicated by the depth information corresponding to the representative coordinates of the human figure region, the weaker the privacy processing performed by the privacy processing unit 24 on the human figure region, and the smaller the depth, the stronger the privacy processing for the human figure region. The privacy processing unit 24 may also be configured to execute the first privacy processing when the depth is equal to or greater than a predetermined threshold value, and execute the second privacy processing when the depth is less than the predetermined threshold value. The second privacy process is stronger than the first privacy process.

As used herein, the term "intensity of privacy processing" is intended to include the degree of ambiguity. In other words, the intensity of the privacy process corresponds to the resolution level. More specifically, the term "privacy processing strong" means that the resolution of an image after being processed is lower than that of an image subjected to weak privacy processing, and the term "privacy processing weak" means that the resolution of an image after being processed is higher than that of an image subjected to strong privacy processing. For example, the procedure described in PTL 5 can be used as a method of privacy processing. In the method of PTL 5, in the weak privacy processing, the spatial resolution may be reduced for a pixel range narrower than the strong privacy processing, and in the strong privacy processing, the spatial resolution may be reduced for a pixel range wider than the weak privacy processing.

Fig. 5 is an explanatory diagram showing another example of the detected original image and the detection result of the human figure region. Image (a) in fig. 5 shows an image of the platform of the station taken by the camera device 10. The image (a) shown in fig. 5 mainly includes four types of human figure regions surrounded by an ellipse of a dotted line. In image (a) in fig. 5, P1 is an example of a person region including a person standing independently on the platform at a position closest to the camera. P2 is an example of a character region of a character standing alone in the middle portion of the platform. P3 is an example of an area of a person standing independently on the far side of the platform. P4 is an example of a person region (crowd region) for multiple persons distal to the platform. Image (b) in fig. 5 shows the detection result of the human figure region in the image shown in image (a) in fig. 5. In the image (b) in fig. 5, a hatched rectangular region represents the detected human figure region.

In the example shown in the image (b) in fig. 5, the size of each person region is input as the depth information. In this case, for example, the privacy processing unit 24 may perform strong privacy processing on a person region that is relatively large (having at least a predetermined size) as seen in a lower part of the image, and perform weak privacy processing on a person region that is relatively small (smaller than the predetermined size) as seen in an upper part of the image. In this case, the privacy processing unit 24 may perform the privacy processing by using a parameter related to the strength of the privacy processing, such as a conversion window size specified according to the size of the person region used for the strength of the privacy processing.

The method of privacy processing is not particularly limited. The privacy processing unit 24 may perform privacy processing on the person region in the image by changing the intensity based on the depth information of the corresponding region using a known method. When the depth is equal to or greater than a predetermined value, a configuration in which privacy processing is not performed may also be applied. The predetermined value may be set to any depth value that provides a sufficient distance to protect privacy without performing privacy processing. It is also contemplated to set a plurality of threshold values to perform a multi-stage determination such as a three-stage determination. As an example of conceivable determination, the first privacy process is performed when the depth is less than the first threshold value; performing a second privacy process when the first threshold is not greater than the depth and the depth is not greater than a second threshold; when the second threshold is less than the depth, the privacy process is not performed. In this case, the first privacy process is a strong privacy process and the second privacy process is a weak privacy process.

The image output unit 25 outputs the image subjected to the privacy processing by the privacy processing unit 24. Here, examples of the image output unit 25 include a liquid crystal display and a small-sized terminal (smartphone, tablet computer).

The operation of the present exemplary embodiment will be described below. Fig. 6 is a flowchart showing an example of the operation of the monitoring system according to the present exemplary embodiment. In the example shown in fig. 6, the image input unit 21 receives an image captured by the camera apparatus 10 as an image to be subjected to privacy processing (step S101). The image input unit 21 may import camera parameters together with the image.

Next, the human figure region detecting unit 22 detects a human figure region from the image by using a predetermined human figure detection process (step S102). It is also applicable to detect the human figure region based on information obtained from the human sensor. Specifically, the region including the human figure is specified by a human sensor that covers an imaging region of an image captured by the camera 10 in the sensing range, and a region in the image corresponding to the specified region is detected as a human area. The human figure region detection unit 22 may also be configured to detect a human figure region by using a human sensor and a human population detection process.

Specifically, the human figure region detecting unit 22 specifies a region including a human figure by a human sensor, and then specifies a first region in the image corresponding to the specified region. Subsequently, the human figure region detecting unit 22 may be further configured to detect a human figure region from the first region by applying a predetermined human figure detection process to the first region.

Next, the depth information acquiring unit 23 acquires depth information of the human figure region (step S103).

Next, the privacy processing unit 24 performs privacy processing on the person region in the image by changing the intensity based on the depth information of the corresponding region (step S104).

Finally, the image output unit 25 outputs the privacy-processed image (step S105).

In the above example, the process in step S103 is executed after the process in step S102. However, the execution timing of step S103 is not limited thereto. For example, the monitoring system may execute the processing in step S103 in parallel with the processing in step S102, or may execute at a predetermined timing such as fixing the position and orientation of the camera each time.

As described above, according to the present exemplary embodiment, a person region is detected from an image captured by the camera apparatus 10, and then privacy processing of different intensities is performed on the detected person region according to the depth or an index related to the depth. Therefore, personal privacy protection is achieved while suppressing a decrease in visibility of an image. Since the range of performing the privacy processing is limited, the processing load can be reduced. In addition, by adopting a method of detecting a human figure region or a method of detecting a human figure region using a human sensor based on a batch of feature amounts of a combination of reference parts including at least one human figure included in an image, it is possible to achieve both suppression of image visibility reduction and privacy protection and further reduction of a processing load.

When the method of detecting a person region by using a human sensor is employed, a region that does not generally require privacy protection, such as a person in a poster, can be advantageously excluded from the object of privacy processing.

Although one camera device 10 is shown in fig. 1, the number of camera devices provided in the monitoring system is not limited to one. For example, a surveillance system may be connected to two or more camera devices 10.

Although fig. 1 shows an example in which the image processing apparatus 20 includes the image output unit 25, a configuration in which the image processing apparatus 20 does not include the image output unit 25 is also applicable. In this case, the image processing device 20 may be configured to output the image subjected to the privacy processing to a predetermined server device connected via a network, for example.

Second example embodiment

A second exemplary embodiment of the present invention will be described below. Fig. 7 is a configuration diagram showing an example of the monitoring system of the present exemplary embodiment. The monitoring system shown in fig. 7 is different from the monitoring system in the first exemplary embodiment shown in fig. 1 in that the image processing apparatus 20 further includes an abnormal behavior sensing unit 26.

The abnormal behavior sensing unit 26 senses an abnormal behavior of a person from an input image. When the information indicating the state of the human figure is, for example, as a result of the analysis by the human figure region detecting unit 22, the abnormal behavior sensing unit 26 may detect the abnormal behavior by determining whether a predetermined collective movement of the human figure has occurred between the images in the time series.

The human figure region detecting unit 22 of the present exemplary embodiment outputs information on the number or orientation of human figures in the human figure region as information indicating the state of the human figures and the representative coordinates and size of the human figure region.

In this case, the abnormal behavior sensing unit 26 may sense the flow of a group of persons, such as aggregation or separation from temporally continuous images, based on information of individual person regions detected from the images. Then, the abnormal behavior sensing unit 26 may sense the abnormal behavior when the degree of aggregation in the collective flow of such persons changes more than a predetermined amount between images within a predetermined period or chronologically in a specific frame.

In addition, when avoidance behavior, the same group as a previously sensed group, or crawling behavior is detected, it may be considered that abnormal behavior is sensed. For example, the method described in PTL 6 can be used as a method of sensing avoidance behavior. The method described in PTL 7 can be used as a method of sensing the same group as a previously detected group. For example, the method described in PTL 8 may be used as a method of sensing crawling behavior.

Note that the abnormal behavior sensing unit 26 may sense the abnormal behavior from the image using only self-processing without using the information indicating the state of the person obtained from the person region detecting unit 22. In this case, the abnormal behavior sensing unit 26 may prepare a collective image for sensing the abnormal behavior.

In the present exemplary embodiment, the privacy processing unit 24 may be configured to perform privacy processing on an image, for example, when the abnormal behavior sensing unit 26 senses abnormal behavior.

When abnormal behavior is sensed, disclosure different from normal time may be implemented so that only recorded images are typically displayed on the monitor. Within the scope of normal disclosure, no privacy processing is required. However, when disclosure different from normal disclosure is implemented, privacy protection may be desired. According to the present exemplary embodiment, the privacy processing is executed only when abnormal behavior is sensed, and therefore an appropriate (automatic and immediate) reaction can be given when abnormal behavior is sensed.

In contrast, the privacy processing unit 24 may be configured not to perform privacy processing on the image, for example, when the abnormal behavior sensing unit 26 senses the abnormal behavior.

When abnormal behavior is sensed, it may be necessary to specify a person who causes the abnormal behavior.

The present exemplary embodiment may be configured to perform the privacy processing under a normal condition in which no abnormal behavior is sensed, and not perform the privacy processing only when abnormal behavior is sensed. Therefore, when an abnormal behavior is sensed, a sufficient (automatic, and faster than the case of performing privacy processing) action can be given.

Third exemplary embodiment

A third exemplary embodiment of the present invention will be described below. Fig. 8 is a configuration diagram showing an example of the monitoring system of the present exemplary embodiment. The monitoring system shown in fig. 8 is different from the monitoring system in the first exemplary embodiment shown in fig. 1 in that the image processing apparatus 20 further includes a face region detection unit 27.

The face-area detecting unit 27 performs face recognition on a predetermined partial area in the input image, and detects a face area where a face exists. Here, the region for which the face region detection unit 27 performs face recognition may be a region that is not recognized as a human region by the human region detection unit 22, or may be a region that satisfies a predetermined condition such that the depth of a corresponding position in real space in the region in the image does not exceed a predetermined value.

The privacy processing unit 24 of the present exemplary embodiment performs privacy processing based on the depth information not only for the person region detected by the person region detecting unit 22 but also for the face region detected by the face region detecting unit 27.

For example, the monitoring system may cause the human region detection unit 22 to first analyze the entire portion of the image acquired by the camera apparatus 10, and then cause the face region detection unit 27 to analyze a region that is not detected as a human region, as an analysis result.

For example, the monitoring system may divide the regions in the image acquired by the camera apparatus 10 into two groups by the size of the depth at the position in the corresponding real space, cause the human region detecting unit 22 to analyze the divided region having a greater depth, and cause the face region detecting unit 27 to analyze the divided region having a smaller depth. When the camera parameters are fixed, the user may divide the regions of the image into two or more groups and specify an analysis method for each divided region. It is noted that specifying an analysis method may include specifying not to perform any analysis. The user may designate the human figure region detecting unit 22 to analyze a region including a face in an image smaller than a predetermined value, and designate the face region detecting unit 27 to analyze a region including a face larger than a predetermined value. The user may also specify that no analysis is performed on areas that do not include the face of a person, such as areas that only include a ceiling or walls. The human figure region detecting unit 22 and the face region detecting unit 27 may perform analysis only on the region specified by the user.

As described above, according to the present exemplary embodiment, the privacy process can be efficiently performed by combining the face recognition process.

The configurations of the above-described exemplary embodiments may be combined. For example, in the second exemplary embodiment, it may be specified whether or not the privacy processing is always executed or executed when an abnormal behavior is sensed. For example, in the third exemplary embodiment, a face recognition process may be combined as a method of detecting a human figure region. The mode of combination is not limited to the above-described mode.

An example of the configuration of a computer according to an exemplary embodiment of the present invention will be described below. Fig. 9 is a block diagram showing a general configuration example of a computer according to an exemplary embodiment of the present invention. The computer 1000 includes a Central Processing Unit (CPU)1001, a main storage device 1002, an auxiliary storage device 1003, an interface 1004, and a display device 1005.

The respective processing components (the image input unit 21, the person region detection unit 22, the depth information acquisition unit 23, the privacy processing unit 24, the image output unit 25, the abnormal behavior sensing unit 26, and the face region detection unit 27) in the monitoring system described above may be realized, for example, by a computer 1000 configured to operate as the image processing apparatus 20. In this case, the operations of the respective processing sections may be stored in the form of a program in the secondary storage device 1003. The CPU 1001 reads out a program from the secondary storage device 1003, deploys the program in the primary storage device 1002, and executes predetermined processing in various exemplary embodiments according to the deployed program.

The secondary storage device 1003 is an example of a non-transitory but tangible medium. Examples of other non-transitory but tangible media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, and semiconductor memories, connected via the interface 1004. When a program is distributed to the computer 1000 via a communication line, the computer 1000 may deploy the program in the main storage device 1002 upon receiving the program, and execute predetermined processing in various exemplary embodiments.

The program may be configured to implement a part of predetermined processing in various exemplary embodiments. Further, the program may be a differential program configured to realize predetermined processing in various exemplary embodiments via combination with other programs already stored in the secondary storage device 1003.

According to the processing contents in the exemplary embodiment, part of the components of the computer 1000 may be omitted. For example, when an image that has been subjected to privacy processing is output to a separate server connected via a network, for example, the display device 1005 may be omitted. Although illustration is omitted in fig. 9, the computer 1000 may be provided with an input device according to the processing content in the example embodiment. For example, when the monitoring system receives a command input related to an analysis method of a specific area in an image acquired by the camera device 10 from a user, an input device for inputting the command may be provided.

Some or all of the components in each device are implemented by a general or specific circuit, processor, or combination. Each of these components may be composed of a single chip, or may include a plurality of chips connected via a bus. Part or all of each component in the respective devices may be realized by a combination of the above-described circuits and programs.

When part or all of the components in each device are implemented by a plurality of information processing devices or circuits, the plurality of information processing devices or circuits may be collectively or discretely located. For example, each information processing device or circuit may be implemented in the form of a connection with a cloud computing system via a communication network such as a client and server system.

Next, an overview of the monitoring system and the image processing apparatus according to the present exemplary embodiment will be described. Fig. 10 is a block diagram showing an outline of the monitoring system according to the present exemplary embodiment. As shown in fig. 10, the monitoring system according to the present exemplary embodiment includes a camera apparatus 600, a person region detection unit 701, and a privacy processing unit 702.

The human figure region detection unit 701 detects a human figure region, which is a region in which a human figure appears in an image captured by the camera apparatus 600.

The privacy processing unit 702 is configured to perform privacy processing on the person region, the intensity of the privacy processing being different according to the depth indicated by the depth information associated with the coordinates of the person region or a predetermined index related to the depth. As used herein, the term "depth" is intended to include the distance in the image relative to the camera device of the location in the corresponding real space of at least the coordinates determined to be a human object region.

In this configuration, personal belonging privacy is protected while suppressing a reduction in visibility of an image.

As shown in fig. 11, the monitoring system may also be provided with a depth information acquisition unit 703.

The depth information acquisition unit 703 may acquire depth information indicating a depth or a predetermined index related to a depth. The depth information acquisition unit 703 may acquire depth information indicating a depth or a predetermined index related to a depth corresponding to at least coordinates of a human figure region in an image.

In this configuration, the privacy processing unit 702 can perform privacy processing whose intensity differs according to the depth information.

The person region detecting unit 701 may determine whether or not a person is present for each specific region set in the image, detect the specific region in which the person is determined to be present as a person region, and the privacy processing unit 702 may perform privacy processing for each detected person region.

The human figure region detection unit 701 may determine whether a human figure is present in the specific region based on the local image including the reference part of the human figure.

For example, the local image may be an image indicating a set of reference locations that extend over a larger range than the face.

The human figure region detection unit 701 may also determine whether a human figure is present in the specific region based on the local image expressing the reference parts of two or more human figures constituting the human figure.

The person region detecting unit 701 may determine whether or not a person is present in the specific region by using a discriminator obtained by machine learning using a combination of the partial image and information indicating the state of the person in the partial image as training data.

The size of the human figure region or the size of the above-described reference part included in the human figure region may be used as a predetermined index relating to the depth.

The human figure region detection unit 701 may set specific regions having different sizes in the image.

The human figure region detection unit 701 may also detect a human figure region from the image based on information obtained by the human sensor, including a region of the image acquired by the camera apparatus within the sensing range.

The monitoring system may further include an abnormal behavior sensing unit (for example, the above-described abnormal behavior sensing unit 26) configured to analyze the image acquired by the camera apparatus 10 and sense an abnormal behavior of a person from the image, and the privacy processing unit 702 may display the image acquired by the camera apparatus without performing privacy processing when the abnormal behavior is sensed.

Although the present application has been described so far with reference to the present exemplary embodiments and examples, the present application is not limited to the above-described exemplary embodiments and examples. In other words, the configuration and details of the present invention can be modified in various ways understood by those skilled in the art without departing from the scope of the present invention.

Some or all of the above example embodiments may be described as additional statements given below, but are not limited to the following.

(supplementary notes 1)

A surveillance system, comprising:

a camera device;

person region detection means for detecting a person region that is a region in which a person appears in an image captured by the camera device, and

privacy processing means for performing privacy processing on the person region, the intensity of which differs depending on the depth associated with the coordinates of the person region or a predetermined index relating to the depth.

(supplementary notes 2)

The monitoring system according to supplementary note 1, comprising;

depth information acquiring means for acquiring depth information indicating a depth or a predetermined index relating to the depth, wherein,

the privacy processing means performs privacy processing whose intensity differs depending on the depth information.

(supplementary notes 3)

The monitoring system according to supplementary note 1 or supplementary note 2, wherein the person region detecting means determines whether or not a person is present for each specific region set in the image and detects the specific region in which it is determined that a person is present as the person region, and

the privacy processing means performs privacy processing for each of the detected person regions.

(supplementary notes 4)

The monitoring system according to supplementary note 3, wherein the human figure region detecting means determines whether or not a human figure is present in each specific region based on the partial image including the reference part of the human figure.

(supplementary notes 5)

The monitoring system according to supplementary note 4, wherein the human figure region detecting means determines whether or not a human figure is present in each specific region based on the partial images of the plurality of reference parts representing two or more human figures constituting the human group.

(supplementary notes 6)

The monitoring system according to supplementary note 4 or supplementary note 5, wherein the human figure region detecting means determines whether or not a human figure is present in each specific region by using a discriminator obtained by machine learning using training data including a combination of a partial image and information indicating a state of a human figure in the partial image.

(supplementary notes 7)

The monitoring system according to supplementary note 4 or supplementary note 5, wherein the size of the human figure region or the size of the reference part included in the human figure region is used as a predetermined index relating to the depth.

(supplementary notes 8)

The monitoring system according to any one of supplementary notes 3 to 7, wherein the human figure region detection means sets specific regions having different sizes in the image.

(supplementary notes 9)

The monitoring system according to any one of supplementary notes 1 to 8, wherein the person region detecting means detects the person region from the image based on information obtained by the human sensor, and the human sensor has a sensing range covering an imaging region of the image acquired by the camera device.

(supplementary notes 10)

The monitoring system according to any one of supplementary notes 1 to 9, comprising:

abnormal behavior sensing means for sensing an abnormal behavior of a person from an image by analyzing the image,

wherein the privacy processing means does not perform the privacy processing when the abnormal behavior is sensed.

(supplementary notes 11)

An image processing apparatus comprising:

person region detection means for detecting a person region that is a region in which a person appears in an image captured by the camera device, and

privacy processing means for performing privacy processing on the person region, the intensity of which differs depending on the depth associated with the coordinates of the person region or a predetermined index relating to the depth.

(supplementary notes 12)

An image processing method comprising:

detecting a person region, which is a region in which a person appears in an image captured by the camera device, and

privacy processing is performed on the person region, the intensity of which differs according to the depth associated with the coordinates of the person region or a predetermined index related to the depth.

(supplementary notes 13)

A program storage medium storing a program for causing a computer to execute:

a process of detecting a person's region, which is a region where a person appears in an image captured by a camera device, and

the process of performing privacy processing on the person region, the intensity of which differs according to the depth associated with the coordinates of the person region or a predetermined index related to the depth.

This application claims priority from Japanese patent application No.2016-058401, filed 2016, month 3, and day 23, the entire contents of which are incorporated herein by reference.

[ Industrial Applicability ]

The present invention can be preferably applied to an application that protects privacy of a personal object in any image, and image quality is kept high so that a viewer can grasp a situation in the image.

[ list of reference numerals ]

10 Camera device

20 image processing apparatus

21 image input unit

22 human figure region detection unit

23 depth information acquisition unit

24 privacy processing unit

25 image output unit

26 abnormal behavior sensing unit

27 face region detection unit

1000 computer

1001 CPU

1002 main storage device

1003 auxiliary storage device

1004 interface

1005 display device

600 camera device

700 image processing apparatus

701 human figure region detection unit

702 privacy processing unit

703 depth information acquiring unit

Claims (13)

1. A surveillance system, comprising:

a camera device;

person region detection means for detecting a person region that is a region where a person appears in an image captured by the camera device, and

privacy processing means for performing privacy processing on the personal area, the intensity of the privacy processing being different according to a depth associated with the coordinates of the personal area or a predetermined index relating to the depth.

2. The monitoring system of claim 1, comprising:

depth information acquiring means for acquiring depth information indicating the depth or a predetermined index related to the depth, wherein

The privacy processing means performs privacy processing whose intensity differs depending on the depth information.

3. The surveillance system according to claim 1 or claim 2, wherein the human figure region detecting means determines whether or not a human figure is present for each specific region set in the image, and detects a specific region in which it is determined that a human figure is present as a human figure region, and

the privacy processing means performs privacy processing for each of the detected person regions.

4. The monitoring system according to claim 3, wherein the human figure region detecting means determines whether or not a human figure is present in each specific region based on a local image including a reference part of the human figure.

5. The monitoring system according to claim 4, wherein the human figure region detecting means determines whether or not a human figure is present in each specific region based on the partial images representing a plurality of reference parts of two or more human figures constituting a human group.

6. The surveillance system according to claim 4 or claim 5, wherein the human figure region detecting means determines whether or not a human figure is present in each specific region by using a discriminator obtained by machine learning using training data including a combination of the partial image and information indicating a human figure state in the partial image.

7. The monitoring system according to claim 4 or claim 5, wherein a size of the human figure region or a size of the reference part included in the human figure region is used as a predetermined index relating to the depth.

8. The monitoring system according to any one of claims 3 to 7, wherein the human figure region detecting means sets specific regions having different sizes in the image.

9. The monitoring system according to any one of claims 1 to 8, wherein the human figure region detecting means detects a human figure region from the image based on information obtained by a human sensor, and the human sensor has a sensing range covering an imaging region of the image acquired by the camera device.

10. The monitoring system of any one of claims 1 to 9, comprising:

abnormal behavior sensing means for sensing an abnormal behavior of a person from the image by analyzing the image,

wherein the privacy processing means does not perform privacy processing when the abnormal behavior is sensed.

11. An image processing apparatus comprising:

person region detection means for detecting a person region that is a region where a person appears in an image captured by the camera device, and

privacy processing means for performing privacy processing on the personal area, the intensity of the privacy processing being different according to a depth associated with the coordinates of the personal area or a predetermined index relating to the depth.

12. An image processing method comprising:

detecting a person region, which is a region in which a person appears in an image captured by a camera device, and

performing privacy processing on the human figure region, the intensity of the privacy processing being different according to a depth associated with the coordinates of the human figure region or a predetermined index related to the depth.

13. A program storage medium storing a program for causing a computer to execute:

a process of detecting a human figure region, which is a region in which a human figure appears in an image captured by a camera device, and

a process of performing a privacy process on the human figure region, the intensity of the privacy process being different according to a depth associated with the coordinates of the human figure region or a predetermined index related to the depth.