JP2019062267A - Imaging device, control method for imaging device and program - Google Patents

Abstract

Kind Code: A1 To provide an imaging apparatus capable of appropriately compressing an image while reducing delay in image distribution. Kind Code: A1 An image capturing device includes image capturing means for capturing an image of a subject to obtain a captured image, detection means for detecting a moving object in the captured image captured by the image capturing means, and compression of the captured image obtained by the image capturing means. Compressing means for generating a compressed image by means of compression; Distribution means for distributing the compressed image generated by the compression means to an external device via a network; Compressing by controlling the compression means so as to compress the moving object area, which is the area that has been captured, in a first mode, and compress the background area other than the moving object area in the captured image in a second mode different from the first mode. and control means for generating an image. [Selection drawing] Fig. 3

Description

  The present invention relates to an imaging device, a control method of an imaging device, and a program.

  A network camera that delivers a captured moving image via a network can compress and record the moving image and can distribute it to an external device. Such video compression methods include H. H.264 / AVC (Advanced Video Coding) and H.264. There are standards such as H.265 / HEVC (High Efficiency Video Coding). According to these standards, it is possible to specify a code amount (quantization parameter: Q value) for a specific area in an image.

A network surveillance camera that applies such a network camera to surveillance has limited storage capacity of the storage device mounted, but has to continue imaging indefinitely, which squeezes storage capacity for storing captured moving images . In order to avoid such compression of storage capacity, it is necessary to increase the compression ratio of moving images and reduce the amount of compression code, but on the other hand, image quality deterioration due to compression must be avoided to achieve the monitoring purpose. , Both are contradictory requests.
Patent Document 1 discloses a surveillance camera system which sets quantization parameters so as to obtain a desired image quality bit rate for each of video data generated by a plurality of cameras.
Specifically, the monitoring camera system of Patent Document 1 determines whether or not there is an area in which movement occurs, and when no movement occurs, video at the lowest bit rate for each camera Control quantization parameters to perform encoding. On the other hand, when motion occurs in one or more cameras, quantum is performed so that the bit rate of the image quality of the set reference image quality evaluation value is obtained for the video data of the camera with the highest face recognition degree. Update the value (Q value) of the conversion parameter. According to the technique described in Patent Document 1, the Q value is set to ensure that a code amount sufficient for subject recognition is allocated to a moving region, to ensure high image quality, while the background region uses a code amount. It can be suppressed and highly compressed.

JP, 2010-279004, A

However, in the surveillance camera system as described above, in order to extract motion in an image, motion must be detected between image frames using a plurality of image frames. For this reason, a delay corresponding to a plurality of image frames occurs until the compressed moving image is generated by compressing the area (moving body area) in which the motion is detected and the background area after imaging the subject at different compression rates.
In particular, since there are cases where a network surveillance camera is used for live surveillance, there is a risk that real-time situation surveillance at the delivery destination may become difficult if there is a delay in streaming the live surveillance image. For example, when the monitoring camera side analyzes the monitoring situation and stores the moving image at that time in the built-in storage device and reports the event to the remote monitoring person, the moving image of the monitoring person notified of the event is delivered live There will be a time lag before confirming the

  The present invention has been made in view of the above problems, and an object thereof is to provide an imaging apparatus capable of appropriately performing image compression while reducing delay in image delivery, and a control method and program for the imaging apparatus. It is in.

  In order to solve the above problems, according to an aspect of an imaging device according to the present invention, an imaging means for imaging a subject and acquiring a captured image, and a moving body in the captured image captured by the imaging means Detection means, compression means for compressing the captured image acquired by the imaging means to generate a compressed image, distribution means for distributing the compressed image generated by the compression means to an external device via a network When the compressed image is not being delivered to the external device, a moving body area, which is an area where the moving body is detected by the detecting means, is compressed in a first mode, and a background area other than the moving body area in the captured image There is provided an imaging apparatus comprising control means for controlling the compression means to generate the compressed image so as to compress in a second mode different from the first mode.

  According to the present invention, it is possible to appropriately perform image compression while reducing delay in image distribution.

BRIEF DESCRIPTION OF THE DRAWINGS The figure which shows an example of the network configuration of the network surveillance camera system which concerns on each embodiment of this invention. The figure which demonstrates the process which sets a different compression rate to a moving body area | region and a background area | region in a monitoring image. FIG. 2 is a diagram showing an example of a hardware configuration and a functional configuration of the network surveillance camera according to the first embodiment. FIG. 2 is a view showing an example of the hardware configuration of an administrator terminal and a server apparatus according to the first embodiment. The figure explaining the delay from the moving body detection in an image to a moving image compression. FIG. 7 is a diagram for explaining compression coding that refers to moving body region information in the first embodiment. FIG. 7 is a diagram for explaining compression encoding without referring to moving body region information in the first embodiment. 6 is a flowchart illustrating an example of a processing procedure of compression coding switching processing according to the first embodiment. 12 is a flowchart illustrating an example of a processing procedure of compression coding switching processing according to the third embodiment. FIG. 14 is a diagram showing an example of a hardware configuration and a functional configuration of a network surveillance camera according to a fourth embodiment. FIG. 14 is a diagram for explaining compression coding that refers to moving body region information in the fourth embodiment. FIG. 16 is a diagram for explaining compression encoding without referring to moving body region information in the fourth embodiment. FIG. 18 is a view showing an example of the processing procedure of the compression encoding switching process according to the fourth embodiment. FIG. 14 is a diagram showing an example of a hardware configuration and a functional configuration of a network surveillance camera according to a fifth embodiment. FIG. 16 is a diagram for explaining compression coding that refers to moving body region information in the fifth embodiment. FIG. 16 is a view for explaining compression coding which refers to a smaller number of image frames in FIG. 15 in the fifth embodiment. 16 is a flowchart illustrating an example of a processing procedure of moving object detection setting switching processing according to the fifth embodiment. FIG. 18 is a view showing an example of the processing procedure of the compression encoding switching process according to the sixth embodiment.

  Hereinafter, an embodiment for carrying out the present invention will be described in detail with reference to the attached drawings. The embodiment described below is an example as a realization means of the present invention, and should be appropriately corrected or changed according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to the embodiment. Moreover, not all combinations of the features described in the present embodiment are essential to the solution means of the present invention.

(Embodiment 1)
In this embodiment, a moving object in an image is detected, and a compression ratio such that an area where the moving object is detected has a sufficient code amount, and a code amount where the code amount is reduced to an area where the moving object is not detected Can be set to compress and encode the image. Then, in the present embodiment, if a compressed image is being delivered to an external device, the entire image is compressed and encoded without referring to the information of the area in which the moving object is detected. As a result, the delay in image delivery of the network camera is reduced, and unless compression is in progress, proper compression coding is applied with a high compression rate applied to the background area while maintaining the image quality of the area where moving objects are detected. To be realized.
Hereinafter, an area where a moving object is detected in an image is referred to as a “moving object area”, and an area where no moving object is detected, ie, an area other than the moving object area is referred to as a “background area”. Further, the code amount after compression coding is referred to as "compression code amount". In order to give a sufficient amount of compression code to the moving object area, a low compression rate is set to the moving object area, and to give a reduced compression code amount to the background area, a higher compression rate is set to the background area than to the moving object area. These compression rates are set by adjusting the quantization parameter (Q value). The Q value is an abbreviation of Quantization Parameter Value.
In the following, although the case of applying a network camera to monitoring is described as an example in the present embodiment, the present embodiment is not limited to this case, and can be applied to other image distribution applications.

<Network Configuration of this Embodiment>
FIG. 1 is a diagram showing an example of a network configuration of a network surveillance camera system according to the present embodiment.
The network surveillance camera system of FIG. 1 includes a network camera 1, an administrator terminal device 2, a server 3, and a NAS (Network Attached Storage) 4. The network camera 1 is connected to the administrator terminal device 2, the server 3, and the NAS 4 via the network 5 to mutually exchange information. The network camera 1 may be further connected to the cloud server 8 via the router 6 and the network 7 such as the Internet.

The network camera 1 includes, for example, a built-in flash memory, an external USB memory, a hard disk (HDD) 14 or an SD card as a local non-volatile storage device, and can capture a captured image on these storage devices. .
The network camera 1 can also upload a captured image to the server 3, the NAS 4, and the cloud server 8 via the network as an external nonvolatile storage device, and record the uploaded image file in each storage device ( Arrow 1A). For uploading the captured image, for example, a protocol such as HTTP (Hyper Text Transfer Protocol) or FTP (File Transfer Protocol) can be used. The uploaded image file may be appropriately reproduced by an external server 3, an image collection device such as a cloud server 8 or the administrator terminal device 2.
Hereinafter, recording to the local nonvolatile storage device of the network camera 1 is referred to as “edge recording” or simply “recording”, and uploading for recording on an external device is referred to as “upload recording” or simply “upload”.
In the network camera 1, recording and uploading may be performed simultaneously, and as described later, a recording destination, a recording start trigger, a recording time, and the like may be appropriately set by an external setting command.

The network camera 1 analyzes the captured image obtained by capturing the monitoring target, and when detecting the occurrence of a preset event, records the moving image at the time of the event occurrence (edge recording), and The occurrence of an event can be notified (arrow 2A). The administrator terminal device 2 notified of the occurrence of the event requests delivery of a live image stream in order to check a live image captured by the network camera 1. In response to the image delivery request, the network camera 1 delivers the requested image to the administrator terminal 2 as a live image stream (arrow 3A). To deliver this live image stream, a protocol such as Real Time Streaming Protocol (RTSP) can be used, for example.
The administrator terminal device 2 can simultaneously reproduce the live image to be streamed. In order to quickly confirm the motion of the subject to be monitored, the captured image compressed from the network camera 1 must be delivered promptly without delay. In the present embodiment, the responsiveness to the image distribution request is enhanced by rapidly compressing and encoding the captured image without referring to the information of the moving body region during the distribution of such a live image stream.

FIG. 2 is a diagram for explaining processing of setting different compression rates for a moving object area and a background area in a monitoring image.
As shown in FIG. 2, the image 201 captured by the network camera 1 includes a moving body area 202 in which a moving body is detected and a background area 203 in which a moving body is not detected. The motion region 202 is compression encoded by applying a low compression ratio to maintain high image quality such that the monitoring target can be accurately detected and recognized. On the other hand, the background area 203 which does not contribute to detection of the monitoring target is compressed and encoded by applying a higher compression rate than the moving body area 202 so as not to compress the storage capacity of the local nonvolatile storage device. In the present embodiment, different compression rates are applied to the moving object area 202 and the background area 203 while the image is not being distributed.

<Configuration of Network Camera 1>
FIG. 3 is a diagram showing an example of a hardware configuration and a functional configuration of the network camera 1 according to the present embodiment.
The network camera 1 in FIG. 3 includes an imaging unit 11, a communication unit 12, a control unit 13, a storage unit 14, a moving object detection unit 15, and a compression encoding unit 16.
The imaging unit 11 includes a lens 111, an imaging element 112, and a development processing unit 113. The lens 11 is an optical lens for focusing incident light from a subject to be imaged on the imaging element 112, and focuses incident light on the imaging element 112. The imaging element 112 is an element that converts light into an image signal, and can be configured by, for example, a CCD, a complementary metal oxide semiconductor (CMOS), or the like. The imaging element 112 outputs the converted image signal (video signal) to the development processing unit 113.
The development processing unit 113 performs development processing on the image signal input from the imaging element 112, stores the image signal subjected to development processing in the storage unit 14, and outputs the image signal to the compression encoding unit 16. This development process includes, but is not limited to, image correction such as color, brightness, contrast, saturation, etc., image size conversion, data format conversion of an image, and the like.
Note that an encoder (coding apparatus) may be configured by components obtained by removing the imaging unit 11 from the network camera 1 in FIG. 3.

  The communication unit 12 provides a wired LAN communication interface between the network camera 1 and an external device including the administrator terminal device 2, the server 3, and the NAS 4. As this wired LAN, for example, the configuration of a plurality of routers, switches, cables, etc. conforming to a communication standard such as Ethernet (registered trademark) can be mentioned. Alternatively, the communication unit 12 may communicate with an external device by wireless communication. The wireless communication includes a wireless personal area network (PAN) such as Bluetooth (registered trademark), ZigBee (registered trademark), UWB (Ultra Wide Band), and the like. Also, a wireless local area network (LAN) such as Wi-Fi (Wireless Fidelity) (registered trademark) or a wireless MAN (Metropolitan Area Network) such as WiMAX (registered trademark) is included. Furthermore, it includes wireless WAN (Wide Area Network) such as LTE / 3G. The communication unit 12 may communicate with an external device, and the communication standard, size, and configuration are not limited to the above.

The control unit 13 is a CPU that functions as an arithmetic processing unit that performs processing such as calculation and logical determination in each unit of the network camera 1, and each component (11, 12, 14, 15, and 16) via a system bus Control. The functions and processing of the network camera 1 may be realized by the control unit 13 reading a program stored in the storage unit 14 and executing this program.
The storage unit 14 includes an electrically erasable nonvolatile memory such as a flash memory or an EEPROM (Electrically Erasable Programmable Read Only Memory) as a built-in memory. The storage unit 14 further includes a RAM constituted by a main memory of the control unit 13 and a volatile memory functioning as a work memory. The storage unit 14 may also include a storage medium configured with a non-volatile memory such as an external flash memory / card, a removable HDD, or a USB memory. Various information used by the control unit 13 is recorded in the storage unit 14. In addition, the storage unit 14 can record a compressed image (edge recording) by storing the captured image which is imaged by the imaging unit 11 and compressed by the compression encoding unit 16.

The moving body detection unit 15 detects a moving body in the captured image from the plurality of image frames output from the development processing unit 113 to the image frame buffer of the storage unit 14 and detects moving body region information which is information of a region in which the moving body is detected. It is output to the compression encoding unit 16. By referring to the moving body area information input from the moving body detecting unit 15, the compression encoding unit 16 can change the compression ratio between the moving body area and the background area in the captured image. The moving body area information is information indicating the position and size of a moving body area in an image such as the moving body area 202, for example.
The compression encoding unit 16 compresses and encodes the image signal output from the development processing unit 113 to the image frame buffer of the storage unit 14 to generate a compressed image. The compression encoding unit 16 stores the generated compressed image in the storage unit 14 and supplies the storage unit 14 to the communication unit 12.

<Hardware Configuration of Administrator Terminal Device 2>
FIG. 4 is a diagram showing an example of the hardware configuration of the administrator terminal device 2.
As shown in FIG. 4, the administrator terminal device 2 includes a CPU 21, a ROM 22, a RAM 23, an external memory 24, an input unit 25, a display unit 26, a communication I / F 27, and a system bus 28. . The CPU 21 centrally controls the operation in the administrator terminal device 2 and controls each component (21 to 27) via the system bus 28. The ROM 22 is a non-volatile memory that stores control programs and the like necessary for the CPU 21 to execute processing. The program may be stored in the external memory 24 or a removable storage medium (not shown). The RAM 23 functions as a main memory, a work area, and the like of the CPU 21. That is, the CPU 21 loads a necessary program and the like from the ROM 22 to the RAM 23 at the time of execution of processing, and realizes various functional operations by executing the program and the like.

The external memory 24 stores, for example, various data, various information, and the like necessary when the CPU 21 performs a process using a program. The external memory 24 stores, for example, various data, various information, and the like obtained by the CPU 21 performing processing using a program or the like. The input unit 25 is configured of a pointing device such as a keyboard or a mouse. The display unit 26 is configured of a monitor such as a liquid crystal display (LCD). The communication I / F 27 is an interface that controls communication between the administrator terminal device 2 and the network camera 1, the server 3, the NAS 4, and the cloud server 8.
The functions of at least a part of the elements of the administrator terminal device 2 shown in FIG. 4 can be realized by the CPU 21 executing a program. However, at least a part of each element of the administrator terminal device 2 shown in FIG. 2 may operate as dedicated hardware. In this case, dedicated hardware operates based on the control of the CPU 21.
The server 3 and the cloud server 8 shown in FIG. 1 can also be realized with the same hardware configuration.

<Operation Overview of Network Camera 1>
Returning to FIG. 3, the operation of each part of the network camera 1 according to various commands and setting information input to the network camera 1 and these various commands and setting information will be described. The control unit 13 controls the operation of each unit of the network camera 1 in accordance with various commands and setting information input to the communication unit 12 from the external device via the network.
In the present embodiment, “delivery” includes streaming of live images unless otherwise defined.

Specifically, the communication unit 12 is connected to the network, and from an external device such as the administrator terminal device 2, an imaging setting command related to imaging for setting the imaging size and the frame rate indicating the number of image frames per unit time To receive. The communication unit 12 also receives a development setting command related to development processing such as exposure control and white balance for a captured subject image. The imaging setting command and the development setting command are hereinafter referred to as "image setting command".
The control unit 13 analyzes the received image setting command, and controls the operation of the imaging device 112 and the development processing unit 113 according to the analysis result.
The analyzed image setting command is simultaneously stored in the storage unit 14 as image setting information for setting the image processing of the network camera 1. At the time of activation of the network camera 1, in accordance with the image setting information stored in the storage unit 14, settings of imaging and development are performed from the control unit 13 to the imaging element 112 and the development processing unit 113. The development processing unit 113 supplies the image subjected to the development processing according to the image setting information to the compression encoding unit 16.

The communication unit 12 also receives, from an external device such as the administrator terminal device 2, a delivery image setting command related to a delivery image for setting a compression code amount of an image to be delivered, and a delivery start / stop command for delivery of compressed images. The control unit 13 analyzes the distribution image setting command, and controls the operation of the compression encoding unit 16 according to the analysis result.
The analyzed distribution image setting command is simultaneously stored in the storage unit 14 as distribution image setting information for setting the compression encoding process of the distribution image of the network camera 1. When the network camera 1 is activated, the control unit 13 sets the compression encoding of the distribution image to the compression encoding unit 16 according to the distribution image setting information stored in the storage unit 14.
The control unit 13 also analyzes the distribution start command, and when distribution of the compressed image is instructed, the external device such as the administrator terminal device 2 that has requested distribution of the compressed image compression-coded by the compression encoding unit 16 Start sending to. Similarly, the control unit 13 analyzes the distribution stop command, and if the compressed image is being distributed, transmission of the compressed image compressed and encoded by the compression encoding unit 16 to an external device such as the administrator terminal device 2 is stopped. Do.

The communication unit 12 also receives a moving object detection setting command for setting a moving object to be detected in the captured image from an external device such as the administrator terminal device 2 or the like. This setting command for moving object detection is limited detection of the type of movement (type of moving object detection) such as removal or leaving, mischief, intrusion, or passage, or detection if the subject to be photographed moves without restriction on the type of movement. Includes settings such as The control unit 13 analyzes the moving object detection setting command, and controls the operation of the moving object detection unit 15 according to the analysis result.
The analyzed moving object detection setting command is simultaneously stored in the storage unit 14 as moving object detection setting information for setting moving object detection processing of the network camera 1. At the time of activation of the network camera 1, in accordance with the moving body detection setting information stored in the storage unit 14, setting of moving body detection is performed from the control unit 13 to the moving body detection unit 15.

In this moving object detection setting command, the compressed image is uploaded for a predetermined time to an image collection device (server 3, NAS 4, cloud server 8 or the like) connected to the network via the communication unit 12 according to the moving object detection type. Includes settings. Here, “upload” means that the control unit 13 uploads the compressed image compressed by the code compression unit 16 via the communication unit 12 to the upload destination that is the image collecting apparatus via the communication unit 12. The uploaded compressed image can be recorded or played back on the image acquisition device.
In addition, the upload performed according to the moving object detection type means that the control unit 13 notified of the moving object detection information from the moving object detection unit 15 uploads a compressed image in the case of the moving object detection type A and in the case of the detection type B, for example. Includes settings such as not uploading a compressed image. The control unit 13 stores the moving object detection setting command including the input upload setting in the storage unit 14 as upload setting information according to the moving object detection type. At the time of activation of the network camera 1, the setting of upload is performed from the control unit 13 to the moving object detection unit 15 according to the upload setting information corresponding to the moving object detection type stored in the storage unit 14.

Similarly, the moving object detection setting command includes a setting for recording the compressed image in the storage unit 14 for a predetermined time according to the moving object detection type. The control unit 13 analyzes the input moving body detection setting command, and controls the storage operation of the compressed image in the storage unit 14 according to the analysis result.
The moving object detection setting command including the analyzed recording setting is simultaneously stored in the storage unit 14 as recording setting information according to the moving object detection type. When the network camera 1 is activated, the control unit 13 restores the recording setting to the storage unit 14 according to the recording setting information corresponding to the moving object detection type stored in the storage unit 14. The recording destination of the storage unit 14 is not limited to a non-volatile memory such as a flash memory built in the network camera 1 and may be a removable medium such as an external flash memory / card, HDD or the like.

<Details of motion detection and compression coding processing>
The moving object detection process in the captured image performed by the moving object detection unit 15 and the compression encoding process of the captured image performed by the compression encoding unit 16 in the present embodiment will be described in detail with reference to FIGS.
FIG. 5 is a diagram for explaining a delay from detection of a moving object in an image to compression of a moving image.
A volatile image buffer memory 141 for temporarily storing an image included in the storage unit 14 of the network camera 1 sequentially stores a plurality of image frames 1 to 5 constituting a moving image, which are developed by the development processing unit 113. Be done. The moving body detection unit 15 sequentially detects the moving body region (motion) in the image based on the moving body detection setting information set by the control unit 13 with reference to the image frames 1 to 5 stored in the image buffer memory 141 sequentially. . In order to detect a moving object from a moving image, the moving object right unit 15 calculates a background difference or an inter-frame difference between a plurality of image frames.
Therefore, when the compression encoding unit 16 tries to apply a compression rate lower than that of the background region to the moving object region in the image, the moving object detection unit 15 calculates moving object region information by referring to a plurality of image frames. I have to wait. FIG. 5 schematically shows that, as an example, the moving object detection unit 15 delays by 4 frames from the first image frame 1 until the compression encoding unit 16 starts image compression.

In the present embodiment, the compression encoding unit 16 compresses the captured image without referring to the moving object region information and a first mode in which the captured image is compressed with reference to the moving object region information output by the moving object detection unit 15. It has a second mode.
Referring to FIG. 6 showing the first mode, a plurality of image frames 1 to 5 developed by development processing unit 113 are sequentially stored in buffer memory 141. The moving body detection unit 15 sequentially detects the moving body region in the image based on the moving body detection setting information set by the control unit 13 with reference to the image frames 1 to 5 stored in the buffer memory 141 sequentially.
In addition, the moving body detection unit 15 outputs moving body area information 5 ′, which is information of the area in which the movement is detected, to the compression encoding unit 16. In the distribution image setting information set in the compression encoding unit 16, the compression code amount is sufficiently given to the moving object area by reducing the Q value, and the Q value is calculated from the Q value of the moving object area in the non-moving background area. By increasing the size, it is set to reduce the amount of compression code. For this reason, the compression encoding unit 16 refers to the moving body area information 5 ′ output from the moving body detection unit 15 to identify the moving body area and the background area. Then, the compression encoding unit 16 compresses the moving area using a low compression rate so as to maintain high image quality in the moving body area, and compresses the background area using a high compression rate so as to reduce the storage capacity of the background area. , Compress image frame 5. In the first mode shown in FIG. 6, the compression encoding unit 16 delays by 4 frames to compress a captured image with reference to moving body region information.

In addition, the moving body area | region which the moving body detection part 15 produces | generates may include a background area | region.
In addition, the compression code amount setting in the image compression performed by the compression encoding unit 16 in the first mode may be such that the compression ratio for the background area is relatively higher than the compression ratio for the moving object area. For example, the compression ratio of the moving object region may be changed to a low value so as to increase the compression coding amount of the moving object region relative to the background region. You may change it high.
The number of image frames referred to by the moving body detection unit 15 shown in FIG. 6 is an example, and the processing time for the moving body detection unit 15 to detect the movement in the captured image is expressed as the number of image frames in the buffer. is there.

Referring to FIG. 7 showing the second mode, buffer memory 141 sequentially stores a plurality of image frames 1 to 5 developed by development processing unit 113. The moving body detection unit 15 sequentially detects the moving body region in the image based on the moving body detection setting information set by the control unit 13 with reference to the image frames 1 to 5 stored in the buffer memory 141 sequentially.
In addition, the moving body detection unit 15 outputs moving body area information 5 ′, which is information of the area in which the movement is detected, to the compression encoding unit 16. However, in the second mode, the compression encoding unit 16 does not refer to the moving subject region information 5 ′ output from the moving subject detection unit 15, and the delivery image setting information for setting different compression rates in the moving subject region and the background region. Also do not refer. Therefore, the compression encoding unit 16 compresses the entire image frame 1 of the buffer memory 141 using the same Q value without distinguishing between the moving object region and the background region. Although the Q value used at this time can take an arbitrary value according to the free space of the non-volatile storage device for recording, etc., the value between two Q values in the first mode as an example It may be. In the second mode, the control unit 13 may control the moving object detection unit 15 to stop the process of generating moving object area information.
In the second mode shown in FIG. 7, the captured image can be compressed without reference to moving body area information, that is, without waiting for the moving body detection unit 15 to calculate moving body area information. Therefore, the image frame 1 output from the development processing unit 113 can be read immediately, and the delay shown in FIG. 6 does not occur. As described above, in the second mode, the delay from when the imaging processing unit 113 develops the captured image to when the compression encoding unit 16 compresses the captured image is shortened with respect to the first mode.

<Processing flowchart of compression coding switching process>
FIG. 8 is a flowchart showing an example of the processing procedure of the compression coding switching process according to the present embodiment. The process shown in FIG. 8 is started, for example, from the time when the network surveillance camera system including the network camera 1 is activated, but the start timing of the process shown in FIG. 8 is not limited to the above. The network camera 1 can execute the processing shown in FIG. 8 by reading out a program required by the CPU constituting the control unit 13 from the storage unit 14 and executing the program. However, the process of FIG. 8 may be realized by operating at least a part of the respective elements shown in FIG. 3 as dedicated hardware. In this case, dedicated hardware operates under control of the CPU.

  In step S81, the control unit 13 determines whether the compressed image is being distributed via the network. Specifically, the control unit 13 distributes the compressed image when the distribution start command is received from the external device such as the administrator terminal device 2 via the communication unit 12 and the distribution stop command has not been received yet. Determined to be medium. If it is determined that the compressed image is being distributed (S81: Y), the process proceeds to S82, and if it is determined that the compressed image is not being distributed (S81: N), the process proceeds to S83.

When it is determined in S81 that the compressed image is being distributed, in S82, the control encoding unit 16 compresses the captured image without referring to the moving object region information output from the moving object detecting unit 15 in S82. Control the compression encoding unit 16 (second mode).
On the other hand, when it is determined in S81 that the compressed image is not being distributed, in S83, in the control unit 13, the compression encoding unit 16 compresses the captured image with reference to the moving body region information output from the moving body detection unit 15. Control the compression encoding unit 16 (first mode). In S83, if the compressed image is being recorded in the storage unit 14 of the network camera 1 and it is determined that the compressed image is not being distributed, the compression encoding unit 16 refers to the moving object region information. The captured image may be compressed.
As a result, when the network camera 1 is delivering the compressed image to the external device, the compression encoding unit 16 compresses the captured image without referring to the moving object region information, so that the delivery delay of the compressed image is reduced. Ru. On the other hand, when the network camera 1 is not delivering the compressed image, the compression encoding unit 16 refers to the moving body area information to reduce the compression rate for the moving body area in the captured image, so the image quality of the moving body area to be monitored Can be maintained. Such delivery delay reduction is particularly effective in the case of live stream delivery, but the present embodiment is also applicable to delivery by uploading to an external device.

When the control unit 13 receives the distribution start command and switches the code compression operation based on the moving object detection area information of the compression coding unit 16, the compressed image of a predetermined time is being uploaded to an external image collection device Alternatively, recording may occur in the storage unit 14. In this case, the control unit 13 starts the process of FIG. 8 when starting or ending uploading or recording of a compressed image for a predetermined time. That is, the reference setting of the moving object region information of the compression encoding unit 16 is changed to ON and OFF, respectively, and uploading or recording to an external device for a predetermined time is continued. In this case, since the compression code amount is not changed during the upload or recording process for a predetermined time, the upload or recording process can be continued with the compressed image for the predetermined time as one container (file).
Alternatively, the control unit 13 analyzes the distribution start / stop command even while uploading or recording a compressed image, and changes whether to reference moving object area information of the compression encoding unit 16 or not. is there. In this case, the compressed image being uploaded or recorded may be divided as separate containers at the timing when the reference setting of the moving object region information is changed.

In addition, regardless of whether or not the compressed image is being distributed, a moving object area reference setting command may be provided to enable moving object area reference setting whether to refer to the moving object area information. When this moving body area reference setting command is received from the external device from the communication unit 12, the control unit 13 analyzes and dynamically switches the moving body area reference from off to on even while distributing the compressed image, and monitoring It is possible to maintain the image quality of the moving object area to be.
Furthermore, the moving body detection setting information may be set in the moving body detection unit 15, and an extended distribution start command (second distribution start command) may be provided to start distribution equivalent to upload according to the moving body detection type.

  As described above, according to the present embodiment, the control unit 13 of the network camera 1 determines whether or not the compressed image is being distributed to the external device based on the reception of the distribution start / stop command. When the compressed image is being distributed, the compression encoding unit 16 compresses the captured image without referring to the moving body area information output by the moving body detecting unit 15, so that the distribution delay of the compressed image is reduced. On the other hand, when the compressed image is not being distributed, the compression encoding unit 16 refers to the moving body area information to set the compression ratio so that the moving body area in the captured image has a sufficient compression code amount. Sets a higher compression rate than the moving body area. Therefore, the storage area can be reduced by highly compressing the background area while maintaining the image quality of the moving object area to be monitored.

Second Embodiment
Hereinafter, the second embodiment will be described in detail only with respect to differences from the first embodiment. In this embodiment, the distribution start / stop command for requesting distribution of the compressed image from the external device to the network camera 1 is provided with distribution settings for display and recording. In the switching process whether or not the compression encoding unit 16 refers to moving object area information, when the distribution start / stop command is the distribution setting for recording, the compression encoding unit 16 refers to moving object area information. Do. In this way, when the delivery of the compressed image is a delivery in which the real-time property is not relatively high, priority is given to maintaining the image quality of the moving object area to be monitored by making the compression encoding unit 16 refer to the moving object area information. Becomes easy.

The configuration of the network camera 1 according to the present embodiment is the same as that of the first embodiment shown in FIG.
In the present embodiment, the distribution start / stop command received by the communication unit 12 is provided with distribution settings for display and for recording, and either display or for recording is set.
Referring to FIG. 3, communication unit 12 receives a distribution start / stop command from an external device such as administrator terminal device 2. In the received distribution start / stop command, one of values for display and for recording is set.
When analyzing the distribution start command for recording, the control unit 13 causes the compression encoding unit 16 to compress the captured image with reference to the moving body area information, and the external device requests the communication unit 12 to distribute the compressed image. Start sending to Further, the control unit 13 analyzes the distribution stop command, and if it is a distribution stop command for recording, the compression encoding unit 16 is set so as not to refer to the moving object area information, and the communication unit 12 externally outputs the compressed image. Stop sending to the device.
On the other hand, the operation of the network camera 1 when the distribution start / stop command set for display is received is the same as that of the first embodiment.

  Note that the moving object detection setting command received by the communication unit 12 includes a setting for uploading a compressed image for a predetermined time to the image collection device connected to the network camera 1 via the communication unit 12 according to the moving object detection type. It is also good. Similarly, the moving object detection setting command may include a setting for recording the compressed image in the storage unit 14 for a predetermined time according to the moving object detection type.

Next, with reference to FIG. 8, the switching procedure of the compression encoding setting based on the dynamic area information performed by the control unit 13 in the present embodiment will be described.
In the present embodiment, in S81, the control unit 13 determines whether the compressed image is being delivered for recording, being delivered for display, or not being delivered via the network. Specifically, the control unit 13 receives a distribution start command for recording from an external device such as the administrator terminal device 2 via the communication unit 12 and has not yet received a distribution stop command for recording. In this case, it is determined that the compressed image is being distributed for recording. If it is determined that the image is not being distributed, and if it is determined that the compressed image is being distributed for recording, the process proceeds to S83. On the other hand, when it is determined that the compressed image is being distributed for display, the process proceeds to S82.
Thereafter, as in the first embodiment, the compression encoding unit 16 executes the second mode for compressing the captured image without referring to the moving body area information output by the moving body detection unit 15 in S82, and in S83, the moving body A first mode of compressing a captured image with reference to area information is executed.

As described above, according to the present embodiment, when a compressed image is being distributed to an external device for display, the compression encoding unit 16 refers to moving body region information output by the moving body detection unit 15. Because the captured image is compressed without any delay, the delivery delay of the compressed image is reduced. On the other hand, when the compressed image is being distributed for recording, the compression encoding unit 16 refers to the moving body region information and the moving body region in the captured image has a sufficient compression code amount, as in the case of not distributing. The compression rate is set, and the compression rate higher than the moving body area is set in the background area. Therefore, the storage area can be reduced by highly compressing the background area while maintaining the image quality of the moving object area to be monitored.
As described above, according to the present embodiment, when the delivery of the compressed image is a delivery in which the real-time property is not relatively high, the moving object region to be monitored is made to refer to the moving object region information by the compression encoding unit 16 It becomes easy to prioritize the maintenance of the image quality of the above and improve the recognition accuracy of the monitoring target.

(Embodiment 3)
Hereinafter, with reference to FIG. 9, the third embodiment will be described in detail only with respect to differences from the above first embodiment. In the present embodiment, triggered by detection of a moving object to be monitored, it is determined whether or not a compressed image is being uploaded to an external image acquisition device for a predetermined time or recorded in the storage unit 14. In the switching process as to whether or not the compression encoding unit 16 refers to moving object area information, when the compressed image is being uploaded to an external image acquisition device or being recorded to a storage unit, the compression encoding unit 16 Refers to motion region information. Triggered by detection of a moving object to be monitored, within a predetermined time after upload or recording of a compressed image is started, there is a high possibility that the moving object to be monitored still exists in the image. In such a case, it is easy to give priority to maintaining the image quality of the moving object area to be monitored by making the compression encoding unit 16 refer to the moving object area information using the first mode.

The configuration of the network camera 1 according to the present embodiment is the same as that of the first embodiment shown in FIG.
Next, with reference to FIG. 9, the switching procedure of the compression encoding setting based on the dynamic area information performed by the control unit 13 in the present embodiment will be described.
In S91, the control unit 13 determines whether uploading to an external image acquisition device or recording on the storage unit 14 is started, and the uploading or recording is still in progress. Specifically, the control unit 13 detects a moving object to be monitored based on the upload setting information corresponding to the moving object detection type. Then, triggered by the detection of the moving object to be monitored, uploading of the compressed image to the image acquisition device is started, and after that, when the predetermined time has elapsed and it has not stopped, it is determined that the compressed image is being uploaded.

Similarly, the control unit 13 starts storage of the compressed image in the storage unit 14 based on the detection of the moving object to be monitored based on the recording setting information corresponding to the moving object detection type, and thereafter, for a predetermined time If it does not stop after elapsed, it is determined that the compressed image is being recorded. If it is determined that uploading or recording is not in progress, the process proceeds to S92, and if it is determined that uploading or recording is in progress, the process proceeds to S93. In S91, it may be determined whether uploading or recording is in progress.
After that, as in the first embodiment, the compression encoding unit 16 executes the second mode for compressing the captured image without referring to the moving body region information output by the moving body detecting unit 15 in S92, and in S93, the moving body A first mode of compressing a captured image with reference to area information is executed.

  When the control unit 13 determines whether the compressed image is being uploaded or recorded and switches the code compression operation based on the moving object detection area information of the compression coding unit 16, the distribution start command is received from the external device After that, the delivery stop command may not have been received. In this case, the control unit 13 does not change the reference setting of the moving object area information of the compression encoding unit 16 until the distribution stop command is received. Then, as soon as the distribution stop command is received, the reference setting of the moving object area information of the compression encoding unit 16 is changed. That is, while the compressed image is being delivered, the compression rate of the compressed image is not changed.

  Even after receiving the distribution stop command, the control unit 13 may store the compressed image being uploaded or recorded as one container for a predetermined time if the compressed image is being uploaded or recorded. Alternatively, the control unit 13 may divide the compressed image being uploaded or recorded as separate containers at the timing when the reference setting of the moving object region information is changed.

Conversely, when the distribution start command is received within a predetermined time after the start of uploading or recording of the compressed image to the image acquisition device, the control unit 13 performs compression encoding even after the predetermined time of upload or recording has elapsed. The compression encoding setting based on the dynamic area information of the unit 16 is not changed. After receiving the distribution start command, the control unit 13 may change the compression encoding setting based on the dynamic region information of the compression encoding unit 16.
The present embodiment may be combined with the above embodiment as appropriate.

As described above, according to the present embodiment, when the compressed image is being uploaded to an external image acquisition device or not being recorded to the storage unit 14, the compression encoding unit 16 outputs the moving object detection unit 15. A captured image is compressed without referring to moving body region information. Therefore, the delivery delay of the compressed image is reduced. On the other hand, when the compressed image is being uploaded to an external image acquisition device or being recorded to the storage unit 14, the compression encoding unit 16 refers to the moving object area information and compresses the sufficient moving object area in the captured image. The compression rate is set to be the amount, and the compression rate higher than the moving body area is set in the background area. Therefore, the storage area can be reduced by highly compressing the background area while maintaining the image quality of the moving object area to be monitored.
Therefore, according to the present embodiment, since it is within a predetermined time after the start of uploading or recording of the compressed image, the moving object area to be monitored when the moving object to be monitored is still likely to be present in the image. It is easy to give priority to maintaining the image quality of

(Embodiment 4)
Hereinafter, the fourth embodiment will be described in detail with reference to FIGS. 10 to 13 only with respect to differences from the first embodiment described above. The network camera 1 according to the present embodiment includes a flash memory, which is a non-volatile memory, and a DRAM, which is a volatile memory, as the storage unit 14. In the present embodiment, the captured image is buffered in the DRAM, and the moving object detection unit 15 writes the moving object area information in the DRAM. The compression encoding unit 16 delivers a compressed image obtained by reading out a captured image from the DRAM and compressing the image to an external device. Therefore, the uncompressed image for upload or for recording for the predetermined time buffered in the DRAM can be compressed by the compression encoding unit 16 with reference to the corresponding moving object region information. As a result, it is easy to highly compress the background area to reduce the storage capacity while maintaining the image quality of the moving object area to be monitored.

FIG. 10 shows an example of the configuration of the network camera 1 according to the present embodiment.
The network 1 according to the present embodiment shown in FIG. 10 is the same as that of the first embodiment shown in FIG. 3, but the storage unit 14 includes a flash memory 142 which is a non-volatile memory and a DRAM 143 which is a volatile memory. It differs in the point. The flash memory and the DRAM are merely examples, and the memory provided in the present embodiment is not limited to these, and any memory may be used as long as it includes nonvolatile memory and volatile memory.

With reference to FIG. 10, differences from the first embodiment in the operation of each unit of the network camera 1 according to various commands and setting information input to the network camera 1 and these various commands and setting information will be described.
Specifically, the imaging element 112 captures an object via the lens 111, and outputs an image signal (video signal) to the development processing unit 113. The development processing unit 113 performs development processing on the image signal input from the imaging element 112, and buffers the image subjected to development processing in the DRAM 143.

The various commands input from the external device to the communication unit 12 are analyzed by the control unit 13 and set in each unit to be set, and are stored in the flash memory 142 as various setting information. At the time of activation of the network camera 1, the control unit 13 controls the respective units to be set according to the various setting information stored in the flash memory 142.
These commands are an image setting command for setting information related to image generation such as imaging and development processing, a delivery image setting command for setting the compression code amount of delivery image, etc., delivery start / stop command, type of movement to be detected, It includes a moving object detection setting command to set an event. These commands further upload the compressed image for a predetermined time according to the moving object detection type, upload setting commands according to the moving object detection type, and recording settings according to the moving object detection type for recording the compressed image for a predetermined time according to the moving object detection type Contains commands.
The various setting information stored in the flash memory 142 by the control unit 13 includes image setting information, distribution image setting information, moving object detection setting information, upload setting information corresponding to the moving object type, and recording setting information corresponding to the moving object detection type. including.
The storage device of the recording destination provided in the network camera 1 includes a removable recording medium such as an external flash memory / card, an HDD, etc. as well as the built-in nonvolatile memory as in the first embodiment.

<Details of Moving Object Detection and Compression Encoding Processing of Fourth Embodiment>
The moving object detection process in the captured image performed by the moving object detection unit 15 and the compression process of the captured image performed by the compression encoding unit 16 in this embodiment will be described in detail with reference to FIGS. 11 and 12.
In the present embodiment, the compression encoding unit 16 refers to the first mode in which the captured image is compressed with reference to the moving subject region information output by the moving subject detection unit 15 as in the above embodiment, and without referring to the moving subject region information. In the second mode to compress the captured image. Further, in the present embodiment, both the captured image output by the development processing unit 113 and the moving object region information output by the moving object detection unit 15 are buffered in the DRAM 143.

Referring to FIG. 11 showing the first mode, in DRAM 143, a plurality of image frames 1 to 5 developed by development processing unit 113 are sequentially buffered. The moving object detection unit 15 detects a moving object region in the image based on the moving object detection setting information set by the control unit 13 by sequentially referring to the image frames 1 to 5 buffered in the DRAM 143.
Also, the moving body detection unit 15 buffers and outputs moving body area information 5 ′, which is information of the area in which the movement is detected, to the DRAM 143. In the distribution image setting information set in the compression encoding unit 16, a setting is made such that a compression code amount is sufficiently given to a moving object region and a compression code amount is reduced to a background region without motion. Therefore, the compression coding unit 16 refers to the moving object region information 5 ′ buffered in the DRAM 143 to identify the moving object region and the background region. Then, the compression encoding unit 16 compresses the moving area using a low compression rate so as to maintain high image quality in the moving body area, and compresses the background area using a high compression rate so as to reduce the storage capacity of the background area. , Compresses the image frame 5 buffered in the DRAM 143. In the first mode shown in FIG. 11, the compression encoding unit 16 delays by 4 frames to compress the captured image with reference to the moving body region information.

The moving body area information generated by the moving body detection unit 15 may include background area information.
In addition, the setting of the Q value in the image compression performed by the compression encoding unit 16 in the first mode may be such that the compression rate for the background area is relatively higher than the compression rate for the moving body area. For example, the Q value for the moving object area may be changed to increase the compression code amount of the moving object area with respect to the background area, and the Q value of the background area may be decreased to reduce the compression coding amount of the background area with respect to the moving object area. You may change it.
Note that the number of image frames referred to by the moving body detection unit 15 shown in FIG. 11 is an example, and the processing time for the moving body detection unit 15 to detect motion in the captured image is represented as the number of image frames in the buffer. is there.

Referring to FIG. 12 showing the second mode, in DRAM 143, a plurality of image frames -4 to 0 that have been subjected to development processing by development processing unit 113 are sequentially buffered. The moving object detection unit 15 detects a moving object region in the image based on the moving object detection setting information set by the control unit 13 by sequentially referring to the image frames -4 to 0 buffered in the DRAM 143.
In addition, the moving body detection unit 15 outputs moving body area information 0 ′, which is information of the area in which the movement is detected, to the DRAM 143. However, in the second mode, the compression encoding unit 16 does not refer to the moving subject region information 0 ′ output from the moving subject detection unit 15 and sets delivery code setting for setting different compression code amounts in the moving subject region and the background region. It does not refer to the information either. Therefore, the compression encoding unit 16 compresses the entire image frame 4 of the DRAM 143 using one compression rate without distinguishing the background area from the moving body area.

Next, at the time when the compression encoding unit 16 compresses the image frame 3, the moving body detection unit 15 detects the movement of the image frames -3 to 1, and buffers and outputs the moving body area information 1 'to the DRAM 143. . Similarly, at the time when the compression encoding unit 16 compresses the image frame 2, the moving object detection unit 15 detects the movement of the image frames -2 and 2 and outputs the moving object area information 2 'to the DRAM 143 as a buffer. .
As described above, the development processing unit 113 and the moving object detection unit 15 buffer the images 0 to 4 not compressed by the compression encoding unit 16 and the moving object area information 0 ′ to 4 ′ in the DRAM 143, respectively. The DRAM 143 can buffer image frames for a predetermined time for uploading or recording.

In the second mode shown in FIG. 12, the compression encoding unit 16 compresses the captured image without referring to the moving body area information buffered in the DRAM 143, that is, without waiting for calculation of moving body area information by the moving body detection unit 15. Therefore, the image frame 4 output from the development processing unit 113 can be read quickly, and the delay shown in FIG. 11 does not occur.
As described above, in the second mode, the delay from the development processing unit 113 developing the captured image to the compression encoding unit 16 compressing the captured image is shortened with respect to the first mode.

Processing Flowchart of Compression Encoding Switching Process of Fourth Embodiment
Next, with reference to FIG. 13, the switching procedure of the compression encoding setting based on the dynamic area information performed by the control unit 13 in the present embodiment will be described.
In the present embodiment, in S131, the control unit 13 determines whether upload setting information or recording setting information is set. If the upload setting information or the recording setting information is set, the process proceeds to S132. If the upload setting information or the recording setting information is not set, the process proceeds to S134.
In S132, the control unit 13 determines whether or not the compressed image is being distributed via the network. Specifically, when the control unit 13 receives a distribution start command from the external device such as the administrator terminal device 2 via the communication unit 12 and does not receive the distribution stop command, the compressed image is being distributed. It is determined that If it is determined that the delivery is not in progress, the process proceeds to S135. On the other hand, if it is determined that the compressed image is being distributed, the process proceeds to step S133.

When it is determined in S132 that the compressed image is being distributed, in S133, the control unit 13 controls the development processing unit 113 to buffer output an image for a predetermined time of uploading or recording to the DRAM 143. Further, the control unit 13 controls the moving body detection unit 15 so that the moving body area information of the image frame buffered in the DRAM 143 is buffered to the DRAM 143. When the buffer of the DRAM 143 is filled with the image for a predetermined time for uploading or recording, the buffer output to the DRAM 143 of the development processing unit 113 and the moving body detection unit 15 is set to be stopped.
In step S134, the control unit 13 controls the compression encoding unit 16 so that the compression encoding unit 16 does not refer to the moving object area information buffered and output to the DRAM 143 by the moving object detection unit 15, that is, to execute the second mode. . In addition, the control unit 13 controls the compression encoding unit 16 so as to compress the latest image that the development processing unit 113 outputs to the DRAM 143.

On the other hand, if it is determined in S132 that the compressed image is not being distributed, in S135, the control unit 13 determines whether or not the DRAM 143 has a capacity capable of buffering an image that has not been uploaded or recorded yet. Do. If it is determined that the image buffer is present, the process proceeds to S136, and if it is determined that the image buffer is not present, the process proceeds to S137.
If it is determined in S132 that the compressed image is not being distributed and it is determined in S135 that there is a buffer for the image, the control unit 13 buffers the image for a predetermined time of uploading or recording to the DRAM 143 in S136, The development processing unit 113 is controlled. Further, the control unit 13 controls the moving body detection unit 15 so that the moving body area information of the image frame buffered in the DRAM 143 is buffered to the DRAM 143.

  As a result, when the time from the distribution start command to the distribution stop command is shorter than the predetermined time, the image to be uploaded or recorded is insufficient and it is possible to buffer output the image to the DRAM 143. Alternatively, the next upload or recording starts when triggered by the detection of a new moving object set as the monitoring target within the processing time during which the compression encoding unit 16 compresses and uploads or records the buffered image. It may be set to be a buffer when it is

In step S137, the control unit 13 compresses the non-compressed image buffered and output by the development processing unit 113 to the DRAM 143 with reference to the moving object area information buffered in the DRAM 143 by the moving object detection unit 15, ie, the first mode. The compression encoding unit 16 is controlled to execute.
In step S138, the control unit 13 uploads the compressed image compressed by the compression encoding unit 16 to the image acquisition apparatus connected via the communication unit 12, or to a non-volatile memory such as the flash memory 142 of the network camera 1. record.

  As described above, according to the present embodiment, when uploading to an external image collecting apparatus or recording to the flash memory 142 is set, the development processing unit 113 performs a predetermined time for uploading or recording. The image is buffered to the DRAM 143. In addition, the moving body detection unit 15 buffers and outputs moving body area information of the image portion buffered in the DRAM 143 to the DRAM 143. The compression encoding unit 16 can compress the uncompressed image buffered in the DRAM 143 with reference to corresponding moving object area information buffered in the DRAM 143 as well. Therefore, the uncompressed image for upload or for recording for a predetermined time buffered in the DRAM 143 can be compressed by the compression encoding unit 16 with reference to the corresponding moving object region information. This makes it easy to highly compress the background area and reduce the storage capacity while maintaining the image quality of the moving object area to be monitored.

Embodiment 5
Hereinafter, the fifth embodiment will be described in detail with reference to FIGS. 14 to 17, with regard to only differences from the above-described embodiments. The network camera 1 according to the present embodiment further includes an audio input unit and an audio detection unit. In the present embodiment, a compressed image is uploaded to an external image acquisition device or recorded in a storage unit for a predetermined period of time using voice as a trigger. Further, although the present embodiment compresses the captured image with reference to the moving body area information even while distributing the compressed image, the moving body area information is generated with a smaller delay compared to the case where it is not distributing. . As a result, even while the compressed image is being delivered, the storage area can be reduced by highly compressing the background area while maintaining the image quality of the moving object area to be monitored.

FIG. 14 shows an example of the configuration of the network camera 1 according to the present embodiment.
The network 1 according to the present embodiment shown in FIG. 14 is the same as the fourth embodiment shown in FIG. 10, but further includes an audio input unit 107 and an audio detection unit 108.
The voice input unit 17 includes, for example, a microphone and an A / D conversion unit that A / D converts an analog voice signal input from the microphone to generate a digital voice signal.
The voice detection unit 18 detects a voice to be monitored from the voice signal input from the voice input unit 17. Triggered by the detection of voice, a compressed image of a predetermined time can be uploaded to the image acquisition device or recorded in a non-volatile memory such as the flash memory 142 of the storage unit.

The differences from the fourth embodiment in the operation of each unit of the network camera 1 according to various commands and setting information input to the network camera 1 and these various commands and setting information will be described with reference to FIG.
Specifically, the communication unit 12 receives a voice input setting command for setting parameters for voice input from an external device such as the manager terminal device 2 or the like. The voice input setting command includes information on the type of microphone connected to the network camera 1 and the setting at the time of A / D conversion of a voice signal input from the microphone.
The voice input setting command is analyzed by the control unit 13 and input to the voice input unit 17. At the same time, the control unit 13 stores the voice input setting command in the flash memory 142 as voice input setting information. The control unit 13 controls voice input by the voice input unit 17 according to the voice input setting information stored in the flash memory 142 when the network camera 1 is activated.

The communication unit 12 also receives a voice detection setting command for setting parameters for voice detection from an external device such as the administrator terminal device 2. The voice detection setting command includes, as an event to be detected, information that limits the type of voice, such as the volume of voice and scream. These voices are acquired as voice data by the voice input unit 17 in association with the captured image.
The voice detection setting command is analyzed by the control unit 13 and input to the voice detection unit 18. At the same time, the control unit 13 stores the voice detection setting command in the flash memory 142 as voice detection setting information. The control unit 13 controls voice detection by the voice detection unit 18 according to the voice detection setting information stored in the flash memory 142 when the network camera 1 is activated.

  Further, the voice detection setting command includes a setting for uploading a compressed image of a predetermined time to an external image acquisition device connected via the communication unit 12 according to the type of voice detection. Here, according to the voice detection type, the control unit 13 notified that the voice is detected from the voice detection unit 18 uploads a compressed image in the case of the detection type A, for example, detection type B If it is, the setting is not to upload the compressed image. Simultaneously with the command analysis, the control unit 13 stores an upload setting command corresponding to the voice detection type in the flash memory 142 as upload setting information corresponding to the voice detection type. The control unit 13 restores the upload setting from the upload setting information corresponding to the voice detection type stored in the flash memory 142 when the network camera 1 is activated.

  Similarly, the voice detection setting command includes a setting for recording a compressed image for a predetermined time in a non-volatile memory such as the flash memory 142 of the network camera 1 according to the type of voice detection. The recording destination includes, besides the flash memory 142, a non-volatile memory built in the network camera 1, a removable recording medium such as an external flash memory / card, an HDD or the like. At the same time as the command analysis, the control unit 13 stores, in the flash memory 142, a recording setting command corresponding to the type of audio detection as recording setting information corresponding to the type of audio detection. The control unit 13 restores the recording setting from the recording setting information corresponding to the voice detection type stored in the flash memory 142 when the network camera 1 is activated.

<Details of Moving Object Detection and Compression Encoding Processing of Fifth Embodiment>
The moving object detection process in the captured image performed by the moving object detection unit 15 and the compression process of the captured image performed by the compression encoding unit 16 in the present embodiment will be described in detail with reference to FIGS. 15 and 16.
In the present embodiment, the compression encoding unit 16 refers to moving body area information output by the moving body detection unit 15 even while the compressed image is being distributed to an external image acquisition device. In the present embodiment, the moving subject detection unit 15 and the compression encoding unit 16 generate moving subject region information from a plurality of image frames and refer to the generated moving subject region information, as in the first mode of the first embodiment. And the first mode for compressing the captured image. The present embodiment further has a second mode in which moving body region information is generated from a smaller number of image frames than in the first mode, and the captured image is compressed with reference to the generated moving body region information.

Referring to FIG. 15 illustrating the first mode, DRAM 143 sequentially buffers a plurality of image frames 1 to 5 of the captured image captured by imaging unit 11. The moving object detection unit 15 detects a moving object region in the image based on the moving object detection setting information set by the control unit 13 by sequentially referring to the image frames 1 to 5 buffered in the DRAM 143.
In addition, the moving body detection unit 15 outputs moving body area information 5 ′, which is information of the area in which the movement is detected, to the compression encoding unit 16. In the distribution image setting information set in the compression encoding unit 16, a setting is made such that a compression code amount is sufficiently given to a moving object region and a compression code amount is reduced to a background region without motion. For this reason, the compression encoding unit 16 refers to the moving body area information 5 ′ output from the moving body detection unit 15 to identify the moving body area and the background area. Then, the compression encoding unit 16 compresses the moving area using a low compression rate so as to maintain high image quality in the moving body area, and compresses the background area using a high compression rate so as to reduce the storage capacity of the background area. , Compresses the image frame 5 of the DRAM 143. In the first mode shown in FIG. 15, the compression encoding unit 16 delays by 4 frames to compress the captured image with reference to the moving body region information.

The moving body area information generated by the moving body detection unit 15 may include background area information.
In addition, the compression code amount setting in the image compression performed by the compression encoding unit 16 in the first mode may be such that the compression ratio for the background area is relatively higher than the compression ratio for the moving object area. For example, the compression ratio of the moving object region may be changed to a low value so as to increase the compression coding amount of the moving object region relative to the background region. You may change it high.
Further, the number of image frames referred to by the moving body detection unit 15 shown in FIG. 15 is an example, and the frame rate at which the imaging unit 11 picks up an image and the frame rate difference time at which the moving body detection unit 15 references an image The delay is expressed as the number of image frames in the buffer.
The moving body area information generated by the moving body detection unit 15 may include background area information.

  Referring to FIG. 16 illustrating the second mode, DRAM 143 sequentially buffers a plurality of image frames 1 to 5 of the captured image captured by imaging unit 11. In the second mode, the moving subject detection unit 15 generates moving subject region information by referring to the number of image frames in the buffer, which is smaller than that in the first mode. That is, the moving body detection unit 15 sequentially detects the moving body region in the image based on the moving body detection setting information set by the control unit 13 by sequentially referring to the image frames 1 and 2 buffered in the DRAM 143.

In addition, the moving body detection unit 15 outputs moving body area information 2 ′, which is information of the area in which the movement has been detected, to the compression encoding unit 16. In the distribution image setting information set in the compression encoding unit 16, a setting is made such that a compression code amount is sufficiently given to a moving object region and a compression code amount is reduced to a background region without motion. For this reason, the compression encoding unit 16 refers to the moving body area information 2 ′ output from the moving body detection unit 15 to identify the moving body area and the background area. Then, the compression encoding unit 16 compresses the moving area using a low compression rate so as to maintain high image quality in the moving body area, and compresses the background area using a high compression rate so as to reduce the storage capacity of the background area. , Compress the image frame 2 of the DRAM 143. In the second mode shown in FIG. 16, the compression encoding unit 16 delays by one frame to compress a captured image with reference to moving body region information. That is, from the first mode shown in FIG. 15, the delay from the imaging unit 11 buffering the captured image in the DRAM 143 to the compression encoding unit 16 compressing the captured image is shortened by three frames.
The number of image frames referred to by the moving body detection unit 15 shown in FIG. 16 is an example, and the processing time for the moving body detection unit 15 to detect a motion is expressed as the number of image frames in the buffer. FIG. 15 shows a standard moving object detection process, while FIG. 16 shows a shortened moving object detection process.

Further, in the second mode of the present embodiment, the moving body detection unit 15 may be realized by a moving body detection method different from the first mode. For example, in the moving object detection process when the number of reference image frames is 2, since the detection accuracy may be lower than that when the number of reference image frames is 5, the difference between the luminance information forming the image is used. Motion may be detected. Alternatively, by detecting the motion vector of the moving object from the image frame 2 and predicting the position of the moving object in the image frame 1, the moving object region information of the image 1 may be generated, for example. Good.
The switching between the standard moving body detection process (first mode) and the shortening moving body detection process (second mode) is set by moving body detection setting information for the moving body detection unit 15. The moving body detection setting command may be received from the external device such as the administrator terminal device 2 via the communication unit 12, and the control unit 13 may analyze the moving body detection setting command to acquire moving body detection setting information.

Processing Flowchart of Compression Encoding Switching Process of Fifth Embodiment
FIG. 17 is a flow chart showing an example of the processing procedure of the compression coding switching process according to the present embodiment.
In step S171, the control unit 13 determines whether the compressed image is being distributed to an external device such as the administrator terminal device 2 via the network. Specifically, the control unit 13 distributes the compressed image when the distribution start command is received from the external device such as the administrator terminal device 2 via the communication unit 12 and the distribution stop command has not been received yet. Determined to be medium. If it is determined that the compressed image is being distributed (S171: Y), the process proceeds to S173, and if it is determined that the compressed image is not being distributed (S171: N), the process proceeds to S172.

When it is determined in S171 that the compressed image is not being distributed, in S172, the control unit 13 detects the moving object so that the processing time of the moving object detection by the moving object detection unit 15 becomes standard, that is, operates in the first mode. Control unit 15;
On the other hand, when it is determined in S171 that the compressed image is not being distributed, in S173, the control unit 13 reduces the processing time of the moving object detection by the moving object detection unit 15, that is, operates in the second mode. The motion detection unit 15 is controlled.
At S174, the control unit 13 controls the compression encoding unit 16 to compress the image buffered in the DRAM 143 with reference to the moving body region information output by the moving body detection unit 15 at S172 or S173.

  As a result, even if the network camera 1 is distributing a compressed image to an external image acquisition device, the compression encoding unit 16 refers to moving body area information generated with a smaller delay time, and captures the captured image. Compression, so the delivery delay of the compressed image is reduced. Furthermore, even if the compressed image is being delivered to an external device, as in the case where it is not being delivered, the compression rate for the moving object region in the captured image can be set low compared to the background region. The image quality of the area can be maintained.

  As described above, according to the present embodiment, even when the network camera 1 is delivering a compressed image to an external device, the image quality of the moving object area to be monitored while reducing the delivery delay of the compressed image. The storage area can be reduced by making the background area highly compressed while maintaining the.

Embodiment 6
Hereinafter, the sixth embodiment will be described in detail with reference to FIG. 18, with regard to only differences from the above fifth embodiment. In this embodiment, when the network camera 1 is distributing a compressed image, it is determined whether the frame rate (imaging cycle) set in the imaging unit 11 is longer than a predetermined time. Then, if the frame rate is longer than a predetermined time, standard moving object detection processing is executed as in the case where delivery is not in progress. As a result, when the compressed image is being distributed, when the imaging cycle is long and the effect of shortening the moving object detection can not be obtained, the moving object area information is generated from more image frames by the standard moving object detection processing. The image quality of the moving object area to be monitored can be further improved.

The configuration of the network camera 1 of the sixth embodiment is the same as that of the fifth embodiment shown in FIG.
The operation and various setting information of the network camera 1 in the sixth embodiment are the same as those in the fifth embodiment.
Next, with reference to FIG. 18, the processing procedure of the compression coding switching process according to the present embodiment will be described.
In S181, the control unit 13 determines whether or not the compressed image is being distributed to an external device such as the administrator terminal device 2 via the network. Specifically, the control unit 13 distributes the compressed image when the distribution start command is received from the external device such as the administrator terminal device 2 via the communication unit 12 and the distribution stop command has not been received yet. Determined to be medium. If it is determined that the compressed image is being distributed (S181: Y), the process proceeds to S184, and if it is determined that the compressed image is not being distributed (S181: N), the process proceeds to S182.

When it is determined in S181 that the compressed image is not being distributed, in S182, the control unit 13 detects the moving object so that the processing time of the moving object detection by the moving object detection unit 15 becomes standard, that is, operates in the first mode. Control unit 15;
At S183, the control unit 13 controls the compression encoding unit 16 to compress the image buffered in the DRAM 143 with reference to the moving body area information output by the moving body detecting unit 15 at S182.

On the other hand, when it is determined in S181 that the compressed image is being distributed, in S184, the control unit 13 determines whether the imaging cycle defined by the imaging frame rate set in the imaging unit 11 is longer than a predetermined time. Determine If it is determined that the imaging cycle is equal to or less than the predetermined time (S181: N), the process proceeds to S185. If it is determined that the imaging cycle is longer than the predetermined time (S181: Y), the process proceeds to S186.
Here, for a predetermined time to be compared with the imaging cycle, the supervisor of the administrator terminal device 2 remotely operates the network camera 1 as a result of the operation and, as a result, the compressed image to be delivered in the live stream changes. An acceptable delay time may be set. Although the threshold value of the imaging cycle is set to 60 msec as one example in FIG. 18, the present invention is not limited to this.
The shortened moving object detection process may have lower accuracy in moving object detection than standard moving object detection processing. In the present embodiment, when the imaging cycle defined by the imaging frame rate set in the imaging unit 11 is longer than the shortened delay time, the moving body detection unit 15 executes standard moving body detection processing. If the imaging cycle is longer than the shortened delay time, the shortening effect of the moving body detection process can not be obtained, and it is more effective to execute the standard moving body detection process because the accuracy of the moving body detection is improved.

When it is determined that the imaging cycle is equal to or less than the predetermined time, in S185, the control unit 13 operates so as to shorten the processing time of the moving object detection by the moving object detection unit 15, that is, operate in the second mode. The detection unit 15 is controlled.
On the other hand, when it is determined that the imaging cycle is longer than the predetermined time, the control unit 13 controls the processing time of the moving object detection by the moving object detection unit 15 in S186, ie, operates in the first mode. The detection unit 15 is controlled.
At S187, the control unit 13 controls the compression encoding unit 16 to compress the image buffered in the DRAM 143 with reference to the moving body area information output by the moving body detection unit 15 at S185 or S186.

  As described above, according to the present embodiment, more images can be obtained by the standard moving body detection processing when the imaging cycle is long and the effect of shortening the moving body detection can not be obtained while distributing the compressed image. Generate moving body area information from the frame. This can further improve the image quality of the moving object area to be monitored.

<Modification>
Further, the configuration of the functional blocks shown in FIG. 3 and the like is an example, and a plurality of functional blocks may constitute one functional block, or any functional block may be divided into blocks performing a plurality of functions. May be In addition, at least a part of the functional blocks illustrated in FIG. 3 and the like may be implemented as hardware. In the case of hardware implementation, for example, by using a predetermined compiler, a dedicated circuit may be automatically generated on a field-programmable gate array (FPGA) from a program for realizing each step. Further, as in the case of the FPGA, a gate array circuit may be formed and implemented as hardware, or may be realized by an application specific integrated circuit (ASIC).

Other Embodiments
In addition, it is possible to implement | achieve each embodiment mentioned above combining the plurality.
The present invention can also be realized by a program that implements one or more of the functions of the embodiments described above. That is, it is possible to supply the program to a system or apparatus via a network or storage medium, and implement processing by one or more processors in the computer (or CPU, MPU, etc.) of the system or apparatus reading and executing the program. is there. Alternatively, the program may be provided by being recorded on a computer readable recording medium.
Further, the functions of the embodiment are not limited to those realized by executing the program read by the computer. For example, an operating system (OS) or the like operating on a computer may perform part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiment may be realized by the processing.

  DESCRIPTION OF SYMBOLS 1 network camera 2 administrator terminal device 3 server 4 NAS 8 cloud server 11 imaging unit 12 communication unit 13 control unit 14 storage unit 15 moving body detection unit 16 compression encoding unit 17 voice input unit 18 voice detection unit 111 lens 112 imaging device 113 development processing unit 141 frame buffer 142 flash memory 143 DRAM

Claims (14)

Imaging means for imaging a subject and acquiring a captured image;
A detection unit that detects a moving body in the captured image captured by the imaging unit;
Compression means for compressing the captured image acquired by the imaging means to generate a compressed image;
Distribution means for distributing the compressed image generated by the compression means to an external device via a network;
When the compressed image is not being delivered to the external device, the moving object area, which is an area in which the moving object is detected by the detecting means, is compressed in the first mode, and background areas other than the moving object area in the captured image Control means for controlling the compression means to generate the compressed image so as to compress in a second mode different from the first mode;
An imaging apparatus comprising: When the compressed image is not being delivered to the external device, the control unit controls the compression unit to generate the compressed image so that the compression ratio of the moving object region is lower than the compression ratio of the background region.
The imaging device according to claim 1, When the compressed image is being delivered to the external device, the control unit controls the compression unit to generate the compressed image so as to compress the captured image at a single compression rate.
The imaging device according to claim 2, characterized in that: When the compressed image is being distributed to the external device, the control unit controls the detection unit to generate a smaller number of image frames from the number of image frames than when the compressed image is not being distributed to the external device. Generate information,
The compression means is controlled to generate the compressed image with reference to the information of the moving body region generated by the detection means.
The imaging device according to claim 1, It further comprises storage means for storing the compressed image generated by the compression means,
The control means compresses the moving body area in the first mode when the compressed image is being stored in the storage means and the compressed image is not being delivered to the external device, The compression unit is controlled to generate the compressed image so that the background area is compressed in the second mode.
The imaging device according to any one of claims 1 to 4, characterized in that: The control means compresses the moving body area in the first mode and compresses the background area in the second mode when the distribution means is distributing the compressed image to the external device for recording. Control the compression means to generate the compressed image,
The imaging device according to any one of claims 1 to 5, characterized in that. It further comprises buffer means for buffering the captured image acquired by the imaging means,
The imaging means outputs the captured image for a predetermined time, and the detection means outputs information on the moving object region for the predetermined time to the buffer means.
The compression means obtains the captured image and the information of the moving body area from the buffer means, and generates the compressed image.
The imaging device according to any one of claims 1 to 6, characterized in that: The apparatus further comprises setting means for setting, to the detection means, a moving object in the captured image to be detected by the detection means or a sound corresponding to the captured image.
The imaging device according to any one of claims 1 to 7, characterized in that: When the compressed image is being delivered to the external device and the imaging cycle of the imaging device is longer than a predetermined time, the control means controls the detection means to deliver the compressed image to the external device. Generating information of the moving body area from the same number of image frames as in the case of
The imaging device according to claim 4, It further comprises storage means for storing the compressed image generated by the compression means,
The delivery means is a second compressed image different from the compressed image generated by the compression means when the compressed image is not being delivered to the external device when the storage means has stored the compressed image. Distributed to the external device,
The imaging device according to any one of claims 1 to 9, characterized in that. The control means changes the case where the compressed image is not being delivered to the external device, and while the uploading of the compressed image to the external device or the storage of the compressed image to the storage device is started, within a predetermined time. Controlling the compression unit to generate the compressed image so as to compress the moving body region in the first mode and compress the background region in the second mode;
The imaging device according to any one of claims 1 to 10, characterized in that: A detection unit that detects a moving body in a captured image captured by the imaging unit;
Compression means for compressing the captured image to generate a compressed image;
Distribution means for distributing the compressed image generated by the compression means to an external device via a network;
When the compressed image is not being delivered to the external device, the moving object area, which is an area in which the moving object is detected by the detecting means, is compressed in the first mode, and background areas other than the moving object area in the captured image Control means for controlling the compression means to generate the compressed image so as to compress in a second mode different from the first mode;
An encoding apparatus comprising: Imaging a subject to obtain a captured image;
Detecting a moving body in the captured image captured;
Compressing the acquired captured image to generate a compressed image;
Delivering the generated compressed image to an external device via a network,
In the step of generating the compressed image, when the compressed image is not being delivered to the external device, a moving object region which is a region in which the moving object is detected is compressed in a first mode, and the moving object in the captured image is Generating the compressed image to compress background areas other than the area in a second mode different from the first mode;
And controlling the imaging device. A program for causing a computer to function as each means of the device according to any one of claims 1 to 12.

Images