JP4479819B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method. Specifically, the presentation image presented by the presentation device is detected from the captured image, and when the detection result of the presentation image satisfies a preset condition, the imaging mode is switched to the presentation image imaging mode. It is intended to easily obtain an optimal captured image when imaging is performed.

  2. Description of the Related Art In recent years, presentations have been frequently performed by connecting a computer device to a projector and projecting materials and images on a screen. Along with this, there are increasing opportunities to image the state using an imaging device such as a video camera. The purpose of imaging includes the use of saving the contents of seminars and meetings as minutes, and the relay conference in which the participants of the conference take images of each other at a remote location. Further, not only images projected on a screen but also a monitor of a computer device and a display screen of a television device are often imaged.

  Here, for example, when imaging a presentation (hereinafter referred to as “presentation”), if the surroundings are dark, strobe light emission may occur and the surroundings may be disturbed. In order to solve such a problem, the invention of Patent Document 1 discloses that an imaging mode for imaging a presentation (hereinafter, presentation) is provided, and the imaging mode is manually switched when imaging a presentation. Yes. In the imaging mode when imaging this presentation, ISO flashing and shutter sound are prohibited so as not to disturb the surroundings, and even if the shutter speed is slowed down by prohibiting flash lighting, ISO is not caused. The sensitivity is set high.

JP 2007-195029 A

  By the way, when an image projected on a screen, a display screen or the like (hereinafter referred to as “presentation image”) is captured by a conventional imaging device, the brightness of the presentation image in the captured image and other images (hereinafter referred to as “background image”) Depending on the color difference, an optimal captured image may not be obtained. For example, there may be a case where a whiteout occurs in a region of a presentation image in a captured image, or a color change occurs because white balance adjustment is not performed correctly. However, it is not possible to prevent overexposure, discoloration, or the like simply by switching the imaging mode to prohibit strobe light emission or increasing the ISO sensitivity. Furthermore, since switching of the shooting mode is performed manually by the user, when capturing a presentation and capturing a captured image that is a moving image, an imaging operation centered on the presentation image and an imaging operation centered on the presenter are continuously performed. In this case, since the imaging mode must be switched each time, the operation becomes very complicated.

  Accordingly, the present invention provides an imaging apparatus and an imaging method that can easily obtain an optimal captured image.

  The concept of the present invention is to detect a presentation image presented by a presentation device from a captured image, for example, an image projected on a screen by a projector, and when the presentation image is detected, overexposure of the presentation image and white balance adjustment The imaging mode is automatically switched so that the image is not correctly performed and the presented image does not change color.

  An imaging apparatus according to the present invention detects an image presented by an image presentation device from the captured image based on an imaging unit that captures an image of a subject and obtains a captured image, and a difference in luminance or color in the captured image. When the detection result of the presentation image to be performed and the detection result of the presentation image satisfy a preset condition, the imaging mode is switched to the presentation image imaging mode, and at least a detection area for calculating a detection value used for exposure control Any of the change and white balance adjustment for each region in the captured image and other image regions excluding the presented image and gamma correction in each region in the captured image for the region of the presented image and the other image And a control unit for performing the above.

  In addition, the imaging method according to the present invention is presented by the image presentation device from the captured image based on an imaging step of capturing an image of a subject by an imaging unit to obtain a captured image and a difference in luminance or color in the captured image. When a presentation image detection step for detecting a presentation image and the detection result of the presentation image satisfy a preset condition, the imaging mode is switched to the presentation image imaging mode, and at least a detection value used for exposure control is calculated. Change of detection area for each region, white balance adjustment for each region of the captured image in the captured image and other image regions excluding the presented image, and for each region in the captured image for the region of the presented image and the other image And a control step for performing any one of the gamma corrections.

  In the present invention, when the process of detecting the presentation image from the imaging apparatus is performed and the detection result of the presentation image satisfies a preset condition, for example, the first presentation image detection and the first are more accurate than the first. When the second presentation image detection for detecting the presentation image is performed and the first presentation image detection satisfies a preset condition, the imaging mode is switched to the presentation image imaging mode. In addition, the detection area is changed. When the imaging mode is switched to the presentation image imaging mode, white balance adjustment and gamma correction for each region are performed based on the detection result obtained by the second presentation image detection. Furthermore, when the detection result does not satisfy a preset condition or when a face image is detected at the center of the captured image, the presentation image capturing mode is terminated.

  According to the present invention, when the process of detecting the presentation image presented by the presentation device from the captured image is performed, and the detection result satisfies a preset condition, the imaging mode is switched to the presentation image imaging mode. Thus, at least a detection region for calculating a detection value used for exposure control, a white balance adjustment for each region of the presented image and the other image region in the captured image, and the presented image and the other in the captured image One of the gamma corrections for each area of the image area is performed. For this reason, an optimal captured image can be easily obtained when the presentation image is captured.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows the configuration of the imaging apparatus. The subject light is incident on the imaging element 211 of the imaging unit 21 such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor via the lens unit 11 and the aperture mechanism 12.

  The imaging element 211 converts the imaging light of the subject image formed on the light receiving surface into an electrical signal for each pixel to generate an image signal of three primary colors. Further, the generated image signal is supplied to a CDS (Correlated Double Sampling circuit) unit 212.

  The CDS unit 212 removes noise from the image signal supplied from the image sensor, and supplies the image signal after the noise removal to the amplification unit 213. The amplification unit 213 adjusts the gain of the image signal supplied from the CDS unit 212 and supplies the image signal Sa after gain adjustment to the A / D (Analog / Digital) conversion unit 23.

  The timing signal generator 22 generates a drive signal TD for generating an image signal by the image sensor 211 and supplies the drive signal TD to the image sensor 211. The timing signal generation unit 22 controls the exposure time (effective charge accumulation time obtained by photoelectric conversion) in the image sensor 211 by the drive signal TD.

  The A / D converter 23 converts the analog image signal Sa into a digital image signal DVa and supplies the digital image signal DVa to the presentation image detector 31, the detector 32, and the image processor 35.

  The presentation image detection unit 31 performs a process of detecting a presentation image from a captured image based on the image signal DVa, generates a detection signal JA indicating a detection result, and supplies the detection signal JA to the control unit 51.

  FIG. 2 shows the configuration of the presentation image detection unit 31. The conversion processing unit 311 of the presentation image detection unit 31 converts the image signal DVa of the three primary colors into a luminance signal DY and a color difference signal DC, the luminance signal DY is converted into a binarization processing unit 312, and the color difference signal DC is converted into a binarization processing unit. 313.

  The binarization processing unit 312 performs processing for binarizing the luminance signal DY, and supplies the obtained luminance binarized signal BY to the signal selection unit 314. In addition, the binarization processing unit 313 performs processing for binarizing the color difference signal DC and supplies the obtained color difference binarized signal BC to the signal selection unit 314. The binarization processing unit 312 binarizes the luminance signal DY by comparing the luminance signal DY with a threshold value, and generates a luminance binarized signal BY. The binarization processing unit 313 binarizes the color difference signal DC by comparing the color difference signal DC with a threshold value, and generates a color difference binarized signal BC. In addition, the binarization processing units 312 and 313 may set the threshold to a fixed level in advance, and set the threshold according to the captured image so that the presented image can be detected from the captured image with high accuracy. It may be a thing.

  The signal selection unit 314 selects the luminance binarized signal BY or the color difference binarized signal BC, and supplies the selected binarized signal to the horizontal distribution acquisition unit 315 and the vertical distribution acquisition unit 316.

  When the binarized signal is the luminance binarized signal BY, the horizontal distribution acquisition unit 315 counts pixels in which the signal level of the luminance binarized signal BY is higher than a threshold value for each line, thereby calculating the horizontal distribution. Then, a horizontal distribution signal DSh indicating this distribution is supplied to the detection processing unit 317. In addition, when the binarized signal is the color difference binarized signal BC, the horizontal distribution acquisition unit 315 counts pixels in which the signal level of the color difference binarized signal BC is higher than a threshold value for each line. A distribution is obtained, and a horizontal distribution signal DSh indicating this distribution is supplied to the detection processing unit 317.

  When the binarized signal is the luminance binarized signal BY, the vertical distribution acquisition unit 316 counts pixels in which the signal level of the luminance binarized signal BY is higher than the threshold value in the vertical direction, An orthogonal vertical distribution is obtained, and a vertical distribution signal DSv indicating this distribution is supplied to the detection processing unit 317. Further, when the binarized signal is the color difference binarized signal BC, the vertical distribution acquisition unit 316 obtains a vertical distribution by counting pixels whose signal level of the color difference binarized signal BC is higher than a threshold value. The vertical distribution signal DSv indicating this distribution is supplied to the detection processing unit 317.

  The detection processing unit 317 detects the upper side and the lower side based on the horizontal distribution signal DSh supplied from the horizontal distribution acquisition unit 315, and detects the left side and the right side based on the vertical distribution signal DSv supplied from the vertical distribution acquisition unit 316. Perform the process. Furthermore, when the detection processing unit 317 determines that the detected upper side, lower side, left side, and right side correspond to each side of the presented image, the image surrounded by the detected upper side, lower side, left side, and right side. Is detected as a presentation image, and a detection signal JA indicating that the position of the presentation image or the presentation image has been detected is generated and supplied to the control unit 51.

  In addition, the detection processing unit 317 controls the signal selection operation of the signal selection unit 314 by the control signal CN, and first selects the luminance binarized signal BY and cannot detect the presentation image from the luminance binarized signal BY. At this time, by causing the signal selection unit 314 to select the color difference binarized signal BC, a process of detecting a presentation image from the color difference binarized signal BC as well as the luminance binarized signal BY is performed.

  The detection unit 32 generates a detection signal JB indicating the brightness of the captured image from the supplied image signal DVa and supplies the detection signal JB to the control unit 51. For example, the detection unit 32 divides the captured image into m × n blocks, and calculates the sum of signals indicating the brightness of each pixel in each block. Further, the detection unit 32 weights the calculated sum for each block, and supplies the detection signal JB indicating the detection value to the control unit 51 using the addition value of the weighted sum as a detection value. Further, the detection unit 32 changes the detection region based on the control signal CTe from the control unit 51. That is, a block with a weight of “0” is not used for calculation of a detection value, and a block with a high weight has a higher contribution ratio to the detection value. Therefore, the detection area is changed by switching the weighting.

  FIG. 3 shows the configuration of the detection unit 32. The block processing unit 321 of the detection unit 32 performs processing for dividing the captured image into m × n blocks, and supplies the image signal DVa to the block-by-block luminance calculation unit 322 as the image signal DBV for each block.

  The block-by-block luminance calculation unit 322 generates a block-by-block luminance value signal BT in which the luminance of each pixel in the block is totaled based on the image signal DBV and supplies the block-by-block luminance value signal BT to the multiplier 324.

  The weighting factor setting unit 323 sets a weighting factor W for each of the blocks based on the control signal CTe supplied from the control unit 51 and supplies the set weighting factor W to the multiplier 324.

  The multiplier 324 multiplies the block-by-block luminance value signal BT supplied from the block-by-block luminance calculation unit 322 by the weighting factor W set by the weighting factor setting unit 323 and supplies the result to the detection value calculation unit 325.

  The detection value calculation unit 325 adds the multiplication result of the block-specific luminance value signal BT and the weighting coefficient W in the captured image, and supplies the addition result to the control unit 51 as the detection signal JB. The detection value calculation unit 325 generates a detection signal JB for each captured image.

  The image processing unit 35 performs various image processing on the image signal DVa supplied from the A / D conversion unit 23. For example, the white balance (WB) adjustment unit 351 performs white balance adjustment so that a white subject in the captured image is correctly displayed as white. Further, when the white balance adjustment unit 351 can divide the captured image into a region of the presentation image and a region of other images (hereinafter referred to as “background image”) based on the detection signal JA, the white balance adjustment unit 351 converts the captured image into a presentation image in the captured image based on the detection signal JA. Corresponding white balance adjustment and white balance adjustment corresponding to the background image area are performed for each area. The image processing unit 35 performs gamma correction on the image signal after white balance adjustment. For example, when the gamma correction unit 352 can divide the captured image into the region of the presentation image and the region of the background image based on the detection signal JA from the presentation image detection unit 31, the overexposure does not occur in the region of the presentation image. Gamma correction is performed, and gamma correction is performed so that no blackout occurs in the background image area.

  FIG. 4 shows the configuration of the white balance adjustment unit 351. The white balance detection unit 351a extracts an image signal in the achromatic region from the image signal DVa, and supplies it as a detection signal JC to the white balance correction coefficient setting unit 351b. The white balance detection unit 351a switches the achromatic color region between the presented image region and the background image region based on the detection signal JA.

  The white balance correction coefficient setting unit 351b determines the white balance correction coefficient K from the detection signal JC supplied from the white balance detection unit 351a and supplies the white balance correction coefficient K to the correction processing unit 351c.

  The correction processing unit 351c performs white balance adjustment by multiplying the image signal DVa by the white balance correction coefficient K, and supplies the image signal DVb after the white balance adjustment to the gamma correction unit 352 shown in FIG.

  Further, the image processing unit 35 converts, for example, an image signal of three primary colors subjected to white balance adjustment, gamma correction, and the like into an image signal such as a luminance signal or a color difference signal according to an image encoding technique of a still image or a moving image. Then, compression / decompression processing is performed, and the obtained still image or moving image compression-encoded signal GDa is output to the recording / reproducing unit 37. Furthermore, a process of outputting an image signal and / or a compression-coded signal of a still image or a moving image that has not been subjected to compression / decompression processing to an external device (not shown) is performed. When the compressed encoded signal GDb is supplied from the recording / reproducing unit 37, the image processing unit 35 performs processing to return the compressed encoded signal GDb to the image signal before the compression encoding process. Further, the image processing unit 35 generates a display image signal HG from the image signal DVa and supplies it to the display unit 38. The image processing unit 35 also generates a display image signal HG in which information indicating that the presentation image capturing mode is set is displayed on the screen based on the display signal HE supplied from the control unit 51. .

  The recording / reproducing unit 37 records the compressed encoded signal GDa supplied from the image processing unit 35 on a recording medium such as a semiconductor memory, an optical disk, or a magnetic tape. Further, the compressed encoded signal GDb recorded on the recording medium is read and supplied to the image processing unit 35.

  The display unit 38 is configured using, for example, a liquid crystal display element or the like, and performs image display based on the display image signal HG supplied from the image processing unit 35.

  The lens driving unit 41 drives a focus lens (not shown) of the lens unit 11 so that the subject image is focused on the light receiving surface of the image sensor 211. Further, the zoom lens (not shown) of the lens unit 11 is driven so that the size of the subject image formed on the light receiving surface of the image sensor 211 becomes a desired size.

  The aperture driving unit 42 drives the aperture mechanism 12 to adjust the amount of light of the subject image formed on the light receiving surface of the image sensor 211.

  The strobe 45 emits strobe light so that a captured image with a desired brightness can be obtained. The face detection unit 46 performs face detection using the image signal DVa and supplies the face detection signal FE to the control unit 51.

  A user interface unit 52 and a motion detection unit 53 are connected to the control unit 51. The user interface unit 52 includes an operation key and an interface for inputting a remote operation signal from the outside. The user interface unit 52 generates an operation signal PS corresponding to a user operation and supplies the operation signal PS to the control unit 51. The motion detection unit 53 is for detecting the movement of the captured image caused by the motion of the imaging device 10, and is configured using, for example, an acceleration sensor or a gyro sensor. The motion detection unit 53 detects the motion of the imaging device 10 and supplies a sensor signal SS indicating the motion detection result to the control unit 51.

  The control unit 51 includes a CPU (Central Processing Unit) 511, a ROM (Read Only Memory) 512, a RAM (Random Access Memory) 513, and the like. The CPU 511 executes a program stored in the ROM 512, generates a control signal corresponding to the operation signal PS, and supplies the control signal to each unit, thereby controlling the operation of the imaging device according to a user operation. To do. The RAM is used as a working area for temporarily storing information when performing various controls.

  Further, the control unit 51 generates a control signal so that the detection value indicated by the detection signal JB supplied from the detection unit 32 becomes a preset reference value, generates a control signal, and generates an exposure adjustment unit. The desired brightness in the captured image is set to the optimum brightness. The exposure adjustment unit adjusts at least one of the amount of light incident on the imaging unit 21, the gain of the image signal Sa output from the imaging unit 21, and the exposure time when the imaging unit 21 obtains the image signal Sa. The mechanism 12, the aperture driving unit 42, the timing signal generating unit 22, and the amplifying unit 213 correspond to an exposure adjusting unit. For example, the control unit 51 generates a control signal CTb based on the detection signal JB and supplies the control signal CTb to the aperture driving unit 42, so that the amount of light incident on the imaging unit 21 so that a desired subject in the captured image has optimum brightness. Adjust. In addition, the control unit 51 generates a control signal CTc based on the detection signal JB and supplies the control signal CTc to the timing signal generation unit 22, thereby adjusting the exposure time so that a desired subject in the captured image has optimum brightness. In addition, the control unit 51 generates the control signal CTd based on the detection signal JB and supplies the control signal CTd to the amplification unit 213, thereby adjusting the gain of the image signal Sa so that a desired subject in the captured image has an optimal brightness. .

  In addition, the control unit 51 controls the detection operation of the presentation image in accordance with the sensor signal SS from the motion detection unit 53 and detects that the presentation image is detected based on the detection signal JA supplied from the presentation image detection unit 31. Is determined, a process of switching to a presentation image imaging mode in which the presentation image in the captured image is an optimal image is performed. In addition, the control unit 51 performs imaging mode switching processing based on the face detection signal FE. Furthermore, the control unit 51 notifies the detection unit 32 and the image processing unit 35 of what imaging mode is set by using the control signal CTe. Further, when the user designates a region of the presentation image, the control unit 51 supplies information indicating the designated region to the presentation image detection unit 31 as a control signal CTf.

  Here, when the presentation image capturing mode is set by the control unit 51, the detection unit 32 switches the detection region so that the presented image in the captured image has an optimum brightness. The image processing unit 35 performs white balance adjustment that corrects the color reproduction of the presentation image and the background image, and performs gamma correction so that the presentation image is not overexposed. Gamma correction is performed to prevent blackout.

  The control unit 51 generates a control signal CTa and supplies the control signal CTa to the lens driving unit 41 to drive the focus lens so that the subject image is focused on the light receiving surface of the image sensor 211. Processing or processing for driving the zoom lens so as to obtain a captured image having a desired angle of view is performed. The control unit 51 also generates a control signal CTh and supplies it to the strobe 45 to control strobe light emission.

  Next, the operation of the imaging apparatus will be described using the flowchart of FIG. In step ST <b> 1, the control unit 51 causes the presentation image detection unit 31 to perform presentation image detection.

  FIG. 6 is a flowchart showing the presentation image detection operation. In step ST101, the presentation image detection unit 31 performs a conversion process. The presentation image detection unit 31 generates a luminance signal DY and a color difference signal DC from the three primary color image signals DVa, and proceeds to step ST102.

  In step ST102, the presentation image detection unit 31 performs binarization processing. The presented image detection unit 31 binarizes the luminance signal DY and the color difference signal DC, generates a luminance binarized signal and a color difference binarized signal, and proceeds to step ST103.

  In the binarization process, as described above, the luminance signal DY and the color difference signal DC are binarized by comparing the luminance signal DY and the color difference signal DC with the threshold value, and the luminance binarized signal BY and the color difference binarization are performed. A signal BC is generated. The threshold used for the binarization process may be set at a fixed level in advance, or may be set according to the captured image so that the presented image can be detected from the captured image with high accuracy.

  Here, when setting a threshold value according to a captured image, for example, the threshold value is set using a statistical method. As a statistical method, a mode method, a discriminant analysis method, or the like is used. In the mode method, the brightness level histogram of the captured image has bimodality, and when the portion of the presented image is separated from the portion of another image (hereinafter referred to as “background image”), the valley position is set as a threshold value. It is a method to set as. The discriminant analysis method is a method in which the histogram is divided into two groups according to the luminance level, and the division level that maximizes the variance between the groups is set as a threshold value.

  In step ST103, the presented image detection unit 31 selects the luminance binarized signal BY and proceeds to step ST104.

  In step ST104, the presentation image detection unit 31 acquires a horizontal distribution and proceeds to step ST105. In step ST105, the presentation image detection unit 31 acquires the vertical distribution and proceeds to step ST106.

  FIG. 7 shows a process for acquiring a horizontal distribution and a vertical distribution. FIG. 7A shows a captured image, and FIG. 7B shows a captured image after binarization processing. The presentation image detection unit 31 obtains a horizontal distribution shown in FIG. 7C by counting pixels in which the signal level of the luminance signal DY is higher than the threshold value for each line. Further, by counting pixels in which the signal level of the luminance signal DY is higher than the threshold value in the vertical direction, as shown in FIG. 7D, a distribution in the vertical direction perpendicular to the horizontal direction is obtained.

  In step ST <b> 106, the presented image detection unit 31 performs processing for detecting the upper side and the lower side of the presented image, and proceeds to step ST <b> 107.

  8 and 9 show the detection processing of the upper side and the lower side of the presented image. 8A shows a captured image after binarization processing, and FIG. 8B shows the horizontal distribution acquired in step ST104. Here, the presentation image detection unit 31 divides the horizontal distribution into a plurality of regions, and obtains the sum of target pixels for each region. For example, the region is divided into regions REh1, REh2, REh3, REh4, and REh5, and the sum of target pixels is obtained for each region. Next, the presented image detection unit 31 sequentially selects areas from the upper end to the lower end of the captured image, compares the sum of the target pixels in the selected area with a threshold value, and detects an area where the sum exceeds the threshold value. By performing such processing, it is possible to detect a region including the upper side of the presented image. Similarly, the presented image detection unit 31 sequentially selects areas from the lower end to the upper end of the captured image, compares the sum of the target pixels in the selected area with the threshold, and detects an area where the sum exceeds the threshold. By performing such processing, it is possible to detect a region including the lower side of the presented image.

  By the way, when the side of the presentation image is close to the boundary of the region, the sum of the target pixels in the region is greatly different, and it may be difficult to detect the region including the side of the presentation image. For example, if the upper side of the presentation image is in the region REh (n) and close to the region REh (n + 1), the sum of the target pixels in the region does not become a large value. Here, if a region where the sum exceeds the threshold value is detected, there is a possibility that the upper side of the presented image is erroneously determined to be included in the region REh (n + 1). Therefore, when the horizontal distribution is divided into a plurality of regions, the regions are provided so as to have portions overlapping with other regions. For example, regions are provided as shown in regions REh1 to REh5 and REh1s to REh4s. In this case, even if the upper side of the presented image is near the boundary between the region REh1 and the region REh2, and the position is in either the region REh1 or the region REh2, the total sum of the target pixels in the region REh1s does not vary greatly. Therefore, if the threshold is set so that the upper side of the presentation image is included in the region REh1s when the total sum of the target pixels in the region REh1s at this time is compared with the threshold, the region including the side of the presentation image Can be detected correctly. The threshold value may be fixed in advance, or may be set from the sum of target pixels in each region using a statistical method as described above.

  Next, in the region REh1s including the side of the detected presentation image, for example, the region REh1s including the upper side, the presented image detection unit 31 sequentially calculates the sum of the target pixels for each line in the horizontal distribution for each line from the upper end to the lower end of the captured image. The line whose sum exceeds the threshold is used as the upper side La of the presentation image. Similarly, in the region REh4s including the lower side, the presented image detection unit 31 sequentially uses the sum of the target pixels for each line in the horizontal distribution from the lower end to the upper end of the captured image for each line and compares it with the threshold value. A line exceeding the threshold is set as the lower side Lb of the presentation image. FIG. 9A shows a captured image after binarization processing, and FIG. 9B shows a horizontal distribution. Here, when the region REh1s is determined as the region including the upper side, the sum of the target pixels for each line in the region REh1s is sequentially used for each line from the upper end to the lower end of the captured image and compared with the threshold value. A line exceeding the upper limit La is set as the upper side La of the presentation image. Similarly, when the region REh4s is determined as the region including the lower side, the sum of the target pixels for each line in the region REh4s is sequentially used for each line from the lower end to the upper end of the captured image and compared with the threshold value. A line that exceeds is set as the lower side Lb of the presentation image.

  In this way, even if the total of one line becomes large due to noise or the like by detecting the area including the upper side and the lower side and detecting the upper side and the lower side of the presentation image from the detected area, this line Can be prevented from being erroneously determined as the upper side of the presented image.

  In step ST107, the presentation image detection unit 31 performs detection processing of the left side and the right side of the presentation image, and proceeds to step ST108.

  10 and 11 show the processing for detecting the left and right sides of the presented image. 10A shows a captured image after binarization processing, and FIG. 10B shows the vertical distribution acquired in step ST105. Here, the presentation image detection unit 31 divides the vertical distribution into a plurality of regions, for example, regions REv1 to REv6 and REv1s to REv5s, and obtains the sum of the target pixels for each region. Next, the presented image detection unit 31 sequentially selects areas from the left end to the right end of the captured image, compares the sum of the target pixels in the selected area with a threshold value, and detects an area where the sum exceeds the threshold value. By performing such processing, the region REv2 including the left side of the presented image can be detected. Similarly, the presented image detection unit 31 sequentially selects regions from the right end to the left end of the captured image, compares the sum of the target pixels in the selected region with a threshold value, and detects a region where the sum exceeds the threshold value. By performing such processing, the region REv5 including the right side of the presented image can be detected. The threshold value at this time may be fixed in advance, or may be set from the sum of target pixels in each region using a statistical method as described above.

  Next, the presented image detection unit 31 sequentially uses the sum of the target pixels for each pixel position in the vertical distribution in the detected region, for example, the region REv2 including the left side, sequentially from the left end to the right end of the captured image for each pixel position. In comparison, the pixel position where the sum exceeds the threshold value is defined as the left side Lc of the presentation image. Similarly, in the region REv5 including the right side, the presented image detection unit 31 sequentially uses the sum of the target pixels for each pixel position in the vertical distribution for each pixel position in the direction from the right end to the left end of the captured image and compares it with the threshold value. The pixel position where the sum exceeds the threshold is defined as the right side Ld of the presented image.

  FIG. 11A shows a captured image after binarization processing, and FIG. 11B shows a vertical distribution. Here, when the region REv2 is determined as a region including the left side, the sum of the target pixels for each pixel position in the region REv2 is sequentially used from the left end to the right end of the captured image and compared with the threshold, and the sum exceeds the threshold. Let the pixel position be the left side Lc of the presentation image. Similarly, when the region REv5 is determined as the region including the right side, the sum of the target pixels for each pixel position in the region REv5 is sequentially used for each line from the right end to the left end of the captured image and compared with the threshold value. A line exceeding the threshold is set as the right side Ld of the presentation image.

  In step ST108, the presentation image detection unit 31 determines whether or not the presentation image is present. The presented image detection unit 31 determines the aspect ratio from the upper and lower sides and the left and right sides detected in steps ST106 and ST107, and this aspect ratio is the aspect ratio of the presented image, for example, the aspect ratio “used in television broadcasting”. When approximate to "4: 3", "16: 9", or the like, that is, when the determined aspect ratio is "4: 3 ± α", "16: 9 ± β", in steps ST106 and ST107 The area surrounded by the detected sides is determined as a presentation image, the presentation image detection process is terminated, and the process proceeds to step ST2 in FIG. “Α” and “β” are constants indicating an allowable range for determining that the image is a presentation image, and are set in advance. In this way, by determining whether or not the aspect ratio approximates the desired aspect ratio, it is possible to easily determine the area of the presentation image from the captured image when a presentation image with a predetermined aspect ratio is captured. It becomes possible to do.

  In addition, when the presented image detection unit 31 cannot detect the upper and lower sides and the left and right sides of the presented image, the aspect ratio determined based on the detected upper and lower sides and the left and right sides approximates the desired aspect ratio. If not, the process proceeds to step ST109 assuming that the presented image is not detected.

  FIG. 12 is a diagram for explaining a determination operation as to whether or not the image is a presentation image. For example, as shown in FIGS. 12A and 12B, the upper side La and the lower side Lb, the left side Lc and the right side Ld are detected, and an image surrounded by the detected upper side, lower side, left side and right side is presented. When the aspect ratio of the image is not approximated, it is determined that the image is not a presentation image, and the process proceeds to step ST109.

  In step ST109, the presentation image detection unit 31 determines whether the presentation image is detected for the first time. When the presentation image is detected using the luminance binarized signal, the presentation image detection unit 31 proceeds to step ST110 because the presentation image is detected for the first time.

  In step ST110, the presentation image detection unit 31 selects the color difference binarized signal, returns to step ST104, and repeats the processing from step ST104.

  The presentation image detection unit 31 detects the presentation image based on the color difference binarized signal by repeating the processing from step ST104. The presented image detection unit 31 cannot detect the upper and lower sides and the left and right sides of the presented image, or the aspect ratio determined based on the detected upper and lower sides and the left and right sides approximates the desired aspect ratio. If not, the process proceeds from step ST108 to step ST109.

  In this way, if the presentation image is detected using not only the luminance binarized signal but also the color difference binary signal, for example, the luminance difference between the presented image and the background image is small. Even when it is difficult to distinguish between the presentation image and the background image based on this, when the presentation image is an image based on a color different from the background image, the presentation image can be detected from the difference in color. This presented image has a feature that it has a larger luminance value and a darker color than natural scenery. Here, the color density can be expressed using the magnitude of the color difference component in the color space of the YCC space or Lab space or the magnitude of the saturation in the HSV space. Therefore, if the presentation image is detected using the intensity of the color difference component and the magnitude of the saturation, the presentation image and the background image can be detected even if it is difficult to distinguish the presentation image and the background image based on the luminance difference. It becomes possible to classify, and a presentation image can be detected with high accuracy.

  In step ST109, the presentation image detection unit 31 determines whether the presentation image is detected for the first time. The presentation image detection unit 31 detects the presentation image using the luminance binarization signal and the color difference binarization signal. Since the presentation image is not detected for the first time, the process ends and step ST2 in FIG. 5 is performed. Proceed to

  The detection of the presented image is not limited to the process shown in FIG. For example, by performing edge detection of the captured image, a contour indicating the boundary between the presentation image and the background image is acquired, and straight lines indicating four vertices and four sides of the presentation image are obtained from the contour information, and the presentation image is obtained. It is good also as what detects.

  In step ST2 of FIG. 5, the control unit 51 determines whether or not the condition for switching to the presentation image capturing mode is satisfied. When it is determined that the condition for switching to the presentation image capturing mode is satisfied, the control unit 51 proceeds to step ST3, and when it is determined that the condition is not satisfied, the control unit 51 proceeds to step ST5.

  The control unit 51 determines whether or not the condition for switching to the presentation image capturing mode is satisfied according to the generation state of the frame in which the presentation image is detected. For example, the control unit 51 counts up every time a frame in which a presentation image is detected is generated, and counts down every time a frame in which a presentation image is not detected occurs. Note that the range of the count value is set to “0” to “about several tens”. Here, when the count value becomes equal to or larger than a threshold value set in advance for imaging mode determination (hereinafter referred to as “mode determination threshold value”), the process proceeds to step ST3. When the count value is less than the mode determination threshold value, step ST5 is performed. Proceed to

  Moreover, the control part 51 is good also as what sets the imaging mode based on the continuous number of the frames from which the presentation image was detected, or the continuous number of the frames from which the presentation image was not detected. For example, when the number of consecutive frames in which the presentation image is detected reaches a predetermined number, the imaging mode is switched from the normal imaging mode to the presentation image imaging mode. The imaging mode may be switched from the presentation image imaging mode to the normal imaging mode when the number of consecutive frames in which no presentation image is detected reaches a predetermined number. In this way, it is possible to stably switch the imaging mode according to the detection state of the presented image.

  In step ST3, the control unit 51 performs a presentation image capturing mode notification process and proceeds to step ST4. The control unit 51 sets the imaging mode to the presentation image imaging mode, and notifies the user that the imaging device is set to the presentation image imaging mode. For example, when the presentation image capturing mode is set, the control unit 51 generates the display signal HE and supplies the display signal HE to the image processing unit 35 to generate the display image signal HG from the image signal DVa and the display signal HE. Thus, information indicating that the captured image and the presentation image capturing mode are set is displayed on the screen of the display unit 38. FIG. 13 shows a case where an icon display MH indicating that the presentation image imaging mode is set is provided on the captured image displayed on the display unit 38.

  In step ST4, the control unit 51 sets the presentation image capturing mode operation setting, and proceeds to step ST7. For example, the control unit 51 controls the detection unit 32 to switch the detection region for calculating the detection value used when performing exposure control to a position where the influence of the presented image is small. The control unit 51 also controls the white balance adjustment unit 351 and the gamma correction unit 352 of the image processing unit 35 to perform white balance adjustment and gamma correction for each region of the presentation image and the background image.

  In step ST5, the control unit 51 performs notification of an imaging mode different from the presentation image imaging mode, for example, normal imaging mode notification processing, and proceeds to step ST6. The control unit 51 sets the imaging mode to another imaging mode, for example, the normal imaging mode, and notifies the user that the imaging apparatus is set to the normal imaging mode. For example, when the display image HE is set in the presentation image capturing mode, the control unit 51 supplies the display signal HE to the image processing unit 35 to perform icon display MH indicating that the presentation image capturing mode is set. The supply of the signal HE is stopped and the icon display MH is deleted, so that the user can identify that the normal imaging mode is set.

  In step ST6, the control unit 51 performs normal imaging mode operation setting and proceeds to step ST7. For example, the control unit 51 controls the detection unit 32 to switch the detection region so as to perform exposure control that places importance on the center of the captured image. In addition, the control unit 51 controls the white balance adjustment unit 351 and the gamma correction unit 352 of the image processing unit 35 to adjust the white balance for each captured image regardless of the region of the presented image or the background image. Enable gamma correction.

  FIG. 14 is a diagram for explaining the operation of the detection unit 32. The detector 32 divides the captured image into a plurality of blocks, for example (m × n), as shown in FIG. In addition, when the imaging mode is set to the presentation image imaging mode, the detection unit 32 increases the weight of the end block in the captured image as shown in FIG. 14B so that the influence of the presentation image is reduced. To do. In addition, when the imaging mode is set to the normal imaging mode, the detection unit 32 sets a weighting factor so that the weight of the central block in the captured image becomes higher as illustrated in FIG.

  When the weighting factor is set according to the imaging mode, the detection unit 32 provides a detection area in the captured image and generates a detection signal JB from the image signal in the detection area. That is, the detection unit 32 divides the captured image into blocks, weights the sum of the signals indicating the brightness of each pixel in the block, weights the weight coefficient W, adds the sum after weighting, and detects the detected value Vjb. , And a detection signal JB indicating the detection value Vjb is generated.

  Here, when the sum of the luminance signals of the pixels contained in the block of coordinates (a, b) is BT (a, b) and the weighting factor for the block of coordinates (a, b) is W (a, b) The detection unit 32 generates a detection value Vjb based on the equation (1).

  It should be noted that the normalized detection value Vjb can be obtained by normalizing the sum of the weight coefficients to 1.

  The control unit 51 performs exposure control based on the detection signal JB generated by the detection unit 32. That is, the control unit 51 generates at least one of the control signals CTb, CTc, and CTd, the generated control signal CTb is the diaphragm drive unit 42, the control signal CTc is the timing signal generation unit 22, and the control signal CTd is the amplification unit. Then, the aperture setting of the aperture mechanism 12 is adjusted, the exposure time is adjusted by the timing signal generation unit 22, and the gain of the image signal Sa is adjusted by the amplification unit 213 so as to obtain an appropriate exposure.

  FIG. 15 shows the position of the achromatic color and the definition range of the achromatic color (hereinafter referred to as “achromatic color region”) obtained by performing the white balance detection. Here, the achromatic color of the presented image on the (R / G)-(B / G) plane detected from the detection signal obtained by performing the white balance detection is the position ACpa, and the achromatic color of the background image is the position ACpb. In such a case, when the imaging mode is the presentation image imaging mode, the white balance adjustment unit 351 obtains a white balance correction coefficient from the achromatic position ACpa of the presentation image and presents it using the white balance correction coefficient. White balance adjustment is performed on the image area. Further, a white balance correction coefficient is obtained from the achromatic color position ACpb of the background image, and white balance adjustment is performed on the background image region using the white balance correction coefficient. In FIG. 15, an area ACS is an achromatic area in an image captured under a general light source (for example, sunlight or incandescent lamp), and an area ACR is an achromatic area experimentally obtained from an image projected on a screen. Show.

  When the imaging mode is set to the normal imaging mode, the white balance adjustment unit 351 detects the position of the achromatic color included in the region ACS from the captured image, and obtains a white balance correction coefficient from the detected position of the achromatic color. The white balance adjustment of the captured image is performed using the white balance correction coefficient. That is, when the imaging mode is set to the normal imaging mode, the white balance adjustment unit 351 performs white balance adjustment without distinguishing between the region of the presentation image and the region of the background image.

  FIG. 16 shows the correction characteristics of the gamma correction unit 352. When the imaging mode is set to the normal imaging mode, the gamma correction unit 352 compresses the luminance level of the high luminance part as compared with the gamma curve CBa used in the normal imaging mode, as shown in FIG. Gamma correction is performed on the region of the presentation image using the gamma curve CBb having the corrected characteristics. Further, as shown in FIG. 16B, the gamma correction unit 352 uses the gamma curve CBc having the characteristics corrected so as to extend the luminance level of the low luminance portion compared to the normal gamma curve CBa. Perform gamma correction on the image area. Further, when the imaging mode is set to the normal imaging mode, the gamma correction unit 352 uses the normal gamma curve CBa shown in FIGS. 16A and 16B to convert the region of the presentation image and the region of the background image. Perform gamma correction without distinction.

  In step ST7, the control unit 51 determines whether or not the imaging operation is finished. The control unit 51 ends the operation when, for example, a power-off operation of the imaging apparatus 10 is performed or an operation of switching from the imaging mode to the reproduction mode for reproducing the recorded image is performed. When the power-off operation or the switching operation to the reproduction mode is not performed, the process returns to step ST1.

  As described above, in the imaging device 10, the imaging mode is automatically set to the presentation image imaging mode or the normal imaging mode according to the detection result of the presentation image. Here, when the presentation image is detected and the imaging mode is set to the presentation image imaging mode, exposure control, white balance adjustment, and gamma correction are performed so that the presentation image and the background image in the captured image become a good image. . Further, when the presented image is not detected and the imaging mode is set to the normal imaging mode, exposure control, white balance adjustment, and gamma correction are performed according to the captured image without distinguishing between the region of the presented image and the background image.

  Therefore, the imaging mode can be automatically switched without manually switching the imaging mode. In addition, since it is not necessary to individually set the operation setting when the presentation image capturing mode is set, a captured image in which the presented image is a good image can be easily obtained. For example, it is possible to easily obtain a good captured image that is free from whiteout and discoloration in the presented image and blackout in the background image. Further, it is possible to prevent unnecessary strobe light emission when the presentation image is captured. Furthermore, since the end of the presentation image capturing mode can be determined automatically by the imaging device, it is possible to save the user from having to end the presentation image capturing mode by operating a menu button, for example.

  The detection of the presentation image is not limited to the case where the horizontal and vertical distributions are obtained from the binarized image and the sides of the presentation image are detected as described above, as disclosed in JP-A-2007-148612. Various methods may be used such as performing contour extraction and detecting the region of the presentation image from the extracted contour.

  By the way, when capturing a presentation image, for example, when capturing a scheduled event such as a presentation, an imaging device is often mounted on a tripod or the like and imaged at a fixed angle of view. Therefore, the imaging mode can be switched in consideration of the state of the imaging apparatus.

  FIG. 17 is a flowchart showing another operation of the imaging apparatus when switching the imaging mode in consideration of the state of the imaging apparatus. In step ST11, the control unit 51 causes the presentation image detection unit 31 to perform presentation image detection in the same manner as in step ST1, and proceeds to step ST12.

  In step ST12, the control unit 51 determines whether or not the condition for switching to the presentation image capturing mode is satisfied. When it is determined that the condition for switching to the presentation image imaging mode is satisfied, the control unit 51 proceeds to step ST13, and when it is determined that the condition is not satisfied, the imaging mode is set to the normal imaging mode and the process proceeds to step ST16. Note that, as described above, the control unit 51 determines whether or not the condition for switching to the presentation image imaging mode is satisfied according to the generation state of the frame in which the presentation image is detected.

  In step ST13, the control unit 51 determines whether or not the presented image in the captured image is in a stable state. When the control unit 51 determines that the position and size of the presented image in the captured image are stable and does not change, the control unit 51 sets the imaging mode to the presented image imaging mode, proceeds to step ST14, and determines that the present state is not stable. If so, the process proceeds to step ST18.

  Here, the change in the position and size of the presented image in the captured image occurs, for example, when the imaging direction of the imaging device 10 changes or a zoom operation is performed. For this reason, the control unit 51 determines whether or not the magnitude of the motion of the imaging device 10 is equal to or larger than a predetermined value based on the sensor signal SS from the motion detection unit 53, and the motion is set in advance. When it is less than the fixed amount, it is determined that the state is stable, and when the movement is equal to or greater than a predetermined amount, it is determined that the state is not stable. Further, based on the operation signal PS from the user interface unit 52, the control unit 51 determines that the zoom operation is being performed and that the control unit 51 is not in a stable state, and when the zoom operation is not being performed, the control unit 51 is in a stable state. Determine.

  In step ST14, the control unit 51 performs a presentation image capturing mode notification process in the same manner as in step ST3, and proceeds to step ST15.

  In step ST15, the control unit 51 performs the presentation image capturing mode operation setting similarly to step ST4, and proceeds to step ST18. For example, the control unit 51 controls the detection unit 32 to switch the detection area for calculating the detection value used when performing exposure control to a position where the influence of the presented image is small. In addition, the control unit 51 causes the white balance adjustment unit 351 and the gamma correction unit 352 of the image processing unit 35 to perform white balance adjustment and gamma correction for each region of the presentation image and the background image.

  In step ST16, the control unit 51 performs normal imaging mode notification processing in the same manner as in step ST5, and proceeds to step ST17.

  In step ST17, the control unit 51 performs normal imaging mode operation setting similarly to step ST6, and proceeds to step ST18. For example, the control unit 51 controls the detection unit 32 to switch the detection region so as to perform exposure control that places importance on the center of the captured image, for example. In addition, the control unit 51 causes the white balance adjustment unit 351 and the gamma correction unit 352 of the image processing unit 35 to perform white balance adjustment and gamma correction regardless of the region of the presentation image or the background image.

  In step ST18, the control unit 51 determines whether or not the imaging operation is finished. The control unit 51 ends the operation when, for example, a power-off operation of the imaging apparatus 10 is performed or an operation of switching from the imaging mode to the reproduction mode for reproducing the recorded image is performed. When the power-off operation or the switching operation to the reproduction mode is not performed, the process returns to step ST11.

  Thus, since the presentation image capturing mode is not set unless the presentation image is in a stable state, frequent mode switching can be prevented. Further, it is possible to prevent the presented image from being switched to the presented image capturing mode in an unstable state.

  The imaging device 10 may switch the imaging mode according to the detected position of the presented image. FIG. 18 is a flowchart illustrating another operation of the imaging apparatus when switching the imaging mode according to the detected position of the presented image. In step ST21, the control unit 51 causes the presentation image detection unit 31 to perform presentation image detection in the same manner as in step ST1, and proceeds to step ST22.

  In step ST22, the control unit 51 determines whether or not the condition for switching to the presentation image capturing mode is satisfied. When it is determined that the condition for switching to the presentation image imaging mode is satisfied, the control unit 51 proceeds to step ST23, and when it is determined that the condition is not satisfied, the imaging mode is set to the normal imaging mode and the process proceeds to step ST27. Note that, as described above, the control unit 51 determines whether or not the condition for switching to the presentation image imaging mode is satisfied according to the generation state of the frame in which the presentation image is detected.

  In step ST23, the control unit 51 determines whether or not the presented image is in a stable state as in step ST13. When it is determined that the presented image is in a stable state, the control unit 51 proceeds to step ST24, and when it is determined that the presented image is not in a stable state, the control unit 51 proceeds to step ST29.

  In step ST24, the control unit 51 determines whether the position of the presentation image detected in step ST21 has moved to the end of the captured image. The control unit 51 proceeds to step ST25 when determining that the position of the presented image has not moved to the end of the captured image, and proceeds to step ST27 when determining that the position of the presentation image has moved to the end.

  FIG. 19 shows the movement of the presented image. Here, for example, when the movement of the imaging device 10 is smaller than a predetermined value set in advance, and the center position GPa of the presented image moves in the direction toward the edge of the captured image to become the position GPb, the presented image becomes the end of the captured image. It is determined that it has moved to. As described above, when the presented image moves to the end of the captured image, the process proceeds to step ST27 on the assumption that the subject focused on by the user has moved from the presented image to another subject. When the control unit 51 determines that the presented image has not moved to the end of the captured image, the control unit 51 proceeds to step ST25.

  In step ST25, the control unit 51 performs a presentation image capturing mode notification process in the same manner as in step ST3, and proceeds to step ST26.

  In step ST26, the control unit 51 sets the presentation image capturing mode operation setting in the same manner as in step ST4, and proceeds to step ST29. For example, the control unit 51 controls the detection unit 32 to switch the detection area for calculating the detection value used when performing exposure control to a position where the influence of the presented image is small. In addition, the control unit 51 causes the white balance adjustment unit 351 and the gamma correction unit 352 of the image processing unit 35 to perform white balance adjustment and gamma correction for each region of the presentation image and the background image.

  In step ST27, the control unit 51 performs normal imaging mode notification processing in the same manner as in step ST5, and proceeds to step ST28.

  In step ST28, the control unit 51 performs the normal imaging mode operation setting in the same manner as in step ST6, and proceeds to step ST29. For example, the control unit 51 controls the detection unit 32 to switch the detection region so as to perform exposure control that places importance on the center of the captured image, for example. In addition, the control unit 51 causes the white balance adjustment unit 351 and the gamma correction unit 352 of the image processing unit 35 to perform white balance adjustment and gamma correction regardless of the region of the presentation image or the background image.

  In step ST29, the control unit 51 determines whether or not the imaging operation is finished. The control unit 51 ends the operation when, for example, a power-off operation of the imaging apparatus 10 is performed or an operation of switching from the imaging mode to the reproduction mode for reproducing the recorded image is performed. When the power-off operation or the switching operation to the reproduction mode is not performed, the process returns to step ST21.

  In this way, by switching the imaging mode according to whether or not the presented image is in a stable state and whether or not the presented image has moved to the end of the captured image, the user focuses on what subject. The imaging mode can be automatically switched in consideration of whether or not the imaging is performed.

  Further, the switching of the imaging mode may be performed based on the detection result of the presented image and an instruction from the user.

  FIG. 20 is a flowchart illustrating an operation when the imaging mode is switched based on the detection result of the presented image and an instruction from the user as another operation of the imaging apparatus. In step ST31, the control unit 51 causes the presentation image detection unit 31 to perform presentation image detection in the same manner as in step ST1, and proceeds to step ST32.

  In step ST32, the control unit 51 determines whether or not the condition for switching to the presentation image capturing mode is satisfied. When it is determined that the condition for switching to the presentation image imaging mode is satisfied, the control unit 51 proceeds to step ST33, and when it is determined that the condition is not satisfied, the imaging mode is set to the normal imaging mode and the process proceeds to step ST37. Note that, as described above, the control unit 51 determines whether or not the condition for switching to the presentation image imaging mode is satisfied according to the generation state of the frame in which the presentation image is detected.

  In step ST33, the control unit 51 notifies the user of the detection result of the presented image, and proceeds to step ST34.

  In step ST <b> 34, the control unit 51 determines whether or not the user has performed an instruction operation on the presented image. When it is determined that the user has performed an instruction operation on the presented image, the control unit 51 proceeds to step ST35. Further, when the user performs an instruction operation for a background image different from the presented image, the process proceeds to step ST37.

  FIG. 21 is a diagram for explaining the detection result of the presented image and the user instruction operation. The control unit 51 generates a display signal HE based on the detection signal JA. Further, the control unit 51 supplies the generated display signal HE to the image processing unit 35 to generate the display image signal HG from the image signal DVa and the display signal HE, as shown in FIG. Then, on the screen of the display unit 38, a display capable of identifying the captured image and the area of the detected presentation image, for example, a detection frame MD is displayed. Here, when the inside of the detection frame MD is instructed by the user, the control unit 51 detects that the area OP pressed by the user is inside the detection frame MD, for example, as shown in FIG. When this is done, it is determined that the user has performed an instruction operation on the presented image, and the process proceeds to step ST35. Further, when the user indicates that the detection frame MD is out of the frame, for example, as shown in (C) of FIG. 21, the control unit 51, as shown in FIG. When it is detected that the area OP pressed by the user is within the detection frame MD, it is determined that the background image instruction operation by the user has been performed, and the process proceeds to step ST37.

  In step ST35, the control unit 51 performs a presentation image capturing mode notification process in the same manner as in step ST3, and proceeds to step ST36.

  In step ST36, the control unit 51 performs the presentation image capturing mode operation setting similarly to step ST4, and proceeds to step ST39. For example, the control unit 51 controls the detection unit 32 to switch the detection area for calculating the detection value used when performing exposure control to a position where the influence of the presented image is small. In addition, the control unit 51 causes the white balance adjustment unit 351 and the gamma correction unit 352 of the image processing unit 35 to perform white balance adjustment and gamma correction for each region of the presentation image and the background image.

  In step ST37, the control unit 51 performs normal imaging mode notification processing in the same manner as in step ST5, and proceeds to step ST38.

  In step ST38, the control unit 51 performs normal imaging mode operation setting similarly to step ST6, and proceeds to step ST39. For example, the control unit 51 controls the detection unit 32 to switch the detection region so as to perform exposure control that places importance on the center of the captured image, for example. In addition, the control unit 51 causes the white balance adjustment unit 351 and the gamma correction unit 352 of the image processing unit 35 to perform white balance adjustment and gamma correction regardless of the region of the presentation image or the background image.

  In step ST39, the control unit 51 determines whether or not the imaging operation is finished. The control unit 51 ends the operation when, for example, a power-off operation of the imaging apparatus 10 is performed or an operation of switching from the imaging mode to the reproduction mode for reproducing the recorded image is performed. When the power-off operation or the switching operation to the reproduction mode is not performed, the process returns to step ST31.

  In this way, the user is notified of the detection result of the presentation image, and when the presentation image is instructed by the user, the presentation image capturing mode is set. Therefore, even if the detection of the presentation image causes an error, an error occurs. Based on the detected result, exposure control, white balance adjustment, gamma correction, and the like can be prevented. In addition, since it is possible to switch to the presented image capturing mode simply by giving an instruction to the notified detection result, the operation menu display operation or the imaging mode switching menu is selected from the operation menu as in the past. It is possible to easily switch to the presentation image imaging mode without performing an operation of selecting the presentation image imaging mode from the operation and imaging mode switching menu. Furthermore, since a detection frame MD is displayed when a presentation image is detected, the user can easily grasp that switching to the presentation image imaging mode is possible.

  By the way, the end of the presentation image capturing mode may be performed according to a change in the subject without using the detection result of the presentation image. For example, when the subject to be imaged changes from a presented image to a person, the presented image capturing mode is terminated.

  FIG. 22 shows a flowchart in the case of switching the imaging mode in accordance with the face detection result. In step ST41, the control unit 51 performs face detection and proceeds to step ST42. In face detection, face detection is performed by focusing on the face shape and skin color area. For example, using a genetic algorithm on a captured image to search for a face shape, configuring a classifier for face detection by machine learning, or other commonly used techniques Detect images.

  In step ST42, the control unit 51 determines whether a face is detected. When it is determined that a face has been detected, the control unit 51 proceeds to step ST43, and when it has not detected a face, the control unit 51 proceeds to step ST46.

  In step ST43, the control unit 51 determines whether or not the detected face position is located at the center of the captured image. When the detected face position is in the center of the captured image, the control unit 51 switches from the presentation image capturing mode to another image capturing mode, for example, the human image capturing mode, and proceeds to step ST44.

  In step ST44, the control unit 51 performs a person imaging mode notification process and proceeds to step ST45. The control unit 51 sets the imaging mode to the person imaging mode, and notifies the user that the imaging apparatus 10 is set to the person imaging mode. For example, when the human imaging mode is set, the control unit 51 generates the display signal HE and supplies the display signal HE to the image processing unit 35 to generate the display image signal HG from the image signal DVa and the display signal HE. Information indicating that the captured image and the person imaging mode are set are displayed on the screen of the unit 38. FIG. 23 shows a case where an icon display MM indicating that the person imaging mode is set is provided on the screen of the display unit 38. FIG. 23 shows a case where a frame display ME indicating the position of the detected face is also displayed.

  In step ST <b> 45, the control unit 51 performs the person imaging mode operation setting and proceeds to step ST <b> 46. The control unit 51 performs exposure control, white balance adjustment, and gamma correction so that the captured face is optimally displayed. For example, the control unit 51 switches the detection area for calculating the detection value used when performing exposure control to the detected face position, or adjusts the white balance so that the human face has an appropriate color. Gamma correction or the like so that the face does not fade out.

  In step ST46, the control unit 51 determines whether or not the imaging operation is finished. The control unit 51 ends the operation when, for example, a power-off operation of the imaging apparatus 10 is performed or an operation of switching from the imaging mode to the reproduction mode for reproducing the recorded image is performed. When the power-off operation or the switching operation to the reproduction mode is not performed, the process returns to step ST41. In this way, it is possible to automatically switch the imaging mode by detecting that the subject that the user is interested in has been changed from a presented image to a person.

  24 and 25 summarize the operation of the above-described imaging apparatus. FIG. 24 illustrates the start of the presentation image imaging mode, the detection area change, the white balance adjustment for each area, and the gamma correction for each area. The relationship of the detection method of a presentation image is shown.

  Simple presentation image detection, for example, a method of acquiring a horizontal distribution and a vertical distribution from a binarized signal obtained by performing a binarization process as shown in FIG. 6 and detecting an upper side and a lower side, and a left side and a right side, A contour indicating the boundary between the presentation image and the background image is obtained by performing highly accurate presentation image detection, for example, edge detection of the captured image, and straight lines indicating four vertices and four sides of the presentation image are obtained from the contour information. Compared to the method of obtaining and detecting the presented image, the presented image can be detected quickly. On the other hand, in the highly accurate presentation image detection, the region of the presentation image in the captured image can be detected with higher accuracy than the simple presentation image detection.

  Here, it is not necessary to detect the area of the presentation image with high accuracy when starting the presentation image imaging mode or switching the detection area, and with high accuracy the white balance adjustment for each area and gamma correction for each area. It is desirable to detect. Therefore, as shown in FIG. 24, it is desirable to use simple presentation image detection when starting the presentation image imaging mode or switching the detection area, and highly accurate presentation images are used for white balance adjustment for each area and gamma correction for each area. It is preferred to use detection.

  Further, in the above-described simple presentation image detection, when the presentation image is captured from the front, each side of the presentation image in the captured image is substantially in the horizontal direction and the vertical direction, so that the presentation image can be detected with high accuracy. However, if the presentation image is captured from an oblique direction, each side of the presentation image in the captured image may have an inclination with respect to the horizontal direction or the vertical direction, so that the detection accuracy of the presentation image decreases. Therefore, in the white balance adjustment for each region and the gamma correction for each region, simple presentation image detection is not preferable when the presentation image is not captured from the front. In FIG. 24, the case where it is preferable is indicated by “◯”, and the case where it is not preferable is indicated by “X”. In addition, a case where conditions are not preferable is indicated by “Δ”.

  FIG. 25 shows the relationship between the detection area change, the white balance adjustment for each area, the gamma correction for each area, and the operation during imaging. When the presented image moves to the position of the end of the captured image, the control unit 51 ends the switching of the detection region and performs exposure control by increasing the weight of the central portion of the captured image, for example. Further, even when the presented image moves to the position of the edge of the captured image, the operation of performing white balance adjustment and gamma correction for each region is maintained.

  The control unit 51 maintains the exposure control in which the weight of the edge of the captured image is increased and the operation of performing the white balance adjustment and the gamma correction for each region when the imaging apparatus is moved greatly or when the zoom operation is performed. .

  In addition, the control unit 51 performs exposure control in which the weight of the edge of the captured image is increased when the position of the face detected by the face detection is the center of the captured image and when the presentation image is not detected, and for each region. Ends the operations for white balance adjustment and gamma correction. That is, the control unit 51 performs exposure control in which the weight of the central portion of the captured image is increased, and white balance adjustment and gamma correction without distinguishing the region of the captured image.

  Further, the detection area may be changed, white balance adjustment for each area, and gamma correction for each area may be performed in accordance with a plurality of operations during imaging. FIG. 26 illustrates a case where the detection area is changed, the white balance is adjusted for each area, and the gamma correction is performed for each area in accordance with a plurality of operations during imaging.

  When the presented image moves to the position of the end of the captured image, the control unit 51 ends the switching of the detection region and performs exposure control by increasing the weight of the central portion of the captured image, for example. Even if the presented image moves to the position of the end of the captured image, the white balance adjustment and gamma correction are maintained in the previous state. Here, when the image pickup apparatus largely moves or a zoom operation is performed, the operation of performing white balance adjustment and gamma correction for each region is ended.

  The control unit 51 also maintains the previous state of detection area switching, white balance adjustment, and gamma correction when the imaging apparatus moves greatly, for example, when the zoom operation is performed or when the zoom operation is performed. . Here, when the presented image moves to the position of the edge of the captured image, the operation of switching the detection area and performing white balance adjustment and gamma correction for each area is ended. That is, the control unit 51 performs exposure control in which the weight of the central portion of the captured image is increased, and white balance adjustment and gamma correction without distinguishing the region of the captured image.

  In this way, the imaging mode can be set to the presentation image imaging mode in a state suitable for the presentation image imaging mode in consideration of various conditions. For example, by using the face detection result and the motion detection result for an imaging scene in which a presentation image such as a presentation is captured and a person is captured, what the user gazes at and shoots Therefore, the shooting mode is switched automatically, so that a good captured image can be obtained in the imaging mode in accordance with the user's intention.

It is a figure which shows the structure of an imaging device. It is a figure which shows the structure of a presentation image detection part. It is a figure which shows the structure of a detection part. It is a figure which shows the structure of a white balance adjustment part. It is a flowchart which shows operation | movement of an imaging device. It is a flowchart which shows a presentation image detection operation | movement. It is a figure which shows the acquisition process of a horizontal distribution and a vertical distribution. It is a figure which shows the detection process of the upper side and lower side of a presentation image. It is a figure which shows the detection process of the upper side and lower side of a presentation image. It is a figure which shows the detection process of the left side and right side of a presentation image. It is a figure which shows the detection process of the left side and right side of a presentation image. It is a figure for demonstrating the discrimination | determination operation | movement of whether it is a presentation image. It is a figure which shows the case where the icon display which shows having been set to the presentation image imaging mode is provided. It is a figure for demonstrating operation | movement of a detection part. It is a figure which shows the position and achromatic region of an achromatic color. It is a figure which shows the correction characteristic of a gamma correction part. It is a flowchart which shows other operation | movement of an imaging device. It is a flowchart which shows other operation | movement of an imaging device. It is a figure which shows the movement of a presentation image. It is a flowchart which shows other operation | movement of an imaging device. It is a figure for demonstrating the detection result of a presentation image, and user instruction | indication operation. It is a flowchart which shows other operation | movement of an imaging device. It is a figure which shows the case where the icon display which shows having been set to person imaging mode is provided. It is a figure which shows the relationship between the start of a presentation image imaging mode, the change of a detection area | region, the white balance adjustment for every area | region, the gamma correction for every area | region, and the detection method of a presentation image. It is a figure which shows the relationship between the operation | movement at the time of the change of a detection area | region, the white balance adjustment for every area | region, the gamma correction for every area | region, and imaging. It is a figure which shows the case where the detection area | region change, the white balance adjustment for every area | region, and the gamma correction for every area | region are performed according to several operation | movement at the time of imaging.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Lens part, 12 ... Diaphragm mechanism, 21 ... Imaging part, 22 ... Timing signal generation part, 23 ... A / D conversion part, 31 ... Presentation image detection unit, 32 ... detection unit, 35 ... image processing unit, 37 ... recording / reproducing unit, 38 ... display unit, 41 ... lens driving unit, 42 ... stop driving 45, strobe, 51 ... control unit, 52 ... user interface unit, 53 ... motion detection unit, 211 ... imaging device, 212 ... CDS unit, 213 ... amplification , 311 ... conversion processing unit, 312, 313 ... binarization processing unit, 314 ... signal selection unit, 315 ... horizontal distribution acquisition unit, 316 ... vertical distribution acquisition unit, 317 ..Detection processing unit, 321... Blocking processing unit, 322. 323 ... Coefficient setting unit, 324 ... Multiplier, 325 ... Detection value calculation unit, 351 ... White balance (WB) adjustment unit, 351a ... White balance detection unit, 351b ... White balance correction coefficient setting unit, 351c ... correction processing unit, 352 ... gamma correction unit

Claims (7)

An imaging unit that images a subject and obtains a captured image;
A presentation image detection unit that detects a presentation image presented by the image presentation device from the captured image based on a difference in luminance or color in the captured image;
When the detection result of the presentation image satisfies a preset condition, the imaging mode is switched to the presentation image imaging mode, and at least the detection area for calculating the detection value used for exposure control is changed, and the imaging image A control unit that performs either white balance adjustment for each region with respect to a region of the presentation image and other image regions excluding the presentation image and gamma correction for each region with respect to the region of the presentation image and the other image in the captured image; An imaging apparatus having The presentation image detection unit performs first presentation image detection and second presentation image detection for detecting the presentation image with higher accuracy than the first,
The control unit switches the imaging mode and changes the detection area based on the detection result obtained by the first presentation image detection, and based on the detection result obtained by the second presentation image detection. The imaging apparatus according to claim 1, wherein white balance adjustment and gamma correction are performed for each region.   The imaging apparatus according to claim 2, wherein the control unit ends the presentation image imaging mode when a detection result of the presentation image does not satisfy the preset condition. A face image detection unit for detecting a face image from the captured image;
The imaging apparatus according to claim 3, wherein the control unit terminates the presentation image imaging mode when the position of the face image detected by the face image detection unit is a central part of the captured image. A display unit that displays the captured image as an identifiable region of the presentation image detected by the presentation image detection unit;
In the captured image displayed, the control unit assumes that a detection result of the presented image satisfies a preset condition when a user operation for designating a region of the presented image is performed. The imaging device according to claim 2, wherein the imaging device is switched to a presentation image imaging mode.   In the displayed captured image, the control unit assumes that the detection result of the presented image does not satisfy a preset condition when a user operation is performed to specify a region of the other image. The imaging apparatus according to claim 5, wherein the presentation image imaging mode is terminated. An imaging step of capturing an image of an object by an imaging unit to obtain a captured image;
A presentation image detection step of detecting a presentation image presented by an image presentation device from the captured image based on a difference in luminance or color in the captured image;
When the detection result of the presentation image satisfies a preset condition, the imaging mode is switched to the presentation image imaging mode, and at least the detection area for calculating the detection value used for exposure control is changed, and the imaging image A control step of performing either white balance adjustment for each region for the region of the presentation image and other image regions excluding the presentation image and gamma correction for each region for the region of the presentation image and the other image in the captured image; An imaging method comprising: