JP2018181357A - Region extraction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a region extraction device that can properly extract a region of a main subject from an input image.SOLUTION: A region extraction device comprises: a first extraction section that extracts a first region from an image on the basis of color information; a second extraction section that extracts a second region from the image on the basis of luminance information; and a setting section that, when an image signal of the first region is less than a threshold, sets the second region as a specific region.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an area extraction apparatus for extracting an area from an image.

  An image processing apparatus is disclosed that extracts a region from a captured image based on a histogram of image feature amounts and identifies the extracted region (see Patent Document 1).

JP-A-4-10079

  However, the image processing apparatus disclosed in Patent Document 1 has a problem that it is not possible to extract an area to which a person pays attention from a captured image, that is, the area of the main subject can not be appropriately extracted.

  The present invention has been made in view of the above-mentioned point, and an object of the present invention is to provide an area extraction device capable of appropriately extracting an area of a main subject from an input image.

  The present invention has been made to solve the above-described problems, and a first extraction unit that extracts a first region from an image based on color information, and a second extraction that extracts a second region from the image based on luminance information. A region extraction device including: a setting unit configured to set the second region as a specific region when the image signal of the first region is less than a threshold.

  According to the present invention, the area extraction device can appropriately extract the area of the main subject from the input image.

It is a block diagram showing composition of a lens barrel, an imaging device provided with a field extraction device, and a storage medium in one embodiment of the present invention. FIG. 5 depicts a histogram of the intensity of a backlit image in an embodiment of the present invention. FIG. 6 is a diagram illustrating a histogram of luminance in one embodiment of the present invention. FIG. 5 depicts an example of a region extracted from an image based on luminance in an embodiment of the present invention. FIG. 6 is a diagram illustrating an example of a region extracted from an image based on color component amounts in an embodiment of the present invention. It is a flowchart showing the procedure of the process to focus on the area | region in one Embodiment of this invention. It is a flowchart (the first half) showing the procedure of the process to focus on the area | region contained in achromatic image in one Embodiment of this invention. It is a flowchart (the second half) showing the procedure of the process to focus on the area | region contained in achromatic image in one Embodiment of this invention.

  An embodiment of the present invention will be described in detail with reference to the drawings. In FIG. 1, the configurations of the lens barrel 111, the imaging device 100, and the storage medium 200 are represented by a block diagram. The imaging device 100 captures an optical image incident from the lens barrel 111, and stores the obtained image in the storage medium 200 in a still image format or a moving image format.

First, the configuration of the lens barrel 111 will be described.
The lens barrel 111 includes a focus adjustment lens (hereinafter referred to as an “AF (Auto Focus) lens”) 112, a lens drive unit 116, an AF encoder 117, and a barrel control unit 118. The lens barrel 111 may be detachably connected to the imaging device 100 or may be integral with the imaging device 100.

  The AF lens 112 is driven by the lens driving unit 116, and guides an optical image to a light receiving surface (photoelectric conversion surface) of the image sensor 119 of the imaging unit 110 described later.

  The AF encoder 117 detects the movement of the AF lens 112, and outputs a signal corresponding to the movement amount of the AF lens 112 to the lens barrel control unit 118. Here, the signal corresponding to the amount of movement of the AF lens 112 is, for example, a sine wave signal whose phase changes according to the amount of movement of the AF lens 112.

  The lens barrel control unit 118 controls the lens drive unit 116 in accordance with a drive control signal input from the CPU 190 of the imaging device 100. Here, the drive control signal is a control signal for driving the AF lens 112 in the optical axis direction. The lens barrel control unit 118 changes, for example, the number of steps of the pulse voltage output to the lens driving unit 116 based on the drive control signal.

  Further, the lens barrel control unit 118 outputs the position (focus position) of the AF lens 112 in the lens barrel 111 to the CPU 190 based on a signal according to the movement amount of the AF lens 112. Here, the lens barrel control unit 118 integrates the signal corresponding to the amount of movement of the AF lens 112 for each moving direction of the AF lens 112, for example, thereby moving the amount of movement of the AF lens 112 in the lens barrel 111 (position ) May be calculated.

  The lens driving unit 116 drives the AF lens 112 according to the control by the lens barrel control unit 118, and moves the AF lens 112 in the optical axis direction in the lens barrel 111.

Next, the configuration of the imaging device 100 will be described.
The imaging device 100 includes an imaging unit 110, an area extraction device 140, a display unit 150, a buffer memory unit 130, an operation unit 180, a storage unit 160, a CPU 190, and a communication unit 170.

  The imaging unit 110 includes an imaging element 119 and an A / D (Analog / Digital) conversion unit 120. The imaging unit 110 is controlled by the CPU 190 in accordance with the set imaging conditions (for example, the aperture value and the exposure value).

  The image sensor 119 has a photoelectric conversion surface, converts an optical image formed on the photoelectric conversion surface by the lens barrel 111 (optical system) into an electrical signal, and converts the electrical signal into an A / D conversion unit 120. Output. The imaging device 119 may be configured by, for example, a complementary metal oxide semiconductor (CMOS). In addition, the imaging device 119 may convert an optical image into an electrical signal only for a partial region of the photoelectric conversion surface (image cutout processing).

  In addition, the imaging device 119 stores the storage medium via the A / D conversion unit 120 as image data based on the electrical signal according to the optical image obtained when receiving the imaging instruction from the user via the operation unit 180. Make it memorize in 200. On the other hand, in a state in which the imaging element 119 does not receive a photographing instruction from the user via the operation unit 180, the A / D conversion unit 120 is used as a through image based on an electrical signal corresponding to an optical image obtained continuously. Through the buffer memory unit 130 and the display unit 150.

  The A / D conversion unit 120 digitizes the electrical signal converted by the imaging device 119 and outputs the digitized image to the buffer memory unit 130.

  The operation unit 180 includes, for example, a power switch, a shutter button, a multi-selector (cross key), or other operation keys. The operation unit 180 receives a user's operation input by being operated by the user, and outputs a signal (hereinafter, referred to as “operation signal”) corresponding to the received operation input to the CPU 190. The operation signal may include an operation signal indicating whether the image feature amount is included in the image by a predetermined amount or more. For example, when the color image is adjusted to an achromatic image by white balance adjustment, the operation signal includes information indicating that the color component amount is not included in the image by more than a predetermined amount.

  The area extraction device 140 executes image processing on the image stored in the buffer memory unit 130 based on the image processing condition stored in the storage unit 160. The image subjected to the image processing is stored in the storage medium 200 via the communication unit 170.

  In addition, the area extraction device 140 extracts an area from the image stored in the buffer memory unit 130 based on the image feature amount, and determines the priority in the extracted area. In addition, the region extraction device 140 causes the storage unit 160 to store information indicating the extracted region. Here, the information indicating the area is, for example, information indicating the distribution of the area, information indicating the image feature amount of the area, and information indicating the priority defined for the area.

  Here, the region extraction device 140 calculates, for each pixel, a first image feature amount (for example, color component amount) and a second image feature amount (for example, light amount (brightness, brightness)) included in the image. . In addition, the region extraction device 140 determines whether each image feature amount is less than a predetermined threshold.

  In addition, when the first image feature amount is equal to or more than the threshold, the region extraction device 140 extracts the region from the image based on the first image feature amount. On the other hand, when the first image feature amount is less than the threshold, the region extraction device 140 extracts the region from the image based on the second image feature amount.

  In addition, when the operation signal indicates that the first image feature amount is not included in the image by a predetermined amount or more, the region extraction device 140 extracts the region from the image based on the second image feature amount. Extract. On the other hand, when the operation signal indicates that the first image feature amount is included in the image by a predetermined amount or more, the region extraction device 140 determines the first image feature amount and the second image feature amount. Extract regions from the image based on

  In addition, if the first image feature amount is less than the threshold, the region extraction device 140 adjusts the resolution for calculating the first image feature amount and the second image feature amount. In addition, when the image is an image captured in a backlit state, the region extraction device 140 determines, based on an evaluation value obtained from an AF evaluation region located at a predetermined position (for example, nine locations) in the image. Extract regions from the image. Here, the AF evaluation area is an area to be subjected to the contrast scan. In addition, if the image is not an image captured in a backlit state, the region extraction device 140 extracts a region determined based on the image feature amount from the image.

  In addition, when the image feature amount (for example, color component amount) is less than the threshold, the region extraction device 140 evaluates the evaluation value obtained from the AF evaluation region located at a predetermined position (for example, nine locations) Extract regions from the image based on In addition, when the image feature amount (for example, color component amount) is equal to or more than a threshold, the region extraction device 140 extracts a region determined based on the image feature amount from the image.

The buffer memory unit 130 stores the image captured by the imaging unit 110.
The display unit 150 displays an image obtained by the imaging unit 110, an operation screen, and the like. The display unit 150 is, for example, a liquid crystal display.

  The storage unit 160 stores a determination condition referred to by the CPU 190 at the time of scene determination, an imaging condition associated with each scene determined by the scene determination, information indicating an area, and the like.

  The CPU 190 controls the imaging unit 110 in accordance with the set imaging conditions (for example, the aperture value and the exposure value). Further, the CPU 190 acquires information indicating an area from the area extraction device 140. Further, the CPU 190 controls the lens barrel control unit 118 of the lens barrel 111 based on the information indicating the area so as to focus on the area having high priority set by the area extraction device 140 among the areas extracted from the image. Adjust the focus by outputting a drive control signal to

  The CPU 190 sets focus adjustment (AF), exposure adjustment (AE), white balance adjustment (AWB), and a subject image as pre-processing or post-processing of image processing based on information indicating a region. Settings for tracking (objects), settings for determining whether or not a night scene, settings for color correction processing, settings for enlarged display of an object image, settings for panning display, settings for optimization of brightness linked to zoom magnification, etc. Control.

  Further, the CPU 190 controls setting of a tone curve (gradation curve), setting of whether or not to perform simultaneous shooting of a still image and a moving image, and the like based on information indicating an area.

  Further, the CPU 190 sets whether or not to continuously shoot as pre-processing or post-processing based on the information indicating the area, and setting of switching of an object image to be focused in continuous shooting, and continuous change of the focal length Setting of the light emission unit (not shown) of the imaging apparatus 100, setting of change of the exposure amount, setting of whether to execute the change of the shutter speed, and the like are controlled.

  Further, based on the operation signal input from the operation unit 180, the CPU 190 causes the region extraction device 140 to process an image in the still image format or the moving image format.

  The communication unit 170 is connected to a removable storage medium 200 (such as a card memory), and executes writing, reading, and erasing of information (image data, information indicating an area, and the like) to the storage medium 200.

  The storage medium 200 is a storage unit that is detachably connected to the imaging apparatus 100, and stores various types of information (for example, image data, information indicating a region). The storage medium 200 may not be removable but may be integral with the imaging device 100.

Next, details of the region extraction device 140 will be described.
The area extraction device 140 includes a feature amount calculation unit 141, a determination unit 142, a reception unit 143, an area extraction unit 144, and a priority setting unit 145.

  The feature amount calculation unit 141 acquires an image captured by the imaging unit 110 via the buffer memory unit 130. Also, the feature amount calculation unit 141 calculates the image feature amount for each of a plurality of features (for example, color component amount, light amount (brightness, lightness)) included in the image acquired via the buffer memory unit 130, The calculated image feature amount is output to the region extraction unit 144.

  The determination unit 142 generates a histogram of the first image feature amount and the second image feature amount, and determines whether each image feature amount is less than a threshold. Here, the determination unit 142 determines whether the image feature amount is less than the threshold value based on the standard deviation σ of the histogram waveform of each image feature amount. For example, the standard deviation σ in the histogram waveform of the color component amount is equal to or less than a predetermined color component amount, that is, the standard deviation σ of the color difference signal Cb ≦ the predetermined color component amount, and the standard deviation σ of the color difference signal Cr. If ≦ predetermined color component amount, the determination unit 142 determines that the captured image is an achromatic image.

  In the following, as an example, the first image feature amount is assumed to be a color component amount, and the second image feature amount is assumed to be a light amount (brightness, lightness).

  The determination unit 142 determines whether the captured image is an image captured in a backlit state (hereinafter, referred to as a “backlit image”). FIG. 2 shows a histogram of the brightness of the backlight image. The horizontal axis represents the luminance Y. Also, the vertical axis represents the frequency N. If the luminance difference from the waveform peak of the histogram to another waveform peak is equal to or greater than a predetermined luminance difference, the determination unit 142 determines that the captured image is a backlight image.

  Returning to FIG. 1, the description of the configuration of the region extraction device 140 will be continued. The receiving unit 143 receives an operation signal from the operation unit 180, and transfers the received operation signal to the area extracting unit 144.

  When it is determined by the determination unit 142 that the captured image is a backlight image, the region extraction unit 144 evaluates an evaluation value obtained from an AF evaluation region located at a predetermined position (for example, nine locations) in the image. Extract regions from the image based on Here, the reason why the area is extracted from the image based on the AF evaluation area is that there is a possibility that the image has few or no suitable areas for which priority has been determined. In addition, when the determination unit 142 determines that the captured image is not a backlight image, the region extraction device 140 extracts a region determined based on the image feature amount calculated by the feature amount calculation unit 141 from the image.

  If the determination unit 142 determines that the image feature amount (for example, color component amount) included in the captured image is less than the threshold, the region extraction unit 144 determines a predetermined position (for example, nine locations) An area is extracted from the image based on the evaluation value obtained from the AF evaluation area existing in. In addition, when the determination unit 142 determines that the image feature amount included in the captured image is not less than the threshold, the region extraction device 140 determines the region determined based on the image feature amount calculated by the feature amount calculation unit 141. , Extract from the image.

  The area extraction unit 144 acquires an operation signal from the reception unit 143. In addition, the region extraction unit 144 acquires the image feature amount calculated by the feature amount calculation unit 141. In addition, the region extraction unit 144 acquires an image captured by the imaging unit 110 from the buffer memory unit 130. The area extraction unit 144 extracts an area from the image captured by the imaging unit 110 based on the image feature amount.

  In addition, the area extraction unit 144 extracts an area from an image binarized into a predetermined range (division) of the image feature amount and the outside of the range (hereinafter, referred to as “binary image”), and extracts the extracted area. Assign a label (labeling). Here, the threshold for determining a certain range (division) of the image feature amount is determined as, for example, “threshold = average value + coefficient × standard deviation” based on the average value and the standard deviation of the histogram waveform of the image feature amount. Be

  If the image feature amount is less than the threshold, the region extraction unit 144 adjusts the resolution for calculating the image feature amount. FIG. 3 shows a histogram of luminance. The horizontal axis represents the luminance Y. Also, the vertical axis represents the frequency N. FIG. 3A shows a histogram of the luminance of the subject area imaged on the screen.

  The region extraction unit 144 determines the resolution of the image feature amount based on the average value Ave and the standard deviation σ of the histogram. For example, a histogram with lower resolution compared to FIG. 3 (C) is represented in FIG. 3 (B). In FIG. 3B, since the resolution of luminance is lower than that in FIG. 3C, it is difficult to detect the subject region because the frequency change of the histogram waveform is not clearly detected. On the other hand, in FIG. 3C, since the resolution is higher than that in FIG. 3B, the frequency change of the histogram waveform is clearly detected, so it is easy to detect the subject region. Thus, the region extraction unit 144 can easily detect the subject region by adjusting the resolution of the image feature amount based on the average value and the standard deviation of the histogram.

  The area extracting unit 144 may set an upper limit on the resolution of the image feature amount so that the subject area too small is not detected. As a result, the area extraction unit 144 can easily distinguish between the subject area and the noise.

  FIG. 4 shows an example of the area extracted from the image based on the luminance. The region extraction unit 144 extracts an image based on luminance (hereinafter, referred to as “Y image”) from the image captured by the imaging unit 110. The region extraction unit 144 corrects (for example, gamma correction) the luminance (input value) of the image captured by the imaging unit 110, and calculates the corrected luminance Y (output value).

  The area extraction unit 144 divides the corrected luminance Y (output value) into a plurality of constant ranges. In FIG. 4, it is assumed that the corrected luminance Y (output value) is divided into divisions 1 to 4 as an example. Here, for example, the range upper limit of category 1 is “average value + K1 × standard deviation” and the range lower limit of category 1 is “average value + K2 × standard deviation” so that the high brightness region is included in the binary image of category 1 It is assumed that The area extraction unit 144 generates a binary image for each section (tone).

  The area extraction unit 144 extracts the high-intensity area 300 as an area image representing the search result by searching for the distribution of the high-intensity area included in the binary image of Section 1. Similarly, the region extraction unit 144 extracts the middle luminance region 310 into the region image representing the search result by searching for the middle luminance region included in the binary image of the division 2. Further, the region extraction unit 144 extracts the low brightness region 320 in the region image representing the search result by searching for the low brightness region included in the binary image of the division 4.

  In FIG. 4, it is assumed that the corrected luminance Y (output value) is divided into divisions 5 to 7 as an example because the resolution of the image feature amount is adjusted by the region extraction unit 144. For example, it is assumed that the division 5 is changed to have higher resolution than the division 1. The region extraction unit 144 extracts the high brightness region 301 in the region image representing the search result by searching for the distribution of the high brightness region included in the binary image of the segment 5 with high resolution.

  Also, for example, it is assumed that the division 7 is changed to have lower resolution than the division 4. The area extraction unit 144 extracts the low luminance area 321 in the area image representing the search result by searching for the distribution of the low luminance area included in the binary image of the low resolution section 7. As described above, the region extraction unit 144 can easily detect the subject region by adjusting the resolution of the image feature amount.

  FIG. 5 shows an example of the area extracted from the image based on the color component amount. The region extraction unit 144 extracts an image (hereinafter referred to as “Cr image”) based on the color component amount (Cr) from the image captured by the imaging unit 110. The region extraction unit 144 corrects (for example, gamma correction) the color component amount (input value) of the image captured by the imaging unit 110, and calculates the corrected color component amount Cr (output value).

  If there is a color component amount equal to or greater than a predetermined threshold (not an achromatic image) in the captured image, the region extraction unit 144 divides the corrected color component amount (Cr) into a plurality of constant ranges. In FIG. 5, as an example, it is assumed that the corrected color component amount Cr (output value) is divided into divisions 8 to 11. Here, for example, the upper limit of division 8 is “average value + K3 × standard deviation” and the lower limit of division 8 is “average value + K4 × standard deviation” so that a reddish region is included in the binary image of division 8 It is assumed that The area extraction unit 144 generates a binary image for each section (tone).

  The area extraction unit 144 extracts the reddish area 410 from the area image representing the search result by searching for the distribution of the reddish area included in the binary image of the segment 8. Similarly, the region extraction unit 144 extracts a pink-colored region 430 as a region image representing a search result by searching for a pink-colored region included in the binary image of the segment 9. Further, the region extraction unit 144 extracts the green region 450 as the region image representing the search result by searching for the green region included in the binary image of the division 11.

  In FIG. 5, it is assumed that the corrected color component amount Cr (output value) is divided into divisions 12 to 16 as an example because the resolution of the image feature amount is adjusted by the region extraction unit 144. Here, for example, it is assumed that the division 12 is changed to have higher resolution than the division 8. The region extraction unit 144 extracts the reddish region 410 from the region image representing the search result by searching for the distribution of the reddish region included in the binary image of the segment 12 with high resolution.

  Also, for example, it is assumed that the division 16 is changed to have higher resolution than the division 11. The region extraction unit 144 extracts the green region 451 as the region image representing the search result by searching for the distribution of the green region included in the binary image of the segment 16 with high resolution. As described above, the region extraction unit 144 can easily detect the subject region by adjusting the resolution of the image feature amount.

  Returning to FIG. 1, the description of the configuration of the region extraction device 140 will be continued. The priority setting unit 145 determines, in the area extracted by the area extraction unit 144, the priority indicating the area that a person may be paying more attention to. The priority setting unit 145 determines the priority of the area according to the evaluation value calculated for each area.

  For example, the priority setting unit 145 determines an evaluation value for each area according to the distance from the center of the angle of view to the center of gravity of the area. Here, the priority setting unit 145 determines a higher evaluation value for the area distributed closer to the center of the angle of view. Also, for example, the priority setting unit 145 determines an evaluation value in the area according to the inertia moment of the area. The priority setting unit 145 sets a high evaluation value in a region where the moment of inertia is small (concentrated at the center of gravity). Also, for example, the priority setting unit 145 determines the evaluation value in the area according to the area (the number of pixels) of the area. The priority setting unit 145 determines a higher evaluation value for the area closer to the predetermined area.

  The priority setting unit 145 causes the storage unit 160 to store the priority determined for each area as information indicating the area.

Next, the procedure of processing of the region extraction device will be described.
FIG. 6 is a flowchart showing the procedure of processing until focusing on the area. The feature amount calculation unit 141 reads the captured image (through image or through image sequence) via the buffer memory unit 130 (step S1). The feature amount calculation unit 141 executes basic image processing (for example, demosaicing, noise reduction, white balance adjustment, pixel defect correction) on the read image (step S2).

  The area extraction unit 144 reads the resized image of the captured image for each image feature amount (for example, YCrCb) via the buffer memory unit 130 (step S3). The determination unit 142 determines whether the read image has a color component amount equal to or greater than a predetermined threshold (is not an achromatic image). If the read image is an achromatic image, the region extraction unit 144 adjusts the resolution of luminance (step S4).

  The region extraction unit 144 generates a binary image based on the luminance when there is no color component amount equal to or larger than a predetermined threshold value in the read image (it is an achromatic image). In addition, if the region extraction unit 144 has a color component amount equal to or greater than a predetermined threshold (not an achromatic image) in the read image, the region extraction unit 144 generates a binary image based on the luminance and color component amounts. (Step S5). The area extraction unit 144 also extracts an area from the binary image (area image), and assigns a label to the extracted area (labeling) (step S6).

  If there are areas in the image that are estimated to be noise, the area extraction unit 144 removes those areas from the image. In addition, when the region presumed to be the background is present in the image, the region extraction unit 144 removes the region from the image (step S7). The priority setting unit 145 sets priorities in the area extracted by the area extraction unit 144 (step S8).

  The area extraction unit 144 shapes the area whose priority is determined by the priority setting unit 145 into a rectangle so as to fit in the AF evaluation area (step S9). In addition, the area extraction unit 144 sets an AF evaluation area at a position predetermined in the image (step S10).

  The CPU 190 executes a contrast scan based on the contrast value of the AF evaluation area (step S11). Further, based on the result of the contrast scan, the CPU 190 gives priority to the area in which the subject in close proximity is imaged, and causes the AF lens 112 to focus on the area (focusing drive) (step S12).

  FIG. 7 is a flowchart (first half) showing a procedure of processing for focusing on an area included in an achromatic image. FIG. 8 is a flowchart (second half) showing the procedure of the process for focusing on the area included in the achromatic image.

  The CPU 190 sets variables used in the exposure adjustment (AE) process and the white balance adjustment (AWB) process to predetermined reference values (step Sa1). The imaging unit 110 continuously captures (exposed) the through image, and outputs the captured through image to the buffer memory unit 130 and the display unit 150 (step Sa2).

  The feature amount calculation unit 141 performs basic image processing (for example, demosaicing, noise reduction, white balance adjustment, pixel defect correction) on the read image (step Sa3). Furthermore, the feature amount calculation unit 141 performs YCrCb (YUV) conversion on the image subjected to the basic image processing (step Sa4).

  The determination unit 142 determines whether white balance adjustment is performed in the imaging mode among the imaging mode and the image reproduction mode (step Sa5). When the white balance adjustment is not performed in the imaging mode (step Sa5-No), the determination unit 142 determines whether the captured image has a color component amount equal to or greater than a predetermined threshold (is not an achromatic image). Determine That is, the determination unit 142 determines whether the degree of achromatic color of the captured image is equal to or less than the threshold (step Sa6).

  If the captured image has a color component amount equal to or greater than a predetermined threshold (not an achromatic image) (step Sa6-No), the determination unit 142 determines whether the captured image is an achromatic image by white balance adjustment It is determined whether or not. Furthermore, the determination unit 142 determines whether the captured image is an achromatic image from the original (regardless of the white balance adjustment) (step Sa7).

  If the captured image is an achromatic image by white balance adjustment, or if the captured image is an achromatic image from the beginning (step Sa7-Yes), the determination unit 142 determines that the captured image is backlit. It is determined whether the image is an image (step Sa8).

  If the captured image is not a backlight image (step Sa8-No), the region extraction unit 144 adjusts the resolution of luminance. Here, the area extraction unit 144 adjusts the resolution of luminance so that many luminance areas are extracted as an area image from an achromatic image such as a snow scene (step Sa9).

  The area extraction unit 144 extracts a luminance area (see FIG. 4) as an area image representing a search result of an area included in the binary image. In addition, when the resolution of the color component amount is adjusted, the region extraction unit 144 extracts the color system region (see FIG. 5) as the region image representing the search result of the region included in the binary image. . The priority setting unit 145 sets the priority in the luminance area extracted by the area extraction unit 144 (step Sa10). The area extraction unit 144 shapes the area whose priority is determined by the priority setting unit 145 into a rectangle so as to fit the AF evaluation area (step Sa11).

  The CPU 190 executes the contrast scan based on the contrast value of the AF evaluation area (step Sa12). Further, the CPU 190 gives priority to the area in which the subject in close proximity is imaged based on the result of the contrast scan, and causes the AF lens 112 to focus on that area (focusing drive) (step Sa13).

  On the other hand, in step Sa5, when the white balance adjustment is performed in the imaging mode (step Sa5-Yes), the determination unit 142 stores the achromatic degree (for example, color component amount) in the storage unit 160 ( Step Sa14). Further, in step Sa6, when there is no color component amount equal to or larger than the predetermined threshold value in the captured image (it is an achromatic image) (step Sa6-No), the determination unit 142 stores the achromatic degree in the storage unit 160. (Step Sa15).

  In step Sa7, when the captured image is not an achromatic image due to white balance adjustment and the captured image is not an achromatic image from the beginning (step Sa7-No), the determination unit 142 captures an image. It is determined whether the color component amount of the obtained image is equal to or less than a predetermined threshold (step Sa16). Further, the region extraction unit 144 adjusts the resolution of the color component amount and the resolution of the luminance (step Sa17). Then, the determination unit 142 and the area extraction unit 144 proceed the process to Step Sa10.

  In step Sa8, if the captured image is a backlit image (step Sa8-Yes), the area extraction unit 144 determines an AF evaluation area at a predetermined position in the image (step Sa18).

As described above, the feature extraction unit 140 calculates the first image feature (for example, color component amount) and the second image feature (for example, luminance) included in the input image. 141, a determination unit 142 that determines whether the first image feature amount is less than a threshold, and if the first image feature amount is less than a threshold, based on the second image feature amount An area (for example, a reddish area) which is determined based on the first image feature amount when the region (for example, high brightness region 300) to be determined is extracted from the binary image and the first image feature amount is equal to or more than a threshold. And a region extraction unit 144 which extracts 410) from the binary image.
With this configuration, if the first image feature amount is less than the threshold, the region extraction unit 144 extracts, from the image, a region determined based on the second image feature amount. Thus, the region extraction device can appropriately extract the region of the main subject from the input image.

Further, the determination unit 142 determines whether the color component amount that is the first image feature amount is less than the threshold, and the region extraction unit 144 determines that the color component amount is less than the threshold. An area defined based on the luminance which is the second image feature quantity is extracted from the image, and when the color component quantity is equal to or more than the threshold value, an area defined based on the color component quantity is extracted from the image.
With this configuration, when the amount of color components is less than the threshold value, the region extraction unit 144 extracts a region determined based on the luminance from the image. Thus, the region extraction device can appropriately extract the region of the main subject from the input image.

In addition, when the first image feature (for example, color component amount) is less than the threshold, the region extraction unit 144 performs the first image feature and the second image feature (for example, luminance). Adjust (e.g., increase) the resolution for calculating.
With this configuration, the region extraction unit 144 adjusts the resolution for calculating the first image feature amount and the second image feature amount. Thus, the region extraction device can appropriately extract the region of the main subject from the input image.

In addition, the region extraction device 140 includes a reception unit 143 that receives an operation signal indicating whether the first image feature amount (for example, color component amount) is included in the image by a predetermined amount or more. When the operation signal indicates that the first image feature amount is not included in the image by a predetermined amount or more (for example, the color image is adjusted by the white balance adjustment) Region is adjusted to an achromatic image), an area (for example, a high brightness area 300) determined based on the second image feature quantity (for example, brightness) is extracted from the image, and the first image feature quantity If the operation signal indicates that the image is included in the image by a predetermined amount or more, an area (for example, a reddish area 410) determined based on the first image feature amount is extracted from the image Do.
With this configuration, when the operation signal indicates that the first image feature amount is not included in the image by a predetermined amount or more, the region extraction unit 144 sets the second image feature amount to the second image feature amount. An area determined based on the image is extracted from the image. Thus, the region extraction device can appropriately extract the region of the main subject from the input image.

In addition, the determination unit 142 determines whether the image is an image captured in a backlit state, and the region extraction unit 144 determines in advance when the image is an image captured in a backlit state. Regions are extracted from the image.
With this configuration, if the image is an image captured in a backlit state, and if the image is an image captured in a backlit state, the region extracting unit 144 extracts a predetermined region from the image. Thus, the region extraction device can appropriately extract the region of the main subject from the input image.

In addition, the region extraction device 140 calculates a first image feature (for example, color component amount) and a second image feature (for example, luminance) included in the input image; A receiving unit 143 configured to receive an operation signal indicating whether the first image feature amount is included in the image by a predetermined amount or more; and the first image feature amount by a predetermined amount or more When the operation signal indicates that the image is not included (for example, when a colored image is adjusted to an achromatic image by white balance adjustment), it is determined based on the second image feature amount If the operation signal indicates that a region (for example, high brightness region 300) is extracted from the image and the first image feature amount is included in the image by a predetermined amount or more, Based on the 1 image feature There are defined region (for example, red areas 410) comprises an area extracting unit 144 is extracted from the image.
With this configuration, when the operation signal indicates that the first image feature amount is not included in the image by a predetermined amount or more, the region extraction unit 144 sets the second image feature amount to the second image feature amount. An area determined based on the image is extracted from the image. Thus, the region extraction device can appropriately extract the region of the main subject from the captured image.

In addition, the region extraction device 140 calculates a feature amount calculation unit 141 that calculates an image feature amount (for example, color component amount) included in the input image, and whether the image is captured in a backlit state If the image is an image captured in a backlit state, an area determined based on the AF evaluation area is extracted from the image, and if the image is not an image captured in a backlit state, And a region extraction unit 144 which extracts a region determined based on the image feature amount calculated by the feature amount calculation unit from the image.
With this configuration, if the image is an image captured in a backlit state, the region extracting unit 144 extracts an area determined based on the AF evaluation region from the image, and the image is not an image captured in a backlit state. In this case, an area determined based on the image feature amount calculated by the feature amount calculation unit is extracted from the image. Thus, the region extraction device can appropriately extract the region of the main subject from the captured image.

In addition, the region extraction device 140 determines a feature amount calculation unit 141 that calculates an image feature amount (for example, color component amount) included in the input image, and determines whether the image feature amount is less than a threshold. A determination unit 142 extracts a region determined based on an AF evaluation region from the image when the image feature amount is less than a threshold, and calculates the feature amount calculation unit 141 when the image feature amount is equal to or more than a threshold And a region extraction unit 144 which extracts a region determined based on the image feature amount from the image.
With this configuration, the region extraction unit 144 extracts the region determined based on the AF evaluation region from the image when the image feature amount is less than the threshold, and calculates the feature amount when the image feature amount is equal to or more than the threshold. An area determined based on the image feature quantity calculated by the unit 141 is extracted from the image. Thus, the region extraction device can appropriately extract the region of the main subject from the input image.

Also, the feature amount calculation unit 141 calculates the color component amount that is the image feature amount.
Thus, the area extraction device can appropriately extract the area of the main subject from the input image based on the color component amount.

Further, the imaging device 100 includes an area extraction apparatus 140, a CPU 190 that adjusts the focus by the AF lens 112 to focus on the area extracted by the area extraction apparatus 140, and an imaging unit 110 that picks up an image including the area. Equipped with
With this configuration, the region extraction device 140 appropriately extracts the region of the main subject from the captured image. Thereby, the imaging device 100 can image the area of the main subject.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within the scope of the present invention.

  For example, the color components may be represented by color difference signals (Cr, Cb) or RGB signals.

  Also, the program for realizing the area extraction device and the imaging device described above is recorded in a computer-readable recording medium, and the computer system reads the program recorded in the recording medium and executes the program. Execution processing may be performed. Note that the “computer system” referred to here may include an OS and hardware such as peripheral devices.

  The "computer system" also includes a homepage providing environment (or display environment) if the WWW system is used. In addition, “computer readable recording medium” refers to flexible disks, magneto-optical disks, ROMs, writable nonvolatile memories such as flash memories, portable media such as CD-ROMs, hard disks incorporated in computer systems, etc. Storage devices.

Furthermore, the “computer-readable recording medium” is a volatile memory (for example, DRAM (Dynamic Memory) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line). As Random Access Memory), it is assumed that the program which holds the program for a fixed time is included.
The program may be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the “transmission medium” for transmitting the program is a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the program may be for realizing a part of the functions described above. Furthermore, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 100 ... Imaging device, 110 ... Imaging part, 111 ... Lens-barrel, 140 ... Area extraction device, 14
1 feature amount calculation unit 142 determination unit 143 reception unit 144 area extraction unit 145 priority setting unit

Claims (1)

A first extraction unit that extracts a first region from an image based on color information;
A second extraction unit that extracts a second region from the image according to luminance information;
A setting unit configured to set the second area as a specific area when the image signal of the first area is less than a threshold value;
An area extraction device comprising:

Images