JP2013201640A - Digital camera - Google Patents

Abstract

A digital camera capable of accurately discriminating between a main subject portion and a non-main subject portion is provided.
An image pickup device for picking up a subject image by a photographing lens and a moving subject portion moved relatively by a predetermined amount or more in two of a plurality of images continuously picked up by the image pickup device. And a subject extraction unit 113 that extracts a non-moving subject part less than a predetermined amount, and a shake that detects the amount of movement of the digital camera 100 in a direction perpendicular to the optical axis of the photographing lens 120 during the imaging of a plurality of images. Based on the amount of movement detected by the detection unit 191 and the shake detection unit 191, a subject determination unit that determines one of the moving subject portion and the non-moving subject portion as the main subject portion and the other as the non-main subject portion. 114.
[Selection] Figure 1

Description

  The present invention relates to a digital camera.

  There is known a method for obtaining a relative movement amount of a subject portion included in a captured moving image. For example, Patent Document 1 describes a moving image processing method for separating a moving image background, that is, a non-animal body and an animal body, based on an optical flow.

JP-A-9-212648

  In the moving image processing method described in Patent Document 1, there is a problem that it is impossible to know which of the separated non-animal body and moving body is the main subject portion.

  According to a first aspect of the present invention, there is provided an image sensor that captures a subject image by a photographing lens, and a moving subject that is relatively moved by a predetermined amount or more in two images out of a plurality of images continuously captured by the image sensor. An extraction means for extracting a portion and a non-moving subject part less than a predetermined amount; a detection means for detecting a movement amount of the digital camera in a direction perpendicular to the optical axis of the photographing lens during imaging of a plurality of images; And determining means for determining one of the moving subject portion and the non-moving subject portion as a main subject portion and determining the other as a non-main subject portion based on the amount of movement detected by the detecting means. It is a digital camera.

  According to the present invention, a main subject portion and a non-main subject portion can be distinguished with high accuracy.

It is a block diagram which shows the structure of the digital camera which concerns on the 1st Embodiment of this invention. FIG. 10 is a schematic diagram for explaining details of determination by a subject determination unit 114. 3 is a flowchart showing moving image shooting processing by the digital camera 100. It is a block diagram which shows the structure of the digital camera which concerns on the 2nd Embodiment of this invention.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of a digital camera according to the first embodiment of the present invention. The digital camera 100 is a lens-integrated digital camera, and includes a photographing lens 120 that forms a subject image on the imaging surface of the imaging element 130. The image sensor 130 is configured by an image sensor such as a CCD or CMOS provided with a plurality of photoelectric conversion elements corresponding to pixels.

  The image sensor 130 captures a subject image formed on the imaging surface and outputs a photoelectric conversion signal (image signal) corresponding to the brightness of the subject image to the control circuit 110. On the imaging surface of the imaging device 130, R (red), G (green), and B (blue) color filters are provided so as to correspond to the pixel positions, respectively.

  The photographing lens 120 includes a plurality of lenses including a shake correction lens (not shown) for correcting image shake of a subject. The shake detection unit 191 detects the amount of movement of the digital camera in the direction perpendicular to the optical axis of the photographic lens 120 and outputs a detection signal to the control circuit 110 and the shake correction unit 190. The shake correction unit 190 corrects the image shake of the subject image by driving the shake correction lens based on the detection signal.

  The control circuit 110 includes a microprocessor and its peripheral circuits, and controls each unit by executing a predetermined control program stored in the flash memory 150 which is a nonvolatile storage medium. The control circuit 110 generates image data based on the output of the image sensor 130 and temporarily stores it in the DRAM 140. The generated image data is stored in a portable storage medium 171 attached to the memory card I / F 170 or displayed on a liquid crystal monitor 161 which is a display device driven by the LCD drive circuit 160.

  Connected to the control circuit 110 is an input device 180 including various operation members provided outside the digital camera 100 such as a release button (not shown). When the user operates each operation member, the input device 180 outputs an operation signal corresponding to the operation to the control circuit 110.

  The control circuit 110 includes a calculation unit 111, a moving image storage unit 112, a subject extraction unit 113, a subject determination unit 114, and an image processing unit 115 in a software form. Each of these units is realized by software when the control circuit 110 executes a predetermined control program stored in the flash memory 150.

  The computing unit 111 computes a motion vector by a well-known method in two images out of a plurality of images continuously captured by the image sensor 130. The subject extraction unit 113 is a moving subject that has moved relatively more than a predetermined amount based on the motion vector calculated by the calculation unit 111 in two of the plurality of images continuously captured by the image sensor 130. A portion and a non-moving subject portion that has moved by less than a predetermined amount are extracted. The subject determination unit 114 determines one of the moving subject part and the non-moving subject part extracted by the subject extraction unit 113 as the main subject part based on the movement amount of the digital camera 100 detected by the shake detection unit 191. The other is determined as a non-main subject portion.

  The image processing unit 115 performs different image processing on the main subject portion and the non-main subject portion based on the determination result by the subject determination unit 114. The moving image storage unit 112 stores, in the storage medium 171, a moving image file including a plurality of images that are continuously captured by the image sensor 130 and subjected to image processing by the image processing unit 115. The moving image file stored by the moving image storage unit 112 is encoded by a predetermined encoding method (compression method) such as MPEG.

(Description of video shooting)
The digital camera 100 of this embodiment has a moving image shooting function. When the user instructs to start moving image shooting using the input device 180, the control circuit 110 causes the image sensor 130 to continuously perform imaging at predetermined intervals (for example, 1/30 second). The control circuit 110 performs various image processes different from those performed by the image processing unit 115 on the image signal output from the image sensor 130 by imaging, and generates image data. The created image data is temporarily stored in the DRAM 140.

  The calculation unit 111 calculates a motion vector with respect to two temporally continuous image data stored in the DRAM 140 by a known method. For example, the whole of one image is divided into a plurality of relatively small blocks, and the position corresponding to each block in the other image is calculated. The method for calculating the motion vector is not limited to this as long as it is a known method. For example, a motion vector can be calculated using a method described in Japanese Patent Laid-Open No. 2000-224593. The motion vector calculation result is stored in the DRAM 140.

  The subject extraction unit 113 extracts the moving subject portion and the non-moving subject portion of the image data stored in the DRAM 140 by a well-known method based on the motion vector calculation result. Specifically, a portion where the motion vector is greater than or equal to a predetermined amount, that is, a portion which is relatively moved by a predetermined amount or more in two consecutive image data is extracted as a moving subject portion. On the other hand, a portion where the motion vector is less than a predetermined amount, that is, a portion that has moved relatively less than a predetermined amount in two consecutive image data is extracted as a non-moving subject portion.

  FIG. 2 is a schematic diagram for explaining the contents of determination by the subject determination unit 114. FIG. 2 shows five pieces of image data P1 to P5 captured continuously. The image data P1 to P5 are captured in the order of P1, P2, P3, P4, and P5. Each of these five image data includes a circular subject portion 10 and a square subject portion 20.

  In these five pieces of image data P1 to P5, the subject portion 10 moves by the movement amount x at every imaging interval. When the movement amount x exceeds the predetermined movement amount, the subject extraction unit 113 extracts the subject portion 10 as the moving subject portion in each of the image data P1 to P5. On the other hand, the other subject portions including the subject portion 20 are not moved at all. Accordingly, the subject extraction unit 113 extracts subject portions other than the subject portion 10 as non-moving subject portions.

  Here, it is assumed that when the image data P1 to P5 illustrated in FIG. 2 is captured, the shake detection unit 191 detects only a movement amount less than a predetermined threshold value. In moving image shooting, it is unlikely to shoot a subject portion (for example, a landscape) that does not move at all with the digital camera 100 fixed (without moving). That is, in FIG. 2, it is considered that the subject portion that the user intends to shoot the moving image is the moving subject portion 10. Accordingly, the subject determination unit 114 determines the subject portion extracted as the moving subject portion by the subject extraction unit 113 as the main subject portion. On the other hand, it is determined that the subject portion extracted as the non-moving subject portion is a non-main subject portion (that is, the background).

  On the other hand, in FIG. 2, when the shake detection unit 191 detects a movement amount equal to or greater than a predetermined threshold, the subject determination unit 114 detects the subject part extracted by the subject extraction unit 113 as a non-moving subject part (ie, subject The subject part other than the subject part 10 such as the part 20) is determined as the main subject part. On the other hand, the subject portion extracted as the moving subject portion (that is, the subject portion 10) is determined to be a non-main subject portion. This is because the subject portion 20 does not move in the image data even though the digital camera 100 is moved greatly during imaging, that is, the user pans the digital camera 100 following the subject portion 20. It is because it is considered.

  As described above, in moving image shooting, the subject determination unit 114 determines that the non-moving subject portion is the main subject portion and moves when the amount of movement detected by the shake detection unit 191 is equal to or greater than a predetermined threshold value. If the amount is less than the predetermined threshold, the moving subject portion is determined as the main subject portion.

  After the determination by the subject determination unit 114 is performed, the image processing unit 115 performs different image processing on the main subject portion and the non-main subject portion on the image data. For example, the white balance of the image is adjusted by paying attention to the main subject portion. Furthermore, an image with an effect of shallow depth of field is created by applying a relatively strong blurring process to the non-main subject portion.

  Finally, the moving image storage unit 112 stores a moving image file including the image data subjected to the image processing by the image processing unit 115 in the storage medium 171. The moving image storage unit 112 of the present embodiment stores a moving image file by an encoding method using a motion vector between frames. This motion vector is calculated by the calculation unit 111. That is, the moving image storage unit 112 stores a moving image file by a predetermined encoding method based on the motion vector calculated by the calculation unit 111.

  FIG. 3 is a flowchart showing the moving image shooting process by the digital camera 100. First, in step S300, the control circuit 110 starts moving image shooting in response to a user instruction. In step S310, the control circuit 110 causes the image sensor 130 to capture a subject image. In step S320, the calculation unit 111 calculates a motion vector in two consecutive image data. In step S330, the subject extraction unit 113 extracts a moving subject portion and a non-moving subject portion.

  In step S340, the subject determination unit 114 determines whether the movement amount of the digital camera 100 detected by the shake detection unit 191 is equal to or greater than a predetermined threshold value. If the movement amount is greater than or equal to the threshold value, an affirmative determination is made, and the process proceeds to step S350. In step S350, the subject determination unit 114 determines that the moving subject portion is a non-main subject portion. On the other hand, if a negative determination is made in step S340, the process proceeds to step S360, and the subject determination unit 114 determines that the moving subject portion is the main subject portion. In step 370, the image processing unit 115 performs different image processing on the main subject portion and the non-main subject portion.

  In step S380, the control circuit 110 determines whether or not the user has instructed to end moving image shooting. If the user has not instructed to finish moving image shooting, the process returns to step S310, and the imaging in step S310 is repeated at predetermined intervals. On the other hand, if the user gives an instruction to end moving image shooting, the process proceeds to step S 390, where the moving image storage unit 112 stores a moving image file including image data stored in the DRAM 140 in the storage medium 171.

According to the digital camera according to the first embodiment described above, the following operational effects can be obtained.
(1) The subject extraction unit 113 includes a moving subject portion that has moved relatively more than a predetermined amount and a non-moving subject that is less than a predetermined amount in two of a plurality of images continuously captured by the image sensor 130. Extract parts. The subject determination unit 114 is non-moving from the moving subject part based on the amount of movement of the digital camera 100 in the direction perpendicular to the optical axis of the photographing lens 120 detected by the shake detection unit 191 during imaging of a plurality of images. One of the subject portions is determined as a main subject portion, and the other is determined as a non-main subject portion. Since it did in this way, a main subject part and a non-main subject part can be distinguished accurately.

(2) The subject determination unit 114 determines that the non-moving subject part is the main subject part when the movement amount detected by the shake detection unit 191 is equal to or greater than a predetermined threshold, and the movement amount is predetermined. If it is less than the threshold value, the moving subject portion is determined as the main subject portion. Since it did in this way, the discrimination of the suitable main to-be-photographed part which took in the user's intention can be performed.

(3) The image processing unit 115 performs different image processing on the main subject portion and the non-main subject portion. Since it did in this way, an effective animation which paid its attention to the main subject part can be photoed.

(4) The calculation unit 111 is used by the subject extraction unit 113 to extract the moving subject portion and the non-moving subject portion, and the moving image storage unit 112 stores the moving image file by a predetermined moving image encoding method. The motion vectors of two images to be used are calculated. Since it did in this way, it is not necessary to perform further calculation in order to extract a moving subject part and a non-moving subject part, and a calculation amount is reduced.

(5) The shake detection unit 191 is used by the shake correction unit 190 to correct image shake of the subject image, and the subject determination unit 114 is used to determine the main subject portion and the non-main subject portion. The amount of movement of the digital camera 100 is detected. Since it did in this way, it is not necessary to add a new sensor for the determination by the subject determination unit 114, and the number of parts and the part cost are reduced.

(Second Embodiment)
FIG. 4 is a block diagram showing a configuration of a digital camera according to the second embodiment of the present invention. In addition, the same code | symbol is attached | subjected about the location similar to the digital camera which concerns on 1st Embodiment shown in FIG. 1, and description is abbreviate | omitted. The digital camera 200 according to the second embodiment includes a focus adjustment unit 292 in addition to each unit of the digital camera 100 according to the first embodiment.

  The focus adjustment unit 292 detects the focus state of the photographic lens 120 and drives the focusing lens included in the photographic lens 120 according to the detection result of the focus state, thereby adjusting the focus of the photographic lens 120. The focus state is detected by a known method such as a contrast method or a phase difference detection method.

  When a focus adjustment instruction is given from the user in the moving image shooting mode, the focus adjustment unit 292 of the present embodiment uses the determination result by the subject determination unit 114 to determine a focus area from the entire subject. Use. That is, the focus adjustment process of the photographing lens 120 is executed based on the main subject portion. For example, when the image data P1 to P5 shown in FIG. 2 are captured by the image sensor 130, a focus adjustment instruction is given from the user, and the subject determination unit 114 determines that the subject portion 10 is the main subject portion by the method described above. In this case, the focus adjustment unit 292 drives the focusing lens so that the subject portion 10 is in focus.

  Further, in the still image shooting mode, the control circuit 110 according to the present embodiment causes the image sensor 130 to capture a subject image at a predetermined interval and causes the liquid crystal monitor 161 to display the imaging result as a so-called through image. The calculation unit 111, the subject extraction unit 113, the subject determination unit 114, and the image processing unit 115 of the present embodiment are the same as those of the first embodiment for image data captured for display as a through image. Process. That is, the main subject portion and the non-main subject portion are determined for the through image, and different image processing is performed. The liquid crystal monitor 161 displays a plurality of image data that has been subjected to image processing by the image processing unit 115 as a through image. When a focus adjustment instruction is given from the user in the still image shooting mode, the focus adjustment unit 292 determines a focus area based on the determination result of the subject determination unit 114 in this through image.

According to the digital camera according to the second embodiment described above, the following functions and effects can be obtained in addition to the functions and effects obtained by the digital camera according to the first embodiment.
(1) On the liquid crystal monitor 161, a plurality of images subjected to image processing by the image processing unit 115 are displayed as through images. Since it did in this way, in still image photography, image processing for supporting photography can be applied to a main subject part and a non-main subject part.

(2) The focus adjustment unit 292 executes focus adjustment processing of the photographing lens 120 based on the main subject portion. Since this is done, it is possible to accurately focus on the main subject portion.

  The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.

(Modification 1)
The subject extraction unit 113 may extract the moving subject portion and the non-moving subject portion based on information other than the motion vector. For example, the moving subject portion and the non-moving subject portion may be extracted by a method using an optical flow described in Patent Document 1.

(Modification 2)
In the first embodiment, the calculation of the motion vector by the calculation unit 111 and the storage of the moving image file by the moving image storage unit 112 are performed based on a plurality of images continuously captured by the image sensor 130. Here, “continuously imaged” does not necessarily mean that the images are consecutive in time series. For example, when imaging is performed every 1 / 60th of a second, the motion vector calculation by the calculation unit 111 may be performed every 1 / 30th of a second. That is, some images may be thinned out without performing motion vector calculation or the like on all captured images. The motion vector calculation may be performed every two frames (30 frames per second), while the image processing by the image processing unit 115 may be performed on all images (60 frames per second). . Alternatively, the motion vector is calculated for all images (60 images per second), while the moving image file stored by the moving image storage unit 112 includes only every second image (30 images per second). You may do it.

(Modification 3)
In the first embodiment, the image processing by the image processing unit 115 may be performed on a moving image file stored in the storage medium 171. In this case, for example, the moving image storage unit 112 may be configured to store a moving image file including the movement amount of the digital camera 100 detected by the shake detection unit 191 in the storage medium 171. Alternatively, the moving image storage unit 112 may be configured to store a moving image file including a determination result by the subject determination unit 114 in the storage medium 171.

(Modification 4)
In the second embodiment, the focus adjustment unit 292 performs focus adjustment using the determination result of the subject determination unit 114 in the through image. That is, the subject determination unit 114 determines a main subject portion and a non-main subject portion for image data captured as a through image. Further, the image processing unit 115 may perform different image processing on the through image in the main subject portion and the non-main subject portion. The image processing performed here may be, for example, image processing similar to the various image processing described in the first embodiment, or may be image processing that emphasizes the outline of the main subject portion.

(Modification 5)
In the second embodiment, in the still image shooting mode, the image processing unit 115 performs image processing on the still image based on the result of the determination performed by the subject determination unit 114 on the through image. May be. That is, when a still image is shot and the still image data is stored in the storage medium 171, the still image data corresponding to the portion determined to be the main subject portion and the portion determined to be the non-main subject portion in the through image. For each region, the image processing unit 115 may perform different image processing.

(Modification 6)
The image processing executed by the image processing unit 115 may be image processing different from that described in the above embodiments. In addition, the subject determination unit 114 may perform determination in consideration of other elements (for example, the size of the subject portion in the entire image data) in addition to the movement amount detected by the shake detection unit 191.

  As long as the characteristics of the present invention are not impaired, the present invention is not limited to the above-described embodiments, and other forms conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention. .

DESCRIPTION OF SYMBOLS 100, 200 ... Digital camera, 110 ... Control circuit, 111 ... Calculation part, 112 ... Moving image memory | storage part, 113 ... Subject extraction part, 114 ... Subject determination part, 115 ... Image processing part, 120 ... Shooting lens, 130 ... Image sensor , 171 ... Storage medium, 180 ... Input device, 190 ... Shake correction unit, 191 ... Shake detection unit, 292 ... Focus adjustment unit

Claims (8)

An image pickup device for picking up a subject image by the taking lens;
Extracting means for extracting a moving subject portion that has moved relatively a predetermined amount or more and a non-moving subject portion that is less than the predetermined amount in two images out of a plurality of images continuously captured by the image sensor. ,
Detecting means for detecting a movement amount of the digital camera in a direction perpendicular to the optical axis of the photographing lens during imaging of the plurality of images;
Determination means for determining one of the moving subject part and the non-moving subject part as a main subject part and determining the other as a non-main subject part based on the amount of movement detected by the detection means;
A digital camera comprising: The digital camera according to claim 1, wherein
The determination unit determines that the non-moving subject part is the main subject part when the movement amount detected by the detection unit is equal to or greater than a predetermined threshold value, and the movement amount is the predetermined amount. The digital camera according to claim 1, wherein the moving subject portion is determined as the main subject portion when the threshold value is less than the threshold value. The digital camera according to claim 1 or 2,
A digital camera further comprising image processing means for performing different image processing on the main subject portion and the non-main subject portion. The digital camera according to claim 3, wherein
A digital camera further comprising storage means for storing, in a storage medium, a moving image file including the plurality of images subjected to image processing by the image processing means. The digital camera according to claim 4, wherein
A calculation means for calculating a motion vector of the two images;
The extraction means extracts the moving subject part and the non-moving subject part based on the motion vector computed by the computing means,
The digital camera characterized in that the storage means stores the moving picture file by a predetermined moving picture coding method based on the motion vector calculated by the calculating means. The digital camera according to any one of claims 3 to 5,
A digital camera further comprising display means for displaying the plurality of images subjected to image processing by the image processing means as through images. The digital camera according to any one of claims 1 to 6,
A digital camera further comprising shake correction means for correcting image shake of the subject image based on the amount of movement detected by the detection means. In the digital camera according to any one of claims 1 to 7,
A digital camera, further comprising: a focus adjustment unit that performs a focus adjustment process of the photographing lens based on the main subject portion.

Images