JP2010041381A - Electronic camera, stereo image generation method, and stereo image generation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide means for acquiring a stereo image without requiring a complicated operation at the time of imaging while making the imaging optical system simpler.
An electronic camera includes an imaging lens, a lens driving unit, an imaging element, and a control unit. The lens driving unit shifts the imaging lens within a plane perpendicular to the optical axis of the imaging lens. The image pickup device picks up an image of the light flux that has passed through the image pickup lens. The control unit obtains a pair of images having parallax by changing the position of the imaging lens at the time of capturing each image.
[Selection] Figure 1

Description

  The present invention relates to an electronic camera suitable for acquiring a stereo image and its peripheral technology.

Conventionally, many proposals have been made on electronic cameras that acquire stereo images (stereoscopic images). As an example, Patent Document 1 discloses a configuration of a camera that acquires a plurality of images with parallax by shifting a mirror included in an imaging optical system.
JP 2003-5311 A

  However, there is room for improvement in the configuration of the above-described Patent Document 1 in that a mirror is required in the imaging optical system in order to obtain a stereo image.

  Accordingly, an object of the present invention is to provide a means for acquiring a stereo image without requiring a complicated operation at the time of imaging while making the imaging optical system simpler.

  An electronic camera according to one aspect includes an imaging lens, a lens driving unit, an imaging element, and a control unit. The lens driving unit shifts the imaging lens within a plane perpendicular to the optical axis of the imaging lens. The image pickup device picks up an image of the light flux that has passed through the image pickup lens. The control unit obtains a pair of images having parallax by changing the position of the imaging lens at the time of capturing each image.

  In the one aspect described above, the imaging lens may include a blur correction lens that optically corrects blurring of the image by movement. The lens driving unit may shift the blur correction lens.

  The electronic camera according to the above aspect may further include a subject distance calculation unit and an image processing unit. The subject distance calculation unit obtains the subject distance of the subject by using the shift amount of the subject position in the pair of images and the shift amount of the imaging lens. The image processing unit generates a stereo pair projection image in which the subject is projected onto a virtual projection plane based on each subject distance.

  The electronic camera may further include a shake detection unit that detects a change in the angle of the optical axis of the imaging lens. Then, the image processing unit may obtain the position of the subject in the projection image in consideration of the angle change that occurs during the acquisition of the pair of images.

  The electronic camera according to the above aspect may further include an image sensor shift unit that shifts the position of the image sensor. Further, the control unit may acquire a pair of images by differentiating the position of the image sensor at the time of capturing each image.

  The electronic camera according to the one aspect may further include a subject distance detection unit that detects a subject distance. The control unit may determine whether to generate the projection image according to the subject distance when the pair of images is acquired.

  The electronic camera according to the aspect described above may further include a shake detection unit that detects a change in the angle of the optical axis of the imaging lens. Further, the control unit may determine whether or not to generate the projection image according to the magnitude of the angle change that occurs during the acquisition of the pair of images.

  The electronic camera according to the one aspect may further include a data output unit that outputs data of the pair of images and data indicating the shift amount of the imaging lens when the pair of images is acquired to an external device. Good.

  A stereo image generation method and a stereo image generation system that generate the projection image using an electronic camera and an image processing apparatus are also effective as specific embodiments of the present invention.

  According to the present invention, a pair of images having parallax can be easily acquired by shifting the imaging lens.

<Configuration of one embodiment>
FIG. 1 is a block diagram illustrating a configuration example of an electronic camera according to an embodiment. The electronic camera of this one embodiment has a stereo image shooting mode as one of the shooting modes. The electronic camera in the stereo image capturing mode captures a pair of images having parallax by shifting the lens in a direction perpendicular to the optical axis of the imaging optical system.

  The electronic camera of one embodiment includes an imaging optical system 11, a first lens driving unit 12, a second lens driving unit 13, a shake detection unit 14, a third lens driving unit 15, an imaging element 16, Image sensor shift unit 17, AFE 18, CPU 19, first memory 20 and second memory 21, recording I / F 22, external I / F 23, wireless communication module 24, monitor 25, and operation unit 26 have. In addition, the 1st lens drive part 12, the 2nd lens drive part 13, the blur detection part 14, the 3rd lens drive part 15, the image pick-up element shift part 17, AFE18, the 1st memory 20, the 2nd memory 21, and recording I / F22 The external I / F 23, the wireless communication module 24, the monitor 25, and the operation unit 26 are each connected to the CPU 19.

  The imaging optical system 11 includes a plurality of lens groups including a zoom lens 11a, a focusing lens 11b, and a shake correction lens 11c. The zoom lens 11a is a lens for optically changing the photographing magnification. The lens position of the zoom lens 11 a is adjusted in the optical axis direction by the first lens driving unit 12. Further, the first lens driving unit 12 incorporates an encoder that detects the position of the zoom lens 11a in the optical axis direction. Information on the lens position of the zoom lens 11a is input to the CPU 19.

  The focusing lens 11b is a lens for performing focus adjustment. The lens position of the focusing lens 11b is adjusted by the second lens driving unit 13 in the optical axis direction. Further, the second lens driving unit 13 incorporates an encoder that detects the position of the focusing lens 11b in the optical axis direction. Information on the lens position of the focusing lens 11b is input to the CPU 19.

  The blur correction lens 11c is a lens that optically corrects image blur due to camera shake by moving the lens. The blur correction lens 11 c is configured to be swingable within a plane perpendicular to the optical axis of the imaging optical system 11.

  The blur detection unit 14 detects a change in the angle of the optical axis of the imaging optical system 11. The shake detection unit 14 includes a vertical angular velocity sensor that detects the pitch of the electronic camera and a horizontal angular velocity sensor that detects the roll of the electronic camera. As each of the above angular velocity sensors, for example, a piezoelectric vibration type angular velocity sensor that detects Coriolis force due to rotation is used. The value of the angular velocity detected by the shake detection unit 14 is input to the CPU 19.

  The third lens driving unit 15 swings the blur correction lens 11 c based on the control of the CPU 19. The third lens driving unit 15 includes a vertical swing mechanism that swings the shake correction lens 11c in the vertical direction and a horizontal swing mechanism that swings the shake correction lens 11c in the horizontal direction.

  The imaging element 16 captures an image formed by the light flux that has passed through the imaging optical system 11 and generates an analog image signal. The output of the image sensor 16 is connected to the AFE 18. Here, the imaging element 16 in the shooting mode captures a still image for recording in response to a full-press operation of a release button described later. Further, the imaging element 16 in the shooting mode continuously takes images for observation (through images) at predetermined intervals even during standby for shooting. Note that the through image acquired in time series is used for moving image display on the monitor 25 and various arithmetic processes by the CPU 19.

  The image sensor shift unit 17 optically corrects image blur due to camera shake by moving the image sensor 16 within a plane perpendicular to the optical axis of the image pickup optical system 11. The image sensor shift unit 17 includes a vertical swing mechanism that swings the image sensor 16 in the vertical direction and a horizontal swing mechanism that swings the image sensor 16 in the horizontal direction.

  The AFE 18 is an analog front end circuit that performs analog signal processing on the output of the image sensor 16. The AFE 18 performs correlated double sampling, image signal gain adjustment, and image signal A / D conversion. The image signal output from the AFE 18 is input to the CPU 19.

  The CPU 19 is a processor that comprehensively controls the operation of each unit of the electronic camera. As an example, the CPU 19 executes an autofocus (AF) calculation and an automatic exposure (AE) calculation based on a through image using a known algorithm. Further, the CPU 19 shifts the shake correction lens 11 c via the third lens driving unit 15 and shifts the image sensor 16 via the image sensor shift unit 17.

  Further, the CPU 19 has a subject distance calculation unit 27 and an image processing unit 28. The subject distance calculation unit 27 geometrically obtains the subject distance of each subject based on the amount of deviation of the subject position in the pair of images in the stereo image shooting mode.

  On the other hand, the image processing unit 28 performs image processing such as color interpolation processing, gradation conversion processing, contour enhancement processing, and white balance adjustment on the image data to be processed. Further, the image processing unit 28 generates a projection image in which each subject is projected on a virtual projection plane based on the subject distance obtained by the subject distance calculation unit 27 in the stereo image shooting mode.

  The first memory 20 is configured by a volatile storage medium (for example, SDRAM). The first memory 20 temporarily stores image data and the like in the pre-process and post-process of image processing.

  The second memory 21 is composed of a nonvolatile storage medium such as a flash memory. The second memory 21 stores programs executed by the CPU 19 and data of various tables. The table stored in the second memory 21 includes a focal length table that indicates the correspondence between the lens position of the zoom lens 11a and the focal length, and a subject distance table that indicates the correspondence between the lens position of the focusing lens 11b and the subject distance. And a determination table used in the stereo image shooting mode.

  Here, the contents of the determination table will be described with reference to FIG. The determination table shows the correspondence between the combination of the focal length and the subject distance and the type of processing in the stereo image shooting mode. There are three types of processing in the stereo image shooting mode described above: “(1) Use the captured image as it is”, “(2): Generate a projection image”, and “(3): Prohibit shooting”. Divided. In FIG. 2, the above processing types are represented by numbers with parentheses.

  The contents of the determination table in FIG. 2 are set according to the magnitude of the parallax generated in the pair of images. For example, when sufficient parallax occurs in the pair of images, the process (1) is set to be selected in the determination table. In addition, when the parallax is small in the pair of images, the process (2) is set to be selected in the determination table. Furthermore, when it can be considered that no parallax occurs in the pair of images, the determination table is set so that the process (3) is selected.

  Here, when the lens is moved in the direction perpendicular to the optical axis, the parallax between images increases when only the value of the focal length is increased. Further, when only the subject distance value is increased when moving the lens in the direction perpendicular to the optical axis, the parallax between images decreases.

  Therefore, in the example of FIG. 2, when the focal length is the longest and the subject distance is the shortest (for example, in the case of macro shooting), the process (1) is selected in the determination table. Further, when the subject distance is made longer or the focal length is made shorter than when the process (1) is selected, the process (2) is selected in the determination table. Further, when the subject distance is made longer or the focal length is made shorter than when the process (2) is selected, the process (3) is selected in the determination table. Therefore, in the example of FIG. 2, when the focal length is the shortest and the subject distance is the longest, the process (3) is selected in the determination table.

  Returning to FIG. 1, the recording I / F 22 has a connector to which an external storage medium 29 is detachably connected. The recording I / F 22 writes / reads data to / from the storage medium 29 connected via the connector. The storage medium 29 includes a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 29.

  The external I / F 23 has a connection terminal of a serial communication standard such as USB (Universal Serial Bus). The external I / F 23 controls data transmission / reception with an external device (such as a computer) connected via a connection terminal in accordance with the communication standard.

  The wireless communication module 24 performs wireless data transmission / reception with an external device (such as a computer) via an antenna. In one embodiment, the wireless communication module 24 performs wireless communication in accordance with a known communication standard such as IEEE 802.11.

  The monitor 25 displays various images in accordance with instructions from the CPU 19. For example, the monitor 25 can perform a moving image display of a finder image by the above-described through image and a reproduction display of a stereo image described later under the control of the CPU 19.

  The operation unit 26 has a plurality of switches that accept operations from the user. The operation unit 26 according to the embodiment includes a release button, a mode dial, a cross key, and a determination button.

  The release button receives from the user an instruction input for starting an AF operation before photographing by a half-press operation and an instruction input for starting an imaging operation by a full-press operation. Further, the mode dial accepts from the user an input for switching the operation mode of the electronic camera. The cross key and the enter button accept various setting operations on the electronic camera from the user. As an example, the CPU 19 switches on / off setting of the projection image generation process at the time of shooting via the cross key and the determination button (note that the projection image generation process at the time of imaging will be described later).

<Description of Operation in One Embodiment>
Hereinafter, an operation example in the stereo image shooting mode of the electronic camera in one embodiment will be described with reference to the flowchart of FIG. 3. The operation of the electronic camera in the stereo image shooting mode is controlled by a program executed by the CPU 19. In the stereo image shooting mode, the CPU 19 monitors the angle change of the optical axis of the imaging optical system 11 by the blur detection unit 14 and stops the blur correction process by the third lens driving unit 15 and the imaging element shift unit 17. To do.

  Here, when the electronic camera is activated in the stereo image capturing mode, the CPU 19 drives the image sensor 16 to start capturing a through image. Then, the CPU 19 performs a moving image display of a finder image based on the through image on the monitor 25. Accordingly, the user can perform framing for determining the shooting composition while referring to the image on the monitor 25. Thereafter, when the user half-presses the release button, the CPU 19 starts a series of processes shown in FIG.

  Step S101: The CPU 19 executes AF by known contrast detection using the through image. Thereafter, the CPU 19 stops the AF control and fixes the focal position of the focusing lens 11b.

  Step S102: The CPU 19 executes a known AE calculation using the through image. Thereafter, the CPU 19 fixes the aperture and shutter speed settings at the time of shooting with the values obtained by the above AE calculation.

  Step S103: The CPU 19 detects the focal length and the subject distance from each lens position of the imaging optical system 11. Specifically, the CPU 19 obtains the lens position information of the zoom lens 11 a from the first lens driving unit 12 and obtains the focal length by referring to the focal length table of the second memory 21. Further, the CPU 19 obtains information on the lens position of the focusing lens 11b from the second lens driving unit 13 and refers to the subject distance table in the second memory 21 to obtain the subject distance to the subject in focus. .

  Step S104: The CPU 19 obtains the type of processing in the stereo image shooting mode from the determination table in the second memory 21 based on the combination of the focal length and the subject distance (S103). Then, the CPU 19 records a flag indicating the type of processing obtained from the determination table in the first memory 20 or the second memory 21.

  Step S105: The CPU 19 refers to the flag (S104) to determine whether or not the type of processing indicated by the determination table is “(3): prohibit photographing”. If the above requirement is satisfied (YES side), the process proceeds to S106. On the other hand, when the above requirement is not satisfied (NO side), the process proceeds to S107.

  Step S106: The CPU 19 outputs a warning display (for example, a message such as “Cannot generate stereo image”) on the monitor 25. And CPU19 complete | finishes the process of the flowchart of FIG.

  Step S107: The CPU 19 determines whether or not the release button has been fully pressed. If the above requirement is satisfied (YES side), the process proceeds to S109. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S108.

  Step S108: The CPU 19 determines whether or not the half-press operation of the release button has been released. If the above requirement is satisfied (YES side), the CPU 19 ends the process of the flowchart of FIG. On the other hand, if the above requirement is not satisfied (NO side), the CPU 19 returns to S107 and repeats the above operation.

  Step S109: The CPU 19 operates the third lens driving unit 15 and the image sensor shift unit 17 to shift the positions of the blur correction lens 11c and the image sensor 16 horizontally such that the positions of the image blur correction lens 11c and the image sensor 16 are more to the left than the center. At this time, the CPU 19 shifts the imaging element 16 to the left by the same amount as the shift amount of the optical axis of the blur correction lens 11c. Note that the movement direction of the shake correction lens 11c and the image sensor 16 in one embodiment is set by the CPU 19 in accordance with how the electronic camera is held (photographing at the normal position, photographing at the vertical position, etc.).

  Step S110: The CPU 19 initializes an integrated value of the angular velocity detected from the shake detecting unit 14, and starts an angular velocity integrating operation. Note that the processing of the CPU 19 in S110 is performed almost simultaneously with an imaging operation in S111 described later.

  Step S111: The CPU 19 drives the image sensor 16 to capture the first frame image (first image) (see FIGS. 4 and 5A). The data of the first image captured in S111 is buffered in the first memory 20 after predetermined image processing is performed by the image processing unit 28.

  Step S112: The CPU 19 operates the third lens driving unit 15 and the image sensor shift unit 17 to shift the positions of the blur correction lens 11c and the image sensor 16 horizontally such that the positions of the image blur correction lens 11c and the image sensor 16 are more to the right than the center, respectively. At this time, the CPU 19 shifts the imaging element 16 to the right by the same amount as the shift amount of the optical axis of the blur correction lens 11c.

  Step S113: The CPU 19 drives the image sensor 16 to capture an image (second image) of the second frame (see FIGS. 4 and 5B). The data of the second image captured in S113 is buffered in the first memory 20 after predetermined image processing is performed by the image processing unit 28.

  In addition, in FIG. 5, the example of a 1st image and a 2nd image is each shown typically. The first image and the second image are images obtained by photographing the same subject, but parallax occurs between the images because the position of each blur correction lens 11c is different in the direction perpendicular to the optical axis.

  Step S114: The CPU 19 ends the angular velocity integration operation started from S110, and acquires the integrated value of each angular velocity. Thereby, CPU19 can detect the angle change of the optical axis of the imaging optical system 11 which arose during acquiring a 1st image and a 2nd image (S111, S113).

  Step S115: The CPU 19 determines whether or not the magnitude of the angle change of the optical axis of the imaging optical system 11 (detected in S114) is larger than the threshold value. Thus, the CPU 19 determines whether or not to generate a projection image according to the magnitude of the angle change that occurs during the acquisition of the first image and the second image. If the above requirement is satisfied (YES side), the process proceeds to S117. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S116.

  Step S116: The CPU 19 refers to the flag (S104) to determine whether or not the type of processing indicated by the determination table is “(2): generate projection image”. Thus, the CPU 19 determines whether to generate a projection image according to the subject distance and the focal distance when the first image and the second image are acquired.

  If the above requirement is satisfied (YES side), the process proceeds to S117. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S119. Note that, in the case of the NO side in S116, the CPU 19 does not generate a projection image, and uses the first image and the second image as they are for a stereo pair.

  Step S117: The CPU 19 determines whether or not the projection image generation process at the time of imaging is set to ON. If the above requirement is satisfied (YES side), the process proceeds to S118. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S120.

  Step S118: The subject distance calculation unit 27 and the image processing unit 28 generate a stereo pair projection image using a pair of images (S111, S113) in response to an instruction from the CPU 19. In one embodiment, the projected image is an image for the right eye of a stereo pair. The first image is used as it is for the left-eye image of the stereo pair.

  Specifically, the subject distance calculation unit 27 and the image processing unit 28 in S118 execute the following processes (a) to (d), respectively.

  (A) The image processing unit 28 performs feature point extraction processing on each of the first image and the second image. As an example, the image processing unit 28 performs a known edge extraction process on the first image and the second image, and uses an intersection (corner) of edges in each image as a feature point.

  (B) The image processing unit 28 associates the feature points of the first image and the feature points of the second image extracted in (a) above. For example, the image processing unit 28 obtains a feature amount for each feature point, and associates feature points that approximate the feature amount between images. As the feature amount of the feature point, for example, information on pixels in the vicinity of the feature point (such as luminance and color difference histograms) can be used.

  (C) The subject distance calculation unit 27 obtains subject distances at the feature points associated in (b) above. Here, the shift amount of the position of the feature point in the first image and the second image becomes larger as the subject distance is shorter. Therefore, the subject distance calculating unit 27 shifts the blur correction lens 11c and the image sensor 16 (distance between the optical axes of the lenses) in the first image and the second image, and the feature points in the first image and the second image. The object distance corresponding to the feature point is geometrically obtained from the amount of deviation of the position of.

  As shown in FIG. 6, when there is an angle change in the optical axis of the imaging optical system 11 during the acquisition of the first image and the second image, the second image is compared with the case where there is no angle change. Since the position of the feature point is shifted, the calculation accuracy of the subject distance is lowered. Therefore, the subject distance calculation unit 27 corrects the displacement amount of each feature point in consideration of the angle change when calculating the subject distance in (C).

  (D) The image processing unit 28 generates a stereo pair projection image from the first image based on the subject distance of each feature point. As an example, first, the image processing unit 28 divides the first image into a plurality of regions by the edges extracted in (a) above. Next, the image processing unit 28 determines the subject distance of each divided region using the subject distance of the feature point obtained in (C) above.

  Then, the image processing unit 28 sets a virtual projection plane (see FIG. 4) on the left side of the imaging plane on which the first image is captured, and each division of the first image based on the subject distance of each division area. A state in which the pixels in the area are respectively projected on the projection plane is obtained. In addition, since the change in the angle of the optical axis of the imaging optical system 11 is added to the subject distance in each divided region, the image processing unit 28 can accurately obtain the position of the subject in the projection image. Thereby, a projection image for the left eye is generated.

  Here, when the images of the divided areas overlap in the projected image, the image processing unit 28 cuts the image of the divided area on the depth side and gives priority to the image of the divided area on the closest side. Further, when a blank portion of the image occurs in the projection image, the image processing unit 28 complements the blank portion based on the image of the divided region located on the depth side of the divided region at the corresponding position in the first image. . For example, the image processing unit 28 interpolates the blank portion with the image of the divided region having a shorter subject distance among the divided regions adjacent to the blank portion. Alternatively, the image processing unit 28 may complement the blank portion image by randomly mixing the images of the adjacent divided regions at a ratio according to the subject distance.

  Step S119: The CPU 19 converts the data of a pair of images for a stereo pair into a stereo image format. As an example, the CPU 19 arranges two images for a stereo pair on the left and right to form a one-frame stereo image. At this time, the CPU 19 determines the left and right positions of the two images according to the viewing method (parallel method or intersection method) of the stereo pair images. Thereafter, the CPU 19 records the above-described stereo image data in the storage medium 29, and ends the processing of the flowchart of FIG.

  Step S120: The CPU 19 generates a stereo image creation file for generating a stereo image in the post-processing step. The stereo image creation file includes (a) data of the first image and the second image, (b) setting information of the imaging optical system 11 at the time of imaging (lens focal length and subject distance information), and (c ) Information on the change in the angle of the optical axis of the imaging optical system 11 that occurred during the acquisition of the first image and the second image; and (d) Information on the distance between the optical axes of the lenses in the first image and the second image. , Is included. Thereafter, the CPU 19 records the above-described stereo image creation file in the storage medium 29, and ends the process of the flowchart of FIG.

  The electronic camera reads the stereo image creation file (S120) from the recording I / F 22, and causes the subject distance calculation unit 27 and the image processing unit 28 to execute the same processing as S118, thereby performing the same processing as S119. A stereo image can be generated afterwards (the content of each process overlaps with the above steps, and the description is omitted).

  Hereinafter, the operational effects of the embodiment will be described. The electronic camera according to one embodiment captures the first image and the second image by shifting the blur correction lens 11c in a plane perpendicular to the optical axis of the imaging optical system 11, and has a pair having these parallaxes. A stereo image is generated from the images. Therefore, in one embodiment, a stereo image can be easily acquired by the configuration of a general blur correction optical system. In one embodiment, the position of the image sensor 16 is also shifted, so that a wider photographing range in a stereo image can be secured.

  In addition, the electronic camera of one embodiment obtains the subject distance of each subject from the first image and the second image, and generates a projection image for a stereo pair in which each subject is projected on a virtual projection plane. As a result, in one embodiment, a high-quality stereo image suitable for stereoscopic vision can be generated even when a large parallax cannot be obtained by moving the lens, for example, when shooting a relatively distant subject. . In one embodiment, since a projection image is generated in consideration of an angle change that occurs during the acquisition of the first image and the second image, the appearance is suitable for stereoscopic viewing even if the camera shakes during shooting. A good stereo image can be generated.

  In addition, the electronic camera of one embodiment can generate a stereo image from a stereo image creation file in a post-processing step. In this case, since it is not necessary to perform time-consuming image processing at the shooting site, the convenience of the user at the time of shooting is further improved.

<Supplementary items of the embodiment>
(1) In the above embodiment, an example has been described in which an electronic camera is used as an image processing apparatus to generate a stereo image in a post-processing step from a stereo image creation file. However, an image processing apparatus that generates a stereo image from a stereo image creation file in a post-processing step is not limited to an electronic camera. For example, a personal computer may be made to function as the subject distance calculation unit 27 and the image processing unit 28 by software to generate a stereo image from a stereo image creation file. The transfer of the stereo image creation file between the electronic camera and the image processing apparatus may be performed by exchanging the storage medium 29 or by data communication via the external I / F 23 or the wireless communication module 24.

  (2) In the above embodiment, the CPU 19 may adjust the amount of movement of the optical axis at the time of shooting according to the subject distance detected from the imaging optical system 11. For example, the CPU 19 may increase the movement amount of the optical axis when the distance to the focused subject is long. Further, the CPU 19 may reduce the amount of movement of the optical axis when the distance to the focused subject is short.

  (3) In the above embodiment, the example in which the position of the image sensor 16 is shifted together with the lens has been described, but it goes without saying that the present invention can also be applied to an electronic camera without a mechanism for shifting the image sensor 16. In the above embodiment, the image sensor and the lens may be combined into one block and shifted perpendicular to the optical axis.

  (4) The stereo image in S119 of the above embodiment is merely an example, and other known stereo images can of course be generated in the present invention. For example, the CPU 19 may generate stereo image data in which left-eye images and right-eye images are alternately displayed corresponding to the liquid crystal shutter. Further, the CPU 19 may generate an anaglyph image having two images of red and blue and different colors as a stereo image. Further, the CPU 19 may generate a stereo image for stereoscopic television in which left-eye images and right-eye images are alternately arranged in a vertical stripe pattern.

  (5) In the above embodiment, an example of generating a stereo image of a still image has been described, but the present invention can also be applied to a case of generating a moving image of a stereo image. Note that when shooting a moving image, the CPU 19 may save the right-eye image and the left-eye image as separate files. In addition, when performing moving image shooting, the CPU 19 may alternately store the right-eye image and the left-eye image in one file.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

1 is a block diagram illustrating a configuration example of an electronic camera according to an embodiment. Schematic diagram showing an example of a judgment table Flow chart showing an example of operation of the electronic camera in stereo image capture mode The figure which shows the various positional relationship at the time of imaging | photography of a 1st image and a 2nd image. (A) The figure which shows an example of a 1st image, (b) The figure which shows an example of a 2nd image The figure which shows the case where there exists an angle change of an optical axis at the time of imaging | photography of a 1st image and a 2nd image

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Imaging optical system, 11a ... Zoom lens, 11b ... Focusing lens, 11c ... Blur correction lens, 12 ... 1st lens drive part, 13 ... 2nd lens drive part, 14 ... Blur detection part, 15 ... 3rd lens drive , 16 ... Imaging device, 17 ... Imaging device shift unit, 18 ... AFE, 19 ... CPU, 20 ... First memory, 21 ... Second memory, 22 ... Recording I / F, 23 ... External I / F, 24 ... Wireless communication module, 25 ... monitor, 26 ... operation unit, 27 ... subject distance calculation unit, 28 ... image processing unit, 29 ... storage medium

Claims (10)

An imaging lens;
A lens driver that shifts the imaging lens in a plane perpendicular to the optical axis of the imaging lens;
An image sensor that captures an image of a light beam that has passed through the imaging lens;
A control unit that obtains a pair of images having parallax by differentiating the position of the imaging lens at the time of capturing each image;
An electronic camera comprising: The electronic camera according to claim 1,
The imaging lens includes a blur correction lens that optically corrects blur of an image by movement,
The electronic camera, wherein the lens driving unit shifts the blur correction lens. The electronic camera according to claim 2,
A subject distance calculation unit for obtaining a subject distance of the subject using a shift amount of the subject position in the pair of images and a shift amount of the imaging lens;
An electronic camera, further comprising: an image processing unit configured to generate a stereo pair projection image obtained by projecting the subject onto a virtual projection plane based on each subject distance. The electronic camera according to claim 3.
Further comprising a blur detection unit for detecting an angle change of the optical axis of the imaging lens;
The electronic camera, wherein the image processing unit obtains the position of the subject in the projection image in consideration of the change in angle generated during the acquisition of the pair of images. The electronic camera according to any one of claims 1 to 4,
An image sensor shift unit for shifting the position of the image sensor;
The said control part makes the position of the said image pick-up element differ at the time of imaging of each image, and acquires the said pair of images, The electronic camera characterized by the above-mentioned. The electronic camera according to claim 3.
A subject distance detection unit for detecting the subject distance;
The electronic camera according to claim 1, wherein the control unit determines whether to generate the projection image according to the subject distance when the pair of images are acquired. The electronic camera according to claim 3.
Further comprising a blur detection unit for detecting an angle change of the optical axis of the imaging lens;
The electronic camera according to claim 1, wherein the control unit determines whether or not to generate the projection image according to a magnitude of the angle change that occurs during acquisition of the pair of images. The electronic camera according to claim 1,
An electronic camera, further comprising: a data output unit that outputs data of the pair of images and data indicating a shift amount of the imaging lens when the pair of images are acquired to an external device. An imaging step of acquiring a pair of images having parallax with an electronic camera by differentiating the position of the imaging lens at the time of capturing each image;
A data reading step in which the image processing device acquires the data of the pair of images and the data indicating the shift amount of the imaging lens when the pair of images is acquired from the electronic camera;
A subject distance calculation step in which the image processing apparatus obtains a subject distance of the subject using a shift amount of a subject position in the pair of images and a shift amount of the imaging lens;
A projection image generating step for generating a stereo pair projection image in which the image processing apparatus projects the subject on a virtual projection plane based on the subject distance;
A stereo image generation method comprising: A stereo image generation system including an electronic camera and an image processing device,
The electronic camera is
An imaging lens;
A lens driver that shifts the imaging lens in a plane perpendicular to the optical axis of the imaging lens;
An image sensor that captures an image of a light beam that has passed through the imaging lens;
A control unit that obtains a pair of images having parallax by differentiating the position of the imaging lens at the time of capturing each image;
A data output unit that outputs data of the pair of images and data indicating a shift amount of the imaging lens when the pair of images are acquired to the image processing device;
The image processing apparatus includes:
A subject distance calculation unit for obtaining a subject distance of the subject using a shift amount of the subject position in the pair of images and a shift amount of the imaging lens;
An image processing unit that generates a stereo pair projection image obtained by projecting the subject on a virtual projection plane based on each subject distance.

Images