JP2004289383A - Imaging device, image data generation method, image data processing device, and image data processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To flexibly correct the distance in focus regarding the photographed image. <P>SOLUTION: The image pickup device 1 photographs images with the same field angle and different distances in focus by one release operation. It can falsely generate the image in the case of changing the distance in focus on the basis of the images and lens information after the photographing. Accordingly, it is possible to flexibly change the distance in focus such as moving the focus forward and backward ex post and changing the depth of field in the photographed images. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image capturing apparatus, an image data generating method, an image data processing apparatus, and an image data processing program for image data including distance information.
[0002]
[Prior art]
2. Description of the Related Art In recent years, digital cameras that digitally shoot an image using an image sensor have become widespread.
A digital camera has a feature that it is easier to correct a photographed image after photographing than a film camera. That is, in a film camera, the image quality of a printed photograph greatly depends on the conditions at the time of shooting, and although it is possible to correct the color tone and the exposure amount after shooting, the image quality is greatly affected. It is impossible to correct the focus shift (shift of the focusing distance). On the other hand, in a digital camera, in addition to correcting the color tone and the exposure amount after shooting, it is also possible to correct the out-of-focus by using a technique such as an unsharp mask or a blur.
[0003]
Here, conventionally, in order to more reliably photograph a high-quality photograph, a subject is photographed a plurality of times.
In particular, in the case of a film camera, in order to prevent a failure in photographing due to overexposure or underexposure, the exposure, by changing the aperture, the shutter speed, or the light emission amount of the flash by a predetermined amount with respect to the imaging conditions assumed to be appropriate. A plurality of shots are performed with different amounts.
[0004]
As a result of photographing in this manner, the plurality of developed photographs have different image quality, and it is possible to select an appropriate one from them.
In digital devices such as digital cameras and scanners, the same subject can be photographed a plurality of times under the same conditions, and the photographed data can be averaged to smooth random noise generated in the imaging device. Is being done.
[0005]
Further, Japanese Patent Application Laid-Open No. 2001-238128 discloses that an appropriate subject having a wide dynamic range is obtained by shooting a plurality of images of the same subject under different shooting conditions, extracting and combining appropriate exposure regions of the shot images. A technique for obtaining a saturation image has been disclosed. Although the present technology relates to an exposure amount, a similar method can be applied to a focus position.
[0006]
[Patent Document 1]
JP 2001-238128 A
[0007]
[Problems to be solved by the invention]
However, the technique described in Japanese Patent Application Laid-Open No. 2001-238128 generates a single image that is assumed to be optimal as a result of a predetermined process from a plurality of captured images. An image obtained as a result of the processing is uniquely determined.
[0008]
In other words, the technique described in Japanese Patent Application Laid-Open No. 2001-238128 is a technique for complementing a defect of an image obtained by one shooting by using another image obtained by performing a plurality of shootings. Yes, this was a common concept in other prior arts.
On the other hand, when the focus position intended at the time of photographing is not always appropriate in the photographed image, or when it is desired to change the in-focus range (depth of field) in the image, the focus is adjusted after the fact. A request to arbitrarily change the focal length may occur.
[0009]
In such a case, in the related art, for an out-of-focus area, the focus can be corrected, but in the image, the focus is shifted back and forth, and the depth of field is changed. It has been difficult to make a correction to change the focusing distance afterwards.
In a digital camera, since the area of the image sensor is small, it is necessary to stop down the light beam passing through the lens so as to have a constant area. For this reason, the depth of field tends to be deep, which is suitable for focusing on the entire screen like a landscape photograph, but focusing only on the main subject like a portrait It was difficult to blur. That is, in the digital camera, there is a problem that it is difficult to flexibly change the focusing distance at the time of shooting as compared with the film camera.
[0010]
An object of the present invention is to make it possible to flexibly correct a focusing distance of a captured image.
[0011]
[Means for Solving the Problems]
In order to solve the above problems, the present invention provides
An image capturing apparatus that captures light from a subject that has passed through a lens with an image capturing element and generates a digital image. The image capturing apparatus includes a plurality of images (for example, a plurality of images set at different focal lengths at the same angle of view corresponding to a release operation). A photographing means (for example, the lens unit 10 and the camera unit 20 in FIG. 1) for continuously photographing “group images” in the embodiment of the present invention, and photographing conditions of each of the plurality of images. The photographing condition acquiring means (for example, the lens unit 10 and the central processing unit 22 in FIG. 1) and the characteristics of the lens (for example, the transfer function of the lens 11 in the embodiment of the invention) or the characteristics of a predetermined lens (for example, Transfer function of a lens selected by a user such as a lens having a shallower depth of field than the provided lens), a plurality of images at different focal lengths, and a plurality of images. Image adjustment means (e.g., FIG. 5) for changing at least one of the focal length and the depth of field of a captured image based on the shooting conditions (for example, the focusing distance at the time of shooting). One lens information storage device 13 and a central processing unit 22).
[0012]
Further, the image adjusting means includes a focused area extracting means (for example, a central processing unit 22 in FIG. 1) for extracting a focused focused area in each of the plurality of images, and a characteristic of the lens or a predetermined characteristic. Focusing area adjusting means (for example, the lens information storage device 13 and the central processing unit 22 in FIG. 1) that changes the image of each of the focusing regions based on the characteristics of the lens and the shooting conditions. It is characterized by.
[0013]
Further, the in-focus area extracting means determines which of the plurality of images is in a more accurate in-focus state for each area in the angle of view of the plurality of images, and It is characterized in that an image of each region determined to be present is extracted.
Further, the imaging means sets the imaging conditions so that the depth of field range of the plurality of images is continuous.
[0014]
Further, based on each area extracted by the in-focus area extracting means, one image having the same angle of view as the plurality of images is generated, and the in-focus image storing means (for example, FIG. One memory unit 30) is further provided.
Further, the image processing apparatus further includes a group image storage unit (for example, the memory unit unit 30 in FIG. 1) that stores a plurality of images captured by the imaging unit in association with each other.
[0015]
Further, when the image is changed by the image adjusting means, an image of an out-of-focus area is generated based on an image other than the focused area in the plurality of images.
Further, the image adjustment means, when changing the focal distance or the depth of field of the captured image, the image assumed when the focal distance or the depth of field is changed at the time of shooting, physically It is characterized in that it is generated in a pseudo manner in accordance with various changes.
[0016]
Further, when changing the focal length or the depth of field of the photographed image, the image adjusting means may change the focal length or the depth of field for an arbitrary region within the angle of view to an arbitrary focal length or a depth of field. It is characterized by being possible.
That is, it is also possible to focus on the far and near areas and to blur the focus on the intermediate distance area without being restricted by the physical change mode.
[0017]
Also, the present invention
An image data generation method for generating image data that can be displayed on a display device,
Shooting a plurality of images set at different focusing distances at the same angle of view, for each area within the angle of view, determine which of the plurality of images is in a more accurate focus state, for each area, It is characterized in that image data including information on the color of the subject and information on the distance is generated by associating the information with the focusing distance of the image determined to be in a more accurate focusing state.
[0018]
Also, the present invention
An image data processing device capable of displaying an image based on digital image data, the image data processing device including: a focus distance of an image related to the image data based on image data including information on a color of a photographed subject and information on a distance. Alternatively, an image adjusting means for changing at least one of the depth of field afterward is provided, and an image of an arbitrarily set focusing distance or depth of field can be displayed.
[0019]
The image adjustment means includes a focus area extracting means for extracting a focused area in each of the plurality of images, a characteristic of a lens used for photographing or a characteristic of a predetermined lens, A focusing area adjusting unit that changes an image of each of the focusing areas based on the condition.
Further, the in-focus area extracting means determines which of the plurality of images is in a more accurate in-focus state for each area in the angle of view of the plurality of images, and The image of each area determined to be present may be extracted.
[0020]
The image processing apparatus further includes a focused image storage unit that generates one image having the same angle of view as the plurality of images based on each region extracted by the focused region extraction unit, and stores the one image. It may be.
Further, the image processing apparatus may further include a group image storage unit that stores a plurality of captured images in association with each other.
[0021]
When the image is changed by the image adjustment unit, an image of an out-of-focus area may be generated based on an image other than the focused area in the plurality of images.
Further, the image adjustment means, when changing the focal distance or the depth of field of the captured image, the image assumed when the focal distance or the depth of field is changed at the time of shooting, physically It may be generated in a pseudo manner in accordance with various changes.
[0022]
Further, the image adjusting means, when changing the focal distance or the depth of field of the photographed image, for an arbitrary area within the angle of view, by changing to an arbitrary focal distance or an arbitrary depth of field Is also good.
Also, the present invention
An image data processing program for displaying an image based on digital image data, the image data processing program focusing on an image related to the image data based on image data including information on a color of a photographed subject and information on a distance. At least one of the distance or the depth of field is realized by a computer with an image adjustment function of changing the ex post facto, and an image of an arbitrarily set focusing distance or a depth of field can be displayed. I have.
[0023]
The image adjustment function includes a focus area extraction function for extracting a focused area in each of the plurality of images, a characteristic of a lens used for photographing or a characteristic of a predetermined lens, A focusing area adjusting function for changing an image of each of the focusing areas based on the condition.
The in-focus area extracting function may determine which of the plurality of images is in a more accurate in-focus state for each area in the angle of view of the plurality of images, and in the more accurate in-focus state. The image of each area determined to be present may be extracted.
[0024]
Further, the apparatus further includes a focused image storage function of generating one image having the same angle of view as the plurality of images based on each area extracted by the focused area extraction function, and storing the one image. It may be.
Further, a group image storage function for storing a plurality of captured images in association with each other may be further provided.
[0025]
In the case where an image is changed by the image adjustment function, an image of an out-of-focus area may be generated based on an image other than a focused area in the plurality of images.
Further, the image adjustment function, when changing the focus distance or depth of field of the captured image, the image assumed when the focus distance or depth of field is changed at the time of shooting, the physical image It may be generated in a pseudo manner in accordance with various changes.
[0026]
Further, the image adjustment function, when changing the focus distance or depth of field of the captured image, for any area within the angle of view, by changing to any focus distance or depth of field Is also good.
According to the present invention, images of different focusing distances are taken at the same angle of view by the release operation, and the images and the characteristics of the lens used for shooting or the characteristics of other lenses and shooting conditions are taken. Based on this, an image in the case where the focusing distance is changed can be generated in a pseudo manner after photographing.
[0027]
Therefore, in a captured image, it is possible to flexibly change the focus distance, such as shifting the focus back and forth or changing the depth of field.
[0028]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of an imaging device according to the present invention will be described with reference to the drawings.
The imaging apparatus according to the present invention continuously captures images focused on different positions at the same angle of view in response to a single release operation. Then, based on a plurality of captured images, it is possible to perform post-correction correction with a change in focusing distance, such as a correction for changing the focus of the captured image or a correction for changing the depth of field. It is a simple imaging device.
[0029]
First, the configuration will be described.
FIG. 1 is a block diagram illustrating a functional configuration of an imaging device 1 to which the present invention has been applied.
In FIG. 1, the imaging apparatus 1 includes a lens unit 10, a camera unit 20, and a memory unit 30. Each of these units can be either an integral structure or a detachable structure, but here, the description will be made assuming that the structure is detachable.
[0030]
The lens unit 10 further includes a lens 11, a diaphragm device 12, and a lens information storage device 13.
The lens 11 is an optical system including a plurality of lenses, and converges reflected light from a subject to form an image.
The diaphragm device 12 has a mechanism capable of changing the opening area by a plurality of blades, and adjusts the amount of light passing through the lens 11. The aperture area of the aperture device 12 is controlled by the central processing unit 22 of the camera unit 20.
[0031]
The lens information storage device 13 stores various information related to the lens 11 such as the transfer function of the lens 11, the depth of field corresponding to the aperture and the focusing distance, the current aperture value and the open aperture value, the focusing distance, and the focal length of the lens. Information is stored as lens information. This lens information is output to the camera unit 20 at the time of photographing or the like.
Here, the transfer function of the lens 11 is a function indicating the relationship between light from a subject incident on the lens 11 and its image formation, and is specified based on the optical characteristics of the lens 11. Therefore, the depth of field with respect to the aperture and the focusing distance can be calculated from the transfer function of the lens 11, but since the calculation involves a complicated calculation, a value calculated in advance is stored. Shall be. However, it is also possible to sequentially calculate the depth of field with respect to the aperture and the focusing distance.
[0032]
The camera unit 20 further includes an input device 21, a central processing unit 22, a shutter device 23, an image sensor 24, a main storage device 25, an auxiliary storage device 26, and a display device 27. .
The input device 21 includes a shutter button pressed by a user to capture an image, and a signal for pressing the shutter button is output to the central processing unit 22. Further, the input device 21 includes various buttons for setting at the time of photographing and processing the photographed image, and operation signals of these buttons are also output to the central processing unit 22. As various buttons, for example, the input device 21 includes a zoom button for adjusting an enlargement ratio when shooting, an aperture setting button for setting an aperture value, a shutter speed setting button for setting a shutter speed, A mode setting button or the like for changing a shooting mode provided in the imaging apparatus 1 is provided. Further, the input device 21 includes a focus position button for changing a focusing distance of a shot image and a depth of field adjustment button for changing a depth of field in a post-shooting correction process described later. Have been.
[0033]
The central processing unit 22 controls the entire image capturing apparatus 1, and reads, for example, an image capturing processing program or a post-photographing correction processing program described later from the auxiliary storage device 26, and executes the image capturing processing or the post-photographing correction processing. Alternatively, various programs for controlling the imaging apparatus 1 are read from the auxiliary storage device 26, and the respective programs are executed.
[0034]
Further, the central processing unit 22 receives a signal corresponding to an operation on the input device 21 and executes a predetermined process corresponding to each operation.
The shutter device 23 includes a mechanism for exposing the light passing through the lens 11 to the image sensor 24 for a predetermined time by performing an opening / closing operation. Note that the shutter device 23 performs an opening / closing operation based on an instruction signal input from the central processing unit 22 in response to the pressing of the shutter button.
[0035]
The imaging device 24 includes an element such as a CCD (Charge Coupled Diode) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. Is image data.
The main storage device 25 is configured by a memory such as an SDRAM (Synchronous Dynamic Random Access Memory), and temporarily holds various data generated when the imaging apparatus 1 operates.
[0036]
The auxiliary storage device 26 is configured by a nonvolatile memory such as a flash ROM, and stores an image photographing processing program, a post-photographing correction processing program, and various programs for controlling the image capturing apparatus 1.
The display device 27 is a small display device such as an LCD (Liquid Crystal Display), and displays an image captured by the imaging device 1, an image stored in the memory unit 30, and the like.
[0037]
The memory unit 30 is configured to include a removable storage medium such as a memory card, and can store a captured image, an image generated by a post-imaging correction process, and the like.
As described above, the imaging device 1 according to the present invention includes the lens unit 10 and the memory unit 30 which are detachable from the camera unit 20. The lens information about the lens 11 in the detachable lens unit 10 is recorded in a lens information storage device 13 built in the lens unit 10. The camera unit section 20 reads the lens information recorded in the lens information storage device 13 and stores it in the main storage device or the auxiliary storage device.
[0038]
Next, the operation will be described.
In describing the operation of the present invention, the basic concept of the present invention will be described step by step.
The imaging device 1 according to the present invention stores lens information including a transfer function of the lens 11 as lens information.
[0039]
Therefore, in the imaging apparatus 1, when the focusing distance is determined, a characteristic that an image of a subject existing at an arbitrary distance changes (hereinafter, referred to as “defocus characteristic”) is grasped.
FIG. 2 is a diagram showing defocus characteristics at various focusing distances. Note that FIG. 2 shows a case where elements other than the focusing distance are defined under specific conditions, in which the horizontal axis represents distance and the vertical axis represents image parameters. The image parameters are each element representing each pixel of the image, such as contrast, color or luminance. Further, in FIG. 2, the abscissa coordinate of the vertex of each characteristic curve indicates the focusing distance.
[0040]
As shown in FIG. 2, in the case of a specific focusing distance (when focusing on one characteristic curve), images of subjects located at various distances change as the focusing distance is increased (so-called out-of-focus state). ).
Further, when the focusing distance changes (when a series of different characteristic curves are noticed), the image of the subject located at a specific distance changes as the distance from the focusing distance increases.
[0041]
Furthermore, as the focusing distance is closer to the imaging device 1, the characteristic curve becomes steeper, and the change becomes more remarkable.
As described above, by capturing an image at a certain focusing distance in a state where the change state of the image is grasped, actual data corresponding to one of the characteristic curves shown in FIG. 2 is obtained.
[0042]
For example, it is assumed that an image is captured at three focusing distances of 2 m, 10 m, and 500 m. Then, image data corresponding to the three focusing distance characteristic curves among the characteristic curves shown in FIG. 2 is obtained.
FIG. 3 is a diagram showing three characteristic curves from which image data has been acquired.
In FIG. 3, for an object located within the depth of field based on three focusing distances, images that can be regarded as being in focus have been acquired. Further, with respect to these images, it is possible to estimate the manner in which the images change when the focusing distance is changed.
[0043]
In addition, since the ranges of the depth of field in the three images are continuously distributed with each other, at least one image that can be regarded as being in focus has been obtained for all the subjects within the angle of view. It will be.
Therefore, for each region of the image, it is determined whether the image at the focusing distance of 2 m, 10 m, or 500 m has the best image quality (focus is more accurately achieved). The determined distance is associated with each area.
[0044]
That is, as shown in FIG. 4, the distance of each subject within the angle of view is classified into one of 2 m, 10 m, and 500 m based on the image quality of the captured image, and I reckon.
Here, paying attention to a certain distance in FIG. 2, the characteristics when the focusing distance is changed can be grasped from a plurality of characteristic curves shown in FIG. For example, for a subject at a distance of 10 m, when the focusing distance is changed, it can be understood that the image changes as shown in FIG.
[0045]
Accordingly, in the case where the focusing distance is changed for the regions of the respective subjects classified as shown in FIG. 4, since the respective regions are regarded as having the same distance, FIG. As shown, it can be changed according to similar characteristics for each region.
FIG. 6 is a diagram illustrating a state in which the image of each region changes correspondingly when the focal distance of the captured image is changed.
[0046]
That is, based on the characteristics as shown in FIG. 5, by changing the focusing distance after shooting while associating the images of the respective regions, an image similar to the one obtained by changing the focusing distance during shooting is simulated. Can be generated dynamically.
Therefore, in the present invention, a plurality of images having different focusing distances are photographed by a single release operation, and the focusing distance is changed in the image to be corrected after photographing based on the plurality of photographed images. It is possible to make.
[0047]
At this time, the image generated when the focal distance is changed after shooting is based on the actually shot image, and the image shot with the focus distance actually changed is reproduced more accurately (Simulation generation). Further, by increasing the number of images to be photographed in one release operation, it is possible to more accurately reproduce an image obtained by changing the focusing distance after photographing.
[0048]
Hereinafter, a specific operation of the imaging device 1 to which the present invention is applied will be described.
First, the image photographing process will be described.
FIG. 7 is a flowchart illustrating an image photographing process performed by the imaging apparatus 1. The image photographing process is started in response to the depression of the shutter button.
In FIG. 7, when the image photographing process is started, the imaging apparatus 1 performs photographing by a release operation according to photographing conditions such as a distance to a subject (focusing distance), a focal length of the lens 11, an aperture value, and the like. A photographing number determination process (described later) for determining the number of a plurality of images (hereinafter, referred to as “group images”) is executed (step S1). In step S1, the focal distance of each image to be photographed is also determined so that the depth of field of the images to be photographed is continuous with each other in accordance with the number of group images.
[0049]
Next, the imaging device 1 shoots the group image determined in step S1, and adds a focusing distance to the shot image as data (step S2).
Then, the imaging apparatus 1 determines whether or not all the group images have been shot (step S3). If it is determined that all of the group images have not been shot, the process proceeds to step S2. .
[0050]
On the other hand, when it is determined in step S3 that the capturing of all the group images has been completed, the imaging apparatus 1 extracts a focused region from each of the captured group images (step S4).
Here, as a method for determining whether or not the captured image is in focus, for each pixel, an image captured at a different focusing distance is compared, and the contrast between adjacent pixels is maximized. There is a known method for selecting an item.
[0051]
Next, the imaging device 1 combines the regions extracted in step S4 to generate an image in which all the subjects within the angle of view are in focus (hereinafter, referred to as a “focused image”) ( Step S5). It should be noted that data of the focal distance of the area to which the pixel belongs is added to each pixel of the focused image.
Then, the imaging device 1 ends the image photographing process.
[0052]
As described above, by executing the image photographing process, for example, in the case of the photographing conditions shown in FIG. 8, the images of FIGS. 9A to 9C are photographed as a group image, and each of these images is The images (a) to (c) are extracted.
Then, based on the images shown in FIGS. 10A to 10C, the image shown in FIG. 11 is generated as a focused image.
[0053]
The data amount of the focused image shown in FIG. 11 is smaller than the sum of the data amounts of the images in FIGS. 9A to 9C. Therefore, in order to refer to the post-imaging correction processing described later, storing the focused image shown in FIG. 11 can use the storage capacity more efficiently.
However, it is also possible to store the images of FIGS. 9A to 9C, respectively. In this case, the image other than the region extracted in step S4, that is, the image of the subject out of focus is also included. Since the image is stored, it is possible to refer to an image in an out-of-focus area in a post-imaging correction process described later.
[0054]
Next, the number-of-shots determination process, which is a sub-flow in the above-described image shooting process, will be described.
The number-of-shots determination process is a process for determining the number of images to be captured as a group image. In order to determine the number of group images, the distance to the subject and the depth of field are important factors. First, these relationships will be described.
[0055]
FIG. 12 is a diagram showing the relationship between the distance to the subject and the position range of the depth of field (range of focus). FIG. 12A shows the case where the focal length of the lens 11 is long. (B) shows a case where the focal length of the lens 11 is short.
In general, the longer the focal length of the lens, the shallower the depth of field, and the smaller the aperture value (opening the aperture), the shallower the depth of field. Further, the shorter the distance to the subject, the shallower the depth of field.
[0056]
Therefore, in order to capture an in-focus image for all subjects included in the angle of view, each subject needs to be included at least once within the depth of field with respect to the focusing distance.
12A and 12B, the width of each line segment indicates the depth of field, and each subject within the angle of view is included at least once within the depth of field based on the focusing distance. This means that each line segment indicating the depth of field covers the entire range where the subject exists on the horizontal axis.
[0057]
That is, according to FIG. 12A, in the case of a shooting condition with a shallow depth of field, it is possible that a small number of images (here, seven images) can cover the entire angle of view at least within the depth of field. Understand. Conversely, according to FIG. 12B, it is understood that a large number of images (here, four images) are required in the case of a shooting condition with a large depth of field.
In order to obtain a more accurate in-focus image, it is ideal to change the in-focus distance at small intervals and to shoot a large number of group images. However, as the number of shots increases, the time required to shoot all of the group images increases, and it becomes necessary to hold the imaging device 1 in the same posture for a long time, and the required storage capacity also increases. Will increase.
[0058]
Therefore, in the photographing number determination process described below, group images are photographed in the range of about 3 to 10 images, and all the subjects within the angle of view are at least once in the depth of field based on the focusing distance. It is assumed that settings are included in the URL.
At this time, regarding the image at which focusing distance is to be photographed, a method of photographing by setting the focusing distance to a predetermined interval as shown in FIGS. 12A and 12B is possible. The method is suitable when the objects within the angle of view are continuously distributed from a short distance to a long distance.
[0059]
Further, when the main subjects present in the angle of view are distributed at discrete distances, for example, as shown in FIG. 8, a person is located at a short distance, a mountain is located at a distance of several hundred meters, and an intermediate distance between them. When buildings are distributed, the focal distance is set for the main subjects (persons, buildings, and mountains), and an image is taken by setting only those focal distances. It is possible to capture an image so as to complement the focal length.
[0060]
Hereinafter, the specific processing content of the photograph number determination processing will be described.
FIG. 13 is a flowchart illustrating the number-of-shots determination process performed by the imaging apparatus 1. The photographing number determination process is started in response to being called in step S1 of the image photographing process.
In FIG. 13, when the photographing number determination process is started, the imaging device 1 determines whether or not there is a change in the state of the lens 11 stored in the lens information storage device 13 (step S101).
[0061]
Here, the change of the state of the lens means that the focal length of the lens, the aperture value, and the distance to the subject change, and the AE (automatic exposure control) function automatically controls the aperture when a zoom operation is performed. This also includes cases where it is changed in some way.
Specifically, a change in the focal length of the lens means that the focal length is changed or the lens is replaced in the zoom lens. In addition, a change in the aperture value means that the photographer operates the aperture setting button to change the aperture, and a change in the distance to the subject means that the imaging device 1 or the subject moves. This means that the distance between the imaging device 1 and the subject changes.
[0062]
If it is determined in step S101 that there is no change in the state of the lens 11, the imaging device 1 determines whether or not the number of images to be captured as a group image is stored in the auxiliary storage device 26 (step S102). If it is determined that the number of images has been captured, the stored number of images is determined (step S103).
Then, the imaging device 1 ends the number-of-shots determination process.
[0063]
On the other hand, when it is determined in step S102 that the number of images to be captured as a group image is not stored in the auxiliary storage device 26, the imaging device 1 proceeds to the process of step S104.
If it is determined in step S101 that the state of the lens 11 has changed, the imaging device 1 reads lens information on the lens 11 from the lens information storage device 13 (step S104), and based on the read lens information, The number of images to be photographed as a group image is determined (step S105).
[0064]
Next, the imaging device 1 stores the number of images determined in step S105 in the auxiliary storage device 26 (step S106), and ends the number-of-images determination process.
Next, the post-shooting correction process will be described.
FIG. 14 is a flowchart illustrating a post-imaging correction process performed by the imaging apparatus 1. The post-shooting correction process is a process for changing the in-focus distance or the depth of field of the focused image generated by the image shooting process after shooting, and the user of the imaging apparatus 1 executes the post-shooting correction process. Is activated in response to the instruction of.
[0065]
In FIG. 14, when the post-shooting correction process is started, the imaging apparatus 1 reads the focused image and necessary lens information (the transfer function of the lens 11 and the like) (step S201), and displays the post-shooting correction process on the display device 27. Is displayed (step S202).
FIG. 15 is a diagram illustrating a user interface of the post-shooting correction process.
[0066]
In FIG. 15, the focus position slider for changing the focus distance indicates the focus distance of the displayed image. The focus position slider moves in response to the operation of the focus position button of the input device 21.
In FIG. 15, a depth of field slider for changing the depth of field indicates which aperture value corresponds to the depth of field. Further, the depth of field slider moves in response to the operation of the depth of field adjustment button of the input device 21.
[0067]
Returning to FIG. 14, the imaging apparatus 1 determines whether the focus position slider or the depth of field slider has been operated on the user interface (step S203), and the operation of the depth of field slider is performed. If it is determined that there is no image, the image of each area in the focused image is corrected according to the selected depth of field (step S204).
[0068]
On the other hand, when it is determined in step S203 that the focus position slider has been operated, the imaging device 1 corrects the image of each area in the focused image according to the selected focus position (step S205).
After step S204 and step S205, the imaging apparatus 1 determines whether or not the end of the post-imaging correction process has been instructed (step S206). If it is determined that the end of the post-imaging correction process has not been instructed, The process moves to step S203. On the other hand, if it is determined that the end of the post-shooting correction process has been instructed, the post-shooting correction process ends.
[0069]
As described above, the imaging apparatus 1 according to the present embodiment captures images at the same angle of view and at different focal lengths by a single release operation, and focuses on images based on those images and lens information. An image when the focal length is changed can be artificially generated after photographing.
Therefore, in a captured image, it is possible to flexibly change the focus distance, such as shifting the focus back and forth or changing the depth of field.
[0070]
Further, the data of the in-focus image generated in this manner includes information on the color of the subject (for example, information on each of R, G, and B) and information on the distance to the subject.
Therefore, the image data can be used as image data for a pseudo three-dimensional display or the like using parallax, and such image data can be easily obtained by applying the present invention.
[0071]
In the present embodiment, it has been described that the focused image is changed based on the transfer function of the lens 11 used for shooting, but the focused image is changed based on the transfer function of a lens other than the lens 11. It may be changed.
That is, a transfer function of a lens other than the lens 11 is stored as lens information, and a transfer function of an arbitrary lens can be selected and used in a post-imaging correction process or the like.
[0072]
In this case, by using a transfer function of a lens having a depth of field shallower than that of the lens 11 used for shooting, it is possible to easily obtain an image intended by the user, such as to obtain an image having a shallow depth of field. It becomes possible. That is, in a digital camera that tends to have a large depth of field, an image with a small depth of field can be easily acquired.
[0073]
Further, in the present embodiment, the imaging device 1 has been described as performing operations from capturing a group image to generating a focused image. However, the captured group images are stored in a memory card or the like and stored in a PC (Personal Computer). The data may be transferred to an information processing device such as (1), and a focused image generation process may be performed on the device.
Hereinafter, a process of capturing a group image (group image capturing process) and a process of generating a focused image (focused image generation process) will be described.
[0074]
First, the group image photographing process will be described.
In the group image photographing process, a plurality of photographed images are stored in association with each other after photographing a group image in the image photographing process shown in FIG.
FIG. 16 is a flowchart showing the group image photographing process.
In FIG. 16, the processing of steps S301 to S303 is the same as the processing of steps S1 to S3 of the image photographing processing shown in FIG. 7, respectively, and thus the description thereof will be omitted, and only the subsequent processing will be described.
[0075]
If it is determined in step S303 that the shooting of all the group images has been completed, the imaging device 1 stores the shot group images in association with each other (step S304).
Then, the imaging device 1 ends the group image photographing process.
Here, examples of a method of storing group images in association with each other include a method of identifying a group by an image file name and a method of identifying a group by a folder or a directory.
[0076]
When a group is identified by an image file name, the format of the file name can be configured by the group name and the file number. For example, file names such as “1343_1.jpg”, “1343_2.jpg”, and “1343_3.jpg” can be assigned to three group images captured at 13:43.
[0077]
When a group is identified by a folder or a directory, it is possible to create a folder or a directory with a group name, and store a group image with a file number in the folder or the directory. For example, in the case of three group images shot at 13:43, a folder having a folder name of “1343” is created, and “1.jpg”, “2.jpg”, and “3. jpg ”can be stored.
[0078]
As another method, a method may be considered in which information indicating that the photos in the group belong to the same group is added to an Exif (Exchangeable Image File Format) tag, and an application in the PC determines this.
In addition, it is necessary to record lens information used for photographing one group image. This lens information can be recorded in another file in the same layer as the image, embedded in a specific image file (for example, the first image captured in a group), or recorded in multiple image files. Conceivable.
[0079]
When recording lens information in an image file, it is conceivable to use a maker note tag section in Exif format.
Next, the focused image generation processing will be described.
FIG. 17 is a flowchart showing the focused image generation processing.
The in-focus image generation process is started by the user instructing execution in an information processing apparatus such as a PC in order to generate an in-focus image.
[0080]
When the in-focus image generation process is started, the information processing apparatus reads an image from an external device such as a memory card (step S401), and classifies the read images into groups (step S402). Note that, in step S401, in addition to the case where the data is read via a storage medium such as a memory card, the data may be directly read from the imaging device 1.
[0081]
Next, the information processing apparatus extracts an in-focus area from each of the images (group images) classified into the group (step S403), and determines whether or not all the processing of the group images has been completed. (Step S404).
If it is determined in step S404 that all the processing of the group image has not been completed, the information processing apparatus proceeds to the processing of step S403, while if it is determined that the processing of the group image has all been completed, The respective regions extracted in step S404 are combined to generate an image (focused image) in which all subjects within the angle of view are in focus (step S405). It should be noted that data of the focal distance of the area to which the pixel belongs is added to each pixel of the focused image.
[0082]
As described above, when only the group image is photographed in the imaging device 1 and the focused image is generated in the information processing device, the processing load on the imaging device 1 is reduced.
Note that, in the present embodiment, when the post-shooting correction processing is performed, the case where the focal distance or the depth of field is changed at the time of photographing is described as a pseudo reproduction, but the focusing distance or the It is also possible to change the depth more flexibly.
[0083]
In other words, in addition to reproducing the state in which the focusing distance and the like are changed at the time of shooting with physical accuracy, it is also possible to arbitrarily reproduce the state of the focusing distance or the depth of field that is physically impossible. Good.
For example, as shown in FIG. 8, when the subject exists at the positions of 2 m, 10 m, and 500 m, it is physically impossible to focus on the position of 2 m and the position of 500 m. In the above, the areas corresponding to these distances are set to the in-focus state, and the areas corresponding to the other distances are set to images that are not in the focus state, whereby a physically impossible focus state can be arbitrarily reproduced. .
[0084]
FIG. 18 is a diagram illustrating a user interface when the focus state is arbitrarily changed. 18, a focused image, a focus position slider, and a depth of field slider are the same as those in the interface shown in FIG. In addition, in FIG. 18, two regions of layer-1 and layer-2 are set in the focused image. FIG. 18A shows an adjustment screen of layer-1, and FIG. , Layer-2 adjustment screen.
[0085]
As a criterion for dividing an area into layer-1 and layer-2, a method of dividing an area by a distance to a subject, a method of arbitrarily selecting a region by a user, and the like are possible. It is divided into an area of 0 to 50 m (referred to as layer-1) and an area of 50 m to ∞ (referred to as layer-2). Since each pixel of the in-focus image has data of the in-focus distance, it can be divided into the right half and the left half of the screen, and the method of division is arbitrary.
[0086]
In FIG. 18A, a small window displayed to the left of the focus position slider and the depth of field slider is a screen showing the operation target area, and is a layer-2 area (such as a background mountain) that is not the operation target. For, the image is masked and the user is notified that the image is out of the operation target. On the adjustment screen shown in FIG. 18A, the user can adjust the focusing distance and the depth of field for the layer-1 area, which is the operation target area. Also, in the in-focus image of FIG. 18A, by adjusting the in-focus distance to the person and reducing the depth of field, only the person is in focus, and the building is corrected to be out of focus. ing.
[0087]
In addition, in FIG. 18A, by clicking the tab indicating the layer-2, the screen shifts to the screen of FIG. 18B.
In the in-focus image of FIG. 18B, the focus distance is adjusted to the mountain for the layer-2 area, and the area is corrected to be in focus.
Therefore, it is possible to achieve a physically impossible focusing state in which a person at a short distance is focused on a mountain at a long distance, and an intermediate building is not focused.
[0088]
Such an image can be used for emphasizing a subject at an arbitrary position, or for making an object that is not desired to be moved to the background less visible.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a functional configuration of an imaging apparatus 1 to which the present invention has been applied.
FIG. 2 is a diagram showing defocus characteristics at various focusing distances.
FIG. 3 is a diagram showing three characteristic curves from which image data has been acquired.
FIG. 4 is a conceptual diagram showing an example of classifying the distance of each subject within the angle of view based on the image quality of a captured image.
FIG. 5 is a diagram illustrating a characteristic that an image changes in response to a change in a focusing distance for a subject at a specific distance.
FIG. 6 is a conceptual diagram showing a state in which a region changes according to similar characteristics for each region.
FIG. 7 is a flowchart illustrating an image photographing process executed by the imaging apparatus 1.
FIG. 8 is a diagram showing photographing conditions of a subject.
FIG. 9 is a diagram illustrating an example of a group image.
FIG. 10 is a diagram illustrating an example of an image of a focused area.
FIG. 11 is a diagram illustrating an example of a focused image.
FIG. 12 is a diagram illustrating a relationship between a distance to a subject and a position range of a depth of field.
FIG. 13 is a flowchart showing the number-of-shots determination process executed by the imaging apparatus 1.
FIG. 14 is a flowchart illustrating a post-shooting correction process performed by the imaging apparatus 1.
FIG. 15 is a diagram illustrating a user interface of a post-shooting correction process.
FIG. 16 is a flowchart showing a group image photographing process.
FIG. 17 is a flowchart illustrating a focused image generation process.
FIG. 18 is a diagram illustrating a user interface when an in-focus state is arbitrarily changed.
[Explanation of symbols]
Reference Signs List 1 imaging device, 10 lens unit, 11 lens, 12 aperture device, 13 lens information storage device, 20 camera unit portion, 21 input device, 22 central processing unit, 23 shutter device, 24 imaging device, 25 main storage device, 26 Auxiliary storage device, 27 display device, 30 memory unit, 30 memory unit

Claims (12)

An imaging device that captures light from a subject that has passed through a lens with an imaging element and generates a digital image,
Photographing means for continuously photographing a plurality of images set at different focusing distances at the same angle of view corresponding to the release operation,
Shooting condition obtaining means for obtaining shooting conditions of each of the plurality of images,
Based on the characteristics of the lens or the characteristics of the predetermined lens, the plurality of images at the different focusing distances, and the shooting conditions of the plurality of images, at least the focusing distance or the depth of field of the captured image. An image adjustment means for changing one of them afterwards,
An imaging device comprising: The image adjustment means,
In each of the plurality of images, a focused area extracting unit that extracts a focused area that is in focus,
A focusing area adjusting unit that changes an image of each of the focusing areas based on the characteristics of the lens or the characteristics of a predetermined lens and the shooting conditions;
The imaging device according to claim 1, further comprising: The in-focus area extracting unit determines which of the plurality of images is in a more accurate in-focus state for each area in the angle of view of the plurality of images, and determines that the in-focus area is in a more accurate in-focus state. 3. The imaging device according to claim 2, wherein an image of each of the determined areas is extracted. The imaging apparatus according to claim 1, wherein the imaging unit sets the imaging conditions such that a depth of field range of the plurality of images is continuous. The apparatus further includes a focused image storage unit that generates one image having the same angle of view as the plurality of images based on each region extracted by the focused region extraction unit, and stores the one image. The imaging device according to claim 2. The imaging apparatus according to claim 1, further comprising a group image storage unit that stores a plurality of images taken by the imaging unit in association with each other. 7. The image processing apparatus according to claim 2, wherein when the image is changed by the image adjusting unit, an image of an out-of-focus area is generated based on an image other than a focused area in the plurality of images. An imaging device according to any one of the above. The image adjustment means, when changing the focus distance or depth of field of the captured image, the image assumed when the focus distance or depth of field is changed during shooting, physical change The imaging device according to claim 1, wherein the imaging device is generated in a pseudo manner in accordance with the mode. The image adjusting means can change the focal distance or the depth of field of a captured image to an arbitrary focal distance or an arbitrary depth of field for an arbitrary area within the angle of view. The imaging device according to any one of claims 1 to 7, wherein: An image data generation method for generating image data that can be displayed on a display device,
Shooting a plurality of images set at different focusing distances at the same angle of view, for each area within the angle of view, determine which of the plurality of images is in a more accurate focus state, for each area, An image data generation method characterized by generating image data including information on a color of a subject and information on a distance by associating the image data with a focus distance of an image determined to be in a more accurate focus state. An image data processing device capable of displaying an image based on digital image data,
Image adjustment means for changing at least one of a focal distance and a depth of field of an image related to the image data based on image data including information on a color of a photographed subject and information on a distance. With
An image data processing apparatus capable of displaying an image having an arbitrarily set focusing distance or depth of field. An image data processing program for displaying an image based on digital image data,
An image adjustment function for changing at least one of a focal distance and a depth of field of an image related to the image data based on image data including information on a color of a photographed subject and information on a distance. To the computer,
An image data processing program capable of displaying an image having an arbitrarily set focusing distance or depth of field.

Images