JP2005229280A - Image processing apparatus and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make a stereoscopic image easily viewed when it is formed by synthesizing a plurality of images. <P>SOLUTION: Two images for forming a stereoscopic image are photographed by means of a digital camera, and information representing the distance to an object in photographing is described to the tagged information of image data S1 and S2 representing respective images. Distance ratio calculating section 114 of an image processor 110 calculates the ratio L2/L1 of object distance between two image data S1 and S2 based on object distance information and an image scaling section 116 scales the size of an image represented by the image data S1 based on the ratio L2/L1. An image converting section 118 synthesizes image data S1' thus obtained and the image data S2 to form stereoscopic image data G1. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and method, and an image processing method for performing image processing on a plurality of images when a plurality of images are combined and a stereoscopic image that can be stereoscopically viewed using parallax is generated. The present invention relates to a program for causing a computer to execute.

  Various methods for generating a stereoscopic image for expressing a planar image as a stereoscopic image are known. This stereoscopic image is realized by intentionally generating the parallax of both eyes generated by the interval between the right eye and the left eye. That is, a three-dimensional feeling that expresses a video by giving different images to the left and right eyes of the person viewing the image is expressed. Specifically, for example, in stereoscopic viewing by the naked-eye parallel method, two images corresponding to the left and right eyes are prepared, and the same object in the two images is shifted to the left and right for stereoscopic viewing. Is realized. This is due to the fact that the object far away from the viewer is almost in the same position when viewed with either left or right eye, but the near object is shifted to the left or right, that is, both eyes have parallax. .

  Various devices such as a mobile phone, a notebook computer, and a television equipped with a 3D liquid crystal for stereoscopic viewing are also known. In these apparatuses, a right-eye image and a left-eye image are generated, and these two images are displayed on a 3D liquid crystal to express a three-dimensional image.

  On the other hand, in order to generate a stereoscopic image, it is necessary to photograph the same subject from a plurality of different viewpoints. For this reason, various methods have been proposed for shooting for the purpose of generating a stereoscopic image. For example, 2 for generating a stereoscopic image by using a camera using two types of optical systems or attaching a stereoscopic image capturing adapter to the camera and capturing the same subject from two different viewpoints. A method of acquiring two images (see Patent Documents 1 and 2) has been proposed.

In addition, when a plurality of images are combined side by side, a composite image having a good appearance is obtained by changing the density level of overlapping portions in the plurality of images based on shooting conditions such as exposure information at the time of shooting. A technique has been proposed (see Patent Document 3). Further, a method has been proposed in which luminance information of each image is obtained from reference points on a plurality of images, and the luminance of each image is corrected based on the obtained information to make the luminance of each image substantially the same (Patent Document 4). reference).
Japanese Patent Laid-Open No. 7-134345 JP-A-10-191395 Japanese Patent Laid-Open No. 9-332040 Japanese Patent Laid-Open No. 11-242737

  However, when photographing the same subject from a plurality of viewpoints, it is difficult to accurately match the subject distance between the subject and the camera, and the distance between the subject and the photographing device differs for each photographing. It often ends up. In such a case, since the size of the subject included in the acquired image varies from image to image, even if a stereoscopic image is generated by combining a plurality of acquired images, the subject is stereoscopically generated. It was difficult to make it visible. In addition, the type of light source may be different for each shooting, but in such a case, the color of the image differs for each image, and as a result, it is difficult to see the stereoscopic image. It was.

  The present invention has been made in view of the above circumstances, and an object thereof is to make a stereoscopic image easy to see.

The image processing apparatus according to the present invention obtains a plurality of images obtained by photographing the same subject from a plurality of viewpoints and provided with photographing information at the time of photographing in order to generate a stereoscopic image. When,
Based on the photographing information given to each of the plurality of images, at least one of the images so that the size of the subject included in the plurality of images and / or the image quality of the plurality of images are substantially the same. Image correcting means for correcting
An image converting means for converting the corrected image and other images into the stereoscopic image is provided.

  "Image acquisition means" include a reading means such as a media drive for reading an image from a recording medium on which an image is recorded, a means for receiving an image transmitted via a network, a scanner for reading an image recorded on a print or film, etc. Any means capable of acquiring an image, such as a reading means, can be used.

  “Convert to a stereoscopic image” means not only to create a single image by combining the left and right subjects with the same orientation of the subjects included in each of the plurality of images, but also when displaying them on a 3D liquid crystal. A plurality of images are combined into a single image so as to be stereoscopically viewable, and a single image file including a plurality of images is generated. The header includes a plurality of images for generating a stereoscopic image. Including the fact that the image is included and the information indicating the arrangement position of the plurality of images, it is possible to reproduce a stereoscopic image in which the plurality of images are arranged side by side when the image file is opened. Including.

  Examples of the “image quality” include image color, brightness, contrast, gradation, sharpness, resolution, noise, and the like.

In the image processing apparatus according to the present invention, when the shooting information includes subject distance information indicating a subject distance at the time of shooting,
The image correcting unit calculates a ratio of the subject distance corresponding to each of the plurality of images based on the subject distance information of the plurality of images, and the size of the subject included in each of the plurality of images. And means for enlarging / reducing at least one of the images in accordance with the ratio, so that
The image conversion means may be means for converting the enlarged and reduced images and other images into the stereoscopic image.

  “Enlarge or reduce at least one image according to a ratio” means that the size of a subject included in another image other than one image among the plurality of images is the same as the size of the subject included in one image. In addition to the above, it includes enlarging and reducing all of the plurality of images so that the sizes of the subjects included in all of the plurality of images are substantially the same.

In the image processing apparatus according to the present invention, when the image quality is tint, and further, when the shooting information includes light source information indicating the type of light source at the time of shooting,
The image correction means is a means for correcting the color of the at least one image based on the light source information,
The image conversion unit may be a unit that converts the color-corrected image and other images into the stereoscopic image.

In this case, the image correcting unit refers to the white balance conversion coefficient table that defines the white balance conversion coefficient that makes the white balance of images of different types of light sources substantially the same, for each of a plurality of types of light sources. Based on light source information corresponding to each of a plurality of images, as means for correcting the color by correcting the white balance of at least one of the images,
The image conversion means may be means for converting the image with the white balance corrected and other images into the stereoscopic image.

The image processing method according to the present invention acquires a plurality of images provided with shooting information at the time of shooting obtained by shooting the same subject from a plurality of viewpoints in order to generate a stereoscopic image.
Based on the photographing information given to each of the plurality of images, at least one of the images so that the size of the subject included in the plurality of images and / or the image quality of the plurality of images are substantially the same. To correct
The corrected image and other images are converted into the stereoscopic image.

  The image processing method according to the present invention may be provided as a program for causing a computer to execute the image processing method.

  According to the present invention, a plurality of images for generating a stereoscopic image are acquired, and the sizes of subjects included in the plurality of images and / or the image quality of the plurality of images based on shooting information given to the plurality of images. Are corrected so that they are substantially the same, and the corrected image and other images are converted into a stereoscopic image. Thereby, since the size of the subject included in the plurality of images and / or the image quality of the plurality of images can be made substantially the same, the stereoscopic image can be easily viewed.

  In particular, when the shooting information includes subject distance information representing the subject distance at the time of shooting, the ratio of the subject distance corresponding to each of the plurality of images is calculated, and the subjects included in each of the plurality of images are approximately the same size. By enlarging and reducing at least one image according to the calculated ratio so that the size of the subject included in the plurality of images can be made substantially the same.

  Further, when the image quality is the color of the image and the shooting information includes light source information indicating the type of light source at the time of shooting, a plurality of images are corrected by correcting the color of at least one image based on the light source information. Can be almost the same color.

  In this case, a white balance conversion coefficient that makes the white balance of images of different types of light sources almost the same is referred to a conversion coefficient table that defines each of the plurality of types of light sources, and the light sources corresponding to each of the plurality of images By correcting the white balance of at least one image based on the information, the colors of the plurality of images can be made substantially the same relatively easily.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. First, a digital camera that acquires image data for generating a stereoscopic image subjected to image processing by an image processing apparatus according to an embodiment of the present invention will be described.

  FIG. 1 is a schematic block diagram showing the configuration of a digital camera. As shown in FIG. 1, the photographing lens 22 constituting the digital camera 1 is composed of one or a plurality of lenses. The photographic lens 22 may be a lens having a single focal length (fixed focus) or a variable focal length such as a zoom lens or a telephoto / wide-angle bifocal switching lens. The light that has passed through the photographic lens 22 is incident on an image sensor 26 made up of a CCD, a CMOS, or the like after the amount of light is adjusted by the diaphragm 24. Photosensors are arranged in a plane on the light receiving surface of the image pickup device 26, and the subject image formed on the light receiving surface of the image pickup device 26 is converted into signal charges corresponding to the amount of incident light by each photosensor. Is done. The signal charges accumulated in this way are sequentially read out as voltage signals corresponding to the signal charges based on the pulses supplied from the image sensor driving unit 28.

  Image data output from the image sensor 26 is sent to the analog signal processing unit 30. The analog signal processing unit 30 includes a sampling hold circuit, a color separation circuit, a gain adjustment circuit, and the like. The analog signal processing unit 30 performs color separation into correlated double sampling (CDS) processing and R, G, and B color signals. Then, the signal level of each color signal is adjusted (pre-white balance process).

  The image data output from the analog signal processing unit 30 is converted into digital data by the A / D converter 40 and then stored in the main memory 48 controlled by the memory control unit 46.

  The image data stored in the main memory 48 is sent to the digital signal processing unit 50 via the bus 70. The digital signal processing unit 50 is an image processing means composed of a digital signal processor (DSP) including a luminance / color difference signal generation circuit, a gamma correction circuit, a sharpness correction circuit, a contrast correction circuit, a white balance correction circuit, and the like. The image data is processed according to the command from

  The CPU 42 receives input of various operations by the photographer of the digital camera from the operation unit 44 including a release button, a cross key, and the like, and outputs a command corresponding to the input operation.

  The image data input to the digital signal processing unit 50 is converted into a luminance signal (Y signal) and a color difference signal (Cr, Cb signal) and subjected to predetermined image processing such as gamma correction and white balance correction. Thereafter, it is stored in the main memory 48.

  The image data stored in the main memory 48 is read according to a command from the CPU 42 and sent to a display control unit 60 having a display memory. The data sent to the display control unit 60 is converted into a predetermined display signal (for example, an NTSC color composite video signal) and then D / A converted to a display unit 62 comprising a liquid crystal monitor or the like. Is output. Thereby, the image content of the image data is reproduced on the display unit 62.

  The image data in the main memory 48 is periodically rewritten by the image data output from the image sensor 26 and the image data is supplied to the display unit 62, whereby the image captured by the image sensor 26 is displayed on the display unit 62 in real time. Played. The photographer can confirm the shooting angle of view from the image reproduced on the display unit 62.

  The CPU 42 is a control unit that controls operations related to photographing of the digital camera. The CPU 42 includes storage means such as a ROM 42A and a RAM 42B. The ROM 42A stores programs processed by the CPU 42 and various data necessary for control. The RAM 42B serves as a work area when the CPU 42 performs various arithmetic processes. Used. The CPU 42 controls the operation of the corresponding circuit based on the input signal from the operation unit 44, and controls display on the display unit 62, autofocus (AF) control, automatic exposure (AE) control, and the like.

  During AF control and AE control, the CPU 42 causes the AE / AF calculation unit 54 to calculate the focus position and AE calculation based on the subject distance output from the distance measuring device 74 and the image data output from the image sensor 26. The lens driving unit 32 including the focus motor is controlled based on the calculation result to move the lens to the in-focus position, and the diaphragm driving unit 34 including the iris motor is controlled to perform an appropriate operation. The aperture is set and the charge accumulation time of the image sensor 26 is controlled.

  That is, at the time of photographing, a distance measurement start instruction signal is issued from the CPU 42 by a half-press operation of a release button included in the operation unit 44, whereby the distance measuring device driving unit 72 drives the distance measuring device 74. Thereby, infrared rays for ranging are emitted from the light emitting unit 76 toward the subject, and the light receiving unit 78 receives the light reflected by the subject. The distance measuring device 74 calculates the subject distance from the time and angle at which the infrared rays are returned. When the information representing the subject distance is input to the AE / AF calculation unit 54 via the CPU 42, the AE / AF calculation unit 54 determines the in-focus position, and the CPU 42 shoots at the in-focus position in response to this. The lens driving unit 32 is controlled to move the lens 22.

  On the other hand, digital image data output from the A / D converter 40 by half-pressing the release button is transmitted to the AE / AF calculation unit 54. The AE / AF calculation unit 54 integrates the R, G, and B signals constituting the image data, and transmits the integrated value to the CPU 42. The CPU 42 detects the average brightness of the subject (subject brightness) based on the input integrated value, and calculates an exposure value (shooting Ev value) suitable for shooting. Then, according to the obtained exposure value and a predetermined program diagram, the aperture value and shutter speed at the time of shooting are determined, and the aperture drive unit 34 and the image sensor drive unit 28 are controlled. Thereby, the optimal exposure amount can be obtained when the release button is fully pressed.

  Further, when a release button included in the operation unit 44 is fully pressed, a recording start instruction signal is issued, thereby starting to capture image data for recording. When the mode for compressing and recording the image data is selected, the CPU 42 sends a command to the compression / decompression processing unit 52, whereby the compression / decompression processing unit 52 converts the image data on the main memory 48 to JPEG or another predetermined predetermined value. Compress according to format. The compressed image data is recorded on a recording medium 58 such as a memory card via the memory control unit 56. If a mode for recording non-compressed image data (non-compression mode) is selected, the compression / decompression processing unit 52 does not perform compression processing, and the image data remains uncompressed on the recording medium 58. To be recorded.

  FIG. 2 is a diagram showing a file structure of image data generated in the present embodiment. As shown in FIG. 2, the image file 100 includes a header portion 102 indicating a file type, tag information 104 describing shooting information, a thumbnail image 106 of shot image data, and shot image data. The main image 108 having a size of, for example, 1600 × 1200 pixels generated based on the image is recorded. Note that the thumbnail image 106 is an image having a size of 160 × 120 pixels used for the heading of the main image 108 and the like.

  The tag information 104 describes the number of pixels of the main image 108, the type of compression, the shooting date and time, the exposure time at the time of shooting, subject distance information, light source information, and the like. The light source information is determined on the basis of RGB color signals constituting the image data.

  Actually, information described in the tag information 104 is specified by a predetermined tag number. For example, the rotation information is assigned with a tag number of 37382, and the numerical value of the subject distance is described in units of meters. Moreover, the 37384th tag number is assigned to the light source information, and the type of the light source is preset by the number. For example, “1” represents daylight light source information, and “2” represents fluorescent light source information. Therefore, such a number is described in the light source information of the tag information 104.

  Then, by using such a digital camera 1, the photographer captures the same subject from a plurality of viewpoints, whereby image data representing a plurality of images for generating a stereoscopic image is acquired and stored in the recording medium 58. To be recorded.

  FIG. 3 is a schematic block diagram showing the configuration of the image processing apparatus according to the first embodiment of the present invention. The image processing apparatus 110 according to the first embodiment of the present invention sets a recording medium 58 on which image data is recorded by shooting with the digital camera 1 shown in FIG. 1 and generates a stereoscopic image from the recording medium 58. Image data representing the stereoscopic image is generated from the read image data. Note that image data for generating a stereoscopic image is acquired by a photographer photographing the same subject from a plurality of viewpoints.

  When the user gives an instruction to display a list of images from the operation unit 128, the memory control unit 112 reads out all image data from the recording medium 58, and thumbnail images attached to all the image data read out from the recording medium 58. 106 is decompressed (compressed) by the compression / decompression processing unit 126, and a list thereof is generated and a list of images is displayed on the display unit 130. Furthermore, when the user selects two images for generating a stereoscopic image from a list of images using the operation unit 128, the memory control unit 112 reads image data representing the image selected by the user from the recording medium 58, and The image data read from the recording medium 58 is decompressed by the compression / decompression processor 126 and input to the distance ratio calculator 114.

  The distance ratio calculation unit 114 refers to the subject distance information described in the tag information 104 of the two image data (hereinafter referred to as S1 and S2), and the subject distance L1 when the two image data S1 and S2 are photographed. The ratio L1 / L2 of L2 is calculated, and the ratio L1 / L2 is input to the image enlargement / reduction processing unit 116.

  The image enlargement / reduction processing unit 116 enlarges / reduces an image represented by the image data S1 (hereinafter also using S1 as a reference symbol for the image and the same for the image represented by the image data S2) by L1 / L2. Reduced image data S1 'is generated. Here, enlargement / reduction will be described. FIG. 4 is a diagram for explaining a difference in size of a subject included in an image according to a subject distance. When shooting is performed at a subject distance L1 that is relatively close to the person who is the subject, the person included in the image S1 obtained by shooting is larger. On the other hand, when shooting is performed with a subject distance L2 that is greater than the subject distance L1, the person included in the image S2 acquired by shooting is smaller than in the case of the subject distance L1. In this case, the ratio of the size of the person included in the images S1 and S2 is L2: L1. Accordingly, the image enlargement / reduction processing unit 116 reduces the image S1 to L1 / L2 times so that the sizes of the persons included in the images S1, S2 are the same.

  Note that, when L1 / L2 <0, the image enlargement / reduction processing unit 116 reduces the size of the image S1, and therefore uses, for example, the pixel values at the periphery of the image S1 ′ around the reduced image S1 ′. Thus, the size of the image S1 ′ is made the same as the size of the image S2. On the other hand, when L1 / L2> 0, the size of the image S1 is enlarged. Therefore, the periphery of the enlarged image S1 ′ is deleted so that the size of the image S1 ′ is the same as the size of the image S2.

  As shown in FIG. 5, the image conversion processing unit 118 converts the image data S1 ′ and the image data S2 so that the image S1 ′ and the image S2 are arranged side by side to generate the stereoscopic image data G1. The generated stereoscopic image data G1 is compressed by the compression / decompression processing unit 126 and recorded on the recording medium 58.

  Note that a program for driving the image processing apparatus 110 is stored in the main memory 124, and the memory control unit 122 reads the program from the main memory 124 according to an instruction from the CPU 120, whereby the CPU 120 generates a stereoscopic image. Perform various processes including. The main memory 124 also serves as a work area for processing performed by the image processing apparatus 110.

  Next, processing performed in the first embodiment will be described. FIG. 6 is a flowchart showing processing performed in the image processing apparatus 110 according to the first embodiment. Here, a process after the user selects two image data S1 and S2 for generating a stereoscopic image will be described. The process is started when the user issues an instruction to select two image data S1 and S2 for generating a stereoscopic image using the operation unit 128, and the CPU 120 drives the memory control unit 112 to be recorded on the recording medium 58. The two existing image data S1, S2 are read from the recording medium 58 (step S1). Then, the compression / decompression processing unit 126 decompresses the image data S1 and S2 (step S2), inputs the decompressed image data S1 and S2 to the distance ratio calculation unit 114, and the distance ratio calculation unit 114 performs the subject distance ratio L1. / L2 is calculated (step S3). Then, the image enlargement / reduction processing unit 116 enlarges / reduces the image represented by the image data S1 by the ratio L1 / L2 to generate enlarged / reduced image data S1 ′ (step S4). Next, the image conversion processing unit 118 converts the image data S1 ′ and the image data S2 to generate stereoscopic image data G1 (step S5), and the compression / decompression processing unit 126 compresses the stereoscopic image data G1 (step S6). ). Then, the CPU 120 records the compressed stereoscopic image data G1 on the recording medium 58 (step S7), and ends the process.

  As described above, in the first embodiment, the two image data S1 and S2 are acquired from the subject distance information described in the tag information 104 of the two image data S1 and S2 for generating a stereoscopic image. Since the ratio L1 / L2 of the subject distances L1 and L2 is calculated and the image S1 is enlarged / reduced by the ratio L1 / L2, the size of the subject included in the image S1 ′ and the size of the subject included in the image S2 Can be made substantially the same. Therefore, when a stereoscopic view is performed using the stereoscopic image represented by the stereoscopic image data G1 generated by combining the image data S1 ′ and the image data S2, the stereoscopic image is easy to see. Can do.

  In the first embodiment, the ratio L1 / L2 is calculated and the image S1 is scaled according to the ratio L1 / L2. However, the ratio L2 / L1 is calculated and the image S2 is converted into the ratio L2 / L. You may enlarge / reduce according to L1. Further, the enlarged image S1 ′ and the image S2 may be further enlarged / reduced at a predetermined enlargement ratio.

  Next, a second embodiment of the present invention will be described. FIG. 7 is a schematic block diagram showing the configuration of the image processing apparatus according to the second embodiment of the present invention. In the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. An image processing apparatus 140 according to the second embodiment replaces the distance ratio calculation unit 114 and the image enlargement / reduction processing unit 116 of the image processing apparatus 110 according to the first embodiment with a white balance conversion processing unit (hereinafter referred to as a WB conversion processing unit). The second embodiment is different from the first embodiment in that a conversion coefficient storage unit 134 and a conversion coefficient storage unit 134 are provided.

  The WB conversion processing unit 132, based on the light source information K1 and K2 described in the tag information 104 of the two image data S1 and S2 that generate the stereoscopic image selected by the user, the white balance of the image S2 and the image S1. With reference to the white balance conversion coefficient table stored in the conversion coefficient storage unit 134 so that the white balance is substantially the same, the white balance of the image S2 is corrected by correcting the white balance of the image S2. Generate.

  In the present embodiment, since the two image data S1 and S2 for generating a stereoscopic image are acquired by shooting the same subject from different directions, the light source at the time of shooting is originally used. Are the same. However, when it is determined that the light sources when the two images S1 and S2 are acquired are different due to the influence of the background at the time of shooting, if the white balance is corrected based on the determination result, the two images S1 and S2 may have different colors. In such a case, the second embodiment corrects the white balance of the image S2.

  FIG. 8 is a diagram illustrating an example of a white balance conversion coefficient table. As shown in FIG. 8, the white balance conversion coefficient table WBT has almost the same white balance as that of the image data S1 based on the light source information K1 of the image data S1 and the light source information K2 of the image data S2. The matrix M is described in which the image data S2 is corrected, more specifically, the RGB color signals constituting the image data S2. Specifically, the light source information K1 in the horizontal direction and the light source information K2 in the vertical direction are determined, the matrix M in the column where the light source information K1 and K2 in the vertical and horizontal directions intersect, the white balance of the image data S2, and the white balance of the image data S1. Can be selected as the matrix M for correcting the image data S2. For example, when the light source information K1 of the image data S1 is daylight and the light source information K2 of the image data S2 is an incandescent lamp, the image data S2 is set so that the white balance of the image data S2 is substantially the same as the white balance of the image data S1. The matrix M2 is selected as the matrix M for correcting.

Here, the matrix M has 3 × 3 elements. Therefore, the WB conversion processing unit 132 selects the matrix M with reference to the white balance conversion coefficient table WBT based on the light source information K1 of the image data S1 and the light source information K2 of the image data S2, and configures the image data S2. By performing the calculation shown in the following equation (1) for each of the RGB color signals, the white balance of the image data S2 is corrected to generate image data S2 ″ with the white balance corrected.
R ″ R
G ″ = MG (1)
B ″ B
However, R ″, G ″, B ″ are color signals constituting the image data S2 ″. Here, the image data S1, S2 are processed by the digital signal processing unit 50 of the digital camera 1 in accordance with the white balance correction process according to the type of the light source. Is given. Accordingly, the WB conversion processing unit 132 performs processing for further correcting the white balance of the image data S2 whose white balance has been corrected in the digital camera.

  Next, processing performed in the second embodiment will be described. FIG. 9 is a flowchart showing processing performed in the image processing apparatus 140 according to the second embodiment. Here, a process after the user selects two image data S1 and S2 for generating a stereoscopic image will be described. The process is started when the user issues an instruction to select two image data S1 and S2 for generating a stereoscopic image using the operation unit 128, and the CPU 120 drives the memory control unit 112 to be recorded on the recording medium 58. The two existing image data S1 and S2 are read from the recording medium 58 (step S11). The compression / decompression processor 126 decompresses the image data S1 and S2 (step S12), inputs the decompressed image data S1 and S2 to the WB conversion processor 132, and the WB conversion processor 132 stores the image data S1. By referring to the white balance conversion coefficient table WBT stored in the conversion coefficient storage unit 134 based on the light source information K1 described in the tag information 104 and the light source information K2 described in the tag information 104 of the image data S2, the image data A matrix M for correcting the image data S2 is selected so that the white balance of S2 is substantially the same as the white balance of the image data S1 (step S13).

  Then, the WB conversion processing unit 132 corrects the white balance of the image data S2 using the matrix M to generate image data S2 ″ with the white balance corrected (step S14). The image data S1 and the image data S2 ″ are converted to generate stereoscopic image data (referred to as G2) (step S15), and the compression / decompression processor 126 compresses the stereoscopic image data G2 (step S16). Then, the CPU 120 records the compressed stereoscopic image data G2 on the recording medium 58 (step S17), and ends the process.

  As described above, in the second embodiment, the conversion coefficient storage unit 134 stores the light source information K1 and K2 described in the tag information 104 of the two image data S1 and S2 for generating a stereoscopic image. With reference to the stored white balance conversion coefficient table WBT, a matrix M for correcting the image data S2 is selected so that the white balance of the image data S2 is substantially the same as the white balance of the image data S1, and the image data is determined by this matrix. Since the white balance of S2 is corrected, the colors of the image S1 and the image S2 ″ can be made substantially the same. Therefore, the stereoscopic image generated by converting the image data S1 ′ and the image data S2 When stereoscopic viewing is performed using the stereoscopic image represented by the data G2, the stereoscopic image is viewed. It can be a casting.

  In the second embodiment, the matrix M for correcting the image data S2 is selected so that the white balance of the image data S2 is substantially the same as the white balance of the image data S1, and the matrix M is used to select the image data S2. Although the white balance is corrected, a matrix M for correcting the image data S1 so that the white balance of the image data S1 is substantially the same as the white balance of the image data S2 with reference to the white balance conversion coefficient table WBT. The matrix M may be selected and the white balance of the image data S1 may be corrected by the matrix M.

  Further, like the image processing apparatus 160 according to the third embodiment of the present invention shown in FIG. 10, the image processing apparatus 110 according to the first embodiment and the image processing apparatus 140 according to the second embodiment are combined to generate an image. It is also possible to perform both the enlargement / reduction processing and the white balance correction processing. In this case, either the enlargement / reduction processing of the image data S1, S2 or the white balance correction processing of the image data S1, S2 may be performed first.

  In the first to third embodiments, the image processing devices 110, 140, and 160 are used alone. However, the digital camera 1 is provided with the image processing devices 110, 140, and 160, and the digital camera 1 has a stereoscopic image. Visual image data G1, G2 may be generated.

  In the first to third embodiments, two image data S1 and S2 are used as image data for generating the stereoscopic image data G1 and G2, but three or more image data are used. It is also possible to generate stereoscopic image data G1, G2.

  In the first to third embodiments, the stereoscopic image data G1 and G2 are generated using the image data acquired by the digital camera. However, the same subject is used for the purpose of generating a stereoscopic image. Stereoscopic image data G1 and G2 can be generated using image data acquired by reading an image recorded on a print or a negative film acquired by photographing with a scanner or the like. In this case, subject distance information and / or light source information may be added to image data obtained by reading an image recorded on a print or negative film, and then recorded on the recording medium 58.

  In the first to third embodiments, the image data S1 and S2 recorded on the recording medium 58 are read to generate a stereoscopic image. For example, the image processing devices 110, 140, and 160 are A stereoscopic image may be generated by connecting to a server that stores image data for generating a stereoscopic image via a network and reading the image data S1, S2 to the image processing devices 110, 140, 160 via the network. .

  In the first to third embodiments, stereoscopic image data G1 and G2 in which the image data S1 and S2 are arranged on the left and right are generated and recorded on the recording medium 58. However, the image data One image file including S1 and S2 is generated, and the header portion includes information indicating that the image file includes an image for generating a stereoscopic image and information indicating the arrangement positions of the images S1 and S2. It may be possible to reproduce a stereoscopic image in which the images S1 and S2 are arranged side by side when opened. Alternatively, the stereoscopic image data G1 and G2 may be generated by combining the image data S1 and S2 so that the image can be stereoscopically viewed when displayed on the 3D liquid crystal.

  As described above, the image processing apparatus according to the embodiment of the present invention has been described. However, the computer includes the memory control unit 112, the distance ratio calculation unit 114, the image enlargement / reduction processing unit 116, the image conversion processing unit 118, and the WB conversion processing unit 132. A program that functions as means corresponding to the conversion coefficient storage unit 134 and performs processing as shown in FIGS. 6 and 9 is also one embodiment of the present invention. A computer-readable recording medium that records such a program is also one embodiment of the present invention. In these cases, the white balance conversion coefficient table WBT may be included in the program or the same recording medium, or may be provided from an external device or a separate medium. Good.

Schematic block diagram showing the configuration of the digital camera The figure which shows the file structure of the image data produced | generated in this embodiment 1 is a schematic block diagram showing the configuration of an image processing apparatus according to a first embodiment of the present invention. The figure for demonstrating the difference in the size of the subject contained in the image according to subject distance Diagram for explaining image composition 5 is a flowchart showing processing performed in the image processing apparatus according to the first embodiment. Schematic block diagram showing a configuration of an image processing apparatus according to a second embodiment of the present invention. The figure which shows the example of a white balance conversion coefficient table The flowchart which shows the process performed in the image processing apparatus by 2nd Embodiment. The schematic block diagram which shows the structure of the image processing apparatus by the 3rd Embodiment of this invention.

Explanation of symbols

58 Recording Medium 110, 140, 160 Image Processing Device 112 Memory Control Unit 114 Distance Ratio Calculation Unit 116 Image Enlargement / Reduction Processing Unit 118 Image Conversion Processing Unit 120 CPU
122 memory control unit 124 main memory 126 compression / decompression processing unit 128 operation unit 130 display unit 132 WB conversion processing unit 134 conversion coefficient storage unit

Claims (7)

Image acquisition means for acquiring a plurality of images provided with shooting information at the time of shooting, obtained by shooting the same subject from a plurality of viewpoints in order to generate a stereoscopic image;
Based on the photographing information given to each of the plurality of images, at least one of the images so that the size of the subject included in the plurality of images and / or the image quality of the plurality of images are substantially the same. Image correcting means for correcting
An image processing apparatus comprising: an image conversion unit that converts the corrected image and other images into the stereoscopic image. In the case where the shooting information includes subject distance information indicating a subject distance at the time of shooting,
The image correcting unit calculates a ratio of the subject distance corresponding to each of the plurality of images based on the subject distance information of the plurality of images, and the size of the subject included in each of the plurality of images. Are means for enlarging or reducing at least one of the images in accordance with the ratio so that they are substantially the same,
The image processing apparatus according to claim 1, wherein the image conversion unit is a unit that converts the scaled image and other images into the stereoscopic image.   The image processing apparatus according to claim 1, wherein the image quality is a color of the image. In the case where the shooting information includes light source information indicating the type of light source at the time of shooting,
The image correction means is means for correcting a color of the at least one image based on the light source information,
The image processing apparatus according to claim 3, wherein the image conversion unit is a unit that converts the image whose color is corrected and other images into the stereoscopic image. The image correcting means refers to the white balance conversion coefficient table that defines white balance conversion coefficients for each of a plurality of types of light sources so that the white balance of images of different types of light sources is substantially the same. Is a means for correcting the color by correcting the white balance of at least one of the images based on the light source information corresponding to each of
5. The image processing apparatus according to claim 4, wherein the image conversion means is means for converting the image with the white balance corrected and other images into the stereoscopic image. In order to generate a stereoscopic image, a plurality of images acquired by shooting the same subject from a plurality of viewpoints and provided with shooting information at the time of shooting are acquired,
Based on the photographing information given to each of the plurality of images, at least one of the images so that the size of the subject included in the plurality of images and / or the image quality of the plurality of images are substantially the same. To correct
An image processing method comprising converting the corrected image and other images into the stereoscopic image. In order to generate a stereoscopic image, a procedure for acquiring a plurality of images obtained by capturing the same subject from a plurality of viewpoints and provided with shooting information at the time of shooting;
Based on the photographing information given to each of the plurality of images, at least one of the images so that the size of the subject included in the plurality of images and / or the image quality of the plurality of images are substantially the same. The procedure to correct
A program for causing a computer to execute an image processing method comprising: converting the corrected image and other images into the stereoscopic image.

Images