JP2011081632A - Image generating apparatus, image generating method and program - Google Patents

Abstract

To provide a stereoscopic image in which the displacement of a main object is appropriately adjusted according to the position and size of the main object.
An image generating apparatus for generating a right eye image to be viewed with a viewer's right eye and a left eye image to be viewed with a viewer's left eye, the right eye image And a main object specifying unit for specifying a main object included in the image for the left eye and a position of the main object on the display screen or a size of the main object in the image for the left eye based on the position or size of the main object. A shift amount determination unit that determines a shift amount, which is a difference in position on the display screen, and an image generation that generates a right eye image and a left eye image in which the positions of main objects on the display screen differ by the shift amount. A part.
[Selection] Figure 1

Description

  The present invention relates to an image generation device, an image generation method, and a program.

  As a technique for visually recognizing a stereoscopic (3D) image with the naked eye without using special glasses, a technique using a parallax barrier or a lenticular lens in front of the display surface is known. For example, a technique is known in which an image for the right eye and an image for the left eye displayed on the display surface are directed toward the right eye and the left eye by a parallax barrier or a lenticular lens, respectively (see, for example, Patent Document 1). ).

JP-A-7-287195

  By shifting the object between the right-eye image and the left-eye image on the display surface and presenting them to the viewer, the object appears farther or closer than the display surface due to parallax fusion. If there is no position shift of the object, it looks to the viewer that the object exists at the position of the display surface. However, the viewer may not be able to perform the parallax fusion well over the entire display surface. For example, when the viewer is located in front of the center of the display surface, the peripheral region of the display surface may not be successfully fused with parallax. If the main object is present at such a position, the viewer may not be able to parallax the main object well, and may feel a strong sense of discomfort. In addition, even if a significantly large main object appears far away, the viewer feels a strong sense of discomfort. For this reason, it is preferable to appropriately adjust the displacement of the main object in accordance with the position and size of the main object.

  In order to solve the above-described problem, in one aspect of the present invention, a right-eye image that should be viewed with the viewer's right eye and a left-eye image that should be viewed with the viewer's left eye are generated. An image generating device for a right eye based on a main object specifying unit for specifying a main object included in a right eye image and a left eye image and a position or size of the main object on a display screen A shift amount determination unit that determines a shift amount, which is a difference in the position of the main object on the display screen, between the image and the left eye image, and a right eye that has a different position on the display screen of the main object by the shift amount An image generation unit configured to generate an image and a left-eye image;

  The image generation unit may generate a right-eye image and a left-eye image obtained by shifting the two-dimensional image left and right on the display screen.

  The shift amount determination unit may determine a shift amount smaller when the position of the main object on the display screen is closer to the end of the display screen.

  The displacement amount determination unit may determine the displacement amount smaller when the size of the main object on the display screen is larger.

  The deviation amount determination unit may determine a deviation amount smaller when the size of the main object is larger than the size of the display screen.

  You may further provide the display part which displays the image for right eyes and the image for left eyes which the image generation part produced | generated.

  The above summary of the invention does not enumerate all necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a figure which shows the structural example of the display apparatus 10 which concerns on one Embodiment. FIG. 6 is a diagram for explaining an operation example of an image processing unit 120. It is a figure explaining the image guide part. 6 is a diagram for explaining an operation example of an image generation unit 110. FIG. It is a figure explaining the example of a process based on the position of a main object. It is a figure explaining the example of a process based on the magnitude | size of a main object. FIG. 10 is a flowchart illustrating an operation example of the display device 10. It is a figure which shows the example of a process with respect to the image edge part of the image for left eyes, and the image for right eyes. It is a figure which shows the other example of a process with respect to an image edge part. It is a figure which shows the further another process example with respect to an image edge part. It is a figure which shows the further another process example with respect to an image edge part. It is a figure which shows the further another process example with respect to an image edge part. It is a figure which shows an example of the hardware constitutions of the computer 800 which concerns on other embodiment.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a diagram illustrating a configuration example of a display device 10 according to an embodiment. The display device 10 presents a stereoscopic image to the viewer. Specifically, the display device 10 generates a right-eye image that should be viewed with the viewer's right eye and a left-eye image that should be viewed with the viewer's left eye, and generates the generated right The stereoscopic image is presented to the viewer by presenting the eye image and the left eye image to the viewer's right eye and the viewer's left eye, respectively.

  Here, the display device 10 may be a display device such as a digital photo frame, a television receiver, and a monitor for a personal computer. The display device 10 may be a display device of an imaging device such as a digital still camera or a digital video camera, or may be a display device of a mobile device such as a mobile phone.

  The display device 10 includes an image acquisition unit 100, an image generation unit 110, an image processing unit 120, a display unit 140, an image guide unit 150, a deviation amount determination unit 160, and a main object specifying unit 170. Among the components included in the display device 10, the functional blocks excluding the image guide unit 150 can function as an image generation device that generates a right-eye image and a left-eye image. For example, a functional block including the image acquisition unit 100, the image generation unit 110, the image processing unit 120, the display unit 140, the deviation amount determination unit 160, and the main object specification unit 170 can function as an image generation device.

  The image acquisition unit 100 acquires an input image that is a source of a display image to be displayed on the display unit 140. For example, the image acquisition unit 100 may acquire one or more images stored in the storage medium from the storage medium as an input image. The image acquisition unit 100 may acquire an image supplied from the outside through an external interface as an input image. The image acquired by the image acquisition unit 100 as an input image may be a still image or a moving image in which still images are continuous. The image acquisition unit 100 acquires a plurality of input images and supplies them to the image generation unit 110.

  As an input image, one two-dimensional image that is an image from one viewpoint can be exemplified. For example, the input image may be a single two-dimensional image obtained by imaging from a single viewpoint with a so-called two-dimensional camera. In addition, the input images may be two two-dimensional images that are images from two viewpoints corresponding to the right eye and the left eye, respectively. For example, the input image may be two two-dimensional images obtained by imaging from two viewpoints with a so-called compound eye camera. The input image is a concept including a non-captured image such as a graphic image (for example, a CG image) in addition to the captured image.

  The image generation unit 110 uses the input image acquired from the image acquisition unit 100 to generate a right eye image and a left eye image included in the display image to be displayed on the display unit 140. Specifically, the image generation unit 110 generates a right-eye image and a left-eye image in which the corresponding positions of corresponding points in at least one object in the image are misaligned on the display screen. In this specification, the difference in the position on the display screen of corresponding points of corresponding objects in the right-eye image and the left-eye image is referred to as a deviation amount. The left-right direction is a direction along the right eye and left eye of the viewer. The image generation unit 110 supplies the generated right eye image and left eye image to the image processing unit 120.

  The image processing unit 120 performs predetermined image processing on the right-eye image and the left-eye image generated by the image generation unit 110 to generate a display image, and supplies the display image to the display unit 140. For example, the image processing unit 120 may perform blurring processing on at least one of the right eye image and the left eye image. For example, the image processing unit 120 may perform blurring processing on at least one of the right-eye image and the left-eye image by averaging a plurality of pixel values for each partial region of the image.

  Rather than presenting a clear image for the right eye and a left eye to the viewer, the viewer may be able to recognize a three-dimensional image with a small burden when the blurring process is performed. is there. Specifically, the spatial information given to the viewer by the display image, which is a pseudo stereoscopic image displayed on the two-dimensional display surface, is completely the same as the spatial information given by the object in the actual three-dimensional space. The viewer may feel some contradiction in the visual information obtained from the display image.

  The viewer's brain tries to recognize one stereoscopic image by fusing the right-eye image and the left-eye image, but contradicts that both the right-eye image and the left-eye image are clear. May stand out and be unable to fuse properly. In addition, in order to fuse the displayed images properly, a great burden is placed on the viewer's brain, and the viewer may get tired.

  Compared with the case where both the right-eye image and the left-eye image are clear, the image processing unit 120 can reduce inconsistency by blurring at least one of the left-eye image and the right-eye image. Even if the image is blurred, the viewer's brain can conveniently recognize the blurred image with the other image, so the display image can be recognized as a three-dimensional image relatively easily. it can. In addition, since the viewer's brain can conveniently recognize a blurred image as a clear image, the image quality of the image recognized by the viewer is not substantially deteriorated.

  By the blurring process by the image processing unit 120, the viewer can recognize the display image as a clear stereoscopic image with a small burden. For this reason, the viewer can continue to appreciate the stereoscopic image provided by the display device 10 for a relatively long time.

  Note that the image processing unit 120 calculates the sharpness of at least one of the input image, the right-eye image, and the left-eye image, and the right eye is provided on the condition that the sharpness is larger than a predetermined value. Blur processing may be performed on at least one of the image for use and the image for the left eye. For example, the image processing unit 120 may determine the definition based on a spatial frequency component included in at least one of the right-eye image and the left-eye image. For example, when the input image has a higher spatial frequency component in a spatial frequency region higher than a predetermined spatial frequency, the image processing unit 120 may determine a sharpness with a larger value.

  In addition, the image processing unit 120 divides each of the right-eye image and the left-eye image into a plurality of stripe images having a predetermined width in a predetermined direction. Then, the image processing unit 120 generates a display image in which the stripe image of the right eye image and the stripe image of the left eye image are alternately arranged in a predetermined image arrangement direction.

  The display image generated by the image processing unit 120 is supplied to the display unit 140. The display unit 140 displays the image for the right eye that should be viewed with the right eye of the viewer and the image for the left eye that should be viewed with the left eye of the viewer, generated by the image generation unit 110. Specifically, the display unit 140 displays a display image in which a stripe image of the right eye image and a stripe image of the left eye image are alternately arranged. As described below, each stripe image (R) from the image for the right eye displayed on the display unit 140 is directed to the right eye of the viewer by the image guide unit 150 and displayed on the display unit 140. Each stripe image (L) from the image for the left eye is directed to the left eye of the viewer by the image guide unit 150.

  The image guide unit 150 may be a so-called parallax barrier in which transmission parts and light shielding parts are alternately arranged. For example, the image guide unit 150 includes alternately transmissive portions and light shielding portions. The transmission part and the light shielding part are alternately arranged in the barrier arrangement direction orthogonal to the extending direction, and the stripe images corresponding to the right eye image and the left eye image displayed by the display unit 140 are emitted in different directions for viewing. Present to the right and left eyes of the viewer. With such a configuration, a stereoscopic image can be presented to a naked eye viewer.

  As described above, when there is a difference in the position of the corresponding point of the corresponding object on the display screen of the display unit 140 between the right-eye image and the left-eye image included in the display image, the viewing is performed. The person can recognize the display image three-dimensionally. For example, if the corresponding points of the corresponding objects in the right eye image and the left eye image are shifted by an amount close to the distance between the pupils of the viewer, the eyes of the viewer's left eye and right eye are substantially It becomes a parallel state. This makes it possible for the viewer to recognize an image as if looking at infinity.

  As will be described below, the display device 10 makes the viewer feel more uncomfortable by adjusting the amount of deviation based on the position or size of the main object included in the right-eye image and the left-eye image. It is possible to present a display image without any image. The main object is specified by the main object specifying unit 170. Specifically, the main object specifying unit 170 may specify the main object from the input image that is the source of the right-eye image and the left-eye image.

  Specifically, the main object specifying unit 170 may compare the template image of the object to be extracted as the main object with the input image by template matching or the like, and specify the main object based on the comparison result. For example, the main object specifying unit 170 may compare an object's face image with an input image, and specify an object having a degree of matching greater than a predetermined value as a main object.

  When the input image is a captured image and the focus information is attached, the main object specifying unit 170 may specify the object at the focus position as the main object. The main object specifying unit 170 may specify not only the object at the focus position but also the object that is consequently in focus as the main object. When the input image is a captured image, the main subject object is an example of the main object.

  In addition, the main object specifying unit 170 may specify the object in the center image area as the main object. For example, the object in the center image area may be specified as the main object on the condition that the main object could not be specified based on the template matching and focus position described above. In addition, when information specifying a main object such as position information where the main object exists is attached to the input image, the main object specifying unit 170 specifies the object specified by the information as the main object. Good.

  The deviation amount determination unit 160 is a deviation amount, which is a difference in the position of the main object on the display screen between the right-eye image and the left-eye image based on the position or size of the main object on the display screen. To decide. Then, the image generation unit 110 generates a right-eye image and a left-eye image in which the position of the main object on the display screen is different by the shift amount determined by the shift amount determination unit 160. Specifically, the image generation unit 110 generates a right-eye image and a left-eye image in which the two-dimensional image is shifted left and right by the shift amount determined by the shift amount determination unit 160 on the display screen.

  Here, the deviation amount determination unit 160 determines the deviation amount by a method described below as an example. For example, the shift amount determination unit 160 determines the shift amount to be smaller when the position of the main object on the display screen is closer to the end of the display screen. When the viewer is looking at the peripheral area of the display screen of the display unit 140, at least part of the right-eye image appears to the left eye, or at least part of the left-eye image appears to the right eye. The possibility of seeing is relatively high. The shift amount determination unit 160 determines a smaller shift amount when the main object is present in the vicinity of the image, so that the right eye image can be seen by the left eye or the left eye image can be viewed by the right eye. Even if it appears, the discomfort felt by the viewer is further reduced.

  In addition, when the image generation unit 110 generates the right-eye image and the left-eye image by shifting the two-dimensional image, the right-eye image and the left-eye image are displayed in the end region of the display screen as described later. There is a region where and do not correspond. In addition, if an attempt is made to express on the display screen a space different from the real space around the display screen, the viewer may feel a strong sense of discomfort due to the difference from the real space in the end area of the display screen. If there is a main object in such an end area, the viewer may not be able to see the main object with either eye, or the main object may feel uncomfortable. . When the main object is present in the vicinity of the image by adjusting the shift amount by the shift amount determination unit 160, it is possible to determine that the main object is positioned in the end region described above by determining a smaller shift amount. May be able to prevent.

  Further, the displacement amount determination unit 160 may determine the displacement amount smaller when the size of the main object on the display screen is larger. For example, it is assumed that the main object is a relatively large object obtained by close-up photography. In this case, if the amount of deviation is increased, it will be recognized that a large main object exists in the distance, so that the viewer will feel strongly uncomfortable. On the other hand, if a large object is brought too far, the difference from an actual three-dimensional space object becomes conspicuous, making it difficult for the viewer to cause parallax fusion. When the larger the size of the main object on the display screen is, the smaller the amount of deviation is determined, so that it is possible to prevent the viewer from feeling uncomfortable or difficult to cause parallax fusion. There is.

  Here, the size of the main object may be the size of the main object with respect to the size of the display screen. That is, the shift amount determination unit 160 may determine a shift amount smaller when the size of the main object is larger than the size of the display screen. When generating a right-eye image and a left-eye image with a shift amount by shifting the entire image, if the shift width is constant, the smaller the display screen is, the more the image is cut off at the edge region of the screen. It gets bigger. Therefore, it is possible to prevent the main object from being cut out and displayed by reducing the shift amount as the display screen is smaller. If the display screen is small, the viewer can feel a sense of depth even if the amount of displacement is made relatively small so that the object surface is slightly behind the display surface. For this reason, even if the deviation amount is reduced by the deviation amount determination unit 160, the viewer can sufficiently enjoy the stereoscopic image.

  When the input image is a two-dimensional image from two viewpoints, such as when the image is captured by a compound eye camera, and already includes a deviation amount, the two-dimensional image from the right viewpoint and the left viewpoint are used. By shifting the two-dimensional images as follows, a right-eye image and a left-eye image are generated, respectively. That is, the main object specifying unit 170 specifies the corresponding main object from the right eye image and the left eye image, and supplies information indicating the position and size of the main object to the deviation amount determining unit 160. The displacement amount determination unit 160 determines the displacement amount of the main object based on the position and size indicated by the information acquired from the main object specifying unit 170, and information indicating the determined displacement amount and information indicating the position of the main object The main object is supplied to the image generation unit 110. Then, based on the position of the main object indicated by the information supplied from the deviation amount determination unit 160, the image generation unit 110 performs a main subject in the two-dimensional image from the right viewpoint and the two-dimensional image from the left viewpoint. By shifting the two-dimensional image from the right viewpoint and the two-dimensional image from the left viewpoint to the right and left so that the amount of deviation becomes the amount of deviation indicated by the information supplied from the deviation amount determination unit 160, the right An eye image and a left eye image are generated.

  FIG. 2 is a diagram for explaining an operation example of the image processing unit 120. As described above, the image processing unit 120 divides the left-eye image and the right-eye image into a plurality of stripe images (L1, L2, L3,..., R1, R2, R3,...). To do. The image processing unit 120 of this example divides the left-eye image and the right-eye image at predetermined intervals along the horizontal arrangement direction.

  The image processing unit 120 generates the display image by arranging the generated stripe images alternately in the arrangement direction of the right-eye image and the left-eye image. Since the display unit 140 simultaneously displays the right eye image and the left eye image included in the display image, the image processing unit 120 arranges the stripe image arrangement for each of the left eye image and the right eye image. The number of display pixels in the direction is approximately halved. More specifically, the image processing unit 120 uses every other stripe image of the left eye image and the right eye image in the arrangement direction.

  FIG. 3 is a diagram illustrating the image guide unit 150. The image guide unit 150 includes a barrier unit 310 and a barrier control unit 320 provided on the viewer side of the display unit 140. The barrier unit 310 guides the right-eye image and the left-eye image in different directions.

  Specifically, the barrier unit 310 includes a transmission unit 312 and a light shielding unit 314 provided alternately along a predetermined arrangement direction on the viewer side of the display unit 140. One transmissive portion 312 is formed for each combination of one stripe image of the left eye image and one stripe image of the right eye image. Each transmission unit 312 emits a corresponding stripe image of the left-eye image in the direction of the left eye of the viewer, and outputs a corresponding stripe image of the right-eye image in the direction of the right eye of the viewer. Eject.

  The barrier unit 310 may be formed by arranging a plurality of shutter elements. Specifically, the barrier unit 310 may include a plurality of shutter elements arranged in a matrix along the horizontal direction and the vertical direction. The barrier control unit 320 controls the extension direction and the arrangement direction of the transmission part and the light shielding part by individually setting on / off of these shutter elements. That is, the barrier control unit 320 turns on the shutter element in the region where the transmission part is to be formed and turns off the shutter element in the region where the light shielding part is to be formed. Form. As described above, the barrier control unit 320 controls whether or not light is transmitted through the plurality of shutter elements, whereby the light shielding unit 314 that shields light and the transmission unit 312 that transmits light are alternately arranged in stripes. Let it form.

  Here, when a so-called two-dimensional image is presented to the viewer instead of a stereoscopic image, the barrier control unit 320 substantially sets all shutter elements corresponding to the display area of the two-dimensional image on the display unit 140. Turn on. Thereby, the two-dimensional image displayed on the display unit 140 is transmitted by the transmission unit and visually recognized by the right and left eyes of the viewer. Even in the state where the light shielding units 314 and the transmission units 312 are alternately arranged, a two-dimensional image can be viewed by generating a display image from the same right-eye image and left-eye image with no deviation. Can be presented to the person.

  In addition to the parallax barrier type image guide unit 150 illustrated in the drawing, a lenticular lens type image guide unit 150 may be illustrated. For example, instead of controlling the guide direction of the image by the barrier unit 310, the guide direction of the image can be changed by a lenticular lens. Specifically, a lens array that guides a plurality of right-eye stripe images and a plurality of left-eye stripe images in different directions causes the corresponding left-eye stripe images to be directed toward the viewer's left eye. The corresponding stripe image of the right-eye image can be directed toward the viewer's right eye. Thereby, the image guide unit 150 can guide the plurality of right-eye stripe images and the plurality of left-eye stripe images in different directions.

  FIG. 4 is a diagram for explaining an operation example of the image generation unit 110. FIG. 4 shows image data whose coordinates are defined with respect to the horizontal direction x and the vertical direction y of the pixel array of the display unit 140. Here, for the sake of simplicity, a process for generating a left-eye image and a right-eye image having a shift amount L, which is a distance equal to or smaller than the interpupillary distance of the viewer, by shifting the entire input image that is a two-dimensional image. explain. Specifically, the image generation unit 110 generates an image obtained by shifting a given two-dimensional image to the left on a predetermined axis by L / 2 as an image for the left eye, and an image obtained by shifting L / 2 to the right on the axis. Is generated as a right-eye image. As described above, the image generation unit 110 generates a right-eye image and a left-eye image in which a two-dimensional image is shifted left and right on the display screen.

  FIG. 5 shows a processing example based on the position of the main object by the deviation amount determination unit 160. The deviation amount determination unit 160 stores a deviation amount in advance for each image area where the main object exists. Here, the shift amount determination unit 160 stores the first shift amount L1 in association with the first image region 500 that is the central image region. Further, the shift amount determination unit 160 stores a second shift amount L2 (<L1) in association with the second image region 510a and the second image region 510b located on the left and right of the first image region 500. Further, the shift amount determination unit 160 associates the third shift amount L3 with the third image region 520a positioned to the left of the second image region 510a and the third image region 520b positioned to the right of the second image region 510b. (<L2) is stored. Note that the deviation amount determination unit 160 may store the deviation amount 0 in association with the image region closest to the end portion (in the example of this figure, the third image region 520).

  The displacement amount determination unit 160 determines the first displacement amount L1 as the displacement amount when the position of the main object specified by the main object specification unit 170 is included in the first image region 500. Further, when the position of the main object is included in the second image region 510, the deviation amount determination unit 160 determines the second deviation amount L2 as the deviation amount. Further, when the position of the main object is included in the third image region 520, the deviation amount determination unit 160 determines the third deviation amount L3 as the deviation amount. Then, the image generation unit 110 shifts the two-dimensional image based on the shift amount determined by the shift amount determination unit 160 to generate a right-eye image and a left-eye image. Thereby, the display image in which the shift amount is adjusted by the shift amount determination unit 160 can be presented to the viewer.

  The main object specifying unit 170 may specify the existence range of the main object as the position of the main object. Then, when it is determined that the main object exists over the first image area 500 and the second image area 510 based on the main object existence range, the deviation amount determination unit 160 is associated with these image areas. The smaller second displacement amount L2 out of the displacement amounts may be determined more preferentially as the display image displacement amount. Similarly, the displacement amount determination unit 160 may determine the third displacement amount L3 as the displacement amount when it is determined that the displacement exists over the second image region 510 and the third image region 520.

  Further, the main object specifying unit 170 may specify a plurality of main objects. In this case, the deviation amount determination unit 160 may determine the deviation amount based on the positions and sizes of the plurality of main objects. For example, when it is determined that at least a part of the main object exists in the first image region 500 and the second image region 510, the displacement amount determination unit 160 determines the displacement amount associated with the image region. The smaller second displacement amount L2 may be determined with higher priority as the display image displacement amount. Similarly, when it is determined that at least a part of the main object exists in the third image region 520, the shift amount determination unit 160 may determine the third shift amount L3 as the shift amount of the display image.

  As described above, the shift amount determination unit 160 stores a smaller shift amount in association with the image region closer to the left and right end portions. For this reason, when the main object is present in the peripheral area where the viewer is difficult to fuse the parallax, it is possible to provide a display image with a relatively small amount of deviation, and the viewer feels a sense of incongruity strongly. Can be prevented. On the other hand, when the main object exists in the central image area, a relatively large amount of deviation is determined. In this case, although it is difficult for the viewer to cause parallax fusion with respect to the image in the peripheral area, the viewer does not feel a sense of discomfort strongly because the viewer can easily focus on the main object. For this reason, when the main object exists in the central image area, a stereoscopic image with a relatively large shift amount and a relatively enhanced depth feeling can be presented to the viewer. As described above, according to the display device 10, it is possible to control the shift amount in a desirable manner in accordance with the position of the main object that is easily noticed by the viewer.

  FIG. 6 shows a processing example based on the size of the main object by the deviation amount determination unit 160. The shift amount determination unit 160 stores a shift amount in advance for each size of the main object. Specifically, the deviation amount determination unit 160 stores the deviation amount α in association with the range of magnitude from 0 to a threshold value T1. Further, the shift amount determination unit 160 stores in advance a shift amount that monotonously decreases from the shift amount α to the shift amount 0 in association with a range of magnitude from the threshold value T1 to the threshold value T2. That is, the shift amount determination unit 160 stores a smaller shift amount in association with a larger main object size. Further, the deviation amount determination unit 160 stores a deviation amount 0 in association with a range having a size equal to or greater than the threshold T2. Then, the shift amount determination unit 160 determines the shift amount stored in advance in association with the range including the size of the main object specified by the main object specifying unit 170 as the shift amount of the display image.

  An example of the size of the main object is the width in the left-right direction of the main object. Specifically, the shift amount determination unit 160 stores a shift amount in advance in association with each of a plurality of ranges related to the width in the left-right direction of the main object. For example, the main object specifying unit 170 specifies the width of the main object in the left-right direction, and supplies information indicating the specified width to the deviation amount determining unit 160. Then, the shift amount determination unit 160 determines the shift amount stored in advance in association with the range including the width of the main object specified by the main object specifying unit 170 as the shift amount of the display image.

  In addition, the area of the main object can be exemplified as the size of the main object. Specifically, the deviation amount determination unit 160 stores a plurality of different deviation amounts in advance in association with a plurality of ranges of the area of the main object. The main object specifying unit 170 specifies the area of the main object and supplies information indicating the specified area to the deviation amount determining unit 160. Then, the shift amount determination unit 160 determines the shift amount stored in advance in association with the range including the area of the main object specified by the main object specifying unit 170 as the shift amount of the display image.

  It should be noted that the size of the main object can be determined by using the ratio of the size of the main object to the size of the display screen of the display unit 140 as an index. Specifically, the size of the main object may be a ratio of the width of the main object to the horizontal width of the display screen of the display unit 140 in the left-right direction. The size of the main object may be the ratio of the area of the main object to the area of the display screen of the display unit 140.

  FIG. 7 is a flowchart illustrating an operation example of the display device 10. In step S <b> 702, the image acquisition unit 100 acquires an input image that is a source of a display image to be displayed on the display unit 140. In S704, the main object specifying unit 170 specifies a main object from the input image. Specifically, in S704, an object similar to a predetermined specific object (for example, a person, an animal head object, etc.) that is easily noticed by the viewer is extracted, and the extracted object is used as the main object. As specified.

  In S706, the main object specifying unit 170 determines whether the main object has been specified. When the main object can be specified, the main object specifying unit 170 supplies information indicating the position and area of the main object to the deviation amount determining unit 160. On the other hand, if the main object cannot be specified, the main object specifying unit 170 supplies information indicating that the main object cannot be specified to the deviation amount determining unit 160.

  If the main object can be identified in S706 (YES), in S708, the shift amount determination unit 160 determines the shift amount based on the position and area of the main object. In this case, the shift amount determination unit 160 compares the shift amount based on the position of the main object described with reference to FIG. 5 with the shift amount based on the area of the main object described with reference to FIG. The smaller shift amount may be determined as the shift amount of the display image.

  If the main object can be detected in S706 (NO), in S710, the shift amount determination unit 160 determines the default shift amount determined for the input image as the shift amount of the display image. When the input image is a captured image, the default shift amount may be a shift amount that is determined according to the distance to the imaged subject. For example, a larger predetermined shift amount may be set as the distance to the subject is larger. The predetermined shift amount may be determined by the shift amount determination unit 160 based on distance information attached to the input image, or the default shift amount itself may be attached to the input image. In addition, the predetermined shift amount may be a shift amount determined by the creator of the input image or the like.

  In step S710, the deviation amount determination unit 160 may determine the deviation amount based on the size of the object located in the center area of the image, for example, by the method described in relation to FIG. Since the viewer can easily focus on the central area of the image, the size of the object in the central area of the image may greatly affect the likelihood of parallax fusion. Therefore, even if a specific object that the viewer is interested in is not specified, the viewer feels uncomfortable by determining the amount of deviation based on the size of the object located in the central area of the image. It is possible to reduce the possibility of being lost.

  In S712, the image generation unit 110 generates a right-eye image and a left-eye image by relatively shifting the input image according to the shift amount determined in S708 or S710. In step S <b> 714, the image processing unit 120 generates a display image from the right eye image and the left eye image and causes the display unit 140 to display the display image. In S716, it is determined whether or not to end the image display. For example, when images are presented for all predetermined input images (YES), it may be determined to end the image display. If there is something that has not been presented (NO), the process proceeds to S702 to select the next input image.

  FIG. 8 shows a processing example of image edge portions in the left eye image and the right eye image. The processing example in this figure shows an example of a left-eye image to which a single-color right-end image is added and an example of a right-eye image to which a single-color left-end image is added. For example, the image processing unit 120 adds a single-color right-end image displayed within a predetermined range from the right end of the display area to the right side of the left-eye image. As an example, the image processing unit 120 adds the right end image of dark color such as black or light color such as white displayed within a predetermined range from the right end of the display area to the right side of the left eye image. As an example, the image processing unit 120 shifts the original two-dimensional image to the left, and the right end of a dark color such as black or a light color such as white is displayed on the right side of the left eye image in the display area. Add an image.

  Further, as an example, the image processing unit 120 adds a left end image of a dark color such as black or a light color such as white, which is displayed within a predetermined range from the left end of the display area, to the left side of the right eye image. As an example, the image processing unit 120 can be obtained by shifting the original two-dimensional image in the right direction, and the left end of a dark color such as black or a light color such as white in the left blank portion of the right-eye image in the display area. Add an image.

  The image processing unit 120 can display such that a frame is conscious at the right end and the left end in the display area by adding such a right end image and a left end image. Thereby, the image processing unit 120 can provide a viewer with a natural three-dimensional image that looks like the scenery outside from the window frame.

  Further, as an example, the image processing unit 120 has a dark frame image such as black or a light color frame such as white having a width equal to or greater than the distance between the pupils on the left side and the right side in the image for the left eye to which the right end image is added. Is further added. Similarly, the image processing unit 120, for example, in the right eye image to which the left end image is added, on each of the right side and the left side, a dark color frame such as black or a light color frame such as white having a width equal to or greater than the interpupillary distance. Add more images. Thereby, the image processing unit 120 can display a clear frame in the display screen.

  FIG. 9 shows another example of processing at the image edge, an example of an image for the left eye to which a right end image obtained by copying a part of the image for the right eye is added, and a left end image obtained by copying a part of the image for the left eye. An example of the added image for the right eye is shown. As an example, the image processing unit 120 adds, as a right end image, a portion displayed within a predetermined range from the right end of the display area in the right eye image on the right side of the left eye image. As an example, the image processing unit 120 copies a portion displayed within a range from the right end of the display area in the right eye image to a position corresponding to the right end of the left eye image, and uses the left eye image as the right end image. Append to the right side of the image.

  For example, the image processing unit 120 adds, as a left end image, a portion displayed within a predetermined range from the left end of the display area in the left eye image on the left side of the right eye image. As an example, the image processing unit 120 copies a portion displayed within a range from the left end of the display region in the left eye image to a position corresponding to the left end of the right eye image, and uses the left eye image as the left end image. Append to the left side of the image.

  When convergence occurs when attention is paid to the right end or left end of the display area, there is a problem of visual field struggle that is different between what is seen between the right eye and the left eye. In this case, the greater the distance between the image given to the right eye and the image given to the left eye, the greater the unnatural feeling given to humans. However, the image processing unit 120 according to the present example gives the same image to the right eye and the left eye at the right end or the left end of the display area. Therefore, the image processing unit 120 according to the present example can provide an image that does not feel unnatural even when congestion occurs in the right end portion or the left end portion of the display area, and the visual field struggle is reduced. it can. It should be noted that, in the central portion of the display area, the condition for fusion of both eyes is maintained, so that it is less likely that congestion will occur compared to the end portion.

  Further, as an example, the image processing unit 120 may add, as a right end image, an image obtained by blurring a portion displayed within a predetermined range from the right end of the display area in the right eye image on the right side of the left eye image. Good. Further, as an example, the image processing unit 120 may add an image obtained by blurring a portion displayed within a predetermined range from the left end of the display area in the left eye image as the left end image on the left side of the right eye image. Good.

  When images with different focus levels are given to the right eye and the left eye, a human feels the given image three-dimensionally. Therefore, such an image processing unit 120 can reduce the visual field struggle and make the viewer feel a stereoscopic effect when congestion occurs in the right end portion or the left end portion of the display area.

  For example, the image processing unit 120 may use the color of the right end image as the average color of the portion of the right eye image that is displayed within a predetermined range from the right end of the display area. Further, in addition to or instead of this, the image processing unit 120 may set the luminance of the right-end image as the average luminance of the portion of the right-eye image that is displayed within a predetermined range from the right end of the display area. . Further, as an example, the image processing unit 120 may set the color of the left end image as an average color of a portion displayed in a predetermined range from the left end of the display area in the left eye image. Further, in addition to or instead of this, the image processing unit 120 may set the luminance of the left-end image as the average luminance of the portion of the left-eye image that is displayed within a predetermined range from the left end of the display area. . Such an image processing unit 120 can give the left eye and the right eye images having the same hue or luminance when congestion occurs in the right end portion or the left end portion of the display area. Can do.

  FIG. 10 shows an example of an image for the left eye enlarged in the right direction and an example of an image for the right eye enlarged in the left direction as still another processing example of the image edge. As an example, the image processing unit 120 enlarges the left eye image to the right and generates an image including the left eye image and the right end image. In this case, the image processing unit 120 enlarges the left-eye image to the right at a larger enlargement ratio as it approaches the left and right ends, and generates an image including the left-eye image and the right-end image. Further, in this case, the image processing unit 120 may not enlarge the image on the left side from a predetermined position (for example, the center position) of the left-eye image.

  Further, as an example, the image processing unit 120 enlarges the right eye image to the left, and generates an image including the right eye image and the left end image. In this case, the image processing unit 120 generates an image including the right-eye image and the left-end image by enlarging the right-eye image to the left with a larger enlargement ratio as it is closer to the left and right ends. Furthermore, in this case, the image processing unit 120 does not have to enlarge the image on the right side from a predetermined position (for example, the center position) of the right-eye image. The image processing unit 120 according to this example can provide a natural image in which images are continuous at the end.

  Further, the image processing unit 120 and the image processing unit 120 may enlarge the image for the left eye and the image for the right eye up and down in accordance with the enlargement ratio in the right direction and the left direction. In this case, the image processing unit 120 and the image processing unit 120 delete portions that are out of the display area in the left-eye image and the right-eye image as a result of being enlarged vertically.

  Further, as an example, the image processing unit 120 displays, on the right side of the left-eye image, a portion displayed within a predetermined range from the right end of the display area in the left-eye image (for example, one column or several columns from the right end). The pixel column) may be added as a right end image. In addition, as an example, the image processing unit 120 displays, on the left side of the right-eye image, a portion displayed within a predetermined range from the left end of the display area in the right-eye image (for example, one column or several columns from the left end). The pixel column) may be added as a left-end image. Such an image processing unit 120 can provide a natural image with continuous images at the end.

  FIG. 11 shows an example of a left-eye image from which the left end portion has been deleted and an example of a right-eye image from which the right end portion has been deleted as still another processing example of the image end portion. For example, instead of adding the right end image to the right side of the left eye image, the image processing unit 120 deletes a predetermined range on the right side of the right eye image. Further, as an example, the image processing unit 120 deletes a predetermined range on the left side of the left eye image instead of adding the left end image to the left side of the right eye image.

  Such an image processing unit 120 can display an image without an image area where a stereoscopic image cannot be provided. Here, the display unit 140 outputs a dark color (for example, a black image) or the like from a portion where the image in the display area is deleted. Therefore, such an image processing unit 120 can provide a natural stereoscopic image as in the case where the outer frame is provided.

  FIG. 12 shows an example of a case where an image for the left eye and an image for the right eye are generated after enlarging the two-dimensional image as still another processing example of the image edge. For example, the image processing unit 120 may perform the following processing.

  First, the image generation unit 110 enlarges the two-dimensional image left and right by a predetermined distance. For example, the image generation unit 110 enlarges the horizontal width in the left-right direction so that the horizontal width becomes a distance obtained by adding the horizontal width of the display area and the inter-pupil distance. In this case, the image generation unit 110 may enlarge the two-dimensional image in the vertical direction at the same magnification as that in the horizontal direction.

  Subsequently, the image generation unit 110 generates a left-eye image and a right-eye image in which the enlarged two-dimensional image is shifted left and right by a predetermined distance within the display area. As an example, the image generation unit 110 shifts the left-eye image to the left by a distance corresponding to ½ of the inter-pupil distance, and shifts the right-eye image to the right side by a distance corresponding to ½ the inter-pupil distance.

  Subsequently, the image processing unit 120 deletes a predetermined distance range on the right side of the right eye image instead of adding the right end image to the right side of the left eye image. As an example, the image processing unit 120 deletes the range corresponding to the interpupillary distance from the right end of the right-eye image.

  Further, the image processing unit 120 deletes a predetermined distance range on the left side of the left eye image instead of adding the left end image to the left side of the right eye image. As an example, the image processing unit 120 deletes the range corresponding to the inter-pupil distance from the left end of the left-eye image.

  Such an image processing unit 120 can display an image without an image area where a stereoscopic image cannot be provided. Therefore, such an image processing unit 120 can provide a natural stereoscopic image. Note that when the two-dimensional image is enlarged in the vertical direction, the image processing unit 120 may delete a portion of the left-eye image and the right-eye image that exceeds the upper and lower boundaries of the display range.

  FIG. 13 shows an example of a hardware configuration of a computer 800 according to another embodiment. The computer 800 corresponds to a given program, the image acquisition unit 100, the image generation unit 110, the image processing unit 120, the display unit 140, the deviation amount determination unit 160, and the main components described with reference to FIGS. It functions as the object specifying unit 170. The computer 800 may further function as the barrier control unit 320.

  A computer 800 according to the present embodiment is connected to a CPU peripheral unit having a CPU 2000, a RAM 2020, a graphic controller 2075, and a display device 2080 connected to each other by a host controller 2082, and to the host controller 2082 by an input / output controller 2084. Input / output unit having communication interface 2030, hard disk drive 2040, and CD-ROM drive 2060, and legacy input / output unit having ROM 2010, flexible disk drive 2050, and input / output chip 2070 connected to input / output controller 2084 With.

  The host controller 2082 connects the RAM 2020 to the CPU 2000 and the graphic controller 2075 that access the RAM 2020 at a high transfer rate. The CPU 2000 operates based on programs stored in the ROM 2010 and the RAM 2020 and controls each unit. The graphic controller 2075 acquires image data generated by the CPU 2000 or the like on a frame buffer provided in the RAM 2020 and displays it on the display device 2080. Instead of this, the graphic controller 2075 may include a frame buffer for storing image data generated by the CPU 2000 or the like.

  The input / output controller 2084 connects the host controller 2082 to the communication interface 2030, the hard disk drive 2040, and the CD-ROM drive 2060, which are relatively high-speed input / output devices. The communication interface 2030 communicates with other devices via a network. The hard disk drive 2040 stores programs and data used by the CPU 2000 in the computer 800. The CD-ROM drive 2060 reads a program or data from the CD-ROM 2095 and provides it to the hard disk drive 2040 via the RAM 2020.

  The input / output controller 2084 is connected to the ROM 2010, the flexible disk drive 2050, and the relatively low-speed input / output device of the input / output chip 2070. The ROM 2010 stores a boot program that is executed when the computer 800 is started and / or a program that depends on the hardware of the computer 800. The flexible disk drive 2050 reads a program or data from the flexible disk 2090 and provides it to the hard disk drive 2040 via the RAM 2020. The input / output chip 2070 connects the flexible disk drive 2050 to the input / output controller 2084 and inputs / outputs various input / output devices via, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like. Connect to controller 2084.

  A program provided to the hard disk drive 2040 via the RAM 2020 is stored in a recording medium such as the flexible disk 2090, the CD-ROM 2095, or an IC card and provided by the user. The program is read from the recording medium, installed in the hard disk drive 2040 in the computer 800 via the RAM 2020, and executed by the CPU 2000.

  A program installed in the computer 800 and executed by the computer 800 works on the CPU 2000 or the like to cause the computer 800 to operate as an image acquisition unit 100, an image generation unit 110, an image processing unit 120, a display unit 140, a deviation amount determination unit 160, And it is made to function as the main object specific | specification part 170 grade | etc.,. The information processing described in these programs is read into the computer 800, whereby the image acquisition unit 100, the image generation unit 110, the image, which are specific means in which the software and the various hardware resources described above cooperate with each other. It functions as the processing unit 120, the display unit 140, the deviation amount determination unit 160, the main object specifying unit 170, and the like. Then, by realizing the calculation or processing of information according to the purpose of use of the computer 800 in the present embodiment by these specific means, the specific image acquisition unit 100, the image generation unit 110, the image according to the purpose of use are realized. A processing unit 120, a display unit 140, a deviation amount determining unit 160, a main object specifying unit 170, and the like are constructed.

  As an example, when communication is performed between the computer 800 and an external device or the like, the CPU 2000 executes a communication program loaded on the RAM 2020, and based on the processing contents described in the communication program, the communication interface A communication process is instructed to 2030. Under the control of the CPU 2000, the communication interface 2030 reads transmission data stored in a transmission buffer area or the like provided on a storage device such as the RAM 2020, the hard disk drive 2040, the flexible disk 2090, or the CD-ROM 2095, and sends it to the network. The reception data transmitted or received from the network is written into a reception buffer area or the like provided on the storage device. As described above, the communication interface 2030 may transfer transmission / reception data to / from the storage device by a DMA (direct memory access) method. Instead, the CPU 2000 transfers the storage device or the communication interface 2030 as a transfer source. The transmission / reception data may be transferred by reading the data from the data and writing the data to the communication interface 2030 or the storage device of the transfer destination.

  The CPU 2000 is all or necessary from among files or databases stored in an external storage device such as a hard disk drive 2040, a CD-ROM drive 2060 (CD-ROM 2095), and a flexible disk drive 2050 (flexible disk 2090). This portion is read into the RAM 2020 by DMA transfer or the like, and various processes are performed on the data on the RAM 2020. Then, CPU 2000 writes the processed data back to the external storage device by DMA transfer or the like. In such processing, since the RAM 2020 can be regarded as temporarily holding the contents of the external storage device, in the present embodiment, the RAM 2020 and the external storage device are collectively referred to as a memory, a storage unit, or a storage device. Various types of information such as various programs, data, tables, and databases in the present embodiment are stored on such a storage device and are subjected to information processing. Note that the CPU 2000 can also hold a part of the RAM 2020 in the cache memory and perform reading and writing on the cache memory. Even in such a form, the cache memory bears a part of the function of the RAM 2020. Therefore, in the present embodiment, the cache memory is also included in the RAM 2020, the memory, and / or the storage device unless otherwise indicated. To do.

  In addition, the CPU 2000 performs various operations, such as various operations, information processing, condition determination, information search / replacement, etc., described in the present embodiment, specified for the data read from the RAM 2020 by the instruction sequence of the program. Is written back to the RAM 2020. For example, when performing the condition determination, the CPU 2000 determines whether or not the various variables shown in the present embodiment satisfy the conditions such as large, small, above, below, equal, etc., compared to other variables or constants. If the condition is satisfied (or not satisfied), the program branches to a different instruction sequence or calls a subroutine.

  Further, the CPU 2000 can search for information stored in a file or database in the storage device. For example, in the case where a plurality of entries in which the attribute value of the second attribute is associated with the attribute value of the first attribute are stored in the storage device, the CPU 2000 displays the plurality of entries stored in the storage device. The entry that matches the condition in which the attribute value of the first attribute is specified is retrieved, and the attribute value of the second attribute that is stored in the entry is read, thereby associating with the first attribute that satisfies the predetermined condition The attribute value of the specified second attribute can be obtained.

  The program or module shown above may be stored in an external recording medium. As the recording medium, in addition to the flexible disk 2090 and the CD-ROM 2095, an optical recording medium such as DVD or CD, a magneto-optical recording medium such as MO, a tape medium, a semiconductor memory such as an IC card, and the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and the program may be provided to the computer 800 via the network.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 10 Display apparatus, 100 Image acquisition part, 110 Image generation part, 120 Image processing part, 140 Display part, 150 Image guidance part, 160 Deviation amount determination part, 170 Main object specific part, 310 Barrier part, 312 Transmission part, 314 Light shielding Part, 320 barrier control part, 500 first image area, 510 second image area, 520 third image area, 800 computer, 2000 CPU, 2010 ROM, 2020 RAM, 2030 communication interface, 2040 hard disk drive, 2050 flexible disk drive 2060 CD-ROM drive, 2070 input / output chip, 2075 graphic controller, 2080 display device, 2082 host controller, 2084 input / output controller, 2090 flexible Disc, 2095 CD-ROM

Claims (8)

An image generation device that generates an image for the right eye that should be viewed with the right eye of the viewer and an image for the left eye that should be viewed with the left eye of the viewer,
A main object specifying unit that specifies main objects included in the right-eye image and the left-eye image;
Based on the position or size of the main object on the display screen, a shift amount, which is a difference in the position of the main object on the display screen, between the right-eye image and the left-eye image is determined. A deviation amount determination unit to perform,
An image generation apparatus comprising: an image generation unit configured to generate the right-eye image and the left-eye image in which the position of the main object on the display screen is different by the shift amount.   The image generation device according to claim 1, wherein the image generation unit generates the right-eye image and the left-eye image obtained by shifting a two-dimensional image left and right on the display screen.   The image generation apparatus according to claim 1, wherein the shift amount determination unit determines the shift amount to be smaller when the position of the main object on the display screen is closer to an end of the display screen. .   The image generation apparatus according to claim 1, wherein the shift amount determination unit determines the shift amount to be smaller when the size of the main object on the display screen is larger.   The image generation apparatus according to claim 4, wherein the shift amount determination unit determines the shift amount to be smaller when the size of the main object is larger than the size of the display screen. The image generation apparatus according to claim 1, further comprising a display unit that displays the right-eye image and the left-eye image generated by the image generation unit. An image generation method for generating an image for the right eye that should be visually recognized by the right eye of the viewer and an image for the left eye that should be visually recognized by the left eye of the viewer,
A main object specifying step of specifying main objects included in the right-eye image and the left-eye image;
Based on the position or size of the main object on the display screen, a shift amount, which is a difference in the position of the main object on the display screen, between the right-eye image and the left-eye image is determined. The amount of deviation determination stage,
An image generation method comprising: an image generation step of generating the right eye image and the left eye image in which the position of the main object on the display screen differs by the amount of displacement. A program for an image generation device that generates an image for the right eye that should be visually recognized by the right eye of the viewer and an image for the left eye that should be visually recognized by the left eye of the viewer,
A main object specifying unit that specifies main objects included in the right-eye image and the left-eye image;
Based on the position or size of the main object on the display screen, a shift amount, which is a difference in the position of the main object on the display screen, between the right-eye image and the left-eye image is determined. Shift amount determination unit,
A program that functions as an image generation unit that generates the right-eye image and the left-eye image in which the position of the main object on the display screen differs by the amount of deviation.

Images