JP2008117113A - Image forming device and method, and image forming program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily provide a 3D expression based on two-dimensional image data even when an image includes a plurality of three-dimensional objects in one screen. <P>SOLUTION: An image forming device has an image forming part 3 for selecting two-dimensional image data conforming to the direction of a virtual camera from among one or more two-dimensional image data stored in a two-dimensional image storage part 1 and for forming an image showing a view seen from the virtual camera within a 3D space, based on the two-dimensional image data selected. The image forming part 3 includes a virtual model layout part 11 for laying out virtual rectangular parallelepiped three-dimensional models within the 3D space, and an image layout part 12 for laying out within the rectangular parallelepipeds the image planes of the two-dimensional image data selected by the image selecting part 2, so that overlap of the image planes laid out within the rectangular parallelepipeds conforms to the overlap of the rectangular parallelepiped three-dimensional models, so that even the overlapped three-dimensional objects on the 3D space are appropriately expressed by the two-dimensional image data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image generation apparatus and method, and an image generation program, and is particularly suitable for use in an apparatus that generates a three-dimensional image from two-dimensional image data.

  Conventionally, as a technique for making an object appear three-dimensionally on a flat display screen of a computer, there is a three-dimensional modeling technique for generating an image that can be seen from a given viewpoint in an object space that is a virtual three-dimensional space. Has been used. There are three types of three-dimensional modeling: a wire frame model that represents an object only by ridgelines, a surface model that represents an object by a combination of surfaces, and a solid model that handles information about the contents of the object in addition to the surface. An image represented by such three-dimensional modeling is generally called a “3D image” or the like.

  The 3D image is applied to, for example, a virtual reality (VR) technology. In this image generation system using the VR technology, a three-dimensional single three-dimensional model (polygon data) that models a display object such as a character or a vehicle is arranged in an object space, and geometry processing (three-dimensional coordinate calculation) is performed. ) To automatically generate an image visible from the virtual camera. Therefore, if one three-dimensional model is prepared, even if this three-dimensional model is viewed from various directions with a virtual camera, it is possible to generate an image having no contradiction.

  However, a 3D image has a very large amount of information, and complicated and enormous processing is required to display it. In particular, when a plurality of three-dimensional models are displayed in 3D on one screen, polygon processing is required for each three-dimensional model, and the calculation load becomes very heavy. is there.

  Further, unlike the two-dimensional data, polygon data of the three-dimensional model is difficult to obtain because there is no simple input device. The polygon data of the object to be displayed needs to purchase existing polygon data, or ask a specialist or a person with specialized skills to create it, which is laborious and costly.

In contrast, a plurality of two-dimensional image data obtained by viewing a single three-dimensional model from different viewpoints are prepared, and two-dimensional image data that matches the relative angle between the direction of the virtual camera and the direction of the three-dimensional model. There is also a technique for generating an image that can be viewed from a virtual camera in the object space based on the selected two-dimensional image data (see, for example, Patent Document 1). According to this technique, an image of a three-dimensional model viewed from a certain viewpoint can be efficiently generated.
JP 2001-84402 A

  However, although the prior art described in Patent Document 1 refers to a method of generating an image viewed from a certain viewpoint for one three-dimensional object from two-dimensional image data, a plurality of three-dimensional objects are included in one screen. No mention is made of a generation method when an object exists. For example, in the case where a plurality of three-dimensional objects are displayed in a partially overlapped manner, for example, when displaying an image in which a plurality of types of products are placed side by side on a display shelf, the prior art described in Patent Document 1 is used. Can not cope.

  The present invention has been made to solve such a problem, and it is possible to easily perform 3D representation of an image in which a plurality of three-dimensional objects exist within one screen based on two-dimensional image data. The purpose is to be able to.

  In order to solve the above problems, in the present invention, a virtual three-dimensional model of a rectangular parallelepiped is arranged in the object space, and two-dimensional image data matching the direction of the virtual camera is selected from the two-dimensional image storage unit. Read out automatically. Then, by arranging the image plane of the selected two-dimensional image data at the position set in the rectangular parallelepiped composed of one or more virtual three-dimensional models arranged as described above, an image seen from the virtual camera in the object space Is generated.

  According to the present invention configured as described above, the three-dimensional model used to perform 3D representation of an image is simple rectangular parallelepiped polygon data, and since the shape is not complicated, the amount of information is not so large. The load is very light. Since 3D representation is realized simply by placing the image plane of the 2D image data in the rectangular parallelepiped of the 3D model, the virtual camera efficiently generates the 3D model image viewed from various directions. can do.

  Further, according to the present invention, the overlapping degree of the image planes arranged in one or more rectangular parallelepipeds follows the overlapping degree of the three-dimensional model of the rectangular parallelepiped. That is, when two rectangular parallelepipeds overlap, the two-dimensional image planes in the same overlap, and it is determined which two-dimensional image comes to the front. As a result, when a three-dimensional object is stacked or arranged in the object space, naturally the overlapping portion is appropriately represented by the two-dimensional image data.

  Therefore, 3D expression can be easily performed on an image in which a plurality of three-dimensional objects exist within one screen based on the two-dimensional image data.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a functional configuration example of the image generation apparatus according to the present embodiment. As shown in FIG. 1, the image generation apparatus of this embodiment includes a two-dimensional image storage unit 1, an image selection unit 2, and an image generation unit 3 as functional configurations. The image generation unit 3 includes a virtual model arrangement unit 11, an image arrangement unit 12, a three-dimensional model processing unit 13, and an image output unit 14.

  FIG. 2 is a diagram illustrating an example of an image (not a 3D image itself but a 3D expression image that looks like a 3D image) generated by the image generation apparatus according to the present embodiment. The 3D representation image shown in FIG. 2 is an image showing a shelf layout in which a plurality of types of vegetables (corresponding to the three-dimensional object of the present invention) are arranged and placed on a display table. Although the image shown in FIG. 2 looks like a 3D image, only the display table is actually a 3D image (an image based on polygon data), and the vegetable portion is a 2D image. That is, as will be described in detail below, simple 3D representation is performed by arranging 2D images as naturally as possible in a 3D object space.

  In FIG. 1, a two-dimensional image storage unit 1 stores one or more two-dimensional image data obtained by viewing a single three-dimensional object from different viewpoints. Here, one or more two-dimensional image data is prepared for a target object of a three-dimensional object to be displayed in the object space. When there are a plurality of objects to be displayed, one or more two-dimensional image data is prepared for each. The number of two-dimensional image data prepared for one object differs depending on the shape of the object and is about 1 to 40 sheets. FIG. 3 is a diagram illustrating an example of a viewpoint when 40 pieces of two-dimensional image data are prepared. The blank portion is two-dimensional image data viewed from the viewpoint between the upper and lower sides (or left and right), that is, two-dimensional image data at an oblique position. Note that the rightmost column in the table shown in FIG. 3 is connected to the leftmost column.

  For example, when the object is a pineapple, as shown in FIG. 4, only three pieces of two-dimensional image data on the front, right above, and diagonally above are used. The grounds for this are that there is no particular change in the shape of the pineapple when viewed from the front, side, or rear part. Therefore, it can be determined that all the rows in the table shown in FIG. That is, the front, right side, left side, rear, and the images between them may all be the same. And because the pineapple is placed on the display table, the lower image is not necessary. For this reason, all the lower and lower rows can be omitted.

  There may be a case where the number of two-dimensional image data to be prepared for one object is the smallest one. That is the case where the object is not much different from where it is seen. For example, in the case of a ball, one piece of two-dimensional image data viewed from the front may be prepared. Thus, the fact that the number of two-dimensional image data can be greatly reduced is one of the points of the algorithm according to the present embodiment.

  The two-dimensional image storage unit 1 stores information representing the actual size of the target object together with the two-dimensional image data of each target object. The size information consists of three values of the height, width, and depth of the object. This size information is used when generating a cuboid virtual three-dimensional model (polygon data), which will be described later.

  The image selection unit 2 is stored in the two-dimensional image storage unit 1 on the basis of information representing the direction of the virtual camera (viewpoint of viewing the object) input by the user operating an operation unit such as a mouse (not shown). Among the one or more two-dimensional image data, the two-dimensional image data matching the direction of the virtual camera is selected and read. When three types of vegetables are displayed as shown in FIG. 2, the image selection unit 2 selects and reads out two-dimensional image data that matches the direction of the virtual camera for each of the three types of vegetables. Details of this selection method will be described later.

  The image generation unit 3 is visible from the virtual camera in the object space based on the two-dimensional image data of the vegetable selected by the image selection unit 2 and the three-dimensional image data of the display table generated by the image generation unit 3 itself. An image, that is, a 3D expression image in which a plurality of types of vegetables are placed on a display table as shown in FIG. 1 is generated. Here, the 3D model processing unit 13 generates the 3D image data of the display table. Since the 3D image data is generated using a normal 3D display algorithm (geometry processing or the like), an image viewed from any direction of 360 degrees can be generated.

  The virtual model placement unit 11 places a cuboid virtual three-dimensional model in the object space. That is, in order to arrange 2D image data on the 3D object space, it is necessary to consider the concept of “depth” that is not found in 2D image data. In order to complement this concept, before placing the two-dimensional image data on the object space, a virtual three-dimensional model (polygon data) of a rectangular parallelepiped is arranged at a place to be arranged. The size of one rectangular parallelepiped is determined using the size information of the object stored in the two-dimensional image storage unit 1.

  FIG. 5 is a diagram illustrating an example of a state in which a plurality of rectangular parallelepipeds (virtual three-dimensional models) are arranged on the object space. The placement process of the virtual three-dimensional model is also performed using a normal 3D display algorithm (geometry processing or the like). The arranged rectangular parallelepiped is not displayed in the end, but only used for calculation.

  The image placement unit 12 displays an image plane of the two-dimensional image data selected by the image selection unit 2 in a rectangular parallelepiped composed of one or more virtual three-dimensional models arranged by the virtual model placement unit 11 as shown in FIG. Deploy. At that time, the image placement unit 12 places the image plane of the two-dimensional image data selected by the image selection unit 2 at a position where the center point of the image plane passes through the center point of the rectangular parallelepiped. The image placement unit 12 places the image plane of the two-dimensional image data by adjusting the placement angle in the rectangular parallelepiped so that the image plane is perpendicular to the direction of the virtual camera.

  The image output unit 14 is a 3D representation in which the 3D image data of the display table generated by the 3D model processing unit 13 and the 2D image data of the vegetables arranged by the image arrangement unit 12 are synthesized on the object space. Output an image.

  FIG. 6 is a diagram illustrating an example of a state in which an image plane of two-dimensional image data is arranged inside a rectangular parallelepiped. In FIG. 6, 21 indicates a rectangular parallelepiped of the three-dimensional model, and 22 indicates an image plane of the two-dimensional image data. The point P is the center point of the rectangular parallelepiped 21, and the image plane 22 of the two-dimensional image data is arranged at a position where the image plane 22 always passes through this point P. In the present embodiment, the cuboid 21 is arranged so that the center point P of the rectangular parallelepiped 21 is the center of the image plane 22. Similarly, when the angle of the image plane 22 is changed in the rectangular parallelepiped 21, the center of the image plane 22 passes through the center point P of the rectangular parallelepiped 21. The angle of the image plane 22 is always perpendicular to the direction of the virtual camera. That is, the image plane 22 is always viewed from the front.

  Here, a method of calculating the position (coordinates) of the image plane 22 arranged in the rectangular parallelepiped 21 will be described with reference to FIG. As shown in FIG. 7, the width and height of the image plane 22 created by the two-dimensional image data are w and h, respectively. Further, the center coordinates of the image plane 22 are assumed to be (a, b, c). The center coordinates are the same as the coordinates of the center point P of the rectangular parallelepiped shown in FIG. 6, and can be easily obtained using a normal 3D display algorithm or the like.

The angle between the reference direction and the image plane 22 (the direction of the virtual camera) is as follows.
Angle on the XY plane (rotation angle when the Z axis is fixed): 0 °
Angle on YZ plane (rotation angle when X axis is fixed): 0 °
Angle on the Z-X plane (rotation angle when the Y axis is fixed): 90 °

In this case, the coordinates of the four corner points A, B, C, D of the image plane 22 created by the two-dimensional image data when the direction of the virtual camera is in the reference direction are as follows.
Point A: (aw−2, b + h / 2, c)
Point B: (a + w / 2, b + h / 2, c)
Point C: (aw / 2, bh / 2, c)
Point D: (a + w / 2, b−h / 2, c)

Hereinafter, when the direction of the virtual camera changes from the reference direction, points A ′, B ′, and C ′ at the four corners of the image plane whose arrangement angles are adjusted to be perpendicular to the direction of the virtual camera after movement. , D ′ coordinates are obtained. When the direction of the virtual camera after movement is expressed as follows, a rotation matrix such as the following (Expression 1) is obtained from a general three-dimensional rotation matrix expression.
Angle on XY plane: xy °
Angle on YZ plane: yz °
Angle on the Z-X plane: zx °

  From the rotation matrix shown in (Equation 1), the coordinates of the four corner points A ′, B ′, C ′, D ′ of the image plane perpendicular to the direction of the virtual camera after movement are as follows (Equation 2): It can ask for. In (Expression 2), (x, y, z) is the coordinates of the points A, B, C, D, and (x ′, y ′, z ′) is the points A ′, B ′, C ′, D ′. Indicates the coordinates.

Next, a method for selecting two-dimensional image data by the image selection unit 2 described above will be described in detail. The image selection unit 2 switches the two-dimensional image data to be selected according to the angle at which the cuboid of the three-dimensional model is viewed, that is, the direction of the virtual camera. As shown in FIG. 8A, the direction of the virtual camera is expressed by two angles, an angle θ _{1 of the} XY coordinate axis and an angle θ _{2 of the} XZ coordinate axis.

The table shown in FIG. 8B shows how the two-dimensional image data is selected according to the two angles θ ₁ and θ ₂ . For example, when the virtual camera is within a range in which the two angles θ ₁ and θ ₂ take values of θ ₁ = 60 ° to 90 ° and θ ₂ = 330 ° to 30 °, the image selection unit 2 selects the object. 2D image data viewed from above (directly above) is selected. In addition, when the virtual camera is within the range of θ ₁ = 300 ° to 30 ° and θ ₂ = 330 ° to 30 °, the image selection unit 2 displays the two-dimensional image data when the object is viewed from the front. select.

There is no problem when there are all 40 pieces of two-dimensional image data shown in the table of FIG. 8B, but when there is not enough, one image closer to the “front” side is used. In other words, if there is no two-dimensional image data on the left side, the image cannot be displayed, so the two-dimensional one cell (θ ₁ = 300 ° to 30 °, θ ₂ = 300 ° to 330 °). Select image data. If not, the front two-dimensional image data is selected.

In the case of a pineapple in which three pieces of two-dimensional image data are prepared in front, right above, and obliquely, the angle θ _{2 of the} XZ coordinate axis is an irrelevant parameter when selecting two-dimensional image data. In this case, the image selection unit 2 selects any two-dimensional image data from the front, directly above, or diagonally based on only the angle θ _{1 of the} XY coordinate axes.

For example, when the virtual camera is within a range in which the angle θ _{1 of the} XY coordinate axes takes a value of θ ₁ = 60 ° to 90 °, the image selection unit 2 displays a two-dimensional image when the object is viewed from directly above. Select data. The image placement unit 12 places the selected 2D image data in the rectangular parallelepiped of the 3D model placed by the virtual model placement unit 11.

In this state, it is assumed that the user operates an operation unit (not shown) to move the direction of the virtual camera downward. At this time, as the virtual camera moves in the direction, the display table and the three-dimensional model of the rectangular parallelepiped are subjected to viewpoint conversion by geometry processing or the like. On the other hand, while the angle θ _{1 of the} XY coordinate axis is in the range of θ ₁ = 60 ° to 90 °, the two-dimensional image data directly above is continuously selected. Meanwhile, as the direction of the virtual camera moves, the arrangement angle of the image plane gradually changes in the rectangular parallelepiped so that the image plane of the two-dimensional image data is always perpendicular to the moved direction. Thereby, even if the direction of the virtual camera is moved, the image plane of the two-dimensional image data being selected is always visible from the front. When the angle θ ₁ moves to a range of θ ₁ = 30 ° to 60 ° due to the movement of the virtual camera, the image selection unit 2 switches so as to select the two-dimensional image data on the upper side.

  Next, the operation of the image generation apparatus according to the present embodiment configured as described above, that is, the image generation method according to the present embodiment will be described. FIG. 9 is a flowchart illustrating an operation example of the image generation apparatus according to the present embodiment.

  In FIG. 9, first, the 3D model processing unit 13 generates 3D image data of a display table (step S1). Next, the virtual model placement unit 11 places one or more cuboid virtual three-dimensional models so as to be arranged on the display table in the object space generated in step S1 (step S2). Here, in order to generate a 3D representation image as shown in FIG. 2, three types of rectangular parallelepipeds are set, and a plurality of them are arranged side by side at appropriate positions on the display table.

  And the image arrangement | positioning part 12 selects the two-dimensional image data of the vegetables suitable for the direction of a virtual camera based on the information showing the direction of a virtual camera, and reads it from the two-dimensional image storage part 1 (step S3). Here, two-dimensional image data of three kinds of vegetables necessary for arranging in three kinds of rectangular parallelepipeds are selected. Further, the image placement unit 12 places the image plane of the two-dimensional image data selected in step S3 in the rectangular parallelepiped placed in step S2 at a position where the angle is adjusted according to the direction of the virtual camera. Then, a 3D representation image as shown in FIG. 2 is generated (step S4).

  The image generation method according to the present embodiment described above can be realized by any of a hardware configuration, a DSP, and software. For example, when realized by software, the image generation apparatus according to the present embodiment is actually configured by including a CPU or MPU of a computer, RAM, ROM, and the like, and can be realized by operating a program stored in the RAM or ROM. .

  Therefore, it can be realized by recording a program that causes a computer to perform the functions of the above-described embodiments on a recording medium such as a CD-ROM and causing the computer to read the program. As a recording medium for recording the program, a flexible disk, a hard disk, a magnetic tape, an optical disk, a magneto-optical disk, a DVD, a nonvolatile memory card, and the like can be used in addition to the CD-ROM. It can also be realized by downloading the program to a computer via a network such as the Internet.

  As described above in detail, in the present embodiment, a virtual three-dimensional model of a rectangular parallelepiped is arranged in the object space, and two-dimensional image data matching the direction of the virtual camera is selected from the two-dimensional image storage unit 1. Read out automatically. Then, by arranging the image plane of the selected two-dimensional image data in a rectangular parallelepiped composed of one or more virtual three-dimensional models arranged as described above, a 3D representation image that can be seen from the virtual camera in the object space is generated. Like to do.

  In the image generating apparatus of the present embodiment configured as described above, the 3D model used to perform 3D representation of an image is simple rectangular parallelepiped polygon data, and since the shape is not complicated, the amount of information is not so much. The calculation load is very light. Since 3D representation is realized simply by placing the image plane of the 2D image data in the rectangular parallelepiped of the 3D model, the virtual camera efficiently generates the 3D model image viewed from various directions. can do.

  Further, according to the present embodiment, the overlapping degree of the image planes arranged in one or more rectangular parallelepipeds follows the overlapping degree of the three-dimensional model of the rectangular parallelepiped. That is, when two rectangular parallelepipeds overlap, the two-dimensional image planes in the same overlap, and it is determined which two-dimensional image comes to the front. As a result, when a three-dimensional object is stacked or arranged in the object space, naturally the overlapping portion is appropriately represented by the two-dimensional image data.

  From the above, 3D expression can be easily performed based on two-dimensional image data even for an image in which a plurality of three-dimensional objects exist within one screen.

  In the above-described embodiment, an example of generating a shelf image in which a plurality of types of products are arranged and arranged on the display table as shown in FIG. 2 has been described, but the present invention is not limited to this.

  In the above-described embodiment, an example in which a normal 3D image is generated based on polygon data for the display table has been described. However, a 2D image may also be generated from 2D image data.

  In the above-described embodiment, the maximum number of two-dimensional image data prepared as images viewed from various viewpoints for one object is 40, but the number may be larger than this. If the number of two-dimensional image data to be prepared is increased, an object viewed from various angles can be expressed more realistically. However, in this case, since the amount of data stored in the two-dimensional image storage unit 1 increases, it is not preferable to increase the number of two-dimensional image data too much.

  In addition, each of the above-described embodiments is merely an example of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. In other words, the present invention can be implemented in various forms without departing from the spirit or main features thereof.

It is a block diagram which shows the function structural example of the image generation apparatus by this embodiment. It is a figure which shows the example of the 3D expression image produced | generated with the image production | generation apparatus of this embodiment. It is a figure which shows the example of a viewpoint in the case of preparing 40 two-dimensional image data. It is a figure which shows the example of the two-dimensional image data of each viewpoint. It is a figure which shows the example of the state which has arrange | positioned several rectangular parallelepiped (virtual three-dimensional model) on object space. It is a figure which shows the example of the state which has arrange | positioned the image plane of two-dimensional image data inside a rectangular parallelepiped. It is a figure for demonstrating the calculation method of the position (coordinate) of the image plane arrange | positioned in a rectangular parallelepiped. It is a figure for demonstrating the selection method of two-dimensional image data. It is a flowchart which shows the operation example of the image generation apparatus by this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 2D image memory | storage part 2 Image selection part 3 Image generation part 11 Virtual model arrangement | positioning part 12 Image arrangement | positioning part 13 3D model processing part 14 Image output part

Claims (6)

A two-dimensional image storage unit that stores one or more two-dimensional image data obtained by viewing a single three-dimensional object from different viewpoints;
Based on the information indicating the direction of the virtual camera, two-dimensional image data matching the direction of the virtual camera is selected and read from one or more two-dimensional image data stored in the two-dimensional image storage unit. An image selector;
An image generation unit that generates an image visible from the virtual camera in the object space based on the two-dimensional image data selected by the image selection unit;
The image generation unit includes a virtual model arrangement unit that arranges a virtual three-dimensional model of a rectangular parallelepiped in the object space;
An image placement unit for placing an image plane of the two-dimensional image data selected by the image selection unit at a position set in a rectangular parallelepiped composed of one or more virtual three-dimensional models placed by the virtual model placement unit; An image generation apparatus comprising: The image arranging unit arranges the image plane of the two-dimensional image data selected by the image selecting unit at a position where the center point of the image plane passes through the center point of the rectangular parallelepiped. The image generating apparatus according to 1. 3. The image arranging unit arranges the image plane by adjusting an arrangement angle in the rectangular parallelepiped so that the image plane is perpendicular to a direction of the virtual camera. The image generating apparatus described in 1. A first step of arranging a virtual three-dimensional model of a rectangular parallelepiped in the object space;
Based on the information indicating the direction of the virtual camera, it matches the direction of the virtual camera from the two-dimensional image storage unit storing one or more two-dimensional image data obtained by viewing a single three-dimensional object from different viewpoints. A second step of selecting and reading out two-dimensional image data;
By arranging the image plane of the two-dimensional image data selected in the second step at the position set in the rectangular parallelepiped composed of one or more virtual three-dimensional models arranged in the first step, And a third step of generating an image visible from the virtual camera in the object space. The image generation program for functioning a computer as each means of any one of Claims 1-3. An image generation program for causing a computer to execute the processing procedure of the image generation method according to claim 4.