JP2013061898A - Program, information storage medium, image generation device, and server - Google Patents

Abstract

An object of the present invention is to provide a program, an information storage medium, an image generation device, and a server capable of generating a stereoscopic image with a small processing load and less discomfort.
When performing processing for generating a stereoscopic image based on a first image and a second image, a first image generation unit applies the first image to each pixel of a specific buffer corresponding to a screen. Process for storing the information of the foremost primitive to the virtual camera of the second camera, and the second image generation unit can divert from the first image for each pixel of the second image before pixel processing A process for determining whether or not image information of a pixel exists is performed. When there is image information of a divertable pixel from the first image, instead of the pixel processing, the pixel information of the first image A process of diverting the image information to the image information of the pixels of the second image is performed.
[Selection] Figure 14

Description

  The present invention relates to a program, an information storage medium, an image generation device, and a server.

  2. Description of the Related Art Conventionally, an image generation apparatus that realizes stereoscopic viewing is known (for example, Patent Document 1). For example, in a twin-lens image generation system, a left-eye image and a right-eye image are generated. Then, using stereoscopic glasses or an optical element such as a parallax barrier or lenticular, the left eye of the observer can only see the image for the left eye, and the right eye can see only the image for the right eye. In this way, stereoscopic viewing is realized. In addition, the N-eye image generation apparatus generates a stereoscopic image from images at N different viewpoints.

  For example, the prior art describes that shading calculation is executed only once for N different views, and processing is performed such that the shaded color is reused for all views (see Reference 1). 0019 paragraph).

JP 2007-514230 A

  However, in the conventional technique as shown in Patent Document 1, it is unclear how and which pixel color is reused, so that an image generated based on N viewpoints (views) is inaccurate. There was a sense of incongruity in the generated stereoscopic image.

  In general, in the image generation process, pixel processing has a very high processing load. Therefore, when performing pixel processing for obtaining image information of all pixels in each image based on N viewpoints, the processing load is high as a whole. turn into. Furthermore, even if an image is generated accurately over time, in the highlight part with many reflected light components, the position and shape of the highlight differ between the left and right images, so that the three-dimensional feeling of that part is uncomfortable. Sometimes. This may occur as a natural phenomenon, but if these can be reduced at the time of video production, it is possible to create an easy-to-see stereoscopic video without a sense of incongruity.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a program, an information storage medium, an image generation apparatus, and a server that can generate a stereoscopic image with a small processing load and a little uncomfortable feeling. It is to provide.

  (1) The present invention is a program for performing a process of arranging a primitive in a virtual three-dimensional space and generating a stereoscopic image, and perspectively projecting the primitive on a screen based on a first virtual camera in the virtual three-dimensional space A first image generation unit that performs processing for generating a first image by performing vertex processing to be converted and pixel processing for obtaining image information of each pixel, and a second virtual in the virtual three-dimensional space A second image generation unit that performs processing for generating a second image by performing vertex processing for perspective projection conversion of a primitive to a screen based on a camera and pixel processing for obtaining image information of each pixel; Based on the first image and the second image, the computer functions as a stereoscopic image generation unit that performs a process of generating a stereoscopic image, and the first image generation unit includes: Each pixel of the specific buffer corresponding to the clean performs processing for storing the information of the foremost primitive for the first virtual camera, and the second image generation unit performs the processing before the pixel processing. Whether or not there is image information of a divertable pixel from the first image by performing processing for comparing the primitive information and the primitive information of the pixel of the specific buffer for each pixel of the second image. In the case where there is image information of a pixel that can be diverted from the first image, the image information of the pixel of the first image is changed to the pixel information of the second image instead of the pixel processing. The present invention relates to a program for performing processing to be used for image information. The present invention also relates to an information storage medium storing the above program.

  Here, the specific buffer is a buffer capable of storing attribute information regarding an image in units of pixels, for example, a stencil buffer. Further, the number of pixels that can be diverted from the first image may be one or plural. The diverting process is, for example, a process of storing image information of a pixel of the first image in a frame buffer or the like as image information of a pixel of the second image, and sampling a plurality of the pixels from the first image ( Extraction) and combining the image information (averaged, weighted average, etc.) into a frame buffer or the like as image information of the pixels of the second image.

  The “primitive information” includes at least one of primitive identification information, primitive three-dimensional space position coordinates, and primitive depth information.

  According to the present invention, before pixel processing, a process is performed to determine whether there is divertable pixel image information from the first image, and divertable pixel image information exists from the first image. In this case, instead of pixel processing, the image information of the pixel of the first image is used for the image information of the pixel of the second image. Therefore, the pixel processing of the pixel in the second image generation processing is performed. Can be omitted, and the processing load can be reduced. In addition, according to the present invention, it is possible to generate a stereoscopic image with less sense of incongruity in, for example, a highlight area by using the image information of the pixel of the first image. In addition, the present invention appropriately determines whether or not the pixel is to be diverted by performing processing for comparing the primitive information and the primitive information of the pixel of the specific buffer for each pixel of the second image. be able to.

  (2) In the program and the information storage medium of the present invention, the first image generation unit has, in each pixel of the specific buffer corresponding to the screen, identification information of a primitive that is in the forefront with respect to the first virtual camera And the second image generation unit compares the primitive identification information with the primitive identification information of the pixels of the specific buffer for each pixel of the second image before the pixel processing. By performing the processing, it may be possible to perform processing for determining whether there is image information of divertable pixels from the first image.

  In the present invention, for each pixel of the second image, a process of comparing the primitive identification information with the primitive identification information of the pixel of the specific buffer is performed, thereby appropriately determining whether or not the pixel is to be diverted. be able to.

  (3) In the program and the information storage medium of the present invention, the first image generation unit is configured such that the virtual image of the primitive at the forefront with respect to the first virtual camera is displayed on each pixel of the specific buffer corresponding to the screen. A process of storing the position coordinates of the three-dimensional space is performed, and the second image generation unit performs the primitive three-dimensional position coordinates of the primitive and the specific buffer for each pixel of the second image before the pixel processing. A process of comparing whether or not there is image information of a divertable pixel from the first image may be performed by performing a process of comparing the position coordinates of the pixel in the virtual three-dimensional space.

  The present invention appropriately determines whether or not a pixel is to be diverted by performing a process of comparing the position coordinates of a primitive virtual three-dimensional space with the position coordinates of a specific buffer pixel in the virtual three-dimensional space. Can do.

  (4) According to the program and the information storage medium of the present invention, the first image generation unit has the depth information of the primitive located at the forefront with respect to the first virtual camera in each pixel of the specific buffer corresponding to the screen. And the second image generation unit, for each pixel of the second image, the position coordinates of the primitive virtual three-dimensional space, the first virtual camera, and the Pixels that can be diverted from the first image by performing a process of comparing the virtual camera information of the second virtual camera and the position coordinates of the virtual three-dimensional space calculated based on the depth information of the pixels of the specific buffer Processing for determining whether or not the image information exists may be performed.

  The virtual camera information of the first virtual camera and the second virtual camera is virtual camera information (for example, a reference virtual camera (center camera) serving as a reference for controlling the first virtual camera and the second virtual camera). , The viewpoint position, direction, and angle of view of the reference virtual camera) and information (for example, the interval between the left and right cameras) that can determine the relative position of each camera from there, or the virtual camera of the first virtual camera Information (for example, the viewpoint position, direction, and angle of view of the first virtual camera) and virtual camera information of the second virtual camera (for example, viewpoint position, direction, and angle of view of the second virtual camera) may be used. .

  The present invention provides a virtual three-dimensional space calculated based on a position coordinate of a primitive virtual three-dimensional space, virtual camera information of the first virtual camera and the second virtual camera, and pixel depth information of the specific buffer. It is possible to accurately determine whether or not the pixel is to be diverted by performing a process of comparing the position coordinates.

  In addition, since the present invention performs processing for storing the depth information of the foremost primitive with respect to the first virtual camera in each pixel of the specific buffer, the position coordinates of the primitive in the virtual three-dimensional space are stored. However, the storage capacity is small, and the storage capacity can be saved.

  (5) The program and the information storage medium according to the present invention are such that the second image generation unit can divert pixels from the first image when the primitive information and the primitive information of the pixels of the specific buffer are the same. The image information of the pixel of the first image corresponding to the pixel of the specific buffer is used as the image information of the pixel of the second image. May be.

  According to the present invention, when the primitive information and the primitive information of the pixel of the specific buffer are the same, it is determined that there is image information of a divertable pixel from the first image. It is possible to determine whether there is image information of divertable pixels from one image.

  (6) In the program and the information storage medium of the present invention, the second image generation unit includes the first virtual camera and the virtual camera information of the second virtual camera and the second virtual camera for each pixel of the second image. The search area of the specific buffer may be determined based on the depth information of the pixels of the second image, and processing for comparing the primitive information and the primitive information of the pixels in the search area may be performed.

  According to the present invention, since the pixels in the search area are to be compared, the number of comparison objects can be reduced, and the processing load can be reduced. Further, according to the present invention, the search area of the specific buffer is determined based on the virtual camera information of the first virtual camera and the second virtual camera and the depth information of the pixel of the second image. Can be determined.

  (7) The program and the information storage medium of the present invention may be configured such that the stereoscopic image generation unit generates the stereoscopic image based on the first to Nth (N ≧ 3) virtual cameras. The image generation unit performs a process of generating a first image by performing the vertex process and the pixel process based on the first virtual camera, and each pixel of the specific buffer is subjected to the first process. A process of storing information on the foremost primitive for the virtual camera is performed, and the second image generation unit performs the vertex process and the pixel process for each of the second to Nth virtual cameras. The second image generation unit performs processing for generating each of the second to Nth images, and the second image generation unit, for each pixel of the second to Nth images, uses a pixel that can be diverted from the first image. Perform processing to determine whether image information exists, If there is image information of a pixel that can be diverted from the image of the image, instead of the pixel processing, a process of diverting the image information of the pixel of the first image to the image information of the pixel of the image is performed. Also good.

  According to the present invention, in each of the second to Nth images, a process for determining whether there is image information of a divertable pixel from the first image for each pixel is performed. When there is image information of a divertable pixel, instead of the pixel processing, the image information of the pixel of the first image is processed as image information of the pixel of the image, so the pixel of the diverted pixel Processing can be omitted and the processing load can be reduced. In addition, by diverting the image information of the pixel of the first image, it is possible to generate a stereoscopic image with little discomfort.

  (8) In the program and the information storage medium of the present invention, the first image generation unit performs processing for processing the first image, and the second image generation unit processes the second image. The stereoscopic image generation unit may perform a process of generating a stereoscopic image based on the processed first image and the processed second image.

  Here, the processing process is, for example, an effect process. According to the present invention, when different processing is performed between the first and second images, a stereoscopic image with less discomfort can be obtained by diverting the image information of the pixels of the first image before performing the processing. Can be generated.

  (9) In the program and information storage medium of the present invention, the image information may be at least one of color information, normal line information, depth information, and translucent information.

  (10) The present invention is an image generation apparatus that performs processing for generating a stereoscopic image by arranging a primitive in a virtual three-dimensional space, and the primitive is displayed on a screen based on a first virtual camera in the virtual three-dimensional space. A first image generation unit that performs processing for generating a first image by performing vertex processing for perspective projection conversion and pixel processing for obtaining image information of each pixel; and second processing in the virtual three-dimensional space. A second image generation unit that performs a process of generating a second image by performing a vertex process for perspective projection conversion of a primitive to a screen based on the virtual camera and a pixel process for obtaining image information of each pixel; A stereoscopic image generation unit that performs a process of generating a stereoscopic image based on the first image and the second image, wherein the first image generation unit corresponds to a screen Each pixel of the specific buffer is configured to store information on the foremost primitive for the first virtual camera, and the second image generation unit performs the second image before the pixel processing. A process for determining whether there is image information of divertable pixels from the first image by performing a process of comparing primitive information and primitive information of pixels of the specific buffer for each pixel When there is image information of a divertable pixel from the first image, the image information of the pixel of the first image is changed to the image information of the pixel of the second image instead of the pixel processing. The present invention relates to an image generation apparatus that performs processing to be diverted.

  (11) The present invention is a server that performs processing for arranging a primitive in a virtual three-dimensional space and generating a stereoscopic image, and perspectively projects the primitive on a screen based on a first virtual camera in the virtual three-dimensional space. A first image generation unit that performs processing for generating a first image by performing vertex processing to be converted and pixel processing for obtaining image information of each pixel, and a second virtual in the virtual three-dimensional space A second image generation unit that performs processing for generating a second image by performing vertex processing for perspective projection conversion of a primitive to a screen based on a camera and pixel processing for obtaining image information of each pixel; A stereoscopic image generation unit that performs a process of generating a stereoscopic image based on the first image and the second image, wherein the first image generation unit corresponds to a screen. Each pixel of the buffer performs processing for storing the information of the foremost primitive for the first virtual camera, and the second image generation unit performs pixel processing of the second image before the pixel processing. Each time, a process for comparing the primitive information and the primitive information of the pixel of the specific buffer is performed to determine whether there is diverted pixel image information from the first image. When there is image information of a pixel that can be diverted from the first image, the image information of the pixel of the first image is diverted to the image information of the pixel of the second image instead of the pixel processing. It relates to a server that performs processing.

The example of a functional block diagram of the image generation device of this embodiment. Explanatory drawing of the stereoscopic vision image generation process of this embodiment. 3A and 3B are explanatory diagrams of stereoscopic images according to the present embodiment. FIG. 3 is a flowchart diagram of an exemplary rendering pipeline. Explanatory drawing of a primitive. Explanatory drawing of the production | generation process of a 1st image (image for left eyes). FIG. 7A is an explanatory diagram of the screen 40. FIG. 7B is an explanatory diagram of the specific buffer 175. The flowchart figure of the rendering pipeline of the 1st image (image for left eyes). Explanatory drawing of the production | generation process of a 2nd image (image for right eyes). Explanatory drawing of the process regarding diversion of a pixel. Explanatory drawing of the process regarding diversion of a pixel. Explanatory drawing of the process regarding diversion of a pixel. Explanatory drawing of the process regarding diversion of a pixel. The flowchart figure of the rendering pipeline of the 2nd image (image for right eyes). Explanatory drawing of the stereoscopic vision image based on three or more virtual cameras. The example of the functional block diagram of the server of this embodiment.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. Configuration FIG. 1 shows an example of a functional block diagram of an image generation apparatus 10 (computer, image generation system, terminal, mobile terminal, game device, mobile phone, smartphone) of the present embodiment. Note that the image generation apparatus 10 of the present embodiment may have a configuration in which some of the components (each unit) in FIG. 1 are omitted.

  The operation unit 160 (input unit) is used by an observer (player) to input operation information (input information), and outputs user operation information to the processing unit 100. The function of the operation unit 160 includes a button, a direction key (cross key), a control stick (analog key) capable of inputting a direction, a lever, a keyboard, a mouse, a touch panel display, an acceleration sensor, a tilt sensor, a controller with a built-in camera, etc. It can be realized by hardware.

  The storage unit 170 serves as a work area for the processing unit 100, the communication unit 196, and the like, and its function can be realized by a RAM (VRAM) or the like. The storage unit 170 of the present embodiment stores a main storage unit 172 used as a work area, a frame buffer 174 for storing images, a specific buffer 175 (for example, a stencil buffer), and object model data. An object data storage unit 176, a texture storage unit 178 for storing textures for each object data, and a Z buffer 179 for storing Z values at the time of object image generation processing. Note that some of these may be omitted.

  The frame buffer 174 includes a first frame buffer 174a (left eye frame buffer) and a second frame buffer 174b (right eye frame buffer). The texture storage unit 178 also includes a first texture buffer 178a (left eye texture buffer) and a second texture buffer 178b (right eye texture buffer). The Z buffer 179 includes a first Z buffer 179a (left eye Z buffer) and a second Z buffer 179b (right eye Z buffer).

  The information storage medium 180 (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (CD, DVD), magneto-optical disk (MO), magnetic disk, hard disk, and magnetic tape. Alternatively, it can be realized by a memory (ROM). The processing unit 100 performs various processes of the present embodiment based on a program (data) stored in the information storage medium 180. The information storage medium 180 can store a program for causing a computer to function as each unit of the processing unit 100 according to the present embodiment (a program for causing a computer to execute the processing of each unit).

  The display unit 190 outputs an image (an image obtained by viewing the virtual three-dimensional space from a given viewpoint) generated according to the present embodiment, and the function is realized by a CRT, LCD, touch panel display, or the like. it can.

  The display unit 190 of the present embodiment displays a stereoscopic image. For example, in the naked eye method, the display unit 190 includes optical elements such as a parallax barrier and a lenticular on the front or back of an LCD composed of a finite number of pixels. For example, in the case of a two-lens system using a vertical barrier or a vertical lenticular, a display area for displaying one (one frame) first image (left-eye image) on the display unit 190 and one sheet The display area for displaying the right-eye image is a strip-shaped first strip image display area (left-eye strip image display area) and second strip image display area (right-eye strip image display area). Are arranged alternately. When a parallax barrier is placed, the left eye strip image displayed in the left eye strip image display area is displayed on the left eye of the observer, and the right eye strip image display area is displayed on the right eye. Only the displayed right eye strip image is visible. When the lenticular is arranged, the left eye strip image displayed in the left eye strip image display area is displayed on the left eye of the observer and the right eye due to the photorefractive effect of the lens corresponding to each strip aggregate image. Only the strip image for the right eye displayed in the strip image display area for the right eye is visible. Further, in the polarizing glasses method, polarizing filters having different polarization directions (for example, a polarizing filter that transmits left circularly polarized light and a polarizing filter that transmits right circularly polarized light) are alternately arranged on the odd and even lines of the display unit 190. The first image (the left eye image) and the second image (the right eye image) are alternately displayed on the odd-numbered line and the even-numbered line of the display unit 190, and this is displayed with polarized glasses (for example, the left circle on the left eye). Stereoscopic viewing is realized by viewing with a polarizing filter that transmits polarized light and glasses with a polarizing filter that transmits right circularly polarized light in the right eye. In the page flip method, the display unit 190 alternately displays the first image (left-eye image) and the second image (right-eye image) every predetermined time (for example, every 1/120 second). . In conjunction with the switching of the display, the left and right liquid crystal shutters of the glasses with the liquid crystal shutter are alternately opened and closed to realize stereoscopic viewing.

  The communication unit 196 performs various controls for communicating with other image generation apparatuses 10 and servers, and the functions can be realized by hardware such as various processors or communication ASICs, programs, and the like. .

  In addition, the program and data for functioning a computer as each part of this embodiment memorize | stored in the information storage medium 280 and the memory | storage part 270 which the server 20 has are received via a network, and the received program and data are information. You may memorize | store in the storage medium 180 or the memory | storage part 170. FIG. The case where the game system is functioned by receiving the program or data as described above is also included in the scope of the present invention.

  The processing unit 100 (processor) performs processing such as game processing, image generation processing, and sound generation processing based on operation data from the operation unit 160, data and programs received via the communication unit 196, and the like. Here, as the game process, a process for starting a game when a game start condition is satisfied, a process for advancing the game, a process for calculating a game result, or a game is ended when a game end condition is satisfied. There is processing. The processing unit 100 performs various processes using the main storage unit 172 as a work area. The functions of the processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.), ASIC (gate array, etc.), and programs.

  The processing unit 100 includes an object space setting unit 110, a movement / motion processing unit 112, a virtual camera control unit 118, a communication control unit 119, an image generation unit 120, and a sound generation unit 130.

  The object space setting unit 110 includes various objects (polygons, free-form surfaces, subdivision surfaces, etc.) representing display objects such as objects (characters, moving objects), buildings, stadiums, cars, trees, pillars, walls, maps (terrain). A process of placing and setting an object composed of primitives) in a virtual three-dimensional space (object space, world coordinate system) is performed. That is, a process for arranging the primitive in the virtual three-dimensional space is performed. For example, the position and rotation angle (synonymous with the direction and direction of the object in the world coordinate system, for example, rotation when rotating clockwise when viewed from the positive direction of each axis of the X, Y, and Z axes in the world coordinate system. The angle is determined, and the object is arranged at the position (X, Y, Z) at the rotation angle (rotation angle about the X, Y, Z axis).

  The movement / motion processing unit 112 performs movement / motion calculation (movement / motion simulation) of an object (moving body, character, etc.). That is, based on operation data input by the observer (player) through the operation unit 160, a program (movement / motion algorithm), various data (motion data), physical laws, and the like, an object is represented in a virtual three-dimensional space (object space, It moves within the world coordinate system) and moves the object (motion, animation). Specifically, object movement information (position, rotation angle, speed, or acceleration) and motion information (position or rotation angle of each part constituting the object) are stored in one frame (for example, 1/60 second or 1 / 30 seconds, etc.) The simulation process is sequentially obtained every time. A frame is a unit of time for performing object movement / motion processing (simulation processing) and image generation processing.

  In addition, the movement / motion processing unit 112 is based on movement information and motion information of an object (an object corresponding to another player) received from another image generation apparatus 10 connected via a network via the communication unit 196. You may make it perform the process which updates the movement information and action information of an object.

  In addition, the movement / motion processing unit 112 may perform the movement / motion calculation of the object based on operation data received from another image generation apparatus via the communication unit 196.

  The virtual camera control unit 118 performs a virtual camera (viewpoint) control process for generating an image that can be seen from a given (arbitrary) viewpoint in the virtual three-dimensional space. Specifically, based on operation data or a program (movement / motion algorithm) input by the player through the operation unit 160, the position (X, Y, Z) of the virtual camera in the world coordinate system or the direction of the optical axis (for example, , X, Y, and Z axes, a rotation angle when rotating clockwise as viewed from the negative direction is performed. In short, processing for controlling the viewpoint position, the orientation of the virtual camera, and the angle of view is performed.

  When displaying a stereoscopic image by a two-eye stereoscopic method, the virtual camera control unit 118 includes a first virtual camera (left-eye virtual camera) VCL and a second virtual camera (for right-eye). Virtual camera) Controls the position, orientation and angle of view of each VCR. Further, the virtual camera control unit 118 controls the reference virtual camera (center camera) VCC serving as a reference for setting the first virtual camera and the second virtual camera, and the position, orientation, and The positions and orientations of the first and second virtual cameras may be controlled based on the distance information between the first and second virtual cameras.

  That is, the virtual camera control unit 118 controls virtual camera information (at least one of position, orientation, and angle of view) of each of the first and second virtual cameras VCL and VCR. Further, the virtual camera control unit 118 controls (at least one of position, orientation, and angle of view) of the reference virtual camera VCC.

  The communication control unit 119 performs processing for transmitting / receiving data to / from another image generation apparatus or the server 20 via the network.

  Note that the image generation apparatus 10 according to the present embodiment may perform processing for acquiring and managing network information necessary for communication control from a server. For example, the image generation device 10 has the identification information of the image generation device 10 that is individually assigned to each image generation device 10 (data and ID individually assigned to identify the image generation device 10 that can participate in the online game). And destination information (such as an IP address) that specifies a transmission destination of a packet associated with the identification information of the image generation apparatus 10 may be acquired and managed.

  The communication control unit 119 performs processing for generating a packet to be transmitted to another image generation device or the server 20, processing for specifying the IP address and port number of the image generation device 10 that is a packet transmission destination, and data included in the received packet. Processing to be stored in the storage unit 170, processing to analyze received packets, control processing related to transmission / reception of other packets, and the like are performed.

  In addition, the communication control unit 119 according to the present embodiment exchanges data with each other in a predetermined cycle (for example, in a cycle of 1 second) until the connection is disconnected after the connection in the other image generation apparatus 10 or the server 20 is established. You may make it perform the process which transmits / receives. Here, the data transmitted / received by the image generation device 10 may be operation information (input information) of the operation unit (input unit), or a position of an object (machine or moving body) to be operated by each image generation device 10. It is good also as information and movement information.

  In addition, when the communication control unit 119 of the present embodiment receives a packet transmitted from another image generation device (second image generation device) or the server 20, the communication control unit 119 analyzes the received packet, and receives other packets included in the packet. A process of storing data such as position information of an operation target object of the image generation apparatus in a storage unit is performed.

  In the case of a network system composed of a plurality of image generation apparatuses 10, a peer-to-peer (so-called P2P) system that executes an online game while transmitting and receiving data between the plurality of image generation apparatuses 10 may be adopted. In addition, the client / server system may be used in which each image generation apparatus 10 executes an online game while transmitting and receiving data (information) via the server 20. In the network system of the present embodiment, data may be transmitted / received not only by wired communication but also by wireless communication.

  For example, the communication control unit 119 of this embodiment may transmit operation information to the server 20 and receive image data (display data) of a stereoscopic image generated in the server 20.

  The image generation unit 120 performs drawing processing based on the results of various processes performed by the processing unit 100, thereby generating an image and outputting the image to the display unit 190.

  In the case of generating a so-called three-dimensional image, the image generation unit 120 first includes object data including vertex data (vertex position coordinates, texture coordinates, color data, normal vector, α value, etc.) of each vertex of the object. Model data) is input, and vertex processing (shading by a vertex shader) is performed based on the vertex data included in the input object data. When performing the vertex processing, vertex generation processing (tessellation, curved surface division, polygon division) for re-dividing the polygon may be performed as necessary.

  In the vertex processing, according to the vertex processing program (vertex shader program, first shader program), vertex movement processing, coordinate transformation, for example, world coordinate transformation, visual field transformation (camera coordinate transformation), clipping processing, projective transformation (perspective transformation) , Projection conversion), viewport conversion (screen coordinate conversion), light source calculation, and other geometric processing are performed, and based on the processing results, the vertex data given to the vertex group constituting the object is changed (updated, adjusted). To do. Object data after the geometry processing (position coordinates of object vertices, texture coordinates, color data (luminance data), normal vector, α value, etc.) is stored in the object data storage unit 176.

  Then, rasterization (scan conversion) is performed based on the vertex data after the vertex processing, and the surface of the primitive (polygon) is associated with the pixel. Subsequent to rasterization, pixel processing (shading or fragment processing by a pixel shader) for drawing pixels (fragments forming a display screen) constituting an image is performed.

  In pixel processing, according to a pixel processing program (pixel shader program, second shader program), various processes such as texture reading (texture mapping), color data setting / change, translucent composition, anti-aliasing, etc. are performed, and an image is processed. The final drawing color of the pixel constituting the image is determined (image information of each pixel constituting the image is obtained), and the image information such as the drawing color of the perspective-transformed object is stored in the frame buffer 174 (image information is stored in pixel units). Output (drawing) to a VRAM. That is, in pixel processing, image information (color information (R, G, B luminance information), normal information, depth information (depth information), translucent information (α value), etc.) is set or changed in units of pixels. Perform per-pixel processing. Thereby, an image that can be seen from the virtual camera (given viewpoint) in the virtual three-dimensional space (object space) is generated.

  Then, the image generation unit 120 performs texture mapping, hidden surface removal processing, α blending, and the like when drawing an object.

  Texture mapping is a process for mapping a texture (texel value) stored in the texture storage unit 178 of the storage unit 170 to an object. Specifically, the texture (surface properties such as color (RGB) and α value) is read from the texture storage unit 178 of the storage unit 170 using the texture coordinates set (given) at the vertex of the object. Then, a texture that is a two-dimensional image is mapped to an object. In this case, processing for associating pixels with texels, bilinear interpolation or the like is performed as texel interpolation.

  As the hidden surface removal processing, hidden surface removal processing by a Z buffer method (depth comparison method, Z test) using a Z buffer 179 (depth buffer) in which a Z value (depth information) of a drawing pixel is stored may be performed. it can. That is, when drawing pixels corresponding to the primitive of the object are drawn, the Z value stored in the Z buffer 179 is referred to. Then, the Z value of the referenced Z buffer 179 is compared with the Z value at the primitive drawing pixel, and the Z value at the drawing pixel is the near side as viewed from the virtual camera (for example, the referenced Z value). If the Z value is smaller than the value, the drawing process of the drawing pixel is performed and the Z value of the Z buffer 179 is updated to a new Z value.

  α blending (α synthesis) is a translucent synthesis process (usually α blending, addition α blending, subtraction α blending, or the like) based on an α value (A value). For example, in normal α blending, a process for obtaining a color obtained by combining two colors by performing linear interpolation using the α value as the intensity of synthesis (synthesis rate, synthesis rate). The α value is information that can be stored in association with each pixel (texel, dot), and is, for example, plus alpha information (translucent information) other than color information representing the luminance of each color component of RGB. The α value can be used as mask information, translucency (equivalent to transparency and opacity), bump information, and the like.

  In particular, the image generation unit 120 of the present embodiment includes a first image generation unit 124, a second image generation unit 126, and a stereoscopic image generation unit 128.

  The first image generation unit 124 (left-eye image generation unit) is a first image that can be seen from the first virtual camera (left-eye virtual camera VCL) in a virtual three-dimensional space (world coordinate system, object space). The image information (at least one of color information, normal line information, depth information, and translucent information) of each pixel of the (left-eye image) is stored in a first buffer (for example, the first frame buffer 174a, the first frame Processing to be stored in the texture buffer 178a and the first Z buffer 179a) is performed.

  In particular, the first image generation unit 124 according to the present embodiment performs vertex processing for perspective projection conversion of a primitive onto a screen based on a first virtual camera in a virtual three-dimensional space, and pixel processing for obtaining image information of each pixel. By performing, the process which produces | generates a 1st image is performed.

  In addition, the first image generation unit 124 stores information on the primitive that is in the forefront with respect to the first virtual camera (for example, primitive identification information, virtual three-dimensional primitive) for each pixel of the specific buffer corresponding to the screen. (Spatial position coordinates, primitive depth information, etc.) are stored.

  For example, the first image generation unit 124 adds, to each pixel of the specific buffer corresponding to the screen, the identification information (primitive number i) of the primitive that is in the forefront with respect to the first virtual camera and the virtual three-dimensional space. Processing for storing at least one of position coordinates (world coordinate values) is performed. For example, when the Z buffer 179a is updated, the first image generation unit 214 assigns each pixel of the specific buffer corresponding to the screen to the identification information (primitive number i of the primitive) that is in the forefront with respect to the first virtual camera. ) And at least one of the position coordinates (world coordinates) of the virtual three-dimensional space is stored.

  In addition, the first image generation unit 124 performs processing for storing the depth information (Z value) of the foremost primitive for the first virtual camera in each pixel of the specific buffer corresponding to the screen. May be. In this case, the depth information stored in the specific buffer 175 is the same content as the depth information of the first Z buffer 179a.

  In addition, the first image generation unit 124 performs processing for processing the first image. For example, the first image may be stored in the first texture buffer 178a, and the final image subjected to effect processing or the like may be stored in the first frame buffer 174a.

  Further, the second image generation unit 126 (right-eye image generation unit) is a second image (right-eye image) that can be seen from the second virtual camera (right-eye virtual camera VCR) in the virtual three-dimensional space. The image information of each pixel is stored in a second buffer (for example, the second frame buffer 174b, the second texture buffer 178b, and the second Z buffer 179b).

  In particular, the second image generation unit 126 of the present embodiment includes a vertex process that performs perspective projection conversion of a primitive onto a screen based on a second virtual camera in a virtual three-dimensional space, a pixel process that obtains image information of each pixel, To generate a second image.

  In addition, the second image generation unit 126 performs a process of comparing the primitive information and the primitive information of the pixels of the specific buffer for each pixel of the second image before the pixel processing. If the image information of the divertable pixel exists from the first image, and the image information of the divertable pixel exists from the first image, the first image is used instead of the pixel processing. A process of diverting the image information of the pixel of the image to the image information of the pixel of the second image is performed.

  For example, the second image generation unit 126 compares the primitive identification information with the primitive identification information of the pixels of the specific buffer for each pixel of the second image before pixel processing, and the primitive virtual By performing at least one of the process of comparing the position coordinates in the three-dimensional space and the position coordinates in the virtual three-dimensional space of the primitive of the pixel of the specific buffer, there is image information of the divertable pixels from the first image. When there is image information of a divertable pixel from the first image, the image information of the pixel of the first image is used as the second image instead of the pixel processing. The processing is applied to the image information of the pixels.

  In addition, the second image generation unit 126 determines the position coordinates of the primitive virtual three-dimensional space and the virtual cameras of the first virtual camera and the second virtual camera for each pixel of the second image before pixel processing. Whether or not there is image information of a divertable pixel from the first image by performing a process of comparing the position coordinates of the virtual three-dimensional space calculated based on the information and the depth information of the pixel of the specific buffer It is also possible to perform a process for determining the above.

  Also, the image information of pixels that can be diverted from the first image may be one or plural. Also, the process of diverting the image information of the pixels of the second image is, for example, the process of storing the image information of the pixels of the first image in the second buffer as the image information of the pixels of the second image, In this process, a plurality of pixels are sampled from one image, and image information obtained by combining (averaging, weighted averaging, etc.) the pixels is stored in the second buffer as image information of the pixels of the second image.

  Further, the second image generation unit 126 performs a process of determining that divertable pixel image information exists from the first image when the primitive information and the primitive information of the pixel of the specific buffer are the same, A process of diverting the image information of the pixel of the first image corresponding to the pixel of the specific buffer to the image information of the pixel of the second image is performed. That is, a process of diverting the image information of the pixel of the first image at the same position coordinate as the pixel of the specific buffer to the image information of the pixel of the second image is performed.

  For example, the second image generation unit 126 performs a process of determining that there is image information of a divertable pixel from the first image when the primitive identification information and the primitive identification information of the pixel in the specific buffer are the same. And processing for diverting the image information of the pixel of the first image corresponding to the pixel of the specific buffer to the image information of the pixel of the second image. That is, a process of diverting the image information of the pixel of the first image at the same position coordinate as the pixel of the specific buffer to the image information of the pixel of the second image is performed.

  In addition, the second image generation unit 126 can divert pixels from the first image when the position coordinates of the primitive virtual three-dimensional space and the position coordinates of the primitive pixel of the specific buffer are the same. The image information of the pixel of the first image corresponding to the pixel of the specific buffer is used as the image information of the pixel of the second image. That is, a process of diverting the image information of the pixel of the first image at the same position coordinate as the pixel of the specific buffer to the image information of the pixel of the second image is performed.

  Further, the second image generation unit 126 specifies, for each pixel of the second image, based on the virtual camera information of the first virtual camera and the second virtual camera and the depth information of the pixel of the second image. A search area of the buffer is determined, and processing for comparing primitive information with primitive information of pixels in the search area is performed.

  For example, the second image generation unit 126 compares the primitive identification information with the primitive identification information of the pixels in the search area, and the position coordinates of the primitive in the virtual three-dimensional space and the pixels in the search area. At least one of processing for comparing the position coordinates of the primitive in the virtual three-dimensional space is performed.

  For example, the search area of the specific buffer is an area for two pixels that are continuously arranged in the same row as the pixel to be processed of the second image.

  For example, in the search area, when there is a pixel having the same identification information of the primitive of the pixel to be processed in the second image and / or the position of the primitive of the pixel to be processed in the second image in the virtual three-dimensional space If there is a pixel with the same coordinate, the pixel of the first image corresponding to the pixel of the search area determined to be the same (at the same position coordinate as the pixel of the search area determined to be the same) The image information (for example, color information) is diverted to the image information (for example, color information) of the processing target pixel of the second image.

  In addition, when all the pixels in the search area are the same as the identification information of the primitive of the pixel to be processed in the second image, and / or in the virtual three-dimensional space of the primitive of the pixel to be processed in the second image If the position coordinates are the same, the image information (for example, color information) of the pixel group of the first image corresponding to all the pixels in the search region is sampled, and some calculation (averaging) is performed based on the sampled image information. , Weighted averaging, etc.) may be performed to apply the processing to the image information of the pixels of the second image.

  When the stereoscopic image generation unit 128 generates a stereoscopic image based on the first to Nth (N ≧ 3) virtual cameras, the second image generation unit 126 includes the second to second images. In each of the N images, for each pixel, processing for determining whether there is image information of a divertable pixel from the first image is performed, and there is image information of a divertable pixel from the first image. In such a case, instead of pixel processing, a process of diverting the image information of the pixel of the first image to the image information of the pixel of the image is performed.

  In addition, the second image generation unit 126 performs processing for processing the second image. For example, the second image may be stored in the second texture buffer 178b, and the final image subjected to effect processing or the like may be stored in the second frame buffer 174b.

  The stereoscopic image generation unit 128 performs processing for generating a stereoscopic image based on the first image and the second image. That is, the stereoscopic image generation unit 128 performs interleaving processing from the first image and the second image according to the stereoscopic method (naked eye method, polarized glasses method, page flip method, etc.) to generate a stereoscopic image. Generate. For example, the stereoscopic image generation unit 128 performs processing for generating a combined image obtained by combining the first image and the second image, switching processing for the first image and the second image at a predetermined cycle, and the like. In addition, the stereoscopic image generation unit 128 performs processing of alternately arranging the first image and the second image in a strip shape, and alternately arranging the first image and the second image on odd lines and even lines. To perform the processing.

  The sound generation unit 130 performs sound processing based on the results of various processes performed by the processing unit 100, generates game sounds such as BGM, sound effects, or sounds, and outputs the game sounds to the sound output unit 192.

  The sound generation unit 130 may generate a sound whose sound image localization changes according to the position of the object.

2. Overview of Stereoscopic Image Generation Processing An overview of a method for generating a stereoscopic image will be described with reference to FIG. As shown in FIG. 2, when a stereoscopic image is generated by the binocular stereoscopic method, the left-eye virtual camera VCL and the right eye separated by a distance corresponding to the width of both eyes of the observer (player). The virtual camera VCR is set in a virtual three-dimensional space (object space, world coordinate system). Then, a left-eye view volume VVL is set corresponding to the left-eye virtual camera VCL, and a right-eye view volume VVR is set corresponding to the right-eye virtual camera VCR.

  The left and right view volumes VVL and VVR have upper, lower, left, and right constituent planes determined by the positions, orientations, angles of view, and screen (projection plane) 40 for the left and right virtual cameras VCL and VCR. This is the area of the viewing frustum. The screen 40 is a surface corresponding to the screen of the display unit 190, and the positions and orientations of the left-eye and right-eye virtual cameras VCL and VCR corresponding to the observer's viewpoint position, the observer's viewpoint position, and the display part. It is set based on the positional relationship with the 190 screen. In FIG. 2, NPL is the near clip plane of the left-eye view volume VVL, FPL is the far-clip plane of the left-eye view volume VVL, NPR is the near-clip plane of the right-eye view volume VVR, and FPR is the right It is a far clip surface of an eye view volume VVR.

  The left-eye image (first image) that can be seen from the left-eye virtual camera VCL is drawn by perspectively projecting primitives (polygons) constituting objects existing in the left-eye view volume VVL onto the screen 40. To be generated. Similarly, the right-eye image (second image) that is an image that can be seen from the right-eye virtual camera VCR is a perspective projection of primitives (polygons) that constitute objects existing in the right-eye view volume VVR onto the screen 40. And generated by drawing.

  Here, when the primitive is positioned on the screen 40 in the virtual three-dimensional space, the drawing position of the primitive in the left-eye image matches the drawing position of the primitive in the right-eye image, and the parallax of the primitive image Will disappear. That is, the screen 40 is a surface where the parallax of the image on which the primitive located on the screen 40 is drawn is eliminated. In this case, it seems to the viewer that the primitive exists on the screen of the display unit 190.

  Further, the first area AR1 in which the primitive is an area on the near side as viewed from the left-eye and right-eye virtual cameras VCL and VCR from the screen 40 (an area sandwiched between the screen 40 and the near clip planes NPL and NPR). 3A, the drawing position of the primitive PTL in the left-eye image is shifted to the right with respect to the drawing position of the primitive PTR in the right-eye image. From the above, it appears that the primitives are popping out from the screen of the display unit 190. Further, the second area AR2 in which the primitive is an area behind the screen 40 from the left and right eye virtual cameras VCL and VCR (an area between the screen 40 and the far clip surfaces FPL and FPR). 3B, the drawing position of the primitive PTL in the left-eye image is shifted to the left with respect to the drawing position of the primitive PTR in the right-eye image. From the above, it appears that the primitive is retracted to the back of the screen of the display unit 190.

3. Description of Typical Rendering Pipeline Next, image generation processing by a typical rendering pipeline RP will be described with reference to FIG. For example, when generating an image in the rendering pipeline RP, first, vertex data of each vertex of each primitive constituting the object (vertex position coordinates, texture coordinates, color data, normal vector, α value, etc.) Primitive data (primitive number, etc.) is input (step S11), and geometry conversion (vertex processing in a broad sense) is performed based on the input vertex data and primitive data (step S12).

  Then, clipping and rasterization (scan conversion) are performed based on the vertex data after the geometry conversion (after the vertex processing) (step S13). That is, the surface of the primitive is associated with the pixel.

  Next, hidden surface removal processing is performed by a Z test (Z buffer method, depth comparison method) using a Z buffer 179 (depth buffer) in which the Z value (depth information) of the drawing pixel is stored (step S14).

  Then, pixel rendering processing (pixel processing in a broad sense) for rendering pixels constituting the image and lighting processing are performed (step S16). In pixel rendering (step S16), texture reading (texture mapping) and texture sampling may be performed (step S15) to determine the final drawing color of the pixels constituting the image.

  Then, a process for outputting (drawing) pixel color information to the frame buffer (drawing buffer) 174, and a process for storing the Z value in the Z buffer 179 when drawing the drawing pixel corresponding to the primitive (step S17). .

  For example, when generating a conventional binocular stereoscopic image, first, an image for the left eye, which is an image seen from the virtual camera for the left eye in the virtual three-dimensional space, is displayed in the frame buffer for the left eye using the rendering pipeline RP. Then, the image for the right eye, which is an image that can be seen from the virtual camera for the right eye in the virtual three-dimensional space, is drawn in the frame buffer for the right eye 174b by the rendering pipeline RP. In other words, since it is necessary to perform pixel rendering (pixel processing) twice for the left-eye image and the right-eye image, there is a problem that the processing load becomes very high.

4). Therefore, in the present embodiment, after rendering the left-eye image (an example of the first image) in the first frame buffer 174a (left-eye frame buffer), the pixels of the left-eye image To determine whether or not the color information of the right eye can be diverted as the color of the pixel drawn in the right eye image (an example of the second image), and the color information of the pixel drawn in the left eye image is used as the right eye When it is determined that the color of the pixel of the image for use can be diverted, a method is adopted in which the color information of the pixel drawn in the image for the left eye is used as the color of the pixel of the image for the right eye.

  For example, when generating the image for the left eye, each pixel of the specific buffer 175 (for example, the stencil buffer) corresponding to the screen 40 has the number i (identification) of the primitive on the forefront with respect to the virtual camera VCL for the left eye. Information) and at least one of world coordinate values (positional coordinates in the virtual three-dimensional space) is stored.

  When the right-eye image is generated, a process of comparing the primitive number i with the primitive number i of the specific buffer 175 for each pixel of the second image before pixel processing, By performing at least one of the process of comparing the world coordinate value and the world coordinate value of the primitive in the specific buffer 175, there is a pixel of color information (R, G, B) that can be diverted from the first image. The process which determines whether to do is performed. Then, when there is color information of a divertable pixel from the first image, the color information (R, G, B) of the pixel from the first image is replaced by the second image instead of pixel processing. Processing to be applied to pixel color information (R, G, B) is performed.

  In this way, for example, pixel processing can be omitted in the image generation processing of the right eye image, and the processing load can be reduced. In addition, by diverting pixel color information from the first image, it is possible to generate a stereoscopic image with less discomfort in, for example, a highlight area. Hereinafter, the details of the processing of this embodiment will be described.

4-1. Preparation First, when generating the right eye image and the left eye image, world coordinate conversion processing and screen coordinate conversion processing are performed.

  When generating the right eye image and the left eye image, the vertex buffer for storing the vertex coordinates Vp (x, y, z) of each vertex of the primitive (polygon) after world coordinate conversion. Prepare VB. In the vertex buffer VB, in addition to the vertex coordinates Vp, color information (R, G, B color values) of each vertex, normals (x, y, z) of each vertex, texture coordinates (U, V) may be stored.

  Further, when generating the right eye image and the left eye image, the position coordinates Wp (Wx, Wy, Wz) of each primitive in the world coordinate system, and as shown in FIG. Holds the primitive number i (index). The primitive number i is identification information that can identify each primitive, and is assigned an integer value.

  A specific buffer 175 (for example, a stencil buffer) having the same size as the size (resolution) of the screen 40 (screen, projection plane) is secured in advance in the storage area of the storage unit 170. For example, when the size of the screen 40 is 640 × 480, the specific buffer 175 has a size of 640 × 480.

4-2. Description of Generation Process of First Image (Left Eye Image) When generating a multi-view stereoscopic image, an image to be generated first is assumed to be a first image. In the present embodiment, the left-eye image is the first image.

  In the left-eye image generation processing, as shown in FIG. 6, after the vertex processing and rasterization based on the left-eye virtual camera VCL, the pixel position coordinates P (x, y) in the screen coordinate system 40 (two-dimensional coordinates). The position coordinate P (x, y) of the pixel of the specific buffer 175 (for example, stencil buffer) at the same position coordinate as the position of the primitive that is in the foreground in the view volume VVL and that is in front of the virtual camera VCL for the left eye The number i (an example of “primitive information”) and a position coordinate Wp (Wx, Wy, Wz) in the world coordinate system (an example of “primitive information”) are stored. That is, the primitive number i and the world coordinate value are stored in a two-dimensional array (x, y). In the present embodiment, when the Z buffer is updated, the primitive number i and the world coordinate value are overwritten in the specific buffer 175, whereby the primitive number i in the forefront with respect to the left-eye virtual camera VCL, Processing for storing the world coordinate values Wp (Wx, Wy, Wz) is performed.

  Note that only the primitive number i may be stored in the pixel position coordinate P (x, y) of the specific buffer 175 at the same position coordinate as the pixel position coordinate P (x, y) of the screen coordinate system 40. Alternatively, only the position coordinates Wp (Wx, Wy, Wz) of the primitive in the world coordinate system may be stored.

  Specifically, as shown in FIG. 6, when generating the left-eye image, the primitive PT1 (primitive number = 1) is perspective-projected on the screen 40 (projection plane) based on the left-eye virtual camera VCL. Conversion (vertex processing in a broad sense) is performed, and then rasterization is performed. Then, after rasterization and pixel processing, as shown in FIG. 7, the primitive PT1 is set to the pixel position coordinate P1 (x1, y1) of the specific buffer 175 that is the same as the pixel position coordinate P1 (x1, y1) of the screen coordinate system 40. The world coordinate system coordinate value Wp1 (Wx1, Wy1, Wz1) and the number “1” of the primitive PT1 are stored.

  That is, when the image for the left eye is drawn in the frame buffer 174a, the position coordinates P (x, y) of each pixel of the screen coordinates and the drawn primitive (viewed from the left-eye virtual camera VCL) are displayed in the specific buffer 175. The number of the frontmost primitive) and the world coordinate value Wp (Wx, Wy, Wz) are associated with each other.

  In the present embodiment, a process for creating a list of primitives visible in the left-eye image is performed. That is, the number of the foremost primitive in the view volume VVL as viewed from the left-eye virtual camera VCL is listed.

  FIG. 8 is a flowchart showing a processing flow of the rendering pipeline RP1 (first rendering pipeline) for the image for the left eye.

  For example, when generating an image for the left eye in the rendering pipeline RP1, first, vertex data and primitive data of each vertex of the primitive are input (step S21), and based on the input vertex data and primitive data, Geometry conversion (“vertex processing” in a broad sense) is performed. That is, vertex processing is performed for perspective projection conversion of the primitive onto the screen based on the left-eye virtual camera (step S22).

  Then, clipping and rasterization (scanning conversion) are performed based on the vertex data after the geometry conversion (step S23). That is, the surface of the primitive is associated with the pixel.

  Next, a hidden surface removal process is performed by a Z test using the Z buffer 179a in which the Z value of the drawing pixel is stored (step S24).

  Then, pixel rendering (pixel processing for obtaining image information of each pixel constituting the image) and lighting processing are performed (step S26). In pixel rendering (step S26), texture reading (texture mapping) or the like may be performed, texture sampling may be performed (step S25), and a final drawing color of pixels constituting the image may be determined.

  Then, a process of storing (storing) the primitive primitive number and the world coordinate value in each pixel of a specific buffer (for example, stencil buffer) is performed (step S27).

  Then, processing for outputting (drawing) the drawing color of each pixel to the left-eye frame buffer (first frame buffer 174a) and processing for storing the Z value in the left-eye Z buffer (first Z buffer 179a) are performed. (Step S28). The process ends here.

4-3. Description of Generation Process of Second Image (Right Eye Image) The second image generation process will be described. The second image is generated after the first image is generated. In the present embodiment, the right-eye image is the second image.

  First, when generating an image for the right eye, control is performed so as to send the image to a shader (vertex shader, vertex processing) in preference to the primitive visible in the image for the left eye. That is, the primitive drawn in the left-eye image is highly likely to be placed in the foreground when viewed from the right-eye virtual camera VCR, and the amount of calculation can be reduced.

  In this embodiment, vertex processing and rasterization are performed based on the right-eye virtual camera VCR, and after rasterization, before pixel processing of the right-eye image, for each pixel, a primitive number i and a specific buffer 175 (for example, , A process for comparing the primitive number i of the pixel of the stencil buffer) and a process of comparing the world coordinate value of the primitive and the world coordinate value of the primitive of the pixel of the specific buffer 175 for the left eye It is determined whether there is color information of a pixel that can be diverted from the image (frame buffer 174a), and when there is color information of a divertable pixel, the color information of the pixel is converted to the pixel information of the right eye image. A process of storing the color information in the second frame buffer 174b is performed. That is, before determining the drawing color for all pixels of the right-eye image, the specific buffer 175 is referenced to determine whether there is a color that can be diverted from the left-eye image.

  If there is color information of the divertable pixel from the left-eye image, the divertable pixel color information is stored in the right-eye frame buffer (frame buffer 174b) as the second pixel color information. To perform the process.

  Specifically, as shown in FIG. 9, when generating the right-eye image, the primitive PT1 (primitive number = 1) is perspective-projected on the screen 40 (projection surface) based on the right-eye virtual camera VCR. Conversion (vertex processing in a broad sense) is performed, and then rasterization is performed. For example, the world coordinate value of the primitive PT1 drawn on the pixel P2 (x2, y2) of the screen 40 is Wp1 (Wx1, Wy1, Wz1).

  Then, the specific buffer 175 is searched for a pixel that stores the same primitive number and world coordinate value as the position coordinate P2 (x2, y2) of the pixel to be processed of the right-eye image. That is, the pixel having the same primitive number i = 1 as the position coordinate P2 (x2, y2) of the pixel to be processed of the right-eye image and the world coordinate value Wp1 (Wx1, Wy1, Wz1) is specified buffer 175. Search from. This is because it is possible to more accurately determine whether there is pixel image information that can be diverted from the left-eye image.

  For example, as shown in FIG. 10, in the specific buffer 175, when the primitive number i = 1 and the pixel P1 (x1, y1) having the world coordinate value Wp1 (Wx1, Wy1, Wz1) exists, It is determined that there is pixel color information that can be diverted from the left-eye image. On the other hand, in the specific buffer 175, when there is no pixel having the primitive number i = 1 and the world coordinate value Wp1 (Wx1, Wy1, Wz1), there is no color information of a pixel that can be diverted from the left-eye image. Is determined.

  In the specific buffer 175, only the primitive number i may be compared. That is, the specific buffer 175 is searched for a pixel storing the same primitive number i = 1 as the position coordinate P2 (x2, y2) of the pixel to be processed of the right-eye image. For example, if there is a pixel P1 (x1, y1) having a primitive number i = 1 in the specific buffer 175, it is determined that there is color information of a pixel that can be diverted from the left-eye image. When there is no pixel with the number i = 1, it is determined that there is no color information of a pixel that can be diverted from the left-eye image. In this way, since the comparison target is only the primitive number i, the processing load can be reduced. For example, low polygons (primitives constituting an object with a low level of detail, objects consisting of a predetermined number of primitives or less) may be compared with the primitive number i.

  Further, in the specific buffer 175, only the world coordinate value of the primitive may be compared. That is, the specific buffer 175 is searched for a pixel storing the world coordinate value Wp1 (Wx1, Wy1, Wz1) of the same primitive as the position coordinate P2 (x2, y2) of the pixel to be processed of the right-eye image. For example, in the specific buffer 175, when there is a pixel P1 (x1, y1) that is the world coordinate value Wp1 (Wx1, Wy1, Wz1) of the primitive, it is determined that there is color information of a pixel that can be diverted from the left-eye image. In the specific buffer 175, when there is no pixel having the primitive world coordinate value Wp1 (Wx1, Wy1, Wz1), it is determined that there is no color information of a pixel that can be diverted from the left-eye image. In this way, it is possible to more accurately determine whether there is image information of pixels that can be diverted from the left-eye image, compared with only the primitive number i as a comparison target. For example, high polygons (primitives constituting objects with a high level of detail, objects consisting of a predetermined number or more of primitives) may be compared using world coordinate values or world coordinate values and primitive numbers. Good.

  In the present embodiment, in order to simplify the search process, the Z value of the pixel P2 (x2, y2) to be processed of the right eye image (Z value of the Z buffer 179b (the right eye Z buffer)). ), The search area B is determined in the specific buffer 175 based on the first and second virtual camera information, and the image information of the pixels that can divert each pixel in the specific buffer 175 search area B from the left-eye image is obtained. It may be determined whether or not it exists.

  For example, the search area B is determined as follows. First, as shown in FIG. 2, the viewpoint 31, the angle of view θC, the right-eye image of the reference virtual camera (center camera) VCC serving as a reference for setting the left-eye virtual camera VCL and the right-eye virtual camera VCR. As shown in FIG. 10, the search region B is determined based on the Z value (depth information, Z value in the right-eye viewpoint coordinate system) of the pixel P2 (x2, y2) to be processed.

  Specifically, when generating the image for the right eye, the position coordinate of the pixel to be rendered of the rasterized primitive is Pn, that is, the position coordinate of the pixel to be processed of the right eye image is Pn, and the reference Based on the viewpoint 31 of the virtual camera (center camera) VCC, the angle of view θC, and the Z value of the pixel Pn to be processed of the right-eye image, as shown in FIG. 11, the reference point Qn on the specific buffer 175 of the position coordinate Pn Ask for. Then, the two pixels Ka and Kb closest to the reference point Qn are determined as the search region B.

  Note that the viewpoint 30L and the angle of view θL of the virtual camera VCL for the left eye, the viewpoint 30R and the angle of view θR of the virtual camera VCR for the right eye, and the Z value (Z value in the viewpoint coordinate system) of the image for the right eye. The search area B may be determined.

  In addition, although the search area B of this embodiment is a continuous area for two pixels, the search area B may be a single pixel or a region composed of three or more pixels.

  Further, since the viewpoint 30L of the left-eye virtual camera VCL and the viewpoint 30R of the right-eye virtual camera VCR are usually arranged in parallel in the world coordinate system, in this embodiment, as shown in FIG. The search region B is determined in the nth row, which is the same row (line) as the pixel P2 (x2, y2) to be processed of the eye image.

  Then, as shown in FIG. 10, when the region (pixels P1, P3) for the second pixel in the nth row of the specific buffer 175 is determined as the search region B, the pixel P2 (x2) to be processed in the right-eye image , Y2) is compared with the primitive number i = 1 of each pixel P1, P3 in the search area B, and the world coordinates of the pixel P2 (x2, y2) to be processed of the right-eye image A process of comparing the value Wp1 (Wx1, Wy1, Wz1) with the world coordinate values of the pixels P1, P3 in the search region B is performed.

  Then, when at least one of the primitive number and the world coordinate value of the pixel P2 (x2, y2) to be processed of the right-eye image exists in the search area B, the color information of the pixel of the left-eye image can be diverted. Is determined.

  For example, as shown in FIG. 10, the primitive number i of the pixel P1 (x1, y1) in the search area B of the specific buffer 175 is “1”, the world coordinate values are Wx1, Wy1, Wz1, and the search of the specific buffer 175 is performed. When the primitive number i of the pixel P3 (x3, y3) in the region B is “3” and the world coordinate values are Wx3, Wy3, Wz3, the primitive number of the pixel P2 (x2, y2) to be processed of the right-eye image A pixel P1 (x1, y1) = (Wx1, Wy1, Wz1, 1) having the same i = 1 and the world coordinate value Wp1 (Wx1, Wy1, Wz1) is present in the search region B of the specific buffer 175. Therefore, it is determined that the color information of the pixel P1 of the left-eye image can be used as the color information of the processing target pixel P2 (x2, y2) of the right-eye image. Then, as shown in FIG. 12, the color information (R, G, B) of the pixel P1 (x1, y1) of the image for the left eye at the same position coordinates as the pixel P1 (x1, y1) of the specific buffer 175 is obtained. Processing for storing the color information (R, G, B) of the pixel P2 (x2, y2) of the right-eye image in the right-eye frame buffer 174b is performed. That is, a process of diverting the color information of the pixel P1 (x1, y1) of the left eye frame buffer 174a as the color information of the pixel P2 (x2, y2) of the right eye frame buffer 174b is performed.

  In the specific buffer 175 (or search area B), when there is no pixel that is the same as the primitive number and world coordinate value of the pixel to be processed of the right-eye image, the color information of the pixel that can be diverted from the left-eye image is displayed. Judge that it does not exist.

  For example, as shown in FIG. 13, when the primitive number i = 2 of the pixel P2 (x2, y2) to be processed of the right-eye image and the world coordinate value Wp2 (Wx2, Wy2, Wz2), the specific buffer In 175 (or search area B), when there is no pixel with primitive number i = 2 and world coordinate value Wp2 (Wx2, Wy2, Wz2), there is no color information of a pixel that can be diverted from the left-eye image. Determination is performed, normal pixel processing is performed on the pixel P2 of the image for the right eye, and processing for obtaining the drawing color of the pixel P2 is performed. That is, for the pixel P2 of the image for the right eye, a calculation process for determining the final drawing color is performed based on the vertex color of the primitive and the color of the texture image.

  In the present embodiment, when there are a plurality of pixels in the specific buffer that are considered to be the same as the primitive number and the world coordinate value of the pixel Pn to be processed in the right-eye image, the plurality of pixels are sampled, May be combined (averaged, etc.) to use the color information. For example, as shown in FIG. 11, among the center points Ka and Kb of the pixels Pa and Pb in the search area B, the pixel Ka closest to the reference point Qn is determined as a pixel to be used for the right-eye image. Good. In short, the pixels to be used for the right-eye image may be determined by the point sampling method. Further, as shown in FIG. 11, the color information of the pixels Pa and Pb may be diverted and used based on the distance between the center points Ka and Kb of the pixels Pa and Pb in the search area B and the reference point Qn. Good. In short, based on the bilinear filter, the color information of the plurality of pixels Pa and Pb in the search area B may be used as the image information of the right eye image.

  As described above, in the present embodiment, pixel processing of the right eye image can be omitted. Further, by diverting the color information of the pixel of the left-eye image to the color information of the pixel of the right-eye image, it is possible to generate a stereoscopic image with less sense of discomfort.

  FIG. 14 is a flowchart illustrating a processing flow of the rendering pipeline RP2 (second rendering pipeline) of the second image (right-eye image).

  For example, when generating an image for the right eye in the rendering pipeline RP2, first, vertex data and primitive data of each vertex of the primitive are input (step S31). Based on the input vertex data and primitive data, Geometry conversion (“vertex processing” in a broad sense) is performed (step S32).

  Then, clipping and rasterization are performed based on the vertex data after the geometry conversion (step S33). That is, the surface of the primitive is associated with the pixel.

  Next, a search area of a specific buffer (for example, stencil buffer) is determined based on the angle of view of the center camera, the viewpoint, and the Z value (depth value) of the virtual camera for the right eye (step S34).

  Then, it is determined whether or not there is a pixel in which at least one of the primitive world coordinate value and the primitive number exists in the search area of the specific buffer (step S35).

  If a pixel having at least one of the primitive world coordinate value and the primitive number is present in the search area of the specific buffer (Y in step S35), the left eye frame buffer 174a at the same position coordinate as the pixel is stored. The pixel color information is used (step S36).

  When there is no pixel having the same at least one of the primitive world coordinate value and the primitive number in the search area of the specific buffer (N in step S35), the Z buffer 179 that stores the Z value of the drawing pixel is used. A hidden surface removal process by a test is performed (step S37).

  Then, pixel rendering processing (pixel processing in a broad sense) and lighting processing of each pixel constituting the image are performed (step S39). In the pixel rendering (step S39), texture reading (texture mapping) or the like may be performed, texture sampling may be performed (step S38), and the final drawing color of the pixels constituting the image may be determined.

  Then, after steps S36 and S39, a process of outputting (drawing) the drawing color of the perspective-converted primitive to the right eye frame buffer 174b and a process of storing the Z value in the right eye Z buffer 179b are performed (step S40). ). The process ends here.

5. Application Example 5.1 Application Example of Diverted Image Information In this embodiment, for each pixel of the right eye image, a process is performed to determine whether or not color information of a divertable pixel exists from the left eye image. In the above description, the color information of the pixel of the left-eye image is used as the color information of the pixel of the right-eye image when there is color information of the pixel that can be diverted from the left-eye image. Not only information but also luminance information, translucent information, normal information, and depth information may be used.

  For example, in the case of stereoscopic viewing of a grayscale image, in the present embodiment, for each pixel of the right eye image, a process is performed to determine whether there is luminance information of a divertable pixel from the left eye image. When there is luminance information of pixels that can be diverted from the left-eye image, the luminance information of pixels of the left-eye image may be diverted to luminance information of pixels of the right-eye image. As described above, the process for determining whether there is luminance information of a divertable pixel from the left-eye image includes the primitive number, the world coordinate value, and the specific buffer 175 (for example, the stencil) at the time of generating the right-eye image. This may be done by comparing the primitive number of the buffer) and the world coordinate value. In this way, the right eye image and the left eye image can be generated with the same brightness for the same primitive and the primitive with the same world coordinate value, and a stereoscopic image with less discomfort. Can be generated.

  That is, in the present embodiment, for each pixel of the right eye image, a process for determining whether there is translucent information (α value) of divertable pixels from the left eye image is performed, and the left eye image When there is translucent information of pixels that can be diverted from the left, the translucent information of the pixels of the left eye image may be diverted to the translucent information of the pixels of the right eye image. As described above, the process of determining whether there is translucent pixel information that can be diverted from the left-eye image includes the primitive number, the world coordinate value, and the primitive number of the specific buffer 175 when the right-eye image is generated. This can be done by comparing the world coordinate values. In this way, in the right eye image and the left eye image, it is possible to perform translucent synthesis processing or the like on the same primitive and the primitive at the same world coordinate value at the same synthesis ratio. A few stereoscopic images can be generated.

  Further, in the present embodiment, for each pixel of the right eye image, processing for determining whether or not there is normal information (normal vector) of a divertable pixel from the left eye image is performed, and the left eye image When there is normal line information of pixels that can be diverted from the image, the normal line information of pixels of the left eye image may be diverted to the normal line information of pixels of the right eye image. As described above, the processing for determining whether there is normal pixel information that can be diverted from the left-eye image includes the primitive number, the world coordinate value, and the primitive number of the specific buffer 175 when the right-eye image is generated. This can be done by comparing the world coordinate values. In this way, the right eye image and the left eye image can be subjected to various processes such as gloss calculation and normal mapping with the same normal information for the same primitive and the primitive at the same world coordinate value. Therefore, it is possible to generate a stereoscopic image with less sense of discomfort.

  Further, in the present embodiment, for each pixel of the right eye image, a process for determining whether there is depth information (depth information) of a divertable pixel from the left eye image is performed. When there is depth information of divertable pixels, the depth information of the pixels of the left eye image may be diverted to the depth information of the pixels of the right eye image. As described above, the process for determining whether there is depth information of a divertable pixel from the left-eye image includes the primitive number, the world coordinate value, the primitive number of the specific buffer 175 at the time of generating the right-eye image, This can be done by comparing the world coordinate values. In this way, in the right-eye image and the left-eye image, various processing can be performed on the same primitive and the primitive at the same world coordinate value with the same depth information, and the stereoscopic image with less uncomfortable feeling. A visual image can be generated.

5.2 Application example of virtual camera In this embodiment, an example in which an image for the left eye is generated first and an image for the right eye is generated later has been described. However, an image for the right eye is first generated as the first image. However, the left-eye image may be generated later as the second image.

  In the present embodiment, the stereoscopic image generation process based on two viewpoints (right eye and left eye) has been described, but a stereoscopic image based on three or more viewpoints may be generated. For example, the first generated virtual camera image is used as the first image, and the world coordinate value and primitive number of the foremost primitive as viewed from the virtual camera are stored in the specific buffer 175 (for example, stencil buffer). When generating images of the second viewpoint (second image to Nth image (N is an integer of 3 or more)), the specific buffer 175 is referred to and pixels that can be diverted from the first image for each pixel. If the image information that can be diverted from the first image exists, the image information of the pixel of the first image is changed to the image information of the pixel of the image. You may make it divert to image information.

  For example, as shown in FIG. 15, when generating five multi-view stereoscopic images of the virtual cameras VC1 to VC5, first, based on the central virtual camera VC3 by the first rendering pipeline RP1. , Generate an image. Then, when generating images of the other virtual cameras VC1, VC2, VC4, and VC5, the images are generated based on the second rendering pipeline RP2. That is, when generating an image based on the virtual cameras VC1, VC2, VC4, and VC5, it is determined whether or not the color information of the image generated based on VC3 can be diverted. Use the color information.

5.3 Processing for omitting the Z test In this embodiment, when the color information of the pixel of the left-eye image is used as the color information of the pixel Pn to be processed of the right-eye image, the right-eye image The Z value of the Z buffer 179b of the pixel Pn may be set to the maximum value. That is, by setting the Z value of the Z buffer 179b of the pixel Pn of the right-eye image to the maximum value, in the Z test, the pixel Pn is always the foremost pixel when viewed from the right-eye virtual camera. The pixel processing can be omitted, and the processing load can be reduced. For example, if it is assumed that the primitive that is in the forefront with respect to the left-eye virtual camera VCL is also in the forefront with respect to the right-eye camera VCR, a processing load is required when generating a stereoscopic image. Is reduced and effective.

5.4 Example of Processing Processing In the present embodiment, processing for processing the left eye image (first image) generated by the first rendering pipeline RP1 is performed, and the processing is performed by the second rendering pipeline RP2. Alternatively, a process for processing the right-eye image (second image) may be performed, and a process for generating a stereoscopic image may be performed based on the processed first and second images.

  For example, the image for the left eye generated by the first rendering pipeline RP1 is stored in the first texture buffer 178a, and the image for the right eye generated by the second rendering pipeline RP2 is stored in the second texture buffer 178b. To store.

  Then, each texture image stored in each texture buffer 178a, 178b is subjected to special processing such as independent effect processing. In this way, when it is appropriate to apply different processing to the right eye image and the left eye image while reducing the processing load, there is little discomfort and the intended glossiness is moderately adjusted. A maintained stereoscopic image can be generated.

5.5 Example of Storing Depth Information in Specific Buffer In this embodiment, each pixel of the specific buffer corresponding to the screen has a primitive on the forefront with respect to the left-eye virtual camera (first virtual camera) VCL. Processing for storing depth information (an example of “primitive information”) may be performed. The depth information (Z value in the left-eye viewpoint coordinate system) stored in the specific buffer 175 is the same content as the depth information in the first Z buffer 179a. Note that the first Z buffer 179a may be used as an example of a specific buffer. This is because the depth information has a smaller amount of data than the position coordinates in the virtual three-dimensional space, so that the storage capacity can be saved.

  When the depth information is stored in the specific buffer, before the pixel processing of the right eye image, for each pixel of the right eye image (second image), the position coordinates of the primitive virtual three-dimensional space, A process of comparing the virtual camera information of the left-eye virtual camera VCL and the right-eye virtual camera VCR and the position coordinates of the virtual three-dimensional space calculated based on the depth information (Z value) of the pixel of the specific buffer. Thus, processing for determining whether there is image information of divertable pixels from the first image is performed.

  For example, based on the “viewpoint 30L of the left-eye virtual camera VCL and the viewpoint 30R of the right-eye virtual camera VCR” and the depth information (Z value) of each pixel stored in the specific buffer, the left-eye virtual camera VCL On the other hand, the world coordinate value Wp (Wx, Wy, Wz) of the primitive at the forefront is obtained. Then, the world coordinate value of the pixel to be processed of the image for the right eye is compared with the obtained world coordinate value Wp (the world coordinate value Wp of the primitive in the forefront with respect to the left-eye virtual camera VCL). It is determined whether there is pixel color information that can be diverted from the eye image. In this method, instead of reducing the storage capacity, the calculation load increases. Therefore, it is preferable to use it when it is desired to reduce the storage capacity rather than the calculation load.

6). Server In the present embodiment, the present invention may be applied to a server 20 of a client / server system. That is, the server 20 may perform a part of processing (for example, at least one processing of the processing unit 100) performed by the image generation apparatus 10 (terminal).

  For example, the server 20 is connected to the plurality of image generation apparatuses 10 </ b> A to 10 </ b> C via a network (for example, the Internet), and the server 20 receives input information from the image generation apparatus 10. Then, the server 20 may generate an image based on the received first, second, and third input information, and transmit the generated image to the image generation apparatus 10.

  FIG. 16 shows an example of a functional block diagram of the server 20. Note that the server 20 of the present embodiment may have a configuration in which some of the components (each unit) in FIG. 16 are omitted.

  The storage unit 270 serves as a work area for the processing unit 200, the communication unit 296, and the like, and its function can be realized by a RAM (VRAM) or the like. The storage unit 270 of the present embodiment stores a main storage unit 272 used as a work area, a frame buffer 274 for storing images, a specific buffer 275 (for example, a stencil buffer), and object model data. An object data storage unit 276, a texture storage unit 278 for storing textures for each object data, and a Z buffer 279 for storing Z values during object image generation processing. Note that some of these may be omitted.

  The frame buffer 274 includes a first frame buffer 274a (a left-eye frame buffer) and a second frame buffer 274b (a right-eye frame buffer). The texture storage unit 178 includes a first texture buffer 278a (left eye texture buffer) and a second texture buffer 278b (right eye texture buffer). The Z buffer 179 includes a first Z buffer 279a (left eye Z buffer) and a second Z buffer 279b (right eye Z buffer).

  The information storage medium 280 (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (CD, DVD), magneto-optical disk (MO), magnetic disk, hard disk, and magnetic tape. Alternatively, it can be realized by a memory (ROM). The processing unit 200 performs various processes of the present embodiment based on a program (data) stored in the information storage medium 280. The information storage medium 280 can store a program for causing a computer to function as each unit of the present embodiment (a program for causing a computer to execute the processing of each unit).

  The communication unit 296 performs various controls for communicating with the outside (for example, the image generation apparatus 10 and other servers), and functions thereof are hardware such as various processors or communication ASICs, and a program. It can be realized by.

  In addition, the program and data for functioning a computer as each part of this embodiment memorize | stored in the information storage medium and memory | storage part which another server has are received via a network, and the received program and data are stored in information. You may memorize | store in the medium 280 or the memory | storage part 270. The case where the image generating apparatus 10 is made to function by receiving the program or data as described above is also included in the scope of the present invention.

  The processing unit 200 (processor) performs processing such as game processing, image generation processing, and sound processing based on the received data and programs.

  The processing unit 200 performs various processes using the main storage unit 271 in the storage unit 270 as a work area. The functions of the processing unit 200 can be realized by hardware such as various processors (CPU, DSP, etc.), ASIC (gate array, etc.), and programs.

  The processing unit 200 includes an object space setting unit 210, a movement / motion processing unit 212, a virtual camera control unit 218, a communication control unit 219, an image generation unit 220, and a sound generation unit 230.

  The object space setting unit 210 includes various objects (polygons, free-form surfaces, subdivision surfaces, etc.) representing display objects such as objects (characters, moving objects), buildings, stadiums, cars, trees, pillars, walls, maps (terrain). A process of placing and setting an object composed of primitives) in a virtual three-dimensional space (object space) is performed. For example, the object space setting unit 210 performs the same processing as the object space setting unit 110 of the image generation device 10.

  The movement / motion processing unit 212 performs a movement / motion calculation (movement / motion simulation) of an object (moving body, character, etc.). The movement / motion processing unit 212 may perform movement / motion calculation of the object based on operation data received from the image generation apparatus 10 via the network. For example, the movement / motion processing unit 212 performs the same processing as the movement / motion processing unit 112 of the image generation apparatus 10.

  The virtual camera control unit 218 performs a virtual camera (viewpoint) control process for generating an image that can be viewed from a given (arbitrary) viewpoint in the virtual three-dimensional space. For example, the virtual camera control unit 218 performs the same processing as the virtual camera control unit 118 of the image generation apparatus 10.

  The communication control unit 219 performs processing for transmitting / receiving data to / from the image generation apparatus 10 or another server 20 via the network.

  Note that the server 20 of the present embodiment may transmit network information necessary for communication control to the image generation apparatus 10. For example, the server 20 manages terminal identification information given to each terminal individually, and destination information that designates a transmission destination of a packet associated with the terminal identification information.

  The communication control unit 219 of the server 20 generates a packet to be transmitted to the image generation apparatus 10, a process for designating the IP address and port number of a packet transmission destination terminal, and data included in the received packet in the storage unit 270. Processing to save, processing to analyze received packets, control processing related to transmission / reception of other packets, and the like are performed.

  In addition, the communication control unit 219 according to the present embodiment performs a process of transmitting and receiving data to and from each other for a predetermined period (for example, every one second) from when the connection with the terminal is established until the connection is disconnected. Here, the data transmitted from the image generation device 10 may be operation information from the operation unit 160 of the image generation device 10 or position information of objects (characters, moving objects) to be operated by the image generation devices 10. It is good also as movement information.

  In addition, when the communication control unit 219 of the present embodiment receives a packet transmitted from the image generation apparatus 10, the communication control unit 219 analyzes the received packet and includes data such as position information of an operation target object of another terminal included in the packet. Is stored in the main storage unit 272.

  In addition, the communication control unit 219 according to the present embodiment is configured so that the image data of each image generated based on the first to Nth virtual cameras (for example, in the case of a twin-lens type, the first image data and the second image data). Alternatively, the stereoscopic image data may be transmitted to the image generation apparatus 10 via a network.

  The image generation unit 220 performs a drawing process based on the results of various processes performed by the processing unit 200, thereby generating an image. The process of the image generation unit 220 performs the same process as that of the image generation unit 120 of the image generation apparatus 10.

  In particular, the image generation unit 220 of the present embodiment includes a first image generation unit 224, a second image generation unit 226, and a stereoscopic image generation unit 228.

  The first image generation unit 224 performs the same processing as that of the first image generation unit 124 of the image generation apparatus 10.

  For example, the first image generation unit 224 (left-eye image generation unit) is the first visible from the first virtual camera (left-eye virtual camera VCL) in the virtual three-dimensional space (world coordinate system, object space). Image information (at least one of color information, normal line information, depth information, and translucent information) of each pixel of the image (image for left eye) of the first buffer (for example, the first frame buffer 274a, 1 texture buffer 278a and first Z buffer 279a).

  In particular, the first image generation unit 224 of the present embodiment performs vertex processing for perspective projection conversion of a primitive onto a screen based on a first virtual camera in a virtual three-dimensional space, and pixel processing for obtaining image information of each pixel. By performing, the process which produces | generates a 1st image is performed.

  In addition, the first image generation unit 224 performs processing for storing the information of the primitive on the forefront with respect to the first virtual camera in each pixel of the specific buffer 275 (for example, stencil buffer) corresponding to the screen. .

  In addition, the first image generation unit 224 performs processing for processing the first image. For example, the first image may be stored in the first texture buffer 278a and the final image subjected to effect processing or the like may be stored in the first frame buffer 274a.

  The second image generation unit 226 performs the same processing as the second image generation unit 126 of the image generation apparatus 10.

  That is, the second image generation unit 226 (right-eye image generation unit) is a second image (right-eye image) that can be seen from the second virtual camera (right-eye virtual camera VCR) in the virtual three-dimensional space. The image information of each pixel is stored in a second buffer (for example, the second frame buffer 274a, the second texture buffer 278b, and the second Z buffer 279b).

  In particular, the second image generation unit 226 of the present embodiment performs vertex processing for perspective projection conversion of a primitive onto a screen based on a second virtual camera in a virtual three-dimensional space, and pixel processing for obtaining image information of each pixel. By performing, the process which produces | generates a 2nd image is performed.

  In addition, the second image generation unit 226 performs a process of comparing the primitive information and the primitive information of the pixels of the specific buffer for each pixel of the second image before the pixel processing, thereby performing the first image processing. A process for determining whether there is image information of a divertable pixel from the image is performed, and when there is image information of a divertable pixel from the first image, the first image is used instead of the pixel processing. The image information of the pixel is used as the image information of the pixel of the second image.

  For example, the second image generation unit 226 performs a process of determining that there is image information of a divertable pixel from the first image when the primitive identification information and the primitive identification information of the pixel of the specific buffer are the same. And processing for diverting the image information of the pixel of the first image corresponding to the pixel of the specific buffer to the image information of the pixel of the second image. In addition, the second image generation unit 226 can use a pixel that can be diverted from the first image when the position coordinate of the primitive virtual three-dimensional space and the position coordinate of the primitive three-dimensional space of the pixel of the specific buffer are the same. The image information of the pixel of the first image corresponding to the pixel of the specific buffer is used as the image information of the pixel of the second image.

  In addition, the second image generation unit 226 specifies for each pixel of the second image based on the virtual camera information of the first virtual camera and the second virtual camera and the depth information of the pixels of the second image. Processing for determining a search area of the buffer, comparing the identification information of the primitive with the identification information of the primitive of the pixel in the search area, and the position coordinates of the primitive in the virtual three-dimensional space and the virtual of the primitive of the pixel in the search area At least one of processing for comparing the position coordinates in the three-dimensional space is performed. What is necessary is just to determine a search area | region by the process similar to the 2nd image generation part 126 of the image generation apparatus 10. FIG.

  When the stereoscopic image generation unit 228 generates a stereoscopic image based on the first to Nth (N ≧ 3) virtual cameras, the second image generation unit 226 includes the second to second images. In each of the N images, for each pixel, processing for determining whether there is image information of a divertable pixel from the first image is performed, and there is image information of a divertable pixel from the first image. In this case, instead of the pixel processing, the image information of the pixel of the first image is used as the image information of the pixel of the image.

  In addition, the second image generation unit 226 performs processing for processing the second image. For example, the final image that has been stored in the second texture buffer 278b and subjected to the effect processing or the like may be stored in the second frame buffer 274b.

  The stereoscopic image generation unit 228 performs processing for generating a stereoscopic image based on the first image and the second image. The stereoscopic image generation unit 228 performs the same processing as the stereoscopic image generation unit 128.

  The sound generation unit 230 performs sound processing based on the results of various processes performed by the processing unit 100.

7). Others The present invention can be applied to a stereoscopic image generation process of various game systems such as a home game system and an arcade game system.

  The present invention is not limited to the one described in the above embodiment, and various modifications can be made. For example, terms cited as broad or synonymous terms in the description in the specification or drawings can be replaced with broad or synonymous terms in other descriptions in the specification or drawings.

  The present invention includes configurations that are substantially the same as the configurations described in the embodiments (for example, configurations that have the same functions, methods, and results, or configurations that have the same objects and effects). In addition, the invention includes a configuration in which a non-essential part of the configuration described in the embodiment is replaced. In addition, the present invention includes a configuration that exhibits the same operational effects as the configuration described in the embodiment or a configuration that can achieve the same object. Further, the invention includes a configuration in which a known technique is added to the configuration described in the embodiment. Note that the examples described in this embodiment can be freely combined.

100 processing unit, 110 object space setting unit, 112 movement / motion processing unit,
118 virtual camera control unit, 119 communication control unit, 120 image generation unit,
124 first image generation unit, 126 second image generation unit, 128 stereoscopic image generation unit,
130 sound generation unit, 160 operation unit, 170 storage unit, 172 main storage unit,
174 frame buffer, 174a first frame buffer,
174b second frame buffer, 175 specific buffer,
176 Object data storage unit, 178 Texture storage unit,
178a first texture buffer, 178b second texture buffer,
179 Z buffer, 179a first Z buffer, 179b second Z buffer,
180 information storage medium, 190 display unit, 192 sound output unit, 196 communication unit,
200 processing unit, 210 object space setting unit, 212 movement / motion processing unit,
218 virtual camera control unit, 219 communication control unit, 220 image generation unit,
224 first image generation unit, 226 second image generation unit, 228 stereoscopic image generation unit,
230 sound generator, 270 storage, 272 main storage,
274 frame buffer, 274a first frame buffer,
274b second frame buffer, 275 specific buffer,
276 Object data storage unit, 278 Texture storage unit,
278a first texture buffer, 278b second texture buffer,
279 Z buffer, 279a first Z buffer, 279b second Z buffer,
280 information storage medium, 296 communication unit,
10 terminals, 20 servers, 40 screens

Claims (12)

A program that arranges primitives in a virtual three-dimensional space and performs a process of generating a stereoscopic image,
In the virtual three-dimensional space, a process of generating a first image by performing a vertex process for perspective projection conversion of a primitive to a screen based on a first virtual camera and a pixel process for obtaining image information of each pixel. A first image generation unit to perform;
In the virtual three-dimensional space, a process of generating a second image by performing a vertex process for perspective projection conversion of a primitive to a screen based on a second virtual camera and a pixel process for obtaining image information of each pixel. A second image generation unit to perform;
Based on the first image and the second image, the computer functions as a stereoscopic image generation unit that performs a process of generating a stereoscopic image,
The first image generation unit
A process of storing the information of the primitive in the forefront with respect to the first virtual camera in each pixel of the specific buffer corresponding to the screen,
The second image generation unit
Prior to the pixel processing, by performing processing for comparing the primitive information and the primitive information of the pixels of the specific buffer for each pixel of the second image, image information of the divertable pixels from the first image Process to determine whether or not exists,
When there is image information of a pixel that can be diverted from the first image, instead of the pixel processing, the image information of the pixel of the first image is diverted to the image information of the pixel of the second image The program characterized by performing. In claim 1,
The first image generation unit
A process of storing identification information of a primitive on the forefront with respect to the first virtual camera in each pixel of a specific buffer corresponding to the screen,
The second image generation unit
Before the pixel processing, for each pixel of the second image, a process of comparing the primitive identification information with the primitive identification information of the pixel of the specific buffer is performed, so that pixels that can be diverted from the first image are compared. A program for performing a process of determining whether image information exists. In claim 1 or 2,
The first image generation unit
A process of storing the position coordinates in the virtual three-dimensional space of the primitive on the forefront with respect to the first virtual camera in each pixel of the specific buffer corresponding to the screen,
The second image generation unit
Before the pixel processing, for each pixel of the second image, a process of comparing the position coordinate of the primitive virtual three-dimensional space with the position coordinate of the pixel of the specific buffer in the virtual three-dimensional space is performed. A program for performing a process of determining whether there is image information of divertable pixels from an image of In any one of Claims 1-3,
The first image generation unit
A process of storing depth information of the foremost primitive for the first virtual camera in each pixel of the specific buffer corresponding to the screen,
The second image generation unit
Before the pixel processing, for each pixel of the second image, the position coordinates of the primitive virtual three-dimensional space, the virtual camera information of the first virtual camera and the second virtual camera, and the pixel of the specific buffer A process for determining whether there is image information of divertable pixels from the first image by performing a process of comparing the position coordinates of the virtual three-dimensional space calculated based on the depth information. A program characterized by In any one of Claims 1-4,
The second image generation unit
When the primitive information and the primitive information of the pixel of the specific buffer are the same, a process of determining that there is image information of a divertable pixel from the first image is performed, and a process corresponding to the pixel of the specific buffer is performed. A program that performs processing for diverting image information of a pixel of one image to image information of a pixel of a second image. In any one of Claims 1-5,
The second image generation unit
For each pixel of the second image, based on the virtual camera information of the first virtual camera and the second virtual camera and the depth information of the pixels of the second image, the search area of the specific buffer is determined, A program for performing processing for comparing primitive information and primitive information of pixels in a search area. In any one of Claims 1-6,
When the stereoscopic image generation unit generates a stereoscopic image based on the first to Nth (N ≧ 3) virtual cameras,
The first image generation unit
Based on the first virtual camera, the vertex processing and the pixel processing are performed to generate a first image, and each pixel of the specific buffer is loaded with respect to the first virtual camera. Perform the process of storing the information of the primitive on the front,
The second image generation unit
By performing the vertex processing and the pixel processing for each of the second to Nth virtual cameras, the second to Nth images are generated,
The second image generation unit
In each of the second to Nth images, for each pixel, a process for determining whether there is image information of a divertable pixel from the first image is performed, and an image of the divertable pixel from the first image A program characterized in that, when information exists, a process of diverting image information of the pixel of the first image to image information of a pixel of the image instead of the pixel processing. In any one of Claims 1-7,
The first image generation unit
Processing to process the first image;
The second image generation unit
Processing to process the second image;
The stereoscopic image generating unit is
A program for performing a process of generating a stereoscopic image based on a processed first image and a processed second image. In any one of Claims 1-8,
The image information is at least one of color information, normal information, depth information, and translucent information.   A computer-readable information storage medium, wherein the program according to any one of claims 1 to 9 is stored. An image generation apparatus that performs a process of arranging a primitive in a virtual three-dimensional space and generating a stereoscopic image,
In the virtual three-dimensional space, a process of generating a first image by performing a vertex process for perspective projection conversion of a primitive to a screen based on a first virtual camera and a pixel process for obtaining image information of each pixel. A first image generation unit to perform;
In the virtual three-dimensional space, a process of generating a second image by performing a vertex process for perspective projection conversion of a primitive to a screen based on a second virtual camera and a pixel process for obtaining image information of each pixel. A second image generation unit to perform;
A stereoscopic image generation unit that performs processing for generating a stereoscopic image based on the first image and the second image,
The first image generation unit
A process of storing the information of the primitive in the forefront with respect to the first virtual camera in each pixel of the specific buffer corresponding to the screen,
The second image generation unit
Prior to the pixel processing, by performing processing for comparing the primitive information and the primitive information of the pixels of the specific buffer for each pixel of the second image, image information of the divertable pixels from the first image Process to determine whether or not exists,
When there is image information of a pixel that can be diverted from the first image, instead of the pixel processing, the image information of the pixel of the first image is diverted to the image information of the pixel of the second image An image generating apparatus characterized by A server that performs a process of arranging a primitive in a virtual three-dimensional space and generating a stereoscopic image,
In the virtual three-dimensional space, a process of generating a first image by performing a vertex process for perspective projection conversion of a primitive to a screen based on a first virtual camera and a pixel process for obtaining image information of each pixel. A first image generation unit to perform;
In the virtual three-dimensional space, a process of generating a second image by performing a vertex process for perspective projection conversion of a primitive to a screen based on a second virtual camera and a pixel process for obtaining image information of each pixel. A second image generation unit to perform;
A stereoscopic image generation unit that performs processing for generating a stereoscopic image based on the first image and the second image,
The first image generation unit
A process of storing the information of the primitive in the forefront with respect to the first virtual camera in each pixel of the specific buffer corresponding to the screen,
The second image generation unit
Prior to the pixel processing, by performing processing for comparing the primitive information and the primitive information of the pixels of the specific buffer for each pixel of the second image, image information of the divertable pixels from the first image Process to determine whether or not exists,
When there is image information of a pixel that can be diverted from the first image, instead of the pixel processing, the image information of the pixel of the first image is diverted to the image information of the pixel of the second image A server characterized by performing

Images