JP2002109561A - Image processing system, device, method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system capable of expressing accurately an image even in the case of a three-dimensional image having translucent images intermingled complicatedly. SOLUTION: Image data including the depth distance from a specific part of the image and color information are generated by plural image generation devices 101-116. Composite devices 117-120 enable the plural image data to be specified (for example, sorted) in the order of the depth distance included therein respectively, and mix the color information of the image data having the relatively long depth distance with the color information of the image data for expressing at least a surface part overlapping thereon on the upper surface of the image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a three-dimensional image processing system and a three-dimensional image processing method for synthesizing a three-dimensional image based on a plurality of image data each including a depth distance and color information.

[0002]

2. Description of the Related Art A frame buffer and a z-buffer, which are widely used in existing computer systems, are used in a three-dimensional image processing apparatus for generating a three-dimensional image (hereinafter, simply referred to as "image processing apparatus"). . That is, an image processing apparatus of this type includes an interpolation calculator that receives drawing data generated by geometry processing from an image processing unit and generates an image data by performing an interpolation operation based on the received drawing data. Buffer and z
And a memory including a buffer.

In the frame buffer, image data including color information such as R value (red), G value (green), and B value (blue) of a three-dimensional image to be processed is drawn. In the z buffer,
A z-coordinate indicating a depth distance from a specific viewpoint, for example, a display surface viewed by an operator is stored. The interpolation calculator receives, for example, a drawing command of a polygon that is a basic configuration figure of a three-dimensional image, vertex coordinates of the polygon in a three-dimensional coordinate system, and color information of each pixel from the image processing unit. The color information and the depth distance are interpolated to generate image data representing the depth distance and color information for each pixel. The depth distance obtained by the interpolation calculation is stored at a predetermined address of the z buffer, and the color information is stored at a predetermined address of the frame buffer.

When a plurality of three-dimensional images overlap, z
Adjusted by the buffer algorithm. The z-buffer algorithm is a hidden surface process performed using a z-buffer, and is a process of erasing an image of an overlapping portion that is present at a position hidden by another image. In the z-buffer algorithm, the z-coordinate of each pixel of a plurality of images to be drawn is compared with each other to determine the anteroposterior relationship of the two images with respect to the display surface. If the depth distance is short, that is, if the position is closer to the viewpoint, Then, the image is drawn, and if it is far away, the image is not drawn, thereby deleting the overlapping portion of the image at the hidden position.

An image processing system for performing complex image processing using a plurality of such image processing apparatuses will be described below. This image processing system has four image processing devices and a z comparison device. Each image processing device draws image data including color information in a frame buffer, and sets z of a pixel constituting each image at that time.
Write coordinates to z-buffer.

[0006] The z comparison device performs hidden surface processing based on the image data written in the frame buffer of each image processing apparatus and the z coordinates written in the z buffer to generate a composite image. Specifically, the z comparison device reads out image data and z coordinates from each image processing device. Then, the image data having the smallest z coordinate among the read z coordinates is set as the image of the three-dimensional image to be processed. That is, an image based on the image data closest to the viewpoint is relatively displayed on the upper surface, and the image data of the image located on the back of the overlapping portion is erased to generate a composite image having the overlapping portion. I do.

[0007] For example, when image data generated by an image processing apparatus for drawing a background, an image processing apparatus for drawing a car, an image processing apparatus for drawing a building, and an image processing apparatus for drawing a person are fetched and overlapping occurs. Are located on the back of the
Erase hidden surfaces with a comparison device. Even in the case of a complicated three-dimensional image, finer image processing can be performed at higher speed by sharing and processing with a plurality of image processing apparatuses than in the case of processing with only one image processing apparatus. The above image processing system is described in the document "Computer Graphic
s Principles and Practice '' and `` Image-Composition-A
rchitectures ".

[0008]

In such a conventional image processing system, outputs from a plurality of image processing apparatuses are output by z
Although they are distinguished by the size of the coordinates, they are basically simple hidden surface processing. Therefore, even if the image having a relatively small z-coordinate is translucent among a plurality of overlapping three-dimensional images, since the hidden surface portion is deleted, it is determined that the three-dimensional image is translucent. There was a problem that it could not be correctly expressed.

In view of such circumstances, the present invention provides an improved image processing mechanism that can accurately represent, for example, even a three-dimensional image in which translucent images are complicatedly mixed. Is the subject.

[0010]

SUMMARY OF THE INVENTION The present invention provides an image processing system, an image processing device, an image processing method, and a computer program. The image processing system according to the present invention generates a plurality of image data for generating image data including a depth distance from a specific part of a three-dimensional image to be processed and color information (R value, G value, B value, A value). Means for capturing the image data from each of the plurality of image data generating means, and enabling the captured plurality of image data to be specified in the order of the depth distances contained in each of the plurality of image data, and an image having a relatively long depth distance. A color information mixing means for mixing color information of data and color information of image data for expressing at least a surface portion overlapping the data on the upper surface of the image. In the image processing system configured as described above, since the color information included in the image data is mixed in the order of the depth distance, even if a translucent image is mixed in a complicated manner, at least the color information is
It becomes possible to express accurately.

The image processing system of the present invention can be a network type system. That is, the image data generating means can be connected to a plurality of networks, and the network can include a plurality of synthesizing means and control means to constitute a network type image processing system. The control means selects one necessary for processing from the plurality of image data generating means and the plurality of synthesizing means. At least one of the plurality of combining means includes:
The color information mixing means is provided.

For example, the color information mixing means mixes the color information of the image data with the longest depth distance and the color information of the image data with the next longest depth distance, and determines the result of the mixing as the depth distance. Further, it is configured to be mixed with the color information of the next longer image data. More specifically, the color information of the image data having the longest depth distance,
It is configured to mix with color information of background image data for expressing the background. The image data having the longest depth distance may be used as background image data for expressing a background.

[0013] From the viewpoint of making the timing of taking in the image data from the plurality of image data generating means accurate and making the three-dimensional image clearer, a synchronizing means for synchronizing with the image processing timing in the own system is further provided. An image processing system is configured.

A part or all of the plurality of image data generating means, the color information mixing means, and the synchronizing means are constituted by a logic circuit, and the logic circuit is mounted in one semiconductor chip. It is also possible according to the present invention.

[0015] The image processing device of the present invention is a data processing device which acquires the image data from each of a plurality of image data generating means for generating image data including a depth distance from a specific portion and color information of a three-dimensional image to be processed. The capturing means and the plurality of captured image data can be specified in the order of the depth distances included in each of the captured image data, and the color information of the image data having a relatively long depth distance and at least the surface portion overlapping with the color information of the image data are displayed. A color information mixing means for mixing the color information of the image data to be displayed on the upper surface with the color information is mounted on a semiconductor chip. As in the image processing system, the image processing device may further include a synchronization unit that synchronizes the timing of capturing the image data from the plurality of image data generation units with the image processing timing of the own device.

Another image processing device of the present invention functions as the above-described image data generating means, and includes a frame buffer for storing image data including color information of a three-dimensional image to be processed, and a frame buffer for each of the image data. A z-buffer for storing a depth distance from a specific part, and the image data from a plurality of devices including the same, which can be specified in the order of the depth distance included in each of the devices, and the color of the image data having a relatively long depth distance. A means for performing communication from color information mixing means for mixing information and color information of image data for expressing at least an overlapping surface portion on the upper surface of the image is mounted on a semiconductor chip.

An image processing method according to the present invention is a method to be executed in an image processing system having a plurality of image data generating means and a synthesizing means connected to these image data generating means. Generating a plurality of image data including depth information and color information from a specific part by the plurality of image data generating means; and
The image data is fetched from each of the plurality of image data generating means at a predetermined synchronization timing, and the fetched image data can be specified in the order of the depth distance included in each of the image data, and the depth distance is relatively large. Mixing the color information of the long image data and the color information of the image data for expressing the overlapping surface portion on the upper surface of the image.

A computer program according to the present invention comprises a computer program for generating a plurality of image data generating means for generating image data including a depth distance from a specific portion and color information of a three-dimensional image to be processed; The image data is fetched from each of the means, and the plurality of fetched image data can be specified in the order of the depth distance included in each of the image data, and the color information of the image data having a relatively long depth distance overlaps with the color information of the image data. Is a computer program for operating as a three-dimensional image processing system including a color information mixing unit that mixes color information of image data for expressing the image data on the upper surface of the image.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the image processing system of the present invention is applied to a system for performing image processing of a three-dimensional model composed of complex image elements such as game characters will be described below.

<Overall Configuration> FIG. 1 is an overall configuration diagram of an image processing system according to the present embodiment. The image processing system 100 includes 16 image generation devices 101 to 116 as an example of an image data generation unit and five synthesis devices 117 to 121 as an example of a color information mixing unit. Each image generation device 101-116,
Each of the synthesizing devices 117 to 121 is configured to include a logic circuit and a semiconductor memory, respectively, and the logic circuit and the semiconductor memory are semiconductor chips mixedly mounted in one semiconductor chip. The number of image generation devices and the number of synthesis devices can be determined as appropriate according to the type and number of three-dimensional images to be processed or the processing mode.

The image generating devices 101 to 116 provide three-dimensional coordinates (x, y, z) of each vertex of a polygon for forming a three-dimensional model, homogeneous coordinates (s, t) and texture of the texture of each polygon. Drawing data including q is generated by a geometry process, and a characteristic rendering process is performed based on the drawing data. Further, upon reception of an external synchronization signal from the synthesis devices 117 to 120 connected at the subsequent stage, color information (R value, G value, B value, A value) as a result of the rendering process is transmitted from the frame buffer, and , The z-coordinate representing the depth distance from the specific part is output from the z-buffer to the subsequent synthesis device. At this time, a write enable signal WE for permitting the subsequent synthesis device to capture the color information (R value, G value, B value, A value) and the z coordinate is also output.
The frame buffer and the z-buffer are the same as those shown in the prior art, and the R value, the G value, and the B value are red, green, and blue luminance values, respectively, and the A value is transmittance (α). Is a numerical value representing.

The synthesizing devices 117 to 121 output data from an image generating device or another synthesizing device through a data capturing mechanism, specifically, x- and y-coordinates indicating the positions of individual pixels, and color information (R). Value, G
Value, B value, A value) and image data including the z coordinate. Then, the image data can be specified by the z-buffer algorithm using the z coordinate (z), and the color information (R) is sequentially arranged from the image data having the longest z coordinate (z) from the viewpoint.
The values (G values, B values, A values) are mixed (blended) to generate data (also image data) for expressing a composite three-dimensional image including a translucent image.

Each of the image generating devices 101 to 116 is connected to one of the subsequent synthesizing devices 117 to 120, and the synthesizing devices 117 to 120 are connected to the synthesizing device 121. As described above, the synthesis device can be connected in multiple stages. In the present embodiment, the image generation devices 101 to 116 are divided into four groups, and one combination device is provided in each group. That is, the image generation devices 101 to 104 are connected to the synthesis device 117, the image generation devices 105 to 108 are connected to the synthesis device 118, the image generation devices 109 to 112 are connected to the synthesis device 119, and the image generation devices 113 to 11
6 is connected to the synthesizing device 120. The image generating devices 101 to 116 and the synthesizing devices 117 to 121 are configured to synchronize the timing of the processing operation with a synchronizing signal described later.

Next, the image generation devices 101 to 116,
Specific configuration examples of the synthesis devices 117 to 121 and their functions will be described.

<Image Generation Device> FIG. 2 shows an overall configuration diagram of the image generation device. All image generation devices 1
Since the components 01 to 116 have the same components, in FIG. 2, the image generation devices are uniformly represented by reference numeral 200 for convenience.

The image generating device 200 is configured by connecting a drawing processor 201, a drawing memory 202, an I / O interface circuit 203 and a rendering circuit 204 to a bus 205. The drawing processor 201 reads necessary graphic data from the drawing memory 202 stored according to the progress of the application and the like, and performs coordinate conversion, clipping processing, lighting processing, and the like on the graphic data. Is performed to generate drawing data. The rendering data is supplied to the rendering circuit 204 via the bus 205.

The I / O interface circuit 203 fetches a control signal for controlling the movement of a three-dimensional model such as a character from an external operation means (not shown), or fetches drawing data generated by an external image processing means. It has the function of rubbing. The control signal is sent to the drawing processor 201 and used for controlling the rendering circuit 204. The drawing data has, for example, an x coordinate and ay coordinate of 1
6 bits, 24 bits for z coordinate, 12 bits for R value, G value, and B value (= 8 + 4), s, t, and q texture coordinates are each composed of 32 bits of floating point value (IEEE format). .

The rendering circuit 204 includes a mapping processing section 2041 and a memory interface (memory I / F).
Circuit 2046, CRT control unit 2047, and D
It has a RAM (Dynamic Random Access Memory) 2049. The rendering circuit 204 in the present embodiment includes:
A logic circuit such as a mapping processing unit 2041 and a DRAM 2049 for storing image data, texture data and the like are mixedly formed in one semiconductor chip.

The mapping processing unit 2041 includes a bus 205
The linear interpolation is performed on the drawing data sent through. By performing linear interpolation, color information (R value, G value, B value, A value) for each vertex of the polygon and drawing data representing only the z coordinate are used to calculate the color of each pixel on the surface of the polygon. Information (R value, G value, B value, A value) and z coordinate can be obtained. Further, by calculating the texture coordinates using the homogeneous coordinates (s, t) and the homogeneous terms q included in the drawing data, and performing texture mapping using the texture data corresponding to the texture coordinates, a more precise display is performed. Images can be obtained. In this way, (x, y, z, R, G, B, A) including (x, y) coordinates, color information, and z coordinates representing the position of each pixel
Is generated.

The memory I / F unit 2046 is provided for the DRAM 2 in response to a request from another circuit in the rendering circuit 204.
049 (write / read). The write path and read path at the time of access are configured as separate paths. That is, in the case of writing, the write address ADRW and the write data DTW are written via the write path, and in the case of reading, the read data DTR is read via the read path.
The memory I / F unit 2046 accesses the DRAM 2049 on a 16-pixel basis, for example, based on predetermined interleaving addressing.

The CRT control unit 2047 synchronizes with the external synchronization signal supplied from the synthesizing device,
Image data from M2049, that is, color information (R value, G value, B value, A value) from frame buffer 2049b, z
A request to read the z coordinate from the buffer 2049c is made via the memory I / F unit 2044. The image data including the read color information (R value, G value, B value, A value) and the z coordinate, and the write enable signal WE as a write signal are output to the subsequent synthesis device.

The DRAM 2049 includes a frame buffer 2049b in which color information (R value, G value, B value, A value) of pixel data is drawn, and a z buffer 2049c in which z coordinate of the pixel data is written. In addition, the DRAM 2049 stores texture data.

<Synthesis Device> FIG. 3 shows the overall configuration of the synthesis device. All synthetic devices 117-121
3 have the same components, and in FIG. 3, for convenience, the combining device is uniformly represented by reference numeral 300.

The synthesizing device 300 includes FIFOs 301 to
304, synchronization signal generation circuit 305 and synthesis block 30
6 is provided. The FIFOs 301 to 304 have a one-to-one correspondence with the four image generation devices provided in each preceding stage, and image data (R value, G value, B value, A value) output from the corresponding image generation device. Temporarily. That is, image data (R value, G value, B value, A value) is written into each of the FIFOs 301 to 304 in synchronization with the write enable signal WE from the corresponding image generation device. The written image data (R value, G value, B value,
A value) is output to the synthesizing block 306 in synchronization with the internal synchronization signal Vsync generated by the synchronization signal generation circuit 305. Since the image data (R value, G value, B value, A value) is output from the FIFOs 301 to 304 in synchronization with the internal synchronization signal Vsync, the input timing of the image data to the synthesizing device 300 is set to some extent freely. be able to.
Therefore, perfect synchronization between the image generation devices is not always necessary. Also, in the synthesizing device 300, the output of each of the FIFOs 301 to 304 is the internal synchronization signal V
Since almost completely synchronized by sync, each F
A synthesis block 30 for the outputs of the IFOs 301 to 304
By rearranging (sorting) in step 6 and performing mixing (α blending) of color information in order from the point farthest from the viewpoint, it is easy to combine four pieces of image data.

Here, an example in which four FIFOs are provided is shown, because four image generation devices are connected to one combining device. F
The number of IFOs is not limited to four, and can be arranged by the number of connected image generation devices. Also, FIF
O301 to O304 may be physically different memories, or one memory may be logically divided into a plurality of areas for use.

From the synchronizing signal generation circuit 305, an external synchronizing signal SYNCIN input from a subsequent device, for example, a display, of the synthesizing device 300 is supplied to the preceding image generating device or synthesizing device at the same timing. An external synchronizing signal SYNCIN supplied from the synthesizing device to the preceding device, and an internal synchronizing signal V of the synthesizing device
The generation timing of the sync will be described with reference to FIG. The synchronization signal generation circuit 305 generates the external synchronization signal SYNCIN and the internal synchronization signal Vsync at this timing. Here, as shown in FIG. 4A, the combination device 121, the combination device 117, and the image generation device 101
Will be described as an example of a case where are connected in three stages. Synthetic device 12
Vsync2 for the internal synchronization signal and SYNC for the external synchronization signal
IN2. The internal synchronization signal of the synthesizing device 117 is Vsync1, and the external synchronization signal is SYNCIN1.

As shown in FIGS. 4B to 4E, the generation timings of the external synchronization signals SYNCIN2 and SYNCIN1 are determined by the internal synchronization signals Vsync2 and Vsyn inside the synthesizing device.
It is moved forward by a predetermined period from c1. Then, to perform multi-stage connection, the internal synchronization signal of the combining device follows the external synchronization signal supplied from the subsequent combining device.
The advance period is because a period from when the image generation device or the like at the preceding stage receives the external synchronization signal SYNCIN to when the actual synchronization operation is started is anticipated, but the FIFOs 301 to 304 are input to the synthesis device. Since they are arranged, there is no problem even if there is slight temporal fluctuation. The advance period is set so that writing of image data to the FIFO is completed before reading of image data from the FIFO. Since the synchronization signal is repeated at a constant period, the advance period can be easily realized by a sequential circuit such as a counter. Further, by resetting a sequential circuit such as a counter with a synchronization signal from the subsequent stage, the internal synchronization signal can follow the external synchronization signal supplied from the synthesis device in the subsequent stage.

The synthesizing block 306 has an internal synchronizing signal Vsy.
4 supplied from FIFO 301 to 304 in synchronization with nc
The two pieces of image data are sorted by the z-coordinate (z) included in each piece of image data, and then color information (R value, G value, B value, A value) is mixed using the A value in order from the farthest viewpoint, That is, α blending is performed, and at a predetermined timing,
Output to the synthesis device 121 of the next stage.

FIG. 5 is a block diagram showing the main configuration of the synthesis block 306. The synthesis block 306 includes a z-sort unit 3061 and a blending unit 3062. The z-sort unit 3061 determines the color information (R1, G1, B1, A1) to (R4, G4, B4, A) of the four image data.
4) and the depth distances z1 to z4,
The magnitude of z4 is compared. Based on the result of the comparison, the image data is sorted in the descending order of the z-coordinate (z), that is, in the order distant from the viewpoint, and color information is supplied to the blending unit 3062 in the order distant from the viewpoint. In the example of FIG. 5, z1
>Z4>z3> z2.

The blending unit 3062 has four blending processors 3062-1 to 3062-4. The number of blending processors may be appropriately determined according to the number of pieces of color information to be combined.

The blending processor 3062-1 outputs the color information (R1, G1, B1, A1) of the image data set farthest from the viewpoint as a result of the sorting, and the display information stored in a register (not shown). A blending process is performed by performing calculations such as the following equations (1) to (3) based on the color information (Rb, Gb, Bb) of the background of the image to be created. The resulting color information (R '
Value, G ′ value, B ′ value)
Feed to 2.

[0042]

R '= R1 * A1 + (1-A1) * Rb (1) G' = G1 * A1 + (1-A1) * Gb (2) B '= B1 * A1 + (1-A1) * Bb ... (3)

As a result of the sorting, the blending processor 3062-2 outputs the color information (R4, G4, B4, A4) of the image data that is the second farthest from the viewpoint, and the calculation result ( R 'value, G' value,
B ′ value), the blending process is performed by performing calculations such as the following equations (4) to (6). The resulting color information (R ″ value, G ″ value, B ″ value) is supplied to the blending processor 3062-3.

[0044]

R ″ = R4 × A4 + (1−A4) × R ′ (4) G ″ = G4 × A4 + (1−A4) × G ′ (5) B ″ = B4 × A4 + ( 1−A4) × B ′ (6)

As a result of the sorting, the blending processor 3062-3 outputs the color information (R3, G3, B3, A3) of the image data which is determined to be the third farthest from the viewpoint, and the calculation result of the blending processor 3062-2. R '' value, G ''
Value, B ″ value), for example, by performing calculations such as the following equations (7) to (9), thereby performing α blending.
The resulting color information (R "'value, G'" value, B '"
) To the blending processor 3062-4.

[0046]

R ′ ″ = R3 × A3 + (1-A3) × R ″ (7) G ′ ″ = G3 × A3 + (1-A3) × G ″ (8) B ′ ″ = B3 × A3 + (1-A3) × B ″ (9)

As a result of the sorting, the blending processing unit 3062-4 determines the value of the color information (R2, G2, B2, A2) of the image data determined to be closest to the viewpoint, and the calculation result of the blending processing unit 3062-3 ( R '''value,G'''
Value, B ″ ′ value), for example, the following expressions (10) to (1)
Α-blending is performed by performing an operation as in 2). As a result, the final color information (Ro value, Go value,
Bo value) is derived.

[0048]

Ro = R2 × A2 + (1-A2) × R ′ ″ (10) Go = G2 × A2 + (1-A2) × G ′ ″ (11) Bo = B2 × A2 + (1- A2) × B ′ ″ (12)
Color information finally obtained (Ro value, Go value, Bo value)
Is sent to the subsequent synthesis device. In the case of the synthesizing device 121 in the last stage, an image based on the obtained color information is displayed on the display.

<Operation Form> Next, the operation form of the image processing system will be described with reference to FIG. 6, focusing on the procedure of the image processing method. When the drawing data is supplied to the rendering circuit 204 of the image generation device via the bus 205, the drawing data is supplied to the mapping processing unit 2041 of the rendering circuit 204 (Step S10)
1). The mapping processing unit 2041 performs linear interpolation, texture mapping, and the like based on the drawing data. The mapping processing unit 2041 first calculates a variation value when the polygon moves by a unit length using the coordinates of the two vertices of the polygon and the distance between the two vertices. Next, interpolation data at each pixel inside the polygon is calculated from the calculated variation value. The interpolation data has coordinates (x, y,
z, s, t, q), R value, G value, B value and A value. Next, the coordinates (s,
The texture coordinates (u, v) are calculated based on the values of (t, q). DRAM2 by texture coordinates (u, v)
049 to the color information (R value, G
Value, B value). Then, the color information (R value, G value, B value) of the read texture data is multiplied by the color information (R value, G value, B value) included in the interpolation data to generate pixel data. Pixel data is
From the mapping processing unit 2041 to the memory I / F unit 2046
Sent to The memory I / F unit 2046 compares the z coordinate of the pixel data input from the mapping processing unit 2041 with the z coordinate stored in the z buffer 2049c, and the image drawn by the pixel data is already It is determined whether the image is located on the near side (viewpoint side) of the image written in the frame buffer 2049b. When the image based on the pixel data is located in the foreground, the z buffer 2049c is added to the z coordinate of the pixel data.
Is updated. In this case, the color information (R value, G value, B value, A value) of the pixel data is stored in the frame buffer 20.
49b is drawn (step S102). The pixel data is obtained under the control of the memory I / F unit 2046.
Adjacent portions in the display area are arranged to be different DRAM modules.

In the synthesizing devices 117 to 120, the synchronizing signal generation circuit 305 generates
The external synchronization signal SYNCIN is supplied to the image generation device (steps S111 and S121).

Each of the image generating devices 101 to 101 receiving the external synchronization signal SYNCIN from the synthesizing devices 117 to 120
In 116, the color information (R value, G value, B value) drawn on the frame buffer 2049b from the CRT control unit 2047 in synchronization with the external synchronization signal SYNCIN.
A request to read the value, the A value) and the z coordinate stored in the z buffer 2049c is sent to the memory I / F unit 2044. Then, the read color information (R value, G value, B value
The image data including the (value, A value) and the z coordinate, and the write enable signal WE as a write signal are sent from the CRT control unit 2047 to the combining devices 117 to 120 connected at the subsequent stage (step S103). Image generation devices 101 to 104 are included in the synthesis device 117.
Thus, the synthesizing device 118 includes the image generating device 105
, The image data and the write enable signal WE are sent from the image generation devices 109 to 112 to the synthesis device 119 and from the image generation devices 113 to 116 to the synthesis device 120, respectively.

In each of the synthesizing devices 117 to 120, image data is written into each of the FIFOs 301 to 304 in synchronization with the write enable signal WE from the image generating device (step S112). Then, in synchronization with the internal synchronization signal Vsync generated with a delay of a predetermined period from the external synchronization signal SYNCIN,
The image data written in each of the image data 04 is read and sent to the synthesizing block 306 (step S11).
3, S114). In the synthesizing block 306 of each of the synthesizing devices 117 to 120, F is synchronized with the internal synchronization signal Vsync.
Upon receiving the image data sent from each of the IFOs 301 to 304, the z-coordinate included in the image data is compared in magnitude, and the image data is sorted based on the comparison result. As a result of the sorting, α blending of the color information (R value, G value, B value, A value) of the image data is performed in the order of being far from the viewpoint (step S115). Image data including new color information (R value, G value, B value, A value) obtained by α blending is output to the synthesizing device 121 in synchronization with the external synchronization signal SYNCIN sent from the synthesizing device 121. (Steps S116 and S122).

The synthesizing device 121 fetches image data from the synthesizing devices 117 to 120 and performs the same processing as that of the synthesizing devices 117 to 120 (step S123). The final image color and the like are determined based on the image data obtained as a result of the processing performed by the synthesizing device 121. The above processing is repeated to generate a moving image. As described above, an image that has been subjected to the transmission processing by α blending is generated.

The synthesizing block 306 includes a z-sorting unit 306
1 and the blending unit 3062, in addition to the hidden surface processing by the conventional z-buffer algorithm performed by the z-sorting unit 3061, the blending unit 30
The transmission processing by α blending performed by 62 can be performed. By performing such processing for all pixels, it becomes easy to generate a composite image in which images generated by a plurality of image generation devices are combined. This makes it possible to accurately process a complicated figure in which translucent figures are mixed. Therefore, it is possible to display a complex translucent object with high quality, and to use a game using three-dimensional computer graphics, VR
(Virtual Reality), design, and the like.

<Other Embodiments> Embodiments of the present invention are not limited to the above-described examples. In the image processing system shown in FIG. 1 described above, four image generating devices are respectively connected to four combining devices 117 to 120, and four image forming devices are further connected.
One combining device 117 to 120 to the combining device 121
However, for example, other forms as shown in FIGS. 7 to 10 are possible.

In the example of FIG. 7, a plurality of image generating devices (four in this case) are connected in parallel to one combining device 135 to obtain a final output. In the example of FIG. 8, even if the combining device 135 can connect four image generation devices, three image generation devices
This is a form in which a final output is obtained by connecting in parallel to one combining device 135. In the example of FIG. 9, four image generating devices 131 to 134 and 136 to 1 are connected to combining devices 135 and 140 to which four image generating devices can be connected.
39 fully connected, and the synthesis devices 135, 14
In this embodiment, a so-called symmetric system in which an output of 0 is input to the synthesizing device 141 is configured. The example shown in FIG. 10 is different from the example shown in FIG. 9 in that, when connecting the synthesizing devices in multiple stages, the four image generating devices can be connected to the synthesizing device 135 which can connect four image generating devices instead of the completely symmetric type as shown in FIG. In this embodiment, an output of the synthesizing device 135 and three image generating devices 136, 137, and 138 are connected to a synthesizing device 141 to which the image forming devices 131 to 134 can be connected, and further four image generating devices can be connected.

<Embodiment in Case of Using Network> The image processing system in the above embodiment is
It is formed by an image generating device and a synthesizing device that are close to each other, and is realized by connecting each device with a short transmission line or the like. Such an image processing system can be housed in one housing, for example. The image processing system of the present invention connects the image generating device and the synthesizing device via a network in addition to the case where the image generating device and the synthesizing device are installed close to each other, even when each is installed in a completely different place. However, the present invention can also be realized by mutually transmitting and receiving data. An embodiment via this network will be described below.

FIG. 11 is a diagram showing a configuration example for realizing an image processing system via a network. In order to realize an image processing system, a plurality of image generation devices 155 and a combination device 156 are connected to a switching unit 154 via a network. The image generation device 155 has the same configuration and function as the image generation device 200 shown in FIG. 2, for example. The synthesizing device 156 has, for example, a configuration and a function similar to those of the synthesizing device 300 shown in FIG. The image data generated by the plurality of image generation devices 155 is transmitted to the synthesis device 156 via the switching unit 154.
Is sent to and synthesized, to generate a synthesized image.

In this embodiment, in addition to this, a video signal input device 1 for receiving an input of image data from the outside.
50, a bus master device 151 for initializing the network and managing each component on the network, a controller 152 for deciding a connection mode between the components,
A drawing data storage unit 153 for storing drawing data is provided. These components are also connected to the switching unit 154 via the network.

At the start of the processing, the bus master device 151 acquires information on the addresses, performances, and processing contents of the components for all the components connected to the switching unit 154. Further, an address map including the obtained information is created. The created address map is
Sent to all components.

The controller 152 selects and determines components to be used when performing image processing, that is, components that constitute an image processing system via a network, based on the address map. Since the address map also includes information on the performance of the component, the component can be selected according to the processing load of the process to be executed or the content of the process. Information indicating the configuration of the image processing system is sent to all components that configure the image processing system. Thus, each component can know which component and data can be transmitted / received. The controller 15
2 can form a link with another network.

The drawing data storage unit 153 is a large-capacity storage device such as a hard disk drive, and holds drawing data to be processed by the image generation device 155. The drawing data is, for example, the video signal input device 1
The data is input from outside through the external device 50.

The switching unit 154 controls a data transmission path so that data is accurately transmitted and received between components. The data transmitted and received between the components via the switching unit 154 is data including data (for example, an address) indicating the component on the receiving side, and is preferably, for example, packet data. The switching unit 154 sends data to the component indicated by the address. The address is a unique element on the network (the bus master device 15).
1 etc.). When the network is the Internet, an IP (Internet Protocol) address can be used.

FIG. 12 shows an example of such data. Each data contains the address of the receiving component. The data “CP” represents data to be a program executed by the controller 152. Data “M0” represents data processed by the combining device 156. When there are a plurality of composite devices, for example, a number is assigned to each composite device so that the composite device can be specified. Thus, “M0” is
Represents data processed by the “0” th synthesis device.
Similarly, "M1" is the first composite device and "M2"
Indicates data to be processed by the second combining device. Data “A0” represents data processed by the image processing device 155. If there are a plurality of devices, a number is assigned to each image generation device so that the image generation device can be specified. Data “V0” represents data processed by the video input device 150. The data “SD” represents data stored in the drawing data storage unit 153. These data are sent to the receiving component individually or in combination.

A procedure up to determination of the components constituting the image processing system will be described with reference to FIG.
First, the bus master device 151 transmits data for confirming information such as processing content, processing performance, and address to all components connected to the switching unit 154. In response to the data from the bus master device 151, each component returns data including information such as processing content, processing performance, and address to the bus master device 151 (step S201). Upon receiving the reply data from each component, the bus master device 151 creates an address map for the processing content, processing performance, and address thereof (step S202). The created address map is provided to all components (step S2).
03).

The controller 152 determines constituent element candidates for executing image processing based on the address map (steps S211 and S212). Controller 152
Transmits confirmation data for confirming whether or not the process to be requested can be executed to the candidate component (step S213). The component that has received the confirmation data from the controller 152 sends data indicating that it can be executed or data indicating that it cannot be executed to the controller 152. The controller 152 analyzes the content of the data indicating that the execution is possible or the data indicating that the execution is not possible, which is sent from each component, and, based on the analysis result, the component to which the processing should be requested (the executable component that can be executed). Finally, it is determined what the component that has received the data to the effect is (step S214).
Then, the configuration of the image processing system according to the present embodiment via the network is determined by the determined combination of the components. The data representing the determined configuration of the image processing system is referred to as “configuration data”. This configuration data is provided to all of the components constituting the image processing system (step S215).

According to the above procedure, the components to be used for the image processing are determined, and the configuration as the image processing system is determined by the configuration data determined thereby. For example, in the case of a combination of 16 image generation devices 155 and 5 synthesis devices 156, an image processing system similar to that shown in FIG. 1 can be configured. Also, seven image generating devices 155 and the synthesizing device 15
In the case where two 6 are used, an image processing system as shown in FIG. 10 can be configured. As described above, the configuration of the image processing system can be freely determined according to the purpose by using various components on the network.

Next, the procedure of image processing by the image processing system of this embodiment will be described. Note that this processing procedure is basically the same as the processing procedure shown in FIG.

The image generation device 155 renders the rendering data supplied from the rendering data storage unit 152 or the rendering data generated by the rendering processor 201 of the rendering device 204 by the rendering circuit 204 to generate image data. Steps S101 and S102).

Of the synthesizing devices 156, the synthesizing device 156 for performing final image synthesizing is the external synchronizing signal SYN.
A CIN is generated, and this external synchronization signal SYNCIN is sent to the synthesis device 156 or the image generation device 155 in the preceding stage. The synthesizing device 156 that has received the external synchronization signal SYNCIN is connected to another synthesizing device 15 at the preceding stage.
6 is provided, an external synchronization signal SYNCIN is sent to the synthesizing device 156. When the image generation device 155 is provided in the preceding stage, the image generation device 15
5 is sent to the external synchronization signal SYNCIN (step S11).
1, S121).

The image generating device 155 sends the generated image data to the subsequent synthesizing device 156 in synchronization with the input external synchronization signal SYNCIN. The address of the destination synthesizing device 156 is added to the head of the image data (step S103).

The combining device 15 to which the image data has been input
6 synthesizes the input image data (step S11).
2 to S115), and sends it to the subsequent synthesizing device 156 in synchronization with the external synchronization signal SYNCIN input at the next timing (steps S122 and S116). Each synthesizing device 156 synthesizes image data and outputs it to the synthesizing device 156 at the subsequent stage. Ultimately Synthetic Device 1
The combined image data obtained at 56 is output as the entire image processing system.

It is difficult for the synthesizing device 156 to synchronously receive the image data from each of the plurality of image generating devices 155. However, as shown in FIG.
Once taken into the IFO 301, the internal synchronization signal V
By supplying the image data to the synthesizing block 306 in synchronization with the sync, the image can be completely synchronized at the time of image synthesis.
Therefore, also in the image processing system according to the present embodiment configured via a network, it becomes easy to synchronize the image data at the time of the synthesis processing.

By utilizing the fact that the controller 152 can be mutually linked to another network, another image processing system formed on another network is used as a part or all of the components. The integrated image processing system used can be realized according to the present invention. That is, the image processing system can be implemented as a “nested structure” image processing system. FIG. 14 is a diagram showing an example of the configuration of this integrated image processing system.
The portion indicated by is the image processing system itself having a controller and a plurality of image generation devices.
In such an integrated image processing system, the controller 152 communicates with the controller of another image processing system 157 to transfer image data while ensuring synchronization.

In such a nested image processing system, data to be sent to the image processing system 157 includes:
It is preferable to use packet data as shown in FIG. The image processing system determined by the controller 152 is defined as n layers, and the image processing system 157 is defined as (n-1) layers. The image processing system 157 sends and receives data to and from the n-level image processing system 157 via the image processing device 155a. Image processing device 155a
Is sent data An0 included in the data Dn.
The data An0 has a structure including the data D (n-1). The data D (n-1) included in the data An0 is sent from the image processing device 155a to the (n-1) -level image processing system 157. In this manner, data is sent from the image processing system of the nth hierarchy to the image processing system of the (n-1) th hierarchy. The image processing device in the image processing system 157 may be provided with another image processing system as the (n-2) th hierarchy. With the data structure as shown in FIG. 15, it is possible to send data from each of the components of the nth layer up to the component of the 0th layer.

Further, instead of one image generation device 155 connected to the network of FIG. 14, an image processing system (for example, the image processing system 100 illustrated in FIG. 1) that can be housed in one housing is used. It is also possible to realize an integrated image processing system. In this case, it is necessary to provide the image processing system with a network interface for connecting to a network used in the integrated image processing system.

In this embodiment, an example has been shown in which the image data generating means, the color information mixing means, and the synchronizing means are all realized in a semiconductor device. However, these means may be implemented by a general-purpose computer and a program. It can also be realized by cooperation. That is, it is also possible to adopt a mode in which the computer reads and executes the program recorded on the recording medium, and thereby the computer functions as image data generating means, color information hybridizing means, and synchronizing means. Further, some of the image data generating means, the color information mixing means, and the synchronizing means may be realized by a semiconductor chip, and the other means may be realized by cooperation of a computer and a program.

[0078]

As described above, according to the present invention,
An image for enabling a plurality of image data to be specified in the order of the depth distances included in each of the image data, and for expressing the color information of the image data having a relatively long depth distance and at least the surface portion overlapping with the color information on the upper surface of the image. Since the color information is mixed with the color information of the data, an effect is obtained that even a three-dimensional image in which translucent images are complicatedly mixed can be accurately represented.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram showing an embodiment of an image processing system according to the present invention.

FIG. 2 is a configuration diagram of an image generation device.

FIG. 3 is a block diagram illustrating a configuration example of a synthesis device according to the present invention.

4A and 4B are diagrams for explaining generation timings of an external synchronization signal and an internal synchronization signal to be supplied to a device at a preceding stage; FIG. 4A is a configuration diagram of an image generation device and a synthesis device; FIG. 4 (C) is an external synchronization signal output from the subsequent synthesis device, 4 (D) is an internal synchronization signal of the previous synthesis device, and FIG. 4 (E) is an internal synchronization signal of the previous synthesis device. This is an external synchronization signal output from the combining device.

FIG. 5 is a block diagram illustrating a configuration example of a main part of a synthesis block according to the present invention.

FIG. 6 is a diagram showing a procedure of an image processing method by the image processing system according to the present invention.

FIG. 7 is a system configuration diagram showing another embodiment of the image processing system according to the present invention.

FIG. 8 is a system configuration diagram showing another embodiment of the image processing system according to the present invention.

FIG. 9 is a system configuration diagram showing another embodiment of the image processing system according to the present invention.

FIG. 10 is a system configuration diagram showing another embodiment of the image processing system according to the present invention.

FIG. 11 is a configuration diagram for realizing an image processing system via a network.

FIG. 12 is an exemplary diagram of data transmitted and received between components.

FIG. 13 is a diagram showing a procedure up to determination of components constituting the image processing system.

FIG. 14 is another configuration diagram for realizing an image processing system via a network.

FIG. 15 is an exemplary diagram of data transmitted and received between components.

[Explanation of symbols]

100, 100A-100D Image processing system 101-116, 131-134, 136-139, 1
55, 155a, 200 Image generation device 117-121, 135, 140, 141, 156, 3
00 synthesis device 150 video signal input device 151 bus master device 152 controller 153 drawing data storage unit 154 switching unit 201 drawing processor 202 drawing memory 203 I / O interface circuit 204 rendering circuit 2041 mapping processing unit 2046 memory I / F unit 2047 CRT control Unit 2049 DRAM 2049b frame buffer 2049c z buffer 301-304 FIFO 305 synchronization signal generation circuit 306 synthesis block 3061 z sort unit 3062 blending unit

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Daisuke Hihara 1-14-10 Higashi Gotanda, Shinagawa-ku, Tokyo F-term in Sony Kihara Laboratory (reference) 5B080 AA13 DA01 FA02 FA03 FA08 FA17 GA02

Claims (17)

[Claims] 1. A plurality of image data generating means for generating image data including a depth distance from a specific part of a three-dimensional image to be processed and color information, and the image data from each of the plurality of image data generating means. A plurality of captured image data can be specified in the order of the depth distance included in each of them, and the color information of the image data having a relatively long depth distance and at least a surface part overlapping with the color information are provided on the upper surface of the image. An image processing system comprising: color information mixing means for mixing color information of image data to be expressed. 2. The color information mixing means mixes the color information of the image data having the longest depth distance and the color information of the image data having the next longest depth distance, and determines the result of the hybridization by determining that the depth distance is equal to the second distance. The image processing system according to claim 1, wherein the image processing system is further configured to be mixed with color information of the next longer image data. 3. The color information mixing unit is configured to mix color information of the image data having the longest depth distance with color information of background image data for expressing a background. 2. The image processing system according to 2. 4. The image processing system according to claim 2, wherein the image data having the longest depth distance is background image data for expressing a background. 5. The image processing system according to claim 1, wherein the color information includes a luminance value for expressing three primary colors and a transmission value for expressing semi-transparency. 6. The image processing apparatus according to claim 1, further comprising a synchronizing means for synchronizing the timing of taking in the image data from the plurality of image data generating means with the image processing timing in the own system. Image processing system. 7. A part or all of the plurality of image data generating means, the color information mixing means, and the synchronizing means are configured to include a logic circuit and a semiconductor memory, and the logic circuit and the semiconductor are provided. The image processing system according to claim 6, wherein the memory is embedded in the semiconductor chip. 8. A data capturing unit that captures the image data from each of a plurality of image data generating units that generate image data including a depth distance from a specific portion and color information of a three-dimensional image to be processed. A plurality of image data, in order to be able to specify in the order of the depth distance included in each of them, and to express the color information of the image data having a relatively long depth distance and at least the surface portion overlapping with the color information on the upper surface of the image. An image processing device comprising: a semiconductor chip mounted with a color information mixing means for mixing color information of image data. 9. The image processing device according to claim 8, further comprising a synchronizing unit for synchronizing the timing of acquiring the image data from the plurality of image data generating units with the image processing timing of the own device. 10. A frame buffer for storing image data including color information of a three-dimensional image to be processed, a z-buffer for storing a depth distance from a specific portion for each image data, and a plurality of devices including itself. The image data can be specified in the order of the depth distances contained in each of the image data, and the color information of the image data having a relatively long depth distance and at least the surface portion overlapping with the color information of the image data for expressing on the upper surface of the image. An image processing device comprising, on a semiconductor chip, means for performing communication from color information mixing means for mixing color information. 11. A method executed in an image processing system having a plurality of image data generating means and a synthesizing means connected to the image data generating means, wherein a depth of the image to be processed from a specific portion is defined. Generating image data including distance and color information by the plurality of image data generating means; and the synthesizing means fetching and fetching the image data from each of the plurality of image data generating means at a predetermined synchronization timing. A plurality of image data can be specified in the order of the depth distance included in each of the image data, and the color information of the image data having a relatively long depth distance and image data for expressing the overlapping surface portion on the upper surface of the image. Mixing the color information with the color information. 12. A computer, comprising: a plurality of image data generating means for generating image data including a depth distance and a color information from a specific part of a three-dimensional image to be processed; The image data is fetched, and the plurality of fetched image data can be specified in the order of the depth distance included in each of the image data, and the color information of the image data having a relatively long depth distance and the surface part overlapping with the color information are displayed on the upper surface of the image. A computer program for operating as an image processing system including a color information mixing unit that mixes color information of image data to be expressed with a color information. 13. A data fetching unit that fetches the image data from each of a plurality of image data generation units that generate image data including color depth and depth information of a three-dimensional image to be processed via a network. Means, and a plurality of image data captured by the data capturing means can be specified in the order of the depth distance included in each of the plurality of image data, and at least overlap with the color information of the image data having a relatively long depth distance. An image processing system comprising: color information mixing means for mixing color information of image data for expressing a surface portion on an upper surface of the image. 14. A plurality of image data generating means for generating image data including a depth distance from a specific part and color information of a three-dimensional image to be processed, and fetching image data generated by the image data generating means. A plurality of synthesizing means for synthesizing, and a control means for selecting the image data generating means and the synthesizing means necessary for processing from the plurality of image data generating means and the plurality of synthesizing means are connected via a network, At least one of the plurality of combining means captures the image data from the image data generating means selected by the control means, and enables the plurality of captured image data to be specified in the order of the depth distance included in each of the plurality of image data. At the same time, the color information of the image data with a relatively long depth distance and at least the surface part overlapping the color information are expressed on the top surface of the image. Includes color information composite unit for hybrid and color information because the image data, the image processing system. 15. At least one of the image data generating means selected by the control means includes another image data generating means connected via a network different from the network, and the image data is The image processing system according to claim 14, wherein the image processing system is configured to be generated by another image data generating unit. 16. The image processing system according to claim 15, wherein said image data is configured to include data for specifying a synthesizing unit to be imported. 17. A method for storing data for specifying an image data generating unit and a synthesizing unit selected by the control unit, and fetching image data generated by the image data generating unit specified by the stored data. 15. The image processing system according to claim 14, further comprising: means for transmitting the captured image data to the synthesizing means specified by the data.