JP2008234087A - Image display method, image display control program, and image display apparatus to which these are applied - Google Patents

Abstract

An image display method, an image display control program, and an image display device to which these can be applied, which can improve the efficiency of processing in the image display device and reflect the image blur in the depth direction of the image.
In a method of displaying a plurality of polygon data having three-dimensional space coordinates into image data having two-dimensional coordinates by using an image processor and displaying the image data drawn in a frame buffer, the image processor receives the frame. A pixel in the contour line is determined by determining a luminance difference with an adjacent pixel for each pixel of the two-dimensional image data drawn in the buffer, and a region formed by pixels arranged in a region having a luminance difference greater than or equal to a predetermined value as a contour line Different blur filter processes are executed on the pixels corresponding to the magnitude of the Z value in the depth direction from the virtual viewpoint in each of the three-dimensional space coordinates.
[Selection] Figure 3

Description

  The present invention relates to an image display method, an image display control program, and an image display apparatus to which these are applied, and in particular, optimizes anti-aliasing when displaying a plurality of polygon data having three-dimensional spatial coordinates on a two-dimensional plane. It relates to image display.

  In a personal computer or a game device, a plurality of polygon data having three-dimensional space coordinates are converted into a two-dimensional plane image and displayed on a display device.

  At this time, when the display device is a digital display device, the display screen has a pixel arrangement fixedly arranged in a matrix. Therefore, when a plurality of polygon data having three-dimensional spatial coordinates are converted into a two-dimensional plane image and displayed on the display device, the pixel position after the coordinate conversion has a decimal point, and the pixel position on the display screen is It is converted to an integer when mapping.

  As a result, jaggies (jagged edges) are conspicuous at the edges between display areas where the luminance difference is large, and if the image is a moving image, the display image flickers and the display quality of the image is degraded. .

  Therefore, an anti-aliasing process for blurring the image is performed to reduce such jaggy.

  Various conventional techniques using such anti-aliasing have been proposed (for example, Patent Documents 1 and 2). The invention described in Patent Document 1 performs anti-aliasing processing on the periphery (edge) of each polygon data at the stage of a plurality of polygon data having three-dimensional spatial coordinates.

  Furthermore, the invention described in Patent Document 2 prepares a predetermined table in which a plurality of semi-transparency coefficients are composed of gradations, and from the data part having a large influence on the luminance among the data constituting the first image, An edge is extracted, and a semi-transparency coefficient is extracted from the predetermined table using the data portion of the edge of the extracted image as an index. Then, using the extracted translucency coefficient, the first image and the second image made up of the first image are synthesized.

In this way, jaggy is reduced mainly by changing the degree of anti-aliasing in accordance with the brightness of the edge portion of the image.
Japanese Patent Laid-Open No. 7-21406 Japanese Patent No. 3529759

  Considering the inventions described in Patent Documents 1 and 2 above, in the invention described in Patent Document 1, at the stage of a plurality of polygon data having three-dimensional spatial coordinates, each polygon data is subjected to anti-peripheries (edges). An aliasing process is executed, and if the image becomes complicated, the amount of polygons becomes enormous, and the load on the processing apparatus increases.

  On the other hand, in the invention described in Patent Document 2, an edge of an image is extracted from a data portion having a large influence on luminance in an image in which a plurality of polygon data having three-dimensional space coordinates are associated with a two-dimensional screen. The degree of anti-aliasing is changed according to the brightness of the part. However, in Patent Document 2, anti-aliasing processing is performed according to the brightness of the edge portion, and image blurring in the depth direction of the image is not taken into consideration, and a problem of unnatural image display is assumed. The

  Accordingly, an object of the present invention is to provide an image display method, an image display control program, and an image display method capable of improving the efficiency of the processing in the image display device and reflecting the image blur in the depth direction of the image in view of the above-described conventional technology. An object is to provide an image display device to be applied.

The present invention that solves the above-mentioned problems has as a first aspect, the image processor converts a plurality of polygon data having three-dimensional spatial coordinates into image data of two-dimensional coordinates, and the image data drawn in the frame buffer In the display method,
In the image processor,
Determining a luminance difference with an adjacent pixel for each pixel of the two-dimensional image data drawn in the frame buffer;
A step of defining a region formed by pixels lined up in a region having a luminance difference equal to or greater than a predetermined value,
A different blur filter process is performed on the pixel corresponding to the magnitude of the Z value in the depth direction from the virtual viewpoint in the three-dimensional space coordinates of each pixel on the contour line; and
Writing the filtered two-dimensional image data back into the frame buffer;
The method includes a step of causing the display device to display the two-dimensional image data written in the frame buffer.

Also, in the above, different blur filter processes executed on the pixels corresponding to the magnitude of the Z value are:
For a pixel having a Z value that is greater than a reference + Z value by a predetermined + Z value or more, the vertical and horizontal moving average values are obtained and set as the luminance value of the pixel,
For a pixel having a Z value that is a predetermined −Z value smaller than the reference Z value by subtracting the Laplacian filter processing value from the original image, the luminance value of the pixel is obtained.
A filtering process for obtaining the horizontal moving average value for pixels in the vicinity of the reference Z value;
A filter process is executed by subtracting a Laplacian filter process value from the original image.

  Furthermore, the luminance is the R.D. G. It is obtained by Ycc conversion for multiplying the B color information by a predetermined coefficient.

  With the above feature, different filter processing is performed according to the Z value from the virtual viewpoint, so that more preferable anti-aliasing can be performed on the image.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of an image display apparatus to which an image display method achieved by executing an image display control program of the present invention is applied. As an example, description will be made based on an example in which the image display device is a game device.

  The image display apparatus has the same hardware configuration as a general game apparatus, and a main CPU 101, a system memory 102, a ROM 103, and input / output interfaces 104 to 108 are connected to a system bus 100.

  An image processing device 110 is connected to the input / output interface 104, and a display device 111 is connected to the image processing device 110.

  The input / output interface 105 is connected to a drive device 112 that writes data to a disk recording medium 113 such as a CD or DVD or reads program data.

  An input device 114 such as a keyboard, a mouse, and a pad controller is connected to the input / output interface 106.

  Further, the input / output interface 107 is a communication interface that is connected to the communication line 115 and transmits / receives signals to / from the center device when the game device is connected to another game device via a center device (not shown) to execute a network game. It is.

  The input / output interface 108 is a sound interface that outputs game sound to the sound device 116.

  The main CPU 101 reads the game program 102 </ b> A from the system memory 102, and controls overall control of devices connected to the system bus 100 and communication control through the communication interface 107 while the game is in progress.

  The image processing apparatus 110 reads the game data 102B attached to the game program 102A from the system memory 102 by the image processing CPU 110A, converts the coordinates into virtual three-dimensional space coordinates on the game, and further, the character composed of polygons according to the game program 102A. The image is converted into two-dimensional coordinates corresponding to the display screen of the display device 111.

  At this time, the texture pasted to the polygon is read from the texture memory 110B and rendered in the video memory 110C.

  The image processing CPU 110A further sends the image data drawn in the video memory 110C to the display device 111 for display.

  The feature of the present invention lies in the anti-aliasing processing in the image processing apparatus 110, and the anti-aliasing processing is performed with another buffer memory function together with the texture memory 110B and the video memory 110C.

  FIG. 2 is a diagram for explaining a buffer memory for the anti-aliasing processing in the image processing apparatus 110 and a processing procedure.

  In FIG. 2, when the execution of the game program is started, the image processing apparatus 110 converts each vertex data of polygons constituting the three-dimensional image data as a vertex buffer / index based on the game data 102B of the system memory 10 as an initial process. Write to buffer 110D.

  Each vertex data includes three-dimensional space coordinates, normal vectors, (RGB) color information, color blending information, and the like.

  In the processing of the image processing CPU 110A, first, a vertex shader program is executed to convert vertex data as 3D image data written in the vertex buffer / index buffer 110D into 2D image data (step S1).

  In the converted two-dimensional image data, the rasterizer associates each pixel of the two-dimensional image data with a coordinate position on a predetermined two-dimensional plane (step S2). At this time, as described above, jaggy occurs due to the process of rounding the decimal point to an integer.

  Next, the texture data from the texture buffer 110B is pasted for each pixel by executing the pixel shader program (step S3).

  Further, as a raster operation (step S4), the pixel to which the texture data is pasted is drawn in the frame buffer 110C while performing hidden surface processing with reference to the Z value of the vertex data (step S4). At the same time, the Z value is written into the Z buffer 110E.

  In the present invention, anti-aliasing processing is performed on the two-dimensional image data drawn in the frame buffer 110C by the above processing according to the principle of the present invention.

  FIG. 3 is a process flow of anti-aliasing according to the present invention.

  As described above with reference to FIG. 2, a plurality of polygon data having three-dimensional space coordinates are converted into two-dimensional image data and rendered in the frame buffer 110C (step S10).

  At this time, the color value and Z value of the pixel to be drawn are stored in the frame buffer 110C and the Z buffer 110E, respectively.

  Next, a contour line with a large luminance difference is extracted by filtering (step S12).

  Here, the luminance is obtained by copying the color information of the pixel held in the frame buffer 110C as a texture in the texture memory 110B, and multiplying the color index of the copied texture, that is, the values of R, G, and B, respectively by a predetermined coefficient. , And obtain them by YCC conversion to sum them

  Further, the filter processing for extracting the contour line is obtained as follows as an embodiment.

  FIG. 4 shows a specific pixel E and pixels A, B, C, and D adjacent to the specific pixel E in an oblique direction.

  A moving average value for the pixel E is obtained. The moving average value is obtained using, for example, the following matrix operator.

  That is, the moving average value Ebb of the luminance for the pixel E is obtained as follows.

Ebb = (Ea * 0.25 + Eb * 0.25 + Ec * 0.25 + Ed * 0.25) / 4
However, Ea, Eb, Ec, Ed are the luminance values of the pixels A, B, C, D, respectively.

  The moving average value Ebb for the pixel E is compared with the luminance value Eb of the pixel E itself. Then, it is determined whether or not the difference between the moving average value Ebb and the luminance value Eb of the pixel E itself is larger than the threshold value.

  FIG. 5 shows an example of a display screen 200 for a two-dimensional image drawn in the frame buffer (video memory) 110C. In this example, there are areas A, B, and C in which a high luminance area (hatched part) X and a low luminance area (non-hatched part) Y are adjacent.

  As described above, the moving average value, the luminance value of the pixel itself, and the magnitude of the difference are obtained for each pixel and compared with the threshold value. In FIG. 5, a part of the boundary of the region C where the high luminance region and the low luminance region are adjacent to each other is shown enlarged below. For pixels a1 (b1), a2 (b2), a3 (b3), a4 (b4), a5 (b5), a6 (b6), and a7 (b7) aligned along the boundary, the magnitude of the difference is greater than the threshold value Become.

  In this way, a contour line is formed by pixels arranged along the boundary.

  The present invention is characterized in that a filter process corresponding to a Z value of a pixel corresponding to the contour line obtained as described above is performed.

  Further, as a raster operation (S4), the Z value held in the Z buffer 110E to be subjected to the filtering process is obtained corresponding to the pixels of the contour line (step S11).

  Next, when the Z value corresponding to the pixel of the contour line is obtained in step S12, different filter processing is executed according to the obtained Z location (steps S13 to S18).

  FIG. 6 is an example of how to set the threshold. The Z value increases in the direction away from the virtual viewpoint O in the three-dimensional space. In FIG. 6, a reference Z value is set, and a threshold value that is greater than or equal to a predetermined + Z value and greater than or equal to a predetermined −Z value is set for the reference Z value.

  When the Z value of the pixel of the contour line is smaller than the reference Z value by −Z threshold (on the side close to the virtual viewpoint O in the three-dimensional space) (step S13, Yes), the moving average of only around the extracted contour line A filter is applied (step S14).

  The moving average filter uses the following matrix as an operator. If the median m and the surrounding value are α,

  It is represented by 8α + m = 1.0.

  Furthermore, in FIG. 3, when the Z value of the pixel of the contour line is larger than the reference Z value + Z threshold (on the far side from the virtual viewpoint in the three-dimensional space) (step average S15, Yes), the diagonal line jaggy is A convolution filter that is efficiently removed is applied (step S16).

To generate an image that removes slanted jaggy,
Image obtained by blurring oblique lines = obtained image by vertical and horizontal moving average + [k * original image−l * Laplacian].

As a specific example,
The original image

age,
Laplacian

So,
Further, a constant k is a coefficient to be applied to each element of the matrix 2, and a constant l is a coefficient to be applied to each element of the matrix 3.

m = −0.20, α = 0.15, k = 0.0, l = 0.28
Then,

  In the specific example, filter processing using the matrix 4 is performed for each target pixel. That is, the luminance of the target pixel is obtained by multiplying the luminance of the target pixel of the original image by 0.92 times, the luminance of the adjacent pixels in the vertical and horizontal directions by -0.13 times, A value obtained by dividing the luminance of the adjacent pixels in the diagonal direction of the target pixel by 0.15 times and dividing the sum by 9.

  Furthermore, between the −Z threshold value and the + Z threshold value, that is, when the brightness of the pixel of the contour line is in the vicinity of the reference threshold value Z (Yes in step S17), the moving average deduction filter (upper (step Linear interpolation is performed (S14) and a convolution filter (step S16) that efficiently removes jaggy in diagonal lines (step S18).

  The above processing is executed for all pixels (step S19).

  As described above, according to the present invention, the blur filter process is changed according to the distance from the virtual viewpoint, so that jaggy can be naturally reduced by blur corresponding to the three-dimensional space position.

1 is a block diagram illustrating a configuration example of an image display device to which an image display method achieved by executing an image display control program of the present invention is applied. FIG. 3 is a diagram for explaining a buffer memory for an anti-aliasing process and a processing procedure in the image processing apparatus 110. 4 is an anti-aliasing processing flow according to the present invention. It is a figure which shows the pixel which adjoins the specific pixel E and the horizontal, vertical direction, and diagonal direction to it. It is an example of a display screen 200 of a two-dimensional image drawn in a frame buffer (video memory) 110C. It is a figure which shows the example of the method of setting a threshold value.

Explanation of symbols

100 System bus 101 Main CPU
102 System memory 103 ROM
110 Image Processing Device 110A Image Processing CPU
110B Texture memory 110C Video memory 111 Display device

Claims (4)

A program executed in an image display system for converting a plurality of polygon data having three-dimensional space coordinates into image data of two-dimensional coordinates by an image processor and displaying the image data drawn in the frame buffer,
In the image processor,
Processing for determining a luminance difference with an adjacent pixel for each pixel of the two-dimensional image data drawn in the frame buffer;
Corresponding to the size of the Z value in the depth direction from the virtual viewpoint in each of the three-dimensional spatial coordinates of the pixels on the contour line, where the region formed by the pixels lined up in the region having a luminance difference greater than or equal to the predetermined contour A process for performing different blur filter processes on the pixels;
Processing to write back the filtered two-dimensional image data to the frame buffer;
Causing the display device to display the two-dimensional image data written in the frame buffer;
A program executed in the image display system. In claim 1,
As a different blur filter processing to be executed on the pixel corresponding to the magnitude of the Z value,
For a pixel having a Z value that is smaller than the reference Z value by a predetermined −Z value or more, the vertical and horizontal moving average values are obtained and set as the luminance value of the pixel.
For a pixel having a Z value larger than the reference Z value by a predetermined + Z value or more, the value of the Laplacian filter processing is subtracted from the original image, and the luminance value of the pixel is obtained.
A filtering process for obtaining the horizontal moving average value for pixels in the vicinity of the reference Z value;
Subtracting the Laplacian filter processing value from the original image to perform the filter processing,
A program executed in the image display system. An image display system for converting a plurality of polygon data having three-dimensional spatial coordinates into image data of two-dimensional coordinates by an image processor and displaying the image data drawn in a frame buffer,
The image processor is
Determining a luminance difference with an adjacent pixel for each pixel of the two-dimensional image data drawn in the frame buffer;
An area formed by pixels lined up in an area having a luminance difference greater than or equal to a predetermined value is defined as a contour line.
Performing different blur filter processes on the pixels corresponding to the magnitudes of the Z values in the depth direction from the virtual viewpoint in the three-dimensional space coordinates of the pixels on the contour line,
Writing the filtered 2D image data back into the frame buffer;
Control to display the two-dimensional image data written in the frame buffer on a display device;
An image display system characterized by that. In claim 3,
As a different blur filter processing to be executed on the pixel corresponding to the magnitude of the Z value,
For a pixel having a Z value that is smaller than the reference Z value by a predetermined −Z value or more, the vertical and horizontal moving average values are obtained and set as the luminance value of the pixel.
For a pixel having a Z value larger than the reference Z value by a predetermined + Z value or more, the value of the Laplacian filter processing is subtracted from the original image, and the luminance value of the pixel is obtained.
A filtering process for obtaining the horizontal moving average value for pixels in the vicinity of the reference Z value;
Subtract the value of Laplacian filter processing from the original image and execute the filter processing.
An image display system characterized by that.

Images