NO303199B1 - graphics System - Google Patents

Description

The present invention relates to computer-controlled image formation in general and in particular techniques for anti-aliasing computer generated images. The invention is described using an anti-aliasing filter at the output of a memory unit for a sub-pixel image to improve the image quality of screens in general and color mosaic screens in particular. The result is smoother and more line-like images.

More specifically, the invention relates to a graphics system that implements a filter to counteract image noise in order to improve image quality on raster screen displays.

For elucidation of the known technique, particular reference should be made to "Dejagging raster graphies displays by flash filtering" by David Oakley and Donald I. Parsons, Proceedings of the SID, Vo. 28/4, 1987, pages 425-429.

The publication describes an algorithm for rapidly reducing image noise on computer graphics displays. The algorithm involves image data being sent from a pixel generator (DVG, fig. 2) to an image memory (FBM). The pixel generator emits both integer (x,y) address data for addressing the image memory, and sub-pixel or fractional sub-address data (u,v). The image memory comprises storage for integer and partial pixel or fractional part address data received from the pixel generator, and an output device for reading out partial pixel or fractional part data. Subpixel or fractional subaddress data (u,v) from the image memory controls a digital filter so that each active pixel of the image data from the image memory is expanded, and high resolution is achieved.

In particular, the publication relates to the generation of video signals for a cathode ray tube, where the known system can rely on the electron beam to intensify several color spots with overlapping Gaussian profiles (which are natural for spots formed on a CRT projector) to form a continuous line. However, this function must be emulated for correct operation in conjunction with active-matrix type displays. Moreover, the known system separately performs vertical and horizontal micro-positioning and then relies on the analog response available in the CRT display.

Incidentally, reference is also made to US patents 4843380 and 4939671.

The problem you face when it comes to flat panel-shaped matrix screens and/or raster screens is that certain characteristics of some screen systems result in an output that is incoherent or has a jagged appearance such as e.g. the aliased line 10 on a simulated screen 11 in fig. 1. This is in contrast to the uniformly accepted high quality line 12 that one would see on a picture tube screen where each line is smooth and continuous (ie an anti-aliased line). Without anti-aliasing, computer-generated images on raster screens will typically show disturbing irregularities such as: bubble patterns, rainbow effects, jagged edges, incoherent movements, loss of detail and/or glitter. These irregularities are all confirmations of a more general problem called aliasing. Aliasing occurs when the detail or spatial frequency content of the image to be rendered exceeds the bandwidth capacity of the selection in the display system. Fortunately, according to selection theory, aliasing can be completely canceled out. To remove this, the signal that is subjected to selection must first be bandwidth limited so as not to contain frequencies that are higher than the system's selection frequency. In order to prevent the irregularities from appearing in an image, the image itself must first be band-limited so that its spatial frequency content satisfies this requirement.

In computer-generated image systems, the image can be expressed as a signal whose amplitude varies as a function of the position across the screen's image surface. When more details are introduced into the image, this signal will change more quickly across the screen. When the signal rises and falls faster, the effect will be to raise the spectral energy content in the image to higher spatial frequency ranges. Therefore, band-limiting a computer-generated image means that the computer must be programmed to prevent sudden transitions. Pixels in the area of details, lines or edges of polygons, for example, are shaded so that the signal rises and falls at the right speed. The right speed is determined by applying the selection theory as mentioned above.

In the past, attempts have been made in connection with the central processing unit to mathematically determine the correct strength for each pixel in the screen. A weighted filter/convolution was introduced over a window area and the result used as an index for the correct value of the strength. Some implementations of hardware imprint a strength value on pixels on either side of the desired pixel. This method is usually chosen when a vector generator is used to render the graphic symbols.

It is a well-known technique to use a write cycle architecture, i.e. band limitation in the data generated image while the image is written into the memory. Many different algorithms have been developed over the years to achieve anti-aliasing. One of the more common techniques was based on how far pixels were from a given detail. For each neighboring pixel, the distance from the detail, e.g. an edge, determined. The further the pixel was from the edge, the less influence the edge had on the strength assigned to this pixel. Smoother transitions in the light density were the result at the same time that the spectral content of the image was suitably limited to lower frequencies and the aliasing irregularities were eliminated.

Another popular approach that is theoretically more accurate assigns brightness or color values to neighboring pixels as a function of the overlap area. The more a pixel was covered by a particular detail, the more impact the detail exerted over the pixel. For example a pixel that was completely covered by a polygon will be assigned the full value required by the polygon. If, on the other hand, the pixel was on the edge of the polygon so that only one third of the pixel was covered, the pixel would be illuminated with only one third of the value prescribed by the polygon. This procedure also led to an effective low-pass filter operation over the image and limited the spatial frequency content as prescribed. Anti-aliasing was also successfully achieved here.

As an alternative and more accurate for non-real-time applications, the method with filter loops has been used. The bandwidth of an image was reduced by running a windowing kernel over an output image whose resolution was higher than in the desired output. A three by three processing core was chosen in typical examples. During processing, the number of kernels corresponding to the spread function for a specific point was stored in a memory. The transfer function and the shape of the point spread function were derived from selection theory. The kernel was passed across and down the output bid, pixel by pixel, resulting in a straightforward convolution process. The convolution with this kernel and a high-resolution output image resulted in an output image whose spatial bandwidth matched the selecting display system, so that aliasing was once again avoided.

The above three categories of anti-aliasing are all based on a write cycle. Because of. this has serious disadvantages when there is a need to work in real time with high resolution. Speed is the critical factor. The increased burden antialiasing imposes on systems even with modest resolutions, e.g. 512 x 512, has pushed the write cycle architectures almost to the limit. The processing elements must be selected for work at much higher levels with the consequent higher costs. Of course, parallel techniques can be introduced to eliminate some of the problems, but control and the flexibility of the system flare up to become the main objections.

The problems with write cycle architecture become even more prominent when color mosaic display technology is used in the display device. The resolution when developing this technology can be many times higher than what has already proven to be difficult. The direction of development points towards even higher resolutions. In addition, the specific color pattern in the mosaic must be taken into account by the graphics generator in order for it to be able to provide successful anti-aliased imaging. Color pattern considerations enable the image generator to shape the frequency content of the image according to specific capabilities of a given color mosaic display. Summarizing this background, anti-aliasing with a write cycle will entail significant costs when producing high-resolution real-time images, especially when aiming at color mosaic screens.

The present invention results in effective anti-aliasing effect by approaching the problem from a different starting point than that mentioned in connection with the state of the art, namely the system level where instead of imposing on the computer the constant task of filtering the image when it exists details in this, the invention will allocate as much as possible of the excess activity to a new component 23 on the output side or, as is usual, the read side of the image memory (fig. 3). In the present invention, each active pixel in the image memory is expanded into a Gaussian profile in a beam former 23 (ie the new component). This is shown pictorially at 13 in fig. 2 which reproduces this expansion.

The system's new component is called a beam former. It has the additional task of performing the anti-aliasing and acting as an adaptation filter between the graphics generator and the display device 22. It limits the bandwidth of the image in accordance with the sampling theory of using sub-pixel information as a means of increasing the apparent sampling frequency of the system, particularly when it comes to screen- the head if color mosaic screens are used. In this new system, the beam former is located between the display device and the computer (Fig. 3b). The most prominent features of the beamformer are: a) It expands impulse data from the image memory 21a, into the point spread function 13, the bandwidth of which lies within the limits of the selection system which may be the screen 22; and b) It uses subpixel information from the graphics generator system 24 to adjust the luminance centroid for

the dot spread function 13, so that it acts between the selection points which can be the ones you have in the screen 22, so that you achieve better addressing possibilities when it comes to the position of the points.

In the computer, only a few function modules need to be modified to be able to work in this new system. In fig. 3b, the first function module, namely the graphics generator 20a, must output partial pixel information in addition to the standard address output with full position numbers. Subpixel information is usually available in this area, but is often used outside of the graphics generator.

The second module, the image memory 21a, must be modified to include a store for the new sub-pixel information emitted by the graphics generator. The image memory must provide this storage in addition to constituting the usual storage for color information and image attribute information. The invention therefore includes the new beam-forming component 23, in addition to these modified components 20a and 21a in the computer.

The advantages of this invention include:

significantly improved treatment review, the system's simple structure and lower costs, especially when it comes to technology for color mosaic screens. It is a further advantage of this invention that it is effectively anti-aliasing for color mosaic displays. In addition, the invention offers a large degree of independence of the graphics processor from the resolution and pixel arrangement in the display technology.

In this way, the beamformer 23 works with data coming from the image memory and produces properly anti-aliased data that is displayed on the screen. It looks at a window of data from the image memory and calculates the correct strength value to display. It moves the function of anti-aliasing from the graphics generator to a separate dedicated hardware section. The beam generator solves several problems that are encountered in the usual system. The graphics generator does not have to do any anti-aliasing work and thus the system's throughput capacity is increased. In addition, the beamformer does all the processing associated with color mosaic screens, which in turn increases the system's throughput by reducing the workload for the graphics generator.

The invention is characterized by the claims in reproduced features and will be explained in more detail below with reference to the drawings therein: Fig 1 shows the difference between the shape of an aliased line and an anti-aliased line. The aliased line is jagged while the anti-aliased line is smooth and continuous. Fig. 2 shows the image-wise input/output ratio for the beam-forming component. The input is impulse data from the graphics generator system. The output is the point spread function from the beamformer combined with the pulse input. The result is centering of the sub-pixel, which leads to higher precision in the placement of color energy on the screen. Fig. 3a shows a common image processing architecture using a write-cycle anti-aliasing method. Anti-aliasing is performed by the graphics processor when the image is drawn and written to the image memory. Fig. 3b shows the invention where it drives a screen. The beamforming module has been added to a modified system which is now an anti-aliasing system in the read cycle. Fig. 4 shows the idea of sub-pixel position in a pixel in the image memory. A sub-pixel position is marked as an example of where in a pixel the invention can be instructed to place the light density centroid. Fig. 5 shows the modified form of graphics processor used in the invention, instead of giving standard whole numbers x and y as address values for addressing the image memory, the modified vector generator also provides partial pixels or fractional address data. Fig. 5a shows the modified form of the image memory. Store for color or attribute data, etc. 71 is shown similarly to store for sub-pixel address information 72, which is obtained from the modified vector generator. Fig. 6 shows an example of a movable window that is used for beamforming treatment and shows how the beamforming works. It receives a data signal from the image memory and performs appropriate processing to produce a single output pixel. The sliding window is moved from left to right on each line and from top to bottom. Fig. 7 shows a block diagram of the internal details of the beam shaper. First, color processing is performed to sort data according to the next output pixel (for color mosaic displays only). Second, a shadow comparison is performed for each active pixel in the window based on a Gaussian profile. Third, all shadows are compared to determine the maximum shadow. Finally, this shadow is passed as output to the display device. Fig. 8 shows an example of color processing. A sliding window is shown in an image memory containing a yellow (red and green) line and a red line. Fig. 9 is an example of a color processing for color sorting performed for a green next output pixel. Fig. 10 is an example of a color processing for color sorting carried out for a red next output pixel. Fig. 11 shows a Gaussian profile in a sliding window. This example of Gaussian profile in moving window is used for shadow comparison values. Fig. 12-17 are examples of sliding windows where one point is shown in the image memory that passes through a sliding window, and represents 6 consecutive time frames. Fig. 18-26 are examples of sliding windows corresponding to fig. 12-17 and shows three points in the image memory close to each other. The 9 figures represent the position of the sliding window for 9 consecutive time frames. Fig. 27 is a Gaussian profile based on 8 shades of gray. A luminance profile is shown for a 0.5 mm wide half-amplitude line drawn in 8 shades of grey. The pixel spacing is assumed to be 0.17 mm. Fig. 28 is a Gaussian profile based on 6 shades of gray. A light density profile is shown for a 0.5 mm wide half-amplitude line drawn with 6 shades of grey. The pixel spacing is assumed to be 0.17 mm. Fig. 29 is a Gaussian profile based on 4 shades of grey. A light density profile is shown for a 0.5 mm wide half-amplitude line drawn with 4 shades of grey. The pixel spacing is assumed to be 0.17 mm.

Fig. 30 shows a vector diagram in an image memory.

Fig. 31 shows a block diagram of a vector generator and fig. 32 shows a block diagram of a vector generator used in conjunction with a beamformer. Fig. 33 shows a vector which is suitable when using the modified vector generator of fig. 32. Fig. 34 shows an embodiment of a double image memory pattern (ping-pong pattern) and fig. 35 shows a modified embodiment of a ping or pong part of the image memory. Fig. 36 shows an embodiment of a block diagram of data paths for processing a 3 x 3 core to produce color point outputs.

In fig. 3A shows a conventional screen system with a graphics generator 20, an image memory 21 and a matrix screen 22. Fig. 3B shows a preferred embodiment of the invention. It includes the modified graphics generator 20a, the modified image memory 21a, and the beam former filter element 23 together with the screen 22. In fig. 5, the modified graphics generator 20a is shown. Instead of simply outputting the standard integer x and y address values 50 for addressing the image memory, the modified graphics generator also outputs subpixel data or fractional address data 51. This subpixel or fractional address information 51 can be used to direct a subsequent system element, namely the beamformer 23, to adjust light energy to areas between separate pixels. Considered with the selection theory as a background, the sub-pixel information provides additional information about the selection point to be written into the image memory. It also provides the phase for the selected amplitude data. Fig. 4 shows that the sub-pixel information can be used to place light energy in any of 16 positions, e.g. within a single pixel 31, in the image memory 32. An enlarged rendering 30 of the pixel 31 shows the sub-pixels that can be made available to the display system with greater precision in the location of the light energy that will form the image. Subpixel 33 e.g. can be designed by the graphics generator and stored in the modified image memory.

In fig. 5a, the modified form of the image memory 21a is shown. Instead of storing only color or attribute data, etc. 71, the modified image memory is capable of also storing the subpixel address information 72, which is obtained from the modified graphics generator. When read out, this image memory will later carry standard video information together with the sub-pixel information of the beamformer.

Reference should again be made to fig. 2 or 3b where the beam former 23 is shown. This input/output ratio shows that it behaves very similar to a standard two-dimensional image processing filter. A pulse from the image memory applied to the input of the beamformer arrives at the output of the beamformer in an expanded state. The expansion is the point spread function that the beamformer has corresponding to this filter's characteristics in terms of spatial frequency transmission. In other words, each active pixel in the image memory 21a is expanded into a Gaussian profile in the beamformer 23. The data that is scanned out of the image memory 21a is collected in the beamformer 23 and interpreted to determine the correct strength level for each pixel.

There are at least four prominent differences between the beamformer and previously known two-dimensional image-processing filters:

1) the beamformer extends the filter technology of two-dimensional imaging to also solve the problem of anti-aliasing raster images in its generality; 2) solving the problem of anti-aliasing color mosaic displays in particular; 3) applying the point spread function derived for the given pixel pattern in a color mosaic; and 4) the centroid of the point spread function is microplaceable under the direction of the input subpixel information. (This means that the filter coefficients are a function of input data, different from standard filter usage.)

Another important feature of this invention which consists of the modified graphics generator, the modified image memory and the beamformer is its reshaping capability during scanning. Because the beam former 23 constitutes an interface between the computer and the screen, the computer's drawing elements, i.e. the graphics processor 20a and the image memory 21a, can be constructed so that they are independent of the resolution and pixel arrangement in the screen. One-to-one correspondence between image memory storage locations and display pixels is not required. The beamformer provides an interpolation function between the computer and any raster display technology. This is particularly useful when it comes to color mosaic displays where the resolution and pixel arrangement can change from display head to display head and while this display technology is still under development. The advantage of the system is that the computer is stabilized and independent of such developments or changes. Another advantage is that the graphics processing can be performed at a lower resolution, which leads to an even greater increase in the effective performance than that achieved by relieving the anti-aliasing burden alone. Furthermore, read/modify/write cycles are not necessary as is the case with prior art equipment, since anti-aliasing data is added to the output side of the image memory and therefore cannot erase underlying or previously drawn/written anti-aliased information. This results in a further increase in the effective process performance.

The beam former uses the information about the position of the sub-pixels from the image memory to more precisely position the light energy on the screen. It can place the light energy even more accurately by adjusting the light density centroid for the point spread function that is the output. As the point spread function covers a collection of pixels, the amount of energy from the pixels can be adjusted in relation to each other, and by doing this you can make the energy appear to come from areas between pixels. In the preferred embodiment, the beam former places light energy within one-third of an arcminute of what is seen with the eye. In a typical case, this applies to one four-thousandth of the screen's width/height in terms of placement accuracy. You get options for fine-tuning that can be used to eliminate the selection artefacts in the display system.

Using the beamformer also eliminates the need for the graphics generator to know the color arrangement of a color mosaic display, as is the case with liquid crystal display (LCD) color matrix technology. Each pixel in the image memory can be processed independently of the pixel arrangement in the LCD. Color can be written into the image memory without regard to the screen's pixel arrangement as it could before, but only with mosaic screen technology without colors in the same way as screen technology with cathode ray tubes. The beamformer may include a color sequencer and comparison mechanism for dividing the color from the image memory into primary color components comprising color mosaic patterns in a particular display. In this way, the different pixel patterns can be easily controlled so that the technology for color mosaic screens becomes as convenient to handle as cathode ray tube technology. Typical primary color patterns used in color mosaic displays include the vertical RGB strip, the diagonal RGB strip, delta or hexagon patterns RGB, Quad-RGBG, Honeywell Ouad-RGBY and Quad-RGBW, where RGB stands for red, green and blue , and where Y and W denote yellow and white.

The beamformer 23 creates anti-aliasing strength levels for each primary array. E.g. in a raster screen on a cathode ray tube, the strength levels of the red, green and blue channels will be interpreted separately independently of the other color channels. Color continuity is thus obtained by using a beam former.

Fig. 4 shows an enlarged rendering 30 of one pixel 31 in the image memory 32, where the addressability of each sub-pixel can be used to improve the apparent resolution. 16 sub-pixels per pixel is shown in fig. 4 as an example. The addressability of the sub-pixel enables micro-positioning of the light density profile, which e.g. at 33 within the enlarged pixel 30. The line shape and location thus become more accurate. The sub-pixel information is stored in the image memory 21a and is used by the path former 23 for fine tuning of the Gaussian beam center. This increases the effective resolution in the display device 22, as the number of bits of sub-pixel addressability is increased. A 512 x 512 image memory with three bits of x and y subaddressability can accommodate a pixel with an effective resolution of a 4k x 4k image memory.

The path shaper 23 forms a new system for using a fixed pattern in the vector generator 20a and the image memory 21a, with changes only required in the beam shaper to drive different display devices. E.g. a change from an RGB diagonal arrangement on an LCD screen to an RGBG quad arrangement would only require a change in the color coding scheme of the path generator 23 without changes being necessary in the vector generator 20a. The vector generator extracts its image with the understanding that each pixel can be any color. The pixel color arrangement is completely determined by the beam former 23 which in turn reads the pixel arrangement from a small PROM unit in the preferred embodiment. The same vector generator system can drive display devices of different sizes without requiring any change in either hardware or software. It is only the beam former that is changed when the screen devices are changed.

The beamformer is a device intended to be placed between the image memory and a raster display to provide anti-aliasing for the display device. The beamformer contains a memory for forming a window of data. The working theory behind the beamformer is to examine a data window in the image memory centered around the next output pixel and to determine the correct power level for the output pixel based on data in the moving window.

Reference should now be made to fig. 6 where one finds an example of a moving window used for the beamformer's processing and the resulting pixel displayed on the screen. This figure shows an example of how the beam former works. It takes a window of data from the image memory and performs appropriate processing to produce a single output pixel. The movable window is moved from left to right on each line and from top to bottom in the same way as a typical raster screen device.

In a typical example, the beamformer contains several line memories which give access to a window of data around the next output pixel. The size of the window is determined by the geometry of the pixel pattern of the display device and the desired line width on the screen. In the examples, a 6 x 6 window will be displayed.

The first step in the beamformer's operation is to sort the active elements in the window by color. In fig. 7 shows a block diagram of the beam former's components. The beam generator 23 comprises a component 50 with strength selection from color bits, a shade comparison component 51, a final maximum shade component 52, and a movement component 53 for the window. Block 50 is dedicated to color processing. The first color processing involves sorting data according to the next output pixel (only for color mosaic displays). A shade comparison is then made for each active pixel in the window based on a Gaussian profile. Third, all shades are compared to determine the maximum shade. This maximum shade becomes the output of the screen. Fig. 8 shows an example of the contents of a movable window before the color processing, as the movable window overlaps an image in the image memory containing a yellow (green and red) line and a red line. Fig. 9 shows an example of the contents of the scrollable window after color processing for a green next output pixel, and Fig. 10 shows an example of the moving window after color processing for a red next output pixel. For picture tubes with rasters where each pixel contains each color, processing can be done in parallel with red in one channel, blue in another, and green in a third channel.

After the color sorting has occurred, a comparison table method is used to determine strength based on distance from the center of the window to determine the strength for each active pixel in the sliding window. A Gaussian profile adapted to the display device is used to determine the strength for each active pixel.

In fig. 11 shows an example of a Gaussian profile within the moving window. Each ring 60, 61, 62, 63, 64, 65 and 66 in the profile represents a strength level in the same way that a topographic map represents elevation data with elevations. A power level is calculated for each active pixel in the sliding window based on this Gaussian profile. Sub-pixel data is used to determine the strength level. Sub-pixel data is used to compare the strength in the main blocks shown in fig. 11. The next step in the processing is to determine the maximum strength level for all the active pixels. This is done using comparators or another comparison table. The maximum strength level is used as the starting strength level.

Fig. 12-17 is considered a group of time frames. Fig. 12 shows a single data point 70 in the image memory as it is to be filtered on one line 83 in the movable window 72. No other points will have any contribution to the image. The thick lines 80 to 86 and 90 to 96 represent pixel boundaries, in this case for a 6 x 6 window of pixels. The thin lines such as 87 and 97 represent sub-pixel addresses, 16 per pixel in the example of fig. 12. Circles 60-66 represent limits for each strength level. In fig. 12-17, the window 72 is shifted one pixel to the right with each successive frame, so that an apparent movement to the left of the one point 70 occurs as the group of frames is examined in sequence. The six consecutive frames in fig. 12-17 and the strength and output strength this one data point 70 contributes to, is shown in the table below:

The sub-pixel address is used to determine the strength value to be used. The strength is based on the radial position, very similar to a typographic map. The inner circle represents strength level 7 and drops one level level per outline to 0 outside the outer circle.

While fig. 12-17 showed an example of a single data point in the image memory as the point is filtered on the line for the moving window, it should now be shown to fig. 18-26 where one finds an example of several data points in the image memory close to each other as they are filtered through the sliding window by a simple passage over the screen. The nine figures represent the moving window for nine consecutive time frames. First a strength comparison is made for each point, then a comparison is made to find the maximum strength. In this case, the maximum strength determined by comparing all active points in the window is used as value output to the screen. Table 2 below summarizes three points 97, 98 and 99 in the image memory close to each other as they are filtered through the sliding window on a single line.

The beam former 23 can be used to produce anti-aliased data for lines with different light profiles. This is a change in the Gaussian profile. Several light profiles can be used in one beamformer alone. Processing of multiple light levels differs only in the original Gaussian profile used to determine the strength level for each active pixel. A comparison is made for each pixel based on subpixel location and strength level setting. Then all the strength levels are compared to find the maximum to be used as output. Different light levels are discussed in the comparison table. Fig. 27, 28 and 29 show three different brightness profiles. A number of these will in a typical example be incorporated into the beam former for anti-aliasing of several light levels.

In this invention, a vector generator is a special purpose graphics generator. Digital vector generators typically draw lines and shapes by incrementally calculating addresses for pixels, and then command a write operation for each of those pixels. The general equation (yl-yO) = m(xl-xO) is solved and used as a basis for calculating the address for each pixel. This equation breaks down into the parametric equivalents which

xA= x0+ dx<*>i and yA= yQ+ dy<*>i

where m = dy/dx and in typical cases either dy or dx is set equal to 1.

Fig. 30 shows a vector drawn in a normal image memory from (0,0) to (7,5). The vector generator solves the basic equation to get a value for m. Since delta Y is less than delta X, this expression is used to increment the Y address as the vector generator moves in steps along the line. The X address is incremented by 1 each time. (If delta Y > delta X, Y is incremented by 1 and X is incremented by l/m.) Fig. 31 shows a functional block diagram of a typical vector generator for the X address. The starting point is first entered in the accumulator. At each clock cycle, the accumulator will add the previous accumulator output (stored in the latch) by the increment specified by delta.

The output of the latch is fed to a rounding circuit to produce the X address of the next pixel. The accumulator has sufficient accuracy to handle fractions. The rounding circuit rounds the numbers to whole values required by the pixel addressing circuit (eg 27.862 is rounded to 28). An identical circuit is used to generate the Y address. For fig. 30, the output values and increments for X and Y will be:

A vector diagram used with the beamformer takes advantage of the fraction of information available from the accumulator. Fig. 32 shows a block diagram for a vector generator used with a beam former. It corresponds to what is shown in fig. 31 with two differences. The rounding circuit used in fig. 31, is replaced by a clipping circuit for generating the pixel address. The latch output is fed to a new block which extracts the fractional parts of the pixel address. This fractional part address information is stored in the pixel as subpixel data along with the normal color and strength data field. Subpixel data is stored in each pixel. As an example, 27.862 would be truncated to 27 for the integer address and with 0.862 stored in the pixel as subpixel data. Fig. 33 shows a vector drawn using the modified vector generator. By storing sub-pixel data in each individual pixel, each pixel can be positioned very precisely. Fig. 33 shows 16 sub-pixel areas per pixel. A comparison of fig. 33 with fig. 30 shows how subpixel data places the line center more accurately. (Fig. 33 is twice as large as Fig. 30.)

In raster scanning screen devices, the electron beam will run across the screen in an orderly manner, most often from left to right and from top to bottom. Because the display devices use fast-reacting phosphorescent substances to reproduce movement on the screen, the images produced will disappear immediately after they have been activated and there is therefore a need to refresh these at regular intervals, in typical cases at a frequency of 60 Hz.

In computer graphics systems, the image is refreshed using a special refresh buffer, an image memory, which in a typical example can use solid state direct storage technology. Each storage point in the image memory is in one-to-one correspondence with each point on the screen. The image memory can then be visualized as a two-dimensional set of storage points addressed with an x-address and a y-address. These addresses are produced synchronously with the scanning movement of the electron beam. The x-address will usually be incremented from 0 to M-l, where M is the number of horizontal picture elements across the screen. When the value M-1 is reached, the x address value is reset to 0 and the y address is incremented. Y can start at 0 at the top of the screen and end at N-l, where N is the number of rows down the screen. M and N are often in the 512 range for cathode ray tube raster displays, but are in the 1280 range for color mosaic displays.

Each storage point contains data that is used to activate the electron guns in the cathode ray tube to some activation level that corresponds to the desired brightness on the screen's surface. Data is usually 8 bits for 256 gray levels per red, green or blue cannon. In this way, each memory point can be 24 bits deep. In color systems for airborne electronics, the number of bits is significantly lower, typically 3 bits to specify any of 8 colors that can be displayed on the screen at the same time. These three bits are passed through a comparison table which divides the designated colors into the primary color components which are the constituents. The ratio between red, green and blue can be selected with 8-bit precision, giving a very wide color palette from which the 8 colors can be selected.

Image memories were previously implemented with static or dynamic direct storage devices, the bandwidths of which were amplified to video rates with subsequent shift registers. A word with a width of 16 pixels would be read into a shift register 16 such as is 16 units (bins) wide and then clocked out at the full speed of the shift register which is far higher than the availability time of direct storage. Today, these proprietary methods from the old days are packaged into a single integrated circuit called Video RAM or VRAM. These chips are very effective in building up image memories and are used in the preferred design. Individually, they can offer sufficient storage options to cover the entire screen for many applications.

The shift register enables access to memory from the computer while data is being read out to the cathode ray tube or display device. This enables an interlacing of read and write cycles. The image can be drawn while it is displayed. However, in real-time applications where a large amount of movement is expected, updating the image by refreshing the screen can lead to visual artifacts. Old and new data can be mixed together on the screen and create gaps in the image and strange floating frequencies in it. To avoid these problems, a structure with double image memory is useful (fig. 34). With this structure, one memory buffer is used to refresh the screen, while another complete image memory buffer is entirely devoted to the graphics generator for image rendering. When the generator has finished drawing the image, it notifies the hardware synchronization circuits. These circuits swap the buffers during the vertical rollback period when the video signal is erased from display. The result is that complete image frames can be rendered at any time without visible breaks in the image content.

When practicing the invention, any of the classic image memory patterns can be used. It does not matter whether dynamic memory chips or direct memory chips are used. What is different is the storage depth and allocation. Storage options exist for subpixel data in addition to the usual color, priority and other attributes. The modified image memory is still available to the graphics generator and is still used to refresh the display device. An inexpensive read/write update device for interleaved image information can be used, or alternatively a dual structure can be chosen. In the preferred embodiment (fig. 34) where VRAM is used, the image memory is 512 x 512 pixels in a double embodiment. Each pixel is 12 bits deep, 6 bits for color and another six bits for storing sub-pixel data, 3 bits for x-sub-addressing and 3 bits for y-sub-addressing. This data, when stored in this way, can be read from the modified image memory for processing in the beam forming device. Thus, this is a reading cycle-oriented method or process. Fig. 35 shows a modified part of the double image memory.

Fig. 34 and 35 are a reproduction of this embodiment. For more information on image memory technology, reference may be made to the book "Fundamentals of Interactive Computer Graphics" by J.D. Foley and A. Van Dam, from July 1984, pp. 129-135.

The block diagram in fig. 36 shows the data paths for processing a 3 x 3 core to produce the color point outputs for an RGGB Quad flat panel. The center pixel in the 3 x 3 matrix is expanded to 4 outputs, one output for each of the color points in the Ouad implementation. This allows the 512 x 512 full-field memory (FFM, discussed in the previous section) to be expanded to drive a 1024 x 1024 quad green flat panel display. This block diagram can be easily extended further to other sizes for processing cores or it can be built in another way to drive other pixel patterns. The requirement for a 3 x 3 core means that pixel data from the FFM must be scanned so that the FFM output is always 1 row and 1 pixel ahead of the pixel address currently being processed. The 3x3 processing of the kernel requires that 9 entries exist for each pixel. This is achieved with the two 512 x 12 bit FIFOs which delay the pixels by one and two lines respectively. For each line there are three 12-bit registers which are filled in order. Thus, the new pixel from each line will be entered into the first register, while the second register receives the previous content of the first register, and the third register is filled with a previous content of the second register. Thus, the 9 registers at their output have the contents for the 3x3 matrix for processing a given pixel. The upper 3 registers hold 3 pixels from the next data line, the middle 3 registers contain 3 pixels from the currently running data line, and the lower 3 registers contain 3 pixels from the previous data line. It must be remembered here that the image memory is always one data line ahead of the processing of the core.

The 12-bit data for each pixel can contain any combination of color, strength, and subpixel addressing. The preferred combination is 6 bits of color codes, 3 bits for x-part addressing and 3 bits for y-part addressing. This results in beamformers creating 64 color/intensity combinations and the equivalent of 4096 x 4096 addressing from the 512 x 512 FFM unit. Other desirable bit assignments are 4 bits for color, 4 bits for strength, and 2 bits for each of the x and y subpixel addresses. This embodiment of the beamformer can drive two sets of bit assignments with one mode bit.

The left set of 8k x 16 bit memories provides the original conversion of pixel data in the outputs with RGGB components. This memory requires 13 bits for addressing: 12 bits from pixel data and one mode bit. The memories can be based on either random access memory (RAM) or a programmable read-only memory

(PROM). The contents of the memories are filled with color point strengths corresponding to the position in the processing window for data belonging to this pixel in each of the 9 matrix positions. Thus, in a typical example, each memory will have different contents. The upper memory can e.g. contain the transform for the pixel that is one line ahead in y and 1 pixel ahead in x. As previously discussed, the pixel position and sub-pixel addressing will compensate for the distance between the pixel and the center of each of the color points when calculating the color contribution this pixel has in the final output for each color point. The 16-bit output of this first set of memories provides 4 bits of strength contribution for each of the color points. With a color code for black, all color point outputs will have a strength of 0, while with a color code for white, each color point contribution will vary depending on the distance to the center of the color point produced. It should be noted that the 2 green outputs will often be different, since the distance from these two points will normally be different in relation to the center of the window for the 3x3 processing. The distance discussed here is from the center of the individual color points produced to the sub-pixel location in the pixel being processed (a total of 9 pixels are processed in parallel to produce 4 color points).

The 16-bit outputs from the first set of memories are divided into 4 units of 4 bits (R, Gl, G2 and B). The second set of memories in the diagram ensures mixing of strength data for each of the three rows so that each memory outputs the strength contribution for this row for each of the four color points. This second set of memories in this embodiment is each 8k x 8 bits, where 13 address bits are composed of 3 strength units of 4 bits and one mode bit. The mode bit ensures that two anti-aliasing strength mixing algorithms exist. The PROM can either select the maximum strength, sum the contributions linearly, or even perform non-linear summation of the strength components. Any mixing function can be introduced into these memories. Each of the three rows contains 4 such memories. Each

memory receives 3 strength units of 4 bits from the first set of memories for a particular color point. Thus, each row will output its strength contribution for each of the 4 color points. Note that only 4 bits from each 8-bit memory are needed (8k x 4-bit memories are not readily available).

The third set of memories combines the outputs from each of the 3 rows to produce the final output for each color point. Each memory receives the 3 four-bit strengths from each row for a particular color point. Thus, all 3 red color points (Ra, Rb, Rc) are routed to one memory (together with a mode bit) where these contributions are mixed to produce the final output for the red color point. This mixing algorithm is programmable, but will normally be the same as that used in the other set of memories. The two sets of green strengths and for blue are also routed to separate memories to produce the final outputs Gl, G2 and B. In this application, only 4 strength bits are produced, while 8 bits are actually available from the 8k x 8 memory .

For flat screens that use a different color dot overlay, it's easy to adapt the architecture. For RGB diagonal or delta patterns, the color dots can still be arranged in groups of four, but the color dots will then move around. However, at most 3 different patterns occur (RGBR, BRGB, and GBRG), so the architecture must be modified to have two mode bits instead of one. Each memory must then have 16k locations instead of 8k. The mode bits then change each clock period to select the memory area reserved for each of the color dot patterns. Note that the final outputs are now based on position instead of just color (for RBBG quad displays, the color assignments will not change due to position). The upper output (of the four) drives R for the RGBR pattern, but B or G for the other two patterns. Thus, 512 x 512 FFM can still be expanded to fill all the color dots in a 1024 x 1024 flat screen regardless of the color dot layout.

Claims (5)

1. Graphics system implementing a filter to counteract image noise to improve image quality on raster display displays, characterized in that the system comprises the combination of: a modified graphics generator (20a) with output means providing standard integer x and y address data (50) for addressing an image memory , the modified graphics generator's output means also providing sub-pixel or fractional address data (51), a modified image memory (21a) which includes storage for integer and sub-pixel or fractional address data provided by the graphics generator (20a), said image memory having an input connected to receive integer and sub-pixel address data from said graphics generator's output means, said image memory having an output means which provides a sub-pixel output, a beam former (23) which has input means connected to the output side of said image memory (21a) and which has output means connected to an input on the raster display viewer (22), in that said beam former receives impulse data from said image memory and expands said data into a point spread function (i.e. to a Gaussian profile) before sending to the monitor, whereby said beamformer is a two-dimensional image filter that uses the received sub-pixel data for adjusting a centroid luminance and for band-limiting the image, and a flat-panel color monitor of the liquid-crystal type as said raster screen monitor . 2. System as stated in claim 1, characterized in that the beam former (23) comprises a means for selecting intensity from color bits which has an input means connected on the output side of said modified image memory (21a) and which has an output, a shading lookup means (51) which has a input connected to the output of the intensity selection means and having an output, means (52) for finding maximum shading having an input connected to the output of the shading lookup means and also having an output, and means (53) for moving a sliding window having a input coupled to the output of the means for finding maximum shading and having an output means coupled to an input of said raster display, the sliding window means producing a sliding window which is moved in a raster fashion in said image memory to provide a window of data centered around an active pixel, as the window of data is examined to determine the appropriate intensity level for the sub-pixel image output based on the data within the window. 3. System as set forth in claim 1, characterized in that the beam former also includes a color coding plan that matches the color pixel pattern of the screen display. 4. System as stated in claim 1, characterized in that the modified graphics generator has the form of a vector generator that draws lines and figures by increment counting to calculate addresses for pixels and then commands a write operation to each of these pixels (fig. 31). 5 . System as stated in claim 4, characterized in that the vector generator used with the beamformer includes an accumulator with sufficient precision to handle fractions, and into which accumulator a starting point address is first loaded, the vector generator including a truncation circuit and a fraction ironing block which irons out the fraction part in the pixel address, said vector generator thereby taking advantage of fractional information available from the accumulator and stores sub-pixel data within each pixel, so that each pixel can be positioned very precisely (Fig. 32).