FR2798033A1 - METHOD AND SYSTEM FOR PROCESSING DIGITAL IMAGES - Google Patents

Abstract

The invention relates to a method and a system for treating digital images. The inventive method comprises a stage in which the width of the image is compared with a predetermined width (L). If the width of the image is greater than the predetermined width (L), the following stages are then carried out: a sequencing stage in which a memory is read in bands of said width (L) and of a height equal to that of the image; a vertical conversion calculation stage; a horizontal conversion calculation stage; and a writing stage in bands, in a memory.

Description

Method and system for processing digital images The present invention relates to the field of digital image processing that is displayed on a screen, in particular computer or television.

In general, the display of computer-generated graphics on a low-resolution television screen with interlaced refreshment has always presented great technical difficulties without achieving a truly satisfactory quality for the users, because of a flickering effect. On a standard television screen, the lines are refreshed interlaced. In other words, the rows of odd rank are refreshed in the first beat and the rows of even rank in a second beat and so on. If two consecutive lines have very different levels of brightness the user's human eye will locally perceive a flicker at a frequency of 50 or 60 Hertz according to the television standard adopted. However, computer-generated images such as graphics, windows or subwindows typically have strong contrasts from one line to another.

Currently, the laptop and TV markets are merging, driven by the development of the internet and interactive television that requires associated technologies. It is therefore important to be able to display images with strong contrasts from one line to another, such as the pages broadcast on the Internet, on a standard television screen with user-acceptable visual comfort.

The systems used so far to suppress or limit flicker effect are often referred to as "flicker" filters. Since the flickering effect is due to the contrast between two consecutive lines, that is to say at high frequencies on a vertical column image, basic anti-flicker filter systems implement low pass vertical digital filtering. After filtering, two consecutive lines are closer in color and the flicker effect is reduced.

The effectiveness of a digital filter is closely related to many of the points taken into account. This type of filtering consists in effect to replace point or pixel by the average of the neighboring points and is all the more effective as the average relates to a large number of points. This has a significant influence on the slope of the transition between the filtered pass and the blocked or filtered part. In most graphical architectures, the data relating to the color of a point or a pixel is read in a buffer in a horizontal scanning order, this requires the implementation of an equal number of lines memory at n - 1 for an n - step filter. This is expensive in line memory only if the filtering is done vertically aligned pixels because, in this case, you must memorize the lines. In the case where the filtering is performed on horizontally aligned pixels, the cost is much lower because it does not use any memory. As a result, the only cases posing problems are the reduction of the flicker, the vertical enlargement / reduction and more generally all phenomena requiring filtering on pixels vertically aligned. This results in a very high cost of line memories whose capacity must be significant, particularly high definition with lines of the order of 1920 pixels. These line memories are so expensive, because they require large areas of silicon, the anti-flicker filters are generally three steps, that is to say with two lines stored, which results in performance of poor filtering. However, the filtering takes place during the steps of displaying the image, which requires high throughput performance even if the filtering is poor.

In addition, graphical objects in the form of windows or sub-windows are often displayed on the screen, the user of which may wish to modify the height and / or the width. The processing required by changing dimensions results in the storage in fast memories of pixels of one or more lines, resulting in high costs.

The present invention aims to overcome the disadvantages mentioned above.

The subject of the present invention is an economical graphic processing method, using only a reasonable memory size and adapted to high quality images regardless of the size of the graphic object being processed.

The graphic processing method, according to the invention, is provided in particular for digital images, and comprises a step of comparing the width of the image and a predetermined width L, if the width of the image is greater than the predetermined width L, a sequencing step performing a reading of a strip memory of said width L and height equal to that of the image, a vertical conversion calculation step, a horizontal conversion calculation step, and a tape write step in a memory.

Advantageously, all the strips are of equal width except the last whose width is equal to the rest of the division of the width of the image by said predetermined width.

In one embodiment, the vertical conversion calculation and horizontal conversion calculation steps achieve an enlargement of the image.

In another embodiment, the vertical conversion calculation and horizontal conversion calculation steps reduce the image.

Advantageously, if the width of the image is less than or equal to the predetermined width L, the sequencing step is deleted. In one embodiment, one or more vertical filtering steps are provided between the vertical conversion calculation step and the horizontal conversion calculation step.

The present invention also relates to a device for generating digital images, of the type comprising an image calculation means. The device comprises a means for comparing the width of the image and a predetermined width L, sequencing means performing a reading of a strip memory of said width L and height equal to that of the image if the width of the image is greater than the predetermined width L, a vertical conversion calculation means, a horizontal conversion calculation means, a strip write means in a memory.

Advantageously, the image calculation means is provided with a plurality of buffer buffers, vertical converter, and a horizontal converter.

In one embodiment, anti-flicker filter means is disposed between the vertical conversion calculation means and the horizontal conversion calculation means.

In one embodiment, the filtering means includes a plurality of line buffers, and means for matching the filtering to local features of the image, so that only areas of the high-flicker image are object of said anti-flicker filtering. The anti-flicker filtering may be adaptive according to a flicker index so that only areas of the flicker image are subject to said anti-flicker filtering.

It is important to consider how the contents of the buffers are refreshed in a graphics subsystem. Most of the time the graphical screen is built as a set of rectangular subwindows that can be moved, zoomed, zoomed, updated, etc. These operations are performed under the control of a software and apply window after window, generally synchronously and relatively slowly compared to the refresh rate of the screen.

The invention thus makes it possible to avoid expensive filter provided with large memories applying to the image to be displayed over its entire width and which from one refresh to the other is often kept, which would lead to perform the same filtering calculations for n consecutive images displayed all strictly identical or at least largely.

For animation applications, the usual technique is to use a dual buffered architecture. The two-dimensional or three-dimensional calculation is performed in one of the buffers while the other is displayed. Each time a new image has been calculated, the buffers are swapped. These buffers can here be buffer zones in a shared external memory. The invention makes it possible to reduce the cost of the filters to be used. The processing is done from memory to memory before the display of the treated area, which has many advantages. This relieves real-time constraints of the display. The filtering performance is not directly related to the processing speed of a pixel. The same filter can process standard definition pictures as well as high definition pictures.

The bandwidth requirements for the display means remain unchanged. As a memory-memory processing, the entire digital image generation device is not forced to follow the standard refresh order of the display screen. The amount of memory required for line buffers can be significantly reduced by slicing the strip image. The anti-flicker filtering means need not be duplicated in the case of a global system connected to several graphic display means. The filter can be selectively applied to specific areas of an image.

In graphics subsystems, a two-dimensional accelerator is usually arranged that acts as an operator on rectangular graphic objects such as windows. This accelerator or bit-block transfer module called "blitter" is used for an update of image areas, and is the best place to integrate the anti-flicker filter and an enlargement / reduction means.

The present invention will be better understood from the detailed description of an embodiment of the invention taken by way of non-limiting example and illustrated by the accompanying drawings, in which FIG. 1 is a schematic view of a system according to the prior art; Figure 2 is a schematic view of a graphic object processed according to an embodiment of the invention; Figure 3 is a step diagram of a method according to one embodiment of the invention; and Figure 4 is a schematic view of a graphics processing system according to one embodiment of the invention.

As can be seen in FIG. 1, a conventional flicker filter comprises a filter element 1 with three steps which is connected to a graphic intermediate stage 2 capable of sending it one after the other, the relative data pixels of the lines of the image to be displayed. A link 3 makes a direct connection between the intermediate stage 2 and the filtering element 1. A buffer memory is arranged between the intermediate stage 2 and the filtering element 1 to supply the latter with the data relating to the pixel of FIG. the line immediately preceding the one whose pixel data is being transmitted on line 3. The output of buffer 4 is connected to filter element 1. Buffer 4 can also be called line memory . A buffer 5 similar to the buffer memory has its input connected to the output of the buffer memory 4 and its output connected to the filtering element 1 to supply the latter with data relating to the pixels of the antepenultimate line with respect to the line whose pixel data is being transmitted on link 3.

The filter element 1 also has an input by which it receives the filter coefficients to be used for the image file being transmitted from the interface 2. The filtering element 1 further comprises an output 7 which is generally connected to the not shown display means.

It can thus be seen that such a three-step anti-flicker filter requires two line buffers while performing for a picture file a filtering which is never a rarely satisfactory compromise between the accuracy of the displayed image and the reduction of the flicker of which it is affected.

In FIG. 2, it can be seen that a graphic object, here a window composed of pixels arranged in a matrix comprising A rows B columns, is processed by bands of width L, in other words comprising L columns in the case where B is greater than L. In the case where B is less than or equal to L, the processing of the graphic object is performed by a single band of B columns. For example, we can choose L equal to 128, especially in the case of high definition television where B is of the order of 2000, for example equal to 1920. The value 128 then represents a good compromise between the size of the necessary memory and the speed of the treatment. Thus, a graphic object of A lines of 1920 pixels will be treated not complete line but in strips of 128 pixels and inside each band line by line. We will treat fifteen bands successively.

B is not divisible by L, the last band will be of width equal to the remainder of the division of B by L. This banding is performed when reading the data relating to the graphic object in a non-volatile memory of RAM type, by a reading means such as an address manager. After the desired operations performed on the graphic object such as size change, filtering, etc. the rewriting is performed in the memory in the same order in L-width bands.

In Figure 3, the steps of a method according to the invention are illustrated. In step 8, an address manager performs testing of the length B of the graphic object, in other words the number of pixels per line. If B is less than or equal to L, the graphic object is read in a conventional manner in the memory where it is stored, see step 9, starting from the first pixel of the first line to the last pixel of the first line, then we read the first pixel of the second line to the last pixel of the second line and so on until the last pixel of the last line. In step 10, the pixels of the graphical object read undergo, in the order of reading, the desired operations and then are rewritten in a memory in step 11, this memory possibly being the same or different from that where the graphic object was originally stored.

If the result of the test of step 8 is different, ie if B is greater than L, the graphic object is read in strips of L pixels in the memory where it is stored, see step 12, starting with the first one. pixel in the first line and in the first coordinate strip (1,1) to the last pixel in the first line and in the first band or Llth pixel of the first coordinate line (L, 1), then reading the first pixel in the second line and in the first coordinate strip (1,2) to the last pixel in the second line and in the first band Lth pixel in the second coordinate line (L , 2) and so on until the last pixel in the first band in the last line or Llth pixel of the last coordinate line (L, A).

However, there is an overlap of the bands because the horizontal converter is N not, with for example N = For the calculation of a pixel, the horizontal converter needs said input pixel and previous N-1 pixels of the same line notwithstanding the tapes.

To reduce the flickering effect, a 3-step filter is used. The coefficients are determined dynamically according to the luminance gradient measured between 3 vertically aligned pixels.

To change the dimensions of a rectangular graphical object, the method used is a polyphase digital filter with N no 2P subheadings.

An increment value is encoded with an integer part and a decimal part. This increment represents the distance separating two pixels at the output of the filter. If this increment is greater than 1, a reduction of the dimension is performed. Otherwise, the object is enlarged.

If the increment is equal to 1.25, the number of pixels generated is less than the number of pixels received at the input of the filter.

If the increment is less than 1, the number of pixels generated is greater than the number of pixels received. The graphic object is enlarged. These pixels are generated as follows. The INC increment is broken down into an integer and decimal part. Each time a pixel is generated, this increment is added to the rest of the accumulated decimal part COUNT = decimal part [COUNT] + INC.

This COUNT variable therefore has an integer and decimal portion. The integer part represents the number of pixels to consider before generating the next pixel. The decimal part represents the intermediate position of the pixel to be generated with respect to the last pixel taken into account.

Once implemented, in order to define the sub-positions, only the first p bits of the decimal part are kept to make 2P sub-positions. At each subheading, there is a filter with N not performing a weighted average depending on the subheading on the N samples present in the filter.

Each time a pixel is generated, the filter's algorithm gives the sub-position of the next pixel to generate, and therefore the number of the filter to be applied (it is the decimal part of COUNT) as well as the number of pixels to take into account (this is the entire part of COUNT).

The filter is positioned as follows to generate the pixels. Initially, pixel 0 is duplicated (N-1) / 2 (odd N) times so that this filter is centered on pixel 0.

As part of the vertical filter, the COUNT value is updated at the end of each line. The algorithm then indicates how many lines must be accessed and what will be sub-position of the next line to generate.

As part of the horizontal filter, the COUNT value is updated each time a pixel is generated.

In the context of the present invention, the aim is to minimize the cost of line memories necessary for storing the lines in order to achieve the vertical dimension change. The proposed solution is to cut the image by 128 pixels strip so that the size of these line memories is reduced to 128 pixels.

In the context of a source image width greater than 128 pixels, the processing will be successively from the first band and until the last.

The problem encountered in this case is a perfect bonding of the strips if in addition to a vertical dimension change, a horizontal dimension change is applied.

As soon as the increment causes the last pixel of the band to be exceeded, the resulting pixel can not be generated since the filter does not have enough pixels for this generation. As a result, this pixel will be generated when the next band is being processed.

The position of the next band is thus determined by the number of the last pixel used in the current band, the value of the increment counter and the number of the filter. If L is the number of the last pixel used in the horizontal filter, [COUNT] the integer part of the increment counter once generated the last pixel of the current band, and N the number of steps of the digital filter, the next band starts then with the pixel number L + [COUNT] -N.

The variable COUNT is updated as new-COUNT = decimal part [COUNT] xINC eg new-COUNT = 0,75 + 3,25 = 4.

The new band starts at: L + 4 - 5 = L - 1.

The next band is requested at the address generator times the last pixel of the last line of the current band generated. The new start position of the new tape is passed to the address generator so that no artifacts appear. The filter stores the sub-position corresponding to the first generated pixel of each row of the next column. Thus, the division in columns does not induce any undesirable effect on the generated image.

To allow identification of bands, there is provided a signaling flag or "flag" of the first line, the last line, the first pixel of a band in a line and the last pixel of a band in a line. The means performing step 13 thus have the information that the next pixel will be the first pixel of a next line in the same band. The signaling of the first line also makes it possible to start the processing by a vertical converter by storing the pixels of the first line in several buffers simultaneously.

4, a graphics processing system 15 for horizontally and vertically modifying the dimensions of a graphic object comprises a vertical five-step converter 16 which receives on its five inputs the data relating to the pixels of the current line and those relating to pixels of the four preceding lines by four buffers 17, 18, 19 and 20 similar to the buffers illustrated in Figure 1 but able to store only L pixels and arranged in the same way to perform the interpolation of the five bits received at the same time. The buffers 17, 18, 19 and 20 operate in a first-in-first-out or FIFO mode.

The graphics processing system 15 comprises a flicker filter 21 provided with a filter element 22 with three steps and two buffers 23 and 24 of capacity limited to L pixels. The general arrangement is analogous to the filter of FIG. 1 except that the capacity of the buffers goes from B pixels to L pixels, for example from 1920 to pixels at 128 pixels, ie a significant reduction in the cost of said memories and the silicon surface used.

The graphics processing system 15 also comprises a horizontal five-step converter disposed at the output of the filtering system 21 and able to change the size of the graphic object in the horizontal direction.

Both vertical and horizontal conversion takes place on a determined number of bits, for example three. We can thus enlarge the graphical object up to eight times in each dimension. Seven pixels of intermediate coordinates can be generated between two original pixels.

The horizontal converter 25 needs five original pixels to generate an output pixel with appropriate weighting of the original five pixels according to the selected magnification / reduction ratio and the output pixel position relative to the original pixels. . For example, if you choose to zoom horizontally by a factor of 2 graphic image, the system will generate pixels for abscissa 1, 1.5, 2, 2.5. If you choose a factor 1.25, the system will generate pixels for abscissa 1, 1.75, 2.5, 3.25, 4 etc. These pixels will then be stored linearly in the memory, not shown in the figures.

With bands of 128 pixels wide, the address of a pixel is coded on ten bits, seven bits of high weight corresponding to the position of a bit from 0 to 127, and three bits of low weight corresponding to the eight bits. - possible positions after treatment.

In the case of enlargement, after treatment, a band of pixels has a width greater than 128. Hence the interest of the particular signaling of the last pixel of a band on a given line which allows the address manager of perform the rewrite operation properly by calculating the coordinates of the place where the next pixel is to be written.

The horizontal converter 25 is not affected by the width of the bands. With a little more than 3 kilobytes of static memory, there is a vertical converter with five effective steps able to process graphic objects of unlimited size associated with a flicker filter and a horizontal converter.

The graphical processing system 15 or bit-block transfer module makes it possible, in a graphics circuit, to speed up the operation by transferring graphic blocks such as a window into a windows environment. This arrangement of the filtering system 21 between the vertical converter 16 and the horizontal converter is particularly interesting because it makes it possible to process the image as it is displayed in the direction of the columns, knowing that the flickering effect is almost entirely due at the juxtaposition pixels presenting a too high contrast on the same column. The filtering system 21 thus operates from memory to memory and avoids the reappearance of the flicker phenomenon when the user implements a zoum function which can tend to increase the contrast between two consecutive pixels of the same column, thanks to the fact that the filtering performed after the vertical conversion.

In other embodiments, provision could be made for the filtering system 21 downstream of the bit block transfer module.

In the example described, the filter works only three pixels of the same column to calculate the data relating to a pixel. could design to use a filter working on a larger number of pixels, but this would tend to increase the cost of the filter because of need to add buffers of lines. It is feasible to make the best use of a three-step filter comprising only two line memories by means of an adaptive strategy making it possible to perform filtering according to the risks of flicker with a dynamic management of the weights of each of the three pixels used to the calculation of the data of one of them.

The length of the line memory is preferably limited to thus benefiting from a two-dimensional rescaling system based on the columns. In other words, the image is cut into areas whose lines are 128 bits to use less memory during filtering. The vertical converter is disposed upstream of the flicker filter. Indeed, a converter with five steps may reintroduce flicker. It is therefore advantageous to carry out the filtering after the vertical conversion. Conversely, the horizontal converter is disposed downstream of the flicker filter so that the size of the line memories is not affected by the horizontal scale change. This is possible because the horizontal conversion is a linear operation that does not produce additional flicker.

The filtering system has no limitation with respect to the size of the images that can be filtered. With a high rate of the order of 100 megapixels per second, it forms an effective solution for standard HDTV screens. Of course, this system described for two-dimensional images is applicable to three-dimensional images based on a matrix of voxels.

Thanks to the invention, flicker is filtered as soon as the image is created and the filtered image is then stored in a memory and can be displayed on a screen without undergoing further filtering during the display. This avoids having to perform filtering each refresh of the screen. Such filtering would be very expensive especially in high definition at about 2000 pixels per line at a frequency of the order of 80 to <B> 100 megahertz. In addition, the filter is forced to read two lines because of the interlaced scanning of the television screens. Such a filter would occupy a large silicon area.

On the contrary, according to the invention, one takes advantage of the characteristics of graphic images which are generally relatively stable in the sense that the user uses them for a duration that can reach several minutes or several tens of minutes. It is therefore particularly advantageous to treat the phenomenon of flicker as soon as the image is created. We are free of real-time constraints and speed constraints. It eliminates the screen resolution and bandwidth constraints. We filter in the order we want. The amount of memory required is divided by an important factor, of the order of 16 compared to a filtering occurring during the steps of displaying a high definition image at about 2000 pixels per line. It is possible to use an adaptive filter giving high quality images with relatively few material resources, especially memory.

 In a more general manner, this method of digital image processing makes it possible to access pixels along a vertical axis while these are arranged by rows in the memory. As a result, this method makes it possible to correct phenomena appearing vertically in an image such as the flicker effect, but also makes it possible to filter over- or under-sampled images as in the case of a vertical enlargement / reduction.

Claims (10)

A digital image processing method comprising: a step of comparing the width of the image and a predetermined width L, if the width of the image is greater than the predetermined width L, a step of sequencing performing a reading of a memory in strips of said width L and height equal to that of the image, - a vertical conversion calculation step, - a horizontal conversion calculation step, and - a write step in strips in a memory. 2. The method of claim 1, wherein all the strips are of equal width except the last whose width is equal to the remainder of the division of the width of the image by said predetermined width. The method of claim 1 or 2, wherein the vertical conversion calculation and horizontal conversion calculation steps enlarge or reduce the image. The method of claim 1 or 2, wherein the strips are overlapping. The method of any of the preceding claims, wherein if the width of the image is less than or equal to the predetermined width L, the sequencing step is suppressed. The method of any of the preceding claims, including one or more vertical filtering steps between the vertical conversion calculation step and the horizontal conversion calculation step. 7. Device for generating digital images, of the type comprising an image calculation means, characterized in that it comprises a means for comparing the width of the image and a predetermined width L, a means sequencing device performing a reading of a strip memory of said width L and height equal to that of the image, if the width of the image is greater than the predetermined width L, a vertical conversion calculation means, horizontal conversion calculating means, and tape writing means in a memory. 8. Device according to claim 7, characterized in that the image calculation means is provided with a plurality of line buffers, a vertical converter, and a horizontal converter. 9. Device according to claim 7 or 8, characterized in that an anti-flicker filtering means is disposed between the vertical conversion calculation means and the horizontal conversion calculation means. 10. Device according to any one of the preceding claims, characterized in that the filtering means comprises a plurality of line buffers, and means for adapting the filtering to local characteristics of the image, so that only areas of the high-flicker image are the subject of said anti-flicker filtering.