GB2352352A

GB2352352A - Image processing to correct for flare effect

Info

Publication number: GB2352352A
Application number: GB0009502A
Authority: GB
Inventors: Stuart William Arundell Hunt
Original assignee: Innovation TK Ltd
Current assignee: Innovation TK Ltd
Priority date: 1999-04-16
Filing date: 2000-04-17
Publication date: 2001-01-24
Also published as: GB9908791D0; GB0009502D0

Abstract

Flare, or blooming, in an image is an unwanted effect wherein the brightness of one pixel alters the brightness of its neighbouring pixels. This effect is corrected by a method of image processing which comprises estimating the component of brightness in a particular pixel due to its neighbouring pixels and adjusting the brightness parameters for that first pixel so as to remove or reduce the flare effect. The flare effect may for example be estimated by using a mathematical model such as an exponential decay or by producing a defocussed image from the image.

Description

2352352 69852/001.609 Imaging Systems The present invention relates to

imaging systems, such as image scanning systems used in telecine and similar apparatus, for example of the type using a spot of light from a scanning cathode ray tube or laser. The invention is particularly concerned with flare effects.

Whilst the concepts of the present invention are of particular use in the field of digital image processing, some at least may be applicable to analogue systems.

It is well known that any optical imaging system will have a certain amount of flare. Flare is the phenomenon of non-imaging light becoming visible. Such examples are seen when images are projected onto a wall in a darkened room. If no flare were present, the image would appear on a black wall. In practice, the wall is diffusely illuminated around the screen by flare light, and this light decreases the contrast between the image and the surrounding area of wall. Further examples of flare include the phenomenon of haloes around bright objects against dark backgrounds. Photographs of street lights at night, for example, often show this phenomenon.

Flare occurs due to light being scattered or reflected to unwanted places. One cause of this is the use of uncoated lenses. These lenses will have unwanted reflections which contribute to flare. Even if lenses are optically coated, there will still be some unwanted reflections. A further contribution to flare is the effect of contaminants. Grease on the faces of lenses is certain to cause flare, as this will scatter the light severely. Atmospheric contaminants in the air, such as smoke, will also cause flare.

Whilst many steps can be taken to minimise flare in optical systems such as by using coated lenses and keeping equipment free from contaminants, there is inevitably residual flare that affects the performance of the systems.

Attempts have been made to use signal processing to correct for this flare. one such patented method is US 155 586, (Levy) assigned to the Sony Corporation of America. Other methods are described in US 5 715 070 (Tone) assigned to Ricoh Company Ltd. A method involving the attempted optical measurement of the flare is described in US 4 974 810 (Fiske) assigned to Eastman Kodak.

The above methods treat flare as causing an overall level shift to the picture's brightness. However, whilst the above is true, there is also a 'local' shift around bright areas of picture, and the present invention seeks to allow for this.

In one broad sense the present invention proposes to estimate an individual flare component that any pixel in the image contains as a result of the brightness of surrounding pixels, and adjusts the parameters for that pixel to remove or reduce the effect of the estimated flare component.

Viewed from one aspect the present invention provides a method of processing image data comprising brightness level parameters for a plurality of picture elements constituting an image, wherein for each picture element there is estimated a flare effect on that element having regard to the brightness levels of at least some surrounding picture elements, and the brightness level parameter for that picture element is adjusted in accordance with the estimated flare effect.

In an ideal situation it might be possible to estimate the effect of surrounding pixels in all directions, but in practice useful results can be obtained by estimating the effects of pixels in two orthogonal directions. The flare effect caused by a particular pixel will decrease with distance, and in preferred embodiments a mathematical model is used to represent this decay of the flare effect. Whilst a linear decay might give reasonable results, it is considered that a more accurate estimation of the actual flare effect can be obtained by using an exponential decay, represented for example by the function e-kd where Ilk" is a constant and I'd" is the distance away from the pixel. In a typical system, distance as such will not be used, but rather a number representing the number of pixels from the pixel concerned.

The flare effect on a particular pixel will be the cumulative effect of the surrounding pixels. A pixel several away will generally have a reduced effect compared to the next adjacent pixel, although of course this will depend on the brightness of the pixels. A very bright pixel several away may have a greater flare effect on a particular pixel than a very dim one which is adjacent.

The invention is particularly, although not exclusively, concerned with digital techniques. Typically, the pixels constituting an entire image of e.g. a scanned film frame will be in a frame store. The brightness parameters for these can be adjusted in sequence, but preferably the flare adjustment parameters for all of the pixels are first calculated and stored, and then applied to the complete set of data. In a preferred system using two orthogonal directions for estimating the flare effect, there are four contributions to the flare effect adjustment on a particular pixel. These are the cumulative flare effect of the pixels to the left of the pixel in question, the cumulative flare effect of those to the right, the cumulative effect of those above and the cumulative effect of those below. Considering image data as represented by lines and columns, one would first scan along a line in a forwards direction, looking at the brightness of each pixel in turn, estimating - using 4 e.g. the mathematical model of exponential decay - the flare effect of that on succeeding pixels and calculating for that pixel the cumulative flare effect from the preceding pixels. This would then be done in the reverse direction, and then in the up and down directions. In practice of course the pixels would be represented by data in a frame store, and the algorithms necessary can be applied effectively simultaneously to calculate the four contributions to flare on each pixel.

These contributions can then be summed and applied to the values for the pixels.

Considering an analogue equivalent, the line data would be fed through analysing circuitry first in one direction, then the other, and then the image would be rotated by 90 degrees and the process repeated to give the four adjustment values to be summed. The estimated decay of the flare effect can be achieved by an infinite impulse response type filter (IIR), for example of a simple type using a capacitor and a resistor. In theory, the decay to a zero value is effective over an infinite period (in the present application of course this is a distance). of course, in a practical embodiment using digital techniques one would use a mathematical model which has no effect after a predetermined distance, i.e. a certain number of pixels away, and this might be, say, five, ten or twenty depending upon the context.

In an alternative embodiment of the invention, the total flare effect on each individual picture element in an image can be estimated as being proportional to the brightness level of each said corresponding individual picture element in a defocussed image produced from the image.

The invention also extends to apparatus for processing image data comprising brightness level parameters for a plurality of picture elements constituting an image, comprising means for estimating a flare effect for each picture element having regard to the brightness levels of at least some surrounding picture elements, and means for adjusting the brightness level parameter for that picture element in accordance with the estimated flare effect.

In one preferred embodiment of the invention, the means for estimating a flare effect for each picture element having regard to the brightness levels of at least some surrounding picture elements, comprise a Digital Video Effects system for producing a defocussed image from the image.

Preferred embodiments of the invention will now be described, by way of example only, and with reference to the accompanying drawings, in which:

Figure I shows a schematic representation of a notional circuit for correcting the flare of an optical signal according to a first embodiment of the invention; and Figure 2 shows a schematic representation of a notional circuit for correcting the flare of an optical signal according to a second embodiment of the invention.

The embodiments of the invention described below relate to a system for correcting for a flare effect in an image for use in a telecine machine. The telecine machine converts images on cinematographic film into electrical signals using a scanning cathode ray tube or laser which produces a light source which passes through the film and is detected by light detecting means such as photomultipliers after passing through the film. Means for transporting the film through the telecine machine are also provided. One such telecine machine with which the system described could be used is the 11YFront" machine sold by Innovation TK Limited or the "URSA" machine sold by Cintel International Ltd.

In television and video signals, for example from a telecine machine or a camera, each picture frame is represented by a set of data giving the brightness level for each of the three primary colours of each pixel making up the image. The brightness data recorded will however be distorted due to flare from the adjacent pixels such that the colour of the actual and recorded images is slightly different. Therefore, the present invention provides a method of correcting the measured signal by modelling the flare content of the signal and subtracting the modelled flare content from the measured signal to obtain a signal which is closer to the actual signal. For optimal processing, the optical signal should be in digital form, with 10 or 12 bits of quantisation per colour.

In order to model the total flare effect, the total is flare effect on each pixel in the frame must be modelled. The necessary calculations may be carried out separately for each of the three sets of colour data for each pixel. In an alternative arrangement, it may be assumed that the flare effect is a function of luminance. Thus a standard RGE signal could be converted to luminance and chrominance form, the flare compensation carried out only once on the luminance values, and then the data converted back into RGB form.

In order to model the flare effect for one pixel, it is assumed that the brightness level of the pixel will decay exponentially over distance (i.e. the flare is modelled as decaying according to e -kd) where d is the distance from the pixel in question. Thus, the flare effect of that pixel at a point a distance d away may be modelled extremely simply. Of course, numbers of pixels may generally be used rather than absolute distances.

In order to model the total flare effect of each pixel on all the other pixels in the frame, the signal data is analysed as follows.

Data processing means such as a powerful computer, although it could be a system integrated in a telecine machine, will obtain and sum four separate sets of data to obtain the total estimated flare effect on each pixel. Thus, the computer firstly uses an algorithm to find the cumulative flare effect on each pixel in each line of data of all the pixels in the same line of data to the left of each of the pixels. Subsequently or simultaneously, using a similar algorithm, it will compute the estimated flare effect on each pixel in each line of data due to all of the pixels to the right of each said pixel in the same line of data. Also subsequently or simultaneously, the computer will read the cumulative estimated flare effect for each of the pixels in each column of data due to all of the pixels above the said pixel in the same column and finally the computer will compute the estimated cumulative flare effect for each pixel in each column due to all of the pixels below the said pixel in the same column.

The four flare values estimated for each pixel will together represent an estimate of the total flare effect on that pixel. Thus, in order to correct the optical signal, these values representing the flare must simply be subtracted from the digital signal.

Figure 1 shows a schematic representation of the series of calculations performed on the optical signal in order to correct it for flare. In Figure 1, elements 2, 5, 8 and 11 correspond to the algorithm which is applied to the data so as to calculate the flare effect of each pixel in the line or column on each of the other pixels in the line or column on each of the other pixels in the line or column using the exponential decay model described above. Elements 3, 4, 9 and 10 show line stores which reverse the order of the data in the line.

Thus, in schematic terms, the signal is input to the system at I and is operated on in parallel by four separate sets of operators, each comprising an algorithm and a line store.

Thus, the signal passing through algorithm 2 will be acted on by the algorithm and then stored, thus providing flare signals for each pixel along the line due to the pixels to the left of it. The signal passing through algorithm 5 will first be stored and will change line direction before being acted on by the algorithm.

This will therefore provide the total flare values for each pixel along a line due to all the pixels to the right of it.

As shown at 7, the signal passing through the other two branches of the parallel circuit is first rotated through 90'. Thus, the signal will be read in columns rather than lines. The signal passing through algorithm 8 is first acted on and then stored so as to give flare values for each pixel in the column due to the other pixels in the column above it. However, the signal passing through algorithm 11 is first stored and changed in line direction and then acted on by the algorithm. Thus, this will provide flare values for each pixel in each column due to the pixels below it in the column.

The two sets of flare readings obtained for the column must be rotated back through 900 before they correspond to the line data and this is shown at 13. The line and column data will then be summed as shown before the total set of flare readings for the optical signal are subtracted from the optical signal as shown at 14. Thus, a corrected signal is obtained which should correspond to the signal which would have been measured if there were no flare.

Although the flare from a particular pixel will decay exponentially such that, in theory, the flare will never be zero however far from the pixel it is measured, for the purposes of the present invention, the flare can be assumed to have reached zero level at a distance of approximately 20 pixels away from its source. Thus, it is not always necessary to include every pixel in a line or column in the calculations.

As an alternative to the flare simulation method outlined above, it would be possible to correct the optical signal for one of the estimated horizontal flare levels and to use this corrected signal to calculate the other of the horizontal flare levels. The signal could then be corrected again before estimating the column flare component and so on. As the flare should only be a small part of the total light measured, for example 2% of the measured signal, estimating the flare cumulatively should not greatly effect the accuracy of the system.

As an example of a typical system, for an 8-bit quantisation (0-255), the flare on a pixel of mid grey (128) could typically be 2-3 units.

The flare model can be derived empirically by, for example adjusting the value of k in the model fae" and using test charts to ascertain at which value of k flare is no longer visible. The initial value of the flare effect, as opposed to the decay characteristics, can also be determined empirically. Effectively, an operator using a telecine machine for example can look at a test chart which is being scanned, and adjust the flare effect up and down until the appearance is considered to be satisfactory.

In an alternative embodiment of the invention, a DVE (Digital Video Effects) system is attached to a telecine machine. A suitable DVE is the Kaleidoscope model made by Grass Valley Group of California, USA. Using the DVE, the digital image obtained by the telecine machine can be defocussed to produce a defocussed image. The inventor has realised that this defocussed image can be used as a model of the flare effect, in which the brightness level of each picture element in the defocussed image is proportional to the cumulative flare effect on that picture element. Thus, by subtracting the defocussed image (or a scaled version thereof) from the actual image obtained by the telecine machine, a flare corrected digital image is obtained.

It will be appreciated that the described methods and apparatus according to the various embodiments of the invention can be used to correct for a flare effect in a digital image obtained by any means and are particularly applicable to digital images obtained from a telecine machine comprising a cathode ray tube or laser scanner.

Claims

1. A method of processing image data comprising brightness level parameters for a plurality of picture elements constituting an image, wherein for each picture element there is estimated a flare effect on that element having regard to the brightness levels of at least some surrounding picture elements, and the brightness level parameter for that picture element is adjusted in accordance with the estimated flare effect.

2. A method as claimed in claim 1, wherein the flare effect on each picture element is estimated from the effects of picture elements in two orthogonal directions on each said picture element.

3. A method as claimed in claim 1 or 2, wherein a mathematical model is used to represent the decay of the flare effect caused by a particular picture element with distance.

4. A method as claimed in claim 3, wherein the mathematical model used is an exponential decay.

5. A method as claimed in any preceding claim, wherein the flare adjustment parameters for all of the picture elements in the image are first calculated and stored, and then applied to the complete set of image data.

6. A method as claimed in any preceding claim, wherein the flare effect on each picture element is estimated by:

scanning along a line in which the picture element is located in a forwards direction, looking at the brightness of each picture element in turn, estimating the flare effect of that picture element on succeeding picture elements and calculating for that picture element the cumulative flare effect from the preceding picture elements; repeating this process in the reverse direction; scanning along a column in which the picture element is located in an upwards direction, looking at the brightness of each picture element in turn, estimating the flare effect of that picture element on succeeding picture elements and calculating for that picture element the cumulative flare effect from the preceding picture elements; repeating this process in the reverse direction; and summing the four cumulative flare effect values obtained.

is

7. A method as claimed in claim 1, wherein a defocussed image is produced from the image and the total flare effect from the surrounding picture elements on each individual picture element in the image is estimated as being proportional to the brightness level of each said corresponding individual picture element in the defocussed image produced from the image.

8. An apparatus for processing image data comprising brightness level parameters for a plurality of picture elements constituting an image, comprising means for estimating a flare effect for each picture element having regard to the brightness levels of at least some surrounding picture elements, and means for adjusting the brightness level parameter for that picture element in accordance with the estimated flare effect.

9. An apparatus as claimed in claim 8, wherein the means for estimating a flare effect for each picture element having regard to the brightness levels of at least some surrounding picture elements comprise processing means for estimating the flare effect for 13 each picture element using a mathematical model.

10. An apparatus as claimed in claim 8, wherein the means for estimating a flare effect for each picture element having regard to the brightness levels of at least some surrounding picture elements comprise means for producing a defocussed image from the image.

11. A method of correcting for flare in an image comprising the steps of:

estimating an individual flare component that any pixel in the image contains as a result of the brightness of surrounding pixels; and adjusting the parameters for that pixel to remove is or reduce the effect of the estimated flare component.

12. A method of processing image data substantially as herein described and with reference to Figures 1 and 2 of the drawings.

13. A method as claimed in any of claims 1 to 7, 12 or 13, wherein the image data is obtained from scanning cinematographic film.

2S

14. An apparatus for processing image data substantially as herein described and with reference to Figures 1 and 2 of the drawings.

15. An apparatus as claimed in any of claims 8 to 10 or 14, for use with image data obtained in a telecine machine by scanning cinematographic film.