HK1137835B - Multiple modulator displays and related methods - Google Patents

Description

Multiple modulator display and associated method

Technical Field

The present invention relates to electronic displays such as computer monitors, televisions, data projectors, and the like. More particularly, the present invention relates to a display that modulates light from a first pixel array by a second pixel array to produce an image.

Background

A Dual modulator display (Dual modulator display) has a first array of pixels that produce a controllable pattern of light on a second array of pixels. Examples of dual modulator displays are described in WO02/069030(PCT/CA2002/000255) and WO03/077013(PCT/CA2003/000350), both entitled "HIGHDYNAMIC RANGE DISPLAY DEVICES". In some embodiments, image data specifying a desired image is supplied to a controller that operates the first pixel array to generate a pattern of light on the second pixel array. The pattern of light approximates the desired image. The controller operates the second pixel array to modulate the pattern of light to produce an image that is closer to the desired image than the pattern of light. In some embodiments, the first pixel array has a lower resolution than the second pixel array (i.e., there are more pixels in the second pixel array than in the first pixel array). The first array of pixels may comprise an array of individually controllable light source pixels, such as a spatial light modulator, or the like. The second array of pixels may comprise reflective or transmissive spatial light modulators.

In some embodiments, the first pixel array comprises a Light Emitting Diode (LED) array and the second pixel array comprises a Liquid Crystal Display (LCD) panel.

The "dual modulator" display architecture may be suitable for high-end displays (some examples of high-end displays are displays for viewing X-rays and other critical images, high-end cinema applications, etc.). In these applications, it is desirable to have the displayed image reproduce the desired image as closely as possible. In these applications, any perceptible deviation from the desired image is not desired.

In an "ideal" dual modulator display, the first array of pixels has a light output that can be varied steplessly from zero to very bright, and the second set of pixels is steplessly controllable between transmitting zero light and passing all incident light. In reality, the composition actually used as the first pixel set and the second pixel set has a limit. For example, where the first set of pixels or the second set of pixels comprise LCD (liquid crystal display) pixels, the pixels have a maximum transmittance of less than 100%, a minimum transmittance of greater than 0%, and the transmittance of each pixel is generally selectable from a set of discrete values. Similarly, where the first set of pixels comprises an array of individually controllable light sources (such as LEDs, for example), the light sources typically have a light output that can be adjusted to some maximum value in discrete steps.

One problem is to determine how to control the first set of pixels such that the pattern of light on the second set of pixels will approach the desired image in a manner that can be corrected to a higher accuracy by the second set of pixels.

Disclosure of Invention

One aspect of the invention provides a method for generating control values for pixels in a first array of pixels in a dual modulator display. The method comprises the following steps: generating a desired light pattern and an initial set of control values from image data defining a desired image; and determining for a pixel a variation Δ d tending to reduce the difference between a desired light pattern at a location corresponding to said pixel and a light pattern estimated to be produced in response to the set of control values_jThe initial set of control values is refined.

Other aspects of the invention and features of particular embodiments of the invention are described below.

Drawings

In the drawings which illustrate non-limiting embodiments of the invention,

fig. 1 is a schematic diagram of a dual modulator display.

Fig. 2A is a diagram containing curves representing a desired image and a light pattern produced by the first pixel array.

Fig. 2B is the high spatial frequency content produced by the second pixel array for the image from fig. 2A.

FIG. 2C is another diagram representing a desired image and a light pattern produced by the first pixel array that is a sub-optimal match of the desired image.

Fig. 2D is the high spatial frequency content produced by the second pixel array for the image from fig. 2C.

Fig. 3 and 3A are flow charts illustrating methods according to the present invention.

Fig. 4 is a schematic view showing an effect of veil glare (veiling glare).

FIG. 5 is a block diagram illustrating functional components of a controller according to an embodiment of the present invention.

FIG. 6 is a block diagram illustrating functional components of a system for generating control signals according to an embodiment of the present invention.

Detailed Description

Throughout the following description, specific details are set forth in order to provide a more thorough understanding of the present invention. However, the invention may be practiced without these particulars. In other instances, well-known elements have not been shown or described in detail to avoid unnecessarily obscuring the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

The inventors have determined that there is a need for displaying an image while reducing the loss of image information despite the capability limitations of the first and second pixel arrays:

dual modulator displays;

a controller for a dual modulator display;

a method for operating a dual modulator display;

a method for generating control values for controlling pixels of a dual modulator display; and

a program product containing software for operating the dual modulator display and generating control values for the pixels of the dual modulator display.

For example, consider the case shown in FIG. 1. The display 10 includes a first array 12 of pixels 12A and a second array 14 of pixels 14A. Controller 16 receives image data 18 and generates control signals 19A and 19B for controlling the pixels of the first and second pixel arrays, respectively. The first array 12 may, for example, comprise an array of Light Emitting Diodes (LEDs) or other light sources. The second array 14 may, for example, comprise an LCD panel.

The controller 16 derives signals 19A and 19B from the image data. The signal 19A may comprise a set of control values d for the pixels 12A of the first array 12. The signal 19B may comprise, for example, a set of control values p for the pixels 14A of the second array 14.

In response to a control signal 19A from the controller 16, the first array 12 emits light. The light emitted by the pixels 12A of the first array 12 produces a pattern 20 of light on the second array 14. Pattern 20 is an approximation of the desired image defined by image data 18. Since the first array 12 has significantly fewer pixels than the second array 14, the aspect of the desired image having the higher spatial frequency will be represented primarily by the control signal 19B. The pattern 20 will track aspects of the desired image having relatively low spatial frequencies.

The characteristics of the pattern 20 depend on the amount of light emitted by the pixels 12A of the first array 12 and the point spread function of the pixels 12A. Optical elements (not shown) such as diffusers, lenses, collimators, and the like may be disposed between the first array 12 and the second array 14. The effect of any such optical element on the pattern 20 can be taken into account by the point spread function of the pixel 12A.

The pattern 20 may be defined by an intensity value B at each pixel 14A of the second array 14_iCharacterization (where i is identifying a particular pixelIndex of 14A). The manner in which control signal 19A is derived from image data 18 may affect the quality of the image produced by display 10. If the control signal 19A results in a "good" pattern 20, the second modulator 14 may modulate the light in the pattern 20 to reproduce the desired image very accurately. On the other hand, if control signal 19A produces a suboptimal pattern 20, it may not be possible to determine a control signal 19B that would cause second modulator 14 to modulate pattern 20 in a manner that matches the desired image without perceptible artifacts.

Consider, for example, the case where the desired image describes stripes that are too high in spatial frequency to be reproduced in the pattern 20 in the region of the desired image. These stripes are superimposed on the background light intensity, which varies at low spatial frequencies. Curve 30 of fig. 2A represents the variation in intensity of the desired image in the direction across the strip. Curve 32 represents the variation in intensity of light in exemplary pattern 20. If the light in the pattern 20 has an intensity that varies in a manner that well matches the low spatial frequency content of the curve 30 (as shown by curve 32), then the second array 14 can be set to produce the high spatial frequency content of the desired image as shown in FIG. 2B, so that the resulting image faithfully reproduces the desired image.

On the other hand, in some cases, the light in the pattern 20 has an intensity that changes in a manner that does not match well with the low spatial frequency components of the curve 30. This may occur, for example, where the point spread function of the pixels 12A of the first array 12 is such that the low spatial frequency of the pattern 20 is lower than the low spatial frequency content of the desired image. Curve 33 in fig. 2C represents the variation in the intensity of light in another exemplary pattern 20, which shows this mismatch between curve 30 and the low spatial frequency content of pattern 20. In this case, as shown in fig. 2D, the second array 14 cannot accurately generate high spatial frequency components of a desired image, so that the resulting image cannot faithfully reproduce the desired image. Loss of high spatial frequency information may occur as a result of quantization of the signal controlling pixel 14A. For example, in B_iSignificantly less than that of pixel 14AWhen the light output of (2) is large, the value p of the pixel 14A_iMay be rounded to zero. In B_iA value p for the pixel 14A of less than or equal to the desired light output of the pixel 14A_iCan be fixed to a maximum value. In either case, the resulting image does not contain all of the information in image data 18.

It can be seen that obtaining the best possible image quality from a dual modulator display of the type described herein, in which the pattern of light from the first array of pixels does not contain the highest spatial frequency content of the image data, may require some optimization of the light pattern 20.

FIG. 3 illustrates steps in a method 40 of obtaining signals 19A and 19B from image data 18, according to an embodiment of the invention. Image data 18 contains valuesSet of (2)In some embodiments, the values are color values. In these embodiments, these values are most conveniently expressed in a color space having the same chromaticity, white point, and primary colors as the display 10 that will display the image.

Block 42 results in an initial set of control signals 19A for the pixels 12A of the first array 12. In an exemplary embodiment, block 42 comprises process 50 shown in FIG. 3A. Process 50 obtains a target light pattern from image data 18Target light patternMay include a value corresponding to each pixel 14A of the second pixel array 14For example, these values may be expressed in photometric units. In some embodiments, the valueA monochrome value in photometric units such as brightness. The frame 52 may contain fixed luminance values so that they do not fall outside the displayable range of the display 10 where the image is to be displayed.

If image data 18 specifies a color image, block 53 may include extracting a monochrome (single channel luminance) representation of image data 18. This may be achieved by taking a maximum of three color channels (e.g., red, green, and blue) for each pixel of the second pixel array.

Block 54 is determined by calculating a function of the monochrome rendering of image data 18The value of (c). In some embodiments, the function includes determining a fractional power of the value of a corresponding pixel in a monochrome representation of image data 18. The power is selected to allocate dynamic range between the first array and the second array. In some embodiments, block 54 comprises taking a square of the value (1/2 th power) in a monochrome representation of image data 18. The optimal division of the dynamic range between the first and second pixel arrays will depend on the ratio of the dynamic range between the first array and the second array. In some dual modulator displays, such as the DR 34P type display available from Brightside Technologies, Inc. of Vancouver, Canada, the ratio of the dynamic range between the first array and the second array is about 1: 1, and powers of about 1/2 are a good basis for obtaining signal 19A.

A single light source that can be completely turned off can be considered to have an infinite dynamic range. However, the dynamic range of a light source within any set of light sources having overlapping point spread functions is determined by the point spread functions of the light sources and the light output of neighboring light sources.

For the case where one of the arrays is given a larger pixel value and the other is given a very small value because the quantized artifacts are relatively larger for smaller values, it is preferable to maintain the pixel values on the first and second pixel arrays to be of the same order, with the other factors being the same. Also, if different combinations of values are used for adjacent pixels of the same intensity, defective alignments that exist in real hardware systems can result in noticeable artifacts.

For calculation purposes, in the range [0, 1 ]]In the middle ofAndthe value of (c) is very convenient. The conversion to such a representation can be done by normalizing the image data. In block 54, an appropriate function of the normalized image data may be calculated. The results may then be scaled to provide in any desired unitsThe value of (c).

In the case where the first array 12 has n pixels 12A and the second array 14 has m pixels 14A, with m > n, the problem of determining the values of the pixels 12A of the array 12 that will result in the desired pattern 20 can be expressed as a system of m n equations. In the case where the first array 12 has a low spatial frequency (i.e., if the point spread function of the first array 12 does not have high spatial frequency content), then the pattern 20 desirably should also have a low spatial frequency. In this case, the product can be obtained by mixingDown-sampling (down sampling) to a lower resolution (it is convenient to have the lower resolution be the same as the resolution of the first array 12) reduces the computation without introducing noticeable artifacts. As a result, the problem can be expressed as a system of n × n equations. Therefore, the temperature of the molten metal is controlled,the value of (A) may comprise n valuesSet of (2), individual valueFor each group of pixels 14A of the second array 14 that is closest to the corresponding pixel 12A of the first array 12.

In block 56, the resulting data is down-sampled to a resolution that may be the same as the resolution of the first array 12. The down-sampling may be implemented in various ways. For example,

● the down-sampling may be implemented in software by any suitable filtered resizing function.

● the down-sampling may be implemented in logic circuitry such as a suitably configured Field Programmable Gate Array (FPGA) as an average of pixel values in the vicinity around a location corresponding to the location of the pixel in the first pixel array.

● downsampling may be implemented in a graphics processor by recursively averaging blocks of pixel values.

Block 58 returns to defining the desired light pattern 20At the resolution of the first array 12.

The method 40 continues in block 44 to determine control values for the pixels of the first array 12 that will result in the light pattern 20 having values corresponding to those returned by block 58Similar or identical values. This can be achieved by solving the minimization problem:

where d is the set of control values for pixel 12A and W is the convolution of the point spread function of pixel 12A with the dirac delta function at the location of pixel 12A.

By considering the pixels of the first array 12 in a neighborhood smaller than the array 12, rather than finding all the pixels of the first array 12 at once, the amount of computation can be reduced. This may be done because, in a typical dual modulator display, the pixels 12A of the first array 12 do not contribute much light at the pixels 14A of the second array 14 that are co-ordinately distant from the pixels 12A.

Also, the performance of the human visual system and the dynamic range of the second pixel array 14 limit which pixels of the first array 12 can be adjusted to obtain a desired amount of light on the pixels 14A of the second pixel array 14. The human visual system cannot detect changes in brightness of less bright areas that are very close to brighter areas. This effect is known as "veil glare". Due to the light scattered in the eyes, the screen glare partially occurs.

Fig. 4 shows veil glare. The desired image is represented by line 75. Image 54 has a bright area 75A adjacent to a dim area 75B. The light emitted by the three pixels 12A is represented by curves 77A, 77B and 77C. The veil glare caused by the bright areas 75A affects the area 76. Within region 76, the human eye cannot discern differences in luminance below level 76A (decreasing with distance to bright region 75A). Outside the area 76, an increase in the intensity of the first array 12 beyond the local desired value is detected. The pixels of the first array 12 within the area 76 may have increased intensity without imperceptibly altering the image. For example, the light output 77B may rise to the value represented by 77B 'without visually affecting the resulting image (since the peak of 77B' is still lower than the curve 76A).

Although the weighting matrix W (see equation (1)) is dense, it can be freely changedThe number of pixels 12A to control the amount of light on a given pixel 14A of the second pixel array 14 is quite limited. The resulting matrix of control values d is a relatively sparse, banded matrix that may be solved accurately or approximately in any suitable manner to obtain control signal values d for pixels 12A_j(j is an index identifying the particular pixel 12A).

In one embodiment, d_jGiven by:

wherein the content of the first and second substances,is the value of the target light pattern on pixel 14A of the second array 14 in the region corresponding to pixel 12A (jth pixel) identified by the index j;is the value of the target light pattern on pixel 14A of the second array 14 in the region corresponding to pixel 12A (ith pixel) identified by index i; n (delta)_j) Contains the pixel 12A in the jth pixel neighborhood; w is a_jjIs the value of the point spread function of the jth pixel on pixel 14A in the region corresponding to the jth pixel, w_jiIs the value of the point spread function for the ith pixel on pixel 14A in the region corresponding to the jth pixel.

The light pattern 20 resulting from the obtained control signal may not be optimal as explained above. The actual light pattern is characterized by a value B corresponding to pixel 14A. In general terms, the amount of the solvent to be used,in general, d can be fine-tuned_jTo produce a light pattern that better approximates the optimal light pattern, improving image quality. Also can be adjusted by fine tuning d_jTo allow more bit depth of the second array 14 to be applied to represent higher spatial frequency detail and color, improving image quality. The corrected control value is determined in block 46 of method 40.

Suppose that it was obtained in block 42Is relatively close to the optimum value, then, for the sake of optimization, d_jOnly a small change in the value of (c) is required. This change is referred to herein as Δ d_j. These variations may be determined by applying a "greedy" algorithm that processes pixels 12A one at a time. The algorithm attempts to reduce B and B at the location corresponding to pixel 12AThe difference between them.

In some embodiments, Δ d may be determined for the jth pixel of the first array 12_jSo that when the jth pixel has the control value d_j+Δd_jSecond matrix in the region corresponding to the jth pixelThe pixels 14A of column 14 may have control values p that are not rounded or fixed. This allows the second array 14 to faithfully reproduce the high frequency spatial components of the desired image without loss of information due to quantization of the signal 19B.

In one embodiment, the algorithm includes determining Δ d satisfying the following equation_jThe value of (c):

where α is a constant (identifiable by a desired average value of the control signal for the second pixel array 14);w and d are as defined hereinbefore; w_jIs the convolution of the point spread function of the currently processed pixel 12A with the dirac delta function at the location of that pixel 12A.

If Wd in equation (3) is replaced with B, then the problem can be expressed as finding a solution of:

wherein, B^(j)Is a set of values characterizing the light pattern 20 (including any change Δ d of the previously processed pixels 12A)_jThe effect of (d).

Expression (4) can be expressed in terms of coordinates (x, y) of the pixel 12A to obtain:

wherein S is_j(x, y) is texture splat processing (texture splat) of the point spread image of the j-th pixel 12A at the (x, y) position; m_j(x, y) is a masking function (masking function) having a value of 1 in a region around the position of the pixel 12A currently under consideration and a value of 0 in other regions (the region may be, for example, a circular region having a defined radius). May be Δ d_jSolving equation (5) to obtain:

can be determined from the inclusion of any Δ d_jD (for computational convenience, in the range 0, 1)]Middle expression) calculation of B^(j). Since the point spread function of pixel 12A generally varies with low spatial frequencies, B can be implemented at reduced resolution^(j)And then up-sampled (upsample). In doing so, it is desirable to ensure that the pixels of the reduced resolution image are aligned with the pixels 12A to avoid rounding errors.

B can be calculated in various ways^(j). For example, in some embodiments, B is calculated as a convolution^(j). In some such embodiments, individual pixels at the location of the convolved pixel 12A with the point spread function of the pixel 12A calibrated in photometric units are set to appropriate control values (d)_jOr d + Δ d_j) All black images (pixel values are all zero). Embodiments may be conveniently implemented in software.

In other embodiments, the amount of light on each pixel 14A may be determined by the following calculations, including: calculating the distance from the pixel 14A to the contributing pixel 12A; and, for each such contributing pixel 12A, the corresponding distance is looked up in the table to obtain the value of the point spread function of contributing pixel 12A on pixel 14A. By the current control value (d) of the appropriate pixel 12A_jOr d + Δ d_j) The values of the point spread functions are modulated (e.g., multiplied). Embodiments may be conveniently implemented in logic circuitry, such as a suitably configured FPGA.

Other embodiments may apply Splatting and will applyThe screen-aligned quadrilateral with the texture of the applicable point spread function is dragged into the frame buffer. Under appropriate circumstances, by means of corresponding control values (d)_jOr d + Δ d_j) Each texture is modulated. Alpha blending may be applied to accumulate the results. The embodiments may be conveniently implemented in a Graphics Processor Unit (GPU).

The tail of the point spread function applicable to pixel 12A may be very long (i.e. light from a particular pixel 12A will reach pixel 14A relatively far from pixel 12A (in terms of the coordinate space of the first and second pixel arrays). the trade-off between accuracy and computational expense may involve truncating the point spread function at some reasonable distance.

Truncation of the point spread function may result in the pixel 14A outside the truncation distance being less computationally intensive than it should be. While not noticeable when compared to the peak brightness of the display, this difference can contribute to a perceptible mismatch in dark areas. Since the spatial frequency of the truncated part of the point spread function is very low, it is possible to compensate by adding a u term to each pixel of the backlight image to represent light that is not considered as a result of the truncation. The value of u may be selected as a fraction (or other suitable function) of the set of control values d. The appropriate value of u may be based on the energy difference between the actual point spread function and the truncated simulation of the point spread function.

The process for solving equation (6) may include operating on pixels 12A in the order of the scan lines (i.e., starting at one corner of first pixel array 12 and working along one row of pixels at a time). Propagating the image S for the point corresponding to the current pixel 12A_j，And B are selected and their elements are multiplied and then added together to get Δ d_j. Then, the corresponding control value d + Δ d_jIs written as a control value d and is therefore determined by the value deltad_jS_jAdding up to B modifies B.

The above process may be iterated two or more times, if desired, to further refine B.

The above-described calculation of the control value d for the improved pixel 12A assumes that the correct set of values B for the actual light pattern is known (e.g., equation (6) contains B). However, in almost all cases, Δ d is performed for any particular pixel 12A_jAt the time of the calculation of (2), Δ d has not been calculated for the other pixels 12A_j. Even if Δ d_jIs generally smaller (as long as it is obtained in block 42)A good approximation of) the accumulated error may also be significant.

If the pixels 12A are processed in a known order, so that the processed Δ d can be processed_jAnd Δ d has not been processed yet_jCan compensate for the error. For example, consider the case where the pixels 12A are processed in scan line order starting at the upper left corner of the first array 12. In this case, the pixels 12A above and to the left of the current pixel 12A have been updated, while the pixels 12A below and to the right of the current pixel 12A have not been updated. Even though Δ d for some k > j_kOf unknown, desired imagesAnd B^(k)The values of (c) are also known. It can be assumed that the control value of the kth pixel 12A will change such that the amount of change of the light emitted by the kth pixel 12A is equal toAnd B^(k)The difference between them.

Image filtering to perform corrective measures may be added to equation (5) to yield:

where F is image filtering. For Δ d_jSolving equation (7) to obtain:

for example, equation (8) can be solved as described above. All that is required is that the filter function F has a positive value of the pixel 12A that has been corrected (i.e. for which deltad_jHas been calculated and added to the corresponding control value) and the negative value of the pixel 12A that has not yet been corrected.

The value of alpha may be set to adjust the displayed image. A value of 1 will result in pixel 12A having an andsame intensity, resulting in the operation as close as possible to the targetAnd (6) matching. Error diffusion may be performed as a second iteration to achieve the desired image. A value of 5 results in a brightness of the first array 12 ofIs doubled, thereby resulting in p_avg0.5. This generally maximizes the number of bits in the control values of the pixels 14A available for correction and minimizes quantized artifacts originating from the control values p of the second array 14.

More complex schemes, such as selecting the value of alpha based on local proximity, are also possible and may be used to provide a characteristic specific tone scale of the light pattern 20.

A control value d may be supplied in signal 19A to drive a corresponding pixel 12A of the first array 12 of the display 10. The control value p for the pixel 14A of the second array 14 may be determined, for example, by:

● estimating the distribution B of light in the pattern 20 resulting from applying the control value d to the pixels of the first array 12; and

● the control value p is calculated according to:

fig. 5 shows an exemplary flow chart of data in an exemplary controller 16 for controlling the display 10. Can be used in general data processingSoftware executing on a processor, logic (e.g., a configured FPGA), a graphics processor, or some combination thereof implements the functional blocks in figure 5. Image data 18 is received in controller 16. Extracting desired images from image data 18The desired light pattern generator 60 generates a desired light pattern to be generated by the first array 12The first array control value generator 62 generates the control value d. The first array control value generator 62 may generate the control value d and the adjustment amount Δ d according to the method described above.

The control signal generator 64A generates the control signal 19A supplied to the first array control circuit 65A. The first array control circuit 65A operates the pixels of the first array 12 according to the first array control value d. The control signal 19A may be provided directly to the first array control circuit 65A or transmitted to the first array control circuit 65A after a delay. For example, the signal 19A may be recorded on a medium (not shown) and played back to the first array control circuit 65A at a later time.

The first array control value d is also supplied to the light pattern simulator 66. Light pattern simulator 66 determines a simulated light pattern B. The second array control value generator 68 is based on the simulated light pattern B and the desired imageA second array control value p is generated. For example, the light pattern B determined by the simulator 66 may also be used by the first array controller value generator 62 to generate a light pattern for use in, for example, calculating Δ d as described above_jB of (A)^(j)。

The control signal generator 64B generates a control signal 19B supplied to the second array control circuit 65B. The second array control circuit 65B operates the pixels of the second array 14 according to the second array control value p. As explained above with reference to signal 19A, data 19B may be provided to second array control circuit 65B directly or after a delay.

Fig. 6 illustrates an exemplary system 70 for generating control signals for a first array of pixels in a dual modulator display. The functional blocks in fig. 6 may be implemented in software executing on a general purpose data processor, logic circuitry (e.g., a configured FPGA), a graphics processor, or some combination thereof. Desired imageIs supplied to the initial control value generator 72. The initial control value generator 72 generates an initial control value d. The initial control value generator 72 may generate the control value d according to the method described above. The initial control value generator 72 may generate a desired light pattern in generating the initial control value d

The control value d is provided to the light pattern simulator 74. The light pattern simulator 74 determines a simulated light pattern B. Simulated light pattern B and desired imageTogether, are provided to the control value adjustment generator 76. The control value adjustment generator 76 generates a control signal adjustment amount Δ d. The control value adjustment generator 76 may generate each control value d according to the method described above_jControl signal adjustment amount deltad of_j. The control signal adjustment Δ d is combined with the initial control value d to produce an adjusted control value d + Δ d.

Certain implementations of the invention comprise a computer processor executing software instructions that cause the processor to perform the methods of the invention. For example, one or more processors in a display controller or one or more processors in a device that outputs a signal containing control signals 19A and 19B for use by a dual modulator display may implement the methods of fig. 3 and 3A by executing software instructions in a program memory accessible to the processors. The present invention may also be provided as a program product. The program product may comprise any medium carrying a set of computer-readable signals comprising instructions which, when executed by a data processor, cause the data processor to carry out the method of the invention. The program product according to the invention may take any of a wide variety of forms. The program product may comprise, for example, media such as magnetic data storage media including floppy disks, hard disk drives, optical data storage media including CD ROMs, DVDs, or electronic data storage media including ROMs, flash RAMs, etc. The computer readable signal on the program product may optionally be compressed or encrypted.

Where a component (e.g., a software module, processor, assembly, device, circuit, etc.) is referred to above, unless otherwise indicated, reference to that component (including a reference to a "means") should be interpreted as including as equivalents of that component any component which performs the function of the described component (i.e., that is functionally equivalent), including components which are structurally equivalent to the disclosed structure which performs the function in the illustrated exemplary embodiments of the invention.

It will be readily appreciated by those skilled in the art, in view of the foregoing disclosure, that numerous variations and modifications are possible in the practice of the invention without departing from the spirit or scope thereof. For example,

● may generate control signals 19A and 19B in real time in response to image data 18 (which may, for example, contain video data), or these control signals may be generated in advance.

Accordingly, the scope of the invention should be construed in accordance with the substance defined by the following claims.

Claims (19)

1. A method for generating control values for pixels in a first array of pixels in a dual modulator display, the dual modulator display comprising a first array of pixels and a second array of pixels illuminated by the first array of pixels, the first array of pixels being configured to generate a controllable pattern of light on the second array of pixels, the method comprising:

generating a desired light pattern and an initial set of control values from image data defining a desired image; and

tending to reduce the correlation with a pixel by determining for that pixelA control value adjustment amount Δ d of a difference between a desired light pattern at a position corresponding to the pixel and a light pattern estimated to be generated in response to the control value set_jThe initial set of control values is improved,

wherein, Δ d_jIs a solution of the following formula or its mathematical equivalent:

wherein the content of the first and second substances,is the desired brightness of the image; wd is the light pattern estimated to be generated in response to the set of control values; α is a constant; and W_jIs the convolution of the point spread function of a pixel with the dirac delta function located on said pixel.

2. The method of claim 1, wherein refining the set of control values comprises performing the following process sequentially for a plurality of pixels:

determining a control value adjustment amount Δ d by determining, for the pixel, a control value adjustment amount Δ d that tends to reduce a difference between a desired light pattern at a point corresponding to the pixel and a light pattern estimated to be generated in response to the set of control values_jAnd processing the pixel.

3. The method of claim 2, wherein the processing of pixels is performed in scan-line order.

4. The method of claim 1, wherein a has a value equal to the midpoint of the optimal range for the set of control values p for the pixels in the second pixel array.

5. The method of claim 1, wherein a is 0.5.

6. The method of claim 1, wherein a has a value selected based at least in part on a desired light pattern in a local vicinity of the pixel.

7. The method of claim 2, wherein Δ d is determined_jComprises the following steps: filtering is applied that distinguishes between pixels that have been processed and pixels that have not yet been processed.

8. The method of claim 7, wherein applying filtering comprises: multiplying by a filter function having one sign for pixels that have been processed and the opposite sign for pixels that have not been processed.

9. The method of claim 1, wherein the refining of the initial set of control values is performed in two or more iterations.

10. The method of claim 1, wherein the image data comprises video data and the method is performed in real-time.

11. The method of claim 2, comprising determining a light pattern estimated to be generated in response to the set of control values at a resolution lower than a resolution of the image data.

12. The method of claim 2, comprising determining the light pattern estimated to be generated in response to the set of control values by convolving the intensities of the pixels with the set of control values provided by their point-spread functions.

13. The method of claim 12 including truncating the point spread function of at least one of the pixels and, for each pixel having a truncated point spread function, adding a value u to the light pattern estimated to be generated in response to the set of control values for pixels outside the truncation boundary to represent light not considered as a result of the truncation.

14. The method of claim 1, wherein Δ d_jIs determined by the following formula or its mathematical equivalent:

wherein S is_jIs a texture splatter treatment of the point spread image of the pixel; m_jIs a masking function with a value of 1 in the region around the location of the pixel and a value of 0 in the other regions; and B^(j)Is a value estimated as a light pattern generated at the pixel in response to a set of control values.

15. The method of claim 1, comprising generating the desired light pattern and the initial set of control values such that an intensity of the light pattern estimated to be generated on a pixel in the second array of pixels exceeds an intensity of a corresponding location of the desired image by an amount within a modulation range of the pixel of the second array of pixels.

16. A system for generating control values for pixels in a first pixel array in a dual modulator display, the dual modulator display comprising a first pixel array and a second pixel array illuminated by the first pixel array, the first pixel array configured to generate a controllable pattern of light on the second pixel array, the system comprising:

an initial control value generator for generating an initial set of control values based on the desired image;

a light pattern simulator for generating a simulated light pattern that is desired to be generated when the first pixel array is controlled with the initial set of control values; and

control value adjustment Δ d for generating, for a pixel, a control value adjustment amount that tends to reduce a difference between a simulated light pattern at a position corresponding to the pixel and a desired image_jThe control value of (2) adjusts the amount generator,

wherein, Δ d_jIs a solution of the following formula or its mathematical equivalent:

wherein the content of the first and second substances,is the desired brightness of the image; wd is the simulated light pattern; α is a constant; and W_jIs the convolution of the point spread function of a pixel with the dirac delta function located on said pixel.

17. The system of claim 16, wherein the initial control value generator generates the desired light pattern prior to generating the initial set of control values.

18. The system of claim 16, wherein the control value adjustment generator generates a set of control value adjustments Δ d for a plurality of pixels in the first pixel array.

19. The system of claim 18, wherein the control value adjustment generator sequentially generates the set of control value adjustments Δ d.