HK1229897B - Light recycling for projectors with high dynamic range - Google Patents

Description

Light recycling for projectors with high dynamic range

Cross Reference to Related Applications

This application claims priority from U.S. provisional priority patent application No.62/018,024, filed on 27/6/2014, which is hereby incorporated by reference in its entirety.

Technical Field

The present invention relates to light recycling (recycling) for projection systems, and in particular to systems and methods for High Dynamic Range (HDR) projection systems.

Background

Projection systems are now improved in terms of dynamic range. Dual and multi-modulator projector display systems are known in the art. However, additional improvements in both rendering and performance of such display systems resulting from improved modeling of light processing in such display systems are possible. Furthermore, as the inventors have recognized, it would be desirable to increase the energy performance of single modulation display systems as well as dual/multi modulation display systems.

Disclosure of Invention

Projection systems and/or methods for efficient use of light by recycling a portion of the light energy for future use are disclosed. In one embodiment, a projector display system is disclosed that includes a light source; an integrating rod receiving light from the light source at a proximal end, the integrating rod including a reflective surface that can reflect/recycle light along the integrating rod; and a modulator comprising at least one movable mirror that reflects light received from the integrating rod in a projection direction or a light recycling direction. In other embodiments, dual and multi-modulator projector display systems are disclosed. The first modulator may implement a pre-modulated halftone image or may implement a highlight modulated image for the desired image to be displayed. A second modulator may be provided for primary modulation of the desired image.

In one embodiment, a projector display system capable of recycling light from a light source is disclosed comprising: a light source; an integrated rod configured to receive light from the light source at a proximal end and wherein the proximal end comprises a reflective surface capable of reflecting light along the integrated rod; and a modulator comprising a movable mirror capable of reflecting light received from the integrating rod in at least one of a projection direction and a light recycling direction, wherein the light recycling direction is substantially in the direction of the integrating rod.

Embodiments for controlling light recycling in response to image characteristics are also presented.

Other features and advantages of the present system are set forth in the following detailed description, which proceeds with reference to the accompanying figures, which are presented in this application.

Drawings

Exemplary embodiments are shown in the drawings. The embodiments and figures disclosed herein are intended to be considered illustrative rather than restrictive.

FIG. 1A depicts a dual modulator projector display system with a light recycling module made in accordance with the principles of the present application and shown schematically.

FIG. 1C depicts a projector display system including a light recycling module on multiple color channels.

Fig. 2 depicts one embodiment of a light recycling module sufficient for the purposes of the present application.

Fig. 3 shows the proximal end of an integrated rod suitable for the purposes of the present application.

Fig. 4 depicts another embodiment of a dual/multi-modulator projection system in which it may be possible and/or desirable to perform light recycling in accordance with the principles of the present application.

Fig. 5 depicts another embodiment of a projection system in which light recycling according to the principles of the present application may be possible and/or desirable.

Fig. 6A and 6B schematically depict many possible embodiments of a projection system that may provide one or more opportunities for light recycling according to principles of the present application.

Fig. 7A is one possible light recycling control system and/or method for a single modulation projector display system.

Fig. 7B and 7C depict response curves and response tables, respectively, for individually modulated color responses of conventional DMD components.

Fig. 8 depicts another possible light recycling control system and/or method for a single modulation projector display system.

Fig. 9 depicts yet another possible light recycling control system and/or method for a single modulation projector display system.

Fig. 10 depicts one possible response table for light recycling for a given illumination pattern.

Fig. 11, 12 and 13 depict three algorithms for efficient light recycling in a display system capable of light recycling.

Fig. 14 depicts an alternative embodiment of a light recycling module in a dual modulator display system.

Detailed Description

As used herein, the terms "component," "system," "interface," and the like are intended to refer to a computer-related entity, either hardware, software (e.g., in operation), and/or firmware. For example, a component may be a process running on a processor, an object, an executable, a program, and/or a computer. By way of example, both an application running on a server and the server can be a component. One or more components can reside within a process and a component can be localized on one computer and/or distributed between two or more computers. Components may also be intended to refer to communication-related entities, being hardware, software (e.g., on the fly), and/or firmware, and may additionally include sufficient wired or wireless hardware to enable communication.

Throughout the following description, specific details are set forth in order to provide a more thorough understanding to persons skilled in the art. However, well known elements may not have been shown or described in detail to avoid unnecessarily obscuring the disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Introduction to the design reside in

In the field of projectors and other display systems, it is desirable to improve both image rendering performance and system efficiency. Several embodiments of the present application describe systems, methods, and techniques to achieve these improvements by applying light field modeling for dual or multi-modulation display systems. In one embodiment, a light source model is developed and used for beneficial effects. A camera picture of a displayed image of a known input image may be evaluated to improve the light model. In some embodiments, the iterative process may accumulate improvements. In some embodiments, these techniques may be used on moving images for real-time adjustment to improve image rendering performance.

Dual modulation projector and display systems are described in commonly owned patents and patent applications, including:

(1) U.S. patent No. 8,125,702 to Ward et al, entitled "SERIAL MODULATION DISPLAY vehicle LIGHT MODULATION STAGE", issued on 28/2/2012 (issue);

(2) U.S. patent publication No.20130148037 to Whitehead et al, entitled "product DISPLAYS", published (publich) 6/13/2013;

(3) U.S. patent publication No.20110227900 to Wallener entitled "CUSTOM PSFs USING CLUSTERED LIGHT SORCES" published on 22.9.2011;

(4) U.S. patent publication No.20130106923 to Shields et al, entitled "SYSTEMS AND METHODS FOR ACCURATERATELYREPRESSING HIGH CONTRAST IMAGE ON HIGH DYNAMIC RANGE DISPLAY SYSTEMS," published ON 5/2/2013;

(5) U.S. patent publication Nos. 20110279749 AND 20110279749 to Erinjippurath et al, entitled "HIGH DYNAMIC RANGE DISPLAYS USINGFILTERLESS LCD (S) FOR INCREASING control AND RESOLUTION", published on 17.11.2011

(6) U.S. patent publication No.20120133689 to Kwong, entitled "REFLECTOR WITH SPATIALLY VARYINGREFLECTANCE/ABSORPTION GRADIENTS FOR COLOR AND LUMINANCE COMPENSATION", published on 31/5/2012.

All of these are incorporated herein by reference in their entirety.

An exemplary physical architecture

Generally, a projector having a single Digital Micromirror Device (DMD) may tend to have limited contrast. To obtain greater contrast, two or more DMDs and/or other reflectors (e.g., MEMS) may be arranged in series. Because a DMD may operate as a time-division or pulse-width modulator, operating two or more DMDs and/or reflectors in series, both as pulse-width modulators, often requires precise time-division alignment of time-division sequences and pixel-to-pixel correspondence. Such alignment and corresponding requirements may be difficult in practice. Thus, in many embodiments of the present application, the projector and/or display system may apply different dual modulation schemes to achieve the desired performance.

For but one example, one embodiment of a projector display system may use a first modulator (e.g., a first DMD/reflector) as a "pre-modulator" or "pre-modulation" — the modulator may spatially modulate a light source with a halftone image that may be maintained for a desired period of time (e.g., a frame or portion thereof). This halftone image may be blurred to produce a spatially reduced bandwidth light field that may be applied to a second DMD/reflector. A second DMD/reflector, referred to as the primary modulator, may pulse width modulate the blurred light field. This arrangement may tend to avoid both of the above-mentioned requirements-e.g., precise time division alignment and/or pixel-to-pixel correspondence. In some embodiments, two or more DMD/reflectors may be frame aligned in time and approximately frame aligned in space. In some embodiments, the blurred light field from the pre-modulated DMD/reflector may substantially cover the primary DMD/reflector. In other embodiments, spatial alignment may be known and considered, for example, to aid in image rendering performance.

Although the present application is presented in the context of a dual, multi-modulation projection system, it should be appreciated that the techniques and methods of the present application will be applicable to single modulation or other dual, multi-modulation display systems. For example, in the context of a projection system, a dual modulation display system including a backlight, a first modulator (e.g., LCD, etc.), and a second modulator (e.g., LCD, etc.) may apply suitable blurring optics and image processing methods and techniques to achieve the performance and efficiency discussed herein.

It should be appreciated that although fig. 1A depicts a two-level or dual modulator display system, the methods and techniques of the present application may also be applied in display systems having only one modulator or display systems having three or more modulators (multi-modulator). The scope of the present application encompasses these various alternative embodiments.

Fig. 1A shows one possible embodiment of a dual/multi-modulator projector display system 100 that may be sufficient for the purposes of the present application. The projection system 100 employs a light source 102, which light source 102 supplies the desired illumination to the projection system so that the final projected image will be sufficiently bright for the intended viewer of the projected image. The light source 102 may include any suitable light source possible including, but not limited to, a xenon lamp, laser(s), coherent light source, partially coherent light source. Since the light source is the main consumption (draw) of power and/or energy of the entire projection system, it may be desirable to advantageously use and/or reuse the light in order to save power and/or energy during its course of operation.

The light 104 may illuminate a first modulator 106, which first modulator 106 may in turn illuminate a second modulator 110 via a set of optional optical components 108. Light from second modulator 110 may be projected by a projection lens 112 (or other suitable optical component) to form a final projected image on a screen 114. The first and second modulators may be controlled by a controller 116-the controller 116 may receive input image and/or video data. The controller 116 may perform certain image processing algorithms, gamut mapping algorithms, or other such suitable processing on the input image/video data and output control/data signals to the first and second modulators to achieve the desired final projected image 114. Furthermore, in some projection systems, depending on the light source, it may be possible to modulate the light source 102 (control lines not shown) in order to achieve additional control over the image quality of the final projected image.

The light recycling module 103 is depicted in fig. 1A as a dashed box that may be placed in the optical path from the light source 102 to the first modulator 106, as will be discussed below. Although the present discussion will be given in the context of this positioning, it should be appreciated that light recycling may be inserted into the projection system at various points in the projection system. For example, light recycling may be placed between the first and second modulators. Furthermore, light recycling may be placed at more than one point on the optical path of the display system. While such embodiments may be more expensive due to the increased number of components, such an increase may be balanced against the energy cost savings due to light recycling at multiple points.

FIG. 1B depicts one embodiment of a projector display system 100B that includes a single modulator 106B. Light 102b is emitted (possibly under control of a controller-not shown) and light beam 104b may be transmitted through light recycling module 103b, as previously described. The modulator 106b may selectively reflect light as desired by the controller-and the modulated light 108b may be transmitted through projection optics 112b and projected onto a screen 114 as the final desired image to be viewed.

Fig. 1C depicts one embodiment of a light recycling module that can perform light recycling on multiple color laser channels (e.g., R, G and B). As can be seen in this example, the display system may include a red light source (R) (e.g., 124 for B and 122 for G) incident on an integration rod 126 that may be transmitted (possibly via internal reflection) to a controllable reflector 120 that may include one or more reflectors that may assume either a recycling position 120B or a transmitting position 120 a. If the light is to be recycled, reflector 120b will reflect the laser light back into integration rod 126, which may reflect multiple times within that path, until the reflector is commanded (via a controller, not shown) to transport position 120 a. As shown, light transmitted by reflector 120a may be directed to red mirror 128. In the case of blue light, the blue light may be combined with the red light at the dichroic combiner 130. Similarly, the green light may thereafter be combined at the dichroic combiner 132, and the light may then be further modulated and/or projected — as simply depicted by the optical element 120. It should be appreciated that this light recycling module may be sufficient for the purpose of implementing a single modulator, dual modulator, and/or multi-modulator display system as desired.

Light recycling embodiment

Fig. 2 depicts one embodiment of a light recycling subsystem and/or module that may be suitable for the purposes of the present application. As discussed above, this light recycling subsystem/module may be placed in the projection system, primarily between the light source 102 and the first modulator 221. Light from the light source 102 may be input to the optical path via an integrated rod/tube/cartridge 202 (e.g., via port 201b, as seen in fig. 3). The integrated rod/tube/cartridge 202 may comprise a substantially reflective surface on its interior such that light incident on its surface may be reflected (e.g., possibly multiple times) until it exits from its extreme right end 203 (exit). Once the light exits the integrated rod/tube/box, the light may be placed in an optical path defined by a set of optical elements, such as lenses 204, 214, and 216, and a set of filters and/or polarizers 208, 210, and 212.

The first modulator 221 may include a reflector 220 and a plurality of prisms 218a, 218 b. Reflector 220 may include a DMD array of reflectors, or a MEMS array-or possibly any other suitable collection of reflectors that may reflect light in at least two or more paths. One such path is depicted in fig. 2. As can be seen, reflector 220 directs light onto the interface of prisms 218a and 218b such that the light is reflected therefrom into lens assembly 222 and thereafter to a second modulator 229 (e.g., comprising lens assembly 224, prisms 226 and 230, and reflector 228). This light may be applied to form the final projected image to be viewed by the viewer.

However, at some time during the rendering of the final projected image, full power/energy of the light source 102 may not be required. If the power of the light source 102 cannot be modulated (or if it is difficult, or if there is another opportunity to conserve light), it may be desirable to recycle the light from the light source 102. In this case, and as can be seen in fig. 2, the reflector 220 can be aligned from its illustrated current position (i.e., light is directed to travel along a path to the second modulator) to an alternate position where the light will be reflected back to the integrator rod/tube/box 202 along substantially the same path described as traveling in a right-to-left direction.

In another embodiment, a third (optional) path (not shown) allows the reflector to direct light from the light source to a light "dump", i.e., the portion of the projection system where the light is absorbed. In this case, the light is wasted as heat dissipated from the projection system. Thus, the projection system may have multiple degrees of freedom when directing light as desired.

Fig. 3 illustrates one embodiment of a proximal end 201 (i.e., the end closest to the light source) that helps achieve light recycling. As can be seen, light may travel back to the proximal end 201 through the integrated rod/tube/cartridge 202 (e.g., via multiple reflections). The proximal end 201 may also include a rear portion 201 a-the rear portion 201a may also include a reflective surface-and a port opening 201b where light from the light source 102 may be input to the projection system. Light striking the back portion 201a may be reflected back along the integrating rod 202 (possibly multiple reflections until the reflector(s) at the first modulator are oriented to transmit light to the second modulator or some other suitable optical path to form the final image). The examples of fig. 2 and 3 may be viewed as one example of a light recycling module (e.g., other examples given herein) capable of recycling light at some point in the light path through the display system.

FIG. 14 is another embodiment of a light recycling module 1400-the light recycling module 1400 can act as a module for at least one laser and/or partially coherent color light source 1402, 1404, 1406. Light from such a source may be transmitted through first optical subsystem 1408 to adjust the light to be input into integrating rod 1412-integrating rod 1412 may include a reflective proximal end 1410, as shown in fig. 3. The second optical subsystem 1414 may further condition the light as desired before the light is input to the first modulator 1416. As with fig. 2 and 3 above, this first leg (leg) of module 1400 may implement an optical recycling mode, as discussed.

After the first modulation, the light may be transmitted through a third optical subsystem 1418 before being input to a second modulator 1420 — the second modulator 1420 modulates the light for transmission through a projection optics subsystem 1422 in order to project a final image for viewing.

Highlight embodiment

In one embodiment, the optional highlight modulator may use a small fraction of the available light to achieve adjustable illumination unless it is combined with a pre-modulator. To accomplish this, both beam steering (beam steering) techniques and mechanical and/or non-mechanical subsystems may be applied — for example, it may be possible to steer portions of the illumination source to various paths in the system using mechanical steering, holograms with spatial light modulators, or other spatial modulation methods. It may be desirable to use such a system to improve efficiency by diverting light to a desired location.

Mechanical beam steering may use a collection of reflective elements that can be controlled over a range of motion in the horizontal and/or vertical directions. These reflective elements direct light that reaches them to the desired area of the modulator behind the high brightness modulator that produces controlled non-uniform illumination.

Non-mechanical beam steering methods may use a spatial light modulator to deflect the phase of the uniform coherent light reaching the modulator. When imaged through a lens, the phase deflected light produces a three-dimensional light field. The three-dimensional light field can be imaged into a two-dimensional light field, wherein planes other than the shrinking dimension are imaged by varying sharpness or PSF properties onto one of the following modulators that generates the two-dimensional light field.

Regardless of the implementation, highlight modulation refers to the use of a modulator to divert light that reaches it to any position on a subsequent modulator. Although there may be limitations (such as location range and granularity), the term "any location" may still be used to distinguish the highlight modulator from other modulators.

Depending on the number of highlight modulation elements, the total coverage that can be achieved by the highlight modulator, and the PSF properties, it may not be necessary to have a premodulation/first modulator between it and the main/second modulator in some embodiments. In some embodiments, it is possible that the highlight modulator may have such a capability that it does not require any modulation (pre-modulation or main modulation) after it.

Highlight to pre/main relay optics control

In some embodiments, it may be possible to adjust the relay optics to control the point spread function shape generated by the highlight modulator to the illumination on the pre-modulator/first modulator or the main/second modulator. In some embodiments, there may be controls to adjust the full width at half maximum size as well as to control the shape or tail of the PSF. Predicting, monitoring, and/or measuring the resulting performance when light recycling is applied may be desirable because additional passes through the integration rod will change the light uniformity and angular diversity, which will in turn affect the resulting PSF.

Pre-modulation/first modulation embodiment

In some embodiments, the pre-modulation/first modulation may require the ability to modulate the light that reaches the pre-modulator en route to the primary modulator. In some cases, pre-modulation may be applied to increase the system contrast. By highlighting (highlightening), it is possible that besides non-imaging pre-modulator illumination, the highlighted image may also illuminate the pre-modulator.

In some embodiments, a suitable pre-modulation/first modulator may be a DMD, LCD, LCoS, or other intensity modulator. Regardless of how implemented, pre-modulation may be used to modulate the light intensity that reaches it onto a subsequent modulator. Each pre-modulator element (e.g., mirror, pixel, etc.) affects a fixed position on the following modulator, or on the screen if there is no additional modulation behind the pre-modulator. Depending on the number of pre-modulation elements, the total coverage and PSF properties that can be achieved by the pre-modulator, it may not be necessary to have a primary modulator after it. It is possible that the pre-modulator may have a capability that does not require any modulation (e.g. highlight or main) before or after it.

Pre-to-primary modulator relay optics control

This refers to the ability to adjust the relay optics to control the point spread function shape of the illumination generated by the highlight or pre-modulator onto the primary modulator. There is control to adjust the full width at half maximum dimension as well as control the shape or tail of the PSF. It is possible to use a pre-modulator for cycling, and monitoring, modeling, predicting and/or measuring the resulting illumination intensity may be desirable, as additional passes through the integration rod will change the light uniformity and angular diversity, which will in turn affect the resulting PSF.

Primary modulator embodiment

Primary/secondary modulation may require the ability to modulate light that reaches the primary modulator on its way to the screen. In some embodiments, this may tend to ensure a resulting quality image with high contrast and desired spatial and intensity resolution. In some embodiments, it is possible that the primary modulator may be illuminated by a highlight and/or pre-modulator image in addition to the non-imaging primary modulator illumination.

In some embodiments, a suitable master/second modulator may be a DMD, LCD, LCoS, or other intensity modulator. Regardless of the implementation, the primary/secondary modulation may be used to modulate the light intensity reaching it onto the screen. Each primary modulator element (e.g., mirror, pixel, etc.) affects a fixed position on the screen. The size and shape of each location should be consistent to form the projected screen image, the overall size and shape of which will be determined by the projection optics. Depending on the primary modulator contrast range, it may not be necessary to use a highlight or a pre-modulator. It is possible that the primary modulator may have a capability that does not require any modulation (highlight or pre) before it. The primary modulator may be used for recycling. It would be desirable to understand the resulting illumination intensity in both horizontal and temporal terms to compensate for the illumination adjustment, or to ensure that the desired image is formed by changing the signal to the modulator. It is possible to measure this level. It is also possible to model and predict this level using an algorithm.

Other projection System embodiments

Fig. 4 depicts another embodiment of a dual/multi-modulator projection system 400 in which it is possible and/or desirable to perform light recycling. As can be seen, the projection system 400 may include one or more light sources (e.g., 402a and/or 402b or other additional light sources). In this embodiment, the light source 402A provides light into the integrated subsystem/box 404a, which may be similar to the embodiment of fig. 2A. The light from 402a may eventually reach the first modulator 406, where the first modulator 406 may be constructed in substantially the same manner as fig. 1A, 1B, 1C, and/or 2 (i.e., with a reflector that may reflect light back to the integration subsystem/box 404 a). The light may then proceed to optical subsystem 408, second modulator 410, and thereafter to projection lens 412, and the final projected image may be formed on screen 414.

However, additional opportunities for light recycling may arise using another (or in other embodiments, multiple) light source 402 b. In one embodiment, light source 402b may be used as another primary light source (i.e., to provide a large amount of light for the final image at a large amount of time). In this embodiment, light from 402b may be further reflected by reflector 403 so that this light may be combined with light from 402a at beam splitter 405, and the combined beam forms the final image a significant amount of time.

In another embodiment, light source 402b may be used for a lesser amount of time in order to provide highlight illumination within a portion of an image. It should be appreciated that reflector 403 may be a single mirror that is movable (e.g., to send light to a dump or another recycling subsystem). Alternatively, reflector 403 may be a group and/or array of reflectors (e.g., MEMS, DMD, etc.) to provide finer control of the additional light from 402 b.

In another embodiment, the light source 402b may be optional, and the integrated subsystem/cartridge 404b may have a fully reflective surface at the end adjacent to where the light source 402b is likely. In this embodiment, the light may have another path (e.g., inside box 404b and box 404 a) in which the light is recycled. In another embodiment, it may be possible to use a one-way mirror for 405. In this case, reflector 403 would just be a controllable mirror that can redirect light into 404b and thus reflector 403 may only be necessary for a "folding" system for recycling. In such an embodiment, light may not be required to be recycled in 404a, instead, light may be recycled in 404. This may be desirable because the possibility of having a recycling reflector without an aperture therein for light input makes it a more efficient recycler.

Fig. 5 is another embodiment where light recycling can be and/or is desired. The projection system 500 may include a light source 502 and an integrated subsystem/box 504, as previously described. Polarizer 505 may be a controllable polarizer, such as an LCD, that will polarize a selectable fraction of light in one orientation. The beam splitter 506 may be a polarizing beam splitter that will pass the light in one orientation directly as a uniform light field 514 for combining onto a primary modulator 518 using 516. Light polarized in the other orientation is redirected 508 by 506. Depending on the design of the system, mirror 510 may be a mirror that folds the system and directs the light to either a pre-modulator or a highlight modulator 512.

The non-uniform light field from 512 is then combined using 516 and 514 to illuminate 518. When 512 is a pre-modulator, light beam 514 can be used to provide some basic level of illumination in the dark less than the first step of 512 for very dark portions of image 522. Alternatively, when 512 is a highlight modulator, 514 is used to provide the uniform light level required for image 522 in areas where no light will be located in the non-uniform light field produced by 512.

In other embodiments, it is possible to place recycling-type integrator rods (similar to those described in fig. 3) between 510 and 512 (or between 506 and 510) and to place non-recycled versions of the integrator rods (e.g., without a back reflector) between 506 and 516. In such embodiments, it may be desirable to remove 504 behind 502 in order to keep the light as a tight beam.

An illustrative embodiment

Fig. 6A and 6B schematically depict one or more possible embodiments for a projection system that may provide these multiple opportunities for light recycling. Fig. 6A schematically depicts a process 600 that can be implemented using a dual/multi-modulator projection system. This processing may include light from various laser, coherent or partially coherent light sources (e.g., where the laser light may be pulsed 602 or supplied by a laser diode 604). Such light may be combined and transmitted (606) in various architectures and in various ways (as described in connection with several embodiments above). The light may then be divided (608) into component parts (e.g., 610 through 620), and this light may be combined and divided (622) to serve various functions, such as highlight illumination (628), release illumination (630), pre-modulated (or first modulator) illumination (626), and main (or second modulator) illumination (624).

In one embodiment, adjusting the laser power tends to uniformly affect the entire display area for global dimming. This may be appropriate for some images and scenes in a projection system that is capable of adjusting the power of the laser and/or light source. However, in some cases it may be advantageous to have a controllable base level uniform illumination applied directly to the highlight, the pre-modulation/first modulator or the main/second modulator at low luminance levels. Controlling this type of laser power adjustment would be considered another form of global dimming.

In one embodiment where multiple laser sources are applied in the display system (either using separate lasers or groups of lasers for each controllable source, or by grouping lasers or groups of lasers into respective controllable sources), they may be spatially arranged such that each affects a portion of the display area to allow local dimming. This approach differs from a highlight modulator in that the local dimming regions are spatially fixed, where the highlight modulation local dimming regions may be spatially adjustable. Mechanical light steering can be used to control laser power adjustment for each region by directing light reaching the mirror to a spatially directed fiber or optical component (such as a segmented integrator rod) that directs the light to a predetermined spatial region on the modulator.

In this case, the mechanical light steering device can be considered part of the laser power adjustment, rather than the highlighting and/or pre-modulator, but these systems where the number of individually controllable elements on the mechanical steering is greater than the number of spatial regions have the additional advantage: the illumination from fixed or variable sources can be spatially redistributed without having to directly alter the source of each region. The spatial application of laser illumination to the modulators may be controlled by illumination optics for each modulator. For global dimming, the illumination of the illumination optics (e.g., lens, integrating rod, etc.) may be designed to uniformly illuminate the modulator. For local dimming, the illumination of the illumination optics (e.g., lenslet arrays, segmented integrator rods, etc.) can be designed to take each optical path and spread it over the desired portion of the modulator to produce a suitable PSF.

In embodiments where the pre-modulation/first modulator is expected to receive most of the illumination, if light recycling is implemented, it may be desirable to have its illumination adjustable in division or using optical power control or by using the modulator to compensate for the contrast-degrading conditions.

Several illustrative embodiments

Fig. 6B schematically depicts a projection system that may implement a process such as that described in fig. 6A. The system 632 may optionally provide highlight illumination 628 to enter an optical path 634 to a highlight modulator 636. This light may be sent into the pre-modulated (or first modulator) optical path at 642 via optical path 644, or the light may be released (638) and possibly recycled at 640.

The pre-modulation/first modulator stage may input light at 626 via an optical path 652. This light may be combined with highlight illumination at the pre-modulation/first modulator 646, as described. This light may be sent to the primary/secondary modulator (e.g., to form a pre-modulated image 654), or it may be released and recycled at 648.

The primary/secondary modulator (660) may receive light from the pre-modulation/primary modulator or primary illumination 624 (e.g., via optical paths 656, 658, respectively). This light may be sent as a primary image 662 to projection optics 664, which forms a projected image 666 on a projection screen (which may vibrate if the light source is coherent or partially coherent) 668 and is viewed in an auditorium 670 or the like. Otherwise, the light may be released and recycled at 674.

It should be appreciated that this schematic diagram may support a variety of possible projection systems and all of them are included within the scope of the present application. For purposes of this application, it may be sufficient that the projection system architecture may support one or more opportunities for light recycling.

Control algorithm embodiment

As mentioned, it may not be desirable to use full power of the light source to form the final projected image many times during projection of an image, group of images, or video. In this case, a portion of the light may be recycled multiple times (essentially indefinitely) until a brighter image needs to be formed. Furthermore, because reflector 220 may actually comprise a set (or array) of reflectors, there may be opportunities to recycle light on a local dimming basis. In one possible embodiment, when not all of the available light is needed to form the final projected image, it is possible to apply light recycling on a global or local dimming basis, and then use it on a targeted basis (e.g., projecting "highlights" in the final projected image). A highlight may be a portion of an image for which it is desirable to direct much more illumination energy than the portion of the image surrounding it in order to emphasize that portion.

In another embodiment, light recycling may again be applied on a global or local dimming basis in order to enhance the illumination of an image or scene that is on average brighter than the image or scene in front of it. These opportunities may arise during illumination of the pre-modulation/first modulator stage or the main/second modulator stage, as can be seen in fig. 6B.

In one embodiment, the projection system may make a determination as to how best to apply light recycling by the controller when it processes the input image/video data. The decision to recycle can be made on the fly as the image data is processed, or ahead of the look-ahead on a frame-by-frame, group-by-group-of-frames, or scene-by-scene basis. In another embodiment, the entire video and/or scene may be analyzed off-line and a control signal may be sent to the controller along with the image/video data as part of an associated metadata stream.

FIG. 7A is one embodiment of a flow chart for performing light recycling. The control system/method 700 may input image data at 702. Based on the response curves and/or tables (e.g., as shown in fig. 7B), the system/method may calculate an Average Picture Level (APL) for each Individually Modulated Color (IMC) of the modulator. As can be seen in the graph of fig. 7B, each individual color may exhibit a different relative brightness for a given DMD fill percentage. It may be desirable to take these color differences into account when performing light recycling in order to eliminate and/or mitigate any tonal visual artifacts. It should be appreciated that the flow diagrams of fig. 7A and 8 may assume that recycling generates a uniform light field, however, the flow diagram of fig. 9 may take into account spatial intensity variations due to recycling and apply the tables of fig. 7C and 10. For example, according to the table depicted in fig. 7C, the input image may be divided into a 5x4 array of image areas, and the light recycling in each image area may be adjusted from 0% to 40% as described.

Returning to FIG. 7A, at 706, the system/method may determine a relative brightness increase for each IMC. Once completed, the system/method may instruct the display system to reduce the illumination source intensity to the reciprocal (reciprocal) of the increase in brightness of each IMC. It should be appreciated that other functional relationships between illumination source intensity and brightness increase may be possible and/or desirable (e.g., some inverse relationship of some function of brightness increase may be possible). Where the term "reciprocal" is used herein, it should be recognized that such other embodiments are also possible. It is possible to adjust the light source intensity in 708, but in some embodiments the recycling may remain the same (e.g., the percentage of recycling may not be changed by source reduction — only the absolute value, so as not to impose too much illumination on the modulator). Since the light travels fast, and even the fastest PWM period is comparatively very slow, the recycling can be considered instantaneous and the resulting illumination level can be achieved immediately after the modulator switches to its current state.

In the case of a system employing DMD(s) as primary modulators (e.g., modulators that spread modulation over several time periods), there may be a modulator state and resulting level of recirculation for each time period, and each may be calculated and compensated for. For systems that apply the DMD(s) as pre-modulators, there may be only one time period, since the system may drive them using a halftone binary pattern that may change only once per frame (e.g., in practice it may change it 1-4 times per frame, but for the master DMD modulator this may be significantly less than the 10's-100's time period). For embodiments employing LCDs and LCoS as the primary modulators, these may be switched slowly (relative to the DMD) while displaying, so that the resulting recirculation can be accumulated over that time to determine how to compensate.

While the control system/method of fig. 7A may be used generally for any dual/multi-modulator display system, this control may also be used in the context of a single modulator projection system (e.g., may be constructed in the same or similar manner as fig. 1B). The recycling on the primary modulator can come from the time-sequential nature of DMD, LCoS, and LCD based systems.

Fig. 8 is another control system/method (800) for light recycling. Control may begin inputting image data at 802. At 804, the system may calculate an APL for each IMC. The system may then determine a relative brightness increase for each IMC at 806. At 808, the system may reduce the illumination source intensity to a setting closest to, and possibly not below, the reciprocal of the increase in brightness of each IMC. In one embodiment, it may be assumed that the system may use a modulator to reduce the light rather than increase it, in which case it may not be desirable that the system can reduce the illumination source below the required level. However, in another embodiment (e.g., in the case of a mostly dark modulator image), the opposite may tend to be true (e.g., the system may reduce the illumination and then set the modulator to allow more light to pass through). In this case, step 808 may continue to reduce the illumination source intensity to a setting closest to the reciprocal of the increase in brightness of each IMC and still allow modulator compensation.

Then at 810, the system can reduce the intensity of the image driven to the modulator to compensate for the difference between the desired reciprocal number of brightness increases and the settings that can be obtained using the illumination source. Alternatively, step 810 may also adjust the intensity of the image driven to the modulator to compensate for the difference between the desired reciprocal number of brightness increases and the settings that can be obtained using the illumination source.

Fig. 9 is yet another embodiment of a control system/method for light recycling. However, this control system/method works well in display systems where photo-non-uniformity due to recycling needs to be accounted for and/or adjusted, and the illumination intensity control is fine-grained or continuous. The system/method 900 may input image data at 902. At 904, the system may calculate an APL for each region of the IMC (i.e., where the image may be divided into different regions). At 906, the system can determine a relative brightness increase for each region in the IMC based on the experimental statistics. The system can drive a pattern (e.g., some regions off while the remaining regions are on) to the modulator and observe the distribution of light. Depending on the location of the dark region, its recycled light may return to the modulator in a non-uniform manner. This non-uniformity needs to be compensated for at the modulator.

At 908, the system may reduce the illumination source intensity to the reciprocal of the region with the lowest brightness increase for each IMC. Based on the illumination source intensity settings, the system may determine the relative brightness increase for each region in each IMC. Then, at 912, the system may reduce the intensity of the image driven to each region of the modulator to compensate for the difference between the desired reciprocal number of brightness increases at that region and the setting of the illumination source.

Given an input image of a 5x4 array divided into image regions, fig. 10 depicts an example table of partially filled (e.g., by measuring, estimating, and/or calculating only center and corner values are filled, the rest may be filled in a similar manner) non-uniform levels of light recycling provided on the modulator given a certain modulator region pattern (e.g., derived as part of experimental statistics). On the other hand, it is possible to show the mode and then adjust the resulting level of recirculation based on its characteristics. For example, table 1 shows luminance characteristics of an image of a 3 × 3 array divided into image regions (e.g., in each region, showing whether the average or peak luminance level is above or below a predetermined luminance threshold (e.g., 10 nit)). For example, as shown in table 2, in an embodiment, because the lower right region is OFF (or below the threshold), most of the light recycling may be performed closer to that region, and then down for more remotely located image regions. Many such tables derived from experimentation may be used at 906.

Table 1 luminance characteristics of test images of a 3x3 array divided into image regions

ON	ON	ON
			ON	ON	ON
ON	ON	OFF

Table 2 percentage of light recycling for 3x3 segmented images as a function of image characteristics

102％	104％	108％
			103％	108％	109％
104％	108％	110％

FIG. 11 is one embodiment of an algorithm (1100) for reducing the intensity of an illumination source as a function of an increase in brightness. In some systems, such brightness increases may occur on the basis of individually modulating the colors.

At 1102, the system can input a desired image for viewing. At 1104, the system may calculate a light field that is expected (or required) to be generated by the pre-modulator for each Individually Modulated Color (IMC). At 1106, the system may calculate an Average Picture Level (APL) for the pre-modulator for each IMC. At 1108, a relative brightness increase may be determined for each IMC based on its APL. Then, at 1110, the system may reduce the illumination source intensity to the reciprocal of the increase in brightness for each IMC.

FIG. 12 is one embodiment of an algorithm (1200) for reducing illumination source intensity, particularly in systems where polarization may be applied to project an image (such as can be seen in FIG. 5).

At 1202, the system can input a desired image for viewing. At 1204, the system may calculate the amount of light (e.g., such as 514 in fig. 5) that will be directly transferred to the primary modulator, possibly for each IMC. Then, at 1206, the system may calculate the light field that needs to be generated by the pre-modulator of each IMC. Then, at 1208, an APL may be calculated for the pre-modulator of each IMC. Then, at 1210, the system may determine a relative brightness increase for each IMC based on its APL. At 1212, the system may reduce the illumination source intensity to a reciprocal of the increase in brightness of each IMC. This may also include the amount of light to be transferred directly to the primary modulator of each IMC. Then, at 1214, the system may adjust a polarizer (e.g., 505) to align the polarization into a beam splitter (e.g., 506) so that the desired amount of light may be transferred directly to the primary modulator.

FIG. 13 is one embodiment of an algorithm (1300) that can input images generated without assuming that the display system can participate in light recycling. In one embodiment, the system can adjust the light recycling in many possible ways (e.g., the ability to take the "EDR master" level and map it to the target display while preserving artistic intent with the metadata).

At 1302, the system can input a desired image for viewing. This image can be generated without assuming that no recycling is to be/was done. At 1304, the system may calculate an APL for each IMC. At 1306, the system may determine a relative brightness increase for each IMC based on its APL. The system may then provide (or otherwise calculate) the range of luminances that can be achieved for each IMC to the display management algorithm at 1308. At 1310, the display management algorithm may generate an image to be displayed based on the recycling range, which may be lower in brightness than an image that can be achieved using recycling each IMC, but may not be greater. Then, at 1312, the system may calculate a New APL (NAPL) for each IMC. At 1314, the system may determine a new relative brightness increase for each IMC based on its NAPL. Thereafter, at 1316, the system may reduce the illumination source intensity to the reciprocal number of NAPLs per IMC.

A detailed description of one or more embodiments of the invention will now be given in conjunction with the appended drawings, illustrating the principles of the invention. It should be appreciated that the invention is described in conjunction with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in this description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

Claims (18)

1. A projector display system capable of recycling light from a plurality of light sources, the projector display system comprising:

a plurality of partially coherent color light sources, each of the partially coherent color light sources emitting color light for modulation;

a light recycling module comprising an integrated rod configured to receive light from the light source at a proximal end and wherein the proximal end comprises a reflective surface capable of reflecting light along the integrated rod;

a first modulator comprising a plurality of movable mirrors capable of reflecting light received from the integrating rod in at least one of a first projection direction and a light recycling direction, wherein the light recycling direction is substantially in the direction of the integrating rod; and

a controller configured to receive input image and/or video data, output a control signal to the first modulator and modulate the light source for adjusting the light source intensity,

the controller is configured to:

receiving an input image;

calculating an average picture level of each individually modulated color light;

determining a relative brightness increase of each individually modulated color light caused by the recycled light; and

the intensity of the illumination from the color light source is reduced according to the reciprocal number of the increase in the relative brightness of the color light,

wherein for each partially coherent color light source an integrating rod and a first modulator are disposed.

2. The projector display system of claim 1 wherein said projector display system further comprises:

a second modulator capable of modulating light received from the first modulator in a first projection direction and transmitting the modulated light for projection.

3. The projector display system of claim 2 wherein said first modulator comprises a pre-modulator.

4. The projector display system of claim 3 wherein said pre-modulator is capable of producing a halftone image of a desired image to be displayed.

5. The projector display system of claim 4 wherein said second modulator comprises a primary modulator.

6. The projector display system of claim 5 wherein said primary modulator is capable of pulse width modulating said halftone image produced by said pre-modulator.

7. The projector display system of claim 1 wherein said first modulator comprises a highlight modulator.

8. The projector display system of claim 7 wherein said highlight modulator is capable of placing additional light energy into the main beam in order to highlight a desired portion of the image to be displayed.

9. The projector display system of claim 8 wherein a second modulator is capable of modulating said main beam and said additional light energy to produce a desired image.

10. The projector display system of claim 1 wherein said projector display system further comprises:

a dichroic combiner capable of combining at least two color light beams from at least two integrated rods to form a primary light beam.

11. A method for recycling light in a projector display system, the display system comprising a plurality of partially coherent color light sources each emitting color light for modulation, a first modulator comprising a plurality of movable mirrors, and a light recycling module; wherein the method comprises the following steps:

receiving an input image;

calculating an average picture level for each individual modulated color light;

determining a relative brightness increase caused by the recycled light for each individual modulated color light; and

the intensity of the illumination from the color light source is reduced according to the reciprocal number of the increase in the relative brightness of the color light,

wherein for each partially coherent color light source an integrating rod and a first modulator are disposed.

12. The method of claim 11, wherein the method further comprises:

the intensity of the image of the control signal driven to the first modulator is reduced to compensate for the difference between the reciprocal of the increase in brightness and the setting that can be obtained by the illumination source.

13. The method of claim 11, wherein the method further comprises:

when it is determined that the relative brightness increases, an adjustment to the light recycling non-uniformity is made to compensate for the light recycling non-uniformity.

14. The method of claim 11, wherein the display system further comprises a second modulator that receives light from the first modulator, and further comprising:

the image of the control signal driven to the second modulator is adjusted based on the increase in brightness due to the circulating light with respect to the first modulator.

15. The method of claim 14, wherein adjusting the image of the control signal driven to the second modulator comprises:

the intensity of the image of the control signal driven to the second modulator is reduced based on a functional relationship between the increase in brightness and the setting available through the illumination source.

16. The method of claim 15, wherein the functional relationship is substantially inverse.

17. The method of claim 14, wherein the method further comprises:

when it is determined that the relative brightness increases, an adjustment to the light recycling non-uniformity is made to compensate for the light recycling non-uniformity.

18. The method of claim 14, wherein the method further comprises:

the source brightness setting is calculated by including the light transferred to the first modulator as the primary modulator.