JP2019012090A - Image processing method and image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a high-luminance display system having enhanced contrast (dynamic range), luminance resolution, and spatial resolution. An image display system includes one or more projectors, one or more screens, and a digital signal processing device, and the image projected by the projectors is triggered by the distribution of hotspots by the projectors on the screens. It is superimposed on the screen to produce a composite image that features good uniformity of maximum radiated brightness and a high level of radiated brightness that can be displayed, and the radiated brightness distribution, radiated characteristics, and accurate overlay of the image. Increase contrast (dynamic range), brightness resolution, and spatial resolution by auto-calibrating the system, including, and calculating the image displayed by each projector, for high-brightness images, extended color range images, and Enables the display of infrared images. [Selection diagram] FIG. 1B

Description

The present invention relates to a projection display characterized by either a front projection or a rear projection. An example of an embodiment is a driving simulator display. In addition, it is also provided as a dedicated display such as a home theater display, a digital cinema display, and a camera evaluation display, an advertising display, a video game display, a virtual reality display, an aircraft simulator display display, and an entertainment simulator display.

Electromagnetic waves of various wavelengths are propagated around the world. The radiance perceived by the observer is a unit of solid angle divided by the apparent basic plane viewed from the viewpoint of the observer, and is defined as the radiant flux emitted by the basic plane in the direction of the observer. Luminance is defined in a similar way except that the luminous flux considered is a luminous flux rather than a radiant flux and can therefore be considered as a specific case of radiance (the luminous flux is visible range [380; 760 nm]). Radiation weighted by the relative sensitivity of the human eye across).

An observer or sensor may be exposed to high radiance and / or brightness, and even high dynamic range radiance and / or brightness. In the real world, the human eye may perceive high dynamic range brightness. The brightness of the surface illuminated by the moon is about 10 ⁻³ cd / m ² , whereas the brightness of the surface illuminated by the sun can exceed 10 ⁵ cd / m ² . In conventional image display systems (cathode tubes, liquid crystal display screens, projectors), the brightness of individual pixels facing the observer is usually about 200-300 cd / m ² and the contrast is usually [100; 300]. Range (contrast is defined as the ratio of the maximum luminance of a pixel in the image to the minimum luminance that another pixel in the same image can have simultaneously).

Recently, several advances in liquid crystal display (LCD) screens and organic light emitting diode (OLED) based screens can display images with individual pixel brightness in the range [500; 1000] cd / m ^2. Became. Special display devices based on liquid crystal display technology backlit by light emitting diode (LED) arrays may display images with individual pixel brightness reaching 8500 cd / m ² (each light emitting in the array). The light flux of the diode is adjusted independently). However, the size of the displayed image is limited by the size of the LCD panel and is typically in the range [32; 46] inches. In addition, the brightness of a particular pixel in the image cannot be accurately controlled because it depends on the intensity of the LED that backlights it, but this LED also backlights the surrounding pixels in the pixel. As a result, the brightness of a set of pixels (ie, a particular pixel and surrounding pixels) are linked together, limiting the ability to control each of them independently. Such display devices are described in documents EP1390806, WO02069030, EP1485904 and WO03077013.

Conventional display systems are characterized by the level of luminance of the pixels viewed by the viewer. A high brightness display system is a display system that can display a high brightness (HL) image, that is, an image in which the brightness of the pixels is higher than the brightness of the pixels in the image displayed by the conventional display system. is there. Usually, the ratio of the maximum brightness of the pixels in the image displayed by the high brightness display system to the maximum brightness of the pixels in the image displayed by the conventional display system is greater than 1 and is in the range [2; 10]. It is. In particular, a high brightness image may induce physiological glare depending on its maximum brightness level.

A high dynamic range image (HDR) is an image having a higher contrast than that of an image displayed by a conventional display system. Typically, the ratio of the contrast of the image displayed by the HDR display to the contrast of the image displayed by the conventional display system is greater than 1 and has a value in the range [2; 10], [10; 100. ] May be within the range. Although an HL image is not necessarily an HDR image, there is a possibility that the image is simultaneously HL and HDR.

A high radiance display system can display a high radiance (HR) image, i.e., an image in which the radiance of a pixel is higher than the radiance of the pixel in an image displayed by a conventional display system. It is a display system. Indeed, a conventional display system emits radiance towards the viewer, even if it is not specifically designed. Usually, the wavelength of emitted radiation is limited to the visible range. Usually, the maximum brightness of the pixels in the image displayed by the present invention and the maximum brightness of the pixels in the image displayed by the conventional display system. The ratio is greater than 1, has a value in the range [2; 10], and may have a value in the range [10; 100]. The term “radiance” means that an image displayed by the present invention emits radiation having a wavelength in the visible range, in the ultraviolet (UV) range, in the infrared (IR) range, and other wavelength ranges. Means. The HR image is not necessarily an HL image. However, the HL image is generally an HR image. An HR image is not necessarily an HDR image, but there is a possibility that the image is simultaneously HR and HDR.

The present invention has various aspects. Embodiments include a method for performing automatic calibration of a display system, a method for calculating the content of an image projected by different projectors, a method for operating a projection display, and a data processor when executed by a data processor. A medium including computer readable instructions to be executed, a method for displaying an image, and a method for processing and displaying image data.

As an example of one aspect of the present invention, a plurality of similar projectors configured to project different images defined by original image data onto a screen, each image calculated based on the relative position and orientation of the projector There is a display system that includes the combined effects of a projector and a screen calibrated in a previous autocalibration phase performed using a sensor.

As an example of another aspect of the present invention, a plurality of similar projectors configured to project different images defined by original image data and arranged to provide a wide or panoramic field of view to the viewer There are display systems that include different screens.

Another example of the present invention is a plurality of different projectors configured to project different images defined by original image data onto a screen, some of which images partially overlap each other ( There is a display system (corner in the screen area where images projected by other projectors are completely superimposed).

In some embodiments, the projector is arranged and positioned relative to the screen so that the image projected onto the screen is superimposed using lens shift and / or keystone correction, and / or the projector's zoom function. Oriented. The generated composite image is characterized by a high uniformity of maximum brightness determined by the distribution of hot spots by the projector on the screen and a high level of displayable radiance.

Some embodiments implement a method for performing automatic calibration of the system including radiance distribution, radiant characterization, and accurate overlay of images. These methods display specific images on the screen of the display system, sensors that measure the characteristics of the screen when these specific images are displayed (eg, cameras, luminance meters, spectrometers, spectroradiometers) And processing of data obtained via a digital signal processor embedded in the digital signal processing device. The result of the auto-calibration phase is that the radiance distribution induced by each projector on the screen, the amount of radiation induced by each channel of each projector on the screen, and the image projected by the different projectors naturally overlap on the screen. Is used to calculate the content of the image displayed by the projector of the system and to process the content of the original image.

In some embodiments, a method for calculating the image displayed by each projector is implemented for the purpose of increasing the contrast (leading to dynamic range), radiance resolution, and spatial resolution of the composite image. Increasing the percentage of transparent or white pixels in the image increases both the contrast and luminance resolution of the composite image, taking advantage of the phenomenon that the native contrast of the image displayed by the projector is reduced. Therefore, the calculation method of the contents of each image projected by each projector takes this relationship into consideration. The spatial resolution of the composite image is increased by using an accurate evaluation of how each pixel array projected by each projector on the screen is superimposed on other pixel arrays projected by other projectors.

Some embodiments are directed to displaying high brightness images. Another embodiment is intended to display a wide color gamut image. Another embodiment is aimed at displaying an infrared image. The type of image that can be displayed by embodiments depends on the projectors used in the system and the radiation characteristics of the radiation they emit: projectors that emit visible radiation display high brightness or extended color gamut images. Used in the intended embodiment. In the latter case, the radiation emitted from the projector should have colorimetric characteristics outside the color gamut of conventional display systems. Projectors that emit infrared radiation are used in embodiments intended to display infrared images. The radiation emitted from the projector is the type of radiation source they contain (light source, infrared source, etc.) and the devices (filters, prisms, dichroic mirrors, etc.) intended to filter or separate the wavelength of these radiations .

In addition to the illustrative aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the drawings and by studying the following detailed description.

The accompanying drawings illustrate non-limiting embodiments.

  FIG. 1A shows an example of the luminance distribution on the screen for two projector configurations. The first configuration includes a single powerful projector that emits 20,000 lumens and a strong hot spot is visible. The second configuration includes nine less powerful projectors that generally radiate 9 × 2400 = 21600 lumens: nine weak hot spots are distributed on the screen, improving brightness uniformity.

  FIG. 1B is a schematic diagram of a display system according to an exemplary embodiment. This exemplary configuration includes four rear projectors.

  FIG. 1C shows an example of the maximum luminance distribution on the screen obtained by four projectors.

  FIG. 2 shows an exemplary curve of the contrast evolution of the image displayed by the projector as a function of the proportion of white pixels to the image.

  FIG. 3A is a schematic diagram of a display system according to an exemplary embodiment. This exemplary configuration includes nine rear projectors, four of which are corner projectors (30, 32, 36 and 38).

  FIG. 3B shows a screen obtained from a corner projector where four of the nine projectors (30, 32, 36, 38) partially overlap with the other projectors (31, 33, 34, 35, and 37). An example of the luminance distribution is shown. When such a corner projector is used, the lens shift function of the corner projector is limited, and the image projected by another projector cannot be completely superimposed on the projected image. Higher uniformity can be obtained.

  FIG. 4A is a close-up shot on the screen when two projectors are incorrectly positioned and oriented to display the same image. The overlay of the two images is inaccurate.

  FIG. 4B is a close-up shot on the screen when the two projectors are correctly positioned and oriented to display the same image. The overlay of the two images is accurate.

  FIG. 5A is a checkerboard image used to evaluate the geometric transformation applied to the image to obtain an accurate overlay on the screen.

  FIG. 5B shows specific points and lengths between these specific points in the checkerboard image.

  FIG. 5C shows a particular point and length between these particular points in the checkerboard image projected by the two projectors on the screen. A checkered pattern image projected by one projector is defined as an image C1. A checkered pattern image projected by another projector is indicated by an image C2. Images C1 and C2 are superimposed on the screen.

  FIG. 6 is a schematic diagram of a digital signal processing device according to an exemplary embodiment comprising one digital data processor and various digital inputs (eg, HDMI) and various digital outputs (eg, HDMI, RS232, USB). It is.

Detailed description

Throughout the following description, specific details are set forth in order to provide a thorough understanding by those skilled in the art. However, well-known elements may not be shown or described in detail to avoid unnecessarily obscuring the disclosure. The description and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.

The present invention allows the observer to see from a specific position and visual orientation with respect to the screen (s) of the display device.
Its scalar three-color component Xpix, Ypix, Zpix, or its vector spectrum Spix (range [1; dimX
The image has n images IP1, IP2,. . . , IPn is a composite image C obtained as an overlay. This device consists of the following elements.
At least one projection screen;
-N (n> = 1) projectors P1, P2,. . . Pn;
A digital signal processing device comprising at least one digital or analog input, at least one digital or analog output, and at least one data processor;

The present invention aims to display high radiance (HR) images. Some embodiments are aimed at displaying HR images that are also TDRs. Some embodiments are high-intensity (HL) display systems because the specific case of high-radiance (HR) images is high-intensity (HL) images. Other embodiments are aimed at displaying images that are also HDR (HL).

Advantageously, the display device is intended for one or more projectors installed in one mount or various mounts, one or more screens installed in other mounts, and on the other hand for automatic calibration, On the other hand, it includes a digital signal processing device for calculating an image displayed by the projector from an input image (which may be either low dynamic range (LDR) or high dynamic range (HDR)). These images include any number of color channels or wavelength channels. The images are superimposed on the screen to produce a high brightness (HR) image. This (HR) image is a composite, that is, obtained from the superposition of various images.

The projector and screen may be arranged for either front projection or rear projection. Projectors used in display devices can have different characteristics (ie, luminous flux, resolution, color gamut, primary color and contrast). Depending on these features, the display device can display images with different sizes, resolutions, contrasts, color gamuts, and radiances.

These different approaches and their permutations and combinations are not limiting and are intended to provide examples of some embodiments.

Some documents describe systems based on the use of various projectors, sometimes to generate either high brightness (HL) or high dynamic range (HDR) images. U.S. Pat. No. 6,871,1961 presents a display device that uses various projectors to generate a composite image, each projector being used to display a portion of the entire image. Images displayed by different projectors are juxtaposed and are not superimposed. In fact, the purpose of the present invention described in US Pat. No. 6,871,961 is to increase the size and resolution of the image, not to increase the dynamic range or maximum brightness displayed.

U.S. Pat. No. 20140043352 describes a display system based on a small number of projectors (typically two), one projector needed to display pixels of an image with high brightness. Characterized by having a higher luminous flux. The second projector is intended to provide a luminous flux necessary for displaying pixels of an image with low brightness.

The present invention differs from the state of the art in various aspects. The first is related to projector selection and projector settings for the screen.

In fact, display devices that use projectors produce hot spots on the screen (regions where the maximum brightness or brightness of the image is higher than elsewhere), usually in the shape of a circle or ellipse. For a given projector attitude (ie, position and orientation) relative to the screen and a given projector, the maximum brightness or brightness on the screen seen by the viewer depends on the luminous flux of the projector. Such a hot spot is shown in FIG. 1A. A rectangle 1 represents a backlight by one powerful projector (for example, a luminous flux of 20000 lm) located behind the screen, and the optical axis of the projector lens intersects the screen at the center of the screen. Each color in the rectangle 1 represents a maximum luminance value that can be displayed at the position of the corresponding pixel according to the luminance scale 3. Hot spots are characterized by an elliptical shape.

This phenomenon is detrimental with respect to radiance or brightness uniformity on the screen, where the maximum radiance or brightness is the center of the hot spot (usually the center of the image) and other places in the image (usually the horizontal and vertical of the image). To produce a composite image that is the same at the edges and corners of the image.

Furthermore, it may be noticed that the price of the projector increases according to a non-linear relationship (but according to a quadratic relationship) to the luminous flux of the projector. This relationship depends in particular on the size and high optical quality of some parts of the projector (reflector, lens) that need to be used to guide the light emitted by the light source inside the projector and at the same time avoid light leakage. Explanation is possible.

In addition, powerful projectors typically use a variety of lamps that require directing light using specific reflectors and / or optical elements. Using different lamps can increase light leakage and reduce image contrast compared to using a single lamp. Therefore, when using various projectors each using a single lamp, the contrast of each image displayed by each projector may be higher, so there is a high possibility of increasing the contrast of the composite image.

Advantageously, the present invention makes it possible to obtain better brightness or brightness uniformity for the composite image, while at the same time reducing the price of the display device. For this purpose, when different projectors are used, the luminous flux is higher than when using projectors that are more powerful than other projectors (the ratio of their luminous flux is usually in the range [2; 10]). It is preferable to use close projectors (the ratio of their luminous flux is usually in the range [1; 2]). This, on the one hand, makes it possible to reduce the global price of the display device (the sum of the two projectors with the luminous flux F).
In the composite image on the screen, hot spots induced by different projectors are distributed. This distribution makes it possible to obtain better brightness or brightness uniformity for the image. This hot spot distribution can usually be obtained using the lens shift or keystone correction function of the projector, and it is possible to introduce an offset between the image displayed by one projector and the lens of this projector To. Therefore, images displayed by projectors with different positions may be superimposed. Since each hot spot is arranged at a different position on the composite image, the different hot spots are not superimposed on the center of the composite image but distributed on the surface thereof.

Such a hot spot distribution is shown in FIG. 1A. The rectangle 2 represents a backlight screen by nine projectors (for example, each luminous flux is 2400 lumens, and the total luminous flux is 2400 × 9 = 21600 lumens). The optical axis of the projector lens is behind the screen. And intersects the screen at nine different points, which are distributed over the rectangle in this example. The images displayed by the nine projectors can be superimposed on the screen using the projector's lens shift function, and the spatial offset of the image displayed on the screen can be applied. Each color in the rectangle 2 represents a maximum luminance value that can be displayed at the position of the corresponding pixel according to the luminance scale 3. Nine elliptical hotspots placed at different positions are superimposed, and the maximum brightness on the screen is obtained with good uniformity.

FIG. 1B shows a display device comprising a screen mount 4, a screen 5, a projector support 6, and four projectors 7, 8, 9 and 10 that backlight the screen 5 to form a composite image. A rectangle 11 in FIG. 1C represents the maximum luminance displayed on the screen by the display device, and each color corresponds to a luminance level corresponding to the scale 12.

In addition to the luminance uniformity of the displayed composite image and the price of the display device, the advantages of the present invention are the size of the displayable image (which may exceed 100 inches diagonal), the high radiance of the pixels and , Including composite image contrast that may be higher than the contrast of state-of-the-art display devices.

In fact, the contrast (ie, the ratio between the maximum radiance or brightness of a pixel and the minimum radiance or brightness of another pixel in the same image) of an individual image displayed by the projector is Decreases as the number increases. The strictly normal decreasing relationship between the contrast of the image projected by the projector and the amount of white pixels in this image is shown in FIG. The three curves in FIG. 2 show the relationship between different environments: a living room with white walls (reflection coefficient close to 1), an optimized room with dark walls, ceiling and floor (reflectance close to 0), And the wall and ceiling floor are three relationships of an ideal room that is completely black (0 reflection coefficient). The difference between these three curves comes from the scattering of light that the screen itself gives to the environment. As the reflectivity increases, the amount of light reflected on the screen itself also increases and the brightness of the screen increases: the brightness increases by a similar amount on the screen, and the brightness of the dark pixels is compared to the clear pixels, It means increasing by a relatively high amount with respect to the value when there is no light scattered on them by the environment. As a result, the contrast of the image is lowered. Such curves for various models of projectors can be found at the website projectiondream. com.

By using multiple projectors, using each of them reduces the number of bright pixels (or more generally bright pixels) in the image to be displayed individually to produce a composite image Can do. This is because the image can be calculated so that the bright pixels in the composite image are distributed over different images displayed by different projectors.

Superposition of images projected by different projectors can be performed using the lens shift and / or keystone correction and / or zoom functions of the projector. The projector and screen may be arranged for front projection or rear projection. Several
Here, F_max is the most powerful light flux or radiant flux of the projector. In other embodiments, the projectors have the same luminous flux or radiant flux. In some embodiments, the screen can have a gain greater than 1 in the normal direction of the surface and is not necessarily planar.

In some embodiments, some projectors (shown as projector set 1) have a physical configuration aimed at obtaining a full overlay of images on the screen characterized by the smallest surface of the area of the screen ( Position and orientation with respect to the screen), lens shift, keystone correction, zoom function, and other projectors (displayed as projector set 2) also partially project at least one image projected by the projector of projector set 1 Physical configuration (position and orientation relative to the screen), lens shift, trapezoidal correction, and zoom function adjustment. As shown in FIG. 3A, the projectors 31, 33, 34, 35, and 37 are in the projector set 1, and the projectors 30, 32, 36, and 38 are in the projector set 2. The projectors 30, 32, 36, and 38 are sometimes called “corner projectors” and are located at the corners of the projector set 1 and the projector set 2, and the projectors (31, 33, 34, 35, 37) (projector set 1). The images projected at the corners of the image projected by are overlapped.

Advantageously, the projectors of the projector set 2 have at least one image projected by the projectors of the projector set 1, for example at least 50% of the image width and 50% of the image height in the overlap region. FIG. 3B shows the maximum luminance distribution obtained with such a configuration. The rectangle 39 in FIG. 3B represents the maximum luminance that can be displayed on the screen by the display device, and each color corresponds to a luminance level by the scale 40.

In order to avoid projecting on the screen light corresponding to a region of the image that is not superimposed on at least one image projected by one projector of the projector set 1, at least one projector of the projector set 2 A physical system intended to limit the aperture of the light beam.

Some embodiments include at least two juxtaposed screens that form a non-zero angle.

In some embodiments, the at least one projection screen has a horizontal and / or vertical size that is smaller than that of the composite image C on the screen (ie, the surface of the screen that the projector commonly illuminates), and the composite image C Cause spatial limitations on the screen.

In some embodiments, different projectors P1, P2,. . . , Pn is used to generate a wide or wide color gamut image, or an image that emits energy in the infrared range or other wavelength range.

In particular, in some embodiments, at least one projector used in a display device has a primary color of an image projected on the screen of the display device that is outside the (x, y) standard colorimetric space (eg, sRGB, Adobe RGB). More generally, an image is projected on a screen whose primary color is a colorimetric coordinate (x, y) that may be outside the standard colorimetric space such as (x1_1; y1_1), (x2_1; y2_1). (X1_2; y1_2), (x2_2; y2_2), (x3_2; y3_2), (x3_2; y3_2), (x3_2; y3_2) are images IP1 projected by the projector P1. The three primary color projectors P2,... Of the image IPn projected by the three primary color projectors Pn. . . (X1_n; y1_n), (x2_n; y2_n), (x3_n; y3_n) represent the color coordinates of the pixels in the composite image C (x1_2; y2_2), (x3_2; y3_2),. . . These are linear combinations (x1_1; y1_1), (x2_1; y2_1), (x2_1 ;, (x2_n; y2_n), (x3_n; y3_n) of (x1_n; y1_n).

In some embodiments, the color coordinates of the primary colors of each image IPi projected onto the screen by the projector Pi (where i is in the range [1; n]) are on the edges of the chromaticity diagram CIE1931.

In some embodiments, the projector is a monochromatic projector that includes a white light source and a color filter, or that includes a colored light source (eg, a colored LED light source, a laser monochromatic light source, a laser diode monochromatic light source). In other embodiments, the projector is a multicolor projector that includes a white light source and a color filter, or includes light sources of different colors (eg, LED, laser, or laser diode). These multicolor projectors can feature DLP technology with wheels with various color filters, or LCD or guidance technology (eg, LCOS) that includes color filters and / or dichroic mirrors.

Some embodiments are light sources that emit infrared radiation (eg, halogen or incandescent lamps) with one or more filters capable of emitting infrared radiation. In that case, the displayed image is infrared, ie those pixels on the projection screen emit infrared.

When the display device uses two or more projectors, it is necessary to superimpose images projected by different projectors on the screen of the display device with sufficient accuracy to generate the composite image C.

Conveniently, in order to produce a sharp composite image, it is necessary that the different images displayed by the different projectors be accurately superimposed on the screen. In other words, each pixel in the image displayed by the projector on the screen needs to be accurately superimposed with the corresponding pixel in the image displayed by the other projector. This accuracy is the same for any set of projectors of the display device, where the center of gravity of the two light emitting surfaces on the screen of the corresponding pixels projected by these projectors is projected by any projector of the display device. The dimension within the maximum dimension of the largest light emitting surface corresponding to the projection of the upper pixel.

This state is generally satisfied naturally only for the specific configuration (including position and orientation) of the projector relative to the screen. FIG. 4A shows a short-distance shooting of a screen in which two images displayed by two different projectors are superimposed. The contents of the two images are the same, emphasizing that the overlap is inaccurate due to the offset between the two images. By changing the position and orientation of the two projectors, the exact overlay shown in FIG. 4B can be obtained. Each pixel displayed by the projector is shown in FIG. 4B, projected onto the screen at the same position as the corresponding pixel displayed by the other projector. However, the physical configurations of such various projectors are difficult to obtain in practice and are not guaranteed over time.

The display device has n images IP1, IP2,. . . , IPn on the screen of the display device n projectors P1, P2,. . . , Pn. The present invention relates to images IP1, IP2,. . . Provides a method for obtaining an accurate overlay of the contents of IPn.

These methods involve images IP1, IP2,. . . , Based on the geometric transformation of the IPn content, each image is transformed in a specific way taking into account the physical configuration of each projector with respect to the screen. This process is practically possible in situations where images projected by different projectors of the display device are accurately superimposed on a common projection area of different images.

In some embodiments, the image projected by one particular projector (represented as a “reference projector”) does not undergo geometric transformation (in other words, the transformation performed on this image is the same transformation). Is). Geometric transformations applied to other images result in an accurate overlay of the images projected by the “reference projector” with these images.

Images IP1, IP2,. . . The purpose of these geometric transformations applied to IPn is to obtain a superposition of an image projected by a projector different from the “reference projector” and an image projected by the “reference projector”. The center of the projection area on the screen in the overlapping area of two pixels having a projection plane corresponding to any one of the same pixels in the composite image C is more accurate than one pixel. In other words, the geometric transformation of each image displayed by each projector (excluding the “reference projector”) is performed, and once projected onto the screen, the pixels of each image are projected onto the screen. Is displayed at the same position as the corresponding pixel displayed by the projector. In this way, the correspondence between each pixel in the image displayed by the reference projector and the accurately superimposed pixel in the image projected by the other projector is known.

In some embodiments, the geometric transformation performed by the image projected by one projector is a homography locally, ie a transformation that transforms a triangular region of the image into another triangular region.

In some embodiments, the geometric transformation applied to the image is a collection of local homography, each of which is applied to a triangular region of the image to be transformed, and the set of these triangular regions is transformed. This is an image segmentation. More precisely, a homography is defined by a set of three points in the transformed triangle area and the transformed triangle area in the image coordinate system. The coordinates of the points are not necessarily integers but may be floating point numbers.

In order to obtain an accurate overlay of all the images, the present invention provides the values of the parameters of the geometric transformation that each image displayed by each projector should undergo, especially when a geometrically shaped image is obtained. Provide a process aimed at determining A transformation can be viewed as a set of local homography. In order to identify the values of these homography parameters, the correspondence of each vertex of the original triangle (before transformation) and each vertex of the transformed triangle must be specified. This is done by applying the same process to each vertex of the triangle prior to transformation.

Once these correspondences are specified, a converted image can be obtained from the non-converted image by mapping the triangular area and converting it into a converted triangular area.

FIG. 5A shows an example of a checkerboard image that is used to evaluate a geometric transformation for application to an image to obtain an accurate overlay on the screen. FIG. 5B shows some specific points and lengths between these specific points in the checkerboard image. FIG. 5C shows these particular points and lengths of the checkerboard image projected on the screen by two projectors (a reference projector projects image C2 and another projector projects image C1). A point of the converted image corresponding to the point P_init (14) of the image before conversion is defined as P_trans. The correspondence between P_trans and P_init can be calculated through the following steps.

On the screen of the display device, a single projector is used to display an image Im having a symbol S0 that can be localized by image processing (eg, circles, polygons, segments, intersection curves). For example, the image Im (13) may be a checkered pattern image as shown in FIG. 5A. The symbol S0 corresponds to the point S0_Im (14). P_init is defined as S0_Im (14).

The displayed image Im is photographed on the screen with a camera. Let the captured image be C1.

On the screen of the display device provided with the “reference projector”, an image including the same symbol S0 as the image Im and two other symbols S1 and S2 are displayed by S1, S2, and S0, respectively. These are not straight lines. These symbols correspond to points S1_Im (15) and S2_Im (16) in the image coordinate system of the image Im (13), respectively. In the image C1, the point S0 corresponds to the point S0_C1 (22).

This image displayed on the screen of the display device by the “reference projector” is taken with the same camera (with the same settings from the same position and orientation) as previously used. Let the captured image be C2.

The point S0_C1 (22) corresponding to the symbol S0 of the image C1 is placed in the image processing process using the data processor of the digital signal processor. Data processor of digital processing device, point S0_C2 (19) corresponding to symbol S0 in image C2, point S1_C2 (20) corresponding to symbol S1 in image C2, and point S2_C2 (corresponding to symbol S2 in image C2) 21). These four symbols are localized in the same coordinate system that is the coordinate system of the image Im.

The coordinates of S0_C1 (22) in the coordinate system (S0_C2 (19); S0_C2 (19) → S1_C2 (20); S0_C2 (19) → S2_C2 (21)) are calculated. To do this, calculate the coordinates of the next intersection.
P1 (23): intersection of the straight line S0_C2 (19) → S1_C2 (20) and the straight line passing through S0_C1 (22) in the direction vector S0_C2 (19) → S2_C2 (21).
P2 (24): intersection of the straight line S0_C2 (19) → S2_C2 (21) and the straight line passing through S0_C1 (22) in the direction vector S0_C2 (19) → S1_C2 (20).

The searched coordinates are the signed length L1 (27) between P1 (23) and S0_C2 (19) (S0_C2 (19) → S1_C2 (20) and S0_C2S0_C2 (19) → S2_C2 (21)) S0_C2 (19) ⇒Scalar product between P2 (22) is positive if positive, otherwise negative) and P2 (24) and S0_C2 (24) are positive, otherwise Negative).

L1_C2 (25) is a length between S0_C2 (19) and S1_C2 (20).
L1_Im (17) is a length between S0_Im (14) and S1_Im (15).
L2_C2 (26) is a length between S0_C2 (19) and S2_C2 (21).
L2_Im (18) is a length between S0_Im (14) and S2_Im (16).

In some embodiments, this process is performed when the symbol of the image Im is a corner. In that case, the image processing uses a corner extractor (for example, a Harris detector) that can detect a corner with sub-pixel accuracy as much as possible.

Alternatively, this processing may be performed when the symbol of the image Im is a circle, in which case the image processing uses a circular extractor that can detect a circle with sub-pixel accuracy as much as possible.

Alternatively, this processing may be performed when the symbol of the image Im is a line segment, in which case the image processing is performed by a segment extractor (for example, a Huff) that can detect a line segment with subpixel accuracy as much as possible. Conversion).

Alternatively, this processing may be performed when the symbol of the image Im is a curve, in which case the image processing is a curve extractor that can detect a curve with sub-pixel accuracy as much as possible (eg, a least square curve). Use).

In some embodiments, the image Im may include various symbols corresponding to various points P_init where the corresponding point P_trans is searched. In this case, the symbols S1 and S2 can be selected from different symbols.

In some embodiments, the image Im includes symbols distributed on the array (eg, the image Im may be a checkerboard image, may be similar, and / or the edges are image edges). May be parallel).

Conveniently, this process (calculates the parameters of the local homography and then superimposes the image on the device screen by transforming the image according to these local homography) is displayed on the screen. This makes it possible to avoid the identification of the intrinsic parameters of the camera used for grasping the intrinsic parameters of the image, the 3D shape of the screen, and the intrinsic parameters of the projector lens used in the display device. These advantages arise from the decomposition of the transform applied to the image content as a collection of local homography that implicitly takes these elements into account, more accurately as the number of homographies used. That is, the image to be converted is decomposed into a large number of triangles. Thus, this process can be applied to different configurations of the display device, including different extrinsic and intrinsic parameters of the projector, different extrinsic and intrinsic parameters of the camera, and different screen shapes.

FIG. 6 shows an example of the digital signal processing device 41. The digital signal processor 42 has two digital inputs 43 and 44 (which can be transmitted via image or video, eg HDMI), two digital inputs or analog inputs 45 and 46, four digital outputs 47, 48, 49, 50 ( 4 digital inputs or analog inputs 51, 52, 53 and 50 54 (projectors and / or sensors can be driven, for example, can be sent to 4 projectors (eg HDMI) via images or video) RS232 or USB).

When the display device is intended to display a high-luminance image, it is necessary that the distribution of the radiance generated by each projector on the screen is also known. The distribution of radiance includes, for each primary color channel or wavelength channel, the relative energy radiation distribution produced by each projector, as well as some characteristics of each primary color channel or wavelength channel itself (eg, Tricolor coordinates x, y, z, or spectrum).

In some embodiments, an automatic calibration step of the maximum brightness distribution on the screen is performed: one or various specific ones intended to allow estimation of the value of the parameter of the maximum brightness distribution model The image is displayed by at least one projector. When various images are displayed (for example, one for each projector), they are displayed in order. The automatic calibration process may be performed for various projectors of the display device. In this case, the automatic calibration of the projector is sequentially performed in order.

In some embodiments, the particular display image is a homogeneous image, i.e., the pixel channel values are the same throughout the image. Furthermore, the pixel channel value may be such that it emits the maximum luminance allowed by the projector towards the viewer.

In some embodiments, the radiance of light emitted from the screen toward the viewer's location is measured by a sensor that can capture an image of the screen connected to a digital signal processor. In other embodiments, the sensor used is a luminance meter that can measure the luminance of one point or various points simultaneously (eg, a video luminance meter).

It is. Here, A, a, b, M, i_max and j_max are six parameters evaluated by the digital processing device, i and j are the coordinates of the pixel displayed on the screen by one projector, and A is The maximum luminance value on the screen, a and b are parameters of the shape of an ellipse that characterizes the distribution of radiance for the image projected on the screen, i_max is the horizontal coordinate of the pixel corresponding to the maximum radiance, and j_max is the maximum radiance The vertical coordinate of the corresponding pixel, M, is related to the decrease in the radiance value around the location characterized by the maximum radiance.

In some embodiments, these parameters are evaluated by an optimization process (eg, least squares minimization) performed by the digital signal processing device.

In some embodiments, an automatic calibration step of the colorimetric or radiative properties of the light emitted by the screen (s) is performed to allow the evaluation of the values of the colorimetric or radiometric model parameters. One or more specific images for the purpose are displayed on the screen. When various images are displayed, they are sequentially displayed. The automatic calibration process may be performed for various projectors of the display device. In this case, the automatic calibration of the projector is sequentially performed in order.

In some embodiments, the particular image is a monochromatic uniform image, i.e., an image with pixel channel values the same throughout the image and all channel values except one are zero. .

In some embodiments, the colorimetric or radiometric characteristics of the light emitted by these particular images on the screen are connected to a digital signal processing device (eg, a chromameter, spectrometer or radio spectrometer). Measured by a color or radiometric sensor. The measurements carried out are the color coordinates x, y or the color components X, Y and Z of the emitted light coordinates in the region of the display image on the screen, or the spectrum (ie the radiant energy per wavelength in W / nm). Is displayed in a certain area of the displayed image on the screen.

When automatic calibration of the display device is performed, the digital processor is used to calculate the image to be displayed by the projector from the original image I transmitted via one or more of its digital or analog inputs. The

In some embodiments, the digital processing display device first performs the following sequential operations from the input image I: “inverse tone mapping” and “adapting to the target optical system”. These operations result in a modified image I_modified that is used in subsequent processing dedicated to the calculation of the image displayed by the projector of the display device. The “inverse tone mapping” process calculates, for each pixel and each radiation that characterizes this pixel, a radiation value defined in the range where this value is extended relative to the range defined in image I. Consists of. The “inverse tone mapping” process is particularly useful when the original image I is LDR (low dynamic range) or has a low radiance resolution (eg, 10, 11 or 12 bits per position pixel) HDR (high dynamic range). It can be applied to One or more pixels can be selected by comparing with what would be expected in subsequent processing (eg, 16 or 24 bits per pixel). The resulting image is shown as I_extended_range.

A specific “inverse tone mapping” process calculates the proportional radiation E of each pixel in the original image I
The channel value, V_max is the maximum value that the channel can have, and gamma is a scalar value, eg, 2.2.

The target optical system of the “adaptation to the target optical system” process may be a device such as a human eye or a camera, for example. In some embodiments, this process applies a transformation of the original image I or image I_extended_range, taking into account the characteristics of the human eye, particularly in relation to the perceived contrast modulation with respect to the spatial frequency of the image. . This processing can include transforming the original image I or image I_extended_range in the frequency domain (eg, performed by a Fourier transform), taking into account the characteristics of the human eye, and transforming the resulting image (eg, (Performed by the inverse Fourier transform).

When these operations are performed, the modified original image I_modified is available for subsequent processing, i.e. the calculation of the image displayed by the projector.

The content of each image displayed by each projector may be calculated according to a process aimed at increasing the contrast of the composite image. That is, by distributing high-brightness pixels across multiple images displayed by different projectors, the number of high-brightness pixels displayed by each projector is reduced, so the contrast of the projector used to generate the pixels A higher image is generated than when the number is smaller. In this case, more lightness or high luminance pixels exist in each image displayed by each projector. Thus, using various projectors improves the contrast.

In some embodiments, the images IP1, IP2,. . . , IPn is calculated from the image I_modified so that the global contrast of the composite image C is improved in consideration of the relationship in which the contrast of the image decreases as the number of lightness or high-luminance pixels in the image increases.

The radiance of a pixel is evaluated as the sum of the radiance generated by each projector from its pixel location on the screen. Another advantage of using multiple projectors is that the resulting composite image C has a luminance resolution. It is to improve. Thus, in some embodiments, from the image I_modified, the images IP1, IP2,. . . , IPn is calculated. That is, each pixel in the composite image C may be characterized by an additional radiance value relative to that obtained using a single projector.

In this way, the scalar color component X and / or Y and / or Z or vector spectrum S of each pixel pix of the image I_modified is obtained from p projection planes (p ≦ n) projected by p projectors. Obtained as a superposition and expressed as the sum of the radiation or photometric properties (spectrum or three color components) induced by projector Pj. In some embodiments, with respect to the order of calculation of the image IPj, the pixels in the image projected by the projector Pj that can produce the maximum of the radiation or photometric characteristics at the position of the projection of the pixel pix on the screen. The characteristics of pix are calculated first.

Advantageously, in some embodiments, the image computation process is as follows. For the sake of clarity, without losing generality, the general radiation amount E (i, j) where the coordinates (i, j) on the screen of the display device are pixels is X (i, j), or It can be either Y (i, j), or Z (i, j), or the pixel radiance over any wavelength range. Further assume that all n projectors are used and the projection overlay of the pixels corresponding to the pixel pix of the image I_modified is accurate. Therefore, the pixel (i, j) of the composite image C and the images IP1, IP2,. . . , IPn is assumed to have a corresponding relationship with the corresponding pixel. For the sake of clarity, different images IP1, IP2,. . . , Corresponding pixels in pixel (i, j) in IPn are denoted by the same coordinates (i, j). Therefore, E (i, j) = E1 (i, j) + E2 (i, j) +. . . It becomes. + En (i, j) + ε. ε is the residual radiation amount that is not displayed, which is inferior to the minimum value E_min (i, j) that can be displayed by one of the projectors at the position of pixel (i, j) on the screen. The order of calculation of the image IPj is selected so that the value of the image channel of the pixel corresponding to the pixel pix projected by the projector Pe can obtain the highest value Ee (i, j) of E (i, j) If it is strictly less than E (i, j), it is calculated first. The second calculated image makes it possible to obtain the highest value Ek (i, j) of E (i, j) on the screen and is more accurate than E (i, j) -Ee (i, j) Is a small image IPk (k different from e). The third calculated image makes it possible to obtain the highest value Ef (i, j) of E (i, j) on the screen, and E (i, j) -Ee (i, j) -Ek ( An image IPf strictly smaller than i, j) (f is different from e and k). It is possible to obtain the minimum value Ed (i, j), and the same process is repeated until the last calculated image Ipd is calculated.

In some embodiments, this process is performed as follows. Without loss of generality, the value of the pixel channel level V (i, j) and the pixel radiation amount E (i, j) in the image IPk (X (i, j) spectrum S ( i, j) integral) is known and obtained from the calibration process (manual or automatic). For example, if gamma = 2.2, V_max = 255, E_max (i, j), E
It becomes the maximum value of the amount of radiation radiated in the direction of the person.

The first stage of the algorithm is a calculation for each pixel pix having coordinates (i, j), and image IPk1 characterized by the maximum amount of radiation that pixel (i, j) is induced on the screen by the projector. Index k1. This process is then performed again to identify the index k2 of the image IPk2 in which the pixel (i, j) is characterized by a second maximum amount of radiation induced by the projector. This process is repeated several times until the index kd of the image IPkd, where the pixel (i, j) is characterized by the minimum amount of radiation induced by the projector on the screen.

Can be stored. Each map stores a value between 1 and n as follows: Each cell (i, j) of the first map Order_1 screens the index k1 of the image projected by the projector k1, whose pixel (i, j) is characterized by the maximum amount of radiation caused by the projector. Store on top. Order_2 stores on the screen the index k2 of the image projected by the projector k2, in which the pixel (i, j) is characterized by a second maximum value of the amount of radiation induced by the projector. The map Order_d stores the index k2 of the image projected by the projector kd where the pixel (i, j) is characterized by the minimum amount of radiation induced by the projector on the screen.

In some embodiments, the calculation process is as follows.
Initializing residual R (i, j) to E (i, j) Fork in range [1; n]
Identify the index k of the image that has not yet been processed in the loop so that the highest proportion of R (i, j) can be reproduced. This is done by simply reading the value of cell (i, j) in the map Order_k.
Using the relationship between this channel value and the radiation value obtained on the screen of pixel (i, j), the channel value of pixel (i, j) in image IPk is calculated. . This value makes it possible to obtain a radiation quantity E (i, j) _k approximating R (i, j) as follows.
R (i, j) _k where | E (i, j) _k−R (i, j) | is minimum and E (i, j) _k <= R (i, j) is Updated.
R (i, j) _k = E (i, j) _k_R (i, j)> = 0
In the next loop, R (i, j) replaces R (i, j) _k End for

loop:
Fork in range [1; n]
Can be replaced with:
While k is smaller than n and R (i, j) is larger than s (s is an arbitrary threshold)

As a result, E (i, j) can be expressed as the sum of q components of q <n, so that images q + 1, q + 2,. . . . , N, the value 0 can be used for the channel of pixel (i, j), improving the contrast of these images and improving the overall contrast of the composite image C. The disadvantage of this process is that the radiation dose E (i, j) is not very accurately reproduced.

In order to avoid the contrast of one image becoming too low (and therefore to avoid the contrast of the composite image C being inferior to an arbitrary threshold S_contrast_composite_image) The use of the image IPk is ruled out as soon as the channel value of the pixel induces contrast below the threshold S_contrast_image.

In other embodiments, the order of image computation is obtained by permutation of the established order.

In some embodiments, the projector's dynamic iris (if any) is utilized to improve the contrast of the composite image C. More precisely, when all projectors are used without using dynamic iris, the average radiation amount of I_modified is inferior to the maximum average radiation amount of the composite image C. In this case, the dynamic iris of the projector is activated so that the average radiation amount of the composite image C becomes equal to the average radiation amount of the image I_modified.

In some embodiments, the resolution of the composite image C may be higher than the resolution of the projector used in the system and is used for a display device that can display the original image I (or I_modified) higher than the resolution of the projector. Is done.

This property is due to the overlap of pixel arrays projected by different projectors, which may be incomplete, due to the different images projected by different projectors being superimposed on the screen of the display system. That is, it may be partially overlapped with the projection of other corresponding pixels in the image projected by another projector.

Projectors that feature lower native resolution because the price of such high-resolution display systems is lower than the price of display systems that aim to display the same resolution using projectors that have inherently higher resolutions This high resolution obtained using is an interesting characteristic of such an embodiment (the person skilled in the art will understand that a high resolution projector is more than a low resolution projector if the characteristics of the two projectors are similar except for the resolution. Knows it is expensive). This technique may also result in a composite image C that is characterized by a resolution that is not reachable by conventional display systems (eg, a resolution of 8K, 16K or higher may be obtained).

From any projector orientation (ie, position and orientation) with respect to the screen, the probability of obtaining a uniform array of small areas on the screen corresponding to the overlay of pixels displayed by the projector increases with the number of projectors. In other words, as the number of projectors is increased, not only are additional pixels provided by the additional projectors, but the resulting array of small regions corresponding to the superimposed pixels is nearly uniform.

For example, if the system comprises two projectors with full HD resolution (1920 × 1080), the projection on the screen of the pixels in the image projected by one projector is four in the image projected by the other projector. It may partially overlap with the projection of one pixel. This superposition results in four projection areas that are smaller than the area corresponding to the projection of one pixel in the image projected by one projector (on average each area is four times smaller). This incomplete overlay is between Full HD (1920 × 1080) and 4K (3840 × 2160) depending on how the images projected by the two projectors are actually superimposed on the screen. Can be obtained.

In order to increase the resolution of such a composite image C, it is necessary to determine with sufficient accuracy how the images projected by different projectors are actually superimposed on the screen. The processes described in [0063]-[0086] are used to derive a model of the superposition of an array of pixels in images projected by different projectors, from which have a resolution higher than the resolution of the projector A system with the composite image C can be obtained.

In fact, the transformation applied to the content of one image defined in [0063]-[0086] uses a pair of points with one point in the image to be transformed and the correspondence in the image to be transformed Should be superimposed on the points to be. The coordinates of these points are based on the use of subpixel image processing techniques aimed at localizing specific points on the screen photo when each projector is displaying a specific image (eg checkered image) Can have sub-pixel accuracy (ie, floating-point accuracy) in a coordinate system linked to the screen. From these floating point coordinates, a shape corresponding to the projection of the pixels in the screen may be derived. Such a shape may be, for example, a quadrilateral in consideration of the height and width of the pixels along the orientation (ie, position and orientation) of the projector with respect to the screen. In this way, an accurate surface model of the projected pixel array in the image projected by the projector on the screen can be evaluated.

From these models, the state of each pixel of the image projected by each projector to obtain a composite image C approximating the original image I (or I_modified) characterized by a higher resolution than one of the projectors in the system A process for calculating (that is, the value of each channel) can be executed. Thereafter, an example of the stage of such processing executed to obtain the high-resolution composite image C will be described.

In the following, for the sake of clarity, it is assumed that the image is monochromatic without loss of generality. It is easy to adapt the processing to a multicolor image separately, for example by considering each color (or wavelength) channel. For the sake of clarity, the amount of radiation to be considered thereafter is the luminance, but other amounts of radiation may be used.

First stage: In the coordinate system linked to the screen, accurately evaluate the position of each pixel projected by each projector on the screen. In this process, each pixel is modeled as a shape (for example, a quadrilateral) corresponding to the projection area of one pixel on the screen. This stage makes it possible to derive a model of the plane arrangement of the pixels projected on the screen by the projector of the display system with floating point precision.

Second stage: An arbitrary projector is selected as a reference projector. This projector introduces a coordinate system for an image projected on a screen. In the following, it is considered that the original image I_IR displayed at a resolution higher than the resolution of the projector of the display system (IR means resolution improvement) is projected onto the surface S_common irradiated by all projectors (this Surface S_common guides the way pixels in image I_IR are projected onto the screen in a coordinate system linked to the screen and guided by a reference projector). Each pixel in the image I_IR is associated with a small shape Sh_IR (eg, a quadrilateral) on the screen that defines a “virtual increased resolution pixel array” on the screen corresponding to the image I_IR.

Third stage: For each shape Sh (eg, a quadrilateral) corresponding to one pixel in the image projected by one of the projectors, a list of small shapes Sh_IR of the image I_IR that intersect all or part of the shape Sh Establish List_Sh.

Fourth stage: The luminance L_Sh of the shape Sh is calculated according to the metric. The metric may be a weighted average of the luminance of the small shape of List_Sh, where the weight is defined taking into account the proportion of the surface of each shape Sh_IR that intersects the shape Sh. The metric may be the minimum brightness of the shape Sh_IR of the list List_Sh. Other indicators may be used.

Fifth stage: a set of shapes Sh corresponding to pixels projected by different projectors is selected so that the projector selected to display the luminance L_Sh of the shape Sh can obtain the maximum luminance at that position ( This makes it possible to improve the contrast of the composite image C). At this stage, the luminance value can be calculated so that the channel values of a smaller set of pixels projected by the projector can be returned. Pixels that are not calculated after this stage are still dark.

Stage 6: The resolution of I_IR, defined as the image I_IR, to which the luminance image obtained by the superposition of the images projected by all projectors of the display system calculated at the end of stage 5 [0129] is subtracted The resolution I_IR_residual similar to is calculated. This calculation requires knowing how the virtual area array of pixels in the image I_IR established at the end of the second stage [0126] overlaps each area array of pixels in each projector. .

Step 7: Repeat steps 4 to 7 [0131] taking into account I_IR_residual and I_IR until all pixels of all images projected by the projector of the system are calculated.

Some embodiments allow active 3D visualization by alternating display of images corresponding to the viewer's right and left eyes, and are synchronized with glasses that prevent the left and right eyes from seeing the wrong image (this property Can be obtained, for example, using liquid crystal-based glasses that can set the transmittance of the glasses). Alternate display of left and right images is obtained using a plurality of projectors that are synchronized with each other and driven by a digital signal processor.

In other embodiments, passive 3D visualization is possible resulting from the simultaneous display of images corresponding to the viewer's right and left eyes, with the light corresponding to each image depending on the specific polarization of the emitted light from the screen. Characterized. The display must be viewed using specific glasses where the transmittance of the left and right eye glasses depends on the polarization of the light so that each eye sees the correct image. Simultaneous display on the screen of images corresponding to the viewer's right and left eyes is obtained using a polarizing filter on the projector lens, some of which is polarized in a different direction than the others.

Some embodiments project at least one image onto a screen by a projector after one or more reflections, transmissions or refractions via at least one refractive or catadioptric system.

The present invention is not limited to the examples described and illustrated, since several modifications can be made without departing from the principles thereof. While several exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize specific changes, substitutions, additions and subcombinations thereof. Accordingly, the following appended claims and the following claims are to be construed to include all such modifications, substitutions, additions and subcombinations that are within the true spirit and scope thereof. Intended.

Claims (106)

The present invention relates to a specific position and viewpoint direction relative to the display device.
And I pixel pix is its scalar three-color component Xpix, Ypix, Zpix, or its vector
Resulting in a radiance value perceived by the observer. The displayed image includes n images IP1, IP2,. . . This is a composite image C obtained as an overlay of IPn and includes the following elements.
At least one projection screen;
-N (n> = 1) projectors P1, P2,. . . , Pn;
A digital signal processing device comprising at least one digital or analog input, at least one digital or analog output, and at least one digital data processor;       Each image IPj projected by the projector Pj (j is included in the range [1; n]) is at least one projected by the projector Pk (k is included in the range [1; n] and different from j) The display device according to claim 1, wherein the display device is superposed on at least one screen with two other images IPk.       Each image IPj projected by the projector Pj (j is included in the range [1; n]) is at least one projected by the projector Pk (k is included in the range [1; n] and different from j) The display device according to claim 1, wherein the display device is partially superimposed on at least one screen with two other images IPk.       An image IPk projected by a projector Pk (k is included in the range [1; n]) installed in an arbitrary state (position, orientation, tuning) is included in the projector IPi (i is in the range [1; n]) The display device according to claim 1, wherein the display device is completely superimposed on the projected image IPi. That is, the area of the screen normally illuminated by image IPk and image IPi is minimal.       Any projector P1, P2,. . . , Each pixel of the image displayed on the screen by Pn is accurately superimposed in the overlap region with the corresponding pixel in the image displayed by the other projector (all these pixels are of the original image I 5. The display device according to claim 1, wherein the display device corresponds to one pixel pix. That is, for an arbitrary pair of projectors of the display device, the distance between the centers of gravity of the two projection planes of the pixels on the screen corresponding to the pixels pix of the original image I and projected onto the overlapping region by the pair of projectors Is shorter than the maximum dimension of the projection plane corresponding to the projection of the largest pixel on the screen projected by any projector of the display device.       The display device according to claim 1, wherein the digital signal processing device can be used in either an automatic calibration mode or an image calculation mode. The automatic calibration mode is for projectors P1, P2,. . . , Pn for some or all of the radiance distribution and characteristic parameter values on the screen, images IP1, IP2,. . . The object of the present invention is to specify the values of parameters of geometric transformation applied to a part or all of IPn. In the image calculation mode, the images IP1, IP2,... Are generated so that the superimposed image C on the screen is a display of the original image I. . . The purpose is to calculate the contents of IPn.       Images IP1, IP2,. . . , IPn is calculated from the original image I so that the global contrast of the composite image C is maximized in consideration of the relationship that the contrast of the image projected by the projector decreases as the number of high-luminance pixels increases. The display device according to claim 1, wherein the display device is characterized.       Images IP1, IP2,. . . , IPn is calculated from the original image I so that the radiance resolution of the composite image C is increased. That is, each pixel in the composite image C may be characterized by an additional radiance value relative to that obtained using a single projector.       The scalar color component X and / or Y and / or Z or vector spectrum S of each pixel pix of the original image I is obtained by superimposing p projection planes (p ≦ n) projected by p projectors. And is expressed as the sum of the radiation or photometric properties (spectrum or three color components) induced by the projector Pj, and with regard to the order of calculation, the maximum of the radiation or photometric properties at the projection position of the pixel pix on the screen. 9. A display device according to any one of the preceding claims, characterized in that the characteristics of the pixels pix in the image projected by the projector capable of generating values are calculated first.       If the maximum radiance that can be emitted by a projector at a pixel location on the screen is higher, the projector's contribution to the radiance of that pixel in the composite image C is higher. The display device according to claim 9.       11. A display device according to any one of claims 7 to 10, characterized in that a dynamic iris (if any) of the projector is used to improve the contrast of the composite image C.       Dynamic iris is used when the average radiation of source image I is inferior to the maximum average radiation of composite image C when all projectors are used without dynamic iris The display device according to claim 7, wherein: In this case, the dynamic iris of the projector is activated so that the average radiation amount of the composite image C is as close as possible to the average radiation amount of the source image I.       The primary color of at least one image projected by one projector presents color coordinates (x, y) outside a standard colorimetric space (eg sRGB, AdobeRGB). 12. The display device according to any one of 12.       The primary color coordinates of at least one image IPi projected on the screen by a projector Pi (i is included in the range [1; n]) are located at the end of the chromaticity diagram CIE1931 Item 14. The display device according to Item 13.       15. The display device according to claim 13, wherein the projector is a monochromatic projector.       The display device according to claim 15, wherein the projector includes a white light source and a color filter.       The display device according to claim 15, wherein the projector includes a colored light source.       The display device according to claim 13, wherein the projector is a light source based on an LED, a laser, or a laser diode.       The display device according to claim 13, wherein the projector is a multicolor projector including a white light source and a color filter.       The display device according to claim 13, wherein the projector is a multicolor projector including light sources of different colors.       21. A display device according to claim 20, characterized in that the light source of the projector is based on LED, laser or laser diode technology.       The display device according to claim 19, wherein the projector technology is DLP, and a wheel including various color filters is provided.       The display device according to claim 19, wherein the projector technology is an LCD including a color filter and / or a dichroic mirror or a related technology (for example, LCOS).       13. A display device according to any one of the preceding claims, characterized in that at least one image projected by one projector is infrared, i.e. its pixels on the projection screen emit infrared.       25. The projector for displaying an infrared image includes a light source that emits infrared light (e.g., a halogen or incandescent lamp) with one or more filters capable of transmitting infrared light. The display device described.       The digital image processing apparatus applies, in an image processing stage, a process including inverse stone mapping, adaptation to a target optical system, and calculation of an image displayed by a projector to an original image I. The display device according to any one of 1 to 25. 27. A display device according to claim 26, characterized in that the inverse tone mapping consists of calculating a proportional radiation amount E of each pixel in the original image I. The E is E = (V /
V_max is the maximum value that the channel can have, and gamma is a scalar value, eg, 2.2.       27. The display device according to claim 26, wherein the target optical system is a human eye.       Said “adaptation to the target optical system” is characterized in that a transformation that models the human eye, characterized for example as a function of contrast modulation depending on the spatial frequency of the image, is applied to the original image I The display device according to claim 28.       30. The transformation to the original image I is performed in the frequency domain (after application of the Fourier transform), followed by transformation in the spatial domain (performed by inverse Fourier transformation). The display device described in 1.       The display device according to claim 1, wherein the digital signal processing device displays an image on a screen using at least one projector during the automatic calibration step.       The digital signal processing device drives one or more sensors connected thereto to measure characteristics of an image displayed on a screen during the automatic calibration stage. 32. A display device according to claim 1 and claim 31.       The digital signal processing device uses at least one of its data processors to process data acquired by the sensor and obtain values of conversion parameters to be applied to a plurality of images projected by a plurality of projectors. 33. A display device according to claim 32, characterized in that an accurate overlay of these images on the screen is obtained.       The digital signal processing device uses at least one of its data processors to process the data acquired by the sensor and to provide the value of a parameter of the radiance distribution model of an image displayed on the screen by at least one projector The display device according to claim 32, wherein the display device is obtained.       The digital signal processing device uses at least one of its data processors to process data acquired by the sensor and to characterize an image displayed on a screen by at least one projector. The display device according to claim 32, wherein a value of the parameter is obtained.       The display device according to claim 5, wherein an accurate overlay is obtained by an accurate physical arrangement (orientation and position) of each projector with respect to the screen.       6. The display device according to claim 5, wherein an accurate overlay is obtained by converting the contents of one image projected by at least one projector.       38. A display device according to claim 37, characterized in that the content of the image projected by one projector indicated as "reference projector" is not subject to any conversion.       39. A display device according to claim 37 and 38, characterized in that the transformation applied to the content of the image projected by the projector is homography.       The transformation applied to the content of the image projected by the projector is defined by at least two homography, each homography is applied to a region of the image, and the set of regions defines a partition of the image The display device according to claim 37 or 38.       41. The display device according to claim 39, wherein the homography is determined by a set of three points including a vertex of a triangle area of the image to be transformed and a corresponding vertex of the transformed triangle area. .       The transformation applied to the image content of one projector P1 is based on the estimation of the coordinates of at least one point of the image projected on the screen in a coordinate system relating to the projection of another image projected by the other projector P2 on the screen. 42. A display device according to any one of claims 37 to 41, characterized in that the display device is defined.       43. The display device according to claim 42, wherein an axis of the coordinate system is an edge of an image on two non-parallel screens projected by the projector P2.       44. A display device according to claim 42 or 43, characterized in that the image comprises at least one symbol from which a localized point can be derived.       The image including the symbol is continuously projected by the projector P1 and the projector P2, and the coordinates of the point obtained from the symbol in the image projected by P1 are obtained from the symbol in the image projected by P2. 45. The display device according to claim 44, wherein the display device indicates an origin of a certain coordinate system.       46. A display device according to claim 45, characterized in that the point and origin of the coordinate system in which the coordinates are represented define two points as defined in claim 41.       A set of point pairs is identified in the same way as above, defining a match between the image to be transformed and the transformed image, and that this transformation is defined as a geometric transformation that preserves this matching. 47. A display device according to claim 46, characterized in that:       46. The camera according to claim 45, wherein the camera captures a screen when an image including a symbol is displayed by the projector P1, and the camera captures a screen when the same image is displayed by the projector P2. Display device.       A point defined in claim 42 is extracted from the symbol by image processing and then the digital signal processing device evaluates the coordinates of the point in the coordinate system defined in any of claims 42 and 43. 49. A display device according to claim 48, characterized in that photographed photographs are processed.       50. A display device according to any of claims 37 to 49, wherein the transformed image is obtained by mapping of pixels in an image that has not been transformed according to the transformation.       45. A display device according to claim 44, wherein the image comprises at least one corner.       52. A display device according to claim 49 and 51, wherein the image processing uses a corner extractor (e.g. Harris detector) characterized in that it has sub-pixel accuracy.       45. A display device according to claim 44, wherein the image comprises at least one circle.       54. A display device according to claim 49 and 53, wherein the image processing uses a circle extractor characterized in that it has sub-pixel accuracy.       45. The display device according to claim 44, wherein the image includes at least one line segment.       56. A display device according to claim 49 and 55, characterized in that a line segment extractor (e.g. Hough transform) characterized in that the image processing has sub-pixel accuracy.       45. A display device according to claim 44, wherein the image comprises at least one curve.       58. A display device according to claim 49 and 57, characterized by using a curve extractor (e.g. least square fitting) characterized in that the image processing has sub-pixel accuracy.       The display device according to any one of claims 44, 51, 53, 55, and 57, wherein the symbols are arranged and distributed.       60. The display device according to claim 44, wherein the image has a checkered pattern.       60. A display device according to claim 49 and 59, wherein the image comprises a quadrilateral.       One or more types of specific images intended to allow estimation of the parameter values of the distribution model of maximum radiance on the screen are projected by at least one projector Item 35. The display device according to Item 34.       63. The display device according to claim 62, wherein the specific image is a uniform image, that is, an image having the same pixel characteristics throughout the image.       64. A display device according to claim 63, characterized in that the specific image is a uniform image, the characteristics of the pixels are the same throughout the image and correspond to the maximum radiance that may be emitted. . The radiance distribution model is
The display apparatus in any one of 2-64. Here, A, a, b, M, i_max and j_max are six parameters evaluated by the digital processing device, i and j are the coordinates of the pixel displayed on the screen by one projector, and A is The maximum luminance value on the screen, a and b are parameters of the shape of an ellipse that characterizes the distribution of radiance for the image projected on the screen, i_max is the horizontal coordinate of the pixel corresponding to the maximum radiance, and j_max is the maximum radiance The vertical coordinate of the corresponding pixel, M, is related to the decrease in the radiance value around the location characterized by the maximum radiance.       66. The display device of claim 65, wherein the value of the model parameter is determined by an optimization process (e.g., least squares minimization).       36. A display device according to claim 35, characterized in that one or more specific images intended to allow evaluation of the values of the parameters of the colorimetric model or of the radiation model are displayed on the screen. .       The particular image is a single color uniform image, i.e. the value of the pixel channel is the same throughout the image and all values except one of the channels describing the state of the pixel are zero. 68. The display device according to claim 67, characterized in that:       For each monochromatic image displayed on the screen, a color or radiometric sensor plugged into a digital signal processing device (eg, a chromameter, spectrometer or spectroradiometer) can measure or display the composite image C displayed on the screen. 69. The display device according to claim 68, wherein a spectrum of chromaticity X, Y, Z or radiation displayed in the region is measured.       The display device according to claim 32, wherein the at least one sensor is a sensor capable of capturing an image on a screen.       33. A display device according to claim 32, characterized in that at least one sensor is a luminance meter (e.g. a video luminance meter) capable of measuring the luminance of one point or various points simultaneously.       The display device according to claim 32, wherein the at least one sensor is a colorimetric sensor capable of measuring the three color components X, Y, Z or the color coordinates x, y of the screen area. .       73. At least one digital output of the digital signal processing device is intended to communicate with at least one projector to set or change its adjustment. Display device.       74. A display device according to claim 73, characterized in that at least one output is RS232 or USB. The original image I is digital and is 3 × 8 bits (encoded in a low dynamic range format) or 3 × 10 bits or 3 × 11 bits or 3 × 12 bits (encoded in a high dynamic range format) The display device according to any one of claims 1 to 74, wherein High dynamic range of image
can do.       The input of the digital signal processing device is digital and compatible with the HDMI 2.0a format, and the output of the digital signal processing device is digital and compatible with the HDMI 2.0a format. Item 76. The display device according to any one of Items 1 to 75.       77. A display device according to claim 1, wherein at least one projector has a lens shift function.       The display device according to claim 79, wherein a lens shift function of the projector is used to superimpose the image projected by the projector on the screen.       77. The display device according to claim 1, wherein at least one projector has a keystone correction function.       The display device according to claim 79, wherein a trapezoidal correction function of the projector is used to superimpose an image projected by the projector on the screen.       77. The display device according to claim 1, wherein at least one projector has a zoom function.       The display device according to claim 79, wherein a zoom function of the projector is used to superimpose an image projected by the projector on the screen.       Some projectors (shown as projector set 1) have a physical configuration (position and orientation with respect to the screen) and lens shift adjustment, keystone correction, aimed at obtaining a full overlay of the images they project onto the screen And a zoom function, characterized by a minimum illuminated area of the screen, and other projectors (shown as projector set 2) are partially connected with at least one image projected by the projectors of projector set 1 The physical structure (position and orientation with respect to the screen) and lens shift adjustment, trapezoidal correction, and zooming functions for obtaining superposition are provided, according to any one of claims 1, 77, 79, 81 The display device described.       The projector of the projector set 1 has at least one image projected by one projector of the projector set 2 so that at least 50% of the image vertical width and 50% of the image horizontal width are in the overlapping region of the two images. 84. A display device according to claim 83, characterized in that it is superimposed on the screen with at least one image projected by.       At least one projector of the projector set 2 in order to avoid projecting on the screen light corresponding to a region of the image that is not superimposed on at least one image projected by one projector of the projector set 1 85. A display device according to claims 83 and 84, characterized in that it comprises a physical system intended to limit the aperture of the light beam.       86. A display device according to any one of the preceding claims, comprising at least two juxtaposed projection screens that form an angle that may be non-zero.       86. The projection screen has a horizontal (vertical) size that is smaller than the combined horizontal (vertical) size of the intersection of light beams projected by different projectors and the surface that maintains the screen. 86. A display device according to 86.       The projection screen allows active 3D visualization by alternating display of images corresponding to the viewer's right and left eyes, and is synchronized with glasses that prevent the left and right eyes from seeing the wrong image (this property is e.g. Obtained using liquid crystal based glasses that can set the transmissivity of), the alternating display of left and right images must be obtained using multiple projectors that are synchronized with each other and driven by a digital signal processor The display device according to claim 1, characterized by:       The projection screen enables passive 3D visualization resulting from simultaneous display of images corresponding to the viewer's right and left eyes, wherein the light corresponding to each image is characterized by a specific polarization. The display apparatus in any one of -88. The display must be viewed using specific glasses where the transmittance of the left and right eye glasses depends on the polarization of the light so that each eye sees the correct image. Polarizing filters are placed on the projector lens, some of which are polarized in a different direction than the others. 90. The display device according to claim 1, wherein the display device has a bundle, and F_max is a luminous flux or a radiant flux of the most powerful projector.       The display device according to claim 90, wherein the projectors have the same luminous flux or radiant flux.       The display device according to claim 1, wherein the at least one screen is a screen having a gain greater than 1 in a normal direction of the screen normal.       95. A display device according to any one of the preceding claims, characterized in that at least one projection screen is a rear projection screen that is not necessarily flat.       94. A display device according to any of claims 1 to 93, wherein the at least one projection screen is a front projection screen that is not necessarily flat.       The at least one image is projected onto a screen by a projector after one or more reflections, transmissions or refractions via at least one refractive or catadioptric optical system. 94. The display device according to any one of 94.       The resolution of the composite image C is higher than the resolution of the projector used in the system, and the original image I having a resolution higher than the resolution of the projector can be displayed. Display device.       The display device according to claim 96, wherein a shape of projection of each pixel in each image projected by each projector on the screen is evaluated.       62. The process of evaluating a shape is derived from the method of claims 37-61, taking into account not only the center of gravity of the shape but also the geometric shape of the projected pixel on the screen. 98. A display device according to claim 97.       99. The surface arrangement of pixels projected on a screen by a projector of the display device is evaluated with floating point precision in a coordinate system linked to the screen. Display device.       The display device according to claim 99, wherein a plane arrangement of pixels corresponding to the original image I is defined on the screen in the same coordinate system.       101. A display as claimed in claim 100, characterized in that the virtual plane arrangement of pixels and each pixel is defined as a shape corresponding to an overlap region of at least two pixels projected onto the screen by two different projectors apparatus.       102. Display according to claim 101, characterized in that the state of each pixel in each image projected by each projector is calculated such that a virtual plane arrangement of pixels constitutes an approximation of the original image I apparatus.       105. A display device according to claim 102, wherein the method of calculation is iterative and a number of pixels in a number of images projected by a number of projectors are calculated at each iteration.       The state of the pixels in the image projected by one projector is calculated metrically using the surface arrangement of the pixels in the original image I and taking into account the superposition of the shapes of the pixels projected on the screen. 104. The display device according to claim 103.       A residual image I_residual is calculated at each iteration, and this residual image is defined as the original image I calculated by iteration and subtracted from the images obtained by superimposing images projected by all projectors. 105. A display device according to any one of claims 103 and 104, wherein:       Said image I_residual is used in place of image I in the next iteration, is updated at each iteration and is repeated until all pixels of all images projected by the projector of the system are calculated, The display device according to claim 105.