DE102023003964A1 - Method for operating an image projection system, device and vehicle - Google Patents

Abstract

The invention relates to a method and a device for operating an image projection system (6) in a vehicle (1), wherein a virtual image is projected onto a projection surface (P) in a field of vision of a driver (2) by means of the image projection system (6). According to the invention, it is provided that
- if the projection surface (P) in one area (B1) has different reflection properties than another area (B2) due to a change in the area, a deviation between the reflection parameters of this area (B1) and the unchanged further area (B2) is determined,
- for the area (B1, B2) with different reflection parameters, image pixels and/or image areas affected by the surface change are determined based on a surface geometry of the projection surface (P),
- the deviating reflection parameters are determined as a function of a projection and/or the surface geometry for each image pixel of the undistorted or distorted image and a matrix of deviating correction parameters is created,
- an inverted deviation of the reflection parameters is applied, taking into account a geometry of a projector and the surface geometry of the projection surface (P) with respect to the undistorted image, so that the projected image appears homogeneous on the projection surface (P). The invention further relates to a vehicle (1) with such a device.

Description

The invention relates to a method for operating an image projection system in a vehicle, wherein the image projection system projects a virtual image onto a projection surface in the field of vision of a driver of the vehicle. Furthermore, the invention relates to a device for implementing the method and a vehicle having such a device.

From the DE 10 2021 127 555 A1 An LCD display unit for use in a reflective display system in a motor vehicle and a reflective display system are known. The LCD display unit comprises an LCD pixel matrix for selectively controlling pixels to display a display image on a display surface; a backlight arrangement divided into selectively controllable dimming zones for illuminating areas with multiple pixels with backlight; a control unit configured to control the LCD pixel matrix so that the display image is output, which is distorted using a warping function to compensate for distortion of the display image due to reflection at a reflective area; wherein the dimming zones have a shape and size determined by the warping function.

In addition, the DE 10 2015 116 160 A1 A head-up display unit for a vehicle for generating a virtual image of the vehicle's surroundings in the driver's field of vision. The head-up display unit comprises an emission unit for generating and emitting an image signal that is projected into the driver's field of vision along a multiply folded projection path. The driver perceives the projected image signal as the virtual image. The display unit comprises a movable reflector that influences part of the projection path. The representation of the virtual image is influenced by a controlled movement of the emission unit and additionally by a modification of the emitted image signal compared to a raw image signal. The movement of the emission unit is carried out depending on a detected driving situation.

The invention is based on the object of specifying a method for operating an image projection system in a vehicle, a device for carrying out the method and a vehicle with such a device.

The object is achieved according to the invention by a method which has the features specified in claim 1, by a device which has the features specified in claim 8 and by a vehicle which has the features specified in claim 9.

Advantageous embodiments of the invention are the subject of the subclaims.

• - wenn die Projektionsfläche in einem Bereich gegenüber einem weiteren Bereich aufgrund einer Flächenveränderung abweichende Reflexionseigenschaften aufweist, eine Abweichung zwischen Reflexionsparametern dieses Bereiches und des unveränderten weiteren Bereiches ermittelt werden,
• - für den Bereich mit den abweichenden Reflexionsparametern von der Flächenveränderung betroffene Bildpixel und/oder Bildbereiche in einem unverzerrtem Bild oder in einem verzerrten Bild anhand einer Oberflächengeometrie der Projektionsfläche ermittelt werden,
• - die abweichenden Reflexionsparameter in Abhängigkeit von einer Projektion und/oder der Oberflächengeometrie und/oder gegebenenfalls einer Position eines Betrachters für jedes Bildpixel des unverzerrten oder des verzerrten Bildes ermittelt werden und eine Matrix abweichender Korrekturparameter erstellt wird,
• - eine invertierte Abweichung der Reflexionsparameter unter Berücksichtigung einer Geometrie eines Projektors und der Oberflächengeometrie der Projektionsfläche in Bezug auf das unverzerrte Bild angewendet wird, so dass das projizierte Bild auf der Projektionsfläche homogen erscheint.

A method for operating an image projection system, in particular a head-up display unit, in a vehicle, wherein a virtual image is projected onto a projection surface in a field of vision of a driver of the vehicle by means of the image projection system, provides according to the invention that

• - if the projection surface in one area has different reflection properties than another area due to a change in the area, a deviation between the reflection parameters of this area and the unchanged other area can be determined,
• - for the area with the deviating reflection parameters, image pixels and/or image areas affected by the surface change are determined in an undistorted image or in a distorted image based on a surface geometry of the projection surface,
• - the deviating reflection parameters are determined as a function of a projection and/or the surface geometry and/or, if applicable, a position of an observer for each image pixel of the undistorted or distorted image and a matrix of deviating correction parameters is created,
• - an inverted deviation of the reflection parameters is applied taking into account a geometry of a projector and the surface geometry of the projection surface with respect to the undistorted image, so that the projected image appears homogeneous on the projection surface.

By applying the method, it is possible for the projected image to have a neutral and homogeneous color reproduction as well as a neutral and homogeneous brightness impression across the entire projection surface despite the area with the deviating reflection parameters, even if the projection surface itself does not have a neutral and/or homogeneous color reproduction.

In particular, the method can be used to compensate for the area of the projection surface which has the deviating reflection parameters, for example due to a laser surface and/or a decoating.

In addition, the procedure provides for the possibility of storing a viewer’s position for further to take into account the subsequent correction of the projected image.

In one embodiment, the brightness and/or color reproduction of the projected image is or are checked to determine the deviating reflection parameters. In particular, the brightness and/or color reproduction is or are checked to determine whether the projected image exhibits visible inhomogeneities due to changes in optical reflection properties due to lasering and/or stripping of the projection surface.

In a further embodiment, the differing reflection parameters for each brightness value of each color channel of the image comprise a correction offset or a correction factor, which is represented as a color-channel-specific gradation curve. Using such a gradation curve, it is possible to edit the pixels of the image, offering a comparatively large number of options for changing the color channels and thus post-processing the image in terms of brightness and/or color.

Furthermore, in one embodiment of the method for determining deviating reflection parameters, the beam entrance angle of a light beam projected by the projector onto the projection surface is determined. In particular, the beam entrance angle is determined to detect inhomogeneities in the projected image and, if necessary, correct them so that the projected image appears homogeneous and uniform to the viewer, even in the altered area of the projection surface.

In one possible embodiment, in order to determine deviating reflection parameters, a position of an observer in the vehicle relative to the projection surface is determined in order to take this into account when correcting the projected image.

For this purpose, in a further development of the method, the viewer's position is determined based on signals acquired by an interior camera and/or a depth sensor and/or an ultrasound-based sensor and/or a time-of-flight camera. In particular, based on the determined position of the viewer relative to the projection surface, a head orientation and, in particular, a viewing direction of the viewer are determined, which are taken into account when correcting the image.

In one embodiment, in order to stabilize transitions between areas with color correction and without color correction, as well as between different corrections in a transition area, an interpolation between the two correction values is carried out in order to optimize a brightness and/or color gradient so that the image projected onto the projection surface appears homogeneous and uniform to the viewer.

Furthermore, the invention relates to a device for carrying out the method. According to the invention, an image source is designed to provide the image to be projected in the form of a pixel matrix, wherein each pixel of the pixel matrix is assigned a brightness value and a color value. Furthermore, a distortion element is provided which is designed to invert any curvature of the projection surface and to convert the image of the image source into a distorted image. The projector is designed to project the distorted image onto the projection surface, wherein the projection surface is designed to reflect the distorted image towards the viewer and the projector and the projection surface are designed to generate an undistorted image based on the distorted image.

By means of the device, it is possible to determine the determined deviating reflection parameters and to correct them with respect to the projected image, so that inhomogeneities of the projection in image brightness and/or color impression can be compensated and the projected image appears homogeneous and uniform to the viewer.

The invention further relates to a vehicle with a device for implementing the method, wherein an image from the vehicle is projected onto a windshield of the vehicle by means of the image projection system, and reflective properties of the windshield are utilized to create the impression, from the viewer's perspective, that the projected image is floating in the air in front of the vehicle. The device projects a homogeneous and uniform image into the viewer's field of vision, even if the windshield as a projection surface has an area with deviating reflection parameters.

Embodiments of the invention are explained in more detail below with reference to a drawing.

• 1 schematisch eine perspektivische Ansicht eines Ausschnittes eines Fahrzeuges mit einem Fahrer auf einem Fahrzeugsitz.

The following shows:

• 1 schematically a perspective view of a section of a vehicle with a driver on a vehicle seat.

The single figure shows a perspective view of a section of a vehicle 1 with a driver 2 on a vehicle seat 3. In addition, 1 an interior camera 5 integrated into a rearview mirror 4 and an exemplary and highly simplified image projection system 6 The image projection system 6 is designed, for example, as a head-up display unit, hereinafter referred to as head-up display unit 6. Alternatively, the image projection system can also be designed differently.

By means of such a head-up display unit 6, an image is projected onto a windshield 7 of the vehicle 1. Reflective properties of the windshield 7 are utilized to create the impression, from the perspective of the driver 2, i.e., a viewer, hereinafter referred to as viewer 2, that the image projected into the field of view of the viewer 2 is floating in the air in front of the vehicle 1. For example, the entire windshield 7 serves as the projection surface P of the head-up display unit 6.

It is generally known that such a windshield 7 is coated. If the coating is metallic, for example, for conducting electricity to generate heat in order to defrost and/or defrost the windshield 7, wherein the windshield 7 is heated or the windshield 7 comprises infrared heat-insulating glazing, the problem may arise that radio communication from the vehicle 1 to and from the vehicle 1 is impaired. In order to still be able to carry out such radio communication, this metallic coating is removed in the vicinity of a radio unit of the vehicle 1, for example by means of a laser-based method.

By removing the metallic coating in an area B1 of the windshield 7, optical properties of the windshield 7 change locally in the area B1 of a decoating, whereby this changed area B1 can be distinguished with the naked eye from another area B2.

Due to the removal of the coating in the area B1, optical reflection properties in this area B1 can also change, so that the area B1 can be visible in an image projected by means of the head-up display unit 6 as an inhomogeneity of the projection in image brightness and/or color impression.

In the following, a device and a method for operating the head-up display unit 6 are described, which enable a neutral and homogeneous color reproduction and a brightness impression to be generated on the entire projection surface P, even if the projection surface P itself does not have a neutral and/or homogeneous color reproduction.

The device in the form of a projection system has an image source which provides the image to be projected in the form of a pixel matrix consisting of rows and columns, wherein each pixel of the pixel matrix is assigned a brightness value and a color value, for example an RGB value.

In addition, the device comprises a distortion element which inverts any curvature of the projection surface P, i.e., the windshield 7, and/or any distortion caused by the device, thus enabling the image provided by the image source to produce a rectangular, undistorted image impression from the viewer's perspective after projection. The distortion element can be embodied as a computer unit which converts the image of the image source into a distorted image using properties of a projector of the device and a surface geometry of the projection surface P, for example, by means of two-dimensional interpolation. Alternatively or additionally, the distortion element can be embodied as a distorting optical element arranged between the projector and the windshield 7.

By means of the projector as a further component of the device, the image distorted by the distortion element is projected onto the projection surface P, which reflects a light beam in the direction of the viewer 2.

By means of the projector and the projection surface P, the distortion of the image caused by the distortion element is eliminated, i.e., corrected again, in order to create an undistorted image impression.

For example, using the distortion element embodied as a computer unit or another computer unit of the device, a pixel-by-pixel, section-by-section, or complete change in the brightness and/or color of the image can be effected, for example, by using a gradation curve, with a gradation curve for each color channel. In this case, either the undistorted image of the image source or the distorted image is changed.

As described above, the optical properties of the projection surface P can be partially or completely changed, for example by lasering and/or other de-coating, in particular in the vicinity of a vehicle-mounted radio unit.

For the modified area B1, whose reflection properties compared to the reflection If the reflection properties of the additional area B2 of the projection surface P deviate, a deviation in the reflection parameters is determined, for example, using a colorimeter. Alternatively, the deviation of the reflection parameters of the additional area B2 from the reflection properties of the modified area B1 of the projection surface P is determined.

The pixels or image areas affected by the area change in the undistorted image or the distorted image are determined based on the surface geometry of the projection surface P.

As an alternative to the determination of the deviation of the reflection parameters between the changed area B1 and the unchanged area B2 described above, the deviation of the reflection parameters compared to a reference surface with color and brightness-true projection reproduction is determined for a completely homogeneous projection surface P.

The deviating reflection parameters can affect the color reproduction and/or the brightness reproduction in the projected image. For example, a correction offset or a correction factor can be provided for each brightness value of each color channel of the image, which can be represented in the form of a color-channel-specific gradation curve.

Furthermore, the deviating reflection parameters can depend on a beam entry angle (elevation) of the light beam projected by the projector onto the projection surface P. In the case of a structured projection surface P, the deviating reflection parameters can additionally depend on an incident direction of the light beam projected by the projector onto the projection surface P relative to the structuring (azimuth). Assuming a known reference position of the observer 2, the deviating reflection parameters are fixed values that can be determined by designing the device, i.e., the projection system, and the projection surface P in the vehicle 1.

The differing reflection parameters may further depend on a position of the observer 2 in the vehicle 1, since when viewing the same pixel from different positions of the observer 2, an angle of light incidence through the projector changes in elevation and azimuth, as does a reflection point of each projected pixel on the projection surface P. For this purpose, the position of the observer 2 is determined based on detected signals from the interior camera 5 of the vehicle 1. Alternatively or additionally, the position of the observer 2 in the vehicle 1 can also be determined based on detected signals from a depth sensor and/or an ultrasound-based sensor and/or a time-of-flight camera.

In one embodiment of the method, the deviating reflection parameters can also be determined for each pixel of the distorted or undistorted image as a function of the projection, in particular the surface geometry of the projection surface P, and optionally the position of the observer 2. The deviating reflection parameters holistically form a matrix of deviating correction parameters for each pixel of the distorted or undistorted image, in particular with respect to the correction offset or the correction factors.

An inverted deviation of the reflection parameters can be applied to the undistorted image by means of the computer unit or the further computer unit, taking into account a geometry of the projector and the surface geometry of the projection surface P, so that the projected image appears homogeneous and uniform to the viewer 2 even in the changed area B1 of the projection surface P or appears the same as the projection on the reference surface in terms of color and brightness reproduction.

In a further embodiment, the position of viewer 2 is taken into account when performing the correction.

Furthermore, to stabilize transitions between the changed area B1 and the unchanged area B2 of the projection surface P with and without color correction as well as between different corrections in a transition area, an interpolation between both correction values can be carried out.

The distorted or undistorted image thus corrected for color and/or brightness is projected by means of the projector and reflected by the projection surface P, whereby undesired reflection properties or altered areas B1 of the projection surface P are no longer visible to the viewer 2 and thus a homogeneous and color-neutral projection surface P is visible in terms of color reproduction and brightness impression.

List of reference symbols

1

vehicle

2

Driver/Viewer

3

vehicle seat

4

rearview mirror

5

Interior camera

6

Image projection system/head-up display unit

7

windshield

B1, B2

changed area

P

Projection surface

QUOTES CONTAINED IN THE DESCRIPTION

This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 10 2021 127 555 A1 [0002]
• DE 10 2015 116 160 A1 [0003]

Claims (9)

Method for operating an image projection system (6) in a vehicle (1), wherein a virtual image is projected onto a projection surface (P) into a field of vision of a driver (2) of the vehicle (1) by means of the image projection system (6), characterized in that - if the projection surface (P) has different reflection properties in one area (B1) compared to a further area (B2) due to a change in the area, a deviation between reflection parameters of this area (B1) and the unchanged further area (B2) is determined, - for the area (B1, B2) with the different reflection parameters, image pixels and/or image areas affected by the change in the area are determined in an undistorted image or in a distorted image based on a surface geometry of the projection surface (P), - the different reflection parameters are determined for each image pixel of the undistorted or distorted image as a function of a projection and/or the surface geometry and/or optionally a position of an observer (2). are determined and a matrix of deviating correction parameters is created, - an inverted deviation of the reflection parameters is applied taking into account a geometry of a projector and the surface geometry of the projection surface (P) with respect to the undistorted image, so that the projected image appears homogeneous on the projection surface (P). Procedure according to Claim 1 , characterized in that in order to determine the deviating reflection parameters, a brightness and/or color reproduction of the projected image is or is checked. Procedure according to Claim 1 or 2 , characterized in that the deviating reflection parameters for each brightness value of each color channel of the image comprise a correction offset or a correction factor which is represented as a color channel-specific gradation curve. Method according to one of the preceding claims, characterized in that in order to determine deviating reflection parameters, a beam entry angle of a light beam projected by the projector onto the projection surface (P) is determined. Method according to one of the preceding claims, characterized in that in order to determine deviating reflection parameters, a position of an observer (2) in the vehicle (1) relative to the projection surface (P) is determined. Procedure according to Claim 5 , characterized in that the position of the observer (2) is determined on the basis of detected signals from an interior camera (5) and/or a depth sensor and/or an ultrasound-based sensor and/or a time-of-flight camera. Method according to one of the preceding claims, characterized in that in order to stabilize transitions between areas (B1, B2) with a color correction and without color correction as well as between different corrections in a transition area, an interpolation between the two correction values is carried out. Device for carrying out the method according to one of the Claims 1 until 7 , characterized in that - an image source is designed to provide the image to be projected in the form of a pixel matrix, each pixel of the pixel matrix being assigned a brightness value and a color value, - a distortion element is designed to invert any curvature of the projection surface (P) that may be present and to convert the image of the image source into a distorted image, - the projector is designed to project the distorted image onto the projection surface (P), - the projection surface (P) is designed to reflect the distorted image in the direction of the viewer (2), wherein - the projector and the projection surface (P) are designed to generate an undistorted image based on the distorted image. Vehicle (1) with a device according to Claim 8 .

Images