DE102024003086A1 - Visual acuity-dependent adjustment of a display unit - Google Patents

Abstract

The invention relates to a method for adapting content shown on a display unit (1), comprising the steps of: capturing (S1) a face of a user of the display unit (1) with a camera (3) and generating a camera image of the captured face; determining (S2) whether the user cannot clearly perceive a display content and, if this is detected, determining a reason for this, in each case by comparing the camera image with a plurality of pre-stored sample images, or by analyzing the camera image and comparing an analysis result with a database of pre-stored features; if it is detected that the user cannot clearly perceive the display content: adapting (S3) the display content depending on the reason.

Description

The invention relates to a method for adapting contents displayed on a display unit, a system for adapting contents displayed on a display unit, and a vehicle with such a system.

Modern passenger cars often incorporate an interior camera. In particular, it is known that this camera can detect the direction of a vehicle occupant's gaze and control a display device in the vehicle's interior accordingly.

The DE 10 2012 206 030 A1 In this context, relates to a method for controlling at least one function of at least one display device for an interior of a vehicle, wherein the vehicle has at least one vehicle camera, wherein the method comprises the following steps: determining whether a viewing direction of at least one vehicle occupant, detected by means of the at least one vehicle camera, is directed towards a device area of the interior of the vehicle in which the at least one display device is installed; and controlling the at least one display device in a first display mode if the detected viewing direction of the at least one vehicle occupant is directed towards the device area, in order to control the at least one function of the at least one display device.

The DE 10 2022 103 606 A1 further relates to a vehicle occupant monitoring device for a vehicle, the vehicle occupant monitoring device comprising: an imaging device configured to image a passenger compartment of the vehicle to monitor the condition of occupants in the vehicle; a display configured to display a screen for viewing by the occupants in the vehicle; and a determining device configured to determine the condition of the occupants in the vehicle using captured image data obtained by imaging the respective occupants by the imaging device while the screen is displayed on the display, wherein a display panel configured to display the screen of the display has a non-display area on an inner side of a rectangular frame surrounding a display area of the screen, and wherein the imaging device is arranged on the back of the non-display area of the display panel of the display.

People with nearsightedness, also known as myopia, have difficulty seeing distant objects clearly. This can cause problems when using a head-up display (abbreviated to "HUD") in a vehicle, as the reflected content is often shown in relation to the surroundings, as if it were floating at a certain distance in front of the vehicle. For nearsighted people, this content therefore usually appears blurry in conventional head-up displays, making it difficult to quickly grasp important information such as speed or navigation instructions. This can lead to distractions and an increased risk of accidents, as the driver may need longer to successfully interpret the content.

The object of the invention is therefore to improve the recognizability, in particular readability, of content displayed on a display unit.

The invention is based on the features of the independent claims. Advantageous developments and refinements are the subject of the dependent claims.

• - Aufnehmen eines Gesichts eines Anwenders der Anzeigeeinheit mit einer Kamera und Erzeugen eines Kamerabilds des aufgenommenen Gesichts;
• - Ermitteln, ob der Anwender einen Anzeigeinhalt nicht klar wahrnehmen kann und, wenn dies erkannt wird, Ermitteln eines Grundes dafür, jeweils indem das Kamerabild mit einer Vielzahl vorabgespeicherter Musterbilder verglichen wird, oder indem das Kamerabild analysiert wird und ein Analyseergebnis mit einer Datenbank vorabgespeicherter Merkmale verglichen wird;
• - wenn erkannt wird, dass der Anwender den Anzeigeinhalt nicht klar wahrnehmen kann: Anpassen des Anzeigeinhalts abhängig vom Grund.

A first aspect of the invention relates to a method for adapting content displayed on a display unit, comprising the steps:

• - capturing a face of a user of the display unit with a camera and generating a camera image of the captured face;
• - Determining whether the user cannot clearly perceive a display content and, if this is detected, determining a reason for this, in each case by comparing the camera image with a plurality of pre-stored sample images or by analyzing the camera image and comparing an analysis result with a database of pre-stored features;
• - if it is detected that the user cannot clearly perceive the display content: Adjust the display content depending on the reason.

The method is particularly suitable for use in a passenger car, as the simple and quick interpretation of the display contents of a display unit is important, especially for the driver of the passenger car. If, for example, a display content with information such as the current vehicle speed, route guidance information or a maximum permissible speed is shown on a head-up display of the passenger car, it is relevant for the driver of the vehicle for safe and comfortable vehicle control to to be able to read and interpret as quickly as possible.

If the method is applied inside a vehicle, the driver's head, particularly their face, can be recorded cyclically at predefined intervals using an interior camera. The camera image thus generated is compared with a large number of pre-stored sample images depicting specific visual impairments. Alternatively, each camera image is analyzed and the analysis result is compared with pre-stored features. This can also be done implicitly by applying a machine learning model with the camera image as input, the output of which is the detection of a problem in the user's perception of the display content and a reason for this problem. For example, by applying the machine learning model to the camera image, nearsightedness, farsightedness, or another cause of a visual impairment can be detected through implicit feature detection. If such a machine learning model is applied, the comparison with pre-stored features of the database is carried out implicitly, since the machine learning model is pre-trained and thus already represents a database itself, but through the links in the machine learning model with corresponding non-interpretable associations and connections, which have arisen from a human-understandable source, so that even new users with their individual facial expressions and behavior patterns can be reliably interpreted and visual impairments can be detected.

A machine learning model can also be used, which considers a large number of camera images, for example, to capture and analyze a temporal progression of patterns. For example, nearsighted people are known to frequently squint their eyes in order to better see distant objects. The duration of the squinting can also be recorded and used as a relevant pattern, especially if the squinting correlates with a distant gaze outside the vehicle interior. Then, by applying the machine learning model to the interaction between gaze direction and squinting, it is possible to infer myopia.

An artificial neural network is a suitable model for machine learning, but other methods for pattern recognition can also be applied. State-of-the-art methods, such as the aforementioned artificial neural networks, can be used for this purpose. Depending on the method used to compare a particular camera image or an analysis result of a particular camera image, the comparison with the sample images or the pre-stored features in the database takes place explicitly or implicitly. As described above, the features in an artificial neural network can be stored implicitly, i.e., in a way that is uninterpretable for a human observer. This is because interpretable information is typically lost during pre-training of such an artificial neural network and is mapped to parameters such as weights.

If such a problem with the user's vision is identified, the display content on the display unit is adjusted depending on the identified reason for the visual impairment, such as the aforementioned farsightedness or nearsightedness.

The display content can also be adjusted dynamically, particularly in real time: Dynamic adjustment is possible if the user's vision is continuously monitored using camera images and the display content is adjusted immediately upon detection. This is particularly useful in changing lighting conditions or for extended applications with the display unit, such as long journeys in a vehicle with such a display unit, where eyes can easily become fatigued.

The advantageous effects of the invention are that road safety is significantly increased and drivers' sense of security is enhanced. Comfort is also increased. Symptoms of fatigue thus occur later or even not at all.

However, the method can also be used outside of vehicles, for example in virtual reality systems, on screens in aircraft or other environments where a precise view of important information is necessary.

According to an advantageous embodiment, in order to determine whether the user cannot clearly perceive a display content, an eye point of the user is measured and a gaze direction of the user is recorded and compared with a database of pre-stored features.

According to a further advantageous embodiment, the sample images and/or the pre-stored features of the database comprise indicators of myopia and/or hyperopia.

Such indicators can be explicitly defined, as described above, or implicitly defined through non-interpretable parameters of a machine learning model. Accordingly, currently generated camera images can be explicitly or implicitly compared with the previously stored indicators of myopia and/or hyperopia, and the display content can be modified accordingly.

• - Text wird vergrößert dargestellt;
• - Ein Icon wird vergrößert dargestellt;
• - Text wird verkleinert dargestellt;
• - Ein Icon wird verkleinert dargestellt;
• - Ein angezeigter Schlagschatten wird zur Kontrasterhöhung verstärkt angezeigt;
• - Ein angezeigter Schlagschatten wird zur Kontrasterhöhung vermindert angezeigt;
• - Angezeigte Farben werden stärker gesättigt;
• - Eine Anzeigeschärfe wird angepasst;
• - Ein Anzeigekontrast wird angepasst;
• - Eine Anordnung von Elementen eines Anzeigeinhalts wird verändert;

According to a further advantageous embodiment, if myopia or hyperopia is detected, a display content is adjusted by at least one of the following measures:

• - Text is displayed enlarged;
• - An icon is displayed enlarged;
• - Text is displayed in a smaller size;
• - An icon is displayed in a smaller size;
• - A displayed drop shadow is displayed intensified to increase contrast;
• - A displayed drop shadow is displayed in a reduced size to increase contrast;
• - Displayed colors become more saturated;
• - A display sharpness is adjusted;
• - A display contrast is adjusted;
• - An arrangement of elements of a display content is changed;

Adjusting the display sharpness, the display contrast, and changing elements of a display content are particularly suitable for adjusting the display content of a head-up display.

According to a further advantageous embodiment, the sample images and/or the pre-stored features of the database comprise indicators of astigmatism and/or presbyopia, i.e. age-related farsightedness.

According to a further advantageous embodiment, in order to determine whether the user cannot clearly perceive a display content, an analysis of the eye movements and the reaction to different lighting conditions is carried out, and the result of the analysis is compared with the database of pre-stored features.

According to a further advantageous embodiment, the determination of whether the user cannot clearly perceive a display content and, if this is detected, the determination of the reason for this are carried out in each case by using an infrared camera to determine a pupil reaction of the user and/or an eye tracking system to detect the user's gaze direction and by comparing the pupil reaction and/or the detected gaze direction with the database of pre-stored features.

This advantageously increases the reliability of recognition and enables even more precise adjustment of the display content.

A further aspect of the invention relates to a system for adapting content shown on a display unit, comprising a camera designed to record a face of a user of the display unit and to generate a camera image of the recorded face; and comprising a computing unit designed to determine whether the user cannot clearly perceive a display content and, if this is detected, to determine a reason for this, in each case by the computing unit comparing the camera image with a plurality of pre-stored sample images, or by analyzing the camera image and comparing an analysis result with a database of pre-stored features; and wherein the computing unit is designed to adapt the display content depending on the reason if it is detected that the user cannot clearly perceive the display content.

Advantages and preferred developments of the proposed system result from an analogous and analogous transfer of the statements made above in connection with the proposed method.

Another aspect of the invention relates to a vehicle having a system as described above and below, wherein the camera is an interior camera and the user is a passenger of the vehicle.

According to a further advantageous embodiment, the display unit is a head-up display.

Further advantages, features and details emerge from the following description, in which - if necessary with reference to the drawing - at least one embodiment is described in detail.

• 1: Ein Fahrzeug mit einem System zur Anpassung von auf einer Anzeigeeinheit dargestellten Inhalten gemäß einem Ausführungsbeispiel der Erfindung.
• 2: Ein Verfahren zur Anpassung von auf einer Anzeigeeinheit dargestellten Inhalten gemäß einem Ausführungsbeispiel der Erfindung.

They show:

• 1 : A vehicle with a system for adjusting information shown on a display unit provided content according to an embodiment of the invention.
• 2 : A method for adapting content displayed on a display unit according to an embodiment of the invention.

The representations in the figures are schematic and not to scale.

1 shows an interior of a vehicle 7 with a system for adapting content displayed on a display unit 1. The display unit 1 is part of the system and, in the present example, is designed as a head-up display. A camera 3 located in the vehicle interior and aimed at the driver cyclically monitors the driver's head at defined intervals. The eye point is measured and the direction of gaze is evaluated. A list of indicators of myopia stored in an external database or the internal memory of the vehicle 7 is queried and evaluated by a computing unit 5. The live images from the camera 3 are compared with the definition images from the database. For example, a nearsighted person often squints to better see objects that are further away. These patterns help an algorithm evaluate whether and to what extent the driver is nearsighted or farsighted. If the computing unit 5 detects nearsightedness or farsightedness, the display content placed on the display unit 1 is adapted. Text or icons are enlarged or reduced, drop shadows are increased or decreased for better contrast, and colors are more saturated depending on whether nearsightedness or farsightedness has been detected.

• - Aufnehmen S1 eines Gesichts eines Anwenders der Anzeigeeinheit 1 mit einer Kamera 3 und Erzeugen eines Kamerabilds des aufgenommenen Gesichts;
• - Ermitteln S2, ob der Anwender einen Anzeigeinhalt nicht klar wahrnehmen kann und, wenn dies erkannt wird, Ermitteln eines Grundes dafür, jeweils indem das Kamerabild mit einer Vielzahl vorabgespeicherter Musterbilder verglichen wird, oder indem das Kamerabild analysiert wird und ein Analyseergebnis mit einer Datenbank vorabgespeicherter Merkmale verglichen wird;
• - wenn erkannt wird, dass der Anwender den Anzeigeinhalt nicht klar wahrnehmen kann: Anpassen S3 des Anzeigeinhalts abhängig vom Grund.

2 shows a method for adapting content displayed on a display unit 1, comprising the steps:

• - Recording S1 a face of a user of the display unit 1 with a camera 3 and generating a camera image of the recorded face;
• - Determining S2 whether the user cannot clearly perceive a display content and, if this is detected, determining a reason for this, in each case by comparing the camera image with a plurality of pre-stored sample images, or by analyzing the camera image and comparing an analysis result with a database of pre-stored features;
• - if it is detected that the user cannot clearly perceive the display content: Adjust S3 the display content depending on the reason.

Although the invention has been illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived therefrom by those skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that embodiments mentioned by way of example really only represent examples that are not to be understood in any way as limiting the scope of protection, possible applications, or configuration of the invention. Rather, the foregoing description and the description of the figures enable those skilled in the art to specifically implement the exemplary embodiments. The skilled person, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

QUOTES CONTAINED IN THE DESCRIPTION

This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 10 2012 206 030 A1 [0003]
• DE 10 2022 103 606 A1 [0004]

Claims (10)

A method for adapting content displayed on a display unit (1), comprising the steps of: - capturing (S1) a face of a user of the display unit (1) with a camera (3) and generating a camera image of the captured face; - determining (S2) whether the user cannot clearly perceive a display content and, if so determined, determining a reason for this, in each case by comparing the camera image with a plurality of pre-stored sample images or by analyzing the camera image and comparing an analysis result with a database of pre-stored features; - if it is determined that the user cannot clearly perceive the display content: adapting (S3) the display content depending on the reason. Procedure according to Claim 1 , whereby to determine whether the user is unable to clearly perceive display content, an eye point of the user is measured and a direction of gaze of the user is recorded and compared with a database of pre-stored features. Method according to one of the preceding claims, wherein the sample images and/or the pre-stored features of the database comprise indicators of myopia and/or hyperopia. Procedure according to Claim 3 , whereby if myopia or hyperopia is detected, the display content is adjusted by at least one of the following measures: - text is displayed larger; - an icon is displayed larger; - text is displayed smaller; - an icon is displayed smaller; - a displayed drop shadow is displayed more strongly to increase the contrast; - a displayed drop shadow is displayed less strongly to increase the contrast; - displayed colors are more saturated; - display sharpness is adjusted; - display contrast is adjusted; - an arrangement of elements of the display content is changed; Method according to one of the preceding claims, wherein the sample images and/or the pre-stored features of the database comprise indicators of astigmatism and/or presbyopia. Procedure according to Claim 5 , whereby an analysis of eye movements and the reaction to different lighting conditions is carried out to determine whether the user is unable to clearly perceive display content, and the result of the analysis is compared with the database of pre-stored features. Method according to one of the preceding claims, wherein determining whether the user cannot clearly perceive a display content and, if this is detected, determining the reason for this, are each carried out by using an infrared camera (3) to determine a pupil reaction of the user and/or an eye tracking system to detect the user's gaze direction and by comparing the pupil reaction and/or the detected gaze direction with the database of pre-stored features. A system for adapting content displayed on a display unit (1), comprising a camera (3) configured to capture a face of a user of the display unit (1) and to generate a camera image of the captured face; and comprising a computing unit (5) configured to determine whether the user cannot clearly perceive display content and, if this is detected, to determine a reason for this, in each case by the computing unit (5) comparing the camera image with a plurality of pre-stored sample images, or by analyzing the camera image and comparing an analysis result with a database of pre-stored features; and wherein the computing unit (5) is configured, upon detection that the user cannot clearly perceive the display content, to adapt the display content depending on the reason. Vehicle (7) with a system according to Claim 8 , wherein the camera (3) is an interior camera (3) and the user is an occupant of the vehicle (7). Vehicle (7) to Claim 9 , wherein the display unit (1) is a head-up display.

Images