TWI822186B - Computer-implemented method of adapting a graphical user interface of a human machine interface of a vehicle, computer program product, human machine interface, and vehicle - Google Patents

Abstract

Described is a computer-implemented method of adapting a graphical user interface of a human machine interface of a vehicle, wherein the graphical user interface is controlled by a controller of the human machine interface, wherein the layout of information presented on the graphical user interface is determined by a configuration, wherein the controller accesses at least two different configurations, wherein each configuration is associated with at least one clutter index or clutter index range, wherein the controller associates at least one of user profile, region, usage information, and/or user alertness with a clutter index or clutter index range and selects a configuration with a compatible clutter index or clutter index range.

Description

Computer implementation of graphical user interface for adapting vehicle human-machine interface Laws, computer program products, human-machine interfaces and vehicles

The present disclosure relates to computer implementation methods, computer program products, human-machine interfaces and vehicles for adapting a graphical user interface of a vehicle.

It is generally known in the prior art that much attention needs to be paid to how information is provided to people, especially when the information is important for decision making, especially in complex and time-sensitive situations. This situation often occurs when a vehicle is driving, and people need real-time information about many key parameters of the vehicle, such as driving speed. This information must be accessible in such a way that the driver is distracted from the traffic situation as little as possible while obtaining it.

What cognitive science has identified is that there is optimal information that an individual driver should receive, neither too little nor too much. If the driver receives too little information, the driver may search for information that the driver desires. If the driver receives too much information, the driver may be overwhelmed and unable to quickly identify the necessary information.

The amount of information to be presented is very individual. It is related to the driver's cognitive ability, age, experience and personal preferences. For example, some drivers would like to know about engine temperature, other drivers may not care about this data. In another example, some drivers may prefer easy access to navigation information, and other drivers may prefer entertainment information. In addition, there are very different types of users; for example, very busy users prefer more information density, while digital novices prefer a reduced interface and few key features.

It is well known that in different markets, people often have different perceptions of the density and quantity of information provided to them on human-computer interfaces. For example, users in China generally prefer to display more information and higher feature density, while in contrast, many users in the EU prefer streamlined and clean digital interfaces.

Therefore, it is well known that designing interfaces suitable for all such markets and user types is a common challenge.

It is also known that many driver assistance systems such as adaptive cruise control and automatic lane change systems have been successfully deployed on the market to improve driver comfort and safety. As these driver assistance systems become more sophisticated, less driver interaction may be required. In some cases, these driver assistance systems may become fully automated during a journey. Therefore, the driver's role has changed from that of an active driver to that of a passenger, at least for some parts of the trip. Highly automated vehicles allow that driver to relinquish control to the automated vehicle and perform other tasks while driving. User requirements for display information in an autonomous driving mode are different from those in a manual driving mode.

EP 3 240 715 B1 discloses a self-adaptive user interface system for a vehicle equipped with an automatic vehicle system. The self-adaptive user interface system includes: a display; and an electronic controller electrically coupled to the display, and configured to generate a signal indicative of the operation of the automated vehicle system Graphical user interface, outputting the graphical user interface on the display, the electronic controller is characterized by being further configured to monitor a mark of a driver's comfort level, and determine when the driver based on the monitored mark is uncomfortable with the operation of the automated vehicle system, and the graphical user interface is modified in response to determining that the driver is uncomfortable with the operation of the automated vehicle system.

It is therefore an object of the present invention to allow a more subtle way of adjusting the way information is displayed to a user of a vehicle so that the user is not overwhelmed but instead receives an appropriate level of information.

The object is a computer implementation method for adapting a graphical user interface of a vehicle human-machine interface according to claim 1, a computer program product according to independent claim 11, a human-machine interface according to independent claim 12 and A vehicle according to independent claim 13. Further embodiments are described in various dependent claims.

Describes a computer implementation method for adapting a graphical user interface of a vehicle's human-machine interface, wherein the graphical user interface is controlled by a controller of the human-machine interface, wherein is presented on the graphical user interface The information layout is determined by a configuration, where the controller accesses at least two different configurations, where each configuration is associated with at least one clutter indicator or clutter indicator range, where the controller accesses a user profile, region, At least one of usage information and/or user alertness is associated with a clutter indicator or clutter indicator range, and a configuration with a compatible clutter indicator or clutter indicator range is selected.

The vehicle may be a car, a truck, a bus or the like.

In particular, the graphical user interface can be a cockpit display, a head-up display or A central display. Using displays instead of gauges has become a standard in many vehicles because they are more configurable than traditional gauges.

Clutter index is an index derived from how a person perceives a certain information configuration. In one embodiment, the clutter metric for a given configuration can be evaluated by a tester. In another more adaptable embodiment, the clutter indicator may be calculated by a processor. The clutter indicator may relate to the amount, positioning and density of information and decorations, fonts used, including, in some embodiments, size and type, colors used and other criteria. The more cluttered the displayed content, the higher the clutter indicator will be. The higher the clutter index, the more precise a user needs to know where to look for a piece of corresponding information in order to find it quickly and efficiently.

These configurations may be pre-configured and/or user-adjustable and/or dynamic. Many dynamic display content can react to certain situations. For example, if a navigation system provides a direction notification that will be acted upon soon, this information may be more visible than other information, which will then be moved. to another location, not displayed or displayed at a smaller size.

In some embodiments, configurations may exist with similar or identical clutter indicators or overlapping clutter indicator ranges. Different configurations may be ordered according to the clutter index. The clutter metrics may be determined within a number of ranges and they may overlap so that more than one clutter metric may be determined to be appropriate.

Depending on the external environment as described above, the controller may select one or more suitable configurations. To do this, the controller must calculate a clutter index or clutter index range from at least one of the user profile, zone, usage information and/or user alertness information provided to the controller. The controller can then compare and match this calculated clutter index or index range with clutter indexes or clutter index ranges associated with available configurations and select one or more the closest matching configuration.

The clutter metric can be calculated as a rounded sum of weighting factors. In some embodiments, the factors can be later normalized and weighted, or the weights can be calculated such that they result in normalization of the factors.

These factors may include factors related to the usage of certain functions of the vehicle's infotainment system, such as an average variation in the usage of certain functions over a given period of time, such as the last month. Another factor may be related to the usage of a smartphone, such as the average variation in the usage of certain features within the smartphone over a given time period. Both factors are related to user preferences regarding the types of applications a user uses. Another factor may be a user-selected information density on the vehicle's human-machine interface. Another factor can be derived from monitored eye tracking and widget usage over a given period of time. Another factor can be derived from monitoring the user's voice interaction with the human-machine interface.

The clutter indicator is linked to a dedicated profile of a user. It can be analyzed at a group level within a cloud application, for example, generating specific weights for different regions and/or demographic groups.

That way, the user can be provided with display configurations that match the user's needs or capabilities.

In a first further embodiment, the selected configuration is presented to the user, wherein if the user confirms the selected configuration, the configuration is changed to the selected configuration.

This allows the user to control the user's preferences and configurations they want to use.

In another further embodiment, the clutter indicator is derived from the density of information displayed on the graphical user interface at a given time.

The information density can be calculated based on the number of different pieces of information displayed simultaneously on the display. For example, if a menu is displayed, a menu with 12 menu items displayed at the same time is more complicated than a menu with only 6 menu items. As another example, a cockpit display that simultaneously displays speed, engine RPM, engine temperature, fuel level, outside temperature, navigation information, etc. often appears more cluttered than a display that only displays speed and navigation information.

In another further embodiment, the controller analyzes usage of functions accessible by the user through the human-machine interface, wherein the controller adjusts at least one configuration based on the usage analysis.

That way, the controller can decide which information is relevant to a specific user and can regularly prioritize the information that that user needs. For example, if the user often wants to view navigation information, this information may be viewed with a higher priority than, for example, information about the current engine status.

In another further embodiment, a driver monitoring system is connected to the controller, wherein driver information captured by the driver monitoring system is processed by the controller, and wherein the controller selects a driver based on the driver information. configuration.

That way, if the driver seems less attentive, the display configuration can be changed to one with a lower clutter index.

In another further embodiment, the human-machine interface includes a voice recognition system, wherein information related to the usage of many voice commands is analyzed by the controller, wherein the controller selects a configuration based on the usage of many voice commands. .

In this embodiment, the information presented on the display can be matched with the commands given by the user, making the information more relevant to the individual user.

In another further embodiment, a selection of a configuration is presented at a predetermined time interval.

For example, such a time interval may be one month so that the user has a regular reminder that the user's needs may have changed. It can also keep the user engaged in customizing the vehicle to the user's preferences, which has been shown to have a beneficial impact on user satisfaction and brand appreciation.

In another further embodiment, a mobile device is connected to the human-machine interface, wherein the controller analyzes a configuration of the mobile device, and wherein the controller selects a configuration based on the configuration of the mobile device.

It has been found that many users tend to have the same preferences for how they display information across many different settings, mobile devices, computers and vehicles. In this way, using the information from the mobile device, the user experience in the vehicle can be improved and the closest configuration can be selected.

In a further embodiment, the controller applies a machine learning algorithm to the selection of the configuration.

Once a certain user group size is reached in a certain area and/or population group, such as 60 or more users, the learning mechanism can be implemented through reinforcement learning.

In another further embodiment, a user preference profile is stored and accessed by the controller.

The user preference profile may be stored in the vehicle and/or in an external storage location such as a server. The server can be accessed via a remote network connection. The profile may also be stored or accessed via a mobile device of the user.

A first independent aspect relates to a computer program product having a non-transitory computer-readable storage medium having a plurality of commands embedded therein, and when the commands are processed by a processor When executed, the processor is caused to perform the method described above.

Another independent aspect relates to a human-machine interface for a vehicle having a controller including a computer program product having a non-transitory computer-readable storage medium and a processor as described above.

Another independent aspect relates to a vehicle having a human-machine interface as described above.

2:Car

4: Human-machine interface

6: Cockpit display

8: Central display

10:Microphone

12: Speaker layout

14: Driver

16:Controller

18: Processor

20: Non-transitory computer-readable storage media

22:Computer program products

26: Driver monitoring system

28:Eye-tracking camera

30:eyes

32: Observation direction

34:Mobile phone

40:Select area

42:Configuration

44:Configuration

46:Configuration

50:Left area

52:Central area

54:Right area

56:Lower area

60:Sliding piece

62: Orbit

64: Stop position

Further features and details will be set forth in the following description, in which at least one exemplary embodiment will be described in detail with reference, where applicable, to the drawings. The features described and/or shown by themselves or in any possible and meaningful combination constitute the subject matter and are ultimately independent of the claims. In particular, they can be the subject of one or more separate applications. The drawings schematically show: [Fig. 1]: a car with multiple displays; [Fig. 2a]: one display configuration; [Fig. 2b]: different display configurations; [Fig. 2c]: different Display configuration; and [Figure 3]: a method of selecting a display configuration.

Figure 1 shows a car 2. The car 2 is equipped with a human machine interface (HMI) 4.

The human-machine interface 4 includes a cockpit display 6 , a central display 8 , a microphone 10 and a loudspeaker arrangement 12 . The central display 8 is touch sensitive. The human-machine interface 4 allows interaction between a driver 14 and various systems of the car 2 . Information is provided to the driver 14 via displays 6 and 8 and the speaker arrangement 12 . The driver 14 can enter commands via the microphone 10 and via the touch-sensitive central display 8 as well as via other well-known but not shown means, such as buttons and levers.

The human-machine interface 4 includes a controller 16 that includes a processor 18 and a non-transitory computer-readable storage medium 20 . A computer program product 22 is stored on the non-transitory computer-readable storage medium 20 . When the computer program product 22 is loaded and executed by the processor 18, the processor implements the methods described herein.

The controller 16 is responsible for providing information to be displayed to the displays 6 and 8 . The controller 16 also controls which information is provided as sounds via the speaker arrangement 12 .

The controller 16 is also connected to a driver monitoring system 26 having an eye tracking camera 28 for tracking the eyes 30 of the driver 14 . The driver monitoring system 26 also utilizes the microphone 10 to record sounds produced by the driver 14 . The eye tracking camera 28 may record a viewing direction 32 of the driver 14 . This information, along with the acoustic information captured via the microphone 10, may be used to determine how focused the driver 14 is at the moment. This information can be used to adjust the display 6 and/or the display Monitor 8 configuration.

The controller 16 is further connected to a mobile phone 34 of the driver 14 via a wired or wireless connection. Information related to the configuration of the mobile device (eg, 34 in the figure) is used by the controller 16 to determine an appropriate configuration of the displays 6, 8.

Figures 2a to 2c show different configurations of the cockpit display 6 and how to make a selection.

Displayed is a screen of the central display 8 for adjusting the configuration of the cockpit display 6 presented at the central display 8 . Below the presentation of the central display 6, a selection area 40 is presented, which the driver 14 can use to choose between different configurations, such as the configuration 42 shown in Figure 2a, the configuration 42 shown in Figure 2b Configuration 44 and configuration 46 as shown in Figure 2c.

The cockpit display 6 is usually divided into four different zones, three upper zones, a left zone 50 , a central zone 52 and a right zone 54 , and a lower linear zone. ,line-like zone)56.

The clutter index is related to the density of information provided on the cockpit display 6 and is calculated based on how much information is provided to the driver 14 at a given time. Based on the location of the information and the type of information, different information is given different weights. For example, information provided in the central area 52 has a higher impact on the clutter indicator than information provided in the left area 50 , the right area 54 , or the lower area 56 . On the other hand, the type of information is also important. Information that is mandatory or frequently retrieved by the driver 14 such as the speed of the car 2 has a lower impact on the clutter index than other less relevant information such as engine temperature.

In a specific exemplary embodiment, the clutter index C can range from 1 to 6: C=round(A1×W1+A2×W2+A3×W3+A4×W4+A5×W5), Among them: A_n=many factors; W_n=many weights; Round()=rounding function.

These factors are described as follows: A1 is the mean variance of the function usage in the human-machine interface last month. The range of A1 is 0 to 1 (1=100% is the percentage value); for example, A1=0.8 (80%). A1 represents the daily usage of many main functions of the infotainment system: navigation, air conditioning, media, driver assistance functions, voice assistance functions, and communication functions. An increased amount of variation will result in functionality being promoted to a higher level, such as the top level, so that more direct shortcut functions are available on the main screen of the human-computer interface, resulting in a higher information density.

W1 is preset equal to 1. In a further embodiment, W1 is adapted due to reinforcement learning, for example for specific regions or population groups.

A2 is the average variation in usage of smartphone functions. The range of A2 is 0 to 1 (1=100% is the percentage value); for example, A2=0.8 (80%). A2 represents the daily usage of all the main functions in the smartphone: messaging, calls, media, games, assistive uses, smart home applications, news, etc. An increased amount of variation will result in functionality being promoted to a higher level, such as the top level, so that more direct shortcut functions are available on the main screen of the human-computer interface, resulting in a higher information density.

W2 is defaulted to equal to 1. In further embodiments described above, W2 is adapted due to reinforcement learning, for example to target specific regions or population groups.

A3 is the information density of the selected human-machine interface. A3 represents many highly relevant components (e.g. open/buttons with labels and descriptions) divided by the maximum number of widgets in a high information density interface (e.g. 20 widgets). For example, 10 highly correlated components result in an A3 of 0.5.

W3 defaults to equal to 1. In a further embodiment, W3 is adapted due to reinforcement learning, such as for specific regions or population groups.

A4 involves eye tracking and component usage. A4 can be composed of multiple sub-factors based on many components within the human-machine interface. For example, a slider or a button can be regarded as a component. A4 can be described as follows: A4=(A4_F1×SW1+A4_F2×SW2+A4_F3×SW3+A4_F4×SW4)×W4

Served with:

F1: The average frequency of checking many components every week; the value range is 0 to 1 (1=100% is the percentage value).

F2: The average frequency of viewing and interacting with many components per week; the value range is 0 to 1 (1=100% is the percentage value).

F3: Check the average variation of many components every month; the value range is 0 to 1 (1=100% is the percentage value).

F4: Check the average variation of many components and interactions with many components every month; the value range is 0 to 1 (1=100% is the percentage value).

In addition, a sub-weight SWn condition applies to: SW1+SW2+SW3+SW4=1

For each component, different sub-weights can be used to adapt the sequence, such as scene settings, and locations within the HMI, such as menu levels, alignment, etc., and generate new HMI variants.

W4 is preset to be equal to 2, so it is weighted higher than the other weights because it is considered the most important factor in the given example. In further embodiments described above, W4 is adapted due to reinforcement learning, for example to target specific regions or population groups.

A5 is related to many voice interactions. A5 equals 1 - A function related to the number of functions performed only by voice but also represented visually, with a functional range of 0 to 1. The range of A5 is 0 to 1 (1=100% is the percentage value). This factor A5 can be used to reduce the number of visual elements corresponding to functions performed solely by the user via voice.

Default is W5=1. In further embodiments described above, W5 is adapted due to reinforcement learning, for example to target specific regions or population groups.

This metric is associated with a user's dedicated profile and is analyzed only at a group level within the cloud application, producing specific weights (W1 to W6) for different regions or demographic groups, for example.

The sum of all weights can be equal to a fixed number, in the given example above, the sum of all weights is 6. If these weights are adapted according to the method described above as a further embodiment, they are adapted interdependently so that the sum of all weights remains the same.

The selection area 40 displayed on the central display 8 includes a slider 60 that can move on a track 62 having a plurality of stop positions 64. for multiple different configurations sorted from left to right as the clutter index increases. The driver 14 can slide the slider 60 to any stop position 64 with the associated cockpit display configuration 42 to 46 displayed above the selection area 40 .

The arrangement 42 as shown in Figure 2a has a very low clutter index because all attention is drawn to a display of a speed of the car 2 in the central area 52, the left area 50, The lines in the right region 54 and the lower region 56 are relatively thin.

The configuration 44 as shown in Figure 2b has a high but still relatively low clutter index because attention is still drawn to a display of a speed of the car 2 in the central area 52, the left area 50 , the lines in the right region 54 and the lower region 56 are relatively thin. However, the information density is increased compared to the configuration 42 due to the additional information displayed in the central area 52 and the lower area 56 .

This configuration 46 as shown in Figure 2c has a relatively high clutter index and is more suitable for experienced drivers. The display of the car's speed is moved to the left area 50 and a charging capacity is visible in the right area 54 . The central area 52 is occupied by a menu with a plurality of list lines. In addition, a lot of additional information of small font or icon size is also displayed in all areas 50, 52, 54, and 56.

After selection, the driver 14 confirms his selection and the corresponding layout will be used for the cockpit display 6 .

Figure 3 shows one method of selecting this display configuration.

A machine learning algorithm is fed large amounts of information. The first source of information is usage statistics derived from the driver's interaction with the human-machine interface 4 .

The second source of information is attention data derived from the driver monitoring system 26 .

The third source of information is data derived from a speech recognition system using the microphone 10 .

The fourth source of information is the configuration of the driver's mobile phone 34.

The machine learning algorithm processes different information sources and determines a tolerable clutter index for an individual driver 14 . The machine learning algorithm selects and/or modifies existing configurations to suit the driver A tolerable range of clutter indicators 14 and a selection of information to be displayed related to the driver 14 .

The components and systems described above may be independent or used by other systems of the vehicle. For example, sensor data such as camera data can be provided to different systems and utilized for different purposes. Systems can be implemented as functions in control units with more functions, such as a function of a driver assistance system with multiple components, such as lane keeping and adaptive cruise control.

Although at least one exemplary embodiment has been presented in the foregoing summary and detailed description and claims, it is to be understood that numerous variations exist. It should also be understood that the exemplary embodiment or exemplary embodiments are examples only, and are not intended to limit the scope, applicability, or configuration in any way. Rather, the foregoing summary and detailed description will provide those skilled in the art with a convenient road map for implementing at least one exemplary embodiment, it being understood that various modifications may be made to the function and arrangement of elements described in the exemplary embodiment. changes without departing from the scope set forth in the attached claims and their legal equivalents.

Any feature disclosed in the claim, description and drawings, including construction details, relative positioning or method steps, may be relevant to the invention taken in isolation or in any meaningful combination with any other features.

2:Car

4: Human-machine interface

6: Cockpit display

8: Central display

10:Microphone

12: Speaker layout

14: Driver

16:Controller

18: Processor

20: Non-transitory computer-readable storage media

22:Computer program products

26: Driver monitoring system

28:Eye-tracking camera

30:eyes

32: Observation direction

34:Mobile phone

Claims (12)

A computer implementation method for adapting a graphical user interface (6, 8) of a human-machine interface (4) of a vehicle (2), wherein the graphical user interface (6, 8) is composed of the human-machine interface (4 ) controlled by a controller (16), wherein the layout of information presented on the graphical user interface (6, 8) is determined by a configuration (42, 44, 46), wherein the controller (16) accesses At least two different configurations (42, 44, 46), wherein each configuration (42, 44, 46) is associated with at least one clutter indicator or range of clutter indicators, wherein the controller (16) assigns a user profile , at least one of a region, a usage information and/or a user alertness is associated with a clutter indicator or a clutter indicator range, and selecting a configuration with a compatible clutter indicator or clutter indicator range (42, 44, 46), wherein the controller (16) analyzes the usage of several functions accessible by a user (14) through the human-machine interface (4), wherein the controller (16) based on the usage Analyze and adjust at least one configuration (42, 44, 46). The computer-implemented method of claim 1, wherein the selected configuration (42, 44, 46) is presented to the user (14), and if the user (14) confirms the selected configuration (42 , 44, 46), the configuration (42, 44, 46) is changed to the selected configuration (42, 44, 46). The computer implementation method of claim 1 or 2, wherein the clutter indicator is derived from an information density displayed on the graphical user interface (6, 8) at a given time. The computer-implemented method of claim 1 or 2, wherein a driver monitoring system (26) is connected to the controller (16), and wherein a driver information captured by the driver monitoring system (26) is obtained by the The controller (16) processes, wherein the controller (16) selects a configuration (42, 44, 46) based on the driver information. The computer implementation method as claimed in claim 1 or 2, wherein the human-machine interface (4) includes a voice recognition system (10), in which information related to the usage of several voice commands is analyzed by the controller (16) , wherein the controller (16) selects a configuration (42, 44, 46) based on usage of the voice commands. The computer-implemented method of claim 1 or 2, wherein a selection of a configuration (42, 44, 46) is presented at a predetermined time interval. The computer implementation method according to claim 1 or 2, wherein a mobile device (34) is connected to the human-machine interface (4), wherein the controller (16) analyzes the configuration of the mobile device (34), wherein the The controller (16) selects a configuration based on the configuration of the mobile device (34). The computer-implemented method of claim 1 or 2, wherein the controller (16) applies a machine learning algorithm to the selection of the configuration (42, 44, 46). The computer implementation method of claim 1 or 2, wherein a user preference profile is stored and accessed by the controller (16). A computer program product is provided with a non-transitory computer-readable storage medium (20). The non-transitory computer-readable storage medium (20) has a plurality of commands embedded therein. When the plurality of commands are processed by a processor (18) ) is executed, causing the processor (18) to execute the method according to any one of claims 1 to 9. A human-machine interface (4) for a vehicle provided with a controller (16), the controller including a computer program product provided with a non-transitory computer-readable storage medium (20) according to claim 10 and a processor(18). A vehicle provided with the human-machine interface (4) according to claim 11.

Images