DE102023001622A1 - Methods for optimizing vehicle functions - Google Patents

Abstract

The invention relates to a method (1) for optimizing vehicle functions, wherein at least one vehicle (2) of a vehicle fleet (3) transmits state values (Zi) of at least one vehicle system to a central data center (5) at a time (t0).
The invention is characterized in that in the central data center (5) control commands (Si) are determined for the state values (Zi) at the time (t0), which map a control target to be tested with a functional mapping to be optimized, wherein the control commands (Si) are sent to the vehicle fleet (3), wherein the vehicles (2) of the vehicle fleet (3) transmit state values to the data center (5) at a time (t1) with t1>t0 in order to improve the functional mapping.

Description

The invention relates to a method for optimizing vehicle functions in vehicle systems which can be controlled directly or indirectly by a user.

Various vehicle functions can be implemented using at least one control unit. The control unit itself is constantly being further developed during vehicle development. This means that different sensors and control units can be used for different vehicle functions in order to set the desired parameters.

The vehicle function can be, for example, the automatic climate control in a vehicle. However, this is just one example, and other vehicle functions can also be controlled in this way. In the case of automatic climate control, for example, with indirect control, when a new target temperature is to be reached, various sensors and control units in the vehicle can be used for control. The sensors can, for example, measure a temperature distribution in the vehicle and, according to a control or control logic that was worked out during the development of the vehicle, elements of the automatic climate control, which in particular include flaps, air nozzles, holding elements and cooling elements, can be controlled.

The control system is designed during the development of the control units based on the individual experience and skills of development engineers. This is based on manual analyses from a test operation, from simulations of the test system and/or from certain parts of the test system. Control logic from an earlier development can also be used.

From the EP 3 307 572 A1 A control unit for a climate control system for a vehicle is known. This comprises a monitoring module configured to monitor at least one request and/or at least one active user input relating to an environmental parameter within the vehicle. Furthermore, the control unit comprises a storage means storing a control policy for configuring at least one climate control element, a feedback module configured to receive the at least one request from the monitoring module and to receive a simulated control policy from a simulation module. An on-policy-reinforcement-learning approach (SARSA) is used, which is used to train an initial control strategy offline in a simulation of an interior climate system.

From the WO 2016/198 307 A1 A control system for air conditioning in a vehicle is known, in which a control is adapted based on a simulated control. The control in a vehicle is considered.

The disadvantage is that only known scenarios and processes can be included in the control system, which limits the vehicle functions. In particular, the disadvantage is that only the range of vehicle functions known at the time of development can be included, with future technologies or further developments and trends not being taken into account. Furthermore, a natural variation in the properties of the system in reality cannot usually be adequately represented in the simulation.

The object of the present invention is to provide a method for optimizing vehicle functions which overcomes the aforementioned disadvantages.

According to the invention, this object is achieved by a method having the features in claim 1, and here in particular in the characterizing part of claim 1. Advantageous embodiments and further developments emerge from the dependent claims.

At the core of the method according to the invention, control commands are determined in the central data center for the state values at time t0, which map a control target to be tested with a functional mapping to be optimized, whereby the control commands are sent to the vehicle fleet, whereby the vehicles of the vehicle fleet transmit state values to the data center at a time t1 with t1>t0 in order to improve the functional mapping. The proposed method or approach makes it possible to make data available to a vehicle fleet, whereby complex vehicle systems can be optimized or their functions expanded. Data or test data with natural and specific variations can be used to expand the complexity of the different vehicle functions. In contrast to known methods, consisting of, for example, a simulated subsystem (software-in-the-loop (SiL)) or individual test systems, which only inadequately map the natural and specific variants of the system behavior, according to the invention the distribution, requirements and loads that occur during use of the vehicle fleet and to which the components are or were exposed can be mapped.

The method is designed to optimize vehicle functions, with at least one vehicle in a vehicle fleet transmitting status values of at least one vehicle system to a central data center at a time t0. A wireless data communication device is preferably used to transmit the data.

A first determination rule (control policy) of the vehicle functions or the functional mapping for the vehicle functions can be created during the development of the vehicle and delivered with the new vehicles. Using the method presented, a so-called behavior policy can be derived from the control policy. In particular, for example, an initial determination of the vehicle functions can be used as a basis, whereby the further system behavior is examined by selecting slightly deviating and random actions based on the first determination.

Preferably, the function mapping can be a control of an automatic climate control system. The vehicle function can be, for example, a lowering and/or raising of an interior temperature by a relative value. Furthermore, a distance measure from a current state, such as the current interior temperature at the driver's seat, to a target value can be defined.

According to a very advantageous development of the idea, it can be provided that additional reward values are determined, which result from a comparison of the control target with actual state changes of the state value from time t0 to time t1. These reward values can represent a criterion with which the control can be evaluated. For example, reaching a target state as quickly as possible from any initial state or from a defined initial state can be evaluated. For example, a negative reward (penalty) can be given which is proportional to the time required to reach the desired goal. This can be combined with a constant positive reward when the goal is reached. In particular, the total reward of the process is only higher if the goal is reached in a particularly short time. Another example of a negative reward (penalty) can be described by the fact that a comfort criterion is violated. This is the case, for example, if a relatively cold air flow results in the area of the driver's face, foot or hand. However, since comfort is assessed subjectively, objective criteria must be defined for this. For example, a relatively cold air flow can be defined as a function of the temperature difference between the average interior temperature and the local air flow as well as an air speed at a location. The penalty is therefore preferably proportional to the value of the comfort-describing function or is set as constant. If the comfort-describing function is below a threshold value, for example, the penalty can be set as constant. Furthermore, a reward can be assumed from a reliability value, a wear value or a combination of these two values. A reliability value is characterized, for example, by how robust the process is in response to small state variations. Wear can be considered to be how much stress the individual components of the system are subjected to.

According to an advantageous embodiment, it can be provided that the functional mapping is optimized on the basis of the state changes and the associated reward values.

A further advantageous embodiment can provide that the improved functional mapping is sent to at least a subset of the vehicles in the vehicle fleet, with a software update of at least the subset of vehicles taking place. This is preferably done by means of the wireless data communication device from a central data center.

It can be provided that state values of at least one vehicle function are transmitted to the central data center at time t0 from all or a subset of vehicles in the vehicle fleet. From the central data center, control commands Si can be sent at a time t0 from one vehicle system, or from several vehicle systems, to all vehicles or to a subset of the vehicles in a vehicle fleet using the wireless data communication connection. The vehicles can then transmit state values Zi at a time t1 that were reached based on the control commands Si at time t0. For example, time t1 can be limited by a speed of the data communication method and/or adapted to the system to be controlled. For example, a time difference for systems whose system states change slowly can be selected to be greater than the maximum data transmission speed would allow.

In an advantageous further development, the subset of vehicles of a vehicle fleet to which control commands Si are sent at a time t0 sent are determined by the state or states of the vehicles in the subset at time t0. In particular, a subset of the vehicles in the vehicle fleet can be determined by the states of the vehicles in the subset being the same or at least similar at time t0.

According to a very advantageous development of the idea, it can be provided that a subset of vehicles of the vehicle fleet is used to transmit status values of at least one vehicle function at the time t0 to the central data center.

According to a very advantageous development of the technical idea, vehicles that have a specific type of older air conditioning system can be examined. For example, a time frame of 5 years can be defined as the minimum or maximum age of the air conditioning system. From all the vehicles in the fleet, those vehicles can be selected that have this type of air conditioning system and whose air conditioning is currently active.

The current status of the air conditioning systems of these participants can be sent to the central data center. For each individual status, an action can be selected in the central data center and sent out. Finally, a functional mapping can be optimized based on a status of the vehicle reported back at a later point in time. In particular, a special functional mapping can be created and optimized for vehicles that have a specific type of older air conditioning system.

In a further advantageous embodiment, an individual control of older air conditioning systems, which have been installed in a vehicle for 10 years, for example, can be improved in such a way that defects in subcomponents, wear and tear, and the associated reduced performance are taken into account. In such an embodiment, the data is not collected centrally from different vehicles as described above, but an individual control model is learned in the vehicle itself or in an external data center based on the data combinations of state-control-reward-future state prevailing in the vehicle.

In a further embodiment, an individual control of current air conditioning systems can be improved in that manufacturing tolerances and thus variations of subcomponents of the air conditioning system are learned through an individual control model. The individual control model can be built in the vehicle itself or in an external data center based on the data combinations of state-control-reward-future state prevailing in the vehicle. Overall, an advantageous automatic air conditioning control can be realized in this way.

According to an advantageous embodiment, it can be provided that the state values include system settings, system parameters and sensor values for the sensors required for functional mapping, and/or the control target(s) include system settings and system parameters which contain target data selected by a vehicle user in a user interface or in the central data center by an algorithm or development engineer. For example, the transmitted state values for the vehicle system of the interior air conditioning can contain a variety of sensor values about the temperatures inside and outside the vehicle, about light irradiation intensity, about temperature distribution in the vehicle, air humidity values and the like. On the other hand, the system settings or the system parameters can be the target states of the air conditioning desired by the user and selected in the user interface (UI), such as a temperature of 20 °C on the left seat and 19 °C on the right seat. In particular, the control target can not be set by the actor/agent, but by a user of the vehicle or for testing in the data center by an algorithm or development engineer.

A further advantageous embodiment can provide that state values of vehicles randomly selected from the vehicle fleet are set as starting points at randomly selected times t0. This allows a realistic distribution of the states of the system in the fleet to be taken into account.

According to an advantageous embodiment, it can be provided that a sequence of state values is defined as a single state value. In particular, this makes it possible to consider sequences over a period of time t without violating Markov properties of the state definition of the system.

According to a very advantageous development of the idea, it can be provided that reinforcement learning algorithms are used to learn the function mapping. In this case, it is advantageous to use of rules or control functions and/or function mappings, in addition to methods of classical reinforcement learning with discrete, discretized continuous or approximated continuous state and action spaces, methods of deep reinforcement learning can also be used. This means that the process can be trained, for example, with continuous action values and state values in one or more deep neural networks. For example, continuous state variables can serve as input to a deep neural network that approximates a specific action, for example one that maximizes the expected future reward. For example, DeepQ networks or combinations of action and critic networks can be used.

Further advantageous embodiments of the method according to the invention also emerge from the embodiment, which is presented in more detail below with reference to the figures.

• 1 eine mögliche Ausführungsform des Verfahrens;
• 2 eine weitere mögliche Ausführungsform des Verfahrens;
• 3 eine weitere mögliche Ausführungsform des Verfahrens; und
• 4 eine weitere mögliche Ausführungsform des Verfahrens.

The following show:

• 1 a possible embodiment of the method;
• 2 another possible embodiment of the method;
• 3 another possible embodiment of the method; and
• 4 another possible embodiment of the method.

The method 1 is designed to optimize vehicle functions, wherein at least one vehicle 2 of a vehicle fleet 3 transmits state values Zi of at least one vehicle system to a central data center 5 at a time t0. This is shown schematically in the following figures.

In the central data center 5, control commands Si are determined for the state values Zi at the time t0, which map a control target to be tested with a functional mapping to be optimized, wherein the control commands Si are sent to the vehicle fleet 3. The vehicles 2 of the vehicle fleet 3 transmit state values Zi at a time t1 with t1>t0 to the data center 5 in order to improve the functional mapping. In particular, a user 6, who is an actor and in particular is located in the central data center, can execute the control commands Si of the state values Zi at the time t0 on the basis of the behavior policy.

In the presentation of the 1 a possible embodiment of the method 1 is shown in a schematic representation. An actor, agent or so-called agent can issue control inputs in the form of control commands Si to a system, whereby the system can be at least one vehicle 2 of a vehicle fleet 3. The control inputs can be determined by a set of rules or a decision function on the basis of current system states known to the system. After the system has executed the control inputs, new system states are communicated to the user 6 by state values Zi. In particular, a reward value 4 can be calculated from the current system states. For example, a reward can be given when a target system state is reached.

A further embodiment of the device 1 is shown in 2 The same elements are provided with the same reference symbols, so that this does not need to be discussed in detail. In contrast to 1 Here, an actor gives control commands Si to many instances of the same system with the same or different states Zi,t0 at time t0, where the time is marked with the index t and the system instance with the index i. Each system instance i responds to the control input Si,t0 at time t0 with a new state value Zi,t1 at a later time t1. A reward can be calculated from the state value Zi,t1.

In 3 The system instances can be different vehicles 2 of a vehicle fleet 3. The communication between the acting user 6 and vehicle 2 takes place via a wireless communication device.

In 3 For example, a control objective to be tested can be the lowering of an average room temperature to 19 °C +/-0.1 °C, whereby the current state values can be the measured temperatures of the vehicles 2 of a vehicle fleet 3 or a subset of the vehicle fleet 3. Elements in the vehicle 2 which then carry out an action can be, for example, flaps, air nozzles, heating elements or cooling elements of an air conditioning system.

In 4 It is shown that it is possible to define different users 6, each of which performs different or identical inputs and communicates only with a subset of the vehicles 2 of the vehicle fleet 3. If off-policy reinforcement algorithms are used, individual sequences of state-action-reward-state data pairs of any length can be stored in so-called called replay buffers, a database 7 of the pairs, are stored and used to train the policy, ie the determination rule.

It is possible, as in 5 shown to restrict the exploration to subgroups of the vehicle fleet 3, so that the exploration only affects a few vehicle users, is not perceived by the vehicle user, ie is below the perception threshold, or is not perceived as disturbing. Advantageously, no users are in the vehicle. Likewise, system states of the subgroups can be in or close to a specific state, so that properties of the vehicle 2 or usage conditions of the vehicles 2 are similar.

If there are no users 6 in the vehicle 2, this can be detected by an interior camera, pressure sensors or the like. For example, when preheating the vehicle 2 with the automatic air conditioning in a garage, the vehicle system can be tested, which does not depend on the state of the vehicle 2 itself. In a further embodiment, testing can be restricted to subgroups of the vehicle fleet 3 whose system states are in a specific state. In this way, the system control can be examined and optimized specifically from this state. For example, subgroups can also be found which are characterized by properties of the vehicles or conditions of use of the vehicles. For example, an age-dependent or user-dependent change in the control of vehicle systems can be detected and implemented. This particularly applies to vehicles with specially designed air conditioning systems which are, for example, 5 years old or 10 years old. In this way, wear and tear that has already set in in older air conditioning systems and the associated reduced performance of the subcomponents can be taken into account. In newer models, manufacturing tolerances can be included.

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• EP 3307572 A1 [0005]
• WO 2016198307 A1 [0006]

Claims (10)

Method (1) for optimizing vehicle functions, wherein at least one vehicle (2) of a vehicle fleet (3) transmits state values (Zi) of at least one vehicle system at a time (t0) to a central data center (5), characterized in that in the central data center (5) control commands (Si) are determined for the state values (Zi) at the time (t0), which map a control target to be tested with a function mapping to be optimized, wherein the control commands (Si) are sent to the vehicle fleet (3), wherein the vehicles (2) of the vehicle fleet (3) transmit state values to the data center (5) at a time (t1) with t1>t0 in order to improve the function mapping. Procedure (1) according to claim 1 , characterized in that the functional mapping is a control of an automatic climate control system. Procedure (1) according to claim 1 or 2 , characterized in that additional reward values (4) are determined which result from a comparison of the control target with actual state changes of the state value (Zi) from the time (t0) to the time (t1). Procedure (1) according to claim 3 , characterized in that the functional mapping is optimized on the basis of the state changes and the associated reward values (4). Method (1) according to one of the Claims 1 until 4 , characterized in that the improved functional mapping is sent to at least a subset of the vehicles (2) of the vehicle fleet (3), wherein a software update of at least the subset of the vehicles (2) takes place. Method (1) according to one of the Claims 1 until 5 , characterized in that a subset of vehicles (2) of the vehicle fleet (3) is used to transmit state values (Zi) of at least one vehicle function at the time (t0) to the central data center (5). Method (1) according to one of the Claims 1 until 6 , characterized in that the state values (Zi) comprise system settings, system parameters and sensor values for the sensors required for functional mapping, and/or the control targets comprise system settings and system parameters which contain target data selected by a user of the vehicle in a user interface. Method (1) according to one of the Claims 1 until 7 , characterized in that state values (Zi) of vehicles (2) randomly selected from the vehicle fleet (3) are set as starting points at randomly selected times (t0). Method (1) according to one of the Claims 1 until 8 , characterized in that a sequence of state values is defined as a single state value (Zi). Method (1) according to one of the Claims 1 until 9 , characterized in that reinforcement learning algorithms are used to learn the function mapping.

Images