DE102019126246A1 - System and method for calibrating a control and regulating device for gas exchange valves of an internal combustion engine - Google Patents

Abstract

The invention relates to a system (100) for finding optimized actions ((A +), (A0), (A-)) for calibrating a control and regulating device (10) for controlling and / or regulating at least one controlled variable (Ri) of gas exchange valves of an internal combustion engine; comprising a learning reinforcement agent (200) which is designed to select at least one action ((A +), (A0), (A-)) for at least one parameter (Pi) for processing data from a data memory (320) for the adaptation of values (70) of the at least one parameter (Pi), and wherein an environment module (400) is provided which is designed to provide a process value (50) for the controlled variable (Ri) on the basis of measurement results, and wherein a status module ( 300) is provided, which is designed to calculate a target value (30) for the controlled variable (Ri) by means of the adjusted values (70) of the at least one parameter (Pi) and to compare the provided process value (50) with the calculated target value (30 ); and wherein the status module (300) is designed to forward the comparison result to a reward module (500), which is designed to determine a reward for the comparison result and to forward this reward to the learning reinforcement agent (200), which is designed based on it Reward for the comparison result to select one or more actions ((A +), (A0), (A-)) again.

Description

The invention relates to a system and method for calibrating a control and regulating device for gas exchange valves of an internal combustion engine.

A valve control or valve drive is used to control the gas exchange valves in a reciprocating engine in order to control the periodic change of the fuel-air mixture in the cylinders of the reciprocating engine by opening and closing the air inlet and exhaust gas outlet ducts of the valves. The term gas exchange describes the periodic change of the cylinder charge, i.e. the emission of exhaust gas and the inflow of the fuel-air mixture. Usually a valve is opened by a camshaft via a tappet, rocker arm or rocker arm. The few exceptions are pneumatic, hydraulic or solenoid valves. Valves are closed by helical springs, rarely by torsion bar springs, gas springs or positively controlled by a closing cam. The camshaft is driven by the crankshaft, in four-stroke engines usually with a ratio of 2: 1, i.e. the camshaft has half the speed of the crankshaft. When the internal combustion engine is in operation, the valves are opened and closed several times every second, i.e. accelerated and brought to a standstill again. The movements of the valves and the piston in the cylinder must be precisely coordinated. The opening function as the time sequence of the valve opening, i.e. the beginning, duration and extent of the opening cross-section are determined in conventional internal combustion engines by the shape of the camshaft.

Furthermore, variable controls are known which control the opening and closing times for the valves and the valve position in a changeable manner during operation. Control is typically based on a proportional-integral-derivative (PID) controller, which is a closed-loop feedback mechanism used in industrial control systems and a variety of other applications that require continuously modulated control.

A PID controller continuously calculates an error value as the difference between a desired setpoint and a measured process variable and carries out a correction on the basis of determined proportional, integral and differentiated quantities. However, H∞ controls or fuzzy controllers are also used, in which the control task is formulated as an optimization problem, which requires a relatively high mathematical effort. The control determines the deviation between a target valve opening time and a target valve closing time and the actual valve opening and valve closing time and corrects it accordingly. In a control unit (ECU = electronic control unit or ECM = electronic control module) of the motor vehicle, a table value or database entry for various parameters and their mutual dependencies and interactions, such as the number of revolutions (rpm ), the oil temperature and the cylinder charge. These table values and database entries are determined and calibrated by an expert such as an engineer and represent the basis for the regulation or control of the valve control for the inlet and outlet valves is based on fixed values.

The DE 10 113538 B4 describes a control device. A controller receives an input variable and uses it to determine a manipulated variable which is fed to an actuator of a controlled system. A neural correction transmitter using a neural network is provided which, on the basis of the input variable, generates a correction signal which is algebraically linked with the manipulated variable to form a corrected manipulated variable. In this way, a multidimensional, non-linear, unknown functional relationship between input variables and additive correction of the output variables can be mapped.

The EP 1097295 B1 describes a variable valve control of gas exchange valves of an internal combustion engine based on a signal that indicates the amount of air sucked in by the engine, detects the valve position and opens the initially completely closed inlet valves in overrun mode, with the valve position at which a reaction of the signal about the air volume occurs as a reference point is used to determine the actual valve position in order to enable automatic calibration of the valve lift sensor or angle encoder in the valve actuation mechanism.

The EP 0966600 B1 describes a method for mixture control in an internal combustion engine. The mixing ratio that actually occurs is compared with a target value and, if there is a deviation from this, stored control information is adapted so that a smaller deviation is achieved when the same operating point is run through in the future, using learning methods by means of neural networks for this purpose.

The object on which the invention is based is now to provide a method and a To provide a system for automatically calibrating a control and regulating device for the opening and closing of intake and exhaust valves in an internal combustion engine, which system is characterized by high reliability and accuracy and is easy to implement.

According to the present invention, a method is proposed by means of which automatic calibration of a target value of a controlled variable is made possible in order to create the basis for a reliable and precise determination of an actuating signal for correcting a process value of this controlled variable by a control and regulation device.

According to the invention, this object is achieved with regard to a system by the features of claim 1 and with regard to a method by the features of claim 8. The further claims relate to preferred embodiments of the invention.

According to a first aspect, the invention relates to a system for finding optimized actions ( A + ), ( A0 ), ( A- ) for the calibration of a control and regulating device for the control and / or regulation of at least one control variable R _i of gas exchange valves of an internal combustion engine. The system comprises a learning reinforcement agent, which is designed to carry out at least one action ( A + ), ( A0 ), ( A- ) for at least one parameter P _i for processing data from a data memory for the adaptation of values of the at least one parameter P _i . There is also an environmental module 400 provided, which is designed to provide a process value for the controlled variable R _i on the basis of measurement results. In addition, a status module is provided which is designed to calculate a target value for the controlled variable R _i by means of the adjusted values of the at least one parameter P _i and for comparing the provided process value with the calculated target value. The status module is designed to forward the comparison result to a reward module, which is designed to determine a reward for the comparison result and to forward this reward to the learning reinforcement agent, which is designed based on this reward for the comparison result again to perform one or more actions ( A + ), ( A0 ), ( A- ).

In an advantageous further development, sensors and / or measuring devices are provided for recording measured values from the gas exchange valves.

The reward module advantageously has a database or function or matrix for evaluating the actions ( A + ), ( A0 ), ( A- ) on.

In a further embodiment, the learning reinforcement agent is designed to use an algorithm from reinforcement learning.

In a further development of the invention, the algorithm is designed as a Markov decision process or as temporal difference learning (TD learning) or as Q-learning or as SARSA or as DYNAQ or as POMDP or as Monte Carlo simulation.

Advantageously, the values are the parameters P _i stored on a control unit of the control and regulating device.

In one embodiment, at least one parameter P _i represent the cylinder load, the number of engine revolutions and / or the oil temperature.

According to a second aspect, the invention relates to a method for finding optimized actions ( A + ), ( A0 ), ( A- ) for the calibration of a control and regulating device for the control and / or regulation of at least one control variable R _i of gas exchange valves of an internal combustion engine. The method includes selecting at least one action ( A + ), ( A0 ), ( A- ) for at least one parameter P _i by a learning reinforcement agent, the processing of data from a data memory using the selected action ( A + ), ( A0 ), ( A- ) for adapting values of the at least one parameter P _i , the provision of a process _{value for the controlled variable R i} from an environment module based on measurement results, the calculation of a target value for the controlled variable R _i in a status module by means of the adjusted values of the at least one parameter P _i , comparing the provided process value with the calculated target value in the status module, forwarding the comparison result to a reward module and determining a reward for the comparison result, forwarding the reward for the comparison result to the learning reinforcement agent and, based on this reward, again selecting one or several actions ( A + ), ( A0 ), ( A- ) from the learning reinforcement agent.

Sensors and / or measuring devices are advantageously provided for recording measured values from the gas exchange valves.

In a further development, the reward module comprises a database or function or matrix for evaluating the actions ( A + ), ( A0 ), ( A- ).

In a further embodiment, the learning reinforcement agent is designed to use an algorithm from reinforcement learning.

The algorithm is advantageously designed as a Markov decision process or as temporal difference learning (TD learning) or as Q-learning or as SARSA or as DYNAQ or as POMDP or as Monte Carlo simulation.

In another embodiment, the values are the parameters P _i stored on a control unit of the control and regulating device.

Advantageously, at least one parameter P _i represent the cylinder load, the number of engine revolutions and / or the oil temperature.

According to a third aspect, the invention provides a computer program product comprising an executable program code which is configured such that it executes the method according to the second aspect when it is executed.

The invention is explained in more detail below with reference to the exemplary embodiments shown in the drawing.

• 1 eine schematische Darstellung einer Steuer- und Regelvorrichtung;
• 2 ein Blockdiagram zur Erläuterung eines Ausführungsbeispiels eines erfindungsgemäßen Systems;
• 3 ein Flussdiagramm zur Erläuterung der einzelnen Verfahrensschritte eines erfindungsgemäßen Verfahrens;
• 4 zeigt schematisch ein Computerprogrammprodukt gemäß einer Ausführungsform des dritten Aspekts der Erfindung.

It shows:

• 1 a schematic representation of a control and regulating device;
• 2 a block diagram to explain an embodiment of a system according to the invention;
• 3 a flow chart to explain the individual method steps of a method according to the invention;
• 4th shows schematically a computer program product according to an embodiment of the third aspect of the invention.

Additional features, aspects and advantages of the invention or its exemplary embodiments will become apparent from the detailed description in conjunction with the claims.

1 shows a control and regulating device 10 for controlling the opening and closing behavior of inlet and outlet valves (gas exchange valves) of an internal combustion engine of a motor vehicle. The control and regulation device 10 includes a control unit 20th , the target values 30th for controlled variables R _i such as the opening time and the closing time of the intake and exhaust valves of the internal combustion engine. These target values 30th are preferably in the control unit 20th deposited. But they can also be stored externally in a separate memory and from the control unit 20th can be retrieved. The control unit also receives 20th currently measured values 40 for measured variables M _i such as the valve positions, the sensors (not shown here) are determined and sent to the control unit 20th will be passed on. In addition, the angular position of a crankshaft and / or camshaft (not shown here) can advantageously be detected. From the measured values 40 calculates the control unit 10 or another processor process values 50 for the controlled variables R _i such as the current time for opening and closing the valves, the duration and, if applicable, the opening cross-section of the valves, which are in the control unit 20th be deposited.

There are also in the control unit 20th Table values, database entries and / or diagrams and functions 70 for various other parameters P _i from a set of parameters { P ₁ , P ₂ , ..., P _n } and their mutual dependencies and interactions. With the parameters P _i For example, it can be the number of revolutions of the engine, the oil temperature, the volume of the cylinder charge, the gas pressure, etc. By means of a controller 80 , which can be designed as a PID controller or H∞ controller, a deviation between the process value 50 and the target value 30th determined for a controlled variable R _i . Using the stored value tables, diagrams, etc. 70 for the parameters P _i are one or more control signals 90 calculated and output to actuators for actuating the valves in order to bring the valve behavior to the target values 30th to adapt the controlled variables R _i. However, control signals can also be used 90 be passed on to sensors or angle sensors for the crankshaft and / or the camshaft in order to influence the valve behavior. Since the calculation of the control signals 90 through the controller 80 from the stored value tables, diagrams, etc. 70 for the parameters P _i depends, an adaptation and calibration of these parameter values is carried out according to the invention 70 performed.

The sensors can be, for example, pressure sensors, piezo sensors, speed sensors, temperature sensors and / or image-recording sensors such as cameras. The sensors can take the measured values 40 to a processor, which processes the data by means of a software application with an algorithm and as process values 50 provides. But it is also conceivable that the data 40 are initially stored in a memory unit or a software module and only at a later point in time by the processor and / or the control unit 20th are processed.

In connection with the invention, a “processor” and / or the “control unit” can be understood to mean, for example, a machine or an electronic circuit. A processor can in particular be a central processing unit (CPU), a microprocessor or a microcontroller, for example an application-specific integrated circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions, etc. act. A processor can also be understood to be a virtualized processor, a virtual machine or a soft CPU. It can also be a programmable processor, for example, which is equipped with configuration steps for executing the aforementioned method according to the invention or is configured with configuration steps in such a way that the programmable processor incorporates the inventive features of the method, the component, the modules, or other aspects and / or implemented partial aspects of the invention.

In connection with the invention, a “memory unit” or “memory module” and the like can mean, for example, a volatile memory in the form of random access memory (RAM) or a permanent memory such as a hard disk or a data carrier or z. B. can be understood as an exchangeable memory module. The storage module can also be a cloud-based storage solution.

In connection with the invention, a “module” can be understood to mean, for example, a processor and / or a memory unit for storing program commands. For example, the processor and / or the control unit 20th specifically designed to execute the program instructions in such a way that the processor and / or the control unit executes functions in order to implement or realize the method according to the invention or a step of the method according to the invention.

In connection with the invention, “measured values” are to be understood as meaning both the raw data and data that has already been processed from the measurement results of the sensors.

2 shows a system according to the invention 100 for calibration of the control and regulation device 10 . In a reinforcement learning agent (LV) 200, actions ( A + ), ( A0 ) and ( A- ) selected in order to adapt the stored parameter values 70 of a parameter P _i in a table of values and the like. At the action ( A + ) it is an action that has the value 70 in a table of values for a parameter P _i from the set of parameters { P ₁ , P ₂ , ..., P _n } increased when the action ( A0 ) is an action in which the value 70 of a parameter P _i remains the same in a table of values and in the action ( A- ) becomes the value 70 of the parameter P _i reduced in a table of values. In addition to typical calculation functions f _i Like addition and subtraction, mathematical functions such as mean values, minimum and maximum values, Fast Fourier Transformations, integral and differential calculations, but also extended Kalman filters, radial basis functions, data fields, convergent neural networks, artificial neural networks and feedback neural networks can be used for this purpose will.

The LV agent 200 also has access to a status module 300 that has a data store 320 with data from various sensors and data sources as well as calculation results. This can be, for example, the current deviation Δ between the measured time of valve opening and the calculated target value for the time of valve opening. The LV agent 200 uses this data and chooses an action for it ( A + ), ( A0 ) or ( A- ) to adjust the value 70 of a parameter P _i in a table of values.

Based on the changed value 70 for the parameter P _i the controller now calculates 80 at least one control signal 90 for an actuator to open the valve and gives this control signal 90 to the actuator. The due to the control signal 90 The time at which the valve was opened is determined by sensors in an environmental module 400 measured and to the status module 300 passed on. In addition, in the environmental module 400 a modeled value 55 for a controlled variable R _i such as the time of valve opening can be calculated, a neural network advantageously being used for the modeling. However, other calculation methods can also be used.

In the state module 300 the calculated value of a controlled variable R _i, such as the time at which the valve was opened, is now compared with the actually measured process value 50 compared to the controlled variable R _i. The calculated value of the controlled variable R _i is based on the changed values 70 for the selected parameters P _i through the agent 200 and sets the newly calibrated target value 30th for the controlled variable R _i . In a reward module 500 becomes the degree of deviation Δ between the calculated target value 30th and the measured process value 50 compared and assigned a reward B _i to the degree of deviation Δ. In addition, the degree of deviation Δ between the modeled value 55 and the calibrated target value 30th compared and assigned a further reward B _j to the degree of deviation Δ. Since the degree of deviation Δ from the selection of the respective action ( A + ), ( A0 ), ( A- ) is dependent, is preferably in a matrix or some other database 520 or function of the selected action ( A + ), ( A0 ), ( A- ) assigned the reward B _i . The reward B _i preferably has the values +1 and -1, with one small deviation Δ between the calculated value W ₁ and the measured value W _{2 is} rewarded with +1 and is thus reinforced, while a considerable deviation Δ is rewarded with -1 and is thus assessed negatively. However, it is also conceivable that values> 1 and values <1 are used.

A second cycle now begins to calibrate the table of values 70 for the parameters P _i with another action ( A + ), ( A0 ), ( A- ). Another calculation function or another parameter can be used here P _i to be chosen. The result is in turn sent to the status module 300 supplied and the result of the comparison in the reward module 500 rated. The LV agent 200 repeats the calibration of the table of values 70 for all parameters provided P _i until there is the greatest possible agreement between the calibrated target value 30th and the measured process value 50 for a controlled variable R _i how the time of valve opening for a valve is reached. The final state of the calibration of the table (s) of values is preferably 70 for the parameters P _i achieved when the deviation is in the range of +/- 5%. In order to avoid the destruction of the facility due to non-optimal calibration, very large negative rewards are given near the system limit.

The preferred algorithm for the LV agent 200 uses a Markov decision-making process. However, provision can also be made to use a temporal difference learning (TD learning) algorithm. An LV agent 200 with a TD learning algorithm, the adaptation of the actions ( A + ), ( A0 ), ( A- ) not only before when he receives the reward, but after each action based on an estimated expected reward. Algorithms such as DYNAQ, POMDP, Q-Learning and SARSA are also conceivable, as are Monte Carlo simulations.

According to the method and system of the present invention, reinforcement learning is used to implement an action ( A + ), ( A0 ), ( A- ) to find the one table of values 70 or a graph for parameters P _i adjusts so that a target value 30th can be automatically calibrated for a controlled variable R _i. This makes it possible to automatically calibrate a control and regulating device for the control and / or regulation of a controlled variable R _i for gas exchange valves of an internal combustion engine. Since the calibration of the control and regulation device 10 takes place automatically and at the same time during the operation of the internal combustion engine, the performance of the internal combustion engine can be increased, since the target values of the controlled variables R _i, such as the time at which the valve is opened, can be individually adapted to the respective engine. This can lead to a reduction in pollutant emissions and thus to an improved environmental balance. In addition, the fuel consumption of the internal combustion engine can be reduced. In addition, costs for a previous manual and thus time-consuming calibration can be reduced.

In 3 are the process steps for finding optimized actions for the calibration of a control and regulating device 10 shown.

In one step S10 will at least one action ( A + ), ( A0 ), ( A- ) for at least one parameter P _i from the learning reinforcement agent 200 selected.

In one step S20 become data from a data memory 320 by means of the selected action A + ), ( A0 ), ( A- ) from the LV agent 200 edited to adjust values 70 of the at least one parameter P _i .

In one step S30 becomes a process value 50 for a controlled variable R _i from an environmental module 400 provided based on measurement results.

In one step S40 becomes a target value 30th for a controlled variable R _i in the status module 300 using the adjusted values 70 of the at least one parameter P _i calculated and with the provided process value 50 compared.

In one step S50 the comparison result is sent to the reward module 500 passed on.

In one step S60 a reward for the comparison result will be in the reward module 500 determined.

In one step S70 becomes the reward for the comparison result to the learning reinforcement agent 200 passed on and one or more actions ( A + ), ( A0 ), ( A- ) are provided by the learning reinforcement agent 200 selected based on the reward for the comparison score.

4th Figure 3 schematically illustrates a computer program product 700 represents an executable program code 750 configured to carry out the method according to the second aspect of the present invention when carried out.

With the method according to the present invention, an optimized combination of actions can be found reliably for the control and regulating device 10 to calibrate reliably and automatically. Through the use of a learning reinforcement agent 200 with an algorithm of reinforcement learning, an autonomous adaptation and calibration for an individual motor vehicle is made possible autonomously and automatically.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 10113538 B4 [0005]
• EP 1097295 B1 [0006]
• EP 0966600 B1 [0007]

Claims (15)

A system (100) for finding optimized actions ((A +), (A0), (A-)) for the calibration of a control and regulating device (10) for the control and / or regulation of at least one controlled variable (R _i ) of Gas exchange valves of an internal combustion engine; comprising a learning reinforcement agent (200) which is designed to select at least one action ((A +), (A0), (A-)) for at least one parameter (P _i ) for processing data from a data memory (320) for the adaptation of values (70) of the at least one parameter (P _i ), and wherein an environment module (400) is provided which is designed to provide a process value (50) for the controlled variable (R _i ) on the basis of measurement results, and wherein a status module (300) is provided which is designed to calculate a target value (30) for the controlled variable (R _i ) using the adjusted values (70) of the at least one parameter (P _i ) and to compare the provided process value (50) with the calculated target value (30); and wherein the status module (300) is designed to forward the comparison result to a reward module (500), which is designed to determine a reward for the comparison result and to forward this reward to the learning reinforcement agent (200), which is designed based on it Reward for the comparison result to select one or more actions ((A +), (A0), (A-)) again. System (100) according to Claim 1 , sensors and / or measuring devices being provided for recording measured values from the gas exchange valves. System (100) according to Claim 1 or 2 , wherein the reward module (500) has a database (520) or function or matrix for the evaluation of the actions ((A +), (A0), (A-)). System (100) according to one or more of the preceding Claims 1 - 3 wherein the learning reinforcement agent (200) is configured to use an algorithm from the reinforcement learning. System (100) according to Claim 4 , whereby the algorithm is designed as a Markov decision-making process or as Temporal Difference Learning (TD-Learning) or as Q-Learning or as DYNAQ or as POMDP or as SARSA or as Monte Carlo simulation. System (100) according to one or more of the Claims 1 to 5 , wherein the values (70) of the parameters (P _i ) are stored on a control unit (20) of the control and regulating device (10). System (100) according to one or more of the Claims 1 to 6th , wherein at least one parameter (P _i ) represents the cylinder loading, the number of engine revolutions and / or the oil temperature. A method for finding optimized actions ((A +), (A0), (A-)) for the calibration of a control and regulating device (10) for the control and / or regulation of at least one controlled variable (R _i ) of gas exchange valves of an internal combustion engine comprising: - selecting (S10) at least one action ((A +), (A0), (A-)) for at least one parameter P _i from a learning reinforcement agent (200); - Processing of data (S20) from a data memory (320) by means of the selected action ((A +), (A0), (A-)) for adapting values (70) of the at least one parameter (P _i ); - Provision of a process value (50) (S30) for the controlled variable (R _i ) from an environmental module (400) on the basis of measurement results; - Calculating (S40) a target value (30) for the controlled variable (R _i ) in a status module (300) by means of the adjusted values (70) of the at least one parameter (P _i ); - Comparing (S50) the provided process value (50) with the calculated target value (30) in the status module (300); - Forwarding (S60) of the comparison result to a reward module (500) and determining a reward for the comparison result; - Forwarding (S70) of the reward for the comparison result to the learning reinforcement agent (200) and, based on this reward, again selecting one or more actions ((A +), (A0), (A-)) by the learning reinforcement agent ( 200). Procedure according to Claim 8 , sensors and / or measuring devices being provided for recording measured values from the gas exchange valves. Method according to one of the preceding Claims 8 or 9 , wherein the reward module (500) comprises a database (520) or function or matrix for the evaluation of the actions ((A +), (A0), (A-)). Method according to one or more of the preceding Claims 8 - 10 wherein the learning reinforcement agent (200) is configured to use an algorithm from the reinforcement learning. Procedure according to Claim 11 , whereby the algorithm is designed as a Markov decision-making process or as Temporal Difference Learning (TD-Learning) or as Q-Learning or as DYNAQ or as POMDP or as SARSA or as Monte Carlo simulation. Method according to one or more of the Claims 8 to 12th , where the values (70) of Parameters (P _i ) are stored on a control unit (20) of the control and regulating device (10). Method according to one or more of the Claims 8 to 13th , wherein at least one parameter (P _i ) represents the cylinder loading, the number of engine revolutions and / or the oil temperature. A computer program product (700), comprising an executable program code (750) which is configured in such a way that, when it is executed, it executes the method according to one of the Claims 8 to 14th executes.

Images