LU101167B1

LU101167B1 - Method and System for Predicting the Time Behavior of an Environment using a Sensing Device, a Physical Model and an Artificial Neural Network

Info

Publication number: LU101167B1
Application number: LU101167A
Authority: LU
Inventors: Hans Peter Beise; Jan Sokolowski; Da Cruz Steve Dias; Udo Schröder
Original assignee: Iee Sa
Priority date: 2019-04-01
Filing date: 2019-04-01
Publication date: 2020-10-02

Abstract

A method of predicting the time behavior of an environment or a physical system that is being monitored by a sensing device (12); the physical systembeing represented by at least one physical model (Q) executable by at least one processor unit (14), the at least one physical model (Q) including at least one time-dependent physical quantity (xs) that is to be sensed by the sensing device (12) and at least one time- independent parameter (%); values for the physical quantity (xs) and the time- independent parameter (zs) are provided as input to both the physical model (Q) and to the artificial neural network (16); operating (22) the artificial neural network (16) or the processor unit (14) for obtaining an update information, using the provided values; providing (24) the update information to the other one out of the artificial neural network (16) and the processor unit (14); and operating (26) the other one out of the artificial neural network (16) and the processor unit (14) for obtaining a prediction value for at least one status vector (ys) characterizing a status of the physical system, using the update information.

Description

| Method and System for Predicting the Time Behavior of an Environment | using a Sensing Device, a Physical Model and an Artificial Neural Network | | Technical field | [0001] The invention relates to a method of predicting the time behavior of an | environment or a physical system that is being monitored by a sensing device and | an automotive sensing system using such method for operation.

| Background of the Invention | | [0002] In the technical field of passenger transportation, and in particular in | automotive technology, it is known to employ vehicle interior occupant sensing | technologies, for instance for detection of left-behind pets and/or children, vital | sign monitoring, vehicle seat occupancy detection in support of a seat belt | reminder (SBR) system and/or activation control for an auxiliary restraint system (ARS) system, or for anti-theft alarm. Further, valuable information that is usable as an important input for Advanced Driver Assistance Systems (ADAS) could be provided by monitoring a vital sign of a detected person.

[0003] An output signal of a seat occupant detection and/or classification system is usually transferred to an electronic control unit of the vehicle to serve, for instance, as a means of assessing a potential activation of an installed vehicle | passenger restraint system, such as an airbag.

[0004] Further valuable information, usable as important input for Advanced | Driver Assistance Systems (ADAS) could be provided by monitoring a vital sign of the detected person, which has been proposed in the art.

[0005] For instance, it has been proposed in the art to use electromagnetic waves in the optical regime or radar technology, for instance for the purpose of vehicle seat occupancy detection.

j [0006] By way of example, WO 2013/037399 A1 describes a system and method | for detecting a vital-related signal pattern of a seated person in a vehicle seat. The seat comprises a substantially horizontal base and a substantially vertical backrest having a front surface accommodating the back of the seated person when in use, and a rear surface. The system further comprises at least one Doppler radar | arranged behind the front surface of the backrest, in such a way that a mainradiation lobe of the emitter/receiver of the Doppler radar is focused towards the front surface of the backrest, by which a movement created by a vital sign of the | person can be detected and a vital-related signal can be obtained. Moreover, the | system comprises a module for detecting, based on a radar signal obtained by the | Doppler radar, a vital-related signal pattern of the seated person. | [0007] Systems based on sensor input experience fast-growing demands in | various fields. Especially in the automotive field they constitute the backbone of | almost all Advanced Driver-Assistance Systems (ADAS) as these monitor an | exterior environment or the interior of a vehicle and its occupants for providing | improved safety by facilitating an optimized reaction of a driver of a vehicle with | appropriate warnings or even by automatically taking over control of the vehicle, | for instance in collision avoidance systems. | [0008] In the described function, such systems are requested to perform tasks of increasing complexity. For example, they should be capable to anticipate potential risks that might occur in complex traffic scenarios within the next few seconds. In conventional ADAS, usually an electronic processing unit such as a central processing unit (CPU) is employed for executing a program code of a software module that has been manually designed for controlling an automatic execution of | a monitoring method.

[0009] US 2014/0139670 A1 describes a system and method directed to | augmenting advanced driver assistance systems (ADAS) features of a vehicle with image processing support in an on-board vehicle platform. Images may be received from one or more image sensors associated with an ADAS of a vehicle. The received images may be processed. An action is determined based upon, at least in part, the processed images. A message is transmitted to an ADAS controller responsive to the determination. To that end, the vehicle may include one or more processor units, networking interfaces, and other computing devices that may enable it to capture image data, process the image data, and augment | ADAS features of the vehicle with image processing support in the on-board vehicle platform. A computing system may include single-feature fixed-function devices such as an ADAS image system on chip (SoC).

[0010] The complexity of tasks to be performed by such ADAS tends to grow more and more as well as in other technical fields such as, for instance, medical diagnostic appliances, smartphone technology and drone technology.

[0011] In such complex sensor-based systems it has been proposed to exploit the capabilities of artificial intelligence (Al) systems and artificial neural networks, respectively. In contrast to conventional processing units, artificial neural networks provide the possibility of learning. | [0012] Artificial neural networks are known to comprise a plurality of | interconnected artificial neurons and to have an input side and an output side. As is well known in the field of artificial neural networks, each artificial neurons of the | plurality of interconnected artificial neurons (also called nodes) can transmit a | signal to another artificial neuron connected to it, and the received signal can further be processed and transmitted to the next artificial neuron. The output of each artificial neuron may be calculated using a non-linear function of the sum of its inputs. In a learning process of an artificial neural network, weights of the non- | linear function usually are being adjusted. A complex task may be learned by determining a set of weights for the artificial neurons such that the output signal of the artificial neural network is close to a desired output signal, which is performed when the artificial neural network is trained.

[0013] Multiple methods for training an artificial neural network are known in the | art. For instance, in supervised learning a function is learned that maps an input to an output based on exemplary input-output pairs. An artificial neural network that has been submitted to a learning scheme is often called a “trained” artificial neural network. Reliability and performance of Al systems including an artificial neural | network improve with quantity and quality of training data.

[0014] Besides employing artificial neural networks as a "black box”; i.e. without | using a priori knowledge about the nature of the input provided to the artificial | neural networks, it has been proposed to develop a modeling strategy called | “hybrid modeling approach” that combines first principles knowledge, in the form of | equations such as mass and energy balances, with neural networks as nonparametric estimators of important process parameters. This is described for biological reactor processes in the article “A Hybrid Neural Network-First Principles Approach to Process Modeling” by Psichogios, Dimitris C. and Ungar, Lyle H.

| P-IEE-527/LU 4 LU101167 (AIChE Journal October 1992, Vol. 38, No. 10, p. 1499-1511). The article “A Hybrid Neural Network Approach for Batch Fermentation Simulation” by Emmanuel, Assidjo N. et al., Australian Journal of Basic and Applied Sciences, 3(4): 3930-3936, 2009, describes a hybrid modeling approach for simulating a | batch fermentation process. | [0015] Further, US 5,461,699 A describes a system and method for forecasting that combines a neural network with a statistical forecast. The neural network has | an input layer, a hidden layer, and an output layer with each layer having one or | more nodes. Each node in the input layer is connected to each node in the hidden | layer and each node in the hidden layer is connected to each node in the output | layer. Each connection between nodes has an associated weight. One node in the input layer is connected to a statistical forecast that is produced by a statistical | model. All other nodes in the input layer are connected to a different historical | datum from the set of historical data. The neural network is operative by outputting | a forecast, the output of the output layer nodes, when presented with input data. The weights associated with the connections of the neural network are first adjusted by a training device. The training device applies a plurality of training sets to the neural network, each training set consisting of historical data, an associated statistical output and a desired forecast. With each set of training data the training device determines a difference between the forecast produced by the neural network given the training data and the desired forecast. The training device then | adjusts the weights of the neural network based on the difference. Object of the invention

[0016] In many sensing systems, i.e. sensor systems whose operation is based on at least one sensor, one or more physical systems have impact on the sensed quantities. This impact might manifest itself as a disturbing signal (noise) or it | might encode additional information. In both cases, it would be valuable to have a prediction model to either compensate for any disturbing effects or to decode any | additional information. In many cases the underlying physical processes are understood in their principles, but the actual system at hand is rather complex as it may depend on unknown material properties, complex geometries, environmental | conditions such as temperature and so forth, all of which are unknown in detail.

| P-IEE-527/LU 5 LU101167

[0017] It is therefore an object of the invention to provide a method of predicting the time behavior of a physical model representing an environment that is being monitored by a sensing system, and of using the predicted behavior for correcting | an output of the sensing system and/or for decoding additional information from | the output of the sensing system. It is another object of the invention to provide a | sensing system using such method for operation. | General Description of the Invention | [0018] In one aspect of the present invention, the object is achieved by a method | of predicting the time behavior of an environment or a physical system that is | being monitored by a sensing device. The physical system or at least certain | aspects thereof is/are represented by at least one physical model. The at least one | physical model is executable by at least one processor unit. The at least one physical model includes at least one time-dependent physical quantity that is to be sensed by the sensing device, and at least one time-independent parameter, wherein the at least one physical quantity and the at least one parameter characterize specific features of the physical system or environment.

[0019] The method comprises the following steps, which are to be executed repeatedly: | - providing a former value for the at least one time-dependent physical quantity and a value of the at least one time-independent parameter as input to both the at least one physical model and to an input side of at least one artificial neural network, - using the provided values, operating one out of the at least one artificial neural network and the at least one processor unit for executing the at least one physical model for obtaining an update information, | - providing the obtained update information to the other one out of the at least | one artificial neural network and the at least one processor unit, and - using the update information, operating the other one out of the at least one | artificial neural network and the at least one processor unit for executing the | at least one physical model for obtaining a prediction value for at least one | status vector characterizing a status of the physical system or environment.

| [0020] The term “status vector’, as used in the present application, shall particularly be understood as an n-tupel of n parameters expressed by real numbers, wherein n is a positive integer equal to or larger than 1.

[0021] An advantage of the invention lies in that a generic concept is provided to | build a prediction model for a potentially complex physical system. The proposed | method can combine solid physical understanding with the capabilities of modern | neural networks to learn from a vast amount of data. The method can be utilized to | improve a functionality of a sensing system that is disturbed by the physical | system, or the method can gain additional information based upon it. | [0022] In principle, the invention is beneficially applicable to any environment or | physical system that is to be monitored by a sensing device. In particular, the | invention is applicable with advantage in the automotive sector. The term | “automotive”, as used in the present application, shall particularly be understood | as being suitable for use in vehicles including passenger cars, trucks, semi-trailer trucks and buses. An important field of application for the present invention may particularly be the field of interior automotive sensing devices for seat occupant detection and/or classification or seat occupant monitoring. | [0023] In this case, the at least one time-dependent physical quantity may be, without being limited to, one out of a position of a seat occupant in any of former time steps; a mechanical tension within a seat foam of a seat occupied by a user, | in any of former time steps; position and mechanical tension of chassis springs in any of former time steps; and an acceleration that is induced by driving behavior and/or road roughness. The at least one time-independent parameter may be, without being limited to, one out of a mechanical parameter of a seat foam serial; a seat spring material parameter; and any one of parameters that are suitable to | characterize a human body stiffness. | [0024] In cases in which the at least one physical model includes a plurality of | time-dependent physical quantities that are to be sensed by the sensing device, | and a plurality of time-independent parameters, the method steps include that | - the at least one artificial neural network is operated for obtaining the update | information, which comprises update values for at least a portion of the | plurality of time-independent parameters,

- the obtained update information is provided to the at least one processor unit, and - using the update information, the at least one processor unit is operated for executing the at least one physical model for obtaining a prediction value for the at least one status vector. | [0025] In this way, the at least one artificial neural network can be employed for obtaining prediction values, i.e. estimates, for any time-independent parameter that forms part of the at least one physical model and that is a priori unknown. The estimates may be obtained while the sensing device is monitoring the physical system or environment, or the estimates may be obtained in a training phase of | the at least one artificial neural network. | [0026] In cases in which the at least one artificial neural network includes a | plurality of network parameters, the method steps include that - the at least one processor unit for executing the at least one physical model | is operated for obtaining the update information, which comprises update values for at least a portion of the plurality of network parameters, | - the obtained update information is provided to the at least one artificial neural | network, and | - using the update information, the at least one artificial neural network is | operated for obtaining a prediction value for the at least one status vector. | [0027] By updating parameters of the at least one artificial neural network in the | described manner, a more precise prediction value for the at least one status | vector can be obtained.

[0028] In preferred embodiments of the method, the steps include that - the at least one processor unit for executing the at least one physical model | is operated for obtaining the update information, which comprises at least | one value for at least one additional physical quantity characterizing a specific feature of the physical system, | - the obtained update information is provided to the at least one artificial neural | network, and - using the update information, the at least one artificial neural network is | operated for obtaining a prediction value for the at least one status vector.

[0029] By using values for at least one additional physical quantity provided by the at least one physical module to the at least one artificial neural network in the ; described manner, a more precise prediction value for the at least one status vector can be generated by the at least one artificial neural network.

[0030] Particularly in the field of interior automotive sensing devices for seat occupant detection and/or classification or seat occupant monitoring, the at least | one status vector that characterizes a status of the environment or physical | system is preferably formed by at least one out of a position and a velocity of at | least one anatomical landmark of a human body. In this way, more precise | prediction values for movements of seat occupants can be enabled. | [0031] In other automotive applications, the at least one status vector that | characterizes a status of the physical system is preferably formed by at least one | out of a position, a velocity, a mechanical tension, a friction status and an engine vibration parameter of the vehicle. In this way, more precise prediction values for vehicle movements can be enabled that may serve as an input for vehicle stability ) systems such as electronic stability control (ESC). Also modeling engine vibration parameters may be used to detect abnormal engine behavior.

[0032] In another aspect of the invention, an automotive sensing system is | provided that includes | - a sensing device that is configured for sensing physical quantities of an | environment or physical system including at least a portion of a vehicle, | - at least one processor unit that is configured to execute at least one physical | model representing the environment or physical system, the physical model | including at least one time-dependent physical quantity that is to be sensed | by the sensing device, and at least one time-independent parameter, wherein | the at least one physical quantity and the at least one parameter characterize specific features of the environment or physical system, - at least one artificial neural network, and - an output unit that is configured for generating output signals that represent prediction values for at least one status vector characterizing a status of the | environment or physical system.

[0033] The at least one processor unit and the at least one artificial neural network are operatively connected for exchanging information. The sensing device is configured for providing an output signal representing values of sensed physical quantities to both the at least one processor unit and the at least one artificial neural network. Eventually, the at least one processor unit and the at least one artificial neural network are configured to execute corresponding steps of the method disclosed herein.

| [0034] The phrase “being configured to”, as used in this application, shall in particular be understood as being specifically programmed, laid out, furnished or arranged.

[0035] The benefits described in context with the disclosed method apply to the proposed automotive sensing system to the full extent.

[0036] In preferred embodiments of the automotive sensing system, the portion of | the vehicle may include an interior of the vehicle. In other preferred embodiments, | the portion of the vehicle may include an engine installed in an engine | compartment of the vehicle. | [0037] Preferably, the at least one physical model representing the environment | or physical system can be expressed as one or more ordinary differential | equations, or one or more partial differential equations, or a linear model. In this way, the environment can be described in an appropriate manner that also allows | for suitable mathematical treatment. | [0038] Preferably, the at least one artificial neural network is designed as a | feedforward neural network, a recurrent neural network such as a long short-term | memory (LSTM) neural network, a residual neural network or a Bayesian neural | network. Artificial neural networks of the proposed kinds can cover a large amount | of specific applications. With artificial neural networks of the described types many | applications can be covered, particularly in the field of automotive sensing devices. | [0039] In preferred embodiments of the automotive sensing system, the sensing | device comprises at least one out of an acceleration sensor, an optical camera | directed towards the interior of the vehicle or towards an exterior of the vehicle, an | interior radar sensor system having at least one radar antenna that is directed | towards the interior of the vehicle or towards the exterior of the vehicle, a LIDAR

(light detection and ranging) sensor and a capacitive sensor that is configured to be arranged within the interior of the vehicle. In this way, a large number of different sensing applications, including interior automotive sensing as well as exterior automotive sensing applications, can be covered.

[0040] It is emphasized herewith that it is within the scope of the invention that the physical system or environment may be represented by more than one physical model, which may be interrelated or not, and that more than one artificial neural network may be employed in any suitable configuration similar to the ones presented above.

[0041] These and other aspects. of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0042] It shall be pointed out that the features and measures detailed individually in the preceding description can be combined with one another in any technically meaningful manner and show further embodiments of the invention. The description characterizes and specifies the invention in particular in connection | with the figures. | Brief Description of the Drawings | [0043] Further details and advantages of the present invention will be apparent | from the following detailed description of not limiting embodiments with reference | to the attached drawings, wherein: | Fig. 1 schematically illustrates a general configuration of an automotive sensing | system operated pursuant to a method in accordance with the invention, | Fig.2 shows a schematic configuration of the automotive sensing system | pursuant to Fig. 1 being operated according to an embodiment of the method of | the invention, Fig. 3 shows a schematic configuration of the automotive sensing system pursuant to Fig. 1 that is operated according to an alternative embodiment of the method of the invention, and Fig. 4 shows a schematic configuration of the automotive sensing system pursuant to Fig. 1 being operated according to another alternative embodiment of the method of the invention.

Description of Preferred Embodiments

[0044] Fig. 1 schematically illustrates a general configuration of an automotive sensing system 10 that is capable to be operated pursuant to a method in accordance with the invention. Without loss of generality, the automotive sensing system 10 may be thought to be intended to serve for monitoring an interior of a vehicle for seat occupancy detection and classification.

[0045] The automotive sensing system 10 includes a sensing device 12 that is configured for sensing physical quantities x; of an environment or physical system including at least a portion of the vehicle, namely the vehicle interior. To this end, | the sensing device 12 may be equipped with a plurality of appropriate sensors (not | shown). Data of the sensed physical quantities x; may for instance include data | from one or more acceleration sensors, data from one or more optical cameras | that are directed towards the interior of the vehicle, data from a radar sensor | system having a radar antenna that is directed towards the interior of the vehicle, | and/or data from a capacitive sensor arranged at the steering wheel for hands off detection.

[0046] The automotive sensing system 10 further comprises a processor unit 14 that is configured to execute a physical model Q representing the monitored environment or physical system. For instance, the processor unit 14 may form an integral part of a microcontroller. Microcontrollers that are suitably equipped and include, for instance, a processor unit 14, a digital data memory unit and a microcontroller system clock are nowadays readily available in many variations.

[0047] In mathematical terms, a physical system can be considered a function P | that takes input data x, which encodes dynamical states from the past, and constant conditions, denoted by z, such as material parameters, or environmental conditions such as temperature or humidity. The output of the system is denoted | by y, is usually vector-valued and encodes a present state of the physical system: | y = P(x,2)

[0048] The physical model Q executed by the processor unit 14 approximates the | physical system P such that an output of the physical model is an approximation of | the output y of the physical system P:

| Qxs 25) = P(x, 2) = y

[0049] The sensed physical quantities x, form part of possible input data x of the physical system, and constant conditions and parameters z, contain a portion of the information of z. For instance, z, may comprise predetermined estimates of average material parameters.

[0050] Thus, the physical model Q includes time-dependent physical quantities x, that are to be sensed by the sensing device 12, and time-independent parameters z,, wherein the physical quantities x; and the parameters z, characterize specific features of the physical system or environment. The physical model Q may be implemented in the form of linear, second-order ordinary differential equations with constant coefficients, wherein the variables may be a | vertical displacement of a vehicle seat/occupant unit and a vertical displacement of | a vehicle body. The constant coefficients may, for instance, represent a mass of | the vehicle seat/occupant unit, a damping coefficient of a vehicle spring damper | system, and a stiffness coefficient of a mechanical connection between the vehicle seat and the vehicle body. In other embodiments, a physical model may be implemented in the form of partial differential equations or a linear model. | [0051] Further, the automotive sensing system 10 comprises an artificial neural | network 16, which may be designed as a recurrent neural network (RNN) such as a long short-term memory (LSTM) neural network. The processor unit 14 and the | artificial neural network 16 are operatively connected for exchanging information. | The sensing device 12 is configured for providing values of the sensed physical quantities x, to both the processor unit 14 as a model input and to the artificial neural network 16. | [0052] As will be described in more detail, the artificial neural network 16 and the | physical model Q executed by the processor unit 14 cooperate to function as an | improved physical model A(xs,zs) such that an outputy, of the physical | model A(xs, zs) is a closer approximation to the output y of the physical system P: | Alxs,z) = ys = P(x, z) =y | [0053] Moreover, the automotive sensing system includes an output unit 18. The output unit 18 is configured for generating output signals that represent prediction | values for at least one status vector y, characterizing a status of the physicalsystem or environment. For the purpose of vehicle seat occupancy detection, the status vector ys may include the position of an anatomic landmark of a seat occupant body.

[0054] Fig. 2 shows a schematic configuration of the automotive sensing system 10 pursuant to Fig. 1 being operated according to a specific embodiment of the method of the invention of predicting the time behavior of the environment that is being monitored by the sensing device 12. The processor unit 14 and the artificial neural network 16 are configured to execute corresponding steps of the method, wherein the steps are to be executed repeatedly, more specifically periodically, for instance controlled by the microcontroller system clock.

[0055] The physical model Q comprises a plurality of unknown time-dependent and/or time-independent parameters w and can thus be denoted as Q (xs, zs, w). Based on the input x, z,, the artificial network 16 is employed to predict these parameters w. | [0056] In one step 20 of the method, former values for the time-dependent | physical quantities x, sensed by the sensing device 12 and values of the time- | independent parameters zs are provided as input to both the physical model Q, i.e. | the processor unit 14, and to an input side of the artificial neural network 16. Using | the provided values xg, ze the artificial neural network 16 is operated in another | step 22 for obtaining an update information, which comprises update values for the | plurality of time-dependent and/or time-independent parameters w. | [0057] The update values may be obtained in a training phase of the artificial | neural network 16. In this case, the artificial neural network 16 predicts values for | time-independent parameters w. The update values may be obtained while the | sensing device 12 is monitoring the physical system. In this specific embodiment | of the method, the artificial neural network 16 predicts values for time-dependent | parameters w(t), as indicated in Fig. 2. | [0058] It should be generally understood that xs,zs vary in time although their time dependency is not explicitly given in the notation. The time dependency has been added explicitly for time-dependent parameters w(t) to underline the difference between time-dependent parameters w(t) and time-independent parameters w.

| P-IEE-527/LU 14 LU101167

[0059] In another step 24, the obtained update information is provided to the processor unit 14. Then, using the update information, the processor unit 14 is | operated in a step 26 for executing the physical model Q for obtaining a prediction | value for the status vector y, characterizing a status of the physical system. The | status vector ys may comprise a position and a velocity of a suitable anatomical | landmark of a human vehicle seat occupant. Data representing the status | vector ys are provided by the output unit 18 for further processing. | [0060] Fig. 3 shows a schematic configuration of the automotive sensing | system 10 pursuant to Fig. 1 that is operated according to an alternative | embodiment of the method of the invention. In order to avoid unnecessary | repetitions, only differences with respect to the first embodiment pursuant to Fig. 2 | will be described. For features in Fig. 3 that are not described in context with the | alternative embodiment, reference is made to the description of the first | embodiment.

[0061] In the embodiment pursuant to Fig. 3, the artificial neural network 16 is shown to include a plurality of network parameters 8 = (05;,0y). Formally, the function of the artificial neural network 16 can be denoted by N(xs, zs, (Ofix» 65). fi denotes parameters that are fixed during life time, and 6, denotes variable parameters that can be continuously updated by the physical model during life time of the automotive sensing system 10.

[0062] Using the provided values xg, zs, the processor unit 14 for executing the physical model is operated in a method step 28 for obtaining the update information, which comprises update values for the plurality of variable | parameters 6,. The update values are then provided to the artificial neural network 16 in another step 30. Eventually in another step 32, using the update values, the artificial neural network 16 is operated for obtaining a prediction value for the status vector ys.

[0063] Fig. 4 shows a schematic configuration of the automotive sensing system 10 pursuant to Fig. 1 being operated according to another alternative embodiment of the method of the invention. Again, only differences with respect to the first embodiment pursuant to Fig. 2 will be described. For features in Fig. 4 thatare not described in context with the alternative embodiment, reference is made to the description of the first embodiment. | [0064] In the embodiment of the method pursuant to Fig. 4, the artificial neural | network 10 uses an additional input, which is formed by an additional physical | quantity u characterizing a specific feature of the physical system. Formally, the | function of the artificial neural network 16 can be denoted by N(x, zg, u). | [0065] Using the provided values xs,zs, the processor unit 14 for executing the | physical model Q is operated in a method step 28 for obtaining the update | information, which comprises a value for the additional physical quantity u. The | value is then provided to the artificial neural network 16 in another step 30. | Eventually in another step 32, using the value for the additional physical | quantity u, the artificial neural network 16 is operated for obtaining a prediction value for the status vector ys.

[0066] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

[0067] Other variations to be disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality, which is meant to express a quantity of at least two. The mere fact that certain measures are recited in mutually different | dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed | as limiting scope.

List of Reference Symbois automotive sensing system 12 sensing device 14 processor unit 16 artificial neural network 18 output unit A combined function of artificial neural network and physical model Q physical model function N artificial neural network function u additional physical quantity xs time-dependent physical quantity | ys physical model output (status vector) | Ze time-independent parameter | Ori, fixed network parameters | 6, variable network parameters | w(t) time-dependent parameters | Method steps: provide former values xs and zg to physical model and artificial neural network 22 operate artificial neural network for obtaining an update information 24 provide update information to processor unit 26 operate processor unit for obtaining a prediction value for status vector yg 28 operate processor unit for obtaining an update information provide update information to artificial neural network 32 operate artificial neural network for obtaining a prediction value for status vector ys

Claims

1. A method of predicting the time behavior of a physical system that is being monitored by a sensing device (12) and that is represented by at least one physical model (Q) executable by at least one processor unit (14), the at least one physical model (Q) including at least one time-dependent physical quantity (xs) that is to be sensed by the sensing device (12), and at least one time-independent parameter (zs), wherein the at least one physical quantity (xs) and the at least one parameter (zs) characterize specific features of the physical system, the method comprising at least the following steps, which are to be executed repeatedly: - providing (20) a former value for the at least one time-dependent physical quantity (xs) and a value of the at least one time-independent | parameter (ze) as input to both the at least one physical model (Q) and to an input side of at least one artificial neural network (16), - using the provided values, operating (22) one out of the at least one artificial neural network (16) and the at least one processor unit (14) for executing the at least one physical model (Q) for obtaining an update information, - providing (24) the obtained update information to the other one out of the at least one artificial neural network (16) and the at least one processor unit (14), and - using the update information, operating (26) the other one out of the at least one artificial neural network (16) and the at least one processor unit (14) for executing the at least one physical model (Q) for obtaining a prediction value for at least one status vector (ys) characterizing a status of the physical system.

|

2. The method as claimed in claim 1, wherein the at least one physical model (Q) includes a plurality of time-dependent physical quantities (xs) that are to be sensed by the sensing device (12) and a plurality of time-independent parameters (zg), and wherein the steps include that

- the at least one artificial neural network (16) is operated (22) for obtaining the update information, which comprises update values for at least a portion (w(t)) of the plurality of time-independent parameters (zg), - the obtained update information is provided (24) to the at least one processor unit (14), and - using the update information, the at least one processor unit (14) is | operated (26) for executing the at least one physical model (Q) for obtaining a prediction value for the at least one status vector (ys).

3. The method as claimed in claim 1 or 2, wherein the at least one artificial neural | 10 network (16) includes a plurality of network parameters (6, 6,), and wherein | the steps include that | - the at least one processor unit (14) for executing the at least one physical | model (Q) is operated (28) for obtaining the update information, which comprises update values for at least a portion (6,) of the plurality of network | 15 parameters (yy, 05), - the obtained update information is provided (30) to the at least one artificial neural network (16), and - using the update information, the at least one artificial neural network (16) is operated (32) for obtaining a prediction value for the at least one status vector (ys).

4. The method as claimed in any one of the preceding claims, wherein the steps include that | - the at least one processor unit (14) for executing the at least one physical model (Q) is operated for obtaining (28) the update information, which comprises at least one value for at least one additional physical quantity (u) characterizing a specific feature of the physical system, - the obtained update information is provided (30) to the at least one artificial neural network (16), and - using the update information, the at least one artificial neural network (16) is operated (32) for obtaining a prediction value for the at least one status vector (ys).

5. The method as claimed in any one of the preceding claims, wherein the at least one status vector (ys) is formed by at least one out of a position and a velocity of at least one anatomical landmark of a human body. |

6. The method as claimed in any one of claims 1 to 4, wherein the at least one status vector (ys) is formed by at least one out of a position, a velocity, a mechanical tension, a friction status and an engine vibration parameter of a vehicle.

7. An automotive sensing system (10), including - a sensing device (12) that is configured for sensing physical quantities (xs) of a physical system including at least a portion of a vehicle, - at least one processor unit (14) that is configured to execute at least one physical model (Q) representing the physical system, the physical model (Q) including at least one time-dependent physical quantity (xs) that is to be sensed by the sensing device (12), and at least one time-independent parameter (zg), wherein the at least one physical quantity (xg) and the at least one parameter (zs) characterize specific features of the physical system, - at least one artificial neural network (16), and - an output unit (18) that is configured for generating output signals that represent prediction values for at least one status vector (ys) characterizing a status of the physical system, wherein - the at least one processor unit (14) and the at least one artificial neural network (16) are operatively connected for exchanging information, - the sensing device (12) is configured for providing an output signal representing values of sensed physical quantities (xs) to both the at least one processor unit (14) and the at least one artificial neural network (16), and - the at least one processor unit (14) and the at least one artificial neural network (16) are configured to execute corresponding steps of the method as claimed in any one of claims 1 to 6.

8. The automotive sensing system (10) as claimed in claim 7, wherein the at least one physical model (Q) representing the physical systemcan be expressed as one or more ordinary differential equations, one or more partial differential equations, or a linear model.

9. The automotive sensing system (10) as claimed in claim 7 or 8, wherein the at least one artificial neural network (16) is designed as a feedforward neural network, a recurrent neural network such as a long short-term memory neural network, a residual neural network or a Bayesian neural network.

10. The automotive sensing system (10) as claimed in any one of claims 7 to 9, wherein the sensing device (12) comprises at least one out of an acceleration sensor, an optical camera directed towards the interior of the vehicle or towards an exterior of the vehicle, an interior radar sensor system having at least one radar antenna that is directed towards the interior of the vehicle or towards the exterior of the vehicle, a LIDAR sensor and a capacitive sensor that is configured to be arranged within the interior of the vehicle.