DE102022205730A1 - Concept for selecting audio snippets of brake squealing noises in a vehicle - Google Patents

Abstract

The present invention relates to a device (10), a method and a computer program for selecting audio excerpts of brake squealing noises in a vehicle (100), and to a method, a computer program and a device (20) for training a machine learning model Making a prediction about the occurrence of brake squeal. The device (10) is designed to receive one or more audio recordings from the one or more microphone modules, to obtain status information about the vehicle from the one or more status data sources of the vehicle, to process the status information by a machine learning model, the machine learning model Model is trained to make a prediction about the occurrence of brake squeal based on the condition information, to select, based on an output of the machine learning model, audio excerpts from the one or more audio recordings, and to store the selected audio excerpts on a storage device ( 16).

Description

The present invention relates to a concept for selecting audio excerpts of brake squealing noises in a vehicle and, more particularly, to an apparatus, a method and a computer program for selecting audio excerpts of brake squealing noises in a vehicle, and to a method, a computer program and an apparatus for training a machine learning model to make a prediction about the occurrence of brake squeal.

During vehicle development, changes are often introduced that may have an impact on vehicle braking. A common challenge associated with vehicle brakes is squeaking noises. Such squeaking noises from the brakes should be identified during vehicle development so that corrective measures can be taken. To record the noise, a measuring system is usually installed in test vehicles, which uses microphones and accelerometers to determine the noise at the brakes. In addition, the measuring system often includes temperature sensors on the brakes.

The evaluation of disturbing noises on the vehicle to classify them into false noises and squeaking noises from the brakes is generally done manually. An engineer listens to thousands of noises per test vehicle in order to classify them. The classification is therefore based, among other things, on the wealth of experience and is subjective. The evaluation is often time-critical but takes many hours. To make the engineer's work easier, measurement software can pre-classify the noise. The basis for this is analysis of the noises (audio recordings) themselves, such as a so-called “impulse analysis” or the use of the “Relative Approach Algorithm”. Machine learning models are also used for this activity (see for example CN 1 09684742A and Stender et Al: “Deep Learning for break squeal: brake noise detection, characterization and prediction”).

However, the pre-classification only slightly limits the manual effort. All audio recordings must also be saved for analysis. The audio signal from the accelerometer and microphone is saved every time you brake and for each brake, which leads to large amounts of data. The large amounts of data also make processing in the evaluation software slow and prone to errors. Data loss can result. In addition, new vehicle classes, such as electric vehicles, bring with them new noise levels. This can be, for example, a sound generator that simulates an engine noise. This further complicates the evaluation based on the audio recordings.

There is a need for an improved approach to obtaining audio recordings of brake noise.

This need is taken into account by the independent claims.

The present invention is based on the following findings. On the one hand, it was recognized that not only the audio recordings allow a statement to be made as to whether brake squeaking is present, but that other vehicle status information, such as vehicle speed, brake service life, brake pressure, steering angle, outside temperature and brake temperature, can also be used to make a prediction regarding the occurrence of brake squeaking. This state information can now be used to train a machine learning model to make this prediction. Optionally, metadata from the audio recordings, such as volume and frequency spectrum, can also be used as further input parameters for the machine learning model. This machine learning model can now be used in the vehicle to carry out pre-selection and to identify audio excerpts where brake squealing is likely to be heard. These audio excerpts can now be saved and the remaining parts of the audio recording(s) can now be discarded, which on the one hand reduces the effort for manual evaluation and on the other hand also reduces or minimizes the required storage space.

A first aspect of the present disclosure relates to an apparatus for selecting audio excerpts of brake squealing noises in a vehicle. The device comprises at least one interface for communication with one or more microphone modules of a measuring device and with one or more status data sources of the vehicle. The device further includes one or more processors configured to receive one or more audio recordings from the one or more microphone modules. The one or more processors are configured to obtain status information about the vehicle from the one or more vehicle status data sources. The one or more processors are configured to process the state information through a machine learning model. The machine learning model is trained to make a prediction about the occurrence of braking based on the status information squeak to meet. The one or more processors are configured to select, based on an output of the machine learning model, audio excerpts from the one or more audio recordings. The one or more processors are configured to store the selected audio excerpts on a storage device. As already written above, this can, on the one hand, reduce the effort for manual evaluation and, on the other hand, reduce or minimize the required storage space.

The one or more processors can in particular be designed to discard parts of the one or more audio recordings that were not selected. These parts are, in most cases, of no interest in identifying brake squeal and can therefore be discarded, thereby reducing the storage space required.

In some embodiments, the one or more processors are further configured to determine metadata about the one or more audio recordings. On the one hand, this metadata can be used as further input data for the machine learning model and, on the other hand, it can be used to determine the relevant times for processing by the machine learning model.

The machine learning model can thus be trained to further make the prediction about the occurrence of brake squeaking based on the metadata about the one or more audio recordings. This can increase the accuracy of the prediction.

For example, the metadata about the one or more audio recordings can include information about a volume and/or information about a frequency spectrum of the respective audio recording. Both the volume and the frequency spectrum are relevant criteria that can be used, on the one hand, to distinguish brake squealing from other noises and, on the other hand, to determine whether the prediction presented above can be carried out for a particular noise.

For example, the status information can be synchronized in time with the one or more audio recordings of the one or more microphone modules. The one or more processors may be configured to determine the times for which the brake squeal occurrence prediction is determined using the machine learning model based on the metadata about the one or more audio recordings. This makes it possible to pre-select potentially relevant noises, which can reduce the processing effort.

In general, brake noise can occur independently from the different brakes of the vehicle. Therefore, in some embodiments, the prediction is made separately for the respective brakes and the respective audio recordings of the brakes. Thus, the one or more processors may be configured to receive a plurality of audio recordings from a plurality of microphone modules. For example, the different microphone modules can be arranged on the different brakes. The machine learning model can be trained to predict the occurrence of brake squeal for each microphone module separately. The one or more processors can be designed to carry out the selection of the audio excerpts separately for the recordings of the respective microphone module. This allows for more precise control over which audio excerpts should be saved, further reducing the storage space required.

For example, the status information can be one or more pieces of information from the group of information about a speed of the vehicle, information about a mileage of the vehicle, information about a service life of the brakes of the vehicle, information about a brake pressure of a braking process, information about a steering angle, information about an outside temperature and information about a temperature of the brakes. In an analysis of the influence of different status information, this status information showed a particularly high influence on the prediction.

In principle, many implementations of the machine learning model are possible. For example, the machine learning model includes a decision tree generated by training the machine learning model. Such decision trees have the advantage that they can be trained with less effort than neural networks, that they are more comprehensible than neural networks and can also be used on systems with lower processing capacity. They can also be efficiently implemented in software and hardware. Alternatively, the machine learning model can be, for example, a neural network, such as a so-called deep neural network (DNN).

While this is a classification task, it can be advantageous to when the result of the machine learning model is not a binary classification (squeak/no squeak), but rather when a non-binary value is output. For example, the machine learning model can be trained to output information about a probability of the occurrence of brake squeaking based on the status information, and optionally based on metadata about the one or more audio recordings. The audio excerpts can be selected based on the probability of brake squealing occurring. This allows the threshold value to be adjusted subsequently without having to adjust the machine learning model.

Another aspect of the present disclosure relates to a corresponding method for selecting audio recordings of brake squealing noises in a vehicle. The method includes obtaining one or more audio recordings from one or more microphone modules of the vehicle. The method further includes obtaining status information about the vehicle from one or more status data sources of the vehicle. The method further includes processing the state information by a machine learning model. The machine learning model is trained to make a prediction about the occurrence of brake squeal based on the condition information. The method further includes selecting, based on an output of the machine learning model, audio excerpts from the one or more audio recordings. The method further includes storing the selected audio excerpts on a storage device within the vehicle.

Another aspect of the present disclosure relates to a corresponding program having program code for performing the above method when the program code is executed on a computer, a processor, a control module, or a programmable hardware component.

In the present disclosure, a trained machine learning model is used. Accordingly, another aspect of the present disclosure relates to a method for training a machine learning model to make a prediction about the occurrence of brake squeal. The method includes obtaining training input data and training output data for training the machine learning model. The training input data includes state information about a condition of the vehicle and the training output data includes information about an occurrence of brake squealing for a given condition of the vehicle. The method further includes training the machine learning model based on the training input data and based on the training output data. Training includes creating a decision tree. The machine learning model can now be used in the previously presented apparatus, method and computer program for selecting audio excerpts of brake squealing noises in a vehicle. By creating a decision tree, a machine learning model is created that can be trained with less effort than, for example, a neural network, that is more comprehensible than a neural network and that can also be used on systems with lower processing capacity.

As already written in connection with the device, method and computer program for selecting audio excerpts of brake squealing noises in a vehicle, the training input data may further include metadata about one or more audio recordings. This can increase the accuracy of the prediction.

For example, the machine learning model can be trained to output information about a probability of brake squealing occurring based on the condition information, and optionally based on metadata about one or more audio recordings. This enables more flexible use of the machine learning model, as a threshold for a classification can now be set subsequently.

A further aspect of the present disclosure relates to a corresponding device with one or more interfaces and with one or more processors, designed to carry out the previously presented method.

Another aspect of the present disclosure relates to a corresponding program with program code for performing the previously presented method when the program code is executed on a computer, a processor, a control module or a programmable hardware component.

• 1a zeigt ein Blockdiagramm einer Vorrichtung zum Auswählen von Audioausschnitten von Bremsquietschgeräuschen in einem Fahrzeug;
• 1b zeigt ein schematisches Diagramm eines Fahrzeugs mit einer Vorrichtung zum Auswählen von Audioausschnitten von Bremsquietschgeräuschen in einem Fahrzeug;
• 1c zeigt ein Flussdiagramm eines Verfahrens zum Auswählen von Audioausschnitten von Bremsquietschgeräuschen in einem Fahrzeug;
• 2a zeigt ein Flussdiagramm eines Verfahrens zum Trainieren eines Maschinenlern-Modells zum Treffen einer Vorhersage über das Vorkommen von Bremsquietschen;
• 2b zeigt ein Blockdiagramm einer Vorrichtung zum Trainieren eines Maschinenlern-Modells zum Treffen einer Vorhersage über das Vorkommen von Bremsquietschen; und
• 3 zeigt ein schematisches Diagramm eines Ausschnitts eines Entscheidungsbaums.

Some examples of devices and/or methods are explained in more detail below with reference to the accompanying figures, merely by way of example. Show it:

• 1a shows a block diagram of an apparatus for selecting audio excerpts from brake squealing noises in a vehicle;
• 1b shows a schematic diagram of a vehicle with an off device selecting audio clips of brake squealing noises in a vehicle;
• 1c shows a flowchart of a method for selecting audio excerpts from brake squeal noises in a vehicle;
• 2a shows a flowchart of a method for training a machine learning model to make a prediction about the occurrence of brake squeal;
• 2 B shows a block diagram of an apparatus for training a machine learning model to make a prediction about the occurrence of brake squeal; and
• 3 shows a schematic diagram of a section of a decision tree.

Some examples will now be described in more detail with reference to the accompanying figures. However, other possible examples are not limited to the features of these embodiments described in detail. These may include modifications to the features as well as equivalents and alternatives to the features. Further, the terminology used herein to describe particular examples is not intended to limit other possible examples.

The same or similar reference numerals throughout the description of the figures refer to the same or similar elements or features, which can each be implemented identically or in a modified form while providing the same or a similar function. Furthermore, in the figures, the thicknesses of lines, layers and/or areas may be exaggerated for clarity.

When two elements A and B are combined using an “or”, it is understood that all possible combinations are disclosed, i.e. H. only A, only B as well as A and B, unless expressly defined otherwise in individual cases. As an alternative formulation for the same combinations, “at least one of A and B” or “A and/or B” can be used. This applies equivalently to combinations of more than two elements.

If a singular form, e.g. For example, if "a, an" and "the, the, the" are used and the use of only a single element is neither explicitly nor implicitly defined as mandatory, further examples can also use multiple elements to implement the same function. Where a function is described below as being implemented using multiple elements, further examples may implement the same function using a single element or processing entity. It is further understood that the terms “comprises,” “comprising,” “having,” and/or “having,” when used, mean the presence of the specified features, integers, steps, operations, processes, elements, components, and/or group describe the same, but do not exclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, components and/or a group thereof.

1a shows a block diagram of a device 10 for selecting audio excerpts of brake squealing noises in a vehicle 100. The device 10 comprises at least one interface 12 for communication with one or more microphone modules 102a of a measuring device 102 and with one or more status data sources 104 of the vehicle. The device 10 further includes one or more processors 14 that are configured to receive (e.g., receive, retrieve, or read) one or more audio recordings from the one or more microphone modules 102a (via the at least one interface 12). The one or more processors 14 are further configured to obtain (such as receive, retrieve or read) status information about the vehicle from the one or more status data sources 104 of the vehicle. The one or more processors 14 are further configured to process the state information through a machine learning model. The machine learning model is trained to make a prediction about the occurrence of brake squeal based on the condition information. The one or more processors 14 are further configured to select, based on an output of the machine learning model, audio excerpts from the one or more audio recordings. The one or more processors 14 are further designed to store the selected audio excerpts on a storage device 16. The storage device 16 can be a storage device of the device 10, or the storage device can be an external storage device, such as an external flash memory, an external hard drive , a USB (Universal Serial Bus) stick or a memory card. The device 10 can, for example, correspond to the measuring device 102, be included in the measuring device 102, or be implemented separately from the measuring device 102.

The present device is used in the vehicle. 1b shows a schematic diagram of the vehicle 100 with the device 10 for selecting audio excerpts of brake squealing noises in a vehicle. The vehicle 100 also includes the measuring equipment device 102 with the one or more microphone modules 102a (whereby the measuring device does not have to be an integral part of the vehicle) and the one or more status data sources 104. If the storage device 16 is not part of the device 10, the vehicle 100 can also use the storage device 16 separately from the device 10 include.

1c shows a flowchart of a corresponding method for selecting audio excerpts from brake squealing noises in a vehicle. The method includes obtaining 110 the one or more audio recordings from the one or more microphone modules of the vehicle. The method further includes obtaining 120 the condition information about the vehicle from the one or more condition data sources of the vehicle. The method further includes processing 130 the state information by the machine learning model. The method further includes selecting 140, based on the output of the machine learning model, audio excerpts from the one or more audio recordings. The method further includes storing 150 the selected audio excerpts on a storage device within the vehicle. For example, the method can be carried out by a computer system, such as the device 10 of 1a .

The functionality of the device 10, the method and a corresponding computer program in connection with the device 10 are explained below. Features that are introduced in connection with the device 10 can also be incorporated into the corresponding method and the corresponding computer program.

The proposed concept deals with the classification of noises based on meta-information, such as the addressed state information and, optionally, metadata about the one or more audio recordings, without relying on the audio recording per se. This is done by using a machine learning model based on parameters that describe the state of the vehicle at the time of the noise (such as speed, mileage, brake pressure, steering angle, outside temperature, brake temperature) and optionally those based on the noise ( such as frequencies or volume of the noise). The noise-based parameters can be determined and stored as metadata about the audio recording while driving by the measuring device or by the device 10. The sound itself does not necessarily have to be saved as an audio recording. The decision as to whether a sound is saved is made in the following parameter. Other parameters, such as the temperature of the brakes, can also be recorded by the measuring device 102. The other parameters of the vehicle occur in the CAN (Controller Area Network) bus and can be read there by the one or more processors 14 via the at least one interface 12. Consequently, the at least one interface 12 may include a CAN bus interface. Furthermore, the at least one interface 12 can include a so-called Ethernet interface or a USB interface for communication with the measuring device. All parameters are measured values and therefore objective.

The trained model predicts with high accuracy what type of noise (missing noise or brake squeak) it is based on the parameters mentioned for new noises. Thanks to the high level of accuracy, the manual effort required to identify brake squealing in a test setup was reduced to approximately 5% of the total number of noises. Only the remaining noises identified as squeaks are still processed manually and are therefore saved for manual processing.

The present concept is based on two types of data - the state information, and the one or more audio recordings. Metadata derived from the audio recordings can also be used to use one or more audio recordings. How this data is obtained is explained below.

The one or more processors are designed to receive, i.e. H. for receiving, reading out or retrieving the one or more audio recordings from the one or more microphone modules. The type of preservation depends on the design of the microphone modules. In some cases the microphone modules are simple microphones. In this case, the one or more audio recordings are received as an audio signal and digitized by the one or more processors. Alternatively, the one or more microphone modules can already include analog-to-digital converters. In this case, the one or more audio recordings can already be received, read or accessed as a data stream or as files. Due to the various possible configurations, the term microphone module was chosen, which, in addition to the respective microphone, can also include the analog-to-digital converter and a communication interface. Furthermore, the microphone module can also include an accelerometer that generates part of the respective audio recording.

The present concept is based on the one or more audio recordings. Several implementations are possible. In some cases, a separate audio recording can be obtained for each sound (and possibly for each microphone module). The division into audio recordings can be done by the measuring device. Alternatively, a longer audio recording, such as a longer audio recording for each microphone module, may be obtained that includes multiple sounds. In this case, the device can process the audio recording(s) in sections.

As previously discussed, metadata about the audio recording(s) can be used. Thus, the one or more processors can further be designed to determine metadata about the one or more audio recordings (e.g. to calculate, extract from the audio recording (s) or obtain it from the measuring device or the one or more microphone modules (s)) . The metadata about the one or more audio recordings can include information about a volume and/or information about a frequency spectrum of the respective audio recording. In particular, the metadata about the one or more audio recordings can include information about a volume and/or information about a frequency spectrum of the respective sounds that are represented by the one or more audio recordings.

The status information in turn can be obtained directly from the respective status data sources (which can be part of the vehicle or the measuring device), i.e. H. received, accessed or read. As already mentioned, the CAN bus, Ethernet, USB or similar systems can be used for communication or data retrieval. State information is information that represents a state of at least one component of the vehicle or an environment of the vehicle. These can be the speed, the mileage (as a proxy for brake wear), the brake pressure, the steering angle, the outside temperature and the temperature of the brakes. In other words, the status information can be one or more pieces of information from the group of information about a speed of the vehicle, information about a mileage of the vehicle, information about a service life of the brakes of the vehicle, information about a brake pressure of a braking process, information about a steering angle, information about include an outside temperature and information about a temperature of the brakes. The status data sources can include, for example, one or more of the CAN bus, an on-board computer, a speed measuring device of the vehicle, a steering angle measuring device of the vehicle, an odometer of the vehicle, a brake pressure measuring device of the vehicle, an outside temperature measuring device of the vehicle and a brake temperature measuring device of the vehicle or the measuring device.

This state information, and optionally the metadata about the one or more audio recordings, are now processed by the machine learning model. In other words, the one or more processors are configured to process the state information (and optionally the metadata about the one or more audio recordings) by the machine learning model, the machine learning model being trained based on the state information (and optionally the metadata via the one or more audio recordings) to make a prediction about the occurrence of brake squeaking. A brief introduction to machine learning is given below to briefly explain the concept of processing by a machine learning model.

Machine learning refers to algorithms and statistical models that computer systems can use to perform a specific task without relying on predefined instructions. In machine learning, for example, instead of a rule-based transformation of data, a transformation of data can be used that can be derived from an analysis of historical and/or training data. For example, the content of images can be analyzed using a machine learning model or using a machine learning algorithm. In order for the machine learning model to analyze the content of an image, the machine learning model can be trained using training images as input and training content information as output. By training the machine learning model with a large number of training images and/or training sequences (e.g. words or sentences) and associated training content information (e.g. labels or annotations), the machine learning model “learns” the content of the images recognize so that the content of images not included in the training data can be recognized using the machine learning model. The same principle can also be used for other types of data, such as existing state data or metadata: By training a machine learning model using training data and a desired output, the machine learning model “learns” a transformation between the data and the output, which can be used to get an output based on that To provide non-training data provided by the machine learning model. The provided data (e.g. the state data or the metadata) can be preprocessed to obtain a feature vector, which is used as input for the machine learning model.

Machine learning models are trained using training input data. The examples above use a training technique called supervised learning. In supervised learning, the machine learning model is trained using a plurality of training data, the training data a plurality of training input data and a plurality of desired training output data, i.e. H. Each training input date is assigned a desired training output date, which can include. By specifying both training input data and desired training input data, the machine learning model “learns” which output date to provide based on an input date that is similar to the training input data provided during training. In addition to supervised learning, semi-supervised learning can also be used. In semi-supervised learning, some of the training input data lacks a desired output value. Supervised learning can be based on a supervised learning algorithm (e.g. a classification algorithm, a regression algorithm, or a similarity learning algorithm). Classification algorithms can be used when the outputs are limited to a limited set of values (categorical variables), i.e. H. the input is classified as one of the limited set of values. Regression algorithms can be used when the outputs show any numerical value (within a range). Similarity learning algorithms can be similar to both classification and regression algorithms, but are based on learning from examples using a similarity function that measures how similar or related two objects are.

In addition to supervised learning or semi-supervised learning, unsupervised learning or reinforcement learning can also be used. In the present case, however, the use of supervised learning or semi-supervised learning makes sense.

Machine learning algorithms are usually based on a machine learning model. In other words, the term “machine learning algorithm” can mean a set of instructions that can be used to create, train, or use a machine learning model. The term “machine learning model” may mean a data structure and/or a set of rules that represents the learned knowledge (e.g. based on the training performed by the machine learning algorithm). In embodiments, the use of a machine learning algorithm may imply the use of an underlying machine learning model (or a plurality of underlying machine learning models). The use of a machine learning model may imply that the machine learning model and/or the data structure/set of rules that is/are the machine learning model is trained by a machine learning algorithm.

In the present case, the state information and optionally the metadata are used as input data (and therefore also as training input data, as in the context of the 2a until 2 B is explained) of the machine learning model (i.e. as a feature vector). This allows the analysis of the noises to be carried out automatically and in a much shorter time (a few seconds for thousands of noises). Thanks to the abstract parameters, the model can also be transferred to other vehicle classes. No large amounts of data in the form of audio files are necessary. The classification is carried out without the noise itself. No additional measuring equipment is required for the process, in addition to the measuring system. The output of the machine learning model, in turn, represents the prediction of the occurrence of brake squeak. In other words, the output of the machine learning model contains the prediction about the occurrence of brake squealing, that is, it can be seen from the output whether brake squealing has occurred or not according to the prediction.

The data on which the machine learning model is run generally corresponds to data recorded at the same time as recorded sounds. The status information can be synchronized in time with the one or more audio recordings of the one or more microphone modules, so that the temporal connection between the noises and the respective status data (and metadata) can be established. The one or more processors can be designed to determine the times for which the prediction about the occurrence of brake squeal is determined using the machine learning model based on the metadata about the one or more audio recordings. In other words, the one or more processors may be configured to detect, based on the metadata, the occurrence of a “loud” sound (ie, a sound with a volume above a threshold) or a “suspicious” sound based on the frequencies (i.e. a sound with a frequency spectrum that does not indicate background noise) and, based on this, trigger the processing of the state data (and metadata) by the machine learning model.

In principle, different types of machine learning models are conceivable for implementing the present machine learning model. For example, the machine learning model may be a decision tree (or include one or more decision trees). The decision tree can be implemented as software, for example. An example of such a decision tree is in 3 shown. Thus, the machine learning algorithm can use a decision tree (or multiple decision trees) as a prediction model. In other words, the machine learning model may include one or more decision tree (or trees) generated by training the machine learning model. In a decision tree, the observations about an item (e.g., a set of input values) can be represented by the branches of the decision tree, and an output value corresponding to the item can be represented by the leaves of the decision tree. Decision trees can support both discrete values and continuous values as output values. When discrete values are used, the decision tree can be called a classification tree; when continuous values are used, the decision tree can be called a regression tree.

Decision trees, for example, are quicker to train than neural networks and are usually best suited for use on tabular data. The implementation as software is more flexible than hardware; the decision tree can be implemented or changed more easily. Standardized libraries are available for implementation in common programming languages (e.g. Python).

Alternatively, other machine learning models can also be used, such as: B. neural networks. For example, the machine learning model can be an artificial neural network (ANN). ANNs are systems inspired by biological neural networks such as those found in a retina or brain. ANNs include a plurality of interconnected nodes and a plurality of connections, called edges, between the nodes. There are usually three types of nodes, input nodes that receive input values, hidden nodes that are (only) connected to other nodes, and output nodes that provide output values. Each node can represent an artificial neuron. Each edge can send information from one node to another. The output of a node can be defined as a (nonlinear) function of the inputs (e.g. the sum of its inputs). A node's inputs can be used in the function based on a "weight" of the edge or node providing the input. The weight of nodes and/or edges can be adjusted in the learning process. In other words, training an artificial neural network may include adjusting the weights of the nodes and/or edges of the artificial neural network, i.e. H. to achieve a desired output for a given input.

The machine learning model can alternatively be implemented as hardware, e.g. B. as electronic circuits. The machine learning model can also be designed in such a way that the classification is also based on the audio recordings. For example, a neural network suitable for image recognition can classify sounds based on spectrograms.

The noises are classified automatically and objectively based on measured values using the machine learning model. The high accuracy of the models significantly reduces manual effort.

Based on this prediction, the selection of audio excerpts from the one or more audio recordings can now be carried out. How this selection is carried out depends on the output of the machine learning model. For example, the machine learning model may be trained to output information about a probability of brake squealing occurring based on the state information, and optionally based on metadata about the one or more audio recordings. In this case, the machine learning model can be a regression model. The selection of audio excerpts can then be made based on the likelihood of brake squealing occurring. For example, an audio excerpt can be selected if the probability of brake squealing occurring at the time of the audio excerpt (based on the output of the machine learning model) is higher than a (user-definable) threshold. Alternatively, the machine learning model can be a classification model. In this case, the output of the machine learning model may contain binary information about whether brake squealing occurred at the time of the audio excerpt. Here the binary information can be used directly as a basis for selection.

In general, vehicles have several brakes, some of which can be triggered independently of each other (e.g. by a vehicle dynamics control or an anti-lock braking system). The occurrence of braking noises usually depends on the direction in which the vehicle is being steered. In this respect, the prediction can be designed in such a way that the prediction is carried out separately for each brake. If a separate microphone module (or microphone modules if the accelerometer and microphone are evaluated separately) is used for each brake, the respective audio excerpts can also be selected separately. Therefore, the one or more processors may be configured to receive a plurality of audio recordings from a plurality of microphone modules. The machine learning model can be trained to make the prediction about the occurrence of brake squeal separately for each microphone module. For example, a separate machine learning model can be used for prediction for each microphone module. The one or more processors can therefore be designed to carry out the selection of the audio excerpts separately for the recordings of the respective microphone module.

Through the proposed concept, i.e. noise classification through a machine learning model, it is possible to reduce the amount of data stored, which can save large amounts of storage space. The one or more processors are designed to store the selected audio excerpts on a storage device 16. If each audio recording includes or represents a single sound, storing the selected audio excerpts may include storing the respective audio recording (or a portion thereof that includes the noise ). If each audio recording includes multiple sounds, storing the selected sounds may include extracting the selected audio excerpts from the one or more audio recordings. The storage device 16 can be part of the vehicle 100, part of the measuring device, and/or part of the device 10. In some systems, the measuring system/the measuring device already makes a preselection of the relevant noises. However, this only limits the amount of data slightly. Thanks to its high prediction accuracy and speed in classification, the machine learning model can decide shortly after the measurement whether it is a relevant noise or not. In addition, non-relevant noises can be deleted immediately. Thus, the one or more processors may be configured to discard (i.e., delete or not extract and store from the audio recordings) portions of the one or more audio recordings that were not selected. For this purpose, the machine learning model is implemented in the vehicle, for example in the measuring system/device. This allows the amount of data accumulated to be limited to 5-10%. This can also mean that the associated memories have to be read out less frequently, meaning that the test vehicles remain operational for longer.

Brake noise analysis is an integral part of vehicle endurance testing. Therefore, the present concept can be used in any vehicle project to reduce the manual effort in brake noise analysis. The concept described can also be applied to other applications of noise classification in which, in addition to the audio recordings, other measured values are recorded by the measuring system or other systems (such as the car in this case).

The at least one interface 12 may, for example, correspond to one or more inputs and/or one or more outputs for receiving and/or transmitting information, for example in digital bit values, based on a code, within a module, between modules, or between modules of different entities .

For example, the one or more processors 14 may correspond to any controller or processor or programmable hardware component. For example, the one or more processors 14 can also be implemented as software that is programmed for a corresponding hardware component. In this respect, the one or more processors 14 can be implemented as programmable hardware with correspondingly adapted software. Any processors, such as digital signal processors (DSPs), can be used. Embodiments are not limited to a specific type of processor. Any processor or even several processors are conceivable for implementation.

The storage device 16 can, for example, be at least one element of the group of computer-readable storage medium, magnetic storage medium, optical storage medium, hard drive, flash memory, floppy disk, random access memory, programmable read only memory (PROM), erasable programmable read only Memory (EPROM), Electronically Erasable Programmable Read Only Memory (EEPROM), and network storage include.

In at least some embodiments, the vehicle 100 may correspond, for example, to a land vehicle, a watercraft, an aircraft, a rail vehicle, a road vehicle, a car, an off-road vehicle, a motor vehicle, or a truck.

More details and aspects of the device 10, the vehicle 100, the storage device 16, the machine learning model, the method and the computer program are mentioned in connection with the concept or examples that come before or after (e.g. 2a and 3 ) to be discribed. The device 10, the vehicle 100, the storage device 16, the machine learning model, the method, and the computer program may include one or more additional optional features that correspond to one or more aspects of the proposed concept or the described examples, as described before or after were described.

2a shows a flowchart of a method for training a machine learning model to make a prediction about the occurrence of brake squeal, such as the machine learning model used by the device 10, the method and computer program of 1a until 1c is used. The method includes obtaining 210 training input data and training output data for training the machine learning model. The training input data includes state information about a condition of the vehicle and the training output data includes information about an occurrence of brake squealing for a given condition of the vehicle. The method further includes training 220 the machine learning model based on the training input data and based on the training output data. Training includes creating a decision tree.

2 B shows a block diagram of a corresponding device 20 for training a machine learning model to make a prediction about the occurrence of brake squeal. The device 20 includes one or more interfaces 22 and one or more processors 24. Optionally, the device 20 further includes one or more storage devices (not shown). The one or more processors 24 are designed to provide the functionality of the device 20, in connection with the one or more interfaces (for exchanging information) and / or with the optional one or more storage devices (for storing information). The device 20 is designed to carry out the method of 2a .

The features of the method, the device 20 and a corresponding computer program are described below with reference to the method of 2a explained. Features that are introduced in connection with the method can also be incorporated into the corresponding device 20 and the corresponding computer program.

While the subject is in the 1 a until 1c concerned with the application of a machine learning model, the subject of the 2a and 2 B with training such a machine learning model. As stated previously, machine learning models are trained using training input data, and in this case also training output data. In the present case, for example, so-called supervised learning is used, or more precisely decision tree learning as a special form of supervised learning. This uses training input data and training output data. The machine learning model, and in particular a decision tree of the machine learning model, is iteratively adjusted so that the desired output is output given the respective sets of training input data. For example, techniques such as gradient descent can be used to reduce or minimize the deviation of the achieved output from the training output data.

In this case, training involves generating (i.e., creating and iteratively adjusting) a decision tree. In particular, training can include generating a classification tree or a regression tree. The nodes (and edges) of the decision tree are selected and adjusted in such a way that the deviation of the achieved output from the training output data, for given training input data, is reduced or minimized. The goal is to create a decision tree that predicts the value of a target variable based on the input data. The value of the target variable is, for example, a classification (i.e. a discrete value that represents a classification, such as “brake squeaking” or “no brake squeaking”) or a value on a scale (such as a probability of brake squeaking). In the latter case, the machine learning model is trained to output information about a probability of the occurrence of brake squeal based on the condition information, and optionally based on metadata about one or more audio recordings.

In addition to the training output data, the training is based on the training input data. These include those related to the 1a until 1c explained status information. Additionally, the training input data may further include the previously introduced metadata about the one or more audio recordings. This state information and (optionally) metadata is applied as an input vector to the machine learning model, and the machine learning model, and in particular the decision tree, is then modified to achieve the desired training output date. If this is achieved for all training input data and corresponding training output data, then the training is successful. Part of the training input data and training output data can be used for the actual training, and part can be retained and used to validate the machine learning model.

The at least one interface 22 may, for example, correspond to one or more inputs and/or one or more outputs for receiving and/or transmitting information, for example in digital bit values, based on a code, within a module, between modules, or between modules of different entities .

For example, the one or more processors 24 may correspond to any controller or processor or programmable hardware component. For example, the one or more processors 24 can also be implemented as software that is programmed for a corresponding hardware component. In this respect, the one or more processors 24 can be implemented as programmable hardware with correspondingly adapted software. Any processors, such as digital signal processors (DSPs), can be used. Embodiments are not limited to a specific type of processor. Any processor or even several processors are conceivable for implementation.

The one or more storage devices can, for example, be at least one element of the group of computer-readable storage medium, magnetic storage medium, optical storage medium, hard drive, flash memory, floppy disk, random access memory, programmable read only memory (PROM), erasable programmable Read Only Memory (EPROM), Electronically Erasable Programmable Read Only Memory (EEPROM), and network storage include.

More details and aspects of the method, the device 20 and the computer program are mentioned in connection with the concept or examples that come before or after (e.g. 1a until 1c , 3 ) to be discribed. The method, apparatus 20, and computer program may include one or more additional optional features corresponding to one or more aspects of the proposed concept or described examples as previously or subsequently described.

3 shows a schematic diagram of a section of a decision tree. Such a decision tree is generally created automatically (e.g. through the 2a and 2 B training shown) and is therefore, on the one hand, comprehensible, because the nodes and edges of the tree are known. On the other hand, it is unknown how the training algorithm arrives at the respective tree structure.

In 3 Only a small section of a decision tree is shown. The tree has other nodes (and especially leaves) and edges, as shown by the dashed arrows. Several decision trees can also be used side by side and cover different parts of the input vector. The decision tree includes three different components - nodes in which a decision between two cases is represented (e.g. KBI_odometer reading ≤ 1518,000), leaf nodes that represent a decision that has been made, and edges that are spanned between the nodes and which are marked "yes". or “No” the path from the nodes in which a decision between two cases is represented and the subsequent node. In 3 The relevant criteria for the decision tree are both status information (“KBI_Kilometerstand” as information about the mileage of the vehicle or information about the service life of the vehicle’s brakes, “ESP_Bremsdruck_Min” as information about the brake pressure of the braking process, “v_calculated” as information about the speed of the vehicle vehicle, “LWI” as information about the steering angle; correlation factors or information about the outside temperature or the brake temperature can also be used) as well as metadata (“Level” as information about the volume, “Tf_HL_1” and “Tf_VL_1” respectively as information about a frequency spectrum).

More details and aspects of the decision tree are mentioned in connection with the concept or examples that come before or after (e.g. 1a until 1c 2b) are described. The decision tree can have one or more additional may include optional features that correspond to one or more aspects of the proposed concept or the described examples, as described before or after.

The aspects and features described in connection with a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the feature into the further example .

Examples may further include or relate to a (computer) program with program code for executing one or more of the above methods when the program is executed on a computer, a processor or other programmable hardware component. Steps, operations or processes of various of the methods described above can also be carried out by programmed computers, processors or other programmable hardware components. Examples may also include program storage devices, e.g. B. digital data storage media, which are machine-, processor- or computer-readable and encode or contain machine-executable, processor-executable or computer-executable programs and instructions. The program storage devices can e.g. B. include or be digital storage, magnetic storage media such as magnetic disks and magnetic tapes, hard disk drives or optically readable digital data storage media. Further examples may include computers, processors, control units, field-programmable logic arrays ((F)PLAs = (Field) Programmable Logic Arrays), field-programmable gate arrays ((F)PGA = (Field) Programmable Gate Arrays), Graphics processors (GPU = Graphics Processor Unit), application-specific integrated circuits (ASIC = application-specific integrated circuit), integrated circuits (IC = Integrated Circuit) or single-chip systems (SoC = System-on-a-Chip). are programmed to perform the steps of the procedures described above.

It is further understood that the disclosure of several steps, processes, operations or functions disclosed in the description or the claims should not be interpreted as necessarily being in the order described, unless this is explicitly stated in the individual case or is absolutely necessary for technical reasons . Therefore, the foregoing description does not limit the performance of multiple steps or functions to a particular order. Further, in further examples, a single step, function, process, or operation may include and/or be broken down into multiple substeps, functions, processes, or operations.

If some aspects have been described in the previous sections in connection with a device or a system, these aspects should also be understood as a description of the corresponding method. For example, a block, a device or a functional aspect of the device or system can correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects that are described in connection with a method are also to be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.

The following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate example. Furthermore, it should be noted that, although a dependent claim refers in the claims to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly suggested, unless it is stated in individual cases that a particular combination is not intended. Furthermore, features of a claim for any other independent claim are also intended to be included, even if that claim is not directly defined as dependent on that other independent claim.

Reference symbol list

10

contraption

12

interface

14

processor

16

storage device

100

vehicle

102

Measuring device

102a

Microphone module

104

State data source

110

Obtaining one or more audio recordings

120

Obtaining state information

130

Processing the status information through a machine learning model

140

Selecting audio excerpts

150

Save the selected audio excerpts

210

Obtaining training input data and training output data

220

Training a machine learning model

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• CN 109684742 A [0003]

Claims (15)

Device (10) for selecting audio excerpts of brake squealing noises in a vehicle (100), the device comprising: at least one interface (12) for communication with one or more microphone modules (102a) of a measuring device (102) and with one or more status data sources (104) of the vehicle; and one or more processors (14), designed to: Receive one or more audio recordings from the one or more microphone modules, Obtaining condition information about the vehicle from the one or more vehicle condition data sources, Processing the condition information by a machine learning model, the machine learning model being trained to make a prediction about the occurrence of brake squeal based on the condition information, selecting, based on an output of the machine learning model, audio excerpts from the one or more Audio recordings, and Save the selected audio excerpts on a storage device (16). The device according to Claim 1 , wherein the one or more processors are configured to discard portions of the one or more audio recordings that were not selected. The device according to one of Claims 1 or 2 , wherein the one or more processors are further configured to determine metadata about the one or more audio recordings. The device according to Claim 3 , wherein the machine learning model is trained to further predict the occurrence of brake squeal based on the metadata about the one or more audio recordings. The device according to Claim 3 , wherein the metadata about the one or more audio recordings includes information about a volume and / or information about a frequency spectrum of the respective audio recording. The device according to one of Claims 3 until 5 , wherein the status information is synchronized in time with the one or more audio recordings of the one or more microphone modules, and wherein the one or more processors are designed to base the times for which the prediction of the occurrence of brake squeal is determined using the machine learning model based on the metadata about the one or more audio recordings. The device according to one of Claims 1 until 6 , wherein the one or more processors are configured to obtain a plurality of audio recordings from a plurality of microphone modules, the machine learning model being trained to predict the occurrence of brake squeal for each microphone module separately, and the one or several processors are designed to carry out the selection of the audio excerpts separately for the recordings of the respective microphone module. The device according to one of Claims 1 until 7 , wherein the status information is one or more pieces of information from the group of information about a speed of the vehicle, information about a mileage of the vehicle, information about a service life of the brakes of the vehicle, information about a brake pressure of a braking process, information about a steering angle, information about an outside temperature and information about a temperature of the brakes. The device according to one of Claims 1 until 8th , wherein the machine learning model includes a decision tree generated by training the machine learning model. The device according to one of Claims 1 until 9 , wherein the machine learning model is trained to output information about a probability of brake squealing occurring based on the condition information, and optionally based on metadata about the one or more audio recordings, wherein the selection of the audio excerpts is based on the probability of the Brake squealing occurs. A method for selecting audio recordings of brake squeal noises in a vehicle, the method comprising: obtaining (110) one or more audio recordings from one or more microphone modules of the vehicle; Obtaining (120) status information about the vehicle from one or more vehicle status data sources; processing (130) the condition information by a machine learning model, the machine learning model being trained to make a prediction about the occurrence of brake squeal based on the condition information; selecting (140), based on an output of the machine learning model, audio excerpts from the one or more audio recordings; and Storing (150) the selected audio excerpts on a storage device within the vehicle. A method for training a machine learning model to make a prediction about the occurrence of brake squeal, the method comprising: obtaining (210) training input data and training output data for training the machine learning model, the training input data comprising state information about a state of the vehicle and the training output data including information about an occurrence of brake squeal at a given state of the vehicle; and Training (220) the machine learning model based on the training input data and based on the training output data, the training comprising generating a decision tree. The procedure according to Claim 12 , wherein the training input data further includes metadata about one or more audio recordings. The procedure according to one of the Claims 12 or 13 , wherein the machine learning model is trained to output information about a probability of brake squeal occurrence based on the condition information, and optionally based on metadata about one or more audio recordings. Program with a program code for carrying out the method according to Claim 11 or the procedure according to one of the Claims 12 until 14 , when the program code is executed on a computer, a processor, a control module or a programmable hardware component.

Images