DE202019006054U1 - Emulated signal processing means and signal processing unit - Google Patents

Abstract

Signal processing unit (300) comprising:- at least one emulated signal processing means (200), the emulated signal processing means comprising a trained neural network system (210), the trained neural network system (210) comprising an input layer (112), at least one hidden layer (114) and an output layer (116), the neural network system (210) being trained on a processor unit (120) with a set of training data (140), the training data (140) being sampled data obtained from a signal processing means (10) to be emulated, the sampled data being obtained at discrete time steps n from: output signal values (yn) as output by the signal processing means (10) to be emulated, and at least one internal state signal value (xn) of an internal state of the signal processing means (10) to be emulated; wherein an input to the neural network system (110) during training is provided by the value of the at least one internal state signal value (xn) at time step n, and an output of the neural network system (10) (a) in case the output signal of the signal processing means (10) is equal to one of the at least one internal state signal, is as follows: a value representing the at least one internal state signal value (xn+1) of the signal processing means (10) at time step n+1, (b) in all other cases: a value representing the at least one internal state signal value (xn+1) of the signal processing means (100) at time step n+1, and the output signal values (yn) of the signal processing means (10);- a storage unit (240) for storing the parameters (111) of the trained neural network system,- an internal sound source (301) configured to provide an input to the trained neural network system (210). and to input an internal state signal of an internal state of the emulated signal processing means (200); and- an output unit (305) configured to receive an output signal of the trained neural network system (210) and to output this output signal.

Description

FIELD OF INVENTION

The present invention generally relates to an emulated signal processing means and a signal processing unit comprising at least one emulated signal processing means.

BACKGROUND OF THE INVENTION

Machine learning, particularly using deep learning neural networks, is used to perform a wide range of tasks based on training mathematical models on data. Because these machine learning structures, algorithms, and related techniques can solve complex tasks without relying on explicit instructions from a human programmer, they are considered a form of artificial intelligence.

Virtual analog modeling in digital signal processing, such as audio signal generation and processing, is a well-established yet specialized field that deals with reproducing the behavior of an originally analog system in digital form. For the purposes of the application context, virtual analog modeling is a suitable subfield of digital signal processing to demonstrate the use of the proposed invention.

In some areas of the state of the art, virtual analog modeling relies on an engineer's understanding of an electrical or electromechanical system to derive equations and convert those equations into code as explicit instructions. There are different approaches to how these explicit models are structured and how processing procedures are optimized. These procedures require a large amount of manual work and expertise, both in the modeling technique and in the individual original system.

EN 102015110938 A1 and US2016328501 A1 reveal automated methods that require less manual work and individual knowledge of the original system, but are limited to modeling a limited class of original systems to a limited level of accuracy.

In the new scientific state of the art of E. Damskägg, L. Juvela, E. Thuillier and V. Välimäki, “Deep Learning for Tube Amplifier Emulation”, 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 2019, pp. 471-475 , highly generalized machine learning techniques, such as convolutional neural networks, have been applied to model the input-output behavior of an original system without any consideration or knowledge of the original system's internal mechanics. Such systems are computationally intensive and, depending on the original system, may be excessively difficult to train using iterative machine learning.

For typical real-time processing applications, a continuous stream of discrete audio data is passed through such a process with a latency of only milliseconds between input and output, creating high demands on the performance, i.e., computational efficiency, of any virtual analog processor.

There is a continuous development in the field of virtual analog modeling to improve digital signal processing techniques in the areas of computational efficiency, achievable complexity of the original system, and the ability to model highly nonlinear or autonomous behavior. In producing such solutions on a commercial scale, the state of the art is not able to meet the above criteria with a reasonable investment of manual effort for many of the original systems in question.

SUMMARY OF THE INVENTION

An object of the invention is to improve the state of the art of digital signal processing techniques by using machine learning for a new data structure and a new computational method, thereby providing higher fidelity processing in reproducing an original system with lower processing requirements and also with reduced manual effort to generate individual models for different original systems.

According to one aspect of the present invention, there is provided a signal processing unit comprising: an emulated signal processing means obtained by the following method for emulating a signal processing means, wherein the emulated signal processing means comprises a trained neural network system; an internal sound source configured to be an input to the trained neural network system; and an output unit configured to receive an output signal of the trained neural network system and to output this output signal.

• Erhalten von abgetasteten Daten in diskreten Zeitschritten n von:
  • Ausgangssignalwerten, wie durch ein zu emulierendes Signalverarbeitungsmittel ausgegeben, und
  • mindestens einem internen Zustandssignalwert eines internen Zustands des zu emulierenden Signalverarbeitungsmittels;
• Training eines neuronalen Netzwerksystems auf einer Prozessoreinheit mit einem Satz von Trainingsdaten, wobei die Trainingsdaten die erhaltenen abgetasteten Daten des zu emulierenden Signalverarbeitungsmittels sind, wobei das neuronale Netzwerksystem eine Eingabeschicht, mindestens eine verborgene Schicht und eine Ausgabeschicht umfasst, wobei eine Eingabe in das neuronale Netzwerksystem während des Trainings durch den Wert des mindestens einen internen Zustandssignalwerts bei einem Zeitschritt n bereitgestellt ist und eine Ausgabe des neuronalen Netzwerksystems wie folgt lautet: im Falle, dass das Ausgangssignal des Signalverarbeitungsmittels gleich einem des mindestens einen internen Zustandssignals ist: ein Wert, der den mindestens einen internen Zustandssignalwert des Signalverarbeitungsmittels zum Zeitschritt n+1 darstellt, in allen anderen Fällen: ein Wert, der den mindestens einen internen Zustandssignalwert des Signalverarbeitungsmittels zum Zeitschritt n+1 darstellt, und die Ausgangssignalwerte des Signalverarbeitungsmittels; und
• Speichern der Parameter des trainierten neuronalen Netzwerksystems in einer Speichereinheit.

The method for emulating a signal processing means comprises the following steps:

• Obtaining sampled data in discrete time steps n of:
  • Output signal values as output by a signal processing means to be emulated, and
  • at least one internal state signal value of an internal state of the signal processing means to be emulated;
• Training a neural network system on a processor unit with a set of training data, the training data being the obtained sampled data of the signal processing means to be emulated, the neural network system comprising an input layer, at least one hidden layer and an output layer, an input to the neural network system during training being provided by the value of the at least one internal state signal value at a time step n and an output of the neural network system being as follows: in case the output signal of the signal processing means is equal to one of the at least one internal state signal: a value representing the at least one internal state signal value of the signal processing means at time step n+1, in all other cases: a value representing the at least one internal state signal value of the signal processing means at time step n+1, and the output signal values of the signal processing means; and
• Storing the parameters of the trained neural network system in a storage unit.

This present invention presents a new method comprising a neural network structure embedded in a discrete-time state space system. This structure can be trained to approximate the behavior of a number of signal processing means using not only input and output measurements, but also measurements of sampled data sets within the signal processing means.

The state space or phase space description of a dynamical system is well known in the field. It is a mathematical geometric space that describes all possible configurations of states of a dynamical system, which is a mathematical system where a function describes the evolution of a point in an abstract geometric space. The state dynamics is a function that defines the evolution of a set of states in time or in discrete time steps over iteration.

wobei x die Zustände des Systems sind, y die Ausgaben sind, u die Eingaben sind, dx/dt die zeitliche Ableitung der Zustände ist und f_a eine Funktion ist, die die Dynamik des Systems definiert.The signal processing means to be emulated in this invention can be described in continuous time using a mathematical model of the following form: $$\begin{array}{c}\dot{x}\left(t\right)\\ y\left(t\right)\end{array}=e_a\left(\begin{array}{c}u\left(t\right)\\ x\left(t\right)\end{array}\right)$$

where x are the states of the system, y are the outputs, u are the inputs, dx/dt is the time derivative of the states, and f _a is a function that defines the dynamics of the system.

wobei u_n, y_n, und x_n Abtastungen der zeitkontinuierlichen Eingaben, Ausgaben und Zustände sind und f_d eine unbekannte Funktion ist.Theoretically, there is an analog discrete-time system that perfectly reproduces the values of the continuous-time system at the discrete time steps of a sampling method: $$\begin{array}{c}{x}_{n+1}\\ y_n\end{array}=e_d\left(\begin{array}{c}{u}_n\\ x_n\end{array}\right)$$

where u _n , y _n , and x _n are samples of the continuous-time inputs, outputs, and states, and f _d is an unknown function.

Signalverarbeitungsmittel, wie etwa ein Verstärker, zeigen häufig ein nichtlineares Verhalten, wobei eine nichtlineare Funktion eine ist, auf die das Überlagerungsprinzip nicht zutrifft. Das neuronale Netzwerk ist dazu trainiert, dieses nichtlineare Verhalten zu imitieren.The neural network is trained to approximate f _d using a new function f^ _d and thus approximate the dynamics of the system: $$\begin{array}{c}{x}_{n+1}\\ y_n\end{array}={\widehat{e}}_d\left(\begin{array}{c}{u}_n\\ x_n\end{array}\right)$$

Signal processing devices, such as an amplifier, often exhibit nonlinear behavior, where a nonlinear function is one to which the superposition principle does not apply. The neural network is trained to imitate this nonlinear behavior.

Throughout the description, the signal processing means to be emulated may be a continuous-time physical system or a discrete-time reference system. Training data is generated from a continuous-time physical system by sampling digital signal values at discrete-time steps. The signal processing system to be emulated may be: a digital or an analog signal processing means, an effects processor, a digital or an analog electrical circuit, or an electromechanical circuit. In its simplest form, it may be an oscillator circuit that does not require any further input signal to generate an oscillating output signal.

Preferably, the method further comprises the step of obtaining sampled input signal values u _n which are provided to the signal processing means to be emulated, wherein the training data set comprises the sampled input signal values u _n , wherein the input signal values u _n are also input to the neural network system at time step n. These input values can also be audio signals.

The method may further comprise the step of applying a control parameter value c _n of the signal processing means as a further input to the neural network system. Such a control parameter may be the input of a controller provided on the signal processing means to be emulated, for example its gain setting. Further control parameters may be signal frequency filter controls, dynamic compression controls, saturation controls, signal offset or polarity.

The method may also comprise the step of providing state indication values s _n of a state of the signal processing means to be emulated as a further output of the neural network system. Such a state indication may be an LED that is turned on on the signal processing device to be emulated. A further state indication may include a mean value of the moving average signal of a volume meter or a binary state of a switching component.

Preferably, the neural network system is trained such that an increment between two consecutive states x _n and x _n+1 is the value representing the at least one internal state signal, wherein the increment is summed with a previous internal state signal value x _n to obtain the next internal state signal value x _n+1 . The use of the small difference between the state values during the temporal evolution of the dynamics can advantageously improve the training process of the neural network system.

A method according to any preceding claim, wherein the signal processing system to be emulated is: a digital or analog signal processing means, an effects processor, a digital or analog electrical circuit or an electromechanical circuit.

The stored parameters of the neural network system may include: the values of trained weights of the neural network system and/or the number of hidden layers and the number of nodes of the input layer, each hidden layer and the output layer of the neural network system. If applicable, the time step of the training data may also be stored.

Further, an emulated signal processing means is provided which is obtained by the method for emulating a signal processing means described above.

• Bereitstellen eines emulierten Signalverarbeitungsmittels, das durch das Verfahren zum Emulieren eines Signalverarbeitungsmittels, wie vorstehend beschrieben, erhalten wird, wobei das emulierte Signalverarbeitungsmittel ein trainiertes neuronales Netzwerksystem umfasst;
• Anwenden eines internen Zustandswerts x_n zum Zeitschritt n als ein Eingangssignal in das trainierte neuronale Netzwerksystem, wobei zum Zeitschritt n=0 der interne Zustandswert x_o ein initialisierter Wert ist, und zu allen anderen Zeitschritten ist der interne Zustandswert x_n eine vorherige Ausgabe des trainierten neuronalen Netzwerksystems, das durch eine Rückkopplungsschleife aus der Ausgabeschicht des neuronalen Netzwerksystems an die Eingabeschicht des neuronalen Netzwerksystems erhalten wurde,
• Empfangen, als ein Ausgangssignal des trainierten neuronalen Netzwerksystems, eines internen Zustandssignalwerts x_n+1 des Zeitschritts n+1, und wenn das Ausgangssignal des zu emulierenden Signalverarbeitungsmittels kein internes Zustandssignal davon ist, eines Ausgangssignalwerts y_n des Zeitschritts n,
• Ausgeben des internen Zustandssignalwerts oder, wenn das Ausgangssignal des zu emulierenden Signalverarbeitungsmittels kein internes Zustandssignal davon ist, des Ausgangssignalwerts y_n des Zeitschritts n als einen Ausgangssignalwert z_n des emulierten Signalverarbeitungsmittels.

Furthermore, a method for processing a signal is provided, comprising the following steps:

• Providing an emulated signal processing means obtained by the method for emulating a signal processing means as described above, wherein the emulated signal processing means comprises a trained neural network system;
• Applying an internal state value x _n at time step n as an input signal to the trained neural network system, where at time step n=0 the internal state value x _o is an initialized value, and at all other time steps the internal state value x _n is a previous output of the trained neural network system obtained by a feedback loop from the output layer of the neural network system to the input layer of the neural network system,
• Receiving, as an output signal of the trained neural network system, an internal state signal value x _n+1 of the time step n+1, and if the output signal of the signal processing means to be emulated is not an internal state signal thereof, an output signal value y _n of the time step n,
• Outputting the internal state signal value or, if the output signal of the signal processing means to be emulated is not an internal state signal thereof, the output signal value y _n of the time step n as an output signal value z _n of the emulated signal processing means.

The method may further comprise the step of applying a discrete input signal value u _n , at time step n as input to the trained neural network system.

The method may further comprise the step of applying a control parameter value c _n as further input to the trained neural network.

The method may further comprise the step of receiving, as a further output to the trained neural network, a state indication value s _n .

In case the neural network system is trained such that an increment between two consecutive states x _n and x _n+1 is the value representing the at least one internal state signal, wherein the increment is summed with a previous internal state signal value x _n to obtain the next internal state signal value x _n+1 , the increment may be multiplied by a gain factor, wherein the gain factor is defined as a fraction of a desired time step used to process a signal divided by a time step used to train the neural network system.

Furthermore, a signal processing unit is provided which comprises at least one emulated signal processing means obtained by the method for emulating a signal processing means described above, wherein the emulated signal processing means comprises a trained neural network system. Furthermore, the following is provided: an internal sound source which is configured to be an input to the trained neural network system; and an output unit which is configured to receive an output signal of the trained neural network system and to output this output signal.

The unit may further comprise: a control input interface configured to receive an external control signal, the control input interface providing additional input to the trained neural network system, and/or an internal control source configured to generate an internal control signal providing additional input to the trained neural network system.

The unit may further comprise a memory for storing the parameters of the at least one emulated signal processing means.

The unit may further comprise a plurality of emulated signal processing means and a selection unit for selecting at least one of the provided emulated signal processing means. This allows the user to select a desired emulated signal processing means for his use.

Also provided is a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to perform the method for emulating a signal processing means as described above and/or the method for processing a signal described above.

• Bereitstellen eines emulierten Signalverarbeitungsmittels, das durch das Verfahren erhalten wurde, das die folgenden Schritte umfasst:
  • Erhalten von abgetasteten Daten zu diskreten Zeitschritten n von Ausgangssignalwerten, wie durch ein zu emulierendes Signalverarbeitungsmittel ausgegeben, und von mindestens einem internen Zustandssignalwert eines internen Zustands des zu emulierenden Signalverarbeitungsmittels;

In one aspect, a method of processing a signal is provided, comprising the steps of:

• Providing an emulated signal processing means obtained by the method comprising the following steps:
  • Obtaining sampled data at discrete time steps n of output signal values as output by a signal processing means to be emulated and of at least one internal state signal value of an internal state of the signal processing means to be emulated;

• Speichern der Parameter des trainierten neuronalen Netzwerksystems in einer Speichereinheit;
• Anwenden eines internen Zustandswerts zum Zeitschritt n als ein Eingangssignal in das trainierte neuronale Netzwerksystem, wobei zum Zeitschritt n=0 der interne Zustandswert ein initialisierter Wert ist, und zu allen anderen Zeitschritten der interne Zustandswert eine vorherige Ausgabe des trainierten neuronalen Netzwerksystems ist, die durch eine Rückkopplungsschleife aus der Ausgabeschicht des neuronalen Netzwerksystems an die Eingabeschicht des neuronalen Netzwerksystems erhalten wurde,
• Empfangen, als ein Ausgangssignal des trainierten neuronalen Netzwerksystems, eines internen Zustandssignalwerts des Zeitschritts n+1, und wenn das Ausgangssignal des zu emulierenden Signalverarbeitungsmittels kein internes Zustandssignal davon ist, eines Ausgangssignalwerts des Zeitschritts n, und
• Ausgeben des internen Zustandssignalwerts oder, wenn das Ausgangssignal des zu emulierenden Signalverarbeitungsmittels kein internes Zustandssignal davon ist, des Ausgangssignalwerts des Zeitschritts n als einen Ausgangssignalwert des emulierten Signalverarbeitungsmittels.

Training a neural network system on a processor unit with a set of training data, the training data being the obtained sampled data of the signal processing means to be emulated, the neural network system comprising an input layer, at least one hidden layer and an output layer, an input to the neural network system during training being provided by the value of the at least one internal state signal value at a time step n and an output of the neural network system being as follows: (a) in case the output signal of the signal processing means is equal to one of the at least one internal state signal: a value representing the at least one internal state signal value of the signal processing means at time step n+1, or (b) in all other cases: a value representing the at least one internal state signal value of the signal processing means at time step n+1, and the output signal values of the signal processing means; and

• Storing the parameters of the trained neural network system in a storage unit;
• Applying an internal state value at time step n as an input signal to the trained neural network system, wherein at time step n=0 the internal state value is an initialized value, and at all other time steps the internal state value is a previous output of the trained neural network system obtained by a feedback loop from the output layer of the neural network system to the input layer of the neural network system,
• Receiving, as an output signal of the trained neural network system, an internal state signal value of the time step n+1, and if the output signal of the signal processing means to be emulated is not an internal state signal thereof, an output signal value of the time step n, and
• Outputting the internal state signal value or, if the output signal of the signal processing means to be emulated is not an internal state signal thereof, the output signal value of the time step n as an output signal value of the emulated signal processing means.

The step of providing an emulated signal processing means may further comprise the step of applying sampled discrete input signal values at time step n as an input to the neural network system, and wherein the step of applying further comprises applying sampled discrete input signal values at time step n as an input to the trained neural network system.

The step of providing an emulated signal processing means may further comprise the step of applying sampled control parameter values of the signal processing means as a further input to the neural network system, and wherein the applying step further comprises applying, as a further input to the trained neural network system, the sampled control parameter values of a control parameter.

The step of providing an emulated signal processing means may further comprise the step of receiving, as a further output of the trained neural network system, state indication values, and wherein the applying step further comprises the step of receiving, as a further output of the trained neural network system, state indication values.

When the neural network system is trained such that an increment between two consecutive internal state signal values at time step n and time step n+1 is the value representing the at least one internal state signal, the increment is summed with a previous internal state signal value to obtain the next internal state signal value. The increment is preferably multiplied by a gain factor, wherein the gain factor is defined as a fraction of a desired time step used to process a signal divided by a time step used to obtain the sampled data for training the neural network system.

The signal processing system to be emulated may be: a digital or analog signal processing means, an effects processor, a digital or analog electrical circuit, or an electromechanical circuit.

The stored parameters of the neural network system may be at least one selected from the group consisting of: the values of trained weights of the neural network system, the number of hidden layers and the number of nodes of the input layer, the number of nodes in each hidden layer, and the number of nodes of the output layer of the neural network system.

• Erhalten von abgetasteten Daten zu diskreten Zeitschritten n von Ausgangssignalwerten, wie durch ein zu emulierendes Signalverarbeitungsmittel ausgegeben, und mindestens eines internen Zustandssignalwerts eines internen Zustands des zu emulierenden Signalverarbeitungsmittels;
• Training eines neuronalen Netzwerksystems auf einer Prozessoreinheit mit einem Satz von Trainingsdaten, wobei die Trainingsdaten die erhaltenen abgetasteten Daten des zu emulierenden Signalverarbeitungsmittels sind, wobei das neuronale Netzwerksystem eine Eingabeschicht, mindestens eine verborgene Schicht und eine Ausgabeschicht umfasst, wobei eine Eingabe in das neuronale Netzwerksystem während des Trainings durch den Wert des mindestens einen internen Zustandssignalwerts zu einem Zeitschritt n bereitgestellt ist und eine Ausgabe des neuronalen Netzwerksystems wie folgt lautet: (a) im Falle, dass das Ausgangssignal des Signalverarbeitungsmittels gleich einem des mindestens einen internen Zustandssignals ist: ein Wert, der den mindestens einen internen Zustandssignalwert des Signalverarbeitungsmittels zum Zeitschritt n+1 darstellt, oder (b) in allen anderen Fällen: ein Wert, der den mindestens einen internen Zustandssignalwert des Signalverarbeitungsmittels zum Zeitschritt n+1 darstellt, und die Ausgangssignalwerte des Signalverarbeitungsmittels; und
• Speichern der Parameter des trainierten neuronalen Netzwerksystems in einer Speichereinheit.

Further, an emulated signal processing means is provided, the means being obtained by the following steps:

• Obtaining sampled data at discrete time steps n of output signal values as output by a signal processing means to be emulated and at least one internal state signal value of an internal state of the signal processing means to be emulated;
• Training a neural network system on a processor unit with a set of training data, the training data being the obtained sampled data of the signal processing means to be emulated, the neural network system comprising an input layer, at least one hidden layer and an output layer, an input to the neural network system during training being provided by the value of the at least one internal state signal value at a time step n and an output of the neural network system being as follows: (a) in case the output signal of the signal processing means is equal to one of the at least one internal state signal: a value representing the at least one internal state signal value of the signal processing means at time step n+1, or (b) in all other cases: a value representing the at least one internal state signal value of the signal processing means at time step n+1, and the output signal values of the signal processing means; and
• Storing the parameters of the trained neural network system in a storage unit.

In addition, a signal processing unit may be provided, comprising: at least one emulated signal processing means obtained by the method described above, the emulated signal processing means comprising a trained neural network system; an internal sound source configured to be an input to the trained neural network system; and an output unit configured to receive an output signal of the trained neural network system and to output this output signal.

The unit may further comprise a control input interface configured to receive an external control signal, the control input interface providing additional input to the trained neural network system, and/or an internal control source configured to generate an internal control signal providing additional input to the trained neural network system.

The unit may further comprise a memory unit configured to store the parameters of the at least one emulated signal processing means.

The unit may further comprise a plurality of sets of stored parameters and a selection unit for selecting at least one of the provided sets of stored parameters to be used in the emulated signal processing means.

There may be provided a computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method of signal processing as described above.

The characteristics, features and advantages of this invention and the manner in which they are obtained as described above will become more apparent and clearly understood in connection with the following description of exemplary embodiments explained with reference to the accompanying drawings.

Short description of the drawings

• 1 veranschaulicht eine erste Ausführungsform eines Signalverarbeitungsmittels 10, das gemäß der vorliegenden Erfindung emuliert werden soll,
• 2 veranschaulicht eine erste Ausführungsform des erfindungsgemäßen Systems 100, das zum Emulieren eines Signalverarbeitungsmittels 10 verwendet wird,
• 3 veranschaulicht eine erste Ausführungsform eines erfindungsgemäßen emulierten Signalverarbeitungsmittels 200,
• 4 veranschaulicht eine zweite Ausführungsform eines erfindungsgemäßen emulierten Signalverarbeitungsmittels 200,
• 5 veranschaulicht eine dritte Ausführungsform eines erfindungsgemäßen emulierten Signalverarbeitungsmittels 200,
• 6 veranschaulicht drei Phasen des Abtastens von Daten, des Trainings des Netzwerks und des Verarbeitens eines Signals,
• 7A veranschaulicht das Abtasten von Daten aus einem ersten Signalverarbeitungsmittel 10,
• 7B veranschaulicht den Zeitbereichsabgleich zwischen dem gemessenen und dem emulierten Ausgangssignal für das erste Signalverarbeitungsmittel 10 von 7A,
• 7C veranschaulicht den Frequenzbereichsabgleich zwischen dem gemessenen und dem emulierten Ausgangssignal für das erste Signalverarbeitungsmittel 10 von 7A,
• 8A veranschaulicht das Abtasten von Daten aus einem zweiten Signalverarbeitungsmittel 10,
• 8B veranschaulicht den Zeitbereichsabgleich zwischen dem gemessenen und dem emulierten Ausgangssignal für das zweite Signalverarbeitungsmittel 10,
• 8C veranschaulicht den Frequenzbereichsabgleich zwischen dem gemessenen und dem emulierten Ausgangssignal für das zweite Signalverarbeitungsmittel 10,
• 9A veranschaulicht das Abtasten von Daten aus einem dritten Signalverarbeitungsmittel 10,
• 9B veranschaulicht die Zeitbereichsansicht zweier periodischer Eingangssignale, „Sägezahn“ und „Gitarre“, für das Signalverarbeitungsmittel 10 von 9A,
• 9C veranschaulicht den Zeitbereichsabgleich zwischen dem gemessenen und dem emulierten Ausgangssignal für das dritte Signalverarbeitungsmittel 10 beim Empfangen des „Sägezahn“-Eingangssignals,
• 9D veranschaulicht den Zeitbereichsabgleich zwischen dem gemessenen und dem emulierten Ausgangssignal für das dritte Signalverarbeitungsmittel 10 beim Empfangen des „Gitarren“-Eingangssignals,
• 9E veranschaulicht den Frequenzbereichsabgleich zwischen dem gemessenen und dem emulierten Ausgangssignal für das dritte Signalverarbeitungsmittel 10,
• 10 veranschaulicht eine erste Audioanwendung 300 für das neuronale Netzwerksystem 210,
• 11 veranschaulicht eine zweite Audioanwendung 300 für das neuronale Netzwerksystem 210,
• 12 veranschaulicht den Prozess des Speicherns mehrerer neuronaler Netzwerkparameter 111 für mehrere emulierte Signalverarbeitungsmittel 200 in einer Speichereinheit.

In the drawings, equal element numbers indicate the same elements in each of the views:

• 1 illustrates a first embodiment of a signal processing means 10 to be emulated according to the present invention,
• 2 illustrates a first embodiment of the system 100 according to the invention, which is used to emulate a signal processing means 10,
• 3 illustrates a first embodiment of an emulated signal processing means 200 according to the invention,
• 4 illustrates a second embodiment of an emulated signal processing means 200 according to the invention,
• 5 illustrates a third embodiment of an emulated signal processing means 200 according to the invention,
• 6 illustrates three phases of sampling data, training the network and processing a signal,
• 7A illustrates the sampling of data from a first signal processing means 10,
• 7B illustrates the time domain alignment between the measured and the emulated output signal for the first signal processing means 10 of 7A ,
• 7C illustrates the frequency range adjustment between the measured and the emulated output signal for the first signal processing means 10 of 7A ,
• 8A illustrates the sampling of data from a second signal processing means 10,
• 8B illustrates the time domain alignment between the measured and the emulated output signal for the second signal processing means 10,
• 8C illustrates the frequency range adjustment between the measured and the emulated output signal for the second signal processing means 10,
• 9A illustrates the sampling of data from a third signal processing means 10,
• 9B illustrates the time domain view of two periodic input signals, “sawtooth” and “guitar”, for the signal processing means 10 of 9A ,
• 9C illustrates the time domain alignment between the measured and the emulated output signal for the third signal processing means 10 when receiving the “sawtooth” input signal,
• 9D illustrates the time domain alignment between the measured and the emulated output signal for the third signal processing means 10 when receiving the “guitar” input signal,
• 9E illustrates the frequency range adjustment between the measured and the emulated output signal for the third signal processing means 10,
• 10 illustrates a first audio application 300 for the neural network system 210,
• 11 illustrates a second audio application 300 for the neural network system 210,
• 12 illustrates the process of storing multiple neural network parameters 111 for multiple emulated signal processing means 200 in a storage unit.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

1 shows schematically an example of a signal processing means 10 to be emulated according to the present invention. In the following, the signal processing means to be emulated may also be referred to as the original system. In general, the signal processing means 10 to be emulated may be any hardware or software system that provides an output signal, for example a digital or an analog signal processing device, a digital or an analog electrical circuit, an electromechanical circuit, an effects processor, an amplifier, etc. In one of its simplest versions, the signal processing means 10 could be an oscillating circuit as in 1 shown. A rudimentary RC oscillator uses resistors 11 and a capacitor 12 together with an amplifier 13 to produce a sine wave output signal y(t) at node 15 in the absence of any incoming signal. The voltage of the capacitor 12 is the internal state signal x(t). The regenerative feedback of the circuit is related to the ability of the capacitor to store an electrical charge. In more complex cases, a signal input u(t) may be provided to the signal processing means which is subsequently processed in the signal processing means to derive an output signal y(t). The input signal u(t) may be an audio input signal and the output signal y(t) may be a processed audio signal provided to a loudspeaker or other device.

wobei u(t) der Vektor von Eingaben in das System ist, y(t) der Vektor von Ausgaben ist und x(t) die internen Zustände' des Systems sind. Dies kann auch durch Verketten der internen Zustände mit der Eingabe oder der Ausgabe in Form einer einzelnen Gleichung geschrieben werden: $$\begin{array}{c}\dot{x}\left(t\right)\\ y\left(t\right)\end{array}=f_a\left(\begin{array}{c}u\left(t\right)\\ x\left(t\right)\end{array}\right).$$

In a state system approach, the states are related to the stored energy in a system, i.e., the states are the voltage stored in the capacitors. This voltage is time-varying due to the oscillation of the circuit. The first step of the provided method for emulating a signal processing means is to sample the output signal y(t) and the state signal x(t) at n different time steps, where n is an integer, to obtain the output signal values y _n and x _n for all time steps n. The state space approach, also known as the phase space approach, is a powerful mathematical formalism that can describe any system that can be characterized by a system of ordinary differential equations. In continuous time, it can be written as: $$\begin{array}{l}\dot{x}\left(t\right)=e\left(u\left(t\right),x\left(t\right)\right)\\ y\left(t\right)=G\left(u\left(t\right),x\left(t\right)\right)\end{array}$$

where u(t) is the vector of inputs to the system, y(t) is the vector of outputs, and x(t) are the internal states of the system. This can also be written in the form of a single equation by concatenating the internal states with the input or the output: $$\begin{array}{c}\dot{x}\left(t\right)\\ y\left(t\right)\end{array}=e_a\left(\begin{array}{c}u\left(t\right)\\ x\left(t\right)\end{array}\right).$$

wodurch das Verhalten des Systems basierend auf seinen Eingaben und vorherigen Zuständen iterativ berechnet werden kann.The function f _a therefore completely describes the behavior of the system. Note that in cases where the state of a system is taken directly as the output, there is no need for any y-variable. The discrete-time analogue assuming constant time steps, where n = τ · t, where τ represents the sampling interval, is given by: $$\begin{array}{c}{x}_{n+1}\\ y_n\end{array}=e_d\left(\begin{array}{c}{u}_n\\ x_n\end{array}\right)$$

which allows the behavior of the system to be iteratively calculated based on its inputs and previous states.

In the present invention, access to the signal processing means to be emulated not only allows one to sample the input and output signals u(t) and y(t) of the system to be emulated, but allows one to obtain access to the signal x(t) corresponding to the states of the system. If the system is fed with test input signals u(t) and the state and output signals x(t) and y(t) are each sampled at discrete times, a large set of data points can be obtained which describe the input and output relationship of the function f _d . To replicate the behavior of the system for an arbitrary input, this data can be used to generate a new function f^ _d which approximates f _d to a suitable degree of accuracy.

This is essentially a regression task and could be approached via any number of standard techniques. However, it is well suited to the application of an artificial neural network (ANN), which are known as universal approximators, i.e. they are able to approximate arbitrarily closely to any continuous function of N real variables. The system can then be simulated iteratively as usual, but using the approximated function f^ _d rather than an f _d derived analytically by discretization. The system can then be written as follows: $$\begin{array}{c}{x}_{n+1}\\ y_n\end{array}={\widehat{e}}_d\left(\begin{array}{c}{u}_n\\ x_n\end{array}\right).$$

The choice of which data sets to take as the states of the system is not as strict as it may first seem. While the real states of the system are the data sets that store energy and thus information, e.g. capacitors, other data sets in the system are related to these states by a mapping that is linear at best but probably not linear. This means that as long as we have sufficient measured data sets compared to the number of states of the system, we should be able to discern its dynamics. The formal upper bound on the number of independent measurements needed is given by Whitney's embedding theorem as 2m+1, where m is the number of states. Furthermore, Taken's theorem allows us to replace some or all of these measurements with time-delayed versions of another measurement. In practice, the dynamics of this transformed state space can be more complicated and thus it is preferable to use measurements that are as close as possible to the real states of the system.

The formulation of the problem in state space terms is advantageous because it means that the function f _d that needs to be approximated is a purely static, i.e. memoryless, mapping. The proposed method is different from autoregressive modeling, which predicts the next most frequent scalar output based on a sequence of past observations. Instead of only considering the input-output behavior of the system, the present invention maps from the state space to itself. In dynamic systems terms, this means that the piecewise stream along the trajectory of the system is learned. An optional input signal u can be considered as a parameter that influences this stream.

The core of the neural network system used in the present invention is a Multilayer Perceptron (MLP), i.e. a series of k closely connected layers with an activation function. The number of layers and the layer width are tailored to suit the specific system being modelled, with small systems potentially requiring only small networks. The activation function can be changed to suit the system, but in general saturating nonlinearities such as tanh produces good results. These functions are advantageous because signal processing means to be emulated, such as electronic circuits, generally contain nonlinearities of the saturating type. Simpler activations such as Rectified Linear Units (ReLU) can be used with the caveat that the size of the network must then generally be larger. However, other architectures and structures of the neural network system can also be used.

The proposed neural network system 110 used in the proposed method for emulating a signal processing means 10 of 1 is used in 2 as a system 100 used to emulate a signal processing means 10.

The obtained sampled data of the signal processing means 10 to be emulated at n discrete time steps are used to train the neural network 110, where n is an integer. Specifically, the obtained sampled state values x _n for at least one state of the signal processing means to be emulated and the output signal values y _n of the signal processing means to be emulated are used to train the neural network system 110. In some cases, however, the output signal is identical to an internal state signal. In such a case, the information is redundant and only the state signal can be used. The neural network system 110 consists of an input layer 112 with at least one input node 113 (here 113-1, 113-2), a number of hidden layers 114 with a plurality of nodes of the hidden layers 115 and an output layer 116 with at least one output node 117 (here 117-1, 117-2). In the event that an internal state of the system is also the output of the signal processing means, then only the state signal values are used to train the neural network system 110.

From the set of training data 120, the input layer 112 iteratively receives internal state values x _n for all time steps n. If necessary, it can also receive sampled input signals nal values u _n received in case the signal processing means to be emulated processes an input signal to derive an output signal as discussed above.

The training of neural network systems 110 is well known in the art. During training of a neural network system 110, the parameters 111 of the system, called weights, are changed such that a norm between the actual output of the neural network system 110 and the sampled data is optimized until a certain criterion for stopping training is met. In the present case, the output layer 116 consists of the signal output values y' _n , and the internal state signal values x' _n+1 generated during training, which are compared with the target sampled output signal values y _n of the original system 10 and the target sampled internal state signal values x _n of the original system 10, respectively. A loss function 150 is calculated as a metric of the approximation between the output values y' _n , x' _n+1 of the output layer 116 of the neural network system 110 during training and the sampled outputs y _n , x _n+1 of the original system 10. The optimization of the iterated neural network system 110 is performed by keeping only changes to the parameters 111 of the neural network 20 that yield a lower loss function, where the loss 150 converges to a certain value after a certain number of iterations, which is accepted as a successful approximation based on a predetermined success condition. It is emphasized that throughout the description an original system may have more than one internal state.

It is advantageous that the states x _n and their values at the new time step x _n+1 are likely to be closely related. Therefore, the neural network system 110, as in 2 shown, structured in a residue-based manner using a skip-layer connection before the output layer 116. Skip-layer connections have been successfully applied in various fields. The conclusion is that the network learns a residue of the state signals compared to their previous value rather than the new values directly. This has steadily increased the training speed and accuracy in the case of the systems considered. Therefore, before the output node 117-2, which refers to the next internal state value x _n+1 , the previous states and their increments are summed at element 119 to obtain the next internal state value x _n+1 . In another embodiment, however, the state values themselves may be used for training without a summer 119. Alternatively, only the increments could be passed through a feedback loop and the summing of the increment and the last current state could take place within the next iteration.

After training is stopped, a set of parameters 111 of the neural network system 110 is stored on a storage unit 140. The set of parameters may include: the values of trained weights of the neural network system 110, the number of hidden layers 114, and the number of nodes of the input layer 112, each hidden layer 114, and the output layer 116 of the neural network system 110.

Consequently, the proposed method for emulating a signal processing means 10 comprises the step S100 of obtaining sampled data at n discrete time steps of the signal processing means 10 to be emulated, S110 training a neural network system 110 with those sampled data and S120 storing a set of parameters of the trained neural network system 110. The step S100 of obtaining sampled data comprises obtaining sampled data at discrete time steps n of output signal values y _n as output by the signal processing means 10 to be emulated and at least one internal state signal value x _n of an internal state of the signal processing means 10 to be emulated. The step S110 of training a neural network system 110 comprises training a neural network system 110 on a processor unit 130 (schematically shown in 2 shown) with a set of training data, the training data being the obtained sampled data of the signal processing means 10 to be emulated, the neural network system 110 comprising: an input layer 112, at least one hidden layer 114 and an output layer 116, wherein an input to the neural network system 110 during training is the at least one internal state signal value x _n at time step n and an output of the neural network system 110 is a value representing the at least one internal state signal value x _n+1 of the signal processing device at time step n+1 and the output signal values y _n of the signal processing device. The step of storing S120 a set of parameters 111 comprises the step of storing a set of parameters 111 of the trained neural network system 110 on a storage unit 140.

3 and 4 show embodiments of the emulated signal processing means 200, derived from the method described above for emulating a signal processing device 10. Thus, the training is finished and now the emulated signal processing means 200, comprising a trained neural network system 210, is used to process a signal. As above, the now trained neural network system 210 again consists of an input layer 212 with at least one input node 213, a number of hidden layers 214 with a plurality of nodes of the hidden layers 215 and an output layer 216 with at least one output node 217. An output signal value y _n is generated at each time step n of the method according to the invention depending on the internal process and possibly also depending on an external signal input u _n . Without signal input, the emulated signal processing means 200 operates as an autonomous signal source.

For each time step or iteration in state space, the hidden layers 214 of the trained neural network system 110 receive at least one input through an input layer 212 and produce outputs through an output layer 216, where the inputs include a current state of the system x _n and the output includes both a signal output y _n and an increment that is summed with the current state of the system at summer 219 to produce a next state of the system x _n+1 . As the processing method proceeds to the next iteration, the next state of the system is passed through a feedback loop 218 to the next iteration where it becomes the new current state of the system upon input to the input node 213-2. The emulated signal processing means 200 provides as an output a signal z _n . Typically, the output z _n of the emulated signal processing means 200 is equal to the output of the trained neural network. However, a further processing step could be applied between the output of the trained neural network and the output of the emulated signal processing means 200, so that z _n may depend on y _n but may not be identical.

The emulated signal processing means 200 according to the invention uses the processor unit 230 (not shown) for processing. The emulated signal processing means 200 can be provided in software, in hardware or in a mixture of both. The output of the emulated signal processing means 200 is a physical/electrical signal.

In an alternative embodiment, only the increments could be fed through a feedback loop and the summation of the increment and the last current state could take place within the next iteration.

4 shows an emulated signal processing means 200 of 3 without the input signal u _n . Everything else remains the same as in 3 .

In an audio application, a specific implementation of the first embodiment is used as a virtual analog emulation of a diode clipper circuit. The digital signal processor runs in the audio domain at a sampling frequency of, for example, 44.1 kHz, with the iterative computation process 2 iterated 44100 times per second with evenly spaced time steps between samples. Without an input signal u _{n ,} the iterative computation method could still produce an output signal y _n , e.g. acting as an oscillator source. With an incoming audio signal u _n , e.g. a digitized audio signal from an electric guitar, the digital signal processing method produces an outgoing sample z _n =y _n for each incoming sample u _n , thus continuously processing the audio data stream.

For a second preferred embodiment of the invention, 5 a schematic structure diagram of an emulated signal processing means 200 derived from the method for emulating a signal processing means. The trained neural network system 210 is used in each processing iteration to produce an output signal value z _n . For each time step or iteration of the process, a control parameter value c _n may optionally be received. Optionally, as above, an input signal value u _n may also be received. The input layer 212 of the neural network system 210 receives the optional signal input u _n , the internal state signal values x _n and a control input c _n at the nodes 213-1 to 213-3. These inputs are connected to the nodes 215 of the hidden layer 214, which consists of neural layers each consisting of at least one neural node 215, the nodes of the successive layers being connected. An output layer 216 outputs an optional state output s _n , an output signal value y _n and a state output x _n+1 representing the state of the next iteration to the nodes 217-1, 217-2. As the computational process proceeds to the next iteration, the next state of the means 200 is passed through a feedback loop 219 to the next iteration where it becomes the new current state. The processing iteration produces for each set of input signal values u _n and the control parameter input c _n an output signal value z _n and also a status s statement output that provides information about the dynamic system.

wobei τ_p, τ_t die Zeitschritte sind, die zum Verarbeiten und zum Training verwendet werden, und gleichermaßen sind F_p und F_t die Abtastfrequenzen, die zum Verarbeiten und zum Training verwendet werden.In a variation of the second preferred embodiment, the variation of the sampling frequency of the iterative method is compensated compared to the sampling frequency of an iterative training. Before an increment is summed with the current state of the system to generate a next state of the system x _n+1 , it is scaled by a gain factor τ 250 , $$H_{\tau }=\frac{{\tau }_p}{{\tau }_t}=\frac{F_t}{F_p}$$

where τ _p , τ _t are the time steps used for processing and training, and similarly F _p and F _t are the sampling frequencies used for processing and training.

Caution should be exercised when using this facility to operate the system at a different sampling frequency than that for which it was trained. Even if the time scale is changed correctly, this does not make the system equivalent to one trained at the new sampling frequency. Operating the system at a lower sampling frequency than that used for training increases the likelihood of aliasing introduced by elements of the learned behavior that exceed the new Nyquist limit. Operating the system at a higher sampling frequency than it was trained at poses fewer risks, although the modeled system potentially lacks high frequency behaviors that would have been present had the system been trained at a higher sampling frequency. Oversampling, compared to audio speed, is recommended in most situations because the system does not employ specific anti-aliasing and thus produces aliasing in the same situations as a normal nonlinear state space system.

It is emphasized that in the above second embodiment of the emulated signal processing means, the control parameter input c _n , the input signal u _n and the state output s _n are merely optional. Depending on the signal processing means to be emulated, they can all be used together or independently as inputs or outputs of the emulated signal processing means.

To further illustrate the embodiments described above, 6 a diagram of three phases I, II, III, wherein the first phase I is a sampling process S100 for obtaining sampled data from a signal processing means 10, wherein in a second phase II the sampled data is used in the method for emulating a signal processing means 10 in training S110 a neural network system 110, which is an iterative training process 160, wherein at the end of the training a set of parameters 111 of the neural network system 110 is stored in a storage unit 140 in step S120 and wherein in a third phase these stored parameters 111 of the neural network system 110 define a trained neural network system 210 which is used in a process iteration 260 of a method for signal processing to provide and output a processed signal z(t).

7-9 show three different signal processing means 10 which were emulated according to the present invention and then used to process signals. A measurement signal was applied to the different signal processing means 10 and the state x _n and output signal y _n were recorded using an analog-to-digital converter (ADC). Unitary gain operational amplifier buffers were used to separate the measurement points from the load of the ADC. In some situations, node voltages were recorded instead of differential voltages across the capacitors (i.e. the required states). In these cases, the differential voltage was recovered in post-processing by a linear combination with other recorded node voltages.

In theory, any signal can be used to generate the measurements used to train the neural network system 210. In practice, some important considerations must be made. Unlike other black-box modeling techniques, measurements with varying frequency input, e.g., a sine sweep, are not strictly necessary to capture the behavior of the system in question. This is because the behavior is captured at all time scales in the learned state trajectories. However, the measurement signal chosen has a number of important influences on the process.

The measurement signal defines the portion of the state space that is sampled. In complicated systems, specific inputs could potentially be needed to access specific parts of the state space. A simple example of this could be a resonant filter circuit. In this case, an input signal at the resonant frequency would excite the circuit much more strongly than signals at other frequencies, potentially revealing nonlinear behavior. which only appear at orders of magnitude of high states.

Care must also be taken in the sampling rate used for measurements to limit the upper frequency of the measurement signal to well below the Nyquist frequency. This is because the process of sampling the state signals and outputs is band-limited. Components that exceed the Nyquist are removed during the sampling process. Thus, harmonics generated in the real system may be lost. This means that the state signals recorded in response to too high a frequency no longer reflect the true state-space trajectory. This leads to measurements that appear to violate the constraint that each point in space can correspond to only one trajectory. In practice, the selection of a measurement signal is a balance between these two constraints and can be tailored to the system in question. Typical input signals to the system may work well (e.g. guitar playing when modeling a distortion pedal), as may carefully specified sweeps. In this paper, we used logarithmic sine sweeps of 10 seconds combined with quiet (-20 dB) white noise. The noise was bandlimited to 22 kHz. The amplitude of this combined signal was then linearly increased from zero to unity gain over half the length of the measurement signal to ensure the collection of sufficient data points close to the origin of the state space. The minimum frequency of the sweep was 20 Hz and the maximum frequency was chosen at 10 kHz to cover a sufficient amount of the state space without corrupting the data with fictitious trajectories. A sampling rate of 192 kHz can be used for measurement and training. Training can be done over 300 epochs with a batch size of 256. Due to the small size of the networks considered, the training process does not require a GPU and is actually slower than training on a CPU with SIMD functionality.

As a first example of the approximate dynamics of a signal processing device, 7A an electrical diagram with access points of a measurement. The shown electrical circuit 10, consisting of wired diodes 14, capacitors 12, resistors 11, and ground connectors cooperate as a so-called "diode clipper" which distorts the input signal u(t) to a clipped output signal y(t) depending on the input signal. For small input signals (i.e. peak voltages below approximately 0.6 V) the current flowing through the diodes is close to zero and the circuit reduces to a simple RC low-pass filter. For larger signal values the diodes cause the output to saturate and the circuit becomes a non-linear low-pass filter, with its instantaneous cutoff frequency increasing as the circuit is driven harder.

The original system 10 is contained in a measurement environment and digital values and a voltage signal input, a voltage signal output y(t), which is also a state of the system x(t) since there is only one capacitor and thus one state, are sampled and the value of this state is the output of the system. A buffered gain element 16 is placed in the circuitry to prevent the voltage measurement from disturbing the dynamics of the original system. A neural network with tanh transfer functions and two hidden layers with eight nodes each was used to train and process the data. 7B shows an excellent match for the time domain signal response between the measurement, ie the sampled output signal 41 shown as dashed lines, and the output signal output 31 of the neural network system 20 trained and used in processing as described above. 7C shows the corresponding frequency range adjustment for the same example.

As a second example of an approximate dynamics of a signal processing device, 8A an electrical diagram with access points of a measurement 18. The electrical circuit shown, consisting of wired diodes 14, capacitors 12, resistors 11, and ground connectors cooperate as a so-called "second order audio signal limiter" which distorts the input signal u(t) to a 'limited' output signal depending on the input signal. The original system 10 is contained in a measurement environment and digital values are sampled for a voltage signal input u(t), a voltage signal output y(t), which is also a state of the system x ₂ (t). A second state x ₁ (t) of the system is calculated as a differential state x ₁ = v - x ₂ between the sampled voltage signal output y(t) and a voltage sampled at an access point 18. Buffered gain elements 16 are placed in the circuitry to prevent the voltage measurement from disturbing the dynamics of the original system. A neural network system 210 with 3 hidden layers each having 4 tanh nodes was used to emulate this signal processing means 10. 8B shows a very good match for the time domain signal response between the measurement, ie the sampled output signal, and the respective output signal output from the neural network system 210, which is described above ben trained and used in processing. 8C shows the corresponding frequency range adjustment for the same example.

As a third, more complex example of an approximate dynamics of a signal processing device, 9A an electrical diagram with access points of a measurement of the internal states x ₁ (t) and x ₂ (t) with respect to two capacitors 12. The shown electrical circuit, consisting of wired diodes 14, capacitors 12, resistors 11, as well as ground connectors and integrated circuits cooperate as a low-pass filter. 9B shows two periodic input signals used in the third example, where the “guitar” input signal was also used in the first and second examples. For the third example, 9C the time domain adjustment for a “sawtooth” signal input, 9D shows the time domain alignment for a “guitar” signal input and 9E shows the frequency range adjustment.

To illustrate a context of an audio application 300 for using the preferred embodiments described above, 10 a real-time audio application 300 as an environment for the inventive method for signal processing, in this case a method for digital signal processing, which includes a neural network system 210. The output signal value y _n of the neural network system 210 is connected to an audio output 305 of the real-time audio application 300, where it can be recorded or heard. The output signal values y _n coming from the neural network system 210 and the actual output signal value z _n output by the audio application 300 are generally the same, but they can also be different if they contain a different signal processing step desired by the user of the system. The audio output 305 in this example outputs the output signal value z _n =y _n for all time steps iteratively. An internal sound source 301 is connected to provide a discretized input signal value u _n to the signal processing method and a control parameter input c _n out is also generated within a real-time audio application means 304. An external control source 302 may be connected via a control input interface 303 to control the internal sound source 301. Specifically, for example, a control keyboard 302 is used to play oscillator and amplifier sections 301 of a software audio synthesizer, which are then processed through a virtual analog filter emulation 200 to an external audio output 305, with the filter cutoff frequency modulated by a low frequency oscillator in the audio application 300.

For another illustration of an application context for using the preferred embodiments described above, 11 a real-time audio application 300 as an environment for a digital signal processing method that includes a neural network system 210. The output signal value y _n of the neural network system 210 is connected to an audio output 305 of the real-time audio application 300, where it can be recorded or heard. An external sound source 306 is connected via an audio input interface 307 to provide a discretized input signal value u _n to the digital signal processing method, and a control parameter input c _n is received from an external control source 302 via a control input interface 303. Specifically, a control controller 302 can be used to change the gain setting for a virtual analog guitar amplifier emulation 200 that processes the signal from an electric guitar 306 to an external audio output 305. A state indication 308 is coupled to a state signal output of the neural network system 210 to indicate a state of the emulated signal processing device.

As an extension of the 6 shown phases shows 12 how several signal processing means 10, 10', 10" are scanned each to emulate the respective signal processing means to train a neural network system 110 within a system 100 used to emulate the signal processing means 10, and thereafter different sets of neural network parameters 111 of the neural network system 110 are stored in a storage unit 140. The data may be stored in the storage unit 140 as a database.

In the signal processing unit 300 employing the inventive method for signal processing comprising an emulated signal processing means 200, a selection unit 320 can be used to select a specific trained neural network system 210 from the storage unit 240 that has received its data content from the storage unit 140 of the system used to emulate the signal processing means 10, 10', 10" and use it in the inventive iterative method for signal processing. In an audio application 300, this can be used to select between several emulated signals using the method for digital signal processing as an audio effect in the application 300. nal processing means 200 at a point in the signal chain.

The signal processing unit 300 according to the invention uses a processor unit 330 for processing. The emulated signal processing means 300 can be provided in software, in hardware or in a mixture of both. The output of the emulated signal processing means 300 is a physical/electrical signal.

While this invention has been described in connection with what are presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims.

LIST OF REFERENCE SYMBOLS

10

Signal processing device to be emulated / original system

11

Resistance

12

capacitor

13

amplifier

14

diode

15

node

16

buffered gain

18

Measurement access point

y(t), yn

Output signal of the signal processing device

u(t), un,

Input signal of the signal processing device

x(t), xn

internal status signal

s(t), cn

Status indicator signal

c(t), yn

Control parameters

z(t), zn

Output signal of the emulated signal processing device

100

System used to emulate the signal processing means to be emulated

110

neural network system

111

parameter

112

Input layer

113

Input layer node

114

hidden layers

115

Hidden layer nodes

116

Output layer

117

Output layer node

119

Summer

120

Training data

130

Processor unit

140

Storage unit

150

Loss function

160

iterative training process

200

emulated signal processing device

210

neural network system

212

Input layer

213

Input layer node

214

hidden layers

215

Hidden layer nodes

216

Output layer

217

Output layer node

218

Feedback loop

219

Summer

230

Processor unit

240

Storage unit

250

Gain factor

260

Processing iteration

300

Signal processing unit

301

internal sound source

302

external tax source

303

Control input interface

304

internal tax source

305

Output unit

308

Condition information

320

Selection unit

330

Processor unit

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely to provide the reader with better information. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• DE 102015110938 A1 [0005]
• US 2016328501 A1 [0005]

Cited non-patent literature

• E. Damskägg, L. Juvela, E. Thuillier and V. Välimäki, “Deep Learning for Tube Amplifier Emulation”, 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 2019, pp. 471-475 [0006]

Claims (12)

Signal processing unit (300) comprising: - at least one emulated signal processing means (200), wherein the emulated signal processing means comprises a trained neural network system (210), wherein the trained neural network system (210) comprises an input layer (112), at least one hidden layer (114) and an output layer (116), wherein the neural network system (210) is trained on a processor unit (120) with a set of training data (140), wherein the training data (140) are sampled data obtained from a signal processing means (10) to be emulated, wherein the sampled data are obtained at discrete time steps n from the following: output signal values (y _n ) as output by the signal processing means (10) to be emulated, and at least one internal state signal value (x _n ) of an internal state of the signal processing means (10) to be emulated; wherein an input to the neural network system (110) during training is provided by the value of the at least one internal state signal value (x _n ) at time step n, and an output of the neural network system (10) (a) in case the output signal of the signal processing means (10) is equal to one of the at least one internal state signal, is as follows: a value representing the at least one internal state signal value (x _n+1 ) of the signal processing means (10) at time step n+1, (b) in all other cases: a value representing the at least one internal state signal value (x _n+1 ) of the signal processing means (100) at time step n+1, and the output signal values (y _n ) of the signal processing means (10); - a storage unit (240) for storing the parameters (111) of the trained neural network system, - an internal sound source (301) configured to be an input to the trained neural network system (210) and to input an internal state signal of an internal state of the emulated signal processing means (200); and - an output unit (305) configured to receive an output signal of the trained neural network system (210) and to output this output signal. Signal processing unit (300) according to Claim 1 , further comprising a control input interface (303) configured to receive an external control signal, the control input interface (303) providing an additional input to the trained neural network system, and/or an internal control source (304) configured to generate an internal control signal providing an additional input to the trained neural network system (210). Signal processing unit (300) according to one of the preceding claims, further comprising a plurality of sets of stored parameters (111) stored in a storage unit and a selection unit (320) for selecting at least one of the sets of stored parameters (111) to be used in the emulated signal processing means (200). Signal processing unit (300) according to one of the preceding claims, wherein the neural network system (210) is configured to receive sampled input signal values (u _n ) at time step n as input to the input layer (112). Signal processing unit (300) according to one of the preceding claims, wherein the trained neural network system (210) is configured to receive sampled control parameter values (c _n ) as a further input to the input layer (112). Signal processing unit (300) according to one of the preceding claims, wherein the trained neural network system (210) is configured to provide state indication values (s _n ) of a state of the signal processing means (10) to be emulated as a further output of the neural network system (210). Signal processing unit (300) according to one of the preceding claims, wherein the signal processing means (200) to be emulated is: a digital or an analog signal processing means, an effects processor, a digital or an analog electrical circuit or an electromechanical circuit. Signal processing unit (300) according to one of the preceding claims, wherein the stored parameters (111) of the neural network system (210) is at least one selected from the group consisting of: the values of trained weights of the neural network system (210), the number of hidden layers (114) and the number of nodes (113) of the input layer (112), the number of nodes (115) in each hidden layer (114) and the number of nodes (117) of the output layer (116) of the neural network system (210). Signal processing unit (300) according to one of the preceding claims, wherein the signal processing unit (300) is an audio application (300), wherein the output unit (10) is provided with a recording or storage means or a loudspeaker. Signal processing unit (300) according to one of Claim 9 further comprising real-time audio application means (304) for generating a control parameter as an input to the trained neural network system. Signal processing unit (300) according to one of the Claims 9 or 10 , further comprising a control input interface (303) for connecting to an external control source (302) for controlling the internal sound source (301). Signal processing unit (300) according to one of the preceding claims, comprising a plurality of emulated signal processing means (200) and a selection unit (320) for selecting an emulated signal processing means (200) from the provided plurality of emulated signal processing means.

Images