CN115600667A - A method and device for modeling multi-dimensional state change time series data of a system - Google Patents

Abstract

The invention discloses a method and a device for modeling multi-dimensional state change time sequence data of a system, wherein a model can learn the change of two time states of the system through the design of a unit time forward change generator (G unit), the mode of integrally learning a time sequence track by a traditional method is converted into the mode of changing and learning the specific time point of the time sequence track, and the problem that the time sequence data with the residual point cannot be learned is solved; the method has the advantages that the system state change rule in unit time is learned through the model, the system state change rule is essentially a model of a system working mechanism, and a new system working mechanism can be superposed on the model to predict what kind of change of system behavior can be generated by changing the existing system working mechanism.

Description

A method and method for modeling time series data of system multi-dimensional state changes device

technical field

The invention belongs to the field of computing methods, and relates to a method and a device for modeling time series data of system multi-dimensional state changes.

Background technique

Establishing a method of establishing a neural network model for multi-dimensional data time series, realizing the learning of the driving mechanism of the state change of the system per unit time, and using the neural network model to simulate the time series change of the internal mechanism of the system is an important aspect of time series analysis in the field of artificial intelligence. Target.

For example, time-series data modeling and prediction of bio-fermentation process is an important technology for efficient bio-manufacturing. The time-series data of the fermentation process refers to the sequence data with time-point labels collected at various time points during the microbial fermentation process. Each time point includes a set of fixed indicators. Common indicators include metabolome data, fermentation process data, transcriptome data. The purpose of time-series data modeling and forecasting in the fermentation process is to explore the fermentation law through simulation, optimize the fermentation process, and obtain better economic and social benefits.

At present, the main method for analyzing the fermentation process is the metabolic flux model, which is not an analysis method for the time series trajectory of the fermentation process, but an analysis method for the possible equilibrium state of the fermentation process. This analysis method relies on partial differential equations to describe chemical reactions, so it has certain limitations. The main reason is that it cannot realize the cross-category joint analysis of metabolome data, fermentation process data, and transcriptome data, and its guidance ability for fermentation process optimization is limited.

On the other hand, regression algorithms based on mathematical statistics and machine learning theory are used in many fields to model various time series data. The mathematical statistics method first designs a reasonable functional framework based on the modeler's understanding of the state change laws of the modeled system; then learns the functional parameters to complete the curve fitting of the multidimensional time series. Commonly used methods of this type include periodic factor method, moving average method, ARIMA model and so on. Because the functional framework designed by the modeler is different from the mechanism that actually drives the state change of the modeled system, the model established by this type of method has an upper limit of accuracy at the system level, and it cannot be broken through additional experimental observation data. Another strategy is to use machine learning methods to complete the curve fitting of multidimensional time series based on general frameworks such as neural networks. The machine learning method solves the problem that the mathematical statistics method requires an accurate design of a fitting functional framework by designing a functional framework that has a good fitting ability for all changing laws. Such methods are commonly used K-nearest neighbor algorithm, SVM algorithm, LSTM model, Seq2seq model, etc. Since general functionals are more difficult to learn their parameters than specially designed functionals, such methods usually require a large number of training examples to achieve good fitting results

Whether it is mathematical statistics or machine learning methods, the state change curve of a batch of modeled systems is usually used as a learning example when it is applied, and it is required that the state indicators in all learning examples cannot be missing. In practical applications, many tracking observations of the state of the modeled system cannot achieve no missing indicators and no missing time points due to sample detection problems. For example, in the time series data modeling of the biological fermentation process, fewer time points are often sampled and observed in the time period with little change, and more time points are sampled and observed in the time period with large change. The sampling frequency in a large time period is used as the benchmark, which is equivalent to the lack of data at some time points in a time period with little change; for the characterization of the system state at a time point in the fermentation process, metabolomes, transcriptomes and other groups are often used. Omics technology can obtain many indicators at one time, but due to technical reasons, it cannot be guaranteed that each indicator obtained by each measurement can have sufficient accuracy for subsequent modeling, resulting in the point-in-time state data of the modeled system There are missing indicators in . Such incomplete time series data cannot be directly learned.

For the lack of time-point state data of the modeled system, if the time-point state data is supplemented by interpolation, the training data will be inaccurate, resulting in a decrease in model accuracy. If the entire batch of time series is used as a learning example, the data size of a single example will become larger, the model will become more complex, the test cost of obtaining examples will increase, and the number of examples will decrease, making it difficult for model learning.

On the other hand, the above-mentioned time series modeling method is a curve fitting method instead of modeling the generation mechanism of the time series, so it cannot provide clues on how to optimize the generation mechanism of the time series. For example, the time-series data of the biological fermentation process is driven by the biological mechanism of the engineered bacteria, and the fermentation process model obtained by the time series modeling method of curve fitting does not directly correspond to the biological mechanism of the engineered bacteria, and it is difficult to guide the transformation of the genome of the engineered bacteria .

The invention discloses that in the biological fermentation system, the neural network model can be used to model the working mechanism of a fermentation system. The neural network model of the working mechanism of the fermentation system can calculate the system state after a unit of time based on the current system state. Compared with the time series numerical fitting model in the general sense, this driving mechanism model for the time series change of the biological fermentation system can better reflect the influence of various regulatory mechanisms of engineering bacteria on the fermentation process, and better guide the process of fermentation. Genome modification of engineering bacteria to obtain better production performance.

Therefore, the current multidimensional time series modeling methods have the following main deficiencies:

1) Time series data with incomplete time points cannot be learned;

2) Using a batch of time series as a learning case makes the data of a single case large, leading to high model complexity; at the same time, the number of cases available for learning is small, which makes model learning difficult;

3) Most time series modeling methods cannot directly correspond to the generation mechanism of time series, and cannot provide clues on how to optimize the generation mechanism of time series.

At present, there is a lack of a general method that can fully realize the multi-dimensional time series data for different missing data, and realize the learning and simulation of the driving mechanism of the state change of the system per unit time. The invention discloses a method of using a neural network model to model the generation mechanism of time series, which can use time series data with incomplete time points without interpolation or taking the time series of the whole batch as an example to make up for the above-mentioned multi-dimensional time series Insufficient modeling methods.

Contents of the invention

The invention discloses a method and device for modeling the multi-dimensional state change time series data of the system. Specifically, the invention is realized through the following technical solutions:

A method for modeling time series data of multidimensional state changes of a system, comprising:

1) Standardize the original observation data to obtain standardized observation data in a unified form;

2) Organize training examples for artificial neural network training based on canonical observation data;

3) Design the structure of the artificial neural network and establish the artificial neural network model;

4) Utilize the training example in step 2) to train the artificial neural network model established in step 3), obtain the parameter matrix of artificial neural network;

5) Using the training example in step 2), evaluate the parameters used in the process of designing the artificial neural network structure and the parameters used in the process of training the established artificial neural network model to the accuracy of the artificial neural network model obtained Influence, select the optimal artificial neural network model under different parameter combinations as the final result model;

The neural network structure of the design described in step 3) has the following characteristics:

Its basic structure is a neural network unit, called the forward change generator per unit time (G unit), whose input layer and output layer have the same dimension; this neural network unit is used to model multi-dimensional observation after a unit time data changes;

The serial training structure is obtained by connecting the G units in series, and the serial training structure is used to model the change rule of the multi-dimensional observation data after multiple unit times.

As a further improvement, step 1) of the present invention is specifically: Obtain multiple batches of multi-dimensional time-series observation data through multiple batches of observations, each batch of observation data includes a group of time points, each The observation data at the time point includes a set of indicators, and the observation data of each indicator is a specific value; the multi-dimensional time series observation data is organized into a four-tuple organization form, that is, batch, time, indicator, and value.

As a further improvement, step 2) of the present invention is specifically: organize the state changes of the modeled system at any two time points in the same batch as a training example, and obtain a set of training examples with different time intervals , where each training example includes the state of the modeled system at two time points, the data at the earlier time point is called the starting time point data of the training example; the data at the later time point is called the training example The end time point data of ; the start time point data and end time point data of each training example are expressed as the organizational form of observation data quadruples.

As a further improvement, the forward change generator (G unit) of the present invention adopts a fully connected structure.

As a further improvement, the tandem training structure of the present invention is established as follows: for training examples of different time intervals, according to the number of time intervals, for each time interval, denoted as n unit times, a G unit is established A serial training structure composed of n G units connected end to end in series.

As a further improvement, the tandem training structure with n time intervals in the present invention is called (G) ⁿ , when n=1, (G) ¹ only contains 1 G unit, and the tandem training structure (G) ¹ The corresponding training data is a training example with a time interval of 1; when n>1, (G) ⁿ is a series training structure obtained by connecting n G units in series, and the training data corresponding to the series training structure (G) ⁿ is a time interval The number of training examples is n.

As a further improvement, the training method of step 4) of the present invention is specifically:

在T_i+n时间点的状态数据为

把T_i时间点的数据

输入到串联训练结构(G)ⁿ，得到网络的输出数据

基于网络输出数据

与T_i+n时间点的真实数据

可计算得模型损失值Loss，公式为

4.1 Calculate the loss value of each serial training structure: for each serial training structure (G) ⁿ , record one of its corresponding training examples as S _i , record the starting time point of S _i as T _i , and the time point of S _i The end time point is T _i+n , record the state data of the modeled system at the time point T _i as

The state data at T _i+n time point is

Put the data at time point T _i

Input to the serial training structure (G) ⁿ to get the output data of the network

Output data based on the network

and the real data at T _i+n time point

The model loss value Loss can be calculated, the formula is

4.2 On the basis of the model loss value, the gradient calculation of the number of backpropagation layers that limits the loss is performed: for a series training structure with a time interval of 1, which only contains 1 G unit, the network uses error backpropagation training Mechanism, directly use the error backpropagation method to obtain the update gradient of the G unit; for the series training structure with the number of time intervals greater than 1, which is composed of multiple G units in series, use the error backpropagation method to calculate the gradient, and intercept the last The gradient on a G unit is used as the update gradient of the G unit of the serial training structure;

4.3 On the basis of the gradient calculation method for limiting the number of loss backpropagation layers, update the weights of multiple serial training structures in sequence, and share the network weight parameters in the process of updating the sequential weights. The specific implementation It is: for the serial training structure of different time intervals, calculate the gradient of the serial training structure according to a certain order, and use the gradient descent method to update the weight value, and the updated weight parameters of a serial training structure are immediately shared to all other serial training structure;

4.4 For K serial training structures (G) ¹ , (G) ² , (G) ³ ... (G) ^K , calculate the loss value of each serial training structure, and judge whether the loss value of each serial training structure is Convergence, if all have converged, the result model will be obtained, otherwise, continue to repeat steps 4.1, 4.2, 4.3 until the loss values of each serial training structure converge, and the result model will be obtained.

As a further improvement, step 5) of the present invention specifically refers to optimizing the modeling process from the level of model hyperparameters, and trying to adjust the hyperparameters used in the remodeling process during the optimization process, specifically including: G unit Structural parameters, including the number of nodes in each layer of the network, the number of hidden layers, the learning rate of the gradient descent method, the number of training data examples for each training, and the number of cycles of cyclic training; using different combinations of the above hyperparameters to complete the establishment of the neural network structure and Neural network training work to obtain a new result model; evaluate the fitting accuracy of the result model to the observed data under each hyperparameter combination, and select the optimal model.

The present invention also discloses a device for modeling multidimensional data time series using a method for modeling multidimensional data time series, including:

Acquisition unit: used to standardize and organize the original observation data to obtain standardized observation data in a unified form;

Organizational unit: used to organize training examples based on canonical observation data for artificial neural network training;

Building unit: used to design the structure of the artificial neural network and establish the artificial neural network model;

Training unit: used to train the established artificial neural network model using training examples to obtain the parameter matrix of the artificial neural network;

Optimization unit: for using training examples to evaluate the influence of parameters used in the process of designing the artificial neural network structure and the parameters used in the process of training the established artificial neural network model on the accuracy of the obtained artificial neural network model, Select the optimal artificial neural network model under different parameter combinations as the final result model;

The neural network structure of described design has following characteristics:

Its basic structure is a neural network unit, called the forward change generator per unit time (G unit), whose input layer and output layer have the same dimension; this neural network unit is used to model multi-dimensional observation after a unit time data changes;

The serial training structure is obtained by connecting the G units in series, and the serial training structure is used to model the change rule of the multi-dimensional observation data after multiple unit times.

The beneficial effects of the present invention are as follows:

1) Through the design of the positive change generator (G unit) per unit time, the model can learn the changes of the two time states of the system, and convert the traditional method of overall learning of the timing trajectory into a specific time point change of the timing trajectory The learning method solves the problem that time series data with defects cannot be learned;

2) Due to the change of the training method, multiple learning examples can be organized through a batch of time series data, which increases the number of examples and improves the information efficiency of the training data;

3) Learning the law of system state changes per unit time through the model, which is essentially a model of the system working mechanism, on which a new system working mechanism can be superimposed to predict changes to the existing system working mechanism that may cause changes in system behavior what a change.

4) The structure and training method of the neural network model of the present invention can be applied in the analysis of biological fermentation time series data.

Description of drawings

Fig. 1 is a schematic diagram of model training steps;

Figure 2 is an example diagram of the organization of training examples;

Fig. 3 is a schematic diagram of a serial training structure composed of multiple G units;

Figure 4 is a schematic diagram of loss-restricted backpropagation for the serial training structure.

detailed description

The positive change generator per unit time (G unit) designed by the present invention can be used to model the overall mechanism of the fermentation biological system, including the mechanism of bacterial physiology, catalytic reaction, corresponding process and nutritional condition regulation. Through the analysis of this mechanism model, possible schemes for mechanism optimization can be obtained to guide the optimization of the fermentation process, including the prediction of the yield of the target product, the genome modification of engineering bacteria and the design of fermentation conditions. Fig. 1 is a schematic diagram of model training steps; the method disclosed in the present invention models the overall mechanism of the fermentation biological system through the following steps.

1. Organize the time series data format into a quaternary organization form

The object of modeling analysis is "biological fermentation system". The biological fermentation system includes strains and fermentation environment, so the model needs to be able to learn and predict indicators reflecting the state of strains and fermentation environment. The strain status and fermentation environment can be characterized by a variety of omics techniques, and the fermentation process is a time series composed of multiple time points; therefore, these fermentation time series data can be organized into such a quaternary organizational form: batch, time, index ,value. The fermentation time series data is divided into training data and test data. The training data is used to build the model and the test data is used to evaluate the accuracy of the model.

Assuming that the training data is the time-series data measured during the fermentation of actinomycetes with acarbose as the target product, there are (M+U) batches (such as M=7, U=3), and each batch has (K+1) time points (eg K=10, each time point is respectively marked as T ₁ , T ₂ , T ₃ . . . T ₁₁ ).

For the samples of (K+1) time points corresponding to each batch of (M+U) batches of actinomycete fermentation process, the abundance of various compounds in each sample was determined by mass spectrometry, and the selected Among them, there are q compounds related to the synthesis of acarbose (for example, q=196), and the abundance of these q compounds is extracted from the compound abundance report of the sample at each time point, as the state of the biological fermentation system at this time point Observed values, the abundance of compounds not observed by mass spectrometry were recorded as missing. Recorded in this batch of samples, the T ₃ time point of the second batch and the T ₂ time point of the third batch were missing due to the failure of the sample determination experiment.

Divide (M+U) batches of actinomycete fermentation data into training data and test data, wherein M batches are training data batches, and U batches are test data batches. Therefore, there are (M+U) batches of all data, each batch has (K+1) time points (T ₁ , T ₂ , T ₃ ... T _K+1 ), each time point has q dimension Index data value, where the q-dimensional data of the p-th time point in the m-th batch is expressed as

2. Organize training data examples

Figure 2 is an example diagram of the organization of training examples; M batches of actinomycete fermentation data are used as training data, and according to the quaternary organization form in step 1, the system state changes at any two time points in the same batch are organized As an example, T _a and T _b represent the start time point and end time point of the example respectively, and the example training data is organized as:

Batch 1 example set D ₁ :

Batch 2 example set D ₂ :

Batch 3 example set D ₃ :

Batch m example set D _m :

where 4≤m≤M-1, m∈N ^*

Batch M example set D _M :

Then summarize the examples in each batch, and reorganize them according to the interval between the two time points T _a and T _b in the example:

3. Establish a serial training structure of G units for training examples at different time intervals

Figure 3 is a schematic diagram of a series training structure composed of multiple G units; first build the network of G units, G units are composed of an input layer, a hidden layer, and an output layer, wherein the number of nodes in each layer is e, and the number of hidden layers is f.

For the training examples of different time intervals obtained in step 2, connect the G units in series according to the number of time intervals to obtain the corresponding series training structure: (G) ¹ , (G) ² , (G) ³ ...( G) ^K , where each G unit corresponds to a set of network weight parameters (for convenience, the network weight parameters and bias parameters are collectively referred to as network weight parameters), and each serial training structure parameter is specifically:

权值参数有

(G) ¹ G unit in 1:

The weight parameter has

权值参数有

There are 2 G cells in (G) ² :

The weight parameter has

权值参数有

There are 3 G units in (G) ³ :

The weight parameter has

权值参数有

(G) There are n G units in ^K :

The weight parameter has

Set the initial weight parameters of each serial training structure to the same random value: W ₀ , namely:

训练开始前时都等于W₀。

It is equal to W ₀ before the training starts.

4. Calculate the model loss value

Input the multidimensional data of the starting time point T _a of group c training examples into the serial training structure corresponding to the number of time intervals obtained in step 3 to calculate the loss value. coming soon

输入(G)ⁿ中，得到输出数据

其中n≤K、n∈N^*。

与T_b时间点多维真实数据

可计算得损失值Loss_n，计算公式为

The multidimensional data of c starting time points T _a in

Input (G) ⁿ to get the output data

where n≦K, n∈N ^* .

Multidimensional real data with T _b time point

The loss value Loss _n can be calculated, and the calculation formula is

5. Calculate the gradient through loss-restricted backpropagation, and update the network weight parameters through sequential loop optimization

Figure 4 is a schematic diagram of loss-restricted backpropagation for the serial training structure. From the loss value of each series training structure obtained in step 4, perform restricted back propagation for each series training structure to calculate the gradient and update the network weight parameters through sequential loop optimization. The specific calculation process is as follows:

For the serial training structure with a time interval of 1, which only contains 1 G unit, the network uses the error backpropagation training mechanism to directly obtain the update gradient of the G unit. For the series training structure with the number of time intervals greater than 1, multiple G units are connected in series, and the gradient is calculated by using the error back propagation mechanism, and the gradient on the last G unit is intercepted as the update gradient of the G unit. The specific calculation formula is:

单元的更新梯度，即

(G) ¹ calculation only

The update gradient of the unit, namely

单元的更新梯度，即

(G) ² calculations only

The update gradient of the unit, namely

单元的更新梯度，即

(G) ³ calculations only

The update gradient of the unit, namely

单元的更新梯度，即

(G) ^K counts only

The update gradient of the unit, namely

The above gradient calculation method only calculates the gradient of the last structure G unit of the serial training structure each time, so the present invention refers to this method as loss-limited backpropagation.

On the basis of the gradient calculated by the above method, for the series training structure of different time intervals, according to the order of the number of time intervals, first calculate the gradient of the series training structure of smaller time intervals, and use the SGD stochastic gradient descent method to update the weight , the updated weights are shared to all other serial training modules, and the present invention refers to this training method as sequential cycle optimization. Set the number of cycles parameter in the cycle training as H, where the t-th cycle training process is:

单元的更新权值，即

其中，α表示SGD梯度下降方法的学习率；

表示第t-1次循环训练训练结束时(G)¹中

单元更新后的权值参数，

表示第t次循环(G)¹中

单元更新后的权值参数；将

单元更新后的梯度

权值参数共享到其它串联训练结构中即：First calculate (G) ¹

The update weight of the unit, namely

Among them, α represents the learning rate of the SGD gradient descent method;

Indicates that at the end of the t-1 cycle training training (G) ¹

The weight parameter after unit update,

Indicates the tth cycle (G) ¹

The updated weight parameters of the unit;

Gradient after cell update

The weight parameters are shared to other serial training structures, namely:

都等于

are equal to

单元的更新权值，即

再将

单元更新后的梯度

权值参数共享到其它所有串联训练结构中。Calculate (G) ²

The update weight of the unit, namely

then

Gradient after cell update

The weight parameters are shared to all other cascade training structures.

单元的更新完权值并共享到其它串联训练结构，本次循环训练结束，进入到下一次循环训练计算，最终要进行H次循环训练计算。Follow the same calculation method to update (G) ³ , (G) ⁴ ... (G) ^K , when this loop training calculates G _K

After the weights of the unit are updated and shared with other serial training structures, this cycle training is over, and the next cycle training calculation is entered, and finally H cycles of training calculations are performed.

6. Get the resulting model after all training structure losses converge

After the loop training in step 5 ends, input all the training examples of the corresponding time intervals for the K serial training structures (G) ¹ , (G) ² , (G) ³ ... (G) ^K according to step 4, The full data loss value for each concatenated training module can be obtained. The specific calculation is:

Calculate the loss value of (G) ¹ , that is,

Compute the loss value of (G) ² , namely

Calculate the loss value of (G) ³ , namely

Calculate the loss value of (G) ^K , namely

Determine whether Loss ₁ , Loss ₂ , Loss ₃ ,...,Loss _K are all convergent, if so, go to the next step, otherwise continue to step 4 and step 5 until Loss ₁ , Loss ₂ , Loss ₃ ,...,Loss _K are all convergence.

7. Adjust the model hyperparameters and optimize the resulting model

Step 6 After completing the training and obtaining the result model, adjust the hyperparameters of the model: the network structure parameters of the G unit (the number of nodes in each layer of the network e, the number of hidden layers f), the learning rate α of the SGD stochastic gradient descent method, and the single cycle training Input the number of training examples c, the number of loops H of the loop training. After adjusting the combination of hyperparameters, repeat steps 4-6 until the network Loss difference converges, and terminate the training.

8. Get the optimal model

After the hyperparameter optimization in step 7, the optimal G unit is obtained, and two sets of parameters of the G unit network structure and network weight are saved. And use the test data to verify the prediction accuracy of the model.

Take the U batches of actinomycete fermentation data described in the first step of the application case as the test data, and each batch has K+1 time points (T ₁ , T ₂ , T ₃ ...T _K+1 ) , there are q indicators at each time point, and the test data is organized as steps 1-2:

中有

共N_l个例子，l≤K、l∈N^*。in,

There are

A total of N _l examples, l≤K, l∈N ^* .

Compute the prediction accuracy of (G) ¹ , i.e.

Compute the prediction accuracy of (G) ² , i.e.

Compute the prediction accuracy of (G) ³ , i.e.

Compute the prediction accuracy of (G) ^K , i.e.

From this, the mechanism model (G unit) of the biological fermentation system is obtained to predict the accuracy expectation when the system state changes in the biological fermentation system at different time intervals.

The present invention also discloses a device for modeling multidimensional data time series using a method for modeling multidimensional data time series, including:

Acquisition unit: used to standardize and organize the original observation data to obtain standardized observation data in a unified form;

Organizational unit: used to organize training examples based on canonical observation data for artificial neural network training;

Building unit: used to design the structure of the artificial neural network and establish the artificial neural network model;

Training unit: used to train the established artificial neural network model using training examples to obtain the parameter matrix of the artificial neural network;

Optimization unit: for using training examples to evaluate the influence of parameters used in the process of designing the artificial neural network structure and the parameters used in the process of training the established artificial neural network model on the accuracy of the obtained artificial neural network model, Select the optimal artificial neural network model under different parameter combinations as the final result model;

The designed neural network structure has the following characteristics:

Its basic structure is a neural network unit, called the forward change generator per unit time (G unit), whose input layer and output layer have the same dimension; this neural network unit is used to model multi-dimensional observation after a unit time data changes;

The serial training structure is obtained by connecting the G units in series, and the serial training structure is used to model the change rule of the multi-dimensional observation data after multiple unit times.

The above is not a limitation to the specific implementation manner of this patent. It should be pointed out that for those skilled in the art, without departing from the essential scope of the present invention, some changes, modifications, additions or substitutions can also be made, and these improvements and modifications should also be considered as part of the present invention. protected range.

Claims (9)

1. A method for modeling multi-dimensional state change time sequence data of a system is characterized in that,

the method comprises the following steps:

1) Carrying out standardized arrangement on the original observation data to obtain standardized observation data with a uniform form;

2) Organizing training examples for artificial neural network training based on normative observation data;

3) Designing the structure of an artificial neural network, and establishing an artificial neural network model;

4) Training the artificial neural network model established in the step 3) by using the training example in the step 2) to obtain a parameter matrix of the artificial neural network;

5) Evaluating the parameters used in the process of designing the artificial neural network structure and the influence of the parameters used in the process of training the established artificial neural network model on the precision of the obtained artificial neural network model by using the training example in the step 2), and selecting the optimal artificial neural network model under different parameter combinations as a final result model;

the neural network structure designed in step 3) has the following characteristics:

the basic structure of the device is a neural network unit, namely a unit time forward change generator (G unit), and an input layer and an output layer of the device have the same dimension; the neural network unit is used for modeling the change of multidimensional observation data after a unit time;

and obtaining a series training structure by adopting a mode of serially connecting the G units, wherein the series training structure is used for modeling the change rule of the multi-dimensional observation data in a plurality of unit times.

2. The method for modeling the multidimensional state change time series data of the system according to claim 1, wherein the step 1) is specifically as follows: the multi-dimensional time series observation data are arranged into a four-tuple organization form, namely, batch, time, indexes and values, the multi-dimensional time series observation data of multiple batches are obtained through observation of the multiple batches, the observation data of each batch comprise a group of time points, the observation data of each time point comprise a group of indexes, and the observation data of each index is a specific value.

3. The method for modeling system multidimensional state change time series data according to claim 1, wherein the step 2) is specifically: organizing system state changes of any two time points in the same batch into a training example to obtain a group of training examples with different time intervals, wherein each training example comprises states of the two time points, and data of an earlier time point is called as starting time point data of the training example; the data of later time points are called as the data of the ending time points of the training example; the start time point data and the end time point data for each training example are represented as an organization of observation data quadruples.

4. The method for modeling system multidimensional state change time series data as recited in claim 1 wherein said forward change generator (G cell) employs a fully connected structure.

5. The method for modeling system multi-dimensional state change time series data according to claim 4,

the series training structure is established as follows: for training examples of different time intervals, recording n unit times for each time interval according to the number of the time intervals, and establishing a series training structure consisting of G units, wherein the series training structure connects the n G units in series end to end.

6. The method of claim 5, wherein the n time intervals of the training structure are in series and are called (G) ⁿ N =1 hour (G) ¹ In the training system, only 1G unit is included, and a series training structure (G) ¹ The corresponding training data is the training example with the time interval number of 1；n>1 hour (G) ⁿ A tandem training architecture for the concatenation of n G units, tandem training architecture (G) ⁿ The corresponding training data is a training example with the time interval number n.

7. The method for modeling the multidimensional state change time series data of the system according to any one of claims 1 to 6, wherein the training mode of the step 4) is specifically as follows:

4.1 Calculate the loss value for each tandem training structure: for each series training structure (G) ⁿ Remember that one of the corresponding training examples is S _i Remember S _i Starting point in time of T _i ，S _i Has an end time point of T _i+n Writing to a modeled System at T _i The state data at the time point is

At T _i+n The state data at the time point is

Handle T _i Data of time point

Input to the series training Structure (G) ⁿ Obtaining network output data

Outputting data based on a network

And Y _i+n Real data of time point

The Loss value Loss of the model can be calculated by the formula

4.2 Gradient calculation of the number of counterpropagating layers limiting the loss is performed on the basis of the model loss value: for the series training structure with the time interval number of 1, the network only comprises 1G unit, and the network obtains the updating gradient of the G unit by directly adopting an error back propagation method by using an error back propagation training mechanism; for the series training structure with the time interval number larger than 1, which is formed by connecting a plurality of G units in series, calculating the gradient by adopting an error back propagation method, and intercepting the gradient on the last G unit as the updating gradient of the G unit of the series training structure;

4.3 Series training structure order weight update: on the basis of the training mechanism for limiting the number of the loss reverse propagation layers, updating the sequence weights of a plurality of series training structures, and sharing the network weight parameters in the sequence weight updating process; the concrete implementation is as follows: calculating the gradient of the series training structures at different time intervals according to a certain sequence, updating the weight by using a gradient descent method, and sharing the updated weight to all other series training structures;

4.4 For K series training structures (G) ¹ 、(G) ² 、(G) ³ ...(G) ^K Calculating the loss value of each series training module, judging whether the loss value of each series training structure is converged, if so, obtaining a result model, otherwise, continuously repeating 4.1), 4.2) and 4.3) until the loss value of each series training structure is converged, and obtaining the result model.

8. The method according to claim 7, wherein the step 5) is specifically optimized from a modeling process of model hyper-parameters, and the hyper-parameters adjusted in the modeling process are: structural parameters of the G unit (the number of nodes of each layer of the network and the number of hidden layers); the learning rate of the gradient descent method; training data example number is input in each training; the number of cycles of the cycle training; adopting different combinations of the hyper-parameters, and completing the work of establishing a neural network structure and training the neural network according to the step 3) and the step 4) to obtain a new result model; and evaluating the fitting precision of the result model to the observation data under each hyper-parameter combination, and selecting an optimal model.

9. An apparatus for modeling multi-dimensional state change time series data of a system, comprising:

an acquisition unit: the system is used for carrying out standardized arrangement on the original observation data to obtain standardized observation data with a uniform form;

organization unit: training examples for organizing the normative-based observation data for artificial neural network training;

a construction unit: the method comprises the steps of designing the structure of the artificial neural network, and establishing an artificial neural network model;

a training unit: the artificial neural network model is used for training the established artificial neural network model by utilizing a training example to obtain a parameter matrix of the artificial neural network;

an optimization unit: the method is used for evaluating the precision influence of parameters used in the process of designing the artificial neural network structure and the process of training the established artificial neural network model on the obtained artificial neural network model by utilizing a training example, and selecting the optimal artificial neural network model under different parameter combinations as a final result model;

the designed neural network structure has the following characteristics:

the basic structure of the device is a neural network unit, namely a unit time forward change generator (G unit), and an input layer and an output layer of the device have the same dimension; the neural network unit is used for modeling the change of multidimensional observation data after a unit time;

and obtaining a series training structure by adopting a mode of serially connecting the G units, wherein the series training structure is used for modeling the change rule of the multidimensional observation data in a plurality of unit times.

Images