CN108734338A - Credit risk forecast method and device based on LSTM models - Google Patents

Abstract

The credit risk prediction method based on the LSTM model includes: obtaining user operation behavior data of the target account within a preset time period; the preset time period is a time series composed of several time intervals with the same time step; The user operation behavior data in the time interval generates the user behavior vector sequence corresponding to each time interval; the generated user behavior vector sequence corresponding to each time interval is input to the LSTM in the trained LSTM model based on the encoding-decoding architecture The encoder performs calculations to obtain hidden state vectors corresponding to each time interval; the LSTM model includes an LSTM encoder and an LSTM decoder that introduces an attention mechanism; the hidden state vectors corresponding to each time interval are used as risk features and input to The LSTM decoder performs calculations to obtain the risk score of the target account in the next time interval; and the weight value of each hidden state vector corresponding to the risk score.

Description

Credit risk prediction method and device based on LSTM model

technical field

This specification relates to the communication field, and in particular to a credit risk prediction method and device based on an LSTM model.

Background technique

In the existing credit risk prevention system, credit risk prediction models have been widely used to prevent credit risk. By providing a large number of risk transactions from risk accounts as training samples, and extracting risk features from these risk transactions for training, a credit risk model is constructed, and then the completed credit risk model is used to predict the credit risk of the user's transaction account and Evaluate.

Contents of the invention

This specification proposes a credit risk prediction method based on the LSTM model, which includes:

Obtain user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step;

Based on the user operation behavior data of the target account in each time interval, generate a user behavior vector sequence corresponding to each time interval;

Input the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtain the hidden state vector corresponding to each time interval; wherein, the LSTM The model includes an LSTM encoder and an LSTM decoder with an attention mechanism;

The hidden state vectors corresponding to each time interval are used as risk features, input to the LSTM decoder for calculation, and the risk score of the target account in the next time interval is obtained; and, each hidden state vector corresponds to the risk score The weight value of ; wherein, the weight value represents the contribution of the hidden state vector to the risk score.

Optionally, the method also includes:

Obtain the user operation behavior data of several sample accounts marked with risk tags within the preset time period;

Based on the user operation behavior data of the several sample accounts in each time interval, generate a user behavior vector sequence corresponding to each time interval;

The generated user behavior vector sequence is used as a training sample to train the LSTM model based on the encoder-decoder architecture.

Optionally, based on the user operation behavior data of the account in each time interval, generate a user behavior vector sequence corresponding to each time interval, including:

Obtain various user operation behavior data of the account in various time intervals;

extracting key factors from the acquired user operation behavior data, and digitizing the key factors to obtain a user behavior vector corresponding to the user operation behavior data;

The user behavior vectors corresponding to various user operation behavior data in each time interval are spliced to generate a sequence of user behavior vectors corresponding to each time interval.

Optionally, the various user behaviors include credit performance behavior, user consumption behavior, financial management payment behavior;

The key factors include the loan order status and loan repayment amount corresponding to the credit performance behavior, the user consumption category and the number of user consumption transactions corresponding to the user consumption behavior, the financial management payment type and the financial management income amount corresponding to the financial management payment behavior.

Optionally, the LSTM encoder adopts a multi-layer many-to-one structure; the LSTM decoder adopts a multi-layer many-to-many structure with symmetrical numbers of input nodes and output nodes.

Optionally, the generated user behavior vector sequence corresponding to each time interval is input to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and the hidden state vector corresponding to each time interval is obtained ,include:

Input the generated user behavior vector sequence corresponding to each time interval into the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture to perform two-way propagation calculation, and obtain the first hidden state vector obtained by the forward propagation calculation; and , the second hidden state vector obtained by backpropagation calculation; wherein, when performing forward propagation calculation and backpropagation calculation, the input order of the user behavior vector sequence corresponding to each time interval is reversed;

Perform splicing processing on the first hidden state vector and the second hidden state vector to obtain final hidden state vectors corresponding to each time interval.

Optionally, the hidden state vectors corresponding to each time interval are used as risk features, input to the LSTM decoder for calculation, and the risk score of the target account in the next time interval is obtained, including:

The hidden state vector corresponding to each time interval is used as the risk feature, input to the LSTM decoder for calculation, and the output vector of the target account in the next time interval is obtained;

The output vector is digitized to obtain the risk score of the target account in the next time interval.

Optionally, the output vector is a multidimensional vector;

The digital processing of the output vector includes any of the following:

Extracting the value of the sub-vector whose value is between 0 and 1 in the output vector as the risk score;

If the output vector contains multiple sub-vectors with values between 0 and 1, calculate the average value of the values of the multiple sub-vectors as the risk score;

If the output vector contains multiple sub-vectors with values between 0 and 1, extract the maximum or minimum value among the values of the multiple sub-vectors as the risk score.

This specification also proposes a credit risk forecasting device based on the LSTM model, the device comprising:

The acquisition module acquires user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step;

A generation module, based on the user operation behavior data of the target account in each time interval, generates a sequence of user behavior vectors corresponding to each time interval;

The first calculation module inputs the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtains the hidden state vector corresponding to each time interval; Wherein, the LSTM model includes an LSTM encoder, and an LSTM decoder that introduces an attention mechanism;

The second calculation module is to input the hidden state vectors corresponding to each time interval as risk features into the LSTM decoder for calculation, and obtain the risk score of the target account in the next time interval; and each hidden state vector corresponds to A weight value of the risk score; wherein, the weight value represents the contribution of the hidden state vector to the risk score.

Optionally, the acquisition module further:

Obtain the user operation behavior data of several sample accounts marked with risk tags within the preset time period;

The generating module further:

Based on the user operation behavior data of the several sample accounts in each time interval, generate a user behavior vector sequence corresponding to each time interval;

The device also includes:

The training module uses the generated user behavior vector sequence as a training sample to train the LSTM model based on the encoding-decoding architecture.

Optionally, the generating module further:

Obtain various user operation behavior data of the account in various time intervals;

extracting key factors from the acquired user operation behavior data, and digitizing the key factors to obtain a user behavior vector corresponding to the user operation behavior data;

The user behavior vectors corresponding to various user operation behavior data in each time interval are spliced to generate a sequence of user behavior vectors corresponding to each time interval.

Optionally, the various user behaviors include credit performance behavior, user consumption behavior, financial management payment behavior;

The key factors include the loan order status and loan repayment amount corresponding to the credit performance behavior, the user consumption category and the number of user consumption transactions corresponding to the user consumption behavior, the financial management payment type and the financial management income amount corresponding to the financial management payment behavior.

Optionally, the LSTM encoder adopts a multi-layer many-to-one structure; the LSTM decoder adopts a multi-layer many-to-many structure with symmetrical numbers of input nodes and output nodes.

Optionally, the first computing module:

Input the generated user behavior vector sequence corresponding to each time interval into the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture to perform two-way propagation calculation, and obtain the first hidden state vector obtained by the forward propagation calculation; and , the second hidden state vector obtained by backpropagation calculation; wherein, when performing forward propagation calculation and backpropagation calculation, the input order of the user behavior vector sequence corresponding to each time interval is reversed;

Perform splicing processing on the first hidden state vector and the second hidden state vector to obtain final hidden state vectors corresponding to each time interval.

Optionally, the second computing module:

The hidden state vector corresponding to each time interval is used as the risk feature, input to the LSTM decoder for calculation, and the output vector of the target account in the next time interval is obtained;

The output vector is digitized to obtain the risk score of the target account in the next time interval.

Optionally, the output vector is a multidimensional vector;

The digital processing of the output vector includes any of the following:

Extracting the value of the sub-vector whose value is between 0 and 1 in the output vector as the risk score;

If the output vector contains multiple sub-vectors with values between 0 and 1, calculate the average value of the values of the multiple sub-vectors as the risk score;

If the output vector contains multiple sub-vectors with values between 0 and 1, extract the maximum or minimum value among the values of the multiple sub-vectors as the risk score.

The specification also proposes an electronic device, comprising:

processor;

memory for storing machine-executable instructions;

Wherein, by reading and executing the machine-executable instructions stored in the memory and corresponding to the control logic of credit risk prediction based on the LSTM model, the processor is prompted to:

Obtain user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step;

Based on the user operation behavior data of the target account in each time interval, generate a user behavior vector sequence corresponding to each time interval;

Input the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtain the hidden state vector corresponding to each time interval; wherein, the LSTM The model includes an LSTM encoder and an LSTM decoder with an attention mechanism;

The hidden state vectors corresponding to each time interval are used as risk features, input to the LSTM decoder for calculation, and the risk score of the target account in the next time interval is obtained; and, each hidden state vector corresponds to the risk score The weight value of ; wherein, the weight value represents the contribution of the hidden state vector to the risk score.

Description of drawings

Fig. 1 is a flow chart of a credit risk prediction method based on an LSTM model provided by an embodiment of this specification;

Fig. 2 is an LSTM model based on an encoder-decoder architecture provided by an embodiment of this specification;

FIG. 3 is a schematic diagram of a variety of multi-layer LSTM network architectures provided by an embodiment of this specification;

Fig. 4 is a schematic diagram of dividing users into groups provided by an embodiment of this specification;

Fig. 5 is a schematic diagram of constructing a user behavior vector sequence for each data node in an LSTM encoder provided by an embodiment of this specification;

Fig. 6 is a hardware structural diagram of a server carrying a credit risk prediction device based on an LSTM model provided by an embodiment of this specification;

Fig. 7 is a logical block diagram of an LSTM model-based credit risk prediction device provided by an embodiment of this specification.

Detailed ways

This specification aims to propose an LSTM model based on the encoder-decoder (encoding-decoding) architecture based on the user operation behavior data of the target account over a period of time in the scenario of credit risk prediction for the target account. The completed LSTM model is a technical solution for predicting the credit risk of the target account in the future.

When implementing, the modeling party can pre-define a target time period that needs to predict credit risk as the performance window, and pre-design a preset time period for observing the user behavior performance of the target account as the observation window, and combine the above performance window with the observation The windows are based on the time steps defined by the modeling party and form the time series.

For example, in an example, assuming that the modeling party needs to predict the credit risk of the target account in the next 6 months based on the user operation behavior data of the target account in the past 12 months, then the performance window can be designed as the past 6 months , design the observation window to be the past 12 months. Assuming that the time step defined by the modeling party is 1 month, then the performance window and observation window can be divided into several time intervals with a time step of 1 month to form a time series. At this time, each time interval is called a data node in the above time series.

The modeling party can prepare several sample accounts marked with risk tags, and obtain the user operation behavior data of these sample accounts in the above observation window, and based on the user operation behavior of each sample account in each time interval in the observation window Data, to construct the user behavior vector sequence corresponding to each time interval as a training sample, to train the LSTM model based on the encoder–decoder architecture; wherein, the above LSTM model includes an LSTM encoder and an LSTM that introduces an attention mechanism (Attention mechanism) decoder.

For example, these training samples can be input to the LSTM encoder for training calculations to train the LSTM encoder, and then the hidden state vectors corresponding to each time interval calculated from the training samples when training the LSTM encoder are used as the training decoder The required feature variables continue to be input to the LSTM decoder for training calculations to train the LSTM decoder, and the above process is performed iteratively until the LSTM model is trained.

When the modeling party predicts the credit risk of the target account in the above-mentioned performance window based on the above-mentioned LSTM model that has been trained, the same method can be used to obtain the user operation behavior data of the target account in the above-mentioned observation window, and based on the target The user operation behavior data of the account in each time interval within the observation window is used to construct the user behavior vector sequence corresponding to each time interval as a prediction sample, and then input these prediction samples into the LSTM encoder of the above LSTM model for calculation. Hidden state vectors corresponding to each time interval.

Further, the hidden state vectors corresponding to each time interval calculated by the LSTM encoder can be used as the risk characteristics of the target account, input to the above LSTM model for calculation, input the risk score of the target account, and each hidden state vector A weight value relative to the above-mentioned risk score; wherein, the weight value represents the contribution of the above-mentioned hidden state vector to the above-mentioned risk score.

In the above technical solution, on the one hand, since the user behavior vector sequence of the target account in each time interval is directly input into the LSTM encoder in the LSTM model based on the encoding-decoding architecture as input data for calculation, the corresponding The hidden state vector of each time interval, and then the obtained hidden state vector can be further input into the LSTM decoder for calculation as a risk feature, so as to complete the risk prediction of the target account and obtain a risk score; The user operation behavior data of the account is used to develop and explore the characteristic variables required for modeling, which can avoid the difficulty of digging out the information contained in the data due to the inaccurate characteristic variables designed based on the experience of the modeler. The accuracy of risk prediction is affected; moreover, there is no need to store and maintain the artificially designed characteristic variables, which can reduce the storage overhead of the system;

On the other hand, due to the introduction of the attention mechanism in the LSTM decoder of the LSTM model based on the encoding-decoding architecture, the hidden feature variables corresponding to each time interval obtained by the LSTM encoder are used as risk features and input into the LSTM decoder for The risk prediction calculation can obtain the weight value of the hidden state vector corresponding to each time interval corresponding to the final risk score, so that the contribution of each hidden feature variable to the final risk score can be intuitively evaluated, and the LSTM model can be improved. interpretability.

The specification is described below through specific embodiments and in combination with specific application scenarios.

Please refer to Figure 1, Figure 1 is a credit risk prediction method based on the LSTM model provided by an embodiment of this specification, which is applied to the server, and the method performs the following steps:

Step 102, acquiring user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step;

Step 104, based on the user operation behavior data of the target account in each time interval, generate a user behavior vector sequence corresponding to each time interval;

Step 106, input the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtain the hidden state vector corresponding to each time interval; wherein, The LSTM model includes an LSTM encoder, and an LSTM decoder that introduces an attention mechanism;

Step 108: Input the hidden state vectors corresponding to each time interval as risk features into the LSTM decoder for calculation, and obtain the risk score of the target account in the next time interval; and, each hidden state vector corresponds to the The weight value of the risk score; wherein, the weight value represents the contribution of the hidden state vector to the risk score.

The above-mentioned target account may include the payment account of the user, and the user may initiate a payment transaction by logging into the target account on a corresponding payment client (such as a payment APP).

The above-mentioned server may include a server, a server cluster, or a cloud platform based on a server cluster that provides services for the user-oriented payment client and performs risk identification on the payment account used by the user to log in to the client.

The above-mentioned operational behavior data may include data generated by a series of transaction-related operational behaviors performed by the user after logging into the target account on the client;

For example, the above-mentioned operation behaviors may include the user's credit performance behavior, user consumption behavior, wealth management and payment bank, store operation behavior, daily friendship behavior, etc. When the user completes the above-mentioned operation behavior through the client, the client can upload the data generated by performing the above operation behavior to the server, and the server will save it as an event in its local database.

In this specification, the modeling party can pre-define a target time period that needs to predict credit risk as the performance window, and pre-design a preset time period for observing the user behavior performance of the target account as the observation window, and combine the above performance window and The observation window is based on the time step defined by the modeling party to form a time series.

Wherein, the value of the time period corresponding to the above-mentioned performance window and observation window can be customized and set by the modeling party based on the actual prediction target, and will not be specifically limited in this specification. Correspondingly, the above-mentioned value of the time step can also be customized by the modeling party based on actual business requirements, and will not be specifically limited in this specification.

In the following example, the modeling party needs to predict the credit risk of the target account in the next 6 months based on the user operation behavior data of the target account in the past 12 months, and the defined time step is 1 month Take this as an example.

In this case, the above-mentioned performance window can be designed as the past 6 months, and the observation window can be designed as the past 12 months. Further, according to the defined time step, the performance window can be divided into 6 time intervals with a time step of 1 month, and then these time intervals can be organized into a time series; and, the observation window can be divided into 12 The time steps are all 1-month time intervals, and these time intervals are then organized into time series.

Please refer to FIG. 2, which is an LSTM model based on an encoder-decoder architecture shown in this specification.

As shown in Figure 2, the above-mentioned LSTM model based on the encoder-decoder architecture can specifically include an LSTM encoder and an LSTM decoder that introduces an attention mechanism.

The above-mentioned LSTM encoder (Encoder) is used to perform feature discovery on the user behavior vector sequence input by each data node in the above-mentioned observation window, and further input the hidden state vector (that is, the finally discovered feature) output by each data node to LSTM decoder. Wherein, the data nodes in the LSTM encoder correspond to each time interval in the above-mentioned observation window. Each time interval in the above observation window corresponds to a data node in the LSTM encoder.

The above-mentioned LSTM decoder (Decoder) is used to analyze the risk characteristics of each data node in the performance window based on the risk characteristics discovered by the LSTM encoder from the input user behavior vector sequence, and the behavior performance of the user in each data node in the observation window. The credit risk is forecasted, and the forecast results corresponding to each data node in the performance window are output. Wherein, the data nodes in the LSTM decoder correspond to each time interval in the above-mentioned representation window. Each time interval in the above performance window corresponds to a data node in the LSTM decoder.

It should be noted that, the time interval corresponding to the first data node in the above-mentioned LSTM decoder is a time interval next to the time interval corresponding to the last data node in the above-mentioned encoder. For example, in FIG. 2 , 0-M1 represents the time interval corresponding to the previous month at the current moment; S represents the time interval corresponding to the current month; P-M1 represents the time interval corresponding to the next month at the current moment.

The above attention mechanism (Attention) is used to mark the weight values corresponding to the prediction results output by the LSTM decoder in the performance window for the features output by each data node in the observation window of the LSTM encoder; wherein , the weight value characterizes the characteristics of the output of each data node in the observation window of the LSTM encoder, and corresponds to the contribution degree (also called the influence degree) of the prediction result output by each data node in the presentation window of the LSTM decoder.

By introducing the attention mechanism, the modeler can intuitively view the features found by the LSTM encoder in the observation window of each data node, and the contribution to the final prediction result output by the final LSTM decoder in the performance window of each data node, Improve the interpretability of LSTM models.

In one embodiment shown, in order to describe the user's operation behavior, both the above-mentioned LSTM encoder and LSTM decoder can adopt a multi-layer LSTM network architecture (for example, more than 3 layers).

Among them, the specific form of the multi-layer LSTM network architecture adopted by the above-mentioned LSTM encoder and LSTM decoder is not particularly limited in this specification; One-to-one, one-to-many, many-to-one, many-to-many with asymmetric number of input and output nodes, many-to-many with symmetrical number of input and output nodes, etc.

In one embodiment shown, since the LSTM encoder finally needs to summarize the hidden state vectors output by each data node in the observation window into one input, the LSTM encoder can use the many-to -one structure. Since the LSTM decoder finally needs to output a corresponding prediction result for each data node in the presentation window, the LSTM encoder can adopt a many-to-many structure with a symmetrical number of input and output nodes as shown in FIG. 3 .

The training and use process of the LSTM model based on the encoder-decoder architecture shown above will be described in detail through specific embodiments.

1) User grouping

In this manual, due to the large differences in the data thickness and credit behavior performance of different user groups, in order to avoid the impact of this difference on the accuracy of the model, the user groups that need to conduct credit risk assessment When modeling, the above-mentioned user groups can be divided into user groups according to these differences, and then for each user group, an LSTM model for credit risk assessment of users in the user group can be trained separately.

Among them, the characteristics adopted when dividing the user groups above, as well as the specific user group division methods, are not particularly limited in this specification;

For example, in practical applications, user groups can be divided according to user data richness, occupation, overdue times, age and other characteristics; for example, as shown in Figure 4, in one example, all users can be divided into data-sparse Groups with rich data and groups with rich data, and then groups with scarce data are further divided into user groups such as salaried and student groups according to their occupations, and groups with rich data are further divided into user groups with good credit and general credit according to the number of overdue times.

2) Training of LSTM model based on encoder–decoder architecture

In this specification, when training the above LSTM model for a certain user group, the modeling party may collect a large number of user accounts that belong to the user group and are marked with risk labels as sample accounts.

Among them, the above-mentioned risk tags may specifically include a tag for indicating that the account has credit risk, and a tag for indicating that the account does not have credit risk; for example, a tag 1 can be marked for a sample account with credit risk; for a sample account without credit risk The sample account of can be marked with a tag of 0.

It should be noted that, among the sample accounts marked with risk labels prepared by the modeling party, there are those marked with a label indicating that the account has credit risk, and the sample accounts marked with a label indicating that the account does not have credit risk The ratio of is not specifically limited in this specification, and the modeling party can set it based on actual modeling requirements.

Further, the modeling party can obtain these sample accounts marked with risk tags, the user operation behavior data in the above observation window, and obtain the user operation behavior data generated by these sample accounts in each time interval in the above observation window , based on the user operation behavior data generated by these sample accounts in the time interval corresponding to each data node in the above observation window, a corresponding user behavior vector sequence is constructed for each data node, and then the constructed user behavior vector sequence is used as a training Samples to train the above LSTM model based on the encoder–decoder architecture.

In one embodiment shown, the modeling party can pre-define a variety of user operation behaviors for constructing user behavior vector sequences, and when constructing corresponding user behavior vector sequences for each data node in the observation window, it can Obtain various user operation behavior data corresponding to the above-mentioned various user operation behaviors generated by the above-mentioned sample accounts in each time interval in the observation window, and extract key factors from the obtained user operation behavior data, and then analyze The extracted key factors are digitized to obtain a user behavior vector corresponding to each user's operation behavior data.

Further, after obtaining the user behavior vectors corresponding to each user operation behavior, the user behavior vectors corresponding to various user operation behavior data in the time interval corresponding to each data node in the above-mentioned observation window can be concatenated to generate a corresponding The sequence of user behavior vectors in each time interval.

Among them, the above-mentioned various user operation behaviors defined by the modeling party are not specifically limited in this specification, and the modeling party can customize based on actual needs; extract from the user operation behavior data corresponding to the above-mentioned various user operation behaviors The key factors of the above-mentioned user operation behavior data are not particularly limited in this specification.

Please refer to FIG. 5 . FIG. 5 is a schematic diagram of constructing a user behavior vector sequence for each data node in the LSTM encoder shown in this specification.

In one embodiment shown, the various user operation behaviors defined by the modeling party may specifically include credit performance behaviors, user consumption behaviors, and wealth management payment behaviors; correspondingly, the above-mentioned key factors may specifically include credit performance behaviors The corresponding loan order status and loan repayment amount, the user consumption category and the number of user consumption transactions corresponding to the user consumption behavior, the financial management payment type and financial income amount corresponding to the financial management payment behavior, etc.

For each time interval in the observation window, the credit performance behavior data, user consumption behavior data, and wealth management payment behavior data generated by the sample account in this time interval can be obtained separately, and then the loan order status can be extracted from the credit performance behavior data ( Shown in Figure 5 are normal and overdue two states) and loan repayment amount (shown in Figure 5 is the actual loan amount and overdue amount; for example, overdue 1/50 means overdue once, overdue amount 50 yuan ; Normal/10 means normal repayment, and the repayment amount is 10 yuan), and the user consumption category is extracted from the user consumption behavior data (shown in Fig. item) and the number of user consumption transactions, and extract the financial management payment type (shown in Figure 5 as two types of financial management products, monetary fund and fund) and the amount of financial management income from the financial management payment behavior data.

Further, the information extracted from credit performance behavior data, user consumption behavior data, and wealth management payment behavior data can be digitized to obtain user behavior vectors corresponding to each time interval for each user operation behavior data, and then the The three types of user operation behavior data shown above are concatenated corresponding to the user behavior vectors of each time interval, to obtain a sequence of user behavior vectors corresponding to each time interval.

In this specification, the calculations involved in the LSTM encoder in the LSTM model based on the encoder-decoder architecture generally include input gate calculations, memory gate (also called forget gate) calculations, unit state calculations, and hidden state vector calculations. Part; where, since in this specification, the hidden state vectors calculated by the LSTM encoder are finally summarized as the input of the LSTM decoder, so for the LSTM encoder, the output gate may not be involved. The calculation formulas involved in the calculation of the above parts are as follows:

f(t)＝f(W _f *X _i +U _f *h(t-1)+b _f )

i(t)＝f(W _i *X _i +U _i *h(t-1)+b _i )

m(t)＝tanh(W _m *X _i +U _m *h(t-1)+b _m )

h(t)=f(t)*h(t-1)+i(t)*m(t)

Among them, in the above formula, f(t) represents the memory gate of the t-th data node of the LSTM encoder; i(t) represents the input gate of the t-th data node of the LSTM encoder; m(t) represents the t-th data node of the LSTM encoder. The unit state of t data nodes (also known as the candidate hidden state); h(t) represents the hidden state vector corresponding to the tth data node (that is, the tth time interval) of the LSTM encoder; h(t-1) Represents the hidden state vector corresponding to the previous data node of the tth data node of the LSTM encoder; f represents the nonlinear activation function, and an appropriate nonlinear activation function can be selected based on actual needs; for example, for the LSTM encoder, the above f can specifically use the sigmoid function. W _f and U _f represent the weight matrix of the memory gate; b _f represents the bias term of the memory gate. W _i and U _i represent the weight matrix of the input gate; b _i represents the bias term of the input gate; W _m and U _m represent the weight matrix of the cell state; b _m represents the bias term of the cell state.

In this specification, the calculations involved in the attention mechanism introduced in the LSTM decoder in the LSTM model based on the encoder-decoder architecture usually include the calculation of the contribution value and the normalization of the contribution value (normalization between 0 and 1) into two parts of the calculation of the weight value. The calculation formulas involved in the calculation of the above parts are as follows:

etj＝tanh(W _a *s(j-1)+U _a *h(t))

atj=exp(etj)/sum_T(exp(etj))

Among them, in the above formula, etj represents the hidden state vector corresponding to the tth data node of the LSTM encoder, and the contribution value of the prediction result corresponding to the jth data node of the LSTM encoder; atj represents the normalization of etj After processing, the obtained weight value; exp(etj) means to perform an exponential function operation on etj; sum_T(exp(etj)) means to sum up the etj of a total of T data nodes of the LSTM encoder. s(j-1) represents the hidden state vector corresponding to the jth data node of the LSTM decoder. W _a and U _a are the weight matrices of the attention mechanism.

Among them, it should be noted that, in the above formula, etj is normalized, and the result of performing exponential function operation on the value of etj is used, which is calculated with the etj of a total of T data nodes of the LSTM encoder. The method of dividing the results of the sum and the value of etj is normalized to the interval [0,1]. In practical applications, in addition to the normalization method shown in the above formula, those skilled in the art will When the technical solution is put into practice, other normalization methods can also be adopted, which will not be listed one by one in this specification.

In this specification, the calculations involved in the LSTM encoder in the LSTM model based on the encoder-decoder architecture generally include input gate calculations, memory gate calculations, output gate calculations, unit state calculations, hidden state vector calculations, and output vector calculations, etc. six parts. The calculation formulas involved in the calculation of the above parts are as follows:

F(j)＝f(W _F *C _j +U _F *S(j-1)+K _F *y(j-1)+b _f )

I(j)＝f(W _I *C _j +U _I *S(j-1)+K _I *y(j-1)+b _I )

O(j)＝f(W _o *C _j +U _O *S(j-1)+K _O *y(j-1)+b _O )

n(j)=tanh(W _n *C _j +U _n *S(j-1)+K _m *y(j-1)+b _n )

S(j)＝F(j)*S(j‐1)+I(j)*n(j)

y(j)=O(j)*tanh(S(j))

C _j = sum_T(atj*h(t))

Among them, in the above formula, F(j) represents the memory gate of the jth data node of the LSTM decoder; I(j) represents the input gate of the jth data node of the LSTM decoder; O(j) represents the memory gate of the jth data node of the LSTM decoder; The output gate of j data nodes; n(j) represents the unit state of the jth data node of the LSTM decoder; S(j) represents the hidden state vector corresponding to the jth data node of the LSTM decoder; S(j-1) Represents the hidden state vector corresponding to the previous data node of the jth data node of the LSTM decoder; y(j) represents the output vector of the jth node of the LSTM decoder; f represents a nonlinear activation function, which can be selected based on actual needs The nonlinear activation function; for example, for the LSTM decoder, the above f can also use the sigmoid function. C _j represents the weighted sum obtained by multiplying the hidden state vector h(t) corresponding to each data node of the LSTM encoder by the attention weight atj calculated based on the attention mechanism of the LSTM decoder; W _F , U _F and K _F represents the weight matrix of the memory gate; b _F represents the bias term of the memory gate. W _I , U _I and K _I represent the weight matrix of the input gate; b _I represents the bias term of the input gate; W _O , U _O and K _O represent the weight matrix of the output gate; b _O represents the bias term of the output gate. W _n , U _n and K _n represent the weight matrix of the cell state; b _n represents the bias item of the cell state.

In this specification, W _f , U _f , b _f , W _i , U _i , _bi , W _m , U _m , b _m , W _a , U _a , W _F , U _F shown in the above formulas , K _F , b _F , W _I , U _I , K _I , b _I , W _O , U _O , K _O , b _o , W _n , U _n and K _n , b _n are the above LSTM model Finally, the trained model parameters are needed.

When training the above LSTM model, the user behavior vector corresponding to each time interval constructed based on the user operation behavior data in each time interval in the observation window of the sample account marked with the risk label shown above can be The sequence is used as a training sample, input into the LSTM encoder for training calculation, and then the calculation result of the LSTM encoder is continuously input into the LSTM decoder for training calculation, and the above model is continuously updated by iterating the above training calculation process Adjust the parameters; when the above parameters are adjusted to the optimal value, the training algorithm of the model converges at this time, and the training of the above LSTM model is completed.

Among them, it should be noted that the training algorithm used in training the above LSTM model is not particularly limited in this specification; for example, in one implementation, the gradient descent method can be used to continuously perform iterative operations to train the above LSTM model.

3) Credit risk prediction of LSTM model based on encoder–decoder architecture

In this specification, according to the model training process shown in the above embodiments, an LSTM model is trained for each divided user group, and based on the trained LSTM model, user accounts belonging to the user group are credited. risk assessment.

When the modeling party needs to conduct risk assessment for a certain target account, the modeling party can obtain the target account and obtain the user operation behavior data generated by the target account in each time interval in the above observation window, based on the target account in For the user operation behavior data generated in the time interval corresponding to each data node in the observation window, a corresponding user behavior vector sequence is respectively constructed for each data node.

Wherein, the process of constructing the user behavior vector sequence for the above-mentioned target account will not be described in detail in this specification, and the description of the previous embodiment may be referred to; for example, the method shown in Figure 5 can still be used to construct and observe The sequence of user behavior vectors corresponding to each time interval in the window.

After the user behavior vector sequence corresponding to each time interval in the observation window is constructed for the target account, the LSTM model corresponding to the user group to which the target account belongs can be determined first from the trained LSTM model, and then the user The behavior vector sequence is used as a prediction sample, which is input to each data node in the LSTM encoder of the LSTM model for calculation.

Among them, for the LSTM model, one of forward propagation calculation or back propagation calculation is usually used. The so-called forward propagation calculation refers to the user behavior vector sequence corresponding to each time interval in the observation window, and the input order in the LSTM model is the same as the propagation direction of each data node in the LSTM model; otherwise, the so-called back propagation Calculation refers to the user behavior vector sequence corresponding to each time interval in the observation window. The input order in the LSTM model is opposite to the propagation direction of each data node in the LSTM model.

That is to say, for the backpropagation calculation and the forward propagation calculation, the sequence of user behavior vectors in each time interval in the observation window is completely reversed as the input data.

For example, taking forward propagation calculation as an example, for the target account corresponding to the user behavior vector sequence X ₁ of the first time interval (i.e., the first month) in the observation window, it can be used as the For the data input of the first data node in the propagation direction, f(1), i(1), m(1) are solved according to the LSTM coding calculation formula shown above, and then based on the calculated f(1), i(1), m(1) further solve the hidden state vector h(1) corresponding to the first time interval. Then, the user behavior vector sequence X ₂ of the second time interval is used as the data input of the second data node in the propagation direction of each data node of the LSTM encoder, and the same calculation method is used for calculation, and so on, in turn The hidden state vectors h(2)-h(12) corresponding to the 2nd to 12th time intervals are respectively calculated.

As another example, taking backpropagation calculation as an example, the user behavior vector sequence X ₁₂ of the target account corresponding to the 12th time interval (that is, the last time interval) in the observation window can be used as each data node of the LSTM encoder For the data input of the first data node in the direction of propagation, use the same calculation method to solve f(1), i(1), m(1), and then based on the calculated f(1), i(1 ), m(1) to further solve the hidden state vector h(1) corresponding to the first time interval. Then, the user behavior vector sequence X ₁₁ of the 11th time interval is used as the data input of the second data node in the propagation direction of each data node of the LSTM encoder, and the same calculation method is used for calculation, and so on, in turn The hidden state vectors h(2)-h(12) corresponding to the 2nd to 12th time intervals are respectively calculated.

In one embodiment shown, in order to improve the calculation accuracy of the LSTM encoder, the calculation in the LSTM encoder may use bidirectional propagation calculation. After the backpropagation calculation and the forward propagation calculation are completed, for each data node in the LSTM encoder, a first hidden state vector obtained by the forward propagation calculation and a first hidden state vector obtained by the backpropagation calculation can be obtained respectively. The second hidden state vector.

In this case, the first hidden state vector and the second hidden state corresponding to each data node in the LSTM encoder can be concatenated as the final hidden state vector corresponding to each data node; for example, the first hidden state vector of the LSTM encoder Take t data nodes as an example, assuming that the first hidden state vector calculated by the data node is recorded as ht_before, the calculated second hidden vector is recorded as ht_after, and the final hidden vector is recorded as ht_final, then ht_final can be expressed as t_final=[ht_before , ht_after].

In this specification, when the user behavior vector sequence corresponding to each time interval in the observation window is constructed for the target account as a prediction sample, and input to each data node in the LSTM encoder of the above LSTM model to complete the calculation, the The hidden state vector calculated by each data node in the LSTM encoder is used as the risk feature extracted from the user operation behavior data of the target account, and is further input into the LSTM decoder in the above-mentioned LSTM model, according to the above-mentioned embodiment. The calculation formula of the LSTM decoder is calculated to predict the credit risk of the above-mentioned target account in each time interval in the above-mentioned performance window.

For example, firstly, based on the attention mechanism of the LSTM decoder, the attention weight atj of the hidden state vector corresponding to each data node in the LSTM encoder can be calculated, and then the attention weight atj corresponding to each data node in the LSTM encoder can be further calculated. The weighted sum C _j of the hidden state vector multiplied by the corresponding attention weight atj. Then, based on the calculation formula of the LSTM decoder shown above, the output vector corresponding to the first data node in the LSTM decoder can be further calculated, and the credit risk of the above-mentioned target account in the first time interval in the performance window can be calculated. Prediction; and so on, based on the same method, according to the calculation formula of the LSTM decoder shown above, the output vector corresponding to the next data node in the LSTM decoder can be calculated sequentially, and the above target account can be displayed in the performance window The credit risk of the next time interval is predicted.

In this specification, after the calculation of the LSTM decoder is completed, the attention weight atj of the hidden state vector corresponding to each data node in the LSTM encoder and the output vector corresponding to each data node in the LSTM decoder can be obtained.

In one embodiment shown, the above-mentioned LSTM model can further digitize the output vectors corresponding to each data node in the LSTM decoder, and convert the output vectors corresponding to each data node into Risk score, as the credit risk prediction result of the target account in each time interval in the performance window.

Wherein, the above-mentioned output vector is digitally processed, and the specific method of converting the above-mentioned output vector into a risk score is not particularly limited in this specification;

For example, in one implementation manner, since the final output vector is a multi-dimensional vector, the output vector usually includes sub-vectors whose values are between 0 and 1. Therefore, during implementation, the value of the sub-vector whose value is between 0 and 1 in the above output vector can be directly extracted as the risk score corresponding to the output vector.

In another implementation shown, if the above output vector contains a plurality of sub-vectors with values between 0 and 1, the maximum or minimum value of the values of the multiple sub-vectors can be extracted as the The risk score corresponding to the output vector; or, the average value of the values of the multiple sub-vectors may also be calculated as the risk score.

After the above calculation is completed, the above-mentioned LSTM decoder can use the risk score corresponding to each data node in the LSTM decoder, and the hidden state vector obtained from each data node in the above-mentioned LSTM encoder, relative to the weight of the above-mentioned risk score value, which is output as the final prediction result.

Among them, in one embodiment shown, the above-mentioned LSTM decoder can also summarize the risk scores corresponding to each data node in the LSTM decoding, and then convert it into a report on whether the above-mentioned target account has credit risk in the above-mentioned performance window. forecast result.

In one implementation, the above-mentioned LSTM decoder can sum the risk scores corresponding to each data node in the LSTM decoding, and then compare the summation result with a preset risk threshold; if the summation result is greater than or equal to the risk threshold, output a 1, indicating that the target account has credit risk in the above realization window; on the contrary, if the summation result is less than the risk threshold, output a 0, indicating that the target account does not have credit risk in the above realization window.

It can be seen from the above embodiments that, on the one hand, since the user behavior vector sequence of the target account in each time interval is directly input as input data into the LSTM encoder in the LSTM model based on the encoding-decoding architecture for calculation, the corresponding The hidden state vector of each time interval, and then the obtained hidden state vector can be further input into the LSTM decoder for calculation as a risk feature, so as to complete the risk prediction of the target account and obtain a risk score; The user operation behavior data of the account is used to develop and explore the characteristic variables required for modeling, which can avoid the difficulty of digging out the information contained in the data due to the inaccurate characteristic variables designed based on the experience of the modeler. The accuracy of risk prediction is affected; moreover, there is no need to store and maintain the artificially designed characteristic variables, which can reduce the storage overhead of the system;

On the other hand, due to the introduction of the attention mechanism in the LSTM decoder of the LSTM model based on the encoding-decoding architecture, the hidden feature variables corresponding to each time interval obtained by the LSTM encoder are used as risk features and input into the LSTM decoder for The risk prediction calculation can obtain the weight value of the hidden state vector corresponding to each time interval corresponding to the final risk score, so that the contribution of each hidden feature variable to the final risk score can be intuitively evaluated, and the LSTM model can be improved. interpretability.

Corresponding to the foregoing method embodiments, this specification also provides device embodiments.

Corresponding to the foregoing method embodiments, this specification also provides an embodiment of an LSTM model-based credit risk prediction device. The embodiment of the credit risk prediction device based on the LSTM model in this specification can be applied to electronic equipment. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for operation by the processor of the electronic device where it is located. From the hardware level, as shown in Figure 6, it is a hardware structure diagram of the electronic equipment where the credit risk prediction device based on the LSTM model in this specification is located, except for the processor, memory, network interface, and non- In addition to the volatile memory, the electronic device where the device in the embodiment is located usually may also include other hardware according to the actual function of the electronic device, which will not be repeated here.

Fig. 7 is a block diagram of an LSTM model-based credit risk prediction device according to an exemplary embodiment of this specification.

Please refer to FIG. 7, the credit risk prediction device 70 based on the LSTM model can be applied to the electronic device shown in FIG. 6, including: an acquisition module 701, a generation module 702, a first calculation module 703 and a second calculation module 704.

The acquisition module 701 is used to acquire user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step;

A generation module 702, based on the user operation behavior data of the target account in each time interval, generating a sequence of user behavior vectors corresponding to each time interval;

The first calculation module 703 inputs the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtains the hidden state vector corresponding to each time interval ; Wherein, the LSTM model includes an LSTM encoder, and an LSTM decoder that introduces an attention mechanism;

The second calculation module 704 is to input the hidden state vectors corresponding to each time interval as risk features into the LSTM decoder for calculation, and obtain the risk score of the target account in the next time interval; and, each hidden state vector A weight value corresponding to the risk score; wherein the weight value represents the contribution of the hidden state vector to the risk score.

In this embodiment, the acquiring module 701 further:

Obtain the user operation behavior data of several sample accounts marked with risk tags within the preset time period;

The generation module 702 further:

Based on the user operation behavior data of the several sample accounts in each time interval, generate a user behavior vector sequence corresponding to each time interval;

The device 70 also includes:

The training module 705 (not shown in FIG. 7 ) uses the generated user behavior vector sequence as a training sample to train the LSTM model based on the encoding-decoding architecture.

In this embodiment, the generating module 702 further:

Obtain various user operation behavior data of the account in various time intervals;

extracting key factors from the acquired user operation behavior data, and digitizing the key factors to obtain a user behavior vector corresponding to the user operation behavior data;

The user behavior vectors corresponding to various user operation behavior data in each time interval are spliced to generate a sequence of user behavior vectors corresponding to each time interval.

In this embodiment, the various user behaviors include credit performance behavior, user consumption behavior, financial management payment behavior;

The key factors include the loan order status and loan repayment amount corresponding to the credit performance behavior, the user consumption category and the number of user consumption transactions corresponding to the user consumption behavior, the financial management payment type and the financial management income amount corresponding to the financial management payment behavior.

In this embodiment, the LSTM encoder adopts a multi-layer many-to-one structure; the LSTM decoder adopts a multi-layer many-to-many structure with symmetrical numbers of input nodes and output nodes.

In this embodiment, the first calculation module 703:

Input the generated user behavior vector sequence corresponding to each time interval into the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture to perform two-way propagation calculation, and obtain the first hidden state vector obtained by the forward propagation calculation; and , the second hidden state vector obtained by backpropagation calculation; wherein, when performing forward propagation calculation and backpropagation calculation, the input order of the user behavior vector sequence corresponding to each time interval is reversed;

Perform splicing processing on the first hidden state vector and the second hidden state vector to obtain final hidden state vectors corresponding to each time interval.

In this embodiment, the second calculating module 704:

The hidden state vector corresponding to each time interval is used as the risk feature, input to the LSTM decoder for calculation, and the output vector of the target account in the next time interval is obtained;

The output vector is digitized to obtain the risk score of the target account in the next time interval.

In this embodiment, the output vector is a multidimensional vector;

The digital processing of the output vector includes any of the following:

Extracting the value of the sub-vector whose value is between 0 and 1 in the output vector as the risk score;

If the output vector contains multiple sub-vectors with values between 0 and 1, calculate the average value of the values of the multiple sub-vectors as the risk score;

If the output vector contains multiple sub-vectors with values between 0 and 1, extract the maximum or minimum value among the values of the multiple sub-vectors as the risk score.

For the implementation process of the functions and effects of each module in the above-mentioned device, please refer to the implementation process of the corresponding steps in the above-mentioned method for details, and details will not be repeated here.

As for the device embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. It can be understood and implemented by those skilled in the art without creative effort.

The systems, devices, modules or modules described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer, which may take the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, e-mail device, game control device, etc. desktops, tablets, wearables, or any combination of these.

Corresponding to the foregoing method embodiments, this specification also provides an embodiment of an electronic device. The electronic device includes: a processor and a memory for storing machine-executable instructions; wherein, the processor and the memory are usually connected to each other through an internal bus. In other possible implementation manners, the device may further include an external interface, so as to be able to communicate with other devices or components.

In this embodiment, by reading and executing the machine-executable instructions stored in the memory and corresponding to the control logic of credit risk prediction based on the LSTM model, the processor is prompted to:

Obtain user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step;

Based on the user operation behavior data of the target account in each time interval, generate a user behavior vector sequence corresponding to each time interval;

Input the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtain the hidden state vector corresponding to each time interval; wherein, the LSTM The model includes an LSTM encoder and an LSTM decoder with an attention mechanism;

The hidden state vectors corresponding to each time interval are used as risk features, input to the LSTM decoder for calculation, and the risk score of the target account in the next time interval is obtained; and, each hidden state vector corresponds to the risk score The weight value of ; wherein, the weight value represents the contribution of the hidden state vector to the risk score.

In this embodiment, by reading and executing the machine-executable instructions stored in the memory and corresponding to the control logic of credit risk prediction based on the LSTM model, the processor is further prompted to:

Obtain user operation behavior data of several sample accounts marked with risk tags within the preset time period; based on the user operation behavior data of the several sample accounts within each time interval, generate user behavior corresponding to each time interval Vector sequence; use the generated user behavior vector sequence as a training sample to train the LSTM model based on the encoding-decoding architecture.

In this embodiment, by reading and executing the machine-executable instructions stored in the memory and corresponding to the control logic of credit risk prediction based on the LSTM model, the processor is further prompted to:

Obtain various user operation behavior data of the account in various time intervals;

extracting key factors from the acquired user operation behavior data, and digitizing the key factors to obtain a user behavior vector corresponding to the user operation behavior data;

The user behavior vectors corresponding to various user operation behavior data in each time interval are spliced to generate a sequence of user behavior vectors corresponding to each time interval.

In this embodiment, by reading and executing the machine-executable instructions stored in the memory and corresponding to the control logic of credit risk prediction based on the LSTM model, the processor is further prompted to:

Input the generated user behavior vector sequence corresponding to each time interval into the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture to perform two-way propagation calculation, and obtain the first hidden state vector obtained by the forward propagation calculation; and , the second hidden state vector obtained by backpropagation calculation; wherein, when performing forward propagation calculation and backpropagation calculation, the input order of the user behavior vector sequence corresponding to each time interval is reversed;

Perform splicing processing on the first hidden state vector and the second hidden state vector to obtain final hidden state vectors corresponding to each time interval.

In this embodiment, by reading and executing the machine-executable instructions stored in the memory and corresponding to the control logic of credit risk prediction based on the LSTM model, the processor is further prompted to:

The hidden state vector corresponding to each time interval is used as the risk feature, input to the LSTM decoder for calculation, and the output vector of the target account in the next time interval is obtained;

The output vector is digitized to obtain the risk score of the target account in the next time interval.

In this embodiment, the output vector is a multidimensional vector; by reading and executing the machine-executable instructions stored in the memory and corresponding to the control logic of credit risk prediction based on the LSTM model, the processor is also prompted to execute any of the following:

Extracting the value of the sub-vector whose value is between 0 and 1 in the output vector as the risk score;

If the output vector contains multiple sub-vectors with values between 0 and 1, calculate the average value of the values of the multiple sub-vectors as the risk score;

If the output vector contains multiple sub-vectors with values between 0 and 1, extract the maximum or minimum value among the values of the multiple sub-vectors as the risk score.

Other embodiments of the specification will readily occur to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This description is intended to cover any modification, use or adaptation of this description. These modifications, uses or adaptations follow the general principles of this description and include common knowledge or conventional technical means in the technical field not disclosed in this description. . The specification and examples are to be considered exemplary only, with a true scope and spirit of the specification being indicated by the following claims.

It should be understood that this specification is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the specification is limited only by the appended claims.

The above descriptions are only preferred embodiments of this specification, and are not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification shall be included in this specification. within the scope of protection.

Claims (17)

1. A credit risk prediction method based on LSTM model, said method comprising: Obtain user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step; Based on the user operation behavior data of the target account in each time interval, generate a user behavior vector sequence corresponding to each time interval; Input the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtain the hidden state vector corresponding to each time interval; wherein, the LSTM The model includes an LSTM encoder and an LSTM decoder with an attention mechanism; The hidden state vectors corresponding to each time interval are used as risk features, input to the LSTM decoder for calculation, and the risk score of the target account in the next time interval is obtained; and, each hidden state vector corresponds to the risk score The weight value of ; wherein, the weight value represents the contribution of the hidden state vector to the risk score. 2. The method of claim 1, further comprising: Obtain the user operation behavior data of several sample accounts marked with risk tags within the preset time period; Based on the user operation behavior data of the several sample accounts in each time interval, generate a user behavior vector sequence corresponding to each time interval; The generated user behavior vector sequence is used as a training sample to train the LSTM model based on the encoder-decoder architecture. 3. The method according to claim 2, based on the user operation behavior data of the account in each time interval, generating a user behavior vector sequence corresponding to each time interval, comprising: Obtain various user operation behavior data of the account in various time intervals; extracting key factors from the acquired user operation behavior data, and digitizing the key factors to obtain a user behavior vector corresponding to the user operation behavior data; The user behavior vectors corresponding to various user operation behavior data in each time interval are spliced to generate a sequence of user behavior vectors corresponding to each time interval. 4. The method according to claim 3, said various user behaviors include credit performance behavior, user consumption behavior, financial management payment behavior; The key factors include the loan order status and loan repayment amount corresponding to the credit performance behavior, the user consumption category and the number of user consumption transactions corresponding to the user consumption behavior, the financial management payment type and the financial management income amount corresponding to the financial management payment behavior. 5. The method according to claim 1, the LSTM encoder adopts a multi-layer many-to-one structure; the LSTM decoder adopts a multi-layer many-to-many structure with symmetrical input nodes and output nodes . 6. The method according to claim 1, wherein the generated user behavior vector sequence corresponding to each time interval is input to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture to obtain the corresponding Hidden state vectors for each time interval, including: Input the generated user behavior vector sequence corresponding to each time interval into the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture to perform two-way propagation calculation, and obtain the first hidden state vector obtained by the forward propagation calculation; and , the second hidden state vector obtained by backpropagation calculation; wherein, when performing forward propagation calculation and backpropagation calculation, the input order of the user behavior vector sequence corresponding to each time interval is reversed; Perform splicing processing on the first hidden state vector and the second hidden state vector to obtain final hidden state vectors corresponding to each time interval. 7. The method according to claim 1, wherein the hidden state vector corresponding to each time interval is used as a risk feature, input to the LSTM decoder for calculation, and the risk score of the target account in the next time interval is obtained ,include: The hidden state vector corresponding to each time interval is used as the risk feature, input to the LSTM decoder for calculation, and the output vector of the target account in the next time interval is obtained; The output vector is digitized to obtain the risk score of the target account in the next time interval. 8. The method according to claim 1, the output vector is a multidimensional vector; The digital processing of the output vector includes any of the following: Extracting the value of the sub-vector whose value is between 0 and 1 in the output vector as the risk score; If the output vector contains multiple sub-vectors with values between 0 and 1, calculate the average value of the values of the multiple sub-vectors as the risk score; If the output vector contains multiple sub-vectors with values between 0 and 1, extract the maximum or minimum value among the values of the multiple sub-vectors as the risk score. 9. A credit risk forecasting device based on LSTM model, said device comprising: The acquisition module acquires user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step; A generation module, based on the user operation behavior data of the target account in each time interval, generates a sequence of user behavior vectors corresponding to each time interval; The first calculation module inputs the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtains the hidden state vector corresponding to each time interval; Wherein, the LSTM model includes an LSTM encoder, and an LSTM decoder that introduces an attention mechanism; The second calculation module is to input the hidden state vectors corresponding to each time interval as risk features into the LSTM decoder for calculation, and obtain the risk score of the target account in the next time interval; and each hidden state vector corresponds to A weight value of the risk score; wherein, the weight value represents the contribution of the hidden state vector to the risk score. 10. The device according to claim 9, the acquisition module further: Obtain the user operation behavior data of several sample accounts marked with risk tags within the preset time period; The generating module further: Based on the user operation behavior data of the several sample accounts in each time interval, generate a user behavior vector sequence corresponding to each time interval; The device also includes: The training module uses the generated user behavior vector sequence as a training sample to train the LSTM model based on the encoding-decoding architecture. 11. The apparatus of claim 10, said generating module further: Obtain various user operation behavior data of the account in various time intervals; extracting key factors from the acquired user operation behavior data, and digitizing the key factors to obtain a user behavior vector corresponding to the user operation behavior data; The user behavior vectors corresponding to various user operation behavior data in each time interval are spliced to generate a sequence of user behavior vectors corresponding to each time interval. 12. The device according to claim 11, the various user behaviors include credit performance behavior, user consumption behavior, financial management payment behavior; The key factors include the loan order status and loan repayment amount corresponding to the credit performance behavior, the user consumption category and the number of user consumption transactions corresponding to the user consumption behavior, the financial management payment type and the financial management income amount corresponding to the financial management payment behavior. 13. The device according to claim 9, wherein the LSTM encoder adopts a multi-layer many-to-one structure; the LSTM decoder adopts a multi-layer many-to-many structure with a symmetrical number of input nodes and output nodes . 14. The apparatus of claim 9, said first computing module: Input the generated user behavior vector sequence corresponding to each time interval into the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture to perform two-way propagation calculation, and obtain the first hidden state vector obtained by the forward propagation calculation; and , the second hidden state vector obtained by backpropagation calculation; wherein, when performing forward propagation calculation and backpropagation calculation, the input order of the user behavior vector sequence corresponding to each time interval is reversed; Perform splicing processing on the first hidden state vector and the second hidden state vector to obtain final hidden state vectors corresponding to each time interval. 15. The apparatus of claim 9, said second computing module: The hidden state vector corresponding to each time interval is used as the risk feature, input to the LSTM decoder for calculation, and the output vector of the target account in the next time interval is obtained; The output vector is digitized to obtain the risk score of the target account in the next time interval. 16. The device according to claim 9, wherein the output vector is a multidimensional vector; The digital processing of the output vector includes any of the following: Extracting the value of the sub-vector whose value is between 0 and 1 in the output vector as the risk score; If the output vector contains multiple sub-vectors with values between 0 and 1, calculate the average value of the values of the multiple sub-vectors as the risk score; If the output vector contains multiple sub-vectors with values between 0 and 1, extract the maximum or minimum value among the values of the multiple sub-vectors as the risk score. 17. An electronic device comprising: processor; memory for storing machine-executable instructions; Wherein, by reading and executing the machine-executable instructions stored in the memory and corresponding to the control logic of credit risk prediction based on the LSTM model, the processor is prompted to: Obtain user operation behavior data of the target account within a preset time period; wherein, the preset time period is a time series composed of several time intervals with the same time step; Based on the user operation behavior data of the target account in each time interval, generate a user behavior vector sequence corresponding to each time interval; Input the generated user behavior vector sequence corresponding to each time interval to the LSTM encoder in the trained LSTM model based on the encoding-decoding architecture for calculation, and obtain the hidden state vector corresponding to each time interval; wherein, the LSTM The model includes an LSTM encoder and an LSTM decoder with an attention mechanism; The hidden state vectors corresponding to each time interval are used as risk features, input to the LSTM decoder for calculation, and the risk score of the target account in the next time interval is obtained; and, each hidden state vector corresponds to the risk score The weight value of ; wherein, the weight value represents the contribution of the hidden state vector to the risk score.