CN111881363A - Recommendation method based on graph interaction network - Google Patents

Abstract

A recommendation method based on graph interaction network is applied in the field of user personalized recommendation. With the rapid development of the Internet industry and the continuous growth of the amount of network data, traditional recommendation methods and deep learning methods are difficult to meet the complex application environment, and there are shortcomings in terms of accuracy and space complexity. Therefore, the present invention proposes a recommendation method based on a graph interaction network, which can ensure the accuracy of personalized recommendation and reduce the complexity of the model space, and has broad application prospects.

Description

A Recommendation Method Based on Graph Interaction Network

technical field

The invention is applied to the field of recommendation system based on U-I relationship, and specifically relates to data mining and deep learning technologies such as graph depth network, attention mechanism, user preference information and item attribute information feature extraction, U-I interaction information modeling and the like.

Background technique

Personalized recommendation is a comprehensive analysis task, which is widely used in social networking, music radio, e-commerce, personalized advertising, movie and video websites and other fields, so it has attracted much attention. With the rapid development of the Internet industry and the continuous growth of the amount of network data, recommendation systems are faced with increasingly complex recommendation tasks and application environments. Especially since entering the Web2.0 era, with the sudden emergence of social network media, Internet people are both consumers of network information and producers of network content, and the amount of information on the Internet has grown exponentially. Due to the limited ability of users to discriminate, they often feel helpless in the face of huge and complex Internet information, which makes the cost of finding useful information on the Internet huge, and the problem of information overload is born. Therefore, for users, how to quickly and accurately locate the content they need in the exponentially growing resources is a very important and challenging thing. It is also quite difficult for merchants to present appropriate items to users in a timely manner, thereby promoting transaction volume and economic growth. The emergence of recommender systems has greatly alleviated this difficulty.

As the most classic traditional recommendation algorithm, the collaborative filtering algorithm has played an important role in the recommendation scenarios in the early years. Collaborative filtering algorithms can naturally model higher-order relationships between users and items, with low model complexity and easy deployment. These advantages make it the most widely used recommendation method to date. However, with the explosive growth of Internet users, the types and attributes of items are becoming more and more complete, and the increasing abundance of Internet resources makes user preference information develop towards a diversified and detailed trend. In this environment, the recommendation accuracy rate of the algorithm has great shortcomings, and the traditional recommendation algorithm represented by the collaborative filtering algorithm is difficult to meet the personalized needs of users.

In recent years, deep learning technology has developed rapidly. Deep learning is more and more widely used in the fields of computer vision, natural language processing, and recommendation systems. Deep learning technology deeply mines user preference information and item attribute information by building a deep network. The model architecture of the recommender system is innovated, and many defects of the traditional model are overcome. The recommendation accuracy rate has been greatly improved, and there are more effective solutions to problems such as system cold start, which have gained widespread attention. However, traditional deep learning techniques cannot naturally combine user and item information. Generally, based on the interaction between users and items, the deep network is used to extract better user and item features, and then other models or another deep network are used to predict the user's preference for recommended items. Although this multi-model stacking method improves the The accuracy of personalized recommendation is improved, but the model is difficult to deploy easily, and the scalability and space complexity of the model cannot be effectively guaranteed.

In recent years, the analysis and processing of non-European data has become a hot topic in academia and industry, and graph deep networks can naturally integrate the interaction between users and items. Effectively extracting graph data features and node features, naturally mining user preferences and item attribute information expression, provides a new direction for personalized recommendation tasks. In recent years, the research of graph deep network has made great progress. Thomas Kpif proposed the concept of graph convolutional network in 2017, which provides a new idea for the processing of graph structure data. The product neural network is applied to graph-structured data. Among them, the graph convolution method based on the spatial domain takes any node as the convolution object and collects the information of the neighbor nodes to construct the graph convolution layer. Because of its flexibility and efficiency, it is more applicable than the graph convolution method based on the frequency domain. for broad. At the same time, Hamilton proposed an inductive learning method suitable for large-scale networks, which can quickly generate node features for new nodes without additional training, which greatly alleviates the scalability problem of graph convolutional neural networks. The rapid development of graph deep networks has pointed out the direction for the construction of personalized recommendation systems.

SUMMARY OF THE INVENTION

In order to realize the user's personalized recommendation system, a personalized recommendation scheme based on graph interaction network is proposed. The method flow is shown in Figure 1. The method takes the U-I interaction diagram as input data, and each module completes feature extraction and feature analysis for user preferences and item attributes, and finally outputs the user's predicted score for the target item. Specifically, the method firstly models the interaction relationship data between users and items on a graph structure, and the datasets are all from the public datasets in the industry. After completing the graph structuring process, the feature distribution of users and items is optimized by stacking multiple mean convolutional layers. After the U-I feature optimization is completed, there will be a greater similarity between user features and item features that meet user preferences. Then, the attention mechanism is used to fuse the node features of the target user and the node features of the item to be recommended to obtain the implicit feature expression of the graph interaction network, that is, the interaction vector of the U-I relationship pair. Finally, the method learns the interaction feature vector through a multi-layer graph fully connected layer network The distribution law of , model the high-order nonlinear relationship between U-I, and get the user's predicted score for the target item. Finally, the personalized recommendation task for users is realized through the top-N recommendation mechanism. The overall block diagram of this method is shown in Figure 1.

The invention contents of each main module of this method are as follows:

1. Optimization of user preference information and item attribute information

The first module is the user preference and item attribute feature optimization module. The purpose of user preference information modeling is to obtain accurate user feature representation according to user interaction behavior, and then fully mine the user's hobby information. Item attribute information modeling is to obtain perfect relationship attributes and content characteristics, and then accurately find the audience of the item and complete the personalized recommendation of the item. User and item feature representations are quickly obtained by using two mean convolutional layers. The characteristics between U-Is that match user preferences and item attributes will be more similar. User and item feature extraction is an important part of building a recommendation system and plays a key role in personalized recommendation.

Firstly, the U-I interaction data is modeled by graph structure, and the connection relationship is established between the user items that interact with each other. After the graph structured modeling is completed, the user preference feature and item attribute feature are initialized, and then the user item feature is accurately modeled, so as to fully mine the user preference information and item attribute information. The method optimizes the features of users and items through two layers of mean convolution layers, in which the feature learning of each layer of target nodes is obtained by averaging the features of the neighbor nodes on the node and the features of the node. After the user and item feature modeling is completed, the user features and item features have a strong distribution regularity, and then the user preference and item attribute information are preliminarily extracted. The mean convolution process greatly reduces the space complexity of the model while ensuring the feature accuracy. The module process framework is shown in Figure 2.

2. U-I interactive feature extraction module

The second module is the U-I interaction feature extraction module, the function of this module is to extract the interaction feature expression between user item pairs. The U-I interactive feature extraction module integrates the graph attention mechanism after the extraction of user preference information and item attribute information is completed, and directly integrates user features and item features on the graph neural network to learn its interactive feature expression. U-I interaction features are the input features of the interaction inference module.

First, splicing user features and user first-order neighbor node features, as well as splicing recommended item features and its first-order neighbor node features, and then inputting the spliced features into the self-attention network to obtain the attention coefficient of the module, where self-attention Force Network The present invention adopts multi-layer fully connected layer network modeling. Finally, the final interaction feature expression is obtained by aggregating the features of the U-I target node and its neighbor nodes through the learned attention coefficient. The specific process of this module is shown in Figure 3.

3. Interactive speculation module

The third module is the interactive inference module. The function of this module is to learn the distribution law of the interactive features after the U-I interactive feature extraction module extracts the interactive features between the user and the target item, and then obtain the user's inference score for the target item. The interactive inference module is an indispensable step for building recommendations. There are many design methods for the interactive inference module, such as traditional feature inner product, logistic regression algorithm, etc. However, these traditional algorithms cannot model the high-dimensional feature relationship between users and items well , so the present invention adopts the classical DNN algorithm to fuse the feature information of users and items to obtain more accurate prediction results.

After obtaining the interaction feature expression between the user and the target item, the interaction vector is directly input into the DNN network to obtain the initial prediction of the model. Then, the predicted value of the module is normalized by the sigmoid function, and the user's predicted score of the target item is modeled as a preference probability expression. The flow of this module is shown in Figure 4.

4.Top-N recommended module

The last module of the present invention is the top-N recommendation module, and the top-N recommendation mechanism is also the most commonly used mechanism for building a recommendation system. After getting the evaluation prediction of the target user's treatment of all items in the recommendation list. Sort all items in descending order according to the scores, and recommend the top N items to the user to realize the user's personalized recommendation.

Description of drawings

Fig. 1 is the overall block diagram based on the interaction graph neural network;

Fig. 2 is the modeling module framework of user preference information and item attribute information;

Figure 3 shows the framework of the U-I interactive feature extraction module

Figure 4 shows the framework of the interactive speculation module

Detailed ways:

The invention proposes a personalized recommendation method based on the interaction graph neural network. The concrete realization steps of this invention are as follows:

Step 1: Select a public recommendation data set, number all users and items, and randomly select 90% of the items that each user interacts with as the training set, and the remaining 10% of the items as the test set. Each of the training and test sets consists of three parts: users, items, and labels. For items that interact with the user, the label of the piece of data is 1, otherwise the label is 0. Through undirected graph structured representation of all pieces of data labeled as 1 in the training set, a connection relationship is established for users and items with interactive behaviors.

Step 2: After completing the undirected graph structuring of the training set, randomly initialize the high-dimensional feature representations of all user nodes and item nodes in the graph, that is, randomly initialize user preference information and item attribute information. Then, according to the node connection relationship of the undirected graph, a two-layer mean convolution layer network is built. All node features in each layer mean convolution layer network are obtained by aggregating the node features in the previous layer network and its first-order neighbor nodes. The features are obtained, where the aggregation method is average processing, and its mathematical expression is:

表示第K层均值卷积层的用户节点u的特征。N(u)表示用户节点u的一阶物品邻居节点。N(v)表示物品节点v的一阶用户邻居节点。

表示第K层均值卷积层的物品节点v的特征。MEAN表示均值化处理，即相关U-I特征每个维度求其平均值。在经过多层均值卷积层处理之后，无向图中所有节点特征有了较大的分布规律性，偏好、属性相似的用户节点和物品节点特征会比较相似。用户偏好信息和物品属性信息得到了初步地建模。in

Represents the feature of the user node u of the K-th mean convolutional layer. N(u) represents the first-order item neighbor nodes of user node u. N(v) represents the first-order user neighbor nodes of item node v.

Represents the feature of the item node v of the K-th mean convolutional layer. MEAN stands for mean processing, that is, the average value of each dimension of related UI features. After the multi-layer mean convolution layer processing, the characteristics of all nodes in the undirected graph have a greater distribution regularity, and the characteristics of user nodes and item nodes with similar preferences and attributes will be similar. User preference information and item attribute information are preliminarily modeled.

Step 3: After the user item feature modeling is completed, for the target user and the item node to be recommended in the undirected graph, the graph attention mechanism is integrated, and the user and its first-order neighbor features, items and their first-order neighbor features are aggregated to obtain the U-I. Interaction feature representation between pairs. Splicing user features and user first-order neighbor node features, as well as splicing recommended item features and its first-order neighbor node features, and inputting the spliced features into the attention network to obtain the attention coefficients corresponding to the user and item features. And perform softmax normalization on the obtained attention coefficient. Among them, the graph attention network adopts two fully connected layers for modeling. The advantages of building a self-attention mechanism network with multi-layer fully connected layers are obvious. It can adapt to the number of first-order neighbor nodes of an undirected graph node, and can effectively model the importance of a node's first-order neighbors to the node. In turn, a more accurate attention coefficient is obtained. Among them, the mathematical expression of the attention coefficient modeling is:

表示的是无向图中目标用户u_i的节点特征。h_a表示的是节点a的特征，节点a是用户u_i及其一阶邻居N(i)的集合中任一节点。N(i)表示用户u_i的一阶物品邻居集合。

表示拼接处理。

表示的是待推荐物品v_j的节点特征。h_b表示的是节点b的特征，节点b是物品v_j及其一阶邻居N(j)的集合中任一节点。N(j)表示物品v_j的一阶用户邻居集合。

和

表示将拼接得到的特征通过attention网络初步得到的权重系数。随后进行softmax归一化处理，得到目标用户及其一阶邻居的attention系数α_ia和待推荐物品及其一阶邻居的attention系数β_jb。Among them, W ₁ , W ₂ represent the parameter matrix of the first and second layers of the two-layer attention network, b ₁ , b ₂ represent the deviation coefficients of the first and second layers of the two-layer attention network, and σ represents the nonlinear activation function, the present invention adopts the Relu activation function.

Represents the node features of the target user _ui in the undirected graph. h _a represents the feature of node a, which is any node in the set of user _ui and its first-order neighbor N(i). N(i) represents the set of first-order item neighbors of user _ui .

Indicates the splicing process.

Represents the node features of the item v _j to be recommended. h _b represents the characteristics of node b, which is any node in the set of item v _j and its first-order neighbor N(j). N(j) represents the set of first-order user neighbors of item v _j .

and

Indicates the weight coefficient initially obtained by passing the spliced features through the attention network. Then perform softmax normalization to obtain the attention coefficient α _ia of the target user and its first-order neighbors, and the attention coefficient β _jb of the item to be recommended and its first-order neighbors.

After obtaining the attention coefficients of the user and its first-order neighbors and the target item and its first-order neighbors, the target node features are weighted by the attention coefficients to obtain the final interactive feature expression. Its mathematical expression is:

z _ij is the obtained interactive feature expression of the target user _ui and the item to be recommended v _j .

Step 4: After obtaining the interactive feature expression between the user and the target item, the model learns the distribution law of the interactive feature through the interactive inference module, and then obtains the user's accurate item rating for the item. Interactive inference module This method uses the classic DNN network. DNN networks can effectively learn feature distributions and model nonlinear relationships between users and items. The interactive inference module directly inputs the interactive features into the DNN network to obtain the user's predicted score for the item, and then uses the sigmoid normalization process to model the model's predicted score as the user's probability expression for the target item. Its mathematical expression is:

g ₁ =z _ij (8)

g ₂ =σ(W ₁ ·g ₁ +b ₁ ) (9)

g ₃ =σ(W ₂ ·g ₂ +b ₂ ) (10)

r′ _ij =sigmoid(W ₃ ·g ₃ ) (11)

Wherein W ₁ , W ₂ , W ₃ represent the parameter matrix of the DNN network, b ₁ , b ₂ represent the deviation coefficient in the DNN network, σ represents the nonlinear activation function, and the present method adopts the Relu activation function. g ₁ , g ₂ , g ₃ are the interaction vector representations of the output of each layer of the DNN network. r′ _ij is the final probability prediction evaluation value expression obtained after sigmoid normalization.

Step 5: In order to optimize the parameters of the model and the feature expression of user item nodes. By constructing the fitting degree of the cross-entropy loss function modeling model, this method minimizes the loss function value through the stochastic gradient descent algorithm, and then achieves the effect of optimizing the model parameters and node characteristics. Among them, the mathematical expression of the cross entropy loss function is:

Where |O| represents all user item node pairs extracted in the undirected graph when training the model, ri _ij represents the label value of the target user _ui and the item to be recommended v _j , and the value range is {0,1}. A label of 0 indicates that the attribute information of item v _j does not conform to the preference information of user _ui , which is used as a negative sample for model training. The label of 1 indicates that the user _ui interacts with the item _vj , which belongs to the positive sample data for model training. r′ _ij represents the predicted score of the model. The stochastic gradient descent algorithm is used to minimize the difference between r′ _ij and the actual label r _ij to optimize the loss function value, and then optimize the parameter matrix of the model to effectively extract user preference information and item attribute information.

Step 6: After the model training is completed, in order to verify the effectiveness of the present invention, the algorithm of the present invention is tested on the data sets huaban and Amazon-Book. After the model training is completed, for the test set of each data set, negative samples are randomly collected at a ratio of 1:100 to participate in the prediction evaluation of the model. At the same time, in order to ensure the validity of the model evaluation, the negative samples collected in the test set did not participate in the training in the training set. After obtaining the predicted evaluation value of all items of each user by the model, sort the items participating in the evaluation from large to small according to the output value of the model, and use the top-N recommendation mechanism to select the top N items after the sorting is completed and recommend them to the user. Target users. And evaluate the effectiveness of the method through effective evaluation indicators. Tables 1 and 2 show the comparison between the algorithm of the present invention and the recall evaluation method of some cutting-edge recommendation algorithms. It can be seen that the algorithm is superior to other recommended algorithms shown.

Table 1: Performance comparison on Amazon-book dataset

Table 2: Performance comparison on the huaban dataset

Claims (3)

1. A recommendation method based on a graph interaction network is characterized in that the following modules are designed and applied: the system comprises a user preference information and article attribute information optimization module, a U-I interactive feature extraction module, an interactive speculation module and a Top-N recommendation module;

firstly, carrying out graph structural modeling on user and article relation data; the user preference information and article attribute information optimization module extracts user characteristics and article characteristic expression by stacking two layers of mean convolution layers; the U-I interactive feature extraction module is used for fusing target user features and the features of the articles to be recommended to obtain implicit feature expression on the graph interactive network by using a graph attention mechanism, namely interactive vectors of U-I relation pairs; the interaction conjecture module learns the distribution rule of the interaction characteristic vectors through a DNN network, models a nonlinear relation between U-I and obtains the prediction score of the user on the target object; and finally, sequencing the target articles through a top-N recommendation module to realize the personalized recommendation task for the user.

2. The method of claim 1, wherein:

1) user preference information and item attribute information optimization

The first module is a user preference and article attribute feature optimization module, and the purpose of user preference information modeling is to obtain accurate user preference feature representation according to user interaction behaviors; obtaining user and article characteristic expressions through two layers of mean convolution layers; the distribution of users and articles in the feature space, wherein the user preferences and the article attributes are consistent, is closer;

firstly, carrying out graph structural modeling on U-I interactive data, and establishing a connection relation between user articles with interactive behaviors; then, initializing user preference characteristics and article attribute characteristics, wherein the user characteristics and the article characteristics are 128-dimensional high-dimensional vectors; optimizing feature representation of users and articles through a mean value convolution layer, wherein feature learning of each layer of target nodes is obtained by averaging processing based on the feature of a neighbor node on the upper layer of the node and the feature of the node, and the feature dimensions of the nodes of each layer of mean value convolution layer are the same;

2) U-I interactive feature extraction module

The second module is a U-I interactive feature extraction module and is used for extracting interactive feature expression between a target user and an article to be recommended; the U-I interactive feature extraction module directly utilizes an attention mechanism network on a graph interactive network to fuse the target user feature and the first-order neighbor feature thereof, the object feature to be recommended and the first-order neighbor feature thereof to obtain an interactive feature expression of the object feature;

firstly, splicing a user characteristic and a first-order neighbor node characteristic of a user, and splicing a to-be-recommended article characteristic and a first-order neighbor node characteristic thereof; inputting the splicing characteristics into a self-attention network to obtain an attention coefficient of the module, wherein the self-attention network adopts two-layer full-connection layer network modeling; finally, aggregating corresponding U-I target node characteristics and neighbor node characteristics thereof through the attention weight coefficient obtained by learning to obtain a final interactive characteristic vector;

3) an interaction speculation module

The third module is an interactive conjecture module, learns the information distribution of the U-I interactive characteristics and calculates the conjecture score of the user on the target object; fusing characteristic information of users and articles through a classical DNN network to obtain a conjecture result;

after the interactive feature expression of the user and the target object is obtained, the interactive vector is directly input into a DNN network to obtain the initial prediction of the model; carrying out normalization processing on the module predicted value through a sigmoid function, and modeling the prediction score of a user on a target article into preference probability expression;

4) Top-N recommendation module

The last module is a top-N recommendation module, and evaluation value prediction of all articles in a list to be recommended by a target user is obtained; and sorting all the articles in a descending order according to the scores, recommending the first N articles to the user, and realizing the personalized recommendation of the user.

3. The method according to claim 1, characterized by the following steps:

firstly, carrying out graph structural modeling according to U-I interaction data; establishing a connection relation between users with interaction records and articles, wherein each user and each article exist in the form of nodes of an undirected graph;

1) modeling of user preference information and item attribute information

After the graph structural modeling is completed, node characteristics of the U-I relational graph are initialized randomly, and any node characteristic is a 128-dimensional high-dimensional vector; according to the undirected graph node connection relation, a two-layer mean convolution layer network is built, all node characteristics in each layer of mean convolution layer network are obtained by aggregating the node characteristics in the upper layer network and the first-order neighbor node characteristics of the node characteristics, the aggregation mode is an averaging processing mode, and the mathematical expression of the aggregation mode is as follows:

wherein

Representing the characteristics of a user node u of the K-th mean convolution layer; n (u) represents a first-order article neighbor node of the user node u; n (v) represents a first order user neighbor set of an item node v;

the characteristic of an article node v of the K-th mean convolution layer is represented; MEAN represents the averaging process, i.e. averaging is carried out on each dimension of the relevant U-I characteristics;

2) U-I interactive feature extraction module

After modeling of the user object features is completed, fusing a graph attention force mechanism aiming at a target user and an object node to be recommended in an undirected graph, and aggregating the user and object features and first-order neighbor features thereof to obtain an interactive feature expression of the user and object features;

firstly, splicing a user characteristic and a first-order neighbor node characteristic of a user, splicing an article characteristic to be recommended and a first-order neighbor node characteristic thereof, inputting the spliced characteristic into a self-attention network to obtain an attention coefficient corresponding to the user and the article node characteristic, and carrying out softmax normalization processing on the attention coefficient; wherein, the graph attention network adopts two fully-connected layers for modeling; the mathematical expression for modeling the attention coefficient is as follows:

wherein, W₁，W₂Parameter matrix representing the first and second layers of two-layer attention network, b₁，b₂Expressing the deviation coefficients of a first layer and a second layer of the two-layer attention network, expressing a nonlinear activation function by sigma, and adopting a Relu activation function;

representing target user u in undirected graph_iThe node characteristics of (a); h is_aRepresenting the characteristics of node a, which is user u_iAnd any node in the set of its first-order neighbors N (i); n (i) denotes user u_iThe first order item neighbor set of (1);

representing splicing treatment;

showing the item v to be recommended_jCharacteristic of the node；h_bRepresenting the characteristics of node b, which is the item v_jAnd any node in the set of its first-order neighbors N (j); n (j) denotes an article v_jThe first-order user neighbor set of (1);

and

representing a weight coefficient obtained by primarily splicing the obtained features through an attention network; then, performing softmax normalization processing to obtain the attention coefficient alpha of the target user and the first-order neighbor thereof_iaAnd the attention coefficient beta of the item to be recommended and the first-order neighbor thereof_jb；

After obtaining attention coefficients of a user and first-order neighbors thereof, a target article and first-order neighbors thereof, fusing the attention coefficients to weight target node characteristics to obtain a final interactive characteristic expression; the mathematical expression is as follows:

z_ijfor the obtained target user u_iAnd its item to be recommended v_jThe interactive feature expression of (2);

3) an interaction speculation module

Inputting the interactive characteristics into a DNN network to obtain a prediction score of a user on the object, and then modeling the prediction score of the model as a probability expression of the user on the target object through sigmoid normalization processing; the mathematical expression is as follows:

g₁＝z_ij(8)

g₂＝σ(W₁·g₁+b₁) (9)

g₃＝σ(W₂·g₂+b₂) (10)

r′_ij＝sigmoid(W₃·g₃) (11)

wherein W₁，W₂，W₃To representParameter matrix of DNN network, b₁，b₂Expressing deviation coefficients in the DNN network, expressing a nonlinear activation function by using a Relu activation function; g₁，g₂，g₃Is the interaction vector expression output by each layer of the DNN network; r'_ijThe final probability prediction evaluation value expression is obtained after sigmoid normalization;

4) Top-N recommendation module

After the predicted evaluation values of the model for all the articles of each user are obtained, the articles participating in evaluation are sorted from large to small according to the output values of the model, the top N sorted articles are made by a top-N recommender and recommended to the target user, and further personalized recommendation for the user is completed.

Images