CN108984904B - Home design method based on deep neural network - Google Patents

Abstract

The invention relates to a home design method based on a deep neural network, and belongs to the technical field of deep neural networks and home design. Firstly, collecting a home design display to label a home, and then generating sequence data based on a marked home placing sequence; then designing a home furnishing prediction model, extracting structural features among furniture characters, learning sequence data and predicting a home furnishing placing sequence based on the designed home furnishing prediction model; designing a bidirectional layered home prediction model based on parameter constraint of the relative size of a home object, and then performing multi-round prediction of hierarchical recursion to enable multi-round prediction results to better accord with the actual situation of home design; finally, a home prediction model aiming at a specific style is designed according to the requirements of different design styles in practical application. The invention uses the three-dimensional engine to draw the three-dimensional home scene, and visually proves the effectiveness and the style learning advantage of the model in a three-dimensional mode.

Description

A home design method based on deep neural network

technical field

The invention relates to a home furnishing design method based on a deep neural network, and belongs to the technical field of deep learning.

Background technique

Deep Neural Network (DNN) is the foundation of many modern AI applications. Different from traditional machine learning algorithms, it can learn features from data autonomously. This process does not require human intervention, is more intelligent, and is more in line with the mechanism of human perception of the world. In addition, in many fields, the accuracy of DNN has surpassed that of human beings. As a parent class, DNN contains the main algorithms can be divided into four systems: convolutional neural network, deep stack auto-encoding network, recurrent neural network, generative confrontation network. Among them, the convolutional neural network is deepened in space, and the recurrent neural network is deepened in time.

Recurrent Neural Network (RNN) is mainly used in natural language processing in practical applications. Its special feature is that the influence of context information on the result output is considered. The current output y _t of the network is not only related to the current input x _t , but also Related to the state h _t-1 of the hidden layer at the previous moment, the new hidden layer state h _t is obtained according to h _t-1 and x _t , which determines the output at the current moment and affects the next moment. Due to the particularity of RNN, in the process of backpropagation, it not only depends on the network of the current layer, but also depends on the network of several previous layers. This propagation algorithm is called backpropagation through time (BPTT). ).

In the current research on home design methods, Gaussian models and Bayesian networks are mostly used to train models to extract structural features between objects. These algorithms are all based on intelligent computing methods, and there is a certain room for improvement in model simplification. Able to learn the design style, the work of the algorithm can be roughly divided into three categories: furniture layout optimization, scene modeling and reconstruction, scene synthesis, these methods have some problems, such as modeling and reconstruction, the demand for collecting specific data is high ( Such as Kinect data), scene synthesis needs to extract overall features and always control the similarity between the generated scene and the prototype.

SUMMARY OF THE INVENTION

The purpose of the present invention is to propose a home furnishing design method based on a deep neural network in view of the technical defects that the existing home furnishing prediction model still has a certain room for improvement in model simplification and the design style cannot be learned.

The core idea of the present invention is as follows: firstly collect the home furnishing design display to mark the home furnishing sequence, and then generate sequence data based on the marked home furnishing sequence, that is, the home furnishing order; and then design a sequential structure home furnishing prediction model, based on the designed sequential structure home furnishing The prediction model extracts the structural features between furniture characters, learns sequence data, and predicts the order of home placement; introduces parameter constraints that limit the relative size of home objects, and builds a bi-directional hierarchical home prediction model. The prediction tasks of each layer are given in advance. Determine the parameter constraints of this layer, and use cluster search to filter parameters in the prediction stage, that is, perform hierarchical recursive multi-round prediction, and the prediction results are more in line with the actual situation of home design; According to the requirements of different design styles in the application, the 3D engine is used to draw 3D home scenes, which directly proves the validity of the model and the advantages of style learning in a 3D way.

The object of the present invention is achieved through the following technical solutions:

A home furnishing design method based on a deep neural network includes a sequential structure home furnishing prediction model and a bidirectional hierarchical home furnishing prediction model, and the realization method is divided into a data preprocessing stage, a training stage, a prediction stage and a model drawing stage;

Among them, the data preprocessing stage of the sequential structure home furnishing prediction model includes the following steps:

Step 1. Collect and obtain each group of home furnishing data, and manually mark them in order to obtain a home furnishing sequence, which is divided into two parts: an input sequence and an output sequence, to generate a complete home furnishing data set;

Step 2. Create the corresponding household data set vocabulary, the vocabulary mainly includes special tags and household tags;

The training phase of the sequential structure home furnishing prediction model includes the following steps:

Step 3. The sequence encoding process, specifically: taking the input sequence of step 1 as the input of the cyclic neural network model, calculating and obtaining a state sequence, which relatively completely retains the structural information of the input sequence;

Step 4. The sequence decoding process, specifically: the weighted summation of the state sequence obtained in step 3 to obtain the current state semantic vector, which is used to determine where the model needs to pay more attention to the state sequence, so that the home The structure information can be transmitted in all neural network models, the current state semantic vector is used as the input of the recurrent neural network model, the probability distribution of the output sequence is calculated, and the sequence structure home sequence prediction model based on the recurrent neural network is obtained;

The prediction stage of the sequential structure home prediction model, specifically:

Step 5. Input a group of partial household sequences into the model obtained in step 4 to generate and obtain a complete household sequence;

The data preprocessing stage of the bidirectional hierarchical home furnishing prediction model includes the following steps:

Step 6. Add size parameters to the home data set in step 1, and create a home data set with size parameters;

Step 7. Parametrize all household conventional sizes and expand them into the vocabulary of step 2, create a hierarchical household model vocabulary, and the vocabulary mainly includes special marks and household marks;

Step 8. A complete household sequence is defined as a set D of subsequences, where D={U ₁ , U ₂ , ..., U _2k-1 , U _2k }, and each subsequence U _m contains several household model vocabulary, namely U _m ={w _m,1 ,w _m,2 ,w _m,3 ...w _m,n }, w _m,n represents the position n of the word in the mth subsequence U _m , which can be represented as the step 7 In the household sequence, U _2i-1 represents the input subsequence, and U _2i represents the output subsequence, where i=1,2,...,k;

The training phase of the bidirectional hierarchical home furnishing prediction model, the process of hierarchical recursive structure modeling and implementation, includes the following steps:

Step 9. As shown in step 8, the home sequence collection is set, and the recursion times are set to k, and the initial value of i is set to 1;

Step 10. The sequence encoding process, specifically: taking the input subsequence U _2i-1 of step 8 as the input of the cyclic neural network, and calculating the encoding vector corresponding to the input subsequence;

Step 11. Use the coding vector corresponding to the input subsequence output in step 10 as the input of the cyclic neural network, and calculate the context vector corresponding to the input subsequence moment;

Step 12. The sequence decoding process is specifically as follows: the context vector output in step 11 and the output subsequence U _2i in step 8 are used in parallel as the input vector of the cyclic neural network, and then the output subsequence is predicted to generate the corresponding output subsequence. The word probability distribution of ;

Step 13. Set i=i+1, use the output subsequence of step 12 and the next input subsequence U _2i-1 to combine as a new input subsequence, and iteratively execute steps 10 to 12 until i=k has been traversed. For all subsequences, a hierarchical recursive structure home prediction model based on recurrent neural network is obtained;

The prediction stage of the bidirectional hierarchical home furnishing prediction model includes the following steps:

Step 14. Based on the home furnishing prediction model in step 13, input the home furnishing subsequence, and predict the corresponding output subsequence;

Step 15. Use beam search to filter and sort the prediction options. Among all the sequences that meet the constraints, select the sequence with the highest probability score as the target output, and form the current hierarchical home prediction sequence with the input, and the final prediction is generated. Complete home sequence;

The model drawing stage is specifically: using 3D software to model the scene, retrieving the corresponding model in the model library for the predicted home sequence scheme, and drawing it in the scene, while preparing data sets of different styles, according to the predicted sequence Retrieve the corresponding style home in the model library to draw;

The model drawing stage is the same for the sequential structure home prediction model and the bidirectional hierarchical home prediction model.

beneficial effect

Compared with the prior art, a home furnishing design method based on a deep neural network of the present invention has the following beneficial effects:

1. The present invention effectively extracts the dependent structural features and completes the home furnishing order prediction;

2. Effectively predict the structural relationship and size parameters between home furnishing characters according to parameter constraints;

3. The realization of home design is more intelligent, and three-dimensional visualization is carried out at the same time.

Description of drawings

Fig. 1 is a kind of home furnishing design method based on deep neural network of the present invention and the flow chart of sequential structure home furnishing prediction model in the embodiment;

FIG. 2 is a flowchart of a bidirectional hierarchical home furnishing prediction model in a home furnishing design method based on a deep neural network and an embodiment of the present invention.

Detailed ways

The embodiments of the method of the present invention will be described in detail below with reference to the accompanying drawings, but the embodiments of the present invention are not limited thereto. The accompanying drawings constituting a part of the present application are used to provide further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

Example 1

A home furnishing design method based on deep neural network, firstly proposes a sequential structure home furnishing prediction model based on dependency features, and designs and uses the model to extract structural features between home furnishing characters and learn the prediction of sequential placement sequence. The specific implementation steps are as follows:

Step 1: Labeling of home data sets and data set arrangement

Collect the home design and display from professional website home design drawings and home floor plans. The standard used in the pictures is that the space is relatively complete and the home display meets the specifications of the medium-sized apartment. Label the homes involved. In the choice of home labeling, it tends to be volume. Relatively large and individual objects are relatively independent, and English words are used to represent the types of objects. The household objects are placed around the wall for a week. The order of these households can be changed into a sequence in a clockwise order; in the sequence Corner (corner) is added to indicate the end of home furnishing output on the first wall and the beginning of home furnishing output on the second wall, which is equivalent to the identification of home group partitions in a certain direction; in the data set, each set of sequence data is represented by Four groups of household objects are formed to maintain the integrity of the characteristic information of the data; the data set is organized into corresponding files of "input" and "output", and .enc (input-Encoder) and .dec (output-Decoder) files are generated.

Step 2, vocabulary creation

The content of the vocabulary file is the related household category, and the function is to convert the input and output into the form of the corresponding id. In the purely structural order household prediction model, the vocabulary contains 32 words (28 household categories and 4 special symbols), the content in the vocabulary: _PAD_GO_EOS_UNK chair divan lounge sofa flowerpot..., the first four kinds of symbols are special symbols, which are used to fill in the input and output of the symbol. _GO marks the start of input, _EOS marks the end of input, _UNK marks characters that do not appear in the vocabulary, and _PAD is used to pad sequences to ensure that sequences in the same batch have the same length.

The ids file converted into input and output, each line in .enc is an input, and each line in .dec is an output. Each id in each row represents the word at the corresponding vocabulary position in the input or output.

Step 3, training of sequential structure home furnishing prediction model

The model has three modes, train, test, serve. Use the train mode for training, the test mode for testing, and the serve mode for verification. The Encoder part of the model selects LSTM, the Decoder part searches for and outputs the optimal prediction sequence during decoding, and the model joins the Attention mechanism. The Encoder is not related to the Attention part. Ordinary Encoder-Decoder models are used for natural language processing tasks. In terms of home design, it is more hoped to predict some dependency structures, symmetrical structures, and global layout distribution. The model flow is shown in Figure 1.

The Encoder part uses LSTM to make it learn to search for subsequence representations corresponding to the source sequence, read the input sequence (x ₁ ,...,x _Tx ) in order, and calculate the current hidden layer state sequence (s ₁ ,...,s _t ); obtain the hidden layer state s _t of each word x _t , the current hidden layer state s _t contains the semantic information of the forward word, and retains the structural information of the input sequence relatively completely; the input gate i processes the input parameters , the forgetting gate f sets the weight of selective forgetting, and discards part of the information, the output gate o processes the output parameters, and then calculates the state of the memory unit c _t , as shown in formula (1):

为点乘，tanh为激活函数。Among them, x _t represents the input data of the network at time t, s _t-1 represents the neuron state of the hidden layer at time t-1, U and W are the weight matrices in the LSTM model, f and i are the forgetting gate and the input gate,

is the dot product, and tanh is the activation function.

Calculate the activation value of the hidden layer of the LSTM at time t, as shown in formula (2):

为点乘，tanh为激活函数。where o is the output gate,

is the dot product, and tanh is the activation function.

Use the hidden layer state information of each input obtained by the Encoder to predict the target output. For each predicted conditional probability, it is calculated as shown in formula (3):

p(y _i |y ₁ ,...,y _i-1 ,x)=g(y _i-1 ,s _i , _ci ) (3)

where s _i is the hidden layer state of the RNN at time _i , g is the softmax activation function, and ci is the semantic vector.

c _i is determined by the _hidden layer state sequence (h ₁ , _. The semantic vector c _i is obtained by the weighted summation of h _i ; the weight parameter α _ij or its associated parameter e _ij embodies the importance of h _j and provides important information for the next hidden layer state s _i and output y _i , This is also the mechanism of the Attention Model, and the Decoder part can determine which position of the source sentence needs to pay more attention based on this information.

Example 2

A home furnishing design method based on deep neural network further introduces parameter constraints that limit the relative size of home objects, and builds a bidirectional hierarchical home furnishing prediction model. In the prediction stage, the cluster search is used to filter the parameters, and the scheme design of the household sequence is completed. The model process is shown in Figure 2. The specific implementation steps are as follows:

Step 1, build a hierarchical phrase-level household data set

Append a number after each household character, that is, the size of the number represents the size of the household, here it represents the length of the wall relative to it. For each type of household, set several different sizes that are commonly used; next Constructing hierarchical input and output pairs. Due to the introduction of size parameters, each wall needs to be predicted separately. A pair of input and output sequence pairs in a non-sequential structure constitutes a complete set of sequences. The hierarchical model Four inputs and predicted outputs are required to implement the full scheme, so the dataset form is more like an array of sequence pairs.

In order to construct the sequence pair array, a set form of 8 elements {s1, s2, s3, s4, s5, s6, s7, s8} is introduced here, which represents the four-round "dialogue" process between the input x and the output y, which is Capture the interactive prediction process, append a special end-of-sequence marker to all training set sequences, and add a flag from the same input or the same output to indicate whether it is currently in the input or output stage. The end of each sequence is marked with </s> to mark the end of the current sequence prediction, and the end of every four rounds of prediction of a complete home sequence is marked with </d>; the <x> mark is currently the input stage, and the <y> mark is currently output stage.

Step 2: Create a Hierarchical Home Model Vocabulary

The format of the hierarchical home model vocabulary is similar to the sequential structure format, and its function is to convert the home characters into the corresponding ids. Unlike the sequence structure, parameter constraints are introduced, so the sequence is based on the phrase level. Based on this, the general dimensions of all households are counted and parameterized, and expanded into the vocabulary. In addition, the table also contains extended special characters, and the corresponding format is shown in Table 1:

Table 1 Glossary Contents List

Glossary content meaning _PAD sequence padding _GO enter start _EOS end of input _UNK strange characters </s> end of subsequence prediction </d> The fourth round of predictions ends chair Home Character - Chair divan Home character - couch lounge Home Character - Recliner ... home character (0.3) parameter-0.3m (0.4) parameter-0.4m (0.5) parameter-0.5m (0.8) parameter-0.8m ... parameter

There are 28 categories of household characters in the vocabulary, 30 description parameters, and 6 special marks. The vocabulary is divided into two corresponding tables: input and output, and the contents of the two tables are basically the same.

Step 3: Two-way hierarchical home furnishing prediction model training

The model consists of three recurrent neural network modules: Encoder, Decoder and Context. The Encoder network module maps each subsequence into a subsequence vector, which is the hidden layer state obtained after the last character in the subsequence is processed. The Context network module realizes the learning and memory of high-level information. It tracks the previous semantic information by iteratively processing each subsequence vector. The Decoder network module utilizes the hidden layer state of the Context network module and generates the probability of the corresponding character when the next subsequence is predicted. distributed;

Consider a complete household sequence (four sets of inputs and predictions) as a set D of eight subsequences, where D={U ₁ ,U ₂ ,U ₃ ,U ₄ ,U ₅ ,U ₆ ,U ₇ ,U ₈ } , involving two interactive parties of the input end and the output end, and each U _m contains several household characters, which means U _m ={w _m,1 ,w _m,2 ,w _m,3 ...w _m,n }, w _m,n is a random variable taken from the value in the vocabulary, representing its position n in the mth subsequence w. The characters w _{m, n} represent both the meaning of the home content and the current behavior state, that is, whether it is the input stage or the prediction stage or the end of the subsequence or the end of the round of prediction. The prediction model parameterizes the distribution of the probability P The process is controlled by the parameter θ in the set of all possible home complete sequences of arbitrary length. The probability P of one round of prediction process can be decomposed as shown in Equation (4):

Further division by character granularity can be decomposed as shown in formula (5):

Among them, U _<m ={U ₁ ,...,U _m-1 }, w _m,<n ={w _m,1 ,...,w _m,n-1 }, which means that the subsequence U _m is before n characters, the prediction is a fixed four-fold hierarchical recursive process, and the number of turns of the dialogue is indeterminate. The sampling of the model can be carried out according to the standard dialogue model, that is, based on the conditional probability p distribution P _θ (w _m,n |w _m,<n ,U _<m ) generated based on the previously sampled characters, one character is sampled from it at a time, A standard n-grams model is used to compute joint probabilities of individual complete home character predictions.

In the training phase, since the output of each Decoder has the correct answer, there is no need to perform redundant processing on the parameters. In the model testing stage, the beam search is used to screen and sort the alternatives, and among all the sequences that satisfy the constraints, the sequence with the highest probability score is selected as the target output, and the input constitutes the current hierarchical home design sequence.

A home design method based on deep neural network, which visualizes the home plan designed by the previous home model in 3D, showing the advantages of the home model based on deep learning in terms of learning style. Use Unity3D for scene modeling, retrieve the corresponding model in the model library for the home sequence scheme predicted above, and draw it in the Unity scene, and use the 3D scene drawing method to model the scene. The effect is more intuitive and vivid. . According to the prediction sequence, the corresponding Chinese-style or European-style furniture is retrieved in the model library, and the drawing effect is natural and elegant with obvious style characteristics.

When the number of training times of the sequential structure home prediction model reaches about 60K, the perplexity of the model is basically stable at about 1.21. It can be intuitively understood that in the average case, when the model predicts the next home character, it is considered that there are 1 to 2 words, etc. may be used as The reasonable choice of the next word indicates that the model can basically find a suitable next target prediction, the model can extract the structural relationship between household characters, and the predicted household design scheme has a certain scene rationality;

In the two-way hierarchical home furnishing prediction model, when the training reaches 50 epochs, the perplexity of the model is stable at about 2.52. The model can effectively predict the structural relationship and size parameters between home furnishing characters according to the parameter constraints, which is practical and reasonable, and is listed in the form of a table. A set of models predicts the home design scheme of the living room scene, and connects several sub-sequences of the living room forecast scheme in turn, there will be symmetrical structures such as "flowerpot, television, flowerpot", and "divan" can also be properly formed with "television". In the right layout, observing the parameter constraints, the length of each sub-sequence corresponding to each group of parameters finally screened will not exceed the length of the attached wall. According to prior knowledge, the size parameters of different home types will be in an appropriate Changes within the range, such as "flowerpot" will not be associated with the parameter "3.0".

Table 1: Living room forecast scenarios

The above descriptions are only the preferred embodiments of the present invention, and the present invention should not be limited to the contents disclosed in the embodiments and the accompanying drawings. All equivalents or modifications accomplished without departing from the disclosed spirit of the present invention fall into the protection scope of the present invention.

Claims (1)

1. A home design method based on a deep neural network is characterized by comprising the following steps: the method comprises a sequential structure home prediction model and a bidirectional hierarchical home prediction model, and the implementation method comprises a data preprocessing stage, a training stage, a prediction stage and a model drawing stage;

the data preprocessing stage of the sequential structure home prediction model comprises the following steps:

step 1, manually labeling each group of collected home furnishing data in sequence to obtain a home furnishing sequence, wherein the sequence is divided into an input sequence and an output sequence to generate a complete home furnishing data set;

step 2, creating a corresponding home data set vocabulary table, wherein the vocabulary table comprises special marks and home marks;

the training stage of the sequential structure home prediction model comprises the following steps:

and 3, a sequence coding process specifically comprises the following steps: taking the input sequence in the step 1 as the input of a recurrent neural network model, calculating and obtaining a state sequence, wherein the state sequence completely reserves the structural information of the input sequence;

step 4, sequence decoding process, which specifically comprises: weighting and summing the state sequences obtained in the step (3) to obtain a current state semantic vector, wherein the semantic vector is used for judging the position of the state sequence where the model needs to put more attention so that the structural information of the home is transmitted in all the neural network models, the current state semantic vector is used as the input of the cyclic neural network model, the probability distribution of an output sequence is obtained through calculation, and a home sequence prediction model with a sequential structure based on the cyclic neural network is obtained;

the prediction stage of the sequential structure household prediction model specifically comprises the following steps:

step 5, inputting a group of partial home sequence into the model obtained in the step 4, and generating and obtaining a complete home sequence;

the data preprocessing stage of the bidirectional layered home prediction model comprises the following steps:

step 6, adding size parameters into the home data set in the step 1, and creating a home data set with the size parameters;

step 7, parameterizing all household conventional sizes, expanding the parameterized household conventional sizes into the vocabulary in the step 2, and creating a hierarchical household model vocabulary, wherein the vocabulary comprises special marks and household marks;

step 8, a complete home sequence is defined as a set D of subsequences, wherein D ═ U ₁ ,U ₂ ,…,U _2k-1 ,U _2k Each subsequence U _m Comprises a plurality of home furnishing moldsType vocabulary, i.e. U _m ＝{w _m,1 ,w _m,2 ,w _m,3 …w _m,n }，w _m,n Indicates that the word is in the m-th sub-sequence U _m The position n in (1) represents the content meaning of the home mark or the current behavior state represented by the special mark as shown in step 7, and U in the home sequence _2i-1 Representing an input subsequence, U _2i Represents an output subsequence, wherein i ═ 1,2, …, k;

the training phase of the bidirectional layered home prediction model and the modeling and implementation process of the layered recursive structure comprise the following steps:

step 9, setting the recursion times as k and setting the initial value of i as 1 in the home sequence set shown in the step 8;

step 10, the sequence coding process specifically comprises: inputting the subsequence U of step 8 _2i-1 As the input of the recurrent neural network, calculating to obtain the coding vector corresponding to the input subsequence;

step 11, taking the coding vector corresponding to the input subsequence output in the step 10 as the input of the recurrent neural network, and calculating to obtain a context vector corresponding to the input subsequence at the moment;

step 12. the sequence decoding process specifically includes: the context vector output in step 11 and the subsequence U output in step 8 are compared _2i The parallel connection is used as the input vector of the cyclic neural network, and then the output subsequence is predicted to generate the vocabulary probability distribution corresponding to the output subsequence;

step 13. put i ═ i +1, use the output subsequence of step 12 and the next input subsequence U _2i-1 Combining the sub-sequences as new input sub-sequences, and iteratively executing the steps 10 to 12 until all the sub-sequences are traversed when i is k, so as to obtain a hierarchical recursive structure home prediction model based on the recurrent neural network;

the prediction stage of the bidirectional layered home prediction model comprises the following steps:

step 14, inputting the home subsequence based on the home prediction model in the step 13, and predicting a corresponding output subsequence;

step 15, screening and sequencing the prediction alternatives by using cluster searching, selecting one sequence with the highest probability score from all sequences meeting constraint conditions as target output, forming a current layered home prediction sequence with the target output and input, and finally predicting to generate a complete home sequence;

a model drawing stage, which specifically comprises the following steps: carrying out scene modeling by using three-dimensional software, retrieving a corresponding model from a model base for the predicted home sequence scheme, drawing in a scene, preparing data sets of different styles and types, and retrieving the home with the corresponding style from the model base according to the predicted sequence for drawing;

the model drawing stages of the sequential structure home prediction model and the bidirectional hierarchical home prediction model are the same.

Images