[go: up one dir, main page]

CN107992937B - Unstructured data judgment method and device based on deep learning - Google Patents

Unstructured data judgment method and device based on deep learning Download PDF

Info

Publication number
CN107992937B
CN107992937B CN201610950494.9A CN201610950494A CN107992937B CN 107992937 B CN107992937 B CN 107992937B CN 201610950494 A CN201610950494 A CN 201610950494A CN 107992937 B CN107992937 B CN 107992937B
Authority
CN
China
Prior art keywords
data
neural network
deep learning
clue
factor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201610950494.9A
Other languages
Chinese (zh)
Other versions
CN107992937A (en
Inventor
朱跃生
罗桂波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to CN201610950494.9A priority Critical patent/CN107992937B/en
Publication of CN107992937A publication Critical patent/CN107992937A/en
Application granted granted Critical
Publication of CN107992937B publication Critical patent/CN107992937B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/044Recurrent networks, e.g. Hopfield networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35Clustering; Classification

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)

Abstract

本发明涉及一种基于深度学习的非结构化数据判决方法,具体包括:获取训练后的深度学习神经网络模型,其中所述训练后的深度学习神经网络模型为经多因子训练样本数据训练后的多因子联合判决的神经网络模型;接收在线实时采集的线索数据,所述线索数据包括多种数据类型的非结构化数据;通过训练后的深度学习神经网络模型对获取的所述线索数据进行联合分析,提取有利于判决的特征线索性信息;根据所述特征线索信息对所述线索数据进行判决,生成判决结果;反馈所述判决结果。上述方法能够对非结构化数据进行更加高效、更加及时的决策分析,实现在线实时分析。

Figure 201610950494

The present invention relates to an unstructured data judgment method based on deep learning, which specifically includes: obtaining a trained deep learning neural network model, wherein the trained deep learning neural network model is the result of training with multi-factor training sample data. A neural network model for multi-factor joint judgment; receive clue data collected online in real time, the clue data includes unstructured data of multiple data types; combine the acquired clue data through a trained deep learning neural network model Analyze and extract characteristic clue information that is beneficial to judgment; make judgment on the clue data according to the characteristic clue information, and generate a judgment result; and feed back the judgment result. The above method can perform more efficient and timely decision analysis on unstructured data, and realize online real-time analysis.

Figure 201610950494

Description

Unstructured data judgment method and device based on deep learning
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an unstructured data judgment method and device based on deep learning.
Background
Data that is not conveniently represented in a database two-dimensional logical table is referred to as unstructured data, as opposed to structured data (i.e., row data, stored in a database, that can be logically represented in a two-dimensional table structure), which includes office documents of all formats, text, pictures, subsets XML in a standard universal markup language, HTML, various types of reports, images, and audio/video information, among others. The unstructured data has the characteristics of large data volume, fast change, multiple types, rich and complex content and non-uniform structure.
In the traditional technology, due to the characteristics of rapid change, non-uniform structure and the like of unstructured data, analysis time consumption is long when unstructured data are analyzed, off-line analysis is generally performed on historical data, and analysis decision efficiency is low.
Disclosure of Invention
In view of the foregoing, there is a need to provide a method and an apparatus for unstructured data decision based on deep learning, which can perform more efficient and timely online decision analysis on unstructured data.
An unstructured data judgment method based on deep learning, the method comprising:
acquiring a trained deep learning neural network model, wherein the trained deep learning neural network model is a multi-factor joint decision neural network model trained by a multi-factor training sample;
receiving clue data acquired online in real time, wherein the clue data is unstructured data comprising a plurality of data types;
performing joint analysis on the received clue data through the trained deep learning neural network model, and extracting characteristic clue information beneficial to judgment;
judging the clue data according to the characteristic clue information to generate a judgment result;
and feeding back the judgment result.
In one embodiment, the step of performing joint analysis on the received clue data through the trained deep learning neural network model to extract characteristic clue information beneficial to decision includes:
performing feature extraction on the received clue data through the trained deep learning neural network model to obtain a high-dimensional feature vector;
and converting the high-dimensional feature vector into a hash code with each dimension incidence relation by adopting a hash retrieval algorithm, and extracting feature clue information favorable for judgment according to the hash code.
In one embodiment, before the step of obtaining a trained deep learning neural network model, where the trained deep learning neural network model is a multi-factor joint decision neural network model trained by multi-factor training samples, the method further includes:
receiving a training sample uploaded by a terminal, wherein the training sample data is a multi-factor training sample;
under an off-line state, a deep learning neural network sub-model is constructed for the training sample corresponding to each factor in the multi-factor training samples by utilizing a deep learning algorithm;
acquiring an incidence relation between the built deep learning neural network submodels;
and fusing the deep learning neural network submodels according to the incidence relation to generate a deep learning neural network model capable of performing multi-factor joint decision.
In one embodiment, the multi-factor training samples include video data, image data, audio data, text data, and network data.
In one embodiment, when the clue data received within a set time contains a data type, the clue data is analyzed and judged by using the deep learning neural network submodel of the factor corresponding to the data type to obtain a judgment result.
An unstructured data decision device based on deep learning, the device comprising:
the neural network model acquisition module is used for acquiring a trained deep learning neural network model, wherein the trained deep learning neural network model is a multi-factor joint decision neural network model trained by a multi-factor training sample;
the real-time data receiving module is used for receiving clue data acquired online in real time, wherein the clue data is unstructured data comprising a plurality of data types;
the characteristic information extraction module is used for carrying out joint analysis on the received clue data through the trained deep learning neural network model and extracting characteristic clue information beneficial to judgment;
the joint judgment module is used for judging the clue data according to the characteristic clue information to generate a judgment result;
and the judgment result feedback module is used for feeding back the judgment result.
In one embodiment, the feature information extraction module is further configured to perform feature extraction on the received clue data through the trained deep learning neural network model to obtain a high-dimensional feature vector; and converting the high-dimensional feature vector into a hash code with each dimension incidence relation by adopting a hash retrieval algorithm, and extracting feature clue information favorable for judgment according to the hash code.
In one embodiment, the apparatus further comprises:
the training sample data receiving module is used for receiving a training sample uploaded by a terminal, wherein the training sample is a multi-factor training sample;
the sub-model training module is used for constructing a deep learning neural network sub-model for the training sample corresponding to each factor in the multi-factor training samples by utilizing a deep learning algorithm in an off-line state;
the incidence relation analysis module is used for acquiring the incidence relation between the built deep learning neural network submodels;
and the combined decision model building module is used for fusing the deep learning neural network submodel according to the incidence relation to generate a deep learning neural network model capable of carrying out multi-factor combined decision.
In one embodiment, the multi-factor training samples include video data, image data, audio data, text data, and network data.
In one embodiment, the apparatus further comprises: and the submodel judging module is used for analyzing and judging the clue data by using the deep learning neural network submodel of the factor corresponding to the data type to obtain a judgment result when the clue data received within the set time contains one data type.
According to the unstructured data judgment method and device based on deep learning, the multi-factor training sample is used for training the deep learning neural network model to obtain the neural network model capable of performing multi-factor joint judgment, the trained neural network model is used for performing joint analysis on real-time unstructured data with various data types, and characteristic clue information beneficial to data judgment is extracted so as to screen analysis data, and then data analysis judgment is performed more efficiently and timely to obtain a judgment result. Namely, the correlation among various types of real-time data is analyzed, and then a faster and more intelligent judgment scheme is confirmed, so that the online real-time judgment of the application data with large data volume and fast change is realized.
Drawings
FIG. 1 is a diagram of an application environment of an unstructured data decision method based on deep learning in one embodiment;
FIG. 2 is a flow diagram of a method for deep learning based unstructured data determination in one embodiment;
FIG. 3 is a flow diagram of training a deep learning neural network model in one embodiment;
FIG. 4 is a block diagram of an unstructured data decision method based on deep learning in one embodiment;
FIG. 5 is a block diagram of the architecture involved in training a deep learning neural network model in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, in one embodiment, an application environment diagram of an unstructured data decision method and apparatus based on deep learning is provided, the application environment includes a data acquisition device 110 and a server 120, wherein the data acquisition terminal 110 can communicate with the server 120 through a network. The data acquisition equipment can comprise a camera terminal, a network terminal, a data storage medium and the like, wherein the camera terminal can capture unstructured data such as video data, image data, audio data and the like; the network terminal can acquire unstructured data such as network data, text data, image data, video data and the like; the data storage medium stores unstructured data such as text data, image data, video data, and audio data collected through other offline channels in advance. The unstructured data including multiple data types acquired by the data acquisition equipment 110 can be received on line in real time through the network connection server 120, and the trained deep learning neural network model capable of performing multi-factor judgment is called to perform rapid judgment on the unstructured data of the multiple data types in real time to obtain a judgment result.
In an embodiment, as shown in fig. 2, an unstructured data decision method based on deep learning is provided, which is described by taking an application in the server 120 shown in fig. 1 as an example, and specifically includes the following steps:
step S202: and acquiring a trained deep learning neural network model, wherein the trained deep learning neural network model is a multi-factor joint decision neural network model trained by a multi-factor training sample.
In particular, deep learning refers to a multi-layer neural network, which may form more abstract high-level representation attribute classes or features by combining low-level features to discover a distributed feature representation of the data. The deep neural network is a process for extracting features layer by layer, a computer automatically extracts the features from data without human intervention, and the essential idea is to stack a plurality of neuron layers, each layer extracts certain features and information, and the output of the layer is used as the input of the next layer. In this way, hierarchical representation of the input information can be achieved. To discover a distributed characterization of the data. Taking image recognition as an example, the first layer extracts boundary information, the second layer extracts boundary contour information, then the contours can be combined into subsections, the subsections are combined into objects, thus the features are extracted layer by layer, a more abstract high-level representation attribute category or feature is formed by combining the features of the lower layer, and which kind of object is in the picture is judged through different combinations of the features or the attributes.
The deep learning neural network model has various types, such as a convolutional neural network model, a denoising autoencoder, a Restricted Boltzmann Machine (RBM) network model, and the like. When the deep learning neural network model is trained, any one of the network models can be selected for training.
In this embodiment, the deep learning neural network model is trained by using a multi-factor training sample, the features of each factor and the association relationship between the features of each factor are learned, and a multi-factor joint decision deep learning neural network model is constructed. The multi-factor here may include any of a video data factor, an image data factor, an audio data factor, a text data factor, and a network data factor.
Step S204: clue data acquired by a line in real time is received, and the clue data is unstructured data comprising a plurality of data types.
The server receives unstructured data acquired by each data acquisition terminal in real time by arranging a plurality of interfaces, wherein the unstructured data received by the server comprise a plurality of data types, specifically, real-time video data, real-time image data, real-time audio data, real-time text data, real-time network data and the like.
Step S206: and performing joint analysis on the received clue data through the trained deep learning neural network model, and extracting characteristic clue information favorable for judgment.
Step S208: and judging the cable data according to the characteristic cable information to generate a judgment result.
Specifically, the server receives unstructured data with various data types in real time, such as video data and network data in real time, the trained deep learning neural network model performs feature extraction and analysis on the video data and the network data received in real time, finds out feature clue information beneficial to decision making, and screens and filters massive video data and/or network data of a real-time structure based on the found feature clue information so as to make a decision more quickly and timely.
Step S210: and feeding back a judgment result.
The feedback of the decision structure can be in the form of triggering alarm or directly pushing the feedback structure to a designated display terminal for display.
For example, the purpose of analyzing unstructured data is to: the method for searching the target object from the video data specifically comprises the following steps:
the method comprises the following steps: continuous video data sent by the camera terminal is received in real time, and data of other data types are received, wherein the other data types can be audio data, text data and the like.
Step two: the method comprises the steps of using a trained deep learning neural network model to carry out feature extraction and analysis on clue data received by video data, text data, audio data and the like, and extracting feature clue information beneficial to retrieving a target from the video data, wherein the feature clue information can be position information of the retrieved target and sound information of the retrieved target, and the deep learning neural network model utilizes the extracted feature clue information beneficial to judging to screen the video data, such as searching the video data near the position information of the target and excluding the video data which do not meet position conditions, so that the data quantity needing judgment and analysis is greatly reduced, the data judgment is quicker, and online real-time judgment can be further realized. For another example, when the audio data is searched for the sound information of the search target, the association relationship between the video data and the audio data in the deep learning neural network model is called to locate the video data segment corresponding to the target sound information, and then the target object of the video is quickly searched.
It should be noted that the deep learning decision method in this embodiment is not limited to the above-mentioned decision task of retrieving the target object from the video data, but may also perform real-time fast decision of classification, identification, tracking, and the like of other application data with large data volume and fast change.
In this embodiment, a multi-factor training sample is used to train a deep learning neural network model to obtain a neural network model capable of performing multi-factor joint decision, the trained neural network model is used to perform joint analysis on real-time unstructured data with multiple data types, and characteristic clue information beneficial to data decision is extracted, so as to screen analysis data, and then more efficiently and timely perform data analysis decision to obtain a decision result. Namely, the correlation among various types of real-time data is analyzed, and then a faster and more intelligent judgment scheme is confirmed, so that the online real-time judgment of the application data with large data volume and fast change is realized.
In one embodiment, step S206: the deep learning neural network model after training is used for carrying out joint analysis on the received clue data, and the extraction of the characteristic clue information favorable for judgment comprises the following steps:
and performing feature extraction on the received clue data through the trained deep learning neural network model to obtain a high-dimensional feature vector.
And converting the high-dimensional feature vector into a hash code with each dimension incidence relation by adopting a hash retrieval algorithm, and extracting the characteristic clue information which is favorable for judgment according to the hash code.
Specifically, the high-dimensional feature vector refers to a feature vector which is generated by performing data analysis on multi-dimensional data (each data type is used as a data dimension) received in real time by using the trained deep learning neural network model and can represent the received unstructured data.
In the embodiment, on the basis of ensuring the proximity relation in the original space of each dimension, the generated high-dimensional feature vector is mapped into the hash code of the binary string (binary code) by adopting the hash retrieval algorithm, so that the storage and communication overhead of data can be obviously reduced, the calculation efficiency and speed are effectively improved, and the real-time analysis and judgment of a large amount of unstructured data are realized.
In one embodiment, as shown in FIG. 3, at step S202: the method comprises the following steps of obtaining a trained deep learning neural network model, wherein before the trained deep learning neural network model is a multi-factor joint decision neural network model trained by a multi-factor training sample, the method also comprises a training step of the deep learning deep neural network model, and specifically comprises the following steps:
step S302: and receiving training samples uploaded by the terminal, wherein the training samples are multi-factor training samples.
Specifically, a multi-factor sample is sample data that includes multiple data types. In one embodiment, the training sample data is multi-factor training sample data including multiple types of data such as video sample data, image sample data, audio sample data, text sample data, and network sample data.
Step S304: and under an off-line state, constructing a deep learning neural network sub-model for training sample data corresponding to each factor in the multi-factor training sample by using a deep learning algorithm.
Specifically, a training sample corresponding to each factor is preprocessed, and features of the training sample corresponding to each factor are respectively extracted by using a deep learning algorithm, for example, moving target features of a video training sample are extracted or target object features of the video training sample are extracted; extracting audio features and the like in the audio data, and respectively establishing a deep learning neural network submodel of training sample data corresponding to each factor, such as a video neural network submodel, an audio neural network submodel, an image neural network submodel, a text neural network submodel and a neural network submodel of network data.
Step S306: and acquiring the association relation between the constructed deep learning neural network submodels.
Step S308: and fusing the deep learning neural network submodels according to the incidence relation to generate a deep learning neural network model capable of performing multi-factor joint decision.
Specifically, the incidence relation among a plurality of built deep learning neural network submodels is mined, the submodels are fused, and then the deep learning neural network model capable of performing multi-factor joint decision is generated.
Specifically, if the fused deep learning neural network model capable of performing multi-factor joint decision is used for retrieving a target object in a video, the essence of mining the association relationship between multiple deep learning neural network submodels is as follows: and establishing an incidence relation between the non-video neural network submodel and the video neural network submodel. The non-video neural network submodel is a submodel constructed based on non-video data and extracted feature information, for example, the image neural network submodel is used for extracting facial features of a target object, and the text neural network submodel is used for extracting position features of the target object. And establishing an incidence relation between the video neural network submodel and the non-video neural network submodel so that the constructed combined decision-making deep learning neural network model can utilize the characteristic information extracted by the non-video neural network submodel to quickly search a target object in the video and improve the analysis and decision efficiency.
In one embodiment, when the clue data collected by the terminal in real time and received within the set time comprises one data type, the clue data is analyzed and judged by using the deep learning neural network submodel of the factor corresponding to the data type.
Specifically, when the real-time clue data received online by the server is only one data type within a period of time, and if the real-time clue data is only video data, the analysis and judgment of the video data are directly performed by using the video deep learning neural network submodel.
In the embodiment, the switching between the joint judgment and the single data form judgment can be carried out according to whether the received data is of multiple data types, and the joint judgment is carried out when the received data is of multiple data types, so that the judgment efficiency is improved; when the data type is a single data type, the single data type is directly analyzed, and data judgment is not stopped because the data type is only the single data type.
In one embodiment, as shown in fig. 4, there is provided an unstructured data decision device based on deep learning, the device comprising:
a neural network model obtaining module 402, configured to obtain a trained deep learning neural network model, where the trained deep learning neural network model is a multi-factor joint decision neural network model trained by a multi-factor training sample.
The real-time data receiving module 404 is configured to receive cue data acquired in real time online, where the cue data is unstructured data including multiple data types.
And the characteristic information extraction module 406 is configured to perform joint analysis on the received clue data through the trained deep learning neural network model, and extract characteristic clue information favorable for judgment.
And the joint decision module 408 is configured to decide the cue data according to the characteristic cue information, and generate a decision result.
And a decision result feedback module 410 for feeding back a decision result.
In one embodiment, the feature information extraction module 406 is further configured to perform feature extraction on the received clue data through the trained deep learning neural network model to obtain a high-dimensional feature vector; and converting the high-dimensional feature vector into a hash code with each dimension incidence relation by adopting a hash retrieval algorithm, and extracting the characteristic clue information which is favorable for judgment according to the hash code.
In one embodiment, as shown in fig. 5, the deep learning decision device further includes:
a training sample data receiving module 502, configured to receive a training sample uploaded by a terminal, where the training sample is a multi-factor training sample.
And the submodel training module 504 is configured to construct a deep learning neural network submodel for the training sample corresponding to each factor in the multi-factor training sample by using a deep learning algorithm in an offline state.
And the incidence relation analysis module 506 is used for acquiring the incidence relation between the built deep learning neural network submodels.
And a joint decision model building module 508, configured to fuse the deep learning neural network submodels according to the association relationship, and generate a deep learning neural network model capable of performing multi-factor joint decision.
In one embodiment, the multi-factor training samples include video data, image data, audio data, text data, and network data.
In one embodiment, the deep learning decision device further includes: and the submodel judging module is used for analyzing and judging the clue data by using the deep learning neural network submodel of the factor corresponding to the data type to obtain a judgment result when the clue data received in the set time contains one data type.
It will be understood by those skilled in the art that all or part of the processes in the methods of the embodiments described above may be implemented by hardware related to instructions of a computer program, and the program may be stored in a computer readable storage medium, for example, in the storage medium of a computer system, and executed by at least one processor in the computer system, so as to implement the processes of the embodiments including the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (11)

1.一种基于深度学习的非结构化数据判决方法,所述方法包括:1. A deep learning-based unstructured data decision method, the method comprising: 获取训练后的深度学习神经网络模型,其中,所述训练后的深度学习神经网络模型为经多因子训练样本训练的多因子联合判决神经网络模型;所述多因子联合判决神经网络模型融合了具有关联关系的、不同因子所对应的深度学习神经网络子模型;Obtain a trained deep learning neural network model, wherein the trained deep learning neural network model is a multi-factor joint judgment neural network model trained by multi-factor training samples; Deep learning neural network sub-models corresponding to different factors in relation to each other; 接收在线实时采集的线索数据,所述线索数据包括视频数据、文本数据、音频数据;Receive clue data collected online in real time, where the clue data includes video data, text data, and audio data; 通过所述训练后的深度学习神经网络模型对接收的所述线索数据进行联合分析,提取有利于从所述视频数据中检索目标的特征线索信息;所述特征线索信息包括检索目标的位置信息或检索目标的声音信息;Jointly analyze the received clue data through the trained deep learning neural network model, and extract characteristic clue information that is beneficial to retrieving the target from the video data; the characteristic clue information includes the position information of the retrieval target or Retrieve the sound information of the target; 根据所述特征线索信息定位所述视频数据中的目标视频数据段,对所述目标视频数据段进行判决,生成判决结果;所述判决结果为所述视频数据中的目标对象;Locate the target video data segment in the video data according to the feature clue information, judge the target video data segment, and generate a decision result; the decision result is the target object in the video data; 反馈所述判决结果。Feedback the judgment result. 2.根据权利要求1所述的方法,其特征在于,所述通过所述训练后的深度学习神经网络模型对接收的所述线索数据进行联合分析,提取有利于从所述视频数据中检索目标的特征线索信息的步骤包括:2 . The method according to claim 1 , wherein the received clue data is analyzed jointly by the deep learning neural network model after training, and extraction is conducive to retrieving the target from the video data. 3 . The steps of characteristic clue information include: 通过所述训练后的深度学习神经网络模型对接收的所述线索数据进行特征提取,得到高维特征向量;Perform feature extraction on the received clue data through the trained deep learning neural network model to obtain a high-dimensional feature vector; 采用散列检索算法将所述高维特征向量转化成带有各维度关联关系的散列码,并根据所述散列码进行有利于从所述视频数据中检索目标的特征线索信息的提取。A hash retrieval algorithm is used to convert the high-dimensional feature vector into a hash code with correlations of various dimensions, and extract feature clue information that is beneficial for retrieving the target from the video data according to the hash code. 3.根据权利要求1所述的方法,其特征在于,在所述获取训练后的深度学习神经网络模型,其中,所述训练后的深度学习神经网络模型为经多因子训练样本训练的多因子联合判决神经网络模型的步骤之前,还包括:3. The method according to claim 1, wherein the deep learning neural network model after the training is obtained, wherein the deep learning neural network model after the training is a multi-factor trained by a multi-factor training sample Before the steps of jointly deciding the neural network model, it also includes: 接收终端上传的训练样本,其中,所述训练样本数据为多因子训练样本;receiving a training sample uploaded by a terminal, wherein the training sample data is a multi-factor training sample; 在离线状态下,利用深度学习算法对所述多因子训练样本中每个因子对应的训练样本构建深度学习神经网络子模型;In an offline state, using a deep learning algorithm to construct a deep learning neural network sub-model for the training samples corresponding to each factor in the multi-factor training samples; 获取构建的所述深度学习神经网络子模型之间的关联关系;Obtain the association relationship between the constructed deep learning neural network sub-models; 根据所述关联关系融合所述深度学习神经网络子模型,生成可进行多因子联合决策的深度学习神经网络模型。According to the association relationship, the deep learning neural network sub-model is fused to generate a deep learning neural network model capable of multi-factor joint decision-making. 4.根据权利要求3所述的方法,其特征在于,所述多因子训练样本包括视频数据、图像数据、音频数据、文本数据和网络数据。4. The method according to claim 3, wherein the multi-factor training samples comprise video data, image data, audio data, text data and network data. 5.根据权利要求2所述的方法,其特征在于,当在设定时间内接收的所述线索数据包含一种数据类型时,则使用所述数据类型对应的所述因子的深度学习神经网络子模型对所述线索数据进行分析判决,得到判决结果。5 . The method according to claim 2 , wherein when the clue data received within a set time includes a data type, a deep learning neural network of the factor corresponding to the data type is used. 6 . The sub-model analyzes and judges the clue data to obtain a judgment result. 6.一种基于深度学习的非结构化数据判决装置,其特征在于,所述装置包括:6. An unstructured data judgment device based on deep learning, wherein the device comprises: 神经网络模型获取模块,用于获取训练后的深度学习神经网络模型,其中,所述训练后的深度学习神经网络模型为经多因子训练样本训练的多因子联合判决神经网络模型;所述多因子联合判决神经网络模型融合了具有关联关系的、不同因子所对应的深度学习神经网络子模型;A neural network model acquisition module, used for acquiring a trained deep learning neural network model, wherein the trained deep learning neural network model is a multi-factor joint judgment neural network model trained by multi-factor training samples; the multi-factor The joint decision neural network model integrates the deep learning neural network sub-models with correlation and corresponding to different factors; 实时数据接收模块,用于接收在线实时采集的线索数据,所述线索数据包括视频数据、文本数据、音频数据;A real-time data receiving module, configured to receive clue data collected online in real time, where the clue data includes video data, text data, and audio data; 特征信息提取模块,用于通过所述训练后的深度学习神经网络模型对接收的所述线索数据进行联合分析,提取有利于从所述视频数据中检索目标的特征线索信息;所述特征线索信息包括检索目标的位置信息或检索目标的声音信息;A feature information extraction module, configured to perform a joint analysis on the received clue data through the trained deep learning neural network model, and extract the feature clue information that is conducive to retrieving the target from the video data; the feature clue information Including the location information of the retrieval target or the sound information of the retrieval target; 联合判决模块,用于根据所述特征线索信息定位所述视频数据中的目标视频数据段,对所述目标视频数据段进行判决,生成判决结果;所述判决结果为所述视频数据中的目标对象;a joint decision module, configured to locate the target video data segment in the video data according to the feature clue information, judge the target video data segment, and generate a decision result; the decision result is the target video data in the video data. object; 判决结果反馈模块,用于反馈所述判决结果。The decision result feedback module is used for feeding back the decision result. 7.根据权利要求6所述的装置,其特征在于,所述特征信息提取模块,还用于通过所述训练后的深度学习神经网络模型对接收的所述线索数据进行特征提取,得到高维特征向量;采用散列检索算法将所述高维特征向量转化成带有各维度关联关系的散列码,并根据所述散列码进行有利于从所述视频数据中检索目标的特征线索信息的提取。7 . The device according to claim 6 , wherein the feature information extraction module is further configured to perform feature extraction on the received clue data through the trained deep learning neural network model to obtain high-dimensional feature vector; using a hash retrieval algorithm to convert the high-dimensional feature vector into a hash code with the relationship of each dimension, and according to the hash code to help retrieve the target feature clue information from the video data extraction. 8.根据权利要求6所述的装置,其特征在于,所述装置还包括:8. The apparatus of claim 6, wherein the apparatus further comprises: 训练样本数据接收模块,用于接收终端上传的训练样本,其中,所述训练样本为多因子训练样本;a training sample data receiving module, configured to receive a training sample uploaded by a terminal, wherein the training sample is a multi-factor training sample; 子模型训练模块,用于在离线状态下,利用深度学习算法对所述多因子训练样本中的每个因子对应的训练样本构建深度学习神经网络子模型;a sub-model training module for constructing a deep learning neural network sub-model for the training samples corresponding to each factor in the multi-factor training samples using a deep learning algorithm in an offline state; 关联关系分析模块,用于获取构建的所述深度学习神经网络子模型之间的关联关系;an association relationship analysis module, used to obtain the association relationship between the constructed deep learning neural network sub-models; 联合决策模型构建模块,用于根据所述关联关系融合所述深度学习神经网络子模型,生成可进行多因子联合决策的深度学习神经网络模型。The joint decision-making model building module is used to fuse the deep learning neural network sub-model according to the association relationship to generate a deep learning neural network model capable of multi-factor joint decision-making. 9.根据权利要求8所述的装置,其特征在于,所述多因子训练样本包括视频数据、图像数据、音频数据、文本数据和网络数据。9. The apparatus according to claim 8, wherein the multi-factor training samples comprise video data, image data, audio data, text data and network data. 10.根据权利要求7所述的装置,其特征在于,所述装置还包括:子模型判决模块,用于当在设定时间内接收的所述线索数据包含一种数据类型时,则使用所述数据类型对应的所述因子的深度学习神经网络子模型对所述线索数据进行分析判决,得到判决结果。10. The apparatus according to claim 7, characterized in that, the apparatus further comprises: a sub-model decision module, configured to use the The deep learning neural network sub-model of the factor corresponding to the data type analyzes and judges the clue data to obtain a judgment result. 11.一种计算机可读存储介质,存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现权利要求1至5中任一项所述的方法的步骤。11. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.
CN201610950494.9A 2016-10-26 2016-10-26 Unstructured data judgment method and device based on deep learning Expired - Fee Related CN107992937B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610950494.9A CN107992937B (en) 2016-10-26 2016-10-26 Unstructured data judgment method and device based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610950494.9A CN107992937B (en) 2016-10-26 2016-10-26 Unstructured data judgment method and device based on deep learning

Publications (2)

Publication Number Publication Date
CN107992937A CN107992937A (en) 2018-05-04
CN107992937B true CN107992937B (en) 2021-12-03

Family

ID=62029459

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610950494.9A Expired - Fee Related CN107992937B (en) 2016-10-26 2016-10-26 Unstructured data judgment method and device based on deep learning

Country Status (1)

Country Link
CN (1) CN107992937B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108733837B (en) * 2018-05-28 2021-04-27 上海依智医疗技术有限公司 Natural language structuring method and device for medical history text
CN109255499B (en) * 2018-10-25 2021-12-07 创新先进技术有限公司 Complaint and complaint case processing method, device and equipment
JP6992733B2 (en) * 2018-11-22 2022-01-13 セイコーエプソン株式会社 Information processing equipment and information processing method
CN110412470B (en) * 2019-04-22 2021-09-21 上海博强微电子有限公司 SOC estimation method for power battery of electric vehicle
CN110046715A (en) * 2019-04-24 2019-07-23 戴文跃 A kind of clue integrated system basis is constituted
CN111400040B (en) * 2020-03-12 2023-06-20 重庆大学 Industrial Internet system based on deep learning and edge calculation and working method

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105022835A (en) * 2015-08-14 2015-11-04 武汉大学 Public safety recognition method and system for crowd sensing big data

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7076594B2 (en) * 2000-12-22 2006-07-11 Cisco Technology, Inc. Apparatus and method for preventing one way connectivity loops in a computer network
US6980976B2 (en) * 2001-08-13 2005-12-27 Oracle International Corp. Combined database index of unstructured and structured columns
US7421443B2 (en) * 2003-01-29 2008-09-02 Microsoft Corporation Filestream data storage attribute
US8682993B1 (en) * 2013-03-01 2014-03-25 Inofile Llc Data capturing and exchange method and system
US9106748B2 (en) * 2013-05-28 2015-08-11 Mattersight Corporation Optimized predictive routing and methods
US9621680B2 (en) * 2013-10-21 2017-04-11 Globalfoundries Inc. Consistent data masking
CN103942240B (en) * 2013-10-29 2018-03-27 北京科东电力控制系统有限责任公司 A kind of building method of intelligent substation integrated data Information application platform
CN103854483A (en) * 2014-03-21 2014-06-11 广东新快易通智能信息发展有限公司 Vehicle dynamic information system based on GPS (global position system) satellite positioning and ECU (electronic control unit)
CN105068661B (en) * 2015-09-07 2018-09-07 百度在线网络技术(北京)有限公司 Man-machine interaction method based on artificial intelligence and system
CN105184513A (en) * 2015-10-16 2015-12-23 北京恒华伟业科技股份有限公司 Evaluation result output method and system

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105022835A (en) * 2015-08-14 2015-11-04 武汉大学 Public safety recognition method and system for crowd sensing big data

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Deep Learning of Binary Hash Codes for Fast Image Retrieval;Kevin Lin 等;《2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops》;20151026;参见第27页 *
Heterogeneous Features Integration via Semi-supervised Multi-model Deep Networks;Lei Zhao 等;《International Conference on Neural Information Processing》;20151118;参见第11-18页 *
Lei Zhao 等.Heterogeneous Features Integration via Semi-supervised Multi-model Deep Networks.《International Conference on Neural Information Processing》.2015, *

Also Published As

Publication number Publication date
CN107992937A (en) 2018-05-04

Similar Documents

Publication Publication Date Title
US11605226B2 (en) Video data processing method and apparatus, and readable storage medium
Sarvakar et al. Facial emotion recognition using convolutional neural networks
CN107992937B (en) Unstructured data judgment method and device based on deep learning
CN112131978B (en) Video classification method and device, electronic equipment and storage medium
CN105808610B (en) Internet picture filtering method and device
CN112132197B (en) Model training, image processing method, device, computer equipment and storage medium
CN109993102B (en) Similar face retrieval method, device and storage medium
CN113705293B (en) Image scene recognition method, device, equipment and readable storage medium
CN110598019B (en) Repeated image identification method and device
CN107423398A (en) Exchange method, device, storage medium and computer equipment
JP2022088304A (en) Methods, devices, electronics, media and computer programs for processing video
CN115187910B (en) Video classification model training method and device, electronic equipment and storage medium
CN117336525A (en) Video processing method, device, computer equipment and storage medium
CN111432206B (en) Video clarity processing method, device and electronic equipment based on artificial intelligence
CN113591530A (en) Video detection method and device, electronic equipment and storage medium
WO2023173552A1 (en) Establishment method for target detection model, application method for target detection model, and device, apparatus and medium
CN111310041A (en) Image-text publishing method, model training method and device and storage medium
CN111324773A (en) Background music construction method, device, electronic device and storage medium
CN118467778B (en) Video information summary generation method, device, electronic device and storage medium
CN114443904B (en) Video query method, device, computer equipment and computer readable storage medium
CN114449342B (en) Video recommendation method, device, computer readable storage medium and computer equipment
CN116932788B (en) Cover image extraction method, device, equipment and computer storage medium
CN114445757A (en) Nomination obtaining method, network training method, device, storage medium and equipment
CN109934302B (en) New category identification method and robot system based on fuzzy theory and deep learning
CN117033308B (en) Multi-mode retrieval method and device based on specific range

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20211203

CF01 Termination of patent right due to non-payment of annual fee