CN108229810A - Industry analysis system and method based on network information resource - Google Patents

Abstract

The invention relates to the field of information analysis, and proposes an industry analysis system based on network information resources, aiming to solve the problem that industry information analysis needs to consume a lot of manpower and material resources, and cannot achieve real-time performance. The system includes: a data collection module, a data preprocessing module, a data analysis module and a front-end interaction module, wherein the data collection module is configured to collect industry-related network information; the data preprocessing module is configured to structure the above network information The data processing is integrated with the platform data to build an industrial structure tree; the data analysis module is configured to analyze the above platform data through natural language processing technology and data mining algorithm, and extract the data related to the above keywords as interactive data; the front-end interaction module, It is configured to interact with the user terminal through the above interaction data. The invention realizes digging valuable data from massive network information, and presents industry analysis results for users in real time.

Description

Industry analysis system and method based on network information resources

technical field

The invention relates to the application field of computer network information, in particular to the application field of data mining of network information resources, and in particular to an industry analysis system and method based on network information resources.

Background technique

With the rapid development of information technology, information data in various fields has shown explosive growth, which has also brought huge challenges and pressures to workers in these industries. How to dig out valuable industry information from these massive data, It is of great reference significance to track changes in industry information in real time, understand the division of labor upstream and downstream of the industry and the development trends of competitors, and assist industry management and decision-makers to make quick and effective coping strategies for market changes.

Industry analysis is the result of systematic industry information integration and analysis, which has important reference significance for enterprises to discover industry business opportunities, grasp the pulse of the market, and evaluate investment risks. Usually internal or professional market research companies collect relevant data and make industry analysis reports based on relevant experience. Since the industry analysis report needs to be compiled after research, it needs to consume a lot of manpower and material resources, and it cannot achieve real-time performance. This is in huge contrast with the rapidly changing information age.

Contents of the invention

In order to solve the above-mentioned problems in the prior art, that is, in order to solve the problem that the industry analysis report needs to be compiled after investigation, it needs to consume a lot of manpower and material resources, and cannot achieve real-time performance, the present invention adopts the following technical solutions to solve the above-mentioned problems:

In the first aspect, the application provides an industry analysis system based on network information resources, the system includes: a data collection module, a data preprocessing module, a data analysis module and a front-end interaction module, wherein the above-mentioned data collection module is configured to collect and communicate with users The network information related to the industry concerned; the above data preprocessing module is configured to perform structural processing on the above network information, integrate with the preset platform data, and construct the domain knowledge tree of the industrial structure and the node of the domain knowledge tree of the industrial structure The above-mentioned data analysis module is configured to analyze the above-mentioned platform data and domain knowledge tree through natural language processing methods and data mining algorithms, and extract data related to the above-mentioned industries as interactive data; the above-mentioned front-end interaction module is configured to pass the above-mentioned The interaction data interacts with the user terminal.

In some examples, the data collection module includes a vertical web crawler and an academic web crawler, and the vertical web crawler is configured to grab webpage information from industry vertical websites by analyzing uniform resource locators according to the preset first initial seed node; The above-mentioned academic web crawler is configured to crawl academic articles from academic websites according to a preset second initial seed node.

In some examples, the above-mentioned data preprocessing module includes a data structuring submodule, a platform data submodule, a domain term extraction submodule and a domain knowledge tree submodule, and the above-mentioned data structuring submodule is configured to collect the above-mentioned vertical web crawler Structural analysis of vertical webpage information; the above-mentioned platform data sub-module is configured to store platform users and collected network information data, and provide data for the above-mentioned analysis module; the above-mentioned field term extraction sub-module is configured to crawl from the above-mentioned academic web crawler Extract domain-related terms from academic articles; the above-mentioned domain knowledge tree sub-module is configured to combine the domain expert knowledge to structurally organize the extracted domain terms, construct a domain knowledge tree of industrial structure, and analyze the above-mentioned domain knowledge tree The industrial association relationship between the nodes.

In some examples, the domain term extraction sub-module is further configured to analyze academic articles obtained by academic web crawlers, use text analysis methods to analyze word frequency in article titles, keywords and abstracts, and extract domain terminology.

In some examples, the above-mentioned data analysis module includes an entity recognition sub-module and a data mining sub-module, and the above-mentioned entity recognition sub-module is configured to construct entity recognition features through text segmentation, part-of-speech tagging and syntactic analysis, and integrate conditional random fields and rule-based method to identify the regional entity, organization name entity, and domain term entity contained in the platform data; the above data mining sub-module is configured to use a supervised machine learning algorithm to associate the identified entity with the domain knowledge tree, and statistically Analyze the relationship between news data, company data and domain knowledge tree, so as to analyze the distribution and change trend of network information data in regions and nodes of the industry chain; infer the industry nodes that users care about based on the user's operation data on the platform , using content-based recommendation algorithms to recommend personalized news, companies, and products for users.

In some examples, the above-mentioned front-end interaction module includes a visualization sub-module and a map sub-module, and the above-mentioned visualization sub-module is configured to synthesize the results of the above-mentioned data analysis module through domain knowledge trees, maps, line graphs, histograms, and lists The data interacts with the user; the above map sub-module is configured to present the regional map of the selected region to the user.

In the second aspect, the present application provides an industry analysis method based on network information resources, the method includes: collecting network information related to the industry concerned by the user; performing structured processing on the network information, and combining the preset platform data Carry out fusion to build an industrial structure tree; analyze the platform data through natural language processing technology and data mining algorithms, and extract data related to the industry as interactive data; interact with user terminals through the interactive data.

In some examples, the above-mentioned industry-related network information includes webpage information and academic articles, and the above-mentioned collection of network information related to the industry concerned by the user includes: according to the preset first initial seed node, using a vertical web crawler to analyze the above-mentioned The uniform resource locator contained in the first initial seed node grabs web page information from industry vertical websites; according to the preset second initial seed node, academic web crawlers are used to grab academic articles from academic websites.

In some examples, the aforementioned network information is structurally processed, fused with preset platform data, and a domain knowledge tree of industrial structure is constructed, including structural analysis of vertical web page information collected by vertical web crawlers; from the above academic Extract domain-related terms from academic articles crawled by web crawlers; combine the domain expert knowledge to structurally organize the extracted domain terms and key technologies, build an industrial structure tree, and analyze the industrial relationship between the nodes of the structure tree.

In some examples, the aforementioned domain-related terminology is extracted from the academic articles crawled by the above-mentioned academic web crawler, including: for analyzing the academic articles obtained by the academic web crawler, using text analysis algorithms to analyze the word frequency in article titles, keywords, and abstracts , to extract domain terminology.

In some examples, the above-mentioned platform data is analyzed through natural language processing methods and data mining algorithms, and data related to the above-mentioned industries is extracted as interactive data, including: constructing entity recognition features through text segmentation, part-of-speech tagging and syntactic analysis, fusion conditions Random field and rule-based methods to identify regional entities, organization name entities, and domain term entities contained in platform data; use supervised machine learning algorithms to associate identified entities with domain knowledge trees, and statistically analyze news data , Company data and the relationship between the domain knowledge tree, so as to analyze the distribution and change trend of network information data in the region and each node of the industrial chain; The recommended algorithm recommends personalized news, companies and products for users.

In some examples, the above-mentioned interaction with the user terminal through the above-mentioned interaction data includes: interacting with the user through the combination of domain knowledge tree, map, line graph, histogram and list; presenting the selected area for the user map.

In the industry analysis system and method based on network information resources provided by this application, the data acquisition module collects information related to the user's industry, and the above information is structured through the data preprocessing module, and the domain knowledge tree of the industry is constructed. The analysis module analyzes and mines the preprocessed information to obtain the analysis results of industry information, and interacts with users through the front-end interaction module. It realizes the mining of valuable industry information from massive data, real-time tracking of changes in industry information, understanding of industry upstream and downstream division of labor and competitor information, and assists industry management and decision-makers to make quick and effective coping strategies for market changes.

Description of drawings

Fig. 1 is a schematic structural diagram of an embodiment of an industry analysis system based on network information resources according to the present application;

Fig. 2 is the basic frame diagram of vertical web crawler crawling web page information flow in the embodiment of the present application;

Fig. 3 is a schematic diagram of an exemplary application of the robot industry industry chain knowledge tree constructed by the domain knowledge tree sub-module in the embodiment of the present application;

Figure 4a is a schematic diagram of the upstream and downstream node relationships of the industrial nodes constructed in the industry industrial chain;

Figure 4b is a schematic diagram of the relationship between upstream and downstream nodes of the system integration industry nodes in the robot industry chain constructed in the industry industry chain;

Fig. 5 is a schematic diagram of an example result of using a text analysis algorithm to perform text segmentation, part-of-speech tagging and syntactic analysis in the embodiment of the present application;

Fig. 6 is a schematic diagram of an embodiment of an industry analysis method based on network information resources applied in this application.

Detailed ways

Preferred embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention, and are not intended to limit the protection scope of the present invention.

It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

The industry analysis system based on network information resources of this application may include a data collection module, a data preprocessing module, a data analysis module and a front-end interaction module, wherein the above-mentioned data collection module is configured to collect network information related to the industry concerned by the user; The above-mentioned data preprocessing module is configured to perform structured processing on the above-mentioned network information, and integrates with the preset platform data to construct a domain knowledge tree of the industrial structure; the above-mentioned data analysis module is configured to use natural language processing methods and data mining algorithms Analyzing the above-mentioned platform data and domain knowledge tree, extracting data related to the above-mentioned keywords as interaction data; the above-mentioned front-end interaction module is configured to interact with the user terminal through the above-mentioned interaction data.

In this embodiment, the above-mentioned data collection module collects industry-related network information according to keywords or key information provided by users. Here, information related to industries that users are concerned about can be collected, such as information about companies in the same industry as users. Information, information of upstream and downstream enterprises; it can also collect information related to the development of the industry that users are concerned about, such as the technological frontier and academic frontier of the industry's development.

The above-mentioned data pre-processing module pre-processes the above-mentioned network information. The above-mentioned pre-processing can be structured processing of the above-mentioned network information related to the industry concerned by the user, and can extract companies related to the industry, company products, and company information from the above-mentioned information. Classified information such as distribution area and company's purchase request; it can also establish classified frontier information for industry development, from which information such as industrial development trends and technological development monitoring can be obtained.

The above-mentioned data analysis module analyzes and mines the structured data and classified information established by the above-mentioned data preprocessing module, and combines the user's operation information on the platform to recommend to the user information such as products, companies, and news that the user may be interested in.

The above-mentioned front-end interaction module can interact with users through an interactive interface, which can display information such as industrial change trends, industrial geographical distribution, upstream and downstream analysis, competitors, potential buyers, etc. in the form of various charts; users can intuitively and at any time , Obtain industry information anywhere.

The system provided by the above embodiments of the present application analyzes and mines the industry-related information concerned by the user, and presents the industry information obtained by the analysis and mining for the user.

FIG. 1 shows an exemplary system structure diagram of a specific embodiment of an industry analysis system based on network information resources to which the present application can be applied.

Specifically, as shown in Figure 1, the data acquisition module, data preprocessing module, data analysis module and front-end interaction module of the industry analysis system based on network information resources respectively realize data acquisition, data preprocessing, data analysis and front-end interaction functions .

The above data collection module includes a vertical web crawler 101 and an academic web crawler 102 . The above-mentioned vertical web crawler 101 is configured to crawl webpage information from industry vertical websites by analyzing uniform resource locators (Uniform ResourceLocator, URL) according to the preset first initial seed node. Specifically, a representative website may be selected according to the industry as the first initial seed node of the vertical web crawler. The above-mentioned vertical web crawler 101 crawls webpage information related to the industry concerned by the user by analyzing the URL of the website. The above-mentioned web page information includes news about relevant companies in the industry, institutional information in the industry, products and purchase information, etc.

The above-mentioned academic web crawler 102 is configured to crawl academic articles from academic websites according to a preset second initial seed node. Here, academic conferences and academic journals can be used as the second initial node of the academic web crawler. The above-mentioned academic web crawler 102 crawls relevant academic articles from academic websites, academic journals or academic conferences according to the above-mentioned second initial seed node, and obtains the development frontier information of the industry.

The above-mentioned vertical web crawler 101 and academic web crawler 102 can run in a fixed cycle according to actual needs, and provide data support for real-time industry analysis. For example, due to the high update frequency of industry information, the above-mentioned vertical web crawler 101 can run once every hour, so that the obtained information can achieve real-time effects as much as possible; while the update frequency of information in the knowledge field or academic field can be relatively low , the above-mentioned academic web crawler 102 can run once a day or once a month.

As an example, FIG. 2 shows the process of crawling industry-related data by the above-mentioned vertical web crawler 101 .

Step 2.1: Select a representative website according to the industry concerned by the user as the seed URL of the module 201, wherein the initial seed of the web crawler is stored in the module 201;

Step 2.2: push the seed URL of module 201 into the URL queue to be grabbed of module 202;

Step 2.3: Module 203 reads and reads URLs from the queue of URLs to be grabbed, and uses URL filter 204 to filter the selected URLs. Specifically, the read URLs can be parsed, and only news and companies are retained. the relevant web page URL;

Step 2.4: the downloader of module 205 crawls the filtered URL webpage from the vertical website, and uses module 206 to save the webpage content;

Step 2.5: the webpage of module 206 is saved in the webpage database 209, meanwhile, the URL of the webpage successfully crawled is pushed into the captured URL queue 208 of module 207;

Step 2.6: Use module 206 to extract URLs from the webpage, filter the crawled URLs, and push uncrawled URLs into module 202;

Step 2.7: Judging whether there are uncrawled web page URLs in the queue of module 202, if so, jump to step 2.3, otherwise, crawling of industry-related data by the web crawler ends.

In this embodiment, the data preprocessing module includes a data structuring submodule 103 , a platform data submodule 105 , a domain term extraction submodule 104 and a domain knowledge tree submodule 106 . The above-mentioned data structuring sub-module 103 is configured to analyze and structure the webpage content crawled by the above-mentioned vertical web crawler 101, and merge it with the platform data in the platform data sub-module 105 to provide basic data for further analysis and processing of data . For the news web pages crawled by the above-mentioned vertical web crawler 101, the above-mentioned data structuring sub-module 103 utilizes web page parsing tools, such as BeautifulSoup, lxml, etc., to extract the news title, release time, web content, etc. of the above-mentioned news web pages. As an example, Table 1 shows a news data table obtained by analyzing the news web pages crawled by the vertical web crawler 101 using a web page analysis tool.

Table 1 news data table,

The corporate organization information obtained by the above vertical web crawler 101 includes the information of companies related to the enterprise, obtains the content of the company's web page, uses a web page analysis tool to analyze the content of the company's web page, and extracts the company's name, address, Products, buy and company introduction and other information. As an example, refer to Table 2, which shows the company data table obtained by analyzing the company web pages crawled by the vertical web crawler 101 using the web page analysis tool.

Table 2 Company Data

The domain term extraction sub-module 104 analyzes the academic articles obtained by the academic web crawler 102 by using a web page analysis tool, and then extracts domain terms of the academic articles. Since the title, keywords and abstract contained in an academic article are the refinement of the core content of the academic article, the analysis of the academic article can be the first analysis of the title, keywords and abstract of the academic article, and then according to the needs of the academic article content is analyzed. Various text analysis algorithms are embedded in the domain term extraction sub-module 104, and the above-mentioned academic articles are analyzed by using the text analysis algorithms embedded in the domain term extraction sub-module 104. Specifically, the above-mentioned analysis of academic articles using text analysis algorithms can extract the text of academic articles using the term frequency-inverse document frequency algorithm (term frequency-inverse document frequency, TF-IDF) and latent semantic analysis algorithm (Latent Dirichilet Allocation, LDA) keywords, using clustering method to analyze the word frequency in titles, keywords, and abstracts of academic articles, and extract words whose frequency of occurrence is greater than the set threshold, as domain terms of child nodes in the domain knowledge tree. The term set formed by the domain terms of each child node in the above domain knowledge tree can be used to analyze the relationship between the network information data and the domain knowledge tree. As an example, refer to Table 3, which shows the data structure of academic articles, and the text analysis algorithm analyzes the academic articles based on the content shown in the data structure of Table 3.

Table 3 Academic Article Data Table

The platform data sub-module 105 is configured to provide basic data and preprocessed data for the data analysis module to perform data analysis. The above-mentioned platform data sub-module 105 stores various types of information, including user operation behaviors in the platform, company products, purchase needs, company news, company information, and regional information. The above-mentioned user operation behavior is the user's operation behavior on the system platform, such as browsing news, clicking on products, publishing requirements, etc., which is used to track and record user behavior information, and provide data support for the algorithm to analyze user interests. The above-mentioned company products can be product information released by company users on the platform, such as product name, product introduction, product function, product parameters and other information. The above request for purchase may be the purchase information released by the user on the platform, such as the name of the product to be purchased, parameters, price, limited area, and the like. The above-mentioned company news may be news information released by company users on the platform, including news title, author, content, etc. The above-mentioned company information can be the registration information of company users in the system platform, such as company name, registered address, main business, etc.; the above-mentioned geographical information can be the geographical information of China constructed on the system platform, including the full name, abbreviation, Latitude and longitude coordinates and areas are used for analyzing network information and locating company location information.

The domain knowledge tree sub-module 106 is configured to combine the expert knowledge and the extracted domain terminology to construct the association relationship between the domain knowledge tree and the nodes of the domain knowledge tree of the industrial structure. The above-mentioned domain knowledge tree sub-module 106 can construct the domain knowledge tree of the industry according to the extracted data information of the industry in which the company user is located. The above construction of the domain knowledge tree of the industry industry first constructs the nodes of the industrial chain, which are the upstream of the industrial chain, the middle of the industrial chain, and the downstream of the industrial chain; then, according to the web page information and expert knowledge crawled by the web crawler, the upstream nodes of the industrial chain are respectively constructed , the sub-nodes of the midstream nodes of the industrial chain and the downstream nodes of the industrial chain; finally, continue to use each of the above-mentioned sub-nodes as intermediate nodes to construct the sub-nodes of each of the above-mentioned intermediate nodes, thereby constructing the domain knowledge tree of the industry chain where the company user is located. As an example, FIG. 3 shows the domain knowledge tree of the robot industry chain constructed by the domain knowledge tree sub-module 106 described above. In the robot industry chain, it is divided into upstream nodes of the industrial chain, midstream nodes of the industrial chain and downstream nodes of the industrial chain. The upstream nodes of the industrial chain are suppliers, including sub-nodes such as raw materials and parts; the downstream nodes of the industrial chain are after-sales services and applications, including sub-nodes of partners, sub-nodes of agents, sub-nodes of third-party services, and sub-nodes of solutions; The middle reaches of the industrial chain are the main business of the industry. As the backbone of the domain tree, it includes the robot body node and the robot integration node. The robot integration node includes multiple sub-nodes. For example, the sub-nodes of the robot integration node include intelligent robot nodes, intelligent robot nodes There are industrial robot sub-nodes under the industrial robot node, and there are moving robot sub-nodes under the industrial robot node.

FIG. 4 shows a schematic diagram of the upstream and downstream industry nodes of the robot industry chain constructed by the above domain knowledge tree sub-module 106 . Among them, Figure 4a shows the relationship between the upstream and downstream nodes of each node in the design, and Figure 4b shows a schematic diagram of the industrial chain nodes of a specific example in the robot industry chain, such as, in the robot industry chain, when the industry node is "system integration" , then the upstream industry nodes include sensors, controllers, etc., and the downstream industry nodes include third parties, agents, etc.

In this embodiment, the data analysis module includes an entity recognition sub-module 107 and a data mining sub-module 108. The entity recognition sub-module 107 is configured to construct entity recognition features through text segmentation, part-of-speech tagging and syntactic analysis, and integrate conditional random fields and a rule-based method to identify regional entities, organization name entities, and domain term entities contained in the platform data; the above-mentioned data mining sub-module 108 is configured to use a supervised machine learning algorithm to combine the identified entities with the domain knowledge tree Correlation, statistical analysis of the relationship between news data, company data and domain knowledge trees, so as to analyze the distribution and change trend of network information data in regions and nodes of the industrial chain; infer the user's concerns based on the user's data on the platform The industry node of , using content-based recommendation algorithms, recommends personalized news, companies and products for users.

The above-mentioned entity recognition sub-module 107 includes six units of text segmentation, part-of-speech tagging, syntactic analysis, region recognition, organization name recognition and field term recognition. The above-mentioned text segmentation, part-of-speech tagging and syntax analysis are used to construct entity recognition features. Taking "a robot is a machine device that automatically performs work" as an example, text segmentation, part-of-speech tagging and syntactic analysis are performed. The results are shown in Figure 5, and the entity recognition features extracted from it are shown in Table 4; then the fusion of conditional random fields ( Conditional Random Fields (CRF) and rule-based methods, for each piece of information, discover and identify the geographical entity, organization name entity and domain term entity contained in it.

Table 4 Entity Recognition Features

The above-mentioned data mining sub-module 108 utilizes supervised learning algorithms to construct associations between identification entities and industry nodes, analyze associations between news data, company data, and domain knowledge trees, and make statistics on industry trend changes, industrial geographical distribution, and Downstream analysis, by analyzing the data of users or companies on the platform, such as released products, purchasing information, etc., reasoning about the industry nodes they care about, and recommending products, companies and news that they are interested in.

In this embodiment, the above-mentioned foreground interaction module includes a visualization sub-module and a map sub-module, and the above-mentioned visualization sub-module is configured to synthesize the data analysis module through domain knowledge tree 109, line graph 111, histogram 112 and list 113 The analyzed result data is interacted with the user; the above map sub-module 110 is configured to present the map of the selected area to the user, which can be used as an area map of the area.

The visualization sub-module for interacting with the user presents various analysis result information for the user through domain knowledge tree 109 , line graph 111 , bar graph 112 and list 113 .

The above domain knowledge tree 109 presents the domain knowledge tree structure of the industry concerned by the user for the user, and allows the user to select the industry node to view.

The above-mentioned map sub-module 110 presents the regions of various provinces and cities in China for the user. When a certain province or city is selected, it will automatically jump to the provincial map of this province.

The above-mentioned line chart 111 presents to the user the trend of news popularity of a certain industry node in a certain region over time.

The above-mentioned histogram 112 presents the news popularity distribution of a certain industry node in a certain region for the user.

The above-mentioned list 113 presents upstream and downstream companies, competitors, potential buyers, and recommended news and other information to the user in the form of a list.

The system provided by the above embodiments of the present application extracts information related to the user's industry from massive data through the data acquisition module; the data preprocessing module performs data structural processing on the extracted information; and builds a domain knowledge tree. The data analysis module analyzes and mines the processed information, and combines expert knowledge to analyze industry development trends, and provides users with industry analysis reports; the front-end interaction module interacts with users to provide users with industry-related information. It enables users to grasp the real-time changes of each node in the industry in a timely manner, understand the upstream and downstream division of labor in the industry and the information of competitors, and assist the industry management or decision-making layer to make quick and effective coping strategies for market changes.

Referring to Fig. 6, the present application provides an industry analysis method based on network information resources, the method includes the following steps:

Step 601, collecting network information related to the industry concerned by the user.

In this embodiment, the electronic device (which may be a server or an application platform) applied to this application uses a web crawler to obtain industry-related network information from industry-related websites. Here, the website related to the industry concerned by the user may be the website of the company in the industry the user is in or engaged in, as well as the upstream and downstream industries, and may also be a technical and academic forum or website related to the industry. The aforementioned web crawler may be a vertical web crawler, and may also be an academic web crawler. The above-mentioned vertical web crawler collects news, institutions, products and purchase information from domain-related websites. The above-mentioned academic web crawler crawls relevant academic articles from the websites of academic conferences and academic journals related to the field. The above-mentioned network information may be the above-mentioned news, institutions, products and purchase information, or academic articles.

In some preferred embodiments, the above-mentioned network information related to the industry concerned by the user includes webpage information and academic articles, and the collection of network information related to the industry concerned by the user includes: according to the preset first initial seed node, using The vertical web crawler grabs web page information from industry vertical websites by analyzing the Uniform Resource Locator of the above-mentioned first initial seed node. According to the preset second initial seed node, an academic web crawler is used to crawl academic articles from academic websites. Here, the above-mentioned first initial seed node is to select a representative website according to the industry as the initial seed node of the web crawler. The above-mentioned second initial seed node may be based on academic conferences and academic journals as the initial seed node. The aforementioned web crawler crawls relevant web page information or academic articles by analyzing the URL.

In step 602, the above-mentioned network information is structured and fused with the preset platform data to build a domain knowledge tree of the industrial structure.

In this embodiment, the server or the application platform performs data preprocessing on the network information to build an industrial structure tree. Here, the above data preprocessing can be structured analysis of vertical webpage information collected by vertical web crawlers; it can also extract domain-related terms and key technologies from academic articles crawled by academic web crawlers, and combine domain expert knowledge, Structurally organize the extracted domain terms and key technologies, build an industrial structure tree, and analyze the industrial relationship between the nodes of the structure tree. Further, extracting industry-related or industry-related terms and key technical information from the academic articles crawled by academic web crawlers can be used to analyze the academic articles obtained by academic web crawlers, using text analysis algorithms to analyze article titles, keywords and The frequency of words in the abstract, extracting domain terminology. The aforementioned text analysis algorithm may be algorithms such as TF-IDF, LDA, and clustering.

Step 603, analyze platform data and domain knowledge tree through natural language processing method and data mining algorithm, and extract industry-related data as interaction data.

In this embodiment, natural language processing methods can be used to identify regional entities, domain term entities, and organization name entities from network information such as news, companies, products, and purchases; data mining algorithms can be used to identify domain term entities and The relationship between domain knowledge tree nodes, classification and analysis of knowledge nodes such as news, company, product, and purchase information, and statistics are made according to the region and release time of these information, based on the change of news popularity of knowledge nodes, a pair of knowledge nodes is formed. Tracking of industry trend changes.

Further, the above-mentioned platform data is analyzed through natural language processing methods and data mining algorithms, and industry-related data is extracted as interactive data, including the construction of entity recognition features through text segmentation, part-of-speech tagging and syntactic analysis, and the fusion of conditional random fields and rule-based method to identify regional entities, organization name entities, and domain term entities contained in platform data; use supervised machine learning algorithms to associate identified entities with domain knowledge trees, and statistically analyze news data, company data, and domain The relationship between the knowledge trees, so as to analyze the distribution and change trend of network information data in the region and each node of the industrial chain; according to the data of the user on the platform, infer the industrial nodes that the user is concerned about, and use the content-based recommendation algorithm to provide User recommendations for personalized news, companies and products.

Step 604, interact with the user terminal through the interaction data.

In this embodiment, the interactive application provided by the application platform performs information interaction with the user. Here, the interactive application may be a visual application, such as displaying analysis results in the form of a line graph, a histogram, or a list. specifically:

Use the line chart to present the industry trend changes of the selected domain knowledge tree nodes within the selected geographical range to the user.

Use the histogram to show the user the geographical distribution of the selected domain knowledge tree nodes within the selected geographical range.

Use the list to present the upstream and downstream enterprises of the selected domain knowledge tree node for the user within the selected geographical range; use the list to recommend the companies they are interested in to the user; use the list to recommend the products they are interested in to the user; Use lists to recommend news of interest to users.

The method provided by the above-mentioned embodiments of the present application can extract effective information from massive data, present real-time changes of each node in the industry for users, understand the division of labor and competitors in the upstream and downstream of the industry, assist industry management, decision-making, etc., and target the market Changes and make quick and effective coping strategies.

So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings, but those skilled in the art will easily understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims (12)

1. a kind of industry analysis system based on network information resource, which is characterized in that the system comprises：Data acquisition module Block, data preprocessing module, data analysis module and foreground interactive module, wherein,

The data acquisition module is configured to acquisition and the relevant network information of user's industry of interest；

The data preprocessing module, be configured to the network information carry out structuring processing, with preset platform data into Row fusion, builds the incidence relation between the domain knowledge tree of the industrial structure and the domain knowledge tree node of the industrial structure；

The data analysis module is configured to analyze the platform data by natural language processing method and data mining algorithm With the domain knowledge tree, extraction is with the relevant data of the industry as interaction data；

The foreground interactive module is configured to interact with user terminal by the interaction data.

2. the industry analysis system according to claim 1 based on network information resource, which is characterized in that the data are adopted Collect module and include perpendicular network reptile and academic web crawlers,

The perpendicular network reptile is configured to according to preset first initial seed node, by analyzing uniform resource locator Webpage information is captured from industrial vertical websites；

The science web crawlers is configured to capture academic article from Academic Web Sites according to preset second initial seed node.

3. the industry analysis system according to claim 2 based on network information resource, which is characterized in that the data are pre- Processing module includes data structured submodule, platform data submodule, field term extracting sub-module and domain knowledge tree Module,

The data structured submodule, the vertical webpage information for being configured to crawl the perpendicular network reptile carry out structure Change analysis；

The platform data submodule is configured to the network information data of storage platform user and collection, and is the analysis Module provides data；

The field term extracting sub-module is configured to extract field from the academic article that the academic web crawlers is crawled Relational language；

The domain knowledge tree module is configured to combine domain-specialist knowledge, and structure is carried out to the field term of extraction Change tissue, build the domain knowledge tree of the industrial structure, and analyze the Industrial Correlation between the domain knowledge tree node of the industrial structure Relationship.

4. the industry analysis system according to claim 3 based on network information resource, which is characterized in that the field art Language extracting sub-module is further configured to analyze the academic article that the academic web crawlers obtains, uses text analyzing method The word frequency in article title, keyword and abstract is analyzed, extracts field technical term.

5. the industry analysis system according to claim 1 based on network information resource, which is characterized in that the data point It analyses module and includes Entity recognition submodule and data mining submodule,

The Entity recognition submodule is configured to segment by text, part-of-speech tagging and syntactic analysis build Entity recognition spy Sign, fusion conditions random field and rule-based method identify that region entity included in the platform data, mechanism name are real Body and field term entity；

The data mining submodule is configured to using the machine learning algorithm for having supervision, the entity that will identify that and the neck Domain knowledge tree is associated, the incidence relation between statistical analysis news data, company data and the domain knowledge tree, so as to divide Network information data is analysed in region and the distribution situation and variation tendency of each node of industrial chain；According to the user in platform Operation data, the industry node of interest of user described in reasoning using content-based recommendation algorithm, are recommended personalized for user News, company and product.

6. the industry analysis system according to claim 1 based on network information resource, which is characterized in that hand on the foreground Mutual module includes visualization submodule and map submodule,

The visualization submodule is configured in a manner that domain knowledge tree, map, line chart, block diagram and list integrate The result data that the data analysis module is analyzed and user are interacted；

The map submodule is configured to the map in region selected by user's presentation.

A kind of 7. industry analysis method based on network information resource, which is characterized in that the method includes：

Acquisition and the relevant network information of user's industry of interest；

Structuring processing is carried out to the network information, is merged with preset platform data, builds the field of the industrial structure Knowledge tree；

The platform data and the domain knowledge tree are analyzed by natural language processing method and data mining algorithm, extraction and The relevant data of industry are as interaction data；

It is interacted by the interaction data with user terminal.

8. the industry analysis method according to claim 7 based on network information resource, which is characterized in that described and industry The relevant network information includes webpage information and academic article, the acquisition and the relevant network information of user's industry of interest, Including：

According to preset first initial seed node, using perpendicular network reptile by analyzing the first initial seed node institute Comprising uniform resource locator from industrial vertical websites capture webpage information；

It is to capture academic article from Academic Web Sites using academic web crawlers according to preset second initial seed node.

9. the industry analysis method according to claim 8 based on network information resource, which is characterized in that described to described The network information carries out structuring processing, is merged with preset platform data, builds the domain knowledge tree of the industrial structure, wraps It includes：

Structured analysis is carried out to the vertical webpage information of perpendicular network reptile acquisition；

Field relational language is extracted in the academic article crawled from the academic web crawlers；

With reference to domain-specialist knowledge, structured organization is carried out to the field term of extraction, builds the domain knowledge tree of the industrial structure, And analyze the Industrial Correlation relationship between the node of the domain knowledge tree.

10. the industry analysis method according to claim 9 based on network information resource, which is characterized in that described from institute Extraction field relational language in the academic article that academic web crawlers is crawled is stated, including：

For the academic article that the academic web crawlers of analysis obtains, using text analyzing Algorithm Analysis article title, keyword and pluck Word frequency in wanting extracts field technical term.

11. the industry analysis method according to claim 7 based on network information resource, which is characterized in that described to pass through Natural language processing method and data mining algorithm analyze the platform data, and extraction is with the relevant data of the industry as friendship Mutual data, including：

It is segmented by text, part-of-speech tagging and syntactic analysis build Entity recognition feature, fusion conditions random field and based on rule Method then identifies region entity included in the platform data, mechanism name entity and field term entity；

Using the machine learning algorithm for having supervision, the entity that will identify that is associated with the domain knowledge tree, and statistical analysis is new The incidence relation between data, company data and the domain knowledge tree is heard, so as to analyze network information data in region and production The distribution situation and variation tendency of each node of industry chain；

According to user in the data of platform, reasoning user industry node of interest, using content-based recommendation algorithm, for Recommend personalized news, company and product in family.

12. the industry analysis method according to claim 11 based on network information resource, which is characterized in that described to pass through The interaction data is interacted with user terminal, including：

The interaction data and user are carried out by way of domain knowledge tree, map, line chart, block diagram and list synthesis Interaction；

Map for region selected by user's presentation.