CN107729006B

CN107729006B - A PC-based intelligent collection tool and method for page information

Info

Publication number: CN107729006B
Application number: CN201711034890.8A
Authority: CN
Inventors: 张�林; 高树; 王立钧; 徐新皎; 郑跃
Original assignee: Nanjing Weishi Technology Co ltd
Current assignee: Nanjing Weishi Technology Co ltd
Priority date: 2017-10-30
Filing date: 2017-10-30
Publication date: 2021-06-04
Anticipated expiration: 2037-10-30
Also published as: CN107729006A

Abstract

The invention provides a PC-based intelligent collection tool and method for page information, including a page extractable data display module, a page data extraction module, a page data processing module and a page data display operation module; the page extractable data display module is in The extractable information is marked and displayed in the process of browsing the page; the page data extraction module extracts and classifies the page data; the page data processing module merges the extracted page data; the page data display operation The module displays the extracted data and its relationships on the canvas in the form of nodes and connections. The invention can help users to extract data from multiple pages, and intelligently merge the extracted data according to a certain relationship, which reduces the analysis time of the extracted data and reduces the workload of the user. At the same time, the extracted data will be displayed and edited by users, which is convenient for users to analyze.

Description

Page information intelligent acquisition tool and method based on pc end

Technical Field

The invention relates to the field of web internet, in particular to a page information intelligent acquisition tool and method based on a pc end.

Background

The existing page data extraction tool extracts the data of the page by utilizing the analysis algorithm of the node and the configuration parameters required by the corresponding webpage information extraction action according to the webpage elements selected by the user, thereby achieving the purpose of extracting the webpage information. Although the data is extracted, it is typically a single bar that is listed and then presented to the user for viewing.

1) Which data on the page can be extracted and which data can not be extracted are not clearly identified, and a certain confusion effect is achieved for the user.

2) The extracted information is scattered in a single piece and has no correlation between the information

3) The extracted information can not be modified, added, deleted, relationship established, grouping added, remark added and the like again when displayed, and browsing operation of a user is facilitated.

4) The operations of screenshot, data storage, export, accessory uploading and the like cannot be performed on the extracted data, and the operation of continuing browsing and backup for the next time by a user is facilitated

5) The extracted data can not be analyzed again, namely, the extracted data is used as an information source, and secondary analysis is performed, so that secondary positioning of a user on problems is facilitated, and the data can be analyzed deeply.

Disclosure of Invention

The purpose of the invention is as follows: in order to overcome the defects in the prior art, the invention provides the page information intelligent acquisition tool and method based on the pc end, the extraction is quick and simple, the problem of quick positioning of a user can be helped, and the working efficiency is greatly improved.

The technical scheme is as follows:

a page information intelligent acquisition tool based on a pc end comprises a page extractable data display module, a page data extraction module, a page data processing module and a page data display operation module;

the page extractable data display module displays the extractable information in a marking way in the process of browsing the page by the user according to the preset data type information on the page;

the page data extraction module extracts and classifies the page data according to the identification data of different data types to obtain classified page data;

the page data processing module judges the relationship between the data by comparing the identification data among the page data extracted by the page data extraction module and merges the extracted page data according to the preset data type information;

and the page data display operation module displays the extracted data and the relation thereof on the canvas in a node and connection mode.

The subsequent analysis module is used for performing subsequent analysis operation on the contents of the plurality of nodes, listing analyzable data, performing secondary analysis on the selected data through the analysis items selected freely by the user, and displaying the analysis result.

The preset data types comprise five types of data, namely task data, case data, card data, relationship and object data and activity track data.

And the mark display adopts highlight prompt for display.

The relationship between the data is specifically as follows: if the data are data of the same node relationship, merging the data into the same node, and if the two data meet the relationship structure, establishing relationship connection.

An intelligent acquisition method for page information comprises the following steps:

(1) presetting data type information on a page, wherein the data type comprises five types of data, namely task data, case data, card data, relation and object data and activity track data;

(2) in the process of browsing the page by the user, according to the preset data type information on the page, marking and displaying the extractable page data, and screening the page data to be extracted by the user;

(3) after the user finishes screening the page data to be extracted, extracting the page data and classifying the page data according to identification data of different data types in the page data to be extracted to obtain classified page data;

(4) judging the relation between the data by comparing the identification data between the page data extracted in the step (3), and merging the page data extracted in the step (3) according to the preset data type information in the step (1);

(5) and (4) displaying the merged result of the step (4) on a canvas in a node and line mode.

In the step (5), the nodes on the canvas can be freely dragged to perform layout, and the nodes and the content on the nodes, the relationship and the content on the relationship line are customized and edited.

The custom editing specifically comprises:

manually adding nodes: selecting node types and key levels, and adding the node types and the key levels to the relational graph; determining a node default display icon according to the type, selecting the color of a node frame according to the key level, and selecting an identity card photo as the node icon if the node is a personnel node;

manually add relationships: selecting a relation line and a relation type, and inputting related contents;

and modifying the relationship: editing the content of the relation, and dragging the relation line to edit the position;

deleting the relationship;

the new grouping function: creating a group for a plurality of or single nodes outside the group;

a grouping editing function: editing the designated groups, including modifying the group names and remarks;

node unbinding function: performing unbinding operation on a single node or a plurality of nodes, and directly unbinding the selected node from the corresponding group;

a packet cancellation function: the designated packet is cancelled.

Has the advantages that: the invention can help the user to extract data from a plurality of pages (the extracted data are all highlighted and can be seen at a glance, so that the user can conveniently identify the extracted data), and the extracted data are intelligently merged or connected according to a certain relation, thereby reducing the analysis time of the user on the extracted data and reducing the workload. Meanwhile, the extracted data is displayed and can be customized and edited by a user, and the user can conveniently analyze the data. The small tool module provided by matching can help a user to carry out overall processing (storage, accessory adding, remarking, additional storage, exporting, navigation, highlighting mode and the like) on the task to which the data belongs, and in addition, the extracted data can be subjected to diversified secondary analysis (follow-up operation), useful data can be extracted, and irrelevant data can be removed. The positioning device can help a user to quickly position, and greatly improves the working efficiency.

Drawings

FIG. 1 is a schematic structural diagram of the present invention.

FIG. 2 is a schematic view of a tool module of the present invention.

FIG. 3 is a schematic diagram of a display operation module according to the present invention.

Detailed Description

The present invention will be further described with reference to the accompanying drawings.

The invention relates to a page information intelligent acquisition based on pc end, which combines a background restful interface, a foreground operating node.js environment and a JavaScript function library D3.js for generating graphics for use, so that single foreground and background frames are connected in series to form a system with a front platform and a back platform separated, comprising:

the page extractable data presentation module: the method is used for marking and highlighting the extractable information in the process of browsing the page by the user, displaying the information to the user and facilitating the screening of the user.

The invention predefines a specific information format on the page, which contains the common data types required by the user, wherein the data types comprise five types of data, namely task data, case data, card data, relationship and object data and activity track data. Wherein the task data is task information data; case data is case related information; the card data is information such as an identity card, a telephone, a person name and the like; the activity track represents activity information, such as what time is in place; the relation and the object data represent the relation among the data, so that the relation is established by matching the data. These types of data are exposed when page fetches are performed. In the present invention, highlighting is used for display.

The page data extraction module: after the user screens the page extractable data, the pages are extracted and classified according to the page identifications of different data types, and classified page data are obtained. The module is also provided with a single extraction function and a one-key extraction function; thereby performing a single data extraction and a one-touch extraction of the user-selected data, respectively.

The page identification is as follows:

task data:

case data:

<span ibox_case_id＝””ibox_case_code＝””ibox_case_name＝””

ibox_case_remark＝””></span>'

card data:

relationship and object data:

<tr>

<td>

</td>

<td>xxxxx</td>

<td>xxxxxxxx</td>

</tr>

activity track data:

the page data processing module: merging the extracted data, and judging whether the data has a relation and what relation by comparing the identification data in the two data according to a specific data structure: if the data are data of the same node relationship, merging the data into the same node, and if the two data meet the relationship structure, establishing relationship connection. The page data become non-scattered and single after being processed by the page data merging module.

The page data display operation module: a visualization module developed on the basis of the D3.js graphic function displays the extracted data and the relation thereof on a canvas; the specific relationship between the nodes and the connecting lines is marked.

And carrying out free dragging and layout on the nodes on the canvas. The content on the node can be edited in a user-defined mode, the content on the relation line can be edited in a user-defined mode, a grouping function can be provided, and the nodes are grouped and remarked; the method specifically comprises the following steps:

manually adding nodes: and selecting the node type and the emphasis level, and adding the node type and the emphasis level to the relational graph. Determining a default display icon of the node according to the type, selecting the color of a node frame according to the key level, and selecting an identity card photo as the node icon if the node is a personnel node;

and modifying the relationship: the content of the relation can be edited by selecting and editing, the position of the relation can be changed by dragging the relation line, and a storage button is clicked to submit the modification operation;

deleting the relationship: the relation can be deleted by selecting deletion, and a save button is clicked to submit deletion operation;

the new grouping function: grouping a plurality of or single nodes outside the group by a right-click menu, and automatically giving a default group name (the number of the new group plus is the default name) to the group when the group is newly established;

a packet cancellation function: the designated packet is cancelled.

A subsequent analysis module: subsequent analysis operation on the contents of the plurality of nodes, selecting attribute functions, wherein the attributes comprise names, identity cards, telephones, vehicles, addresses and the like; and for subsequent analysis operation of the contents of the plurality of nodes, analyzable data is listed, analysis items can be selected freely, secondary analysis is carried out on the selected data, the secondary analysis can jump to a result page of a corresponding analysis module, the analysis result is displayed, and the two functions of extraction and module analysis are chained, so that the extracted data is processed in a diversified manner.

The invention also provides a tool module comprising:

manually creating a new task: the system is used for manually adding tasks by a user, inputting task names and storing the tasks;

data extraction creation task: after a user extracts service module data associated with a task identifier, automatically creating a task according to the data;

and (3) modifying the task: when the existing task is selected and opened, the remarks of the task are modified and stored;

and (4) deleting the task: the task association system is used for selecting the existing task, deleting the task and completely deleting the task association;

refreshing: for loading up-to-date data in a library

Text extraction adding attribute: the function of adding attributes is provided for text extraction, and the attributes can be added in a user-defined mode;

uploading pictures: supporting user-defined uploading of node pictures and relationship pictures;

uploading accessories: accessory supporting custom uploading task

The task is stored as follows: a user designates a task, stores the task as an operation, copies the original task and all the contents of nodes, relations and the like of the original task together, and stores the original task and all the contents of the nodes, relations and the like in a new task name;

navigation: providing a navigation button and displaying a navigation map;

and (3) navigation picture derivation: and providing an export button to export the picture in the current navigation frame.

The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.

Claims

1. a pc-based page information intelligent collection tool is characterized in that: comprising page extractable data display module, page data extraction module, page data processing module and page data display operation module;

The page extractable data display module marks and displays the extractable information in the process of the user browsing the page according to the data type information preset on the page; the preset data type includes five types of data, which are: Task data, case data, card data, relationship and object data, and activity track data. On the page, task data is prefixed with ibox_task, case data is prefixed with ibox_case, card data is prefixed with ibox_relation, relation and object data is prefixed with ibox_pro_type, and activity track is prefixed with ibox_pro_type. The data is prefixed with ibox_pro_value;

The page data extraction module extracts and classifies the page according to the identification data of different data types, and obtains the classified page data; the page data extraction module is used to realize a single extraction function and a one-key extraction function;

The page data processing module judges the relationship between the data by comparing the identification data between the page data extracted by the page data extraction module, and merges the extracted page data according to preset data type information;

The page data display operation module displays the extracted data and its relationship on the canvas in the form of nodes and connections.

2. The intelligent collection tool for page information according to claim 1, characterized in that: further comprising a follow-up analysis module, the follow-up analysis module performs a follow-up analysis operation on the contents of a plurality of nodes, lists analyzable data, and is freed by the user The checked analysis items perform secondary analysis on the checked data, and display the analysis results.

3 . The intelligent collection tool for page information according to claim 1 , wherein the mark display adopts a highlight prompt to display. 4 .

4. the page information intelligent collection tool according to claim 1, is characterized in that: the relation between described data is specifically: if it is the data of the same node relationship then merged under the same node, if two pieces of data satisfy the relational structure Then build a relationship.

5. A page information intelligent collection method of the page information intelligent collection tool according to claim 1, characterized in that: comprising the steps of:

(1) Pre-set data type information on the page, and the data type includes five types of data, which are task data, case data, card data, relationship and object data, and activity track data;

(2) During the page browsing process of the user, according to the data type information preset on the page, the extractable page data is marked and displayed, and the user filters the page data that needs to be extracted;

(3) After the user has screened the page data that needs to be extracted, data is extracted from the page and classified according to the identification data of different data types in the page data to be extracted to obtain the page data after classification;

(4) Judging the relationship between the data by comparing the identification data between the page data extracted in the step (3), and combining the page data extracted in the step (3) according to the data type information preset in the step (1). ;

(5) Display the combined result of step (4) on the canvas in the form of nodes and lines.

6. The method for intelligently collecting page information according to claim 5, wherein: in the step (5), the nodes on the canvas can be freely dragged for layout, and the nodes and the content on the nodes, The relationship and the content on the relationship line are custom edited.

7. The method for intelligently collecting page information according to claim 6, wherein the self-defined editing specifically comprises:

Manually add a node: select the node type and key level, and add it to the relationship diagram; the default display icon of the node is determined according to the type, and the node border color is selected according to the key level. If it is a personnel node, the ID card photo is selected as the node icon;

Manually add a relationship: select the relationship line and relationship type, and enter the relevant content;

Modify relationship: edit the content of the relationship, drag the relationship line to edit the position;

delete relationship;

New grouping function: create a new group for multiple or single nodes outside the group;

Group editing function: Edit the specified group, including the modification of the group name and remarks;

Node unbinding function: Unbind a single or multiple nodes, and the selected node will be directly unbound from the corresponding group;

Group cancel function: cancel the specified group.