CN101763432A - Method for constructing lightweight webpage dynamic view - Google Patents

Abstract

The invention discloses a method for quickly constructing a dynamic view of a lightweight web page, and belongs to the technical field of computer applications. The method is as follows: 1) extracting the local script and the static embedded link of the page to be analyzed; 2) extracting the static embedded script from the static embedded link; 3) using the script execution engine to dynamically execute the local script and the static embedded script, Identifying the dynamic embedded link of the page; 4) forming an embedded document set from the document pointed to by the dynamic embedded link and the document pointed to by the static embedded link; 5) extracting the document in the page and the embedded document set 6) Take the identified static embedded pages and dynamic embedded pages as the pages to be analyzed respectively, repeat steps 1) to 5), and according to the embedded document collection and cited-cited relationship, and establish a dynamic view of the page. The invention completes the construction of the page dynamic view with less time cost and less system cost.

Description

A Lightweight Webpage Dynamic View Rapid Construction Method

technical field

The invention belongs to the technical field of computer applications, and in particular relates to a method for quickly constructing low-interaction dynamic webpage views by combining static analysis of pages and dynamic execution of scripts.

Background technique

Early web pages simply included some text and pictures. With the development of web technology in recent years, web page writers often use some embedded links and combined scripts to run more active pages in a way that does not require user interaction. Elements and more exciting content are displayed on the page.

Embedded links are a special form of hyperlinks with src attributes in HTML pages. Its characteristic is that when a browser accesses a page with an embedded link, the content pointed to by the link will be automatically loaded without the user clicking. , such as <iframe> with src attribute, <frame> points to an embedded page, and the src of <script> points to an embedded script file. The script in the page is used as a client to execute the code in real time, which greatly improves the browsing speed and interactive ability of the webpage, and realizes a real-time relationship between the webpage and the user. Often, some embedded links are generated through the dynamic execution of the script. and load it into the page.

The present invention abstracts the concept of webpage dynamic view, which corresponds to when a user visits a page, the browser automatically loads the hierarchical structure diagram of the embedded document pointed to by the embedded link. As shown in Figure 1, when a user visits page X, all embedded documents in the figure will be automatically loaded or executed by the browser even if there is no further interaction. Web pages bring users a richer experience through dynamic views, but it also brings challenges to some tasks. For example, the vulnerability attack code of a webpage Trojan can exist anywhere in the dynamic view of the webpage and be automatically loaded and executed by the browser. A comprehensive detection of a webpage Trojan must be based on the dynamic view of the webpage; for example, some public opinion analysis must also be based on the dynamic view of the webpage. Perform comprehensive content analysis of web pages.

When a user visits a page, a dynamic view of that page is automatically loaded in the browser. Therefore, a highly interactive method can be used to obtain the dynamic view of the webpage, that is, to monitor the behavior of the browser when browsing the page, and capture all the document access content without user interaction, but this method often has a large time cost and system cost.

Contents of the invention

The present invention deeply analyzes the automatic loading mechanism of current page elements, and aims to provide a method for quickly constructing a lightweight web page dynamic view, which combines static analysis of pages and dynamic execution of scripts to simulate a browser, and can take up less time. While saving system resources, quickly build a dynamic view of the web page.

For ease of description, the present invention introduces the following concepts:

1. Embedded link: refers to a special type of hyperlink containing the src attribute, which points to different types of embedded documents, such as <iframe>/<frame> embedded pages and <script> embedded scripts, etc. Embedded documents can be automatically loaded by the browser without user clicking;

2. Static embedded link: an embedded link specified in the form of a static tag in the page source file;

3. Dynamic embedded link: an embedded link automatically generated through the dynamic execution of the script;

4. Local script: the script content between <script> and </script>, relative to the embedded script with src;

5. Web page dynamic view: when the page is accessed, the hierarchical structure diagram of the embedded link is automatically loaded (as shown in Figure 1), which has the following three characteristics:

a) The web page dynamic view is composed of the visited page and the documents pointed to by embedded links such as embedded pages and embedded scripts automatically loaded by the browser when the page is accessed;

b) The embedded link in the dynamic view of the web page is either specified in the form of a static tag in the page source file or generated through the dynamic execution of the script;

c) The dynamic view of the web page has a certain hierarchical structure—that is, embedded pages also have embedded links.

Synthesize the three points of a), b) and c) in 5: the dynamic view of the webpage corresponds to when the user visits the page, the browser automatically loads the hierarchical structure diagram for the embedded document. The web page dynamic view is a tree structure, in which the document to be analyzed and the embedded document are used as node sets, the reference-reference relationship between documents is used as an edge, and the document to be analyzed is the root node of the tree. When a user visits a certain page, all embedded documents in the dynamic view of the page are automatically loaded by the browser in order.

The invention simulates the browser in a low-interactive manner, analyzes all loaded embedded documents and reference-reference relationships when the page is accessed by the browser, and thus constructs a dynamic view of the web page. The main idea is to obtain embedded links in the current page through static analysis of the page combined with dynamic script execution, further recursively analyze the embedded page through a certain strategy, and finally construct a dynamic view of the webpage corresponding to the page accessed by the browser.

The low-interaction web page dynamic view rapid construction method proposed by the present invention mainly includes the following steps:

(1) Designate the page to be analyzed as the current page

(2) Perform local script analysis and static embedded link identification on the current page through static analysis of the page. The specific processing method is as follows:

a) Local script parsing: extract the local script and submit it to step (3) to be processed

b) Static embedded link identification, identifying all static embedded links and further performing the following operations:

i. Submit the identified static embedded script to step (3) for processing

ii. Submit the identified static embedded page to step (5) for processing

(3) With the script execution engine as the core, the script is dynamically executed to identify the dynamic embedded link of the current page, mainly in the following two aspects:

a) Perform DOM simulation and use it as the context environment for script execution, which is mainly divided into the following two parts:

i. Use a scripting language to simulate the application programming interface (API) provided by the browser to the script

ii. Simulate the browser with a scripting language to automatically build a DOM tree (DocumentObject Model, Document Object Model) corresponding to the page

b) Dynamic embedded link identification: Capture the dynamically generated embedded link in the key functions related to the automatic loading of embedded links and further process it as follows:

i. The identified script is executed immediately in step (3)

ii. The identified dynamic embedded pages are processed by step (5)

(4) Steps (2) and (3) analyze the static embedded links and dynamic embedded links of the current page. The documents specified by these embedded links constitute an embedded document collection, and the current page and the embedded document collection Each document has a reference and a referenced relationship

(5) Analyze the embedded page of the current page through a certain embedded page recursive analysis strategy: the static embedded page analyzed by (2)-b)-ii and the dynamic analysis analyzed by (3)-b)-ii Embedded pages, designate them as the current page one by one, and return to step (2) for analysis

(6) According to the embedded document collection and reference-reference relationship obtained in step (4) each time, the page dynamic view construction is completed.

Advantage of the present invention and positive effect are as follows:

1. Combining static analysis of pages with dynamic execution of scripts, it can comprehensively analyze all loaded embedded documents and the reference-reference relationship between each document when the page is accessed by the browser.

2. Lightweight and low-interaction simulates the browser, and completes the construction of the dynamic view of the page with less time cost and less system cost.

Description of drawings

An example of a dynamic view of a page in Figure 1

Fig. 2 is a structural diagram of a method for quickly constructing a lightweight web page dynamic view in the present invention

Detailed ways

Below in conjunction with accompanying drawing 2, the method among the present invention is described in detail, and a concrete implementation uses the example set of data structure 1 as the structured expression of web page dynamic view,

struct node_link{

Node_Type child_type;

string parentURL;

string childURL;

                    }

Definition of data structure 1

where enum Node_Type{InlineScript, InlinePage...}

The steps of this embodiment are as follows:

(1) Use the JavaScript scripting language to simulate various DOM APIs provided by the browser to the script, and put the code into the defineDOMAPI.js file, where:

a) Define window.open(), and output the url in the parameter through print;

b) Define document.write(), which calls the callback function to further analyze the content of its parameters. The parsed script content is executed immediately in the current context by executing the eval function, and the parsed embedded link is output through print;

c) Define onmouseover, onmouseout, onload events to trigger automatically;

d) The rest of the attributes in the DOM API are assigned default values, and the rest of the functions are defined as empty function bodies;

(2) Set the URL of the document to be analyzed as the URL of the current page, and the collection S is empty

(3) Create a new folder with the URL of the current page as the name, and download the page, based on SAX (Sample API for XML), that is, "XML simple API", perform static analysis on the page, during the analysis process:

a) Every time a label is parsed, write out the JavaScript statement that builds the DOM object corresponding to the label according to the API defined in (1), and add the JavaScript statement to the buildDOMTree variable (string type) of the current page;

b) Put the URL pointed to by src into the inline_linking variable (list type) of the current page every time a static inline link is parsed;

also:

c) The parsed local script is put into the jscript variable (string type) of the current page;

d) parse to <script> with src, download its content and put it into the jscript variable of the current page;

e) Parse to <iframe> or <frame> with src, and put the URL pointed to by src into the inlinepage variable (List type) of the current page

(4) Perform script dynamic execution. The specific steps are to first create a temporary empty file tmp.js in the current directory, and then:

a) Write the content in the defineDOMAPI.js file in (1) to tmp.js

b) Write the buildDOMTree of the current page in (3)-a) into tmp.js

c) Write the jscript of the current page in (3) into tmp.js

d) Execute tmp.js with the script engine, and put the result into the output.js file in the current directory

(5) The merge module of the main program extracts the embedded link in the output.js file, puts it into the inline_linking variable of the current page, and puts the URL pointed to by the src of <iframe> or <frame> into the inlinepage variable of the current page;

(6) The inline_linking variable obtained in step (3)-b) and step (5) is a set of static inline links and dynamic inline links in the current page. Next step: according to the URL of the current page and the URL in inline_linking, construct different instances of the data structure 1, and put those instances into the set S

(7) recursively analyze the embedded pages of the current page, promptly the URL in the current page inlinepage variable is set to the URL of the current page successively, and get back to step (3);

(8) The set S represents the data representation of the dynamic view of the web page with the analysis page

Although specific embodiments and drawings of the present invention are disclosed for the purpose of illustration, the purpose is to help understand the content of the present invention and implement it accordingly, but those skilled in the art can understand that: without departing from the present invention and the appended claims Various substitutions, changes and modifications are possible within the spirit and scope of . The present invention should not be limited to the content disclosed in the preferred embodiment of the specification and the accompanying drawings, and the protection scope of the present invention is subject to the scope defined in the claims.

Claims (6)

1. a method for constructing lightweight webpage dynamic view the steps include:

1) the local script of the extraction page to be analyzed and static inline link;

2) from described static inline link, extract the static inline script;

3) utilize script executing engine dynamically to carry out described local script and described static inline script, identify the dynamic embedded link of this page;

4) document and described static inline link document pointed described dynamic embedded link is pointed constitute an embedded collection of document;

That 5) extracts document in this page and the described embedded collection of document quotes-is cited relation;

6) the static inline page that will identify according to the link of described static inline and the dynamic embedded page that identifies according to described dynamic embedded link are respectively as the page to be analyzed, repeating step 1)～5), according to the embedded collection of document that obtains and quote-be cited relation at every turn, set up page dynamic view.

2. the method for claim 1 is characterized in that described page dynamic view is a tree structure, and document wherein to be analyzed and embedded document are set of node; Quoting and be cited that to close be the limit between the document, document to be analyzed is the root node of tree.

3. method as claimed in claim 1 or 2, when it is characterized in that described script executing engine is carried out described local script and described static inline script, embedded link to the dynamic generation that captures is handled, its method is: identify the script of the embedded link of described dynamic generation, and utilize described script executing engine to carry out this script; Identify the embedded link dynamic embedded page pointed of described dynamic generation simultaneously, and with this page as the page to be analyzed.

4. method as claimed in claim 1 or 2, it is characterized in that described script executing engine utilizes DOM Document Object Model as the script executing environment, its method is: utilize script simulation browser to offer the application programming interfaces of script, utilize script simulation browser to construct the document object model tree corresponding with the page to be analyzed then.

5. method as claimed in claim 4 is characterized in that adopting JavaScript script simulation browser to offer the application programming interfaces of the various document object model tree of script.

6. the method for claim 1 is characterized in that based on SAX the described page to be analyzed being carried out static state resolves, and extracts the local script and the static inline link of this page to be analyzed.

Images