CN111597422B

CN111597422B - Buried point mapping method, device, computer equipment and storage medium

Info

Publication number: CN111597422B
Application number: CN202010406915.8A
Authority: CN
Inventors: 罗嘉辉
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-05-14
Filing date: 2020-05-14
Publication date: 2025-08-22
Anticipated expiration: 2040-05-14
Also published as: CN111597422A

Abstract

The present application relates to a method, device, computer equipment and storage medium for mapping buried points. The method comprises: obtaining a target page, and obtaining a page snapshot corresponding to the target page based on the target page; obtaining the target buried point in the page snapshot, and the data of the first element where the target buried point is located; when it is determined based on the data of the first element that there is no buried point corresponding to the target buried point in the target page, obtaining the data of each second element in the target page; matching the data of the first element with the data of each second element respectively, and determining the similarity between each second element and the first element respectively; determining the target element from each second element based on the similarity between each second element and the first element, and mapping the target buried point to the target element in the target page. This method can be used to map the target buried point in the page snapshot to the target page.

Description

Buried point mapping method, buried point mapping device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method and apparatus for mapping a buried point, a computer device, and a storage medium.

Background

With the development of network and computer technology, more and more user information is generated. Enterprises can collect the user information for analysis, so that products which meet the requirements of users better are produced. In a traditional information acquisition method, buried points are added in a network page, and information of a user is acquired through the buried points.

However, when the web page is changed to obtain a new web page, the buried point in the web page will also fail, and there is a problem that the buried point cannot be mapped to the new web page.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a buried point mapping method, apparatus, computer device, and storage medium capable of mapping a buried point to a new page.

A buried point mapping method, the method comprising:

acquiring a target page and acquiring a page snapshot corresponding to the target page based on the target page;

acquiring a target embedded point in the page snapshot and data of a first element where the target embedded point is located;

when judging that no buried point corresponding to the target buried point exists in the target page based on the data of the first element, acquiring the data of each second element in the target page;

Matching the data of the first element with the data of each second element respectively, and determining the similarity between each second element and the first element respectively;

A target element is determined from each of the second elements based on a similarity between each of the second elements and the first element, and the target buried point is mapped to the target element in the target page.

A buried point mapping apparatus, said apparatus comprising:

The system comprises a target page and page snapshot acquisition module, a target page generation module and a page snapshot generation module, wherein the target page and page snapshot acquisition module is used for acquiring a target page and acquiring a page snapshot corresponding to the target page based on the target page;

The data acquisition module of the first element is used for acquiring a target embedded point in the page snapshot and the data of the first element where the target embedded point is located;

a data acquisition module of second elements, configured to acquire data of each second element in the target page when it is determined that no buried point corresponding to the target buried point exists in the target page based on the data of the first element;

the similarity determining module is used for respectively matching the data of the first element with the data of each second element and respectively determining the similarity between each second element and the first element;

And the embedded point mapping module is used for determining target elements from the second elements based on the similarity between the second elements and the first elements, and mapping the target embedded points to the target elements in the target page.

A computer device comprising a memory storing a computer program and a processor implementing the steps of the method described above when the processor executes the computer program.

A computer readable storage medium storing a computer program which, when executed by a processor, performs the steps of the method described above.

The embedded point mapping method, the embedded point mapping device, the computer equipment and the storage medium acquire a target page, and acquire a page snapshot corresponding to the target page based on the target page, acquire target embedded points in the page snapshot and data of first elements where the target embedded points are located, acquire data of second elements in the target page when judging that the embedded points corresponding to the target embedded points do not exist in the target page based on the data of the first elements, match the data of the first elements with the data of the second elements respectively, determine similarity between the second elements and the first elements, acquire the similarity between the second elements and the first elements where the target embedded points are located through the similarity between the second elements and the first elements, and therefore, the target elements can be determined from the second elements based on the similarity between the second elements and the first elements, and map the target embedded points to the target elements in the target page.

Drawings

FIG. 1 is a diagram of an application environment for a buried point mapping method in one embodiment;

FIG. 2 is a flow chart of a buried point mapping method in one embodiment;

FIG. 3 is a schematic diagram of a visual buried point configuration system in one embodiment;

FIG. 4 is a block diagram of a visual buried point configuration system in one embodiment;

FIG. 5 is an interface diagram of a newly built Web crawling task in one embodiment;

FIG. 6 is a diagram of the architecture of a service layer of a crawler in one embodiment;

FIG. 7 is a schematic diagram of a buried point mapping method in one embodiment;

FIG. 8 is a flow diagram of the steps for determining similarity between text elements and first elements in one embodiment;

FIG. 9 is a schematic diagram of a buried point mapping method according to another embodiment;

FIG. 10 is a block diagram of a buried point mapping apparatus in one embodiment;

FIG. 11 is an internal block diagram of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The buried point mapping method provided by the application can be applied to an application environment shown in figure 1. Wherein the terminal 102 communicates with the server 104 via a network. The terminal 102 sends a target page to the server 104, the server 104 acquires the target page and acquires a page snapshot corresponding to the target page based on the target page, acquires target embedded points in the page snapshot and data of first elements where the target embedded points are located, acquires data of second elements in the target page when judging that the embedded points corresponding to the target embedded points do not exist in the target page based on the data of the first elements, respectively matches the data of the first elements with the data of the second elements, respectively determines similarity between the second elements and the first elements, determines target elements from the second elements based on the similarity between the second elements and the first elements, and maps the target embedded points to the target elements in the target page. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices, and the server 104 may be implemented by a stand-alone server or a server cluster composed of a plurality of servers.

In one embodiment, as shown in fig. 2, a method for mapping a buried point is provided, and the method is applied to a computer device, where the computer device may be one of a server, a terminal or other devices, and includes the following steps:

Step 202, obtaining a target page, and obtaining a page snapshot corresponding to the target page based on the target page.

A page snapshot refers to a copy of a page at some point in time. The page snapshot may include text, buttons, input boxes, navigation bars, pictures, links, video, audio, and the like. Likewise, in the target page, elements such as text, buttons, input boxes, navigation bars, pictures, links, video, audio, and the like may be included.

Specifically, the computer device obtains a target page, obtains an identification of the target page, and obtains a page snapshot matching the identification of the target page from the memory based on the identification of the target page. The identification of the target page may include at least one of a number, a character string, a network address of the target page, and the like.

For example, the computer device obtains a target page, obtains a network address of the target page, matches the network address of the target page with the network address of each page snapshot in the memory in sequence, and when there is a page snapshot consistent with the network address of the target page, the page snapshot is a page snapshot corresponding to the target page.

In one embodiment, a computer device may receive target pages sent by other computer devices. In another embodiment, the computer device may also obtain a target page accessed by the access instruction input by the user.

Step 204, obtaining the target embedded point in the page snapshot and the data of the first element where the target embedded point is located.

Buried points refer to program code that is captured, processed, and transmitted for user-specific actions or events. The target buried point refers to a buried point in the page snapshot. The number of target embedded points in the page snapshot is not limited, and may be one or at least two.

The target embedded point has a specific identification, and the computer device can acquire the target embedded point from the page snapshot based on the specific identification. The specific identification of the target buried point can be set according to the needs of a user.

The first element in the page snapshot may be at least one of text, buttons, input boxes, navigation bars, pictures, links, video, audio, and the like. When the first element where the target embedded point is located is a button, the data such as the number of times the button is clicked, the time when the button is clicked, and the like can be acquired through the target embedded point. When the first element where the target buried point is located is a text, data such as the number of times the text is browsed, the duration of the text is browsed, the frequency of the text is browsed and the like can be acquired through the target buried point. When the first element where the target embedded point is located is an input box, data such as content input in the input box, duration of the input content of the input box and the like can be acquired through the target embedded point. When the first element where the target embedded point is located is a video, data such as the number of times the video is watched, the time period of being watched, and the like can be acquired through the target embedded point.

The data of the first element where the target embedded point is located refers to data related to the first element, and may include an attribute of the first element, a tag of the first element, a first element structure of the first element in a page structure of the page snapshot, and the like.

Wherein the attribute (classname) of the first element is, for example, a position of the first element, a color of the first element, a size of a font in the first element, a color of a font in the first element, or the like.

The label of the first element, such as "icon", "button", etc., can obtain the type of the first element through the semantic meaning of the label of the first element, such as "icon" of the first element, the semantic meaning article can know that the first element is an article, the label of the first element is "button", the semantic meaning button can know that the first element is a button.

The first element structure of the first element in the page structure of the page snapshot refers to the structural relationship of the first element in the DOM (Document Object Model ) tree of the page snapshot. The page snapshot comprises a DOM tree, and the structural relation of the first element in the page structure can be obtained by analyzing the DOM tree. For example, the first element A is an article, the first element B is a picture in the article, the first element C is a link in the article, and the structural relationship among the first element A, the first element B and the first element C is that the first element B and the first element C are all child nodes in the first element A, and the first element B and the first element C are in parallel relationship.

For example, when the first element where the target embedded point is located is text, the data of the first element may be the content of the text, the font of the text, the color of the text, the position of the text, the size of the area occupied by the text in the page snapshot, the first element structure of the text in the page structure of the page snapshot, and so on. When the first element where the target embedded point is located is a button, the data of the first element may be the position of the button, the size of the button, the first element structure of the button in the page structure of the page snapshot, and so on. When the first element where the target embedded point is located is a picture, the data of the first element may be the data of the size of the picture, the position of the picture, the address of the picture, the first element structure of the picture in the page structure of the page snapshot, and the like.

In step 206, when it is determined that there is no buried point corresponding to the target buried point in the target page based on the data of the first element, the data of each second element in the target page is acquired.

It can be understood that when the page structure of the page changes, the buried point in the page fails, so that operations such as data acquisition, data processing and sending cannot be completed. For example, when a target embedded point a exists in the page snapshot, the first element where the target embedded point a is a text, that is, < divid=solid > </div >, and when the structure of the page corresponding to the page snapshot is changed, for example, the structure of < divid=solid > </div > is adjusted to be < solid > </solid >, so as to obtain a target page, then the target embedded point a fails, no embedded point corresponding to the target embedded point a exists in the target page, and data of the text cannot be acquired.

The second element in the target page may be at least one of text, buttons, input boxes, navigation bars, pictures, links, video, audio, and the like. The data of the second element refers to data related to the second element, and may include an attribute of the second element, a tag of the second element, a second element structure of the second element in a page structure of the target page, and the like.

Wherein the attribute (classname) of the second element is, for example, a position of the second element, a color of the second element, a size of a font in the second element, a color of a font in the second element, or the like.

The label of the second element, such as "icon", "button", etc., can obtain the type of the second element through the semantic meaning of the label of the second element, such as "icon" of the second element, the semantic meaning article can know that the second element is an article, the label of the second element is "button", the semantic meaning button can know that the second element is a button.

The second element structure of the second element in the page structure of the target page refers to the structural relationship of the second element in the DOM tree of the target page. The target page comprises a DOM tree, and the structural relation of the second element in the page structure can be obtained by analyzing the DOM tree. For example, the second element A is an article, the second element B is a picture in the article, the second element C is a link in the article, and the structural relationship among the second element A, the second element B and the second element C is that the second element B and the second element C are all child nodes in the second element A, and the second element B and the second element C are in parallel relationship.

For example, when the second element where the target embedded point is located is text, the data of the second element may be the content of the text, the font of the text, the color of the text, the position of the text, the size of the area occupied by the text in the target page, the structure of the second element of the text in the page structure of the target page, and so on. When the second element where the target embedded point is located is a button, the data of the second element may be the position of the button, the size of the button, the structure of the second element in the page structure of the target page, and the like. When the second element where the target embedded point is located is a picture, the data of the second element may be data such as a size of the picture, a position of the picture, an address of the picture, a second element structure of the picture in a page structure of the target page, and the like.

In one embodiment, the computer device may query through puppeteer $ eval interface whether there is a buried point in the target page that corresponds to the target buried point.

Step 208, matching the data of the first element with the data of each second element, and determining the similarity between each second element and the first element.

The degree of similarity between the second element and the first element refers to the degree of similarity between the second element and the first element. The higher the similarity, the more similar the second element and the first element are.

The data of the first element can comprise data of at least one dimension, the data of the second element can comprise data of at least one dimension, for each dimension, the data of the first element and the data of the same dimension of the second element are matched through diff commands to obtain a matching result of each dimension, and then the similarity between the second element and the first element is determined according to the matching result of each dimension.

For example, the data of the first element and the data of the second element comprise two dimensions, namely a size and a position, the size of the first element and the size of the second element are matched through diff commands for the dimension of the size to obtain a matching result, the position of the first element and the position of the second element are matched through diff commands for the dimension of the position to obtain a matching result, and the similarity between the second element and the first element is determined according to the matching result obtained through the dimension of the size and the matching result obtained through the dimension of the position.

It should be noted that, when the second element does not have data in a certain dimension, and the first element has data in the dimension, the matching result may be directly generated as the data of the second element that does not have the element. Similarly, when the first element does not have data in a certain dimension and the second element does have data in the dimension, the matching result may be directly generated as the data of the first element that does not have the element.

In step 210, a target element is determined from each second element based on the similarity between each second element and the first element, and the target buried point is mapped to the target element in the target page.

In one embodiment, the computer device may determine the second element with the highest similarity as the target element. In another embodiment, the computer device may also determine the second element of the next highest similarity as the target element. In other embodiments, the computer device may also determine the second element of other similarity as the target element, without limitation.

The method comprises the steps of adding a target embedded point in a target element in a target page by computer equipment, and updating embedded point information of the target page based on the target page after the target embedded point is added. In a memory of the computer device, buried point information of the target page is stored. The buried point information may include information of a second element where the buried point is located, an event monitored by the buried point, a time when the buried point is created, and the like.

In one embodiment, after adding the target embedded point in the target element in the target page, the computer device obtains embedded point information of the target page after adding the target embedded point, and replaces the embedded point information of the target page with the embedded point information in the memory.

In another embodiment, after adding the target embedded point in the target element in the target page, the computer device obtains embedded point information of the target embedded point in the target page after adding the target embedded point, and adds the embedded point information of the target embedded point to the memory.

The embedded point mapping method comprises the steps of obtaining a target page, obtaining a page snapshot corresponding to the target page based on the target page, obtaining target embedded points in the page snapshot and data of first elements where the target embedded points are located, obtaining data of second elements in the target page when judging that the embedded points corresponding to the target embedded points do not exist in the target page based on the data of the first elements, matching the data of the first elements with the data of the second elements respectively, determining similarity between the second elements and the first elements respectively, obtaining the similarity between the second elements and the first elements where the target embedded points are located through the similarity between the second elements and the first elements, determining the target elements from the second elements based on the similarity between the second elements and the first elements, and mapping the target embedded points to the target elements in the target page.

In addition, other functions related to the embedded point, such as embedded point test, embedded point configuration, embedded point data reporting and the like, exist in the computer equipment, and the target embedded point is mapped to the target element through the embedded point mapping, so that the functions related to the embedded point in the computer equipment can be perfected.

In another embodiment, when the page snapshot corresponding to the target page is not found based on the target page, which indicates that the target page is opened for the first time, the target page is stored as the page snapshot in a memory of the computer device for subsequent embedded point mapping.

In another embodiment, when it is determined that there is a buried point corresponding to the target buried point in the target page based on the data of the first element, the computer device may perform operations such as data acquisition, processing, and transmission through the buried point in the target page.

In one embodiment, when the similarity between each second element and the first element is lower than a preset threshold, the target element corresponding to the target embedded point cannot be found in the target page, and the target embedded point can be confirmed to be permanently lost, so that alarm information is generated.

In one embodiment, a computer device may obtain a buried point configuration request, and configure a buried point in a target page according to the buried point configuration request. The buried point configuration request may include a location of the buried point configuration, an element of the buried point configuration, an event monitored by the buried point, a location of data storage collected by the buried point, and the like. The buried point configuration request may be generated by a visual buried point configuration system.

FIG. 3 is a schematic diagram of a visual buried point configuration system in one embodiment. As shown in fig. 3, in the item list, item a is included, and the user can view the buried point configured in item a by clicking the button where item a is located. The user can also configure the buried point in the newly added item by clicking a button where the newly added item is located, thereby generating a buried point configuration request.

FIG. 4 is a block diagram of a visual buried point configuration system in one embodiment. In fig. 4, the visual buried point configuration system includes three modules, SDK (Software Development Kit ) module 402, configuration module 404, and reporting module 406, respectively.

The SDK module 402 includes a unique id generator therein, which can generate an id of a buried point having a uniqueness. The SDK module 402 further performs different operations according to different policies, for example, sending a buried point configuration instruction to the configuration module 404 according to a buried point configuration instruction input by a user, communicating with the configuration module 404, receiving a buried point configuration request sent by the configuration module 404, and sending the buried point configuration request to the computer device, and for example, sending a data reporting instruction to the reporting module 406 according to a data reporting instruction input by the user, and receiving the data pulled by the reporting module 406.

After receiving the embedded point configuration instruction sent by the SDK module, the configuration module 404 may generate an embedded point configuration request through operations such as configuration pulling, configuration service, and the like, and send the embedded point configuration request to the SDK module 402. Buffering may also be provided in configuration module 404 to relieve server stress. The configuration module may also store the buried point configuration request to the ckv system. ckv is a key-value memory system. The configuration module 404 may also perform data presentation on the buried points configured in the buried point configuration request, and provide a visual interface for the user.

The reporting module 406 may receive a data reporting instruction of the SDK module 402, pull data from the buried point according to the data reporting instruction, perform a data analysis service on the pulled data, and send the pulled data to the SDK module 402. The reporting module 406 may also store the pulled data to the tdw system. tdw the system is a storage system for a distributed database.

In one embodiment, the computer equipment comprises a crawler program, a target page is acquired through the crawler program, a page snapshot corresponding to the target page is acquired based on the target page, target embedded points in the page snapshot and data of first elements where the target embedded points are located are acquired, when the fact that embedded points corresponding to the target embedded points do not exist in the target page is judged based on the data of the first elements, data of each second element in the target page are acquired, the data of the first elements are respectively matched with the data of each second element, similarity between each second element and the first elements is respectively determined, and the target elements are determined from the second elements based on the similarity between each second element and the first elements. The crawler program refers to a program for automatically executing a preset operation according to a certain rule. For example, a crawler may crawl data, may intercept interface functions called by a computer device, and so on.

In one embodiment, as shown in FIG. 5, a computer device may create a Web crawling task by a crawler. The newly built Web crawling task is divided into three steps, namely inputting a website, grabbing data and setting. In the step of inputting the website, the step of capturing the data can be executed by inputting the information such as the address of the web page to be captured, the task name, the project name, whether login is needed, and the like, and clicking the next button. After the data is captured, the captured data may be set.

In one embodiment, an architecture diagram of the service layer of the crawler is shown in FIG. 6. The crawler may provide a scheduling service that is distributed, supporting large-scale crawling of data. The scheduling service comprises a timing scheduling function, a permission management module, a crawling module, a data crawling module, a cleaning module, a warehousing configuration and warehousing distribution operation, wherein the timing scheduling function can be used for performing task configuration and agent configuration from a configuration module at regular time, the permission management is performed through the permission module, the permission is checked, after the check is passed, the crawling module can be called for performing operations such as anti-crawling, browser instance management, pre-preprocessing, content identification and the like through a browser engine, so that data crawling is performed, the crawled data is sent to the cleaning module for performing operations such as duplication removal check, sensitive word cleaning and general cleaning, and clear data is sent to the warehousing module for warehousing configuration and warehousing distribution operation. The dispatch service also includes a machine monitoring function and a task monitoring keep-alive function.

The crawler program also comprises a Web interface, an external calling interface, a service interface, a storage interface and the like, wherein Web services can be accessed through the Web interface, data can be called from the outside through the external calling interface, each service module of the crawler program can be called through the service interface, and the data can be stored in a memory or obtained from the memory through the storage interface.

The crawler program can perform visual task configuration through a visual task configuration platform, and can analyze and monitor data through a data analysis and monitoring platform. The crawler also includes an interface service. The crawler program also includes Redis (Remote Dictionary Server, remote dictionary service) services, and an elastesearch service, which is a Lucene-based search server.

In one embodiment, the data of the first element comprises a first element structure of the first element in a page structure of the page snapshot, and the method for judging whether the embedded point corresponding to the target embedded point exists in the target page based on the data of the first element comprises the steps of comparing the first element structure with second element structures of second elements included in the target page respectively, when the second element consistent with the first element structure exists, the embedded point corresponding to the target embedded point exists in the target page, and when the second element structures of the second elements included in the target page are inconsistent with the first element structure, the embedded point corresponding to the target embedded point does not exist in the target page.

The first element structure of the first element in the page structure of the page snapshot refers to the structural relationship of the first element in the DOM tree of the page snapshot. The target page includes data of a second element, the data of the second element including a second element structure of the second element in a page structure of the target page. The second element structure of the second element in the target page of the page snapshot refers to the structural relationship of the second element in the DOM tree of the target page.

The computer equipment compares the first element structure with the second element structures of the second elements included in the target page respectively, when the second element consistent with the first element structure exists, the second element structure of the second element in the target page is unchanged, the buried point corresponding to the target buried point exists in the target page, and the buried point corresponding to the second element is still valid.

When the second element structures of the second elements included in the target page are inconsistent with the first element structures, the second element structures of the second elements in the target page are changed, no buried point corresponding to the target buried point exists in the target page, and the target buried point in the page snapshot fails in the target page.

In this embodiment, by comparing the first element structures with the second element structures of the respective second elements included in the target page, it is possible to more accurately determine whether or not there is a buried point corresponding to the target buried point in the target page, that is, whether or not the target buried point in the page snapshot is valid in the target page.

In one embodiment, the data of the first element comprises the attribute of the first element and the label of the first element, the data of the second element comprises the attribute of the second element and the label of the second element, the data of the first element is matched with the data of each second element respectively, the similarity between each second element and the first element is determined respectively, the data of the first element is matched with the attribute of each second element respectively, the first value of each second element is determined according to the matching result, the label of the first element is matched with the label of each second element respectively, the second value of each second element is determined according to the matching result, and the similarity between the second element and the first element is determined based on the first value and the second value for each second element.

The attribute of the first element is, for example, the size of the area occupied by the first element, the color of the first element, the font size of the text in the first element, the font color of the text in the first element, and the like. The attribute of the second element, for example, the size of the area occupied by the second element, the color of the second element, the font size of the text in the second element, the font color of the text in the second element, and the like.

The tag of the first element indicates the type of the first element and the tag of the second element indicates the type of the second element. For example, "art" means an article, "title" means a title, "button" means a button, "character" means a paragraph, and so on.

The computer equipment matches the attribute of the first element with the attribute of the second element, when the attribute of the first element is consistent with the attribute of the second element, the matching result is that the matching is successful, the first numerical value corresponding to the matching success can be obtained, and when the attribute of the first element is inconsistent with the attribute of the second element, the matching result is that the matching is unsuccessful, and the first numerical value corresponding to the matching unsuccessful can be obtained. The first values corresponding to the different matching results can be set according to the needs of the user.

The computer equipment matches the label of the first element with the label of the second element, when the label of the first element is consistent with the label of the second element, the matching result is that the matching is successful, and a second numerical value corresponding to the matching success can be obtained, and when the label of the first element is inconsistent with the label of the second element, the matching result is that the matching is unsuccessful, and a second numerical value corresponding to the matching unsuccessful can be obtained. The second values corresponding to the different matching results may be set according to the user's needs.

In one embodiment, the first value and the second value are added for each second element, and the resulting sum is added as the similarity between the second element and the first element.

For example, the attribute of the first element is A1, the tag is B1, when the attribute of the second element is A1, the first value of the second element is 10, and when the tag of the second element is B1, the second value of the second element is 10, so that the similarity between the second element and the first element is 10+10=20.

In another embodiment, a first weight factor corresponding to an attribute of the second element is obtained, a second weight factor corresponding to a label of the second element is obtained, the first value and the first weight factor are multiplied for each second element, the second value and the second weight factor are multiplied, the obtained products are added, and finally the obtained sum is used as the similarity between the second element and the first element.

For example, the first weight factor corresponding to the attribute of the second element is 0.4, the second weight factor corresponding to the label of the second element is 0.6, the first value of the second element a is 10, the second value is 20, the first value of the second element B is 20, the first value is 15, the similarity between the second element a and the first element is 0.4x10+0.6x20=16, and the similarity between the second element B and the first element is 0.4x20+0.6x15=17.

In the embodiment, the attribute of the first element is respectively matched with the attribute of each second element, the first value of each second element is respectively determined according to the matching result, the label of the first element is respectively matched with the label of each second element, the second value of each second element is respectively determined according to the matching result, and the similarity between the second element and the first element can be more accurately determined according to the first value and the second value for each second element.

In one embodiment, the data of the first element comprises a first element structure of the first element in a page structure of the page snapshot, the first element structure is compared with second element structures of all second elements included in the target page respectively, when second elements consistent with the first element structure exist, buried points corresponding to the target buried points exist in the target page, when the second element structures of all second elements included in the target page are inconsistent with the first element structure, buried points corresponding to the target buried points do not exist in the target page, the data of the first element comprises attributes of the first element and labels of the first element, the data of the second element comprises attributes of the second element and labels of the second element, when the buried points corresponding to the target buried points do not exist in the target page based on the data of the first element, the data of all second elements in the target page are obtained, the attributes of the first element are matched with the attributes of all second elements respectively, first numerical values of all second elements are determined according to the matching results, the labels of the first element and the second element are not matched with the first numerical values of all second elements respectively, and the numerical values of all second elements are determined according to the matching results of the first numerical values and second element are determined according to the first numerical values.

In this embodiment, through the first element structure and the second element structures of the second elements included in the target page, whether the target embedded point corresponding to the target embedded point exists in the target page can be more accurately determined, when the embedded point corresponding to the target embedded point does not exist in the target page, the attribute of the first element is further matched with the attribute of the second element, and the tag of the first element is matched with the tag of the second element, so that the target element corresponding to the target embedded point can be more accurately determined from the target page.

As shown in fig. 7, the buried point mapping algorithm of the present embodiment combines DOM structure analysis, attribute analysis and tag analysis, and can more accurately determine a target element corresponding to a target buried point in a target page, thereby more accurately performing buried point mapping.

In one embodiment, as shown in fig. 8, matching the data of the first element with the data of each second element, and determining the similarity between each second element and the first element includes:

When the content of the first element includes text, a text element whose content includes text is selected from the second elements of the target page, step 802.

When the first element is an element of an input box, an article, a comment, or the like, then the content of the first element includes text. The text element refers to a second element in the target page whose content includes text.

In step 804, the text included in the content of the first element is matched with the text included in the content of each text element, so as to determine the similarity between each text element and the first element.

The text included in the content of the first element includes data of the text itself, the number of words of the text, the font of the text, and the like. Similarly, the text included in the content of the text element includes data such as the content of the text itself, the number of words of the text, and the font of the text.

Specifically, for each dimension of text included in the content of the first element and text included in the content of the text element, matching data of the same dimension, and determining similarity between the text element and the first element according to a matching result.

Determining a target element from each second element based on a similarity between each second element and the first element, comprising:

A target element is determined from each text element based on the similarity between each text element and the first element, step 806.

In the embodiment, when the content of the first element comprises text, the text elements with the content comprising the text are screened from the second elements of the target page, then the text comprising the content of the first element is matched with the text comprising the content of the text elements respectively, the similarity between the text elements and the first elements is determined respectively, the matching of the first element and all the second elements is avoided, the matching efficiency can be improved, the target element can be found more quickly, and the target buried point can be mapped to the target element more quickly.

In one embodiment, the method further comprises determining a first text concentration of the first element based on text included in the content of the first element, the first text concentration being positively correlated with a number of text included in the content of the first element, determining a second text concentration of the text element based on text included in the content of the text element, the second text concentration being positively correlated with a number of text included in the content of the text element. The method for matching the text included in the content of the first element with the text included in the content of each text element respectively to determine the similarity between each text element and the first element comprises the steps of matching the first text concentration of the first element with the second text concentration of each text element respectively to determine the similarity between each text element and the first element.

Text intensity refers to the degree of text intensity. The more text that the content of an element includes, the greater the text concentration of the element. The first text concentration refers to the degree of concentration of text included in the content of the first element. The second text concentration refers to the degree of concentration of text included in the content of the second element.

For example, the first element is an article, the content of the article includes 1000 words of text, the second element is a comment, the content of the comment includes 100 words of text, and the first text concentration of the first element is high and the second text concentration of the second element is low.

It will be appreciated that the text included in the content of the text elements in the page is typically user entered, and the developer typically modifies the page structure of the page, but does not modify the content included in the text elements in the page, so that the amount of text included in the content of the text elements does not change, i.e. the second text concentration of the text elements does not change, and by matching the first text concentration of the first element with the second text concentration of the text elements, respectively, the similarity between the first element and each text element may be more accurate, thereby more accurately determining the target element.

In one embodiment, the data of the first element comprises visual data of the first element and the data of the second element comprises visual data of the second element. The method comprises the steps of matching the visual data of the first element with the visual data of each second element respectively, and determining the similarity between each second element and the first element respectively.

The visual data includes data of colors of elements, sizes of elements, font sizes, font colors, and the like.

In a simple UI (User Interface) page, two elements are respectively a title and a text, and after the position of the title and the id of the element are adjusted, the buried point where the title is located is invalid. In this embodiment, the visual data of the first element is respectively matched with the visual data of each second element, so that the target element, i.e. the title, can be quickly found, and the target buried point is mapped to the title. For the UI test system, after the change of the page is detected, the change content can be further analyzed through a buried point mapping method.

In one embodiment, the method is applied to a server, when judging that no buried point corresponding to a target buried point exists in a target page based on data of a first element, the method further comprises the steps of acquiring second terminal identifiers of second terminals connected with the server after acquiring data of second elements in the target page, wherein the second terminal identifiers are identifiers of other terminals except the first terminal identifiers, the first terminal identifiers are identifiers of terminals for transmitting the target page to the server, generating a buried point failure instruction, and transmitting the buried point failure instruction to the second terminals where the second terminal identifiers are located, and the buried point failure instruction is used for indicating the second terminals to intercept a buried point mapping request.

The first terminal identification is an identification of a terminal that sends the target page to the server. And when the server acquires the target page and judges that the target page does not have the buried point corresponding to the target buried point, acquiring each second terminal identifier connected with the server. The second terminal identification is an identification of other terminals than the first terminal identification.

It can be understood that when the target page does not have the buried point corresponding to the target buried point, the server needs to process the target page every time it receives the target page sent by one terminal, which increases the processing pressure of the server. In this embodiment, when the server receives the target page sent by the first terminal and determines that the target embedded point does not exist in the target page, an embedded point failure instruction is generated and sent to each second terminal, where the embedded point failure instruction is used to instruct the second terminals to intercept the embedded point mapping request, which indicates that the target embedded point of the target page is being updated, and the embedded point mapping request does not need to be repeatedly sent, so that the processing pressure of the server can be reduced.

Further, after the server maps the target embedded point to the target element in the target page, the server further comprises generating embedded point effective instructions, and sending the embedded point effective instructions to the second terminals where the second terminal identifiers are located respectively, wherein the embedded point effective instructions are used for indicating the second terminals to release interception of the embedded point mapping request. The buried point mapping request includes a target page.

It can be understood that after the target embedded point is mapped to the target element in the target page, the target embedded point in the target page is effective, and operations such as data collection, specific event monitoring and the like can be performed through the target embedded point. And the server sends the embedded point effective instruction to each second terminal to instruct the second terminals to release interception of the embedded point mapping request, so that the server can receive the embedded point mapping request sent by the second terminals.

In one embodiment, as shown in fig. 9, step 1, a first terminal sends a target page to a server, step 2, the server compares the target page with a corresponding page snapshot through a diff command, step 3, when no buried point corresponding to a target buried point exists in the target page, the similarity of a first element and each second element is calculated respectively, step 4, the similarity of each second element is stored in a queue, and step 5, the second element with the highest similarity in the queue is determined to be the target element.

In the step 1, when the first terminal detects that the target page is opened, the target page is sent to the server. Further, the DOM tree of the target page may be sent to the server as a parameter. The target page may be a page in a browser.

In one embodiment, a buried point mapping method is provided, comprising the steps of:

step 1, acquiring a target page and acquiring a page snapshot corresponding to the target page based on the target page.

And 2, acquiring the target embedded point in the page snapshot and the data of the first element where the target embedded point is located.

And 3, comparing the first element structure with the second element structures of the second elements included in the target page respectively, and when the second element consistent with the first element structure exists, embedding points corresponding to the target embedding points exist in the target page.

And 4, when the second element structures of the second elements included in the target page are inconsistent with the first element structures, no buried point corresponding to the target buried point exists in the target page.

And 5, when judging that the buried points corresponding to the target buried points do not exist in the target page based on the data of the first elements, acquiring the data of each second element in the target page.

Step 6, the data of the first element comprises the attribute of the first element and the label of the first element, and the data of the second element comprises the attribute of the second element and the label of the second element;

and 7, respectively matching the labels of the first elements with the labels of the second elements, and respectively determining the second numerical values of the second elements according to the matching results.

And 8, matching the visual data of the first element with the visual data of each second element, and determining a third numerical value of each second element according to a matching result.

Step 9, for each second element, determining the similarity between the second element and the first element based on the first value, the second value and the third value.

In one embodiment, the first value, the second value, and the third value may be added, and the resulting sum added as the similarity between the second element and the first element. In another embodiment, a first weight factor corresponding to an attribute of the second element may be obtained, a second weight factor corresponding to a label of the second element may be obtained, a third weight factor corresponding to visual data of the second element may be obtained, for each second element, the first value and the first weight factor may be multiplied, the second value and the second weight factor may be multiplied, the third value and the third weight factor may be multiplied, the obtained products may be added, and finally the obtained sum may be added as the similarity between the second element and the first element.

And step 10, determining target elements from the second elements based on the similarity between the second elements and the first elements, and mapping the target buried points to the target elements in the target page.

And 11, when the content of the first element comprises text, screening text elements of which the content comprises text from the second elements of the target page.

And step 12, determining a first text concentration of the first element based on the text included in the content of the first element, wherein the first text concentration is positively correlated with the number of the texts included in the content of the first element.

And step 13, determining a second text concentration of the text element based on the text included in the content of the text element, wherein the second text concentration is positively correlated with the number of the texts included in the content of the text element.

Step 14, matching the first text concentration of the first element with the second text concentration of each text element, matching the attribute of the first element with the attribute of each second element, matching the label of the first element with the label of each second element, matching the visual data of the first element with the visual data of each second element, and determining the similarity between each text element and the first element.

And step 15, determining target elements from the text elements based on the similarity between the text elements and the first elements, and mapping the target buried points to the target elements in the target page.

And step 16, obtaining second terminal identifiers of all second terminals connected with the server, wherein the second terminal identifiers are identifiers of other terminals except the first terminal identifier, and the first terminal identifier is an identifier of a terminal for transmitting the target page to the server.

And 17, generating a buried point failure instruction, and respectively sending the buried point failure instruction to the second terminals where the second terminal identifiers are located, wherein the buried point failure instruction is used for indicating the second terminals to intercept the buried point mapping request.

It should be understood that, although the steps in the flowcharts of fig. 2 and 8 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least a portion of the steps in fig. 2 and 8 may include a plurality of steps or stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the execution of the steps or stages is not necessarily sequential, but may be performed in rotation or alternatively with at least a portion of the steps or stages in other steps or steps.

In one embodiment, as shown in fig. 10, a buried point mapping apparatus 1000 is provided, which may be a software module or a hardware module, or a combination of the two forms a part of a computer device, and specifically includes a target page and page snapshot acquiring module 1002, a first element data acquiring module 1004, a second element data acquiring module 1006, a similarity determining module 1008, and a buried point mapping module 1010, where:

the target page and page snapshot obtaining module 1002 is configured to obtain a target page, and obtain a page snapshot corresponding to the target page based on the target page.

The data obtaining module 1004 of the first element is configured to obtain the target embedded point in the page snapshot and the data of the first element where the target embedded point is located.

And a second element data acquisition module 1006, configured to acquire data of each second element in the target page when it is determined that there is no buried point corresponding to the target buried point in the target page based on the data of the first element.

And the similarity determining module 1008 is configured to match the data of the first element with the data of each second element, and determine the similarity between each second element and the first element.

The embedded point mapping module 1010 is configured to determine a target element from the second elements based on a similarity between the second elements and the first elements, and map the target embedded point to the target element in the target page.

The embedded point mapping device acquires a target page, acquires a page snapshot corresponding to the target page based on the target page, acquires target embedded points in the page snapshot and data of first elements where the target embedded points are located, acquires data of second elements in the target page when judging that the embedded points corresponding to the target embedded points do not exist in the target page based on the data of the first elements, respectively matches the data of the first elements with the data of the second elements, respectively determines similarity between the second elements and the first elements, and can acquire the similarity between the second elements and the first elements where the target embedded points are located through the similarity between the second elements and the first elements, so that the target elements are determined from the second elements based on the similarity between the second elements and the first elements, and the target embedded points are mapped to the target elements in the target page.

In one embodiment, the data of the first element includes a first element structure of the first element in a page structure of the page snapshot. The embedded point mapping device further comprises a judging module, wherein the judging module is used for comparing the first element structure with the second element structures of the second elements included in the target page respectively, when the second elements consistent with the first element structure exist, embedded points corresponding to the target embedded points exist in the target page, and when the second element structures of the second elements included in the target page are inconsistent with the first element structure, embedded points corresponding to the target embedded points do not exist in the target page.

In one embodiment, the data of the first element includes an attribute of the first element and a tag of the first element, and the data of the second element includes an attribute of the second element and a tag of the second element. The similarity determining module 1008 is further configured to match the attribute of the first element with the attribute of each second element, determine a first value of each second element according to a matching result, match the tag of the first element with the tag of each second element, determine a second value of each second element according to a matching result, and determine, for each second element, a similarity between the second element and the first element based on the first value and the second value.

In one embodiment, the similarity determining module 1008 is further configured to, when the content of the first element includes text, screen text elements whose content includes text from each second element of the target page, match the text included in the content of the first element with the text included in the content of each text element, and determine the similarity between each text element and the first element. The above-mentioned buried point mapping module 1010 is further configured to determine a target element from each text element based on a similarity between each text element and the first element.

In one embodiment, the above-mentioned buried point mapping apparatus further includes a text density obtaining module, configured to determine a first text density of the first element based on the text included in the content of the first element, where the first text density is positively correlated with the number of texts included in the content of the first element, determine a second text density of the text element based on the text included in the content of the text element, and where the second text density is positively correlated with the number of texts included in the content of the text element. The similarity determining module 1008 is further configured to match the first text concentration of the first element with the second text concentration of each text element, and determine a similarity between each text element and the first element.

In one embodiment, the data of the first element comprises visual data of the first element and the data of the second element comprises visual data of the second element. The similarity determining module 1008 is further configured to match the visual data of the first element with the visual data of each second element, and determine the similarity between each second element and the first element.

In one embodiment, the embedded point mapping module further comprises an embedded point failure instruction generating module, wherein the embedded point failure instruction generating module is used for acquiring second terminal identifiers of all second terminals connected with the server, the second terminal identifiers are identifiers of other terminals except the first terminal identifier, the first terminal identifier is an identifier of a terminal for sending a target page to the server, an embedded point failure instruction is generated and is respectively sent to the second terminals where all second terminal identifiers are located, and the embedded point failure instruction is used for indicating the second terminals to intercept embedded point mapping requests.

For specific limitations of the buried point mapping device, reference is made to the above limitation of the buried point mapping method, and no further description is given here. The above-described individual modules in the buried point mapping apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 11. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing buried point mapping data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a buried point mapping method.

It will be appreciated by those skilled in the art that the structure shown in FIG. 11 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.

Claims

1. A buried point mapping method, the method comprising:

acquiring a target page, and acquiring a page snapshot corresponding to the target page based on the target page, wherein the page snapshot refers to a copy of the page at a historical time point;

acquiring a target embedded point in the page snapshot and data of a first element where the target embedded point is located, wherein the target embedded point refers to the embedded point in the page snapshot;

when the second element structures of the second elements included in the target page are inconsistent with the first element structures, no buried point corresponding to the target buried point exists in the target page, and when judging that no buried point corresponding to the target buried point exists in the target page based on the data of the first element, the data of the second elements in the target page are acquired;

The method comprises the steps of respectively matching data of a first element with data of each second element, respectively determining the similarity between each second element and each first element, respectively matching the attribute of the first element with the attribute of each second element, respectively determining a first value of each second element according to a matching result, respectively matching a label of the first element with a label of each second element, respectively determining a second value of each second element according to a matching result, and determining the similarity between each second element and each first element based on the first value and the second value;

2. The method of claim 1, wherein the data of the first element comprises a first element structure of the first element in a page structure of the page snapshot;

The method for judging whether the embedded point corresponding to the target embedded point exists in the target page based on the data of the first element comprises the following steps:

And comparing the first element structure with the second element structures of the second elements included in the target page, and when the second elements consistent with the first element structure exist, embedding points corresponding to the target embedding points exist in the target page.

3. The method of claim 1, wherein the step of determining the position of the substrate comprises,

The matching of the data of the first element with the data of each second element respectively, and the determination of the similarity between each second element and the first element respectively comprise:

Obtaining first weight factors corresponding to the attributes of the second elements, obtaining second weight factors corresponding to the labels of the second elements, multiplying the first numerical value by the first weight factors, multiplying the second numerical value by the second weight factors for each second element, adding the obtained products, and finally adding the obtained sum to be used as the similarity between the second element and the first element.

4. The method of claim 1, wherein said matching the data of the first element with the data of each of the second elements, respectively, determines a similarity between each of the second elements and the first element, respectively, comprises:

when the content of the first element comprises text, text elements with the content comprising the text are screened from the second elements of the target page;

matching texts included in the content of the first element with texts included in the content of each text element respectively, and determining similarity between each text element and the first element respectively;

the determining a target element from each of the second elements based on the similarity between each of the second elements and the first element includes:

a target element is determined from each of the text elements based on a similarity between each of the text elements and the first element.

5. The method according to claim 4, wherein the method further comprises:

determining a first text concentration of the first element based on text included in the content of the first element, wherein the first text concentration is positively correlated with the number of text included in the content of the first element;

Determining a second text concentration of the text element based on text included in the content of the text element, the second text concentration being positively correlated with a number of text included in the content of the text element;

Matching the text included in the content of the first element with the text included in the content of each text element, and determining the similarity between each text element and the first element, wherein the matching comprises the steps of:

and matching the first text concentration degree of the first element with the second text concentration degree of each text element, and determining the similarity between each text element and the first element.

6. The method of claim 1, wherein the data of the first element comprises visual data of the first element and the data of the second element comprises visual data of the second element;

and matching the visual data of the first element with the visual data of each second element, and determining the similarity between each second element and the first element.

7. The method according to claim 1, wherein when it is determined that there is no buried point corresponding to the target buried point in the target page based on the data of the first element, after obtaining the data of each second element in the target page, the method further comprises:

Acquiring second terminal identifiers of all second terminals connected with the server, wherein the second terminal identifiers are identifiers of other terminals except a first terminal identifier, and the first terminal identifier is an identifier of a terminal for transmitting the target page to the server;

generating a buried point failure instruction, and respectively sending the buried point failure instruction to the second terminals where the second terminal identifiers are located, wherein the buried point failure instruction is used for indicating the second terminals to intercept buried point mapping requests.

8. A buried point mapping apparatus, said apparatus comprising:

the system comprises a target page and a page snapshot acquisition module, wherein the target page and the page snapshot acquisition module are used for acquiring the target page and acquiring a page snapshot corresponding to the target page based on the target page, and the page snapshot refers to a copy of the page at a historical time point;

the data acquisition module of the first element is used for acquiring a target embedded point in the page snapshot and the data of the first element where the target embedded point is located, wherein the target embedded point refers to the embedded point in the page snapshot;

A second element data acquisition module, configured to, when second element structures of each second element included in a target page are inconsistent with a first element structure, obtain data of each second element in the target page when it is determined that no buried point corresponding to the target buried point exists in the target page based on the data of the first element;

The similarity determining module is used for respectively matching the data of the first element with the data of each second element to respectively determine the similarity between each second element and the first element, and comprises the steps of respectively matching the attribute of the first element with the attribute of each second element, respectively determining a first value of each second element according to a matching result, respectively matching the label of the first element with the label of each second element, respectively determining a second value of each second element according to a matching result, and determining the similarity between each second element and the first element based on the first value and the second value;

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.

10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the method of any one of claims 1 to 7.