CN110611713A - A data download method, system, electronic device and storage medium - Google Patents

Abstract

The present application discloses a data download method and system, an electronic device and a computer-readable storage medium. The method includes: using a regular expression to screen nodes to be processed to obtain a target node; generating a clone node of the target node, and moving Remove the original monitoring event in the clone node; add a target monitoring event to the clone node, and use the target monitoring event to obtain the download link of the clone node; use the download link to perform a data download operation, the present application Can improve data download stability.

Description

A data download method, system, electronic device and storage medium

technical field

The present application relates to the field of computer technology, and more specifically, to a data downloading method and system, an electronic device, and a computer-readable storage medium.

Background technique

With the development of network technology, obtaining data through network download has become a common data acquisition method.

It is a relatively common data download method to realize data download by obtaining the download link of the third-party webpage in the APP, but the third-party webpage often has situations such as web page jumps, resulting in the inability to obtain stable and efficient download data.

Therefore, how to improve data download stability is a problem to be solved by those skilled in the art.

Contents of the invention

The purpose of the present application is to provide a data downloading method and system, an electronic device and a computer-readable storage medium, which improve the stability of data downloading.

In order to achieve the above purpose, the application provides a data download method, including:

Use regular expressions to filter nodes to be processed to obtain target nodes;

Generate a clone node of the target node, and remove the original listening event in the clone node;

Adding a target monitoring event to the clone node, and using the target monitoring event to obtain a download link of the clone node;

Use the download link to perform a data download operation.

Optionally, also include:

Call the stopPropagation method of the target event to prevent events added through event delegation.

Optionally, also include:

Calling the PreventDefault method of the target event prevents the element's default behavior.

Optionally, also include:

Judging whether the URL of the target node changes;

If so, generate a new clone node and remove the original listening event in the new clone node;

Adding the target listening event to the new clone node, and using the target listening event to obtain a download link of the new clone node.

Optionally, after obtaining the download link of the new clone node by using the target listening event, the method further includes:

A data download operation is performed using the download link of the new clone node.

Optionally, generating the clone node of the target node includes:

A clone node of the target node is generated by the method of cloneNode.

Optionally, using regular expressions to filter nodes to be processed to obtain target nodes includes:

The node to be processed whose download link suffix matches the regular expression is set as the target node.

In order to achieve the above purpose, the application provides a data download system, including:

A node screening module, used to filter nodes to be processed by regular expressions to obtain target nodes;

A monitoring event removal module, configured to generate a clone node of the target node, and remove the original monitoring event in the clone node;

A link obtaining module, configured to add a target monitoring event to the clone node, and use the target monitoring event to obtain a download link of the clone node;

A data download module, configured to use the download link to perform a data download operation.

To achieve the above object, the present application provides an electronic device, the electronic device includes a memory and a processor, the memory stores a data download program that can run on the processor, and the data download program is executed by the When the processor executes, it realizes: using regular expressions to filter nodes to be processed to obtain a target node; generating a clone node of the target node, and removing the original monitoring event in the clone node; adding target monitoring to the clone node event, and use the target listener event to obtain the download link of the clone node; use the download link to execute a data download operation.

To achieve the above object, the present application provides a computer-readable storage medium, on which a data download program is stored, and when the data download program is executed by a processor, the steps of the above-mentioned data download method are implemented.

To achieve the above purpose, the present application provides a computer program product, including computer instructions, which, when run on a computer, enable the computer to execute the data downloading method described in any one of the above.

It can be seen from the above scheme that a data download method provided by the present application includes: using regular expressions to filter the nodes to be processed to obtain the target node; generating a clone node of the target node, and removing the original listening event in the clone node ; Adding a target monitoring event to the clone node, and using the target monitoring event to obtain a download link of the clone node; using the download link to perform a data download operation. This application uses a regular expression to select a node for data download. By cloning the node and removing the original listening event in the cloned node, web page jumps can be avoided, and the download link of the cloned node can be obtained by using the target listening event, and the data provided by the node can be downloaded. Data, the above process improves the stability of data download and improves the success rate of obtaining download links.

The application also discloses a data downloading system, an electronic device and a computer-readable storage medium, which can also achieve the above-mentioned technical effects.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a flowchart of a data download method disclosed in an embodiment of the present application;

FIG. 2 is a flow chart of another data downloading method provided by the embodiment of the present application;

FIG. 3 is a structural diagram of a data download system provided by an embodiment of the present application;

FIG. 4 is a structural diagram of an electronic device provided in an embodiment of the present application;

FIG. 5 is a structural diagram of another electronic device disclosed in an embodiment of the present application.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

It should be noted that the descriptions involving "first", "second", etc. in the present invention are only for descriptive purposes, and should not be understood as indicating or implying their relative importance or implicitly indicating the number of indicated technical features . Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In addition, the technical solutions of the various embodiments can be combined with each other, but it must be based on the realization of those skilled in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist , nor within the scope of protection required by the present invention.

In the prior art, the download link is obtained by directly adding a monitoring event to the target node to provide download service for the user, but the download link obtained by using the prior art is easily affected by the webpage jump, and the stability of data download is relatively low. Low.

Therefore, the present application provides a new data download method through the following several embodiments, which can improve the stability of data download.

Referring to FIG. 1, a flow chart of a data download method disclosed in the embodiment of the present application, as shown in FIG. 1, includes:

S101: Use regular expressions to filter nodes to be processed to obtain target nodes;

Wherein, the purpose of this step is to obtain a target node from a large number of nodes to be processed. The node to be processed may be a node with a label named a, that is, a node with a download link. The node in this embodiment may be a JavaScript web page containing a download link.

This embodiment does not limit the specific form of the regular expression. The following example illustrates the process of filtering nodes to be processed through regular expressions:

For example, regular expressions can be judged by http and ftp link suffixes to filter. The regular expression for video download link can be: /^(.*\.)? (wmv|asf|asx|rm|rmvb|mpeg|mp4|mpg|3gp|3g2|mov|m4v|avi|dat|mkv|flv|vob|webm|ogv|ogx|swf)[]*$/; for Can the regular expression of the audio download link be /^(.*\.)? (mp3|m4a|wav|aac|aif|au|ram|wma|mmf|amr|flac|weba|oga|ogg)[]*$/; the regular expression for the text file download link can be /^(. *\.)? (txt|nfo|info|doc|docx|htm|xhtml|html|xml|ppt|pptx|pdf)[]*$/; for image download links, the regular expression can be: /^(.*\.)? (bmp|jpg|png|tif|gif|pcx|tga|exif|fpx|svg|psd|cdr|pcd|dxf|ufo|eps|ai|raw|WMF|webp)[]*$/; for compressed files The download link regular expression can be: /^(.*\.)? (zip|rar|7z|tar|tgz)[]*$/; For the download link of the installation package, the regular expression can be /^(.*\.)? apk$/; for other types of download links, the regular expression can be /^(thunder|flashget|qqdl|magnet|ed2k):/.

Nodes that do not match the regular expression can be filtered out through the above method, for example, the following tags exist:

(1) <aclass="download" href="http://www.foo.bar/asdas">;

(2) <aclass="download" href="http://www.foo.bar/a.mp4">;

(3) <divclass="a-tag">;

(4) <ahref="magnet:?xt=urn:btih:AC9AE47134CE7A...">;

Among them, (3) is not a tag, (1), (2) and (4) are all a tags, but the href attribute of (1) does not pass the verification of the above two rules, (2) is http and the suffix is mp4, Pass the first type of regular verification; (4) is a magnet link, pass the second type of regular verification, and pass the verification. So (2) and (4) can be marked as download links and can be used as target nodes.

S102: Generate a clone node of the target node, and remove the original monitoring event in the clone node;

Specifically, in this embodiment, the clone node can be obtained by cloning the target node through the cloneNode method, so as to remove the listening event in the addEventListener manner. The events monitored by the addEventListener method cannot be seen intuitively, so they are easy to be ignored. Clone and replace the node of the label to remove all events added by the addEventListener method to prevent unpredictable web events from affecting hijacking.

In this step, there may also be an operation of removing other original listening events in the cloned node, for example, removing the original listening events written in the node by removing corresponding attributes.

S103: Add a target monitoring event to the clone node, and use the target monitoring event to obtain a download link of the clone node;

The purpose of this step is to add a custom event to the clone node, that is, the target monitoring event, and the target monitoring event can be used to obtain the download link of the clone node. Of course, the target listening event can also prevent bubbling by means of stopPropagation, which can prevent triggering of events added through event delegation.

Specifically, the process of using the target listening event to obtain the download link of the cloned node can be as follows:

S104: Perform a data download operation by using the download link.

On the basis of obtaining the download link of the target node, this embodiment can use the download link to download data.

In this embodiment, the node for data download is selected through a regular expression. By cloning the node and removing the original monitoring event in the clone node, the situation of webpage jump can be avoided, and the download link of the clone node can be obtained by using the target monitoring event, and the node can be downloaded. The above process improves the stability of data downloading and the success rate of obtaining download links.

The embodiment of the present application discloses a data downloading method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Please refer to FIG. 2 for details. FIG. 2 is a flow chart of another data downloading method provided by the embodiment of the present application, as shown in FIG. 2 , including:

S201: Set the node to be processed whose download link suffix matches the regular expression as the target node;

S202: Generate a clone node of the target node through the cloneNode method, and remove the original listening event in the clone node;

Wherein, the process of removing the original listening event of the cloned node may include, for example, removing the listening event onclick written in HTML, $element.removeAttribute('onclick').

S203: Call the stopPropagation method of the target event to stop the event added through event delegation.

Wherein, this embodiment can prevent bubbling by calling the stopPropagation method in the custom event. Blocking bubbling prevents events added via event delegation from firing.

The process of preventing bubbling can be:

$element.addEventListener('click',(e)=>{

e. stopPropagation();

}).

S204: Call the PreventDefault method of the target event to prevent the default behavior of the element.

Among them, calling the preventDefault method in the target event to prevent the default behavior of the element can prevent the default jump behavior of the a tag. The specific implementation method can be as follows:

S205: Add a target monitoring event to the clone node, and use the target monitoring event to obtain a download link of the clone node;

S206: Perform a data download operation by using the download link.

In this embodiment, multiple methods are used to remove the original monitoring event and add a target monitoring event, which ensures the completeness of link acquisition and prevents link acquisition such as page jump from being affected.

As a feasible implementation manner, after S206, there may be a situation of monitoring the Url change of the current node, judging that the node has performed a single-page jump, and repeating the above steps. Specifically, the following steps may be included:

Step 1: judging whether the URL of the target node has changed; if so, proceed to step 2;

Step 2: Generate a new clone node and remove the original listening event in the new clone node;

Step 3: Add the target monitoring event to the new clone node, and use the target monitoring event to obtain a download link of the new clone node.

Step 4: Perform a data download operation using the download link of the new clone node.

Wherein, URL (Uniform Resource Locator) is a Uniform Resource Locator, and this embodiment can generate a new clone node for the target node when the URL changes. Remove the original listening time of the new clone node and add the target listening event so as to obtain the download link again. It can be understood that after the new clone node is generated, there may also be an operation of calling the stopPropagation method of the target event to prevent the event added through event delegation, or there may be a call of the PreventDefault method of the target event to prevent the default behavior of the element operation in order to improve the completeness of link acquisition.

A data download system provided in the embodiment of the present application is introduced below, and a data download system described below and a data download method described above may refer to each other.

Referring to FIG. 3, a structural diagram of a data downloading system provided in an embodiment of the present application, as shown in FIG. 3, includes:

A node screening module 100, configured to utilize regular expressions to screen nodes to be processed to obtain target nodes;

A monitoring event removal module 200, configured to generate a clone node of the target node, and remove the original monitoring event in the clone node;

A link obtaining module 300, configured to add a target monitoring event to the clone node, and use the target monitoring event to obtain a download link of the clone node;

The data downloading module 400 is configured to use the download link to perform a data downloading operation.

In this embodiment, the node for data download is selected through a regular expression. By cloning the node and removing the original monitoring event in the clone node, the situation of webpage jump can be avoided, and the download link of the clone node can be obtained by using the target monitoring event, and the node can be downloaded. The above process improves the stability of data downloading and the success rate of obtaining download links.

Further, it also includes:

The bubbling prevention module is used to call the stopPropagation method of the target event to prevent events added through event delegation.

Further, it also includes:

The default behavior prevention module is used to call the PreventDefault method of the target event to prevent the default behavior of the element.

Further, it also includes:

The URL judging module is used to judge whether the URL of the target node changes; if so, then generate a new clone node and remove the former listening event in the new clone node;

A new link obtaining module, configured to add the target monitoring event to the new clone node, and use the target monitoring event to obtain the download link of the new clone node, so as to use the download link of the new clone node The link performs a data download operation.

Further, the monitoring event removal module 200 includes:

a cloning unit, configured to generate a clone node of the target node through a cloneNode method;

an event removal unit, configured to remove the original listening event in the clone node;

Further, the node screening module 100 is specifically a module for setting a node to be processed whose download link suffix conforms to the regular expression as a target node.

The present application also provides an electronic device, which may be a PC (Personal Computer, personal computer), or may be a terminal device such as a smart phone, a tablet computer, a palmtop computer, a portable computer, or an intelligent router.

Referring to FIG. 4 , a structural diagram of an electronic device provided by an embodiment of the present application, as shown in FIG. 4 , may include a memory 11 , a processor 12 and a bus 13 .

Wherein, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (eg, SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device, such as a hard disk of the electronic device. The memory 11 may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk equipped on the electronic device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, Flash card (FlashCard), etc. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store the application software and various data installed in the electronic device, such as the code of the data download program 01, etc., but also can be used to temporarily store the data that has been output or will be output.

In some embodiments, the processor 12 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips for running program codes or processing stored in the memory 11. Data, implement the data download method provided by any of the above embodiments, for example, execute the data download program 01 and so on.

The bus 13 may be a peripheral component interconnect standard (PCI for short) bus or an extended industry standard architecture (EISA for short) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 4 , but it does not mean that there is only one bus or one type of bus.

On the basis of the foregoing embodiments, as a preferred implementation manner, referring to FIG. 5 , the electronic device further includes:

The input interface 14 is used to obtain computer programs, parameters and instructions imported from the outside, which are stored in the memory 11 under the control of the processor 12 . The input interface 14 can be connected with an input device to receive parameters or instructions manually input by the user. The input device may be a touch layer covered on the display screen, or may be a button, a trackball, or a touch pad provided on the terminal shell, or may be a keyboard, a touch pad, or a mouse.

The display unit 15 is used for displaying data processed by the processor 12 and for displaying a visualized user interface. The display unit 15 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, Organic Light-Emitting Diode) touch device, and the like.

The network port 16 is used for communicating with various external terminal devices. The communication technology used in the communication connection can be wired communication technology or wireless communication technology, such as mobile high-definition link technology (MHL), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, communication technology based on IEEE802.11s, etc.

Figure 5 only shows an electronic device with components 11-16 and a data download program 01, those skilled in the art can understand that the structure shown in Figure 5 does not constitute a limitation on the electronic device, and may include more Fewer or more components, or combinations of certain components, or different arrangements of components.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present invention will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, or a magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, a Solid State Disk (SSD)).

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes.

It should be noted that the serial numbers of the above embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments. And herein the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, apparatus, article or method comprising a set of elements includes not only those elements, but also includes the elements not expressly included. other elements listed, or also include elements inherent in the process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional same elements in the process, apparatus, article or method comprising the element.

The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process conversion made by using the description of the present invention and the contents of the accompanying drawings, or directly or indirectly used in other related technical fields , are all included in the scope of patent protection of the present invention in the same way.

Claims (10)

1. A method for downloading data, comprising:

screening nodes to be processed by using a regular expression to obtain target nodes;

generating a clone node of the target node, and removing an original monitoring event in the clone node;

adding a target monitoring event into the clone node, and acquiring a download link of the clone node by using the target monitoring event;

and executing data downloading operation by utilizing the downloading link.

2. The data downloading method of claim 1, further comprising:

and calling the stopPropropagation method of the target event to stop the event added by the event delegation mode.

3. The data downloading method of claim 1, further comprising:

invoking a PreventDefault method of the target event prevents element default behavior.

4. The data downloading method of claim 1, further comprising:

judging whether the URL of the target node changes or not;

if so, generating a new clone node and removing an original monitoring event in the new clone node;

and adding the target monitoring event into the new clone node, and acquiring a download link of the new clone node by using the target monitoring event.

5. The data downloading method of claim 1, after acquiring the download link of the new clone node using the target snoop event, further comprising:

and executing data downloading operation by utilizing the downloading link of the new clone node.

6. The data download method of claim 1, wherein generating the clone node of the target node comprises:

generating a clone node of the target node by a method of a cloneNode.

7. The data downloading method of any one of claims 1 to 6, wherein the screening the nodes to be processed to obtain the target node by using the regular expression comprises:

and setting the nodes to be processed with the download link suffixes conforming to the regular expression as target nodes.

8. A data download system, comprising:

the node screening module is used for screening the nodes to be processed by utilizing the regular expression to obtain target nodes;

a monitoring event removing module, configured to generate a clone node of the target node, and remove an original monitoring event in the clone node;

the link acquisition module is used for adding a target monitoring event into the clone node and acquiring a download link of the clone node by using the target monitoring event;

and the data downloading module is used for executing data downloading operation by utilizing the downloading link.

9. An electronic device comprising a memory and a processor, the memory having stored thereon a data download program operable on the processor, the data download program when executed by the processor, to: screening nodes to be processed by using a regular expression to obtain target nodes; generating a clone node of the target node, and removing an original monitoring event in the clone node; adding a target monitoring event into the clone node, and acquiring a download link of the clone node by using the target monitoring event; and executing data downloading operation by utilizing the downloading link.

10. A computer-readable storage medium, having stored thereon a data downloading program, which when executed by a processor, implements the steps of the data downloading method according to any one of claims 1 to 7.