CN107196998B - Mobile Web request processing method, device and system based on data deduplication - Google Patents

Abstract

The invention discloses a mobile Web request processing method, equipment and system based on data deduplication, and belongs to the technical field of mobile Web application and data processing. Aiming at the problem that the Web performance is reduced by network transmission of most unnecessary repeated contents when a client cache file is updated in a small amount in the field of mobile Web, the method provides that a mobile Web request data deduplication mechanism is adopted in the mobile Web, duplicated data in a new version of the update file is deleted according to the cache file to generate a differential file, only the differential file is returned to the client during network transmission, and the client performs restoration according to the differential file and the cache file, so that the network transmission data amount is reduced; the method for asynchronously processing the differential file analysis and the server rendering is provided, so that the pressure of a client is reduced, the network request times of dynamic data are reduced, and the overall Web performance is improved.

Description

Mobile Web request processing method, device and system based on data deduplication

technical field

The invention belongs to the technical field of mobile web application and data processing, and more particularly, relates to a method, device and system for processing mobile web requests based on data deduplication.

Background technique

The Internet is the most important information dissemination and communication platform today, and the World Wide Web is one of the most important platforms in the Internet. With the entry of the Web 2.0 era, the content of pages is gradually diversified and complicated. How to obtain better Web performance has become a problem. research hotspots.

With the popularization of mobile devices and the rapid development of the mobile Internet, many traditional Internet applications are changing to the mobile Internet. Due to the relatively lack of system resources on the mobile side, such as small memory and storage capacity, weak processing power, poor network and connectivity, the speed of requesting to render a page is several seconds to ten seconds slower than that on the PC side. Therefore, optimization for mobile terminals is more worthy of attention.

In the era of information explosion, mobile devices with limited resources are obviously insufficient in processing too much information. Relevant studies have shown that there is a lot of redundant data in a large amount of data information in Internet applications. Removing redundant data and reducing the amount of data transmission can save limited storage space, reduce network transmission time, and save users’ time and cost. Significance.

At present, two methods are mainly used to reduce the amount of data transmission: compression technology and caching technology. The compression technology compresses each original file to reduce the file transmission volume, but cannot compress the redundancy between the files. The compressed file obtained after compressing two similar files separately still has a lot of redundant data. Cache technology can effectively reduce the number of network requests, but the cache capacity is limited and cannot eliminate the problem of data redundancy. In file systems and backup systems, data deduplication technology is used to identify and eliminate redundant data on a large scale, reducing data storage costs. However, applying the redundancy elimination technology in the traditional field directly to the mobile web will add extra operations for calculation and recovery. For mobile terminals with weak processing capabilities, it may affect the overall web performance. Therefore, for mobile web applications, To implement an efficient method that both eliminates redundant data and improves overall web performance.

SUMMARY OF THE INVENTION

In view of the above defects or improvement requirements of the prior art, the present invention provides a mobile Web request processing method, device and system based on data deduplication, the purpose of which is to reduce the transmission of a large amount of unnecessary redundant data in the process of mobile Web request processing. The problem of reducing web performance can not only effectively eliminate redundant data, but also improve overall web performance.

In order to achieve the above object, according to an aspect of the present invention, a method for processing mobile Web requests based on data deduplication is provided, comprising:

S1, the client sends a page request to the server, so that the server requests dynamic data from the general gateway interface CGI and fills in, and renders a page file, wherein the page request includes the page file;

S2, the client receives the rendered page file sent by the server;

S3. The client extracts the file version number information in the received page file, and compares it with the version number of the old version file in the cache. If it is found that the file is updated, step S4 is performed; if the file is not updated, the cache is directly used. old version files in;

S4. The client sends a delta file request to the server, the delta file request is used to request a delta file, the delta file is generated offline by the server when the file is updated, and is The difference file includes the block number corresponding to the newly added data and the unchanged data content;

S5. The client receives the delta file sent by the server, and merges the delta file with the old version file in the cache according to the block numbers corresponding to the newly added data and unchanged data content in the delta file Restore the new version of the file.

Preferably, the step S5 includes the following steps:

S51, initialize the empty string variable strResult to represent the calculated new file;

S52, scan the array in the differential file;

S53. If the content of the currently scanned array item is a string, it means that it is newly added data, and the newly added data is added to strResult; if the content of the currently scanned array item is not a character string, it means that the record is unchanged data The block number of the content, calculate the start and end positions of the content corresponding to the block number in the old version file according to the block size corresponding to the block number and the description rule, and use the interception string method to obtain the content corresponding to the description rule from the old version file. , and add that content to strResult;

S54, determine whether the scanning is over, if so, execute step S55, otherwise go to S52;

S55, returning the variable strResult as the new version file, and ending.

To achieve the above object, according to another aspect of the present invention, there is provided a mobile Web request processing method based on data deduplication, comprising:

A1, the server receives the page request sent by the client, requests dynamic data from the general gateway interface CGI and fills it, and renders the page file, wherein the page request includes the page file;

A2. The server sends the page file with the rendered data to the client, so that the client extracts the file version number information in the received page file and compares it with the version number of the old version file in the cache , and send a delta file request to the server when the file is updated, wherein the delta file request is used to request a delta file, and the delta file is generated offline by the server when the file is updated , and the block number corresponding to the newly added data and the unchanged data content is included in the difference file;

A3. The server receives the delta file request from the client, and sends the delta file to the client, so that the client corresponds to the content of the unchanged data from the newly added data in the delta file The block number of the received delta file is merged with the old version file in the cache to restore the new version file.

Preferably, the step of generating the difference file is:

T1. Divide the old version file into fixed length blocks, and calculate the hash check code value of each block as the block identifier to number each block;

T2. Perform a scrolling search in the new version file, and compare the hash check code value of the block content of the new version file in turn to see if it corresponds to the block ID of the old version file; if so, record the corresponding block number, and scroll to subscript a block Length distance; otherwise, push the data into the new data buffer, and scroll the subscript a character length distance;

T3, determine whether the content in the new version file is scrolled and searched, if so, execute step T4, otherwise go to step T2;

T4. Synthesize the new data in the new data buffer and generate a difference file.

To achieve the above object, according to another aspect of the present invention, a client is provided, comprising:

a first sending module, configured to send a page request to the server, so that the server requests dynamic data from the general gateway interface CGI and fills in and renders the page file, wherein the page request includes the page file;

a receiving module, configured to receive the rendered page file sent by the server;

The comparison and judgment module is used to extract the file version number information in the received page file, compare it with the version number of the old version file in the cache, and determine whether the file is updated;

The second sending module is configured to send a delta file request to the server when the file is updated, the delta file request is used to request a delta file, and the delta file is updated by the server in the file Time off-line generation, and the block number corresponding to the newly added data and the unchanged data content is included in the difference file;

The update module is used to receive the delta file sent by the server, and combine and restore the delta file and the old version file in the cache by the block number corresponding to the newly added data and unchanged data content in the delta file new version file.

Preferably, the update module includes:

The initialization module is used to initialize the empty string variable strResult to represent the calculated new file;

Scan module for scanning arrays in delta files;

The scan processing module is used to add new data to strResult when the content of the currently scanned array item is a string; when the content of the currently scanned array item is not a string, it means yes Record the block number of the unchanged data content, calculate the start and end positions of the content corresponding to the block number in the old version file according to the block size corresponding to the block number and the description rule, and obtain the description from the old version file by intercepting the string method The content corresponding to the rule, and add the content to strResult;

The judgment processing module is used for returning the variable strResult as the new version file when the scanning ends; when the scanning is not finished, returning to execute the operation of the scanning module.

To achieve the above purpose, according to another aspect of the present invention, a server is provided, comprising:

The first receiving module is used to receive the page request sent by the client, request dynamic data from the general gateway interface CGI and fill it, and render the page file, wherein the page request includes the page file;

The first sending module is used to send the page file with the rendered data to the client, so that the client extracts the file version number information in the received page file, and the version number of the old version file in the cache. Compare, and when the file is updated, send a delta file request to the server, wherein the delta file request is used to request a delta file;

A difference file generation module, used for offline generation of a difference file when the file is updated, and the block number corresponding to the newly added data and the unchanged data content is included in the difference file;

a second receiving module, configured to receive a differential file request from the client;

The second sending module is configured to send the delta file to the client, so that the client will receive the delta file from the block number corresponding to the newly added data and the unchanged data content in the delta file Merge with the old version of the file in the cache to restore the new version of the file.

Preferably, the delta file generation module includes:

The block module is used to divide the old version file into fixed length blocks, and calculate the hash check code value of each block as a block identifier to number each block;

The search processing module is used to perform a rolling search in the new version file, and sequentially compare whether the hash check code value of the block content of the new version file corresponds to the block ID of the old version file; if so, record the corresponding block number and scroll Subscript a block length distance; otherwise, push the data into the new data buffer, and scroll the subscript a character length distance;

The judgment processing module is used to synthesize new data in the new data buffer and generate a difference file when the content scrolling search in the new version file ends; when the content scrolling search in the new version file is not completed, return to execute the described Find the action of the processing module.

In order to achieve the above object, according to another aspect of the present invention, a mobile Web request processing system based on data deduplication is provided, including a client and a server, wherein:

The client is configured to send a page request to the server, wherein the page request includes a page file;

The server is used for receiving the page request, requesting dynamic data from the general gateway interface CGI and filling it, and rendering the page file; sending the page file with the rendered data to the client;

The client is also used to extract the file version number information in the received page file, and compare it with the version number of the old version file in the cache. If it is found that the file is updated, it will send a differential file request to the server. The difference file request is used to request the difference file, and the difference file is generated offline by the server when the file is updated; if the file is not updated, the old version file in the cache is directly used;

The server is also used to receive a delta file request from the client, send a delta file to the client, and include the newly added data and the block number corresponding to the unchanged data content in the delta file;

The client is further configured to restore the new version file by merging the received delta file with the old version file in the cache according to the block number corresponding to the newly added data and unchanged data content in the delta file.

Preferably, the server is also used to divide the old version file into fixed-length blocks, calculate the hash check code value of each block as a block identifier to number each block; perform a rolling search in the new version file, Check whether the hash check code value of the new version of the file block content corresponds to the block ID of the old version of the file; if so, record the corresponding block number, scroll the subscript a block length distance; otherwise, push the data into the new data buffer area, scroll subscript a character length distance; judge whether the content in the new version file is finished scrolling, if so, synthesize the new data in the new data buffer and generate a difference file, otherwise continue to scroll in the new version file;

The client is also used to initialize the empty string variable strResult to represent the calculated new file; scan the array in the differential file; if the content of the currently scanned array item is a string, it means that it is newly added data, and the The newly added data is added to strResult; if the content of the currently scanned array item is not a string, it means that it is the block number that records the content of the unchanged data, and the content corresponding to the block number is calculated according to the block size and description rules corresponding to the block number. Start and end positions in the old version file, use the interception string method to obtain the content corresponding to the description rule from the old version file, and add the content to strResult; judge whether the scan is over, and if so, return the variable strResult as the new version file, otherwise continue scanning the array in the delta file.

The above data deduplication-based mobile Web request processing method, device and system, referred to as DWPF (Deduplication-based Web-request Processing Framework) for short, are different from existing technical solutions: cache-based technology and compression-based technology. In general, compared with the prior art solutions, the method of the present invention can achieve the following beneficial effects:

1. Compared with the existing cache technology, the present invention takes into account the elimination of redundant data in the file and the limited cache capacity, effectively eliminates redundant data, and reduces the amount of data transmission and the size of the cache.

2. Compared with the existing compression technology, the present invention considers the elimination of redundant data between similar files, further eliminates the redundancy in the files, and reduces the amount of data transmission.

Description of drawings

1 is a schematic flowchart of a method for processing a Web request based on data deduplication disclosed in an embodiment of the present invention;

2 is a schematic flowchart of another Web request processing method based on data deduplication disclosed in an embodiment of the present invention;

3 is a schematic structural diagram of a Web request processing system based on data deduplication disclosed in an embodiment of the present invention;

4 is a schematic flowchart of a method for generating a differential file disclosed in an embodiment of the present invention;

FIG. 5 is a schematic flowchart of a method for merging a difference file and an old version file into a new version file disclosed in an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

The present invention provides a mobile Web request processing method based on data deduplication, including the following mechanisms:

1. Mobile Web request data deduplication mechanism

Existing methods to improve Web performance include reducing the amount of data transmission through compression technology or increasing the number of access requests through caching technology, but these two technologies cannot effectively improve Web performance in the case of a small amount of updates to old cached files. The present invention proposes a data deduplication mechanism for mobile web requests. When a client that caches an old version of a file requests the server for the content of a new version of the file, the server eliminates redundant data according to the version numbers of the old and new versions of the file, and generates a new version of the file. The difference between the files is sent to the client instead of the complete file or compressed file, which effectively reduces the redundant data of the file, reduces the amount of data transmission, and improves the performance of Web page requests.

2. Asynchronous deduplication and rendering mechanism

The use of mobile Web request data deduplication mechanism can effectively reduce the amount of data transmission, but the client must merge the difference file with the old file to restore the new version of the file, and the client needs to perform dynamic data request and page rendering process, which will increase Computational stress on clients with less processing power. The invention proposes an asynchronous deduplication and rendering mechanism, so that the dynamic data request and rendering page of the server and the differential file parsing of the client are performed asynchronously. This mechanism can reduce the number of network requests for dynamic data, balance the load of the server and the client, and effectively Improve overall web performance.

The present invention provides an example to illustrate the implementation process of the method of the present invention. In this example, it is assumed that the content of the old version file cached by the client is chunk1-chunk2-chunk3-chunk4-chunk5 (each chunk represents a different string divided by a fixed length), and the content of the new version file is chunk1-"hdfjkhlf"-chunk2- chunk3-"uygfiuy"-chunk5.

1 is a schematic flowchart of a method for processing a mobile Web request based on data deduplication disclosed in an embodiment of the present invention, including the following steps:

S1, the client sends a page request to the server, so that the server requests dynamic data from a common gateway interface (CommonGateway Interface, CGI) and fills in, and renders a page file, wherein the page request includes the page file;

Among them, transferring the page rendering process to the server can reduce the client's subsequent computational pressure on differential file parsing and recovery.

S2, the client receives the rendered page file sent by the server;

S3. The client extracts the file version number information in the received page file, and compares it with the version number of the old version file in the cache. If it is found that the file is updated, step S4 is performed; if the file is not updated, the cache is directly used. old version files in;

S4. The client sends a delta file request to the server, the delta file request is used to request a delta file, the delta file is generated offline by the server when the file is updated, and is The difference file includes the block number corresponding to the newly added data and the unchanged data content;

S5. The client receives the delta file sent by the server, and merges the delta file with the old version file in the cache according to the block numbers corresponding to the newly added data and unchanged data content in the delta file To restore the new version of the file, the differential file parsing process and the server-side rendering process are performed asynchronously, and the processing ends when the parsing is completed.

FIG. 2 is a schematic flowchart of another method for processing mobile Web requests based on data deduplication disclosed in an embodiment of the present invention, including the following steps:

A1, the server receives the page request sent by the client, requests dynamic data from the general gateway interface CGI and fills it, and renders the page file, wherein the page request includes the page file;

A2. The server sends the page file with the rendered data to the client, so that the client extracts the file version number information in the received page file and compares it with the version number of the old version file in the cache , and send a delta file request to the server when the file is updated, wherein the delta file request is used to request a delta file, and the delta file is generated offline by the server when the file is updated , and the block number corresponding to the newly added data and the unchanged data content is included in the difference file;

A3. The server receives the delta file request from the client, and sends the delta file to the client, so that the client corresponds to the content of the unchanged data from the newly added data in the delta file The block number of the received delta file is merged with the old version file in the cache to restore the new version file.

The embodiment of the present invention also provides a client, including:

a first sending module, configured to send a page request to the server, so that the server requests dynamic data from the general gateway interface CGI and fills in and renders the page file, wherein the page request includes the page file;

a receiving module, configured to receive the rendered page file sent by the server;

The comparison and judgment module is used to extract the file version number information in the received page file, compare it with the version number of the old version file in the cache, and determine whether the file is updated;

The second sending module is configured to send a delta file request to the server when the file is updated, the delta file request is used to request a delta file, and the delta file is updated by the server in the file Time off-line generation, and the block number corresponding to the newly added data and the unchanged data content is included in the difference file;

The update module is used to receive the delta file sent by the server, and combine and restore the delta file and the old version file in the cache by the block number corresponding to the newly added data and unchanged data content in the delta file new version file.

The embodiment of the present invention also provides a server, including:

The first receiving module is used to receive the page request sent by the client, request dynamic data from the general gateway interface CGI and fill it, and render the page file, wherein the page request includes the page file;

The first sending module is used to send the page file with the rendered data to the client, so that the client extracts the file version number information in the received page file, and the version number of the old version file in the cache. Compare, and when the file is updated, send a delta file request to the server, wherein the delta file request is used to request a delta file;

A difference file generation module, used for offline generation of a difference file when the file is updated, and the block number corresponding to the newly added data and the unchanged data content is included in the difference file;

a second receiving module, configured to receive a differential file request from the client;

The second sending module is configured to send the delta file to the client, so that the client will receive the delta file from the block number corresponding to the newly added data and the unchanged data content in the delta file Merge with the old version of the file in the cache to restore the new version of the file.

Figure 3 is a schematic structural diagram of a mobile Web request processing system based on data deduplication disclosed in an embodiment of the present invention. The system shown in Figure 3 includes a client and a server, wherein:

Client: The old version of the file is cached, which is used to send a web request to the server, as well as the subsequent request and parsing process for the difference file between different versions of the file.

Server-side: Contains a delta file generator and page parser for handling web requests from clients. The delta file generator is used to generate delta files between different versions of files offline when the file is updated, and the page parser is used to request dynamic data and render pages.

FIG. 4 is a schematic flowchart of a method for generating a difference file disclosed in an embodiment of the present invention, including the following operations:

T1. According to the configuration, the old version file is divided into fixed length chunks, and the hash check code value of each chunk is calculated and used as the chunk identifier and numbered for each chunk. The chunked content is chunk1, chunk2, chunk3 , chunk4, chunk5 are consecutive 5 blocks, the block numbers are 1, 2, 3, 4, 5 respectively;

T2. Perform a rolling search in the new version file, and compare the hash check code value of the block content of the new version file in turn to see if it corresponds to the block ID of the old version file; if so, record the corresponding block number, and scroll to subscript one Block length distance; otherwise, push the data into the new data buffer, and scroll the subscript a character length distance; in this embodiment, after scrolling search, the content of chunk1 corresponds to the old version, and record 1 to the difference file; " hdfjkhlf" does not correspond to the old version, and is recorded to the difference file; and so on, write 2, 3, "uygfiuy", 5 to the difference file respectively;

T3. The content of the difference file is finally obtained as {1, "hdfjkhlf", 2, 3, "uygfiuy", 5}.

5 is a schematic flowchart of a method for merging a difference file and an old version file into a new version file disclosed in an embodiment of the present invention, including the following operations:

S51. Use the variable strResult to represent the calculated new file, and initialize it to an empty string "";

S52, scan the differential file array s={1, "hdfjkhlf", 2, 3, "uygfiuy", 5};

S53. If the content of the current array item is a string, it means that it is newly added data and is directly added to strResult; if it is not a string, it means that it is the block number that records the unchanged content, which is calculated according to the block size and description rules This part of the content is at the start and end positions of the original file, and the real content corresponding to the description rule is obtained from the original file by the method of intercepting the string, and the content is added to strResult; in this embodiment, after scanning, it is found that s[ 0]=1, not a string, find the corresponding block content in the old version file, write the content of chunk1 to strResult; s[1]="hdfjkhlf", is a string, add it directly to strResult; and so on, continue to add chunk2, chunk3, "uygfiuy", chunk5 to strResult;

S54. The new version file is chunk1-"hdfjkhlf"-chunk2-chunk3-"uygfiuy"-chunk5.

Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.

Claims (8)

1. a mobile web request processing method based on data deduplication, is characterized in that, comprises: S1, the client sends a page request to the server, so that the server requests dynamic data from the general gateway interface CGI and fills it in, and renders a page file, wherein the page request includes the page file; transfer the page rendering process to the service The client can reduce the subsequent calculation pressure of the client for differential file parsing and recovery; S2, the client receives the rendered page file sent by the server; S3. The client extracts the file version number information in the received page file, and compares it with the version number of the old version file in the cache. If it is found that the file is updated, step S4 is performed; if the file is not updated, the cache is directly used. old version files in; S4. The client sends a delta file request to the server, the delta file request is used to request a delta file, the delta file is generated offline by the server when the file is updated, and is The difference file includes the block number corresponding to the newly added data and the unchanged data content; S5. The client receives the delta file sent by the server, and merges the delta file with the old version file in the cache according to the block numbers corresponding to the newly added data and unchanged data content in the delta file Restore the new version of the file; the differential file parsing process is asynchronous with the server-side rendering process. 2. The method according to claim 1, wherein the step S5 comprises the following steps: S51, initialize the empty string variable strResult to represent the calculated new file; S52, scan the array in the differential file; S53. If the content of the currently scanned array item is a string, it means that it is newly added data, and the newly added data is added to strResult; if the content of the currently scanned array item is not a character string, it means that the record is unchanged data The block number of the content, calculate the start and end positions of the content corresponding to the block number in the old version file according to the block size corresponding to the block number and the description rule, and use the interception string method to obtain the content corresponding to the description rule from the old version file. , and add that content to strResult; S54, determine whether the scanning is over, if so, execute step S55, otherwise go to S52; S55, returning the variable strResult as the new version file, and ending. 3. a mobile Web request processing method based on data deduplication, is characterized in that, comprises: A1, the server receives the page request sent by the client, requests dynamic data from the general gateway interface CGI and fills it, and renders the page file, wherein the page request includes the page file; A2. The server sends the page file with the rendered data to the client, so that the client extracts the file version number information in the received page file and compares it with the version number of the old version file in the cache , and send a delta file request to the server when the file is updated, wherein the delta file request is used to request a delta file, and the delta file is generated offline by the server when the file is updated , and the block number corresponding to the newly added data and the unchanged data content is included in the difference file; A3. The server receives the delta file request from the client, and sends the delta file to the client, so that the client corresponds to the content of the unchanged data from the newly added data in the delta file The block number of the received difference file is merged with the old version file in the cache to restore the new version file; The steps of generating the difference file are: T1. Divide the old version file into fixed length blocks, and calculate the hash check code value of each block as the block identifier to number each block; T2. Perform a scrolling search in the new version file, and compare the hash check code value of the block content of the new version file in turn to see if it corresponds to the block ID of the old version file; if so, record the corresponding block number, and scroll to subscript a block Length distance; otherwise, push the data into the new data buffer, and scroll the subscript a character length distance; T3, determine whether the content in the new version file is scrolled and searched, if so, execute step T4, otherwise go to step T2; T4. Synthesize the new data in the new data buffer and generate a difference file. 4. A client, characterized in that, comprising: a first sending module, configured to send a page request to the server, so that the server requests dynamic data from the general gateway interface CGI and fills in and renders the page file, wherein the page request includes the page file; a receiving module, configured to receive the rendered page file sent by the server; The comparison and judgment module is used to extract the file version number information in the received page file, compare it with the version number of the old version file in the cache, and determine whether the file is updated; The second sending module is configured to send a delta file request to the server when the file is updated, the delta file request is used to request a delta file, and the delta file is updated by the server in the file Time off-line generation, and the block number corresponding to the newly added data and the unchanged data content is included in the difference file; The update module is used to receive the delta file sent by the server, and combine and restore the delta file and the old version file in the cache by the block number corresponding to the newly added data and unchanged data content in the delta file new version file. 5. client according to claim 4, is characterized in that, described update module comprises: The initialization module is used to initialize the empty string variable strResult to represent the calculated new file; Scan module for scanning arrays in delta files; The scan processing module is used to add new data to strResult when the content of the currently scanned array item is a string; when the content of the currently scanned array item is not a string, it means yes Record the block number of the unchanged data content, calculate the start and end positions of the content corresponding to the block number in the old version file according to the block size corresponding to the block number and the description rule, and obtain the description from the old version file by intercepting the string method The content corresponding to the rule, and add the content to strResult; The judgment processing module is used for returning the variable strResult as the new version file when the scanning ends; when the scanning is not finished, returning to execute the operation of the scanning module. 6. A server, characterized in that, comprising: The first receiving module is used to receive the page request sent by the client, request dynamic data from the general gateway interface CGI and fill it, and render the page file, wherein the page request includes the page file; The first sending module is used to send the page file with the rendered data to the client, so that the client extracts the file version number information in the received page file, and the version number of the old version file in the cache. Compare, and when the file is updated, send a delta file request to the server, wherein the delta file request is used to request a delta file; A difference file generation module, used for offline generation of a difference file when the file is updated, and the block number corresponding to the newly added data and the unchanged data content is included in the difference file; a second receiving module, configured to receive a differential file request from the client; The second sending module is configured to send the delta file to the client, so that the client will receive the delta file from the block number corresponding to the newly added data and the unchanged data content in the delta file Merge with the old version of the file in the cache to restore the new version of the file; The delta file generation module includes: The block module is used to divide the old version file into fixed length blocks, and calculate the hash check code value of each block as a block identifier to number each block; The search processing module is used to perform a rolling search in the new version file, and sequentially compare whether the hash check code value of the block content of the new version file corresponds to the block ID of the old version file; if so, record the corresponding block number and scroll Subscript a block length distance; otherwise, push the data into the new data buffer, and scroll the subscript a character length distance; The judgment processing module is used to synthesize new data in the new data buffer and generate a difference file when the content scrolling search in the new version file ends; when the content scrolling search in the new version file is not completed, return to execute the described Find the action of the processing module. 7. a mobile Web request processing system based on data deduplication, is characterized in that, comprises client and server, wherein: The client is configured to send a page request to the server, wherein the page request includes a page file; The server is used for receiving the page request, requesting dynamic data from the general gateway interface CGI and filling it, and rendering the page file; sending the page file with the rendered data to the client; The client is also used to extract the file version number information in the received page file, and compare it with the version number of the old version file in the cache. If it is found that the file is updated, it will send a differential file request to the server. The difference file request is used to request the difference file, and the difference file is generated offline by the server when the file is updated; if the file is not updated, the old version file in the cache is directly used; The server is also used to receive a delta file request from the client, send a delta file to the client, and include the newly added data and the block number corresponding to the unchanged data content in the delta file; The client is further configured to restore the new version file by merging the received delta file with the old version file in the cache according to the block number corresponding to the newly added data and unchanged data content in the delta file. 8. The system of claim 7, wherein: The server is also used to divide the old version file into blocks according to a fixed length, and calculate the hash check code value of each block as a block identifier to number each block; perform a rolling search in the new version file, and compare them in turn. Whether the hash check code value of the block content of the new version of the file corresponds to the block ID of the old version of the file; if so, record the corresponding block number, and scroll the subscript a block length distance; otherwise, push the data into the new data buffer and scroll Subscript a character length distance; judge whether the content in the new version file is finished scrolling, if so, synthesize the new data in the new data buffer and generate a difference file, otherwise continue to scroll in the new version file; The client is also used to initialize the empty string variable strResult to represent the calculated new file; scan the array in the differential file; if the content of the currently scanned array item is a string, it means that it is newly added data, and the The newly added data is added to strResult; if the content of the currently scanned array item is not a string, it means that it is the block number that records the content of the unchanged data, and the content corresponding to the block number is calculated according to the block size and description rules corresponding to the block number. Start and end positions in the old version file, use the interception string method to obtain the content corresponding to the description rule from the old version file, and add the content to strResult; judge whether the scan is over, and if so, return the variable strResult as the new version file, otherwise continue scanning the array in the delta file.

Images