CN112579321A - Method, device and equipment for downloading service data - Google Patents
Method, device and equipment for downloading service data Download PDFInfo
- Publication number
- CN112579321A CN112579321A CN202011539109.4A CN202011539109A CN112579321A CN 112579321 A CN112579321 A CN 112579321A CN 202011539109 A CN202011539109 A CN 202011539109A CN 112579321 A CN112579321 A CN 112579321A
- Authority
- CN
- China
- Prior art keywords
- service data
- download
- target service
- request message
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Human Computer Interaction (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the application provides a method, a device and equipment for downloading service data, wherein the method comprises the following steps: the terminal equipment sends a downloading request message to the server, the server acquires target service data from a service database according to the downloading request message, writes the target service data into a file, stores the file into a file system, then generates a downloading log corresponding to the downloading request message according to the storage address of the file in the file system, and downloads the file to the local according to the storage address of the downloading log in the file system after scanning the downloading log. Through the process, the downloading process of the service data is simplified, and the downloading efficiency of the service data is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a method, a device and equipment for downloading service data.
Background
Asset secured Securitization (ABS) is a Securitization financing approach that supports the assets to which an item belongs. The ABS service system generates a large amount of service data and stores the service data in an ABS service database.
In the prior art, the big data platform extracts service data from the ABS service database at specific time intervals (for example, extracting service data of the previous day at zero point of each day), and stores the extracted service data in the big data platform. And when the user needs to download the service data, downloading from the big data platform. Specifically, the user submits a request form to the big data platform, and service data required to be downloaded is indicated in the request form. And the administrator of the big data platform checks the application form, saves the service data required to be downloaded by the user into a file, stores the file into a specific file system, and then informs the user to download the file from the file system.
In the process of implementing the present application, the inventor finds that at least the following problems exist in the prior art: the downloading process of the service data is complex and the efficiency is low.
Disclosure of Invention
The embodiment of the application provides a method, a device and equipment for downloading service data, which are used for improving the downloading efficiency of the service data.
In a first aspect, an embodiment of the present application provides a method for downloading service data, including:
receiving a downloading request message from a terminal device, wherein the downloading request message is used for indicating target service data requested to be downloaded;
according to the downloading request message, acquiring the target service data from a service database, writing the target service data into a file, and storing the file into a file system;
and generating a download log corresponding to the download request message according to the storage address of the file in the file system, wherein the download log is used for downloading the file from the storage address after scanning the download log by the terminal equipment.
In a possible implementation manner, the download request message includes at least one query condition input by the user through the terminal device for querying the target service data; obtaining the target service data from a service database according to the download request message, including:
and inquiring the target service data from the service database according to the at least one inquiry condition.
In a possible implementation manner, the target service data includes a plurality of fields, and the download request message further includes a display parameter corresponding to at least one field input by the user through the terminal device; writing the target service data into a file, including:
and processing the corresponding field of the target service data according to the display parameter corresponding to the at least one field, and writing the processed target service data into a file.
In a possible implementation manner, the presentation parameter corresponding to the field indicates at least one of the following: and displaying the field, not displaying the field, desensitizing to display the field, and encrypting to display the field.
In a possible implementation manner, acquiring the target service data from a service database according to the download request message includes:
determining the type of the target service data according to the downloading request message;
if the type of the target service data is a first type, acquiring the target service data from the service database;
and if the type of the target service data is a second type, generating a download task corresponding to the download request message, adding the download task into a task queue, scanning the task queue, and executing the download task to acquire the target service data from the service database when the download task is scanned.
In a possible implementation manner, determining the type of the target service data according to the download request message includes:
if the data volume of the target service data is smaller than the preset data volume, determining that the type of the target service data is the first type;
and if the data volume of the target service data is greater than or equal to the preset data volume, determining that the type of the target service data is the second type.
In one possible implementation, writing the target service data into a file, and storing the file in a file system includes:
writing the target service data into a plurality of files by taking a preset data volume as a unit;
and compressing the plurality of files, and storing the compressed files in a file system.
In a second aspect, the present application provides a method for downloading service data, including:
sending a downloading request message to a server, wherein the downloading request message is used for indicating target service data requested to be downloaded and is used for enabling the server to acquire the target service data from a service database so as to write the target service data into a file and store the file into a file system;
and after scanning a download log corresponding to the download request message, determining a storage address of the file in the file system according to the download log, and downloading the file from the storage address.
In a possible implementation manner, before sending the download request message to the server, the method further includes:
acquiring at least one query condition which is input by a user and used for querying the target service data;
and generating the downloading request message according to the at least one query condition.
In a possible implementation manner, the generating the download request message according to the at least one query condition includes:
obtaining a display parameter corresponding to at least one field input by the user;
and generating the download request message according to the at least one query condition and the display parameter corresponding to the at least one field.
In a possible implementation manner, the presentation parameter corresponding to the field indicates at least one of the following: and displaying the field, not displaying the field, desensitizing to display the field, and encrypting to display the field.
In a possible implementation manner, before sending the download request message to the server, the method further includes:
and determining that the user has the downloading authority of the service data.
In a third aspect, an embodiment of the present application provides a device for downloading service data, including:
a receiving module, configured to receive a download request message from a terminal device, where the download request message is used to indicate target service data requested to be downloaded;
the downloading module is used for acquiring the target service data from a service database according to the downloading request message, writing the target service data into a file and storing the file into a file system;
and the generating module is used for generating a download log corresponding to the download request message according to the storage address of the file in the file system, wherein the download log is used for enabling the terminal equipment to download the file from the storage address after scanning the download log.
In a possible implementation manner, the download request message includes at least one query condition input by the user through the terminal device for querying the target service data; the download module is specifically configured to:
and inquiring the target service data from the service database according to the at least one inquiry condition.
In a possible implementation manner, the target service data includes a plurality of fields, and the download request message further includes a display parameter corresponding to at least one field input by the user through the terminal device; the download module is specifically configured to:
and processing the corresponding field of the target service data according to the display parameter corresponding to the at least one field, and writing the processed target service data into a file.
In a possible implementation manner, the presentation parameter corresponding to the field indicates at least one of the following: and displaying the field, not displaying the field, desensitizing to display the field, and encrypting to display the field.
In a possible implementation manner, the downloading module is specifically configured to:
determining the type of the target service data according to the downloading request message;
if the type of the target service data is a first type, acquiring the target service data from the service database;
and if the type of the target service data is a second type, generating a download task corresponding to the download request message, adding the download task into a task queue, scanning the task queue, and executing the download task to acquire the target service data from the service database when the download task is scanned.
In a possible implementation manner, the downloading module is specifically configured to:
if the data volume of the target service data is smaller than the preset data volume, determining that the type of the target service data is the first type;
and if the data volume of the target service data is greater than or equal to the preset data volume, determining that the type of the target service data is the second type.
In a possible implementation manner, the downloading module is specifically configured to:
writing the target service data into a plurality of files by taking a preset data volume as a unit;
and compressing the plurality of files, and storing the compressed files in a file system.
In a fourth aspect, an embodiment of the present application provides a device for downloading service data, including:
a sending module, configured to send a download request message to a server, where the download request message is used to indicate target service data requested to be downloaded, and is used to enable the server to obtain the target service data from a service database, so as to write the target service data into a file, and store the file in a file system;
and the downloading module is used for determining the storage address of the file in the file system according to the downloading log after scanning the downloading log corresponding to the downloading request message, and downloading the file from the storage address.
In a possible implementation manner, the apparatus further includes a generation module; the generation module is configured to:
acquiring at least one query condition which is input by a user and used for querying the target service data;
and generating the downloading request message according to the at least one query condition.
In a possible implementation manner, the target service data includes a plurality of fields, and the generating module is specifically configured to:
acquiring a display parameter corresponding to at least one field input by a user;
and generating the download request message according to the at least one query condition and the display parameter corresponding to the at least one field.
In a possible implementation manner, the presentation parameter corresponding to the field indicates at least one of the following: and displaying the field, not displaying the field, desensitizing to display the field, and encrypting to display the field.
In a possible implementation manner, the generating module is further configured to:
and determining that the user has the downloading authority of the service data.
In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory for storing a computer program and a processor for executing the computer program to implement the method according to any of the first aspect.
In a sixth aspect, an embodiment of the present application provides an electronic device, including: a memory for storing a computer program and a processor for executing the computer program to implement the method according to any of the second aspect.
In a seventh aspect, this application provides a computer-readable storage medium including a computer program which, when executed by a processor, implements the method according to any one of the first aspect, or implements the method according to any one of the second aspect.
In an eighth aspect, the present application provides a computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of the first aspect, or implements the method according to any one of the second aspect.
The method, the device and the equipment for downloading the service data provided by the embodiment of the application comprise the following steps: the terminal equipment sends a downloading request message to the server, the server acquires target service data from a service database according to the downloading request message, writes the target service data into a file, stores the file into a file system, then generates a downloading log corresponding to the downloading request message according to the storage address of the file in the file system, and downloads the file to the local according to the storage address of the downloading log in the file system after scanning the downloading log. Through the process, the downloading process of the service data is simplified, and the downloading efficiency of the service data is improved. Furthermore, the server directly downloads data from the service database, so that the real-time service data downloaded by the user is ensured.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic architecture diagram of a service data downloading system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a method for downloading service data according to an embodiment of the present application;
fig. 3 is a schematic view of a downloading process of ABS service data according to an embodiment of the present application;
fig. 4 is a flowchart illustrating a method for downloading service data according to another embodiment of the present application;
fig. 5 is a schematic structural diagram of a device for downloading service data according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a device for downloading service data according to another embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
First, terms related to embodiments of the present application will be explained:
ABS: the method is a project financing method for collecting funds by issuing bonds in the capital market on the basis of assets owned by a project and expected income brought by the project assets.
Dividing the table: when the data volume of one data table is large, the data table is split into a plurality of data tables, and each data table obtained through splitting is called a sub-table.
ABS service systems generate a large amount of service data. And storing the service data generated by the ABS service system into an ABS service database in real time.
In the prior art, the big data platform extracts service data from the ABS service database at specific time intervals (for example, extracting service data of the previous day at zero point of each day), and stores the extracted service data in the big data platform. And when the user needs to download the service data, downloading from the big data platform. Specifically, the user submits a request form to the big data platform, and service data required to be downloaded is indicated in the request form. And the administrator of the big data platform checks the application form, saves the service data required to be downloaded by the user into a file, stores the file into a specific file system, and then informs the user to download the file from the file system.
In the process of implementing the present application, the inventor finds that at least the following problems exist in the prior art:
(1) because the big data platform extracts the service data from the ABS service database according to a specific time interval, the service data stored in the big data platform is not updated in real time, and thus, a user cannot download the real-time service data. For example, it is assumed that the big data platform extracts the service data of the previous day at the zero point of each day, so that only the service data of the previous day and the previous day is stored in the big data platform, and the service data of the current day is not stored in the big data platform, and therefore, when the user downloads the service data from the big data platform, the service data of the current day cannot be downloaded.
(2) The downloading process of the service data needs the user to manually submit the application form, and needs the administrator to manually check and download the data, so that the downloading process of the service data is complex, the time consumption is long, and the efficiency is low.
(3) In the process, the service data which needs to be downloaded by the user is saved into the file by the administrator, so that the files generated by the service data downloaded by a plurality of users all adopt the same display mode, and the personalized display requirements of different users cannot be met.
In order to solve at least one of the above technical problems, the present application provides a system for downloading service data, and provides a method for downloading service data based on the system. The following first describes a system for downloading service data provided by the present application with reference to fig. 1.
Fig. 1 is a schematic structural diagram of a service data downloading system according to an embodiment of the present application. As shown in fig. 1, the service data downloading system of this embodiment includes: terminal equipment, a server and a service database. And the terminal equipment is in communication connection with the server. The server may access the service database and obtain service data from the service database.
The service database is in communication connection with the service system, and the service data generated by the service system is stored in the service database in real time. Optionally, the service database may be an ABS service database.
The terminal device may be an electronic device with information processing capability, such as a smart phone, a tablet computer, or a desktop computer. The terminal equipment can access the server in a webpage or client-side mode, and sends a downloading request message to the server to request the server to download the target service data from the service database.
The server provides the downloading service of the service data to the terminal equipment. The server can be a common server or a cloud server. The server may be a clustered server or a non-clustered server. The server may be a distributed server.
In this embodiment, when a user needs to download service data, the terminal device sends a download request message to the server, and after receiving the download request message, the server obtains target service data from the service database according to the download request message, writes the target service data into a file, and stores the file in the file system. And the server generates a download log corresponding to the download request message according to the storage address of the file in the file system. Therefore, after scanning the download log, the terminal device obtains the storage address of the file in the file system according to the download log, and downloads the file from the storage address. Through the process, the downloading process of the service data is simplified, and the downloading efficiency of the service data is improved. And the server directly downloads data from the service database, so that the real-time service data downloaded by the user is ensured.
It should be noted that the service data in this application may be service data of an ABS service system, and may also be service data of other service systems, which is not limited in this embodiment. For convenience of description, the following examples only take the service data of the ABS service system as an example for illustration.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 2 is a schematic flowchart of a method for downloading service data according to an embodiment of the present application. As shown in fig. 2, the method of the present embodiment includes:
s201: the terminal device sends a download request message to the server.
Accordingly, the server receives a download request message from the terminal device.
The downloading request message is used for indicating the target service data requested to be downloaded.
Specifically, when the user needs to download the service data, the user inputs a download instruction to the terminal device according to the download requirement of the user. For example, the user may specify which time range of service data to download, and/or which geographic range of service data to download, and/or which type of service data to download.
With the ABS service data type, a user can specify to download the asset detail type service data generated in a certain time period, and can also specify to download the asset summary type service data generated in a certain time period.
After receiving a downloading instruction input by a user, the terminal equipment sends a downloading request message to the server according to the downloading instruction, so that the server can download target service data required by the user according to the downloading request message.
Optionally, before the terminal device receives a download instruction input by the user, it may be further determined that the user has the download authority of the service data.
For example, taking ABS service scenario as an example, ABS service data generally relates to multiple service roles, and the ABS service data has a certain sensitivity, and the ABS service data is not visible to all service roles. That is, some service roles have the downloading authority of the ABS service data, and some service roles do not have the downloading authority of the ABS service data.
In this embodiment, the terminal device may first determine that the user has the downloading right of the service data. And if the user is determined not to have the downloading authority of the service data, the user is not allowed to input the downloading instruction. For example, in the case that it is determined that the user has the downloading authority of the service data, the downloading control is displayed in the display interface of the terminal device. And under the condition that the user is determined not to have the downloading authority of the service data, not displaying the downloading control in the display interface of the terminal equipment. Thereby realizing the download authority control of the service data.
The downloading authority of the user is controlled through the terminal equipment, so that the manual checking process in the prior art is avoided, and the downloading process of the service data is simplified.
S202: and the server acquires the target service data from the service database according to the downloading request message, writes the target service data into a file, and stores the file into a file system.
After receiving the download request message, the server inquires and acquires target service data through interaction with the service database, stores the target service data into a file, and stores the file into a file system.
The file system may be an entity file system or a virtual file system. The file system may be provided in a server or in another device. This embodiment is not limited to this.
In a possible implementation manner, the server may determine the type of the target service data according to the download request message, and acquire the target service data from the service database in a synchronous or asynchronous manner according to the type of the target service data.
The type of the target service data may be divided in various ways.
Optionally, the server may determine the type of the target service data according to the data size of the target service data. For example, if the data amount of the target service data is less than the preset data amount, the type of the target service data is determined to be the first type, and if the data amount of the target service data is greater than or equal to the preset data amount, the type of the target service data is determined to be the second type. That is, the data amount of the second type of traffic data is greater than the data amount of the first type of traffic data.
In this embodiment, if the type of the target service data is the first type, the server may acquire the target service data from the service database in a synchronous downloading manner. The server adopts a synchronous downloading mode, namely, the server directly executes the action of acquiring the target service data from the service database after receiving the downloading request message, and executes subsequent other operations after the action of acquiring the target service data is finished. The synchronous downloading mode is suitable for scenes with small data volume of the target service data.
If the type of the target service data is the second type, the server may acquire the target service data from the service database in an asynchronous downloading manner. The server adopts an asynchronous downloading mode, namely, after receiving the downloading request message, the server firstly generates a downloading task corresponding to the downloading request message and adds the downloading task into a task queue. And the server scans each task in the task queue, and executes the downloading task to acquire target business data from the business database after scanning the downloading task. The asynchronous downloading mode is suitable for scenes with large data volume of target service data.
Optionally, the servers in this embodiment may include a server that generates a task and a server that executes the task. And after receiving the downloading request message, the task generating server generates a corresponding downloading task and adds the downloading task into the task queue. And the server executing the task scans the task queue, and executes the downloading task after scanning the downloading task. The number of the servers for executing the tasks can be multiple, so that the multiple downloading tasks can be executed in parallel, and the downloading efficiency of the service data is improved.
In the following, with reference to fig. 3, the procedure of downloading service data is described by taking an ABS service scenario as an example.
Fig. 3 is a schematic view of a downloading process of ABS service data according to an embodiment of the present application. The flow shown in fig. 3 is performed by the server. In the ABS service scene, a user can select to download the asset detail type service data and also select to download the asset summary type service data. It can be understood that the data volume of the asset detail type of business data is much larger than that of the asset summary type of business data. Therefore, as shown in fig. 3, after receiving the download request message of the ABS service data, the server determines whether the type of the ABS service data to be downloaded is the asset summary type or the asset detail type according to the download request message. And if the asset summarizing type is adopted, acquiring target service data from the ABS service database in a synchronous downloading mode, and if the asset detail type is adopted, acquiring the target service data from the service database in an asynchronous downloading mode. And the server writes the acquired target service data into a file and stores the file into a file system.
In the embodiment, the load is carried out in a synchronous or asynchronous mode according to the type of the target service data, so that the load of the large-batch data load on the service database can be avoided.
Optionally, in a case that the data volume of the target service data is large, the following feasible method may be further adopted in the embodiment to write the target service data into the file: and writing the target service data into a plurality of files by taking a preset data volume as a unit, compressing the plurality of files, and storing the compressed files into a file system. For example, every 50 ten thousand pieces of business data may be saved as one file.
By storing the target service data into a plurality of files, the data volume in each file is prevented from being overlarge, so that the service data in the files can be conveniently displayed and subsequently processed.
S203: and the server generates a download log corresponding to the download request message according to the storage address of the file in the file system.
S204: and after scanning the download log corresponding to the download request message, the terminal equipment determines the storage address of the file in the file system according to the download log and downloads the file from the storage address.
In this embodiment, after the server stores the file in the file system, a download log corresponding to the download request message is generated. In one example, the name of the download log may be an identifier of the download request message, and the content of the download log is a storage address of the file in the file system.
In this embodiment, the storage location of the download log is not limited, and the storage location may be a location agreed in advance with the terminal device. For example, the server may store the download log in a specific storage space of the server, or in a storage space of a file system, or in a specific location of a database.
And after the terminal equipment sends a download request message to the server, starting a timing scanning task, scanning the storage position at preset time intervals, and downloading the file to the local at a storage address in the file system after scanning a download log corresponding to the download request message.
In the method for downloading service data provided by this embodiment, the terminal device sends a download request message to the server, the server obtains target service data from the service database according to the download request message, writes the target service data into a file, and stores the file in the file system, then the server generates a download log corresponding to the download request message according to a storage address of the file in the file system, and after scanning the download log, the terminal device downloads the file to the local according to the storage address of the download log in the file system. Through the process, the downloading process of the service data is simplified, and the downloading efficiency of the service data is improved. And the server directly downloads data from the service database, so that the real-time service data downloaded by the user is ensured.
On the basis of any of the above embodiments, the following describes an asynchronous downloading flow of service data with reference to an example.
In this example, it is assumed that the target service data relates to n sub-tables, that is, the server needs to query the n sub-tables to obtain the target service data, and the server writes one file for every 50 ten thousand pieces of service data queried.
After scanning the download task, the server firstly initializes the task. Specifically, the task initialization may include: the number of queried pieces (i.e., the maximum ID of queried data) corresponding to each sub-table is set to 0.
Starting to execute the downloading task, namely starting an inquiry thread to inquire each sub-table, wherein the method specifically comprises the following steps:
and for the 1 st sub-table, determining the number of the remaining service data according to the number of the inquired pieces of the sub-table, if the number of the remaining service data is more than 50 ten thousand, acquiring the subsequent 50 ten thousand pieces of service data in the sub-table from the maximum ID of the inquired data of the sub-table, writing the acquired data into a file, and updating the number of the inquired pieces of the sub-table. The above process is repeatedly executed until the number of the remaining service data of the sub-table is less than 50 ten thousand.
Then start to look up the 2 nd sub-table. It should be understood that the query process of the 2 nd sub-table and the subsequent sub-tables is similar to the query process of the 1 st sub-table, and is not described herein.
According to the process, after the n sub-tables are inquired, the number of the remaining service data in the n sub-tables is less than 50 ten thousand. Then, the remaining service data in each sub-table is sequentially inquired, and 50 ten thousand service data are written into one file every time the inquiry is made. Until the query of the n sub-tables is completed. The number of pieces of service data in the last file is less than or equal to 50 ten thousand.
It should be noted that the above query process for multiple sub-tables is only an example, and in practical applications, other query modes may also be adopted, which is not limited in this embodiment.
On the basis of any of the above embodiments, the following describes the downloading method of the service data in more detail with reference to the embodiment shown in fig. 4. The method of the present embodiment is performed by a server.
Fig. 4 is a flowchart illustrating a method for downloading service data according to another embodiment of the present application. As shown in fig. 4, the method of the present embodiment includes:
s401: receiving a download request message from a terminal device, the download request message including: the terminal equipment is used for inputting at least one query condition for querying the target service data and the display parameter corresponding to at least one field in the target service data.
S402: and inquiring the target service data from a service database according to the at least one inquiry condition.
S403: and processing the corresponding field of the target service data according to the display parameter corresponding to the at least one field, and writing the processed target service data into a file.
It should be noted that, in an actual implementation, the download request message may also include only the at least one query condition, but does not include the presentation parameter corresponding to the at least one field. In this case, in S403, after the server queries the service database to obtain the target service data, the server may write the target service data into the file directly, and may not process the target service data.
S404: and generating a download log corresponding to the download request message according to the storage address of the file in the file system.
Similar to the above embodiment, the download log is used to enable the terminal device to download the file from the storage address after scanning the download log.
On the basis of the above embodiment, when the user inputs the download instruction in the terminal device, the query condition may be set, where the query condition is a condition for querying the target service data.
In other words, the terminal device obtains at least one query condition input by a user for querying the target service data; and generating a downloading request message according to the at least one query condition. The terminal equipment sends the downloading request message to the server. After receiving the download request message, the server can acquire the target service data by executing the query condition.
Optionally, when the user inputs the downloading instruction in the terminal device, the display parameter corresponding to the field may also be set to specify a display mode of the corresponding field in the target service data.
In other words, the terminal device obtains at least one query condition for querying the target service data, which is input by the user, and the display parameter corresponding to at least one field, which is input by the user; and generating a downloading request message according to the at least one query condition and the display parameter corresponding to the at least one field. The terminal equipment sends the downloading request message to the server.
Optionally, the presentation parameter corresponding to the field indicates at least one of the following: and displaying the field, not displaying the field, desensitizing to display the field, and encrypting to display the field.
When the display parameter corresponding to a certain field indicates that the field is not displayed, the server sets the field in the target service data to be not displayed, or the server does not write the field into the file. When the display parameter corresponding to a certain field indicates encryption display or desensitization display, the server performs encryption processing or desensitization processing on the field in the target service data, and writes the processed field into a file.
In this embodiment, the display parameters corresponding to the fields are included in the download request message, so that the user can customize the display mode of each field of the target service data according to the actual requirement, and this embodiment is applicable to various different application scenarios.
Fig. 5 is a schematic structural diagram of a device for downloading service data according to an embodiment of the present application. The apparatus of this embodiment may be in the form of software and/or hardware, and the apparatus of this embodiment may be disposed in a server. As shown in fig. 5, the apparatus 10 for downloading service data provided in this embodiment includes: a receiving module 11, a downloading module 12 and a generating module 13.
The receiving module 11 is configured to receive a download request message from a terminal device, where the download request message is used to indicate target service data requested to be downloaded;
a downloading module 12, configured to obtain the target service data from a service database according to the downloading request message, write the target service data into a file, and store the file in a file system;
a generating module 13, configured to generate a download log corresponding to the download request message according to a storage address of the file in the file system, where the download log is used to enable the terminal device to download the file from the storage address after scanning the download log.
In a possible implementation manner, the download request message includes at least one query condition input by the user through the terminal device for querying the target service data; the download module 12 is specifically configured to:
and inquiring the target service data from the service database according to the at least one inquiry condition.
In a possible implementation manner, the target service data includes a plurality of fields, and the download request message further includes a display parameter corresponding to at least one field input by the user through the terminal device; the download module 12 is specifically configured to:
and processing the corresponding field of the target service data according to the display parameter corresponding to the at least one field, and writing the processed target service data into a file.
In a possible implementation manner, the presentation parameter corresponding to the field indicates at least one of the following: and displaying the field, not displaying the field, desensitizing to display the field, and encrypting to display the field.
In a possible implementation manner, the downloading module 12 is specifically configured to:
determining the type of the target service data according to the downloading request message;
if the type of the target service data is a first type, acquiring the target service data from the service database;
and if the type of the target service data is a second type, generating a download task corresponding to the download request message, adding the download task into a task queue, scanning the task queue, and executing the download task to acquire the target service data from the service database when the download task is scanned.
In a possible implementation manner, the downloading module 12 is specifically configured to:
if the data volume of the target service data is smaller than the preset data volume, determining that the type of the target service data is the first type;
and if the data volume of the target service data is greater than or equal to the preset data volume, determining that the type of the target service data is the second type.
In a possible implementation manner, the downloading module 12 is specifically configured to:
writing the target service data into a plurality of files by taking a preset data volume as a unit;
and compressing the plurality of files, and storing the compressed files in a file system.
The device for downloading service data provided in this embodiment may be used to implement the method for downloading service data executed by the server side in any of the above method embodiments, and the implementation principle and technical effect are similar, which are not described herein again.
Fig. 6 is a schematic structural diagram of a device for downloading service data according to another embodiment of the present application. The apparatus of this embodiment may be in the form of software and/or hardware, and the apparatus of this embodiment may be disposed in a terminal device. As shown in fig. 6, the apparatus 20 for downloading service data provided in this embodiment may include: a sending module 21 and a downloading module 22.
The sending module 21 is configured to send a download request message to a server, where the download request message is used to indicate target service data requested to be downloaded, and is used to enable the server to obtain the target service data from a service database, so as to write the target service data into a file, and store the file in a file system;
and the downloading module 22 is configured to, after scanning the downloading log corresponding to the downloading request message, determine a storage address of the file in the file system according to the downloading log, and download the file from the storage address.
In a possible implementation manner, as shown in fig. 6, the apparatus of this embodiment may further include a generating module 23. The generating module 23 is configured to:
acquiring at least one query condition which is input by a user and used for querying the target service data;
and generating the downloading request message according to the at least one query condition.
In a possible implementation manner, the target service data includes a plurality of fields, and the generating module 23 is specifically configured to:
acquiring a display parameter corresponding to at least one field input by a user;
and generating the download request message according to the at least one query condition and the display parameter corresponding to the at least one field.
In a possible implementation manner, the presentation parameter corresponding to the field indicates at least one of the following: and displaying the field, not displaying the field, desensitizing to display the field, and encrypting to display the field.
In a possible implementation manner, the generating module 23 is further configured to:
and determining that the user has the downloading authority of the service data.
The apparatus for downloading service data provided in this embodiment may be used to implement the method for downloading service data executed by the terminal device in any of the above method embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device can be used as a server, and the electronic device can also be a terminal device. As shown in fig. 7, the electronic device 30 of the present embodiment includes: a processor 31 and a memory 32; a memory 32 for storing a computer program; the processor 31 is configured to execute a computer program stored in the memory to implement the method for downloading service data executed by the server in the foregoing embodiment, or implement the method for downloading service data executed by the terminal device in the foregoing embodiment. Specifically, reference may be made to the related descriptions in the foregoing method embodiments, which have similar implementation principles and technical effects, and this embodiment is not described herein again.
Alternatively, the memory 32 may be separate or integrated with the processor 31.
When the memory 32 is a device independent of the processor 31, the electronic device 30 may further include: a bus 33 for connecting the memory 32 and the processor 31.
Optionally, when the electronic device 30 is a server, the electronic device 30 may further include a communication component 34 for communicating with a terminal device.
Optionally, when the electronic device 30 is a terminal device, the electronic device 30 may further include a communication component 34 for communicating with a server.
An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium includes a computer program, and the computer program is configured to implement the method for downloading service data executed by a server in any method embodiment, or implement the method for downloading service data executed by a terminal device in any method embodiment, where implementation principles and technical effects are similar, and are not described herein again.
An embodiment of the present application further provides a chip, including: the present invention relates to a method for downloading service data, and more particularly, to a method for downloading service data, which is executed by a server in any one of the above method embodiments, or a method for downloading service data, which is executed by a terminal device in any one of the above method embodiments, and which is implemented by a memory, a processor, and a computer program, where the computer program is stored in the memory, and the processor runs the computer program to execute the method for downloading service data executed by the server in any one of the above method embodiments.
An embodiment of the present application further provides a computer program product, including a computer program, where the computer program is executed by a processor to implement the method for downloading service data executed by a server in any of the above method embodiments, or to implement the method for downloading service data executed by a terminal device in any of the above method embodiments, and implementation principles and technical effects thereof are similar, and details are not described here.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware form, and can also be realized in a form of hardware and a software functional unit.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application.
It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in the incorporated application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.
The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.
The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
Claims (16)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011539109.4A CN112579321A (en) | 2020-12-23 | 2020-12-23 | Method, device and equipment for downloading service data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011539109.4A CN112579321A (en) | 2020-12-23 | 2020-12-23 | Method, device and equipment for downloading service data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112579321A true CN112579321A (en) | 2021-03-30 |
Family
ID=75139103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011539109.4A Pending CN112579321A (en) | 2020-12-23 | 2020-12-23 | Method, device and equipment for downloading service data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112579321A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114218156A (en) * | 2021-12-07 | 2022-03-22 | 深圳市科脉技术股份有限公司 | Data export method, apparatus, computer equipment and storage medium |
| CN115189956A (en) * | 2022-07-18 | 2022-10-14 | 中原银行股份有限公司 | File security sharing method |
| CN115203306A (en) * | 2022-07-19 | 2022-10-18 | 平安付科技服务有限公司 | Data exporting method and device, computer equipment and readable storage medium |
| CN115442360A (en) * | 2022-08-24 | 2022-12-06 | 微梦创科网络科技（中国）有限公司 | Log file downloading method and device and electronic equipment |
| CN115834575A (en) * | 2023-02-28 | 2023-03-21 | 新云网科技集团股份有限公司 | Log reporting method, system, equipment and computer readable storage medium |
| CN115941794A (en) * | 2022-11-16 | 2023-04-07 | 北京奇艺世纪科技有限公司 | A file storage method, system, device and electronic equipment |
| CN116319757A (en) * | 2023-04-04 | 2023-06-23 | 平安国际融资租赁有限公司 | File downloading method and device, storage medium and electronic equipment |
| CN117056288A (en) * | 2023-08-17 | 2023-11-14 | 齐鲁空天信息研究院 | Method and system for searching and downloading server file |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100121894A1 (en) * | 2008-07-11 | 2010-05-13 | Sony Corporation | Information processing apparatus, information processing system, information recording medium, information processing method, and program |
| CN101727504A (en) * | 2010-01-29 | 2010-06-09 | 成都市华为赛门铁克科技有限公司 | Method and device for migrating data of file system |
| CN102255866A (en) * | 2010-05-17 | 2011-11-23 | 腾讯科技（深圳）有限公司 | Method and device for downloading data |
| CN102752326A (en) * | 2011-04-19 | 2012-10-24 | 腾讯科技（深圳）有限公司 | Method, client, server and system for processing data in file downloading |
| US20140359080A1 (en) * | 2013-05-30 | 2014-12-04 | Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. | File download method, system, and computing device |
| CN106372153A (en) * | 2016-08-30 | 2017-02-01 | 陕西千山航空电子有限责任公司 | Recording equipment fault diagnosis method based on log file system |
| CN106874781A (en) * | 2015-12-11 | 2017-06-20 | 阿里巴巴集团控股有限公司 | A kind of tables of data encryption method and server |
| CN109981745A (en) * | 2019-02-28 | 2019-07-05 | 深圳市买买提信息科技有限公司 | A kind of journal file processing method and server |
| WO2019178979A1 (en) * | 2018-03-21 | 2019-09-26 | 平安科技（深圳）有限公司 | Method for querying report data, apparatus, storage medium and server |
| CN110727949A (en) * | 2019-09-06 | 2020-01-24 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Data storage method and device, computer equipment and storage medium |
| CN111125731A (en) * | 2019-12-11 | 2020-05-08 | 支付宝(杭州)信息技术有限公司 | A business processing method, device and device based on verifiable statement |
| CN111343269A (en) * | 2020-02-25 | 2020-06-26 | 江苏苏宁物流有限公司 | Data downloading method and device, computer equipment and storage medium |
-
2020
- 2020-12-23 CN CN202011539109.4A patent/CN112579321A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100121894A1 (en) * | 2008-07-11 | 2010-05-13 | Sony Corporation | Information processing apparatus, information processing system, information recording medium, information processing method, and program |
| CN101727504A (en) * | 2010-01-29 | 2010-06-09 | 成都市华为赛门铁克科技有限公司 | Method and device for migrating data of file system |
| CN102255866A (en) * | 2010-05-17 | 2011-11-23 | 腾讯科技（深圳）有限公司 | Method and device for downloading data |
| CN102752326A (en) * | 2011-04-19 | 2012-10-24 | 腾讯科技（深圳）有限公司 | Method, client, server and system for processing data in file downloading |
| US20140359080A1 (en) * | 2013-05-30 | 2014-12-04 | Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. | File download method, system, and computing device |
| CN106874781A (en) * | 2015-12-11 | 2017-06-20 | 阿里巴巴集团控股有限公司 | A kind of tables of data encryption method and server |
| CN106372153A (en) * | 2016-08-30 | 2017-02-01 | 陕西千山航空电子有限责任公司 | Recording equipment fault diagnosis method based on log file system |
| WO2019178979A1 (en) * | 2018-03-21 | 2019-09-26 | 平安科技（深圳）有限公司 | Method for querying report data, apparatus, storage medium and server |
| CN109981745A (en) * | 2019-02-28 | 2019-07-05 | 深圳市买买提信息科技有限公司 | A kind of journal file processing method and server |
| CN110727949A (en) * | 2019-09-06 | 2020-01-24 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Data storage method and device, computer equipment and storage medium |
| CN111125731A (en) * | 2019-12-11 | 2020-05-08 | 支付宝(杭州)信息技术有限公司 | A business processing method, device and device based on verifiable statement |
| CN111343269A (en) * | 2020-02-25 | 2020-06-26 | 江苏苏宁物流有限公司 | Data downloading method and device, computer equipment and storage medium |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114218156A (en) * | 2021-12-07 | 2022-03-22 | 深圳市科脉技术股份有限公司 | Data export method, apparatus, computer equipment and storage medium |
| CN115189956A (en) * | 2022-07-18 | 2022-10-14 | 中原银行股份有限公司 | File security sharing method |
| CN115203306A (en) * | 2022-07-19 | 2022-10-18 | 平安付科技服务有限公司 | Data exporting method and device, computer equipment and readable storage medium |
| CN115442360A (en) * | 2022-08-24 | 2022-12-06 | 微梦创科网络科技（中国）有限公司 | Log file downloading method and device and electronic equipment |
| CN115941794A (en) * | 2022-11-16 | 2023-04-07 | 北京奇艺世纪科技有限公司 | A file storage method, system, device and electronic equipment |
| CN115834575A (en) * | 2023-02-28 | 2023-03-21 | 新云网科技集团股份有限公司 | Log reporting method, system, equipment and computer readable storage medium |
| CN116319757A (en) * | 2023-04-04 | 2023-06-23 | 平安国际融资租赁有限公司 | File downloading method and device, storage medium and electronic equipment |
| CN117056288A (en) * | 2023-08-17 | 2023-11-14 | 齐鲁空天信息研究院 | Method and system for searching and downloading server file |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112579321A (en) | Method, device and equipment for downloading service data | |
| CN113364853B (en) | Business service system, business request method and gateway equipment | |
| CN108734028B (en) | Blockchain-based data management method, blockchain node and storage medium | |
| CN110245069B (en) | Page version testing method and device and page display method and device | |
| CN110557284B (en) | Data aggregation method and device based on client gateway | |
| CN108959430B (en) | Advertisement promotion data acquisition method, device and equipment | |
| CN107862352B (en) | Two-dimensional code sharing method and two-dimensional code sharing equipment | |
| WO2016074370A1 (en) | Keyvalue database data table updating method and data table updating device | |
| JP2017507376A (en) | Method, apparatus and system for communicating and presenting merchandise information | |
| CN113626512A (en) | Data processing method, device, equipment and readable storage medium | |
| CN109544207B (en) | Information processing method, storage medium and server | |
| CN112784195A (en) | Page data publishing method and system | |
| CN110286981A (en) | The display methods and display system of the use state of virtual cloud desktop server | |
| CN110598072B (en) | Feature data aggregation method and device | |
| CN113467823A (en) | Configuration information acquisition method, device, system and storage medium | |
| CN112988311A (en) | On-line experimental method based on WEB server | |
| CN112733510A (en) | Financial certificate generation method, device, equipment and computer readable storage medium | |
| CN111814029A (en) | Data query method, system and computing device | |
| CN113032820A (en) | File storage method, access method, device, equipment and storage medium | |
| CN114579127B (en) | Page deployment method and device, computer equipment and storage medium | |
| CN112732547A (en) | Service testing method and device, storage medium and electronic equipment | |
| CN113630612B (en) | Data processing and data displaying method and device, electronic equipment and medium | |
| CN115858905A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN114298699A (en) | Generation method, acquisition method and device of non-fungible token | |
| CN112464099A (en) | Method, device, electronic equipment and medium for generating document based on query data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Beijing Economic and Technological Development Zone, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210330 |
|
| RJ01 | Rejection of invention patent application after publication |