Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
For any field, the accuracy and timeliness of data acquisition and data analysis, and the consumption of manpower, time and equipment are closely related to the data storage mode. Generally, data generated by system operation is directly stored in a server of the system as online data, for example, the online data is stored in a central relational database or NoSQL, and data can be obtained by directly accessing the database. In order to avoid the influence on the online service, the data generated by the online system may be sliced in the dimensions of daily level, monthly level, etc., and the sliced data may be stored offline, however, the data stored offline has a delay of at least 1 day compared with the data generated by the online system, and the latest and complete data cannot be provided to the data demanding party.
In order to solve the above problem, in the embodiments of the present application, data generated by an online system is stored in real time through a blockchain network, so that a time delay between the stored data and the online data is avoided, so that the blockchain network can provide the completed data to a data requesting party, and the stored data is read through a plurality of blockchain nodes distributed in the blockchain network, and pressure of reading the data is dispersed on each blockchain node, thereby avoiding an influence on an online service.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application. As shown in fig. 1, the blockchain network 100 is composed of a plurality of blockchain nodes 110, and the blockchain nodes 110 may be any terminal devices, such as personal computers PC, notebook computers, mobile phones, mining machines, servers, and so on.
It should be understood that an account of a user is logged in the blockchain node 110, and is used to complete a corresponding operation according to an instruction of the user, in this embodiment of the present application, the account logged in the blockchain node 110 may be an account corresponding to a data producer, or an account corresponding to a data consumer, optionally, the same account may have a data reading permission or a data writing permission, or both the data reading permission and the data writing permission, for example, the data producer may also be the data consumer, and this application is not limited thereto.
Exemplary operations that blockchain node 110 may perform include, but are not limited to, invoking smart contracts, recording transactions in the blockchain's blocks, determining the sequence of transactions and whether the transactions were successful based on a consensus algorithm.
Optionally, the intelligent contract invoked in the embodiment of the present application may be implemented by any programming language, such as a contractual programming language identity, a static strongly typed compiled language Go, an object-oriented programming language java, and the like. The blockchain in the embodiments of the present application may be any type or blockchain applied to any platform, such as a public chain, a federation chain, or a private chain.
In order to realize real-time storage of data generated by a data producer, the embodiment of the application proposes an implementation manner as shown in fig. 2.
Fig. 2 is a flowchart illustrating a data processing method 200 according to an embodiment of the present disclosure. As shown in fig. 2, the method includes:
s201: and receiving a data writing request sent by a data producer.
Illustratively, the data producing direction block link point sends a data write request for requesting the block link point to write the first data into the block chain. It should be understood that the data producer continuously sends a data write request by using data generated in real time as first data, and requests the blockchain to perform write processing on the first data, where the first data may be one or more data to be stored, or may be a data packet containing a plurality of data.
It should be understood that the first data should be included in the data write request.
S202: write data is determined based on the first data and the data write contract.
The data write contract is an intelligent contract preset in the block chain, and can determine whether to write the first data into the block chain, and the writing position and writing mode of the first data. Illustratively, the writing method includes: writing the first data line by line; or filling the first data into the corresponding table according to the variable of the first data, the value of the variable, and the preset table row and the table header.
Illustratively, the first data includes a variable and a value, the variable being a definition of the data, such as a user's age, a user's gender, a user's name, etc., and the value being used to represent a value of the variable, such as 30 years, female, lie, etc.
Further, the data write contract may also determine whether the first data has at least one second data associated therewith based on the first data.
In this step, it may be determined, by calling a data write contract, to write the first data, or to write at least one second data related to the first data, or to write the first data and the at least one second data, based on the first data.
S203: write data is written to the block chain.
Illustratively, the block chain link point generates a write transaction according to the determined write data, the write position of the data and the write mode, and synchronizes the write transaction in the block chain network to realize writing the write data into the block chain.
For example, the first data includes a variable and a value, and if the variable of the first data is a stored variable, the process of writing the first data may be understood as a process of updating the value of the variable.
According to the embodiment of the application, the data writing request sent by the data producer is received, the written data is determined according to the first data and the data writing contract requested to be written by the data writing request, and then the written data is written into the block chain. According to the embodiment of the application, when the data producer continuously sends the first data acquired in real time, the written data is written into the block chain in real time, and the data is stored in real time by utilizing a synchronization mechanism between block chain nodes in the block chain network, so that the problem of time delay between offline data and online data is avoided, and complete data can be provided for the data demander to use.
For example, in the process of writing the write data into the blockchain at the blockchain link point, after the blockchain network agrees with the write transaction according to the agreement algorithm, it is determined that the write transaction is completed, that is, the step of writing the write data into the blockchain is completed. For example, if two or more write transactions are received simultaneously by the blockchain network, the sequence of the write transactions is identified according to an identification algorithm, and the write data is written into the blockchain according to the sequence determined by the identification.
Fig. 3 is a flowchart illustrating a data processing method 300 according to an embodiment of the present disclosure. As shown in fig. 3, after receiving a write data request sent by a corresponding data producer through an Application Programming Interface (Application Programming Interface), a plurality of block chain nodes generate corresponding write transactions according to an intelligent contract, for example, a data write contract, and perform consensus on whether each write transaction is successful and the sequence of each write transaction according to a consensus algorithm, and then complete the process of writing the write data into the block chain.
Fig. 4 is a flowchart illustrating a data processing method 400 according to an embodiment of the present application. On the basis of any of the above embodiments, as shown in fig. 4, the embodiments of the present application provide the following possible implementations for how to determine write data according to the first data and the data write contract:
s401: by the data write contract, it is determined whether the first data is used to determine the at least one second data.
In this embodiment, in order to facilitate data reading and improve data reading efficiency, data processing may be performed on data that needs to be stored in advance, and the processed data may also be stored.
In order to achieve the above-described effect, in this step, it is first necessary to determine whether the first data is root data for determining the at least one second data through a data write contract.
Referring to fig. 5, E and F are both first data, that is, root data, and data B and data C are obtained after data processing is performed by an X contract, and data a is obtained according to data B, data C, and root data D by a Y contract. Wherein, data B, data C or data a may be referred to as second data. Optionally, the X contract or the Y contract implements different data processing according to the actual application scenario, such as data screening, data merging, data mapping, and the like. Intelligent contracts containing variable logic relations, such as an X contract and a Y contract, are maintained in a block chain, and when the logic relations are changed, any block chain link point can sense the change of the logic relations, so that the problem of variable semantic ambiguity caused by processing data by a data production party and storing the processed data in the block chain is avoided.
After step S401, if the first data is used to determine at least one second data, steps S402 to S404 are performed.
S402: it is determined that the write data includes the first data and at least one second data.
If the first data is used as root data to determine at least one second data, the first data and the at least one second data need to be written into the block chain for storage, so that a data demand side can acquire the second data quickly.
Illustratively, the first data includes a variable and a value, and if the variable of the first data is a stored variable, and the variable of the second data is also a stored variable, the process of writing the first data may be understood as a process of updating a value corresponding to the variable of the first data, and the process of writing the second data may be understood as a process of updating data corresponding to the variable of the second data.
S403: at least one second datum is determined from the first datum by the data processing contract.
After determining that the write data should include the first data and the at least one second data, the present embodiment determines the at least one second data from the first data by invoking a data processing contract. Optionally, the block link point calls a data processing contract to determine the at least one second datum by a data write contract.
The data processing contract is an intelligent contract preset in the block chain, and at least one second data can be obtained after the first data are processed. The data processing contracts may include one or more intelligent contracts, such as X contracts and Y contracts in the embodiments described above.
S404: writing the first data and the at least one second data to the block chain by a data write contract; alternatively, the first data is written to the block chain via a data write contract and the at least one second data is written to the block chain via a data processing contract.
After determining the at least one second data, receiving the at least one second data returned by the data processing contract through the data writing contract, and writing both the received at least one second data and the first data into the block chain; or, the data write contract invokes the data processing contract to determine at least one second data, that is, to write the first data into the block chain, and the data processing contract determines at least one second data, that is, to write the at least one second data into the block chain.
After step S401, if the first data is not used to determine any one of the second data, steps S402-1 and S404-1 are performed.
S402-1: it is determined that the write data includes the first data.
If the first data is not used to determine any data, then the first data need only be stored as write data in the blockchain.
S404-1: the first data is written to the block chain by a data write contract.
In the embodiment of the application, whether the first data is used for determining the at least one second data is determined according to the data write contract, and when the first data is root data used for determining the at least one second data, the at least one second data is also stored in the block chain, so that a data demand side can directly read the at least one second data without performing data processing on the first data, and the data reading efficiency is improved.
In order to ensure the safety of data storage and ensure the authenticity and validity of data, the embodiment of the application needs to confirm whether a data producer has write-in authority. As a specific implementation manner, before writing the write data into the block chain, the embodiment of the present application further includes: and determining whether the data producer has the write-in authority of the first data or not according to the authority information of the data producer carried by the data write-in request through an authentication contract, writing the write-in data into the block chain when the data producer has the write-in authority of the first data, and stopping the process of writing into the block chain when the data producer does not have the write-in authority of the first data. The authority information includes account information of the data producer or a certificate of the data producer, and for example, the certificate of the data producer is a certificate of a corresponding authority issued by the blockchain to the data producer with the authority in advance.
It should be understood that the write permission of the first data possessed by the data producer may generally refer to the permission of the data producer to write any data to the blockchain. However, in order to ensure the real validity of data, the write authority of the first data possessed by the data producer may only represent the write authority of one or several variables, for example, different service systems have different services and write data to the blockchain at the same time, each service system may be understood as a data producer, the service system a can only modify, add and delete the data with the variables of user age and user gender, and the service system b can only modify, add and delete the data with the variables of user name and user telephone.
The authentication contract is an intelligent contract preset in the block chain, and can determine whether the data producer has the write-in authority and the write-in range of the write-in authority according to the authority information of each data producer.
On the basis of any of the above embodiments, after data produced by a data producer in real time is stored, in order to avoid increasing the load of an online system by reading the data online and causing an influence on online services of the data producer, in the embodiments of the present application, a block chain technology is used to provide data services to data consumers, and at the same time, different data consumers can only read data through one block chain node, so that the reading requests of different data consumers are respectively responded by a plurality of block chain nodes, and the problem of excessive load caused by reading data by a plurality of data consumers at the same time can be dispersed. This will be described in detail with reference to fig. 5.
Fig. 6 is a flowchart illustrating a data processing method 600 according to an embodiment of the present application. As shown in fig. 6:
s601: and receiving a data reading request sent by a data demand party.
Illustratively, the data request direction sends a data read request to the blockchain node, for requesting the blockchain node to read the third data from the blockchain. It should be understood that the third data should be included in the data read request.
It should be understood that since the data writing process is real-time, i.e., data is written into the block chain continuously during the data production process of the data producer, the data reading process may occur after the data writing process is completed or during the data writing process.
S602: and determining third data through the data reading contract, and displaying the third data to the data demand side.
The data reading contract is an intelligent contract preset in the block chain and can determine whether to read the third data and the storage position of the third data. Further, the data reading contract can also determine whether the third data has at least one fourth data related to the third data according to the third data.
In this step, it is determined that there are two possible implementations of the third data:
the method comprises the steps that firstly, third data are inquired in a block chain through a data reading contract, and after the third data are inquired, the third data are displayed to a data demand side; and querying third data in the block chain through a data reading contract, determining at least one fourth data related to the third data through the data reading contract if the third data is not queried, reading the at least one fourth data in the block chain, and determining the third data according to the at least one fourth data. Referring to fig. 5, at this time, a is the third data, B, C, D, E, F is the fourth data, and the third data a can be obtained according to the read fourth data by the X contract and the Y contract, for example, if B or C is already stored in the blockchain, the third data a can be determined according to B, C, D data by the Y contract directly.
Determining whether the third data has at least one related fourth data through a data reading contract, and directly reading the third data if the third data does not have at least one fourth data; if the third data has at least one fourth data related to the third data, determining whether the third data is stored in the block chain, if the third data is stored, directly reading the third data, and if the third data is not stored, acquiring the fourth data.
On the basis of the embodiment shown in fig. 6, in order to ensure the security of data in the process of reading data, the embodiment of the present application needs to confirm whether a data consumer has a read right. Before the third data is displayed to the data demander, the method further includes: and determining whether the data demand party has the read right of the third data or not according to the right information of the data demand party carried by the data read request through an authentication contract, displaying the third data to the data demand party when the data demand party has the read right of the third data, and stopping inquiring the third data or not displaying the third data to the data demand party when the data demand party does not have the read right of the third data. The authority information comprises account information of the data demand party or a certificate of the data demand party.
It should be understood that the read permission of the third data possessed by the data demander can generally refer to the permission of the data demander to read any data from the blockchain. However, in order to ensure data security, the data consumer may have a third data read right that only indicates a read right for one or more variables.
It should be understood that the authentication contract authenticating the data producer and the authentication contract authenticating the data consumer may be the same contract or may be different sub-contracts belonging to the same authentication contract.
In a specific implementation manner, the determining whether the data demander has the read permission of the third data includes determining whether the data demander has the read permission of the fourth data, and when the data demander has the read permission of the third data and also has the read permission of each fourth data related to the third data, the data demander is considered to have the read permission of the third data.
Optionally, in order to ensure the security of the stored data, the data to be written may be encrypted in the data writing process, so as to avoid the plaintext information being directly read by the data requiring party. Correspondingly, when a data demanding party with authority reads data, the read encrypted data needs to be decrypted, and then decrypted plaintext information is displayed to the data demanding party.
Fig. 7 is a schematic structural diagram of a block link point 700 according to an embodiment of the present disclosure, and as shown in fig. 7, the node 700 includes:
a receiving unit 710, configured to receive a data write request sent by a data producer, where the data write request is used to request to write first data;
a processing unit 720, configured to determine write data according to the first data and the data write contract; the write data comprises the first data and/or at least one second data related to the first data;
a read/write unit 730, configured to write the write data into the block chain.
The block link point 700 provided by this embodiment includes a receiving unit 710, a processing unit 720, and a reading and writing unit 730, and by receiving a data write request sent by a data producer, determining write data according to a first data and a data write contract requested to be written by the data write request, and then writing the write data into a block chain, and using a synchronization mechanism between block chain nodes in a block chain network, the data is stored in real time, so as to avoid a delay problem between offline data and online data, so that complete data can be provided for a data consumer.
In one possible design, the processing unit 720 is specifically configured to:
determining, by the data write contract, whether the first data is used to determine the at least one second data;
determining that the write data includes the first data and the at least one second data if the first data is used to determine the at least one second data;
otherwise, determining that the write data includes the first data.
In one possible design, the read/write unit 730 is specifically configured to:
writing the first data to the block chain via the data write contract.
In one possible design, the read/write unit 730 is specifically configured to:
determining, by the data processing contract, the at least one second datum from the first datum;
writing the first data and the at least one second data to the block chain by the data write contract; or, the first data is written into the block chain through the data write contract, and the at least one second data is written into the block chain through the data processing contract.
In one possible design, processing unit 720 may further be configured to:
determining whether the data producer has the write-in authority of the first data or not according to the authority information of the data producer carried by the data write-in request through the authentication contract; the authority information comprises account information of the data producer or a certificate of the data producer;
and if the data producer has the write permission of the first data, executing the step of writing the write data into the block chain.
In a possible design, the receiving unit 710 is further configured to receive a data read request sent by a data demander, where the data read request is used to request to read third data;
the processing unit 720 is further configured to determine the third data according to the data reading contract, and display the third data to the data demander.
In one possible design, the processing unit 720 is specifically configured to:
querying the blockchain for the third data via the data read contract;
if the third data exists in the block chain, reading the third data;
if the third data does not exist in the block chain, reading at least one fourth data; and determining the third data according to the at least one fourth data.
In one possible design, processing unit 720 may further be configured to:
determining whether the data demand party has the read authority of the third data or not according to the authority information of the data demand party carried by the data read request through the authentication contract; the authority information comprises account information of the data demander or a certificate of the data demander;
and if the data demander has the reading permission of the third data, executing the step of displaying the third data to the data demander.
In one possible design, the read/write unit 730 is specifically configured to:
generating a write transaction according to the write data;
and after the block chain network passes the consensus of the write transaction, writing the write data into the block chain according to the sequence determined by the consensus.
The block link point provided in this embodiment may be used to implement the method of authenticating a participating node side in any of the above embodiments, and the implementation effect is similar to that of the method embodiment, and is not described here again.
Fig. 8 is a schematic hardware structure diagram of a node 800 of a blockchain according to an embodiment of the present disclosure. As shown in fig. 8, in general, a blockchain node 800 includes: a processor 801 and a memory 802.
The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 801 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 801 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.
Memory 802 may include one or more computer-readable storage media, which may be non-transitory. Memory 802 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 802 is used to store at least one instruction for execution by processor 801 to implement the methods provided by the method embodiments herein.
Optionally, as shown in fig. 8, the blockchain node 800 may further include a transceiver 803, and the processor 801 may control the transceiver 803 to communicate with other devices, and specifically, may transmit information or data to the other devices or receive information or data transmitted by the other devices.
The transceiver 803 may include a transmitter and a receiver, among others. The transceiver 803 may further include an antenna, and the number of antennas may be one or more.
Optionally, the blockchain node 800 may implement corresponding processes in the methods of the embodiments of the present application, and for brevity, details are not described here again.
Those skilled in the art will appreciate that the architecture shown in fig. 8 does not constitute a definition of a blockchain node 900, and may include more or fewer components than shown, or combine certain components, or employ a different arrangement of components.
Embodiments of the present application also provide a non-transitory computer-readable storage medium, where instructions in the storage medium, when executed by a processor of a node of a blockchain, enable the blockchain node to perform the method provided by the above embodiments.
The computer-readable storage medium in this embodiment may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, etc. that is integrated with one or more available media, and the available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., SSDs), etc.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
The embodiment of the present application also provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the method provided by the above embodiment.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.