HK1186793B - Service data push method and device thereof - Google Patents

Description

Business data pushing method and device

Technical Field

The application relates to the field of internet, in particular to a business data pushing method and device.

Background

When a user queries data in an internal operation service system, the system can provide a database-based data retrieval function, and the database-based data retrieval function can quickly respond to the query requirement of the user in the case of small data quantity in the database (for example, the case of less than one million data). However, as the amount of data in the database of the system increases, for example, after the amount of data exceeds one million, the above-mentioned database-based query function cannot quickly respond to the query requirement of the user, i.e., the response speed of the system to the query request of the user decreases rapidly, and therefore, the data query function provided by the system has poor performance in the case that the amount of data exceeds a certain limit.

In order to solve the above problems, the prior art may adopt a search engine technology to improve the performance of the query function, and the response speed of the technology does not have obvious performance reduction along with the increase of the data volume. Specifically, in the prior art, a data query function is realized by creating a data synchronization relationship between database data in a business system and a search engine server, and meanwhile, the search engine server actively accesses data changed by a database on the database server in the business system at a certain frequency, queries static data in the database in an SQL query manner, and constructs an index of new data when the database data on the database server in the business system is changed. Specifically, the index construction process can be realized by the following steps:

in step S1, when the operation database data of the service system is changed, such as adding, modifying, or deleting database data, the database trigger records the ID of the changed data object in the database and stores the ID in a log table.

Step S2, the search engine server starts a timing task, periodically and circularly reads the ID of the changed data object from the log table in step S1, and the search engine queries the database view of a plurality of table combinations according to the ID of the ID, thereby obtaining the corresponding data.

In step S3, the search engine server builds an index from the data acquired in step S2.

The above prior art realizes that, after data in a database in a business system is changed, a search engine server may read identification numbers ID of the changed data from the database in the business system at certain time intervals (for example, about 1-5 minutes), acquire corresponding changed data according to the identification numbers ID, and establish an index of the changed data.

However, the above solution has the following problems: firstly, the time delay of data synchronization between a database in a business system and a search engine server is too long, and a user can query the changed data in the database in the business system by using the search engine server after a certain time interval (for example, about 1-5 minutes); secondly, static data in a database in a business system is acquired in an SQL query mode, and a serious performance problem exists in a specific business scene. For example, in a certain service scenario in the service system, if there is a lot of data changes in a short time, the search engine server will build a delay exceeding a predetermined time interval after obtaining data from the server in the service system; in addition, as the data volume is larger and larger, the SQL query performance is slower and slower, thereby reducing the efficiency of the search engine server acquiring the updated data in the server in the business system.

At present, no effective solution is provided for the problem that the way of regularly reading the database data in the database server of the business system by the search engine server in the related art causes inconsistent synchronization data between the search engine server and the database in the business system under the condition that the database data in the business system is greatly changed.

Disclosure of Invention

The present application mainly aims to provide a method and an apparatus for pushing service data, so as to solve the above problem, where a manner of regularly reading database data in a database server of a service system by a search engine server in the related art may cause inconsistency of synchronous data between the search engine server and the database in the service system when the database data in the service system is changed greatly.

In order to achieve the above object, according to an aspect of the present application, there is provided a method for pushing service data, the method including: when detecting that a service layer of a client calls a data object access layer to perform change operation on service data, intercepting the service data which is subjected to the change operation to obtain change data; analyzing the intercepted change data according to a pushing rule to acquire pushing data needing to execute a pushing task; after the database transaction is submitted, a push task is asynchronously initiated to send the push data to the search engine server.

Further, before intercepting the service data subjected to the change operation to obtain the change data, the method further includes: presetting an interception condition for carrying out interception operation and a name space for determining a pushing task in a client so as to obtain a configuration file; and starting the operation of intercepting the service data according to the intercepting condition in the configuration file.

Further, analyzing the intercepted change data according to a pushing rule to acquire the pushing data required to execute the pushing task includes: analyzing the changed data to obtain the attribute of the changed data, wherein the entity object is analyzed under the condition that the obtained attribute indicates that the changed data is determined to be the entity object, and if the analysis result is the same as the preset name space, the entity object is marked as the push data needing to execute the push task; and under the condition that the obtained attribute indicates that the changed data is the SQL statement, extracting a name space in the SQL statement, comparing the name space in the SQL statement with a preset name space, and under the same condition, marking the operation data of the SQL statement as push data needing to execute a push task.

Further, asynchronously initiating a push task to send push data to the search engine server after the database transaction is committed comprises: performing section-oriented editing (AOP) initialization operation on a data object access layer (DAO) to monitor the submission operation of database transactions in real time, and calling a push function after monitoring that the database transactions are submitted; starting to read the saved pushed data according to the pushing function; and sending the pushed data to a search engine server.

Further, before sending the pushed data to the search engine server, the method further comprises: extracting key field values in the pushed data; searching one or more corresponding data objects in a database according to the key field value; and assembling all the acquired data objects according to a preset format to acquire the push data according with the format regulation.

Further, after parsing the intercepted change data according to the push rule to obtain push data that needs to execute a push task, the method further includes: and caching the obtained push data into a thread local variable.

In order to achieve the above object, according to another aspect of the present application, there is provided a service data pushing device, including: the intercepting module is used for intercepting the business data which is subjected to change operation when the business layer of the client calls the data object access layer to carry out the change operation of the business data so as to obtain the change data; the processing module is used for analyzing the intercepted change data according to a pushing rule so as to acquire pushing data needing to execute a pushing task; and the pushing module is used for asynchronously starting a pushing task after the database transaction is submitted so as to send the pushing data to the search engine server.

Further, the apparatus further comprises: the configuration module is used for presetting an interception condition for carrying out interception operation in the client and determining a name space for pushing a task so as to obtain a configuration file; and the starting module is used for starting the operation of intercepting the service data according to the intercepting condition in the configuration file.

Further, the processing module includes: the analysis module is used for analyzing the changed data to acquire the attribute of the changed data; the first marking module is used for analyzing the entity object under the condition that the obtained attribute indication determines that the changed data is the entity object, and marking the entity object as push data needing to execute a push task if the analysis result is the same as a preset name space; and the second marking module is used for extracting the name space in the SQL statement and comparing the name space in the SQL statement with a preset name space under the condition that the obtained attribute indicates that the changed data is the SQL statement, and marking the operation data of the SQL statement as the push data needing to execute the push task under the same condition.

Further, the push module includes: the monitoring module is used for monitoring the submission operation of the database transaction in real time by performing section-oriented editing (AOP) initialization operation on the DAO, and calling a push function after monitoring that the database transaction is submitted; the reading module is used for starting to read the stored push data according to the push function; and the sending module is used for sending the pushed data to the search engine server.

Further, the apparatus further comprises: the extraction module is used for extracting key field values in the pushed data; the query module is used for searching one or more corresponding data objects in the database according to the key field value; and the assembling module is used for assembling all the acquired data objects according to a preset format so as to acquire the push data meeting the format regulation.

By the method, when the business layer of the client calls the data object access layer to change the business data, the business data which is changed is intercepted to obtain the changed data; analyzing the intercepted change data according to a pushing rule to acquire pushing data needing to execute a pushing task; the embodiment intercepts changed business data before the business data are updated to the database, determines that the intercepted changed data belong to the pushing task according to a configured pushing rule, and asynchronously starts to push the pushing data to the search engine server after the database transaction is submitted, wherein the process is independent of the updating process of the data in the database in the business system. Since the changed data in the business system is intercepted and pushed before reaching the database, and the pushed data is sent to the search engine server after the transaction of the database is submitted and the asynchronous starting is started, the search engine server does not need to read the stored updated data from the database in the business system at regular time, namely, the data is updated almost synchronously in the database in the business system and the search engine server. Therefore, the problem that the synchronous data between the database server in the business system and the database in the business system are inconsistent under the condition that the database data in the business system is greatly changed due to the mode that the database server in the business system is read at regular time by the search engine server is solved, and further the delay time of the synchronous data between the database in the business system and the search engine server is greatly shortened, so that the maintenance cost of the search engine server is reduced, and the user experience effect is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic structural diagram of a pushing device for service data according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a business system framework including a pushing device of business data shown in FIG. 1;

fig. 3 is a functional structure diagram of a service data pushing device according to the embodiment shown in fig. 1;

fig. 4 is a flowchart of a pushing method of service data according to an embodiment of the present application; and

fig. 5 is a detailed flowchart of a pushing method of service data according to the embodiment shown in fig. 4.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Fig. 1 is a schematic structural diagram of a pushing device for service data according to an embodiment of the present application; FIG. 2 is a schematic diagram of a business system framework including a pushing device of business data shown in FIG. 1; fig. 3 is a functional structure diagram of the service data pushing apparatus according to the embodiment shown in fig. 1.

As shown in fig. 1, the service data pushing apparatus provided in this embodiment of the present application may include: the intercepting module 10 is configured to intercept the service data subjected to the change operation to obtain change data when it is detected that a service layer of the client calls a data object access layer to perform the change operation on the service data; the processing module 30 is configured to analyze the intercepted change data according to a push rule to obtain push data that needs to execute a push task; and the pushing module 50 is used for asynchronously starting a pushing task after the database transaction is submitted so as to send the pushing data to the search engine server.

In the above embodiment of the present application, before the service data is updated to the database, the pushing device first intercepts the changed service data through the intercepting module 10, and sends the obtained changed data to the processing module 30, the processing module 30 determines that the intercepted changed data belongs to a pushing task according to the configured pushing rule, and after the transaction of the database is submitted, asynchronously starts to push the pushed data to the search engine server, and the process is independent of the updating process of the data in the database in the service system. Because the changed data in the business system is intercepted and pushed before reaching the database, and the pushed data is sent to the search engine server after the transaction of the database is submitted and the asynchronous starting is started, the search engine server does not need to read the stored updated data from the database in the business system at regular time, namely the data is almost synchronously updated in the database in the business system and the search engine, compared with the existing mode that the search engine server reads the data in the database in the business system at regular time, the embodiment of the application realizes the effect that the synchronous data between the search engine server and the database in the business system is kept consistent under the condition that the data change of the database in the business system is large, and greatly shortens the delay time of the synchronous data between the database in the business system and the search engine server, therefore, the maintenance cost of the search engine server is reduced, and the user experience is improved. In the above embodiment, after the service data is intercepted, the service data is sent to the data object access layer for updating the data in the database in the service system.

Specifically, as can be seen from fig. 2 and fig. 3, the service data pushing apparatus in the foregoing embodiment is a real-time data pushing framework embedded in a service system architecture.

As shown in fig. 2, the service system structure includes: a web layer, a service layer, a data object access layer, a real-time data push framework shown in fig. 3 in this application is located at the bottom layer of the whole service system architecture, wherein, an interception module for intercepting data change operations in fig. 2 is located between the "service layer" and the "data object access layer", the interception module performs the data interception function in fig. 3, that is, when the service layer calls the data object access layer to perform data operations, any change operation on a data object by the "service layer" is intercepted by the interception module, that is, the change data is intercepted, after the changed data object is intercepted, the intercepted change data is sent to a processing module for completing the "real-time data push" service, the processing module performs the data analysis function in fig. 3 by analyzing the intercepted change data, that is, according to the configured push rule, and after a task thread is started to finish data analysis and data assembly, if the transaction monitoring module in the figure 2 monitors the condition of submitting the database transaction, the system asynchronously starts a thread, intercepts the changed data from a service layer in a service system and then pushes the changed data to the search engine server, namely, the data pushing monitoring function and the asynchronous data pushing function in the figure 3 are finished.

In most application scenarios, the embodiment greatly shortens the delay time of synchronously changing data between the database and the search engine, and greatly improves the user experience. In addition, the index of the search engine is further created by adopting the scheme, so that the delay time for creating the index can be greatly shortened.

Preferably, the intra-thread communication function in fig. 3 further implements that after it is determined that the intercepted changed data is data that needs to be pushed, the pushed data is cached in the intra-thread communication channel, and the changed data that needs to be pushed in the communication channel is consumed after the database transaction is submitted, specifically, the obtained pushed data can be cached in the thread local variable, and the thread local variable ThreadLocal can ensure that each thread has its own local variable and cannot be modified by other threads, thereby improving the stability of the pushed data. In addition, the thread pool management function in FIG. 3 provides management of threads, such as setting thread capacity and thread lifecycle.

As can be seen from the above, the key point of the present application is that compared with the existing push technology, data is extracted after data is updated, and the real-time data push framework shown in fig. 3 intercepts the service data before updating before the service data is updated to the database, and asynchronously pushes the intercepted data to the search engine server after the intercepted data is analyzed.

The pushing device of the service data in the foregoing embodiment of the present application may further include: the configuration module is used for presetting an interception condition for carrying out interception operation in the client and determining a name space for pushing a task so as to obtain a configuration file; and the starting module is used for starting the operation of intercepting the service data according to the intercepting condition in the configuration file. The embodiment presets the interception condition, for example, which database tables can be intercepted, thereby eliminating the service data which does not need to be pushed, and avoiding wasting system resources in the process of pushing and changing data by the system.

Specifically, the real-time data pushing frame in the present application may be a plug-in of a spring frame, as shown in fig. 2, when the business system is started, the spring frame is initialized, and then, the content of the configuration file in the real-time data pushing frame is initialized. The contents of the configuration file may include: setting the name space of the database table needing to be intercepted, the field needing to be intercepted (such as ID number) and the like.

The processing module 30 in the above embodiments of the present application may include: the analysis module 301 is configured to analyze the changed data to obtain an attribute of the changed data; a first marking module 302, configured to, when the obtained attribute indicates that the changed data is determined to be an entity object, analyze the entity object, and if an analysis result is the same as a preset namespace, mark the entity object as push data that needs to execute a push task; the second marking module 303 is configured to, when the obtained attribute indicates that the changed data is determined to be an SQL statement, extract a namespace in the SQL statement, compare the namespace in the SQL statement with a preset namespace, and, under the same condition, mark the operation data of the SQL statement as push data that needs to execute a push task.

Specifically, after the intercepting operation of the service data is completed, the data analysis function in fig. 3 first determines, through the analysis module 301, whether the intercepted changed data is an entity object or an SQL statement, if the intercepted changed data is an entity object, the entity object is analyzed, and if a namespace obtained through the analysis is the same as a preset namespace of the database, it is marked that the database operation needs to start a data pushing task; and if the data is changed in the SQL language mode, directly intercepting and extracting the name space and comparing the name space with the preset name space of the database, and marking that the data pushing task needs to be started in the current database operation under the same condition.

The pushing module 50 in the above embodiments of the present application may include: the monitoring module 501 is configured to perform a section-oriented editing AOP initialization operation on the data object access layer DAO to monitor a commit operation of a database transaction in real time, and call a push function after monitoring that the database transaction is committed; a reading module 502, configured to start reading the saved pushed data according to a pushing function; a sending module 503, configured to send the pushed data to the search engine server. Preferably, the pushing device may further include: and the storage module is used for caching the obtained pushed data into the thread local variable.

Specifically, as shown in fig. 3, the data push monitoring function implements that, in the process of initializing the service system when starting up, the initialization operation of editing the AOP facing the section is performed on the data object access layer DAO of the service system, so as to ensure that any database operation can be monitored, and after monitoring that a database transaction is submitted, a push function callback is called, so as to trigger the real-time push framework to consume the data in the thread local variable thread local. The push module 50 adopts an asynchronous data push mode and a thread asynchronous mode to perform a data push task, thereby ensuring that the data push does not affect normal business operation.

In addition, the data pushing monitoring function also provides monitoring for the operation process of other modules in the whole frame, once any step is abnormal, abnormal information of the system operation at that time can be recorded, whether the intercepted data is normal or not is mainly included, and technical personnel is informed of the abnormal information, and the technical personnel can trigger the data pushing task again according to the information.

The apparatus of the foregoing embodiment of the present application may further include: the extraction module is used for extracting key field values in the pushed data; the query module is used for searching one or more corresponding data objects in the database according to the key field value; and the assembling module is used for assembling all the acquired data objects according to a preset format so as to acquire the push data meeting the format regulation. According to the embodiment of the application, before the pushed data is sent to the search engine server, the key field value of the pushed data, such as the ID number of the pushed data, can be extracted through the extraction module, other data objects are inquired in the database according to the key field value, and all the data objects are sent to the search engine database, so that the integrity of the pushed data is ensured. For example, the database may be queried for data for the customer's order object based on the customer's ID number.

Fig. 4 is a flowchart of a pushing method of service data according to an embodiment of the present application; fig. 5 is a detailed flowchart of a pushing method of service data according to the embodiment shown in fig. 4.

As shown in fig. 4, the method includes the steps of:

step S102, by using the intercepting module 10 in fig. 1, when it is detected that the service layer of the client calls the data object access layer to perform a change operation on the service data, intercepting the service data that is subjected to the change operation to obtain the change data. Specifically, this step is implemented by the data interception function in fig. 3, and when the service layer calls the data object access layer to perform a change operation on the service data, the service data of the change operation is intercepted.

Step S104, parsing the intercepted changed data according to the push rule by the processing module 30 in fig. 1 to obtain the push data that needs to execute the push task. Specifically, the step is implemented by using the data analysis function in fig. 3 to determine whether the intercepted service data needs to be pushed to the search engine server according to the configured interception rule, where the pushing rule is one or more preset conditions, and the intercepted change data is determined to belong to the pushing task according to the conditions.

Step S106, the pushing module 50 in fig. 1 is used to asynchronously start the pushing task after the database transaction is submitted, so as to send the pushed data to the search engine server. In the step, a thread asynchronous mode is adopted to carry out a data pushing task, so that the data pushing does not influence normal business operation.

In the embodiment of the application, the changed business data is intercepted before the business data is updated to the database, the intercepted changed data is determined to belong to a pushing task according to the configured pushing rule, and the pushing of the pushing data to the search engine server is asynchronously started after the database transaction is submitted, wherein the process is independent of the updating process of the data in the database. Because the changed data is intercepted and pushed before reaching the database, and the pushed data is sent to the search engine after the transaction of the database is submitted by asynchronous starting, the search engine does not need to read the stored updated data from the database regularly, namely the data is almost synchronously updated in the database and the search engine, compared with the existing mode that the search engine server reads the data in the database regularly, the embodiment of the application realizes the effect of keeping the synchronous data between the search engine server and the database consistent under the condition that the data of the database is greatly changed, greatly shortens the delay time of the synchronous data between the database and the search engine, thereby achieving the effects of reducing the maintenance cost of the search engine server and improving the user experience. In the above embodiment, after the service data is intercepted, the service data is sent to the data object access layer for updating the data in the database.

Preferably, after step S104 in the above embodiment of the present application, the obtained pushed data may be cached in the thread local variable by the saving module in fig. 1. This step is implemented by the in-thread communication function of fig. 3, after determining that the intercepted changed data is data that needs to be pushed, the pushed data is cached in the in-thread communication channel, that is, the pushed data is temporarily stored in a thread local variable of a thread local type. The thread local variable ThreadLocal may ensure that each thread has its own local variable and that data is not modified by other threads.

In the above embodiment of the present application, before intercepting the service data subjected to the change operation in step S102 to obtain the change data, the method may further include the following steps: presetting an interception condition for carrying out interception operation and a name space for determining a pushing task in a client so as to obtain a configuration file; and starting the operation of intercepting the service data according to the intercepting condition in the configuration file. The embodiment presets the interception condition, for example, which database tables can be intercepted, thereby eliminating the service data which does not need to be pushed, and avoiding wasting system resources in the process of pushing and changing data by the system.

Specifically, the real-time data pushing frame in the present application may be a plug-in of a spring frame, and when the business system is started, the spring frame is initialized, so that the content of the configuration file in the real-time data pushing frame is initialized. Specifically, the initialized configuration file may mainly include the following:

1) the interception condition is configured to indicate which database table (e.g., client table) needs to be intercepted.

For example, an example of configuration content that intercepts operations of which database tables is as follows:

2) the namespace used to determine the push task is configured, i.e., the namespace ibatis namespace in the data object access framework that includes the database table is configured.

3) Fields, such as an ID field, that need to intercept traffic data are configured.

4) Configuration data object access initialization of the DAO layer for profile-oriented programming AOP. When a business system is started, a real-time pushing framework firstly initializes the AOP on a DAO layer of the system, so that any database operation can be guaranteed to be monitored by the framework, for example, when any data change operation aiming at a client table is monitored, the framework is triggered to execute an interception operation, and the intercepted business data is triggered to be subjected to data analysis.

In the above embodiment of the present application, the step S104 of analyzing the intercepted change data according to the push rule to obtain the push data that needs to execute the push task includes: analyzing the changed data to obtain the attribute of the changed data, wherein the entity object is analyzed under the condition that the obtained attribute indicates that the changed data is determined to be the entity object, and if the analysis result is the same as the preset name space, the entity object is marked as the push data needing to execute the push task; and under the condition that the obtained attribute indicates that the changed data is the SQL statement, extracting a name space in the SQL statement, comparing the name space in the SQL statement with a preset name space, and under the same condition, marking the operation data of the SQL statement as push data needing to execute a push task.

Specifically, the above embodiment completes the screening process of the push data that needs to execute the push task through the real-time data push framework in fig. 3, and specifically includes the following steps:

and step A, after the interception operation of the business data is finished, a data analysis function in the data pushing framework analyzes the intercepted business data.

And step B, judging whether the intercepted business data is a solid object or an SQL statement, if the intercepted business data is the solid object, entering step C, and if the intercepted business data is the SQL statement, entering step D.

And step C, if the data object is the entity object, analyzing the entity object, wherein the analysis result is the namespace of the technical framework ibatis accessed by the data object. If the parsed namespace is matched with the namespace of the configured database table (the configuration of the namespace is configured before the system is started), marking that the database operation corresponding to the entity object needs to start a data pushing task.

And D, if the data is changed in the SQL mode, directly matching the intercepted namespace with the namespace of the configured database table, and marking that the database operation corresponding to the SQL statement needs to start a data pushing task if the intercepted namespace and the namespace are the same.

Preferably, after the business data pushing framework obtains the matching result, the intercepted pushing data needs to be temporarily stored in a thread local variable thread type variable. The thread local variable ThreadLocal ensures that each thread has its own local variable and is not modified by other threads. The stability of the push data is improved.

In the above embodiment of the present application, in step S106, after the database transaction is submitted, the step of asynchronously starting the push task to send the push data to the search engine server may include: performing section-oriented editing (AOP) initialization operation on a data object access layer (DAO) to monitor the submission operation of database transactions in real time, and calling a push function after monitoring that the database transactions are submitted; starting to read the push data stored in the thread local variable according to the push function; and sending the pushed data to a search engine server. The function adopts an asynchronous data pushing mode and a thread asynchronous mode to carry out data pushing tasks, and ensures that normal business operation is not influenced by data pushing.

Specifically, in the foregoing embodiment, the real-time data pushing framework shown in fig. 3 develops a plug-in for managing spring database transactions, which can monitor the commit operation of the database transactions. When the spring database transaction is submitted, a push function callback is called, so that the real-time push framework is triggered to consume the data in the thread local variable ThreadLocal.

In the above embodiment of the present application, before sending the pushed data in the thread local variable to the search engine server, the method may further include the following steps: extracting key field values in the pushed data; searching one or more corresponding data objects in a database according to the key field value; and assembling all the acquired data objects according to a preset format to acquire the push data according with the format regulation. The embodiment realizes that before the push data is sent to the search engine server, the key field value of the push data, such as the ID number of the push data, can be extracted, other data objects can be inquired in the database according to the key field value, and all the data objects are sent to the search engine database, so that the integrity of the push data is ensured.

Specifically, with reference to fig. 5, the steps of the foregoing embodiment of the present application are as follows:

step S20, when the service system in the service system is started, initializing the configuration file pushed in real time, and executing an operation of intercepting the changed service data according to the configuration file before the service data is updated to the database.

And step S40, analyzing the intercepted change data to obtain the change data needing to be pushed. Specifically, the step a to the step D may be implemented to determine the changed data that needs to execute the push task, so as to obtain the push data.

Step S60, cache the parsed push data meeting the configuration requirement in the thread local variable, obtain the changed data that needs to execute the push task, and save the changed data as the push data in the linear local variable.

Step S80, after the service system is initialized, starting monitoring database transactions (including addition, modification, and deletion operations) of the current thread, and after the monitoring database transactions are submitted and the intercepted push data is processed, starting a data push task in real time, and sending the push data to the search engine server.

The above step 80 comprises the steps of:

step S801, after monitoring that the database transaction is committed, reading the saved pushed data from the thread local variable.

In step S802, key field values, such as primary key IDs of data objects, are extracted from the changed data to be pushed, i.e., from the pushed data.

In step S803, the static data that has been submitted and saved is searched for in the database according to the key field value (e.g., the primary key ID). For example, the data of the order object of the customer is queried in the database according to the customer's ID.

Step S804, meanwhile, acquiring other related data objects according to the data object of the stored static data, and acquiring the static data of other data objects from the database.

Step S805, after all the data objects are acquired, the framework assembles the intercepted pushed data according to the data format required by the predefined search engine according to the pre-configured data assembly logic, so as to assemble the service data required to be pushed into the data format meeting the requirement of the search engine.

Step S806, the search engine client is called, and the assembled push data is pushed to the search engine server. Specifically, the assembled service data completely meets the requirement of a data format of a search engine, and after the service data is assembled, the real-time data pushing framework initiates communication with the search engine and pushes data to the search engine after the real-time data pushing framework is connected with the search engine.

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.

The application can also provide a computer program for operating the pushing method of the business data and a storage device for storing the computer program.

From the above description, it can be seen that the following technical effects are achieved by the present application: the problem that the synchronous data between the search engine server and the database are inconsistent under the condition that the database data is greatly changed due to the mode that the search engine server reads the data in the database at regular time is solved, and further the delay time of the synchronous data between the database and the search engine is greatly shortened, so that the maintenance cost of the search engine server is reduced, and the user experience effect is improved.

It will be apparent to those skilled in the art that the modules or steps of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present application is not limited to any specific combination of hardware and software.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (11)

1. A method for pushing service data is characterized by comprising the following steps:

when detecting that a service layer of a client calls a data object access layer to perform change operation on service data, intercepting the service data which is subjected to the change operation to obtain change data;

analyzing the intercepted change data according to a pushing rule to acquire pushing data needing to execute a pushing task;

and after the database transaction is submitted, asynchronously starting the push task to send the push data to the search engine server.

2. The method of claim 1, wherein before intercepting the business data undergoing change operations to obtain change data, the method further comprises:

presetting an interception condition for carrying out interception operation and a name space for determining a pushing task in the client so as to obtain a configuration file;

and starting the operation of intercepting the service data according to the intercepting condition in the configuration file.

3. The method of claim 2, wherein parsing the intercepted change data according to a push rule to obtain push data needed to perform a push task comprises:

parsing the change data to obtain attributes of the change data, wherein,

under the condition that the obtained attribute indication determines that the changed data is an entity object, analyzing the entity object, and if the analysis result is the same as a preset name space, marking the entity object as push data needing to execute a push task;

and under the condition that the obtained attribute indication determines that the changed data is the SQL statement, extracting a name space in the SQL statement, comparing the name space in the SQL statement with a preset name space, and under the same condition, marking the operation data of the SQL statement as push data needing to execute a push task.

4. The method of claim 3, wherein asynchronously initiating the push task to send the push data to the search engine server after the database transaction is committed comprises:

performing section-oriented editing (AOP) initialization operation on the DAO to monitor the submission operation of the database transaction in real time, and calling a push function after monitoring that the database transaction is submitted;

starting to read the saved pushed data according to the pushing function;

and sending the push data to a search engine server.

5. The method of claim 1, wherein prior to sending the pushed data to a search engine server, the method further comprises:

extracting key field values in the push data;

searching one or more corresponding data objects in the database according to the key field value;

and assembling all the acquired data objects according to a preset format to acquire the push data according with the format regulation.

6. The method according to any of claims 1 to 5, wherein after parsing the intercepted change data according to a push rule to obtain push data that needs to perform a push task, the method further comprises: and caching the obtained push data into a thread local variable.

7. A device for pushing service data, comprising:

the intercepting module is used for intercepting the business data which is subjected to change operation when the business layer of the client calls the data object access layer to carry out the change operation of the business data so as to obtain the change data;

the processing module is used for analyzing the intercepted change data according to a pushing rule so as to acquire pushing data needing to execute a pushing task;

and the pushing module is used for asynchronously starting the pushing task after the database transaction is submitted so as to send the pushing data to the search engine server.

8. The apparatus of claim 7, further comprising:

the configuration module is used for presetting an interception condition for carrying out interception operation in the client and a name space for determining a pushing task so as to obtain a configuration file;

and the starting module is used for starting the operation of intercepting the service data according to the intercepting condition in the configuration file.

9. The apparatus of claim 8, wherein the processing module comprises:

the analysis module is used for analyzing the changed data to acquire the attribute of the changed data;

the first marking module is used for analyzing the entity object under the condition that the obtained attribute indication determines that the changed data is the entity object, and marking the entity object as push data needing to execute a push task if the analysis result is the same as a preset name space;

and the second marking module is used for extracting the name space in the SQL statement and comparing the name space in the SQL statement with a preset name space under the condition that the obtained attribute indication determines that the changed data is the SQL statement, and marking the operation data of the SQL statement as the push data needing to execute the push task under the same condition.

10. The apparatus of claim 9, wherein the push module comprises:

the monitoring module is used for monitoring the submission operation of the database transaction in real time by performing section-oriented editing (AOP) initialization operation on the data object access layer (DAO), and calling a push function after monitoring that the database transaction is submitted;

the reading module is used for starting to read the saved pushed data according to the pushing function;

and the sending module is used for sending the push data to a search engine server.

11. The apparatus of claim 7, further comprising:

the extraction module is used for extracting key field values in the push data;

the query module is used for searching one or more corresponding data objects in the database according to the key field value;

and the assembling module is used for assembling all the acquired data objects according to a preset format so as to acquire the push data meeting the format regulation.