CN111427917A - Search data processing method and related product - Google Patents

Abstract

The embodiment of the present application discloses a search data processing method and related products. The method includes: receiving a first search data set transmitted by a first data source, and receiving multiple search service records transmitted by a second data source; Multiple search business records generate a second search data set, and asynchronously archive the second search data set and the first search data set as data to be consumed in a preset message queue; extract all the data from the preset message queue. A third search data set is obtained by performing a consumption operation on the data to be consumed; and the third search data set is stored in a preset first database and a second database. Implementing the embodiments of the present application has the advantage of high user experience.

Description

Search data processing methods and related products

technical field

The present application relates to the field of data storage, in particular to a search data processing method and related products.

Background technique

With the rapid development of Internet technology, search has become the basic function of current Internet products. Search is an extremely basic and core part of B-side applications in the traditional Internet era or mobile Internet C-side applications. The search function of an application is powerful and No has a great effect on enhancing user experience and improving user stickiness.

In the search workflow, the key point is the storage of search metadata. Traditional search metadata archiving basically adopts synchronous processing, that is, one-time collection and storage. However, this processing method may be in the face of emergencies. It is impossible to restore data in a timely manner, and when processing business data, the concurrent performance of this archiving operation is poor, which may occupy system physical resources on a large scale, resulting in blockage of other system services, delaying the overall network interaction of the application, and user experience. low.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a search data processing method and related products. A third search data set is obtained by asynchronously filing a first search data set and a second search data set in a preset message queue and then performing a consumption operation. The data set is stored in the first database and the second database to realize data decoupling, which is beneficial to reduce the occupancy rate of system physical resources, thereby reducing network interaction delay and improving user experience.

In a first aspect, an embodiment of the present application provides a method for archiving search data, including:

receiving the first search data set transmitted by the first data source, and receiving multiple search service records transmitted by the second data source;

generating a second search data set according to the plurality of search service records, and asynchronously storing the second search data set and the first search data set as data to be consumed in a preset message queue;

Extract the data to be consumed from the preset message queue and perform a consumption operation to obtain a third search data set;

The third search data set is stored in a preset first database and a second database.

In a second aspect, an embodiment of the present application provides a search data filing device, including:

a receiving unit, configured to receive the first search data set transmitted by the first data source, and receive a plurality of search service records transmitted by the second data source;

a generating unit, configured to generate a second search data set according to the plurality of search service records, and asynchronously store the second search data set and the first search data set as data to be consumed in a preset message queue;

a consumption unit, configured to extract the data to be consumed from the preset message queue and perform a consumption operation to obtain a third search data set;

A storage unit, configured to store the third search data set in a preset first database and a second database.

In a third aspect, an embodiment of the present application provides an electronic device, which is characterized in that it includes a processor, a memory, a communication device, and one or more programs, wherein the processor, the memory, and the communication device are connected to each other, wherein the communication device is used to communicate with each other. The external device performs information interaction, the one or more programs are stored in the memory and are configured to be executed by the processor, and the above program includes steps for executing any method in the first aspect of the embodiments of the present application instruction.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the computer program as described in the first embodiment of the present application. Some or all of the steps described in the method of an aspect.

In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute as implemented in the present application. For example, some or all of the steps described in the method described in the first aspect. The computer program product may be a software installation package.

It can be seen that, in this embodiment of the present application, the electronic device receives the first search data set transmitted by the first data source, and receives multiple search service records transmitted by the second data source; and generates a second search service record according to the multiple search service records. Searching for a data set, asynchronously storing the second search data set and the first search data set as data to be consumed in a preset message queue; extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain A third search data set; storing the third search data set in a preset first database and a second database. In this way, the third search data set is obtained by asynchronously filing the first search data set and the second search data set in the preset message queue and then performing the consumption operation, and the third search data set is stored in the first database and the second database. Data decoupling is conducive to reducing the occupancy rate of system physical resources, thereby reducing network interaction delay and improving user experience.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic structural diagram of a search data processing method provided by an embodiment of the present application;

2 is a schematic flowchart of another search data processing method provided by an embodiment of the present application;

3 is a schematic flowchart of another search data processing method provided by an embodiment of the present application;

4 is a schematic flowchart of another search data processing method provided by an embodiment of the present application;

5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 6 is a block diagram of functional units of a search data processing apparatus provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms "first", "second", "third" and "fourth" in the description and claims of the present invention and the accompanying drawings are used to distinguish different objects, rather than to describe a specific order. . Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

Reference herein to an "embodiment" means that a particular feature, result, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

Electronic devices may include various handheld devices with wireless communication capabilities, in-vehicle devices, wearable devices (eg, smart watches, smart bracelets, pedometers, etc.), computing devices or other processing devices connected to wireless modems, and various In the form of user equipment (User Equipment, UE), mobile station (Mobile Station, MS), terminal device (terminal device) and so on. For convenience of description, the devices mentioned above are collectively referred to as electronic devices.

Data archiving is the process of moving data that is no longer frequently used to a separate storage device for long-term preservation. The data archive consists of old data, but it is necessary and important data for future reference, and its data must be kept according to the rules. The data archive has indexing and search capabilities so that files can be found easily.

The embodiments of the present application will be described in detail below.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a search data processing method provided by an embodiment of the present application, which is applied to an electronic device. The search data processing method includes:

Step 101: Receive a first search data set transmitted by a first data source, and receive multiple search service records transmitted by a second data source;

Optionally, before receiving the first search data set transmitted by the first data source and receiving multiple search service records transmitted by the second data source, further comprising: setting a timing controller, where the timing controller is used to control the first data data transmission between the source and the second data source, when the time of the timing controller is equal to the preset time, a first data acquisition request and a search service acquisition request are generated, and the first data acquisition request is sent to the first data source, Send the search service acquisition request to the second data source, where the first data acquisition request is used to request the first data source to transmit the first search data set within a preset time, and the search service acquisition request is used to request the second data The source transmits multiple search transaction records within the preset time.

Optionally, before receiving the first search data set transmitted by the first data source and receiving multiple search service records transmitted by the second data source, the method further includes: when detecting the data acquisition instruction of the target user, requesting the target user Perform verification, if the verification is successful, generate a first data acquisition request and a search service acquisition request, send the first data acquisition request to the first data source, and send the search service acquisition request to the second data source, if the verification fails, Then the data acquisition instruction is rejected.

Further, verifying the target user includes: receiving the verification data in the data acquisition instruction, and verifying the target user according to the verification data, for example, if the verification data is fingerprint data, acquiring a preset fingerprint template set , compare the fingerprint data in the fingerprint template set, if the comparison is successful, the verification is successful; if the verification data is face data, then obtain a preset face template set, in the face template set to the person The face data is compared, and if the comparison is successful, the verification is successful.

Step 102, generating a second search data set according to the plurality of search service records, and asynchronously filing the second search data set and the first search data set as data to be consumed in a preset message queue;

Wherein, any one of the search service records in the plurality of search service records includes: search service flow.

Wherein, the first search data set includes at least one first search metadata, and the second search data set includes at least one second search metadata.

In the embodiments of the present application, the data to be searched is abstracted into data of various data organizations due to structural differences and association differences, which are collectively referred to as metadata. Their index structure is also different.

In the embodiment of this application, asynchronous archiving means that the collection sources of different metadata are different, the data is written to the message queue with different synchronization information, and then decoupled through the asynchronous mechanism of the message queue middleware, and allocated to different server instances for processing. The process of consuming, and then persisting to the database for storage. Among them, due to the difference of different metadata institutions and different information associations, it is necessary to provide different consumption instances for asynchronous consumption; and the data table format and fields stored in the database also exist differences, so perhaps a separate service should be provided.

Step 103, extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain a third search data set;

Optionally, a preset data extraction rule is acquired, the data to be consumed in the preset message queue is acquired according to the data extraction rule, and a consumption operation is performed on the to-be-consumed data to obtain the third search data set.

The data extraction rules may include: first-in, first-out rules, first-in, last-out rules, etc., which are not limited herein.

In the embodiments of the present application, data consumption is a consumption activity that takes statistical data products and statistical data services as consumption objects, is a relatively special form of information consumption, and is an important part of social consumption. It is the purpose and driving force of statistical data production, and is the embodiment of the final purpose of statistical work. The quantity, quality, structure and method of data consumption not only affect the production and dissemination of statistical data, but also directly reflect the quality of the use of statistical data, reflecting the overall function of statistics, but also affect the quality and level of data user consumption, which is related to statistics. career and the development of the whole society and economy. The trend of modern scientific development shows that almost all disciplines and all fields have accumulated data, and all need to use data to speak, and it is necessary to analyze the data to discover the rules. And, with the development of computer science and network technology, its situation is becoming more and more obvious, and the speed is getting faster and faster. Therefore, data consumption is of great practical significance to promoting various social undertakings, especially the development of statistical undertakings and information service industries, as well as promoting consumption upgrades, improving people's quality and level of life, and achieving sustainable economic development.

Step 104: Store the third search data set in a preset first database and a second database.

Wherein, the first database is used for search and retrieval, and the second database is used for record backtracking.

In a possible example, the generating the second search data set according to the multiple search service records includes: extracting multiple search service flows from the multiple search service records, wherein the multiple search service records are: Any search business flow in the flow includes at least one search keyword; performing a search keyword extraction operation on the multiple search business flows to obtain multiple search keyword sets corresponding to the multiple search business flows; synthesizing the multiple search business flows A set of search keywords is obtained to obtain the second set of search data.

Wherein, any one of the multiple search service flows includes: at least one search keyword and at least one search time corresponding to the at least one search keyword.

Optionally, obtain a preset keyword position set, where the keyword position set includes: at least one keyword position, and according to the keyword position set, a search keyword extraction operation is performed for each search service flow of the plurality of search business flows , obtain multiple search keyword sets, wherein the multiple search keyword sets are in one-to-one correspondence with the multiple search business flows, and extract multiple search keyword sets contained in the multiple search keyword sets to generate a second search data set , wherein the plurality of search keywords are the second metadata corresponding to the second search data set.

In a possible example, the extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain a third search data set includes: acquiring multiple listeners corresponding to multiple preset consuming servers, The multiple listeners are used to monitor data transmission operations of the first data source and the second data source; when the multiple listeners monitor the first data source and/or the second data source When the data transmission operation is performed, the plurality of consumption servers are started; the third search data set is obtained according to the consumption operation performed by the plurality of consumption servers on the data to be consumed.

The multiple listeners correspond to the multiple consumer servers one-to-one.

The listener among the multiple listeners is used to monitor the first data source and the second data source, and monitor the data transmission operations of the first data source and the second data source. /or when the second data source transmits data to the preset message queue, start the consumption model corresponding to the listener.

Optionally, obtaining the data to be consumed in the preset message queue and performing a consumption operation to obtain the third search data set further includes: setting a listener to the preset message queue, and when the listener monitors the data to be consumed in the preset message queue When writing, a consumption server start request is sent to the application processor, where the consumption server start request is used to request the application processor to start a plurality of consumption servers.

In a possible example, the obtaining the third search data set by performing a consumption operation on the data to be consumed according to the plurality of consumption servers includes: extracting data from the preset message queue according to a preset data extraction rule. extract the data to be consumed from the data; obtain the data format to be consumed of the data to be consumed, and determine whether the preset data format set includes the data format to be consumed; if the data format set includes the data format to be consumed, Determine the target consumption server corresponding to the data format to be consumed according to the mapping relationship between the preset data format and the consumption server and the plurality of consumption servers; generate a data consumption instruction according to the data to be consumed, and send it to the target consumption server The data consumption instruction, wherein the data consumption instruction is used to instruct the target consumption server to perform a consumption operation on the data to be consumed; receive a data consumption response returned by the target consumption server, and obtain a data consumption response from the data consumption response. Extracting consumption data; generating the third search data set according to the consumption data.

The format of the data to be consumed may include: JSON format, XML format, YAML format, etc., which are not limited herein.

Wherein, the data format set includes: multiple data formats, wherein any one of the multiple data formats may include: JSON format, XML format, YAML format, etc., which is not limited herein.

Wherein, the plurality of consumer servers are in one-to-one correspondence with the plurality of data formats included in the data format set.

The data format (data format) described in the embodiments of the present application is to describe the rules for saving data in files or records. It can be a text format in the form of characters, or a compressed format in the form of binary data. The text format in the form of characters occupies more storage space but has high transparency, and the compressed format in the form of binary numbers occupies less storage space but lacks transparency.

In the specific implementation process, the data to be consumed is extracted from the preset message queue according to a preset data extraction rule (for example, a first-in, first-out rule), that is, the data to be consumed at the end of the queue in the preset message queue is obtained, and the determination is made. The to-be-consumed data format of the to-be-consumed data, assuming that the to-be-consumed data format is a text format, the to-be-consumed data is to-be-consumed text data, obtains a preset data format set, and determines whether the data format set includes the text format, If the data format set includes the text format, obtain the mapping relationship between the data format and the consuming server, determine the text consuming server corresponding to the text format according to the mapping relationship between the data format and the consuming server, and determine the text consuming server corresponding to the text format according to the mapping relationship between the data format and the consuming server. The data generates a data consumption instruction, wherein the data consumption instruction is used to instruct the text consumption server to perform a consumption operation on the to-be-consumed text data, send the to-be-consumed text data to the text consumption server, and receive the data consumption returned by the text consumption server In response, the consumption text data corresponding to the to-be-consumed text data is extracted from the data consumption response, and the consumption text data is stored in the third search data set.

In a possible example, the method further includes: if the data format set does not include the data format to be consumed, generating a data consumption reminder according to the data format to be consumed, and the data consumption reminder is used to prompt the The consumption server corresponding to the data format to be consumed is missing; the data consumption reminder is sent to the preset mobile terminal.

Wherein, the data consumption reminder is used to send a reminder of the missing format of the data to be consumed to the preset mobile terminal. In the specific implementation process, assuming that the data format to be consumed is a text format, and it is judged that the data format set does not contain the text format, a data consumption reminder is generated according to the text format, wherein the data consumption reminder includes: text format, to-be-consumption reminder data, the data consumption reminder is used to prompt that the consumption server corresponding to the to-be-consumed data format is missing, that is, it is used to prompt that the text consumption server corresponding to the text format is missing in the current consumption server, and send a data consumption reminder to the preset mobile terminal.

In a possible example, after storing the third search data in the preset first database and the second database, the method further includes: when receiving input data from a target user, comparing the input data with the first database. The third search data set in a database is compared to obtain a plurality of comparison values; the third search data corresponding to the maximum value among the plurality of comparison values is determined as the search result; corresponding to the target user The mobile terminal sends the search result.

In the specific implementation process, the input data of the target user is received, and the input data may include: text data, audio data, etc., which are not limited here; the input data is compared with the third search data set stored in the first database to obtain Multiple comparison values, determine that the third search data corresponding to the maximum value among the multiple comparison values is the search result, and send the search result to the user terminal corresponding to the target user, wherein, if the input data is audio data, then Before the matching between the input data and the third search data stored in the first database, the method further includes: taking the audio data as the input of the preset audio recognition model, obtaining the audio text data corresponding to the audio data, and comparing the input data according to the audio text data. data is updated.

It can be seen that, in this embodiment of the present application, the electronic device receives the first search data set transmitted by the first data source, and receives multiple search service records transmitted by the second data source; and generates a second search service record according to the multiple search service records. Searching for a data set, asynchronously storing the second search data set and the first search data set as data to be consumed in a preset message queue; extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain A third search data set; storing the third search data set in a preset first database and a second database. In this way, the third search data set is obtained by asynchronously filing the first search data set and the second search data set in the preset message queue and then performing the consumption operation, and the third search data set is stored in the first database and the second database. Data decoupling is conducive to reducing the occupancy rate of system physical resources, thereby reducing network interaction delay and improving user experience.

Please refer to FIG. 2. FIG. 2 is a schematic flowchart of another search data processing method provided by an embodiment of the present application, which is applied to an electronic device. The search data processing method includes:

Step 201: Receive a first search data set transmitted by a first data source, and receive multiple search service records transmitted by a second data source;

Step 202, extracting multiple search service flows from the multiple search service records, wherein any search service flow in the multiple search service flows includes at least one search keyword;

Step 203, performing a search keyword extraction operation on the multiple search service flows, to obtain multiple search keyword sets corresponding to the multiple search service flows;

Step 204, combining the multiple search keyword sets to obtain the second search data set, and asynchronously filing the second search data set and the first search data set as data to be consumed in a preset message queue;

Step 205, extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain a third search data set;

Step 206: Store the third search data set in a preset first database and a second database.

The specific description of the above steps 201 to 206 may refer to the corresponding steps of the search data processing method described in FIG. 1 , and details are not repeated here.

It can be seen that, in this embodiment of the present application, the electronic device receives the first search data set transmitted by the first data source, and receives multiple search service records transmitted by the second data source; extracts multiple search service records from the multiple search service records Search business flows, wherein any one of the multiple search business flows includes at least one search keyword; perform a search keyword extraction operation on the multiple search business flows to obtain multiple search business flows corresponding to the multiple search business flows Search keyword sets; combine the multiple search keyword sets to obtain the second search data set, and asynchronously archive the second search data set and the first search data set as data to be consumed to a preset message queue; Suppose that the data to be consumed is extracted from the message queue and a consumption operation is performed to obtain a third search data set; the third search data set is stored in a preset first database and a second database. In this way, metadata decoupling can be achieved through asynchronous archiving, which is beneficial to reducing the occupancy rate of system physical resources, thereby reducing network interaction delay and improving user experience.

Please refer to FIG. 3. FIG. 3 is a schematic flowchart of another search data processing method provided by an embodiment of the present application, which is applied to an electronic device. The search data processing method includes:

Step 301: Receive a first search data set transmitted by a first data source, and receive multiple search service records transmitted by a second data source;

Step 302, generating a second search data set according to the plurality of search service records, and asynchronously filing the second search data set and the first search data set as data to be consumed in a preset message queue;

Step 303: Acquire multiple listeners corresponding to multiple preset consumer servers, where the multiple listeners are used to monitor data transmission operations of the first data source and the second data source;

Step 304, when the multiple listeners monitor the data transmission operation of the first data source and/or the second data source, start the multiple consumer servers;

Step 305: Obtain a third search data set by performing a consumption operation on the data to be consumed according to the plurality of consumption servers;

Step 306: Store the third search data set in a preset first database and a second database.

For the specific description of the above steps 301 to 306, reference may be made to the corresponding steps of the search data processing method described in FIG. 1, which will not be repeated here.

It can be seen that, in the embodiment of the present application, the electronic device receives the first search data set transmitted by the first data source, and receives multiple search service records transmitted by the second data source; and generates a second search data set according to the multiple search service records. A data set, the second search data set and the first search data set are asynchronously archived to a preset message queue as to-be-consumed data; multiple preset listeners corresponding to multiple consumer servers are obtained, and the multiple listeners are used for Monitor the data transmission operations of the first data source and the second data source; when the plurality of listeners monitor the data transmission operations of the first data source and/or the second data source, start the plurality of consumer servers ; perform a consumption operation on the data to be consumed according to the plurality of consumption servers to obtain a third search data set; store the third search data set in a preset first database and a second database. In this way, the first data source and the second data source can be monitored through the listener, and when data transmission occurs, the consumer server is started to perform a consumption operation on the data to be consumed to obtain a third search data set, thereby realizing the decoupling of metadata of different data sources. , to improve the user experience.

Please refer to FIG. 4. FIG. 4 is a schematic flowchart of another search data processing method provided by an embodiment of the present application, which is applied to an electronic device. The search data processing method includes:

Step 401: Receive a first search data set transmitted by a first data source, and receive multiple search service records transmitted by a second data source;

Step 402, generating a second search data set according to the plurality of search service records, and asynchronously filing the second search data set and the first search data set as data to be consumed in a preset message queue;

Step 403, extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain a third search data set;

Step 404, storing the third search data set to a preset first database and a second database;

Step 405, when receiving the input data of the target user, compare the input data with the third search data set in the first database to obtain a plurality of comparison values;

Step 406, determining that the third search data corresponding to the maximum value among the multiple comparison values is the search result;

Step 407: Send the search result to the mobile terminal corresponding to the target user.

For the specific description of the above steps 401 to 407, reference may be made to the corresponding steps of the search data processing method described in FIG. 1 , which will not be repeated here.

It can be seen that, in the embodiment of the present application, the electronic device receives the first search data set transmitted by the first data source, and receives multiple search service records transmitted by the second data source; and generates a second search data set according to the multiple search service records. A data set, the second search data set and the first search data set are asynchronously archived to a preset message queue as data to be consumed; extract the data to be consumed from the preset message queue and perform a consumption operation to obtain third search data set; store the third search data set in the preset first database and second database; when receiving the input data of the target user, compare the input data with the third search data set in the first database to obtain multiple comparison values; determine the third search data corresponding to the maximum value among the multiple comparison values as the search result; and send the search result to the mobile terminal corresponding to the target user. In this way, a third search data set can be obtained by asynchronously filing the first search data set and the second search data set in the preset message queue and then performing a consumption operation, and storing the third search data set in the first database and the second database, Implementing data decoupling is beneficial to reduce the occupancy rate of system physical resources, thereby reducing network interaction delay, and comparing the input data in the first database when receiving input data can clean search data and improve user experience.

Please refer to FIG. 5. FIG. 5 is a schematic structural diagram of an electronic device 500 provided by an embodiment of the present application. As shown in the figure, the electronic device 500 includes an application processor 510, a memory 520, a communication interface 530, and one or more Program 521, wherein the one or more programs 521 are stored in the aforementioned memory 520 and configured to be executed by the aforementioned application processor 510, the one or more programs 521 including instructions for performing the following steps:

receiving the first search data set transmitted by the first data source, and receiving multiple search service records transmitted by the second data source;

generating a second search data set according to the plurality of search service records, and asynchronously storing the second search data set and the first search data set as data to be consumed in a preset message queue;

Extract the data to be consumed from the preset message queue and perform a consumption operation to obtain a third search data set;

The third search data set is stored in a preset first database and a second database.

It can be seen that, in this embodiment of the present application, the electronic device receives the first search data set transmitted by the first data source, and receives multiple search service records transmitted by the second data source; and generates a second search service record according to the multiple search service records. Searching for a data set, asynchronously storing the second search data set and the first search data set as data to be consumed in a preset message queue; extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain A third search data set; storing the third search data set in a preset first database and a second database. In this way, the third search data set is obtained by asynchronously filing the first search data set and the second search data set in the preset message queue and then performing the consumption operation, and the third search data set is stored in the first database and the second database. Data decoupling is conducive to reducing the occupancy rate of system physical resources, thereby reducing network interaction delay and improving user experience.

In a possible example, the second search data set is generated according to the plurality of search business records, and the instructions in the program are specifically used to perform the following operations: extracting a plurality of search data sets from the plurality of search business records business flow, wherein any one of the multiple search business flows includes at least one search keyword; performing a search keyword extraction operation on the multiple search business flows to obtain the corresponding Multiple search keyword sets; combining the multiple search keyword sets to obtain the second search data set.

In a possible example, the third search data set is obtained by extracting the data to be consumed from the preset message queue and performing a consumption operation, and the instructions in the program are specifically used to perform the following operations: obtaining a preset Multiple listeners corresponding to multiple consumer servers, where the multiple listeners are used to monitor data transmission operations of the first data source and the second data source; when the multiple listeners monitor the first data source and the second data source During the data transmission operation of the data source and/or the second data source, the plurality of consumption servers are started; the third search data set is obtained according to the consumption operation of the data to be consumed by the plurality of consumption servers.

In a possible example, the third search data set is obtained by performing a consumption operation on the data to be consumed according to the plurality of consumption servers, and the instructions in the program are specifically used to perform the following operations: according to a preset The data extraction rule extracts the data to be consumed from the preset message queue; obtains the data format to be consumed of the data to be consumed, and determines whether the preset data format set includes the data format to be consumed; if the The data format set includes the to-be-consumed data format, and the target consumer server corresponding to the to-be-consumed data format is determined according to the preset mapping relationship between the data format and the consumer server and the plurality of consumer servers; Data consumption instruction, sending the data consumption instruction to the target consumption server, wherein the data consumption instruction is used to instruct the target consumption server to perform a consumption operation on the to-be-consumed data; receive the data returned by the target consumption server Data consumption response, extracting consumption data from the data consumption response; generating the third search data set according to the consumption data.

In a possible example, the instructions in the program are further configured to perform the following operations: if the data format set does not include the data format to be consumed, generate a data consumption reminder according to the data format to be consumed, the data The consumption reminder is used to prompt that the consumption server corresponding to the data format to be consumed is missing; the data consumption reminder is sent to the preset mobile terminal.

In a possible example, after the third search data is stored in the preset first database and the second database, the instructions in the program are further used to perform the following operations: when receiving input data from the target user , compare the input data with the third search data set in the first database to obtain multiple comparison values; determine the third search data corresponding to the maximum value in the multiple comparison values is the search result; the search result is sent to the mobile terminal corresponding to the target user.

The foregoing mainly introduces the solutions of the embodiments of the present application from the perspective of the method-side execution process. It can be understood that, in order to realize the above-mentioned functions, the electronic device includes corresponding hardware structures and/or software modules for executing each function. Those skilled in the art should easily realize that the present application can be implemented in hardware or a combination of hardware and computer software with the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In this embodiment of the present application, the electronic device may be divided into functional units according to the foregoing method examples. For example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one control unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical function division, and other division methods may be used in actual implementation.

FIG. 6 is a block diagram of functional units of the search data processing apparatus 600 involved in the embodiment of the present application. The search data processing apparatus 600 is applied to electronic equipment, and the search data processing apparatus 600 includes a receiving unit 601, a generating unit 602, a consuming unit 603 and a storage unit 604, wherein:

A receiving unit 601, configured to receive the first search data set transmitted by the first data source, and receive multiple search service records transmitted by the second data source;

A generating unit 602, configured to generate a second search data set according to the plurality of search service records, and asynchronously archive the second search data set and the first search data set as data to be consumed to a preset message queue;

A consumption unit 603, configured to extract the data to be consumed from the preset message queue and perform a consumption operation to obtain a third search data set;

The storage unit 604 is configured to store the third search data set in a preset first database and a second database.

It can be seen that, in this embodiment of the present application, the electronic device receives the first search data set transmitted by the first data source, and receives multiple search service records transmitted by the second data source; and generates a second search service record according to the multiple search service records. Searching for a data set, asynchronously storing the second search data set and the first search data set as data to be consumed in a preset message queue; extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain A third search data set; storing the third search data set in a preset first database and a second database. In this way, the third search data set is obtained by asynchronously filing the first search data set and the second search data set in the preset message queue and then performing the consumption operation, and the third search data set is stored in the first database and the second database. Data decoupling is conducive to reducing the occupancy rate of system physical resources, thereby reducing network interaction delay and improving user experience.

In a possible example, the generating the second search data set according to the multiple search service records, the generating unit 602 is specifically configured to: extract multiple search service flows from the multiple search service records, Wherein, any one of the plurality of search business flows includes at least one search keyword; the keyword extraction operation is performed on the plurality of search business flows, and multiple search business flows corresponding to the plurality of search business flows are obtained. Search keyword sets; combine the multiple search keyword sets to obtain the second search data set.

In a possible example, the third search data set is obtained by extracting the data to be consumed from the preset message queue and performing a consumption operation, and the consuming unit 603 is specifically configured to: obtain a plurality of preset consumptions Multiple listeners corresponding to the server, the multiple listeners are used to monitor the data transmission operations of the first data source and the second data source; when the multiple listeners monitor the first data source and the second data source /or during the data transmission operation of the second data source, start the multiple consuming servers; and obtain the third search data set according to the multiple consuming servers performing the consuming operation on the data to be consumed.

In a possible example, the third search data set is obtained by performing a consumption operation on the data to be consumed according to the plurality of consumption servers, and the consumption unit 603 is specifically configured to: extract according to a preset data The rule extracts the to-be-consumed data from the preset message queue; obtains the to-be-consumed data format of the to-be-consumed data, and determines whether the preset data format set includes the to-be-consumed data format; if the data format set Including the to-be-consumed data format, determining a target consumer server corresponding to the to-be-consumed data format according to a preset mapping relationship between the data format and the consumer server and the plurality of consumer servers; generating a data consumption instruction according to the to-be-consumed data , send the data consumption instruction to the target consumption server, where the data consumption instruction is used to instruct the target consumption server to perform a consumption operation on the data to be consumed; receive a data consumption response returned by the target consumption server , extracting consumption data from the data consumption response; and generating the third search data set according to the consumption data.

In a possible example, the consuming unit 603 is further configured to: if the data format set does not include the data format to be consumed, generate a data consumption reminder according to the data format to be consumed, and the data consumption reminder uses In order to prompt that the consumption server corresponding to the format of the data to be consumed is missing; send the data consumption reminder to the preset mobile terminal.

In a possible example, after the third search data is stored in the preset first database and the second database, the storage unit 604 is further configured to: when receiving the input data of the target user, store the search data into the preset first database and the second database. The input data is compared with the third search data set in the first database to obtain multiple comparison values; the third search data corresponding to the maximum value among the multiple comparison values is determined as the search result ; send the search result to the mobile terminal corresponding to the target user.

Embodiments of the present application further provide a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes the computer to execute part or all of the steps of any method described in the above method embodiments , the above computer includes electronic equipment.

Embodiments of the present application further provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of the method embodiments described above. some or all of the steps of the method. The computer program product may be a software installation package, and the computer includes an electronic device.

It should be noted that, for the sake of simple description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence. Because in accordance with the present application, certain steps may be performed in other orders or concurrently. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The above-mentioned integrated units, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art, or all or part of the technical solution, and the computer software product is stored in a memory, Several instructions are included to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above-mentioned methods in the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash disk , Read-only memory (English: Read-Only Memory, referred to as: ROM), random access device (English: Random Access Memory, referred to as: RAM), magnetic disk or optical disk, etc.

The embodiments of the present application have been introduced in detail above, and the principles and implementations of the present application are described in this paper by using specific examples. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; at the same time, for Persons of ordinary skill in the art, based on the idea of the present application, will have changes in the specific implementation manner and application scope. In summary, the contents of this specification should not be construed as limitations on the present application.

Claims (10)

1. a search data processing method, is characterized in that, comprises: receiving the first search data set transmitted by the first data source, and receiving multiple search service records transmitted by the second data source; generating a second search data set according to the plurality of search service records, and asynchronously filing the second search data set and the first search data set as data to be consumed in a preset message queue; Extract the data to be consumed from the preset message queue and perform a consumption operation to obtain a third search data set; The third search data set is stored in a preset first database and a second database. 2. The method according to claim 1, wherein the generating a second search data set according to the plurality of search business records comprises: Extracting multiple search business flows from the multiple search business records, wherein any one of the multiple search business flows includes at least one search keyword; performing a search keyword extraction operation on the multiple search business flows, to obtain multiple search keyword sets corresponding to the multiple search business flows; The second search data set is obtained by combining the plurality of search keyword sets. 3. The method according to claim 1, wherein the extracting the data to be consumed from the preset message queue and performing a consumption operation to obtain a third search data set, comprising: Acquiring multiple listeners corresponding to multiple preset consumer servers, where the multiple listeners are used to monitor data transmission operations of the first data source and the second data source; When the multiple listeners monitor the data transmission operation of the first data source and/or the second data source, start the multiple consuming servers; The third search data set is obtained by performing a consumption operation on the data to be consumed according to the plurality of consumption servers. 4. The method according to claim 3, wherein the obtaining the third search data set by performing a consumption operation on the data to be consumed according to the plurality of consumption servers comprises: Extracting the data to be consumed from the preset message queue according to a preset data extraction rule; Obtain the data format to be consumed of the data to be consumed, and determine whether the preset data format set includes the data format to be consumed; If the data format set includes the to-be-consumed data format, determining a target consumer server corresponding to the to-be-consumed data format according to a preset mapping relationship between the data format and the consumer server and the plurality of consumer servers; generating a data consumption instruction according to the data to be consumed, and sending the data consumption instruction to the target consumption server, wherein the data consumption instruction is used to instruct the target consumption server to perform a consumption operation on the to-be-consumed data; receiving a data consumption response returned by the target consumption server, and extracting consumption data from the data consumption response; The third search data set is generated according to the consumption data. 5. The method according to claim 4, wherein the method further comprises: If the data format set does not include the data format to be consumed, a data consumption reminder is generated according to the data format to be consumed, and the data consumption reminder is used to prompt that the consumption server corresponding to the data format to be consumed is missing; Send the data consumption reminder to the preset mobile terminal. 6. The method according to claim 1, wherein after the storing the third search data to the preset first database and the second database, the method further comprises: When receiving the input data of the target user, comparing the input data with the third search data set in the first database to obtain a plurality of comparison values; Determine that the third search data corresponding to the maximum value in the plurality of comparison values is the search result; Send the search result to the mobile terminal corresponding to the target user. 7. A search data processing device, characterized in that, comprising: a receiving unit, configured to receive the first search data set transmitted by the first data source, and receive a plurality of search service records transmitted by the second data source; a generating unit, configured to generate a second search data set according to the plurality of search service records, and asynchronously archive the second search data set and the first search data set as data to be consumed in a preset message queue; a consumption unit, configured to extract the data to be consumed from the preset message queue and perform a consumption operation to obtain a third search data set; A storage unit, configured to store the third search data set in a preset first database and a second database. 8. The apparatus according to claim 7, wherein, in the aspect of generating the second search data set according to the plurality of search service records, the generating unit is specifically configured to: Extracting multiple search business flows from the multiple search business records, wherein any one of the multiple search business flows includes at least one search keyword; performing a search keyword extraction operation on the multiple search business flows, to obtain multiple search keyword sets corresponding to the multiple search business flows; The second search data set is obtained by combining the plurality of search keyword sets. 9. An electronic device, characterized in that it comprises a processor, a memory, a communication device, and one or more programs, wherein the processor, the memory, and the communication device are connected to each other, wherein the communication device is used for information interaction with an external device, so that the The one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing steps in the method of any of claims 1-6. 10. A computer-readable storage medium, characterized by storing a computer program for electronic data exchange, wherein the computer program causes a computer to perform the method according to any one of claims 1-6.

Images