CN116561292A - Data search method, device, electronic device and computer readable medium - Google Patents

Abstract

The invention discloses a data searching method, a data searching device, electronic equipment and a computer readable medium, and relates to the technical field of big data processing. One embodiment of the method comprises the following steps: receiving operation and maintenance data pushed by each operation and maintenance system, or pulling the operation and maintenance data from each operation and maintenance system; adding classification fields and tag fields to the operation and maintenance data, writing the operation and maintenance data and the corresponding classification fields and tag fields into a database, creating a document according to the operation and maintenance data and the corresponding classification fields and tag fields, and updating the document into an index; receiving a search request sent by a searcher, wherein the search request carries a search sentence input by a user, identifying a target field from the search sentence, searching a plurality of matched documents in an index according to the target field, and sequencing the plurality of documents, so that the document with the front sequencing result is returned to the searcher. The implementation mode can solve the technical problem that full-ecological operation and maintenance data cannot be conveniently searched.

Description

Data search method, device, electronic device and computer readable medium

technical field

The present invention relates to the technical field of big data processing, in particular to a data search method, device, electronic equipment and computer readable medium.

Background technique

With the popularization of computer systems and the promotion of intelligent office systems, the scale and frequency of various operation and maintenance operations are also increasing. The various information generated by the operation and maintenance system during the operation process can help analyze the content of the work, solve the problems found, and provide inquiries about relevant operation and maintenance data, which is very valuable for research and preservation, so most of the operation and maintenance The system will record its own operation information for subsequent use.

However, due to the huge gap in function between different operation and maintenance systems and the complexity of operation and maintenance data brought about by the complexity of operations, these operation and maintenance data have not been connected with each other. Even if some systems implement cross-platform data storage, they cannot provide complex search support, and it is difficult to find out the whole ecological operation and maintenance data of a hardware or network in real time according to a certain dimension, which causes operation and maintenance personnel to search for data It is very inconvenient when you have to search multiple systems or conduct multiple searches and then manually splicing to get the desired data.

Contents of the invention

In view of this, the embodiments of the present invention provide a data search method, device, electronic device, and computer-readable medium to solve the technical problem that it is impossible to conveniently search the operation and maintenance data of the whole ecology.

To achieve the above purpose, according to an aspect of the embodiments of the present invention, a data search method is provided, including:

Receive the operation and maintenance data pushed by each operation and maintenance system, or pull the operation and maintenance data from each operation and maintenance system;

Adding classification fields and label fields to the operation and maintenance data, writing the operation and maintenance data and their corresponding classification fields and label fields into the database, creating documents according to the operation and maintenance data and their corresponding classification fields and label fields, and update said document into the index;

receiving a search request sent by a searcher, the search request carrying a search statement input by a user, and identifying a target field from the search statement;

Searching for a plurality of matched documents in the index according to the target field, and sorting the multiple documents, so as to return the documents with higher ranking results to the searcher.

Optionally, writing the operation and maintenance data and its corresponding classification fields and label fields into a database includes:

Generate a tree-shaped directory according to the classification field, and add the operation and maintenance data to the tree-shaped directory;

Writing the tree directory, and the corresponding relationship between the operation and maintenance data and the label field into a database.

Optionally, create a document according to the operation and maintenance data and its corresponding classification field and label field, and update the document into the index, including:

generating an index field according to the operation and maintenance data;

Assembling the index field, the operation and maintenance data and its corresponding classification field and label field into a document, and writing the document into an index creator, thereby updating the document into the index.

Optionally, an index field is generated according to the operation and maintenance data, including:

generating an index field according to the metadata of the operation and maintenance data; and/or,

Word segmentation is performed on the operation and maintenance data to obtain index fields.

Optionally, identifying a target field from the search statement includes:

Preprocessing the search sentence, the preprocessing includes at least one of pinyin conversion processing, completion processing and synonym supplement processing;

extracting keywords and/or label fields from the preprocessed search statement, identifying user intentions according to the keywords and/or the label fields, so as to obtain associated fields;

performing word segmentation processing on the search statement, thereby obtaining word segmentation;

Wherein, the target field includes keyword and/or label field, associated field, and word segmentation.

Optionally, searching out multiple matching documents in the index according to the target field, and sorting the multiple documents includes:

For each document in the index, calculate the correlation score between each target field and each index field, each classification field and label field in the document, and perform a weighted summation of the calculated correlations, Thereby obtaining the BM25 value of the search statement and the document;

Sort each document according to the order of BM25 value from large to small.

In addition, according to another aspect of the embodiments of the present invention, a data search device is provided, including:

The receiving module is used to receive the operation and maintenance data pushed by each operation and maintenance system, or to pull the operation and maintenance data from each operation and maintenance system;

A storage module, configured to add classification fields and label fields to the operation and maintenance data, write the operation and maintenance data and their corresponding classification fields and label fields into a database, and according to the operation and maintenance data and their corresponding classification fields and Create a document with the label field and update said document into the index;

A processing module, configured to receive a search request sent by a searcher, the search request carrying a search sentence input by a user, and identifying a target field from the search sentence;

A search module, configured to find multiple matching documents in the index according to the target field, and sort the multiple documents, so as to return the documents with higher ranking results to the searcher.

Optionally, the storage module is also used for:

Generate a tree-shaped directory according to the classification field, and add the operation and maintenance data to the tree-shaped directory;

Writing the tree directory, and the corresponding relationship between the operation and maintenance data and the label field into a database.

Optionally, the storage module is also used for:

generating an index field according to the operation and maintenance data;

Assembling the index field, the operation and maintenance data and its corresponding classification field and label field into a document, and writing the document into an index creator, thereby updating the document into the index.

Optionally, the storage module is also used for:

generating an index field according to the metadata of the operation and maintenance data; and/or,

Word segmentation is performed on the operation and maintenance data to obtain index fields.

Optionally, the processing module is also used for:

Preprocessing the search sentence, the preprocessing includes at least one of pinyin conversion processing, completion processing and synonym supplement processing;

extracting keywords and/or label fields from the preprocessed search statement, identifying user intentions according to the keywords and/or the label fields, so as to obtain associated fields;

performing word segmentation processing on the search statement, thereby obtaining word segmentation;

Wherein, the target field includes keyword and/or label field, associated field, and word segmentation.

Optionally, the search module is also used for:

For each document in the index, calculate the correlation score between each target field and each index field, each classification field and label field in the document, and perform a weighted summation of the calculated correlations, Thereby obtaining the BM25 value of the search statement and the document;

Sort each document according to the order of BM25 value from large to small.

According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including:

one or more processors;

storage means for storing one or more programs,

When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the foregoing embodiments.

According to another aspect of the embodiments of the present invention, there is also provided a computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, the method described in any of the above-mentioned embodiments is implemented.

According to another aspect of the embodiments of the present invention, there is also provided a computer program product, including a computer program, when the computer program is executed by a processor, the method described in any of the foregoing embodiments is implemented.

An embodiment of the above invention has the following advantages or beneficial effects: because the operation and maintenance data and its corresponding classification fields and label fields are written into the database by adding classification fields and label fields to the operation and maintenance data, according to the operation and maintenance data and its The corresponding classification field and label field create a document, update the document to the index, identify the target field from the search statement, find multiple matching documents in the index according to the target field, and sort the multiple documents means, so it overcomes the technical problem that the existing technology cannot conveniently search for the operation and maintenance data of the whole ecology. The embodiments of the present invention can not only flexibly store a large amount of complex and multi-type data information, but also accurately search for corresponding operation and maintenance data according to different needs, so as to conveniently search for the whole ecological operation and maintenance data.

The further effects of the above-mentioned non-conventional alternatives will be described below in conjunction with specific embodiments.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work. in:

Fig. 1 is a flowchart of a data search method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a system architecture for implementing a data search method according to an embodiment of the present invention;

FIG. 3 is a flow chart of a data search method according to a reference embodiment of the present invention;

FIG. 4 is a flowchart of a data search method according to another reference embodiment of the present invention;

5 is a schematic diagram of a data search device according to an embodiment of the present invention;

FIG. 6 is an exemplary system architecture diagram to which the embodiment of the present invention can be applied;

Fig. 7 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It should be noted that in the technical solution of the present invention, the collection, analysis, use, transmission, storage, etc. of the user's personal information involved are in compliance with the provisions of relevant laws and regulations, and are used for legal and reasonable purposes. Sharing, divulging or selling other than legitimate use, and accepting the supervision and management of regulatory authorities. Necessary measures should be taken for users' personal information to prevent illegal access to such personal information and data, ensure that those who have access to personal information and data abide by relevant laws and regulations, and ensure the security of users' personal information. Risk should be minimized by limiting or even prohibiting data collection and/or deleting data once such user personal data is no longer required.

The embodiment of the present invention mainly realizes two aspects of functions, that is, the collection of complex and multi-type system information and data and the efficient, accurate and multi-dimensional search function based on this, so as to conveniently search for the entire ecological operation and maintenance information.

Fig. 1 is a flowchart of a data search method according to an embodiment of the present invention. As an embodiment of the present invention, as shown in Figure 1, the data search method may include:

Step 101, receiving the operation and maintenance data pushed by each operation and maintenance system, or pulling the operation and maintenance data from each operation and maintenance system.

As shown in Figure 2, the data processing layer service can be built to actively receive the latest operation and maintenance data pushed by various operation and maintenance systems (such as operation and maintenance system A, operation and maintenance system B, operation and maintenance system C, etc.). In addition, as an auxiliary measure, the data processing layer service will regularly scan the data of each connected operation and maintenance system actively, which has achieved the effect of checking for leaks and filling in vacancies and ensuring the integrity of the data.

Sometimes due to network problems or traffic restrictions, the operation and maintenance system cannot guarantee to push the latest operation and maintenance data in real time, so if only relying on the interface to passively receive operation and maintenance data may cause data loss, so the service also needs an active data scanning mechanism. For all the operation and maintenance systems connected to the system, the service can flexibly set a scanning cycle for each system according to the data volume or request frequency, so as to meet the requirement of continuously scanning the operation and maintenance system at a certain frequency. It should be pointed out that the operation and maintenance data scanned actively will be screened, and the information that has been entered into the database will be ignored and will not be repeatedly entered.

Therefore, the operation and maintenance data is obtained in parallel in two ways, active and passive, to ensure the integrity and real-time performance of the access data.

Step 102, adding classification fields and label fields to the operation and maintenance data, writing the operation and maintenance data and their corresponding classification fields and label fields into the database, and according to the operation and maintenance data and their corresponding classification fields and label fields A document is created and said document is updated into the index.

In order to store a large amount of operation and maintenance data in a flexible structure, the embodiment of the present invention uses a Nosql distributed database cluster for data storage. There are two reasons for this choice: the storage data structure is flexible and can be adapted to the operation and maintenance data of various operation and maintenance systems ; A large amount of stored data is convenient for expansion at any time. Therefore, the Nosql distributed database cluster is used to store operation and maintenance data, which not only ensures the storage of a large amount of data, but also protects the original format of the data.

Since the Nosql database is used for data storage, the interface service for receiving operation and maintenance data has no strict restrictions on the transmitted data structure, so that the original operation and maintenance data structure can be preserved in the data storage. While maintaining the original data structure, the data processing layer will add some additional system information for subsequent search and management, such as:

1) Add classification fields: According to the data source, that is, the operation and maintenance data obtained from which operation and maintenance system, add classification fields to the operation and maintenance data, such as the name, number, type, etc. of the operation and maintenance system where it is located, and deploy and run information. For example, information such as network segments, deployment equipment rooms, deployment equipment, and deployment areas, such as names and numbers, facilitate subsequent information searches based on specific classification fields as conditions, such as searching by network segments or deployment equipment rooms.

2) Add label fields: The difference between label fields and classification fields is that classification is a limited number of clusters that reflect the overall information of the operation and maintenance system, while labels are some phrases or keywords that can describe the functional details of the system to the greatest extent. According to the preset tag type, some keywords/keywords are extracted from the operation and maintenance data as tag fields. Tags are very valuable search terms that can greatly increase search accuracy in subsequent searches. It should be pointed out that, according to the tag hit status of the operation and maintenance data, one tag field or multiple tag fields can be added to the operation and maintenance data.

3) Set directory: Generate a tree directory according to the above classification fields, and add operation and maintenance data to the directory information. By using the format of the tree directory, you can manage and find the corresponding operation and maintenance data more conveniently, such as according to the unit, department, computer room, etc., so that you can quickly view the operation and maintenance data under the same information.

Optionally, writing the operation and maintenance data and its corresponding classification fields and label fields into the database includes: generating a tree directory according to the classification fields, adding the operation and maintenance data to the tree directory; The above tree directory, and the corresponding relationship between the operation and maintenance data and the label field is written into the database. In the embodiment of the present invention, after receiving the operation and maintenance data pushed by each operation and maintenance system or pulling the operation and maintenance data from each operation and maintenance system, adding classification fields and label fields for each operation and maintenance data, and then according to the classification fields Generate a tree directory and add the operation and maintenance data to the tree directory. The data processing layer will push the tree directory with the operation and maintenance data and the corresponding relationship between the operation and maintenance data and the label field to the data storage layer for persistent storage, and Follow up.

In order to facilitate the storage and management of a large amount of heterogeneous operation and maintenance data, the embodiment of the present invention uses a NoSql distributed database cluster as the underlying database application of the data storage layer service. In the data storage layer, it also includes the following two parts of business: use the flexible data structure of Nosql to store a large amount of heterogeneous operation and maintenance data information passed by the data processing layer; for subsequent searches, process the stored operation and maintenance data into The search index format data used by the search service is to create documents based on the operation and maintenance data and their corresponding classification fields and label fields, and update the documents to the index.

The data pushed from the data processing layer corresponds to adding or modifying two different processing methods. The data that does not exist in the database is new data, and the data that already exists in the system will modify and update the corresponding fields to ensure the real-time and correctness of the data. Specifically, according to the data type, it is stored in a distributed database cluster. The data type reflects the information category of the system, such as system information of number type or text type, various system calendars of date type, system log and system manual of file type, and so on. The corresponding relationship between the data identification of the operation and maintenance data and the label field can be stored in the database table, which is used to represent the corresponding relationship between the operation and maintenance data and the label field.

Optionally, creating a document according to the operation and maintenance data and its corresponding classification field and label field, and updating the document into the index includes: generating an index field according to the operation and maintenance data; adding the index field, The operation and maintenance data and its corresponding classification fields and label fields are assembled into a document, and the document is written into the index creator, thereby updating the document into the index. As shown in Figure 2, in order to facilitate the use of subsequent data search services, these newly entered data will activate a process of updating the search index to ensure that it will be searched by users in the first time. It mainly includes the following steps: First, generate index fields (Fields) based on the operation and maintenance data, then assemble the classification fields, label fields and index fields of the operation and maintenance data into a document (Document), and finally A document is written to an index writer (IndexWriter), which updates the document into the index. That is to say, organize the operation and maintenance data of multiple operation and maintenance systems, and finally form an index for easy search. After the database and corresponding indexes are updated, the data update process ends. Finally, the data store stores the generated search index data into the database for easy searching.

Optionally, generating an index field according to the operation and maintenance data includes: generating an index field according to metadata of the operation and maintenance data; and/or performing word segmentation processing on the operation and maintenance data to obtain an index field. For each piece of operation and maintenance data, all types of data are extracted to generate index fields one by one. Among them, according to the classification of data and the type of information content, some index fields directly store unprocessed metadata, such as data identification, network type, system deployment region, etc. To search, it needs to be stored after word segmentation, such as system logs, system files, system profiles, etc.

Step 103, receiving a search request sent by a searcher, the search request carrying a search sentence input by a user, and identifying a target field from the search sentence.

For the search request of operation and maintenance data, the search service layer will identify the search statement entered by the user to identify the target field, and then find the operation and maintenance data that meets the user's expectations from the data storage layer according to the identified target field. As shown in Figure 2, for example, input processing such as keyword extraction, pinyin conversion, and intent recognition is performed on the search sentence input by the user, so as to identify target keywords, tag fields, associated fields, word segmentation, and other target fields, so that the search service Can more accurately obtain the user's real search intention.

Step 104 , find multiple matching documents in the index according to the target field, and sort the multiple documents, so as to return the documents with the highest ranking results to the searcher.

As shown in Figure 2, use the target field identified in step 103 to find out qualified documents in the search index of the data storage layer, and for qualified documents, sort each document according to the correlation between the target field and the document, and Return the top-ranked documents to the searcher. Therefore, the embodiment of the present invention can find the operation and maintenance data that meets user expectations from the data storage layer according to multi-dimensional and user requirements.

The embodiment of the present invention collects a large amount of operation and maintenance data, and can provide vertical and horizontal complete data of an operation and maintenance node (such as a server). The longitudinal data of an operation and maintenance node refers to all the operation and maintenance data generated from its birth to its demise, while the horizontal data of operation and maintenance refers to its environment and the content of adjacent nodes. Therefore, the embodiment of the present invention can open up the vertical and horizontal data of the operation and maintenance node, so as to effectively mine the value of the node data.

According to the various embodiments described above, it can be seen that the embodiment of the present invention writes the operation and maintenance data and its corresponding classification fields and label fields into the database by adding classification fields and label fields to the operation and maintenance data, and according to the operation and maintenance data Create a document with its corresponding classification field and label field, and update the document to the index, identify the target field from the search statement, find multiple matching documents in the index according to the target field, and sort the multiple documents The technical means solves the technical problem that the existing technology cannot conveniently search the operation and maintenance data of the whole ecology. The embodiments of the present invention can not only flexibly store a large amount of complex and multi-type data information, but also accurately search for corresponding operation and maintenance data according to different needs, so as to conveniently search for the whole ecological operation and maintenance data.

Fig. 3 is a flowchart of a data search method according to a reference embodiment of the present invention. As another embodiment of the present invention, as shown in FIG. 3, the data search method may include:

Step 301, receiving the operation and maintenance data pushed by each operation and maintenance system, or pulling the operation and maintenance data from each operation and maintenance system.

Step 302, adding classification fields and label fields to the operation and maintenance data, writing the operation and maintenance data and their corresponding classification fields and label fields into the database, and according to the operation and maintenance data and their corresponding classification fields and label fields A document is created and said document is updated into the index.

Step 303, receiving a search request sent by a searcher, the search request carrying a search sentence input by a user.

Step 304, perform preprocessing on the search sentence, the preprocessing includes at least one of pinyin conversion processing, completion processing and synonym supplement processing.

Step 305 , extract keywords and/or label fields from the preprocessed search statement, and identify user intentions according to the keywords and/or the label fields, so as to obtain associated fields.

The embodiment of the present invention adopts Lucene full-text search in the data search, which has higher accuracy and flexibility than the traditional Like search, and can return the results that best meet the user's requirements on the basis of trying to understand the user's intention .

In order to search out the operation and maintenance data that meets the requirements according to the conditions, the search statement entered by the user is first processed to help the user express the search intention more clearly, mainly including the following processing steps:

1) Pinyin conversion processing: This step is two-way. For the non-Chinese characters input by the user, check whether it is the pinyin of the corresponding Chinese word. If it can be found, it will be converted into a Chinese word and added to the search statement; for the input Chinese, it will be converted into Corresponding pinyin to deal with homonyms or typos.

Such as: database -> shujuku

Ceshi -> test

2) Completion processing: according to the phrases extracted from the search index and label fields, assist the user to complete the input of the search sentence. Because the operation and maintenance system is a relatively independent field, there are many proper nouns that need to be memorized, so this function can help users improve the input information more conveniently. For example, when the user enters "Orac", it will complete the word "Oracle". A database noun.

3) Supplementary processing of synonyms: For the input search sentence, it will be inquired in a maintained synonym table whether there are other phrases with similar meanings. If so, it will be added to the search sentence to expand the possibility of search hits. For the application scenario of the operation and maintenance system, some nouns can be grouped into synonyms, such as: software and application, framework and architecture, etc.

4) Keyword extraction: As the operation and maintenance data, there are many professional terms or key sentences, such as IP address, etc., according to the pre-maintained keyword list and IP format, etc., the search keywords can be extracted, and in the subsequent search Searches will be biased based on these keywords.

5) Label check: The input search statement may contain labels, and if the label is hit, the subsequent search will focus on searching for data containing labels. For example, for the search statement "what information systems are deployed in the intranet test environment?", if the system contains "intranet", "test environment", "single-site deployment", and "information system" These tags can be considered to basically meet the search criteria.

6) Intent recognition: According to the extracted keywords and checked tags, it is possible to guess the user's search intention, such as the type of information system searched, the tendency of search, etc. These intentions can help the follow-up process to more accurately find out what matches the user. data needed. For example, when a user searches for "systems with high performance pressure recently", it can judge and understand the intent, and return systems with more network requests and higher data saturation such as CPU and memory.

Step 306, performing word segmentation processing on the search sentence, so as to obtain word segmentation.

Due to the particularity of Chinese ideograms, if you want to have a good search effect, the focus is on the word segmentation of the search sentence and the completion of the recalled data. Since some fields of the index data are word-segmented when they are created, in order to find the data to the greatest extent, it is also necessary to perform word-segmentation on the search statement to match the search content with the stored content.

Even the best tokenizer, due to the extremely high complexity of Chinese word segmentation, it is impossible to perfectly express the user's intention. And sometimes you may search directly based on the sentence without word segmentation, that is, the most suitable result is found in the form of LIKE. Therefore, in addition to word segmentation search, in the case of fewer recall results or lower recall scores, additional means will be used to expand the scope of retrieved data:

1) Word-by-word word segmentation search: split each word of the search statement, and search the word segmentation results in the search index. This form has a good effect when the input word is not long and the recall data is very small. For the application scenario of operation and maintenance information system search, sometimes the search statement may be more complicated, or the original data content contains some mixed information, which will affect the result of word segmentation. At this time, no matter what word segmentation method is used, it is not easy to match For the documents stored in the index, if you disassemble them word by word and search, and return all the matching results, you may be able to find the appropriate result.

2) Wordless search: that is, use the LIKE form of database search, treat the search statement as a complete phrase, and search the index for data containing this phrase to recall. For search sentences with mixed Chinese and English or poor word segmentation, using this form to complete the recall data can often help find the desired results. For example, when searching for the numbers or names of certain functions or components of the information system, these contents are sometimes mixed with Chinese numbers and letters. At this time, it may be more accurate to use a complete search statement without word segmentation.

No matter what form of recalled data is used, the data is not arranged in order at this time. In order to give priority to returning the results that best meet the user's intentions, it needs to be sorted.

Step 307 , find multiple matching documents in the index according to the target field, and sort the multiple documents.

Step 308 , returning the documents with the highest ranking results to the searcher.

In addition, the specific implementation content of the data search method in a reference embodiment of the present invention has been described in detail in the above data search method, so the repeated content will not be described here.

Fig. 4 is a flowchart of a data search method according to another reference embodiment of the present invention. As another embodiment of the present invention, as shown in Figure 4, the data search method may include:

Step 401, receiving the operation and maintenance data pushed by each operation and maintenance system, or pulling the operation and maintenance data from each operation and maintenance system.

Step 402, adding classification fields and label fields to the operation and maintenance data, writing the operation and maintenance data and their corresponding classification fields and label fields into the database, and according to the operation and maintenance data and their corresponding classification fields and label fields A document is created and said document is updated into the index.

Step 403, receiving a search request sent by a searcher, the search request carrying a search sentence input by a user.

Step 404, identifying the target field from the search statement.

Step 405, for each document in the index, calculate the correlation score between each target field and each index field, each classification field and label field in the document, and weight the calculated correlation sum, so as to obtain the BM25 value of the search statement and the document.

Data sorting is to adjust the order of the returned data according to certain rules. The goal is to put the results that are most in line with the search semantics at the head as much as possible, and put the data that is most likely to be selected by the user at the head. The focus is on the sorting dimension employed and the sorting algorithm used. The embodiment of the present invention uses the BM25 algorithm to score the recalled data, and after comparing the scores of the recalled data, there is a related sequence.

Optionally, the label field has the highest weight and is greater than 1. However, there are many types of fields in the information system, and the importance of these contents is not the same. For example, the system name or system introduction is obviously more valuable than the system log or system file when searching. Therefore, when calculating the BM25 value, it is also It is necessary to perform a weighted summation based on the specific sources of the fields matched by the search before sorting. for example:

It can be seen that tags are very important to express the specific content of a system. For the content matched in the tags, a higher weight will be given to ensure that the matched documents are ranked first. In addition, for important fields such as system name or system introduction, the matching results will also be given a positive weight calculation. For systems such as system logs or system files that contain many fields with mixed information, in order to prevent confusing results, the reverse weighting will be used to guide the sorting method appropriately, and the results will be displayed later.

Step 406, sort each document according to the descending order of BM25 values.

Step 407, returning the documents with the highest ranking results to the searcher.

In addition, the specific implementation content of the data search method in another reference embodiment of the present invention has been described in detail in the above data search method, so the repeated content will not be described here.

Fig. 5 is a schematic diagram of a data search device according to an embodiment of the present invention. As shown in FIG. 5 , the data search device 500 includes a receiving module 501, a storage module 502, a processing module 503, and a searching module 504; wherein, the receiving module 501 is used to receive the operation and maintenance data pushed by each operation and maintenance system, or, Pull the operation and maintenance data from each operation and maintenance system; the storage module 502 is used to add classification fields and label fields for the operation and maintenance data, write the operation and maintenance data and its corresponding classification fields and label fields into the database, according to the Create a document based on the operation and maintenance data and its corresponding classification fields and label fields, and update the document into the index; the processing module 503 is used to receive the search request sent by the searcher, and the search request carries the search statement input by the user, A target field is identified from the search statement; the search module 504 is configured to find multiple matching documents in the index according to the target field, and sort the multiple documents, so that the sorting result is placed at the front of documents are returned to the searcher.

Optionally, the storage module 502 is also used for:

Generate a tree-shaped directory according to the classification field, and add the operation and maintenance data to the tree-shaped directory;

Writing the tree directory, and the corresponding relationship between the operation and maintenance data and the label field into a database.

Optionally, the storage module 502 is also used for:

generating an index field according to the operation and maintenance data;

Assembling the index field, the operation and maintenance data and its corresponding classification field and label field into a document, and writing the document into an index creator, thereby updating the document into the index.

Optionally, the storage module 502 is also used for:

generating an index field according to the metadata of the operation and maintenance data; and/or,

Word segmentation is performed on the operation and maintenance data to obtain index fields.

Optionally, the processing module 503 is further configured to:

Preprocessing the search sentence, the preprocessing includes at least one of pinyin conversion processing, completion processing and synonym supplement processing;

extracting keywords and/or label fields from the preprocessed search statement, identifying user intentions according to the keywords and/or the label fields, so as to obtain associated fields;

performing word segmentation processing on the search statement, thereby obtaining word segmentation;

Wherein, the target field includes keyword and/or label field, associated field, and word segmentation.

Optionally, the search module 504 is also used for:

For each document in the index, calculate the correlation score between each target field and each index field, each classification field and label field in the document, and perform a weighted summation of the calculated correlations, Thereby obtaining the BM25 value of the search statement and the document;

Sort each document according to the order of BM25 value from large to small.

It should be noted that the specific implementation content of the data search device of the present invention has been described in detail in the above data search method, so the repeated content will not be described here.

Fig. 6 shows an exemplary system architecture 600 to which the data search method or data search device according to the embodiment of the present invention can be applied.

As shown in FIG. 6 , a system architecture 600 may include terminal devices 601 , 602 , and 603 , a network 604 and a server 605 . The network 604 is used as a medium for providing communication links between the terminal devices 601 , 602 , 603 and the server 605 . Network 604 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.

Users can use terminal devices 601 , 602 , 603 to interact with server 605 via network 604 to receive or send messages and the like. Various communication client applications can be installed on the terminal devices 601, 602, 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (just for example).

The terminal devices 601, 602, 603 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers and the like.

The server 605 may be a server that provides various services, such as a background management server that provides support for shopping websites browsed by users using the terminal devices 601 , 602 , 603 (just an example). The background management server can analyze and process the received data such as item information query requests, and feed back the processing results to the terminal device.

It should be noted that the data search method provided by the embodiment of the present invention is generally executed by the server 605 , and correspondingly, the data search device is generally set in the server 605 .

It should be understood that the numbers of terminal devices, networks and servers in FIG. 6 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

Referring now to FIG. 7 , it shows a schematic structural diagram of a computer system 700 suitable for implementing a terminal device according to an embodiment of the present invention. The terminal device shown in FIG. 7 is only an example, and should not limit the functions and application scope of this embodiment of the present invention.

As shown in FIG. 7 , a computer system 700 includes a central processing unit (CPU) 701 that can operate according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage section 708 into a random-access memory (RAM) 703 Instead, various appropriate actions and processes are performed. In the RAM 703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701 , ROM 702 , and RAM 703 are connected to each other via a bus 704 . An input/output (I/O) interface 705 is also connected to the bus 704 .

The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 708 including a hard disk, etc. and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc. is mounted on the drive 710 as necessary so that a computer program read therefrom is installed into the storage section 708 as necessary.

In particular, according to the disclosed embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the disclosed embodiments of the present invention include a computer program, including a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication portion 709 and/or installed from removable media 711 . When this computer program is executed by a central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are performed.

It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program codes are carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. . Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer programs according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that includes one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a A combination of dedicated hardware and computer instructions.

The modules involved in the embodiments described in the present invention may be implemented by software or by hardware. The described modules can also be set in the processor, for example, it can be described as: a processor includes a receiving module, a storage module, a processing module and a search module, wherein the names of these modules do not constitute a pair of The module itself is defined.

As another aspect, the present invention also provides a computer-readable medium. The computer-readable medium may be contained in the device described in the above embodiments, or it may exist independently without being assembled into the device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by a device, the device implements the following method: receiving the operation and maintenance data pushed by each operation and maintenance system, or receiving the operation and maintenance data from each operation and maintenance system The maintenance system pulls the operation and maintenance data; adds classification fields and label fields to the operation and maintenance data, writes the operation and maintenance data and its corresponding classification fields and label fields into the database, and according to the operation and maintenance data and its corresponding Classification field and label field create a document, and update the document into the index; receive the search request sent by the searcher, the search request carries the search statement entered by the user, and identify the target field from the search statement; according to the Find multiple matching documents in the index by using the target field, and sort the multiple documents, so as to return the documents with higher ranking results to the searcher.

As another aspect, an embodiment of the present invention further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described in any of the foregoing embodiments is implemented.

According to the technical solution of the embodiment of the present invention, since the operation and maintenance data and its corresponding classification fields and label fields are written into the database by adding classification fields and label fields to the operation and maintenance data, according to the operation and maintenance data and its corresponding classification fields and Create a document with a label field, update the document to the index, identify the target field from the search statement, find multiple matching documents in the index according to the target field, and sort the multiple documents, so it overcomes the In the existing technology, it is impossible to conveniently search for the technical problem of the whole ecological operation and maintenance data. The embodiments of the present invention can not only flexibly store a large amount of complex and multi-type data information, but also accurately search for corresponding operation and maintenance data according to different needs, so as to conveniently search for the whole ecological operation and maintenance data.

The above specific implementation methods do not constitute a limitation to the protection scope of the present invention. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (15)

1. A data search method, characterized in that, comprising: Receive the operation and maintenance data pushed by each operation and maintenance system, or pull the operation and maintenance data from each operation and maintenance system; Adding classification fields and label fields to the operation and maintenance data, writing the operation and maintenance data and their corresponding classification fields and label fields into the database, creating documents according to the operation and maintenance data and their corresponding classification fields and label fields, and update said document into the index; receiving a search request sent by a searcher, the search request carrying a search statement input by a user, and identifying a target field from the search statement; Searching for a plurality of matched documents in the index according to the target field, and sorting the multiple documents, so as to return the documents with higher ranking results to the searcher. 2. The method according to claim 1, wherein writing the operation and maintenance data and its corresponding classification fields and label fields into the database includes: Generate a tree-shaped directory according to the classification field, and add the operation and maintenance data to the tree-shaped directory; Writing the tree directory, and the corresponding relationship between the operation and maintenance data and the label field into a database. 3. The method according to claim 1, wherein creating a document according to the operation and maintenance data and its corresponding classification field and label field, and updating the document into the index, comprising: generating an index field according to the operation and maintenance data; Assembling the index field, the operation and maintenance data and its corresponding classification field and label field into a document, and writing the document into an index creator, thereby updating the document into the index. 4. The method according to claim 3, wherein generating an index field according to the operation and maintenance data includes: generating an index field according to the metadata of the operation and maintenance data; and/or, Word segmentation is performed on the operation and maintenance data to obtain index fields. 5. The method of claim 1, wherein identifying a target field from the search statement comprises: Preprocessing the search sentence, the preprocessing includes at least one of pinyin conversion processing, completion processing and synonym supplement processing; extracting keywords and/or label fields from the preprocessed search statement, identifying user intentions according to the keywords and/or the label fields, so as to obtain associated fields; performing word segmentation processing on the search statement, thereby obtaining word segmentation; Wherein, the target field includes keyword and/or label field, associated field, and word segmentation. 6. The method according to claim 5, characterized in that, finding a plurality of matching documents in the index according to the target field, and sorting the plurality of documents, comprising: For each document in the index, calculate the correlation score between each target field and each index field, each classification field and label field in the document, and perform a weighted summation of the calculated correlations, Thereby obtaining the BM25 value of the search statement and the document; Sort each document according to the order of BM25 value from large to small. 7. A data search device, characterized in that it comprises: The receiving module is used to receive the operation and maintenance data pushed by each operation and maintenance system, or to pull the operation and maintenance data from each operation and maintenance system; A storage module, configured to add classification fields and label fields to the operation and maintenance data, write the operation and maintenance data and their corresponding classification fields and label fields into a database, and according to the operation and maintenance data and their corresponding classification fields and Create a document with the label field and update said document into the index; A processing module, configured to receive a search request sent by a searcher, the search request carrying a search sentence input by a user, and identifying a target field from the search sentence; A search module, configured to find multiple matching documents in the index according to the target field, and sort the multiple documents, so as to return the documents with higher ranking results to the searcher. 8. The device according to claim 7, wherein the storage module is also used for: Generate a tree-shaped directory according to the classification field, and add the operation and maintenance data to the tree-shaped directory; Writing the tree directory, and the corresponding relationship between the operation and maintenance data and the label field into a database. 9. The device according to claim 7, wherein the storage module is also used for: generating an index field according to the operation and maintenance data; Assembling the index field, the operation and maintenance data and its corresponding classification field and label field into a document, and writing the document into an index creator, thereby updating the document into the index. 10. The device according to claim 9, wherein the storage module is also used for: generating an index field according to the metadata of the operation and maintenance data; and/or, Word segmentation is performed on the operation and maintenance data to obtain index fields. 11. The device according to claim 7, wherein the processing module is further used for: Preprocessing the search sentence, the preprocessing includes at least one of pinyin conversion processing, completion processing and synonym supplement processing; extracting keywords and/or label fields from the preprocessed search statement, identifying user intentions according to the keywords and/or the label fields, so as to obtain associated fields; performing word segmentation processing on the search statement, thereby obtaining word segmentation; Wherein, the target field includes keyword and/or label field, associated field, and word segmentation. 12. The device according to claim 11, wherein the search module is also used for: For each document in the index, calculate the correlation score between each target field and each index field, each classification field and label field in the document, and perform a weighted summation of the calculated correlations, Thereby obtaining the BM25 value of the search statement and the document; Sort each document according to the order of BM25 value from large to small. 13. An electronic device, characterized in that it comprises: one or more processors; storage means for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-6. 14. A computer-readable medium, on which a computer program is stored, wherein, when the program is executed by a processor, the method according to any one of claims 1-6 is implemented. 15. A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the method according to any one of claims 1-6 is implemented.