CN105528411B - Device and method for full-text retrieval of ship equipment interactive electronic technical manual - Google Patents

Abstract

A full-text search device for ship equipment interactive electronic technical manuals designed by the present invention includes a public source database, a professional vocabulary extraction module, an abbreviation extraction module, a first word segmentation module, a technical information term database, an equipment component name database, an abbreviation Abbreviation database, common vocabulary database, retrieval record database, user retrieval command communication module, retrieval module, second participle module, index database and index module. The present invention integrates the characteristics of element tags and document content in the data module document, uses professional vocabulary to query and increases the weight of professional vocabulary in documents and retrieval keywords, so that the system can query at a certain semantic level, and the returned retrieval results are closer The user's retrieval intention ensures the high recall and precision of the retrieval system.

Description

Device and method for full-text retrieval of ship equipment interactive electronic technical manual

technical field

The invention relates to the technical field of information retrieval, in particular to a full-text retrieval device and method for an interactive electronic technical manual of ship equipment.

technical background

At present, most of the technical data of ship equipment exist in paper form, which leads to increasingly heavy management tasks of technical data, increased data repetition rate and redundancy, and is difficult to update, and the interoperability, real-time transmission and sharing of data are difficult. In order to solve the above problems, an Interactive Electronic Technical Manual (IETM, Interactive Electronic Technical Manual) is usually compiled to manage technical data, that is, compiled according to standard digital format standards, using text, graphics, tables, audio and video, etc., through man-machine Technical publications that provide the basic principles, operation and maintenance support of the equipment in an interactive manner. Due to the large amount of information involved in the IETM system, users usually need to use the information retrieval function to quickly find the required content, and full-text retrieval is one of the most commonly used methods. In the past, most of IETM's full-text retrieval methods used general-purpose retrieval schemes, which did not fully consider the characteristics of technical materials in specialized fields, resulting in unsatisfactory retrieval results.

Full-text search refers to a search method that matches all text of a document with search keywords. Since in the Chinese context, there is no space between words as a separator, and there is no obvious distinguishing mark between words, it is necessary to divide the Chinese string into individual words according to certain specifications, in order to achieve the effect of automatic recognition of the meaning of the sentence by the computer , to complete the matching work between the text in the document and the retrieval keywords, therefore, Chinese word segmentation technology has also become the core technology of Chinese full-text retrieval. Among the currently commonly used word segmentation methods, the word segmentation method based on strings is the most widely used method. This method is to match the strings that need to be segmented with a thesaurus according to a certain strategy to obtain word segmentation results. In the professional field Among them, if there is a lack of professional vocabulary in the thesaurus, the word segmentation method based on strings cannot achieve the ideal word segmentation effect, and the number of professional vocabulary in the thesaurus directly affects the accuracy of word segmentation.

In the field of marine equipment IETM, there are mainly two types of professional vocabulary, one is the names of marine equipment components, such as "SMR-7200 marine radar", "05106 current type propeller anemometer" and so on. The other category is technical information terms, such as "tactical technical indicators", "principle of direction finding", "maintenance envelope diagram" and so on. Therefore, the acquisition of these two types of professional vocabulary is the first problem to be solved by IETM full-text retrieval. Only by using both professional vocabulary and general vocabulary to perform word segmentation and matching on data module (DM, Data Model) documents can users quickly find the desired Equipment technical information.

The structure of the full name of the ship's equipment is complicated, and the name often contains numbers, symbols, letters and other types of characters. Users usually use abbreviations to replace the full name. Usually, "H1604A bulk carrier" or "Ilkos Dignity" is used instead. Therefore, it is not enough to only include the full name of the equipment name in the thesaurus, and the treatment of abbreviations is also an unavoidable problem in word segmentation matching in the field of ship equipment IETM. For equipment names, there are mainly two forms from the original language to the abbreviation: condensation and truncation. Condensation refers to dividing the original language into several parts, and selecting the words or words that best represent the original meaning in each part to form an abbreviation. For example, "H1604A bulk carrier" in the example; truncation refers to obtaining a continuous substring in the original language as an abbreviation, such as "Ilkos Dignity" in the above example.

After solving the problem of professional vocabulary acquisition, the existing word segmentation methods do not match the characteristics of professional vocabulary, and there are certain problems in the word segmentation effect. good matching effect.

After the required information is retrieved, how to sort the various retrieval results is also one of the core problems that the full-text retrieval device and method need to solve. Due to the large number of elements in the data module document, the importance varies, and the importance of different documents There are also differences, and the importance of different query keywords is not the same. Therefore, it is necessary to comprehensively consider the above three factors, design a reasonable search result sorting method, and obtain search results that satisfy users.

From the above content, it can be seen that acquisition of professional vocabulary, acquisition of abbreviations, word segmentation and sorting of retrieval results are the four major problems to be solved by the IETM full-text retrieval device and method for ship equipment.

Contents of the invention

The purpose of the present invention is to provide a full-text search device and method for an interactive electronic technical manual of ship equipment, which can facilitate users to quickly and accurately find required ship equipment technical information.

To achieve this goal, the present invention designs a device for full-text retrieval of interactive electronic technical manuals for marine equipment, which includes databases and functional modules, wherein the databases include public source databases, technical information terminology databases, equipment parts name databases, abbreviations Database, general vocabulary database, search record database and index database, the functional modules include professional vocabulary extraction module, abbreviation extraction module, first word segmentation module, user search command communication module, retrieval module, second word segmentation module and index module, Among them, the public source database provides vocabulary extraction sources for the professional vocabulary extraction module and abbreviation extraction module and provides word segmentation processing content for the first word segmentation module, and the professional vocabulary extraction module is used to extract vocabulary and store it in the technical information terminology database and equipment part name database , the abbreviation extraction module is used to extract vocabulary and store it in the abbreviation database, and the first word segmentation module is used to import the processed word segmentation content into the index module for processing;

The index module is used to build an index and store it in the index database. The index database is used to receive the retrieval content processed by the word segmentation of the second word segmentation module for matching and searching, and return the matched result set to the retrieval module for sorting. The retrieval module is used to sort the user The retrieval content is sent to the second word segmentation module for word segmentation processing. The retrieval module is also used to receive the retrieval command from the user retrieval command communication module and return the sorted result set to the user retrieval command communication module. The user retrieval command communication module is used to The user's search command is sent to the search record database, and the search record database is used to provide a vocabulary extraction source for the abbreviation extraction module;

The technical information term database, equipment part name database, abbreviation database and general vocabulary database respectively provide matching word sets for the first word segmentation module and the second word segmentation module.

A method for searching by using the above-mentioned interactive electronic technical manual full-text retrieval device for ship equipment, which includes the following steps:

Step 1: Import the data module document edited according to the selected interactive electronic technical manual document writing standard (ie S1000D standard) into the public source database, and the professional vocabulary extraction module is based on the selected interactive electronic technical manual document writing standard According to the requirements of the public source database, two types of technical information terminology and equipment part names are extracted, and the mapping relationship with the data module coding information in the corresponding data module file is established, and the above two types of professional vocabulary and the mapping The relationship is stored in the corresponding technical information term database and equipment part name database;

Step 2: The abbreviation extraction module extracts the feature quantity corresponding to the abbreviation from the equipment part name of the public source database, and the feature quantity is the number or common name part in the equipment part name;

Step 3: The abbreviation extraction module matches and searches the above-mentioned feature quantity with the user search records in the data module document and search record database in the public source database, and determines the specific content of each element in the feature quantity in the data module document and user search record. Location;

Step 4: The abbreviation extraction module determines the first and last character strings of the abbreviation where the feature quantity is located, and identifies the boundary segment of the abbreviation corresponding to the feature quantity, so that the recognized abbreviation is a complete abbreviation, and the complete abbreviation is defined as a candidate abbreviation;

Step 5: The abbreviation extraction module calculates the weight of the above candidate abbreviations through the following formula 1:

In the formula, n _mic is the number of times that the candidate abbreviation appears in the specific content, and the specific content includes the same data module document content as the equipment type identification code of the equipment part name and the retrieval keyword in the data module document content retrieval record; n _all is the total number of occurrences of candidate abbreviations in all data module documents and all search records in the search record database; D _all is the sum of the total number of all data module documents and all search records; D _mic is the sum of candidate abbreviations The sum of the total number of data module documents of the abbreviation and the total number of retrieval records containing the candidate abbreviation; W _a is the weight of the candidate abbreviation, which is used to measure the ability of the candidate abbreviation to measure the subject. The threshold of W _a is a given value, when When the weight of _a candidate abbreviation is greater than or equal to the threshold of W _a , the candidate abbreviation can be regarded as a formal abbreviation, and the candidate abbreviation is stored in the abbreviation database. Abbreviations are not processed;

Step 6: In the first word segmentation module and the second word segmentation module, respectively perform word segmentation processing on the user search keywords provided by the data module document and the retrieval module; the specific process of word segmentation processing is as follows:

Assume that the character string to be segmented is S ₁ = w ₁ w ₂ w ₃ ... _wi ... w _n , wherein the character string to be segmented S ₁ is the string of keywords retrieved by the user or each sentence in the data module document Content, w _i is a single character in S ₁ , n is the length of the string, n≥1, i is the character number between 1 and n;

Use the abbreviation database to scan the character string S ₁ to be segmented, and when the abbreviation hits, restore the hit character substring in the character string S ₁ to be segmented to the corresponding primitive until the character string to be segmented Until the scanning of S ₁ is completed, a character string S ₂ = u ₁ u ₂ ... u _i ... u _m is formed at this time, where u _i is a single character in S ₂ , and m is the length of the character string;

In the first word segmentation module and the second word segmentation module, use the string S ₂ to establish a directed acyclic graph G with the number of nodes m+1, and the numbers of the nodes of the directed acyclic graph G are v ₀ , v ₁ , v ₂ …v _m , m is the length of the string, establish a directed edge <v _k ,v _k+1 > between two adjacent vertices v _k , v _k+ 1, the directed edge <v _k ,v _k+1 >The corresponding vocabulary is u _k+1 , (k=0,1,2...m-1, m is the length of the string), if any two directed acyclic graph G nodes exist Directly connected directed edges, it is considered that the distance between these two nodes is 1, if the character substring h ₁ =u _p u _p+1 …u _q of the string S ₂ , (1≤p<q) is in The original language after the abbreviation is restored, the node v _p-1 , v _q is used as the start node and the end node to establish a directed edge <v _p-1 ,v _q >, the directed edge <v _p-1 ,v _q >corresponding vocabulary is character substring h ₁ of character string S ₂ ;

Use the technical information terminology database and the equipment part name database to match the character string S ₂ respectively, if there is a matching maximum word length character substring h ₂ =u _a u _a+1 ... u _b , (1≤a<b), And there is no directed edge <v _a-1 , v _b > between node v _a-1 and node v _b of the maximum word length character substring h ₂ , and a≥p+1 or b≤q-1 holds, Then take the node v _a-1 as the starting node, and take the node v _b as the ending node to establish a directed edge <v _a-1 , v _b >, and the corresponding vocabulary of this edge is the character substring h ₂ of the maximum word length;

Use the general vocabulary database to match the string S ₂ , if there is a matching string h ₃ =u _c u _c+1 … u _d , (1≤c<d), and the node v _c-1 of the string h ₃ There is no directed edge <v _c-1 , v _d > between v _d and v c-1, then the node v _c-1 of the string h ₃ is used as the starting node, and the node v _d of the string h ₃ is used as the terminal node to establish a The directional edge <v _c-1 , v _d >, the corresponding vocabulary of the directional edge <v _c-1 , v _d > is the string h ₃ ; if the node v _c-1 and the node v _d of the string h ₃ exist There is a directed edge <v _c-1 , v _d >, and the string type of the directed edge <v _c-1 , v _d > is the character substring h ₂ of the maximum word length, then it indicates that the character substring h ₂ of the maximum word length It exists in the general vocabulary database, so its type maximum word length character substring h ₂ is changed to character substring h ₄ ;

Count the first N paths from node v 0 to v m in the directed acyclic graph G after the directed edges are generated. The length of the path from node v ₀ to v _m is from short to long. Both the second shortest path and the third shortest path ignore the directed edges whose string types are h ₁ and h ₂ , and only consider the directed edges whose corresponding vocabulary strings are h ₃ and h ₄ , that is, in the non-most In the optimal path, only the matching results of the general lexicon are considered, the repeated directed edges existing in the above three paths are eliminated, and the words corresponding to the remaining directed edges in each path are respectively output, and the resulting result set is the final word segmentation result;

Step 7: In the first word segmentation module, store the final word segmentation results obtained above into each field of the index document in the index database, and set the weight value of each field. Each field of the index document includes the title field, path field, link text field, subtitle field and body field;

Step 8: Set the weight of the index document in the index database, and form multiple index documents into segments and finally form an index file; the index document weight setting is divided into standard coding system coding weight setting and information code weight setting, according to the data module document coding characteristics , to set the weights of different standard coding system codes and information codes. The standard coding system coding weight setting is based on the rule that the lower the standard coding system coding equipment level is, the higher the corresponding weight factor is set. The information code weight setting is based on subcategory information The code sets a higher weight than the main category, and then multiplies the weight of the standard coding system and the weight of the information code to obtain the weight of the index document;

Step 9: Use the retrieval module to provide full-text retrieval service to the user. The retrieval module receives the user's retrieval request and invokes the query method to perform the retrieval. Match the word segmentation content in each field of the document in the formed index library, and find all matching documents as the result set.

The present invention aims at the problems of lack of professional vocabulary and abbreviations, lack of adapted word segmentation algorithm and unoptimized sorting of search results when existing full-text retrieval devices and methods are used in the professional field of ship equipment interactive electronic technical manuals. The selected interactive electronic technical manual document writing standard (i.e. S1000D standard) data module document structure and specific element label characteristics, combined with the types and characteristics of professional vocabulary appearing in the technical data of ship equipment, complete the extraction of professional vocabulary and its abbreviations, and According to the characteristics of multiple types of vocabulary, the word segmentation algorithm is designed in a targeted manner, and the content of the data module document is segmented and stored in the index to facilitate the rapid positioning of information, and the weight values of various factors are set to solve the problem of sorting the retrieval results, and the full text of the interactive electronic technical manual is completed. Construction of retrieval device and method. The full-text retrieval device and method integrates the characteristics of element tags and document content in the data module document, uses professional vocabulary to query and increases the weight of professional vocabulary in the document and retrieval keywords, so that the system can query at a certain semantic level, and the returned The retrieval result is closer to the user's retrieval intention, thereby ensuring a high recall rate and accuracy rate of the retrieval device.

Description of drawings

Fig. 1 is a structural schematic diagram of a full-text retrieval device for an interactive electronic technical manual of ship equipment in the present invention.

Among them, 1—public source database, 2—professional vocabulary extraction module, 3—acronym extraction module, 4—first word segmentation module, 5—technical information term database, 6—equipment part name database, 7—abbreviation database, 8 - general vocabulary database, 9 - search record database, 10 - user search command communication module, 11 - search module, 12 - second word segmentation module, 13 - index database, 14 - index module.

Detailed ways

Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

The ship equipment interactive electronic technical manual full-text retrieval device as shown in Figure 1, it includes database and function module, wherein, said database includes public source database 1, technical information terminology database 5, equipment part name database 6, abbreviation database 7, general vocabulary database 8, retrieval record database 9 and index database 13, described function module comprises professional vocabulary extraction module 2, abbreviation extraction module 3, the first participle module 4, user retrieval order communication module 10, retrieval module 11, Second word segmentation module 12 and index module 14, wherein public source database 1 provides vocabulary extraction source for professional vocabulary extraction module 2 and abbreviation extraction module 3 and provides the content of word segmentation processing for the first word segmentation module 4, professional vocabulary extraction module 2 uses To extract vocabulary and store it in the technical information terminology database 5 and the equipment part name database 6, the abbreviation extraction module 3 is used to extract vocabulary and store it in the abbreviation database 7, and the first word segmentation module 4 is used to import the processed word segmentation content into the index module 14 processing;

The indexing module 14 is used to build an index and store it in the indexing database 13. The indexing database 13 is used to receive the retrieval content processed by the second word segmentation module 12 for word segmentation to perform matching search, and return the result set obtained by matching to the retrieval module 11 for sorting, retrieval Module 11 is used to send the user's retrieval content to the second word segmentation module 12 for word segmentation processing, and retrieval module 11 is also used to receive the retrieval command of the user retrieval command communication module 10 and return the sorted result set to the user retrieval command communication module 10 so as to check, the user's retrieval command communication module 10 is used to send the user's retrieval command to the retrieval record database 9, and the retrieval record database 9 is used to provide the vocabulary extraction source for the abbreviation extraction module 3;

The technical information terminology database 5 , equipment part name database 6 , abbreviation database 7 and common vocabulary database 8 provide matching word sets for the first word segmentation module 4 and the second word segmentation module 12 respectively.

A method for searching by using the above-mentioned interactive electronic technical manual full-text retrieval device for ship equipment, which includes the following steps:

Step 1: Import in the public source database 1 the data module document compiled according to the selected interactive electronic technical manual document (this embodiment is selected as the interactive electronic technical manual document of S1000D), and the professional vocabulary extraction module 2 is based on the selected interactive electronic technical manual document. According to the requirements of the selected interactive electronic technical manual document writing standards, extract two types of technical information terms and equipment part names in the data module document (DM, Data Model) in the public source database 1, and establish a corresponding data module document The mapping relationship between the coding information of the data module in the data module, and store the above-mentioned two types of professional vocabulary and the mapping relationship in the corresponding technical information term database 5 and equipment component name database 6;

Step 2: The abbreviation extraction module 3 extracts the feature quantity corresponding to the abbreviation from the equipment part name (full name) of the public source database 1, and the feature quantity is a digital number or a common name part in the equipment part name (for example, the equipment name primitive " H1604A "Ilkos Dignity" Bulk Carrier", its abbreviation must include the number "1604" and commonly known as "Ilkos Dignity" or one of the two, therefore, such characteristic quantities can be used to locate the possible existence of abbreviations position, and then use other strings of the original language of the equipment name to match the strings before and after the feature, identify the boundary segment of the abbreviation, make the identified abbreviation contain the longest word, calculate the weight of the abbreviation and determine the threshold, Build the mapping relationship between the equipment name primitive and the abbreviation and store it in the abbreviation dictionary to complete the abbreviation extraction);

The specific method for the above-mentioned abbreviation extraction module 3 to extract the corresponding abbreviation feature quantity from the equipment part name (full name) of the public source database 1 includes the following steps: extract the feature quantity of the abbreviation in the original language of the ship equipment name; Ship equipment has fixed naming rules, so the naming rules can be used to determine the type of equipment name and segment the components of the equipment name according to the rules to complete the extraction of feature quantities. The original language of ship equipment name is W ₀ = w ₁ w ₂ …w _n , w _i is the i-th character of the name primitive, first use JAPE (a Java Annotation Patterns Engine) and other grammatical tools to formulate regular expressions for various equipment naming rules, and use these regular expressions to determine step 1 The name type of each W ₀ in the formed equipment part name lexicon, and segment W ₀ according to the matching rules to obtain the abbreviation feature quantity W ₁ = w _p ... w _q , 1≤p<q≤ n;

Step 3: the abbreviation extraction module 3 matches and searches the above-mentioned feature quantity with the data module document in the public source database 1 and the user search record in the search record database 9, and determines that each element in the feature quantity is in the data module document and the user search record The specific position in , the specific method of step ₃ is to set the hit character string as W ₂ , then satisfy W ₂ =W ₁ . Or the equipment type identification code MIC of the access link corresponding to the retrieval record must satisfy the same requirement as the equipment type identification code MIC mapped to the primitive W ₀ corresponding to W ₁ ;

Step 4: The abbreviation extraction module 3 determines the first and last character strings of the abbreviation where the feature quantity is located, and identifies the boundary segment of the abbreviation corresponding to the feature quantity, so that the recognized abbreviation is a complete abbreviation, and the complete abbreviation is defined as a candidate abbreviation (For example, "HMZ-360 radar recognizes the target", in this sentence, "360" is the feature quantity, "HMZ-360 radar" is the longest word of the abbreviation, if only "HMZ-360" or "360 radar" is recognized " are all incomplete identification);

Step 5: The abbreviation extraction module 3 calculates the weight of the above-mentioned candidate abbreviations through the following formula 1:

In the formula, n _mic is the number of times that the candidate abbreviation appears in the specific content, and the specific content includes the same data module document content as the equipment type identification code (MIC, Model identification code) of the equipment part name and the retrieval of the data module document content Retrieval keywords in the record; n _all is the sum of the number of occurrences of candidate abbreviations in all data module documents and all retrieval records in the retrieval record database 9 (the quotient of the two measures the frequency of candidate abbreviations, and the higher the value is High, indicating that the candidate abbreviation appears more times in a specific IETM system; D _all is the sum of the total number of all data module documents and all retrieval records; D _mic is the total number of data module documents containing candidate abbreviations and the total number of candidate abbreviations The sum of the total number of retrieved records of the abbreviations (the logarithm value is used to measure the universality of the candidate abbreviations, the higher the value, it means that the candidate abbreviations are concentrated in a few data module documents); W _a is the weight of the candidate abbreviations, using To measure the ability of candidate abbreviations to measure topics, the threshold value of W _a is a given value, and the threshold value is set to 2. When the weight of candidate abbreviations is greater than or equal to the threshold value of W _a (indicating that it is related to the specific equipment IETM The system subject correlation degree is higher), the candidate abbreviation can be regarded as the official abbreviation, the candidate abbreviation is stored in the abbreviation database 7, when the weight value of the candidate abbreviation is less than the threshold of W _a , the candidate abbreviation is not processed;

Step 6: The first word segmentation module 4 and the second word segmentation module 12 carry out word segmentation processing to the user's retrieval keywords provided by the data module document and the retrieval module 11 respectively, and extract the multiple words formed in the professional vocabulary extraction module 2 and the abbreviation extraction module 3 In the category of vocabulary, there are compound words composed of multiple simple words. These words have multiple correct paths after being segmented through the thesaurus. For example, the equipment name "radar test device" can be further divided into "radar/test/ device", if only a single segmentation result is used for this type of compound vocabulary, a large number of correct matching methods will be discarded, and the result of scoring words cannot meet the needs of user retrieval. The present invention adopts the original N-shortest path segmentation method. On this basis, combined with the characteristics of the generated multi-category professional vocabulary thesaurus and the existing general vocabulary thesaurus, when performing word segmentation, a total of three thesaurus matching processes are carried out. First, the abbreviation thesaurus obtained in step 2 is used for matching , scan the abbreviations in the technical information, and restore them to the corresponding equipment part name primitives; secondly, match the missing text content through the technical information term lexicon obtained in step 1 and the equipment part name lexicon; and then pass the general The thesaurus matches all the text content after restoring the original language; when the matching is completed, output N paths that meet the requirements, and the result set composed of multiple paths is the final word segmentation result. The specific process of word segmentation processing is as follows:

Assume that the character string to be segmented is S ₁ = w ₁ w ₂ w ₃ ... _wi ... w _n , wherein the character string to be segmented S ₁ is the string of keywords retrieved by the user or each sentence in the data module document Content, w _i is a single character in S ₁ , n is the length of the string, n≥1, i is the character number between 1 and n;

Use the abbreviation database ₇ to scan the character string S1 to be segmented, and when the abbreviation hits, restore the hit character _substring in the character string S1 to be segmented to the corresponding primitive until the character to be segmented Until the string S ₁ is scanned, a character string S ₂ = u ₁ u ₂ ... u _i ... u _m is formed at this time, where u _i is a single character in S ₂ , and m is the length of the string;

In the first word segmentation module 4 and the second word segmentation module 12, use the string S ₂ to establish a directed acyclic graph G with the number of nodes m+1, and the numbers of the nodes of the directed acyclic graph G are v ₀ , v ₁ in turn , v ₂ ...v _m , m is the length of the string, and a directed edge <v _k ,v _k+1 > is established between two adjacent vertices v _k and v _k+ 1, the directed edge <v _k , v _k+1 > the corresponding vocabulary is u _k+1 , (k=0,1,2...m-1, m is the length of the string), if any two directed acyclic graph G nodes If there is a directed edge directly connected between the two nodes, the distance between these two nodes is considered to be 1, if the character substring h ₁ of the string S ₂ =u _p u _p+1 …u _q , (1≤p<q) is the original language after the abbreviation is restored, and the node v _p-1 , v _q is used as the starting node and the ending node to establish a directed edge <v _p-1 , v _q >, and the directed edge <v _p-1 , v _q > the corresponding vocabulary is character substring h ₁ of character string S ₂ ;

Use the technical information terminology database 5 and the equipment part name database 6 to match the character string S ₂ , if there is a matching maximum word length character substring h ₂ =u _a u _a+1 ... u _b , (1≤a<b ), and there is no directed edge <v _a-1 , v _b > between node v _a-1 and node v _b of the maximum word length character substring h ₂ , and there is a≥p+1 or b≤q-1 If it is established, the node v _a-1 is used as the starting node, and the node v _b is used as the ending node to establish a directed edge <v _a-1 , v _b >, and the corresponding vocabulary of this edge is the maximum word length character substring h ₂ ;

Use the general vocabulary database 8 to match the string S ₂ , if there is a matching string h ₃ = u _c u _{c + 1} ... u _d , (1≤c<d), and the node v _c- of the string h ₃ If there is no directed edge <v _c-1 ,v _d > between ₁ and v _d , then the node v _c-1 of the string h ₃ is used as the starting node, and the node v _d of the string h ₃ is established as the ending node There is a directed edge <v _c-1 , v _d >, and the corresponding vocabulary of the directed edge <v _c-1 , v _d > is the string h ₃ ; if the node v _c-1 and the node v _d of the string h ₃ There is a directed edge <v _c-1 , v _d >, and the string type of the directed edge <v _c-1 , v _d > is the maximum word length character substring h ₂ , then the maximum word length character substring h ₂ exists in the general vocabulary database 8, so its type maximum word length character substring h ₂ is changed to character substring h ₄ to facilitate subsequent output processing;

Count the first N paths from node v 0 to v m in the directed acyclic graph G after the directed edges are generated. The length of the path from node v ₀ to v _m is from short to long. Both the second shortest path and the third shortest path ignore the directed edges whose string types are h ₁ and h ₂ , and only consider the directed edges whose corresponding vocabulary strings are h ₃ and h ₄ , that is, in the non-most In the optimal path, only the matching results of the general lexicon are considered (to prevent the above N-shortest path word segmentation method from three segmentations that cannot meet the retrieval requirements, and to avoid the situation where the N value is too large to achieve a better segmentation granularity), and the above-mentioned For the repeated directed edges existing in the three paths, respectively output the vocabulary corresponding to the remaining directed edges in each path, and the resulting set is the final word segmentation result;

Step 7: In the first word segmentation module 4, the final word segmentation results obtained above are stored in each field of the index document in the index database 13 respectively, and the weight value of each field is set to provide parameters for the sorting of the final retrieval results, multiple The document forms a segment and finally forms an index file, which is stored in the disk or memory. The fields of the index document include the title field, the path field, the link text field, the subtitle field and the text field;

Step 8: Set the weight of the index document in the index database 13, and form a plurality of index documents into segments and finally form an index file, and store it in disk or memory; the index document weight setting is divided into standard numbering systems (StandardNumbering Systems, SNS) Coding weight setting and information code weight setting, according to the data module document coding characteristics, set the weight of different standard coding system coding and information code, the standard coding system coding weight setting is based on the standard coding system The lower the level of coding equipment, the corresponding weight The higher the factor is set, the information code weight is set according to the rule that the subcategory information code sets a higher weight than the main category, and then the weight of the index document is obtained by multiplying the standard coding system coding weight and the information code weight;

Step 9: Utilize the retrieval module 11 to provide full-text retrieval service to the user. The retrieval module 11 receives the user's retrieval request and invokes the query method to perform the retrieval. The word segmentation content of each field of the document in the index database formed in step 7 is matched, and all matching documents are found as the result set.

In step 7 of the above technical solution, the basis for setting each field of the index document and the corresponding weight value is as follows:

The title field stores the word segmentation result of the data module name <dmtitle>, and the entries appearing in the title field reflect the theme of the entire data module document, and the weight of the title field is set to 10;

The path field is used to identify the document access path, and stores the data module encoding information to realize the path identification function. The path field does not participate in the word segmentation and retrieval process, and the path field does not need to set the weight;

The link text field is used to store the participle results of the data module encoding link restoration text content (same as in the webpage, there are links in the data module content, and the links appear in the form of data module encoding, users can click on the link to access other data modules, in step 1 Here, the mapping between the data module code and the vocabulary is formed, here is the result of using this mapping relationship to restore the code to the vocabulary content and then segment the word), it is also used to realize the retrieval of the link anchor text, when the search keyword is in the link text When the field hits, the data module document pointed to by the link may be what the user is looking for, and the weight of the link text field is set to 3;

The subtitle field is used to store word segmentation results reflecting the local topic information <title> (the label of the local topic, which stores the content of the local topic), and the weight of the subtitle field is set to 5;

The text field is used to store word segmentation results of other technical information in the data module document (other technical information is the content of the text excluding subtitles and link information), and the weight of the text field is set to 1.

Step 1 of the above-mentioned technical solution specifically includes the following steps:

Step 101: Select specific text content to extract two types of professional vocabulary, equipment part name and technical information term, wherein specific elements include technical name <techname> and information name <infoname>, in the data module name, technical name <techname> is used for description The name of the equipment part and the information name <infoname> are used to describe technical information terms, so the text information of these two types of elements is extracted to complete the extraction of professional vocabulary;

Step 102: Establish a mapping relationship between professional vocabulary and corresponding data module code (Data Model Code, DMC) information, wherein the mapping relationship refers to the standard numbering system (Standard Numbering Systems, SNS) and equipment component names, information code <incode >The mapping relationship with technical information terms, link access information is an important resource in the retrieval process, but because the link reference of the data module document does not provide the anchor text information, but realizes it by referencing the data module code, so it needs to be The data module code information can only be entered into the search scope after it is restored to text. The sub-element quasi-code system SNS of the data module code is used to describe the hierarchical position of the component described in the current data module document in the entire equipment, so it can be described with the technical name <techname> The name of the equipment part forms a mapping relationship, so that the retrieval of the coding system SNS is completed by using the name of the equipment part, and the mapping relationship between the sub-element information code <incode> of the data module code DMC and the information name <infoname> is established, and the technical information term is used to complete Retrieval of information codes Because in different ship equipment interactive electronic technical manual IETM systems, the codes corresponding to the same technical information or equipment part names may be different, in order to prevent such mapping inconsistencies, the corresponding information codes and codes Add the corresponding equipment type identification code (Model identification code, MIC) to the SNS code of the system. The MIC code plays the role of defining the name and model of the equipment, and is the only code to determine the equipment formulated by the authoritative organization;

Step 103: Store the extracted vocabulary and corresponding coding information into the equipment part name lexicon and the technical information term lexicon respectively, wherein the equipment part name lexicon is used to store equipment names or parts names and corresponding coding system SNS coding information, The technical information terminology database is used to store technical information terms and corresponding information code encoding information.

In step 4 of the above technical solution, since the abbreviation of ship equipment appears in two forms of condensation and truncation, the character strings appearing in the abbreviation must be the characters in the original language (that is, the full name corresponding to the "abbreviation"), and It is satisfied that the sequence of characters in the abbreviation remains unchanged relative to the original language; read in a character on the left or right side of W ₂ , set the candidate character as w _c , and determine whether w _c exists in W ₀ and satisfies the requirement of W ₂ The arrangement order of W ₀ does not change. If the condition is satisfied, w _c is determined as the boundary character of the candidate abbreviation, and W ₂ is equal to w _c W ₂ or W ₂ w _c . If the condition is not satisfied, w _c is not is the character in the abbreviation, the character determination in the current direction is terminated, the boundary is determined, and the above process is repeated until the character boundary determination in the two directions is all terminated. At this time, W ₂ is the final candidate abbreviation.

In step 7 of the above technical solution, the index is used to quickly locate the required text information, thereby avoiding a large number of read and write operations in the retrieval process, and the index uses a specific data structure to quickly locate the entry. Based on the retrieval toolkit Lucene, an index structure suitable for the IETM full-text retrieval device and method is designed. The index structure in Lucene is divided into five levels from high to low: index, segment, document, field and entry. The entry is the basic unit of the index, storing each word-segmented character string; the domain is used to contain different information in a single document, such as title, body, and link, and the domain is a structure that users can design by themselves, so as to realize Retrieval to different types of documents; document is the basic unit of indexing. In the present invention, an index document stores the information processed by a data module document; a segment is composed of multiple documents, which can be regarded as a small index, and multiple segments Finally constitute the index.

In step 8 of the above-mentioned technical solution, the standard numbering system (Standard Numbering Systems, SNS) coding weight is determined according to the equipment part level represented by the standard numbering system, and the number of the SNS code describes the equipment level where the equipment part is in the current data module, and the SNS Code 00-00-00, 0a-00-00, 0a-b0-00, 0a-bd-00 and 0a-bd-fg, (a≠0, b≠0, d≠0, f≠0∪g≠ 0) respectively describe the equipment components at the equipment level, system level, subsystem level, sub-subsystem level and lower equipment division level in the equipment hierarchy. When the search keyword hits the document, the data module with a higher SNS code level Documents may have only part of the content linked to the information required by users. On the contrary, data module documents with lower levels of SNS codes reflect that the information required by users accounts for a higher proportion of document content. Therefore, the lower the level of SNS code equipment, the weight of the corresponding documents The higher the factor is set, the SNS code weights of equipment level, system level, subsystem level, sub-subsystem level and lower equipment division level are set to 1, 2, 3, 4 and 5 respectively;

The weight of the information code is determined according to the size of the information category described by the information code. The information codes a00 and abc, (b≠0, c≠0) respectively describe the major category and subcategory of technical information. When the retrieval keyword hits the document, the granularity The smaller information code level is more likely to be associated with the content required by the user. Therefore, the subcategory information code is set with a higher weight than the large category. The present invention sets the large category weight value to 1, and the subcategory weight value to 2 .

In step 9 of the above technical solution, the sorting of the result set is calculated according to the vector space model (VSM, Vector SpaceModel), and the specific formula is as follows:

Let the document in the index be d, the user's search keyword be q, and the result of q after word segmentation is t ₁ /t ₂ /.../t _n (S _dt is the result of i from 1 to n, including t _n ) , where n is the total number of entries after segmentation, t _i is a single keyword entry, n≥1, i is the character number between 1 and n, and S _qd represents the matching retrieval keyword q in the index document d Score, which is the result sorting factor, the higher the value, the higher the ranking of documents in the result set, coord(q,d) is used to measure the number of unique entries in the index document d, by calculating the existence of unique entries in the index document d The quotient of the number of entries Num _dt and the number of non-repeated entries Num _qt in the search keyword q is obtained. querytnorm(q) is an adjustment factor, which does not affect the scoring and sorting results. This value can be set to adjust the size of the score as a whole. S _dt represents the sum of the scores of all single keyword entries t _i hit in the index document d, tf(t _i ,d) represents the word frequency score of a single keyword entry t _i appearing in the index document d, idf(t _i ) Indicates how many documents a single keyword entry t _i has appeared in. The higher the value, the fewer documents t _i appears in, and the greater the relevance of a single keyword entry t _i to a specific topic. Boost _ti is a single keyword The weight of entry t _i is determined according to the matching lexicon of single keyword entry t _i during word segmentation, norm(t,d) is the weight and length factor summary value of index document d, where Boost _d is the weight of index document d, the The value is determined according to the weight setting of each field of the index document of the index module described in step 7. Boost _f is the weight of the field that hits a single keyword entry t _i in the index document d, and the value is determined according to the index module described in step 7. The weight setting of each field of the index document is determined. Num _term is the total number of segmentation terms in the index document d. The larger the value, the lower the norm(t,d) score;

The weight of the search keyword entry is determined according to the matching thesaurus type during word segmentation, and the setting basis is as follows:

(1) The hit entries from the abbreviation lexicon, technical information term lexicon, and equipment part name lexicon reflect the user's retrieval intention, and the weight value is set to 5.

(2) The matching thesaurus in the general thesaurus reflects the user's one-sided retrieval intention, and the weight value is set to 2.

(3) The granularity of word segmentation in the word segmentation process is too fine, resulting in too much noise data during retrieval, and the weight value is set to 1.

After the sorting is completed, the retrieval module outputs the sorting results of the result set in a certain form, and the returned result page has ten retrieval results per page, and each result outputs the information segment where the hit entry is located and highlights the hit entry in red, and Given the title and data module code (Data Model Code, DMC) information of the hit document, the user can click on the hyperlink of the title to access the original data module document.

The content not described in detail in this specification belongs to the prior art known to those skilled in the art.

Claims (7)

1. A ship equipment interactive electronic technical manual full-text search device is characterized in that: it includes a database and a functional module, wherein the database includes a public source database (1), a technical information terminology database (5), an equipment component name Database (6), abbreviation database (7), common vocabulary database (8), retrieval record database (9) and index database (13), described function module comprises professional vocabulary extraction module (2), abbreviation extraction module ( 3), the first word segmentation module (4), the user retrieval command communication module (10), the retrieval module (11), the second word segmentation module (12) and the index module (14), wherein the public source database (1) is a professional vocabulary The extraction module (2) and the abbreviation extraction module (3) provide vocabulary extraction sources and provide word segmentation processing content for the first word segmentation module (4), and the professional vocabulary extraction module (2) is used to extract vocabulary and store it in the technical information terminology database (5) and equipment parts name database (6), the abbreviation extracting module (3) is used for extracting vocabulary and storing in the abbreviation database (7), and the first word segmentation module (4) is used for importing the word segmentation content after processing into the index module (14) Processing; Indexing module (14) is used for setting up index and is stored in indexing database (13), and indexing database (13) is used for receiving the retrieval content that the second participle module (12) participle handles to carry out matching search, and the result set that matching obtains is returned Retrieval module (11) sorts, and retrieval module (11) is used for sending the retrieval content of user to the second word segmentation module (12) and carries out word segmentation process, and retrieval module (11) is also used for receiving user's retrieval order communication module (10 ) of the retrieval command and the returned sorted result set are sent to the user retrieval command communication module (10), and the user retrieval command communication module (10) is used to send the user's retrieval command to the retrieval record database (9), and the retrieval record database ( 9) It is used to provide a vocabulary extraction source for the abbreviation extraction module (3); The technical information term database (5), the equipment parts name database (6), the abbreviation database (7) and the common vocabulary database (8) respectively provide word segmentation for the first word segmentation module (4) and the second word segmentation module (12) matching word set. 2. a method utilizing the full-text retrieval device of the interactive electronic technical manual of the ship equipment described in claim 1 to retrieve, is characterized in that, it comprises the steps: Step 1: Import the data module document edited according to the selected interactive electronic technical manual document writing standard in the public source database (1), and the professional vocabulary extraction module (2) is written according to the selected interactive electronic technical manual document The standard requires extracting two types of technical information terms and equipment part names in the data module documents in the public source database (1), and establishing a mapping relationship with the data module coding information in the corresponding data module documents, and combining the above two categories The professional vocabulary and mapping relationship are stored in the corresponding technical information terminology database (5) and equipment part name database (6); Step 2: the abbreviation extraction module (3) extracts the feature quantity corresponding to the abbreviation from the equipment part name of the public source database (1), and the feature quantity is a digital number or a common name part in the equipment part name; Step 3: the abbreviation extraction module (3) matches the above-mentioned feature quantity with the user search record in the data module document in the public source database (1) and the search record database (9), and determines that each element in the feature quantity is included in the data Specific locations in the module documentation and user search records; Step 4: The abbreviation extraction module (3) determines the first and last character strings of the abbreviation where the feature quantity is located, and identifies the boundary segment of the abbreviation corresponding to the feature quantity, so that the recognized abbreviation is a complete abbreviation, and the complete abbreviation is determined as a candidate Abbreviations; Step 5: The abbreviation extraction module (3) calculates the weight of the above-mentioned candidate abbreviations by the following formula 1: In the formula, n _mic is the number of times that the candidate abbreviation appears in the specific content, and the specific content includes the same data module document content as the equipment type identification code of the equipment part name and the retrieval keyword in the data module document content retrieval record; n _all is the total number of occurrences of candidate abbreviations in all data module documents and all retrieval records in the retrieval record database (9); D _all is the sum of the total number of all data module documents and all retrieval records; D _mic is the sum of the total number of data module documents containing candidate abbreviations and the total number of retrieval records containing candidate abbreviations; W _a is the weight of candidate abbreviations, which is used to measure the ability of candidate abbreviations to measure the subject, and the threshold of W _a is given value, when the weight of the candidate abbreviation is greater than or equal to the threshold of W _a , the candidate abbreviation can be regarded as a formal abbreviation, and the candidate abbreviation is stored in the abbreviation database (7), and the weight of the candidate abbreviation is less than W _a When the threshold is reached, the candidate abbreviations are not processed; Step 6: Carry out word segmentation processing to the user's retrieval keywords that data module document and retrieval module (11) provide respectively in the first word segmentation module (4) and the second word segmentation module (12); The specific process of word segmentation processing is: Assume that the character string to be segmented is S ₁ = w ₁ w ₂ w ₃ ... _wi ... w _n , wherein the character string to be segmented S ₁ is the string of keywords retrieved by the user or each sentence in the data module document Content, w _i is a single character in S ₁ , n is the length of the string, n≥1, i is the character number between 1 and n; Use the abbreviation database (7) to scan the character string S ₁ to be segmented, and when the abbreviation hits, restore the character substring hit in the character string S ₁ to be segmented to the corresponding primitive until the segment to be segmented Until the character string S ₁ of is scanned, a character string S ₂ = u ₁ u ₂ ... u _i ... u _m is formed at this time, where u _i is a single character in S ₂ , and m is the length of the character string; In the first word segmentation module (4) and the second word segmentation module (12), set up a directed acyclic graph G whose number of nodes is m+ ₁ with character string S2, and the numbering of the directed acyclic graph G node is v successively ₀ , v ₁ , v ₂ ... v _m , m is the length of the string, and a directed edge <v _k , v _k+1 > is established between two adjacent vertices v _k , v _k+ 1, the directed The vocabulary corresponding to edge <v _k , v _k+1 > is u _k+1 , k=0,1,2...m-1, m is the length of the string, if any two directed acyclic graphs If there is a directed edge directly connected between G nodes, the distance between these two nodes is considered to be 1. If the character substring h ₁ =u _p u _p+1 ...u _q of the string S ₂ is after the reduction of the abbreviations , where 1≤p<q, then a directed edge <v _p-1 ,v _q > is established with nodes v _p-1 and v _q as the start node and end node, and the directed edge <v _{p- 1} , v _q > the corresponding vocabulary is character substring h ₁ of character string S ₂ ; Use the technical information terminology database (5) and the equipment part name database (6) to match the character string S ₂ , if there is a matching maximum word length character substring h ₂ =u _a u _a+1 ... u _b , 1≤ a<b, and there is no directed edge <v _a-1 , v _b > between node v _a-1 and node v _b of the maximum word length character substring h ₂ , and a≥p+1 or b≤q If -1 is established, a directed edge <v _a-1 , v _b > is established with node v _a-1 as the starting node and node v _b as the ending node, and the corresponding vocabulary of this edge is the character substring h ₂ of maximum word length ; Use the general vocabulary database (8) to match the string S ₂ , if there is a matching string h ₃ =u _c u _c+1 ... u _d , 1≤c<d, and the node v _c- of the string h ₃ If there is no directed edge <v _c-1 ,v _d > between ₁ and v _d , then the node v _c-1 of the string h ₃ is used as the starting node, and the node v _d of the string h ₃ is established as the ending node There is a directed edge <v _c-1 , v _d >, and the corresponding vocabulary of the directed edge <v _c-1 , v _d > is the string h ₃ ; if the node v _c-1 and the node v _d of the string h ₃ There is a directed edge <v _c-1 , v _d >, and the string type of the directed edge <v _c-1 , v _d > is the maximum word length character substring h ₂ , then the maximum word length character substring h ₂ exists in the general vocabulary database (8), so its type maximum word length character substring h ₂ is changed into character substring h ₄ ; Count the first N paths from node v 0 to v m in the directed acyclic graph G after the directed edges are generated. The length of the path from node v ₀ to v _m is from short to long. Both the second shortest path and the third shortest path ignore the directed edges whose string types are h ₁ and h ₂ , and only consider the directed edges whose corresponding vocabulary strings are h ₃ and h ₄ , that is, in the non-most In the optimal path, only the matching results of the general lexicon are considered, the repeated directed edges existing in the above three paths are eliminated, and the words corresponding to the remaining directed edges in each path are respectively output, and the resulting result set is the final word segmentation result; Step 7: In the first word segmentation module (4), the final word segmentation results obtained above are respectively stored in each field of the index document in the index database (13), and the weight value of each field is set, and each field of the index document includes the title field , path field, link text field, subtitle field, and body text field; Step 8: set the weight of the index document in the index database (13), and form a plurality of index documents into segments and finally form an index file; the index document weight setting is divided into standard coding system coding weight setting and information code weight setting, according to the data module Document coding features, set the weights of different standard coding system codes and information codes, standard code system code weight settings are based on the rule that the lower the level of standard code system coding equipment, the higher the corresponding weight factor setting, information code weight settings are based on The subcategory information code sets a rule with a higher weight than the main category, and then multiplies the standard coding system coding weight and the information code weight to obtain the weight of the index document; Step 9: Utilize the retrieval module (11) to provide full-text retrieval service to the user. The retrieval module (11) receives the user's retrieval request and invokes the query method for retrieval. The query method is specifically: call the keywords retrieved by the user in step 6 to perform word segmentation After processing, match the word segmentation content of each domain in the document in the index library formed in step 7, and find all matching documents as the result set. 3. The retrieval method according to claim 2, characterized in that: in said step 7, the title field stores the word segmentation result of the data module document name, and the entry that appears in the title field reflects the theme of the entire data module document, and the title The weight of the domain is set to 10. 4. The retrieval method according to claim 2, characterized in that: in said step 7, the path field is used to identify the document access path, and stores the data module document encoding information to realize the function of identifying the path, and the path field does not participate in word segmentation and In the retrieval process, the path domain does not need to set weight. 5. The retrieval method according to claim 2, characterized in that: in said step 7, the link text domain is used to store the word segmentation results of the data module document code link restoration text content, and is also used to realize the retrieval of the link anchor text , when the search keyword hits in the link text field, the data module document pointed to by the link may be what the user is looking for, and the weight of the link text field is set to 3. 6. The retrieval method according to claim 2, characterized in that: in the step 7, the subtitle field is used to store word segmentation results of tags reflecting local topic information, and the weight of the subtitle field is set to 5. 7. The retrieval method according to claim 2, characterized in that: in the step 7, the text field is used to store word segmentation results of other technical information in the data module document, and the weight of the text field is set to 1.