JP3554459B2 - Text data registration search method - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a text data registration / search method for searching registered text data, and more particularly, to a desired document by searching a document database for the entire text of a document by designating a search word that is a predetermined character string. The present invention relates to a technique that is effective when applied to a text data registration and retrieval method to be obtained.
[0002]
[Prior art]
Conventionally, various document search methods for searching a document database storing a large number of documents have been proposed. Japanese Patent Application Laid-Open No. 8-190571 discloses a document search method for efficiently performing a full-text search using connected character information, reducing search noise, reducing processing time, and reducing disk usage. It is described in.
[0003]
The outline is described with reference to a connected character storing step of storing connected characters describing a co-occurrence relationship of a plurality of characters in text data of a document in a connected character file without duplication, and referring to the connected characters stored in the connected character file. In a document search method in which a document including a concatenated character included in a search term in a designated conditional expression is a candidate for a search result, as the concatenated character storage step, the type of a concatenated character component appearing in text data and each concatenated character The number of documents in which the component appears is calculated. If the calculated number of documents is larger than a predetermined threshold, the position corresponding to the document number of the corresponding document is registered as a bit string of “1”. Stores the document number of the document as binary data.
[0004]
In this method, the document numbers of all the connected character components in the document are registered in the connected character component table, and at the time of retrieval, a document including all the connected character components included in the search word is searched for by referring to the connected character component table. The connected character component table used here lists, for each connected character component, an identifier of a document in which the connected character component appears, that is, a document number.
[0005]
[Problems to be solved by the invention]
By the way, in the conventional document search method, the document numbers of all the connected character components in the document are registered in the connected character component table. However, in a database storing many documents, the number of the connected characters appearing in those documents is determined. Will be huge, and the registration will require file access for the huge number of connected character types. Therefore, when a document is registered in a large-scale database, or when a registered document is updated or deleted, a very long processing time is required.
[0006]
For example, when deleting a specific document from the database, it is necessary to delete all the corresponding document numbers for all the character components registered in the character component file. However, in a large-scale database, the capacity of the character component file may be in the GByte order, and such a large-scale database update process online is impossible.
[0007]
An object of the present invention is to provide a technique capable of further improving transaction processing performance.
[0008]
The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.
[0009]
[Means for Solving the Problems]
The outline of typical inventions among the inventions disclosed by the present application will be briefly described as follows.
[0010]
That is, in a text data registration / search method for searching for a document including a specified search word with reference to the appearance relation of characters in the document, the appearance relation of characters of the text data requested to be registered is temporarily registered in an update buffer. Is used for searching.
[0011]
The text data registration and search method solves the above problem by using a text data registration and search method having the following steps.
[0012]
(1) Update document number registration step
The document number of the document for which a new update, deletion, or registration request has been made is registered in the updated document number list.
[0013]
(2) Update text buffer creation step
The text data of a document for which a new update or registration request has been made is registered in an update text buffer.
[0014]
The normal registration processing is performed in the update document number registration step (1) and the update text buffer creation step.
[0015]
(3) Character component file merge condition determination step
The timing for registering the text data temporarily stored in the update text buffer in the character component file is determined. This timing is when one or more of the following conditions are satisfied.
[0016]
(A) When there is a request from the system administrator
(B) When a predetermined fixed number of documents are registered in the update text buffer
(C) When the time required for the process of registering the updated document number list and the updated text buffer in the updated document number registering step and the updating text buffer creating step exceeds a predetermined amount.
(D) When the time required for processing of the update text buffer search step and the search result merge step exceeds a predetermined amount.
(E) When a predetermined time has come.
[0017]
(4) Character component file merge step
In the character component file merge condition determination step (3) described above, the operation is performed when the determined conditions are satisfied, and the text data temporarily stored in the update text buffer is registered in the character component file.
[0018]
The document numbers registered in the update document number list created in the above (1) update document number registration step are deleted from the character component file, and the text of the update text buffer created in the above (2) update text buffer creation step is deleted. For the character component of the data, the corresponding document number is additionally registered in the character component file.
[0019]
(5) Character component file search step
At the time of search, data already registered in the character component file is searched.
[0020]
(6) Update text buffer search step
At the time of retrieval, data not yet registered in the character component file, that is, text data stored in the update text buffer is retrieved.
[0021]
(7) Search result merge step
At the time of retrieval, the document that has been updated and deleted, that is, all the document numbers stored in the updated document number list, are deleted from the retrieval result as invalid documents with respect to the result retrieved in the character component file retrieval step (5). (6) Add the result obtained in the update text buffer search step.
[0022]
By these steps, when the document is registered, the process ends with the registration of the updated document number list and the update text buffer that are light in processing, and the response time can be shortened.
[0023]
By providing the updated document number list and the update text buffer as described above, the search process is to search both the character component file and the update text buffer.
[0024]
However, this update text buffer search process depends on the size of the update text buffer, and a small update text buffer has little effect on the search process. That is, according to the text data registration search method, it is possible to minimize the deterioration of the search processing efficiency and improve the speed performance of the registration processing.
[0025]
The size of the update text buffer also affects the time required for the registration process. In other words, the time required for the registration and search processing of this update text buffer increases with the size of the capacity.
[0026]
Therefore, according to the conditions shown in the (3) character component file merge condition determination step, new information is additionally registered in the character component file based on the text data in the updating text buffer in the (4) character component file merge step.
[0027]
As a result, the size of the update text buffer is maintained at an appropriate amount, and appropriate response performance can be maintained for both registration and search processing. That is, it is possible to minimize the influence on the performance of the search processing and improve the registration processing performance.
[0028]
As described above, according to the text data registration and retrieval method, when a document is registered, the process can be completed by registering the updated document number list and the update text buffer with light processing, thereby further improving the transaction processing performance. It is possible to do.
[0029]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, in the text data registration / search method of the present invention, a text data registration / search method according to an embodiment for obtaining a desired document by searching a document database by specifying a search word that is a predetermined character string will be described. The device will be described.
[0030]
FIG. 1 is a diagram illustrating a schematic configuration of an implementation device that implements the text data registration and search method according to the present embodiment. In FIG. 1, 100 is a memory, 110 is a storage medium, 200 is a CPU, 300 is an updated document number list, 400 is an update text buffer, 500 is a character component file, 600 is a clock, and 700 is a memory.
[0031]
As shown in FIG. 1, in an apparatus for implementing the text data registration / search method of the present embodiment, a memory 100 for storing programs for registration / search, a storage medium 110 for the programs, a CPU 200 for executing those programs, An updated document number list 300 for storing a document number, an update text buffer 400 for storing data of an updated document, a character component file 500 storing character components and document numbers including the character components in association with each other, a current time Is provided to the character component file merge determination step at regular intervals, and a memory 700 for storing each specified value used as a determination condition in the character component merge determination step.
[0032]
First, a description will be given of a method of registering a document by an execution apparatus that performs the text data registration search method of the present embodiment.
[0033]
FIG. 2 is a PAD diagram showing a flow of a document registration process of the embodiment apparatus that implements the text data registration search method of the present embodiment.
[0034]
As shown in FIG. 2, in the document registration process of the apparatus for executing the text data registration search method according to the present embodiment, the document to be registered has a document number and text data for identifying the document. And The registration process includes an update document number registration step of registering a document number in the update document number list 300 (step 201) and an update text buffer creation step of registering text information in the update text buffer 400 (step 202). ).
[0035]
In the update document number registration step of step 201, the document number of the document to be registered is registered in the update document number list 300 stored on the memory or the magnetic disk.
[0036]
FIG. 3 is a diagram showing the structure of the updated document number list 300 of the embodiment that implements the text data registration and search method of the present embodiment.
[0037]
As shown in FIG. 3, in the updated document number list 300 of the apparatus that implements the text data registration and search method of the present embodiment, the updated document number, which is the document number of the document to be registered, is stored in an array format of numerical data. In this structure, the number of stored document numbers is stored as the first data of the array.
[0038]
In the update text buffer creation step of step 202, a process of adding document data to the update text buffer 400 is performed.
[0039]
FIG. 4 is a diagram showing a data structure of the updating text buffer 400 of the embodiment that executes the text data registration / search method of the present embodiment.
[0040]
As shown in FIG. 4, the updating text buffer 400 of the embodiment that implements the text data registration / retrieval method according to the present embodiment includes a data directory and a data part, and the data directory contains the text data stored in the data part. And the length of each data are stored together with the document number. When registering a new document, it is only necessary to register the position and data length of the data portion in which new data is written in the directory portion after the data portion, together with the document number.
[0041]
By providing the updated document number list 300 and the updating text buffer 400 in this manner, the registration process can be easily performed.
[0042]
Next, a description will be given of a document search method of an embodiment apparatus that performs the text data registration search method of the present embodiment.
[0043]
FIG. 5 is a PAD diagram illustrating a flow of a document search process performed by the apparatus that performs the text data registration / search method according to the present embodiment.
[0044]
As shown in FIG. 5, in the document search process of the apparatus that implements the text data registration search method of the present embodiment, a search term is specified, a document containing the search term is searched for from the already registered document data, and The document numbers to be output are output as an array.
[0045]
The search processing of the apparatus that implements the text data registration search method of the present embodiment includes a character component file search step for searching the character component file 500 (step 501) and a search for data in the update text buffer 400. The process of the update text buffer search step (step 502), the document number registered in the update document number list 300 is deleted from the search result of the character component file search step, and the result of the update text buffer search step is added. It consists of a search result merge step (step 503).
[0046]
The character component file search step, which is step 501, reads data of character components constituting a search word of the character component file 500, and outputs a document including all character components, that is, a document number including the search word.
[0047]
FIG. 6 is a diagram showing the structure of the character component file 500 of the embodiment that implements the text data registration and search method of the present embodiment.
[0048]
As shown in FIG. 6, the character component file 500 of the embodiment that implements the text data registration and search method of the present embodiment includes a character component data directory and character component data.
[0049]
The character component data directory stores one or more contiguous character strings in a document as character components and stores pointer information to character component data for storing a document number where the character component appears and an appearance position of each document. ing.
[0050]
For example, FIG. 6 shows that the character component "document" has character component data for 568 bytes from the 23rd and 654th bytes from the head of the character component data file.
[0051]
The character component data is composed of a document number and the appearance position of the corresponding character component. To distinguish whether each data means a document number or an appearance position, a document number flag is added to each data. I have.
[0052]
That is, when the document number flag is "1", the data means a document number, and when the document number flag is "0", the data means an appearance position. In the example of FIG. "Document" indicates that the document exists at the 56th and 145th characters of the document number 23.
[0053]
In the character component file search step, the document in which the character component constituting the search word exists and the appearance position are read from the character component file 500, and the document in which each character component is adjacent is extracted.
[0054]
For example, in the case of searching by the word “document search”, in the example of FIG. 6, the document numbers and the appearance positions of the two character components “document” and “search” are read, and the respective character components are read. By comparing at the appearance position, a document in which the word “document search” appears accurately can be searched.
[0055]
In an update text buffer search step of step 502, a search process of a newly registered document after the creation of the character component file is performed.
[0056]
The newly registered document is stored in the update text buffer 400 as shown in FIG. Therefore, by reading the document number of each document and the length in the buffer from the data directory, and scanning the corresponding text data, that is, by shifting the text to be searched one character at a time, it is determined whether the text matches the search word. In addition, a search process of a newly registered document can be performed.
[0057]
In the apparatus that implements the text data registration and search method of the present embodiment, the search process of the update text buffer 400 is realized by scanning the text data stored in the update text buffer 400. Apart from the component file 500, it is also possible to create a small character component file 500 separately.
[0058]
In such a case, the configuration of the updating text buffer 400 shown in FIG. 4 is the same as the configuration of the character component file 500 shown in FIG. However, the data stored in the character component file 500 of the update text buffer 400 is only newly registered text data.
[0059]
In this way, by changing the configuration of the update text buffer from flat text to the same configuration as the character component file 500, there is an effect of speeding up the search process.
[0060]
Further, the update text buffer 400 may be on a real memory or on a secondary storage medium such as a magnetic disk device.
[0061]
When the update text data is stored in the real memory, registration and search processing can be performed at high speed, but only the capacity of the real memory can be registered in the update text buffer 400, and before the update text buffer 400 runs out of capacity. , A character component file merge process described later is required.
[0062]
Further, when the update text data is stored in the magnetic disk device, registration and search processing become slower, but more update text data can be stored.
[0063]
The process of deleting the document number registered in the updated document number list 300 in the search result merge step in step 503 is to return an incorrect search result for a document that has been newly updated after being registered in the character component file 500. The corresponding document is deleted from the character component file search result so that there is no such document.
[0064]
For example, as a result of the character component file search step, the document numbers 21, 87, 104, 251,. . . Is obtained, and the updated document number list 300 is as shown in FIG. 3, the document number 104 is deleted from the search result set.
[0065]
Next, in a search result merging step, which is step 503, the result of the text buffer search step for updating is added to the search result of the character component file search step in which the document numbers registered in the updated document number list 300 have been deleted.
[0066]
As described above, in accordance with the document data registration and search procedures of the apparatus that implements the text data registration and search method of the present embodiment, an update document number registration step, an update text buffer creation step, a character component file search step, and an update text buffer The search step and the search result merge step have been described.
[0067]
Hereinafter, the character component file merging step and the character component file merging condition determination step of the embodiment that implements the text data registration and retrieval method of the present embodiment will be described.
[0068]
The character component file merging step executes a process when it is determined in the character component file merging condition determination step described later that it is necessary to merge the character component files 500. Specifically, a process of adding the character component data of the updating text buffer 400 shown in FIG. 4 to the character component file 500 shown in FIG. 6 is performed.
[0069]
Of course, it is also possible to use the small character component file 500 as the configuration of the update text buffer 400 as described above. In this case, a process of merging each character component of the update text buffer 400 into a regular large character component file 500 is performed.
[0070]
The character component file merge condition determination step determines that it is necessary to merge the data in the update text buffer 400 with the character component file 500 under the following conditions.
[0071]
(A) When there is a request from the system administrator
(B) When a predetermined fixed number of documents are registered in the update text buffer 400
(C) When the time required for the registration processing of the update document number list 300 and the update text buffer 400 in the update document number registration step and the update text buffer 400 creation step exceeds a predetermined amount.
(D) When the time required for the processing of the character component file search step and the search result merge step exceeds a predetermined amount
(E) When a predetermined time has come.
[0072]
In the character component file merge determination step, it is determined that the merging process is performed based on such criteria, and the character component file merge step is started. For this reason, the system waits for a request from the system administrator, the number of documents stored in the update text buffer 400, the processing time per unit document in the update text buffer 400 creation step, the character component file search step, and the search result merge step. Always monitor the processing time and current time.
[0073]
FIG. 7 is a diagram showing an outline of the processing of the character component file merge determination step of the execution apparatus that performs the text data registration and search method of the present embodiment.
[0074]
As shown in FIG. 7, in the character component file merge determination step of the execution apparatus that implements the text data registration and retrieval method of the present embodiment, the registration time is determined by the update document number in the update document number registration step and the update document buffer creation step. The registration time in the list 300 and the update text buffer 400 is shown, and the search time is the time required for processing in the character component file search step and the search result merge step.
[0075]
As shown in the figure, the character component file merge determination step includes (1) a request from the system administrator, (2) the number of documents stored in the update text buffer 400, (3) registration time, and (4) search time. (5) A signal indicating the current time indicated by the clock 600 is input and compared with a predetermined number of designated documents, a designated registration time, a designated search time, and a designated time. Invoke the step.
[0076]
FIG. 8 is a PAD diagram showing a flow of a character component file merge determination step of the apparatus for executing the text data registration / search method of the present embodiment.
[0077]
As shown in FIG. 8, in the character component file merge determination step of the text data registration and retrieval method according to the present embodiment, the above signal is waited, and if any signal is input, the above signal is determined depending on the type of the signal. A determination is made as to the set value, and a character component file merging step is started.
[0078]
In the case of a request from the system administrator, the character component file merge step is started immediately. When the number of documents stored in the update text buffer 400 is reported, the number of documents is compared with the set number of designated documents. Start
[0079]
If the registration time is reported, the registration time is compared with the designated registration time, and if the registration processing takes longer than the designated registration time, a character component file merging step is started. By executing the character component file merge step, the update text buffer 400 becomes empty, and the registration process from the next time can be accelerated.
[0080]
If the search time is reported, it is compared with the specified search time, and if the search process takes longer than the specified search time, the character component file merge step is started and the update temporary By reflecting the information in the character component table or the text buffer 400 for updating that has been created in a regular character component table, the search processing time from the next time is expedited.
[0081]
When the current time is reported, the current time is compared with the specified time, and when the specified time has passed, the character component file merging step is started.
[0082]
The numerical values used for these determinations shown in (b) to (e) are set by the system administrator. For example, the number of designated documents is set to 1000, the designated registration time is set to 10 seconds, the designated search time is set to 5 seconds, and the designated time is set to 24:00.
[0083]
These are determined from the viewpoints of emphasis on registration time performance, search performance, and operation. Of course, these values can be set in advance as default values of the program.
[0084]
The character component file merge determination step has been described above. In the apparatus for implementing the text data registration and search method of the present embodiment, the character component file merge determination step is realized by the above-described determination conditions (a) to (e). It can also be used as an execution determination condition of the character component file merge step.
[0085]
As described above, according to the apparatus for implementing the text data registration and search method of the present embodiment, when a document is registered, the process can be completed by a lightly updated document number list and registration in the update text buffer. Therefore, it is possible to further improve the transaction processing performance.
[0086]
As described above, the present invention has been specifically described based on the embodiment. However, the present invention is not limited to the embodiment, and it is needless to say that various changes can be made without departing from the gist of the invention.
[0087]
【The invention's effect】
The effects obtained by the typical inventions among the inventions disclosed in the present application will be briefly described as follows.
[0088]
In other words, when a document is registered, the process can be completed by registering the updated document number list and the update text buffer with light processing, so that the transaction processing performance can be further improved.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a schematic configuration of an implementation device that implements a text data registration and search method according to an embodiment.
FIG. 2 is a PAD diagram showing a flow of a document registration process of an embodiment apparatus that implements the text data registration / search method of the embodiment.
FIG. 3 is a diagram illustrating a structure of an updated document number list 300 of an embodiment that implements the text data registration and search method according to the embodiment.
FIG. 4 is a diagram showing a data structure of an update text buffer 400 of the embodiment that implements the text data registration / search method of the present embodiment.
FIG. 5 is a PAD diagram showing a flow of a document search process of an embodiment apparatus that implements the text data registration / search method of the embodiment.
FIG. 6 is a diagram showing a structure of a character component file 500 of an embodiment for executing the text data registration / search method of the embodiment.
FIG. 7 is a diagram showing an outline of a process of a character component file merge determination step of the embodiment that executes the text data registration / search method of the embodiment.
FIG. 8 is a PAD diagram showing a flow of a character component file merge determination step of the apparatus for executing the text data registration / retrieval method of the embodiment.
[Explanation of symbols]
100: memory, 110: storage medium, 200: CPU, 300: update document number list, 400: update text buffer, 500: character component file, 600: clock, 700: memory.

Claims (4)

In a text data registration search method for registering the appearance relation of characters in a document in a character component file and searching for a document including a specified search word by referring to the appearance relation of the character component file,
An update document number registration step of registering the document number of the document for which a new update, deletion or registration request has been made in the update document number list;
An update text buffer creating step of registering the text data of a document for which a new update or registration request has been made in an update text buffer having the same configuration as the character component file ;
A character component file merging step of registering the text data temporarily stored in the update text buffer in a character component file;
A character component file merge condition determining step for determining a timing of executing the character component file merge step;
A character component file search step for searching data registered in the character component file;
An update text buffer search step for searching text data stored in the update text buffer;
A search result obtained by deleting the document number of a document that has been updated and deleted from the search result as an invalid document with respect to the result searched in the character component file search step, and adding the result obtained in the update text buffer search step A text data registration / retrieval method characterized by having a merge step. 2. The search result merging step according to claim 1, wherein the result obtained in the update text buffer search step is added to a character component file search result in which a document number of the updated document has been deleted. Text data registration search method described. An update document number registration step of registering the document number of the document for which a new update, deletion or registration request has been made in the update document number list;
An update text buffer creating step of registering the text data of a document for which a new update or registration request has been made in an update text buffer having the same configuration as the character component file ;
A character component file merging step of registering the text data temporarily stored in the update text buffer in a character component file;
A character component file merge condition determining step for determining a timing of executing the character component file merge step;
A character component file search step for searching data registered in the character component file;
An update text buffer search step for searching text data stored in the update text buffer;
A search result obtained by deleting the document number of a document that has been updated and deleted from the search result as an invalid document with respect to the result searched in the character component file search step, and adding the result obtained in the update text buffer search step A medium on which a program for causing a computer to execute a merge step is recorded. 4. The search result merging step according to claim 3, wherein the result obtained in the update text buffer search step is added to a character component file search result obtained by deleting a document number of the updated document. The medium described.

Images