JP2018194881A - Document classification system and document classification method - Google Patents

Abstract

In a document classification system, when a document is classified based on a keyword, the basis for the classification is specified, and it is possible to verify whether the keyword that is the basis for the classification is valid. A document classification system for assigning a classification to a document, comprising model information created by reading a classified document as teacher data, reading the document to be classified, and using the model information to classify the document A classification unit that assigns a plurality of classifications to the document, and outputs a word or sentence to be classified as a basis for assigning the classifications. In particular, a list of words or sentences that are the basis and a document in which the words or sentences that are the basis in the document are emphasized are output. [Selection] Figure 1

Description

  The present invention relates to a document classification system and a document classification method, and more particularly to a document classification system and a document classification method suitable for explicitly displaying a classification basis when a patent document is classified by machine learning.

  Patent applications such as International Patent Class (IPC), FI (File Index), and F-term are assigned to patent applications. Conventionally, since these patent classifications are assigned manually, the labor and cost for the classification are not small. For this reason, it has been studied to automatically assign a patent classification by a computer.

  For example, Patent Document 1 is an example of automatic document classification. Patent Document 1 discloses a technique for automatically classifying a document by analyzing the document and counting the classified items for each classification item based on the appearance frequency of keywords that match the word.

JP 2003-36261 A

  In the technology related to automatic document classification described in Patent Document 1, keywords are classified for each classification in advance and the information is retained (FIG. 3), and documents in which the keywords frequently appear belong to the classification items. It is classified as a thing. Then, the classification result for the user is displayed in a format as shown in FIGS. 20 and 21 (paragraph numbers 0101 and 0103).

  However, for the user who receives the document classification result, the classification basis is not shown, and it is impossible to quantitatively evaluate how reliable the classification result is. In addition, since the keyword that is the basis for classification is not shown to the user who confirms the classification result, the validity of the classification based on the keyword cannot be evaluated.

  In addition, related documents that include the same keyword stored in the database and belong to the same category cannot be displayed.

  When categorizing a vast number of documents, such as a classification of patent documents, a user (for example, an examiner of the Patent Office) verifies whether or not the keyword that is the basis of the classification is valid. A request to do so arises. It is also important to examine documents related to a patent document during examination. The request to search for patent documents belonging to the same technical field is the same even when the person in charge of the company accesses the patent documents in connection with research or infringement cases.

  An object of the present invention is to provide a document classification system that, when classifying documents based on keywords, clearly indicates the classification basis and verifies whether the keyword that is the basis of the classification is valid.

  It is another object of the present invention to provide a document classification system that can display related documents that include the same keyword stored in the database and belong to the same classification.

  In order to solve the above problems, the document classification system of the present invention is preferably a document classification system for assigning a classification to a document, comprising model information created by reading a classified document as teacher data, A document to be assigned is read, using a model information, has a classification unit that assigns a plurality of classifications to the document to be classified, and outputs a word or sentence to be classified that is a basis for giving the classification.

  According to the present invention, when a document is classified based on a keyword, it is possible to provide a document classification system that clearly indicates the classification basis and can verify whether the keyword that is the basis of the classification is valid.

  In addition, according to the present invention, it is possible to provide a document classification system that can display related documents that include the same keyword stored in the database and belong to the same classification.

1 is an overall configuration diagram of a document classification system. 6 is a diagram illustrating an example of a document information table 221. FIG. 5 is a diagram illustrating an example of a classification information table 222. FIG. It is a figure which shows an example of the classification provision accuracy information table 223. It is a figure which shows an example of the word weight table 224. It is a figure which shows an example of the sentence weight table 225. 6 is a diagram illustrating an example of a document file 226. FIG. It is a flowchart which shows a model information creation process. It is a figure explaining the outline | summary of the learning and classification by NAM. It is a flowchart which shows the outline | summary of a document classification | category process. It is a flowchart which shows the process until it displays a classification result from literature classification. It is a figure which shows the classification result display screen of the document classification system which concerns on this embodiment (the 1). It is a figure which shows the classification result display screen of the document classification system which concerns on this embodiment (the 2). It is a figure which shows the classification result display screen of the document classification system which concerns on this embodiment (the 3). It is a figure which shows the classification result display screen of the document classification system which concerns on this embodiment (the 4).

  Hereinafter, an embodiment according to the present invention will be described with reference to FIGS.

  In the present embodiment, an example of a document classification system that analyzes patent documents based on appearance words and classifies them according to F-term and FI will be described.

  For example, IPC is given to foreign patent documents such as US and EP, but classification by F-term and FI, which are classification systems of the Japan Patent Office, is not usually given. In the document classification system according to the embodiment, an example will be described in which a classification based on F terms and FIs is given to foreign patent documents to which F terms and FIs are not given in such a situation. At this time, since Japanese patent documents are assigned F-term and FI classifications, a document obtained by machine translation of the Japanese patent documents can be used as learning data for classification. The accuracy (described later) can also be evaluated.

First, the configuration of the document classification system according to the present invention will be described with reference to FIG.
FIG. 1 shows the overall functional configuration of a document classification system. As shown in FIG. 1, the document classification system of this embodiment includes a user terminal 10, an AP (Application) server 100, and a DB server. 200, a document classification server 300 is connected via a network 5.

  The user terminal 10 is a client terminal that accesses the AP server 100 and receives various functions of the document classification system. The user terminal 10 has a UI (User Interface) unit 11 and inputs various commands and data and displays image data created by the AP server 100.

  An AP (Application) server 100 is a server that executes application software for document classification, and creates image data for displaying a document search and information related to the document search. As shown in FIG. 1, the AP server 100 includes a calculation unit 110, and the calculation unit 110 includes functional units such as a search unit 111, an assignment basis editing unit 112, and a screen display data creation unit 113. Have. The search unit 111 is a functional unit that receives an input such as a document number from the user terminal 10 and searches for a document. The grant basis editing unit 112 is a functional unit that edits information for class grant for classifying documents in association with the document number. The screen display data creation unit 113 is a functional unit that creates screen display data to be displayed on the user terminal 10. For example, the AP server 100 is a Web server, and the screen display data creation unit 113 creates an HTML (HyperText Markup Language) file and transmits it to the user terminal 10 using HTTP (Hypertext Transfer Protocol). The UI unit 11 of the user terminal 10 is equipped with a browser, and the transmitted HTML file is displayed by the browser. However, dedicated screen display software is installed in the UI unit 11 of the user terminal 10, and the screen display data creation unit 113 creates screen display data in a unique format and transmits it to the user terminal 10. You may do it.

  The DB server 200 is a server for storing and accessing various data such as literature information and information related to classification, and includes a calculation unit 210 and a storage unit 220 as shown in FIG. The calculation unit 210 includes functional units such as a conversion processing unit 211 and a totalization processing unit 212. Then, the conversion processing unit 211 edits the classification and classification basis calculated by the classification assignment processing unit 312 of the document classification server 300 so as to match the database, and stores the classification basis information in the classification information table 222 as the classification assignment information. As a functional unit stored in the word weight table 224 and the sentence weight table 225. The aggregation processing unit 212 is a functional unit that calculates the accuracy of classification based on the classification stored in the classification information table 222 and creates the classification provision accuracy information table 223 as the provision accuracy information. The storage unit 220 is a functional unit that stores various data and tables, and holds a document information table 221, a classification information table 222, a classification assignment accuracy information table 223, a word weight table 224, a sentence weight table 225, and a document file 226. . Details of the data and the contents of the table will be described in detail.

  The document classification server 300 is a server that provides an automatic document classification function, and includes a calculation unit 310 and a storage unit 320 as shown in FIG. The calculation unit 310 includes functional units such as a learning processing unit 311, a classification assignment processing unit 312, a word context vector weight calculation unit 313, and a sentence context vector weight calculation unit 314. The learning processing unit 311 is a functional unit that learns document data (teacher data) that has been classified and creates / updates model information. The classification assignment processing unit 312 is a functional unit that performs classification assignment on a classification target document and also specifies a classification basis. The word context vector weight calculation unit 313 is a functional unit that adjusts a weight α (described later) so as to be appropriate during learning. The sentence context vector weight calculation unit 314 is a functional unit that adjusts a weight β (described later) so as to be appropriate during learning. The storage unit 320 is a functional unit that stores model information 321 of the learned model. The model information 321 of the learned model will be described in detail later.

  Each of the AP server 100, the DB server 200, and the document classification server 300 is realized by executing necessary application software on a general server device, and the CPU assists the application software on the main storage device such as an HDD. Functions are realized by loading and executing from a storage device. The user terminal 10 is also realized by executing necessary application software in an information processing device such as a general PC (Personal Computer), and the CPU stores the application software on the main storage device such as an HDD. The function is realized by loading and executing from above.

  The AP server 100, the DB server 200, and the document classification server 300 may not be separated as shown in FIG. FIG. 1 shows a configuration in which the document classification system is realized by a server and client configuration. However, a so-called all-in-one configuration in which application software necessary for one information processing apparatus is installed may be used.

  Next, the data structure used in the document classification system will be described with reference to FIGS.

  The document information table 221 is a table for storing information on documents to be classified. As shown in FIG. 2, the document information table 221 includes fields of a document ID 221a, a path 221b, a document file name 221c, a language 221d, and a family document ID 221e. Become. The document ID 221a stores an identifier for identifying a document. In the path 221b, a directory in the file system of the DB server 200 representing the storage location of the document is stored. The document file name 221c stores the file name of the document. In this embodiment, since patent documents are handled, for example, an application number or a publication number is used as a document file name. The language 221d stores a language code used in the literature. The family document ID 221c stores the document ID of the parent application when a priority claim or divisional application is made.

  The classification information table 222 is a table for storing information related to the classification of documents, and includes fields of a document ID 222a, a theme 222b, a classification 222c, and an FI 222d as shown in FIG. The document ID 222a stores an identifier for identifying a document. The theme 222b stores themes in the F-term classification system. The classification 222c stores the classification symbols of the F-term list in the F-term classification system. The FI 222d stores the FI symbol. Here, F-term is a classification system based on technical features described in Japanese patent documents compiled by the Japan Patent Office, and FI is a Japan-specific classification that subdivides the International Patent Classification (IPC). It is a system. In the classification information table 222, a plurality of F terms and FIs can be associated with one document.

  The classification imparting accuracy information table 223 is a table for maintaining the accuracy of classification in machine learning. As shown in FIG. 4, the classification assignment accuracy information table 223 includes fields of the theme 223a, classification 223b, language 223c, accuracy 223d, and creation date 223e. Become. The theme 223a stores themes in the F-term classification system. The classification 223b stores the classification symbols of the F-term list in the F-term classification system. The language 223c stores the language code of the document for which accuracy is created. The accuracy in the classification and the language of the document is stored in the accuracy 223d in% notation. The accuracy is an index indicating the validity of the classification of documents, and the higher the accuracy, the more accurate the classification. A specific calculation method of accuracy will be described in detail later. The creation date 223e stores the date for which the accuracy is calculated.

  The word weight table 224 is a table for storing the weight of each word (Word) in the document classification by machine learning. As shown in FIG. 5, the theme 224a, the classification 224b, the word 224c, and the weight 224d. , The fields of appearance order 224e and document ID 224f. The theme 224a stores themes in the F-term classification system. The classification 224b stores the classification symbols of the F-term list in the F-term classification system. The word for evaluating the weight is stored in the word 224c. The weight 224d stores the weight of the word in the document. Here, the word weight is an amount that serves as an index for evaluating how much the word contributes to the classification of the document by machine learning. The weight of this word will be described in detail later. The appearance order 224e stores the appearance order of the word in the document. The document ID 224f stores an identifier of a document including the word.

  The sentence weight table 225 is a table for storing the weight of each sentence (Statement) in the classification of the document by machine learning. As shown in FIG. 6, the theme 225a, the classification 225b, the word 225c, and the weight 225d. , The appearance order 225e and the document ID 225f. The theme 225a stores themes in the F-term classification system. The classification 225b stores a classification symbol of the F-term list in the F-term classification system. A sentence for evaluating the weight is stored in the word 225c. The weight 225d stores the weight of the sentence in the document. Here, the sentence weight is an amount that serves as an index for evaluating how much the sentence contributes to the classification of the document by machine learning. The weight of this sentence will also be described in detail later. The appearance order 225e stores the appearance order of the sentence in the document. The document ID 225f stores the identifier of the document including the sentence.

  The document file 226 is electronic data of a patent document as shown in FIG. In this embodiment, XML (eXtensible Markup Language), which is a kind of markup language, is used as the format of the patent document.

  Next, processing of the document classification system will be described with reference to FIGS.

  First, a process in which the learning processing unit 311 of the document classification server 300 creates the model information 321 will be described with reference to FIGS. 8 and 9.

  The document classification process in the document classification system of the present embodiment is a process based on a neural attention model (hereinafter referred to as “NAM”) which is an application example of a neural network. The NAM is a model that predicts an output based on a context vector obtained by weighted averaging the units of hidden layers (intermediate layers) on the input side by an attention mechanism. In particular, in the model of this embodiment, an RNN (Recurrent Neural Network) that performs recursive input in a hidden layer of the NAM is used.

  First, a word string of a document is read (S01 in FIG. 8, PH1 in FIG. 9). In FIG. 9, w1, w2,... Of PH1 are word strings.

  Next, a word vector is set for each word as an embedded expression of the word (S02, PH2). The word vector is a vector that expresses the feature amount of the word in the context. In FIG. 9, PH1, x1, x2,... Are word vectors.

  Next, the context vector of the word is set by the calculation of the RNN hidden layer (S03, PH3). The word context vector is a vector that expresses a feature amount according to the context in which the word appears. PH3 indicates that the context vectors h11, h12,... In the forward direction and the context vectors h21, h22,.

Next, at the time of learning, the teacher data is adjusted to be appropriate by the word context vector weight calculation unit 313, and the weight α is assigned to the word context vector. Then, the sentence vectors are calculated by adding the context vectors of the words in the same sentence (S04, PH4). In PH4, the context vector weight of each word is α ₁ , α ₂ ,..., And the sentence vector is s ₁ , s ₂ ,.

  Next, the context vector of the sentence is set by RNN hidden layer calculation while considering the context of each sentence of the sentence vector (S05, PH5). In PH5, the context vector of the sentence is set to hs1, hs2,.

Next, a weight β is assigned to the context vector of the sentence, and the resultant is added to obtain a document vector (S06, PH6). In PH6, the context vector weight of each word is set to β ₁ , β ₂ ,..., And the document vector is set to v 1, v 2,.

  Then, the document vector and the classification (teacher data) previously assigned to the document are compared, and the word context vector weight calculation unit 313 and the sentence context vector weight calculation unit 314 are adjusted so as to be appropriate. Then, an appropriate word context vector weight α and sentence context vector weight β for each context vector are learned (S07, PH8, PH9).

  The word context vector weight α and the sentence context vector weight β for the classification are stored in the weight 224d of the word weight table 224 shown in FIG. 5 and the weight 225d of the sentence weight table 225 shown in FIG.

  Next, a process in which the classification assignment processing unit of the document classification server 300 performs classification assignment for a document will be described with reference to FIGS. 10 and 9.

  In the model information generation process by learning according to the flowchart of FIG. 8, the word context vector weight α and the sentence context vector weight β for the classification are learned based on the teacher data. In the process related to the classification for the document, the classification for the document is given based on the weight α and the weight β.

First, a word string of a document is read (S11 in FIG. 10, PH1 in FIG. 9).
Next, a word vector is set for each word as an embedded expression of the word (S12, PH2).

  Next, the context vector of the word is set by the calculation of the RNN hidden layer (S13, PH3).

  Next, the weight α of the model information 321 (weight 224d of the word weight table 224) is assigned to the context vector of the word, and the sentence vector is calculated by adding the context vectors of the words in the same sentence (S14, PH4).

  Next, the context vector of the sentence is set by RNN hidden layer calculation while considering the context of each sentence of the sentence vector (S15, PH5).

  Next, the weight β of the model information 321 (weight 225d of the sentence weight table 225) is assigned to the context vector of the sentence, and the resultant is added as a document vector (S16, PH6).

  And the classification | category with respect to a literature is provided based on a literature vector (S17, PH7).

Next, processing from document classification to result display will be described with reference to FIG.
First, the document classification server 300 reads a document (reference file 226) to be classified via the DB server 200 (S21).

  Next, according to the process shown in the flowchart of FIG. 10, the classification assignment processing unit 312 of the document classification server 300 assigns a classification to a document (S22).

  Next, in the conversion processing unit 211 of the DB server 200, the classification assignment processing unit 312 of the document classification server 300 stores the classification assignment information for the document in the classification information table 222 illustrated in FIG. 3 (S23).

  Next, the conversion processing unit 212 of the DB server 200 obtains the classification accuracy for the classification of the document and stores it in the classification assignment accuracy information table 223 shown in FIG. 4 (S24).

  Here, as described above, the accuracy of classification is an index representing the validity of classification of documents, and the higher it is, the more accurate the classification is, and the following (formula 1) Desired.

  In addition, the recall of (Formula 1) is obtained by the following (Formula 2).

  Moreover, the precision of (Formula 1) is obtained by the following (Formula 3).

  Next, the classification result is displayed (S25). In this process, the grant basis editing unit 112 of the AP server 100 edits the sort grant basis for the document to which the category is given, and the screen display data creation unit creates screen display data. Delivered to the terminal 10. The AP server 100 reads the document file 226 (FIG. 7) of the target document via the DB server 200 and extracts the classification of the document from the classification information table 222 (FIG. 3). Then, information corresponding to the classification is extracted from the classification grant basis information (word weight table 224 (FIG. 5), sentence weight table 225 (FIG. 6)). Then, the word / sentence in the document file is matched with the word / sentence in the extracted classification grant basis information, and the processing / editing process such as adding the weight in the classification grant basis information to the matched word / sentence is performed. Do it.

  Next, the user interface of the document classification system according to this embodiment will be described with reference to FIGS.

  The classification result display screen 500 of the document classification system is a screen that displays document classification and information related thereto as shown in FIG. This is a screen displayed by the UI unit 11 of the user terminal 10 based on information received from the AP server 100.

  The document display column 501 is a column for displaying the contents of the document for each tab 502. When the user wants to view the document, the user clicks on the tab 502. When applicable, the document display left column 503 displays the original text of the document to be classified, and the document display right column 504 displays the content in a machine-translated sentence or the contents of the family document. Is displayed.

  The classification display field 510 displays a list of classifications assigned to the target document and the classification accuracy.

  The online correction button 520 is a button for accessing a classification input screen (not shown) for manually inputting and correcting a classification of a target document.

  When a category displayed in the category display column 510 is clicked, a word or sentence with a large weight displayed in the document display left column 503 or both of them are highlighted and displayed for the selected category. The highlighting method is changed according to the weight and accuracy. For example, a word with a large weight is emphasized with a dark color or a clear color, and a word with a small weight is emphasized with a light color. The highlighted word or sentence is not limited to the word with the highest weight, but may be a word or sentence with a high ranking such as the top three words with the highest weight.

  When a plurality of categories in the category display field 510 are selected (dragging with the mouse, selecting while pressing the CTRL key, etc.), the color system is changed for each category and highlighted. In the example shown in FIG. 12, for example, words related to 5B075 are emphasized in red, and words related to 5B077 are emphasized in blue.

  Moreover, you may consider the similarity of classification. For example, since 5B075 and 5B077 have the same upper hierarchy, they may be emphasized with a similar red system, and words that are not similar to this field may be emphasized with a blue system.

  Further, the document display right column 504 is also displayed in the document display left column 503 when the category displayed in the category display column 510 is selected and the word or sentence is matched by machine translation. Highlight and / or display words and / or sentences with high weight.

  When the highlighted word is clicked in the document display left column 503 and the document display right column 504, as shown in FIG. 13, a document for selecting a list of document file names of documents having a large weight of the selected word. When the list menu 530 is displayed and the user selects the document list menu 530, a new document display field 501 for displaying the selected document is opened.

  Further, as shown in FIG. 14, by clicking a category displayed in the category display field 510 and inputting a display command by right-clicking or the like, the word / sentence information display field 540 displays the selected category. Information about words / sentences with high weights is displayed in a ranking format.

  Further, when a word in the word / sentence information display field 540 is clicked, as shown in FIG. 15, a document list menu 530 for selecting a list of document file names of documents whose weight of the selected word is large is displayed. When the user selects the document list menu 530, a new document display field 501 for displaying the selected document is opened.

  As described above, according to the document classification system of the present embodiment, it is possible to automatically classify documents based on machine-learned model information for patent documents to which no classification is given.

  At that time, in order to explicitly display the basis of document classification, that is, the words / sentences with high weight in the machine-learned model from the documents in the document, the user reviews the classification by machine learning. Can be validated and corrected if necessary.

In addition, since the basis of document classification, that is, a word / sentence having a large weight in a machine-learned model can be selected and related patent documents having a large contribution to classification of the word / sentence can be displayed, related patents can be displayed. There is an advantage that it is easy to search for documents.
In this embodiment, the classification of the document is assigned and the related patent document is displayed using the information output in the classification process. However, only the related patent document is displayed without assigning the classification. May be. This is because companies and research laboratories that use patent information can use classifications granted by the Patent Office and the like, and therefore do not need to assign new classifications.
Even in this case, the system configuration is the same as in FIG. This is because there is a lot of information that is not disclosed about the classification grounds other than the assigned classification. First, learning model information 321 is created using patent documents (that is, teacher data) classified and assigned by the JPO. Thereafter, using the model information 321, classification grant processing is performed on the patent documents already classified. Thereby, each information memorize | stored in the memory | storage part 220 of DB server 200 is formed, and the display of a related patent is attained.

  As described above, the document classification system according to the present embodiment can be effectively used in the examination department related departments of the JPO that grants F-terms and FIs for examination, and companies and laboratories that use patent information. In Japan, the effective use of patent information can be promoted for research and development.

  DESCRIPTION OF SYMBOLS 10 ... User terminal, 100 ... AP server, 200 ... DB server, 300 ... Document classification server

Claims (15)

A document classification system for assigning classifications to documents,
With model information created by reading classified documents as teacher data,
A classification unit that reads a document to be classified and uses the model information to assign a plurality of classifications to the document to be classified;
A document classification system that outputs the classification target word or sentence that is a basis for the classification. In claim 1,
A document classification system that outputs a document in which a word or sentence that is the basis of the document is emphasized. In claim 1,
A document classification system that outputs a list of words or sentences that serve as a basis. In claim 3,
A document classification system, characterized in that a weight contributing to classification of a word or sentence is displayed in a list of words or sentences as a basis. In any one of Claims 1 thru | or 4,
A document classification system that accepts selection of the assigned classification and displays a basis for giving the selected classification. In claim 5,
The document classification system is characterized in that the basis for assigning the selected classification displays a word or sentence having a large weight that is the basis for classification. In any one of Claims 1 thru | or 6.
A document classification system for displaying classification accuracy for each of the assigned classifications. In any one of Claims 1 thru | or 7,
A document classification system, characterized in that a basis for providing the classification is displayed in a state where the plurality of classifications are distinguishable. In claim 8,
A document classification system, which outputs based on the similarity of the plurality of classifications, according to the similarity of the classifications, by changing the output mode of the words or sentences to be classified. In claims 1 to 9,
A document classification system, wherein a word or sentence to be classified is selected as a basis for classification, and a list of documents having a large contribution of classification of the word or sentence to be classified is displayed. In claim 3,
A document classification system, wherein a word or sentence in a list of words or sentences as a basis is selected and a list of documents having a large contribution of classification of the word or sentence to be classified is displayed. A document classification method for assigning a classification to a document,
Creating model information created by reading classified documents as teacher data;
Reading a document to be classified and assigning a plurality of classifications to the document to be classified using the model information;
Displaying the assigned classification for the relevant document;
Receiving the selection of the assigned classification for the corresponding document, and displaying the basis for the assignment of the selected classification. In claim 12,
A document classification method, comprising: outputting a document in which a word or sentence used as a basis in the document is emphasized as a classification grant basis. In claim 12,
A document classification method characterized by outputting a list of words or sentences as a basis for providing classification. In claim 14,
A document classification method, characterized in that a weight contributing to classification of a word or sentence is displayed in a list of words or sentences as a basis.

Images