JP2009282913A - Personal-adaptive web information search device, method, and program - Google Patents

Abstract

Personal adaptive search is performed by appropriately acquiring personal profile information in order to enable Internet information search according to personal interests.
The present invention also extracts a set of feature words as profile information of a user from a Web page text indicated by a URL included in the user's bookmark and a linked Web page to which the page refers. Following the change due to the bookmark update, the profile information (feature word group) is compared before and after the update to generate a feature word group as a common part, and the profile information is updated. The search results are sorted and output based on profile information having a certain degree of similarity or higher in the updated profile information and the updated link destination profile information.
[Selection] Figure 1

Description

  The present invention relates to a personal adaptive Web information search apparatus, method, and program, and more particularly to a personal adaptive Web information search apparatus and method capable of updating profile information when scoring Web search results based on a personal profile. And the program.

  In the conventional personal adaptive information search technology, there are the following methods for changing the suitability ranking of information search results using profile information created according to the personal interest content.

The search formula used by the user in normal information search, the character string displayed in the search result and the word obtained by removing the stop word from the word included in the content sentence of the document saved by the user A set of frequently used words (feature words) is used as the profile information of the user, and a similarity ranking between the user and the document vector of the document obtained as a result of the information search is set as the search result rank (for example, Non-Patent Document 1). reference).
Namiyama Hiroshi "Personal Adaptive Information Retrieval System: Hierarchical Memory Model for Learning Individual Interests and Its Application to Collaborative Filtering" IPSJ SIG Notes Vol. 96, No. 70 pp. 49-56 (1996)

  However, the conventional method has the following problems.

  In the above method, since the entire search result includes many Web pages that do not match the user request, many feature words that do not match the user's interest are extracted from the feature words extracted therefrom.

  In addition, Web pages are not usually saved for permanent Web sites. For this reason, in the conventional method described above, it is difficult to extract an appropriate feature word as profile information from the target Web page, and thus personal adaptation of search results using it is not appropriately performed.

  The present invention has been made in view of the above points. In order to enable Internet information search according to personal interests, it is possible to perform personal adaptive search by appropriately acquiring personal profile information. An object of the present invention is to provide a personal adaptive web information retrieval apparatus, method and program.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is a personal adaptive information retrieval method for retrieving information on a Web page according to an individual's interest,
A URL extracting step (step 1) in which the URL extracting unit obtains a URL stored as a useful Web page that the user browsed and stored in the past from the bookmark storing unit;
A first feature word extraction unit acquires a Web page based on the URL, and stores a feature word group WAa extracted from the text of the Web page in the first profile information storage unit (step 2) )When,
The first updating means compares the second profile information storage means storing the previous feature word group WBa with the feature word groups WAa and WBa stored in the first profile information storage means, and compares them. A first update step (step 3) for updating the contents of the second profile information storage unit by generating a feature word group WCa that is a common part of the second profile information and storing it in the second profile information storage unit ,
The second feature word extraction unit acquires a reference destination Web page based on the acquired URL, and stores the feature word group WAb extracted from the text of the reference destination Web page in the first additional profile information storage unit. A reference page acquisition step (step 4);
The second updating unit compares the second additional profile information storage unit storing the previous feature word group WBb with the WAb and WBb stored in the first additional profile storage unit, and uses them in common. A second update step (step 5) for updating the contents of the second additional profile information storage means by generating a feature word group WCb to be a part and storing it in the second additional profile information storage means; ,
The feature word synthesizing unit compares the similarity with each of the feature word groups WAa and WAb stored in the second profile information storage unit and the second additional profile information storage unit, A feature word synthesis step (step 6) for extracting a feature word group WD;
When a search expression is input (step 7), the search result sorting means performs a search result sorting step (step 8) in which the search result is sorted and output based on the feature word group WD.

Further, the present invention (Claim 2) provides a reference page acquisition step (Step 4).
Based on the acquired URL of the reference destination and the reference level n, the process of acquiring the source indicated by the URL is repeated until the reference level reaches a predetermined level N.

Further, the present invention (Claim 3) provides a search result sorting step (Step 8).
Generate a document vector from each document included in the search results,
An inner product with a vector based on the feature word group WD is calculated, and the documents are sorted in descending order of the inner product.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 4) is a personal adaptive information retrieval apparatus for performing information retrieval of Web pages according to personal interests,
Bookmark storage means 1 storing a Web page selected by the user as a bookmark;
URL extraction means 2 for acquiring a URL stored as a useful Web page from the bookmark storage means 1 that the user browsed and saved in the past;
Page acquisition means 3 for acquiring a web page based on the URL;
First feature word extraction means 4 for storing the feature word group WAa extracted from the text of the Web page in the first profile information storage means 9;
The second profile information storage means 8 in which the previous feature word group WBa is stored and the feature word groups WAa and WBa stored in the first profile information storage means are compared to form a common part of them. A first updating unit 10 for generating the word group WCa and storing the word group WCa in the second profile information storage unit 8 to update the contents of the second profile information storage unit 8;
A reference destination page acquisition unit 13 that extracts a reference destination URL from the Web page acquired by the page acquisition unit 5 and acquires a reference destination Web page;
A second feature word extraction unit 11 that stores the feature word group WAb extracted from the text of the reference destination Web page in the first additional profile information storage unit 17;
The feature word that is the common part of the second additional profile information storage means 16 storing the previous feature word group WBb and the WAb and WBb stored in the first additional profile storage means 17 are compared. A second updating unit 18 for updating the contents of the second additional profile information storage unit 16 by generating a group WCb and storing it in the second additional profile information storage unit 16;
Similarity levels are compared for each of the feature word groups WAa and WAb stored in the second profile information storage unit 8 and the second additional profile information storage unit 16, and a feature word group WD having a predetermined similarity or higher is compared. Feature word synthesizing means 19 for extracting
And search result sorting means 22 for sorting the search results based on the feature word group WD and outputting the results searched by the search means 21 based on the input search formula.

Further, the present invention (Claim 5) is provided in the page acquisition means 5,
Means for outputting the reference URL and reference level n of the acquired web page to the reference page acquisition means 13;
The reference page acquisition means 13
Based on the acquired URL of the reference destination and the reference level n, means for repeating the process of acquiring the source indicated by the URL until the reference level reaches a predetermined level N is included.

Further, the present invention (Claim 6) is provided in the search result sorting means 22,
Means for generating a document vector from each document included in the search results;
Means for calculating an inner product with a vector based on the feature word group WD and sorting the documents in descending order of the inner product.

  The present invention (Claim 7) is a personal adaptive Web information search program for causing a computer to function as each means constituting the personal adaptive Web information search apparatus according to any one of Claims 4 to 6.

  As described above, according to the present invention, the bookmark is included in the bookmark as compared with the conventional feature word extraction method in which the user extracts the entire search result page temporarily browsed during the Internet search or the text of the saved Web page. By extracting feature words from the Web page indicated by the URL and another Web page referenced by the page, profile information more personally suited to the user can be created. As a result, search results using the profile information can be created. The fitness ranking is more personalized.

  In addition, the content of the profile information can be appropriately updated by appropriately calculating the feature word group that changes in accordance with the change of the content from the comparison of the feature word groups before and after the update after the bookmark is updated. .

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 shows the configuration of a personal adaptive Internet information retrieval apparatus according to an embodiment of the present invention.

  The apparatus shown in the figure includes a bookmark storage unit 1, a URL extraction unit 2, a timing control unit 3, a URL filtering unit 4, a basic HTML acquisition unit 5, a text extraction unit 6, a feature word acquisition unit 7, and a basic feature word storage unit A8. , Basic feature word storage unit B9, comparison update unit 10, link acquisition unit 11, URL filtering unit 12, HTML acquisition unit 13, text extraction unit 14, feature word acquisition unit 15, additional feature word storage unit A16, additional feature word storage A part B 17, a comparison / update part 18, a feature word composition part 19, a search expression input part 20, a search engine part 21, a search result sort part 22, and a result display part 23.

  The operation in the above configuration is shown below.

  The personal adaptive Internet information retrieval device acquires Web page information, such as a URL stored in a bookmark as a useful Web page stored by the user in the past, from the browser bookmark, and the URL These feature word groups are extracted from the web page and the page reference destination web page body, and are used as profile information. Further, when the bookmark is subsequently updated, the feature word groups obtained from the reference Web pages before and after the update are compared to update the profile information to a more appropriate one.

  At the time of retrieval, the information retrieval results are sorted into suitability rankings that are personally adapted by the feature word group in the profile information.

  FIG. 4 is a flowchart of the overall processing in one embodiment of the present invention.

  It is assumed that the user stores in advance in the bookmark storage unit 1 a web page that is of interest to the user prior to information retrieval.

  Step 101) The URL extraction unit 2 extracts the URL stored in the bookmark storage unit 1 and sends it to the URL filtering unit. The timing for extracting the URL is specified based on a timing specified by the user specified by the timing control unit 3 or a timing scheduled in advance such as every X hours.

  Step 102) The URL filtering unit 4 performs filtering on the URL, removes unnecessary URLs, and sends other URLs to the basic HTML acquisition unit 5. As a specific filtering method, for example, a method of removing a URL that matches a URL black list created in advance or a method based on a matching rule based on a URL character string pattern is used.

  The processing in this step will be described in detail with reference to FIG.

  Step 103) The basic HTML acquisition unit 5 acquires the Web page (basic page) of the designated URL from the Internet, and sends the HTML source of the page to the text extraction unit 6 and the link extraction unit 11. At this time, reference level information is added.

  Step 104) The text extracting unit 6 removes the tag from the HTML source of the basic page, extracts only the text of the sentence, and sends it to the feature word acquiring unit 7.

  Step 105) The feature word extraction unit 7 analyzes the extracted text to extract words, removes stop words from the extracted word, and then selects feature words from those words by a feature word extraction method. And it preserve | saves at basic feature word preservation | save part B9. However, the feature word extraction method used here may be any existing method such as the TFIDF method.

  Step 106) The comparison / update unit 10 acquires the feature word group WB from the basic feature word storage unit B9. If nothing is stored in the basic feature word storage unit A8, the acquired feature word group WB is Save in the basic feature word storage A8. However, if a feature word group already exists in the basic feature word storage unit A8, the feature word group WA is acquired, and both the WA and WB are compared to generate a feature word group WC that is a common part of them. The content of the basic feature word storage unit A8 is updated with WC.

  Step 107) In the link extraction unit 11, in step 103, the link destination URL indicated by the anchor tag is acquired from the target HTML source sent from the basic HTML acquisition unit 5 and sent to the URL filtering unit 12. The processing in this step will be described in detail with reference to FIG.

  Step 108) The URL filtering unit 12 filters the link destination URL in the same manner as the URL filtering unit 4, and sends the obtained URL to the HTML acquisition unit 13.

  The processing in this step will be described in detail with reference to FIG.

  Step 109) The HTML acquisition unit 13 acquires the Web page of the specified URL from the Internet, sends the HTML source of the page to the text extraction unit 14, and the URL is within a predetermined N link from the basic page. Is sent to the link extraction unit 11. Here, the link extraction unit 11, the URL filtering unit 12, and the HTML acquisition unit 13 repeatedly acquire Web pages within N links from the basic page. However, in order to obtain an equivalent processing result, these blocks are continuously arranged in N stages. It may be configured as follows. The processing of this step will be described in detail with reference to FIG.

  Step 110) The text extracting unit 14 removes the tag from the HTML source acquired by the HTML acquiring unit 13, extracts only the text of the sentence, and sends it to the feature word acquiring unit 15.

  Step 111) The feature word extraction unit 15 analyzes the extracted text to extract words, removes stop words from the extracted word, selects feature words from those words by a feature word extraction method, and adds an additional feature word storage unit. Store in B17. However, as a feature word extraction method used here, any existing method such as the TFIDF method may be used.

  Step 112) The comparison / update unit 18 acquires the feature word group WB from the additional feature word storage unit B17. If nothing is stored in the additional feature word storage unit A16, the acquired feature word group is added to the additional feature word storage unit A16. Save in the word storage unit A16. However, if a feature word group already exists in the additional feature word storage unit A16, the feature word group WA is acquired, and both the WA and WB are compared to generate a feature word group WC that is a common part of them. The content of the additional feature word storage unit A16 is updated with WC.

  Step 113) The feature word synthesis unit 19 acquires the feature word group WA of the basic feature word storage unit A8 and sends it to the search result sorting unit 17. The feature words included in the additional feature word storage unit A16 are also acquired, the degree of similarity is compared with all the words in the feature word group WA for each of them, and a certain degree of similarity with any feature word of the WA Those having the above are sent to the search result sorting unit 17. However, any existing similarity such as cosine similarity may be used as the similarity used here. Detailed processing of this step will be described in detail with reference to FIG.

  Step 114) The search expression input unit 20 receives the input of the search expression from the user, and sends the obtained search expression to the search engine unit 21.

  Step 115) The search engine unit 21 performs an Internet search using the search formula obtained by the search formula input unit 20, and sends the search result to the search result sorting unit 17.

  Step 116) The search result sorting unit 17 generates a document vector from each document included in the search result obtained from the search engine unit 21, and calculates an inner product with the vector based on the feature word group obtained from the feature word synthesis unit 19. Then, the original documents are sorted in the order of the inner product, and the result is sent to the result display unit 23.

  Step 117) Finally, the result display unit 23 displays the search results obtained by the search result sorting unit 22 to the user.

  Next, URL filtering in steps 102 and 108 will be described. Hereinafter, an example of the black list method will be described.

  FIG. 5 is a flowchart of URL filtering processing according to an embodiment of the present invention. Below, although the example of the URL filtering part 4 is shown, the URL filtering part 12 which filters URL of a link destination performs the same process.

  Step 201) The URL is acquired from the URL extraction unit 2.

  Step 202) The acquired URL is checked against a black list database (not shown) in which the URL to be removed is registered.

  Step 203) If the URL matches the black list as a result of the collation, this processing ends. If no match is found, the process proceeds to step 204.

  Step 204) The URL is output to the basic HTML acquisition unit 5.

  Next, the operation in the link acquisition unit 11 in step 107 will be described.

  FIG. 6 is a flowchart of link acquisition processing according to an embodiment of the present invention.

  Step 301) The link acquisition unit 11 acquires an HTML document and its reference level n = k from the basic HTML acquisition unit 5.

  Step 302) The HREF attribute is extracted as a URL for all anchor tags in the HTML document.

  Step 303) For each URL, output its reference level n as k + 1.

  Next, the operation of the HTML acquisition unit 13 in step 109 will be described.

  FIG. 7 is a flowchart showing processing of the HTML acquisition unit in the embodiment of the present invention.

  Step 401) The URL and its reference level are acquired from the URL filtering unit 12.

  Step 402) The source of the Web page indicated by the URL is acquired.

  Step 403) The reference level n is compared with N (a preset maximum value of the reference level). If n <N, the page source and the reference level n are output to the link acquisition unit 11 and the process proceeds to Step 404. To do. If n ≧ N, the process proceeds to step 404.

  Step 404) The page source is output to the text extracting unit 14.

  Next, the processing of the feature word synthesis unit 19 in step 113 will be described.

  FIG. 8 is a flowchart of the feature word synthesis process according to the embodiment of the present invention.

Step 501) Basic feature words W1,..., Wn are acquired from the basic feature word storage unit A8, and these are output. Step 502) Loop variables k and i are initialized to 1.

  Step 503) If i> m (the maximum number of additional feature words), the process ends. Otherwise, the process proceeds to step 504.

  Step 504) If k> n (the maximum number of basic feature words), the process proceeds to Step 509. Otherwise, the process proceeds to step 505.

  Step 505) Ski, which is the similarity between the basic feature word Wk and the additional feature word Ti, is calculated.

  Step 506) If the similarity Ski is equal to or greater than a predetermined threshold C, the process proceeds to Step 507. Otherwise, the process proceeds to step 508.

  Step 507) The additional feature word Ti is output and the process proceeds to Step 509.

  Step 508) Set k = k + 1 and proceed to Step 504.

  Step 509) Set i = i + 1, and proceed to Step 503.

  The operations of the components shown in FIG. 3 described above can be constructed as a program and installed in a computer used as a personal adaptive Internet information retrieval apparatus for execution or distributed via a network.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to Internet Web search technology.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the personal adaptive type | mold web information search device in one embodiment of this invention. It is a flowchart of the whole process in one embodiment of this invention. It is a flowchart of the URL filtering process in one embodiment of this invention. It is a flowchart which shows the process of the link acquisition part in one embodiment of this invention. It is a flowchart which shows the process of the HTML acquisition part in one embodiment of this invention. It is a flowchart which shows the process of the feature word synthetic | combination part in one embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Bookmark memory | storage means, Bookmark memory | storage part 2 URL extraction means, URL extraction part 3 Timing control part 4 URL file talling part 5 Page acquisition means 6 Text extraction part 7 1st feature word extraction means, feature word acquisition part 8 2nd Profile information storage unit, basic feature word storage unit A
9 First profile information storage means, basic feature word storage B
10 first update means, comparison update section 11 second feature word extraction means, link acquisition section 12 URL filtering section 13 reference page acquisition means, HTML acquisition section 14 text extraction section 15 feature word acquisition section 16 second addition Profile information storage means, additional feature word storage unit A
17 First additional profile information storage means, additional feature word storage B
18 second update means, comparison update section 19 feature word composition means, feature word composition section 20 search expression input section 21 search means, search engine 22 search result sort means, search result sort section 23 result output section

Claims (7)

A personal adaptive information retrieval method for retrieving information on a web page according to an individual's interest,
A URL extracting step in which the URL extracting means acquires from the bookmark storage means a URL that the user has browsed in the past and saved as a useful Web page;
A first feature word extraction unit that acquires a Web page based on the URL and stores a feature word group WAa extracted from the text of the Web page in the first profile information storage unit;
The first updating means compares the second profile information storage means storing the previous feature word group WBa with the feature word groups WAa and WBa stored in the first profile information storage means. A first update step of updating the contents of the second profile information storage unit by generating a feature word group WCa serving as a common part thereof and storing it in the second profile information storage unit;
The second feature word extraction unit acquires a reference destination Web page based on the acquired URL, and stores the feature word group WAb extracted from the text of the reference destination Web page in the first additional profile information storage unit. A reference page acquisition step to be performed;
The second updating unit compares the second additional profile information storage unit storing the previous feature word group WBb with the WAb and WBb stored in the first additional profile storage unit, and compares them. A second update step of updating the contents of the second additional profile information storage unit by generating a feature word group WCb to be a common part and storing it in the second additional profile information storage unit;
The feature word synthesizing unit compares the similarity with each of the feature word groups WAa and WAb stored in the second profile information storage unit and the second additional profile information storage unit to obtain a predetermined similarity A feature word synthesis step for extracting the above feature word group WD;
A search result sorting step, wherein the search result sorting means sorts the search results based on the feature word group WD and outputs the results searched by the input search formula;
A personal adaptive Web information retrieval method characterized by: In the reference page acquisition step,
The personal adaptive Web information search method according to claim 1, wherein the process of acquiring the source indicated by the URL is repeated until the reference level reaches a predetermined level N based on the acquired URL of the reference destination and the reference level n. In the search result sorting step,
Generating a document vector from each document included in the search results;
2. The personal adaptive Web information retrieval method according to claim 1, wherein inner products with vectors by the feature word group WD are calculated, and the documents are sorted in descending order of the inner products. A personal adaptive information retrieval device for retrieving information on a web page according to an individual's interest,
Bookmark storage means for storing a web page selected by the user as a bookmark;
URL extraction means for obtaining a URL stored as a useful one of the Web pages that the user browsed and saved in the past from the bookmark storage means;
Page acquisition means for acquiring a web page based on the URL;
First feature word extraction means for storing in the first profile information storage means a feature word group WAa extracted from the text of the Web page;
A feature that is a common part of the second profile information storage means storing the previous feature word group WBa and the feature word groups WAa, WBa stored in the first profile information storage means. A first updating means for updating the contents of the second profile information storage means by generating a word group WCa and storing it in the second profile information storage means;
A reference destination page acquisition unit that extracts a reference destination URL from the Web page acquired by the page acquisition unit and acquires a reference destination Web page;
Second feature word extraction means for storing in the first additional profile information storage means the feature word group WAb extracted from the text of the reference Web page;
The feature word group which is the common part of the second additional profile information storage means storing the previous feature word group WBb and the WAb and WBb stored in the first additional profile storage means A second updating means for updating the contents of the second additional profile information storage means by generating a WCb and storing it in the second additional profile information storage means;
Similarity levels are compared for each of the feature word groups WAa and WAb stored in the second profile information storage unit and the second additional profile information storage unit, and a feature word group WD having a predetermined similarity or higher is compared. Feature word synthesis means for extracting
Search result sorting means for sorting and outputting the search results based on the feature word group WD based on the input search expression;
A personal adaptive Web information retrieval apparatus characterized by comprising: The page acquisition means
Means for outputting the reference URL and reference level n of the acquired web page to the reference page acquisition means;
The reference page acquisition means includes
5. The personal adaptive Web according to claim 4, further comprising means for repeating the process of acquiring the source indicated by the URL based on the acquired URL of the reference destination and the reference level n until the reference level reaches a predetermined level N. Information retrieval device. The search result sorting means includes:
Means for generating a document vector from each document included in the search results;
5. The personal adaptive Web information retrieval method according to claim 4, further comprising means for calculating an inner product with a vector based on the feature word group WD and sorting the documents in descending order of the inner product.   A personal adaptive Web information search program for causing a computer to function as each means constituting the personal adaptive Web information search device according to any one of claims 4 to 6.

Images