JP2003036267A - Information retrieval device and information retrieval method - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide an apparatus and a method capable of easily retrieving information related to an object of an image based only on the image. SOLUTION: Based on image data captured by a digital camera (imaging device) provided in a portable information terminal 1, an image database 5 is searched to extract information related to an image object such as a keyword. Then, based on the information, the Web database 7 is further searched, and U
Retrieve information for specifying information related to the information, such as RL, and return it to the user's portable information terminal or send it to the specified e-mail address.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieving apparatus and method for retrieving a Web address or the like which describes various information relating to an object such as a subject included in image data. is there.

[0002]

2. Description of the Related Art At present, a WWW (World Wide Web) describing various information is constructed on the Internet. A large number of Internet users can access the information on the Web regardless of whether it is wired or wireless by using a computer equipped with Web browsing software or a mobile information terminal.

The above-mentioned Web is, for example, a Web address (URL: Universal Reso) in a format such as http://www.canon.co.jp/
urce Locator) is specified on the Web browsing software, the user can retrieve the text information, image information, and audio information described in the above address. The above Web address can be directly specified by the user if the user correctly recognizes it in advance. Even if it is not, you can access it by specifying a Web address in the form called a link to the address of another Web on the Web. Furthermore, as another means, by inputting a keyword related to the information that the user wants to obtain, the
A web search system for searching and displaying addresses is built on the Internet.
You can also get a web address.

In contrast to such conventional technology, for example, as in Japanese Patent Laid-Open No. 09-231244, GPS (Global Posi) is used so that a Web address can be searched without specifying a keyword.
Based on the location information using the image processing system, search the Web address related to the location, or use the image data to compare the image data stored in advance with the captured image data. A technique for obtaining a web address related to data is also being provided.

[0005]

However, the above conventional technique has a problem that it is difficult to perform a desired search when an appropriate keyword cannot be attached.

Further, in the example disclosed in Japanese Patent Laid-Open No. 09-231244, it is possible to retrieve only a Web address related to a position-dependent object such as a store, a famous place, a historic site, a river, a road, or a city. Also, the biggest problem is that you have to wait until you get the search results for the web address.
The above-mentioned conventional technology is based on the position information and the image data.
Since the address is searched, it is premised that it is used outdoors in a wireless manner by a portable information terminal having a position detection means and an image input means, but depending on the place of use, it may take a long time. There are often situations where it is not possible to secure a line or when it is difficult to wait for a web address to be searched. Therefore, there is a problem that the system becomes an information retrieval system that is difficult to use in actual operation.

The present invention has been made in order to solve the above-mentioned problems of the prior art, and its object is an apparatus and method capable of simply searching for information related to an object of an image based on only the image. To provide.

[0008]

According to the present invention, by having the following means, it is not necessary to input subjective information of a user such as a keyword, and only an image of an object is used,
(EN) An information search device and method for providing an information search system which is significantly easier to operate than conventional ones.

That is, according to the present invention, an image data receiving means for receiving image data, and based on the received image data, an image information database storing the image data and the information about the object of the image data in association with each other is searched. Then, first search means for extracting the information related to the received image data, and search basic information as a basis for the search based on the information related to the image data extracted by the first search means. To search the specified information database that stores the specified information for specifying the information related to the search basic information in association with each other, and specify the information related to the image data extracted by the first searching means. Second search means for extracting the specified information of and the search result output for outputting the specified information extracted by the second search means. And the step, the information retrieval device including a.

Further, it may include image capturing means for capturing an image, and captured image data transmitting means for transmitting data of the image captured by the image capturing means to the image data receiving means.

The image capturing means and the captured image data transmitting means may be constituted by a portable information terminal with an image pickup device.

The image capturing means may be composed of a digital camera or a digital video camera.

The search result output means may include at least non-responsive output means for outputting the search result non-responsively to the extraction of the specific information.

Further, the non-prompt output means may have a function of outputting the specific information and the image data provided to the first searching means together when the specific information is extracted. Good.

The image data output together with the specification information may be thumbnail images.

Further, the search result output means further comprises:
It is also possible to include prompt output means for outputting a search result in response to extraction of the specific information, and output selection request means for requesting selection of which of the search result output means to output.

Also, operating status acquisition means for acquiring information on the operating status of the first searching means and the second searching means, and end time information for estimating time information until the end of search based on the operating status. The estimation means and the display means for displaying time information until the end of the search may be included.

Further, it is possible to include confirmation requesting means for requesting confirmation of whether or not to continue the search after displaying the time information until the end of the search on the display means.

Further, the specific information may be a URL of a Web page including information related to the search basic information.

The present invention is also based on image data,
A method for retrieving specified information for specifying information related to image data, comprising: a step of receiving image data; and an image data and a contrast object of the image data based on the received image data. A first search step of searching an image information database that stores information associated with the information and extracting information related to the received image data, and a search basis based on the information related to the extracted image data For searching a specified information database in which the search basic information and the specified information for specifying the information related to the searched basic information are stored in association with each other, and for specifying the information related to the extracted image data A second search step for extracting the specified information;
Outputting the extracted specified information.

Further, the first search step and the second search step
Of acquiring the information on the execution status of the search step, estimating the time information until the end of the search based on the information on the execution status, and displaying the time information until the end of the search, Good.

Further, a step of requesting confirmation of whether or not to continue the search may be included after the step of displaying the time information until the end of the search.

Furthermore, when the specified information extracted in the second search step is output, it is output in response to the second search step, or in a non-responsive manner to the second search step. The step of outputting may be included.

Further, when the specified information extracted by the second search step is output, it is output in response to the second search step, or is non-responsive to the second search step. It may be possible to include a step of selecting whether to output to.

The present invention is also based on image data,
An information search program for extracting specified information for specifying information related to image data, the step of receiving image data, and the image data and a contrast object of the image data based on the received image data. A first search step of searching an image information database that stores information associated with the information and extracting information related to the received image data, and a search basis based on the information related to the extracted image data For searching a specified information database in which the search basic information and the specified information for specifying the information related to the searched basic information are stored in association with each other, and for specifying the information related to the extracted image data A second search step of extracting the specified information, a step of outputting the extracted specified information,
It is an information search program including.

Further, according to the present invention, based on the image data,
A computer-readable recording medium recording an information search program for extracting specified information for specifying information related to image data, the step of receiving image data, and based on the received image data,
First, a search is performed for an image information database that stores image data and information about a contrast object of the image data in association with each other, and information related to the received image data is extracted.
Of the search step, and based on the information related to the extracted image data, the search basic information that is the basis of the search and the specific information for specifying the information related to the search basic information are stored in association with each other. Information including a second search step of searching a specialized information database for extracting specific information for identifying information related to the extracted image data, and a step of outputting the extracted specific information It is a computer-readable recording medium recording a search program.

[0027]

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

(First Embodiment) FIG. 1 shows a first embodiment of the present invention.
FIG. 3 is a diagram showing a configuration of an information search device that performs a Web address search using a portable information terminal with an imaging device, which is an embodiment of The operation method in the first embodiment will be described with reference to the flowchart shown in FIG.

In FIG. 1, 1 is a portable information terminal with an image pickup device, 2 is a relay wireless station, 3 is a host computer for interfacing with users of the information retrieval system, and 4 is a keyword representing a subject included in image data. Using a keyword output by the first search means 4 and relating to the keyword.
A second search means for searching a Web address, 7 is a Web address database, and 8 is an e-mail server.

The operation of the first embodiment of the present invention will be described below.

Although the portable information terminal 1 with an image pickup device is already on the market, it has a built-in small digital camera and has a function capable of connecting to the host computer 3 of the Internet provider through a wireless telephone line. is there. FIG. 2 shows a schematic configuration of the portable information terminal 1 with an imaging device. CPU (Central Processing Unit) 11 is ROM
The RAM (Random Access Memory) performs predetermined processing according to the program stored in the (Read Only Memory) 12.
Reference numeral 13 is a work area in which a program is expanded or data for the CPU 11 to perform a predetermined process is temporarily stored. The image captured by the image capturing device as the image capturing unit is converted into digital image data and output to the CPU 11 via the interface 15. The image pickup device is composed of, for example, a digital camera or a digital video camera. The imaging device may be detachable from the mobile information terminal. A modem (acquired image data transmitting means) 17 transmits / receives a predetermined signal to / from the relay station 2 via the wireless telephone line in accordance with an instruction from the CPU 11, and the received signal is supplied to the CPU 11. The display 18 as a display means is a CPU
According to the instruction from 11, a URL or output method selection request message, a search continuation confirmation request message, or a Web page is displayed, and is composed of, for example, a liquid crystal display. Operation part 1
9, operation information such as capturing an image from the imaging device 14 and selecting one of a plurality of URLs displayed on the display 18 is input, and, for example,
It can be configured by buttons, dials, or the like.

The image photographed by the portable information terminal 1 with the image pickup device is sent to the relay radio station 2 through the radio telephone line and to the host computer 3. FIG. 3 shows a schematic configuration of the host computer 3. The CPU 31 controls the respective devices of the morphological information terminal with an imaging device 1, the first searching means 4, the second searching means 5 and the electronic mail server 8 via the relay station 2 to execute information search. The ROM 32 stores programs such as an information search program described later and various data. The RAM 33 serves as a work area in which the information retrieval program is expanded or data for the CPU 31 to perform predetermined processing is temporarily stored. Operation unit 39
Is composed of, for example, a keyboard and a mouse, and the user inputs various instructions and information. Auxiliary storage device 34
Is a hard disk, a floppy (registered trademark) disk, a CD-ROM, or the like, and stores programs and various data. The network interface (search result output means) 37 is a LAN (Local Area Network) or WAN.
(Wide Area Network) or communication with each device via various networks such as the Internet. Also,
A display 38 for displaying various information is provided, and each unit of the host computer 3 is connected by a bus 301. Here, when the host computer 3 transmits the search result to the portable information terminal 1 with an imaging device via the Internet or the like, the network interface 37 corresponds to the prompt output means.

The first searching means 4 is, for example, Japanese Patent Laid-Open No. 10-260.
A natural image retrieval device and method, such as in 983, can be used. The image database (image information database) 5 stores a large number of image data together with keywords representing the contents of the image and is attached to the similar image data searched by the first searching means 4. Extract keywords. FIG. 4 schematically shows the structure of the data stored in the image database. The image data includes, for example, 00001,00002,00003.
An identification number such as ... Is given, and keywords representing the contents of the image, KW (0001,1), KW (0001,2), KW (0001,3), ... Are stored in association with the image data ( Here, the keywords are shown by symbols, but the natural languages correspond to each other.).

The extracted keyword (search basic information) is given to the second search means 6. The second search means 6 can use a Web address search system using existing keywords. The Web database (specification information database) 7 stores, for example, a keyword and a URL that is information that specifies a storage location on the network of a Web page related to the keyword. FIG. 5 schematically shows the structure of data stored in the Web database 7. For example, KW (0001,
URL of http: //abcd@def1.com is associated with 1), and KW
The URL of http: //abcd@def2.com is associated with (0001,2).

In this way, the Web address relating to the subject of the image photographed by the portable information terminal 1 with the image pickup device can be obtained. The obtained Web address is returned to the portable information terminal 1 with the image pickup device as it is, or sent to the electronic mail server 8 together with the image taken by the portable information terminal 1 with the image pickup device. The first search means 4, the second search means 6 and the electronic mail server 8 can be configured in the same manner as the host computer. When sending the Web address by e-mail, the e-mail server 8 corresponds to the non-responsive output means.

Regarding the operation of the host computer 3,
This will be described below with reference to the flowchart of FIG.

In step 1, the user is first identified, and the electronic mail address of the user and the information necessary for charging are acquired. In step 2, the number of jobs currently in progress is acquired by the first search means 4 and the second search means 6 in FIG. In step 3, the time required for the search for the user is estimated from the number of Jobs. Step 4
At, the time required for the search is displayed to the user, and the user is allowed to select whether to continue or end the search service. If the user selects to end the search service in step 5, the process proceeds to step 11 to end the search service. When the user selects to continue the search service in step 5, the user is allowed to wait until the search result of the web address is obtained in step 6 or to select whether to use the result via electronic mail. In step 7,
If the user chooses to wait until the search results are available, proceed to step 8; if in step 7 the user chooses to obtain results via email,
Go to step 12. In step 8, the web address is searched by the first search means and the second search means shown in FIG. In step 9, you get a web address. In step 10, the web address is displayed to the user. In step 11, the process ends. On the other hand, in step 12, the web address is searched by the first search means 4 and the second search means 6 of FIG. In step 13, the Web address is obtained and the image data used in the first searching means in FIG. 1 is obtained. In step 14, the Web address and the image data are stored in the electronic mail server of FIG. 1 by the address of the electronic mail address obtained in step 1. The process ends in step 11.

Here, the CPU 31 of the host computer 3 which executes the information retrieval program constitutes an image data receiving means, an output selection requesting means, an operation status acquiring means, an end time estimating means and a confirmation requesting means.

By the above operation, when the user waits until the retrieval result of the Web address is obtained, the desired Web address can be obtained at the end of the retrieval, and the portable information terminal with the image pickup device of FIG. 1 is used. You can refer to the WWW page of the web address. Alternatively, when the result is acquired by e-mail, the user can access the e-mail server using the portable information terminal and obtain the Web address after an appropriate time. If a long time has passed since the search and the web address was obtained from the e-mail server, even if the memory of what the web address was searched for was ambiguous, the attached image data used for the search Can be remembered from

The operation of the above-mentioned host computer 3 is described as a program stored in a storage device such as a ROM or a hard disk, and this program is read into RAM and C
It is realized by executing by PU. Such a program is not limited to the above-mentioned built-in storage device,
Floppy disk, optical disk, magneto-optical disk, CD
-It may be recorded in a portable recording medium such as a ROM, a CD-R, a CD-RW, or a non-volatile memory card, and may be read and executed by a corresponding drive.

(Second Embodiment) FIG. 7 shows a second embodiment of the present invention.
FIG. 3 is a diagram showing a configuration of an information search device that performs a Web address search using the image pickup device and a personal computer, which is the embodiment of FIG.

In FIG. 7, 1 is a digital camera device, 2 is a personal computer, 3 is a host computer for interfacing with users of the information retrieval system, and 4 is a first keyword for retrieving a keyword representing a subject included in image data. Search means, 5 is an image database stored in advance, 6 is a second search means for searching a Web address related to the keyword using the keyword output by the first search means of 4, and 7 is W
A database of eb addresses, and 8 is an e-mail server.

The operation of the second embodiment of the present invention will be described below.

The image data taken by the digital camera device 1 is loaded on the personal computer 2. The loading is performed by the user. The image captured by the digital camera device 1 is sent to the host computer 3 by a wired telephone line or a dedicated computer network line. The following operation is performed in the same procedure as in the first embodiment shown in the flowchart of FIG.

As described above, the present invention can be configured in a commercially available digital camera device and personal computer.

[0046]

As described above, according to the present invention,
Related to position-dependent objects as in the prior art
It is possible to search for specific information regarding all objects based not only on specific information such as a Web address but also on images. Further, according to the present invention, if it is not necessary to wait until the end of the search for the specific information according to the user's judgment, the work can be ended when the request for the search for the specific information is made. In that case, the search result of the specific information is obtained by non-responsive means such as electronic mail.
Further, according to the present invention, even when it takes a long time for a user to request a specific information search and obtain a search result, the image data used for the specific information search is attached. Therefore, it is possible to remember what the request for the specific information search has been requested.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a personal digital assistant with an imaging device.

FIG. 3 is a block diagram showing a schematic configuration of a host computer.

FIG. 4 is a diagram showing an example of a data structure of an image database.

FIG. 5 is a diagram showing an example of a data structure of a Web database.

6 is a flowchart of the operation of the host computer of FIG.

FIG. 7 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.

[Explanation of symbols]

1 Personal digital assistant with imaging device 2 relay radio stations 3 Host computer 4 First search means 5 Image database 6 Second search means 7 Web address database E-mail server

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G06F 13/00 540 G06F 13/00 540E

Claims (18)

[Claims] 1. An image data accepting means for accepting image data, and based on the accepted image data, an image information database that stores the image data and information about an object of the image data in association with each other is searched and accepted. First search means for extracting information related to the image data, based on the information related to the image data extracted by the first search means, search basic information serving as a basis for search and the search basic information Specific information for identifying the information related to the image data extracted by the first searching means by searching the specific information database that is stored in association with the specific information for specifying the information related to And a search result output means for outputting the specified information extracted by the second search means. Search device. 2. An image capturing device for capturing an image, and a captured image data transmitting device for transmitting data of an image captured by the image capturing device to the image data receiving device. Information retrieval device. 3. The information retrieving apparatus according to claim 2, wherein the image capturing means and the captured image data transmitting means are constituted by a portable information terminal with an image capturing device. 4. The image capturing means comprises a digital camera or a digital video camera.
Information retrieval device described in. 5. The information search device according to claim 1, wherein the search result output means includes at least non-responsive output means for outputting the search result non-responsively to the extraction of the specified information. 6. The non-responsive output means includes specific information,
The information search device according to claim 5, further comprising a function of outputting the image data provided to the first search means when the specified information is extracted. 7. The information search device according to claim 6, wherein the image data output together with the specific information is a thumbnail image. 8. The search result output means further includes a prompt output means for outputting a search result in response to the extraction of the specified information, and requests which of the search result output means outputs the search result. 8. The information search device according to claim 5, further comprising output selection requesting means. 9. An operation status acquisition unit that acquires information about the operation status of the first search unit and the second search unit, and end time information that estimates time information until the end of the search based on the operation status. 9. The information search device according to claim 1, further comprising: an estimation unit and a display unit that displays time information until the end of the search. 10. The information retrieving apparatus according to claim 9, further comprising confirmation requesting means for requesting confirmation of whether or not to continue the search after displaying the time information until the end of the search on the display means. 11. The information search device according to claim 1, wherein the specified information is a URL of a Web page including information related to search basic information. 12. An information retrieval method for extracting specified information for specifying information related to image data based on the image data, the method comprising: accepting the image data, and based on the received image data. A first search step of searching the image information database that stores the image data and the information relating to the contrast of the image data in association with each other, and extracting the information related to the received image data; and the extracted image data Based on the information related to, the search is performed by searching the specific information database that stores the search basic information that is the basis of the search and the specific information for specifying the information related to the search basic information in association with each other. A second search step for extracting specified information for specifying information related to the image data, and outputting the extracted specified information Information retrieval method includes a step that, a. 13. The method further comprising: acquiring information regarding execution status of the first search step and second search step; estimating time information until completion of search based on the execution status information; The information search method according to claim 12, further comprising: displaying time information until the end of the search. 14. The information search method according to claim 13, further comprising a step of requesting confirmation of whether or not to continue the search after the step of displaying time information until the end of the search. 15. When outputting the specified information extracted in the second search step, the specified information is output in response to the second search step, or in a non-responsive manner to the second search step. The information search method according to claim 12, further comprising: a step of outputting. 16. When outputting the specified information extracted by the second search step, the specific information is output in response to the second search step, or is non-responsive to the second search step. 16. The information search method according to claim 15, further comprising the step of selecting whether to output to. 17. An information search program for extracting specified information for specifying information related to image data based on the image data, the step of receiving the image data, and the step of receiving the image data based on the received image data. A first search step of searching the image information database that stores the image data and the information relating to the contrast of the image data in association with each other, and extracting the information related to the received image data; and the extracted image data Based on the information related to, the search is performed by searching the specific information database that stores the search basic information that is the basis of the search and the specific information for specifying the information related to the search basic information in association with each other. A second search step for extracting the specified information for specifying the information related to the image data, and the extracted specified information Information retrieval program comprising the steps of outputting. 18. A computer-readable recording medium recording an information retrieval program for extracting specified information for specifying information related to image data based on the image data, the step of receiving the image data. And searching the image information database that stores the image data and the information related to the contrast of the image data in association with each other based on the received image data, and extracting the information related to the received image data. Of the search step and the identification information stored in association with the retrieval basic information that is the basis of the retrieval and the specific information for identifying the information related to the retrieval basic information, based on the information related to the extracted image data. A second step of searching the specialized information database and extracting the specific information for specifying the information related to the extracted image data; A computer-readable recording medium recording an information search program including a search step and a step of outputting the extracted specified information.