JP2003323440A - Information providing system for captured image using mobile terminal, information providing method for captured image, and program for causing computer to execute the method - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To enable retrieval of image information based on an image photographed by a user, and in particular, to immediately obtain this retrieval result on the go using a portable terminal. SOLUTION: When the user encounters an interested object 300 on the road, he or she photographs the object 300 using a mobile phone 301 with a built-in digital camera, and transmits image data to an information providing apparatus 102 via a network 100. Request search.
At this time, the mobile phone 301 sends out the shooting time, the shooting position information, and the distance information to the object 300 in addition to the image data. The information providing apparatus 102 searches the database 310 to 312 and the providing links 320 to 323 for the object 300 at that time and place based on the information, and sends the search result to the mobile phone 301.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention can obtain detailed information of a photographed image through a network, and more specifically, details of an object photographed at a place where a person uses a mobile terminal such as a mobile phone. The present invention relates to a captured image information providing system that uses a mobile terminal that can be used to obtain information on the spot and specify an object, etc., a captured image information providing method, and a program that causes a computer to execute the method.

[0002]

2. Description of the Related Art With the spread of mobile phones, various mobile phone services have been provided. In recent years, mobile phones have GPS
Some have come equipped with a direction detection system, and it is possible to grasp the current position using this. Mobile terminals including mobile phones connected to these networks have excellent portability and can be easily connected to the network when going out (on the go) so that various contents and information can be accessed by accessing the server. It has become.

Further, it is possible to easily take an image of a target object using a mobile phone or a digital camera even when the user is out and transfer data to another mobile terminal or server via a network. The digital camera includes a type built in a mobile phone and a mobile terminal, and a type externally connected to the mobile phone and the mobile terminal to transfer image data.

[0004]

However, at the present time,
It is only possible to easily take an image of an object and transfer data via a network using a mobile terminal including these mobile phones. Therefore, when the user encounters an object that is unknown at the destination or whose details are desired to be known, he can only photograph an image and describe the characteristics (create a sentence). At that time, it is possible to obtain information by calling an acquaintance by phone or e-mail,
The information obtained is uncertain.

As a mobile phone service, a service for searching for peripheral information on the go is provided, and at present it is convenient to obtain a search result by voice input. No search service is provided for.

In order to solve the above-mentioned problems of the prior art, the present invention can retrieve image information based on an image taken by a user, and in particular, the retrieval result can be immediately obtained on the go using a portable terminal. System for providing information on captured images using a portable terminal capable of providing information on captured images,
And a program that causes a computer to execute the method.

[0007]

[Means for Solving the Problems]
In order to achieve the object, an information providing system for photographed images using a portable terminal according to the invention of claim 1 transmits an image photographed by a user from a portable terminal to a center via a network, and the center transmits the image via a network. In the information providing system for a photographed image using a mobile terminal that receives information regarding image data, the mobile terminal includes a photographing means for photographing an object and obtaining image data, and a photographing means for photographing the object. Position information acquisition means for acquiring photographing position information on the map, distance information acquisition means for acquiring distance information to the object when the object is imaged, and the object imaged when a search request is made. Image data of
Communication means for sending the photographing position information and distance information from the portable terminal to the center and for receiving the search result returned from the center, operating means for performing the operation necessary for the search, and the search result An output means for outputting the image using an image, a character, a sound, etc., and the center stores a search database in which image data related to the object, shooting position information, and search information related to distance information are stored. And a photographing position specifying means for specifying a photographing place of the image data by using the photographing position information and the distance information sent from the mobile terminal, and accessing the search database to identify the photographing place. A search means for searching information related to the image data using the image data as a search item; and a search result obtained by the search for the portable terminal. Characterized in that and an information sending means for sending to.

According to the first aspect of the present invention, the user can obtain the search result of the information of the target object only by photographing the target object, requesting the search, and sending the image data. It becomes possible to obtain the information of the object of interest or the object to be investigated by using the mobile terminal. On this occasion,
The user can easily obtain accurate information of the object by a simple operation of sending image data without performing a special operation. Further, since the object can be specified with high accuracy based on the photographing position and the distance information, it becomes possible to appropriately provide the information desired by the user.

Further, in the information providing system for photographed images using the portable terminal according to the invention of claim 2, in the invention according to claim 1, the distance information acquiring means is an object detected by a distance measuring sensor. Is used as the distance information.

According to the second aspect of the present invention, the distance information can be obtained with a simple structure, the distance information can be acquired by using the structure of the digital camera or the like, and the information regarding the object can be easily obtained. You will be able to obtain it with high accuracy.

Further, in the information providing system for photographed images using the portable terminal according to the invention of claim 3, in the invention according to claim 1, the distance information acquisition means is a focal length at the time of photographing the object. Information is used as the distance information.

According to the third aspect of the present invention, the distance information can be obtained with a simple structure, and the distance information can be acquired by using the structure of the digital camera or the like to easily and easily obtain the information about the object. You will be able to obtain it with high accuracy.

Further, in the information providing system of the photographed image using the portable terminal according to the invention of claim 4, in the invention according to any one of claims 1 to 3, the searching means is
When a plurality of search results are output during the search, the search candidates that list each search result are output to the mobile terminal, and the search candidates are narrowed down based on the selection of the search candidates by the mobile terminal. Characterize.

According to the invention of claim 4, even when a plurality of similar search candidates are generated in the search of the image data, the information desired by the user can be easily obtained by the narrowed search. In this case, the list of search results can be used to easily narrow down.

Further, in the information providing system for photographed images using the portable terminal according to the invention of claim 5, in the invention according to any one of claims 1 to 4, the searching means is:
The object is classified into a plurality of fields in advance, and the corresponding field is designated and accessed in the search database based on the field designation by the mobile terminal.

According to the invention of claim 5, the efficiency of the search process and the search accuracy can be improved only by the user designating the field.

Further, in the information providing system of the photographed image using the portable terminal according to the invention of claim 6, in the invention according to any one of claims 1 to 5, the searching means is
It is characterized in that the search result is composed of basic information indicating an outline and detailed information transmitted based on selection of the mobile terminal with respect to the basic information.

According to the sixth aspect of the present invention, the output of the search result can be hierarchized, more detailed information can be obtained after the necessary information is determined, and the search operation can be facilitated and obtained. You will be able to examine the search results in as much detail as you need.

Further, in the information providing system for photographed images using the portable terminal according to the invention of claim 7, in the invention according to claim 6, the searching means is information related to the basic information as the detailed information. It is characterized in that a destination link having "" is searched and transmitted to the mobile terminal.

According to the invention of claim 7, it becomes possible to access the provider link as a search result, and obtain a wide range of information published on the Internet based on the photographing of the object. Will be able to.

According to the invention of claim 8, there is provided a photographed image information providing system using a portable terminal, in the invention according to any one of claims 1 to 7, the photographing means is:
It is characterized in that the object is photographed independently of the mobile terminal and the photographed data can be transferred to the mobile terminal.

According to the invention of claim 8, the object can be photographed by a general-purpose digital camera, and the information of the image data can be retrieved using a portable personal computer or the like. Can be provided.

According to a ninth aspect of the present invention, in a photographic image information providing system using a portable terminal, in the invention according to any one of the first to eighth aspects, the information sending means is set in advance by a user. It is characterized in that the search result is sent to a predetermined computer device.

According to the invention of claim 9, the search result can be received not only by the mobile terminal but also by the computer device at home, and the search service in a form corresponding to the user's request such as the communication environment and the arrangement of the search result. Will be able to provide.

Further, the information providing system for photographed images using the portable terminal according to the invention of claim 10 provides the information providing system of claims 1-9.
In the invention described in any one of the above, the position information acquisition means of the mobile terminal sends the direction on the map detected by the gyro at the time of shooting the object as a part of the shooting position information, The photographing position specifying means of the center specifies the photographing place of the image data including the direction.

According to the tenth aspect of the present invention, since the search processing is performed based on the more detailed position information, it is possible to improve the efficiency of the search processing and improve the search accuracy.

An information providing system for a photographed image using a portable terminal according to the invention of claim 11 provides the information providing system of claims 1 to 1.
0. In the invention according to any one of 0, the mobile terminal or the photographing means includes an editing means for editing the image data in order to extract only the object from the photographed image data. Image data is sent to the center.

According to the eleventh aspect of the present invention, by editing the image data before starting the search, it becomes possible to obtain the information suitable for the object in detail and accurately. The editing operation itself may be a simple operation such as leaving the object and deleting the periphery.

A photographic image information providing system using the portable terminal according to the twelfth aspect of the present invention is the first aspect of the present invention.
In the invention described in any one of 1, the mobile terminal is a mobile phone device equipped with a digital camera.

According to the twelfth aspect of the present invention, the object can be photographed in a state that it is easy to carry, and the required object can be photographed immediately by a simple operation when needed at the place of going, etc., and can be searched after photographing. All requests and retrieval results can be easily obtained using the same mobile phone.

Further, the information providing system for photographed images using the portable terminal according to the invention of claim 13 is defined by claims 1 to 1.
In the invention described in any one of 2 above, the portable terminal captures a plurality of image data or moving images of the same object, and the search means of the center stores the plurality of image data or moving images. It is characterized by executing a search process based on the above.

According to the thirteenth aspect of the present invention, by increasing the image data of the object used at the time of retrieval,
The search accuracy can be improved. At this time, the user can improve the search accuracy by an operation of increasing the number of shots or a simple operation of shooting a moving image.

According to a fourteenth aspect of the present invention, there is provided a photographed image information providing method, a photographing step in which a user photographs an object to obtain image data, and photographing position information on a map at the time of photographing the object. And a distance information acquisition step of acquiring distance information up to the object when the object is photographed, image data of the object photographed, photographing position information and distance information Based on
It is characterized by including a search step of searching for information on the photographed object and a search result output step of outputting the search result.

According to the fourteenth aspect of the present invention, the retrieval result of the information of the object can be obtained only by photographing the object and requesting the retrieval, and the object of interest or the object to be investigated can be obtained. Information can be easily obtained. At this time, since the object can be specified with high accuracy based on the shooting position and the distance information, the information desired by the user can be appropriately provided.

Further, in the information providing method of the photographed image according to the invention of claim 15, the image photographed by the user is transmitted from the portable terminal to the center via the network, and the information about the image data is transmitted from the center via the network. In a method of providing information of a captured image using a mobile terminal that receives the provision, the mobile terminal includes a capturing step in which a user captures an image of an object and obtains image data, and an image capturing position on a map when capturing the object. A position information acquisition step of acquiring information,
A distance information acquisition step of acquiring distance information to the object when the object is imaged, and a search request step of sending image data of the object, the shooting position information and the distance information to the center. And a search result output step of receiving a search result returned from the center, wherein the center uses the shooting position information and the distance information sent from the mobile terminal to determine a shooting place of the image data. A photographing position specifying step of specifying, a searching step of accessing the search database to search for information related to the specified photographing position and the image data as a search item, and the search result in the portable form. And a search result sending step of outputting to a terminal.

According to the fifteenth aspect of the present invention, the user can obtain the search result of the information of the target object only by photographing the target object, requesting the search, and sending the image data. It becomes possible to obtain the information of the object of interest or the object to be investigated by using the mobile terminal. At this time, the user can easily obtain accurate information of the object by a simple operation of sending image data without performing a special operation. Further, since the object can be specified with high accuracy based on the photographing position and the distance information, it becomes possible to appropriately provide the information desired by the user.

According to a sixteenth aspect of the present invention, there is provided a photographed image information providing method according to any one of the fourteenth and fifteenth aspects, wherein the distance information acquisition step is an object detected by a distance measuring sensor. Is used as the distance information.

According to the sixteenth aspect of the present invention, the distance information can be obtained with a simple structure, and the distance information can be acquired by using the structure of the digital camera or the like to easily and easily obtain the information about the object. You will be able to obtain it with high accuracy.

According to a seventeenth aspect of the present invention, there is provided a photographed image information providing method according to any one of the fourteenth and fifteenth aspects, wherein the distance information acquisition step is a focal length when the object is photographed. Information is used as the distance information.

According to the seventeenth aspect of the present invention, the distance information can be obtained with a simple structure, and the distance information can be acquired by using the structure of the digital camera or the like to easily and easily obtain the information about the object. You will be able to obtain it with high accuracy.

The information providing method of photographed images according to an eighteenth aspect of the present invention is the method according to any one of the fifteenth to seventeenth aspects, wherein the searching step includes a plurality of searches during processing of the searching step. When the results are output, the search candidates in which the respective search results are listed are output to the mobile terminal, and the search candidates are narrowed down and searched based on the selection of the search candidates by the mobile terminal.

According to the eighteenth aspect of the present invention, even if a plurality of similar search candidates are generated in the image data search, it is possible to easily obtain the information desired by the user by the narrowed search. In this case, the list of search results can be used to easily narrow down.

According to a nineteenth aspect of the present invention, there is provided a photographed image information providing method as defined in any one of the fifteenth to eighteenth aspects, wherein the searching step is performed by preliminarily classifying the object into a plurality of fields. It is characterized in that the user classifies and accesses the database for search by designating the corresponding field based on the field specification by the mobile terminal.

According to the nineteenth aspect of the present invention, the efficiency of the search process and the search accuracy can be improved only by the user designating the field.

According to a twentieth aspect of the present invention, there is provided a photographed image information providing method according to any one of the fifteenth to nineteenth aspects of the present invention, wherein the search step is a basic procedure for summarizing the search results. It is characterized by including information and detailed information transmitted based on the selection of the mobile terminal with respect to the basic information.

According to the twentieth aspect of the present invention, the output of the search result can be hierarchically structured, more detailed information can be obtained after the necessary information is determined, and the search operation can be facilitated and obtained. You will be able to examine the search results in as much detail as you need.

A method of providing information of a photographed image according to the invention of claim 21 is the same as that of the invention of claim 20,
In the searching step, a providing destination link having information related to the basic information as the detailed information is searched and sent to the mobile terminal.

According to the invention of claim 21, it becomes possible to access the provider link as a search result, and obtain a wide range of information published on the Internet based on the photographing of the object. Will be able to.

According to a twenty-second aspect of the present invention, there is provided a method for providing information of a photographed image, wherein in the invention according to any one of the fifteenth to twenty-first aspects, the photographing step is independent of the portable terminal. In the step of photographing the object,
It is characterized by including a step of transferring photographing data to the mobile terminal.

According to the twenty-second aspect of the present invention, the object can be photographed with a general-purpose digital camera, and the information of the image data can be retrieved using a portable personal computer or the like, and the information can be obtained using a handheld device. Can be provided.

According to a twenty-third aspect of the present invention, there is provided a photographed image information providing method according to any one of the fifteenth to twenty-second aspects, wherein the search result sending step is a computer device predetermined by a user. Is sent to the search result.

According to the twenty-third aspect of the present invention, the retrieval result can be received not only by the mobile terminal but also by the computer device at home, and the retrieval service in a form corresponding to the user's request such as the communication environment and the arrangement of the retrieval result. Will be able to provide.

Further, a method of providing information of a photographed image according to the invention of claim 24 is the invention according to any one of claims 15 to 23, wherein the position information acquisition step is performed when the object is photographed. The direction on the map detected by the gyro is transmitted as part of the photographing position information, and the photographing position specifying step specifies the photographing place of the image data including the direction.

According to the twenty-fourth aspect of the present invention, since the search processing is performed based on the more detailed position information, the search processing can be made more efficient and the search accuracy can be improved.

According to a twenty-fifth aspect of the present invention, in the information providing method of the photographed image, in the invention according to any one of the fifteenth to twenty-fourth aspects, only the object is extracted from the photographed image data. In order to do so, an edit step is provided for editing the image data, and the edited image data is sent to the center.

According to the twenty-fifth aspect of the present invention, by editing the image data before starting the search, it becomes possible to obtain the information suitable for the object in detail and accurately. The editing operation itself may be a simple operation such as leaving the object and deleting the periphery.

The information providing method of the photographed image according to the invention of claim 26 is the invention according to any one of claims 15 to 25, wherein the photographing step is performed by a plurality of images for the same object. Data or a moving image is photographed, and the searching step executes a search process based on the plurality of image data or moving images.

According to the twenty-sixth aspect of the present invention, by increasing the image data of the object used at the time of retrieval,
The search accuracy can be improved. At this time, the user can improve the search accuracy by an operation of increasing the number of shots or a simple operation of shooting a moving image.

Further, the program according to the invention of claim 27 can cause a computer to execute the method described in any one of claims 14 to 26.

[0060]

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of a captured image information providing system, a captured image information providing method, and a program for causing a computer to execute the method will be described below with reference to the accompanying drawings. Embodiments will be described in detail.

(Structure of Photographed Image Information Providing System) First, the structure of the photographed image information providing system according to the embodiment of the present invention will be described. FIG. 1 is a system configuration diagram showing the overall configuration of a captured image information providing system according to an embodiment of the present invention. In FIG.
Reference numeral 101 is a mobile terminal such as a mobile phone or a mobile terminal carried by a user, and 102 is an information providing device (information providing server) installed in the center of a business operator such as an information providing company. They are connected by a network 100 such as a telephone network.

The mobile terminal 101 has a digital camera 104.
Is built in or is configured to be connectable externally, an image of an object is captured using the digital camera 104, and the image data of the captured object is transferred to the mobile terminal 101. The image data transferred to the mobile terminal 101 is transferred to the information providing apparatus 102 via the network 100. At this time, in addition to the image data, the shooting time, position information, and distance information to the object at the time of shooting are also sent to the information providing apparatus 102. These shooting time, position information, and distance information can be performed without the user's operation, and the user only needs to operate the image data transmission.
Operability can be improved. The operation-transferred image data is temporarily stored in the information providing device 102 to search the detailed information of the image data. The search result is sent to the mobile terminal 101 via the network 100.

A terminal device 105 such as a personal computer (PC) installed at the user's home or the like is connected to the network 100, and the information providing device 10 is connected as necessary.
It is structured so that the information provision of 2 can be received at home.

(Hardware Configuration of Photographed Image Information Providing System) Next, the hardware of the portable terminal 101, the information providing device 102, and the terminal device 105 constituting the photographed image information providing system according to this embodiment of the present invention. The ware configuration will be described. FIG. 2 is a portable terminal 1 that constitutes a captured image information providing system according to the embodiment of the present invention.
1 is a block diagram showing the hardware configuration of 01, the information providing device 102, and the terminal device 105.

In FIG. 2, 201 indicates a CPU for controlling the entire apparatus, 202 indicates a ROM storing a basic input / output program, and 203 indicates a RAM used as a work area of the CPU 201.

Reference numeral 204 denotes an HDD (hard disk drive) that controls reading / writing of data from / to an HD (hard disk) 205 under the control of the CPU 201, and 205 denotes an HD that stores written data under the control of the HDD 204. Shown respectively. In addition, 206 is F under the control of the CPU 201.
Reference numeral 207 denotes an FDD (flexible disk drive) that controls reading / writing of data from / to a D (flexible disk) 207, and reference numeral 207 denotes a removable FD that stores data written under the control of the FDD 206.

Reference numeral 208 is a display for displaying various data such as cursors, menus, windows, characters and images (for example, CRT, TFT liquid crystal display, plasma display, etc.), and 209 is the network 100 via the communication line 210. I / F (interface) connected to the network 100 and functioning as an interface between the network 100 and the CPU 201. Reference numeral 211 denotes a keyboard having a plurality of keys for inputting characters, numerical values, various instructions, etc.
Reference numeral 12 denotes a mouse for selecting and executing various instructions, selecting a processing target, and moving a cursor.

Reference numeral 213 is a scanner for engineeringly reading characters and images, 214 is a printer for printing characters and images under the control of the CPU 201, 215 is a removable recording medium CD-ROM, and 216 is a recording medium. CD-R
CD-R that controls the reading of data to OM215
The OM drive 217 is a speaker (including headphones and earphones) for outputting voice and music,
Reference numeral 8 is a tablet for inputting handwritten characters. And
The above components are connected by a bus 200.

(Functional Configuration of Photographic Image Information Providing System) Next, the functional configuration of the photographed image information providing system according to the embodiment of the present invention will be described. Figure 3
FIG. 1 is a block diagram showing a functional configuration of a captured image information providing system according to an embodiment of the present invention.

(Mobile Terminal 101) In FIG. 3, the user holds the mobile terminal 101 and mainly uses it on the go. A personal computer (PC) 1 is installed in the user's house.
05 is installed. The digital camera 104 is externally connected to the mobile terminal 101. Alternatively, the digital camera 10
It can be configured using the mobile phone 301 having the built-in four functions. When the user takes an image of the object 300 of interest at the destination or the like using this digital camera 104, the image data of the object 300 is stored in the memory (not shown) in the mobile terminal 101.

In the portable terminal 101, a built-in position detecting means (not shown) detects a radio wave transmitted from the GPS satellite 303 to obtain current position information. This position information obtains the latitude and longitude on the map, and can be displayed as map information (map) of surrounding roads / buildings on the display screen of the mobile terminal 101 as necessary.

The digital camera 104 is constructed so that the built-in distance measuring means (not shown) detects the distance to the object 300 and obtains distance information at the time of photographing. As the distance measuring means, a distance measuring sensor such as an infrared sensor built in the digital camera 104 is used. In addition to this, it is also possible to drive the autofocus mechanism at the time of shooting and use the focus information when focusing on the object 300. The distance information to the object 300 detected by the digital camera 104 is
It is sent to the mobile terminal 101 together with the image data at the time of shooting.

Acquisition of position information when the digital camera 104 is externally connected to the portable terminal 101 is as follows.
The digital camera 104 captures the object 300 near the mobile terminal 101. Digital camera 104
If the mobile terminal 101 has a built-in, there is no such limitation.
Further, the digital camera 104 may be provided with a position detecting means to send position information at the time of shooting to the mobile terminal 101.

The mobile terminal 101 can request the information provision of the photographed image (information provision request) and obtain the information provision by performing a predetermined operation after photographing the object 300. The information provision request, the image data captured by operating the mobile terminal 101, and the position information and distance information are sent to the information providing device 102 of the business operator via the network 100. The information providing apparatus 102 searches the mobile terminal 101 (or PC 1) for detailed information regarding the captured image based on the image data, the position information, and the distance information.
This detailed information is sent to 05).

(Information providing device 102) Information providing device 10
2 includes a plurality of search databases as shown. The search database is an image database (DB) 310 that stores and holds detailed image information, a place database (DB) 311 that stores and holds detailed position information, and a link related to the object identified by the detailed information about the image and position. An information database that stores and stores information (D
B) It consists of 312 etc.

These search databases are search means 3
It is accessed during the search process of 15. Search means 315
Is an image search means 31 for accessing the image DB 310.
6, a place search means 31 for accessing the place DB 311
7. Information retrieval means 318 for accessing the information DB 312
Have.

When an information providing request is input via the network 100, the searching means 315 stores the input image data, position information and distance information in the temporary storage means.

The place searching means 317 receives the position information and the distance information and accesses the place DB 311 to search the shooting place to obtain the corresponding shooting place. At this time, the shooting position is specified by searching the latitude and longitude on the map with reference to the input position information. In addition, the target object 3 based on the distance information
A distance to 00 is specified and an object included in a distance range (imaging range) centered on the imaging position is searched. When there are a plurality of search results, the plurality of search candidates are listed.

The image search means 316 receives the input image data, accesses the image DB 310 to search the image contents, and obtains detailed information of the object 300 indicated by the image.
At the stage where there are multiple search targets, these multiple search candidates are listed. At this time, the place search means 317
Of the search target of the object 300 included in the shooting range is used as a part of the search condition.

The search means 315 uses the search means 316 described above.
To 317, and controls the procedure related to image information search with the mobile terminal 101. The processing function of the search means 315 is as follows: The detailed information of the image obtained by the search processing of the image search means 316 is sent to the mobile terminal 101. 2. When a plurality of search results are obtained, these search results are sent as search candidates to the mobile terminal 101 via the network 100, and a narrowed search is performed by operating the mobile terminal 101. 3. An information menu such as the procedure at the time of the search process is sent to the mobile terminal 101, and the search process based on the selected information menu is executed. 4. Information search means 31 during and after the search process
The destination link of the related information related to the detailed information provided by 8 is sent to the mobile terminal 101.

The information retrieving means 318 accesses the information database 312 and sends to the portable terminal 101 a link for providing the related information related to the detailed information of the image obtained by the retrieval process. The information DB 312 stores an index of an external destination link related to detailed information. As shown in the figure, the destination link is the broadcasting station 32.
0, publisher 321, newspaper publisher 322, network 100
The information providing site 323 and the like connected to the. By this information search, it becomes possible to acquire the related information in addition to the detailed information of the object 300.

Further, the search means 315, when performing the narrow-down search processing, has the location DB 311 and the image DB 31.
Not limited to 0, the information DB 312 can be accessed to perform a narrowed search. At this time, the search unit 315 can access the external provider link, such as the broadcasting station 320, the publisher 321, the newspaper publisher 322, etc., to obtain detailed information related to the object 300.

Detailed information provided by the destination link is as follows: 1. Program, image, sound, and character data included in the program, article about the object 300 already broadcast, published, and published by a broadcasting station, radio station, newspaper company, or publisher 2. 2. Image, voice, and character data regarding the object 300, which is open to the public on the information providing site 323 on the Internet. The information includes information on other provider links related to the detailed information of the object 300.

The information retrieval means 318 is, specifically,
For example, the ROM 202, RAM 203, HD shown in FIG.
205, FD 207, CD-ROM 215 and other programs (application software) recorded in C
The function is realized by the execution of the PU 201.

(Structure of Mobile Terminal 101 for Image Shooting) FIG. 4 is a diagram showing a specific structure of the mobile terminal. In the illustrated example, the digital camera 104 is used as the mobile terminal 101.
This is an example using a mobile phone 301 having a built-in. The mobile phone 301 includes a display screen 401 for displaying a captured image and detailed information at the time of providing information, and an operation button 402 for operating the mobile phone 301. By operating the photographing button 402a with a photographing function assigned to a specific button 402a of the operation buttons 402, the mobile phone 30
An imaging unit (not shown) provided in No. 1 captures the object 300. The captured object 300 is displayed as an image on the display screen 401.

(Basic processing contents concerning information provision of photographed image) Next, processing concerning information provision until the object 300 is photographed using the portable terminal 101 having the above-mentioned configuration and detailed information of the object 300 is obtained. Explain the contents. FIG. 5 is a flowchart showing the overall processing procedure for providing information on a captured image. In the following description, it is assumed that the object 300 is a stationary object.

The user is the target object 3 that is interesting on the go.
If 00 is found, the object 300 is photographed by the digital camera 104 (step S501).

At the same time as the photographing, the photographing time and the distance information to the object 300 detected by the distance measuring sensor,
The position information detected by the GPS satellite 303 is obtained. The position information and the distance information are added to the image data (step S502).

The user transfers the image data of the object 300 photographed by the digital camera 104 to the portable terminal 101 (step S503). The transfer method can be performed via a cable connection, a memory card, or communication such as Bluetooth. At this time, if the mobile phone 301 with the built-in digital camera 104 is used, connection and transfer processing is unnecessary.

When the user wants to obtain detailed information of the photographed object 300, the user makes a search request to the information providing apparatus 102. Then, the image data is transmitted from the mobile terminal 101 to the information providing apparatus 102 via the network 100 (step S504). Shooting time, position information, and distance information are added to this image data.

Search means 315 of the information providing apparatus 102
Searches for the object indicated by the image data based on the image data, the position information, and the distance information received from the mobile terminal 101 (step S505). Location DB3 at the time of this search
11, the image DB 310, the information DB 312 is accessed to obtain the search result (basic information) of the object 300 included in the captured position (distance range) on the map. When a plurality of search results are listed instead of one, a headline (index) of each search result is generated.

The information providing device 102 includes the object 300.
The basic information indicating the outline of the search results of the user's mobile terminal 1
Send to 01. When a plurality of search results are listed, the generated indexes are listed and sent to the user's mobile terminal 101. This list is configured by appropriately combining images, voices, and characters, and prompts the user to select (step S50).
6).

The user confirms whether the basic information of the search result received by the portable terminal 101 matches the desired one, and instructs the information providing apparatus 102 to confirm the confirmation result (step S507). When the search candidate is sent, the optimum one is selected from the listed indexes and returned to the information providing apparatus 102.

When the user confirms the basic information on the portable terminal 101 or after confirming the basic information through the selection operation of the index list, the information providing apparatus 102 carries the detailed information in which the basic information is further detailed. It is sent to the terminal 101 (step S508). The user obtains detailed information of the object 300 with the mobile terminal 101.

When the charging contract is made for the information provision, the business operator uses the charging system of the information providing apparatus 102 to perform the charging process corresponding to the information provision this time, and ends the processing (step S509).

Through the series of search processing described above, the user can immediately obtain the search result of the target object 300 that he / she is interested in or wants to check on the spot. This information providing service enables the business operator to expand service business such as mobile phones.

(Details of Retrieval Process Designated by Field-No. 1) Next, a processing procedure performed by designating a field at the time of the above-described search processing will be described. FIG. 6 is a diagram for explaining the field specifying process of the search target in the mobile terminal 101, and FIG. 7 is a flowchart showing a processing procedure of information provision of a captured image when the field of the search target is specified. The steps shown in FIG. 7 are basically the same as the steps shown in FIG. 5, the same steps as those in FIG. 5 are designated by the same step numbers, and only new steps are described. is there.

After step S503, the portable terminal 1
When 01 accesses the information providing apparatus 102, the information providing apparatus 102 displays the search target selection screen 60 shown in FIG.
0 is transmitted to the mobile terminal 101 (mobile phone 301) (step S701). This selection screen 600 shows the object 3
Displayed to specify the field when searching for 00.
The user uses the operation button 402 of the mobile terminal 101 to specify each field such as a human / animal / plant / vegetable system, a building system, a sound system, and a vehicle system as shown in the figure (step S702). The information on the designated field is sent to the information providing device 102 together with the image data at the time of the search request (step S).
504).

As a result, the information providing apparatus 102 narrows down the designated field as the search target in the search process (step S505), and starts the search process for the object 300. As a result, the processing time and processing efficiency required for the search processing and the search accuracy can be improved.

(Example of Processing for Specifying Field 1-Building System) FIG.
FIG. 9 is a diagram for explaining a process progress when a building system is designated as a field at the time of search, and FIG. 9 is a flowchart showing a processing procedure of providing information of a photographed image when a field is designated at the time of search.
At the time of shooting (step S501),
“Building system” is specified in the field specification shown in FIG. 6 (step S
702), image data (including position information and distance information) is sent to the information providing apparatus 102 (step S50).
4).

As a result, the information providing apparatus 102 searches for the object 300. After that, the processes of steps S505 to S508 of FIG. 5 are basically executed. The processing contents of each of these steps when the field is designated will be described in more detail. First, the basic information 8 of the search result shown in FIG.
00 is transmitted (display output) to the display screen 401 of the mobile phone 301 (step S901). Basic information 8 in the illustrated example
00 includes the name of the photographed object 300 (building), the designer's name, and the construction company. In addition, an address, a telephone number, etc. can be configured to be displayed by scrolling the screen.

After the acquisition of the basic information 800, the desired basic information 800 is designated by an item (step S
902), the information providing apparatus 102 sends out the detailed information index 801 shown in FIG. 8B (step S903).
When the index 801 of the desired detailed information is specified by the mobile phone 301 (step S904), the corresponding detailed information can be transmitted and displayed on the mobile phone 301 (step S905).

Further, in the detailed information index 801, a search database of the information providing apparatus 102 (image DB 310,
Not only the detailed information stored in the location DB 311) but also link information to the destination link (broadcasting station 320, publisher 321, newspaper publisher 322, information providing site 323, etc.) is included via the information DB 312. By this, step S
At 903, the link to the destination of the "Haneda Airport Building", which is the object 300, is displayed as a part of the detailed information index 801 (FIG. 8B), and is selected (step S904), and step S905. Then, it becomes possible to access the detailed information 802 ("Haneda Airport homepage" which is the information providing site 323) shown in FIG. 8C. After this, the process of step S509 in FIG. 7 is processed and terminated.

(Processing example of field designation 2-vehicle system) FIG.
FIG. 6 is a diagram for explaining a process progress when a vehicle system is designated in a field at the time of search. 10A shows the display state of the basic information 800 of the search result, FIG. 10B shows the display state of the detailed information index 801, and FIG. 10C shows the display state of the detailed information. . Detailed information index 801
9 is different only in that items related to vehicles are displayed, and even when the field of search is specified in the vehicle system, FIG.
The search processing is executed in the same processing procedure. Note that FIG.
Detailed information (and basic information) 802 as shown in (c)
The output of is not limited to images, characters, etc. using the display screen 401, and may be configured to be output with the voice content 1000 from the speaker of the mobile phone 301.

(Search Processing Considering Shooting Time) During the above search processing, the shooting time information is sent from the mobile terminal 101 (mobile phone 301) in addition to the image data, position information, and distance information. . This shooting time can also be added to the items to be searched. As a result, the search result regarding the object 300 photographed at the photographing location at the photographing time can be made more accurate. Also, detailed information can be obtained in more detail.

For example, in the above-mentioned field "vehicle system",
It becomes possible to provide more basic information about the vehicles passing through the shooting location at the shooting time. Figure 10
In the example of (a), the basic information about the relevant airplane 80
As 0, not only the model name of the airplane but also the airline that is flying the shooting location at that time, the flight number, and the destination can be obtained. In this way, by adding the photographing time as an item at the time of search, it is possible to improve the search accuracy even for the object 300 that moves over time in the field “vehicle system”.

Further, as shown in FIG. 10B, the detailed information index 801 can also index the airline company and flight number based on the basic information 800 obtained at the photographing time. Correspondingly, the detailed information can also specify the object 300 and provide more detailed information. For example, when the flight number is selected in the detailed information index 801 shown in FIG. 10B, the voice content 100 of the detailed information shown in FIG.
With 0, timetable information such as flight schedule (departure point, departure time, arrival point, arrival time, etc.) for this flight number can be provided, and appropriate and detailed details can be obtained even for the object 300 that moves over time. The information 802 (sound content 1000) can be obtained.

Other objects 300 that move over time in the field "vehicle system" include trains, buses, ships, etc.,
By adding the shooting time to each item as a search item, more detailed information can be provided.
The detailed information 802 itself can also guide more appropriate provided links 320 to 323 based on the specified flight number and train name. It goes without saying that the information obtained at this time is not limited to the present information but includes future (future) schedule information.

Even if the object is a stationary object, for example, a "building system" building, it is possible to effectively utilize the photographing time and provide appropriate information. For example, it becomes possible to obtain information on temporary halls, stages, buildings under construction, etc. for a limited time.

(Operation by field designation at the time of retrieval) By the way, at the time of photographing, unnecessary images other than the desired object 300 may be photographed in the same image photographed by the digital camera. For example, in the example shown in FIG. 4, images of three fields, “building system”, “vehicle system”, and “human / animal / plant / vegetable system”, which are different in search fields, are captured in the same image. ing. Therefore, the object 300
After the image is captured, the information providing apparatus 102 specifies the field before the search processing is started by the information providing apparatus 102.
Can extract the object 300 in the specified field in the single image data and perform the search process.

(Structural Example 1 for Extracting Object from Image Data) In addition, after the user has photographed the object 300, only the field in which the user wants to check the image data on the display screen of the portable terminal 101 to be searched The image data may be edited as shown in FIG. FIG. 11 is a diagram showing a shooting state of an object including a plurality of fields. When the images shown in this figure are taken, a building-based object 300a, a vehicle-based object 300b, a human / animal / plant-based object 300c.
The composition is included in the same image.

Even in such a case, the information providing apparatus 102 can detect the object 300 based on the distance information at the time of photographing.
Can be specified. For example, since the distance information to the foremost person / animal / plant type object 300c is short, an image of the person / animal / plant type object 300c located closest to this side is based on this distance information. Only the data is extracted, the search process is executed, and the search targeting the object 300c can be performed.

(Structural Example 2 for Extracting Object from Image Data) Further, the digital camera 104 is not limited to the distance information.
By transmitting the focal length information (focus information) of the object from the mobile terminal 101 to the information providing apparatus 102, the clearest image data is extracted by the edge detection or the like of the image processing by using the focal length information together. Can also be extracted.

Further, since the general-purpose digital camera 104 has a structure in which the center position of the photographed image is in focus, image processing such as projection is performed on the image data with the center of the image data as a reference, and the object 300c is removed. It is also possible to perform a search process by extracting and specifying the target object 300c. When the other objects 300a and 300b are focused at the time of photographing, these other objects 300a and 300b are extracted based on this focus information.

(Structural Example 3 of Extracting Object from Image Data) A simpler and more reliable method for extracting object 300 is to edit the image data by the user operation on the portable terminal 101 side. It is configured to operate. Figure 11
As an editing operation for the captured image data shown in
An operation of extracting only the target object 300 in the field to be searched is performed. Alternatively, the enlargement / reduction processing is performed so that the target object 300 in the field to be searched becomes the largest ratio with respect to the image size.

FIG. 12 is a diagram showing a state after the image data editing process. If “human / animal / plant / vegetable system” is desired as the field to be searched, the target object 30 in the image data
Image editing is performed by deleting the peripheral image (or cutting only the center and pasting it as new image data) so as to extract only the substantially central portion where 0c (person) is located. Image data of a person). At this time, the edited image data may be scaled up or down as necessary. In addition, by performing a size reduction process on an image with a necessary portion as the center of the image, it is possible to perform a process of moving an unnecessary image around the image outside the image size frame.

Then, the "person / animal / vegetable system" is designated by the field on the portable terminal 101, and the image data of the object 300c is transmitted. As a result, the information providing apparatus 102 can perform a search by narrowing down the field to "human / animal / plant / vegetable system" from the start of the search, and even at the time of the search, the object 300c is deleted based on the image data in which unnecessary peripheral images are deleted. The feature extraction can be easily performed, and the efficiency of the search processing and the search accuracy can be improved.

(Processing Example 3 for Specifying Fields-Human / Animal / Plant System) FIG. 13 is a diagram for explaining the processing progress when a field is specified for a human / animal / plant system during retrieval. FIG. 13 (a),
(B) is a display state of the basic information 800 of each search result, and when the search result is limited to race identification as in (a), an individual may be identified as in (b). In the case of (b), a celebrity or a celebrity can be specified.

When an individual can be specified, the display state of the detailed information index 801 shown in FIG. 13C is obtained. In this case, the detailed information index 801 displays items related to the specified individual. The above search processing can be executed by the same processing procedure as in FIG. At this time, the target object 30
As 0, the processing efficiency and the search accuracy are improved by applying the respective processes described in (Configuration examples 1 to 3 for extracting the target object from the image data) in order to extract the person (target object) 300c. You can also let it.

(Processing Example 4 for Specifying Field 4-Sound System) FIG.
It is a figure for explaining the processing progress when the sound system is designated at the time of a search. FIG. 14A shows the display state of the basic information 800 of the search result, and FIG. 14B shows the detailed information index 80.
1 is a display state, and FIG. 14C is a diagram showing an output state of the detailed information 802. In the case of sound system, sound is recorded by the microphone provided in the mobile terminal 101. At this time, the sound-producing target object 300 is simultaneously photographed, and these sounds, image data, position information, distance information, and photographing time (recording time) are sent from the mobile terminal 101 to the information providing apparatus 102 to request a search. . It is also possible to make a search request to the information providing apparatus 102 excluding the image data.

In the illustrated example, the recorded sound is a musical piece,
As shown in FIG. 14A, the basic information 800 is a state in which the search results of the song, the lyricist, the composer, and the artist (singer) are obtained. After this, in the detailed information index 801, “1.
By selecting "Watch this promotion video", detailed information 802 can be obtained by accessing the corresponding information destination link (providing site 323 or the like). As the detailed information 802, there is a case where a moving image with sound such as a promotion video is distributed as shown in the figure.

(Regarding Utilization of Shooting Direction) In the above-described embodiment of the present invention, the portable terminal 101 is the target object 30.
The image data of 0, the position information at the time of shooting, the distance information, and the shooting time are sent to the information providing device 102. In addition to these, a configuration can also be adopted in which direction information at the time of shooting is also transmitted. The direction at the time of shooting can be obtained by the gyro provided in the mobile phone 301. Information providing device 102
After specifying the distance range (shooting range) around the shooting position on the map based on the position information and distance information, this direction information can detect the shooting direction (range) on the map. It becomes possible to improve the search processing efficiency by searching the target object included in.

(Another example of information provision destination-PC installed at home
(Etc.) In the processing step of the search processing described with reference to FIG. 5, the information providing apparatus 102 is configured to send detailed information to the mobile terminal 101 that has issued the search request. Not limited to this,
The configuration may be such that it is sent to the PC 105 (see FIG. 1) at the user's home or the like. Mobile terminal 1 when requesting a search for processing steps
You may add the process which specifies the transmission destination of a search result using 01.

For example, the detailed information 80 on the home PC 105
By obtaining 2, the communication environment (communication speed, communication charge, etc.) of the PC 105, the amount of data of the detailed information 802, the organization of search results, etc. can be performed at home, and the convenience of the search service can be improved. . Information providing device 102
Stores a plurality of image data previously sent from the mobile terminal 101, creates an album by using these, and PC1
It is also possible to have a configuration in which it is sent to 05.

As the detailed information (index) 802 described above, besides, advertisements of companies, stores, broadcast programs of television / radio, articles of newspapers / magazines, voices of users (repeaters, persons familiar with the region / industry). It can be linked to a wide range of information.

(Regarding Retrieval Process Using Plural Image Data or Video) In the above embodiment, the object 300 is photographed only once and a single image data is sent to the information providing device 102. The configuration for obtaining the search result has been described. In order to improve the search accuracy of the object 300, it is effective to take a plurality of images of the same object 300 at different angles and shooting magnifications. The plurality of pieces of image data are sent to the information providing apparatus 102 at the time of a search request, and the next image data is sent in the course of the narrowing search processing of the information providing apparatus 102.

At this time, the information providing device 102 may be configured to specify the photographing angle and the photographing place of the object 300 which is effective for specifying the object 300. Furthermore, not only still images but also search processing using moving images is possible. For example, the information providing apparatus 102 may generate a plurality of frame-based still images from a moving image to extract a feature, or the image DB 310.
The search is executed by comparing with the search image of.

The search method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program
Hard disk, flexible disk, CD-RO
It is recorded on a computer-readable recording medium such as M, MO, and DVD, and executed by being read from the recording medium by the computer. Further, this program may be a transmission medium that can be distributed via a network such as the Internet.

[0129]

As described above, according to the first aspect of the invention, the image photographed by the user is transmitted from the portable terminal to the center via the network, and the center relates to the image data via the network. In a captured image information providing system using a mobile terminal that receives information, the mobile terminal includes a photographing means for photographing an object and obtaining image data, and a map on a map when the object is imaged. Position information acquisition means for acquiring imaging position information, distance information acquisition means for acquiring distance information to the object when the object is imaged, image data of the object imaged at the time of a search request, Communication means for sending the photographing position information and the distance information from the portable terminal to the center, and for receiving the search result returned from the center;
The center is provided with an operation means for performing an operation necessary for the search and an output means for outputting the search result by using an image, a character, a sound, and the like. Shooting position specifying means for specifying a shooting place of the image data by using a search database in which information for searching regarding information and distance information is stored, and the shooting position information and the distance information sent from the mobile terminal. And a search means for accessing the search database to search for the specified shooting location and information related to the image data using the image data as a search item.
Since the information sending means for sending the search result obtained by the search to the mobile terminal is provided, the information of the object can be obtained only by the user photographing the object, requesting the search and sending the image data. The search result can be obtained, and the information of the object of interest or the person who wants to investigate can be obtained by using the mobile terminal. At this time, the user can easily obtain accurate information of the object by a simple operation of sending image data without performing a special operation. Further, since the object can be specified with high accuracy based on the shooting position and the distance information, the above-described effects can be provided in which the information desired by the user can be appropriately provided.

According to a second aspect of the present invention, in the first aspect of the invention, the distance information acquisition means uses the distance to the object detected by the distance measuring sensor as the distance information. Therefore, the distance information can be obtained with a simple structure, and the distance information can be acquired by using the structure of the digital camera or the like to easily and accurately obtain the information about the object.

According to the invention of claim 3, in the invention of claim 1, the distance information acquisition means uses focal length information at the time of photographing the object as the distance information. The distance information can be obtained with a simple structure, and the distance information can be acquired by using the structure of a digital camera or the like to easily and accurately obtain the information about the object.

Further, according to the invention described in claim 4, in the invention described in any one of claims 1 to 3, when the search means outputs a plurality of search results during the search, The search candidates that list each search result are output to the mobile terminal, and the search candidates are narrowed down based on the selection of the search candidates by the mobile terminal, so that a plurality of similar search candidates are generated in the image data search. Even in such a case, it is possible to easily obtain the information desired by the user by the narrowing search, and at this time, it is possible to easily perform the narrowing using the list of search results.

According to a fifth aspect of the invention, in the invention according to any one of the first to fourth aspects, the search means classifies the object in advance into a plurality of fields, Since the corresponding field is specified and accessed to the search database based on the field specification by the mobile terminal,
It is possible to improve the efficiency of search processing and the accuracy of search simply by the user designating a field.

Further, according to the invention described in claim 6, in the invention described in any one of claims 1 to 5, the searching means includes the search result, basic information showing an outline,
Since it is composed of the detailed information transmitted based on the selection of the mobile terminal with respect to the basic information, the output of the search result can be hierarchized, more detailed information can be obtained after the necessary information is fixed, and the search operation is easy. The present invention has the effect that the search results that can be obtained can be detailed and only the necessary search results can be examined in detail.

[0135] According to the invention described in claim 7, in the invention described in claim 6, the searching means searches for a provider link having information related to the basic information as the detailed information, and Since it is sent to the mobile terminal, it becomes possible to access the destination link as a search result, and it is possible to obtain a wide range of information published on the Internet based on the shooting of the object. .

[0136] According to the invention described in claim 8, in the invention described in any one of claims 1 to 7, the photographing means is independent of the portable terminal. Since the configuration is such that the object can be photographed and the photographed data can be transferred to the portable terminal, the object can be photographed with a general-purpose digital camera and the information of the image data can be retrieved using a portable personal computer or the like. Therefore, there is an effect that information can be received by using a device on hand.

According to a ninth aspect of the present invention, in the invention according to any one of the first to eighth aspects, the information sending means causes the computer to preset the search result by a user. Since the search result is sent out, the search result can be received not only by the mobile terminal but also by the computer device at home, and the search service in the form corresponding to the user's request such as the communication environment and the arrangement of the search result can be provided.

Further, according to the invention of claim 10,
In the invention according to any one of claims 1 to 9, the position information acquisition means of the mobile terminal is a part of the shooting position information indicating a direction on a map detected by a gyro at the time of shooting the object. Since the photographing position specifying means of the center specifies the photographing position of the image data including the direction, the search process is performed based on more detailed position information, so that the efficiency of the search process and the search accuracy are improved. There is an effect that it is possible to improve.

According to the invention described in claim 11,
In the invention according to any one of claims 1 to 10,
Since the portable terminal or the photographing means includes an editing means for editing the image data in order to extract only the object from the photographed image data, and sends the edited image data to the center, the search is performed. By editing the image data before the start, it becomes possible to obtain detailed and accurate information suitable for the object. There is an effect that the editing operation itself may be a simple operation such as leaving the object and deleting the periphery.

According to the invention described in claim 12,
In the invention according to any one of claims 1 to 11,
Since the mobile terminal uses a mobile phone device equipped with a digital camera, it is possible to shoot an object in a state that is easy to carry, and it is possible to immediately shoot an object that is necessary when needed at a place such as a destination, Even after shooting, a search request and a search result can all be easily obtained using the same mobile phone.

According to the invention described in claim 13,
In the invention according to any one of claims 1 to 12,
The mobile terminal captures a plurality of image data or moving images of the same object, and the search means of the center executes a search process based on the plurality of image data or moving images. By increasing the image data of the target object, it becomes possible to improve the search accuracy. At this time, the user can improve the search accuracy by an operation of increasing the number of captured images or a simple operation of capturing a moving image.

According to the invention described in claim 14,
A photographing step in which a user photographs an object to obtain image data, a position information acquisition step of acquiring photographing position information on a map at the time of photographing the object, and the object at the time of photographing the object A distance information acquisition step of acquiring distance information to an object; a search step of searching information related to the photographed object based on the image data of the photographed object, photographing position information and distance information; A search result output step of outputting a result is included, so that it is possible to obtain the search result of the information of the object only by photographing the object and requesting the search, and the object of interest or the object to be investigated You will be able to easily get the information of. At this time, since the object can be specified with high accuracy based on the shooting position and the distance information, it is possible to appropriately provide the information desired by the user.

Further, according to the invention of claim 15,
An information providing method for a captured image using a mobile terminal, wherein an image captured by a user is transmitted from a mobile terminal to a center via a network, and the center provides information on image data via the network. A photographing step in which a user photographs an object to obtain image data, a position information acquisition step of acquiring photographing position information on a map at the time of photographing the object, and a step of acquiring the object at the time of photographing the object. A distance information acquisition step of acquiring distance information to the object, image data of the photographed object,
It includes a search request step of sending shooting position information and distance information to the center, and a search result output step of receiving a search result returned from the center, wherein the center sends the shooting position information sent from the mobile terminal. A photographing position specifying step of specifying a photographing place of the image data using the distance information and the distance information, and accessing the search database to relate the specified photographing place and the image data to the image data as a search item. A search step for searching for information to be performed,
Since the search result sending step of outputting the search result to the mobile terminal is included, the user only needs to photograph the object, request the search, and send the image data, and the search result of the information of the object is obtained. Therefore, it becomes possible to obtain the information of the target object which is of interest at the time of going out or to be investigated by using the portable terminal. At this time, the user can easily obtain accurate information of the object by a simple operation of sending image data without performing a special operation. Further, since the object can be specified with high accuracy based on the shooting position and the distance information, it is possible to appropriately provide the information desired by the user.

Further, according to the invention of claim 16,
In the invention according to any one of claims 14 and 15, in the distance information acquisition step, since the distance to the object detected by the distance measuring sensor is used as the distance information, the distance information can be obtained with a simple configuration. Therefore, it is possible to obtain the distance information by using the configuration of the digital camera or the like, and to obtain the information about the object easily and accurately.

Further, according to the invention of claim 17,
In the invention according to any one of claims 14 and 15, since the distance information acquisition step uses focal length information at the time of photographing the object as the distance information, the distance information can be obtained with a simple configuration. In addition, it is possible to obtain the distance information using the configuration of the digital camera or the like, and obtain the information about the object easily and accurately.

Further, according to the invention of claim 18,
In the invention according to any one of claims 15 to 17, in the search step, when a plurality of search results are output during the process of the search step, the search candidates in which each search result is listed are the mobile terminal. Since the search candidates are narrowed down based on the selection of the search candidates by the mobile terminal, even if a plurality of similar search candidates occur in the search of the image data, the narrowed search allows the user to select the search candidates. It is possible to easily obtain the information to be obtained, and at this time, it is possible to easily perform the narrowing using the list of the search results.

Further, according to the invention of claim 19,
The invention according to any one of claims 15 to 18, wherein in the search step, the object is classified into a plurality of fields in advance, and the field corresponding to the search database is specified based on the field designation by the mobile terminal. Since the user specifies and accesses the field, it is possible to improve the efficiency of the search process and the accuracy of the search only by the user specifying the field.

Further, according to the invention of claim 20,
The invention according to any one of claims 15 to 19, wherein the search step includes basic information indicating an outline of the search result and detailed information to be transmitted based on selection of the mobile terminal with respect to the basic information. Since it has been included, the output of search results can be hierarchical, more detailed information can be obtained after the necessary information is determined, the search operation can be facilitated, and only the necessary search results can be obtained. The effect that it can be examined in detail is produced.

Further, according to the invention of claim 21,
In the invention according to claim 20, the searching step searches for a provider link having information related to the basic information as the detailed information and sends it to the mobile terminal. Therefore, the provider link is accessed as a search result. As a result, it is possible to obtain a wide range of information published on the Internet based on the shooting of the object.

Further, according to the invention of claim 22,
The invention according to any one of claims 15 to 21, wherein the photographing step includes a step of photographing the object in a state independent of the mobile terminal, and photographing data for the mobile terminal. Since it includes a transfer process, the object can be photographed with a general-purpose digital camera, the information of the image data can be searched using a portable personal computer, etc., and the information can be received using a handheld device. Has the effect.

According to the invention of claim 23,
In the invention according to any one of claims 15 to 22, in the search result sending step, the user sends the search result to a predetermined computer device. Therefore, the search result is not limited to a mobile terminal and can be stored at home. It is possible to receive it by a computer device, and it is possible to provide a search service in a form corresponding to a user's request such as organizing a communication environment and search results.

According to the invention of claim 24,
In the invention according to any one of claims 15 to 23, in the position information acquisition step, a direction on a map detected by a gyro at the time of photographing the object is sent as a part of the photographing position information. Since the photographing position specifying step specifies the photographing place of the image data including the direction, the search process is performed based on more detailed position information, which improves the efficiency of the search process and improves the search accuracy. The effect that it can be achieved is produced.

According to the invention of claim 25,
The invention according to any one of claims 15 to 24, further comprising an editing step capable of editing the image data in order to extract only an object from the captured image data, Since it is sent to the center, by editing the image data before starting the search, it becomes possible to obtain the information suitable for the object in detail and accurately. There is an effect that the editing operation itself may be a simple operation such as leaving the object and deleting the periphery.

Further, according to the invention of claim 26,
In the invention according to any one of claims 15 to 25, the photographing step photographs a plurality of image data or moving images of the same object, and the searching step includes a plurality of the image data or Since the search process based on the moving image is executed, the search accuracy can be improved by increasing the image data of the target object used at the time of search. At this time, there is an effect that the user can improve the search accuracy by an operation of increasing the number of shots or a simple operation of shooting a moving image.

Further, according to the invention of claim 27,
27. By causing a computer to execute the method according to any one of claims 14 to 26,
It is possible to obtain a program capable of realizing any one of the operations 26 to 26 by a computer.

[Brief description of drawings]

FIG. 1 is a system configuration diagram showing an overall configuration of a captured image information providing system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a hardware configuration of a mobile terminal, an information providing device, and a terminal device which constitute the captured image information providing system according to the embodiment of the present invention.

FIG. 3 is a block diagram showing a functional configuration of a captured image information providing system according to the embodiment of the present invention.

FIG. 4 is a diagram showing a specific configuration of a mobile terminal.

FIG. 5 is a flowchart showing an overall processing procedure for providing information on a captured image.

FIG. 6 is a diagram illustrating a field designation process of a search target in a mobile terminal.

FIG. 7 is a flowchart showing a processing procedure for providing information on a captured image when a field to be searched is designated.

FIG. 8 is a diagram for explaining a process progress when a building system is designated as a field at the time of search.

FIG. 9 is a flowchart illustrating a processing procedure for providing information of a captured image when a field is designated during search.

FIG. 10 is a diagram for explaining a process progress when a vehicle system is designated in a field at the time of search.

FIG. 11 is a diagram showing a shooting state of an object including a plurality of fields.

FIG. 12 is a diagram showing a state after image data editing processing.

FIG. 13 is a diagram for explaining a process progress when a person / animal / plant system is designated as a field at the time of search.

FIG. 14 is a diagram for explaining a process progress when a sound system is designated in a field during search.

[Explanation of symbols]

100 networks 101 mobile terminal 102 information providing device 104 digital camera 105 Terminal device (PC) 200 bus 201 CPU 202 ROM 203 RAM 204 HDD 205 HD 206 FDD 207 FD 208 display 209 I / F 210 communication line 211 keyboard 212 mouse 213 Scanner 214 printer 215 CD-ROM 216 CD-ROM drive 217 speaker 218 tablets 300 object 300a Object (building system) 300b Object (vehicle system) 300c Object (human / animal and plant system) 301 mobile phone 310 Image database (DB) 311 Place database (DB) 312 Information database (DB) 315 Search method 316 image retrieval means 317 Place Search Method 318 Information retrieval means 320 broadcasting stations 321 Publisher 322 newspaper 323 Information site 401 display screen 402 Operation button 402a shooting button 600 Search target selection screen 800 Basic information 801 Detailed information index 802 Detailed information 1000 voice content

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 7/173 630 H04B 7/26 109M H04Q 7/20 H04Q 7/04 Z 7/38 H04N 5/91 L F Term (Reference) 5B075 KK07 ND20 NK07 PQ02 PQ04 QM08 UU14 5C053 FA08 HA29 LA02 LA14 5C064 BA07 BC16 BC18 BC23 BC24 BC25 BD02 BD05 BD07 BD13 5K067 AA21 BB04 BB21 BB36 DD52 EE02 EE15 FF23EE02 FF07 EE02 FF07EE02 EE07 FF07EE02EE16 FF07EE02EE16

Claims (27)

[Claims] 1. An information providing system for a captured image using a mobile terminal, which transmits an image captured by a user from a mobile terminal to a center via a network and receives information about image data from the center via the network. The mobile terminal includes: a photographing unit for photographing an object and obtaining image data; a position information acquiring unit for acquiring photographing position information on a map at the time of photographing the object; Distance information acquisition means for acquiring distance information to the object at the time of shooting, and sending image data, shooting position information and distance information of the taken object at the time of a search request from the portable terminal to the center, A communication means for receiving the search result returned from the center, an operation means for performing an operation necessary for the search, and the search result as an image. Output means for outputting using characters, sounds, and the like, wherein the center stores a search database in which image data related to the object, shooting position information, and search information related to distance information are stored, A photographing position specifying means for specifying a photographing place of the image data using the photographing position information and the distance information sent from the mobile terminal; and accessing the search database, and the specified photographing place, A mobile phone comprising: a search unit that searches for information related to the image data using the image data as a search item; and an information sending unit that sends the search result obtained by the search to the mobile terminal. An information provision system for captured images using a terminal. 2. The information of a photographed image using a mobile terminal according to claim 1, wherein the distance information acquisition means uses a distance to an object detected by a distance measuring sensor as the distance information. Offer system. 3. The information providing system for a photographed image using a mobile terminal according to claim 1, wherein the distance information acquisition means uses focal length information at the time of photographing the object as the distance information. . 4. The search means, when a plurality of search results are output during the search, outputs the search candidates listing each search result to the mobile terminal, and based on the selection of the search candidates by the mobile terminal. The information providing system for photographed images using the mobile terminal according to claim 1, wherein the search candidates are narrowed down and a search process is performed. 5. The search means classifies an object into a plurality of fields in advance, and specifies and accesses a corresponding field in the search database based on the field specification by the mobile terminal. An information providing system for a captured image using the mobile terminal according to claim 1. 6. The search means is constituted by basic information indicating an outline and detailed information transmitted based on selection of the mobile terminal for the basic information, the search result. An information providing system for photographed images using the mobile terminal according to any one of items 1 to 5. 7. The mobile terminal according to claim 6, wherein the search means searches for a destination link having information related to the basic information as the detailed information and sends it to the mobile terminal. Information system for captured images. 8. The photographing means is configured to photograph the object independently of the portable terminal and transfer the photographed data to the portable terminal. An information providing system for captured images using the mobile terminal according to any one of 1 to 7. 9. The photographing using the mobile terminal according to claim 1, wherein the information sending unit sends the search result to a computer device predetermined by a user. Image information system. 10. The position information acquisition means of the mobile terminal is,
The direction on the map detected by the gyro at the time of photographing the object is transmitted as a part of the photographing position information, and the photographing position specifying means of the center specifies the photographing place of the image data including the direction. An information providing system of a captured image using the mobile terminal according to any one of claims 1 to 9. 11. The portable terminal or the photographing means includes an editing means for editing image data to extract only an object from photographed image data, and the edited image data is stored in the center. The information providing system for a captured image using the mobile terminal according to claim 1, which transmits the information. 12. The portable terminal is a portable telephone device having a digital camera, and the portable terminal is a portable telephone device.
An information providing system for photographed images using the mobile terminal according to any one of 1. 13. The mobile terminal captures a plurality of image data or moving images of the same object, and the search means of the center executes a search process based on the plurality of image data or moving images. An information providing system for a captured image using the mobile terminal according to any one of claims 1 to 12. 14. A photographing step in which a user photographs an object and obtains image data, a position information acquisition step of acquiring photographing position information on a map at the time of photographing the object, and the object photographing A distance information acquisition step of acquiring distance information to the object at the time, and a search for retrieving information on the imaged object based on the image data of the object, the shooting position information and the distance information. A method of providing information on a captured image, comprising: a step; and a search result output step of outputting the search result. 15. A method of providing a captured image information using a portable terminal, which transmits an image captured by a user from a portable terminal to a center via a network and receives information about image data from the center via the network. The mobile terminal includes a photographing step in which a user photographs an object to obtain image data, a position information acquisition step of acquiring photographing position information on a map when the object is photographed, and the object. A distance information acquisition step of acquiring distance information to the object at the time of shooting; a search request step of sending the image data of the object, the shooting position information, and the distance information to the center; A search result output step of receiving a returned search result, wherein the center has the photographing position information and the distance information sent from the mobile terminal. And a database for search is accessed to search for information related to the image data by using the specified shooting location and the image data as search items. And a search result sending step of outputting the search result to the mobile terminal. 16. The captured image information according to claim 14, wherein the distance information acquisition step uses a distance to an object detected by a distance measuring sensor as the distance information. How to provide. 17. The information providing method for a photographed image according to claim 14, wherein the distance information acquisition step uses focal length information at the time of photographing the object as the distance information. . 18. The search step outputs, when a plurality of search results are output during the processing of the search step, a search candidate listing each search result to the mobile terminal, and the search candidates by the mobile terminal are output. 18. The method for providing information of a photographed image according to claim 15, wherein the search candidates are narrowed down and searched based on the selection. 19. The searching step is characterized in that the object is classified into a plurality of fields in advance, and the corresponding field is designated and accessed in the search database based on the field designation by the mobile terminal. The method for providing information on a captured image according to any one of claims 15 to 18. 20. The search step includes basic information indicating an outline of the search result and detailed information to be transmitted based on selection of the mobile terminal for the basic information. The method for providing information on a captured image according to any one of 1. 21. The searching step searches for a provider link having information related to the basic information as the detailed information and sends it to the mobile terminal.
The method for providing information on a captured image according to 0. 22. The photographing step includes a step of photographing the object independently of the mobile terminal, and a step of transferring photographing data to the mobile terminal. Item 22. A method for providing information on a captured image according to any one of Items 15 to 21. 23. Information provision of a photographed image according to claim 15, wherein the search result sending step sends the search result to a computer device predetermined by a user. Method. 24. The position information acquisition step sends a direction on a map detected by a gyro at the time of photographing the object as a part of the photographing position information, and the photographing position specifying step includes the image data. 24. The method for providing information of a photographed image according to claim 15, wherein the photographing location is specified by including the direction. 25. An image data editing step is provided for extracting only an object from the photographed image data, and the edited image data is sent to the center. Item 25. A method for providing information on a captured image according to any one of Items 15 to 24. 26. The photographing step photographs a plurality of image data or moving images of the same object, and the searching step executes a search process based on the plurality of image data or moving images. Claim 15
25. A method for providing information on a captured image according to any one of 25 to 25. 27. A program for causing a computer to execute the method according to any one of claims 14 to 26.