JP2008217660A - Search method and apparatus - Google Patents

Abstract

When searching for words or images in content for which detailed information is desired, the burden of input work is reduced and a comfortable content viewing environment is provided.
A user-designated position on a content or user instruction information command1 such as a user's movement is imaged by an imaging means INPUT1 to obtain input images IMGin1 and IMGin2. The image IMGin1 holds user instruction information or motion information for the search target contents, and the image IMGin2 holds user search result presentation position instruction information. The search key detection means SD detects a search keyword or an image (hereinafter collectively referred to as a search key COMin1) by image recognition. In the search means SEAR1, the search algorithm is applied to the search key COMin1, and the search result Sdat is acquired.
[Selection] Figure 1

Description

  The present invention relates to an apparatus that enables searching without requiring a laborious input operation for words or images in content for which detailed information is desired by a user.

  In recent years, the cost of high-speed communication has been reduced, and anyone can easily access the network, and a large amount of information is disclosed on the network. As a result, web search has become the most common means in a wide range of research work, such as when there are unknown phrases during reading, when planning a trip, or when conducting research on advanced technology.

  A typical example is the use of a web browser that comes standard with a PC. This is a method of inputting a search key into an input box using an input device. When entering a search key (in this case, a character string), keyboard operation is indispensable.

  FIG. 15 is a block diagram showing a configuration of a conventional search device PROC4 using an input device such as a keyboard. The search device PROC4 includes input means INPUT3, search means SEAR1, image generation means GEN3, and presentation means OUTPUT2. The input means INPUT3 reads a search command command2 from a user input by an input device such as a keyboard and outputs it as a search key COMin2. The search means SEAR1 applies a search algorithm to the search key COMin2 and acquires the search result Sdat. The presentation image generation means GEN3 creates a presentation image IMGout3 that presents the search result Sdat to the user according to the search engine or application. The presentation means OUTPUT2 outputs the presentation image IMGout3 as a search result search result 2 on the display.

  A flowchart of the operation of the apparatus is shown in FIG. In step S109, the user directly inputs a character string serving as the search key COMin2 from an input device such as a keyboard. In step S105, the search key COMin2 input in step S109 is applied to the search algorithm, and the search result Sdat is obtained. In step S110, a presentation image IMGout3 for presenting the search result Sdat to the user is created according to the search engine or application. In step S111, the presentation image IMGout3 created in step S110 is output as search result 2.

  In the search device, in order to perform smooth input using the keyboard, training is necessary, and even if the user gets used to the operation, typographical errors can occur frequently, and it can be said that the operation is complicated. Considering this point, keyboards with various key arrangements based on ergonomic design are on the market. However, at present, the purchase of such a keyboard is limited to users who have a particular commitment among people who have a long time to operate the keyboard for work or hobbies. Moreover, even if a skilled person uses the devised keyboard, it is impossible to completely eliminate typing errors, and it cannot be said that the complexity of the key type itself has been solved.

  Currently, many technological developments aimed at improving the efficiency of search work have been promoted, but most of them are focused on search algorithms. Statistical algorithms such as Bayesian theory have been introduced for search algorithms related to the presentation order of search results, and improvements are still being made. Search engines such as Google are representative examples that employ advanced search algorithms. Such search engines offer a variety of search options that allow detailed specification of search conditions such as search key, language, file type, etc., and can also categorize and present each item such as images, news, maps, etc. It corresponds to a wide range of purposes. However, the improvement of the search algorithm is aimed at how the desired information is presented to the user, and the input of the search condition is premised on the input of a character string by the user. Even Google search sites with a very concise design are built around an input box, and the interface puts a burden on user input.

  Under these circumstances, the development of new input devices for search work is also beginning to attract attention. One of them is the use of cameras mounted on mobile phones. This is put into practical use as a service using image recognition and mobile phone software. Patent Document 1 proposes an apparatus that takes a picture of a signboard or the like while moving in an automobile, recognizes the contents thereof, and performs navigation. Since cell phones are more complicated to input characters than keyboards, image retrieval as described in Patent Document 1 has a high potential need as a technology that takes advantage of the characteristics of highly mobile devices.

  Patent Document 2 proposes an apparatus for making a program reservation by photographing a TV column such as a newspaper with a camera-equipped mobile phone as an indoor technique. However, even when shooting with a camera-equipped mobile phone capable of character string recognition as in Patent Document 2, it takes time to adjust the shooting range. Furthermore, pointing the camera at the content many times during reading or viewing images and videos is a troublesome task, similar to web search using a personal computer, and hinders comfortable content viewing.

The problem with the conventional search means is that the device is completely independent for searching the content that the user is watching. Searching on a computer display while reading a book or newspaper or watching a movie involves time and space movement, which is very time-consuming and requires a comfortable content viewing environment. Hinder. Even in the search means using the camera-equipped mobile phone, it is not possible to completely satisfy the requirement for efficient search work because the user needs to stop viewing the content.
JP-A-2005-98939 JP 2004-341977

  In the search means by a search engine, a method of entering a search key into an input box with a keyboard is common. Even if the keyboard itself has been improved, there is a first problem that input work is still necessary and a burden on the user is heavy.

  In addition, in web search, when a user performs a search between viewing contents, it is necessary to look at the display of the personal computer. Therefore, the user looks away from the contents and the viewing is temporarily suspended. Also, when the search target is shot with a camera-equipped mobile phone, adjusting the imaging range by pointing the camera at the content is a very complicated task as the number of times increases. The search means has a second problem that it cannot be searched while viewing the content.

  The present invention has been made to solve the above problems.

A search method for detecting a search keyword or image data (hereinafter collectively referred to as a search key) by image processing for a phrase or image for which detailed information is desired by a user in a content. ,
An imaging step for obtaining an instruction for a user-desired search location on the content as a moving image or a still image;
A detection step of inputting a moving image or a still image, specifying a search target based on user-designated information such as a position designated by the user or a user's movement, and detecting a search key;
A search step of applying the search key to a search algorithm and obtaining a search result;
An image generation step of creating a presentation image for presenting the search result;
A presenting step of presenting the presented image;
It is characterized by providing.

The search method according to claim 2 of the present invention includes:
A search method for detecting a search key by image processing for a word or image for which detailed information is desired by a user in a content,
An imaging step for obtaining an instruction for a user-desired search location on the content as a moving image or a still image;
A detection step of inputting a moving image or a still image, specifying a search target based on user-designated information such as a position designated by the user or a user's movement, and detecting a search key;
A search step of applying the search key to a search algorithm and obtaining a search result;
An image generation step of creating a presentation image for presenting the search result;
A presentation position determining step for determining a spatial position for presenting the presented image;
A presenting step of presenting the presented image at the position determined by the presenting position determining step;
The search method according to claim 1, further comprising:

The search method according to claim 3 of the present invention includes:
The search method according to claim 1, wherein the detection step performs character recognition or image identification on the moving image or still image obtained in the imaging step to detect a search key. .

The search method according to claim 4 of the present invention comprises:
The presenting position determining step detects an instruction of a presenting position from a user with respect to the moving image or the still image obtained in the imaging step, and determines the presenting position of the presenting image. The search method according to claim 2.

The search method according to claim 5 of the present invention includes:
The search method according to claim 1, wherein the content is an image presented by a step different from the presentation step, or a physical medium such as a printed material.

The search method according to claim 6 of the present invention includes:
The content is virtual content presented by the presenting step,
The image generating step also creates an image for presenting the virtual content, and further combines the search result with an image for presenting the virtual content to form the presented image. The search method according to 1.

The search method according to claim 7 of the present invention comprises:
The search method according to claim 6, wherein the detection step is capable of detecting a search key only by detecting a designated position by holding position information presenting the virtual content.

The search method according to claim 8 of the present invention comprises:
The presenting step presents auxiliary information for specifying the search target on the content according to the user instruction information,
The search method according to claim 1, wherein the auxiliary information includes a predetermined mark and a highlight.

The search method according to claim 9 of the present invention includes:
The search method according to claim 1, wherein the detecting step detects a search key using metadata embedded in the content.

The search device according to claim 10 of the present invention comprises:
A search device for detecting a search key by image processing for a phrase or image for which detailed information is desired by a user in a content,
Imaging means for acquiring an instruction for a user-desired search location on the content as a moving image or a still image;
Detecting means for inputting a moving image or a still image, specifying a search target based on user designation information such as a position designated by a user or a user's movement, and detecting a search key;
Search means for applying the search key to a search algorithm and obtaining a search result;
Image generating means for creating a presentation image for presenting the search result;
Presenting means for presenting the presented image;
It is characterized by providing.

The program of claim 11 of the present invention is:
A program for realizing a search method for detecting a search key by image processing for a phrase or image for which a user desires detailed information in a content,
An imaging step for obtaining an instruction for a user-desired search location on the content as a moving image or a still image;
A detection step of inputting a moving image or a still image, specifying a search target based on user-designated information such as a position designated by the user or a user's movement, and detecting a search key;
A search step of applying the search key to a search algorithm and obtaining a search result;
An image generation step of creating a presentation image for presenting the search result;
A presenting step of presenting the presented image;
It is characterized by providing.

An integrated circuit according to claim 12 of the present invention comprises:
An integrated circuit for realizing a search method for detecting a search key by image processing for a phrase or image for which detailed information is desired by a user in a content,
Imaging means for acquiring an instruction for a user-desired search location on the content as a moving image or a still image;
Detecting means for inputting a moving image or a still image, specifying a search target based on user designation information such as a position designated by a user or a user's movement, and detecting a search key;
Search means for applying the search key to a search algorithm and obtaining a search result;
Image generating means for creating a presentation image for presenting the search result;
Presenting means for presenting the presented image;
It is characterized by providing.

  As described above, according to the present invention, since the search key can be detected from the position and movement designated by the user by image recognition, the user can obtain a desired search result without performing a complicated keyboard input operation. And its practical value is high.

  In addition, since the search result presentation position determining means is provided, the degree of freedom of the presentation position is high and a comfortable content viewing environment can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing the operation of the search device PROC1 of the present invention. The search device PROC1 can detect the search key COMin1 by image recognition of a video in which the operation is captured by the user specifying a position as a search target or performing an operation as instruction information when the search key COMin is input. Further, since the search result presentation image IMGout1 can be determined as the presentation position Pdat intended by the user, presentation with a high degree of freedom is possible. The two points are different from the conventional search method.

  The search device PROC1 includes an imaging means INPUT1, search key detection means SD, search means SEAR1, image generation means GEN1, presentation position determination means POS, and presentation means OUTPUT1.

  The imaging means INPUT1 acquires an image IMGin1 obtained by capturing user instruction information command1 such as a position designated by the user or a user's movement. The search key detecting means SD detects a search key COMin1 designated by the user by image recognition for the image IMGin1. The search means SEAR1 applies a search algorithm to the search key COMin1 and acquires a search result Sdat. The image generation means GEN1 creates a presentation image IMGout1 for presenting the search result Sdat to the user according to the search engine or application. The presentation position determination unit POS determines a position Pdat for presenting the presentation image IMGout1 based on the user's action taken by the imaging unit INPUT1 or the position of the content being viewed by the user. The presentation means OUTPUT1 outputs the presentation image IMGout1 as a search result search result 1 by the presentation device.

  As an embodiment of this, a search device equipped with a camera and a projector is shown in FIG. In this apparatus, a user's instruction or motion is imaged by a camera as an imaging apparatus, and a presentation image generated based on the search result is presented by a projector as a presentation apparatus. The search target in this apparatus may be a phrase or an image included in a physical medium (hereinafter referred to as actual content) such as a printed matter such as a book or a newspaper, or a sentence or an image presented by a projector (hereinafter referred to as virtual content). ) The virtual content may be presented in a blank booklet. The virtual content can be acquired from the network.

  The camera or projector may be mounted on a compact device as shown in FIG. 2, or may be installed on the ceiling, wall, or pillar. In the future, when further miniaturization of the projector and the camera is realized, a portable terminal having functions of the projector and the camera can be configured as shown in FIG. 3 and the present invention can be applied. In addition, as shown in FIG. 4, a glasses-type search device equipped with a projector and a camera can be configured to acquire the user instruction information command1 or present the search result search result 1 according to the user's line of sight. is there. Note that the user's line-of-sight detection may be performed within the eyeglass-type search device, or may be performed externally by a method of transmitting the line-of-sight detection result.

  Next, the operation of the search device PROC1 according to the embodiment of the present invention will be described. FIG. 5 shows a flowchart of the operation of the search device PROC1.

  In step S101, content contents including a search target are presented within the camera imaging range. The camera may be fixed, or may follow the user's movement.

  In step S102, the user performs an instruction or operation for specifying the position to be searched. This is designated as instruction information command1. A dedicated tool for giving instructions may be used, or a stick-shaped object such as a fingertip or a commercially available pen may be used as a virtual marker. At this time, if a dedicated tool is used, search key detection from user instruction information command1 by image recognition becomes easy. The search target instruction basically indicates one point in an area where the search target exists, such as a person, an object in a video, or a character in a sentence. However, in order to specify the search target more accurately, it is possible to move the search target in a circle or draw a ruled line.

  In step S103, the user instruction information command1 and the search target contents are imaged. The search target is acquired as a moving image or a still image. When imaging is periodically performed and an operation indicating the instruction information command1 is detected, search key detection in step S104 or presentation position determination in step S107 is performed. If the user's instruction and the search target are imaged, the image IMGin1 is input in step S104, and if the user's presentation position instruction is imaged, the image IMGin2 is input in step S105. The determination may be based on the presence / absence of a search target at the user's designated position, may be determined in advance by the user's designated position, or may be designated by the user via an interface. The camera used for imaging may be a commercially available NTSC camera, or a monochrome camera or an infrared camera may be used depending on the content to be presented and the configuration of the presentation device.

  In step S104, the search key COMin1 is detected by using the instruction information command1 from the user and the image IMGin1 in which the search target contents are captured. The processing here can deal with any situation, such as when the search target is a character or an image, or when the content presentation is a projector presentation image or a real thing. The search key COMin1 obtained here may be the same as the search key COMin2 that is input to the input box by a conventional web search or may be unique to the search system. Before proceeding to the next step, the user is inquired as to whether the search key COMin1 is intended by the user. If the search key COMin1 is not the desired search key COMin1, processing can be performed again from step S102.

  In step S105, the search key COMin1 detected in step S104 is applied to the search algorithm to obtain the search result Sdat. The search may be an Internet search or a local search. Local search refers to a search performed on a narrow range of facilities such as a library, educational institution, amusement facility, or a network constructed in a region. In the local search, while the search range is limited as compared with the Internet search, it is also possible to aggregate information highly related to the content. The Internet search and local search may be performed by a predetermined server on the network.

  In step S106, a presentation image IMGout1 reflecting the search result Sdat obtained in S105 is created. The search result may be synthesized on the content data according to the application and interface, or may be independent of the content.

  In step S107, the position Pdat where the image IMGout1 created in S106 is presented is determined based on the input image IMGin2 captured in step S103. In this processing, an operation of designating a presentation position desired by the user may be performed, and this may be captured in step S103, or the optimal presentation position may be determined on the apparatus side from the input image in S103. If the position where the presentation image IMGout1 is presented is fixed, this step is omitted.

  In step S108, the presentation image IMGout1 created in step S106 is output as the search result search result 1 at the presentation position determined in step S107. The search can be continued with the information in the presented image IMGout1 as the next search target. In addition, it is possible to improve the visibility of the interface by presenting a mark or highlight to the user as auxiliary information for specifying the search target. Note that when presenting search results for virtual content as a search target, it is possible to separate presentation means for presenting virtual content and presentation means for presenting a presentation image created based on the search results. As one embodiment, there is a method of using two projectors in FIG. Thereby, the freedom degree of the position which shows a presentation image can be improved.

  FIG. 7 is a flowchart showing details of step S104.

  In step S1041, an image IMGin1 obtained by capturing a user instruction is input. The image IMGin1 is an image sequence captured from the start to the end of the user instruction.

  In step S1042, the coordinates of the tip of a dedicated tool or a virtual tool such as a user's fingertip or a bar-like real object are detected. The target image to be detected is basically the first frame of the image IMGin1 in which the start of the instruction from the user is the first frame and the end is the last frame, but any frame in the image sequence may be used.

  In step S1043, it is determined whether the content presented at the coordinates obtained in S1042 is an image or a character. It is also possible to specify text or image via the user interface.

  In step S1044, when the search target is a character, character recognition is performed and the search key COMin1 is detected. Examples of character recognition include the following methods. The designated area is extracted based on the designated position coordinates of the user or the designated vector obtained as a result of motion detection. At this time, the instruction area is extracted by extending the instruction coordinates or the instruction vector according to the character direction. Character recognition is performed in the designated area, and the obtained word / phrase is used as a search key COMin1.

  In step S1045, when the search target is an image, image identification is performed, and a search key COMin1 corresponding to the image is detected. Image identification is roughly classified into two methods. The first method is a method that uses metadata added to an image. In the present invention, metadata refers to a search key added or embedded in content. Since the virtual content is electronic data presented by the projector of the search device as shown in FIG. 2, FIG. 3, or FIG. 4, metadata can be added as digital data in the content. For example, a search key such as the product name, manufacturer, and ingredient can be added as metadata to an image showing a drink can. When the search target is instructed, the metadata is used as a search key. Detected. On the other hand, in the case of real content, a search key can be embedded as a barcode, QR code, or digital watermark information. The metadata may be given for each character or object in the image, and may be held in one frame and the user may be requested to select, or may be held in divided areas. In addition, a plurality of metadata may be added to one target. In that case, the user may be asked to select, or all may be used as the search key COMin1. The second method is a method of performing feature extraction and matching with pattern information stored in a server. At this time, the search key COMin1 corresponding to the pattern information is also stored. Similar to the first method, a plurality of search keys COMin1 can be prepared for one pattern information.

  When virtual content is a search target, if the position information for presenting the virtual content is retained, only the indication area is detected without performing character recognition in step S1044 or image identification in step S1045. It is also possible to acquire the search key COMin1.

  In step S1046, the search key COMin1 obtained in step S1044 or step S1045 is output. When a plurality of search keys are obtained, each search key may be a single search key COMin1, or a search expression using logical operators such as AND and OR may be configured.

  In the detection of the search key COMin1, when a plurality of search targets are specified, the embodiment shown in the flowchart of FIG. 8 can be realized. As a difference from FIG. 7, an instruction end determination is added as step S1047. Until the user's instruction is completed, steps S1041 to S1045 are repeated, a plurality of search targets can be specified, and the search key COMin1 can be acquired.

  As described above, according to the present invention, the search key COMin1 can be detected from the position and movement specified by the user by image recognition, so that the user can obtain a desired search result without performing a complicated keyboard input operation. And its practical value is high. In addition, since the search result presentation position determining means is provided, the degree of freedom of the presentation position is high and a comfortable content viewing environment can be provided.

(Embodiment 2)
Next, the operation of the search device PROC2 according to Embodiment 2 of the present invention will be described. FIG. 9 is a block diagram showing the operation of the search device PROC2. The search device PROC2 can present and refer to the content cont. Although the flowchart of the operation is the same as that of FIG. 5 shown in the first embodiment, since the electronic content is held, the imaging means INTPUT2 only captures the instruction information command1 from the user. In the second embodiment, since the signal equivalent to the search target contents shown in FIG. 1 is obtained by referring to the content cont, the search key COMin1 can be easily detected in step S109. If the position information for presenting the content is held, the content is referred to and the search key COMin1 can be acquired when the target area is extracted from the position designated by the user. By the above method, complicated image recognition processing such as content determination or character recognition and image identification can be omitted, and the processing amount can be reduced. Metadata can be added to the content cont and can be used for search key detection.

  As a configuration in the case where additional information related to content contents is held in the content file FILE, a search can be performed by the search device PROC3 shown in the block diagram of FIG. The search device PROC3 is different from the search unit SEAR1 of the search device PROC2 shown in FIG. 9 in that the search unit SEAR2 can refer to the content file FILE. In the second embodiment, the content file and the network server or the local server may be referred to in order, or the reference side may be selected by the user side or designated by the server side. There may be.

<User interface in search device>
Hereinafter, although the user interface screen of the search device of this embodiment is shown, the screen shown here is a typical example and is not limited to this.

  FIGS. 11A and 11B are diagrams showing examples of user interface screens that present search results. In this example, the content 001 may be actual content that is a physical medium such as a book or a newspaper, or may be virtual content presented by a projector. The virtual content may be presented in a blank booklet. The search result 002 may be presented independently of the content as shown in FIG. 11 (a), or may be presented in a form combined with the content as shown in FIG. 11 (b). The control panel 003 presents operation buttons and a navigation screen that enable feedback to the apparatus by the user. The control panel 003 allows the user to issue an operation command to the search device in starting and ending search, instructing search targets, browsing search results, and the like. The operation command may be given as a motion and extracted by image recognition.

  FIG. 12 is a diagram illustrating an example of a user interface screen when a user searches for a phrase in a sentence. On the content 001, the user gives instruction information command1. At this time, the search target is highlighted to improve the interface visibility. Further, in the control panel 003, it is possible to inquire whether or not the search key is the one desired by the user.

  FIG. 13 is a diagram illustrating an example of a user interface screen when searching for an object in an image on the content 001. When the user designates a search target, a search key recorded in metadata for virtual content and in digital watermark information for real content is presented to the control panel 003. Similar to FIG. 12, the search target can be highlighted. When the user indicates the search target with a point, an object including the point is selected, and when the user indicates an operation surrounding the search target, an object in the region is selected. When there are a plurality of search keys, selection is requested as shown in FIG. Any number can be selected from the presented search keys. If there is no desired search key, “OK” may be designated in the absence of the selected search key. In addition, the search keys can be classified hierarchically, in which case “Next” or “Prev” is designated to move the hierarchy.

(Embodiment 3)
Further, by recording a program for realizing the search method shown in each embodiment on a recording medium such as a flexible disk, the processing shown in each embodiment is independent of a computer system. It becomes possible to carry out easily.

  FIG. 14 is an explanatory diagram when the search method of each of the above embodiments is implemented by a computer system using a program recorded on a recording medium such as a flexible disk.

  FIG. 14B shows an appearance, a cross-sectional structure, and a flexible disk as seen from the front of the flexible disk, and FIG. 14A shows an example of a physical format of the flexible disk that is a recording medium body. The flexible disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk storing the program, the program is recorded in an area allocated on the flexible disk FD.

  FIG. 14C shows a configuration for recording and reproducing the program on the flexible disk FD. When the program for realizing the search method is recorded on the flexible disk FD, the program is written from the computer system Cs via the flexible disk drive. Further, when the search method for realizing the search method by the program in the flexible disk is constructed in the computer system, the program is read from the flexible disk by the flexible disk drive and transferred to the computer system.

  In the above description, the flexible disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as an IC card or a ROM cassette capable of recording a program can be similarly implemented.

  The search device and search method according to the present invention have been described based on the above embodiments, but the present invention is not limited to these embodiments. The present invention also includes modifications made to the present embodiment by those skilled in the art without departing from the scope of the present invention.

  For example, an optical disk recording device including a search device according to the present embodiment, a moving image transmission device, a digital television broadcast transmission device, a Web server, a communication device, a portable information terminal, and the like, or a moving image including the image processing device according to the present embodiment. It goes without saying that a receiving device, a moving image recording device, a still image recording device, a digital television broadcast receiving device, a communication device, a portable information terminal, and the like are also included in the present invention. Here, the moving image recording device includes a camcorder, Web, and the like, and the still image recording device includes a digital still camera and the like.

  Each functional block in the block diagrams (FIG. 1, FIG. 9, FIG. 10, etc.) is typically realized as an LSI (Large Scale Integration) which is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

  Although referred to as LSI here, it may be called IC (Integrated Circuit), system LSI, super LSI, or ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI, and implementation with a dedicated circuit or a general-purpose processor is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of the circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied.

  Further, among the functional blocks, only the means for storing the data to be searched may be configured separately instead of being integrated into one chip.

  According to the present invention, since the search key can be detected from the position and movement specified by the user by image recognition, the user can acquire a desired search result without performing complicated keyboard input work. Effectiveness is high.

The block diagram which shows one embodiment of this invention Schematic diagram of a search device according to an embodiment of the present invention Schematic diagram of a mobile terminal type search device according to an embodiment of the present invention Illustrated diagram of a glasses-type search device according to an embodiment of the present invention Flow chart showing the operation of the present invention Illustration of a search device equipped with two presentation devices according to an embodiment of the present invention Flowchart showing search key detection operation of the embodiment A flowchart showing the operation of the search condition extraction means for a plurality of search targets Block diagram of an embodiment in which a search device holds electronic content files Block diagram of an embodiment including a content file at the search destination The figure which shows the example of the user interface screen in the search device as embodiment of this invention The figure which shows the example of the user interface screen in the search command when the search object is a character as embodiment of this invention The figure which shows the example of the user interface screen in the search command in case search object is an image as embodiment of this invention Explanatory drawing about the storage medium for storing the program for implement | achieving the search method of each said embodiment with a computer system Block diagram showing a conventional search device A flowchart showing the operation of a conventional search device

Explanation of symbols

INPUT1 Imaging means
INPUT2 imaging means
INPUT3 input means
SD search key detection means
SEAR search means
Server on SEAV network
GEN image generation means
POS presentation position determination means
OUTPUT presentation means
FILE Content file

Claims (12)

A search method for detecting a search keyword or image data (hereinafter collectively referred to as a search key) by image processing for a phrase or image for which detailed information is desired by a user in a content. ,
An imaging step for obtaining an instruction for a user-desired search location on the content as a moving image or a still image;
A detection step of inputting a moving image or a still image, specifying a search target based on user-designated information such as a position designated by the user or a user's movement, and detecting a search key;
A search step of applying the search key to a search algorithm and obtaining a search result;
An image generation step of creating a presentation image for presenting the search result;
A presenting step of presenting the presented image;
A search method comprising: A search method for detecting a search key by image processing for a word or image for which detailed information is desired by a user in a content,
An imaging step for obtaining an instruction for a user-desired search location on the content as a moving image or a still image;
A detection step of inputting a moving image or a still image, specifying a search target based on user-designated information such as a position designated by the user or a user's movement, and detecting a search key;
A search step of applying the search key to a search algorithm and obtaining a search result;
An image generation step of creating a presentation image for presenting the search result;
A presentation position determining step for determining a spatial position for presenting the presented image;
A presenting step of presenting the presented image at the position determined by the presenting position determining step;
The search method according to claim 1, further comprising: The search method according to claim 1, wherein the detection step detects a search key by performing character recognition or image identification on the moving image or still image obtained in the imaging step. The presenting position determining step detects an instruction of a presenting position from a user with respect to the moving image or the still image obtained in the imaging step, and determines the presenting position of the presenting image. The search method according to claim 2. The search method according to claim 1, wherein the content is an image presented by a step different from the presentation step or a physical medium such as a printed matter. The content is virtual content presented by the presenting step,
The image generating step also creates an image for presenting the virtual content, and further combines the search result with an image for presenting the virtual content to form the presented image. The search method according to 1. The search method according to claim 6, wherein the detection step can detect a search key only by detecting a designated position by holding position information for presenting the virtual content. The presenting step presents auxiliary information for specifying the search target on the content according to the user instruction information,
The search method according to claim 1, wherein the auxiliary information includes a predetermined mark and a highlight. The search method according to claim 1, wherein the detection step detects a search key using metadata embedded in the content. A search device for detecting a search key by image processing for a phrase or image for which detailed information is desired by a user in a content,
Imaging means for acquiring an instruction for a user-desired search location on the content as a moving image or a still image;
Detecting means for inputting a moving image or a still image, specifying a search target based on user designation information such as a position designated by a user or a user's movement, and detecting a search key;
Search means for applying the search key to a search algorithm and obtaining a search result;
Image generating means for creating a presentation image for presenting the search result;
Presenting means for presenting the presented image;
A search device comprising: A program for realizing a search method for detecting a search key by image processing for a phrase or image for which a user desires detailed information in a content,
An imaging step of acquiring an instruction for a search location desired by the user on the content as a moving image or a still image;
A detection step of inputting a moving image or a still image, specifying a search target based on user-designated information such as a position designated by the user or a user's movement, and detecting a search key;
A search step of applying the search key to a search algorithm and obtaining a search result;
An image generation step of creating a presentation image for presenting the search result;
A presenting step of presenting the presented image;
A program comprising: An integrated circuit for realizing a search method for detecting a search key by image processing for a phrase or image for which detailed information is desired by a user in a content,
Imaging means for acquiring an instruction for a user-desired search location on the content as a moving image or a still image;
Detecting means for inputting a moving image or a still image, specifying a search target based on user designation information such as a position designated by a user or a user's movement, and detecting a search key;
Search means for applying the search key to a search algorithm and obtaining a search result;
Image generating means for creating a presentation image for presenting the search result;
Presenting means for presenting the presented image;
An integrated circuit comprising:

Images