JP2014021802A - Search system, search server and program for portable terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a search system for allowing the user of a portable terminal to intuitively know which of ranges viewable from his or her current standing position is associated with a base matching search conditions.SOLUTION: A portable terminal sends search conditions and the current position information of its own terminal via a communication network to a search section, and acquires a search result relating to a base matching search conditions including the position information of a base matching the search conditions from the search section. The portable terminal includes detection means for detecting the photographic direction of a camera section. The portable terminal estimates a camera video by the camera section displayed on a display surface from the detected photographic direction of the camera section and map data stored by a map data acquisition and storage part, and estimates the position on the estimated camera video of the base matching the search conditions, and performs the superimposed display of a mark associated with the base matching the search conditions at the estimated screen position in the camera video of the display screen.

Description

  The present invention relates to a search system for searching a base such as a store that matches a set search condition. The present invention also relates to a search server and a portable terminal program used in the search system.

  Various searches related to a base (POI (Point Of Interest)) displayed on a map have been proposed, for example. Here, the base (POI) is, for example, a restaurant such as a restaurant or a department store, a golf course, a facility such as a mountain, a park, a beach, a waterfall, etc. where a representative point is determined. Including natural products. The bases (POI) are not limited to those that can be displayed as points on the map, but also include those defined by polygons (polygons) such as parks and golf courses.

  For example, various searches can be performed by accessing a search site from a high-function mobile phone terminal called a smartphone. As an example of this search, for example, when a user of a mobile phone terminal sets a search condition and requests a search from the search site, the search site, for example, a base that matches the set search condition, such as a restaurant, ramen shop, or karaoke shop Stores, hotels, parks, etc. are searched within a predetermined range centered on the current position of the mobile phone terminal, for example, and the search result is provided to the user of the mobile phone terminal (for example, Patent Documents) 1 (Japanese Unexamined Patent Application Publication No. 2011-197523), Japanese Patent Application Laid-Open No. 2008-122227, Japanese Patent Application Laid-Open No. 2010-124185, etc.).

  In the above-described conventional search system, as a search result, map information including bases that match the search conditions is provided, or a list of bases that match the search conditions is provided.

JP 2011-197523 A JP 2008-122227 A JP 2010-124185 A

  However, if a location that matches the search criteria is displayed on a map or displayed in a list, the location that matches the search criteria is actually from the current position of the user. There is a problem that it is not possible to intuitively know which one is visible.

  If the user is familiar with the map, if the current position is displayed on the map, it is possible to determine, to some extent, which bases that match the search criteria are visible in the user's field of view. Sometimes you can. However, it is not easy for a general user to determine the correspondence between a map and what is actually visually recognized.

  In view of the above points, the present invention provides a search system in which a user can intuitively know which of the bases that match the search condition is actually visible from the current standing position. The purpose is to provide.

In order to solve the above problems, the invention of claim 1
A search system configured by connecting a search unit and a portable terminal through a communication network,
The portable terminal is
A camera unit;
A display screen;
Means for acquiring current location information of the terminal;
Detecting means for detecting a shooting direction of the camera unit;
The search condition and the current location information of the terminal are sent to the search unit through the communication network, and the search result relating to the base that matches the search condition includes the location information of the base that matches the search condition. Means for obtaining
Map data acquisition and holding means for holding map data around at least the current position of the terminal;
The camera image by the camera unit displayed on the display screen of the portable terminal is estimated from the shooting direction of the camera unit and the map data held in the map data acquisition holding unit, and the search condition The estimated position on the camera image of the base that matches the position is estimated, and in the camera video displayed on the display screen, the estimated screen position is associated with the base that matches the search condition. Display control means for displaying superimposed marks;
A search system characterized by comprising:

  In the search system according to the first aspect of the present invention, the portable terminal sends the search condition and the current position of the portable terminal to the search unit, and the search unit determines a predetermined based on the current position of the portable terminal. A search result regarding a base that matches the search condition in the range is received. In addition, the portable terminal performs, for example, acquiring map data around the current position of the terminal from the map data providing unit, and stores the acquired map data in the map data acquisition and holding means.

  And the display control means of the portable terminal estimates the camera video by the camera unit displayed on the display screen from the information on the shooting direction of the camera unit and the map data held in the map data acquisition holding unit, A mark associated with the site that matches the search condition is superimposed on the screen position estimated as the position on the camera image of the site that matches the search condition.

  That is, on the display screen of the portable terminal, a mark associated with a base that matches the search condition obtained from the search unit as a search result is displayed on the same camera image as the current position (standing position). The superimposed image using the augmented reality (AR) technology is displayed.

  Therefore, the user of the portable terminal can see the base that matches the search condition in the range where the user can actually see the current standing position by viewing the image using the AR technology displayed on the display screen. Intuitively know if there is.

The invention of claim 8
A search system configured by connecting a server and a portable terminal through a communication network,
The portable terminal is
A camera unit;
A display screen;
Means for acquiring current location information of the terminal;
Detecting means for detecting a shooting direction of the camera unit;
Means for sending a search condition, current position information of the terminal itself, and information on a shooting direction of the camera unit to the server through the communication network, and acquiring a search result regarding a base that matches the search condition from the server When,
With
The server
A base that matches the search condition from the portable terminal is searched in a range based on the current position of the portable terminal, and matches the search condition including position information of the base that matches the search condition. Means for sending a search result relating to a base to the portable terminal through the communication network;
The camera image by the camera unit displayed on the display screen of the portable terminal from the information on the current position from the portable terminal, the information on the shooting direction of the camera unit, and the held map data. And estimating the screen position estimated as the position on the camera image of the base that matches the search condition, and information on the screen position of the base that matches the estimated search condition is transmitted through the communication network. Means for sending to the portable terminal;
With
Further, the portable terminal is configured to use the estimated screen in the camera image displayed on the display screen based on the information on the screen position of the base that matches the search condition received from the server. There is provided a detection system characterized by comprising display control means for superimposing and displaying a mark associated with a base that matches the search condition at a position.

  In the invention of claim 8 configured as described above, the portable terminal sends the search condition, the current position of the portable terminal, and information on the shooting direction of the camera unit to the server. The server searches for a base that matches a search condition in a predetermined range based on the current position of the portable terminal, and transmits the search result to the portable terminal. In addition, the server displays a camera image by the camera unit displayed on the display screen of the portable terminal from the information on the current position from the portable terminal, the information on the shooting direction of the camera unit, and the held map data. Estimate and estimate the position of the base that matches the search condition on the camera image, and send the information about the screen position of the base that matches the estimated search condition to the portable terminal via the communication network. .

  The mobile phone terminal matches the search condition with the estimated screen position in the camera image by the camera unit displayed on the display screen based on the information on the screen position of the site that matches the search condition received from the server. The mark associated with the base is superimposed and displayed.

  That is, in the invention of claim 8 as well, the display screen of the portable terminal matches the search condition obtained from the search unit as a search result on the same camera image as the current position (standing position). An image using an augmented reality (AR) technology in which a mark associated with a base to be superimposed is superimposed is displayed.

  Therefore, the user of the portable terminal can see the base that matches the search condition in the range where the user can actually see the current standing position by viewing the image using the AR technology displayed on the display screen. Intuitively know if there is.

  According to the present invention, the user of the portable terminal simply sees the camera image using the AR technology displayed on the display screen of the portable terminal, and the site that matches the search condition is actually present. It is possible to know intuitively which range is visible from the standing position.

It is a figure for demonstrating the outline | summary of the whole of embodiment of the search system by this invention. It is a block diagram which shows the structural example of the mobile telephone terminal as an example of the portable terminal used for embodiment of the search system by this invention. FIG. 3 is a diagram illustrating a part of a flowchart for explaining an example of a processing operation in the mobile phone terminal of the example of FIG. 2. FIG. 3 is a diagram illustrating a part of a flowchart for explaining an example of a processing operation in the mobile phone terminal of the example of FIG. 2. FIG. 3 is a diagram illustrating a part of a flowchart for explaining an example of a processing operation in the mobile phone terminal of the example of FIG. 2. FIG. 3 is a diagram illustrating a part of a flowchart for explaining an example of a processing operation in the mobile phone terminal of the example of FIG. 2. It is a figure used for description of embodiment of the search system by this invention. It is a figure used for description of embodiment of the search system by this invention. It is a figure used for description of embodiment of the search system by this invention. It is a figure used for description of embodiment of the search system by this invention. It is a figure which shows the flowchart for demonstrating the processing operation example of the search server which comprises embodiment of the search system by this invention.

[Outline of Embodiment of Search System]
FIG. 1 is a diagram showing an outline of an example of a communication system including an embodiment of a search system according to the present invention. As shown in FIG. 1, the search system of this embodiment is configured by connecting a search site (Web site) 1 and a mobile phone terminal 2 as an example of a mobile terminal through a communication network (including the Internet) 3. Has been. In the embodiment described below, for the sake of easy understanding, the description will be made assuming that the search target base is a store.

  In this embodiment, the search site 1 includes a search unit (search server) 11 and a map data providing unit (server) 12.

  Although not shown in the drawing, the map data providing unit 12 includes a map database that stores map data for all of Japan, which is a search target area. The map data stored in this map database includes, for example, the position (latitude / longitude) for each building, the number of floors, store name information, and the like. Therefore, it is possible to determine from which map data of this map database each store is located.

  Although not shown, the search unit 11 includes a search database that stores information on bases to be searched such as stores all over Japan. In this search database, for example, when the base is a store, for example, store name information, location information (address and latitude / longitude), genre (gourmet, shopping, play, etc.), usage fee, etc. Is stored. Further, the search site 1 periodically obtains vacancy information at that point in time from each store and stores it as temporary information in association with the identification information of each store.

  When the search site 1 receives a search request including the search condition and the position information of the current position from the mobile phone terminal 2 through the communication network 3, the search unit 1 performs a predetermined operation centering on the current position of the mobile phone terminal 2 in the search unit 11. Stores that match the specified search condition are searched in the area range of the size of. Then, the search site 1 extracts store information that matches the search condition by the search in the search unit 11, and requests to search for store information that matches the extracted search condition via the communication network 3. Return to the mobile phone terminal 2.

  In this case, the search result information about the store extracted as matching the search condition includes location information, store name, and other information of the store that matches the search condition. In this embodiment, In particular, the search server 11 determines the rank of the search result for the store that matches the extracted search condition based on the priority condition including the distance from the current position of the mobile phone terminal 2 and the availability, and information on the search result To include the rank information.

  Note that the priority conditions in this case are not limited to the distance from the current position and the availability, but other conditions such as low cost may be used, and these are further added as conditions. You may do it. In this embodiment, the priority condition is determined in advance on the search server 11 side. However, the priority condition is specified by the user of the mobile phone terminal 2 and sent to the search site 1 together with the search condition. Anyway. Of course, the priority condition may be included in the search condition.

  Further, in this embodiment, the search site 1 receives from the map data providing unit 11 the surroundings of the current position based on the current position information of the mobile phone terminal 2 included in the search request sent from the mobile phone terminal 2. The map data is read out and transmitted to the mobile phone terminal 2 that has requested the search.

  The mobile phone terminal 2 temporarily stores the map data from the search site 1 in the map data memory. The mobile phone terminal 2 uses the map data stored in the map data memory to display a map around the current position on the display screen, and stores that match the search conditions of the search results received from the search site 1 Are superimposed on the map of the display screen with a predetermined mark.

  The above functions are functions for searching around the current position that the search site 1 has conventionally provided as one of the search services.

  In this embodiment, the mobile phone terminal 2 is configured to be able to perform a function of searching for the vicinity of the current position with display using the AR technology described below, in addition to the function of searching for the vicinity of the current position described above. Yes. In this embodiment, the function of searching for the vicinity of the current position accompanied by the display using the AR technology is mainly executed in the mobile phone terminal 2 by the support program provided from the search site 1 to the mobile phone terminal 2. It is configured.

  In this embodiment, a function for searching around the current position with display using the AR technology is added to the function for searching around the current position provided in the related art. It is almost equivalent to what added the function.

  In other words, in this embodiment, the mobile phone terminal 2 uses the support program acquired from the search site 1 to execute the function of searching for the vicinity of the current location with display using the AR technology. A mark corresponding to the store that matches the search condition as a search result is superimposed on the camera image of the camera unit and displayed in AR. In this case, in this embodiment, the direction of the optical axis of the imaging lens of the camera unit of the mobile phone terminal 2, that is, the shooting direction of the camera unit is an elevation angle in the vicinity of the horizontal direction for shooting the surrounding buildings. If the display screen is facing upward and the camera unit is facing downward for the user to view the display screen while using the AR technology, the AR display function is The search result is displayed in the same manner as in the conventional case.

  In the following, this embodiment having the function of searching around the current position with display using the AR technology will be described in more detail.

[Hardware configuration example of mobile phone terminal 2]
FIG. 2 is a diagram illustrating a hardware configuration example of the mobile phone terminal 2 as an example of the mobile terminal used in the search system according to the embodiment of the present invention. This example is a case having a configuration of a mobile phone terminal called a so-called smartphone.

  That is, as shown in FIG. 2, in the mobile phone terminal 2 of this example, a control unit 101 configured by a microcomputer is connected via a system bus 100 to a wireless communication circuit unit 102, a transmission / reception information processing unit 103, a call circuit unit 104. , Display interface 105, touch panel interface 106, GPS positioning unit 107, memory 108, camera unit 109, map data memory 110, gyro sensor 111, geomagnetic sensor 112, and acceleration sensor 113, respectively.

  The control unit 101 is for controlling the telephone function of the mobile phone terminal 2 and other various control processes. In this example, the control unit 101 executes a program stored in the memory 108 to support processing for accessing the search site 1 to be described later and peripheral search processing with display using AR technology. Various application processes such as programs are executed and controlled.

  The wireless communication circuit unit 102 is for connecting to the communication network 3 through a base station of a mobile phone network. The transmission / reception information processing unit 103 analyzes information received through the wireless communication circuit unit 102, transfers the information to each necessary unit through the system bus 100, and generates transmission information to be transmitted to the communication network, under the control of the control unit 101. Then, the data is transferred to the wireless communication circuit unit 102.

  The call circuit unit 104 is a telephone voice process, to which a speaker 114 constituting a receiver and a microphone 115 constituting a transmitter are connected.

  The display interface 105 is connected to a display 116 made of, for example, an LCD (Liquid Crystal Display). A touch panel 117 is provided on the display screen of the display 116, and the touch panel 117 is connected to the touch panel interface 106.

  In this embodiment, operation inputs such as operation buttons can be performed through the touch panel 117, and the display screen of the display 116 is a character input key operation screen, a setting reception screen, or the like according to the control of the control unit 101. Is displayed. The user can perform a predetermined operation input by performing a predetermined behavior of touching the touch panel 117 with a finger or sliding the touch panel 117 with a finger.

  The touch panel interface 106 sends an operation detection signal corresponding to an operation instruction input by the user through the touch panel 117 to the control unit 101 through the system bus 100. The control unit 101 detects what operation input has been made by the user from the operation detection signal received from the touch panel interface 106, and performs control processing according to the detection result.

  The GPS positioning unit 107 receives a radio wave from a GPS satellite, measures and detects the current position of the mobile phone terminal 2, and transfers position information of the detected current position to the control unit 101 through the system bus 100.

  The memory 108 is a storage unit for various data used in the mobile phone terminal 2. In this embodiment, the memory 108 is an application program such as a peripheral search support program with display using augmented reality acquired from the search site 1 described later. Is provided with a program storage unit 108M.

  The camera unit 109 includes an imaging lens (not shown) and has a function of imaging a subject in a predetermined field angle range in the optical axis direction of the imaging lens. When the camera unit 109 is activated, a camera image (monitor image) captured through the imaging lens and captured as a still image or a moving image is displayed on the display screen 117D of the display 116.

  In this embodiment, the map data memory 110 is a memory for storing map data in a predetermined range centered on the current position acquired from the map data providing unit 12 of the search site 1.

  The gyro sensor 111, the geomagnetic sensor 112, and the acceleration sensor 113 are sensors for detecting the orientation of the camera unit 109 by detecting the attitude of the mobile phone terminal 2. The gyro sensor 111 is a sensor for detecting the angular velocity and angle of the mobile phone terminal 2. The geomagnetic sensor 112 is a sensor for detecting which direction the mobile phone terminal 2 indicates east, west, south, and north. The acceleration sensor is a sensor for detecting acceleration in the front-rear direction, the left-right direction, and the up-down direction.

  In order to detect the orientation of the mobile phone terminal 2, only the geomagnetic sensor 112 may be used. However, in this embodiment, the control unit 101 includes the gyro sensor 111 and the acceleration sensor 113 in addition to the geomagnetic sensor 112. By using this, the orientation of the mobile phone terminal 2 is detected more precisely, and thereby the orientation of the optical axis of the imaging lens of the camera unit 109, that is, the shooting direction by the camera unit 109 is detected more precisely. .

[Processing operation for searching around the current position in the mobile phone terminal 2]
Processing operations at the time of executing the function of searching for the vicinity of the current position accompanied by display using the AR technology in the mobile phone terminal 2 having the above configuration will be described with reference to the flowcharts of FIGS.

  First, the control unit 101 determines whether any function item in the menu display is selected through the touch panel 117 of the display 116 (step S101), and when determining that the function item is not selected. Other processes are performed (step S102), and the process returns to step S101.

  When it is determined in step S101 that any one of the function items in the menu display has been selected, the control unit 101 determines whether or not the selected function item is a current location search (with AR display) ( Step S103). When it is determined in step S103 that the selected function item is not the current location vicinity search (with AR display), the control unit 101 performs processing of the other selected function items (step S104). After the process ends, the process returns to step S101.

  If it is determined in step S103 that the selected function item is the current location periphery search (with AR display), the control unit 101 accesses the search site 1 through the communication network 3 to search the current location periphery (AR display). Applicable support program for executing the function is acquired from the search site 1 and activated (step S105). The following processing of the control unit 101 is processing according to the support program.

  The control unit 101 displays a search condition input screen on the display screen of the display 116 to prompt the user to input the search condition (step S106). The search condition input includes, for example, a selection operation of a genre such as karaoke, bar, late-night business, hotel, park, and a direct input operation or selection operation of conditions such as the number of people, a budget, and a scene.

  Then, the control unit 101 determines whether or not an input of a user's search condition through the touch panel 117 has been accepted (step S107). When it is determined that an input of the search condition has not been accepted, whether or not to accept an end operation. (Step S108). If it is determined in step S108 that an end operation has not been accepted, the control unit 101 returns the process to step S107, and repeats the processes in and after step S107. If it is determined in step S108 that the termination operation has been accepted, the control unit 101 terminates the current location search (with AR display) function, such as stopping the support program and disconnecting from the search site 1. Perform (step S109), and then return the process to step S101.

  When it is determined in step S107 that an input of search conditions has been received, the control unit 101 determines whether an operation instruction for a search request transmission instruction has been received (step S110). When it is determined in step S110 that an operation instruction for transmitting a search request is not accepted, the control unit 101 determines whether or not the search condition has been changed (step S111), and determines that the search condition has not been changed. Sometimes, the process returns to step S110. If it is determined in step S111 that the search condition has been changed, the control unit 101 returns the process to step S107 to accept the changed search condition.

  When it is determined in step S110 that an operation instruction for a search request transmission instruction has been received, the control unit 101 acquires the current position information measured by the GPS positioning unit 107 (step S121 in FIG. 4), and has received it. A search request with the search condition and the current position information is transmitted to the search site 1 (step S122).

  In response to the transmission of the search request, the search site 1 sends information such as a store that matches the search condition as search result information. Therefore, the control unit 101 receives the search result information from the search site 1, and displays the search result display information based on the search result information on the display screen of the display 116 (step S123).

  An example of the search result display screen displayed in step S123 is shown in FIG. That is, in response to a search request from the mobile phone terminal 2, the search site 1 searches for a store that matches the search conditions in the surrounding area of the current location of the mobile phone terminal 2, and stores the store name, address, location information, The distance from the current position and availability information are included in the search result information and sent. Furthermore, in this embodiment, as described above, the distance from the current position and the availability are used as priority conditions, and ranks corresponding to the priority conditions are included in the search results and sent.

  Therefore, based on the search result information, the control unit 101 of the mobile phone terminal 2 displays the evaluation obtained by converting the store name, the distance from the current position, the free space, and the rank into marks as shown in FIG. . In this example, the evaluation is displayed with a ☆ mark so that the higher the rank corresponding to the priority condition, the larger the number of ☆. It goes without saying that the mark is not limited to the ☆ mark.

  In this embodiment, since the map data of the area around the current position of the mobile phone terminal 2 is sent from the search site 1, the control unit 101 acquires the map data and stores it in the map data memory 110. Temporary storage is performed (step S124).

  Next, the control unit 101 determines whether or not an end operation has been performed by the user of the mobile phone terminal 2 (step S125). When it is determined in this step S125 that the termination operation has been performed, the control unit 101 terminates the current location surrounding search (with AR display) function such as stopping the support program and disconnecting from the search site 1. (Step S126) After that, the process returns to Step S101. In this example, the map data temporarily stored in the map data memory 110 is deleted from the map data memory 110 in this end process.

  If it is determined in step S126 that the user has not performed the end operation, the control unit 101 sets the shooting direction of the camera unit 109 to a predetermined elevation angle range in the vicinity of the horizontal direction for shooting the surrounding buildings. It is determined whether or not (step S127). That is, in this embodiment, in step S127, as shown in FIG. 8, the optical axis direction AR of the imaging lens 109L of the camera unit 109 of the mobile phone terminal 2 is set in the vertical direction centering on the horizontal direction, for example, θ It is determined whether or not it is within the range of 15 degrees (± 15 degrees). Here, θ = 15 degrees is an example, and is not limited to this range. Further, the value of θ may be set by the user.

  When it is determined in step S127 that the shooting direction of the camera unit is not within the predetermined elevation angle range, the control unit 101 accepts an instruction to display the search result on the map from the user via the touch panel 117. It is determined whether or not (step S131 in FIG. 5). When it is determined in this step S131 that an instruction to display the search result on the map has been received, the control unit 101 searches the map display image around the current position based on the map data held in the map data memory 110. An image in which the search results acquired from the site 1 are superimposed and displayed is generated and displayed on the display screen 116D of the display 116 (step S132).

  FIG. 9 shows an example of an image in which the search result is superimposed and displayed on the map displayed in step S132. In FIG. 9, a mark Ma indicated by hatching is superimposed on the position of the store that matches the search condition. As shown in the map display screen in the example of FIG. 9, since the map is a two-dimensional display, if stores as bases that match the search conditions exist on different floors of the same building, the search conditions The mark display indicating the matching stores overlaps, making it difficult to determine, and it is not known on which floor of the building there are stores that match each search condition.

  After step S132, the control unit 101 returns the process to step S125, and repeats the processes after step S125. In addition, although illustration is abbreviate | omitted, on the display screen of the search result on this map, you may enable it to request | require guidance to the store, for example, specifying the specific mark Ma as a target store. . In that case, the navigation request is sent to the search site 1 and a navigation support program is sent from the search site 1 in response to the navigation request. Navigation can be performed.

  In step S131, when it is determined that the display instruction of the search result on the map is not received from the user, the control unit 101 instructs the user to return to the list display through the touch panel 117. It is determined whether or not it has been accepted (step S133). If it is determined in step S133 that an instruction to return to the list display from the user is not accepted, the control unit 101 returns the process to step S125, and repeats the processes after step S125.

  If it is determined in step S133 that an instruction to return to list display from the user has been received, the control unit 101 displays the display image on the display screen 116D of the display 116 as a list of search results as illustrated in FIG. The display image is returned to the display (step S134). After step S134, the control unit 101 returns the process to step S125, and repeats the processes after step S125.

  Next, when it is determined in step S127 that the shooting direction of the camera unit 109 is within a predetermined elevation angle range in the vicinity of the horizontal direction for shooting a surrounding building, the control unit 101 displays the display screen 116D on the display 116. Then, a monitor image (referred to as a camera image) being shot by the camera unit 109 is displayed (step S128).

  Then, the control unit 101 corresponds to the current position of the mobile phone terminal 2, the shooting direction of the camera unit 109, and the angle of view shot by the camera unit 109 (the range of the camera video displayed on the display screen 116D of the display 116). ) And the map data stored in the map data memory 110, the camera image displayed on the display screen 116D is estimated (step S141 in FIG. 6).

  That is, from the current position of the mobile phone terminal 2, the shooting direction of the camera unit 109, and the angle of view shot by the camera unit 109, the camera unit 109 starts from the standing position of the user holding the mobile phone terminal 2. The shooting direction and range are specified, and from the map data in the specified range, it is possible to estimate which building is in the range and how the road looks. A camera image displayed on the screen 116D is estimated.

  Next, the control unit 101 uses the position information of the store that matches the search condition as link information, examines the relationship with the map data, and estimates the position of the store that matches the search condition on the camera image estimated in step S141. (Step S142). Here, the estimated position on the camera image is a display screen position indicating which position in the display screen 116D.

  In this case, as described above, in this embodiment, the store location information in the map data exists on any floor of the building if the store exists in the building as described above. That information is also included. Therefore, the store position (display screen position) that matches the search condition on the camera image estimated in step S141 can be estimated in consideration of the number of floors in the building where the store exists.

  Next, the control unit 101 refers to the rank that is the evaluation of the store that matches each estimated search condition, and the number of ☆ marks corresponding to the rank is displayed on the display screen 116D of the display 116. It is superimposed on the video and displayed at each display screen position (step S143).

  An example of a display image on the display screen 116D of the display 116 at this time is shown in FIG. The background portion Px in FIG. 10 is a camera video captured by the camera unit 109, and corresponds to a store that matches the search condition estimated from the camera video as described above and the display screen position determined. * The mark MK is displayed superimposed.

  The user of the mobile phone terminal 2 matches the search condition as a search result by looking at the image displayed on the display screen 116D and comparing with the scenery actually viewed from his / her standing position. It is possible to intuitively recognize which store the evaluation rank is reflected on.

  Next, the control unit 101 determines whether or not a specific ☆ mark position, that is, a specific store position is touched by the user through the touch panel 117 on the display screen 116D (step S144). When it is determined in step S144 that the specific store position is touched, the control unit 101 includes a request for detailed information on the store corresponding to the touched ☆ mark, for example, including the location information of the store as identification information. And transmitted to the search site 1 (step S145).

  Then, the detailed information of the designated store sent from the search site 1 is received, and the received detailed information is displayed on the display screen 116D of the display 116 (step S146).

  Next, the control unit 101 determines whether or not the shooting direction of the camera unit 109 is out of the predetermined elevation angle range (step S147). If it is determined in step S144 that the specific ☆ mark position is not touched through the touch panel 117, the control unit 101 bypasses steps S145 and S146, and proceeds to step S147.

  When it is determined in step S147 that the shooting direction of the camera unit 109 is in the predetermined elevation angle range, the control unit 101 determines whether or not the shooting direction of the camera unit 109 is changed (step S148). When it is determined that the direction has not been changed, the process returns to step S144, and the processes after step S144 are repeated.

  If it is determined in step S148 that the shooting direction of the camera unit 109 has been changed, the control unit 101 returns the process to step S141, and repeats the processes after step S141 described above.

  If it is determined in step S147 that the shooting direction of the camera unit 109 is no longer within the predetermined elevation angle range, in this example, the control unit 101 returns the process to step S132 and displays the display screen 116 on the display 116. The image is changed to a display image in which the search result is superimposed on the map. And the process after step S132 is repeated.

[Processing operation for searching around the current position in the search server 1]
Next, the processing operation in the search server 1 in this embodiment will be described with reference to the flowchart of FIG.

  The search server 1 determines whether or not an access for current location search (with AR display) has been received from the mobile phone terminal 2 (step S201), and has not received access for current location search (with AR display). Is determined, other processing is performed (step S202), and then the processing returns to step S201.

  If it is determined in step S201 that the current location search (with AR display) access from the mobile phone terminal 2 has been received, the search server 1 transmits the support program to the accessed mobile phone terminal 2 (step S201). S203). Next, the search server 1 determines whether or not a search request has been received from the mobile phone terminal 2 (step S204). When it is determined that the search request has not been received, the search server 1 issues an end instruction from the mobile phone terminal 2. It is determined whether or not it has been received (step S205).

  If the search server 1 determines in step S205 that an end instruction has not been received from the mobile phone terminal 2, the search server 1 returns the process to step S204 and waits for reception of a search request. If it is determined in step S205 that an end instruction has been received from the mobile phone terminal 2, an end process such as disconnecting the communication path with the mobile phone terminal 2 is performed (step S206), and then the process returns to step S201. .

  If the search server 1 determines in step S204 that a search request has been received from the mobile phone terminal 2, the search request 1 within a predetermined range centered on the current position of the mobile phone terminal 2 included in the search request. Search for stores that match the search criteria included in the. Then, for each store that matches the search condition extracted by the search, the rank based on the priority condition such as the distance from the current position of the mobile phone terminal 2 and the free space reported and registered from each store. Is granted. Then, the search result including the store information that matches the search condition and the rank information is transmitted to the mobile phone terminal 2 that has requested the search (step S207).

  Next, the search server 1 causes the map data providing unit 12 to transmit the map data around the current position included in the search request from the mobile phone terminal 2 to the mobile phone terminal 2 (step S208). ).

  Next, the search server 1 determines whether or not the detailed information acquisition request for the specific store that matches the search condition is received from the mobile phone terminal 2 (step S209), and receives the detailed information acquisition request. When it is determined that the search has been made, the search unit 11 searches for detailed information about the specified store, and transmits the detailed information of the specified store obtained as a result of the search to the mobile phone terminal 2 (step S210). ).

  Following step S210, the search server 1 determines whether an end instruction has been received from the mobile phone terminal 2 (step S211). If the search server 1 determines in step S211 that it has not received an end instruction from the mobile phone terminal 2, the search server 1 returns the process to step S209. If it is determined in step S211 that an end instruction has been received from the mobile phone terminal 2, an end process such as disconnecting the communication path with the mobile phone terminal 2 is performed (step S206), and then the process returns to step S201. .

[Effect of the embodiment]
As described above, the mobile phone terminal 2 in the search system of the above-described embodiment makes a search request to the search server 1 and sets the shooting direction of the camera unit 109 within a predetermined elevation angle range near the horizontal direction. Thus, a mark corresponding to a base such as a store that matches the search condition is superimposed on the camera image and displayed as an AR. Therefore, the user compares the scenery that he / she actually sees with the image displayed on the display screen of the mobile phone terminal 2, so that the base that matches the search condition can be actually found from the current standing position. You can intuitively know which area is visible.

  Moreover, according to this embodiment, it is possible to display the detailed information of the store on the display screen simply by touching the mark corresponding to the store that matches the search condition in the AR-displayed camera image. Convenient to.

  Further, when the shooting direction of the camera unit 109 is not within a predetermined elevation range near the horizontal direction, the camera image is not a view of the field of view from the user's standing position, but a camera image of the user's feet or the sky. However, according to this embodiment, when the shooting direction of the camera unit 109 is such a direction, the search result is superimposed on a list of search results or a map image. Since things are displayed, it is easy to use.

  Further, in the above-described embodiment, the map data includes information on the floor of the building for the store. Therefore, in the AR display in which the mark MK is superimposed on the camera video, the camera video is displayed. A mark MK corresponding to a store that matches the search condition can be displayed corresponding to each floor of the building that appears. In this respect as well, it is possible to intuitively grasp the store as a search result.

[Other Embodiments]
In the above-described embodiment, estimation of the camera image by the camera unit 109 of the mobile phone terminal 2 and estimation of the display screen position on which the mark corresponding to the base corresponding to the search condition is superimposed is performed by the mobile phone terminal 2. However, it is also possible to perform the search at the search site 1 by sending information about the shooting direction of the camera unit 109 from the mobile phone terminal 2 to the search site 1.

  That is, in this case, after receiving the search result, the control unit 101 of the mobile phone terminal 2 monitors the shooting direction of the camera unit 109, and the shooting direction of the camera unit 109 is a predetermined elevation angle near the horizontal direction. When it is within the range, information on the shooting direction of the camera unit 109 at that time is transmitted to the search site 1 through the communication network 3.

  In the search site 1, after the search result is transmitted to the mobile phone terminal 2, reception of information on the shooting direction of the camera unit 109 is monitored from the mobile phone terminal 2. When the search site 1 receives information on the shooting direction of the camera unit 109 from the mobile phone terminal 2, the search site 1 and the current information on the mobile phone terminal 2 included in the search request received earlier. From the information on the current position of the mobile phone terminal 2, the information on the angle of view of the camera unit 109 and the map data of the map data providing unit 12, which are sent together with the information on the position or the shooting direction of the camera unit 109, In the same manner as described above, a camera image that will be displayed on the display screen of the mobile phone terminal 2 is estimated.

  In this case, the angle of view of the camera unit 109 may be sent from the mobile phone terminal 2 to the search site 1 together with information on the shooting direction, or the search site 1 may receive the identification information of the mobile phone terminal 2 of each company. The correspondence between each and the angle of view of the mounted camera unit 109 is registered and stored, and the identification information is sent from the mobile phone terminal 2 to the search site 1. 1 may grasp.

  Then, the search server 1 further estimates and calculates the position (display screen position) on the camera video of a store that matches the search conditions estimated to be displayed on the estimated camera video. Then, the search server 1 transmits the calculated position on the camera image (display screen position) information to the mobile phone terminal 2 in association with the store identification information that matches the search condition.

  Then, the mobile phone terminal 2 receives the information on the position (display screen position) on the camera image and the store identification information that matches the search condition, and performs the camera image on the display screen 116D in the same manner as described above. In addition, the number of ☆ marks corresponding to the rank of the store that matches the search condition are displayed in a superimposed manner.

  In the case of this embodiment, since the support program of the mobile phone terminal 2 does not require the estimation of the camera image and the estimation process of the position where the mark is superimposed, it can be simple and the load is light. Become.

[Other Embodiments or Modifications]
In the above-described embodiment, the shooting direction of the camera unit of the mobile phone terminal 2 is set to a predetermined elevation angle range in the vicinity of the horizontal direction, so that the display screen 116D automatically corresponds to the store that matches the search condition on the camera image The image on which the ☆ mark MK is superimposed is displayed, but only when there is a user's explicit operation such as a predetermined operation through the touch panel, the display screen 116D matches the search condition on the camera video. Of course, it is also possible to display an image on which the ☆ mark MK corresponding to the store to be displayed is superimposed.

  In the above-described embodiment, the mark associated with the base that matches the search condition displays the store name, the distance from the current position, the availability, and the rating converted into the mark. However, it goes without saying that the mark associated with the base that matches the search condition is not limited to the one that displays such an evaluation. As described above, the mark is not limited to the ☆ mark, and various marks can be used.

  Further, in the above-described embodiment, the map data acquisition and holding means of the mobile phone terminal 2 sends the current position information to the search site 1 from the map data providing unit 12 of the search site 1 to the current location information. The map data around the location was acquired. However, the map data held in the map data acquisition and holding means is not limited to this method.

  For example, when the mobile phone terminal 2 includes a loading unit for a large-capacity card-type memory, a large-capacity card-type memory that stores map data for a region to be used is loaded. The data acquisition and holding means may read and hold map data around the current position from the card type memory. Further, instead of using the card-type memory, the mobile phone terminal 2 may incorporate in advance a large-capacity memory that stores map data of the user's action range.

  In the above-described embodiment, the function of searching for the vicinity of the current position with display using augmented reality technology is executed in the mobile phone terminal 2 by the support program provided from the search site 1 to the mobile phone terminal 2. Was composed. However, instead of acquiring and executing the support program only when the search site 1 is accessed, an application program similar to the support program is installed in the mobile phone terminal 2 in advance. Also good.

  In the above-described embodiment, the mobile phone terminal 2 acquires the current position as measured by the GPS positioning unit 107. However, information on the current position of the mobile phone terminal 2 can be obtained from a mobile base station or Wi -You may make it use what was acquired from the access point of Fi (Wireless Fidelity).

  In the above-described embodiment, the portable terminal is described as a mobile phone terminal. However, the portable terminal to which the present invention can be applied is not limited to the mobile phone terminal, and has a display screen and a camera unit. And any portable terminal can be used as long as it can acquire current position information and can access the search server through a communication network.

  DESCRIPTION OF SYMBOLS 1 ... Search site, 2 ... Mobile phone terminal, 3 ... Communication network, 101 ... Control part, 107 ... GPS positioning part, 109 ... Camera part, 110 ... Map data memory, 111 ... Gyro sensor, 112 ... Geomagnetic sensor, 113 ... Accelerometer 116, display

Claims (8)

A search system configured by connecting a search unit and a portable terminal through a communication network,
The portable terminal is
A camera unit;
A display screen;
Means for acquiring current location information of the terminal;
Detecting means for detecting a shooting direction of the camera unit;
The search condition and the current location information of the terminal are sent to the search unit through the communication network, and the search result relating to the base that matches the search condition includes the location information of the base that matches the search condition. Means for obtaining
Map data acquisition and holding means for holding map data around at least the current position of the terminal;
The camera image by the camera unit displayed on the display screen of the portable terminal is estimated from the shooting direction of the camera unit and the map data held in the map data acquisition holding unit, and the search condition The estimated position on the camera image of the base that matches the position is estimated, and in the camera video displayed on the display screen, the estimated screen position is associated with the base that matches the search condition. Display control means for displaying superimposed marks;
A detection system comprising: The search unit is configured to send, to the portable terminal, information related to the rank that has been ranked based on a predetermined priority condition for the base that matches the search condition.
The mark associated with the base that matches the search condition displayed in a superimposed manner on the camera image displayed on the display screen in the portable terminal differs according to the rank. The search system described in. The search system according to claim 2, wherein the predetermined priority condition is a vacancy status of the base and a distance from the current position of the portable terminal. The portable terminal, when a predetermined mark superimposed on the camera image is specified on the display screen, a request for acquiring detailed information of a site that matches the search condition corresponding to the specified mark To the search unit,
The search system according to any one of claims 1 to 3, wherein the search unit transmits detailed information of a base that matches the search condition corresponding to the designated mark to the portable terminal. . The display control means of the portable terminal is
The camera image on which the mark is superimposed is displayed on a display screen when the determination unit determines that the shooting direction of the camera unit is in the predetermined elevation angle range. The search system in any one of -4. The display control means of the portable terminal is
When the determination unit determines that the shooting direction of the camera unit is not within the predetermined elevation angle range, the search condition is displayed on a map displayed on the display screen based on the stored map data. The search system according to any one of claims 1 to 4, wherein a map image on which information related to a base that matches is displayed. A computer provided in a portable terminal comprising a camera unit, a display screen, and means for acquiring current position information of the terminal itself;
Detecting means for detecting a shooting direction of the camera unit;
Means for obtaining current location information of the terminal itself;
The search condition and the current location information of the terminal are sent to the search unit through the communication network, and the search unit includes a search result regarding the base that matches the search condition including the location information of the base that matches the search condition. Means to obtain,
Map data acquisition and holding means for acquiring and holding map data around the current position of the terminal, from a map data providing unit;
The camera image by the camera unit displayed on the display screen of the portable terminal is estimated from the shooting direction of the camera unit and the map data held in the map data acquisition holding unit, and the search condition The estimated position on the camera image of the base that matches the position is estimated, and in the camera video displayed on the display screen, the estimated screen position is associated with the base that matches the search condition. Display control means for superimposing and displaying the mark,
A program for portable terminals to function as A search system configured by connecting a server and a portable terminal through a communication network,
The portable terminal is
A camera unit;
A display screen;
Means for acquiring current location information of the terminal;
Detecting means for detecting a shooting direction of the camera unit;
Means for sending a search condition, current position information of the terminal itself, and information on a shooting direction of the camera unit to the server through the communication network, and acquiring a search result regarding a base that matches the search condition from the server When,
With
The server
A base that matches the search condition from the portable terminal is searched in a range based on the current position of the portable terminal, and matches the search condition including position information of the base that matches the search condition. Means for sending a search result relating to a base to the portable terminal through the communication network;
The camera image by the camera unit displayed on the display screen of the portable terminal from the information on the current position from the portable terminal, the information on the shooting direction of the camera unit, and the held map data. And estimating the screen position estimated as the position on the camera image of the base that matches the search condition, and information on the screen position of the base that matches the estimated search condition is transmitted through the communication network. Means for sending to the portable terminal;
With
Further, the portable terminal is configured to use the estimated screen in the camera image displayed on the display screen based on the information on the screen position of the base that matches the search condition received from the server. A detection system comprising: a display control unit that superimposes and displays a mark associated with a base that matches the search condition at a position.

Images