JP5315111B2 - Terminal device, information presentation system, and terminal screen display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a terminal device, an information presentation system and a terminal screen display method, capable of preventing complication of a screen, even if an amount of information displayed in a video of a real space increases. <P>SOLUTION: The terminal device includes: a camera 106 for acquiring the video of the real space; a display unit 102 for displaying the screen; a memory 107 for storing extended reality sense information associated to position information; and a control unit 108 for controlling operation of each part, and for achieving an information processing function. The control unit 108 makes the display unit 102 display the video of the real space acquired by the camera 106, and performing processing of determining a display position of a graphics object corresponding to the extended reality sense information, superposing the graphics object on the video of the real space in the display position, and making the display unit 102 draw it. The control unit 108 performs processing of making the display unit 102 display a plurality of pieces of the extended reality sense information wherein the display positions approach as one set graphics object. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a terminal device, an information presentation system, and a terminal screen display method.

  The mobile communication technology in the mobile phone business has gone from the first generation of the analog system to the second generation of the PDC system and the GSM system, etc. that realized data communication by digital technology in addition to voice calls, and now it is faster data Development continues to the third generation such as W-CDMA, which provides various services using communication and multimedia. In addition, in mobile communication terminals that use such communication technology, as a result of technological innovations in the fields of processors such as CPUs, memories, various peripheral devices, and mounting technologies, both multi-functionality and high functionality have been achieved. Has achieved tremendous progress.

  As a result of the advanced development of mobile communication and terminal technology, various communication services such as video calling, e-mail, Web browsing, chat, etc. are realized on mobile phones in addition to voice calls. In addition, various devices such as liquid crystal / organic EL display modules, camera modules, non-contact IC card modules, infrared communication modules, Bluetooth (registered trademark) communication modules, WiFi communication modules, and GPS modules are installed in multifunctional terminals. Has been. In addition, multi-functionalization of terminals is also progressing in terms of software, and various application software for controlling various modules and various application software such as viewers, multimedia players, browsers, mailers, games, PIMs, etc. are installed on the operating system. It is implemented, and extremely advanced functions can be realized by simultaneously executing multiple applications by the multitask function.

  On the other hand, in the field of information processing, virtual reality (virtual reality) that constitutes a virtual reality space by computer graphics, augmented reality (AR; augmented reality) in which images in the real space are processed by a computer and information is added. Research has been promoted, and recently, augmented reality technology (AR technology) has attracted attention. There are various modes for realizing the AR technology, and one of them is one using a camera-equipped mobile phone. For example, MOBILIZY, Inc. of the United States provides a travel guide application product for Android, which is one of the open platforms for mobile phones, adding tourist information to a real-world landscape image captured by a camera. This application sends the location information of the terminal (by GPS positioning or manual input by the user) to the server, causes the peripheral tourist information of the information registered in the server to respond to the terminal from the server, and acquires the acquired tourist information. Augmented reality is realized by adding to the landscape video and displaying it.

  Japanese Patent No. 3700021 acquires an image with a camera, measures the position and orientation of the camera, and reads and reads data stored at a predetermined address of the storage device from the measured position and orientation of the camera. AR technology is disclosed that displays the combined data with an image acquired by a camera.

Japanese Patent No. 3700021

  In augmented reality technology that displays information on real-space video captured by a camera, information is superimposed on the real-space video as graphics objects such as information balloons and icons. When the number of screens increases, the screen becomes very complicated, and when graphics objects overlap, it becomes difficult to visually recognize and select them as operation targets. Such a problem is particularly remarkable in mobile terminals such as mobile phones and PDAs in which small display devices are used. Furthermore, although some recent mobile terminals employ a touch panel, it is extremely difficult to operate the touch panel in a state where graphics objects are densely arranged on the screen.

  In view of the above circumstances, the present invention provides a terminal device, an information presentation system, and a terminal screen display method capable of suppressing the complexity of the screen even when the amount of information to be displayed on the real space video is increased. Objective.

That is, the terminal device according to the present invention includes a camera unit that acquires a real space image, a display unit that displays a screen, a storage unit that stores augmented reality information associated with position information, and the operations of the units. A control unit that controls and realizes an information processing function, wherein the control unit displays the video of the real space acquired by the camera unit on the display unit and also displays a graphic corresponding to the augmented reality information A display device that determines a display position of a virtual object and performs processing for superimposing the graphics object on the video in real space at the display position and drawing on the display unit, wherein the control unit includes the augmented reality the display position determined based on the position information that is associated with the sensitive information, the position information of the plurality in close proximity to each other the augmented reality information one Performs processing for displaying on the display unit as if the graphics object, the position information between a plurality of augmented reality information about the Augmented Reality information distance is associated with a point far from the current point is close to each other It is characterized by expanding the range that is determined to be.

According to the terminal device of the present invention, the control unit can display a plurality of augmented reality information whose display positions are close to each other as one collective graphics object on the display unit. For this reason, since a plurality of adjacent augmented reality information is not displayed, complication of the screen can be avoided. In addition, since objects close to each other in the real space can be displayed together, it is possible to visually represent the positional relationship in the real space and to make the display easy to understand for the user. Further, the control unit facilitates the display of the graphics objects as one collective graphics object on the display unit as the display position of the graphics objects is farther from the shooting position. For this reason, since it can display in one, so that a display becomes difficult to visually recognize, simplification of a screen can be achieved further.

  Further, the control unit performs a process of drawing the graphics object corresponding to the type of the augmented reality information on the display unit, and the collective graphics object includes a plurality of graphics arranged under a predetermined condition. It is preferable that the objects have overlapping shapes. With this configuration, it is possible to easily recognize that the collective graphics object includes a plurality of graphics objects, and for example, it is possible to provide a display that allows the user to easily search for information.

  Further, when the control unit detects an operation of a user who selects the collective graphics object, the control unit displays the graphics of the plurality of augmented reality information displayed on the display unit as the collective graphics object. It is preferable to perform a process of arranging the objects in a ring shape and displaying them on the display unit. With this configuration, when the control unit detects a user operation, graphics objects collected as a collective graphics object are developed in a ring shape. For this reason, it is possible to provide a user interface capable of displaying detailed information only when the user needs it while simplifying the screen.

  In addition, the control unit has a larger number of the augmented reality information displayed on the display unit as the one collective graphics object than the number of the graphics objects that can be displayed in a circle at a time. In the case where the display unit displays the graphics objects arranged in a ring shape as many as the number that can be displayed, and when the user's operation that clearly indicates the moving direction of the graphics objects arranged in a ring shape is detected, It is preferable to perform a process of rotating the graphics objects arranged in a ring shape along the circumferential direction and replacing the displayed graphics objects. With this configuration, the control unit causes the graphics object arranged in a ring according to the operation to specify the moving direction to rotate and move along the circumferential direction, and the graphics object is replaced and displayed on the display unit. Is done. Therefore, it is possible to provide a user interface that is intuitive and easy to understand for the user and has excellent operability.

  In addition, when performing the process of arranging the graphics objects of the plurality of augmented reality information in a circle and displaying them on the display unit, the control unit adjusts the size of the graphics object based on a predetermined condition. It is preferable to change and display on the display unit. With this configuration, the control unit changes the size of the graphics objects arranged in a ring based on a predetermined condition. Therefore, it is possible to provide a user interface that is intuitive and easy to understand for the user.

  The control unit may change the transparency of the graphics object based on a predetermined condition when performing processing for arranging the graphics objects of the plurality of augmented reality information in a circular pattern and displaying the graphics objects on the display unit. Then, it is preferable to display on the display unit. With this configuration, the transparency of the graphics objects arranged in a circle is changed based on a predetermined condition by the control unit. Therefore, it is possible to provide a user interface that is intuitive and easy to understand for the user.

  Further, it is preferable that the control unit displays the graphics object of the augmented reality information satisfying a predetermined condition on the display unit so as to be positioned at the head of the collective graphics object. With this configuration, the graphics object of augmented reality information that satisfies a predetermined condition is arranged at the head of the set graphics by the control unit. For this reason, it can be set as the display which information search is easy for a user, for example.

  The control unit may detect a user's operation of dragging along the graphics object as a user's operation that clearly indicates a moving direction of the graphics objects arranged in a circle. Preferably, when the operation is slower than the threshold, the displayed graphics objects are replaced one by one, and when the operation is faster than the threshold, all the displayed graphics objects are replaced. It is. With this configuration, the control unit causes the graphics objects arranged in a ring according to the user's operation dragging at a lower speed than the threshold to rotate one by one and rotate, and the user dragging at a higher speed than the threshold. All of the graphics objects arranged in a ring according to the operation are switched and displayed on the display unit. Therefore, it is possible to provide a user interface that is intuitive and easy to understand for the user.

  In addition, when the control unit detects an operation of a user who selects the graphics object, the control unit loops the graphics object that displays an executable process for the augmented reality information related to the graphics object. If the user's operation for selecting one of the graphics objects displayed in a circle is displayed on the display unit so as to be arranged in a circle, processing related to the graphics object is detected. Is preferably performed. With this configuration, the control unit executes processing related to one graphics object selected by the user from among the circularly arranged graphics objects. Therefore, it is possible to provide a user interface that is intuitive and easy to understand for the user.

  Moreover, it is preferable that the control unit displays the augmented reality information on the display unit by changing the size of the graphics object based on a predetermined condition. With this configuration, the size of the graphics object is changed by the control unit based on a predetermined condition and displayed on the display unit. For this reason, it is possible to prevent information from being displayed on a flat plate in a real space image, and thus it is possible to provide an excellent augmented reality.

  Furthermore, it is preferable that the control unit displays the augmented reality information on the display unit by changing an information amount of the graphics object based on a predetermined condition. With this configuration, the control unit changes the information amount of the graphics object based on a predetermined condition and displays the information on the display unit. For this reason, it is possible to prevent information from being displayed on a flat plate in a real space image, and thus it is possible to provide an excellent augmented reality.

In addition, an information presentation system according to the present invention includes a terminal device that displays a screen by a display unit, and a server that is configured to be communicable with the terminal device and includes augmented reality information associated with position information. In the presentation system, the server displays an image of real space on the display unit, determines the display position of the graphics object corresponding to the augmented reality information based on the position information of the terminal device, The graphics object is superimposed on the video in real space at the display position and is drawn on the display unit, and a plurality of the augmented reality information close to the display position is displayed as one collective graphics object. A server control unit that performs processing to be displayed on the unit, and the server control unit is associated with the augmented reality information before The display position determined based on the position information, performs the location information to be displayed on the display unit a plurality of augmented reality information are close to each other as one set graphics object processing, the terminal device The range of determining that the position information is close to each other among a plurality of augmented reality information as the augmented reality information associated with a point far from the current point is expanded. Composed.

  According to the information presentation system of the present invention, the server control unit can display a plurality of augmented reality information whose display positions are close to each other as one collective graphics object on the display unit of the terminal device. For this reason, since a plurality of adjacent augmented reality information is not displayed on the display unit of the terminal device, complication of the screen of the terminal device can be avoided.

In addition, the terminal screen display method according to the present invention is a terminal that superimposes a graphics object corresponding to augmented reality information associated with position information on a real space image acquired by a camera unit and draws it on the display unit. A screen display method for determining a display position of a graphics object corresponding to the augmented reality information, and determining whether the display positions of a plurality of graphics objects are close to each other A proximity determining step, and a display step of displaying a plurality of the augmented reality information as one collective graphics object on the display unit when the display positions are close to each other, the proximity determining step Then, the display position is determined based on the position information associated with the augmented reality information, That serial said positional information between about augmented reality information more augmented reality information distance is associated with a point far from the current position of the terminal device to expand the determining range are close to each other Configured as a feature.

  According to the terminal screen display method of the present invention, a plurality of augmented reality information whose display positions are close to each other can be displayed on the display unit as one collective graphics object. For this reason, since a plurality of adjacent augmented reality information is not displayed, complication of the screen can be avoided.

  According to the present invention, it is possible to suppress complication of the screen even when the amount of information to be displayed on the real space video is increased.

It is the schematic which shows the structure of the information presentation system which has the terminal device which concerns on embodiment. It is a functional block diagram of the terminal device concerning an embodiment It is an example of a list of position information contents. It is a screen figure which displays augmented reality information by a round icon. It is explanatory drawing of the icon and collective graphics object which adjoin. It is a screen figure which superimposed the augmented reality information on the image | video of real space. It is an example of the icon which attached the pattern for every type. It is a screen figure explaining the screen operation | movement of an icon. It is a screen figure explaining the screen operation | movement of an icon. It is a screen figure explaining other screen operations of an icon. It is explanatory drawing explaining the other screen operation | movement of an icon. It is a screen figure explaining other screen operations of an icon. It is a functional block diagram of the augmented reality server which concerns on embodiment. It is a flowchart which shows operation | movement from starting of the terminal device which concerns on embodiment until it will be in the state of waiting for operation. It is a flowchart which shows the other operation | movement from starting of the terminal device which concerns on embodiment until it will be in the state of waiting for operation. It is a flowchart which shows the spiral menu operation | movement of the terminal device which concerns on embodiment. It is a flowchart which shows the disc menu operation | movement of the terminal device which concerns on embodiment. It is an indicator of disk variations.

  Hereinafter, modes for carrying out the present invention will be described. FIG. 1 schematically shows the configuration of an augmented reality system (information presentation system) 1 including a terminal device 100 according to an embodiment of the present invention and an augmented reality server 300 that provides augmented reality information to the terminal device 100. It is drawing shown in. The terminal device 100 is a mobile terminal such as a mobile phone that communicates by connecting to the communication network 150, and includes a camera unit and a display unit having an image acquisition function as described later. The augmented reality server 300 is a server computer system connected to the network 200 connected to the communication network 150. Similarly, the map information server 351, the route information server 361, and the location information are external servers connected to the network 200. The content server 371 and the terminal device 100 can communicate with each other. The augmented reality system 1 having such a configuration superimposes the information provided by the augmented reality server 300 on the image acquired by the user of the terminal device 100 with the camera directed to the real space, thereby allowing the user to augment the augmented reality. It provides services that give Hereinafter, the terminal device 100 and the augmented reality server 300 will be described in detail.

  First, the terminal device 100 will be described. FIG. 2 is a functional block diagram of the terminal device 100 in the present embodiment. The terminal device 100 is mainly directed to a position information acquisition unit 101 that acquires terminal position information, a display unit 102 that displays a screen, a key input unit 103 that receives input by a user's key operation, and a terminal. An orientation sensor 104 that detects an orientation, a communication unit 105 that communicates by connecting to a communication network 150, a camera unit 106 that acquires an image of real space, a storage unit 107 that stores a predetermined program and data, and each unit And a control unit 108 that realizes predetermined information processing.

  The position information acquisition unit 101 can use, for example, a GPS module that receives radio waves from a plurality of GPS (Global Positioning System) satellites and acquires current terminal position information from their intensities. Further, in addition to the GPS method, communication with the communication network 150 via the communication unit 105 acquires information on satellites that exist near the terminal as assist data, and an AGPS (Assisted Global Positioning System) method that performs positioning is performed. Use of a corresponding module is preferable because terminal location information can be acquired more quickly and accurately. However, the position information acquisition system is not limited to the one using GPS or AGPS, and can be configured to acquire terminal position information without depending on GPS. For example, a method may be used in which a plurality of wireless communication base stations such as WiFi and Bluetooth (registered trademark) are arranged in the space, and terminal location information is acquired based on the radio field intensity therefrom. It is also possible to use a method used in combination with GPS or AGPS.

  The display unit 102 includes an LCD module 113 and an LCD controller 112 for performing screen display, and a sensor module 114 and a position detection unit 111 that detect a contact position of the user in response to an operation in which the user touches the screen. A screen display function and a touch panel function are realized. That is, the display unit 102 receives data and control signals from the control unit 108 by the LCD controller 112 and displays a predetermined screen on the LCD module 113, and receives a user's operation for touching the screen by the sensor module 114. The detection unit 111 detects the contact position, and notifies the control unit 108 of an input of a position signal related to the contact position. In the terminal device 100 according to the present embodiment, augmented reality is provided to the user by adding and displaying the information acquired from the augmented reality server to the video of the real space acquired by the camera unit 106 according to the operation described later. In addition, a user interface is provided by a display function and a touch panel function.

  The key input unit 103 is an input device such as a keypad provided in the casing of the terminal device 100 and other buttons, and a key event that is a signal corresponding to each key or button is operated by the user. 108 is used for various operations or control.

  The direction sensor 104 is a module that detects a posture / angle in a direction in which the direction sensor 104 is facing and outputs a signal corresponding to the posture / angle to the control unit 108. The control unit 108 can recognize the posture / angle at which the user holds the terminal device 100 from the detection value of the direction sensor 104. A combination of a plurality of geomagnetic sensors can be used as the orientation sensor 104, and a motion sensor that combines a plurality of acceleration sensors in addition to the geomagnetic sensor may be used. A six-axis sensor that combines a triaxial geomagnetic sensor and a triaxial acceleration sensor, a five-axis sensor that combines a triaxial geomagnetic sensor and a biaxial acceleration sensor, and a gyro instead of an acceleration sensor Can be used for the direction sensor 104 as a motion sensor. When the motion sensor is used in this way, it is possible to detect not only information on the attitude and angle of the terminal device 100 but also the direction and speed of the operation in which the user moves or tilts the terminal device 100. .

  The communication unit 105 is a module for connecting to the communication network 150 and communicating with other terminal devices and server systems. Examples of the communication here include circuit-switched communication performed by communication connection with other terminal devices, packet-switched communication performed with a server system connected to the network 200 via the communication network 150, and the like. . Further, when acquiring the terminal location information by the AGPS method, the communication unit 105 cooperates with the location information acquisition unit 101 to assist the acquisition of the terminal location information.

  The camera unit 106 is a module that has a predetermined optical system and an image receiving element and provides a function of acquiring a digital image. When the terminal device 100 captures a digital image, the activated camera unit 106 normally repeats the process of acquiring data by the image receiving element at a predetermined exposure time from the subject image acquired by the optical system at a predetermined time interval. The live view function is realized by displaying the data on the display unit 102. In this state, when the user further performs a predetermined operation with the key input unit 103, the camera unit 106 generates image data with the shooting conditions set from the subject image acquired by the optical system, and the generated image data Is stored in the storage unit 107. The augmented reality system according to the present embodiment displays information that brings augmented reality to a real space image displayed on the display unit 102 by the above-described live view function.

  The storage unit 107 is a memory for storing predetermined information under the control of the control unit 108 and providing the stored information to the control unit 108. Further, the storage unit 107 stores various programs executed by the control unit 108, and the control unit 108 reads and executes them as appropriate. The storage unit 107 does not need to have a uniform configuration, and can be configured by appropriately combining various memories such as ROM, RAM, and Flash ROM, and further combined with a secondary storage device such as an HDD. It doesn't matter. The unique terminal ID assigned to the terminal device 100 is also stored in the storage unit 107 together with other data. The storage unit 107 stores various augmented reality information provided from an augmented reality server described later. FIG. 3 shows, as an example, a list of position information contents in the augmented reality information stored in the storage unit 107. The location information content is acquired from the location information content server 371 by the augmented reality server 300 in response to a request from the terminal device 100 and returned as a response to the terminal device 100. The list of the storage unit 107 includes content ID, content type, content name, and position information (point information) for each position information content, and further, the distance from the current terminal position of the terminal device 100 and the viewing angle. An item of whether or not is within is provided. Among these items, the information acquired from the augmented reality server 300 is recorded as it is for items other than the distance and the viewing angle, and the items calculated and determined by the control unit 108 of the terminal device 100 for the items of the distance and the viewing angle are recorded. It is recorded.

  The control unit 108 is a functional block that is virtually configured by executing a program stored in the storage unit 107 on a CPU (not shown). The control unit 108 includes a position information acquisition unit 101, a display unit 102, and the like of the terminal device 100. Various functions of the terminal device 100 are realized by exchanging data and control signals with the functional blocks such as the key input unit 103, the orientation sensor 104, the communication unit 105, the camera unit 106, and the storage unit 107. . In addition, the control unit 108 of the terminal device 100 according to the present embodiment transmits the request for the terminal location information acquired by the location information acquisition unit 101 to request augmented reality information corresponding to the terminal location information from the augmented reality server 300. In response to the response, the augmented reality is provided to the user. As a characteristic function in this case, the display unit 102 determines a display position on which the augmented reality information is superimposed on the real space image while displaying the real space image captured by the camera unit on the display unit 102. When a graphics object corresponding to the augmented reality information is superimposed and drawn at the display position, a plurality of augmented reality information close to the superimposed display position is displayed on the display unit 102 as one collective graphics object. To perform the process. Hereinafter, such processing will be described in detail.

  First, a process of determining the display position of the augmented reality information based on the attitude and angle of the camera unit 106 and superimposing it on the real space image will be described. In this process, first, the orientation / direction of the camera facing is derived from the detection value of the azimuth sensor 104, and the augmented reality stored in the storage unit 107 based on the horizontal viewing angle and the vertical viewing angle of the camera. It is determined whether or not the sense information falls within the field of view of the camera unit 106, and the result is recorded in a list of augmented reality information (see FIG. 3). Subsequently, it is assumed that the augmented reality information determined to be in the field of view is arranged in the coordinate space with the current terminal position of the terminal device 100 as the origin, and the horizontal position of the camera with the predetermined height of the origin as the center point. Based on the direction viewing angle, the vertical viewing angle, and the video size, the augmented reality information is transmitted and projected onto a rectangular plane determined at a predetermined distance from the origin, and the display position is determined from the position of the transmissive projection. Here, it is preferable to set the height of the center point to be transparently projected to a height of about 1.5 m in order to reproduce the situation in which the user uses the terminal device 100 while standing.

  After the above process, a process is performed in which each augmented reality information is drawn at the determined display position so as to be superimposed on the real space image. Although there is no particular limitation on such processing, each augmented reality information is drawn at a predetermined display position to create a transparent screen layer, and this screen layer is overlaid on the real space video on the display unit 102. The augmented reality information can be displayed at an appropriate position in the real space image. At this time, the augmented reality information in the present embodiment is displayed on the screen as a graphics object such as an icon, an information balloon, or a subwindow.

  At this time, the control unit 108 causes the display unit 102 to display a plurality of augmented reality information with close overlapping display positions as one collective graphics object. 4A, 4 </ b> B, and 4 </ b> C show examples of a screen that displays augmented reality information with a round icon, and FIG. 4A is an image of a real space acquired by the camera unit 106. (B) shows the image which drew the icon by calculating | requiring the display position of each augmented reality information by the above-mentioned method. Here, on this screen, the size of the icon is changed for each icon and displayed. The transparency of the icon may be changed for each icon and displayed. The change in size and transparency is changed according to predetermined conditions. As the predetermined condition, for example, the distance from the current location to the position indicated by the position information content, the height of the position indicated by the position information content, the date of construction of the building indicated by the position information content, the update date of the position information content, Information freshness, number of accesses, access time, recommendation, tags used by users and services for categorization, and the like are employed. For example, when the distance is adopted as the predetermined condition, the position information indicated by the icon is displayed larger as the position is closer to the current terminal point, and the position information indicated by the icon is closer to the current terminal point. Is displayed smaller. That is, an icon indicating a position closer to the current terminal point is displayed larger and clearly. On the other hand, an icon indicating a position far from the current terminal point is displayed with a smaller and more blurred (hazy) display. That is, the information amount of the icon is changed according to the distance. Note that the transparency display method may use a conventional 3D lettering technique, or may express perspective using brightness or a blue color. By displaying in this way, it is possible to prevent the augmented reality from being impaired by the information superimposed on the flat plate.

  However, as shown in FIG. 4, when the amount of augmented reality information displayed on the screen increases, the screen becomes complicated and the icons overlap and are difficult to view. Furthermore, in the present embodiment, the display unit 102 is provided with a touch panel function, but it is extremely difficult to touch and operate icons that are dense in the state illustrated in FIG.

  Therefore, in the present embodiment, as shown in FIG. 5A, for the icons (icon groups α, β, γ) whose display positions to be superimposed are close, as shown in FIG. As shown in b), it is displayed on the display unit 102 as a collective graphics object indicating that the icons are dense. As shown in FIG. 5B, the collective graphics object here is a symbol icon in which a plurality of icons overlap each other. FIG. 6 shows a screen on which augmented reality information is displayed on an image in real space by using such a collective graphics object. As shown in FIG. 6, the display of the screen is simplified by using the collective graphics object, and it is possible to prevent the screen from becoming complicated and difficult to view.

  The target to be displayed with such a collective graphics object can be, for example, a plurality of augmented reality information in which graphics objects overlap when rendered at a determined display position. As a result, it is possible to prevent the icons from overlapping and becoming invisible. Whether or not the graphics objects overlap may be detected by an AND condition by actually drawing individual graphics objects, or may be detected by calculation without drawing. Further, as shown in FIG. 10A, the screen is divided into a plurality of areas for management, and as shown in FIG. 10B, a plurality of augmented reality images in which the display positions to be superimposed are determined in the same area. Information can also be displayed on the display unit 102 as one collective graphics object. The area division management may be performed by dividing the projection plane. In this case, since it is not necessary to determine whether the graphics objects overlap each other, it is possible to reduce the load of detecting augmented reality information in which the display positions to be superimposed are close to each other. It is also possible to display a plurality of augmented reality information with close positional information (point information) on the display unit 102 as one collective graphics object. Details will be described with reference to FIG. FIG. 11 is a plan view of the map information in the imaging direction of the camera, and indicates the position information indicated by the position information content with a cross. As shown in FIG. 11, position information content (portion surrounded by a dotted line) where the position information is close among the position information contents indicated by X marks is displayed as a collective graphics object. In this case, the position information content can be displayed on the display unit 102 as a collective graphics object for each address. In this case, between the augmented reality information associated with points far away from the terminal device, the range determined to be close is expanded in consideration of the perspective effect in the real space image. It is preferable to do. As a result, since the position information content indicating the position far from the current position can be displayed as the collective graphics object, it is possible to avoid complication of the screen.

  FIG. 7A shows an example in which an icon for displaying position information content in the augmented reality information is provided with a pattern for each type. By using such a graphics object, the visibility of augmented reality information can be improved. In addition, as shown in FIG. 7B, the collective graphics object of the position information content can be provided with a pattern according to the type of the position information content in a place that satisfies a predetermined condition. As predetermined conditions, for example, the distance from the current location to the position indicated by the position information content, the height of the position indicated by the position information content, the date of construction of the building indicated by the position information content, or the update date of the position information content Information freshness, number of accesses, access time, recommendation degree, tags used by users and services for categorization, and the like are employed. For example, when distance is adopted as the predetermined condition, a pattern corresponding to the type of position information content at a location closest to the terminal device 100 can be attached. Further, the collective graphics object is displayed by arranging the graphics objects under the predetermined condition described above. By displaying in this way, the visibility of augmented reality information can be improved, and information retrieval can be facilitated.

  As described above, by displaying a plurality of augmented reality information with close display positions on the display unit 102 as one collective graphics object, it is possible to prevent the screen from becoming complicated and difficult to view. . Here, it is necessary to design an interface for individual augmented reality information from the collective graphics object. The simplest method is to use an interface in which a list of information is displayed on the display unit 102 by performing a key operation to move the cursor to a set graphics object, and target information is selected by performing a key operation again. is there. However, in the present embodiment, the following screen transition interface is proposed using the display unit 102 having a touch panel function.

  In this interface, when it is detected that the user touches the collective icon α shown in FIG. 6, the real space image displayed on the display unit 102 is fixed to that at the time of the touch, and the brightness of the screen is changed to dark. The At the same time, other icons and the like are deleted from the screen, and the collective icon α is moved to the center of the screen (FIG. 8A). Subsequently, in order to display each of the augmented reality information displayed as the collective icon α, an icon corresponding to each augmented reality information is displayed. Here, each icon is expanded from the collective icon α so as to spread while rotating (FIG. 8B), and a predetermined number of icons are arranged in a ring (ring shape) in order from the position information content with a short distance ( FIG. 8 (c)).

  In this way, when the user drags the surface of the display unit 102 along the icon ring on the screen displaying the icon, the icon ring displayed on the screen is dragged in the direction (circumferential direction of the ring). When the icon is rotated to the left and dragged leftward, the leading icon α1 disappears from the screen, and α9 following α8 is displayed instead (FIG. 9A). That is, the icon to be replaced is determined based on the rotational movement direction of the icon and the display position (icon placement position on the ring). In such an interface, the state where the augmented reality information displayed as the collective icon α is arranged in order from the closest distance on the spiral perpendicular to the screen as viewed from the front of the spiral. Modeled. That is, as in the real space image, the augmented reality information is displayed continuously from the near side to the back side of the user, and the icon display area spirals by dragging to rotate the screw. Go to the back of the screen. According to such an interface, the user holds the terminal device 100 with one hand, drags the screen with a thumb, moves a desired icon to a position where the thumb can reach, and performs a touch operation. 100 can be operated.

  Further, the screen shown in FIG. 8C may be displayed like the screen shown in FIG. The screen shown in FIG. 12 is a screen that visually represents the above-described spiral model. In this screen, the size of the icon is changed for each icon and displayed. The transparency of the icon may be changed for each icon and displayed. The change in size and transparency is changed according to predetermined conditions. As the predetermined condition, for example, the distance from the current location to the position indicated by the position information content, the height of the position indicated by the position information content, the date of construction of the building indicated by the position information content, the update date of the position information content, Information freshness, number of accesses, access time, recommendation, tags used by users and services for categorization, and the like are employed. For example, when the distance is adopted as the predetermined condition, the icon is displayed larger as the position information indicated by the position information content is closer to the current terminal point, and the position information indicated by the position information content is closer to the current terminal point. The closer to the position, the smaller the icon transparency. That is, the position information content indicating a position closer to the current terminal point is displayed with a larger and clearer icon. On the other hand, as the position information content indicates a position far from the current terminal point, the icon is displayed with a smaller blur (hazy). Note that the transparency display method may use a conventional 3D lettering technique, or may express perspective using brightness or a blue color. The icons are arranged in the order in which the predetermined conditions are satisfied and displayed in a ring shape, so that the size and transparency of the icons can be displayed so as to gradually change in the circumferential direction. Can be provided. Then, as shown in FIG. 12B, the spiral can be rotated in the dragged direction as in the screen shown in FIG. When dragging leftward, the leading icon α1 disappears from the screen and the icon α2 is moved to the position where the icon α1 was displayed. Then, the size and transparency of the icon α2 are changed so as to be equal to the size and transparency of the icon α1 positioned at the head. As described above, the ring-shaped icon is spirally displayed, so that time, distance, and the like can be intuitively expressed.

  When one of the icons is touched in the state shown in FIG. 8C or FIG. 9A, a set of menu icons that can be used for the augmented reality information displayed by the icon is displayed in a ring shape. (FIG. 9B). Even in this state, the icon can be rotated along the ring by dragging the screen in the same manner as described above. However, since the screen of FIG. 9B displays a fixed menu, the icons are not replaced while rotating as described above. A set of menu icons is represented as a disc in the center of the screen, and the screen transitions to the next or previous disc (set of menu icons) by quickly dragging the surface of the display unit 102. As described above, the terminal device according to the present embodiment provides an excellent augmented reality by using an interface with excellent visibility and recognizability with a physical operation using the touch panel function.

  Next, the augmented reality server 300 will be described. FIG. 13 is a functional block diagram of the augmented reality server 300 in the present embodiment. The augmented reality server 300 is a server system in which a Web server is resident. The augmented reality server 300 mainly includes a communication unit 301, a server control unit 310, and a user DB 316. Requests from the terminal device 100 are transmitted via the communication network 150 and the network 200. The data is acquired from the map information server 351, the route information server 361, or the location information content server 371 connected to the network 200 as needed, and a process of responding to the terminal device 100 as augmented reality information is performed.

  The communication unit 301 is a network module that communicates with an external server such as the terminal device 100, the map information server 351, the route information server 361, and the location information content server 371 by connecting to a network, and a publicly known one can be used. . The user DB 316 is a database machine in the system area of the augmented reality server 300, and records and manages the terminal ID, terminal position information, and handle name received from the terminal device 100 in association with each other.

  The server control unit 310 is a functional block that is virtually configured by executing a predetermined program by a CPU (not shown). More specifically, the server control unit 310 includes a map data acquisition unit 311 that acquires map data from the map information server 351, a route data acquisition unit 312 that acquires route data from the route information server 361, and a position from the position information content server 371. It comprises a location information content acquisition unit 313 that acquires information content, and a user data unit 315 that accesses the user DB 316. These map data acquisition unit 311, route data acquisition unit 312, location information content acquisition unit 313, and user data unit 315 are all based on the request transmitted from the terminal device 100, the augmented reality server API that is a corresponding CGI program. A predetermined process is performed by calling. Hereinafter, the map data acquisition unit 311, the route data acquisition unit 312 and the location information content acquisition unit 313 will be described in detail.

  The map data acquisition unit 311 is an API that is called when the terminal device 100 requests map data, and uses a public API for map data for a predetermined range centered on terminal location information transmitted from the terminal device 100. To the map information server 351. In response to this, the map information server 351 extracts corresponding map data from the map information DB 352 and returns it to the augmented reality server 300, and the augmented reality server 300 can display the acquired map data on the terminal device 100. To the terminal device 100. At this time, when the terminal device 100 requests map data, if the range and scale of the map data are specified as parameters, a process for specifying these and calling an API published by the map information server 351 I do.

  The route data acquisition unit 312 is an API that is called when the terminal device 100 requests route data, and the terminal device 100 is a location transmitted together with the terminal location information acquired by the location information acquisition unit 101 as a starting point. The route information server 361 is requested for route data having information (point information) as an end point. In response to this, the route information server 361 generates corresponding route data from the route information DB 362 and returns it to the augmented reality server 300, and the augmented reality server 300 can handle the acquired route data in the terminal device 100. To the terminal device 100. At this time, the route information server 361 may calculate and add the total required time and the total fee to the route data for reply.

  The location information content acquisition unit 313 is an API that is called when the terminal device 100 requests location information content. Here, the location information content refers to the location information (stations, airports, ports, bus stops, interchanges, service areas, gas stations, restaurants, taverns, convenience stores, department stores, hospitals, schools, signals, corners, intersections, etc.) Information on the facility associated with the (point information). For example, for restaurants, location information contents including addresses, telephone numbers, fax numbers, e-mail addresses, homepage URLs, introductions, business hours information, store images, etc. are published on services provided by information service providers. .

  The location information content acquisition unit 313 uses location information content existing within a predetermined distance centered on the terminal location information acquired by the location information acquisition unit 101 of the terminal device 100 as location information published by an information service provider or the like. Requests to the content server 371. In response to this, the location information content server 371 returns a list in which the location information content satisfying the conditions from the location information content DB 372 is extracted to the augmented reality server 300, and the augmented reality server 300 transmits the obtained location information content to the terminal device 100. Is replied to the terminal device 100 as a usable data format. At this time, the location information content acquisition unit 313 can request location information (point information) from the plurality of location information content servers 371, and in this case, the augmented reality server 300 has a plurality of location information content servers 371. Are integrated into a common format and replied to the terminal device 100. In addition, if the location information content server 371 used for each field of the location information content is registered, it is possible to request the location information content only for the field requested from the terminal device 100.

  In addition, the terminal ID and terminal position information of another terminal device are recorded in the user DB 316 together with the handle name of the user. However, such contents are also contents associated with the position information (point information). It can be handled equivalent to content. That is, the position information content acquisition unit 313 requests the user data unit 315 to request the user handle name recorded in association with the position information (point information) within a predetermined distance from the current terminal position of the terminal device 100. If the user data part returns the handle name list extracted from the user DB 316 in response to the augmented reality server 300 and replies from the augmented reality server 300 to the terminal device 100 in a common format, other location information content And can be used as content.

  Hereinafter, the operation of the terminal device 100 in the augmented reality system 1 configured as described above will be described with reference to flowcharts.

  FIG. 14 is a flowchart illustrating the operation on the terminal device 100 side in the present embodiment, from starting to displaying augmented reality information and waiting for a user operation.

  This operation is started when the user operates the key input unit 103 of the terminal device 100 to start the augmented reality program. First, the camera unit 106 and various sensors (position information acquisition unit 101 and direction sensor 104) are activated, and live view display is started as an initial screen (ST101). Live view display is a process in which video data is generated by an image receiving element with a predetermined exposure time from a subject image acquired by the optical system of the activated camera unit 106 and displayed on the display unit 102 according to a conventional method. Is repeated at predetermined time intervals. Subsequently, terminal position information relating to the current terminal position of the terminal device 100 is acquired from the detection value of the position information acquisition unit 101 (ST102).

  Next, the terminal device 100 transmits the terminal location information acquired in ST102 to the augmented reality server 300 to request augmented reality information (ST103), and the augmented surroundings of the terminal device 100 that is responded by the augmented reality server 300 A list of reality information is received and recorded in the storage unit 107 (ST104).

  Subsequently, for the augmented reality information of the list received in ST104, the information described in the list based on the attitude and angle information of the camera unit 106 detected by the direction sensor 104 and the terminal position information acquired in ST102. Are individually determined as to whether or not the associated point enters the field of view of the camera unit 106, and is recorded in the list of augmented reality information stored in the storage unit 107 (ST105). Hereinafter, the list of the storage unit 107 is referred to, and the augmented reality information determined to enter the field of view of the camera unit 106 is processed.

  Next, assuming that the extracted augmented reality information is arranged in the coordinate space with the current terminal position of the terminal device 100 as the origin, the horizontal viewing angle and the vertical of the camera with the predetermined height of the origin as the center point The display position of each piece of information is calculated by performing a process of transparently projecting each piece of augmented reality information onto a rectangular plane determined at a predetermined distance from the origin based on the direction viewing angle and the video size (ST106).

  Subsequently, based on the calculated display position of each piece of information, it is determined whether there is augmented reality information in which the display position interval is equal to or less than a predetermined value (ST107), and the display position interval is equal to or less than the predetermined value. If there is augmented reality information, the augmented reality information is displayed as a collective icon (ST108), and the remaining augmented reality information is displayed as a normal icon (ST109). As described above, the operation of superimposing the augmented reality information on the video in the real space is finished and the operation waits for the user's operation. It is determined whether or not there is an end instruction (ST110). When it is determined that there is an end instruction, the series of operations is ended. If it is not determined that an end instruction has been issued, it is subsequently determined whether or not an operation for touching the collective icon has been performed (ST111). If it is determined that an operation for touching the collective icon has been performed, a spiral described later is performed. Proceed to menu operation. on the other hand. If it is not determined that an operation for touching the collective icon has been performed, it is determined whether an operation for touching a normal icon has been performed (ST112). If it is determined that an operation of touching a normal icon is performed, the operation proceeds to an operation of a disc menu described later. If it is not determined in ST112 that an operation of touching a normal icon is performed, the process returns to ST110 and the subsequent operations are repeated.

  Next, another operation from the start to the display of augmented reality information until the user waits for an operation will be described with reference to the flowchart of FIG.

  In this operation, first, similarly to the above operation, the camera unit 106 and various sensors are activated to start live view display (ST101 ′), and the position information acquisition unit 101 acquires terminal position information (ST102 ′). The location information is transmitted to the augmented reality server 300 to request augmented reality information (ST103 ′), and the list of augmented reality information responded by the augmented reality server 300 is recorded in the storage unit 107 (ST104 ′). Then, it is determined whether or not the augmented reality information in the list enters the field of view of the camera unit 106 (ST105 ′).

  Subsequently, for the augmented reality information determined to fall within the viewing angle of the camera unit 106 in the list recorded in the storage unit 107, the distance from the point related to the position information (point information) to the terminal device 100 is respectively set. Calculate and record in the list of the storage unit 107 (ST120). Next, the display position of each information is calculated in the same manner as the above operation (ST121).

  In this operation, when superimposing position information content in augmented reality information on a video in real space, a large icon (graphics object) is drawn when the point related to the position information (point information) is a short distance. In the case of distance, by drawing a small icon (graphics object), the augmented reality information is given a sense of perspective according to the actual distance, resulting in excellent augmented reality. Therefore, in this operation, when an icon having a size corresponding to the distance is displayed on the display unit 102, it is determined whether or not there is augmented reality information that overlaps another augmented reality information icon (ST122). If it is determined that there is augmented reality information that overlaps with other augmented reality information icons, the augmented reality information is displayed on the display unit 102 as a collective icon (ST123), and then the remaining information is displayed. Is displayed on the display unit 102 as a normal icon corresponding to the distance (ST124). On the other hand, if it is not determined in ST123 that there is augmented reality information that overlaps with another augmented reality information icon, each augmented reality information is displayed as a normal icon according to the distance (ST124).

  Thereafter, as in the above-described operation, the operation waits for the user's operation, and it is determined whether or not there is an end instruction (ST110 ′), whether or not an operation of touching the collective icon has been performed (ST111 ′), and a normal icon The determination of whether or not an operation for touching is performed (ST112 ′) is repeated.

  Next, in the operation waiting for the user's operation, in ST111 or ST111 ′, the spiral menu operation performed when it is determined that the user has performed an operation of touching the collective icon is shown in the flowchart of FIG. The description will be given with reference.

  In the spiral menu operation of the present embodiment, when it is determined in ST111 or ST111 ′ that the user has performed an operation of touching the collective icon, the live view image displayed on the display unit 102 is first displayed. When touched, it is locked and the brightness is reduced (ST201). Here, to lock the live view image is to fix the video in the real space displayed on the display unit 102 to the one at the time of touch. By doing in this way, even if the attitude angle of the terminal changes during the menu operation, it is possible to continue displaying the video of the point associated with the augmented reality information related to the collective icon. Further, by reducing the brightness of the real space video, it is possible to display the augmented reality information such as icons more emphasized. Subsequently, icons other than the touched collective icon are deleted from the screen (ST202), and the touched collective icon is moved to the center of the screen (ST203, FIG. 8A). The icon is rewritten to an image in which normal icons are superimposed, and a predetermined number of icons in the augmented reality information displayed as a collective icon are rotated around the center of the screen in order from the closest distance to the terminal device 100. They are moved and deployed in a ring shape (ST204, FIG. 8B). A spiral menu related to the collective icon touched as described above is displayed on the display unit 102.

  The subsequent processing is in accordance with user input to the spiral menu. First, it is determined whether or not an operation defined as an instruction to end the spiral menu, such as an operation of touching the center portion of the ring formed by the icon, has been performed (ST205). If it is determined that the operation defined as the spiral menu end instruction has been performed, the currently displayed menu is cleared (ST211), the lock of the live view image is released, and the brightness of the real space image is returned (ST212). The icons and the collective icons are displayed again (ST213), and the process returns to the user operation waiting state in the processing after ST110 in FIG. 14 or ST110 ′ in FIG.

  On the other hand, if it is determined that the operation defined as the spiral menu end instruction has not been performed, it is first determined whether or not a touch operation has been performed on each icon (ST206). If it is determined in ST206 that a touch operation has been performed on each icon, a disc menu operation described later is started. If it is not determined in ST206 that a touch operation has been performed, it is subsequently determined whether a predetermined drag operation has been performed (ST207). Here, the predetermined drag operation is an input operation to the spiral menu, and specifically, an operation of dragging to rotate along icons arranged in a ring shape forming the spiral menu. If it is determined that such a drag operation has been performed by the touch panel function of the display unit 102, it is subsequently determined whether or not the direction of the drag operation is the left direction (ST208). If it is determined that the direction of the drag operation is the left direction, the icons arranged in a ring shape are rotated to the left in the ring-shaped circumferential direction, and the displayed icons one by one in the long distance direction (ST209). If it is not determined in ST208 that the direction of the drag operation is the left direction, it is assumed that the icon is dragged to the right, and the icon arranged in the ring shape is rotated to the right in the ring-shaped circumferential direction. The displayed icons are shifted one by one in the short distance direction (ST210). After shifting the icon in ST209 or ST210, if it is not determined in ST207 that a predetermined drag operation has been performed, both return to ST205 and repeat the above operations.

  Next, when it is determined in ST112 in FIG. 14 or ST112 ′ in FIG. 15 that an operation for touching a normal icon has been performed, and it is determined that a touch operation for each icon has been performed in the spiral menu operation in ST206 in FIG. The disc menu operation performed in this case will be described with reference to the flowchart of FIG.

  In this disc menu operation, a menu that can be selected for augmented reality information relating to each icon is displayed as a disc menu distributed to a set of numbers that can be displayed at a time on the display unit 102, and the touch panel function of the display unit 102 is displayed. It provides an interface that can be easily operated with one hand.

  First, when starting the disc menu operation from ST112 in FIG. 14 or ST112 ′ in FIG. 15 without going through the spiral menu described above, first, the live view image is locked and the brightness is reduced as in ST201 described above. The untouched icon is erased (ST250), and the touched icon is moved to the center of the screen (ST251). When starting the disc menu operation from ST206 of the spiral menu operation, it starts from ST251.

  Subsequently, the icon set of the initial menu disk is expanded and arranged in a ring shape by the same operation as ST204 of the spiral menu operation (ST252). Thereafter, the process is in accordance with the user input to the disc menu. First, it is determined whether or not an operation defined as a predetermined disc menu end instruction has been performed (ST253), and when it is determined that an end instruction operation has been performed. The displayed menu is cleared (ST254), and it is determined whether the collective icon is selected (ST265). If it is determined that the collective icon is selected, the icon and the collective icon are redrawn and the process returns to ST203 of the spiral menu operation. If it is not determined that the collective icon is selected, ST110 of FIG. 14 or FIG. Return to ST110 '.

  Subsequently, it is determined whether or not a touch operation has been performed on the currently displayed disc menu icon (ST255). If it is determined that a touch operation has been performed, processing corresponding to the icon is executed (ST256). ). On the other hand, if it is not determined that a touch operation has been performed, it is subsequently determined whether a predetermined drag operation has been performed (ST257). Here, the predetermined drag operation is an input operation to the disc menu, and is an operation of dragging so as to rotate along icons arranged in a ring shape forming the disc menu. If it is determined that such a drag operation has been performed, it is subsequently determined whether or not the direction of the drag operation is the left direction (ST258). If it is determined that the direction of the drag operation is the left direction, it is subsequently determined whether or not the drag operation is at a low speed (ST259). When it is determined that the drag operation is slow, the set of icons displayed as the disc menu is rotated to the left one by one in the ring-shaped circumferential direction (ST260), and the drag operation is not determined to be slow. In this case, the icon being displayed is deleted from the screen and the icon set of the next menu disk is displayed (ST261), and the operations after ST253 are repeated. If the drag operation is not determined to be in the left direction in ST258, it is subsequently determined whether or not the drag operation has been performed at a low speed (ST262). If it is determined that the drag operation is slow, the set of icons displayed as the disc menu is rotated to the right one by one in the ring-shaped circumferential direction (ST263). If it is not determined that the drag operation is slow, the currently displayed icon is deleted from the screen to display the icon set of the previous menu disc (ST264), and the operations after ST253 are repeated.

  FIG. 18 is an indicator of a disc variation displayed in the center of the icons arranged in a ring shape in the disc menu operation shown in FIG. As shown in FIG. 18, the disc menu is roughly divided into a set of menu icons for operations related to an initial menu, communication, navigation, and related blogs as an example. In the disc menu operation, the highlighted portion (shaded portion in the figure) of the indicator is changed according to the type of the disc menu being displayed. For this reason, an interface with excellent visibility is realized.

  Through the processing as described above, the terminal device 100 according to the present embodiment corresponds to the augmented reality technology that displays information in a real-space image, and the display position that is superimposed is close even if the amount of information to be displayed increases. Since the augmented reality information is displayed as a collective graphics object, it is possible to suppress complication of the screen. In addition, the spiral menu and disc menu provide an intuitive interface that is intuitive and excellent in visibility through physical operations using the touch panel, providing an augmented reality with an excellent user experience. Can do. Furthermore, since information can be superimposed with a visual or informational perspective according to the distance between the object on which information is superimposed and the terminal device 100, augmented reality is realized by the information superimposed on the flat plate. Can be prevented from being damaged.

  Note that the terminal device, the information presentation system, and the terminal screen display method of the present invention are not limited to the above-described embodiment, and various modifications can be made. For example, although the case where the graphics object is a round icon has been described in the above embodiment, the present invention is not limited to this, and the icon may be a square icon or an information balloon. .

  In the above-described embodiment, an example in which the graphics object is developed in a ring shape has been described.

  In the above-described embodiment, the example in which the control unit 108 of the terminal device 100 performs the process of determining the display position of the augmented reality information and the like has been described. However, the server controller 310 of the augmented reality server 300 performs the augmented reality information. It is also possible to perform a process for determining the display position and the like and return only the result to the terminal device 100.

  In the above-described embodiment, the case where the user operation for dragging the graphic object displayed in the ring shape is detected and it is determined whether or not it is the left direction (ST208) has been described as an example, but ST259 in FIG. As shown in ST262, a process of replacing all display icons according to the drag speed may be executed.

  In the above embodiment, when calling a server function, the function is constructed by CGI and the call is an API, but the method is configured by any communication means between applications such as other protocols and distributed objects. Also good.

  In the above-described embodiment, an example in which the display unit 102 includes a touch panel function has been described. However, the present invention is not limited to this, and the terminal device 100 may have a function of detecting a user operation. For example, instead of the touch panel function, a screen operation by a hardware button such as a pointing device, a cross key, and a numeric keypad may be detected.

  DESCRIPTION OF SYMBOLS 1 ... Augmented reality system (information presentation system), 100 ... Terminal device, 101 ... Position information acquisition part, 102 ... Display part, 103 ... Key input part, 104 ... Direction sensor, 105 ... Communication part, 106 ... Camera part, 107: Storage unit, 108: Control unit.

Claims (13)

A camera unit that acquires a real space image, a display unit that displays a screen, a storage unit that stores augmented reality information associated with position information, and controls the operation of each unit and realizes an information processing function A control unit, and the control unit displays the video of the real space acquired by the camera unit on the display unit, determines a display position of the graphics object corresponding to the augmented reality information, A terminal device that performs processing of superimposing the graphics object on the video in real space at the display position and rendering the display unit;
The control unit determines the display position based on the position information associated with the augmented reality information, and displays the plurality of augmented reality information in which the position information is close to each other as one collective graphic. The position information is closer to each other among the plurality of augmented reality information as the augmented reality information associated with a point far from the current point is displayed on the display unit as a scan object. Expanding the range of judgment
A terminal device characterized by the above. The control unit performs processing for causing the display unit to draw the graphics object corresponding to the type of the augmented reality information,
The terminal device according to claim 1, wherein the collective graphics object has a shape in which a plurality of the graphics objects arranged under a predetermined condition overlap each other.   When the control unit detects a user operation to select the collective graphics object, the control unit displays the graphics objects of the plurality of augmented reality information displayed on the display unit as the collective graphics object. The terminal device according to claim 1, wherein processing is performed such that the display units are arranged in a ring shape and displayed on the display unit.   When the number of the augmented reality information displayed on the display unit as one set graphics object is larger than the number of the graphics objects that can be displayed in a circle and displayed at a time, When the display unit displays the graphics objects arranged in a ring shape as many as the number that can be displayed on the display unit, and detects a user operation that clearly indicates the moving direction of the graphics objects arranged in a ring shape, 4. The terminal device according to claim 3, wherein each of the arranged graphics objects is rotated and moved along a circumferential direction, and a process of replacing the displayed graphics objects is performed. 5.   The control unit changes the size of the graphics object based on a predetermined condition when performing processing for arranging the graphics objects of a plurality of the augmented reality information in a circle and displaying them on the display unit. The terminal device according to claim 3, wherein the terminal device is displayed on the display unit.   The control unit changes the transparency of the graphics object based on a predetermined condition when performing processing for arranging the graphics objects of the plurality of augmented reality information in a circular pattern and displaying the graphics objects on the display unit. The terminal device according to claim 3, wherein the terminal device is displayed on the display unit.   The said control part displays the said graphics object of the said augmented reality information which satisfy | fills predetermined conditions on the said display part so that it may be located in the head of the said collective graphics object. The terminal device according to any one of the above.   The control unit detects a user's operation of dragging along the graphics object as a user's operation that clearly indicates a moving direction of the graphics objects arranged in a ring, and the user's operation of dragging is detected. When the speed is lower than the threshold, the displayed graphics objects are replaced one by one, and when the speed is higher than the threshold, the displayed graphics objects are all replaced. The terminal device according to any one of claims 3 to 7. When the control unit detects an operation of a user who selects the graphics object, the control unit displays the graphics object that displays an executable process for the augmented reality information related to the graphics object in a ring shape. As shown on the display section,
The processing related to a graphics object is performed when a user operation for selecting one of the graphics objects displayed in a circle is detected. The terminal device according to any one of 8.   The said control part changes the magnitude | size of the said graphics object based on predetermined conditions about the said augmented reality information, and displays it on the said display part, The any one of Claims 1-9 characterized by the above-mentioned. The terminal device described in 1.   The said control part changes the information content of the said graphics object based on predetermined conditions about the said augmented reality information, and displays it on the said display part, The any one of Claims 1-10 characterized by the above-mentioned. The terminal device described in 1. A terminal device for displaying a screen by a display unit;
A server configured to be communicable with the terminal device and having augmented reality information associated with position information;
An information presentation system comprising:
The server displays an image of real space on the display unit, determines a display position of a graphics object corresponding to the augmented reality information based on position information of the terminal device, and displays the image in the real space. A process of superimposing the graphics object at the display position and drawing the display part on the display unit, and displaying the plurality of augmented reality information close to the display position on the display unit as one collective graphics object. A server control unit to perform,
The server control unit determines the display position based on the position information associated with the augmented reality information, and collects a plurality of the augmented reality information in which the position information is close to each other. It performs processing for displaying on the display unit as a graphics object, the position information about the Augmented reality information distance is associated with a point far from the current point between the plurality of augmented reality information of the terminal device Expanding the range that is determined to be close to each other ;
An information presentation system characterized by A terminal screen display method in which a graphics object corresponding to augmented reality information associated with position information is superimposed on a real space image acquired by a camera unit and rendered on a display unit of the terminal device,
A display position determining step for determining a display position of a graphics object corresponding to the augmented reality information;
A proximity determination step of determining whether or not the display positions of the plurality of graphics objects are close to each other ;
When the display positions are close to each other, a display step of displaying a plurality of augmented reality information on the display unit as one collective graphics object;
With
In the proximity determination step, the display position is determined based on the position information associated with the augmented reality information, and the augmented reality information associated with a point far from the current point of the terminal device Enlarging the range for determining that the position information is close to each other among the plurality of augmented reality information ,
A terminal screen display method characterized by the above.