JP2012155362A - Information processing apparatus, augmented reality system, information processing method, and information processing program - Google Patents

Abstract

An information processing apparatus capable of guaranteeing the relationship between a physical object and an annotation is provided.
An image acquisition unit 302, identification information detection units 303, 305, and 306 for detecting physical object identification information, physical object identification information, physical object annotations, physical objects and physical annotations Whether or not the communication unit 307 that acquires the correspondence level information indicating the correspondence level, the physical object identification information detected by the identification information detection unit, and the physical object identification information acquired by the communication unit 307 substantially match. The usage policy determination unit 310 that determines whether or not the correspondence information of the annotation of the physical object satisfies a predetermined usage policy, and the usage policy When the determination unit 310 determines that the usage policy is satisfied, an annotation is presented It includes a radical 113 312, a.
[Selection] Figure 5

Description

  The present invention relates to an information processing apparatus, an augmented reality system, an information processing method, and an information processing program. In particular, the present invention relates to a technology used in an augmented reality (AR) system that provides information (annotations) on an object (physical object) in a real environment to a user.

  In recent years, “augmented reality” (also referred to as augmented reality) is becoming known as a technique for additionally presenting information to a real environment using a computer and a term indicating the environment in which information is additionally presented. In the augmented reality technology, electronic information (annotation) including explanation and related information regarding a specific object in a real environment is presented in association with an actual object (physical object) to be explained.

  The following technologies are known as technologies related to augmented reality. For example, the similarity between the visual feature information extracted from the partial video of the captured video and the visual feature information of the physical object stored in the storage device is calculated, and the physical object is determined based on the calculated similarity. It is determined whether or not it exists in the captured video. An apparatus that does not display an annotation when a physical object does not exist in a captured video is known (for example, see Patent Document 1).

  There is also known an apparatus that changes the display of an annotation in accordance with the distance between a physical object projected on a real world video and the viewpoint of the real world video and the acquired distance (see, for example, Patent Document 2).

  There is also known an apparatus that provides a user with information in which context information of a user's current, past, and future is reflected regardless of a person's subjectivity or production intention in association with a place (patent) Reference 3). In this apparatus, information on the position is searched based on the history of the user.

Japanese Patent No. 4488233 International Publication No. 2010/073616 Pamphlet JP 2001-306604 A

  In the prior art, annotations are displayed on physical objects based on the above feature information, distance information, and position information. However, for the user, how to recognize physical objects and annotations. It is unknown. Therefore, the user cannot grasp the relationship between the physical object and the annotation. In the future, augmented reality systems are expected to become widespread, but it is thought that augmented reality systems cannot be used with peace of mind unless the relationship between physical objects and annotations can be sufficiently guaranteed.

  The present invention has been made in view of the above circumstances, and provides an information processing device, an augmented reality system, an information processing method, and an information processing program capable of guaranteeing the relationship between a physical object and an annotation The purpose is to do.

  An information processing apparatus of the present invention identifies an image acquisition unit that acquires an image, an identification information detection unit that detects physical object identification information for identifying a physical object included in the image, and a predetermined physical object Detected by the identification information detector, an information acquisition unit that acquires physical object identification information, an annotation of the physical object, and correspondence information indicating a physical correspondence between the physical object and the annotation. The identification information determination unit for determining whether or not the physical object identification information substantially matches the physical object identification information acquired by the information acquisition unit, and the identification information determination unit Whether the correspondence information of the annotation of the physical object satisfies a predetermined usage policy And determining usage policy determining section whether, if the it is determined to satisfy the usage policy by utilizing policy judgment section, and a annotation presentation unit for presenting the annotation.

  With this configuration, it is possible to guarantee the relationship between the physical object and the annotation. That is, it is possible to present to the user so that a highly reliable annotation among the annotations corresponding to the physical object can be easily identified. Therefore, the user can use the presented annotation with peace of mind.

  Further, in the information processing apparatus of the present invention, the information acquisition unit acquires relevance information indicating a semantic relevance between the physical object and the annotation, and the usage policy determination unit includes the identification information determination unit. If it is determined that the values substantially match, it is determined whether the correspondence level information and the relevance level information of the annotation of the physical object satisfy a predetermined usage policy.

  With this configuration, it is possible to guarantee the relevance between the physical object and the annotation by the correspondence level and the relevance level. The reliability is further improved as compared with the case where the reliability is judged only by the degree of correspondence.

  Also, the information processing apparatus of the present invention displays that the usage policy determination unit satisfies the usage policy when the usage policy determination unit determines that the usage policy is satisfied.

  With this configuration, for example, by presenting that the usage policy is satisfied with characters, icon colors, etc., the user can surely recognize that the annotation satisfies the usage policy.

  In the information processing device of the present invention, the identification information detection unit detects a position information related to the position of the information processing device, and includes a position information detection unit from the information processing device to a physical object included in the image. A distance information detection unit that detects distance information related to a distance, and the information acquisition unit is based on position information related to a position of the information processing device and distance information related to a distance to the physical object, Position information relating to the position of a predetermined physical object is acquired as the physical object identification information, and the identification information determination unit detects the position information acquired by the information acquisition unit and the position information detected by the position information detection unit. And the distance information acquired by the information acquisition unit and the distance information detected by the distance information detection unit are substantially matched.

  With this configuration, it is possible to confirm a match between the physical object related to the physical object identification information detected by the information processing apparatus owned by the poster and the physical object currently displayed by the viewer. That is, by identifying a physical object based on position information and distance information, only highly reliable annotations can be presented to the user.

  Further, in the information processing apparatus according to the present invention, the identification information detection unit includes a feature information extraction unit that extracts feature information related to a feature of the physical object included in the image, and the information acquisition unit includes the physical information Feature information related to the feature of the physical object is acquired as the object identification information, and the identification information determination unit is substantially the feature information acquired by the information acquisition unit and the feature information extracted by the feature information extraction unit. It is determined whether or not they match.

  With this configuration, by identifying the physical object based on the feature information together with the position information and the distance information, only highly reliable annotation can be presented to the user.

  In the information processing device of the present invention, the identification information detection unit detects a position information related to the position of the information processing device, and includes a position information detection unit from the information processing device to a physical object included in the image. A distance information detection unit that detects distance information related to a distance; and a feature information extraction unit that extracts feature information related to a feature of the physical object included in the image, wherein the information acquisition unit includes the physical object identification As the information, position information related to the position of a predetermined physical object, distance information related to the distance to the physical object, and feature information related to the characteristic of the physical object are acquired, and the identification information determination unit The position information acquired by the information acquisition unit and the position information detected by the position information detection unit substantially match and acquired by the information acquisition unit. Whether the distance information detected by the distance information detection unit substantially matches the feature information acquired by the information acquisition unit and the feature information acquired by the feature information extraction unit. Determine whether or not.

  With this configuration, by identifying a physical object based on position information, distance information, and feature information, only highly reliable annotations can be presented to the user.

  In the information processing apparatus of the present invention, the annotation presentation unit displays the annotation in association with a physical object included in the image acquired by the image acquisition unit.

  With this configuration, it is possible to easily understand to which physical object the annotation is added.

  The information processing apparatus according to the present invention is an information processing apparatus in which the annotation presenting unit presents the annotations in descending order of the degree of correspondence or the degree of association.

  With this configuration, when a plurality of annotations are posted on one physical object, highly reliable annotations can be preferentially presented.

  In the information processing apparatus of the present invention, the predetermined usage policy is that each of the correspondence level and the relevance level or a total value is equal to or greater than a predetermined value.

  With this configuration, an annotation can be presented only when the degree of correspondence and the degree of association are greater than or equal to a predetermined value.

  The information processing apparatus according to the present invention is an information processing apparatus in which the annotation presenting unit presents at least one of the correspondence level and the relevance level simultaneously with the annotation.

  With this configuration, the user can recognize the degree of correspondence or the degree of association.

  The information processing apparatus according to the present invention includes an image acquisition unit that acquires an image, an identification information detection unit that detects physical object identification information for identifying a physical object included in the image, and an annotation of the physical object. An annotation generation unit to be generated, and a transmission unit that transmits, for each physical object, the annotation generated by the annotation generation unit and the physical object identification information detected by the identification information detection unit.

  With this configuration, the degree of correspondence and the degree of association can be generated based on the physical object identification information, and the generated annotation can be determined to be a highly reliable annotation.

  In the information processing device of the present invention, the identification information detection unit detects a position information related to the position of the information processing device, and includes a position information detection unit from the information processing device to a physical object included in the image. A distance information detection unit that detects distance information related to a distance; and a feature information extraction unit that extracts feature information related to a feature of the physical object included in the image, wherein the sending unit includes the physical object identification information The position information, the distance information, and the feature information are transmitted.

  With this configuration, it is possible to determine that the annotation is highly reliable by identifying the physical object based on the position information, the distance information, and the feature information.

  Further, the information processing apparatus according to the present invention includes a physical object identification information for identifying a physical object, an information acquisition unit that acquires the annotation of the physical object, and the physical object based on the physical object identification information. A correspondence calculation unit that calculates a physical correspondence with the annotation of the physical object.

  With this configuration, by calculating the degree of correspondence between the physical object and the annotation, the reliability of the annotation can be provided, for example, as a numerical value.

  The information processing apparatus according to the present invention further includes a relevance calculation unit that calculates a semantic relevance between the physical object and the annotation based on the identification information of the physical object and the annotation.

  With this configuration, by further calculating the degree of association between the physical object and the annotation, the reliability of the annotation can be provided as a numerical value, for example.

  In addition, the information processing apparatus of the present invention sends out correspondence information related to the correspondence calculated by the correspondence calculation unit and relevance information related to the correlation calculated by the association calculation unit. An information processing apparatus comprising a unit.

  With this configuration, since detailed annotation information can be provided to the viewer, the viewer can check the reliability of the annotation.

  In the information processing apparatus of the present invention, the physical object identification information includes position information related to a position of the information processing apparatus, distance information related to a distance to the physical object, and feature information related to a characteristic of the physical object. The correspondence degree calculating unit calculates an identification level based on the feature information and the distance information, calculates a positioning level based on the position information, and based on the identification level and the positioning level The correspondence degree is calculated.

  With this configuration, it is possible to calculate the degree of correspondence with high accuracy from the identification level and the positioning level.

  The information processing apparatus according to the present invention further includes a physical object information storage unit that associates and stores the physical object identification information and physical object information including phrase information related to the physical object identified by the physical object identification information. A text information acquisition unit that acquires text information related to the annotation from the annotation acquired by the information acquisition unit, and the relevance calculation unit calculates the text information from the annotation acquired by the information acquisition unit. Among the phrase information included in the text information acquired by the acquisition unit, included in the physical object information associated with the physical object identification information acquired by the information acquisition unit and stored in the physical object information storage unit Based on the phrase information, the degree of association Calculated to.

  With this configuration, it is possible to calculate a relevance level with high accuracy from text information related to the annotation.

  Further, in the information processing apparatus of the present invention, the phrase information includes information on the name of the physical object, information on a related word that is a phrase related to the physical object, and a phrase that is not related to the physical object. And information on certain unrelated words.

  With this configuration, it is possible to calculate a relevance level with high accuracy from the names, related words, and unrelated words of the physical object related to the annotation.

  An augmented reality system of the present invention includes an augmented reality system comprising: an information providing device that provides an annotation corresponding to a physical object; an information management server that manages the annotation; and an information presentation device that presents the annotation. In the system, the information providing device acquires an image, an identification information detection unit that detects physical object identification information for identifying a physical object included in the image, and an annotation of the physical object An annotation generation unit that generates a transmission unit that transmits, for each physical object, an annotation generated by the annotation generation unit, and physical object identification information detected by the identification information detection unit, The information management server receives information from the information providing device. An information acquisition unit that acquires the physical object identification information and the annotation; a correspondence calculation unit that calculates a physical correspondence between the physical object and the annotation of the physical object based on the physical object identification information; A transmission unit that transmits the physical object identification information, the annotation, and the correspondence information, and the information presentation device acquires an image, and a physical object included in the image An identification information detector for detecting physical object identification information for identifying the physical object identification information for identifying a predetermined physical object from the information management server, an annotation for the physical object, and the correspondence level Information acquisition unit for acquiring information and the identification information detection unit It is determined by the identification information determination unit that the physical object identification information thus issued and the physical object identification information acquired by the information acquisition unit substantially match, and by the identification information determination unit. When the correspondence information of the annotation of the physical object is determined to satisfy the usage policy by the usage policy determination unit that determines whether or not a predetermined usage policy is satisfied, and the usage policy determination unit And an annotation presenting unit for presenting the annotation.

  With this configuration, the degree of correspondence can be generated based on the physical object identification information, and the generated annotation can be determined to be a highly reliable annotation. Further, by calculating the correspondence between the physical object and the annotation, the reliability of the annotation can be provided, for example, in the form of a numerical value. Furthermore, it is possible to guarantee the relationship between the physical object and the annotation. That is, it is possible to present to the user so that a highly reliable annotation among the annotations corresponding to the physical object can be easily identified. Therefore, the user can use the presented annotation with peace of mind.

  The first information processing method of the present invention includes a step of acquiring an image, a step of detecting physical object identification information for identifying a physical object included in the image, and a predetermined physical object. Physical object identification information, an annotation of the physical object, and correspondence information indicating physical correspondence between the physical object and the annotation; and the detected physical object identification information; The step of determining whether or not the acquired physical object identification information substantially matches, and if it is determined that the physical object identification information substantially matches, the correspondence information of the annotation of the physical object indicates a predetermined usage policy. A step of determining whether or not to satisfy, and it is determined that the usage policy is satisfied If, and a step of presenting the annotations.

  By this method, it is possible to guarantee the relationship between the physical object and the annotation. That is, it is possible to present to the user so that a highly reliable annotation among the annotations corresponding to the physical object can be easily identified. Therefore, the user can use the presented annotation with peace of mind.

  According to the second information processing method of the present invention, the step of acquiring an image, the step of detecting physical object identification information for identifying the physical object included in the image, and the annotation of the physical object are generated. And sending the generated annotation and the detected physical object identification information for each physical object.

  With this method, it is possible to generate the degree of correspondence and the degree of association based on the physical object identification information, and it is possible to determine that the generated annotation is a highly reliable annotation.

  According to a third information processing method of the present invention, the physical object identifying information for identifying the physical object and the annotation of the physical object are acquired, and the physical object is based on the physical object identifying information. And calculating a physical correspondence between the annotation of the physical object and the annotation of the physical object.

  By calculating the degree of correspondence between the physical object and the annotation by this method, the reliability of the annotation can be provided, for example, as a numerical value.

  The information processing program of the present invention is a program for causing a computer to execute each step of the first information processing method.

  With this program, it is possible to guarantee the relationship between the physical object and the annotation. That is, it is possible to present to the user so that a highly reliable annotation among the annotations corresponding to the physical object can be easily identified. Therefore, the user can use the presented annotation with peace of mind.

  The information processing program of the present invention is a program for causing a computer to execute each step of the second information processing method.

  With this program, the degree of correspondence and the degree of association can be generated based on the physical object identification information, and the generated annotation can be determined to be a highly reliable annotation.

  The information processing program of the present invention is a program for causing a computer to execute each step of the third information processing method.

  By calculating the degree of correspondence between the physical object and the annotation by this program, the reliability of the annotation can be provided, for example, in the form of a numerical value.

  According to the present invention, it is possible to guarantee the relationship between a physical object and an annotation.

The image figure for demonstrating the correspondence and relevance in embodiment of this invention The block diagram which shows the structural example of the augmented reality system in embodiment of this invention The block diagram which shows the structural example of the information provision apparatus in embodiment of this invention The block diagram which shows the structural example of the information management server in embodiment of this invention The block diagram which shows the structural example of the information display apparatus in embodiment of this invention The figure which shows an example of the physical object identification information in embodiment of this invention The flowchart which shows the operation example of the information provision apparatus in embodiment of this invention. The flowchart which shows the operation example of the information provision apparatus in embodiment of this invention. The flowchart which shows the operation example of the information provision apparatus in embodiment of this invention (continuation of FIG. 8). The flowchart which shows the operation example of the information provision apparatus in embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  In this embodiment, in order to recognize the relationship between a physical object and an annotation, the concepts of “correspondence” and “relevance” are introduced. The correspondence level indicates a physical relationship (physical correspondence level) between the physical object and the annotation. In other words, the correspondence degree indicates the degree to which the annotation is accurately added to the physical object. The association degree indicates a semantic relationship (semantic association degree) between the physical object and the annotation. In other words, the degree of association indicates how much the content of the annotation is related to the physical object. The degree of correspondence and the degree of association are mainly used in numerical form.

  In addition, as a physical object in this embodiment, a feature, a moving body, a place, etc. are assumed. The feature is a building fixed at a specific position such as a building or a monument. A moving body is a movable object such as a person or a car. A place is a district uniquely identified by a latitude and longitude and uniquely identified by a specific name such as a predetermined point, a park, or a street. As annotations, text information such as character information, image information such as 2D / 3D graphics, audio information, and the like are assumed.

  FIG. 1 is an image diagram for explaining correspondence and relevance. In the example shown in FIG. 1, the maximum value of correspondence and relevance is 5 and the minimum value is 0. In the example shown in FIG. 1, an image of a five-storied pagoda as the physical object O is displayed. To this physical object, character information “Five-storied Pagoda” as the first annotation A1 and character information “Taiko Shotoku” as the second annotation A2 are added.

  In the example shown in FIG. 1, the degree of correspondence between the physical object O and the first annotation A1 is 5, and the degree of association is 4. Therefore, it can be recognized that the first annotation A1 is accurately added to the physical object O based on the degree of correspondence. In other words, the character information “Five-storied Pagoda” as the first annotation A1 is originally intended to be added to another physical object, but it is not accidentally added to the “Five-storied Pagoda” as the physical object O. Can understand and improve annotation reliability. Further, it can be recognized from the degree of relevance that “the five-storied pagoda” as the physical object O and the character information “the five-storied pagoda” as the first annotation A1 are substantially closely related. That is, it can be understood that the word “quintuple tower” is almost mistaken as content related to the physical object O, and the reliability of the annotation is improved.

  In the example shown in FIG. 1, the degree of correspondence between the physical object O and the second annotation A2 is 4, and the degree of association is 2. Therefore, it can be recognized that the second annotation A2 is added to the physical object O almost accurately based on the degree of correspondence. In other words, the character information “acquired prince” as the second annotation A2 is originally intended to be added to another physical object, but may be erroneously added to the “5-storied pagoda” as the physical object O. It can be understood that the sex is quite low. Further, it can be recognized from the degree of relevance that “the five-storied pagoda” as the physical object O and the character information “Saintoku Prince” as the second annotation A2 are not significantly related. That is, it can be understood that the word “Saintoku Prince” may be mistaken as the content related to the physical object O.

  Next, the configuration of the augmented reality system of the present invention will be described.

  FIG. 2 is a block diagram showing a configuration example of the augmented reality system in the embodiment of the present invention. The augmented reality system 1 shown in FIG. 2 includes an information providing apparatus 100, an information management server 200, and an information display apparatus 300. The information providing apparatus 100, the information management server 200, and the information display apparatus 300 are all examples of information processing apparatuses.

  The information providing apparatus 100 is one or a plurality of apparatuses that generate an annotation corresponding to a physical object, and is an apparatus owned by a contributor (Submitter) who posts an annotation. As the information providing apparatus 100, for example, mobile devices such as a mobile phone, a portable information terminal, and a laptop computer, and a photographing apparatus such as a digital camera, a digital video camera, and an in-vehicle camera can be considered. The information providing apparatus 100 captures a physical object, generates an annotation for the captured physical object, and transmits the generated annotation to the information management server 200.

  The information management server 200 is one or a plurality of servers that acquire and manage annotations from the information providing apparatus 100, and is a server owned by a service provider that provides an augmented reality system service. . The information management server 200 generates the degree of correspondence and the degree of association based on the physical object identification information for identifying the physical object and the annotation posted to the physical object. Then, together with the annotation of the physical object, the correspondence level information including the generated correspondence level and the relevance level information including the relevance level are transmitted to the information display device 300.

  The information display device 300 is one or a plurality of devices that acquire annotations, correspondence information, and relevance information from the information management server 200 and present annotations based on the correspondence information and relevance information. It is a device owned by a viewer (viewer) who browses (uses). As the information display device 300, as with the information providing device 100, for example, mobile devices such as a mobile phone, a portable information terminal, and a laptop computer, and a photographing device such as a digital camera, a digital video camera, and an in-vehicle camera can be considered.

First, the configuration of the information providing apparatus 100 will be described.
FIG. 3 is a block diagram illustrating a configuration example of the information providing apparatus 100 according to the embodiment of the present invention.

  The information providing apparatus 100 includes a camera unit 101, an image acquisition unit 102, a feature extraction unit 103, a region specification unit 104, a distance measurement unit 105, a positioning unit 106, an information setting unit 107, an operation unit 108, a content generation unit 109, and a transmission unit. 110.

  The camera unit 101 is a photographing device that captures 2D or 3D still images or moving images, and is, for example, a CCD image sensor or a CMOS image sensor.

  The image acquisition unit 102 acquires an image (video through image) from the camera unit 101.

  The feature extraction unit 103 performs feature extraction processing for extracting the features of the image acquired by the image acquisition unit 102. For this feature extraction process, a known feature extraction method such as feature point extraction or edge extraction is used. For example, when the five-storied tower shown in FIG. 1 is included in the image, the outline (feature pattern) of the five-storied tower and parameters (feature parameters) that faithfully represent the five-storied tower are extracted.

  The feature extraction unit 103 may perform recognition processing for recognizing a physical object from the extracted features. For this recognition processing, a known image recognition method such as pattern matching is used. For example, it is recognized as a physical object called a five-storied pagoda from the extracted feature pattern and feature parameter set of five-storied pagoda.

  The area designating unit 104 designates an area of a physical object to which an annotation is to be added from among the physical objects whose features have been extracted or recognized by the feature extracting unit 103. At this time, for example, by touching the physical object on the screen via the touch panel as the operation unit 108, the area of the physical object is designated. At this time, a plurality of physical objects may be selected. If no special designation is made by the area designation unit 104, all physical objects that have been feature-extracted or object-recognized may be designated.

  The distance measuring unit 105 measures the distance from the camera unit 101 to the physical object specified by the area specifying unit 104. As a distance measuring method, for example, distance measurement using an optical or radar distance measuring sensor can be considered.

  The positioning unit 106 measures the position of the information providing apparatus 100. As a position measurement method, for example, GPS information (latitude, longitude, altitude, etc.) is acquired using a GPS (Global Positioning System) satellite, and position information is acquired using a WLAN (wireless LAN). It is done.

  The information setting unit 107 sets physical object identification information for identifying a physical object as additional information of an image captured by the camera unit 101. For example, object identification information is added as image metadata. For example, when the image is a JPEG image, object identification information is added to Exif (Exchangeble Image File Format) information as metadata. The information setting unit 107 adds object identification information for each physical object.

  Here, the physical object identification information includes the feature information extracted by the feature extraction unit 103 or the feature information of the physical object specified by the region specifying unit 104, and the distance related to the distance measured by the distance measuring unit 105. Information, and position information related to the position measured by the positioning unit 106 is included.

  Further, the feature information includes feature extraction method information, object feature information, and feature extraction accuracy information. The feature extraction method information is information indicating a feature extraction method (face authentication, image authentication, etc.) by the feature extraction unit 103. The object feature information is information on feature points of the image, information on markers included in the physical object, RFID (Radio Frequency Identification), QR code (Quick Response code), car number, nameplate, name tag, etc. It is information which shows. Further, the type (attribute) of a physical object such as text (character) information, image information, audio information or the like can be identified by the object feature information. Further, the type (attribute) of a physical object such as a car, a person, a feature, or a place can be identified by the object feature information. The feature extraction accuracy information is information indicating a state that can influence feature extraction, such as radio wave intensity and brightness. The feature extraction accuracy information may be specific numerical information indicating accuracy.

  The distance information includes distance measurement method information, object distance information, and distance measurement accuracy information. The distance measurement method information is information indicating a method of distance measurement by the distance measurement unit 105 (optical distance measurement method, radar distance measurement method, etc.). The object distance information is information indicating the distance from the information providing apparatus 100 to the physical object specified by the area specifying unit 104. The distance measurement accuracy information is information indicating a state that can affect distance measurement, such as radio wave intensity and brightness. The distance measurement accuracy information may be specific numerical information indicating accuracy.

  The position information includes positioning method information, information providing apparatus position information, and positioning accuracy information. The positioning method information is information indicating a positioning method (GPS method, WLAN method, etc.) by the positioning unit 106. The object position information is information indicating the position (latitude, longitude, etc.) of the information providing apparatus 100. The positioning accuracy information is information indicating a state that can affect distance measurement, such as radio wave intensity and brightness. The positioning accuracy information may be specific numerical information indicating accuracy. The object position information may include object direction information indicating the direction in which the camera unit 101 of the information providing apparatus 100 is facing. The direction related to the object direction information is measured by a geomagnetic sensor included in the positioning unit 106, for example.

  A more detailed description of the physical object identification information will be described later.

  The operation unit 108 receives an input operation by a user, and is, for example, a numeric keypad, a cross key, a touch panel, or the like. For example, the operation unit 108 selects a physical object to be annotated from among a plurality of physical objects.

  The content generation unit 109 generates content information as an annotation for a physical object to which an annotation is to be added. The content information includes text information, image information such as still images and moving images, audio information such as various sounds, sounds, and music. The content information may be generated via the operation unit 108 or the like, may be downloaded from the content server, or may be stored in advance in a memory (not shown) provided in the information providing apparatus 100.

  The transmission unit 110 transmits physical object identification information and content information to the information management server 200. The physical object identification information may be transmitted as image metadata as described above. The transmission unit 110 may transmit information providing device identification information for identifying the information providing device 100 to the information management server 200. The information providing device identification information includes an information providing device identification ID and information indicating the name of the information providing device 100. Instead of sending to the information management server 200 by communication, it may be sent to the information management server 200 via a storage medium (CD-R, SD card, etc.), for example.

Next, the configuration of the information management server 200 will be described.
FIG. 4 is a block diagram showing a configuration example of the information management server 200 in the embodiment of the present invention.

  The information management server 200 includes a reception unit 201, an identification level calculation unit 202, a positioning level calculation unit 203, a correspondence level calculation unit 204, a physical object information acquisition unit 205, a text conversion unit 206, a word number calculation unit 207, and a relevance level calculation unit 208. A signature generation unit 209, a transmission unit 210, a physical object database (physical object DB) 211, and a storage unit 212.

  The reception unit 201 receives physical object identification information, content information, and the like from the transmission unit 110 of the information providing apparatus 100. Instead of acquiring the information from the information providing apparatus 100 by communication, the information may be acquired from the information providing apparatus 100 via a storage medium (CD-R, SD card, etc.), for example.

  The identification level calculation unit 202 has an identification level calculation function, and calculates the identification level used when the correspondence level calculation unit 204 calculates the correspondence level. The identification level calculation function receives a feature extraction method information, feature extraction accuracy information, distance measurement method information, and distance measurement accuracy information of physical object identification information, and outputs a function for outputting an identification level corresponding to these information. Have. This function is expressed by, for example, (Equation 1) below.

  Identification level = feature extraction method level × extraction accuracy factor + ranging method level × ranging accuracy factor (Equation 1)

  Here, the feature extraction method level is a feature extraction method related to feature extraction method information (face authentication, image authentication, etc.) classified according to the performance of the feature extraction method and assigned, for example, five levels. It is. For example, level 4 is assigned when the feature extraction method is image recognition, level 5 is assigned when barcode recognition is performed, and the like. The extraction accuracy coefficient is classified according to the extraction accuracy (radio wave status, brightness status, etc.) according to the extraction accuracy information according to the status (strong radio wave state, weak radio wave state, daytime, nighttime, sunny, cloudy, rain, etc.) For example, a numerical value of 0% to 100% is assigned.

  In addition, the distance measurement method level is classified into five distance measurement methods (optical, radar, autofocus, etc.) according to the performance of the distance measurement method. Are assigned levels. For example, level 3 is assigned when the ranging method is AF, level 5 is assigned when the millimeter wave radar is used, and the like. The distance measurement accuracy factor is the distance measurement accuracy (radio wave condition, brightness condition, etc.) related to the distance measurement accuracy self-report, and the status (strong radio wave condition, weak radio wave condition, daytime, nighttime, sunny, cloudy, rain, etc.) And a numerical value of, for example, 0% to 100% is assigned.

  Therefore, the identification level calculation unit 202 calculates the identification level as, for example, level 4 × 70% + level 3 × 80% = 5.2. The maximum value of the identification level here is 10.

  The positioning level calculation unit 203 has a positioning level calculation function, and calculates a positioning level used when the correspondence degree calculation unit 204 calculates the correspondence degree. The positioning level calculation function has a function that receives the positioning method information and the positioning accuracy information of the physical object identification information and outputs a positioning level corresponding to the information. This function is expressed by, for example, the following (Formula 2).

  Positioning level = Positioning method level x Positioning accuracy factor (Equation 2)

  Here, the positioning method level is obtained by classifying the positioning method (GPS, WLAN, geomagnetic sensor, etc.) related to the positioning method information according to the performance of the positioning method, and assigning, for example, five levels. For example, when the positioning method is GPS, level 4 is assigned. The positioning accuracy coefficient is classified according to the positioning accuracy (radio wave status, brightness status, etc.) according to the positioning accuracy information according to the status (strong radio wave status, weak radio wave status, daytime, nighttime, sunny, cloudy, rain, etc.) For example, a numerical value of 0% to 100% is assigned.

  Therefore, the positioning level calculation unit 203 calculates the positioning level as, for example, level 5 × 60% = 3.0. The maximum value of the positioning level here is 5.

  The correspondence level calculation unit 204 calculates the correspondence level based on the identification level calculated by the identification level calculation unit 202 and the positioning level calculated by the positioning level calculation unit 203. The formula for calculating the correspondence is expressed by, for example, the following (Formula 3).

  Correspondence = identification level × identification weight coefficient + positioning level × positioning weight coefficient (Equation 3)

  Here, each weighting factor of the identification weighting factor and the positioning weighting factor is set according to the type of the physical object, for example, a numerical value of 0% to 100% is assigned. For example. For example, in the case of a physical object having a small relationship with a position such as a moving object, a large value is set as the identification weight coefficient and a small value is set as the positioning weight coefficient. On the other hand, when the physical object has a large relationship with a position such as a place, a small value is set as the identification weight coefficient, and a large value is set as the positioning weight coefficient.

  Therefore, the correspondence level calculation unit 204 calculates the correspondence level as 5.2 × 80% + 3.0 × 30% = 5.06. The maximum value of the degree of correspondence here is 15.

  The physical object DB 211 is a database that receives position information and feature information included in the physical object identification information and outputs physical object information corresponding to the information. Here, the physical object information includes physical object ID, name information, general identification information, attribute information, and keyword information.

  The physical object ID is an ID for identifying a physical object, and is information serving as a key of the physical object DB 211.

  The name information is a proper noun such as the name and name of the physical object identified by the physical object ID, and may be text information or image information or audio information other than the text information.

  The general identification information is information such as the address, IP address, URL, and telephone number of the physical object identified by the physical object ID.

  The attribute information is information indicating the attribute (type) of the physical object identified by the physical object ID. For example, the attribute information is information indicating any one of a moving object such as a person, a feature such as a fixed object and a building, and a place. is there.

  The keyword information includes information on related words (keywords) related to the physical object identified by the physical object ID. Further, the keyword information may include information on unrelated words having low relevance with the physical object identified by the physical object ID. Since the keyword information changes depending on the context of the physical object, it is preferable to update the keyword information periodically.

  The physical object information acquisition unit 205 searches for a physical object ID, and acquires physical object information corresponding to these pieces of information from the physical object identification information received by the reception unit 201.

  When the content information received by the receiving unit 201 is other than text information (audio information, image information, etc.), the text conversion unit 206 converts the information into text information. In this case, the image information is converted into text information by, for example, a Web image search technique that obtains text information corresponding to the image information from the image information. Also, the voice information is converted into text information by a predetermined voice recognition process.

  For example, when image information of a five-storied pagoda is acquired as content information, “5-storied pagoda” as text information is obtained by a Web image search technique. Further, when the five-storied pagoda audio information is acquired as the content information, “5-storied pagoda” as the text information is obtained by the voice recognition process.

  The number-of-words calculation unit 207 obtains information on proper nouns such as names of physical objects, related words, and unrelated words included in the physical object information acquired by the physical object information acquisition unit 205 from the content information as necessary. The number of words included in the converted text information is calculated. Note that if the keyword information held in the physical object DB 211 does not include unrelated word information, the word count calculation unit 207 may infer an unrelated word based on the related word.

  For example, when the proper noun of the physical object is “5-storied pagoda”, the number of “5-storied pagoda” as text information is counted. Further, for example, the number of words / phrases that are highly relevant to the five-storied pagoda such as “wooden”, “oldest in Japan”, “1000” as text information is counted as related words. Also, for example, the number of words or phrases that are not related to the five-storied pagoda such as “concrete” as text information is counted as unrelated words in the text information.

  The degree of association calculation unit 208 calculates the degree of association based on the number of proper nouns, the number of related words, and the number of unrelated words of the physical object calculated by the word number calculation unit 207. The degree of association is expressed by, for example, the following (Formula 4).

  Relevance = (number of proper nouns x proper noun weighting coefficient + number of related words x related weighting coefficient-unrelated word x unrelated weighting coefficient) / total number of words (Equation 4)

  Here, the proper noun weighting factor is a weighting factor set according to the importance of the proper noun. The related weighting factor is a weighting factor set according to the importance of the related word. The unrelated weight coefficient is a weight coefficient set according to the importance of the unrelated word. Each weighting factor is set between 0% and 100%. The total number of words is the total number of words included in the text information. Each weighting factor is set based on the attribute of the physical object.

  For example, if the physical object has a proper noun that can be uniquely identified, the proper noun weighting coefficient is set to a large value. For a physical object that does not have a uniquely identifiable proper noun, the related weight coefficient is set to a large value. If it is assumed that there is a lot of similar information of the physical object, the unrelated weight coefficient is set to a small value.

  Therefore, the degree-of-association calculation unit 208 calculates the degree of association as, for example, (24 × 80% + 46 × 40% −13 × 80%) / 254 = 11%. The maximum value of the identification level here is 10. The degree of association here is a value of 0% to 100%.

  In addition, when the reception unit 201 receives a plurality of pieces of physical object identification information, the degree-of-association calculation unit 208 calculates the degree of association for each physical object and further calculates the degree of association for the entire physical object.

  Further, as described above, by considering unrelated words when calculating the degree of relevance, it is possible to filter similar information that is likely to be erroneously detected only with the proper nouns and related words of the physical object.

  The signature generation unit 209 generates signature information in order to guarantee the validity of the content information for the physical object. At this time, signature information is generated based on at least information management server identification information for identifying the information management server 200. Further, as other information, for example, signature information may be generated based on physical object identification information, physical object information, content information, correspondence information, relevance information, and the like. This signature information is a digital signature meaning annotation approval permission approval of the information management server 200. As a signature generation method, there is a general digital signature method such as RSA.

  The storage unit 212 stores various information. For example, a predetermined issue policy is stored. The issuance policy is information that describes a criterion for permitting the issuance of annotations, and guarantees that content information, correspondence level information, relevance level information, and the like do not violate public good sense or service content. The issue policy may be updated as necessary.

  The transmission unit 210 transmits physical object identification information, content information, correspondence level information, and relevance level information to the information display device 300. Further, the signature information including the signature generated by the signature generation unit 209 may be transmitted together with the information. Instead of sending to the information display device 300 by communication, it may be sent to the information display device 300 via a storage medium (CD-R, SD card, etc.), for example.

  Further, the transmission unit 210 may transmit information to the information display device 300 only when the predetermined issue policy stored in the storage unit 212 is satisfied. For example, the transmission unit 210 may have at least one of the correspondence degree calculated by the correspondence degree calculation unit 204 and the association degree calculated by the association degree calculation unit 208 equal to or greater than each threshold value, or both may be equal to or greater than each threshold value. In this case, it is determined that the issued policy is satisfied, and each piece of information is transmitted.

  The example in which the information management server 200 confirms the correspondence information, the relevance information, the physical object information, and the content information and confirms the issuance with the digital signature has been described. According to such a configuration, since the information management server 200 can be confirmed centrally, there is an advantage that the configuration is simplified.

  On the other hand, as another configuration, in the physical object management server owned or operated by the owner who actually owns the physical object, the correspondence information, the relevance information, the physical object information, and the content information are based on the owner's policy. It may be configured to confirm and confirm the approval with the digital signature, and further confirm the issue with the digital signature in the information management server. With such a configuration, there is an advantage that the intention of the owner of the physical object can be accurately reflected.

  Further, the policy information of the owner may be provided in the information management server, and the issue may be confirmed based on both the issue policy and the owner policy. Such a configuration has an advantage that the configuration can be simplified while reflecting the intention of the owner.

Next, the configuration of the information display device 300 will be described.
FIG. 5 is a block diagram illustrating a configuration example of the information display device 300 according to the embodiment of the present invention.

  The information display device 300 includes a camera unit 301, an image acquisition unit 302, a feature extraction unit 303, a distance measurement unit 305, a positioning unit 306, a communication unit 307, a verification analysis unit 308, an identification information determination unit 309, a usage policy determination unit 310, A synthesis unit 311, a display unit 312, and a storage unit 313 are provided.

  The camera unit 301 is a photographing device that captures 2D or 3D still images or moving images, and is, for example, a CCD image sensor or a CMOS image sensor.

  The image acquisition unit 302 acquires an image (video through image) from the camera unit 301.

  The feature extraction unit 303 performs feature extraction processing for extracting the features of the image acquired by the image acquisition unit 302. For this feature extraction process, a known feature extraction method such as feature point extraction or edge extraction is used.

  The feature extraction unit 303 may perform recognition processing for recognizing a physical object from the extracted features. For this recognition processing, a known image recognition method such as pattern matching is used.

  The distance measurement unit 305 measures the distance from the camera unit 301 to the feature extracted by the feature extraction unit or the physical object recognized by the object. As a distance measuring method, for example, distance measurement using an optical or radar distance measuring sensor can be considered.

  The positioning unit 306 measures the position of the information display device 300. As a position measuring method, for example, GPS information (latitude, longitude, altitude, etc.) is acquired using a GPS satellite, and position information is acquired using a WLAN. Further, the positioning unit 306 may measure the position of a predetermined physical object. At this time, the positioning unit 306 first measures the position of the information display device 300, measures the distance to the physical object by the distance measuring unit 305, and the physical object viewed from the information display device 300 by a sensor (geomagnetic sensor) (not shown). Measure the direction. Based on these measurement results, the positioning unit 306 or the communication unit 307 measures the position of the physical object. The calculation of the measurement of the position of the physical object may be performed on the information management server 200 side.

  The communication unit 307 receives physical object identification information, content information (physical object annotation), correspondence information, relevance information, signature information, and the like from the transmission unit 210 of the information management server 200. Instead of acquiring these pieces of information from the information management server 200 through communication, the information may be acquired from the information management server 200 via a storage medium (CD-R, SD card, etc.), for example. Receiving the relevance information can be omitted.

  The verification analysis unit 308 verifies and analyzes the signature information received by the communication unit 307. For example, a verification analysis method includes a verification algorithm of a general digital signature method such as RSA. Then, when the generation source of the information received by the communication unit 307 is the information management server 200 and the completeness of the content of the received information is confirmed, the verification analysis unit 308 causes the communication unit 307 to It is determined that the received information is valid.

  The identification information determination unit 309 compares the physical object identification information received by the communication unit 307 with the physical object identification information measured by the information display device 300, and determines whether or not both are substantially the same. . The term “substantially coincidence” includes the case of coincidence, taking into consideration the difference in standing position, orientation, height, etc. between the user of the information providing apparatus 100 and the user of the information display apparatus 300. The intention is to identify information. The extent to which the error is allowed can be described in advance in the usage policy.

  The physical object identification information measured by the information display device 300 includes position information related to the position of the information display device 300 or physical object measured by the positioning unit 306, feature information related to the feature extracted by the feature extraction unit 303, The distance information measured by the distance measuring unit 305 is included. These position information, feature information, and distance information are the same type as the position information, feature information, and distance information obtained by the information providing apparatus 100 through measurement or the like.

  The storage unit 313 stores various information. For example, a predetermined usage policy is stored. The usage policy is information that describes the criteria for judging the usage of annotation. The content related to the content information, the degree of correspondence related to the degree of correspondence information, the degree of relevance related to the degree of association information, etc. are based on the user's preference, the organization, etc. It guarantees that it is not against. The usage policy may be updated as necessary. The storage unit 313 stores, for example, information that the degree of correspondence and the degree of association are greater than or equal to a predetermined value as a usage policy.

  The storage unit 313 stores importance level information indicating the importance level of the annotation. For example, in the case where importance level information indicating that a correspondence level and a high degree of association are important is stored, the presentation order of annotations can be determined based on this information.

  The usage policy determination unit 310 determines whether the correspondence level information and the relevance level information received by the communication unit 307 satisfy a predetermined usage policy stored in the storage unit 313. For example, when at least one of the correspondence level and the relevance level is equal to or higher than each threshold value, or both are equal to or higher than each threshold value, it is determined that the usage policy is satisfied. Note that the comparison of the relevance information may be omitted, and it may be determined whether or not the correspondence information satisfies a predetermined usage policy.

  The synthesizing unit 311 is a physical object included in the image captured by the camera unit 301 and is identified by the physical object identification information received by the communication unit 307 and the annotation received by the communication unit 307. The content information is synthesized in association with each other.

  The display unit 312 is a liquid crystal display (LCD) or a head mounted display (HMD), and displays physical objects and annotations synthesized in association with each other by the synthesis unit 311. In the display by the display unit 312, for example, an annotation is overlaid (superimposed) displayed on the physical object. In addition, the display unit 312 may project and display an annotation on a physical object using a projector.

  When the annotation is image information or text information, the display unit 312 displays the physical object and the annotation in association with each other. However, when the annotation is audio information, the physical object is used instead of the display unit 312. Annotation associated with may be presented by voice based on voice information via a speaker or the like (not shown).

Next, a specific example of physical object identification information will be described.
FIG. 6 is a diagram illustrating an example of physical object identification information.

  In this embodiment, the information providing apparatus 100 can check the reliability of the annotation corresponding to the physical object using the information of the image captured by the camera unit 101. From the image, the feature extraction unit 103, the distance measurement unit 105, and the positioning unit 106 can identify the feature, the moving object, and the place. Therefore, the relationship between the physical object and the annotation can be clarified by adding the image to the annotation with information for enhancing authentication. As a basic concept, the information setting unit 107 divides a video through image at the time of feature extraction in units of physical objects, and manages feature information for each unit.

  As a method of mounting physical object identification information when an annotation is provided from the information providing apparatus 100 to the information management server 200, a method of extending Exif information by the information setting unit 107, a method of adding unique header information, or the image itself For example, a method of embedding information using information hiding such as digital watermarking can be considered.

  Here, a method for extending Exif information will be described. As feature information managed for each physical object unit, in addition to marker, QR code, and feature point information of the edge portion that is the outline of the physical object, which is information that can be acquired from the image, a reference point for identifying the physical object And the distance from the identification position to the physical object. For the identification position, the latitude, longitude, and direction information of the GPS Info tag can be used as it is. The Exif information includes subject distance information (Subject Distance) as distance information. The information setting unit 107 sets this information as extended information for each physical object.

  The information display device 300 verifies the validity of the annotation. At the time of this verification, an autofocus (AF) distance measuring function which is a basic function of a camera included in many mobile devices such as mobile phones is used. At this time, the identification information determination unit 309 compares the distance related to the distance information set as the additional information of the physical object with the distance actually measured by the information display device 300. The identification information determination unit 309 can distinguish between an actual physical object and an image of the physical object by comparing a plurality of distances from the reference point in addition to comparing images and feature points.

  Further, the information providing apparatus 100 can set different contents for different observation planes of the same physical object group by positioning the content from different reference points and setting the content even in the same physical object group. it can.

  As shown in FIG. 6, as physical object identification information, attribute information indicating attributes of a moving object, a feature, a place, etc., feature extraction method information indicating a feature extraction method, object feature information indicating features of a physical object, feature extraction Feature extraction accuracy information indicating accuracy, distance measurement method information indicating distance measurement method, distance measurement accuracy information indicating distance measurement accuracy, and object indicating a distance from a reference point (for example, a point where the camera unit 101 exists) to a physical object Each information of distance information and physical object angle information indicating an angle formed by a shooting direction by the camera unit 101 and a physical object can be included. The minus sign (−) of the angle related to the physical object angle information indicates that the angle is the opposite side of the plus sign angle with reference to the shooting direction by the camera unit 101. These pieces of information are set for each physical object.

  Further, as shown in FIG. 6, the basic information of the physical object identification information includes each information of positioning method information indicating the positioning method, object position information indicating the position of the physical object, and positioning accuracy information indicating the positioning accuracy. Can do. These pieces of information are not set for each physical object, but are set for each captured image including one or a plurality of physical objects.

Next, the operation of the information providing apparatus 100 will be described.
FIG. 7 is a flowchart illustrating an operation example of the information providing apparatus 100.

  First, the image acquisition unit 102 acquires an image captured by the camera unit 101 (step S101), and the feature extraction unit 103 extracts features of the acquired image (step S102).

  Here, when the feature of the image indicates a place, the address display, the street name, and the like are recognized as characters. In addition, when the feature of the image indicates a person, the name tag or the like worn by the person is recognized or the feature point is extracted. Further, when the feature of the image indicates a car, the number plate of the car is recognized or characters are extracted. When the feature of the image indicates a feature, the building name, the bronze image name, etc. are recognized as characters or the feature points are extracted.

  Subsequently, the area designating unit 104 designates the area of the physical object to which the annotation is to be added from the physical objects whose feature points have been extracted or recognized by the feature extracting unit 103. At this time, for example, an area of the physical object is specified by touching the physical object on the screen via the touch panel as the operation unit 108 (step S103).

  Subsequently, the distance measuring unit 105 measures the distance from the information providing apparatus 100 to the physical object specified by the area specifying unit 104 (step S104).

  Subsequently, the positioning unit 106 measures the position of the information providing apparatus 100 using GPS or the like (step S105).

  Subsequently, the information setting unit 107 sets the object identification information in the metadata of the image. For example, feature information and distance information are set (added) to the Exif information for each physical object, and position information is set as common information (step S106).

  Subsequently, the content generation unit 109 generates content information as an annotation corresponding to the physical object specified by the region specification unit 104 via the operation unit 108 or the like (step S107).

  Subsequently, the transmission unit 110 transmits the image information including the object identification information set by the information setting unit 107 and the content information generated by the content generation unit 109 to the information management server 200 (step S108). .

  According to such an information providing apparatus 100, the degree of correspondence and the degree of association can be generated based on the physical object identification information, and it can be determined that the generated annotation is a highly reliable annotation.

Next, the operation of the information management server 200 will be described.
8 and 9 are flowcharts illustrating an operation example of the information management server 200.

  First, the receiving unit 201 receives image information including object identification information and content information from the information providing apparatus 100 (step S201).

  Subsequently, the physical object information acquisition unit 205 refers to the physical object DB 211 and acquires physical object information corresponding to the physical object feature information received by the reception unit 201 (step S202).

  Subsequently, the identification level calculation unit 202 assigns a feature extraction method level from the feature extraction method information included in the feature information included in the object identification information, and assigns a distance measurement method level from the distance measurement method information included in the distance information. Also, the identification level calculation unit 202 assigns an extraction accuracy coefficient from the feature extraction accuracy information included in the feature information, and assigns a ranging accuracy coefficient from the distance measurement accuracy information included in the distance information (step S203). For example, the information management server 200 has the conversion table that associates the patent extraction method information and the like with the feature extraction method level and the like, so that the above assignment is possible.

  Subsequently, the identification level calculation unit 202 calculates an identification level based on the feature extraction method level, the distance measurement method level, the extraction accuracy coefficient, and the distance measurement accuracy coefficient (step S204). For example, the identification level is calculated by the arithmetic expression shown in (Expression 1) described above.

  Subsequently, the positioning level calculation unit 203 assigns a positioning method level from the positioning method information included in the position information included in the object identification information, and assigns a positioning accuracy coefficient from the positioning accuracy information included in the position information (step S205). For example, the information management server 200 has the conversion table that associates the positioning method information and the like with the positioning method level and the like, so that the above assignment is possible.

  Subsequently, the positioning level calculation unit 203 calculates a positioning level based on the positioning method level and the positioning accuracy coefficient. For example, the positioning level is calculated by the arithmetic expression shown in (Expression 2) described above (step S206).

  Subsequently, the correspondence degree calculation unit 204 assigns an identification weight coefficient and a positioning weight coefficient according to attribute information (car, person, place, etc.) included in the physical object information acquired by the physical object information acquisition unit 205. (Step S207).

  Subsequently, the correspondence level calculation unit 204 calculates the correspondence level based on the identification level, the positioning level, the identification weight coefficient, and the positioning weight coefficient (step S208). For example, the correspondence degree is calculated by the arithmetic expression shown in (Expression 3) described above.

  Subsequently, when the content information received by the receiving unit 201 is image information other than text information, audio information, or the like, the text conversion unit 206 converts the information into text information (step S209).

  Subsequently, the word count calculation unit 207 adds the proper noun, the related word, the text information as the content information or the text information converted by the text conversion unit 206, which is included in the physical object information acquired by the physical object information acquisition unit 205, The number of unrelated words is calculated (step S210). The word number calculation unit 207 also calculates the total number of words included in the text information.

  Subsequently, the relevance calculation unit 208 assigns a proper noun weight coefficient, a related weight coefficient, and an unrelated weight coefficient based on attribute information included in the physical object information acquired by the physical object information acquisition unit 205 (step S211). ). For example, the information management server 200 has the conversion table that associates the attribute information with each weighting factor, so that the above assignment is possible.

  Subsequently, the relevance calculation unit 208 calculates the relevance based on the number of proper nouns, the number of related words, the number of unrelated words, the total number of words, the proper noun weight coefficient, the related weight coefficient, and the non-related weight coefficient (step S212). ). For example, the relevance is calculated by the arithmetic expression shown in (Expression 4) described above.

  In addition, the signature generation unit 209 generates signature information. For example, when receiving a request for annotation from the information display device 300, the transmission unit 210 transmits object identification information, annotation, correspondence information, relevance information, signature information, and the like.

  According to such an operation of the information management server 200, the reliability of the annotation can be provided, for example, in numerical form by calculating the degree of correspondence and the degree of association (at least the degree of correspondence) between the physical object and the annotation.

Next, the operation of the information display device 300 will be described.
FIG. 10 is a flowchart illustrating an operation example of the information display apparatus 300.

  First, the positioning unit 306 measures the position, orientation, and the like of the information display device 300 using a GPS, a geomagnetic sensor, or the like (step S301).

  Subsequently, the communication unit 307 uses the position, orientation, and the like of the information display device 300 as information on one or more physical object identification information, annotations corresponding to the physical object identification information, correspondence information, relevance information, and the like. Obtained from the management server 200 (step S302). At this time, information such as the position of the information display device 300 may be transmitted to the information management server 200, and physical object identification information, annotations, and the like corresponding to the position may be acquired from the information management server 200. In addition, information (object identification information) detected by the identification information detection unit such as the feature extraction unit 303, the distance measurement unit 305, or the positioning unit 306 is transmitted to the information management server 200, and annotations corresponding to the physical object identification information are transmitted. May be acquired from the information management server 200.

  Subsequently, the verification analysis unit 308 verifies the signature information received by the communication unit 307. As a result of the verification, when it is determined that the information from the information management server 200 is valid, that is, when the identity of the transmission source and the transmission content is guaranteed by the signature information, the object identification received by the communication unit 307 is identified. Information is analyzed (step S303). The signature may be omitted.

  Subsequently, the image acquisition unit 302 acquires an image captured by the camera unit 301 (step S304), and the feature extraction unit 303 extracts the feature of the acquired image (step S305).

  Here, when the feature of the image indicates a place, the address display, the street name, and the like are recognized as characters. In addition, when the feature of the image indicates a person, the name tag or the like worn by the person is recognized or the feature point is extracted. Further, when the feature of the image indicates a car, the number plate of the car is recognized or characters are extracted. When the feature of the image indicates a feature, the building name, the bronze image name, etc. are recognized as characters or the feature points are extracted.

  Subsequently, the distance measurement unit 305 measures the distance from the information display device 300 to the feature point extracted by the feature extraction unit 303 or the physical object recognized by the object (step S306).

  Subsequently, the identification information determination unit 309 determines whether the physical object identification information measured by the feature extraction unit 303, the distance measurement unit 305, and the positioning unit 306 substantially matches the physical object identification information received by the communication unit 307. Is determined (step S307). For example, measured feature information and received feature information (especially object feature information), measured distance information and received distance information, measured position information and received position information are almost the same. In this case, it is determined that they are substantially the same.

  When it is determined that the two physical object identification information substantially match, the usage policy determination unit 310 refers to the usage policy stored in the storage unit 313, and the correspondence level information and the relevance level information received by the communication unit 307. It is determined whether or not satisfies the usage policy (step S308). For example, it is possible to determine that the usage policy is satisfied when the degree of correspondence according to the degree of correspondence information and the degree of association according to the degree of association information are equal to or greater than a predetermined value. When the usage policy is satisfied, it can be determined that the annotation corresponding to the physical object has high reliability.

  If the usage policy is satisfied, the display unit 312 combines and displays the annotation that satisfies the usage policy in association with the physical object (step S309). At this time, when there are a plurality of annotations, they may be presented in order of importance stored in the storage unit 313. For example, they can be displayed in descending order of correspondence and relevance. Further, the display unit 312 may display at least one of the correspondence level and the relevance level simultaneously with the annotation. In addition, when the usage policy is satisfied, the display unit 312 may display that the usage policy is satisfied simultaneously with the annotation. The fact that the usage policy is satisfied is displayed by, for example, characters, icons, colors, and the like. It may be presented by a method other than display.

  According to such an operation of the information display device 300, it is possible to present to the user so that a highly reliable annotation among the annotations corresponding to the physical object can be easily identified. Therefore, the user can use the presented annotation with peace of mind.

  According to such a virtual reality system of this embodiment, it is possible to guarantee the relationship between a physical object and an annotation. For example, even if the information providing apparatus 100 intends to give an annotation to a desired physical object, the information providing apparatus 100 is replaced as an annotation for another physical object during management by the information management server 200. Even in such a case, an unexpected replacement of the annotation can be found according to the correspondence level and the relevance level calculated by the information management server 200. In this way, unauthorized use or impersonation of annotation as evaluation information can be detected.

  In addition, a large number of negative annotations are pasted, and it is possible to prevent a store, an individual, or the like corresponding to the annotation from being damaged by reputation because the annotation is misidentified.

  In addition, by checking the degree of correspondence, it is possible to determine how physically the physical object and the annotation are related. In addition, regardless of whether the physical object is a moving object, a place, or a feature, the degree of correspondence between the physical object and the annotation can be accurately calculated. Further, by checking the degree of association, it is possible to determine how semantically related the physical object and the annotation are.

  In addition, the information management server 200 does not receive physical object identification information, content information, and the like by the receiving unit 201, but the information management server 200 itself measures information included in the physical object identification information and generates content information. You may do it.

  Further, the information providing apparatus 100 may have a function as the information management server 200.

  In addition, as the augmented reality system 1 of the present embodiment, a server-client type system including the information providing apparatus 100, the information management server 200, and the information display apparatus 300 is assumed, and a plurality of information is provided via the information management server 200. Although it has been described that annotations are shared between clients, a stand-alone type system may be used.

  The present invention is useful for an information processing apparatus, an augmented reality system, an information processing program, and the like that can guarantee the relationship between a physical object and an annotation.

DESCRIPTION OF SYMBOLS 1 Augmented reality system 100 Information provision apparatus 101 Camera part 102 Image acquisition part 103 Feature extraction part 104 Area designation part 105 Distance measurement part 106 Positioning part 107 Information setting part 108 Operation part 109 Content generation part 110 Transmission part 200 Information management server 201 Receiver 202 Identification level calculator 203 Positioning level calculator 204 Correspondence calculator 205 Physical object information acquisition unit 206 Text converter 207 Word count calculator 208 Relevance calculator 209 Signature generator 210 Transmitter 211 Physical object DB
212 storage unit 300 information display device 301 camera unit 302 image acquisition unit 303 feature extraction unit 305 distance measurement unit 306 positioning unit 307 communication unit 308 verification analysis unit 309 identification information determination unit 310 usage policy determination unit 311 synthesis unit 312 display unit 313 storage Part

An information processing apparatus of the present invention identifies an image acquisition unit that acquires an image, an identification information detection unit that detects physical object identification information for identifying a physical object included in the image, and a predetermined physical object Detected by the identification information detection unit, an information acquisition unit that acquires physical object identification information, an annotation of the physical object, and relevance information indicating a semantic relevance between the physical object and the annotation. The identification information determination unit for determining whether or not the physical object identification information substantially matches the physical object identification information acquired by the information acquisition unit, and the identification information determination unit the relevance information of the annotation of the physical objects, meets a predetermined usage policy And determining usage policy determining section whether, if the it is determined to satisfy the usage policy by utilizing policy judgment section, and a annotation presentation unit for presenting the annotation.

In the information processing apparatus according to the present invention, the information acquisition unit acquires correspondence level information indicating a physical correspondence level between the physical object and the annotation, and the usage policy determination unit includes the identification information determination unit. If it is determined that the values substantially match, it is determined whether the correspondence level information and the relevance level information of the annotation of the physical object satisfy a predetermined usage policy.

With this configuration, it is possible to guarantee the relevance between the physical object and the annotation by the correspondence level and the relevance level. The reliability is further improved as compared with the case where the reliability is determined based on the relevance alone.

The information processing apparatus of the present invention, and annotations generator for generating an annotation before Symbol physical objects, for each of the physical objects, and annotations generated by the annotation generation unit, which is detected by the identification information detecting section A physical object identification information; and a transmission unit for transmitting the physical object identification information.

Further, the information processing apparatus according to the present invention includes a physical object identification information for identifying a physical object, an information acquisition unit that acquires the annotation of the physical object, and the physical object based on the physical object identification information. A degree- of- association calculating unit that calculates a degree of semantic association with the annotation of the physical object.

With this configuration, by calculating the degree of association between the physical object and the annotation, the reliability of the annotation can be provided, for example, in the form of a numerical value.

The information processing apparatus according to the present invention further includes a correspondence calculation unit that calculates a physical correspondence between the physical object and the annotation based on the identification information of the physical object and the annotation.

With this configuration, by further calculating the degree of correspondence between the physical object and the annotation, the reliability of the annotation can be provided, for example, as a numerical value.

An augmented reality system according to the present invention is an augmented reality system including an information providing device that provides an annotation corresponding to a physical object, an information management server that manages the annotation, and an information presentation device that presents the annotation. The information providing apparatus generates an image acquisition unit for acquiring an image, an identification information detection unit for detecting physical object identification information for identifying a physical object included in the image, and an annotation for the physical object An annotation generation unit that performs, for each physical object, a transmission unit that transmits the annotation generated by the annotation generation unit and the physical object identification information detected by the identification information detection unit. The management server sends the information from the information providing device. An information acquisition section for acquiring said the object identification information annotation, the degree-of-association calculation unit based on said physical object identification information, calculates the semantic relevance between the physical object and annotation of the physical object, the A transmission unit that transmits physical object identification information, the annotation, and the degree of association; and the information presentation device identifies an image acquisition unit that acquires an image, and a physical object included in the image An identification information detecting unit for detecting physical object identification information for detecting, a physical object identification information for identifying a predetermined physical object from the information management server, an annotation of the physical object, the relevance information, Detected by the information acquisition unit and the identification information detection unit When it is determined that the physical object identification information and the physical object identification information acquired by the information acquisition unit substantially match, and the identification information determination unit that determines whether the physical object identification information substantially matches, When it is determined by the usage policy determination unit that the relevance level of the annotation of the physical object satisfies a predetermined usage policy, and the usage policy determination unit determines that the usage policy is satisfied, the annotation is An annotation presentation unit to present.

With this configuration, the degree of association can be generated based on the physical object identification information, and the generated annotation can be determined to be a highly reliable annotation. Further, by calculating the degree of association between the physical object and the annotation, the reliability of the annotation can be provided, for example, as a numerical value. Furthermore, it is possible to guarantee the relationship between the physical object and the annotation. That is, it is possible to present to the user so that a highly reliable annotation among the annotations corresponding to the physical object can be easily identified. Therefore, the user can use the presented annotation with peace of mind.

The first information processing method of the present invention includes a step of acquiring an image, a step of detecting physical object identification information for identifying a physical object included in the image, and a predetermined physical object. Physical object identification information, an annotation of the physical object, and relevance information indicating a semantic relevance between the physical object and the annotation; and the detected physical object identification information; The step of determining whether or not the acquired physical object identification information substantially matches, and if it is determined that the physical object identification information substantially matches, the relevance information of the annotation of the physical object indicates a predetermined usage policy. A step of determining whether or not to satisfy, and it is determined that the usage policy is satisfied If, and a step of presenting the annotations.

The second information processing method of the present invention, the transmission and generating an annotation before Symbol physical objects, for each of the physical objects, and annotations said generated physical object identification information the detected, the And a step of performing.

According to a second information processing method of the present invention, the physical object identification information for identifying a physical object and the annotation of the physical object are acquired, and the physical object is based on the physical object identification information. And calculating a semantic relevance between the physical object annotation and the physical object annotation.

By calculating the degree of association between a physical object and an annotation by this method, the reliability of the annotation can be provided, for example, in numerical form.

By calculating the degree of association between the physical object and the annotation by this program, the reliability of the annotation can be provided, for example, as a numerical value.

Claims (25)

An image acquisition unit for acquiring images;
An identification information detection unit for detecting physical object identification information for identifying a physical object included in the image;
An information acquisition unit for acquiring physical object identification information for identifying a predetermined physical object, an annotation of the physical object, and correspondence information indicating a physical correspondence between the physical object and the annotation;
An identification information determination unit that determines whether or not the physical object identification information detected by the identification information detection unit substantially matches the physical object identification information acquired by the information acquisition unit;
A usage policy determination unit that determines whether or not the correspondence information of the annotation of the physical object satisfies a predetermined usage policy when it is determined by the identification information determination unit that they substantially match;
When it is determined by the usage policy determination unit that the usage policy is satisfied, an annotation presentation unit that presents the annotation;
An information processing apparatus comprising: The information processing apparatus according to claim 1,
The information acquisition unit acquires relevance information indicating a semantic relevance between the physical object and the annotation,
The usage policy determination unit determines whether or not the correspondence information and the relevance information of the annotation of the physical object satisfy a predetermined usage policy when it is determined that the identification information determination unit substantially matches the usage policy. Information providing apparatus for determining. The information processing apparatus according to claim 1, wherein:
The annotation presenting unit displays an information indicating that the usage policy is satisfied simultaneously with the annotation when the usage policy determination unit determines that the usage policy is satisfied. The information processing apparatus according to any one of claims 1 to 3,
The identification information detection unit detects position information related to the position of the information processing device, and distance information detects distance information related to the distance from the information processing device to the physical object included in the image. A detection unit;
The information acquisition unit acquires position information related to a position of a predetermined physical object as the physical object identification information based on position information related to the position of the information processing apparatus and distance information related to a distance to the physical object. And
The identification information determination unit substantially matches the position information acquired by the information acquisition unit and the position information detected by the position information detection unit, and the distance information acquired by the information acquisition unit and the distance information detection. An information processing apparatus that determines whether or not the distance information detected by the unit substantially matches. The information processing apparatus according to any one of claims 1 to 3,
The identification information detection unit includes a feature information extraction unit that extracts feature information related to a feature of the physical object included in the image,
The information acquisition unit acquires feature information relating to features of the physical object as the physical object identification information,
The identification information determination unit is an information processing apparatus that determines whether or not the feature information acquired by the information acquisition unit substantially matches the feature information extracted by the feature information extraction unit. The information processing apparatus according to any one of claims 1 to 3,
The identification information detection unit detects position information related to the position of the information processing device, and distance information detects distance information related to the distance from the information processing device to the physical object included in the image. A detection unit, and a feature information extraction unit that extracts feature information related to the feature of the physical object included in the image,
The information acquisition unit acquires, as the physical object identification information, position information related to a position of a predetermined physical object, distance information related to a distance to the physical object, and feature information related to a feature of the physical object. And
The identification information determination unit substantially matches the position information acquired by the information acquisition unit and the position information detected by the position information detection unit, and the distance information acquired by the information acquisition unit and the distance information detection. Information processing for determining whether or not the distance information detected by the unit substantially matches and the feature information acquired by the information acquisition unit and the feature information extracted by the feature information extraction unit substantially match apparatus. The information processing apparatus according to claim 1, wherein:
The annotation presentation unit is an information processing apparatus that displays the annotation in association with a physical object included in an image acquired by the image acquisition unit. An information processing apparatus according to any one of claims 1 to 7,
The information processing apparatus that presents the annotations in descending order of the degree of correspondence or the degree of association. The information processing apparatus according to any one of claims 1 to 8,
The predetermined usage policy is an information processing apparatus in which each of the correspondence level and the relevance level or a total value is a predetermined value or more. An information processing apparatus according to any one of claims 1 to 9,
The annotation presentation unit is an information processing apparatus that presents at least one of the correspondence degree and the association degree simultaneously with the annotation. An image acquisition unit for acquiring images;
An identification information detection unit for detecting physical object identification information for identifying a physical object included in the image;
An annotation generator for generating an annotation of the physical object;
A sending unit that sends out the annotation generated by the annotation generation unit and the physical object identification information detected by the identification information detection unit for each physical object;
An information processing apparatus comprising: The information processing apparatus according to claim 11,
The identification information detection unit detects position information related to the position of the information processing device, and distance information detects distance information related to the distance from the information processing device to the physical object included in the image. A detection unit, and a feature information extraction unit that extracts feature information related to the feature of the physical object included in the image,
The transmission unit is an information processing apparatus that transmits the position information, the distance information, and the feature information as the physical object identification information. An information acquisition unit for acquiring physical object identification information for identifying a physical object and an annotation of the physical object;
A correspondence calculation unit that calculates a physical correspondence between the physical object and the annotation of the physical object based on the physical object identification information;
An information processing apparatus comprising: The information processing apparatus according to claim 13, further comprising:
An information processing apparatus comprising: a degree-of-association calculating unit that calculates a degree of semantic association between the physical object and the annotation based on the identification information of the physical object and the annotation. The information processing apparatus according to claim 13 or 14, further comprising:
An information processing apparatus comprising: a sending unit that sends correspondence degree information related to the degree of correspondence calculated by the correspondence degree calculation unit and relevance degree information related to the degree of association calculated by the association degree calculation unit. The information processing apparatus according to any one of claims 13 to 15,
The physical object identification information includes position information related to the position of the information processing device, distance information related to a distance to the physical object, and feature information related to the characteristics of the physical object,
The correspondence calculation unit calculates an identification level based on the feature information and the distance information, calculates a positioning level based on the position information, and calculates the correspondence based on the identification level and the positioning level. Information processing apparatus. The information processing apparatus according to any one of claims 13 to 16, further comprising:
A physical object information storage unit that associates and stores the physical object identification information and physical object information including phrase information related to the physical object identified by the physical object identification information;
A text information acquisition unit that acquires text information related to the annotation from the annotation acquired by the information acquisition unit;
With
The relevance calculation unit is acquired by the information acquisition unit from the phrase information included in the text information acquired by the text information acquisition unit from the annotation acquired by the information acquisition unit, and the physical object information storage unit An information processing apparatus that calculates the degree of association based on word / phrase information included in the physical object information associated with the physical object identification information stored. The information processing apparatus according to claim 17,
The phrase information includes information on the name of the physical object, information on a related word that is a phrase related to the physical object, and information on an unrelated word that is a phrase that is not related to the physical object. Information processing device. An augmented reality system comprising an information providing device that provides an annotation corresponding to a physical object, an information management server that manages the annotation, and an information presentation device that presents the annotation,
The information providing apparatus includes:
An image acquisition unit for acquiring images;
An identification information detection unit for detecting physical object identification information for identifying a physical object included in the image;
An annotation generator for generating an annotation of the physical object;
A sending unit that sends out the annotation generated by the annotation generation unit and the physical object identification information detected by the identification information detection unit for each physical object;
With
The information management server
An information acquisition unit for acquiring the physical object identification information and the annotation from the information providing device;
A correspondence calculation unit that calculates a physical correspondence between the physical object and the annotation of the physical object based on the physical object identification information;
A sending unit that sends the physical object identification information, the annotation, and the correspondence information;
With
The information presentation device includes:
An image acquisition unit for acquiring images;
An identification information detection unit for detecting physical object identification information for identifying a physical object included in the image;
An information acquisition unit that acquires the physical object identification information for identifying a predetermined physical object from the information management server, the annotation of the physical object, and the correspondence information;
An identification information determination unit that determines whether or not the physical object identification information detected by the identification information detection unit substantially matches the physical object identification information acquired by the information acquisition unit;
A usage policy determination unit that determines whether or not the correspondence information of the annotation of the physical object satisfies a predetermined usage policy when it is determined by the identification information determination unit that they substantially match;
When it is determined by the usage policy determination unit that the usage policy is satisfied, an annotation presentation unit that presents the annotation;
Augmented reality system with Acquiring an image;
Detecting physical object identification information for identifying a physical object included in the image;
Obtaining physical object identification information for identifying a predetermined physical object, an annotation of the physical object, and correspondence information indicating a physical correspondence between the physical object and the annotation;
Determining whether the detected physical object identification information substantially matches the acquired physical object identification information;
Determining that the correspondence information of the annotation of the physical object satisfies a predetermined usage policy if it is determined to be substantially the same;
Presenting the annotation when it is determined that the usage policy is satisfied;
An information processing method comprising: Acquiring an image;
Detecting physical object identification information for identifying a physical object included in the image;
Generating an annotation of the physical object;
Sending the generated annotation and the detected physical object identification information for each physical object;
An information processing method comprising: Obtaining physical object identification information for identifying a physical object and an annotation of the physical object;
Calculating a physical correspondence between the physical object and the annotation of the physical object based on the physical object identification information;
An information processing method comprising:   A program for causing a computer to execute each step of the information processing method according to claim 20.   A program for causing a computer to execute each step of the information processing method according to claim 21. A program for causing a computer to execute each step of the information processing method according to claim 22.

Images