JP2006178804A - Subject object information providing method and subject object information providing server - Google Patents

Abstract

【Task】
Provides information about objects displayed in images taken with a mobile phone, provides information that does not depend on the passage of time for photographic images to viewers of images, and further recognizes the recognition status of reality reflected in photographic images It provides useful information for improvement.
[Solution]
Object search means and object matching means for identifying an object shown in a photographic image by using the position and orientation and the photographic image transmitted from the mobile phone in the subject object information providing apparatus server and the object information held in the server And a subject object information providing method having object name writing means for writing an object name specified in a photographic image.
[Selection] Figure 1

Description

  The present invention relates to a subject object information providing method and a subject object information providing server used therefor.

  The fall of the image taken with the mobile phone is performed by adding information added value to the entertainment and the image. For example, Patent Document 1 discloses a mobile mobile communication in which a plurality of processed and combined images stored on an image combining server are processed only by a mobile mobile communication terminal, and a composite image can be displayed on the screen of the display unit. An image processing system in a terminal is described.

  Patent Document 2 describes that place name / facility name data is printed on a photographed image photographed by a digital camera.

JP 2004-32663 A JP 2003-209785 A

  In the conventional technology, it is possible to process an image photographed by a mobile phone by a user's operation of the mobile phone, and this is effective in adding information added value related to entertainment and images. For example, it describes in patent document 1 (Unexamined-Japanese-Patent No. 2004-32663). However, in the conventional technology, the relationship between the captured image and the object to be captured is not maintained, and therefore the information of the object shown in the captured image does not exceed the recognition range of the photographer or the individual who views the image. Therefore, there is a problem that knowledge about individual objects displayed in the image at the time of shooting is too little for the existing huge amount of information, such as causing the loss of storage of information about the shot image over time To do. In Patent Document 2, when some objects such as buildings are included in a photographed image, the specification of each object is not mentioned.

  It is an object of the present invention to provide information about an object displayed in an image taken with a mobile phone, provide information not related to the passage of time regarding a photographic image to a viewer of the image, and further, is reflected in a photographic image. It provides useful information to improve the recognition status of reality.

  The present invention provides an object matching for specifying an object shown in a photographic image by using the position and orientation transmitted from the mobile phone in the subject object information providing apparatus server and the photographic image and object information held in the server. There is provided a subject object information providing method having means and object name writing means for writing an object name specified in a photographic image.

  Specifically, the present invention provides a subject object information providing server for providing subject object information connected to a mobile phone carrier through a communication circuit, and a photograph of the subject position acquisition means and direction acquisition means, thereby acquiring a photograph image. In the subject object information providing apparatus comprising a mobile phone connected to the mobile carrier by a communication circuit, the mobile phone acquires the own position and orientation acquired by the own position acquisition means. The azimuth obtained by the means and the photographic image obtained by the photographic image obtaining means are transmitted to the subject object information providing server, and the subject object information providing server sends at least the object height for the object on the map to the object storage means. Object features, including object Stores name and longitude / latitude of object, specifies subject object information displayed on the photographic image from the received own position, orientation and photographic image, calculates object distance and calculates object distance The object feature stored by the means is searched, subject object information is matched with the object by the object matching means, and the object name is specified from the specified object by specifying the object for the subject object information. Providing a provision method.

  The subject object information providing server further transmits the specified object and object name to the mobile phone, and the mobile phone displays the object name in the subject object information on the photographic image displayed on the display means, The object information and the object name are combined and stored in the storage means.

  Further, the present invention provides a subject object information providing server for providing subject object information connected by a communication circuit to a mobile phone carrier to which a mobile phone is connected by a communication circuit, and at least an object on a map as the subject object information. Object storage means for storing object features including object height, object name and object longitude and latitude, and receiving the mobile phone's own position, orientation, and photographic image transmitted from the mobile phone via the mobile phone carrier Identifying object object information from the received own position, orientation, and photographic image, calculating object distances to calculate object distances, and retrieving object features; and object objects retrieved from object object information Performing matching between transfected characterized in that it comprises an object matching means for performing a specific object name from a particular and specific objects of the object for the subject object information.

  Further, the subject object information providing server includes storage means for storing the subject object information and an object name written on the subject object information.

  According to the present invention, it is possible to provide a plurality of objects reflected in an image photographed using a mobile phone, each object when the objects seem to overlap each other, and easily obtain information on the object. It is possible to save the image as it is, to prevent a decrease in memory over time, and to contribute to improving the recognition state of the reality shown in the image. By applying the present invention, it becomes possible to develop a highly functional electronic album function and navigation software for a mobile phone that is easier to use, and the usefulness of the mobile phone itself can be significantly improved.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

FIG. 1 is a configuration diagram of a subject object information providing apparatus according to the present invention. In FIG. 1, a subject object information providing apparatus 1 includes a subject object information providing server 2 and a mobile phone 3.
The mobile phone carrier 4 is connected to the subject object information providing server 2 using a wired data communication line 5 and to the mobile phone 3 using a wireless data communication line 6. The subject object information providing server 2 includes a receiving unit 25 that receives a photographic image, an object storage unit 23 that stores information about an object related to the received photographic image, an object searching unit 22 that searches for an object from the object storing unit, The object searched using 22 and the subject object information displayed on the photographic image received from the mobile phone 3 using the receiving means 25 are specified, and the subject object information is matched with the stored object information. An object matching means 21 that performs the above processing, an object name writing means 24 that writes the matched object on the subject object information, and a transmission means 26 that transmits the processed photographic image to the mobile phone.

  The mobile phone 3 includes a position acquisition unit 31 that acquires its own position, an orientation acquisition unit 32 that acquires a direction in which the camera of the mobile phone captures, a photographic image acquisition unit 33 that captures a photograph and acquires a photographic image, and an acquired photograph The transmission means 34 for transmitting the image, position and orientation, the reception means 35 for receiving the photographic image processed by the subject object information providing server 2, and the photographic image and subject object information providing server 2 processed by the photographic image acquisition means 33 are processed. The display unit 36 displays an image received via the receiving unit 35.

  The processed image is stored in the storage unit 37. The storage means 37 stores subject object information and object names written on the subject object information as a pair of photographic images. These pieces of information may be transmitted to the subject object information providing server 2 and stored in the storage unit 38. The information stored in the storage unit 38 is appropriately searched by the mobile phone 3.

  FIG. 2 is a system configuration diagram showing an embodiment to which the present invention is applied. The subject object information providing apparatus 1 includes a subject object information providing server 2 and a plurality of mobile phones 3, and the subject object information providing server 2 and the mobile phone 3 are connected via a mobile phone carrier 4. The subject object information providing server 2 and the mobile phone carrier 4 are connected by a wired data communication line 5, and the mobile phone carrier 4 and the mobile phone are connected by a wireless data communication line 6.

  The operation of the embodiment shown in FIGS. 1 and 2 will be described below with reference to the flowcharts shown in FIGS. 3, 5, and 7 and the diagrams shown in FIGS.

  In FIG. 3, the mobile phone 3 takes a picture, transmits the acquired photographic image to the subject object information providing server 2, processes the image in the subject object information providing server 2, transmits the image to the mobile phone 3, and displays it on the mobile phone 3. The flow of the entire subject object information providing apparatus 1 is shown. The mobile phone 3 first measures its own position using the position measuring means 31 such as GPS built in the mobile phone 3 and uses the direction acquisition means 32 such as an electronic compass built in the mobile phone 3. The direction in which the camera shoots is measured, and a photographic image is acquired by the photographic image acquisition means 33 such as a camera built in the mobile phone 3 (S3). The photograph image acquired at this time shows object information such as a building as well as a landscape such as a mountain. However, the names of the subject object information are rarely specified in the objects themselves, and the photographer cannot even know the names of these objects when there is no knowledge about the area called land intuition. When a photograph image with little knowledge for a photographer is stored as it is and then referred to after a time, there is a problem that it becomes very ambiguous where the photograph was taken and the value of the photograph image decreases with time. An example in which a photograph is taken with the mobile phone 3 and displayed as it is on the display means is shown in FIG.

  Therefore, the mobile phone 3 displays the own position measured in (S1), the orientation measured in (S2), and the photographic image acquired in (S3) on the display means (36). Some of the objects in the photographic image are specified as subject object information. This specification may be made automatically with a reference. The specified subject object information, own position and orientation information are transmitted to the subject object information providing server 2 by the transmission means (34) (S4). The position, orientation and photographic image received using the receiving means 25 of the subject object information providing server 2 are transmitted to the object searching means 22 and the object matching means 21. The object search means 22 searches for the object feature of the object from the received own position, orientation, and photographic image. The object matching means 21 determines the feature of the object including the height of the building that can be seen in the direction from the own position direction, and performs matching (verification) between the object feature and the object on the map. Further, an object name is added to the photographic image in the subject object information corresponding to the matched object (S5). It may be added to a method of switching to a photographic image and describing the object name in the photographic image. The subject object information to which the object name is added is transmitted to the cellular phone 3 via the transmitting unit 26 and the receiving unit of the cellular phone carrier 4 (S6). The cellular phone 3 displays the photographic image in which the additional information is written on the display means 36 (S7). At the same time, the subject object information in which additional information is written and stored in the storage unit 37 is stored in the object storage unit 23 or other storage unit 38 via the transmission unit 34 and the reception unit 25.

Next, a main part process flow of the object name adding process (S5) will be described with reference to FIG.
The subject object information providing server 2 is in a waiting state for a photographic image from the mobile phone 3 or subject object information (S11). When the receiving means 25 receives the position, azimuth, and at least subject object information (S12) (S13), it uses the position and orientation at which the photograph image was taken to calculate the geographical range shown in the subject object information, particularly the distance and height. Then, an object search is performed using the object search means 22 on the condition of the geographical range (S14), and the position of the subject object information on the photographic image corresponding to the number of objects searched by the object matching means 21 (S15). To identify the subject object and the object on the map (S16).

  An example of the retrieved object list is shown in FIG. The object storage means 23 includes an object type such as a building or a tower as an object, and, when the object type is a building, an object feature such as a floor number, an object height, an object name, an object longitude and latitude coordinate, Such information is stored and can be used for matching with subject object information. A specific method of matching can perform image matching with subject object information after image conversion of the object. Known template matching is famous for matching between images, and further classification includes residual matching, normalized correlation method, phase-only correlation method, geometric matching method, vector correlation method, generalized Hough transform method, etc. The vector correlation method, which is not easily affected by overlapping or hiding, is a promising candidate for ensuring high matching accuracy. Next, when the subject object matches the object, a check mark “◯” is checked in FIG. 6 indicating that it matches the image match sequence of the object in the object list (FIG. 6) (S17). Then, the object name of the checked object is added to the subject object information (S18). In addition, an object name may be added by arbitrarily selecting some from the circles.

  Next, a main part process flow of the process of adding an object name to an image (S18) will be described with reference to FIG. The object list shown in FIG. 6 subjected to the object name addition process shown in FIG. 5 is checked with reference to the image match string check information set in advance by the object name addition process (S17) shown in FIG. It is determined that the contained object information matches the photographic image, and the longitude and latitude coordinates of the object are taken out and converted into coordinates on the photographic image (S23). Specifically, the longitude and latitude coordinates at which the object is present are converted into pixel coordinates on the photograph image using the longitude and latitude coordinate position at which the photograph image was photographed and the photographed orientation. As a calculation method, a known image processing technique such as a three-dimensional projection method can be used. Further, the character size and arrangement are adjusted in accordance with the length of the character string of the object name and the size of the object (S24). That is, when the character string is long and protrudes from the building, a line feed is inserted in the middle to display it in two lines, or when the building is small, a small character is selected. Then, the character is converted into an image as preprocessing for synthesizing the photographic image and the character (S25). Specifically, a rectangular temporary image area is prepared, and a binary image is created by drawing characters therein. For example, a bit string of 0 is treated as an image for a pixel where a character exists and a pixel where a character does not exist. Then, the subject object information and the temporary character image created in step S25 are synthesized (S26). Specifically, the temporary character string image is masked, and only the pixel having a pixel value of 1 is written in the subject object information. Can be realized. The image in which the object name is written is completed by the above method. An example displayed on a mobile phone is shown in FIG. Names are assigned to the three building objects, respectively. In this state, when the image is stored, it is possible to recognize at a glance where the image was taken even if the image is stored again several years later. In addition, by using this embodiment in a place where there is no land intuition, photographing a nearby landmark object and obtaining the object name can give a great hint to the user's geographical recognition, which is one of the navigation functions. Can also be used.

  According to the above-described embodiment, the subject object information providing apparatus 1 includes the subject object information providing server 2 that provides the subject object information connected to the mobile phone carrier 4 through the communication circuit, and the own location acquisition unit 31 and the orientation acquisition. The mobile phone 3 includes a mobile phone 3 having a photographic image acquisition means 33 for taking a photograph and acquiring a photographic image with the means 32 and connected to the mobile carrier by a communication circuit. Furthermore, it has a subject object information specifying means for specifying the subject imaged in the photographic image as subject object information. As described above, this designation can be performed automatically.

  The mobile phone 3 transmits the subject position acquired by the subject position acquisition unit 31, the orientation acquired by the orientation acquisition unit 32, and the photographic image acquired by the photographic image acquisition unit 33 to the subject object information providing server 2.

  The subject object information providing server 2 stores at least object characteristics including height, object name, and object longitude and latitude as the subject object information in the object storage means 23 as shown in FIG. The object feature and object longitude and latitude stored by the object search means 23 are searched from the information of the own position, orientation and subject object, and the photographic image and the object are matched by the object matching means 21 to find the object object information about the subject object information. The object name is specified from the specified and specified object.

  The subject object information providing server 2 transmits the specified object and object name to the mobile phone 3, and the mobile phone 3 designates the object on the photographic image displayed on the display means 36, that is, the subject object information. The object name is displayed and stored in the storage means 37. This information can also be stored in the storage means 38.

  In addition, the subject object information providing server 2 that provides subject object information connected by the communication circuit to the mobile phone carrier 4 to which the mobile phone 3 is connected by the communication circuit has at least an object feature, object name as an object as subject object information. And object storage means 23 for storing the object longitude and latitude, and the mobile phone 3's own position, orientation, and subject object information transmitted from the mobile phone 3 via the mobile phone carrier 4 are received. Object search means 22 for searching for object features and object longitude and latitude from subject object information, and matching of a photographic image with the searched object to identify and identify an object for subject object information. Having features comprising an object matching means 21 for a particular object name from the specified object.

  The subject object information providing server 2 includes object name writing means for writing the specified object name on the object that is the photographic image, that is, subject object information.

  The subject object information providing server 2 includes storage means for storing a photographic image and an object name written on the object of the photographic image.

  As described above, the subject object information providing apparatus or the subject object information providing server according to the present embodiment uses the cellular phone with a built-in data communication function and a positioning electronic compass by photography or GPS, thereby adding information to the photographic image. It can be used as an additional service to services and navigation functions.

The block diagram of the to-be-photographed object information provision apparatus of this invention. The system block diagram of the Example of this invention. Process flow diagram. The figure which shows the mobile phone picked-up image display example. The principal part processing flowchart of an object name addition process. Object search result list diagram. The principal part process flow figure of the process which adds an object name to an image. The figure which shows the example of a mobile telephone screen display.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Subject object information provision apparatus, 2 ... Subject object information provision server, 3 ... Mobile telephone, 4 ... Mobile phone carrier, 5 ... Wired data communication line, 6 ... Wireless data communication line, 21 ... Object matching means, 22 ... Object Search means 23 ... Object storage means 24 ... Object name writing means 25 ... Reception means 26 ... Transmission means 31 ... Position acquisition means 32 ... Direction acquisition means 33 ... Photo image acquisition means 34 ... Transmission means, 35 ... receiving means, 36 ... display means, 37 ... storage means.

Claims (5)

A subject object information providing server for providing subject object information connected to a mobile phone carrier through a communication circuit, and a photograph image obtaining means for taking a photograph with a self-position obtaining means and a direction obtaining means, and obtaining a photograph image; In a subject object information providing method comprising a mobile phone connected to a mobile carrier by a communication circuit,
The mobile phone transmits the own position acquired by the own position acquisition unit, the direction acquired by the direction acquisition unit, and the photographic image acquired by the photographic image acquisition unit to the subject object information providing server,
The subject object information providing server stores an object feature including at least the height of an object, an object name, and an object longitude and latitude for an object on a map in an object storage means, and the received self position, orientation, and photographic image The object object information displayed on the photographic image is identified, the distance to the object object information is calculated, the object feature stored by the object search means is searched, the object object information and the object A subject object information providing method comprising: performing object matching and identifying an object for the subject object information and identifying an object name from the identified object.   The subject object information providing server according to claim 1, wherein the subject object information providing server transmits the specified object and the object name to the mobile phone, and the mobile phone sets the object name in the subject object information on the photographic image displayed on the display means. A method for providing subject object information, comprising: displaying and storing subject object information and an object name together in a storage unit. In a subject object information providing server for providing subject object information connected by a communication circuit to a mobile phone carrier to which a mobile phone is connected by a communication circuit,
Object storage means for storing at least object height including object height, object name and object longitude / latitude for objects on the map as the subject object information, and the mobile phone transmitted from the mobile phone via the mobile phone carrier Object search means for receiving the subject position, orientation, and photographic image of the subject, specifying subject object information from the received subject position, orientation, and photographic image, calculating a distance to the subject object information, and retrieving object features And object matching that performs matching between the subject object information and the retrieved object to identify an object for the subject object information and identify an object name from the identified object. Subject object information providing server, characterized in that it comprises means.   4. The subject object information providing server according to claim 3, further comprising object name writing means for writing the specified object name on the subject object information. 5. The subject object information providing server according to claim 4, further comprising storage means for storing the subject object information and an object name written on the subject object information.

Images