JP2017111034A - Terminal having camera, navigation information providing system, and navigation information display method - Google Patents

Abstract

Provided is a technique for accurately navigating a user in an environment where sufficient accuracy cannot be expected from a GPS or a gyro sensor. In the navigation information display of this embodiment, the three-dimensional relative position between the camera and the marker is not used. The relative angle between the camera and the marker is obtained as the angle between the camera optical axis vector in the camera model and the vector to the marker from the marker position projected on the image and the optical axis position obtained by calibrating in advance. Based on the obtained relative angle, the image taken on the screen and the navigation arrows and guidance are superimposed and displayed at a more appropriate angle to navigate the user to the destination. [Selection] Figure 6

Description

  The present invention relates to a camera-equipped terminal, a navigation information providing system, and a navigation information display method.

  Conventionally, there are navigation techniques and products that use GPS to reach a location and direction on the map. In recent years, smartphones equipped with a camera, GPS, gyro sensor, acceleration sensor, and magnetic sensor have become widespread, and many AR technologies for displaying navigation information superimposed on an image taken with a camera in real time have been announced.

  On the other hand, GPS cannot be used underground or indoors where GPS satellites cannot be supplemented. Further, for navigation in a wide space such as outdoors, the sensor position / posture error is allowed, but in a narrow indoor space, the sensor position / posture error is not allowed as compared to the outdoors. In order to reach the destination indoors, for example indoors, in which GPS is not present, the user must rely on a whole guide map and a signpost with arrows.

  However, the general guide map is generally only at the entrance / exit of the facility and the elevator hall of each floor and cannot be referred to when necessary. In addition, in the guidance with arrows, representative destinations with high use frequency are often written, but it is not expected to be written about destinations with low use frequency.

  In order to cope with this problem, for example, as disclosed in Patent Document 1, there is a method of recognizing a marker reflected on a camera and obtaining a positional relationship with the marker without using a GPS or a gyro sensor.

JP 2005-115500 A

  Patent Document 1 uses a two-dimensional barcode printed in black and white, and simultaneously with reading information, the normal direction of the two-dimensional barcode plane, the relative angle in the camera optical axis direction, the two-dimensional barcode, and the camera The three-dimensional relative position is specified.

  However, in reality, there are many cases where an image other than the two-dimensional barcode is displayed on the image, and in such a situation, it is difficult to specify the relative angle and the relative position. In addition, the distortion is corrected so as to face the two-dimensional barcode image with an angle, and the information of the two-dimensional barcode is merely read, and accurate navigation information is not provided to the user.

  The present invention has been made in view of such a situation, and provides a technique for accurately navigating a user in an environment where sufficient accuracy cannot be expected from a GPS or a gyro sensor.

  In order to achieve the above object, the present invention provides a camera-equipped terminal that reads a marker with a camera and displays navigation information based on the marker on a screen. The camera-equipped terminal calculates an angle between the optical axis of the camera device and the marker, rotates the navigation information on the projection plane of the navigation information by the calculated angle, and captures the image obtained by the camera device. Superimposed display.

  Further features related to the present invention will become apparent from the description of the present specification and the accompanying drawings. The embodiments of the present invention can be achieved and realized by elements and combinations of various elements and the following detailed description and appended claims.

  It should be understood that the description herein is merely exemplary and is not intended to limit the scope of the claims or the application of the invention in any way.

  According to the present invention, a user can be accurately navigated in an environment where sufficient accuracy cannot be expected from a GPS or a gyro sensor.

It is a figure which shows the example of a rough function structure of the navigation information provision system (it can also be called a map relevant-information delivery system) by embodiment of this invention. It is a sketch of an indoor environment as an example using the present invention. It is a figure which shows the data structure example of the navigation information 0300. It is a figure showing the environment which the terminal 0101 with a camera image | photographs. It is a model figure (upper surface) showing the relationship between a to-be-photographed object and the image projected in the environment which a camera image | photographs. It is a model figure showing the relationship between the to-be-photographed image with the to-be-photographed object and the projection information in the environment which a camera image | photographs. It is a figure which shows the example of a superimposed display of the image image | photographed with the camera, and navigation information. It is a flowchart for demonstrating the navigation information display process (process from a navigation information acquisition to a display) in the navigation information provision system 0100 of embodiment of this invention. It is a figure which shows the schematic hardware structural example of the navigation information provision system by embodiment of this invention.

  Embodiments of the present invention are equipped with a display device and read the coordinates of markers and embedded information detected in a captured image for a fixed or mobile terminal device using a camera, The captured image and navigation information are superimposed and displayed in a manner that matches the user's intuition.

  In the navigation information display according to the present embodiment, the three-dimensional relative position between the camera and the marker is not used. The relative angle between the camera and the marker is obtained as the angle between the camera optical axis vector in the camera model and the vector to the marker from the marker position projected on the image and the optical axis position obtained by calibrating in advance. Based on the obtained relative angle, the image taken on the screen and the navigation arrows and guidance are superimposed and displayed at a more appropriate angle to navigate the user to the destination.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be denoted by the same numbers. The attached drawings show specific embodiments and implementation examples based on the principle of the present invention, but these are for understanding the present invention and are not intended to limit the present invention. Not used.

  This embodiment has been described in sufficient detail for those skilled in the art to practice the present invention, but other implementations and configurations are possible without departing from the scope and spirit of the technical idea of the present invention. It is necessary to understand that the configuration and structure can be changed and various elements can be replaced. Therefore, the following description should not be interpreted as being limited to this.

  Furthermore, as will be described later, the embodiment of the present invention may be implemented by software running on a general-purpose computer, or may be implemented by dedicated hardware or a combination of software and hardware.

  In the following description, each information of the present embodiment will be described in the “table” format. However, the information does not necessarily have to be represented by a data structure using a table. It may be expressed in other than. Therefore, “table”, “list”, “DB”, “queue”, etc. may be simply referred to as “information” to indicate that they do not depend on the data structure.

  In addition, when explaining the contents of each information, the expressions “identification information”, “identifier”, “name”, “name”, “ID” can be used, and these can be replaced with each other. It is.

  In the following, each processing in the embodiment of the present invention will be described with each processing unit (image analysis unit, information acquisition unit, display unit, etc.) as a subject (operation subject). Since the processing is performed using the memory and the communication port (communication control device), the description may be made with the processor as the subject. Part or all of the program may be realized by dedicated hardware, or may be modularized. Various programs may be installed in each computer by a program distribution server or a storage medium.

<System function configuration>
FIG. 1 is a diagram illustrating a schematic functional configuration example of a navigation information providing system (also referred to as a map-related information distribution system) according to an embodiment of the present invention.

  The navigation information providing system 0100 includes a camera-equipped terminal 0101, a navigation information distribution server 0102, and a navigation database (DB) 0103. For example, the camera-equipped terminal 0101 and the navigation information distribution server 0102 can communicate with each other via a communication network (network) 0111 to distribute and acquire information. Further, the navigation information distribution server 0102 and the navigation DB 0103 may be directly connected by a cable, or may be connected by another network.

  The camera-equipped terminal 0101 includes an imaging unit 0104, an image analysis unit 0105, an information acquisition unit 0106, and a display unit 0110.

  The navigation information distribution server 0102 includes a distribution request reception unit 0107, an information acquisition unit 0108, and an information distribution unit 0109.

  In the camera-equipped terminal 0101, the photographing unit 0104 includes a camera. A user photographs using a camera attached to the photographing unit 0104 and acquires an image of the object. The image analysis unit 0105 recognizes a two-dimensional barcode from an image obtained by photographing and extracts information included in the two-dimensional barcode. At the same time, the image analysis unit 0105 calculates the center coordinates and optical axis coordinates of the two-dimensional barcode in the image coordinate system, and further calculates the correction angle of the navigation information from the projection model of the camera. The information acquisition unit 0106 inquires of the navigation information distribution server 0102 about information corresponding to the two-dimensional barcode information on the condition of the two-dimensional barcode information obtained by the image analysis unit 0105.

  The distribution request reception unit 0107 of the navigation information distribution server 0102 receives an inquiry from the information acquisition unit 0106 of the camera-equipped terminal 0101. The information acquisition unit 0108 acquires navigation information corresponding to the two-dimensional barcode information and passes it to the information distribution unit 0109. The information distribution unit 0109 provides the navigation information acquired from the information acquisition unit 0108 to the inquiring camera-equipped terminal 0101.

  The camera-equipped terminal 0101 that has acquired the navigation information corresponding to the two-dimensional barcode information uses the display unit 0110 to superimpose and display the image captured by the imaging unit 0104 and the navigation information. At this time, the label or arrow of the navigation information to be superimposed is rotated by the navigation information correction angle.

  A configuration in which the navigation information distribution server 0102 and the navigation DB 0103 are arranged in the camera-equipped terminal 0101 without using a communication network can be realized. Further, the camera-equipped terminal 0101 may include only the photographing unit 0104, the information acquisition unit 0106, and the display unit 0110, and all the processing executed by the image analysis unit 0105 may be executed on the navigation information distribution server 0102 side. .

<System hardware configuration>
FIG. 9 is a diagram showing a schematic hardware configuration example of the navigation information providing system according to the embodiment of the present invention.

  As shown in FIG. 9, the camera-equipped terminal 0101 and the navigation information distribution server 0102 can be configured by a general computer, for example.

  The camera-equipped terminal 0101 includes, for example, a CPU (also simply referred to as a processor) 0901 for executing various programs, a camera device 0902, an input device 0903 for a user to input instructions and information, and processing by the CPU An output device 0904 for outputting results, a storage device 0905 for storing various programs and processing results, and temporarily storing information in the middle of processing, and communication for communicating with the navigation information distribution server 0102 and the like Device 0906.

  The navigation information distribution server 0102 outputs, for example, a CPU (also simply referred to as a processor) 0911 that executes various programs, an input device 0912 for a server administrator to input instructions and information, and processing results by the CPU An output device 0913 for storing, a storage device 0914 for storing various programs and processing results, temporarily storing information in the middle of processing, a communication device 0915 for communicating with the camera-equipped terminal 0101, and the like, have.

  The input devices 0903 and 0912 include, for example, various buttons, a keyboard, a touch panel, a microphone, and the like. The output devices 0904 and 0913 include, for example, a display that displays images and character strings, a speaker that outputs sound, and the like. The storage devices 0905 and 0914 are configured by, for example, a ROM or a RAM.

  The CPU 0901 of the camera-equipped terminal 0101 includes an image capturing unit 09011 that is a program for controlling image capturing processing by the camera device, and an image analysis unit 09012 that is a program for analyzing image data and information processed by the image capturing unit 09011. An information acquisition unit 09013 that is a program for acquiring information (navigation information) from the navigation information distribution server 0102, and a display unit 09014 that is a program for displaying image data and navigation information on the output device 0904 in a superimposed manner. Run.

  The CPU 0911 of the navigation information distribution server 0102 receives a distribution request reception unit 09111 that is a program for receiving a navigation information distribution request from the camera-equipped terminal 0101 and a two-dimensional barcode ID ( Information acquisition unit 09112 which is a program for acquiring navigation information corresponding to CodeID (see FIG. 3) from the navigation DB 0103, and an information distribution unit which is a program for transmitting the obtained navigation information to the camera-equipped terminal 0101 09113 are executed.

  FIG. 9 shows a state in which various programs are expanded in the internal memory of the CPU. Before execution, various programs are stored in, for example, the storage device 0905 or 0914, and are read and executed by the CPU 0901 or 0911 at the time of execution.

<Use scene>
FIG. 2 is a diagram illustrating a scene example (indoor environment) where navigation information is provided by the navigation information providing system 0100 according to the embodiment of the present invention.

  The navigation information is provided in a space configured by, for example, a wall front surface 0201, a wall left surface 0202, and a wall right surface 0203. A scene is assumed in which the user 0204 is placed at the center of the space. There is a passage between each wall. Each passage leads to conference room A, office room D0205, conference room B0206, and entrance C0207, respectively.

  It is assumed that the user is destined for the office room D at the end of the passage of the conference room A, for example. On the wall front 0201, a two-dimensional barcode 0208 is arranged at a height approximately equal to the user's line of sight. Then, the user takes an image of the two-dimensional barcode 0208, and the navigation information is superimposed and displayed on the screen of the camera-equipped terminal 0101 based on the two-dimensional barcode information acquired thereby.

<Example data structure of navigation information>
FIG. 3 is a diagram showing a data structure example of the navigation information 0300 in the embodiment of the present invention. The navigation information 0300 has a table structure, NavigationID 0301 for uniquely identifying and recognizing navigation information, and CodeID 0302 for uniquely identifying and recognizing a two-dimensional barcode (information read from the two-dimensional code). ), A Label 0303 indicating a character string superimposed on the photographed image, and an arrow angle (direction of a location indicated by Label 0303) superimposed on the photographed image when the camera faces the two-dimensional barcode. Angle 0304 is included as a configuration item.

  The information acquisition unit 0108 of the navigation information distribution server 0102 acquires corresponding navigation information from the navigation DB 0103 using CodeID 0302 as a key. That is, information of Label 0303 and Angle 0304 corresponding to Navigation ID 0301 including Code ID 0302 is acquired. In the example of FIG. 3, the two-dimensional barcode with CodeID “1” corresponds to three pieces of navigation information.

<Environment where the camera-equipped device shoots>
FIG. 4 is a diagram illustrating an environment in which the camera-equipped terminal 0101 shoots. The camera-equipped terminal 0101 has, for example, a camera on the opposite surface (back surface) to the display screen, and is configured to be able to display an image being shot on the screen as a preview.

  When using the camera-equipped terminal 0101, the camera included in the imaging unit 0104 captures a wall surface (front wall) 0401 and a two-dimensional barcode 0402. Then, the camera-equipped terminal 0101 displays the shooting range 0403 on the screen of the display unit 0110. In FIG. 4, a coordinate 0404 indicates a camera optical axis obtained in advance by camera calibration.

  FIG. 4 illustrates a scene in which the optical axis of the camera and the normal vector from the wall surface 0401 to be photographed do not coincide with each other and form a certain angle.

<Relationship between subject and projected image>
FIG. 5 is a model diagram (upper surface) showing a relationship between a subject and a projected image under an environment where the camera captures an image.

  There is a camera 0501, and an optical axis 0502 extends from the camera 0501. The range 0506 of the image projection plane 0503 is determined from the angle of view 0504 of the camera lens and the focal length 0505 of the lens.

  The relative angle 0509 between the two-dimensional barcode 0507 and the optical axis 0502 of the camera is obtained from the distance 0508 between the coordinate 0510 and the optical axis obtained by projecting the two-dimensional barcode 0507 onto the projection plane 0503, and the focal length 0505. The relative angle 0509 is expressed clockwise with the direction of the optical axis 0502 being 0 °, as viewed from above.

<Relationship between subject and projected image and superimposed navigation information>
FIG. 6 is a model diagram showing a relationship between a subject, a projected image, and navigation information to be superimposed under an environment where the camera captures an image. FIG. 6 corresponds to the model of FIG.

  There is a camera 0601, and an optical axis 0602 extends from the camera 0601. In the state shown by the model in FIG. 6, an angle 0605 is formed between the vector 0604 from the camera 0601 toward the two-dimensional barcode 0603 and the optical axis 0602.

  In FIG. 6, a projection plane 0606 for a captured image and a projection plane 0608 for projecting navigation information 0607 with a vector 0604 as a normal vector are defined. That is, the projection surface 0608 does not indicate a wall surface on which the two-dimensional barcode 0603 is arranged, but indicates a virtual surface on which the navigation information 0607 composed of labels and arrows is projected.

  If the projection plane 0608 passes through the intersection 0610 of the projection plane 0606 and the optical axis 0602, the angle 0609 between the projection plane 0606 and the projection plane 0608 is equal to the angle 0605. That is, the geometrical relationship seen from the upper surface of the projection surface 0606 and the projection surface 0608 is known. Note that the geometric relationship similar to that in FIG.

<Example of superimposed display of navigation information>
FIG. 7 is a diagram illustrating a superimposed display example of an image captured by a camera and navigation information. As shown in FIG. 7, the user is photographing the wall front 0701 and the two-dimensional barcode 0702 with the camera included in the photographing unit 0104. At that time, the camera-equipped terminal 0101 displays the shooting range 0703 on the screen of the display unit 0110. A coordinate 0704 indicates a camera optical axis obtained in advance by camera calibration.

  The navigation information 0705 is composed of a label and an arrow, and is displayed as a pair on the screen. The direction of the arrow is 0 ° upward and clockwise. When the navigation information obtained using the two-dimensional barcode as a key is, for example, “conference room A” as the label and 90 ° as the angle (direction), the projection plane 0608 and the plane 0606 (see FIG. 6) Navigation information before the angle 0609 is reflected is indicated by an arrow 0706. An angle 0609 between the projection plane 0608 and the plane 0606 is further added to the direction of the arrow 0706, and navigation information 0705 reflecting the angle is displayed. In this example, since the angle between the projection plane 0608 and the plane 0606 is counterclockwise, subtraction is performed.

  As a result of superimposing the screen, the image is an image obtained by photographing the horizontal direction, but the navigation information represents the vertical direction. For this reason, the display of the navigation information may be uncomfortable for explanation, but it is easy for the user to understand intuitively.

<Contents of navigation information display processing>
FIG. 8 is a flowchart for explaining navigation information display processing (processing from acquisition of navigation information to display) in the navigation information providing system 0100 according to the embodiment of the present invention.

(I) Step 0801
First, when a user photographs a wall surface, the photographing unit 0104 of the camera-equipped terminal 0101 acquires an image i.

(Ii) Step 0802
The image analysis unit 0105 detects a two-dimensional barcode from the image i. At this time, it is assumed that a plurality of two-dimensional barcodes are detected. Also, the two-dimensional barcode to be used is assumed to be a color code that can be photographed and recognized in a plurality of distances, and can be detected by adopting an existing detection method.

(Iii) Step 0803
The image analysis unit 0105 determines whether a two-dimensional barcode is included in the image i based on the processing result of step 0802. If the image i does not include a two-dimensional barcode (No in step 0803), the process proceeds to step 0804. If the image i includes a two-dimensional barcode (Yes in step 0803), the process proceeds to step 0805.

(Iv) Step 0804
The display unit 0110 displays only the image i on the screen and ends the navigation information display process.

(V) Step 0805
The image analysis unit 0105 sets the processing target to the one having the largest area among a plurality of detected two-dimensional barcodes. Which two-dimensional barcode has the maximum area is calculated by using the coordinates of the four corners obtained by the detection method using the color code that is the existing detection method and the library method. And can be determined by comparison.

(Vi) Step 0806
The image analysis unit 0105 reads a processing target two-dimensional barcode and extracts an ID (CodeID) that is two-dimensional barcode information. At the same time, the image analysis unit 0105 calculates center coordinates from the coordinates of the four corners of the two-dimensional barcode to be processed.

(Vii) Step 0807
The image analysis unit 0105 calculates an angle between the camera optical axis and the processing target two-dimensional barcode based on the geometrical relationship shown in FIGS.

(Viii) Step 0808
The information acquisition unit 0106 acquires the ID (CodeID) of the processing target two-dimensional barcode from the image analysis unit 0105, and inquires the navigation information corresponding to the navigation information distribution server 0102 (navigation information distribution request) using it as a key. . The information acquisition unit 0108 of the navigation information distribution server 0102 acquires a navigation information distribution request (including CodeID) via the distribution request reception unit 0107, and checks whether there is corresponding navigation information (see FIG. 3) from the navigation DB 0103. If it exists, get it. Then, the information distribution unit 0109 of the navigation information distribution server 0102 transmits navigation information corresponding to the processing target two-dimensional barcode to the camera-equipped terminal 0101. If there is no corresponding navigation information, this is notified to the camera-equipped terminal 0101.

(Ix) Step 0809
The information acquisition unit 0106 of the camera-equipped terminal 0101 confirms the number of acquired navigation information based on the information received from the navigation information distribution server 0102. If the number of navigation information is 0 (No in step 0809), the process proceeds to step 0804. If the number of navigation information is one or more (Yes in step 0809), the process proceeds to step 0810.

(X) Step 0810
As shown in FIG. 7, the display unit 0110 superimposes and displays the image i captured by the camera and the navigation information.

<Summary>
(I) In this embodiment, when the navigation information is superimposed on the captured image, the angle between the optical axis of the camera device and a marker (for example, a two-dimensional barcode) is calculated, and navigation is performed on the projection plane of the navigation information. The information is rotated by the angle. By doing so, it is possible to give an intuitive direction suggestion that does not give the user a sense of incongruity simply by reading the information of the marker (two-dimensional barcode).

  More specifically, the camera-equipped terminal acquires marker identification information from the marker, transmits the marker identification information to the navigation information distribution server, and acquires navigation information corresponding to the marker identification information from the server. By doing in this way, it is not necessary to give much information to the marker, and it is not necessary to provide a database of navigation information in the camera-equipped terminal, so that the processing can be performed quickly. However, this does not deny that the camera-equipped terminal has navigation information. If it is a camera-equipped terminal having a processing capacity and a large storage device capacity, there is no inconvenience for the user even if the navigation DB is provided.

  When calculating the angle, the angle between the center coordinate of the marker and the vector between the camera device and the optical axis of the camera device is calculated. By doing in this way, an angle can be easily known using a geometric relationship. This angle corresponds to the angle between the projection plane of the image obtained by photographing with the camera device and the projection plane of the navigation information.

  In the present embodiment, the captured image is allowed to include a plurality of markers. In this case, the marker having the maximum area is extracted from the image as the marker to be processed. By doing in this way, even if a some marker is detected, the navigation information which should be superimposed-displayed easily can be acquired now. Although different from the present embodiment, all navigation information may be displayed in the vicinity of a plurality of photographed markers so as not to overlap each other. At this time, using the area ratio of the corresponding marker, the navigation information may be displayed by weighting the transparency and the label size.

(Ii) The present invention can also be realized by software program codes that implement the functions of the embodiments. In this case, a storage medium in which the program code is recorded is provided to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention. As a storage medium for supplying such program code, for example, a flexible disk, CD-ROM, DVD-ROM, hard disk, optical disk, magneto-optical disk, CD-R, magnetic tape, nonvolatile memory card, ROM Etc. are used.

  Also, based on the instruction of the program code, an OS (operating system) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. May be. Further, after the program code read from the storage medium is written in the memory on the computer, the computer CPU or the like performs part or all of the actual processing based on the instruction of the program code. Thus, the functions of the above-described embodiments may be realized.

  Further, by distributing the program code of the software that realizes the functions of the embodiment via a network, it is stored in a storage means such as a hard disk or memory of a system or apparatus, or a storage medium such as a CD-RW or CD-R And the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage means or the storage medium when used.

  Finally, it should be understood that the processes and techniques described herein are not inherently related to any particular apparatus, and can be implemented by any suitable combination of components. In addition, various types of devices for general purpose can be used in accordance with the teachings described herein. It may prove useful to build a dedicated device to perform the method steps described herein. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined. Although the present invention has been described with reference to specific examples, these are in all respects illustrative rather than restrictive. Those skilled in the art will appreciate that there are numerous combinations of hardware, software, and firmware that are suitable for implementing the present invention. For example, the described software can be implemented in a wide range of programs or script languages such as assembler, C / C ++, perl, shell, PHP, Java (registered trademark).

  Furthermore, in the above-described embodiment, control lines and information lines are those that are considered necessary for explanation, and not all control lines and information lines on the product are necessarily shown. All the components may be connected to each other.

  In addition, other implementations of the invention will be apparent to those skilled in the art from consideration of the specification and embodiments of the invention disclosed herein. The specification and specific examples are merely exemplary, and the scope and spirit of the invention are indicated in the following claims.

0101 Terminal with camera, 0102 Navigation information distribution server, 0103 Navigation DB, 0104 Imaging section, 0105 Image analysis section, 0106 Information acquisition section, 0107 Distribution request reception section, 0108 Information acquisition section, 0109 Information distribution section, 0110 Display section, 0201 Wall Front, 0202 Wall Left, 0203 Wall Right, 0204 User, 0205 Navigation Information Right, 0206 Navigation Information Left, 0207 Navigation Information Back, 0208 2D Barcode, 0300 Navigation Information, 0301 Navigation ID, 0302 2D Barcode ID , 0303 Navigation label, 0304 Navigation angle, 0401 Front of wall, 0402 Two-dimensional barcode, 0403 Shooting range, 0404 coordinates (Optical axis position), 0501 camera, 0502 optical axis (vector), 0503 plane (projection plane), 0504 angle of view, 0505 focal length, 0506 distance from optical axis to shooting boundary (range), 0507 2D barcode, 0508 Distance from optical axis to projected 2D barcode, 0509 relative angle (angle from optical axis to 2D barcode), 0601 camera, 0602 optical axis (vector), 0603 2D barcode, 0604 2 from camera Vector to dimensional barcode, 0605 Angle between optical axis vector and vector to two-dimensional barcode, 0606 plane (projection plane of captured image), 0607 navigation information, 0608 projection plane (navigation information projection reference plane), 0609 projection Angle between plane and navigation information projection reference plane, 0701 Surface, 0702 2D barcode, 0703 shooting range, 0704 coordinates (optical axis position), 0705 navigation information (navigation label and arrow)

Claims (10)

A camera-equipped terminal that reads a marker with a camera and displays navigation information based on the marker on a screen,
A camera device,
A storage device for storing various programs;
The various programs are read from the storage device, and processing for displaying the navigation information based on the marker on the screen is superimposed on an image obtained by photographing with the camera device based on the various programs. And a processor to
The processor is
A process of calculating an angle between the optical axis of the camera device and the marker;
Processing for rotating the navigation information on the projection surface of the navigation information by the calculated angle and displaying it superimposed on an image obtained by photographing with the camera device;
Run the camera terminal. In claim 1,
The processor further executes processing for acquiring marker identification information from the marker, transmitting the marker identification information to an external server, and acquiring the navigation information corresponding to the marker identification information from the external server. , A camera-equipped device. In claim 1,
In the process of calculating the angle, the processor is a camera-equipped terminal that calculates the angle using a center coordinate of the marker, a vector between the camera devices, and an optical axis of the camera device. In claim 3,
A camera-equipped terminal, wherein a projection plane of an image photographed by the camera device and a projection plane of the navigation information form the calculated angle. A camera-equipped terminal according to claim 1;
A navigation information distribution server,
The camera-equipped terminal further includes a communication device,
The processor of the camera-equipped terminal acquires marker identification information from the marker, transmits the marker identification information to the navigation information distribution server, and transmits the navigation information corresponding to the marker identification information from the navigation information distribution server. Execute the process to get,
The navigation information delivery server processor comprises:
Processing for obtaining the navigation information corresponding to the marker identification information from a navigation database;
Processing for transmitting the acquired navigation information to the camera-equipped terminal;
A navigation information provision system that executes A navigation information display method for reading a marker with a camera and displaying navigation information based on the marker on a screen,
The processor of the camera-equipped terminal extracts the marker from the image captured by the camera device;
The processor calculates an angle between the optical axis of the camera device and the marker;
The processor obtains the navigation information associated with the marker;
The processor rotates the navigation information on the projection surface of the navigation information by the calculated angle and superimposes it on an image obtained by photographing with the camera device;
Including navigation information display method. In claim 6,
In extracting the marker, the processor extracts a marker having the maximum area from the image as a marker to be processed, and displays the navigation information. In claim 6,
In acquiring the navigation information, the processor acquires marker identification information from the marker, transmits the marker identification information to an external server, and the navigation information corresponding to the marker identification information from the external server. To get the navigation information display method. In claim 6,
The navigation information display method, wherein in calculating the angle, the processor calculates the angle using a center coordinate of the marker, a vector between the camera devices, and an optical axis of the camera device. In claim 9,
A navigation information display method, wherein a projection plane of an image photographed by the camera device and a projection plane of the navigation information form the calculated angle.

Images