JP2011191892A - Image display system, mobile information terminal and image display program - Google Patents

Abstract

【Task】
Realize indoor air tag service with low introduction cost and high location accuracy.
[Solution]
Provided in the portable terminal 1 and equipment master data 13 storing information of an image indicating a predetermined range and display information in which a position in the predetermined range and information to be displayed corresponding to the position are associated with each other The camera 3, the direction sensor 5 that detects the direction in which the camera 3 is capturing, and the image captured by the camera 3 are displayed on the display 4 as captured images, and the current position information input by the user is acquired. And recognizing a range of the captured image in combination with the detected direction, and displaying information corresponding to a position included in the recognized range of the display information superimposed on a corresponding position of the captured image. And
[Selection] Figure 1

Description

  The present invention relates to an image display system, a portable information terminal, and an image display program, and more particularly to a system that displays information associated with position information superimposed on a captured image.

  In recent years, an application service called Sekai Camera (registered trademark) has been provided in portable communication terminals. In this application service, a current position is specified using a positioning system mounted on a portable communication terminal. Further, the direction of the camera mounted on the portable communication terminal is recognized by the function of the electronic compass mounted on the portable communication terminal, thereby specifying the range of the image currently displayed on the camera.

  The user of this application service can specify a position within the range displayed on the camera and paste character information called “air tag” at the position. The air tag information is stored on the server via the network. The air tag information stored on the server includes character information pasted by the user and information on a position where the character information is pasted. Thereby, when another user uses the above application service to copy the same position to the camera, the character information of the air tag already stored on the server is displayed.

  As a positioning technique, for example, a position information specifying member that is regularly and integrally mounted on a sheet-like member or a mesh-like member can be easily installed and is subject to radio interference both outdoors and indoors. There has been proposed a position information specifying member that can accurately specify position information (see, for example, Patent Document 1).

  In addition, as a technique for displaying information by adding an imaging position and an imaging direction to a captured image, a technique for displaying information on a camera image from the latitude, longitude, and orientation of the camera has been proposed (for example, a patent). Reference 2).

JP 2007-155383 A JP 2000-512781 A Patent No. 4294093 Specification

  In the application service as described above (hereinafter referred to as an air tag service), it is necessary to recognize the current position of the terminal as described above, and at present, GPS (Global Positioning System) is used as a positioning system. For example, since an object to which an air tag is attached outdoors is a relatively large object such as a building, an error of about several meters is allowed. Therefore, in the case where an air tag is displayed outdoors by designating the position of a building or the like, position accuracy is not a problem in positioning by GPS.

  However, because the target to which the air tag is attached indoors is relatively small, such as equipment and fixtures in the building, if there is an error of several meters, the position where the air tag is displayed is the intended position, that is, the equipment And the position of equipment will be greatly deviated. In other words, the indoor room size is generally several meters to several tens of meters on one side, and the indoor position information system has higher positioning accuracy (tens of centimeters) than the outdoor position information system. Realization becomes a challenge.

  The positioning accuracy of GPS used for outdoor positioning has an error of several meters to several tens of meters, and the error further increases indoors. Therefore, it is a reality to use GPS for positioning in a system that handles indoor location information. Not right.

  As an indoor positioning system, there may be IMES (Indoor Messaging System), wireless LAN (Local Area Network), RFID (Radio Frequency IDentification), and the like in addition to GPS. However, in any case, installation of equipment for positioning is required, and the installation cost and operation cost increase in proportion to the number and size of rooms in which the equipment is installed. In addition, it is not always easy to realize the positional accuracy as described above.

  An object of the present invention is to realize an air tag service with low introduction cost and high positional accuracy.

  In order to solve the above-described problem, one aspect of the present invention is an image display system that displays other information superimposed on a captured image, and stores a range image that stores image information indicating a predetermined range An information storage unit, a display information storage unit storing display information in which a position in the predetermined range and information to be displayed corresponding to the position are associated, and imaging provided in a portable information terminal A current position information acquisition unit that acquires current position information of the information terminal input by the user, a direction detection unit that detects a direction in which the imaging unit is capturing, and an image captured by the imaging unit A display processing unit for displaying an image on the display unit as a captured image, wherein the display processing unit recognizes a range of the captured image based on the acquired current position information and the detected direction, And displaying the information to be displayed corresponding to a position included in the recognition range of the display information being 憶 superimposed on the corresponding position of the captured image.

  Further, another aspect of the present invention is a portable information terminal that displays other information superimposed on the captured image, the imaging unit that captures the image, and the information input in response to an operation by the user A current position information acquisition unit that acquires current position information of the terminal, and a display information storage unit that stores display information in which a position in the predetermined range and information to be displayed corresponding to the position are stored A display information acquisition unit that acquires display information, a direction detection unit that detects a direction in which the imaging unit is imaging, and a display processing unit that displays an image captured by the imaging unit on the display unit as a captured image; The display processing unit recognizes a range of the captured image based on the acquired current position information and the detected direction, and includes the acquired display information in the recognized range. The information to be displayed corresponding to the position superimposed on the corresponding position of the captured image and displaying that.

  According to still another aspect of the present invention, there is provided an image display program for a portable information terminal that displays other information superimposed on a captured image, the current information terminal being input in response to an operation by a user. Obtaining position information; obtaining the display information from a display information storage unit storing display information in which a position in the predetermined range and information to be displayed corresponding to the position are associated; Obtaining information indicating the direction in which the imaging unit provided in the portable information terminal is capturing, and indicating the direction detected by the direction detecting unit provided in the portable information terminal; The step of causing the display unit to display an image captured by the imaging unit as a captured image, and the imaging based on the acquired current position information and the detected direction A step of recognizing a range of an image, and a step of superimposing and displaying information to be displayed corresponding to a position included in the recognized range of the acquired display information on a corresponding position of the captured image; Is executed by the portable information terminal.

  By using the present invention, an air tag service with low introduction cost and high positional accuracy can be realized.

It is a block diagram which shows the operation | use form of the image processing system which concerns on embodiment of this invention. It is a figure showing an example of information contained in equipment master data concerning an embodiment of the present invention. It is a flowchart which shows operation | movement of the portable terminal in the image processing system which concerns on embodiment of this invention. It is a figure which shows the example in which the portable terminal operator which concerns on embodiment of this invention is located in the entrance / exit vicinity of a room. It is a figure which shows the example of a display of a portable terminal position input means when the portable terminal operator which concerns on embodiment of this invention is located in the doorway vicinity. It is a figure which shows the example of a display of a portable terminal when the portable terminal operator which concerns on embodiment of this invention is located in the doorway vicinity. It is a figure which shows the example which the portable terminal operator which concerns on embodiment of this invention moved to room vicinity vicinity. It is a figure which shows the example of a display of a portable terminal when the portable terminal operator which concerns on embodiment of this invention moves to room center vicinity. It is a figure which shows the example of a display of a portable terminal position input means when the portable terminal operator which concerns on embodiment of this invention moves to room center vicinity. It is a figure which shows the example of a display of a portable terminal when the portable terminal operator which concerns on embodiment of this invention inputs the correct position with a portable terminal position input means, when moving to the center vicinity of a room. It is a figure which shows the hardware constitutions of the information processing terminal which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, an air tag service that displays prestored facility name data at a set position in a building will be described as an example.

  FIG. 1 is a diagram showing an operation mode of an air tag service according to an embodiment of the present invention. As shown in FIG. 1, the air tag service according to the present embodiment is realized by wireless communication between the mobile terminal 1 and the facility management server 2. The portable terminal 1 and the facility management server 2 operate as an image display system that displays captured information and air tag information. In the air tag service according to the present embodiment, the facility management server 2 provides an air tag service in one building.

  As shown in FIG. 1, the equipment management server 2 holds equipment master data 13 for equipment installed indoors. The equipment master data management means 12 performs management such as registration / deletion / update of the equipment master data 13. The equipment management server communication means 11 transmits equipment master data of the room where the mobile terminal 1 is located to the mobile terminal 1. The equipment master data includes room shape data, equipment position, name, and the like.

  The information stored in the equipment master data 13 will be described with reference to FIG. As shown in FIG. 2, the equipment master data 13 according to the present embodiment includes building data, room data, and equipment data. The building data is information about a building for which the air tag service is provided, and includes a floor ID, floor image information, and building position information, as shown in FIG.

  The floor ID is identification information for identifying each floor constituting the building. As shown in FIG. 2, the floor image information is an image of a floor map of each floor constituting the building. The building position information is information on the position where the building is built, and in this embodiment, is information corresponding to a position signal by GPS.

  The room data is information on rooms existing on each floor constituting the building, and includes a room ID, room image information, and room position information as shown in FIG. The room ID is identification information for identifying each room. The room image information is a plane image of each room as shown in FIG. The room position information is information on the position where the room is arranged on each floor of the building. In the present embodiment, the room position information is information corresponding to the coordinates in the floor image information included in the building data.

  As described above, in the present embodiment, the facility master data 13 functions as a range image information storage unit that stores image information indicating a room as a predetermined range. The equipment master data 13 also functions as a floor image information storage unit that stores information on floor images in the building.

  The equipment data is information on equipment arranged in each room, and includes equipment ID, room ID, equipment name, and equipment location information as shown in FIG. The facility ID is identification information for identifying each facility. The room ID is identification information for identifying a room in which each facility is installed, and corresponds to the room ID included in the room data.

  The facility name is character information indicating the name of each facility. In this embodiment, the character information of the facility name is displayed as an air tag. The equipment position is information on a position where each equipment is installed, and in the present embodiment, the equipment position is information corresponding to coordinates in the room image information included in the room data. An air tag is displayed at a position specified by the equipment position.

  As described above, the equipment master data 13 stores display information in which the above-described room, that is, a position in a predetermined range, and information on the equipment name, which is information to be displayed corresponding to the position, are associated. It functions as a display information storage unit.

  In the mobile terminal 1, the camera 3 as an imaging unit converts optical information into an electrical signal and generates image information. The direction sensor 5 detects the direction in which the camera 3 of the mobile terminal 1 is facing and outputs a detection signal to the facility data display means 6. Note that the direction sensor 5 can detect the vertical direction by an acceleration sensor or the like in addition to the east, west, north, and south directions. The mobile terminal position input means 7 is a user interface through which the user inputs information to the mobile terminal 1. In this embodiment, the user uses the mobile terminal position input means 7 to display information on the position where the user is standing. Input through.

  The portable terminal communication means 8 receives the equipment master data transmitted by the equipment management server communication means 11. The equipment data management means 9 stores the equipment master data received by the mobile terminal communication means 8 as equipment data 10. The equipment data management means 9 inputs information stored as equipment data to the equipment data display means 6.

  The equipment data display means 6 recognizes the current position of the mobile terminal 1 based on the information input via the mobile terminal position input means 7. Furthermore, the facility data display means 6 recognizes the direction in which the camera 3 is facing based on the detection signal from the direction sensor 5 and, based on the relationship with the recognized current position, the range currently captured by the camera 3 (hereinafter referred to as “the camera 3”). The imaging range). That is, the direction sensor 5 functions as a direction detection unit.

  The equipment data display means 6 acquires the air tag to be displayed in the determined imaging range, that is, the character information of the equipment name from the equipment data management means 9, and based on the information on the position at which the air tag is to be displayed, that is, the equipment position information. The air tag is superimposed on the image information generated by the camera 3 and displayed on the display 4. The display 4 functions as a display unit that displays an image, and also functions as an input unit through which a user inputs information as a touch panel.

  Next, the operation of the air tag service according to the present embodiment will be described with reference to FIG. As shown in FIG. 3, first, the mobile terminal communication means 8 receives equipment master data from the equipment management server 2 based on the control of the equipment data management means 9. The equipment data management means 0 stores the received equipment master data as equipment data 10 (S301). That is, the facility data management means 9 functions as a display information acquisition unit. Next, the equipment data display means 6 acquires floor data from the information included in the equipment data 10 from the equipment data management means 9, and displays the floor image on the display 4 (S302).

  The floor image shown in FIG. 2 is displayed in S302. On this screen, the user selects one of the rooms included in the floor. The user selects the room by touching the room on the displayed screen, and the coordinate information on the screen is input. After the conversion process between the coordinate information on the screen and the coordinate information on the floor image, It is recognized by the correspondence with the coordinate information in the room data.

  When a room is selected by the user (S303), the equipment data display means 6 acquires the room data of the room selected from the equipment data management means 9 and the equipment data of equipment included in the room (S304). Then, the facility data display means 6 displays an image of the selected room on the display 4 (S305). The room image displayed in S305 is a plan view of the room shown in FIG.

  When an image of the room is displayed in S305, the user inputs the current position via the portable terminal position input means 7 (S306). In S306, for example, when the user is standing at the position shown in FIG. 4, the position indicated by the star mark in FIG. 5 is input as the current position of the user. This input is performed when the user touches the position to be designated, that is, the position of the star mark on the screen of the room displayed as shown in FIG. It is recognized as a coordinate on the image by a conversion process with the coordinate information of the image information.

  That is, the facility data display means 6 functions as current position information acquisition means for acquiring current position information of the mobile terminal 1 in accordance with an operation on the displayed room image. As described above, in this embodiment, the display 4 configured as a touch panel is used to display a plan view of the room on the display 4, and then the current position information portable terminal 1 displays the current position information by touching the screen. Since it is acquired, an intuitive operation is possible, and it is possible to acquire current position information easily and with high accuracy.

  When the position information is input by the process of S306, the facility data display means 6 displays the image information generated by the camera 3 on the display 4, and the direction information detected by the direction sensor 5, the portable terminal position input. Based on the position information input via the means 7, the position and range of the image displayed on the display 4 are recognized.

  And the equipment data display means 6 judges the position which should display each equipment name as an air tag based on the equipment position of the equipment data acquired in S304, and it superimposes on the image information produced | generated by the said camera 3, and an air tag Is displayed (S307). That is, the facility data display means 6 functions as a display processing unit. FIG. 6 shows an example of the screen display in S307 when the camera 3 is facing the back side of the room in the state of FIG.

  As shown in FIG. 6, the text information of the equipment names of equipment A and equipment B is displayed superimposed on the image captured by the camera 3 as an air tag. In the process of S307, the equipment data display means 6 monitors the detection signal from the direction sensor 5 in real time, and when the detection signal from the direction sensor 5 changes, the display position of the air tag changes according to the change. Let

  If the end of the application is instructed after the process of S307 (S308 / YES), the facility data display means 6 stops the display on the display 4 and ends the process. On the other hand, when the termination of the application is not instructed (S308 / NO), the processing from S305 is repeated. Here, processing when the user moves from the state of FIG. 4 to the state of FIG. 7 will be described.

  When the user moves from the state of FIG. 4 to the state of FIG. 7 and continues to image in the same direction as the state of FIG. 6, the image displayed on the display 4 is the actual equipment and equipment as shown in FIG. It will be in the state which shifted | deviated from the air tag which displayed the name. This is because the position information already input is not updated from the state of FIG.

  On the other hand, the user operates the portable terminal 1 to display the room data again on the display 4 (S305), and designates and inputs his / her position after the movement as shown in FIG. 9 (S306). . Thereby, the installation data display means 6 updates the display position of installation data, and displays installation data as an air tag by the updated display position (S307).

  FIG. 10 shows a screen display example on the display 4 after the update. As shown in FIG. 10, by inputting the correct position information, the display position of the equipment data is updated, and the state where the actual equipment and the air tag displaying the equipment name are shifted is eliminated.

  The mobile terminal 1 and the facility management server 2 are realized by an information processing terminal. With reference to FIG. 11, a hardware configuration of an information processing terminal that realizes the mobile terminal 1 and the facility management server 2 according to the present embodiment will be described. FIG. 11 is a block diagram showing a hardware configuration of the information processing terminal according to the present embodiment. As shown in FIG. 11, the information processing terminal according to the present embodiment has the same configuration as a general server, a PC (Personal Computer), or the like.

  That is, the information processing terminal according to the present embodiment includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, an HDD (Hard Disk Drive) 104, and an I / F 105. Connected through. Further, an LCD (Liquid Crystal Display) 106 and an operation unit 107 are connected to the I / F 105.

  The CPU 101 is a calculation unit and controls the operation of the entire information processing terminal. The RAM 102 is a volatile storage medium capable of reading and writing information at high speed, and is used as a work area when the CPU 101 processes information. The ROM 103 is a read-only nonvolatile storage medium and stores a program such as firmware.

  The HDD 104 is a nonvolatile storage medium that can read and write information, and stores an OS (Operating System), various control programs, application programs, and the like. The I / F 105 connects and controls the bus 108 and various hardware and networks. In addition, it connects with the portable terminal 1 in I / F105 of other devices, such as the camera 3 and the direction sensor 5. FIG. The LCD 106 is a visual user interface for the user to check the state of the information processing terminal. The operation unit 107 is a user interface such as a keyboard and a mouse for the user to input information to the information processing terminal.

  In such a hardware configuration, a program stored in a recording medium such as the ROM 103, the HDD 104, or an optical disk (not shown) is read into the RAM 103, and the CPU 101 performs an operation according to the program, thereby configuring a software control unit. . The function of the information processing terminal according to the present embodiment is realized by a combination of the software control unit configured as described above and hardware.

  Functions realized as the software control unit include equipment data display means 6 in the portable terminal 1, portable terminal communication means 8, equipment data management means 9, equipment management server communication means 11 in the equipment management server 2, equipment master data. The management means 12 is mentioned. The equipment master data 13 is realized by a non-volatile storage medium such as the HDD 104. Further, the facility data 10 is realized in a storage area secured in a part of the RAM 102 in addition to the nonvolatile storage medium.

  As described above, in the present embodiment, when providing an indoor air tag service, the current position of the mobile terminal 1 is acquired by causing the user to input the current position on the floor or room, thereby reducing the introduction cost. An indoor air tag service with low and high positional accuracy can be realized.

  In the above embodiment, as shown in FIG. 3, the case where all the equipment master data 13 is acquired from the equipment management server 2 at the start of the air tag service has been described as an example. In addition, in S301, only building data is acquired among the information shown in FIG. 2, and corresponding equipment data and room data may be acquired in S304. Thereby, the amount of data downloaded to the mobile terminal 1 can be reduced, and the storage capacity and communication load on the mobile terminal 1 can be reduced.

  Moreover, in the said embodiment, the equipment management server 2 demonstrated as an example the case where the equipment master data about one building is managed. In addition, one equipment management server 2 may manage equipment master data of a plurality of buildings. In this case, the user selects one from a plurality of buildings managed by the facility management server 2. At this time, if the mobile terminal 1 is equipped with a GPS, the building data to be acquired can be determined by comparing the current position of the mobile terminal 1 with the building position information of the building data shown in FIG. It is.

  Moreover, in the said embodiment, it demonstrated as an example the system which presupposed existence of the equipment master data 13 constructed | assembled previously. In addition, when a user designates a position on an image displayed on the display 4 of the portable terminal 1 and inputs character information, information can be added to the equipment master data 13 and information can be updated. it can.

  The above embodiment is applied not only to a system for displaying an equipment name as described above as an air tag but also as a system capable of displaying an arbitrary character string associated with position information superimposed on a real-time captured image. It is also possible to do. Furthermore, the information displayed as an air tag is not limited to character information, and can be applied to other information such as image information as long as it is information associated with position information.

  Further, in the system that displays information corresponding to equipment as an air tag as described above, other information such as a description of the equipment may be displayed as an air tag without being limited to the equipment name.

1 mobile device,
2 facilities management server,
3 Camera,
4 display,
5 direction sensor,
6 Equipment data display means,
7 Terminal position input means,
8 Mobile terminal communication means,
9 Equipment data management means,
10 Equipment data,
11 facility management server communication means,
12 Equipment master data management means,
13 Equipment master data

Claims (7)

An image display system for displaying other information superimposed on a captured image,
A range image information storage unit storing information of an image indicating a predetermined range;
A display information storage unit storing display information in which a position in the predetermined range and information to be displayed corresponding to the position are associated;
An imaging unit provided in a portable information terminal;
A current position information acquisition unit that displays the predetermined range on the display unit based on the stored image information and acquires the current position information of the information terminal that is input in response to an operation on the displayed range When,
A direction detection unit that detects a direction in which the imaging unit is imaging;
A display processing unit that displays an image captured by the imaging unit on a display unit as a captured image,
The display processing unit recognizes a range of the captured image based on the acquired current position information and the detected direction, and sets the position included in the recognized range of the stored display information. An image display system characterized in that information to be displayed correspondingly is displayed superimposed on a corresponding position of the captured image. The predetermined range is a room in the building;
A floor image information storage unit that stores information on floor images in the building;
The current position information acquisition unit displays a floor in the building on the display unit based on the stored floor image information, and selects one of a plurality of rooms included in the displayed floor. The image display system according to claim 1, wherein the predetermined range to be displayed on the display unit is determined by selection.   The display processing unit acquires and holds the display information corresponding to a position included in the range of the selected room from the display information storage unit after one of the plurality of rooms is selected. The image display system according to claim 2, wherein The position in the predetermined range is a position where predetermined equipment is installed in the predetermined range,
The image display system according to claim 2 or 3, wherein the information to be displayed corresponding to the position is information to be displayed corresponding to the equipment.   The image display system according to claim 4, wherein the information to be displayed corresponding to the position is a name of the facility. A portable information terminal that displays other information superimposed on a captured image,
An imaging unit that captures an image;
Obtaining image information indicating the predetermined range from a range image information storage unit storing image information indicating the predetermined range, displaying the predetermined range on the display unit, and performing an operation on the displayed range A current position information acquisition unit for acquiring current position information of the information terminal input according to
A display information acquisition unit that acquires the display information from a display information storage unit that stores display information in which a position in the predetermined range and information to be displayed corresponding to the position are associated;
A direction detection unit that detects a direction in which the imaging unit is imaging;
A display processing unit that displays an image captured by the imaging unit on a display unit as a captured image,
The display processing unit recognizes a range of the captured image based on the acquired current position information and the detected direction, and corresponds to a position included in the recognized range of the acquired display information. And displaying the information to be displayed superimposed on the corresponding position of the captured image. An image display program for a portable information terminal that displays other information superimposed on a captured image,
Obtaining image information indicating the predetermined range from a range image information storage unit storing image information indicating the predetermined range, displaying the predetermined range on the display unit, and performing an operation on the displayed range Obtaining the current location information of the information terminal input according to
Obtaining the display information from a display information storage unit storing display information in which a position in the predetermined range and information to be displayed corresponding to the position are associated;
Obtaining information indicating the direction in which the imaging unit provided in the portable information terminal is capturing, and indicating the direction detected by the direction detection unit provided in the portable information terminal;
Displaying an image captured by the imaging unit on the display unit as a captured image;
Recognizing a range of the captured image based on the acquired current position information and the detected direction;
Causing the portable information terminal to execute a step of superimposing and displaying information to be displayed corresponding to a position included in the recognized range of the acquired display information on a corresponding position of the captured image. An image display program characterized by the above.

Images