JP2004219664A - Information display system and information display method - Google Patents

Abstract

A head mounted display device displays information corresponding to a driver's line of sight.
An information display system includes an image creating device connected to a head-mounted display device, a direction specifying camera, and the like. The head-mounted display device 2 has markings 2d to 2f on the frame portion 2a, and the image forming device 3 is used as the head-mounted display device 2 based on the positional relationship between the markings 2d to 2f of the image captured by the direction specifying camera 4. Is calculated, and the calculated direction is specified as the gaze direction of the driver D. The image creating device 3 stores a three-dimensional map associated with the position, reads out information on the position corresponding to the specified line-of-sight direction, and displays the information on the head-mounted display device 2.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention is to display various information on a display device in a vehicle, and by using a head mounted display device having a transmissive display unit, various information is displayed in a field of view where a driver looks outside the vehicle. Information display system and information display method.
[0002]
[Prior art]
Conventionally, there is an information display system in which a display device is provided on an upper part of an instrument panel or a center console or the like in a vehicle, and various information such as traffic congestion information and a map screen searched by a navigation device is displayed on a screen of the display device. Existing.
[0003]
Further, the information display system has various modes other than the above modes. For example, a head-up display device that uses a head-up display device that displays information projected by a projector on a combiner provided in a part of a front window, and a head-mounted display device that has a transmissive display unit that allows the outside world to be seen even when mounted. Some are used.
[0004]
An information display system using a head-up display device is disclosed in Patent Document 1 below.
[0005]
[Patent Document 1]
JP-A-9-2102
[0006]
[Problems to be solved by the invention]
In a system in which a display device is provided on an instrument panel, a center console, or the like, the driver has to shift his / her gaze from the front of the vehicle to the display device when viewing information to be displayed. There is a problem. In addition, in the case of an information display system using a head-up display device, since information is displayed on the combiner, the load on the driver's line of sight movement is reduced, but the display resolution of the combiner is lower than that of a general display device. Therefore, there is a problem that it is difficult to appropriately display a fine image such as a map related to route guidance of a navigation device.
[0007]
Furthermore, even with an information display system using a head-mounted display device, it is difficult to display a detailed image such as a map screen, and depending on the manner of displaying information, it may obstruct the driver's view.
[0008]
On the other hand, the above three types of information display systems only display information according to the current position of the vehicle, and cannot display information according to the direction in which the driver is looking. Is the current situation. Further, these three types of display systems cannot cope with appropriately warning that an obstacle exists when an obstacle or the like exists in front of the vehicle.
[0009]
The present invention has been made in view of the above circumstances, and detects a direction of a head-mounted display device using a head-mounted display device, thereby enabling information corresponding to a driver's line of sight to be displayed. It is an object to provide a display system.
[0010]
Further, according to the present invention, when the road of the route searched by the navigation device is present in the line of sight of the driver, a mark corresponding to the searched road is displayed on the transmissive display section of the head mounted display device, so that the map is displayed. It is an object of the present invention to provide an information display system capable of recognizing a road corresponding to a search route without displaying the information.
[0011]
A further object of the present invention is to provide an information display system that includes a detection unit that detects an obstacle, a person, and the like located in front of a vehicle, and can appropriately display a warning.
[0012]
[Means for Solving the Problems]
An information display system according to a first aspect of the present invention is an information display system for displaying information on a transmissive display unit included in a head mounted display device mounted on a head of a vehicle driver, wherein the storage stores information associated with a position. Means, position detecting means for detecting a vehicle position, calculating means for calculating a driver's seat position from the vehicle position detected by the position detecting means, and detecting a direction in which the transmissive display unit of the head mounted display device is directed. A direction detecting unit, a position specifying unit that specifies a position in the direction detected by the direction detecting unit from the driver's seat position calculated by the calculating unit, and an association from the storage unit based on the position specified by the position specifying unit. Means for reading out the information and displaying the information on the transmissive display unit.
[0013]
An information display system according to a second invention is characterized in that the information is a content related to a building corresponding to a position associated with the information.
[0014]
An information display system according to a third aspect of the present invention is an information display system for displaying information on a transmissive display unit of a head mounted display device mounted on the head of a vehicle driver, a search unit for searching for a route, and a vehicle position. Position detecting means for detecting the position of the vehicle, the calculating means for calculating the driver's seat position from the vehicle position detected by the position detecting means, a direction detecting means for detecting the direction in which the transmissive display unit of the head mounted display device is directed, Range specifying means for specifying a position range including the direction detected by the direction detecting means from the driver's seat position calculated by the calculating means; and the position range specified by the range specifying means includes the route searched by the search means. Determining means for determining whether or not the searched route is included in the position range; A mark generating means for generating, characterized in that it comprises means for displaying indicia indicia producing means in the transmissive display section.
[0015]
An information display system according to a fourth invention further comprises a detecting means for detecting a seat height of the driver's seat, wherein the calculating means sets a driver in the driver's seat based on the seat height detected by the detecting means. Means for calculating a driver's head position when the driver is seated as a driver's seat position.
[0016]
An information display system according to a fifth aspect of the present invention includes a map storage unit that stores a three-dimensional map, and an extraction unit that extracts a part of the three-dimensional map corresponding to the position range specified by the range specification unit from the map storage unit. And a shape specifying means for specifying a road shape of the route searched by the search means included in the part of the three-dimensional map extracted by the extracting means, wherein the mark creating means includes a road specified by the shape specifying means. It is characterized by comprising means for creating a mark along the shape.
[0017]
In an information display system according to a sixth aspect, the direction detecting means includes an absolute azimuth detecting means for detecting an absolute azimuth centered on a driver's seat position calculated by the calculating means, and a transmissive display unit for the head mounted display device Photographing means for photographing from the side opposite to the photographing means; identifying means for identifying the direction of the head-mounted display device from the video imaged by the photographing means; and an absolute direction in which the absolute direction detecting means detects the direction identified by the identifying means. Direction conversion means for converting the direction into a direction based on the azimuth.
[0018]
The information display system according to the seventh invention further includes a detection unit that detects an object located in front of the vehicle, and a unit that, when the detection unit detects an object, displays that the detection is performed on the transmission display unit. It is characterized by the following.
[0019]
An information display method according to an eighth aspect of the present invention is the information display method in which the information output device displays information on the transmissive display unit of the head mounted display device mounted on the head of the vehicle driver. And a position detecting means for detecting a vehicle position, wherein the information output device calculates a driver's seat position from the vehicle position detected by the position detecting means, and a transmission type of the head mounted display device. Detecting the direction in which the display unit is turned, specifying the position in the detected direction from the calculated driver's seat position, reading the associated information from the storage unit based on the specified position, and reading the transmission type display. It is displayed on a section.
[0020]
An information display method according to a ninth invention is a search method for searching for a route in an information display method in which an information output device displays information on a transmission type display unit included in a head mounted display device mounted on the head of a vehicle driver. And a position detecting means for detecting a vehicle position, wherein the information output device calculates a driver's seat position from the vehicle position detected by the position detecting means, and the transmission type display section of the head mounted display device is directed. Detecting a direction, specifying a position range including the detected direction from the calculated driver's seat position, determining whether the specified position range includes the route searched by the search unit, and determining the position range. If it is determined that the searched route is included, a mark indicating the searched route is created, and the created mark is displayed on the transparent display unit.
[0021]
According to the first invention and the eighth invention, the information corresponding to the driver's line of sight is displayed by detecting the direction of the head mounted display device and displaying information associated with the position of the detected direction. it can. It is preferable that information to be displayed is stored in a storage medium such as a CD, a DVD, or a hard disk as a storage unit, and the direction detected from these storage media is read out and displayed at any time as a clue.
[0022]
According to the second aspect of the present invention, since the information related to the building corresponding to the position can be displayed as information, various useful information can be provided to the driver. The contents related to the building include the type and name of the building, and if the building is a store, the contents can include the business hours, the type of product for sale, the selling price, and the like.
[0023]
According to the third invention and the ninth invention, a search means for searching for a route to a destination is combined, and when a road corresponding to the search route exists in the driver's line of sight, an actual view through the transparent display unit is provided. Since a mark indicating that the road is a search route is displayed on the road, a guide route can be directly displayed to the driver without displaying a map. In addition, by using the head-mounted display device, the driver can recognize the road of the search route without moving his line of sight, and the burden on the road recognition of the driver can be reduced.
[0024]
According to the fourth aspect, the driver's seat position is specified three-dimensionally in order to detect the driver's seat height and calculate the head position when the driver is seated in the driver's seat as the driver's seat position. At the same time, the position range in the line of sight from the driver's seat position can be specified three-dimensionally. This makes it possible to display the mark to be displayed in a form corresponding to the three-dimensional position range, thereby preventing a situation in which the mark is displayed at a visual field position that hinders the driver's field of view.
[0025]
According to the fifth aspect, since the three-dimensional map is stored and the part of the three-dimensional map is extracted from the viewpoint from the driver's seat position, the state of the outside world that the driver actually sees through the transparent display unit can be grasped. . Furthermore, since a mark is created and displayed along the road shape of the search route in the grasped situation, the road viewed through the transparent display unit and the displayed mark correspond, and the driver can easily search without a map. The road corresponding to the route can be recognized. It is preferable that the mark to be displayed is formed in a slightly smaller shape than the road that is actually seen, and is displayed so as to overlap the road that is actually seen through the transmissive display unit.
[0026]
According to the sixth aspect, by providing the photographing means for photographing the head-mounted display device from the opposite side, the direction of the head-mounted display device can be reliably specified. Further, since the direction in which the transmission type display unit of the head mounted display device is turned is detected by converting the specified direction into the absolute direction, the position according to the detected direction, and the position range included in the detected direction are specified. Processing can be performed accurately and easily.
[0027]
According to the seventh aspect, when the detecting means detects an object in front of the vehicle, the detection is displayed on the transmissive display unit, so that the driver can quickly see an obstacle, a falling object, A response that recognizes a person or the like becomes possible. The distance from the vehicle to the detected vehicle may be calculated and displayed in consideration of the time required for the detection by the detecting unit and the current vehicle speed.
[0028]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, the present invention will be described with reference to the drawings showing the embodiments.
[0029]
1 and 2 show an overall configuration of an information display system 1 according to an embodiment of the present invention. The information display system 1 captures a head-mounted display device (referred to as an HMD device) 2 that is worn like glasses on the head of a driver D of a vehicle, an image creating device 3 corresponding to an information output device, and an image of the HMD device 2. The system includes a camera 4 for specifying a direction to be searched, a navigation device 5 for searching the position and route of the vehicle, an obstacle detection sensor 6 for detecting an obstacle in front of the vehicle, and a seat height sensor 7 for detecting the height of the driver's seat.
[0030]
As shown in FIG. 1, the HMD device 2 protrudes left and right handle portions 2b and 2c to be placed on the ears of the driver D from both ends of the frame portion 2a and faces the driver D's both eyes of the frame portion 2a. Is mounted with a combiner 10 of a transmission type display unit through which light is transmitted. The left and right handle parts 2b and 2c are connected by an arch-shaped support part 2g that passes over the driver D's head so that the HMD device 2 is stably mounted on the driver D's head. .
[0031]
The frame 2a has markings 2d, 2e, and 2f for direction identification on the left, right, and center of the outer front, respectively. As shown in FIG. 3, a projector 20 and a display controller 21 for the combiner 10 are provided on the inner surface side of each of the patterns 2b and 2c. The display controller 21 is connected to the image creating device 3 via the connection line S1 shown in FIG. 1, and projects the image created by the image creating device 3 with the projector 20 and displays the image on the combiner 10. Since the combiner 10 is of a transmission type, the driver D can secure a field of view through the combiner 10 that does not hinder driving even when the HMD device 2 is mounted.
[0032]
In the HMD device 2, a geomagnetic sensor 8 is provided on the top of the support 2g. The terrestrial magnetism sensor 8 detects the absolute azimuth of the east, west, north and south as the absolute azimuth detecting means, and transmits the detected absolute azimuth to the image creating apparatus 3 via the connection line S1.
[0033]
Further, the HMD device 2 is provided with a first tilt sensor 9a, a second tilt sensor 9b, and a third tilt sensor 9c at the center of the upper end of the frame portion 2a and the left and right portions of the support portion 2g.
[0034]
The first tilt sensor 9a detects an angle (pitch angle) when the driver D tilts the head back and forth as shown in FIG. 4A, and is based on a horizontal line (Y direction in the figure). And the angle inclined from the horizontal line is detected as the pitch angle. The second and third tilt sensors 9 detect an angle (roll angle) when the driver D tilts the head to the left and right as shown in FIG. 4B, and a vertical line (Z in the figure). Direction) is set to 0 degree as a reference, and an angle inclined from a vertical line is detected as a roll angle. In addition, each of the inclination sensors 9a to 9c transmits data of the detected angle to the image forming apparatus 3 via the connection line S1.
[0035]
The direction identifying camera 4 shown in FIG. 1 is installed on an instrument panel (not shown) at a location facing the driver's seat of the vehicle, and serves as a photographing unit that photographs the HMD device 2 from the side facing the combiner 10. It functions and transmits the data of the photographed image to the image creation device 3 via the connection line S2. The transmitted image is used in the process in the image creating device 3 for specifying the angle (the yaw angle) at which the driver D turns his or her head to the left and right shown in FIG. 7C.
[0036]
The navigation device 5 shown in FIG. 1 includes a receiving antenna 5a for receiving position data relating to latitude and longitude from a satellite and a required operation unit (not shown) having various switches. The navigation device 5 functions as a position detecting means for detecting a vehicle position and a searching means for searching for a route. Specifically, as shown in FIG. 5, the navigation device 5 detects the center position of the road R on which the vehicle A travels based on the data received from the artificial satellite, corrects the detected center position, and corrects the detected center position. The center point A1 is detected as a vehicle position.
[0037]
The navigation device 5 also calculates the traveling direction of the vehicle A by detecting the vehicle position continuously for a certain period of time. The information display system 1 of the present embodiment grasps the traveling direction of the vehicle A thus calculated as the Y direction, the direction orthogonal to the traveling direction as the X direction, and the vertical direction as the Z direction. The same X, Y and Z directions are used for the pitch angle and roll angle shown in (b) and the like.
[0038]
Further, the navigation device 5 has a function of searching for an optimal route from the current position to the destination by performing a required operation on the operation unit. The navigation device 5 transmits the calculated latitude and longitude, the traveling direction, and the data related to the searched route to the image creating device 3 through the connection line S4.
[0039]
The obstacle detection sensor 6 is attached in front of the vehicle A as shown in FIG. 6, and detects an obstacle M located on a road R in front of the vehicle as a detection means. The obstacle detection sensor 6 of the present embodiment uses a millimeter wave radar to determine the presence or absence of an obstacle M in front of the vehicle according to the state of the reflected millimeter wave, and to detect the presence of the millimeter wave reflected from the output of the millimeter wave. The time until input is also detected. The obstacle detection sensor 6 can detect not only an object but also a person.
[0040]
When the obstacle detection sensor 6 detects the presence of the obstacle M, the obstacle detection sensor 6 transmits, to the image creating apparatus 3, notification data for notifying that the obstacle M has been detected by the connection line S5 and data of a time related to the detection. Note that a laser radar, a stereo camera, or the like can be applied to the obstacle detection sensor 6 in addition to the millimeter wave radar.
[0041]
The seat height sensor 7 shown in FIG. 1 serves as detecting means for detecting the height from the upper surface of the floor in the vehicle to the headrest of the driver's seat (not shown), and outputs data relating to the detected height through a connection line S3. 3.
[0042]
The image creation device 3 includes operation units such as a display control switch 3a and a vehicle type selection switch 3b, and also has a storage unit 14, a view range identification unit 11, a route creation unit 12, a target information creation unit 13, and a danger as shown in FIG. An information creation unit 15 and an HMD image synthesis unit 16 are provided. The storage unit 14 functions as a storage unit and a map storage unit, and employs a hard disk in this embodiment. The storage unit 14 stores a digital three-dimensional map on the hard disk. The three-dimensional map includes a topographic map in a form corresponding to various directions of the driver from a viewpoint. Information associated with each position is also included.
[0043]
That is, the three-dimensional map associates various types of information with each position corresponding to the absolute azimuth, latitude, and longitude, and the information includes, for example, the content of a building built at the corresponding position. The building corresponds to restaurants, department stores, gas stations, etc., business hours and menus are stored as contents related to restaurants, sale items and entertainment guides, etc. are stored as contents related to department stores. Of gasoline and service information are stored.
[0044]
The field-of-view range specifying unit 11 functions as a range specifying unit that specifies a line-of-sight direction of the driver D and performs a process of determining a position range serving as the driver's field of view, and includes a position specifying unit 11a and a line-of-sight direction specifying unit 11a inside. Have.
[0045]
The position identification unit 11a functions as a calculation unit that calculates the driver's seat position, and calculates data on the vehicle position from the navigation device 5, height data detected by the seat height sensor 7, and data of the absolute azimuth detected by the geomagnetic sensor 8. Accepted. The position identification unit 11a according to the present embodiment calculates the head position D1 of the driver D sitting in the driver's seat A2 as the driver's seat position as shown in FIG.
[0046]
The position identification unit 11a stores offset distances x1 and y1 in the X and Y directions from the center point A1 of the vehicle A to the head position D1 of the driver D, which are vehicle position data transmitted from the navigation device 5 The X and Y coordinates of the head position D1 of the driver D are calculated by subtracting the offset distances x1 and y1 from the center point A1 of the vehicle A.
[0047]
In addition, the position specifying unit 11a calculates the height of the headrest of the driver's seat A2 transmitted from the seat height sensor 7 as position coordinates (Z coordinates) in the Z-axis direction relating to the head position D1 of the driver D. The position specifying unit 11a converts the X and Y coordinates calculated as described above into a position in the absolute azimuth from the geomagnetic sensor 8, and converts the head position D1 of the driver D into a three-dimensional unit called latitude, longitude and height. To specify.
[0048]
The line-of-sight direction identification unit 11b functions as direction detection means for detecting the direction in which the combiner 10 of the HMD device 2 is facing from the driver's seat position calculated by the position identification unit 11a. The direction is detected based on the data, the image data from the direction specifying camera 4, and the data from the inclination sensors 9a to 9c.
[0049]
In response to the direction detection, the line-of-sight direction identification unit 11b analyzes the image data from the direction identification camera 4 and calculates the yaw angle with respect to the traveling direction (Y direction: 0 degrees) of the vehicle A shown in FIG. Identify. As shown in FIGS. 7A and 7B, the yaw angle is specified by marking portions 2d, 2e, and 2f attached to the frame portion 2a of the HMD device 2 in the image data and the vertical center with respect to the head of the driver D. This is performed by the gaze direction identification unit 11b determining the positional relationship with the line T.
[0050]
For example, as shown in FIG. 7A, when the transmitted image data has the center marking portion 2 e overlapping the vertical center line T, the HMD device 2 is in a direction in which the direction identifying camera 4 is directly viewed. The line-of-sight direction specifying unit 11b determines and determines the yaw angle as 0 degree. Also, as shown in FIG. 7 (b), when only the marking part 2d and the center marking part 2e on the left side in the figure are shown as shown in FIG. 7 (b), from the one marking part 2d to the vertical center line T The yaw angle is calculated from the distance T1 and the distance T2 from the other marking portion 2e to the vertical center line T. As described above, when the yaw angle is other than 0 degrees, the line-of-sight direction specifying unit 11b calculates and specifies the yaw angle from the distance between each of the marking units 2d to 2f and the vertical center line T.
[0051]
Further, the line-of-sight direction specifying unit 11b converts the yaw angle specified as described above as a direction conversion unit into a direction in the absolute azimuth transmitted from the geomagnetic sensor 8 around the head position D1 of the driver D, and the conversion is performed. The specified direction is specified as the line-of-sight direction on the XY plane. Further, the line-of-sight direction identification unit 11b determines the direction of the HMD device 2 on the YZ plane and the XZ plane based on the pitch angle and the roll angle transmitted from each of the inclination sensors 9a to 9c, and identifies the line-of-sight direction. In addition, the driver's D gaze direction is specified three-dimensionally with the gaze direction on the XY plane.
[0052]
As described above, the visual field range specifying unit 11 determines the driver D shown in FIGS. 5 and 6 based on the head position D1 of the driver D specified by the position specifying unit 11a and the gaze direction specified by the gaze direction specifying unit 11b. Specify the position range included in the field of view. The position range corresponds to a visual field range K that is a normal human visual field angle centered on the specified visual line direction (indicated by K1 in the figure), and the visual field range specifying unit 11 of the present embodiment moves in the horizontal direction. A substantially conical range having the top at the head position D1 at 75 degrees each and 30 degrees each in the vertical direction is specified as the visual field range K.
[0053]
In addition, in this embodiment, the depth of the visual field range K is set within a range of 100 m or less from the head position D1, but this numerical value can be appropriately changed. The visual field range specifying unit 11 outputs the specified head position D1, the line of sight direction K1, and the visual field range K of the driver D to the route creating unit 12, the target information creating unit 13, and the danger information creating unit 15.
[0054]
The route creation unit 12 is connected to the storage unit 14 and functions as a mark creation unit that creates a mark indicating a road corresponding to the route searched by the navigation device 5, and includes a route determination unit 12a and a three-dimensional image generation unit 12b. ing.
[0055]
The route determination unit 12a receives search route data from the navigation device 5, and determines as a determination unit whether or not the transmitted search route is included in the current visual field range K. When the route determination unit 12a determines that the search route is included in the visual field range K, the three-dimensional image generation unit 12b sets a mark indicating a road within the visual field range K of the search route in accordance with the driver's D line of sight. Create with shape.
[0056]
That is, the three-dimensional image generation unit 12b first stores partial data of the three-dimensional map corresponding to the head position D1, the line-of-sight direction K1, and the field-of-view range K of the driver D transmitted from the field-of-view range identification unit 11 as extraction means. It is extracted and read from the unit 14. Further, the three-dimensional image generation unit 12b specifies the read partial data as a topographic map actually viewed by the driver D through the combiner 10, and determines a road shape of a search route in the specified topographic map as shape specifying means. Identify.
[0057]
Therefore, the route creation unit 12 forms a mark along the road shape specified by the three-dimensional image generation unit 12b, and specifies the location where the formed mark is displayed so as to overlap the searched road in the specified topographic map. are doing.
[0058]
Explaining a specific processing example in the route creating unit 12 described above, the topographic map that the driver D actually sees through the combiner 10 of the HMD device 2 is a situation as shown in FIG. It is assumed that the device 5 searches for a route so that the lane R1 of the road R on which the vehicle is currently traveling is turned left ahead and proceeds to the lane R2. In this case, the route determination unit 12a determines that the road of the search route is included in the visual field range K, and the three-dimensional image generation unit 12b reads the map data from the viewpoint equivalent to FIG.
[0059]
Further, the three-dimensional image generation unit 12b specifies the road position and the shape of the search route in the read map data, and displays the mark 30 along the specified three-dimensional road shape as shown in FIG. 10 and is generated at a position corresponding to an actual road. In addition, the three-dimensional image generation unit 12a adds a flag icon 30a to a corner of the generated mark 30 that is a left turn point, and simultaneously creates a message 31 indicating the distance to the left turn and an arrow 32 indicating the left turn. I have.
[0060]
Since the selection signal of the vehicle type selection switch 3b of the image creation device 3 is input to the three-dimensional image generation unit 12a, the lower end position W of the front window of the vehicle A according to the selection signal (see FIG. 8A). Are read from the built-in memory, and a mark 30, a message 31, an arrow 32, and the like are generated at a position above the lower end position W. In this manner, when various instruments and indicator lights of the instrument panel are viewed through the combiner 10, the various instruments and indicator lights and the like are surely not interrupted by the mark 30 and the message 31 displayed on the combiner 10. You can see.
[0061]
The route creating unit 12 outputs to the HMD image combining unit 16 the route guidance image data related to the mark 30, the message 31, the arrow 32, and the like, generated by the three-dimensional image generating unit 12a.
[0062]
The target information creating unit 13 shown in FIG. 2 identifies the position of the driver D in the line-of-sight direction K1 as a position identifying unit, extracts target information of the identified position from the storage unit 14, and creates display data. And a target information extracting unit 13b. The target specifying unit 13a specifies an object that intersects with an extension of the driver D in the line of sight and is closest to the vehicle.
[0063]
That is, the target specifying unit 13a stores the partial data of the three-dimensional map included in the visual field range K from the storage unit 14 based on the data of the visual line direction K1 and the visual field range K shown in FIGS. , And the closest building T is specified from the object located in the line-of-sight direction K1 in the read partial data, for example, as shown in FIG. Further, the target information extracting unit 13b reads out information on the building T specified as described above from the storage unit 14. When the processing is simplified, the target position in the line-of-sight direction may be specified based on data only on the XY plane.
[0064]
Therefore, the target information creating unit 13 creates the target information message 33 as shown in FIG. 9A based on the read information, and outputs the data related to the created target information message 33 to the HMD image combining unit. 16 is output. The target information creating unit 13 is connected to a display control switch 3a for switching a detailed information display mode, and the display control switch 3a is used to display information of an object located in the line of sight on the combiner 10 of the HMD device 2. Is a switch for determining whether or not to perform. Therefore, when the display control switch 3a is on, the target information creating unit 13 performs the various processes described above, and when the display control switch 3a is off, the target information creating unit 13 does not perform the various processes described above.
[0065]
The danger information creation unit 15 shown in FIG. 2 creates a message for notifying the driver D when a thing ahead of the vehicle is detected, and includes a danger state identification unit 15a and a danger state output unit 15b. The danger state specifying unit 15a is connected to the obstacle detection sensor 6, and when the obstacle detection sensor 6 detects the obstacle M as shown in FIG. Data for notifying that M has been detected and the time required for detection has been received, and it has been specified whether or not the state is dangerous.
[0066]
The dangerous state output unit 15b is in a dangerous state as shown in FIG. 9B, for example, based on the data output from the obstacle detection sensor 6 when the dangerous state identifying unit 15a identifies the dangerous state. Of the message 34 indicating the fact, and outputs the created data to the HMD image synthesizing unit 16.
[0067]
Finally, the HMD image synthesizing unit 16 shown in FIG. 2 receives various display data output from the route creating unit 12, the target information creating unit 13, and the danger information creating unit 15, and outputs the data to the HMD device 2, The control for displaying various data on the combiner 10 of the HMD device 2 is performed.
[0068]
The HMD image synthesizing unit 16 treats the data output from the danger information creating unit 15 with the highest priority, and when the data from the danger information creating unit 15 is received, the route creating unit 12 and the target information creating unit Even if data is received from the unit 13, only data from the danger information creating unit 15 is output to the HMD device 2. By performing such processing, the driver D can recognize an obstacle ahead of the vehicle at an early stage.
[0069]
Next, a series of processes related to the information display method of the information display system 1 described above will be described based on the flowchart of FIG.
[0070]
First, when the information display system 1 is started, the image creating device 3 specifies the head position D1 of the driver D of the vehicle A (S1), and calculates the line-of-sight direction K1 of the driver D (S2). Further, the image creating device 3 specifies the visual field range K based on the head position D1 specified as described above and the calculated visual line direction K1 (S3), and a three-dimensional image corresponding to the specified visual field range K. The partial data of the map is read from the storage unit 14.
[0071]
The image creating device 3 determines whether the navigation device 5 has performed a route search (S4). When it is determined that the navigation device 5 has performed the route search (S4: YES), the image creating device 3 determines whether or not the road of the searched route is included in the specified visual field range K in the partial data of the three-dimensional map. The determination is made based on (S5). When the road of the search route is included in the visual field range K (S5: YES), the route determination unit 12 of the image creating device 3 sets the map in a form that matches the actually visible road 25 as shown in FIG. The route image data such as the mark 30 indicating the search route and the message 31 as shown in FIG. 8B is created (S6).
[0072]
When the navigation device 5 is not performing the route search (S4: NO), and when the road of the search route is not included in the position range (S5: NO), the image creating device 3 does not create the route image data.
[0073]
Further, the image creating device 3 determines whether or not the display control switch 3a is turned on to enter the detailed information display mode (S7). When the display control switch 3a is on (S7: YES), the image creating device 3 determines whether or not the target exists in the specified line of sight (S8). If there is an object in the line of sight (S8: YES), the object information creating unit 13 of the image creating apparatus 3 reads out information on the corresponding object from the storage unit 14, and outputs a message as shown in FIG. Create target object information data corresponding to (S9)
When the display control switch 3a is turned off and the detailed information display mode is not set (S7: NO), when no object exists in the line of sight (S8: NO), the image creating device 3 creates the object information data. do not do.
[0074]
Further, the image creating device 3 determines whether or not an obstacle exists in front of the vehicle based on the data relating to the detection from the obstacle detection sensor 6 (S10), and determines that no obstacle exists. If it has been (S10: NO), the data created by the route creation unit 12 and the target information creation unit 13 is output to the HMD device 2 and displayed (S11). By creating the data and displaying the data in this way, the driver D shown in FIG. 8A is made to correspond to the lanes R1 and R2 of the road 25 actually seen through the combiner 10 as shown in FIG. The route image data thus selected is displayed on the combiner 10.
[0075]
As a result, the driver D sees the mark 30 displayed on the combiner 10 so as to overlap the actual road 25 passing through the combiner 10 as shown in FIG. 8C, and the driver D is searched. A road can be easily determined directly without a map without moving the line of sight in correspondence with the road actually seen. When there is a building, facility, or the like corresponding to the line of sight, a message 33 is displayed on the combiner 10 as shown in FIG. In addition, since the display location of the message 33 is a position where the building to be displayed can be seen through, the driver can easily grasp the message and the building to be subjected to the message.
[0076]
When the image creation device 3 determines that an obstacle is present in front of the vehicle (S10: YES), the image creation device 3 creates danger information data based on the data from the obstacle detection sensor 6 (S12) and outputs the danger information data to the HMD device 2. Then, as shown in FIG. 9B, the message 34 relating to the danger information data is displayed on the combiner 10 through which the actual road 25 is seen through (S13). Therefore, the driver D can quickly recognize the presence of the obstacle in front of the vehicle, and can quickly recognize the dangerous state without moving his line of sight.
[0077]
Note that the information display system 1 according to the present invention is not limited to the above-described embodiment, and various modifications can be applied. For example, when the obstacle sensor 6 detects an obstacle by connecting a vehicle speed pulse sensor for detecting the vehicle speed to the image creation device 3 so that the image creation device 3 can recognize the current vehicle speed, detection is required. The distance to the obstacle may be calculated from the time and the current vehicle speed, and the calculated distance may be displayed.
[0078]
Further, the signal from the above-described vehicle speed pulse sensor may be used for display processing of target object information. That is, information on buildings and facilities located in the line of sight is displayed when the display control switch 3a is turned on and the signal from the vehicle speed pulse sensor is at a vehicle speed of 0 km / h. By not displaying the information, it is possible to provide a display form that does not hinder the driving operation. It is to be noted that the image creating device 3 is provided with a function of measuring time, and information such as a building and a facility located in the line of sight is displayed only when the line of sight is the same for a predetermined time (for example, 5 seconds) or more. It may be.
[0079]
Further, a driver's seat is provided with a seating sensor for detecting the seating of a human body, and a detection signal relating to the seating detection of the seating sensor is input to the image creating device 3 so that when a detection signal from the seating sensor is input. Alternatively, the present system may be activated. By doing in this way, various kinds of information are displayed only when the driver D is seated in the driver's seat in a normal state, so that the actual external object viewed through the combiner 10 and the actual object are displayed on the combiner 10. It is possible to prevent a situation in which the positions of the mark and the message are largely shifted.
[0080]
Furthermore, when the navigation device 5 includes a storage unit for storing necessary map data and a reading unit for reading map data, the image creating device 3 does not need to include the storage unit 14 shown in FIG. Alternatively, the above-described map data of the navigation device 5 may be shared. Further, in order to detect the orientation of the HMD device 2, in addition to marking the HMD device 2, it is also possible to detect the direction of the driver's iris and nose that can be seen through the combiner 10. Further, the HMD device 2 may be of a helmet type in addition to the glasses type.
[0081]
Furthermore, the height position (coordinates) of the head position D1 of the driver D may be set by providing a height selection switch in the image creation device 3 and inputting the selection switch. Alternatively, it may be determined by performing calibration from the position of the driver D's eye in the image data captured by the direction specifying camera 4. Furthermore, the pitch angle of the driver D's head can be calculated based on the difference in the line-of-sight position from the forward direct vision reference coordinates measured in advance for each driver instead of the inclination sensor 9a. May be calculated from the average line-of-sight position of the driver. In addition, a gyro sensor other than the geomagnetic sensor 8 can be used for detecting the absolute azimuth.
[0082]
Also, the information display system 1 displays all of the searched route, the display of the target information located in the line of sight direction, and the notification display of the object located in front of the vehicle. May be displayed. In this case, it is preferable that the configuration of the information display system 1 and the internal configuration of the image creating apparatus 3 include only the elements necessary for display to simplify the configuration.
[0083]
【The invention's effect】
As described in detail above, in the first invention and the eighth invention, by detecting the direction of the head mounted display device and displaying the information associated with the position in the position range of the driver with respect to the detected direction. In addition, information corresponding to the driver's line of sight can be displayed, and the driver can check various types of information without moving the driver's line of sight.
[0084]
According to the second aspect of the present invention, the information relating to the building corresponding to the position can be displayed as information, so that the driver can check various useful information.
[0085]
In the third invention and the ninth invention, the mark indicating the road of the search route is displayed on the combiner of the HMD device, so that the driver directly faces the front of the vehicle without displaying the map around the vehicle. The road of the search route can be intermittently recognized, and the burden on the driver involved in confirming the road of the search route can be reduced.
[0086]
According to the fourth aspect, the driver's seat position is calculated in accordance with the driver's seat height, so that the driver's seat position can be specified three-dimensionally, and accordingly, the position range in the line of sight from the driver's seat position is specified. Also become three-dimensional, and can be used for three-dimensional information display.
[0087]
According to the fifth invention, the three-dimensional map is stored, and the part of the three-dimensional map is extracted from the viewpoint from the driver's seat position. In this state, a mark indicating the road of the search route can be displayed, and the driver can more easily and reliably recognize the road of the search route.
[0088]
According to the sixth aspect, since the head-mounted display device is photographed from the opposite side, the orientation of the head-mounted display device can be reliably specified, and the accuracy of displaying various information can be improved.
[0089]
In the seventh invention, when the detecting means detects an object in front of the vehicle, the detection is displayed on the combiner, so that the driver can quickly recognize an obstacle, a falling object, a person, etc. in front of the vehicle by the display of the combiner. It is possible to quickly respond to the driving operation that has been performed, and it is possible to contribute to avoiding a dangerous state before it occurs.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of an information display system according to an embodiment of the present invention.
FIG. 2 is a block diagram of an information display system.
FIG. 3 is a sectional view of a combiner of the head mounted display device.
FIG. 4A is a schematic diagram illustrating a tilt angle (pitch angle) of a driver's head in the front-rear direction, and FIG. 4B is a schematic diagram illustrating a horizontal tilt angle (roll angle) of the driver's head. is there.
FIG. 5 is a schematic plan view showing the relationship between the center point of the vehicle and the position of the driver's head.
FIG. 6 is a schematic diagram showing a gaze direction of a driver, detection of an obstacle, and the like.
7A is a schematic diagram showing a state in which the driver is directly looking, FIG. 7B is a schematic diagram showing a state in which the driver turns his / her head to the left, and FIG. FIG. 3 is a schematic diagram showing a rotation angle (yaw angle) in the left-right direction of FIG.
8A is a diagram showing a situation of an external road seen by a driver through a combiner, FIG. 8B is a diagram of image data related to a mark corresponding to a road of a search route, and FIG. FIG. 7 is a diagram illustrating a state in which image data is displayed on a combiner so as to be superimposed on a road that can be seen.
9A is a diagram illustrating a state in which a message related to an object in the line of sight is displayed on the combiner, and FIG. 9B is a diagram illustrating a state in which a message when an obstacle is detected ahead of the vehicle is displayed on the combiner; FIG.
FIG. 10 is a flowchart showing a series of processes according to the information display method of the present invention.
[Explanation of symbols]
1 Information display system
2 Head mounted display device
3 Image creation device
4 direction identification camera
5 Navigation device
6 Obstacle detection sensor
7 Seat height sensor
8 Geomagnetic sensor
9a-9c tilt sensor
11 Field of view range identification unit
12 Route creation unit
13 Target information creation unit
14 Storage unit
15 Danger Information Creation Department
16 HMD image synthesis unit
A vehicle
D1 Head position

Claims (9)

In an information display system that displays information on a transmission type display unit included in a head mounted display device mounted on the head of a vehicle driver,
Storage means for storing information associated with the position;
Position detecting means for detecting a vehicle position;
Calculating means for calculating a driver's seat position from the vehicle position detected by the position detecting means;
Direction detection means for detecting a direction in which the transmission type display unit of the head mounted display device is directed,
Position identification means for identifying a position in the direction detected by the direction detection means from the driver's seat position calculated by the calculation means,
Means for reading associated information from the storage means based on the position specified by the position specifying means and displaying the information on the transmissive display unit. The information display system according to claim 1, wherein the information is content related to a building corresponding to the associated position. In an information display system that displays information on a transmission type display unit included in a head mounted display device mounted on the head of a vehicle driver,
Search means for finding a route;
Position detecting means for detecting a vehicle position;
Calculating means for calculating a driver's seat position from the vehicle position detected by the position detecting means;
Direction detection means for detecting a direction in which the transmission type display unit of the head mounted display device is directed,
Range specifying means for specifying a position range including the direction detected by the direction detecting means from the driver's seat position calculated by the calculating means,
Determining means for determining whether or not the position range specified by the range specifying means includes the route searched by the search means;
A marking creating means for creating a mark indicating the searched route when the determining means determines that the searched route is included in the position range;
Means for displaying the mark created by the mark creating means on the transmissive display unit. Further comprising a detecting means for detecting the height of the driver's seat,
The calculating means,
4. The information display system according to claim 3, further comprising: a unit configured to calculate a driver's head position when the driver sits in the driver's seat as a driver's seat position based on the seat height detected by the detection unit. 5. Map storage means for storing a three-dimensional map,
Extracting means for extracting a portion of the three-dimensional map corresponding to the position range specified by the range specifying means from the map storage means,
Shape identifying means for identifying the road shape of the route searched by the searching means included in the part of the three-dimensional map extracted by the extracting means,
The mark creating means,
The information display system according to claim 4, further comprising a unit that creates a mark along the road shape specified by the shape specifying unit. The direction detecting means,
Absolute azimuth detecting means for detecting an absolute azimuth about the driver's seat position calculated by the calculating means,
Photographing means for photographing the head-mounted display device from a side facing the transmission display unit;
Specifying means for specifying the direction of the head-mounted display device from the image captured by the image capturing means,
The information display system according to claim 1, further comprising: a direction conversion unit configured to convert the direction specified by the specifying unit into a direction based on the absolute azimuth detected by the absolute azimuth detection unit. Detecting means for detecting an object located in front of the vehicle;
The information display system according to any one of claims 1 to 6, further comprising: when the detection means detects an object, a means for displaying the detection on the transmission type display unit. In an information display method in which an information output device displays information on a transmission type display unit included in a head mounted display device mounted on a head of a vehicle driver,
Storage means for storing information associated with the position;
Position detecting means for detecting a vehicle position,
The information output device,
Calculating the driver's seat position from the vehicle position detected by the position detecting means,
Detecting the direction in which the transmission type display unit of the head mounted display device is directed,
Identifying a position in the detected direction from the calculated driver's seat position,
An information display method, comprising: reading information associated with the storage means based on the specified position; and displaying the information on the transmission display unit. In an information display method in which an information output device displays information on a transmission type display unit included in a head mounted display device mounted on a head of a vehicle driver,
Search means for finding a route;
Position detecting means for detecting a vehicle position,
The information output device,
Calculating the driver's seat position from the vehicle position detected by the position detecting means,
Detecting the direction in which the transmission type display unit of the head mounted display device is directed,
Specifying a position range including the detected direction from the calculated driver's seat position,
Determining whether the specified position range includes the route searched by the search means,
When it is determined that the searched route is included in the position range, a mark indicating the searched route is created,
Displaying the created mark on the transmissive display unit.

Images