JP2019142297A - Method of detecting viewpoint of driver, device thereof, and head-up display device using the same - Google Patents

Abstract

To provide a method of detecting a viewpoint of a driver which uses a monocular camera that is easily mounted inside a vehicle and can be inexpensively manufactured, a device thereof, and a head-up display device using the same.SOLUTION: A device of detecting a viewpoint of a driver for detecting a viewpoint of a driver inside a vehicle includes: a monocular camera which is mounted by fixing at a position where a two-dimensional image including a face of the driver can be captured in a part in the front of a three dimensional space (an x-axis, a y-axis, a z-axis) inside the vehicle; and an operation part which detects the viewpoint of the driver from a two-dimensional camera image captured by the monocular camera, specifies a linear line going to the viewpoint of the driver detected in the three-dimensional space, and obtains a point where the specified linear line intersects with a vertical plane including the viewpoint of the driver for detecting three-dimensional viewpoint coordinates (x,y,z) of the driver.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a viewpoint detection method and apparatus for detecting the viewpoint of a driver, for example, and in particular, a driver viewpoint suitable for use in a head-up display device that projects / reflects an image on transparent glass or the like to display it as a virtual image. The present invention relates to a detection method, a device therefor, and a head-up display device using the same.

  For example, in vehicles such as automobiles, information such as vehicle speed and engine speed is usually displayed on an instrument panel in the dashboard. In addition, a screen such as a car navigation is incorporated in the dashboard or displayed on a display installed on the dashboard. When the driver (driver) visually recognizes such information, it is necessary to move the line of sight largely. Therefore, as a technique for reducing the amount of movement of the line of sight, information such as information on the vehicle speed, information such as instructions related to car navigation, etc. A head-up display (Head Up Display, hereinafter sometimes referred to as “HUD”) device that projects and displays the image on a windshield, a combiner, or the like is already known.

  As a technique related to the HUD, for example, Patent Document 1 below discloses a technique in which a HUD and a driver photographing camera that is a viewpoint detection camera are combined.

Japanese Patent Application Laid-Open No. 2015-92346

  However, the above-described prior art is particularly intended to provide a display device that easily obtains a sense of distance to an obstacle or the like, and has the following problems, that is, can be easily attached. However, there is no description about detecting a driver's viewpoint using a simple and inexpensive monocular camera.

  Therefore, in the present invention, a driver viewpoint detection method and apparatus using a monocular camera that can be easily installed in a vehicle and can be manufactured at low cost, and a head-up display device using the same Is to provide.

  The outline of typical ones of the present invention will be briefly described as follows.

In the present invention, as an example, there is a driver viewpoint detection method for detecting a driver's viewpoint in a vehicle, and a monocular camera is placed in front of a three-dimensional space (x axis, y axis, z axis) in the vehicle. A fixed two-dimensional image including a face of the driver is fixed at a position where the driver's face can be captured, and the driver's viewpoint is detected from the two-dimensional camera image captured by the monocular camera, and the monocular camera in the three-dimensional space A straight line from the detected driver to the viewpoint of the driver is identified, and a point at which the identified straight line intersects a vertical plane including the viewpoint of the driver is obtained, thereby obtaining the three-dimensional viewpoint coordinates (x _d , y _{d of the} driver _). , Z _d ), a driver viewpoint detection method is provided.

Further, in the present invention, as an example, a driver viewpoint detection device that detects a driver's viewpoint in a vehicle, in a part of a front of a three-dimensional space (x axis, y axis, z axis) in the vehicle. A monocular camera fixedly mounted at a position where a two-dimensional image including the driver's face can be imaged, and a two-dimensional camera image captured by the monocular camera, and detecting the driver's viewpoint in the three-dimensional space. By identifying a straight line directed to the detected driver's viewpoint and obtaining a point where the identified straight line intersects a vertical plane including the driver's viewpoint, the three-dimensional viewpoint coordinates (x _d , y _d , There is provided a driver viewpoint detection device including an arithmetic unit for detecting z _d ).

Furthermore, in the present invention, as a head-up display device using the method or apparatus described above, a video display device having a light source and a display element and forming an image on the display element, and light emitted from the video display device A virtual image optical system that displays a virtual image in front of the vehicle by reflecting the image with a windshield or combiner, a viewpoint detection device that detects the viewpoint of the driver, and light emission that adjusts the direction in which light is emitted from the video display device Direction adjustment means, and the viewpoint detection device can capture a two-dimensional image including the driver's face in a part of the front of a three-dimensional space (x axis, y axis, z axis) in the vehicle. A viewpoint of the driver is detected from a monocular camera fixedly attached at a certain position and a two-dimensional camera image captured by the monocular camera, and is detected in the three-dimensional space. Identifying a straight line toward the point of view of the detected driver, the by specified straight lines determine the point of intersection with a vertical plane including the viewpoint of the driver, the viewpoint coordinates (x _d of the three-dimensional of the driver, y _d, z _d ), and the light emission direction adjustment means is based on the three-dimensional viewpoint coordinates (x _d , y _d , z _d ) of the driver obtained by the calculation unit. A head-up display device is provided that adjusts a direction in which the light is emitted.

  According to the present invention, a driver viewpoint detection method and apparatus using a monocular camera that can be easily installed in a vehicle and can be manufactured at low cost, and further, a head-up using the same. It provides a practically excellent effect of providing a display device.

It is the figure which showed the outline | summary of the operation | movement concept of the head-up display apparatus for vehicles which concerns on one embodiment of this invention. It is explanatory drawing explaining adjustment of the position of the virtual image in the head-up display apparatus which concerns on one embodiment of this invention. It is a block diagram which shows an example of a structure of the viewpoint detection control part in the head-up display apparatus which concerns on one embodiment of this invention. It is a block diagram which shows an example of a structure of the HUD control part in the head-up display apparatus which concerns on one embodiment of this invention. (A), (b) is a figure for demonstrating the subject by the change of the driving | running state of the driver in the head-up display apparatus which concerns on one embodiment of this invention. (A), (b) is a figure for demonstrating the principle of the viewpoint detection method of the driver which concerns on one embodiment of this invention, and its apparatus. It is a figure for demonstrating the principle of the viewpoint detection method of the driver which concerns on one embodiment of this invention, and its apparatus. It is a figure for demonstrating the principle of the viewpoint detection method of the driver which concerns on one embodiment of this invention, and its apparatus. It is a figure for demonstrating the other subject in an actual head-up display apparatus. (A)-(c) is a figure for demonstrating the solution of another subject. It is the figure which showed the whole processing flow of the driver viewpoint detection in the head-up display apparatus which concerns on one embodiment of this invention, and the position adjustment of the concave mirror based on the result.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted. On the other hand, parts described with reference numerals in some drawings may be referred to with the same reference numerals although not illustrated again in the description of other drawings. In the following embodiments, details of an example in which the driver viewpoint detection method and apparatus of the present invention are applied mainly to a vehicle head-up display (HUD) device installed in a vehicle such as an automobile. Explained. The HUD device can be applied not only to vehicles but also to other vehicles such as trains and airplanes, and also to HUD devices used for purposes other than vehicles.

  Prior to the viewpoint detection method and apparatus of the present invention, a head-up display (HUD) device for a vehicle as an embodiment to which the driver viewpoint detection method and apparatus of the present invention is applied will be described first. FIG. 1 is a diagram showing an outline of an operation concept of the HUD device. In the HUD device 1 according to the present embodiment, an image displayed by the image display device 30 disposed in a casing (not shown) or detachable from the casing is displayed on a necessary optical element (here, a reflection). The light is reflected by the concave mirror 52 via the mirror 51) and projected onto the windshield 3 of the vehicle 2.

  Here, the member to be projected is not limited to the windshield 3 and may be another member such as a combiner as long as the image is projected. The video display device 30 is configured by, for example, a projector having a backlight, an LCD (Liquid Crystal Display), or the like. A self-luminous VFD (Vacuum Fluorescent Display) or the like may be used. An image may be displayed on a screen by a projection device. As such a screen, for example, a microlens array in which microlenses are two-dimensionally arranged may be used.

  The concave mirror 52 is constituted by, for example, a free-form surface mirror, a mirror having an optical axis asymmetric shape, or the like. More specifically, the shape of the concave mirror 52 is reduced in order to reduce the distortion of the virtual image, for example, in the upper region (that is, the light beam reflected here is reflected below the windshield 3, In the case where the distance from the driver's (driver) viewpoint 5 is short), the radius of curvature is relatively decreased so that the enlargement ratio is increased. On the other hand, in the lower region of the concave mirror 52 (that is, the light beam reflected here is reflected above the windshield 3 so that the distance from the driver's viewpoint 5 is relatively long), the enlargement ratio is reduced. The radius of curvature is made relatively large. By disposing the video display device 30 so as to be inclined with respect to the optical axis of the concave mirror 52, the difference in image magnification as described above may be corrected to reduce the generated distortion itself.

  The driver views the image projected as a virtual image in front of the transparent windshield 3 by viewing the image projected on the windshield 3. By adjusting the angle of the concave mirror 52 and adjusting the position at which the image is projected onto the windshield 3, the display position of the virtual image seen by the driver may be adjusted in the vertical direction. In addition, the content displayed as a virtual image is not specifically limited, For example, the vehicle information, navigation information, the image | video of the landscape image | photographed with the camera etc. which is not shown in figure etc. can be displayed suitably.

  Reference numeral 6 in the figure denotes a monocular camera that employs the driver viewpoint detection method of the present invention, and constitutes means for detecting the driver viewpoint. It should be noted that the monocular camera as the viewpoint detection means only needs to be able to detect the driver's viewpoint. For example, an example in which the viewpoint is attached to a part of the dashboard 61 will be described here, or a part of the windshield 3 is described. It may be attached to.

  Next, the adjustment of the position of the virtual image in the HUD device 1 whose configuration has been described above will be described in detail below with reference to FIG. Basically, in the HUD device of the present invention, a concave surface corresponding to the viewpoint 5 of the driver is obtained by the monocular camera 6 for viewpoint detection which is a means for detecting the viewpoint 5 of the driver attached to a part of the dashboard 61. By controlling the tilt angle of the mirror 52, the position of the virtual image projected on the windshield 3 of the vehicle 2 is adjusted / controlled. In the figure, corresponding to the positions A, B, and C of the viewpoint 5 of the driver, the inclined position of the concave mirror 52, the projection position of the virtual image on the windshield 3, and the position of the virtual image that can be seen from the viewpoint 5 of the driver Are indicated by A, B and C, respectively.

  In the control described above, for example, the communication unit 24 of the viewpoint detection control unit 20 shown in the block diagram of FIG. 3 notifies the HUD control unit 31 shown in FIG. 4 of the viewpoint information, and the HUD control unit 31 detects the concave mirror 52. It is realized by moving. In the viewpoint detection control unit 20, acquisition of information from each device such as the monocular camera 6 and the engine start sensor 109 is performed under the control of an ECU (Electronic Control Unit) (1) 21, for example. In the HUD control unit 31, the acquisition of the vehicle information 4, the control of the speaker 60, the control of the video display device 30, and the driving of the concave mirror 52 are performed under the control of the ECU (2) 32 of the HUD control unit 31. . However, the present invention is not necessarily limited to these, and other types of devices may be provided.

  As described above, the viewpoint detection system for HUD includes information acquisition devices such as various sensors installed in each part of the vehicle 2, detects various events occurring in the vehicle 2, and relates to the driving situation at predetermined intervals. The vehicle information 4 is acquired and output by detecting and acquiring values of various parameters. Although not shown here, the vehicle information 4 includes, for example, temperature information (temperature sensor), speed information (vehicle speed sensor), distance information (ranging sensor), camera image information (camera), engine Start information (engine start sensor) or the like may be included.

  The viewpoint detection control unit 20 has a function of notifying the detected viewpoint information to the HUD control unit 31, for example, a CPU (Central Processing Unit) (not shown) and software executed thereby (for example, a viewpoint detection control unit) 20 non-volatile memories 22 and 23). It may be implemented by hardware such as a microcomputer or FPGA (Field Programmable Gate Array). The detected viewpoint information is notified to the HUD control unit 31 via CAN or Ethernet (registered trademark) via the communication unit 24. Further, the HUD control unit 31 forms a video to be displayed as a virtual image by driving the video display device 30 based on the vehicle information acquired from the vehicle information acquisition unit 10, and reflects this by the concave mirror 52. Then project onto the windshield 3. At the same time, the mirror driving unit 45 controls the tilt angle of the concave mirror 52. The engine start sensor 109 is for grasping the engine start, and starts the system when the engine is started.

  Further, according to the viewpoint detection control unit 20 described above, the infrared LED 120 is lit to detect the driver's viewpoint at night via the infrared LED control unit 121.

  The HUD control unit 31 has a function of moving the concave mirror 52 based on the projection of the video based on the vehicle information 4 and the viewpoint information notified from the viewpoint detection control unit 20, and is executed by the CPU. (For example, stored in the nonvolatile memory 35 or the memory 36 of the HUD control unit 31 in FIG. 4). It may be implemented by hardware such as a microcomputer or FPGA.

<Problem in viewpoint detection system and driver viewpoint detection method of the present invention>
In the head-up display device described above, as shown in FIG. 2, the position of the virtual image to be displayed is moved according to the viewpoint 5 of the driver.

  Therefore, in the conventional viewpoint detection system, when the viewpoint coordinates (u, v) detected by detecting the viewpoint 5 of the driver are converted into 3D viewpoint coordinates (x, y, z), the imaging position of the monocular camera 6 is set. It was necessary to set the corresponding parameters.

  In such a viewpoint detection system using a monocular camera, distance measurement from the camera to the face (viewpoint) is calculated based on the face size detected by the face detection function of the monocular camera. It has been pointed out that the accuracy is poor due to fluctuations. As one means for solving this problem with a viewpoint detection system using a monocular camera, it is conceivable to perform distance measurement up to the driver's face (viewpoint) using a plurality (two or more) of monocular cameras. If the distance can be measured with one monocular camera, a merit in cost can be obtained.

  Therefore, the present invention proposes a driver's viewpoint detection method and apparatus that are sufficiently accurate even when using a monocular camera, and can therefore be easily installed in a vehicle and inexpensively. A viewpoint detection system that can be manufactured is realized, and further, a head-up display device using the same is provided. More specifically, for example, as shown in FIG. 5, even when the driver shifts from the normal driving state (FIG. 5A) to the forward bending state (FIG. 5B), the monocular camera 6 It becomes possible to identify and detect the viewpoint 5 of the driver in the vehicle from the image.

  Next, a driver viewpoint detection method using the monocular camera of the present invention will be described in detail below.

<Arrangement and coordinates of monocular camera>
As shown in FIG. 6A, a position where the driver's face (viewpoint) can be imaged with the monocular camera 6 inside the vehicle, for example, a part of the dashboard 61 on the front side of the driver (right corner) in this example. Place it fixed to. At this time, the monocular camera 6 is arranged at the center in the three-dimensional space coordinates inside the vehicle, the x-axis is in the vehicle length direction (vehicle traveling direction), the y-axis is in the vehicle width direction, and the z-axis is in the vehicle. Set in the height (vertical) direction. FIG. 6B shows the situation when this state is viewed from above. That is, as is apparent from the figure, the monocular camera 6 captures an image including the face (viewpoint) from the front of the driver in a three-dimensional space in the vehicle centering on the monocular camera 6 (shown by shading in the figure). (See “Shooting Range”).

  Next, in FIG. 7, based on the above assumption, first, an eye (viewpoint) is detected from a two-dimensional camera image captured by the monocular camera 6. Subsequently, a straight line from the monocular camera 6 (the origin of the three-dimensional space) toward the driver's eye (viewpoint) 5 (that is, connecting the monocular camera 6 and the viewpoint) is obtained. As a result, it is possible to specify a straight line directed toward the driver's eyes (viewpoint) 5 when viewed from the origin of the three-dimensional space that is the monocular camera 6. In other words, if one point on the camera image (two-dimensional) (in the above, the eye position of the driver) is designated from the relationship between the installation direction of the monocular camera 6 and the angle of view, the monocular camera 6 and one point on the image A straight line in the three-dimensional space connecting However, it is not possible to limit where the eyes are located on the straight line.

  Subsequently, as shown in FIG. 8, the three-dimensional straight line obtained above and the three-dimensional vertical plane (z-x plane) passing through the seat center (or the central axis of the steering wheel 7) An intersection is obtained, and the obtained intersection is a viewpoint position that is the eyes of the driver. A three-dimensional vertical plane (zx plane) passing through the seat center (center axis of the steering wheel 7) is a distance L from the monocular camera 6 (origin) to the seat center in the vehicle width direction (y-axis). Zx planes that are separated from each other, and the distance can be easily obtained by setting or measuring in advance when the monocular camera 6 is placed inside the vehicle.

That is, in the HUD device 1, although not shown in the figure, the HUD control unit 31 in FIG. 4 starts processing when the power source such as an ignition is turned on, and the driver imaged by the monocular camera 6 is started. Based on the face, the viewpoint coordinates (x _d , y _d , z _d ) of the driver are obtained by the viewpoint detection method of the present invention described above, and a series of processes for moving the concave mirror 52 based on the obtained viewpoint is repeated. .

  By the way, in the viewpoint detection method described above, the driver's face is usually based on the premise that the face of the driver is located on the plane when viewed from above. However, in the actual HUD device 1, FIG. As shown, the face may not be in the center of the seat. In this case, as indicated by a broken line or a one-dot chain line in the figure, a difference (displacement) occurs between the viewpoint position that is the driver's eyes and the center of the seat, and the virtual image is obtained by adjusting the angle of the concave mirror 52 by the viewpoint detection system. In some cases, the display position adjustment is not performed properly.

Accordingly, the orientation of the driver's face is estimated from the image captured by the monocular camera 6, and it is determined whether the driver's face is located at the center of the seat based on the estimated face orientation. It is proposed to perform a position adjustment. For example, as shown in FIG. 10A, the direction of the driver's face (the directions of the three directions of roll, pitch, and yaw as shown by the arrows in the figure) is based on the image captured by the monocular camera 6. If it is estimated and the face is determined to face the front of the vehicle, the display position of the virtual image by adjusting the angle of the concave mirror 52 based on the driver's viewpoint coordinates (x _d , y _d , z _d ) obtained above. As shown in FIGS. 10B and 10C, when the driver's face is deviated from the front of the vehicle, the virtual image display position is adjusted by adjusting the angle of the concave mirror 52. Do not.

  More specifically, FIG. 11 shows the processing in the viewpoint detection system in detail including the detection of the position of the driver's eye (viewpoint) 5 described above. This process is executed based on software stored in the nonvolatile memory 22 or the memory 23 in the ECU (1) 21 constituting the viewpoint detection control unit 20.

As shown in FIG. 11, this process is started by turning on the power supply (ignition) at the time of starting the engine. Is detected (step S111). Then, a straight line from the origin of the three-dimensional space where the monocular camera 6 is arranged to the viewpoint (eyes) 5 detected on the two-dimensional image is specified (step S112). Further, the viewpoint coordinates (x _d , y _d , z _d ) of the driver are obtained from the intersection of the straight line and the vertical plane (z-x plane) passing through the seat center (step S113).

Thereafter, the display position adjustment of the virtual image is performed by adjusting the angle of the concave mirror 52 by the viewpoint detection system based on the obtained viewpoint coordinates (x _d , y _d , z _d ) of the driver. As described above, the direction of the driver's face (inclination with respect to the front of the vehicle) is estimated from the two-dimensional image captured by the monocular camera 6 by the existing image processing technique to determine the direction (roll, pitch, yaw) in three directions ( Step S114). Thereafter, these three orientations (roll, pitch, yaw) are compared with a predetermined range (for example, ± 10 °), and the orientations in the respective directions are within a predetermined allowable range. It is determined whether or not. Specifically, it is determined whether or not the angle in the roll direction is within a predetermined range (step S115), whether or not the angle in the pitch direction is within the predetermined range (step S116), and It is determined whether or not the direction angle is within a predetermined range (step S117).

As a result, when it is determined that the obtained orientations in the three directions are within a predetermined range (Yes), the driver's viewpoint coordinates are obtained assuming that the driver's face is facing the front of the vehicle. The virtual image display position adjustment is performed by adjusting the angle of the concave mirror 52 based on the viewpoint coordinates (x _d , y _d , z _d ) of the driver (step S118), the process returns to step S111, and a series of processing is repeated again. . On the other hand, if it is determined that the predetermined range set in advance in at least any one of the three directions is outside (No), the driver's face is assumed to face other than the front of the vehicle, and the concave mirror Without adjusting the display position of the virtual image by the angle adjustment of 52, the process returns to the above step S111, and the series of processes is repeated again.

  In the above description, it has been described that the two-dimensional camera image is captured by the monocular camera 6 under natural light. However, it is possible to capture the necessary two-dimensional camera image by increasing the sensitivity of the monocular camera 6 at night or in a dark environment in a tunnel. In addition, the viewpoint of the driver is detected at night as described above. It is also possible to use the infrared LED 120 of the viewpoint detection control unit 20 (see FIG. 3) that is turned on for this purpose. In this case, it goes without saying that the infrared LED 120 is turned on via the infrared LED control unit 121.

  As is clear from the detailed description given above, according to the processing in the viewpoint detection system described above, a monocular camera that can be easily installed in a vehicle and that is advantageous from the viewpoint of cost is used. By providing such a viewpoint detection method and apparatus, it is possible to provide a viewpoint detection system that can be manufactured at low cost, and further, Can provide a head-up display device using the same.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiments. However, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention. Needless to say. For example, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to the one having all the configurations described. A part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. . Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  The present invention is particularly applicable to a head-up display device that displays a virtual image to a driver via a windshield of a vehicle.

  DESCRIPTION OF SYMBOLS 1 ... HUD apparatus, 2 ... Vehicle, 3 ... Wind shield, 4 ... Vehicle information, 5 ... Driver's viewpoint, 6 ... Monocular camera, 61 ... Dashboard, 7 ... Steering wheel, 20 ... Viewpoint detection control part, 21 ... ECU (1), 22 ... Non-volatile memory, 23 ... Memory, 24 ... Communication unit, 30 ... Video display device, 31 ... HUD control unit, 32 ... ECU (2), 51 ... Reflective mirror, 52 ... Concave mirror, 35 ... Non-volatile memory, 36 ... memory, 45 ... mirror drive unit, 60 ... speaker, 109 ... engine start sensor, 120 ... infrared LED, 121 ... infrared LED control unit.

Claims (10)

A monocular camera is fixedly attached to a position in front of a three-dimensional space (x axis, y axis, z axis) in a vehicle at a position where a two-dimensional image including a driver's face can be captured,
Detecting a viewpoint of the driver from a two-dimensional camera image captured by the monocular camera;
Identifying a straight line from the monocular camera toward the detected driver's viewpoint in the three-dimensional space;
A driver viewpoint detection method of detecting a three-dimensional viewpoint coordinate (x _d , y _d , z _d ) of the driver by obtaining a point where the identified straight line intersects a vertical plane including the driver viewpoint. The driver viewpoint detection method according to claim 1,
The length direction of the vehicle is the x-axis of the three-dimensional space in the vehicle, the width direction of the vehicle is the y-axis, and the height direction of the vehicle is the z-axis,
The vertical plane is a three-dimensional vertical plane (zx plane) passing through the center of the driver's seat,
The driver's viewpoint detection method, wherein the monocular camera is arranged in the y-axis direction by a predetermined distance with respect to the vertical plane (zx plane). A monocular camera fixedly mounted at a position where a two-dimensional image including a driver's face can be captured in a part of the front of a three-dimensional space (x-axis, y-axis, z-axis) in the vehicle;
A vertical plane in which the driver's viewpoint is detected from a two-dimensional camera image captured by the monocular camera, a straight line directed to the detected driver's viewpoint is identified in the three-dimensional space, and the identified straight line includes the driver's viewpoint. A driver viewpoint detection device comprising: an arithmetic unit that detects a three-dimensional viewpoint coordinate (x _d , y _d , z _d ) of the driver by obtaining a point that intersects with the driver. The driver viewpoint detection device according to claim 3,
The vehicle length direction is the x-axis of the three-dimensional space in the vehicle, the vehicle width direction is the y-axis, and the height direction is the z-axis,
The vertical plane is a three-dimensional vertical plane (zx plane) passing through the center of the driver's seat,
The driver's viewpoint detection device, wherein the monocular camera is arranged in the y-axis direction by a predetermined distance with respect to the vertical plane (zx plane). An image display device having a light source and a display element, and forming an image on the display element;
A virtual image optical system that displays a virtual image in front of a vehicle by reflecting light emitted from the video display device with a windshield or a combiner;
A viewpoint detection device for detecting the viewpoint of the driver;
A light emission direction adjusting means for adjusting a direction of emitting light from the video display device,
The viewpoint detection device includes:
A monocular camera fixedly mounted at a position where a two-dimensional image including the face of the driver can be captured in a part of the front of a three-dimensional space (x axis, y axis, z axis) in the vehicle;
A vertical plane in which the driver's viewpoint is detected from a two-dimensional camera image captured by the monocular camera, a straight line directed to the detected driver's viewpoint is identified in the three-dimensional space, and the identified straight line includes the driver's viewpoint. An arithmetic unit that detects the three-dimensional viewpoint coordinates (x _d , y _d , z _d ) of the driver by obtaining a point that intersects with
The light emission direction adjusting means adjusts the direction in which the light is emitted based on the three-dimensional viewpoint coordinates (x _d , y _d , z _d ) of the driver obtained by the calculation unit. . The head-up display device according to claim 5,
The vehicle length direction is the x-axis of the three-dimensional space in the vehicle, the vehicle width direction is the y-axis, and the height direction is the z-axis,
The vertical plane is a three-dimensional vertical plane (zx plane) passing through the center of the driver's seat,
The monocular camera is a head-up display device that is arranged in the y-axis direction by a predetermined distance with respect to the vertical plane (z-x plane). The head-up display device according to claim 6,
A face direction estimating means for estimating an inclination of the driver's face relative to the front of the vehicle from a two-dimensional camera image captured by the monocular camera;
The light emitting direction adjusting unit adjusts a direction in which the light is emitted based on an inclination of the driver's face with respect to the front of the vehicle estimated by the face direction estimating unit. The head-up display device according to claim 7,
When the inclination of the driver's face with respect to the front of the vehicle estimated by the face direction estimating unit is within a predetermined allowable range, the light emission direction adjusting unit determines the three-dimensional viewpoint coordinates (x _The light emission direction is adjusted when the direction of light emission is adjusted based on ( _d , y _d , z _d ), and the inclination of the driver's face relative to the front of the vehicle estimated by the face direction estimation means is outside a predetermined range. Head-up display device that does not adjust. The head-up display device according to claim 8,
The head-up display device, wherein the inclination of the driver's face with respect to the front of the vehicle estimated by the face direction estimation means includes at least one inclination angle of a roll direction, a pitch direction, and a yaw direction with respect to the front of the vehicle. The head-up display device according to claim 9, wherein
The head-up display device, wherein an allowable range of inclination of the driver's face with respect to the front of the vehicle is ± 10 °.

Images