JP2019038342A - Image processing unit and head-up display device including the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid deterioration of visibility of a virtual image. An image processing unit applied to a display controller 200 includes a generation unit 230 that generates a display image including a display element corresponding to a virtual image visually recognized by a user. The virtual image on the virtual plane S is visually recognized by the user as if the virtual plane S is substantially parallel to the road surface H and the virtual plane S rises above the road surface H by a predetermined height. The generation unit 230 determines the planar shape of the display element so that the size of the virtual image visually recognized by the user changes based on the height of the viewpoint of the user. Specifically, when the object approaches the user's viewpoint 102 in the own vehicle and the user's viewpoint height changes to a low position, the generation unit 230 determines the size of the plane shape of the virtual image on the virtual plane S. Decide to change direction. [Selection diagram] Fig. 1

Description

  The present invention relates to an image processing unit capable of generating a display image based on a user's viewpoint position, a head-up display device including the same, and the like.

  For example, the head-up display device of Patent Document 1 recognizes a lane marking such as a white line on a road existing in front of the vehicle, and displays corresponding to the virtual image so that the lane marking (actual scenery) and the virtual image overlap. Images can be projected or displayed on the windshield (windscreen) of the vehicle. In particular, the head-up display device of Patent Document 1 is linked with a driving support device such as a lane departure warning device (LDW device) or a lane maintenance assist device (LKA device), and indicates that the driving support device is active by a virtual image. . Specifically, as shown in FIGS. 3 and 4 of Patent Document 1, for example, the virtual image is displayed on the vehicle side as display elements M1, M2, M3, M4, M5, and M6, and the user (driver). Visible to.

JP 2016-110627 A

  According to the description in paragraph [0018] of Patent Document 1, the driver's eye position (viewpoint position) is calculated by analyzing the driver's face image captured by the in-vehicle camera. And according to the viewpoint position obtained by the analysis, the projection positions of the display elements M1 to M6 are adjusted so that the display elements M1 to M6 are visually recognized superimposed on the lane markings R1 to R6. However, when the viewpoint position is lowered, the present inventors have recognized that it is difficult for the user to visually recognize the virtual image, that is, the visibility is lowered.

  Specifically, in the example of Patent Document 1, the user is caused to visually recognize that the display elements M1 to M6 are substantially parallel to the road surface and the display elements M1 to M6 are raised by a predetermined height from the road surface. Assuming that, the present inventors recognized a decrease in the visibility of the display elements M1 to M6 and / or the partition lines R1 to R6.

  One object of the present invention is to provide an image processing unit capable of avoiding a decrease in visibility of a virtual image and a head-up display device including the image processing unit. Other objects of the present invention will become apparent to those skilled in the art by referring to the aspects and best embodiments exemplified below and the accompanying drawings.

  In the following, in order to easily understand the outline of the present invention, embodiments according to the present invention will be exemplified.

In the first aspect, the image processing unit includes:
A display image including a display element corresponding to a virtual image on the virtual plane that is visually recognized by a user such that the virtual plane is substantially parallel to the road surface and the virtual plane is raised by a predetermined height from the road surface; It has a generation part to generate,
The generation unit determines a planar shape of the display element so that a size of the virtual image visually recognized by the user changes based on a viewpoint height of the user,
When the viewpoint height changes from a reference position to a position lower than the reference position, the generation unit is configured to cause the planar shape of the virtual image on the virtual plane to move in a predetermined direction. Determine the planar shape.

  In the first aspect, when the viewpoint height decreases, the planar shape of the virtual image on the virtual plane moves in a predetermined direction. Accordingly, the visibility of the virtual image on the virtual plane can be avoided by the amount of movement.

In a second aspect dependent on the first aspect,
The predetermined direction may be a direction in which the planar shape of the virtual image on the virtual plane approaches a vehicle having a projection member onto which the virtual image is projected.

  In the second aspect, when the viewpoint height decreases, the planar shape of the virtual image on the virtual plane approaches the vehicle, and visibility can be improved.

In a third aspect subordinate to the second aspect,
The predetermined direction may be a direction from the reference point of the planar shape of the virtual image on the virtual plane toward the user driving the vehicle.

  In the third aspect, when the viewpoint height decreases, the planar shape of the virtual image on the virtual plane approaches the user (driver), and the visibility can be effectively improved.

In a fourth aspect subordinate to any one of the first to third aspects,
The planar shape of the virtual image on the virtual plane may be a shape surrounding an object that can be recognized by the user,
The predetermined direction is a direction in which an area on the virtual plane is defined between a boundary between the object and the road surface and an outer edge of the planar shape of the virtual image on the virtual plane. May be.

  In the fourth aspect, when the viewpoint height decreases, the gap (area on the virtual plane) between the boundary between the object and the road surface (outer edge of the object) and the outer edge of the virtual image on the virtual plane The visibility can be further improved by increasing.

In a fifth aspect subordinate to any one of the first to fourth aspects,
The generation unit may dynamically determine an amount of movement of the planar shape of the virtual image on the virtual plane in response to a change in the viewpoint height.

  In the fifth aspect, when the viewpoint height changes, the visibility can be improved in response to the change.

In a sixth aspect subordinate to the fifth aspect,
When the amount of change in the viewpoint height per unit time is equal to or greater than a reference value, the generation unit weakens or invalidates display of the display element corresponding to the virtual image moving in the predetermined direction on the virtual plane. Also good.

  In the sixth aspect, when the change in the viewpoint height is large, the improvement in visibility can be stopped.

In a seventh aspect dependent on the fifth or sixth aspect,
The generation unit changes the planar shape of the display element so that the movement amount of the planar shape of the virtual image on the virtual plane is zero only while the user's line-of-sight direction is facing the virtual image. You may decide.

  In the seventh aspect, the improvement in visibility can be stopped only when the user's line-of-sight direction faces the display element (virtual image). In other words, since the user cannot visually recognize the movement of the virtual shape of the virtual image on the virtual plane, it is possible to avoid erroneous information presentation to the user.

In an eighth aspect dependent on any one of the fifth to seventh aspects, the image processing unit includes:
When the amount of movement of the planar shape of the virtual image on the virtual plane is not zero, the user may be notified that the amount of movement has occurred as the viewpoint height decreases.

  In the eighth aspect, it is possible to notify the movement of the virtual image according to the viewpoint height of the user.

In a ninth aspect subordinate to any one of the first to fourth aspects,
The viewpoint height may be a fixed value acquired when the vehicle is started,
The generation unit may determine a movement amount of the planar shape of the virtual image on the virtual plane based on the fixed value.

  In the ninth aspect, the visibility during driving of the vehicle can be improved with the viewpoint height (fixed value) acquired when the vehicle is started.

In a tenth aspect subordinate to any one of the first to ninth aspects,
When the viewpoint height changes from the reference position to a position higher than the reference position, the generation unit displays the display so that the movement amount of the planar shape of the virtual image on the virtual plane is zero. The planar shape of the element may be determined.

  In the tenth aspect, when the viewpoint height increases, the planar shape of the virtual image on the virtual plane does not have to be moved. In other words, the visibility does not decrease when the viewpoint height increases.

In an eleventh aspect, the head-up display device comprises:
The image processing unit according to any one of the first to tenth aspects;
A display for displaying the display image generated by the image processing unit;
Is provided.

  In the eleventh aspect, the head-up display device can display a display image including a display element corresponding to a virtual image having a planar shape that moves in a predetermined direction on the virtual plane when the viewpoint height decreases. Accordingly, it is possible to avoid a decrease in visibility by the amount of movement.

  Those skilled in the art will readily understand that the illustrated embodiments according to the present invention can be further modified without departing from the spirit of the present invention.

FIG. 1A shows a schematic configuration example of an image processing unit according to the present invention and a head-up display device including the image processing unit, and FIG. 1B is an explanatory diagram of a virtual image display surface and a virtual plane of FIG. Indicates. FIG. 2B shows a modification of the virtual image display surface of FIG. 1A, and FIG. 2B shows an explanatory diagram of the virtual image display surface and the virtual plane of FIG. 3A shows an example of an object (actual scenery) recognized through the windshield, and FIG. 3B shows a virtual image (display element) superimposed on the object of FIG. 3A. FIG. 3C shows an explanatory diagram (plan view) of a planar shape of a virtual image in a real space or a planar shape on a virtual plane. 4A shows an example of a virtual image recognized by the user, and FIG. 4B shows an example of a change in the size of the virtual image recognized by the user when the viewpoint height decreases. 4 (C) shows an example of the virtual image moved on the virtual plane, and FIG. 4 (D) is an explanatory diagram (plan view) of the planar shape of the moved virtual image in the real space or the planar shape on the virtual plane. Figure). 5 (A) shows a plurality of setting examples of the movement amount of the planar shape of the virtual image in the real space or the planar shape on the virtual plane based on the viewpoint height of the user, and 5 (B) shows the distance to the vehicle. The setting example of the movement amount of the planar shape of the virtual image in the real space or the planar shape on the virtual plane based on the above is shown. FIG. 6A shows an example of a virtual image recognized by the user when the viewpoint height decreases, and FIG. 6B shows a planar shape or virtual plane of the virtual image of FIG. 6A in a side view. An explanatory diagram of the upper planar shape is shown. FIG. 7A shows an example of a virtual image moved on a virtual plane that is recognized by the user when the viewpoint height is lowered, and FIG. 7B is a side view of FIG. FIG. 7C illustrates another modified example of the virtual image of FIG. 7A. FIG. 7C illustrates an explanatory diagram of the planar shape of the virtual image or the planar shape on the virtual plane. FIG. 8A shows an explanatory diagram of the positional relationship between the object (the vehicle ahead) and the host vehicle, and FIG. 8B shows an explanatory diagram of a setting example of a predetermined direction.

  The best mode described below is used to easily understand the present invention. Accordingly, those skilled in the art should note that the present invention is not unduly limited by the embodiments described below.

  FIG. 1A shows a schematic configuration example of a head-up display device (or head-up display system) including an image processing unit according to the present invention. As shown in FIG. 1A, the head-up display system includes, for example, an imaging device 103, an image processing unit 300 that processes a captured image from the imaging device 103, and a display based on information from the image processing unit 300. The display controller 200 which processes the display image to 20 and controls the display device 20 based on the display image, and the display mechanism 100 including, for example, the display device 20 and the reflector 21 can be provided. Note that the processing unit 210 of the display controller 200 in FIG. 1A does not have a function of controlling the display device 20, and the function thereof is another device (not shown) or, for example, the display shown in the figure. It may be realized by the device 20 or the image processing unit 300.

  In this specification, an apparatus having the function of the generation unit 230 that generates a display image including a display element corresponding to a virtual image visually recognized by a user can be referred to as an image processing unit according to the present invention. As an example, since the generation unit 230 is applied to the display controller 200 in FIG. 1A, the display controller 200 including the generation unit 230 is an image processing unit. The image processing unit essentially has a function of processing a display image (generation unit 230), but further has a function of processing a captured image (object information acquisition unit 310 and / or viewpoint information acquisition unit 320). May be. In other words, when the image processing unit 300 includes the function of the generation unit 230 that generates a display image including display elements, the image processing unit 300 can be called an image processing unit according to the present invention.

  An apparatus comprising an image processing unit according to the present invention (for example, the display controller 200 in FIG. 1A) and a display 20 that displays a display image generated by the image processing unit is a head-up display device according to the present invention. Can be called.

  In FIG. 1A, the imaging device 103 is, for example, a first imaging unit 103-1 that can image a road (an actual landscape on the road) that exists in front of the vehicle, and a user, for example. For example, it has the 2nd imaging part 103-2 which can image a driver | operator's face. The display mechanism 100 projects a display image (including display elements) onto a display range AR (projection member) set in, for example, a part of the windshield 101 of the vehicle, and displays the display element from the viewpoint 102 of the driver of the vehicle. The display image can be displayed in the display range AR so that the virtual image on the virtual image display surface V overlaps with the actual scenery (for example, the vehicle ahead). The display mechanism 100 and the display controller 200 are typically housed inside the dashboard, but a part of the display mechanism 100 and the display controller 200 may be disposed outside the dashboard. The display device 20 may be called a projector because it projects a display element or a display image onto the windshield 101, for example.

  The image processing unit 300 in FIG. 1A analyzes the captured image from the first imaging unit 103-1, and recognizes an object such as a forward vehicle, a pedestrian, and a white line superimposed on the display element from the captured image. A possible object information acquisition unit 310 may be included. Preferably, the object information acquisition unit 310 acquires position information of the recognized object (for example, a relative three-dimensional position in real space when the host vehicle is used as a reference), and more preferably, the recognized object position. The size (for example, the volume of the object in real space) can be acquired. The first imaging unit 103-1 is, for example, a stereo camera, and the object information acquisition unit 310 determines the distance between the object and the vehicle (own vehicle) (relative distance from the own vehicle to the object) from the left and right captured images. , The position of the object on the road surface H (horizontal plane) and the vehicle (own vehicle) (relative distance from the own vehicle to the object), and the length of three sides (the width and depth of the object) that define the rectangular parallelepiped surrounding the object Height), and can be obtained.

  The image processing unit 300 in FIG. 1A includes a viewpoint information acquisition unit 320 that analyzes the captured image from the second imaging unit 103-2 and recognizes the driver's eyes from the captured image. it can. Preferably, the viewpoint information acquisition unit 320 is the position information of the recognized eye (for example, in a real space when a predetermined position between the ceiling or the floor of the vehicle interior or the floor is used as a reference). Relative one-dimensional position (viewpoint height)), and more preferably, the recognized eye direction (gaze direction) can be acquired.

  The imaging device 103 is installed, for example, in the vicinity of an inner rear view mirror (back mirror) (not shown). As an example, the first imaging unit 103-1 is located on the windshield 101 side of the inner rear view mirror, and the second imaging unit 103-2 is located on the driver seat (not shown) side of the inner rear view mirror. To do.

  A display controller 200 in FIG. 1A includes a processing unit 210. The processing unit 210 includes a generation unit 230, and preferably may further include, for example, a determination unit 220. The processing unit 210 or the determination unit 220 can determine whether or not the driver's viewpoint 102 acquired by the viewpoint information acquisition unit 320 changes in the vertical direction. The determination unit 220 can determine whether or not the change is large when the viewpoint height changes. In addition, the determination unit 220 can determine whether or not the driver's line-of-sight direction acquired by the viewpoint information acquisition unit 320 faces the display element.

  Next, the processing unit 210 or the generation unit 230 generates a display image including a display element corresponding to the virtual image visually recognized by the user. Preferably, the virtual image of the display element is an actual scene at the viewpoint 102 of the vehicle driver. A display image including the display element is generated so as to overlap (for example, the preceding vehicle acquired by the object information acquisition unit 310).

  Specifically, the generation unit 230 can determine the position information and the size of the display element based on the viewpoint height of the driver who is a user, for example, and the position information of the front vehicle which is a recognized object, for example. it can. More specifically, when the display element is visually recognized by the user, the generation unit 230 makes the virtual image corresponding to the display element have a planar shape in real space and does not depend on the viewpoint height of the user. Thus, the planar shape of the display element is determined so that the planar shape of the virtual image in the real space is the same. In other words, the generation unit 230 changes the size of the display element superimposed on the front vehicle when, for example, the size of the front vehicle (actual scenery) viewed by the user changes according to the viewpoint height of the user. However, the planar shape of the display element is determined according to the viewpoint height of the user so as to be equal to the change in the size of the vehicle ahead. That is, the generation unit 230 does not depend on the viewpoint height of the user, for example, the relative positional relationship in the real space between the front vehicle (actual landscape) and the display element that are visually recognized by the user and the size in the real space. The position of the display element and the appearance shape in the display image are determined according to the viewpoint height of the user so that the relationship does not change.

  However, the present inventors have recognized that the visibility of virtual images decreases when the user's viewpoint height decreases. Although details will be described later, the generation unit 230 determines the planar shape of the display element according to the user's viewpoint height in order to avoid a decrease in visibility, and then when the user's viewpoint height decreases, The planar shape of the display element is determined so that the planar shape of the virtual image on the virtual plane moves in a predetermined direction. Therefore, the visibility of the virtual image of the display element is improved by the amount of the movement.

  The display controller 200 of FIG. 1A further includes, for example, a storage unit 240, and the storage unit 240 stores various data necessary for processing or calculation of the processing unit 210 such as the determination unit 220 and the generation unit 230, for example. be able to. The display controller 200 is typically configured by a microcomputer, for example, and can include, for example, a CPU, a memory such as a ROM and a RAM, an input / output interface, and the like. The processing unit 210 is typically configured with a CPU and a RAM (work area), and the storage unit 240 is typically configured with a ROM (eg, EEPROM). For example, the ROM may store a program that causes the CPU to execute a predetermined operation (a method of generating a display image based on the user's viewpoint 102), and the RAM can form a work area of the CPU. The ROM can store data necessary for determining or calculating the shape of the display element, for example.

  The display controller 200 is disposed or mounted on a vehicle and is connected to, for example, the image processing unit 300 or the like via an in-vehicle network LAN that is, for example, a CAN (Controller Area Network). The display controller 200 may generally be referred to as an ECU (Electronic Control Unit).

  The display controller 200 in FIG. 1A may input information from the information acquisition unit 400, for example, and generate a display image including a display element corresponding to a virtual image visually recognized by the user based on the information. Specifically, the processing unit 210 or the generation unit 230 is, for example, a road surface based on the position (for example, current position) of the vehicle (own vehicle) via the in-vehicle network LAN, the road information acquisition unit 410, and the position information acquisition unit 420. Information on the gradient of H may be input.

  Note that the projection position of the display element generated based on the information from the information acquisition unit 400 may be fixed to a part of the display range AR set on the windshield 101, and may be fixed to a predetermined object (actual landscape). It does not have to be superimposed. For example, when the road information includes sign information, a road sign based on the current position is input to the generation unit 230, and the display element representing the road sign is the display range AR set in the windshield 101. It may be fixed and displayed on a part of.

  Moreover, the process part 210 or the production | generation part 230 may input the speed of a vehicle (own vehicle) via the vehicle-mounted network LAN and the vehicle speed information acquisition part 430, for example. The speed of the vehicle (own vehicle) based on the current time is input to the generation unit 230, and the display element indicating the speed of the vehicle is fixed and displayed in a part of the display range AR set in the windshield 101. May be. In other words, the head-up display device fixes the instrument information such as the speed of the vehicle regardless of the position of a predetermined object (actual landscape) when displaying the display image, while the obstacle such as the vehicle ahead, the pedestrian, etc. The object information may be changed according to the position of a predetermined object (actual scenery) [0]. The display controller 200 may not input information from the information acquisition unit 400 such as the road information acquisition unit 410, the position information acquisition unit 420, and the vehicle speed information acquisition unit 430.

  In FIG. 1A, the display mechanism 100 or the display device 20 is controlled by the display controller 200 or the processing unit 210 so that the user can visually recognize the virtual image on the virtual image display surface V through the windshield 101. The display device 20 can generate or emit display light L (projection light) based on a display image including a display element corresponding to a virtual image. The reflector 21 guides the optical path of the display light L from the display device 20 to the windshield 101, and the driver can recognize the display light L (display element) on the windshield 101 as a virtual image. The display 20 includes a light source unit such as an LED, a display element such as a DMD (Digital Micromirror Device), a screen that receives light from the display element, and displays a display image (including display elements). .

  A vehicle (own vehicle) on which a display controller 200 that is an image processing unit and a head-up display device including, for example, a display mechanism 100 is mounted is, for example, an automobile in FIG. Is possible. From the viewpoint 102 of the driver, the height or depth (superimposition distance) of the virtual image display surface V in FIG. 1A is set to a predetermined distance from a distance D1 to a distance D3, for example. In other words, at the driver's viewpoint 102, the lower end of the virtual image display surface V exists forward by a distance D1 from the vehicle, and the upper end of the virtual image display surface V exists forward by a distance D3 from the vehicle. The midpoint in the up and down direction is ahead of the vehicle by a distance D2. The distance D1, the distance D2, and the distance D3 are, for example, 20 [m], for example, 30 [m], and for example, 50 [m].

  Although one reflector 21 is shown in FIG. 1A, the reflector 21 can be composed of, for example, two reflectors, in other words, display light from the display 20 to the windshield 101. The optical path length of L may be set by the reflector 21, and the distance D1, the distance D2, and the distance D3 are the ideal driver viewpoint height (ideal driver seat height) and display light. It may be set according to the optical path length of L. Further, the reflector 21 generally expands the display light L from the display device 20, and in addition, the reflector 21 or the display controller 200 (for example, the processing unit 210) is generally set as a part of the windshield 101. Distortion in the display range AR (for example, distortion of the glass surface) can be corrected.

  FIG. 1B is an explanatory diagram of the virtual image display surface and the virtual plane of FIG. The virtual image of the display element is recognized, for example, on the virtual image display surface V that is raised by a predetermined height from the road surface H. For example, the virtual image display surface V is set on the virtual plane S that is raised by a predetermined height from the road surface H. Is done. Note that the virtual image display surface V may not be set parallel to the road surface H and may be inclined by a predetermined angle from the road surface H. The inclination of the virtual image display surface V is set by the angle of the screen (display surface on which the display device 20 displays a display image) or the reflector 21.

  An actuator (not shown) that can rotationally drive the reflector 21 may be provided in the reflector 21. In this case, the processing unit 210 of the display controller 200 controls the actuator to control the virtual image display surface V. The inclination may be adjusted. In the example of FIG. 1A or FIG. 1B, the virtual image display surface V is substantially parallel to the road surface H, and the inclination of the virtual image display surface V from the road surface H is 0 [degree] as an example. ] To 30 [degree] may be set.

  In FIG. 1A or FIG. 1B, the virtual image display surface V rises from the road surface H by a predetermined height, but may be set to the road surface H. In addition, the virtual plane S rises from the road surface H by a predetermined height.

  FIG. 2B shows a modification of the virtual image display surface of FIG. 1A, and FIG. 2B shows an explanatory diagram of the virtual image display surface and the virtual plane of FIG. In the example of FIG. 2A or 2B, the virtual image display surface V ′ is substantially perpendicular to the road surface H, and the inclination θ ′ of the virtual image display surface V from the road surface H is, for example, It may be set in a range from 80 [degree] to 90 [degree]. In FIG. 2A, the actual landscape range (overlapping distance) superimposed by the virtual image on the virtual image display surface V ′ at the driver's viewpoint 102 is set to, for example, a predetermined distance from the distance D1 ′ to the distance D3 ′. Has been. Note that the optical path length of the display light L is such that the distances D1 ′, D2 ′, and D3 ′ in FIG. 2A are the same as the distances D1, D2, and D3 in FIG. And the angle of the reflector 21 may be set. Also, the virtual plane S in FIG. 2A rises from the road surface H by a predetermined height.

  Note that the virtual image display surface is not limited to the virtual image display surface V in FIG. 1A or the virtual image display surface V ′ in FIG. 2A, and for example, Japanese Patent Application Laid-Open Nos. 2014-181025 and 2006-068576. As disclosed in JP 2016-212338 A, a combination of a plurality of virtual image display surfaces may be used. In other words, the virtual image display surface may be set to an arbitrary plane in real space, or may be a virtual image display surface of a so-called 3D head-up display device.

  FIG. 3A shows an example of an object OB (actual scenery) recognized through the windshield 101. As shown in FIG. 3 (A), for example, a driver who is a user can drive a vehicle on one side of three lanes with his own vehicle, for example, and can recognize a forward vehicle which runs on the left adjacent lane. At this time, for example, the object information acquisition unit 310 in FIG. 1A recognizes the vehicle (front vehicle) as the object OB on the road ahead of the host vehicle, and the position of the recognized object OB (front vehicle). Information can be acquired.

  FIG. 3B shows an example of a virtual image V1 (display element) superimposed on the object OB in FIG. For example, the generation unit 230 in FIG. 1A generates a virtual image V1 at the driver's viewpoint 102 based on the position information from the object information acquisition unit 310 and the viewpoint height of the driver from the viewpoint information acquisition unit 320. The position information of the display element corresponding to the virtual image V1 can be determined so as to overlap the object OB (front vehicle). For example, the virtual image V1 is formed on the virtual image display surface V that coincides with the virtual plane S in FIG.

  FIG. 3C is an explanatory diagram (plan view) of the planar shape of the virtual image V1 in the real space or the planar shape on the virtual plane S. For example, as shown in FIG. 1B, when the virtual image display surface V coincides with the virtual plane S, the virtual image V1 in the real space is formed on the virtual plane S. Therefore, the virtual image V1 in the real space The planar shape matches the planar shape on the virtual plane S in real space. As shown in FIG. 3C, the planar shape on the virtual plane S in the real space is a rectangle defined by, for example, the first to fourth sides S1, S2, S3, and S4. In the example of FIG. 3C, the center C of the planar shape (rectangle) on the virtual plane S coincides with, for example, the center of the object OB (front vehicle), and the generation unit 230 determines the center of the object OB (front vehicle). Can be acquired or calculated based on information from the object information acquisition unit 310, for example.

  In FIG. 3C, the planar shape on the virtual plane S in the real space is a rectangular parallelepiped virtual plane S (virtual plane S) surrounding the object OB (front vehicle) in the real space acquired by the object information acquisition unit 310. May be a shape that defines a cut surface by a predetermined height from the xz plane when the road surface H is the xz plane. In other words, when the object information acquisition unit 310 surrounds the object OB (front vehicle) with a rectangular parallelepiped (virtual rectangular parallelepiped), the length of the three sides (the width of the object (x-axis direction) and the depth) that define the rectangular parallelepiped (virtual rectangular parallelepiped). Of the (z-axis direction) and height (y-axis direction), a predetermined value (= of2 ++ 3) and a predetermined value (= of1 + of4) can be added to the width and depth of the object OB (front vehicle), respectively.

  Note that the size of the object OB (the vehicle in front) may not be strictly acquired by the object information acquisition unit 310, and the object information acquisition unit 310 may determine the category of the obstacle or approaching object (for example, “vehicle” or “walking”). May be assigned a fixed size for each category, and for each type within the category (eg, “normal car”, “heavy truck”, or “motorcycle” within the vehicle category) May be assigned a fixed size.

  For example, when the virtual image V1 is formed on the virtual image display surface V ′ that does not coincide with the virtual plane S in FIG. 2B, the virtual plane S on which the planar shape of the virtual image V1 in the real space visually recognized by the user is mapped. The upper planar shape is a rectangle defined by, for example, the first to fourth sides S1, S2, S3, and S4, as shown in FIG.

  4A shows an example of the virtual image V1 recognized by the user, and FIG. 4B shows an example of a change in the size of the virtual image V1 recognized by the user when the viewpoint height is lowered. 4C shows an example of the virtual image V1 moved on the virtual plane S, and FIG. 4D shows the plane shape of the moved virtual image V1 in the real space or the plane on the virtual plane S. An explanatory view (plan view) of the shape is shown.

  4A to 4C, the object OB (front vehicle) in FIG. 3B is simplified and represented as a rectangular parallelepiped (real rectangular parallelepiped) represented by a two-dot chain line. The planar shape or virtual image of the virtual image V1 in the real space visually recognized by the user when the user's viewpoint height decreases from the height assumed in FIG. 4 (A) to the height assumed in FIG. 4 (B). The generation unit 230 determines the planar shape of the display element corresponding to the virtual image V1 so that the planar shape on the virtual plane S to which the planar shape of V1 is mapped is the same. Therefore, in a situation where the planar shape on the virtual plane S in the real space is fixed or maintained by a rectangle (virtual rectangular parallelepiped) defined by the first to fourth sides S1, S2, S3, S4, for example, the viewpoint height The present inventors have recognized that it is difficult for the user to visually recognize the virtual image V1, that is, that the visibility is lowered (see FIG. 4B).

  In the example of FIG. 4B, the outer edge (boundary between the object OB and the road surface H) of the rectangular parallelepiped (real rectangular parallelepiped) in which the object OB (front vehicle) is represented by a two-dot chain line, and the virtual image V1 on the virtual plane S There is no gap between the outer edge (side S1) of the planar shape. In other words, in the example of FIG. 4B, a part (side S1) of the virtual image V1 is located at the boundary between the object OB and the road surface H. As described above, in the example of FIG. 4B, the visibility of the side S1 and / or the boundary is lowered.

  Therefore, in the generation unit 230 according to the present invention, as shown in FIGS. 4A and 4B, the planar shape (first to fourth) on the virtual plane S surrounding the object OB (front vehicle) is shown. The change in the size of the display element (virtual image V1) superimposed on the rectangle defined by the sides S1, S2, S3, S4 is the change in the size of the object OB (front vehicle) (a rectangular parallelepiped represented by a two-dot chain line). So that the planar shape of the display element is temporarily determined according to the viewpoint height of the user. That is, the generation unit 230 does not depend on the viewpoint height of the user, and the relative positional relationship in the real space between the object OB (front vehicle) visually recognized by the user and the display element (virtual image V1) and the real space. The position of the display element in the display image and the appearance shape are temporarily determined according to the viewpoint height of the user so that the size relationship does not change. Thereafter, the generation unit 230 determines the planar shape of the display element that is temporarily determined so that the planar shape (rectangular shape) of the virtual image V1 on the virtual plane S moves (FIG. 4C and FIG. 4). D)).

  As shown in FIGS. 4C and 4D, the planar shape on the virtual plane S in the real space is defined by the first to fourth sides S1, S2, S3, S4 as an example. Is a rectangle. That is, the sides S1, S2, S3, and S4 in FIG. 4B are maintained at S1, S2, S3, and S4 in FIG. 4C, and the size (area) of the planar shape on the virtual plane S is Fixed. Next, when the viewpoint height is lowered, the planar shape of the virtual image V1 in the real space visually recognized by the user or the planar shape on the virtual plane S on which the planar shape of the virtual image V1 is mapped is the front side (of4 ′ + of4). And by the amount of (of1′−of1 = of4 ′ + of4) in the same direction on the rear side (see FIG. 4D). As illustrated in FIG. 4C, the generation unit 230 determines the planar shape of the display element that is temporarily determined so that the planar shape (rectangle) of the virtual image V1 on the virtual plane S moves. The visibility can be improved by the amount of movement.

  The generation unit 230 can dynamically determine the amount of movement of the planar shape of the virtual image V1 on the virtual plane S in response to a change in the viewpoint height of the user. When the viewpoint height of the user changes, the visibility can be improved in response to the change. However, when the change in viewpoint height is large, the improvement in visibility may be stopped. Specifically, the determination unit 220 determines whether or not the viewpoint height changes within a predetermined range within a predetermined period, and when the amount of change in the viewpoint height per unit time is equal to or greater than a reference value, The generation unit 230 may hold the planar shape of the display element that is temporarily determined as it is. Alternatively, when the amount of change in the viewpoint height per unit time is equal to or greater than the reference value, the generation unit 230 displays temporarily determined so that the planar shape of the virtual image V1 on the virtual plane S moves. Even if the planar shape of the element is maintained, the display luminance of the display element may be lowered, or the display of the display element (virtual image V1) may be turned off. Thereafter, when the change in the viewpoint height becomes small, the generation unit 230 can resume improving the visibility. When the change in the height of the viewpoint is large, stopping the improvement of the visibility may cause an erroneous recognition that the distance between the object OB (front vehicle) and the vehicle (own vehicle) has changed, or an error to the driver Information presentation can be avoided.

  On the other hand, the generation unit 230 may not respond to a change in the viewpoint height of the user. Specifically, the generation unit 230 can improve the visibility during driving of the vehicle with the viewpoint height (fixed value) acquired when the vehicle is started. In other words, for example, when a woman with a relatively low sitting height drives the vehicle, the visibility during driving of the vehicle can be improved at the height of the viewpoint of the woman.

  By the way, the generation unit 230 holds the temporarily determined planar shape of the display element so that the planar shape of the virtual image V1 on the virtual plane S recognized by the user moves when the viewpoint height decreases. Therefore, the visibility can be improved by the amount of the movement. However, the determination unit 220 can determine whether or not the driver's line-of-sight direction acquired by the viewpoint information acquisition unit 320 faces the display element (virtual image V1). That is, only when the driver's line-of-sight direction faces the display element (virtual image V1), the generation unit 230 improves the visibility so that the movement amount of the planar shape of the virtual image V1 on the virtual plane S is zero. May be canceled. By stopping the improvement in visibility, erroneous information presentation to the driver can be avoided.

  When the viewpoint height is low and the driver's line-of-sight direction does not face the display element (virtual image V1), the generation unit 230 may not move the planar shape of the virtual image V1 on the virtual plane S.

  In addition, the generation unit 230 moves the planar shape of the virtual image V1 on the virtual plane S in order to improve the visibility, that is, when the movement amount of the planar shape of the virtual image V1 on the virtual plane S is not zero. It is possible to notify the user that the movement has occurred as the viewpoint height of the user decreases. As an example, the generation unit 230 can display a dedicated display element (not shown) indicating that the display element (virtual image V1) has moved according to the viewpoint height of the user in the display range AR. In addition to this, or instead of this, the processing unit 210 controls a vehicle speaker (not shown) to indicate that the display element (virtual image V1) has moved according to the viewpoint height of the user. Audio can be output. Thereby, the user can distinguish the movement of the virtual image V1 according to the user's viewpoint height and the movement of the virtual image V1 according to the approach of the object OB (front vehicle).

  FIG. 5A shows a plurality of setting examples of the movement amount of the planar shape of the virtual image V1 in the real space or the planar shape on the virtual plane S based on the viewpoint height of the user, and FIG. 3 shows an example of setting the movement amount of the planar shape of the virtual image V1 in the real space or the planar shape on the virtual plane S based on the distance up to.

  As shown in FIG. 5A, the determination unit 220 preferably determines whether or not the user's viewpoint height is lower than the reference position THα, and the user's viewpoint height is equal to or higher than the reference position THα. Sometimes, the movement amount of the planar shape of the virtual image V1 on the virtual plane S is zero. In other words, when the user's viewpoint height is equal to or higher than the reference position THα, the visibility does not decrease even if the planar shape of the virtual image V1 on the virtual plane S does not move, for example, as in FIG. .

  On the other hand, when the viewpoint height of the user is lower than the reference position THα, the movement amount (α) of the planar shape of the virtual image V1 on the virtual plane S is increased as shown by the solid line in FIG. . Note that Tall and Short in FIG. 5A indicate the maximum value and the minimum value of the viewpoint height of the driver who is a user assumed in the vehicle interior, respectively. “Normal” in FIG. 5A indicates an ideal viewpoint height of the driver. The movement amount (α) in FIG. 5A may be determined by a one-dot chain line other than the solid line in FIG. The alternate long and short dash line can represent the movement amount (α) in FIG. 5A by a convex curve, which means that the visibility can be further improved when the viewpoint height decreases. To do.

  Moreover, in order to improve visibility, the production | generation part 230 can add the expansion rate ((beta)) shown, for example by FIG. 5 (B). That is, the generation unit 230 determines the plane of the display element so that the plane shape on the virtual plane S is moved by the movement amount (α) in FIG. 5A + the movement amount (β) in FIG. The shape can be determined (confirmed). In FIG. 5B, when the distance between the object OB (the preceding vehicle) and the vehicle (the host vehicle) is larger than the threshold value THβ, a movement amount (β) larger than 0 can be set.

  5A and 5B, the minimum values of the movement amount (α) and the movement amount (β) are 0 or more.

  FIG. 6A shows an example of the virtual image V1 recognized by the user when the viewpoint height decreases, and FIG. 6B shows the planar shape of the virtual image V1 in FIG. An explanatory view of a planar shape on the virtual plane S is shown. 6A and 6B correspond to FIG. 4B, and the planar shape on the virtual plane S in the real space is, for example, the first to fourth sides S1, S2, S3, S4. In a situation where the rectangle is fixed or maintained by the prescribed rectangle, when the viewpoint height decreases, the present inventors cannot easily view the virtual image V1 and / or the boundary (the boundary between the object OB and the road surface H). That is, a reduction in visibility was recognized (see FIG. 6A).

  FIG. 7A shows an example of the moving virtual image V1 on the virtual plane S that is recognized by the user when the viewpoint height is reduced, and FIG. 7B is a side view of FIG. 7A. An explanatory view of the planar shape of the virtual image V1 or the planar shape on the virtual plane S is shown. 7 (A) and 7 (B) correspond to FIG. 4 (C), and in the situation where the planar shape on the virtual plane S in the real space is moved, when the viewpoint height decreases, the present invention They recognized the improvement in visibility (see FIG. 7A).

  FIG. 7C shows another modification of the virtual image V1 of FIG. In the example of FIG. 7C, in order to maintain the visibility of the object OB (front vehicle) itself, the object OB (front vehicle) is prioritized and a part of the planar shape on the virtual plane S, that is, The generation unit 230 can determine the shape of the display element corresponding to the virtual image V1 so as to omit the portion of the virtual image V1 behind the object OB (front vehicle).

FIG. 8A shows an explanatory diagram of the positional relationship between the object OB (front vehicle) and the host vehicle, and FIG. 8B shows an explanatory diagram of a setting example of a predetermined direction. In the example of FIG. 8 (A), the coordinates in the real space, x _w-axis, can be represented by y _w-axis and z _w-axis may also be referred to as a world coordinate. In the example of FIG. 8B, the predetermined direction DR1 _w is parallel to the _zw axis, and the first side when the first side S1 of the planar shape on the virtual plane S moves in the predetermined direction DR1 _w S1 approaches the own vehicle. In other words, when the forward direction of the host vehicle is the forward direction and the backward direction of the host vehicle is the rearward direction, the predetermined direction DR1 _w matches the backward direction of the host vehicle.

Predetermined direction in the real space, x _l-axis may be defined in the local coordinates of the object OB (the vehicle ahead) which can be represented by y _l-axis and z _l axis. In FIG. 8B, the boundary L1 between the object OB (the vehicle ahead) and the road surface H is perpendicular to the _zl axis, and the predetermined direction DR1 _l coincides with the predetermined direction DR1 _w .

The predetermined direction DR1 _w or DR1 _l is not limited to the backward direction of the host vehicle in FIG. 8A or the direction in which the first side S1 is separated from the boundary L1, and the planar shape on the virtual plane S approaches the host vehicle. If it is. In FIG. 8B, the predetermined direction DR _m is represented by a dotted line, and the viewpoint of the driver of the host vehicle from a reference point R (eg, center, center of gravity, etc.) on the road surface H of the object OB (front vehicle). The predetermined direction DR _m can be set on a straight line (virtual straight line) connecting, for example, the center of the object OB (front vehicle) and the viewpoint 102 of the driver of the host vehicle so as to coincide with the direction M toward 102. When the first side S1 approaches the user (specifically, the viewpoint 102) in the host vehicle, the visibility can be effectively improved.

Alternatively, in the predetermined direction DR1 _w or DR1 _l , or in the predetermined direction DR _m , the planar shape of the virtual image V1 on the virtual plane S is the boundary L1 between the object OB (front vehicle) and the road surface H, and the virtual It may be a direction in which a gap or an area on the virtual plane S defined between the outer edge S1 of the virtual shape V1 on the plane S and the planar shape increases. When the viewpoint height decreases, the gap on the virtual plane S (area on the virtual plane) can be increased, and the visibility can be further improved.

Further, when the object OB is an obstacle traveling toward the host vehicle, the predetermined direction DR1 _w or DR1 _l or the predetermined direction DR _m is set to the traveling direction of the object OB (obstacle), The visibility of the side S1 and / or the boundary may be improved.

  The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art will be able to easily modify the above-described exemplary embodiments to the extent included in the claims. .

DESCRIPTION OF SYMBOLS 20 ... Display device, 21 ... Reflector, 100 ... Display mechanism, 101 ... Windshield, 102 ... Viewpoint of user (driver), 103 ... Imaging device, 200 ... Display controller (image processing unit in a broad sense), 210 ... processing unit, 220 ... determination unit, 230 ... generation unit, 240 ... storage unit, 300 ... image processing unit, 310 ... object information acquiring unit, 320 ... viewpoint information acquisition unit, 400 ... information acquisition unit, AR ... display _{range, DR1 w, DR1 l, DR} m ··· predetermined direction, L ... Display light, L1 ... boundary, H ... road surface, OB ... object, R ... reference point, V ... virtual image display surface, V1 ... virtual image.

Claims (11)

A display image including a display element corresponding to a virtual image on the virtual plane that is visually recognized by a user such that the virtual plane is substantially parallel to the road surface and the virtual plane is raised by a predetermined height from the road surface; It has a generation part to generate,
The generation unit determines a planar shape of the display element so that a size of the virtual image visually recognized by the user changes based on a viewpoint height of the user,
When the viewpoint height changes from a reference position to a position lower than the reference position, the generation unit is configured to cause the planar shape of the virtual image on the virtual plane to move in a predetermined direction. An image processing unit for determining a planar shape.   The image processing unit according to claim 1, wherein the predetermined direction is a direction in which the planar shape of the virtual image on the virtual plane approaches a vehicle having a projection member onto which the virtual image is projected.   The image processing unit according to claim 2, wherein the predetermined direction is a direction from the reference point of the planar shape of the virtual image on the virtual plane toward the user who drives the vehicle. The planar shape of the virtual image on the virtual plane is a shape surrounding an object that can be recognized by the user;
The predetermined direction is a direction in which an area on the virtual plane is defined between a boundary between the object and the road surface and an outer edge of the planar shape of the virtual image on the virtual plane. The image processing unit according to claim 1, wherein the image processing unit is an image processing unit.   The said generation part determines the movement amount of the said planar shape of the said virtual image on the said virtual plane dynamically in response to the change of the said viewpoint height, The any one of Claim 1 thru | or 4 characterized by the above-mentioned. The image processing unit according to Item.   When the amount of change in the viewpoint height per unit time is equal to or greater than a reference value, the generation unit weakens or invalidates display of the display element corresponding to the virtual image moving in the predetermined direction on the virtual plane. The image processing unit according to claim 5.   The generation unit changes the planar shape of the display element so that the movement amount of the planar shape of the virtual image on the virtual plane is zero only while the user's line-of-sight direction is facing the virtual image. The image processing unit according to claim 5, wherein the image processing unit is determined.   6. The user is notified that the amount of movement has occurred as the viewpoint height decreases when the amount of movement of the planar shape of the virtual image on the virtual plane is not zero. 8. The image processing unit according to any one of items 1 to 7. The viewpoint height is a fixed value acquired when the vehicle is started,
The image processing unit according to claim 1, wherein the generation unit determines a movement amount of the planar shape of the virtual image on the virtual plane based on the fixed value.   When the viewpoint height changes from the reference position to a position higher than the reference position, the generation unit displays the display so that the movement amount of the planar shape of the virtual image on the virtual plane is zero. The image processing unit according to claim 1, wherein the planar shape of an element is determined. The image processing unit according to any one of claims 1 to 10,
A display for displaying the display image generated by the image processing unit;
A head-up display device comprising:

Images