JP2018151903A - Virtual image display device and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a virtual image display device and a computer program in which a virtual image is appropriately superimposed on an object so as to be visually recognized by a vehicle occupant and chattering of a virtual image display position is suppressed. SOLUTION: A virtual image such as an icon for giving a warning is superimposed on a warning object in front of the line of sight of the occupant to be visually recognized, and is visually recognized by the occupant based on the behavior of the vehicle on which the occupant is riding and the behavior of the warning object. Warning The displacement of the future position of the object is predicted, and the display position of the virtual image is corrected based on the predicted displacement of the position. In addition, the superimposed state of the warning object and the virtual image is acquired, and the presence / absence of correction is switched based on the superimposed state of the warning object and the virtual image. [Selection diagram] FIG. 8

Description

  The present invention relates to a virtual image display device and a computer program that display a virtual image visually recognized by a vehicle occupant.

  Conventionally, various means have been used as information providing means for providing driving information such as route guidance and obstacle warnings to passengers of moving bodies such as vehicles. For example, display on a liquid crystal display installed on a moving body, sound output from a speaker, and the like. In recent years, as one of such information providing means, a head-up display device (hereinafter referred to as HUD) is used in a space different from a position where an image is actually displayed using an illusion of human eyes. There is a virtual image display device for visually recognizing an image.

  Here, in the case where guidance is performed by superimposing a virtual image displayed by the HUD on the surrounding environment, it is important that the virtual image is superimposed on an appropriate position with respect to the surrounding environment. For example, in the case of displaying a mark that warns the obstacle ahead as a virtual image, by displaying the virtual image at a position superimposed on the obstacle to be warned, the occupant can be surely grasped the obstacle, Effective guidance can be performed. Therefore, the position of the target object (obstacle, etc.) on which the virtual image is to be superimposed is detected using a sensor or camera, and the display position of the virtual image is controlled so as to be superimposed on the detected position of the target object. . However, when the vehicle on which the occupant rides moves, the position of the object visually recognized by the occupant (that is, the relative position of the object with respect to the occupant) changes, and the position of the virtual image to be displayed deviates from the position of the object. It was. In particular, when a moving body such as another vehicle or a pedestrian is the target object, there is a possibility that a larger displacement occurs due to the movement of the target object.

  Therefore, a technique for preventing the displacement of the position by predicting the displacement of the future position of the object visually recognized by the occupant can be considered. For example, Japanese Patent Application Laid-Open No. 2011-223242 discloses a technique for predicting and tracking a future movement of a subject in order to perform focus adjustment, exposure control, and the like in an electronic camera. It is also possible to use it.

JP2011-223242A (page 8-9)

  Here, when the tracking technique described in Patent Document 1 is used for HUD, an appropriate virtual image can be superimposed and displayed when the moving speed and moving direction of the object viewed from the occupant are constant. It is believed that there is. However, compared to the subject of the electronic camera, the object moving on the road as seen from the occupant is larger in frequency and amount of change in the movement speed and direction of movement, so the virtual image is displayed by performing the tracking correction described above. There was a problem that position chattering (blurring, rattling, shaking, etc.) occurred, giving the passenger a sense of incongruity.

  The present invention has been made in order to solve the above-described problems in the related art. The virtual image is appropriately superimposed on the object so that the vehicle occupant can visually recognize the virtual image, and the occurrence of chattering at the display position of the virtual image is suppressed. An object is to provide a display device and a computer program.

In order to achieve the above object, a virtual image display device according to the present invention is mounted on a vehicle and causes a vehicle occupant to visually recognize an image displayed on a video display surface, thereby moving the virtual image of the image forward of the occupant's line of sight. A virtual image display device that is superimposed on a certain object and visually recognized, vehicle information acquisition means for acquiring the behavior of the vehicle, object information acquisition means for acquiring the behavior of the object, behavior of the vehicle, and Based on the behavior of the object, the correction means for predicting the displacement of the future position of the object visually recognized by the occupant, and correcting the display position of the virtual image based on the displacement of the predicted position, A superimposition state acquisition unit that acquires a superimposition state of the object and the virtual image; and a correction switching unit that switches whether the correction unit performs correction based on the superimposition state of the object and the virtual image. .
The “virtual image display position” is defined not by an absolute position but by a relative position with respect to the line of sight of an occupant who visually recognizes the virtual image.

  In addition, the computer program according to the present invention is mounted on a vehicle and causes the occupant of the vehicle to visually recognize the image displayed on the image display surface, thereby superimposing a virtual image of the image on an object in front of the sight line of the occupant. It is a computer program to be visually recognized. Specifically, the computer is based on vehicle information acquisition means for acquiring the behavior of the vehicle, object information acquisition means for acquiring the behavior of the object, behavior of the vehicle, and behavior of the object. A correction means for predicting a future displacement of the object visually recognized by the occupant and correcting a display position of the virtual image based on the predicted displacement of the object; and a superposition state of the object and the virtual image And a correction switching means for switching whether or not correction is performed by the correction means based on a superposition state of the object and the virtual image.

  According to the virtual image display device and the computer program according to the present invention having the above-described configuration, the virtual image can be appropriately superimposed on the object and visually recognized by the vehicle occupant by correcting the display position of the virtual image. Therefore, by switching whether to perform correction based on the superimposed state of the object and the virtual image, it is possible to suppress the correction to the minimum necessary and to reduce the occurrence of chattering of the display position of the virtual image as much as possible. .

It is a schematic block diagram of the virtual image display apparatus which concerns on this embodiment. It is the block diagram which showed the navigation apparatus which concerns on this embodiment. It is the figure which showed the internal structure of HUD which concerns on this embodiment. It is a flowchart of the virtual image generation processing program which concerns on this embodiment. It is the figure which showed an example of the virtual image visually recognizable from the passenger | crew of a vehicle. It is the figure which showed the example of prediction of the future position of the warning target object visually recognized from a passenger | crew. It is the figure which showed an example of the virtual image which can be visually recognized from a passenger | crew when the position of the warning target object visually recognized from a passenger | crew is displaced. It is a flowchart of the sub process program of a correction implementation determination process. It is the figure which showed the case where the correction process of the display position of a virtual image is not performed when there exists another warning target object used as the superimposition target of a virtual image around the warning target object. It is the figure which showed the case where the correction process of the display position of a virtual image is performed when there exists another warning target object used as the superimposition target of a virtual image around the warning target object. It is the figure which showed the case where the correction process of the display position of a virtual image is not performed when there exists no other warning target object used as the superimposition target of a virtual image around a warning target object. It is the figure which showed the case where the correction process of the display position of a virtual image is performed when there is no other warning target object used as the superimposition target of a virtual image around the warning target object.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, an embodiment embodying a virtual image display device according to the invention will be described in detail with reference to the drawings. First, a schematic configuration of the virtual image display device 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a schematic configuration diagram of a virtual image display device 1 according to the present embodiment.

  As shown in FIG. 1, a virtual image display device 1 includes a navigation device 3 mounted on a vehicle 2, a head-up display device (hereinafter referred to as HUD) 4 mounted on the vehicle 2 and connected to the navigation device 3. Basically has.

  Here, the navigation device 3 searches for a recommended route to the destination, detects the current position of the vehicle 2, obtains it from the server, or based on map data stored in the memory, around the current position of the vehicle 2 The map image is displayed, and the movement guidance along the set guidance route and the warning guidance for obstacles are performed together with the HUD 4. Note that the navigation device 3 does not have to have all of the above functions, and the present invention is configured if it has at least a function of performing movement guidance along a set guidance route and warning guidance for obstacles. Is possible. Details of the structure of the navigation device 3 will be described later.

  On the other hand, the HUD 4 is installed inside the dashboard 5 of the vehicle 2 and has a liquid crystal display 6 as a video display surface on which video is displayed. Then, the image projected on the liquid crystal display 6 is reflected on the front window 8 in front of the driver's seat through the concave mirror 7 provided in the HUD 4 as will be described later, so that the passenger 9 of the vehicle 2 can visually recognize the image. ing. The video displayed on the liquid crystal display 6 includes information related to the vehicle 2 and various information used for assisting the driving of the occupant 9. For example, a warning for an object (another vehicle or a pedestrian) that is a warning target for the occupant 9, guidance information set by the navigation device 3, guidance information based on the guidance route (such as an arrow indicating a right / left turn direction) There are warnings to be displayed (such as rear-end collision warnings, speed limits, etc.), current vehicle speed, information signs, map images, traffic information, news, weather forecasts, times, connected smartphone screens, TV programs, and the like.

  Further, in the HUD 4 of the present embodiment, when the occupant 9 visually reflects the image displayed on the liquid crystal display 6 by reflecting the front window 8, the occupant 9 is not at the position of the front window 8 but at the tip of the front window 8. The image displayed on the liquid crystal display 6 at a distant position is visually recognized as a virtual image 10. The virtual image 10 that can be seen by the occupant 9 is an image displayed on the liquid crystal display 6, but the vertical direction and the horizontal direction may be reversed through the concave mirror 7 and other mirrors. Therefore, it is necessary to display an image on the liquid crystal display 6. In addition, the size is changed through the concave mirror 7.

  Here, regarding the position where the virtual image 10 is generated, more specifically, the distance from the occupant 9 to the virtual image 10 (hereinafter referred to as the virtual image generation distance) L, the curvature of the concave mirror 7 included in the HUD 4, the liquid crystal display 6 and the concave mirror 7 It is possible to set as appropriate depending on the relative position of the. For example, if the curvature of the concave mirror 7 is fixed, the virtual image generation distance L is determined by the distance (optical path length) along the optical path from the position where the image is displayed on the liquid crystal display 6 to the concave mirror 7. For example, the optical path length is set so that the virtual image generation distance L is 25 m. Details of the structure of the HUD 4 will be described later.

  A front camera 11 is installed above the front bumper of the vehicle, behind the rearview mirror, and the like. The front camera 11 is an imaging device having a camera using a solid-state imaging device such as a CCD, for example, and is installed with the optical axis direction facing forward in the traveling direction of the vehicle. Then, by performing image processing on the captured image captured by the front camera 11, the situation of the front environment (that is, the environment in which the virtual image 10 is superimposed) that is visually recognized by the occupant 9 through the front window 8 is detected. Is done. A sensor such as a millimeter wave radar may be used instead of the front camera 11.

  Next, a schematic configuration of the navigation device 3 constituting the virtual image display device 1 will be described with reference to FIG. FIG. 2 is a block diagram showing the navigation device 3 according to the present embodiment.

  As shown in FIG. 2, the navigation device 3 according to the present embodiment includes a current position detection unit 12 that detects the current position of the vehicle 2 on which the navigation device 3 is mounted, and a data recording unit 13 that records various data. The navigation ECU 14 that performs various arithmetic processes based on the input information, the operation unit 15 that receives operations from the user, and the liquid crystal display 16 that displays facility information related to the map and facilities around the vehicle to the user. Communication for communicating with an information center such as a VICS (registered trademark: Vehicle Information and Communication System) center, and a speaker 17 that outputs voice guidance related to route guidance, a DVD drive 18 that reads a DVD as a storage medium, and the like. And a module 19. The navigation device 3 is connected to the HUD 4 and the front camera 11 described above via an in-vehicle network such as CAN.

Below, each component which the navigation apparatus 3 has is demonstrated in order.
The current position detection unit 12 includes a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, and the like, and can detect the current vehicle position, direction, vehicle traveling speed, current time, and the like. . Here, in particular, the vehicle speed sensor 22 is a sensor for detecting a moving distance and a vehicle speed of the vehicle, generates a pulse according to the rotation of the driving wheel of the vehicle, and outputs a pulse signal to the navigation ECU 14. And navigation ECU14 calculates the rotational speed and moving distance of a driving wheel by counting the generated pulse. In addition, it is not necessary for the navigation device 3 to include all the four types of sensors, and the navigation device 3 may include only one or a plurality of types of sensors.

  The data recording unit 13 is a driver for reading out an external storage device and a hard disk (not shown) as a recording medium, a map information DB 31 and a predetermined program recorded in the hard disk, and writing predetermined data in the hard disk And a recording head (not shown). The data recording unit 13 may include a flash memory, a memory card, an optical disk such as a CD or a DVD, instead of the hard disk. The map information DB 31 may be stored in an external server, and the navigation device 3 may be configured to acquire by communication.

  Here, the map information DB 31 is, for example, link data 32 regarding roads (links), node data 33 regarding node points, point data 34 regarding points such as facilities, intersection data regarding each intersection, and map display data for displaying a map. The storage means stores search data for searching for routes, search data for searching for points, and the like.

  Here, as the link data 32, for example, a link ID for identifying the link, end node information for specifying a node located at the end of the link, a road type of a road formed by the link, and the like are stored. . The node data 33 stores a node ID for identifying the node, position coordinates of the node, connection destination node information for specifying a connection destination node to which the node is connected through a link, and the like. Further, as the point data 34, various types of information related to the facility to be set as the destination are stored. For example, an ID for specifying a facility, a facility name, position coordinates, a genre, an address, and the like are stored.

  On the other hand, the navigation ECU (Electronic Control Unit) 14 is an electronic control unit that controls the entire navigation device 3, and includes a CPU 41 as an arithmetic device and a control device, and a working memory when the CPU 41 performs various arithmetic processes. Read out from the ROM 43 and the ROM 43 in which the RAM 42 storing the route data when the route is searched, the control program, the virtual image generation processing program (FIG. 4) described later, and the like are recorded. An internal storage device such as a flash memory 44 for storing the program is provided. The navigation ECU 14 has various means as processing algorithms. For example, the vehicle information acquisition unit acquires the behavior of the vehicle 2. The target object information acquisition unit acquires the behavior of the target object on which the virtual image is superimposed. The correction means predicts the displacement of the future position of the object visually recognized by the occupant 9 based on the behavior of the vehicle 2 and the behavior of the object, and displays the virtual image based on the predicted displacement of the position. Correct. A superimposition state acquisition means acquires the superimposition state of a target object and a virtual image. The correction switching unit switches whether or not correction is performed by the correction unit based on the superimposed state of the object and the virtual image.

  The operation unit 15 is operated when inputting a starting point as a travel start point and a destination as a travel end point, and has a plurality of operation switches (not shown) such as various keys and buttons. The navigation ECU 14 performs control to execute various corresponding operations based on switch signals output by pressing the switches. The operation unit 15 may have a touch panel provided on the front surface of the liquid crystal display 16. Moreover, you may have a microphone and a speech recognition apparatus.

  In addition, the liquid crystal display 16 includes a map image including a road, traffic information, operation guidance, operation menu, key guidance, guidance route from the departure point to the destination, guidance information along the guidance route, news, weather forecast, Time, mail, TV program, etc. are displayed. In this embodiment, since the HUD 4 is provided as information display means, the liquid crystal display 16 may be omitted if the map image or the like is displayed on the HUD 4.

  In addition, the speaker 17 outputs voice guidance for guiding traveling along the guidance route based on an instruction from the navigation ECU 14 or traffic information guidance.

  The DVD drive 18 is a drive that can read data recorded on a recording medium such as a DVD or a CD. Based on the read data, music and video are reproduced, the map information DB 31 is updated, and the like. A card slot for reading / writing a memory card may be provided instead of the DVD drive 18.

  The communication module 19 is a communication device for receiving traffic information including information such as traffic jam information, regulation information, and traffic accident information transmitted from a traffic information center, for example, a VICS center or a probe center. For example, a mobile phone or DCM is applicable.

  Next, a schematic configuration of the HUD 4 included in the virtual image display device 1 will be described with reference to FIG. FIG. 3 is a diagram showing an internal configuration of the HUD 4 according to the present embodiment.

  As shown in FIG. 3, the HUD 4 basically includes a liquid crystal display 6, a concave mirror 7, a cover glass 51, a control circuit unit 52, and a CAN interface 53.

  Here, the liquid crystal display 6 is a video display device that uses a backlight as a light source and has a function of displaying video on a video display surface provided on the front surface. As the backlight, for example, a CCFL (cold cathode tube) or a white LED is used. In addition to the liquid crystal display, an organic EL display, a combination of a projector and a screen, or the like may be used as a means for displaying an image. When a combination of a projector and a screen is used, the screen corresponds to a liquid crystal display surface on which an image is displayed.

  On the other hand, the concave mirror 7 is a projection mirror that generates a virtual image of the image in front of the occupant 9 by enlarging and reflecting the image displayed on the liquid crystal display 6 and causing the occupant 9 to visually recognize the image. As the concave mirror 7, a spherical concave mirror, an aspheric concave mirror, or a free-form curved mirror for correcting distortion of a projected image is used. Since the image projected onto the liquid crystal display 6 is reflected by the concave mirror 7, the generated virtual image 10 is an image that is vertically inverted from the image displayed on the liquid crystal display 6.

  The cover glass 51 is a transmissive plate-like member disposed on the upper surface of the HUD 4. The image displayed on the liquid crystal display 6 is reflected by the concave mirror 7 and is visually recognized by the occupant 9 through the cover glass 51. As the cover glass 51, a Fresnel lens may be used. Further, when a Fresnel lens is used as the cover glass 51, a flat mirror can be used instead of the concave mirror 7.

  The control circuit unit 52 is an electronic control unit that controls the entire HUD 4. Here, the control circuit unit 52 includes a CPU as an arithmetic device and a control device, a RAM used as a working memory when the CPU performs various arithmetic processes, a control program, and a control for generating a virtual image. An internal storage device such as a ROM in which programs are recorded is provided. The control circuit unit 52 is connected to the liquid crystal display 6 and performs drive control of the liquid crystal display 6.

  The CAN (controller area network) interface 53 is an interface for inputting / outputting data to / from CAN, which is a vehicle-mounted network standard that performs multiplex communication between various vehicle-mounted devices and control devices for vehicle equipment installed in the vehicle. It is. The HUD 4 is connected to various on-vehicle devices and vehicle equipment control devices (for example, the navigation device 3 and the AV device) via the CAN so as to communicate with each other. Accordingly, the HUD 4 is configured to be able to project information acquired from the navigation device 3 or the AV device.

  Next, a virtual image generation processing program executed in the navigation device 3 in the virtual image display device 1 having the above configuration will be described with reference to FIG. FIG. 4 is a flowchart of the virtual image generation processing program according to this embodiment. Here, the virtual image generation processing program is a program that is executed after an ACC power supply (accessory power supply) of the vehicle is turned on, and generates a virtual image 10 that is visually recognized by the passenger 9 of the vehicle 2 using the HUD 4. 4 and 8 are stored in the RAM 42 and the ROM 43 provided in the navigation device 3 and executed by the CPU 41.

  First, in step (hereinafter abbreviated as S) 1 in the virtual image generation processing program, the CPU 41 has a target object (hereinafter referred to as a warning target object) as a warning target for the vehicle occupant in front of the traveling direction of the vehicle. It is determined whether or not. Here, the warning object is an object to be watched by a passenger who rides the vehicle, for example, another vehicle, a pedestrian, or a two-wheeled vehicle. It should be noted that other objects that should be watched by the passenger, such as signs and traffic lights, may be included in the warning object.

  The presence or absence of a warning object in front of the traveling direction of the vehicle may be detected, for example, by performing image processing such as template matching on the captured image captured by the front camera 11, You may detect using a sensor. Moreover, you may acquire by communication with a vehicle-to-vehicle communication or an external server.

  And when it determines with there being a warning target object ahead of the advancing direction of a vehicle (S1: YES), it transfers to S2. On the other hand, when it is determined that there is no warning object ahead in the traveling direction of the vehicle (S1: NO), the virtual image generation processing program is terminated.

  In S <b> 2, the CPU 41 acquires a specific position of the warning object that is determined to be ahead of the traveling direction of the vehicle. Note that the position of the warning object may be acquired by performing image processing on a captured image captured by the front camera 11, for example, or may be acquired using a sensor such as a millimeter wave radar. Moreover, you may acquire by communication with a vehicle-to-vehicle communication or an external server. Further, the position of the warning object may be an absolute position, or a relative position with respect to an occupant (own vehicle). In addition, when there are a plurality of warning objects, the positions of the plurality of warning objects are acquired.

  For example, a method for specifying the position of a warning object by performing image processing on a captured image captured by the front camera 11 will be described below.

  First, the CPU 41 generates template image data of a warning object. For example, when the warning object is a pedestrian, the template image data of the pedestrian is generated, and when the warning object is another vehicle, the template image data of the vehicle is generated and stored in the flash memory 44 or the like. To do. The template image data generation method is already known (see, for example, Japanese Patent Laid-Open No. 2011-223242).

  Then, the CPU 41 uses the generated template image to continuously detect the position of the warning object ahead of the traveling direction of the vehicle by template matching processing until the virtual image generation processing program ends.

  Specifically, first, a target image is set for a captured image of the front camera 11 as a comparison target image for performing template matching processing. The area of the template image and the area of the target image are the same.

  Thereafter, the CPU 41 sets a search area for searching for a warning object in the captured image and moves the target image one pixel at a time or a predetermined pixel set in advance in the search area. The similarity with the template image is calculated. And CPU41 specifies the position of the target image which satisfy | fills conditions with the relatively largest similarity in a search area as a movement destination of a warning target object. Specifically, the sum of differences in each pixel between the template image and the target image is calculated. The smaller the difference sum, the greater the degree of similarity, and it is determined that the template image and the target image are similar.

  Then, the CPU 41 can continuously specify the current position of the warning object by specifying the movement destination of the warning object for each frame of the captured image captured by the front camera 11.

  Next, in S <b> 3, the CPU 41 transmits a control signal to the HUD 4 and outputs an image for generating the virtual image 10 to the liquid crystal display 6 of the HUD 4. For example, in this embodiment, it is set as the structure which outputs the image | video of the icon which shows that it is a warning target object. The display position of the image on the liquid crystal display 6 is a position where the virtual image 10 based on the image is visually recognized by the vehicle occupant when the vehicle occupant visually recognizes the virtual image 10, and the latest image acquired in S2. It is determined based on the position of the warning object. In addition, when a certain condition is satisfied as will be described later, a display position correction process for superimposing the virtual image 10 on the warning object is also performed (S6).

  Then, by outputting the image in S3, the virtual image 10 superimposed on the warning object 61 in the forward direction of the vehicle as shown in FIG. 6 is separated from the vehicle occupant by the virtual image generation distance L (for example, 25 m). It will be visually recognized at the position. As a result, the line of sight of the occupant can be guided to the warning object 61 by the virtual image 10, and the warning object 61 can be surely visually recognized by the user. In this embodiment, the virtual image generation distance L is a fixed value. However, the virtual image generation distance L can be arbitrarily changed by moving the liquid crystal display 6, for example. Moreover, the display of the virtual image 10 is basically performed until the warning object disappears from the front of the vehicle thereafter.

  Further, when there are a plurality of warning objects, the virtual image 10 is basically displayed only for the warning object closest to the passenger (own vehicle). However, the virtual image 10 may be displayed superimposed on each of the plurality of warning objects. Moreover, you may select the warning target object on which the virtual image 10 is superimposed based on the kind of warning target object. For example, when a pedestrian is included as a warning object, it is possible to preferentially select a pedestrian and superimpose the virtual image 10 thereon.

  Next, in S4, the CPU 41 executes a correction execution determination process (FIG. 8) described later. Note that the correction execution determination process is a process for determining whether or not to execute a correction process of the display position of the virtual image based on the future position prediction of the warning object as described later.

  Subsequently, in S5, the CPU 41 determines whether or not to execute the correction processing of the display position of the virtual image based on the processing result of S4.

  And when it determines with implementing the correction process of the display position of a virtual image (S5: YES), it transfers to S6. On the other hand, if it is determined not to perform the virtual image display position correction process (S5: NO), the process proceeds to S7.

  In S <b> 6, the CPU 41 starts a process for correcting the display position of the virtual image based on the future position prediction of the warning object. Note that the started correction process is continued until the function of the correction process is turned off in S7 described later.

Here, the virtual image display position correction processing performed in S6 will be described below.
First, the CPU 41 obtains the behavior of the host vehicle on which the occupant gets and the behavior of the warning object that is the object of superimposition of the virtual image. In particular, the movement speed and the movement direction are acquired as the behavior, and the movement speed and the movement direction of the own vehicle are acquired using various sensors such as the vehicle speed sensor 22 and the gyro sensor 24. On the other hand, the behavior of the warning object is acquired by comparing the position of the warning object specified in the current frame by the template matching process in S2 described above and the position of the warning object specified in the previous frame. In addition, you may acquire using sensors, such as a millimeter wave radar.

  Subsequently, the CPU 41 predicts a future displacement of the warning object visually recognized by the occupant based on the behavior of the host vehicle on which the occupant gets on and the behavior of the warning object on which the virtual image is superimposed. Specifically, first, the CPU 41 calculates a vector (hereinafter referred to as a movement vector) indicating the moving direction and moving speed of the warning object visually recognized by the occupant.

  Then, the CPU 41 predicts the destination of the warning object from the calculated movement vector. For example, as shown in FIG. 6, when the warning object 61 visually recognized by the occupant is located in the region G and the movement vector 62 can be specified, the warning object 61 will be moved from the region G to the region H in the future. , It can be predicted to move to region I. Therefore, the CPU 41 calculates a difference value between the AE evaluation value indicating the amount of change in brightness of the region G and the AE evaluation value of the region H.

  Thereafter, the CPU 41 corrects the brightness level of the template image based on the difference value of the AE evaluation values. Then, the CPU 41 performs template matching processing using the corrected template image in the next frame. As a result, the luminance component of the template image is corrected following the change in the luminance component of the target image in the direction predicted as the destination, so that the detection accuracy of the warning object can be increased in the template matching process.

  In S6, the search area may be narrowed based on the movement vector of the warning object. In that case, the destination of the warning object can be estimated based on the movement vector, and the search area can be narrowed.

  By performing the correction process of the display position of the virtual image in S6, even if the position of the warning object 61 visually recognized by the occupant is displaced as shown in FIG. Since the display position of the virtual image 10 based on the correction is performed, the occupant can visually recognize the virtual image 10 while being superimposed on the warning object 61 as much as possible without causing the position of the virtual image 10 to greatly deviate from the position of the warning object 61. Is possible.

  On the other hand, in S7, the CPU 41 ends the correction processing of the virtual image display position based on the future position prediction of the warning object started in S6. If the correction processing of the display position of the virtual image is not performed, there is a possibility that a deviation occurs between the position of the virtual image 10 and the position of the warning object 61 when the position of the warning object 61 viewed from the passenger is displaced. On the other hand, chattering (blurring, rattling, shaking, etc.) of the display position of the virtual image can be prevented. Thereafter, the process returns to S1.

  Next, a sub-process of the correction execution determination process executed in S4 will be described with reference to FIG. FIG. 8 is a flowchart of a sub process program of the correction execution determination process.

  First, in S11, the CPU 41 reads the position of the warning object specified in S2. If there are a plurality of warning objects, the position of each warning object is read out.

  Next, in S <b> 12, the CPU 41 determines whether there is another warning object around (for example, within 5 m) around the warning object to be superimposed on the virtual image 10. In the present embodiment, when there are a plurality of warning objects, the virtual image 10 is basically displayed only for the warning object closest to the occupant (own vehicle). Therefore, the warning object other than the warning object closest to the occupant (own vehicle) is another warning object.

  When it is determined that there is another warning object around the warning object to be superimposed with the virtual image 10 (S12: YES), the process proceeds to S13. On the other hand, if it is determined that there is no other warning object around the warning object to be superimposed with the virtual image 10 (S12: NO), the process proceeds to S16.

  In S <b> 13, the CPU 41 obtains a superimposition state of the warning object and the virtual image 10 to be superimposed on the virtual image 10, and determines whether or not a ratio of superimposing the warning object on the entire virtual image 10 is 50% or more. judge. Note that the superposition state of the warning object and the virtual image 10 (more specifically, the ratio of superposition of the entire virtual image 10 with respect to the warning object) is the current position of the warning object specified in S2. It is calculated based on the shape of the warning object and the display position of the image on the liquid crystal display 6. Further, the threshold value serving as the determination criterion in S13 can be changed as appropriate, and may be 40% or 60%, for example. However, the threshold value of the criterion is set higher than S16 described later.

  And when it determines with the ratio which overlaps with respect to a warning target object among the whole virtual images 10 being 50% or more (S13: YES), it determines with not correcting the display position of the virtual image 10 (S14). On the other hand, when it is determined that the ratio of superimposing the warning object on the entire virtual image 10 is less than 50% (including the case where it is not superimposed) (S13: NO), the virtual image 10 is displayed. It is determined that position correction processing is to be performed (S15). Thereafter, the process proceeds to S5, and whether or not the display position correction process of the virtual image 10 is performed is switched based on the determination result of S14 or S15.

  Here, as shown in FIG. 9, even when another warning target object 65 exists around the warning target object 61 to be superimposed with the virtual image 10, the degree of superimposition of the warning target object 61 and the virtual image 10 is high. In the case (specifically, the ratio of overlapping with the warning object in the entire virtual image 10 is 50% or more), the warning object is not the warning object 65 but the warning object 65 by the virtual image 10 at the present time. The occupant 9 can clearly grasp that it is 61. Therefore, in such a case, it is possible to prevent the occurrence of chattering of the display position of the virtual image by not performing the correction process of the display position of the virtual image.

  On the other hand, as shown in FIG. 10, when another warning object 65 exists around the warning object 61 to be superimposed with the virtual image 10 and the degree of superimposition of the warning object 61 and the virtual image 10 is low. (Specifically, in the entire virtual image 10, the ratio of overlapping with the warning object is less than 50%), the warning object 61 or the warning object 65 is the warning object by the virtual image 10. There is a possibility that the occupant 9 cannot clearly grasp whether there is any. Therefore, in such a case, priority is given to superimposing the display position of the virtual image 10 on the warning object 61 by performing the correction process of the display position of the virtual image.

  In S <b> 16, the CPU 41 acquires the superimposition state of the warning object to be superimposed on the virtual image 10 and the virtual image 10, and determines whether at least a part of the virtual image 10 is superimposed on the warning object (that is, the virtual image). 10), it is determined whether or not the ratio of overlapping with the warning object is larger than 0%. The superimposed state of the warning object and the virtual image 10 is calculated based on the current position of the warning object specified in S2, the shape of the warning object, and the display position of the image on the liquid crystal display 6 as in S13. Is done. In S16, it may be determined whether or not the ratio of superimposing on the warning object in the entire virtual image 10 is a predetermined ratio or more. However, the predetermined ratio is lower than S13.

  And when it determines with at least one part of the virtual image 10 superimposing with respect to a warning target object (S16: YES), it determines with not correcting the display position of the virtual image 10 (S17). On the other hand, when it is determined that the virtual image 10 is not superimposed on the warning object (S16: NO), it is determined that the display position correction process of the virtual image 10 is performed (S18). Thereafter, the process proceeds to S5, and whether or not to perform the correction process of the display position of the virtual image 10 is switched based on the determination result of S17 or S18.

  Here, as shown in FIG. 11, when there is no other warning object around the warning object 61 to be superimposed with the virtual image 10, even if the degree of superimposition of the warning object 61 and the virtual image 10 is low, However, if they overlap, the occupant 9 can grasp that the warning object 61 is the warning object by the virtual image 10. Therefore, in such a case, it is possible to prevent the occurrence of chattering of the display position of the virtual image by not performing the correction process of the display position of the virtual image.

  On the other hand, as shown in FIG. 12, even when there is no other warning object around the warning object 61 to be superimposed with the virtual image 10, the warning object 61 and the virtual image 10 do not overlap. , There is a possibility that the occupant 9 cannot clearly grasp whether the warning object 61 is the warning target by the virtual image 10. Therefore, in such a case, priority is given to superimposing the display position of the virtual image 10 on the warning object 61 by performing the correction process of the display position of the virtual image.

  As described above in detail, in the virtual image display device 1 and the computer program executed by the virtual image display device 1 according to the present embodiment, the virtual image 10 is superimposed on the warning object 61 in front of the line of sight of the occupant 9 and visually recognized. Based on the behavior of the vehicle on which the occupant 9 rides and the behavior of the warning object, the future position displacement of the warning object 61 visually recognized by the occupant 9 is predicted, and the virtual image is based on the predicted position displacement. 10 display position is corrected (S6). Further, the superimposition state of the warning object 61 and the virtual image 10 is acquired (S13, S16), and whether or not correction is performed is switched based on the superposition state of the warning object 61 and the virtual image 10 (S14, S15, S17, S18). Therefore, by correcting the display position of the virtual image 10, the virtual image 10 can be appropriately superimposed on the warning target object and can be visually recognized by the vehicle occupant, while being based on the superimposed state of the warning target object and the virtual image 10. By switching the presence / absence of the correction, the correction can be suppressed to the minimum necessary and the occurrence of chattering of the display position of the virtual image 10 can be reduced as much as possible.

Note that the present invention is not limited to the above-described embodiment, and various improvements and modifications can be made without departing from the scope of the present invention.
For example, in the present embodiment, the virtual image is generated in front of the front window 8 of the vehicle 2 by the HUD 4, but the virtual image may be generated in front of a window other than the front window 8. In addition, the object to be reflected by the HUD 4 may be a visor (combiner) installed around the front window 8 instead of the front window 8 itself.

  In the present embodiment, the virtual image display device 1 is installed on the vehicle 2. However, the virtual image display device 1 may be installed on a moving body other than the vehicle 2. For example, it can be installed on a ship or an aircraft. Moreover, you may install in the ride type attraction installed in an amusement facility. In that case, a virtual image can be generated around the ride so that the rider can visually recognize the virtual image.

  Moreover, in this embodiment, the virtual image of the icon that warns the warning object is displayed as the virtual image 10 that is guided by being superposed on the surrounding environment and visually recognized by the occupant 9. It may be a virtual image. For example, a virtual image of an arrow, a guide sign, a map image, traffic information, news, weather forecast, time, a connected smartphone screen, a television screen, and the like may be used.

  In the present embodiment, the virtual image is superimposed on the warning object that is the warning target for the occupant, but the target object on which the virtual image is superimposed may be other than the warning target object. For example, a road surface or a facility may be used.

  In the present embodiment, when there are other warning objects around the warning object, the threshold value for determining the degree of superimposition is set higher than when there is no other warning object. Even when an object other than the warning object (for example, a road surface or a facility) is around the warning object, the threshold value for determining the degree of superimposition may be set high.

  In the present embodiment, as a parameter indicating the superimposition state (superimposition degree) of the warning object and the virtual image in the correction execution determination process in S4, a ratio of superimposing the warning object on the entire virtual image is calculated. However, other parameters may be calculated as long as the parameters indicate the superimposed state of the warning object and the virtual image. For example, you may calculate the ratio which overlaps with respect to a virtual image among the warning object whole. Further, the time during which the warning object and the virtual image are continuously superimposed may be calculated.

  Further, in the present embodiment, the processing of the virtual image generation processing program (FIG. 4) is configured to be executed by the navigation ECU 14 of the navigation device 3, but the execution subject can be changed as appropriate. For example, it is good also as a structure which the control part of HUD4, vehicle control ECU20, and another vehicle equipment perform. In addition, when the control part of HUD4 performs, the virtual image display apparatus which concerns on this invention can also be comprised only with HUD4.

  Moreover, although the embodiment which actualized the virtual image display device according to the present invention has been described above, the virtual image display device can also have the following configuration, and in that case, the following effects can be obtained.

For example, the first configuration is as follows.
By making the vehicle occupant (9) visually recognize the image mounted on the vehicle (2) and displayed on the image display surface (5), the virtual image (10) of the image is located in front of the occupant's line of sight ( 61) a virtual image display device (1) to be superimposed and visually recognized, vehicle information acquisition means (41) for acquiring the behavior of the vehicle, and object information acquisition means (41) for acquiring the behavior of the object. And, based on the behavior of the vehicle and the behavior of the object, predict the displacement of the future position of the object visually recognized by the occupant, and display the virtual image based on the predicted displacement of the position Correction means (41) for correcting the image, superimposition state acquisition means (41) for acquiring a superposition state of the object and the virtual image, and correction by the correction means based on the superposition state of the target object and the virtual image Correction to switch on / off Having a switch means (41), the.
According to the virtual image display device having the above configuration, by correcting the display position of the virtual image, it is possible to appropriately superimpose the virtual image on the object and allow the vehicle occupant to visually recognize the object. By switching whether or not the correction is performed based on the superimposition state, it is possible to suppress the correction to the minimum necessary and to reduce the occurrence of chattering of the display position of the virtual image as much as possible.

The second configuration is as follows.
When the degree of superimposition of the object (61) and the virtual image (10) is less than a threshold value, the correction unit (41) performs correction, while the degree of superimposition of the object and the virtual image is equal to or greater than the threshold value. In such a case, the correction by the correction means is not performed.
According to the virtual image display device having the above-described configuration, the correspondence between the virtual image and the object is clear, and chattering of the display position of the virtual image occurs by not performing the correction particularly in a situation where the correction is not essential. Can be reduced as much as possible.

The third configuration is as follows.
When there is another object (65) to be superimposed on the virtual image (10) around the object (61), the threshold value is greater than when there is no other object around the object. Set high.
According to the virtual image display device having the above configuration, in a situation where the correspondence between the virtual image and the object is difficult to understand, the correspondence between the virtual image and the object is misunderstood by the occupant by performing correction as much as possible. It becomes possible to prevent.

The fourth configuration is as follows.
When correction by the correction means (41) is performed when the object (61) and the virtual image (10) are not superimposed, while at least a part of the object and the virtual image is superimposed Correction by the correction means is not performed.
According to the virtual image display device having the above-described configuration, the correspondence between the virtual image and the object is clear, and chattering of the display position of the virtual image occurs by not performing the correction particularly in a situation where the correction is not essential. Can be reduced as much as possible.

The fifth configuration is as follows.
The correction means (41) performs the correction when the ratio of the virtual image (10) to be superimposed on the object (61) is less than a predetermined ratio, while when the ratio is greater than or equal to the predetermined ratio, No correction is performed by the correction means.
According to the virtual image display device having the above-described configuration, the correspondence between the virtual image and the object is clear, and chattering of the display position of the virtual image occurs by not performing the correction particularly in a situation where the correction is not essential. Can be reduced as much as possible.

DESCRIPTION OF SYMBOLS 1 Virtual image display apparatus 2 Vehicle 3 Navigation apparatus 4 HUD
6 Liquid crystal display 8 Front window 10 Virtual image 14 Navigation ECU
41 CPU
42 RAM
43 ROM
61 Warning object 65 Other warning object

Claims (6)

A virtual image display device that is mounted on a vehicle and causes a vehicle occupant to visually recognize a video displayed on a video display surface so as to superimpose a virtual image of the video on an object in front of the sight line of the occupant. ,
Vehicle information acquisition means for acquiring the behavior of the vehicle;
Object information acquisition means for acquiring the behavior of the object;
Based on the behavior of the vehicle and the behavior of the object, the displacement of the future position of the object visually recognized by the occupant is predicted, and the display position of the virtual image is corrected based on the displacement of the predicted position. Correction means to
Superimposition state obtaining means for obtaining a superposition state of the object and the virtual image;
A virtual image display device comprising: correction switching means for switching whether or not correction by the correction means is performed based on a superimposed state of the object and the virtual image.   When the degree of superimposition of the object and the virtual image is less than a threshold value, the correction unit performs correction. On the other hand, when the degree of superimposition of the object and the virtual image is greater than or equal to a threshold value, the correction unit performs correction. The virtual image display device according to claim 1, wherein the virtual image display device is not.   3. The threshold value is set higher when there is another object to be superimposed with the virtual image around the object than when there is no other object around the object. Virtual image display device.   The correction by the correction unit is performed when the object and the virtual image are not superimposed, and the correction by the correction unit is not performed when at least a part of the object and the virtual image is superimposed. The virtual image display device according to claim 1.   The correction by the correction unit is performed when the ratio of the virtual image to be superimposed on the object is less than a predetermined ratio, and the correction by the correction unit is not performed when the ratio is equal to or greater than the predetermined ratio. The virtual image display device according to claim 1. A computer program that is mounted on a vehicle and causes a vehicle occupant to visually recognize a video displayed on a video display surface so that a virtual image of the video is superimposed on an object in front of the sight line of the occupant,
Computer
Vehicle information acquisition means for acquiring the behavior of the vehicle;
Object information acquisition means for acquiring the behavior of the object;
Based on the behavior of the vehicle and the behavior of the object, the displacement of the future position of the object visually recognized by the occupant is predicted, and the display position of the virtual image is corrected based on the displacement of the predicted position. Correction means to
Superimposition state obtaining means for obtaining a superposition state of the object and the virtual image;
Correction switching means for switching the execution of correction by the correction means based on the superimposed state of the object and the virtual image;
Computer program to make it function.

Images