JP2018195971A - Overhead image generation device, overhead image display system, overhead image generating method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately confirm a relative positional relationship between a vehicle and a surrounding area. SOLUTION: An image data acquisition unit 42, an overhead image generation unit 44 for generating an overhead image, a reference target detection unit 45 for detecting a reference target around the vehicle, and a reference target detected by the reference target detection unit 45. A direction specifying unit 46 for specifying a relative direction to the vehicle, and a superimposed image in which information indicating the relative direction between the reference target and the vehicle is superimposed on the overhead image as a relative direction based on the own vehicle icon. And a display control unit 49 for displaying the superimposed image on the display panel 31, and the bird's-eye view image generating unit 44 satisfies the predetermined condition in the relative positional relationship between the reference target and the vehicle. At this time, the second bird's-eye view image is generated instead of the bird's-eye view image based on the own vehicle icon. [Selection diagram] Figure 1

Description

  The present invention relates to an overhead view video generation device, an overhead view video display device, an overhead view video generation method, and a program.

  When the vehicle is parked, the driver parks the vehicle on a reference object such as a parking lot line, a vehicle stop, a roadway outer line, and a curb, for example. A technique related to a vehicle periphery display device that assists parking by displaying an overhead video of a vehicle together with a vehicle image when the vehicle is parked is known (see, for example, Patent Document 1 and Patent Document 2). The technique described in Patent Document 1 sets a boundary between adjacent parking areas from a car stop extending in the vehicle width direction in a parking lot where a white line is not drawn on the road surface as a boundary line between the parking areas, Superimposed display. The technique described in Patent Literature 2 displays an extended line image indicating an extended line of the parking frame superimposed on the overhead video when the predetermined portion of the parking frame is outside the display range.

JP2013-116696A JP 2008-083990 A

  When the vehicle is parked, the relative orientation of the vehicle and the reference line including the parking lot line and the vehicle stop varies in conjunction with the steering operation. In the bird's-eye view video in which the direction of the host vehicle icon is fixed, the direction of the reference object changes in conjunction with the steering operation with respect to the host vehicle icon. For this reason, it may be difficult to quickly determine how to perform the steering operation in order to match the direction of the vehicle with the direction of the reference object. As described above, there is room for improvement in displaying the relative positional relationship between the vehicle and the surroundings so that the relative positional relationship can be appropriately confirmed.

  The present invention has been made in view of the above, and an object of the present invention is to make it possible to appropriately confirm the relative positional relationship between the vehicle and the periphery.

  In order to solve the above-described problems and achieve the object, an overhead image generation apparatus according to the present invention includes a video data acquisition unit that acquires peripheral video data captured by a capturing unit that captures the periphery of a vehicle, and the video data. An overhead video generation unit that performs a viewpoint conversion process and a synthesis process including a host vehicle icon indicating a vehicle on the peripheral video acquired by the acquisition unit, and generates an overhead video that displays a predetermined display range from the vehicle, and surroundings of the vehicle A reference object detection unit that detects a reference object in the vehicle, a direction specifying unit that specifies a relative direction between the reference object detected by the reference object detection unit and the vehicle, and a relative relationship between the reference object and the vehicle. A superimposition processing unit that generates a superimposed video by superimposing information indicating a direction on the overhead video generated by the overhead video generation unit as a relative direction with respect to the vehicle icon; A display control unit that displays on the display unit the superimposed video generated by the vehicle, wherein the overhead video generation unit is configured to display the own vehicle when a relative positional relationship between the reference target and the vehicle satisfies a predetermined condition. Instead of the overhead view video based on the icon, the overhead view video based on the information indicating the relative direction is generated.

  An overhead video display device according to the present invention is at least one of the above-described overhead video generation device, a photographing unit that supplies peripheral video data to the video data acquisition unit, and a display unit that causes the display control unit to display a superimposed video. It is characterized by providing.

  An overhead view video generation method according to the present invention includes a video data acquisition step of acquiring peripheral video data captured by a video capturing unit that captures the periphery of a vehicle, a viewpoint conversion process and a vehicle on the peripheral video acquired by the video data acquisition step. An overhead view video generation step for generating a bird's-eye view video for displaying a predetermined display range from the vehicle, a reference target detection step for detecting a reference target around the vehicle, and the reference A direction specifying step for specifying a relative direction between the reference object detected by the object detection step and the vehicle, and information indicating a relative direction between the reference object and the vehicle relative to the vehicle icon. A superimposition processing step for generating a superimposed video superimposed on the overhead video generated by the overhead video generation step as a general orientation; A display control step for displaying a superimposed image generated by the control step on a display unit, and the overhead image generation step includes a step in which the relative positional relationship between the reference object and the vehicle satisfies a predetermined condition, Instead of the bird's-eye view image based on the own vehicle icon, the bird's-eye view image based on the information indicating the relative direction is generated.

  The program according to the present invention includes a video data acquisition step of acquiring peripheral video data captured by a photographing unit that captures the periphery of a vehicle, and a host vehicle that indicates a viewpoint conversion process and a vehicle in the peripheral video acquired by the video data acquisition step. An overhead view video generation step for performing a synthesis process including an icon and generating an overhead view video for displaying a predetermined display range from the vehicle, a reference target detection step for detecting a reference target around the vehicle, and the reference target detection step A direction specifying step for specifying a relative orientation between the reference object detected by the vehicle and the vehicle, and information indicating a relative orientation between the reference object and the vehicle relative to the vehicle icon as a reference A superimposition processing step for generating a superimposed video superimposed on the overhead video generated by the overhead video generation step, and the superimposition processing step. And a display control step for displaying a superimposed image generated by the top-up on a display unit, and the overhead image generation step includes a step when the relative positional relationship between the reference object and the vehicle satisfies a predetermined condition. Instead of the overhead view video based on the host vehicle icon, the computer operating as the overhead view video generation device is caused to generate the overhead view video based on the information indicating the relative direction.

  According to the present invention, it is possible to appropriately confirm the relative positional relationship between the vehicle and the periphery.

FIG. 1 is a block diagram illustrating a configuration example of the overhead video generation apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of an overhead video generated by the overhead video generation device according to the first embodiment. FIG. 3 is a diagram illustrating an example of the superimposed video generated by the overhead view video generation device according to the first embodiment. FIG. 4 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the first embodiment. FIG. 5 is a flowchart showing a flow of processing in the overhead view video generation apparatus according to the first embodiment. FIG. 6 is a diagram illustrating an example of a positional relationship between the host vehicle, the reference object, the display range, and the imaging range. FIG. 7 is a diagram illustrating another example of the positional relationship among the host vehicle, the reference object, the display range, and the shooting range. FIG. 8 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the first embodiment. FIG. 9 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the first embodiment. FIG. 10 is a diagram illustrating another example of the positional relationship among the host vehicle, the reference object, the display range, and the imaging range. FIG. 11 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the first embodiment. FIG. 12 is a flowchart showing a flow of processing in the overhead view video generation device according to the second embodiment. FIG. 13 is a diagram illustrating an example of the positional relationship between the host vehicle, the reference object, the display range, and the imaging range. FIG. 14 is a diagram illustrating an example of a superimposed video generated by the overhead view video generation device according to the second embodiment. FIG. 15 is a diagram illustrating another example of the positional relationship among the host vehicle, the reference object, the display range, and the imaging range. FIG. 16 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the second embodiment. FIG. 17 is a diagram illustrating an example of a positional relationship among the own vehicle, the reference object, the display range, and the imaging range. FIG. 18 is a diagram illustrating an example of the superimposed video generated by the overhead view video generation device according to the third embodiment. FIG. 19 is a diagram illustrating another example of the positional relationship among the host vehicle, the reference object, the display range, and the shooting range. FIG. 20 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the third embodiment. FIG. 21 is a diagram illustrating another example of the superimposed video generated by the overhead video generation device according to the third embodiment.

  DESCRIPTION OF EMBODIMENTS Embodiments of an overhead video generation device 40, an overhead video display device 1, an overhead video generation method, and a program according to the present invention will be described below in detail with reference to the accompanying drawings. In addition, this invention is not limited by the following embodiment.

[First embodiment]
FIG. 1 is a block diagram illustrating a configuration example of the overhead video generation apparatus according to the first embodiment. The overhead view video display device 1 generates a first overhead view image 100 (see FIG. 2), a superimposed video 100A (see FIG. 3), or a superimposed video 100B (see FIG. 9) of the vehicle. The overhead view video generation device 40 and the overhead view video display device 1 are mounted on a vehicle. In addition to what is mounted on the vehicle, the overhead image generation device 40 and the overhead image display device 1 may be portable devices that can be used in the vehicle.

  The overhead view video display device 1 will be described with reference to FIG. The overhead view video display device 1 includes a front camera (shooting unit) 11, a rear camera (shooting unit) 12, a left side camera (shooting unit) 13, a right side camera (shooting unit) 14, and a display panel (display unit). ) 31 and a bird's-eye view video generation device 40.

  The front camera 11 is an overhead video camera. The front camera 11 is disposed in front of the vehicle and captures a periphery around the front of the vehicle. The front camera 11 shoots the first shooting range A1 (see FIG. 6) of about 180 °, for example. The first shooting range A1 includes a wider range in front of the host vehicle V than the display range A of the first bird's-eye view image 100 (see FIG. 6). The front camera 11 outputs the captured video to the video data acquisition unit 42 of the overhead video generation device 40.

  The rear camera 12 is an overhead video camera. The rear camera 12 is disposed behind the vehicle and photographs a periphery around the rear of the vehicle. The rear camera 12 shoots the second shooting range A2 (see FIG. 6) of about 180 °, for example. The second shooting range A <b> 2 includes a wider range behind the host vehicle V than the display range A of the first bird's-eye view image 100. The rear camera 12 outputs the captured video to the video data acquisition unit 42 of the overhead video generation device 40.

  The left side camera 13 is an overhead video camera. The left-side camera 13 is disposed on the left side of the vehicle and captures the periphery around the left side of the vehicle. The left side camera 13 shoots a third shooting range A3 (see FIG. 6) of about 180 °, for example. The third shooting range A3 includes a wider range on the left side of the host vehicle V than the display range A of the first bird's-eye view video 100. The left side camera 13 outputs the captured video to the video data acquisition unit 42 of the overhead video generation device 40.

  The right side camera 14 is an overhead video camera. The right-side camera 14 is disposed on the right side of the vehicle and captures the periphery around the right side of the vehicle. The right side camera 14 shoots a fourth shooting range A4 (see FIG. 6) of about 180 °, for example. The fourth shooting range A4 includes a wider range on the right side of the host vehicle V than the display range A of the first bird's-eye view video 100. The right-side camera 14 outputs the captured video to the video data acquisition unit 42 of the overhead view video generation device 40.

  The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 photograph all directions of the vehicle.

  The display panel 31 is, for example, a display including a liquid crystal display (LCD) or an organic EL-Organic Electro-Luminescence (EL) display. The display panel 31 displays the first overhead video 100, the superimposed video 100A, or the superimposed video 100B based on the video signal output from the overhead video generation device 40 of the overhead video display device 1. The display panel 31 may be dedicated to the overhead view video display device 1 or may be used jointly with other systems including a navigation system, for example. The display panel 31 is disposed at a position that is easily visible to the driver.

  The overhead video generation device 40 includes a control unit 41 and a storage unit 50.

  The control unit 41 is an arithmetic processing unit configured with, for example, a CPU (Central Processing Unit). The control unit 41 loads the program stored in the storage unit 50 into the memory and executes instructions included in the program. The control unit 41 includes a video data acquisition unit 42, a vehicle information acquisition unit 43, an overhead video generation unit 44, a reference target detection unit 45, a direction identification unit 46, a superimposition processing unit 48, and a display control unit 49. Have The control unit 41 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the control unit 41.

  The video data acquisition unit 42 acquires peripheral video data obtained by photographing the periphery of the vehicle. More specifically, the video data acquisition unit 42 acquires video data output from the front camera 11, the rear camera 12, the left camera 13, and the right camera 14. The video data acquisition unit 42 outputs the acquired video data to the overhead video generation unit 44 and the reference target detection unit 45. Video data acquired from each camera is, for example, a moving image composed of 30 frames of images per second.

  The vehicle information acquisition unit 43 acquires vehicle information indicating a vehicle situation that triggers the start of display of a bird's-eye view image, such as vehicle gear operation information, from a CAN (Controller Area Network) or various sensors that sense the state of the vehicle. To do. The vehicle information acquisition unit 43 acquires a reverse trigger, for example. The vehicle information acquisition unit 43 acquires vehicle speed information, for example. The vehicle information acquisition unit 43 outputs the acquired vehicle information to the overhead view video generation unit 44.

  The bird's-eye view video generation unit 44 performs a viewpoint conversion process and a synthesis process including the host vehicle icon 110 indicating the vehicle from the surrounding image acquired by the video data acquisition unit 42 so as to look down from the upper side. A first bird's-eye view video 100 displaying the display range A is generated. More specifically, the bird's-eye view image generation unit 44 generates the first bird's-eye view image 100 based on images taken by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the first bird's-eye view image 100 may be any known method and is not limited. The overhead video generation unit 44 outputs the generated first overhead video 100 to the superimposition processing unit 48 and the display control unit 49.

  When the relative positional relationship between the reference object B and the host vehicle V does not satisfy the predetermined condition, the bird's-eye view image generation unit 44 generates the first bird's-eye view image 100 based on the host vehicle icon 110. In other words, the overhead view video generation unit 44 fixes the orientation and position of the host vehicle icon 110 while the relative positional relationship between the reference object B and the host vehicle V does not satisfy the predetermined condition. 100 is generated. In other words, the first bird's-eye view image 100 based on the host vehicle icon 110 is also the first bird's-eye view image based on the host vehicle V.

  The first bird's-eye view video 100 will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of an overhead video generated by the overhead video generation device according to the first embodiment. The first bird's-eye view image 100 has a vertically long rectangular shape. The first bird's-eye view image 100 is a central portion surrounded by a front image 101, a rear image 102, a left side image 103, a right side image 104, a front image 101, a rear image 102, a left side image 103, and a right side image 104. And the own vehicle icon 110 located in the area. The host vehicle icon 110 indicates the position and orientation of the host vehicle V. The host vehicle icon 110 is arranged in the center with the front-rear direction parallel to the front-rear direction of the first bird's-eye view image 100. In the rear image 102, a vehicle stop B1 and a rear frame line B2 are displayed as objects to be photographed. The front video 101, the rear video 102, the left side video 103, the right side video 104, and the host vehicle icon 110 may be separated by a frame-like boundary line. The driver knows the position and orientation of the vehicle stop B1 and the rear frame line B2 with respect to the host vehicle icon 110. Thus, the driver steers the host vehicle V according to the vehicle stop B1 and the rear frame line B2.

  In FIG. 2, diagonal broken lines indicating boundaries between the front video 101, the rear video 102, the left side video 103, and the right side video 104 are illustrated for explanation, but are actually displayed on the display panel 31. The broken line may not be displayed in the first bird's-eye view video 100. The same applies to the other figures.

  When the relative positional relationship between the reference object B and the host vehicle V satisfies a predetermined condition, the overhead view video generation unit 44 replaces the first overhead image 100 with the host vehicle icon 110 as a reference, A bird's-eye view video (not shown) (hereinafter referred to as “second bird's-eye view video”) based on information indicating a relative orientation with the host vehicle V is generated. In other words, while the relative positional relationship between the reference target B and the host vehicle V satisfies the predetermined condition, the overhead video generation unit 44 generates an overhead video in which the arrangement of information indicating the relative direction is fixed.

  In this embodiment, the bird's-eye view video generation unit 44 replaces the first bird's-eye view video 100 when the relative orientation difference between the reference object B specified by the direction specifying unit 46 and the host vehicle V is less than the threshold. The second overhead view video is generated. In this embodiment, the threshold is 5 °.

  In the second bird's-eye view video, the information indicating the relative direction is relative to the first bird's-eye view video 100 when it is determined that the difference in relative direction between the reference object B and the host vehicle V is less than the threshold value. You may fix by arrangement | positioning of the information which shows a right direction. Alternatively, in the second bird's-eye view video, the information indicating the relative direction may be arranged and fixed in parallel with the front-rear direction or the left-right direction of the bird's-eye view video. In the present embodiment, information indicating a relative direction is arranged and fixed in parallel with the horizontal direction of the overhead view video.

  The difference in the relative direction between the reference object B and the host vehicle V is the difference in the relative direction between the reference object B and the target portion of the host vehicle V that aligns with the reference object B. In the present embodiment, it is a difference in relative orientation between the reference object B and the rear end portion Va of the host vehicle V. More specifically, it is the difference in angle between the reference object B and the rear end portion Va of the host vehicle V.

  The bird's-eye view video generation unit 44 includes a viewpoint conversion processing unit 441, a cut-out processing unit 442, and a composition processing unit 443.

  The viewpoint conversion processing unit 441 performs viewpoint conversion processing on the surrounding video data acquired by the video data acquisition unit 42 so that the host vehicle V is looked down from above. More specifically, the viewpoint conversion processing unit 441 generates a video on which the viewpoint conversion processing has been performed based on peripheral video data captured by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method of viewpoint conversion processing may be any known method and is not limited. The viewpoint conversion processing unit 441 outputs the peripheral video data that has undergone the viewpoint conversion processing to the clipping processing unit 442.

  The cut-out processing unit 442 performs a cut-out process for cutting out a predetermined range of video from the peripheral video data subjected to the viewpoint conversion process. Which range is the cut-out range is registered and stored in advance. The cutout processing unit 442 outputs the video data of the video that has undergone the cutout processing to the composition processing unit 443.

  When the relative positional relationship between the reference target B and the host vehicle V satisfies a predetermined condition, the cutout processing unit 442 generates a second overhead view video from the peripheral video data subjected to the viewpoint conversion process. A cut-out process is performed to cut out the video in a cut-out range different from the cut-out range of the one-view image 100.

  The composition processing unit 443 performs composition processing for compositing the video data that has been subjected to the clipping processing. The composition processing unit 443 generates the first overhead image 100 or the second overhead image in which the host vehicle icon 110 is displayed on the synthesized image.

  The reference object detection unit 45 detects the reference object B present around the host vehicle V. The reference object detection unit 45 detects the reference object B imaged in the first imaging range A1, the second imaging range A2, the third imaging range A3, and the fourth imaging range A4. For this reason, the reference object B detected by the reference object detection unit 45 includes the reference object B that exists farther from the display range A. The reference target detection unit 45 performs object recognition processing on the peripheral video acquired by the video data acquisition unit 42 and detects the reference target B. The reference object detection unit 45 may detect, as the reference object B, an object to be photographed having a linearity of a predetermined length or more from the peripheral video by edge detection processing. The reference object detection unit 45 may detect the reference object B from the surrounding video using a recognition dictionary that stores the reference object B. The object recognition process may be an existing white line detection process, a specific object detection process using a recognition dictionary, or the like, or may be used in combination.

  The reference object B is an object that has a linearity of a predetermined length or more and serves as a reference for aligning the direction of the host vehicle V during parking. The reference object B is, for example, a curb, a vehicle stop, a parking lot line, a roadway outer line, or a line drawn on the road surface. The reference object B has directionality. In other words, the reference object B exists on a straight line along a predetermined direction. The reference object B extends straight along a predetermined direction without bending or bending. In the object recognition process, an object having a condition as the reference object B is detected.

  When the reference object detection unit 45 detects a plurality of reference objects B, the reference object B may be a reference object B having a small distance from the host vehicle V. Alternatively, the reference object detection unit 45 may select the reference object B present in the traveling direction of the host vehicle V when detecting a plurality of reference objects B. In this case, the reference object detection unit 45 detects the reference object B from the peripheral video in the traveling direction of the host vehicle V. Alternatively, when the reference object detection unit 45 detects a plurality of reference objects B, the reference object B may have a large linear continuity as the reference object B. Alternatively, for example, the reference object detection unit 45 may use the rear frame line as the reference object B when detecting a U-shaped parking frame line.

  The peripheral image in the traveling direction of the host vehicle V is image data output by at least one of the front camera 11, the rear camera 12, the left camera 13, and the right camera 14 facing the traveling direction of the host vehicle V. It is. For example, when the traveling direction of the host vehicle V is the rear, the video data captured by the rear camera 12 and the video data of the rear side of the vehicle among the video data captured by the left side camera 13 and the right side camera 14. It is.

  For the surrounding video in the traveling direction of the host vehicle V, priority may be given to video data output by a camera facing in a direction that is more likely to contact the host vehicle V. For example, when the traveling direction of the host vehicle V is backward, the video data captured by the rear camera 12 is the video data in the rear side of the vehicle among the video data captured by the left camera 13 and the right camera 14. More priority may be given. Alternatively, for example, when the traveling direction of the host vehicle V is being steered backward, the vehicle data in the video data captured by the rear camera 12 or the video data captured by the left-side camera 13 or the right-side camera 14 in the steering direction. Priority may be given to the video data in the rear side range of the vehicle among the video data in the rear side range and the video data taken by the right side camera 14 or the left side camera 13 in the direction opposite to the steering direction.

  The direction specifying unit 46 specifies the relative direction between the reference object B detected by the reference object detection unit 45 and the host vehicle V. More specifically, the direction specifying unit 46 specifies the relative direction between the reference target B and the host vehicle V from the direction and position in the video data of the reference target B detected by the reference target detection unit 45. For example, the orientation specifying unit 46 determines the relative relationship between the reference target B and the host vehicle V based on the direction and position in the video data of the reference target B detected by the reference target detection unit 45 and the direction of the optical axis of the photographed camera. Specific orientation may be specified. For example, the orientation specifying unit 46 determines the relative relationship between the reference target B and the host vehicle V based on the orientation and position of the reference target B in the video data detected by the reference target detection unit 45 and the orientation of the subject of the video data. Specific orientation may be specified. In the present embodiment, the relative direction between the reference object B and the host vehicle V is a relative direction with respect to the rear end portion Va of the host vehicle V.

  The superimposition processing unit 48 superimposes the superimposed image 100A on the first overhead image 100 with information indicating the relative direction between the reference object B and the host vehicle V as a relative direction based on the host vehicle icon 110. Generate. In the present embodiment, the superimposition processing unit 48 indicates information indicating a relative direction between the reference object B specified by the direction specifying unit 46 and the host vehicle V as a reference direction icon (information indicating the direction) 120. A superimposed image 100A superimposed on the overhead image 100 is generated. More specifically, in the superimposed image 100A, the superimposition processing unit 48 has the relative orientation specified by the orientation specifying unit 46 in the relative orientation between the reference orientation icon 120 and the rear end portion 110a of the host vehicle icon 110. Thus, the reference orientation icon 120 is superimposed.

  When the relative positional relationship between the reference object B and the host vehicle V satisfies a predetermined condition, the superimposition processing unit 48 superimposes the reference orientation icon 120 on the second overhead image instead of the first overhead image 100. A superimposed image 100B is generated.

  The reference orientation icon 120 is an icon that notifies the orientation of the reference target B. In the present embodiment, the reference direction icon 120 is a broken line extending along the extending direction of the reference object B.

  The superimposition processing unit 48 superimposes the reference orientation icon 120 on the side where the reference target B exists with respect to the host vehicle icon 110. For example, when the reference object B is detected behind the host vehicle V, the superimposition processing unit 48 superimposes the reference direction icon 120 on the rear video 102 that is behind the host vehicle icon 110. More specifically, for example, when the reference object B is detected behind the host vehicle V, the superimposition processing unit 48 sets the reference orientation icon 120 with the center of the rear image 102 as a reference, in other words, the center of the rear image 102. You may superimpose so that it may pass. For example, when the reference object B is detected behind the host vehicle V, the superimposition processing unit 48 may superimpose the reference orientation icon 120 so that only the rear video 102 passes. For example, when the reference object B is detected behind the host vehicle V, the superimposition processing unit 48 passes the reference direction icon 120 through a position away from the end of the host vehicle icon 110 on the traveling direction side by a predetermined distance. You may superimpose on.

  The superimposed image 100A will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of the superimposed video generated by the overhead view video generation device according to the first embodiment. The superimposed image 100 </ b> A has the same vertically long rectangular shape as the first overhead image 100. The superimposed video 100A includes a front video 101A, a rear video 102A, a left side video 103A, a right side video 104A, a host vehicle icon 110, and a superimposed reference direction icon 120. The reference orientation icon 120 is superimposed on the rear image 102A. The reference direction icon 120 indicates the direction with respect to the reference target B, for example, the rear frame line B2. The reference direction icon 120 allows the driver to know that there is a reference object B along the direction of the reference direction icon 120 in the traveling direction of the vehicle. Accordingly, the driver steers the host vehicle V in accordance with the reference direction icon 120.

  In FIG. 3, the rotation center 120 a is illustrated in a cross shape, but the cross shape may not be displayed in the superimposed image 100 </ b> A actually displayed on the display panel 31. The same applies to the other figures.

  When the relative positional relationship between the reference object B and the host vehicle V changes while the superimposed image 100A is displayed, the reference orientation icon 120 rotates around the rotation center 120a in the superimposed image 100A.

  While the superimposed image 100 </ b> B is displayed, when the relative direction of the reference object B and the host vehicle V changes, the direction of the host vehicle icon 110 changes with respect to the reference direction icon 120.

  Alternatively, the superimposition processing unit 48 may superimpose the reference orientation icon 120 on the basis of the center of the host vehicle icon 110, in other words, so as to pass through the center of the host vehicle icon 110.

  The superimposed image 100A will be described with reference to FIG. FIG. 4 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the first embodiment. The superimposed image 100A is different from the superimposed image 100A shown in FIG. 3 in that the reference orientation icon 120 is superimposed so as to pass through the center of the host vehicle icon 110.

  The display control unit 49 displays the first bird's-eye view image 100, the superimposed image 100A, or the superimposed image 100B on the display panel 31.

  The storage unit 50 stores data and various processing results required for various processes in the overhead view video generation device 40. The storage unit 50 is, for example, a semiconductor memory device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

  Next, a processing flow in the overhead view video generation device 40 will be described with reference to FIG. FIG. 5 is a flowchart showing a flow of processing in the overhead view video generation apparatus according to the first embodiment.

  The control unit 41 determines whether or not to start the overhead view video display (step S11). As an example of the determination to start the overhead view video display, the control unit 41 determines whether to start the overhead view video display based on the presence or absence of the reverse trigger. The reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle is rearward in the front-rear direction of the vehicle. When there is no reverse trigger, the control unit 41 determines not to start the overhead view video display (No in step S11), and executes the process of step S11 again. When there is a backward trigger, the control unit 41 determines to start the overhead view video display (Yes in Step S11), and proceeds to Step S12.

  The control part 41 produces | generates the 1st bird's-eye view image | video 100 (step S12). More specifically, the control unit 41 generates a first bird's-eye view image 100 in which the viewpoint conversion is performed so that the vehicle is looked down from above from the surrounding image acquired by the image data acquisition unit 42 in the bird's-eye view image generation unit 44. The control unit 41 proceeds to step S13.

  The control unit 41 determines whether or not the reference target B has been detected (step S13). More specifically, the control unit 41 determines whether or not the reference target B is detected by the reference target detection unit 45. When it is determined that the reference target B is detected by the reference target detection unit 45 (Yes in step S13), the control unit 41 proceeds to step S14. When the reference target detection unit 45 determines that the reference target B is not detected (No in step S13), the control unit 41 proceeds to step S20.

  The control unit 41 determines whether or not the reference object B exists in the display range A of the first overhead video 100 (step S14). More specifically, the control unit 41 determines whether or not the position where the reference object B is detected by the reference object detection unit 45 is included in the display range A of the first overhead image 100. If the control unit 41 determines that the reference object B is included in the display range A of the first overhead video 100 (Yes in step S14), the control unit 41 proceeds to step S20. When the control unit 41 determines that the reference object B is not included in the display range A of the first overhead video 100 (No in step S14), the control unit 41 proceeds to step S15.

  The control part 41 may change to step S15, without performing the process of step S14, after it determines with Yes by step S13. The control unit 41 performs the process of step S14 when the reference object B is detected for the first time after the display of the overhead view video in step S11 is started, and after the superimposed overhead view video is once displayed in step S19, in step S13. After the determination is Yes, the process may proceed to step S15 without performing the process of step S14.

  The control unit 41 specifies a relative direction (step S15). More specifically, the control unit 41 specifies the relative direction between the reference target B and the host vehicle V from the direction and position in the video data of the reference target B detected by the reference target detection unit 45 by the direction specifying unit 46. Let The control unit 41 proceeds to step S16.

  The control unit 41 determines whether or not the relative orientation difference is less than the threshold (step S16). More specifically, the control unit 41 determines whether or not the relative direction difference between the reference object B specified by the direction specifying unit 46 and the host vehicle V is less than a threshold value. If the control unit 41 determines that the relative orientation difference is less than the threshold value (Yes in step S16), the control unit 41 proceeds to step S17. If the control unit 41 determines that the relative direction difference is not less than the threshold (No in step S16), the control unit 41 proceeds to step S18.

  The control unit 41 generates the superimposed video 100B with the reference object B as a reference (step S17). More specifically, the control unit 41 generates a second overhead view video with the reference target B as a reference at the overhead view video generation unit 44. And the control part 41 produces | generates the superimposed image | video 100B which made the superimposition process part 48 superimpose the reference | standard direction icon 120 which shows the direction of the reference | standard object B on the 2nd bird's-eye view image | video. In the superimposition processing unit 48, the control unit 41 generates a superimposed image 100 </ b> B in which the reference direction icon 120 is superimposed on the side where the reference target B is present on the host vehicle icon 110. The control unit 41 proceeds to step S19.

  The control unit 41 generates a superimposed image 100A based on the host vehicle icon 110 (step S18). More specifically, the control unit 41 uses the overhead view video generation unit 44 to generate the first overhead view video 100 based on the host vehicle icon 110. The control unit 41 causes the superimposition processing unit 48 to generate a superimposed image 100A in which the reference orientation icon 120 is superimposed on the first overhead image 100 with the relative orientation with respect to the host vehicle icon 110 as a reference. In the superimposition processing unit 48, the control unit 41 superimposes the reference orientation icon 120 on the side where the reference target B exists with respect to the host vehicle icon 110. The control unit 41 proceeds to step S19.

  The control unit 41 displays the superimposed video 100A or the superimposed video 100B (step S19). More specifically, the control unit 41 causes the display control unit 49 to display the superimposed video 100A or the superimposed video 100B generated by the superimposition processing unit 48 on the display panel 31. The control unit 41 proceeds to step S21.

  The control unit 41 displays the first bird's-eye view video 100 (step S20). More specifically, the control unit 41 causes the display control unit 49 to display the first overhead image 100 on the display panel 31. The control unit 41 proceeds to step S21.

  The control unit 41 determines whether or not to end the overhead view video display (step S21). More specifically, the control unit 41 determines whether or not to end the overhead view video display including the display of the first overhead view video 100, the superimposed video 100A, or the superimposed video 100B based on the presence or absence of the reverse end trigger. The reverse end trigger means, for example, that the shift position has changed from “reverse” to another position. In step S21, the displayed video is the first overhead video 100, the superimposed video 100A, or the superimposed video 100B. When there is a reverse end trigger, the control unit 41 determines to end the display of the first bird's-eye view image 100, the superimposed image 100A, or the superimposed image 100B (Yes in step S21), and ends the process. When there is no reverse end trigger, the control unit 41 determines that the display of the first bird's-eye view image 100, the superimposed image 100A, or the superimposed image 100B is not finished (No in step S21), and executes the process of step S13 again.

  In this way, when the reference object B is detected around the host vehicle V, the overhead view video display device 1 generates a video signal for displaying the first overhead view video 100, the superimposed video 100A, or the superimposed video 100B on the display panel 31. Output. The display panel 31 displays, for example, the superimposed video 100A together with the navigation screen based on the video signal output from the overhead video display device 1.

  For example, referring to FIGS. 6, 3, 7 to 11, when a vehicle stop B1 and a rear frame line B2 exist at the rear when the vehicle moves backward, the superimposed image 100A generated by the overhead view video display device 1 is used. Alternatively, an example of the superimposed image 100B will be described. FIG. 6 is a diagram illustrating an example of a positional relationship between the host vehicle, the reference object, the display range, and the imaging range. FIG. 7 is a diagram illustrating another example of the positional relationship among the host vehicle, the reference object, the display range, and the shooting range. FIG. 8 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the first embodiment. FIG. 9 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the first embodiment. FIG. 10 is a diagram illustrating another example of the positional relationship among the host vehicle, the reference object, the display range, and the imaging range. FIG. 11 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the first embodiment.

  First, the vehicle is positioned in the vicinity of the parking position, and the shift position is set to “reverse”. The vehicle information acquisition unit 43 acquires a reverse trigger. At this time, as shown in FIG. 6, it is assumed that the host vehicle V, the vehicle stop B1, and the rear frame B2 are separated from each other. In the parking section shown in FIG. 6, there is no side frame line that partitions the vehicle width direction. The vehicle stop B <b> 1 and the rear frame line B <b> 2 exist in the second imaging range A <b> 2 and are located farther from the display range A of the first overhead view image 100. In the present embodiment, in the reference object detection unit 45, a reference object B that is present in the traveling direction of the host vehicle V and has a large linear continuity is set as the reference object B. For this reason, in this embodiment, the back frame line B2 is set as the reference object B. The difference in angle between the rear frame B2 that is the reference object B and the rear end portion Va of the host vehicle V is equal to or greater than a threshold value.

  In step S <b> 11, the control unit 41 determines that there is a reverse trigger and starts the overhead view video display. In step S <b> 12, the control unit 41 generates the first bird's-eye view image 100 using the bird's-eye view image generation unit 44. In step S <b> 13, the control unit 41 determines that, for example, the rear frame B <b> 2 is detected as the reference target B. In step S <b> 14, the control unit 41 determines that the rear frame line B <b> 2 does not exist in the display range A of the first overhead image 100. And in step S15, the control part 41 specifies the relative direction of the back frame line B2. In step S16, the control unit 41 determines that the relative orientation difference is not less than the threshold (No in step S16). In step S <b> 18, the control unit 41 generates a superimposed image 100 </ b> A in which the reference orientation icon 120 is superimposed on the first overhead image 100 based on the host vehicle icon 110. In step S <b> 19, the control unit 41 displays the generated superimposed video 100 </ b> A on the display panel 31.

  For example, the control unit 41 displays a superimposed image 100A as shown in FIG. The superimposed image 100A is based on the host vehicle icon 110. The reference orientation icon 120 is superimposed on the rear image 102A. The difference in angle between the reference orientation icon 120 and the rear end 110a of the host vehicle icon 110 is equal to or greater than a threshold value.

  FIG. 7 shows a state in which the host vehicle V moves backward while being steered from the state shown in FIG. 6 so that the host vehicle V and the rear frame line B2 approach each other. The vehicle stop B <b> 1 and the rear frame line B <b> 2 exist in the second imaging range A <b> 2 and are located farther from the display range A of the first overhead view image 100. The difference in angle between the rear frame line B2 and the rear end portion Va of the host vehicle V is equal to or greater than a threshold value.

  For example, the control unit 41 displays a superimposed image 100A as shown in FIG. The superimposed image 100A is based on the host vehicle icon 110. The reference orientation icon 120 rotates counterclockwise around the rotation center 120a with respect to the state shown in FIG. The difference in angle between the reference orientation icon 120 and the rear end 110a of the host vehicle icon 110 is equal to or greater than a threshold value.

  Furthermore, the state where the host vehicle V is steered from the state shown in FIG. 7 and the angle between the rear frame line B2 and the rear end portion Va of the host vehicle V is less than the threshold will be described.

  In step S13, the control unit 41 determines that the rear frame line B2 has been detected. In step S <b> 14, the control unit 41 determines that the rear frame line B <b> 2 does not exist in the display range A of the first overhead image 100. And in step S15, the control part 41 specifies the relative direction of the back frame line B2. In step S16, the control unit 41 determines that the relative orientation difference is less than the threshold (Yes in step S16). In step S <b> 17, the control unit 41 generates a superimposed image 100 </ b> B in which the reference orientation icon 120 is superimposed on the overhead view image with the rear frame line B <b> 2 as a reference. In step S <b> 19, the control unit 41 displays the generated superimposed video 100 </ b> B on the display panel 31.

  For example, the control unit 41 displays a superimposed image 100B as shown in FIG. The superimposed image 100B is based on the reference orientation icon 120. The reference orientation icon 120 is arranged in parallel with the left-right direction. The difference in angle between the reference orientation icon 120 and the rear end 110a of the host vehicle icon 110 is less than the threshold value. The own vehicle icon 110 is arranged to be inclined with respect to the reference orientation icon 120. More specifically, the host vehicle icon 110 rotates counterclockwise with respect to the state shown in FIG.

  For example, when a turn-back operation is performed to adjust the parking position in the state where the superimposed image 100B is displayed, the host vehicle icon is linked to the change in the relative direction between the host vehicle V and the rear frame line B2. The host vehicle icon 110 rotates around the center of 110 as the rotation center. The reference orientation icon 120 is fixed.

  Further, FIG. 10 shows a state where the host vehicle V moves backward while being steered and the host vehicle V is positioned in the vicinity of the rear frame line B2. The vehicle stop B1 and the rear frame line B2 exist in the second imaging range A2 and exist in the display range A of the first overhead image 100. The difference in angle between the rear frame line B2 and the rear end portion Va of the host vehicle V is zero. The rear frame line B2 and the rear end portion Va of the host vehicle V are parallel.

  In step S13, the control unit 41 determines that the rear frame line B2 has been detected. In step S <b> 14, the control unit 41 determines that the rear frame line B <b> 2 exists in the display range A of the first overhead view video 100. In step S <b> 20, the control unit 41 displays the first bird's-eye view image 100 on the display panel 31.

  For example, the control unit 41 displays the first overhead image 100 as shown in FIG. 11 on the display panel 31. In the rear image 102, a vehicle stop B1 and a rear frame line B2 are displayed as objects to be photographed. In step S20, as shown in FIG. 11, the reference orientation icon 120 may be superimposed and displayed together with the vehicle stop B1 and the rear frame line B2. Alternatively, the reference orientation icon 120 may not be displayed in step S20.

  As described above, when the difference in relative orientation between the reference object B and the host vehicle V is less than the threshold value, the reference orientation icon 120 is used as a reference instead of the first bird's-eye view image 100 using the host vehicle icon 110 as a reference. The superimposed image 100B is displayed on the display panel 31. According to this embodiment, while the relative difference between the reference object B and the host vehicle V is less than the threshold value, the superimposed image 100B in which the direction of the reference direction icon 120 is fixed is displayed. Thus, according to the present embodiment, when the difference in the relative direction between the reference object B and the host vehicle V is less than the threshold value, a display that makes it easy to quickly determine the direction of the host vehicle icon 110 with respect to the reference direction icon 120 is displayed. can do.

  In particular, according to the present embodiment, when the parking position is fine-adjusted because the difference in relative orientation between the reference object B and the host vehicle V becomes small at the time of parking, the host vehicle is compared with the fixed reference direction icon 120. The direction of the icon 110 changes. Thereby, this embodiment can be operated so that the own vehicle V becomes an appropriate direction while confirming whether or not the own vehicle V is suitable for the direction of the reference target B.

  As a modification of the first embodiment, for example, in the process of the flowchart shown in FIG. 5, step S <b> 16 in a state where the superimposed overhead view image 100 </ b> A based on the host vehicle icon 110 is generated is determined as Yes, The threshold of the relative direction difference for starting the generation of the superimposed overhead view video 100B with reference to B is set as a first threshold, and the threshold is set to 5 °. Further, when the superimposed overhead view video 100B based on the reference object B is generated, the determination in step S16 is No, and relative for starting generation of the superimposed overhead view video 100A based on the host vehicle icon 110. The threshold value for the difference in direction may be a second threshold value that is larger than the first threshold value, and the threshold value may be 15 °. Thereby, the display of the superimposed overhead view video 100B with reference to the reference object B is continued only when the relative orientation difference between the reference object B and the host vehicle V is small and the parking position is finely adjusted. be able to.

  As another modification of the first embodiment, for example, in the process of the flowchart shown in FIG. 5, the determination in step S <b> 16 is Yes when the superimposed overhead view image 100 </ b> A based on the host vehicle icon 110 is generated. After the generation of the superimposed overhead view video 100B based on the reference target B is started, the determination of step S16 is not performed, and the generation of the superimposed overhead view video 100B based on the reference target B is determined until the determination in step S21 is Yes. May be continued. Thereby, it can suppress that the direction of the image | video displayed is switched frequently.

  Thus, this embodiment can make it possible to appropriately confirm the relative positional relationship between the vehicle and the periphery.

[Second Embodiment]
The overhead image display apparatus 1 according to the present embodiment will be described with reference to FIGS. FIG. 12 is a flowchart showing a flow of processing in the overhead view video generation device according to the second embodiment. FIG. 13 is a diagram illustrating an example of the positional relationship between the host vehicle, the reference object, the display range, and the imaging range. FIG. 14 is a diagram illustrating an example of a superimposed video generated by the overhead view video generation device according to the second embodiment. FIG. 15 is a diagram illustrating another example of the positional relationship among the host vehicle, the reference object, the display range, and the imaging range. FIG. 16 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the second embodiment. The overhead image display device 1 has the same basic configuration as the overhead image display device 1 of the first embodiment. In the following description, the same components as those of the overhead video display device 1 are denoted by the same reference numerals or corresponding reference numerals, and detailed description thereof is omitted.

  When the relative orientation between the reference object B identified by the orientation identifying unit 46 and the host vehicle V matches the orientation indicated by the host vehicle icon 110, the overhead view video generation unit 44 replaces the first overhead view video 100 with the first Generates a two-view video. In other words, the bird's-eye view video generation unit 44 generates a bird's-eye view video in which the orientation of the reference orientation icon 120 is fixed when the orientations of the reference target B and the target portion of the host vehicle V that aligns with the reference target B match. In the present embodiment, the bird's-eye view image generation unit 44 generates a bird's-eye view image in which the direction and position of the reference direction icon 120 are fixed when the directions of the reference object B and the rear end portion Va of the host vehicle V coincide.

  Next, a processing flow in the overhead view video generation device 40 will be described with reference to FIG. The process of the flowchart shown in FIG. 12 performs the process of step S36 instead of the process of step S16 of the flowchart shown in FIG. The process of step S31-step S35 and step S37-step S41 performs the process similar to step S11-step S15 and step S17-step S21.

  The control unit 41 determines whether or not the relative directions match (step S36). More specifically, the control unit 41 determines whether or not the directions of the reference target B specified by the direction specifying unit 46 and the target part of the host vehicle V match. If the control unit 41 determines that the relative orientations match (Yes in step S36), the control unit 41 proceeds to step S37. When it is determined that the relative directions do not match (No in step S36), the control unit 41 proceeds to step S38. The term “match” as used herein may include a range of about ± 2 °, for example, in addition to perfect match. Here, the judgment of coincidence is a case where the coincidence is temporarily coincident in a state where the relative orientation is fluctuating.

  When the difference in the direction of the host vehicle V with respect to the reference target B becomes zero, the direction of the host vehicle V matches the reference target B. The direction of the host vehicle V with respect to the reference target B is when the direction of the host vehicle V becomes parallel to the reference target B or becomes vertical. In other words, the direction of the host vehicle V with respect to the reference target B is when the direction of the host vehicle V is the correct direction with respect to the reference target B.

  For example, with reference to FIGS. 13 to 16, an example of the superimposed image 100 </ b> B generated by the overhead view video display device 1 when the vehicle stop B <b> 1 and the rear frame line B <b> 2 exist behind when the vehicle is reversing will be described.

  As shown in FIG. 13, a state in which the rear frame line B2 and the rear end portion Va of the host vehicle V coincide with each other will be described. The difference in angle between the rear frame line B2 and the rear end portion Va of the host vehicle V is zero. The rear frame line B2 and the rear end portion Va of the host vehicle V are parallel.

  In step S33, the control unit 41 determines that the rear frame line B2 has been detected. In step S <b> 34, the control unit 41 determines that the rear frame line B <b> 2 does not exist in the display range A of the first overhead image 100. And in step S35, the control part 41 specifies the relative direction of the back frame line B2. In step S36, the control unit 41 determines that the relative directions match (Yes in step S36). In step S <b> 37, the control unit 41 generates a superimposed image 100 </ b> B in which the reference orientation icon 120 is superimposed on the bird's-eye view image based on the rear frame line B <b> 2. In step S39, the control unit 41 displays the generated superimposed video 100B on the display panel 31.

  For example, the control unit 41 displays a superimposed image 100B as shown in FIG. The superimposed image 100B is based on the reference orientation icon 120. The reference orientation icon 120 is arranged in parallel with the left-right direction. The orientations of the reference orientation icon 120 and the rear end portion 110a of the host vehicle icon 110 are the same.

  FIG. 15 shows a state turned back to adjust the parking position from the state shown in FIG. The direction of the rear frame line B2 and the rear end portion Va of the host vehicle V do not match.

  For example, the control unit 41 displays a superimposed image 100B as shown in FIG. The superimposed image 100B has the same orientation and position as the reference orientation icon 120 of the superimposed image 100B shown in FIG. The own vehicle icon 110 is arranged to be inclined with respect to the reference orientation icon 120. More specifically, the host vehicle icon 110 rotates counterclockwise with respect to the state shown in FIG.

  As described above, when the relative direction between the reference object B and the host vehicle V matches the direction indicated by the host vehicle icon 110, the reference direction icon is used instead of the first bird's-eye view image 100 based on the host vehicle icon 110. A superimposed image 100B based on 120 is displayed on the display panel 31. According to this embodiment, when the relative orientations of the reference object B and the host vehicle V coincide with each other, the superimposed image 100B in which the orientation of the reference orientation icon 120 is fixed is displayed. Thereby, especially this embodiment can be made into the display which is easy to confirm having shifted | deviated after the relative direction of the reference | standard object B and the own vehicle V corresponded, when turning back at the time of parking. . Thus, this embodiment can make it possible to appropriately confirm the relative positional relationship between the vehicle and the periphery.

  As a modified example of the second embodiment, for example, in the process of the flowchart shown in FIG. 12, step S <b> 36 in a state where the superimposed overhead view image 100 </ b> A based on the host vehicle icon 110 is generated is determined as Yes. The determination in step S36 after the generation of the superimposed overhead view video 100B with reference to B is started, as in step S16 in the first embodiment, for example, whether or not the threshold is less than about 5 ° to 15 °. Thus, step S36 may be determined as No. Thereby, after step S36 is judged Yes, only when the difference in relative orientation between the reference object B and the host vehicle V is less than the threshold value, the superimposed overhead view video 100B based on the reference object B is displayed. The display can be continued.

  As another modification of the second embodiment, for example, in the process of the flowchart shown in FIG. 12, the determination in step S <b> 36 is Yes when the superimposed overhead view image 100 </ b> A based on the host vehicle icon 110 is generated. After the generation of the superimposed overhead video 100B based on the reference object B is started, the determination of step S36 is not performed, and the generation of the superimposed overhead video 100B based on the reference object B is determined until the determination in step S41 is Yes. May be continued. Thereby, it can suppress that the direction of the image | video displayed is switched frequently.

[Third embodiment]
The overhead image display apparatus 1 according to the present embodiment will be described with reference to FIGS. FIG. 17 is a diagram illustrating an example of a positional relationship among the own vehicle, the reference object, the display range, and the imaging range. FIG. 18 is a diagram illustrating an example of the superimposed video generated by the overhead view video generation device according to the third embodiment. FIG. 19 is a diagram illustrating another example of the positional relationship among the host vehicle, the reference object, the display range, and the shooting range. FIG. 20 is a diagram illustrating another example of the superimposed video generated by the overhead view video generation device according to the third embodiment. FIG. 21 is a diagram illustrating another example of the superimposed video generated by the overhead video generation device according to the third embodiment.

  In this embodiment, it is assumed that the host vehicle V is parked in parallel along the roadway outer line B3 that is the reference target B. There are another vehicle V1 and another vehicle V2 across a space where the host vehicle V is to be parked.

  The reference target detection unit 45 performs object recognition processing on the peripheral video acquired by the video data acquisition unit 42, and detects the roadway outer line B3 as the reference target B existing around the host vehicle V.

  For example, when the reference object B is detected on the left rear side of the host vehicle V, the superimposition processing unit 48 changes the reference direction icon 120 into the rear video 102 and the left side video 103 that are left rear with respect to the host vehicle icon 110. Superimpose. More specifically, for example, when the reference object B is detected to the left rear of the host vehicle V, the superimposition processing unit 48 displays the reference orientation icon 120 at the midpoint of the boundary line between the rear image 102A and the left image 103A. You may superimpose so that it may pass. For example, when the reference object B is detected to the left rear of the host vehicle V, the superimposition processing unit 48 passes the reference direction icon 120 through a position away from the left rear end of the host vehicle icon 110 by a predetermined distance. You may superimpose.

  Alternatively, the superimposition processing unit 48 may superimpose the reference orientation icon 120 so as to pass through the center of the host vehicle icon 110.

  For example, referring to FIG. 17 to FIG. 21, when a vehicle roadside parking line B3 that is the reference target B exists at the left rear when the vehicle is parked in parallel, the superimposed video 100A or the superimposed video 100B generated by the overhead view video display device 1 is used. An example will be described.

  First, the vehicle is positioned in the vicinity of the parking position, and the shift position is set to “reverse”. The vehicle information acquisition unit 43 acquires a reverse trigger. At this time, as shown in FIG. 17, it is assumed that the host vehicle V and the roadway outside line B3 are separated. The roadway outer line B3 exists across the second shooting range A2 and the third shooting range A3, and is located farther from the display range A of the first overhead view video 100. The difference in angle between the roadway outer line B3 and the left side Vb of the host vehicle V is equal to or greater than a threshold value.

  For example, the control unit 41 displays a superimposed image 100A as shown in FIG. The superimposed image 100A is based on the host vehicle icon 110. The reference orientation icon 120 is superimposed on the rear video 102A and the left video 103A. The difference in angle between the reference orientation icon 120 and the left side 110b of the host vehicle icon 110 is equal to or greater than a threshold value.

  FIG. 19 shows a state where the host vehicle V moves backward while being steered from the state shown in FIG. 17 and the host vehicle V is positioned in the vicinity of the roadway outer line B3. The roadway outer line B3 exists across the first shooting range A1, the second shooting range A2, and the third shooting range A3, and is located farther from the display range A of the first overhead image 100. The other vehicle V1 exists in the display range A and exists in the first shooting range A1. The other vehicle V2 exists in the display range A and exists in the second imaging range A2.

  For example, the control unit 41 displays a superimposed image 100A as shown in FIG. The superimposed image 100A is based on the host vehicle icon 110. In the superimposed video 100A, the other vehicle V1 and the other vehicle V2 are displayed. The reference orientation icon 120 rotates clockwise around the rotation center 120a with respect to the state shown in FIG.

  Furthermore, a state in which the host vehicle V is steered and the relative direction difference between the roadway outer line B3 and the left side Vb of the host vehicle V is less than the threshold will be described.

  For example, the control unit 41 displays a superimposed image 100B as shown in FIG. The superimposed image 100B is based on the reference orientation icon 120. The reference direction icons 120 are arranged in parallel in the front-rear direction. The difference in angle between the reference orientation icon 120 and the left side 110b of the host vehicle icon 110 is less than the threshold value. The host vehicle icon 110 is displayed tilted with respect to the reference orientation icon 120. More specifically, the host vehicle icon 110 rotates clockwise with respect to the state shown in FIG.

  As described above, when the parallel parking is performed, if the difference in relative orientation between the reference object B and the host vehicle V is less than the threshold, the first overhead image 100 based on the host vehicle icon 110 is used as a reference. The superimposed image 100B based on the orientation icon 120 is displayed on the display panel 31. Thus, this embodiment can make it possible to appropriately confirm the relative positional relationship between the vehicle and the surroundings even during parallel parking.

  Now, the overhead video display device 1 according to the present invention has been described so far, but the present invention may be implemented in various different forms other than the above-described embodiment.

  Each component of the overhead view video display apparatus 1 shown in the figure is functionally conceptual, and may not necessarily be physically configured as illustrated. That is, the specific form of each device is not limited to the one shown in the figure, and all or a part of them is functionally or physically distributed or integrated in arbitrary units according to the processing load or usage status of each device. May be.

  The configuration of the overhead view video display device 1 is realized by, for example, a program loaded into a memory as software. The above embodiment has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The components described above include those that can be easily assumed by those skilled in the art and those that are substantially the same. Furthermore, the above-described configurations can be appropriately combined. In addition, various omissions, substitutions, or changes in the configuration can be made without departing from the scope of the present invention.

  The superimposition processing unit 48 may superimpose the reference orientation icon 120 by changing the display mode according to the relative orientation between the reference target B and the host vehicle V. For example, the reference direction icon 120 may change at least one of the thickness, the color, and the line type according to the direction of the host vehicle V with respect to the reference target B. The reference direction icon 120 is indicated by a thin line when the difference in the direction of the host vehicle V with respect to the reference target B is greater than a predetermined value, and the line becomes thicker as the difference in the direction of the host vehicle V with respect to the reference target B becomes smaller. It may be displayed. The reference direction icon 120 is displayed in red when the difference in the direction of the host vehicle V with respect to the reference target B is greater than a predetermined value, and changes to green as the difference in the direction of the host vehicle V with respect to the reference target B decreases. May be displayed. The reference direction icon 120 is displayed by a broken line when the difference in the direction of the host vehicle V with respect to the reference target B is larger than a predetermined value, and changes to a solid line as the difference in the direction of the host vehicle V with respect to the reference target B decreases. May be displayed.

  The host vehicle icon 110 may be displayed by coloring an end portion in the traveling direction of the host vehicle V, in other words, a target portion of the host vehicle V whose direction is aligned with the reference orientation icon 120. Thereby, confirmation of direction of standard object B and self-vehicle V becomes easier by superposition picture 100A. The target portion whose direction is aligned with the reference orientation icon 120 performs a relative positional relationship between the traveling direction of the host vehicle V and the reference target B and image processing on the captured data, and a positional relationship with surrounding objects. You may specify by. Alternatively, the target portion whose direction is aligned with the reference orientation icon 120 may specify a portion selected by the user on the host vehicle icon 110 displayed on the display panel 31 as the target portion.

  The reference orientation icon 120 has been described as being configured with a broken line, but is not limited thereto. The reference orientation icon 120 may be, for example, a strip shape.

1 Overhead video display device 11 Front camera (shooting unit)
12 Rear camera (shooting unit)
13 Left-side camera (shooting unit)
14 Right-side camera (shooting unit)
31 Display panel (display section)
40 Overhead Image Generation Device 41 Control Unit 42 Video Data Acquisition Unit 43 Vehicle Information Acquisition Unit 44 Overhead View Video Generation Unit 441 Viewpoint Conversion Processing Unit 442 Cutout Processing Unit 443 Compositing Processing Unit 45 Reference Object Detection Unit 46 Orientation Identification Unit 48 Superimposition Processing Unit 49 Display Control Unit 50 Storage Unit 100 First Bird's-eye View Video 100A Superimposed Video 110 Own Vehicle Icon 120 Reference Direction Icon (Information indicating Direction)

Claims (9)

A video data acquisition unit that acquires peripheral video data captured by the imaging unit that captures the periphery of the vehicle;
A bird's-eye view video generation unit that performs a viewpoint conversion process on the peripheral video acquired by the video data acquisition unit and a synthesis process including a host vehicle icon indicating a vehicle, and generates a bird's-eye view video that displays a predetermined display range from the vehicle;
A reference object detection unit for detecting a reference object around the vehicle;
A direction specifying unit for specifying a relative direction between the reference target detected by the reference target detection unit and the vehicle;
Superimposition processing for generating a superimposed image in which information indicating the relative orientation of the reference object and the vehicle is superimposed on the overhead view image generated by the overhead view image generation unit as a relative orientation based on the own vehicle icon And
A display control unit that causes the display unit to display the superimposed video generated by the superimposition processing unit;
With
When the relative positional relationship between the reference object and the vehicle satisfies a predetermined condition, the overhead image generation unit replaces the overhead image with the own vehicle icon as a reference, and indicates the relative direction. To generate a bird's-eye view video based on
A bird's-eye view image generation apparatus characterized by the above. When the difference in relative orientation between the reference object specified by the orientation specifying unit and the vehicle is less than a threshold, the overhead view video generation unit is replaced with the overhead view video based on the own vehicle icon. Generate a bird's-eye view image based on information indicating the relative orientation,
The overhead image generation apparatus according to claim 1. When the relative orientation between the reference object specified by the orientation specifying unit and the vehicle coincides with the orientation indicated by the own vehicle icon, the overhead view video generation unit replaces the overhead view image based on the own vehicle icon. A bird's-eye view image based on the information indicating the relative orientation,
The overhead image generation apparatus according to claim 2. The overhead view video generation unit replaces the overhead view video with the own vehicle icon as a reference while the difference in relative direction between the reference target specified by the orientation specifying unit and the vehicle is less than a threshold value. Generate a bird's-eye view image based on information indicating the relative orientation,
The overhead image generation apparatus according to claim 2 or 3. The superimposing processing unit superimposes information indicating a relative direction between the reference object and the own vehicle by changing a display mode according to a relative direction between the reference object and the own vehicle.
The overhead view image generation device according to any one of claims 1 to 4. The reference object detection unit detects a reference object that exists farther from the display range around the vehicle.
The overhead image generation device according to any one of claims 1 to 5. An overhead view video generation device according to any one of claims 1 to 6,
An overhead video display device comprising at least one of a photographing unit that supplies peripheral video data to the video data acquisition unit and a display unit that causes the display control unit to display a superimposed video. A video data acquisition step of acquiring peripheral video data captured by a capturing unit that captures the periphery of the vehicle;
An overhead view video generation step of performing a viewpoint conversion process on the peripheral video acquired by the video data acquisition step and a synthesis process including a host vehicle icon indicating a vehicle, and generating an overhead video that displays a predetermined display range from the vehicle;
A reference object detection step for detecting a reference object around the vehicle;
A direction specifying step for specifying a relative direction between the reference object detected by the reference object detection step and the vehicle;
Superimposition processing for generating a superimposed video in which information indicating the relative orientation between the reference object and the vehicle is superimposed on the overhead video generated by the overhead video generation step as a relative orientation based on the own vehicle icon Steps,
A display control step of displaying the superimposed video generated by the superimposition processing step on a display unit;
Including
When the relative positional relationship between the reference object and the vehicle satisfies a predetermined condition, the overhead image generation step replaces the overhead image based on the own vehicle icon with information indicating the relative direction. To generate a bird's-eye view video based on
Overhead video generation method. A video data acquisition step of acquiring peripheral video data captured by a capturing unit that captures the periphery of the vehicle;
An overhead view video generation step of performing a viewpoint conversion process on the peripheral video acquired by the video data acquisition step and a synthesis process including a host vehicle icon indicating a vehicle, and generating an overhead video that displays a predetermined display range from the vehicle;
A reference object detection step for detecting a reference object around the vehicle;
A direction specifying step for specifying a relative direction between the reference object detected by the reference object detection step and the vehicle;
Superimposition processing for generating a superimposed video in which information indicating the relative orientation between the reference object and the vehicle is superimposed on the overhead video generated by the overhead video generation step as a relative orientation based on the own vehicle icon Steps,
A display control step of displaying the superimposed video generated by the superimposition processing step on a display unit;
Including
When the relative positional relationship between the reference object and the vehicle satisfies a predetermined condition, the overhead image generation step replaces the overhead image based on the own vehicle icon with information indicating the relative direction. To generate a bird's-eye view video based on
A program for causing a computer that operates as a bird's-eye view image generation device to execute this.

Images