JP2018050277A - Overhead video generation device, overhead video generation system, overhead video generation method and program - Google Patents

Abstract

An object of the present invention is to make it possible to appropriately check obstacles around a vehicle. An image acquisition unit for acquiring a peripheral image obtained by photographing the periphery of a vehicle, an obstacle information acquisition unit for acquiring obstacle information of an obstacle detected in the vicinity of the vehicle, and an image acquisition unit. Information indicating an obstacle based on the obstacle information acquired by the obstacle information acquisition unit 43 and the overhead image generation unit 45 that generates an overhead image obtained by performing the viewpoint conversion process so that the vehicle is looked down from above. And a display control unit 47 for displaying the overhead image generated by the superimposed image generation unit 46 on the display panel 31. Features. [Selection] Figure 1

Description

  The present invention relates to an overhead video generation device, an overhead video generation system, an overhead video generation method, and a program.

  A technique related to a vehicle periphery display device that displays an overhead view video of a vehicle together with a vehicle image is known (for example, see Patent Document 1).

Japanese Patent Laying-Open No. 2015-0776645

  In order to make it easier to confirm the periphery of the vehicle, there is a technique for displaying obstacle information of an obstacle detected in the vicinity of the vehicle so as to be superimposed on the overhead image. However, when the obstacle information is superimposed on the overhead view video, the visibility of the obstacle reflected in the overhead view video may be hindered.

  The present invention has been made in view of the above, and an object thereof is to enable appropriate confirmation of obstacles around a vehicle.

  In order to solve the above-described problems and achieve the object, an overhead image generation apparatus according to the present invention includes a video acquisition unit that acquires a peripheral video obtained by photographing the periphery of a vehicle, and obstacles detected in the vicinity of the vehicle. An obstacle information acquisition unit that acquires obstacle information, an overhead image generation unit that generates an overhead image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit, and the obstacle Based on the obstacle information acquired by the object information acquisition unit, a superimposed video generation unit that generates an overhead video by superimposing information indicating the obstacle on a central part surrounded by the overhead video, and the superimposed video generation unit includes: And a display control unit that displays the generated overhead image on the display unit.

  An overhead video generation system according to the present invention includes the above-described overhead video generation device, an imaging unit that captures the periphery of the vehicle and supplies a peripheral image to the image acquisition unit, detects an obstacle in the vicinity of the vehicle, and It has an obstacle detection part which supplies obstacle information to an obstacle information acquisition part, It is characterized by the above-mentioned.

  The bird's-eye view video generation method according to the present invention includes a video acquisition step of acquiring a peripheral video obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring obstacle information of an obstacle detected in the periphery of the vehicle, Based on the obstacle information acquired in the obstacle information acquisition step, an overhead image generation step of generating an overhead image that performs viewpoint conversion processing so as to look down on the vehicle from above from the surrounding image acquired in the image acquisition step, A superimposed video generation step of generating a bird's-eye view video in which information indicating the obstacle is superimposed on a central portion surrounded by the overhead view video; and a display control step of causing the display unit to display the overhead view video generated in the superimposed video generation step; including.

  The program according to the present invention includes a video acquisition step of acquiring a peripheral video obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring obstacle information of an obstacle detected in the vicinity of the vehicle, and the video acquisition step. Based on the obstacle information acquired in the obstacle information acquisition step, an overhead image generation step of generating an overhead image that has been subjected to viewpoint conversion processing so as to look down on the vehicle from above from the surrounding image acquired in And a display control step for displaying the overhead image generated in the superimposed image generation step on the display unit, and a display control step for displaying the overhead image generated in the superimposed image generation step on the display unit. A computer operating as a generation device is executed.

  According to the present invention, there is an effect that obstacles around the vehicle can be appropriately confirmed.

FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment. FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment. FIG. 3 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the first embodiment. FIG. 4 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment. FIG. 5 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the first embodiment. FIG. 6 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the second embodiment. FIG. 7 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the second embodiment. FIG. 8 is a flowchart showing the flow of processing in the overhead video generation device of the overhead video generation system according to the third embodiment. FIG. 9 is a diagram showing an example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 10 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 11 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 12 is a graph showing an example of the relationship between the distance between the arcs and the distance to the obstacle in the obstacle notification icon. FIG. 13 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the sixth embodiment. FIG. 14 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment. FIG. 15 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.

  DESCRIPTION OF EMBODIMENTS Embodiments of an overhead video generation device 40, an overhead video generation system 1, an overhead video generation method, and a program according to the present invention will be described below in detail with reference to the accompanying drawings. In addition, this invention is not limited by the following embodiment.

[First embodiment]
FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment. The bird's-eye view video generation system 1 generates a bird's-eye view video 100 (see FIG. 2). The overhead image generation device 40 and the overhead image generation system 1 are mounted on a vehicle. In addition to what is mounted on the vehicle, the overhead image generation device 40 and the overhead image generation system 1 may be a portable device that can be used in the vehicle.

  The overhead view video generation system 1 will be described with reference to FIG. The overhead view video generation system 1 includes a front camera (shooting unit) 11, a rear camera (shooting unit) 12, a left side camera (shooting unit) 13, a right side camera (shooting unit) 14, and a front left sensor (failure). (Object detection unit) 21, front center sensor (obstacle detection unit) 22, front right sensor (obstacle detection unit) 23, rear left sensor (obstacle detection unit) 24, and rear center sensor (obstacle detection) Unit) 25, a rear right sensor (obstacle detection unit) 26, a display panel 31, and an overhead image generation device 40.

  The front camera 11 is disposed in front of the vehicle and captures a periphery around the front of the vehicle. The front camera 11 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.

  The rear camera 12 is disposed behind the vehicle and photographs a periphery around the rear of the vehicle. The rear camera 12 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.

  The left-side camera 13 is disposed on the left side of the vehicle and captures the periphery around the left side of the vehicle. The left side camera 13 outputs the captured video to the video acquisition unit 42 of the overhead view video generation device 40.

  The right-side camera 14 is disposed on the right side of the vehicle and captures the periphery around the right side of the vehicle. The right side camera 14 outputs the captured video to the video acquisition unit 42 of the overhead view video generation device 40.

  The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 photograph all directions of the vehicle.

  The front left sensor 21 is disposed on the front left side of the vehicle and detects an obstacle on the front left side of the vehicle. The front left sensor 21 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The front left sensor 21 detects an object having a height from the ground that may come into contact with the vehicle. The front left sensor 21 detects an obstacle at a distance of about 5 m from the vehicle, for example. The front left sensor 21 detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the front left sensor 21 may overlap with a part of the detection range of the front center sensor 22. The front left sensor 21 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front left sensor 21, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

  The front center sensor 22 is disposed at the front center of the vehicle and detects an obstacle at the front center of the vehicle. The front center sensor 22 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The front center sensor 22 detects an object having a height from the ground that may come into contact with the vehicle. The front center sensor 22 detects an obstacle at a distance of about 5 m from the vehicle, for example. The front center sensor 22 detects an obstacle in a range of, for example, about 40 ° with the center of the sensor as the center when viewed in the vertical direction. The detection range of the front center sensor 22 may overlap with a part of the detection range of the front left sensor 21 and the front right sensor 23. The front center sensor 22 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front center sensor 22, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

  The front right sensor 23 is disposed on the front right side of the vehicle and detects an obstacle on the front right side of the vehicle. The front right sensor 23 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The front right sensor 23 detects an object having a height from the ground that may come into contact with the vehicle. The front right sensor 23 detects an obstacle at a distance of about 5 m from the vehicle, for example. The front right sensor 23 detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the front right sensor 23 may overlap with a part of the detection range of the front center sensor 22. The front right sensor 23 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front right sensor 23, the distance to the obstacle, and the existence range of the obstacle in the horizontal direction.

  The front left sensor 21, the front center sensor 22, and the front right sensor 23 detect an obstacle ahead of the vehicle.

  The rear left sensor 24 is disposed on the rear left side of the vehicle and detects an obstacle on the rear left side of the vehicle. The rear left sensor 24 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The rear left sensor 24 detects an object having a height from the ground that may come into contact with the vehicle. The rear left sensor 24 detects an obstacle at a distance of about 5 m from the vehicle, for example. The rear left sensor 24 detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the rear left sensor 24 may overlap with a part of the detection range of the rear center sensor 25. The rear left sensor 24 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead image generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear left sensor 24, the distance to the obstacle, and the existence range of the obstacle in the horizontal direction.

  The rear center sensor 25 is disposed at the rear center of the vehicle and detects an obstacle at the rear center of the vehicle. The rear center sensor 25 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The rear center sensor 25 detects an object having a height from the ground that may come into contact with the vehicle. The rear center sensor 25 detects an obstacle at a distance of about 5 m from the vehicle, for example. The rear center sensor 25 detects an obstacle in a range of, for example, about 40 ° with the center of the sensor as the center when viewed in the vertical direction. The detection range of the rear center sensor 25 may overlap a part of the detection range of the rear left sensor 24 and the rear right sensor 26. The rear center sensor 25 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead image generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear center sensor 25, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

  The rear right sensor 26 is disposed on the rear right side of the vehicle and detects an obstacle on the rear right side of the vehicle. The rear right sensor 26 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The rear right sensor 26 detects an object having a height from the ground that may come into contact with the vehicle. The rear right sensor 26 detects an obstacle at a distance of about 5 m from the vehicle, for example. The rear right sensor 26 detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the rear right sensor 26 may overlap with a part of the detection range of the rear center sensor 25. The rear right sensor 26 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear right sensor 26, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

  The rear left sensor 24, the rear center sensor 25, and the rear right sensor 26 detect an obstacle behind the vehicle.

  The display panel 31 is, for example, a display including a liquid crystal display (LCD) or an organic EL-Organic Electro-Luminescence (EL) display. The display panel 31 displays the overhead video 100 based on the video signal output from the overhead video generation device 40 of the overhead video generation system 1. The display panel 31 may be dedicated to the overhead view video generation system 1 or may be used jointly with other systems including a navigation system, for example. The display panel 31 is disposed at a position that is easily visible to the driver.

  The overhead video generation device 40 includes a control unit 41 and a storage unit 49.

  The control unit 41 is an arithmetic processing unit configured with, for example, a CPU (Central Processing Unit). The control unit 41 loads the program stored in the storage unit 49 into the memory and executes instructions included in the program. The control unit 41 includes a video acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, an overhead video generation unit 45, a superimposed video generation unit 46, and a display control unit 47.

  The video acquisition unit 42 acquires a peripheral video that captures the periphery of the vehicle. More specifically, the video acquisition unit 42 acquires video output from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The video acquisition unit 42 outputs the acquired video to the overhead view video generation unit 45.

  The obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in the vicinity of the vehicle. In the present embodiment, the obstacle information acquisition unit 43 acquires obstacle information including the distance to the detected obstacle. More specifically, the obstacle information acquisition unit 43 acquires the obstacle information output by the front left sensor 21, the front center sensor 22, the front right sensor 23, the rear left sensor 24, the rear center sensor 25, and the rear right sensor 26. To do. The obstacle information acquisition unit 43 outputs the acquired obstacle information to the superimposed video generation unit 46.

  The vehicle information acquisition unit 44 acquires vehicle information serving as a trigger for displaying a bird's-eye view video such as vehicle gear operation information from a CAN (Controller Area Network) or various sensors that sense the state of the vehicle. The vehicle information acquisition unit 44 outputs the acquired vehicle information to the overhead view video generation unit 45.

  The bird's-eye view video generation unit 45 generates the bird's-eye view video 100 that has been subjected to the viewpoint conversion process so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. More specifically, the bird's-eye view image generation unit 45 generates the bird's-eye view image 100 based on images taken by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the bird's-eye view image 100 may be any known method and is not limited. The overhead view video generation unit 45 outputs the generated overhead view video 100 to the display control unit 47.

  The overhead view image 100 will be described with reference to FIG. FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment. The bird's-eye view image 100 has a vertically long rectangular shape. The bird's-eye view image 100 is located at the center surrounded by the front image 101, the rear image 102, the left side image 103, the right side image 104, the front image 101, the rear image 102, the left side image 103, and the right side image 104. And a central image 105 to be displayed. The front video 101, the rear video 102, the left side video 103, the right side video 104, and the central video 105 may be separated by a frame-like boundary line.

  The center video 105 is generated in a vertically long rectangular shape. In the central video 105, the boundaries of the front video 101, the rear video 102, the left video 103, and the right video 104 are separated by lines. The center image 105 shows the position of the vehicle in the overhead view image 100.

  In FIG. 2, diagonal broken lines indicating boundaries between the front video 101, the rear video 102, the left side video 103, and the right side video 104 are illustrated for explanation, but are actually displayed on the display panel 31. The broken line is not displayed in the overhead view video 100. The same applies to the other figures.

  Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed video generation unit 46 generates an overhead video 100 in which information indicating an obstacle is superimposed on the central video 105 of the overhead video 100. In the present embodiment, the superimposed video generation unit 46 superimposes information indicating the direction in which the obstacle is detected on the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. 100 is generated. In the present embodiment, the superimposed video generation unit 46 indicates information indicating the direction in which the obstacle is detected as an obstacle notification icon (information indicating the obstacle) 110. The obstacle notification icon 110 schematically shows the detection direction in the horizontal direction of each sensor that detects an obstacle, and the arrangement direction of the plurality of arcs is a detection direction starting from the sensor mounting position, or a vehicle. For example, a radial arrangement with the center. The width of the arc constituting the obstacle notification icon 110 may indicate the detection range of each sensor that detects the obstacle, or may be a fixed width corresponding to the detection direction regardless of the detection range of each sensor. .

  The obstacle notification icon 110 is an icon for notifying an obstacle. The obstacle notification icon 110 indicates the distance and direction to the obstacle. The obstacle notification icon 110 includes a front left icon (information indicating an obstacle) 111, a front center icon (information indicating an obstacle) 112, a front right icon (information indicating an obstacle) 113, and a rear left icon (an obstacle is displayed). Information) 114, a rear center icon (information indicating an obstacle) 115, and a rear right icon (information indicating an obstacle) 116.

  The front left icon 111 is an icon for notifying an obstacle on the left front side of the vehicle. More specifically, the front left icon 111 is an icon that notifies that an obstacle has been detected by the front left sensor 21. The front left icon 111 is superimposed on the upper left of the central image 105 of the overhead view image 100 in FIG.

  In the present embodiment, the front left icon 111 is composed of a triple arc-shaped curve. The arc-shaped curve bulges toward the outside of the overhead view image 100. The triple arc-shaped curve has a radius that decreases from the outside of the bird's-eye view image 100 toward the center. The triple arc-shaped curve is shortened from the outside of the overhead view image 100 toward the center side.

  The front left icon 111 may change its color according to the distance to the obstacle. For example, the front left icon 111 is displayed in green when the distance to the obstacle is greater than or equal to the first predetermined distance. The front left icon 111 is displayed in yellow when the distance to the obstacle is greater than or equal to a second predetermined distance that is less than the first predetermined distance and less than the first predetermined distance. The front left icon 111 is displayed in red when the distance to the obstacle is less than the second predetermined distance.

  The front left icon 111 may change the number of arcuate curves according to the distance to the obstacle. For example, the front left icon 111 displays the outermost arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is equal to or greater than the first predetermined distance. The front left icon 111 displays an outermost arc-shaped curve and a center arc-shaped curve among triple arc-shaped curves when the distance to the obstacle is less than the first predetermined distance and greater than or equal to the second predetermined distance. To do. The front left icon 111 displays all triple arc-shaped curves when the distance to the obstacle is less than the second predetermined distance.

  The front center icon 112 is an icon for notifying an obstacle at the front center of the vehicle. More specifically, the front center icon 112 is an icon that notifies that an obstacle has been detected by the front center sensor 22. The front center icon 112 is superimposed on the upper center of the center image 105 of the overhead image 100 in FIG. The front center icon 112 is configured in the same manner as the front left icon 111.

  The front right icon 113 is an icon that notifies an obstacle on the right front side of the vehicle. More specifically, the front right icon 113 is an icon notifying that an obstacle has been detected by the front right sensor 23. The front right icon 113 is superimposed on the upper right of the central video 105 of the overhead view video 100 in FIG. The front right icon 113 is configured in the same manner as the front left icon 111.

  The rear left icon 114 is an icon for notifying an obstacle on the left rear side of the vehicle. More specifically, the rear left icon 114 is an icon for notifying that an obstacle has been detected by the rear left sensor 24. The rear left icon 114 is superimposed on the lower left of the central video 105 of the overhead video 100 in FIG. The rear left icon 114 is configured in the same manner as the front left icon 111.

  The rear center icon 115 is an icon that notifies an obstacle at the rear center of the vehicle. More specifically, the rear center icon 115 is an icon that notifies that an obstacle has been detected by the rear center sensor 25. The rear center icon 115 is superimposed on the lower center of the center image 105 of the overhead image 100 in FIG. The rear center icon 115 is configured in the same manner as the front left icon 111.

  The rear right icon 116 is an icon for notifying an obstacle on the right rear side of the vehicle. More specifically, the rear right icon 116 is an icon for notifying that the rear right sensor 26 has detected an obstacle. The rear right icon 116 is superimposed on the lower right of the center image 105 of the overhead view image 100 in FIG. The rear right icon 116 is configured in the same manner as the front left icon 111.

  The display control unit 47 causes the display panel 31 to display the overhead view video 100 generated by the superimposed video generation unit 46.

  The storage unit 49 stores data and various processing results required for various processes in the overhead view video generation device 40. The storage unit 49 is, for example, a semiconductor memory device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described with reference to FIG. FIG. 3 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the first embodiment.

  The control unit 41 determines whether or not to start the overhead view video display (step S11). As an example of the determination to start the overhead view video display, the control unit 41 determines whether or not to disclose the overhead view video display based on the presence or absence of a reverse trigger. The reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle is rearward in the front-rear direction of the vehicle. When there is no reverse trigger, the control unit 41 determines not to start the overhead view video display (No in step S11), and executes the process of step S11 again. When there is a backward trigger, the control unit 41 determines to start the overhead view video display (Yes in Step S11), and proceeds to Step S12.

  The control unit 41 generates and displays the overhead view video 100 (step S12). More specifically, the control unit 41 causes the overhead video generation unit 45 to generate the overhead video 100 that has been subjected to the viewpoint conversion processing so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. Then, the control unit 41 causes the display control unit 47 to display the generated overhead image 100 on the display panel 31.

  The control unit 41 determines whether an obstacle has been detected (step S13). More specifically, the control unit 41 determines whether or not obstacle information has been acquired by the obstacle information acquisition unit 43. When it is determined that the obstacle information acquisition unit 43 has acquired the obstacle information (Yes in step S13), the control unit 41 proceeds to step S14. When it is determined that the obstacle information acquisition unit 43 has not acquired the obstacle information (No in step S13), the control unit 41 proceeds to step S15.

  The control unit 41 superimposes and displays the obstacle notification icon 110 indicating an obstacle at the center of the overhead view video 100 (step S14). More specifically, the control unit 41 causes the superimposed image generation unit 46 to indicate the direction in which the obstacle is detected in the central image 105 of the overhead view image 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. A bird's-eye view video 100 in which the obstacle notification icon 110 is superimposed is generated. Then, the control unit 41 causes the display control unit 47 to display the generated overhead image 100 on the display panel 31.

  When the control unit 41 acquires obstacle information from a plurality of sensors including the front left sensor 21, the front center sensor 22, the front right sensor 23, the rear left sensor 24, the rear center sensor 25, and the rear right sensor 26, the superimposed image The generation unit 46 generates the overhead video 100 in which a plurality of obstacle notification icons 110 are superimposed on the central video 105 of the overhead video 100.

  The control unit 41 determines whether or not to end the overhead view video display (step S15). More specifically, the control unit 41 determines whether or not to end the bird's-eye view video display based on the presence or absence of the reverse trigger. When there is no backward trigger, in other words, when the backward trigger is released, the control unit 41 determines to end the overhead view video display (Yes in step S15), and ends the process. When there is a reverse trigger, the control unit 41 determines not to end the overhead view video display (No in step S15), returns to step S13, and continues the process.

  In this way, when an obstacle is detected, the overhead image generation system 1 superimposes the obstacle notification icon 110 on the center of the overhead image 100 so as to indicate the direction in which the obstacle is detected, and displays the display panel. 31 outputs a video signal. The display panel 31 displays the bird's-eye view video 100 together with the navigation screen, for example, based on the video signal output from the bird's-eye view video generation system 1.

  For example, an example of detecting an obstacle on the left rear side when the vehicle is moving backward will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment. In FIG. 4, the rear image 102 of the overhead view image 100 includes an obstacle image 200 of the obstacle on the left rear side.

  In step S13, the controller 41 determines that an obstacle has been detected. In step S14, the control unit 41 detects the obstacle in the central video 105 of the overhead view video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43 in the superimposed video generation unit 46, and then left A bird's-eye view image 100 on which the obstacle notification icon 110 is superimposed so as to indicate the direction is generated. Then, the control unit 41 causes the display control unit 47 to display the generated overhead image 100 on the display panel 31.

  Another example in the case where an obstacle is detected on the rear left side when the vehicle is moving backward will be described with reference to FIG. FIG. 5 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the first embodiment.

  In FIG. 5, a vehicle icon 120, which is a vehicle icon indicating a vehicle when the vehicle is viewed from above, is combined with the central image 105 of the overhead view image 100. In step S <b> 14, the control unit 41 causes the superimposed video generation unit 46 to generate an overhead video 100 in which the obstacle notification icon 110 is superimposed on the own vehicle icon 120 so as to indicate the left direction after the obstacle is detected.

  As described above, according to the present embodiment, when an obstacle is detected in the vicinity of the vehicle, the obstacle notification icon 110 is superimposed on the central image 105 of the overhead image 100 so as to indicate the direction in which the obstacle is detected. The video 100 is displayed on the display panel 31. In the present embodiment, since the obstacle notification icon 110 is superimposed on the central image 105 of the overhead image 100, the obstacle notification icon 110 and the obstacle reflected in the overhead image 100 are not superimposed. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.

  In the present embodiment, the obstacle detection icon 110 can notify the direction and distance in which an obstacle is detected.

  In the present embodiment, as shown in FIG. 4, the visibility of the obstacle notification icon 110 can be further improved when the vehicle icon 120 is not displayed in the central image 105 of the overhead view image 100. Thus, this embodiment can make it possible to more appropriately check obstacles around the vehicle.

  In the present embodiment, as shown in FIG. 5, the vehicle icon 120 may be displayed on the center image 105 of the overhead view image 100. Thereby, this embodiment can display the bird's-eye view image 100 which can recognize the direction of the obstacle with respect to a vehicle more easily. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.

[Second Embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. FIG. 6 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the second embodiment. FIG. 7 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the second embodiment. The overhead image generation system 1 has the same basic configuration as the overhead image generation system 1 of the first embodiment. In the following description, components similar to those in the overhead view video generation system 1 are denoted by the same reference numerals or corresponding reference numerals, and detailed description thereof is omitted. The overhead image generation system 1 of the present embodiment is different from the overhead image generation system 1 of the first embodiment in the processing in the obstacle information acquisition unit 43, the superimposed image generation unit 46, and the control unit 41 of the overhead image generation device 40. .

  The obstacle notification icon 110 includes a second front left icon 111, a front center icon 112, a front right icon 113, a rear left icon 114, a rear center icon 115, and a rear right icon 116 in the first embodiment. Icon (information indicating obstacle), second front center icon (information indicating obstacle), second front right icon (information indicating obstacle), second rear left icon (information indicating obstacle) and second A rear center icon (information indicating an obstacle) and a second rear right icon (information indicating an obstacle) are included. In FIG. 7, the second front left icon, the second front center icon, the second front right icon, the second rear center icon, and the second rear right icon are not shown. Information).

  The front left icon 111 is an icon for notifying an obstacle on the left front side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front left icon 111 is an icon for informing that the front left sensor 21 has detected an obstacle whose distance to the obstacle is less than a threshold.

  The front center icon 112 is an icon that notifies an obstacle at the front center of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front center icon 112 is an icon for notifying that the front center sensor 22 has detected an obstacle whose distance to the obstacle is less than a threshold.

  The front right icon 113 is an icon for notifying an obstacle on the right front side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front right icon 113 is an icon for notifying that the front right sensor 23 has detected an obstacle whose distance to the obstacle is less than a threshold.

  The rear left icon 114 is an icon for notifying an obstacle on the left rear side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the rear left icon 114 is an icon for notifying that the rear left sensor 24 has detected an obstacle whose distance to the obstacle is less than a threshold.

  The rear center icon 115 is an icon for notifying an obstacle at the rear center of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the rear center icon 115 is an icon for notifying that the rear center sensor 25 has detected an obstacle whose distance to the obstacle is less than a threshold value.

  The rear right icon 116 is an icon for notifying an obstacle on the right rear side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the rear right icon 116 is an icon for notifying that the rear right sensor 26 has detected an obstacle whose distance to the obstacle is less than a threshold.

  A 2nd front left icon is an icon which alert | reports the obstruction of the vehicle front left side where the distance to an obstruction is more than a threshold value. More specifically, the second front left icon is an icon for notifying that the front left sensor 21 has detected an obstacle whose distance to the obstacle is not less than a threshold value. The second front left icon is superimposed on the upper left outside the central image 105 of the overhead image 100. The second front left icon is configured in the same manner as the front left icon 111.

  A 2nd front center icon is an icon which alert | reports the obstruction of the front center of a vehicle whose distance to an obstruction is more than a threshold value. More specifically, the second front center icon is an icon for notifying that the front center sensor 22 has detected an obstacle whose distance to the obstacle is a threshold value or more. The second front center icon is superimposed on the upper center outside the center image 105 of the overhead view image 100. The second front center icon is configured in the same manner as the front left icon 111.

  The second front right icon is an icon that notifies an obstacle on the right front side of the vehicle whose distance to the obstacle is equal to or greater than a threshold value. More specifically, the second front right icon is an icon for notifying that the front right sensor 23 has detected an obstacle whose distance to the obstacle is not less than a threshold value. The second front right icon is superimposed on the upper right outside the center image 105 of the overhead view image 100. The second front right icon is configured in the same manner as the front left icon 111.

  A 2nd back left icon is an icon which alert | reports the obstruction of the back left side of a vehicle whose distance to an obstruction is more than a threshold value. More specifically, the second rear left icon is an icon for informing that the rear left sensor 24 has detected an obstacle whose distance to the obstacle is a threshold value or more. The second rear left icon is superimposed on the lower left outside the center image 105 of the overhead view image 100. The second rear left icon is configured in the same manner as the front left icon 111.

  A 2nd back center icon is an icon which alert | reports the obstruction of the back center of a vehicle whose distance to an obstruction is more than a threshold value. More specifically, the second rear center icon is an icon for notifying that the rear center sensor 25 has detected an obstacle whose distance to the obstacle is a threshold value or more. The second rear center icon is superimposed on the lower center outside the center image 105 of the overhead image 100. The second rear center icon is configured in the same manner as the front left icon 111.

  A 2nd back right icon is an icon which alert | reports the obstacle on the back right side of a vehicle whose distance to an obstacle is more than a threshold value. More specifically, the second rear right icon is an icon for informing that the rear right sensor 26 has detected an obstacle whose distance to the obstacle is a threshold value or more. The second rear right icon is superimposed on the lower right outside the center image 105 of the overhead view image 100. The second rear right icon is configured in the same manner as the front left icon 111.

  The obstacle information acquisition unit 43 acquires information including the distance to the detected obstacle. The obstacle information acquisition unit 43 outputs information including the acquired distance to the obstacle to the superimposed video generation unit 46.

  If the distance to the detected obstacle is greater than or equal to a predetermined threshold based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 46 displays the vehicle icon 120 in the central image 105 of the overhead image 100. Is displayed, and an overhead video 100 in which an obstacle notification icon 110, which is information indicating an obstacle, is superimposed on the outside of the central portion of the overhead video 100 is generated.

  If the distance to the detected obstacle is less than a predetermined threshold based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 46 displays the obstacle notification icon 110 in the center of the overhead image 100. The overhead view video 100 superimposed on the video 105 is generated.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described with reference to FIG. The processing in step S21, step S23, step S26, and step S27 in the flowchart shown in FIG. 6 is the same as the processing in step S11, step S13, step S14, and step S15 in the flowchart shown in FIG.

  The control part 41 produces | generates and displays the bird's-eye view image 100 on which the own vehicle icon 120 was superimposed (step S22). More specifically, the control unit 41 causes the overhead video generation unit 45 to generate the overhead video 100 that has been subjected to the viewpoint conversion processing so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. The control unit 41 synthesizes the vehicle icon 120 in the center of the generated bird's-eye view image 100 with the display control unit 47. Then, the control unit 41 causes the display control unit 47 to display the generated overhead image 100 on the display panel 31.

  The control unit 41 determines whether or not the distance to the obstacle is greater than or equal to a threshold value (step S24). More specifically, when the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold (Yes in step S24), the control unit 41 proceeds to step S25. When the distance to the detected obstacle acquired by the obstacle information acquiring unit 43 is less than the predetermined threshold (No in step S24), the control unit 41 proceeds to step S26.

  The predetermined threshold indicates that the position at which the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon are displayed is an overhead image 100. It is set to a value that does not overlap with the obstacle video. More specifically, the predetermined threshold is the outermost arc shape of the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon. Any value that is equal to or greater than the distance from the vehicle corresponding to the position of the curve may be used. For example, the predetermined threshold may be about 2 m.

  The control unit 41 superimposes and displays the obstacle notification icon 110 on the bird's-eye view image 100 outside the host vehicle icon 120 (step S25). More specifically, the control unit 41 uses the superimposed image generation unit 46 to display a second front left icon and an overhead image 100 outside the host vehicle icon 120 based on the obstacle information acquired by the obstacle information acquisition unit 43. An overhead view image 100 in which at least one of a second front center icon, a second front right icon, a second rear left icon, a second rear center icon, and a second rear right icon is superimposed is generated. Then, the control unit 41 causes the display control unit 47 to display the generated overhead image 100 on the display panel 31.

  In this manner, when the distance to the obstacle is equal to or greater than the predetermined threshold, the overhead view video generation system 1 displays the second forward left icon, the second forward center icon, and the first forward icon 100 on the overhead view video 100 outside the host vehicle icon 120. A bird's-eye view image 100 in which at least one of a second front right icon, a second rear left icon, a second rear center icon, and a second rear right icon is superimposed is generated, and a video signal is output to the display panel 31. When the distance to the obstacle is less than the predetermined threshold, the overhead view video generation system 1 includes a front left icon 111, a front center icon 112, a front right icon 113, a rear left icon 114, and a rear side. An overhead view video 100 in which at least one of the center icon 115 and the rear right icon 116 is superimposed is generated, and a video signal is output to the display panel 31.

  For example, an example in the case where an obstacle is detected at a distance above a predetermined threshold on the left rear side when the vehicle is moving backward will be described with reference to FIG.

  In step S23, the control unit 41 determines that an obstacle has been detected. In step S24, the control unit 41 determines that the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold (Yes in step S24). Then, in step S25, the control unit 41 superimposes the second rear left icon 117 indicating the left direction after detecting the obstacle on the overhead view video 100 outside the host vehicle icon 120, as shown in FIG. Is generated. Then, the control unit 41 displays the generated overhead image 100 on the display panel 31.

  As described above, in the present embodiment, when the distance to the obstacle is equal to or greater than the predetermined threshold, the overhead image 100 in which the obstacle notification icon 110 is superimposed on the overhead image 100 outside the host vehicle icon 120 is generated and displayed. Display on the panel 31. In the present embodiment, when the distance to the obstacle is equal to or greater than a predetermined threshold, the obstacle notification icon 110 is superimposed on the overhead image 100 outside the host vehicle icon 120, so that the obstacle notification icon 110 and the overhead image 100 are reflected. Obstacles do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.

  In the present embodiment, when the distance to the obstacle is less than a predetermined threshold, the overhead image 100 in which the obstacle notification icon 110 is superimposed on the central image 105 of the overhead image 100 is generated and displayed on the display panel 31. In the present embodiment, when the distance to the obstacle is less than the predetermined threshold, the obstacle notification icon 110 is superimposed on the central image 105 of the overhead view video 100, so that the obstacle reflected in the obstacle notification icon 110 and the overhead view image 100 is displayed. And do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.

  Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle according to the distance to the obstacle.

  In the present embodiment, the display position of the obstacle notification icon 110 in the overhead view video 100 changes according to the distance to the obstacle. For this reason, this embodiment can make it easier to grasp the distance to the obstacle.

[Third embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing the flow of processing in the overhead video generation device of the overhead video generation system according to the third embodiment. The overhead image generation system 1 of the present embodiment is different from the overhead image generation system 1 of the second embodiment in the processing of the obstacle information acquisition unit 43, the superimposed image generation unit 46, and the control unit 41 of the overhead image generation device 40. .

  The front left sensor 21, the front center sensor 22, the front right sensor 23, the rear left sensor 24, the rear center sensor 25, and the rear right sensor 26 have a range farther than the range displayed as the overhead image 100 as a detection range.

  The obstacle information acquisition unit 43 acquires obstacle information of an obstacle detected in a range far from the range displayed as the overhead view video 100.

  Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed video generation unit 46 is located at the center of the overhead video 100 if the distance to the detected obstacle is far from the range displayed as the overhead video 100. The vehicle icon 120 is displayed, and the overhead view image 100 in which the obstacle notification icon 110 is superimposed on the overhead view image 100 outside the vehicle icon 120 is generated.

  The superimposed video generation unit 46 is based on the obstacle information acquired by the obstacle information acquisition unit 43, and is an obstacle notification icon if the distance to the detected obstacle is included in the range displayed as the overhead image 100. The overhead image 100 is generated by superimposing 110 on the central image 105 of the overhead image 100 so as to indicate the direction in which the obstacle is detected.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described with reference to FIG. The processes in steps S31 to S33 and steps S35 to S37 in the flowchart shown in FIG. 8 are the same as the processes in steps S21 to S23 and steps S25 to S27 in the flowchart shown in FIG.

  The control unit 41 determines whether or not the distance to the obstacle is greater than or equal to the display range of the overhead view image 100 (step S34). When the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is a range far from the display range of the overhead view image 100 (Yes in step S34), the control unit 41 proceeds to step S35. When the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is included in the display range of the overhead image 100 (No in step S34), the control unit 41 proceeds to step S36.

  In this way, the overhead view video generation system 1 displays the obstacle notification icon 110 on the overhead view video 100 outside the vehicle icon 120 when the distance to the obstacle is a range farther than the display range of the overhead view video 100. The superimposed bird's-eye view image 100 is generated and a video signal is output to the display panel 31. When the distance to the obstacle is included in the display range of the overhead image 100, the overhead image generation system 1 generates the overhead image 100 in which the obstacle notification icon 110 is superimposed on the central image 105 of the overhead image 100, and the display panel 31 outputs a video signal.

  As described above, in the present embodiment, when the distance to the obstacle is a range farther than the display range of the bird's-eye view image 100, the obstacle notification icon 110 is superimposed on the bird's-eye view image 100 outside the host vehicle icon 120. An overhead image 100 is generated and displayed on the display panel 31. In the present embodiment, when the distance to the obstacle is a range farther than the display range of the overhead view video 100, the obstacle notification icon 110 is superimposed on the outside of the vehicle icon 120, so the obstacle notification icon 110 and the overhead view video are displayed. The obstacle reflected in 100 does not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.

  In the present embodiment, when the distance to the obstacle is included in the display range of the overhead image 100, the overhead image 100 in which the obstacle notification icon 110 is superimposed on the central image 105 of the overhead image 100 is generated and displayed on the display panel 31. Let In the present embodiment, when the distance to the obstacle is included in the display range of the overhead view video 100, the obstacle notification icon 110 is superimposed on the center video 105 of the overhead view video 100. The obstacles reflected in the do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.

  Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle according to the distance to the obstacle.

  In the present embodiment, the obstacle notification icon 110 can be displayed even when there is an obstacle in a range far from the display range of the overhead view image 100. For this reason, this embodiment can notify the obstacle of the range far from the display range of the bird's-eye view image 100.

[Fourth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described. The overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in processing in the superimposed video generation unit 46 and the control unit 41 of the overhead view video generation device 40. More specifically, it differs from the bird's-eye view image generation system 1 of the first embodiment in that an obstacle located in the traveling direction of the vehicle is detected in determining whether an obstacle is detected in the control unit 41.

  Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 46 superimposes the obstacle notification icon 110 on the central image 105 of the overhead view video 100 for the obstacle located in the traveling direction of the vehicle. The overhead view video 100 is generated.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described.

  In step S <b> 13, the control unit 41 determines whether an obstacle is detected in the traveling direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43. When it is determined that the obstacle is detected in the traveling direction of the vehicle (Yes in Step S13), the control unit 41 proceeds to Step S14. When it is determined that no obstacle is detected in the traveling direction of the vehicle (No in step S13), the control unit 41 proceeds to step S15.

  In this way, the control unit 41 displays the obstacle notification icon 110 for the obstacle approaching the vehicle, and hides the obstacle notification icon 110 for the obstacle moving away from the vehicle.

  As described above, this embodiment can make it possible to more appropriately check obstacles around the vehicle.

[Fifth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. 9 to 12. FIG. 9 is a diagram showing an example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 10 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 11 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 12 is a graph showing an example of the relationship between the distance between the arcs and the distance to the obstacle in the obstacle notification icon. The overhead video generation system 1 of the present embodiment differs from the overhead video generation system 1 of the first embodiment in the processing in the superimposed video generation unit 46 of the overhead video generation device 40.

  As shown in FIGS. 9 to 11, the obstacle notification icon 110 has a plurality of components facing in the direction in which the obstacle is detected, and indicates the distance to the obstacle by the interval between the plurality of components. In the present embodiment, the obstacle notification icon 110 includes a triple arc as a constituent element. In the present embodiment, the obstacle notification icon 110 indicates the distance to the obstacle at intervals of arcs. In the present embodiment, the obstacle notification icon 110 indicates a wider arc interval as the distance to the obstacle is larger, and a smaller arc interval as the distance to the obstacle is smaller.

  The interval between the arcs of the obstacle notification icon 110 may be set linearly so as to increase as the distance to the obstacle increases, as indicated by a solid line in FIG.

  The interval between the arcs of the obstacle notification icon 110 may be set in a staircase shape so as to increase as the distance to the obstacle increases as indicated by a broken line in FIG. More specifically, when the distance to the obstacle is not less than the first threshold, for example, not less than 2 m, the arc interval is set as the first interval d1. The first threshold may be a distance of a boundary included in a range to be displayed as the overhead view video 100. When the distance to the obstacle is equal to or greater than the first threshold, the outermost arc-shaped curve may be displayed outside the central image 105 of the overhead image 100. When the distance to the obstacle is less than the first threshold and greater than or equal to the second threshold, for example, less than 2 m and greater than or equal to 1 m, the arc interval is set to a second interval d2 that is smaller than the first interval d1. When the distance to the obstacle is less than the second threshold, for example, less than 1 m, the arc interval is set to a third interval d3 that is smaller than the second interval d2. Further, when the distance to the obstacle becomes smaller, the arcs may be overlapped with the interval between the arcs set to zero.

  The distance to the obstacle is a distance from each sensor that detects the obstacle to the obstacle. The distance to the obstacle substantially coincides with the distance from the end of the vehicle in the obstacle direction to the obstacle.

  The superimposed video generation unit 46 displays the obstacle notification icon 110 in which the arc interval is changed based on the detected distance to the obstacle acquired by the obstacle information acquisition unit 43 in the central video 105 of the overhead image 100. The superimposed bird's-eye view image 100 is generated.

  An example in which an obstacle is detected on the rear left side when the vehicle is moving backward will be described with reference to FIGS.

  For example, an example in which an obstacle is detected on the left rear side when the vehicle is reversing and the distance to the obstacle is 2 m will be described with reference to FIG. In step S14, the control unit 41 causes the superimposed image generation unit 46 to indicate the left direction in the center image 105 of the overhead view image 100 after detecting the obstacle in the arc bulge direction, and to the obstacle at intervals of the arc. The overhead view image 100 on which the obstacle notification icon 110 indicating the distance is superimposed is generated. In the obstacle notification icon 110, the arc interval is the first interval d1. In the obstacle notification icon 110, the outermost arc-shaped curve is located outside the central image 105. The control unit 41 causes the superimposed video generation unit 46 to display the generated overhead video 100 illustrated in FIG. 9 on the display panel 31. Since the arc interval of the obstacle notification icon 110 and the outermost arc-shaped curve are located outside the central image 105, the obstacle is located near the boundary of the range to be displayed as the overhead image 100. I understand that.

  For example, an example in which an obstacle is detected on the rear left side when the vehicle is reversing and the distance to the obstacle is 1.5 m will be described with reference to FIG. In step S14, the control unit 41 causes the superimposed image generation unit 46 to indicate the left direction in the center image 105 of the overhead view image 100 after detecting the obstacle in the arc bulge direction, and to the obstacle at intervals of the arc. The overhead view image 100 on which the obstacle notification icon 110 indicating the distance is superimposed is generated. In the obstacle notification icon 110, the arc interval is the second interval d2. The control unit 41 causes the superimposed video generation unit 46 to display the generated overhead video 100 illustrated in FIG. 10 on the display panel 31. It can be seen from the arc interval of the obstacle notification icon 110 that the obstacle is located within the range displayed as the bird's-eye view video 100.

  For example, an example in which an obstacle is detected on the left rear side when the vehicle is reversing and the distance to the obstacle is 1 m will be described with reference to FIG. In step S14, the control unit 41 causes the superimposed image generation unit 46 to indicate the left direction in the center image 105 of the overhead view image 100 after detecting the obstacle in the arc bulge direction, and to the obstacle at intervals of the arc. The overhead view image 100 on which the obstacle notification icon 110 indicating the distance is superimposed is generated. In the obstacle notification icon 110, the arc interval is the third interval d3. The control unit 41 causes the superimposed video generation unit 46 to display the generated overhead video 100 illustrated in FIG. 11 on the display panel 31. Since the arc interval of the obstacle notification icon 110 is narrow, it can be seen that the obstacle is located in the vicinity of the rear end of the vehicle.

  In this way, the control unit 41 generates the overhead view video 100 in which the obstacle notification icon 110 in which the arc interval is changed is superimposed on the central video 105 of the overhead view 100 based on the detected distance to the obstacle. .

  As described above, in the present embodiment, when an obstacle is detected in the vicinity of the vehicle, the center image 105 of the overhead view image 100 indicates the direction in which the obstacle is detected, and the distance to the obstacle is indicated by the arc interval. As shown, the overhead image 100 on which the obstacle notification icon 110 is superimposed is displayed on the display panel 31. In this embodiment, the obstacle notification icon 110 can display the direction of the obstacle and the distance to the obstacle. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.

[Sixth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. 13 to 15. FIG. 13 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the sixth embodiment. FIG. 14 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment. FIG. 15 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment. The overhead video generation system 1 of the present embodiment differs from the overhead video generation system 1 of the first embodiment in the processing in the superimposed video generation unit 46 of the overhead video generation device 40.

  As shown in FIGS. 13 to 15, the obstacle notification icon 110 has a horizontal detection range of the sensor detecting the obstacle and a horizontal direction of the detected obstacle depending on the horizontal width of the icon itself. Indicates the width. In this embodiment, the obstacle notification icon 110 has a larger arc length as the detection range in the horizontal direction of the sensor detecting the obstacle is larger, and the detection range in the horizontal direction of the sensor detecting the obstacle. The narrower is the smaller the arc length. In the present embodiment, the obstacle notification icon 110 indicates that the arc length increases as the obstacle's horizontal width increases, and the arc length decreases as the obstacle's horizontal width decreases.

  More specifically, for example, the obstacle notification icon 110 may have a projected width from the center of the overhead image 100 to both ends in the horizontal direction of the obstacle as the length of the arc.

  Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 46 sets the width of the information indicating the obstacle as a width corresponding to the horizontal width of the detected obstacle. An overhead image 100 superimposed on the central image 105 is generated.

  An example in which an obstacle is detected on the left rear side when the vehicle is reversing will be described with reference to FIGS.

  For example, an example in which an obstacle is detected on the rear left side when the vehicle is reversing and the distance to the obstacle is 2 m will be described with reference to FIG. In step S <b> 14, the control unit 41 includes a detection range in the horizontal direction of the sensor that detects the obstacle, based on the horizontal width of the icon itself, in the central video 105 of the overhead image 100 in the superimposed video generation unit 46. An overhead view image 100 on which an obstacle notification icon 110 indicating the horizontal width of the detected obstacle is superimposed is generated. The horizontal width of the obstacle here is the apparent width detected by the sensor, that is, the projected width. The control unit 41 causes the superimposed image generation unit 46 to display the generated overhead image 100 illustrated in FIG. 13 on the display panel 31. It can be seen from the horizontal width of the obstacle notification icon 110 that the detection range in the horizontal direction of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrow.

  For example, an example when the distance from the state shown in FIG. 13 to the obstacle is 1 m will be described with reference to FIG. In step S <b> 14, the control unit 41 causes the superimposed video generation unit 46 to generate the overhead video 100 in which the obstacle notification icon 110 is superimposed on the central video 105 of the overhead video 100. Since the projected width from the center of the overhead image 100 to both ends of the obstacle in the horizontal direction is wide, the obstacle notification icon 110 has a horizontal width that is the horizontal width of the obstacle notification icon 110 in FIG. Wider. The control unit 41 causes the superimposed video generation unit 46 to display the generated overhead video 100 shown in FIG. 14 on the display panel 31. Since the horizontal width of the obstacle notification icon 110 is wider than that in FIG. 13, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are widened. That is, it can be seen that the obstacle is approaching the rear end of the vehicle.

  As described above, the superimposed video generation unit 46 changes the horizontal width of the obstacle notification icon 110 as shown in FIGS. 13 to 14 and displays the obstacle notification icon 110 according to the proximity state of the obstacle. A change in the width of the obstacle notification icon 110 in the horizontal direction indicates whether the obstacle is approaching or moving away from the vehicle.

  For example, an example in the case where an obstacle having a width smaller in the horizontal direction than the obstacle shown in FIG. 14 is detected on the left rear side when the vehicle is reversing will be described with reference to FIG. In step S <b> 14, the control unit 41 causes the superimposed video generation unit 46 to generate the overhead video 100 in which the obstacle notification icon 110 is superimposed on the central video 105 of the overhead video 100. The obstacle notification icon 110 has a horizontal width smaller than the horizontal width of the obstacle notification icon 110 in FIG. The control unit 41 causes the superimposed video generation unit 46 to display the generated overhead video 100 illustrated in FIG. 15 on the display panel 31. Since the horizontal width of the obstacle notification icon 110 is narrower than that in FIG. 14, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are reduced. That is, it can be seen that the horizontal width of the obstacle is narrow.

  As described above, in the present embodiment, when an obstacle is detected around the vehicle, the center image 105 of the bird's-eye view image 100 indicates the direction in which the obstacle is detected, and the obstacle width is indicated by the horizontal width of the arc. The overhead image 100 on which the obstacle notification icon 110 is superimposed so as to indicate the horizontal width is displayed on the display panel 31. In the present embodiment, the obstacle notification icon 110 can display the direction of the obstacle and the horizontal width of the obstacle. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.

  Further, in the present embodiment, when the distance to the obstacle changes, the horizontal width of the obstacle notification icon 110 is changed and displayed as shown in FIGS. 13 to 14, for example. Thereby, according to this embodiment, it is possible to check the proximity state of the obstacle by a change in the horizontal width of the obstacle notification icon 110.

  Now, the overhead video generation system 1 according to the present invention has been described so far, but may be implemented in various different forms other than the above-described embodiment.

  Each component of the illustrated bird's-eye view video generation system 1 is functionally conceptual and may not necessarily be physically configured as illustrated. That is, the specific form of each device is not limited to the one shown in the figure, and all or a part of them is functionally or physically distributed or integrated in arbitrary units according to the processing load or usage status of each device. May be.

  The configuration of the overhead view video generation system 1 is realized by, for example, a program loaded in a memory as software. The above embodiment has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The components described above include those that can be easily assumed by those skilled in the art and those that are substantially the same. Furthermore, the above-described configurations can be appropriately combined. In addition, various omissions, substitutions, or changes in the configuration can be made without departing from the scope of the present invention.

  For example, when the distance to the nearest obstacle is less than a predetermined threshold, the superimposed image generation unit 46 may display the distance to the obstacle as a numerical value together with the obstacle notification icon 110 in the central image 105 of the overhead view image 100. Good. Alternatively, for example, when the distance to the nearest obstacle is less than a predetermined threshold, the superimposed image generation unit 46 displays the shape or the shape according to the distance to the obstacle together with the obstacle notification icon 110 in the central image 105 of the overhead view image 100. A figure whose color changes may be displayed. The overhead image generation system 1 can notify the distance to the obstacle in addition to the change of the obstacle notification icon 110 according to the distance to the obstacle. Thereby, the bird's-eye view video generation system 1 can make it possible to more appropriately check obstacles around the vehicle.

  When there are a plurality of obstacles, the control unit 41 may display the obstacle notification icon 110 for an obstacle with a high priority. For example, the priority order may be the highest priority order with the smallest distance to the obstacle. Alternatively, for example, the priority may be the highest priority when the obstacle is a moving object. Alternatively, for example, the priority may be the highest priority when the obstacle is moving and is approaching the vehicle.

  When it is detected that the obstacle is a moving object, an icon indicating that the obstacle is a moving object may be displayed in the central image 105 of the overhead view video 100. The icon indicating a moving object may be, for example, an icon indicating a pedestrian or an icon indicating a vehicle. Thereby, the bird's-eye view video generation system 1 can make it possible to more appropriately check obstacles around the vehicle.

  Although the obstacle notification icon 110 has been described as being configured with a triple arc-shaped curve, the present invention is not limited to this. The obstacle notification icon 110 may be, for example, an arrow-shaped figure indicating the direction of the obstacle. In this case, the obstacle notification icon 110 may be an arrow-shaped figure whose thickness and size are changed according to the distance to the obstacle.

  In the present embodiment, the sensor that is an obstacle detection unit includes six sensors including a front left sensor 21, a front center sensor 22, a front right sensor 23, a rear left sensor 24, a rear center sensor 25, and a rear right sensor 26. However, the number of sensors is not limited. If it is desired to detect the direction of an obstacle more finely, the number of sensors may be increased.

  In the present embodiment, the obstacle notification icons 110 correspond to the number of sensors, the front left icon 111, the front center icon 112, the front right icon 113, the rear left icon 114, the rear center icon 115, and the rear right icon 116. However, the present invention is not limited to this. In other words, the obstacle notification icon 110 only needs to associate the detection result of each sensor with the icon to be displayed.

1 Overhead video generation system 11 Front camera (shooting unit)
12 Rear camera (shooting unit)
13 Left-side camera (shooting unit)
14 Right-side camera (shooting unit)
21 Front left sensor (obstacle detection unit)
22 Front center sensor (obstacle detection unit)
23 Front right sensor (obstacle detection unit)
24 Rear left sensor (obstacle detection unit)
25 Rear center sensor (obstacle detection unit)
26 Rear right sensor (obstacle detection unit)
Reference Signs List 31 Display Panel 40 Overhead Video Generation Device 41 Control Unit 42 Video Acquisition Unit 43 Obstacle Information Acquisition Unit 44 Vehicle Information Acquisition Unit 45 Overhead Video Generation Unit 46 Superimposed Video Generation Unit 47 Display Control Unit 49 Storage Unit 100 Overhead Video 110 Obstacle Notification Icon (information indicating an obstacle)
111 Front left icon (information indicating an obstacle)
112 Front center icon (information indicating an obstacle)
113 Front right icon (information indicating an obstacle)
114 Rear left icon (information indicating an obstacle)
115 Rear center icon (information indicating obstacles)
116 Rear right icon (information indicating an obstacle)
120 own car icon

Claims (11)

A video acquisition unit that acquires a peripheral video of the periphery of the vehicle;
An obstacle information acquisition unit for acquiring obstacle information of obstacles detected in the vicinity of the vehicle;
An overhead video generation unit that generates an overhead video obtained by performing a viewpoint conversion process so as to look down on the vehicle from above from the peripheral video acquired by the video acquisition unit;
Based on the obstacle information acquired by the obstacle information acquisition unit, a superimposed video generation unit that generates an overhead video in which information indicating the obstacle is superimposed on a central part surrounded by the overhead video;
A bird's-eye-view video generation device, comprising: a display control unit that causes the display unit to display the bird's-eye-view video generated by the superimposed video generation unit. The obstacle information acquisition unit acquires obstacle information including a distance to the detected obstacle,
If the distance to the detected obstacle is greater than or equal to a predetermined threshold based on the obstacle information acquired by the obstacle information acquisition unit, the superimposed image generation unit is configured to display the vehicle in a central portion surrounded by the overhead image. The vehicle icon is displayed to generate an overhead video in which the information indicating the obstacle is superimposed on the overhead video, and the detected obstacle is acquired based on the obstacle information acquired by the obstacle information acquisition unit. The bird's-eye view video generation apparatus according to claim 1, wherein if the distance is less than a predetermined threshold, a bird's-eye view image is generated by superimposing information indicating the obstacle on a central portion surrounded by the bird's-eye view image. The obstacle information acquisition unit acquires obstacle information of an obstacle detected in a range farther than a range to be displayed as the overhead image,
The superimposed video generation unit is surrounded by the overhead view video if the distance to the detected obstacle is farther than the range displayed as the overhead video based on the obstacle information acquired by the obstacle information acquisition unit. Based on the obstacle information acquired by the obstacle information acquisition unit, displaying an own vehicle icon indicating the vehicle in the center, generating an overhead image in which information indicating the obstacle is superimposed on the overhead image. If the distance to the detected obstacle is a range included in the range to be displayed as the overhead image, the overhead image is generated by superimposing information indicating the obstacle on a central portion surrounded by the overhead image. The overhead view video generation device according to 2.   The overhead image generation device according to any one of claims 1 to 3, wherein the information indicating the obstacle is information indicating a direction in which the obstacle is detected.   The overhead image generation device according to any one of claims 1 to 4, wherein the information indicating the obstacle changes according to a distance to the detected obstacle. The information indicating the obstacle has a plurality of components facing the direction in which the obstacle is detected, and indicates the distance to the obstacle by the interval of the plurality of components,
The superimposed video generation unit changes the interval between the plurality of components based on the distance to the detected obstacle.
The overhead image generation apparatus according to any one of claims 1 to 5. The obstacle information acquisition unit acquires obstacle information including a horizontal width of the detected obstacle,
The information indicating the obstacle includes information indicating a horizontal width of the detected obstacle,
The superimposed video generation unit, based on the obstacle information acquired by the obstacle information acquisition unit, sets the width of information indicating the obstacle as a width corresponding to the horizontal width of the detected obstacle. Generate a bird's-eye view image superimposed on the center part surrounded by
The overhead image generation device according to any one of claims 1 to 6.   The superimposed image generation unit is configured to surround information indicating the obstacle with the overhead view image for the obstacle located in the traveling direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit. The overhead image generation apparatus according to any one of claims 1 to 7, which generates an overhead image superimposed on a central portion. An overhead view video generation device according to any one of claims 1 to 8,
A photographing unit for photographing the periphery of the vehicle and supplying the peripheral image to the video acquisition unit;
An overhead view video generation system comprising: an obstacle detection unit that detects an obstacle around the vehicle and supplies the obstacle information to the obstacle information acquisition unit. A video acquisition step for acquiring a peripheral video of the periphery of the vehicle;
Obstacle information acquisition step for acquiring obstacle information of obstacles detected in the vicinity of the vehicle;
An overhead view video generation step of generating an overhead view image obtained by performing a viewpoint conversion process so as to look down on the vehicle from above from the surrounding video acquired in the video acquisition step;
Based on the obstacle information acquired in the obstacle information acquisition step, a superimposed video generation step of generating an overhead video in which information indicating the obstacle is superimposed on a central portion surrounded by the overhead video;
And a display control step of displaying the overhead view video generated in the superimposed video generation step on a display unit. A video acquisition step for acquiring a peripheral video of the periphery of the vehicle;
Obstacle information acquisition step for acquiring obstacle information of obstacles detected in the vicinity of the vehicle;
An overhead view video generation step of generating an overhead view image obtained by performing a viewpoint conversion process so as to look down on the vehicle from above from the surrounding video acquired in the video acquisition step;
Based on the obstacle information acquired in the obstacle information acquisition step, a superimposed video generation step of generating an overhead video in which information indicating the obstacle is superimposed on a central portion surrounded by the overhead video;
A program for causing a computer that operates as an overhead video generation device to execute a display control step of displaying the overhead video generated in the superimposed video generation step on a display unit.

Images