JP2018158664A - Driving support device, driving support method and program - Google Patents

Abstract

A vehicle peripheral image is displayed in a more easily viewable manner. A driving support device includes an image acquisition unit that acquires a vehicle surrounding image by capturing the surroundings of a vehicle, an image analysis unit that detects a reference object included in the vehicle surrounding image, and an image sensor existing around the reference object. A data acquisition unit that acquires data indicating a relative positional relationship of at least one of an obstacle and a road marking to a reference object from the outside of the own vehicle 50; An image generator for generating a driving assistance image in which an image representing at least one of an obstacle and a road marking is superimposed on a vehicle peripheral image based on the position of the included reference object, and a display for displaying the driving assistance image on a display device. and a control unit. [Selection drawing] Fig. 4

Description

  The present invention relates to a driving support device, a driving support method, and a program.

  2. Description of the Related Art In recent years, there has been known an apparatus that presents a driver with a vehicle periphery image captured by an in-vehicle camera for the purpose of assisting driving during parking. For example, a rear image obtained by imaging the rear of the vehicle is displayed, or an overhead image generated by combining images from a plurality of in-vehicle cameras is displayed. In addition, a technique for displaying an overhead image by synthesizing an image captured by a side camera of a target parking lot and an image captured by a rear camera when retreating toward the target parking lot has been proposed (for example, Patent Document 1).

JP 2002-29350 A

  During daytime or in fine weather, white lines and obstacles around the vehicle can be clearly captured by the in-vehicle camera. The tendency is also likely to decline. Further, when the host vehicle is positioned on the road marking, the road marking under the host vehicle cannot be grasped from the vehicle peripheral image. Thus, in a situation where obstacles and road markings around the vehicle cannot be partially imaged, the convenience of the vehicle periphery image is reduced.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to provide a technique for displaying a vehicle peripheral image more easily.

  A driving support apparatus according to an aspect of the present invention includes an image acquisition unit that acquires a vehicle peripheral image obtained by imaging the periphery of the host vehicle, an image analysis unit that detects a reference object included in the vehicle peripheral image, and a periphery of the reference object. A data acquisition unit that acquires data indicating the relative positional relationship with respect to at least one of the obstacles and road markings that are present from the outside of the host vehicle, and data that the data acquisition unit acquires, An image generating unit that generates a driving assistance image in which an image showing at least one of an obstacle and a road marking is superimposed on a vehicle peripheral image with reference to the position of the included reference object, and a display that displays the driving assistance image on a display device A control unit.

  Another aspect of the present invention is a driving support method. In this method, a vehicle peripheral image obtained by capturing an image of the periphery of the host vehicle is acquired, and a reference object included in the vehicle peripheral image is detected, and an obstacle existing around the reference object and at least one reference object of road markings A step of acquiring data indicating a relative positional relationship with respect to the vehicle from outside the host vehicle, and using the acquired data, the position of the reference object included in the vehicle peripheral image is used as a reference, and an obstacle and a road surface are displayed on the vehicle peripheral image. A step of generating a driving assistance image on which an image showing at least one of the signs is superimposed; and a step of displaying the driving assistance image on a display device.

  Note that any combination of the above-described constituent elements and the constituent elements and expressions of the present invention replaced with each other among methods, apparatuses, systems, etc. are also effective as an aspect of the present invention.

  According to the present invention, a vehicle periphery image can be displayed more easily.

It is a figure which shows typically the condition which parks the own vehicle in a parking space. It is a figure which shows typically an example of the bird's-eye view image at the time of parking. It is a figure which shows an example of the additional image superimposed. It is a figure which shows typically an example of the driving assistance image which concerns on embodiment. It is a functional block diagram which shows typically the structure of a driving assistance device. It is a flowchart which shows the flow of a driving assistance method typically.

  Before describing the present invention in detail, an outline will be described. The present embodiment is a driving support device for displaying a vehicle periphery image obtained by imaging the periphery of the host vehicle with an in-vehicle camera. The vehicle periphery image presented by the driving support device is a rear image obtained by imaging the rear of the vehicle, an overhead image obtained by performing viewpoint conversion processing as if the periphery of the host vehicle is viewed from above. The driver refers to the surrounding image of the vehicle when the host vehicle is moved backward toward the parking space defined by the road marking such as a white line, so that the driver is adjacent to an obstacle such as a pillar or a wall displayed in the surrounding image of the vehicle. You can drive while grasping the position of other vehicles in the parking space.

  During daytime or in fine weather, white lines and obstacles around the vehicle can be clearly captured by the in-vehicle camera. The tendency is also likely to decline. Further, when the host vehicle is located on the white line, the position of the white line below the host vehicle cannot be grasped from the vehicle peripheral image. Thus, in a situation where white lines and obstacles around the vehicle cannot be partially imaged, the convenience of the vehicle periphery image captured from the host vehicle is reduced.

  Therefore, in the present embodiment, data indicating the position of obstacles and road markings present around the host vehicle is acquired wirelessly from the outside of the host vehicle, and the positions of obstacles and road markings are determined based on the acquired data. The displayed image is superimposed on the vehicle surrounding image currently captured. At this time, alignment is performed so that the reference object included in the vehicle peripheral image where the host vehicle is imaged corresponds to the position of the reference object of the acquired data. According to the present embodiment, it is possible to compensate for a missing portion in a vehicle peripheral image captured by the host vehicle with data acquired from the outside, and it is possible to provide a driving support image that makes it easier to grasp obstacles and the like around the vehicle.

  Embodiments of the present invention will be described below with reference to the drawings. Specific numerical values and the like shown in the embodiments are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  FIG. 1 is a diagram schematically illustrating a situation in which the host vehicle 50 is parked in a parking space, and shows a state in which the host vehicle 50 is moved backward to enter the parking space. In the parking lot 52, a plurality of parking spaces are arranged side by side, and the first parking space 60a and the second parking space 60b are vacant. In the vicinity of the first parking space 60 a, there are obstacles such as the columns 54 of the parking lot 52 and the disaster prevention equipment 56. Each parking space is partitioned by road markings such as white lines 58. Other vehicles 62a and 62b exist in a section adjacent to the vacant parking spaces 60a and 60b. The driver of the host vehicle 50 needs to perform a driving operation so that the host vehicle 50 is within the section of the white line 58 while paying attention to obstacles and other vehicles located around the host vehicle.

  FIG. 2 is a diagram schematically illustrating an example of the overhead image 70 at the time of parking. The bird's-eye view image 70 is a vehicle peripheral image obtained by imaging the periphery of the host vehicle 50, and is an image that has undergone viewpoint conversion processing as if the periphery of the host vehicle 50 is viewed from above. The bird's-eye view image 70 includes obstacles and road markings located around the host vehicle 50 around the position of the host vehicle 50. In the illustrated example, a support 54, a disaster prevention facility 56, a white line 58, and other vehicles 62a and 62b are included.

  The bird's-eye view image 70 as shown can be generated using a known algorithm. For example, by acquiring and synthesizing images captured by each of a plurality of cameras provided at four locations on the front, rear, left and right of the host vehicle 50, an overhead image 70 as if the periphery of the host vehicle 50 is viewed from above is obtained. Can be generated. In addition, the range included in the bird's-eye view image 70 in the situation around the host vehicle 50 is not limited to that illustrated, and may be wider or narrower than the illustrated range. Further, not all the front, rear, left and right situations of the host vehicle 50 may be included, and for example, an image of only the rear half of the host vehicle 50 may be used.

  In FIG. 2, the host vehicle 50 is positioned so as to straddle both the first parking space 60a and the second parking space 60b, and is one of the road markings that divide the space between the first parking space 60a and the second parking space 60b. The part is hidden by the own vehicle 50 and cannot be seen. As a result, it may be difficult to accurately grasp the positional relationship between the boundary between the first parking space 60a and the second parking space 60b and the host vehicle 50 only by looking at the overhead image 70 shown in FIG. Therefore, in the present embodiment, a part of the white line 58 hidden by the host vehicle 50 is superimposed so that the entire white line 58 is visualized so that the position of the host vehicle 50 can be easily understood.

  FIG. 3 is a diagram schematically illustrating an example of the additional image 72 to be superimposed. The additional image 72 is an image including obstacles and road markings in the parking lot 52, and is an image showing the internal structure of the parking lot 52 as an overhead image. The additional image 72 includes obstacles such as the pillars 54 and the disaster prevention equipment 56 and road markings such as the white line 58, but does not include other vehicles located in each parking space. This is because the positions of obstacles and road markings are fixed regardless of the passage of time, while the positions of other vehicles located in each parking space change with the passage of time.

  FIG. 4 is a diagram schematically illustrating an example of the driving support image 74 according to the embodiment. 4 is generated by superimposing a partial area A of the additional image 72 of FIG. 3 on the overhead image 70 of FIG. Specifically, a common reference point included in FIGS. 2 and 3 is specified, and the two are overlapped so that the positions of the reference points correspond to each other. Here, the common reference point is an obstacle such as the column 54 or the disaster prevention facility 56 or a road marking such as the white line 58, and a feature point can be extracted or easily. If it is a flap type parking lot, the flap board installed on a road surface can also be used as a reference point. As shown in FIG. 4, by presenting the driving support image 74 on which the additional image 72 is superimposed, the situation of the host vehicle 50 can be shown more clearly than when the overhead image 70 of FIG. 2 is presented.

  In FIG. 3, as an example of the additional image 72, a case where a range wider than the partial region A superimposed on the overhead image 70 is included. Such an additional image 72 can be generated from, for example, drawing data indicating an arrangement of an obstacle such as a parking lot or a road marking. The additional image 72 may be generated based on a vehicle surrounding image captured by the own vehicle or another vehicle, and the range included in the additional image 72 in this case may be similar to the overhead image 70, It may be larger or smaller than the overhead image 70. Specifically, only the same range as the partial area A in FIG. 3 may be included in the additional image 72. Further, the image data that is the source of the additional image 72 is not limited to a bird's-eye view image, and may be an image obtained by capturing the front, side, or rear of the vehicle from the own vehicle or another vehicle.

  FIG. 5 is a functional block diagram schematically showing the configuration of the driving support device 10. Each functional block shown in the figure can be realized by hardware such as a computer CPU and memory, or a mechanical device, and can be realized by software by a computer program or the like. It is drawn as a functional block to be realized. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by a combination of hardware and software.

  The driving support device 10 includes a touch display 12 and a control device 18. The touch display 12 includes a display device 14 and an input device 16. The display device 14 is a display device such as a liquid crystal display, and is attached to the position of the center console or dashboard of the vehicle. The input device 16 is a so-called touch panel sensor, and is provided in the display area of the display device 14. Note that the input device 16 is not a touch type, and may be configured with buttons or the like arranged around the display device 14. Moreover, the display apparatus 14 may be provided in the position of the rear view mirror for visually recognizing the vehicle rear, and may be comprised as a head-up display. The driving support device 10 may not include the touch display 12 and may control display on a display device such as a display different from the driving support device 10.

  The control device 18 generates a driving support image 74 and causes the display device 14 to display the generated image. The control device 18 is connected to the wireless communication device 30, the in-vehicle camera 32, the vehicle speed sensor 34, the gyro sensor 36, and the navigation device 38, and acquires information necessary for generating a display image therefrom. The wireless communication device 30 is connected to a network through a wireless communication line, and exchanges data with a server (not shown) provided outside the host vehicle 50.

  The control device 18 includes an image acquisition unit 20, an image analysis unit 22, a data acquisition unit 24, a data transmission unit 25, an image generation unit 26, and a display control unit 28.

  The image acquisition unit 20 acquires an image obtained by capturing the periphery of the host vehicle 50 from the in-vehicle camera 32. The image acquisition unit 20 acquires, for example, images captured by each of a plurality of cameras provided at four locations on the front, rear, left, and right of the host vehicle 50, and performs viewpoint conversion processing as if the periphery of the host vehicle 50 is viewed from above. It is possible to generate a bird's-eye view image that has been subjected to. Note that the image acquired by the image acquisition unit 20 is not particularly limited, and an arbitrary image necessary for generating a vehicle peripheral image indicating a peripheral situation of the host vehicle 50 such as a rear image of the host vehicle 50 may be acquired. .

  The image analysis unit 22 detects a reference object included in the vehicle peripheral image acquired by the image acquisition unit 20 using an analysis technique such as pattern matching. For example, in the bird's-eye view image 70 shown in FIG. 2, the support 54, the disaster prevention facility 56, the white line 58, and the like included in the image are detected as reference objects. If it is a flap type parking lot, you may detect the flap board on a road surface as a reference | standard thing. The image analysis unit 22 may detect a reference object set in advance from the vehicle periphery image, or specify an object that is a candidate for the reference object from the vehicle periphery image, and use either of them as a reference object. It may be detected.

  The data acquisition unit 24 acquires data necessary for generating the additional image 72 from the outside of the host vehicle 50 via the wireless communication device 30. The data acquired by the data acquisition unit 24 includes information and images indicating the relative positional relationship between the reference object analyzed by the image analysis unit 22 and obstacles and road markings present around the reference object. . The additional image 72 is generated by the image generation unit 26 based on the data acquired by the data acquisition unit 24.

  An example of data acquired by the data acquisition unit 24 is image data, and is based on an image captured in advance in the parking lot 52, an overhead image captured in the past by a vehicle using the parking lot 52, or the like. When images captured in advance include images such as other vehicles and passers-by whose positions can change over time, it is preferable to remove them in advance by image processing. Moreover, when using the vehicle periphery image which the vehicle-mounted camera of the other vehicle imaged, it is preferable to use the image whose position of the vehicle-mounted camera is equivalent to the host vehicle 50. This is because if the camera mounting position and orientation are different, the viewpoint and angle of view of the captured image are different, and the appearance of the captured image is different. For example, it is preferable to acquire image data captured by another vehicle of the same type as that of the host vehicle 50 using a genuine in-vehicle camera.

  The data acquired by the data acquisition unit 24 may be drawing data indicating the arrangement of the reference object, obstacles around the reference object, and road markings. Even if the additional image 72 is generated based on the drawing data. Good. In this case, the additional image 72 can be generated by generating a two-dimensional or three-dimensional virtual space based on the drawing data and rendering a position corresponding to the angle of view of the overhead image 70. The image generation unit 26 generates an additional image 72 corresponding to the angle of view of the overhead image 70 based on the drawing data acquired by the data acquisition unit 24.

  The data transmission unit 25 transmits the vehicle periphery image captured by the host vehicle 50 to the server via the wireless communication device 30. The image transmitted from the host vehicle 50 is stored in a server and used as original data of the additional image 72 acquired by the host vehicle or another vehicle. The data transmission unit 25 may transmit image attribute information such as the vehicle type, imaging date and time, imaging location, and weather at the time of imaging to the server in addition to the captured vehicle surrounding image.

  The image generation unit 26 generates a driving support image 74 by superimposing an additional image on the vehicle periphery image. The image generation unit 26 superimposes the two images so that the positions of the reference object included in the currently captured vehicle peripheral image and the reference object included in the additional image based on the data acquired from the outside correspond to each other. For example, in the example shown in FIGS. 2 to 4, the column 54, the disaster prevention equipment 56, the tip of the white line 58, or the like can be used as a reference for alignment. If it is a flap type parking lot, the flap board on a road surface can be used as a reference | standard thing.

  When superimposing the additional image 72 on the overhead image 70, the image generation unit 26 may perform an enlargement / reduction process on the additional image 72 so that the size of the additional image 72 matches the overhead image 70. The original data of the additional image 72 may include size information indicating the size and scale of obstacles and road markings included in the additional image 72, and the image generation unit 26 performs enlargement / reduction processing based on the size information. An additional image 72 subjected to the above may be generated. In addition, when using a vehicle peripheral image captured by another vehicle of the same vehicle type as the own vehicle 50 as the original data of the additional image 72, the viewpoint and the angle of view of the captured image coincide with each other. You may superimpose on the bird's-eye view image 70, without performing reduction processing.

  Depending on the data stored in the server, the additional image 72 corresponding to the current situation around the host vehicle 50 may not be obtained. Further, since the reference object cannot be appropriately detected from the vehicle peripheral image, the additional image 72 may not be superimposed. In such a case, the image generation unit 26 generates the overhead image 70 as a driving assistance image without superimposing the additional image 72.

  When the own vehicle 50 moves and the vehicle surrounding image currently being captured changes, the image generation unit 26 switches the additional image 72 to be superimposed according to the change. For example, the image generation unit 26 acquires and superimposes different additional images 72 according to the position of the host vehicle 50 that is the movement destination, and causes the driving support image 78 to be updated and displayed. The image generation unit 26 may update and display the driving support image 78 by changing the superimposed position and orientation of the additional image 72 when the change in the position of the host vehicle 50 is small.

  The display control unit 28 causes the display device 14 to display an image generated by the image generation unit 26. The display control unit 28 displays, for example, a driving assistance image 74 as shown in FIG. 4 on the display device 14 and presents a vehicle surrounding image in which the surrounding situation is easier to understand to the driver.

  According to the above configuration, the driving support device 10 displays the image supplemented by the additional image 72 on the overhead image 70 captured by the host vehicle 50 to compensate for a portion that is not visible or difficult to see with the overhead image 70 alone. Can do. For example, it is possible to visualize a location that is hidden by the host vehicle 50 or a location that does not appear as a blind spot of the in-vehicle camera 32 of the host vehicle 50. Moreover, even if it is within the range of the field angle of the in-vehicle camera 32, it is possible to easily display a portion that is difficult to see at night or is shaded because it is not exposed to illumination light. Therefore, according to the present embodiment, it is possible to present a driving assistance image in which the position of an obstacle around the vehicle is easy to see even in a situation where the situation around the vehicle is not sufficiently projected. Thereby, a user's convenience can be improved.

  FIG. 6 is a flowchart schematically showing the flow of the driving support method. The image acquisition unit 20 acquires a vehicle periphery image from the in-vehicle camera 32 (S10). The image analysis unit 22 analyzes the acquired current vehicle periphery image to identify a reference object, and the data acquisition unit 24 acquires data related to obstacles and road markings around the host vehicle (S12). If the additional image indicating the position of the obstacle and the road marking around the host vehicle can be superimposed (Y in S14), the image generation unit 26 corresponds to the position of the reference object common to the vehicle peripheral image and the additional image. In this manner, the additional image is superimposed on the vehicle peripheral image and synthesized (S16). The display control unit 28 displays the driving support image synthesized by the image generation unit 26 (S18). On the other hand, if the additional image cannot be superimposed (N in S14), the captured vehicle peripheral image is displayed (S20).

  As described above, the present invention has been described with reference to the above-described embodiment. However, the present invention is not limited to the above-described embodiment, and the configuration shown in each display example is appropriately combined or replaced. Are also included in the present invention.

  In the above-described embodiment, the case where the additional image 72 is displayed superimposed on the overhead image 70 of the host vehicle 50 has been shown. In a modified example, an additional image may be superimposed and displayed on the front image, the side image, or the rear image of the host vehicle 50.

  In the above-described embodiment, the driving assistance image displayed when the host vehicle 50 is moved backward with the steering wheel turned to the left is illustrated. In the modified example, the above-described contents may be applied to the case where the steering wheel of the host vehicle 50 is moved backward while being turned to the right, the vehicle 50 is moved back in a straight traveling state, the host vehicle 50 is moved forward, or the like.

  DESCRIPTION OF SYMBOLS 10 ... Driving assistance apparatus, 14 ... Display apparatus, 20 ... Image acquisition part, 22 ... Image analysis part, 24 ... Data acquisition part, 25 ... Data transmission part, 26 ... Image generation part, 28 ... Display control part, 32 ... In-vehicle Camera, 50 ... own vehicle, 74 ... driving assistance image.

Claims (8)

An image acquisition unit for acquiring a vehicle periphery image obtained by imaging the periphery of the own vehicle;
An image analysis unit for detecting a reference object included in the vehicle periphery image;
A data acquisition unit for acquiring data indicating a relative positional relationship with respect to the reference object of at least one of obstacles and road markings present around the reference object from outside the host vehicle;
Using the data acquired by the data acquisition unit, an image indicating at least one of the obstacle and the road marking is superimposed on the vehicle peripheral image with reference to the position of the reference object included in the vehicle peripheral image. An image generation unit for generating a driving support image;
And a display control unit that displays the driving support image on a display device.   The driving support device according to claim 1, wherein the data acquisition unit acquires image data obtained by imaging the reference object and at least one of an obstacle and a road marking present around the reference object. .   The driving support apparatus according to claim 1, wherein the data acquisition unit acquires data based on a vehicle peripheral image captured by an in-vehicle camera of another vehicle different from the host vehicle.   The driving support device according to claim 3, wherein the data acquisition unit acquires data based on a vehicle surrounding image captured by an in-vehicle camera of another vehicle that is the same type as the host vehicle.   5. The driving support device according to claim 1, further comprising a data transmission unit configured to transmit data based on a vehicle peripheral image captured by an in-vehicle camera of the host vehicle to the outside of the host vehicle.   The driving support device according to claim 1, wherein the data acquisition unit acquires drawing data including the reference object and at least one of an obstacle and a road marking present around the reference object. Acquiring a vehicle peripheral image obtained by imaging the periphery of the host vehicle, and detecting a reference object included in the vehicle peripheral image;
Obtaining from the outside of the host vehicle data indicating a relative positional relationship of at least one of obstacles and road markings present around the reference object with respect to the reference object;
A driving assistance image obtained by superimposing an image indicating at least one of the obstacle and the road marking on the vehicle peripheral image using the acquired data as a reference based on the position of the reference object included in the vehicle peripheral image. Generating step;
Displaying the driving assistance image on a display device. A driving assistance method comprising: A function of acquiring a vehicle peripheral image obtained by imaging the periphery of the host vehicle and detecting a reference object included in the vehicle peripheral image;
A function of acquiring data indicating a relative positional relationship of at least one of an obstacle present around the reference object and a road marking with respect to the reference object from outside the host vehicle;
A driving assistance image obtained by superimposing an image indicating at least one of the obstacle and the road marking on the vehicle peripheral image using the acquired data as a reference based on the position of the reference object included in the vehicle peripheral image. The function to generate,
A program for causing a computer to realize a function of displaying the driving assistance image on a display device.

Images