JP2010081385A - Driving assistance device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem in which when a vehicle turns, a background is rotated in an all-round bird's-eye view image without rotation of the image of the vehicle. <P>SOLUTION: In a driving assistance device, an image generation unit 30 generates a bird's-eye view image around the vehicle based on an image taken by at least one imaging device installed in the vehicle. A display device 14 displays the bird's-eye view image generated by the image generation unit 30. The image generation unit 30 generates the bird's-eye view image in which an image of a parking space is fixed when a relative relation between the direction of the vehicle and the direction of the parking space changes. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to driving support technology, and more particularly to a driving support device that supports driving based on an image captured by an imaging device installed in a vehicle.

A bird's-eye view image (hereinafter referred to as an all-round view as appropriate) that converts the images captured by a plurality of cameras installed around the vehicle into bird's-eye view images and combines the bird's-eye view images. A technique for generating a bird's eye view image) has been proposed. This all-around bird's-eye view image can provide useful information to the driver and can support driving. In particular, driving during parking can be supported.
JP 2007-269060 A

  The all-around bird's-eye view image is generated by synthesizing images captured by a plurality of imaging devices fixed around the vehicle (more specifically, images converted into bird's-eye view images). Therefore, an all-around bird's-eye view image in which the vehicle orientation is usually vertical (usually the vehicle head is up) is generated and displayed. Therefore, when the vehicle turns, the vehicle does not rotate but the background rotates. When parking with turning, the parking space, not the vehicle, rotates.

  The present inventor has recognized the above situation and made the present invention. The purpose of the present inventor is to provide a technology capable of displaying a bird's eye view image that makes it easier for the driver to realize the positional relationship between the parking space and the vehicle. It is to be.

  A driving support device according to an aspect of the present invention is generated by an image generation unit that generates a bird's-eye view image around a vehicle based on an image captured by at least one imaging device installed in the vehicle, and an image generation unit. And a display unit for displaying a bird's eye view image. When the relative relationship between the direction of the vehicle and the direction of the parking space changes, the image generation unit generates a bird's eye view image in which the direction of the parking space is fixed.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the bird's-eye view image which a driver | operator can feel more easily the positional relationship between a parking space and a vehicle can be displayed.

  FIG. 1 shows a configuration of a vehicle 100 equipped with a driving support apparatus 10 according to an embodiment of the present invention. The vehicle 100 includes a driving support device 10, a first imaging device 12a, a second imaging device 12b, a third imaging device 12c, and a fourth imaging device 12d (hereinafter referred to as a first imaging device 12a, a second imaging device 12b, and a third imaging device). 12c and fourth imaging device 12d are collectively referred to as imaging device 12), and display device 14. The vehicle 100 includes various components such as an engine, a chassis, a steering wheel, a brake, and the like, but are omitted here. Further, the left side of the vehicle 100 shown in FIG. 1 corresponds to the front side.

  The imaging device 12 continuously captures images and outputs the captured images to the driving support device 10. In this specification, an image itself or image data is referred to as an “image” without distinction. The first imaging device 12 a and the second imaging device 12 b are installed in the front part of the vehicle 100, and the third imaging device 12 c and the fourth imaging device 12 d are installed in the rear part of the vehicle 100. Here, the first imaging device 12a to the fourth imaging device 12d form a stereo camera in an arbitrary combination. A stereo camera is a camera that can record information in the depth direction by simultaneously photographing an object from a plurality of different directions. The number of imaging devices 12 is not limited to “4”.

  The driving assistance device 10 receives images captured by the first imaging device 12a, the second imaging device 12b, the third imaging device 12c, and the fourth imaging device 12d. The driving support device 10 converts the input images into bird's-eye view images, and synthesizes the bird's-eye view images to generate a synthesized bird's-eye view image. The driving support device 10 outputs the synthesized bird's-eye view image to the display device 14. The display device 14 displays a composite bird's-eye view image input from the driving support device 10 (more specifically, generated by the image generation unit 30 described later). The display device 14 may be a dedicated terminal device, or a display device of a car navigation device. Further, the driving support device 10 and the display device 14 may be configured integrally.

  The driving support device 10 detects an obstacle around the vehicle. When the driving support device 10 detects an obstacle, the driving support device 10 outputs a warning to that effect to the driver.

  FIG. 2 shows a configuration of the driving support apparatus 10 according to the embodiment. The driving support device 10 includes a first frame buffer 20a, a second frame buffer 20b, a third frame buffer 20c, a fourth frame buffer 20d (a first frame buffer 20a, a second frame buffer 20b, a third frame buffer 20c, and a fourth frame buffer 20c). The frame buffer 20d is collectively referred to as a frame buffer 20), an image generation unit 30, a control unit 40, and an operation unit 50.

  The configuration of the driving support device 10 can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in software, it is realized by a program loaded in the memory. Describes functional blocks realized by collaboration. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The first frame buffer 20a, the second frame buffer 20b, the third frame buffer 20c, and the fourth frame buffer 20d are provided by the first imaging device 12a, the second imaging device 12b, the third imaging device 12c, and the fourth imaging device 12d. Each captured image is input. The frame buffer 20 temporarily holds the image input from the imaging device 12 and outputs the image to the image generation unit 30.

  The conversion table storage unit 24 stores a conversion formula for converting an image captured by the imaging device 12 into a bird's eye view image. This conversion formula is for converting the viewpoint of the actually installed imaging device 12 into an image viewed from the viewpoint of the virtual imaging device (hereinafter referred to as a virtual viewpoint). The position of this virtual viewpoint can be arbitrarily set by the designer. The conversion table storage unit 24 may be constructed in the form of a bird's eye conversion mapping lookup table. In this case, the image conversion unit 32 converts the coordinates of the image input from the frame buffer 20 with reference to the lookup table.

  The image generation unit 30 captures an image captured by at least one imaging device (for example, the first imaging device 12a, the second imaging device 12b, the third imaging device 12c, and the fourth imaging device 12d) installed in the vehicle 100. Based on this, a bird's eye view image around the vehicle 100 is generated. This bird's-eye view image may be the all-around bird's-eye view image. In addition, the above-described bird's-eye view image around the vehicle 100 preferably has a composition including the entire vehicle 100, but the composition may not necessarily include the entire vehicle 100. That is, a composition that includes a part of the vehicle 100 or a composition that does not include the vehicle 100 may be used. Hereinafter, an example in which the image generation unit 30 generates the all-around bird's-eye view image will be described.

  When images are continuously input from the imaging device 12 to the driving support device 10, the image generation unit 30 continuously generates all-around bird's-eye view images. That is, the image generation unit 30 generates a moving image of the all-around bird's-eye view.

  In the parking mode, when the relative relationship between the direction of the vehicle 100 and the direction of the parking space changes, the image generation unit 30 generates an all-around bird's-eye view image in which the direction of the parking space is fixed. For example, the all-around bird's-eye view image in which the direction of the parking space is fixed in the vertical direction is generated. Here, the parking mode includes a garage entry mode when the vehicle 100 is put into the parking space and a garage exit mode where the vehicle 100 is taken out from the parking space. The driver enters the garage entry mode or the garage exit mode by operating the operation unit 50.

  The all-around bird's-eye view image in which the direction of the parking space is fixed is an image in which the direction of the background is fixed and the vehicle 100 turns when the vehicle 100 turns. Normally, since the imaging device 12 is fixed to the vehicle 100, when the vehicle 100 turns, the orientation of the vehicle 100 is fixed and the background rotates in the all-around bird's-eye view image. The image generation unit 30 can generate an all-around bird's-eye view image in which the direction of the parking space is fixed regardless of the direction of the vehicle 100 by being controlled by the control unit 40.

  The image generation unit 30 includes an image conversion unit 32 and an image synthesis unit 34. The image conversion unit 32 converts the respective images input from the first frame buffer 20a, the second frame buffer 20b, the third frame buffer 20c, and the fourth frame buffer 20d into conversion formulas held in the conversion table storage unit 24. And converted into a bird's-eye view image.

  The image synthesis unit 34 generates a full-circle bird's-eye view image by synthesizing the plurality of bird's-eye view images converted by the image conversion unit 32.

  For the conversion and generation of the all-around bird's-eye view image, a known technique, for example, a technique disclosed in Japanese Patent Application Laid-Open No. 2008-48345 can be used.

  The motion detection unit 42 detects the rotation angle of the parking space in the all-around bird's-eye view image generated by the image generation unit 30 due to the turning of the vehicle 100. The angle is output to the control unit 40. Of course, not only the parking space but also the entire background in the all-around bird's-eye view image rotates as the vehicle 100 turns. As the starting point of the rotation, the all-around bird's-eye view image based on the image captured when the driver enters the garage entry mode or the garage exit mode by operating the operation unit 50 is used. The motion detection unit 42 can detect the rotation angle of the background including the parking space by detecting the optical flow in the all-around bird's-eye view image generated by the image generation unit 30.

  The optical flow is one of moving body analysis techniques, and is a technique for analyzing a motion from luminance information in an image and representing a motion of the moving body by a velocity vector. Since detection of such an optical flow is a known technique, a detailed description thereof is omitted here.

  The motion detection unit 42 can detect a motion vector between the frames in the continuously generated all-around bird's-eye view image. The motion detection unit 42 integrates this motion vector from the all-round bird's-eye view image that is the starting point to the current all-round bird's-eye view image, thereby detecting the amount of movement and the amount of rotation (that is, the rotation angle) between them. Can do. Note that the motion detection unit 42 may set a plurality of feature points in the all-around bird's-eye view image and estimate the motion vector of the entire background based on the motion vectors of these feature points.

  The control unit 40 controls the image generation unit 30 based on signals input from the motion detection unit 42 and the operation unit 50. The control unit 40 controls the image generation unit 30 to output to the display device 14 after reversely rotating the generated all-around bird's-eye view image based on the rotation angle detected by the motion detection unit 42. Thereby, the direction of the background is fixed on the display device 14, and the all-around bird's-eye view image in which the vehicle 100 rotates is displayed.

  In the present embodiment, it is preferable that the image generation unit 30 generates an all-around bird's-eye view image that is larger than the display area of the display device 14. In this case, after the all-around bird's-eye view image is reversely rotated by the control unit 40, the image generation unit 30 can cut out a rectangular area corresponding to the display area and output it to the display device 14.

  The operation unit 50 receives an instruction from the driver and outputs the instruction to the control unit 40. For example, the operation unit 50 may include a garage entry mode button and a garage exit mode button. One parking mode button may be provided. In this case, the garage entry mode and the garage exit mode may be distinguished depending on the number of times the parking mode button is pressed or whether or not the parking mode button is pressed. Further, a rotation button for changing the direction of the all-around bird's-eye view image displayed on the display device 14 may be provided. For example, every time the rotation button is pressed, the all-around bird's-eye view image may be rotated 1 °, 45 °, or 90 ° clockwise or counterclockwise.

  FIG. 3 is a diagram for explaining the all-around bird's-eye view image displayed on the display device 14 when leaving the garage. FIG. 3 shows an example in which the vehicle 100 parked forward in the parking space 70 is moved backward while turning and is taken out from the parking space 70. In FIG. 3, dotted arrows indicate the longitudinal direction of the vehicle 100 in the all-around bird's-eye view images 60 and 62. A solid line arrow (straight line) indicates a short direction of the parking space 70 in the all-around bird's-eye view images 60 and 62. In the garage exit mode, an axis parallel to the longitudinal direction of the parking space 70 is set as a reference axis for obtaining the background rotation angle θ.

  Fig.3 (a) is a figure which shows the state parked forward before garage removal. When starting the garage removal, the driver enters the garage removal mode by operating the operation unit 50. At this time, since the longitudinal direction of the vehicle 100 and the longitudinal direction of the parking space 70 are generally substantially parallel, the longitudinal direction of the parking space 70 in the all-around bird's-eye view image 60 displayed on the display device 14 is vertical. It is reflected. This composition is a composition in which the driver can easily grasp the positional relationship between the parking space 70 and the vehicle 100.

  FIG.3 (b) is a figure which shows the state in the middle of the garage removal based on a prior art. In this example, the driver retreats the vehicle 100 to a predetermined position on the right rear side of the parking space 70 while turning the vehicle 100 counterclockwise, and turns counterclockwise from that position until it is substantially parallel to the roadway. Move forward and go straight when it is almost parallel.

  In the driving support device according to the related art, since the orientation of the vehicle 100 in the all-round bird's eye view image 62 displayed on the display device is fixed, the vehicle 100 is rotated counterclockwise by the rotation angle θ in the real world. Then, the background including the parking space 70 in the all-around bird's-eye view image 62 is rotated clockwise by the rotation angle θ. Since the vehicle 100 moves in the real world and the parking space 70 does not move as a matter of course, the all-around bird's-eye view image 62 displayed on the display device of the driving assistance device according to the related art is an unnatural image.

  FIG.3 (c) is a figure which shows the state in the middle of the garage removal which concerns on embodiment. The driving support device 10 according to the embodiment includes the parking space 70 in the all-around bird's-eye view image generated by the image generation unit 30 when the vehicle 100 is rotated counterclockwise by the rotation angle θ in the real world. The background is in a state of being rotated clockwise by the rotation angle θ (the same state as in FIG. 3b). Under the control of the control unit 40, the image generation unit 30 rotates the all-around bird's-eye view image counterclockwise by the rotation angle θ. Therefore, the all-around bird's-eye view image 60 displayed on the display device 14 has a composition in which the vehicle 100 is rotated counterclockwise by the rotation angle θ with respect to the reference axis. The all-around bird's-eye view image 60 displayed on the display device 14 of the driving assistance apparatus 10 according to the embodiment is a natural image that matches the movement of the real world.

  FIG. 4 is a diagram for explaining the all-around bird's-eye view image displayed on the display device 14 when entering the garage. FIG. 4 shows an example in which the vehicle 100 is moved backward while turning into the parking space 70. Even in the garage entry mode, an axis parallel to the longitudinal direction of the parking space 70 is set as a reference axis for obtaining the background rotation angle θ.

  Fig.4 (a) is a figure which shows the state before putting in a garage. When starting the garage entry, the driver enters the garage entry mode by operating the operation unit 50. Since the orientation of the background including the parking space 70 in the all-around bird's-eye view image 60 displayed on the display device 14 at this time is fixed during the garage entry mode, the driver adjusts the orientation of the background so that it can be easily seen. Good. As shown in FIG. 4A, when the vehicle 100 is stopped parallel to the short direction of the parking space 70, the parking space 70 in the all-around bird's-eye view image 60 displayed on the display device 14 is: The longitudinal direction appears horizontally.

  The driver presses the rotation button until the longitudinal direction of the parking space 70 in the all-around bird's-eye view image 60 becomes vertical (more specifically, the entrance of the parking space 70 appears above). The all-around bird's-eye view image 60 can be rotated. When the driver rotates the all-around bird's-eye view image 60 to a desired direction, the driver determines the direction of the background as a reference by pressing the determination button.

  When a frame line is drawn in the parking space 70, the motion detection unit 42 can detect the frame line from the all-around bird's-eye view image generated by the image generation unit 30 at the start of the garage entry mode. The control unit 40 controls the image generation unit 30 so as to rotate the all-around bird's-eye view image so that the short straight line of the frame line is horizontal and the long straight line is vertical. The control unit 40 determines the background orientation of the all-round bird's eye view image after the rotation as a reference background orientation. The motion detector 42 can specify the reference axis from the frame line.

  When no frame line is drawn in the parking space 70, the motion detection unit 42 can specify the reference axis as follows. That is, as shown in FIG. 4A, the driver presses the garage entry mode button or the like in a state where the vehicle 100 is stopped parallel to the short direction of the parking space 70. The operation unit 50 notifies the motion detection unit 42 of the timing via the control unit 40. The motion detection unit 42 specifies a horizontal line in the all-around bird's-eye view image generated at the timing as the reference axis. At this timing, the horizontal line is parallel to the longitudinal direction of the parking space 70.

  FIG.4 (b) is a figure which shows the state in the middle of putting in the garage which concerns on a prior art. In this example, the driver advances the vehicle 100 to a predetermined position on the right front side of the road while turning the vehicle 100 counterclockwise, and turns the vehicle 100 counterclockwise until it is substantially parallel to the parking space 70. Retreat, and when it becomes almost parallel, retreat straight.

  In the driving support device according to the related art, since the orientation of the vehicle 100 in the all-round bird's-eye view image 62 displayed on the display device is fixed, when the vehicle 100 is rotated counterclockwise by the rotation angle θ, the entire circumference The background including the parking space 70 in the bird's-eye view image 62 is rotated clockwise by the rotation angle θ.

  FIG.4 (c) is a figure which shows the state in the middle of the garage putting concerning embodiment. In the driving support device 10 according to the embodiment, when the vehicle 100 is rotated counterclockwise by the rotation angle θ, the background including the parking space 70 in the all-around bird's-eye view image generated by the image generation unit 30 is The rotation angle θ is the clockwise rotation (the same state as in FIG. 4B). Under the control of the control unit 40, the image generation unit 30 rotates the all-around bird's-eye view image counterclockwise by the rotation angle θ. Therefore, the all-around bird's-eye view image 60 displayed on the display device 14 has a composition in which the vehicle 100 is rotated counterclockwise by the rotation angle θ with respect to the reference axis.

  Next, a process of changing the composition of the all-around bird's-eye view image 60 displayed on the display device 14 during the garage exit mode and the garage entry mode will be described. The image generation unit 30 changes the composition of the all-around bird's-eye view image 60 displayed on the display device 14 according to the positional relationship between the vehicle 100 and the parking space 70. This composition change can be performed by an instruction from the control unit 40.

  For example, when the displacement of the vehicle 100 from the reference position exceeds a predetermined value, the image generation unit 30 generates a bird's eye view image that preferentially shows a region located in the traveling direction of the vehicle 100. Here, the reference position of the vehicle 100 may be a position when entering the garage exit mode or the garage entry mode, or may be a position when the vehicle 100 starts to move backward. The displacement may be the rotation angle θ or a movement amount from the reference position. As described above, the motion detection unit 42 can identify the rotation angle θ and the movement amount of the vehicle 100 by detecting the optical flow.

  The predetermined value is the relationship between the positional relationship (more specifically, the distance) between the vehicle 100 and the parking space 70 and the displacement of the vehicle 100 from the reference position, and various positional relationships between the vehicle 100 and the parking space 70. Is set based on the part of the vehicle 100 where the driver should pay more attention. The designer can determine the predetermined value by experiment or simulation. For example, it may be 45 °.

  Further, the image generation unit 30 generates bird's-eye view images having different viewpoint heights according to the displacement of the vehicle 100 from the reference position. For example, in the garage taking out mode in reverse, until the displacement exceeds the predetermined value, a full-view bird's-eye view image of the lower viewpoint among the two viewpoints having different heights is generated, and the displacement After the predetermined value is exceeded, an all-around bird's-eye view image with a higher viewpoint is generated. In the garage entry mode in reverse, until the displacement exceeds the predetermined value, an all-round bird's-eye view image of the higher viewpoint of the two viewpoints having different heights is generated, and the displacement is After exceeding the predetermined value, the all-round bird's-eye view image of the lower viewpoint is generated.

  The control unit 40 controls the image generation unit 30 to change the composition of the all-around bird's-eye view image 60 displayed on the display device 14 based on the displacement from the reference position of the vehicle 100 specified by the motion detection unit 42. To do. Hereinafter, two methods for changing the composition of the all-around bird's-eye view image 60 displayed on the display device 14 will be described.

  The first method is a method in which the control unit 40 changes a range to be displayed on the display device 14 in the all-around bird's-eye view image generated by the image generation unit 30. In this method, the image generation unit 30 generates an all-around bird's-eye view image larger than the display area of the display device 14. The control unit 40 adaptively determines a region to be actually displayed on the display device 14 in the all-around bird's-eye view image generated by the image generation unit 30 according to the displacement.

  For example, in the garage exit mode in reverse, until the displacement exceeds the predetermined value, the control unit 40 selects an area in which the area located in front of the vehicle 100 is preferentially captured in the all-around bird's-eye view image. After the displacement exceeds the predetermined value, control is performed so that a region where the region located behind the vehicle 100 is preferentially captured is employed. Further, in the garage unloading mode in reverse, until the displacement exceeds the predetermined value, the control unit 40 selects an image of a preset size located in the center area of the all-around bird's-eye view image. The image generation unit 30 is controlled to cut out. The control unit 40 controls the image generation unit 30 to enlarge the clipped image to the size of the all-around bird's-eye view image and display the enlarged image on the display device 14. After the displacement exceeds the predetermined value, the control unit 40 controls the image generation unit 30 to display the all-around bird's-eye view image on the display device 14.

  Further, in the garage entry mode in reverse, until the displacement exceeds the predetermined value, the control unit 40 selects an area in which the area located behind the vehicle 100 is preferentially reflected in the all-around bird's-eye view image. After the displacement exceeds the predetermined value, control is performed so as to adopt a region in which a region located in front of the vehicle 100 is preferentially captured. In the garage entry mode in reverse, the control unit 40 controls the image generation unit 30 to display the all-around bird's-eye view image on the display device 14 until the displacement exceeds the predetermined value. After the displacement exceeds the predetermined value, the control unit 40 controls the image generation unit 30 to cut out an image of a preset size located in the central region of the all-around bird's-eye view image. . The control unit 40 controls the image generation unit 30 to enlarge the clipped image to the size of the all-around bird's-eye view image and display the enlarged image on the display device 14.

  The second method is a method of changing the position of the virtual viewpoint when the image captured by the imaging device 12 is converted into a bird's eye view image. In order to simplify the description, the composition of the all-around bird's-eye view image displayed on the display device 14 includes a first example in which there are two patterns with importance on the front direction and importance on the rear direction, and two patterns with a low viewpoint and a high viewpoint. A second example will be described.

  In the first example, the conversion table storage unit 24 holds in advance conversion formulas corresponding to two patterns of importance on the forward direction and importance on the backward direction. For example, a lookup table for coordinate conversion of two patterns is held. The control unit 40 instructs the image conversion unit 32 to use which conversion formula depending on whether or not the displacement exceeds the predetermined value.

  For example, in the garage removal mode in reverse, until the displacement exceeds the predetermined value, the control unit 40 converts each image captured by the imaging device 12 with the virtual viewpoint set in front of the reference position. Each of the equations is used to control to generate a plurality of bird's-eye view images, and after the displacement exceeds the predetermined value, a plurality of conversion equations in which the virtual viewpoint is set behind the reference position are used, respectively. Control to generate a bird's eye view image.

  In the garage entry mode in reverse, until the displacement exceeds the predetermined value, the control unit 40 converts each image captured by the imaging device 12 into a conversion formula in which the virtual viewpoint is set behind the reference position. Each is used to control generation of a plurality of bird's-eye view images, and after the displacement exceeds the predetermined value, a plurality of bird's-eye view images are respectively used by using conversion equations in which a virtual viewpoint is set ahead of the reference position. Control to generate

  In the second example, the conversion table storage unit 24 holds in advance conversion expressions corresponding to two patterns with a low viewpoint and a high viewpoint. For example, a lookup table for coordinate conversion of two patterns is held. The control unit 40 instructs the image conversion unit 32 to use which conversion formula depending on whether or not the displacement exceeds the predetermined value.

  For example, in the garage exit mode in reverse, until the displacement exceeds the predetermined value, the control unit 40 sets each image captured by the imaging device 12 to the height of the virtual viewpoint as the reference position. Using a conversion formula or a conversion formula set lower than the reference position, control is performed to generate a plurality of bird's-eye view images, and after the displacement exceeds the predetermined value, the height of the virtual viewpoint is higher than the reference position. Control is performed so as to generate a plurality of bird's-eye view images using conversion formulas set high.

  In the garage entry mode in reverse, until the displacement exceeds the predetermined value, the control unit 40 converts each image captured by the imaging device 12 with the virtual viewpoint height set lower than the reference position. Each of the equations is used to control to generate a plurality of bird's-eye view images, and after the displacement exceeds the predetermined value, the height of the virtual viewpoint is set higher than the conversion equation or the reference position set as the reference position. Control is performed to generate a plurality of bird's-eye view images using the converted equations.

  Here, the case where the composition pattern of the all-around bird's-eye view image displayed on the display device 14 is two has been described. However, by setting the predetermined value in multiple stages, the composition pattern is made three or more. It is readily understood by those skilled in the art that this can be done. Note that the image conversion unit 32 may correct and use the reference conversion formula according to the movement of the virtual viewpoint.

  FIG. 5 is a diagram illustrating a virtual viewpoint 80 that serves as a reference for the vehicle 100. In FIG. 5, the left side of the vehicle 100 is the front (the same applies to FIGS. 6 to 8). This virtual viewpoint 80 has a predetermined height H from the ground, and is set at the center position of the vehicle 100 when the vehicle 100 is viewed from the bird's-eye view.

  FIG. 6 is a diagram illustrating an example of the virtual viewpoint 80 set in the garage removal mode in reverse. 6A shows the position of the virtual viewpoint 80 when the displacement from the reference position does not exceed the predetermined value, and FIG. 6B shows the position when the displacement from the reference position exceeds the predetermined value. The position of the virtual viewpoint 80 is shown. In this example, during the entire process of taking out the garage, it is possible to sufficiently check whether the left front part or the right front part of the vehicle 100 is not in contact with the adjacent vehicle by turning the vehicle 100 in the first half. It can be sufficiently confirmed whether the left rear portion or the right rear portion of the vehicle 100 does not contact the vehicle in the parking space at the position facing the parking space where the vehicle is parked.

  FIG. 7 is a diagram illustrating an example of the virtual viewpoint 80 set in the garage entry mode in reverse. FIG. 7A shows the position of the virtual viewpoint 80 when the displacement from the reference position does not exceed the predetermined value, and FIG. 7B shows the position when the displacement from the reference position exceeds the predetermined value. The position of the virtual viewpoint 80 is shown. In this example, in the first half of the garage entry, due to the turning of the vehicle 100, the left front part or the right front part of the vehicle 100 does not contact the vehicle in the parking space at the position facing the parking space to be parked. In the second half, it can be sufficiently confirmed whether the left rear part or the right rear part of the vehicle 100 does not contact the adjacent vehicle.

  FIG. 8 is a diagram illustrating another example of the virtual viewpoint 80 set in the garage removal mode in reverse. FIG. 8A shows the position of the virtual viewpoint 80 when the displacement from the reference position does not exceed the predetermined value, and FIG. 8B shows the position when the displacement from the reference position exceeds the predetermined value. The position of the virtual viewpoint 80 is shown. In this example, in the entire process of taking out the garage, by setting the height of the virtual viewpoint 80 relatively low in the first half, the area close to the vehicle 100 can be confirmed in detail, and in the second half, the vehicle It is possible to confirm safety around 100. In the garage entry mode in reverse, the relationship between FIG. 8 (a) and FIG. 8 (b) is reversed.

  FIG. 9 is a flowchart illustrating an operation example of the driving support apparatus 10 according to the embodiment. This operation example is an operation in the garage exit mode in reverse. First, the garage removal mode starts when the driver operates the operation unit 50, such as pressing the garage removal mode button (Y in S10).

  The image generation unit 30 generates an all-around bird's-eye view image (S12). The motion detection unit 42 sets the reference axis based on the all-around bird's-eye view image at the start of the garage removal mode. The motion detection unit 42 specifies the rotation angle θ from the reference axis in the current all-around bird's-eye view image generated by the image generation unit 30 (S14). The control unit 40 controls the image generation unit 30 to reversely rotate the bird's eye view image of the entire circumference by the rotation angle θ (S16).

  The control unit 40 compares the rotation angle θ with the set angle A (S18). While the rotation angle θ does not exceed the set angle A (Y in S18), the control unit 40 controls the image generation unit 30 to generate the all-around bird's-eye view image in which the virtual viewpoint is set in front of the vehicle 100 (S20). ). After the rotation angle θ exceeds the set angle A (N in S18), the control unit 40 controls the image generation unit 30 to generate an all-around bird's-eye view image with the virtual viewpoint set behind the vehicle 100 (S22). ).

  When the driver operates the operation unit 50 such as pressing the garage removal mode button again, the garage removal mode ends (Y in S24). While the garage removal mode does not end (N in S24), the process proceeds to step S12, and the garage removal mode continues.

  As described above, according to the embodiment, when the vehicle turns, by rotating the all-around bird's-eye view image in reverse so as to cancel the rotation of the background of the all-around bird's-eye view image generated by the image generation unit 30. A natural all-around bird's-eye view image with a fixed background orientation can be displayed. For example, an all-around bird's-eye view image in which the direction of the parking space is fixed in the vertical direction can be displayed. Such an all-around bird's-eye view image makes it easier for the driver to feel the positional relationship between the parking space and the vehicle, and enables more accurate steering operation.

  In the entire process of garage removal or garage entry, the composition of the bird's-eye view image displayed on the display device 14 is automatically changed according to the progress, which is more beneficial without increasing the burden on the driver. Information can be provided to the driver. In other words, the driver is more useful at each time point by preferentially displaying an area where the driver should pay more attention at each time point, or by displaying a bird's eye view image of a suitable range at each time point. Information can be obtained. In addition, since the composition is automatically changed, the driver does not need to perform an operation such as pressing a button, and the convenience and safety are high.

  Further, since the rotation of the background due to turning of the vehicle 100 can be specified by detecting the optical flow, it is not necessary to connect the driving support device 10 to the steering angle sensor, the vehicle speed sensor, etc., and the system is simplified. can do.

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

  In the above-described embodiment, the rotation angle θ is specified by integrating optical flows between frames. However, in order to reduce errors due to the integration, line segments on the vehicle 100 are detected at regular intervals. The rotation angle θ may be corrected by calculating the angle between the line segment and the frame line of the parking space 70.

  In the above-described embodiment, the example in which the axis parallel to the longitudinal direction of the parking space 70 is set as the reference axis for obtaining the background rotation angle θ has been described. In this respect, the driver operates the operation unit 50 (for example, by pressing the rotation button) at any time when leaving the garage and entering the garage, so that the axis other than the axis parallel to the longitudinal direction of the parking space 70 is the above. Can be set to the reference axis. For example, an image that is captured in a state where the longitudinal direction of the parking space 70 is rotated 45 ° or 90 ° clockwise or counterclockwise is set as the reference image, and the background direction in the reference image is set in the subsequent image. The orientation of the background can be fixed.

  When the position where the vehicle 100 is not horizontal or vertical with respect to the parking space 70 is set as the garage unloading start position or the garage entry start position, the driver operates the operation unit 50 in the longitudinal direction of the parking space 70. A parallel axis can be set as the reference axis. That is, the image may be rotated until the longitudinal direction of the parking space 70 becomes vertical, and the image is set as the reference image.

  The timing for setting these reference images is not limited to when starting garage removal or starting garage entry. For example, it may be in the middle of garage removal or garage entry. Therefore, it is possible to change the orientation of the background to be fixed during garage entry or garage entry.

It is a figure which shows the structure of the vehicle carrying the driving assistance device which concerns on embodiment of this invention. It is a figure which shows the structure of the driving assistance device which concerns on embodiment. It is a figure for demonstrating the all-around bird's-eye view image displayed on a display apparatus at the time of garage removal. Fig.3 (a) is a figure which shows the state parked forward before garage removal. FIG.3 (b) is a figure which shows the state in the middle of the garage removal based on a prior art. FIG.3 (c) is a figure which shows the state in the middle of the garage removal which concerns on embodiment. It is a figure for demonstrating the all-around bird's-eye view image displayed on a display apparatus at the time of garage entry. Fig.4 (a) is a figure which shows the state before putting in a garage. FIG.4 (b) is a figure which shows the state in the middle of putting in the garage which concerns on a prior art. FIG.4 (c) is a figure which shows the state in the middle of the garage putting concerning embodiment. It is a figure which shows the virtual viewpoint used as the reference | standard with respect to a vehicle. It is a figure which shows an example of the virtual viewpoint set in the garage removal mode in reverse. FIG. 6A shows the position of the virtual viewpoint when the displacement from the reference position does not exceed a predetermined value. FIG. 6B shows the position of the virtual viewpoint when the displacement from the reference position exceeds a predetermined value. It is a figure which shows an example of the virtual viewpoint set in the garage entry mode in reverse. FIG. 7A shows the position of the virtual viewpoint when the displacement from the reference position does not exceed a predetermined value. FIG. 7B shows the position of the virtual viewpoint when the displacement from the reference position exceeds a predetermined value. It is a figure which shows another example of the virtual viewpoint set in the garage removal mode in reverse. FIG. 8A shows the position of the virtual viewpoint when the displacement from the reference position does not exceed a predetermined value. FIG. 8B shows the position of the virtual viewpoint when the displacement from the reference position exceeds a predetermined value. It is a flowchart which shows the operation example of the driving assistance device which concerns on embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Driving assistance apparatus, 12 Imaging apparatus, 14 Display apparatus, 20 Frame buffer, 24 Conversion table memory | storage part, 30 Image generation part, 32 Image conversion part, 34 Image composition part, 40 Control part, 42 Motion detection part, 50 Operation part 100 vehicles.

Claims (5)

An image generation unit that generates a bird's-eye view image around the vehicle based on an image captured by at least one imaging device installed in the vehicle;
A display unit for displaying the bird's eye view image generated in the image generation unit,
The said image generation part produces | generates the bird's-eye view image with which the direction of the parking space was fixed, when the relative relationship of the direction of a vehicle and the direction of a parking space changes. A motion detection unit that detects an angle of rotation of the parking space in the bird's-eye view image generated in the image generation unit by turning the vehicle;
A control unit that controls the image generation unit to output to the display unit after reversely rotating the bird's eye view image based on the angle detected in the motion detection unit;
The driving support device according to claim 1, further comprising:   The driving support device according to claim 1, wherein the image generation unit changes a composition of a bird's eye view image displayed on the display unit according to a positional relationship between a vehicle and a parking space.   The said image generation part produces | generates the bird's-eye view image which preferentially reflected the area | region located in the advancing direction of a vehicle, when the displacement from the reference position of a vehicle exceeds a predetermined value. The driving assistance apparatus as described.   The driving support device according to claim 3, wherein the image generation unit generates bird's-eye view images having different viewpoint heights according to displacement from a reference position of the vehicle.

Images