JP2010221863A - Vehicle surrounding monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle surrounding monitoring device capable of efficiently detecting an obstacle in an entire peripheral bird's eye view image. <P>SOLUTION: A first image pickup device 2a to a fourth image pickup device 2d provided to a vehicle store a picked-up image in a frame buffer 10. An image reading part 12 supplies each image read from the frame buffer 10 to an image processing part 14, and supplies the image picked up by the second image pickup device 2b and the fourth image pickup device 2d to a moving amount detection part 16. The image processing part 14 converts each image received from the image reading part 12 into a bird's eye view image, converts the bird's eye view image to cancel the moving amount of the vehicle based on a moving vector of the entire vehicle received from the moving amount detection part 16, and supplies it to an obstacle detection part 18. The moving amount detection part 16 detects the moving vector of the entire vehicle based on the image received from the image reading part 12. The obstacle detection part 18 detects an obstacle for each image pickup device 2 by taking a difference of the bird's eye view images received from the image processing part 16. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring technique, and more particularly to a vehicle periphery monitoring device that monitors the periphery of a vehicle based on an image captured by an imaging device installed in the vehicle.

  A monitoring device has been developed that installs a camera in a vehicle and displays a captured image of a surrounding situation that is likely to be a blind spot of a driver on a screen such as a car navigation system.

  In addition to simply displaying captured images, multiple cameras are installed around the vehicle, each captured image is converted into a bird's-eye view image, and these bird's-eye view images are combined to create the entire periphery of the vehicle. Some display a bird's-eye view image of the entire circumference that allows the bird's-eye view to be viewed (for example, Patent Document 1).

  Furthermore, one camera is installed in the vehicle, and two captured images with a time difference captured are converted into two bird's-eye view images belonging to the same bird's-eye view coordinate system, and by taking these differences, Some detect an obstacle (for example, Patent Document 2).

JP 2007-269060 A JP 2006-268076 A

  When two captured images with a time difference are converted to the same bird's-eye view coordinate system, the movement amount of the vehicle between the two captured images is calculated, and one bird's-eye view image is converted to the other bird's-eye view image based on this movement amount. Is transformed to the same bird's eye view image system. At this time, in order to calculate the movement amount of the vehicle from the captured image, the feature points (for example, the edge of the white line on the road surface) included in the captured image are tracked by the gradient method or the block matching method. The calculation for tracking the feature points is heavy, and in order to detect obstacles in the all-around bird's-eye view image, tracking feature points for each bird's-eye view image constituting the all-around bird's-eye view image requires enormous processing time. Efficient obstacle detection was difficult.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a vehicle surrounding monitoring apparatus that can efficiently detect an obstacle in a bird's eye view image of the entire circumference.

  One embodiment of the present invention is a vehicle surrounding monitoring apparatus. This vehicle surrounding monitoring device captures images for each of N (N is a natural number of 2 or more) imaging devices installed in the vehicle, and generates first and second bird's-eye view images from two images having a time difference. An image generation unit that performs image selection, an image selection unit that selects N−1 or less images among N images captured by the N imaging devices, and a feature included in the N−1 or less images A movement amount detection unit that detects a movement amount of the vehicle by tracking a point, and a coordinate conversion unit that converts the coordinates of the second bird's-eye view image based on the movement amount to generate a third bird's-eye view image And an obstacle detection unit that detects an obstacle by taking a difference between the first bird's-eye view image and the third bird's-eye view image.

  Moreover, it is preferable that the said image selection part selects the image which imaged the right direction and the left direction of the vehicle from the said N images.

  Further, it is preferable that the image selection unit selects the N−1 or less images in descending order of image quality from the N images.

  In addition, it is preferable that the image selection unit selects the N−1 or less images from the N images in the order in which many feature points are included.

  Moreover, it is preferable that the said image selection part selects the said N-1 image or less from the said N images in order with a large depression angle with respect to the horizontal surface of a corresponding imaging device.

  Moreover, it is preferable that the said image selection part selects the said N-1 image or less from the said N images in order with many white lines on a road surface.

  According to the present invention, obstacle detection in an all-around bird's-eye view image can be performed efficiently.

The conceptual diagram which shows the structure of the vehicle 1 carrying the vehicle periphery monitoring apparatus 3 in embodiment of this invention. All-round bird's-eye view image displayed on the display device The conceptual diagram which shows the structure of the vehicle periphery monitoring apparatus 3 in embodiment of this invention Flow chart showing the procedure for detecting an obstacle from a bird's eye view image

  The outline will be described before the present invention is specifically described. Embodiments described herein relate generally to a vehicle surrounding monitoring apparatus that detects an obstacle in an all-around bird's-eye view image. The vehicle periphery monitoring device acquires captured images from a total of four in-vehicle cameras installed at the front, rear, left, and right sides of the vehicle, and synthesizes a bird's eye view image of the entire circumference.

  In addition, the vehicle surrounding monitoring apparatus converts each of the two captured images having a time difference captured for each in-vehicle camera into two bird's-eye view images belonging to the same bird's-eye view coordinate system. And the obstacle is detected for every vehicle-mounted camera by taking these differences. Furthermore, the detected obstacle is displayed on the all-around bird's-eye view image to alert the driver.

  At this time, in order to convert two captured images with a time difference into the same bird's eye view coordinate system, the feature points included in the captured images are tracked to detect the amount of movement of the vehicle. The calculation of feature point tracking is heavy, and if feature points are tracked for each in-vehicle camera, the calculation load increases according to the number of in-vehicle cameras.

  Here, since the moving amount of the vehicle is common to all installed in-vehicle cameras, for example, if the moving amount of the vehicle can be detected by tracking the feature points included in the captured image captured by a certain in-vehicle camera, the remaining in-vehicle cameras There is no need to track feature points with a camera.

  In general, in-vehicle cameras at the front and rear of the vehicle are installed so that they can image far away from the vehicle (with a small depression angle relative to the horizontal plane), and the in-vehicle cameras on the left and right of the vehicle image near the vehicle ( Installed at a large depression angle with respect to the horizontal plane). For this reason, the captured images of the left and right vehicle-mounted cameras have better image quality than the captured images of the vehicle-mounted cameras before and after the vehicle, and tracking the feature points with the captured images of the left and right vehicle-mounted cameras often improves the accuracy. .

  Therefore, in the vehicle surrounding monitoring apparatus according to the embodiment of the present invention, a captured image suitable for detecting the amount of movement of the vehicle is selected from captured images captured by a plurality of in-vehicle cameras installed in the vehicle. Then, feature points are tracked for the selected captured image.

  As a result, it is possible to convert captured images with a time difference taken for each in-vehicle camera into the same bird's-eye view coordinate system without tracking feature points for all in-vehicle cameras installed in the vehicle. Therefore, the obstacle detection in the all-around bird's-eye view image can be performed efficiently.

  FIG. 1 is a conceptual diagram showing a configuration of a vehicle 1 in an embodiment of the present invention. The vehicle 1 includes a first imaging device 2 a, a second imaging device 2 b, a third imaging device 2 c, a fourth imaging device 2 d, a vehicle surrounding monitoring device 3, and a display device 4 that are collectively referred to as the imaging device 2. The vehicle 1 includes components for realizing the original operation of the vehicle 1 such as an engine, a chassis, a handle, a brake, etc., but is omitted in FIG. 1 for simplification of the drawing. To do.

  The imaging device 2 continuously captures images and supplies the captured images to the vehicle surrounding monitoring device 3. The first imaging device 2a is installed in the front part of the vehicle, and the second imaging device 2b is installed in the right part of the vehicle. The third imaging device 2c is installed in the rear part of the vehicle, and the fourth imaging device 2d is installed in the left part of the vehicle. The number of imaging devices 2 is not limited to “4”. The vehicle surroundings monitoring device 3 generates an all-round bird's-eye view image by converting the image received from the imaging device 2 into a bird's-eye view image and further combining the bird's-eye view images corresponding to each of the plurality of imaging devices 2.

  The vehicle surroundings monitoring device 3 is configured to detect the entire vehicle based on an image received from the second imaging device 2b (hereinafter also referred to as a right image) and an image received from the fourth imaging device 2d (hereinafter also referred to as a left image). Detect motion vectors.

  In addition, the vehicle surroundings monitoring device 3 includes a bird's-eye view image (hereinafter also referred to as a first bird's-eye view image) at a certain point in time and a predetermined period (for example, 1) from the bird's-eye view image corresponding to each of the imaging devices 2. A bird's-eye view image (hereinafter also referred to as a second bird's-eye view image) at a time point away from the frame period) is extracted.

  When each bird's-eye view image generated from two images with different imaging points is converted to the same bird's-eye view coordinate system, they match in the planar area of the imaging target, but they do not match in the area where the imaging target is high Produce.

  Here, the coordinate system of the second bird's-eye view image is shifted from the coordinate system of the first bird's-eye view image by the amount of movement of the vehicle that has occurred within a predetermined period. Therefore, the vehicle periphery monitoring device 3 calculates the amount of movement of the vehicle that has occurred within a predetermined period from the motion vector of the entire vehicle, and performs coordinate conversion for canceling this amount of movement on the second bird's-eye view image. Then, by taking the difference between the first bird's-eye view image and the bird's-eye view image generated by performing coordinate conversion on the second bird's-eye view image (hereinafter also referred to as the third bird's-eye view image), there is a height of the imaging target. An area, that is, an obstacle is detected.

  The vehicle surrounding monitoring device 3 supplies the display device 4 with a special symbol (for example, exclamation) or blinks the obstacle so that the detected obstacle can be identified in the all-around bird's-eye view image.

  The display device 4 is a liquid crystal monitor, for example, and displays the all-around bird's-eye view image received from the vehicle surroundings monitoring device 3 as shown in FIG. In FIG. 2, a vehicle is displayed at the center of the screen, a bird's-eye view image corresponding to the imaging device 2a above the vehicle, a bird's-eye view image corresponding to the right imaging device 2b, a bird's-eye view image corresponding to the lower imaging image 2c, An all-around bird's-eye view image composed of bird's-eye view images corresponding to the captured image 2d is displayed. In addition, since the detected obstacles are displayed in an identifiable manner on the all-around bird's-eye view image, it is easy for the driver to feel the situation around the vehicle, and an accurate steering operation is possible.

  FIG. 3 is a conceptual diagram showing a configuration of the vehicle periphery monitoring device 3 according to the embodiment of the present invention. The vehicle surrounding monitoring apparatus 3 includes a first frame buffer 10a, a second frame buffer 10b, a third frame buffer 10c, a fourth frame buffer 10d, an image reading unit 12, an image processing unit 14, A detection unit 16, an obstacle detection unit 18, and a control unit 20 are included.

  The frame buffer 10 is arranged in association with each imaging device 2 (not shown). That is, the first frame buffer 10a to the fourth frame buffer 10d are associated with the first imaging device 2a to the fourth imaging device 2d, respectively. The frame buffer 10 stores the image received from the imaging device 2.

  The image reading unit 12 reads the images stored in the first frame buffer 10a, the second frame buffer 10b, the third frame buffer 10c, and the fourth frame buffer 10d, and supplies each of them to the image processing unit 14. The image reading unit 14 also selects an image instructed by the control unit 20 from among the images stored in the first frame buffer 10a, the second frame buffer 10b, the third frame buffer 10c, and the fourth frame buffer 10d. Read and supply to the movement amount detector 16. In the embodiment of the present invention, the motion vector of the entire vehicle is detected based on the right image and the left image. Therefore, the image reading unit 12 that has received an instruction to read the right image and the left image from the control unit 20 reads the images from the second frame buffer 10b and the fourth frame buffer 10d, and the movement amount detection unit 16 To supply.

  The image processing unit 14 converts each image received from the image reading unit 12 into a bird's eye view image. Then, an all-around bird's-eye view image is generated by combining the bird's-eye view images. The bird's-eye view image is converted to cancel the amount of vehicle movement, and is supplied to the obstacle detection unit 16. Further, the position information of the obstacle received from the obstacle detection unit 16 is referred to and displayed in a manner that the driver can recognize on the all-around bird's-eye view image.

  The image processing unit 14 includes an image conversion unit 22, an image composition unit 24, and a conversion table storage unit 26.

  The image conversion unit 22 receives an image for each frame buffer 10 from the image reading unit 12, and converts each image into a bird's-eye view image by referring to the conversion formula stored in the conversion table storage unit 26. That is, the bird's eye view image is associated with each imaging device 2 that has captured the image before conversion. The conversion into the bird's-eye view image may be performed by using a known technique, for example, the technique disclosed in Japanese Patent Application Laid-Open No. 2007-269060, and the description thereof is omitted here.

  The image conversion unit 22 extracts a set of bird's-eye view images (first bird's-eye view image and second bird's-eye view image) to be processed for obstacle detection from the associated bird's-eye view images for each imaging device 2. . If the first bird's eye view image is discretely extracted, the load of the obstacle detection process can be reduced. On the other hand, if the first bird's eye view images are continuously extracted, obstacles can be detected in real time.

  The image conversion unit 22 performs conversion for canceling the movement amount of the vehicle with respect to the second bird's-eye view image by referring to the conversion table storage unit 26 based on the motion vector of the entire vehicle received from the movement amount detection unit 16. . Thereby, the 3rd bird's-eye view image which shifted the virtual viewpoint in the direction opposite to movement of vehicles is generated. In other words, if the motion vector of the entire vehicle is a translational motion vector, the image conversion unit 22 performs a conversion for translating the virtual viewpoint in the opposite direction to the second bird's-eye view image. Further, if the motion vector of the entire vehicle is a motion vector due to a rotational motion, conversion is performed to rotate and move the virtual viewpoint in the opposite direction with respect to the second bird's-eye view image. The image conversion unit 22 supplies the first bird's-eye view image and the third bird's-eye view image to the obstacle detection unit 18.

  The image synthesis unit 24 synthesizes the bird's-eye view image associated with each imaging device 2 by referring to the conversion table 26 to generate an all-round bird's-eye view image. Note that the technique disclosed in Japanese Patent Application Laid-Open No. 2007-269060 may be used for synthesizing the all-around bird's-eye view image.

  Further, the image composition unit 24 receives the position information of the obstacle on the bird's eye view image from the obstacle detection unit 18. By referring to this position information, the pixel data around the corresponding position on the all-around bird's-eye view image is supplied to the display device 4 by blinking display or highlight display.

  The conversion table storage unit 26 stores a conversion formula used when converting an image into an all-around bird's-eye view image. In addition, a conversion formula for converting the second bird's-eye view image into parallel translation or rotational conversion is also stored.

  The movement amount detection unit 16 detects the motion vector of the entire vehicle based on the right side image and the left side image received from the image reading unit 12. The movement amount detection unit 16 includes a motion detection unit 28 and a motion estimation unit 30.

  The motion detection unit 28 detects a global motion vector (hereinafter also referred to as a global motion vector) in each of the right image and the left image received from the image reading unit 12. Therefore, the motion vector detection unit 28 extracts feature points in the image. For example, the KLT (Kanade-Lucas-Tomasi) method is used for extracting feature points. The KLT method is, for example, C.I. Tomasi and T. Kanade, “Detection and Tracking of Point Features,” Carnegie Mellon University Technical Report CMU-CS-91-132, April 1991. Therefore, the description thereof is omitted here.

  The motion detection unit 28 tracks the detected feature points in the time axis direction for each of the right image and the left image received from the image reading unit 12. For this reason, when detecting the feature point in the image at time T, the motion detection unit 28 detects the corresponding point in the image at time T + 1 by the block matching method.

  Specifically, a unit block including a feature point in the image at time T (for example, a square of 8 pixels in length × 8 pixels in width) is used as a template block, and a predetermined search area (for example, 16 units in length) on the image at time T + 1. Block × horizontal 16 unit block) is set. The sum of absolute differences of pixel values is calculated between the template block in the search area and the unit block candidates predicted to include corresponding points. The motion detection unit 28 predicts that the corresponding point is included in the unit block having the minimum value (hereinafter also referred to as a prediction unit block). Then, a vector indicating the motion from the unit block at the position corresponding to the template block on the image at time T + 1 to the prediction unit block is detected as a motion vector of the feature point (hereinafter also referred to as a local motion vector).

  The motion detection unit 28 detects a plurality of feature points in the image at the time T, and detects a plurality of local motion vectors corresponding to the feature points. Statistical processing is performed on a plurality of local motion vectors (for example, processing for averaging local motion vectors included in a predetermined error range), global motion vectors are detected for each of the right image and the left image, and motion is detected. It supplies to the estimation part 30.

  The motion estimation unit 30 receives the global motion vector detected from the right image (hereinafter also referred to as the right global motion vector) and the global motion vector detected from the left image (hereinafter referred to as the left) received from the motion detection unit 28. The motion vector of the entire vehicle is detected from the global motion vector.

  Here, when the vehicle is in translation (for example, when traveling straight), the right global motion vector and the left global motion vector have the same magnitude. On the other hand, when the vehicle is rotating (for example, when turning right), the magnitudes of the right global motion vector and the left global motion vector are different.

  Therefore, the motion estimation unit 30 compares the right global motion vector and the left global motion vector to determine whether the vehicle is performing translational motion or rotational motion. When it is determined that the vehicle is moving in translation, one of the right global motion vector and the left global motion vector, or an average of both is supplied to the image processing unit 14 as the motion vector of the entire vehicle.

  When the entire vehicle is rotating, the magnitude of the right global motion vector is proportional to the product of the distance to the second imaging device 2b and the amount of rotation, and the magnitude of the left global motion vector is the center of rotation. It is proportional to the product of the distance from the point to the fourth imaging device 2d and the rotation amount. Moreover, the relative distance between the second imaging device 2b and the fourth imaging device 2d is known. Therefore, if the magnitude of the left global motion vector and the magnitude of the right global motion vector are known, the amount of rotation can be calculated. Therefore, when the motion estimation unit 30 determines that the vehicle is rotating, the motion estimation unit 30 calculates a rotation amount from the magnitude of the right global motion vector and the magnitude of the left global motion vector, and calculates the rotation amount and the rotation direction. The motion vector of the entire vehicle is supplied to the image processing unit 14.

  The obstacle detection unit 18 calculates a difference between the first bird's-eye view image and the third bird's-eye view image received from the image processing unit 14. As described above, in the region where the imaging target is high, the difference results are inconsistent. Therefore, the obstacle detection unit 18 detects the obstacle for each imaging device 2 by taking the difference between the first bird's-eye view image and the third bird's-eye view image received from the image processing unit 14, and the position of the obstacle on the bird's-eye view image Information is supplied to the image processing unit 14. In addition, the obstacle detection unit 18 measures the coordinate position on the first bird's-eye view image of the area where the obstacle is detected, and supplies this to the image processing unit 14 as position information of the obstacle on the bird's-eye view image.

  The operation of the vehicle periphery monitoring device 3 having the above configuration will be described. FIG. 4 is a flowchart showing a procedure for detecting an obstacle from a bird's eye view image.

  The motion detection unit 28 detects a right global motion vector from the right image received from the image reading unit 12, and detects a left global motion vector from the left image (S10). If the right global motion vector and the left global motion vector are not detected (N in S12), the motion detection unit 28 determines that the vehicle is stopped and ends the process.

  On the other hand, when the right global motion vector and the left global motion vector are detected (Y in S12), the motion estimation unit 30 compares the size of the right global motion vector with the size of the left global motion vector ( S14).

  When the magnitude of the right global motion vector is equal to the magnitude of the left global motion vector (Y in S14), the motion estimation unit 30 determines that the entire vehicle is performing translational motion, for example, the right global motion The vector is supplied to the image processing unit 14 as a motion vector for the entire vehicle. Furthermore, the motion estimation unit 30 instructs the image processing unit 14 to perform conversion for translating the virtual viewpoint of the second bird's eye view image in the direction opposite to the vehicle motion. Based on the instruction from the motion estimation unit 30, the image conversion unit 22 generates a third bird's-eye view image in which the virtual viewpoint of the second bird's-eye view image is shifted back and forth, and supplies the third bird's-eye view image to the obstacle detection unit 18 (S16).

  When the magnitude of the right global motion vector is different from the magnitude of the left global motion vector (N in S14), the motion estimation unit 30 determines that the entire vehicle is performing rotational motion, and determines the right global motion vector. The amount of rotation is calculated from the magnitude and the magnitude of the left global motion vector. The motion estimation unit 30 supplies the calculated rotation amount and rotation direction to the image processing unit 14 as a motion vector for the entire vehicle. Furthermore, the motion estimation unit 30 instructs the image processing unit 14 to perform conversion in which the virtual viewpoint of the second bird's-eye view image is rotated and moved in the direction opposite to the vehicle and motion. The image conversion unit 22 generates a third bird's-eye view image obtained by rotating the virtual viewpoint of the second bird's-eye view image based on an instruction from the motion estimation unit 30, and supplies the third bird's-eye view image to the obstacle detection unit 18 (S18).

  The obstacle detection unit 18 takes the difference between the first bird's eye view image and the third bird's eye view image received from the image processing unit 14 and detects an obstacle (S20).

  According to such an embodiment of the present invention, of the first imaging device 2a, the second imaging device 2b, the third imaging device 2c, and the fourth imaging device 2d installed in the vehicle, the second imaging device 2b and Since the feature points are tracked for the image captured by the fourth imaging device 2d, the calculation load of motion vector detection can be reduced.

  Further, since the motion vector is detected based on the right image of the vehicle imaged by the second imaging device 2b and the left image of the vehicle imaged by the fourth imaging device 2d, the feature points are compared with the front image and the rear image of the vehicle. Is easy to detect and the motion vector detection accuracy is improved. In addition, since feature points are tracked for high-quality images near the vehicle, motion vector detection accuracy is improved. In addition, when the vehicle is moving straight ahead, a larger motion vector can be detected than when detecting a motion vector for the front image and the rear image, so that the motion vector can be accurately detected for the entire vehicle. In addition, it is possible to accurately detect whether the vehicle is moving in translation or rotating. Therefore, it is possible to efficiently perform obstacle detection for the all-around bird's-eye view image.

  As mentioned above, although the form for implementing this invention has been demonstrated, this invention is not limited to the structure of this embodiment, It exists in the application range of this invention prescribed | regulated by the claim. Various modifications are possible as long as the functions of the configuration of the above-described embodiment can be achieved.

  For example, in the embodiment of the present invention, the movement amount detection unit 16 tracks feature points of images captured by the second imaging device 2b and the fourth imaging device 2d among the four imaging devices 2. However, feature points may be tracked only for one of them.

  More specifically, a plurality of feature points are tracked with respect to either one of images captured by the second imaging device 2b or the fourth imaging device 2d, and corresponding local motion vectors are detected. . Then, the motion vector of the entire vehicle is detected by determining whether or not the plurality of local motion vectors are within a predetermined error range. In order to respond to the rotational movement of the vehicle, information on the amount of rotation is acquired from a rotation angle sensor installed in the vehicle. Further, the rotation amount may be calculated by a known technique disclosed in Japanese Patent Laid-Open No. 2006-268076. According to this modification, the calculation load of motion vector detection can be further reduced.

  In the embodiment of the present invention, the control unit 20 supplies the movement amount detection unit 16 with images captured by the second imaging device 2b and the fourth imaging device 2d among the four imaging devices 2. Although the image reading unit 12 is controlled, the image reading unit 12 may be controlled so that the image having the highest resolution among the images captured by the four imaging devices 2 is supplied to the movement amount detection unit 16. According to this modification, since the feature points are tracked for a high-quality image, the motion vector detection accuracy is improved.

  Further, the control unit 20 may control the image reading unit 12 so as to supply the moving amount detection unit 16 with the image having the highest luminance among the images captured by the four imaging devices 2. According to this modification, since the feature points are tracked for a high-quality image, the motion vector detection accuracy is improved.

  Further, the control unit 20 may control the image reading unit 12 so as to supply the movement amount detection unit 16 with an image in which the most feature points can be detected among the images captured by the four imaging devices 2. According to this modification, since tracking is performed for more feature points, the motion vector detection accuracy is improved.

  Further, the control unit 20 controls the image reading unit 12 so as to supply the movement amount detection unit 16 with the image of the imaging device 2 having the largest depression angle with respect to the horizontal plane among the images captured by the four imaging devices 2. Also good. According to this modification, since the feature points are tracked for a high-quality image close to the vehicle, the motion vector detection accuracy is improved.

  In addition, the control unit 20 may control the image reading unit 12 so as to supply the movement amount detection unit 16 with an image including the most white lines on the road surface among images captured by the four imaging devices 2. . According to the present modification, the feature points can be easily tracked, so that the motion vector detection efficiency is improved.

  Moreover, the control part 20 may switch the selection mode of the imaging device 2 demonstrated above according to a driver | operator's instruction | indication. According to this modification, the convenience for the driver is improved.

10 frame buffer, 12 image reading unit, 14 image processing unit, 16 movement amount detection unit, 18 obstacle detection unit, 20 control unit, 22 image conversion unit, 24 image composition unit, 26 conversion table storage unit, 28 motion detection unit 30 Motion estimation unit.

Claims (6)

An image generation unit that generates images of the first and second bird's-eye view images from each of two images that are imaged for each of N (N is a natural number of 2 or more) imaging devices installed in the vehicle, and have a time difference;
An image selection unit that selects N-1 or less images among N images captured by the N imaging devices;
A movement amount detection unit for detecting a movement amount of the vehicle based on the N-1 or less images;
A coordinate conversion unit that generates a third bird's-eye view image from the second bird's-eye view image based on the amount of movement;
An obstacle detection unit that detects an obstacle by taking a difference between the first bird's-eye view image and the three bird's-eye view images;
A vehicle surroundings monitoring device comprising:   2. The vehicle surrounding monitoring apparatus according to claim 1, wherein the image selection unit selects an image obtained by imaging a right direction and a left direction of the vehicle from the N images.   2. The vehicle surrounding monitoring apparatus according to claim 1, wherein the image selection unit selects the N−1 or less images from the N images in descending order of image quality.   2. The vehicle surrounding monitoring apparatus according to claim 1, wherein the image selection unit selects the N−1 or less images in the order in which many feature points are included from the N images.   2. The vehicle surrounding monitoring apparatus according to claim 1, wherein the image selection unit selects, from the N images, the N−1 or less images in descending order of a depression angle with respect to a horizontal plane of a corresponding imaging device. . 2. The vehicle surrounding monitoring apparatus according to claim 1, wherein the image selection unit selects the N−1 or less images from the N images in the order in which many white lines on the road surface are included.

Images