JP2008141649A - Vehicle periphery monitoring apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle periphery monitoring apparatus capable of suppressing the display of unsuitable vehicle periphery monitoring images and capable of displaying a vehicle periphery monitoring image corresponding to variation in the posture of a camera which may be previously supposed according to the movement of a movable part. <P>SOLUTION: When a variation in the postures of cameras 3, 4, 5 installed on movable portions is previously supposed even if the movement of the movable portion is detected, vehicle periphery monitoring images 9, 10 generated by using a mapping table 20 corrected by using correction information corresponding to the moved quantity of the movable portions are displayed, and when variation in the posture of a camera 2 installed on a position other than the movable portions is detected or when variations in the postures of the cameras 3, 4, 5 installed on the movable portions are not previously supposed, the display of the vehicle periphery monitoring images 9, 10 is stopped or limited. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring device, and in particular, a vehicle periphery suitable for supporting safe and smooth driving operation of a vehicle by displaying a vehicle periphery monitoring image for monitoring the periphery of the vehicle on a display unit. It relates to a monitoring device.

  In recent years, a vehicle periphery monitoring device that generates a vehicle periphery monitoring image, which is an image for monitoring the periphery of the vehicle, and displays it on the display of the in-vehicle device is employed as a technique for improving the safety of driving operation of the vehicle. It became so.

  This type of vehicle periphery monitoring device is a vehicle periphery monitoring image based on a photographed image of a road surface or the like captured by a camera installed in the vehicle, and an overhead view image of the vehicle and its surroundings viewed from above the vehicle, A side view image for monitoring the side of the image is displayed on the display.

  By the way, about the camera installed in the movable parts of vehicles, such as a door mirror on the right and left of a vehicle, and a rear hatch, a posture (for example, height and direction with respect to a road surface) changes with movement of a movable part, and an appropriate vehicle In some cases, it may not be possible to capture a video image necessary for generating the peripheral monitoring image.

  Therefore, as a means for correcting such a problem, for example, a vehicle periphery monitoring device described in Patent Document 1 has been proposed so far. In the vehicle periphery monitoring device described in Patent Document 1, the movable part in which the camera is installed is a normal position of the movable part when the camera is photographed around the vehicle (for example, a position in a closed state in the case of a rear hatch). ), The display of the vehicle periphery monitoring image is stopped or restricted, and a warning message is displayed (see, for example, Patent Document 1 FIGS. 6 and 7).

Japanese Patent No. 3599639 Japanese Patent No. 3670647

  However, in the past, the focus was only on the movement of the movable part, so the attitude of the camera installed on the movable part fluctuated independently of the movable part, or the attitude of the camera installed on other than the movable part fluctuated In some cases, an inappropriate vehicle periphery monitoring image may be displayed.

  Furthermore, in the case of a camera installed on a movable part, the movement range of the movable part is determined in advance based on the design of the vehicle, so that it is possible to assume a change in the posture of the camera with the movement of the movable part. Nevertheless, conventionally, when the movable part moves, the display of the vehicle periphery monitoring image has to be stopped or restricted.

  As described above, the conventional vehicle periphery monitoring device still has insufficient points to support safe and smooth driving operation.

  Therefore, the present invention has been made in view of such a point, and copes with a change in posture independent of the movable portion of the camera installed in the movable portion and a change in posture of the camera installed in other than the movable portion. In addition, it is possible to suppress the display of an inappropriate vehicle periphery monitoring image, and even when the movable part moves, the vehicle periphery monitoring corresponding to a presumed change in the posture of the camera accompanying the movement of the movable part An object of the present invention is to provide a vehicle periphery monitoring device that can display an image and can sufficiently support a safe and smooth driving operation.

  In order to achieve the above-described object, a vehicle periphery monitoring device according to the present invention monitors a vehicle periphery monitoring image for monitoring the periphery of the vehicle based on a captured image of a camera that captures the periphery of the vehicle installed in the vehicle. And a mapping table storage unit for storing a mapping table indicating a correspondence relationship between the video captured by the camera and the vehicle periphery monitoring image, and the mapping table storage unit An image processing unit that generates the vehicle periphery monitoring image using the mapping table stored in the display unit, and displays the generated vehicle periphery monitoring image on the display unit; and a movable unit of the vehicle. Correction information used for correcting the mapping table corresponding to the camera, and when the camera is photographed around the vehicle. Correction information storage means for storing correction information having a correspondence relationship with the amount of movement of the movable part from the normal position of the movable part, and the movement state of the movable part from which the camera is installed from the normal position, The movable part movement detection means that detects the movement amount together with the movement amount, and the correction corresponding to the detected movement amount of the movable part when the movement state and the movement amount of the movable part are detected by the movable part movement detection means. Using the correction information stored by the information storage means, a mapping table correction means for correcting the mapping table and a camera in which a posture change from a normal posture at the time of photographing around the vehicle is detected. Posture changing camera detection means, and whether the camera detected by the posture changing camera detection means is a camera installed on the movable part. A camera type determination unit that performs the determination, and the camera type determination unit determines that the camera detected by the posture variation camera detection unit is a camera installed in the movable unit. A camera posture variation determining means for determining whether or not a camera installed in a section exhibits a posture change assumed in advance as the movable portion moves, and the image processing means includes the camera posture When the change determining means determines that the camera installed in the movable part shows a change in posture assumed in advance as the movable part moves, the correction by the mapping table correcting means The vehicle periphery monitoring image generated using a later mapping table is displayed on the display unit, and the posture variation camera detection unit is detected by the camera type determination unit. When it is determined that the camera detected by the stage is not a camera installed in the movable part, or the camera installed in the movable part is moved by the camera posture variation determining means. Along with this, it is configured to stop or limit the display of the vehicle periphery monitoring image on the display unit when it is determined that the posture change is not assumed in advance. It is said.

  And according to such a structure, even if the movement of the movable part in which the camera is installed is detected, a change in the posture of the camera installed in the movable part accompanying the movement of the movable part is assumed in advance. If it is, the mapping table is corrected using the correction information corresponding to the moving amount of the movable part, and the vehicle periphery monitoring image generated using the corrected mapping table can be displayed. In addition, when a change in the posture of a camera installed other than the movable unit is detected or when a change in the posture of the camera installed in the movable unit accompanying the movement of the movable unit is not assumed in advance, the vehicle It is possible to stop or limit the display of the peripheral monitoring image.

  Another vehicle periphery monitoring device according to the present invention displays a warning message on the display unit when the image processing unit stops or restricts display of the vehicle periphery monitoring image on the display unit. It is characterized by being formed.

  According to such a configuration, it is possible to notify the user that an unexpected posture change has occurred in the camera by displaying a warning message.

  Furthermore, in another vehicle periphery monitoring device according to the present invention, the vehicle periphery monitoring image is an overhead image obtained by looking down the vehicle and its periphery from above the vehicle, which is generated based on the captured images of the plurality of cameras. The moving part movement detection means is an overhead image generated by connecting a plurality of converted images corresponding to each camera obtained by converting each captured video of each camera using the mapping table. However, it is characterized in that the moving state of the movable part is detected based on the connection state of the converted images in the overhead image generated by the image processing means.

  And according to such a structure, it becomes possible to detect the movement state of a movable part based on the connection state of the some conversion image which forms a bird's-eye view image.

  Furthermore, in another vehicle periphery monitoring apparatus according to the present invention, the camera posture variation determination means is connected to the converted image in the overhead image generated using the mapping table corrected by the mapping table correction means. Based on the above, it is characterized in that a determination is made as to whether or not a camera installed in the movable part shows a change in posture assumed in advance as the movable part moves.

  And according to such a structure, based on the connection state of the some conversion image which forms the bird's-eye view image produced | generated using the mapping table after correction | amendment, the camera installed in the movable part can move a movable part. Accordingly, it is possible to determine whether or not the posture change assumed in advance is indicated.

  Further, in another vehicle periphery monitoring device according to the present invention, the vehicle periphery monitoring image is a side view image for monitoring a side of the vehicle, and the movable part movement detection means is included in the side view image. It is characterized in that it is formed so as to detect the moving state of the movable part based on the reflected state of the vehicle.

  And according to such a structure, it becomes possible to detect the movement state of a movable part based on the reflection state of the vehicle in a side view image.

  Furthermore, in another vehicle periphery monitoring device according to the present invention, the camera posture variation determining means reflects the vehicle in the side view image generated using the mapping table corrected by the mapping table correcting means. It is characterized in that it is configured to determine whether or not a camera installed in the movable part shows a change in posture assumed in advance as the movable part moves, based on a state of being depressed Yes.

  According to such a configuration, the camera installed in the movable part moves with the movement of the movable part, based on the reflected state of the vehicle in the side view image generated using the corrected mapping table. Thus, it is possible to determine whether or not the posture variation assumed in advance is indicated.

  According to the vehicle periphery monitoring device according to the present invention, the vehicle periphery is inappropriate in response to a change in posture independent of the movable portion of the camera installed in the movable portion and a change in posture of the camera installed outside the movable portion. It is possible to suppress the display of the monitoring image and to display the vehicle periphery monitoring image corresponding to the presumed change in the posture of the camera accompanying the movement of the movable part even when the movable part moves. As a result, safe and smooth driving operation can be sufficiently supported.

  In addition, according to another vehicle periphery monitoring device according to the present invention, it is possible to point out an abnormality in the posture of the camera to the user by displaying a warning message, so that convenience can be improved.

  Furthermore, according to the other vehicle periphery monitoring device according to the present invention, it is possible to detect the moving state of the movable part based on the connection state of the plurality of converted images forming the overhead image, and therefore, an inappropriate overhead view. It is possible to simply and reliably perform suppression of image display and display of an overhead image corresponding to a presumed change in the posture of the camera.

  Furthermore, according to another vehicle periphery monitoring device according to the present invention, in addition, based on the connection state of a plurality of converted images forming a bird's-eye view image generated using the corrected mapping table, in other words, Based on whether or not an appropriate overhead image with a good connection state of the converted image could be generated by using a mapping table in which the correction corresponding to the movement amount of the movable part was performed, the installation was performed on the movable part. Since it can be determined whether or not the camera shows a presumed change in posture with the movement of the movable part, it is possible to suppress the inappropriate display of the overhead view image and the presumed change in the posture of the camera. The corresponding overhead image can be displayed more easily and reliably.

  In addition, according to another vehicle periphery monitoring device according to the present invention, it is possible to detect the moving state of the movable part based on the reflected state of the vehicle in the side view image. It is possible to simply and reliably perform the suppression of the display of the view image and the display of the side view image corresponding to the presumed change in the posture of the camera.

  Furthermore, according to another vehicle periphery monitoring apparatus according to the present invention, further, based on the reflected state of the vehicle in the side view image generated using the corrected mapping table, in other words, the movable part Whether or not an appropriate side view image in which the state of the reflection of the vehicle in the side view image matches the one assumed in advance can be generated by using the mapping table in which the correction corresponding to the amount of movement is performed Therefore, it is possible to determine whether or not the camera installed in the movable part shows a change in posture assumed in advance with the movement of the movable part. Suppression and display of a side view image corresponding to a presumed change in the posture of the camera can be performed more simply and reliably.

  Hereinafter, an embodiment of a vehicle periphery monitoring device according to the present invention will be described with reference to FIGS. 1 to 6.

  As shown in FIG. 1, the vehicle periphery monitoring apparatus 1 in the present embodiment has a total of four cameras 2, 3, 4, and 5 installed at different locations of the vehicle. One of the four cameras 2, 3, 4 and 5 is a front camera 2 installed in the front part (for example, an emblem part) of the vehicle. An object (such as a road surface) existing in front of the vehicle is photographed. One of the other three cameras 3, 4, 5 is a left side camera 3 installed on a left door mirror of the vehicle as a movable part, and this left side camera 3 is a vehicle An object (such as a road surface) existing on the left side of is photographed. One of the other two cameras 4 and 5 is the right side camera 4 installed on the right door mirror of the vehicle as a movable part, and this right side camera 4 is the right side of the vehicle. The object (road surface etc.) which exists in the direction is photographed. The remaining one camera 5 is a back camera 5 installed in the rear hatch of the vehicle as a movable part, and this back camera 5 captures an object (road surface or the like) existing behind the vehicle. It has become.

  These cameras 2, 3, 4, and 5 may be cameras that include an ultra-wide-angle lens such as a fish-eye lens and form an image of an object on the imaging surface of a solid-state imaging device such as a CCD.

  An image processing unit 7 serving as an image processing unit is connected to each of the cameras 2, 3, 4, 5. The image processing unit 7 is connected to each camera 2, 3, 4, 5 from each camera 2, 3, 4, 5. 4 and 5 captured images are acquired, and a vehicle periphery monitoring image for monitoring the periphery of the vehicle is generated based on the acquired captured images.

  In the present embodiment, as shown in FIG. 2, the image processing unit 7 includes an overhead view image 9 in which the vehicle and its periphery as a vehicle periphery monitoring image are looked down from above the vehicle (vertically above), and a vehicle periphery monitoring image. An image 8 including a side view image 10 for monitoring the side of the vehicle is generated.

  As shown in FIG. 2, the bird's-eye view image 9 is a forward monitoring image 11 as a converted image generated based on a captured image of the front camera 2 and a converted image generated based on a captured image of the left side camera 3. Left side monitoring image 12, right side monitoring image 13 as a converted image generated based on the captured image of the right side camera 4, and rear monitoring as a converted image generated based on the captured image of the back camera 5. It is formed by the image 14 and the illustration image 15 of the vehicle pasted on the blind spot area (the area not photographed) of the cameras 2, 3, 4, and 5. As shown in FIG. 2, the front monitoring image 11 is connected to the left side monitoring image 12 and the right side monitoring image 13. The rear monitoring image 14 is also connected to the left side monitoring image 12 and the right side monitoring image 13. The illustration image 15 of the vehicle is positioned at the center of the image 9 so as to be connected to all the monitoring images 11, 12, 13, and 14.

  On the other hand, the side view image 10 includes an image 16 in which the road surface on the left side of the vehicle is looked down obliquely toward the front in the traveling direction of the vehicle, and a reflected image 17 of the vehicle in which a part of the vehicle (left side portion) is reflected. Is formed by.

  Returning to FIG. 1, the image processing unit 7 is connected to a mapping table storage unit 18 as a mapping table storage unit. The mapping table storage unit 18 includes a mapping table for the overhead image 9 and a side view image. A mapping table for 10 and an illustration image of the vehicle are stored. Note that the mapping table for the overhead image 9 may be independent for each of the cameras 2, 3, 4, and 5.

  Here, the mapping table describes the correspondence between the captured images of the cameras 2, 3, 4, and 5 and the vehicle periphery monitoring images 9 and 10.

Specifically, as shown in FIG. 3, the mapping table 20 includes coordinate points (Xc _i , Yc _j ) (i, j are arbitrary natural numbers) in the XY coordinate system of the cameras 2, 3, 4, and 5. One pixel in the captured image of the cameras 2, 3, 4 and 5 positioned at (note that i ≠ 5, j ≠ 2) is a coordinate point (Xt _m , Yt) in the XY coordinate system of the display 21 described later. _n ) (m and n are used to generate one pixel in the vehicle periphery monitoring images 9 and 10 displayed in an arbitrary natural number (where m ≠ 4, n ≠ 3)). Note that the mapping table 20 may describe that two or more pixels in the captured images of the cameras 2, 3, 4, and 5 are used to generate one pixel of the vehicle periphery monitoring images 9 and 10. (For example, see Patent Document 1, paragraphs 0041 and 0044). In addition, the mapping table 20 may be accompanied by coordinate conversion in consideration of correcting distortion of a captured image when an ultra-wide angle lens is used as a lens for the cameras 2, 3, 4, and 5.

  Then, the image processing unit 7 uses the mapping table 20 for the bird's-eye view image 9, the captured video of the front camera 2 as the front monitoring image 11, the captured video of the left side camera 3 as the left side monitoring image 12, and the right side By converting (viewpoint conversion) the captured video of the camera 4 to the right side monitoring image 13 and the captured video of the back camera 5 to the rear monitoring image 14, respectively, and further combining the illustration image 15 of the host vehicle, An overhead image 9 is generated.

  Further, the image processing unit 7 generates the side view image 10 by converting the captured video of the left side camera 3 into the side view image 10 using the mapping table 20 for the side view image 10.

  Returning to FIG. 1, a display 21 is connected to the image processing unit 7. The vehicle periphery monitoring images 9 and 10 generated by the image processing unit 7 are output and displayed on the display 21. It has become.

  A movable part movement detection unit 22 as a movable part movement detection unit is connected to the image processing unit 7. The movable part movement detection unit 22 is generated by the image processing unit 7 from the image processing unit 7. Data of the vehicle periphery monitoring images 9 and 10 is input. Then, based on the input data of the vehicle periphery monitoring images 9 and 10, the movable part movement detection unit 22 has a left door mirror, a right door mirror, and a rear hatch as a movable part of the vehicle. The movement (moving state) from the normal position at the time of photographing is detected together with the movement amount of each movable part. The normal positions of the left door mirror and the right door mirror are the positions where the left and right doors to which the mirrors are attached are completely closed, and the normal position of the rear hatch is the position where the rear hatch is closed. When the movable part is at the normal position, the mapping table 20 need not be corrected.

  More specifically, the movable part movement detection unit 22 is based on the connection state of the monitoring images 11, 12, 13, and 14 in the overhead view image 9 and the reflection state of the vehicle in the side view image 10. The movement state and movement amount of the door mirror and the rear hatch are detected.

  For example, the movable part movement detection unit 22 detects the front from both the image of the joint portion with the left side monitoring image 12 in the front monitoring image 11 and the image of the joint portion with the front monitoring image 11 in the left side monitoring image 12. When the feature points that are pixels respectively corresponding to the shooting points adjacent to each other in the camera 2 and the left side camera 3 are extracted and the extracted feature points are connected to each other, the moving state of the left door mirror is not detected. Become. In addition, it is preferable that the said feature point is a point which is easy to extract the boundary part etc. of the white line drawn on the road surface and a road surface. Moreover, the movable part movement detection part 22 is a left door mirror, when the position and range (size) of the vehicle reflection image 17 in the side view image 10 correspond with the suitable position and range stored beforehand. The moving state is not detected.

  On the other hand, when the feature points extracted from the front monitoring image 11 and the left side monitoring image 12 are not connected to each other or the position and range of the vehicle reflection image 17 in the side view image 10 ( When the size does not match the appropriate position and range, the movement state and the movement amount of the left door mirror are detected.

  The movable part movement detection part 22 is a left door mirror corresponding to an amount of separation between a feature point of the front monitoring image 11 and a feature point of the left side monitoring image 12 that are originally connected in a stationary state of the left door mirror from the normal position. A table in which the amount of movement from the normal position is described in advance, and the amount of movement of the left door mirror may be detected by referring to this table. In this table, as the amount of movement of the left door mirror from the normal position, the amount of movement of the left door mirror in the vehicle width direction and the amount of movement of the vehicle in the full length direction may be described. In addition, the rotation angle of the left door mirror from the normal position (0 °) may be described in the table as the amount of movement of the left door mirror.

  Alternatively, the movable part movement detection unit 22 may include a deviation amount (for example, a deviation amount in the vehicle width direction and the full length direction of the vehicle) of the reflected image 17 of the vehicle and the corresponding left door mirror. A table in which the amount of movement from the normal position is described in advance, and the amount of movement of the left door mirror may be detected by referring to this table.

  Of course, the detection of the moving state of the movable part based on the connection state of the bird's-eye view image 9 may be applied to a movable part other than the left door mirror.

  A mapping table correction unit 23 as a mapping table correction unit is connected to the movable part movement detection unit 22, and a correction information storage unit 24 as a correction information storage unit is connected to the mapping table correction unit 23. Yes.

In the correction information storage unit 24, correction information of the mapping table 20 corresponding to the amount of movement from the normal position for each of the left side camera 3, the right side camera 4, and the back camera 5 is stored. This correction information may be, for example, information in a table format as shown in FIG. That is, in the table shown in FIG. 4, the amount of movement of the left door mirror is expressed as the amount of movement (mm) in the vehicle width direction and the full length direction, and correction information corresponding to each amount of movement is stored in the left side camera. 3 is described for each pixel of each coordinate point in the XY coordinate system. For example, when the movement amount of the left side camera 3 is i (mm) in the vehicle width direction and j (mm) in the full length direction of the vehicle, the coordinate point (Xc _i , Yc) in the XY coordinate system of the left side camera 3 _j ) from the pixel on the coordinate point (Xt _m , Yt _n ) in the XY coordinate system of the display 21 to the corrected coordinate point (Xt _{m + 1} , Yt) _{n + 1} ) is changed (corrected) to the upper pixel. Note that the correction information is not limited to describing the corrected coordinate point in the XY coordinate system of the display 21 as shown in FIG. 4, but as the correction amount, the X direction and Y of the coordinate point of the XY coordinate system of the display 21 are used as the correction amount. A movement correction amount in the direction or a rotation correction amount (for example, a rotation matrix for rotating the coordinate point before correction to the coordinate point after correction) may be described. Of course, these examples may be applied to movable parts other than the left door mirror.

  Returning to FIG. 1, when the movement of the movable part from the normal position and the amount of movement thereof are detected by the movable part movement detection unit 22, the mapping table correction unit 23 detects the movement of the movable part that has been detected. Correction information of the mapping table 20 corresponding to the movement amount is acquired from the correction information storage unit 24. Then, the mapping table correction unit 23 corrects the mapping table 20 stored in the mapping table storage unit 18 using the acquired correction information, and records the corrected mapping table 20 in the mapping table storage unit 18. It is like that.

  In addition to the above configuration, the image processing unit 7 is further connected to a posture variation camera detection unit 26 as posture variation camera detection means. The posture variation camera detection unit 26 is generated by the image processing unit 7. Cameras that have acquired the data of the vehicle periphery monitoring images 9 and 10, and based on the acquired data of the vehicle periphery monitoring images 9 and 10, change in posture from the normal posture when photographing the periphery of the vehicle Is supposed to be detected.

  Specifically, similar to the movable part movement detection unit 22, the posture variation camera detection unit 26 determines the posture from the normal posture based on the connection state of the monitoring images 11, 12, 13, and 14 in the overhead image 9. Cameras 2, 3, 4, and 5 in which fluctuations occur are detected.

  Similarly to the movable part movement detection unit 22, the posture change camera detection unit 26 is a left side camera as a camera in which a change in posture from the normal posture occurs based on the reflected state of the vehicle in the side view image 10. The camera 3 is detected.

  The normal postures of the cameras 2, 3, 4, and 5 are the normal (original) postures of the cameras 2, 3, 4, and 5 when the surroundings are photographed to generate the vehicle periphery monitoring images 9 and 10. In the case of the cameras 3, 4 and 5 installed in the movable part, it is assumed that the movable part has not moved from the normal position.

  A camera type determination unit 27 as a camera type determination unit is connected to the posture variation camera detection unit 26, and the camera type determination unit 27 acquires the detection result of the posture variation camera detection unit 26. Yes. Then, the camera type determination unit 27 detects that the camera detected by the posture variation camera detection unit 26 from the detection result of the acquired posture variation camera detection unit 26 is a camera (left side camera 3, right side) It is determined whether the camera is a side camera 4 or a back camera 5), and the determination result is output to the image processing unit 7.

  In the present embodiment, when the image processing unit 7 acquires from the camera type determination unit 27 a determination result that the camera detected by the posture variation camera detection unit 26 is not a camera installed on the movable unit. The display of the vehicle periphery monitoring images 9 and 10 generated using the corrected mapping table 20 described above on the display 21 is restricted.

  That is, when the camera detected by the posture variation camera detection unit 26 is not a camera installed in the movable unit, the image processing unit 7 has an abnormality in the front camera 2 and the front as shown in FIG. A warning message image 29 for notifying that the monitoring image 11 cannot be displayed is generated and displayed at a position corresponding to the front monitoring image 11 in the overhead image 9, thereby limiting the display of the overhead image 9. It is like that.

  In addition to the above configuration, the camera type determination unit 27 is connected to a camera posture variation determination unit 31 as a camera posture variation determination unit. Judgment results are acquired. This camera posture variation determination unit 31 only obtains a determination result that the camera detected by the posture variation camera detection unit 26 is a camera installed on the movable unit as the determination result of the camera type determination unit 27. Then, it is determined whether or not the camera exhibits a posture change assumed in advance as the movable part moves (hereinafter referred to as posture change determination).

  At this time, the camera posture variation determination unit 31 acquires the data of the vehicle periphery monitoring images 9 and 10 generated using the corrected mapping table 20 from the image processing unit 7, and based on the acquired data. Attitude variation determination is performed. Specifically, the camera posture variation determination unit 31 performs posture variation determination based on the connection state of the monitoring images 11, 12, 13, and 14 in the overhead view image 9 generated using the corrected mapping table 20. To do. In this case, for example, when the right-side monitoring image 13 generated using the corrected mapping table is appropriately connected to the front monitoring image 11, the camera posture variation determination unit 31 is connected to the right door mirror as a movable unit. It is determined that the installed right side camera 4 shows a change in posture assumed in advance as the right door mirror moves. In addition, the camera posture variation determination unit 31 performs posture variation determination based on the reflection state of the vehicle in the side view image 10 generated using the corrected mapping table 20. Then, when the position and range of the reflected image 17 of the vehicle in the side view image 10 matches the position and range of the appropriate reflected image 17 stored in advance, the camera posture variation determination unit 31 It is determined that the side camera 3 shows a change in posture assumed in advance with the movement of the left door mirror as the movable part.

  The camera posture variation determination unit 31 outputs the determination result of the posture variation determination to the image processing unit 7.

  In the present embodiment, the image processing unit 7 assumes that the determination result of the posture variation determination output from the camera posture variation determination unit 31 is assumed in advance as the camera installed in the movable unit moves. In the case of the determination that the posture is changed, the vehicle periphery monitoring images 9 and 10 generated using the above-described corrected mapping table 20 are displayed on the display 21.

  On the other hand, in the image processing unit 7, the determination result of the posture variation determination output from the camera posture variation determination unit 31 is not assumed in advance as one of the cameras installed in the movable unit moves with the movable unit. In the case of the determination that the posture is changed, the display of the vehicle periphery monitoring images 9 and 10 generated using the corrected mapping table 20 is stopped or limited. For example, when the camera in which the posture change that is not assumed in advance with the movement of the movable unit is the left side camera 3 installed in the left door mirror as the movable unit, the image processing unit 7 As shown in FIG. 6, a warning message image 33 for notifying that the left side camera 3 is abnormal and the side view image 10 cannot be displayed is generated and displayed on the display 21. The display of the side view image 10 is stopped. At this time, as shown in FIG. 6, the image processing unit 7 generates a warning message image 34 for notifying that the left side monitoring image 12 in the overhead image 9 cannot be displayed and displays the warning message image 34 on the display 21. This restricts the display of the bird's-eye view image 9 as the vehicle periphery monitoring image.

  Next, the operation of this embodiment will be described.

  In the present embodiment, first, the image processing unit 7 uses the mapping table 20 for the bird's-eye view image 9 stored in the mapping table storage unit 18 to capture the captured images of the cameras 2, 3, 4, and 5. While converting into the monitoring images 11, 12, 13, and 14, the illustration image 15 of the own vehicle is produced | generated, By combining these images, the bird's-eye view image 9 as a vehicle periphery monitoring image is produced | generated.

  At the same time, the image processing unit 7 converts the captured video of the left side camera 3 into the side view image 10 by using the mapping table 20 for the side view image 10 stored in the mapping table storage unit 18. A side view image 10 as a monitoring image is generated.

  Next, the movable part movement detection unit 22 detects the movement state and the movement amount of the movable part (left door mirror, right door mirror, or rear hatch) from the normal position, and the posture change camera detection unit 26 detects the posture from the normal posture. The cameras 2, 3, 4, and 5 in which the fluctuations are generated are detected. These detections are performed based on the connection state of the monitoring images 11, 12, 13, and 14 in the overhead image 9 generated by the image processing unit 7 and the reflection state of the vehicle in the side view image 10.

  As a result, when the cameras 2, 3, 4 and 5 in which the moving state of the movable part and the posture change are not detected at all, the image processing unit 7 generates the generated vehicle periphery monitoring images 9, 10 Is displayed on the display 21 as it is.

  On the other hand, when the cameras 2, 3, 4, and 5 in which the posture variation from the normal posture is detected by the posture variation camera detection unit 26, the camera type determination unit 27 detects the cameras 2, 3 , 4, 5 are determined using communication with the cameras 3, 4, 5 or the like.

  Then, as a result of the determination by the camera type determination unit 27, in the case of the front camera 2 in which the cameras 2, 3, 4, and 5 in which the posture change from the normal posture has occurred other than the movable unit are shown in FIG. As shown, the image processing unit 7 generates a warning message image 29 for notifying that the front monitoring image 11 cannot be displayed, and the other monitoring images 12, 13, 14 and the own vehicle illustration image 15 By combining and displaying on the display 21, the display of the overhead image 9 is limited.

  On the other hand, if the cameras 2, 3, 4, and 5 in which the posture variation from the normal posture is generated as a result of the determination by the camera type determination unit 27 are the cameras 3, 4, and 5 installed in the movable unit, The process shifts to posture change determination by the posture change determination unit 31.

  By the way, when shifting to this posture variation determination, naturally, the moving state of the movable portion from the normal position is detected together with the amount of movement by the movable portion movement detecting portion 22. Then, the mapping table correction unit 23 corrects the mapping table 20 using the correction information corresponding to the movement amount of the movable part, and the image processing unit 7 uses the corrected mapping table 20 to create a new vehicle periphery. The monitoring images 9 and 10 are generated.

  The posture variation determination is performed based on the vehicle periphery monitoring images 9 and 10 generated using the mapping table 20 after such correction.

  In other words, the camera posture variation determination unit 31 performs posture variation determination based on the connection state of the monitoring images 11, 12, 13, and 14 in the overhead view image 9 generated using the corrected mapping table 20. Then, when the monitoring images 11, 12, 13, and 14 are appropriately connected (when the above-described feature points are connected), the camera posture variation determination unit 31 moves the camera 3 with the movement of the movable unit. If it is determined that 4,5 and 5 indicate a presumed posture change, and the camera is not properly connected, the camera 3,4,5 does not have a presumed posture change as the movable part moves. Is determined.

  In addition, the camera posture variation determination unit 31 performs posture variation determination based on the reflected state of the vehicle in the side view image 10 generated using the corrected mapping table 20. Then, when the position and range of the reflected image 17 of the vehicle in the side view image 10 matches the position and range of the appropriate reflected image 17 stored in advance, the camera posture variation determination unit 31 When it is determined that the side camera 3 shows a change in posture assumed in advance with the movement of the left door mirror as the movable part and does not match, the left side camera 3 moves with the movement of the left door mirror. It is determined that the posture change is not assumed in advance.

  The image processing unit 7 determines that the cameras 3, 4, and 5 installed in the movable unit show a variation in posture assumed in advance as the movable unit moves by the camera posture variation determination unit 31. In this case, the vehicle periphery monitoring images 9 and 10 generated using the corrected mapping table 20 are displayed on the display 21.

  On the other hand, the image processing unit 7 determines by the camera posture variation determination unit 31 that the cameras 3, 4, and 5 installed in the movable unit exhibit a posture variation that is not assumed in advance as the movable unit moves. In such a case, the display of the vehicle periphery monitoring images 9 and 10 is stopped or restricted by displaying the warning message images 33 and 34 as shown in FIG.

  As described above, according to this embodiment, the camera 3, 4, 5 installed in the movable part can cope with a change in posture independent of the movable part and a change in posture of the camera 2 installed other than the movable part. Thus, inappropriate display of the vehicle periphery monitoring images 9 and 10 can be suppressed, and even if the movable part moves, the cameras 3, 4 and 5 assumed in advance with the movement of the movable part can be used. It is possible to display the vehicle periphery monitoring images 9 and 10 corresponding to the change in posture, and as a result, it is possible to sufficiently support a safe and smooth driving operation.

  In addition, this invention is not limited to embodiment mentioned above, A various change is possible as needed.

  For example, in the above-described embodiment, the display control of the vehicle periphery monitoring image is performed assuming the change in the posture of one camera and the movement of one movable part. It is not limited to the form. For example, in the case of a bird's-eye view image, when a change in two or more movable parts where the camera is installed is detected, and a change in the posture of the camera of each movable part is not assumed in advance, A warning message image indicating that two or more monitoring images 12, 13, and 14 cannot be displayed may be displayed.

  Further, the overhead image 9 and the side view image 10 may be displayed on separate screens.

The block diagram which shows embodiment of the vehicle periphery monitoring apparatus which concerns on this invention The figure which shows the bird's-eye view image and side view image as a vehicle periphery monitoring image in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. The figure which shows a mapping table in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. The figure which shows correction information in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. The figure which shows the restriction | limiting state of the display of a bird's-eye view image in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. The figure which shows the restriction | limiting state of the display of a bird's-eye view image, and the stop state of the display of a side view image in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle periphery monitoring apparatus 2 Front camera 3 Left side camera 4 Right side camera 5 Back camera 7 Image processing part 9 Overhead image 10 Side view image 20 Mapping table 21 Display 22 Movable part movement detection part 23 Mapping table correction part 24 Correction information storage Unit 26 posture variation camera detection unit 27 camera type determination unit 31 camera posture variation determination unit

Claims (6)

A vehicle periphery monitoring device that generates a vehicle periphery monitoring image for monitoring the periphery of the vehicle based on a captured image of a camera that captures the periphery of the vehicle installed in the vehicle and displays the image on a display unit,
Mapping table storage means for storing a mapping table indicating a correspondence relationship between the video captured by the camera and the vehicle periphery monitoring image;
Image processing means for generating the vehicle periphery monitoring image using the mapping table stored in the mapping table storage means, and performing processing for displaying the generated vehicle periphery monitoring image on the display unit;
Correction information used for correction of the mapping table corresponding to the camera installed in the movable part of the vehicle, the movable part from a normal position of the movable part when photographing the periphery of the vehicle by the camera Correction information storage means for storing correction information having a correspondence relationship with the movement amount of
Movable part movement detection means for detecting the movement state of the movable part from which the camera is installed from the normal position together with the movement amount;
When the movement state and the movement amount of the movable part are detected by the movable part movement detection means, the correction information stored by the correction information storage means corresponding to the detected movement amount of the movable part is used. Mapping table correction means for correcting the mapping table;
Posture change camera detection means for detecting a camera in which a change in posture from a normal posture occurs when photographing around the vehicle;
Camera type determination means for determining whether or not the camera detected by the posture variation camera detection means is a camera installed in the movable portion;
When the camera type determination unit determines that the camera detected by the posture variation camera detection unit is a camera installed in the movable unit, the camera installed in the movable unit is moved. Camera posture variation determining means for determining whether or not the posture variation assumed in advance as the part moves is included,
The image processing means
When the camera posture variation determining means determines that the camera installed in the movable portion exhibits a posture variation assumed in advance as the movable portion moves, the mapping table correction is performed. Displaying the vehicle periphery monitoring image generated using the mapping table corrected by the means on the display unit,
When the camera type determination unit determines that the camera detected by the posture variation camera detection unit is not a camera installed on the movable unit, or the camera posture variation determination unit Display of the vehicle periphery monitoring image on the display unit is canceled when it is determined that the camera installed in the camera shows a posture change that is not assumed in advance as the movable unit moves. Or the vehicle periphery monitoring apparatus characterized by being formed so that it may restrict | limit. The image processing means
The vehicle periphery monitoring device according to claim 1, wherein a warning message is displayed on the display unit when the display of the vehicle periphery monitoring image on the display unit is stopped or restricted. . The vehicle periphery monitoring image is an overhead image obtained by looking down the vehicle and its surroundings from above the vehicle, which are generated based on the captured images of the plurality of cameras, and each of the captured images of each camera is the mapping It is an overhead image generated by connecting a plurality of converted images corresponding to each camera converted using a table,
The movable part movement detection means is formed so as to detect a movement state of the movable part based on a connection state of the converted images in the overhead image generated by the image processing means. The vehicle periphery monitoring apparatus according to claim 1 or 2. The camera posture variation determining means is
Based on the connection state of the converted image in the overhead image generated using the mapping table corrected by the mapping table correcting means, a camera installed in the movable unit is assumed in advance as the movable unit moves. The vehicle periphery monitoring device according to claim 3, wherein the vehicle periphery monitoring device is configured to determine whether or not a change in posture is indicated. The vehicle periphery monitoring image is a side view image for monitoring the side of the vehicle,
The movable part movement detection means is formed so as to detect a movement state of the movable part based on a reflection state of the vehicle in the side view image. Item 3. The vehicle periphery monitoring device according to Item 2. The camera posture variation determining means is
Based on the reflected state of the vehicle in the side view image generated using the mapping table corrected by the mapping table correcting means, the camera installed on the movable part moves with the movement of the movable part. The vehicle periphery monitoring device according to claim 5, wherein the vehicle periphery monitoring device is configured to determine whether or not a change in posture assumed in advance is indicated.

Images