JP4393659B2 - In-vehicle preview sensor position adjustment device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a position deviation adjusting device for a vehicle-mounted preview sensor.
[0002]
[Prior art]
In recent years, a pair of vehicle-mounted cameras incorporating a solid-state image sensor (CCD), that is, a stereo-type vehicle exterior monitoring device using a stereo camera has attracted attention. This monitoring device recognizes the traveling situation ahead of the host vehicle based on a pair of images captured by the stereo camera. Then, depending on the recognition result, the driver is alerted or vehicle control such as deceleration by downshifting is performed. Specifically, first, a positional deviation, that is, parallax, of the same object displayed in the pair of captured images is specified (stereo matching). From the principle of triangulation, this parallax corresponds to the distance to the object. Then, based on the parallax and the two-dimensional position on the image plane, the three-dimensional information ahead of the vehicle, for example, the three-dimensional shape of the road (curve and undulation), or the inter-vehicle distance from the host vehicle to the preceding vehicle, etc. Can be sought. Conventionally, a stereo camera in which cameras are assembled to the left and right in the vehicle width direction has been attached to a vehicle body member such as a front rail in the vicinity of a room mirror in a vehicle interior.
[0003]
[Problems to be solved by the invention]
However, when a stereo camera is attached in the vicinity of the rearview mirror, some drivers may feel uncomfortable.
[0004]
Further, when a stereo camera is mounted in the vehicle interior, there is a problem that the mounting interval between the left and right cameras, that is, the camera base line length cannot be set too large. The camera baseline length varies to some extent due to camera mounting errors, dimensional errors of the vehicle body itself, and the like. At this time, if these errors are approximately the same, the larger the camera baseline length, the smaller the variation in the calculated distance, and thus the monitoring accuracy is improved. However, when a stereo camera is installed in the passenger compartment, it is difficult to increase the camera base length because of the front visibility and the wiper wiping area. As a result, there is a limit to improving the accuracy of the distance calculated by the stereo method.
[0005]
The present invention has been made in view of such circumstances, and an object thereof is to provide an in-vehicle preview sensor mounting structure that does not give a driver a sense of incongruity.
[0006]
[Means for Solving the Problems]
In order to solve this problem, the present invention provides an in-vehicle preview sensor misalignment adjusting device that obtains a captured image by capturing an image of the situation in front of the vehicle. The positional deviation adjusting device is attached to the left and right vehicle body front ends in front of the vehicle, and is a captured image obtained by imaging a pair of light emitting sources attached in the field of view of the preview sensor and the light emitting source pair emitting light. On the basis of this, it has an image processing means for detecting a shift in the mounting position of the preview sensor and performing an image conversion process on the captured image so that the detected shift becomes small.
[0007]
Here, in the present invention, the light emission source is preferably an infrared light emitter.
[0008]
In the present invention, the light source pair is preferably a pair of fender markers attached to the front of the vehicle body.
[0009]
Moreover, in this invention, while attaching to the center part ahead of a vehicle body, you may provide the ornament which has a light emission source. In this case, the image processing means detects the shift of the attachment position of the preview sensor based on the captured image obtained by capturing the state in which the ornament emits light.
[0010]
In the present invention, it is desirable that the above-described preview sensor is a stereo camera constituted by a pair of cameras.
[0011]
In the present invention, the preview sensor is attached to the roof portion of the vehicle body, and may be installed at the front end portion of the roof rail extending in the vehicle length direction on the vehicle front side.
[0012]
In addition, in the present invention, in order to accommodate the preview sensor inside, it further has a cover attached to the tip of the roof rail, and the space in this cover forms a part of the ventilation passage for ventilating the vehicle interior. It is preferable.
[0013]
Further, in the present invention, the preview sensor is a stereo camera composed of a pair of cameras, and one of the cameras is installed at the front end of the vehicle on the one roof rail, and the other of the preview sensors is It is desirable that the other roof rail is installed at the front end portion of the vehicle front side.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is an external perspective view of a vehicle equipped with a roof rail. Roof rails 1a and 1b extending in the vehicle length direction are attached to the left and right sides of the roof panel of the vehicle body. A right camera 2a is installed at the front end of the right roof rail 1a in front of the vehicle, and a left camera 2b is installed at the front end of the left roof rail 1b in front of the vehicle. The left and right cameras 2a and 2b constituting the stereo camera are attached separately from the roof rails 1a and 1b. However, in order to ensure an aesthetic appearance, the left and right cameras 2a and 2b have an integral shape with the front ends of the roof rails 1a and 1b. Have. The stereo camera is a preview sensor that monitors the situation in front of the vehicle, and includes an image sensor. As is well known, the object can be imaged from different positions with a stereo camera, and the distance to the object can be specified based on the parallax of the object displayed in the pair of captured images. On the other hand, fender markers 3a and 3b, which are light emission sources, are attached to the left and right of the fender tip. As will be described later, these fender markers 3a and 3b have a role of providing a positional reference when adjusting the positional deviation of the cameras 2a and 2b.
[0015]
FIG. 2 is a cross-sectional view of the main part showing the vicinity of the front ends of the roof rails 1a and 1b. Since the left and right roof rails 1a and 1b have the same cross-sectional shape in the vicinity of the left and right roof rails 1a and 1b, hereinafter, the left and right roof rails 1a and 1b are collectively denoted by reference numeral 1 and the left and right cameras 2a and 2b are represented by Collectively, it is represented by reference numeral 2. The roof rail 1 is fixed to the roof panel 4 of the vehicle body by bolts and nuts. The joint between the roof rail 1 and the roof panel 4 is sealed by packings 5a and 5b in order to prevent intrusion of rainwater or the like. On the other hand, an outer cover 6 having a shape integral with the tip end portion is attached to the tip end portion of the roof rail 1, and a joint portion with the roof rail 1 and a joint portion with the roof panel 4 are respectively provided with packings 5c and 5d. It is sealed. The outer cover 6 is formed with a vent for ventilating the inside, and a camera unit body including a semiconductor element, an electronic device and the like is accommodated in the inner space.
[0016]
The camera unit body includes a pedestal 7, a CCD 8, an interface unit 9, an inner case 10, and a CCD subcase 11. An interface unit 9 is attached to a pedestal 7 fixed to the roof panel 4 with bolts and nuts, and the unit 9 is covered with an inner case 10. In this way, the inner case 10 is provided separately from the outer cover 6, and a slight space is formed between these members, so that the electronic equipment in the interface unit housed in the inner case 10 is in use in the vehicle. Makes it difficult to be placed in a high temperature environment or a low temperature environment. A CCD sub case 11 is attached to the vehicle front side of the inner case 10, and the CCD 8 having a light receiving surface facing the vehicle front is attached to the sub case 11.
[0017]
In order to secure the camera field of view, in other words, a window 12 made of transparent glass or the like is arranged in front of the light receiving surface of the CCD 8 so as not to interfere with light reception on the light receiving surface of the CCD 8. The window 12 is made of transparent glass or the like that opens a part of the outer cover 6 and is fitted into the opening. The outside of the glass is coated with a titanium oxide coating, a fluorine coating, or the like in order to prevent poor visibility due to raindrops or dust. In place of such a coating process, a hydrophilic film, a water repellent film, or the like may be attached. Moreover, the heat ray reflective coating is given to the inner side of the glass. The space sealed by the window 12 and the CCD sub case 11 is filled with dry air or the like. In order to secure the camera field of view, a small wiper that is driven by a motor or the like may be attached to the outside of the window 12.
[0018]
On the other hand, two ventilation openings 14a and 14b are formed in the base 7. The ventilation port 14 a communicates with an air intake port 18 provided in the indoor trim 17 via the hose 15. On the other hand, the ventilation port 14 b communicates with a blower fan negative pressure portion of an air conditioner (not shown) via a hose 16. In a state where negative pressure is generated in the negative pressure portion, air (atmospheric pressure) in the passenger compartment is sucked from the air intake port 18, and the ventilation passage formed by the hose 15, the storage space of the camera unit body, and the hose 16. Flowing inside. Thereby, even in an environment where the temperature of the storage space of the camera unit main body is likely to rise due to direct sunlight or the like, it is possible to prevent the storage space from becoming high temperature.
[0019]
In particular, a small fan may be installed in the storage space of the camera unit main body as a countermeasure against high temperatures in leaving the vehicle in summer. In this case, according to the internal temperature detected by the temperature sensor, the small fan is driven to radiate heat in the storage space. Thereby, even in a situation (when the air conditioner is not in operation) in which the above-described airflow flowing in the ventilation passage is not generated, heat can be appropriately radiated from the storage space. As a power source for the small fan, it is preferable to use a solar cell or the like that can secure the power source even when the vehicle is left.
[0020]
Optical information detected by the CCD 8 is converted into an electrical signal. The electrical signal output from the CCD 8 is subjected to predetermined signal processing in the interface unit 9 and then transferred to the image recognition unit 20 shown in FIG. Based on a pair of captured images obtained from a stereo camera, the image recognition unit 20 recognizes a road shape ahead of the vehicle, a three-dimensional object including a preceding vehicle, etc., using a distance measurement technique based on the stereo method and an image recognition technique. To do. The details of the recognition method are described in, for example, Japanese Patent Laid-Open No. 5-265547, so refer to them if necessary.
[0021]
As described above, in the stereo camera mounting structure as described above, the stereo camera that has been conventionally installed in the vehicle interior is installed on the roof portion of the vehicle body. Therefore, since the projection in the passenger compartment caused by the stereo camera can be eliminated, the driver does not feel uncomfortable.
[0022]
In addition, the camera base line length can be set larger than in a conventional mounting structure in which a stereo camera is mounted in the vehicle interior. As described above, variations in the camera base line length caused by the mounting error of the left and right cameras 2a and 2b, the dimensional error of the vehicle body shape, etc. appear as errors in the calculated distance due to the characteristics of the stereo method. At this time, if the mounting error, the dimensional error, and the like are approximately the same, the larger the camera base line length, the smaller the variation in the calculated distance. Therefore, in order to improve the accuracy of the calculated distance, it is advantageous to increase the camera baseline length. In the mounting structure according to the present embodiment, it is possible to secure a camera base line length comparable to the mounting interval of the roof rails 1a and 1b. Therefore, it is possible to improve the accuracy of vehicle exterior monitoring compared to the conventional mounting structure. Become.
[0023]
In addition, since the stereo camera is attached to the roof portion of the vehicle body that is higher than the room mirror, it is possible to improve the monitoring accuracy in the distance and the vicinity. In a situation where the vehicle is traveling on a road or a highway with a difference in elevation, the distant visibility is improved by increasing the stereo camera installation position, and a farther preceding vehicle can be recognized. In addition, the blind spot in the immediate vicinity of the host vehicle can be reduced by the height of the mounting position.
[0024]
Furthermore, since the mounting rigidity of the stereo camera can be improved, the imaging blur of the cameras 2a and 2b can be reduced. Originally, the roof rails 1a and 1b themselves are designed to carry heavy objects, and the rigidity of the vehicle body member at the location where the roof rails 1a and 1b are attached is high. Therefore, if a stereo camera is attached to such a highly rigid portion, camera shake can be reduced, so that a reduction in monitoring accuracy can be suppressed. In addition, the external appearance of the stereo camera, that is, the shape of the outer cover 6 is integrated with the roof rails 1a and 1b, so that the appearance is not impaired. In addition, it is advantageous in comparison with the conventional mounting structure in terms of design or option setting.
[0025]
In the above description, a stereo camera is used as the preview sensor. The present invention is suitable for a stereo camera mounting structure, but is not limited thereto, and is widely used for various preview sensors (including laser and millimeter wave sensors) for monitoring the situation in front of the vehicle. Of course it can be applied.
[0026]
Next, detection of the displacement of the stereo camera mounting position and its adjustment method will be described. As described above, the accuracy of the distance calculated by the stereo method depends on the mounting accuracy of the stereo camera and the dimensional accuracy of the vehicle body. However, these mechanical processing accuracy is low compared with the accuracy required for appropriately performing the distance measurement by the stereo method. Therefore, by imaging a target object (for example, a test chart) placed at a known distance, a shift in the position of the stereo camera is detected, and this shift is equivalently corrected by image conversion processing such as affine transformation. A technique has been proposed (for example, see Japanese Patent Laid-Open No. 10-307352). In the present embodiment, the positional deviation, that is, the orientation and the inclination of the stereo camera is adjusted by causing the fender markers 3a and 3b to emit light.
[0027]
FIG. 3 is a diagram for explaining misalignment detection using the fender markers 3a and 3b. The positions of the left and right fender markers 3a and 3b with respect to the stereo camera are known, and each of the fender markers 3a and 3b includes a light bulb 21 and an LED 22 (colorless LED) that emits infrared light. The light bulb 21 is lit in conjunction with a lighting switch such as a headlight. On the other hand, the LED 22 blinks when a positional deviation adjustment of the stereo camera is instructed. At that time, the image recognition unit 20 blinks the left and right LEDs 22 at a light emission timing shifted by a half cycle. Thereby, based on the captured images of the cameras 2a and 2b, the image recognition unit 20 can discriminate without misidentifying the left and right fender markers 3a and 3b, and is less susceptible to disturbances such as sunlight. In addition, in order to accurately detect the fender markers 3a and 3b displayed in the captured image, as shown in FIG. 4, the lens unit 23 is formed on the fender markers 3a and 3b, and directivity is applied to the infrared light of the LED 22. Is given. The directivity peak is set to face the stereo camera.
[0028]
Hereinafter, although the positional deviation adjustment will be described by taking the right camera 2a as an example, the same processing is also performed for the left camera 2b. FIG. 5 is a diagram illustrating the search range S1 of the fender marker 3a in the captured image. An effective area R2 that is a target area for monitoring the traveling state is set in an area excluding the end of the entire area R1 of the captured image. The right fender marker 3a is displayed at the target position T if there is no position shift of the right camera 2a. However, actually, the fender marker 3a is displayed at a position shifted from the target position T due to an attachment error or the like of the camera 2a. Therefore, in a state immediately after the camera 2a is attached (at the time of system initialization), a relatively wide range in consideration of the maximum deviation amount that can actually occur, centered on a preset target position T (appropriate position), Set as search range S1. The image processing unit 20 searches the search range S1 to identify the imaging position (two-dimensional coordinates on the image plane) of the right fender marker 3a emitting infrared light. FIG. 5 shows a state in which the camera 2a is facing left with respect to the target direction.
[0029]
When the position (imaging position) where the right fender marker 3a is projected is specified, the image recognition unit 20 performs image conversion such as affine transformation on the captured image. That is, the effective area R2 is shifted in the horizontal / vertical direction so that the imaging position of the right fender marker 3a matches the target position T (matching point P) (see FIG. 6). Next, the position of the left fender marker 3b is specified by searching the predetermined search range S2 set with the coincidence point P as a reference. Since the distance between the left and right fender markers 3a and 3b is known, if the coincidence point P is specified, the position of the left fender marker 3b is also generally specified. Therefore, the search range S2 is set in consideration of the maximum amount of deviation that can actually occur (particularly, the amount of deviation in the rotational direction about the coincidence point P). FIG. 6 shows a state where the mounting position of the camera 2a is shifted clockwise.
[0030]
When the position where the left fender marker 3b is projected is specified, the image recognition unit 20 performs image conversion such as affine conversion on the captured image. That is, the effective area R2 is rotated counterclockwise around the coincidence point P (see FIG. 7). Then, an area where the line connecting the position of the left fender marker 3b and the coincidence point P is parallel to the horizontal line of the effective area R2 (not the horizontal line of the pixel) is set as the final effective area R2.
[0031]
FIG. 8 is a diagram showing the effective area R2 finally set by the adjustment as described above. As described above, the displacement of the mounting position of the camera 2a is equivalently corrected by performing linear image conversion processing (typically affine transformation) on the captured image based on the imaging positions of the fender markers 3a and 3b. be able to. By tilting the effective area R2, the horizontal direction of the effective area R2 does not coincide with the arrangement direction of the horizontal pixels. Therefore, when the image in the effective area R2 is scanned in the horizontal direction, the image recognition unit 20 scans each pixel in a saw-tooth shape as shown in FIG. Handled as data on the line ((b) in the figure).
[0032]
After the adjustment at the time of system initial is once completed, the imaging positions of the fender markers 3a and 3b usually do not change greatly. Therefore, after that, if a relatively narrow search range S3, S4 is set based on the current imaging position of each of the fender markers 3a, 3b, and the fender markers 3a, 3b are searched in the search ranges S3, S4, respectively. Good. Since the amount of calculation required for these searches is relatively small, it is possible to adjust the positional deviation of the cameras 2a and 2b in real time in parallel with the monitoring control.
[0033]
As described above, in the positional deviation adjustment method according to the present embodiment, the fender markers 3a and 3b fixedly attached to known positions of the vehicle body are used without preparing a special test chart or adjustment pattern. . Since the imaging positions of the fender markers 3a and 3b as viewed from the stereo camera are always the same regardless of the driving situation, if the effective area is set based on these imaging positions, the positional deviation of the stereo camera is corrected appropriately. can do.
[0034]
Moreover, since the infrared light emitted from the LED 22 in the fender markers 3a and 3b is the light with the highest sensitivity of the CCD 8, the positions of the fender markers 3a and 3b can be accurately detected from the captured images. In particular, if directivity is imparted to the infrared light, it is difficult to be affected by disturbances such as sunlight. Also, since humans cannot sense infrared light, the driver will not feel uncomfortable even if infrared light is emitted (flashing) while driving. In addition, as a light emission source, you may use a telescopic cornering marker etc. instead of the fender markers 3a and 3b.
[0035]
Furthermore, an ornament having a light emission source of infrared light may be attached to a central portion in front of the vehicle body (near the center of the left and right fender markers 3a and 3b). In this case, the displacement of the camera mounting position is adjusted using the three points of the imaging position of the emitting ornament and the imaging positions of the left and right fender markers 3a and 3b. Thereby, it is possible to further improve the positional deviation adjustment accuracy.
[0036]
The above-described position adjustment method is not limited to a stereo camera, and can be widely applied to image sensors (including a monocular camera) that can detect light emitted from a light source that provides a positional reference. It is.
[0037]
【The invention's effect】
As described above, according to the present invention, since the vehicle-mounted preview sensor is attached to the front end portion of the roof rail, the driver does not feel uncomfortable. In particular, if the stereo camera is mounted using the mounting structure according to the present invention, the camera base line length can be increased as compared with the case where the stereo camera is mounted in the vehicle interior, so that the three-dimensional including the distance calculated by the stereo method is included. It becomes possible to improve the reliability of information.
[Brief description of the drawings]
FIG. 1 is an external perspective view of a vehicle equipped with a roof rail. FIG. 2 is a cross-sectional view of the main part showing the vicinity of the front end of the roof rail. FIG. FIG. 5 is a diagram showing a search range in a captured image. FIG. 6 is a diagram showing an effective area shifted in the horizontal / vertical direction. FIG. 7 is an effective area shifted in the rotation direction. Fig. 8 is a diagram showing an effective area finally set by adjustment. Fig. 9 is an explanatory diagram showing the handling of data on one horizontal line when the effective area is rotated.
1a, 1b roof rail, 2a, 2b camera,
3a, 3b fender marker, 4 roof panel,
5a, 5b, 5c, 5d packing, 6 outer cover,
7 base, 8 CCD,
9 Interface unit, 10 Inner case,
11 CCD subcase, 12 windows,
13 Harness, 14a, 14b Ventilation port,
15,16 hose, 17 indoor trim,
18 Air intake, 20 Image recognition unit,
21 bulbs, 22 LEDs,
23 Lens section

Claims (8)

In the in-vehicle preview sensor misalignment adjusting device that obtains a captured image by imaging the situation in front of the vehicle,
A pair of light-emitting sources attached to the left and right vehicle body front ends in front of the vehicle, and attached within the field of view of the preview sensor;
Image processing for detecting a shift in the attachment position of the preview sensor based on a captured image obtained by capturing a state in which the light source pair emits light, and performing an image conversion process on the captured image so that the detected shift is reduced A position deviation adjusting device for a vehicle-mounted preview sensor.   The apparatus of claim 1, wherein the light source is an infrared light emitter.   The position adjustment apparatus for a vehicle-mounted preview sensor according to claim 1, wherein the pair of light emitting sources is a pair of fender markers attached to the front of the vehicle body. It is attached to the front center part of the vehicle body, and further has an ornament having a light source,
The position of the in-vehicle preview sensor according to claim 3, wherein the image processing unit detects a shift in an attachment position of the preview sensor based on a captured image obtained by capturing a state in which the ornament emits light. Deviation adjustment device.   The apparatus for adjusting a positional deviation of a vehicle-mounted preview sensor according to any one of claims 1 to 4, wherein the preview sensor is a stereo camera including a pair of cameras.   6. The preview sensor according to claim 1, wherein the preview sensor is attached to a roof portion of a vehicle body and is installed at a front end portion of a roof rail extending in a vehicle length direction on a vehicle front side. The in-vehicle preview sensor misalignment adjusting apparatus described. In order to accommodate the preview sensor inside, it further has a cover attached to the tip of the roof rail,
The position adjustment apparatus for a vehicle-mounted preview sensor according to any one of claims 1 to 6, wherein the space in the cover forms a part of a ventilation passage for ventilating the vehicle interior. The preview sensor is a stereo camera composed of a pair of cameras,
One of the cameras is installed at a front end of the roof rail on the front side of the vehicle, and the other of the preview sensors is installed at a front end of the other roof rail on the front side of the vehicle. A position deviation adjusting device for a vehicle-mounted preview sensor according to any one of claims 1 to 7.

Images