WO2003002375A1 - Device for image detecting objects, people or similar in the area surrounding a vehicle - Google Patents

Abstract

The invention relates to a device (100) for detecting objects, people or similar, especially in the area surrounding a vehicle (10), comprising a stereo sensitive image display unit with at least two image sensors (14, 14'), and an evaluating unit (32) for evaluating the signals emitted by the image sensors. According to the invention, in order to detect the three-dimensional geometry of the area surrounding a vehicle in a variable and simple manner offering wide coverage and an angular viewpoint, the optical axes (16, 16') of the image sensors (14, 14') are disposed in such a manner that they are adjustable in relation to each other and/or in relation to the vehicle (10).

Description

DEVICE FOR DETECTING THE IMAGE OF OBJECT, PEOPLE OR

THE SAME IN THE ENVIRONMENT OF A VEHICLE

Technical field

The invention relates to a device for detecting objects, people or the like, in particular in the vicinity of a vehicle.

State of the art

From the publication IEEE Intelligent Vehicles Symposium, Oct. 2000, "Real-time stereo vision for urban traffic scene understanding", U. Franke, it is known to capture the surroundings of a vehicle with an image recording unit which comprises a stereo camera for taking three-dimensional images , By evaluating the recorded scene, the position, for example of other vehicles, people or objects, can be determined in the area surrounding the vehicle. On the basis of this information, the longitudinal and / or transverse guidance of the vehicle can be influenced by a controller, so that the driver is supported in the vehicle guidance and driving comfort and traffic safety can be improved. For example, in the event of an impending collision, an automatic avoidance or braking maneuver can avoid the collision or at least reduce the collision energy.

From the document MIT Press, 1995, "Three dimensional computer vision: a geometric viewpoint", O. Faugeras, it is known how a stereo camera rasystem determines the three-dimensional geometry of the environment projected onto the pair of images by tangling, that is to say by determining the intersection of the light rays incident in the left and right cameras, respectively. It is determined that the position of the camera heads, ie the alignment of their optical axes, is a stereo camera influences the detection properties considerably and some objects make stereosensing difficult due to their position or texture

The choice of the optical focal length of a rigid stereo arrangement basically requires a compromise between range and viewing angle. However, high demands are made on both sizes in vehicle applications. For example, at speeds of around 200 km / h - for example according to the so-called "half-speedometer rule" "- a range of more than 100 m required At the same time, reliable detection of traffic signs requires large viewing angles of, for example, 70 °. These requirements cannot be met with the known stereo cameras

To remedy this, it was proposed in the IEEE Intelligent Vehicles Symposium, Oct 2000, "EMS Vision Gaze control in autonomous vehicles", M Pellkofer and ED Dickmanns, to mount individual mono cameras with different focal lengths on a rotatable platform. This makes it possible to enlarge the viewable angle The camera form is oriented here to a distant object or to an object located laterally to the direction of travel, for example a traffic sign Presentation of the invention, task, solution, advantages

The invention is based on the object of specifying a device of the generic type by means of which a three-dimensional geometry of a vehicle environment can be variably ascertained both with a long range and with a large viewing angle.

According to the invention, this object is achieved by a device having the features mentioned in claim 1. The fact that the image recording unit comprises at least two image sensors, the optical axes of the image sensors being adjustable relative to one another and / or relative to the vehicle, advantageously makes it possible to adapt the geometry of the image recording unit to the currently required range or viewing angle. A generally long range is made possible by choosing a long focal length, with an instantly smaller viewing angle being acceptable, since the device according to the invention allows the viewing angle to be enlarged again at any time. Furthermore, it is advantageously possible to achieve a variation of a stereo area of the image recording unit by means of the optical axes of the image sensors that can be aligned relative to one another. According to the alignment of the optical axes relative to one another, the overlap area of the detection areas of the individual image sensors can be enlarged or reduced. Variants are preferably also conceivable here according to which at least one of the image sensors is oriented in the direction of travel, while another of the image sensors is oriented transversely to the direction of travel. In this way, a temporarily monocular image evaluation of each of the signals supplied by the image sensors can be achieved if this appears necessary or sufficient in a corresponding situation. Thereby, the fact that the optical axes can also be adjusted relative to the vehicle, the overlap area of the detection areas of the image sensors can preferably also be pivoted relative to the vehicle and the size thereof can also be varied there is, for example, located to the side of the vehicle. It is also advantageous that, due to the variable adjustability of the optical axes, bodies or the like which are located in the surroundings and are difficult to stereosensitive due to their shape and / or texture can be detected and evaluated

Further preferred embodiments of the invention result from the other features mentioned in the subclaims

Brief description of the drawings

The invention is explained in more detail in exemplary embodiments with reference to the accompanying drawings

1a shows schematic views of a vehicle with and 1 b of a device according to the invention,

Fig. 2 schematically different Ausπchtrnoglich- to 6 sides of the device according to the invention and

7 shows a flowchart of a control of the invention

contraption Best way to carry out the invention

1a and 1b, a motor vehicle 10 is shown schematically in plan view and in side view. The motor vehicle 10 has a device, designated overall by 100, for the detection of objects, people or the like in an environment of the motor vehicle. The device 100 comprises an image recording unit 12 which has two image sensors 14 and 14 '(hereinafter also called cameras 14 and 14'). The cameras 14 and 14 'are formed, for example, by CMOS cameras. Corresponding to the construction of the cameras 14 or 14 ', these optical axes 16 or 16' have. These are only indicated schematically in Fig. 1 a. The optical axes 16 and 16 'determine the orientation of the cameras 14 and 14' and thus their image acquisition area.

A first drive device 18 is assigned to the camera 14 and a second drive device 20 is assigned to the camera 14 '. The drive devices 18 and 20 are arranged on a carrier 22, which in turn is assigned a third drive device 24. The drive devices 18, 20 and 24 are formed, for example, by stepper motors or continuously adjustable motors. The relative position of the optical axes 16 and 16 'of the cameras 14 and 14' to one another and / or to the motor vehicle 10 can be changed by means of the drive devices 18, 20 and 24. In principle, a pivoting of the optical axes 16 or 16 ′ to a vehicle longitudinal axis 26, to a vehicle transverse axis 28 and / or to a vehicle vertical vehicle axis 30 take place. According to the exemplary embodiment shown, it is assumed that the cameras 14 or 14 'can be rotated about the vertical vehicle axis 30 by means of the drive devices 18, 20 or 24. Here, the camera 14 is via the secondary device 18, the camera 14'. Via the drive device 20 and the carrier 22 via the drive device 24, each can be rotated separately about the vertical axis 30 of the vehicle, so that according to the controls of the drive devices 18, 20 and 24 there are any orientations of the optical axes 16 or 16 ′ in the horizontal plane a possible angular position of the cameras 14 and 14 'and thus a corresponding angular position of the optical axes 16 and 16' to the horizontal plane, these optical axes 16 and 16 'can also be adjustable in relation to one another and / or relative to the motor vehicle 10 on a conical surface area. The image recordings Unit 12 is preferably arranged on a roof module of vehicle 10

Both the cameras 14 and 14 'and the drive devices 18, 20 and 24 are connected to an evaluation unit 32, which comprises at least one signal processing processor. The image signals supplied by the cameras 14 and 14' are processed by the evaluation unit 32 and control signals for the drive devices 18 , 20 and 24 generated. Further actuators of the motor vehicle 10 can be controlled via the evaluation unit 32, for example depending on the images supplied by the image recording unit 12. Such actuators can be, for example, actuators of braking devices, e-gas devices, steering devices or the like the evaluation unit can also be connected to further sensors, not shown, from which additional information, such as operating parameters of the motor vehicle 10, can be provided The function of the device according to the invention is explained in more detail with reference to FIGS. 2 to 7

7 shows a flowchart, according to which, in a first step 34, the evaluation unit 32 detects an image pair recorded by the cameras 14 or 14 '. In a next step 36, corresponding image areas in the two, from slightly different viewing angles (corresponding to the distance between the cameras 14 or 14 'to each other) images sought Where the corresponding image areas constrict, the evaluation unit 32 creates from the approximate knowledge of the camera positions (known from the control of the drive devices 18, 20 and 24) and possibly from the previous evaluation of the calibration and the three - Dimensional geometry of the recorded scene With the help of this rough knowledge of corresponding image positions, the exact locations of corresponding areas in the two images can be determined using known search methods for estimating displacement. From the knowledge now available of the cor responding image areas, the relative orientation between the two cameras 14 and 14 'can be determined (step 38). This can be done, for example, by determining the so-called essential matrix E by suitable resolution of the system of equations

The 3x3 matrix fulfills the conditions of an essential matrix, and x and y the corresponding coordinates in the image on the right and left camera 14 and 14 'forms. Thus one obtains an equation of determination for each corresponding pair of points. The Essential Matrix can therefore be determined with a sufficient number of point pairs. The previous control of the drive devices 18, 20 or 24, as well as knowledge of the scene geometry from previous measurements, can advantageously also be used as prior knowledge to stabilize the system of equations. Standard methods, for example a robust extended caiman filter, can be used as a suitable solution method.

In a next step 40, the captured scene is subjected to a stereo image evaluation with known calibration, that is to say the geometry of the street and of objects, people or the like is measured three-dimensionally. The previously determined orientation of the two cameras 14 and 14 'and the point correspondence are brought together in the triangularization method known per se.

In a next step 42, the recorded scene is analyzed and suitable areas for mono detection and stereo detection are identified depending on the situation. In accordance with this identification of mono areas and stereo areas, the drive devices 18, 20 and 24 are controlled accordingly by the evaluation unit 32, so that there is a relative adjustment of the optical axes 16 or 16 'to one another or possibly to the motor vehicle 10. In accordance with the alignment of the optical axes 16 and 16 ', the image acquisition areas of the cameras 14 and 14' are more or less superimposed. An emerging overlay area then defines the stereo acquisition of the recorded scene and the non-overlaid areas define the mono acquisition of the recorded scene. A few examples are schematically indicated with reference to FIGS. 2 to 6, but without claiming to be complete for the possible variants.

2 to 6, the image capture area of camera 14 is designated 43 and the image capture area of camera 14 'is designated 44. Corresponding to the alignment of the optical axes 16 and 16 ', there is an overlay area 46 of the two image acquisition areas 43 and 44. The overlay area 46 defines the stereo acquisition of the recorded scene and the remaining sections of the image acquisition areas 43 and 44 (outside the overlay area 46) the mono capture of the recorded scene.

2 shows by way of example that the cameras 14 and 14 ′ are aligned parallel to one another in the longitudinal axis 26 of the vehicle. In this way, a favorable stereo detection area 46 is obtained in the direction of travel, while edge areas are of minor importance.

FIG. 3 illustrates a variant in which the cameras 14 and 14 ', respectively, are aligned parallel and pivoted to the right to the longitudinal axis 26 of the vehicle. This results in a large stereo detection area 46 oriented to the right, which is advantageous, for example, when measuring parking scenes or the like.

In contrast to FIG. 3, an embodiment variant is shown in FIG. 4, in which the cameras 14 and 14 ′ are not pivoted individually by the drive devices 18 and 20 about the vehicle vertical axis 30, but rather the carrier 22 as a whole by the drive device 24 is pivoted about the vehicle vertical axis 30. This also maintains a parallel alignment of the cameras 14 and 14 'and at the same time realizes an enlarged viewing angle compared to FIG. 3.

FIG. 5 shows a variant in which the camera 14 'remains aligned in the longitudinal axis 26 of the vehicle, while the camera 14 is pivoted to the right to the longitudinal axis 26 of the vehicle. This makes it possible, for example, to detect traffic signs or the like located on the right-hand lane edge with a large viewing angle, but at the same time to observe the scene in front of the motor vehicle in the direction of travel.

6 shows an embodiment variant in which the camera 14 is pivoted on the right to the vehicle longitudinal axis 26 and the camera 14 'on the left to the vehicle longitudinal axis 26. As a result, the overall viewing angle of the image recording unit 12 is enlarged, while the stereo detection area 46 is reduced. Such an alignment of the cameras 14 or 14 ′ can in particular be used for monocular image processing, that is to say with relatively large mono detection areas 43 or 44. This is sufficient for traffic sign recognition, for example.

The exemplary embodiments shown clearly illustrate the variable use of the image recording unit 12 by the individually controllable drive devices 18, 20 and 24. Thus, the most varied of viewing angles and ranges can be combined in a simple manner with a wide variety of stereo detection areas 46 and mono detection areas 43 and 44, respectively. In particular, this advantageously also makes it possible to resolve the uncertainty of the stereo analysis, which occurs when the texture of an object is parallel to the epipolar (straight line, which is the location of possible point correspondences describes) is projected into the image. By rotating the camera base around at least one of the axes 26, 28 and 30, the epipolar can be rotated in the image, so that the texture is no longer parallel and the point correspondence is clearly established.

Claims

claims 1 device (100) for detecting objects, people or the like, in particular in the vicinity of a vehicle (10), with a stereosensitive image recording unit (12) which comprises at least two image sensors (14, 14 '), and an evaluation unit ( 32) for evaluating the signals supplied by the image sensors, characterized in that the optical axes (16, 16 ') of the image sensors (14, 14') can be adjusted so as to be changeable relative to one another and / or relative to the vehicle (10)

2 Device according to claim 1, characterized in that the optical axes (16, 16 ') to a vehicle longitudinal axis (26) and / or a vehicle transverse axis (28) and / or a vehicle vertical axis (30) are adjustable adjustable 3 Device according to one of the preceding claims, characterized in that each of the image sensors (14, 14 ') is assigned a drive device (18, 20) 4. Device according to one of the preceding claims, characterized in that the image sensors (14, 14 ') are arranged on a carrier (22) to which a drive device (24) is assigned 5 Device according to one of the preceding claims, characterized in that the optical axes (16, 16 ') can be aligned in a horizontal plane

Device according to one of the preceding claims, characterized in that the optical axes (16, 16 ') can be aligned on conical surface areas Device according to one of the preceding claims, characterized in that the image recording unit (12) is arranged in, on or on a roof area of the vehicle (10) Device according to one of the preceding claims, characterized in that the image sensors (14, 14 ') are CMOS cameras Device according to one of the preceding claims, characterized in that the drive devices (18, 20, 24) are stepper motors or continuously adjustable motors Device according to one of the preceding claims, characterized in that control signals for the drive devices (18, 20, 24) can be generated by the evaluation unit (32) as a function of supplied image signals from the image sensors (14, 14 ') - 14-