WO2014090957A1

WO2014090957A1 - Method for switching a camera system to a supporting mode, camera system and motor vehicle

Info

Publication number: WO2014090957A1
Application number: PCT/EP2013/076423
Authority: WO
Inventors: Michael Newell
Original assignee: Connaught Electronics Ltd
Current assignee: Connaught Electronics Ltd
Priority date: 2012-12-12
Filing date: 2013-12-12
Publication date: 2014-06-19
Anticipated expiration: 2015-06-12
Also published as: DE102012024289A1

Abstract

The invention relates to a method for operating a camera system (2) of a motor vehicle (1) upon approach of the motor vehicle (1) to a roadway (31) extending transversely to the longitudinal axis (x) of the motor vehicle (1), wherein the camera system (2) is switched to a supporting mode, in which an image (18') of the roadway (31) is displayed on a display device (16) of the camera system (2), which is captured by means of the camera system (2) from a point of view (17), which is disposed on the motor vehicle (1) and is closer to the roadway (31) than a point of view of a driver of the motor vehicle (1), characterized by detecting the approach to the roadway (31) by means of an acquisition device (15) of the motor vehicle (1) and autonomously switching the camera system (2) to the supporting mode by means of a control device (15) due to the detection.

Description

Method for switching a camera system to a supporting mode,

camera system and motor vehicle

The invention relates to a method for operating a camera system of a motor vehicle upon approach of the motor vehicle to a roadway extending transversely to the longitudinal axis of the motor vehicle, wherein the camera system is switched to a supporting mode. In the supporting mode, an image of the roadway is displayed on a display device of the camera system from a point of view, which is disposed on the motor vehicle and is closer to the roadway than a point of view of the driver. In addition, the invention relates to a camera system for performing such a method as well as to a motor vehicle with a camera system.

Presently, the interest is directed to the so-called junction view, which can be presented on a display in the interior of a motor vehicle. It is already prior art to switch a camera system of a motor vehicle to a supporting mode, in which such a junction view is displayed. Here, images are concerned, which are provided by means of a camera of the camera system, which is disposed on the front - in particular on the front bumper. Here, for example, a camera can be centrally mounted on the front bumper, which has a very wide capturing angle or opening angle of 180° or even 200° and the camera axis of which is oriented in vehicle longitudinal direction. Such a camera then captures the entire environment in front of the motor vehicle including the lateral regions to the left and to the right of the motor vehicle. Alternatively, two cameras can also be mounted on the front bumper, which then are oriented to the left on the one hand and to the right on the other hand in order to capture the respective front lateral environmental regions of the motor vehicle. Independently of the employed number of the cameras, the driver is supported in the supporting mode in particular in such situations, in which the motor vehicle

approaches a crossroad with right of way and the sight to this road to the left and/or to the right is restricted by obstacles - such as for example other vehicles. In this case, the driver moves the motor vehicle towards the crossroad until the front of the motor vehicle reaches the crossroad and this road comes in the field of view of the at least one camera. Now, the images of the road can be presented on the display and the driver is supported to the effect that he is able to recognize on the display if other motor vehicles come from the left or from the right side.

Such a junction view mode of the camera system can be activated by the driver himself in the prior art by the driver performing a corresponding operating input. For example, this can be effected by touching a touch-sensitive display. Herein, the driver can usually select between different views. For example, the so-called "bird eye view" is also available, i.e. a plan view representation showing the motor vehicle and the environment from a birds's eye view. Such a plan view representation can be generated by combining images of a plurality of cameras, which are disposed distributed on the motor vehicle.

It is an object of the invention to demonstrate a solution how in a method of the initially mentioned kind the driver of the motor vehicle can be supported in driving the motor vehicle even more reliably than in the prior art.

According to the invention, this object is solved by a method, by a camera system as well as by a motor vehicle having the features according to the respective independent claims. Advantageous implementations of the invention are the subject matter of the dependent claims, of the description and of the figures.

A method according to the invention serves for operating a camera system of a motor vehicle upon approach of the motor vehicle to a roadway, which extends transversely to the longitudinal axis of the motor vehicle. The camera system is switched to a supporting mode, in which an image of the roadway is displayed on a display device of the camera system, which is captured by means of the camera system from a point of view, which is disposed on the motor vehicle and is closer to the roadway than a point of view of the driver. The approach to the roadway is detected by means of an acquisition device of the motor vehicle, and the camera system is autonomously switched to the supporting mode by means of a control device due to the detection.

Such an approach has the advantage that the driver does not have to perform an operating input in order to switch the camera system to the supporting mode. Namely, the switching of the camera system to the supporting mode is effected automatically with the aid of a control device without a corresponding operating input of the driver having to be received. Thus, the driver can concentrate on the road situation and his attention is not distracted from the road situation. In this way, the driver is supported by the camera system even more reliably than in the prior art.

In the supporting mode, an image is presented on the display device, which is captured from a point of view, which is located on the motor vehicle on the one hand and is closer to the roadway than the point of view of the driver on the other hand. This in particular means that the image is captured by means of a camera, which is placed on the motor vehicle at a location, which is closer to the roadway (in the longitudinal direction of the vehicle) than the driver himself or the driver's seat or the steering wheel. The mentioned point of view is therefore a point of view of a camera, by means of which the image is captured. For example, this camera can be disposed on a front bumper.

Preferably, the camera is a CCD camera or a CMOS camera. It can be a video camera providing a sequence of images per second. These images are therefore provided in real time such that a video of the roadway can be presented on the display device. The camera can have a relatively wide opening angle, which can be in a range of values from 120 ° to 200 °. If only a single camera is attached to the front bumper, thus, it can be centrally disposed on the bumper and have an opening angle greater than 180°, namely for example 200°. If two cameras are placed on the bumper, thus, they can be positioned in lateral corner regions of the bumper. In this case, a smaller opening angle can also be selected.

The autonomous switching of the camera system to the supporting mode means that the switching to the supporting mode is effected without input of the driver and thus independent of the driver. To this, a control signal is generated by means of the control device, which then causes that the image of the roadway from the mentioned point of view is superimposed on the display device.

With respect to the detection of the approach to the roadway, very different embodiments can be provided. Therein, a situation is preferably detected, in which the motor vehicle moves forward and approaches a road, which extends transversely to the vehicle longitudinal axis and has the right of way as against the motor vehicle. Therein, an angle between the longitudinal axis of the motor vehicle on the one hand and the roadway on the other hand, which is in a range of values from 10° to 90 °, in particular in a range of values from 50° to 90°, preferably is 90°, is understood by the term "transversely". This can for example be the case at a junction, at which the motor vehicle has to yield right of way to other motor vehicles. However, this can also be the case upon exit from a garage, when the motor vehicle is driven out of a garage and moves towards a roadway. In this case, the sight to the road is restricted by the walls of the garage and the driver can orient himself by the images presented on the display device.

However, the invention is not restricted to a forward movement of the vehicle. The mentioned point of view can also be located at the rear of the vehicle if the vehicle is reversed and approaches a crossroad behind the vehicle. Here, for example, a rear camera can be used, which is disposed on the rear bumper or on a tailgate. In an embodiment, it is provided that the approach to the roadway is detected by evaluating a current operating state of a direction indicator. The switching to the supporting mode is then performed considering the current operating state of the direction indicator. Hereby, a situation can be reliably detected, in which the motor vehicle is located before a junction or else is driven out of a garage. Namely, the activation of the direction indicator suggests that the motor vehicle approaches a crossroad and the driver wants to turn left or right. Especially in such situations, the supporting mode proves particularly advantageous.

The consideration of the current operating mode of the direction indicator can be configured as follows: on the one hand, it can be provided that the switching to the supporting mode is effected every time the direction indicator is switched to the active operating state and thus activated. This means that the activation of the supporting mode is temporally coupled to the activation of the direction indicator, namely without additional requirements having to be satisfied. However, on the other hand, it can also be provided that the activation of the direction indicator presents at least one condition, on which switching of the camera system to the supporting mode can be effected. Here, further conditions can optionally also be defined, which are for example attached to other parameters.

Additionally or alternatively, it can be provided that the approach to the roadway is detected by evaluating the current velocity of the motor vehicle. The switching to the supporting mode is then performed considering the current velocity. This embodiment is based on the realization that the switching to the supporting mode is basically only advantageous with a low velocity. Namely, a low velocity or deceleration of the motor vehicle suggests that the driver of the motor vehicle has to yield right of way to other traffic participants. Considering the current velocity of the motor vehicle, the switching to the supporting mode can therefore be effected even more precisely and adequately.

Also with respect to the current velocity, two alternative embodiments can be provided: on the one hand, the switching to the supporting mode can be effected every time the current velocity falls below a preset limit value. Independently of other parameters, thus, the camera system is switched to the supporting mode every time the current velocity falls below the preset limit value. However, alternatively, it can also be provided that falling below the preset limit value presents only one condition for switching the camera system to the supporting mode. Optionally, other conditions can also be defined here, which can be attached to other parameters. Particularly preferably, therein, the evaluation of the current velocity is combined with the evaluation of the current operating state of the direction indicator. This can be configured such that the camera system is switched to the supporting mode if the direction indicator is switched to the active operating state on the one hand and the current velocity also falls below the preset limit value on the other hand.

Furthermore, additionally or alternatively, it can be provided that the detection of the approach includes that an image of the camera system is subjected to pattern recognition with respect to at least one preset road sign by means of the acquisition device, and thus it is searched for a preset road sign in the image of the camera system. The result of this examination can then be taken into account in switching the camera system to the supporting mode. By identification of road signs in the images of the camera system, situations can be reliably recognized, in which the motor vehicle approaches a crossroad and has to yield right of way to other traffic participants. Especially in such situations, the supporting mode is particularly advantageous for the diver.

For example, this can be configured such that the switching to the supporting mode is effected every time the preset road sign is identified in the image. Alternatively, the detection of the preset road sign can present at least one condition, on which the camera system is switched to the supporting mode. Optionally, here, additional conditions or criteria can also be defined. The detection of the road sign can thus either directly result in automatically switching to the supporting mode or else serve for making plausible the decision whether or not the camera system is to be switched to the supporting mode.

Preferably, the image is subjected to pattern recognition with respect to a STOP sign and/or a give way sign, which specifies that the driver has to yield right of way to other traffic participants and the detected roadway is a road with right of way. Then, it can be assumed that the driver will stop the motor vehicle and thus will require support by the display device.

As already explained, a situation can also be detected, in which the motor vehicle is driven out of a garage, namely directly to a road extending perpendicularly to the garage or a pedestrian way. In such situations too, the supporting mode proves particularly advantageous because the driver has only restricted sight to the road or the pedestrian way. Namely, this sight is restricted by the lateral walls of the garage. In order to detect such a situation, the acquisition device can evaluate distance values, which are acquired by means of at least one distance sensor of the motor vehicle. The switching to the supporting mode can then be performed considering the distance values. Therein, it can preferably be provided that the distance values specify a current distance to an obstacle located laterally beside the motor vehicle (walls of the garage). For example, this distance can be measured with the aid of an ultrasonic sensor. If the measured distance remains the same over a preset time interval such that deviation of the distance values over the time is below a preset limit value, the camera system can be switched to the supporting mode. Namely, such a constant distance suggests that the motor vehicle is driven parallel to a wall of the garage. Here too, further criteria can optionally be defined for switching to the supporting mode.

Furthermore, additionally or alternatively, the approach to the roadway can also be detected based on positional data of the motor vehicle, which is provided with the aid of a navigation system of the motor vehicle. Therein, the navigation system detects information about the respectively current geographic position of the motor vehicle, and this information can be combined with stored map data. Based on this data, it can then be checked if the motor vehicle approaches a crossroad and thus the supporting mode is to be activated. Based on stored map data, it can also be optionally checked if the driver of the motor vehicle has the right of way or else has to yield right of way.

In the supporting mode, thus, an image is displayed, which shows the roadway from a point of view located on the motor vehicle, which is between the driver and the roadway. Preferably, the displayed image shows the roadway from a direction of view, which extends transversely to the longitudinal axis of the motor vehicle and thus preferably parallel to the roadway. Thus, the driver can recognize on the display device without much effort whether or not obstacles are on the roadway.

In addition, the invention relates to a camera system for a motor vehicle, in particular a passenger car, wherein the camera system is formed for performing a method according to the invention. A motor vehicle according to the invention, in particular a passenger car, includes a camera system according to the invention.

The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the camera system according to the invention as well as to the motor vehicle according to the invention.

Further features of the invention are apparent from the claims, the figures and the description of figures. All of the features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations or else alone.

Now, the invention is explained in more detail based on a preferred embodiment as well as with reference to the attached drawings.

There show:

Fig. 1 in schematic illustration a motor vehicle with a camera system according to an embodiment of the invention;

Fig. 2 in schematic illustration an image, wherein a method according to an

embodiment of the invention is explained in more detail; and

Fig. 3 in schematic illustration a road situation with a motor vehicle, wherein a method according to an embodiment is explained in more detail.

A motor vehicle 1 illustrated in Fig. 1 is for example a passenger car. The motor vehicle 1 has a camera system 2 with for example four cameras 3, 4, 5, 6, which are disposed distributed on the motor vehicle 1 . The cameras 3, 4, 5, 6 overall capture the environment around the motor vehicle 1 . For example, the entire environment of the motor vehicle 1 and therefore a 360 ° image can be captured. Therein, the camera 3 is a front camera disposed in the front region of the motor vehicle 1 , for example on a front bumper 7. Thus, the camera 3 is disposed on a front of the motor vehicle 1 . The second camera 4 is for example a rear camera, which is mounted in the rear region of the motor vehicle 1 , for example on a rear bumper 8 or on a tailgate. The lateral cameras 5, 6 can for example be integrated in the respective exterior mirrors. The first camera 3 captures an environmental region 9 in front of the motor vehicle 1 . In the embodiment, the camera 3 is centrally disposed on the front bumper 7 and has a relatively wide opening angle a, which can for example be in a range of values from 170° to 200°. Therein, the camera axis of the camera 3 extends along a center longitudinal axis of the motor vehicle 1 and thus parallel to a vehicle longitudinal axis x. A vehicle coordinate system x, y is defined to the vehicle 1 .

For example, the camera 3 can be a fish-eye camera, which has such a wide opening angle a. Therein, the angle a is defined between two lines 10, 1 1 . Thus, the camera 3 also captures the environmental regions at the front laterally beside the motor vehicle 1 , namely both to the left and to the right. Alternatively to the camera 3, two cameras can also be attached to the front bumper 7, which can then be placed in the respective corner regions of the bumper 7. These cameras can then be oriented at an angle to the vehicle longitudinal axis x. If two cameras are used, thus, the respective opening angle can then also be selected smaller and for example be in a range of values from 100° to 180°.

Correspondingly, the second camera 4 captures an environmental region 12 behind the motor vehicle 1 . This camera 4 too can be centrally disposed on the rear bumper 8. The camera 4 also can have an opening angle, which is relatively wide and for example is in a range of values from 160° to 200°. The camera 4 too can thus be a fish-eye camera.

The lateral cameras 5, 6 respectively capture an environmental region 13 and 14, respectively, laterally beside the motor vehicle 1 .

The number as well as the arrangement of the cameras 3, 4, 5, 6 are only exemplarily illustrated in Fig. 1 and can vary according to embodiment.

For example, the cameras 3, 4, 5, 6 can be CCD cameras or CMOS cameras. The cameras 3, 4, 5, 6 are capturing devices detecting light in the visible spectral range and thus being able to provide images. The cameras 3, 4, 5, 6 can be video cameras, which are each able to provide a plurality of frames per second. The images are communicated to a central control device 15, which processes the images of all of the cameras 3, 4, 5, 6.

The control device 15 is coupled to an optical display device 16, which can for example be an LCD display. On the display 16, very different views can be presented, which can be selected according to the driving situation.

For example, the control device 15 can generate an overall presentation from the images of all of the cameras 3, 4, 5, 6, which shows the motor vehicle 1 and its environment 9, 12, 13, 14 from a bird's eye view and thus from a point of view, which is located above the motor vehicle 1 . Such a "bird eye view" is already prior art and can be generated by image processing by means of the control device 15.

The camera system 2 can also be switched to a supporting mode, in which the images or at least image regions of the camera 3 are presented on the display 16. Because the camera 3 is disposed on the front bumper 7, the point of view, from which the images are captured, are also located on the bumper 7, namely at the location, at which the camera 3 is placed. The point of view denoted by 17 in Fig. 1 therefore is directly on the front of the motor vehicle 1 .

An image 18 captured by means of the camera 3 is shown in schematic and abstract representation in Fig. 2. For the supporting mode, in particular, lateral edge regions 19, 20 are of interest, which correspond to respective edge regions a1 , a2 of the opening angle a according to Fig. 1 . The edge region 19 shows the environmental region 9 in front of and laterally beside the motor vehicle 1 from a direction of view, which extends substantially parallel to the vehicle transverse axis y. Correspondingly, the edge region 20 of the image 18 shows the environmental region 9 in front of and laterally to the right beside the motor vehicle 1 , wherein the direction of view here too is oriented substantially along the vehicle transverse axis y. In the supporting mode, these edge regions 19, 20 of the image 18 can be displayed together next to each other on the display 16, as it is schematically shown in Fig. 2. Therein, an image region 21 between the edge regions 19, 20 can be partially cut out, such that the two edge regions 19, 20 as well as a section 21 ' of the center region 21 are contained in the displayed image 18'. If two separate cameras are attached to the front bumper 7, thus, the images of these cameras can be displayed next to each other on the display 16.

In the supporting mode, thus, images 18' are displayed on the display 16, which allow the driver of the motor vehicle 1 to recognize objects, which he cannot notice himself due to restricted sight.

With reference again to Fig. 1 , the control device 15 can be coupled to a communication bus 22 of the motor vehicle 1 , to which other components of the motor vehicle 1 can also be connected, such as for example a navigation system 23 and/or a speedometer 24 and/or a direction indicator controller 25 and/or a sensor controller 26. The navigation system 23 communicates current positional data of the motor vehicle 1 to the

communication bus 22; the speedometer 24 communicates the current velocity of the motor vehicle 1 ; the direction indicator controller 25 communicates information about the current operating state of the direction indicators of the motor vehicle 1 ; the sensor controller 26 communicates current distance values, which are measured by means of at least one distance sensor 27, 28. Therein, ultrasonic sensors 27, 28 can be used, which are disposed in respective corner regions of the front bumper 7 and sense distances to obstacles along the vehicle transverse axis y. The control device 15 also represents an acquisition device, which picks up the information on the communication bus 22 and is able to detect a situation, in which the motor vehicle 1 approaches a roadway extending transversely to the vehicle longitudinal direction x. The control device 15 then activates the mentioned supporting mode (junction view), if any combination of the following criteria is satisfied: if the current velocity of the motor vehicle 1 falls below a preset limit value, for example 20 km/h or 15 km/h or 10 km/h; and/or

if at least one of the direction indicators is activated; and/or

if a preset road sign is identified in the images of the camera 3 or of the other cameras 5, 6, 4; and/or

if the measured distance values of at least one of the sensors 28, 27 remain constant for a preset time interval and herein deviation (standard deviation) of these distance values over the time is smaller than a present limit value; and/or if it is recognized based on positional data of the navigation system 23 that the motor vehicle 1 approaches a junction, at which the driver of the motor vehicle 1 has to yield right of way.

Now, these criteria can be arbitrarily combined with each other. The supporting mode can for example already be activated if one of the criteria is satisfied. However, plausibility check can also be performed such that for example two criteria are combined with each other. For example, it can be provided that the criterion with respect to the velocity as well as the criterion with respect to the activated direction indicator are combined with each other. In this case, the supporting mode can be automatically activated if the current velocity of the motor vehicle 1 falls below the limit value and additionally a direction indicator is also activated.

An exemplary road situation with the motor vehicle 1 is illustrated in Fig. 3. The motor vehicle 1 moves on a road 29 towards a junction 30, at which the road 29 is merged with another roadway 31 . Therefore, the road 29 is disposed perpendicularly and thus at a right angle to the roadway 31 . Thus, the roadway 31 also extends perpendicularly to the vehicle longitudinal direction x. In the situation according to Fig. 3, the control device 15 recognizes that the driver reduces the velocity on the one hand and a direction indicator 32 is activated on the other hand. If this is detected, thus, the control device 15 automatically activates the supporting mode, in which the images 18' according to Fig. 2 are presented on the display 16. For plausibility check, the control device 15 also can subject the images of the camera 3 to pattern recognition and herein identify a give way sign 33, which specifies that the driver of the motor vehicle 1 has to yield right of way to the other traffic participants. The control device 15 can also identify road signs 34 in the images, which are applied to the road 29 itself. In the example according to Fig. 3, the STOP sign 34 is identified. Thereby, the switching to the supporting mode is made plausible.

If the motor vehicle 1 is directly at the junction 30, thus, the sight to the roadway 31 is restricted by obstacles 35. However, the driver can reliably recognize in the presented images 18' if other vehicles are on the roadway 31 and come from the right.

Claims

1 . Method for operating a camera system (2) of a motor vehicle (1 ) upon approach of the motor vehicle (1 ) to a roadway (31 ) extending transversely to the longitudinal axis (x) of the motor vehicle (1 ), wherein the camera system (2) is switched to a supporting mode, in which an image (18') of the roadway (31 ) is displayed on a display device (16) of the camera system (2), which is captured by means of the camera system (2) from a point of view (17), which is disposed on the motor vehicle (1 ) and is closer to the roadway (31 ) than a point of view of the driver of the motor vehicle (1 ),

characterized by

- detecting the approach to the roadway (31 ) by means of an acquisition device (15) of the motor vehicle (1 ), and

- autonomously switching the camera system (2) to the supporting mode by means of a control device (15) due to the detection.

2. Method according to claim 1 ,

characterized in that

the approach to the roadway (31 ) is detected by evaluating the current operating state of a direction indicator (32) and the switching to the supporting mode is performed considering the current operating state of the direction indicator (32).

3. Method according to claim 2,

characterized in that

- the switching to the supporting mode is effected at least on condition that the direction indicator (32) is switched to an active operating state, or

- the switching to the supporting mode is effected every time the direction

indicator (32) is switched to the active operating state.

4. Method according to any one of the preceding claims,

characterized in that

the approach to the roadway (31 ) is detected by evaluating the current velocity of the motor vehicle (1 ) and the switching to the supporting mode is performed considering the current velocity.

5. Method according to claim 4,

characterized in that

- the switching to the supporting mode is effected at least on condition that the current velocity falls below a preset limit value, or

- the switching to the supporting mode is effected every time the current velocity falls below the preset limit value.

6. Method according to any one of the preceding claims,

characterized in that

the detection of the approach includes that an image (18) of the camera system (2) is subjected to pattern recognition with respect to at least one preset road sign (33, 34) by means of the acquisition device (15).

7. Method according to claim 6,

characterized in that

- the switching to the supporting mode is effected at least on condition that the preset road sign (33, 34) is identified in the image (18), or

- the switching to the supporting mode is effected every time the preset road sign (33, 34) is identified in the image (18).

8. Method according to claim 6 or 7,

characterized in that

the image (18) is subjected to pattern recognition with respect to a stop sign (34) and/or a give way sign (33), which specifies that the roadway (31 ) is a roadway with right of way.

9. Method according to any one of the preceding claims,

characterized in that

the approach to the roadway (31 ) is detected by evaluating distance values, which are sensed by means of at least one distance sensor (27, 28) of the motor vehicle (1 ), wherein the switching to the supporting mode is performed considering the distance values.

10. Method according to claim 9,

characterized in that

the distance values specify a current distance to an obstacle located laterally beside the motor vehicle (1 ), wherein

- the switching to the supporting mode is effected every time a deviation of the distance values from each other is below a preset limit value over a preset time interval, or

- the switching to the supporting mode is effected at least on condition that the deviation of the distance values from each other is below the preset limit value over the preset time interval.

1 1 . Method according to any one of the preceding claims,

characterized in that

the approach to the roadway (31 ) is detected by evaluating positional data of the motor vehicle (1 ), which is provided by means of a navigation system (23) of the motor vehicle (1 ).

12. Method according to any one of the preceding claims,

characterized in that

the image (18') presented on the display device (16) in the supporting mode shows the roadway (31 ) from a direction of view extending transversely to the longitudinal axis (x) of the motor vehicle (1 ).

13. Camera system (2) for a motor vehicle (1 ), wherein the camera system (2) is formed for performing a method according to any one of the preceding claims.

14. Motor vehicle (1 ) with a camera system (2) according to claim 13.