WO2025031546A1 - Method for the creation of a representation of surroundings representing vehicle surroundings, control device, driver assistance system, and vehicle - Google Patents

Abstract

A method for creating a representation (15) of surroundings representing vehicle surroundings, involving the steps of: - providing camera images (12) captured by one or more vehicle (1) cameras (3), each of said camera images (12) representing at least a portion of the vehicle surroundings; - producing three-dimensional surroundings data from at least one of the camera images (12), the at least one camera image (12) having been captured at a first capturing time; - identifying a concealed sector (14) of the surroundings for at least one of the cameras from the surroundings data; - acquiring image data for filling in the concealed sector (14) of the surroundings, the image data being acquired from at least one additional camera image captured by another one of the cameras (2) at the first capturing time and/or from at least one stored camera image captured by the one or more cameras (3) at a second capturing time that precedes the first capturing time, creating the representation of the surroundings from the at least one camera image (12) and from the image data, the image data being used to represent, in the representation (15) of the surroundings, at least a portion of the sector (14) of the surroundings that is concealed in the at least one camera image (12).

Description

Description

Method for generating an environmental representation representing a vehicle environment, control device, driver assistance system and vehicle

The invention relates to a method for generating an environmental representation representing a vehicle environment. The invention also relates to a control device, a driver assistance system and a vehicle.

Modern vehicles are increasingly being equipped with driver assistance systems that support the driver in carrying out driving maneuvers by displaying a view of the surroundings. Such environmental views can, for example, be so-called surround view displays, which show at least part of the vehicle's surroundings. Such an environmental view is usually generated on the basis of camera images recorded using one or more of the vehicle's cameras.

The representation of the surrounding area can be based on a virtual camera position that differs from the actual arrangement position of the cameras on the vehicle. For this purpose, the camera images can be projected onto a virtual projection surface, for example, whereby the projection surface is used to determine a view of the surroundings from a different point of view. However, it can happen that individual areas of the vehicle's surroundings are obscured by objects in the vehicle's surroundings.

DE 10 2015 223 176 A1 discloses a driver assistance system for a vehicle with environmental sensors that use sensors to detect the vehicle's surroundings. The sensor data from the environmental sensors is evaluated to detect obstacles in the vehicle's surroundings, and masking areas in the vehicle's surroundings are determined depending on the obstacles detected. The obscuration areas are covered by the obstacles and limit the field of view of the driver assistance system's optical environmental sensors. The determined obscuration areas can be covered with textured surfaces when the camera images are displayed.

DE 10 2018 203 590 A1 describes a surround view system for a vehicle. An evaluation unit of the surround view system is used to detect an object in recorded environmental data and to determine the 3D shape of the object. A projection surface used to generate a surround view is then supplemented with the specific 3D shape of the object. Image areas hidden by the object can also be filled with a predefined color, with an interpolation of surrounding areas and/or with a mirroring of environmental data of the front of the object.

In order to provide the driver with as complete an overview of the vehicle's surroundings as possible, it is desirable that when the environment is displayed from a virtual camera position that does not correspond to the actual camera position, there are as few areas as possible in which the actual vehicle surroundings are not displayed.

The invention is based on the object of specifying an improved method for generating an environmental representation representing a vehicle environment, which in particular avoids or at least reduces the occurrence of areas without a representation of the actual vehicle environment.

This object is achieved according to the invention by a method for generating an environmental representation representing a vehicle environment, the method comprising the following steps:

Providing camera images recorded with one or more cameras of the vehicle, wherein at least part of the vehicle environment is shown in the camera images, - generating three-dimensional environmental data from at least one of the camera images, wherein the at least one camera image was recorded at a first recording time,

- Determining a hidden environmental area for at least one of the cameras from the environmental data,

- Determining image data for at least partially filling the concealed surrounding area, wherein the image data are determined from at least one further camera image of another of the cameras recorded at the first recording time and/or from at least one stored camera image of the one or more cameras recorded at a second recording time prior to the first recording time,

- generating the environment representation from the at least one camera image and the image data, wherein at least a part of the environment region hidden in the at least one camera image is represented in the environment representation using the image data.

The camera images used to generate the representation of the surroundings can be generated, for example, by one or more cameras in the vehicle and transmitted to a control device designed to carry out the method via a communication connection to provide the camera images. The camera images can in particular be generated and transmitted continuously. At least some of the camera images can also be stored in a storage device in the vehicle, at least temporarily. In this way, camera images recorded at previous recording times are also available for generating the representation of the surroundings.

The part of the vehicle surroundings depicted in the camera images depends on the position of the camera generating the respective camera images on the vehicle. It is possible that several cameras arranged at different positions on the vehicle are used to generate the camera images. For example, at least one camera on the front of the A front camera arranged on the vehicle, one or more side cameras arranged on the side of the vehicle and/or at least one rear view camera arranged on the rear of the vehicle can be used. According to the invention, several cameras of a camera arrangement designed as a surround view system of the vehicle can be used.

In particular, a surround view can be generated as the environment representation. The environment representation can be generated in particular starting from a virtual camera position, which differs from the actual arrangement positions of the one or more cameras on the vehicle. The change in perspective can be achieved by projecting the camera images onto a projection surface, for example a bowl-shaped projection surface that virtually surrounds the vehicle, whereby the environment representation can then be derived from the projected camera images starting from the virtual camera position.

By changing the perspective of the display from the actual camera position from which the camera images are generated to the position of the virtual camera, it can happen that image areas in the camera images that are hidden by an object in the environment are no longer hidden from the perspective of the virtual camera. However, for such a hidden environment area, the camera image does not contain any information describing the vehicle environment within the hidden environment area.

In order to determine these hidden environmental areas, three-dimensional environmental data is first generated from at least one of the camera images, with the at least one camera image having been recorded at a first recording time. The three-dimensional environmental data describes the vehicle environment, in particular in a three-dimensional coordinate system. In particular, the three-dimensional environmental data can describe the geometry of environmental objects and the respective position of the environmental objects in relation to the vehicle. Using the environmental data, a hidden environmental area can then be determined. The hidden environmental area refers to one of the cameras and describes the part of the vehicle's surroundings that is hidden from the camera by an object and therefore cannot be seen. In other words, the hidden environmental area assigned to one of the cameras is a description of the part of the vehicle's surroundings for which no image information is contained in the camera images recorded with the respective camera.

Image data is then determined to fill in the hidden surrounding area. The image data used to fill in the hidden surrounding area in particular shows at least part of the vehicle surroundings within the hidden surrounding area. In other words, the image data used to fill in the hidden surrounding area contains a description of at least part of the actual vehicle surroundings in the hidden image area, so that they can be used in the environment view to display the hidden surrounding area. The image data can describe the entire hidden surrounding area or image data can be determined which only describes one or more sub-areas of the hidden surrounding area.

The image data for filling the concealed surrounding area can be taken from at least one camera image taken at the first recording time from another camera. This can be the case, for example, if the vehicle has several cameras, each of which takes camera images, with the detection areas of the cameras and thus also the surrounding areas shown in the camera images partially overlapping. The image data can be taken, for example, from one of the overlapping areas of a camera image from the other camera. In addition or alternatively, the invention allows image data from a stored camera image recorded at a second recording time to be used. The second recording time is before the first recording time, so that the image data is taken from a previously recorded camera image, which can also be referred to as a historical camera image. The stored camera image can be stored in a storage device in the vehicle, for example.

The previously recorded camera image can be used if an image area is covered by a static surrounding object, especially if the vehicle has moved between the first recording time and the second recording time. If there are a number of stored camera images, a stored camera image can be selected that was recorded from a different position of the vehicle, and thus also from a different camera position.

The stored camera image, the image data of which is used to replace the hidden surrounding area determined for one of the cameras, can have been recorded by the same camera whose hidden surrounding area is to be replaced. In addition or as an alternative, a stored camera image recorded by another camera in the vehicle can also be used.

The environment representation is then generated from the at least one camera image and the image data. In this case, at least a part of the surrounding area hidden in the at least one camera image is represented in the environment representation using the image data. In other words, in the environment representation, the hidden surrounding area is at least partially filled with the image data of another of the cameras and/or with image data recorded at an earlier point in time. This means that the environment representation contains additional information about the actual vehicle environment, which is available when the

Environment representation from only one camera image or only from several, at their camera images merged in overlapping areas would not be included. The use of image data from a previously recorded camera image represents at least an approximation of the current vehicle surroundings, especially if the first recording time and the second recording time are only a short period of time apart.

The representation of the surroundings can be shown visibly to a driver, in particular on a display device of the vehicle, in particular on a display arranged in a vehicle interior. In this way, a driver of the vehicle is provided with assistance, for example when carrying out driving maneuvers in a confined space.

The method according to the invention can advantageously lead to an improved perception of the situation by the driver. By inserting the image data into the representation of the environment, additional information or additional context is provided, which can make it easier for the driver to drive the vehicle. In particular, gaining an understanding of the situation is made easier, which advantageously allows the driver to react more quickly and/or better to the current traffic situation in the vehicle's surroundings.

By filling in hidden areas of the vehicle's surroundings, the occurrence of blind spots can be prevented or at least reduced, which has the advantage of increasing road safety. The driver's better understanding of the situation can reduce the risk of collisions or accidents due to areas of the surroundings that cannot be seen.

In particular, the insertion of image data from a stored camera image makes it possible to implement additional functions such as object tracking, object recognition and/or augmentation of the environment representation with additional information when evaluating the three-dimensional vehicle environment, which can be carried out, for example, on the basis of the generated environment representation. The implementation of further This simplifies functions that depend on the most complete possible understanding of the current traffic situation.

Particularly in the case of static environmental objects, valuable information regarding the presence and position of objects, landmarks and/or obstacles can also be obtained from previously recorded camera images if these are combined with a current camera image of the vehicle's surroundings in the environmental representation.

By reducing hidden image areas in which no image content or only a schematic image content such as a static color or the like is displayed, it is also achieved that the vehicle surroundings can be continuously reproduced in a consistent manner. This particularly simplifies the detection of the surroundings when dynamic conditions, such as changing lighting conditions and/or dynamic objects in the surroundings of the vehicle, are present. In such cases, the method according to the invention can advantageously be used to generate a consistent representation of the surroundings that essentially corresponds to human perception and can be viewed by a driver of the vehicle.

According to the invention, it can be provided that it is first checked whether the image data for filling the concealed surrounding area are present in at least one further camera image of another of the cameras recorded at the first recording time, wherein if the image data from the at least one further camera image are present, these image data are used to generate the representation of the surroundings and wherein if the image data from the at least one further camera image are not present, the image data from the at least one stored camera image are used to generate the representation of the surroundings.

This two-stage procedure advantageously allows for checking whether there are any current camera images, i.e. images taken at the same time as the first image, which provide the image data needed to fill in the hidden surrounding area. This can be done, for example, using the three-dimensional surrounding data or the hidden surrounding areas derived from it, if these were determined for several cameras in the vehicle. If such current image data is available, it can be used preferentially to fill in the hidden surrounding area.

If there are no further camera images from another camera at the first recording time from which the image data for the hidden surrounding area could be taken, then it can be checked whether there are any stored camera images recorded by the camera or another camera of the vehicle at an earlier, second recording time which contain the image data for filling the hidden surrounding area. Accordingly, the image data from the at least one stored camera image can then be used to generate the view of the surroundings.

In the case of an environmental representation which is formed from several camera images recorded at the first recording time, it can be checked for each of the camera images whether the image data of another camera at the first recording time or historical image data which are taken from a camera image recorded at a second recording time are used for a possibly existing hidden environmental area.

In a preferred embodiment of the invention, it can be provided that the image data for filling the concealed surrounding area are determined on the basis of the surrounding data, wherein the concealed surrounding area of the at least one camera at the first recording time is compared with the concealed surrounding area of the at least one further camera and/or with concealment information associated with the stored camera image, which describes the concealed surrounding area of the camera generating the stored camera image at the second recording time. By comparing the hidden surrounding areas for two of the cameras, it is easy to determine whether image data for the hidden surrounding area in the camera images of one of the cameras is contained in the camera images of the other camera. This is the case if the surrounding area hidden for one camera is not hidden for the other camera. This applies equally to the comparison of two camera images recorded at the first recording time and to the comparison between a camera image recorded at the first recording time and a stored camera image recorded at a second recording time.

In addition or as an alternative, it is possible for the image data to be determined using masking information that is associated with a stored camera image. The masking information can describe which part of the vehicle's surroundings is masked in the stored camera image, so that checking whether the camera image contains image data for filling a masked area of the surroundings can be simplified. The masking information can be determined, for example, at or shortly after the time the camera image is recorded and stored with the camera image in a storage device.

According to the invention, it can be provided that the environmental data are determined as a function of depth information associated with the camera image, as a function of map information describing the position of the vehicle and the vehicle environment and/or as a function of the measurement data of at least one distance sensor of the vehicle. The three-dimensional environmental data can be generated from the camera image by determining depth information from the image content reproduced in the camera image.

In addition or as an alternative, it is possible for the three-dimensional environment data to be determined as a function of map information describing the vehicle environment. The map information can, for example, by means of a method such as simultaneous localization and mapping (SLAM) and/or comparable methods.

It is also possible for the three-dimensional environmental data to be determined as a function of the measurement data from at least one distance sensor of the vehicle. Using the distance sensor, a distance between the sensor or the vehicle and an object in the vehicle's surroundings can be determined. The distance sensor can be designed, for example, as a radar sensor, as a lidar sensor and/or as an ultrasonic sensor.

According to the invention, the environment representation can be generated starting from a virtual camera position, wherein the image data for representing at least part of the concealed environment area is transformed depending on the virtual camera position. Because the image data can be taken from a camera image whose perspective differs from the perspective starting from the virtual camera, an improved reproduction of the actual vehicle environment can be achieved by transforming the image data to fill in the concealed environment area in the environment representation. By transforming the image data, the difference in perspective can be at least partially compensated, resulting in a coherent overall image.

In a preferred embodiment of the invention, it can be provided that the image data and the at least one camera image are combined to generate the view of the surroundings depending on at least one image harmonization function. With the help of the image harmonization function, for example, differences in exposure, color and/or resolution between the camera image and the inserted image data can be at least partially compensated. By using at least one image harmonization fusion, a more harmonious overall image can thus be achieved in the representation of the surroundings. According to the invention, it can be provided that several cameras of a camera arrangement designed as a surround view system of the vehicle are used. Accordingly, the representation of the environment can preferably be a surround view representation which, starting from a virtual camera position, reproduces, for example, a virtual model of the vehicle as well as the part of the vehicle surroundings visible from the position of the virtual camera.

The invention further relates to a control device which is designed to carry out a method according to the invention. The step of providing the camera images can be carried out, for example, by the control device being designed to receive the camera images via a communication connection and/or to retrieve the camera images from a storage device in the vehicle.

For a driver assistance system according to the invention, it is provided that it comprises at least one camera which is designed to provide camera images representing at least part of a vehicle environment, as well as a control device set up to carry out a method according to the invention.

For an invention according to the vehicle it is provided that it has a driver assistance system according to the invention.

All advantages and configurations described above in relation to the method according to the invention also apply accordingly to the control device according to the invention, the driver assistance system according to the invention and the vehicle according to the invention and vice versa. The advantages and configurations described in relation to the control device according to the invention, the driver assistance system according to the invention and the vehicle according to the invention can also be transferred analogously to the other objects. Further advantages and details of the invention emerge from the embodiments described below and from the drawings. These are schematic representations and show:

Fig. 1 shows an embodiment of a vehicle according to the invention comprising an embodiment of a driver assistance system according to the invention,

Fig. 2 is a block diagram of an embodiment of a method according to the invention,

Fig. 3 shows an example of a camera image taken with a camera of the vehicle and

Fig. 4 shows an embodiment of an environment representation generated using the method according to the invention.

Fig. 1 shows an embodiment of a vehicle 1. The vehicle 1 can be, for example, a motor vehicle, in particular a

The vehicle 1 can be a passenger car, a truck or another type of commercial vehicle. It is also possible that the vehicle 1 is a non-motorized vehicle such as a trailer or that it is a combination of a towing vehicle and a trailer. The vehicle 1 can also be a rail-bound vehicle such as a tram or the like, or a robot.

The vehicle 1 comprises an embodiment of a driver assistance system 2, which comprises one or more cameras 3 and a control device 4. In the present embodiment, the vehicle 1 comprises four cameras 3, which form a camera arrangement designed as a surround view system. A first camera 5 is arranged as a front camera of the vehicle 1, a second camera 6 as a rear view camera of the vehicle 1, and a third camera 7 and a fourth camera 8 each as a side camera of the vehicle 1. The Control device 4 can be designed, for example, as a microcontroller, as a processor or as another type of computing device.

The driver assistance system 2 further comprises a display device 9, via which graphic information can be displayed for a driver or a user of the vehicle 1. The display device 9 can, for example, be one or more screens or displays arranged in an interior of the vehicle 1.

The vehicle 1 also comprises at least one distance sensor 10. The distances to objects, in particular to objects arranged in front of, next to and behind the vehicle, can be determined via the at least one distance sensor 10. The distance sensor 10 can be designed, for example, as a radar sensor, as a lidar sensor and/or as an ultrasonic sensor. The vehicle 1 can in particular have a plurality of distance sensors 10, which can be designed as any combination of different sensor types and can be arranged at different positions on the vehicle 1.

The cameras 3 can be used to generate image data from a sub-area of the surroundings of the vehicle 1. The cameras 3 and the at least one distance sensor 10 are connected to the control device 4 via a communication connection 11. The communication connection 11 can, for example, comprise a plurality of point-to-point connections or be a bus connection such as a CAN bus or the like. Image data or camera images are transmitted from the cameras 3 to the control device 4 via the communication connection 11, in particular continuously.

The control device 4 is designed to carry out a method for generating an environmental representation representing the vehicle's surroundings. The control device 4 uses several camera images that were recorded by the cameras 3 of the vehicle 1. In order to gain access to camera images recorded at an earlier point in time or at an earlier vehicle position, at least some of the recorded camera images can be stored in the vehicle 1, for example in a storage device of the control device 4 and/or in a storage device of the respective camera. The storage device can be, for example, a temporary data storage device such as a working memory or a permanent data storage device such as a hard disk.

Fig. 2 shows a block diagram of an embodiment of a method for generating an environmental representation representing the vehicle environment.

In a first step S1 of the method, several camera images are provided which were recorded using the cameras 3 of the vehicle 1, wherein the camera images each show a partial area of the vehicle's surroundings. The camera images can be current camera images which are generated at a first recording time and transmitted to the control device. Additionally or alternatively, historical camera images can also be provided which were recorded at a second recording time before the first recording time and which are stored in the vehicle 1 so that they can be retrieved by the control device 4.

Subsequently, in a step S2, three-dimensional environmental data is generated from at least one of the camera images that was recorded at a first recording time. The three-dimensional environmental data describe objects and/or features of the environmental structure with regard to their geometric shape and their arrangement, in particular in relation to the position of the vehicle 1.

The environmental data can be derived from the image content of at least one camera image. The environmental data can be determined by the control device 4, which, for example, uses one or more algorithms to determine the image content of one or more of the camera images. can be evaluated. The image content can be evaluated using methods such as semantic segmentation, structure-from-motion, SLAM or similar methods.

The environmental data can be determined depending on depth information associated with the camera image. The depth information can indicate which image areas have which depth, i.e. which distance from the vehicle 1. The depth information can, for example, be determined by the control device 4 using one or more of the algorithms, as described above. For stored camera images, it is possible that corresponding depth information for the stored camera images is also stored in a storage device so that it can be retrieved by the control device 4.

It is also possible that the environmental data are determined depending on map information describing the position of the vehicle and the vehicle surroundings. The map information can contain information about the vehicle surroundings as well as the position of the vehicle within the vehicle surroundings. A

Map information can in particular be continuously updated depending on the camera images from the cameras 3 and/or the measurement data from the distance sensor 10.

It is also possible for the measurement data of the distance sensor 10, which describe the distances of objects in the surroundings of the vehicle 1 to the vehicle 1, to be used to determine the environmental data. For this purpose, the measurement data of the distance sensor 10 can be transmitted from the distance sensor 10 to the control device 4 via a communication connection.

Subsequently, in a step S3 of the method, hidden environmental areas are determined for the individual cameras 3 of the vehicle 1 from the previously determined three-dimensional environmental data. In particular, the areas are identified in which a part of the The vehicle surroundings cannot be seen due to being obscured by an object in the surrounding area.

Fig. 3 shows an example of a camera image 12 in which a part of the vehicle surroundings is obscured by an object 13. The camera image 12 can, for example, have been recorded by one of the side cameras 7, 8 of the vehicle 1. The obscured image area 14 comprises those sub-areas which lie behind the objects 13 designed as posts. There is no image information in the camera image 12 for the areas of the vehicle surroundings which lie behind the objects 13. The obscured image areas 14 each relate to a camera image 12 or to one of the cameras 3, so that the obscured image areas 14 can differ for different cameras 3.

In step S4, image data is determined to fill in the hidden surrounding area 14. In a first sub-step S4A, the image data can be taken from at least one additional camera image recorded at the first recording time from one of the additional cameras 3 of the vehicle 1. For example, a camera image from the front camera 5 or the rear view camera 6, also recorded at the first recording time, can be used if they capture the hidden surrounding area 14 of the side cameras 7, 8 uncovered due to an overlap of the detection areas with the side cameras 7, 8.

In a sub-step S4B, the image data can be determined from at least one stored camera image of the one or more cameras 3 of the vehicle 1, recorded at a second recording time before the first recording time. In this sub-step, the image data are therefore taken from a historical camera image that was recorded at an earlier time. This stored camera image can be loaded by the control device 4 from a storage device of the vehicle 1. The stored camera image can be recorded by the same camera as the camera image showing the hidden surrounding area. or it can be a stored camera image taken by another of the cameras 3.

The sub-steps S4A and S4B can be carried out alternatively or one after the other. In particular, it can first be checked whether the image data for filling the concealed surrounding area 14 are present in at least one further camera image recorded at the first recording time from another of the cameras 3, wherein if the image data from the at least one further camera image are present, these image data are then used to generate the representation of the surroundings.

The image data for filling the hidden surrounding area can be determined, for example, using the three-dimensional surrounding data. The hidden surrounding area 14 of the at least one camera 3 can be compared with the hidden surrounding area of the at least one other camera at the first recording time. In this way, it can be checked whether image data for the hidden surrounding areas 14 are present in the other camera image.

If another camera image from the first recording time is not available or if the image data cannot be obtained from the other existing camera images, a stored camera image can then be determined and used to generate the representation of the surroundings. The stored camera image can in particular be determined in such a way that it depicts the vehicle surroundings starting from a different camera position. This can be the case, for example, if the vehicle has moved between the second recording time and the first recording time. This means that image data for the surrounding areas 14 hidden by the camera image recorded at the first recording time are present in the stored camera image.

When using a stored camera image, the image data can be

Filling the hidden surrounding area 14, for example, with a The masking information can be determined from the masking information associated with the stored camera image. The masking information can describe the masked surrounding area of the camera 3 generating the stored camera image at the second recording time. The masking information can be used to compare the masked surrounding area of the stored camera image and the other camera image. The masking information can be associated with the stored camera image, for example, and also stored in the storage device. It is possible that the masking information was already generated when the stored camera image was recorded and stored in the storage device for later use.

Finally, in a step S5, the environment representation is generated on the basis of the at least one camera image 12 and the image data used to fill the hidden environment area. The image data previously determined in step S4 are inserted into the hidden environment area 14 so that, in particular when the environment representation is displayed starting from the position of a virtual camera, it can be displayed at least partially with image data that at least approximately describe the actual environment of the vehicle 1.

Fig. 4 shows an environmental representation 15. The environmental representation 15 was generated based on a virtual camera position, so that the viewing angle of the vehicle's surroundings does not depend on an actual position of one of the cameras 3. This makes it possible for the vehicle 1, or a virtual model of the vehicle 1, to also be shown in the environmental representation. The image data for representing at least part of the hidden environmental area 14 can be transformed in particular depending on the virtual camera position in order to compensate for the difference in perspective.

In the environment representation 15 shown in Fig. 4, the image data were used to identify the areas covered by the objects 13 formed as posts. Fill in sub-areas of the hidden surrounding area 14 with data about the actual vehicle surroundings. Using the image data enables the driver to view a complete image of the vehicle surroundings that has no or at least fewer gaps and/or black areas or areas not filled with image data. Preferably, image data for all hidden areas 14 are determined for the environment representation 15 so that a complete, gap-free reproduction of the environment is possible. In the event that image data is not available for all hidden surrounding areas 14, the remaining hidden sub-areas for which no image data is available can be shown in black, for example, or filled in by extrapolating the respective surrounding image areas.

Furthermore, it is possible for the image data and the at least one camera image 12 to be combined to generate the environmental representation 15 depending on at least one image harmonization function. With the help of image harmonization, for example, different brightnesses, different color saturations and the like can be compensated for in order to generate a harmonious environmental representation 15.

The environmental representation 15 shown in Fig. 4 is purely exemplary; in particular, other types of environmental representations can also be generated, for example environmental representations 15 recorded at a different angle or a different virtual camera position. For example, it is possible that a top view, which shows a bird's eye view of the vehicle 1, is also generated as the environmental representation 15.

The environment display 15 can be shown to a driver of the vehicle 1 on the display device 9 so that it can be viewed by the driver 1 for carrying out vehicle maneuvers. This significantly simplifies the execution of precise driving maneuvers, especially maneuvers carried out in a confined space such as parking maneuvers. In addition, the environment display 15 can also be used for other driver assistance functions of the Vehicle 1, which include, for example, tracking objects, landmarks or obstacles in the vicinity of the vehicle 1.

Claims

patent claims 1 . Method for generating an environmental representation (15) representing a vehicle environment, comprising the steps: - providing camera images (12) recorded with one or more cameras (3) of the vehicle (1), wherein at least a part of the vehicle environment is shown in each of the camera images (12), - generating three-dimensional environmental data from at least one of the camera images (12), wherein the at least one camera image (12) was recorded at a first recording time, - determining a hidden environmental area (14) for at least one of the cameras from the environmental data, - determining image data for at least partially filling the concealed surrounding area (14), wherein the image data are determined from at least one further camera image of another of the cameras (3) recorded at the first recording time and/or from at least one stored camera image of the one or more cameras (3) recorded at a second recording time prior to the first recording time, - generating the environment representation from the at least one camera image (12) and the image data, wherein at least a part of the environment region (14) hidden in the at least one camera image (12) is represented in the environment representation (15) using the image data. 2. Method according to claim 1, characterized in that it is first checked whether the image data for filling the concealed surrounding area (14) are present in at least one further camera image recorded at the first recording time from another of the cameras (3), wherein if the image data from the at least one further camera image, these image data are used to generate the environmental representation (15), and wherein, if the image data from the at least one further camera image are not present, the image data from the at least one stored camera image are used to generate the environmental representation (15). 3. Method according to claim 1 or 2, characterized in that the image data for filling the concealed surrounding area (14) are determined on the basis of the surrounding data, wherein the concealed surrounding area (14) of the at least one camera (3) at the first recording time is compared with the concealed surrounding area (14) of the at least one further camera (3) and/or with concealment information associated with the stored camera image, which describes the concealed surrounding area (14) of the camera (3) generating the stored camera image at the second recording time. 4. Method according to one of the preceding claims, characterized in that the environmental data are determined as a function of depth information associated with the camera image (12), as a function of map information describing the position of the vehicle (1) and the vehicle environment and/or as a function of the measurement data of at least one distance sensor (10) of the vehicle (1). 5. Method according to one of the preceding claims, characterized in that the environmental representation (15) is generated starting from a virtual camera position, wherein the image data for representing the at least part of the hidden environmental area (14) are transformed depending on the virtual camera position. 6. Method according to one of the preceding claims, characterized in that the image data and the at least one camera image (12) are combined to generate the environmental view (15) depending on at least one image harmonization function. 7. Method according to one of the preceding claims, characterized in that several cameras (3) of a camera arrangement designed as a surround view system of the vehicle (1) are used. 8. Control device arranged to carry out a method according to one of the preceding claims. 9. Driver assistance system comprising at least one camera (3) which is designed to provide camera images representing at least part of a vehicle environment, and a control device (4) according to claim 8. 10. Vehicle comprising a driver assistance system (2) according to claim 9.