DE102022211267A1 - Method and system for a motor vehicle for checking the timeliness of an image of a vehicle environment of the motor vehicle by means of a digital stamp - Google Patents

Abstract

The invention relates to a method and a system for checking the timeliness of an image 1 of a vehicle environment 2 of a motor vehicle 3 by means of a digital stamp. The method comprises the steps:
a) generating the image (1) by means of an optical sensor unit (4) of the motor vehicle (3);
b) introducing a digital stamp (5) into the image (1) by means of the optical sensor unit (4), wherein the stamp (5) is introduced according to a time-dependent rule;
c) transmitting the image (1) with the stamp (5) from the optical sensor unit (4) to a receiving unit (6) of the motor vehicle (3);
d) Checking the timeliness of the stamp (5) of the transmitted image (1) based on the time-dependent rule by means of the receiving unit (6).

Description

The invention relates to a method for checking the timeliness of an image of a vehicle environment of a motor vehicle. The invention further relates to an associated system which comprises an optical sensor unit for generating the image and a receiving unit for receiving the image, as well as an associated motor vehicle.

Transmission errors can occur on a transmission path, for example between an optical sensor unit of a motor vehicle and a receiving unit of the motor vehicle, which can be caused, for example, by interference in the transmission path. This means that incorrect, temporally delayed or static images can be transmitted to the receiving unit. It is therefore not impossible that a driver, a driver assistance system or an electronic vehicle guidance system makes a decision based on the incorrect information, which can cause a dangerous situation.

The invention is based on the object of providing a method, a system and a vehicle by means of which it can be ensured that a received image is current.

The object is achieved by the subject matter of the independent patent claims. Advantageous developments of the invention are described by the dependent patent claims, the following description and the figures.

1. a) Erzeugen des Bildes mittels einer optischen Sensoreinheit des Kraftfahrzeuges;
2. b) Einbringen eines digitalen Stempels in das Bild mittels der optischen Sensoreinheit, wobei der Stempel gemäß einer zeitabhängigen Vorschrift eingebracht wird;
3. c) Übertragen des Bildes mit dem Stempel von der optischen Sensoreinheit an eine Empfangseinheit des Fahrzeuges;
4. d) Überprüfen der Aktualität des Stempels des übertragenen Bildes anhand der zeitabhängigen Vorschrift mittels der Empfangseinheit.

One aspect of the invention provides a method for checking the timeliness of an image of a vehicle environment of a motor vehicle. The method comprises in particular the following steps:

1. a) generating the image by means of an optical sensor unit of the motor vehicle;
2. b) inserting a digital stamp into the image by means of the optical sensor unit, wherein the stamp is inserted according to a time-dependent rule;
3. c) transmitting the image with the stamp from the optical sensor unit to a receiving unit of the vehicle;
4. d) Checking the timeliness of the stamp of the transmitted image against the time-dependent rule by means of the receiving unit.

Such a process can be used to check the timeliness of the received image in an extremely reliable and robust manner. In other words, the transmission path of the image, from the optical sensor unit to the receiving unit, is secured "end-to-end", so that the received image can be assessed as current with absolute certainty. Accordingly, the driver, the driver assistance system or the electronic vehicle guidance system of the motor vehicle can make a safe decision about driving the motor vehicle based on the current image, which actually reflects the current vehicle environment. This significantly increases road safety for all road users.

For example, in a situation of reversing a parking space, the optical sensor unit can be designed as a rear view camera or can comprise a rear view camera, wherein the rear view camera detects a rear vehicle environment and generates images of the rear vehicle environment. The images of the rear view camera can be displayed on a display unit for the driver, for example, so that the driver reverses the vehicle based on the displayed images. By checking the timeliness of the image, the method can ensure that the driver reverses the motor vehicle based on the current image.

Timeliness can also be understood as temporal originality or temporal validity. The check can also be referred to as plausibility. In other words, the method is suitable for checking the validity and temporal originality of the image.

The vehicle environment can in particular be an external environment of the motor vehicle. The optical sensor unit can in particular detect an area or a section of the vehicle environment, for example the rear vehicle environment, be a component of a driver assistance system (ADAS system) of the vehicle and/or be mounted or integrated on a roof, front or rear of the motor vehicle. The optical sensor unit can be or include a camera, for example.

The optical sensor unit can be designed in particular to generate a single image of the vehicle surroundings. Preferably, the optical sensor unit can be designed to generate a plurality of temporally successive images, in particular a video recording. Consequently, the method can be suitable for checking the timeliness of the plurality of temporally successive images. In other words, the method can be used to check whether the image or the video recording was taken in real time, i.e. "live" received by the receiving unit. The image generated can in particular be a digital image and preferably comprise a plurality of pixels.

The digital stamp can be inserted into the image immediately after it has been created or together with the creation of the image. In particular, the stamp can be inserted using the optical sensor unit that creates the image, so that it can be ensured that the stamp is already inserted into the image immediately at the start of the transmission.

The digital stamp can in particular be incorporated into the image in an unchangeable manner, which means that the digital stamp no longer changes once it has been incorporated into the image. Incorporating the digital stamp into the image can in particular be understood to mean that at least one or more pixels of the image of the vehicle surroundings are replaced by at least one or more pixels of the digital stamp.

In particular, the optical sensor unit, preferably a generation unit of the sensor unit, can generate the stamp according to the time-dependent rule and insert the generated stamp into the image. In particular, the time-dependent rule, which can also be referred to below as just a rule, can prescribe which pixel or pixels of the image should be replaced by one or more pixels. In particular, a color of the pixel or pixels can also be prescribed according to the rule.

The rule can in particular be time-dependent. Preferably, the rule can depend on the time at which the image was captured or the stamp is inserted. In other words, the rule can prescribe the insertion of the stamp as a function of time, in particular how the stamp is to be inserted into the image. As a result, the insertion of the stamp can be at least indirectly dependent on time.

The image with the stamp can then be sent, for example as a digital image signal, from the optical sensor unit or received by the receiving unit. The image with the stamp can be transmitted from the optical sensor unit to the receiving unit via a transmission path. The transmission path can, for example, have a cable connection, such as an Ethernet cable, routers and/or electronic processing units that can, for example, convert or amplify the image signal to be transmitted.

The receiving unit at the end of the transmission path can be designed in particular to check whether the received image is up to date, in particular whether the stamp on the image is up to date, whereby the check can be carried out using the known, time-dependent rule. The check can be used to assess whether the image is up to date, i.e. whether it is transmitted in real time or is live.

The receiving unit can be designed, for example, as a computing unit or as a module of a computing unit, for example of an ADAS system of the motor vehicle. The receiving unit can alternatively or additionally be designed as a module of a display unit of the motor vehicle, for example a display in the vehicle interior of the motor vehicle.

For example, the receiving unit can generate a stamp according to the time-dependent rule and compare the generated stamp with the stamp inserted into the image. The rule for generating the stamp can depend on an expected time of creation of the received image. If the image is transmitted in real time, the expected time can correspond to the time of reception. Accordingly, an expected transmission time can be deducted from the time of reception. In other words, the inserted stamp can be compared with an expected stamp in order to check whether the stamp and thus the image are up to date. If, for example, the inserted stamp corresponds to the expected stamp, the rule is met and the image can be assessed as up to date. If this is not the case, the image can be assessed as not up to date.

For example, it can be provided that the receiving unit recognizes the stamp inserted in the image and reciprocally determines the time of creation of the image based on the time-dependent rule. The determined time can thus be compared with an expected time of creation of the image in order to check the timeliness of the stamp and thus of the image. If, for example, the determined time corresponds to the expected time, the rule is met and the image can be assessed as current. If this is not the case, the image can be assessed as not current.

In particular, the timeliness of the stamp can also be checked in other ways that are not explicitly mentioned here.

If, for example, the image is judged to be out of date, the next step in the process can be Among other things, it can be provided that the transmitted image is not displayed on the display unit. Instead, an error message or the like can be displayed. Alternatively or additionally, it can be provided that the received image is not processed further, for example by the ADAS system of the motor vehicle. For example, it can be provided as a further step that an error analysis of the transmission path is initiated and/or the transmission path is reinitialized.

for checking the timeliness of an image (1) of a vehicle environment (2) of a motor vehicle (3),

In particular, the method can be carried out not only by motor vehicle systems, but also by other systems, for example by systems for object protection monitoring to check the timeliness of an image of an object's surroundings, or other camera monitor systems.

One embodiment provides that the image is only displayed, for example by the display unit, if a result of the stamp check shows that the stamp satisfies the time-dependent regulation. One embodiment provides that the image is only further processed, for example by the ADAS system, if the result of the stamp check shows that the stamp satisfies the time-dependent regulation.

One advantage of this embodiment is that the driver and/or the ADAS system do not make incorrect decisions about driving the vehicle based on the incorrectly transmitted image. Instead, only those images are displayed or processed if their timeliness has been reliably verified. This can significantly increase road safety.

One embodiment provides that the stamp on the image is not recognizable to a user, in particular a driver or passenger, when the image is shown on a display unit. This has the advantage, among other things, that the user is not irritated or distracted by the stamp when viewing the image. Driving safety is therefore not impaired in any way.

In particular, the stamp cannot be seen when the user looks at the display unit as intended, for example from an upright sitting position. For example, the stamp cannot be seen because it is small and/or does not have enough contrast for human perception, although the stamp can still be clearly recognized by the receiving unit.

An alternative embodiment provides that the stamp applied to the image is recognizable to a user when the image is displayed on a display unit. This allows the user to convince themselves that the image is current. This is particularly advantageous when there are a large number of consecutive images, since the stamp changes over the course of the images displayed and, in particular, the change in the stamp can be recognized by the user.

One embodiment provides that a graphic representation and/or a position of the stamp in the image is defined according to the time-dependent rule. In particular, with a large number of images, the graphic representation and/or the position of the stamp can change over time.

For example, the stamp can be graphically represented as a pointer of an imaginary clock, with the time-dependent rule defining the position of the pointer of the imaginary clock. For example, the pointer according to the rule can assume a "12 o'clock" position at a first point in time and a "1 o'clock" position at a second point in time.

In particular, any graphical representation or position of the stamp that can be defined according to the time-dependent rule is conceivable.

One embodiment provides that a time signal is generated by a central timing unit of the vehicle and is received by the optical sensor unit and the receiving unit. The time-dependent rule preferably depends on the generated time signal.

In particular, the central timing unit comprises a clock, wherein the time signal can comprise information about the current time. The time signal can in particular be generated in a cycle, for example in seconds, tenths of a second or milliseconds.

Preferably, the optical sensor unit and the receiving unit can be synchronized by means of the time signal. In particular, the stamp is inserted according to the rule which is dependent on the time signal. In particular, the timeliness of the stamp is checked based on the rule which is dependent on the time signal. This can ensure that the transmitted image was transmitted in real time. With other In other words, the time signal can be used to check that the image is not outdated or delayed. A driving decision is therefore always made based on a current image.

An alternative embodiment provides that a first image is generated at a first point in time and a second image at a subsequent second point in time using the optical sensor unit. Preferably, a first digital stamp is introduced into the first image and a second digital stamp is introduced into the second image using the sensor unit. Preferably, the first stamp is introduced according to the regulation depending on the first point in time and the second stamp is introduced according to the regulation depending on the second point in time, so that the first stamp and the second stamp are designed differently. Preferably, the second stamp is checked for a difference or a change to the first stamp.

For example, in the situation of a stationary motor vehicle, the camera can generate essentially identical images if the vehicle surroundings do not change. In particular, if a large number of images are generated in this situation, a "still image" is generated. However, this embodiment can be used to check that it is a current, non-frozen image, since the stamp changes over time despite the still image. This at least ensures that the received image is not frozen.

An advantage of this embodiment is that a time signal is not required for synchronization to at least determine whether the image is a "moving" image or a frozen image. Rather, the optical sensor unit and the receiving unit can insert or check the stamp independently of one another.

One embodiment provides that the stamp is inserted with information about the time at which the image was created. For example, the stamp is a code that can be read out at least in a machine-readable manner, for example by means of a reading unit of the receiving unit. It is also conceivable that the stamp graphically represents a time, for example accurate to the second, which can also be read by a user. The information about the time can advantageously be compared directly with an expected time in order to check that the transmitted image is up to date.

Alternatively or additionally, the stamp can be inserted with a pattern consisting of one or more pixels. If the pattern consists of just one pixel, for example, this can move over the course of several images, so that a change in the stamp can be recognized when comparing several images. This makes it particularly easy and unobtrusive to insert the stamp into the image.

One embodiment provides that the stamp is inserted in an edge area of the image. One advantage is that the stamp leaves the essential image information of the image untouched, so that no image information is lost when the stamp is inserted. Another advantage is that the stamp in the edge area does not disturb or distract the user.

A further aspect of the invention provides a system for a motor vehicle for checking whether an image of a vehicle's surroundings is up to date. The system comprises at least one optical sensor unit for generating the image and a receiving unit for receiving the image. These can be connected to one another in particular by a transmission path for transmitting image signals and/or control signals. The optical sensor unit is designed in particular to insert a digital stamp into the image. The stamp can preferably be inserted according to a time-dependent rule. The receiving unit is designed in particular to check whether the stamp of the received image is up to date using the time-dependent rule.

For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and/or a request to enter user feedback is issued and/or a default setting and/or a predetermined initial state is set.

Further embodiments of the system according to the invention follow directly from the various embodiments of the method according to the invention and vice versa. In particular, individual features and corresponding explanations as well as advantages relating to the various embodiments of the method according to the invention can be transferred analogously to corresponding embodiments of the system according to the invention. In particular, the system according to the invention is designed or programmed to carry out a method according to the invention. In particular, the system according to the invention carries out the method according to the invention.

In particular, the system can also be provided for an object, for example a building, to check the timeliness of an image of an object's surroundings, or for other applications.

A further aspect of the invention provides a motor vehicle which comprises a system according to the invention.

The invention also includes combinations of the features of the described embodiments.

• 1 ein Ablaufdiagramm eines Ausführungsbeispiels eines erfindungsgemäßen Verfahrens;
• 2 eine skizzierte Darstellung eines Ausführungsbeispiels eines erfindungsgemäßen Verfahrens;
• 3 eine schematische Darstellung eines Stempels in einem zeitlichen Verlauf;
• 4 eine schematische Darstellung eines Ausführungsbeispiels eines Kraftfahrzeuges mit einem System.

In the following, embodiments of the invention are described.

• 1 a flow chart of an embodiment of a method according to the invention;
• 2 a sketched representation of an embodiment of a method according to the invention;
• 3 a schematic representation of a stamp over time;
• 4 a schematic representation of an embodiment of a motor vehicle with a system.

The exemplary embodiments explained below are preferred exemplary embodiments of the invention. In the exemplary embodiments, the components described each represent individual features of the invention that are to be considered independently of one another, which also develop the invention independently of one another and are therefore to be viewed as part of the invention individually or in a combination other than that shown. Furthermore, the exemplary embodiments described can also be supplemented by other features of the invention already described.

In the figures, functionally identical elements are provided with the same reference symbols.

1 shows a flow chart of an embodiment of a method according to the invention for checking the timeliness of an image 1 of a vehicle environment 2 of a motor vehicle 3. In a first step S1, the image 1 is generated by means of an optical sensor unit 4 of the motor vehicle 3. In particular, the optical sensor unit 4 detects a section, for example a rear area 18 of the vehicle environment 2, and consequently generates the image 1 of the detected vehicle environment 2. For example, the optical sensor unit 4 is a rear view camera.

In a second step S2, a digital stamp 5 is introduced into the image 1 by means of the optical sensor unit 4, in particular by means of a module of the optical sensor unit 4. The stamp 5 is preferably introduced into the image 1 according to a time-dependent rule. For example, the stamp 5 is first generated according to the time-dependent rule by a generation unit of the optical sensor unit 4 and then introduced into the image 1. For this purpose, for example, one or more pixels of the image 1 of the vehicle environment 2 can be replaced by pixels of the stamp 5 in order to introduce the stamp 5.

In a third step S3, the image 1 with the stamp 5 is transmitted from the optical sensor unit 4 to a receiving unit 6 of the motor vehicle 3, for example via a transmission path 19. The transmission path 19 can, for example, have wired and/or wireless transmission paths, as well as components 20 for processing and/or amplifying an image signal of the image 1. In this transmission path 19, or already in the optical sensor unit 4, interference can occur, for example, so that the image 1 may not be transmitted or may only be transmitted with a time delay.

In order to detect and counteract such disturbances, in a fourth step S4 the timeliness of the stamp 5 of the transmitted image 1 is checked using the time-dependent rule by means of the receiving unit 6. For example, the stamp 5 can be compared with an expected stamp 14 according to the time-dependent rule that the receiving unit 6 has generated by a comparison unit of the receiving unit 6. For example, the image 1 can be assessed as current if the stamp 5 corresponds to the expected stamp 14.

For example, in a fifth step S5, the image 1 can only be displayed and/or further processed on a display unit 7 if a result of the check of the stamp 5 shows that the stamp 5 satisfies the time-dependent regulation. If the stamp 5 does not satisfy the time-dependent regulation, an error message can be issued and/or troubleshooting can be initiated.

In the 2 a sketched representation of an embodiment of a method according to the invention is shown. For example, the optical sensor unit 4 generates the image 1 and places the stamp 5 in an edge area 16 of the image 1. The image 1 with the stamp 5 can then be transmitted to the receiving unit 6 via the transmission path 19, which can include one or more processing units 20. Example The receiving unit 6 checks the timeliness of the stamp 5 based on the time-dependent regulation, for example by generating the expected stamp 14 by the receiving unit based on the time-dependent regulation and comparing it with the transmitted stamp 5. If the result of the test shows that the stamp 5 meets the regulation, the image 1 can be displayed on the display unit 7.

3 shows a schematic representation of a stamp 5 in a time course at the respective time of insertion of the stamp 5 along a time axis t. At a first time 11, a first stamp 5 can be inserted according to the time-dependent rule. In this example, the stamp 5 comprises two black pixels 15, which are arranged one pixel apart. The first stamp 5 can be inserted into a first image 1, whereby the stamp 5, once inserted, preferably no longer changes.

At a second point in time 12, which follows the first point in time 11 at a time interval Δt, a second stamp 9 can be generated and inserted according to the time-dependent rule, for example. In this example, the second stamp 9 can comprise two pixels 15 arranged next to one another. At a third point in time 13, which follows the second point in time 12 at a time interval Δt, a third stamp 10 can be generated and inserted according to the time-dependent rule, for example. For example, a pattern of the stamp 5 can repeat itself over time after a large number of time intervals Δt.

In the 4 a schematic representation of an embodiment of a motor vehicle 3 with a system 17 is shown. The system 17 of the motor vehicle 3 can in particular comprise the optical sensor unit 4 and the receiving unit 6, which can be connected to one another via the transmission path 19. In this example, the optical sensor unit 4 comprises a rear view camera that can capture the rear area 18 of the vehicle environment 2.

For example, the motor vehicle 3 can also comprise a central timing unit 8, which can be designed to generate a time signal that can be received by the optical sensor unit 4 and the receiving unit 6. Preferably, the time-dependent regulation, and thus the stamp, depends on the generated time signal.

List of reference symbols

1

Picture

2

Vehicle environment

3

Motor vehicle

4

Optical sensor unit

5

Digital stamp

6

Receiving unit

7

Display unit

8th

Timer unit

9

Second digital stamp

10

Third digital stamp

11

First point in time

12

Second point in time

13

Third point in time

14

Expected digital stamp

15

pixel

16

Edge area

17

system

18

Rear area

19

Transmission path

20

Processing unit

t

Time

Δt

temporal distance

Claims (10)

Method for checking the timeliness of an image (1) of a vehicle environment (2) of a motor vehicle (3), comprising the steps: a) generating the image (1) by means of an optical sensor unit (4) of the motor vehicle (3); b) inserting a digital stamp (5) into the image (1) by means of the optical sensor unit (4), the stamp (5) being inserted according to a time-dependent rule; c) transmitting the image (1) with the stamp (5) from the optical sensor unit (4) to a receiving unit (6) of the motor vehicle (3); d) checking the timeliness of the stamp (5) of the transmitted image (1) based on the time-dependent rule by means of the receiving unit (6). Procedure according to Claim 1 , characterized in that the image (1) is only displayed and/or further processed if a result of the inspection of the stamp (5) shows that the stamp (5) satisfies the time-dependent regulation. Procedure according to Claim 1 or 2 , characterized in that the introduced stamp (5) of the image (1) is not recognizable to a user when the image (1) is displayed on a display unit (7). Method according to one of the preceding claims, characterized in that a graphic representation and/or a position of the stamp (5) is defined according to the time-dependent rule. Method according to one of the preceding claims, characterized in that a time signal is generated by a central timer unit (8) of the motor vehicle (3) and is received by the optical sensor unit (4) and the receiving unit (6), wherein the time-dependent regulation depends on the generated time signal. Method according to one of the Claims 1 until 4 , characterized in that a first image (1) at a first point in time (11) and a second image (1) at a subsequent second point in time (12) are generated by means of the optical sensor unit (4), wherein a first digital stamp (5) is introduced into the first image (1) and a second digital stamp (9) is introduced into the second image (1) by means of the optical sensor unit (4), wherein the first stamp (5) is introduced according to the regulation depending on the first point in time (11) and the second stamp (9) is introduced according to the regulation depending on the second point in time (12), wherein the second stamp (9) is checked for a difference to the first stamp (5). Method according to one of the preceding claims, characterized in that the stamp (5) is introduced with information about a time of generation of the image (1), and/or the stamp (5) is introduced with a pattern consisting of one or more pixels (15). Method according to one of the preceding claims, characterized in that the stamp (5) is introduced into an edge region (16) of the image (1). System (17) for a motor vehicle (3) for checking the timeliness of an image (1) of a vehicle environment (2), comprising an optical sensor unit (4) for generating the image (1) and a receiving unit (6) for receiving the image (1), characterized in that - the optical sensor unit (4) is designed to insert a digital stamp (5) into the generated image (1), wherein the stamp (5) can be inserted according to a time-dependent rule; and - the receiving unit (6) is designed to check the timeliness of the stamp (5) of the received image (1) based on the time-dependent rule. Motor vehicle comprising a system according to Claim 9 .

Images