DE102015201317B4 - Measuring a dimension on a surface - Google Patents

Abstract

Device for measuring a dimension between at least two points on at least one surface for a vehicle (1), comprising
- at least one image generating device (4, 7, 10, 12) for scanning the vehicle surroundings,
- a display device (8) for displaying the image of the vehicle surroundings,
- an input device (9) for defining at least two points as measuring points (6) between which the dimensions are to be determined, in the illustrated illustration,
- at least one environment sensor (7, 10, 12, 13) for detecting the distance and direction of the measuring points (6) with respect to the vehicle (1) and
- an evaluation device (3) for determining the dimension based on the detected distances and directions of the measuring points (6) and for outputting the determined dimension, and designed such that the curvature of the at least one surface between the measuring points (6) is detected by detecting the distance and direction of further points that lie on an imaginary, possibly projected line onto the surface between the measuring points (6) from the image generation device (4, 7, 10, 12) and/or from the environment sensor (7, 10, 12, 13), and is taken into account when determining the dimension.

Description

The invention relates to a device, a vehicle and a method for measuring a dimension between at least two points on surfaces.

Different methods and devices are known for the two-dimensional measurement of surfaces, such as vehicle surfaces, and for the three-dimensional measurement of objects, in particular vehicles and shipping packages.

The DE 10 2012 005 966 A1 discloses a device for generating a planar representation of a three-dimensional body by using parallel light to project a shadow image of the three-dimensional body.

The US 2004/0075847 A1 describes a sensor arrangement for determining the height of a vehicle, wherein the vehicle to be examined is movable between two columns in which light-emitting and light-receiving devices are arranged in such a way that the vehicle or a respective section of the vehicle interrupts the light beams between these devices, so that the height of the vehicle section can be determined on the basis of the interrupted light beams.

The CH 701 106 A2 discloses a method and a device for measuring the spatial extent of an object, wherein by means of two measuring arrangements, one of which is designed as a horizontally extending light barrier arrangement and the other has vertically extending laser beams with a camera as the associated sensor, the height profile and the ground contour of a vehicle can be determined when the vehicle is moved through both measuring arrangements during the measuring process.

The US 6,195,019 B1 explains a device for classifying vehicles and a toll system, wherein the device for classifying vehicles is arranged above a roadway on which vehicles move through a detection zone defined by a laser scanner. The detection zone is spanned on the one hand longitudinally to the direction of travel and on the other hand transversely to the direction of travel.

The JP 3110725 B2 reveals an area measurement with several measuring sensors from above.

The EP 0 828 990 B1 describes a device for determining the dimensions of an object, in particular an object on a moving conveyor for parcel sorting and handling applications. The objects to be detected move along a conveyor belt and are scanned from above by a laser scanner.

The EP 0 858 055 A2 explains a device and a method for measuring mail pieces, wherein the mail pieces are captured by a camera, preferably from above, and the dimensions of the mail piece are determined from the captured images.

The WO 03/025498 A1 discloses a system and a method for measuring the dimensions of moving packages, wherein light rays are directed onto the object and their reflections are detected via a parabolic mirror with the aid of a light detector and the dimensions of the object in one dimension are determined from the detected reflections.

The DE 10 2013 103 897 A1 describes a camera module for scanning an object line by line, comprising at least one line-shaped sensor and a lens for imaging the object onto the sensor. The camera module has multiple line sensors arranged at different distances from the lens, so that image lines at different distances from the lens are imaged onto the respective line sensors.

The WO 1999/060335 A1 discloses a surveying device that uses a camera to capture a target area and a laser rangefinder to measure the distance to a target point. The camera displays the image on a screen, and the distance is determined using the laser.

The US 2010/0134325 A1 Describes a vehicle vision system that uses multiple cameras with overlapping fields of view to create a 3D projection of the environment. It combines sensor data to generate projections, taking vehicle motion into account.

The DE 696 35 891 T2 discloses a camera system for generating three-dimensional images by distance measurement. A laser illuminates the target area, and a CCD array detects the reflected radiation to determine the distance to the objects.

The EP 2 120 009 A1 describes a system for measuring road features using laser and camera data. 3D point clouds are generated and the position of the features is taking into account the vehicle movement.

The present invention is based on the object of creating a device and a method for a vehicle with which dimensions of one object or several objects or between objects in the surroundings of the vehicle can be accurately measured.

The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments are contained in the respective subclaims.

• - zumindest eine Bilderzeugungseinrichtung zum Abtasten der Fahrzeugumgebung,
• - eine Anzeigeeinrichtung zum Darstellen des Abbilds der Fahrzeugumgebung,
• - eine Eingabeeinrichtung zum Festlegen zumindest zweier Punkte als Messpunkte, zwischen denen die Abmessungen zu ermitteln sind, in der dargestellten Abbildung,
• - zumindest einen Umfeldsensor zum Erfassen der Entfernung und der Richtung der Messpunkte bzgl. des Fahrzeugs und
• - eine Auswerteeinrichtung zum Ermitteln der Abmessung anhand der erfassten Entfernungen und Richtungen der Messpunkte sowie zum Ausgeben der ermittelten Abmessung.

The present invention relates to a device for measuring a dimension between at least two points on at least one surface for a vehicle, comprising

• - at least one image generating device for scanning the vehicle surroundings,
• - a display device for displaying the image of the vehicle's surroundings,
• - an input device for defining at least two points as measuring points between which the dimensions are to be determined, in the illustrated figure,
• - at least one environment sensor for detecting the distance and direction of the measuring points with respect to the vehicle and
• - an evaluation device for determining the dimension based on the recorded distances and directions of the measuring points and for outputting the determined dimension.

The curvature of the surface between the measurement points is detected by recording the distance and direction of additional points that lie on an imaginary, possibly projected, line on the at least one surface between the measurement points from the image generation device and/or the environment sensor. The resulting curvature of the surface is taken into account when determining the dimension.

With the device for measuring a dimension between measuring points on at least one surface, such dimensions can be determined easily and precisely using the sensors already present on the vehicle.

Precise measurement of two points on at least one surface in the vicinity of a vehicle allows, for example, the measurement of width and/or height of passages, such as garage or gate entrances. Passages between parked or stopped vehicles can also be measured quickly and easily, allowing a driver to decide whether they can drive their vehicle through this passage. Such situations occur, for example, when trucks are parked in narrow streets and block a large part of the roadway. If you want to enter warehouses or storage facilities, it can be advantageous to measure the relevant passages in advance. This is especially true if the vehicle is a large truck that cannot be easily turned around. Furthermore, goods to be transported can be measured. This is particularly useful in connection with vehicles used to transport such goods, such as industrial trucks, tractors, or the like. The recorded dimensions of the goods can be saved and used for further handling of the goods. For example, passageways or storage areas can be measured using the method according to the invention, from which it can be determined whether the goods can fit through or be deposited there. Furthermore, items of luggage to be stowed in the vehicle can be measured in advance. The method is preferably further developed in such a way that it checks whether the items of luggage can be accommodated in the cargo space and automatically determines suggestions for the arrangement of the individual items of luggage within the cargo space and displays them on the display device. This allows for both dense and correct loading of the vehicle.

• - Kamera,
• - Stereokamera,
• - Wärmebildkamera,
• - 3D-Laufzeitkamera,
• - Laserscanner bzw. Rotationslaser bzw. Lidar,
• - Radar.

The device according to the invention can comprise one or more image generating devices from the following group:

• - Camera,
• - stereo camera,
• - thermal imaging camera,
• - 3D runtime camera,
• - Laser scanner or rotating laser or lidar,
• - Radar.

Such imaging devices are often already present on vehicles.

In a preferred embodiment of the device according to the invention, one or more of the image generating devices are designed to be rotatable or pivotable and/or zoomable.

Such rotatability or pivoting makes it possible to define measurement points that would otherwise be impossible using fixed imaging devices. Providing zoomable imaging devices increases the accuracy of determining the measurement points. by allowing a measuring point to be defined more precisely in the image section enlarged by zooming.

• - Stereokamera,
• - 3D-Laufzeitkamera,
• - Kamera mit Autofokuseinrichtung bzw. Entfernungsmesser,
• - optisches Abstandsmessgerät mit Zeilenkamera,
• - Laserscanner bzw. Rotationslaser bzw. Lidar,
• - Radar,
• - Ultraschallsensor.

The device according to the invention can comprise one or more environmental sensors from the following group:

• - stereo camera,
• - 3D runtime camera,
• - Camera with autofocus device or rangefinder,
• - optical distance measuring device with line scan camera,
• - Laser scanner or rotating laser or lidar,
• - radar,
• - Ultrasonic sensor.

In a preferred embodiment of the device according to the invention, one or more environment sensors are designed to be rotatable or pivotable.

Such rotation or pivoting makes it possible to record the distance and direction of measuring points that would otherwise not be detectable using rigidly arranged environmental sensors.

In a further embodiment of the device according to the invention, the at least one image generating device also represents the environment sensor or both are integrated in a sensor (housing).

The resulting advantages are that the vehicle can be cost-effectively equipped with sensors, that the sensors already on the vehicle can be used efficiently, and that, in particular, the integration avoids an offset between the image generation device and the environment sensor. In the case of non-identical or non-integrated sensors, such an offset can lead to a situation where, although a point on a surface can be defined as a measurement point using the image generation device, this measurement point cannot be detected due to an obstacle located in a direct line between the environment sensor and the measurement point. This is avoided by designing one and the same sensor as both the image generation device and the environment sensor, since the viewing direction is identical in both cases.

In a preferred embodiment of the device according to the invention, a plurality of image generating devices and/or a plurality of environment sensors are arranged such that their viewing direction points in different directions.

Such an arrangement makes it possible to capture the entire vehicle surroundings without moving the vehicle.

In a further preferred embodiment of the device according to the invention, one or more image generation devices and/or one or more environmental sensors are arranged such that they scan the ground area below the vehicle from the vehicle, in particular from the underside or underbody of the vehicle. Additionally, these sensors can be configured with a light source as an illumination device.

By scanning the ground area, it is possible to define and record points underneath the vehicle as measurement points. This is particularly advantageous when markings on the ground are to be included in a survey. A light source enables the definition and recording of measurement points that would otherwise be impossible to define or record without lighting due to the darker light conditions beneath the vehicle body.

In a preferred embodiment of the device according to the invention, one or more vehicle movement sensors are provided to detect the movement of the vehicle. The evaluation device is designed such that a measuring point can be scanned at different vehicle positions. The movement of the vehicle is also taken into account when determining the measurement.

• Raddrehsensor, Lenkwinkelsensor, Beschleunigungssensor,
• Geschwindigkeitsmesser, Drehzahlmesser in Verbindung mit einem Getriebesensor, Neigungssensor, Federwegsensor bzw. Fahrwerksensor.

This makes it possible to define or record measurement points that are outside the field of view of the respective sensor in vehicles with only a small number of imaging devices or environmental sensors. Vehicle motion sensors are sensors that record the movement or trajectory of the vehicle's path. Examples of these are listed in the following group:

• Wheel rotation sensor, steering angle sensor, acceleration sensor,
• Speedometer, tachometer in conjunction with a transmission sensor, inclination sensor, suspension travel sensor or chassis sensor.

• - Abtasten und Erzeugen eines Abbilds der Fahrzeugumgebung,
• - Darstellen des Abbilds der Fahrzeugumgebung,
• - Festlegen zumindest zweier Punkte als Messpunkte, zwischen denen die Abmessungen zu ermitteln sind, in der dargestellten Abbildung,
• - Erfassen der Entfernung und der Richtung der Messpunkte bzgl. des Fahrzeugs,
• - Ermitteln der Abmessung anhand der erfassten Entfernungen und Richtungen der Messpunkte sowie Ausgeben der ermittelten Abmessung.

According to a further aspect of the present invention, a method for measuring a dimension between at least two points provided on at least one surface for a vehicle, in particular with the device explained above, comprising the steps

• - Scanning and generating an image of the vehicle environment,
• - Displaying the image of the vehicle environment,
• - Specify at least two points as measuring points between which the dimensions are to be determined, in the figure shown,
• - Recording the distance and direction of the measuring points with respect to the vehicle,
• - Determine the dimension based on the recorded distances and directions of the measuring points and output the determined dimension.

The curvature of the surface between the measurement points is detected by recording the distance and direction of additional points that lie on an imaginary, possibly projected, line on the at least one surface between the measurement points from the image generation device and/or the environment sensor. The resulting curvature of the surface is taken into account when determining the dimension.

This allows the three-dimensional shape of surfaces to be taken into account when determining a dimension.

In a further embodiment of the method according to the invention, a plurality of measurement points are scanned. A plan or a 3D model of the vehicle's surroundings is created from the data obtained by the scanning.

This makes it possible to create such a plan or 3D model easily and efficiently using the sensors available on the vehicle.

The plan may include a floor plan and/or an elevation.

In a further embodiment of the method according to the invention, predetermined objects are automatically filtered out from the acquired data.

Through automatic filtering, objects of interest for surveying can be identified and offered for surveying. Furthermore, objects unsuitable for surveying can be identified and excluded from the survey. Moving objects are particularly unsuitable for surveying because the measurement points cannot be determined or scanned without significant deviations.

• - Zaun,
• - Grenzstein,
• - Wand,
• - Gebäudedecke,
• - Boden,
• - Außen- oder Innenkante eines Gebäudes,
• - Fahrweg,
• - Nivellierlatte,
• - Bake,
• - Pfosten,
• - Durchfahrt,
• - Einfahrt,
• - Flurfördergerät,
• - Transportgut,
• - Transportpalette,
• - Lagerregal,
• - Gepäckstück,
• - Fahrzeug.

In a further embodiment of the method according to the invention, one or more of the following types of predetermined objects are automatically detected by means of image analysis:

• - Fence,
• - boundary stone,
• - Wall,
• - building ceiling,
• - Floor,
• - outer or inner edge of a building,
• - roadway,
• - leveling staff,
• - Bake,
• - posts,
• - passage,
• - entrance,
• - industrial truck,
• - transport goods,
• - transport pallet,
• - storage rack,
• - piece of luggage,
• - vehicle.

This makes determining dimensions and creating plans faster and more efficient.

In an advantageous embodiment of the method according to the invention, the movement of the vehicle is detected when the vehicle is moved between the detection of the distance and the direction of two measuring points. This detected movement is taken into account when determining the dimension.

This makes it possible to determine dimensions between measuring points on surfaces using sensors on the vehicle that cannot cover all areas in the vehicle's surroundings. Likewise, a measurement can also be carried out if an obstacle obstructs the direct view of an imaging device and/or an environmental sensor to a measuring point, for example, an object in its respective line of sight. Often, by moving the vehicle, a difference can be made between the recording of the distance and direction of one measuring point and the recording of the distance and direction of another. At a different measuring point, the measurement can be carried out despite the obstructing object.

In a further embodiment of the method according to the invention, the vehicle moves autonomously between the detection of the distance and the direction of the measuring points.

Autonomous vehicle movement enables automatic surveying of even larger outdoor or indoor spaces. This allows for more efficient planning, in particular.

In an advantageous embodiment of the method according to the invention, after a measuring point has been determined, it is checked whether the measuring point can be detected with one of the environmental sensors.

If an imaging device and an environmental sensor on a vehicle are offset from one another because the two sensors are not identical or are not integrated in the same sensor (housing), it may happen that a measuring point on a surface can be determined in the image of the vehicle's surroundings created by the imaging device, but the environmental sensor cannot determine the distance and direction to the measuring point due to an obstructing object or its orientation. Such a check makes it possible to display a message to the vehicle occupant performing the measurement that a specified measuring point cannot be detected. Additionally or alternatively, in such a case, a trajectory can be calculated that is either displayed for the driver to follow manually, or that the vehicle follows autonomously to enable one of the vehicle's environmental sensors to detect the measuring point anyway. When generating such a trajectory, other measuring points in the vehicle's surroundings are advantageously recorded and included in the trajectory calculation to avoid collisions. In order to be able to calculate such a trajectory more precisely, other systems or devices present on the vehicle or their data can also be included, such as a distance measuring device.

Dimensions can be 3D coordinates, distances, angles, diameters, radii, etc.

According to a further aspect of the present invention, a vehicle is provided which has one of the devices according to the invention explained above, wherein the device can in particular carry out one of the methods according to the invention explained above.

The vehicle according to the invention may have four-wheel drive and/or at least a ground clearance of at least 15 cm and preferably of at least 20 cm.

By means of such four-wheel drive and/or such minimal ground clearance, it is possible to measure measuring points even off-road or away from paved roads.

• 1 schematisch den Aufbau einer Vorrichtung zum Vermessen einer Abmessung zwischen zumindest zwei Punkten auf Oberflächen,
• 2 schematisch ein Fahrzeug mit der Vorrichtung zum Vermessen einer Abmessung zwischen zumindest zwei Punkten auf Oberflächen in einer Seitenansicht,
• 3 schematisch das Fahrzeug aus 2 in einer Heckansicht,
• 4 ein Verfahren zum Vermessen einer Abmessung zwischen zumindest zwei Punkten auf Oberflächen in einem Flussdiagramm,
• 5 schematisch eine autonome Vermessung eines Innenbereichs eines Gebäudes,
• 6 schematisch ein Bewegen des Fahrzeugs zwischen dem Festlegen bzw. der Erfassung zweier verschiedener Messpunkte, und
• 7 schematisch eine Vermessung zweier verschiedener Messpunkte eines Objekts.

The invention is explained in more detail below by way of example with reference to the drawings. The drawings show:

• 1 schematically shows the structure of a device for measuring a dimension between at least two points on surfaces,
• 2 schematically a vehicle with the device for measuring a dimension between at least two points on surfaces in a side view,
• 3 schematic view of the vehicle 2 in a rear view,
• 4 a method for measuring a dimension between at least two points on surfaces in a flowchart,
• 5 schematically an autonomous survey of an interior area of a building,
• 6 schematically a movement of the vehicle between the setting or recording of two different measuring points, and
• 7 schematically a measurement of two different measuring points of an object.

A first embodiment of a device for measuring a dimension between at least two points on surfaces for a vehicle is explained below. A vehicle 1 is equipped with such a measuring device 2 ( 1 , 2 , 3 ). In particular, vehicle 1 is a motor vehicle.

The measuring device 2 has an evaluation device 3, which is designed as a computer with a CPU memory device and suitable interfaces and centrally controls the measuring device 2. A measuring software module for processing two-dimensional and three-dimensional data is stored and executable on the evaluation device 3. Such data can be transmitted to the evaluation device 3 from various sensors arranged on the vehicle 1, in particular from image generation devices and/or environmental sensors.

The vehicle 1 is provided with a camera 4/1, which is arranged centrally at the upper edge of a windscreen of the vehicle 1 ( 2 ). The camera 4/1 is connected to the evaluation device 3 and transmits image data to the evaluation device 3 via this connection and receives control commands from the evaluation device 3 via this connection. The camera 4/1 is arranged on the vehicle 1 such that its viewing direction is directed diagonally downwards and forwards in the direction of travel 5 and preferably comprises servo motors (not shown) with which it can be aligned horizontally and vertically and zoomed. The servo motors are preferably connected to the evaluation device 3 and are controlled by it. The camera 4/1 represents an image generation device within the meaning of the present invention, which provides image data on the basis of which points on surfaces can be defined as measuring points 6.

Vehicle 1 is equipped with a Lidar 7/1, which is positioned centrally above the camera 4/1 at the upper edge of the windscreen ( 2 ). The lidar 7/1 is connected to the evaluation device 3 and transmits 3D data to the evaluation device 3 via this connection and receives control commands from the evaluation device 3 via this connection. The lidar 7/1 is arranged on the vehicle 1 such that its viewing direction is directed obliquely upwards and forwards in the direction of travel 5 and preferably comprises servo motors (not shown) with which it can be aligned horizontally and vertically. The servo motors are preferably connected to the evaluation device 3 and are controlled by it. The lidar 7/1 represents an environmental sensor within the meaning of the present invention, which provides three-dimensional data, in particular the distance and direction of measuring points 6, on the basis of which dimensions can be determined.

The evaluation device 3 is connected to a display device 8, such as an LCD screen, located inside the vehicle 1. An image of the vehicle surroundings recorded by the camera 4/1 is displayed on the display device 8. The display device 8 can be designed to be touch-sensitive, so that the measuring device 2 can be operated via it, i.e., its functions can be triggered or controlled.

Furthermore, the evaluation device 3 is connected to an input device 9, which is also located inside the vehicle 1. The input device 9 can be used to operate the measuring device 2.

The operation of the measuring device 2 comprises controlling an input cursor that overlays the image displayed on the display device 8. The input cursor can be positioned at any point on the image of the vehicle's surroundings, which can be selected as measurement points 6. The operation of the measuring device 2 can further comprise manually controlling the orientation and zoom range of the camera 4/1 and, if necessary, manually controlling the orientation of the lidar 7/1.

Preferably, the measuring device 2 is connected to one or more vehicle motion sensors 11, from which it receives vehicle motion data that can be included in the determination of the dimensions. In particular, this allows the vehicle 1 to be moved during a measurement of a dimension without distorting the measurement results. The vehicle motion sensors are one or more of the following sensors: wheel rotation sensor, steering angle sensor, acceleration sensor, speedometer, tachometer in conjunction with a transmission sensor, inclination sensor, a spring travel sensor or chassis sensor, and/or other suitable sensors that detect the movement or trajectory of the vehicle.

In the following, a method for measuring a dimension between at least two points on surfaces is explained in more detail, which is carried out by means of the measuring device 2 explained above ( 4 ).

The method begins in step S1. In step S2, an image of the vehicle's surroundings is captured using camera 4/1.

In the subsequent step S3, the captured image is displayed on the display device 8.

This is followed by the execution of step S4, in which one or more measuring points 6 are determined by a user of the measuring device 2 using the input device 9 to mark a measuring point 6 in the displayed image with the input cursor. Here, the user can manually control the camera 4/1, as explained above, so that the image shows the desired measuring points 6.

In the subsequent step S5, the evaluation device 3 uses the lidar 7/1 to record the distance and direction of the measuring point 6 or the measuring points 6 previously defined in step S4.

The process then proceeds to step S6, in which the user indicates whether he wants to record another measuring point 6. If If this is the case, step S2 is executed again.

After execution of step S6 and before execution of step S2, step S7 can optionally be executed, in which a vehicle movement takes place and is detected so that the movement data of the vehicle 1 can be saved for later determination of the dimensions. After execution of the optional step S7, step S2 follows. The detection of the vehicle movement or the trajectory traveled can be carried out using the data from additional driving movement sensors 11 present on the vehicle 1. By detecting and taking into account the movement or trajectory of the vehicle 1 between the detection of the measuring points 6, it is possible for the vehicle 1 to move during the execution of the method without falsifying the measurement result. This also makes it possible to determine dimensions between measuring points 6 that are not simultaneously displayed in an image captured by the camera 4/1.

If it is determined in step S6 that no further measuring point 6 is to be recorded, step S8 is executed, in which the dimensions between the measuring points 6 are determined from the recorded distance and direction of the respective measuring point 6 and output on the display device 8. In addition, the user can be prompted to specify which dimensions he or she would like to have displayed.

This is followed by the execution of step S9, in which it is checked whether operation should be continued. If operation should be continued, step S2 is executed again. If operation should not be continued, step S10 is executed, where the method ends.

A second embodiment of a device for measuring a dimension between at least two points on surfaces for a vehicle is explained below.

This embodiment is similar to the first embodiment and includes the same devices, apparatus, devices, etc., with the same reference numerals being used for the same parts as those already explained above. 1 the additional sensors of the second embodiment are connected to the evaluation device 3 with a dashed line.

The evaluation device 3 is connected to a stereo camera 10/1. The stereo camera 10/1 can be provided together with one or more cameras 4 and one or more lidars 7. Two cameras 4 can also be used together as a stereo camera 10. For this purpose, the evaluation device 3 has a control program module with which the two cameras 4 each simultaneously record an image, which is forwarded to the evaluation device 3. Furthermore, the relative positions of the two cameras 4 to one another and their viewing directions must be known to the evaluation device 3. The evaluation device 3 can then use this information and the two captured images to determine the distances to specific points in the stereo image.

The stereo camera 10/1 is arranged in an exterior mirror on a passenger side of the vehicle 1 such that its viewing direction is directed downwards away from the vehicle 1. The stereo camera 10/1 preferably comprises servo motors with which it can be aligned horizontally and vertically and zoomed. The servo motors are preferably connected to the evaluation device 3 and are controlled by the latter. The stereo camera 10/1 represents an image generation device within the meaning of the present invention, which provides image data based on which points on surfaces can be defined as measurement points 6. In addition, the stereo camera 10/1 represents an environmental sensor within the meaning of the present invention, which provides three-dimensional data, in particular the distance and direction of measurement points 6, based on which dimensions can be determined.

The method explained above for the camera 4/1 and Lidar 7/1 can be carried out accordingly with the stereo camera 10/1, whereby the stereo camera 10/1 is used both as an image generation device and as an environment sensor.

In particular, such a stereo camera 10/1 can be used to measure luggage before the start of a journey. If the dimensions of a trunk of the vehicle 1 are stored in the evaluation device 3, the evaluation device 3 can determine, based on the dimensions of the luggage, whether there is sufficient space in the trunk. In addition, the evaluation device 3 can display loading suggestions on the display device 8, whereby the evaluation device 3 can take into account a correct loading.

To measure the luggage, the vehicle can also drive autonomously around the luggage to record any depth dimensions of the luggage that may be in the shadow area.

Different detection areas of the stereo camera 10/1, which can be adjusted using a servo motor, are shown in 3 shown schematically.

The method explained above can be modified such that the movement or trajectory of the vehicle 1 is continuously recorded, in particular between steps S4 and S5. Such continuous recording makes it possible to perform a measurement even if there is no overlap between the detection areas of the image generation device and the environment sensor and/or if the environment sensor cannot detect a measuring point 6 due to an obstacle or an offset, since the vehicle 1 can be moved such that each of the measuring points 6 to be detected can be detected by the image generation device and subsequently by the environment sensor.

If a measuring point 6 is specified by the user on the display device 8 which cannot be detected by one of the surrounding sensors of the vehicle 1, an instruction to move the vehicle 1 can be displayed to the user on the display device 8 so that the surrounding sensor can detect the measuring point 6 after the vehicle 1 has moved.

Alternatively, such a movement of the vehicle 1 can also be performed autonomously, with the evaluation device 3 controlling the vehicle 1. The data from the described image generation devices and/or environmental sensors can be used to determine a collision-free trajectory. Further data from other devices or equipment with which the vehicle 1 is equipped can also be evaluated, such as data from a distance warning system, a parking assistance system (ultrasonic sensors), etc.

Additionally, an automatic surveying mode can also be provided, with which the vehicle 1, after activation by the user, fully automatically surveys interior areas of buildings (e.g., underground garages, warehouses, workshops) or exterior areas (e.g., construction sites, parking lots, property boundaries) by scanning a plurality of measuring points 6 and, if necessary, autonomously traverses the respective surveying area during the survey. When scanning a plurality of measuring points 6, characteristic measuring points, such as edges, corners of walls, and fences, are detected, for example, using automatic image recognition software and scanned at regular or irregular intervals. By defining specific image patterns for such measuring points, fully automatic scanning of an interior area of a building or exterior area is possible. Preferably, predetermined obstacles or additional objects, such as people or animals, are also automatically removed from the data. From the data obtained in this way, a plan, in particular a floor plan and/or elevation, or a 3D model of the vehicle's surroundings can be created. Such an autonomous survey of an interior of a building is shown schematically in 5 shown, where the dashed line indicates a collision-free path that is driven autonomously.

Additionally or alternatively, the detection of dimensions while the vehicle 1 is traveling can be carried out in such a way that obstacles on the travel path or passage widths are automatically detected and their dimensions determined. If the vehicle 1 cannot pass an obstacle or pass through a passage, a warning is preferably output on the display device 8 and/or on a loudspeaker (not shown) connected to the evaluation device 3. During such detection of passage widths, the dimensions of loaded goods, in particular when moved by an industrial truck (forklift truck, pallet truck, etc.), which represents the vehicle 1, can also be detected manually or automatically and taken into account.

In addition, in the method explained above, a check can be performed between steps S4 and S5 to determine whether a measuring point 6 marked in the image of the vehicle's surroundings can be detected by the environment sensor. This is particularly advantageous in the case of an offset between the image generation device and the environment sensor, i.e., when the two are not identical to one another, as well as in the case of obstacles that do not impede the image generation device but do impede the detection path of the environment sensor. Such a check can ensure that a measuring point 6 previously marked in the two-dimensional image can also be detected three-dimensionally. Such a check can be performed using various methods that depend on the available sensors. Two methods with which such a check can be performed are explained below.

In the first verification method, a camera 4 is used as an image generation device and a lidar 7 as an environment sensor. After defining a measuring point 6 in step S4, the evaluation device 3 directs the laser beam of the lidar 7 onto the measuring point 6 and determines with the camera 4 whether the measuring point 6 is illuminated by the laser beam. The alignment of the laser beam can be iterative, i.e. the evaluation device 3 corrects the alignment of the laser beam with the aid of the images captured by the camera 4 until the measuring point 6 is illuminated by the laser beam or a predetermined number of illumination attempts is reached or a predetermined time has elapsed. If the measuring point 6 is illuminated by the laser beam cannot be illuminated, an error message can be issued, possibly with driving instructions for the vehicle driver, or the evaluation device moves the vehicle 1 autonomously in such a way that the measuring point 6 can be illuminated by the laser beam.

In the second verification method, a three-dimensional scan of the vehicle's surroundings is performed using at least one of the environmental sensors and compared or linked with the image of the vehicle's surroundings generated by the image generation device using suitable image analysis methods. In particular, edge contours as well as the structure and/or color of surfaces can be taken into account in order to merge the two-dimensional image with the three-dimensional data. The distortions of the three-dimensional structures compared to the two-dimensional image, which occur in particular due to an offset between the image generation device and the environmental sensor, can be eliminated using interpolation methods. By applying such a method, the presence of an obstacle blocking the path from the environmental sensor to measuring point 6 can be determined. The three-dimensional scan of the vehicle's surroundings can be carried out very roughly compared to determining the distance with the environmental sensor and can only be used to estimate the presence of objects between the measuring devices and the target areas.

Additionally, in the method explained above, between steps S2 and S3, the image processing software module can extract objects to be measured or measurable from the images captured with one or more image generation devices, which are then suggested to the user for measurement when the image is displayed in step S3. Such objects are preferably objects that delimit a room or area, such as fences, boundary stones, walls, building ceilings, floors, outer or inner edges of a building, driveways, leveling rods, beacons, posts, passageways, entrances, industrial trucks, transport goods, transport pallets, storage racks, luggage, and vehicles. Object extraction can be performed using a pattern comparison based on the acquired 2D data. Moving or moving objects are eliminated, leaving only static or immobile or unmoving objects. To perform object extraction more accurately and efficiently, 3D data from the environmental sensors can be included. For this purpose, a three-dimensional scan of the environment can be carried out, as already explained above.

Alternatively or additionally, moving objects can be eliminated by repeatedly capturing 2D and/or 3D data over a predetermined time interval and comparing the acquired data. If moving objects are present in the data, they have changed their position and can therefore be detected or eliminated.

Additionally or alternatively, the measuring device 2 can also suggest dimensions to be recorded in the image of the objects to be measured or measurable, such as 3D coordinates, distances (e.g., height, width, depth, length), angles, diameters, and radii for measurement, which is then carried out automatically. Further differentiation can also be made according to, for example, minimum/maximum/average dimensions. For example, the query can be: "Should the maximum height/width of the object be measured?" A measurement command can also be entered via voice control, such as: "Calculate the average of all radii along an edge of the object" or "Determine the minimum and maximum angle of the zoomed-in area."

Additionally or alternatively, the curvature of the surface between the measuring points 6 can be detected by recording the distance and direction of additional points that lie on an imaginary, possibly projected, line onto the surface between the measuring points 6 by the image generation device and/or the environment sensor. This allows the three-dimensional shape of surfaces to be taken into account when determining a dimension, so that, for example, not only a direct distance between two measuring points 6 can be determined, but also the length of a route between the two measuring points 6, whereby the route can extend along the surface. The determination and scanning of the additional points is preferably carried out automatically. In particular, 3D models of surfaces or objects in the vehicle environment can be created using the data obtained in this way.

The vehicle 1 can be provided with one or more further cameras 4/2, 4/3 and/or 4/4 in addition to or as an alternative to the camera 4/1, as shown in 2 shown. While camera 4/2 is centrally located at the upper edge of a rear window of vehicle 1, camera 4/3 is centrally located on a front bumper of vehicle 1 and camera 4/4 is centrally located on a rear bumper of vehicle 1. While the viewing direction of camera 4/2 is directed diagonally downwards and backwards in the direction of travel 5, the viewing direction of camera 4/3 is horizontally forwards in the direction of travel 5 and the viewing direction of camera 4/4 directed horizontally rearward in the direction of travel 5. Cameras 4/2, 4/3, and 4/4 are connected to the evaluation device 3, have the same functions, and preferably include servo motors, as explained above for camera 4/1. Cameras 4 can also be part of one of the stereo cameras 10. The method explained above for camera 4/1 can be carried out accordingly with cameras 4/2, 4/3, and 4/4.

Likewise, the vehicle 1 can be equipped with one or more additional lidars 7 in addition to or as an alternative to the lidar 7/1. 2 A lidar 7/2 is shown, which is arranged centrally above the camera 4/2 at the upper edge of the rear window and whose viewing direction is directed diagonally upwards and backwards in the direction of travel 5. The lidar 7/2 is connected to the evaluation device 3, has the same functions, and preferably includes servo motors, as explained above for the lidar 7/1. The method explained above for the lidar 7/1 can be carried out accordingly with the lidar 7/2.

In addition or alternatively to the stereo camera 10/1, the vehicle 1 can be provided with further stereo cameras 10/2, 10/3 and/or 10/4, which are arranged on the underbody of the vehicle 1, as shown in 2 shown. The viewing directions of the stereo cameras 10/2, 10/3 and 10/4 are directed towards the ground beneath the vehicle 1. With the stereo cameras 10/2, 10/3 and 10/4, for example, markings on the ground can be detected and scanned and thus measured or included in measurements. The stereo cameras 10/2, 10/3 and 10/4 are connected to the evaluation device 3, have the same functions and preferably include servo motors, as explained above for the stereo camera 10/1. The stereo cameras 10/2, 10/3 and 10/4 can have light sources as illumination devices, so that the ground beneath the vehicle 1 can be illuminated and thus the quality of the images displayed on the display device 8 is improved. The procedure explained above for the stereo camera 10/1 can be carried out accordingly with the stereo cameras 10/2, 10/3 and 10/4.

The device and method described above for measuring a dimension between at least two points on surfaces can be modified in a variety of ways, and these modifications are explained below. 1 the additional sensors of the modifications are connected to the evaluation device 3 with a dotted line.

According to a first modification, the vehicle 1, in addition to or alternatively to the stereo camera 10/1, has a stereo camera (not shown) on an exterior mirror on a driver's side of the vehicle 1. This stereo camera, like the stereo camera 10/1, is arranged such that its viewing direction is directed downwards away from the vehicle 1 and is connected to the evaluation device 3. It has the same functions and preferably comprises servo motors, as explained above for the stereo camera 10/1. The method explained above for the stereo camera 10/1 can be carried out accordingly with this stereo camera on the driver's side.

According to a second modification, in addition to or as an alternative to the above-explained environment sensors (Lidar 7, stereo camera 10), the vehicle 1 can be provided with a radar 12/1 and/or 12/2 as an environment sensor, as in 1 and 2 shown. The radar 12/1 is arranged above the camera 4/3 in the center of the front bumper of the vehicle 1, and the radar 12/1 is arranged above the camera 4/4 in the center of the rear bumper of the vehicle 1. While the viewing direction of the radar 12/1 is directed horizontally forward in the direction of travel 5, the viewing direction of the radar 12/2 is directed horizontally rearward in the direction of travel 5. The radars 12/1 and 12/2 are connected to the evaluation device 3, have the same functions, and preferably include servo motors, as explained above for the lidar 7/1.

In addition or alternatively to the environment sensors explained above, the vehicle 1 can be provided with one or more ultrasonic sensors 13 as an environment sensor, as in 1 shown. The ultrasonic sensors 13 are connected to the evaluation device 3, have the same functions, and preferably include servo motors, as explained above for the lidar 7/1. Using the ultrasonic sensors 13, which are often already present on vehicles 1 as part of a parking assistance system, measurements can be taken, particularly in the immediate vicinity of the vehicle 1.

In the 6 and 7 A vehicle 1 is shown with a stereo camera 10/5, which is arranged centrally at the upper edge of the windshield of the vehicle 1 such that its viewing direction is directed forward from the vehicle 1 in the direction of travel 5. The stereo camera 10/5 is connected to the evaluation device 3, has the same functions, and preferably includes servo motors, as explained above for the stereo camera 10/1. The method explained above for the stereo camera 10/1 can be carried out accordingly with the stereo camera 10/5.

6 shows schematically a movement of the vehicle 1 between the setting or the recording of two different measuring points 6/6 and 6/7 by means of the stereo camera 10/5 in a Interior of a building. The dashed line indicates the path of vehicle 1 during the surveying process. First, measuring point 6/6 is determined and measured, then vehicle 1 is moved, and then measuring point 6/7 is determined and measured.

7 shows a schematic of the measurement of two different measuring points 6/8 and 6/9 of an object using the stereo camera 10/5. The object is a sculpture. First, measuring point 6/8 is determined and measured, followed by measuring point 6/9.

In principle, several different sensor types are suitable as image generation devices or environmental sensors, which can be arranged at different locations on the vehicle 1 and directed in different directions. The respective positions of the installed sensors on the vehicle 1 and their possible orientation are stored in the evaluation device.

An image generation device within the meaning of the present invention is a camera 4, a stereo camera 10, a thermal imaging camera or infrared camera, a 3D time-of-flight camera, a lidar 7, and a radar 12. Images of the vehicle's surroundings can be calculated in the form of two-dimensional data from the 3D data generated by the three-dimensional sensors. The thermal imaging camera or infrared camera can also be used to detect living beings, particularly people and animals. These living beings can be removed from the data, for example, when plans are created automatically.

An environmental sensor within the meaning of the present invention is a lidar 7, a stereo camera 10, a 3D time-of-flight camera, a camera with autofocus device or rangefinder, an optical distance measuring device with line scan camera, a radar 12 and/or an ultrasonic sensor 13. In principle, any environmental sensor can also be used as an image generation device, since images of the vehicle environment in the form of two-dimensional data can be calculated from the generated 3D data.

This data from the vehicle's surroundings can be used as reference points relative to a measurement object. For example, the 3D object coordinates can be measured relative to spatial coordinates. For example, it may be useful to position and align an object at a specific spatial coordinate. It is useful to use the environment sensors to determine the position/coordinates of the vehicle relative to the spatial coordinates and, in a subsequent step, the object coordinates relative to the vehicle coordinates, in order to be able to determine the object coordinates relative to space.

In principle, the settings of the servo motors explained above can be made by the operator using the input device 9 and/or a touch-sensitive display device 8. The zoom function enables the operator to determine a measuring point 6 more precisely by the image generation device capturing a zoomed-in, i.e., enlarged, image of the vehicle's surroundings, which is displayed on the display device 8.

Reference symbol

1

vehicle

2

Surveying device

3

Evaluation device

4

camera

5

Direction of travel

6

measuring point

7

Lidar

8

Display device

9

Input device

10

stereo camera

11

Driving motion sensor

12

radar

13

Ultrasonic sensor

Claims (20)

Device for measuring a dimension between at least two points on at least one surface for a vehicle (1), comprising - at least one image generation device (4, 7, 10, 12) for scanning the vehicle surroundings, - a display device (8) for displaying the image of the vehicle surroundings, - an input device (9) for defining at least two points as measuring points (6) between which the dimensions are to be determined, in the displayed image, - at least one environment sensor (7, 10, 12, 13) for detecting the distance and the direction of the measuring points (6) with respect to the vehicle (1), and - an evaluation device (3) for determining the dimension based on the detected distances and directions of the measuring points (6) and for outputting the determined dimension, and designed such that the curvature of the at least one surface between the measuring points (6) is determined by detecting the distance and the direction of further Points that lie on a line imaginary, possibly projected, onto the surface between the measuring points (6) by the image generation device (4, 7, 10, 12) and/or by the environment sensor (7, 10, 12, 13) are detected and taken into account when determining the dimension. Device according to Claim 1 , characterized in that the device has one or more image generation devices from the following group: - camera (4), - stereo camera (10), - thermal imaging camera, - 3D time-of-flight camera, - laser scanner or rotating laser or lidar (7), - radar (12). Device according to Claim 1 or 2 , characterized in that one or more image generating devices (4, 7, 10, 12) are designed to be rotatable or pivotable and/or zoomable. Device according to one of the Claims 1 until 3 , characterized in that the device has one or more environmental sensors from the following group: - stereo camera (10), - 3D time-of-flight camera, - camera with autofocus device or rangefinder, - optical distance measuring device with line camera, - laser scanner or rotating laser or lidar (7), - radar (12), - ultrasonic sensor (13). Device according to one of the Claims 1 until 4 , characterized in that one or more environmental sensors (7, 10, 12, 13) are designed to be rotatable or pivotable. Device according to one of the Claims 1 until 5 , characterized in that the at least one image generating device also represents the environment sensor. Device according to one of the Claims 1 until 6 that a plurality of image generating devices (4, 7, 10, 12) and/or a plurality of environmental sensors (7, 10, 12, 13) are arranged such that their viewing direction points in different directions. Device according to one of the Claims 1 until 7 , characterized in that one or more image generation devices (4, 10) and/or one or more environment sensors (10, 13) are arranged in such a way that they scan the ground area below the vehicle (1) from the vehicle (1), in particular from the underside or underbody of the vehicle (1), and are optionally designed with a light source as an illumination device. Device according to one of the Claims 1 until 8 , characterized in that one or more driving movement sensors (11) are provided for detecting the movement of the vehicle (1) and the evaluation device (3) is designed such that a measuring point (6) can be scanned at different vehicle positions, the movement of the vehicle (1) being taken into account when determining the dimension. Method for measuring a dimension between at least two points on at least one surface for a vehicle (1), in particular with a device according to one of the Claims 1 until 9 , comprising the steps of - scanning and generating an image of the vehicle surroundings, - displaying the image of the vehicle surroundings, - defining at least two points as measuring points (6) between which the dimensions are to be determined in the displayed image, - detecting the distance and direction of the measuring points (6) with respect to the vehicle (1), - determining the dimension based on the detected distances and directions of the measuring points (6) and outputting the determined dimension, wherein the curvature of the at least one surface between the measuring points (6) is detected by detecting the distance and direction of further points which lie on a line which is imaginary, possibly projected, onto the surface between the measuring points (6) from the image generation device (4, 7, 10, 12) and/or from the environment sensor (7, 10, 12, 13), and is taken into account when determining the dimension. Procedure according to Claim 10 , characterized in that a plurality of measuring points (6) are scanned and a plan or a 3D model of the vehicle environment is created from the data determined by the scanning. Procedure according to Claim 11 , characterized in that the plan comprises a floor plan and/or an elevation. Method according to one of the Claims 10 until 12 , characterized in that predetermined objects are automatically filtered out from the recorded data. Method according to one of the Claims 10 until 13 , characterized in that one or more of the following types of predetermined objects are automatically detected by means of image analysis: - Fence, - Boundary stone, - Wall, - Building ceiling, - Floor, - Outer or inner edge of a building, - Driveway, - Leveling rod, - Beacon, - Post, - Passage, - Entrance, - Industrial truck, - Goods to be transported, - Transport pallet, - Storage rack, - Piece of luggage, - Vehicle. Method according to one of the Claims 10 until 14 , characterized in that the vehicle (1) is moved between the detection of the distance and the direction of two measuring points (6), wherein the movement of the vehicle (1) is detected and taken into account when determining the dimension. Procedure according to Claim 15 , characterized in that the vehicle (1) moves autonomously between the detection of the distance and direction of the measuring points (6). Method according to one of the Claims 10 until 16 , characterized in that after defining a measuring point (6), it is checked whether the measuring point (6) can be detected with one of the environmental sensors (7, 10, 12, 13). Method according to one of the Claims 10 until 17 , characterized in that dimensions are 3D coordinates, distances, angles, diameters, radii, which can be further classified into minimum, maximum and average values. Vehicle incorporating a device according to one of the Claims 1 until 9 wherein the device in particular comprises a method according to one of the Claims 10 until 18 executes. Vehicle to Claim 19 , characterized in that the vehicle (1) has a four-wheel drive and/or a ground clearance of at least 15 cm and preferably of at least 20 cm.

Images