DE102019201912A1 - Control of an agricultural machine based on the detection of a boundary on agricultural land - Google Patents

Abstract

A method for controlling an agricultural machine (50), comprising receiving data on the surroundings which are generated by a surroundings sensor system (22) when detecting an agricultural area (60); Evaluating the environmental data in order to detect a cultivation area (62), a weed area (64) and a boundary (63) between the cultivation area (62) and the weed area (64) based on a segmentation of the environmental data; and generating a control signal for the agricultural machine (50) based on the detected boundary profile (63).

Description

TECHNICAL AREA

The present invention relates to the field of autonomously or partially autonomously driving agricultural machines. In particular, the present invention relates to a device, a method, a computer program product, a computer-readable storage medium and a data carrier signal for controlling such an agricultural machine.

TECHNICAL BACKGROUND

Agricultural machines which can be controlled by means of a navigation system during their movements on an agricultural area are known from the prior art. For example disclosed US 2018/0113473 A1 an agricultural machine with an automated steering system based on the Global Navigation Satellite System (GNSS). The GNSS comprises an antenna and a receiver which is arranged at a certain distance from the antenna. Movement parameters of the agricultural machine such as position, alignment (roll-pitch-yaw angle) and speed can be determined in this way. On the basis of the determined movement parameters, control signals for steering the agricultural machine can be generated.

When cultivating agricultural land such as fields, it is particularly important that the means used for this, in particular an agricultural machine, only move within a predefined limit for certain processing operations. Unintentional crossing of the limit by the agricultural machine can lead to harvests being damaged. There is also the risk of a collision with other agricultural machinery or people, which can have fatal consequences.

However, these systems known from the prior art have the disadvantage that the detection of the objects on the usable area does not function with sufficient accuracy and without interference. As a result, the boundaries of field pieces cannot be recognized with the required reliability. In addition, GNSS-based solutions are very expensive due to the often high license fees for the use of GNSS systems.

The present invention is therefore based on the object of carrying out the agricultural cultivation of the usable areas with increased safety while at the same time keeping the cost level at least the same.

The object is achieved by a device, a method, a computer program product, a computer-readable storage medium and a data carrier signal according to the independent claims.

The agricultural machine can be designed as a one-piece vehicle or articulated vehicle. It can comprise a swather, a hay machine, a combine harvester, a spraying machine, a sprayer or another vehicle that can be used in agriculture. The agricultural machine is an autonomous or semi-autonomous vehicle. This means that the agricultural machine can act entirely (in the case of a fully autonomous vehicle) or partially (in the case of a partially autonomous vehicle) without the influence of a human driver.

The device for controlling the agricultural machine can be, for example, an electronic control or regulating unit (ECU), an electronic control or regulating module (ECM = Electronic Control Module) or a control / regulating unit for autonomous driving (e.g. an "autopilot"). The control device can be located in the agricultural machine, or outside or partially outside the agricultural machine. For example, the control device can be part of a central monitoring device for the agricultural area. Communication can be wireless, for example via BlueTooth, infrared, near-field communication (NFC), radio, Internet, intranet, cloud systems and / or wired systems.
The environment sensor system for generating environment data can comprise one or more environment sensors. The environment sensors preferably include a radar sensor (for example a broadband radar sensor or a ground penetrating radar sensor, GPR), a lidar sensor, an ultrasonic sensor and / or a camera, for example a stereo camera or 3D camera or an infrared camera. The environment sensor system can be attached to the agricultural machine and thus record the environment of the agricultural machine traveling on the agricultural area in real time. Alternatively, the environment sensor system can be part of a central monitoring device for monitoring the usable area, the monitoring device being connected to the control device and / or the agricultural machine. The multiple environment sensors of the environment sensor system can be arranged at different positions on the agricultural machine and / or the central monitoring device. At least one of the environment sensors can be arranged on another vehicle, for example on another agricultural machine on the agricultural area.

The environment data are analyzed by means of the evaluation unit in order to identify a plurality of objects in the environment data. The Object recognition is based on a segmentation of the surrounding data, in which a specific algorithm or a plurality of algorithms are used which are designed to recognize regions with coherent content by combining neighboring pixels or voxels according to a specific homogeneity criterion. The recognized objects are classified by means of the evaluation unit in order to assign an object class (for example cultivation area, weed area, paths) to each of the objects. The classification is preferably based on an analysis of the environment data with regard to reflection properties of the various detected objects. For example, crops such as cabbage vegetables, maize and cereals have a higher reflection coefficient than weeds at certain wavelengths of light. A reflection measurement can therefore be used to differentiate between the various objects.

In this way, a growing area, a weed area and a boundary between the growing area and the weed area are detected. The borderline relates to a transition from the cultivation area to the weed area or vice versa, the transition comprising a dimensionless line or curve or having a width that can be detected by the environment sensor. The segmentation is preferably a semantic segmentation.

Based on the above sensor-based object recognition and detection of the boundary profile, a control signal is generated which brings about a suitable measure for the agricultural machine. The control signal can be output to an external entity, such as a central control device, which is connected to the agricultural machine, or the agricultural machine itself (for example an actuator of the agricultural machine). According to one embodiment, the method further comprises the control signal causing the agricultural machine to brake (e.g. full braking), change the direction of travel and / or emit a warning signal (e.g. a warning light signal such as a red light signal, a warning sound signal or a vibration signal).

The control of the agricultural machine according to the invention is particularly advantageous because it allows different areas on the agricultural area such as cultivation areas and weed areas to be taken into account when generating the control signal. Adjacent cultivation areas or fields are often separated from one another by a weed area. Therefore, such weed areas mark the border between two neighboring fields. If only one of the two adjacent field pieces is to be processed by the agricultural machine at a given point in time, according to the invention it can be particularly reliably avoided that the other field piece, which is not to be entered, is inadvertently impaired by the agricultural machine. Agricultural management can therefore be automated or partially automated in a secure manner.

Advantageous refinements and developments are specified in the subclaims.

According to one embodiment, the method further comprises creating a raster map based on the environment data, the raster map having a plurality of raster elements, an object class being assigned to each of the raster elements.

Each grid element corresponds to a surface element of the agricultural area. In this way, the object classes can be assigned area by area. The grid map provides particularly precise information about how the usable area is divided into different areas. This is advantageous for reliable control of the agricultural machine on the usable area.

According to a further embodiment, an attribute is assigned to at least one of the grid elements, the attribute comprising a height and / or density of an object recognized in the at least one grid element.

In this way, more extensive information about the various surface elements of the usable area can be determined, which further favors the more reliable control of the agricultural machine on the usable area.

According to a further embodiment, the control signal is generated under the condition that, viewed from the agricultural machine, the cultivation area is in front of the weed area.

The weed area is therefore further away from the agricultural machine than the cultivation area. In this way, an unnecessary measure such as braking when entering the cultivation area to be worked can be avoided and at the same time a necessary measure can be ensured before entering a (temporarily) not to be worked field piece.

According to one embodiment, the method further comprises evaluating the environment data in order to detect a path area between two weed areas adjoining the path area.

In many cases, such a path area marks the boundary between two adjacent field pieces. Another control signal for braking or changing the direction of travel of the agricultural machine can be used when the distance range is detected to be triggered. In this way, unintentional stepping onto an adjacent field piece which is not (temporarily) to be worked on can be avoided particularly reliably.

As an alternative or in addition, the method also includes evaluating the surroundings data in order to detect a path area within a cultivation area. In this case, the path area does not mark the boundary of the cultivation area. The agricultural machine does not have to be braked. Therefore, no control signal based on the detection of the route area or an all-clear signal (e.g. a green light signal) is triggered.

The computer program product according to the invention is designed to be loaded into a memory of a computer and comprises software code sections with which the method steps of the method according to the invention are carried out when the computer program product is running on the computer.

A program belongs to the software of a data processing system, for example an evaluation device or a computer. Software is a collective term for programs and associated data. The complement to software is hardware. Hardware describes the mechanical and electronic alignment of a data processing system. A computer is an evaluation device.

Computer program products generally comprise a sequence of instructions which, when the program is loaded, cause the hardware to carry out a specific method that leads to a specific result. When the program in question is used on a computer, the computer program product produces the inventive technical effect described above.

The computer program product according to the invention is platform independent. That means it can run on any computing platform. The computer program product is preferably executed on an evaluation device according to the invention for detecting the surroundings of the vehicle.

The software code sections are written in any programming language, for example Python.

The computer-readable storage medium is, for example, an electronic, magnetic, optical or magneto-optical storage medium.

The data carrier signal is a signal which the computer program product from a storage medium on which the computer program product is stored to another entity, for example another storage medium, a server, a cloud system or a data processing facility , transmits.

• 1 eine schematische Darstellung einer erfindungsgemäßen Vorrichtung gemäß einer Ausführungsform;
• 2 eine schematische Darstellung einer Rasterkarte, die basierend auf Erfassung einer landwirtschaftlichen Nutzfläche erstellt ist; und
• 3 eine schematische Darstellung der Nutzfläche zur Veranschaulichung der erfindungsgemäßen Steuerung.

Embodiments will now be described by way of example and with reference to the accompanying drawings. Show it:

• 1 a schematic representation of a device according to the invention according to one embodiment;
• 2 a schematic representation of a grid map that is created based on the detection of an agricultural area; and
• 3 a schematic representation of the usable area to illustrate the control according to the invention.

In the figures, the same reference numerals relate to the same or functionally similar reference parts. The relevant reference parts are identified in the individual figures.

1 shows schematically an apparatus 10 for controlling an agricultural machine 50 . The control device 10 includes an input interface 12 for receiving environmental data from an environmental sensor system 22nd when recording an agricultural area 60 ( 2 ) are generated. The environment data can be obtained directly from the environment sensors 22nd , as in 1 shown by way of example, or alternatively received by another device to which the environment data was previously fed.

The control device 10 also includes an evaluation unit 14th for evaluating the surrounding data. Using the evaluation unit 14th the environment data are analyzed in order to identify a plurality of objects in the environment data. The object recognition is based on a segmentation of the surrounding data, in which a certain algorithm or a plurality of algorithms are used which are designed to recognize regions with coherent content by combining neighboring pixels or voxels according to a certain homogeneity criterion. The recognized objects are evaluated by means of the evaluation unit 14th classified in order to assign an object class (for example cultivation area, weed area, paths) to each of the objects. The classification is preferably based on an analysis of the environment data with regard to reflection properties of the various detected objects. For example, crops such as cabbage vegetables, maize and cereals have a higher reflection coefficient than weeds at certain wavelengths of light. A reflection measurement can therefore be used to differentiate between the various objects.

This way you become a growing area 62 , a weed area 64 and a boundary 63 between the growing area 62 and the weed area 64 detected. 2 shows a schematic representation of the evaluation result, with that of the environment sensors 22nd recorded usable area 60 in this example as a raster map 24 with a variety of grid elements 242 is summarized. The exemplary borderline 63 is shown as a dashed line.

The control device 10 also includes a signal unit 16 for generating a control signal based on the detected boundary profile 63 . The control signal is designed to bring about a suitable measure for the agricultural machine. The control signal can be sent to an external entity such as a central control device associated with the agricultural machine 50 connected, or the agricultural machine 50 itself (e.g. an actuator of the agricultural machine 50 ) are output. The control signal can, for example, cause the agricultural machine 50 braking (e.g. full braking), changing the direction of travel and / or sending out a warning signal (e.g. a warning light signal such as a red light signal, a warning sound signal or a vibration signal).

3 shows a schematic representation of the usable area 60 who have favourited two adjacent growing areas 62a , 62b includes. The cultivation areas 62a , 62b are each covered by a weed area 64a , 64b enclosed. The weed areas 64a , 64b are in turn surrounded by an outer path area 66 enclosed and laterally by a central path area 66c spaced from each other. There is also an inner path area 66a , 66b in the respective cultivation area 62a , 62b .

When the agricultural machine 50 in the in 3 position shown, the attachment areas fall 62a , 62b who have favourited Weed Areas 64a , 64b , as well as the path areas 66a , 66b , 66c each partially under the detection area 222 the environment sensors 22nd in this example on the agricultural machine 50 is appropriate.

Here is a first boundary 63a between the first growing area 62a and the first area of weeds 64a detected, the first boundary 63a as a dashed line in 3 is shown. The first area of weeds 64a is wider than the first cultivation area 62a from the agricultural machine 50 away. This means that the agricultural machine 50 starting from a cultivation area 62a its limit is approaching. The signal unit 16 can then trigger a first control signal, which causes the agricultural machine 50 brakes.

At the same time, as in 3 shown, second boundary 63b between the second cultivation area 62b and the second weed area 64b detected, the second boundary 63b as a dashed line in 3 is shown. The second area of weeds 64b is wider than the second cultivation area 62b from the agricultural machine 50 away. This means that the agricultural machine 50 starting from the environment of a cultivation area 62b its limit is approaching. In this case, the signal unit generates 16 no control signal in response to the detection of the second boundary 63b . In the in 3 shown example are both limit courses 63a , b at the same time through the environment sensors 22nd captured and detected. The first control signal is in spite of the detection of the second limit profile 63b triggered.

Also, as in 3 shown, the middle distance range 66c detected between two adjacent weed areas 64a , 64b is arranged. This means that the agricultural machine 50 a boundary between two neighboring cultivation areas 62a , 62b approximates. The signal unit 16 can then trigger a second control signal, which causes the agricultural machine 50 brakes to entry into the second cultivation area 62b should be avoided if this should not be (temporarily) processed.

Furthermore, as in 3 shown within the first and second cultivation areas 62a , 62b one path area each 66a , 66b detected. In this case, the signal unit generates 16 no control signal in response to, this having no effect on the generation of the other control signals.

The control device 10 can be designed to trigger the first and second control signals described above as a common or single control signal, since the first boundary and the separation of the two weed areas 64a , b through the middle distance range 66c simultaneously under the detection area 222 the environment sensors 22nd fall.

List of reference symbols

10

Control device

12

Input interface

14th

Evaluation unit

16

Signal unit

18th

Output interface

22nd

Environment sensors

222

Detection area

24

Raster map

242

Raster map elements

50

Agricultural machine (sprayer)

60

Usable area

62, 62a, b

Cultivation area

63, 63a, b

Borderline

64, 64a, b

Weed area

66

Path area

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 2018/0113473 A1 [0002]

Claims (12)

A method for controlling an agricultural machine (50) comprising: - Obtaining environment data that are generated by an environment sensor system (22) when detecting an agricultural area (60); - Evaluation of the environment data in order to detect a cultivation area (62), a weed area (64) and a boundary (63) between the cultivation area (62) and the weed area (64) based on a segmentation of the environment data; and - Generating a control signal for the agricultural machine (50) based on the detected boundary profile (63). Procedure according to Claim 1 , further comprising creating a grid map (24) based on the environment data, the grid map (24) having a plurality of grid elements (242), each of the grid elements (242) being assigned an object class determined by means of the segmentation. Procedure according to Claim 2 wherein at least one of the grid elements (242) is assigned an attribute, the attribute comprising a height and / or density of an object recognized in the at least one grid element (242). Method according to one of the preceding claims, wherein the control signal is generated under the condition that the weed area (64) is further away from the agricultural machine (50) than the cultivation area (62). Method according to one of the preceding claims, further comprising evaluating the environment data in order to detect a path area (66) between two weed areas (64) adjoining the path area (66). Method according to one of the preceding claims, wherein the control signal causes the agricultural machine (50) to brake, change the direction of travel and / or send out a warning signal. Device (10) for controlling an agricultural machine (50), comprising: - An input interface (12) for receiving environment data which are generated by an environment sensor system (22) when detecting an agricultural area (60); - An evaluation unit (14) for evaluating the environment data in order to detect a cultivation area (62), a weed area (64) and a boundary (63) between the cultivation area (62) and the weed area (64) based on a segmentation of the environment data; and - A signal unit (16) for generating a control signal for the agricultural machine (50) based on the detected boundary profile (63). System for controlling an agricultural machine (50), comprising a device (10) according to Claim 7 , the system further comprising an environment sensor system (22) for generating environment data when recording an agricultural area (60). Use of the device (10) after Claim 7 and / or the system Claim 8 in an agricultural machine (50). Computer program product, comprising instructions which, when the program is executed by a computer, cause the computer to perform the method according to Claim 1 execute. Computer-readable storage medium on which the computer program product is based Claim 10 is stored. Data carrier signal that the computer program product according to Claim 10 transmits.

Images