DE102022000701A1 - Autonomous manipulation system for maintenance and inspection work on track systems - Google Patents

Abstract

Apparatus for autonomous manipulation installed on a mobile vehicle, consisting of at least one robotic arm or manipulator and a system for object recognition, characterized in that the apparatus is configured to recognize and manipulate objects in the area of the railway.

Description

field of use

The invention relates to an autonomous manipulation system installed on a mobile base for works on railways, in particular vegetation control (removal of vegetation), removal of unwanted objects or garbage, inspection and maintenance.

State of the art

Various procedures have been established in the railway sector for the maintenance and inspection of tracks. Vegetation control often occurs through the application of herbicides via road-rail vehicles traveling on the track. But it is also common to dispatch personnel to remove them manually, especially right next to the tracks. Railway lines are regularly inspected (track inspection) and also measured using measurement trains.

Permanent monitoring using predictive maintenance is currently being established, e.g. via vibration sensors on points. These detect unusual vibrations using machine learning and can thus predict a failure of the switch.

Description

• • einem Roboterarm (8) oder Aktuator,
• • einem Endeffektor, wie einem Greifer (10),
• • einem typischerweise kamerabasierten System zur Objekt- oder Umgebungserkennung,
• • einem System zur Planung bzw. Steuerung des Manipulationsvorgangs, sowie optional aus
• • einem Sammelbehälter (6) für aufgesammelte Pflanzen und Müll,
• • einem System zur Fernsteuerung (Teilautonomie),
• • einer Kamera am Greifer/Endeffektor,
• • weiteren, austauschbaren Endeffektoren, wie z.B. einer Schere, Messer, Säge, Sprüheinrichtung für Flüssigkeiten.

The manipulation system described here consists in particular of at least

• • a robot arm (8) or actuator,
• • an end effector, such as a gripper (10),
• • a typically camera-based system for object or environment recognition,
• • a system for planning or controlling the manipulation process, as well as optionally
• • a collection container (6) for collected plants and waste,
• • a system for remote control (partial autonomy),
• • a camera on the gripper/end effector,
• • Additional, exchangeable end effectors, such as scissors, knives, saws, spray devices for liquids.

The manipulation system is installed on a mobile base that moves on, along or in the track or railway. Manipulation system and mobile base together form an autonomous robot. The manipulation system can control the mobile base, e.g. stop it during a manipulation process.

In one embodiment, the manipulation system is configured in such a way that it detects certain objects (such as plants (13) or other objects in the track) via an object recognition system, positions the robot arm (8) to the object, grabs the object, tears it out in the case of a plant and finally deposits or packs in a collection container (6). The collection container can be equipped with a shredder or other tool for chopping up the plants.

In a preferred embodiment, the mobile robot is constructed in such a way that it is limited to the use of the space (2) between the rails (3) ("rail gap"), see operation under railway vehicles in operation) and does not interfere with the ongoing train operation. For this purpose, the robot - at least when a train (1) is crossing - must not penetrate into the so-called loading gauge of the railway vehicles. The robot arm is either mechanically constructed, in particular by appropriate kinematics or configuration of the joints (kinematics), so that it cannot leave the space between the rails, or it is brought into the space between the rails by active control when a train passes over it. The latter requires an absolutely reliable detection of moves and the ability to control the arm.

Advantages

The current problem is that vegetation control on tracks in Europe can no longer be carried out with the very effective herbicide glysophate. Alternative methods, such as manual treatment, are expensive, time-consuming and involve safety risks for personnel.

The manipulation system described removes plants in a targeted manner, e.g. by grabbing and tearing, and thus with minimal impact on the environment. Even a targeted application of chemical agents using the manipulation system reduces the environmental impact compared to large-scale spraying.

applications

• • Entfernung von Bewuchs im Gleis
• • Entfernung von Müll, insbesondere auch in Bahnhöfen, und unerwünschten Gegenstände im Gleis
• • Inspektion und Prüfungen, z.B. Klopf- oder Rütteltests an
  • ◯ Gleisen und Weichen
  • ◯ Einbauten im Gleis
  • ◯ Unterseiten von Bahnfahrzeugen
• • Visuelle Prüfung von Gleiskomponenten mit gezielter Kameraführung, z.B. nach Meldung eines Problemverdachts durch einen Lokführer oder automatisiert über einen Sensor am Zug
• • Durchführung der regulatorisch vorgeschriebenen Streckenbegehungen
• • Erfassung des Schienenprofils, z.B. über taktile Sensoren oder die gezielte Führung von Kameras, Tiefensensoren oder anderer Sensoren
• • Erstellung eines „Digitalen Zwillings“ des Schienennetzes, der den aktuellen Zustand der Schiene und der Umgebung darstellt, wobei eine Vielzahl von auch durch aktive Manipulation erfasste Daten integriert werden können
• • Baubegleitende Dokumentation
• • Prüfung von Neubaustrecken
• • Beladen, Transport und Abladen von Gegenständen

Applications of the manipulation system include:

• • Removal of vegetation in the track
• • Removal of rubbish, especially in train stations, and unwanted objects on the track
• • Inspection and testing, eg knocking or shaking tests
  • ◯ Tracks and points
  • ◯ Installations in the track
  • ◯ Undersides of rail vehicles
• • Visual inspection of track components with targeted camera guidance, eg after a train driver reports a suspected problem or automatically via a sensor on the train
• • Carrying out the route inspections prescribed by the regulations
• • Recording of the rail profile, eg via tactile sensors or the targeted guidance of cameras, depth sensors or other sensors
• • Creation of a "digital twin" of the rail network, which shows the current condition of the rail and the environment, whereby a large number of data recorded through active manipulation can be integrated
• • Construction accompanying documentation
• • Examination of new lines
• • Loading, transporting and unloading of objects

object detection

The mobile robot moves in an unstructured environment, i.e. the nature of the environment and the objects in it are not fully known. The manipulation system is therefore equipped with a system for object and environment recognition (“object recognition system”). This recognizes at least the objects relevant to the task of the robot and their positions, such as the track, installations in the track, objects/waste in the track, obstacles, components to be inspected (e.g. cables, screws, etc.). The detection of people or approaching trains may also be necessary to plan the movement. The object recognition system provides the position of objects, e.g. in the 2D image by means of a frame ("bounding box") or a more precise, often pixel-by-pixel, segmentation. When detection occurs in the 2D image, the distance is determined using a depth camera. In the case of known objects, the distance can also be calculated from the 2D camera image. However, object recognition is also possible, for example, in depth images or point clouds generated from them.

In order to remove plants/growth, plants in the track in particular are recognized together with their distance. Typically, the geometry of plants cannot be fully captured because it is too complex and there are many self-coverings. Depth values are often incomplete and sometimes incorrect. The manipulation system can grab and uproot the plant at many points as long as the grabbing point is sufficiently stable to uproot the plant. So many grip points are possible. It is preferable to aim for a trunk or main branch in a low part of the plant ("stem base") as this is where the plant is most stable. In a preferred embodiment, the object recognition system therefore first searches for plants and—if recognizable—the base of the trunk in the camera image. In order to determine possible robust grasping points, regions in low-lying areas of the plant with a stably sensed distance/depth and a certain size are determined. The procedure is repeated over several frames and the possible robust grasp points are evaluated.

The same applies to the removal of rubbish or other objects, although a greater variety of object classes must be taken into account here. The geometry can often be recognized better than with plants.

The object recognition system usually uses at least one camera (7) (e.g. color camera, black-and-white camera, infrared camera), which images the relevant surrounding area. Depth values are also required for manipulation, which are determined, for example, via at least one depth camera, stereo camera, laser scanner, LiDAR or other depth or distance sensor. Combined color and depth cameras (RGBD cameras) are common. Modern AI-based object recognition systems can reliably recognize a large number of objects in a 2D camera image. The combination of different sensors - e.g. color and depth image (RGBD) - can improve the detection quality.

Known SLAM methods can be used to capture an environment model, e.g. a 2D or 3D map with obstacles. These also enable the localization of the mobile robot. It should be noted, however, that these methods do not work well when primarily capturing the long, linear rails without features. Additional features, such as sleepers or gravel, should therefore be recorded.

The camera/s are intrinsically and extrinsically calibrated. It is possible to convert from the coordinates of a camera to coordinates of the robot arm, coordinates of another camera/depth camera or another sensor.

For the successful autonomous use of the system in all weather conditions and to avoid manual intervention, the cameras (7) and sensors must be protected from moisture and dirt. In order to avoid breakdowns due to dirty cameras, the system can be able to operate independently using the manipulator to clean. Furthermore, the operating time of the system can be extended to include the entire day and night thanks to integrated artificial lighting. This can be attached to the mobile platform or in the gripper (15).

gripping process and monitoring

Grippers (10) and fingers (12) should be designed to be fault-tolerant, so that the geometry of the gripper passively enables a secure grip on the object, especially the plant (13), even if the gripping position is imprecise or unsuitable. A radial gripper can be used for this, in which the fingers form a funnel when open, see 3 , and allow a wide opening of the gripper. When starting the plant, a larger part of the plant is pushed between the fingers. In addition, fingers should have a high coefficient of friction on the inside in order to apply enough force to pull out the plant.

Due to various sources of error, it can still happen that the arm does not grip the plant or only insufficiently when it reaches the target position. Causes of errors are e.g. calibration errors, position errors of the arm, bending of the arm, inaccurate depth measurement, unfavorable geometry of the plant.

Therefore, a visual grip check is useful during the gripping process. For this purpose, in a further embodiment, a gripper camera (14) is installed in the end effector or gripper (10), see 4 . The parts of the plant in the gripper are detected in the image of the gripper camera, for example by object recognition or simple visual tracking in front of the fingers as the background. Depending on where the majority of the plant is recognized (e.g. in front of/left/right of the gripping fingers), a gripping correction is planned.

Grip control can also be tactile. For this purpose, a tactile sensor is installed on the gripper fingers, which provides an impression of the objects in the gripper. The number/size of the plant parts in the grab can be used to decide whether the grab was successful. A camera-based tactile sensor can also be used.

When the plant is torn out, object recognition or simple visual tracking is used, for example, to determine whether most of the plant parts remain between the fingers. In this way, any slipping out can be detected at an early stage. After being torn out, the cameras (7) or (14) carry out another object detection to ensure that the area around the plant is clear. Otherwise, a new attempt is made.

In addition, deviations in the movement of the arm can also arise, e.g. due to flexing of the arm, tolerances, backlash in the gears, wear, movement of the mobile platform, etc. The design of the arm can be designed in such a way that e.g. backlash in the gears and bending are provided for are to achieve other design goals such as low cost, robust mechanics, simple mechanics, good weather resistance or light weight. The camera (7) on the mobile base (4) or the gripper camera (14) can be used to correct the position (visual servoing).

An extended concept envisages that the movement of the arm is controlled entirely via a camera (preferably on the mobile base (7)). The joint angles of the joints (11) are measured visually by camera and software, with features or markers on the arm (9) being recorded by the camera (“camera-based sensors”). There is no need for any encoders (angle sensors) in the arm. Deflection of the arm and backlash of the gears are detected directly.

mobile base

• • ein Bahnfahrzeug (Lok, Wagon, Zug, Wartungsfahrzeug, etc.),
• • ein Straßenbahn- oder U-Bahnfahrzeug,
• • Ein Zweiwegefahrzeug,
• • ein Hilfsfahrzeug, z.B. eine Eisenbahn-Draisine, das ggf. mit einem Antrieb ausgestattet wird,
• • eine mobile Roboter-Plattform (z.B. AGV, automatic guided vehicle, oder AMR, autonomous mobile robot),
• • ein mobiler Roboter/ein Fahrzeug, das sich im Bereich zwischen den Schienen unter den Zügen bewegt (2), ohne den Zugverkehr auf der Schiene zu behindern („Schienenzwischenraumfahrzeug“ (4)).

The manipulation system is installed on a mobile base, which ensures movement on, along, next to or even in the rail, such as

• • a railway vehicle (locomotive, wagon, train, maintenance vehicle, etc.),
• • a tram or metro vehicle,
• • A two-way vehicle,
• • an auxiliary vehicle, e.g. a railway handcar, which may be equipped with a drive,
• • a mobile robot platform (e.g. AGV, automatic guided vehicle, or AMR, autonomous mobile robot),
• • a mobile robot/vehicle that moves in the area between the rails under the trains (2) without impeding train traffic on the rails (“gap rail vehicle” (4)).

Operation under railway vehicles in operation

The use of the manipulation system on an interrail vehicle (4), which represents a novel concept, is particularly advantageous and is mainly presented here. With such a vehicle, maintenance tasks can be carried out in/on the track without having to close the track for train operations. In other words, a second route is being established between the rails. Maintenance and inspection tasks In this way, tests can be carried out more frequently and with higher quality. Safety risks on trains or railways can thus be identified earlier. Likewise, the safety risks for the inspection and maintenance personnel are reduced. The reduction in line closures for a wide variety of maintenance and inspection work increases the capacity and availability of railway lines.

The entire robot system must not intrude into the clearance of a rail vehicle, at least when it passes over it. The permissible structure gauges for railway vehicles are defined in the relevant standards. This means that the frontal profile of the robotic system is limited to the space between the rails, typically to the top of the rails. The rail space vehicle must be secured against being lifted out by suction, e.g. with holding arms (5).

A rail gap vehicle cannot negotiate points and certain installations without leaving the rail gap. It is therefore transferred manually at these points. An extended version of the rail gap vehicle can "climb over" such obstacles, in which case the route must then be closed until safe re-entry into the rail gap. Waiting times at these obstacles must be taken into account when planning operations. In the case of manual transfers, these can amount to many hours.

Claims (7)

Apparatus for autonomous manipulation installed on a mobile vehicle, consisting of at least one robotic arm or manipulator and a system for object recognition, characterized in that the apparatus is configured for the recognition and manipulation of objects in the area of the railway. apparatus after claim 1 , characterized in that the apparatus and the mobile vehicle operate in a railway track among operating railway vehicles. apparatus after claim 1 or 2 , characterized in that a gripper or other tool for carrying out the manipulation tasks is installed on the robot arm apparatus after claim 1 , 2 or 3 , characterized in that the apparatus is configured to detect and remove plants or vegetation. apparatus after claim 1 , 2 or 3 , characterized in that the apparatus is configured to detect and remove unwanted objects in the track area. apparatus after claim 1 , 2 or 3 , characterized in that the apparatus is configured to detect and inspect components of the railway. apparatus after claim 1 , 2 or 3 , characterized in that the apparatus is configured for the detection and testing of components on railway vehicles.

Images