DE102024206901A1 - Method for determining a movement sequence for a robot - Google Patents

Abstract

The invention relates to a method for determining a movement sequence (250) for a robot to grasp and/or move an object in a work environment by means of an end effector of the robot, comprising: providing (206) environmental information (202, 204) that has been acquired from the work environment by means of at least one sensor; determining (210), based on the environmental information, a pose for one or each of several objects in the work environment to obtain an object pose data set (228); determining (230), based on the object pose data set, one or more poses for one or each of several end effectors, in each case for one or at least one of the several objects of the object pose data set, to obtain an end effector pose data set (236); Determine (240), based on the end effector pose data set, the motion sequence (250) for the robot to grasp and/or move the object in the work environment using one or more end effectors of the robot; and provide (252) the motion sequence (250), and in particular cause the robot to execute the motion sequence.

Description

The present invention relates to a method for determining a movement sequence for a robot in order to grasp and/or move an object in a working environment by means of an end effector of the robot, a computing unit and a computer program for its execution, and a robot.

Background of the invention

Robots, often also referred to as kinematics, can be used in production facilities, logistics, and other systems to, for example, transport or assemble parts. Typical types of such robots include Cartesian robots, SCARA robots, and articulated robots.

Disclosure of the invention

According to the invention, a method for determining a movement sequence for a robot, a computing unit and a computer program for its execution, as well as a mobile device with the features of the independent claims, are proposed. Advantageous embodiments are the subject of the dependent claims and the following description.

The invention relates generally to robots that can be used, for example, in production plants, logistics, or other facilities. Types of such robots, often also referred to as kinematics, include, for example, Cartesian robots, SCARA robots, and articulated robots. Such robots can be used to grasp and/or move objects (i.e., items or parts). A typical application is removing an object from a box containing, for example, a large number of (identical or different) objects and placing it somewhere or possibly assembling it. For this purpose, such a robot has, for example, a gripper or, more generally, an end effector. Such robots can also be referred to as manipulators or manipulation devices.

In order for the robot to grasp and/or move an object in a work environment, e.g. in a box, using the end effector, the robot, in particular its end effector, must perform a movement sequence to move the end effector to the position of the object and then, e.g. after the object has been grasped, to move the end effector away from the position of the object again, e.g. to a desired storage position.

The movements or tasks that the robot is to perform can be programmed, for example, by an application engineer (technician or engineer). All positions that the robot is to move to in the specific application must then be specified by the application engineer or programmer.

If the points in the application are static, i.e., points that do not change during the application's runtime, the applicator can precisely move the robot to them using a handheld control device and transfer them to the program.

Dynamic points, meaning points that change during application runtime, can be defined using sensors, especially 3D sensors. Optical sensors are particularly suitable for this purpose. Dynamic points are often gripping points on moving objects. The sensor can capture the environment or workspace, for example, as a point cloud. This captured environmental information (data) can then be analyzed, for example, using a software component, to determine or locate the object in 3D space, including its position and orientation (generally and in combination also referred to as pose).

Especially in modern systems, 3D sensors can be used to pick parts from boxes or pallets, or to lift them from conveyor belts. This technology significantly simplifies the mechanical aspects of the system, as precise mechanical locking and pre-singulation of objects are no longer necessary.

In this way, the entire system can become more cost-effective, as standardized 3D sensors and software components can be used. Adaptations to new products or objects, or the reuse of components, are possible with minimal effort. Particularly in intralogistics, especially in parts handling and order picking, there is a very large, sometimes unmanageable variety of different objects with varying surfaces, shapes, packaging materials, and other physical properties.

Against this background, a way is proposed to meet these requirements, specifically through adaptive software that uses intelligent strategies to decide which objects should be removed from a box, how, and when. In particular, it is proposed to analyze the complex movement sequence of the robot with its end effector for grasping or moving an object, i.e., for example, the removal of the object. The process of picking an object from a box is to be generically structured and optimized to simplify and simultaneously optimize the selection, the end effector (gripper), and the gripping of the objects. The individual steps can be implemented as modules in software, so that different modules are used sequentially during application, executing the various steps, at least as applicable.

This process begins by providing environmental information captured from the work environment by at least one sensor. This includes, in particular, a camera (especially a 2D camera) and/or a Time-of-Flight (TOF) sensor, another depth sensor, or another 3D sensor. The environmental information then comprises, in particular, 2D and/or 3D environmental information. While the 2D environmental information can be images, such as color, grayscale, or black-and-white images, the 3D environmental information can be point clouds or depth maps, and possibly also 3D images. The sensors can be mounted on the robot or be part of the robot; alternatively, they can be provided and used separately to capture environmental information about the work environment, such as a box containing objects, as accurately as possible.

Based on the environmental information, a pose is then determined for one or more of the objects in the workspace. Depending on whether the environmental information includes 2D and/or 3D data, for example, only 3D environmental information might be processed initially, followed by only 2D environmental information. It is also conceivable that 2D and 3D environmental information are processed (together). This can also be done differently for different parts of the environmental information.

Based on the object pose data set, one or more end effector poses are then determined for one or at least one of the multiple objects in the object pose data set to obtain an end effector pose data set. It should be noted that the robot can also have multiple end effectors, from which a suitable one can be selected, for example, depending on the type of object to be grasped. In the case of multiple end effectors, one or more poses can be determined for each of them.

Based on the end-effector pose data set, the motion sequence for the robot is then determined to grasp and/or move an object in a work environment using one or more of the robot's end effectors. The resulting motion sequence is then deployed; in particular, the robot is instructed to execute it. Specifically, control information for moving the robot can be determined based on the motion sequence and then deployed, or used to move the robot.

This approach is particularly advantageous for the automated picking of objects at order picking workstations in intralogistics. The automatic selection of an end effector and the automatic generation of various grips based on the end effector's geometry increase both the success rate of picking objects and the success rate of completely emptying boxes. At order picking workstations, a wide variety of objects are removed from boxes. Product portfolios often comprise more than 100,000 different items.

As mentioned, the environmental information can include 2D and/or 3D environmental information, which can then be processed in various ways. It should be noted that the 2D and/or 3D environmental information, or corresponding 2D and 3D images, are recalibrated with respect to each other, meaning that their relative poses, especially in relation to the respective sensor, are known.

In one embodiment, processing only 3D environment information includes determining, based on the 3D environment information, the position and/or orientation for one or each of the multiple objects; and/or assigning, based on the 3D environment information, one or at least one of the multiple objects to an object class, wherein the object pose record includes information about the assigned object class.

In one embodiment, processing only 2D environment information includes identifying, based on the 2D environment information, a part, e.g., an edge, of one or at least one of the multiple objects; and/or assigning, based on the 2D environment information, one or at least one of the multiple objects to an object class, wherein the object pose record includes information about the assigned object class.

In one embodiment, the processing of 2D and 3D environment information includes classifying and/or segmenting, based on the 2D environment information, one or at least one of the multiple objects; and determining, based on the 3D environment information, the pose for one or at least one of the multiple classified and/or segmented objects.

The separate processing of 3D and 2D environment information allows for efficient and fast processing of the respective specific information.

However, the combined processing of 3D and 2D environment information allows for the acquisition of more accurate information, e.g., more accurate poses.

In one embodiment, at least one of the following steps is provided to obtain the object-pose data set. For example, based on at least one object selection criterion, one or more selected objects are determined from the one or more objects. The object-pose data set then only includes poses for the one or more selected objects. This allows, for example, poses of objects that lie outside a valid or predefined range, or that exhibit excessive tilt, to be disregarded. Alternatively, a sequence can be assigned to each of the multiple objects in the object-pose data set according to at least one object ordering criterion. Here, for example, higher-ordered poses can be sorted to the front so that they are processed preferentially later, since such objects are, for example, easier to grasp.

In one embodiment, to obtain the end-effector pose data set, at least one of the following steps is provided. For example, based on at least one end-effector selection criterion, one or more selected poses are determined for one or at least one of the multiple end effectors, specifically from the one or more poses of the respective end effector. The end-effector pose data set then comprises only the one or more selected poses for the respective end effector. Similar to the poses of the objects, poses for an end effector that are difficult to access can be disregarded. Alternatively, a sequence can be assigned to each of the multiple poses for one or at least one of the multiple end effectors in the end-effector pose data set, according to at least one end-effector ordering criterion. Here, for example, higher-ordered poses can be sorted to the front so that they are given priority later, since such poses are, for example, easier to reach for an end effector.

In one embodiment, determining the motion sequence includes determining an axis configuration data set, particularly based on inverse kinematics. This axis configuration data set comprises an axis configuration of the robot for one or at least one of the multiple poses of one or at least one of the multiple end effectors. Preferably, a potential motion sequence is also determined for one or at least one of the multiple axis configurations. In the case of a potential motion sequence, this is used as the final motion sequence, or in the case of multiple potential motion sequences, one of them is determined as the motion sequence to be used. The multiple potential motion sequences can represent different poses for one end effector, or they can represent different end effectors, each with one or more different poses. The selection of the final motion sequence thus may also include a selection of the end effector.

The motion sequence can, in particular, comprise a first sub-sequence and a second sub-sequence. The first sub-sequence involves a movement of one or more end effectors toward a position of the object, and the second sub-sequence involves a movement of one or more end effectors away from the object's position. This also applies accordingly to potential motion sequences.

A computing unit according to the invention (i.e., generally a system for data processing), e.g., a control unit or a control unit of a robot, or a central server or other computing system, is, in particular in terms of programming, equipped to carry out a method according to the invention.

The invention also relates to a robot configured to receive control information as described above. In addition, or alternatively, the robot comprises a computing unit according to the invention. Furthermore, the robot comprises, in particular, a control unit and a drive unit for moving the robot. The robot may also have at least one sensor for acquiring environmental information, e.g., a camera and/or a 3D sensor.

Implementing a method according to the invention in the form of a computer program or computer program product with program code for carrying out all method steps is also advantageous, as this incurs particularly low costs, especially if an executing control unit is already available for other tasks. Finally, a machine-readable storage medium is provided with a computer program stored on it as described above. Suitable storage media or data carriers for providing the computer program are, in particular, magnetic, optical, and electrical storage media, such as hard drives, flash memory, EEPROMs, DVDs, etc. Downloading a program via computer networks (Internet, intranet, etc.) is also possible. Such a download can be wired or wireless (e.g., via a WLAN network, a 3G, 4G, 5G, or 6G connection, etc.).

Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawing.

The invention is schematically illustrated in the drawing using an exemplary embodiment and is described below with reference to the drawing.

Brief description of the drawings

• 1 The image schematically shows a robot to illustrate the invention.
• 2 schematically shows a sequence of steps in one embodiment of a process.
• 3 , 4 , 5 until 6 The images schematically show objects and robots to illustrate the invention.

embodiment(s) of the invention

In 1 A robot 100 is shown schematically and by way of example, to illustrate the invention. By way of example, the robot 100 has stand and arm components 102, 104, 106, so-called axes, which are each movably and movably connected by means of joints 112, 114.

Furthermore, the robot 100 has an end effector 108, e.g., a gripper. The end effector 108 is movably and movably connected to the arm component 106 by means of a joint 116.

Furthermore, the robot 100 has a drive system 120, shown only schematically here, as well as a computing unit 122 designed, for example, as a control or regulation unit. This allows the drive system 120 to be controlled, for example, using control information, in order to move the robot according to a desired sequence of movements. This can include, for example, moving the axes relative to each other by means of the joints, but also rotating the axes themselves, provided that appropriate drives are present.

Furthermore, a box 132 is shown in a work environment 130, containing, for example, an object 140. The robot 100 can now be controlled, for example, in such a way that it grasps and/or moves the object 140 using the end effector 108.

Furthermore, an example sensor 124 is shown, which could be, for example, a camera or a 3D sensor. Both types of sensors can also be used. Likewise, several identical sensors can be used. The sensor 124 can, for example, be arranged in a suitable manner in the work environment, e.g., on a ceiling and thus separately from the robot 100. However, the sensor 124 could also be part of the robot and, for example, be arranged on the arm component 106 or the end effector 108.

The sensor 124 can now detect the work environment 130, and in particular the box 132 and its interior, including the object 140. Based on this environmental information, a movement sequence can be created for the robot to grasp and/or move the object 140 using the end effector 108. This will be explained in more detail below.

In 2 The schematic representation depicts the sequence of a procedure in one embodiment. Various steps are explained, which, when implemented in software, would then run, for example, on the computing unit 122 according to... 1 or is implemented in other ways, as modules may be provided, as will be explained in more detail below.

This also includes attention to the 3 , 4 , 5 until 6 Reference is made to objects and/or robots, which are used to explain various steps in more detail.

In step or module 200, environmental information is captured from the work environment using at least one sensor. Specifically, this could be, for example, a scene recording of the box or the work environment in which objects are to be detected. The sensor captures, for example, one or more 3D and/or 2D images – generally referred to as 3D environmental information 202 and 2D environmental information 204 – which are then processed. The data is processed in several steps or modules. For example, the 2D image is an RGB image or a grayscale image. The 3D image is, for example, a point cloud or depth image. Typically, the 2D and 3D images are calibrated to each other, meaning they are aligned.

This environmental information, thus captured, is then made available for further use or processing in step 206.

In step 210, based on the environment information, a pose is determined for one or each of several objects in the work environment in order to obtain an object pose record 228.

Step 210 can comprise three different steps or modules: processing only 3D environment information in step or module 212, processing only 2D environment information in step or module 214, and processing 2D and 3D environment information in step or module 216.

Step or module 212 can include determining the pose for one or each of the multiple objects based on the 3D environment information, and assigning one or at least one of the multiple objects to an object class based on the 3D environment information, with the object pose record then including information about the assigned object class.

Specifically, 3D images can be processed and instantiated in image processing workflows. Such workflows can range from simple surface segmentation to complex algorithms using neural networks to classify and segment complex objects. The result is, for example, a vector containing object poses. These poses can also be referred to as vision poses, as they are derived from visual environmental information. These poses represent all localized objects. Optionally, algorithm-specific parameters, such as surface information, can be assigned to the poses and/or the objects in the object-pose dataset.

Step or module 214 may include identifying a part of one or at least one of the multiple objects based on the 2D environment information, or assigning one or at least one of the multiple objects to an object class based on the 2D environment information, with the object pose record including information about the assigned object class.

Specifically, 2D images can be processed and instantiated in image processing workflows. Such workflows might include barcode recognition, object edge detection, or a classifier that identifies different object classes. Algorithms (with and without AI, i.e., artificial intelligence) can also be used to extract 6D poses. This can be achieved, for example, using shape-based CAD matchers or AI-based methods, such as generating a synthetic depth image that enables 3D point cloud processing.

Step or module 216 may include classifying and/or segmenting one or at least one of the multiple objects based on the 2D environment information, and determining the pose for one or at least one of the multiple classified and/or segmented objects based on the 3D environment information.

Specifically, hybrid image processing allows for the instantiation of images or data that combine 2D and 3D representations. For example, objects in the 2D image are classified and segmented, and then correspondingly cut out in the 3D point cloud or depth image. This approach enables the segmentation of complex objects within the 3D image. The result is, for instance, a vector containing object poses. These poses can also be referred to as vision poses, as they are derived from visual environment information. These poses represent all localized objects. Optionally, algorithm-specific parameters, such as area information, can be assigned to the poses and/or objects within the object-pose dataset.

Using steps or modules 212, 214, 216 - depending on the situation, for example, only some of them may be used - a preliminary object-pose data set 218 is then obtained, which includes poses and, if applicable, the other information mentioned about the recognized objects.

To obtain the (final) object pose data record 228, further steps or modules 222 and/or 224 may be required. Step or module 222 includes, for example, determining one or more selected objects based on at least one object selection criterion, and selecting the object from this one or more objects. objects (i.e., from the preliminary object poses record 218), whereby the (final) object poses record 228 then only includes poses for the one or more selected objects.

Specifically, various processes and/or filters can be instantiated here. For example, poses from steps or modules 212, 214, and 216 that lie outside a valid or otherwise predefined range can be filtered out. For instance, all poses of objects that exhibit a significant tilt relative to the bottom of the box can be filtered out.

This is in 3 a box 332 was shown, which, for example, corresponds to box 132. 1 This can correspond to... Box 332 contains four objects 340, 341, 342, and 343 as examples, which can, for instance, have any pose. 350 shows a normal to a bottom 334 of box 332 (i.e., a box bottom), and 352 shows a normal 352 to a surface of object 340. This normal 352 can represent the pose of object 340.

The value 354 now denotes a difference or difference vector between the normals 350 and 352, which is a measure of the tilt of object 340 relative to the box bottom 332. In this way, poses with a very strong tilt relative to the box bottom, such as object 343, can be filtered out. Suitable threshold values can be specified here.

Step or module 224 includes, for example, assigning a sequence to each of the multiple objects in the object-pose data record according to at least one object ordering criterion. This could simply be reordering the entries in a list corresponding to the object-pose data record. A concrete example is sorting the poses according to their height within the box.

In the example in 3 For example, object 340 would be the first one, followed by objects 341 and 342. Object 343 would already be sorted out at this point due to the tilting.

The result is, for example, a vector containing the remaining and possibly reordered poses of objects, i.e., the (final) object-pose data set 228.

In a step or module 230, based on the object poses record 228, one or more poses are determined for one or each of several end effectors, in each case for one or at least one of the several objects of the object poses record 228, in order to obtain an end effector poses record 236.

Specifically, in this step or module, various processes can be instantiated that serve to convert object poses (vision poses) into end effector poses (gripping poses). Depending on the image processing algorithm used and the available end effectors (grippers), different end effector poses can be generated.

For example, a cylinder fitting algorithm outputs a vision pose that is axially aligned in the z-direction.

In 4 On the left, object 440 is shown, which has the shape of a cylinder or at least approximates a cylinder. Based on a Cartesian coordinate system shown in the lower left, the z-axis in the indicated pose 452 of object 440 is aligned along an axis of the cylinder.

The required gripping poses, however, lie on the cylinder's surface. Using a vision-pose parameter set that includes the radius of the detected cylinder for the cylinder-fit algorithm, the gripping positions can be calculated, for example.

This is in 4 On the right, an end effector or gripper 408 (or part thereof) is shown as an example, next to object 440. The radius can be used, for example, to select a suitable tool for gripping the cylinder. In other words, if multiple end effectors are available, the one that can grip the object with the specified radius of the cylinder's surface can be selected (possibly even later). The generated grip poses include information on the vision pose, the generated grip pose (end effector pose), the end effector itself, and, for example, cost information describing the grip quality. Grip quality refers to how precisely the end effector can grasp the object, i.e., how accurately the end effector can be positioned. For example, grips located on the top of the cylinder's surface have a higher quality than those tilted at an angle on the side.

Compared to pose 452 of the object, pose 460 of the end effector 408 is modified again; it can be obtained, for example, by translation and rotation. The cylinder is axially symmetric; therefore, any number of handles can be orthogonally positioned on the surface of the cylinder using the known radius. The surface area can be calculated. Using the cylinder length information (transmitted by the vision), a translational shift is also conceivable to generate further handles on the surface.

Specifically, further processes and filters can also be instantiated in this module. For example, processes can be used to generate secondary gripping positions using the selected end effector or gripper from the previous step, thus expanding the number of possible poses of the end effector.

For example, if it is a rotationally symmetrical gripper, correspondingly rotationally symmetrical handles can be generated, which ultimately increases the success rate of the gripping action.

This is in 5 An object 540 with an end effector 508 is shown, which corresponds to the object 440 with end effector 408 according to 4 This can correspond to the following: 562 here indicates a rotation of the end effector 508 around its axis. Depending on the degree of rotation, e.g., in specific angular increments, several new poses can be created. Processes that generate the gripping poses generate them, for example, based on the gripping poses that are passed when the module is called.

Furthermore, for example, a step or module 232 may be provided in which, based on at least one end-effector selection criterion, one or more selected poses are determined for one or at least one of the several end-effectors, specifically from the one or more poses of that one of the several end-effectors. The end-effector pose data set 236 then comprises only the one or more selected poses for the one or at least one of the several end-effectors.

This can, for example, also include a step or a module 234 in which an order is assigned to each of the several poses for one or at least one of the several end effectors in the end effector poses data set 236 according to at least one end effector ordering criterion.

To filter the generated gripping poses, filter processes can be instantiated to, for example, remove gripping poses with excessively high costs or excessive tilt relative to the box bottom. Processes can also be implemented to sort the gripping poses or evaluate or filter them according to specific criteria such as height, gripping area, or localization accuracy.

The result is a vector containing the remaining gripper poses and their costs, i.e., the end effector pose dataset 236.

In a step or module 240, based on the end effector pose data set 236, the motion sequence 250 for the robot is determined to grasp and/or move the object in the work environment using one or more end effectors of the robot.

This can, for example, include a step or module 242 in which an axis configuration data set 244 is determined, in particular based on inverse kinematics. The axis configuration data set includes an axis configuration of the robot for one or at least one of the several poses of one or at least one of the several end effectors.

Specifically, for example, the corresponding specific axis configurations of the robot used for each gripping pose can be calculated using inverse kinematics. This allows for the calculation of possible movements, particularly joint rotations, as shown in... 1 mentioned, taken into consideration.

Gripping poses that the robot cannot physically reach have no solution in the inverse kinematic calculation and are therefore automatically eliminated.

The result is a vector containing the valid robot axis configurations at the gripping points, i.e., axis configuration data set 244. The vector elements include, for example, the robot axis configuration of all movable axes at the gripping point and the corresponding gripping pose.

This can also include a step or module 246 in which a potential motion sequence 248 is determined for one or at least one of the several axis configurations 244. In the case of (only) one potential motion sequence, this will be used as motion sequence 250; in the case of several potential motion sequences, one of them will be determined as motion sequence 250, e.g., according to a quality criterion that determines how or which object can be grasped most quickly.

Specifically, processes or filters can be instantiated to, for example, filter out unwanted robot axis configurations. Specifically, axis configurations where a particular axis is positioned by more than +/- 90° (e.g., relative to a standard position) should be excluded. This could, for example, be the axis 106 according to 1 It is evident here that the same pose of the end effector, as shown, could also be achieved with an axis 106 rotated by 180°; however, this may be undesirable for various reasons, e.g. because it would increase the load on the robot or restrict subsequent movements.

The motion sequence 250 can comprise a first partial motion sequence and a second partial motion sequence, wherein the first partial motion sequence comprises a movement of one or one of the several end effectors towards a position of the object, and wherein the second partial motion sequence comprises a movement of one or one of the several end effectors away from the position of the object.

For each available robot axis configuration at the gripping point, a collision-free movement using the end effector into the box to the object and out of the box can be calculated, provided that physical accessibility from the given starting position is possible.

Specifically, processes for calculating the approach path (first part of the movement sequence) can be instantiated in a module. Such processes serve to determine segments of the approach path or the entire collision-free approach path to the specified gripping point. Information about the robot, gripper, and interfering contours in the system and on the robot should be taken into account to ensure a collision-free path.

Accordingly, processes for calculating the retraction path (second part of the motion sequence) can be instantiated, for example, in a separate module. Such processes serve to determine segments of the retraction path or the entire collision-free retraction path to the specified gripping point. Information about the robot, gripper, and interfering contours in the system and on the robot should be taken into account to ensure a collision-free path.

In 6 This includes box 632, comparable to box 332 according to 3 The diagram shows examples of various objects 640, 641, 642, 643, and 644. The goal is to grasp object 640 and remove it from the box. An approach path, or first part of the movement sequence 651, is shown, along which the end effector moves to object 640 so that it can be grasped. Additionally, a retraction path, or second part of the movement sequence 652, is shown, along which the end effector can be moved with object 640 out of the box.

The motion sequence 250 thus created is then provided in step 252, so that the robot can be instructed to grasp the object based on it.

Claims (13)

Method for determining a motion sequence (250) for a robot (100) to grasp and/or move an object (140) in a work environment (130) using an end effector (108) of the robot, comprising: Providing (206) environmental information (202, 204) acquired from the work environment by means of at least one sensor (124); Determining (210), based on the environmental information, a pose for one or each of several objects in the work environment to obtain an object-pose data set (228); Determining (230), based on the object-pose data set, one or more poses for one or each of several end effectors, for each of the objects in the object-pose data set, to obtain an end-effector pose data set (236); Determine (240), based on the end effector pose data set, the motion sequence (250) for the robot to grasp and/or move the object in the work environment using one or more end effectors of the robot; and provide (252) the motion sequence (250), and in particular cause the robot to execute the motion sequence. Procedure according to Claim 1 , wherein the environment information comprises 3D environment information (202), in particular one or more 3D images, depth images or point clouds, and/or 2D environment information (204), in particular one or more 2D images, and wherein determining (210) the object pose data set comprises at least one of the following steps: - processing (212) only 3D environment information; - processing (214) only 2D environment information; - processing (216) 2D and 3D environment information. Procedure according to Claim 2 , wherein the processing (212) only includes: - determining, based on the 3D environment information, the pose for one or each of the multiple objects; and/or - assigning, based on the 3D environment information, the one or at least one of the multiple objects to an object class, wherein the The object pose record includes information about the associated object class; wherein the processing (214) of 2D environment information only includes: - identifying, based on the 2D environment information, a part of the one or at least one of the multiple objects; and/or - assigning, based on the 2D environment information, the one or at least one of the multiple objects to an object class, wherein the object pose record includes information about the associated object class; and wherein the processing (216) of 2D and 3D environment information includes: - classifying and/or segmenting, based on the 2D environment information, of the one or at least one of the multiple objects; and - determining, based on the 3D environment information, the pose for the one or at least one of the multiple classified and/or segmented objects. A method according to any of the preceding claims, further comprising, in order to obtain the object pose data set (228), at least one of the following steps: - Determining (222), based on at least one object selection criterion, one or more selected objects from the one or more objects, wherein the object pose data set comprises only poses for the one or more selected objects; - Assigning (224) a sequence to each of the multiple objects in the object pose data set according to at least one object ordering criterion. A method according to any of the preceding claims, further comprising, in order to obtain the end-effector pose data set (236), at least one of the following steps: - Determining (232), based on at least one end-effector selection criterion, one or more selected poses for the one or at least one of the multiple end-effectors, from the one or more poses of the one of the multiple end-effectors, wherein the end-effector pose data set comprises only the one or more selected poses for the one or at least one of the multiple end-effectors; - Assigning (234) a sequence to each of the multiple poses for the one or at least one of the multiple end-effectors in the end-effector pose data set according to at least one end-effector ordering criterion. Method according to one of the preceding claims, wherein determining (240) the motion sequence (246) comprises: Determining (242) an axis configuration data set (244), in particular based on inverse kinematics, wherein the axis configuration data set comprises an axis configuration of the robot for one or at least one of the several poses of one or at least one of the several end effectors. Procedure according to Claim 6 , furthermore comprehensive: Determine (246) for one or at least one of the several axis configurations of a potential motion sequence; and in the case of one potential motion sequence: Use the potential motion sequence as the motion sequence, or in the case of several potential motion sequences: Determine one of the several potential motion sequences as the motion sequence. Method according to one of the preceding claims, wherein the motion sequence (250) comprises a first partial motion sequence (651) and a second partial motion sequence (652), wherein the first partial motion sequence comprises a movement of one or one of the multiple end effectors towards a position of the object, and wherein the second partial motion sequence comprises a movement of one or one of the multiple end effectors away from the position of the object. A method according to any of the preceding claims, further comprising: Determining, based on the motion sequence, control information for moving the robot, and providing the control information and/or moving the robot based on the control information. Computing unit comprising means for carrying out the method according to one of the preceding claims. A robot that is set up to receive control information via a procedure according to Claim 9 have been determined to obtain, and/or with a computing unit according Claim 10 , and with a drive system and a control or regulation unit for controlling the drive system, with one or more, in particular several selectable, end effectors for gripping and/or moving an object, and preferably with at least one sensor, in particular a camera and/or a 3D sensor, for capturing environmental information of a working environment. A computer program comprises commands that are issued by a computer during the execution of the program. Computers cause this to follow the procedure Claim 1 until 9 to execute. Computer-readable storage medium on which the computer program is stored. Claim 12 is stored.